The value of valgrind

This weekend I got quite a lot done on liboca, mainly working on the network stack, and the basic implementation of OCP.1, which is the TCP/IP-based protocol for OCA (There is room in the OCA standard for other transports). Anyhow, at one point, my unit tests broke in a suite of tests that tested code that should not have been affected by the code I was working on. In the end the fact that I had set up Valgrind, ended up showing me how to solve my problem, so I learned that hard way that it is an essential tool to run along with your standard test suite.

What is Valgrind?

"Valgrind is an instrumentation framework 
for building dynamic analysis tools. There 
are Valgrind tools that can automatically 
detect many memory management and threading 
bugs, and profile your programs in detail. You 
can also use Valgrind to build new tools. " 
                                      - valgrind.org

In its simplest form, it takes your code, instruments it and runs it on a processor emulator, and tells you if you’ve leaked memory, accessed memory you shouldn’t have, and a whole bunch of other stuff – I encourage you to check out the website for more detail.

So what happened?

So we already got to the fact that I had a theoretically non-impacted test stubbornly crashing with:

Segmentation fault (Core Dumped)

Frustratingly, I couldn’t reproduce the crash in a debugger, which left me thinking it was a race condition for a while, but all the tricks I tried to hunt down the race didn’t give me any inclination as to what was going on. Also, the core was not being dumped, by default Ubuntu doesn’t do that. You need to enable core dumps with:

$ ulimit -c unlimited

Anyhow, I then debugged the core dump:

$ gdb build/tests/all core

Finding that the Seg-fault was inside Google Test! Googling for this issue showed no known issues around this part of the code, so I was a little stumped.

So what next?

Looking further up the stack trace showed that it happened inside _malloc_consolidate which previous investigations have lead me to attribute to memory corruption, most often heap trashing. Anyhow, I have a bunch of verification scripts set up to automatically run the unit tests, static analysis and dynamic analysis (valgrind!), and was indeed running them. Lo and behold, valgrind (memcheck) was turning up 47 issues, all of which were relating to a few lines of code (that I had been working on) where I was deleting an object which was still needed by an asynchronous process. I fixed the errors rather easily, and hey presto! No more Segmentation Fault.

You see the problem was not even the unit test in question, but about 10 tests earlier, the heap was getting trashed, which was causing the test runner to seg-fault later on.

So what did I learn?

In the past I’ve had policies on repositories that no code makes it in without a successful build and test-run. Usually I compile with -Werror (or similar) so that my code never has any warnings (because I have to fix them, since they’re treated as errors). This time I added cppcheck and valgrind because I figured they would be useful. This experience taught me that if I ever encounter weird issues while I’m working, I should run the full verification suite because I will probably find something related to the weird issues that way.

My first ever audio plugin

I’ve been procrastinating working on the OCA library project that I’ve started, so I ended up writing a really naive distortion plugin.

My SI-D1 distortion running as a standalone JACK application.
My SI-D1 distortion running as a standalone JACK application.

It works as a LADSPA, LV2, VST (untested) plugin, or as a standalone application (as pictured above). I used the DISTRHO Plugin Framework which is how I can target all the platforms. It should, in theory, build on Windows as well.

The code itself is quite simple: it pre-amplifies the signal, and then applies a very hard clipping algorithm. The clipping levels are determined by the “Range” and “Bias” parameters. Range sets the distance between the thresholds, around 0.0, and then Bias moves those thresholds up or down. It doesn’t sound great but its a start.

I haven’t provided a binary download as yet, but you can get it on github.

Try it out and let me know what you think.

Towards a cross-platform C/C++ dev environment

As you’ve probably read in my recent posts, I’ve been getting involved in the linux audio community. I’ve discovered that I really like linux – especially the concept of a system-wide package manager, I wish Windows had one that everybody used.

What I miss is Visual Studio. I really can’t overstate the hit my velocity has taken without intellisense. Its not that I’m incapable of programming without it, but it takes such a long time to look through html SDK/API docs. So, I’ve been trying to sort out a system where I can lint my code, build my projects, get some form of auto-complete, and have a few other C/C++ specific conveniences, like switching between .h* and .c* files with the same base name. I also don’t want to be tied to a build system. I’m kinda greedy, I want a graphical debugger if I can. I’ve got a solution now which shows potential.

As text editors go, I’ve grown to like Atom by GitHub over the last year or so. It is built in JavaScript (mostly, they moved a few things down to native code for performance, it is now quite slick, and a lot better than it was even 6 months ago), and it is super extensible, but in a nice way, not like Eclipse.

Triggering builds

One of the challenges with Linux is that there are quite a few build systems in common use: autotools, scons, waf, cmake to name a few. I like waf, but autotools is super common. Anyway, I didn’t want to be tied to a build system, but I do want to do basic things like build or configure my project from a keyboard shortcut. There’s a nice plugin for this: build. It has a bunch of built-in build systems that are mainly web-focused, but it lets you basically tie any shell command to the keyboard shortcut if you want, by providing a file called .atom-build.json.

Here’s what mine looks like:

{
     "cmd": "./waf",
     "name": "Build",
     "sh": "true",
     "targets" : [
         {
               "cmd": "./waf configure",
               "name": "Configure",
               "sh": "true"
         }
   ]
}

It’s not perfect – if you switch target it remembers which is a pain for doing things like configure, or clean, which you do once in a while, then go back to building. I’ll probably fork it and make a more C/C++ centric version one day, but it works well enough. Install it as follows:

> apm install build
Linting

Ok, so I can build, and each time I build, I get to see what’s wrong but sometimes deciphering a big blob of compiler output is difficult, and slow. Atom has a plugin which gives a generic linting infrastructure. I guess it was originally built for JSLint or something. Users can create providers to this linter to handle various languages, and it just so happens there is one which uses GCC, and one which uses Clang, so take your pick. I’m using the GCC one.

> apm install linter linter-gcc

You can change the default compiler flags, so I added -Werror, to make warnings appear as errors – I use that in my build script as well. linter-gcc supports a per-project config file which allows you to configure the settings per project, called .gcc-flags.json. I haven’t had to add any flags to it as yet, but at some point I’ll write a waf plugin to dump my CFLAGS and CXXFLAGS into this file whenever I configure. If you use autotools, I’m sure your configure script could do something like that too. A note about this one: at the time of writing (version 0.36.0), this plugin is actually a little broken. I fixed it and the PR was merged, so you should probably get it from source instead, or install it with apm and then replace the main.js with one from github.

Auto-complete

Again, Atom has existing auto-complete infrastructure in the form of the autocomplete-plus plugin, which can be extended by providers – in this case autocomplete-clang. Now this is one of the key reasons LLVM and Clang were created, to be used in tool integrations. You’ll need to have Clang installed such that the clang executable is on your path, but that’s easy on all platforms. Install the plugins with:

> apm install autocomplete-plus autocomplete-clang

You can customize your cflags etc per project with a .clang_complete file, which is the same file used by the clang-complete plugin for VIM, which is neat. Again, I haven’t had to modify these yet, but getting waf to auto-generate the file should be pretty easy.

Switching files

Ok, so we can build, lint, and auto-complete now. All that’s left from my wish-list is switching between header and source files with the same name, and debugging. Lets deal with the former, there’s a plugin for that.

> apm install switch-header-source
Debugging

This one is probably the least satisfying. I’ve installed a plugin called atom-debugger, which basically bootstraps gdb. At the moment it is very basic, but I think it could be improved quite quickly, so I’ll probably find myself contributing to it. Install it with:

> apm install atom-debugger
Conclusion

Linux is actually a really awesome environment to write code in, mainly because of system wide package management, and a defined scheme for where libraries, headers etc go, which means that you very seldom have to add special include or lib directory paths to your compiler flags. Visual Studio is an incredibly powerful IDE, which you miss when you are relegated to a mere text editor. Atom, is a great, highly extensible text editor which can be extended to give you quite a functional IDE for developing in C/C++, without being tied to a buildsystem or compiler, and it can be done on Linux, OSX and Windows. Boom.

 

Building Ardour on Ubuntu

[UPDATE] I’ve found that the instructions below cause jackd2 to be uninstalled, which causes some problems. Reinstalling the jackd2, libjack-jackd2-0, libjack-jackd2-dev and pulseaudio-module-jack packages should fix those issues[/UPDATE]

I’m taking a journey into audio software, and I’ve been playing around on Linux because there’s quite a good open-source audio community out there. The shining star (in my opinion) is Ardour, a really excellent, open-source DAW. In fact Harrison Mixbus is based on it.

Anyway, many of the developers who work on Ardour use Debian, but I prefer UbuntuStudio, so here are some simple steps to building on Ubuntu. I’m assuming that you already have a dev environment set up (ie you’ve gone ahead and installed build-essentials, git etc).

These instructions are pulled largely from the Ardour instructions, with a few other useful tips given to me by Robin Gareus and Paul Davis in IRC.

I’m running Ubuntu Studio 15.04.

Step 1 – The dependencies

The Ardour website has a list of Ardour’s dependencies. You’ll notice that there are a few libs which they have modified versions of. You don’t need these special ones – you can get by without them. I learned yesterday that APT has very neat mechanism for installing these dependencies. A package maintainer can specify the list of dependencies required to build the package, not just to install it. So, to install the dependencies do the following:

> sudo apt-get build-dep ardour3

On Debian, that would actually give you all the dependencies for Ardour 4 (their Ardour3 package is actually version 4.x). On Ubuntu there are a couple more dependencies you need to install:

> sudo apt-get install vamp-plugin-sdk libtag1-dev libaubio-dev liblrdf0-dev librubberband-dev

Now you should have all the dependencies for Ardour.

Step 2 – Get the code

Assuming you have already changed to the directory where you want to clone ardour

> git clone git://git.ardour.org/ardour/ardour.git

Alternatively you could go to their github mirror and fork that, and then clone that to your machine. If you want to submit changes doing them via github PRs is by far the easiest way.

Step 3 – Build

Next change into the ardour directory that was cloned

> cd ardour

Then we build

> ./waf configure
> ./waf

If you are missing any depenencies then you should find out during the waf configure step.

Step 4 – Run

To run the version you just built

> cd gtk2_ardour
> ./ardev

Waf also lets you do install/uninstall/clean etc.

Linux Audio

I’ve recently started learning about signal processing, and a programming language called faust. They are very linux focused, and so I figured the best way to get up and running with it would be to dive into linux, it’s been a while since I really gave linux a fair look, so I was due for it.

Without digressing too far, I would like to announce that I now quite like linux, and I could pretty much use if for all of my home computing apart from gaming.

Anyhow I’m running Ubuntu Studio which is an audio-focused Ubuntu derivative, and there was one little snag Continue reading “Linux Audio”