Don’t forget the cross-fade

A quick post about a little lesson I learned tonight whilst editing some audio: don’t forget the cross-fade.

I recorded some synth drum tracks to audio (so that I wouldn’t have to open up the drum machine again), and then trimmed the tracks down so that they ended tidily at the end of the bar: BAD! When I later when to loop those tracks I realised that I hadn’t left room for the quick fade from one clip to another that you generally need when putting clips right next to eachother. What I needed to do was leave a tiny bit extra on the end so that the clips could overlap without messing up the timing.

Luckily my trimming wasn’t destructive, so I could extend the clips back out.

Here’s a screenshot of what I mean.

Ardour Clip XFade

The value of valgrind

This weekend I got quite a lot done on liboca, mainly working on the network stack, and the basic implementation of OCP.1, which is the TCP/IP-based protocol for OCA (There is room in the OCA standard for other transports). Anyhow, at one point, my unit tests broke in a suite of tests that tested code that should not have been affected by the code I was working on. In the end the fact that I had set up Valgrind, ended up showing me how to solve my problem, so I learned that hard way that it is an essential tool to run along with your standard test suite.

What is Valgrind?

"Valgrind is an instrumentation framework 
for building dynamic analysis tools. There 
are Valgrind tools that can automatically 
detect many memory management and threading 
bugs, and profile your programs in detail. You 
can also use Valgrind to build new tools. " 
                                      - valgrind.org

In its simplest form, it takes your code, instruments it and runs it on a processor emulator, and tells you if you’ve leaked memory, accessed memory you shouldn’t have, and a whole bunch of other stuff – I encourage you to check out the website for more detail.

So what happened?

So we already got to the fact that I had a theoretically non-impacted test stubbornly crashing with:

Segmentation fault (Core Dumped)

Frustratingly, I couldn’t reproduce the crash in a debugger, which left me thinking it was a race condition for a while, but all the tricks I tried to hunt down the race didn’t give me any inclination as to what was going on. Also, the core was not being dumped, by default Ubuntu doesn’t do that. You need to enable core dumps with:

$ ulimit -c unlimited

Anyhow, I then debugged the core dump:

$ gdb build/tests/all core

Finding that the Seg-fault was inside Google Test! Googling for this issue showed no known issues around this part of the code, so I was a little stumped.

So what next?

Looking further up the stack trace showed that it happened inside _malloc_consolidate which previous investigations have lead me to attribute to memory corruption, most often heap trashing. Anyhow, I have a bunch of verification scripts set up to automatically run the unit tests, static analysis and dynamic analysis (valgrind!), and was indeed running them. Lo and behold, valgrind (memcheck) was turning up 47 issues, all of which were relating to a few lines of code (that I had been working on) where I was deleting an object which was still needed by an asynchronous process. I fixed the errors rather easily, and hey presto! No more Segmentation Fault.

You see the problem was not even the unit test in question, but about 10 tests earlier, the heap was getting trashed, which was causing the test runner to seg-fault later on.

So what did I learn?

In the past I’ve had policies on repositories that no code makes it in without a successful build and test-run. Usually I compile with -Werror (or similar) so that my code never has any warnings (because I have to fix them, since they’re treated as errors). This time I added cppcheck and valgrind because I figured they would be useful. This experience taught me that if I ever encounter weird issues while I’m working, I should run the full verification suite because I will probably find something related to the weird issues that way.

My first ever audio plugin

I’ve been procrastinating working on the OCA library project that I’ve started, so I ended up writing a really naive distortion plugin.

My SI-D1 distortion running as a standalone JACK application.
My SI-D1 distortion running as a standalone JACK application.

It works as a LADSPA, LV2, VST (untested) plugin, or as a standalone application (as pictured above). I used the DISTRHO Plugin Framework which is how I can target all the platforms. It should, in theory, build on Windows as well.

The code itself is quite simple: it pre-amplifies the signal, and then applies a very hard clipping algorithm. The clipping levels are determined by the “Range” and “Bias” parameters. Range sets the distance between the thresholds, around 0.0, and then Bias moves those thresholds up or down. It doesn’t sound great but its a start.

I haven’t provided a binary download as yet, but you can get it on github.

Try it out and let me know what you think.

Building Ardour on Windows with MSYS2

NOTE: This post was written for Ardour 4. There is at least one more dependency (liblua, possibly more). Since I wrote this article, I’ve mostly shifted my development environment to linux, hence I haven’t kept this post up to date. So, caveat emptor, YMMV, and other suitable disclaimers. If you just want to use Ardour on Windows, it is now supported on Windows, so why not donate to the Ardour developers and get an official download? I’m currently toying with building Ardour for Windows via the Windows Subsystem for Linux on Windows 10, but a post on that is a bit far away.

Building Ardour on Linux is quick and easy. Building Ardour on Windows…isn’t. That being said, I bashed my way through it, and have come up with a strategy that will get you up and running.

Who is this post for?

This post is most definitely for developers, familiarity with MSYS2 is handy, but as I was basically learning how MSYS2 works as I went, this post won’t assume terribly much on that score. If you just want to use Ardour on Windows I highly recommend downloading a nightly build and using that instead. Keep in mind that Ardour is un-ashamedly built for linux, and it is fantastic on that platform, so if you want the most out of Ardour, use it on linux. If you MUST use windows, know that at present it isn’t supported, and sometimes the nightly builds might be broken. That being said the Ardour developer community is looking into how support could become possible one day.

Other things you might want installed

I find it is handy to have JACK and ASIO4ALL installed.

Getting Started

The first thing you’ll need to do is install MSYS2, but what is MSYS2? Google says this:

MSYS2 is a successor of MSYS and MinGW-builds. 
MSYS2-shell uses “pacman” for downloading packages from
repo, and these are GPG signed & verified. Packages are
by developer: lexx83 (Alexpux), mingwandroid, niXman.

Essentially it is a very light gnu-like environment for Windows, like a very stripped down cygwin. It makes it a lot easier to build linux programs and libraries for windows that don’t use Cmake. There is a good set of instructions here on how to install it and set it up, so I won’t copy them.

Great, now that you’ve followed all the way through that, you should have an up-to-date msys2 system. Now we have a few packages that need to be installed, well, quite a few, most of them from source, but we’ll do it in four batches: tools, prebuilt dependencies, source dependencies, and then some more pre-built packages which require some of our source packages. Before that we should discuss the three shells available to you:

  • MSYS2 shell
  • MinGW-w64 Win64 shell
  • MinGW-w64 Win32 shell

Essentially, the MSYS2 shell is a runtime environment with full posix support. This is most useful for the unix tools. The other two shells have less posix compliance, but play much nicer with the windows APIs, so if you want to build a windows application, you’ll use these. Of these latter two, I’m only using the MinGW-w64 Win64 shell, which uses a 64bit toolchain. That being said, it is easiest of you do all of the package installation (both pre-built, and from source) in the MSYS2 shell. There’s a bunch more detail here.

Install various tools
> pacman -S mingw-w64-x86_64-toolchain python3 \
python3-setuptools mingw-w64-x86_64-python3 \
mingw-w64-x86_64-python3-setuptools python2 \
python2-setuptools pkg-config \
mingw-w64-x86_64-pkg-config autoconf automake perl \
gtk-doc flex bison patch libtool \
mingw-w64-x86_64-libtool wget git nasm \
mingw-w64-x86_64-nasm dos2unix mingw-w64-x86_64-cmake

This will take a while based on your internet connection, etc.

Install pre-built dependencies
> pacman -S mingw-w64-x86_64-glib2 \
mingw-w64-x86_64-gobject-introspection \
mingw-w64-x86_64-pkg-config mingw-w64-x86_64-c-ares \
mingw-w64-x86_64-ca-certificates \
mingw-w64-x86_64-gnutls mingw-w64-x86_64-libidn \
mingw-w64-x86_64-libssh2 mingw-w64-x86_64-rtmpdump \
mingw-w64-x86_64-gnutls libgnutls-devel \
libutil-linux-devel gtk-doc \
mingw-w64_x86_64-docbook-xsl intltool \
mingw-w64-x86_64-libjpeg-turbo \
mingw-w64-x86_64-jbigkit \
mingw-w64-x86_64-ladspa-sdk

Build and install source dependencies

So the thing about MSYS2 is that it stays very “bleeding edge”, whereas most of the Ardour devs use Debian, which has quite a long update cycle. Both approaches have their merit, but they come into conflict when you want to build something that expects debian-ish packages, but have only the most up-to-date packages available. Thankfully, others have blazed this trail before me and have created repositories of package build scripts for most of the packages you’ll need for Ardour. Most of these are quite straight-forward to install, but there are some mighty strange quirks, which we’ll discuss as we get to them.

To get these package scripts, clone my github fork of them:

> git clone https://github.com/guysherman/MINGW-packages

NB: Make sure to switch to the guypkgs branch!

As the README says, to make a given package do the following:

> cd ${package-name}
> makepkg-mingw

And to install it do:

> pacman -U ${package-name}*.pkg.tar.xz

Alternatively, you can do this to build and install at once:

> cd ${package-name}
> makepkg-mingw --install

There are also a few useful flags that you’ll need at times:

--nocheck - disables the 'check' step, which is usually
            unit tests
--nodeps - disables dependency checking, which is 
           useful because a few packages have cyclic 
           runtime dependencies
--skippgpcheck - doesn't check the signature of the 
                 package, I needed this on a couple of
                 packages, only use it if you have 
                 errors relating to unknow public keys,
                 or the pgp part of the process.

So, now that we’ve covered how to build and install the packages, build and install the following list of packages:

 mingw-w64-x86_64-icu
 mingw-w64-x86_64-boost
 mingw-w64-x86_64-curl
 mingw-w64-x86_64-fftw
 mingw-w64-x86_64-libusb
 mingw-w64-x86_64-libxml2
 mingw-w64-x86_64-libogg
 mingw-w64-x86_64-libflac
 mingw-w64-x86_64-libvorbis
 mingw-w64-x86_64-libsndfile
 mingw-w64-x86_64-libsamplerate
 mingw-w64-x86_64-soundtouch
 mingw-w64-x86_64-wineditline
 mingw-w64-x86_64-pcre with --nocheck
 mingw-w64-x86_64-cppunit
 mingw-w64-x86_64-taglib
 mingw-w64-x86_64-dlfcn
 mingw-w64-x86_64-gobject-introspection with --nodeps
 mingw-w64-x86_64-gnome-doc-utils
 mingw-w64-x86_64-gtk-doc
 mingw-w64-x86_64-gnome-common
 mingw-w64-x86_64-atk
 mingw-w64-x86_64-libpng

These should all be straight forward, but some of the will take a long time. Next we get into the weirdest quirk I’ve seen in a while. There are two libraries, harfbuzz and freetype. Turns out they have a circular dependency on eachother. So what you have to do is

First, build and install harfbuzz with –nodeps

> cd mingw-w64-x86_64-harfbuzz
> makepkg-mingw --nodeps --install

This will end up installing the following packages from the pre-built repositories, but the freetype will have no harfbuzz support:

mingw-w64-x86_64-cairo
mingw-w64-x86_64-fontconfig
mingw-w64-x86_64-freetype
mingw-w64-x86_64-pixman

So, next step delete the harfbuzz package that you already built and rebuild it (assuming your still in the harfbuzz directory):

> rm -rf mingw-w64-x86_64-harfbuzz*.pkg.tar.gz
> makepkg-mingw --nodeps --install

Next build and install the following packages:

mingw-w64-x86_64-freetype
mingw-w64-x86_64-fontconfig
mingw-w64-x86_64-pixman

Ok, now that’s the harfbuzz freetype weirdness out of the way. You should be good to go ahead and install the rest of the source packages:

 mingw-w64-x86_64-pango
 mingw-w64-x86_64-libjpeg-turbo
 mingw-w64-x86_64-jasper
 mingw-w64-x86_64-libtiff
 mingw-w64-x86_64-gdk-pixbuf2 
 mingw-w64-x86_64-shared-mime-info
 mingw-w64-x86_64-gtk2 
 mingw-w64-x86_64-libsigc++
 mingw-w64-x86_64-cairomm
 mingw-w64-x86_64-glibmm
 mingw-w64-x86_64-atkmm
 mingw-w64-x86_64-pangomm
 mingw-w64-x86_64-gtkmm
 mingw-w64-liblo
 mingw-w64-serd
 mingw-w64-sord
 mingw-w64-lv2
 mingw-w64-sratom
 mingw-w64-lilv
 mingw-w64-aubio
 mingw-w64-portaudio
 mingw-w64-jack
 mingw-w64-libltc

The second wave of pre-built packages

Ok, so now there are two more pre-built packages that you need to install:

> pacman -S mingw-w64-x86_64-rubberband \
mingw-w64-x86_64-vamp-plugin-sdk
Actually Building Ardour

Ok, so now you have all the dependencies you need to build and run Ardour (although we’ll discuss one other work around a bit later which solved a problem that I had, that I’m not sure if you will encounter or not).

For now you’ll need to clone my fork of Ardour. I’ll update when my pull request is accepted. Get the ardour code (do this from somewhere you’d like the code to go):

> git clone https://github.com/guysherman/ardour.git
> git clone git://git.ardour.org/ardour/ardour.git
> cd ardour

Ardour uses the waf build system. I found there was a tiny quirk with running it under MSYS. Run it as follows to configure Ardour:

> MSYSTEM=''
> ./waf configure \
  --dist-target=mingw \
  --prefix=/mingw64 \
  --configdir=/share

Then to build it, run

> ./waf

If you want to install it you can do so with

> ./waf install

I’m more interested in running it:

> cd gtk2_ardour
> ./ardev-win

Now, you might find that you get white text on white buttons, like I did, and you’ll probably see warnings about the gtk ‘clearlooks’ engine. I worked out a weird work around for this.

Weird clearlooks workaround

Go to a folder somewhere (ie ~/), and get the gtk-engines source code:

> cd ~/
> wget http://ftp.gnome.org/pub/GNOME/sources/gtk-engines/2.20/gtk-engines-2.20.2.tar.gz
> cd gtk-engines-2.20.2
> ./configure \
  --build=x86_64-w64-mingw32 \
  --host=x86_64-w64-mingw32 \
  --prefix=/mingw64
> make
> make install

You’ll get a whole lot of warnings that it can’t find various dlls. That’s no biggie in this case.

Next you’ll want to download a nightly build of ardour for windows and install it.

Then copy C:\Program Files\Ardour4\lib\gtk-2.0\engines\libclearlooks.la to C:\msys64\mingw64\lib\gtk-2.0\2.10.0\engines. It’ll want to over-write a file, which is all good. Now if you go back and run Ardour, it should look fine.

UPDATE: ACKNOWLEDGEMENTS

It occurred to me after I wrote this that I should have acknowledged all the help the Ardour devs gave me while I was asking pesky questions. They are a very helpful and welcoming group.

Building Ardour on Ubuntu

[UPDATE] I’ve found that the instructions below cause jackd2 to be uninstalled, which causes some problems. Reinstalling the jackd2, libjack-jackd2-0, libjack-jackd2-dev and pulseaudio-module-jack packages should fix those issues[/UPDATE]

I’m taking a journey into audio software, and I’ve been playing around on Linux because there’s quite a good open-source audio community out there. The shining star (in my opinion) is Ardour, a really excellent, open-source DAW. In fact Harrison Mixbus is based on it.

Anyway, many of the developers who work on Ardour use Debian, but I prefer UbuntuStudio, so here are some simple steps to building on Ubuntu. I’m assuming that you already have a dev environment set up (ie you’ve gone ahead and installed build-essentials, git etc).

These instructions are pulled largely from the Ardour instructions, with a few other useful tips given to me by Robin Gareus and Paul Davis in IRC.

I’m running Ubuntu Studio 15.04.

Step 1 – The dependencies

The Ardour website has a list of Ardour’s dependencies. You’ll notice that there are a few libs which they have modified versions of. You don’t need these special ones – you can get by without them. I learned yesterday that APT has very neat mechanism for installing these dependencies. A package maintainer can specify the list of dependencies required to build the package, not just to install it. So, to install the dependencies do the following:

> sudo apt-get build-dep ardour3

On Debian, that would actually give you all the dependencies for Ardour 4 (their Ardour3 package is actually version 4.x). On Ubuntu there are a couple more dependencies you need to install:

> sudo apt-get install vamp-plugin-sdk libtag1-dev libaubio-dev liblrdf0-dev librubberband-dev

Now you should have all the dependencies for Ardour.

Step 2 – Get the code

Assuming you have already changed to the directory where you want to clone ardour

> git clone git://git.ardour.org/ardour/ardour.git

Alternatively you could go to their github mirror and fork that, and then clone that to your machine. If you want to submit changes doing them via github PRs is by far the easiest way.

Step 3 – Build

Next change into the ardour directory that was cloned

> cd ardour

Then we build

> ./waf configure
> ./waf

If you are missing any depenencies then you should find out during the waf configure step.

Step 4 – Run

To run the version you just built

> cd gtk2_ardour
> ./ardev

Waf also lets you do install/uninstall/clean etc.

Linux Audio

I’ve recently started learning about signal processing, and a programming language called faust. They are very linux focused, and so I figured the best way to get up and running with it would be to dive into linux, it’s been a while since I really gave linux a fair look, so I was due for it.

Without digressing too far, I would like to announce that I now quite like linux, and I could pretty much use if for all of my home computing apart from gaming.

Anyhow I’m running Ubuntu Studio which is an audio-focused Ubuntu derivative, and there was one little snag Continue reading “Linux Audio”