What about PortAudio? It seems to have some kind of custom license, so I don't know if it's boost-compatible: http://www.portaudio.com/license.html
It does look quite permissive, so if it's boost-compatible I see no reason not to use it. It takes care of handling playback for you on a multitude of platforms, with support for multiple audio driver standards (ASIO/WDM/*nix types..). All that's left is to make routines that can read/write different types of audio files. libsndfile is a good library, but that one is licensed under LGPL. I've been using it myself and it works fine for high-latency and low-latency audio playback and sequencing.
