On Monday, September 26, 2016 2:26:53 AM EDT Florian Jung wrote: > Hey Tim, hey all!
Howdy! > > i just tried the audio stretching branch (with rubberband), i'm quite > impressed! Good work :) > > But one thing, will you make the wave parts just stretch automatically > with the song's playback speed? > i'd expect stuff to get slower when i change the overall tempo of the > song (e.g. if I bounced a track to a wave track, then change the tempo. > it should change the wave track's tempo as well). Unfortunately I have not implemented tempo changes yet. I reasoned that it was *most* important to get manual stretching in order before attempting tempo, 'cause tempo is /real/ tricky. If manual stretching could be demonstrated, we're on our way... At least one could begin to have some fun with the manual stretching. > But hell, this must have been a huge pain under the hood, wasn't it? How > are you doing this? are you operating the stretcher "live", with a > feedback loop between current position and "where it should be"? There are two new tool buttons in the wave editor. (No icons yet, borrowed the [ ] icon for now). One is called 'samplerate' and one is called 'stretch'. Selecting one of these tools allows you to place special markers on the canvas, called 'samplerate' and 'stretch' markers, denoted by their colour. Once these markers are placed, they can be *moved* which resamples or stretches the wave as can be seen graphically and heard, a la Pro Tools etc., you should see and hear the wave squish or stretch. Select a marker and hit delete to remove it. I am using 'live' realtime resampling and stretching. There are actually *two* resampling mechanisms going on: 1) One is automatic - it automatically resamples /any/ open wave. No more wave rewriting headaches at open time - MusE does not care what samplerate a wave is or what samplerate Jack is running, it's all automatic, /including/ anything that is related to frames - specifically the audio automation drawing and playback for example. (Later we will add easy rendering features to let users decide to render the wave at open time, or at any time later with a right-click. I have temporarily disabled the section Andrew added which re-writes waves at open time.) 2) The other mechanism is the moveable /manual/ resampling markers. This allows you to resample any /portion/ of a wave for interesting effects - think of the ending of 'War Pigs' by Black Sabbath, although I really should add easier 'ramping' facilities for that. So, after that we have the moveable 'stretch' markers: Straight forward, they stretch or squish /any/ portion of a wave. (I am working on /overall/ event and part stretch for easier simple part 'border' manipulation rather than having to use markers.) About the configuration, and library usage: I chose a highly modular approach since the variety of stretch/resample libraries and their purposes and quality are not known fully until one tries them. Each supported library is wrapped in a self-contained dynamic library so that it may be /discovered/ at run time. This was essential if we are to support new libraries quickly. Each dynamic library contains everything required: The process functions and the special graphical options dialog, unique to each 'brand' of library. Essentially it is identical to say a LADSPA or DSSI plugin framework for example, except the inputs are /streaming/ inputs. You see, unfortunately stretching is the /one/ feature that is not supported by most plugin frameworks. GStreamer is an exception. So I basically had to invent a new plugin framework. I brought this up on LAD, I proposed adding /streamable/ inputs and outputs to LV2 for example, and Dave Robillard was kind to respond and said it is possible but he didn't see the benefit. I haven't responded yet, I'll wait 'till I investigate the possibility more. There are three basic techniques most libraries use: SOLA time domain shifting (cheap), phase vocoding (better), and wavelets (best). Unfortunately that last item (the best) is not easy, there seem to be no readily available libraries. Commercial libraries are out there, but since there's no way to distribute this stretch technology in a plugin, they usually only offer APIs for commercial product developers, not individuals. So I contacted one company (THE company) and explained the situation and asked if it was possible for an /individual/ to purchase a license, since we here in Linux Land compile our /own/ software. The president kindly responded they would not be opposed per se, if it could work out favourably, and said they'd consider it. We'll see. Hopefully. I have been studying and looking forward to using that technology for almost 20 years, I would like to see it in action. The configuration (the user's choice of which library to use on which function and so on) may seem like overkill, but so far I am still feeling that it is a necessary evil. Eventually I hope to whittle away some of that in favour of say, just Rubberband for the graphical since it's not terribly important. The ability to use a library for graphics /hinges/ on its ability to start drawing a wave /cleanly/, without transients, at any place since the graphics wants to jump around and draw at random locations triggered by update() for example. I rewrote the drawing routines for special optimization - it used to seek the wave file on /every/ graphic slice drawing (waaay slow) but now it only seeks at the 'edge' and then progresses naturally. And yes, Flo this is where it got real tricky because getting rid of those 'first seek' transients was tough. I managed to get libsamplerate to work fine - it should look and sound great ! * Be sure to use 'linear' with SRC for the graphics - no filtering because that distorts the drawing. I'll work on making that more permanent. However at the moment I'm having some difficulty with Rubberband. I got past one difficult issue of flushing 'hidden' internal buffers such that /full/ graphic updates should look somewhat OK but you'll notice weird graphic distortion on small updates only if you zoom in . These difficulties are experienced /only/ when zoomed in enough for the 'live' realtime drawing to kick in, where it suddenly becomes real demanding. Otherwise at zoomed out levels it should look just fine since those are /cached/ wave displays :-) * For now, when you use Rubberband for graphics be sure to choose certain options from its unique dialog such as smoothing (I can't remember the exact ones that make it look best at the moment, try experimenting.) These are just graphic issues. In all cases the resulting audio however, should be just fine. Whew... A lot to say. Well enough for now. The code is fairly tight and debugged right now but obviously is a complete mess with comments to be removed from /all/ that tough experimentaion, eh... Thanks. Tim. > > or are you setting the tempo keypoints before starting the playback? > > Cheers, > flo ------------------------------------------------------------------------------ _______________________________________________ Lmuse-developer mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/lmuse-developer
