Re: [music-dsp] Real-time pitch shifting?

[Chris Cannam]

On Tue, 22 May 2018, at 14:09, Sound of L.A. Music and Audio wrote:
> With the voice it is even more tricky, since the formant shaping is 
> different for other frequencies. One reason is, that there are more than 
> one "equalizer" involved. Just putting the whole track to a hight frequ 
> will disreagard this.

Yes, with this sort of thing it's important to remember that what you're trying 
to do is not really possible.

Ultimately you're asking to produce the sound that would have resulted from the 
same performers playing the same thing faster, slower, or in a different key. 
There's nothing like enough information in the audio to do that. If a 
convenient fudge like the phase vocoder happens to work usefully in some cases, 
great.


Chris
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] Real-time pitch shifting?

[Chris Cannam]

On Tue, 22 May 2018, at 18:31, Matt Ingalls wrote:
> As far as I can understand it, it seems to be along the same lines as 
> that Bernsee blog, but with peak detection.

I think that should work a lot better than what you have -- although Eric 
Brombaugh's experience earlier in the thread may be more relevant.

Note that the Laroche-Dolson "identity phase locking" time-stretcher (a) is 
patented, although that may be about to expire, and (b) doesn't usually sound 
as good as more computationally-expensive methods that adjust phases 
individually but influenced by local gradient, as in Rubber Band or the newer, 
interesting approach in the recent paper Corey K just cited. (It tends to 
introduce a metallic edge.) The same might be true of the related pitch-shifter.

I guess it should be possible to do formant adjustment at the same time, which 
would be nice.


Chris
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] Real-time pitch shifting?

[Chris Cannam]

On Mon, 21 May 2018, at 22:08, robert bristow-johnson wrote:
> From: "Chris Cannam" <can...@all-day-breakfast.com>
> > I assume this wouldn't work with your use-case as you want to keep the 
> > frames synchronised so you can resynthesise everything in one go.
> it would need more buffering, but you can do it real-time, even with the 
> frames being reset and initially aligned with a transient.

Yep, agreed.

But Matt said "i have a bunch of streams in the frequency domain and wanted to 
independently manipulate their pitch without having to go back into time domain 
for each". I was guessing that meant he wanted to pitch-shift the streams 
individually, perhaps by different amounts, without changing the frame 
alignments for the individual streams, so they could be summed in the frequency 
domain and fed into a single IFFT/overlap/add at the end.


Chris
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] Real-time pitch shifting?

[Chris Cannam]

On Mon, 21 May 2018, at 19:54, Matt Ingalls wrote:
> "Rubber Band implements pitch-shifting using the standard technique of 
> resampling…
> This method has the advantage of making the amount of pitch adjustment 
> extremely accurate and stable, even for small changes.”

It also means you only have to solve the auditory problem once -- if you can 
keep the artifacts down for the time stretcher, then pitch-shifting is free. 
Unfortunately it does create some complications in terms of managing the proper 
frame timing and so on.

I assume this wouldn't work with your use-case as you want to keep the frames 
synchronised so you can resynthesise everything in one go.

> And I hear what you are saying about the the post, looking at the code 
> was easier for me: 
> http://blogs.zynaptiq.com/bernsee/repo/smbPitchShift.cpp

Yes, the actual shifting step is bizarrely crude considering the work done in 
the rest of the code (and the post). He must have known that. It could surely 
be substantially improved by working on the shifting interpolation a bit -- not 
that I've tried it.


Chris
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] Real-time pitch shifting?

[Chris Cannam]

On Sat, 19 May 2018, at 21:34, RJ Skerry-Ryan wrote:
> It may not be the state of the art, but RubberBand
>  is, I believe, the best open source
> pitch shift / time stretch library out there at the moment, and can run in
> realtime on modern CPUs.

See here for a page very briefly summarising how Rubber Band works:
https://breakfastquay.com/rubberband/technical.html
In short, it's a phase vocoder that uses a handful of the available tricks to 
try to reduce phasiness, without doing any very expensive analysis such as 
sinusoidal modelling.

There is actually a fine sinusoidal-modelling time stretcher hiding in 
Audacity, using the SBSMS library by Clayton Otey. This isn't a real-time 
method as far as I can see, and is slow to run, but it's worth checking out -- 
you activate it by selecting the Change Tempo or Change Pitch effect and 
checking the option labelled "Use high quality stretching". Code at 
https://github.com/audacity/audacity/tree/master/lib-src/sbsms.

Stephan Bernsee's old post is a bit of a puzzle, since it contains quite a lot 
about analysis/resynthesis but very little about actual pitch shifting.


Chris
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] proper scaling of an FFT

[Chris Cannam]

On Wed, Nov 4, 2015, at 04:56 PM, Phil Burk wrote:
> Is there a "right" way or a "wrong" way or is there just "my" way?

I think in the context of a library there are two questions here -- what
scaling should an FFT implementation use (behind the scenes in the
functions it provides), and what scaling is appropriate for a particular
application that makes use of an FFT implementation.

Without getting into the second question, most libraries I've used
either scale by 1 in both directions (FFTW, KissFFT) or scale the
forward transform by 1 and the inverse by 1/M (MATLAB, NumPy).

The exceptions that come to mind are:

 * Apple's Accelerate framework scales the complex-complex forward
 transform by 1, but the real-complex forward transform by 2 (that's 2,
 not 2/M). In both cases it scales the inverse by 1/M

 * Fixed-point implementations, e.g. OpenMAX, which has you specify the
 scale factor when you call the function

My preference is for scaling by 1 in both directions and leaving it up
to the caller.


Chris
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] warts in JUCE

[Chris Cannam]

On Sun, Sep 6, 2015, at 01:50 PM, robert bristow-johnson wrote:
> otherwise, you're always going to be passing sampleRate 
> along with every AudioSampleBuffer.

This bit here -- this is the part that doesn't match real practice. You
would typically pass the sample rate when you construct a specific
processing object (i.e. literally when you call its constructor, in C++
terms). It's not global, it's not hardcoded, but you still don't need to
pass it to every processing call. It's just fixed for a particular
instance of a processor. Say you have a filter object, you construct
that to run at a particular sample rate, and then you just feed it the
sample values each time.

You mentioned channel count as well -- same thing applies. I imagine (as
Ethan suggested earlier) that channel count is only included in the
buffer object because it wouldn't be possible to unpick the raw C++
object without it. Arguably that's a limitation of C/C++ (pointer-based
arrays don't know their size) rather than by design.


Chris
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] warts in JUCE

[Chris Cannam]

On Sat, Sep 5, 2015, at 11:37 PM, Daniel Varela wrote:
> Robert is right, sample rate is part of the audio information

Robert is certainly right -- an encapsulation of an audio buffer in
isolation must include the sample rate. I don't expect anyone in this
thread disagrees with that.

All the same, a lot of software breaks down the abstraction differently
and associates the sample rate with a processing context (or graph)
instead of with individual buffers. You could say the buffers then are
incomplete, just series of values rather than self-contained
representations of audio. As I understood it, Robert was wondering why
anyone would do that (assuming some reason other than "programmer is
just stupid" or "programmer hasn't learned the first thing about sampled
audio") and I was trying to articulate an answer. It makes no sense to
represent that as a question of right or wrong.

(Perhaps Jules's mistake was to name the class AudioSampleBuffer in the
first place.)


Chris
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] warts in JUCE

[Chris Cannam]

On Sat, Sep 5, 2015, at 01:23 AM, robert bristow-johnson wrote:
> same problem (if that *is* a problem) if you pass sampleRate alongside 
> the AudioSampleBuffer rather than inside of it.

I agree, that would be the worst of both worlds. (At least from this
perspective -- there may be other reasons to make an API that way.)

The most relevant point I think is that, because this is a class-based
object oriented language, your filter function doesn't have to be just a
function: it will generally be implemented as a class (with a processing
function, but also a constructor) instead. And the class can be supplied
the sample rate on construction. So you don't have to pass the sample
rate in to the processing function at all, but by constructing different
instances of the class you can still set up different versions of the
function according to the sample rate in use. There is a middle ground
between "a single fixed sample rate throughout the whole application"
(or, worse, 44100 hardcoded) and "every process call gets the sample
rate supplied to it".

> now suppose the source of audio changed (like a different file or 
> different internal buffer is now gonna be sent out) and it's a different 
> sample rate attached to it.  now the function sees that it's a different 
> input rate and the interpolation step-size must change and it's 
> recalculated.

A function whose job is to convert between sample rates is something of
a special case.

> no, it's the opposite.  i gotta filter method and you send me an 
> AudioSampleBuffer and some parameters for the filter such as type (LPF, 
> etc), Q, and cutoff frequency (in units of physical frequency).  i 
> cannot do the job until i know what the sample rate is.  the best thing 
> is if i look at the AudioSampleBuffer itself

That's likely to be too late. When you get audio data, you may be
already in a realtime processing context and you may not be able to do
rate dependent initialisation any more. You want to know the sample rate
before that, and often you aren't in a position to reinitialise on the
fly if it changes. So it makes sense for a processing class to get it on
construction.


Chris
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] warts in JUCE

[Chris Cannam]

(I have never used JUCE, don't know its interfaces, and have no side to
take)

On Fri, Sep 4, 2015, at 08:38 PM, robert bristow-johnson wrote:
> i find it odd that it seems to all inherent to a parcel of sound 
> represented in a computer are the number of samples and the number of 
> channels (and both are in an AudioSampleBuffer in addition to the audio 
> samples themselves) yet the sample rate is considered "ancillary".

If your sample buffer has the sample rate attached, then any library
function that consumes a sample buffer cannot be correct until its
author has made a decision about what to do if the sample rate changes
between calls.

In most cases the library won't be able to handle this, and in normal
processing use it won't anyway, so the obvious thing would be to fail,
or else to ignore the changed rate. Few developers would actually write
(correctly) the code necessary to support a change in sample rate
between process calls.

If the sample rate can't change between calls, then the right thing is
to enforce that in the API. In an object-oriented world, that means that
unchanging properties like sample rate and channel count are given when
the object is constructed, rather than when its methods are called. So
they're passed in to the constructor instead of to the methods that
accept audio buffers.

That changes the conception of what the function does -- it no longer
accepts an audio buffer at a particular sample rate and works on that;
instead it has already been constructed to work at a particular sample
rate -- it produces a function that processes audio at a particular
sample rate -- and it accepts all audio buffers on the assumption that
they match it. That's much simpler to deal with and probably also
simpler to call.

The way you propose things is also very sound in outline, I think, but
it's more like a functional-programming paradigm. In an object-oriented
world you would bind the context as soon as you construct the object. So
the things that aren't going to change during the lifetime of the object
go into its constructor, and that way the API itself enforces the fact
that they can't change, so that the caller doesn't have to consider them
in future method calls and the implementor doesn't have to worry about
what might happen if they do change.

In a functional world you don't have state in the object, and so you
work by taking a package of data, processing the relevant parts, and
returning the new package. In that world it's a lot more likely that you
have unchanging data in the same package as the stuff you want to
process. There you look at self-contained data bundles -- you have this,
you process it in such-and-such a way, you get that -- and there's no
context as such.

And as for sample rate, so for channel count -- if you can't easily cope
with it changing, it shouldn't be in the function call but in the object
constructor. Same would go for block size, if that's supposed to be
fixed.

(Disclaimer: my own code surely fails this all over the place)


Chris
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

[music-dsp] Last call to register for SoundSoftware 2014!

[Chris Cannam]

[Apologies for crossposting]

A reminder that the third annual SoundSoftware Workshop, a full day of
presentation sessions about all aspects of software in audio and music
research with a sociable and friendly atmosphere, takes place on Tuesday
8th July in London.

The speaking programme is now complete and you can read all about it
here:

  http://soundsoftware.ac.uk/soundsoftware2014

Registration has just been extended for a few more days -- it will now
close at the end of Tuesday 1st July. Attendance is free, but you must
register to secure a place! The registration page is here:

  http://www.eventbrite.com/e/soundsoftware-workshop-2014-tickets-11733734917

The previous two workshops have been varied and interesting, and
relevant to every audio and music researcher with an interest in getting
their work out so that other people can use it. The programme for this
year's event suggests it will be just as rewarding. We look forward to
seeing you there!


Chris Cannam
SoundSoftware.ac.uk
--
dupswapdrop -- the music-dsp mailing list and website:
subscription info, FAQ, source code archive, list archive, book reviews, dsp 
links
http://music.columbia.edu/cmc/music-dsp
http://music.columbia.edu/mailman/listinfo/music-dsp

[music-dsp] SoundSoftware 2014: Third Workshop on Software and Data for Audio and Music Research

[Chris Cannam]

(please forward as appropriate!)

*** CALL FOR PARTICIPATION ***

SoundSoftware 2014: Third Workshop on Software and Data for Audio and
Music Research

Queen Mary University of London, 8 July 2014

About the workshop:
  http://soundsoftware.ac.uk/soundsoftware2014

Registration link:
  http://www.eventbrite.com/e/soundsoftware-workshop-2014-tickets-11733734917

 Registration Deadline: 27 June 2014 ***

Software and data have increasingly become an integral part of our
research in the audio and music community. In the process, the way that
we do research in this field is changing, with methods and software
becoming increasingly complex, and reliable datasets becoming
increasingly important.

The third SoundSoftware.ac.uk one-day workshop on Software and Data for
Audio and Music Research will include talks and discussions on issues
such as robust software development for audio and music research,
reproducible research, management of research data, and open access.

While our project is concentrating on the audio and music research
community, we believe many of these issues will have wider interest, and
we want to link with other communities for which proper handling of
research software and data are important.

The workshop will take place on Tuesday, 8 July 2014 at Queen Mary,
University of London.

Confirmed speakers:

Oliver Bown (Sydney University)
Dimitrios Bountouridis (Utrecht University)
Nela Brown and Katerina Kosta (Queen Mary, University of London)
John ffitch (Codemist)
Di Mainstone (Artist) and Alessia Milo (Queen Mary, University of
London)
Wadud Miah (Queen Mary, University of London)
Guillaume Pellerin (Parisson, Telemeta)
Bob Sturm (Aalborg University)
Tillman Weyde (City University)

*** Registration ***

Workshop registration is free, but you must register to secure a
place. To register please visit:

  http://www.eventbrite.com/e/soundsoftware-workshop-2014-tickets-11733734917
--
dupswapdrop -- the music-dsp mailing list and website:
subscription info, FAQ, source code archive, list archive, book reviews, dsp 
links
http://music.columbia.edu/cmc/music-dsp
http://music.columbia.edu/mailman/listinfo/music-dsp

[music-dsp] Last call: SoundSoftware 2013: Workshop on Software and Data for Audio and Music Research

[Chris Cannam]

*** LAST CALL FOR PARTICIPATION ***
*** New Registration Deadline: 24 June 2013 ***

SoundSoftware 2013: Workshop on Software and Data for Audio and Music Research

Queen Mary University of London
26 June 2013

http://soundsoftware.ac.uk/soundsoftware2013

If you're interested in attending our second workshop on Software and
Data for Audio and Music Research, you'll be delighted to learn that
registration is still open and we have a few places left.

(Owing to an administrative error, registration was closed earlier
than our stated deadline -- so if you have already tried and failed to
register, please try again with our apologies.)


*** About the workshop ***

Software and data have increasingly become an integral part of our research
in the audio and music community. In the process, the way that we do research
in this field is changing, with methods and software becoming increasingly
complex, and reliable datasets becoming increasingly important.

The second SoundSoftware.ac.uk one-day workshop on Software and Data for
Audio and Music Research will include talks and discussions on issues such as
robust software development for audio and music research, reproducible research,
management of research data, and open access.

While our project is concentrating on the audio and music research community,
we believe many of these issues will have wider interest, and we want to link
with other communities for which proper handling of research software and data
are important.

Confirmed speakers:

Nick Barnes (Climate Code Foundation)
Angus Hewlett (FXpansion)
Rebecca Stewart (Codasign)
Mahendra Mahey (British Library, BL Labs)
Jamie Bullock (Birmingham Conservatoire)
Bryan Pardo (Northwestern University)

Registration for this workshop is free, but delegates must register to
secure a place.

To register, and for further information about the workshop, please visit:
http://soundsoftware.ac.uk/soundsoftware2013

You may also like to refer to the programme and footage from last
year's event, at http://soundsoftware.ac.uk/soundsoftware2012.

Note that this workshop is the day after the Listening in the Wild
workshop, also at Queen Mary; delegates may be interested in attending
both. See http://listeninginthewild.eventbrite.co.uk/.
--
dupswapdrop -- the music-dsp mailing list and website:
subscription info, FAQ, source code archive, list archive, book reviews, dsp 
links
http://music.columbia.edu/cmc/music-dsp
http://music.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] stuck with filter design

[Chris Cannam]

On 18 November 2012 22:24, Bjorn Roche bj...@xowave.com wrote:
 Great. I guess that means LADSPA does not use the usual [-1,1] range.

LADSPA doesn't enforce anything -- it's really up to the host. But the
spec in header does say For audio it is generally assumed that 1.0f
is the `0dB' reference amplitude and is a `normal' signal level.


Chris
--
dupswapdrop -- the music-dsp mailing list and website:
subscription info, FAQ, source code archive, list archive, book reviews, dsp 
links
http://music.columbia.edu/cmc/music-dsp
http://music.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] Window presum synthesis

[Chris Cannam]

On 20 April 2012 14:15, Domagoj Šarić dsar...@gmail.com wrote:
 ...but unfortunately all of these however seem to stop at the
 analysis stage (i.e. we've gotten a more precise spectrum and that's
 all we need).

Mark Dolson's CARL phase vocoder
(http://www.crca.ucsd.edu/cmusic/cmusic.html) from 1984 uses window
presum if the FFT is shorter than the window, and includes
resynthesis.

I suppose the idea is to reduce some of the annoying artifacts of
phase vocoder scaling at the expense of time-domain aliasing, which
may be more tolerable for some signals and listeners. My impression is
it doesn't work all that well, but it may have made a significant
difference to computational cost in 1984. The code is worth reading
though, it's clear and has illuminating comments.


Chris
--
dupswapdrop -- the music-dsp mailing list and website:
subscription info, FAQ, source code archive, list archive, book reviews, dsp 
links
http://music.columbia.edu/cmc/music-dsp
http://music.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] looking for a flexible synthesis system technically and legally appropriate for iOS development

[Chris Cannam]

On 8 February 2011 15:50, Stefan Kersten s...@k-hornz.de wrote:
 the known precedents make it a risky undertaking trying to distribute _any_
 GPL'd application through the app store, because apple might decide to take it
 out in any moment; not a sound foundation to build any business model on ...

I imagine it's more a question of whether other copyright holders in
the GPL software might object (in which case Apple would pull it --
this is what happened with VLC and Gnu Go) than of Apple spontaneously
deciding to remove it.

I'm not finding it easy to work out the exact situation from the TCs
at http://www.apple.com/legal/itunes/us/terms.html#APPS.  This
document notes that individual applications may have different EULAs
from the default Apple-provided one, so the licensed application
EULA (which is definitely GPL-incompatible) wouldn't necessarily have
to apply -- but the implication seems to be that the App Store Product
Usage Rules would still apply because they are not part of the section
identified as the EULA.  And they also appear incompatible with the
GPL (e.g. section 6 of GPLv2), though it may depend on who the You
in the GPL is understood to refer to in this context (Apple or you).

Basically I think if you own the copyright or have the agreement of
everyone who does, then in practice you probably can get away with
putting GPL'd code in the Apple stores -- but as soon as someone might
reasonably object, then you're quite likely stuffed regardless of
whether you made your best effort to comply with the GPL or not.  A
pity it's such a grey area, it does make things rather unpredictable.

Doesn't look like there's any such problem with the Android market, as
far as I can see
(http://www.google.com/mobile/android/market-tos.html,
http://www.android.com/us/developer-distribution-agreement.html).


Chris
--
dupswapdrop -- the music-dsp mailing list and website:
subscription info, FAQ, source code archive, list archive, book reviews, dsp 
links
http://music.columbia.edu/cmc/music-dsp
http://music.columbia.edu/mailman/listinfo/music-dsp