Re: [music-dsp] Real-time pitch shifting?

[robert bristow-johnson]

 Original Message 

Subject: Re: [music-dsp] Real-time pitch shifting?

From: "Chris Cannam" 

Date: Mon, May 21, 2018 3:35 pm

To: music-dsp@music.columbia.edu

--



>

> On Mon, 21 May 2018, at 19:54, Matt Ingalls wrote:

>> "Rubber Band implements pitch-shifting using the standard technique of

>> resampling…

>> This method has the advantage of making the amount of pitch adjustment

>> extremely accurate and stable, even for small changes.”
Rubber Band sounds pretty good to me and to my ears made of clay.� i thought, 
back in my day, that SPEED (SoundToys) sounded pretty good.

�
> It also means you only have to solve the auditory problem once -- if you can 
> keep the artifacts down for the time stretcher, then pitch-shifting is free.
unless you're gonna worry about formants, pitch shifting is just resampling 
applied to the output (or input) of a time
scaler.� i would do whichever operation increases the data first.� so pitch 
shifting up would be time-stretch first, then resample.� pitch shifting down 
would be resample first and then time-compress.




> Unfortunately it does create some complications in terms of managing the 
> proper frame timing and so on.
resetting the frame alignment whenever a transient is detected is a real pain 
in the ass.� but the idea to do that then let the phases drift (glitchlessly) 
after the transient is a
good idea.� doesn't seem to solve the phasiness problem for human voice when 
pitches change but no transient is detected (then phases can get outa alignment 
again).� i really don't know a clean way of keeping those phases well aligned.
�
> I assume this wouldn't work with your use-case as you want to keep the frames 
> synchronised so you can resynthesise everything in one go.
it would need more buffering, but you can do it real-time, even with the frames 
being reset and initially aligned with a transient.�
but it gets complicated and the throughput delay gets large.

--



r b-j� � � � � � � � � � � � �r...@audioimagination.com



"Imagination is more important than knowledge."

�
�
�
�
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] Real-time pitch shifting?

[Eric Brombaugh]

On 05/21/2018 12:35 PM, Chris Cannam wrote:



And I hear what you are saying about the the post, looking at the code
was easier for me:
http://blogs.zynaptiq.com/bernsee/repo/smbPitchShift.cpp


Yes, the actual shifting step is bizarrely crude considering the work done in 
the rest of the code (and the post). He must have known that. It could surely 
be substantially improved by working on the shifting interpolation a bit -- not 
that I've tried it.


I've used the Bernsee stuff as a starting point for a real time phase 
vocoder-based audio effect running on an embedded ARM processor. I found 
that there was a lot of additional work required to get good sounding 
fractional shifts, so going beyond the crude approach used in the 
example code took more effort than I expected.


tl;dr - working in the frequency domain is more complicated than you 
might think at first...


Eric

___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] Real-time pitch shifting?

[Chris Cannam]

On Mon, 21 May 2018, at 19:54, Matt Ingalls wrote:
> "Rubber Band implements pitch-shifting using the standard technique of 
> resampling…
> This method has the advantage of making the amount of pitch adjustment 
> extremely accurate and stable, even for small changes.”

It also means you only have to solve the auditory problem once -- if you can 
keep the artifacts down for the time stretcher, then pitch-shifting is free. 
Unfortunately it does create some complications in terms of managing the proper 
frame timing and so on.

I assume this wouldn't work with your use-case as you want to keep the frames 
synchronised so you can resynthesise everything in one go.

> And I hear what you are saying about the the post, looking at the code 
> was easier for me: 
> http://blogs.zynaptiq.com/bernsee/repo/smbPitchShift.cpp

Yes, the actual shifting step is bizarrely crude considering the work done in 
the rest of the code (and the post). He must have known that. It could surely 
be substantially improved by working on the shifting interpolation a bit -- not 
that I've tried it.


Chris
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] Real-time pitch shifting?

[Matt Ingalls]

Ok thanks everyone for the info. and thanks RJ for the Rubber Band tech notes 
page, which states:

"Rubber Band implements pitch-shifting using the standard technique of 
resampling…
This method has the advantage of making the amount of pitch adjustment 
extremely accurate and stable, even for small changes.”

(This certainly is not the case with that shifting algorithm I was using from 
that Bernsee blog - even small adjustments created undesirable artifacts, 
And I hear what you are saying about the the post, looking at the code was 
easier for me: http://blogs.zynaptiq.com/bernsee/repo/smbPitchShift.cpp)

It seems that the freq-domain solutions people mentioned here are also 
stretch+resampling. I would be interested to know exactly how Melodyne does it!

I wish these sites had sound examples of just simple tones, it’s kind of hard 
to hear for artifacts with full-spectrum music examples..

Fyi - someone was wondering what exactly I was after. 
Basically, i have a bunch of streams in the frequency domain and wanted to 
independently manipulate
their pitch without having to go back into time domain for each.

Thanks,
Matt


> On May 21, 2018, at 6:46 AM, Chris Cannam  
> wrote:
> 
> 
> On Sat, 19 May 2018, at 21:34, RJ Skerry-Ryan wrote:
>> It may not be the state of the art, but RubberBand
>>  is, I believe, the best open source
>> pitch shift / time stretch library out there at the moment, and can run in
>> realtime on modern CPUs.
> 
> See here for a page very briefly summarising how Rubber Band works:
> https://breakfastquay.com/rubberband/technical.html
> In short, it's a phase vocoder that uses a handful of the available tricks to 
> try to reduce phasiness, without doing any very expensive analysis such as 
> sinusoidal modelling.
> 
> There is actually a fine sinusoidal-modelling time stretcher hiding in 
> Audacity, using the SBSMS library by Clayton Otey. This isn't a real-time 
> method as far as I can see, and is slow to run, but it's worth checking out 
> -- you activate it by selecting the Change Tempo or Change Pitch effect and 
> checking the option labelled "Use high quality stretching". Code at 
> https://github.com/audacity/audacity/tree/master/lib-src/sbsms.
> 
> Stephan Bernsee's old post is a bit of a puzzle, since it contains quite a 
> lot about analysis/resynthesis but very little about actual pitch shifting.
> 
> 
> Chris
> ___
> dupswapdrop: music-dsp mailing list
> music-dsp@music.columbia.edu
> https://lists.columbia.edu/mailman/listinfo/music-dsp
> 

___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] Real-time pitch shifting?

[Corey K]

"Phase Vocoder Done Right" is a fairly interesting new paper I came across,
that talks about a strategy to preserve both vertical and horizontal phase
coherence. Examples (which sound pretty good) and link to paper are here:
http://ltfat.github.io/notes/050/

-Corey

On Mon, May 21, 2018, 06:46 Chris Cannam 
wrote:

>
> On Sat, 19 May 2018, at 21:34, RJ Skerry-Ryan wrote:
> > It may not be the state of the art, but RubberBand
> >  is, I believe, the best open
> source
> > pitch shift / time stretch library out there at the moment, and can run
> in
> > realtime on modern CPUs.
>
> See here for a page very briefly summarising how Rubber Band works:
> https://breakfastquay.com/rubberband/technical.html
> In short, it's a phase vocoder that uses a handful of the available tricks
> to try to reduce phasiness, without doing any very expensive analysis such
> as sinusoidal modelling.
>
> There is actually a fine sinusoidal-modelling time stretcher hiding in
> Audacity, using the SBSMS library by Clayton Otey. This isn't a real-time
> method as far as I can see, and is slow to run, but it's worth checking out
> -- you activate it by selecting the Change Tempo or Change Pitch effect and
> checking the option labelled "Use high quality stretching". Code at
> https://github.com/audacity/audacity/tree/master/lib-src/sbsms.
>
> Stephan Bernsee's old post is a bit of a puzzle, since it contains quite a
> lot about analysis/resynthesis but very little about actual pitch shifting.
>
>
> Chris
> ___
> dupswapdrop: music-dsp mailing list
> music-dsp@music.columbia.edu
> https://lists.columbia.edu/mailman/listinfo/music-dsp
>
>
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] Real-time pitch shifting?

[Chris Cannam]

On Sat, 19 May 2018, at 21:34, RJ Skerry-Ryan wrote:
> It may not be the state of the art, but RubberBand
>  is, I believe, the best open source
> pitch shift / time stretch library out there at the moment, and can run in
> realtime on modern CPUs.

See here for a page very briefly summarising how Rubber Band works:
https://breakfastquay.com/rubberband/technical.html
In short, it's a phase vocoder that uses a handful of the available tricks to 
try to reduce phasiness, without doing any very expensive analysis such as 
sinusoidal modelling.

There is actually a fine sinusoidal-modelling time stretcher hiding in 
Audacity, using the SBSMS library by Clayton Otey. This isn't a real-time 
method as far as I can see, and is slow to run, but it's worth checking out -- 
you activate it by selecting the Change Tempo or Change Pitch effect and 
checking the option labelled "Use high quality stretching". Code at 
https://github.com/audacity/audacity/tree/master/lib-src/sbsms.

Stephan Bernsee's old post is a bit of a puzzle, since it contains quite a lot 
about analysis/resynthesis but very little about actual pitch shifting.


Chris
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp

Re: [music-dsp] Real-time pitch shifting?

[Neil Goldman]

SoundTouch is time domain, based on a Synchronous-OverLap-Add (SOLA). I
haven't gone through the source, but an article on their website explains a
few different methods and states at the bottom SoundTouch is SOLA based.

https://www.surina.net/article/time-and-pitch-scaling.html

I've used SoundTouch a lot since it's one of Reaper's built-in algorithm
options, anecdotally I can say it works well enough on monophonic sources
like a voice but anything complex enough quickly turns into "glitch as an
effect" type of artefacts rather than remaining very useful for its
original purpose.

I think Elastique Pitch is the best among Reaper's built-in realtime
algorithms, it has "Soloist" (for monophonic) and "Pro" options too, but it
seems that one is proprietary...

On Sun, May 20, 2018 at 10:17 PM, robert bristow-johnson <
r...@audioimagination.com> wrote:

>
>
>  Original Message 
> Subject: Re: [music-dsp] Real-time pitch shifting?
> From: "RJ Skerry-Ryan" 
> Date: Sat, May 19, 2018 4:34 pm
> To: music-dsp@music.columbia.edu
> --
>
> > It may not be the state of the art, but RubberBand
> >  is, I believe, the best open
> source
> > pitch shift / time stretch library out there at the moment, and can run
> in
> > realtime on modern CPUs. SoundTouch 
> is
> > another good option that is cheaper to compute (and therefore easier to
> run
> > in realtime on e.g. a mobile CPU).
> >
>
> these both sound interesting (in their demos).  thanks for filling me in
> on these.  the SoundTouch at stretch to double mode (or cut tempo in half)
> has splicing sounds or "glitches", many per second, so i sorta suspect that
> one to be time domain, but maybe it's not.  i wonder what that might sound
> like for monophonic vocal or something that might expose the "phasiness"
> that i dunno how to beat using a phase vocoder.  a good time-domain pitch
> shifter (not multiband pitch shifting) naturally is able to keep phases
> between harmonics synchronized (because it doesn't separate the harmonics
> and deal with each separately).  but there is the potential for glitches
> when the input is not sufficiently periodic.
>
>
>
> --
>
> r b-j r...@audioimagination.com
>
> "Imagination is more important than knowledge."
>
>
>
>
>
>
>
>
> ___
> dupswapdrop: music-dsp mailing list
> music-dsp@music.columbia.edu
> https://lists.columbia.edu/mailman/listinfo/music-dsp
>
___
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp