Re: [music-dsp] DC blocking (again :)
On 2012-08-01, robert bristow-johnson wrote: well, it works pretty fine. theoretically, of course, DC is constant. but really we think of DC (or the coefficient applied to any other frequency component) to be slowly-varying. Or in other words, something like the stuff that happens below 1Hz. Nobody hears that, so that it can be safely filtered away, using even the kinds of nasty first order IIR filters which would wreak havoc if used higher up the band. In DC removal we can just say that it removed DC, and sleep well at night. if DC is slowly varying, small displacements of a windowed section of DC (which is what comes out of any weighted moving-average filter) does not change it much. More precisely, quite a number of waveforms like these can be put into closed form, or if not that, at least into the form of a combination of piecewise linear functions, impulses, certain second degree polynomials, gaussians, plus sparse convolutions between them. In the end that kind of an analysis will yield a second degree error term in the error term, which will be negligible in usual audio practice. At the same time it will yield an in-band frequency modulation signal, which will also be negligible, also by empirical A/B/X measurement from something like the 60es. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-50-5756111, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] DC blocking (again :)
On 2012-08-01, Domagoj Saric wrote: Then there is the modification (http://www.dsprelated.com/showmessage/80739/2.php, Andor's post) to subtract the moving average from the _current_ sample (instead of the one corresponding to the middle of the moving average filter) but this supposedly makes the filter minimal-phase instead of linear-phase so it is still a no go (at least AFAICT from my limited knowledge). Certain high theoreticians have surmised that the minimum phase version could be better. So, at least at high sample rates (88kHz+) you could go with that design as well, and at least in theory do it even better than with the linear phase, more complex design. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-50-5756111, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] DC blocking (again :)
On 1.8.2012. 18:13, robert bristow-johnson wrote: Since we seem to be starting to run in circles let me try to restate my main question as simple as possible: does using an IIR DC filter defeat the purpose of using a (linear phase) FIR (anti-aliasing) LPF in the same signal chain? If not, why? If yes, how can it be considering that this is exactly what ITU-R and EBU recommend for true peak measurement? i dunno about ITU-R or EBU (i s'pose i could click on those links you mention), but the old analog meters had meter ballistics that were analog. IIR filters most closely follow analog filters than do FIR, unless you make the FIR very long. The new loudness standards are all about declaring those old analog meters exactly that, old, useless and obsolete :) You might want to skim through the papers because pretty much all public broadcasting will have to behave according to those regulations in most parts of the world... ;) note that when R is very close to 1, the DC block filter is almost zero-phase for frequencies high enough. the DC blocker does nothing to frequencies high enough because it's just an HPF. So, as I suspected in the first post, the answer is in JOS (i.e. it does not touch content not-very-close-to-DC)...thanks ;) This however is not the end of the (to me) unclear parts of the standard proposal: a) the standard also recommends an optional high frequency pre-emphasis shelving IIR filter (zero at 14.1 kHz, pole at 20 kHz) for the true peak measurements - the question you are guessing is the same as for the DC filter: doesn't that IIR filter defeat the purpose of using a linear phase FIR filter (and/or more importantly, doesn't the non linear phase response of an IIR filter possibly distort the very peaks we are trying to meassure) b) as you can see on page 18 of http://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1770-2-201103-I!!PDF-E.pdf they placed both the DC and pre-emphasis filters _after_ the upsampler...isn't that just plain inefficient? -- Domagoj Saric Software Architect www.LittleEndian.com -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] DC blocking (again :)
On 31.7.2012. 12:54, Wen Xue wrote: 5ms moving-average doesn't sound very right for it cuts off anything below 200Hz, no matter how much one upsamples it. However it is probably just fine to subtract a DC measured 500ms ago from the current waveform because the DC shouldn't change much in that time or it can't be DC. This subtraction you can write as a anti-DC filter which is FIR with a big delay on the unwanted part but no delay on the wanted part. There might be some DC left - but probably significantly suppressed - depending on how DC-ish it is. Is this the same/similar to what I mentioned in the first post: Then there is the modification (http://www.dsprelated.com/showmessage/80739/2.php, Andor's post) to subtract the moving average from the _current_ sample (instead of the one corresponding to the middle of the moving average filter) but this supposedly makes the filter minimal-phase instead of linear-phase so it is still a no go (at least AFAICT from my limited knowledge). ? -- Domagoj Saric Software Architect www.LittleEndian.com -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] DC blocking (again :)
On 1.8.2012. 6:29, robert bristow-johnson wrote: if DC is slowly varying, small displacements of a windowed section of DC (which is what comes out of any weighted moving-average filter) does not change it much. the difference between the IIR vs FIR, minimum phase vs. linear phase, is just the shape of that window function (it's the same as the impulse response) I must admit you kind of lost me with that last sentence... :) Unless the answer is in the JOS link (the near zero-phase)..? you mean the zero-phase filters using truncated IIR? for DC blocking (or for LPF metering) or whatever, using that Powell/Chau/Smith/Wang kind of linear-phase TIIR filtering is using a cannon to kill a fly, no? No, I mean https://ccrma.stanford.edu/~jos/filters/DC_Blocker_Frequency_Response.html which shows that when R=0 you get a linear phase FIR but when R approaches 1 (which is the case in DC filters) the phase response approaches zero for frequencies not near zero/DC. for a moving average or for DC blocking, just use an IIR. simple and plenty good enough. if it's done in fixed-point arithmetic, then using noise shaping will help the filter behave itself. Since we seem to be starting to run in circles let me try to restate my main question as simple as possible: does using an IIR DC filter defeat the purpose of using a (linear phase) FIR (anti-aliasing) LPF in the same signal chain? If not, why? If yes, how can it be considering that this is exactly what ITU-R and EBU recommend for true peak measurement? -- Domagoj Saric Software Architect www.LittleEndian.com -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] DC blocking (again :)
On 8/1/12 5:25 AM, Domagoj Saric wrote: On 1.8.2012. 6:29, robert bristow-johnson wrote: if DC is slowly varying, small displacements of a windowed section of DC (which is what comes out of any weighted moving-average filter) does not change it much. the difference between the IIR vs FIR, minimum phase vs. linear phase, is just the shape of that window function (it's the same as the impulse response) I must admit you kind of lost me with that last sentence... :) i'm saying that these are all filters. the output of whatever DC blocker, however you implement it, is a weighted sum of the current and previous samples. if you were to plot the weighting coefficients vs. displacement from the current sample, you would see how your signal is windowed and the only difference between these is the shape of the window. this window is actually the same as the impulse response flipped around backward. for a moving average, the coefficients must all add to 1. there are many shapes that you can have that have coefficients that add to 1. for a DC blocker, the coefficients must all add to 0. again, many ways to do that. if the shape of this window is perfectly symmetrical, you have linear phase. that filter has a constant time delay, no matter what the frequency content of the audio going in is. if the filter is minimum phase, the shape of that window will emphasize the more current samples than the much older samples. Unless the answer is in the JOS link (the near zero-phase)..? you mean the zero-phase filters using truncated IIR? for DC blocking (or for LPF metering) or whatever, using that Powell/Chau/Smith/Wang kind of linear-phase TIIR filtering is using a cannon to kill a fly, no? No, I mean https://ccrma.stanford.edu/~jos/filters/DC_Blocker_Frequency_Response.html which shows that when R=0 you get a linear phase FIR but when R approaches 1 (which is the case in DC filters) the phase response approaches zero for frequencies not near zero/DC. that FIR case (when R=0) is a digital differentiator. you might like the linear phase (a constant delay of 1/2 sample), but you won't like the amplitude component of the frequency response. that JOS DC block filter is the simplest IIR filter. for a moving average or for DC blocking, just use an IIR. simple and plenty good enough. if it's done in fixed-point arithmetic, then using noise shaping will help the filter behave itself. Since we seem to be starting to run in circles let me try to restate my main question as simple as possible: does using an IIR DC filter defeat the purpose of using a (linear phase) FIR (anti-aliasing) LPF in the same signal chain? If not, why? If yes, how can it be considering that this is exactly what ITU-R and EBU recommend for true peak measurement? i dunno about ITU-R or EBU (i s'pose i could click on those links you mention), but the old analog meters had meter ballistics that were analog. IIR filters most closely follow analog filters than do FIR, unless you make the FIR very long. note that when R is very close to 1, the DC block filter is almost zero-phase for frequencies high enough. the DC blocker does nothing to frequencies high enough because it's just an HPF. -- r b-j r...@audioimagination.com Imagination is more important than knowledge. -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] DC blocking (again :)
Ah sorry I missed that. Yes, pretty much the same. And this has to be minimal-phase although there's no attempt to make it so. And no, this is no linear phase BUT if the moving average filter has low sidelobes then it is ALMOST linear phase in audible range, because there's hardly any subtraction there. This applies to FIR as well as IIR filters as long as the DC-blocking is constructed by subtraction. If the moving average is FIR then the whole DC-blocking is FIR; if that is IIR then so is this. Regarding linear-phaseness their outcomes are the same in audio range, as long as the moving average output is kept below. Xue -Original Message- From: Domagoj Saric Sent: Wednesday, August 01, 2012 9:39 AM To: music-dsp@music.columbia.edu Subject: Re: [music-dsp] DC blocking (again :) On 31.7.2012. 12:54, Wen Xue wrote: 5ms moving-average doesn't sound very right for it cuts off anything below 200Hz, no matter how much one upsamples it. However it is probably just fine to subtract a DC measured 500ms ago from the current waveform because the DC shouldn't change much in that time or it can't be DC. This subtraction you can write as a anti-DC filter which is FIR with a big delay on the unwanted part but no delay on the wanted part. There might be some DC left - but probably significantly suppressed - depending on how DC-ish it is. Is this the same/similar to what I mentioned in the first post: Then there is the modification (http://www.dsprelated.com/showmessage/80739/2.php, Andor's post) to subtract the moving average from the _current_ sample (instead of the one corresponding to the middle of the moving average filter) but this supposedly makes the filter minimal-phase instead of linear-phase so it is still a no go (at least AFAICT from my limited knowledge). ? -- Domagoj Saric Software Architect www.LittleEndian.com -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] DC blocking (again :)
On 30.7.2012. 20:51, robert bristow-johnson wrote: i didn't have anything to do with the subtract-the-moving-average DC block filter. I apologize...at least I attributed too much rather than too little ;) if you can put up with delay (which is what you must for a causal and linear-phase filter), you can subtract the moving-average from the sample in the middle of the buffer, not the most current sample. Well, for a live meter delay is obviously very undesirable. Taking into account the laziness of human senses, I guess up to ~5ms might be tolerable, that's about 220 samples@44.1kHz. If the DC filter is placed after the upsampler (as they seem to imply in the standard) and we upsample by a factor of 8 that becomes ~1760 samples...would that be enough for real-world DC offset tracking? But, more importantly, this might not be needed at all because, as I pointed out in my first mail, they (the ITU-R and EBU standard developers) obviously think/imply that using a plain IIR DC blocking filter is just fine (and one would certainly expect the standard not to err in such fundamentals, especially considering the amount of people that worked on it). The question, again, is how (can it be just fine)? Unless the answer is in the JOS link (the near zero-phase)..? -- Domagoj Saric Software Architect www.LittleEndian.com -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] DC blocking (again :)
On 30.7.2012. 22:18, Theo Verelst wrote: So: a relatively low sample rate signal (like 44.1 or 48 kS/s) can contain easily high frequency components (and possibly transients) which during reconstruction in the DA converter become larger than the highest sample value. Yes, this is clear from the explanations in the standard itself. It does not however touch the subject of my question... -- Domagoj Saric Software Architect www.LittleEndian.com -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] DC blocking (again :)
On 7/31/12 4:45 AM, Domagoj Saric wrote: On 30.7.2012. 20:51, robert bristow-johnson wrote: i didn't have anything to do with the subtract-the-moving-average DC block filter. I apologize...at least I attributed too much rather than too little ;) no sweatsky. i generally try to actively avoid credit for whence it is not due me. i'm a bit sloppier about credit due (or not due) to others. i just remember that article for IEEE Signal Processing Magazine that Rick and Randy did, and the IIR with noise shaping came from an idea of mine. the other half of that article came from something Rick Lyons had contributed. if you can put up with delay (which is what you must for a causal and linear-phase filter), you can subtract the moving-average from the sample in the middle of the buffer, not the most current sample. Well, for a live meter delay is obviously very undesirable. Taking into account the laziness of human senses, I guess up to ~5ms might be tolerable, that's about 220 samples@44.1kHz. If the DC filter is placed after the upsampler (as they seem to imply in the standard) and we upsample by a factor of 8 that becomes ~1760 samples...would that be enough for real-world DC offset tracking? But, more importantly, this might not be needed at all because, as I pointed out in my first mail, they (the ITU-R and EBU standard developers) obviously think/imply that using a plain IIR DC blocking filter is just fine (and one would certainly expect the standard not to err in such fundamentals, especially considering the amount of people that worked on it). The question, again, is how (can it be just fine)? well, it works pretty fine. theoretically, of course, DC is constant. but really we think of DC (or the coefficient applied to any other frequency component) to be slowly-varying. if DC is slowly varying, small displacements of a windowed section of DC (which is what comes out of any weighted moving-average filter) does not change it much. the difference between the IIR vs FIR, minimum phase vs. linear phase, is just the shape of that window function (it's the same as the impulse response) this is for metering? Unless the answer is in the JOS link (the near zero-phase)..? you mean the zero-phase filters using truncated IIR? for DC blocking (or for LPF metering) or whatever, using that Powell/Chau/Smith/Wang kind of linear-phase TIIR filtering is using a cannon to kill a fly, no? for a moving average or for DC blocking, just use an IIR. simple and plenty good enough. if it's done in fixed-point arithmetic, then using noise shaping will help the filter behave itself. -- r b-j r...@audioimagination.com Imagination is more important than knowledge. -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] DC blocking (again :)
Think about sampling theory and a *lot* of tricks which influence these measurements, I'm sure I've seen some guys in the field present stuff with which it would be possible to mess in more ways than people count on. So: a relatively low sample rate signal (like 44.1 or 48 kS/s) can contain easily high frequency components (and possibly transients) which during reconstruction in the DA converter become larger than the highest sample value. When talking about perceived loudness, the Equal Loudness Curve should be part of the equation, and when on top of that it is safe sound levels are the target, oh boy, that is complicated. Ir. Theo Verelst http://www.theover.org/Prod -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
