Hi all, This is a very interesting discussion, thanks.
Since Dario picks up on Andy's comment about 1-dimensional data and neural nets, I must comment: Andy's assertion is a bit out-of-date, and there are lots of NN methods that are designed/applied for one-dimensional scalar signals. A famous one for audio is deepmind's "WaveNet"; it's just one example of the more general idea of "time-domain CNN" or one-dimensional CNN. You can also apply recurrent NNs (RNNs) or even the very fashionable Transformer networks, all of these are reasonable things to apply to 1D signals such as audio. I'm no expert on sigma-delta at all, but my hunch about Dario's idea is that it could well be a great idea. Xue comments that sigma-delta systems are tuned to minimise SNR across the whole audio spectrum. One thing that might be possible with a NN method might be to move beyond that, e.g. optimising for (a proxy of) perceptual quality rather than for raw SNR. Also, Brian's comment about dithering strongly implies to me that, despite other commenters treating SDM as a strictly deductive system, there may be a benefit of some "inference" that a NN can provide, to produce the best approximation of an output given uncertain (e.g. dithered) input data. Best wishes to all, Dan Op za 9 jan. 2021 om 17:04 schreef Dario Sanfilippo < [email protected]>: > Dear all, > > I would like to thank you for your precious comments; I am really grateful > for the time you have taken to write your answers. > > About Brian's comments on summation and multiplication: in the case of > delta-sigma multiplication, for example using the 1-bit multiplier > described in [Ng 2009] (http://dx.doi.org/10.5353/th_b4163384, page 12), > my assumption (perhaps entirely wrong) was that multiplying two signals in > the multi-bit domain and then converting them into delta-sigma streams > would always result in a better SNR than multiplying two delta-sigma > streams through 1-bit multipliers. My hope was that I could have used an NN > as an efficient 1-bit multiplier using the delta-sigma-encoded multi-bit > multiplication as training data. > > About Andy's comments on the mono-dimensionality of DSM streams: perhaps a > single audio stream feeds multi-input NNs using the FFT outputs or the > outputs of a filter bank; in the case of a DSM stream, could it make sense > to feed an M-input NN with a sliding chunk of the 1-bit stream, hence > having x[n] to x[n-(M-1)] from the top to the bottom input of the NN? > > On Fri, 8 Jan 2021 at 08:00, Xue Wen <[email protected]> wrote: > >> As for extracting features from 1-bit format, given the fact that >> multi-bit and one-bit versions are only a filter away from each other, >> there seems no pressing reason to favour one over the other, unless the >> cost of the filter itself (which upper bounds their difference) is >> considered significant. >> > > I guess that my ultimate goal would have been to be able to process DSM > streams or extract information from them without moving from the 1-bit > domain to the other and vice versa. If we think of brightness information, > provided that an NN ever really worked for that, it would be about > comparing the requirements of an FFT for the spectral centroid, for > example, to that of the NN to achieve the same accuracy in the measurements. > > Given that DSM requires considerable oversampling to actually work, I'm > wondering whether the number of inputs required to process a long-enough > slice of audio would make even binarised NN unreasonable for the > computational requirements. > > Thank you so much for your help, > Dario > > >> >> >> NNs may be useful, for example, to help decide which of several >> delta-sigma configurations best suites some given data. >> >> >> >> --------- *Original Message* --------- >> >> *Sender* : Andy Farnell <[email protected]> >> >> *Date* : 2021-01-07 19:18 (GMT+8) >> >> *Title* : Re: Neural nets for delta-sigma DSP >> >> >> >> I think Brian has already done a fantastic job of steering you away from >> some rocks Dario. When it comes to audio applications, and particularly >> music, the analysis and prediction aspects of NN are where you might look. >> >> As a general CS issue, NN are best for many inputs and few outputs >> where information simultaneity/parallelism can be presented in the >> first layer. >> >> My knowledge and use of NN is old and rusty, but a >> a couple of examples from memory; >> >> Melodies can be compactly captured by a series of "tri-state" (or >> explicitly timed binary) transitions. Either the notes goes up, down >> or stays the same. Given sequences or fragments you can get NN to make >> a fair job of identifying, extrapolating, interpolating and regressing >> them to do useful musical tasks like improvising or assisting composition. >> >> Given a short attack portion of an audio signal you can use NN to do >> fast prediction of likely spectrum or period, so super fast pitch >> estimates of bass notes in less that 1/4 cycle. >> >> Delta-sigma is really a "one-dimensional" stream and on the face of >> it not amenable to traditional NN. However, if you shift to >> techniques that map sequence to state then things get more interesting. >> LPC, Kalman filters and suchlike are fun - lots of applications here, >> automatic SDR radio tuning, voice correction, and more to my >> present interests cryptanalysis looking for information leaking >> from what ought to be random (secure) signals. >> >> On Wed, Jan 06, 2021 at 08:21:11PM -0800, Brian Willoughby wrote: >> > Hello Dario, >> > >> > Some of your goals seem completely inappropriate for solving with neural >> > networks. >> > Perhaps it would help to break down each one, and then you can explain >> > your thought >> > process in each case. >> > >> > 1) summation - the sum of two numbers, or the sum of two streams of >> > numbers, has >> > exactly one defined result. There is no noise associated with summation. >> > Neural >> > networks are generally used when the results for a specific data set are >> > not known, >> > but where training against a large number of similar data sets for which >> > the results >> > are known can lead towards a fuzzy approximation of the correct result for >> > the >> > unknown data set in question. In the case of a basic arithmetic sum, only >> > one result >> > is acceptable, and a fuzzy approximation will significantly increase >> > noise, i.e., >> > decrease SNR. >> > >> > 2) multiplication - the product of two numbers, or the product of a number >> > and a >> > stream of numbers, has exactly one defined result. There is no noise >> > associated with >> > multiplication, provided that you're willing to increase the bit depth in >> > order to >> > hold all significant digits. Again, neural networks would appear to be >> > completely >> > inappropriate for this task. If you need to reduce the bit depth of the >> > results, then >> > there's a small chance that the results might benefit from the application >> > of NN >> > processing, but in this case you would actually be using NN for dither, >> > not for >> > multiplication. >> > >> > 3) extracting noisiness - do you want to isolate the noise from the stream >> > and keep >> > the noise? ... or do you want to remove noise? >> > >> > 4) extracting brightness - again, when you say that you want to extract >> > brightness, >> > does that mean you want to isolate the bright sounds and retain them? ... >> > or is the >> > goal to reduce brightness in the signal? This example seems like one where >> > a solution >> > based on a standard DSP filter would be appropriate, without NN >> > calculations. >> > >> > If you have any examples where basic arithmetic is performed via neural >> > networks - >> > particularly where it is explained how this expensive approach is superior >> > to >> > directly performing the basic math - then I would be very interested in >> > references. >> > >> > Similarly, when it comes to extracting noise or brightness, any examples >> > where neural >> > networks have achieved similar results would be helpful. In other words, >> > what >> > successes have you observed that lead you to believe that neural networks >> > would be >> > appropriate for these tasks? >> > >> > In general, neural networks are useful for approximation, analysis, >> > prediction, >> > classification, and processing that is lossy. It would seem that basic >> > arithmetic >> > does not fall into any of these application classes. In addition to A. K. >> > Dewdney's >> > comments on neural networks, I would add that neural networks are >> > incredibly >> > inefficient for processing audio, especially in terms of energy >> > expenditure. >> > >> > Brian Willoughby >> > >> > >> > On Jan 6, 2021, at 14:45, Dario Sanfilippo wrote: >> > > I am new to neural networks and I would like to start some investigation >> > > for a >> > > 1-bit audio DSP project. >> > > >> > > The idea is to use NN to perform basic arithmetic on delta-sigma streams >> > > such as >> > > summation and multiplication and, hopefully, achieve better SNR than the >> > > standard >> > > techniques currently available. And perhaps, some of these networks >> > > could also be >> > > trained to extract low-level information from delta-sigma streams such >> > > as noisiness >> > > or brightness. >> > > >> > > One thing that should be convenient is that the specific training data >> > > can be >> > > generated in the multi-bit domain and then converted with a delta-sigma >> > > modulator >> > > to train the network. >> > > >> > > I am not sure whether the nature of the streams makes binarised neural >> > > networks >> > > ideal, but I will sure look into them for implementation on FPGAs. >> > > >> > > For now, I would kindly ask for your help to understand which strategy >> > > might be most successful: I have seen several types of neural networks >> > > for audio and I am guessing that investigating the right one could be >> > > crucial. >> > > >> > > If any of you has worked in this field and has some useful information >> > > or hints on which NN type could suit this project, that would be greatly >> > > appreciated. >> >> >> >> >> > > -- > Dr Dario Sanfilippo > http://dariosanfilippo.com > -- http://www.mcld.co.uk
