Hi Ted, thanks for your answer. I don't yet have a clear idea of the scope of this project, so I was just asking if DSP filters are something in the scope of the project. I have an already developed library of DSP filters, so if you think they are useful or will be useful or will fall into the scope of the project, I can donate them.
I have used DSP filters for statistical analysis of long time series (like logs) a couple of years ago, but not for preparation of data for ingestion into a ML algorithm, simply to isolate and notify to the user when something strange happened, like an uncommon frequency of traffic from a certain ip. So, as you correctly pointed out, I have not yet felt the pain of combining DSP with other ML algorithms. If you feel like it could give some benefit, code is there ready to be donated. Simone Ted Dunning wrote: > To my mind, the key questions in general here are whether the problem needs > parallel processing at scale and whether map/reduce can be profitably > applied to the problem. > > FIR and IIR input filters for time-series are definitely important in some > domains and for extremely large time series or large filters, these may > actually benefit from map-reduce implementation. > > So the next question is whether you would have benefited by having these > operations operate in parallel. I definitely can say that all of the time > series problems I have had to deal with had very moderate scale and thus > didn't really need any subsequent processing. You sound like you have had a > different experience. If so, then please do implement these since we would > all learn from your experience. > > On the other hand, implementing something just because somebody might > someday find it useful would not be very helpful. The biggest reason for > this is that if you don't feel the pain already, you probably can't guess > what pains somebody else will feel when they need this capability. > > On 2/17/08 2:03 PM, "Simone Gianni" <[EMAIL PROTECTED]> wrote: > > >> Hi all, >> are DSP filters (FIR,IIR etc..) something useful for this project? >> Digital Signal Processing is used in some fields of Machine Learning, >> mainly to pre-process or to filter data before ingestion in Machine >> Learning algorithms. >> >> Simone >> > >
