AD2136x doesn't seem to have FP division instructions (?) therefore I guess one has to engage some homemade routine to do that. Some chips have a built-in lookup table of low-precision reciprocals to help cooking faster division routines. A usual digital arithmetic textbook will have the algorithms. Indeed one has to debug into the division routine to find out what's going wrong.
Xue -----Original Message----- From: music-dsp-boun...@music.columbia.edu [mailto:music-dsp-boun...@music.columbia.edu] On Behalf Of Alan Wolfe Sent: 27 April 2011 06:50 To: A discussion list for music-related DSP Subject: Re: [music-dsp] Floating Point Division i don't know that chip but have you thought about re-arranging your math? with limited precision, order of operations can matter. This is a bigger problem with fixed point (and integers) than floating point but it can still be a problem even in floating point. whats the specific error message? On Tue, Apr 26, 2011 at 8:51 PM, <resea...@ottomaneng.com> wrote: > Hello everyone, > > I have been going back to working on a DSP processor and recently hit a precision problem with division on an ADI SHARC 21369. I get something to the effect of overflow or fold over. Anyone have any suggestions or references if I want e-7 precision division? > > > Thanks, > > Omer Osman > -- > dupswapdrop -- the music-dsp mailing list and website: > subscription info, FAQ, source code archive, list archive, book reviews, dsp links > http://music.columbia.edu/cmc/music-dsp > http://music.columbia.edu/mailman/listinfo/music-dsp > -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
