Quoting Jaroslav Kysela <[EMAIL PROTECTED]>: > I don't think so. It seems that my brain still remembers assembler ;-) ... > sample = *sum; > s16 s; > - if (unlikely(sample & 0xffff0000)) > + if (unlikely(sample & 0x7fff0000)) > s = sample > 0 ? 0x7fff : -0x8000; > else > s = sample;
I think I remember some of the x86 assembly myself and this correction does not fix the problem. This code will still "saturate" all negative samples to -8000. You cannot detect an overflow into the upper half of the register with a simple bitwise and. The actual test should be as follows : - extend the sign of the lower half - check if the upper half is the same as the effect of expansion if it is - there is no overflow if it differs - there was overflow and you need to saturate. examples : value 0x 0000 0335 ext 0x 0000 0335 -> no overflow value 0x 0002 43b1 ext 0x 0000 43b1 -> overflow value 0x ffff f25b ext 0x ffff f25b -> no overflow value 0x ff1c 35c9 ext 0x 0000 35c9 -> overflow to put it in asm: mov ebx,eax cwde cmp eax,ebx The problem is cwde operates only on ax/eax. This may sound complicated but in fact it amounts to a very simple question : does the sample fit in a 16 bit int, or does it not, so I guess in C it could look something like : s16 s=sample; if (unlikely(sample != (s32)s)) The cast is just there for clarity I believe it would be done implicitly anyway. But don't take my word for it - I did not test this. -------------- Fycio (J.Sobierski) [EMAIL PROTECTED] ------------------------------------------------------- This SF.net email is sponsored by: SlickEdit Inc. Develop an edge. The most comprehensive and flexible code editor you can use. Code faster. C/C++, C#, Java, HTML, XML, many more. FREE 30-Day Trial. www.slickedit.com/sourceforge _______________________________________________ Alsa-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/alsa-devel