On Mon, Oct 08, 2018 at 05:04:43PM +0200, Martin Vignali wrote: > > > > also there are 2 divisions in this that you can trivially eliminate > > /255 and /65535 (extra precission beyond IEEE float/double could change > > these) > > > > also the whole could be done with fewer floats and no extra complexity > > for example: > > int64_t tmp2 = 16843009LL * i; > > (float)((double)tmp2 / (1LL<<32)) > > and > > int64_t tmp2 = 4295032833LL * i or uint64_t 281479271743489 > > > > > Still raise the assert for me. > Maybe we can go back to the initial patch i propose. > Let the simple initial code, for non bitexact conversion > and use a dedicated calc for bit exact version, without float calc
does it fail on actual platforms to produce the same result ? [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB While the State exists there can be no freedom; when there is freedom there will be no State. -- Vladimir Lenin
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
