On Thu, 2016-06-09 at 17:01 -0400, Ronald S. Bultje wrote: > Hi, > > On Thu, Jun 9, 2016 at 4:02 PM, Dan Parrot <dan.par...@mail.com> wrote: > > > Line 72 of libswscale/input.c is: > > dstU[i] = (ru*r + gu*g + bu*b + (0x10001<<(RGB2YUV_SHIFT-1))) >> > > RGB2YUV_SHIFT; > > > > The definition of macro RGB2YUV_SHIFT in libswscale/swscale_internal.h > > is on line 417: > > #define RGB2YUV_SHIFT 15 > > > > By examining the result of executing line 72 in input.c it appears that > > the radix used for macro RGB2YUV_SHIFT is hexadecimal. So that 0x10001 > > is shifted left by 20 in subexpression 0x10001<<(RGB2YUV_SHIFT-1) with > > the result being 0x100000. > > > > I was expecting a left-shift of 14, given the macro definition. > > > > Should the macro be interpreted as decimal 15 or hexadecimal 15? > > > It's decimal 15, it should shift by 14. Check disassembly, though... It's > almost impossible that your compiler would read this as hex and still be > able to produce runnable applications. Maybe submit a bug report to your > compiler's bug tracker? > > Ronald > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
There is no issue with the software. I have been examining the code in gdb and what confused me was having changed the default gdb radix with "set radix 16". Thanks Ronald. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
