Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-12-22 Thread Bernhard Übelacker
A small addition to the test case in Message #114: In test-768369.c lines 193 and 194 are swapped therefore an undefined value is given to malloc. When cleaning up this leads to a crash as now the stack smashing is fixed. Kind regards, Bernhard -- To UNSUBSCRIBE, email to

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-22 Thread roucaries bastien
control: tags -1 + confirmed control: severity - 1 serious justification: does not need imagemagick On Sat, Nov 22, 2014 at 7:43 AM, Bernhard Übelacker bernha...@vr-web.de wrote: Hello, I created a minimal test case in around 200 lines. It uses a file with the intercepted scanlines of the

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-22 Thread DRC
I can readily reproduce the failure with the supplied test case, but what I'm tripping on right now is understanding why a Huffman-encoded block can grow so much larger than the size of the source block (128 bytes.) While this test case is very unusual, there may be others out there, and I

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-22 Thread roucaries bastien
On Sat, Nov 22, 2014 at 6:58 PM, DRC dcomman...@users.sourceforge.net wrote: I can readily reproduce the failure with the supplied test case, but what I'm tripping on right now is understanding why a Huffman-encoded block can grow so much larger than the size of the source block (128 bytes.)

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-22 Thread DRC
Never mind. I think I answered my own question. Although I don't understand the Huffman algorithm well enough to know whether this is algorithmically possible, a naive analysis of the code shows that it calls PUT_BITS 128 times for each block, and the size argument in all of those cases can

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-22 Thread DRC
I find that by setting the AC coefficients to alternating values of 32767 and -32768 in the JPEG scanning order (1, 8, 16, 9, 2, 3, etc.), I can make the Huffman encoder exceed 200 bytes/block every single time. So that further confirms that 256 is the worst case. I've checked in an upstream

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-21 Thread DRC
Hi, guys. So what's still tripping me up on this is that I can't reproduce it using only libjpeg-turbo. There is really no action I can take until I am able to do that, because without the ability to reproduce the issue irrespective of ImageMagick, I have no way of knowing whether this is

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-15 Thread Bernhard Übelacker
Hello, probably the attached patch could help in diagnose the issue. It prints an error message and aborts, when the current buffer pointer is advanced past the _buffer. In debugger it shows this happens a little before what roucaries bastien in message 47 wrote. (Because he stopped at the stack

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-15 Thread Bernhard Übelacker
Hello, DRC suggested to have a look at the newer upstream version. In jchuff.c the buffer in question is there really grown. But only by 8 bytes. [1] When increasing by 28 bytes the stack smashing and writing beyond the buffer goes away. The resulting image looks good. (Input file from the

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-14 Thread DRC
Have you tried libjpeg-turbo 1.4 beta? I believe this has already been fixed. Another user reported a very low-probability event (literally 1 in a million) that would cause the 128-byte buffer to be exceeded when encoding a JPEG image using quality 100. If you have a specific image that

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-12 Thread roucaries bastien
We have investigated a little bit (thansk y dlemstra ) was able to reproduce the problem under Windows that uses libturbo 1.3.0. It looks like the problem is inside the STORE_BUFFER macro at line 421 of jchuff.c: Code: Select all buffer = _buffer; \ while (bytes 0) { \ bytestocopy =

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-07 Thread Ondřej Surý
Did anyone report that to the upstream? Since it affects multiple distributions that should have been the first thing to do... Cheers, Ondrej On Thu, Nov 6, 2014, at 22:22, roucaries bastien wrote: More information could be found here

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-07 Thread roucaries bastien
Le 7 nov. 2014 10:14, Ondřej Surý ond...@sury.org a écrit : Did anyone report that to the upstream? Since it affects multiple distributions that should have been the first thing to do... I Sent a mail to upstream contact adress. Bastien Cheers, Ondrej On Thu, Nov 6, 2014, at 22:22,

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-07 Thread DRC
Happy to fix it, but I need to be able to reproduce it first, using only libjpeg-turbo. Currently I cannot. I tried running jpegtran -optimize -rotate 270 003632r270.jpg out.jpg and jpegtran -progressive -optimize -rotate 270 003632r270.jpg out.jpg with valgrind, and no issues were

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-07 Thread roucaries bastien
On Fri, Nov 7, 2014 at 4:57 PM, DRC dcomman...@users.sourceforge.net wrote: Happy to fix it, but I need to be able to reproduce it first, using only libjpeg-turbo. Currently I cannot. I tried running Here a backtrace, do you want to get some argument of the call function ? #0

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-07 Thread DRC
I want exactly what I asked for: a way to reproduce this issue. Currently I cannot. A backtrace from your machine is not helpful, as it does not tell me anything regarding how the library is being used by ImageMagick. On 11/7/14 11:26 AM, roucaries bastien wrote: On Fri, Nov 7, 2014 at

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-07 Thread roucaries bastien
On Fri, Nov 7, 2014 at 6:26 PM, roucaries bastien roucaries.bastien+deb...@gmail.com wrote: On Fri, Nov 7, 2014 at 4:57 PM, DRC dcomman...@users.sourceforge.net wrote: Happy to fix it, but I need to be able to reproduce it first, using only libjpeg-turbo. Currently I cannot. I tried running

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-07 Thread roucaries bastien
On Fri, Nov 7, 2014 at 6:36 PM, DRC dcomman...@users.sourceforge.net wrote: I want exactly what I asked for: a way to reproduce this issue. Currently I cannot. A backtrace from your machine is not helpful, as it does not tell me anything regarding how the library is being used by ImageMagick.

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-07 Thread DRC
I don't know why you would expect me to have tried that, given that there was no reasonable expectation that I would know any of that information prior to now. But I did just try it, and it works fine. I will re-iterate: You need to produce a test case that demonstrates this failure using

Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

2014-11-06 Thread roucaries bastien
More information could be found here http://www.imagemagick.org/discourse-server/viewtopic.php?f=3t=26482sid=81658bc2f51a8d9893279cd01e83783f On Thu, Nov 6, 2014 at 10:21 PM, Debian Bug Tracking System ow...@bugs.debian.org wrote: Thank you for filing a new Bug report with Debian. This is an