[jira] [Commented] (IMAGING-373) OutOfMemory with invalid ICO input file
[ https://issues.apache.org/jira/browse/IMAGING-373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17822423#comment-17822423 ] Milan Nikl commented on IMAGING-373: I'm using the library to load selected types of images (GIF, ICO) inside a flink pipeline within a taskmanager with JVM Heap Size: 564 MB. That's why I was getting OOM. I tried using the imaging jar from master branch [commit 6519bad|https://github.com/apache/commons-imaging/commit/6519bad07f5ebcdfa0e10830c3e57064619a2cb7] and after 24h +/- in the testing environment, there hasn't been any unexpected exception so far. > OutOfMemory with invalid ICO input file > --- > > Key: IMAGING-373 > URL: https://issues.apache.org/jira/browse/IMAGING-373 > Project: Commons Imaging > Issue Type: Bug > Components: Format: BMP, Format: ICO >Affects Versions: 1.0-alpha3 >Reporter: Milan Nikl >Priority: Major > Fix For: 1.0-M1 > > Attachments: epine.ico > > > While trying to _use org.apache.commons.imaging.formats.ico.IcoImageParser_ > to read an icon file, I'm getting OutOfMemory Error. The file is corrupted, > but that should not lead to an Error. > Icon is downloaded from [https://epine.es/assets/icos/epine.ico] and > hopefully attached to this issue. > [^epine.ico] > > Trying to debug the problem, I found out that the cause is in misinterpreting > some values in > _org.apache.commons.imaging.formats.bmp.BmpImageParser#readBmpHeaderInfo(java.io.InputStream, > org.apache.commons.imaging.FormatCompliance)_ method while trying to load > data of the single image bitmap in given ico file. > While _fileSize_ value is 5990, the _bitmapDataOffset_ value parsed is > 989265922. At the same time _expectedDataOffset_ value is 70, which leads to > _extraBytes_ value of 989265852. > Then > _org.apache.commons.imaging.common.BinaryFunctions#readBytes(java.lang.String, > java.io.InputStream, int, java.lang.String)_ tries to allocate byte array of > size 989265852. Which exhausts available memory of the JVM. > Maybe just a simple check of _extraBytes > bhi.fileSize_ could help? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IMAGING-373) OutOfMemory with invalid ICO input file
[
https://issues.apache.org/jira/browse/IMAGING-373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821717#comment-17821717
]
Gary D. Gregory commented on IMAGING-373:
-
For me locally, without the change to {{main}}, I do not get an OOME, I get a
{{IOException}}:
{noformat}
org.opentest4j.AssertionFailedError: Unexpected exception type thrown,
expected: but was:
at
org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:67)
at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:35)
at org.junit.jupiter.api.Assertions.assertThrows(Assertions.java:3115)
at
org.apache.commons.imaging.formats.ico.IcoImageParserTest.testImageWithInvalidBmpHeaders(IcoImageParserTest.java:37)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at java.util.ArrayList.forEach(ArrayList.java:1259)
Caused by: java.io.IOException: Not a Valid BMP File, name: BitmapDataOffset,
length: 989265852
at
org.apache.commons.imaging.common.BinaryFunctions.readBytes(BinaryFunctions.java:255)
at
org.apache.commons.imaging.formats.bmp.BmpImageParser.readImageContents(BmpImageParser.java:577)
at
org.apache.commons.imaging.formats.bmp.BmpImageParser.getBufferedImage(BmpImageParser.java:123)
at
org.apache.commons.imaging.formats.ico.IcoImageParser.readBitmapIconData(IcoImageParser.java:450)
at
org.apache.commons.imaging.formats.ico.IcoImageParser.readIconData(IcoImageParser.java:527)
at
org.apache.commons.imaging.formats.ico.IcoImageParser.readImage(IcoImageParser.java:567)
at
org.apache.commons.imaging.formats.ico.IcoImageParser.getAllBufferedImages(IcoImageParser.java:258)
at
org.apache.commons.imaging.AbstractImageParser.getAllBufferedImages(AbstractImageParser.java:282)
at
org.apache.commons.imaging.formats.ico.IcoImageParserTest.lambda$0(IcoImageParserTest.java:37)
at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:53)
... 6 more
{noformat}
Please check your use case with git master or a snapshot build from
https://repository.apache.org/content/repositories/snapshots/org/apache/commons/commons-imaging/1.0-M1-SNAPSHOT/
We do different internal checking than in alpha3 now, but the suggested fix
still looks valid. I'll see if we can fail faster with the suggestion.
> OutOfMemory with invalid ICO input file
> ---
>
> Key: IMAGING-373
> URL: https://issues.apache.org/jira/browse/IMAGING-373
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: BMP, Format: ICO
>Affects Versions: 1.0-alpha3
>Reporter: Milan Nikl
>Priority: Major
> Attachments: epine.ico
>
>
> While trying to _use org.apache.commons.imaging.formats.ico.IcoImageParser_
> to read an icon file, I'm getting OutOfMemory Error. The file is corrupted,
> but that should not lead to an Error.
> Icon is downloaded from [https://epine.es/assets/icos/epine.ico] and
> hopefully attached to this issue.
> [^epine.ico]
>
> Trying to debug the problem, I found out that the cause is in misinterpreting
> some values in
> _org.apache.commons.imaging.formats.bmp.BmpImageParser#readBmpHeaderInfo(java.io.InputStream,
> org.apache.commons.imaging.FormatCompliance)_ method while trying to load
> data of the single image bitmap in given ico file.
> While _fileSize_ value is 5990, the _bitmapDataOffset_ value parsed is
> 989265922. At the same time _expectedDataOffset_ value is 70, which leads to
> _extraBytes_ value of 989265852.
> Then
> _org.apache.commons.imaging.common.BinaryFunctions#readBytes(java.lang.String,
> java.io.InputStream, int, java.lang.String)_ tries to allocate byte array of
> size 989265852. Which exhausts available memory of the JVM.
> Maybe just a simple check of _extraBytes > bhi.fileSize_ could help?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IMAGING-373) OutOfMemory with invalid ICO input file
[
https://issues.apache.org/jira/browse/IMAGING-373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821583#comment-17821583
]
Milan Nikl commented on IMAGING-373:
Using [https://github.com/drewnoakes/metadata-extractor] to check given file's
metadata, I can see:
* [ICO] Image Size Bytes - 5960
* [ICO] Image Offset Bytes - 22
Using org.apache.commons.imaging.formats.ico.IcoImageParser#readIconInfo I can
see the same:
* imageSize = 5960
* imageOffset = 22
So it looks like the problem is in transition to bitmap in
_org.apache.commons.imaging.formats.ico.IcoImageParser#readBitmapIconData_
where:
* _colorsUsed = 247316463_ which leads to {_}bitmapPixelsOffset =
989265922{_}, while _bitmapSize = 5990_ and _restOfFile.length = 5920_
> OutOfMemory with invalid ICO input file
> ---
>
> Key: IMAGING-373
> URL: https://issues.apache.org/jira/browse/IMAGING-373
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: BMP, Format: ICO
>Affects Versions: 1.0-alpha3
>Reporter: Milan Nikl
>Priority: Major
> Attachments: epine.ico
>
>
> While trying to _use org.apache.commons.imaging.formats.ico.IcoImageParser_
> to read an icon file, I'm getting OutOfMemory Error. The file is corrupted,
> but that should not lead to an Error.
> Icon is downloaded from [https://epine.es/assets/icos/epine.ico] and
> hopefully attached to this issue.
> [^epine.ico]
>
> Trying to debug the problem, I found out that the cause is in misinterpreting
> some values in
> _org.apache.commons.imaging.formats.bmp.BmpImageParser#readBmpHeaderInfo(java.io.InputStream,
> org.apache.commons.imaging.FormatCompliance)_ method while trying to load
> data of the single image bitmap in given ico file.
> While _fileSize_ value is 5990, the _bitmapDataOffset_ value parsed is
> 989265922. At the same time _expectedDataOffset_ value is 70, which leads to
> _extraBytes_ value of 989265852.
> Then
> _org.apache.commons.imaging.common.BinaryFunctions#readBytes(java.lang.String,
> java.io.InputStream, int, java.lang.String)_ tries to allocate byte array of
> size 989265852. Which exhausts available memory of the JVM.
> Maybe just a simple check of _extraBytes > bhi.fileSize_ could help?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
