[
https://issues.apache.org/jira/browse/COMPRESS-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15814044#comment-15814044
]
Stefan Bodewig commented on COMPRESS-358:
-----------------------------------------
We use an internal lock size of 32k and this one is only configurable for the
raw stream, It seems the IWA files use offsets larger than that.
https://github.com/evernote/iwana/blob/master/iwana-extract/src/main/java/com/evernote/iwana/SnappyNoCRCFramedInputStream.java#L33
uses 128k which is way bigger than the size the "spec" talks about
{quote}
Note that the data portion of the chunk, i.e., the compressed contents,
can be at most 16777211 bytes (2^24 - 1, minus the checksum).
However, we place an additional restriction that the uncompressed data
in a chunk must be no longer than 65536 bytes. This allows consumers to
easily use small fixed-size buffers.
{quote}
Yet another deviation, I guess.
I'll make the block size configurable for the framed streams as well, probably
won't manage to do so today, though.
> Offset is larger than block size in IWA dialect of
> FramedSnappyCompressorInputStream
> ------------------------------------------------------------------------------------
>
> Key: COMPRESS-358
> URL: https://issues.apache.org/jira/browse/COMPRESS-358
> Project: Commons Compress
> Issue Type: Bug
> Components: Compressors
> Affects Versions: 1.12
> Reporter: Tim Allison
> Priority: Trivial
> Attachments: DocumentStyleSheet_commons_unsnapped.iwa,
> DocumentStyleSheet_fromZipStream.iwa, DocumentStylesheet.iwa,
> DocumentStylesheet_iwana_unsnapped.iwa, DocumentStylesheet_uncompressed.iwa,
> testNumbersNew.numbers
>
>
> I finally was able to run the FramedSnappyCompressorInputStreamon a larger
> number of .iwa files. I got the following on two files:
> {noformat}
> java.io.IOException: Offset is larger than block size
> at
> org.apache.commons.compress.compressors.snappy.SnappyCompressorInputStream.expandCopy(SnappyCompressorInputStream.java:341)
> at
> org.apache.commons.compress.compressors.snappy.SnappyCompressorInputStream.fill(SnappyCompressorInputStream.java:212)
> at
> org.apache.commons.compress.compressors.snappy.SnappyCompressorInputStream.read(SnappyCompressorInputStream.java:134)
> at
> org.apache.commons.compress.compressors.snappy.FramedSnappyCompressorInputStream.readOnce(FramedSnappyCompressorInputStream.java:166)
> at
> org.apache.commons.compress.compressors.snappy.FramedSnappyCompressorInputStream.read(FramedSnappyCompressorInputStream.java:122)
> at java.io.InputStream.read(InputStream.java:101)
> at java.nio.file.Files.copy(Files.java:2908)
> at java.nio.file.Files.copy(Files.java:3027)
> {noformat}
> No good deed goes unpunished... :) Thank you, again!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
