[
https://issues.apache.org/jira/browse/AVRO-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía updated AVRO-4286:
-------------------------------
Description:
{{DataFileReader.HasNext()}} in
{{lang/csharp/src/apache/main/File/DataFileReader.cs}} calls
{{_codec.Decompress(...)}}, and the codec implementations (deflate, snappy,
zstandard, bzip2, xz) decompress via {{Stream.CopyTo}} into an unbounded
{{MemoryStream}} (or a full-buffer decode) with no size limit.
Enforce a configurable maximum decompressed size and fail decoding with a clear
error when a block would exceed it. The built-in deflate codec is bounded
during decompression (inflated in chunks and rejected before the full output is
materialized); {{DataFileReader}} additionally checks the decompressed size of
every codec output as a safeguard (the snappy, zstandard, bzip2 and xz codecs
are only bounded through this reader-level check, not inside the codec itself).
Add a test with a high-ratio block.
was:
{{DataFileReader.HasNext()}} in
{{lang/csharp/src/apache/main/File/DataFileReader.cs}} calls
{{_codec.Decompress(...)}}, and the codec implementations (deflate, snappy,
zstandard, bzip2, xz) decompress via {{Stream.CopyTo}} into an unbounded
{{MemoryStream}} (or a full-buffer decode) with no size limit.
Enforce a configurable maximum decompressed size in the codecs and fail
decoding with a clear error when a block would exceed it. Add a test with a
high-ratio block.
> [csharp] Enforce a maximum decompressed block size
> --------------------------------------------------
>
> Key: AVRO-4286
> URL: https://issues.apache.org/jira/browse/AVRO-4286
> Project: Apache Avro
> Issue Type: Sub-task
> Components: csharp
> Affects Versions: 1.11.5, 1.12.1
> Reporter: Ismaël Mejía
> Assignee: Ismaël Mejía
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.2
>
> Time Spent: 4h
> Remaining Estimate: 0h
>
> {{DataFileReader.HasNext()}} in
> {{lang/csharp/src/apache/main/File/DataFileReader.cs}} calls
> {{_codec.Decompress(...)}}, and the codec implementations (deflate, snappy,
> zstandard, bzip2, xz) decompress via {{Stream.CopyTo}} into an unbounded
> {{MemoryStream}} (or a full-buffer decode) with no size limit.
> Enforce a configurable maximum decompressed size and fail decoding with a
> clear error when a block would exceed it. The built-in deflate codec is
> bounded during decompression (inflated in chunks and rejected before the full
> output is materialized); {{DataFileReader}} additionally checks the
> decompressed size of every codec output as a safeguard (the snappy,
> zstandard, bzip2 and xz codecs are only bounded through this reader-level
> check, not inside the codec itself). Add a test with a high-ratio block.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
