[
https://issues.apache.org/jira/browse/AVRO-4307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated AVRO-4307:
---------------------------------
Labels: pull-request-available (was: )
> [javascript] BlockDecoder should destroy itself on decompression error to
> stop upstream I/O
> -------------------------------------------------------------------------------------------
>
> Key: AVRO-4307
> URL: https://issues.apache.org/jira/browse/AVRO-4307
> Project: Apache Avro
> Issue Type: Improvement
> Components: javascript
> Reporter: Ismaël Mejía
> Assignee: Ismaël Mejía
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> BlockDecoder decompresses blocks asynchronously. When a block callback fails
> (corrupt block, invalid sync marker, unknown codec, or the decompression size
> guard from AVRO-4287), the failure path called emit(*error*) once per failing
> block with a bare emit and never destroyed the stream.
> Confirmed empirically on plain master (no dependency on the
> decompression-limit work) with a 394 KB multi-block file decoded through the
> real createFileDecoder path (fs.createReadStream(path).pipe(new
> BlockDecoder())) using a custom codec that always errors:
> * errorCount: 947 -- one *error* event per failing block (an error storm)
> * bytesReadAfterFirstError: 65536 -- the upstream file stream kept reading a
> file already known to be bad
> * srcDestroyed: false, decDestroyed: false -- neither stream was torn down;
> the file descriptor leaked
> Root cause is in BlockDecoder._createBlockCallback (bare emit, no destroy, no
> guard), which predates and is independent of AVRO-4287. createFileDecoder
> using plain .pipe() compounds it: a destination error only unpipes the
> source, it never destroys it.
> Fix: route fatal errors through a guarded _onError() that calls destroy(err)
> so exactly one error surfaces and the stream unwinds; switch
> createFileDecoder to stream.pipeline() so the source file stream is destroyed
> and its descriptor released. After the fix: errorCount 1,
> bytesReadAfterFirstError 0, source and decoder both destroyed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
