[
https://issues.apache.org/jira/browse/BEAM-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía updated BEAM-2716:
-------------------------------
Component/s: (was: sdk-java-core)
io-java-avro
> AvroReader should refuse dynamic splits while in the last block
> ---------------------------------------------------------------
>
> Key: BEAM-2716
> URL: https://issues.apache.org/jira/browse/BEAM-2716
> Project: Beam
> Issue Type: Bug
> Components: io-java-avro
> Reporter: Eugene Kirpichov
> Assignee: Eugene Kirpichov
> Priority: Minor
>
> AvroReader is able to detect when it's in the last block:
> https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/io/AvroSource.java#L728
> It could also use this information to avoid wastefully producing dynamic
> splits starting in the range of the current block.
> One way to do this would be to have OffsetRangeTracker have a "claim range"
> operation: claim range of [a, b) is, in terms of correctness, equivalent to
> claiming "a" (it checks whether "a" is within the range), but sets the last
> claimed position to "b" rather than "a", thus protecting more positions from
> being split away.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
