[
https://issues.apache.org/jira/browse/CRUNCH-551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Wills updated CRUNCH-551:
------------------------------
Attachment: CRUNCH-551.patch
Patch for this that makes sure that both CrunchInputSplit and and
CrunchRecordReader are using a version of the
TaskAttemptContext.getConfiguration() settings that have been modified w/the
FormatBundle settings for the InputSplit.
> Make the use of Configuration objects inside of
> CrunchRecordReader/CrunchInputSplit consistent
> ----------------------------------------------------------------------------------------------
>
> Key: CRUNCH-551
> URL: https://issues.apache.org/jira/browse/CRUNCH-551
> Project: Crunch
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.12.0
> Reporter: Josh Wills
> Assignee: Josh Wills
> Fix For: 0.13.0
>
> Attachments: CRUNCH-551.patch
>
>
> Ran into an issue when testing a Crunch-on-Spark pipeline that was caused by
> an inconsistency in the way we handle Configuration objects inside of
> CrunchInputSplit and CrunchRecordReader. Basically, we can have different
> Configuration objects in play at different parts of the
> InputFormat/InputSplit initialization process depending on when/how the split
> is deserialized and initialized.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
