Hadoop should serialize the Configration after the call to getSplits() to the
backend such that any changes to the Configuration in getSplits() is serialized
to the backend
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Key: MAPREDUCE-1620
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1620
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 0.20.2, 0.20.1
Reporter: Pradeep Kamath
In 0.20.1 and 0.20.2, when using the new API, while working on the next pig
release we discovered that the hadoop code makes a copy of the Configuration
and hands a copy to the getSplits() call. Any changes to the Configuration made
in getSplits() are on that copy. However the original Configuraiton is the one
which gets serialized to the backend - hence any changes made to the
Configuration in the getSplits() implementation does not get serialized to the
backend. In a framework like Pig, there are usecases for writing information
into the Configuration during getSplits - it would be helpful if Hadoop would
ensure that these changes get serialized to the backend.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
