-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7695/
-----------------------------------------------------------
Review request for pig.
Description
-------
Streaming fails with a NullPointerException when input to the streaming binary
is from a file. This is because when input is asynchronous, the exec() method
is called by the FileInputThread not by the ExecutableManager thread, and the
JobConf is not set in the FileInputThread.
In the attached patch, I made UdfContext (which includes JobConf) copied from
ExecutableManager to FileInputThread when input is asynchronous. I decided to
copy UdfContext instead of JobConf because something else besides JonConf from
UdfContext might be needed in the future.
I also removed tabs and tailing whites paces from the code, so you may want to
hide white space changes.
Thanks!
This addresses bug PIG-2975.
https://issues.apache.org/jira/browse/PIG-2975
Diffs
-----
src/org/apache/pig/impl/streaming/ExecutableManager.java 1899e10
test/org/apache/pig/test/TestStreaming.java c3b6237
Diff: https://reviews.apache.org/r/7695/diff/
Testing
-------
Added a new unit test case that verifies that JobConf is passed to the
streaming binary via environment even when input is asynchronous:
testAddJobConfToEnvironmentWithASynchInput.
ant test -Dtestcase=TestStreaming passes with both hadoop 20 and 23.
Thanks,
Cheolsoo Park
