[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112449#comment-16112449 ]
liyunzhang_intel commented on PIG-5283: --------------------------------------- [~szita]: In [PigInputFormatSpark#createRecordReader|https://github.com/apache/pig/blob/trunk/src/org/apache/pig/backend/hadoop/executionengine/spark/running/PigInputFormatSpark.java#L59], it initialize pigSplit configuration. In my understanding,the configuration of pigSplit is initialized correctly, so currently you met a problem of invalid initialization of pigSplit's configuration? If yes, can you provide simple script to show your problem. > Configuration is not passed to SparkPigSplits on the backend > ------------------------------------------------------------ > > Key: PIG-5283 > URL: https://issues.apache.org/jira/browse/PIG-5283 > Project: Pig > Issue Type: Bug > Components: spark > Reporter: Adam Szita > Assignee: Adam Szita > Attachments: PIG-5283.0.patch > > > When a Hadoop ObjectWritable is created during a Spark job, the instantiated > PigSplit (wrapped into a SparkPigSplit) is given an empty Configuration > instance. > This happens > [here|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SerializableWritable.scala#L44] -- This message was sent by Atlassian JIRA (v6.4.14#64029)