[
https://issues.apache.org/jira/browse/PIG-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15300586#comment-15300586
]
Rohini Palaniswamy commented on PIG-4908:
-----------------------------------------
bq. Without a testcase, I'm not seeing how the incorrect udfs.py was picked.
Explanation is in description. We put the absolute path in the
JythonFunction in the pig client (Oozie launcher). If a different file with
same name exists on that backend task node in that path it runs into errors. It
is hard to write a testcase for it as it you can't have two different files in
the local filesytem on the same machine.
I am going to look a little bit more into ScriptEngine.getScriptAsStream() to
see if there is a better way to fix this.
> JythonFunction refers to Oozie launcher script absolute path
> ------------------------------------------------------------
>
> Key: PIG-4908
> URL: https://issues.apache.org/jira/browse/PIG-4908
> Project: Pig
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.16.0
>
> Attachments: PIG-4908-1.patch, PIG-4908-2-fixtest.patch
>
>
> We had a scenario where a user had multiple udfs all named udfs.py.
> JythonFunction was referring to the absolute localized path of udfs.py in the
> Oozie launcher. Tasks which ran on a node and which had a different version
> of udfs.py localized to same path (hashcode conflict) as the Oozie launcher
> failed. We should be referring to relative path of the files. The current
> code check is the canonical path starts with cwd, but that does not work as
> the files are downloaded to a different location and symlinked to the current
> working directory of the task attempt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
