[
https://issues.apache.org/jira/browse/TEZ-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134302#comment-14134302
]
Bikas Saha commented on TEZ-1563:
---------------------------------
That is correct. Though in this case changes made by Tez are actually stuff the
user should do in theory. Tez is making it convenient to add resources to every
vertex from the common resources in the DAG at compile time.
Thanks for the reviews. Committing.
> TezClient.submitDAGSession alters DAG local resources regardless of DAG
> submission
> ----------------------------------------------------------------------------------
>
> Key: TEZ-1563
> URL: https://issues.apache.org/jira/browse/TEZ-1563
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.5.0
> Reporter: Josh Elser
> Assignee: Bikas Saha
> Attachments: TEZ-1563.1.patch, TEZ-1563.2.patch
>
>
> In {{TezClient#submitDAGSesssion(Dag)}}, a {{DAGPlan}} is created from the
> {{DAG}} before the {{DAGClientAMProtocolBlockingPB}} is instantiated. When
> the application isn't running, {{waitForProxy()}} will throw a
> {{SessionNotRunning}} Exception.
> The problem is that the internal state of the {{DAG}} is modified, regardless
> of whether the DAG is actually run or not.
> {code}
> DAGPlan dagPlan = dag.createDag(amConfig.getTezConfiguration());
> {code}
> The {{createDag}} method will ultimately call {{addTaskLocalFiles}} for each
> {{Vertex}} in the {{DAG}}
> {code}
> // add common task files for this DAG
> vertex.addTaskLocalFiles(commonTaskLocalFiles);
> {code}
> Because the {{DAG}}'s state is modified, {{Vertex#addTaskLocalFiles(Map)}}
> will fail if any resources are added multiple times. As such, if the
> application is not running and {{SessionNotRunning}} is thrown, that same DAG
> cannot be passed in to run the DAG after the application is started again.
> Additionally, {{DAG}} is missing a getTaskLocalFiles method as compared to
> {{Vertex}} which would be good to add to make the two classes more uniform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
