[
https://issues.apache.org/jira/browse/TEZ-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182059#comment-15182059
]
Zhiyuan Yang commented on TEZ-3155:
-----------------------------------
Thanks for reviewing my code! I've uploaded another patch which fixes the
problems you mentioned. Sorry for the mistakes and coding style problems.
About TezClient, ratio 0.8 is not necessary especially with large max IPC size,
so I changed it to a fixed gap to the max IPC size.
About TestTezClient, I've added a test about the negative case.
About DAGClientServer, the configuration instance is necessary to get a
FileSystem object so I have to pass the configuration object to the constructor.
About DAGClientServer, I didn't move the deserialization to DAGAppMaster
because that requires change of DAGClientHandler's submitDAG signature which is
bigger change than that I did.
Also, now both client and server side hold one FileSystem instance without
getting one each time which is costly.
> Support a way to submit DAGs to a session where the DAG plan exceeds hadoop
> ipc limits
> ---------------------------------------------------------------------------------------
>
> Key: TEZ-3155
> URL: https://issues.apache.org/jira/browse/TEZ-3155
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Hitesh Shah
> Assignee: Zhiyuan Yang
> Attachments: TEZ-3155.1.patch, TEZ-3155.2.patch, TEZ-3155.3.patch,
> TEZ-3155.4.patch
>
>
> Currently, dag submissions fail if the dag plan exceeds the hadoop ipc
> limits. One option would be to fall back to local resources if the dag plan
> is too large.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
