[
https://issues.apache.org/jira/browse/TEZ-2144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337428#comment-14337428
]
Rohini Palaniswamy commented on TEZ-2144:
-----------------------------------------
bq. However, a question on the payload. To confirm, is the input split
information in the payload of only the required descriptor i.e. the input of
the map stage and not replicated for to user payloads of all descriptors?
No. It is set only for the vertex that does the particular LOAD.
bq. For such cases, can you try creating the TezClient in non-session mode by
changing its input configuration. The DAG will be send as a local resource in
that case and things should work.
That will be a bigger change to Pig code and also not worth identifying what
should be in session mode and non-session mode and switching between both.
> Compressing user payload
> ------------------------
>
> Key: TEZ-2144
> URL: https://issues.apache.org/jira/browse/TEZ-2144
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rohini Palaniswamy
>
> Pig sets the input split information in user payload and when running against
> a table with 10s of 1000s of partitions, DAG submission fails with
> java.io.IOException: Requested data length 305844060 is longer than maximum
> configured RPC length 67108864
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
