[
https://issues.apache.org/jira/browse/PIG-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15394155#comment-15394155
]
Siddharth Seth commented on PIG-4958:
-------------------------------------
bq. . Currently Pig client is hitting AM with dagClient.getDAGStatus() every
second to check completion status and have not seen issues with AM. (Different
issue - 1 sec is too low. Should be at least 5 secs)
That's interesting. TEZ-2149 changed getDagStatus(StatusOptions, long timeout)
to wait on the AM for the specified timeout, or until a final state is reached.
This would occupy the single RPC handler thread configured on this interface.
I'm guessing the timed version of the call is not being used. Something to be
aware of, if attempting multiple connections to the same AM.
> Tez autoparallelism estimation for order by is higher than mapreduce
> --------------------------------------------------------------------
>
> Key: PIG-4958
> URL: https://issues.apache.org/jira/browse/PIG-4958
> Project: Pig
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.17.0
>
> Attachments: PIG-4958-withoutsecurity.patch
>
>
> The input size is calculated from the size of the samples in memory. Size
> in memory is usually 4x or more than the serialized size. Mapreduce estimates
> the number of reducers based on serialized size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
