[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-07-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083231#comment-16083231 ] Rajesh Balamohan commented on TEZ-3754: --- This is not an infinite loop similar to the one which got

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-07-11 Thread Prasanth Jayachandran (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083137#comment-16083137 ] Prasanth Jayachandran commented on TEZ-3754: Can we have a counter for shuffle

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-07-11 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083027#comment-16083027 ] Siddharth Seth commented on TEZ-3754: - [~rajesh.balamohan] - what is the corner case in which this is

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-07-10 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081281#comment-16081281 ] Rajesh Balamohan commented on TEZ-3754: --- Agreed that this would block the cases which could have

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-07-10 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080529#comment-16080529 ] Jonathan Eagles commented on TEZ-3754: -- The removal of drop behind also helps in the broadcast fetch

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-07-10 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080500#comment-16080500 ] Jason Lowe commented on TEZ-3754: - Note that we mitigated some of the effects of shuffle hammering in

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-07-10 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080464#comment-16080464 ] Jason Lowe commented on TEZ-3754: - bq. do you why mapreduce explicitly turned this off? As Zhiyuan

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-07-10 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080445#comment-16080445 ] Jonathan Eagles commented on TEZ-3754: -- [~jlowe], do you why mapreduce explicitly turned this off? I

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-07-10 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079940#comment-16079940 ] Rajesh Balamohan commented on TEZ-3754: --- Thanks for the review [~aplusplus]. Pinging the host in

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-07-10 Thread Zhiyuan Yang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079913#comment-16079913 ] Zhiyuan Yang commented on TEZ-3754: --- One side-effect is single large fetch can block other small fetches,

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-06-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068038#comment-16068038 ] Rajesh Balamohan commented on TEZ-3754: --- [~sseth], [~jeagles], [~aplusplus] - Can you plz review when

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-06-07 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041503#comment-16041503 ] Gopal V commented on TEZ-3754: -- [~jeagles]: Netty4 uses the kernel defaults instead, so in LLAP we're reading

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-06-07 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041022#comment-16041022 ] Jonathan Eagles commented on TEZ-3754: -- Agree about the intention of the jira. However, can you comment

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-06-07 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040766#comment-16040766 ] TezQA commented on TEZ-3754: {color:red}-1 overall{color}. Here are the results of testing the latest