[jira] [Updated] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3291: -- Attachment: TEZ-3291.4.patch Attaching the patch which explicitly checks whether all splits

[jira] [Commented] (TEZ-3297) Deadlock scenario in AM during ShuffleVertexManager auto reduce

2016-06-12 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326688#comment-15326688 ] TezQA commented on TEZ-3297: {color:red}-1 overall{color}. Here are the results of testing the latest

[jira] [Updated] (TEZ-3297) Deadlock scenario in AM during ShuffleVertexManager auto reduce

2016-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3297: -- Attachment: TEZ-3297.2.branch-0.7.patch Thanks a lot [~sseth], [~bikassaha]. Will commit it

[jira] [Commented] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-12 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326689#comment-15326689 ] Bikas Saha commented on TEZ-3291: - The comment could be more explicit like "this is a workaround for systems

[jira] [Assigned] (TEZ-3297) Deadlock scenario in AM during ShuffleVertexManager auto reduce

2016-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-3297: - Assignee: Rajesh Balamohan > Deadlock scenario in AM during ShuffleVertexManager auto

[jira] [Updated] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3291: -- Attachment: TEZ-3291.3.patch Attaching patch to address review comments. S3 urls can be

[jira] [Commented] (TEZ-3296) Tez job can hang if two vertices at the same root distance have different task requirements

2016-06-12 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326738#comment-15326738 ] Jonathan Eagles commented on TEZ-3296: -- bq. Now - (1*24*3) + 20*3 = 150 = (2*24*3) + 2*3 The formula is

[jira] [Commented] (TEZ-3216) Support for more precise partition stats in VertexManagerEvent

2016-06-12 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326650#comment-15326650 ] Bikas Saha commented on TEZ-3216: - /cc [~rajesh.balamohan] in case he is interested in this optimization. >

[jira] [Commented] (TEZ-3297) Deadlock scenario in AM during ShuffleVertexManager auto reduce

2016-06-12 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326660#comment-15326660 ] Bikas Saha commented on TEZ-3297: - looking at the code further, looks like the crucial change is not holding

[jira] [Updated] (TEZ-3302) Add a version of processorContext.waitForAllInputsReady and waitForAnyInputReady with a timeout

2016-06-12 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3302: Description: This is useful when a Processor needs to check on whether it has been aborted or not,

[jira] [Created] (TEZ-3302) Add a version of processorContext.waitForAllInputsReady and waitForAnyInputReady with a timeout

2016-06-12 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-3302: --- Summary: Add a version of processorContext.waitForAllInputsReady and waitForAnyInputReady with a timeout Key: TEZ-3302 URL: https://issues.apache.org/jira/browse/TEZ-3302

[jira] [Commented] (TEZ-3300) Tez UI: A wiki must be created with info about each page in Tez UI

2016-06-12 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326638#comment-15326638 ] Bikas Saha commented on TEZ-3300: - Could pages to the wiki be linked directly from the UI page for quick

[jira] [Comment Edited] (TEZ-3300) Tez UI: A wiki must be created with info about each page in Tez UI

2016-06-12 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326638#comment-15326638 ] Bikas Saha edited comment on TEZ-3300 at 6/12/16 9:22 PM: -- Could pages to the wiki

[jira] [Commented] (TEZ-3296) Tez job can hang if two vertices at the same root distance have different task requirements

2016-06-12 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326673#comment-15326673 ] Bikas Saha commented on TEZ-3296: - bq. Today each vertex uses a set of three priority values, the low, the

[jira] [Commented] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-12 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326649#comment-15326649 ] Bikas Saha commented on TEZ-3291: - Why the numLoc=1 check only in the size < min case? A comment before the

[jira] [Updated] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3291: -- Attachment: TEZ-3291.2.patch Thanks for the review [~bikassaha]. Attaching the revised patch.

Failed: TEZ-3297 PreCommit Build #1793

2016-06-12 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-3297 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1793/ ### ## LAST 60 LINES OF THE CONSOLE ###

[jira] [Commented] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-12 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326684#comment-15326684 ] Bikas Saha commented on TEZ-3291: - Would the split not have the URLs with S3 in them? Wondering how ORC

[jira] [Commented] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-12 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326714#comment-15326714 ] Gopal V commented on TEZ-3291: -- The numDistinctLocations worries me since this impl leaks into HDFS runs as

[jira] [Commented] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326302#comment-15326302 ] Rajesh Balamohan commented on TEZ-3291: --- It is ready for review [~bikassaha]. haven't renamed the