[jira] [Resolved] (TEZ-1725) Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

2014-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-1725. --- Resolution: Fixed Fix Version/s: 0.5.2 Thanks [~gopalv]. Committed to master and branch-0

[jira] [Updated] (TEZ-1698) Cut down on ResourceCalculatorProcessTree overheads in Tez

2014-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1698: -- Attachment: TEZ-1725.addendum.patch Looks like IDEA setup caused this. Attaching the addendum patc

[jira] [Updated] (TEZ-1522) Scheduling can result in out of order execution and slowdown of upstream work

2014-10-30 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-1522: Attachment: TEZ-1522.1.wip.txt Patch I was experimenting with earlier today which (temporarily) change

[jira] [Commented] (TEZ-1724) Refactoring - Consolidate ROOT_INPUT and INPUT

2014-10-30 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191465#comment-14191465 ] Siddharth Seth commented on TEZ-1724: - Changes sound good. Maybe VertexEventRootInputFa

[jira] [Commented] (TEZ-1698) Cut down on ResourceCalculatorProcessTree overheads in Tez

2014-10-30 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191458#comment-14191458 ] Siddharth Seth commented on TEZ-1698: - [~rajesh.balamohan] - I think the license needs t

[jira] [Commented] (TEZ-1725) Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

2014-10-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191455#comment-14191455 ] Gopal V commented on TEZ-1725: -- The error bar seems to be within a percentage, even on a 10Tb q

[jira] [Commented] (TEZ-1725) Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

2014-10-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191449#comment-14191449 ] Gopal V commented on TEZ-1725: -- Comparing LinuxResourceCalculatorPlugin vs MXBeanResourceCalcul

[jira] [Commented] (TEZ-1703) Exception handling for InputInitializer

2014-10-30 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191404#comment-14191404 ] Jeff Zhang commented on TEZ-1703: - [~sseth] The origin patch has one race issue in test case

[jira] [Updated] (TEZ-1725) Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

2014-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1725: -- Attachment: TEZ-1725.3.patch > Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

[jira] [Updated] (TEZ-1725) Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

2014-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1725: -- Attachment: TEZ-1725.2.patch > Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

[jira] [Updated] (TEZ-1725) Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

2014-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1725: -- Attachment: (was: TEZ-1725.2.patch) > Fix nanosecond to millis conversion in TezMxBeanResource

[jira] [Updated] (TEZ-1547) Make use of state change notifier in VertexManagerPlugins

2014-10-30 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated TEZ-1547: Attachment: TEZ-1547.8.patch Patch addresses review comments. Notably, Exception propagation added and tested

[jira] [Updated] (TEZ-1725) Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

2014-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1725: -- Attachment: TEZ-1725.2.patch > Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

[jira] [Updated] (TEZ-1725) Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

2014-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1725: -- Attachment: TEZ-1725.1.patch > Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

[jira] [Created] (TEZ-1725) Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

2014-10-30 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-1725: - Summary: Fix nanosecond to millis conversion in TezMxBeanResourceCalculator Key: TEZ-1725 URL: https://issues.apache.org/jira/browse/TEZ-1725 Project: Apache Tez

[jira] [Resolved] (TEZ-1698) Cut down on ResourceCalculatorProcessTree overheads in Tez

2014-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-1698. --- Resolution: Fixed Fix Version/s: 0.5.2 Hadoop Flags: Reviewed Thanks [~gopalv]. Com

[jira] [Commented] (TEZ-1698) Cut down on ResourceCalculatorProcessTree overheads in Tez

2014-10-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191204#comment-14191204 ] Gopal V commented on TEZ-1698: -- +1 - Thanks Rajesh, this looks good. This isn't on by default,

[jira] [Updated] (TEZ-1698) Cut down on ResourceCalculatorProcessTree overheads in Tez

2014-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1698: -- Attachment: TEZ-1698.4.patch Thanks [~gopalv]. Fixing the test case as cumulativeRSS would return

[jira] [Updated] (TEZ-1724) Refactoring - Consolidate ROOT_INPUT and INPUT

2014-10-30 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-1724: Description: Some places we use ROOT_INPUT , while other places we use INPUT (Actually Input can been attache

[jira] [Updated] (TEZ-1724) Refactoring - Consolidate ROOT_INPUT and INPUT

2014-10-30 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-1724: Description: Some places we use ROOT_INPUT , while other places we use INPUT, we need to consolidate them, he

[jira] [Created] (TEZ-1724) Refactoring - Consolidate ROOT_INPUT and INPUT

2014-10-30 Thread Jeff Zhang (JIRA)
Jeff Zhang created TEZ-1724: --- Summary: Refactoring - Consolidate ROOT_INPUT and INPUT Key: TEZ-1724 URL: https://issues.apache.org/jira/browse/TEZ-1724 Project: Apache Tez Issue Type: Bug Affec

[jira] [Updated] (TEZ-1698) Cut down on ResourceCalculatorProcessTree overheads in Tez

2014-10-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1698: - Attachment: TEZ-1698.3.patch [~rajesh.balamohan]: Can you test this version? Minor changes, the plugin is built on

[jira] [Commented] (TEZ-1703) Exception handling for InputInitializer

2014-10-30 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191192#comment-14191192 ] Jeff Zhang commented on TEZ-1703: - Thanks [~sseth], Committed to master and branch-0.5 > E

[jira] [Updated] (TEZ-1698) Cut down on ResourceCalculatorProcessTree overheads in Tez

2014-10-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1698: - Summary: Cut down on ResourceCalculatorProcessTree overheads in Tez (was: Use ResourceCalculatorPlugin instead of R

[jira] [Comment Edited] (TEZ-1547) Make use of state change notifier in VertexManagerPlugins

2014-10-30 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191181#comment-14191181 ] Bikas Saha edited comment on TEZ-1547 at 10/31/14 1:47 AM: --- Fixed

[jira] [Commented] (TEZ-1547) Make use of state change notifier in VertexManagerPlugins

2014-10-30 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191181#comment-14191181 ] Bikas Saha commented on TEZ-1547: - Fixed the race condition. Added test cases for that. bq.

[jira] [Commented] (TEZ-1703) Exception handling for InputInitializer

2014-10-30 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191180#comment-14191180 ] Siddharth Seth commented on TEZ-1703: - +1. Looks good. Thanks [~zjffdu] > Exception han

[jira] [Commented] (TEZ-1711) Don't cache outputSpecList in VertexImpl.getOutputSpecList(taskIndex)

2014-10-30 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191129#comment-14191129 ] Jeff Zhang commented on TEZ-1711: - Thanks [~bikassaha] Committed to master and branch-0.5

[jira] [Created] (TEZ-1723) Flaky test: TestVertexImpl::testVertexWithMultipleInitializers1

2014-10-30 Thread Hitesh Shah (JIRA)
Hitesh Shah created TEZ-1723: Summary: Flaky test: TestVertexImpl::testVertexWithMultipleInitializers1 Key: TEZ-1723 URL: https://issues.apache.org/jira/browse/TEZ-1723 Project: Apache Tez Issue

[jira] [Commented] (TEZ-1703) Exception handling for InputInitializer

2014-10-30 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191078#comment-14191078 ] Jeff Zhang commented on TEZ-1703: - [~sseth] Update the patch * Revert he renaming changes *

[jira] [Updated] (TEZ-1703) Exception handling for InputInitializer

2014-10-30 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-1703: Attachment: TEZ-1703-4.patch > Exception handling for InputInitializer > -

[jira] [Resolved] (TEZ-1722) DAG should be related to Application Id in ATS data

2014-10-30 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved TEZ-1722. -- Resolution: Fixed Fix Version/s: 0.5.2 Committed to master and branch 0.5 > DAG should be related

[jira] [Updated] (TEZ-1722) DAG should be related to Application Id in ATS data

2014-10-30 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-1722: - Attachment: TEZ-1722.1.patch > DAG should be related to Application Id in ATS data > --

[jira] [Created] (TEZ-1722) DAG should be related to Application Id in ATS data

2014-10-30 Thread Hitesh Shah (JIRA)
Hitesh Shah created TEZ-1722: Summary: DAG should be related to Application Id in ATS data Key: TEZ-1722 URL: https://issues.apache.org/jira/browse/TEZ-1722 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-1703) Exception handling for InputInitializer

2014-10-30 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190731#comment-14190731 ] Siddharth Seth commented on TEZ-1703: - One bit to fix in the patch - V_ROOT_INPUT_FAILED

[jira] [Commented] (TEZ-1711) Don't cache outputSpecList in VertexImpl.getOutputSpecList(taskIndex)

2014-10-30 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190554#comment-14190554 ] Bikas Saha commented on TEZ-1711: - Yes. The RPC would have issues because the next call to g

[jira] [Commented] (TEZ-1547) Make use of state change notifier in VertexManagerPlugins

2014-10-30 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190491#comment-14190491 ] Siddharth Seth commented on TEZ-1547: - Scratch that; I see how this can happen. {code} s

[jira] [Commented] (TEZ-1547) Make use of state change notifier in VertexManagerPlugins

2014-10-30 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190485#comment-14190485 ] Siddharth Seth commented on TEZ-1547: - [~rajesh.balamohan] - I had discounted this case

[jira] [Commented] (TEZ-1704) Derive from Edge configs

2014-10-30 Thread Kostas Tzoumas (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190345#comment-14190345 ] Kostas Tzoumas commented on TEZ-1704: - Siddharth, thank you. I am trying out your sugges

[jira] [Updated] (TEZ-1698) Use ResourceCalculatorPlugin instead of ResourceCalculatorProcessTree in Tez

2014-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1698: -- Attachment: TEZ-1698.2.patch Attaching a patch which has TezMxBeanResourceCalculator to get CPU/Me

[jira] [Commented] (TEZ-1711) Don't cache outputSpecList in VertexImpl.getOutputSpecList(taskIndex)

2014-10-30 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189826#comment-14189826 ] Jeff Zhang commented on TEZ-1711: - [~bikassaha] Please ignore my last patch (TEZ-1711-2.patc

[jira] [Updated] (TEZ-1711) Don't cache outputSpecList in VertexImpl.getOutputSpecList(taskIndex)

2014-10-30 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-1711: Attachment: TEZ-1711-3.patch > Don't cache outputSpecList in VertexImpl.getOutputSpecList(taskIndex) > ---