[jira] [Updated] (TEZ-3235) Modify Example TestOrderedWordCount job to test the IPC limit for large dag plans

2016-07-12 Thread Sushmitha Sreenivasan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushmitha Sreenivasan updated TEZ-3235: --- Attachment: Tez-3235.3.patch > Modify Example TestOrderedWordCount job to test the IPC

[jira] [Commented] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-07-12 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374379#comment-15374379 ] Tsuyoshi Ozawa commented on TEZ-3303: - Thanks for your review and your committing :-) > Have

[jira] [Commented] (TEZ-3343) sqoop import can't success

2016-07-12 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374251#comment-15374251 ] Jeff Zhang commented on TEZ-3343: - Could you attach the yarn app log ? You can ask this kind of question in

[jira] [Created] (TEZ-3343) sqoop import can't success

2016-07-12 Thread lishaoguang (JIRA)
lishaoguang created TEZ-3343: Summary: sqoop import can't success Key: TEZ-3343 URL: https://issues.apache.org/jira/browse/TEZ-3343 Project: Apache Tez Issue Type: Bug Environment:

Success: TEZ-3303 PreCommit Build #1848

2016-07-12 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-3303 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1848/ ### ## LAST 60 LINES OF THE CONSOLE ###

[jira] [Commented] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-07-12 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374133#comment-15374133 ] TezQA commented on TEZ-3303: {color:green}+1 overall{color}. Here are the results of testing the latest

[jira] [Commented] (TEZ-3334) Tez Custom Shuffle Handler

2016-07-12 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374071#comment-15374071 ] Bikas Saha commented on TEZ-3334: - Also reporting errors properly in the response such that 1 error does not

[jira] [Commented] (TEZ-3334) Tez Custom Shuffle Handler

2016-07-12 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374065#comment-15374065 ] Bikas Saha commented on TEZ-3334: - YARN-4577 for classpath isolation of aux services. Perhaps the first

Success: TEZ-3337 PreCommit Build #1847

2016-07-12 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-3337 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1847/ ### ## LAST 60 LINES OF THE CONSOLE ###

[jira] [Commented] (TEZ-3337) Not log empty fields of TaskAttemptFinishedEvent to avoid confusion

2016-07-12 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374021#comment-15374021 ] TezQA commented on TEZ-3337: {color:green}+1 overall{color}. Here are the results of testing the latest

[jira] [Commented] (TEZ-3331) Add operation specific HDFS counters for Tez UI

2016-07-12 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373981#comment-15373981 ] Hitesh Shah commented on TEZ-3331: -- Also, TEZ-3168 has a wip patch that shows how the shims could be

[jira] [Updated] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-07-12 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3303: Attachment: TEZ-3303.003.02.patch Uploading the modified patch for precommit. > Have

[jira] [Commented] (TEZ-3303) Have ShuffleVertexManager consume more precise partition stats

2016-07-12 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373969#comment-15373969 ] Siddharth Seth commented on TEZ-3303: - Very minor: can we make this an "else if

[jira] [Commented] (TEZ-3331) Add operation specific HDFS counters for Tez UI

2016-07-12 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373947#comment-15373947 ] Ming Ma commented on TEZ-3331: -- [~hitesh] thanks for the info about hadoop shim. bq. Mind adding more details

[jira] [Updated] (TEZ-3340) Add support for YARN Shared Cache

2016-07-12 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated TEZ-3340: - Description: YARN provides shared cache in functionality YARN-1492. According to [~ctrezzo] most of the YARN

[jira] [Created] (TEZ-3342) Have Tez AM generate thread dump on task attempts timeout before killing them

2016-07-12 Thread Ming Ma (JIRA)
Ming Ma created TEZ-3342: Summary: Have Tez AM generate thread dump on task attempts timeout before killing them Key: TEZ-3342 URL: https://issues.apache.org/jira/browse/TEZ-3342 Project: Apache Tez

[jira] [Moved] (TEZ-3340) Add support for YARN Shared Cache

2016-07-12 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma moved YARN-5365 to TEZ-3340: Key: TEZ-3340 (was: YARN-5365) Project: Apache Tez (was: Hadoop YARN) > Add

[jira] [Comment Edited] (TEZ-3337) Not log empty fields of TaskAttemptFinishedEvent to avoid confusion

2016-07-12 Thread Zhiyuan Yang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373841#comment-15373841 ] Zhiyuan Yang edited comment on TEZ-3337 at 7/12/16 10:43 PM: - [~hitesh],

[jira] [Updated] (TEZ-3337) Not log empty fields of TaskAttemptFinishedEvent to avoid confusion

2016-07-12 Thread Zhiyuan Yang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhiyuan Yang updated TEZ-3337: -- Attachment: TEZ-3337.1.patch > Not log empty fields of TaskAttemptFinishedEvent to avoid confusion >

[jira] [Created] (TEZ-3339) Add Tez Counters for bytes-read-by-network-distance FileSystem metrics

2016-07-12 Thread Ming Ma (JIRA)
Ming Ma created TEZ-3339: Summary: Add Tez Counters for bytes-read-by-network-distance FileSystem metrics Key: TEZ-3339 URL: https://issues.apache.org/jira/browse/TEZ-3339 Project: Apache Tez Issue

[jira] [Commented] (TEZ-3337) Not log empty fields of TaskAttemptFinishedEvent to avoid confusion

2016-07-12 Thread Zhiyuan Yang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373787#comment-15373787 ] Zhiyuan Yang commented on TEZ-3337: --- This issue doesn't apply for ATS related things. When we convert

[jira] [Created] (TEZ-3338) Support classloader isolation

2016-07-12 Thread Ming Ma (JIRA)
Ming Ma created TEZ-3338: Summary: Support classloader isolation Key: TEZ-3338 URL: https://issues.apache.org/jira/browse/TEZ-3338 Project: Apache Tez Issue Type: Improvement Reporter:

[jira] [Comment Edited] (TEZ-3337) Not log empty fields of TaskAttemptFinishedEvent to avoid confusion

2016-07-12 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373781#comment-15373781 ] Hitesh Shah edited comment on TEZ-3337 at 7/12/16 9:58 PM: --- If this is the case,

[jira] [Commented] (TEZ-3337) Not log empty fields of TaskAttemptFinishedEvent to avoid confusion

2016-07-12 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373781#comment-15373781 ] Hitesh Shah commented on TEZ-3337: -- If this is the case, the conversion to ATS should also not set the

[jira] [Updated] (TEZ-3331) Add operation specific HDFS counters for Tez UI

2016-07-12 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-3331: - Attachment: TEZ-3331.wip.4.patch Some tests added. > Add operation specific HDFS counters for Tez UI >

[jira] [Created] (TEZ-3337) Not log empty fields of TaskAttemptFinishedEvent to avoid confusion

2016-07-12 Thread Zhiyuan Yang (JIRA)
Zhiyuan Yang created TEZ-3337: - Summary: Not log empty fields of TaskAttemptFinishedEvent to avoid confusion Key: TEZ-3337 URL: https://issues.apache.org/jira/browse/TEZ-3337 Project: Apache Tez

[jira] [Commented] (TEZ-3334) Tez Custom Shuffle Handler

2016-07-12 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373702#comment-15373702 ] Ming Ma commented on TEZ-3334: -- [~bikassaha], for the new YARN aux service isolation, do you mean YARN-1593? >

[jira] [Commented] (TEZ-3336) Hive map-side join job sometimes fails with ROOT_INPUT_INIT_FAILURE

2016-07-12 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373690#comment-15373690 ] Jason Lowe commented on TEZ-3336: - Seems like one fix would be to simply have the MR input initializers

[jira] [Commented] (TEZ-3336) Hive map-side join job sometimes fails with ROOT_INPUT_INIT_FAILURE

2016-07-12 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373680#comment-15373680 ] Jason Lowe commented on TEZ-3336: - One example of the failure: {noformat} Vertex failed, vertexName=Map 1,

[jira] [Commented] (TEZ-3336) Hive map-side join job sometimes fails with ROOT_INPUT_INIT_FAILURE

2016-07-12 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373670#comment-15373670 ] Hitesh Shah commented on TEZ-3336: -- \cc [~hagleitn] > Hive map-side join job sometimes fails with

[jira] [Created] (TEZ-3336) Hive map-side join job sometimes fails with ROOT_INPUT_INIT_FAILURE

2016-07-12 Thread Jason Lowe (JIRA)
Jason Lowe created TEZ-3336: --- Summary: Hive map-side join job sometimes fails with ROOT_INPUT_INIT_FAILURE Key: TEZ-3336 URL: https://issues.apache.org/jira/browse/TEZ-3336 Project: Apache Tez

[jira] [Commented] (TEZ-3334) Tez Custom Shuffle Handler

2016-07-12 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373610#comment-15373610 ] Hitesh Shah commented on TEZ-3334: -- Other feature asks: - control channel to query the shuffle service

[jira] [Commented] (TEZ-3335) DAG client thinks app is still running when app status is null

2016-07-12 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373411#comment-15373411 ] Jason Lowe commented on TEZ-3335: - I thought about fixing this on the YARN side. The YarnClient currently

[jira] [Commented] (TEZ-3335) DAG client thinks app is still running when app status is null

2016-07-12 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373352#comment-15373352 ] Hitesh Shah commented on TEZ-3335: -- \cc [~gtCarrera9] [~vinodkv] > DAG client thinks app is still running

[jira] [Comment Edited] (TEZ-3335) DAG client thinks app is still running when app status is null

2016-07-12 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373330#comment-15373330 ] Hitesh Shah edited comment on TEZ-3335 at 7/12/16 5:58 PM: --- Seems like a bug in

[jira] [Commented] (TEZ-3335) DAG client thinks app is still running when app status is null

2016-07-12 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373330#comment-15373330 ] Hitesh Shah commented on TEZ-3335: -- Seems like a bug in YARN that should be fixed too? Where if the RM does

[jira] [Created] (TEZ-3335) DAG client thinks app is still running when app status is null

2016-07-12 Thread Jason Lowe (JIRA)
Jason Lowe created TEZ-3335: --- Summary: DAG client thinks app is still running when app status is null Key: TEZ-3335 URL: https://issues.apache.org/jira/browse/TEZ-3335 Project: Apache Tez Issue

[jira] [Commented] (TEZ-3330) Error on avro M/R job with Tez: missing configuration property

2016-07-12 Thread Manuel Godbert (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372798#comment-15372798 ] Manuel Godbert commented on TEZ-3330: - I already tried that actually, with no success: the configuration

[jira] [Updated] (TEZ-3334) Tez Custom Shuffle Handler

2016-07-12 Thread Tsuyoshi Ozawa (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated TEZ-3334: Issue Type: New Feature (was: Bug) > Tez Custom Shuffle Handler > -- > >

[jira] [Commented] (TEZ-3334) Tez Custom Shuffle Handler

2016-07-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372216#comment-15372216 ] Rajesh Balamohan commented on TEZ-3334: --- +1 for custom shuffle handler. >> "fetching 100 pieces