[jira] [Issue Comment Deleted] (TEZ-2507) mapreduce.{map|reduce}.java.opts should override tez.task.launch.cmd-opts

2015-05-31 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2507: Comment: was deleted (was: 2 things need to be done for this issue: * Make tez.task.launch.cmd-opts as vertex

[jira] [Updated] (TEZ-2507) mapreduce.{map|reduce}.java.opts should override tez.task.launch.cmd-opts

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2507: Attachment: TEZ-2507-1.patch > mapreduce.{map|reduce}.java.opts should override tez.task.launch.cmd-opts > ---

[jira] [Commented] (TEZ-2507) mapreduce.{map|reduce}.java.opts should override tez.task.launch.cmd-opts

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14567104#comment-14567104 ] Jeff Zhang commented on TEZ-2507: - Currently Tez use both of tez.task.launch.cmd-opts and V

[jira] [Commented] (TEZ-2509) YarnTaskSchedulerService should not try to allocate containers if AM is shutting down

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568396#comment-14568396 ] Jeff Zhang commented on TEZ-2509: - Should also stop assign containers in DelayedContainerMan

[jira] [Commented] (TEZ-2509) YarnTaskSchedulerService should not try to allocate containers if AM is shutting down

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568397#comment-14568397 ] Jeff Zhang commented on TEZ-2509: - Should also stop assigin container in DelayedContainerMan

[jira] [Issue Comment Deleted] (TEZ-2509) YarnTaskSchedulerService should not try to allocate containers if AM is shutting down

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2509: Comment: was deleted (was: Should also stop assign containers in DelayedContainerManager ? ) > YarnTaskSched

[jira] [Comment Edited] (TEZ-2509) YarnTaskSchedulerService should not try to allocate containers if AM is shutting down

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568397#comment-14568397 ] Jeff Zhang edited comment on TEZ-2509 at 6/2/15 2:16 AM: - Should als

[jira] [Comment Edited] (TEZ-2509) YarnTaskSchedulerService should not try to allocate containers if AM is shutting down

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568397#comment-14568397 ] Jeff Zhang edited comment on TEZ-2509 at 6/2/15 2:19 AM: - Should als

[jira] [Commented] (TEZ-2509) YarnTaskSchedulerService should not try to allocate containers if AM is shutting down

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568403#comment-14568403 ] Jeff Zhang commented on TEZ-2509: - Please ignore my last comment, Looks like it is fine. It

[jira] [Comment Edited] (TEZ-2509) YarnTaskSchedulerService should not try to allocate containers if AM is shutting down

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568403#comment-14568403 ] Jeff Zhang edited comment on TEZ-2509 at 6/2/15 2:21 AM: - Please ign

[jira] [Commented] (TEZ-1387) Add proper diagonstics message for disk issues

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568445#comment-14568445 ] Jeff Zhang commented on TEZ-1387: - [~hitesh] We didn't report exit code directly, instead we

[jira] [Resolved] (TEZ-1387) Add proper diagonstics message for disk issues

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang resolved TEZ-1387. - Resolution: Won't Fix Close this jira as disk Issue has been propagated to diagnostics. > Add proper diag

[jira] [Created] (TEZ-2511) Add exitCode to diagnostics when container fails

2015-06-01 Thread Jeff Zhang (JIRA)
Jeff Zhang created TEZ-2511: --- Summary: Add exitCode to diagnostics when container fails Key: TEZ-2511 URL: https://issues.apache.org/jira/browse/TEZ-2511 Project: Apache Tez Issue Type: Improvement

[jira] [Created] (TEZ-2512) Create Tez's ContainerExitStatus to handle different container fail scenarios

2015-06-01 Thread Jeff Zhang (JIRA)
Jeff Zhang created TEZ-2512: --- Summary: Create Tez's ContainerExitStatus to handle different container fail scenarios Key: TEZ-2512 URL: https://issues.apache.org/jira/browse/TEZ-2512 Project: Apache Tez

[jira] [Comment Edited] (TEZ-1387) Add proper diagonstics message for disk issues

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568644#comment-14568644 ] Jeff Zhang edited comment on TEZ-1387 at 6/2/15 6:40 AM: - Create TEZ

[jira] [Commented] (TEZ-1387) Add proper diagonstics message for disk issues

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568644#comment-14568644 ] Jeff Zhang commented on TEZ-1387: - Create TEZ-2511 for adding exitCode to diagnostics when c

[jira] [Updated] (TEZ-2511) Add exitCode to diagnostics when container fails

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2511: Description: Currently we only support for PREEMPTED/DISKS_FAILED, that means except for these 2 cases, there

[jira] [Updated] (TEZ-2511) Add exitCode to diagnostics when container fails

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2511: Description: Currently we only support for PREEMPTED/DISKS_FAILED, that means except for these 2 cases, there

[jira] [Updated] (TEZ-2511) Add exitCode to diagnostics when container fails

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2511: Issue Type: Sub-task (was: Improvement) Parent: TEZ-1240 > Add exitCode to diagnostics when container

[jira] [Updated] (TEZ-2511) Add exitCode to diagnostics when container fails

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2511: Description: Currently we only support for PREEMPTED/DISKS_FAILED, that means except for these 2 cases, there

[jira] [Updated] (TEZ-2512) Create Tez's ContainerExitStatus to handle different container fail scenarios

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2512: Description: Currently we only support for PREEMPTED/DISKS_FAILED, that means except for these 2 cases, there

[jira] [Updated] (TEZ-2511) Add exitCode to diagnostics when container fails

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2511: Description: Currently we only identify the container failure cause for PREEMPTED / DISKS_FAILED, that means

[jira] [Updated] (TEZ-2512) Create Tez's ContainerExitStatus to handle different container fail scenarios

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2512: Issue Type: Sub-task (was: Improvement) Parent: TEZ-1240 > Create Tez's ContainerExitStatus to handle

[jira] [Updated] (TEZ-2511) Add exitCode to diagnostics when container fails

2015-06-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2511: Attachment: TEZ-2511-1.patch > Add exitCode to diagnostics when container fails >

[jira] [Commented] (TEZ-2509) YarnTaskSchedulerService should not try to allocate containers if AM is shutting down

2015-06-02 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568785#comment-14568785 ] Jeff Zhang commented on TEZ-2509: - Patch 2 lgtm except one thing in the unit test. not sure

[jira] [Commented] (TEZ-2509) YarnTaskSchedulerService should not try to allocate containers if AM is shutting down

2015-06-02 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569986#comment-14569986 ] Jeff Zhang commented on TEZ-2509: - +1 > YarnTaskSchedulerService should not try to allocate

[jira] [Updated] (TEZ-2376) Remove TaskAttemptEventType.TA_DIAGNOSTICS_UPDATE

2015-06-02 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2376: Attachment: TEZ-2376-2.patch Patch rebased. > Remove TaskAttemptEventType.TA_DIAGNOSTICS_UPDATE > --

[jira] [Updated] (TEZ-2304) InvalidStateTransitonException TA_SCHEDULE at START_WAIT during recovery

2015-06-02 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2304: Target Version/s: 0.5.4, 0.6.2, 0.7.1 > InvalidStateTransitonException TA_SCHEDULE at START_WAIT during recove

[jira] [Commented] (TEZ-2304) InvalidStateTransitonException TA_SCHEDULE at START_WAIT during recovery

2015-06-02 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14570084#comment-14570084 ] Jeff Zhang commented on TEZ-2304: - Thanks [~hitesh] for review, committed to branch-0.5, bra

[jira] [Updated] (TEZ-2304) InvalidStateTransitonException TA_SCHEDULE at START_WAIT during recovery

2015-06-02 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2304: Fix Version/s: 0.7.1 0.6.2 > InvalidStateTransitonException TA_SCHEDULE at START_WAIT durin

[jira] [Commented] (TEZ-2376) Remove TaskAttemptEventType.TA_DIAGNOSTICS_UPDATE

2015-06-02 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14570122#comment-14570122 ] Jeff Zhang commented on TEZ-2376: - Thanks [~hitesh], Committed to master > Remove TaskAttem

[jira] [Updated] (TEZ-2391) TestVertexImpl timing out at times on jenkins builds

2015-06-02 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2391: Attachment: TEZ-2391-3.patch Upload new patch to address the review comment > TestVertexImpl timing out at ti

[jira] [Commented] (TEZ-2391) TestVertexImpl timing out at times on jenkins builds

2015-06-02 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14570232#comment-14570232 ] Jeff Zhang commented on TEZ-2391: - Committed to branch-0.7 & master > TestVertexImpl timing

[jira] [Created] (TEZ-2531) Support fuzzy search for dag name

2015-06-02 Thread Jeff Zhang (JIRA)
Jeff Zhang created TEZ-2531: --- Summary: Support fuzzy search for dag name Key: TEZ-2531 URL: https://issues.apache.org/jira/browse/TEZ-2531 Project: Apache Tez Issue Type: Improvement Comp

[jira] [Created] (TEZ-2532) Add button for displaying tez related configuration

2015-06-02 Thread Jeff Zhang (JIRA)
Jeff Zhang created TEZ-2532: --- Summary: Add button for displaying tez related configuration Key: TEZ-2532 URL: https://issues.apache.org/jira/browse/TEZ-2532 Project: Apache Tez Issue Type: Improvem

[jira] [Updated] (TEZ-2532) Add button for displaying tez related configuration

2015-06-02 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2532: Attachment: 2015-06-03_1303.png > Add button for displaying tez related configuration > --

[jira] [Created] (TEZ-2533) AM deadlock when shutdown

2015-06-03 Thread Jeff Zhang (JIRA)
Jeff Zhang created TEZ-2533: --- Summary: AM deadlock when shutdown Key: TEZ-2533 URL: https://issues.apache.org/jira/browse/TEZ-2533 Project: Apache Tez Issue Type: Bug Affects Versions: 0.8.0

[jira] [Updated] (TEZ-2533) AM deadlock when shutdown

2015-06-03 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2533: Attachment: syslog_dag_1433145984561_0024_244 > AM deadlock when shutdown > - > >

[jira] [Updated] (TEZ-2533) AM deadlock when shutdown

2015-06-03 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2533: Attachment: jstack.log > AM deadlock when shutdown > - > > Key: TEZ-25

[jira] [Updated] (TEZ-2533) AM deadlock when shutdown

2015-06-03 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2533: Description: AM is shutdown due to AM_REBOOT signal {noformat} 2015-06-03 15:44:25,637 INFO [Dispatcher thread

[jira] [Created] (TEZ-2534) Error handling summary event when shutdown AM

2015-06-03 Thread Jeff Zhang (JIRA)
Jeff Zhang created TEZ-2534: --- Summary: Error handling summary event when shutdown AM Key: TEZ-2534 URL: https://issues.apache.org/jira/browse/TEZ-2534 Project: Apache Tez Issue Type: Bug Affect

[jira] [Updated] (TEZ-2534) Error handling summary event when shutdown AM

2015-06-03 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2534: Description: When AM is shutting down, it will close the summary stream, but there may be still some events i

[jira] [Updated] (TEZ-2534) Error handling summary event when shutdown AM

2015-06-03 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2534: Description: When AM is shutting down, it will close the summary stream, but there may be still some events i

[jira] [Updated] (TEZ-2534) Error handling summary event when shutting down AM

2015-06-03 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2534: Summary: Error handling summary event when shutting down AM (was: Error handling summary event when shutdown

[jira] [Updated] (TEZ-2533) AM deadlock when shutdown

2015-06-04 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2533: Attachment: TEZ-2533-1.patch > AM deadlock when shutdown > - > > Key:

[jira] [Commented] (TEZ-2533) AM deadlock when shutdown

2015-06-04 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14572637#comment-14572637 ] Jeff Zhang commented on TEZ-2533: - [~sseth] [~bikassaha] [~hitesh] [~rajesh.balamohan] Pleas

[jira] [Commented] (TEZ-2533) AM deadlock when shutdown

2015-06-04 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573854#comment-14573854 ] Jeff Zhang commented on TEZ-2533: - Thanks [~hitesh] Committed to branch-0.5, branch-0.6, bra

[jira] [Updated] (TEZ-2229) bower ESUDO Cannot be run with sudo -- during build

2015-06-04 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2229: Target Version/s: 0.6.2, 0.8.0, 0.7.1 > bower ESUDO Cannot be run with sudo -- during build >

[jira] [Commented] (TEZ-2511) Add exitCode to diagnostics when container fails

2015-06-07 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14576513#comment-14576513 ] Jeff Zhang commented on TEZ-2511: - Thanks [~hitesh] Committed to 0.6,0.7 & master > Add exi

[jira] [Commented] (TEZ-2488) Tez AM crashes if a submitted DAG is configured to use invalid resource sizes.

2015-06-08 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578107#comment-14578107 ] Jeff Zhang commented on TEZ-2488: - bq. Why is the isLocal change needed? Will this break wit

[jira] [Updated] (TEZ-2534) Error handling summary event when shutting down AM

2015-06-09 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2534: Target Version/s: 0.6.2, 0.8.0, 0.7.1 > Error handling summary event when shutting down AM > -

[jira] [Updated] (TEZ-2534) Error handling summary event when shutting down AM

2015-06-09 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2534: Attachment: TEZ-2534-1.patch > Error handling summary event when shutting down AM > --

[jira] [Commented] (TEZ-2534) Error handling summary event when shutting down AM

2015-06-09 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578764#comment-14578764 ] Jeff Zhang commented on TEZ-2534: - [~hitesh] Please help review it. > Error handling summa

[jira] [Created] (TEZ-2544) Incorrect dag result due to wrong TaskSpec in recovering

2015-06-09 Thread Jeff Zhang (JIRA)
Jeff Zhang created TEZ-2544: --- Summary: Incorrect dag result due to wrong TaskSpec in recovering Key: TEZ-2544 URL: https://issues.apache.org/jira/browse/TEZ-2544 Project: Apache Tez Issue Type: Bug

[jira] [Updated] (TEZ-2544) Incorrect dag result due to wrong TaskSpec in recovering

2015-06-09 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2544: Description: Expected TaskSpec {noformat} DAGName : OrderedWordCount, VertexName: Summation, VertexParallelism

[jira] [Updated] (TEZ-2544) Incorrect dag result due to wrong TaskSpec in recovering

2015-06-09 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2544: Description: Expected TaskSpec {noformat} DAGName : OrderedWordCount, VertexName: Summation, VertexParallelism

[jira] [Updated] (TEZ-2544) Incorrect dag result due to wrong TaskSpec in recovering

2015-06-09 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2544: Description: Expected TaskSpec {noformat} DAGName : OrderedWordCount, VertexName: Summation, VertexParallelism

[jira] [Updated] (TEZ-2544) Incorrect dag result due to wrong TaskSpec in recovering

2015-06-09 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2544: Issue Type: Sub-task (was: Bug) Parent: TEZ-15 > Incorrect dag result due to wrong TaskSpec in recove

[jira] [Updated] (TEZ-2107) Recovery failure in the case of Auto-reduce parallelism

2015-06-09 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2107: Description: The following errors happens when recovering in the case of auto-reduce parallelism. The task n

[jira] [Updated] (TEZ-2107) Recovery failure in the case of Auto-reduce parallelism

2015-06-09 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2107: Description: The following errors happens when recovering in the case of auto-reduce parallelism. The task n

[jira] [Commented] (TEZ-2544) Incorrect dag result due to wrong TaskSpec in recovering

2015-06-09 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14579772#comment-14579772 ] Jeff Zhang commented on TEZ-2544: - Actually recovery don't work when auto parallelism estima

[jira] [Commented] (TEZ-2534) Error handling summary event when shutting down AM

2015-06-09 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14579823#comment-14579823 ] Jeff Zhang commented on TEZ-2534: - Committed to 0.6, 0.7 & master > Error handling summary

[jira] [Created] (TEZ-2545) Test timed out on TestCommit.testDAGKilledWhileRunning_OnVertexSuccess

2015-06-09 Thread Jeff Zhang (JIRA)
Jeff Zhang created TEZ-2545: --- Summary: Test timed out on TestCommit.testDAGKilledWhileRunning_OnVertexSuccess Key: TEZ-2545 URL: https://issues.apache.org/jira/browse/TEZ-2545 Project: Apache Tez

[jira] [Updated] (TEZ-2545) Test timed out on TestCommit.testDAGKilledWhileRunning_OnVertexSuccess

2015-06-09 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2545: Description: {noformat} Failed org.apache.tez.dag.app.dag.impl.TestCommit.testDAGKilledWhileRunning_OnVertexS

[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs

2015-06-10 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580426#comment-14580426 ] Jeff Zhang commented on TEZ-1961: - [~bikassaha] If you have no further comment, I will commi

[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs

2015-06-10 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581320#comment-14581320 ] Jeff Zhang commented on TEZ-1961: - Thanks [~sseth] [~bikassaha], committed to branch-0.7 & m

[jira] [Commented] (TEZ-2548) TezClient submitDAG can hang if the AM is in the process of shutting down

2015-06-10 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581367#comment-14581367 ] Jeff Zhang commented on TEZ-2548: - Not sure what the log [~gopalv] see, Is it related to TEZ

[jira] [Commented] (TEZ-2548) TezClient submitDAG can hang if the AM is in the process of shutting down

2015-06-10 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581477#comment-14581477 ] Jeff Zhang commented on TEZ-2548: - Link with TEZ-2550, this seems AM hang issue > TezClient

[jira] [Commented] (TEZ-2548) TezClient submitDAG can hang if the AM is in the process of shutting down

2015-06-11 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582712#comment-14582712 ] Jeff Zhang commented on TEZ-2548: - [~hitesh] Should set sessionStopped before DAGAppMaster#s

[jira] [Comment Edited] (TEZ-2548) TezClient submitDAG can hang if the AM is in the process of shutting down

2015-06-11 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582712#comment-14582712 ] Jeff Zhang edited comment on TEZ-2548 at 6/12/15 12:08 AM: --- [~hite

[jira] [Comment Edited] (TEZ-2548) TezClient submitDAG can hang if the AM is in the process of shutting down

2015-06-11 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582712#comment-14582712 ] Jeff Zhang edited comment on TEZ-2548 at 6/12/15 12:09 AM: --- [~hite

[jira] [Commented] (TEZ-2548) TezClient submitDAG can hang if the AM is in the process of shutting down

2015-06-11 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582851#comment-14582851 ] Jeff Zhang commented on TEZ-2548: - +1 > TezClient submitDAG can hang if the AM is in the pr

[jira] [Updated] (TEZ-2545) It is not necessary to

2015-06-11 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2545: Summary: It is not necessary to (was: Test timed out on TestCommit.testDAGKilledWhileRunning_OnVertexSuccess

[jira] [Updated] (TEZ-2545) It is not necessary to start the vertex group commit when DAG is in TERMINATING

2015-06-11 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2545: Summary: It is not necessary to start the vertex group commit when DAG is in TERMINATING (was: It is not nece

[jira] [Commented] (TEZ-2545) It is not necessary to start the vertex group commit when DAG is in TERMINATING

2015-06-11 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582957#comment-14582957 ] Jeff Zhang commented on TEZ-2545: - The reason is that even when the dag is in terminating st

[jira] [Comment Edited] (TEZ-2545) It is not necessary to start the vertex group commit when DAG is in TERMINATING

2015-06-13 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582957#comment-14582957 ] Jeff Zhang edited comment on TEZ-2545 at 6/14/15 4:23 AM: -- The caus

[jira] [Updated] (TEZ-2545) It is not necessary to start the vertex group commit when DAG is in TERMINATING

2015-06-14 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2545: Attachment: TEZ-2545-1.patch > It is not necessary to start the vertex group commit when DAG is in > TERMINAT

[jira] [Commented] (TEZ-2545) It is not necessary to start the vertex group commit when DAG is in TERMINATING

2015-06-15 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586063#comment-14586063 ] Jeff Zhang commented on TEZ-2545: - [~bikassaha] Please help review it. > It is not necessa

[jira] [Comment Edited] (TEZ-1008) Vertex V_INIT should be sent to downstream vertices only after vertex is initialized

2015-06-15 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237506#comment-14237506 ] Jeff Zhang edited comment on TEZ-1008 at 6/16/15 12:00 AM: --- [~hite

[jira] [Comment Edited] (TEZ-1008) Vertex V_INIT should be sent to downstream vertices only after vertex is initialized

2015-06-15 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237506#comment-14237506 ] Jeff Zhang edited comment on TEZ-1008 at 6/16/15 12:00 AM: --- [~hite

[jira] [Updated] (TEZ-2545) It is not necessary to start the vertex group commit when DAG is in TERMINATING

2015-06-15 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2545: Attachment: TEZ-2545-2.patch > It is not necessary to start the vertex group commit when DAG is in > TERMINAT

[jira] [Updated] (TEZ-2545) It is not necessary to start the vertex group commit when DAG is in TERMINATING

2015-06-16 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2545: Attachment: TEZ-2545-3.patch Abstract the logic into method isCommittable > It is not necessary to start the

[jira] [Commented] (TEZ-2545) It is not necessary to start the vertex group commit when DAG is in TERMINATING

2015-06-16 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589007#comment-14589007 ] Jeff Zhang commented on TEZ-2545: - This issue affect TestCommit which is introduced in 0.7.

[jira] [Commented] (TEZ-2557) Port TEZ-1910 to branch-0.5

2015-06-16 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589277#comment-14589277 ] Jeff Zhang commented on TEZ-2557: - +1, build with hadoop 2.2 passed > Port TEZ-1910 to bran

[jira] [Commented] (TEZ-2545) It is not necessary to start the vertex group commit when DAG is in TERMINATING

2015-06-16 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589303#comment-14589303 ] Jeff Zhang commented on TEZ-2545: - Committed to master & branch-0.7. > It is not necessary

[jira] [Created] (TEZ-2564) Branch-0.7 compilation fails

2015-06-17 Thread Jeff Zhang (JIRA)
Jeff Zhang created TEZ-2564: --- Summary: Branch-0.7 compilation fails Key: TEZ-2564 URL: https://issues.apache.org/jira/browse/TEZ-2564 Project: Apache Tez Issue Type: Bug Reporter: Jeff

[jira] [Created] (TEZ-2566) Allow TaskAttemptFinishedEvent without TaskAttemptStartedEvent when it is KILLED/FAILED

2015-06-17 Thread Jeff Zhang (JIRA)
Jeff Zhang created TEZ-2566: --- Summary: Allow TaskAttemptFinishedEvent without TaskAttemptStartedEvent when it is KILLED/FAILED Key: TEZ-2566 URL: https://issues.apache.org/jira/browse/TEZ-2566 Project: Apac

[jira] [Updated] (TEZ-2566) Allow TaskAttemptFinishedEvent without TaskAttemptStartedEvent when it is KILLED/FAILED

2015-06-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2566: Attachment: TEZ-2566-1.patch > Allow TaskAttemptFinishedEvent without TaskAttemptStartedEvent when it is > KI

[jira] [Commented] (TEZ-2566) Allow TaskAttemptFinishedEvent without TaskAttemptStartedEvent when it is KILLED/FAILED

2015-06-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591050#comment-14591050 ] Jeff Zhang commented on TEZ-2566: - Attach patch to fix it. Also add some unit test. [~hites

[jira] [Commented] (TEZ-2562) Multiline log event generated on recovery.

2015-06-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591057#comment-14591057 ] Jeff Zhang commented on TEZ-2562: - Would it be more readable with multiline log ? Especially

[jira] [Updated] (TEZ-2566) Allow TaskAttemptFinishedEvent without TaskAttemptStartedEvent when it is KILLED/FAILED

2015-06-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2566: Description: TEZ-2304 allow logging TaskAttempFinishedEvent even without TaskAttemptStartedEvent but don't ch

[jira] [Commented] (TEZ-2561) Port for TaskAttemptListenerImpTezDag should be configurable

2015-06-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591157#comment-14591157 ] Jeff Zhang commented on TEZ-2561: - Is this due to some firewall issue ? > Port for TaskAtte

[jira] [Comment Edited] (TEZ-2296) Add option to print counters for tez-examples

2015-06-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591282#comment-14591282 ] Jeff Zhang edited comment on TEZ-2296 at 6/18/15 5:35 AM: -- [~saikat

[jira] [Commented] (TEZ-2296) Add option to print counters for tez-examples

2015-06-17 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591282#comment-14591282 ] Jeff Zhang commented on TEZ-2296: - [~saikatr] The patch mostly lgtm, but the patch seems for

[jira] [Commented] (TEZ-2566) Allow TaskAttemptFinishedEvent without TaskAttemptStartedEvent when it is KILLED/FAILED

2015-06-18 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591380#comment-14591380 ] Jeff Zhang commented on TEZ-2566: - release audit warnings is not related with this ticket. W

[jira] [Commented] (TEZ-1273) Refactor DAGAppMaster to state machine based

2015-06-18 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591400#comment-14591400 ] Jeff Zhang commented on TEZ-1273: - bq. writeLock is used in the statemachine handle() call b

[jira] [Comment Edited] (TEZ-1273) Refactor DAGAppMaster to state machine based

2015-06-18 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591400#comment-14591400 ] Jeff Zhang edited comment on TEZ-1273 at 6/18/15 7:44 AM: -- bq. writ

[jira] [Commented] (TEZ-2561) Port for TaskAttemptListenerImpTezDag should be configurable

2015-06-18 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591426#comment-14591426 ] Jeff Zhang commented on TEZ-2561: - make sense to specify port-range. > Port for TaskAttemp

[jira] [Updated] (TEZ-1273) Refactor DAGAppMaster to state machine based

2015-06-18 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-1273: Attachment: TEZ-1273-10.patch > Refactor DAGAppMaster to state machine based > ---

[jira] [Commented] (TEZ-2566) Allow TaskAttemptFinishedEvent without TaskAttemptStartedEvent when it is KILLED/FAILED

2015-06-18 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591442#comment-14591442 ] Jeff Zhang commented on TEZ-2566: - Committed to 0.5,0.6,0.7 & master > Allow TaskAttemptFin

<    3   4   5   6   7   8   9   10   11   12   >