[jira] [Commented] (TEZ-2076) Tez framework to extract/analyze data stored in ATS for specific dag

2015-05-04 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14528000#comment-14528000 ] Rajesh Balamohan commented on TEZ-2076: --- >> What if there are 100,000 attempts? or mor

[jira] [Commented] (TEZ-2401) Tez UI: All-dag page has duration keep counting for KILLED dag.

2015-05-04 Thread Sreenath Somarajapuram (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527996#comment-14527996 ] Sreenath Somarajapuram commented on TEZ-2401: - +1 LGTM > Tez UI: All-dag page h

[jira] [Commented] (TEZ-2366) Pig tez MiniTezCluster unit tests fail intermittently after TEZ-2333

2015-05-04 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527993#comment-14527993 ] TezQA commented on TEZ-2366: {color:green}+1 overall{color}. Here are the results of testing th

Success: TEZ-2366 PreCommit Build #622

2015-05-04 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2366 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/622/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Commented] (TEZ-776) Reduce AM mem usage caused by storing TezEvents

2015-05-04 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527957#comment-14527957 ] TezQA commented on TEZ-776: --- {color:red}-1 overall{color}. Here are the results of testing the lat

Failed: TEZ-776 PreCommit Build #621

2015-05-04 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-776 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/621/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 2

[jira] [Commented] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527937#comment-14527937 ] Jeff Zhang commented on TEZ-2221: - {code} If VertexGroup("A", v1) and VertexGroup("B", v1) a

[jira] [Commented] (TEZ-2392) Have all readers throw an Exception on incorrect next() usage

2015-05-04 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527935#comment-14527935 ] TezQA commented on TEZ-2392: {color:green}+1 overall{color}. Here are the results of testing th

[jira] [Commented] (TEZ-2076) Tez framework to extract/analyze data stored in ATS for specific dag

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527932#comment-14527932 ] Hitesh Shah commented on TEZ-2076: -- bq. the zip doesn't need versioning, because it is an A

Success: TEZ-2392 PreCommit Build #620

2015-05-04 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2392 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/620/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Comment Edited] (TEZ-2076) Tez framework to extract/analyze data stored in ATS for specific dag

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527925#comment-14527925 ] Hitesh Shah edited comment on TEZ-2076 at 5/5/15 5:09 AM: -- Comments

[jira] [Commented] (TEZ-2076) Tez framework to extract/analyze data stored in ATS for specific dag

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527925#comment-14527925 ] Hitesh Shah commented on TEZ-2076: -- Comments: {code] usage: java -cp $HADOOP_CONF_DIR/:.

[jira] [Updated] (TEZ-2366) Pig tez MiniTezCluster unit tests fail intermittently after TEZ-2333

2015-05-04 Thread Prakash Ramachandran (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakash Ramachandran updated TEZ-2366: -- Attachment: TEZ-2366.2.patch patch 2 * added check for shufflemetadata being null. [~hite

[jira] [Commented] (TEZ-2076) Tez framework to extract/analyze data stored in ATS for specific dag

2015-05-04 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527899#comment-14527899 ] Gopal V commented on TEZ-2076: -- Also, the import tool is only the "other" way to produce the .z

Failed: TEZ-2366 PreCommit Build #619

2015-05-04 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2366 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/619/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Commented] (TEZ-2366) Pig tez MiniTezCluster unit tests fail intermittently after TEZ-2333

2015-05-04 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527897#comment-14527897 ] TezQA commented on TEZ-2366: {color:red}-1 overall{color}. Here are the results of testing the

[jira] [Commented] (TEZ-2076) Tez framework to extract/analyze data stored in ATS for specific dag

2015-05-04 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527896#comment-14527896 ] Gopal V commented on TEZ-2076: -- [~hitesh]: the zip doesn't need versioning, because it is an AT

[jira] [Commented] (TEZ-2076) Tez framework to extract/analyze data stored in ATS for specific dag

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527888#comment-14527888 ] Hitesh Shah commented on TEZ-2076: -- bq. How is versioning being handled in the serialized z

[jira] [Updated] (TEZ-2411) Offload DataMovement event creation from the AM to the tasks

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated TEZ-2411: Issue Type: Sub-task (was: Task) Parent: TEZ-753 > Offload DataMovement event creation from the AM to

[jira] [Commented] (TEZ-2369) Add a few unit tests for RootInputInitializerManager

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527884#comment-14527884 ] Hitesh Shah commented on TEZ-2369: -- The patch does not include the "Integer successfulAttem

[jira] [Comment Edited] (TEZ-2369) Add a few unit tests for RootInputInitializerManager

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527884#comment-14527884 ] Hitesh Shah edited comment on TEZ-2369 at 5/5/15 4:29 AM: -- The (mis

[jira] [Updated] (TEZ-2411) Offload DataMovement event creation from the AM to the tasks

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated TEZ-2411: Description: Today the AM creates a new DataMovement event from the original event sent by the producer task a

[jira] [Comment Edited] (TEZ-2366) Pig tez MiniTezCluster unit tests fail intermittently after TEZ-2333

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527881#comment-14527881 ] Hitesh Shah edited comment on TEZ-2366 at 5/5/15 4:26 AM: -- [~pramac

[jira] [Created] (TEZ-2411) Offload DataMovement event creation from the AM to the tasks

2015-05-04 Thread Bikas Saha (JIRA)
Bikas Saha created TEZ-2411: --- Summary: Offload DataMovement event creation from the AM to the tasks Key: TEZ-2411 URL: https://issues.apache.org/jira/browse/TEZ-2411 Project: Apache Tez Issue Type

[jira] [Commented] (TEZ-2366) Pig tez MiniTezCluster unit tests fail intermittently after TEZ-2333

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527881#comment-14527881 ] Hitesh Shah commented on TEZ-2366: -- [~pramachandran] Can you confirm that this path is not

[jira] [Updated] (TEZ-2410) VertexGroupCommitFinishedEvent is not logged correctly

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated TEZ-2410: Priority: Blocker (was: Major) > VertexGroupCommitFinishedEvent is not logged correctly > ---

[jira] [Updated] (TEZ-2410) VertexGroupCommitFinishedEvent is not logged correctly

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated TEZ-2410: Description: VertexGroupCommitFinishedEvent may be logged for non-vertex group commits. VertexGroupCommitFinis

[jira] [Commented] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527872#comment-14527872 ] Bikas Saha commented on TEZ-2221: - If VertexGroup("A", v1) and VertexGroup("B", v1) and conn

[jira] [Created] (TEZ-2410) VertexGroupCommitFinishedEvent is not logged correctly

2015-05-04 Thread Jeff Zhang (JIRA)
Jeff Zhang created TEZ-2410: --- Summary: VertexGroupCommitFinishedEvent is not logged correctly Key: TEZ-2410 URL: https://issues.apache.org/jira/browse/TEZ-2410 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527869#comment-14527869 ] Jeff Zhang commented on TEZ-2221: - I think this is must to have to disallow {code} dag.creat

[jira] [Commented] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527850#comment-14527850 ] Bikas Saha commented on TEZ-2221: - Unless there is a technical reason to not support v1,v2 i

[jira] [Commented] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527846#comment-14527846 ] Bikas Saha commented on TEZ-2221: - By definition oA and oB cannot be part of 2 different gro

[jira] [Comment Edited] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527846#comment-14527846 ] Bikas Saha edited comment on TEZ-2221 at 5/5/15 3:33 AM: - By definit

[jira] [Commented] (TEZ-776) Reduce AM mem usage caused by storing TezEvents

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527845#comment-14527845 ] Bikas Saha commented on TEZ-776: prepareForRouting is guarded by synchronized in Edge which c

[jira] [Updated] (TEZ-776) Reduce AM mem usage caused by storing TezEvents

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated TEZ-776: --- Attachment: TEZ-776.12.patch > Reduce AM mem usage caused by storing TezEvents >

[jira] [Updated] (TEZ-2366) Pig tez MiniTezCluster unit tests fail intermittently after TEZ-2333

2015-05-04 Thread Prakash Ramachandran (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakash Ramachandran updated TEZ-2366: -- Attachment: TEZ-2366.1.patch > Pig tez MiniTezCluster unit tests fail intermittently after

[jira] [Comment Edited] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527830#comment-14527830 ] Hitesh Shah edited comment on TEZ-2221 at 5/5/15 3:06 AM: -- This imp

[jira] [Commented] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527830#comment-14527830 ] Hitesh Shah commented on TEZ-2221: -- This implies that oA ( or oB ) cannot belong to 2 diffe

[jira] [Updated] (TEZ-2392) Have all readers throw an Exception on incorrect next() usage

2015-05-04 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2392: -- Attachment: TEZ-2392.3.patch Thanks @sseth, [~hitesh]. - Yes, the condition in valuesIterator is

[jira] [Commented] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527804#comment-14527804 ] Bikas Saha commented on TEZ-2221: - The commit behavior is different. Only the participating

Failed: TEZ-776 PreCommit Build #618

2015-05-04 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-776 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/618/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 2

[jira] [Commented] (TEZ-776) Reduce AM mem usage caused by storing TezEvents

2015-05-04 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527766#comment-14527766 ] TezQA commented on TEZ-776: --- {color:red}-1 overall{color}. Here are the results of testing the lat

[jira] [Commented] (TEZ-2408) TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527696#comment-14527696 ] Hitesh Shah commented on TEZ-2408: -- Committed to master. > TestTaskAttempt fails to compi

[jira] [Commented] (TEZ-2408) TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527694#comment-14527694 ] Hitesh Shah commented on TEZ-2408: -- Committing shortly. Thanks for the review [~bikassaha].

[jira] [Commented] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527692#comment-14527692 ] Hitesh Shah commented on TEZ-2221: -- I guess the question boils down to what the behavior sh

[jira] [Commented] (TEZ-2408) TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2

2015-05-04 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527690#comment-14527690 ] TezQA commented on TEZ-2408: {color:red}-1 overall{color}. Here are the results of testing the

Failed: TEZ-2408 PreCommit Build #617

2015-05-04 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2408 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/617/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Commented] (TEZ-776) Reduce AM mem usage caused by storing TezEvents

2015-05-04 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527677#comment-14527677 ] Siddharth Seth commented on TEZ-776: - Minor: {code}Target input indices. The number must

[jira] [Updated] (TEZ-2237) Valid events should be sent out when an Output is not started

2015-05-04 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-2237: Summary: Valid events should be sent out when an Output is not started (was: Complex DAG freezes and

[jira] [Updated] (TEZ-2237) Complex DAG freezes and fails (was BufferTooSmallException raised in UnorderedPartitionedKVWriter then DAG lingers)

2015-05-04 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-2237: Attachment: TEZ-2237.3.branch6.txt Patch with the log line fixed for branch-0.6. Thanks for the revie

[jira] [Commented] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527549#comment-14527549 ] Bikas Saha commented on TEZ-2221: - Disallowing this should be ok and sounds related to the j

[jira] [Commented] (TEZ-2392) Have all readers throw an Exception on incorrect next() usage

2015-05-04 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527538#comment-14527538 ] Siddharth Seth commented on TEZ-2392: - Patch looks good to me - ValuesIterator has the c

[jira] [Commented] (TEZ-2237) Complex DAG freezes and fails (was BufferTooSmallException raised in UnorderedPartitionedKVWriter then DAG lingers)

2015-05-04 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527511#comment-14527511 ] Rajesh Balamohan commented on TEZ-2237: --- lgtm. +1. Might need to fix log statement

[jira] [Commented] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527479#comment-14527479 ] Rohini Palaniswamy commented on TEZ-2221: - bq. dag.createVertexGroup("group_1", v1,v

[jira] [Commented] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527463#comment-14527463 ] Hitesh Shah commented on TEZ-2221: -- \cc [~bikassaha] in case he has any input. > VertexGr

[jira] [Commented] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527448#comment-14527448 ] Hitesh Shah commented on TEZ-2221: -- [~rohini] Yes - I believe both are being dis-allowed i.

[jira] [Updated] (TEZ-776) Reduce AM mem usage caused by storing TezEvents

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated TEZ-776: --- Attachment: TEZ-776.11.patch Uploading new patch that creates a new abstract class for on-demand routing APIs, l

[jira] [Comment Edited] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527425#comment-14527425 ] Rohini Palaniswamy edited comment on TEZ-2221 at 5/4/15 10:12 PM:

[jira] [Updated] (TEZ-2408) TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2408: - Affects Version/s: (was: 0.7.0) > TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2 >

[jira] [Updated] (TEZ-2408) TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2408: - Target Version/s: 0.7.0 > TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2 >

[jira] [Updated] (TEZ-2408) TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2408: - Affects Version/s: 0.7.0 > TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2 > ---

[jira] [Commented] (TEZ-2221) VertexGroup name should be unqiue

2015-05-04 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527425#comment-14527425 ] Rohini Palaniswamy commented on TEZ-2221: - bq. what happens if someone does the foll

[jira] [Commented] (TEZ-2198) Fix sorter spill counts

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527423#comment-14527423 ] Hitesh Shah commented on TEZ-2198: -- \cc [~gopalv] [~sseth] for review > Fix sorter spill c

[jira] [Created] (TEZ-2409) Allow different edges to have different routing plugins

2015-05-04 Thread Bikas Saha (JIRA)
Bikas Saha created TEZ-2409: --- Summary: Allow different edges to have different routing plugins Key: TEZ-2409 URL: https://issues.apache.org/jira/browse/TEZ-2409 Project: Apache Tez Issue Type: Task

[jira] [Commented] (TEZ-1529) ATS and TezClient integration in secure kerberos enabled cluster

2015-05-04 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527361#comment-14527361 ] Zhijie Shen commented on TEZ-1529: -- The patch looks good to me overall. Two nits: 1. In ge

[jira] [Commented] (TEZ-2408) TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527324#comment-14527324 ] Bikas Saha commented on TEZ-2408: - lgtm. I remember fixing these (perhaps was TestTaskImpl)

[jira] [Commented] (TEZ-2393) Tez pickup PATH env from gateway machine

2015-05-04 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527296#comment-14527296 ] Jason Lowe commented on TEZ-2393: - I think the main problems will be from anyone who expecte

[jira] [Updated] (TEZ-2408) TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2408: - Attachment: TEZ-2408.1.patch [~bikassaha] [~sseth] review please. > TestTaskAttempt fails to compile again

[jira] [Created] (TEZ-2408) TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2

2015-05-04 Thread Hitesh Shah (JIRA)
Hitesh Shah created TEZ-2408: Summary: TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2 Key: TEZ-2408 URL: https://issues.apache.org/jira/browse/TEZ-2408 Project: Apache Tez I

[jira] [Updated] (TEZ-2408) TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2408: - Priority: Minor (was: Major) > TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2 > --

[jira] [Comment Edited] (TEZ-2407) Drop references to the old DataInputBuffer impl in PipelinedSorter

2015-05-04 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527144#comment-14527144 ] Gopal V edited comment on TEZ-2407 at 5/4/15 7:49 PM: -- No, the issue is

[jira] [Commented] (TEZ-2407) Drop references to the old DataInputBuffer impl in PipelinedSorter

2015-05-04 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527144#comment-14527144 ] Gopal V commented on TEZ-2407: -- No, the issue is that {{DataInputBuffer::getLength()}} has bad

[jira] [Commented] (TEZ-2407) Drop references to the old DataInputBuffer impl in PipelinedSorter

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527135#comment-14527135 ] Hitesh Shah commented on TEZ-2407: -- If refactor then 0.8 makes sense. Was not sure if this

[jira] [Updated] (TEZ-2379) org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: T_ATTEMPT_KILLED at KILLED

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2379: - Attachment: TEZ-2379.branch-0.5.patch > org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid

[jira] [Commented] (TEZ-2405) PipelinedSorter can throw NPE with custom compartor

2015-05-04 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527124#comment-14527124 ] Gopal V commented on TEZ-2405: -- [~hitesh]: nope, this was introduced during 0.7 release cycle i

[jira] [Commented] (TEZ-2407) Drop references to the old DataInputBuffer impl in PipelinedSorter

2015-05-04 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527112#comment-14527112 ] Gopal V commented on TEZ-2407: -- This is code-cleanliness refactoring - this does not add perfor

[jira] [Commented] (TEZ-2379) org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: T_ATTEMPT_KILLED at KILLED

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527102#comment-14527102 ] Hitesh Shah commented on TEZ-2379: -- Re-ran TestFaultTolerance locally without any problems.

[jira] [Commented] (TEZ-2379) org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: T_ATTEMPT_KILLED at KILLED

2015-05-04 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527095#comment-14527095 ] TezQA commented on TEZ-2379: {color:red}-1 overall{color}. Here are the results of testing the

Failed: TEZ-2379 PreCommit Build #616

2015-05-04 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2379 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/616/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Commented] (TEZ-1897) Create a concurrent version of AsyncDispatcher

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527081#comment-14527081 ] Bikas Saha commented on TEZ-1897: - Thanks for catching it. My bad. Fixed. commit 5218f481dba

[jira] [Commented] (TEZ-1897) Create a concurrent version of AsyncDispatcher

2015-05-04 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527068#comment-14527068 ] Siddharth Seth commented on TEZ-1897: - Looks like this went in with concurrentDispatcher

[jira] [Commented] (TEZ-2379) org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: T_ATTEMPT_KILLED at KILLED

2015-05-04 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527041#comment-14527041 ] TezQA commented on TEZ-2379: {color:red}-1 overall{color}. Here are the results of testing the

Failed: TEZ-2379 PreCommit Build #615

2015-05-04 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2379 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/615/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Commented] (TEZ-2379) org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: T_ATTEMPT_KILLED at KILLED

2015-05-04 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527033#comment-14527033 ] Siddharth Seth commented on TEZ-2379: - One thing to consider here is that the individual

[jira] [Updated] (TEZ-2379) org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: T_ATTEMPT_KILLED at KILLED

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2379: - Attachment: TEZ-2379.3.patch Final patch with doc comment. > org.apache.hadoop.yarn.state.InvalidStateTran

[jira] [Commented] (TEZ-2379) org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: T_ATTEMPT_KILLED at KILLED

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526952#comment-14526952 ] Hitesh Shah commented on TEZ-2379: -- Will update the final patch with the relevant note rela

[jira] [Commented] (TEZ-2379) org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: T_ATTEMPT_KILLED at KILLED

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526935#comment-14526935 ] Bikas Saha commented on TEZ-2379: - lgtm pending jenkins. If possible, could you put a commen

[jira] [Comment Edited] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526931#comment-14526931 ] Hitesh Shah edited comment on TEZ-2404 at 5/4/15 5:54 PM: -- BUmping

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526931#comment-14526931 ] Hitesh Shah commented on TEZ-2404: -- BUmping up priority as this means recovery is potential

[jira] [Updated] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2404: - Priority: Critical (was: Major) > Handle DataMovementEvent before its TaskAttemptCompletedEvent > -

[jira] [Updated] (TEZ-2325) Route status update event directly to the attempt

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2325: - Priority: Critical (was: Major) > Route status update event directly to the attempt >

[jira] [Updated] (TEZ-2325) Route status update event directly to the attempt

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2325: - Priority: Major (was: Critical) > Route status update event directly to the attempt >

[jira] [Commented] (TEZ-2405) PipelinedSorter can throw NPE with custom compartor

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526922#comment-14526922 ] Hitesh Shah commented on TEZ-2405: -- Does this affect anyone using pipelinedsorter in 0.5 or

[jira] [Commented] (TEZ-2407) Drop references to the old DataInputBuffer impl in PipelinedSorter

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526921#comment-14526921 ] Hitesh Shah commented on TEZ-2407: -- Any reason why should not be targeted to 0.7.0 or a 0.7

[jira] [Updated] (TEZ-2379) org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: T_ATTEMPT_KILLED at KILLED

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2379: - Attachment: TEZ-2379.2.patch Attached patch with handling for killed attempt on failed/killed task states.

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526916#comment-14526916 ] Bikas Saha commented on TEZ-2404: - TEZ-1897 is not enabled yet. So we dont have to fix this

[jira] [Commented] (TEZ-2076) Tez framework to extract/analyze data stored in ATS for specific dag

2015-05-04 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526833#comment-14526833 ] TezQA commented on TEZ-2076: {color:green}+1 overall{color}. Here are the results of testing th

Success: TEZ-2076 PreCommit Build #614

2015-05-04 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2076 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/614/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Commented] (TEZ-2401) Tez UI: All-dag page has duration keep counting for KILLED dag.

2015-05-04 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526779#comment-14526779 ] TezQA commented on TEZ-2401: {color:red}-1 overall{color}. Here are the results of testing the

Failed: TEZ-2401 PreCommit Build #613

2015-05-04 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2401 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/613/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

  1   2   >