[jira] [Updated] (TEZ-1024) Fix determination of failed attempts

2014-08-05 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-1024: Attachment: Tez-1024-2.patch Fix determination of failed attempts

[jira] [Commented] (TEZ-1372) Fix preWarm to work after recent API changes

2014-08-05 Thread Thaddeus Diamond (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086221#comment-14086221 ] Thaddeus Diamond commented on TEZ-1372: --- I like [~gopalv]'s idea, but am happy to make

[jira] [Updated] (TEZ-1236) Implement common functionality in base abstract class for EdgeManager

2014-08-05 Thread Chen He (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen He updated TEZ-1236: - Attachment: TEZ-1236.patch In progress patch, remove input parameter of

[jira] [Updated] (TEZ-1369) Provide a configuration variable to reuse containers that had unsuccessful tasks

2014-08-05 Thread Thaddeus Diamond (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thaddeus Diamond updated TEZ-1369: -- Attachment: (was: reuse-failed-containers.thaddeus.patch) Provide a configuration variable

[jira] [Updated] (TEZ-1369) Provide a configuration variable to reuse containers that had unsuccessful tasks

2014-08-05 Thread Thaddeus Diamond (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thaddeus Diamond updated TEZ-1369: -- Attachment: (was: reuse-failed-containers.thaddeus.patch.1) Provide a configuration

[jira] [Commented] (TEZ-1369) Provide a configuration variable to reuse containers that had unsuccessful tasks

2014-08-05 Thread Thaddeus Diamond (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086462#comment-14086462 ] Thaddeus Diamond commented on TEZ-1369: --- I will repost these a bit later. If anyone

[jira] [Commented] (TEZ-1368) TestSecureShuffle failing

2014-08-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086575#comment-14086575 ] Siddharth Seth commented on TEZ-1368: - [~rajesh.balamohan] - the patch looks good

[jira] [Commented] (TEZ-1372) Fix preWarm to work after recent API changes

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086579#comment-14086579 ] Bikas Saha commented on TEZ-1372: - It will still be session.preWarm(FOO) where FOO is Vertex

[jira] [Commented] (TEZ-1317) Simplify MRinput/MROutput configuration

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086599#comment-14086599 ] Bikas Saha commented on TEZ-1317: - Those examples are using generate splits in client and

[jira] [Updated] (TEZ-1372) Fix preWarm to work after recent API changes

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated TEZ-1372: Attachment: TEZ-1372.1.patch Fix preWarm to work after recent API changes

[jira] [Created] (TEZ-1379) EdgeConfigurers should accept a Partitioner configuration, accept parameters for compression and secure shuffle

2014-08-05 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-1379: --- Summary: EdgeConfigurers should accept a Partitioner configuration, accept parameters for compression and secure shuffle Key: TEZ-1379 URL:

[jira] [Commented] (TEZ-1379) EdgeConfigurers should accept a Partitioner configuration, accept parameters for compression and secure shuffle

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086735#comment-14086735 ] Bikas Saha commented on TEZ-1379: - Some tabs are not formatted correctly. Ideally, enabling

[jira] [Commented] (TEZ-1379) EdgeConfigurers should accept a Partitioner configuration, accept parameters for compression and secure shuffle

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086738#comment-14086738 ] Bikas Saha commented on TEZ-1379: - Looks good for the rest. At some point, we need to write

[jira] [Commented] (TEZ-1372) Fix preWarm to work after recent API changes

2014-08-05 Thread Thaddeus Diamond (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086839#comment-14086839 ] Thaddeus Diamond commented on TEZ-1372: --- This looks right on. I'll try it tomorrow

[jira] [Commented] (TEZ-1379) EdgeConfigurers should accept a Partitioner configuration, accept parameters for compression and secure shuffle

2014-08-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086857#comment-14086857 ] Siddharth Seth commented on TEZ-1379: - bq. Ideally, enabling or disabling compression

[jira] [Updated] (TEZ-1360) Provide vertex parallelism to each vertex task

2014-08-05 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1360: - Attachment: TEZ-1332.1.patch Provide vertex parallelism to each vertex task

[jira] [Updated] (TEZ-1379) EdgeConfigurers should accept a Partitioner configuration, accept parameters for compression and secure shuffle

2014-08-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-1379: Attachment: TEZ-1379.2.txt Updated patch based on previous comments. EdgeConfigurers should accept

[jira] [Updated] (TEZ-1347) Consolidate MRHelpers

2014-08-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-1347: Attachment: TEZ-1347-initial-review.txt Took a first pass at the various methods in MRHelpers, and

[jira] [Commented] (TEZ-1379) EdgeConfigurers should accept a Partitioner configuration, accept parameters for compression and secure shuffle

2014-08-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086926#comment-14086926 ] Siddharth Seth commented on TEZ-1379: - bq. configurations since the free-form

[jira] [Updated] (TEZ-1368) TestSecureShuffle failing

2014-08-05 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1368: -- Attachment: TEZ-1368.3.patch Thanks [~sseth]. setFromConfiguration() was already in the patch.

[jira] [Comment Edited] (TEZ-1368) TestSecureShuffle failing

2014-08-05 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086930#comment-14086930 ] Rajesh Balamohan edited comment on TEZ-1368 at 8/5/14 10:49 PM:

[jira] [Commented] (TEZ-1380) Refactor SpecificBuilder's enableEncryption / enableCompression to accept parameters (which then allows them to be used via conditionals)

2014-08-05 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086943#comment-14086943 ] Rajesh Balamohan commented on TEZ-1380: --- Patch for

[jira] [Resolved] (TEZ-1380) Refactor SpecificBuilder's enableEncryption / enableCompression to accept parameters (which then allows them to be used via conditionals)

2014-08-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth resolved TEZ-1380. - Resolution: Duplicate Refactor SpecificBuilder's enableEncryption / enableCompression to accept

[jira] [Commented] (TEZ-1380) Refactor SpecificBuilder's enableEncryption / enableCompression to accept parameters (which then allows them to be used via conditionals)

2014-08-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086945#comment-14086945 ] Siddharth Seth commented on TEZ-1380: - Yep. TEZ-1379 should take care of this.

[jira] [Commented] (TEZ-1379) EdgeConfigurers should accept a Partitioner configuration, accept parameters for compression and secure shuffle

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086952#comment-14086952 ] Bikas Saha commented on TEZ-1379: - We might be trying to do 2 things that are potentially at

[jira] [Created] (TEZ-1381) Set EditLogFileOutputStream.setShouldSkipFsyncForTesting(true) for MiniTezCluster

2014-08-05 Thread Bikas Saha (JIRA)
Bikas Saha created TEZ-1381: --- Summary: Set EditLogFileOutputStream.setShouldSkipFsyncForTesting(true) for MiniTezCluster Key: TEZ-1381 URL: https://issues.apache.org/jira/browse/TEZ-1381 Project: Apache

[jira] [Updated] (TEZ-1372) Fix preWarm to work after recent API changes

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated TEZ-1372: Attachment: (was: TEZ-1372.1.patch) Fix preWarm to work after recent API changes

[jira] [Resolved] (TEZ-1381) Set EditLogFileOutputStream.setShouldSkipFsyncForTesting(true) for MiniTezCluster

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha resolved TEZ-1381. - Resolution: Won't Fix Tried it out. Did not show much gain for us because we are probably not writing

[jira] [Updated] (TEZ-1379) EdgeConfigurers should accept a Partitioner configuration, accept parameters for compression and secure shuffle

2014-08-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-1379: Attachment: (was: TEZ-1379.3.txt) EdgeConfigurers should accept a Partitioner configuration,

[jira] [Commented] (TEZ-1379) EdgeConfigurers should accept a Partitioner configuration, accept parameters for compression and secure shuffle

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087074#comment-14087074 ] Bikas Saha commented on TEZ-1379: - Looks like a reasonable middle ground that also allows

[jira] [Assigned] (TEZ-1153) Allow object caching for more than one vertex

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha reassigned TEZ-1153: --- Assignee: Bikas Saha Allow object caching for more than one vertex

[jira] [Comment Edited] (TEZ-1372) Fix preWarm to work after recent API changes

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087139#comment-14087139 ] Bikas Saha edited comment on TEZ-1372 at 8/6/14 2:21 AM: - That

[jira] [Commented] (TEZ-1372) Fix preWarm to work after recent API changes

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087139#comment-14087139 ] Bikas Saha commented on TEZ-1372: - That latest patch builds fine for me. Fix preWarm to

[jira] [Created] (TEZ-1382) Change ObjectRegistry API to allow for future extensions

2014-08-05 Thread Bikas Saha (JIRA)
Bikas Saha created TEZ-1382: --- Summary: Change ObjectRegistry API to allow for future extensions Key: TEZ-1382 URL: https://issues.apache.org/jira/browse/TEZ-1382 Project: Apache Tez Issue Type:

[jira] [Commented] (TEZ-1372) Fix preWarm to work after recent API changes

2014-08-05 Thread Thaddeus Diamond (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087162#comment-14087162 ] Thaddeus Diamond commented on TEZ-1372: --- Sorry, grabbed the first one accidentally.

[jira] [Updated] (TEZ-1382) Change ObjectRegistry API to allow for future extensions

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated TEZ-1382: Attachment: TEZ-1382.1.patch Change ObjectRegistry API to allow for future extensions

[jira] [Commented] (TEZ-1153) Allow object caching for more than one vertex

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087164#comment-14087164 ] Bikas Saha commented on TEZ-1153: - TEZ-1382 allows this jira to become major as an API

[jira] [Commented] (TEZ-1372) Fix preWarm to work after recent API changes

2014-08-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087174#comment-14087174 ] Bikas Saha commented on TEZ-1372: - TestContainerReuse is known to occasionally fail. The

[jira] [Updated] (TEZ-1194) Make TezUserPayload user facing for payload specification and change to ByteBuffer

2014-08-05 Thread Tsuyoshi OZAWA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated TEZ-1194: Attachment: TEZ-1194.3-wip.patch Attached WIP patch which changed the semantics of *ContextImpl. In

[jira] [Updated] (TEZ-1368) TestSecureShuffle failing

2014-08-05 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1368: -- Fix Version/s: 0.5.0 TestSecureShuffle failing -