[jira] [Created] (TEZ-2979) FlakyTest: org.apache.tez.history.TestHistoryParser

2015-12-08 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-2979: - Summary: FlakyTest: org.apache.tez.history.TestHistoryParser Key: TEZ-2979 URL: https://issues.apache.org/jira/browse/TEZ-2979 Project: Apache Tez Issue Ty

[jira] [Commented] (TEZ-2974) Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles

2015-12-07 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15044541#comment-15044541 ] Rajesh Balamohan commented on TEZ-2974: --- test failure is unrelated to this change. >

[jira] [Updated] (TEZ-2974) Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles

2015-12-06 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2974: -- Attachment: TEZ-2974.1.patch Attaching .1 patch. Instead of reading entire TFile into buffer (getV

[jira] [Assigned] (TEZ-2974) Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles

2015-12-06 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-2974: - Assignee: Rajesh Balamohan > Tez tools: TFileRecordReader in tez-tools should support readi

[jira] [Created] (TEZ-2974) Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles

2015-12-06 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-2974: - Summary: Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles Key: TEZ-2974 URL: https://issues.apache.org/jira/browse/TEZ-2974 Project: Ap

[jira] [Commented] (TEZ-2964) Tez AMScheduler - assign to unallocated container

2015-11-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15029734#comment-15029734 ] Rajesh Balamohan commented on TEZ-2964: --- Should "tez.dag.recovery.max.unflushed.events

[jira] [Commented] (TEZ-2956) Handle auto-reduce parallelism when the totalNumBipartiteSourceTasks is 0

2015-11-24 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025719#comment-15025719 ] Rajesh Balamohan commented on TEZ-2956: --- lgtm. +1 minor: Should "Assert.assertTrue(m

[jira] [Updated] (TEZ-2956) Handle auto-reduce parallelism when the totalNumBipartiteSourceTasks is 0

2015-11-22 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2956: -- Attachment: TEZ-2956.2.patch Attaching the revised patch. Retaining the assumptions about the min

[jira] [Commented] (TEZ-2956) Handle auto-reduce parallelism when the totalNumBipartiteSourceTasks is 0

2015-11-19 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015337#comment-15015337 ] Rajesh Balamohan commented on TEZ-2956: --- [~bikassaha] - Please review when you find ti

[jira] [Updated] (TEZ-2956) Handle auto-reduce parallelism when the totalNumBipartiteSourceTasks is 0

2015-11-19 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2956: -- Attachment: TEZ-2956.1.patch > Handle auto-reduce parallelism when the totalNumBipartiteSourceTask

[jira] [Updated] (TEZ-2956) Handle auto-reduce parallelism when the totalNumBipartiteSourceTasks is 0

2015-11-19 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2956: -- Attachment: With_Patch.png Without_Patch.png TEZ-2956_DAG.png > Ha

[jira] [Created] (TEZ-2956) Handle auto-reduce parallelism when the totalNumBipartiteSourceTasks is 0

2015-11-19 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-2956: - Summary: Handle auto-reduce parallelism when the totalNumBipartiteSourceTasks is 0 Key: TEZ-2956 URL: https://issues.apache.org/jira/browse/TEZ-2956 Project: Apache

[jira] [Commented] (TEZ-2950) Poor performance of UnorderedPartitionedKVWriter

2015-11-18 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15012492#comment-15012492 ] Rajesh Balamohan commented on TEZ-2950: --- sizePerBuffer seems to be low as per the log

[jira] [Commented] (TEZ-2480) TEZ-2003: exception when closing output (ignored)

2015-11-18 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15012273#comment-15012273 ] Rajesh Balamohan commented on TEZ-2480: --- lgtm. +1 > TEZ-2003: exception when closing

[jira] [Commented] (TEZ-1491) Tez reducer-side merge's counter update is slow

2015-11-17 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009894#comment-15009894 ] Rajesh Balamohan commented on TEZ-1491: --- In merger case, this would be beneficial if w

[jira] [Resolved] (TEZ-2030) TEZ-UI some vertex level counters are not populated in dag-view

2015-11-05 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-2030. --- Resolution: Cannot Reproduce Haven't seen this in a while with recent builds. Not sure if it got

[jira] [Commented] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093

2015-11-03 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988526#comment-14988526 ] Rajesh Balamohan commented on TEZ-2849: --- [~hitesh] - Yes, .2 patch is good to commit.

[jira] [Updated] (TEZ-2918) Make progress notifications in IOs

2015-11-02 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2918: -- Attachment: jmh-microbenchmark.tar.gz Tried out a simple JMH microbenchmark which updates boolean

[jira] [Comment Edited] (TEZ-2918) Make progress notifications in IOs

2015-11-01 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14984592#comment-14984592 ] Rajesh Balamohan edited comment on TEZ-2918 at 11/2/15 3:22 AM: --

[jira] [Commented] (TEZ-2918) Make progress notifications in IOs

2015-11-01 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14984592#comment-14984592 ] Rajesh Balamohan commented on TEZ-2918: --- Patch introduces volatile updates in inner lo

[jira] [Updated] (TEZ-2814) ATSImportTool has a return statement in a finally block

2015-10-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2814: -- Attachment: TEZ-2814.1.patch [~hitesh] - Please review when you find sometime. > ATSImportTool ha

[jira] [Comment Edited] (TEZ-2814) ATSImportTool has a return statement in a finally block

2015-10-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14980686#comment-14980686 ] Rajesh Balamohan edited comment on TEZ-2814 at 10/29/15 4:06 PM: -

[jira] [Assigned] (TEZ-2814) ATSImportTool has a return statement in a finally block

2015-10-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-2814: - Assignee: Rajesh Balamohan > ATSImportTool has a return statement in a finally block > ---

[jira] [Updated] (TEZ-2244) PipelinedSorter: Progressive allocation for sort-buffers

2015-10-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2244: -- Attachment: TEZ-2244.8.patch Fixing the javadoc issue. Will commit shortly. > PipelinedSorter: Pr

[jira] [Commented] (TEZ-2244) PipelinedSorter: Progressive allocation for sort-buffers

2015-10-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14978808#comment-14978808 ] Rajesh Balamohan commented on TEZ-2244: --- [~hitesh] - It depends on the sort buffers. S

[jira] [Comment Edited] (TEZ-2244) PipelinedSorter: Progressive allocation for sort-buffers

2015-10-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14978808#comment-14978808 ] Rajesh Balamohan edited comment on TEZ-2244 at 10/28/15 5:23 PM: -

[jira] [Updated] (TEZ-2244) PipelinedSorter: Progressive allocation for sort-buffers

2015-10-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2244: -- Attachment: TEZ-2244.7.patch "tez.runtime.pipelined.sorter.lazy-allocate.memory" - (defaults to fa

[jira] [Comment Edited] (TEZ-2244) PipelinedSorter: Progressive allocation for sort-buffers

2015-10-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14978725#comment-14978725 ] Rajesh Balamohan edited comment on TEZ-2244 at 10/28/15 4:44 PM: -

[jira] [Commented] (TEZ-2244) PipelinedSorter: Progressive allocation for sort-buffers

2015-10-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14978669#comment-14978669 ] Rajesh Balamohan commented on TEZ-2244: --- Thanks [~gopalv]. Will upload the new patch w

[jira] [Updated] (TEZ-2244) PipelinedSorter: Progressive allocation for sort-buffers

2015-10-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2244: -- Attachment: TEZ-2244.6.patch Made minor changes to comments. > PipelinedSorter: Progressive alloc

[jira] [Assigned] (TEZ-2912) Add inProgress and completedLogs details in TaskAttemptStartedEvent

2015-10-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-2912: - Assignee: Rajesh Balamohan > Add inProgress and completedLogs details in TaskAttemptStarted

[jira] [Updated] (TEZ-2912) Add inProgress and completedLogs details in TaskAttemptStartedEvent

2015-10-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2912: -- Attachment: TEZ-2912.1.patch [~sseth], [~hitesh] - Please review when you find time. > Add inProg

[jira] [Updated] (TEZ-2912) Add inProgress and completedLogs details in TaskAttemptStartedEvent

2015-10-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2912: -- Target Version/s: 0.8.2 > Add inProgress and completedLogs details in TaskAttemptStartedEvent > --

[jira] [Created] (TEZ-2912) Add inProgress and completedLogs details in TaskAttemptStartedEvent

2015-10-27 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-2912: - Summary: Add inProgress and completedLogs details in TaskAttemptStartedEvent Key: TEZ-2912 URL: https://issues.apache.org/jira/browse/TEZ-2912 Project: Apache Tez

[jira] [Assigned] (TEZ-2898) tez tools : swimlanes.py is broken in master

2015-10-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-2898: - Assignee: Rajesh Balamohan > tez tools : swimlanes.py is broken in master > ---

[jira] [Updated] (TEZ-2898) tez tools : swimlanes.py is broken in master

2015-10-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2898: -- Attachment: TEZ-2898.1.patch [~gopalv] - Please review when you find time. We might need to intro

[jira] [Commented] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093

2015-10-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976975#comment-14976975 ] Rajesh Balamohan commented on TEZ-2849: --- lgtm. +1. Is the explicit cast on SpanItera

[jira] [Updated] (TEZ-2021) Tez tool to analyze shuffle performance in large clusters by mining task logs

2015-10-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2021: -- Attachment: TEZ-2021.3.patch Rebasing the patch as the log formats have changed in the recent rele

[jira] [Updated] (TEZ-2244) PipelinedSorter: Progressive allocation for sort-buffers

2015-10-26 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2244: -- Attachment: TEZ-2244.5.patch - TEZ_RUNTIME_PIPELINED_SORTER_PRE_ALLOCATE_MEMORY (true by default)

[jira] [Commented] (TEZ-2888) Make critical path calculation resilient to AM crash

2015-10-26 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14974768#comment-14974768 ] Rajesh Balamohan commented on TEZ-2888: --- +1. lgtm. VertexInfo's vertexId (ENTITY vs

[jira] [Updated] (TEZ-2882) Consider improving fetch failure handling

2015-10-22 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2882: -- Target Version/s: 0.7.1 (was: 0.8.2) Fix Version/s: 0.7.1 Description: Committed

[jira] [Updated] (TEZ-2907) NPE in IFile.Reader.getLength during final merge operation

2015-10-22 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2907: -- Attachment: TEZ-2907.1.patch [~sseth] - Please review when you find time. > NPE in IFile.Reader

[jira] [Created] (TEZ-2907) NPE in IFile.Reader.getLength during final merge operation

2015-10-22 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-2907: - Summary: NPE in IFile.Reader.getLength during final merge operation Key: TEZ-2907 URL: https://issues.apache.org/jira/browse/TEZ-2907 Project: Apache Tez I

[jira] [Updated] (TEZ-2882) Consider improving fetch failure handling

2015-10-22 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2882: -- Attachment: TEZ-2882.5.branch-0.7.patch rebasing patch for branch-0.7 (does not include TestShuffl

[jira] [Commented] (TEZ-2882) Consider improving fetch failure handling

2015-10-22 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14969336#comment-14969336 ] Rajesh Balamohan commented on TEZ-2882: --- [~sseth] - Should we include this in 0.7 as w

[jira] [Commented] (TEZ-2882) Consider improving fetch failure handling

2015-10-22 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14969335#comment-14969335 ] Rajesh Balamohan commented on TEZ-2882: --- [~jeagles] - It should not. In case of blackl

[jira] [Updated] (TEZ-2882) Consider improving fetch failure handling

2015-10-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2882: -- Attachment: TEZ-2882.5.patch Thanks @sseth This is the one I'm concerned about - and think is a c

[jira] [Comment Edited] (TEZ-2882) Consider improving fetch failure handling

2015-10-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14967481#comment-14967481 ] Rajesh Balamohan edited comment on TEZ-2882 at 10/21/15 5:09 PM: -

[jira] [Created] (TEZ-2898) tez tools : swimlanes.py is broken in master

2015-10-14 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-2898: - Summary: tez tools : swimlanes.py is broken in master Key: TEZ-2898 URL: https://issues.apache.org/jira/browse/TEZ-2898 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-2896) Fix thread names used during Input/Output initialization

2015-10-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14958092#comment-14958092 ] Rajesh Balamohan commented on TEZ-2896: --- +1. lgtm > Fix thread names used during Inpu

[jira] [Created] (TEZ-2897) Consider plotting concurrent task details in critical path analyzer

2015-10-14 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-2897: - Summary: Consider plotting concurrent task details in critical path analyzer Key: TEZ-2897 URL: https://issues.apache.org/jira/browse/TEZ-2897 Project: Apache Tez

[jira] [Updated] (TEZ-2897) Consider plotting concurrent task details in critical path analyzer

2015-10-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2897: -- Issue Type: Improvement (was: Bug) > Consider plotting concurrent task details in critical path a

[jira] [Commented] (TEZ-2888) Make critical path calculation resilient to AM crash

2015-10-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14957991#comment-14957991 ] Rajesh Balamohan commented on TEZ-2888: --- Comment on getLastDataEvents() was an observa

[jira] [Updated] (TEZ-2882) Consider improving fetch failure handling

2015-10-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2882: -- Attachment: TEZ-2882.4.patch - Configs - annotate as private, unstable. Also fraction instead of p

[jira] [Commented] (TEZ-2888) Make critical path calculation resilient to AM crash

2015-10-13 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14956381#comment-14956381 ] Rajesh Balamohan commented on TEZ-2888: --- TaskAttemptInfo::getPostDataExecutionTimeInte

[jira] [Updated] (TEZ-2885) Remove counter logs from AMWebController

2015-10-13 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2885: -- Fix Version/s: 0.7.1 > Remove counter logs from AMWebController >

[jira] [Updated] (TEZ-2885) Remove counter logs from AMWebController

2015-10-13 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2885: -- Target Version/s: 0.7.1 > Remove counter logs from AMWebController > -

[jira] [Commented] (TEZ-2885) Remove counter logs from AMWebController

2015-10-13 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14956281#comment-14956281 ] Rajesh Balamohan commented on TEZ-2885: --- Added to branch-0.7 >> branch 0.7: commit 32

[jira] [Created] (TEZ-2890) Consider removing "tez.runtime.shuffle.notify.readerror" can be removed

2015-10-13 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-2890: - Summary: Consider removing "tez.runtime.shuffle.notify.readerror" can be removed Key: TEZ-2890 URL: https://issues.apache.org/jira/browse/TEZ-2890 Project: Apache T

[jira] [Assigned] (TEZ-2885) Remove counter logs from AMWebController

2015-10-13 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-2885: - Assignee: Rajesh Balamohan > Remove counter logs from AMWebController > ---

[jira] [Updated] (TEZ-2885) Remove counter logs from AMWebController

2015-10-13 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2885: -- Attachment: TEZ-2885.1.patch [~hitesh] - Please review. Moved log in constructCounterMapInfo to de

[jira] [Created] (TEZ-2885) Remove counter logs from AMWebController

2015-10-13 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-2885: - Summary: Remove counter logs from AMWebController Key: TEZ-2885 URL: https://issues.apache.org/jira/browse/TEZ-2885 Project: Apache Tez Issue Type: Bug

[jira] [Updated] (TEZ-2882) Consider improving fetch failure handling

2015-10-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2882: -- Attachment: TEZ-2882.3.patch Attaching the refactored patch based on the following. In case of fe

[jira] [Updated] (TEZ-2882) Consider improving fetch failure handling

2015-10-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2882: -- Attachment: TEZ-2882.2.patch Minor fix in test case and added null check in copySucceeded. > Cons

[jira] [Updated] (TEZ-2882) Consider improving fetch failure handling

2015-10-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2882: -- Attachment: TEZ-2882.1.patch Changes: - Reduced abortFailureLimit from 30 to 15. This should be fi

[jira] [Created] (TEZ-2882) Consider improving fetch failure handling

2015-10-11 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-2882: - Summary: Consider improving fetch failure handling Key: TEZ-2882 URL: https://issues.apache.org/jira/browse/TEZ-2882 Project: Apache Tez Issue Type: Bug

[jira] [Created] (TEZ-2869) Exception in TestUnorderedPartitionedKVWriter

2015-10-07 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-2869: - Summary: Exception in TestUnorderedPartitionedKVWriter Key: TEZ-2869 URL: https://issues.apache.org/jira/browse/TEZ-2869 Project: Apache Tez Issue Type: Bu

[jira] [Commented] (TEZ-2602) Throwing EOFException when launching MR job

2015-10-02 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942099#comment-14942099 ] Rajesh Balamohan commented on TEZ-2602: --- [~hitesh] This is due to a corner case of fix

[jira] [Updated] (TEZ-2716) DefaultSorter.isRleNeeded not thread safe

2015-09-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2716: -- Target Version/s: 0.6.3 (was: 0.7.0) > DefaultSorter.isRleNeeded not thread safe > --

[jira] [Updated] (TEZ-2716) DefaultSorter.isRleNeeded not thread safe

2015-09-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2716: -- Fix Version/s: 0.6.3 > DefaultSorter.isRleNeeded not thread safe > ---

[jira] [Commented] (TEZ-2716) DefaultSorter.isRleNeeded not thread safe

2015-09-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938936#comment-14938936 ] Rajesh Balamohan commented on TEZ-2716: --- Thanks [~jeagles]. Committed to branch-0.6. >

[jira] [Commented] (TEZ-2716) DefaultSorter.isRleNeeded not thread safe

2015-09-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936097#comment-14936097 ] Rajesh Balamohan commented on TEZ-2716: --- It does not cause task failures. During spill

[jira] [Commented] (TEZ-2859) TestMergeManager.testLocalDiskMergeMultipleTasks failing

2015-09-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14934527#comment-14934527 ] Rajesh Balamohan commented on TEZ-2859: --- That is right [~sseth]. Patch takes care of t

[jira] [Assigned] (TEZ-2861) TestPipelinedSorter should use target dir for tests and not /tmp

2015-09-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-2861: - Assignee: Rajesh Balamohan > TestPipelinedSorter should use target dir for tests and not /t

[jira] [Updated] (TEZ-2716) DefaultSorter.isRleNeeded not thread safe

2015-09-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2716: -- Attachment: TEZ-2716.branch-0.6-and-0.5.patch Rebasing the patch for branch-0.6 (should apply for

[jira] [Commented] (TEZ-2858) Stop using System.currentTimeMillis in TestInputReadyTracker

2015-09-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14934192#comment-14934192 ] Rajesh Balamohan commented on TEZ-2858: --- lgtm. +1. > Stop using System.currentTimeMil

[jira] [Updated] (TEZ-2859) TestMergeManager.testLocalDiskMergeMultipleTasks failing

2015-09-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2859: -- Attachment: TEZ-2859.1.patch Introducing artificial delay so that interrupting thread can get a ch

[jira] [Commented] (TEZ-2836) Avoid setting framework/system counters for tasks running in threads

2015-09-25 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14907910#comment-14907910 ] Rajesh Balamohan commented on TEZ-2836: --- lgtm +1. Minor comment: TestTaskExecution2.t

[jira] [Updated] (TEZ-2716) DefaultSorter.isRleNeeded not thread safe

2015-09-22 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2716: -- Attachment: TEZ-2716.2.patch Thanks [~sseth]. Addressing review comments in .2 patch. > DefaultSo

[jira] [Commented] (TEZ-2844) Backport TEZ-2775 to branch-0.7

2015-09-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902060#comment-14902060 ] Rajesh Balamohan commented on TEZ-2844: --- lgtm. +1. Plz remove unused import in Merge

[jira] [Updated] (TEZ-2716) DefaultSorter.isRleNeeded not thread safe

2015-09-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2716: -- Attachment: TEZ-2716.1.patch attaching .1 patch. Main change is to set the last seen value of tot

[jira] [Resolved] (TEZ-2833) Dont create extra directory during ATS file download

2015-09-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-2833. --- Resolution: Fixed Fix Version/s: 0.8.1 Target Version/s: 0.8.1 Thanks [~bikassa

[jira] [Comment Edited] (TEZ-2383) Cleanup input/output/processor contexts in LogicalIOProcessorRuntimeTask

2015-09-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14901585#comment-14901585 ] Rajesh Balamohan edited comment on TEZ-2383 at 9/21/15 11:04 PM: -

[jira] [Commented] (TEZ-2383) Cleanup input/output/processor contexts in LogicalIOProcessorRuntimeTask

2015-09-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14901585#comment-14901585 ] Rajesh Balamohan commented on TEZ-2383: --- [~jeagles] - I have rebased the patch for bra

[jira] [Updated] (TEZ-2833) Dont create extra directory during ATS file download

2015-09-20 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2833: -- Attachment: TEZ-2833.1.txt Patch to download in the same folder instead of creating dagId sub-fold

[jira] [Updated] (TEZ-2383) Cleanup input/output/processor contexts in LogicalIOProcessorRuntimeTask

2015-09-20 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2383: -- Attachment: TEZ-2383.branch-0.6.patch Rebased patch for branch-0.6 > Cleanup input/output/process

[jira] [Updated] (TEZ-1997) Remove synchronization DefaultSorter::isRLENeeded() (Causes sorter to hang indefinitely in large jobs)

2015-09-20 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1997: -- Attachment: TEZ-1997.addendum.patch [~sseth] - Please review addendum patch when you find time. >

[jira] [Commented] (TEZ-2834) tez app hangs at large scale (~30TB)

2015-09-20 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14900142#comment-14900142 ] Rajesh Balamohan commented on TEZ-2834: --- lgtm. +1 Very minor: {noformat} !highestPri

[jira] [Updated] (TEZ-2775) Reduce logging in runtime components

2015-09-20 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2775: -- Attachment: TEZ-2775.5.txt lgtm. +1. Made minor change in ShuffleUtils.logIndividualFetchComplet

[jira] [Commented] (TEZ-2834) tez app hangs at large scale (~30TB)

2015-09-18 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876745#comment-14876745 ] Rajesh Balamohan commented on TEZ-2834: --- Job completed with the patch. RM sometimes re

[jira] [Commented] (TEZ-2816) Build with hadoop 2.4 fails

2015-09-18 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14805329#comment-14805329 ] Rajesh Balamohan commented on TEZ-2816: --- Tried with commit 04571cc53f81b8f05380a062345

[jira] [Commented] (TEZ-2816) Build with hadoop 2.4 fails

2015-09-18 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14805233#comment-14805233 ] Rajesh Balamohan commented on TEZ-2816: --- Latest patch needs a rebase to master? > Bui

[jira] [Commented] (TEZ-2832) Support tests for both SimpleHistory logging and ATS logging

2015-09-17 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14805008#comment-14805008 ] Rajesh Balamohan commented on TEZ-2832: --- lgtm. +1. Were the changes related to "track

[jira] [Commented] (TEZ-814) Improve heuristic for determining a task has failed outputs

2015-09-17 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804876#comment-14804876 ] Rajesh Balamohan commented on TEZ-814: -- lgtm. +1. Even when tez.task.max.allowed.output.

[jira] [Updated] (TEZ-2775) Reduce logging in runtime components

2015-09-17 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2775: -- Attachment: TEZ-2775.3.txt - Added logProgress() in ShuffleEventHandler. Added it in UnorderedKVI

[jira] [Updated] (TEZ-2834) tez app hangs at large scale (~30TB)

2015-09-16 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2834: -- Description: Will attach the DAG. Repro for reference: TPC-DS q_70 @ 30 TB scale. "Map 7" comple

[jira] [Updated] (TEZ-2834) tez app hangs at large scale (~30TB)

2015-09-16 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2834: -- Attachment: application_1442254312093_0095.1.log.gz application_1442254312093_0095.

[jira] [Created] (TEZ-2834) tez app hangs at large scale (~30TB)

2015-09-16 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-2834: - Summary: tez app hangs at large scale (~30TB) Key: TEZ-2834 URL: https://issues.apache.org/jira/browse/TEZ-2834 Project: Apache Tez Issue Type: Bug Aff

[jira] [Comment Edited] (TEZ-2833) Dont create extra directory during ATS file download

2015-09-15 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746711#comment-14746711 ] Rajesh Balamohan edited comment on TEZ-2833 at 9/16/15 2:47 AM: --

[jira] [Commented] (TEZ-2833) Dont create extra directory during ATS file download

2015-09-15 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746711#comment-14746711 ] Rajesh Balamohan commented on TEZ-2833: --- Mainly created separate directory, so that it

<    3   4   5   6   7   8   9   10   11   12   >