[jira] [Commented] (TEZ-2635) Limit number of attempts being downloaded in unordered fetch

[ https://issues.apache.org/jira/browse/TEZ-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14651247#comment-14651247 ] Rajesh Balamohan commented on TEZ-2635: --- Sure [~sseth]. backported 0.7, 0.6, 0.5

[jira] [Commented] (TEZ-2646) Add scheduling casual dependency for attempts

[ https://issues.apache.org/jira/browse/TEZ-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14651281#comment-14651281 ] Rajesh Balamohan commented on TEZ-2646: --- lgtm. +1. I believe dependencies like DME

[jira] [Commented] (TEZ-2552) CRC errors can cause job to run for very long time in large jobs

[ https://issues.apache.org/jira/browse/TEZ-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14648467#comment-14648467 ] Rajesh Balamohan commented on TEZ-2552: --- Fixed CHANGES.txt. Cherry-picked to

[jira] [Updated] (TEZ-2635) Limit number of attempts being downloaded in unordered fetch

[ https://issues.apache.org/jira/browse/TEZ-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2635: -- Attachment: TEZ-2635.3.patch Limit number of attempts being downloaded in unordered fetch

[jira] [Commented] (TEZ-2659) Enhance ATSImportTool to download additional data related to the job

[ https://issues.apache.org/jira/browse/TEZ-2659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647320#comment-14647320 ] Rajesh Balamohan commented on TEZ-2659: --- For e.g, current Hive has got ATSHook

[jira] [Commented] (TEZ-2633) Allow VertexManagerPlugins to receive and report based on attempts instead of tasks

[ https://issues.apache.org/jira/browse/TEZ-2633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14648441#comment-14648441 ] Rajesh Balamohan commented on TEZ-2633: --- Checked mainly on the vertexManagers -

[jira] [Commented] (TEZ-2635) Limit number of attempts being downloaded in unordered fetch

[ https://issues.apache.org/jira/browse/TEZ-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646836#comment-14646836 ] Rajesh Balamohan commented on TEZ-2635: --- [~bikassaha] - Will be posting the revised

[jira] [Commented] (TEZ-2552) CRC errors can cause job to run for very long time in large jobs

[ https://issues.apache.org/jira/browse/TEZ-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646891#comment-14646891 ] Rajesh Balamohan commented on TEZ-2552: --- [~bikassaha] - Given that there may not be

[jira] [Updated] (TEZ-2645) Provide standard analyzers for job analysis

[ https://issues.apache.org/jira/browse/TEZ-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2645: -- Attachment: TEZ-2645.addendum.patch Attaching addendum patch. Provide standard analyzers for

[jira] [Commented] (TEZ-2645) Provide standard analyzers for job analysis

[ https://issues.apache.org/jira/browse/TEZ-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647041#comment-14647041 ] Rajesh Balamohan commented on TEZ-2645: --- Committed addendum to master. commit

[jira] [Updated] (TEZ-2552) CRC errors can cause job to run for very long time in large jobs

[ https://issues.apache.org/jira/browse/TEZ-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2552: -- Attachment: TEZ-2552.branch-0.6.patch TEZ-2552.branch-0.5.patch Attaching

[jira] [Updated] (TEZ-2552) CRC errors can cause job to run for very long time in large jobs

[ https://issues.apache.org/jira/browse/TEZ-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2552: -- Fix Version/s: 0.5.5 0.6.2 CRC errors can cause job to run for very long time

[jira] [Updated] (TEZ-2635) Limit number of attempts being downloaded in unordered fetch

[ https://issues.apache.org/jira/browse/TEZ-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2635: -- Attachment: TEZ-2635.2.patch Changes in recent patch: - includedMaps is incremented for allowed

[jira] [Commented] (TEZ-2552) CRC errors can cause job to run for very long time in large jobs

[ https://issues.apache.org/jira/browse/TEZ-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647179#comment-14647179 ] Rajesh Balamohan commented on TEZ-2552: --- Committed to branch-0.5 and branch-0.6

[jira] [Created] (TEZ-2659) Enhance ATSImportTool to download additional data related to the job

Rajesh Balamohan created TEZ-2659: - Summary: Enhance ATSImportTool to download additional data related to the job Key: TEZ-2659 URL: https://issues.apache.org/jira/browse/TEZ-2659 Project: Apache Tez

[jira] [Updated] (TEZ-2645) Provide standard analyzers for job analysis

[ https://issues.apache.org/jira/browse/TEZ-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2645: -- Attachment: TEZ-2645.3.patch -Added findbugs-excludes.xml -Added license header in pom.xml -Fixed

[jira] [Updated] (TEZ-2645) Provide standard analyzers for job analysis

[ https://issues.apache.org/jira/browse/TEZ-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2645: -- Attachment: TEZ-2645.addendum.1.patch Attaching the revised addendum which fixes pom.xml and the

[jira] [Commented] (TEZ-2613) Fetcher(unordered) using List to store InputAttemptIdentifier can lead to some inefficiency during remove() operation

[ https://issues.apache.org/jira/browse/TEZ-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14645155#comment-14645155 ] Rajesh Balamohan commented on TEZ-2613: --- Lgtm. +1. Minor comment - Should

[jira] [Updated] (TEZ-2645) Provide standard analyzers for job analysis

[ https://issues.apache.org/jira/browse/TEZ-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2645: -- Attachment: TEZ-2645.1.patch Attaching patch for review. [~bikassaha] - Can you please review

[jira] [Resolved] (TEZ-2641) Update jar licenses

[ https://issues.apache.org/jira/browse/TEZ-2641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-2641. --- Resolution: Fixed [~bikassaha] - This is committed to TEZ-2592 branch. Marking it as resolved.

[jira] [Created] (TEZ-2645) Provide standard analyzers for job analysis

Rajesh Balamohan created TEZ-2645: - Summary: Provide standard analyzers for job analysis Key: TEZ-2645 URL: https://issues.apache.org/jira/browse/TEZ-2645 Project: Apache Tez Issue Type:

[jira] [Commented] (TEZ-2602) Throwing EOFException when launching MR job

[ https://issues.apache.org/jira/browse/TEZ-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14642132#comment-14642132 ] Rajesh Balamohan commented on TEZ-2602: --- This is specific to 0.8 codebase and does not

[jira] [Commented] (TEZ-2613) Fetcher(unordered) using List to store InputAttemptIdentifier can lead to some inefficiency during remove() operation

[ https://issues.apache.org/jira/browse/TEZ-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14642131#comment-14642131 ] Rajesh Balamohan commented on TEZ-2613: --- Thanks for sharing the patch [~saikatr]. Very

[jira] [Commented] (TEZ-2613) Fetcher(unordered) using List to store InputAttemptIdentifier can lead to some inefficiency during remove() operation

[ https://issues.apache.org/jira/browse/TEZ-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639962#comment-14639962 ] Rajesh Balamohan commented on TEZ-2613: --- [~saikatr] -

[jira] [Updated] (TEZ-2641) Update jar licenses

[ https://issues.apache.org/jira/browse/TEZ-2641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2641: -- Attachment: TEZ-2641.1.patch Update jar licenses --- Key:

[jira] [Created] (TEZ-2641) Update jar licenses

Rajesh Balamohan created TEZ-2641: - Summary: Update jar licenses Key: TEZ-2641 URL: https://issues.apache.org/jira/browse/TEZ-2641 Project: Apache Tez Issue Type: Sub-task

[jira] [Updated] (TEZ-2641) Update jar licenses

[ https://issues.apache.org/jira/browse/TEZ-2641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2641: -- Attachment: TEZ-2641.2.patch Update jar licenses --- Key:

[jira] [Commented] (TEZ-2634) Provide SimpleHistoryParser to parse SimpleHistoryLog and normalize to in-memory datamodel provided in tez-history

[ https://issues.apache.org/jira/browse/TEZ-2634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636359#comment-14636359 ] Rajesh Balamohan commented on TEZ-2634: --- [~hitesh] - Would TEZ-2628 be backported to

[jira] [Created] (TEZ-2635) Limit number of attempts being downloaded in unordered fetch

Rajesh Balamohan created TEZ-2635: - Summary: Limit number of attempts being downloaded in unordered fetch Key: TEZ-2635 URL: https://issues.apache.org/jira/browse/TEZ-2635 Project: Apache Tez

[jira] [Updated] (TEZ-2635) Limit number of attempts being downloaded in unordered fetch

[ https://issues.apache.org/jira/browse/TEZ-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2635: -- Description: {noformat} 2015-07-22 23:39:14,221 WARN [Fetcher [Map_3] #4] shuffle.Fetcher: Fetch

[jira] [Assigned] (TEZ-2635) Limit number of attempts being downloaded in unordered fetch

[ https://issues.apache.org/jira/browse/TEZ-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-2635: - Assignee: Rajesh Balamohan Limit number of attempts being downloaded in unordered fetch

[jira] [Updated] (TEZ-2635) Limit number of attempts being downloaded in unordered fetch

[ https://issues.apache.org/jira/browse/TEZ-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2635: -- Attachment: TEZ-2635.1.patch tez2635.tar.gz Was able to reproduce this with small

[jira] [Updated] (TEZ-2634) Provide SimpleHistoryParser to parse SimpleHistoryLog and normalize to in-memory datamodel provided in tez-history

[ https://issues.apache.org/jira/browse/TEZ-2634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2634: -- Attachment: TEZ-2634.1.patch Attaching .1 patch for review. [~bikassaha], [~hitesh] - Plz review

[jira] [Updated] (TEZ-2634) Provide SimpleHistoryParser to parse SimpleHistoryLog and normalize to in-memory datamodel provided in tez-history

[ https://issues.apache.org/jira/browse/TEZ-2634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2634: -- Description: Would be useful when only SimpleHistoryLogging is available ATS data is not

[jira] [Updated] (TEZ-2001) Support pipelined data transfer for ordered output

[ https://issues.apache.org/jira/browse/TEZ-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2001: -- Fix Version/s: 0.7.0 Support pipelined data transfer for ordered output

[jira] [Updated] (TEZ-2496) Consider scheduling tasks in ShuffleVertexManager based on the partition sizes from the source

[ https://issues.apache.org/jira/browse/TEZ-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2496: -- Attachment: TEZ-2496.9.patch Tested @ 200 GB scale in small scale cluster (launched 35 mappers

[jira] [Updated] (TEZ-2496) Consider scheduling tasks in ShuffleVertexManager based on the partition sizes from the source

[ https://issues.apache.org/jira/browse/TEZ-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2496: -- Attachment: TEZ-2496.9.addendum.patch Unused math import from DATA_RANGE_IN_MB caused builds

[jira] [Commented] (TEZ-2602) Throwing EOFException when launching MR job

[ https://issues.apache.org/jira/browse/TEZ-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625727#comment-14625727 ] Rajesh Balamohan commented on TEZ-2602: --- Thanks [~hitesh] Throwing EOFException when

[jira] [Created] (TEZ-2615) Consider allocating completed fetcher memory to other running tasks for more in-memory fetches

Rajesh Balamohan created TEZ-2615: - Summary: Consider allocating completed fetcher memory to other running tasks for more in-memory fetches Key: TEZ-2615 URL: https://issues.apache.org/jira/browse/TEZ-2615

[jira] [Created] (TEZ-2612) Critical path analyzer for DAGs

Rajesh Balamohan created TEZ-2612: - Summary: Critical path analyzer for DAGs Key: TEZ-2612 URL: https://issues.apache.org/jira/browse/TEZ-2612 Project: Apache Tez Issue Type: Bug

[jira] [Updated] (TEZ-2602) Throwing EOFException when launching MR job

[ https://issues.apache.org/jira/browse/TEZ-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2602: -- Attachment: TEZ-2602.1.patch Thanks [~ozawa]. Posting the refined version. Added 2 more test

[jira] [Commented] (TEZ-2610) Swimlane for DAGs that use containers from previous DAGs

[ https://issues.apache.org/jira/browse/TEZ-2610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14622303#comment-14622303 ] Rajesh Balamohan commented on TEZ-2610: --- +1. tried it on local vm with couple of

[jira] [Updated] (TEZ-2602) Throwing EOFException when launching MR job

[ https://issues.apache.org/jira/browse/TEZ-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2602: -- Attachment: TEZ-2602.WIP.1.patch Attaching a WIP patch. Basically spill() was not invoked when

[jira] [Commented] (TEZ-2602) Throwing EOFException when launching MR job

[ https://issues.apache.org/jira/browse/TEZ-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14620098#comment-14620098 ] Rajesh Balamohan commented on TEZ-2602: --- Not yet [~ozawa], tried with different

[jira] [Commented] (TEZ-2609) Consider calling TaskReporter...-..heartbeat() when there are no more events to be processed in LogicalIOProcessorRuntimeTask

[ https://issues.apache.org/jira/browse/TEZ-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621411#comment-14621411 ] Rajesh Balamohan commented on TEZ-2609: --- This JIRA is for the task side improvements

[jira] [Commented] (TEZ-2591) Remove unneeded methods from EdgeManagerPludinOnDemand

[ https://issues.apache.org/jira/browse/TEZ-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618507#comment-14618507 ] Rajesh Balamohan commented on TEZ-2591: --- lgtm from patch perspective. Only concern is

[jira] [Updated] (TEZ-2496) Consider scheduling tasks in ShuffleVertexManager based on the partition sizes from the source

[ https://issues.apache.org/jira/browse/TEZ-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2496: -- Attachment: TEZ-2496.8.patch - Enabled tez.runtime.report.partition.stats=true - Moved

[jira] [Commented] (TEZ-2496) Consider scheduling tasks in ShuffleVertexManager based on the partition sizes from the source

[ https://issues.apache.org/jira/browse/TEZ-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619670#comment-14619670 ] Rajesh Balamohan commented on TEZ-2496: --- Thanks [~bikassaha]. Will implement both

[jira] [Updated] (TEZ-2609) Consider calling TaskReporter...-..heartbeat() when there are no more events to be processed in LogicalIOProcessorRuntimeTask

[ https://issues.apache.org/jira/browse/TEZ-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2609: -- Summary: Consider calling TaskReporter...-..heartbeat() when there are no more events to be

[jira] [Created] (TEZ-2609) Consider calling TaskReporter.heartbeat() when there are no more events to be processed in LogicalIOProcessorRuntimeTask

Rajesh Balamohan created TEZ-2609: - Summary: Consider calling TaskReporter.heartbeat() when there are no more events to be processed in LogicalIOProcessorRuntimeTask Key: TEZ-2609 URL:

[jira] [Commented] (TEZ-2591) Remove unneeded methods from EdgeManagerPludinOnDemand

[ https://issues.apache.org/jira/browse/TEZ-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617646#comment-14617646 ] Rajesh Balamohan commented on TEZ-2591: --- Earlier it was handling both cases (i.e

[jira] [Updated] (TEZ-2496) Consider scheduling tasks in ShuffleVertexManager based on the partition sizes from the source

[ https://issues.apache.org/jira/browse/TEZ-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2496: -- Attachment: TEZ-2496.7.patch Based on VertexManagerEvent being sent to ShuffleVertexManager

[jira] [Commented] (TEZ-2604) PipelinedSorter doesn't use number of items when creating SortSpan

[ https://issues.apache.org/jira/browse/TEZ-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617873#comment-14617873 ] Rajesh Balamohan commented on TEZ-2604: --- [~ozawa] - This is covered as a part of

[jira] [Commented] (TEZ-2602) Throwing EOFException when launching MR job

[ https://issues.apache.org/jira/browse/TEZ-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617917#comment-14617917 ] Rajesh Balamohan commented on TEZ-2602: --- [~Tsuyoshi Ozawa] - I am trying to reproduce

[jira] [Comment Edited] (TEZ-2602) Throwing EOFException when launching MR job

[ https://issues.apache.org/jira/browse/TEZ-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617917#comment-14617917 ] Rajesh Balamohan edited comment on TEZ-2602 at 7/8/15 3:38 AM: ---

[jira] [Updated] (TEZ-2496) Consider scheduling tasks in ShuffleVertexManager based on the partition sizes from the source

[ https://issues.apache.org/jira/browse/TEZ-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2496: -- Attachment: TEZ-2496.8.patch missed out the sorter test changes. ignoring the findbugs warning in

[jira] [Updated] (TEZ-2496) Consider scheduling tasks in ShuffleVertexManager based on the partition sizes from the source

[ https://issues.apache.org/jira/browse/TEZ-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2496: -- Attachment: TEZ-2496.6.patch Minor codefix in ShuffleVertexManager. Consider scheduling tasks

[jira] [Commented] (TEZ-2496) Consider scheduling tasks in ShuffleVertexManager based on the partition sizes from the source

[ https://issues.apache.org/jira/browse/TEZ-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615847#comment-14615847 ] Rajesh Balamohan commented on TEZ-2496: --- - For 50K x 20K, it would end up with ~760MB

[jira] [Updated] (TEZ-2496) Consider scheduling tasks in ShuffleVertexManager based on the partition sizes from the source

[ https://issues.apache.org/jira/browse/TEZ-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2496: -- Attachment: TEZ-2496.5.patch Addressing review comments. Attaching the patch and comments here

[jira] [Updated] (TEZ-2575) Handle KeyValue pairs size which do not fit in a single block

[ https://issues.apache.org/jira/browse/TEZ-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2575: -- Attachment: TEZ-2575.4.patch Added

[jira] [Comment Edited] (TEZ-2591) Remove unneeded methods from EdgeManagerPludinOnDemand

[ https://issues.apache.org/jira/browse/TEZ-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611533#comment-14611533 ] Rajesh Balamohan edited comment on TEZ-2591 at 7/2/15 6:08 AM: ---

[jira] [Commented] (TEZ-2591) Remove unneeded methods from EdgeManagerPludinOnDemand

[ https://issues.apache.org/jira/browse/TEZ-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611533#comment-14611533 ] Rajesh Balamohan commented on TEZ-2591: --- [~bikassaha] EdgeManagerPluginBase.java

[jira] [Commented] (TEZ-2599) Dont send obsoleted data movement events to tasks

[ https://issues.apache.org/jira/browse/TEZ-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14612716#comment-14612716 ] Rajesh Balamohan commented on TEZ-2599: --- In Edge.maybeAddTezEventForDestinationTask(),

[jira] [Commented] (TEZ-2575) Handle KeyValue pairs size which do not fit in a single block

[ https://issues.apache.org/jira/browse/TEZ-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14612636#comment-14612636 ] Rajesh Balamohan commented on TEZ-2575: --- lgtm. Thanks [~saikatr]. One very minor

[jira] [Commented] (TEZ-2575) Handle KeyValue pairs size which do not fit in a single block

[ https://issues.apache.org/jira/browse/TEZ-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611176#comment-14611176 ] Rajesh Balamohan commented on TEZ-2575: --- bq. 3.10 sort() sees spills the span, resets

[jira] [Commented] (TEZ-2575) Handle KeyValue pairs size which do not fit in a single block

[ https://issues.apache.org/jira/browse/TEZ-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609529#comment-14609529 ] Rajesh Balamohan commented on TEZ-2575: --- bq.I had one question, with your proposed

[jira] [Commented] (TEZ-2575) Handle KeyValue pairs size which do not fit in a single block

[ https://issues.apache.org/jira/browse/TEZ-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606903#comment-14606903 ] Rajesh Balamohan commented on TEZ-2575: --- Thanks for the patch [~saikatr]. Minor

[jira] [Created] (TEZ-2582) Consider removing DataInputBuffer sync overheads in pipelinedsorter

Rajesh Balamohan created TEZ-2582: - Summary: Consider removing DataInputBuffer sync overheads in pipelinedsorter Key: TEZ-2582 URL: https://issues.apache.org/jira/browse/TEZ-2582 Project: Apache Tez

[jira] [Created] (TEZ-2584) Investigate: Reducers with local errors can run indefinitely

Rajesh Balamohan created TEZ-2584: - Summary: Investigate: Reducers with local errors can run indefinitely Key: TEZ-2584 URL: https://issues.apache.org/jira/browse/TEZ-2584 Project: Apache Tez

[jira] [Updated] (TEZ-2584) Investigate: Reducers with local errors can run indefinitely

[ https://issues.apache.org/jira/browse/TEZ-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2584: -- Target Version/s: 0.8.0 Investigate: Reducers with local errors can run indefinitely

[jira] [Commented] (TEZ-2574) Make a better Metadata Value split choice in Pipeline sort

[ https://issues.apache.org/jira/browse/TEZ-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605071#comment-14605071 ] Rajesh Balamohan commented on TEZ-2574: --- Thanks for uploading the patch [~saikatr]

[jira] [Commented] (TEZ-1314) Port MAPREDUCE-5821 to Tez

[ https://issues.apache.org/jira/browse/TEZ-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14602084#comment-14602084 ] Rajesh Balamohan commented on TEZ-1314: --- committed to branch-0.7 as well. commit

[jira] [Updated] (TEZ-1314) Port MAPREDUCE-5821 to Tez

[ https://issues.apache.org/jira/browse/TEZ-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1314: -- Target Version/s: 0.8.0, 0.7.1 (was: 0.8.0) Fix Version/s: 0.7.1 Port MAPREDUCE-5821 to

[jira] [Updated] (TEZ-2565) Consider scanning unfinished tasks in VertexImpl::constructStatistics to reduce merge overhead

[ https://issues.apache.org/jira/browse/TEZ-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2565: -- Attachment: (was: TEZ-2565.3.patch) Consider scanning unfinished tasks in

[jira] [Updated] (TEZ-2565) Consider scanning unfinished tasks in VertexImpl::constructStatistics to reduce merge overhead

[ https://issues.apache.org/jira/browse/TEZ-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2565: -- Attachment: TEZ-2565.3.patch Consider scanning unfinished tasks in

[jira] [Updated] (TEZ-1314) Port MAPREDUCE-5821 to Tez

[ https://issues.apache.org/jira/browse/TEZ-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1314: -- Summary: Port MAPREDUCE-5821 to Tez (was: Investigate MAPREDUCE-5821 import to Tez) Port

[jira] [Updated] (TEZ-2565) Consider scanning unfinished tasks in VertexImpl::constructStatistics to reduce merge overhead

[ https://issues.apache.org/jira/browse/TEZ-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2565: -- Attachment: mem_usage_without_patch.png mem_usage_with_patch.png

[jira] [Commented] (TEZ-2378) In case Fetcher (unordered) fails to do local fetch, log in debug mode to reduce log size

[ https://issues.apache.org/jira/browse/TEZ-2378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599199#comment-14599199 ] Rajesh Balamohan commented on TEZ-2378: --- Pasting ./build-tools/test-patch.sh output

[jira] [Commented] (TEZ-2575) Handle KeyValue pairs size which donot fit in a single block

[ https://issues.apache.org/jira/browse/TEZ-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600449#comment-14600449 ] Rajesh Balamohan commented on TEZ-2575: --- In regular of PipelinedSorter, requested

[jira] [Commented] (TEZ-2575) Handle KeyValue pairs size which donot fit in a single block

[ https://issues.apache.org/jira/browse/TEZ-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600649#comment-14600649 ] Rajesh Balamohan commented on TEZ-2575: --- That would be a good addition. Currently,

[jira] [Comment Edited] (TEZ-2378) In case Fetcher (unordered) fails to do local fetch, log in debug mode to reduce log size

[ https://issues.apache.org/jira/browse/TEZ-2378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14598819#comment-14598819 ] Rajesh Balamohan edited comment on TEZ-2378 at 6/24/15 3:41 AM:

[jira] [Updated] (TEZ-2378) In case Fetcher (unordered) fails to do local fetch, log in debug mode to reduce log size

[ https://issues.apache.org/jira/browse/TEZ-2378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2378: -- Attachment: TEZ-2378.1.patch - For broadcast, fetcher tries to download and cache it locally when

[jira] [Assigned] (TEZ-2378) In case Fetcher (unordered) fails to do local fetch, log in debug mode to reduce log size

[ https://issues.apache.org/jira/browse/TEZ-2378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-2378: - Assignee: Rajesh Balamohan In case Fetcher (unordered) fails to do local fetch, log in

[jira] [Assigned] (TEZ-1314) Investigate MAPREDUCE-5821 import to Tez

[ https://issues.apache.org/jira/browse/TEZ-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-1314: - Assignee: Rajesh Balamohan Investigate MAPREDUCE-5821 import to Tez

[jira] [Updated] (TEZ-1314) Investigate MAPREDUCE-5821 import to Tez

[ https://issues.apache.org/jira/browse/TEZ-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1314: -- Attachment: TEZ-1314.1.patch Unordered case is not valid anymore. Attaching patch with simple

[jira] [Resolved] (TEZ-2571) TEZ-UI: Some vertice's stats show up as undefined in graphical view page

[ https://issues.apache.org/jira/browse/TEZ-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-2571. --- Resolution: Invalid Checked it with recent version and works fine. Closing it as a invalid.

[jira] [Commented] (TEZ-2565) Consider scanning unfinished tasks in VertexImpl::constructStatistics to reduce merge overhead

[ https://issues.apache.org/jira/browse/TEZ-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14597182#comment-14597182 ] Rajesh Balamohan commented on TEZ-2565: --- - clear()/setupIOStats() can be removed and

[jira] [Commented] (TEZ-2565) Consider scanning unfinished tasks in VertexImpl::constructStatistics to reduce merge overhead

[ https://issues.apache.org/jira/browse/TEZ-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595254#comment-14595254 ] Rajesh Balamohan commented on TEZ-2565: --- Nit: final SetTezTaskID taskSet; - this

[jira] [Updated] (TEZ-2565) Consider scanning unfinished tasks in VertexImpl::constructStatistics to reduce merge overhead

[ https://issues.apache.org/jira/browse/TEZ-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2565: -- Attachment: TEZ-2565.2.patch Consider scanning unfinished tasks in

[jira] [Comment Edited] (TEZ-2565) Consider scanning unfinished tasks in VertexImpl::constructStatistics to reduce merge overhead

[ https://issues.apache.org/jira/browse/TEZ-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595254#comment-14595254 ] Rajesh Balamohan edited comment on TEZ-2565 at 6/22/15 12:16 AM:

[jira] [Commented] (TEZ-2555) TestHttpConnection test fails

[ https://issues.apache.org/jira/browse/TEZ-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590797#comment-14590797 ] Rajesh Balamohan commented on TEZ-2555: --- [~mitdesai], [~saikatr] - Connection is not

[jira] [Created] (TEZ-2565) Consider scanning unfinished tasks in VertexImpl::constructStatistics to reduce merge overhead

Rajesh Balamohan created TEZ-2565: - Summary: Consider scanning unfinished tasks in VertexImpl::constructStatistics to reduce merge overhead Key: TEZ-2565 URL: https://issues.apache.org/jira/browse/TEZ-2565

[jira] [Updated] (TEZ-2565) Consider scanning unfinished tasks in VertexImpl::constructStatistics to reduce merge overhead

[ https://issues.apache.org/jira/browse/TEZ-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2565: -- Issue Type: Improvement (was: Bug) Consider scanning unfinished tasks in

[jira] [Updated] (TEZ-2552) CRC errors can cause job to run for very long time in large jobs

[ https://issues.apache.org/jira/browse/TEZ-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2552: -- Attachment: TEZ-2552.3.patch Resubmitting the patch. Since the conf value was changed to 0.1 in

[jira] [Updated] (TEZ-2565) Consider scanning unfinished tasks in VertexImpl::constructStatistics to reduce merge overhead

[ https://issues.apache.org/jira/browse/TEZ-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2565: -- Attachment: TEZ-2565.1.patch [~bikassaha] - Can you please have a look at this when you find

[jira] [Assigned] (TEZ-2565) Consider scanning unfinished tasks in VertexImpl::constructStatistics to reduce merge overhead

[ https://issues.apache.org/jira/browse/TEZ-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-2565: - Assignee: Rajesh Balamohan Consider scanning unfinished tasks in

[jira] [Created] (TEZ-2559) Compilation issue in tez master

Rajesh Balamohan created TEZ-2559: - Summary: Compilation issue in tez master Key: TEZ-2559 URL: https://issues.apache.org/jira/browse/TEZ-2559 Project: Apache Tez Issue Type: Bug

[jira] [Updated] (TEZ-2552) CRC errors can cause job to run for very long time in large jobs

[ https://issues.apache.org/jira/browse/TEZ-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2552: -- Attachment: TEZ-2552.2.patch Thanks [~bikassaha]. Attaching the patch with review comments

[jira] [Commented] (TEZ-2496) Consider scheduling tasks in ShuffleVertexManager based on the partition sizes from the source

[ https://issues.apache.org/jira/browse/TEZ-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14589272#comment-14589272 ] Rajesh Balamohan commented on TEZ-2496: --- Review board link:

[jira] [Updated] (TEZ-2496) Consider scheduling tasks in ShuffleVertexManager based on the partition sizes from the source

[ https://issues.apache.org/jira/browse/TEZ-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2496: -- Attachment: TEZ-2496.4.patch Addressing [~sseth]'s comments In JavaDocs - replace destination

<    5   6   7   8   9   10   11   12   13   14   >