[jira] [Updated] (TEZ-4088) Create in-memory ifile writer for transferring smaller payloads (follow up of TEZ-4075)

2019-10-07 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4088: -- Attachment: TEZ-4088.1.patch > Create in-memory ifile writer for transferring smaller payloads

[jira] [Commented] (TEZ-4088) Create in-memory ifile writer for transferring smaller payloads (follow up of TEZ-4075)

2019-10-07 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945765#comment-16945765 ] Rajesh Balamohan commented on TEZ-4088: --- Initial patch. Will create a PR after ptests. > Create

[jira] [Updated] (TEZ-4088) Create in-memory ifile writer for transferring smaller payloads (follow up of TEZ-4075)

2019-10-07 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4088: -- Attachment: TEZ-4088.1.patch > Create in-memory ifile writer for transferring smaller payloads

[jira] [Commented] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-10-07 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945636#comment-16945636 ] Rajesh Balamohan commented on TEZ-4075: --- Merged in master.  Created

[jira] [Created] (TEZ-4088) Create in-memory ifile writer for transferring smaller payloads (follow up of TEZ-4075)

2019-10-07 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4088: - Summary: Create in-memory ifile writer for transferring smaller payloads (follow up of TEZ-4075) Key: TEZ-4088 URL: https://issues.apache.org/jira/browse/TEZ-4088

[jira] [Commented] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-10-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945590#comment-16945590 ] Rajesh Balamohan commented on TEZ-4075: --- Attached TEZ-4075.enable-dme.16 (PR:

[jira] [Updated] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-10-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4075: -- Attachment: TEZ-4075.enable-dme.16.patch > Tez: Reimplement

[jira] [Commented] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-10-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945581#comment-16945581 ] Rajesh Balamohan commented on TEZ-4075: --- Thanks [~sseth] . I had a miscalculation in the checksum

[jira] [Updated] (TEZ-4087) Shuffle: Check for thread's liveliness regularly to avoid infinite wait in merger & referee threads

2019-10-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4087: -- Description: In certain cases, Shuffle's cleanupIgnoreErrors() is not called. This leaves 4

[jira] [Updated] (TEZ-4087) Shuffle: Check for thread's liveliness regularly to avoid infinite wait in merger & referee threads

2019-10-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4087: -- Description: In certain cases, Shuffle's cleanupIgnoreErrors() is not called. This leaves 4

[jira] [Created] (TEZ-4087) Shuffle: Check for thread's liveliness regularly to avoid infinite wait in merger & referee threads

2019-10-06 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4087: - Summary: Shuffle: Check for thread's liveliness regularly to avoid infinite wait in merger & referee threads Key: TEZ-4087 URL: https://issues.apache.org/jira/browse/TEZ-4087

[jira] [Commented] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-10-04 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944252#comment-16944252 ] Rajesh Balamohan commented on TEZ-4075: --- Thanks [~sseth] for the review comments. Closing the

[jira] [Commented] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-10-03 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943376#comment-16943376 ] Rajesh Balamohan commented on TEZ-4075: --- Thanks [~sseth] . I will post the revised patch on github

[jira] [Commented] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-30 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941022#comment-16941022 ] Rajesh Balamohan commented on TEZ-4075: --- Thanks for the review [~sseth] . I will fix a corner case

[jira] [Commented] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-26 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938315#comment-16938315 ] Rajesh Balamohan commented on TEZ-4075: --- Attached {{16.patch}} version to fix test case error in

[jira] [Updated] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-26 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4075: -- Attachment: TEZ-4075.16.patch > Tez: Reimplement tez.runtime.transfer.data-via-events.enabled >

[jira] [Comment Edited] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-25 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938265#comment-16938265 ] Rajesh Balamohan edited comment on TEZ-4075 at 9/26/19 4:43 AM: Additional

[jira] [Comment Edited] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-25 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938265#comment-16938265 ] Rajesh Balamohan edited comment on TEZ-4075 at 9/26/19 4:42 AM: Additional

[jira] [Comment Edited] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-25 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938265#comment-16938265 ] Rajesh Balamohan edited comment on TEZ-4075 at 9/26/19 4:42 AM: Changes:

[jira] [Commented] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-25 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938265#comment-16938265 ] Rajesh Balamohan commented on TEZ-4075: --- Changes: 1. Extended IFile to have filebacked in-mem

[jira] [Comment Edited] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-25 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938265#comment-16938265 ] Rajesh Balamohan edited comment on TEZ-4075 at 9/26/19 4:41 AM: Changes:

[jira] [Updated] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-25 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4075: -- Attachment: TEZ-4075.15.patch > Tez: Reimplement tez.runtime.transfer.data-via-events.enabled >

[jira] [Comment Edited] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-19 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933951#comment-16933951 ] Rajesh Balamohan edited comment on TEZ-4075 at 9/20/19 2:38 AM: Attaching 

[jira] [Comment Edited] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-19 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933951#comment-16933951 ] Rajesh Balamohan edited comment on TEZ-4075 at 9/20/19 2:19 AM: Attaching 

[jira] [Updated] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-19 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4075: -- Attachment: TEZ-4075.10.patch > Tez: Reimplement tez.runtime.transfer.data-via-events.enabled >

[jira] [Commented] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-19 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933951#comment-16933951 ] Rajesh Balamohan commented on TEZ-4075: --- Attaching {{TEZ-4075.10.patch}}. This is based out of 0.8

[jira] [Comment Edited] (TEZ-4071) shuffle throws exceptions with an external table with multiple hdfs files

2019-09-18 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932615#comment-16932615 ] Rajesh Balamohan edited comment on TEZ-4071 at 9/18/19 4:24 PM: Is this

[jira] [Commented] (TEZ-4071) shuffle throws exceptions with an external table with multiple hdfs files

2019-09-18 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932615#comment-16932615 ] Rajesh Balamohan commented on TEZ-4071: --- Is this consistent? > shuffle throws exceptions with an

[jira] [Commented] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-09-04 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923056#comment-16923056 ] Rajesh Balamohan commented on TEZ-4075: --- Thanks a lot for sharing the patch [~rzhappy]. Currently

[jira] [Commented] (TEZ-4082) Reduce excessive getFileLinkInfo calls in Tez

2019-08-26 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916221#comment-16916221 ] Rajesh Balamohan commented on TEZ-4082: --- Thanks [~jeagles]. +1. > Reduce excessive getFileLinkInfo

[jira] [Commented] (TEZ-4082) Reduce excessive getFileLinkInfo calls in Tez

2019-08-26 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915591#comment-16915591 ] Rajesh Balamohan commented on TEZ-4082: --- Thanks for sharing the patch [~jeagles] . Would the change

[jira] [Commented] (TEZ-4035) Tez master breaks with YARN 3.2.0

2019-02-19 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16771765#comment-16771765 ] Rajesh Balamohan commented on TEZ-4035: --- Thanks for sharing the patch [~jeagles]. Patch works fine

[jira] [Created] (TEZ-4035) Tez master breaks with YARN 3.2.0

2019-02-01 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-4035: - Summary: Tez master breaks with YARN 3.2.0 Key: TEZ-4035 URL: https://issues.apache.org/jira/browse/TEZ-4035 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-4013) Allow intermediate output/spill data encryption

2018-10-23 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660139#comment-16660139 ] Rajesh Balamohan commented on TEZ-4013: --- \cc [~cnauroth] > Allow intermediate output/spill data

[jira] [Updated] (TEZ-3953) Restore ABI-compat for DAGClient for TEZ-3951

2018-06-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3953: -- Summary: Restore ABI-compat for DAGClient for TEZ-3951 (was: Restore API-compat for DAGClient

[jira] [Updated] (TEZ-3953) Restore API-compat for DAGClient for TEZ-3951

2018-06-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3953: -- Summary: Restore API-compat for DAGClient for TEZ-3951 (was: Restore ABI-compat for DAGClient

[jira] [Commented] (TEZ-3924) TestDefaultSorter fails intermittently due random keys and interaction with RLE and partition collisions

2018-05-01 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460534#comment-16460534 ] Rajesh Balamohan commented on TEZ-3924: --- >> For this scenario, DefaultSorter appends key values and

[jira] [Updated] (TEZ-3869) Analyzer: Fix VertexInfo::getLastTaskToFinish comparison

2017-12-03 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3869: -- Fix Version/s: 0.9.next > Analyzer: Fix VertexInfo::getLastTaskToFinish comparison >

[jira] [Assigned] (TEZ-3870) Analyzer: Add script to run all analyzers

2017-11-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-3870: - Assignee: Rajesh Balamohan > Analyzer: Add script to run all analyzers >

[jira] [Created] (TEZ-3870) Analyzer: Add script to run all analyzers

2017-11-29 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3870: - Summary: Analyzer: Add script to run all analyzers Key: TEZ-3870 URL: https://issues.apache.org/jira/browse/TEZ-3870 Project: Apache Tez Issue Type: Bug

[jira] [Updated] (TEZ-3869) Analyzer: Fix VertexInfo::getLastTaskToFinish comparison

2017-11-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3869: -- Attachment: TEZ-3869.2.patch > Analyzer: Fix VertexInfo::getLastTaskToFinish comparison >

[jira] [Updated] (TEZ-3869) Analyzer: Fix VertexInfo::getLastTaskToFinish comparison

2017-11-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3869: -- Attachment: TEZ-3869.1.patch [~gopalv] - Can you please review when you find time? > Analyzer:

[jira] [Created] (TEZ-3869) Analyzer: Fix VertexInfo::getLastTaskToFinish comparison

2017-11-29 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3869: - Summary: Analyzer: Fix VertexInfo::getLastTaskToFinish comparison Key: TEZ-3869 URL: https://issues.apache.org/jira/browse/TEZ-3869 Project: Apache Tez

[jira] [Commented] (TEZ-3805) Analyzer: Add an analyzer to find out scheduling misses in 1:1 edges

2017-11-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270638#comment-16270638 ] Rajesh Balamohan commented on TEZ-3805: --- In case of hive with LLAP, I observed misses for a specific

[jira] [Commented] (TEZ-3805) Analyzer: Add an analyzer to find out scheduling misses in 1:1 edges

2017-11-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267954#comment-16267954 ] Rajesh Balamohan commented on TEZ-3805: --- [~kshukla] - https://issues.apache.org/jira/browse/HIVE-17037

[jira] [Commented] (TEZ-3738) TestUnorderedPartitionedKVWriter fails due to RejectedExecutionException

2017-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226188#comment-16226188 ] Rajesh Balamohan commented on TEZ-3738: --- I do not think it is due to recent changes. But it is quite

[jira] [Updated] (TEZ-3861) PipelineSorter setting negative progess

2017-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3861: -- Attachment: TEZ-3861.2.patch > PipelineSorter setting negative progess >

[jira] [Assigned] (TEZ-3861) PipelineSorter setting negative progess

2017-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-3861: - Assignee: Rajesh Balamohan > PipelineSorter setting negative progess >

[jira] [Commented] (TEZ-3861) PipelineSorter setting negative progess

2017-10-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224525#comment-16224525 ] Rajesh Balamohan commented on TEZ-3861: --- Thanks for reporting this [~prasanth_j]. Do you have logs for

[jira] [Commented] (TEZ-3855) Allow vertex manager to send event to processor

2017-10-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16221967#comment-16221967 ] Rajesh Balamohan commented on TEZ-3855: --- In VertexManager::sendEventToProcessor, it always points to

[jira] [Commented] (TEZ-3809) The buffer size allocated for InMemoryMapOutput can be optimized

2017-08-06 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116030#comment-16116030 ] Rajesh Balamohan commented on TEZ-3809: --- Thanks for the patch [~samirkhan]. Patch looks good to me.

[jira] [Created] (TEZ-3808) ShuffleHandler: optimize fadvise calls for broadcast edge

2017-07-30 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3808: - Summary: ShuffleHandler: optimize fadvise calls for broadcast edge Key: TEZ-3808 URL: https://issues.apache.org/jira/browse/TEZ-3808 Project: Apache Tez

[jira] [Commented] (TEZ-3212) IFile throws NegativeArraySizeException for value sizes between 1GB and 2GB

2017-07-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105858#comment-16105858 ] Rajesh Balamohan commented on TEZ-3212: --- Thanks [~samirkhan], [~jeagles]. +1. No other comments from

[jira] [Commented] (TEZ-3212) IFile throws NegativeArraySizeException for value sizes between 1GB and 2GB

2017-07-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104018#comment-16104018 ] Rajesh Balamohan commented on TEZ-3212: --- Thanks for sharing the patch [~samirkhan]. Very minor

[jira] [Updated] (TEZ-3805) Analyzer: Add an analyzer to find out scheduling misses in 1:1 edges

2017-07-26 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3805: -- Attachment: TEZ-3805.1.patch {noformat} java -cp `hadoop classpath`:./*:

[jira] [Created] (TEZ-3805) Analyzer: Add an analyzer to find out scheduling misses in 1:1 edges

2017-07-26 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3805: - Summary: Analyzer: Add an analyzer to find out scheduling misses in 1:1 edges Key: TEZ-3805 URL: https://issues.apache.org/jira/browse/TEZ-3805 Project: Apache Tez

[jira] [Commented] (TEZ-3802) Tez Shuffle Handler TezSpillRecord Cache hit rate is very low

2017-07-20 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095782#comment-16095782 ] Rajesh Balamohan commented on TEZ-3802: --- Hive in LLAP has its own shuffle handler. HIVE-16740 is on

[jira] [Assigned] (TEZ-3794) tez-tools: swimlane does not recognize "HistoryEventHandler.criticalEvents" based logs

2017-07-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-3794: - Resolution: Fixed Assignee: Rajesh Balamohan Hadoop Flags: Reviewed

[jira] [Created] (TEZ-3794) tez-tools: swimlane does not recognize "HistoryEventHandler.criticalEvents" based logs

2017-07-12 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3794: - Summary: tez-tools: swimlane does not recognize "HistoryEventHandler.criticalEvents" based logs Key: TEZ-3794 URL: https://issues.apache.org/jira/browse/TEZ-3794

[jira] [Updated] (TEZ-3794) tez-tools: swimlane does not recognize "HistoryEventHandler.criticalEvents" based logs

2017-07-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3794: -- Description: TEZ-3581 moved the history logs to "HistoryEventHandler.criticalEvents".

[jira] [Updated] (TEZ-3794) tez-tools: swimlane does not recognize "HistoryEventHandler.criticalEvents" based logs

2017-07-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3794: -- Attachment: TEZ-3794.1.patch \cc [~gopalv] > tez-tools: swimlane does not recognize

[jira] [Updated] (TEZ-3794) tez-tools: swimlane does not recognize "HistoryEventHandler.criticalEvents" based logs

2017-07-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3794: -- Description: e.g {noformat} 2017-07-12 04:03:21,152 [INFO] [Dispatcher thread {Central}]

[jira] [Created] (TEZ-3793) Consider reducing the number of times "DiskChecker.doDiskIo" needs to be invoked for writing to local folders

2017-07-11 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3793: - Summary: Consider reducing the number of times "DiskChecker.doDiskIo" needs to be invoked for writing to local folders Key: TEZ-3793 URL:

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-07-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083231#comment-16083231 ] Rajesh Balamohan commented on TEZ-3754: --- This is not an infinite loop similar to the one which got

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-07-10 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081281#comment-16081281 ] Rajesh Balamohan commented on TEZ-3754: --- Agreed that this would block the cases which could have

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-07-10 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079940#comment-16079940 ] Rajesh Balamohan commented on TEZ-3754: --- Thanks for the review [~aplusplus]. Pinging the host in

[jira] [Created] (TEZ-3789) Consider avoiding buffer copies in TezMerger when lots of unique keys are present in reducer side

2017-07-09 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3789: - Summary: Consider avoiding buffer copies in TezMerger when lots of unique keys are present in reducer side Key: TEZ-3789 URL: https://issues.apache.org/jira/browse/TEZ-3789

[jira] [Updated] (TEZ-3784) Submitting very large DAG throws com.google.protobuf.CodedInputStream exception

2017-07-02 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3784: -- Attachment: TEZ-3784.1.patch Attaching .1 patch. > Submitting very large DAG throws

[jira] [Commented] (TEZ-3784) Submitting very large DAG throws com.google.protobuf.CodedInputStream exception

2017-07-02 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071820#comment-16071820 ] Rajesh Balamohan commented on TEZ-3784: --- Appears that hive has hit this issue as well HIVE-11592. >

[jira] [Created] (TEZ-3784) Submitting very large DAG throws com.google.protobuf.CodedInputStream exception

2017-07-02 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3784: - Summary: Submitting very large DAG throws com.google.protobuf.CodedInputStream exception Key: TEZ-3784 URL: https://issues.apache.org/jira/browse/TEZ-3784 Project:

[jira] [Commented] (TEZ-3754) With large ordered fetches, stalling shuffle could lead to slowdown of job

2017-06-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068038#comment-16068038 ] Rajesh Balamohan commented on TEZ-3754: --- [~sseth], [~jeagles], [~aplusplus] - Can you plz review when

[jira] [Commented] (TEZ-3769) Unordered: Fix wrong stats being sent out in the last event, when final merge is disabled

2017-06-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066109#comment-16066109 ] Rajesh Balamohan commented on TEZ-3769: --- Thanks [~aplusplus]. Committed to master. > Unordered: Fix

[jira] [Commented] (TEZ-3769) Unordered: Fix wrong stats being sent out in the last event, when final merge is disabled

2017-06-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066094#comment-16066094 ] Rajesh Balamohan commented on TEZ-3769: --- Created TEZ-3780 for #1. Thanks for the review

[jira] [Created] (TEZ-3780) Fix ADDITIONAL_SPILLS_BYTES_WRITTEN counter in unordered case

2017-06-28 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3780: - Summary: Fix ADDITIONAL_SPILLS_BYTES_WRITTEN counter in unordered case Key: TEZ-3780 URL: https://issues.apache.org/jira/browse/TEZ-3780 Project: Apache Tez

[jira] [Assigned] (TEZ-3769) Unordered: Fix wrong stats being sent out in the last event, when final merge is disabled

2017-06-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-3769: - Assignee: Rajesh Balamohan > Unordered: Fix wrong stats being sent out in the last event,

[jira] [Updated] (TEZ-3769) Unordered: Fix wrong stats being sent out in the last event, when final merge is disabled

2017-06-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3769: -- Attachment: TEZ-3769.3.patch Uploading .3 with review comments addressed. Agreed that

[jira] [Commented] (TEZ-3767) Shuffle should not report error to AM during inputContext.killSelf()

2017-06-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065743#comment-16065743 ] Rajesh Balamohan commented on TEZ-3767: --- Thanks for the review [~sseth]. Test failure earlier was not

[jira] [Commented] (TEZ-3777) Avoid buffer copies by passing RLE flag to TezMerger from PipelinedSorter

2017-06-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065662#comment-16065662 ] Rajesh Balamohan commented on TEZ-3777: --- \cc [~gopalv] > Avoid buffer copies by passing RLE flag to

[jira] [Updated] (TEZ-3777) Avoid buffer copies by passing RLE flag to TezMerger from PipelinedSorter

2017-06-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3777: -- Attachment: TEZ-3777.2.patch > Avoid buffer copies by passing RLE flag to TezMerger from

[jira] [Updated] (TEZ-3767) Shuffle should not report error to AM during inputContext.killSelf()

2017-06-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3767: -- Attachment: TEZ-3767.3.patch Uploading .3 patch; addressing review comments. > Shuffle should

[jira] [Updated] (TEZ-3777) Avoid buffer copies by passing RLE flag to TezMerger from PipelinedSorter

2017-06-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3777: -- Attachment: TEZ-3777.1.patch > Avoid buffer copies by passing RLE flag to TezMerger from

[jira] [Assigned] (TEZ-3777) Avoid buffer copies by passing RLE flag to TezMerger from PipelinedSorter

2017-06-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-3777: - Assignee: Rajesh Balamohan > Avoid buffer copies by passing RLE flag to TezMerger from

[jira] [Created] (TEZ-3777) Avoid buffer copies by passing RLE flag to TezMerger from PipelinedSorter

2017-06-27 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3777: - Summary: Avoid buffer copies by passing RLE flag to TezMerger from PipelinedSorter Key: TEZ-3777 URL: https://issues.apache.org/jira/browse/TEZ-3777 Project:

[jira] [Updated] (TEZ-3769) Unordered: Fix wrong stats being sent out in the last event, when final merge is disabled

2017-06-26 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3769: -- Attachment: TEZ-3769.2.patch Rebasing the patch (TEZ-3762 got committed) and added few more

[jira] [Commented] (TEZ-3762) When final merge is disabled in unordered case, it should create index file instead of relying on cache

2017-06-26 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063885#comment-16063885 ] Rajesh Balamohan commented on TEZ-3762: --- lgtm. +1. Thanks [~aplusplus] > When final merge is disabled

[jira] [Commented] (TEZ-3774) hive use tez engine occur org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0)

2017-06-26 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062997#comment-16062997 ] Rajesh Balamohan commented on TEZ-3774: --- This is more of a hive problem than Tez issue. Can you plz

[jira] [Updated] (TEZ-3767) Shuffle should not report error to AM during inputContext.killSelf()

2017-06-22 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3767: -- Attachment: TEZ-3767.2.patch Thanks [~sseth]. Attaching revised patch. > Shuffle should not

[jira] [Updated] (TEZ-3769) Unordered: Fix wrong stats being sent out in the last event, when final merge is disabled

2017-06-22 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3769: -- Attachment: TEZ-3769.1.patch [~sseth], [~aplusplus], [~harishjp], [~jeagles] - Please review when

[jira] [Created] (TEZ-3769) Unordered: Fix wrong stats being sent out in the last event when final merge is disabled

2017-06-22 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3769: - Summary: Unordered: Fix wrong stats being sent out in the last event when final merge is disabled Key: TEZ-3769 URL: https://issues.apache.org/jira/browse/TEZ-3769

[jira] [Updated] (TEZ-3769) Unordered: Fix wrong stats being sent out in the last event, when final merge is disabled

2017-06-22 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3769: -- Summary: Unordered: Fix wrong stats being sent out in the last event, when final merge is

[jira] [Commented] (TEZ-3761) NPE in Fetcher under load

2017-06-22 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058953#comment-16058953 ] Rajesh Balamohan commented on TEZ-3761: --- Thanks for sharing the patch [~jeagles]. Would be good to add

[jira] [Commented] (TEZ-3761) NPE in Fetcher under load

2017-06-19 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16053518#comment-16053518 ] Rajesh Balamohan commented on TEZ-3761: --- This could be due to a impact of recent checkin in for

[jira] [Commented] (TEZ-3761) NPE in Fetcher under load

2017-06-18 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1605#comment-1605 ] Rajesh Balamohan commented on TEZ-3761: --- Sure. I will add this patch and try it out. > NPE in Fetcher

[jira] [Commented] (TEZ-3762) When final merge is disabled in unordered case, it should create index file instead of relying on cache

2017-06-16 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16051801#comment-16051801 ] Rajesh Balamohan commented on TEZ-3762: --- Thanks for sharing the patch [~aplusplus]. Patch lgtm.

[jira] [Created] (TEZ-3762) When final merge is disabled in unordered case, it should create index file instead of relying on cache

2017-06-14 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3762: - Summary: When final merge is disabled in unordered case, it should create index file instead of relying on cache Key: TEZ-3762 URL:

[jira] [Created] (TEZ-3761) NPE in Fetcher under load

2017-06-13 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3761: - Summary: NPE in Fetcher under load Key: TEZ-3761 URL: https://issues.apache.org/jira/browse/TEZ-3761 Project: Apache Tez Issue Type: Bug

[jira] [Updated] (TEZ-3757) Integer overflow in PipelinedSorter

2017-06-13 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3757: -- Priority: Critical (was: Major) > Integer overflow in PipelinedSorter >

[jira] [Updated] (TEZ-3698) UnorderedKV writer should be able to honor tez.runtime.enable.final-merge.in.output without pipelinedshuffle

2017-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3698: -- Attachment: TEZ-3698.6.patch Thanks [~sseth] for the review. Fixed minor spelling mistake in

[jira] [Comment Edited] (TEZ-3757) Integer overflow in PipelinedSorter

2017-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047190#comment-16047190 ] Rajesh Balamohan edited comment on TEZ-3757 at 6/12/17 11:07 PM: - Thanks for

[jira] [Comment Edited] (TEZ-3757) Integer overflow in PipelinedSorter

2017-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047190#comment-16047190 ] Rajesh Balamohan edited comment on TEZ-3757 at 6/12/17 11:06 PM: - Thanks for

<    1   2   3   4   5   6   7   8   9   10   >