[jira] [Commented] (TEZ-4518) Limit number of spill files getting created

2023-10-14 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775145#comment-17775145 ] Rajesh Balamohan commented on TEZ-4518: --- Number of spills can be different between DefaultSorter and

[jira] [Commented] (TEZ-4518) Limit number of spill files getting created

2023-10-13 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774809#comment-17774809 ] Rajesh Balamohan commented on TEZ-4518: --- There can be spills in reducer side in merger. IAC, I was

[jira] [Commented] (TEZ-4518) Limit number of spill files getting created

2023-10-08 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773062#comment-17773062 ] Rajesh Balamohan commented on TEZ-4518: --- Spills are there in multiple places including merging in

[jira] [Comment Edited] (TEZ-4404) Hive on tez report IndexOutofBoundary Exception when query hive external table

2022-07-10 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564698#comment-17564698 ] Rajesh Balamohan edited comment on TEZ-4404 at 7/10/22 3:25 PM: Stacktrace

[jira] [Commented] (TEZ-4404) Hive on tez report IndexOutofBoundary Exception when query hive external table

2022-07-10 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564698#comment-17564698 ] Rajesh Balamohan commented on TEZ-4404: --- Stacktrace points to give codebase. You may have to move

[jira] [Resolved] (TEZ-4433) Login is not working with correct user name and password

2022-07-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-4433. --- Resolution: Invalid > Login is not working with correct user name and password >

[jira] [Resolved] (TEZ-4406) Wrong FS Exception when warehouse and scratchdir are on different FS

2022-04-25 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-4406. --- Fix Version/s: 0.10.2 Resolution: Fixed Thanks for the contribution

[jira] [Resolved] (TEZ-4397) Open Tez Input splits asynchronously

2022-03-23 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-4397. --- Fix Version/s: 0.10.2 Resolution: Fixed > Open Tez Input splits asynchronously >

[jira] [Commented] (TEZ-4142) TezUtils.createConfFromByteString on Configuration larger than 32MB throws com.google.protobuf.CodedInputStream exception

2021-11-29 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450873#comment-17450873 ] Rajesh Balamohan commented on TEZ-4142: --- setSizeLimit will just adjust the threshold and not allocate

[jira] [Commented] (TEZ-4139) Tez should consider node information for computing failure fraction

2021-08-29 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406479#comment-17406479 ] Rajesh Balamohan commented on TEZ-4139: --- >> can these changes go into the same patch? Sure, link both

[jira] [Commented] (TEZ-4245) Optimise split grouping when locality information is set to null/empty

2021-04-24 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17331241#comment-17331241 ] Rajesh Balamohan commented on TEZ-4245: --- Not yet [~jeagles] . Need to think through more corner cases

[jira] [Commented] (TEZ-4250) Optimise TaskImpl::getCounters

2021-02-24 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290595#comment-17290595 ] Rajesh Balamohan commented on TEZ-4250: --- [~maheshk114], Issue is that,

[jira] [Updated] (TEZ-4296) Consider using listStatusIterator instead of listStatus in DatePartitionedLogger

2021-02-21 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4296: -- Description: DatePartitionedLogger should make use of {{listStatusIterator}} instead of

[jira] [Created] (TEZ-4296) Consider using listStatusIterator instead of listStatus in DatePartitionedLogger

2021-02-21 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4296: - Summary: Consider using listStatusIterator instead of listStatus in DatePartitionedLogger Key: TEZ-4296 URL: https://issues.apache.org/jira/browse/TEZ-4296

[jira] [Commented] (TEZ-4250) Optimise TaskImpl::getCounters

2021-02-18 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286800#comment-17286800 ] Rajesh Balamohan commented on TEZ-4250: --- [~ashutoshc]: Need to check if TestVertexImpl failure is

[jira] [Updated] (TEZ-4250) Optimise TaskImpl::getCounters

2021-02-18 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4250: -- Attachment: (was: TEZ-4250.02.patch) > Optimise TaskImpl::getCounters >

[jira] [Updated] (TEZ-4250) Optimise TaskImpl::getCounters

2021-02-18 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4250: -- Attachment: TEZ-4250.02.patch > Optimise TaskImpl::getCounters >

[jira] [Commented] (TEZ-4281) dag_*_priority.dot files should go into a valid log directory

2021-02-07 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280687#comment-17280687 ] Rajesh Balamohan commented on TEZ-4281: --- +1. Can you exclude set/getLogDirs findbugs before commit?

[jira] [Commented] (TEZ-4273) Clear off staging files when TezYarnClient is unable to submit applications

2021-02-03 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17277876#comment-17277876 ] Rajesh Balamohan commented on TEZ-4273: --- Thanks [~abstractdog]. resources is cleared up via Hive.

[jira] [Commented] (TEZ-4273) Clear off staging files when TezYarnClient is unable to submit applications

2021-02-02 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17277539#comment-17277539 ] Rajesh Balamohan commented on TEZ-4273: --- Sure. Thanks [~abstractdog]. > Clear off staging files when

[jira] [Updated] (TEZ-4273) Clear off staging files when TezYarnClient is unable to submit applications

2021-01-27 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4273: -- Attachment: TEZ-4273.1.patch > Clear off staging files when TezYarnClient is unable to submit

[jira] [Comment Edited] (TEZ-4273) Clear off staging files when TezYarnClient is unable to submit applications

2021-01-27 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17273262#comment-17273262 ] Rajesh Balamohan edited comment on TEZ-4273 at 1/28/21, 4:00 AM: - Lines of

[jira] [Comment Edited] (TEZ-4273) Clear off staging files when TezYarnClient is unable to submit applications

2021-01-27 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17273262#comment-17273262 ] Rajesh Balamohan edited comment on TEZ-4273 at 1/28/21, 3:59 AM: - Lines of

[jira] [Commented] (TEZ-4273) Clear off staging files when TezYarnClient is unable to submit applications

2021-01-27 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17273262#comment-17273262 ] Rajesh Balamohan commented on TEZ-4273: --- Lines of interest:

[jira] [Created] (TEZ-4273) Clear off staging files when TezYarnClient is unable to submit applications

2021-01-27 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4273: - Summary: Clear off staging files when TezYarnClient is unable to submit applications Key: TEZ-4273 URL: https://issues.apache.org/jira/browse/TEZ-4273 Project:

[jira] [Comment Edited] (TEZ-3985) Correctness: Throw a clear exception for DMEs sent during cleanup

2021-01-27 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17273204#comment-17273204 ] Rajesh Balamohan edited comment on TEZ-3985 at 1/27/21, 11:53 PM: --

[jira] [Commented] (TEZ-4271) Add config to limit desiredNumSplits

2021-01-26 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17272469#comment-17272469 ] Rajesh Balamohan commented on TEZ-4271: --- Hi [~amagyar]: yes, from hive side. > Add config to limit

[jira] [Commented] (TEZ-4271) Add config to limit desiredNumSplits

2021-01-26 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17272068#comment-17272068 ] Rajesh Balamohan commented on TEZ-4271: --- "tez.grouping.by-count" is kind of experimental in nature

[jira] [Updated] (TEZ-3985) Correctness: Throw a clear exception for DMEs sent during cleanup

2021-01-26 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3985: -- Attachment: TEZ-3985.7.patch > Correctness: Throw a clear exception for DMEs sent during cleanup

[jira] [Updated] (TEZ-3985) Correctness: Throw a clear exception for DMEs sent during cleanup

2021-01-25 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3985: -- Attachment: TEZ-3985.6.patch > Correctness: Throw a clear exception for DMEs sent during cleanup

[jira] [Commented] (TEZ-3985) Correctness: Throw a clear exception for DMEs sent during cleanup

2021-01-25 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17271889#comment-17271889 ] Rajesh Balamohan commented on TEZ-3985: --- Uploaded 0.5 version. > Correctness: Throw a clear

[jira] [Updated] (TEZ-3985) Correctness: Throw a clear exception for DMEs sent during cleanup

2021-01-25 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3985: -- Attachment: TEZ-3985.5.patch > Correctness: Throw a clear exception for DMEs sent during cleanup

[jira] [Commented] (TEZ-3985) Correctness: Throw a clear exception for DMEs sent during cleanup

2021-01-25 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17271886#comment-17271886 ] Rajesh Balamohan commented on TEZ-3985: --- 0.4 patch would need a rebase. EventHandler is missing in

[jira] [Commented] (TEZ-4271) Add config to limit desiredNumSplits

2021-01-25 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17271883#comment-17271883 ] Rajesh Balamohan commented on TEZ-4271: --- {{tez.grouping.split-count}} mainly helps in initializing

[jira] [Commented] (TEZ-4254) Don't unset the tez config if both mr and tez config have same value.

2021-01-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259660#comment-17259660 ] Rajesh Balamohan commented on TEZ-4254: --- Had offline conversation with [~mustafaiman] . If config has

[jira] [Commented] (TEZ-4254) Don't unset the tez config if both mr and tez config have same value.

2021-01-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259551#comment-17259551 ] Rajesh Balamohan commented on TEZ-4254: --- No, it need not. As long as "tez.runtime.shuffle.ssl.enable"

[jira] [Commented] (TEZ-4254) Don't unset the tez config if both mr and tez config have same value.

2021-01-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259509#comment-17259509 ] Rajesh Balamohan commented on TEZ-4254: --- [~mustafaiman] : There is some basic confusion on

[jira] [Commented] (TEZ-4254) Don't unset the tez config if both mr and tez config have same value.

2021-01-05 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259375#comment-17259375 ] Rajesh Balamohan commented on TEZ-4254: --- I am missing it. It should be the other way around in the

[jira] [Commented] (TEZ-4254) Don't unset the tez config if both mr and tez config have same value.

2021-01-05 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259277#comment-17259277 ] Rajesh Balamohan commented on TEZ-4254: --- Curious to know why "mapreduce.shuffle.ssl.enabled=true" is

[jira] [Updated] (TEZ-4256) Reduce key comparisons in reducer side

2020-12-07 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4256: -- Description: !Screenshot 2020-12-07 at 12.00.08 PM.png|width=1025,height=366!  

[jira] [Commented] (TEZ-4256) Reduce key comparisons in reducer side

2020-12-07 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245559#comment-17245559 ] Rajesh Balamohan commented on TEZ-4256: --- Thanks [~gopalv] , [~ashutoshc] > Reduce key comparisons in

[jira] [Commented] (TEZ-4256) Reduce key comparisons in reducer side

2020-12-07 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245185#comment-17245185 ] Rajesh Balamohan commented on TEZ-4256: --- Adding one more profiler screenshot for ref. !Screenshot

[jira] [Updated] (TEZ-4256) Reduce key comparisons in reducer side

2020-12-07 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4256: -- Attachment: Screenshot 2020-12-07 at 6.15.19 PM.png > Reduce key comparisons in reducer side >

[jira] [Updated] (TEZ-4256) Reduce key comparisons in reducer side

2020-12-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4256: -- Attachment: TEZ-4256.1.patch > Reduce key comparisons in reducer side >

[jira] [Assigned] (TEZ-4256) Reduce key comparisons in reducer side

2020-12-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-4256: - Assignee: Rajesh Balamohan > Reduce key comparisons in reducer side >

[jira] [Commented] (TEZ-4256) Reduce key comparisons in reducer side

2020-12-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245013#comment-17245013 ] Rajesh Balamohan commented on TEZ-4256: --- E.g: Q97 in TPCDS > Reduce key comparisons in reducer side

[jira] [Created] (TEZ-4256) Reduce key comparisons in reducer side

2020-12-06 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4256: - Summary: Reduce key comparisons in reducer side Key: TEZ-4256 URL: https://issues.apache.org/jira/browse/TEZ-4256 Project: Apache Tez Issue Type:

[jira] [Updated] (TEZ-4244) Consider using RawLocalFileSystem in LocalDiskFetchedInput

2020-12-01 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4244: -- Attachment: TEZ-4244.3.patch > Consider using RawLocalFileSystem in LocalDiskFetchedInput >

[jira] [Updated] (TEZ-4244) Consider using RawLocalFileSystem in LocalDiskFetchedInput

2020-11-30 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4244: -- Attachment: TEZ-4244.2.patch > Consider using RawLocalFileSystem in LocalDiskFetchedInput >

[jira] [Assigned] (TEZ-4244) Consider using RawLocalFileSystem in LocalDiskFetchedInput

2020-11-30 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-4244: - Assignee: Rajesh Balamohan > Consider using RawLocalFileSystem in LocalDiskFetchedInput >

[jira] [Updated] (TEZ-4244) Consider using RawLocalFileSystem in LocalDiskFetchedInput

2020-11-30 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4244: -- Attachment: TEZ-4244.1.patch > Consider using RawLocalFileSystem in LocalDiskFetchedInput >

[jira] [Commented] (TEZ-4250) Optimise TaskImpl::getCounters

2020-11-30 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241144#comment-17241144 ] Rajesh Balamohan commented on TEZ-4250: --- Thanks [~maheshk114] for sharing the patch. Changes LGTM;

[jira] [Commented] (TEZ-4251) Acquiring locks for getInputVertices and getOutputVertices is not consistent

2020-11-23 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237928#comment-17237928 ] Rajesh Balamohan commented on TEZ-4251: --- Committed to master. Thanks [~kkasa]. > Acquiring locks for

[jira] [Commented] (TEZ-4251) Acquiring locks for getInputVertices and getOutputVertices is not consistent

2020-11-19 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235347#comment-17235347 ] Rajesh Balamohan commented on TEZ-4251: --- +1. LGTM pending tests. > Acquiring locks for

[jira] [Created] (TEZ-4250) Optimise TaskImpl::getCounters

2020-11-17 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4250: - Summary: Optimise TaskImpl::getCounters Key: TEZ-4250 URL: https://issues.apache.org/jira/browse/TEZ-4250 Project: Apache Tez Issue Type: Improvement

[jira] [Commented] (TEZ-4246) Avoid uneven local disk usage for spills

2020-11-08 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228365#comment-17228365 ] Rajesh Balamohan commented on TEZ-4246: --- Can you share more details on this [~okumin] ? This should

[jira] [Commented] (TEZ-4245) Optimise split grouping when locality information is set to null/empty

2020-10-27 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221238#comment-17221238 ] Rajesh Balamohan commented on TEZ-4245: --- PR: [https://github.com/apache/tez/pull/78] > Optimise

[jira] [Updated] (TEZ-4245) Optimise split grouping when locality information is set to null/empty

2020-10-27 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4245: -- Attachment: TEZ-4245.1.patch > Optimise split grouping when locality information is set to

[jira] [Created] (TEZ-4245) Optimise split grouping when locality information is set to null/empty

2020-10-26 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4245: - Summary: Optimise split grouping when locality information is set to null/empty Key: TEZ-4245 URL: https://issues.apache.org/jira/browse/TEZ-4245 Project: Apache

[jira] [Created] (TEZ-4244) Consider using RawLocalFileSystem in LocalDiskFetchedInput

2020-10-21 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4244: - Summary: Consider using RawLocalFileSystem in LocalDiskFetchedInput Key: TEZ-4244 URL: https://issues.apache.org/jira/browse/TEZ-4244 Project: Apache Tez

[jira] [Commented] (TEZ-4234) Compressor can cause IllegalArgumentException in Buffer.limit where limit exceeds capacity

2020-10-05 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208010#comment-17208010 ] Rajesh Balamohan commented on TEZ-4234: --- LGTM. +1. Resetting the conf in the codec is the crux of

[jira] [Commented] (TEZ-4233) Map task should be blamed earlier for local fetch failures

2020-09-23 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17201142#comment-17201142 ] Rajesh Balamohan commented on TEZ-4233: --- LGTM. +1. > Map task should be blamed earlier for local

[jira] [Commented] (TEZ-4233) Map task should be blamed earlier for local fetch failures

2020-09-22 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17200433#comment-17200433 ] Rajesh Balamohan commented on TEZ-4233: --- Thanks [~abstractdog] for the revised patch. 1.

[jira] [Commented] (TEZ-4233) Map task should be blamed earlier for local fetch failures

2020-09-17 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197624#comment-17197624 ] Rajesh Balamohan commented on TEZ-4233: --- Thanks [~abstractdog]  for the patch. I went through the

[jira] [Assigned] (TEZ-4207) Provide approximate number of input records to be processed in UnorderedKVInput

2020-08-13 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-4207: - Fix Version/s: 0.10.1 Assignee: Rajesh Balamohan Resolution: Fixed >

[jira] [Commented] (TEZ-4207) Provide approximate number of input records to be processed in UnorderedKVInput

2020-08-13 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176856#comment-17176856 ] Rajesh Balamohan commented on TEZ-4207: --- Thanks for the review [~ashutoshc]. Committed to master. >

[jira] [Commented] (TEZ-4223) Adding new jars or resources after the first DAG runs does not work.

2020-08-12 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176731#comment-17176731 ] Rajesh Balamohan commented on TEZ-4223: --- LGTM. +1 > Adding new jars or resources after the first DAG

[jira] [Commented] (TEZ-4222) Sync issues during IFile::Writer init phase due to SerializationFactory

2020-08-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172762#comment-17172762 ] Rajesh Balamohan commented on TEZ-4222: --- Yes [~jeagles] , TEZ-3645 fixes the issue.  

[jira] [Updated] (TEZ-4222) Sync issues during IFile::Writer init phase due to SerializationFactory

2020-08-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4222: -- Attachment: image-2020-08-07-06-49-38-729.png > Sync issues during IFile::Writer init phase due

[jira] [Commented] (TEZ-4211) Optimise MergeManager final merge

2020-08-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172227#comment-17172227 ] Rajesh Balamohan commented on TEZ-4211: --- Thanks for the note [~abstractdog] . I had offline

[jira] [Created] (TEZ-4222) Sync issues during IFile::Writer init phase due to SerializationFactory

2020-08-06 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4222: - Summary: Sync issues during IFile::Writer init phase due to SerializationFactory Key: TEZ-4222 URL: https://issues.apache.org/jira/browse/TEZ-4222 Project: Apache

[jira] [Updated] (TEZ-4216) RLE check in MergeManager::finalMerge could be disabled

2020-08-04 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4216: -- Attachment: TEZ-4216.1.patch > RLE check in MergeManager::finalMerge could be disabled >

[jira] [Created] (TEZ-4216) RLE check in MergeManager::finalMerge could be disabled

2020-08-04 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4216: - Summary: RLE check in MergeManager::finalMerge could be disabled Key: TEZ-4216 URL: https://issues.apache.org/jira/browse/TEZ-4216 Project: Apache Tez

[jira] [Commented] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

2020-08-04 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170723#comment-17170723 ] Rajesh Balamohan commented on TEZ-4208: --- Attaching .2 patch with test case. > Pipelinesorter uses

[jira] [Updated] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

2020-08-04 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4208: -- Attachment: TEZ-4208.2.patch > Pipelinesorter uses single SortSpan after spill >

[jira] [Updated] (TEZ-4211) Optimise MergeManager final merge

2020-08-03 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4211: -- Attachment: TEZ-4211.2.patch > Optimise MergeManager final merge >

[jira] [Commented] (TEZ-4211) Optimise MergeManager final merge

2020-07-30 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167729#comment-17167729 ] Rajesh Balamohan commented on TEZ-4211: --- Attaching wip patch > Optimise MergeManager final merge >

[jira] [Updated] (TEZ-4211) Optimise MergeManager final merge

2020-07-30 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4211: -- Attachment: TEZ-4211.wip.patch > Optimise MergeManager final merge >

[jira] [Created] (TEZ-4211) Optimise MergeManager final merge

2020-07-30 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4211: - Summary: Optimise MergeManager final merge Key: TEZ-4211 URL: https://issues.apache.org/jira/browse/TEZ-4211 Project: Apache Tez Issue Type: Bug

[jira] [Resolved] (TEZ-4210) Use task counter information to compute keycount during hashtable loading

2020-07-29 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-4210. --- Resolution: Won't Fix > Use task counter information to compute keycount during hashtable

[jira] [Created] (TEZ-4210) Use task counter information to compute keycount during hashtable loading

2020-07-29 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4210: - Summary: Use task counter information to compute keycount during hashtable loading Key: TEZ-4210 URL: https://issues.apache.org/jira/browse/TEZ-4210 Project:

[jira] [Resolved] (TEZ-4209) Use task counter information to compute keycount during hashtable loading

2020-07-29 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-4209. --- Resolution: Won't Fix > Use task counter information to compute keycount during hashtable

[jira] [Commented] (TEZ-4209) Use task counter information to compute keycount during hashtable loading

2020-07-29 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167635#comment-17167635 ] Rajesh Balamohan commented on TEZ-4209: --- Supposed to be created in Hive project. Ignore this ticket.

[jira] [Created] (TEZ-4209) Use task counter information to compute keycount during hashtable loading

2020-07-29 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4209: - Summary: Use task counter information to compute keycount during hashtable loading Key: TEZ-4209 URL: https://issues.apache.org/jira/browse/TEZ-4209 Project:

[jira] [Updated] (TEZ-4207) Provide approximate number of input records to be processed in UnorderedKVInput

2020-07-29 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4207: -- Attachment: TEZ-4207.1.patch > Provide approximate number of input records to be processed in >

[jira] [Updated] (TEZ-4203) Findbugs: MergeThread.shuffleSchedulerThread; locked 80% of time

2020-07-28 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4203: -- Attachment: TEZ-4203.1.patch > Findbugs: MergeThread.shuffleSchedulerThread; locked 80% of time

[jira] [Comment Edited] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

2020-07-28 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166857#comment-17166857 ] Rajesh Balamohan edited comment on TEZ-4208 at 7/29/20, 3:59 AM: - Q67

[jira] [Commented] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

2020-07-28 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166857#comment-17166857 ] Rajesh Balamohan commented on TEZ-4208: --- Q67 runtime with/without patch in internal cluster @ 10 TB

[jira] [Updated] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

2020-07-28 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4208: -- Attachment: TEZ-4208.1.patch > Pipelinesorter uses single SortSpan after spill >

[jira] [Created] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

2020-07-28 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4208: - Summary: Pipelinesorter uses single SortSpan after spill Key: TEZ-4208 URL: https://issues.apache.org/jira/browse/TEZ-4208 Project: Apache Tez Issue Type:

[jira] [Updated] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

2020-07-28 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4208: -- Attachment: q67_sorter.log > Pipelinesorter uses single SortSpan after spill >

[jira] [Updated] (TEZ-4207) Provide approximate number of input records to be processed in UnorderedKVInput

2020-07-27 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4207: -- Attachment: TEZ-4207.wip.patch > Provide approximate number of input records to be processed in

[jira] [Updated] (TEZ-4207) Provide approximate number of input records to be processed in UnorderedKVInput

2020-07-27 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4207: -- Summary: Provide approximate number of input records to be processed in UnorderedKVInput (was:

[jira] [Moved] (TEZ-4207) Provide approximate number of input records to be processed in broadcast reader

2020-07-27 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan moved HIVE-23936 to TEZ-4207: -- Key: TEZ-4207 (was: HIVE-23936) Project: Apache Tez (was:

[jira] [Commented] (TEZ-4175) Consider removing YarnConfiguration where it's possible

2020-07-23 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164098#comment-17164098 ] Rajesh Balamohan commented on TEZ-4175: --- [~abstractdog] , thanks for sharing the patch. It would be

[jira] [Commented] (TEZ-4128) Logging: Fix ArrayOutOfBound in PipelineSorter

2020-07-23 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163447#comment-17163447 ] Rajesh Balamohan commented on TEZ-4128: --- [~rameshkumar]: Is this still an issue? I believe this was

[jira] [Commented] (TEZ-4203) Findbugs: MergeThread.shuffleSchedulerThread; locked 80% of time

2020-07-22 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17162529#comment-17162529 ] Rajesh Balamohan commented on TEZ-4203: --- This isn't a real sync issue. "shuffleSchedulerThread" is

[jira] [Created] (TEZ-4199) MergeManager::finalMerge should make use of compression

2020-07-13 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4199: - Summary: MergeManager::finalMerge should make use of compression Key: TEZ-4199 URL: https://issues.apache.org/jira/browse/TEZ-4199 Project: Apache Tez

[jira] [Updated] (TEZ-4194) NPE in FetcherOrderedGrouped

2020-06-23 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4194: -- Attachment: NPE_TASK_syslog_attempt_1592898862823_0002_1_01_000120_0_apache.log > NPE in

[jira] [Created] (TEZ-4194) NPE in FetcherOrderedGrouped

2020-06-23 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created TEZ-4194: - Summary: NPE in FetcherOrderedGrouped Key: TEZ-4194 URL: https://issues.apache.org/jira/browse/TEZ-4194 Project: Apache Tez Issue Type: Bug Affects

  1   2   3   4   5   6   7   8   9   10   >