[jira] [Assigned] (TEZ-1205) Remove profiling keyword from APIs/configs

2014-07-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-1205: Assignee: Gopal V (was: Rajesh Balamohan) Remove profiling keyword from APIs/configs

[jira] [Commented] (TEZ-1205) Remove profiling keyword from APIs/configs

2014-07-29 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078913#comment-14078913 ] Gopal V commented on TEZ-1205: -- This is passed in as 2 properties - one declaring the actual

[jira] [Assigned] (TEZ-1360) Provide vertex parallelism to each vertex task

2014-08-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-1360: Assignee: Gopal V Provide vertex parallelism to each vertex task

[jira] [Updated] (TEZ-1360) Provide vertex parallelism to each vertex task

2014-08-05 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1360: - Attachment: TEZ-1332.1.patch Provide vertex parallelism to each vertex task

[jira] [Commented] (TEZ-1372) Fix preWarm to work after recent API changes

2014-08-07 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14089848#comment-14089848 ] Gopal V commented on TEZ-1372: -- [~bikassaha]: getting to this after HIVE-7601 and HIVE-7639

[jira] [Commented] (TEZ-1372) Fix preWarm to work after recent API changes

2014-08-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091011#comment-14091011 ] Gopal V commented on TEZ-1372: -- Missed PreWarmVertex.java in diff? Fix preWarm to work after

[jira] [Updated] (TEZ-1205) Remove profiling keyword from APIs/configs

2014-08-11 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1205: - Assignee: Rajesh Balamohan (was: Gopal V) Remove profiling keyword from APIs/configs

[jira] [Commented] (TEZ-1360) Provide vertex parallelism to each vertex task

2014-08-11 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093368#comment-14093368 ] Gopal V commented on TEZ-1360: -- [~hitesh]: looks like I'll have to start over after the API

[jira] [Updated] (TEZ-1360) Provide vertex parallelism to each vertex task

2014-08-11 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1360: - Attachment: TEZ-1360.2.patch Provide vertex parallelism to each vertex task

[jira] [Commented] (TEZ-1411) Address initial feedback on swimlanes

2014-08-15 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098965#comment-14098965 ] Gopal V commented on TEZ-1411: -- [~jeagles]: sure, that shouldn't be a big issue. The reason to

[jira] [Commented] (TEZ-1411) Address initial feedback on swimlanes

2014-08-15 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099297#comment-14099297 ] Gopal V commented on TEZ-1411: -- [~jeagles]: You can produce a zoomed out view by modifing the

[jira] [Commented] (TEZ-1390) Replace byte[] with ByteBuffer as the type of user payload in the API

2014-08-18 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14101350#comment-14101350 ] Gopal V commented on TEZ-1390: -- Need to put a warning around {code} ... +byte[] bytes

[jira] [Created] (TEZ-1466) Fix JDK8 builds of tez

2014-08-19 Thread Gopal V (JIRA)
Gopal V created TEZ-1466: Summary: Fix JDK8 builds of tez Key: TEZ-1466 URL: https://issues.apache.org/jira/browse/TEZ-1466 Project: Apache Tez Issue Type: Bug Affects Versions: 0.5.0

[jira] [Updated] (TEZ-1360) Provide vertex parallelism to each vertex task

2014-08-19 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1360: - Attachment: TEZ-1360.4.patch Provide vertex parallelism to each vertex task

[jira] [Comment Edited] (TEZ-1469) AM/Session LRs are not shipped to vertices in new API use-case

2014-08-20 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104286#comment-14104286 ] Gopal V edited comment on TEZ-1469 at 8/20/14 6:26 PM: --- I think this

[jira] [Commented] (TEZ-1469) AM/Session LRs are not shipped to vertices in new API use-case

2014-08-20 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104286#comment-14104286 ] Gopal V commented on TEZ-1469: -- I think this is a breakage between Session and on-Session

[jira] [Commented] (TEZ-1469) AM/Session LRs are not shipped to vertices in new API use-case

2014-08-20 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104378#comment-14104378 ] Gopal V commented on TEZ-1469: -- Fixed that issue in the DAGAppMaster, but still only getting

[jira] [Commented] (TEZ-1469) AM/Session LRs are not shipped to vertices in new API use-case

2014-08-20 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104462#comment-14104462 ] Gopal V commented on TEZ-1469: -- I'm fine with the DAG as well, if that is guaranteed to not

[jira] [Updated] (TEZ-1469) AM/Session LRs are not shipped to vertices in new API use-case

2014-08-20 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1469: - Attachment: TEZ-1469.2.patch Patch to DAGAppMaster to add AM Local Resources even if not in session mode.

[jira] [Created] (TEZ-1479) Disambiguate between ShuffleInputEventHandler and ShuffleInputEventHandlerImpl (which are not related)

2014-08-21 Thread Gopal V (JIRA)
Gopal V created TEZ-1479: Summary: Disambiguate between ShuffleInputEventHandler and ShuffleInputEventHandlerImpl (which are not related) Key: TEZ-1479 URL: https://issues.apache.org/jira/browse/TEZ-1479

[jira] [Commented] (TEZ-1332) tez swimlanes UI tool

2014-08-22 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14106580#comment-14106580 ] Gopal V commented on TEZ-1332: -- This patch was all python - can't quite figure out how this

[jira] [Updated] (TEZ-1360) Provide vertex parallelism to each vertex task

2014-08-23 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1360: - Attachment: TEZ-1360.6.patch Patch with doc changes from me and unit test from [~rajesh.balamohan]. Provide

[jira] [Created] (TEZ-1489) Broadcast Shuffle should call freeResources() on FetchedInput

2014-08-24 Thread Gopal V (JIRA)
Gopal V created TEZ-1489: Summary: Broadcast Shuffle should call freeResources() on FetchedInput Key: TEZ-1489 URL: https://issues.apache.org/jira/browse/TEZ-1489 Project: Apache Tez Issue Type:

[jira] [Created] (TEZ-1491) Tez reducer-side merge's counter update is slow

2014-08-24 Thread Gopal V (JIRA)
Gopal V created TEZ-1491: Summary: Tez reducer-side merge's counter update is slow Key: TEZ-1491 URL: https://issues.apache.org/jira/browse/TEZ-1491 Project: Apache Tez Issue Type: Bug Affects

[jira] [Updated] (TEZ-1491) Tez reducer-side merge's counter update is slow

2014-08-24 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1491: - Attachment: perf-top-counters.png Tez reducer-side merge's counter update is slow

[jira] [Updated] (TEZ-1488) Implement HashComparatorBytesWritable in TezBytesComparator

2014-08-24 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1488: - Attachment: TEZ-1488.1.patch Implement HashComparatorBytesWritable in TezBytesComparator

[jira] [Commented] (TEZ-1083) Enable IFile RLE for DefaultSorter

2014-08-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109453#comment-14109453 ] Gopal V commented on TEZ-1083: -- This looks alright - this just needs a roll-over check for the

[jira] [Commented] (TEZ-1492) IFile RLE not kicking in due to bug in BufferUtils.compare()

2014-08-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109461#comment-14109461 ] Gopal V commented on TEZ-1492: -- The BufferUtils class needs re-namespacing as well, as part of

[jira] [Commented] (TEZ-1489) Broadcast Shuffle should call freeResources() on FetchedInput

2014-08-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109472#comment-14109472 ] Gopal V commented on TEZ-1489: -- The buffer is being cleared up correctly - but the unreserve()

[jira] [Commented] (TEZ-1489) Broadcast Shuffle should call freeResources() on FetchedInput

2014-08-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109476#comment-14109476 ] Gopal V commented on TEZ-1489: -- UnorderedKVReader::moveToNextInput() maybe is a good place for

[jira] [Created] (TEZ-1497) Add tez-broadcast-example into tez-examples/

2014-08-25 Thread Gopal V (JIRA)
Gopal V created TEZ-1497: Summary: Add tez-broadcast-example into tez-examples/ Key: TEZ-1497 URL: https://issues.apache.org/jira/browse/TEZ-1497 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-1492) IFile RLE not kicking in due to bug in BufferUtils.compare()

2014-08-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110264#comment-14110264 ] Gopal V commented on TEZ-1492: -- Thanks [~rajesh.balamohan], can I have the same diff with git

[jira] [Resolved] (TEZ-1497) Add tez-broadcast-example into tez-examples/

2014-08-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V resolved TEZ-1497. -- Resolution: Not a Problem Add tez-broadcast-example into tez-examples/

[jira] [Commented] (TEZ-1503) UnorderedKVInput.getReader() should return KeyValuesReader

2014-08-26 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111095#comment-14111095 ] Gopal V commented on TEZ-1503: -- The Unordered input will not satisfy the contract of key values

[jira] [Commented] (TEZ-1492) IFile RLE not kicking in due to bug in BufferUtils.compare()

2014-08-26 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111819#comment-14111819 ] Gopal V commented on TEZ-1492: -- Alright, this looks good - +1 IFile RLE not kicking in due to

[jira] [Commented] (TEZ-1503) UnorderedKVInput.getReader() should return KeyValuesReader

2014-08-26 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111859#comment-14111859 ] Gopal V commented on TEZ-1503: -- Yes, I think this change would make it easier to write wrong

[jira] [Commented] (TEZ-1509) Set a useful default value for java opts

2014-08-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112988#comment-14112988 ] Gopal V commented on TEZ-1509: -- -1 on the -XX:+UseCompressedStrings. The rest looks good.

[jira] [Commented] (TEZ-1501) Add a test dag to generate load on the getTask RPC

2014-08-28 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114052#comment-14114052 ] Gopal V commented on TEZ-1501: -- Looks good - +1 Just needs an fs.deleteOnExit() for the

[jira] [Commented] (TEZ-1488) Implement HashComparatorBytesWritable in TezBytesComparator

2014-08-28 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114448#comment-14114448 ] Gopal V commented on TEZ-1488: -- I think we should call the interface what it really - a

[jira] [Updated] (TEZ-1488) Implement HashComparatorBytesWritable in TezBytesComparator

2014-08-29 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1488: - Attachment: TEZ-1488.3.patch Patch includes renames - will only apply with {{git apply -v -p0 TEZ-1488.3.patch}}

[jira] [Updated] (TEZ-1488) Rename HashComparator to ProxyComparator and implement in TezBytesComparator

2014-08-29 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1488: - Summary: Rename HashComparator to ProxyComparator and implement in TezBytesComparator (was: Implement

[jira] [Updated] (TEZ-1488) Rename HashComparator to ProxyComparator and implement in TezBytesComparator

2014-08-29 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1488: - Fix Version/s: 0.6.0 Rename HashComparator to ProxyComparator and implement in TezBytesComparator

[jira] [Updated] (TEZ-1488) Rename HashComparator to ProxyComparator and implement in TezBytesComparator

2014-08-29 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1488: - Attachment: TEZ-1488.4.patch Rename HashComparator to ProxyComparator and implement in TezBytesComparator

[jira] [Commented] (TEZ-1524) getDAGStatus seems to fork out the entire JVM

2014-08-29 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115806#comment-14115806 ] Gopal V commented on TEZ-1524: -- The cache does not cache misses. getDAGStatus seems to fork

[jira] [Updated] (TEZ-1524) getDAGStatus seems to fork out the entire JVM

2014-08-29 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1524: - Attachment: TEZ-1524.1.patch getDAGStatus seems to fork out the entire JVM

[jira] [Created] (TEZ-1525) BroadcastLoadGen testcase

2014-08-29 Thread Gopal V (JIRA)
Gopal V created TEZ-1525: Summary: BroadcastLoadGen testcase Key: TEZ-1525 URL: https://issues.apache.org/jira/browse/TEZ-1525 Project: Apache Tez Issue Type: Test Affects Versions: 0.6.0

[jira] [Updated] (TEZ-1525) BroadcastLoadGen testcase

2014-08-29 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1525: - Attachment: TEZ-1525.1.patch BroadcastLoadGen testcase - Key: TEZ-1525

[jira] [Updated] (TEZ-1524) getDAGStatus seems to fork out the entire JVM

2014-09-11 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1524: - Attachment: TEZ-1524.2.patch getDAGStatus seems to fork out the entire JVM

[jira] [Updated] (TEZ-1524) getDAGStatus seems to fork out the entire JVM on non-secure clusters

2014-09-12 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1524: - Summary: getDAGStatus seems to fork out the entire JVM on non-secure clusters (was: getDAGStatus seems to fork out

[jira] [Updated] (TEZ-1524) getDAGStatus seems to fork out the entire JVM on non-secure clusters

2014-09-12 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1524: - Attachment: TEZ-1524.3.patch Removed the stray println. getDAGStatus seems to fork out the entire JVM on

[jira] [Updated] (TEZ-1157) Optimize broadcast :- Tasks pertaining to same job in same machine should not download multiple copies of broadcast data

2014-09-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1157: - Attachment: TEZ-1157.7.patch Optimize broadcast :- Tasks pertaining to same job in same machine should not

[jira] [Updated] (TEZ-1157) Optimize broadcast :- Tasks pertaining to same job in same machine should not download multiple copies of broadcast data

2014-09-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1157: - Attachment: TEZ-1157.8.patch Optimize broadcast :- Tasks pertaining to same job in same machine should not

[jira] [Updated] (TEZ-1157) Optimize broadcast :- Tasks pertaining to same job in same machine should not download multiple copies of broadcast data

2014-09-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1157: - Attachment: TEZ-1157.9.patch Optimize broadcast :- Tasks pertaining to same job in same machine should not

[jira] [Updated] (TEZ-1535) Minor bug in computing min-time the reducer should run without getting killed

2014-09-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1535: - Labels: performance timeunits (was: performance) Minor bug in computing min-time the reducer should run without

[jira] [Updated] (TEZ-1535) Minor bug in computing min-time the reducer should run without getting killed

2014-09-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1535: - Fix Version/s: 0.6.0 Minor bug in computing min-time the reducer should run without getting killed

[jira] [Updated] (TEZ-1157) Optimize broadcast :- Tasks pertaining to same job in same machine should not download multiple copies of broadcast data

2014-09-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1157: - Attachment: TEZ-1157.10.patch Fix TeztTezRuntimeConfiguration failures for missing keys. This is the patch for

[jira] [Updated] (TEZ-1157) Optimize broadcast :- Tasks pertaining to same job in same machine should not download multiple copies of broadcast data

2014-09-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1157: - Description: Currently tasks (belonging to same job) running in the same machine download its own copy of

[jira] [Created] (TEZ-1593) PipelinedSorter::compare() makes a key-copy to satisfy RawComparator interface

2014-09-18 Thread Gopal V (JIRA)
Gopal V created TEZ-1593: Summary: PipelinedSorter::compare() makes a key-copy to satisfy RawComparator interface Key: TEZ-1593 URL: https://issues.apache.org/jira/browse/TEZ-1593 Project: Apache Tez

[jira] [Updated] (TEZ-1596) Secure Shuffle utils is extremely expensive for fast queries

2014-09-18 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1596: - Attachment: shuffle-secure.png Secure Shuffle utils is extremely expensive for fast queries

[jira] [Updated] (TEZ-1141) DAGStatus.Progress should include number of failed attempts

2014-09-22 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1141: - Affects Version/s: 0.5.0 DAGStatus.Progress should include number of failed attempts

[jira] [Assigned] (TEZ-1609) Add hostname to logIdentifiers of fetchers for easy debugging

2014-09-22 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-1609: Assignee: Gopal V Add hostname to logIdentifiers of fetchers for easy debugging

[jira] [Assigned] (TEZ-1141) DAGStatus.Progress should include number of failed attempts

2014-09-22 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-1141: Assignee: Gopal V Hit this issue in a bad way today, need a way to debug this. DAGStatus.Progress should

[jira] [Created] (TEZ-1622) Implement a tez jar equivalent script to avoid the complexities of hadoop jar

2014-09-24 Thread Gopal V (JIRA)
Gopal V created TEZ-1622: Summary: Implement a tez jar equivalent script to avoid the complexities of hadoop jar Key: TEZ-1622 URL: https://issues.apache.org/jira/browse/TEZ-1622 Project: Apache Tez

[jira] [Commented] (TEZ-1609) Add hostname to logIdentifiers of fetchers for easy debugging

2014-09-26 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149779#comment-14149779 ] Gopal V commented on TEZ-1609: -- Yes, they do print the URL and speed. Add hostname to

[jira] [Commented] (TEZ-1634) BlockCompressorStream.finish() is called twice in IFile.close leading to Shuffle errors

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14153290#comment-14153290 ] Gopal V commented on TEZ-1634: -- Change looks good, but is hard to read. Can you move the

[jira] [Updated] (TEZ-1634) BlockCompressorStream.finish() is called twice in IFile.close leading to Shuffle errors

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1634: - Attachment: TEZ-1634.2.patch Small cosmetic change, for easier debugging. Please review - [~rajesh.balamohan].

[jira] [Updated] (TEZ-1634) BlockCompressorStream.finish() is called twice in IFile.close leading to Shuffle errors

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1634: - Fix Version/s: 0.6.0 BlockCompressorStream.finish() is called twice in IFile.close leading to Shuffle errors

[jira] [Updated] (TEZ-1277) Tez Spill handler should truncate files to reserve space on disk

2014-10-06 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1277: - Description: Occasionally tasks fail due to full disks because the disks had space when the task was allocating

[jira] [Commented] (TEZ-1083) Enable IFile RLE for DefaultSorter

2014-10-14 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171963#comment-14171963 ] Gopal V commented on TEZ-1083: -- +1 - this enables RLE on the map-side spill. Reduce-side

[jira] [Commented] (TEZ-1277) Tez Spill handler should truncate files to reserve space on disk

2014-10-14 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171964#comment-14171964 ] Gopal V commented on TEZ-1277: -- Need to add a NativeIO impl to do {{fallocate}}. Tez Spill

[jira] [Updated] (TEZ-1525) BroadcastLoadGen testcase

2014-10-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1525: - Attachment: TEZ-1525.2.patch Rebase after TEZ-1479 BroadcastLoadGen testcase -

[jira] [Commented] (TEZ-1141) DAGStatus.Progress should include number of failed attempts

2014-10-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14175874#comment-14175874 ] Gopal V commented on TEZ-1141: -- LGTM, I found that this doesn't track NM blacklisting, but that

[jira] [Updated] (TEZ-1525) BroadcastLoadGen testcase

2014-10-20 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1525: - Attachment: TEZ-1525.3.patch BroadcastLoadGen testcase - Key: TEZ-1525

[jira] [Updated] (TEZ-1690) TestMultiMRInput tests fail because of user collisions

2014-10-20 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1690: - Affects Version/s: 0.5.2 TestMultiMRInput tests fail because of user collisions

[jira] [Created] (TEZ-1693) ARCHIVE local resources are not supported in Tez DAGs

2014-10-21 Thread Gopal V (JIRA)
Gopal V created TEZ-1693: Summary: ARCHIVE local resources are not supported in Tez DAGs Key: TEZ-1693 URL: https://issues.apache.org/jira/browse/TEZ-1693 Project: Apache Tez Issue Type: Bug

[jira] [Updated] (TEZ-1593) Refactor PipelinedSorter to remove all MMAP based ByteBuffer references

2014-10-21 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1593: - Summary: Refactor PipelinedSorter to remove all MMAP based ByteBuffer references (was: PipelinedSorter::compare()

[jira] [Updated] (TEZ-1593) Refactor PipelinedSorter to remove all MMAP based ByteBuffer references

2014-10-21 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1593: - Description: The current implementation of PipelinedSorter has a slow section which revolves around key

[jira] [Commented] (TEZ-1141) DAGStatus.Progress should include number of failed attempts

2014-10-22 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14180900#comment-14180900 ] Gopal V commented on TEZ-1141: -- The hive progress UI is already crowded with 4 numbers per

[jira] [Commented] (TEZ-1141) DAGStatus.Progress should include number of failed attempts

2014-10-22 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14180910#comment-14180910 ] Gopal V commented on TEZ-1141: -- The UI should be able to extract this information from

[jira] [Commented] (TEZ-1688) Add applicationId as a primary filter for all Timeline data for easier export

2014-10-22 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14180934#comment-14180934 ] Gopal V commented on TEZ-1688: -- LGTM - +1. I noticed that this uses the same naming convention

[jira] [Commented] (TEZ-1688) Add applicationId as a primary filter for all Timeline data for easier export

2014-10-22 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14180944#comment-14180944 ] Gopal V commented on TEZ-1688: -- Running through my extraction pipelines -

[jira] [Updated] (TEZ-1596) Secure Shuffle utils is extremely expensive for fast queries

2014-10-23 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1596: - Description: Generating the hash for YARN's secure shuffle is more expensive than the actual HTTP call once

[jira] [Updated] (TEZ-1698) Use ResourceCalculatorPlugin instead of ResourceCalculatorProcessTree in Tez

2014-10-23 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1698: - Attachment: ProcfsBasedProcessTree.png Use ResourceCalculatorPlugin instead of ResourceCalculatorProcessTree in

[jira] [Created] (TEZ-1698) Use ResourceCalculatorPlugin instead of ResourceCalculatorProcessTree in Tez

2014-10-23 Thread Gopal V (JIRA)
Gopal V created TEZ-1698: Summary: Use ResourceCalculatorPlugin instead of ResourceCalculatorProcessTree in Tez Key: TEZ-1698 URL: https://issues.apache.org/jira/browse/TEZ-1698 Project: Apache Tez

[jira] [Updated] (TEZ-1698) Use ResourceCalculatorPlugin instead of ResourceCalculatorProcessTree in Tez

2014-10-23 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1698: - Description: ResourceCalculatorProcessTree scraps all of /proc/ for PIDs which are part of the current task's

[jira] [Updated] (TEZ-1698) Use ResourceCalculatorPlugin instead of ResourceCalculatorProcessTree in Tez

2014-10-23 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1698: - Attachment: ProcfsFiles.png Use ResourceCalculatorPlugin instead of ResourceCalculatorProcessTree in Tez

[jira] [Updated] (TEZ-1634) BlockCompressorStream.finish() is called twice in IFile.close leading to Shuffle errors

2014-10-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1634: - Fix Version/s: 0.5.2 BlockCompressorStream.finish() is called twice in IFile.close leading to Shuffle errors

[jira] [Commented] (TEZ-1596) Secure Shuffle utils is extremely expensive for fast queries

2014-10-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14185973#comment-14185973 ] Gopal V commented on TEZ-1596: -- Works as expected - +1 Secure Shuffle utils is extremely

[jira] [Commented] (TEZ-1701) ATS fixes to flush all history events and also using batching

2014-10-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14186000#comment-14186000 ] Gopal V commented on TEZ-1701: -- Event counts add up with this patch. Within the ATS level DB,

[jira] [Created] (TEZ-1719) Allow IFile reducer merge-sort to disable crc32 checksums

2014-10-29 Thread Gopal V (JIRA)
Gopal V created TEZ-1719: Summary: Allow IFile reducer merge-sort to disable crc32 checksums Key: TEZ-1719 URL: https://issues.apache.org/jira/browse/TEZ-1719 Project: Apache Tez Issue Type: Bug

[jira] [Updated] (TEZ-1719) Allow IFile reducer merge-sort to disable crc32 checksums

2014-10-29 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1719: - Labels: Performance (was: ) Allow IFile reducer merge-sort to disable crc32 checksums

[jira] [Updated] (TEZ-1698) Cut down on ResourceCalculatorProcessTree overheads in Tez

2014-10-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1698: - Summary: Cut down on ResourceCalculatorProcessTree overheads in Tez (was: Use ResourceCalculatorPlugin instead of

[jira] [Updated] (TEZ-1698) Cut down on ResourceCalculatorProcessTree overheads in Tez

2014-10-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1698: - Attachment: TEZ-1698.3.patch [~rajesh.balamohan]: Can you test this version? Minor changes, the plugin is built

[jira] [Commented] (TEZ-1698) Cut down on ResourceCalculatorProcessTree overheads in Tez

2014-10-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191204#comment-14191204 ] Gopal V commented on TEZ-1698: -- +1 - Thanks Rajesh, this looks good. This isn't on by default,

[jira] [Commented] (TEZ-1725) Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

2014-10-31 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191449#comment-14191449 ] Gopal V commented on TEZ-1725: -- Comparing LinuxResourceCalculatorPlugin vs

[jira] [Commented] (TEZ-1725) Fix nanosecond to millis conversion in TezMxBeanResourceCalculator

2014-10-31 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191455#comment-14191455 ] Gopal V commented on TEZ-1725: -- The error bar seems to be within a percentage, even on a 10Tb

[jira] [Created] (TEZ-1733) TezMerger should sort FileChunks on decompressed size

2014-11-03 Thread Gopal V (JIRA)
Gopal V created TEZ-1733: Summary: TezMerger should sort FileChunks on decompressed size Key: TEZ-1733 URL: https://issues.apache.org/jira/browse/TEZ-1733 Project: Apache Tez Issue Type: Bug

[jira] [Updated] (TEZ-1733) TezMerger should sort FileChunks on decompressed size

2014-11-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1733: - Attachment: TEZ-1733.1.patch TezMerger should sort FileChunks on decompressed size

[jira] [Updated] (TEZ-1733) TezMerger should sort FileChunks on decompressed size

2014-11-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1733: - Priority: Critical (was: Major) TezMerger should sort FileChunks on decompressed size

[jira] [Updated] (TEZ-1733) TezMerger should sort FileChunks on decompressed size

2014-11-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-1733: - Target Version/s: 0.5.2 (was: 0.6.0) TezMerger should sort FileChunks on decompressed size

  1   2   3   4   5   6   >