[jira] [Commented] (TEZ-3379) Tez analyzer: Move sysout to log4j

2016-07-26 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15395030#comment-15395030 ] Rajesh Balamohan commented on TEZ-3379: --- [~hitesh], [~zjffdu] - Trivial change. Can yo

[jira] [Created] (TEZ-3379) Tez analyzer: Move sysout to log4j

2016-07-26 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3379: - Summary: Tez analyzer: Move sysout to log4j Key: TEZ-3379 URL: https://issues.apache.org/jira/browse/TEZ-3379 Project: Apache Tez Issue Type: Bug

[jira] [Assigned] (TEZ-3379) Tez analyzer: Move sysout to log4j

2016-07-26 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-3379: - Assignee: Rajesh Balamohan > Tez analyzer: Move sysout to log4j > -

[jira] [Commented] (TEZ-3356) Fix initializing of stats when custom ShuffleVertexManager is used

2016-07-18 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383589#comment-15383589 ] Rajesh Balamohan commented on TEZ-3356: --- Thanks for sharing the details. In this case,

[jira] [Commented] (TEZ-3356) Fix initializing of stats when custom ShuffleVertexManager is used

2016-07-18 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383551#comment-15383551 ] Rajesh Balamohan commented on TEZ-3356: --- [~petersla] - Thanks for uploading the patch.

[jira] [Commented] (TEZ-3351) Handle ATS performance issues for Hive-LLAP.

2016-07-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378831#comment-15378831 ] Rajesh Balamohan commented on TEZ-3351: --- Quick question: Do we need to check if this w

[jira] [Updated] (TEZ-3344) ATS does not contain certain vertex details

2016-07-13 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3344: -- Attachment: dag_succeeded_screenshot.png > ATS does not contain certain vertex details > -

[jira] [Commented] (TEZ-3344) ATS does not contain certain vertex details

2016-07-13 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374836#comment-15374836 ] Rajesh Balamohan commented on TEZ-3344: --- for the missing vertex, data is not in ATS.

[jira] [Updated] (TEZ-3344) ATS does not contain certain vertex details

2016-07-13 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3344: -- Attachment: dag.json vertex.json \cc [~Sreenath] > ATS does not contain certain v

[jira] [Updated] (TEZ-3344) ATS does not contain certain vertex details

2016-07-13 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3344: -- Attachment: all_vertices_page_showing_only_3_vertices.png > ATS does not contain certain vertex de

[jira] [Updated] (TEZ-3344) ATS does not contain certain vertex details

2016-07-13 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3344: -- Attachment: dag_details.png > ATS does not contain certain vertex details > --

[jira] [Created] (TEZ-3344) ATS does not contain certain vertex details

2016-07-13 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3344: - Summary: ATS does not contain certain vertex details Key: TEZ-3344 URL: https://issues.apache.org/jira/browse/TEZ-3344 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-3334) Tez Custom Shuffle Handler

2016-07-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372216#comment-15372216 ] Rajesh Balamohan commented on TEZ-3334: --- +1 for custom shuffle handler. >> "fetchin

[jira] [Updated] (TEZ-3324) Add kerberos support in ATSImportTool

2016-07-08 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3324: -- Description: When running ATSImportTool on kerberos environment, usergroup information needs to be

[jira] [Comment Edited] (TEZ-3327) ATS Parser: Populate config details available in dag

2016-07-07 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366877#comment-15366877 ] Rajesh Balamohan edited comment on TEZ-3327 at 7/7/16 10:21 PM: --

[jira] [Commented] (TEZ-3327) ATS Parser: Populate config details available in dag

2016-07-07 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366877#comment-15366877 ] Rajesh Balamohan commented on TEZ-3327: --- Thanks [~hitesh]. Will commit shortly. > ATS

[jira] [Commented] (TEZ-3327) ATS Parser: Populate config details available in dag

2016-07-07 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366848#comment-15366848 ] Rajesh Balamohan commented on TEZ-3327: --- that would have been ideal, btu the version o

[jira] [Comment Edited] (TEZ-3327) ATS Parser: Populate config details available in dag

2016-07-07 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366848#comment-15366848 ] Rajesh Balamohan edited comment on TEZ-3327 at 7/7/16 9:57 PM: ---

[jira] [Commented] (TEZ-3328) [Umbrella] UI does not work well when there are separate DAG and session-level ACLs

2016-07-07 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365813#comment-15365813 ] Rajesh Balamohan commented on TEZ-3328: --- Proposal in TEZ-3327 was to provide configs s

[jira] [Comment Edited] (TEZ-3327) ATS Parser: Populate config details available in dag

2016-07-06 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365432#comment-15365432 ] Rajesh Balamohan edited comment on TEZ-3327 at 7/7/16 1:16 AM: ---

[jira] [Updated] (TEZ-3327) ATS Parser: Populate config details available in dag

2016-07-06 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3327: -- Attachment: TEZ-3327.1.patch ATSImport tool downloads ATS information pertaining to TEZ_APPLICATIO

[jira] [Assigned] (TEZ-3327) ATS Parser: Populate config details available in dag

2016-07-06 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-3327: - Assignee: Rajesh Balamohan > ATS Parser: Populate config details available in dag > ---

[jira] [Created] (TEZ-3327) ATS Parser: Populate config details available in dag

2016-07-06 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3327: - Summary: ATS Parser: Populate config details available in dag Key: TEZ-3327 URL: https://issues.apache.org/jira/browse/TEZ-3327 Project: Apache Tez Issue T

[jira] [Created] (TEZ-3324) Add kerberos support in ATSImportTool

2016-07-05 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3324: - Summary: Add kerberos support in ATSImportTool Key: TEZ-3324 URL: https://issues.apache.org/jira/browse/TEZ-3324 Project: Apache Tez Issue Type: Improvemen

[jira] [Updated] (TEZ-3313) ATSFileParser : Wrong args passed in VersionInfo

2016-07-03 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3313: -- Attachment: TEZ-3313.2.patch > ATSFileParser : Wrong args passed in VersionInfo >

[jira] [Commented] (TEZ-3313) ATSFileParser : Wrong args passed in VersionInfo

2016-07-03 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15360702#comment-15360702 ] Rajesh Balamohan commented on TEZ-3313: --- [~ozawa] - Thanks for the review. 1. What's

[jira] [Updated] (TEZ-3314) Double counting input bytes in MultiMRInput

2016-06-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3314: -- Fix Version/s: 0.9.0 > Double counting input bytes in MultiMRInput > -

[jira] [Commented] (TEZ-3314) Double counting input bytes in MultiMRInput

2016-06-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356201#comment-15356201 ] Rajesh Balamohan commented on TEZ-3314: --- lgtm. Thanks [~harishjp]. Will commit shortly

[jira] [Commented] (TEZ-3308) Add counters to capture input split length.

2016-06-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352101#comment-15352101 ] Rajesh Balamohan commented on TEZ-3308: --- +1 pending jenkin tests > Add counters to ca

[jira] [Updated] (TEZ-3313) ATSFileParser : Wrong args passed in VersionInfo

2016-06-27 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3313: -- Attachment: TEZ-3313.1.patch > ATSFileParser : Wrong args passed in VersionInfo >

[jira] [Created] (TEZ-3313) ATSFileParser : Wrong args passed in VersionInfo

2016-06-27 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3313: - Summary: ATSFileParser : Wrong args passed in VersionInfo Key: TEZ-3313 URL: https://issues.apache.org/jira/browse/TEZ-3313 Project: Apache Tez Issue Type:

[jira] [Created] (TEZ-3312) CriticalPathAnalyzer broken when loading 0.7 dataset

2016-06-27 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3312: - Summary: CriticalPathAnalyzer broken when loading 0.7 dataset Key: TEZ-3312 URL: https://issues.apache.org/jira/browse/TEZ-3312 Project: Apache Tez Issue T

[jira] [Commented] (TEZ-3308) Add counters to capture input split length.

2016-06-24 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348944#comment-15348944 ] Rajesh Balamohan commented on TEZ-3308: --- [~harishjp] - Thanks for the patch. Minor com

[jira] [Resolved] (TEZ-2582) Consider removing DataInputBuffer sync overheads in pipelinedsorter

2016-06-24 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-2582. --- Resolution: Resolved > Consider removing DataInputBuffer sync overheads in pipelinedsorter > ---

[jira] [Commented] (TEZ-2582) Consider removing DataInputBuffer sync overheads in pipelinedsorter

2016-06-24 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348935#comment-15348935 ] Rajesh Balamohan commented on TEZ-2582: --- Thanks [~ozawa]. I will close this jira as HA

[jira] [Commented] (TEZ-3311) ATSFileParser resource leak: InputStream should be closed

2016-06-24 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348930#comment-15348930 ] Rajesh Balamohan commented on TEZ-3311: --- Thanks for sharing the patch [~ozawa]. Is c

[jira] [Commented] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15342655#comment-15342655 ] Rajesh Balamohan commented on TEZ-3291: --- Thanks [~bikassaha]. Created TEZ-3310. Will r

[jira] [Updated] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3291: -- Attachment: TEZ-3291.004.patch > Optimize splits grouping when locality information is not availab

[jira] [Created] (TEZ-3310) Handle splits grouping better when locality information is not available (or only when localhost is available)

2016-06-21 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3310: - Summary: Handle splits grouping better when locality information is not available (or only when localhost is available) Key: TEZ-3310 URL: https://issues.apache.org/jira/browse/

[jira] [Commented] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15342512#comment-15342512 ] Rajesh Balamohan commented on TEZ-3291: --- [~bikassaha] - 0.4 patch has been working fin

[jira] [Updated] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-20 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3291: -- Attachment: TEZ-3291.5.patch Modified the patch to do the following. Tez does not need to handle

[jira] [Commented] (TEZ-3305) TestAnalyzer fails on Hadoop 2.7

2016-06-16 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334077#comment-15334077 ] Rajesh Balamohan commented on TEZ-3305: --- Thanks for the patch [~jeagles]. lgtm. +1. >

[jira] [Commented] (TEZ-3304) TestHistoryParser fails with Hadoop 2.7

2016-06-16 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15333419#comment-15333419 ] Rajesh Balamohan commented on TEZ-3304: --- Thanks [~jeagles]. lgtm. +1 > TestHistoryPar

[jira] [Commented] (TEZ-3216) Support for more precise partition stats in VertexManagerEvent

2016-06-15 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331423#comment-15331423 ] Rajesh Balamohan commented on TEZ-3216: --- MB should be fine. "memory_optimized" and "p

[jira] [Updated] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3291: -- Attachment: TEZ-3291.4.patch Attaching the patch which explicitly checks whether all splits having

[jira] [Updated] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3291: -- Attachment: TEZ-3291.3.patch Attaching patch to address review comments. S3 urls can be explicitl

[jira] [Assigned] (TEZ-3297) Deadlock scenario in AM during ShuffleVertexManager auto reduce

2016-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-3297: - Assignee: Rajesh Balamohan > Deadlock scenario in AM during ShuffleVertexManager auto reduc

[jira] [Updated] (TEZ-3297) Deadlock scenario in AM during ShuffleVertexManager auto reduce

2016-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3297: -- Attachment: TEZ-3297.2.branch-0.7.patch Thanks a lot [~sseth], [~bikassaha]. Will commit it shortl

[jira] [Updated] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3291: -- Attachment: TEZ-3291.2.patch Thanks for the review [~bikassaha]. Attaching the revised patch. It

[jira] [Commented] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15326302#comment-15326302 ] Rajesh Balamohan commented on TEZ-3291: --- It is ready for review [~bikassaha]. haven't

[jira] [Updated] (TEZ-3297) Deadlock scenario in AM during ShuffleVertexManager auto reduce

2016-06-09 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3297: -- Attachment: TEZ-3297.2.patch Thanks for the review [~sseth]. Added comments + added writeLock in

[jira] [Updated] (TEZ-3297) Deadlock scenario in AM during ShuffleVertexManager auto reduce

2016-06-09 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3297: -- Attachment: TEZ-3297.1.patch > Deadlock scenario in AM during ShuffleVertexManager auto reduce > -

[jira] [Commented] (TEZ-3297) Deadlock scenario in AM during ShuffleVertexManager auto reduce

2016-06-09 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15323734#comment-15323734 ] Rajesh Balamohan commented on TEZ-3297: --- Attaching .1 patch for review. > Deadlock sc

[jira] [Commented] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-09 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322781#comment-15322781 ] Rajesh Balamohan commented on TEZ-3291: --- [~bikassaha] - The fix is the single line che

[jira] [Updated] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-05 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3291: -- Attachment: TEZ-3291.WIP.patch E.g TPC-DS query_22 @ 200GB scale: *Without Patch: 291.59 seconds*

[jira] [Comment Edited] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-05 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316094#comment-15316094 ] Rajesh Balamohan edited comment on TEZ-3291 at 6/6/16 12:38 AM: --

[jira] [Created] (TEZ-3291) Optimize splits grouping when locality information is not available

2016-06-05 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3291: - Summary: Optimize splits grouping when locality information is not available Key: TEZ-3291 URL: https://issues.apache.org/jira/browse/TEZ-3291 Project: Apache Tez

[jira] [Commented] (TEZ-3265) Add preconditions check in SortSpan when available mem is less than metasize

2016-05-26 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15303369#comment-15303369 ] Rajesh Balamohan commented on TEZ-3265: --- agreed. Saw this "reserved.remaining()=146800

[jira] [Updated] (TEZ-3265) Add preconditions check in SortSpan when available mem is less than metasize

2016-05-26 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3265: -- Attachment: TEZ-3265.1.patch > Add preconditions check in SortSpan when available mem is less than

[jira] [Created] (TEZ-3265) Add preconditions check in SortSpan when available mem is less than metasize

2016-05-23 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3265: - Summary: Add preconditions check in SortSpan when available mem is less than metasize Key: TEZ-3265 URL: https://issues.apache.org/jira/browse/TEZ-3265 Project: Apa

[jira] [Commented] (TEZ-3237) Corrupted shuffle transfers to disk are not detected during transfer

2016-05-12 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281610#comment-15281610 ] Rajesh Balamohan commented on TEZ-3237: --- Earlier comment was for all the jobs that has

[jira] [Commented] (TEZ-2442) Support DFS based shuffle in addition to HTTP shuffle

2016-05-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281197#comment-15281197 ] Rajesh Balamohan commented on TEZ-2442: --- Haven't caught up on the latest patch yet. Pl

[jira] [Commented] (TEZ-2741) Hive on Tez does not work well with Sequence Files Schema changes

2016-05-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281195#comment-15281195 ] Rajesh Balamohan commented on TEZ-2741: --- Can you please provide details on which versi

[jira] [Commented] (TEZ-3237) Corrupted shuffle transfers to disk are not detected during transfer

2016-05-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281069#comment-15281069 ] Rajesh Balamohan commented on TEZ-3237: --- [~ozawa] Having checksum will definitely help

[jira] [Commented] (TEZ-3237) Corrupted shuffle transfers to disk are not detected during transfer

2016-05-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281001#comment-15281001 ] Rajesh Balamohan commented on TEZ-3237: --- Checksumming can be enabled by default. But o

[jira] [Commented] (TEZ-3237) Corrupted shuffle transfers to disk are not detected during transfer

2016-05-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280974#comment-15280974 ] Rajesh Balamohan commented on TEZ-3237: --- [~jlowe] - Thanks for sharing the patch. Woul

[jira] [Updated] (TEZ-3250) TezTaskRunner2 should accept ExecutorService

2016-05-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3250: -- Fix Version/s: 0.8.4 0.9.0 > TezTaskRunner2 should accept ExecutorService > ---

[jira] [Updated] (TEZ-3251) Allow ability to add custom counters to TaskRunner2Callable

2016-05-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3251: -- Fix Version/s: 0.9.0 > Allow ability to add custom counters to TaskRunner2Callable > -

[jira] [Updated] (TEZ-3251) Allow ability to add custom counters to TaskRunner2Callable

2016-05-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3251: -- Fix Version/s: 0.8.4 > Allow ability to add custom counters to TaskRunner2Callable > -

[jira] [Commented] (TEZ-3251) Allow ability to add custom counters to TaskRunner2Callable

2016-05-10 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15279542#comment-15279542 ] Rajesh Balamohan commented on TEZ-3251: --- lgtm. +1. Will wait for jenkins to finish.

[jira] [Updated] (TEZ-3250) TezTaskRunner2 should accept ExecutorService

2016-05-10 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3250: -- Summary: TezTaskRunner2 should accept ExecutorService (was: TezTaskRunner2 should not accept List

[jira] [Commented] (TEZ-3250) TezTaskRunner2 should not accept ListeningExecutorService

2016-05-10 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15279373#comment-15279373 ] Rajesh Balamohan commented on TEZ-3250: --- Thanks for the contribution [~prasanth_j].

[jira] [Commented] (TEZ-3248) Allowing one fetcher to go past thresholds could cause OOM in corner cases (in OrderedShuffle)

2016-05-10 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278149#comment-15278149 ] Rajesh Balamohan commented on TEZ-3248: --- [~sseth] - Can you plz check when you find so

[jira] [Updated] (TEZ-3248) Allowing one fetcher to go past thresholds could cause OOM in corner cases (in OrderedShuffle)

2016-05-10 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3248: -- Attachment: TEZ-3248.1.patch attaching .1 patch > Allowing one fetcher to go past thresholds coul

[jira] [Created] (TEZ-3248) Allowing one fetcher to go past thresholds could cause OOM in corner cases (in OrderedShuffle)

2016-05-10 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3248: - Summary: Allowing one fetcher to go past thresholds could cause OOM in corner cases (in OrderedShuffle) Key: TEZ-3248 URL: https://issues.apache.org/jira/browse/TEZ-3248

[jira] [Commented] (TEZ-3247) Tez does not compile against hadoop-trunk

2016-05-09 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15277306#comment-15277306 ] Rajesh Balamohan commented on TEZ-3247: --- [~hitesh] - This change would not affect anal

[jira] [Created] (TEZ-3220) Add license file details in tez job parser's fat jar

2016-04-20 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3220: - Summary: Add license file details in tez job parser's fat jar Key: TEZ-3220 URL: https://issues.apache.org/jira/browse/TEZ-3220 Project: Apache Tez Issue T

[jira] [Commented] (TEZ-3202) Reduce the memory need for jobs with high number of segments

2016-04-13 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15239028#comment-15239028 ] Rajesh Balamohan commented on TEZ-3202: --- [~jeagles] - Sorry about the delay. Patch LGT

[jira] [Updated] (TEZ-3211) tez-ui2 : build issue

2016-04-11 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3211: -- Attachment: debug.log > tez-ui2 : build issue > - > > Key: TEZ

[jira] [Created] (TEZ-3211) tez-ui2 : build issue

2016-04-11 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3211: - Summary: tez-ui2 : build issue Key: TEZ-3211 URL: https://issues.apache.org/jira/browse/TEZ-3211 Project: Apache Tez Issue Type: Bug Reporter:

[jira] [Commented] (TEZ-3198) Shuffle failures for the trailing task in a vertex are often fatal to the entire DAG

2016-04-07 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15230335#comment-15230335 ] Rajesh Balamohan commented on TEZ-3198: --- That is correct. I took the example of 500 in

[jira] [Commented] (TEZ-3198) Shuffle failures for the trailing task in a vertex are often fatal to the entire DAG

2016-04-06 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15229416#comment-15229416 ] Rajesh Balamohan commented on TEZ-3198: --- Thanks [~jlowe] for sharing the details. Sinc

[jira] [Commented] (TEZ-3198) Shuffle failures for the trailing task in a vertex are often fatal to the entire DAG

2016-04-05 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15227422#comment-15227422 ] Rajesh Balamohan commented on TEZ-3198: --- [~jlowe] - Would it be possible to share the

[jira] [Updated] (TEZ-3197) Tez-tools: create zeppelin notebook for tez analyzers

2016-04-05 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3197: -- Attachment: screen-shot-3.png screen-shot-2.png screen-shot-1.png

[jira] [Updated] (TEZ-3197) Tez-tools: create zeppelin notebook for tez analyzers

2016-04-05 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3197: -- Attachment: TEZ-3197.WIP.patch Attaching WIP patch. > Tez-tools: create zeppelin notebook for tez

[jira] [Created] (TEZ-3197) Tez-tools: create zeppelin notebook for tez analyzers

2016-04-05 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3197: - Summary: Tez-tools: create zeppelin notebook for tez analyzers Key: TEZ-3197 URL: https://issues.apache.org/jira/browse/TEZ-3197 Project: Apache Tez Issue

[jira] [Commented] (TEZ-3192) IFile#checkState creating unnecessary objects though auto-boxing

2016-03-31 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15220880#comment-15220880 ] Rajesh Balamohan commented on TEZ-3192: --- +1. lgtm. Thanks [~jeagles] > IFile#checkSta

[jira] [Commented] (TEZ-3187) Pig on tez hang with java.io.IOException: Connection reset by peer

2016-03-30 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219108#comment-15219108 ] Rajesh Balamohan commented on TEZ-3187: --- scope_6778 --> scope_6780, scope_6785 scope_6

[jira] [Commented] (TEZ-3181) History parser : Handle invalid/unsupported history event types gracefully

2016-03-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205803#comment-15205803 ] Rajesh Balamohan commented on TEZ-3181: --- This is mainly for HistoryEvent (VERTEX_PARAL

[jira] [Updated] (TEZ-3181) History parser : Handle invalid/unsupported history event types gracefully

2016-03-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3181: -- Attachment: TEZ-3181.1.patch [~bikassaha] - Please review when you find time. > History parser :

[jira] [Created] (TEZ-3181) History parser : Handle invalid/unsupported history event types gracefully

2016-03-21 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3181: - Summary: History parser : Handle invalid/unsupported history event types gracefully Key: TEZ-3181 URL: https://issues.apache.org/jira/browse/TEZ-3181 Project: Apach

[jira] [Updated] (TEZ-3179) ATS data being downloaded needs versioning

2016-03-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3179: -- Summary: ATS data being downloaded needs versioning (was: ATS parser needs versioning) > ATS dat

[jira] [Created] (TEZ-3179) ATS parser needs versioning

2016-03-20 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3179: - Summary: ATS parser needs versioning Key: TEZ-3179 URL: https://issues.apache.org/jira/browse/TEZ-3179 Project: Apache Tez Issue Type: Bug Repo

[jira] [Commented] (TEZ-2756) MergeManager close should not try merging files on close if invoked after a shuffle exception

2016-03-03 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15179111#comment-15179111 ] Rajesh Balamohan commented on TEZ-2756: --- lgtm. +1 Can you please remove TezRawKeyValu

[jira] [Updated] (TEZ-3154) Debuggability : Add an option to take threaddump from a specific vertex/task

2016-03-01 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3154: -- Description: tez.task-specific.launch.cmd-opts and tez.task-specific.launch.cmd-opts.list (e.g "M

[jira] [Updated] (TEZ-3154) Debuggability : Add an option to take threaddump from a specific vertex/task

2016-03-01 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3154: -- Description: tez.task-specific.launch.cmd-opts and tez.task-specific.launch.cmd-opts.list (e.g "M

[jira] [Created] (TEZ-3154) Debuggability : Add an option to take threaddump from a specific vertex/task

2016-03-01 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-3154: - Summary: Debuggability : Add an option to take threaddump from a specific vertex/task Key: TEZ-3154 URL: https://issues.apache.org/jira/browse/TEZ-3154 Project: Apa

[jira] [Updated] (TEZ-3148) Invalid event TA_TEZ_EVENT_UPDATE on TaskAttempt

2016-03-01 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3148: -- Description: Got the following when executing one of the DAG. Tez details: versionInfo=[ compone

[jira] [Updated] (TEZ-3149) Tez-tools: Add username in DagInfo

2016-02-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3149: -- Fix Version/s: 0.8.3 > Tez-tools: Add username in DagInfo > -- > >

[jira] [Updated] (TEZ-3149) Tez-tools: Add username in DagInfo

2016-02-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3149: -- Attachment: TEZ-3149.1.patch [~bikassaha] - Please review when you find time. > Tez-tools: Add us

<    1   2   3   4   5   6   7   8   9   10   >