[jira] [Updated] (TEZ-3208) Changes for 0.8.3 release

2016-04-08 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3208: Attachment: TEZ-3208.part1.txt First patch - updates master to 0.8.4-SNASHOT, and adds sections in

[jira] [Created] (TEZ-3208) Changes for 0.8.3 release

2016-04-08 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-3208: --- Summary: Changes for 0.8.3 release Key: TEZ-3208 URL: https://issues.apache.org/jira/browse/TEZ-3208 Project: Apache Tez Issue Type: Task

[jira] [Created] (TEZ-3205) Update tez poms to remove hadoop 2.2/2.4 profiles

2016-04-08 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-3205: --- Summary: Update tez poms to remove hadoop 2.2/2.4 profiles Key: TEZ-3205 URL: https://issues.apache.org/jira/browse/TEZ-3205 Project: Apache Tez Issue Type:

[jira] [Commented] (TEZ-3180) Update master docs to declare hadoop-2.6.x as a minimum requirement

2016-04-08 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232506#comment-15232506 ] Siddharth Seth commented on TEZ-3180: - Not waiting for jenkins since this is a doc update. Committed to

[jira] [Commented] (TEZ-3180) Update master docs to declare hadoop-2.6.x as a minimum requirement

2016-04-07 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15231445#comment-15231445 ] Siddharth Seth commented on TEZ-3180: - +1 > Update master docs to declare hadoop-2.6.x as a minimum

[jira] [Updated] (TEZ-3188) Move tez.submit.hosts out of TezConfiguration

2016-04-07 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3188: Attachment: TEZ-3188.3.txt Updated patch. Back to TezConstants. The ConfigurationProperty annotation

[jira] [Commented] (TEZ-3188) Move tez.submit.hosts out of TezConfiguration

2016-04-07 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230706#comment-15230706 ] Siddharth Seth commented on TEZ-3188: - Since these config properties are marked as private - the

[jira] [Commented] (TEZ-3120) Remove TaskCommContext.getCurrentDagName, Identifier

2016-04-07 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230680#comment-15230680 ] Siddharth Seth commented on TEZ-3120: - +1. Committing. Thanks [~ozawa] > Remove

[jira] [Commented] (TEZ-3201) Tez-UI build broken

2016-04-07 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229906#comment-15229906 ] Siddharth Seth commented on TEZ-3201: - [~Sreenath] - I think the fix will be needed on branch-0.7 as

[jira] [Commented] (TEZ-3201) Tez-UI build broken

2016-04-07 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229901#comment-15229901 ] Siddharth Seth commented on TEZ-3201: - Thanks [~Sreenath] - starting the pending pre-commit builds. >

[jira] [Commented] (TEZ-3201) Tez-UI build broken

2016-04-07 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229877#comment-15229877 ] Siddharth Seth commented on TEZ-3201: - Tried the patch locally, and this fixes my local build. The

[jira] [Updated] (TEZ-3188) Move tez.submit.hosts out of TezConfiguration

2016-04-07 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3188: Attachment: TEZ-3188.2.txt Renamed to TezConfigurationPublished - still open to better names. Tried

[jira] [Updated] (TEZ-3199) Rename getCredentials in TaskCommunicatorContext to be less confusing

2016-04-06 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3199: Attachment: TEZ-3199.1.jenkins.txt Patch for jenkins with the UI temporarily disabled. > Rename

[jira] [Updated] (TEZ-3120) Remove TaskCommContext.getCurrentDagName, Identifier

2016-04-06 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3120: Attachment: TEZ-3120.1.jenkins.txt Patch for jenkins with the UI temporarily disabled. > Remove

[jira] [Updated] (TEZ-3183) Change the taskFailed method on plugin contexts to specify the type of failure

2016-04-06 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3183: Attachment: TEZ-3183.2.jenkins.txt Patch for jenkins with the UI temporarily disabled. > Change the

[jira] [Commented] (TEZ-3201) Tez-UI build broken

2016-04-06 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228601#comment-15228601 ] Siddharth Seth commented on TEZ-3201: - Correction - tez-ui2 build is also broken. Partial output from

[jira] [Created] (TEZ-3201) Tez-UI build broken

2016-04-06 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-3201: --- Summary: Tez-UI build broken Key: TEZ-3201 URL: https://issues.apache.org/jira/browse/TEZ-3201 Project: Apache Tez Issue Type: Bug Affects Versions: 0.8.3

[jira] [Updated] (TEZ-3199) Rename getCredentials in TaskCommunicatorContext to be less confusing

2016-04-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3199: Attachment: TEZ-3199.1.txt > Rename getCredentials in TaskCommunicatorContext to be less confusing >

[jira] [Updated] (TEZ-3199) Rename getCredentials in TaskCommunicatorContext to be less confusing

2016-04-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3199: Attachment: TEZ-3199.1.txt [~hitesh] - please review. > Rename getCredentials in

[jira] [Created] (TEZ-3199) Rename getCredentials in TaskCommunicatorContext to be less confusing

2016-04-05 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-3199: --- Summary: Rename getCredentials in TaskCommunicatorContext to be less confusing Key: TEZ-3199 URL: https://issues.apache.org/jira/browse/TEZ-3199 Project: Apache Tez

[jira] [Updated] (TEZ-3130) A bad NodeManager can end up occupying all container launcher threads, delaying new launches

2016-04-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3130: Target Version/s: 0.8.3 Fix Version/s: (was: 0.8.3) > A bad NodeManager can end up

[jira] [Updated] (TEZ-3198) Shuffle failures for the trailing task in a vertex are often fatal to the entire DAG

2016-04-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3198: Target Version/s: 0.7.1 Fix Version/s: (was: 0.8.3) (was: 0.7.1)

[jira] [Commented] (TEZ-3195) TezMerger OOM: unreserve called while memory still held

2016-04-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15227042#comment-15227042 ] Siddharth Seth commented on TEZ-3195: - [~jeagles] - could you please provide a little more information

[jira] [Commented] (TEZ-3161) Allow task to report different kinds of errors - fatal / kill

2016-04-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15226748#comment-15226748 ] Siddharth Seth commented on TEZ-3161: - [~oae] - I noticed you watching the jira. Is this something you

[jira] [Updated] (TEZ-3161) Allow task to report different kinds of errors - fatal / kill

2016-04-05 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3161: Attachment: TEZ-3161.6.txt Updated patch. Thanks for the reviews [~hitesh]. Will commit after the

[jira] [Commented] (TEZ-3177) Non-DAG events should use the session domain or no domain if the data does not need protection

2016-04-04 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15225189#comment-15225189 ] Siddharth Seth commented on TEZ-3177: - +1. > Non-DAG events should use the session domain or no domain

[jira] [Updated] (TEZ-3161) Allow task to report different kinds of errors - fatal / kill

2016-04-02 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3161: Attachment: TEZ-3161.5.txt Updated patch with the test fixed. > Allow task to report different kinds

[jira] [Commented] (TEZ-3077) TezClient.waitTillReady should support timeout

2016-04-02 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223058#comment-15223058 ] Siddharth Seth commented on TEZ-3077: - Thanks for the updated patch [~kshukla]. This looks a lot better

[jira] [Commented] (TEZ-3177) Non-DAG events should use the session domain or no domain if the data does not need protection

2016-04-02 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223051#comment-15223051 ] Siddharth Seth commented on TEZ-3177: - {code} + if (historyACLPolicyManager != null + &&

[jira] [Updated] (TEZ-3161) Allow task to report different kinds of errors - fatal / kill

2016-04-02 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3161: Attachment: TEZ-3161.4.txt Updated patch with the following changes. - FailureType renamed to

[jira] [Commented] (TEZ-3161) Allow task to report different kinds of errors - fatal / kill

2016-03-30 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15218546#comment-15218546 ] Siddharth Seth commented on TEZ-3161: - Thanks for the review. bq. FailureType seems to be specific to

[jira] [Updated] (TEZ-3108) Add support for external services to local mode

2016-03-26 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3108: Attachment: TEZ-3108.2.txt Updated patch with the earlier comment addressed, and local mode docs to

[jira] [Commented] (TEZ-3175) Add tez client submit host

2016-03-24 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210634#comment-15210634 ] Siddharth Seth commented on TEZ-3175: - Created TEZ-3188 to track the move out of TezConfiguration.

[jira] [Created] (TEZ-3188) Move tez.submit.hosts out of TezConfiguration

2016-03-24 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-3188: --- Summary: Move tez.submit.hosts out of TezConfiguration Key: TEZ-3188 URL: https://issues.apache.org/jira/browse/TEZ-3188 Project: Apache Tez Issue Type:

[jira] [Moved] (TEZ-3186) teztask event problem when running repeated queries on LLAP

2016-03-24 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth moved HIVE-13347 to TEZ-3186: Key: TEZ-3186 (was: HIVE-13347) Project: Apache Tez (was: Hive) >

[jira] [Commented] (TEZ-3108) Add support for external services to local mode

2016-03-23 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209342#comment-15209342 ] Siddharth Seth commented on TEZ-3108: - bq. External services and local mode seems a bit confusing. If

[jira] [Updated] (TEZ-3161) Allow task to report different kinds of errors - fatal / kill

2016-03-23 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3161: Attachment: TEZ-3161.3.txt Updated to fix the unit test. Can't do much about the deprecation. >

[jira] [Commented] (TEZ-3165) Parallel initialization of inputs, outputs, and processor can cause NoSuchMethodException

2016-03-23 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209205#comment-15209205 ] Siddharth Seth commented on TEZ-3165: - Also minor - please make the new boolean fields final before

[jira] [Updated] (TEZ-3165) Allow Inputs/Outputs to be initialized serially, control processor initialization relative to Inputs/Outputs

2016-03-23 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3165: Summary: Allow Inputs/Outputs to be initialized serially, control processor initialization relative

[jira] [Updated] (TEZ-3161) Allow task to report different kinds of errors - fatal / kill

2016-03-23 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3161: Attachment: TEZ-3161.2.txt Updated patch with findbugs and javac warnings fixed. > Allow task to

[jira] [Created] (TEZ-3185) Treat certain kinds of failures in task execution as fatal errors by default

2016-03-23 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-3185: --- Summary: Treat certain kinds of failures in task execution as fatal errors by default Key: TEZ-3185 URL: https://issues.apache.org/jira/browse/TEZ-3185 Project: Apache

[jira] [Created] (TEZ-3184) Make use of fail fatal method in various Inputs / Outputs

2016-03-23 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-3184: --- Summary: Make use of fail fatal method in various Inputs / Outputs Key: TEZ-3184 URL: https://issues.apache.org/jira/browse/TEZ-3184 Project: Apache Tez Issue

[jira] [Updated] (TEZ-3161) Allow task to report different kinds of errors - fatal / kill

2016-03-22 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3161: Attachment: TEZ-3161.1.txt First iteration of the patch. Allows a failure to be marked as

[jira] [Commented] (TEZ-3108) Add support for external services to local mode

2016-03-21 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205288#comment-15205288 ] Siddharth Seth commented on TEZ-3108: - Ping [~jeagles], [~hitesh] - could one of you please review the

[jira] [Commented] (TEZ-3175) Add tez client submit host

2016-03-21 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15204791#comment-15204791 ] Siddharth Seth commented on TEZ-3175: - Sorry for coming in late. Should we move such informational

[jira] [Commented] (TEZ-3166) Counters aren't fully updated and sent for failed tasks

2016-03-19 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1521#comment-1521 ] Siddharth Seth commented on TEZ-3166: - +1 for the latest patch. [~aplusplus] - could you please post a

[jira] [Updated] (TEZ-3108) Add support for external services to local mode

2016-03-19 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3108: Attachment: TEZ-3108.1.txt Allows local mode (in-proc am) to submit work to an external entity.

[jira] [Commented] (TEZ-3168) Provide a more predictable approach for total resource guidance for wave/split calculation

2016-03-19 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200446#comment-15200446 ] Siddharth Seth commented on TEZ-3168: - bq. Looking at the queue capacity could be very wrong in cases

[jira] [Updated] (TEZ-3166) Counters aren't fully updated and sent for failed tasks

2016-03-19 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3166: Summary: Counters aren't fully updated and sent for failed tasks (was: Counters aren't fully updated

[jira] [Commented] (TEZ-3168) Provide a more predictable approach for total resource guidance for wave/split calculation

2016-03-19 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198874#comment-15198874 ] Siddharth Seth commented on TEZ-3168: - Is the patch ready for review (the name says wip) ? Quickly

[jira] [Commented] (TEZ-3166) Counters aren't fully updated and sent for failed tasks

2016-03-19 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197741#comment-15197741 ] Siddharth Seth commented on TEZ-3166: - [~aplusplus] - there's a couple more cases which need to be

[jira] [Updated] (TEZ-3166) Counters aren't fully updated and sent for failed tasks

2016-03-19 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3166: Target Version/s: 0.7.1 > Counters aren't fully updated and sent for failed tasks >

[jira] [Commented] (TEZ-3166) Counters aren't fully updated and sent for failed tasks

2016-03-18 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200700#comment-15200700 ] Siddharth Seth commented on TEZ-3166: - +1. Committing. Thanks [~aplusplus]. I'm going to create a follow

[jira] [Commented] (TEZ-3165) Parallel initialization of inputs, outputs, and processor can cause NoSuchMethodException

2016-03-14 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15194399#comment-15194399 ] Siddharth Seth commented on TEZ-3165: - [~jeagles] - is this happening when relocalization kicks in ?

[jira] [Commented] (TEZ-2954) Container launch timeouts should count towards node blacklisting

2016-03-14 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15194355#comment-15194355 ] Siddharth Seth commented on TEZ-2954: - [~ozawa] - the problem is highlighted in

[jira] [Commented] (TEZ-2936) Create ATS implementation that enables support for YARN-4265

2016-03-14 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15194298#comment-15194298 ] Siddharth Seth commented on TEZ-2936: - +1. The 2.8 profile would be built as follows, correct ? mvn

[jira] [Commented] (TEZ-3164) Surface error histograms from the AM

2016-03-14 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15194133#comment-15194133 ] Siddharth Seth commented on TEZ-3164: - Big +1 for doing this. An external script could be used for such

[jira] [Commented] (TEZ-3162) Add a mechanism for tasks to provide some diagnostics information even on success

2016-03-10 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190167#comment-15190167 ] Siddharth Seth commented on TEZ-3162: - A specific file could be setup - that's something that apps could

[jira] [Commented] (TEZ-3162) Add a mechanism for tasks to provide some diagnostics information even on success

2016-03-10 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189977#comment-15189977 ] Siddharth Seth commented on TEZ-3162: - As an example, a structured log of what happened during task

[jira] [Created] (TEZ-3162) Add a mechanism for tasks to provide some diagnostics information even on success

2016-03-10 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-3162: --- Summary: Add a mechanism for tasks to provide some diagnostics information even on success Key: TEZ-3162 URL: https://issues.apache.org/jira/browse/TEZ-3162 Project:

[jira] [Commented] (TEZ-3161) Allow task to report different kinds of errors - fatal / kill

2016-03-10 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189975#comment-15189975 ] Siddharth Seth commented on TEZ-3161: - The same functionality could be added on other user plugin points

[jira] [Commented] (TEZ-3155) Support a way to submit DAGs to a session where the DAG plan exceeds hadoop ipc limits

2016-03-10 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189950#comment-15189950 ] Siddharth Seth commented on TEZ-3155: - A log line indicating that the dag plan is being serialized to

[jira] [Commented] (TEZ-3145) Reduce message size when empty partitions is high

2016-03-08 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15185900#comment-15185900 ] Siddharth Seth commented on TEZ-3145: - [~jeagles] - have you seen issues with the size of the events

[jira] [Commented] (TEZ-2954) Container launch timeouts should count towards node blacklisting

2016-03-07 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15183434#comment-15183434 ] Siddharth Seth commented on TEZ-2954: - [~ozawa] - I'll try looking at the patch by the end of the week.

[jira] [Updated] (TEZ-3152) Tez UI 2: Build fails when run by multiple users or when node_modules is old

2016-03-03 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3152: Attachment: err2 Still failing. Attached output of mvn clean install -X from the tez-ui2 directory

[jira] [Commented] (TEZ-3156) Tez client keep trying to talk to RM if RM does not know application

2016-03-03 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15178372#comment-15178372 ] Siddharth Seth commented on TEZ-3156: - +1. > Tez client keep trying to talk to RM if RM does not know

[jira] [Commented] (TEZ-3077) TezClient.waitTillReady should support timeout

2016-03-02 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176507#comment-15176507 ] Siddharth Seth commented on TEZ-3077: - Comments on the patch, and apologies for the late review. - We

[jira] [Commented] (TEZ-3115) Shuffle string handling adds significant memory overhead

2016-03-02 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176305#comment-15176305 ] Siddharth Seth commented on TEZ-3115: - +1. Thanks [~jeagles] Wasn't aware of the improvements to

[jira] [Commented] (TEZ-3140) Reduce AM memory usage while serialization

2016-03-01 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174892#comment-15174892 ] Siddharth Seth commented on TEZ-3140: - I think it's better to set this up in it's own test class - not

[jira] [Commented] (TEZ-3153) build uses tmp, that is problematic on a shared machine

2016-03-01 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174412#comment-15174412 ] Siddharth Seth commented on TEZ-3153: - cc [~Sreenath] > build uses tmp, that is problematic on a shared

[jira] [Commented] (TEZ-3115) Shuffle string handling adds significant memory overhead

2016-03-01 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174281#comment-15174281 ] Siddharth Seth commented on TEZ-3115: - I think the interning needs to be done via

[jira] [Commented] (TEZ-3140) Reduce AM memory usage while serialization

2016-03-01 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174181#comment-15174181 ] Siddharth Seth commented on TEZ-3140: - +1. Could you please move the test into a new file before commit

[jira] [Commented] (TEZ-3151) expose DAG credentials to plugins

2016-02-29 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172855#comment-15172855 ] Siddharth Seth commented on TEZ-3151: - +1. Looks good. Waiting for jenkins before committing. [~hitesh]

[jira] [Commented] (TEZ-3151) expose DAG credentials to plugins

2016-02-29 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172714#comment-15172714 ] Siddharth Seth commented on TEZ-3151: - [~sershe] - can we add a getCredentials api into DagInfo itself.

[jira] [Commented] (TEZ-3120) Remove TaskCommContext.getCurrentDagName, Identifier

2016-02-29 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172596#comment-15172596 ] Siddharth Seth commented on TEZ-3120: - [~ozawa] - from a very quick look, the patch looks good. I did

[jira] [Commented] (TEZ-3147) Intermediate mem-to-mem: Fix early exit when only one segment can fit into memory

2016-02-26 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170347#comment-15170347 ] Siddharth Seth commented on TEZ-3147: - +1 > Intermediate mem-to-mem: Fix early exit when only one

[jira] [Commented] (TEZ-3141) mapreduce.task.timeout is not translated to container heartbeat timeout

2016-02-26 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169837#comment-15169837 ] Siddharth Seth commented on TEZ-3141: - +1. The property is used in the AM in any case, so the runtime

[jira] [Commented] (TEZ-1911) MergeManager's unconditionalReserve() should check for memory limits before allocating memory to IntermediateMemoryToMemoryMerger

2016-02-26 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169536#comment-15169536 ] Siddharth Seth commented on TEZ-1911: - According to findbugs {code} Unsynchronized access at

[jira] [Commented] (TEZ-3128) Avoid stopping containers on the AM shutdown thread

2016-02-26 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169448#comment-15169448 ] Siddharth Seth commented on TEZ-3128: - +1. Committing. Thanks [~ozawa]. [~jlowe] - should this be pulled

[jira] [Commented] (TEZ-1911) MergeManager's unconditionalReserve() should check for memory limits before allocating memory to IntermediateMemoryToMemoryMerger

2016-02-26 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169442#comment-15169442 ] Siddharth Seth commented on TEZ-1911: - Please go ahead. > MergeManager's unconditionalReserve() should

[jira] [Commented] (TEZ-3128) Avoid stopping containers on the AM shutdown thread

2016-02-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168481#comment-15168481 ] Siddharth Seth commented on TEZ-3128: - [~ozawa] - looking at the scheduler, we already release all held

[jira] [Commented] (TEZ-1911) MergeManager's unconditionalReserve() should check for memory limits before allocating memory to IntermediateMemoryToMemoryMerger

2016-02-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168468#comment-15168468 ] Siddharth Seth commented on TEZ-1911: - +1. Thanks! > MergeManager's unconditionalReserve() should check

[jira] [Commented] (TEZ-3141) mapreduce.task.timeout is not translated to container heartbeat timeout

2016-02-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168438#comment-15168438 ] Siddharth Seth commented on TEZ-3141: - I completely forgot about the separate maps in DeprecatedKeys.

[jira] [Commented] (TEZ-1911) MergeManager's unconditionalReserve() should check for memory limits before allocating memory to IntermediateMemoryToMemoryMerger

2016-02-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168127#comment-15168127 ] Siddharth Seth commented on TEZ-1911: - bq. One more question - does the interrupt status have to be

[jira] [Commented] (TEZ-3114) Shuffle OOM due to EventMetaData flood

2016-02-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168124#comment-15168124 ] Siddharth Seth commented on TEZ-3114: - +1 > Shuffle OOM due to EventMetaData flood >

[jira] [Commented] (TEZ-1911) MergeManager's unconditionalReserve() should check for memory limits before allocating memory to IntermediateMemoryToMemoryMerger

2016-02-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15166911#comment-15166911 ] Siddharth Seth commented on TEZ-1911: - One more question - does the interrupt status have to be checked

[jira] [Commented] (TEZ-1911) MergeManager's unconditionalReserve() should check for memory limits before allocating memory to IntermediateMemoryToMemoryMerger

2016-02-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15166910#comment-15166910 ] Siddharth Seth commented on TEZ-1911: - {code} LOG.info(inputContext.getSourceVertexName() + ": " +

[jira] [Created] (TEZ-3138) Consider ignoring successful empty partition fetches in Shuffle failure handling

2016-02-24 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-3138: --- Summary: Consider ignoring successful empty partition fetches in Shuffle failure handling Key: TEZ-3138 URL: https://issues.apache.org/jira/browse/TEZ-3138 Project:

[jira] [Updated] (TEZ-3121) Allow exceptions thrown by service plugins to be non-fatal

2016-02-24 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3121: Target Version/s: 0.8.3 Fix Version/s: (was: 0.8.3) > Allow exceptions thrown by service

[jira] [Updated] (TEZ-3120) Remove TaskCommContext.getCurrentDagName, Identifier

2016-02-24 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3120: Target Version/s: 0.8.3 Fix Version/s: (was: 0.8.3) > Remove

[jira] [Commented] (TEZ-3132) Flaky test: TestContainerReuse.testReuseConflictLocalResources

2016-02-23 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159231#comment-15159231 ] Siddharth Seth commented on TEZ-3132: - There's another jira for TestContainerReuse - almost all of the

[jira] [Updated] (TEZ-3128) Avoid stopping containers on the AM shutdown thread

2016-02-23 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3128: Attachment: amJstack Here's the stack trace. TezContainerLauncher invokes shutdownAllContainers -

[jira] [Updated] (TEZ-3128) Avoid stopping containers on the AM shutdown thread

2016-02-23 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3128: Target Version/s: 0.8.3 > Avoid stopping containers on the AM shutdown thread >

[jira] [Created] (TEZ-3130) A bad NodeManager can end up occupying all container launcher threads, delaying new launches

2016-02-19 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-3130: --- Summary: A bad NodeManager can end up occupying all container launcher threads, delaying new launches Key: TEZ-3130 URL: https://issues.apache.org/jira/browse/TEZ-3130

[jira] [Commented] (TEZ-3128) Avoid stopping containers on the AM shutdown thread

2016-02-19 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154982#comment-15154982 ] Siddharth Seth commented on TEZ-3128: - Yep. That works. The timeouts while an app are running are also

[jira] [Created] (TEZ-3128) Avoid stopping containers on the AM shutdown thread

2016-02-19 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-3128: --- Summary: Avoid stopping containers on the AM shutdown thread Key: TEZ-3128 URL: https://issues.apache.org/jira/browse/TEZ-3128 Project: Apache Tez Issue Type:

[jira] [Commented] (TEZ-3123) Containers can get re-used even with conflicting local resources

2016-02-19 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154655#comment-15154655 ] Siddharth Seth commented on TEZ-3123: - +1. > Containers can get re-used even with conflicting local

[jira] [Commented] (TEZ-3123) Containers can get re-used even with conflicting local resources

2016-02-19 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153942#comment-15153942 ] Siddharth Seth commented on TEZ-3123: - The change to the scheduler looks good. I don't think the test

[jira] [Created] (TEZ-3121) Allow exceptions thrown by service plugins to be non-fatal

2016-02-17 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-3121: --- Summary: Allow exceptions thrown by service plugins to be non-fatal Key: TEZ-3121 URL: https://issues.apache.org/jira/browse/TEZ-3121 Project: Apache Tez

[jira] [Updated] (TEZ-3120) Remove TaskCommContext.getCurrentDagName, Identifier

2016-02-17 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3120: Fix Version/s: 0.8.3 Setting fix version temporarily - till the target version field is restored. >

<    2   3   4   5   6   7   8   9   10   11   >