[jira] [Commented] (MAPREDUCE-5831) Old MR client is not compatible with new MR application

[ https://issues.apache.org/jira/browse/MAPREDUCE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13977689#comment-13977689 ] Wangda Tan commented on MAPREDUCE-5831: --- Link this issue with MAPREDUCE-4150

[jira] [Commented] (MAPREDUCE-5844) Reducer Preemption is too aggressive

[ https://issues.apache.org/jira/browse/MAPREDUCE-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007184#comment-14007184 ] Wangda Tan commented on MAPREDUCE-5844: --- Hi [~maysamyabandeh], Thanks for your

[jira] [Commented] (MAPREDUCE-5196) CheckpointAMPreemptionPolicy implements preemption in MR AM via checkpointing

[ https://issues.apache.org/jira/browse/MAPREDUCE-5196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14016357#comment-14016357 ] Wangda Tan commented on MAPREDUCE-5196: --- Hi [~curino], While trying to

[jira] [Commented] (MAPREDUCE-5900) Container preemption interpreted as task failures and eventually job failures

[ https://issues.apache.org/jira/browse/MAPREDUCE-5900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14016365#comment-14016365 ] Wangda Tan commented on MAPREDUCE-5900: --- Hi [~mayank_bansal], Thanks for your

[jira] [Commented] (MAPREDUCE-5196) CheckpointAMPreemptionPolicy implements preemption in MR AM via checkpointing

[ https://issues.apache.org/jira/browse/MAPREDUCE-5196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018357#comment-14018357 ] Wangda Tan commented on MAPREDUCE-5196: --- Hi [~curino], Thanks for your

[jira] [Commented] (MAPREDUCE-5831) Old MR client is not compatible with new MR application

[ https://issues.apache.org/jira/browse/MAPREDUCE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039768#comment-14039768 ] Wangda Tan commented on MAPREDUCE-5831: --- +1 for migrating towards MAPREDUCE-4421

[jira] [Commented] (MAPREDUCE-5900) Container preemption interpreted as task failures and eventually job failures

[ https://issues.apache.org/jira/browse/MAPREDUCE-5900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14048406#comment-14048406 ] Wangda Tan commented on MAPREDUCE-5900: --- Make sense, LGTM, +1. Container

[jira] [Assigned] (MAPREDUCE-5956) MapReduce AM should not use maxAttempts to determine if this is the last retry

[ https://issues.apache.org/jira/browse/MAPREDUCE-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan reassigned MAPREDUCE-5956: - Assignee: Wangda Tan (was: Vinod Kumar Vavilapalli) MapReduce AM should not use

[jira] [Work started] (MAPREDUCE-5956) MapReduce AM should not use maxAttempts to determine if this is the last retry

[ https://issues.apache.org/jira/browse/MAPREDUCE-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on MAPREDUCE-5956 started by Wangda Tan. MapReduce AM should not use maxAttempts to determine if this is the last retry

[jira] [Commented] (MAPREDUCE-5956) MapReduce AM should not use maxAttempts to determine if this is the last retry

[ https://issues.apache.org/jira/browse/MAPREDUCE-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050924#comment-14050924 ] Wangda Tan commented on MAPREDUCE-5956: --- Assigned it to me, already started

[jira] [Commented] (MAPREDUCE-5956) MapReduce AM should not use maxAttempts to determine if this is the last retry

[ https://issues.apache.org/jira/browse/MAPREDUCE-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056026#comment-14056026 ] Wangda Tan commented on MAPREDUCE-5956: --- Thanks thoughts provided by [~vinodkv],

[jira] [Updated] (MAPREDUCE-5956) MapReduce AM should not use maxAttempts to determine if this is the last retry

[ https://issues.apache.org/jira/browse/MAPREDUCE-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-5956: -- Attachment: MR-5956.patch MR-5956.patch Attached a fix for this, please

[jira] [Commented] (MAPREDUCE-5956) MapReduce AM should not use maxAttempts to determine if this is the last retry

[ https://issues.apache.org/jira/browse/MAPREDUCE-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14057260#comment-14057260 ] Wangda Tan commented on MAPREDUCE-5956: --- Hi [~mayank_bansal], It should say, we

[jira] [Updated] (MAPREDUCE-5956) MapReduce AM should not use maxAttempts to determine if this is the last retry

[ https://issues.apache.org/jira/browse/MAPREDUCE-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-5956: -- Affects Version/s: 2.4.0 Status: Patch Available (was: In Progress)

[jira] [Updated] (MAPREDUCE-5956) MapReduce AM should not use maxAttempts to determine if this is the last retry

[ https://issues.apache.org/jira/browse/MAPREDUCE-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-5956: -- Attachment: (was: MR-5956.patch) MapReduce AM should not use maxAttempts to determine

[jira] [Commented] (MAPREDUCE-5956) MapReduce AM should not use maxAttempts to determine if this is the last retry

[ https://issues.apache.org/jira/browse/MAPREDUCE-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14058110#comment-14058110 ] Wangda Tan commented on MAPREDUCE-5956: --- Hi [~mayank_bansal] and [~hitesh],

[jira] [Updated] (MAPREDUCE-5956) MapReduce AM should not use maxAttempts to determine if this is the last retry

[ https://issues.apache.org/jira/browse/MAPREDUCE-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-5956: -- Attachment: MR-5956.patch Zhijie, thanks for review! Uploaded a patch addressed your

[jira] [Commented] (MAPREDUCE-5956) MapReduce AM should not use maxAttempts to determine if this is the last retry

[ https://issues.apache.org/jira/browse/MAPREDUCE-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14058565#comment-14058565 ] Wangda Tan commented on MAPREDUCE-5956: --- Thanks [~zjshen] for review and commit!

[jira] [Created] (MAPREDUCE-6002) MR task should prevent report error to AM when process is shutting down

Wangda Tan created MAPREDUCE-6002: - Summary: MR task should prevent report error to AM when process is shutting down Key: MAPREDUCE-6002 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6002

[jira] [Updated] (MAPREDUCE-6002) MR task should prevent report error to AM when process is shutting down

[ https://issues.apache.org/jira/browse/MAPREDUCE-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6002: -- Status: Patch Available (was: Open) MR task should prevent report error to AM when

[jira] [Updated] (MAPREDUCE-6002) MR task should prevent report error to AM when process is shutting down

[ https://issues.apache.org/jira/browse/MAPREDUCE-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6002: -- Attachment: MR-6002.patch Attached a patch for review. MR task should prevent report

[jira] [Commented] (MAPREDUCE-6002) MR task should prevent report error to AM when process is shutting down

[ https://issues.apache.org/jira/browse/MAPREDUCE-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074228#comment-14074228 ] Wangda Tan commented on MAPREDUCE-6002: --- Hi [~zjshen], Thanks for review I

[jira] [Commented] (MAPREDUCE-6002) MR task should prevent report error to AM when process is shutting down

[ https://issues.apache.org/jira/browse/MAPREDUCE-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075183#comment-14075183 ] Wangda Tan commented on MAPREDUCE-6002: --- Jason, thanks for your comments, bq. I

[jira] [Commented] (MAPREDUCE-5956) MapReduce AM should not use maxAttempts to determine if this is the last retry

[ https://issues.apache.org/jira/browse/MAPREDUCE-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14106425#comment-14106425 ] Wangda Tan commented on MAPREDUCE-5956: --- Hi [~ashwinshankar77], You can check my

[jira] [Updated] (MAPREDUCE-5831) Old MR client is not compatible with new MR application

[ https://issues.apache.org/jira/browse/MAPREDUCE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-5831: -- Assignee: Junping Du (was: Tan, Wangda) Old MR client is not compatible with new MR

[jira] [Created] (MAPREDUCE-6098) org.apache.hadoop.mapreduce.lib.input.TestMRCJCFileInputFormat intermittently failed in trunk

Wangda Tan created MAPREDUCE-6098: - Summary: org.apache.hadoop.mapreduce.lib.input.TestMRCJCFileInputFormat intermittently failed in trunk Key: MAPREDUCE-6098 URL:

[jira] [Commented] (MAPREDUCE-6130) Mapreduce tests fail with IllegalArgumentException in trunk

[ https://issues.apache.org/jira/browse/MAPREDUCE-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14175486#comment-14175486 ] Wangda Tan commented on MAPREDUCE-6130: --- Assigned to me, looking at it.

[jira] [Assigned] (MAPREDUCE-6130) Mapreduce tests fail with IllegalArgumentException in trunk

[ https://issues.apache.org/jira/browse/MAPREDUCE-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan reassigned MAPREDUCE-6130: - Assignee: Wangda Tan Mapreduce tests fail with IllegalArgumentException in trunk

[jira] [Commented] (MAPREDUCE-6130) Mapreduce tests fail with IllegalArgumentException in trunk

[ https://issues.apache.org/jira/browse/MAPREDUCE-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14175526#comment-14175526 ] Wangda Tan commented on MAPREDUCE-6130: --- Opened YARN-2705 for resolving default

[jira] [Updated] (MAPREDUCE-6212) Hadoop 2.6.0: Basic error “starting MRAppMaster” after installing

[ https://issues.apache.org/jira/browse/MAPREDUCE-6212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6212: -- Priority: Major (was: Critical) Hadoop 2.6.0: Basic error “starting MRAppMaster” after

[jira] [Updated] (MAPREDUCE-6212) UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative() happened when starting MRAppMaster

[ https://issues.apache.org/jira/browse/MAPREDUCE-6212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6212: -- Summary: UnsatisfiedLinkError:

[jira] [Commented] (MAPREDUCE-6212) Hadoop 2.6.0: Basic error “starting MRAppMaster” after installing

[ https://issues.apache.org/jira/browse/MAPREDUCE-6212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268354#comment-14268354 ] Wangda Tan commented on MAPREDUCE-6212: --- It's more like a configuration issue

[jira] [Resolved] (MAPREDUCE-6130) Mapreduce tests fail with IllegalArgumentException in trunk

[ https://issues.apache.org/jira/browse/MAPREDUCE-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved MAPREDUCE-6130. --- Resolution: Done Problem solved by YARN-2705 already. Mapreduce tests fail with

[jira] [Updated] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6304: -- Description: Per the discussion on YARN-796, we need a mechanism in MAPREDUCE to specify

[jira] [Created] (MAPREDUCE-6326) Cleanup ResourceManagerAdministrationProtocol interface audience

Wangda Tan created MAPREDUCE-6326: - Summary: Cleanup ResourceManagerAdministrationProtocol interface audience Key: MAPREDUCE-6326 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6326 Project:

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14515853#comment-14515853 ] Wangda Tan commented on MAPREDUCE-6304: --- bq. Well my view should be to support

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14523543#comment-14523543 ] Wangda Tan commented on MAPREDUCE-6304: --- [~Naganarasimha], Patch generally

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514669#comment-14514669 ] Wangda Tan commented on MAPREDUCE-6304: --- Hi [~Naganarasimha], Just took a look

[jira] [Updated] (MAPREDUCE-6302) deadlock in a job between map and reduce cores allocation

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6302: -- Status: Patch Available (was: Open) deadlock in a job between map and reduce cores

[jira] [Commented] (MAPREDUCE-6302) deadlock in a job between map and reduce cores allocation

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533618#comment-14533618 ] Wangda Tan commented on MAPREDUCE-6302: --- [~kasha], Thanks for working on this.

[jira] [Commented] (MAPREDUCE-6302) deadlock in a job between map and reduce cores allocation

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533619#comment-14533619 ] Wangda Tan commented on MAPREDUCE-6302: --- Linked to YARN-1680, one is for more

[jira] [Updated] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6304: -- Status: Open (was: Patch Available) Specifying node labels when submitting MR jobs

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14527488#comment-14527488 ] Wangda Tan commented on MAPREDUCE-6304: --- [~Naganarasimha], remove from

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14528957#comment-14528957 ] Wangda Tan commented on MAPREDUCE-6304: --- [~Naganarasimha], thanks for pointing

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539024#comment-14539024 ] Wangda Tan commented on MAPREDUCE-6304: --- Thanks [~Naganarasimha] for working on

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540701#comment-14540701 ] Wangda Tan commented on MAPREDUCE-6304: --- Thanks update, [~Naganarasimha], +1 for

[jira] [Updated] (MAPREDUCE-6302) deadlock in a job between map and reduce cores allocation

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6302: -- Target Version/s: 2.8.0 Mark its target version to be 2.8.0 deadlock in a job between map

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544102#comment-14544102 ] Wangda Tan commented on MAPREDUCE-6304: --- [~Naganarasimha], Actually now CS will

[jira] [Resolved] (MAPREDUCE-1439) Learning Scheduler

[ https://issues.apache.org/jira/browse/MAPREDUCE-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved MAPREDUCE-1439. --- Resolution: Not A Problem [~jaideep], thanks for sharing this, but JIRA is to track

[jira] [Reopened] (MAPREDUCE-1439) Learning Scheduler

[ https://issues.apache.org/jira/browse/MAPREDUCE-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan reopened MAPREDUCE-1439: --- Assignee: Jaideep My bad, this should be a new feature, reopen and assign to

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546230#comment-14546230 ] Wangda Tan commented on MAPREDUCE-6304: --- [~Naganarasimha], Thanks for replying,

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14545831#comment-14545831 ] Wangda Tan commented on MAPREDUCE-6304: --- [~Naganarasimha], I think I made a

[jira] [Commented] (MAPREDUCE-6302) deadlock in a job between map and reduce cores allocation

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14494812#comment-14494812 ] Wangda Tan commented on MAPREDUCE-6302: --- bq. In other words, I guess I am

[jira] [Commented] (MAPREDUCE-6189) TestMRTimelineEventHandling fails in trunk

[ https://issues.apache.org/jira/browse/MAPREDUCE-6189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485641#comment-14485641 ] Wangda Tan commented on MAPREDUCE-6189: --- bq. We do have

[jira] [Commented] (MAPREDUCE-6311) AM JVM hungs after job unregister and finished

[ https://issues.apache.org/jira/browse/MAPREDUCE-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485675#comment-14485675 ] Wangda Tan commented on MAPREDUCE-6311: --- This seems like a big issue, thanks for

[jira] [Commented] (MAPREDUCE-6302) deadlock in a job between map and reduce cores allocation

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14391208#comment-14391208 ] Wangda Tan commented on MAPREDUCE-6302: --- Moved to mapreduce. And [~shurong.mai],

[jira] [Moved] (MAPREDUCE-6302) deadlock in a job between map and reduce cores allocation

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan moved YARN-3416 to MAPREDUCE-6302: - Component/s: (was: fairscheduler) Affects Version/s:

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487941#comment-14487941 ] Wangda Tan commented on MAPREDUCE-6304: --- Just found I'm not in the watcher list,

[jira] [Updated] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6304: -- Resolution: Fixed Fix Version/s: 2.8.0 Hadoop Flags: Reviewed Status:

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561824#comment-14561824 ] Wangda Tan commented on MAPREDUCE-6304: --- Just committed to branch-2/trunk.

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14556454#comment-14556454 ] Wangda Tan commented on MAPREDUCE-6304: --- It's the last call of comments, I plan

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625631#comment-14625631 ] Wangda Tan commented on MAPREDUCE-6304: --- [~Naganarasimha], I just found the

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625618#comment-14625618 ] Wangda Tan commented on MAPREDUCE-6304: --- [~Naganarasimha], I just found there's

[jira] [Updated] (MAPREDUCE-6302) Preempt reducers after a configurable timeout irrespective of headroom

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6302: -- Target Version/s: 2.8.0, 2.6.3, 2.7.3 (was: 2.8.0) > Preempt reducers after a configurable

[jira] [Updated] (MAPREDUCE-6302) Preempt reducers after a configurable timeout irrespective of headroom

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6302: -- Attachment: MAPREDUCE-6302.branch-2.7.0001.patch

[jira] [Commented] (MAPREDUCE-6541) Exclude scheduled reducer memory when calculating available mapper slots from headroom to avoid deadlock

[ https://issues.apache.org/jira/browse/MAPREDUCE-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14996998#comment-14996998 ] Wangda Tan commented on MAPREDUCE-6541: --- [~varun_saxena], yes you're correct, updated

[jira] [Updated] (MAPREDUCE-6541) Exclude scheduled reducer memory when calculating available mapper slots from headroom to avoid deadlock

[ https://issues.apache.org/jira/browse/MAPREDUCE-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6541: -- Summary: Exclude scheduled reducer memory when calculating available mapper slots from

[jira] [Updated] (MAPREDUCE-6541) Exclude scheduled reducer memory when calculating available mapper slots from headroom to avoid deadlock

[ https://issues.apache.org/jira/browse/MAPREDUCE-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6541: -- Description: We saw a MR deadlock recently: - When NM restarted by framework without

[jira] [Commented] (MAPREDUCE-6302) Preempt reducers after a configurable timeout irrespective of headroom

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007092#comment-15007092 ] Wangda Tan commented on MAPREDUCE-6302: --- Update: We run jobs in test cluster with this fix for

[jira] [Commented] (MAPREDUCE-6541) Exclude pending reducer memory when calculating available mapper slots from headroom to avoid deadlock

[ https://issues.apache.org/jira/browse/MAPREDUCE-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994678#comment-14994678 ] Wangda Tan commented on MAPREDUCE-6541: --- [~Naganarasimha], that's also different: MAPREDUCE-6514

[jira] [Commented] (MAPREDUCE-6541) Exclude pending reducer memory when calculating available mapper slots from headroom to avoid deadlock

[ https://issues.apache.org/jira/browse/MAPREDUCE-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994582#comment-14994582 ] Wangda Tan commented on MAPREDUCE-6541: --- [~Naganarasimha], think for pointing me MAPREDUCE-6513,

[jira] [Commented] (MAPREDUCE-6513) MR job got hanged forever when one NM unstable for some time

[ https://issues.apache.org/jira/browse/MAPREDUCE-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994592#comment-14994592 ] Wangda Tan commented on MAPREDUCE-6513: --- I linked MAPREDUCE-6541 to this JIRA, they're different

[jira] [Commented] (MAPREDUCE-6541) Exclude pending reducer memory when calculating available mapper slots from headroom to avoid deadlock

[ https://issues.apache.org/jira/browse/MAPREDUCE-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994785#comment-14994785 ] Wangda Tan commented on MAPREDUCE-6541: --- Thanks for sharing your thoughts [~varun_saxena]. And

[jira] [Commented] (MAPREDUCE-6541) Exclude pending reducer memory when calculating available mapper slots from headroom to avoid deadlock

[ https://issues.apache.org/jira/browse/MAPREDUCE-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994507#comment-14994507 ] Wangda Tan commented on MAPREDUCE-6541: --- +[~jlowe], [~kasha]. > Exclude pending reducer memory

[jira] [Created] (MAPREDUCE-6541) Exclude pending reducer memory when calculating available mapper slots from headroom to avoid deadlock

Wangda Tan created MAPREDUCE-6541: - Summary: Exclude pending reducer memory when calculating available mapper slots from headroom to avoid deadlock Key: MAPREDUCE-6541 URL:

[jira] [Commented] (MAPREDUCE-6302) Preempt reducers after a configurable timeout irrespective of headroom

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961050#comment-14961050 ] Wangda Tan commented on MAPREDUCE-6302: --- +1 to backport this issue to 2.6.x and 2.7.x > Preempt

[jira] [Commented] (MAPREDUCE-6478) Add an option to skip cleanupJob stage or ignore cleanup failure during commitJob().

[ https://issues.apache.org/jira/browse/MAPREDUCE-6478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14790844#comment-14790844 ] Wangda Tan commented on MAPREDUCE-6478: --- Thanks [~djp], patch looks good, will wait for a few

[jira] [Updated] (MAPREDUCE-6478) Add an option to skip cleanupJob stage or ignore cleanup failure during commitJob().

[ https://issues.apache.org/jira/browse/MAPREDUCE-6478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6478: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status:

[jira] [Created] (MAPREDUCE-6574) MR AM should print host of failed tasks.

Wangda Tan created MAPREDUCE-6574: - Summary: MR AM should print host of failed tasks. Key: MAPREDUCE-6574 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6574 Project: Hadoop Map/Reduce

[jira] [Commented] (MAPREDUCE-6476) InvalidResourceException when Nodelabel don't have access to queue should be handled

[ https://issues.apache.org/jira/browse/MAPREDUCE-6476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15075343#comment-15075343 ] Wangda Tan commented on MAPREDUCE-6476: --- Looks good, +1, thanks [~bibinchundatt]. >

[jira] [Commented] (MAPREDUCE-6574) MR AM should print host of failed tasks.

[ https://issues.apache.org/jira/browse/MAPREDUCE-6574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15067083#comment-15067083 ] Wangda Tan commented on MAPREDUCE-6574: --- Hi [~mohdshahidkhan], Thanks for working on this, some

[jira] [Updated] (MAPREDUCE-6574) MR AM should print host of failed tasks.

[ https://issues.apache.org/jira/browse/MAPREDUCE-6574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6574: -- Assignee: Mohammad Shahid Khan > MR AM should print host of failed tasks. >

[jira] [Updated] (MAPREDUCE-6574) MR AM should print host of failed tasks.

[ https://issues.apache.org/jira/browse/MAPREDUCE-6574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6574: -- Status: Patch Available (was: Open) > MR AM should print host of failed tasks. >

[jira] [Commented] (MAPREDUCE-6476) InvalidResourceException when Nodelabel don't have access to queue should be handled

[ https://issues.apache.org/jira/browse/MAPREDUCE-6476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15073040#comment-15073040 ] Wangda Tan commented on MAPREDUCE-6476: --- Thanks [~bibinchundatt], Some comments: 1) You can put

[jira] [Updated] (MAPREDUCE-6574) MR AM should print host of failed tasks.

[ https://issues.apache.org/jira/browse/MAPREDUCE-6574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6574: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status:

[jira] [Commented] (MAPREDUCE-6476) InvalidResourceException when Nodelabel don't have access to queue should be handled

[ https://issues.apache.org/jira/browse/MAPREDUCE-6476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022729#comment-15022729 ] Wangda Tan commented on MAPREDUCE-6476: --- [~bibinchundatt], I think approach#1 seems more

[jira] [Updated] (MAPREDUCE-6476) Label-related invalid resource request exception should be properly handled by application

[ https://issues.apache.org/jira/browse/MAPREDUCE-6476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6476: -- Summary: Label-related invalid resource request exception should be properly handled by

[jira] [Commented] (MAPREDUCE-6579) Test failure : TestNetworkedJob

[ https://issues.apache.org/jira/browse/MAPREDUCE-6579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120898#comment-15120898 ] Wangda Tan commented on MAPREDUCE-6579: --- Thanks [~ajisakaa]/[~Naganarasimha], A couple of

[jira] [Commented] (MAPREDUCE-6579) Test failure : TestNetworkedJob

[ https://issues.apache.org/jira/browse/MAPREDUCE-6579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122804#comment-15122804 ] Wangda Tan commented on MAPREDUCE-6579: --- Agree, latest patch LGTM, thanks! > Test failure :

[jira] [Commented] (MAPREDUCE-6579) JobStatus#getFailureInfo should not output diagnostic information when the job is running

[ https://issues.apache.org/jira/browse/MAPREDUCE-6579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149092#comment-15149092 ] Wangda Tan commented on MAPREDUCE-6579: --- Apologize for the delays, I was on vacation in recent a

[jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs

[ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149292#comment-15149292 ] Wangda Tan commented on MAPREDUCE-6304: --- [~sunilg], I would suggest don't backport this issue,

[jira] [Commented] (MAPREDUCE-6579) JobStatus#getFailureInfo should not output diagnostic information when the job is running

[ https://issues.apache.org/jira/browse/MAPREDUCE-6579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151219#comment-15151219 ] Wangda Tan commented on MAPREDUCE-6579: --- Hi [~Naganarasimha], I think we need to fix both: -

[jira] [Commented] (MAPREDUCE-6579) JobStatus#getFailureInfo should not output diagnostic information when the job is running

[ https://issues.apache.org/jira/browse/MAPREDUCE-6579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152840#comment-15152840 ] Wangda Tan commented on MAPREDUCE-6579: --- [~Naganarasimha], [~ajisakaa]. I agree to only handle

[jira] [Updated] (MAPREDUCE-6579) JobStatus#getFailureInfo should not output diagnostic information when the job is running

[ https://issues.apache.org/jira/browse/MAPREDUCE-6579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6579: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status:

[jira] [Commented] (MAPREDUCE-6579) JobStatus#getFailureInfo should not output diagnostic information when the job is running

[ https://issues.apache.org/jira/browse/MAPREDUCE-6579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195842#comment-15195842 ] Wangda Tan commented on MAPREDUCE-6579: --- Hi [~ajisakaa], Sorry for my delays. I thought I have

[jira] [Commented] (MAPREDUCE-6513) MR job got hanged forever when one NM unstable for some time

[ https://issues.apache.org/jira/browse/MAPREDUCE-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237699#comment-15237699 ] Wangda Tan commented on MAPREDUCE-6513: --- Patch looks good to me, thanks [~varun_saxena]! > MR

[jira] [Updated] (MAPREDUCE-6302) Preempt reducers after a configurable timeout irrespective of headroom

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6302: -- Fix Version/s: 2.6.5 2.7.3 > Preempt reducers after a configurable

[jira] [Commented] (MAPREDUCE-6302) Preempt reducers after a configurable timeout irrespective of headroom

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233001#comment-15233001 ] Wangda Tan commented on MAPREDUCE-6302: --- Done committed to branch-2.6/2.7 > Preempt reducers

[jira] [Commented] (MAPREDUCE-6302) Preempt reducers after a configurable timeout irrespective of headroom

[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232987#comment-15232987 ] Wangda Tan commented on MAPREDUCE-6302: --- Apologize I forgot backporting patches to maintenance

[jira] [Commented] (MAPREDUCE-6108) ShuffleError OOM while reserving memory by MergeManagerImpl

[ https://issues.apache.org/jira/browse/MAPREDUCE-6108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281053#comment-15281053 ] Wangda Tan commented on MAPREDUCE-6108: --- [~kasha], [~vinodkv] is this still an issue in existing

  1   2   >