[jira] [Created] (YARN-7668) Remove unused variables from ContainerLocalizer

Ray Chiang created YARN-7668: Summary: Remove unused variables from ContainerLocalizer Key: YARN-7668 URL: https://issues.apache.org/jira/browse/YARN-7668 Project: Hadoop YARN Issue Type: Task

[jira] [Created] (YARN-7521) Add some misisng @VisibleForTesting annotations

Ray Chiang created YARN-7521: Summary: Add some misisng @VisibleForTesting annotations Key: YARN-7521 URL: https://issues.apache.org/jira/browse/YARN-7521 Project: Hadoop YARN Issue Type: Bug

Re: [DISCUSS] A final minor release off branch-2?

On 11/15/17 10:34 AM, Andrew Wang wrote: Hi Junping, On Wed, Nov 15, 2017 at 1:37 AM, Junping Du wrote: 3. Beside incompatibilities, there is also possible to have performance regressions (lower throughput, higher latency, slower job running, bigger memory footprint or

[jira] [Resolved] (YARN-6142) Support rolling upgrade between 2.x and 3.x

[ https://issues.apache.org/jira/browse/YARN-6142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang resolved YARN-6142. -- Resolution: Information Provided Fix Version/s: 3.0.0 Protobuf and JACC analysis done

[jira] [Created] (YARN-7322) Remove annotations from org.apache.hadoop.yarn.server classes

Ray Chiang created YARN-7322: Summary: Remove annotations from org.apache.hadoop.yarn.server classes Key: YARN-7322 URL: https://issues.apache.org/jira/browse/YARN-7322 Project: Hadoop YARN

[jira] [Created] (YARN-7219) Fix AllocateRequestProto difference between branch-2/branch-2.8 and trunk

Ray Chiang created YARN-7219: Summary: Fix AllocateRequestProto difference between branch-2/branch-2.8 and trunk Key: YARN-7219 URL: https://issues.apache.org/jira/browse/YARN-7219 Project: Hadoop YARN

Re: Branch merges and 3.0.0-beta1 scope

On 8/22/17 3:20 AM, Steve Loughran wrote: On 21 Aug 2017, at 22:22, Vinod Kumar Vavilapalli wrote: Steve, You can be strict & ruthless about the timelines. Anything that doesn’t get in by mid-September, as was originally planned, can move to the next release - whether

[jira] [Created] (YARN-6868) Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml

Ray Chiang created YARN-6868: Summary: Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml Key: YARN-6868 URL: https://issues.apache.org/jira/browse/YARN-6868 Project: Hadoop

[jira] [Created] (YARN-6798) NM startup failure with old state store due to version mismatch

Ray Chiang created YARN-6798: Summary: NM startup failure with old state store due to version mismatch Key: YARN-6798 URL: https://issues.apache.org/jira/browse/YARN-6798 Project: Hadoop YARN

[jira] [Created] (YARN-6717) [Umbrella] API related cleanup for Hadoop 3

Ray Chiang created YARN-6717: Summary: [Umbrella] API related cleanup for Hadoop 3 Key: YARN-6717 URL: https://issues.apache.org/jira/browse/YARN-6717 Project: Hadoop YARN Issue Type: Task

[jira] [Created] (YARN-6273) TestAMRMClient#testAllocationWithBlacklist fails intermittently

Ray Chiang created YARN-6273: Summary: TestAMRMClient#testAllocationWithBlacklist fails intermittently Key: YARN-6273 URL: https://issues.apache.org/jira/browse/YARN-6273 Project: Hadoop YARN

[jira] [Created] (YARN-6272) TestAMRMClient#testAMRMClientWithContainerResourceChange fails intermittently

Ray Chiang created YARN-6272: Summary: TestAMRMClient#testAMRMClientWithContainerResourceChange fails intermittently Key: YARN-6272 URL: https://issues.apache.org/jira/browse/YARN-6272 Project: Hadoop

[jira] [Resolved] (YARN-5230) Document FairScheduler's allowPreemptionFrom flag

[ https://issues.apache.org/jira/browse/YARN-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang resolved YARN-5230. -- Resolution: Duplicate > Document FairScheduler's allowPreemptionFrom f

[jira] [Resolved] (YARN-5644) Define exit code for allowing NodeManager health script to mar

[ https://issues.apache.org/jira/browse/YARN-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang resolved YARN-5644. -- Resolution: Duplicate > Define exit code for allowing NodeManager health script to

[jira] [Created] (YARN-5644) Define exit code for allowing NodeManager health script to mar

Ray Chiang created YARN-5644: Summary: Define exit code for allowing NodeManager health script to mar Key: YARN-5644 URL: https://issues.apache.org/jira/browse/YARN-5644 Project: Hadoop YARN

[jira] [Resolved] (YARN-5549) AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately

[ https://issues.apache.org/jira/browse/YARN-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang resolved YARN-5549. -- Resolution: Fixed Fix Version/s: (was: 3.0.0-alpha2) 2.8.0 Pushed

[jira] [Resolved] (YARN-5567) Fix script exit code checking in NodeHealthScriptRunner#reportHealthStatus

[ https://issues.apache.org/jira/browse/YARN-5567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang resolved YARN-5567. -- Resolution: Fixed Hadoop Flags: Incompatible change,Reviewed (was: Reviewed) Release Note

[jira] [Created] (YARN-5595) Update documentation and Javadoc to match change to NodeHealthScriptRunner#reportHealthStatus

Ray Chiang created YARN-5595: Summary: Update documentation and Javadoc to match change to NodeHealthScriptRunner#reportHealthStatus Key: YARN-5595 URL: https://issues.apache.org/jira/browse/YARN-5595

[jira] [Created] (YARN-5564) Fix typo in .RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE

Ray Chiang created YARN-5564: Summary: Fix typo in .RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE Key: YARN-5564 URL: https://issues.apache.org/jira/browse/YARN-5564 Project: Hadoop YARN

[jira] [Created] (YARN-5529) Create new DiskValidator class with metrics

Ray Chiang created YARN-5529: Summary: Create new DiskValidator class with metrics Key: YARN-5529 URL: https://issues.apache.org/jira/browse/YARN-5529 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Created] (YARN-5495) Clean up imports in CapacityScheduler

Ray Chiang created YARN-5495: Summary: Clean up imports in CapacityScheduler Key: YARN-5495 URL: https://issues.apache.org/jira/browse/YARN-5495 Project: Hadoop YARN Issue Type: Task

[jira] [Created] (YARN-5399) Add configuration to remember ad-hoc queues upon configuration reload

Ray Chiang created YARN-5399: Summary: Add configuration to remember ad-hoc queues upon configuration reload Key: YARN-5399 URL: https://issues.apache.org/jira/browse/YARN-5399 Project: Hadoop YARN

[jira] [Created] (YARN-5398) Standardize whitespace trimming and splitting in FairScheduler code

Ray Chiang created YARN-5398: Summary: Standardize whitespace trimming and splitting in FairScheduler code Key: YARN-5398 URL: https://issues.apache.org/jira/browse/YARN-5398 Project: Hadoop YARN

[jira] [Created] (YARN-5397) [Umbrella] Usability improvements in FairScheduler

Ray Chiang created YARN-5397: Summary: [Umbrella] Usability improvements in FairScheduler Key: YARN-5397 URL: https://issues.apache.org/jira/browse/YARN-5397 Project: Hadoop YARN Issue Type

Minor fix request on Jenkins web page link

For the Jenkins jobs: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/ https://builds.apache.org/job/PreCommit-YARN-Build/ The link to PreCommit-Admin is incorrect. There's an extra "/hudson" in the path. -Ray

[jira] [Created] (YARN-5285) Refactor common code in initScheduler across schedulers

Ray Chiang created YARN-5285: Summary: Refactor common code in initScheduler across schedulers Key: YARN-5285 URL: https://issues.apache.org/jira/browse/YARN-5285 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-5284) Refactor handle across schedulers

Ray Chiang created YARN-5284: Summary: Refactor handle across schedulers Key: YARN-5284 URL: https://issues.apache.org/jira/browse/YARN-5284 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Created] (YARN-5283) Refactor container assignment into AbstractYarnScheduler#assignContainers

Ray Chiang created YARN-5283: Summary: Refactor container assignment into AbstractYarnScheduler#assignContainers Key: YARN-5283 URL: https://issues.apache.org/jira/browse/YARN-5283 Project: Hadoop YARN

[jira] [Created] (YARN-5282) Fix typos in CapacityScheduler documentation

Ray Chiang created YARN-5282: Summary: Fix typos in CapacityScheduler documentation Key: YARN-5282 URL: https://issues.apache.org/jira/browse/YARN-5282 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-5137) Make DiskChecker pluggable

Ray Chiang created YARN-5137: Summary: Make DiskChecker pluggable Key: YARN-5137 URL: https://issues.apache.org/jira/browse/YARN-5137 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Created] (YARN-5129) Update FairScheduler to use SchedulerHealth

Ray Chiang created YARN-5129: Summary: Update FairScheduler to use SchedulerHealth Key: YARN-5129 URL: https://issues.apache.org/jira/browse/YARN-5129 Project: Hadoop YARN Issue Type: Task

[jira] [Created] (YARN-5128) Investigate potential race condition in the scheuduler nodeUpdate() method

Ray Chiang created YARN-5128: Summary: Investigate potential race condition in the scheuduler nodeUpdate() method Key: YARN-5128 URL: https://issues.apache.org/jira/browse/YARN-5128 Project: Hadoop YARN

Re: Different JIRA permissions for HADOOP and HDFS

Similar issue for me. I can't modify HDFS and HADOOP JIRAs now. -Ray On Tue, May 17, 2016 at 9:54 AM, Varun Saxena wrote: > Hi, > > I do not have permissions for any project other than YARN as well. Unable > to assign a JIRA in MAPREDUCE for instance. > Can somebody

[jira] [Created] (YARN-5078) [Umbrella] NodeManager health checker improvements

Ray Chiang created YARN-5078: Summary: [Umbrella] NodeManager health checker improvements Key: YARN-5078 URL: https://issues.apache.org/jira/browse/YARN-5078 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-5047) Refactor nodeUpdate() from FairScheduler and CapacityScheduler

Ray Chiang created YARN-5047: Summary: Refactor nodeUpdate() from FairScheduler and CapacityScheduler Key: YARN-5047 URL: https://issues.apache.org/jira/browse/YARN-5047 Project: Hadoop YARN

[jira] [Created] (YARN-5046) [Umbrella] Refactor scheduler code

Ray Chiang created YARN-5046: Summary: [Umbrella] Refactor scheduler code Key: YARN-5046 URL: https://issues.apache.org/jira/browse/YARN-5046 Project: Hadoop YARN Issue Type: Task

[jira] [Created] (YARN-4911) Bad placement policy in FairScheduler causes the RM to crash

Ray Chiang created YARN-4911: Summary: Bad placement policy in FairScheduler causes the RM to crash Key: YARN-4911 URL: https://issues.apache.org/jira/browse/YARN-4911 Project: Hadoop YARN

[jira] [Created] (YARN-4704) TestResourceManager#testResourceAllocation() fails when using FairScheduler

Ray Chiang created YARN-4704: Summary: TestResourceManager#testResourceAllocation() fails when using FairScheduler Key: YARN-4704 URL: https://issues.apache.org/jira/browse/YARN-4704 Project: Hadoop YARN

[jira] [Created] (YARN-4579) Allow container directory permissions to be configurable

Ray Chiang created YARN-4579: Summary: Allow container directory permissions to be configurable Key: YARN-4579 URL: https://issues.apache.org/jira/browse/YARN-4579 Project: Hadoop YARN Issue

[jira] [Created] (YARN-4569) Remove incorrect documentation on maxResources in FairScheduler

Ray Chiang created YARN-4569: Summary: Remove incorrect documentation on maxResources in FairScheduler Key: YARN-4569 URL: https://issues.apache.org/jira/browse/YARN-4569 Project: Hadoop YARN

[jira] [Created] (YARN-4568) Fix message when NodeManager runs into errors initializing the recovery directory

Ray Chiang created YARN-4568: Summary: Fix message when NodeManager runs into errors initializing the recovery directory Key: YARN-4568 URL: https://issues.apache.org/jira/browse/YARN-4568 Project

[jira] [Created] (YARN-4560) Make

Ray Chiang created YARN-4560: Summary: Make Key: YARN-4560 URL: https://issues.apache.org/jira/browse/YARN-4560 Project: Hadoop YARN Issue Type: Bug Components: yarn Affects

[jira] [Created] (YARN-4541) Change log message in LocalizedResource#handle() to DEBUG

Ray Chiang created YARN-4541: Summary: Change log message in LocalizedResource#handle() to DEBUG Key: YARN-4541 URL: https://issues.apache.org/jira/browse/YARN-4541 Project: Hadoop YARN Issue

[jira] [Resolved] (YARN-4406) RM Web UI continues to show decommissioned nodes even after RM restart

[ https://issues.apache.org/jira/browse/YARN-4406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang resolved YARN-4406. -- Resolution: Duplicate > RM Web UI continues to show decommissioned nodes even after RM rest

[jira] [Created] (YARN-4406) RM Web UI continues to show decommissioned nodes even after RM restart

Ray Chiang created YARN-4406: Summary: RM Web UI continues to show decommissioned nodes even after RM restart Key: YARN-4406 URL: https://issues.apache.org/jira/browse/YARN-4406 Project: Hadoop YARN

Re: Reservation ambiguity

Sorry for the delayed response. My two cents: 1) Hold has some ambiguity as a word, since it can be both a noun and a verb. Using hold in a sentence will be confusing in documentation. 2) Here are some alternatives. Sadly, a lot of these words are quite long and could be painful for coding:

[jira] [Created] (YARN-3912) Fix typos in hadoop-yarn-project module

Ray Chiang created YARN-3912: Summary: Fix typos in hadoop-yarn-project module Key: YARN-3912 URL: https://issues.apache.org/jira/browse/YARN-3912 Project: Hadoop YARN Issue Type: Task

[jira] [Created] (YARN-3823) Fix mismatch in default values for yarn.scheduler.maximum-allocation-vcores property

Ray Chiang created YARN-3823: Summary: Fix mismatch in default values for yarn.scheduler.maximum-allocation-vcores property Key: YARN-3823 URL: https://issues.apache.org/jira/browse/YARN-3823 Project

[jira] [Created] (YARN-3825) Add automatic search of default Configuration variables to TestConfigurationFieldsBase

Ray Chiang created YARN-3825: Summary: Add automatic search of default Configuration variables to TestConfigurationFieldsBase Key: YARN-3825 URL: https://issues.apache.org/jira/browse/YARN-3825 Project

[jira] [Created] (YARN-3824) Fix two minor nits in member variable properties of YarnConfiguration

Ray Chiang created YARN-3824: Summary: Fix two minor nits in member variable properties of YarnConfiguration Key: YARN-3824 URL: https://issues.apache.org/jira/browse/YARN-3824 Project: Hadoop YARN

New unit tests for *-default.xml verification

I was going to update the Wiki with the information, but it seems to be down for me at the moment. A few people have already bumped into some issues, so I'm informing via the *-dev lists for now. In trunk, there are three new unit tests: TestMapreduceConfigFields (MAPREDUCE-6192)

Re: [DISCUSS] Developing features in branches

Following up on Zhijie's comments, there's nothing to prevent periodically pulling updates from the main branch (e.g. branch-2 or trunk) into the feature branch, is there? Or cherry-picking some changes to alleviate conflict management during branch merging? I've seen other projects use one of

Re: Removal of unused properties

We have done this before where the properties were documented in the .xml file, but didn't exist anywhere in the Configuration files or the rest of the code. HDFS-7566 (committed) YARN-2460 (committed) MAPREDUCE-6057 (pending) -Ray On Thu, Apr 9, 2015 at 4:33 AM, Akira AJISAKA

[jira] [Created] (YARN-3182) Cleanup switch statement in ApplicationMasterLauncher#handle()

Ray Chiang created YARN-3182: Summary: Cleanup switch statement in ApplicationMasterLauncher#handle() Key: YARN-3182 URL: https://issues.apache.org/jira/browse/YARN-3182 Project: Hadoop YARN

[jira] [Created] (YARN-3179) Update use of Iterator to Iterable

Ray Chiang created YARN-3179: Summary: Update use of Iterator to Iterable Key: YARN-3179 URL: https://issues.apache.org/jira/browse/YARN-3179 Project: Hadoop YARN Issue Type: Improvement

[jira] [Resolved] (YARN-3107) Update TestYarnConfigurationFields to flag missing properties in yarn-default.xml with an error

[ https://issues.apache.org/jira/browse/YARN-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang resolved YARN-3107. -- Resolution: Duplicate Merge code changes into YARN-3069 instead. Update TestYarnConfigurationFields

[jira] [Created] (YARN-3107) Update TestYarnConfigurationFields to flag missing properties in yarn-default.xml with an error

Ray Chiang created YARN-3107: Summary: Update TestYarnConfigurationFields to flag missing properties in yarn-default.xml with an error Key: YARN-3107 URL: https://issues.apache.org/jira/browse/YARN-3107

[jira] [Created] (YARN-3105) Add generic state transition metrics to existing framework

Ray Chiang created YARN-3105: Summary: Add generic state transition metrics to existing framework Key: YARN-3105 URL: https://issues.apache.org/jira/browse/YARN-3105 Project: Hadoop YARN Issue

[jira] [Created] (YARN-3069) Document missing properties in yarn-default.xml

Ray Chiang created YARN-3069: Summary: Document missing properties in yarn-default.xml Key: YARN-3069 URL: https://issues.apache.org/jira/browse/YARN-3069 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-2957) Create unit test to automatically compare YarnConfiguration and yarn-default.xml

Ray Chiang created YARN-2957: Summary: Create unit test to automatically compare YarnConfiguration and yarn-default.xml Key: YARN-2957 URL: https://issues.apache.org/jira/browse/YARN-2957 Project: Hadoop

[jira] [Created] (YARN-2927) YARN InMemorySCMStore properties need fixing

Ray Chiang created YARN-2927: Summary: YARN InMemorySCMStore properties need fixing Key: YARN-2927 URL: https://issues.apache.org/jira/browse/YARN-2927 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-2868) Add metric for initial container launch time

Ray Chiang created YARN-2868: Summary: Add metric for initial container launch time Key: YARN-2868 URL: https://issues.apache.org/jira/browse/YARN-2868 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-2610) Hamlet doesn't close table tags

Ray Chiang created YARN-2610: Summary: Hamlet doesn't close table tags Key: YARN-2610 URL: https://issues.apache.org/jira/browse/YARN-2610 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-2460) Remove obsolete entries from yarn-default.xml

Ray Chiang created YARN-2460: Summary: Remove obsolete entries from yarn-default.xml Key: YARN-2460 URL: https://issues.apache.org/jira/browse/YARN-2460 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-2461) Fix PROCFS_USE_SMAPS_BASED_RSS_ENABLED property in YarnConfiguration

Ray Chiang created YARN-2461: Summary: Fix PROCFS_USE_SMAPS_BASED_RSS_ENABLED property in YarnConfiguration Key: YARN-2461 URL: https://issues.apache.org/jira/browse/YARN-2461 Project: Hadoop YARN

[jira] [Created] (YARN-2450) Fix typos in log messages

Ray Chiang created YARN-2450: Summary: Fix typos in log messages Key: YARN-2450 URL: https://issues.apache.org/jira/browse/YARN-2450 Project: Hadoop YARN Issue Type: Bug Affects Versions

[jira] [Resolved] (YARN-2275) When log aggregation not enabled, message should point to NM HTTP port, not IPC port

[ https://issues.apache.org/jira/browse/YARN-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang resolved YARN-2275. -- Resolution: Won't Fix Unable to fix this using a single Configuration property. Patch which hacks

[jira] [Created] (YARN-2284) Find missing config options in YarnConfiguration and yarn-default.xml

Ray Chiang created YARN-2284: Summary: Find missing config options in YarnConfiguration and yarn-default.xml Key: YARN-2284 URL: https://issues.apache.org/jira/browse/YARN-2284 Project: Hadoop YARN

[jira] [Created] (YARN-2201) TestRMWebServicesAppsModification dependent on yarn-default.xml

Ray Chiang created YARN-2201: Summary: TestRMWebServicesAppsModification dependent on yarn-default.xml Key: YARN-2201 URL: https://issues.apache.org/jira/browse/YARN-2201 Project: Hadoop YARN

[jira] [Created] (YARN-2159) allocateContainer() in SchedulerNode needs a clearer LOG.info message

Ray Chiang created YARN-2159: Summary: allocateContainer() in SchedulerNode needs a clearer LOG.info message Key: YARN-2159 URL: https://issues.apache.org/jira/browse/YARN-2159 Project: Hadoop YARN