[jira] [Resolved] (YARN-8849) DynoYARN: A simulation and testing infrastructure for YARN clusters

2021-09-20 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung resolved YARN-8849. - Resolution: Fixed FYI we have open source DynoYARN on Github: https://github.com/linkedin

[jira] [Created] (YARN-10297) TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently

2020-05-29 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-10297: Summary: TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently Key: YARN-10297 URL: https://issues.apache.org/jira/browse/YARN-10297

[jira] [Created] (YARN-10263) Application summary is logged multiple times due to RM recovery

2020-05-11 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-10263: Summary: Application summary is logged multiple times due to RM recovery Key: YARN-10263 URL: https://issues.apache.org/jira/browse/YARN-10263 Project: Hadoop YARN

[jira] [Created] (YARN-10260) Allow transitioning queue from DRAINING to RUNNING state

2020-05-06 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-10260: Summary: Allow transitioning queue from DRAINING to RUNNING state Key: YARN-10260 URL: https://issues.apache.org/jira/browse/YARN-10260 Project: Hadoop YARN

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2020-04-16 Thread Jonathan Hung
Source code has been deleted from branch-2. Thanks Akira for taking this up! Jonathan Hung On Thu, Apr 16, 2020 at 11:40 AM Jonathan Hung wrote: > Makes sense. I've cherry-picked the commits in branch-2 that were missed > in branch-2.10. > > Jonathan Hung > > > On Wed, Ap

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2020-04-16 Thread Jonathan Hung
Makes sense. I've cherry-picked the commits in branch-2 that were missed in branch-2.10. Jonathan Hung On Wed, Apr 15, 2020 at 2:25 AM Akira Ajisaka wrote: > Hi folks, > > I am still seeing some changes are being committed to branch-2. > I'd like to delete the source code f

[jira] [Created] (YARN-10212) Create separate configuration for max global AM attempts

2020-03-27 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-10212: Summary: Create separate configuration for max global AM attempts Key: YARN-10212 URL: https://issues.apache.org/jira/browse/YARN-10212 Project: Hadoop YARN

[jira] [Created] (YARN-10200) Add number of containers to RMAppManager summary

2020-03-16 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-10200: Summary: Add number of containers to RMAppManager summary Key: YARN-10200 URL: https://issues.apache.org/jira/browse/YARN-10200 Project: Hadoop YARN Issue

[jira] [Created] (YARN-10192) CapacityScheduler stuck in loop rejecting allocation proposals

2020-03-11 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-10192: Summary: CapacityScheduler stuck in loop rejecting allocation proposals Key: YARN-10192 URL: https://issues.apache.org/jira/browse/YARN-10192 Project: Hadoop YARN

[jira] [Created] (YARN-10134) Periodically sync backend scheduler configuration changes to capacity-scheduler.xml

2020-02-12 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-10134: Summary: Periodically sync backend scheduler configuration changes to capacity-scheduler.xml Key: YARN-10134 URL: https://issues.apache.org/jira/browse/YARN-10134

[jira] [Created] (YARN-10116) Expose diagnostics in RMAppManager summary

2020-02-04 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-10116: Summary: Expose diagnostics in RMAppManager summary Key: YARN-10116 URL: https://issues.apache.org/jira/browse/YARN-10116 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-10039) Allow disabling app submission from REST endpoints

2019-12-17 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-10039: Summary: Allow disabling app submission from REST endpoints Key: YARN-10039 URL: https://issues.apache.org/jira/browse/YARN-10039 Project: Hadoop YARN Issue

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-12-09 Thread Jonathan Hung
old branch-2.10 - Rename branch-2 to (new) branch-2.10 - Set version in new branch-2.10 to 2.10.1-SNAPSHOT - Renamed fix versions from 2.11.0 to 2.10.1 - Removed 2.11.0 as a version in HADOOP/YARN/HDFS/MAPREDUCE Jonathan Hung On Wed, Dec 4, 2019 at 10:55 AM Jonathan Hung wrote: > FYI, s

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-12-04 Thread Jonathan Hung
FYI, starting the rename process, beginning with INFRA-19521. Jonathan Hung On Wed, Nov 27, 2019 at 12:15 PM Konstantin Shvachko wrote: > Hey guys, > > I think we diverged a bit from the initial topic of this discussion, which > is removing branch-2.10, and changing the version

[jira] [Created] (YARN-10012) Guaranteed and max capacity queue metrics for custom resources

2019-12-03 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-10012: Summary: Guaranteed and max capacity queue metrics for custom resources Key: YARN-10012 URL: https://issues.apache.org/jira/browse/YARN-10012 Project: Hadoop YARN

[jira] [Created] (YARN-9992) Max allocation per queue is zero for custom resource types on RM startup

2019-11-26 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9992: --- Summary: Max allocation per queue is zero for custom resource types on RM startup Key: YARN-9992 URL: https://issues.apache.org/jira/browse/YARN-9992 Project: Hadoop

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-11-21 Thread Jonathan Hung
to use java 11 on branch-2, we can always revive branch-2. But for now I think the convenience of not needing to port to both branch-2 and branch-2.10 (and below) outweighs the cost of potentially needing to revive branch-2. Jonathan Hung On Wed, Nov 20, 2019 at 10:50 AM Eric Yang wrote: >

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-11-18 Thread Jonathan Hung
. But this proposal would reduce a lot of confusion IMO. Jonathan Hung On Fri, Nov 15, 2019 at 11:41 AM epa...@apache.org wrote: > Thanks Jonathan for opening the discussion. > > I am not in favor of this proposal. 2.10 was very recently released, and > moving to 2.10 will take some time for t

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-11-14 Thread Jonathan Hung
Some other additional items we would need: - Mark all fix-versions in YARN/HDFS/MAPREDUCE/HADOOP from 2.11.0 to 2.10.1 - Remove 2.11.0 as a version in these projects Jonathan Hung On Thu, Nov 14, 2019 at 6:51 PM Jonathan Hung wrote: > Hi folks, > > Given the release

[DISCUSS] Making 2.10 the last minor 2.x release

2019-11-14 Thread Jonathan Hung
This way we get all the current branch-2 fixes into the 2.10.x release line. Then the commit chain will look like: trunk -> branch-3.2 -> branch-3.1 -> branch-2.10 -> branch-2.9 -> branch-2.8 Thoughts? Jonathan Hung [1] https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg29479.html

[jira] [Created] (YARN-9964) Queue metrics turn negative when relabeling a node with running containers to default partition

2019-11-07 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9964: --- Summary: Queue metrics turn negative when relabeling a node with running containers to default partition Key: YARN-9964 URL: https://issues.apache.org/jira/browse/YARN-9964

[jira] [Created] (YARN-9954) Configurable max application tags and max tag length

2019-11-05 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9954: --- Summary: Configurable max application tags and max tag length Key: YARN-9954 URL: https://issues.apache.org/jira/browse/YARN-9954 Project: Hadoop YARN Issue

[ANNOUNCE] Apache Hadoop 2.10.0 release

2019-10-31 Thread Jonathan Hung
Hi all, I am happy to announce that the Apache Hadoop 2.10.0 has been released. Apache Hadoop 2.10.0 is the first release in the Apache Hadoop 2.10 line. The release details, including links to downloads, list of major features, release notes, and changelog, are on the 2.10.0 announcement page

[jira] [Created] (YARN-9945) Fix javadoc in FederationProxyProviderUtil in branch-2

2019-10-31 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9945: --- Summary: Fix javadoc in FederationProxyProviderUtil in branch-2 Key: YARN-9945 URL: https://issues.apache.org/jira/browse/YARN-9945 Project: Hadoop YARN Issue

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-29 Thread Jonathan Hung
+1 from me too. The vote passed, so I'll continue with the rest of the release. Thanks everyone! Jonathan Hung On Tue, Oct 29, 2019 at 1:40 PM Giovanni Matteo Fumarola < giovanni.fumar...@gmail.com> wrote: > +1 (non-binding). > > - Built from source on Ubuntu with OpenJDK 11.0

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-28 Thread Jonathan Hung
Thanks Eric! I sent out an RC1 earlier last week, not sure if you saw that. The only diff between RC1 and RC0 is HDFS-14667. If RC1 looks good to you then it'd be great to get your testing results on that thread. Jonathan Hung On Mon, Oct 28, 2019 at 1:06 PM epa...@apache.org wrote

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-26 Thread Jonathan Hung
Hi Eric, I took a quick look, are you using mapreduce.application.framework.path to run your MR jobs? If not, this seems like expected behavior if AM and tasks get launched on different NMs with different locally installed hadoop versions? Jonathan Hung On Sat, Oct 26, 2019 at 8:55 AM epa

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-25 Thread Jonathan Hung
be up to date, but I think that's fine. I'll go ahead with this plan if no objections. Jonathan Hung On Fri, Oct 25, 2019 at 12:19 PM Jonathan Hung wrote: > Thanks for looking Erik. > > For the release notes, yeah I think it's because there's no release notes > for the corresponding

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-25 Thread Jonathan Hung
to be correct. I think we could easily overwrite the current RC1 artifacts with ones containing proper release notes. Not sure what to do about the javadoc issue though, that would require overwriting the release-2.10.0-RC1 tag which I don't want to do. What do others think? Jonathan Hung On Fri

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-23 Thread Jonathan Hung
in dominant resource calculator, but root cause remains to be seen. Jonathan Hung On Wed, Oct 23, 2019 at 9:02 AM epa...@apache.org wrote: > Hi Jonathan, > > Thanks very much for all of your work on this release. > > I have a concern about cross-queue (inter-queue) preemption in 2

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-22 Thread Jonathan Hung
Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and 2.10.0 clients and datanodes. Everything worked as expected. Jonathan Hung On Tue, Oct 22, 2019 at 3:04 PM Eric Badger wrote: > Hi Jonathan, > > Thanks for putting this RC together. You stated that there are >

[VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-22 Thread Jonathan Hung
://repository.apache.org/content/repositories/orgapachehadoop-1243/ My public key is available here: https://dist.apache.org/repos/dist/release/hadoop/common/KEYS The vote will run for 5 weekdays, until Tuesday, October 29 at 3:00 pm PDT. Thanks, Jonathan Hung [1] https://issues.apache.org/jira/issues/?jql

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-22 Thread Jonathan Hung
Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar (HDFS-14667). Since this is the first of a minor release, we would like to get it into 2.10.0. HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1 shortly. Jonathan Hung On Tue, Oct 22, 2019 at 1:39

[VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-16 Thread Jonathan Hung
/repositories/orgapachehadoop-1241/ My public key is available here: https://dist.apache.org/repos/dist/release/hadoop/common/KEYS The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm PDT. Thanks, Jonathan Hung [1] https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C

Re: [DISCUSS] Hadoop 2.10.0 release plan

2019-10-16 Thread Jonathan Hung
I've moved all jiras with target version 2.10.0 to 2.10.1. Also I've created branch-2.10 and branch-2.10.0, please commit any 2.10.x bug fixes to branch-2.10. I'll send out a vote thread for 2.10.0-RC0 shortly. Jonathan Hung On Fri, Oct 11, 2019 at 10:32 AM Jonathan Hung wrote: > Edit: se

Re: [DISCUSS] Hadoop 2.10.0 release plan

2019-10-11 Thread Jonathan Hung
Edit: seems a 2.10.0 blocker was reopened (HDFS-14305). I'll continue watching this jira and start the release once this is resolved. Jonathan Hung On Thu, Oct 10, 2019 at 5:13 PM Jonathan Hung wrote: > Hi folks, as of now all 2.10.0 blockers have been resolved [1]. So I'll >

Re: [DISCUSS] Hadoop 2.10.0 release plan

2019-10-10 Thread Jonathan Hung
Hi folks, as of now all 2.10.0 blockers have been resolved [1]. So I'll start the release process soon (cutting branches, updating target versions, etc). [1] https://issues.apache.org/jira/issues/?filter=12346975 Jonathan Hung On Mon, Aug 26, 2019 at 10:19 AM Jonathan Hung wrote: > Hi fo

[jira] [Created] (YARN-9869) Create scheduling policy to auto-adjust queue elasticity based on cluster demand

2019-09-30 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9869: --- Summary: Create scheduling policy to auto-adjust queue elasticity based on cluster demand Key: YARN-9869 URL: https://issues.apache.org/jira/browse/YARN-9869 Project

[jira] [Created] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-26 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9858: --- Summary: Optimize RMContext getExclusiveEnforcedPartitions Key: YARN-9858 URL: https://issues.apache.org/jira/browse/YARN-9858 Project: Hadoop YARN Issue

Re: Incompatible changes between branch-2.8 and branch-2.9

2019-09-24 Thread Jonathan Hung
2.9+. But rolling upgrade would still work IIUC. Jonathan Hung On Tue, Sep 24, 2019 at 2:52 PM Eric Badger wrote: > * For YARN-6616, for branch-2.8 and below, it was only committed to > 2.7.8/2.8.6 which have not been released (as I understand). Perhaps we can > revert YARN-6616 fro

[jira] [Created] (YARN-9855) Fix ApplicationReportProto submitTime id in branch-2.8/branch-2.7

2019-09-24 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9855: --- Summary: Fix ApplicationReportProto submitTime id in branch-2.8/branch-2.7 Key: YARN-9855 URL: https://issues.apache.org/jira/browse/YARN-9855 Project: Hadoop YARN

Re: Incompatible changes between branch-2.8 and branch-2.9

2019-09-24 Thread Jonathan Hung
Sorry, let me edit my first point. We can just create addendums for YARN-6616 in branch-2.7 and branch-2.8 to edit the submitTime field to the correct id 28. We don’t need to revert YARN-6616 from these branches completely. Jonathan From: Jonathan Hung Sent

Re: Incompatible changes between branch-2.8 and branch-2.9

2019-09-24 Thread Jonathan Hung
be to move the intraQueuePreemptionDisabled field from id 12 to id 13 in branch-2.8, then users would upgrade from 2.8.4/2.8.5 to 2.8.6 (someone would have to release this), then upgrade from 2.8.6 to 2.9+. Jonathan Hung On Tue, Sep 24, 2019 at 9:23 AM Eric Badger wrote: > We (Verizon Media) are currently m

[jira] [Resolved] (YARN-6684) TestAMRMClient tests fail on branch-2.7

2019-09-19 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-6684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung resolved YARN-6684. - Resolution: Won't Fix branch-2.7 EOL, closing as won't fix > TestAMRMClient tests fail on bra

[jira] [Resolved] (YARN-8825) Print application tags in ApplicationSummary

2019-09-19 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-8825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung resolved YARN-8825. - Resolution: Duplicate > Print application tags in ApplicationSumm

[jira] [Resolved] (YARN-9844) TestCapacitySchedulerPerf test errors in branch-2

2019-09-19 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung resolved YARN-9844. - Resolution: Fixed > TestCapacitySchedulerPerf test errors in branc

[jira] [Created] (YARN-9825) Changes for initializing placement rules with ResourceScheduler in branch-2

2019-09-09 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9825: --- Summary: Changes for initializing placement rules with ResourceScheduler in branch-2 Key: YARN-9825 URL: https://issues.apache.org/jira/browse/YARN-9825 Project

[jira] [Created] (YARN-9824) Fall back to configured queue ordering policy class name

2019-09-09 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9824: --- Summary: Fall back to configured queue ordering policy class name Key: YARN-9824 URL: https://issues.apache.org/jira/browse/YARN-9824 Project: Hadoop YARN

[jira] [Created] (YARN-9810) Add queue capacity/maxcapacity percentage metrics

2019-09-03 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9810: --- Summary: Add queue capacity/maxcapacity percentage metrics Key: YARN-9810 URL: https://issues.apache.org/jira/browse/YARN-9810 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-9806) TestNMSimulator#testNMSimulator fails in branch-2

2019-08-30 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9806: --- Summary: TestNMSimulator#testNMSimulator fails in branch-2 Key: YARN-9806 URL: https://issues.apache.org/jira/browse/YARN-9806 Project: Hadoop YARN Issue Type

[jira] [Resolved] (YARN-7585) NodeManager should go unhealthy when state store throws DBException

2019-08-30 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung resolved YARN-7585. - Fix Version/s: 2.10.0 Resolution: Fixed Committed to branch-2. > NodeManager should

Re: [VOTE] Merge YARN-8200 to branch-2 and branch-3.0

2019-08-29 Thread Jonathan Hung
Thanks all, +1 from me too. There's three binding +1, two non-binding +1, and no -1 so I'll merge YARN-8200 to branch-2 shortly. I'll skip branch-3.0 since it's EOL as others have mentioned. Jonathan Hung On Tue, Aug 27, 2019 at 11:49 AM Konstantin Shvachko wrote: > +1 for the merge. >

[DISCUSS] Hadoop 2.10.0 release plan

2019-08-26 Thread Jonathan Hung
. We're targeting a release at end of September. Please share any thoughts you have about this. Thanks! [1] https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg29461.html [2] https://www.mail-archive.com/mapreduce-dev@hadoop.apache.org/msg21293.html [3] https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg33440.html Jonathan Hung

[VOTE] Merge YARN-8200 to branch-2 and branch-3.0

2019-08-22 Thread Jonathan Hung
=csrqgysvpdf7mh-iu17femgx89dhr+1...@mail.gmail.com%3e Jonathan Hung

[jira] [Created] (YARN-9770) Create a queue ordering policy which picks child queues with equal probability

2019-08-21 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9770: --- Summary: Create a queue ordering policy which picks child queues with equal probability Key: YARN-9770 URL: https://issues.apache.org/jira/browse/YARN-9770 Project

Re: [DISCUSS] Merging YARN-8200 to branch-3.0 and branch-2

2019-08-21 Thread Jonathan Hung
Reviving this thread: we tested YARN RU starting with a cluster running 2.7.4, to running branch-2 + YARN-8200. Ran some simple MR/Spark jobs concurrently with the RM/NM upgrades and did not see any issues. If no other concerns I'll continue with a vote. Jonathan Hung On Thu, Apr 18, 2019 at 5

Re: [VOTE] Mark 2.6, 2.7, 3.0 release lines EOL

2019-08-20 Thread Jonathan Hung
+1. Thanks! Jonathan Hung On Tue, Aug 20, 2019 at 8:03 PM Wangda Tan wrote: > Hi all, > > This is a vote thread to mark any versions smaller than 2.7 (inclusive), > and 3.0 EOL. This is based on discussions of [1] > > This discussion runs for 7 days and will conclude on Aug

[jira] [Created] (YARN-9764) Print application submission context label in application summary

2019-08-19 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9764: --- Summary: Print application submission context label in application summary Key: YARN-9764 URL: https://issues.apache.org/jira/browse/YARN-9764 Project: Hadoop YARN

[jira] [Created] (YARN-9763) Print application tags in application summary

2019-08-19 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9763: --- Summary: Print application tags in application summary Key: YARN-9763 URL: https://issues.apache.org/jira/browse/YARN-9763 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-9762) Add submission context label to audit logs

2019-08-19 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9762: --- Summary: Add submission context label to audit logs Key: YARN-9762 URL: https://issues.apache.org/jira/browse/YARN-9762 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-9761) Allow overriding application submissions based on server side configs

2019-08-19 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9761: --- Summary: Allow overriding application submissions based on server side configs Key: YARN-9761 URL: https://issues.apache.org/jira/browse/YARN-9761 Project: Hadoop YARN

[jira] [Created] (YARN-9760) Support configuring application priorities on a workflow level

2019-08-19 Thread Jonathan Hung (Jira)
Jonathan Hung created YARN-9760: --- Summary: Support configuring application priorities on a workflow level Key: YARN-9760 URL: https://issues.apache.org/jira/browse/YARN-9760 Project: Hadoop YARN

[jira] [Created] (YARN-9751) Separate queue and app ordering policy capacity scheduler configs

2019-08-15 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9751: --- Summary: Separate queue and app ordering policy capacity scheduler configs Key: YARN-9751 URL: https://issues.apache.org/jira/browse/YARN-9751 Project: Hadoop YARN

Re: [DISCUSS] Hadoop 2019 Release Planning

2019-08-12 Thread Jonathan Hung
Hi Wangda, Thanks for starting the discussion. We would also like to release 2.10.0 which was discussed previously <https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg29479.html> and at various contributor meetups. I'm interested in being release manager for that. Thanks, Jonatha

[jira] [Created] (YARN-9736) Recursively configure app ordering policies

2019-08-09 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9736: --- Summary: Recursively configure app ordering policies Key: YARN-9736 URL: https://issues.apache.org/jira/browse/YARN-9736 Project: Hadoop YARN Issue Type: Task

[jira] [Created] (YARN-9730) Support forcing configured partitions to be exclusive based on app node label

2019-08-08 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9730: --- Summary: Support forcing configured partitions to be exclusive based on app node label Key: YARN-9730 URL: https://issues.apache.org/jira/browse/YARN-9730 Project

[jira] [Created] (YARN-9668) UGI conf doesn't read user overridden configurations on RM and NM startup

2019-07-08 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9668: --- Summary: UGI conf doesn't read user overridden configurations on RM and NM startup Key: YARN-9668 URL: https://issues.apache.org/jira/browse/YARN-9668 Project: Hadoop

[jira] [Created] (YARN-9615) Add dispatcher metrics to RM

2019-06-10 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9615: --- Summary: Add dispatcher metrics to RM Key: YARN-9615 URL: https://issues.apache.org/jira/browse/YARN-9615 Project: Hadoop YARN Issue Type: Task

[jira] [Created] (YARN-9559) Create AbstractContainersLauncher for pluggable ContainersLauncher logic

2019-05-15 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9559: --- Summary: Create AbstractContainersLauncher for pluggable ContainersLauncher logic Key: YARN-9559 URL: https://issues.apache.org/jira/browse/YARN-9559 Project: Hadoop

[jira] [Created] (YARN-9529) Log correct cpu controller path on error

2019-05-03 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9529: --- Summary: Log correct cpu controller path on error Key: YARN-9529 URL: https://issues.apache.org/jira/browse/YARN-9529 Project: Hadoop YARN Issue Type

Re: [DISCUSS] Merging YARN-8200 to branch-3.0 and branch-2

2019-04-18 Thread Jonathan Hung
Sorry for the delay, had to deprioritize this. Hoping to get to this next week. Jonathan From: Jim Brennan Sent: Thursday, April 18, 2019 7:28 AM To: Jonathan Hung Cc: yarn-dev@hadoop.apache.org; mapreduce-...@hadoop.apache.org Subject: Re: [DISCUSS] Merging

[jira] [Created] (YARN-9438) launchTime not written to state store for running applications

2019-04-03 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9438: --- Summary: launchTime not written to state store for running applications Key: YARN-9438 URL: https://issues.apache.org/jira/browse/YARN-9438 Project: Hadoop YARN

Re: [DISCUSS] Merging YARN-8200 to branch-3.0 and branch-2

2019-04-03 Thread Jonathan Hung
Hi Jim, we have not tested rolling upgrade. I don’t foresee this being an issue, but we’ll try it out and report back. Jonathan From: Jim Brennan Sent: Tuesday, April 2, 2019 9:17 AM To: Jonathan Hung Cc: yarn-dev@hadoop.apache.org; mapreduce

[DISCUSS] Merging YARN-8200 to branch-3.0 and branch-2

2019-03-29 Thread Jonathan Hung
'll start a merge vote next week. Thanks. Jonathan Hung

[jira] [Resolved] (YARN-9412) Backport YARN-6909 to branch-2

2019-03-27 Thread Jonathan Hung (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung resolved YARN-9412. - Resolution: Fixed This ended up being a clean port. Closing. > Backport YARN-6909 to branc

[jira] [Created] (YARN-9412) Backport YARN-6909 to branch-2

2019-03-26 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9412: --- Summary: Backport YARN-6909 to branch-2 Key: YARN-9412 URL: https://issues.apache.org/jira/browse/YARN-9412 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Created] (YARN-9409) Port resource type changes from YARN-7237 to branch-3.0/branch-2

2019-03-25 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9409: --- Summary: Port resource type changes from YARN-7237 to branch-3.0/branch-2 Key: YARN-9409 URL: https://issues.apache.org/jira/browse/YARN-9409 Project: Hadoop YARN

[jira] [Created] (YARN-9397) Fix empty NMResourceInfo object test failures in branch-2

2019-03-18 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9397: --- Summary: Fix empty NMResourceInfo object test failures in branch-2 Key: YARN-9397 URL: https://issues.apache.org/jira/browse/YARN-9397 Project: Hadoop YARN

[jira] [Created] (YARN-9291) Backport YARN-7637 to branch-2

2019-02-08 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9291: --- Summary: Backport YARN-7637 to branch-2 Key: YARN-9291 URL: https://issues.apache.org/jira/browse/YARN-9291 Project: Hadoop YARN Issue Type: Task

Re: [VOTE] Moving branch-2 precommit/nightly test builds to java 8

2019-02-07 Thread Jonathan Hung
My non-binding +1 to finish. This vote passes with 6 binding +1, 3 non-binding +1, and no vetoes. We will make the changes as part of HADOOP-15711, please follow there. Thanks all! Jonathan Hung On Tue, Feb 5, 2019 at 11:38 PM Akira Ajisaka wrote: > +1 > > -Akira > > On Wed, F

[jira] [Created] (YARN-9289) Backport YARN-7330 for GPU in UI to branch-2

2019-02-07 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9289: --- Summary: Backport YARN-7330 for GPU in UI to branch-2 Key: YARN-9289 URL: https://issues.apache.org/jira/browse/YARN-9289 Project: Hadoop YARN Issue Type: Sub

[VOTE] Moving branch-2 precommit/nightly test builds to java 8

2019-02-04 Thread Jonathan Hung
-Build/ [3] https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/ Jonathan Hung

[jira] [Created] (YARN-9280) Backport YARN-6620 to YARN-8200/branch-2

2019-02-04 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9280: --- Summary: Backport YARN-6620 to YARN-8200/branch-2 Key: YARN-9280 URL: https://issues.apache.org/jira/browse/YARN-9280 Project: Hadoop YARN Issue Type: Sub

Re: [DISCUSS] Moving branch-2 to java 8

2019-02-04 Thread Jonathan Hung
. Jonathan Hung On Mon, Feb 4, 2019 at 10:45 AM Anu Engineer wrote: > Konstantin, > > Just a nitpicky thought, if we move this branch to Java-8 on Jenkins, but > still hope to release code that can run on Java 7, how will we detect > Java 8 only changes? I am asking because till

Re: [DISCUSS] Moving branch-2 to java 8

2019-02-04 Thread Jonathan Hung
Yeah, it's possible with yetus, there's one example here <https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86-jhung/60/console> which runs compilation on openjdk7 (and openjdk8), and runs tests on openjdk8 only. Jonathan Hung On Mon, Feb 4, 2019 at 10

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

2019-02-01 Thread Jonathan Hung
+1. Thanks Wangda. Jonathan Hung On Fri, Feb 1, 2019 at 2:25 PM Dinesh Chitlangia < dchitlan...@hortonworks.com> wrote: > +1 (non binding), thanks Wangda for organizing this. > > Regards, > Dinesh > > > > On 2/1/19, 5:24 PM, "Wangda Tan" wrote: >

[jira] [Created] (YARN-9272) Backport YARN-7738 for refreshing max allocation for multiple resource types

2019-02-01 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9272: --- Summary: Backport YARN-7738 for refreshing max allocation for multiple resource types Key: YARN-9272 URL: https://issues.apache.org/jira/browse/YARN-9272 Project

[jira] [Created] (YARN-9271) Backport YARN-6927 for resource type support in MapReduce

2019-02-01 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9271: --- Summary: Backport YARN-6927 for resource type support in MapReduce Key: YARN-9271 URL: https://issues.apache.org/jira/browse/YARN-9271 Project: Hadoop YARN

Re: [DISCUSS] Moving branch-2 to java 8

2019-02-01 Thread Jonathan Hung
compatibility we can do that at a later point. Also based on discussion with others in the community at the contributors meetup this past Wednesday, seems we are generally in favor of testing against java8. I'll start a vote soon. Jonathan Hung On Tue, Jan 29, 2019 at 4:11 AM Steve Loughran wrote

[jira] [Resolved] (YARN-9261) Backport YARN-7270 addendum to YARN-8200

2019-01-31 Thread Jonathan Hung (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung resolved YARN-9261. - Resolution: Fixed Clean backport. Pushed to YARN-8200 > Backport YARN-7270 addendum to YARN-8

[jira] [Created] (YARN-9261) Backport YARN-7270 addendum to YARN-8200

2019-01-31 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9261: --- Summary: Backport YARN-7270 addendum to YARN-8200 Key: YARN-9261 URL: https://issues.apache.org/jira/browse/YARN-9261 Project: Hadoop YARN Issue Type: Sub

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Jonathan Hung
+1. This is important for improving the deep learning on hadoop story. There's recently a lot of momentum for this, and decoupling submarine/hadoop will help it continue. Jonathan Hung On Thu, Jan 31, 2019 at 11:04 AM Wangda Tan wrote: > Hi devs, > > Since we started submarine-relat

[DISCUSS] Moving branch-2 to java 8

2019-01-28 Thread Jonathan Hung
source and binary compatibility. I don't currently have a great answer, but one initial thought is to build source/binary against java 7 to ensure compatibility and run the rest of the build as java 8. Thoughts? Jonathan Hung

[jira] [Resolved] (YARN-9222) Change startTime semantics for RMApp

2019-01-23 Thread Jonathan Hung (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung resolved YARN-9222. - Resolution: Fixed darn it, seems this is a dupe of YARN-7088 > Change startTime semant

[jira] [Created] (YARN-9222) Change startTime semantics for RMApp

2019-01-22 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9222: --- Summary: Change startTime semantics for RMApp Key: YARN-9222 URL: https://issues.apache.org/jira/browse/YARN-9222 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-9188) Port YARN-7136 to branch-2

2019-01-09 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9188: --- Summary: Port YARN-7136 to branch-2 Key: YARN-9188 URL: https://issues.apache.org/jira/browse/YARN-9188 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Created] (YARN-9187) Backport YARN-6852 for GPU-specific native changes to branch-2

2019-01-08 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9187: --- Summary: Backport YARN-6852 for GPU-specific native changes to branch-2 Key: YARN-9187 URL: https://issues.apache.org/jira/browse/YARN-9187 Project: Hadoop YARN

[jira] [Created] (YARN-9182) Backport YARN-6445 resource profile performance improvements to branch-2

2019-01-07 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9182: --- Summary: Backport YARN-6445 resource profile performance improvements to branch-2 Key: YARN-9182 URL: https://issues.apache.org/jira/browse/YARN-9182 Project: Hadoop

[jira] [Created] (YARN-9181) Backport YARN-6232 for generic resource type usage to branch-2

2019-01-07 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9181: --- Summary: Backport YARN-6232 for generic resource type usage to branch-2 Key: YARN-9181 URL: https://issues.apache.org/jira/browse/YARN-9181 Project: Hadoop YARN

[jira] [Created] (YARN-9180) Port YARN-7033 NM recovery of assigned resources to branch-3.0/branch-2

2019-01-07 Thread Jonathan Hung (JIRA)
Jonathan Hung created YARN-9180: --- Summary: Port YARN-7033 NM recovery of assigned resources to branch-3.0/branch-2 Key: YARN-9180 URL: https://issues.apache.org/jira/browse/YARN-9180 Project: Hadoop

  1   2   >