[jira] [Commented] (YARN-5767) Fix the order that resources are cleaned up from the local Public/Private caches

[ https://issues.apache.org/jira/browse/YARN-5767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15608897#comment-15608897 ] Jason Lowe commented on YARN-5767: -- Thanks for the patch, Chris! Looks good overall, just some minor nits

[jira] [Updated] (YARN-5765) LinuxContainerExecutor creates appcache and its subdirectories with wrong group owner.

[ https://issues.apache.org/jira/browse/YARN-5765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5765: - Affects Version/s: 2.8.0 Target Version/s: 2.8.0 (was: 3.0.0-alpha2) > LinuxContainerExecutor

[jira] [Commented] (YARN-5765) LinuxContainerExecutor creates appcache and its subdirectories with wrong group owner.

[ https://issues.apache.org/jira/browse/YARN-5765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15605347#comment-15605347 ] Jason Lowe commented on YARN-5765: -- Linking the two tickets. YARN-5287 went into 2.8, but this is marked

[jira] [Commented] (YARN-4126) RM should not issue delegation tokens in unsecure mode

[ https://issues.apache.org/jira/browse/YARN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595010#comment-15595010 ] Jason Lowe commented on YARN-4126: -- I'm just going off of [~rkanter]'s post in the yarn-dev mailing list

[jira] [Comment Edited] (YARN-4126) RM should not issue delegation tokens in unsecure mode

[ https://issues.apache.org/jira/browse/YARN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15591896#comment-15591896 ] Jason Lowe edited comment on YARN-4126 at 10/20/16 2:02 PM: Note that this

[jira] [Commented] (YARN-4126) RM should not issue delegation tokens in unsecure mode

[ https://issues.apache.org/jira/browse/YARN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15591896#comment-15591896 ] Jason Lowe commented on YARN-4126: -- Note that this change has broken Oozie, see YARN-5750. As such I

[jira] [Updated] (YARN-4763) RMApps Page crashes with NPE

[ https://issues.apache.org/jira/browse/YARN-4763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-4763: - Fix Version/s: (was: 2.9.0) 2.8.0 Thanks, [~bibinchundatt]! I committed this to

[jira] [Commented] (YARN-5732) Run auxiliary services in system containers

[ https://issues.apache.org/jira/browse/YARN-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15573374#comment-15573374 ] Jason Lowe commented on YARN-5732: -- Looks like a duplicate of YARN-1593. > Run auxiliary services in

[jira] [Commented] (YARN-5551) Ignore file backed pages from memory computation when smaps is enabled

[ https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15565698#comment-15565698 ] Jason Lowe commented on YARN-5551: -- +1 for the latest patch, committing this. > Ignore file backed pages

[jira] [Commented] (YARN-5641) Localizer leaves behind tarballs after container is complete

[ https://issues.apache.org/jira/browse/YARN-5641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15565643#comment-15565643 ] Jason Lowe commented on YARN-5641: -- Thanks for the patch, Eric! This has primarily become a change to the

[jira] [Commented] (YARN-5551) Ignore file backed pages from memory computation when smaps is enabled

[ https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562454#comment-15562454 ] Jason Lowe commented on YARN-5551: -- +1 lgtm. I'll commit this tomorrow if there are no objections. >

[jira] [Updated] (YARN-5491) Random Failure TestCapacityScheduler#testCSQueueBlocked

[ https://issues.apache.org/jira/browse/YARN-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5491: - Fix Version/s: (was: 2.9.0) 2.8.0 Thanks, [~bibinchundatt]! I committed this to

[jira] [Updated] (YARN-4543) TestNodeStatusUpdater.testStopReentrant fails + JUnit misusage

[ https://issues.apache.org/jira/browse/YARN-4543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-4543: - Fix Version/s: (was: 2.9.0) 2.8.0 Thanks [~suda]! I committed this to branch-2.8

[jira] [Updated] (YARN-5655) TestContainerManagerSecurity#testNMTokens is asserting

[ https://issues.apache.org/jira/browse/YARN-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5655: - Hadoop Flags: Reviewed Summary: TestContainerManagerSecurity#testNMTokens is asserting (was:

[jira] [Updated] (YARN-5540) scheduler spends too much time looking at empty priorities

[ https://issues.apache.org/jira/browse/YARN-5540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5540: - Attachment: YARN-5540-branch-2.8.004.patch Thanks for the review, Arun! Posting the branch-2.8 patch

[jira] [Updated] (YARN-5540) scheduler spends too much time looking at empty priorities

[ https://issues.apache.org/jira/browse/YARN-5540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5540: - Attachment: YARN-5540-branch-2.7.004.patch YARN-5540-branch-2.8.004.patch Thanks for the

[jira] [Commented] (YARN-5655) TestContainerManagerSecurity is failing

[ https://issues.apache.org/jira/browse/YARN-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15496277#comment-15496277 ] Jason Lowe commented on YARN-5655: -- The NPE looks like a case where either a container never started or

[jira] [Commented] (YARN-5655) TestContainerManagerSecurity is failing

[ https://issues.apache.org/jira/browse/YARN-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15494375#comment-15494375 ] Jason Lowe commented on YARN-5655: -- git bisect points to this commit when it started failing in

[jira] [Created] (YARN-5655) TestContainerManagerSecurity is failing

Jason Lowe created YARN-5655: Summary: TestContainerManagerSecurity is failing Key: YARN-5655 URL: https://issues.apache.org/jira/browse/YARN-5655 Project: Hadoop YARN Issue Type: Bug

[jira] [Commented] (YARN-5545) App submit failure on queue with label when default queue partition capacity is zero

[ https://issues.apache.org/jira/browse/YARN-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15494248#comment-15494248 ] Jason Lowe commented on YARN-5545: -- Yes, that's essentially the idea. Users can work around the issue

[jira] [Commented] (YARN-5545) App submit failure on queue with label when default queue partition capacity is zero

[ https://issues.apache.org/jira/browse/YARN-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493271#comment-15493271 ] Jason Lowe commented on YARN-5545: -- bq. This could be configured to set max-apps per queue level in

[jira] [Commented] (YARN-5545) App submit failure on queue with label when default queue partition capacity is zero

[ https://issues.apache.org/jira/browse/YARN-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15490671#comment-15490671 ] Jason Lowe commented on YARN-5545: -- The problem with changing queues to use the max apps conf directly is

[jira] [Commented] (YARN-5540) scheduler spends too much time looking at empty priorities

[ https://issues.apache.org/jira/browse/YARN-5540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15490557#comment-15490557 ] Jason Lowe commented on YARN-5540: -- The two test failures appear to be unrelated. Filed YARN-5652 for the

[jira] [Created] (YARN-5653) testNonLabeledResourceRequestGetPreferrenceToNonLabeledNode fails intermittently

Jason Lowe created YARN-5653: Summary: testNonLabeledResourceRequestGetPreferrenceToNonLabeledNode fails intermittently Key: YARN-5653 URL: https://issues.apache.org/jira/browse/YARN-5653 Project: Hadoop

[jira] [Commented] (YARN-5652) testRefreshNodesResourceWithResourceReturnInRegistration fails intermittently

[ https://issues.apache.org/jira/browse/YARN-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15490519#comment-15490519 ] Jason Lowe commented on YARN-5652: -- Looks closely related to YARN-4893 and YARN-5318. YARN-4893 added a

[jira] [Created] (YARN-5652) testRefreshNodesResourceWithResourceReturnInRegistration fails intermittently

Jason Lowe created YARN-5652: Summary: testRefreshNodesResourceWithResourceReturnInRegistration fails intermittently Key: YARN-5652 URL: https://issues.apache.org/jira/browse/YARN-5652 Project: Hadoop

[jira] [Commented] (YARN-5547) NMLeveldbStateStore should be more tolerant of unknown keys

[ https://issues.apache.org/jira/browse/YARN-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15490475#comment-15490475 ] Jason Lowe commented on YARN-5547: -- To be clear, the skipping containers during recovery is _never_ the

[jira] [Updated] (YARN-5009) NMLeveldbStateStoreService database can grow substantially leading to longer recovery times

[ https://issues.apache.org/jira/browse/YARN-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5009: - Fix Version/s: 2.6.5 I committed this to branch-2.6. > NMLeveldbStateStoreService database can grow

[jira] [Commented] (YARN-5547) NMLeveldbStateStore should be more tolerant of unknown keys

[ https://issues.apache.org/jira/browse/YARN-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15488333#comment-15488333 ] Jason Lowe commented on YARN-5547: -- Yes, having the ability to recover unknown keys that cannot be ignored

[jira] [Updated] (YARN-5009) NMLeveldbStateStoreService database can grow substantially leading to longer recovery times

[ https://issues.apache.org/jira/browse/YARN-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5009: - Attachment: YARN-5009-branch-2.6.002.patch The merge conflicts for branch-2.6 were trivial context

[jira] [Updated] (YARN-5540) scheduler spends too much time looking at empty priorities

[ https://issues.apache.org/jira/browse/YARN-5540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5540: - Attachment: YARN-5540.004.patch Oops, just realized patch 003 is missing the TODO comment removal. Fixed

[jira] [Updated] (YARN-5540) scheduler spends too much time looking at empty priorities

[ https://issues.apache.org/jira/browse/YARN-5540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5540: - Attachment: YARN-5540.003.patch Thanks for the review, Wangda! Updated the method names per the

[jira] [Commented] (YARN-5640) Issue while accessing resource manager webapp rest service

[ https://issues.apache.org/jira/browse/YARN-5640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15487316#comment-15487316 ] Jason Lowe commented on YARN-5640: -- JIRA is used for tracking bugs and features against the Hadoop code

[jira] [Commented] (YARN-5630) NM fails to start after downgrade from 2.8 to 2.7

[ https://issues.apache.org/jira/browse/YARN-5630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484465#comment-15484465 ] Jason Lowe commented on YARN-5630: -- Thanks for the +1, Arun! I'll commit this later today if there are no

[jira] [Commented] (YARN-5630) NM fails to start after downgrade from 2.8 to 2.7

[ https://issues.apache.org/jira/browse/YARN-5630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484295#comment-15484295 ] Jason Lowe commented on YARN-5630: -- I'm not a fan of the "prepare for rollback" approach if we can avoid

[jira] [Commented] (YARN-5547) NMLeveldbStateStore should be more tolerant of unknown keys

[ https://issues.apache.org/jira/browse/YARN-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15478519#comment-15478519 ] Jason Lowe commented on YARN-5547: -- Skipping the container entirely would be very bad. The NM would not

[jira] [Commented] (YARN-5547) NMLeveldbStateStore should be more tolerant of unknown keys

[ https://issues.apache.org/jira/browse/YARN-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15478509#comment-15478509 ] Jason Lowe commented on YARN-5547: -- That sounds like an excellent idea. If the old software could consult

[jira] [Commented] (YARN-5632) UPDATE_EXECUTION_TYPE causes UpdateContainerRequestPBImpl to throw

[ https://issues.apache.org/jira/browse/YARN-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15478487#comment-15478487 ] Jason Lowe commented on YARN-5632: -- Thanks for the review, Arun! Committing this. >

[jira] [Commented] (YARN-5630) NM fails to start after downgrade from 2.8 to 2.7

[ https://issues.apache.org/jira/browse/YARN-5630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477825#comment-15477825 ] Jason Lowe commented on YARN-5630: -- We could do the refcounting thing, but what I'm thinking is that we

[jira] [Commented] (YARN-5630) NM fails to start after downgrade from 2.8 to 2.7

[ https://issues.apache.org/jira/browse/YARN-5630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477609#comment-15477609 ] Jason Lowe commented on YARN-5630: -- Yeah I'm not sure what to do about the schema version. Ideally I'd

[jira] [Commented] (YARN-5633) Update Container Version in NMStateStore only if Resources have changed

[ https://issues.apache.org/jira/browse/YARN-5633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477501#comment-15477501 ] Jason Lowe commented on YARN-5633: -- Took a look at the patch. It's removing any support for storing a

[jira] [Commented] (YARN-5633) Update Container Version in NMStateStore only if Resources have changed

[ https://issues.apache.org/jira/browse/YARN-5633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477490#comment-15477490 ] Jason Lowe commented on YARN-5633: -- This looks closely related to YARN-5630, although the patch in

[jira] [Updated] (YARN-5632) UPDATE_EXECUTION_TYPE causes UpdateContainerRequestPBImpl to throw

[ https://issues.apache.org/jira/browse/YARN-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5632: - Attachment: YARN-5632-branch-2.8.001.patch Looks like the most straightforward fix is to remove the extra

[jira] [Updated] (YARN-5632) UPDATE_EXECUTION_TYPE causes UpdateContainerRequestPBImpl to throw

[ https://issues.apache.org/jira/browse/YARN-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5632: - Priority: Major (was: Critical) > UPDATE_EXECUTION_TYPE causes UpdateContainerRequestPBImpl to throw >

[jira] [Commented] (YARN-5221) Expose UpdateResourceRequest API to allow AM to request for change in container properties

[ https://issues.apache.org/jira/browse/YARN-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477459#comment-15477459 ] Jason Lowe commented on YARN-5221: -- The branch-2.8 patch also has an issue with UPDATE_EXECUTION_TYPE

[jira] [Commented] (YARN-5632) UPDATE_EXECUTION_TYPE causes UpdateContainerRequestPBImpl to throw

[ https://issues.apache.org/jira/browse/YARN-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477437#comment-15477437 ] Jason Lowe commented on YARN-5632: -- Looks like this was missed in the branch-2.8 patch for YARN-5221.

[jira] [Created] (YARN-5632) UPDATE_EXECUTION_TYPE causes UpdateContainerRequestPBImpl to throw

Jason Lowe created YARN-5632: Summary: UPDATE_EXECUTION_TYPE causes UpdateContainerRequestPBImpl to throw Key: YARN-5632 URL: https://issues.apache.org/jira/browse/YARN-5632 Project: Hadoop YARN

[jira] [Updated] (YARN-5630) NM fails to start after downgrade from 2.8 to 2.7

[ https://issues.apache.org/jira/browse/YARN-5630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5630: - Attachment: YARN-5630.002.patch The TestQueuingContainerManager failure is not related and being tracked

[jira] [Commented] (YARN-5221) Expose UpdateResourceRequest API to allow AM to request for change in container properties

[ https://issues.apache.org/jira/browse/YARN-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477048#comment-15477048 ] Jason Lowe commented on YARN-5221: -- This broke rolling downgrades from 2.8 to 2.7. See YARN-5630 for

[jira] [Updated] (YARN-5630) NM fails to start after downgrade from 2.8 to 2.7

[ https://issues.apache.org/jira/browse/YARN-5630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5630: - Attachment: YARN-5630.001.patch Patch that only stores the container version key if the version is

[jira] [Commented] (YARN-5630) NM fails to start after downgrade from 2.8 to 2.7

[ https://issues.apache.org/jira/browse/YARN-5630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476980#comment-15476980 ] Jason Lowe commented on YARN-5630: -- This was introduced by YARN-5221. Sample stacktrace: {noformat}

[jira] [Created] (YARN-5630) NM fails to start after downgrade from 2.8 to 2.7

Jason Lowe created YARN-5630: Summary: NM fails to start after downgrade from 2.8 to 2.7 Key: YARN-5630 URL: https://issues.apache.org/jira/browse/YARN-5630 Project: Hadoop YARN Issue Type: Bug

[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory

[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470986#comment-15470986 ] Jason Lowe commented on YARN-4954: -- We're seeing this unit test still failing sporadically in our builds,

[jira] [Commented] (YARN-5617) AMs only intended to run one attempt can be run more than once

[ https://issues.apache.org/jira/browse/YARN-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459242#comment-15459242 ] Jason Lowe commented on YARN-5617: -- See TEZ-3426 for an example of an app framework that was depending

[jira] [Created] (YARN-5617) AMs only intended to run one attempt can be run more than once

Jason Lowe created YARN-5617: Summary: AMs only intended to run one attempt can be run more than once Key: YARN-5617 URL: https://issues.apache.org/jira/browse/YARN-5617 Project: Hadoop YARN

[jira] [Commented] (YARN-5549) AMLauncher.createAMContainerLaunchContext() should not log the command to be launched indiscriminately

[ https://issues.apache.org/jira/browse/YARN-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456586#comment-15456586 ] Jason Lowe commented on YARN-5549: -- +1 lgtm. > AMLauncher.createAMContainerLaunchContext() should not log

[jira] [Commented] (YARN-5549) AMLauncher.createAMContainerLaunchContext() should not log the command to be launched indiscriminately

[ https://issues.apache.org/jira/browse/YARN-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449059#comment-15449059 ] Jason Lowe commented on YARN-5549: -- Storing launch info in ATSv2 is fine with me and sounds preferable as

[jira] [Commented] (YARN-5549) AMLauncher.createAMContainerLaunchContext() should not log the command to be launched indiscriminately

[ https://issues.apache.org/jira/browse/YARN-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447251#comment-15447251 ] Jason Lowe commented on YARN-5549: -- Totally agree with Daniel that this is only useful for debugging a

[jira] [Commented] (YARN-5560) Clean up bad exception catching practices in TestYarnClient

[ https://issues.apache.org/jira/browse/YARN-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446310#comment-15446310 ] Jason Lowe commented on YARN-5560: -- +1, committing this. > Clean up bad exception catching practices in

[jira] [Commented] (YARN-5560) Clean up bad exception catching practices in TestYarnClient

[ https://issues.apache.org/jira/browse/YARN-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439630#comment-15439630 ] Jason Lowe commented on YARN-5560: -- Thanks, Sean! Patch looks good overall with just one minor nit.

[jira] [Updated] (YARN-5540) scheduler spends too much time looking at empty priorities

[ https://issues.apache.org/jira/browse/YARN-5540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5540: - Attachment: YARN-5540.002.patch Updating the patch to use a ConcurrentSkipListMap instead of a TreeMap.

[jira] [Commented] (YARN-5551) Ignore deleted file mapping from memory computation when smaps is enabled

[ https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439274#comment-15439274 ] Jason Lowe commented on YARN-5551: -- bq. Memory pressure could cause either of them to spill to disk, so it

[jira] [Commented] (YARN-5551) Ignore deleted file mapping from memory computation when smaps is enabled

[ https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439246#comment-15439246 ] Jason Lowe commented on YARN-5551: -- bq. Yes, deleted files is a red-herring OK, cool. That's been my

[jira] [Commented] (YARN-5547) NMLeveldbStateStore should be more tolerant of unknown keys

[ https://issues.apache.org/jira/browse/YARN-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15438921#comment-15438921 ] Jason Lowe commented on YARN-5547: -- Thanks for the patch! What I meant about the leak is a scenario like

[jira] [Commented] (YARN-5551) Ignore deleted file mapping from memory computation when smaps is enabled

[ https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437655#comment-15437655 ] Jason Lowe commented on YARN-5551: -- The more I think about this, the more I feel ignoring deleted files is

[jira] [Commented] (YARN-5551) Ignore deleted file mapping from memory computation when smaps is enabled

[ https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437602#comment-15437602 ] Jason Lowe commented on YARN-5551: -- Sorry I'm confused, so apologies if this is obvious to everyone else.

[jira] [Commented] (YARN-5551) Ignore deleted file mapping from memory computation when smaps is enabled

[ https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437397#comment-15437397 ] Jason Lowe commented on YARN-5551: -- bq. Actually, that's just a safety rail to cut down IO here - when the

[jira] [Commented] (YARN-5551) Ignore deleted file mapping from memory computation when smaps is enabled

[ https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437316#comment-15437316 ] Jason Lowe commented on YARN-5551: -- Special casing buffer cache pages is one thing, but I guess where I'm

[jira] [Commented] (YARN-5551) Ignore deleted file mapping from memory computation when smaps is enabled

[ https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437067#comment-15437067 ] Jason Lowe commented on YARN-5551: -- The "deleted" here refers to the fact that the file path no longer

[jira] [Updated] (YARN-5389) TestYarnClient#testReservationDelete fails

[ https://issues.apache.org/jira/browse/YARN-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5389: - Summary: TestYarnClient#testReservationDelete fails (was: TestYarnClient#testReservationDelete fails in

[jira] [Commented] (YARN-5389) TestYarnClient#testReservationDelete fails in trunk

[ https://issues.apache.org/jira/browse/YARN-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437008#comment-15437008 ] Jason Lowe commented on YARN-5389: -- +1 for the latest patch. Committing this. >

[jira] [Commented] (YARN-5389) TestYarnClient#testReservationDelete fails in trunk

[ https://issues.apache.org/jira/browse/YARN-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435845#comment-15435845 ] Jason Lowe commented on YARN-5389: -- Thanks for updating the patch! Nit: The getNewReservation method is

[jira] [Commented] (YARN-5540) scheduler spends too much time looking at empty priorities

[ https://issues.apache.org/jira/browse/YARN-5540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435138#comment-15435138 ] Jason Lowe commented on YARN-5540: -- Thanks for the review! bq. you can remove the TODO: Shouldn't we

[jira] [Commented] (YARN-5389) TestYarnClient#testReservationDelete fails in trunk

[ https://issues.apache.org/jira/browse/YARN-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435087#comment-15435087 ] Jason Lowe commented on YARN-5389: -- Thanks for updating the patch! I'm curious why we don't just let

[jira] [Updated] (YARN-5540) scheduler spends too much time looking at empty priorities

[ https://issues.apache.org/jira/browse/YARN-5540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5540: - Attachment: YARN-5540.001.patch The main problem is that a scheduler key is never being removed from the

[jira] [Commented] (YARN-1503) Continuous resource-localization for YARN containers

[ https://issues.apache.org/jira/browse/YARN-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432920#comment-15432920 ] Jason Lowe commented on YARN-1503: -- Thanks for driving this, Jian! Seems reasonable overall with just one

[jira] [Updated] (YARN-5540) scheduler spends too much time looking at empty priorities

[ https://issues.apache.org/jira/browse/YARN-5540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5540: - Summary: scheduler spends too much time looking at empty priorities (was: Capacity Scheduler spends too

[jira] [Commented] (YARN-5547) NMLeveldbStateStore should be more tolerant of unknown keys

[ https://issues.apache.org/jira/browse/YARN-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430982#comment-15430982 ] Jason Lowe commented on YARN-5547: -- Also see the backwards-compatibility discussions in YARN-3998 and

[jira] [Created] (YARN-5547) NMLeveldbStateStore should be more tolerant of unknown keys

Jason Lowe created YARN-5547: Summary: NMLeveldbStateStore should be more tolerant of unknown keys Key: YARN-5547 URL: https://issues.apache.org/jira/browse/YARN-5547 Project: Hadoop YARN Issue

[jira] [Commented] (YARN-3998) Add support in the NodeManager to re-launch containers

[ https://issues.apache.org/jira/browse/YARN-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430973#comment-15430973 ] Jason Lowe commented on YARN-3998: -- bq. it looks like we throw an exception when we encounter a field we

[jira] [Commented] (YARN-5389) TestYarnClient#testReservationDelete fails in trunk

[ https://issues.apache.org/jira/browse/YARN-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430955#comment-15430955 ] Jason Lowe commented on YARN-5389: -- Thanks for the patch! We should not add longer sleeps in tests. It

[jira] [Commented] (YARN-3998) Add support in the NodeManager to re-launch containers

[ https://issues.apache.org/jira/browse/YARN-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430822#comment-15430822 ] Jason Lowe commented on YARN-3998: -- I believe the old software will ignore unrecognized keys in the state

[jira] [Commented] (YARN-5049) Extend NMStateStore to save queued container information

[ https://issues.apache.org/jira/browse/YARN-5049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430799#comment-15430799 ] Jason Lowe commented on YARN-5049: -- The major version should change when an older version of the software

[jira] [Commented] (YARN-1529) Add Localization overhead metrics to NM

[ https://issues.apache.org/jira/browse/YARN-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427301#comment-15427301 ] Jason Lowe commented on YARN-1529: -- bq. One comment that I have is we are adding a new API, albeit a small

[jira] [Updated] (YARN-5049) Extend NMStateStore to save queued container information

[ https://issues.apache.org/jira/browse/YARN-5049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5049: - Hadoop Flags: Incompatible change Sorry for getting here late, as I just ran into this by accident. This

[jira] [Updated] (YARN-2695) Support continuously looking reserved container with node labels

[ https://issues.apache.org/jira/browse/YARN-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-2695: - Attachment: YARN-2695.001.patch We've run into a number of situations where the lack of

[jira] [Commented] (YARN-5532) Continuous resource-localization for YARN containers

[ https://issues.apache.org/jira/browse/YARN-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424592#comment-15424592 ] Jason Lowe commented on YARN-5532: -- This looks like a dup of YARN-1503 or vice-versa. > Continuous

[jira] [Commented] (YARN-3388) Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit

[ https://issues.apache.org/jira/browse/YARN-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421800#comment-15421800 ] Jason Lowe commented on YARN-3388: -- [~nroberts] the patch no longer applies to trunk. Could you please

[jira] [Updated] (YARN-4566) TestMiniYarnClusterNodeUtilization sometimes fails on trunk

[ https://issues.apache.org/jira/browse/YARN-4566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-4566: - Fix Version/s: (was: 2.9.0) 2.8.0 Thanks, [~bwtakacy]! I committed this to

[jira] [Updated] (YARN-4916) TestNMProxy.tesNMProxyRPCRetry fails.

[ https://issues.apache.org/jira/browse/YARN-4916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-4916: - Fix Version/s: (was: 2.9.0) 2.8.0 Thanks, [~tibor.k...@gmail.com]! I committed

[jira] [Commented] (YARN-5393) [Umbrella] Optimize YARN tests runtime

[ https://issues.apache.org/jira/browse/YARN-5393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419470#comment-15419470 ] Jason Lowe commented on YARN-5393: -- A lot of this bloat is from over-sleeping. It's sad to watch the CPU

[jira] [Updated] (YARN-1529) Add Localization overhead metrics to NM

[ https://issues.apache.org/jira/browse/YARN-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-1529: - Attachment: YARN-1529.v04.patch I've attached a version 4 of the patch upmerged to trunk which is what

[jira] [Commented] (YARN-4953) Delete completed container log folder when rolling log aggregation is enabled

[ https://issues.apache.org/jira/browse/YARN-4953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417388#comment-15417388 ] Jason Lowe commented on YARN-4953: -- Sorry for the delay in responding. I think we can make deletion of

[jira] [Commented] (YARN-5483) Optimize RMAppAttempt#pullJustFinishedContainers

[ https://issues.apache.org/jira/browse/YARN-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415656#comment-15415656 ] Jason Lowe commented on YARN-5483: -- +1 for the 2.7 and 2.6 patch as well. Committing this. > Optimize

[jira] [Commented] (YARN-5483) Optimize RMAppAttempt#pullJustFinishedContainers

[ https://issues.apache.org/jira/browse/YARN-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414349#comment-15414349 ] Jason Lowe commented on YARN-5483: -- bq. seems we should merge YARN-5262 to 2.6/2.7 too. Good catch! I

[jira] [Updated] (YARN-5262) Optimize sending RMNodeFinishedContainersPulledByAMEvent for every AM heartbeat

[ https://issues.apache.org/jira/browse/YARN-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5262: - Fix Version/s: (was: 2.9.0) 2.7.4 2.6.5 2.8.0

[jira] [Commented] (YARN-5382) RM does not audit log kill request for active applications

[ https://issues.apache.org/jira/browse/YARN-5382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414251#comment-15414251 ] Jason Lowe commented on YARN-5382: -- +1 for the latest trunk and branch-2.7 patches. I'll commit this

[jira] [Commented] (YARN-5492) TestSubmitApplicationWithRMHA is failing sporadically during precommit builds

[ https://issues.apache.org/jira/browse/YARN-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414225#comment-15414225 ] Jason Lowe commented on YARN-5492: -- {noformat} Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time

[jira] [Created] (YARN-5492) TestSubmitApplicationWithRMHA is failing sporadically during precommit builds

Jason Lowe created YARN-5492: Summary: TestSubmitApplicationWithRMHA is failing sporadically during precommit builds Key: YARN-5492 URL: https://issues.apache.org/jira/browse/YARN-5492 Project: Hadoop

[jira] [Commented] (YARN-5382) RM does not audit log kill request for active applications

[ https://issues.apache.org/jira/browse/YARN-5382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413740#comment-15413740 ] Jason Lowe commented on YARN-5382: -- Scratch that commit, the javadoc error flagged above is related to the

<    7   8   9   10   11   12   13   14   15   16   >