[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881097#comment-15881097 ] Wangda Tan commented on YARN-4954: -- [~ebadger], I believe YARN-5389 is the issue. Since we're using MiniYarnCluster which by default inits host's memory to 4G, I don't think MiniYarnCluster tests related to host's actual physical memory. Let's wait for [~GergelyNovak]'s comment before closing this ticket. > TestYarnClient.testReservationAPIs fails on machines with less than 4 GB > available memory > - > > Key: YARN-4954 > URL: https://issues.apache.org/jira/browse/YARN-4954 > Project: Hadoop YARN > Issue Type: Test > Components: test >Affects Versions: 3.0.0-alpha1 >Reporter: Gergely Novák >Assignee: Gergely Novák >Priority: Critical > Attachments: YARN-4954.001.patch, YARN-4954.002.patch > > > TestYarnClient.testReservationAPIs sometimes fails with this error: > {noformat} > java.lang.AssertionError: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416) > Caused by: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237) > ... 10 more > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227) > {noformat} > This is caused by really not having enough available memory to complete the > reservation (4 * 1024 MB). In my opinion lowering the required memory (either > by lowering the number of containers to 2, or the memory to 512 MB) would > make the test more stable. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881074#comment-15881074 ] Eric Badger commented on YARN-4954: --- [~leftnoteasy], YARN-5389 fixed the timeout failures related to the NMs not heartbeating quickly enough to update the RM's capacity. But this JIRA is more about the requests being too high in general. I can't get this test to fail anymore locally and I have forgotten how exactly it was failing for me in the past. I might be getting Jiras mixed up in my head, but I remember something about capacities being updated in different places and not propagating quickly enough once we got past the test check. However, that doesn't seem to be mentioned here, so it is quite possible that I'm thinking of a different Jira. [~GergelyNovak] addressed the issue of machines/VMs with less than 4GB of memory and confirmed that it still worked, so I'll take his word for it. I'm ok closing this and reopening if the issue shows again. > TestYarnClient.testReservationAPIs fails on machines with less than 4 GB > available memory > - > > Key: YARN-4954 > URL: https://issues.apache.org/jira/browse/YARN-4954 > Project: Hadoop YARN > Issue Type: Test > Components: test >Affects Versions: 3.0.0-alpha1 >Reporter: Gergely Novák >Assignee: Gergely Novák >Priority: Critical > Attachments: YARN-4954.001.patch, YARN-4954.002.patch > > > TestYarnClient.testReservationAPIs sometimes fails with this error: > {noformat} > java.lang.AssertionError: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416) > Caused by: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237) > ... 10 more > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227) > {noformat} > This is caused by really not having enough available memory to complete the > reservation (4 * 1024 MB). In my opinion lowering the required memory (either > by lowering the number of containers to 2, or the memory to 512 MB) would > make the test more stable. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879485#comment-15879485 ] Wangda Tan commented on YARN-4954: -- I believe YARN-5389 have already fixed this issue, [~jlowe], [~ebadger], [~GergelyNovak], correct? > TestYarnClient.testReservationAPIs fails on machines with less than 4 GB > available memory > - > > Key: YARN-4954 > URL: https://issues.apache.org/jira/browse/YARN-4954 > Project: Hadoop YARN > Issue Type: Test > Components: test >Affects Versions: 3.0.0-alpha1 >Reporter: Gergely Novák >Assignee: Gergely Novák >Priority: Critical > Attachments: YARN-4954.001.patch, YARN-4954.002.patch > > > TestYarnClient.testReservationAPIs sometimes fails with this error: > {noformat} > java.lang.AssertionError: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416) > Caused by: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237) > ... 10 more > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227) > {noformat} > This is caused by really not having enough available memory to complete the > reservation (4 * 1024 MB). In my opinion lowering the required memory (either > by lowering the number of containers to 2, or the memory to 512 MB) would > make the test more stable. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470986#comment-15470986 ] Jason Lowe commented on YARN-4954: -- We're seeing this unit test still failing sporadically in our builds, so it'd be good to get some more traction on this. It looks like [~GergelyNovak]'s original patch fixes the test. If it's too painful or people are too busy to address all the comments I propose we proceed with a not-perfect-but-it-fixes-the-test-for-now patch and address the remaining issues in a followup JIRA. I agree that MiniYARNCluster is not going to be dynamically sized based on the real hardware size, since hardware detection is disabled by default. It will always come up with the same sized nodes (unless overridden in a particular test), so I question how dynamic this test code really needs to be in practice. If we defer the dynamic stuff to another JIRA (or decide it's not necessary at all) then that just leaves Vinod's other comment about the memory check against 6000 possibly still being too high. > TestYarnClient.testReservationAPIs fails on machines with less than 4 GB > available memory > - > > Key: YARN-4954 > URL: https://issues.apache.org/jira/browse/YARN-4954 > Project: Hadoop YARN > Issue Type: Test > Components: test >Affects Versions: 3.0.0-alpha1 >Reporter: Gergely Novák >Assignee: Gergely Novák >Priority: Critical > Attachments: YARN-4954.001.patch > > > TestYarnClient.testReservationAPIs sometimes fails with this error: > {noformat} > java.lang.AssertionError: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416) > Caused by: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237) > ... 10 more > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227) > {noformat} > This is caused by really not having enough available memory to complete the > reservation (4 * 1024 MB). In my opinion lowering the required memory (either > by lowering the number of containers to 2, or the memory to 512 MB) would > make the test more stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385594#comment-15385594 ] Gergely Novák commented on YARN-4954: - This JIRA seems outdated to me: unfortunately I'm not familiar with the latest developments affecting the YARN reservation system, but it looks like the MiniYARNCluster is now using hard coded config values for 'detecting' container memory (NodeManagerHardwareUtils.getContainerMemoryMB()): {code} memoryMb = conf.getInt(YarnConfiguration.NM_PMEM_MB, YarnConfiguration.DEFAULT_NM_PMEM_MB); {code} This means that it is irrelevant how much memory is actually available in the machine, I tested this with VMs containing 6, 4, 3 and 2 GBs of RAM (with swapping disabled) and on my local machine with 16 GBs: the container memory is *always* 4096 MBs, and the plan capacity is 4096 * 0.8 * 2 \[total * planAbsCap * number of node managers\] = 6553 MBs, so the block you quoted works (at least in my test environments). Also, with fixed container memory the fixed reservation request memory sizes seem reasonable to me. [~vinodkv] Can you give me a link to the environment (Jenkins?) where the tests are consistently failing so that I can have a further analysis? > TestYarnClient.testReservationAPIs fails on machines with less than 4 GB > available memory > - > > Key: YARN-4954 > URL: https://issues.apache.org/jira/browse/YARN-4954 > Project: Hadoop YARN > Issue Type: Test > Components: test >Affects Versions: 3.0.0-alpha1 >Reporter: Gergely Novák >Assignee: Gergely Novák >Priority: Critical > Attachments: YARN-4954.001.patch > > > TestYarnClient.testReservationAPIs sometimes fails with this error: > {noformat} > java.lang.AssertionError: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416) > Caused by: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237) > ... 10 more > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227) > {noformat} > This is caused by really not having enough available memory to complete the > reservation (4 * 1024 MB). In my opinion lowering the required memory (either > by lowering the number of containers to 2, or the memory to 512 MB) would > make the test more stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail:
[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353000#comment-15353000 ] Hadoop QA commented on YARN-4954: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s {color} | {color:red} YARN-4954 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12798472/YARN-4954.001.patch | | JIRA Issue | YARN-4954 | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/12156/console | | Powered by | Apache Yetus 0.3.0 http://yetus.apache.org | This message was automatically generated. > TestYarnClient.testReservationAPIs fails on machines with less than 4 GB > available memory > - > > Key: YARN-4954 > URL: https://issues.apache.org/jira/browse/YARN-4954 > Project: Hadoop YARN > Issue Type: Test > Components: test >Affects Versions: 3.0.0-alpha1 >Reporter: Gergely Novák >Assignee: Gergely Novák >Priority: Minor > Attachments: YARN-4954.001.patch > > > TestYarnClient.testReservationAPIs sometimes fails with this error: > {noformat} > java.lang.AssertionError: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416) > Caused by: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237) > ... 10 more > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227) > {noformat} > This is caused by really not having enough available memory to complete the > reservation (4 * 1024 MB). In my opinion lowering the required memory (either > by lowering the number of containers to 2, or the memory to 512 MB) would > make the test more stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352998#comment-15352998 ] Hadoop QA commented on YARN-4954: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color} | {color:red} YARN-4954 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12798472/YARN-4954.001.patch | | JIRA Issue | YARN-4954 | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/12155/console | | Powered by | Apache Yetus 0.3.0 http://yetus.apache.org | This message was automatically generated. > TestYarnClient.testReservationAPIs fails on machines with less than 4 GB > available memory > - > > Key: YARN-4954 > URL: https://issues.apache.org/jira/browse/YARN-4954 > Project: Hadoop YARN > Issue Type: Test > Components: test >Affects Versions: 3.0.0-alpha1 >Reporter: Gergely Novák >Assignee: Gergely Novák >Priority: Minor > Attachments: YARN-4954.001.patch > > > TestYarnClient.testReservationAPIs sometimes fails with this error: > {noformat} > java.lang.AssertionError: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416) > Caused by: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237) > ... 10 more > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227) > {noformat} > This is caused by really not having enough available memory to complete the > reservation (4 * 1024 MB). In my opinion lowering the required memory (either > by lowering the number of containers to 2, or the memory to 512 MB) would > make the test more stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15252523#comment-15252523 ] Junping Du commented on YARN-4954: -- Hi [~vinodkv], I saw you move this one into subtask of YARN-4478. Here we noticed timeout issue for 4 test classes: {noformat} org.apache.hadoop.yarn.client.cli.TestYarnCLI org.apache.hadoop.yarn.client.api.impl.TestYarnClient org.apache.hadoop.yarn.client.api.impl.TestAMRMClient org.apache.hadoop.yarn.client.api.impl.TestNMClien {noformat} Shall we file JIRA under YARN-4478 to track them? If so, as a whole or filed separately? > TestYarnClient.testReservationAPIs fails on machines with less than 4 GB > available memory > - > > Key: YARN-4954 > URL: https://issues.apache.org/jira/browse/YARN-4954 > Project: Hadoop YARN > Issue Type: Sub-task > Components: test >Affects Versions: 3.0.0 >Reporter: Gergely Novák >Assignee: Gergely Novák >Priority: Minor > Attachments: YARN-4954.001.patch > > > TestYarnClient.testReservationAPIs sometimes fails with this error: > {noformat} > java.lang.AssertionError: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416) > Caused by: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237) > ... 10 more > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227) > {noformat} > This is caused by really not having enough available memory to complete the > reservation (4 * 1024 MB). In my opinion lowering the required memory (either > by lowering the number of containers to 2, or the memory to 512 MB) would > make the test more stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15252115#comment-15252115 ] Hadoop QA commented on YARN-4954: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 25s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s {color} | {color:green} trunk passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 14s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 24s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 12s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 35s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s {color} | {color:green} trunk passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 19s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 14s {color} | {color:green} the patch passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 14s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 11s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 22s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 11s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 45s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s {color} | {color:green} the patch passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 11s {color} | {color:red} hadoop-yarn-client in the patch failed with JDK v1.8.0_77. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 26s {color} | {color:red} hadoop-yarn-client in the patch failed with JDK v1.7.0_95. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 18s {color} | {color:green} Patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 147m 7s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.8.0_77 Failed junit tests | hadoop.yarn.client.api.impl.TestAMRMProxy | | | hadoop.yarn.client.TestGetGroups | | JDK v1.8.0_77 Timed out junit tests | org.apache.hadoop.yarn.client.cli.TestYarnCLI | | | org.apache.hadoop.yarn.client.api.impl.TestYarnClient | | | org.apache.hadoop.yarn.client.api.impl.TestAMRMClient | | | org.apache.hadoop.yarn.client.api.impl.TestNMClient | | JDK v1.7.0_95 Failed junit tests | hadoop.yarn.client.api.impl.TestAMRMProxy | | | hadoop.yarn.client.TestGetGroups | | JDK v1.7.0_95 Timed out junit tests | org.apache.hadoop.yarn.client.cli.TestYarnCLI | | |
[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251884#comment-15251884 ] Junping Du commented on YARN-4954: -- Agree. I just kick off test again to see if timeout issue still exists for TestYarnClient. If yes, we should file a separated JIRA to track it if we don't have yet. > TestYarnClient.testReservationAPIs fails on machines with less than 4 GB > available memory > - > > Key: YARN-4954 > URL: https://issues.apache.org/jira/browse/YARN-4954 > Project: Hadoop YARN > Issue Type: Test > Components: test >Affects Versions: 3.0.0 >Reporter: Gergely Novák >Assignee: Gergely Novák >Priority: Minor > Attachments: YARN-4954.001.patch > > > TestYarnClient.testReservationAPIs sometimes fails with this error: > {noformat} > java.lang.AssertionError: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416) > Caused by: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237) > ... 10 more > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227) > {noformat} > This is caused by really not having enough available memory to complete the > reservation (4 * 1024 MB). In my opinion lowering the required memory (either > by lowering the number of containers to 2, or the memory to 512 MB) would > make the test more stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247565#comment-15247565 ] Gergely Novák commented on YARN-4954: - Tried to reproduce the timeout issues but couldn't. However, it is certain that they have nothing to do with my patch and in the subsequent builds they are not present. > TestYarnClient.testReservationAPIs fails on machines with less than 4 GB > available memory > - > > Key: YARN-4954 > URL: https://issues.apache.org/jira/browse/YARN-4954 > Project: Hadoop YARN > Issue Type: Test > Components: test >Affects Versions: 3.0.0 >Reporter: Gergely Novák >Assignee: Gergely Novák >Priority: Minor > Attachments: YARN-4954.001.patch > > > TestYarnClient.testReservationAPIs sometimes fails with this error: > {noformat} > java.lang.AssertionError: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416) > Caused by: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237) > ... 10 more > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227) > {noformat} > This is caused by really not having enough available memory to complete the > reservation (4 * 1024 MB). In my opinion lowering the required memory (either > by lowering the number of containers to 2, or the memory to 512 MB) would > make the test more stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245658#comment-15245658 ] Junping Du commented on YARN-4954: -- Thanks [~GergelyNovak] for reporting this issue and deliver the patch. The fix looks OK to me. However, the timeout issue still get report in TestYarnClient. Can you investigate and fix it in the same JIRA? > TestYarnClient.testReservationAPIs fails on machines with less than 4 GB > available memory > - > > Key: YARN-4954 > URL: https://issues.apache.org/jira/browse/YARN-4954 > Project: Hadoop YARN > Issue Type: Test > Components: test >Affects Versions: 3.0.0 >Reporter: Gergely Novák >Assignee: Gergely Novák >Priority: Minor > Attachments: YARN-4954.001.patch > > > TestYarnClient.testReservationAPIs sometimes fails with this error: > {noformat} > java.lang.AssertionError: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416) > Caused by: > org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException: > The request cannot be satisfied > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55) > at > org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237) > ... 10 more > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227) > {noformat} > This is caused by really not having enough available memory to complete the > reservation (4 * 1024 MB). In my opinion lowering the required memory (either > by lowering the number of containers to 2, or the memory to 512 MB) would > make the test more stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory
[ https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15239085#comment-15239085 ] Hadoop QA commented on YARN-4954: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 24s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 47s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s {color} | {color:green} trunk passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 18s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 26s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 15s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 38s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s {color} | {color:green} trunk passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 20s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 15s {color} | {color:green} the patch passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 15s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 18s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 12s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 22s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 11s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 46s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s {color} | {color:green} the patch passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 24s {color} | {color:red} hadoop-yarn-client in the patch failed with JDK v1.8.0_77. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 23s {color} | {color:red} hadoop-yarn-client in the patch failed with JDK v1.7.0_95. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 16s {color} | {color:green} Patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 148m 11s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.8.0_77 Failed junit tests | hadoop.yarn.client.api.impl.TestAMRMProxy | | | hadoop.yarn.client.TestGetGroups | | JDK v1.8.0_77 Timed out junit tests | org.apache.hadoop.yarn.client.cli.TestYarnCLI | | | org.apache.hadoop.yarn.client.api.impl.TestYarnClient | | | org.apache.hadoop.yarn.client.api.impl.TestAMRMClient | | | org.apache.hadoop.yarn.client.api.impl.TestNMClient | | JDK v1.7.0_95 Failed junit tests | hadoop.yarn.client.api.impl.TestAMRMProxy | | | hadoop.yarn.client.TestGetGroups | | JDK v1.7.0_95 Timed out junit tests | org.apache.hadoop.yarn.client.cli.TestYarnCLI | | |