[jira] [Assigned] (YARN-9654) "Null Pointer Exception" when there is Disk Error Exception occurs during Localization

2019-06-27 Thread Akshay Agarwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-9654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akshay Agarwal reassigned YARN-9654:


Assignee: Akshay Agarwal  (was: Bilwa S T)

> "Null Pointer Exception" when there is Disk Error Exception occurs during 
> Localization
> --
>
> Key: YARN-9654
> URL: https://issues.apache.org/jira/browse/YARN-9654
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: nodemanager
>Reporter: Akshay Agarwal
>Assignee: Akshay Agarwal
>Priority: Minor
>
> Currently Null Pointer Exception is thrown when there is Disk Error Exception 
> occurs during Localization
>  
> !http://dts.huawei.com/net/dts/fckeditor/download.ashx?Path=4Ycc0VSuBbmQDP4v02iGYJn%2bVPgAqwGG720AiKXBplWJ7wBLEazrAQ%3d%3d!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Assigned] (YARN-9654) "Null Pointer Exception" when there is Disk Error Exception occurs during Localization

2019-06-27 Thread Akshay Agarwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-9654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akshay Agarwal reassigned YARN-9654:


Assignee: Bilwa S T

> "Null Pointer Exception" when there is Disk Error Exception occurs during 
> Localization
> --
>
> Key: YARN-9654
> URL: https://issues.apache.org/jira/browse/YARN-9654
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: nodemanager
>Reporter: Akshay Agarwal
>Assignee: Bilwa S T
>Priority: Minor
>
> Currently Null Pointer Exception is thrown when there is Disk Error Exception 
> occurs during Localization
>  
> !http://dts.huawei.com/net/dts/fckeditor/download.ashx?Path=4Ycc0VSuBbmQDP4v02iGYJn%2bVPgAqwGG720AiKXBplWJ7wBLEazrAQ%3d%3d!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Created] (YARN-9654) "Null Pointer Exception" when there is Disk Error Exception occurs during Localization

2019-06-27 Thread Akshay Agarwal (JIRA)
Akshay Agarwal created YARN-9654:


 Summary: "Null Pointer Exception" when there is Disk Error 
Exception occurs during Localization
 Key: YARN-9654
 URL: https://issues.apache.org/jira/browse/YARN-9654
 Project: Hadoop YARN
  Issue Type: Bug
  Components: nodemanager
Reporter: Akshay Agarwal


Currently Null Pointer Exception is thrown when there is Disk Error Exception 
occurs during Localization

 

!http://dts.huawei.com/net/dts/fckeditor/download.ashx?Path=4Ycc0VSuBbmQDP4v02iGYJn%2bVPgAqwGG720AiKXBplWJ7wBLEazrAQ%3d%3d!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Assigned] (YARN-8824) App Nodelabel missed after RM restart for finished apps

2018-10-24 Thread Akshay Agarwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-8824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akshay Agarwal reassigned YARN-8824:


Assignee: Bibin A Chundatt  (was: Akshay Agarwal)

> App Nodelabel missed after  RM restart for finished apps
> 
>
> Key: YARN-8824
> URL: https://issues.apache.org/jira/browse/YARN-8824
> Project: Hadoop YARN
>  Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Major
> Fix For: 3.2.0, 3.1.2
>
> Attachments: YARN-8824-branch-3.1.001.patch, YARN-8824.001.patch
>
>
> Similar to YARN-8815 nodelabel for application is lost for finished 
> application after restart



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Assigned] (YARN-8824) App Nodelabel missed after RM restart for finished apps

2018-10-24 Thread Akshay Agarwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-8824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akshay Agarwal reassigned YARN-8824:


Assignee: Akshay Agarwal  (was: Bibin A Chundatt)

> App Nodelabel missed after  RM restart for finished apps
> 
>
> Key: YARN-8824
> URL: https://issues.apache.org/jira/browse/YARN-8824
> Project: Hadoop YARN
>  Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Bibin A Chundatt
>Assignee: Akshay Agarwal
>Priority: Major
> Fix For: 3.2.0, 3.1.2
>
> Attachments: YARN-8824-branch-3.1.001.patch, YARN-8824.001.patch
>
>
> Similar to YARN-8815 nodelabel for application is lost for finished 
> application after restart



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Updated] (YARN-8942) PriorityBasedRouterPolicy throws exception if all sub-cluster weights have negative value

2018-10-24 Thread Akshay Agarwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-8942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akshay Agarwal updated YARN-8942:
-
Description: 
In *PriorityBasedRouterPolicy* if all sub-cluster weights are *set to negative 
values* it is throwing exception while running a job.

Ideally it should handle the negative priority as well according to the home 
sub cluster selection process of the policy.

 *Exception Details:*
{code:java}
java.io.IOException: org.apache.hadoop.yarn.exceptions.YarnException: Unable to 
insert the ApplicationId application_1540356760422_0015 into the 
FederationStateStore
at 
org.apache.hadoop.yarn.server.router.RouterServerUtil.logAndThrowException(RouterServerUtil.java:56)
at 
org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.submitApplication(FederationClientInterceptor.java:418)
at 
org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.submitApplication(RouterClientRMService.java:218)
at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:282)
at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:579)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:818)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2678)
Caused by: 
org.apache.hadoop.yarn.server.federation.store.exception.FederationStateStoreInvalidInputException:
 Missing SubCluster Id information. Please try again by specifying Subcluster 
Id information.
at 
org.apache.hadoop.yarn.server.federation.store.utils.FederationMembershipStateStoreInputValidator.checkSubClusterId(FederationMembershipStateStoreInputValidator.java:247)
at 
org.apache.hadoop.yarn.server.federation.store.utils.FederationApplicationHomeSubClusterStoreInputValidator.checkApplicationHomeSubCluster(FederationApplicationHomeSubClusterStoreInputValidator.java:160)
at 
org.apache.hadoop.yarn.server.federation.store.utils.FederationApplicationHomeSubClusterStoreInputValidator.validate(FederationApplicationHomeSubClusterStoreInputValidator.java:65)
at 
org.apache.hadoop.yarn.server.federation.store.impl.ZookeeperFederationStateStore.addApplicationHomeSubCluster(ZookeeperFederationStateStore.java:159)
at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
at com.sun.proxy.$Proxy84.addApplicationHomeSubCluster(Unknown Source)
at 
org.apache.hadoop.yarn.server.federation.utils.FederationStateStoreFacade.addApplicationHomeSubCluster(FederationStateStoreFacade.java:402)
at 
org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.submitApplication(FederationClientInterceptor.java:413)
... 11 more
{code}
 

  was:
In PriorityBasedRouterPolicy if all sub-cluster weights are set to negative 
values it is throwing exception while running a job.

Ideally it should handle the negative priority as well according to the home 
sub cluster selection process of the policy.

 Exception Details:
{code:java}
java.io.IOException: org.apache.hadoop.yarn.exceptions.YarnException: Unable to 
insert the ApplicationId application_1540356760422_0015 into the 
FederationStateStore
at 
org.apache.hadoop.yarn.server.router.RouterServerUtil.logAndThrowException(RouterServerUtil.java:56)
at 
org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.submitApplication(FederationClientInterceptor.java:418)
at 
org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.submitApplication(RouterClientRMService.java:218)
at 

[jira] [Created] (YARN-8942) PriorityBasedRouterPolicy throws exception if all sub-cluster weights have negative value

2018-10-24 Thread Akshay Agarwal (JIRA)
Akshay Agarwal created YARN-8942:


 Summary: PriorityBasedRouterPolicy throws exception if all 
sub-cluster weights have negative value
 Key: YARN-8942
 URL: https://issues.apache.org/jira/browse/YARN-8942
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Akshay Agarwal


In PriorityBasedRouterPolicy if all sub-cluster weights are set to negative 
values it is throwing exception while running a job.

Ideally it should handle the negative priority as well according to the home 
sub cluster selection process of the policy.

 Exception Details:
{code:java}
java.io.IOException: org.apache.hadoop.yarn.exceptions.YarnException: Unable to 
insert the ApplicationId application_1540356760422_0015 into the 
FederationStateStore
at 
org.apache.hadoop.yarn.server.router.RouterServerUtil.logAndThrowException(RouterServerUtil.java:56)
at 
org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.submitApplication(FederationClientInterceptor.java:418)
at 
org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.submitApplication(RouterClientRMService.java:218)
at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:282)
at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:579)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:818)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2678)
Caused by: 
org.apache.hadoop.yarn.server.federation.store.exception.FederationStateStoreInvalidInputException:
 Missing SubCluster Id information. Please try again by specifying Subcluster 
Id information.
at 
org.apache.hadoop.yarn.server.federation.store.utils.FederationMembershipStateStoreInputValidator.checkSubClusterId(FederationMembershipStateStoreInputValidator.java:247)
at 
org.apache.hadoop.yarn.server.federation.store.utils.FederationApplicationHomeSubClusterStoreInputValidator.checkApplicationHomeSubCluster(FederationApplicationHomeSubClusterStoreInputValidator.java:160)
at 
org.apache.hadoop.yarn.server.federation.store.utils.FederationApplicationHomeSubClusterStoreInputValidator.validate(FederationApplicationHomeSubClusterStoreInputValidator.java:65)
at 
org.apache.hadoop.yarn.server.federation.store.impl.ZookeeperFederationStateStore.addApplicationHomeSubCluster(ZookeeperFederationStateStore.java:159)
at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
at com.sun.proxy.$Proxy84.addApplicationHomeSubCluster(Unknown Source)
at 
org.apache.hadoop.yarn.server.federation.utils.FederationStateStoreFacade.addApplicationHomeSubCluster(FederationStateStoreFacade.java:402)
at 
org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.submitApplication(FederationClientInterceptor.java:413)
... 11 more
{code}


 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Updated] (YARN-8897) LoadBasedRouterPolicy throws "NPE" in case of sub cluster unavailability

2018-10-17 Thread Akshay Agarwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-8897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akshay Agarwal updated YARN-8897:
-
Summary: LoadBasedRouterPolicy throws "NPE" in case of sub cluster 
unavailability   (was: LoadBasedRouterPolicy throws "NullPointerException" if 
no sub cluster is available with cluster weight=1 )

> LoadBasedRouterPolicy throws "NPE" in case of sub cluster unavailability 
> -
>
> Key: YARN-8897
> URL: https://issues.apache.org/jira/browse/YARN-8897
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: federation, router
>Reporter: Akshay Agarwal
>Assignee: Bilwa S T
>Priority: Minor
>
> If no sub clusters are available for "*Load Based Router Policy*" with 
> *cluster weight* as *1*  in Router Based Federation Setup , throwing 
> "*NullPointerException*".
>  
> *Exception Details:*
> {code:java}
> java.lang.NullPointerException: java.lang.NullPointerException
>  at 
> org.apache.hadoop.yarn.server.federation.policies.router.LoadBasedRouterPolicy.getHomeSubcluster(LoadBasedRouterPolicy.java:99)
>  at 
> org.apache.hadoop.yarn.server.federation.policies.RouterPolicyFacade.getHomeSubcluster(RouterPolicyFacade.java:204)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.submitApplication(FederationClientInterceptor.java:362)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.submitApplication(RouterClientRMService.java:218)
>  at 
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:282)
>  at 
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:579)
>  at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
>  at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
>  at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)
>  at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:818)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at javax.security.auth.Subject.doAs(Subject.java:422)
>  at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
>  at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2678)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>  at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>  at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
>  at 
> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateRuntimeException(RPCUtil.java:85)
>  at 
> org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:122)
>  at 
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.submitApplication(ApplicationClientProtocolPBClientImpl.java:297)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498)
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
>  at com.sun.proxy.$Proxy15.submitApplication(Unknown Source)
>  at 
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:288)
>  at 
> org.apache.hadoop.mapred.ResourceMgrDelegate.submitApplication(ResourceMgrDelegate.java:300)
>  at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:331)
>  at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:254)
>  at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570)
>  at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at javax.security.auth.Subject.doAs(Subject.java:422)
>  at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
>  at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
>  at 

[jira] [Created] (YARN-8897) LoadBasedRouterPolicy throws "NullPointerException" if no sub cluster is available with cluster weight=1

2018-10-17 Thread Akshay Agarwal (JIRA)
Akshay Agarwal created YARN-8897:


 Summary: LoadBasedRouterPolicy throws "NullPointerException" if no 
sub cluster is available with cluster weight=1 
 Key: YARN-8897
 URL: https://issues.apache.org/jira/browse/YARN-8897
 Project: Hadoop YARN
  Issue Type: Bug
  Components: federation, router
Reporter: Akshay Agarwal


If no sub clusters are available for "*Load Based Router Policy*" with *cluster 
weight* as *1*  in Router Based Federation Setup , throwing 
"*NullPointerException*".

 

*Exception Details:*
{code:java}
java.lang.NullPointerException: java.lang.NullPointerException
 at 
org.apache.hadoop.yarn.server.federation.policies.router.LoadBasedRouterPolicy.getHomeSubcluster(LoadBasedRouterPolicy.java:99)
 at 
org.apache.hadoop.yarn.server.federation.policies.RouterPolicyFacade.getHomeSubcluster(RouterPolicyFacade.java:204)
 at 
org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.submitApplication(FederationClientInterceptor.java:362)
 at 
org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.submitApplication(RouterClientRMService.java:218)
 at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:282)
 at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:579)
 at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
 at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)
 at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:818)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
 at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2678)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
 at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
 at 
org.apache.hadoop.yarn.ipc.RPCUtil.instantiateRuntimeException(RPCUtil.java:85)
 at org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:122)
 at 
org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.submitApplication(ApplicationClientProtocolPBClientImpl.java:297)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
 at 
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
 at 
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
 at 
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
 at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
 at com.sun.proxy.$Proxy15.submitApplication(Unknown Source)
 at 
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:288)
 at 
org.apache.hadoop.mapred.ResourceMgrDelegate.submitApplication(ResourceMgrDelegate.java:300)
 at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:331)
 at 
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:254)
 at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570)
 at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
 at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
 at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1588)
 at 
org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:307)
 at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:360)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
 at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:368)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at 

[jira] [Commented] (YARN-8829) Cluster metrics can fail with IndexOutOfBound Exception

2018-09-27 Thread Akshay Agarwal (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16630234#comment-16630234
 ] 

Akshay Agarwal commented on YARN-8829:
--

[~sunilg]

After cluster setup, did shutdown of all the sub-clusters , in that case it is 
showing this behaviour.

> Cluster metrics can  fail with IndexOutOfBound Exception
> 
>
> Key: YARN-8829
> URL: https://issues.apache.org/jira/browse/YARN-8829
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Akshay Agarwal
>Assignee: Akshay Agarwal
>Priority: Minor
> Attachments: YARN-8829.001.patch
>
>
> If no sub clusters are available in Router Based Federation Setup ,cluster 
> metrics can throw "IndexOutOfBoundException".
> Additional check is required for sub cluster is Empty.
> *Exception details:*
> {noformat}
> Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, 
> Size: 0
>  at java.util.ArrayList.rangeCheck(ArrayList.java:653)
>  at java.util.ArrayList.get(ArrayList.java:429)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.invokeConcurrent(FederationClientInterceptor.java:654)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.getClusterMetrics(FederationClientInterceptor.java:604)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.getClusterMetrics(RouterClientRMService.java:232)
>  at 
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getClusterMetrics(ApplicationClientProtocolPBServiceImpl.java:253)
>  at 
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:585)
>  at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
>  at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
>  at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Updated] (YARN-8829) Cluster metrics can fail with IndexOutOfBound Exception

2018-09-27 Thread Akshay Agarwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akshay Agarwal updated YARN-8829:
-
Attachment: YARN-8829.001.patch

> Cluster metrics can  fail with IndexOutOfBound Exception
> 
>
> Key: YARN-8829
> URL: https://issues.apache.org/jira/browse/YARN-8829
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Akshay Agarwal
>Assignee: Akshay Agarwal
>Priority: Minor
> Attachments: YARN-8829.001.patch
>
>
> If no sub clusters are available in Router Based Federation Setup ,cluster 
> metrics can throw "IndexOutOfBoundException".
> Additional check is required for sub cluster is Empty.
> *Exception details:*
> {noformat}
> Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, 
> Size: 0
>  at java.util.ArrayList.rangeCheck(ArrayList.java:653)
>  at java.util.ArrayList.get(ArrayList.java:429)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.invokeConcurrent(FederationClientInterceptor.java:654)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.getClusterMetrics(FederationClientInterceptor.java:604)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.getClusterMetrics(RouterClientRMService.java:232)
>  at 
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getClusterMetrics(ApplicationClientProtocolPBServiceImpl.java:253)
>  at 
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:585)
>  at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
>  at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
>  at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Commented] (YARN-8829) Cluster metrics can fail with IndexOutOfBound Exception

2018-09-27 Thread Akshay Agarwal (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16630195#comment-16630195
 ] 

Akshay Agarwal commented on YARN-8829:
--

Attached the patch ! Please review.

> Cluster metrics can  fail with IndexOutOfBound Exception
> 
>
> Key: YARN-8829
> URL: https://issues.apache.org/jira/browse/YARN-8829
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Akshay Agarwal
>Assignee: Akshay Agarwal
>Priority: Minor
> Attachments: YARN-8829.001.patch
>
>
> If no sub clusters are available in Router Based Federation Setup ,cluster 
> metrics can throw "IndexOutOfBoundException".
> Additional check is required for sub cluster is Empty.
> *Exception details:*
> {noformat}
> Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, 
> Size: 0
>  at java.util.ArrayList.rangeCheck(ArrayList.java:653)
>  at java.util.ArrayList.get(ArrayList.java:429)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.invokeConcurrent(FederationClientInterceptor.java:654)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.getClusterMetrics(FederationClientInterceptor.java:604)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.getClusterMetrics(RouterClientRMService.java:232)
>  at 
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getClusterMetrics(ApplicationClientProtocolPBServiceImpl.java:253)
>  at 
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:585)
>  at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
>  at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
>  at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Updated] (YARN-8829) Cluster metrics can fail with IndexOutOfBound Exception

2018-09-27 Thread Akshay Agarwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akshay Agarwal updated YARN-8829:
-
Description: 
If no sub clusters are available in Router Based Federation Setup ,cluster 
metrics can throw "IndexOutOfBoundException".

Additional check is required for sub cluster is Empty.

*Exception details:*
{noformat}
Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
 at java.util.ArrayList.rangeCheck(ArrayList.java:653)
 at java.util.ArrayList.get(ArrayList.java:429)
 at 
org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.invokeConcurrent(FederationClientInterceptor.java:654)
 at 
org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.getClusterMetrics(FederationClientInterceptor.java:604)
 at 
org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.getClusterMetrics(RouterClientRMService.java:232)
 at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getClusterMetrics(ApplicationClientProtocolPBServiceImpl.java:253)
 at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:585)
 at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
 at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)
{noformat}

  was:
If no sub clusters are available in Router Based Federation Setup ,cluster 
metrics can throw "IndexOutOfBoundException".

Additional check is required for sub cluster is Empty.

*Exception details:*
Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
 at java.util.ArrayList.rangeCheck(ArrayList.java:653)
 at java.util.ArrayList.get(ArrayList.java:429)
 at 
org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.invokeConcurrent(FederationClientInterceptor.java:654)
 at 
org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.getClusterMetrics(FederationClientInterceptor.java:604)
 at 
org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.getClusterMetrics(RouterClientRMService.java:232)
 at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getClusterMetrics(ApplicationClientProtocolPBServiceImpl.java:253)
 at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:585)
 at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
 at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)


> Cluster metrics can  fail with IndexOutOfBound Exception
> 
>
> Key: YARN-8829
> URL: https://issues.apache.org/jira/browse/YARN-8829
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Akshay Agarwal
>Assignee: Akshay Agarwal
>Priority: Minor
>
> If no sub clusters are available in Router Based Federation Setup ,cluster 
> metrics can throw "IndexOutOfBoundException".
> Additional check is required for sub cluster is Empty.
> *Exception details:*
> {noformat}
> Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, 
> Size: 0
>  at java.util.ArrayList.rangeCheck(ArrayList.java:653)
>  at java.util.ArrayList.get(ArrayList.java:429)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.invokeConcurrent(FederationClientInterceptor.java:654)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.getClusterMetrics(FederationClientInterceptor.java:604)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.getClusterMetrics(RouterClientRMService.java:232)
>  at 
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getClusterMetrics(ApplicationClientProtocolPBServiceImpl.java:253)
>  at 
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:585)
>  at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
>  at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
>  at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Created] (YARN-8829) Cluster metrics can fail with IndexOutOfBound Exception

2018-09-27 Thread Akshay Agarwal (JIRA)
Akshay Agarwal created YARN-8829:


 Summary: Cluster metrics can  fail with IndexOutOfBound Exception
 Key: YARN-8829
 URL: https://issues.apache.org/jira/browse/YARN-8829
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Akshay Agarwal


If no sub clusters are available in Router Based Federation Setup ,cluster 
metrics can throw "IndexOutOfBoundException".

Additional check is required for sub cluster is Empty.

*Exception details:*
Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
 at java.util.ArrayList.rangeCheck(ArrayList.java:653)
 at java.util.ArrayList.get(ArrayList.java:429)
 at 
org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.invokeConcurrent(FederationClientInterceptor.java:654)
 at 
org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.getClusterMetrics(FederationClientInterceptor.java:604)
 at 
org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.getClusterMetrics(RouterClientRMService.java:232)
 at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getClusterMetrics(ApplicationClientProtocolPBServiceImpl.java:253)
 at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:585)
 at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
 at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Assigned] (YARN-8829) Cluster metrics can fail with IndexOutOfBound Exception

2018-09-27 Thread Akshay Agarwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akshay Agarwal reassigned YARN-8829:


Assignee: Akshay Agarwal

> Cluster metrics can  fail with IndexOutOfBound Exception
> 
>
> Key: YARN-8829
> URL: https://issues.apache.org/jira/browse/YARN-8829
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Akshay Agarwal
>Assignee: Akshay Agarwal
>Priority: Minor
>
> If no sub clusters are available in Router Based Federation Setup ,cluster 
> metrics can throw "IndexOutOfBoundException".
> Additional check is required for sub cluster is Empty.
> *Exception details:*
> Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, 
> Size: 0
>  at java.util.ArrayList.rangeCheck(ArrayList.java:653)
>  at java.util.ArrayList.get(ArrayList.java:429)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.invokeConcurrent(FederationClientInterceptor.java:654)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.getClusterMetrics(FederationClientInterceptor.java:604)
>  at 
> org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.getClusterMetrics(RouterClientRMService.java:232)
>  at 
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getClusterMetrics(ApplicationClientProtocolPBServiceImpl.java:253)
>  at 
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:585)
>  at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
>  at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
>  at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Created] (YARN-8671) Container Launch failed stating "TaskAttempt killed because it ran on unusable node , Container released on a *lost* node"

2018-08-16 Thread Akshay Agarwal (JIRA)
Akshay Agarwal created YARN-8671:


 Summary: Container Launch failed stating "TaskAttempt killed 
because it ran on unusable node , Container released on a *lost* node"
 Key: YARN-8671
 URL: https://issues.apache.org/jira/browse/YARN-8671
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn
Affects Versions: 3.1.1
Reporter: Akshay Agarwal


Pre-requisites:
{code:java}
1. Install HA cluster.
2.Set yarn.nodemanager.opportunistic-containers-max-queue-length=(positive 
integer value)[NodeManager->yarnsite.xml]
3. Set yarn.resourcemanager.opportunistic-container-allocation.enabled= 
true[ResourceManager->yarnsite.xml]
{code}
 

Steps to reproduce:
{code:java}
1.Keep All NodeManagers Up
2.Stop 2 Nodemanagers and immediately follow step 3.
3.Submit a job with -Dmapreduce.job.num-opportunistic-maps-percent="100" and 
run with 50 mappers  
{code}
Expected Result:
{code:java}
Job should be successfull {code}
Actual Result:
{code:java}
Job is getting successfull but some containers are failing stating 

TaskAttempt killed because it ran on unusable node , Container released on a 
*lost* node"
{code}
 

Log Details:
{code:java}
TaskAttempt killed because it ran on unusable node Container released on a 
*lost* node Container launch failed for container_1534149133116_0019_01_06 
: java.net.ConnectException: Call From hostname/IP to hostname:portNumber 
failed on connection exception: java.net.ConnectException: 
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Created] (YARN-8663) Opportunistic Container property "mapreduce.job.num-opportunistic-maps-percent" is throwing wrong exception at wrong sequence

2018-08-14 Thread Akshay Agarwal (JIRA)
Akshay Agarwal created YARN-8663:


 Summary: Opportunistic Container property 
"mapreduce.job.num-opportunistic-maps-percent" is throwing wrong exception at 
wrong sequence
 Key: YARN-8663
 URL: https://issues.apache.org/jira/browse/YARN-8663
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn
Affects Versions: 3.1.1
 Environment: Secure Installation with Kerberos ON.
Reporter: Akshay Agarwal


Pre-requisites:

{code:java}
1. Install HA cluster.
2.Set yarn.nodemanager.opportunistic-containers-max-queue-length=(positive 
integer value)[NodeManager->yarnsite.xml]
3. Set yarn.resourcemanager.opportunistic-container-allocation.enabled= 
true[ResourceManager->yarnsite.xml]
{code}

 

Steps to reproduce:

{code:java}
1.Keep All NodeManagers Up
2. Submit a job with -Dmapreduce.job.num-opportunistic-maps-percent="abh" or 
"2.5" 
{code}


Expected Result: 
{code:java}
Should through an Exception stating "NumberFormatException" before writing the 
input for mappers.
{code}

Log Details:
{code:java}
2018-08-14 18:15:54,049 INFO mapreduce.Job:  map 0% reduce 0%
2018-08-14 18:15:54,069 INFO mapreduce.Job: Job job_1534236847054_0005 failed 
with state FAILED due to: Application application_1534236847054_0005 failed 2 
times due to AM Container for appattempt_1534236847054_0005_02 exited with  
exitCode: 1
Failing this attempt.Diagnostics: [2018-08-14 18:15:53.110]Exception from 
container-launch.
Container id: container_e31_1534236847054_0005_02_01
Exit code: 1

[2018-08-14 18:15:53.113]Container exited with a non-zero exit code 1. Error 
file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier; 
support was removed in 8.0
Aug 14, 2018 6:15:51 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver 
as a provider class
Aug 14, 2018 6:15:51 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a 
provider class
Aug 14, 2018 6:15:51 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a 
root resource class
Aug 14, 2018 6:15:51 PM 
com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
Aug 14, 2018 6:15:51 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory 
getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to 
GuiceManagedComponentProvider with the scope "Singleton"
Aug 14, 2018 6:15:52 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory 
getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to 
GuiceManagedComponentProvider with the scope "Singleton"
Aug 14, 2018 6:15:52 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory 
getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices to 
GuiceManagedComponentProvider with the scope "PerRequest"
log4j:WARN No appenders could be found for logger 
(org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more 
info.
{code}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org