[jira] [Updated] (STORM-3728) Workers are not able to connect to Pacemaker if pacemaker.auth.method is KERBEROS

2020-12-23 Thread Ethan Li (Jira) 


 [ 
https://issues.apache.org/jira/browse/STORM-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ethan Li updated STORM-3728:

Fix Version/s: 2.3.0

> Workers are not able to connect to Pacemaker if pacemaker.auth.method is 
> KERBEROS
> -
>
> Key: STORM-3728
> URL: https://issues.apache.org/jira/browse/STORM-3728
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1.0, 2.2.0
>Reporter: Ethan Li
>Assignee: Ethan Li
>Priority: Major
> Fix For: 2.3.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> When pacemaker.auth.method is KERBEROS,  worker will fail to connect to 
> KERBEROS because of exceptions like the following:
>  
> {code:java}
> 2020-12-21 20:07:00.786 o.a.s.c.PaceMakerStateStorage 
> executor-heartbeat-timer [ERROR] Timed out waiting for channel ready. Failed 
> to set_worker_hb. Will make 2 more attempts.
> 2020-12-21 20:07:00.902 o.a.s.m.n.KerberosSaslClientHandler 
> openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [INFO] Connection established 
> from /10.215.73.209:45548 to 
> openstorm3blue-n10.blue.ygrid.yahoo.com/10.215.79.152:6699
> 2020-12-21 20:07:00.903 o.a.s.m.n.KerberosSaslNettyClient 
> openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [INFO] Creating Kerberos Client.
> 2020-12-21 20:07:00.906 o.a.s.m.n.KerberosSaslNettyClient 
> openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [INFO] Kerberos Client Callback 
> Handler got callback: class javax.security.auth.callback.PasswordCallback
> 2020-12-21 20:07:00.906 o.a.s.m.n.Login 
> openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [ERROR] Login using jaas conf 
> /home/y/lib/storm/current/conf/storm_jaas.conf failed
> 2020-12-21 20:07:00.906 o.a.s.m.n.KerberosSaslNettyClient 
> openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [ERROR] Client failed to login 
> in principal:javax.security.auth.login.LoginException: No password provided
> javax.security.auth.login.LoginException: No password provided
> at 
> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:923)
>  ~[?:1.8.0_262]
> at 
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:764)
>  ~[?:1.8.0_262]
> at 
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:618) 
> ~[?:1.8.0_262]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[?:1.8.0_262]
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[?:1.8.0_262]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_262]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_262]
> at 
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) 
> ~[?:1.8.0_262]
> at 
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) 
> ~[?:1.8.0_262]
> at 
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) 
> ~[?:1.8.0_262]
> at 
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) 
> ~[?:1.8.0_262]
> at java.security.AccessController.doPrivileged(Native Method) 
> ~[?:1.8.0_262]
> at 
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) 
> ~[?:1.8.0_262]
> at 
> javax.security.auth.login.LoginContext.login(LoginContext.java:587) 
> ~[?:1.8.0_262]
> at org.apache.storm.messaging.netty.Login.login(Login.java:301) 
> ~[storm-client-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at org.apache.storm.messaging.netty.Login.(Login.java:83) 
> ~[storm-client-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at 
> org.apache.storm.messaging.netty.KerberosSaslNettyClient.(KerberosSaslNettyClient.java:66)
>  [storm-client-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at 
> org.apache.storm.messaging.netty.KerberosSaslClientHandler.channelActive(KerberosSaslClientHandler.java:59)
>  [storm-client-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at 
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:213)
>  [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at 
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:199)
>  [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at 
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:192)
>  [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at 
> org.apache.storm.shade.io.netty.channel.ChannelInboundHandlerAdapter.channelActive(ChannelInboundHandlerAdapter.java:64)
>  [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at 
> org.apache.storm.shade.

[jira] [Updated] (STORM-3728) Workers are not able to connect to Pacemaker if pacemaker.auth.method is KERBEROS

2020-12-23 Thread Ethan Li (Jira) 


 [ 
https://issues.apache.org/jira/browse/STORM-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ethan Li updated STORM-3728:

Affects Version/s: 2.2.0
   2.0.0
   2.1.0

> Workers are not able to connect to Pacemaker if pacemaker.auth.method is 
> KERBEROS
> -
>
> Key: STORM-3728
> URL: https://issues.apache.org/jira/browse/STORM-3728
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1.0, 2.2.0
>Reporter: Ethan Li
>Assignee: Ethan Li
>Priority: Major
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> When pacemaker.auth.method is KERBEROS,  worker will fail to connect to 
> KERBEROS because of exceptions like the following:
>  
> {code:java}
> 2020-12-21 20:07:00.786 o.a.s.c.PaceMakerStateStorage 
> executor-heartbeat-timer [ERROR] Timed out waiting for channel ready. Failed 
> to set_worker_hb. Will make 2 more attempts.
> 2020-12-21 20:07:00.902 o.a.s.m.n.KerberosSaslClientHandler 
> openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [INFO] Connection established 
> from /10.215.73.209:45548 to 
> openstorm3blue-n10.blue.ygrid.yahoo.com/10.215.79.152:6699
> 2020-12-21 20:07:00.903 o.a.s.m.n.KerberosSaslNettyClient 
> openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [INFO] Creating Kerberos Client.
> 2020-12-21 20:07:00.906 o.a.s.m.n.KerberosSaslNettyClient 
> openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [INFO] Kerberos Client Callback 
> Handler got callback: class javax.security.auth.callback.PasswordCallback
> 2020-12-21 20:07:00.906 o.a.s.m.n.Login 
> openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [ERROR] Login using jaas conf 
> /home/y/lib/storm/current/conf/storm_jaas.conf failed
> 2020-12-21 20:07:00.906 o.a.s.m.n.KerberosSaslNettyClient 
> openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [ERROR] Client failed to login 
> in principal:javax.security.auth.login.LoginException: No password provided
> javax.security.auth.login.LoginException: No password provided
> at 
> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:923)
>  ~[?:1.8.0_262]
> at 
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:764)
>  ~[?:1.8.0_262]
> at 
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:618) 
> ~[?:1.8.0_262]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[?:1.8.0_262]
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[?:1.8.0_262]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_262]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_262]
> at 
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) 
> ~[?:1.8.0_262]
> at 
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) 
> ~[?:1.8.0_262]
> at 
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) 
> ~[?:1.8.0_262]
> at 
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) 
> ~[?:1.8.0_262]
> at java.security.AccessController.doPrivileged(Native Method) 
> ~[?:1.8.0_262]
> at 
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) 
> ~[?:1.8.0_262]
> at 
> javax.security.auth.login.LoginContext.login(LoginContext.java:587) 
> ~[?:1.8.0_262]
> at org.apache.storm.messaging.netty.Login.login(Login.java:301) 
> ~[storm-client-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at org.apache.storm.messaging.netty.Login.(Login.java:83) 
> ~[storm-client-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at 
> org.apache.storm.messaging.netty.KerberosSaslNettyClient.(KerberosSaslNettyClient.java:66)
>  [storm-client-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at 
> org.apache.storm.messaging.netty.KerberosSaslClientHandler.channelActive(KerberosSaslClientHandler.java:59)
>  [storm-client-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at 
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:213)
>  [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at 
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:199)
>  [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at 
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:192)
>  [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at 
> org.apache.storm.shade.io.netty.channel.ChannelInboundHandlerAdapter.channelActive(ChannelInboundHandlerAdapter.java:64)
>  [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
> at

[jira] [Updated] (STORM-3728) Workers are not able to connect to Pacemaker if pacemaker.auth.method is KERBEROS

2020-12-21 Thread Ethan Li (Jira) 


 [ 
https://issues.apache.org/jira/browse/STORM-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ethan Li updated STORM-3728:

Description: 
When pacemaker.auth.method is KERBEROS,  worker will fail to connect to 
KERBEROS because of exceptions like the following:
 
{code:java}
2020-12-21 20:07:00.786 o.a.s.c.PaceMakerStateStorage executor-heartbeat-timer 
[ERROR] Timed out waiting for channel ready. Failed to set_worker_hb. Will make 
2 more attempts.
2020-12-21 20:07:00.902 o.a.s.m.n.KerberosSaslClientHandler 
openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [INFO] Connection established from 
/10.215.73.209:45548 to 
openstorm3blue-n10.blue.ygrid.yahoo.com/10.215.79.152:6699
2020-12-21 20:07:00.903 o.a.s.m.n.KerberosSaslNettyClient 
openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [INFO] Creating Kerberos Client.
2020-12-21 20:07:00.906 o.a.s.m.n.KerberosSaslNettyClient 
openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [INFO] Kerberos Client Callback 
Handler got callback: class javax.security.auth.callback.PasswordCallback
2020-12-21 20:07:00.906 o.a.s.m.n.Login 
openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [ERROR] Login using jaas conf 
/home/y/lib/storm/current/conf/storm_jaas.conf failed
2020-12-21 20:07:00.906 o.a.s.m.n.KerberosSaslNettyClient 
openstorm3blue-n10.blue.ygrid.yahoo.com-pm-1 [ERROR] Client failed to login in 
principal:javax.security.auth.login.LoginException: No password provided
javax.security.auth.login.LoginException: No password provided
at 
com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:923)
 ~[?:1.8.0_262]
at 
com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:764)
 ~[?:1.8.0_262]
at 
com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:618) 
~[?:1.8.0_262]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[?:1.8.0_262]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_262]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_262]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_262]
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) 
~[?:1.8.0_262]
at 
javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) 
~[?:1.8.0_262]
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) 
~[?:1.8.0_262]
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) 
~[?:1.8.0_262]
at java.security.AccessController.doPrivileged(Native Method) 
~[?:1.8.0_262]
at 
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) 
~[?:1.8.0_262]
at javax.security.auth.login.LoginContext.login(LoginContext.java:587) 
~[?:1.8.0_262]
at org.apache.storm.messaging.netty.Login.login(Login.java:301) 
~[storm-client-2.3.0.y.jar:2.3.0-SNAPSHOT]
at org.apache.storm.messaging.netty.Login.(Login.java:83) 
~[storm-client-2.3.0.y.jar:2.3.0-SNAPSHOT]
at 
org.apache.storm.messaging.netty.KerberosSaslNettyClient.(KerberosSaslNettyClient.java:66)
 [storm-client-2.3.0.y.jar:2.3.0-SNAPSHOT]
at 
org.apache.storm.messaging.netty.KerberosSaslClientHandler.channelActive(KerberosSaslClientHandler.java:59)
 [storm-client-2.3.0.y.jar:2.3.0-SNAPSHOT]
at 
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:213)
 [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
at 
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:199)
 [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
at 
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:192)
 [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
at 
org.apache.storm.shade.io.netty.channel.ChannelInboundHandlerAdapter.channelActive(ChannelInboundHandlerAdapter.java:64)
 [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
at 
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:213)
 [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
at 
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:199)
 [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
at 
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:192)
 [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
at 
org.apache.storm.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1422)
 [storm-shaded-deps-2.3.0.y.jar:2.3.0-SNAPSHOT]
at 
org.apache.storm.shad