erik3001 opened a new issue #10991: URL: https://github.com/apache/pulsar/issues/10991
While upgrading a 2.7.2 cluster to 2.8.0 a client with an mTLS certificate cannot connect anymore. Had to revert the proxies to 2.7.2. Traceback for proxy at v2.8.0: ``` Jun 20 16:44:28 proxy-1 pulsar[8062]: 16:44:28.994 [main] INFO org.eclipse.jetty.server.Server - Started @5274ms Jun 20 16:44:28 proxy-1 pulsar[8062]: 16:44:28.995 [main] INFO org.apache.pulsar.proxy.server.WebServer - Server started at end point http://0.0.0.0:8080 Jun 20 16:44:46 proxy-1 pulsar[8062]: 16:44:46.164 [pulsar-proxy-io-2-1] INFO org.apache.pulsar.proxy.server.ProxyConnection - [/87.233.176.139:58758] New connection opened Jun 20 16:44:46 proxy-1 pulsar[8062]: 16:44:46.224 [pulsar-proxy-io-2-1] WARN org.apache.pulsar.proxy.server.ProxyConnection - [/87.233.176.139:58758] Unable to authenticate: Jun 20 16:44:46 proxy-1 pulsar[8062]: java.lang.IllegalStateException: Field 'auth_data' is not set Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at org.apache.pulsar.common.api.proto.CommandConnect.getAuthDataSlice(CommandConnect.java:90) ~[org.apache.pulsar-pulsar-common-2.8.0.jar:2.8.0] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at org.apache.pulsar.common.api.proto.CommandConnect.getAuthData(CommandConnect.java:83) ~[org.apache.pulsar-pulsar-common-2.8.0.jar:2.8.0] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at org.apache.pulsar.proxy.server.ProxyConnection.handleConnect(ProxyConnection.java:308) [org.apache.pulsar-pulsar-proxy-2.8.0.jar:2.8.0] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:166) [org.apache.pulsar-pulsar-common-2.8.0.jar:2.8.0] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at org.apache.pulsar.proxy.server.ProxyConnection.channelRead(ProxyConnection.java:183) [org.apache.pulsar-pulsar-proxy-2.8.0.jar:2.8.0] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [io.netty-netty-codec-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) [io.netty-netty-codec-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1504) [io.netty-netty-handler-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1265) [io.netty-netty-handler-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1302) [io.netty-netty-handler-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:508) [io.netty-netty-codec-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:447) [io.netty-netty-codec-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) [io.netty-netty-codec-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [io.netty-netty-transport-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [io.netty-netty-transport-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) [io.netty-netty-transport-native-epoll-4.1.63.Final-linux-x86_64.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) [io.netty-netty-transport-native-epoll-4.1.63.Final-linux-x86_64.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) [io.netty-netty-transport-native-epoll-4.1.63.Final-linux-x86_64.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [io.netty-netty-common-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [io.netty-netty-common-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.63.Final.jar:4.1.63.Final] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at java.lang.Thread.run(Thread.java:829) [?:?] Jun 20 16:44:46 proxy-1 pulsar[8062]: 16:44:46.239 [pulsar-proxy-io-2-1] WARN io.netty.channel.DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception. Jun 20 16:44:46 proxy-1 pulsar[8062]: java.lang.NullPointerException: null Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at org.apache.pulsar.proxy.server.ProxyConnection.close(ProxyConnection.java:406) ~[org.apache.pulsar-pulsar-proxy-2.8.0.jar:2.8.0] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at org.apache.pulsar.proxy.server.ProxyConnection.handleConnect(ProxyConnection.java:346) ~[org.apache.pulsar-pulsar-proxy-2.8.0.jar:2.8.0] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:166) ~[org.apache.pulsar-pulsar-common-2.8.0.jar:2.8.0] Jun 20 16:44:46 proxy-1 pulsar[8062]: #011at org.apache.pulsar.proxy.server.ProxyConnection.channelRead(ProxyConnection.java:183) ~[org.apache.pulsar-pulsar-proxy-2.8.0.jar:2.8.0] ``` Using proxy v2.7.2: ``` Jun 20 16:54:16 proxy-1 pulsar[8204]: 16:54:16.174 [pulsar-proxy-io-2-1] INFO org.apache.pulsar.proxy.server.ProxyConnection - [/87.233.176.139:59762] New connection opened Jun 20 16:54:16 proxy-1 pulsar[8204]: 16:54:16.429 [pulsar-proxy-io-2-1] INFO org.apache.pulsar.proxy.server.ProxyConnection - [/87.233.176.139:59762] complete connection, init proxy handler. authenticated with tls role critdev, hasProxyToBrokerUrl: true ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
