Sijie, The files I attached where the only ones I could capture, the standalone broker log files were never written to unless I am looking in the wrong directory.
Lowell From: Sijie Guo <guosi...@gmail.com> Sent: Monday, August 20, 2018 4:26 PM To: users@pulsar.incubator.apache.org Subject: Re: EXTERNAL: Re: Runtime error after upgrading to 2.1.0 Hi Schaffert, Sorry that I missed your email last week. The stackoverflow exception seems coming from server side. Do you happen to check the broker log? Is there anything suspicious in the broker log? - Sijie On Wed, Aug 15, 2018 at 3:09 PM Schaffert, Lowell <lowell.schaff...@lmco.com<mailto:lowell.schaff...@lmco.com>> wrote: Temp1 is the log from the producer app run 1 temp3 is the log from the producer app run 2 Temp2 is the console log from the broker Any insight would be appreciated. Lowell ________________________________ From: Schaffert, Lowell (US) Sent: Wednesday, August 15, 2018 3:44:50 PM To: users@pulsar.incubator.apache.org<mailto:users@pulsar.incubator.apache.org> Subject: Re: EXTERNAL: Re: Runtime error after upgrading to 2.1.0 Actually I was able to reproduce the problem from the command line running Spring Boot, looking for the root cause now. Lowell ________________________________ From: Schaffert, Lowell (US) Sent: Wednesday, August 15, 2018 3:10:32 PM To: users@pulsar.incubator.apache.org<mailto:users@pulsar.incubator.apache.org> Subject: Re: EXTERNAL: Re: Runtime error after upgrading to 2.1.0 The issue is a strange combination of events caused by some change in the Pulsar client (2.0.1 -> 2.1.0) that causes a Spring Boot run within IntelliJ to recursively create producers (about 1640 verses the 2 that it normally creates). This occurs if the Spring Boot app is restarted, no issue on the first run. If the Spring Boot application is run from the command line I do not have this issue. Very strange. Lowell ________________________________ From: Sijie Guo <guosi...@gmail.com<mailto:guosi...@gmail.com>> Sent: Wednesday, August 15, 2018 12:01:53 PM To: users@pulsar.incubator.apache.org<mailto:users@pulsar.incubator.apache.org> Subject: EXTERNAL: Re: Runtime error after upgrading to 2.1.0 Hi Schaffert, It seems the `StackoverflowException` is coming from broker side. Did you see any logs at broker side? Do you mind giving a bit more context on your application? For example, are you using a Schema? and what schema are you using? > Another data point: running standalone, if I stop the broker and delete the > contents of the data directory things will run until I close and restart a > producer. You mean the producer keeps running when you kill standalone? - Sijie On Wed, Aug 15, 2018 at 8:06 AM Schaffert, Lowell <lowell.schaff...@lmco.com<mailto:lowell.schaff...@lmco.com><mailto:lowell.schaff...@lmco.com<mailto:lowell.schaff...@lmco.com>>> wrote: Another data point: running standalone, if I stop the broker and delete the contents of the data directory things will run until I close and restart a producer. Lowell ________________________________ From: Schaffert, Lowell (US) Sent: Wednesday, August 15, 2018 8:29:13 AM To: Pulsar User Group Subject: Re: Runtime error after upgrading to 2.1.0 I have been using the 2.1.0 broker since it came out, this is a result of updating my pom to use 2.1.0 Lowell ________________________________ From: Schaffert, Lowell (US) Sent: Wednesday, August 15, 2018 8:27:13 AM To: Pulsar User Group Subject: Re: Runtime error after upgrading to 2.1.0 2018-08-15 08:20:56.911 WARN 31687 --- [r-client-io-5-1] org.apache.pulsar.client.impl.ClientCnx : [id: 0xaa2cec05, L:/127.0.0.1:58224<http://127.0.0.1:58224><http://127.0.0.1:58224> - R:localhost/127.0.0.1:6650<http://127.0.0.1:6650><http://127.0.0.1:6650>] Received error from server: java.lang.StackOverflowError 2018-08-15 08:20:56.912 ERROR 31687 --- [r-client-io-5-1] o.a.pulsar.client.impl.ProducerImpl : [persistent://public/default/omni.diamd.unit_type] [null] Failed to create producer: java.lang.StackOverflowError 2018-08-15 08:20:56.912 ERROR 31687 --- [enerContainer-1] updSvcMessageLogger : publishUnitTypeToTopic Lowell ________________________________ From: Schaffert, Lowell (US) Sent: Wednesday, August 15, 2018 8:25:22 AM To: Pulsar User Group Subject: Runtime error after upgrading to 2.1.0 org.apache.pulsar.client.api.PulsarClientException: java.lang.StackOverflowError at org.apache.pulsar.client.impl.ClientCnx.getPulsarClientException(ClientCnx.java:760) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.client.impl.ClientCnx.handleError(ClientCnx.java:497) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.common.api.PulsarDecoder.channelRead(PulsarDecoder.java:154) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1414) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:945) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:146) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at org.apache.pulsar.shade.io<http://org.apache.pulsar.shade.io><http://org.apache.pulsar.shade.io>.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[pulsar-client-2.1.0-incubating.jar:2.1.0-incubating] at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_171] Lowell
