Re: Any hope for a fix of STORM-3032 ?

[Stig Rohde Døssing]

Looking at the stack trace, the issue is not in the Kafka bolt, but in some
internal Storm statistics code. It seems likely to me that it has been
fixed in 2.0.0, as we haven't heard of anyone encounter it there. Are you
able to upgrade to 2.x to see if that fixes this?

If not, the exception is coming from
https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131.
Looks like either "amt" or " (stats-rate stats) " in that line is null. My
guess would be stats. The equivalent code in 2.x uses primitive ints, so
won't be throwing NPEs
https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63

If you have the ability to upgrade to 2.0.0 that would be best.

I don't think simply catching the NPE is a good solution, we should prevent
the NPE from being thrown in the first place. If you want to take a look at
it, feel free to raise a PR. May want to also lobby to get a 1.2.4 release
out after that.

Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
avermeerber...@gmail.com>:

> Hello,
>
> I have seen couple of occurrences of the NullPointerException in Storm
> Kafka Spout with same stack trace as in
> https://issues.apache.org/jira/browse/STORM-3032
>
> Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected to a
> cluster of Kafka Brokers at 2.2.1 version.
>
> Since the stack traces seems to tell that the NullPointerException
> occurs in some code related to statistics computation, wouldn't it be
> better to fix the issue by catching the NullPointerException to avoid
> blocking Kafka consumption ?
>
> Kind regards,
> Alexandre Vermeerbergen
>

[GitHub] [storm] cjljohnson opened a new pull request #3131: STORM-3510: Track overflow count per taskId for resending backpressur…

[GitBox]

cjljohnson opened a new pull request #3131: STORM-3510: Track overflow count 
per taskId for resending backpressur…
URL: https://github.com/apache/storm/pull/3131
 
 
   …e status
   
   This is a fix to the WorkerState.transferLocalBatch logic for keeping track 
of how often to resend backpressure status updates as the overflow queue grows.
   
   Previously a single counter was shared and overwritten for all executors and 
not retained between mini-batches.
   
   This fix stores the overflow count in a map keyed by taskId and stored in 
BackPressureTracker.java so values from different executors don't overwrite 
each other and a status update is only sent once per 1 tuples rather than 
every mini batch once the threshold has been reached.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services