[GitHub] [metron] merrimanr commented on issue #1490: METRON-2169: Upgrade to Kafka 2.0.0 and Storm 1.2.1

[GitBox]

merrimanr commented on issue #1490: METRON-2169: Upgrade to Kafka 2.0.0 and 
Storm 1.2.1
URL: https://github.com/apache/metron/pull/1490#issuecomment-524999219
 
 
   I still this error happening intermittently:
   ```
   2019-08-26 13:50:36 ERROR util:0 - Async loop died!
   java.lang.RuntimeException: java.lang.IllegalStateException: This consumer 
has already been closed.
        at 
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522)
        at 
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)
        at 
org.apache.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:477)
        at org.apache.storm.disruptor$consume_batch.invoke(disruptor.clj:70)
        at 
org.apache.storm.daemon.executor$fn__4975$fn__4990$fn__5021.invoke(executor.clj:634)
        at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484)
        at clojure.lang.AFn.run(AFn.java:22)
        at java.lang.Thread.run(Thread.java:748)
   Caused by: java.lang.IllegalStateException: This consumer has already been 
closed.
        at 
org.apache.kafka.clients.consumer.KafkaConsumer.acquireAndEnsureOpen(KafkaConsumer.java:2202)
        at 
org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1984)
        at 
org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1963)
        at 
org.apache.storm.kafka.spout.metrics.KafkaOffsetMetric.getValueAndReset(KafkaOffsetMetric.java:79)
        at 
org.apache.storm.daemon.executor$metrics_tick$fn__4899.invoke(executor.clj:345)
   ```
   I am able to narrow it down to the `FluxTopologyComponent.reset` that gets 
called after every test.  I changed this test to call 
`FluxTopologyComponent.stop` instead, similar to how other integration tests 
work.  I was able to get through several runs without an error but I will keep 
testing.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services