[ https://issues.apache.org/jira/browse/METRON-1584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494406#comment-16494406 ]
ASF GitHub Bot commented on METRON-1584: ---------------------------------------- Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1036 I want to take a step back a moment here. I think there are some useful, clarifying refactorings in this PR. I'm generally in favor of refactoring improvements e.g. "handleTick(tuple)" that sum up the code nicely. I also like adding sensible clarifying comments and making tests more clear. However, I'm not sure how I feel about mixing refactoring like this with the bug fix itself. It seems to me that the actual bug fix is roughly 5-10 lines of code. It makes it more difficult to determine code that's specific to the change in mind. I noticed this while reviewing https://github.com/apache/metron/pull/1015 as well. Might it be better to do separate refactoring PR's? I am certainly not suggesting we be pedantic about it - I'm not even clear how I would even suggest a reasonable rule to follow, to be honest. Part of me thinks "ok, broken window theory and boyscout rules - this is great stuff." But in general, I think that splitting the refactoring will allow reviewers to voice opinions specific to coding style and refactoring changes as distinct from a feature or bug fix itself. It's clear that we all have varying opinions on what reads best in our code, and I think splitting those discussions off will make the review process more expedient. > Indexing Topology Crashes with Invalid Message > ---------------------------------------------- > > Key: METRON-1584 > URL: https://issues.apache.org/jira/browse/METRON-1584 > Project: Metron > Issue Type: Bug > Reporter: Nick Allen > Assignee: Nick Allen > Priority: Major > > Per Mohan Venkateshaiah: > I published message "adkadknalkda;LK;ad;Da;dD;" to indexing topic , I see > that the random access indexing topology worker thread died and couldn't > recover until the kafka topic was deleted and recreated. > {code:java} > Caused by: java.lang.IllegalStateException: Unable to parse > adkadknalkda;LK;ad;Da;dD; due to null > at > org.apache.metron.common.message.JSONFromPosition.get(JSONFromPosition.java:49) > ~[stormjar.jar:?] > at > org.apache.metron.common.message.JSONFromPosition.get(JSONFromPosition.java:25) > ~[stormjar.jar:?] > at > org.apache.metron.writer.bolt.BulkMessageWriterBolt.execute(BulkMessageWriterBolt.java:237) > ~[stormjar.jar:?] > at > org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735) > ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292] > at > org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466) > ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292] > at > org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40) > ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292] > at > org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) > ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292] > ... 6 more > Caused by: org.json.simple.parser.ParseException > at org.json.simple.parser.Yylex.yylex(Yylex.java:610) ~[stormjar.jar:?] > at org.json.simple.parser.JSONParser.nextToken(JSONParser.java:269) > ~[stormjar.jar:?] > at org.json.simple.parser.JSONParser.parse(JSONParser.java:118) > ~[stormjar.jar:?] > at org.json.simple.parser.JSONParser.parse(JSONParser.java:81) > ~[stormjar.jar:?] > at org.json.simple.parser.JSONParser.parse(JSONParser.java:75) > ~[stormjar.jar:?] > at > org.apache.metron.common.message.JSONFromPosition.get(JSONFromPosition.java:47) > ~[stormjar.jar:?] > at > org.apache.metron.common.message.JSONFromPosition.get(JSONFromPosition.java:25) > ~[stormjar.jar:?] > at > org.apache.metron.writer.bolt.BulkMessageWriterBolt.execute(BulkMessageWriterBolt.java:237) > ~[stormjar.jar:?] > at > org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735) > ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292] > at > org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466) > ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292] > at > org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40) > ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292] > at > org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) > ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292] > ... 6 more > 2018-05-24 09:21:22.236 o.a.s.util Thread-9-indexingBolt-executor[3 3] > [ERROR] Halting process: ("Worker died") > java.lang.RuntimeException: ("Worker died") > at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341) > [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292] > at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?] > at org.apache.storm.daemon.worker$fn__10799$fn__10800.invoke(worker.clj:763) > [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292] > at > org.apache.storm.daemon.executor$mk_executor_data$fn__10011$fn__10012.invoke(executor.clj:276) > [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292] > at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:494) > [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292] > at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161] > 2018-05-24 09:21:22.237 o.a.s.d.worker Thread-20 [INFO] Shutting down worker > random_access_indexing-24-1527147389 703b5bf7-6c9d-46f3-8136-0c4877a69375 6700 > 2018-05-24 09:21:22.237 o.a.s.d.worker Thread-20 [INFO] Terminating messaging > context > 2018-05-24 09:21:22.238 o.a.s.d.worker Thread-20 [INFO] Shutting down > executors > 2018-05-24 09:21:22.238 o.a.s.d.executor Thread-20 [INFO] Shutting down > executor > __metricsorg.apache.hadoop.metrics2.sink.storm.StormTimelineMetricsSink:[2 2] > 2018-05-24 09:21:22.239 o.a.s.util Thread-6-disruptor-executor[2 > 2]-send-queue [INFO] Async loop interrupted! > 2018-05-24 09:21:22.239 o.a.s.util > Thread-7-__metricsorg.apache.hadoop.metrics2.sink.storm.StormTimelineMetricsSink-executor[2 > 2] [INFO] Async loop interrupted! > 2018-05-24 09:21:22.240 o.a.h.m.s.s.StormTimelineMetricsSink Thread-20 [INFO] > Stopping Storm Metrics Sink > 2018-05-24 09:21:22.240 o.a.s.d.executor Thread-20 [INFO] Shut down executor > __metricsorg.apache.hadoop.metrics2.sink.storm.StormTimelineMetricsSink:[2 2] > 2018-05-24 09:21:22.241 o.a.s.d.executor Thread-20 [INFO] Shutting down > executor indexingBolt:[3 3] > {code} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)