[
https://issues.apache.org/jira/browse/METRON-253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345111#comment-15345111
]
ASF GitHub Bot commented on METRON-253:
---------------------------------------
GitHub user cestella opened a pull request:
https://github.com/apache/incubator-metron/pull/170
METRON-253: Reloading parser configurations from zookeeper doesn't call
init()
This results in a NPE if you do not specify an output column (this output
column gets inferred as part of init())
To validate this:
* Get a running topology
* Run `/usr/metron/0.2.0BETA/bin/zk_load_configs.sh -m PUSH -i
/usr/metron/0.2.0BETA/config/zookeeper -z node1:2181`
* Wait a full 2 minutes
* Check the `parser_error` topic via the kafka console consumer and ensure
you don't see any weird NPEs
The problem here is that init() is not getting run on the sensor parser
config, so some syntactic sugar in the FieldTransformer isn't getting updated.
Specifically, if you only specify `input` and not `output`, then it will NPE.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cestella/incubator-metron yaf_npe
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/170.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #170
----
commit b0d391c143e3f14a16cadcac6ab91e6863b1be8d
Author: cstella <[email protected]>
Date: 2016-06-22T20:27:31Z
METRON-253: Reloading parser configurations from zookeeper doesn't call
init()
----
> Reloading parser configurations from zookeeper doesn't call init()
> ------------------------------------------------------------------
>
> Key: METRON-253
> URL: https://issues.apache.org/jira/browse/METRON-253
> Project: Metron
> Issue Type: Bug
> Reporter: Casey Stella
> Assignee: Casey Stella
>
> This results in a NPE if you do not specify an output column (this output
> column gets inferred as part of init()):
> {"exception":"java.lang.NullPointerException","hostname":"ip-10-0-0-119.us-west-2.compute.internal","stack":"java.lang.NullPointerException\n\tat
>
> org.apache.metron.common.field.transformation.SimpleFieldTransformation.map(SimpleFieldTransformation.java:38)\n\tat
>
> org.apache.metron.common.configuration.FieldTransformer.transform(FieldTransformer.java:107)\n\tat
>
> org.apache.metron.common.configuration.FieldTransformer.transformAndUpdate(FieldTransformer.java:112)\n\tat
> org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:123)\n\tat
> backtype.storm.daemon.executor$fn__5495$tuple_action_fn__5497.invoke(executor.clj:670)\n\tat
>
> backtype.storm.daemon.executor$mk_task_receiver$fn__5418.invoke(executor.clj:426)\n\tat
>
> backtype.storm.disruptor$clojure_handler$reify__4994.onEvent(disruptor.clj:58)\n\tat
>
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)\n\tat
>
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)\n\tat
>
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)\n\tat
>
> backtype.storm.daemon.executor$fn__5495$fn__5508$fn__5559.invoke(executor.clj:808)\n\tat
> backtype.storm.util$async_loop$fn__543.invoke(util.clj:475)\n\tat
> clojure.lang.AFn.run(AFn.java:22)\n\tat
> java.lang.Thread.run(Thread.java:745)\n","time":1466617043534,"rawMessage_bytes":[50,48,49,54,45,48,54,45,50,50,32,49,55,58,51,53,58,53,49,46,56,56,56,124,50,48,49,54,45,48,54,45,50,50,32,49,55,58,51,55,58,48,57,46,55,49,51,124,32,32,55,55,46,56,50,53,124,32,32,32,48,46,49,55,51,124,32,32,54,124,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,49,57,50,46,49,54,56,46,49,51,56,46,49,53,56,124,52,57,50,48,54,124,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,57,53,46,49,54,51,46,49,50,49,46,50,48,52,124,32,32,32,56,48,124,32,32,32,32,32,32,32,83,124,32,32,32,32,32,65,80,82,124,32,32,32,32,32,32,65,83,124,32,32,32,32,32,32,65,80,124,97,56,48,100,97,97,54,101,124,98,57,51,97,48,52,100,102,124,48,48,48,124,48,48,48,124,32,32,32,32,32,32,49,48,124,32,32,32,32,50,50,54,53,124,32,32,32,32,32,32,49,50,124,32,32,32,32,55,49,57,52,124,32,32,32,32,48,124],"message":null,"rawMessage":"2016-06-22
> 17:35:51.888|2016-06-22 17:37:09.713| 77.825| 0.173| 6|
> 192.168.138.158|49206| 95.163.121.204| 80|
> S| APR| AS| AP|a80daa6e|b93a04df|000|000| 10| 2265|
> 12| 7194| 0|","source.type":"yaf_error"}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
