Github user cestella commented on the issue:
https://github.com/apache/incubator-metron/pull/303
I figured out what was going on with this PR and it's kinda interesting, so
forgive the long-winded explanation.
@ottobackwards is indeed right, it is not apparent why his PR about a
FieldTransformation and Stellar Function would affect the `YafIntegrationTest`.
Unfortunately, it became apparent when it consistently failed on this PR in
both travis and locally that something was happening.
It turns out the problem is
[here](https://github.com/apache/incubator-metron/blob/master/metron-platform/metron-integration-test/src/main/config/zookeeper/global.json#L14).
The keen observer will notice in the global config we are ensuring that the
`src_ip_addr` and `dst_ip_addr` are valid IP addresses for the integration
tests. This, of course, is a typo (that I made :P ). The correct field names
are `ip_src_addr` and `ip_dst_addr`.
Now, you may ask:
* How has this ever worked?
* Given that it did work before, why is it broken now?
Well, for the first question, it was never correct in the integration
tests, but since we did not exercise the situation where we send in invalid IP
addresses. Furthermore, the default behavior for fields which do not exist is
to return `true` (probably wrong).
For the second question, @ottobackwards has changed the default behavior
for fields which do not resolve is to return `false` (almost assuredly right).
For clarity to show how I go about debugging these problems, I'll be rather
explicit in how I figured out what was happening:
* In my git branch, I added @ottobackwards 's repo as a remote repo and
checked out the branch:
* `git remote add otto
https://github.com/ottobackwards/incubator-metron.git`
* `git checkout otto/METRON-424`
* `git branch METRON-424-debug`
* `git checkout METRON-424-debug`
* In Intellij, I ran the YafIntegrationTest to verify that it continued to
fail
* I set breakpoints at the exit points of the `ParserBolt`:
* [Line
135](https://github.com/apache/incubator-metron/blob/4a4cb8b117dbb66bbfb4915bca9d871a06682c28/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java#L135)
* [Line
147](https://github.com/apache/incubator-metron/blob/4a4cb8b117dbb66bbfb4915bca9d871a06682c28/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java#L147)
* [Line
131](https://github.com/apache/incubator-metron/blob/4a4cb8b117dbb66bbfb4915bca9d871a06682c28/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java#L131)
* It became apparent that we were exiting because the message was not
valid. I then set breakpoints inside of @ottobackwards 's code and validated
that it was doing the right thing, but the fields were coming in as `null`,
which means the fields were not in the message.
* I tracked the `global.json` down and looked at the field names and there
the bug became apparent.
I submitted a PR to your PR branch, @ottobackwards, so if you merge that
in, the build *should* work (fingers crossed ;)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
