Github user ottobackwards commented on the issue:
https://github.com/apache/incubator-metron/pull/303
That is awesome. Thank you.
I have been looking at the bro test and getting the failures, but I
havenât
gotten hits inside the IP Validator, so iâm a few steps behind.
I will take care of your pull request right way.
The good news, is in true âlearning by carnageâ fashion, I know know a
bit
about the integration tests ;)
On October 13, 2016 at 09:43:23, Casey Stella ([email protected])
wrote:
I figured out what was going on with this PR and it's kinda interesting, so
forgive the long-winded explanation.
@ottobackwards <https://github.com/ottobackwards> is indeed right, it is
not apparent why his PR about a FieldTransformation and Stellar Function
would affect the YafIntegrationTest. Unfortunately, it became apparent when
it consistently failed on this PR in both travis and locally that something
was happening.
It turns out the problem is here
<https://github.com/apache/incubator-metron/blob/master/metron-platform/metron-integration-test/src/main/config/zookeeper/global.json#L14>.
The keen observer will notice in the global config we are ensuring that the
src_ip_addr and dst_ip_addr are valid IP addresses for the integration
tests. This, of course, is a typo (that I made :P ). The correct field
names are ip_src_addr and ip_dst_addr.
Now, you may ask:
- How has this ever worked?
- Given that it did work before, why is it broken now?
Well, for the first question, it was never correct in the integration
tests, but since we did not exercise the situation where we send in invalid
IP addresses. Furthermore, the default behavior for fields which do not
exist is to return true (probably wrong).
For the second question, @ottobackwards <https://github.com/ottobackwards>
has changed the default behavior for fields which do not resolve is to
return false (almost assuredly right).
For clarity to show how I go about debugging these problems, I'll be rather
explicit in how I figured out what was happening:
- In my git branch, I added @ottobackwards
<https://github.com/ottobackwards> 's repo as a remote repo and checked
out the branch:
- git remote add otto
https://github.com/ottobackwards/incubator-metron.git
- git checkout otto/METRON-424
- git branch METRON-424-debug
- git checkout METRON-424-debug
- In Intellij, I ran the YafIntegrationTest to verify that it continued
to fail
- I set breakpoints at the exit points of the ParserBolt:
- Line 135
<https://github.com/apache/incubator-metron/blob/4a4cb8b117dbb66bbfb4915bca9d871a06682c28/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java#L135>
- Line 147
<https://github.com/apache/incubator-metron/blob/4a4cb8b117dbb66bbfb4915bca9d871a06682c28/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java#L147>
- Line 131
<https://github.com/apache/incubator-metron/blob/4a4cb8b117dbb66bbfb4915bca9d871a06682c28/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java#L131>
- It became apparent that we were exiting because the message was not
valid. I then set breakpoints inside of @ottobackwards
<https://github.com/ottobackwards> 's code and validated that it was
doing the right thing, but the fields were coming in as null, which
means the fields were not in the message.
- I tracked the global.json down and looked at the field names and there
the bug became apparent.
I submitted a PR to your PR branch, @ottobackwards
<https://github.com/ottobackwards>, so if you merge that in, the build
*should* work (fingers crossed ;)
â
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<https://github.com/apache/incubator-metron/pull/303#issuecomment-253516607>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABD1_wxd18U5OE1jrhDTJjPWRGPJZJqTks5qzjV6gaJpZM4KTvIX>
.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
