Github user cestella commented on the issue:
https://github.com/apache/metron/pull/585
@mmiklavc it depends on which test case you're talking about. We have two
modes of operation in the pcap topology and 2 test cases in the integration
test and these are defined by the flux property `kafka.pcap.ts_scheme`. These
modes define the deserialization logic used in the topology to convert kafka
key/values to bytes suitable for writing to HDFS:
* `FROM_PACKET`: which expects a fully-formed packet (with headers) and
parses the packet and extracts the timestamp from the value. This is a legacy
mode, which functioned with pycapa prior to rewriting. We should eventually
deprecate this and remove it. This is associated with the
`FromPacketDeserializer`
* `FROM_KEY` : which expects raw data and a timestamp from the key. This
is by far the dominant mode of operation and the one you will see in `pycapa`
or `fastcapa`. This is associated with the `FromKeyDeserializer`
It appears that you are doing the null check in the `HDFSWriterCallback`.
I would recommend doing this null check in `FromKeyDeserializer` as a null key
is not an illegal state for the `FromPacketDeserializer`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
