Github user pwawrzyniak commented on the issue:
https://github.com/apache/incubator-samoa/pull/64
Thank you nicolas-kourtellis for your feedback!
Regarding your comments, I removed copyright notices from the code, fixed
and clened-up the code according to your comments.
Regarding AVRO code provided within SAMOA-65, the first difference between
avro loader and our kafka avro mapper is that avro loader works with json files
containing schema header and payload data (json or binary). In our case avro
mapper works with byte stream received from kafka and the schema is defined in
separate file. The other difference is that avro loader produces Instance
object while Kafka Mapper was designed to work with InstanceContentEvent.
Moreover kafka avro mapper serializes whole InstanceContentEvent object while
avro loader reads data file, creates avro structure and, based on that, builds
Instance object.It should be possible to use the concept from avro loader for
processing kafka but I suppose it would require implementation of new generic
loader, instead of using the old one. Other thing is that we need to have two
way serialization and avro loader is used only for reading data, not writing.
And regarding JSON parser, as of now it is prepared to parse messages
coming from Apache Kafka in "one-by-one" style, serializing
InstanceContentEvent class. It could be potenatially used to parse file (in
line-by-line manner for example), but I believe currently, as the mapper
accepts byte array as the input, it can be easily used to this task (i.e. as
the parser when reading data from text/json file).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
