Hello, Currently we are researching fast and resources efficient way to save enriched data in Hive for further Analytics.
There are two scenarios that we consider: a) Use Ozzie Java job that uses Metron enrichment classes to "manually" enrich each line of the source data that is picked up from the source dir (the one that we have developed already and using). That is something that we developed on our own. Downside: custom code that built on top of Metron source code. b) Use NiFi to listen for indexing Kafka topic -> split stream by source type -> Put every source type in corresponding Hive table. I wonder, if someone was going any of this direction and if there are best practices for this? Please advise. Thank you. - Dima
