Re: Enrichment (enrichments?) in Metron
Thank you, Nick! From: Nick Allen Reply-To: "user@metron.apache.org" Date: Friday, January 17, 2020 at 11:16 AM To: "user@metron.apache.org" Subject: Re: Enrichment (enrichments?) in Metron Hi Tom - You will want to look at using the Flatfile Loader [1] for loading enrichment data into HBase. This supports loading smaller data sets from a local file or much larger data sets using an MR job against data stored in HDFS. [1] https://github.com/nickwallen/metron/tree/master/metron-platform/metron-data-management#flatfile-loader On Fri, Jan 17, 2020 at 1:23 PM Yerex, Tom wrote: Good morning, I am exploring enrichments, currently looking at the Host Enrichment field in Ambari (Metron/Configs/Enrichment tab). In EnrichmentIntegrationTest.java, the test seems to connect to an HBase table “enrichments”. In HBase there is a table called “enrichment” (minus the “s”), and the values displayed in Ambari do not appear in the HBase table. I believe the “enrichment” table I see on the tutorial at hxxps://cwiki.apache.org/confluence/display/METRON/2016/06/16/Metron+Tutorial+-+Fundamentals+Part+6%3A+Streaming+Enrichment. Are these two the same table, and is there a typo? Does the Ambari Host Enrichment field (Metron/Config/Enrichments tab), store data in HBase? I would like to populate HBase with a lot of data about our local network and use it in Metron. The data is large enough that a copy/paste exercise into the field I see in Ambari is not practical, but importing the data directly into HBase makes sense as long as the field I see in Ambari would not overwrite the data and I put the data into the correct table. Happy Friday and thank you, --Tom. smime.p7s Description: S/MIME cryptographic signature
Re: Enrichment (enrichments?) in Metron
Hi Tom - You will want to look at using the Flatfile Loader <https://github.com/nickwallen/metron/tree/master/metron-platform/metron-data-management#flatfile-loader> [1] for loading enrichment data into HBase. This supports loading smaller data sets from a local file or much larger data sets using an MR job against data stored in HDFS. [1] https://github.com/nickwallen/metron/tree/master/metron-platform/metron-data-management#flatfile-loader On Fri, Jan 17, 2020 at 1:23 PM Yerex, Tom wrote: > Good morning, > > > > I am exploring enrichments, currently looking at the Host Enrichment field > in Ambari (Metron/Configs/Enrichment tab). In > EnrichmentIntegrationTest.java, the test seems to connect to an HBase table > “enrichments”. In HBase there is a table called “enrichment” (minus the > “s”), and the values displayed in Ambari do not appear in the HBase table. > > > > I believe the “enrichment” table I see on the tutorial at hxxps:// > cwiki.apache.org/confluence/display/METRON/2016/06/16/Metron+Tutorial+-+Fundamentals+Part+6%3A+Streaming+Enrichment > . > > > >- Are these two the same table, and is there a typo? >- Does the Ambari Host Enrichment field (Metron/Config/Enrichments >tab), store data in HBase? > > > > I would like to populate HBase with a lot of data about our local network > and use it in Metron. The data is large enough that a copy/paste exercise > into the field I see in Ambari is not practical, but importing the data > directly into HBase makes sense as long as the field I see in Ambari would > not overwrite the data and I put the data into the correct table. > > > > Happy Friday and thank you, > > > > --Tom. > > > > > > >
