Dima, Here are the CSV, extractor and enrichment configs that I uploaded via flatfile_loader command.
Regards, Tyler Regards, Tyler Moore Software Engineer Flyball Labs On Sat, Oct 29, 2016 at 3:19 PM, Dima Kovalyov <[email protected]> wrote: > Hello Tyler, > > Can you send csv sample, extractor and enrichment/threatintel config and > flatfile_loader command that you using to load your enrichment/threatintel? > > - Dima > > On 10/29/2016 07:54 PM, Tyler Moore wrote: > Hey everyone, > > I was having some trouble with creating a custom enrichment configuration > for my bro sensor and hopefully someone can clue me in on what i'm missing. > > So basically I created a custom hostname enrichment config and an > extractor config file that I pushed to zookeeper that extract and data from > a csv file i pushed into my enrichment table in Hbase and maps this > enrichment to the ip_src_addr and ip_dst_addr to see if they match. If one > of the fields matches the "ip" key from hbase the "host" value should be > added via the "hostname" enrichment mapping. > > The problem is that the enrichment isn't being written. I attached > screenshots of the bolts and there stats after I pushed some data in via > tcpreplay. > Also this is what my bro.json file looks like : > > { > "index" : "bro", > "batchSize" : 5, > "enrichment" : { > "fieldMap" : { > "geo" : [ "ip_dst_addr", "ip_src_addr" ], > "host" : [ "ip_src_addr", "ip_dst_addr" ], > "hbaseEnrichment" : [ "ip_src_addr", "ip_dst_addr" ] > }, > "fieldToTypeMap" : { > "ip_dst_addr" : [ "hostname" ], > "ip_src_addr" : [ "hostname" ] > }, > "config" : { } > }, > "threatIntel" : { > "fieldMap" : { > "hbaseThreatIntel" : [ "ip_src_addr", "ip_dst_addr" ] > }, > "fieldToTypeMap" : { > "ip_src_addr" : [ "malicious_ip" ], > "ip_dst_addr" : [ "malicious_ip" ] > }, > "config" : { }, > "triageConfig" : { > "riskLevelRules" : { }, > "aggregator" : "MAX", > "aggregationConfig" : { } > } > }, > "configuration" : { } > } > > The built in host enrichment works fine and is able to enrich via info in > enrichment.host.known_hosts but we will be adding a custom parser to this > to stream host data. > If you have any ideas please let me know! > > Regards, > > Tyler Moore > Software Engineer > Flyball Labs > >
hostname_extractor_config.json
Description: application/json
hostname_enrichment_config.json
Description: application/json
0.0.0.0, "IGMP" 10.113.145.135, "GLAZER" 10.113.145.137, "GLAZER" 10.113.145.138, "GLAZER" 10.113.145.140, "GLAZER" 10.113.145.142, "GLAZER" 10.113.145.143, "GLAZER" 10.113.145.144, "GLAZER" 10.113.145.147, "GLAZER" 10.113.145.149, "GLAZER" 10.113.145.159, "GLAZER" 10.113.145.161, "GLAZER" 10.113.145.166, "GLAZER" 10.113.145.168, "GLAZER" 10.113.145.197, "GLAZER" 10.113.145.2, "GLAZER" 10.113.145.20, "GLAZER" 10.113.145.201, "GLAZER" 10.113.145.202, "GLAZER" 10.113.145.21, "GLAZER" 10.113.145.23, "GLAZER" 10.113.145.26, "GLAZER" 10.113.145.27, "GLAZER" 10.113.145.29, "GLAZER" 10.113.145.30, "GLAZER" 10.113.145.31, "GLAZER" 10.113.145.33, "GLAZER" 10.113.145.34, "GLAZER" 10.113.145.36, "GLAZER" 10.113.145.50, "GLAZER" 10.113.145.57, "GLAZER" 10.113.145.62, "GLAZER" 10.113.145.63, "GLAZER" 10.113.145.64, "GLAZER" 10.113.145.68, "GLAZER" 10.113.145.69, "GLAZER" 10.113.145.70, "GLAZER" 10.113.145.71, "GLAZER" 10.113.145.74, "GLAZER" 10.113.145.75, "GLAZER" 10.113.145.80, "GLAZER" 10.113.145.81, "GLAZER" 10.113.145.82, "GLAZER" 10.113.145.83, "GLAZER" 10.113.145.84, "GLAZER" 10.113.145.85, "GLAZER" 10.113.145.87, "GLAZER" 10.113.145.88, "GLAZER" 10.113.145.90, "GLAZER" 10.113.145.91, "GLAZER" 10.113.145.92, "GLAZER" 10.113.145.93, "GLAZER" 10.113.145.94, "GLAZER" 10.113.145.96, "GLAZER"
