[
https://issues.apache.org/jira/browse/METRON-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15230162#comment-15230162
]
ASF GitHub Bot commented on METRON-93:
--------------------------------------
Github user cestella commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/66#discussion_r58862697
--- Diff: deployment/roles/metron_streaming/tasks/main.yml ---
@@ -109,8 +111,10 @@
- { regexp: "bolt.hbase.table.name=", line: "bolt.hbase.table.name={{
pcap_hbase_table }}" }
- { regexp: "threat.intel.tracker.table=", line:
"threat.intel.tracker.table={{ tracker_hbase_table }}" }
- { regexp: "threat.intel.tracker.cf=", line:
"threat.intel.tracker.cf=t" }
- - { regexp: "threat.intel.ip.table=", line: "threat.intel.ip.table={{
threatintel_ip_hbase_table }}" }
- - { regexp: "threat.intel.ip.cf=", line: "threat.intel.ip.cf=t" }
+ - { regexp: "threat.intel.simple.hbase.table=", line:
"threat.intel.simple.hbase.table={{ threatintel_hbase_table }}" }
+ - { regexp: "threat.intel.simple.hbase.cf=", line:
"threat.intel.simple.hbase.cf=t" }
+ - { regexp: "enrichment.simple.hbase.table=", line:
"enrichment.simple.hbase.table={{ enrichment_hbase_table }}" }
+ - { regexp: "enrichment.simple.hbase.cf=", line:
"enrichment.simple.hbase.cf=t" }
- { regexp: "mysql.ip=", line: "mysql.ip={{ groups.mysql[0] }}" }
- { regexp: "mysql.password=", line: "mysql.password={{
mysql_root_password }}" }
- { regexp: "index.hdfs.output=", line: "index.hdfs.output={{
metron_hdfs_output_dir }}/enrichment/indexed" }
--- End diff --
I think the problem is deeper than this. We need separate properties files
for the sensor topologies than we have for the enrichment topologies. As it
stands, we're using the same config for everything and that's just confusing.
I'm creating https://issues.apache.org/jira/browse/METRON-99 to capture it.
> Generalize the HBase threat intel infrastructure to support enrichments
> -----------------------------------------------------------------------
>
> Key: METRON-93
> URL: https://issues.apache.org/jira/browse/METRON-93
> Project: Metron
> Issue Type: Improvement
> Reporter: Casey Stella
> Assignee: Casey Stella
> Original Estimate: 504h
> Remaining Estimate: 504h
>
> As it stands, the threat intel infrastructure is awkward. Namely, different
> threat intelligence sources must be pushed into separate hbase tables
> (malicious_ips separate form malicious_hosts, for instance). We'd rather
> have one table where the type is brought into the rowkey. Since this
> infrastructure is generalized, also add a simple hbase enrichment adapter.
> Furthermore, the configuration for a new enrichment should be added to
> zookeeper as part of the data load.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
