[
https://issues.apache.org/jira/browse/METRON-283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15838012#comment-15838012
]
ASF GitHub Bot commented on METRON-283:
---------------------------------------
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/421#discussion_r97813018
--- Diff:
metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/bolt/GenericEnrichmentBolt.java
---
@@ -149,9 +154,10 @@ public JSONObject load(CacheKey key) throws Exception {
cache = CacheBuilder.newBuilder().maximumSize(maxCacheSize)
.expireAfterWrite(maxTimeRetain, TimeUnit.MINUTES)
.build(loader);
+
GeoLiteDatabase.INSTANCE.update((String)getConfigurations().getGlobalConfig().get(GeoLiteDatabase.GEO_HDFS_FILE));
boolean success = adapter.initializeAdapter();
--- End diff --
Does this mean that we will attempt to load the geo database into memory in
every `GenericEnrichmentBolt`; even ones not doing geo-enrichment? Maybe that
is one reason, we have singleton for the GeoLiteDatabase; to avoid repeated
initialization?
Along the same lines, If I choose to not do geo-enrichment, do I still need
to have a valid Maxmind file in HDFS? Or would that cause all
GenericEnrichmentBolts to fail to initiailize?
Would it make sense to update the EnrichmentAdapter interface to accept
configuration values, so that this initialization occurs in the GeoAdapter
where it makes more sense? Then only the bolts doing geo-enrichment attempt to
initialize the geo data?
> Migrate Geo Enrichment outside of MySQL
> ---------------------------------------
>
> Key: METRON-283
> URL: https://issues.apache.org/jira/browse/METRON-283
> Project: Metron
> Issue Type: Improvement
> Reporter: James Sirota
> Assignee: Justin Leet
> Priority: Minor
>
> We need to migrate our enrichment SQL store from MySQL to Phoenix or some
> other SQL on Hbase library. Or alternatively come up with a way to do this
> without using SQL. This way we don't have a dependency on MySQL and there is
> one less thing that we need to install on our platform
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
