Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/334#discussion_r86190240
--- Diff:
metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/GetProfile.java
---
@@ -226,14 +262,23 @@ private ColumnBuilder getColumnBuilder(Map<String,
Object> global) {
* @param global The global configuration.
*/
private RowKeyBuilder getRowKeyBuilder(Map<String, Object> global) {
- /*
- * WARNING: the row key builder is not currently configurable. by
invoking
- * the default constructor below, this defaults to generating keys
using a
- * period duration of 15 minutes. this function will NOT be able to
read
- * profiles created by a profiler running with any other period
duration, but
- * 15 minutes.
- */
- return new SaltyRowKeyBuilder();
+
+ // how long is the profile period?
+ String configuredDuration = (String)
global.getOrDefault(PROFILER_PERIOD, PROFILER_PERIOD_DEFAULT);
+ long duration = Long.parseLong(configuredDuration);
+ LOG.debug("profiler client: {}={}", PROFILER_PERIOD, duration);
+
+ // which units are used to define the profile period?
+ String configuredUnits = (String)
global.getOrDefault(PROFILER_PERIOD_UNITS, PROFILER_PERIOD_UNITS_DEFAULT);
+ TimeUnit units = TimeUnit.valueOf(configuredUnits);
+ LOG.debug("profiler client: {}={}", PROFILER_PERIOD_UNITS, units);
+
+ // what is the salt divisor?
+ String configuredSaltDivisor = (String)
global.getOrDefault(PROFILER_SALT_DIVISOR, PROFILER_SALT_DIVISOR_DEFAULT);
+ int saltDivisor = Integer.parseInt(configuredSaltDivisor);
+ LOG.debug("profiler client: {}={}", PROFILER_SALT_DIVISOR,
saltDivisor);
+
+ return new SaltyRowKeyBuilder(saltDivisor, duration, units);
--- End diff --
I had thought about the profiler having that limitation, I'm glad you
opened the jira, and I definitely agree that can be deferred. Regarding the
client, though, I'm thinking about this scenario:
(a) There are one or two clients doing on-going routine analysis of
up-to-date profiles, perhaps they are updaters for ML modules;
(b) Suddenly a security event requires analyzing some historical profile
data from 2 months ago, that was done with different profiles -- different
table, different period, something.
We shouldn't need to turn off (a) in order to do (b).
I won't insist on this, because certainly what you've already done is a
great improvement. But I'd still like to suggest that a small addition to the
mods you've made in GetProfile would give this flexibility. Do you mind if I
just whomp it up? Then you can look at it, and if you still don't like it,
I'll shut up :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
