Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/334#discussion_r86137602
--- Diff:
metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/GetProfile.java
---
@@ -226,14 +262,23 @@ private ColumnBuilder getColumnBuilder(Map<String,
Object> global) {
* @param global The global configuration.
*/
private RowKeyBuilder getRowKeyBuilder(Map<String, Object> global) {
- /*
- * WARNING: the row key builder is not currently configurable. by
invoking
- * the default constructor below, this defaults to generating keys
using a
- * period duration of 15 minutes. this function will NOT be able to
read
- * profiles created by a profiler running with any other period
duration, but
- * 15 minutes.
- */
- return new SaltyRowKeyBuilder();
+
+ // how long is the profile period?
+ String configuredDuration = (String)
global.getOrDefault(PROFILER_PERIOD, PROFILER_PERIOD_DEFAULT);
+ long duration = Long.parseLong(configuredDuration);
+ LOG.debug("profiler client: {}={}", PROFILER_PERIOD, duration);
+
+ // which units are used to define the profile period?
+ String configuredUnits = (String)
global.getOrDefault(PROFILER_PERIOD_UNITS, PROFILER_PERIOD_UNITS_DEFAULT);
+ TimeUnit units = TimeUnit.valueOf(configuredUnits);
+ LOG.debug("profiler client: {}={}", PROFILER_PERIOD_UNITS, units);
+
+ // what is the salt divisor?
+ String configuredSaltDivisor = (String)
global.getOrDefault(PROFILER_SALT_DIVISOR, PROFILER_SALT_DIVISOR_DEFAULT);
+ int saltDivisor = Integer.parseInt(configuredSaltDivisor);
+ LOG.debug("profiler client: {}={}", PROFILER_SALT_DIVISOR,
saltDivisor);
+
+ return new SaltyRowKeyBuilder(saltDivisor, duration, units);
--- End diff --
Right now the entire Profiler runs with one period duration. If the
Profiler is generating multiple profiles, they will each have the same profile
period. I don't foresee multiple clients trying to read profiles with
different periods at the same time. At least, not right now.
In the future though, I think it would be very useful to enhance the
Profiler so that each profile can define its own period; 15 minutes, 4 hours,
etc. The period that you want to use is going to depend on the data itself and
how you want to use the data. I just opened
[METRON-530](https://issues.apache.org/jira/browse/METRON-530) to track this.
When we complete METRON-530, then I think we definitely need to make this
improvement that you are suggesting. A Profiler Client will need to be able to
read from many different profiles with unique durations. But at the same time,
we will need [METRON-450](https://issues.apache.org/jira/browse/METRON-450),
and probably some other enhancements.
My gut tells me that we could defer this enhancement until we prioritize
METRON-530. What do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
