[
https://issues.apache.org/jira/browse/METRON-529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15626337#comment-15626337
]
ASF GitHub Bot commented on METRON-529:
---------------------------------------
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/334#discussion_r85994108
--- Diff:
metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/GetProfile.java
---
@@ -226,14 +262,23 @@ private ColumnBuilder getColumnBuilder(Map<String,
Object> global) {
* @param global The global configuration.
*/
private RowKeyBuilder getRowKeyBuilder(Map<String, Object> global) {
- /*
- * WARNING: the row key builder is not currently configurable. by
invoking
- * the default constructor below, this defaults to generating keys
using a
- * period duration of 15 minutes. this function will NOT be able to
read
- * profiles created by a profiler running with any other period
duration, but
- * 15 minutes.
- */
- return new SaltyRowKeyBuilder();
+
+ // how long is the profile period?
+ String configuredDuration = (String)
global.getOrDefault(PROFILER_PERIOD, PROFILER_PERIOD_DEFAULT);
+ long duration = Long.parseLong(configuredDuration);
+ LOG.debug("profiler client: {}={}", PROFILER_PERIOD, duration);
+
+ // which units are used to define the profile period?
+ String configuredUnits = (String)
global.getOrDefault(PROFILER_PERIOD_UNITS, PROFILER_PERIOD_UNITS_DEFAULT);
+ TimeUnit units = TimeUnit.valueOf(configuredUnits);
+ LOG.debug("profiler client: {}={}", PROFILER_PERIOD_UNITS, units);
+
+ // what is the salt divisor?
+ String configuredSaltDivisor = (String)
global.getOrDefault(PROFILER_SALT_DIVISOR, PROFILER_SALT_DIVISOR_DEFAULT);
+ int saltDivisor = Integer.parseInt(configuredSaltDivisor);
+ LOG.debug("profiler client: {}={}", PROFILER_SALT_DIVISOR,
saltDivisor);
+
+ return new SaltyRowKeyBuilder(saltDivisor, duration, units);
--- End diff --
1. If GetProfile attempts to read a profile with the wrong window duration,
will the user get a reasonable error message? I was unable to find this in
GetProfile.java or GetProfileTest.java.
2. Unlike the profiler, it seems to me that there may be need for multiple
simultaneous clients for different users, some of which may be consuming
historical profile data that was taken with different window duration (and
other settings) than is currently in use at the time the client is run.
Therefore, I suggest that GetProfile should also take an optional override set
of config arguments (any or all of profiler.client.* , possibly excepting
table.provider) so that clients accessing different data won't stomp on each
other.
> Allow Configurable Periods for Profiler Client
> ----------------------------------------------
>
> Key: METRON-529
> URL: https://issues.apache.org/jira/browse/METRON-529
> Project: Metron
> Issue Type: Improvement
> Reporter: Nick Allen
> Assignee: Nick Allen
>
> By default, the Profiler creates Profiles with a period duration of 15
> minutes. This means that data is accumulated, summarized and flushed every 15
> minutes. The Client API must also have knowledge of this duration to
> correctly retrieve the profile data. If the client API is expecting 15 minute
> periods, it will not be able to read data generated by a Profiler that has
> been configured with a 1 hour period.
> The period duration can be configured in the Profiler by altering the
> Profiler topology's static properties file. The Stellar Client API currently
> provides no means to configure the period duration and defaults also to 15
> minutes. This means that the Stellar Client API can only read profiles with a
> period duration of 15 minutes. This is a known limitation that has not yet
> been addressed.
> https://github.com/apache/incubator-metron/blob/e31705029fd7518bd03c22eece0c2a8eb1187abf/metron-analytics/metron-profiler-client/README.md#period-duration
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
