[
https://issues.apache.org/jira/browse/METRON-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876029#comment-15876029
]
ASF GitHub Bot commented on METRON-690:
---------------------------------------
GitHub user cestella reopened a pull request:
https://github.com/apache/incubator-metron/pull/450
METRON-690: Create a DSL-based timestamp lookup for profiler to enable
sparse windows
Creating a small DSL to allow specifying profiles from windows of time that
may:
* repeat non-contiguously (e.g. the same hour every day for a week)
* have inclusions and exclusions (e.g. the same hour every day for a week
excluding holidays and weekends)
This also provides a `PROFILE_WINDOW` Stellar function which accepts this
DSL as a string and can return the set of profiler periods based on the times
selected and suitable for passing to `PROFILE_GET`. To be clear, you can
compose the two functions.
i.e. `PROFILE_GET('profile', 'entity', PROFILE_WINDOW('1 hour window every
24 hours starting from 14 days ago including the current day of the week
excluding weekends, holidays:us'))` would retrieve all the profile measurements
written for the profile "profile" and entity "entity" for the last hour on the
same weekday excluding weekends and US holidays across the last 14 days
For a complete description with examples, see
[here](https://github.com/cestella/incubator-metron/tree/METRON-690-squashed/metron-analytics/metron-profiler-client#profile_window).
Acceptance testing plan will be submitted as a follow-on comment.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cestella/incubator-metron METRON-690-squashed
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/450.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #450
----
commit 04474a61c8f8cf85a491c6a693897a5cb0a6107f
Author: cstella <[email protected]>
Date: 2017-02-08T19:14:37Z
Create a DSL-based timestamp lookup for profiler to enable sparse windows
commit eb729a821a7b8ee80a1b033ad884633a4d174138
Author: cstella <[email protected]>
Date: 2017-02-09T22:46:57Z
Merge branch 'master' into METRON-690-squashed
commit 51381e70d66e01b10787779fcc10e69e39f05a83
Author: cstella <[email protected]>
Date: 2017-02-10T00:38:52Z
Removed jodatime dependency as unnecessary
commit 877ba3df06a8cfb1e1a2f6101d8db9932d664d21
Author: cstella <[email protected]>
Date: 2017-02-10T01:27:52Z
Updating docs and dependencies.
commit 8c36be979407f4bdfced1c49c351ba02b95a71d6
Author: cstella <[email protected]>
Date: 2017-02-10T02:18:04Z
Finished readme
commit 9c0bc91df672725486ad4be251103e3877aa88a1
Author: cstella <[email protected]>
Date: 2017-02-10T02:36:48Z
Fixed weird spacing on AST class.
commit b5cc6d454a6dcf2989611945f4f8ca2184fa73e8
Author: cstella <[email protected]>
Date: 2017-02-10T02:38:24Z
Missed a few.
commit c1398d5388f7420d197b2f7584f902a069c0970f
Author: cstella <[email protected]>
Date: 2017-02-10T14:08:45Z
Making exceptions throw properly
commit fe6c80360d14ab78bd687c3a1bc5ad5f65d9d187
Author: cstella <[email protected]>
Date: 2017-02-10T14:57:21Z
Moved from a stack to a deque, becuase @justinleet made fun ot of
offline...and it's the right thing to do.
commit 8d3d298cbec999d24a2928c2e5ce281ab09ea97f
Author: cstella <[email protected]>
Date: 2017-02-10T16:31:25Z
Updating test.
commit 51faaa2d74bb8f2924cda1c7ad25eee8518264c6
Author: cstella <[email protected]>
Date: 2017-02-17T18:03:22Z
updating.
commit 1fe525635fbe2db63f06cb41efbad7e43782b168
Author: cstella <[email protected]>
Date: 2017-02-17T18:05:02Z
Merge branch 'master' into METRON-690-squashed
commit 928d59e510db64017dd6afaabc9d8f968a412fe6
Author: cstella <[email protected]>
Date: 2017-02-17T18:35:49Z
Updating readme to react to james
----
> Create a DSL-based timestamp lookup for profiler to enable sparse windows
> -------------------------------------------------------------------------
>
> Key: METRON-690
> URL: https://issues.apache.org/jira/browse/METRON-690
> Project: Metron
> Issue Type: New Feature
> Reporter: Casey Stella
>
> I propose that we support the following features:
> * A starting point that is not current time
> * Sparse bins (i.e. the last hour for every tuesday for the last month)
> * The ability to skip events (e.g. weekends, holidays)
> This would result in a new function with the following arguments:
> from - The lookback starting point (default to now)
> fromUnits - The units for the lookback starting point
> to - The ending point for the lookback window (default to from + binSize)
> toUnits - The units for the lookback ending point
> including - A list of conditions which we would skip.
> weekend
> holiday
> sunday through saturday
> excluding - A list of conditions which we would skip.
> weekend
> holiday
> sunday through saturday
> binSize - The size of the lookback bin
> binUnits - The units of the lookback bin
> Given the number of arguments and their complexity and the fact that many,
> many are optional,
> PROFILE_LOOKBACK accept a string backed by a DSL to express these criteria
> Base Case: A lookback of 1 hour ago
> PROFILE_LOOKBACK( '1 hour bins from now')
> Example 1: The same time window every tuesday for the last month starting one
> hour ago
> Just to make this as clear as possible, if this is run at 3PM on Monday
> January 23rd, 2017, it would include the following bins:
> January 17th, 2PM - 3PM
> January 10th, 2PM - 3PM
> January 3rd, 2PM - 3PM
> December 27th, 2PM - 3PM
> PROFILE_LOOKBACK( '1 hour bins from 1 hour to 1 month including tuesdays')
> Example 2: The same time window every sunday for the last month starting one
> hour ago skipping holidays
> Just to make this as clear as possible, if this is run at 3PM on Monday
> January 22rd, 2017, it would include the following bins:
> January 16th, 2PM - 3PM
> January 9th, 2PM - 3PM
> January 2rd, 2PM - 3PM
> NOT December 25th
> PROFILE_LOOKBACK( '1 hour bins from 1 hour to 1 month including tuesdays
> excluding holidays')
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
