Github user arunmahadevan commented on a diff in the pull request:
https://github.com/apache/storm/pull/2090#discussion_r118845346
--- Diff:
storm-client/src/jvm/org/apache/storm/windowing/TimeTriggerPolicy.java ---
@@ -62,7 +62,9 @@ public void reset() {
@Override
public void start() {
- executorFuture = executor.scheduleAtFixedRate(newTriggerTask(),
duration, duration, TimeUnit.MILLISECONDS);
+ // initial delay is slightly less than the duration so that the
initial tuples wont't expire due to time drift
+ long initialDelay = duration - Math.min((long) (duration * .1),
10);
+ executorFuture = executor.scheduleAtFixedRate(newTriggerTask(),
initialDelay, duration, TimeUnit.MILLISECONDS);
--- End diff --
@srdo, I removed the initial delay and changed the logic so that the
initial events are processed in the first window. Subsequent windows will be
adjusted so that even if the trigger is slightly off, the events are
considered.
The processing time windows collects the events into windows based on
system timestamp so the user may not see all the events. However the processed
events are still guaranteed at-least once based on storm's acking mechanisms.
With the proposed patch we do a best effort to slightly adjust the window
sizes if the triggers are off and log warnings if it exceeds a threshold.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
