[
https://issues.apache.org/jira/browse/BEAM-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862897#comment-15862897
]
ASF GitHub Bot commented on BEAM-920:
-------------------------------------
GitHub user amitsela opened a pull request:
https://github.com/apache/beam/pull/1987
[BEAM-920] Add support for Watermarks in the Spark runner.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.txt).
---
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/amitsela/beam watermarks
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/1987.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1987
----
commit 33288a7c2a30f65886cdf93007f202dfcbdfb842
Author: Sela <[email protected]>
Date: 2017-02-12T16:28:19Z
A global Watermark holder to update and broadcast to workers.
commit a44c5de1a043d3aa14fc187b5621efad683d7015
Author: Sela <[email protected]>
Date: 2017-02-12T16:30:47Z
Ingest the input watermarks into the GlobalWatermarkHolder.
commit 656e2fb0ad35e11661a14a9dc2db489abf1880f3
Author: Sela <[email protected]>
Date: 2017-02-12T16:31:14Z
Advance watermarks onBatchCompleted hook.
commit 8300b3f363dbbffabeb7d7148198ae146a59bca4
Author: Sela <[email protected]>
Date: 2017-02-12T16:32:06Z
Handle QueuedStream (for testing) and track sources upstream.
commit 8aa0268dbd607037407aaae30e8968872c3e5146
Author: Sela <[email protected]>
Date: 2017-02-12T16:33:16Z
Refactor according to changes.
commit 0d1b1ce202c48763153cfab4147e28f48e4c28e1
Author: Sela <[email protected]>
Date: 2017-02-12T16:33:55Z
Exopse RegexMatcher as part of the SDK.
commit 4c8fec02f8b7181fcc192268ec48907dadca3c40
Author: Sela <[email protected]>
Date: 2017-02-12T16:34:30Z
Watermark tests.
commit 1150eb517c35352a62611444ca24030e3bdd9e7c
Author: Sela <[email protected]>
Date: 2017-02-12T17:23:28Z
Streaming sources tracking test.
commit 3d3f19566e98c684e68f34afffc6c01904852b0a
Author: Sela <[email protected]>
Date: 2017-02-12T17:47:00Z
This relied on a wrong functionality as descrined in BEAM-1444 and should
be revisited there.
commit e3593b5cc1d4004b49b2f5d44ca07c6ec63806b3
Author: Sela <[email protected]>
Date: 2017-02-12T18:08:40Z
fixup! licenese.
----
> Support triggers, panes and watermarks.
> ---------------------------------------
>
> Key: BEAM-920
> URL: https://issues.apache.org/jira/browse/BEAM-920
> Project: Beam
> Issue Type: New Feature
> Components: runner-spark
> Reporter: Amit Sela
> Assignee: Amit Sela
>
> Implement event-time based aggregation using triggers, panes and watermarks.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
