GitHub user aljoscha opened a pull request:
https://github.com/apache/flink/pull/1215
Stream API Refactoring
This is a WIP of the refactoring. I still want to add Javadocs and a new
join operator based on tagged union that uses the new windowing operators.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aljoscha/flink api-rework
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1215.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1215
----
commit 560090348f8df93160501a55051a913625c7215e
Author: Aljoscha Krettek <[email protected]>
Date: 2015-09-30T13:05:13Z
[FLINK-2666] Add timestamp extraction operator
This adds a user function TimestampExtractor and an operator
ExtractTimestampsOperator that can be used to extract timestamps and
attach them to elements to do event-time windowing.
Users can either use an AscendingTimestampExtractor that assumes that
timestamps are monotonically increasing. (This allows it to derive the
watermark very easily.) Or they use a TimestampExtractor, where they
also have to provide the watermark.
The ExtractTimestampOperator periodically (on the auto watermark
interval) calls the extractor to get the current watermark and forwards
it.
This also adds an ITCase for this behaviour.
commit 5b843231fdc5dee8d4ceada02f3ff8c41daa0281
Author: Aljoscha Krettek <[email protected]>
Date: 2015-10-01T13:58:52Z
Simplify Stream Java API Class Names
KeyedDataStream -> KeyedStream
KeyedWindowDataStream -> WindowedStream
NonParallelWindowDataStream -> AllWindowedStream
KeyedWindowFunction -> WindowFunction
WindowFunction -> AllWindowFunction
(along with rich functions and reduce function wrappers)
WindowedStream.mapWindow -> WindowedStream.apply
AllWindowedStream.mapWindow -> AllWindowedStream.apply
Also renamed the tests to match the new names.
commit 187ed701f548666a5daaf244ee69d43032c39c6f
Author: Aljoscha Krettek <[email protected]>
Date: 2015-10-01T15:07:11Z
Rename ConnectedDataStream to ConnectedStreams, Remove some operations
The removed operations are tricky and some of them are not working
correctly. For now, co-reduce, stream-cross and stream-join are
removed.
I'm planning to add a new join implementation based on tagged union
that uses the new windowing code.
commit 7892f321d0b0900a4331c7ee307a34778a8476c7
Author: Aljoscha Krettek <[email protected]>
Date: 2015-10-01T15:56:13Z
Remove groupBy and GroupedDataStream
Their functionality is subsumed by keyBy and KeyedStream
commit fd729f616c4386ddc72ecc817ea166df7a8f76aa
Author: Aljoscha Krettek <[email protected]>
Date: 2015-10-01T19:23:56Z
Add Scala API for new Windowing
This adds window/timeWindow to KeyedStream along with
windowAll/timeWindowAll
on DataStream.
The added API classes are AllWindowedStream and WindowedStream.
This also adds Translations tests similar to those for the Java API:
- AllWindowTranslationTest.scala
- WindowTranslationTest.scala
commit fcaa0fef2700730b768364861fc10e6bab628f47
Author: Aljoscha Krettek <[email protected]>
Date: 2015-10-02T14:48:13Z
WIP on Javadoc
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
