[
https://issues.apache.org/jira/browse/FLINK-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14641755#comment-14641755
]
ASF GitHub Bot commented on FLINK-2405:
---------------------------------------
GitHub user gyfora opened a pull request:
https://github.com/apache/flink/pull/936
[FLINK-2405] [streaming] Added stateful functions to Scala DataStreams
This PR introduces 3 new API methods for the Scala DataStreams that extend
the standard Map, FlatMap, and Filter function with state.
**These methods are:**
*for a DataStream[T]*
* **mapWithState**(fun: (T, Option[S]) => (R, Option[S]), stateByKey:
Boolean = false): DataStream[R]
* **flatMapWithState**( fun: (T, Option[S]) => (TraversableOnce[R],
Option[S]), stateByKey: Boolean = false): DataStream[R]
* **filterWithState**(fun: (T, Option[S]) => (Boolean, Option[S]),
stateByKey: Boolean = false): DataStream[T]
Each stateful transformation takes an optional boolean flag, stateByKey,
which determines whether the state of the function should be kept as local
state (stateByKey=false) or partitioned state (stateByKey = true). Please refer
to these concepts in
https://cwiki.apache.org/confluence/display/FLINK/Stateful+Stream+Processing.
I am open to suggestions whether we should create separate methods for
partitioned state use instead of the boolean flag. I personally prefer the
flags.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gyfora/flink master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/936.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #936
----
commit 462b864bff588f5d2c9dfc9c2de95b931a8805c2
Author: Gyula Fora <[email protected]>
Date: 2015-07-25T17:05:56Z
[FLINK-2405] [streaming] Added stateful functions to Scala DataStreams
----
> Add stateful transformations using lambdas to the Streaming Scala API
> ---------------------------------------------------------------------
>
> Key: FLINK-2405
> URL: https://issues.apache.org/jira/browse/FLINK-2405
> Project: Flink
> Issue Type: New Feature
> Components: Scala API, Streaming
> Reporter: Gyula Fora
> Priority: Minor
>
> I propose to extend the Streaming Scala API methods (map, flatmap, filter
> etc) with versions that take stateful functions as lambdas. This would allow
> for a nice functional way of defining stateful transformations:
> Example:
> def mapWithState( fun: (I, Option[S]) => (O, Option[S]), stateByKey: Boolean
> = false)
> The stateByKey optional flag allows the user to partition the state by key
> when applied on a KeyedDataStream.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
