[
https://issues.apache.org/jira/browse/FLINK-1344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paris Carbone updated FLINK-1344:
---------------------------------
Description:
Source definitions in the scala-api pass a collector to the UDF, thus enforcing
an imperative style for defining custom streams. In order maintain a purely
functional coding style in the streaming scala-api while also adding some
interoperability with scala constructs (ie. Seq and Streams)
[UPDATE] Since there is already a fromCollection(Seq) function we can simply
add an implicit conversion from Seq[T] to DataStream[T].
(An additional idea would be for sinks could also support wrapping up flink
streams to scala streams for full interoperability with scala streaming code.)
was:
Source definitions in the scala-api pass a collector to the UDF, thus enforcing
an imperative style for defining custom streams. In order maintain a purely
functional coding style in the streaming scala-api while also adding some
interoperability with scala constructs it would be nice to add a source
definition that gets a generating lazily collection or a scala/akka stream and
just delegates internally the calls to the collector while iterating them. Mind
that source parallelism should be 1.
As an additional idea, sinks could also support wrapping up flink streams to
scala streams for full interoperability with scala streaming code.
> Add support for lazy collections and scala streams for functional source
> definitions
> ------------------------------------------------------------------------------------
>
> Key: FLINK-1344
> URL: https://issues.apache.org/jira/browse/FLINK-1344
> Project: Flink
> Issue Type: New Feature
> Components: Streaming
> Reporter: Paris Carbone
> Assignee: Paris Carbone
> Priority: Trivial
>
> Source definitions in the scala-api pass a collector to the UDF, thus
> enforcing an imperative style for defining custom streams. In order maintain
> a purely functional coding style in the streaming scala-api while also adding
> some interoperability with scala constructs (ie. Seq and Streams)
> [UPDATE] Since there is already a fromCollection(Seq) function we can simply
> add an implicit conversion from Seq[T] to DataStream[T].
> (An additional idea would be for sinks could also support wrapping up flink
> streams to scala streams for full interoperability with scala streaming code.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
