[
https://issues.apache.org/jira/browse/SPARK-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tathagata Das updated SPARK-1332:
---------------------------------
Priority: Blocker (was: Critical)
> Improve Spark Streaming's Network Receiver and InputDStream API for future
> stability
> ------------------------------------------------------------------------------------
>
> Key: SPARK-1332
> URL: https://issues.apache.org/jira/browse/SPARK-1332
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 0.9.0
> Reporter: Tathagata Das
> Assignee: Tathagata Das
> Priority: Blocker
>
> The current Network Receiver API makes it slightly complicated to right a new
> receiver as one needs to create an instance of BlockGenerator as shown in
> SocketReceiver
> https://github.com/apache/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala#L51
> Exposing the BlockGenerator interface has made it harder to improve the
> receiving process. The API of NetworkReceiver (which was not a very stable
> API anyways) needs to be change if we are to ensure future stability.
> Additionally, the functions like streamingContext.socketStream that create
> input streams, return DStream objects. That makes it hard to expose
> functionality (say, rate limits) unique to input dstreams. They should return
> InputDStream or NetworkInputDStream.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
