[ https://issues.apache.org/jira/browse/SPARK-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tathagata Das updated SPARK-1332: --------------------------------- Priority: Blocker (was: Critical) > Improve Spark Streaming's Network Receiver and InputDStream API for future > stability > ------------------------------------------------------------------------------------ > > Key: SPARK-1332 > URL: https://issues.apache.org/jira/browse/SPARK-1332 > Project: Spark > Issue Type: Improvement > Components: Streaming > Affects Versions: 0.9.0 > Reporter: Tathagata Das > Assignee: Tathagata Das > Priority: Blocker > > The current Network Receiver API makes it slightly complicated to right a new > receiver as one needs to create an instance of BlockGenerator as shown in > SocketReceiver > https://github.com/apache/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala#L51 > Exposing the BlockGenerator interface has made it harder to improve the > receiving process. The API of NetworkReceiver (which was not a very stable > API anyways) needs to be change if we are to ensure future stability. > Additionally, the functions like streamingContext.socketStream that create > input streams, return DStream objects. That makes it hard to expose > functionality (say, rate limits) unique to input dstreams. They should return > InputDStream or NetworkInputDStream. -- This message was sent by Atlassian JIRA (v6.2#6252)