[ https://issues.apache.org/jira/browse/SPARK-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma closed SPARK-15842. ----------------------------------- Resolution: Not A Problem > Add support for socket stream. > ------------------------------ > > Key: SPARK-15842 > URL: https://issues.apache.org/jira/browse/SPARK-15842 > Project: Spark > Issue Type: Sub-task > Components: SQL, Streaming > Reporter: Prashant Sharma > Assignee: Prashant Sharma > > Streaming so far has offset based sources with all the available sources like > file-source and memory-source that do not need additional capabilities to > implement offset for any given range. > Socket stream at OS level has a very tiny buffer. Many message queues have > the ability to keep the message lingering until it is read by the receiver > end. ZeroMQ is one such example. However in the case of socket stream, this > is not supported. > The challenge here would be to implement a way to buffer for a configurable > amount of time and discuss strategies for overflow and underflow. > This JIRA will form the basis for implementing sources which do not have > native support for lingering a message for any amount of time until it is > read. It deals with design doc if necessary and supporting code to implement > such sources. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org