[
https://issues.apache.org/jira/browse/SPARK-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prashant Sharma closed SPARK-15842.
-----------------------------------
Resolution: Not A Problem
> Add support for socket stream.
> ------------------------------
>
> Key: SPARK-15842
> URL: https://issues.apache.org/jira/browse/SPARK-15842
> Project: Spark
> Issue Type: Sub-task
> Components: SQL, Streaming
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
>
> Streaming so far has offset based sources with all the available sources like
> file-source and memory-source that do not need additional capabilities to
> implement offset for any given range.
> Socket stream at OS level has a very tiny buffer. Many message queues have
> the ability to keep the message lingering until it is read by the receiver
> end. ZeroMQ is one such example. However in the case of socket stream, this
> is not supported.
> The challenge here would be to implement a way to buffer for a configurable
> amount of time and discuss strategies for overflow and underflow.
> This JIRA will form the basis for implementing sources which do not have
> native support for lingering a message for any amount of time until it is
> read. It deals with design doc if necessary and supporting code to implement
> such sources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
