GitHub user arunmahadevan opened a pull request:
https://github.com/apache/spark/pull/21199
[SPARK-24127][SS] Continuous text socket source
## What changes were proposed in this pull request?
Support for text socket stream in spark structured streaming "continuous"
mode. This is roughly based on the idea of ContinuousMemoryStream where the
executor queries the data from driver over an RPC endpoint.
This makes it possible to create Structured streaming continuous pipeline
to ingest data via "nc" and run examples.
## How was this patch tested?
Ran spark examples in structured streaming continuous mode.
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/arunmahadevan/spark SPARK-24127
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21199.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21199
----
commit f7807fac2c39185893f78c7968eff8e9cafce991
Author: Arun Mahadevan <arunm@...>
Date: 2018-04-30T17:04:23Z
SPARK-24127: Continuous text socket source
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
