[
https://issues.apache.org/jira/browse/SPARK-24466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-24466.
----------------------------------
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 21497
[https://github.com/apache/spark/pull/21497]
> TextSocketMicroBatchReader no longer works with nc utility
> ----------------------------------------------------------
>
> Key: SPARK-24466
> URL: https://issues.apache.org/jira/browse/SPARK-24466
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
> Affects Versions: 2.4.0
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
> Priority: Major
> Fix For: 2.4.0
>
>
> While playing with Spark 2.4.0-SNAPSHOT, I found nc command exits before
> reading actual data so the query also exits with error.
>
> The reason is due to launching temporary reader for reading schema, and
> closing reader, and re-opening reader. While reliable socket server should be
> able to handle this without any issue, nc command normally can't handle
> multiple connections and simply exits when closing temporary reader.
>
> Given that socket source is expected to be used from examples on official
> document or some experiments, which we tend to simply use netcat, this is
> better to be treated as bug, though this is a kind of limitation on netcat.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
