Hi ,
I have submitted a JIRA for this issue.
The link is
https://issues.apache.org/jira/browse/SPARK-21147
thanks
Fei Shao
---Original---
From: "Michael Armbrust"<mich...@databricks.com>
Date: 2017/6/20 03:06:49
To: "??????????"<1427357...@qq.com>;
Cc: "user"<user@spark.apache.org>;"dev"<d...@spark.apache.org>;
Subject: Re: the scheme in stream reader
The socket source can't know how to parse your data. I think the right thing
would be for it to throw an exception saying that you can't set the schema
here. Would you mind opening a JIRA ticket?
If you are trying to parse data from something like JSON then you should use
from_json` on the value returned.
On Sun, Jun 18, 2017 at 12:27 AM, ?????????? <1427357...@qq.com> wrote:
Hi all,
L set the scheme for DataStreamReader but when I print the scheme.It just
printed:
root
|--value:string (nullable=true)
My code is
val line = ss.readStream.format("socket")
.option("ip",xxx)
.option("port",xxx)
.scheme(StructField("name",StringType??::(StructField("age", IntegerType))).load
line.printSchema
My spark version is 2.1.0.
I want the printSchema prints the schema I set in the code.How should I do
please?
And my original target is the received data from socket is handled as schema
directly.What should I do please?
thanks
Fei Shao
