Gabor Somogyi created SPARK-26845:
-------------------------------------
Summary: Avro from_avro to_avro roundtrip fails if data type is
string
Key: SPARK-26845
URL: https://issues.apache.org/jira/browse/SPARK-26845
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.4.0, 3.0.0
Reporter: Gabor Somogyi
I was playing with AvroFunctionsSuite and creates a situation where test fails
which I believe it shouldn't:
{code:java}
test("roundtrip in to_avro and from_avro - string") {
val df = spark.createDataset(Seq("1", "2",
"3")).select('value.cast("string").as("str"))
val avroDF = df.select(to_avro('str).as("b"))
val avroTypeStr = s"""
|{
| "type": "string",
| "name": "str"
|}
""".stripMargin
checkAnswer(avroDF.select(from_avro('b, avroTypeStr)), df)
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
