Eylon Ronen created SPARK-47661:
-----------------------------------
Summary: Using from_json with columnNameOfCorruptRecord and
ArrayType schema not working
Key: SPARK-47661
URL: https://issues.apache.org/jira/browse/SPARK-47661
Project: Spark
Issue Type: Bug
Components: Java API, SQL
Affects Versions: 3.5.1
Reporter: Eylon Ronen
When using schemas of type ArrayType using the columnNameOfCorruptRecord
parameter is not changing anything... even though the schema does contain the
field it is not populated during parsing
The cause is line 574 in JsonExpressions.scala and onward in running
FailureSafeParser where only when the schema is StructType this field is
actually handled and taken into account
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
