[
https://issues.apache.org/jira/browse/SPARK-33865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17252612#comment-17252612
]
Apache Spark commented on SPARK-33865:
--------------------------------------
User 'AngersZhuuuu' has created a pull request for this issue:
https://github.com/apache/spark/pull/30869
> When HiveDDL, we need check avro schema too like parquet & orc
> --------------------------------------------------------------
>
> Key: SPARK-33865
> URL: https://issues.apache.org/jira/browse/SPARK-33865
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: angerszhu
> Priority: Major
>
> {code:java}
> Caused by: org.apache.avro.SchemaParseException: Illegal initial character:
> (IF((1 = 1), 1, 0))
> at org.apache.avro.Schema.validateName(Schema.java:1147)
> at org.apache.avro.Schema.access$200(Schema.java:81)
> at org.apache.avro.Schema$Field.<init>(Schema.java:403)
> at org.apache.avro.Schema$Field.<init>(Schema.java:396)
> at
> org.apache.hadoop.hive.serde2.avro.TypeInfoToSchema.createAvroField(TypeInfoToSchema.java:76)
> at
> org.apache.hadoop.hive.serde2.avro.TypeInfoToSchema.convert(TypeInfoToSchema.java:61)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerDe.getSchemaFromCols(AvroSerDe.java:170)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:114)
> at
> org.apache.spark.sql.hive.execution.HiveOutputWriter.<init>(HiveFileFormat.scala:121)
> at
> org.apache.spark.sql.hive.execution.HiveFileFormat$$anon$1.newInstance(HiveFileFormat.scala:104)
> at
> org.apache.spark.sql.execution.datasources.SingleDirectoryDataWriter.newOutputWriter(FileFormatDataWriter.scala:126)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
