Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22037#discussion_r209083275
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/SchemaConverters.scala
---
@@ -139,7 +152,22 @@ object SchemaConverters {
case FloatType => builder.floatType()
case DoubleType => builder.doubleType()
- case _: DecimalType | StringType => builder.stringType()
+ case StringType => builder.stringType()
+ case d: DecimalType =>
+ val avroType = LogicalTypes.decimal(d.precision, d.scale)
+ val fixedSize = minBytesForPrecision(d.precision)
+ // Use random name to avoid conflict in naming of fixed field.
+ // Field names must start with [A-Za-z_], while the charset of
Random.alphanumeric contains
+ // [0-9]. So add a single character "f" to ensure the name is
valid.
+ val name = "f" + Random.alphanumeric.take(32).mkString("")
+ if (nullable) {
+ val schema = avroType.addToSchema(
+ SchemaBuilder.builder().fixed(name).size(fixedSize))
+ builder.`type`(schema)
+ } else {
+ avroType.addToSchema(builder.fixed(name).size(fixedSize))
--- End diff --
Why we don't need call `type` as above?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
