Marwen94 commented on issue #11492: URL: https://github.com/apache/hudi/issues/11492#issuecomment-2623041174
Hello @ad1happy2go , I'm having the same issue with AWS Glue 5.0. runtime versions (https://docs.aws.amazon.com/fr_fr/glue/latest/dg/release-notes.html): > Spark 3.5.2 > Python 3.11 > Scala 2.12.18 > Hudi 0.15.0 Full trace : ``` Py4JJavaError( py4j.protocol.Py4JJavaError: An error occurred while calling o263.save. : org.apache.hudi.internal.schema.HoodieSchemaException: Failed to convert struct type to avro schema: StructType(StructField(actor_created_at,TimestampType,true),StructField(actor_event_id,StringType,true),StructField(actor_time,TimestampType,true),StructField(actor_type,StringType,true),StructField(actor_user_access_groups,ArrayType(StructType(StructField(id,LongType,true),StructField(name,StringType,true),StructField(type,StringType,true),StructField(uuid,StringType,true)),true),true),StructField(actor_user_avatar_file_name,StringType,true),StructField(actor_user_avatar_url_150,StringType,true),StructField(actor_user_avatar_url_40,StringType,true),StructField(actor_user_avatar_url_original,StringType,true),StructField(actor_user_card_access_revoked_source,StringType,true),StructField(actor_user_cost_center,StringType,true),StructField(actor_user_department,StringType,true),StructField(actor_user_elevator_access_groups,Ar ava:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374) at py4j.Gateway.invoke(Gateway.java:282) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182) at py4j.ClientServerConnection.run(ClientServerConnection.java:106) at java.base/java.lang.Thread.run(Thread.java:840) Caused by: org.apache.avro.SchemaParseException: Illegal character in: additional_info_Area Name at org.apache.avro.Schema.validateName(Schema.java:1626) at org.apache.avro.Schema.access$400(Schema.java:94) at org.apache.avro.Schema$Field.<init>(Schema.java:558) at org.apache.avro.SchemaBuilder$FieldBuilder.completeField(SchemaBuilder.java:2258) at org.apache.avro.SchemaBuilder$FieldBuilder.completeField(SchemaBuilder.java:2254) at org.apache.avro.SchemaBuilder$FieldBuilder.access$5100(SchemaBuilder.java:2150) at org.apache.avro.SchemaBuilder$GenericDefau lt.noDefault(SchemaBuilder.java:2557) at org.apache.hudi.org.apache.spark.sql.avro.SchemaConverters$.$anonfun$toAvroType$2(SchemaConverters.scala:205) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at org.apache.spark.sql.types.StructType.foreach(StructType.scala:102) at org.apache.hudi.org.apache.spark.sql.avro.SchemaConverters$.toAvroType(SchemaConverters.scala:202) at org.apache.hudi.org.apache.spark.sql.avro.HoodieSparkAvroSchemaConverters$.toAvroType(HoodieSparkAvroSchemaConverters.scala:37) at org.apache.hudi.AvroConversionUtils$.convertStructTypeToAvroSchema(AvroConversionUtils.scala:149) ... 52 more ``` Could you please help? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
