luky777 commented on issue #6558: URL: https://github.com/apache/hudi/issues/6558#issuecomment-1278677131
Hi, I have exact same issue and can replicate it if I try to upsert the record with the extra or missing column. Using very much the same config as @parisni. I use Glue and Hudi Connector with Hudi Version 0.10.1 and Spark 3.1. I am also syncing Hudi to the glue catalog. As part of my code, I am trying to evolve the schema by pulling the Full schema from the Glue catalog and then merging it with schema from new records. Even schema has the same number of columns it keeps failing. It seems like the schema has to have columns in the same order as it is in the parquet file. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
