mzheng-plaid opened a new issue, #11865:
URL: https://github.com/apache/hudi/issues/11865
**Describe the problem you faced**
We're upgrading from Hudi 0.12.2 to Hudi 0.14.1 and are running into
failures on all of our log ingestion jobs on:
```
Caused by: org.apache.hudi.exception.HoodieException: Failed to read
schema/check compatibility for base path s3://...
at
org.apache.hudi.table.HoodieTable.validateSchema(HoodieTable.java:844)
at
org.apache.hudi.table.HoodieTable.validateUpsertSchema(HoodieTable.java:854)
writerSchema: ...
tableSchema: ...
```
The only diff in the schema seems to be the snippet below in all of our jobs
where `meta` is a `record` type (`table_name` is a placeholder for every
affected table):
```
➜ ✗ diff writer_schema_table_name.json table_schema_table_name.json
65,66c65
< "name": "meta",
< "namespace": "hoodie.table_name.table_name_record",
---
> "name": "table_name_meta",
```
Oddly, the table version was bumped even though the commit failed, so we
ended up having to run a tedious bulk downgrade command. That seems super
surprising.
**To Reproduce**
Unclear, it seems like some one-time upgrade step of the table version did
not run for whatever reason?
**Expected behavior**
1. Is this expected behavior? How do we manually upgrade our tables if not?
2. The table version should not be bumped if the commit fails
**Environment Description**
We are running on EMR 7.2
* Hudi version : 0.14.1
* Spark version : 3.5.1
* Hive version :
* Hadoop version :
* Storage (HDFS/S3/GCS..) : S3
* Running on Docker? (yes/no) :
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
