pan3793 commented on code in PR #15814: URL: https://github.com/apache/iceberg/pull/15814#discussion_r3019444313
########## docs/docs/spark-ddl.md: ########## @@ -326,6 +330,21 @@ ALTER TABLE prod.db.sample DROP COLUMN id; ALTER TABLE prod.db.sample DROP COLUMN point.z; ``` +!!! warning "Hive Catalog Limitation" + When using a **Hive catalog**, dropping a column that is not the last column in the table schema + may fail with an error from the Hive Metastore (HMS). This occurs because HMS validates schema + changes by comparing column types **positionally** — dropping a middle column shifts subsequent + columns, which HMS interprets as incompatible type changes + (`MetaStoreUtils#throwExceptionIfIncompatibleColTypeChange`). + + To work around this, disable the HMS schema compatibility check by setting + `hive.metastore.disallow.incompatible.col.type.changes=false` in `hive-site.xml`, or by passing + `--conf spark.hadoop.hive.metastore.disallow.incompatible.col.type.changes=false` when starting Spark. Review Comment: @jackylee-ch, I see, and have double-checked with our hive team, there are additional internal changes to make client-side overriding work. Sorry about the incomplete information. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
