sd4324530 opened a new pull request, #9538:
URL: https://github.com/apache/paimon/pull/9538
### Purpose
The current docs for `ALTER TABLE ... DROP (col)` against a `hive` catalog
tell users they can work around the
`The following columns have types incompatible with the existing columns in
their respective positions` error by either:
1. disabling `hive.metastore.disallow.incompatible.col.type.changes` on the
hive server (correct), or
2. setting
`hadoop.hive.metastore.disallow.incompatible.col.type.changes=false` on the
Paimon catalog / via Flink SQL `SET` / via `spark-sql --conf
spark.hadoop.hive.metastore.disallow.incompatible.col.type.changes=false`
(incorrect for a remote HMS).
Option 2 does not work against a real (remote) Hive Metastore. The
compatibility check lives in the Hive Metastore **server**, in
`HiveAlterHandler#alterTable` (hive-metastore):
```java
if (HiveConf.getBoolVar(hiveConf,
HiveConf.ConfVars.METASTORE_DISALLOW_INCOMPATIBLE_COL_TYPE_CHANGES,
false)
&& !oldt.getTableType().equals(TableType.VIRTUAL_VIEW.toString())) {
MetaStoreUtils.throwExceptionIfIncompatibleColTypeChange(...);
}
```
The hiveConf here is the server-side HiveConf (loaded from hive-site.xml),
and the alter is invoked over Thrift via
client.alter_table_with_environmentContext(...) from
HiveAlterTableUtils#alterTableWithEnv. The Thrift RPC does not carry the
client's HiveConf to the server, so any client-side setting (including SET
hadoop.hive.metastore.disallow.incompatible.col.type.changes=false in Flink
SQL, the equivalent WITH (...) catalog option, or spark-sql --conf
spark.hadoop.hive.metastore.disallow.incompatible.col.type.changes=false) has
no effect. The check runs with the server's own value of
hive.metastore.disallow.incompatible.col.type.changes, whose default in
HiveConf is true.
That's why users following the documented workaround still hit the error.
Paimon's own unit test for this scenario passes only because it uses an
embedded HMSHandler in the same JVM, where the client HiveConf and the server
HiveConf happen to be the same object.
This PR drops option 2 and rewrites the note in
docs/docs/flink/sql-alter.md and docs/docs/spark/sql-alter.md
(plus the related tip in docs/docs/flink/sql-ddl.md) so the only recommended
path is to sethive.metastore.disallow.incompatible.col.type.changes=false on
the Hive Metastore server (hive-site.xml, then restart HMS).
Refs: HIVE-17832.
Tests
Manual: re-rendered the three updated doc pages; the new :::warning block
reads as the only supported way to opt out of the compatibility check, and the
misleading "or set ... in your paimon catalog" bullet is removed.
No code / no test changes; this is a docs-only fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
