deepakpanda93 opened a new pull request, #19554:
URL: https://github.com/apache/hudi/pull/19554
### Describe the issue this Pull Request addresses
The Hive Sync troubleshooting entry for
```
Unable to alter table. The following columns have types incompatible with
the existing columns in their respective positions : col1,col2
```
tells the reader to run:
```sql
set hive.metastore.disallow.incompatible.col.type.changes=false;
```
That statement writes only to the client's session `HiveConf`. The metastore
evaluates this check against **its own**
configuration — `HiveAlterHandler#alterTable` reads `handler.getConf()` — so
against a remote metastore the recommended
`set` has no effect and the error persists. The reader does the one thing
the page tells them to do, sees no change, and
has nothing to go on.
No GitHub issue for this one; it surfaced while working on #19551, which
documents the same property from the Schema
Evolution side.
### Summary and Changelog
The existing paragraph and its snippet are left untouched — they are correct
for an embedded metastore. Appended a short
paragraph plus one snippet covering the remote case:
- Set the property in the metastore's `hive-site.xml` and restart the
service, or
- override it for a single **Beeline** session with `set metaconf:...`,
which pushes the value to the metastore for that
connection.
Plus a one-line caveat that the `metaconf:` form applies over HiveServer2
and not the legacy Hive CLI (see below).
Applied to `website/docs/troubleshooting.md` (next) and
`website/versioned_docs/version-1.2.0/troubleshooting.md` (current released
docs), per the next-plus-current convention
used in #19473. The nine older versioned copies carry the same text and were
left alone — happy to widen if preferred.
Purely additive: no existing sentence, snippet, or heading was changed or
removed, so inbound links and anchors are
unaffected.
### Where this comes from in the code
Hive `rel/release-3.1.3`:
- `HiveAlterHandler.java:164-169` — the check reads `handler.getConf()`, the
metastore's conf, not the client's.
- `SetProcessor.java:202-210` — only a `metaconf:`-prefixed variable is
routed to `Hive#setMetaConf`; a plain `set` just
updates the session conf.
- `Hive.java:4625-4631` — `setMetaConf` forwards to
`IMetaStoreClient#setMetaConf`.
- `MetastoreConf.java:275` — `DISALLOW_INCOMPATIBLE_COL_TYPE_CHANGES` is in
`metaConfVars`, so the server accepts the
override.
### Reproduction
Verified against a standalone Hive **3.1.3** metastore in Docker. Table
starts as `(id int, name string, ts bigint)`;
the statement under test is `ALTER TABLE ts_test.t REPLACE COLUMNS (id int,
age int, name string, ts bigint)`, which
shifts an `int` into a position a `string` occupied — incompatible per
`ColumnType#areColTypesCompatible`.
| Client | plain `set ...=false` | `set metaconf:...=false` |
|---|---|---|
| Beeline (HiveServer2) | :x: fails | :white_check_mark: **succeeds** —
`describe` returns `id, age, name, ts` |
| Hive CLI | :x: fails | :x: fails |
The plain-`set` rows are the point of this PR: the currently-documented
remedy does not work against a remote metastore
on either client.
Separately verified that the other two remedies do work against the same
remote metastore: adding the property to the
metastore's `hive-site.xml` and restarting lifts the check with no
client-side setting at all, and a raw
`HiveMetaStoreClient` calling `setMetaConf` then `alter_table` on one client
object succeeds. Both were measured while
testing #19551.
#### Why the caveat about the Hive CLI
Worth spelling out, because the Hive CLI does not fail at the protocol level
— the metastore visibly accepts the value:
```
hive> set metaconf:hive.metastore.disallow.incompatible.col.type.changes;
metaconf:hive.metastore.disallow.incompatible.col.type.changes=true
hive> set
metaconf:hive.metastore.disallow.incompatible.col.type.changes=false;
hive> set metaconf:hive.metastore.disallow.incompatible.col.type.changes;
metaconf:hive.metastore.disallow.incompatible.col.type.changes=false
```
…and the very next `ALTER` in that same session is still rejected.
`setMetaConf` binds to the metastore connection that
received it — `HMSHandler` holds it in a thread-local conf — and the CLI's
DDL path does not run on that connection.
HiveServer2 does keep one session on one connection, which is why Beeline
works. Since a user could easily read the
value back, conclude it took effect, and be stuck, the page names Beeline
explicitly and points Hive CLI users at the
`hive-site.xml` option.
### Site verification
`npm run build` passes with the warning set byte-identical to a baseline
build of the same base commit — no new broken
links or anchors. `/docs/troubleshooting` and `/docs/next/troubleshooting`
were loaded from `npm run serve`; the new
paragraph and `sql` snippet render in the right section, the surrounding
`#### ` anchors (`hive-sync`,
`sqlexception-following-columns-have-types-incompatible`, and the following
`hoodiehivesyncexception-...`) are all
intact, and the `<h4>` count is unchanged from the untouched older versions.
### Impact
Documentation only. No code, config, or behaviour change.
### Risk Level
none
### Documentation Update
This PR is the documentation update — the Troubleshooting page,
`/docs/troubleshooting` and `/docs/next/troubleshooting`.
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
