jiangxt2 opened a new issue, #11881:
URL: https://github.com/apache/gravitino/issues/11881
### Version
main branch
### Describe what's wrong
ClickHouse distinguishes column default value kinds via
`system.columns.default_kind`:
DEFAULT, MATERIALIZED, and ALIAS. The ClickHouse catalog does not fetch this
field, so
MATERIALIZED and ALIAS columns are indistinguishable from DEFAULT columns on
round-trip.
Additionally, the ClickHouse JDBC driver's `getColumns()` implementation
hardcodes `'NO'` as
`IS_GENERATEDCOLUMN` in its SQL query (verified in clickhouse-jdbc 0.7.1
source), independent
of the actual column definition. This is the same class of driver limitation
as the one already
worked around in `getIndexes()` (clickhouse-java#1625). The code at
`ClickHouseColumnDefaultValueConverter.java`:109 has a TODO acknowledging
this limitation, and
relies solely on the hardcoded `IS_GENERATEDCOLUMN` to determine
`isExpression`, causing
expression-based columns to be treated as literal values.
### Error message and/or stacktrace
N/A. Found via code review. No runtime error — MATERIALIZED/ALIAS columns
are silently
downgraded to DEFAULT.
### How to reproduce
1. Create a table in ClickHouse with all three default kinds:
```sql
CREATE TABLE test_default_kind (
id Int64,
created_date Date DEFAULT today(),
computed_date Date MATERIALIZED today(),
alias_date Date ALIAS today()
) ENGINE=MergeTree ORDER BY id;
```
2. Verify ClickHouse stores the distinction:
```sql
SELECT name, default_kind, default_expression
FROM system.columns
WHERE database = 'default' AND table = 'test_default_kind';
```
Returns: `created_date` → DEFAULT, `computed_date` → MATERIALIZED,
`alias_date` → ALIAS
3. Load the table via Gravitino REST API and inspect column `defaultValue`
fields
4. Expected: columns should have distinct default value kinds (DEFAULT /
MATERIALIZED / ALIAS)
5. Actual: all three columns have identical `defaultValue: {type:
"unparsed", unparsedExpression: "today()"}`
— no way to distinguish MATERIALIZED or ALIAS from DEFAULT
### Additional context
- `ClickHouseColumnDefaultValueConverter.java`:109 — TODO acknowledging the
hardcoded IS_GENERATEDCOLUMN limitation
- `JdbcTableOperations.getBasicJdbcColumnInfo()` — uses `IS_GENERATEDCOLUMN`
only, never queries `default_kind`
- `ClickHouseTableOperations` does not override `getColumns()` to query
`system.columns`
- `getIndexes()` in `ClickHouseTableOperations.java` — existing workaround
for a similar driver limitation (clickhouse-java#1625), establishing the
pattern of querying system tables directly
- Fix: override `getColumns()` in `ClickHouseTableOperations` to query
`system.columns` (including `default_kind`), following the same pattern as
`getIndexes()` which already bypasses the JDBC driver. Then pass `default_kind`
to the default value converter to correctly distinguish MATERIALIZED and ALIAS
from DEFAULT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
