[
https://issues.apache.org/jira/browse/IMPALA-15617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arnab Karmakar resolved IMPALA-15617.
-------------------------------------
Fix Version/s: Impala 5.0.0
Resolution: Fixed
> ALTER COLUMN ... DROP DEFAULT fails on TIMESTAMP columns (Kudu, Iceberg) and
> BINARY columns (Iceberg)
> -----------------------------------------------------------------------------------------------------
>
> Key: IMPALA-15617
> URL: https://issues.apache.org/jira/browse/IMPALA-15617
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Zoltán Borók-Nagy
> Assignee: Arnab Karmakar
> Priority: Major
> Labels: impala-iceberg
> Fix For: Impala 5.0.0
>
>
> ALTER TABLE ... ALTER COLUMN ... DROP DEFAULT fails during analysis on
> TIMESTAMP columns of Kudu and Iceberg V3 tables, and on BINARY columns of
> Iceberg V3 tables. It fails whether or not the column has a default, so these
> defaults cannot be removed from Impala.
> Repro on master:
> {code:sql}
> create table k (id bigint primary key, ts timestamp default now()) stored as
> kudu;
> alter table k alter column ts drop default;
> {code}
> {noformat}
> AnalysisException: NULL cannot be cast to a TIMESTAMP literal.
> CAUSED BY: InternalException: Error converting timestamp expression:
> NullLiteral{}
> {noformat}
> {code:sql}
> create table t (i int, ts timestamp, b binary, s string)
> stored by iceberg tblproperties('format-version'='3');
> alter table t alter column ts drop default;
> alter table t alter column b drop default;
> alter table t alter column s drop default;
> {code}
> {noformat}
> AnalysisException: NULL cannot be cast to a TIMESTAMP literal.
> CAUSED BY: InternalException: Error converting timestamp expression:
> NullLiteral{}
> AnalysisException: Default values are not supported for type: BINARY
> Column has been altered.
> {noformat}
> For Kudu, the only way to drop a TIMESTAMP default from Impala is to recreate
> the table. {{ALTER COLUMN ts SET DEFAULT NULL}} and {{CREATE TABLE ... (ts
> TIMESTAMP DEFAULT NULL)}} fail with the same error.
> For Iceberg, Impala cannot define TIMESTAMP or BINARY defaults, but other
> engines can. An INSERT that omits such a column fails
> (IcebergUtil.resolveWriteDefault()). The user has to specify the value in
> every INSERT, because Impala cannot drop the write default.
> *Analysis*
> AlterTableAlterColStmt.createDropDefaultStmt() represents DROP DEFAULT as a
> ColumnDef with {{DEFAULT NullLiteral}}, so ColumnDef.analyze() analyzes NULL
> as a default value:
> * For TIMESTAMP columns, ColumnDef.analyzeKuduOptions() converts the default
> to Unix time micros with ExprUtil.utcTimestampToUnixTimeMicros() and fails
> for NULL. This conversion came with IMPALA-5338. The Kudu tests
> (kudu_create.test tbl_with_null_defaults, kudu_alter.test DROP DEFAULT) do
> not cover TIMESTAMP columns.
> * For Iceberg tables, AlterTableAlterColStmt.analyze() calls
> IcebergSchemaConverter.validateIcebergDdlDefaultType() whenever
> newColDef_.hasDefaultValue() is true, including for DROP DEFAULT. That method
> rejects TIMESTAMP, BINARY and complex types (IMPALA-14589).
> *Suggested fix*
> * ColumnDef: skip the TIMESTAMP conversion for a NULL default, so that
> DEFAULT NULL works on nullable TIMESTAMP columns as on other types.
> * AlterTableAlterColStmt: call validateIcebergDdlDefaultType() only for SET
> DEFAULT, not for DROP DEFAULT.
> * Add tests for DROP DEFAULT and SET DEFAULT NULL on Kudu TIMESTAMP columns,
> and for DROP DEFAULT on Iceberg TIMESTAMP and BINARY columns whose write
> default was set outside Impala.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
