[
https://issues.apache.org/jira/browse/IMPALA-13625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dániel Gábor Vankó reassigned IMPALA-13625:
-------------------------------------------
Assignee: Dániel Gábor Vankó
> Allow reading Parquet int32/int64 as Decimal without Logical types
> ------------------------------------------------------------------
>
> Key: IMPALA-13625
> URL: https://issues.apache.org/jira/browse/IMPALA-13625
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Csaba Ringhofer
> Assignee: Dániel Gábor Vankó
> Priority: Major
> Labels: ramp-up
>
> Currently Impala can only read int32/int64 as decimal from Parquet if the
> scale/precision is provided by logical type in the Parquet metadata.
> For schema evolution it could be useful to to read files that were written as
> INT but the column was altered to a suitable DECIMAL.
> For example:
> {code}
> create table t (i int) stored as parquet;
> insert into t values (1);
> alter table t change i i decimal(10, 0);
> select * from t;
> {code}
> Curently this returns an error "2024-12-18 18:41:52 [Exception] ERROR: Query
> ...failed:
> File... column 'i' does not have the decimal precision set."
> The error comes from here:
> https://github.com/apache/impala/blob/aefd1b0920150feff31922a6979affc005a6a7d4/be/src/exec/parquet/parquet-metadata-utils.cc#L373
> ParquetMetadataUtils::ValidateColumn could be more tolerant and consider an
> int32 as decimal(9,0) or an int64 as decimal(19,0)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
