[
https://issues.apache.org/jira/browse/DRILL-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati updated DRILL-7225:
---------------------------------------------
Description:
Merging of columnTypeInfo from two files with different schemas throws
nullpointerexception. For example if a directory Orders has two files:
* orders.parquet (with columns order_id, order_name, order_date)
* orders_with_address.parquet (with columns order_id, order_name, address)
When refresh table metadata is triggered, metadata such as total_null_count for
columns in both the files is aggregated and updated in the ColumnTypeInfo.
Initially ColumnTypeInfo is initialized with the first file's ColumnTypeInfo
(i.e., order_id, order_name, order_date). While aggregating, the existing
ColumnTypeInfo is looked up for columns in the second file and since some of
them don't exist in the ColumnTypeInfo, a npe is thrown. This can be fixed by
initializing ColumnTypeInfo for columns that are not yet present.
was:
Merging of columnTypeInfo from two files with different schemas throws
nullpointerexception. For example if a directory Orders has two files:
* orders.parquet (with columns order_id, order_name, order_date)
* orders_with_address.parquet (with columns order_id, order_name, address)
When refresh table metadata is triggered, metadata such as total_null_count for
columns in both the files is aggregated and updated in the ColumnTypeInfo.
Initially ColumnTypeInfo is initialized with the first file's ColumnTypeInfo.
While aggregating, the existing ColumnTypeInfo is looked up for columns in the
second file and since some of them don't exist in the ColumnTypeInfo, a npe is
thrown. This can be fixed by initializing ColumnTypeInfo for columns that are
not yet present.
> Merging of columnTypeInfo for file with different schema throws
> NullPointerException during refresh metadata
> ------------------------------------------------------------------------------------------------------------
>
> Key: DRILL-7225
> URL: https://issues.apache.org/jira/browse/DRILL-7225
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Venkata Jyothsna Donapati
> Assignee: Venkata Jyothsna Donapati
> Priority: Major
>
> Merging of columnTypeInfo from two files with different schemas throws
> nullpointerexception. For example if a directory Orders has two files:
> * orders.parquet (with columns order_id, order_name, order_date)
> * orders_with_address.parquet (with columns order_id, order_name, address)
> When refresh table metadata is triggered, metadata such as total_null_count
> for columns in both the files is aggregated and updated in the
> ColumnTypeInfo. Initially ColumnTypeInfo is initialized with the first file's
> ColumnTypeInfo (i.e., order_id, order_name, order_date). While aggregating,
> the existing ColumnTypeInfo is looked up for columns in the second file and
> since some of them don't exist in the ColumnTypeInfo, a npe is thrown. This
> can be fixed by initializing ColumnTypeInfo for columns that are not yet
> present.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
