[
https://issues.apache.org/jira/browse/IMPALA-11780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltán Borók-Nagy resolved IMPALA-11780.
----------------------------------------
Fix Version/s: Impala 4.3.0
Resolution: Fixed
> Wrong FILE__POSITION values for multi row group Parquet files when page
> filtering is used
> -----------------------------------------------------------------------------------------
>
> Key: IMPALA-11780
> URL: https://issues.apache.org/jira/browse/IMPALA-11780
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg
> Fix For: Impala 4.3.0
>
>
> Impala generates wrong values for the FILE__POSITION column when the Parquet
> file contains multiple row groups and page filtering is being used.
> We are using the value of 'current_row_' to populate the file position slot:
> https://github.com/apache/impala/blob/8dbc8b733d3712035f8615f477f74c66ec9d63ea/be/src/exec/parquet/parquet-column-readers.cc#L906
> Problem is that 'current_row_' denotes the index of the row within the row
> group and not withing the file. We cannot change 'current_row_' as page
> filtering depends on its value, and the page index also uses the row
> group-based indexes of the rows, not the file indexes.
> The value of FILE__POSITION is critical for Iceberg V2 tables as position
> delete files store file positions of the deleted rows.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
