[
https://issues.apache.org/jira/browse/IMPALA-15409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18119131#comment-18119131
]
ASF subversion and git services commented on IMPALA-15409:
----------------------------------------------------------
Commit 5ad4bf7afc380eb6e6a4352ba7846c4fca8431c6 in impala's branch
refs/heads/master from Noemi Pap-Takacs
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=5ad4bf7af ]
IMPALA-15409: Fix file paths in OPTIMIZE and replace deprecated
ContentFile.path()
OptimizeStmt with file filtering sends the paths of the selected
data files to the catalog, which matches them against the
DataFile locations to decide which files need to be replaced.
With a partition value containing '/', '"' or '=' and other
special characters, no path matched and OPTIMIZE failed with
"Files to delete cannot be empty".
It was a hidden bug because every existing OPTIMIZE test used
plain ASCII partition values.
Using ContentFile.location() to retrieve the paths fixes the issue.
This patch also replaces the now deprecated Iceberg
ContentFile.path() calls with ContentFile.location() throughout
the frontend.
Testing:
- new e2e test test_optimize_special_char_partitions
Change-Id: Ia6df5387f520acd556468ee888c127d99f8fc00d
Assisted-by: Claude Opus 5 (Claude Code)
Reviewed-on: http://gerrit.cloudera.org:8080/24927
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Fix file paths in OPTIMIZE and replace deprecated ContentFile.path()
> --------------------------------------------------------------------
>
> Key: IMPALA-15409
> URL: https://issues.apache.org/jira/browse/IMPALA-15409
> Project: IMPALA
> Issue Type: Bug
> Reporter: Noémi Pap-Takács
> Assignee: Noémi Pap-Takács
> Priority: Major
>
> OptimizeStmt with file filtering sends the paths of the selected data files
> to the catalog, which matches them against the DataFile locations to decide
> which files need to be replaced.
> With a partition value containing '/', '"' or '=', no path matches and
> OPTIMIZE fails with "Files to delete cannot be empty". It was a hidden bug
> because every existing OPTIMIZE test uses plain ASCII partition values.
> Using ContentFile.location() to retrieve the file paths can fix this issue.
> Also, we should replace the now deprecated Iceberg ContentFile.path() calls
> with ContentFile.location() throughout the frontend. It is not expected to
> impact performance because we call .toString() on ContentFile.path() anyway.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
