Noémi Pap-Takács created IMPALA-15222:
-----------------------------------------
Summary: OPTIMIZE can create orphan files
Key: IMPALA-15222
URL: https://issues.apache.org/jira/browse/IMPALA-15222
Project: IMPALA
Issue Type: Bug
Components: Frontend
Reporter: Noémi Pap-Takács
Assignee: Noémi Pap-Takács
OPTIMIZE TABLE compacts the Iceberg tables by rewriting small files. When given
a file size threshold, it can automatically detect when the table is already in
a compact form. It flags the case with NO-OP mode internally and does not
create a new commit. However, the executors still get the ExecRequest to scan
the table and rewrite the files, and just silently leave the new files on the
file system as orphan files.
We should short-circuit OPTIMIZE operation on the Frontend when it is NO-OP.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
