sreejasahithi opened a new pull request, #11919:
URL: https://github.com/apache/ozone/pull/11919
## What changes were proposed in this pull request?
Iceberg rewrite-path accepts optional --start-version and --end-version to
scope metadata and snapshot rewrites.
If users pass inverted bounds (e.g. --start-version = current/newest,
--end-version = an older file), validation previously did not reject the
command. The action could still report “Rewrite completed successfully” while
doing the wrong work:
- Metadata — Rewrite anchors on --end-version and walks only backward
through that file’s previousFiles. A start bound newer than end never appears
on that walk, so the exclusive stop never runs, and an unintended slice of
older metadata can be rewritten (e.g. vol6: end=v10 rewrote v10 down through
older log entries, not v10–v15).
- Manifests / data — With start set to newer metadata (e.g. v15) and end
on older metadata (e.g. v10), validSnapshots becomes `snapshots(end) −
snapshots(start)`, which is typically empty, so no manifest lists, data files,
or position deletes appear in the file-list even though many metadata JSON
files were staged.
This change rejects the configuration when --start-version is set and is not
chronologically older than --end-version in the loaded table’s metadata log
(including start equals end). CLI help is updated to state the ordering
requirement. Unit tests cover inverted and equal versions.
## What is the link to the Apache JIRA
[HDDS-17690](https://issues.apache.org/jira/browse/HDDS-17690)
## How was this patch tested?
Added tests to TestRewriteTablePathOzoneAction
Manually tested in docker ozone cluster:
```
bash-5.1$ ozone iceberg rewrite-path -l
"ofs://om:9862/vol1/buck1/my_db/test_table/metadata/00003-cdf55b43-4632-43c2-8c9e-350e1ee2855c.metadata.json"
-s "ofs://om:9862/vol1/buck1/my_db/test_table" -t
"s3://buck1link/my_db/test_table" --start-version
"00003-cdf55b43-4632-43c2-8c9e-350e1ee2855c.metadata.json" --end-version
"00001-99eca872-5250-4496-a432-25bd26553712.metadata.json"
Starting Iceberg table path rewrite
Table location:
ofs://om:9862/vol1/buck1/my_db/test_table/metadata/00003-cdf55b43-4632-43c2-8c9e-350e1ee2855c.metadata.json
Source prefix: ofs://om:9862/vol1/buck1/my_db/test_table
Target prefix: s3://buck1link/my_db/test_table
Table loaded: ofs://ozone-om:9862/vol1/buck1/my_db/test_table
Threads: 10
Start version: 00003-cdf55b43-4632-43c2-8c9e-350e1ee2855c.metadata.json
End version: 00001-99eca872-5250-4496-a432-25bd26553712.metadata.json
Start version (00003-cdf55b43-4632-43c2-8c9e-350e1ee2855c.metadata.json)
must be chronologically older than end version
(00001-99eca872-5250-4496-a432-25bd26553712.metadata.json) in the table
metadata log. ...
bash-5.1$ echo $?
255
bash-5.1$
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
