weimingdiit commented on code in PR #2430:
URL: https://github.com/apache/auron/pull/2430#discussion_r3662292896
##########
thirdparty/auron-iceberg/src/test/scala/org/apache/auron/iceberg/AuronIcebergIntegrationSuite.scala:
##########
@@ -713,6 +713,92 @@ class AuronIcebergIntegrationSuite
}
}
+ test("iceberg changelog scan falls back for unsupported changelog
operations") {
+ withTable("local.db.t_changelog_unsupported_operation") {
+ withTempView("t_changelog_unsupported_operation_changes") {
+ sql("""
+ |create table local.db.t_changelog_unsupported_operation (id
int, v string)
+ |using iceberg
+ |tblproperties (
+ | 'format-version' = '2',
+ | 'write.delete.mode' = 'merge-on-read'
Review Comment:
Yes, that matches my reading after checking #2435 and the Iceberg 1.10.1
changelog planning path.
For a planable changelog scan, Iceberg 1.10.1 creates AddedRowsScanTask for
added data-file entries and DeletedDataFileScanTask for deleted data-file
entries. Both are created with empty delete lists. If a snapshot contains
delete manifests, orderedChangelogSnapshots throws before Auron receives any
planned changelog tasks.
Therefore, after #2435, all tasks produced by this test are supported: the
added tasks are INSERT tasks with empty deletes, the deleted task is a DELETE
task with empty existingDeletes(), and all files are Parquet. The fallback
expectation in #2430 would no longer be valid.
I will remove the mixed insert/delete fallback test from #2430 and keep the
mixed-file-format fallback coverage. #2435 already covers the mixed insert and
full-data-file delete range with the positive native expectation, so keeping
that scenario there avoids merge-order-dependent tests.
I will also update the PR description to remove the unsupported-operation
fallback claim. Thanks for catching the conflict.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
