weiqingy commented on code in PR #2430:
URL: https://github.com/apache/auron/pull/2430#discussion_r3660606544
##########
thirdparty/auron-iceberg/src/test/scala/org/apache/auron/iceberg/AuronIcebergIntegrationSuite.scala:
##########
@@ -713,6 +713,92 @@ class AuronIcebergIntegrationSuite
}
}
+ test("iceberg changelog scan falls back for unsupported changelog
operations") {
+ withTable("local.db.t_changelog_unsupported_operation") {
+ withTempView("t_changelog_unsupported_operation_changes") {
+ sql("""
+ |create table local.db.t_changelog_unsupported_operation (id
int, v string)
+ |using iceberg
+ |tblproperties (
+ | 'format-version' = '2',
+ | 'write.delete.mode' = 'merge-on-read'
Review Comment:
Thanks for confirming.
I don't see the new commit yet (branch is still on `713c2018`), so this is
off your description rather than the code.
Worth settling before the rename lands: #2435 makes this exact range native.
Its `toNativeChangelogDataFileTask` accepts a `DeletedDataFileScanTask` with
operation `DELETE` and empty `existingDeletes()`, which is what this whole-file
delete produces, so all three tasks in the range (two `AddedRowsScanTask`, one
`DeletedDataFileScanTask`, all PARQUET) convert and
`assert(!plan.contains("NativeIcebergTableScan"))` flips. #2435 hit this on the
sibling test and rewrote "falls back when delete changes exist" into "supports
mixed insert and full-data-file delete changelog scan", but it branched off
master so this test isn't in its tree.
So the two PRs would describe the same scenario with opposite expectations,
and merge order decides which one breaks. Which way should it go: #2435 updates
this test too, or it gets reshaped here to still fall back? The second may not
be available. In Iceberg 1.10.1 the changelog scan only builds
`BaseAddedRowsScanTask` and `BaseDeletedDataFileScanTask`, both of which #2435
accepts, and `orderedChangelogSnapshots` throws
`UnsupportedOperationException("Delete files are currently not supported in
changelog scans")` once a snapshot in range carries a delete manifest. If that
is right, nothing reaches the task-type check afterwards. Does that match your
reading?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
