homar commented on PR #4660:
URL: https://github.com/apache/iceberg/pull/4660#issuecomment-1118323572
@rdblue I may be missing something but using metadata tree is not very
precise/not very feasible.
Consider following scenario(I will use SQL for readability):
```
1. CREATE TABLE orders (orderkey bigint, orderstatus varchar);
2. INSERT INTO orders VALUES(1, 'ok')
3. INSERT INTO orders VALUES(2, 'invalid')
4. INSERT INTO orders VALUES(3, 'ok')
5. Do some OPTIMIZE/COMPACT or something that will merge these 3 data files
into 1 data file
6. DELETE FROM orders WHERE orderkey = 2; // this will create position
delete
7. DELETE FROM orders WHERE EXISTS(SELECT 1) // delete everything - so 2
rows with orderkey 1 and 3
```
if you now check dataManifests from currentSnapshot it will show that 3 rows
were deleted.
When in fact only 2 were deleted in the last operation and 1 in the previous
one.
And I don't see a way from the data manifest to figure out if some rows were
deleted in the current snapshot or if the deletions were merged from the
previous position deletes/equality deletes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
