rdblue commented on code in PR #6775:
URL: https://github.com/apache/iceberg/pull/6775#discussion_r1199488231
##########
python/pyiceberg/table/__init__.py:
##########
@@ -401,9 +423,38 @@ def plan_files(self) -> Iterator[FileScanTask]:
metrics_evaluator,
)
for manifest in manifests
+ if (manifest.content is None or manifest.content ==
ManifestContent.DATA)
+ or (
+ # Not interested in deletes that are older than
the data
+ manifest.content == ManifestContent.DELETES
+ and (manifest.sequence_number or
INITIAL_SEQUENCE_NUMBER) >= min_sequence_number
+ )
],
)
+ ):
+ if datafile.content is None or datafile.content ==
DataFileContent.DATA:
+ data_datafiles.append(datafile)
+ elif datafile.content == DataFileContent.POSITION_DELETES:
+ deletes_positional.append(datafile)
+ elif datafile.content == DataFileContent.EQUALITY_DELETES:
Review Comment:
This is fine for now, though. We can move on with this, but it would be nice
to separate them out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
