JingsongLi commented on code in PR #7711:
URL: https://github.com/apache/paimon/pull/7711#discussion_r3145830757
##########
paimon-python/pypaimon/write/file_store_commit.py:
##########
@@ -170,15 +170,23 @@ def overwrite(self, overwrite_partition, commit_messages:
List[CommitMessage], c
partition_filter = None
# sanity check, all changes must be done within the given partition,
meanwhile build a partition filter
if len(overwrite_partition) > 0:
- predicate_builder =
PredicateBuilder(self.table.partition_keys_fields)
- sub_predicates = []
+ # Build two predicates from overwrite_partition in one pass:
+ # - check_filter: indexed by partition keys only, used for the
sanity check
+ # below because msg.partition only contains partition key values.
+ # - partition_filter: indexed by full table schema, used by
FileScanner's
Review Comment:
Can we just let `FileScanner` supports partition_predicate？
##########
paimon-python/pypaimon/write/file_store_commit.py:
##########
@@ -170,15 +170,23 @@ def overwrite(self, overwrite_partition, commit_messages:
List[CommitMessage], c
partition_filter = None
# sanity check, all changes must be done within the given partition,
meanwhile build a partition filter
if len(overwrite_partition) > 0:
- predicate_builder =
PredicateBuilder(self.table.partition_keys_fields)
- sub_predicates = []
+ # Build two predicates from overwrite_partition in one pass:
+ # - check_filter: indexed by partition keys only, used for the
sanity check
+ # below because msg.partition only contains partition key values.
+ # - partition_filter: indexed by full table schema, used by
FileScanner's
Review Comment:
Can we just let `FileScanner` supports partition_predicate just like Java
and Rust？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
