TGooch44 commented on a change in pull request #2115:
URL: https://github.com/apache/iceberg/pull/2115#discussion_r561240703
##########
File path: python/iceberg/api/partition_spec.py
##########
@@ -117,9 +118,9 @@ def compatible_with(self, other):
def lazy_fields_by_source_id(self):
if self.fields_by_source_id is None:
- self.fields_by_source_id = dict()
+ self.fields_by_source_id = defaultdict(list)
for field in self.fields:
- self.fields_by_source_id[field.source_id] = field
+ self.fields_by_source_id[field.source_id].append(field)
Review comment:
I believe this is because there may be multiple hidden partitions
derived from the same table column. I think Ryan Blue introduced this in the
java implementation here:
https://github.com/apache/iceberg/commit/649cbdde83693ebda8e8dc6e75857426d25414ec#diff-d1905822d843dea78ebe5404ee9ce885b7adbc46970fbaf931a87ae2758abeb6
Let me double check, I had written this in our internal repo a while back
and it's a little foggy now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
