kevinjqliu commented on PR #3478:
URL: https://github.com/apache/iceberg-python/pull/3478#issuecomment-5087239687
The `DeleteFileSet` approach makes sense, but I suggest making its identity
explicit with a `DeleteFileKey` rather than using an anonymous tuple.
```python
@dataclass(frozen=True, slots=True)
class DeleteFileKey:
file_path: str
content_offset: int | None
content_size_in_bytes: int | None
@classmethod
def from_file(cls, delete_file: DataFile) -> "DeleteFileKey":
return cls(
file_path=delete_file.file_path,
content_offset=delete_file.content_offset,
content_size_in_bytes=delete_file.content_size_in_bytes,
)
```
`DeleteFileSet` could then be backed by:
```python
self._files: dict[DeleteFileKey, DataFile]
```
For example:
```python
def add(self, delete_file: DataFile) -> None:
key = DeleteFileKey.from_file(delete_file)
self._files.setdefault(key, delete_file)
def discard(self, delete_file: DataFile) -> None:
key = DeleteFileKey.from_file(delete_file)
self._files.pop(key, None)
```
This makes the intended identity clearer:
- A traditional delete file is identified by `DeleteFileKey(file_path, None,
None)`.
- A DV is identified by its physical range: `DeleteFileKey(file_path,
offset, size)`.
- Multiple DVs can share the same Puffin or binary file without being
incorrectly deduplicated.
- Adding the same DV range more than once still behaves like a normal set.
Using a named, immutable key also avoids relying on tuple ordering and keeps
this specialized identity separate from the existing path-based
`DataFile.__eq__` behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
