sungwy commented on code in PR #3474:
URL: https://github.com/apache/iceberg-python/pull/3474#discussion_r3454112883
##########
pyiceberg/table/puffin.py:
##########
@@ -114,3 +152,99 @@ def __init__(self, puffin: bytes) -> None:
def to_vector(self) -> dict[str, "pa.ChunkedArray"]:
return {path: _bitmaps_to_chunked_array(bitmaps) for path, bitmaps in
self._deletion_vectors.items()}
+
+
+class PuffinWriter:
+ """Writes a Puffin file containing a single deletion-vector-v1 blob to an
output file."""
+
+ _output_file: OutputFile
+ _blobs: list[PuffinBlobMetadata]
+ _blob_payloads: list[bytes]
+ _created_by: str
+
+ def __init__(self, output_file: OutputFile, created_by: str | None = None)
-> None:
+ self._output_file = output_file
+ self._blobs = []
+ self._blob_payloads = []
+ self._created_by = (
+ created_by if created_by is not None else f"PyIceberg version
{importlib.metadata.version('pyiceberg')}"
Review Comment:
small nit: pyiceberg already exposes `__version__` in
`pyiceberg/__init__.py`, so we could do `from pyiceberg import __version__`
here instead of `importlib.metadata.version('pyiceberg')` like we do in
[cli/console.py](https://github.com/apache/iceberg-python/blob/8c7912fe61fb73e722d4e8b354763fcc29683a60/pyiceberg/cli/console.py#L29).
Besides matching the existing pattern, `importlib.metadata.version` raises
`PackageNotFoundError` when pyiceberg isn't pip-installed, which would crash
the writer over the cosmetic `created-by` field.
##########
pyiceberg/table/puffin.py:
##########
@@ -114,3 +152,99 @@ def __init__(self, puffin: bytes) -> None:
def to_vector(self) -> dict[str, "pa.ChunkedArray"]:
return {path: _bitmaps_to_chunked_array(bitmaps) for path, bitmaps in
self._deletion_vectors.items()}
+
+
+class PuffinWriter:
Review Comment:
+1 to @ebyhr's point
Once #3491 lands I think it would be worth rebasing this implementation on
top of it. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
