XiaoHongbo-Hope commented on code in PR #9375:
URL: https://github.com/apache/paimon/pull/9375#discussion_r3851990640
##########
paimon-python/pypaimon/write/writer/data_writer.py:
##########
@@ -104,15 +105,33 @@ def __init__(self, table, partition: Tuple, bucket: int,
max_seq_number: int, op
# the table schema is fixed for the lifetime of this writer.
self._paimon_field_id: Dict[str, int] = {pf.name: pf.id for pf in
self.table.fields}
+ # Set by the composite writers when a flush landed its normal data file
but a
+ # later phase of the same flush failed; see their
``_close_current_writers``.
+ _pending_normal_meta: Optional[DataFileMeta] = None
+
+ @property
+ def pending_row_count(self) -> int:
+ """Number of buffered rows not yet written to a file."""
+ return self._buffer.num_rows
Review Comment:
pending_row_count always reads the base _buffer, but DataVectorWriter and
DedicatedFormatWriter buffer their normal rows in _normal_buffer. For example,
after either composite writer buffers three normal rows, this property still
returns zero.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
