JingsongLi commented on code in PR #10541:
URL: https://github.com/apache/paimon/pull/10541#discussion_r4492357263
##########
paimon-python/pypaimon/write/write_builder.py:
##########
@@ -35,15 +36,53 @@ def __init__(self, table):
self.table: FileStoreTable = table
self.commit_user = self._create_commit_user()
self.restore_snapshot_id = None
+ self.dynamic_options = None
+
+ def with_options(self, options: Optional[dict] = None):
+ """Derive a new builder with the given dynamic options applied.
+
+ Returns a new builder with options applied through table.copy(), so
+ table.options stays the single source of truth. Because self is left
+ untouched, callers must use the return value.
+
+ Args:
+ options: Dynamic option key/value pairs. None or empty returns
self.
+
+ Returns:
+ A new builder with the options applied, or self when nothing to
apply.
+ """
+ if not options:
+ return self
+ new_builder = type(self)(self.table.copy(options))
+ new_builder.dynamic_options = options
+ self._carry_state_to(new_builder)
+ return new_builder
+
+ def _carry_state_to(self, new_builder) -> None:
+ """Copy builder state that must survive ``with_options``.
+
+ ``with_options`` rebuilds the builder from scratch, so anything already
+ configured on it is lost unless copied here. A subclass that adds its
+ own configuration overrides this and calls super, or that configuration
+ is silently dropped the moment a caller applies options.
+ """
+ new_builder.restore_snapshot_id = self.restore_snapshot_id
Review Comment:
[P2] Preserve the configured commit identity in derived builders
`with_options()` reconstructs the builder, but `_carry_state_to()` does not
copy `commit_user`, so applying any nonempty options silently discards an
earlier public `with_commit_user(...)`. This can break stream recovery rather
than merely change a diagnostic field.
On a real ordinary append table, I created the writer/committer with
`new_stream_write_builder().with_commit_user('checkpoint-app').with_options({'target-file-size':
'1 MB'})`, committed checkpoint 7, then reconstructed the same configuration
and replayed checkpoint 7 with `CommitDedup(on_first_attempt=True,
base_snapshot_id=0)`. The two builders use different random UUIDs, so the
replay publishes snapshot 2 and the row appears twice. Putting
`with_commit_user` after `with_options` instead leaves one snapshot/row;
copying only `self.commit_user` into the derived builder also makes the
original ordering deduplicate correctly.
Please retain the configured commit identity alongside
restore/static-partition state, and cover both configuration orders. The actor
currently resets its identity after applying options, so this finding concerns
the newly exposed shared builder API, not an unconditional actor-mode replay
failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]