tvalentyn commented on code in PR #34698:
URL: https://github.com/apache/beam/pull/34698#discussion_r2053086234
##########
sdks/python/apache_beam/internal/dill_pickler.py:
##########
@@ -376,9 +377,16 @@ def new_log_info(msg, *args, **kwargs):
logging.getLogger('dill').setLevel(logging.WARN)
-def dumps(o, enable_trace=True, use_zlib=False) -> bytes:
+def dumps(
+ o,
+ enable_trace=True,
+ use_zlib=False,
+ enable_best_effort_determinism=False) -> bytes:
"""For internal use only; no backwards-compatibility guarantees."""
with _pickle_lock:
+ if enable_best_effort_determinism:
+ dill.dill.pickle(set, save_set)
Review Comment:
does this create a side-effect (for future invocation where we have
`enable_best_effort_determinism=False` again) ?
##########
sdks/python/apache_beam/internal/dill_pickler.py:
##########
@@ -376,9 +377,16 @@ def new_log_info(msg, *args, **kwargs):
logging.getLogger('dill').setLevel(logging.WARN)
-def dumps(o, enable_trace=True, use_zlib=False) -> bytes:
+def dumps(
+ o,
+ enable_trace=True,
+ use_zlib=False,
+ enable_best_effort_determinism=False) -> bytes:
"""For internal use only; no backwards-compatibility guarantees."""
with _pickle_lock:
+ if enable_best_effort_determinism:
+ dill.dill.pickle(set, save_set)
Review Comment:
does this create a side-effect (for future invocations where we have
`enable_best_effort_determinism=False` again) ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@beam.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
