claudevdm commented on code in PR #36271: URL: https://github.com/apache/beam/pull/36271#discussion_r2392303427
########## CHANGES.md: ########## @@ -94,6 +94,10 @@ * PulsarIO has now changed support status from incomplete to experimental. Both read and writes should now minimally function (un-partitioned topics, without schema support, timestamp ordered messages for read) (Java) ([#36141](https://github.com/apache/beam/issues/36141)). +* (Python) Logical type and coder registry are saved for pipelines with `save_main_session` pipeline option enabled in Review Comment: https://github.com/apache/beam/blob/bb340c2f66ac8730334160d6ed5ecd18822d059d/sdks/python/apache_beam/options/pipeline_options.py#L1647 I am still worried that changing the default behavior for cloudpickle when `save_main_session=True` which used to be a no-op (since cloudpickle doesnt need main session to unpickle types defined in main) can lead to some kind of errors. I think having an explicit flag so that users who run into a logical type issue can use is reasonable instead of pickling registries for all users that unintentionally set the `save_main_session=True` flag but don't use custom logical types at all. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
