[
https://issues.apache.org/jira/browse/BEAM-13595?focusedWorklogId=713830&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713830
]
ASF GitHub Bot logged work on BEAM-13595:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Jan/22 15:20
Start Date: 24/Jan/22 15:20
Worklog Time Spent: 10m
Work Description: AnandInguva commented on a change in pull request
#16589:
URL: https://github.com/apache/beam/pull/16589#discussion_r790847557
##########
File path: sdks/python/apache_beam/runners/worker/sdk_worker_main.py
##########
@@ -252,7 +252,8 @@ def _load_main_session(semi_persistent_directory):
else:
_LOGGER.warning(
'No session file found: %s. Functions defined in __main__ '
- '(interactive session) may fail.',
+ '(interactive session) may fail. Ignore this warning'
Review comment:
Added an If condition to check whether the pickle_library is
cloud_pickle or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 713830)
Time Spent: 1h 20m (was: 1h 10m)
> Disable save_main_session when using cloudpickle
> ------------------------------------------------
>
> Key: BEAM-13595
> URL: https://issues.apache.org/jira/browse/BEAM-13595
> Project: Beam
> Issue Type: New Feature
> Components: sdk-py-core
> Reporter: Ryan Thompson
> Assignee: Anand Inguva
> Priority: P3
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> [save_main_session|https://github.com/apache/beam/blob/17b62ad9e050f80a88793457aee710ea4711d47b/sdks/python/apache_beam/options/pipeline_options.py#L1089]
> is a flag in the python sdk that is used by the dill library to save all
> classes/variables/lambas in memory.
>
> When the cloudpickle library is used, no session is saved (it is a no op).
> However, if the runner sees the save_main_session option set it may try to
> access/move/manager the save_main_session saved file.
>
> To avoid this, when cloudpickle is the main library save_main_session should
> be false.
>
> See also:
> https://issues.apache.org/jira/browse/BEAM-13386
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
