cisaacstern commented on PR #27618:
URL: https://github.com/apache/beam/pull/27618#issuecomment-1778641046
It seems we are hitting the cloupickle code path after all, and drilling a
bit deeper, I am getting this trace:
```
Traceback (most recent call last):
File
"/Users/charlesstern/Dropbox/pangeo/scratch/distributed/distributed/protocol/pickle.py",
line 63, in dumps
result = pickle.dumps(x, **dump_kwargs)
AttributeError: Can't pickle local object
'ParDo.apply.<locals>.apply_dofn_to_bundle'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/Users/charlesstern/Dropbox/pangeo/scratch/distributed/distributed/protocol/pickle.py",
line 68, in dumps
pickler.dump(x)
AttributeError: Can't pickle local object
'ParDo.apply.<locals>.apply_dofn_to_bundle'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/Users/charlesstern/Dropbox/pangeo/scratch/distributed/distributed/protocol/serialize.py",
line 352, in serialize
header, frames = dumps(x, context=context) if wants_context else dumps(x)
File
"/Users/charlesstern/Dropbox/pangeo/scratch/distributed/distributed/protocol/serialize.py",
line 75, in pickle_dumps
frames[0] = pickle.dumps(
File
"/Users/charlesstern/Dropbox/pangeo/scratch/distributed/distributed/protocol/pickle.py",
line 82, in dumps
result = cloudpickle.dumps(x, **dump_kwargs)
File
"/Users/charlesstern/Dropbox/pangeo/scratch/beam-dask/lib/python3.10/site-packages/cloudpickle/cloudpickle_fast.py",
line 73, in dumps
cp.dump(obj)
File
"/Users/charlesstern/Dropbox/pangeo/scratch/beam-dask/lib/python3.10/site-packages/cloudpickle/cloudpickle_fast.py",
line 632, in dump
return Pickler.dump(self, obj)
AttributeError: Can't pickle local object
'_get_arg_placeholders.<locals>.ArgPlaceholder.__init__'
```
Not sure if I may be missing something basic about how this serialization is
expected to work 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
