Hi guys. Currently i have the class SaptmCustomerUpsertPipeline that inherited from the GenericPipeline class. Locally the pipeline works fine, but when i try to run it in dataflow i got this error:
severity: "ERROR" textPayload: "Error message from worker: generic::unknown: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/apache_beam/internal/dill_pickler.py", line 285, in loads return dill.loads(s) File "/usr/local/lib/python3.9/site-packages/dill/_dill.py", line 275, in loads return load(file, ignore, **kwds) File "/usr/local/lib/python3.9/site-packages/dill/_dill.py", line 270, in load return Unpickler(file, ignore=ignore, **kwds).load() File "/usr/local/lib/python3.9/site-packages/dill/_dill.py", line 472, in load obj = StockUnpickler.load(self) File "/usr/local/lib/python3.9/site-packages/dill/_dill.py", line 462, in find_class return StockUnpickler.find_class(self, module, name) AttributeError: Can't get attribute 'SaptmCustomerUpsertPipeline' on <module 'apache_beam.runners.worker.sdk_worker_main' from '/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker_main.py'> Seems that there is an error in the serialization of the child. class. Someone has faced with this problem? . The pipeline works fine with the direct runner.
