Thanks!!’ On Sat, 24 Feb 2024 at 12:23 PM Valentyn Tymofieiev via user < [email protected]> wrote:
> You might need to pass the --save_main_session pipeline option. See: > https://beam.apache.org/documentation/sdks/python-pipeline-dependencies/#pickling-and-managing-the-main-session > > Alternatively, structure your pipeline as a package: > https://beam.apache.org/documentation/sdks/python-pipeline-dependencies/#nonpython > > > On Fri, Feb 23, 2024 at 6:14 PM Juan Romero <[email protected]> wrote: > >> Hi guys. >> >> Currently i have the class SaptmCustomerUpsertPipeline that inherited >> from the GenericPipeline class. Locally the pipeline works fine, but when i >> try to run it in dataflow i got this error: >> >> severity: "ERROR" >> textPayload: "Error message from worker: generic::unknown: Traceback >> (most recent call last): File >> "/usr/local/lib/python3.9/site-packages/apache_beam/internal/dill_pickler.py", >> line 285, in loads return dill.loads(s) File >> "/usr/local/lib/python3.9/site-packages/dill/_dill.py", line 275, in loads >> return load(file, ignore, **kwds) File >> "/usr/local/lib/python3.9/site-packages/dill/_dill.py", line 270, in load >> return Unpickler(file, ignore=ignore, **kwds).load() File >> "/usr/local/lib/python3.9/site-packages/dill/_dill.py", line 472, in load >> obj = StockUnpickler.load(self) File >> "/usr/local/lib/python3.9/site-packages/dill/_dill.py", line 462, in >> find_class return StockUnpickler.find_class(self, module, name) >> AttributeError: Can't get attribute 'SaptmCustomerUpsertPipeline' on >> <module 'apache_beam.runners.worker.sdk_worker_main' from >> '/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker_main.py'> >> >> Seems that there is an error in the serialization of the child. class. >> Someone has faced with this problem? . The pipeline works fine with the >> direct runner. >> >>
