[
https://issues.apache.org/jira/browse/BEAM-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16975495#comment-16975495
]
Valentyn Tymofieiev commented on BEAM-8668:
-------------------------------------------
This happens when SDK uses dill <=0.3.0 for pickling and worker >= 0.3.1.1 for
unpickling.
Where do you see this error? Is your branch synced to master?
https://github.com/apache/beam/blob/7beb9ee50a60d2d6ded098ebfad53852aa368107/sdks/python/setup.py#L111
?
> TypeError: _create_function() takes from 2 to 6 positional arguments but 7
> were given
> -------------------------------------------------------------------------------------
>
> Key: BEAM-8668
> URL: https://issues.apache.org/jira/browse/BEAM-8668
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-harness
> Reporter: Kyle Weaver
> Assignee: Kyle Weaver
> Priority: Major
>
> This error looks to me a lot like a mismatch between SDK versions (Dill
> dependencies), but I've had a couple reports of it so I thought it worth
> making it discoverable here.
>
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 158, in _execute
> response = task()
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 191, in <lambda>
> self._execute(lambda: worker.do_instruction(work), work)
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 343, in do_instruction
> request.instruction_id)
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 363, in process_bundle
> instruction_id, request.process_bundle_descriptor_reference)
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 306, in get
> self.data_channel_factory)
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 578, in __init__
> self.ops = self.create_execution_tree(self.process_bundle_descriptor)
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 622, in create_execution_tree
> descriptor.transforms, key=topological_height, reverse=True)])
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 621, in <listcomp>
> for transform_id in sorted(
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 546, in wrapper
> result = cache[args] = func(*args)
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 605, in get_operation
> in descriptor.transforms[transform_id].outputs.items()
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 604, in <dictcomp>
> for tag, pcoll_id
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 603, in <listcomp>
> tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 546, in wrapper
> result = cache[args] = func(*args)
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 608, in get_operation
> transform_id, transform_consumers)
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 867, in create_operation
> return creator(self, transform_id, transform_proto, payload, consumers)
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 1110, in create
> serialized_fn, parameter)
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 1148, in _create_pardo_operation
> dofn_data = pickler.loads(serialized_fn)
> File
> "/usr/local/lib/python3.6/site-packages/apache_beam/internal/pickler.py",
> line 265, in loads
> return dill.loads(s)
> File "/usr/local/lib/python3.6/site-packages/dill/_dill.py", line 317, in
> loads
> return load(file, ignore)
> File "/usr/local/lib/python3.6/site-packages/dill/_dill.py", line 305, in load
> obj = pik.load()
> TypeError: _create_function() takes from 2 to 6 positional arguments but 7
> were given
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
