Hey,
I created a transform method in Java and now I want to use it in Python
using Cross-language.

I got pretty stuck with the following problem:
p
| GenerateSequence(...)
|ExternalTransform(...)
*=> is working like a charm *


p
| Create(...)
| ExternalTransform(...)
*=> getting assert pardo_payload.do_fn.urn ==
python_urns.PICKLED_DOFN_INFO *

Based on https://www.mail-archive.com/[email protected]/msg04887.html
it seems like a Create transform is being registered as a Java transform
but executed as Python transform.
Do you have any idea what is going on here? Thanks a lot for any help :)


Traceback (most recent call last):
File
"/Users/urban/projects/beam/sdks/python/apache_beam/io/external/snowflake_test.py",
line 92, in test_snowflake_write_read
self.run_write()
File
"/Users/urban/projects/beam/sdks/python/apache_beam/io/external/snowflake_test.py",
line 129, in run_write
expansion_service=self.expansion_service,
File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line
528, in __exit__
self.run().wait_until_finish()
File
"/Users/urban/projects/beam/sdks/python/apache_beam/testing/test_pipeline.py",
line 112, in run
False if self.not_use_test_runner_api else test_runner_api))
File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line
501, in run
self._options).run(False)
File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line
858, in from_runner_api
p.transforms_stack = [context.transforms.get_by_id(root_transform_id)]
File
"/Users/urban/projects/beam/sdks/python/apache_beam/runners/pipeline_context.py",
line 103, in get_by_id
self._id_to_proto[id], self._pipeline_context)
File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line
1231, in from_runner_api
part = context.transforms.get_by_id(transform_id)
File
"/Users/urban/projects/beam/sdks/python/apache_beam/runners/pipeline_context.py",
line 103, in get_by_id
self._id_to_proto[id], self._pipeline_context)
File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line
1231, in from_runner_api
part = context.transforms.get_by_id(transform_id)
File
"/Users/urban/projects/beam/sdks/python/apache_beam/runners/pipeline_context.py",
line 103, in get_by_id
self._id_to_proto[id], self._pipeline_context)
File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line
1231, in from_runner_api
part = context.transforms.get_by_id(transform_id)
File
"/Users/urban/projects/beam/sdks/python/apache_beam/runners/pipeline_context.py",
line 103, in get_by_id
self._id_to_proto[id], self._pipeline_context)
File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line
1231, in from_runner_api
part = context.transforms.get_by_id(transform_id)
File
"/Users/urban/projects/beam/sdks/python/apache_beam/runners/pipeline_context.py",
line 103, in get_by_id
self._id_to_proto[id], self._pipeline_context)
File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line
1170, in from_runner_api
transform = ptransform.PTransform.from_runner_api(proto, context)
File
"/Users/urban/projects/beam/sdks/python/apache_beam/transforms/ptransform.py",
line 685, in from_runner_api
context)
File
"/Users/urban/projects/beam/sdks/python/apache_beam/transforms/core.py",
line 1380, in from_runner_api_parameter
assert pardo_payload.do_fn.urn == python_urns.PICKLED_DOFN_INFO
AssertionError

Reply via email to