[
https://issues.apache.org/jira/browse/BEAM-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789685#comment-16789685
]
Valentyn Tymofieiev commented on BEAM-6711:
-------------------------------------------
Thanks, [~ttanay] for helping to look into this, I agree with your description
of the rootcause.
The questions we need to answer:
* What are the collections in Beam that store TableReference objects. Do we
need to discern between two different instances of TableReference with the same
contents when store them in these collection.
* Can we reuse _get_hashable_destination as a workaround, see:
https://github.com/apache/beam/blob/97bf4696aed75dd05d520e7663719b2032a929b4/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py#L165
> Bigquery Tornadoes IT is broken in Python3 PostCommit test suite.
> ------------------------------------------------------------------
>
> Key: BEAM-6711
> URL: https://issues.apache.org/jira/browse/BEAM-6711
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Pablo Estrada
> Priority: Major
> Fix For: 2.12.0
>
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> First failure was observed in
> https://builds.apache.org/job/beam_PostCommit_Python3_Verify/54 , after
> https://github.com/apache/beam/commit/cdea885872b3be7de9ba22f22700be89f7d53766
> was merged.
> [~pabloem], could you please take a look? I suggest we do a rollback +
> rollforward with a fix.
> {noformat}
> root: ERROR: Exception at bundle
> <apache_beam.runners.direct.bundle_factory._Bundle object at 0x7f35e5331240>,
> due to an exception.
> Traceback (most recent call last):
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/runners/common.py",
> line 727, in process
> return self.do_fn_invoker.invoke_process(windowed_value)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/runners/common.py",
> line 556, in invoke_process
> windowed_value, additional_args, additional_kwargs, output_processor)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/runners/common.py",
> line 622, in _invoke_per_window
> self.process_method(*args_for_process, **kwargs_for_process))
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/runners/common.py",
> line 823, in process_outputs
> for result in results:
> File "/home/jenkins/jenkins-slave/works
> pace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py",
> line 191, in process
> if destination in self._destination_to_file_writer:
> TypeError: unhashable type: 'TableReference'
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
