[
https://issues.apache.org/jira/browse/BEAM-14016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17500992#comment-17500992
]
Ning commented on BEAM-14016:
-----------------------------
Just looking at the code, the
[self.service_metric.call('ok')|https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/experimental/spannerio.py#L1251]
is misplaced:
* If all elements within the bundle are indeed handled by a spanner batched
write mutation to the same table, the metric should be built only once when the
bundle is not empty
* If each element needs to build their own metric, the "call" should be invoked
inside the for-loop for each element in the bundle.
The only feasible execution route to cause the error is when the bundle is
empty.
>
> apache_beam.io.gcp.experimental.spannerio_write_it_test.SpannerWriteIntegrationTest.test_spanner_update
> fails in Python3.7 Postcommits
> ----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-14016
> URL: https://issues.apache.org/jira/browse/BEAM-14016
> Project: Beam
> Issue Type: Bug
> Components: io-py-gcp
> Reporter: Valentyn Tymofieiev
> Assignee: Ning
> Priority: P1
>
> For some reason only Python 3.7 suite is failing with this error.
> {noformat}
> > self)
> E
> apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException:
> Dataflow pipeline failed. State: FAILED, Error:
> E Traceback (most recent call last):
> E File "apache_beam/runners/common.py", line 1198, in
> apache_beam.runners.common.DoFnRunner.process
> E File "apache_beam/runners/common.py", line 537, in
> apache_beam.runners.common.SimpleInvoker.invoke_process
> E File
> "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/experimental/spannerio.py",
> line 1251, in process
> E self.service_metric.call('ok')
> E AttributeError: 'NoneType' object has no attribute 'call'
> E
> E During handling of the above exception, another exception occurred:
> E
> E Traceback (most recent call last):
> E File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 646, in do_work
> E work_executor.execute()
> E File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py", line
> 210, in execute
> E op.finish()
> E File "apache_beam/runners/worker/operations.py", line 736, in
> apache_beam.runners.worker.operations.DoOperation.finish
> E File "apache_beam/runners/worker/operations.py", line 738, in
> apache_beam.runners.worker.operations.DoOperation.finish
> E File "apache_beam/runners/worker/operations.py", line 739, in
> apache_beam.runners.worker.operations.DoOperation.finish
> E File "apache_beam/runners/common.py", line 1253, in
> apache_beam.runners.common.DoFnRunner.finish
> E File "apache_beam/runners/common.py", line 1234, in
> apache_beam.runners.common.DoFnRunner._invoke_bundle_method
> E File "apache_beam/runners/common.py", line 1265, in
> apache_beam.runners.common.DoFnRunner._reraise_augmented
> E File "apache_beam/runners/common.py", line 1232, in
> apache_beam.runners.common.DoFnRunner._invoke_bundle_method
> E File "apache_beam/runners/common.py", line 475, in
> apache_beam.runners.common.DoFnInvoker.invoke_finish_bundle
> E File "apache_beam/runners/common.py", line 480, in
> apache_beam.runners.common.DoFnInvoker.invoke_finish_bundle
> E File "apache_beam/runners/common.py", line 1401, in
> apache_beam.runners.common._OutputProcessor.finish_bundle_outputs
> E File "apache_beam/runners/worker/operations.py", line 215, in
> apache_beam.runners.worker.operations.SingletonConsumerSet.receive
> E File "apache_beam/runners/worker/operations.py", line 707, in
> apache_beam.runners.worker.operations.DoOperation.process
> E File "apache_beam/runners/worker/operations.py", line 708, in
> apache_beam.runners.worker.operations.DoOperation.process
> E File "apache_beam/runners/common.py", line 1200, in
> apache_beam.runners.common.DoFnRunner.process
> E File "apache_beam/runners/common.py", line 1281, in
> apache_beam.runners.common.DoFnRunner._reraise_augmented
> E File "apache_beam/runners/common.py", line 1198, in
> apache_beam.runners.common.DoFnRunner.process
> E File "apache_beam/runners/common.py", line 537, in
> apache_beam.runners.common.SimpleInvoker.invoke_process
> E File
> "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/experimental/spannerio.py",
> line 1251, in process
> E self.service_metric.call('ok')
> E AttributeError: 'NoneType' object has no attribute 'call' [while
> running 'WriteToSpanner/Writing to spanner']
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
