[
https://issues.apache.org/jira/browse/BEAM-13388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17462886#comment-17462886
]
Valentyn Tymofieiev commented on BEAM-13388:
--------------------------------------------
- When using --save_main_session, test pipeline fails with a different error
(some transient object in main session cannot be pickled at job submission).
- Confirmed that the error still happening when launching a pipeline from
gcr.io/cloud-dataflow/v1beta3/python37:2.35.0 container.
Using this command to iterate:
{noformat}
python -m pytest -o log_cli=True -o log_level=Debug
apache_beam/ml/gcp/cloud_dlp_it_test.py::^CoudDLPIT::test_inspection
--test-pipeline-options='--runner=TestDataflowRunner
--project=apache-beam-testing --temp_location=gs://valentyn-test/tmp
--sdk_location=dist/apache-beam-2.35.0.dev0.tar.gz --region=us-central1 '
{noformat}
> [beam_PostCommit_Python38]
> [apache_beam.ml.gcp.cloud_dlp_it_test.CloudDLPIT.test_deidentification]
> Failure summary
> ------------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-13388
> URL: https://issues.apache.org/jira/browse/BEAM-13388
> Project: Beam
> Issue Type: Bug
> Components: test-failures
> Reporter: Yichi Zhang
> Assignee: Valentyn Tymofieiev
> Priority: P1
> Labels: currently-failing
> Fix For: 2.35.0
>
> Time Spent: 11h 10m
> Remaining Estimate: 0h
>
> _Use this form to file an issue for test failure:_
> * [Jenkins
> Job|https://ci-beam.apache.org/job/beam_PostCommit_Python38/2012/testReport/apache_beam.ml.gcp.cloud_dlp_it_test/CloudDLPIT/test_deidentification/]
> * [Gradle Build Scan|TODO]
> * [Test source code|TODO]
> Initial investigation:
> {code:java}
> E
> apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException:
> Dataflow pipeline failed. State: FAILED, Error:
> E Traceback (most recent call last):
> E File "apache_beam/runners/common.py", line 1239, in
> apache_beam.runners.common.DoFnRunner._invoke_lifecycle_method
> E File "apache_beam/runners/common.py", line 465, in
> apache_beam.runners.common.DoFnInvoker.invoke_setup
> E File
> "/usr/local/lib/python3.8/site-packages/apache_beam/ml/gcp/cloud_dlp.py",
> line 193, in setup
> E 'parent': self.client.project_path(self.project)
> E AttributeError: 'DlpServiceClient' object has no attribute
> 'project_path'
> E
> E During handling of the above exception, another exception occurred:
> E
> E Traceback (most recent call last):
> E File
> "/usr/local/lib/python3.8/site-packages/dataflow_worker/batchworker.py", line
> 644, in do_work
> E work_executor.execute()
> E File
> "/usr/local/lib/python3.8/site-packages/dataflow_worker/executor.py", line
> 208, in execute
> E op.start()
> E File "apache_beam/runners/worker/operations.py", line 699, in
> apache_beam.runners.worker.operations.DoOperation.start
> E File "apache_beam/runners/worker/operations.py", line 701, in
> apache_beam.runners.worker.operations.DoOperation.start
> E File "apache_beam/runners/worker/operations.py", line 702, in
> apache_beam.runners.worker.operations.DoOperation.start
> E File "apache_beam/runners/worker/operations.py", line 300, in
> apache_beam.runners.worker.operations.Operation.start
> E File "apache_beam/runners/worker/operations.py", line 306, in
> apache_beam.runners.worker.operations.Operation.start
> E File "apache_beam/runners/worker/operations.py", line 648, in
> apache_beam.runners.worker.operations.DoOperation.setup
> E File "apache_beam/runners/worker/operations.py", line 697, in
> apache_beam.runners.worker.operations.DoOperation.setup
> E File "apache_beam/runners/common.py", line 1245, in
> apache_beam.runners.common.DoFnRunner.setup
> E File "apache_beam/runners/common.py", line 1241, in
> apache_beam.runners.common.DoFnRunner._invoke_lifecycle_method
> E File "apache_beam/runners/common.py", line 1281, in
> apache_beam.runners.common.DoFnRunner._reraise_augmented
> E File "apache_beam/runners/common.py", line 1239, in
> apache_beam.runners.common.DoFnRunner._invoke_lifecycle_method
> E File "apache_beam/runners/common.py", line 465, in
> apache_beam.runners.common.DoFnInvoker.invoke_setup
> E File
> "/usr/local/lib/python3.8/site-packages/apache_beam/ml/gcp/cloud_dlp.py",
> line 193, in setup
> E 'parent': self.client.project_path(self.project)
> E AttributeError: 'DlpServiceClient' object has no attribute
> 'project_path' [while running 'MaskDetectedDetails/ParDo(_DeidentifyFn)']
> {code}
> Seems google-cloud-dlp package has backward uncompatible change.
>
> (Add any investigation notes so far)
> ----
> _After you've filled out the above details, please [assign the issue to an
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
> Assignee should [treat test failures as
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
> helping to fix the issue or find a more appropriate owner. See [Apache Beam
> Post-Commit
> Policies|https://beam.apache.org/contribute/postcommits-policies]._
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
