See
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/604/display/redirect>
Changes:
------------------------------------------
[...truncated 2.74 MB...]
return self._RunMethod(config, request, global_params=global_params)
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/apitools/base/py/base_api.py:731:
in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/apitools/base/py/base_api.py:737:
in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
<apache_beam.runners.dataflow.internal.clients.dataflow.dataflow_v1b3_client.DataflowV1b3.ProjectsLocationsJobsService
object at 0x7f0135ba8f40>
method_config = <ApiMethodInfo
relative_path: 'v1b3/projects/{projectId}/locations/{location}/jobs/{jobId}'
method_id: 'dataflow.pro...DataflowProjectsLocationsJobsUpdateRequest'
response_type_name: 'Job'
request_field: 'job'
supports_download: False>
http_response = Response(info={'vary': 'Origin, X-Origin, Referer',
'content-type': 'application/json; charset=UTF-8', 'date': 'Wed,
1...is.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2023-09-12_19_25_39-15653423758417993759?alt=json')
request = <DataflowProjectsLocationsJobsUpdateRequest
job: <Job
requestedState: RequestedStateValueValuesEnum(JOB_STATE_CANCEL...les: []>
jobId: '2023-09-12_19_25_39-15653423758417993759'
location: 'us-central1'
projectId: 'apache-beam-testing'>
def __ProcessHttpResponse(self, method_config, http_response, request):
"""Process the given http response."""
if http_response.status_code not in (http_client.OK,
http_client.CREATED,
http_client.NO_CONTENT):
> raise exceptions.HttpError.FromResponse(
http_response, method_config=method_config, request=request)
E apitools.base.py.exceptions.HttpBadRequestError: HttpError
accessing
<https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2023-09-12_19_25_39-15653423758417993759?alt=json>:
response: <{'vary': 'Origin, X-Origin, Referer', 'content-type':
'application/json; charset=UTF-8', 'date': 'Wed, 13 Sep 2023 02:35:44 GMT',
'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0',
'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff',
'transfer-encoding': 'chunked', 'status': '400', 'content-length': '284',
'-content-encoding': 'gzip'}>, content <{
E "error": {
E "code": 400,
E "message": "(bb5d53da3f3b103b): Workflow modification failed.
Causes: (5dd8c73ccbd9261f): Cannot perform operation 'cancel' on Job:
2023-09-12_19_25_39-15653423758417993759. Job has no backend assigned..",
E "status": "FAILED_PRECONDITION"
E }
E }
E >
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/apitools/base/py/base_api.py:603:
HttpBadRequestError
----------------------------- Captured stdout call -----------------------------
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-12_19_25_39-15653423758417993759?project=apache-beam-testing
------------------------------ Captured log call -------------------------------
INFO apache_beam.internal.gcp.auth:auth.py:130 Setting socket default
timeout to 60 seconds.
INFO apache_beam.internal.gcp.auth:auth.py:132 socket default timeout is
60.0 seconds.
INFO apache_beam.runners.portability.stager:stager.py:322 Copying Beam SDK
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/sdks/python/build/apache-beam.tar.gz";>
to staging location.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:397
Pipeline has additional dependencies to be installed in SDK worker container,
consider using the SDK container image pre-building workflow to avoid
repetitive installations. Learn more on
https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO root:environments.py:313 Using provided Python SDK container image:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20230913-022352423501358
INFO root:environments.py:320 Python SDK container image set to
"us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20230913-022352423501358"
for Docker environment
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function pack_combiners at 0x7f0149d99820>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function sort_stages at 0x7f0149d9a040>
====================
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668
Starting GCS upload to
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0913022539-115377-zxz6xnth.1694571939.115551/dataflow_python_sdk.tar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684
Completed GCS upload to
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0913022539-115377-zxz6xnth.1694571939.115551/dataflow_python_sdk.tar
in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668
Starting GCS upload to
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0913022539-115377-zxz6xnth.1694571939.115551/pipeline.pb...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684
Completed GCS upload to
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0913022539-115377-zxz6xnth.1694571939.115551/pipeline.pb
in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:843
Create job: <Job
clientRequestId: '20230913022539116472-5768'
createTime: '2023-09-13T02:25:40.099852Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-12_19_25_39-15653423758417993759'
location: 'us-central1'
name: 'beamapp-jenkins-0913022539-115377-zxz6xnth'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-13T02:25:40.099852Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:845
Created job with id: [2023-09-12_19_25_39-15653423758417993759]
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:846
Submitted job: 2023-09-12_19_25_39-15653423758417993759
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:847 To
access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-12_19_25_39-15653423758417993759?project=apache-beam-testing
INFO
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:58
Console log:
INFO
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:59
https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-12_19_25_39-15653423758417993759?project=apache-beam-testing
ERROR
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:102
Console URL:
https://console.cloud.google.com/dataflow/<regionId>/2023-09-12_19_25_39-15653423758417993759?project=<projectId>
=============================== warnings summary ===============================
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/hdfs/config.py:28
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/hdfs/config.py:28
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/hdfs/config.py>:28:
DeprecationWarning: the imp module is deprecated in favour of importlib; see
the module's documentation for alternative uses
from imp import load_source
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/google/rpc/__init__.py:18
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/google/rpc/__init__.py:18
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/google/rpc/__init__.py>:18:
DeprecationWarning: pkg_resources is deprecated as an API. See
https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2871:
38 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py>:2871:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2871:
32 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py>:2871:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.cloud')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2350
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py>:2350:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(parent)
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2871
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2871
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py>:2871:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.logging')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2871
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py:2871
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/pkg_resources/__init__.py>:2871:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.iam')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/google/rpc/__init__.py:20
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/google/rpc/__init__.py:20
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/google/rpc/__init__.py>:20:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.rpc')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
pkg_resources.declare_namespace(__name__)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/sdks/python/pytest-beam_python3.9_sdk.xml>
-
=========================== short test summary info ============================
FAILED
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_fnapi_it
- apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing
<https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2023-09-12_19_25_39-15653423758417993759?alt=json>:
response: <{'vary': 'Origin, X-Origin, Referer', 'content-type':
'application/json; charset=UTF-8', 'date': 'Wed, 13 Sep 2023 02:35:44 GMT',
'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0',
'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff',
'transfer-encoding': 'chunked', 'status': '400', 'content-length': '284',
'-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "(bb5d53da3f3b103b): Workflow modification failed. Causes:
(5dd8c73ccbd9261f): Cannot perform operation 'cancel' on Job:
2023-09-12_19_25_39-15653423758417993759. Job has no backend assigned..",
"status": "FAILED_PRECONDITION"
}
}
>
====== 1 failed, 3 passed, 17 skipped, 88 warnings in 1857.48s (0:30:57) =======
cleanup_container
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20230913-022352423501358
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:b090085b7c747d5b9e812b249795d84d4ee6dbe894afc8f41e794a9670c1640a
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:ed3e5aaf-6ed7-4898-b2b9-c0188df80a97
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:d061e6dcaaa7683cf9eacabb4d3de9f7fc06c7741ec5fbaab34d246172812465
Deleted: sha256:03db528e9605be1c82597f5269e335dec65e0ef69ddd62ac43721c0acece4bb1
Deleted: sha256:c37858d5e6b35ebd80ba5a06b00ead5937df842d92f2eb14d1c78c9bfdb56cd9
Deleted: sha256:6049bef2e18a88842e103adb30ea925d741795804dc6a0117674fd5c0ab7bedf
Deleted: sha256:237d488918a6b38f8aa89bee7d572b16faa1e7dac60923faf785d433d9f05ab2
Deleted: sha256:90b43df479ef1012302878157e7e0de5aa73687b6cc0c2688dca98b3880425a7
Deleted: sha256:569f780c64618b806b315f3f56e0005f7c99c67f1601678fb4caaf993540cbf8
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:b090085b7c747d5b9e812b249795d84d4ee6dbe894afc8f41e794a9670c1640a
Associated tags:
- 20230913-022352423501358
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20230913-022352423501358
> Task :sdks:python:test-suites:dataflow:py311:validatesContainer
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python311_sdk/beam_python_prebuilt_sdk@sha256:8810e8047c031007e9000105fd536f8c986c2f7143f8d32a00ba0abdd8534621
Associated tags:
- 220b1494-3bca-450d-b55f-06274d73b7f5
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python311_sdk/beam_python_prebuilt_sdk:220b1494-3bca-450d-b55f-06274d73b7f5].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python311_sdk/beam_python_prebuilt_sdk@sha256:8810e8047c031007e9000105fd536f8c986c2f7143f8d32a00ba0abdd8534621].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python311_sdk/beam_python_prebuilt_sdk@sha256:dd92d3ad58b523fa428780c2e41570e04ed0cec1290e81ce6ac8627020eabf3d
Associated tags:
- e59adb09-4634-4493-812d-4a5781c73fee
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python311_sdk/beam_python_prebuilt_sdk:e59adb09-4634-4493-812d-4a5781c73fee].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python311_sdk/beam_python_prebuilt_sdk@sha256:dd92d3ad58b523fa428780c2e41570e04ed0cec1290e81ce6ac8627020eabf3d].
Removed the container
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20230913-022352423501358].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:b090085b7c747d5b9e812b249795d84d4ee6dbe894afc8f41e794a9670c1640a].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:d061e6dcaaa7683cf9eacabb4d3de9f7fc06c7741ec5fbaab34d246172812465
Associated tags:
- ed3e5aaf-6ed7-4898-b2b9-c0188df80a97
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:ed3e5aaf-6ed7-4898-b2b9-c0188df80a97].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:d061e6dcaaa7683cf9eacabb4d3de9f7fc06c7741ec5fbaab34d246172812465].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:b214f2c1e5b143f99980117e988eb00f2ef94e349ff3712eecfee47deaee73e5
Associated tags:
- 8374a1eb-f96f-4196-ad23-34d286f33e87
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:8374a1eb-f96f-4196-ad23-34d286f33e87].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:b214f2c1e5b143f99980117e988eb00f2ef94e349ff3712eecfee47deaee73e5].
Removed the container
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 351
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 351
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py310:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_with_RC/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 351
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py39:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 43m 6s
66 actionable tasks: 55 executed, 4 from cache, 7 up-to-date
Publishing build scan...
https://ge.apache.org/s/3yr2srkfdg35s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
