See
<https://ci-beam.apache.org/job/beam_PostCommit_Python311/737/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #28788 from apache/aaltay-patch-1
------------------------------------------
[...truncated 12.04 MB...]
(
4,
b'\xab\xac\xad',
datetime.date(2000, 1, 1),
datetime.time(0, 0, 0),
)])
]
args = self.test_pipeline.get_full_options_as_args(
on_success_matcher=hc.all_of(*pipeline_verifiers))
> with beam.Pipeline(argv=args) as p:
[1m[31mapache_beam/io/gcp/bigquery_write_it_test.py[0m:369:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[1m[31mapache_beam/pipeline.py[0m:596: in __exit__
self.result = self.run()
[1m[31mapache_beam/pipeline.py[0m:549: in run
self._options).run(False)
[1m[31mapache_beam/pipeline.py[0m:573: in run
return self.runner.run_pipeline(self, self._options)
[1m[31mapache_beam/runners/dataflow/test_dataflow_runner.py[0m:53: in
run_pipeline
self.result = super().run_pipeline(pipeline, options)
[1m[31mapache_beam/runners/dataflow/dataflow_runner.py[0m:479: in
run_pipeline
self.dataflow_client.create_job(self.job), self)
[1m[31mapache_beam/utils/retry.py[0m:275: in wrapper
return fun(*args, **kwargs)
[1m[31mapache_beam/runners/dataflow/internal/apiclient.py[0m:720: in
create_job
return self.submit_job_description(job)
[1m[31mapache_beam/utils/retry.py[0m:275: in wrapper
return fun(*args, **kwargs)
[1m[31mapache_beam/runners/dataflow/internal/apiclient.py[0m:820: in
submit_job_description
response = self._client.projects_locations_jobs.Create(request)
[1m[31mapache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py[0m:722:
in Create
return self._RunMethod(config, request, global_params=global_params)
[1m[31m../../build/gradleenv/2050596099/lib/python3.11/site-packages/apitools/base/py/base_api.py[0m:731:
in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
[1m[31m../../build/gradleenv/2050596099/lib/python3.11/site-packages/apitools/base/py/base_api.py[0m:737:
in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
<apache_beam.runners.dataflow.internal.clients.dataflow.dataflow_v1b3_client.DataflowV1b3.ProjectsLocationsJobsService
object at 0x7ff38abfa0d0>
method_config = <ApiMethodInfo
relative_path: 'v1b3/projects/{projectId}/locations/{location}/jobs'
method_id: 'dataflow.projects.lo...DataflowProjectsLocationsJobsCreateRequest'
response_type_name: 'Job'
request_field: 'job'
supports_download: False>
http_response = Response(info={'vary': 'Origin, X-Origin, Referer',
'content-type': 'application/json; charset=UTF-8', 'date': 'Tue, 0...',
request_url='https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs?alt=json')
request = <DataflowProjectsLocationsJobsCreateRequest
job: <Job
clientRequestId: '20231003181924137226-4098'
environment: <En...empFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
location: 'us-central1'
projectId: 'apache-beam-testing'>
def __ProcessHttpResponse(self, method_config, http_response, request):
"""Process the given http response."""
if http_response.status_code not in (http_client.OK,
http_client.CREATED,
http_client.NO_CONTENT):
> raise exceptions.HttpError.FromResponse(
http_response, method_config=method_config, request=request)
[1m[31mE apitools.base.py.exceptions.HttpBadRequestError: HttpError
accessing
<https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs?alt=json>:
response: <{'vary': 'Origin, X-Origin, Referer', 'content-type':
'application/json; charset=UTF-8', 'date': 'Tue, 03 Oct 2023 18:20:34 GMT',
'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0',
'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff',
'transfer-encoding': 'chunked', 'status': '400', 'content-length': '492',
'-content-encoding': 'gzip'}>, content <{[0m
[1m[31mE "error": {[0m
[1m[31mE "code": 400,[0m
[1m[31mE "message": "(633ffc3f3e41be08): The workflow could not
be created. Causes: (952165878c397953): Dataflow quota error for
jobs-per-project quota. Project apache-beam-testing is running 300 jobs. Please
check the quota usage via GCP Console. If it exceeds the limit, please wait for
a workflow to finish or contact Google Cloud Support to request an increase in
quota. If it does not, contact Google Cloud Support.",[0m
[1m[31mE "status": "FAILED_PRECONDITION"[0m
[1m[31mE }[0m
[1m[31mE }[0m
[1m[31mE >[0m
[1m[31m../../build/gradleenv/2050596099/lib/python3.11/site-packages/apitools/base/py/base_api.py[0m:603:
HttpBadRequestError
------------------------------ Captured log call -------------------------------
[32mINFO [0m apache_beam.io.gcp.bigquery_tools:bigquery_tools.py:812
Dataset apache-beam-testing:python_write_to_table_1696357161ecc060 does not
exist so we will create it as temporary with location=None
[32mINFO [0m
apache_beam.io.gcp.bigquery_write_it_test:bigquery_write_it_test.py:72 Created
dataset python_write_to_table_1696357161ecc060 in project apache-beam-testing
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:761
Executing command:
['<https://ci-beam.apache.org/job/beam_PostCommit_Python311/ws/src/build/gradleenv/2050596099/bin/python3.11',>
'-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r',
'/tmp/tmp3iphas9s/tmp_requirements.txt', '--exists-action', 'i', '--no-deps',
'--implementation', 'cp', '--abi', 'cp311', '--platform',
'manylinux2014_x86_64']
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:322 Copying
Beam SDK
"<https://ci-beam.apache.org/job/beam_PostCommit_Python311/ws/src/sdks/python/build/apache_beam-2.52.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl";>
to staging location.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:395 Pipeline
has additional dependencies to be installed in SDK worker container, consider
using the SDK container image pre-building workflow to avoid repetitive
installations. Learn more on
https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
[32mINFO [0m root:environments.py:313 Using provided Python SDK container
image: gcr.io/cloud-dataflow/v1beta3/beam_python3.11_sdk:beam-master-20230927
[32mINFO [0m root:environments.py:320 Python SDK container image set to
"gcr.io/cloud-dataflow/v1beta3/beam_python3.11_sdk:beam-master-20230927" for
Docker environment
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function pack_combiners at 0x7ff3bcf13d80>
====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function sort_stages at 0x7ff3bcf14680>
====================
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/requirements.txt...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/requirements.txt
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/mock-2.0.0-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/mock-2.0.0-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/seaborn-0.12.2-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/seaborn-0.12.2-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/seaborn-0.13.0-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/seaborn-0.13.0-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/PyHamcrest-1.10.1-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/PyHamcrest-1.10.1-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/inflection-0.5.1-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/inflection-0.5.1-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/beautifulsoup4-4.12.2-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/beautifulsoup4-4.12.2-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/parameterized-0.7.5-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/parameterized-0.7.5-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/google_cloud_aiplatform-1.33.1-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/google_cloud_aiplatform-1.33.1-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/google_cloud_aiplatform-1.34.0-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/google_cloud_aiplatform-1.34.0-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.7.2-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.7.2-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.7.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.7.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.7.3-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.7.3-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.8.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.8.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 2 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/tensorflow-2.13.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/tensorflow-2.13.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 28 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/tensorflow-2.13.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/tensorflow-2.13.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 25 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.7.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.7.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.7.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.7.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.8.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/matplotlib-3.8.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/scikit_learn-1.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/scikit_learn-1.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 1 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/scikit_learn-1.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/scikit_learn-1.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 1 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/scikit_learn-1.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/scikit_learn-1.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 1 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/scikit_learn-1.2.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/scikit_learn-1.2.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/apache_beam-2.52.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/apache_beam-2.52.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 1 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:668 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/pipeline.pb...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:684 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1003181924-136307-e6275y3s.1696357164.136451/pipeline.pb
in 0 seconds.
[32mINFO [0m
apache_beam.io.gcp.bigquery_write_it_test:bigquery_write_it_test.py:79 Deleting
dataset python_write_to_table_1696357161ecc060 in project apache-beam-testing
[33m=============================== warnings summary
===============================[0m
../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28
../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28
../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28
../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28
../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28
../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28
../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28
../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28
../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28
<https://ci-beam.apache.org/job/beam_PostCommit_Python311/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py>:28:
DeprecationWarning: the imp module is deprecated in favour of importlib and
slated for removal in Python 3.12; see the module's documentation for
alternative uses
from imp import load_source
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
<https://ci-beam.apache.org/job/beam_PostCommit_Python311/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py>:17:
DeprecationWarning: The distutils package is deprecated and slated for removal
in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
from distutils import util
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_it
apache_beam/examples/complete/game/game_stats_it_test.py::GameStatsIT::test_game_stats_it
apache_beam/examples/complete/game/leader_board_it_test.py::LeaderBoardIT::test_leader_board_it
apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_file_loads
apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_streaming_inserts
<https://ci-beam.apache.org/job/beam_PostCommit_Python311/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63:
PendingDeprecationWarning: Client.dataset is deprecated and will be removed in
a future version. Use a string like 'my_project.my_dataset' or a
cloud.google.bigquery.DatasetReference object, instead.
dataset_ref = client.dataset(unique_dataset_name, project=project)
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
<https://ci-beam.apache.org/job/beam_PostCommit_Python311/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100:
PendingDeprecationWarning: Client.dataset is deprecated and will be removed in
a future version. Use a string like 'my_project.my_dataset' or a
cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
<https://ci-beam.apache.org/job/beam_PostCommit_Python311/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:47:
FutureWarning: The default value of numeric_only in DataFrame.mean is
deprecated. In a future version, it will default to False. In addition,
specifying 'numeric_only=None' is deprecated. Select only valid columns or
specify the value of numeric_only to silence this warning.
return airline_df[at_top_airports].mean()
apache_beam/io/gcp/bigquery_read_it_test.py::ReadTests::test_native_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python311/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:170:
BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use
ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_native_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python311/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:706:
BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use
ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python311/ws/src/sdks/python/pytest_postCommitIT-df-py311.xml>
-
[36m[1m=========================== short test summary info
============================[0m
[31mFAILED[0m
apache_beam/io/gcp/bigquery_write_it_test.py::[1mBigQueryWriteIntegrationTests::test_big_query_write_without_schema[0m
- apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing
<https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs?alt=json>:
response: <{'vary': 'Origin, X-Origin, Referer', 'content-type':
'application/json; charset=UTF-8', 'date': 'Tue, 03 Oct 2023 18:20:34 GMT',
'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0',
'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff',
'transfer-encoding': 'chunked', 'status': '400', 'content-length': '492',
'-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "(633ffc3f3e41be08): The workflow could not be created. Causes:
(952165878c397953): Dataflow quota error for jobs-per-project quota. Project
apache-beam-testing is running 300 jobs. Please check the quota usage via GCP
Console. If it exceeds the limit, please wait for a workflow to finish or
contact Google Cloud Support to request an increase in quota. If it does not,
contact Google Cloud Support.",
"status": "FAILED_PRECONDITION"
}
}
>
[31m====== [31m[1m1 failed[0m, [32m86 passed[0m, [33m50 skipped[0m,
[33m26 warnings[0m[31m in 9023.18s (2:30:23)[0m[31m ======[0m
> Task :sdks:python:test-suites:dataflow:py311:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python311/ws/src/sdks/python/test-suites/direct/common.gradle'>
line: 52
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py311:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python311/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 139
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py311:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 2h 38m 2s
214 actionable tasks: 149 executed, 59 from cache, 6 up-to-date
Publishing build scan...
https://ge.apache.org/s/p73fp2ouxglc4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
