See
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/4428/display/redirect?page=changes>
Changes:
[noreply] Set maxConnections optional in incoming schema for
JdbcSchemaIOProvider
[noreply] Document that yield and return shouldn't be mixed. (#25060)
------------------------------------------
[...truncated 145.30 KB...]
pipeline
| beam.Create([
{
'name': 'foo', 'timestamp': MIN_TIMESTAMP
},
{
'name': 'foo', 'timestamp': 0
},
{
'name': 'bar', 'timestamp': 33
},
{
'name': 'bar', 'timestamp': 0
},
])
| beam.Map(
lambda element: beam.window.TimestampedValue(
element, element['timestamp'])))
# Reshuffle the PCollection above and assign the timestamp of an element
# to that element again.
after_reshuffle = before_reshuffle | beam.Reshuffle()
# Given an element, emits a string which contains the timestamp and the
# name field of the element.
def format_with_timestamp(element, timestamp=beam.DoFn.TimestampParam):
t = str(timestamp)
if timestamp == MIN_TIMESTAMP:
t = 'MIN_TIMESTAMP'
elif timestamp == MAX_TIMESTAMP:
t = 'MAX_TIMESTAMP'
return '{} - {}'.format(t, element['name'])
# Combine each element in before_reshuffle with its timestamp.
formatted_before_reshuffle = (
before_reshuffle
| "Get before_reshuffle timestamp" >>
beam.Map(format_with_timestamp))
# Combine each element in after_reshuffle with its timestamp.
formatted_after_reshuffle = (
after_reshuffle
| "Get after_reshuffle timestamp" >>
beam.Map(format_with_timestamp))
expected_data = [
'MIN_TIMESTAMP - foo',
'Timestamp(0) - foo',
'Timestamp(33) - bar',
'Timestamp(0) - bar'
]
# Can't compare formatted_before_reshuffle and formatted_after_reshuffle
# directly, because they are deferred PCollections while equal_to only
# takes a concrete argument.
assert_that(
formatted_before_reshuffle,
equal_to(expected_data),
label="formatted_before_reshuffle")
assert_that(
formatted_after_reshuffle,
equal_to(expected_data),
> label="formatted_after_reshuffle")
[1m[31mapache_beam/transforms/util_test.py[0m:805:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[1m[31mapache_beam/pipeline.py[0m:600: in __exit__
self.result = self.run()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.testing.test_pipeline.TestPipeline object at 0x7f8c84be9a90>
test_runner_api = True
def run(self, test_runner_api=True):
result = super().run(
test_runner_api=(
False if self.not_use_test_runner_api else test_runner_api))
if self.blocking:
state = result.wait_until_finish()
> assert state in (PipelineState.DONE, PipelineState.CANCELLED), \
"Pipeline execution failed."
[1m[31mE AssertionError: Pipeline execution failed.[0m
[1m[31mE assert 'FAILED' in ('DONE', 'CANCELLED')[0m
[1m[31mapache_beam/testing/test_pipeline.py[0m:117: AssertionError
------------------------------ Captured log call -------------------------------
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:772
Executing command:
['<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967052/bin/python3.7',>
'-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r',
'/tmp/tmp0isa8ykz/tmp_requirements.txt', '--exists-action', 'i', '--no-deps',
'--implementation', 'cp', '--abi', 'cp37m', '--platform',
'manylinux2014_x86_64']
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:330 Copying
Beam SDK
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/build/apache-beam.tar.gz";>
to staging location.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:444 Pipeline
has additional dependencies to be installed in SDK worker container, consider
using the SDK container image pre-building workflow to avoid repetitive
installations. Learn more on
https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
[32mINFO [0m root:environments.py:376 Default Python SDK image for
environment is apache/beam_python3.7_sdk:2.46.0.dev
[32mINFO [0m root:environments.py:296 Using provided Python SDK container
image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20221205
[32mINFO [0m root:environments.py:304 Python SDK container image set to
"gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20221205" for Docker
environment
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/requirements.txt...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/requirements.txt
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/0-0.0.0-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/0-0.0.0-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/mock-2.0.0-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/mock-2.0.0-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/seaborn-0.12.1-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/seaborn-0.12.1-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/seaborn-0.12.2-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/seaborn-0.12.2-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/PyHamcrest-1.10.1-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/PyHamcrest-1.10.1-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/transformers-4.24.0-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/transformers-4.24.0-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/transformers-4.25.1-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/transformers-4.25.1-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/inflection-0.5.1-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/inflection-0.5.1-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/beautifulsoup4-4.11.1-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/beautifulsoup4-4.11.1-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/parameterized-0.7.5-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/parameterized-0.7.5-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/tensorflow_transform-1.12.0-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/tensorflow_transform-1.12.0-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/torch-1.13.0-cp38-cp38-manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/torch-1.13.0-cp38-cp38-manylinux1_x86_64.whl
in 43 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/torch-1.13.1-cp38-cp38-manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/torch-1.13.1-cp38-cp38-manylinux1_x86_64.whl
in 35 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/torch-1.13.0-cp37-cp37m-manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/torch-1.13.0-cp37-cp37m-manylinux1_x86_64.whl
in 41 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/torchvision-0.14.0-cp38-cp38-manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/torchvision-0.14.0-cp38-cp38-manylinux1_x86_64.whl
in 1 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/torchvision-0.14.1-cp38-cp38-manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/torchvision-0.14.1-cp38-cp38-manylinux1_x86_64.whl
in 1 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/torchvision-0.14.0-cp37-cp37m-manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/torchvision-0.14.0-cp37-cp37m-manylinux1_x86_64.whl
in 1 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/Pillow-9.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/Pillow-9.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/Pillow-9.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/Pillow-9.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/Pillow-9.3.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/Pillow-9.3.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/matplotlib-3.5.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/matplotlib-3.5.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/tfx_bsl-1.12.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/tfx_bsl-1.12.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
in 1 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/matplotlib-3.6.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/matplotlib-3.6.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/dataflow_python_sdk.tar...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/dataflow_python_sdk.tar
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:723 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/pipeline.pb...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:742 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0119045906-118808-v42xagsj.1674104346.118996/pipeline.pb
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:898 Create job:
<Job
clientRequestId: '20230119045906120074-2075'
createTime: '2023-01-19T05:01:18.062836Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-01-18_21_01_17-13524008252908449462'
location: 'us-central1'
name: 'beamapp-jenkins-0119045906-118808-v42xagsj'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-01-19T05:01:18.062836Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:900 Created job
with id: [2023-01-18_21_01_17-13524008252908449462]
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:901 Submitted job:
2023-01-18_21_01_17-13524008252908449462
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:907 To access the
Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2023-01-18_21_01_17-13524008252908449462?project=apache-beam-testing
[32mINFO [0m
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:58
Console log:
[32mINFO [0m
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:59
https://console.cloud.google.com/dataflow/jobs/us-central1/2023-01-18_21_01_17-13524008252908449462?project=apache-beam-testing
[33mWARNING [0m
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:65
Waiting indefinitely for streaming job.
[33m=============================== warnings summary
===============================[0m
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py>:15:
DeprecationWarning: the imp module is deprecated in favour of importlib; see
the module's documentation for alternative uses
from imp import load_source
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py37-xdist.xml>
-
[36m[1m=========================== short test summary info
============================[0m
[31mFAILED[0m
apache_beam/transforms/util_test.py::[1mReshuffleTest::test_reshuffle_preserves_timestamps[0m
- AssertionError: Pipeline execution failed.
assert 'FAILED' in ('DONE', 'CANCELLED')
[31m======= [31m[1m1 failed[0m, [32m31 passed[0m, [33m7 skipped[0m,
[33m9 warnings[0m[31m in 3903.07s (1:05:03)[0m[31m =======[0m
> Task :sdks:python:test-suites:dataflow:py37:validatesRunnerStreamingTests
> FAILED
> Task :sdks:python:test-suites:dataflow:py310:validatesRunnerStreamingTests
[32mPASSED[0m
[33m=============================== warnings summary
===============================[0m
../../build/gradleenv/2050596098/lib/python3.10/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/hdfs/config.py>:15:
DeprecationWarning: the imp module is deprecated in favour of importlib and
slated for removal in Python 3.12; see the module's documentation for
alternative uses
from imp import load_source
../../build/gradleenv/2050596098/lib/python3.10/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/google/api_core/operations_v1/abstract_operations_client.py>:17:
DeprecationWarning: The distutils package is deprecated and slated for removal
in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
from distutils import util
../../build/gradleenv/2050596098/lib/python3.10/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/tenacity/_asyncio.py>:42:
DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use
"async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/typehints/pandas_type_compatibility_test.py:67
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:67:
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas
in a future version. Use pandas.Index with the appropriate dtype instead.
}).set_index(pd.Int64Index(range(123, 223), name='an_index')),
apache_beam/typehints/pandas_type_compatibility_test.py:90
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:90:
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas
in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(123, 223), name='an_index'),
apache_beam/typehints/pandas_type_compatibility_test.py:91
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:91:
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas
in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(475, 575), name='another_index'),
apache_beam/runners/dataflow/dataflow_exercise_streaming_metrics_pipeline_test.py::ExerciseStreamingMetricsPipelineTest::test_streaming_pipeline_returns_expected_user_metrics_fnapi_it
apache_beam/runners/dataflow/dataflow_exercise_streaming_metrics_pipeline_test.py::ExerciseStreamingMetricsPipelineTest::test_streaming_pipeline_returns_expected_user_metrics_fnapi_it
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/httplib2/__init__.py>:147:
DeprecationWarning: ssl.PROTOCOL_TLS is deprecated
context = ssl.SSLContext(DEFAULT_TLS_VERSION)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py310-noxdist.xml>
-
[33m==== [32m1 passed[0m, [33m[1m6 skipped[0m, [33m[1m6800
deselected[0m, [33m[1m8 warnings[0m[33m in 880.24s (0:14:40)[0m[33m
=====[0m
FAILURE: Build failed with an exception.
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 241
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py37:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 15m 44s
20 actionable tasks: 14 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/wqsd6s5rnh2zy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
