See
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/4999/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-10212] Clean-up comments, remove rawtypes usage.
[Kyle Weaver] [BEAM-12976] Test a whole pipeline using projection pushdown in
BQ IO.
[Kyle Weaver] [BEAM-12976] Enable projection pushdown for Java pipelines on
Dataflow,
[Kyle Weaver] [BEAM-14123] Fix typo in hdfsIntegrationTest task name.
[noreply] [BEAM-13893] improved coverage of jobopts package (#17003)
[noreply] Merge pull request #16977 from [BEAM-12164] Added integration test
for
------------------------------------------
[...truncated 26.04 MB...]
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py";,>
line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py";,>
line 634, in _read_inputs
for elements in elements_iterator:
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 426, in __next__
return self._next()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that
terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string =
"{"created":"@1647584210.694169106","description":"Error received from peer
ipv6:[::1]:44701","file":"src/core/lib/surface/call.cc","file_line":903,"grpc_message":"Socket
closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py";,>
line 990, in pull_responses
for response in responses:
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 426, in __next__
return self._next()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that
terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string =
"{"created":"@1647584210.694187651","description":"Error received from peer
ipv6:[::1]:39325","file":"src/core/lib/surface/call.cc","file_line":903,"grpc_message":"Socket
closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py";,>
line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 426, in __next__
return self._next()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that
terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string =
"{"created":"@1647584210.694187200","description":"Error received from peer
ipv6:[::1]:37245","file":"src/core/lib/surface/call.cc","file_line":903,"grpc_message":"Socket
closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT
E0318 06:16:59.275644068 2799510 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
> Task :sdks:python:test-suites:portable:py37:xlangSpannerIOIT
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT
E0318 06:18:06.808705192 2799596 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 06:19:30.775033727 2799569 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 06:26:02.531518067 2799510 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 06:26:43.506890639 2799485 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 06:27:23.363040161 2799596 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 06:28:43.317199738 2799569 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 06:35:43.892138613 2799485 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 06:36:03.834080762 2799596 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 06:46:53.453025108 2799485 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 06:47:28.161238227 2799569 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 06:56:30.353256144 2799485 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 07:05:28.790888919 2799485 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 07:05:59.882824375 2799569 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 07:14:37.571745022 2799485 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 07:16:29.069038049 2799535 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 07:24:21.106946156 2799485 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 07:24:28.046002768 2799569 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 07:29:50.865590982 2799535 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 07:34:00.223476807 2799485 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 07:43:55.260560984 2799485 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 07:53:28.622551877 2799485 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 08:04:32.579121811 2799485 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
> Task :sdks:python:test-suites:dataflow:py37:spannerioIT
>>> RUNNING integration tests with pipeline options:
>>> --runner=TestDataflowRunner --project=apache-beam-testing
>>> --region=us-central1
>>> --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it
>>> --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it
>>> --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
>>> --sdk_location=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/build/apache-beam.tar.gz>
>>> --requirements_file=postcommit_requirements.txt --num_workers=1
>>> --sleep_secs=20
>>> --dataflow_worker_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-2.38.0-SNAPSHOT.jar>
>>>
>>> --kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test
>>>
>>> --dataflow_kms_key=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test
>>> pytest options: --capture=no --timeout=4500 --color=yes
>>> --log-cli-level=INFO --numprocesses=8 --dist=loadfile
>>> collect markers: -m=spannerio_it
[1m============================= test session starts
==============================[0m
platform linux -- Python 3.7.12, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python,>
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.4.0, requests-mock-1.9.3
timeout: 4500.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I
[1m[gw0] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw1] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw2] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw3] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw4] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw5] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw6] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw7] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0mgw0 [15] / gw1 [15] / gw2 [15] / gw3 [15] / gw4 [15] / gw5 [15] / gw6 [15]
/ gw7 [15]
scheduling tests via LoadFileScheduling
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
[gw1] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
[gw1] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
E0318 08:14:30.377024582 3116423 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
E0318 08:14:30.840272972 3116420 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
[gw1] [32mPASSED[0m
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
E0318 08:23:55.662457612 3116423 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
[gw0] [32mPASSED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
E0318 08:24:00.767480522 3116420 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
[gw1] [32mPASSED[0m
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
E0318 08:34:10.324385900 3116423 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
[gw0] [32mPASSED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
[gw0] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
[gw0] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
[gw0] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
[gw0] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
[gw0] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
[gw0] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
[gw0] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw0] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw1] [32mPASSED[0m
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[33m=============================== warnings summary
===============================[0m
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190:
FutureWarning: WriteToSpanner is experimental. No backwards-compatibility
guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128:
FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility
guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171:
FutureWarning: WriteToSpanner is experimental. No backwards-compatibility
guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117:
FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility
guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135:
FutureWarning: WriteToSpanner is experimental. No backwards-compatibility
guarantees.
max_batch_size_bytes=250))
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml>
-
[33m[1m============= 5 passed, 10 skipped, 5 warnings in 1688.82 seconds
==============[0m
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'>
line: 151
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:directRunnerIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py37:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'>
line: 51
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 2h 41m 51s
222 actionable tasks: 153 executed, 63 from cache, 6 up-to-date
Publishing build scan...
https://gradle.com/s/mbthutw7ynczy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
