See
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/6255/display/redirect?page=changes>
Changes:
[noreply] Fix a couple typos caught by an internal linter (#25188)
[noreply] Update Dataflow container versions (#25192)
------------------------------------------
[...truncated 2.90 GB...]
def setUp(self):
self.database_id = f'xlang_beam{uuid.uuid4()}'.replace('-', '')[:30]
> self.spanner_helper.create_database(self.database_id)
apache_beam/io/gcp/tests/xlang_spannerio_it_test.py:114:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/io/gcp/tests/xlang_spannerio_it_test.py:293: in create_database
database.create().result(120)
../../build/gradleenv/2022703441/lib/python3.7/site-packages/google/cloud/spanner_v1/database.py:415:
in create
future = api.create_database(request=request, metadata=metadata)
../../build/gradleenv/2022703441/lib/python3.7/site-packages/google/cloud/spanner_admin_database_v1/services/database_admin/client.py:823:
in create_database
metadata=metadata,
../../build/gradleenv/2022703441/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py:113:
in __call__
return wrapped_func(*args, **kwargs)
../../build/gradleenv/2022703441/lib/python3.7/site-packages/google/api_core/timeout.py:120:
in func_with_timeout
return func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
args = (parent: "projects/beam-testing/instances/beam-test"
create_statement: "CREATE DATABASE xlang_beam8f22b66d6acb4b8da1b4...G(1024) NOT
NULL,\n f_int64 INT64,\n f_boolean BOOL\n
) PRIMARY KEY (f_string)"
,)
kwargs = {'metadata': [('google-cloud-resource-prefix',
'projects/beam-testing/instances/beam-test/databases/xlang_beam8f22b66d...stances/beam-test'),
('x-goog-api-client', 'gl-python/3.7.12 grpc/1.51.1 gax/2.11.0
gapic/3.27.0')], 'timeout': 3600.0}
@functools.wraps(callable_)
def error_remapped_callable(*args, **kwargs):
try:
return callable_(*args, **kwargs)
except grpc.RpcError as exc:
> raise exceptions.from_grpc_error(exc) from exc
E google.api_core.exceptions.ServiceUnavailable: 503 failed to
connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:38733: Failed to
connect to remote host: Connection refused
../../build/gradleenv/2022703441/lib/python3.7/site-packages/google/api_core/grpc_helpers.py:74:
ServiceUnavailable
---------------------------- Captured stdout setup -----------------------------
apache_beam/io/gcp/tests/xlang_spannerio_it_test.py::CrossLanguageSpannerIOTest::test_spanner_read_table
------------------------------ Captured log setup ------------------------------
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
org.apache.beam.fn.harness.FnHarness$$Lambda$110/890160158.apply(Unknown Source)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType(BeamFnControlClient.java:151)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0(BeamFnControlClient.java:116)
------------------------------ Captured log call -------------------------------
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver$$Lambda$117/1241330181.run(Unknown
Source)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
org.apache.beam.sdk.util.UnboundedScheduledExecutorService$ScheduledFutureTask.run(UnboundedScheduledExecutorService.java:162)
--------------------------- Captured stdout teardown ---------------------------
FAILED [ 71%]
---------------------------- Captured log teardown -----------------------------
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0(BeamFnControlClient.java:116)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver$$Lambda$117/1241330181.run(Unknown
Source)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
org.apache.beam.sdk.util.UnboundedScheduledExecutorService$ScheduledFutureTask.run(UnboundedScheduledExecutorService.java:162)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
java.lang.Thread.run(Thread.java:750)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Jan 27,
2023 2:44:25 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 WARNING:
Operation ongoing in bundle 91 for at least 01h09m20s without outputting or
completing:
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at
sun.misc.Unsafe.park(Native Method)
_______________ CrossLanguageSpannerIOTest.test_spanner_replace ________________
args = (parent: "projects/beam-testing/instances/beam-test"
create_statement: "CREATE DATABASE xlang_beamc89ca307b6294ea78d43...G(1024) NOT
NULL,\n f_int64 INT64,\n f_boolean BOOL\n
) PRIMARY KEY (f_string)"
,)
kwargs = {'metadata': [('google-cloud-resource-prefix',
'projects/beam-testing/instances/beam-test/databases/xlang_beamc89ca307...stances/beam-test'),
('x-goog-api-client', 'gl-python/3.7.12 grpc/1.51.1 gax/2.11.0
gapic/3.27.0')], 'timeout': 3600.0}
@functools.wraps(callable_)
def error_remapped_callable(*args, **kwargs):
try:
> return callable_(*args, **kwargs)
../../build/gradleenv/2022703441/lib/python3.7/site-packages/google/api_core/grpc_helpers.py:72:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <grpc._channel._UnaryUnaryMultiCallable object at 0x7fc0c72efa50>
request = parent: "projects/beam-testing/instances/beam-test"
create_statement: "CREATE DATABASE xlang_beamc89ca307b6294ea78d43"...ING(1024)
NOT NULL,\n f_int64 INT64,\n f_boolean BOOL\n
) PRIMARY KEY (f_string)"
timeout = 3600.0
metadata = [('google-cloud-resource-prefix',
'projects/beam-testing/instances/beam-test/databases/xlang_beamc89ca307b6294ea78d43'...ects/beam-testing/instances/beam-test'),
('x-goog-api-client', 'gl-python/3.7.12 grpc/1.51.1 gax/2.11.0 gapic/3.27.0')]
credentials = None, wait_for_ready = None, compression = None
def __call__(self,
request,
timeout=None,
metadata=None,
credentials=None,
wait_for_ready=None,Exception in thread wait_until_finish_read:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py";,>
line 572, in read_messages
for message in self._message_stream:
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 426, in __next__
return self._next()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py";,>
line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that
terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "UNKNOWN:Error received from peer
ipv6:%5B::1%5D:41613 {grpc_message:"Socket closed", grpc_status:14,
created_time:"2023-01-27T02:47:31.46835422+00:00"}"
>
> Task :sdks:python:test-suites:portable:py37:xlangSpannerIOIT FAILED
> Task :sdks:python:test-suites:dataflow:py37:spannerioIT
>>> RUNNING integration tests with pipeline options:
>>> --runner=TestDataflowRunner --project=apache-beam-testing
>>> --region=us-central1
>>> --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it
>>> --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it
>>> --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
>>> --sdk_location=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/build/apache-beam.tar.gz>
>>> --requirements_file=postcommit_requirements.txt --num_workers=1
>>> --sleep_secs=20
>>> --kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test
>>>
>>> --dataflow_kms_key=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test
>>> pytest options: --capture=no --timeout=4500 --color=yes
>>> --log-cli-level=INFO --numprocesses=8 --dist=loadfile
>>> collect markers: -m=spannerio_it
[1m============================= test session starts
==============================[0m
platform linux -- Python 3.7.12, pytest-7.2.1, pluggy-1.0.0
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python,>
configfile: pytest.ini
plugins: xdist-2.5.0, timeout-2.1.0, forked-1.4.0, requests-mock-1.10.0,
hypothesis-6.65.1
timeout: 4500.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I
[1m[gw1] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw0] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw2] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw3] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw5] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw4] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw6] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0m[1m[gw7] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0]
[0mgw0 [15] / gw1 [15] / gw2 [15] / gw3 [15] / gw4 [15] / gw5 [15] / gw6 [15]
/ gw7 [15]
scheduling tests via LoadFileScheduling
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
[gw0] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
[gw0] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
[gw0] [32mPASSED[0m
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
[gw1] [32mPASSED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
[gw0] [32mPASSED[0m
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[gw1] [32mPASSED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
[gw1] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
[gw1] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
[gw1] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
[gw1] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
[gw1] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
[gw1] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
[gw1] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw1] [33mSKIPPED[0m
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw0] [32mPASSED[0m
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[33m=============================== warnings summary
===============================[0m
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/hdfs/config.py>:15:
DeprecationWarning: the imp module is deprecated in favour of importlib; see
the module's documentation for alternative uses
from imp import load_source
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190:
FutureWarning: WriteToSpanner is experimental. No backwards-compatibility
guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128:
FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility
guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171:
FutureWarning: WriteToSpanner is experimental. No backwards-compatibility
guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117:
FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility
guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135:
FutureWarning: WriteToSpanner is experimental. No backwards-compatibility
guarantees.
max_batch_size_bytes=250))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml>
-
[33m=========== [32m5 passed[0m, [33m[1m16 skipped[0m, [33m[1m14
warnings[0m[33m in 2199.92s (0:36:39)[0m[33m ============[0m
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'>
line: 326
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py37:postCommitPy37IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'>
line: 371
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py37:xlangSpannerIOIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 4h 38m 4s
213 actionable tasks: 149 executed, 58 from cache, 6 up-to-date
Publishing build scan...
The build scan publishing request was rejected as the data exceeded the maximum
size.
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
