See
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/345/display/redirect?page=changes>
Changes:
[noreply] exception handling for loading models (#27186)
------------------------------------------
[...truncated 193.35 KB...]
[32mPASSED[0m
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_write_with_beam_rows
[1m-------------------------------- live log call
---------------------------------[0m
[32mINFO [0m apache_beam.io.gcp.bigquery_tools:bigquery_tools.py:815
Dataset apache-beam-testing:python_xlang_storage_write_1687358408_8f0880 does
not exist so we will create it as temporary with location=None
[32mINFO [0m
apache_beam.io.external.xlang_bigqueryio_it_test:xlang_bigqueryio_it_test.py:113
Created dataset python_xlang_storage_write_1687358408_8f0880 in project
apache-beam-testing
[32mINFO [0m
apache_beam.io.external.xlang_bigqueryio_it_test:xlang_bigqueryio_it_test.py:115
expansion port: 36623
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function annotate_downstream_side_inputs at
0x7fbe1e721200> ====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function fix_side_input_pcoll_coders at 0x7fbe1e721320>
====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function pack_combiners at 0x7fbe1e721830>
====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function lift_combiners at 0x7fbe1e7218c0>
====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function expand_sdf at 0x7fbe1e721a70>
====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function expand_gbk at 0x7fbe1e721b00>
====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function sink_flattens at 0x7fbe1e721c20>
====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function greedily_fuse at 0x7fbe1e721cb0>
====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function read_to_impulse at 0x7fbe1e721d40>
====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function impulse_to_input at 0x7fbe1e721dd0>
====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function sort_stages at 0x7fbe1e71f050>
====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function add_impulse_to_dangling_transforms at
0x7fbe1e71f170> ====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function setup_timer_mapping at 0x7fbe1e721f80>
====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function populate_data_channel_coders at 0x7fbe1e71f0e0>
====================
[32mINFO [0m apache_beam.runners.worker.statecache:statecache.py:234
Creating state cache with size 104857600
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:908
Created Worker handler
<apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler
object at 0x7fbe37ca5f10> for environment
ref_Environment_default_environment_2 (beam:env:embedded_python:v1, b'')
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:510
starting control server on port 42799
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:511
starting data server on port 43453
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:512
starting state server on port 39153
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:513
starting logging server on port 33297
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:908
Created Worker handler
<apache_beam.runners.portability.fn_api_runner.worker_handlers.DockerSdkWorkerHandler
object at 0x7fbe37b10390> for environment external_10beam:env:docker:v1
(beam:env:docker:v1, b'\n apache/beam_java8_sdk:2.49.0.dev')
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:764
Attempting to pull image apache/beam_java8_sdk:2.49.0.dev
Error response from daemon: manifest for apache/beam_java8_sdk:2.49.0.dev not
found: manifest unknown: manifest unknown
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:769
Unable to pull image apache/beam_java8_sdk:2.49.0.dev, defaulting to local
image if it exists
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:790
Waiting for docker to start up. Current status is running
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:796
Docker container is running. container_id =
b'a3e222b85beca00e06753089dae1f55f7d90045e55fbfe6a247368045cb1aa70', worker_id
= worker_13
[32mINFO [0m root:worker_handlers.py:415 severity: INFO
timestamp {
seconds: 1687358416
nanos: 594000000
}
message: "Fn Harness started"
log_location: "org.apache.beam.fn.harness.FnHarness"
thread: "1"
[32mINFO [0m root:worker_handlers.py:415 severity: INFO
timestamp {
seconds: 1687358416
nanos: 763000000
}
message: "Entering instruction processing loop"
log_location: "org.apache.beam.fn.harness.FnHarness"
thread: "1"
[33mWARNING [0m root:worker_handlers.py:415 severity: WARN
timestamp {
seconds: 1687358418
nanos: 617000000
}
message: "Application name is not set. Call Builder#setApplicationName."
instruction_id: "bundle_45"
transform_id:
"StorageWriteToBigQuery/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigQueryIO.Write/StorageApiLoads/Convert/Convert
to message"
log_location: "com.google.api.client.googleapis.services.AbstractGoogleClient"
thread: "30"
[33mWARNING [0m root:worker_handlers.py:415 severity: WARN
timestamp {
seconds: 1687358419
nanos: 531000000
}
message: "Application name is not set. Call Builder#setApplicationName."
instruction_id: "bundle_45"
transform_id:
"StorageWriteToBigQuery/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigQueryIO.Write/StorageApiLoads/StorageApiWriteUnsharded/Write
Records"
log_location: "com.google.api.client.googleapis.services.AbstractGoogleClient"
thread: "30"
[33mWARNING [0m root:worker_handlers.py:415 severity: WARN
timestamp {
seconds: 1687358420
nanos: 33000000
}
message: "Application name is not set. Call Builder#setApplicationName."
instruction_id: "bundle_45"
transform_id:
"StorageWriteToBigQuery/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigQueryIO.Write/StorageApiLoads/StorageApiWriteUnsharded/Write
Records"
log_location: "com.google.api.client.googleapis.services.AbstractGoogleClient"
thread: "30"
[32mINFO [0m root:worker_handlers.py:415 severity: INFO
timestamp {
seconds: 1687358420
nanos: 363000000
}
message: "Trying to create BigQuery table:
apache-beam-testing:python_xlang_storage_write_1687358408_8f0880.write_with_beam_rows"
instruction_id: "bundle_45"
transform_id:
"StorageWriteToBigQuery/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigQueryIO.Write/StorageApiLoads/StorageApiWriteUnsharded/Write
Records"
log_location: "org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl"
thread: "30"
[32mINFO [0m root:worker_handlers.py:415 severity: INFO
timestamp {
seconds: 1687358421
nanos: 253000000
}
message: "Start connecting stream:
projects/apache-beam-testing/datasets/python_xlang_storage_write_1687358408_8f0880/tables/write_with_beam_rows/streams/Cic2N2E4NjY0Ny0wMDAwLTI4YjktODlhYS04ODNkMjRmOGIyNzg6czk
id: c9e839f2-8418-4116-b253-1e1e91dce23c"
log_location: "com.google.cloud.bigquery.storage.v1.StreamWriter"
thread: "51"
[32mINFO [0m root:worker_handlers.py:415 severity: INFO
timestamp {
seconds: 1687358421
nanos: 260000000
}
message: "Finish connecting stream:
projects/apache-beam-testing/datasets/python_xlang_storage_write_1687358408_8f0880/tables/write_with_beam_rows/streams/Cic2N2E4NjY0Ny0wMDAwLTI4YjktODlhYS04ODNkMjRmOGIyNzg6czk
id: c9e839f2-8418-4116-b253-1e1e91dce23c"
log_location: "com.google.cloud.bigquery.storage.v1.StreamWriter"
thread: "51"
[33mWARNING [0m root:worker_handlers.py:415 severity: WARN
timestamp {
seconds: 1687358421
nanos: 686000000
}
message: "Application name is not set. Call Builder#setApplicationName."
instruction_id: "bundle_47"
transform_id:
"StorageWriteToBigQuery/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigQueryIO.Write/StorageApiLoads/StorageApiWriteUnsharded/Finalize
writes/ParMultiDo(StorageApiFinalizeWrites)"
log_location: "com.google.api.client.googleapis.services.AbstractGoogleClient"
thread: "30"
[32mINFO [0m root:worker_handlers.py:415 severity: INFO
timestamp {
seconds: 1687358422
nanos: 170000000
}
message: "BatchCommit succeeded for tableId
projects/apache-beam-testing/datasets/python_xlang_storage_write_1687358408_8f0880/tables/write_with_beam_rows
response commit_time {\n seconds: 1687358422\n nanos: 156351000\n}\n"
instruction_id: "bundle_47"
transform_id:
"StorageWriteToBigQuery/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigQueryIO.Write/StorageApiLoads/StorageApiWriteUnsharded/Finalize
writes/ParMultiDo(StorageApiFinalizeWrites)"
log_location: "org.apache.beam.sdk.io.gcp.bigquery.StorageApiFinalizeWritesDoFn"
thread: "30"
[33mWARNING [0m root:worker_handlers.py:415 severity: WARN
timestamp {
seconds: 1687358422
nanos: 179000000
}
message: "Hanged up for url: \"localhost:43453\"\n."
log_location: "org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer"
thread: "22"
a3e222b85beca00e06753089dae1f55f7d90045e55fbfe6a247368045cb1aa70
[32mINFO [0m
apache_beam.io.gcp.tests.bigquery_matcher:bigquery_matcher.py:121 Attempting to
perform query SELECT * FROM
python_xlang_storage_write_1687358408_8f0880.write_with_beam_rows to BQ
[32mINFO [0m
apache_beam.io.gcp.tests.bigquery_matcher:bigquery_matcher.py:158 Result of
query is: [(1, 0.1, Decimal('1.11'), 'a', True, b'a', datetime.datetime(1970,
1, 1, 0, 16, 40, 100, tzinfo=datetime.timezone.utc)), (2, 0.2, Decimal('2.22'),
'b', False, b'b', datetime.datetime(1970, 1, 1, 0, 33, 20, 200,
tzinfo=datetime.timezone.utc)), (3, 0.3, Decimal('3.33'), 'c', True, b'd',
datetime.datetime(1970, 1, 1, 0, 50, 0, 300, tzinfo=datetime.timezone.utc)),
(4, 0.4, Decimal('4.44'), 'd', False, b'd', datetime.datetime(1970, 1, 1, 1, 6,
40, 400, tzinfo=datetime.timezone.utc))]
[32mINFO [0m
apache_beam.io.external.xlang_bigqueryio_it_test:xlang_bigqueryio_it_test.py:121
Deleting dataset python_xlang_storage_write_1687358408_8f0880 in project
apache-beam-testing
[32mPASSED[0m
[33m=============================== warnings summary
===============================[0m
../../build/gradleenv/1398941891/lib/python3.7/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1398941891/lib/python3.7/site-packages/hdfs/config.py>:15:
DeprecationWarning: the imp module is deprecated in favour of importlib; see
the module's documentation for alternative uses
from imp import load_source
../../build/gradleenv/1398941891/lib/python3.7/site-packages/google/rpc/__init__.py:18
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1398941891/lib/python3.7/site-packages/google/rpc/__init__.py>:18:
DeprecationWarning: pkg_resources is deprecated as an API. See
https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
../../build/gradleenv/1398941891/lib/python3.7/site-packages/pkg_resources/__init__.py:2871:
18 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1398941891/lib/python3.7/site-packages/pkg_resources/__init__.py>:2871:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/1398941891/lib/python3.7/site-packages/pkg_resources/__init__.py:2871:
13 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1398941891/lib/python3.7/site-packages/pkg_resources/__init__.py>:2871:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.cloud')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/1398941891/lib/python3.7/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/1398941891/lib/python3.7/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/1398941891/lib/python3.7/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/1398941891/lib/python3.7/site-packages/pkg_resources/__init__.py:2350
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1398941891/lib/python3.7/site-packages/pkg_resources/__init__.py>:2350:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(parent)
../../build/gradleenv/1398941891/lib/python3.7/site-packages/pkg_resources/__init__.py:2871
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1398941891/lib/python3.7/site-packages/pkg_resources/__init__.py>:2871:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.logging')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/1398941891/lib/python3.7/site-packages/pkg_resources/__init__.py:2871
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1398941891/lib/python3.7/site-packages/pkg_resources/__init__.py>:2871:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.iam')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/1398941891/lib/python3.7/site-packages/google/rpc/__init__.py:20
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1398941891/lib/python3.7/site-packages/google/rpc/__init__.py>:20:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.rpc')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
pkg_resources.declare_namespace(__name__)
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_all_types
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_nested_records_and_lists
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_at_least_once
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_auto_sharding
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_with_at_least_once_semantics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2028:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_all_types
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_nested_records_and_lists
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_at_least_once
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_auto_sharding
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_with_at_least_once_semantics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2034:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/sdks/python/pytest_gcpCrossLanguage.xml>
-
[33m==== [32m7 passed[0m, [33m[1m9 skipped[0m, [33m[1m6948
deselected[0m, [33m[1m52 warnings[0m[33m in 140.24s (0:02:20)[0m[33m
====[0m
> Task :sdks:python:test-suites:direct:py37:gcpCrossLanguageCleanup
Stopping expansion service pid: 2416936.
Skipping invalid pid: 2416937.
> Task :sdks:python:test-suites:xlang:fnApiJobServerCleanup
Killing process at 2416912
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':sdks:python:test-suites:direct:py311:setupVirtualenv'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 4m 1s
111 actionable tasks: 75 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/fpghaiglidnco
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
