See
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/654/display/redirect?page=changes>
Changes:
[noreply] adds maven to image (#28348)
------------------------------------------
[...truncated 668.21 KB...]
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/external/xlang_bigqueryio_it_test.py: 10 warnings
apache_beam/io/gcp/bigtableio_it_test.py: 6 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/transforms/external.py>:679:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
self._expansion_service, pipeline.options)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/pytest_gcpCrossLanguage.xml>
-
[36m[1m=========================== short test summary info
============================[0m
[31mFAILED[0m
apache_beam/examples/snippets/snippets_test.py::[1mSnippetsTest::test_model_bigqueryio_xlang[0m
- RuntimeError:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar>
not found. Please build the server with
cd
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;>
./gradlew sdks:java:io:google-cloud-platform:expansion-service:build
[31m= [31m[1m1 failed[0m, [32m13 passed[0m, [33m17 skipped[0m,
[33m7149 deselected[0m, [33m75 warnings[0m[31m in 6064.53s
(1:41:04)[0m[31m =[0m
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava
> FAILED
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguageCleanup
Stopping expansion service pid: 2803492.
Skipping invalid pid: 2803493.
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204
2023-09-07T15:49:48.032Z: JOB_MESSAGE_BASIC: Worker pool stopped.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:154 Job
2023-09-07_08_43_22-9183506566346642196 is in state JOB_STATE_DONE
[32mINFO [0m
apache_beam.io.gcp.bigtableio_it_test:bigtableio_it_test.py:190 Deleting table
[test-table-1694100947-bee7e4]
[32mPASSED[0m
[1m------------------------------ live log teardown
-------------------------------[0m
[32mINFO [0m
apache_beam.io.gcp.bigtableio_it_test:bigtableio_it_test.py:198 Deleting
instance [bt-write-xlang-1694099019-83ad22]
=================================== FAILURES ===================================
[31m[1m___________________ SnippetsTest.test_model_bigqueryio_xlang
___________________[0m
self = <apache_beam.examples.snippets.snippets_test.SnippetsTest
testMethod=test_model_bigqueryio_xlang>
@pytest.mark.uses_gcp_java_expansion_service
@unittest.skipUnless(
os.environ.get('EXPANSION_PORT'),
"EXPANSION_PORT environment var is not provided.")
def test_model_bigqueryio_xlang(self):
p = TestPipeline()
p.options.view_as(GoogleCloudOptions).temp_location = 'gs://mylocation'
> snippets.model_bigqueryio_xlang(p)
[1m[31mapache_beam/examples/snippets/snippets_test.py[0m:768:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[1m[31mapache_beam/examples/snippets/snippets.py[0m:1065: in
model_bigqueryio_xlang
quotes | "StorageWriteAPIWithFrequency" >> beam.io.WriteToBigQuery(
[1m[31mapache_beam/pvalue.py[0m:137: in __or__
return self.pipeline.apply(ptransform, self)
[1m[31mapache_beam/pipeline.py[0m:655: in apply
return self.apply(
[1m[31mapache_beam/pipeline.py[0m:666: in apply
return self.apply(transform, pvalueish)
[1m[31mapache_beam/pipeline.py[0m:712: in apply
pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
[1m[31mapache_beam/runners/dataflow/dataflow_runner.py[0m:102: in apply
return super().apply(transform, input, options)
[1m[31mapache_beam/runners/runner.py[0m:203: in apply
return self.apply_PTransform(transform, input, options)
[1m[31mapache_beam/runners/runner.py[0m:207: in apply_PTransform
return transform.expand(input)
[1m[31mapache_beam/io/gcp/bigquery.py[0m:2241: in expand
| "StorageWriteToBigQuery" >> StorageWriteToBigQuery(
[1m[31mapache_beam/io/gcp/bigquery.py[0m:2540: in __init__
expansion_service or _default_io_expansion_service())
[1m[31mapache_beam/io/gcp/bigquery.py[0m:2480: in
_default_io_expansion_service
return BeamJarExpansionService(
[1m[31mapache_beam/transforms/external.py[0m:983: in __init__
path_to_jar = subprocess_server.JavaJarServer.path_to_beam_jar(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cls = <class 'apache_beam.utils.subprocess_server.JavaJarServer'>
gradle_target = 'sdks:java:io:google-cloud-platform:expansion-service:build'
appendix = None, version = '2.51.0.dev'
artifact_id = 'beam-sdks-java-io-google-cloud-platform-expansion-service'
@classmethod
def path_to_beam_jar(
cls,
gradle_target,
appendix=None,
version=beam_version,
artifact_id=None):
if gradle_target in cls._BEAM_SERVICES.replacements:
return cls._BEAM_SERVICES.replacements[gradle_target]
gradle_package = gradle_target.strip(':').rsplit(':', 1)[0]
if not artifact_id:
artifact_id = 'beam-' + gradle_package.replace(':', '-')
project_root = os.path.sep.join(
os.path.abspath(__file__).split(os.path.sep)[:-5])
local_path = os.path.join(
project_root,
gradle_package.replace(':', os.path.sep),
'build',
'libs',
cls.jar_name(
artifact_id,
version.replace('.dev', ''),
classifier='SNAPSHOT',
appendix=appendix))
if os.path.exists(local_path):
_LOGGER.info('Using pre-built snapshot at %s', local_path)
return local_path
elif '.dev' in version:
# TODO: Attempt to use nightly snapshots?
> raise RuntimeError(
(
'%s not found. '
'Please build the server with \n cd %s; ./gradlew %s') %
(local_path, os.path.abspath(project_root), gradle_target))
[1m[31mE RuntimeError:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar>
not found. Please build the server with [0m
[1m[31mE cd
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;>
./gradlew sdks:java:io:google-cloud-platform:expansion-service:build[0m
[1m[31mapache_beam/utils/subprocess_server.py[0m:243: RuntimeError
[33m=============================== warnings summary
===============================[0m
../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py>:28:
DeprecationWarning: the imp module is deprecated in favour of importlib and
slated for removal in Python 3.12; see the module's documentation for
alternative uses
from imp import load_source
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/rpc/__init__.py:18
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/google/rpc/__init__.py>:18:
DeprecationWarning: pkg_resources is deprecated as an API. See
https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2871:
19 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py>:2871:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2871:
16 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py>:2871:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.cloud')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2350
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py>:2350:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(parent)
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2871
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py>:2871:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.logging')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2871
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py>:2871:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.iam')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/rpc/__init__.py:20
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/google/rpc/__init__.py>:20:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.rpc')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
pkg_resources.declare_namespace(__name__)
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py>:17:
DeprecationWarning: The distutils package is deprecated and slated for removal
in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
from distutils import util
apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/examples/snippets/snippets_test.py>:767:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location = 'gs://mylocation'
apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_all_types
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_nested_records_and_lists
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_at_least_once
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_auto_sharding
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_with_at_least_once_semantics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2101:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_all_types
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_nested_records_and_lists
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_at_least_once
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_auto_sharding
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_with_at_least_once_semantics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2107:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/external/xlang_bigqueryio_it_test.py: 10 warnings
apache_beam/io/gcp/bigtableio_it_test.py: 6 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/transforms/external.py>:679:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
self._expansion_service, pipeline.options)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/pytest_gcpCrossLanguage.xml>
-
[36m[1m=========================== short test summary info
============================[0m
[31mFAILED[0m
apache_beam/examples/snippets/snippets_test.py::[1mSnippetsTest::test_model_bigqueryio_xlang[0m
- RuntimeError:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar>
not found. Please build the server with
cd
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;>
./gradlew sdks:java:io:google-cloud-platform:expansion-service:build
[31m= [31m[1m1 failed[0m, [32m13 passed[0m, [33m17 skipped[0m,
[33m7149 deselected[0m, [33m76 warnings[0m[31m in 6341.36s
(1:45:41)[0m[31m =[0m
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava
> FAILED
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguageCleanup
Stopping expansion service pid: 2802757.
Skipping invalid pid: 2802758.
> Task :sdks:python:test-suites:xlang:fnApiJobServerCleanup
Killing process at 2802146
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 1h 51m 43s
119 actionable tasks: 83 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/2fei56be65dee
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
