See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/650/display/redirect>
Changes: ------------------------------------------ [...truncated 642.18 KB...] apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_at_least_once apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_auto_sharding apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_with_at_least_once_semantics <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2107: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported experiments = p.options.view_as(DebugOptions).experiments or [] apache_beam/io/external/xlang_bigqueryio_it_test.py: 10 warnings apache_beam/io/gcp/bigtableio_it_test.py: 6 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/transforms/external.py>:679: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported self._expansion_service, pipeline.options) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/pytest_gcpCrossLanguage.xml> - [36m[1m=========================== short test summary info ============================[0m [31mFAILED[0m apache_beam/examples/snippets/snippets_test.py::[1mSnippetsTest::test_model_bigqueryio_xlang[0m - RuntimeError: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar> not found. Please build the server with cd <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;> ./gradlew sdks:java:io:google-cloud-platform:expansion-service:build [31m= [31m[1m1 failed[0m, [32m13 passed[0m, [33m17 skipped[0m, [33m7147 deselected[0m, [33m77 warnings[0m[31m in 5904.87s (1:38:24)[0m[31m =[0m > Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava > FAILED > Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguageCleanup Stopping expansion service pid: 1009063. Skipping invalid pid: 1009064. > Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-06T15:43:31.668Z: JOB_MESSAGE_BASIC: Worker pool stopped. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:154 Job 2023-09-06_08_37_17-12285574955782609231 is in state JOB_STATE_DONE [32mINFO [0m apache_beam.io.gcp.bigtableio_it_test:bigtableio_it_test.py:190 Deleting table [test-table-1694014617-d443d7] [32mPASSED[0m [1m------------------------------ live log teardown -------------------------------[0m [32mINFO [0m apache_beam.io.gcp.bigtableio_it_test:bigtableio_it_test.py:198 Deleting instance [bt-write-xlang-1694013017-2988aa] =================================== FAILURES =================================== [31m[1m___________________ SnippetsTest.test_model_bigqueryio_xlang ___________________[0m self = <apache_beam.examples.snippets.snippets_test.SnippetsTest testMethod=test_model_bigqueryio_xlang> @pytest.mark.uses_gcp_java_expansion_service @unittest.skipUnless( os.environ.get('EXPANSION_PORT'), "EXPANSION_PORT environment var is not provided.") def test_model_bigqueryio_xlang(self): p = TestPipeline() p.options.view_as(GoogleCloudOptions).temp_location = 'gs://mylocation' > snippets.model_bigqueryio_xlang(p) [1m[31mapache_beam/examples/snippets/snippets_test.py[0m:768: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [1m[31mapache_beam/examples/snippets/snippets.py[0m:1065: in model_bigqueryio_xlang quotes | "StorageWriteAPIWithFrequency" >> beam.io.WriteToBigQuery( [1m[31mapache_beam/pvalue.py[0m:137: in __or__ return self.pipeline.apply(ptransform, self) [1m[31mapache_beam/pipeline.py[0m:655: in apply return self.apply( [1m[31mapache_beam/pipeline.py[0m:666: in apply return self.apply(transform, pvalueish) [1m[31mapache_beam/pipeline.py[0m:712: in apply pvalueish_result = self.runner.apply(transform, pvalueish, self._options) [1m[31mapache_beam/runners/dataflow/dataflow_runner.py[0m:102: in apply return super().apply(transform, input, options) [1m[31mapache_beam/runners/runner.py[0m:203: in apply return self.apply_PTransform(transform, input, options) [1m[31mapache_beam/runners/runner.py[0m:207: in apply_PTransform return transform.expand(input) [1m[31mapache_beam/io/gcp/bigquery.py[0m:2241: in expand | "StorageWriteToBigQuery" >> StorageWriteToBigQuery( [1m[31mapache_beam/io/gcp/bigquery.py[0m:2540: in __init__ expansion_service or _default_io_expansion_service()) [1m[31mapache_beam/io/gcp/bigquery.py[0m:2480: in _default_io_expansion_service return BeamJarExpansionService( [1m[31mapache_beam/transforms/external.py[0m:983: in __init__ path_to_jar = subprocess_server.JavaJarServer.path_to_beam_jar( _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cls = <class 'apache_beam.utils.subprocess_server.JavaJarServer'> gradle_target = 'sdks:java:io:google-cloud-platform:expansion-service:build' appendix = None, version = '2.51.0.dev' artifact_id = 'beam-sdks-java-io-google-cloud-platform-expansion-service' @classmethod def path_to_beam_jar( cls, gradle_target, appendix=None, version=beam_version, artifact_id=None): if gradle_target in cls._BEAM_SERVICES.replacements: return cls._BEAM_SERVICES.replacements[gradle_target] gradle_package = gradle_target.strip(':').rsplit(':', 1)[0] if not artifact_id: artifact_id = 'beam-' + gradle_package.replace(':', '-') project_root = os.path.sep.join( os.path.abspath(__file__).split(os.path.sep)[:-5]) local_path = os.path.join( project_root, gradle_package.replace(':', os.path.sep), 'build', 'libs', cls.jar_name( artifact_id, version.replace('.dev', ''), classifier='SNAPSHOT', appendix=appendix)) if os.path.exists(local_path): _LOGGER.info('Using pre-built snapshot at %s', local_path) return local_path elif '.dev' in version: # TODO: Attempt to use nightly snapshots? > raise RuntimeError( ( '%s not found. ' 'Please build the server with \n cd %s; ./gradlew %s') % (local_path, os.path.abspath(project_root), gradle_target)) [1m[31mE RuntimeError: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar> not found. Please build the server with [0m [1m[31mE cd <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;> ./gradlew sdks:java:io:google-cloud-platform:expansion-service:build[0m [1m[31mapache_beam/utils/subprocess_server.py[0m:243: RuntimeError [33m=============================== warnings summary ===============================[0m ../../build/gradleenv/-1734967051/lib/python3.8/site-packages/hdfs/config.py:28 <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/hdfs/config.py>:28: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses from imp import load_source ../../build/gradleenv/-1734967051/lib/python3.8/site-packages/google/rpc/__init__.py:18 <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/google/rpc/__init__.py>:18: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html import pkg_resources ../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2871: 20 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) ../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2871: 16 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.cloud')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) ../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2350 ../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2350 ../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2350 ../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2350 <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2350: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(parent) ../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2871 <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.logging')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) ../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2871 <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.iam')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) ../../build/gradleenv/-1734967051/lib/python3.8/site-packages/google/rpc/__init__.py:20 <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/google/rpc/__init__.py>:20: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.rpc')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages pkg_resources.declare_namespace(__name__) apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/examples/snippets/snippets_test.py>:767: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported p.options.view_as(GoogleCloudOptions).temp_location = 'gs://mylocation' apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_all_types apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_nested_records_and_lists apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_at_least_once apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_auto_sharding apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_with_at_least_once_semantics <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2101: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported is_streaming_pipeline = p.options.view_as(StandardOptions).streaming apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_all_types apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_nested_records_and_lists apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_at_least_once apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_auto_sharding apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_with_at_least_once_semantics <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2107: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported experiments = p.options.view_as(DebugOptions).experiments or [] apache_beam/io/external/xlang_bigqueryio_it_test.py: 10 warnings apache_beam/io/gcp/bigtableio_it_test.py: 6 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/transforms/external.py>:679: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported self._expansion_service, pipeline.options) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/pytest_gcpCrossLanguage.xml> - [36m[1m=========================== short test summary info ============================[0m [31mFAILED[0m apache_beam/examples/snippets/snippets_test.py::[1mSnippetsTest::test_model_bigqueryio_xlang[0m - RuntimeError: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar> not found. Please build the server with cd <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;> ./gradlew sdks:java:io:google-cloud-platform:expansion-service:build [31m= [31m[1m1 failed[0m, [32m13 passed[0m, [33m17 skipped[0m, [33m7147 deselected[0m, [33m76 warnings[0m[31m in 5721.18s (1:35:21)[0m[31m =[0m > Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava > FAILED > Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguageCleanup Stopping expansion service pid: 1009814. Skipping invalid pid: 1009815. > Task :sdks:python:test-suites:xlang:fnApiJobServerCleanup Killing process at 1008398 FAILURE: Build completed with 2 failures. 1: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. ============================================================================== 2: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 1h 45m 23s 119 actionable tasks: 83 executed, 32 from cache, 4 up-to-date Publishing build scan... https://ge.apache.org/s/2ndwewkulfpgu Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
