See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/5197/display/redirect?page=changes>
Changes: [kevinsijo] Setting up a basic directory [kevinsijo] Mirroring Python SDK's directory structure [kerrydc] Adds initial tests [kevinsijo] 'runners' is the correct directory name [Pablo Estrada] sketching the core API for JS SDK [jonathanlui] add .gitignore for node/ts project [Robert Bradshaw] Worker directory. [Robert Bradshaw] Fix complile errors with explicit any for callables. [Robert Bradshaw] Add worker entry point. [Robert Bradshaw] Add proto generation code. [Robert Bradshaw] Add generated proto files. [Robert Bradshaw] Attempts to get ts protos to compile. [Robert Bradshaw] Exclude ts protos for now. [Robert Bradshaw] More changes to get ts protos working. [Robert Bradshaw] Update scripts and config to get protos compiling. [Robert Bradshaw] Update geenrated files. [jonathanlui] add build and clean script to compile ts [Robert Bradshaw] Generate server for loopback worker. [Robert Bradshaw] Generated grpc servers for loopback. [Robert Bradshaw] Add typescript formatter. [Robert Bradshaw] Loopback server (that does nothing). [Robert Bradshaw] Working server. [Pablo Estrada] Starting expansion of primitive transforms [Pablo Estrada] Starting to implement and support standard coders [Robert Bradshaw] Also generate grpc clients. [Robert Bradshaw] Basic implementation of worker harness. [Pablo Estrada] fix the build [Robert Bradshaw] Add some missing files for worker harness. [Robert Bradshaw] Refactor operators to use registration. [jonathanlui] enable ts in mocha [jonathanlui] update readme [jonathanlui] --save-dev @types/mocha [jonathanlui] translate core_test.js to typescript [Robert Bradshaw] Encapsulate worker service in a class. [Kenneth Knowles] Port standard_coders_test to typescript (superficially) [Pablo Estrada] Starting the proto translation of Impulse, ParDo, GBK [Robert Bradshaw] Add some tests for the worker code. [Robert Bradshaw] Fixing old lock file error. [Pablo Estrada] Adding transform names and fixing GBK coder issue [Robert Bradshaw] npx tsfmt -r src/apache_beam/base.ts src/apache_beam/transforms/core.ts [Kenneth Knowles] switch to import style require() statements [Kenneth Knowles] Add Coder interface using protobufjs classes [Kenneth Knowles] BytesCoder with some failures [noreply] Added GeneralObjectCoder and using it as coder for most transforms (#9) [Kenneth Knowles] Fix order of arguments to deepEqual [Kenneth Knowles] Encode expected encoding as binary [Robert Bradshaw] Refactor API to allow for composites. [jrmccluskey] Initial setup for automated Java expansion startup [jrmccluskey] Update exp_service.ts [Kenneth Knowles] Fix up coder deserialization [Robert Bradshaw] Simplify GBK coder computation. [Robert Bradshaw] Remove top-level PValue. [Pablo Estrada] Make tests green [Robert Bradshaw] Rename PValueish to PValue. [jonathanlui] node runner [jonathanlui] whitespaces [Robert Bradshaw] Make Runner.run async. [jonathanlui] bson and fast-deep-equal should not be listed as devdependency [jrmccluskey] Add basic Dockerfile that starts ExternalWorkerPool [Robert Bradshaw] Direct runner. [kevinsijo] Testing expansion service communication [Robert Bradshaw] Added flatten, assertion checkers. [Pablo Estrada] progress on basic coders [Robert Bradshaw] Fixing the build. [Robert Bradshaw] Cleanup, simplify access. [Pablo Estrada] Adding limited support for KVCoder and IterableCoder [Robert Bradshaw] Introduce PipelineContext. [Robert Bradshaw] Add toProto to all coders. [Robert Bradshaw] Some work with coders. [Robert Bradshaw] Remove debug logging. [Robert Bradshaw] Use coders over data channel. [Kenneth Knowles] explicitly sequence sub-coder serializations [Kenneth Knowles] no more need to extend FakeCoder [Kenneth Knowles] actually advance reader [Kenneth Knowles] autoformat [Kenneth Knowles] protobufjs already can write and read signed varints [Kenneth Knowles] with improved test harness, kv has many more failures [Kenneth Knowles] read bytescoder from correct position [Kenneth Knowles] no more fake coders [Kenneth Knowles] varint examples all work [Kenneth Knowles] simplify coder value parsing [Kenneth Knowles] global window coder [Kenneth Knowles] fix swapEndian32 [Robert Bradshaw] Add P(...) operator. [kevinsijo] Implementing RowCoder encoding. [jrmccluskey] remove unused container dir [kevinsijo] Corrected sorting of encoded positions to reflect an argsort instead. [Robert Bradshaw] Populate environments. [kevinsijo] Implementing RowCoder decoding. [Kenneth Knowles] preliminary unbounded iterable coder [Kenneth Knowles] friendlier description of standard coder test case [Kenneth Knowles] fix test harness; iterable works [jrmccluskey] first pass at boot.go [jonathanlui] update package-lock.json [jonathanlui] make NodeRunner a subclass of Runner [jonathanlui] add waitUntilFinish interface member [Pablo Estrada] Adding double coder [Kenneth Knowles] scaffolding for windowed values [Pablo Estrada] Adding type information to PColleciton and PTransform [jonathanlui] fix direct runner [Pablo Estrada] Adding typing information for DoFns [Kenneth Knowles] add interval window [Robert Bradshaw] Export PValue. [Robert Bradshaw] Add CombineFn interface. [Robert Bradshaw] Typed flatten. [jonathanlui] add runAsync method to base.Runner [Kenneth Knowles] add Long package [Pablo Estrada] Adding more types. Making PValue typed [Kenneth Knowles] instant coder draft [Robert Bradshaw] Return job state from direct runner. [Kenneth Knowles] type instant = long [jonathanlui] implement NodeRunner.runPipeline [Kenneth Knowles] autoformat [kevinsijo] Completed implementation of basic row coder [Kenneth Knowles] Fix IntervalWindowCoder, almost [Kenneth Knowles] fix interval window coder [Kenneth Knowles] autoformat [Robert Bradshaw] loopback runner works [Kenneth Knowles] move core element types into values.ts [Kenneth Knowles] just build object directly to be cool [Robert Bradshaw] GBK working on ULR. [Robert Bradshaw] Async transforms. [Robert Bradshaw] External transform grpah splicing. [Kenneth Knowles] progress on windowed value: paneinfo encoding [Robert Bradshaw] Fix merge. [Robert Bradshaw] autoformat [Kenneth Knowles] full windowed value coder [kerrydc] Updates tests to use correct types, adds generics where needed to DoFns [Robert Bradshaw] Add serialization librarires.' [Robert Bradshaw] Add Split() PTransform, for producing multiple outputs from a single [Robert Bradshaw] Schema-encoded external payloads. [kevinsijo] Adding Schema inference from JSON [Pablo Estrada] Removing unused directories [Pablo Estrada] Support for finishBundle and improving typing annotations. [Pablo Estrada] A base implementation of combiners with GBK/ParDo [Robert Bradshaw] Fully propagate windowing information in both remote and direct runner. [Robert Bradshaw] Make args and kwargs optional for python external transform. [Robert Bradshaw] Infer schema for external transforms. [Pablo Estrada] Implementing a custom combine fn as an example. Small fixes [Robert Bradshaw] Fix missing windowing information in combiners. [Robert Bradshaw] PostShuffle needn't group by key as that's already done. [Robert Bradshaw] Guard pre-combine for global window only. [Robert Bradshaw] WindowInto [Robert Bradshaw] Fix optional kwargs. [Robert Bradshaw] A couple of tweaks for js + py [Robert Bradshaw] Add windowing file. [Robert Bradshaw] CombineBy transform, stand-alone WordCount. [Robert Bradshaw] cleanup [Robert Bradshaw] Actually fix optional external kwargs. [Robert Bradshaw] Demo2, textio read. [Robert Bradshaw] Add command lines for starting up the servers. [Robert Bradshaw] Run prettier on the full codebase. [Robert Bradshaw] Update deps. [Pablo Estrada] Adding docstrings for core.ts. Prettier dependency [Pablo Estrada] Documenting coder interfaces [Pablo Estrada] Added documentation for a few standard coders [Robert Bradshaw] Unified grouping and combining. [Robert Bradshaw] Allow PCollection ids to be lazy. [Robert Bradshaw] Reorganize module structure. [Robert Bradshaw] A couple more renames. [Robert Bradshaw] Simplify. [Robert Bradshaw] Consolidation. [Robert Bradshaw] Fix build. [Robert Bradshaw] Add optional context to ParDo. [Robert Bradshaw] fixup: iterable coder endian sign issue [Robert Bradshaw] omit context for map(console.log) [Robert Bradshaw] Fix ReadFromText coders. [Robert Bradshaw] Flesh out README with overview and current state. [noreply] Readme typo [Robert Bradshaw] Two more TODOs. [noreply] Add a pointer to the example wordcount to the readme. [Pablo Estrada] Documenting coders and implementing unknown-length method [Robert Bradshaw] UIID dependency. [Robert Bradshaw] Artifact handling. [Robert Bradshaw] Properly wait on data channel for bundle completion. [Robert Bradshaw] Automatic java expansion service startup. [Robert Bradshaw] Process promises. [Robert Bradshaw] Implement side inputs. [Robert Bradshaw] Cleanup. [Robert Bradshaw] Put complex constext stuff in its own file. [Robert Bradshaw] Rename BoundedWindow to just Window. [Robert Bradshaw] Alternative splitter class. [Pablo Estrada] Documenting internal functions [Robert Bradshaw] Take a pass clarifying the TODOs. [Robert Bradshaw] Sql transform wrapper. [Robert Bradshaw] Incorporate some feedback into the TODOs. [Robert Bradshaw] More TODOs. [Robert Bradshaw] Remove app placeholder. [Robert Bradshaw] Apache license headers. [Robert Bradshaw] More TODOs [jankuehle] Suggestions for TypeScript todos [dannymccormick] Add actions for typescript sdk [dannymccormick] Fix test command [noreply] Add missing version [dannymccormick] Fix codecovTest command [noreply] Only do prettier check on linux [noreply] Only get codecov on linux [Robert Bradshaw] Resolve some comments. [Robert Bradshaw] Fix compile errors. [Robert Bradshaw] Prettier. [Robert Bradshaw] Re-order expandInternal arguments pending unification. [Robert Bradshaw] More consistent and stricter PTransform naming. [Robert Bradshaw] Notes on explicit, if less idiomatic, use of classes. [Robert Bradshaw] Let DoFn be an interface rather than a class. [Robert Bradshaw] Provide DoFn context to start and finish bundle. [Robert Bradshaw] Optional promise code simplification. [Robert Bradshaw] Cleanup todos. [Robert Bradshaw] Avoid any type where not needed. [Robert Bradshaw] Apache RAT excludes for typescript. [Robert Bradshaw] Remove empty READMEs. [Robert Bradshaw] Add licences statement to readme files. [Robert Bradshaw] More RAT fixes. [Robert Bradshaw] Another unsupported coder. [Robert Bradshaw] Remove debugging code. [noreply] Fix automatic naming with code coverage. [Robert Bradshaw] Coders cleanup. [Robert Bradshaw] Add tests for RowCoder. [Robert Bradshaw] Normalize capitalization, comments. [Robert Bradshaw] Install typescript closure packages. [Robert Bradshaw] npm audit fix [Robert Bradshaw] Move more imports out of base. [Robert Bradshaw] Changes needed to compile with ts closure plugin. [Robert Bradshaw] Use ttsc and ts-closure-transform plugin. [Robert Bradshaw] Serialization registration to actually get serialization working. [Robert Bradshaw] Container images working on local runner. [Robert Bradshaw] Add a portable job server that proxies the Dataflow backend. (#17189) [Robert Bradshaw] Improvements to dataflow job service for non-Python jobs. [Robert Bradshaw] Get dataflow working. [Robert Bradshaw] User friendly pipeline options. [Robert Bradshaw] Less classes, more functions. [Robert Bradshaw] Add new nullable standard coder. [Robert Bradshaw] Make Apache Rat happy. [Robert Bradshaw] Disable broken codecov. [Robert Bradshaw] Remove last uses of base.ts. [Robert Bradshaw] Remove unneedd file. [Robert Bradshaw] Remove more uneeded/unused files. [Robert Bradshaw] Cleanup tests. [Robert Bradshaw] Minor cleanups to coder tests. [Robert Bradshaw] Addressing issues from the review. [noreply] Apply suggestions from code review. [Robert Bradshaw] Post-merge fixes. [dannymccormick] Delete tags.go [Robert Bradshaw] Update tests to use our actual serialization libraries. [Robert Bradshaw] Another pass at TODOs, removing finished items. [Heejong Lee] [BEAM-14146] Python Streaming job failing to drain with BigQueryIO write [Kenneth Knowles] Add parameter for service account impersonation in GCP credentials [noreply] Merge pull request #17490 from [BEAM-14370] [Website] Add new page about [noreply] [BEAM-14332] Refactored cluster management for Flink on Dataproc ------------------------------------------ [...truncated 57.04 MB...] apache_beam/io/gcp/bigquery.py:2150 apache_beam/io/gcp/bigquery.py:2150 apache_beam/io/gcp/bigquery.py:2150 apache_beam/io/gcp/bigquery.py:2150 apache_beam/io/gcp/bigquery.py:2150 apache_beam/io/gcp/bigquery.py:2150 apache_beam/io/gcp/bigquery.py:2150 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2150: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported self.table_reference.projectId = pcoll.pipeline.options.view_as( apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 apache_beam/io/gcp/bigquery_file_loads.py:1128 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = p.options.view_as(GoogleCloudOptions).temp_location apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 apache_beam/io/gcp/bigquery_file_loads.py:1130 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME') apache_beam/io/gcp/tests/utils.py:100 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead. table_ref = client.dataset(dataset_id).table(table_id) apache_beam/io/gcp/bigquery.py:2515 apache_beam/io/gcp/bigquery.py:2515 apache_beam/io/gcp/bigquery.py:2515 apache_beam/io/gcp/bigquery.py:2515 apache_beam/io/gcp/bigquery.py:2515 apache_beam/io/gcp/bigquery.py:2515 apache_beam/io/gcp/bigquery.py:2515 apache_beam/io/gcp/bigquery.py:2515 apache_beam/io/gcp/bigquery.py:2515 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2515: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported project_id = pcoll.pipeline.options.view_as(GoogleCloudOptions).project apache_beam/io/gcp/bigquery.py:2543 apache_beam/io/gcp/bigquery.py:2543 apache_beam/io/gcp/bigquery.py:2543 apache_beam/io/gcp/bigquery.py:2543 apache_beam/io/gcp/bigquery.py:2543 apache_beam/io/gcp/bigquery.py:2543 apache_beam/io/gcp/bigquery.py:2543 apache_beam/io/gcp/bigquery.py:2543 apache_beam/io/gcp/bigquery.py:2543 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2543: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported | _PassThroughThenCleanupTempDatasets(project_to_cleanup_pcoll)) apache_beam/dataframe/io.py:632 apache_beam/dataframe/io.py:632 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental. sink=lambda _: _WriteToPandasFileSink( apache_beam/io/fileio.py:550 apache_beam/io/fileio.py:550 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported p.options.view_as(GoogleCloudOptions).temp_location or apache_beam/io/gcp/bigquery_read_it_test.py:169 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:169: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead. beam.io.BigQuerySource(query=self.query, use_standard_sql=True))) apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84 apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead. kms_key=kms_key)) apache_beam/runners/dataflow/ptransform_overrides.py:323 apache_beam/runners/dataflow/ptransform_overrides.py:323 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead. kms_key=self.kms_key)) apache_beam/ml/gcp/cloud_dlp_it_test.py:77 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental. inspection_config=INSPECT_CONFIG)) apache_beam/ml/gcp/cloud_dlp_it_test.py:87 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental. | beam.ParDo(extract_inspection_results).with_outputs( apache_beam/io/gcp/bigquery_read_it_test.py:566 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:566: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead. beam.io.BigQuerySource(query=self.query, use_standard_sql=True))) apache_beam/io/gcp/bigquery_read_it_test.py:680 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:680: FutureWarning: ReadAllFromBigQuery is experimental. | beam.io.ReadAllFromBigQuery()) apache_beam/io/gcp/bigquery.py:2645 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2645: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name apache_beam/io/gcp/bigquery.py:2646 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2646: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project apache_beam/io/gcp/bigquery.py:2659 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2659: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported options=pcoll.pipeline.options, -- Docs: https://docs.pytest.org/en/latest/warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> - [31m[1m======= 1 failed, 73 passed, 8 skipped, 196 warnings in 7267.44 seconds ========[0m > Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED > Task :sdks:python:test-suites:dataflow:py37:spannerioIT >>> RUNNING integration tests with pipeline options: >>> --runner=TestDataflowRunner --project=apache-beam-testing >>> --region=us-central1 >>> --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it >>> --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it >>> --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output >>> --sdk_location=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/build/apache-beam.tar.gz> >>> --requirements_file=postcommit_requirements.txt --num_workers=1 >>> --sleep_secs=20 >>> --dataflow_worker_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-2.40.0-SNAPSHOT.jar> >>> >>> --kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test >>> >>> --dataflow_kms_key=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test >>> pytest options: --capture=no --timeout=4500 --color=yes >>> --log-cli-level=INFO --numprocesses=8 --dist=loadfile >>> collect markers: -m=spannerio_it [1m============================= test session starts ==============================[0m platform linux -- Python 3.7.12, pytest-4.6.11, py-1.11.0, pluggy-0.13.1 rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python,> inifile: pytest.ini plugins: xdist-1.34.0, timeout-1.4.2, forked-1.4.0, requests-mock-1.9.3 timeout: 4500.0s timeout method: signal timeout func_only: False gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I [1m[gw0] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0] [0m[1m[gw1] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0] [0m[1m[gw3] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0] [0m[1m[gw5] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0] [0m[1m[gw4] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0] [0m[1m[gw7] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0] [0m[1m[gw2] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0] [0m[1m[gw6] Python 3.7.12 (default, Jan 15 2022, 18:42:10) -- [GCC 9.3.0] [0mgw0 [15] / gw1 [15] / gw2 [15] / gw3 [15] / gw4 [15] / gw5 [15] / gw6 [15] / gw7 [15] scheduling tests via LoadFileScheduling apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql [gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call [gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error [gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table [gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update [gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call [gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call [gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call [gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call [gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call [gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call [gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call [gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call [gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call [gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches [gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches [33m=============================== warnings summary ===============================[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees. sql="select * from Users") apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees. database_id=self.TEST_DATABASE)) apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees. columns=["UserId", "Key"]) apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees. database_id=self.TEST_DATABASE)) apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135 <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees. max_batch_size_bytes=250)) -- Docs: https://docs.pytest.org/en/latest/warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> - [33m[1m============= 5 passed, 11 skipped, 5 warnings in 1339.62 seconds ==============[0m FAILURE: Build failed with an exception. * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 121 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 2h 26m 16s 225 actionable tasks: 155 executed, 64 from cache, 6 up-to-date Publishing build scan... https://gradle.com/s/woislnu63jui2 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
