See <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Flink_Streaming/679/display/redirect?page=changes>
Changes: [aydar.zaynutdinov] [BEAM-13329][Playground] [aydar.zaynutdinov] [BEAM-13329][Playground] [aydar.zaynutdinov] [BEAM-13360][Playground] [noreply] Merge pull request #16011 from [BEAM-12164] Add models for Spanner [noreply] [BEAM-13335] Use signed int64 range for DataFrame read unique indexes [iyi] Add test case for untriggered expansion with temp tables. [zyichi] Fix small typo in ProcessBundleHandlerTest.java [noreply] Merge pull request #16079 from [BEAM-13241] [Playground] Frontend [noreply] Ricardo Case Study (#16087) [noreply] Merge pull request #16086 from [BEAM-13258][Playground] Get and store [noreply] [BEAM-11936] Fix errorprone UnusedVariable in Runners (#16035) [noreply] [BEAM-8123] Add cloudpickle as optional library (#15472) [noreply] [BEAM-12572] Beam python examples continuously exercised on at least 2 [alexander.zhuravlev] [BEAM-13370] Deleted unused prints & strings [ilya.kozyrev] Fix pylint issues and apply yapf with Beam config [stranniknm] [BEAM-13112]: playground embedded version [aydar.zaynutdinov] [BEAM-13363][Playground] [ilya.kozyrev] fix white spaces [noreply] Don't pin a particular version of Tensorflow. (#16102) [noreply] [BEAM-12733] Fix failing integration tests for Java Recommendation AI [marco.robles] Update README with latest PreCommit Jobs [noreply] [BEAM-13288] improve logging for no rows present error (#16096) [marco.robles] Update Postcommit jobs with latest jobs [marco.robles] Update Performace job tests in readme [marco.robles] update load job tests with latest updates [marco.robles] update other jobs test with latest updates [marco.robles] mismatch links fix [marco.robles] update trigger phrase for some postCommit jobs [aydar.zaynutdinov] [BEAM-13357][Playground] [thiagotnunes] [BEAM-12164] Add Spanner Change Stream DAOs [aydar.zaynutdinov] [BEAM-13357][Playground] [noreply] Merge pull request #16117 from [BEAM-13368][Playground][Bugfix] Fix CI [noreply] Run python SpannerIO IT with python 3.7 only to avoid overload spanner [noreply] Merge pull request #15378 from [RFC] Define and document per-key [noreply] Bump python containers to beam-master-20211202 (#16129) [Luke Cwik] [BEAM-13015] Simplify fake state API testing client and make it [noreply] [BEAM-13354, BEAM-13015, BEAM-12802, BEAM-12588] Support prefetch for [zyichi] Fix failing RecommendationAICatalogItemIT [Valentyn Tymofieiev] Clarify instructions on how to get contributor list. [noreply] [BEAM-11936] Fix errorprone UnusedVariable in IO (#16036) [zyichi] [BEAM-13380] Fix broken seedjob. [noreply] Merge pull request #16118 from [BEAM-13362][Playground] python ci cd [slebedev] [BEAM-13384] Re-exported the metrics subpackage in [aydar.zaynutdinov] [BEAM-13357][Playground] [noreply] Merge pull request #16137 from [BEAM-13381][Playground][Bugfix] Update [noreply] Merge pull request #16107 from [BEAM-13332] [Playground] Add logs output [noreply] Merge pull request #16109 from [BEAM-13330][Playground] Save Java logs [noreply] Merge pull request #16125 from [BEAM-13322][Playground] Support Java [zyichi] [BEAM-13388] Fix broken google cloud dlp test. [Robert Bradshaw] Better type hints for Count combiners. [melissapa] [BEAM-11758] Update basics page: Window, Watermark [relax] don't store unserializable DatasetService as a member variable [noreply] [BEAM-13236] Properly close kinesis producer on teardown (#15955) [noreply] Merge pull request #16150 from [BEAM-13396][Playground][Bugfix] Issues [noreply] Merge pull request #16148 from [BEAM-13394][Playground] [Bugfix] Fix [noreply] Avoid overriding explicit portable job submission disabling. (#16143) [noreply] Merge pull request #16151 from [BEAM-13350][Playground] Support running [zyichi] [BEAM-13373] Increase python post commit timeout to reduce chance of [zyichi] [BEAM-13401] Wait for worker start to reduce flakiness in [melissapa] [BEAM-11758] Final cleanup for Beam Basics doc content [msbukal] Exclude FhirIOPatientEverything from v2 dataflow runner intg test. [noreply] [BEAM-13371] Fix bug where DataFrame overview snippets don't show up [dpcollins] Add a workaround for https://github.com/googleapis/gax-java/issues/1577 [noreply] [BEAM-12976] Implement pipeline visitor to get global field access in… [noreply] [BEAM-13388] Use 3.0.0 as lower bound for google-cloud-dlp (#16164) [noreply] Merge pull request #16127 from [BEAM-13366] [Playground] Add support [stranniknm] fix playground frontend licences [noreply] Merge pull request #16167 from [BEAM-13409][Playground] [Bugfix] Change [noreply] Merge pull request #16136 from [BEAM-13365] [Playground] Add Pipelines [noreply] [BEAM-13244] Support STS Assume role credentials provider for AWS SDK v2 [noreply] [BEAM-11936] Fix errorprone UnusedVariable in core,examples,harness.. [marco.robles] correct trigger phrases in readme [noreply] [BEAM-11936] Remove suppressUnusedVariable flag (#16171) [zyichi] Fix pytest unknown markers warning. [noreply] [BEAM-13090] Adding SDK harness container overrides option to Java SDK [noreply] [BEAM-11936] Fix errorprone warnings (#15890) [Kyle Weaver] Include name of missing tag in error message. [noreply] [BEAM-13015] Start integrating a process wide cache. (#16130) [aydar.zaynutdinov] [BEAM-13424][Playground] [aydar.zaynutdinov] [BEAM-13424][Playground] [stranniknm] [BEAM-13426] clear log and output panels on example change [aydar.zaynutdinov] [BEAM-13428][Playground] [aydar.zaynutdinov] [BEAM-13424][Playground] [daria.malkova] add type of precompiled object [noreply] [BEAM-11936] Remove suppression in ReadFromKafkaDoFn (#16174) [noreply] [BEAM-12561] method truncate on series and dataframe (#15833) [noreply] [BEAM-13294] Widen key schema for all keys before use (#16158) [noreply] [BEAM-13171] Support for stopReadTime on KafkaIO SDF (#15951) [noreply] [BEAM-13399] Add functionality to download Beam JARs from Maven [noreply] [BEAM-13402] Add another workaround for [stranniknm] [BEAM-13423]: fix frontend failure if no examples [Alexey Romanenko] [BEAM-13171] Fix Duration ambiguity [daria.malkova] change return type of 2 methods [mmack] [BEAM-13441] Use quiet delete for S3 batch deletes. In quiet mode only [mmack] [BEAM-13410][BEAM-11788] Integration test for S3Filesystem using [Alexey Romanenko] [BEAM-13434] Bump up Apache log4j2 vulnerability to 2.15.0 [noreply] Merge pull request #16138 from [BEAM-13343][Playground] Support go unit [noreply] [BEAM-12560] Dataframe idxmin and idxmax implementation (#15827) [noreply] [BEAM-12565] Dataframe compare implementation (#16027) [noreply] [BEAM-12976] Use a map to pass all pushdown requests at once. (#16189) [noreply] Updating Grafana from v8.1.2 to v8.1.6 [noreply] [BEAM-12683] Fix failing integration tests for Python Recommendation AI [noreply] [BEAM-13015] Plumb the cache through contexts and transform executors. [dpcollins] [BEAM-13402] Simplify PubsubLiteSink [daria.malkova] Docs for validators tests [daria.malkova] change context type [noreply] Merge pull request #16140 from [BEAM-13377][Playground] Update CI/CD [noreply] Merge pull request #16120 from [BEAM-13333][Playground] Save Python logs [noreply] Merge pull request #16185 from [BEAM-13425][Playground][Bugfix] Support [mmack] [BEAM-13445] Correctly set data limit when flushing S3 upload buffer and [noreply] Merge pull request #16121 from [BEAM-13334][Playground] Save Go logs to [aydar.zaynutdinov] [BEAM-13438][Playground] [noreply] Merge pull request #16179 from [BEAM-13344][Playground] support python [noreply] Merge pull request #16208 from [BEAM-13442][Playground] Filepath to log [stranniknm] [BEAM-13446]: fix incorrect tab symbol [noreply] [BEAM-13276] bump jackson-core to 2.13.0 for .test-infra (#16062) [noreply] Change Pub/Sub Lite PollResult to set explicit watermark (#16216) [noreply] Removes the comment that seems no longer relevant. [noreply] [BEAM-13454] Fix and test dataframe read_fwf. (#16064) [Valentyn Tymofieiev] Update a few dependencies that may depend on log4j transitively. [noreply] [BEAM-12976] Pipeline visitor to discover pushdown opportunities. [noreply] [BEAM-13015] Allow decoding a set of elements until we hit the block [noreply] [BEAM-13159] Add Redis Stream (XADD) Write Support (#15858) [noreply] Merge pull request #15994: [BEAM-13263] Support OnWindowExpiration in [noreply] Bump dataflow container version to beam-master-20211213 (#16213) [noreply] Merge pull request #16198 from [BEAM-13437][Playground] Add [noreply] Merge pull request #16195 from a[BEAM-13436][Playground] Add [noreply] [BEAM-13355] add Big Query parameter to enable users to specify load_… [noreply] Updated the base images to use debian:bullseye (#16221) [noreply] [BEAM-13434] Bump log4j to 2.16.0. (#16237) [mmack] [BEAM-13209] Fix DynamoDBIO.write to properly handle partial success [mmack] [BEAM-13209] Fix DynamoDBIO.write to properly handle partial success [stranniknm] [BEAM-13466]: sort categories and examples by name [noreply] [BEAM-13341] Serialize reflection parameter in AvroCoder cloud object [johnjcasey] [BEAM-12391] update avro sink to close the opened file handle, instead [noreply] Update grafana from 8.1.6 to 8.1.8 [noreply] [BEAM-13015] Update FakeBeamFnStateClient to generate elements that stop [noreply] [BEAM-13218] Sickbay [noreply] [BEAM-13399] Add infrastructure to start JARs from Go functions (#16214) [noreply] Merge pull request #16061 from [BEAM-13428] [Playground] Integrate [noreply] Clarify CoGroupByKey creates Iterable, not list. (#16099) [noreply] [BEAM-12931] Allow for DoFn#getAllowedTimestampSkew() when checking the [noreply] [BEAM-13467] Properly handle null argument types for logical types. [noreply] [BEAM-10277] Initial implementation for encoding position in Python [noreply] [BEAM-11545] State & timer for batched RPC calls pattern (#13643) [noreply] Automatically prune local images before building an RC. (#16238) [noreply] Add verbose error messages to container-related scripts. (#16056) [noreply] [BEAM-13456] Rollback #15890 to fix timeout in Java PostCommit (#16257) [noreply] [BEAM-13015] Add a state backed iterable that can be mutated under [noreply] [BEAM-13388] Update Cloud DLP after breaking changes. (#16236) [rsl4] [BEAM-13397] Bump numpy to 1.21 for M1 chip [noreply] [BEAM-13434] Bump google pubsublite on master. (#16265) [mmack] [adhoc] Fix guava imports in tests [aydar.zaynutdinov] [BEAM-13485][Playground] [Steve Niemitz] [BEAM-13488] Work around bug in classpath detection [alexander.zhuravlev] [BEAM-13476] Changed Timeout Error Text [rsl4] re-generate requirements.txt [alexander.zhuravlev] [BEAM-13474] Changed 'Playground' logo text color [mmack] [BEAM-13443] Avoid blocking put to Kinesis records queue to shutdown [noreply] Merge pull request #16278 from [BEAM-13479] [Playground] Change logic [noreply] Merge pull request #16281 from [BEAM-13475] [Playground] [Bugfix] Error [noreply] Merge pull request #16279 from [BEAM-13473] [Playground] [Bugfix] Reset [noreply] Merge pull request #16232 from [BEAM-13285][Playground] Add steps to [noreply] [BEAM-13399] Add check for dev versions of JARs to download code [noreply] Merge pull request #16122 from [BEAM-13345] [Playground] add resizable [benjamin.gonzalez] [BEAM-13088] Make tempDataset final [noreply] Merge pull request #16170 from [BEAM-13411][Playground] Add getting of [noreply] Merge pull request #16172 from [BEAM-13417] [Playground] Add java [noreply] Merge pull request #16240 from [BEAM-13465][Playground] Change error [noreply] Merge pull request #16234 from [BEAM-13461][Playground] [Bugfix] Error [noreply] Merge pull request #15489 from [BEAM-12865] Allow custom batch_duration [noreply] [BEAM-13483] Increase timeout of Java Examples Dataflow suite. (#16226) [Kyle Weaver] [BEAM-13496] Upgrade Flink runner to include log4j patches. [Kyle Weaver] [BEAM-13497] Correct class name in Flink tests. [noreply] Pin transitive log4j dependencies to 2.17.0 in :sdks:java:io:hcatalog [noreply] [BEAM-13434] Bump solr to 8.11.1 (#16296) [Kyle Weaver] [BEAM-13498] Sickbay portable Flink testProcessElementSkew. [noreply] Use a patched shadow 6.1.0 plugin using Log4j 2.16.0 (#16269) [noreply] [BEAM-12830] Replace GoGradle plugin with Shell Scripts. (#16291) [stranniknm] [BEAM-13502]: fix loading example on embedded version [noreply] [BEAM-13430] Start the process of upgrading the Gradle 7. (#16292) [noreply] [BEAM-13430] Introduce new testRuntimeMigration configuration that [noreply] Update flink cluster to use a supported dataproc version (1.2 -> 1.5) [Robert Bradshaw] Add microbenchmark for row coder. [noreply] [BEAM-13499] Add warning about hcatalog to release notes and javadoc [Robert Bradshaw] Compile RowCoderImpl with Cython. [Robert Bradshaw] RowCoder optimizations. [Robert Bradshaw] Further optimizations for common case of components in order. [noreply] [BEAM-13430] Remove propdeps and replace with compileOnly (#16308) [Robert Bradshaw] Optimize rows with null fields. [noreply] Merge pull request #16304 from [BEAM-13491] [Playground] Examples' [noreply] Merge pull request #16283 from [BEAM-13448] [Playground] track run code [noreply] Merge pull request #16244 from [BEAM-13463] [Playground] add retries to [noreply] Merge pull request #16241 from [BEAM-13440] [Playground] Implement [noreply] [BEAM-13399] Add integration test for Go SDK expansion service JAR [noreply] [BEAM-13421] Fix bug with xs called with non-tuple key (#16258) [Robert Bradshaw] RowCoder lint. [Robert Bradshaw] [BEAM-13482] Python fully qualified name external transforms. [Robert Bradshaw] Add Python expansion service entry point. [mmack] [BEAM-13494] Avoid draining Kinesis API limits & CPU when reading from [alexander.zhuravlev] [BEAM-13489] Updated manifest.json and index.html to follow best [arietis27] [BEAM-13517] Unable to write nulls to columns with logical types [stranniknm] [BEAM-13501] playground - add log about processing and precompiled run [noreply] Make S3 streaming more efficient (#15931) [noreply] [Playground][BEAM-13358]Deploy Go service via Ci/CD (#16210) [dpcollins] [BEAM-13430] Clean up tests that override the time of the JVM [noreply] [BEAM-12830] Install go version w/host platform. (#16330) [noreply] [BEAM-13399] Add extra time buffer to reduce likelihood of flake in [noreply] [BEAM-12865] Fix triggering_frequency validation (#16315) [noreply] [BEAM-13525] Sickbay new PardoTests for Dataflow V2 Java Streaming VR [noreply] [BEAM-12830] Avoid flock and prepare task once. (OSX fix.) (#16333) [noreply] Bump Python beam-master containers (#16332) [arietis27] [BEAM-13517] Unable to write nulls to columns with logical types [stranniknm] [BEAM-13531] playground - fetch example info only once [mmack] [BEAM-7559] Add integration test for DynamoDBIO (SDK v2) [mmack] [BEAM-7559] Add integration test for DynamoDBIO (SDK v1) [mmack] [adhoc] Forbid to import guava and others from org.testcontainers.shaded [alexander.zhuravlev] [BEAM-13530] Fixed scrollbar in the Output section [noreply] [BEAM-13399] Disable expansion service integration test on Samza and [noreply] [BEAM-13430] Upgrade beam to gradle 7.3.2 (#16319) [Kyle Weaver] [BEAM-13521] flink pvr batch precommit [noreply] Merge pull request #16320 from [BEAM-13512] [Playground] send request to [noreply] Merge pull request #16209 from [Playground][BEAM-13304]Deploy Python [noreply] Merge pull request #16105 from [BEAM-13328] [Playground] Add [noreply] Merge pull request #16103 from [BEAM-13252] [Playground] Feedback Thumbs [noreply] Better type inference for GroupBy. (#16318) [noreply] [BEAM-13534] Update TestAutomatedExpansionService liveness check to not [noreply] [BEAM-13490] Bump log4j version to 2.17.0 (#16276) [noreply] Merge pull request #16355 from [BEAM-13529] [Playground] [Bugfix] Fix [noreply] Fixed grammar error in Go ParDo function comments. (#16329) [noreply] Merge pull request #16285 from [BEAM-13492][Playground] Update backend [stranniknm] [BEAM-13555] playground - quick fix for safari [mmack] [BEAM-13009] Fix tests to ensure absence of duplicates only per request [noreply] [BEAM-13526] Kafka.IO: make DeserializerProvider a public interface [mmack] [BEAM-13509] Stop sharing SQS client between readers of same source. [Kyle Weaver] [BEAM-13571] Fix ClassNotFound exception in Flink tests [Kyle Weaver] [BEAM-13498] [BEAM-13573] exclude new tests on Flink [noreply] Exclude UsesOnWindowExpiration by category from Dataflow v2 streaming [noreply] [BEAM-13052] Increment pubsub python version and fix breakages. (#16126) [noreply] [BEAM-13052] Add Pub/Sub Lite xlang transforms in python (#15727) [noreply] [BEAM-13402] Version bump Pub/Sub Lite and implement changes to ensure [aydar.zaynutdinov] [BEAM-13565][Playground] [relax] update bom [noreply] [BEAM-13578] Validate that repeatable PipelineOptions need to be [mmack] [adhoc] Use predefined min/max combiners in SQS reader to gather stats [noreply] [BEAM-13430] Update comment over eclipse plugin. (#16387) [zyichi] [BEAM-13430] Fix broken ULR validates runner tests [noreply] Update venv creation commands in release scripts. (#16381) [Steve Niemitz] [BEAM-13459] Cache artifacts across job runs on python+dataflow [noreply] [BEAM-12777] Stable URL for current API docs (#16327) [noreply] [BEAM-13539] correct resource override dependencies (#16392) [Kyle Weaver] [BEAM-13581] Remove previous job name for Flink PVR precommit. [emilyye] Moving to 2.37.0-SNAPSHOT on master branch. [noreply] Release website update for Beam 2.35.0 (#16115) [noreply] [BEAM-13430] Re-enable checkerframework for the project excluding the [zyichi] Fix class not found in perf integration tests. [noreply] Update [noreply] [BEAM-13430] Fix class not found error for example integration tests and [noreply] [BEAM-13430] Re-enable dependency analysis for modules. (#16395) [Kyle Weaver] [BEAM-13569] Change Spark dependencies to implementation. [Kyle Weaver] remove redundant dependency [noreply] [BEAM-13484][Playground] Improve Terraform Scripts For Deploy [noreply] [BEAM-13541] More intelligent caching of CoGBK values. (#16354) [noreply] Merge pull request #15769 from [BEAM-13031] [Playground] Code editor - [noreply] [BEAM-13541] Fix spotbugs issue with DLS in tail (#16407) [noreply] [BEAM-13588] Add missing module for PVR tests and also sickbay known [noreply] [BEAM-13575] Sickbay test that is flaky to restore precommit test signal [noreply] [BEAM-12092] Bump jedis to version 3.8.0 (#16403) [noreply] [BEAM-13498, BEAM-13522] Sickbay broken test and test category for Spark [blais] python sdk examples: Fixed typo in wordcount example. [wuren] [BEAM-13591] Bump log4j2 version to 2.17.1 [Steve Niemitz] [BEAM-13459] Update CHANGES.md, add note about artifact caching for [mmack] [BEAM-13587] Attempt to load AWS region from default provider chain in [relax] Add Flink runner support for OrderedListState. This version reads the [noreply] Merge pull request #16404: [BEAM-13586] Fix NPE in [zyichi] Fix sdk_container_builder too many values to unpack error [noreply] [BEAM-13480] Sickbay PubSubIntegrationTest.test_streaming_data_only on [Kyle Weaver] remove redundant testImplementation dependencies [noreply] [BEAM-13430] Swap to use "mainClass" instead of "main" since it was [zyichi] Fix remaining failing perf IT tests. [noreply] [BEAM-13430] Replace deprecated "appendix" with "archiveAppendix" [noreply] [BEAM-13015] Add jamm as a java agent to the Java SDK harness container [noreply] [BEAM-13430] Partially revert [noreply] Merge pull request #15863 from [BEAM-13184] Autosharding for [noreply] [BEAM-11936] Enable FloatingPointAssertionWithinEpsilon errorprone check [noreply] [BEAM-11936] Enable LockNotBeforeTry errorprone check (#16259) [noreply] [BEAM-11936] Enable errorprone unused checks (#16262) [noreply] Add Nexmark Query 14 (#16337) [noreply] [BEAM-13015] Migrate all user state and side implementations to support [zyichi] Upgrade python library versions in base_image_requirements.txt [noreply] [BEAM-13015] Use 20% of memory when the maximum has been configured. [noreply] [BEAM-13567] Consolidate runner flag definition. (#16426) [noreply] [BEAM-13601] Don't cache Row types for a schema. (#16427) [noreply] [BEAM-13430] Re-enable checkerframework (#16429) [noreply] [BEAM-13430] Ensure that testRuntimeMigration depends on "default" [noreply] Merge pull request #16277 from [BEAM-13124][Playground] Create readiness [noreply] Merge pull request #16314 from [BEAM-13260][Playground]Implement setup [noreply] Merge pull request #16383 from [BEAM-13566][Playground] Add logic of [noreply] Merge pull request #16365 from [BEAM-13559][Playground] Remove tag in [noreply] Merge pull request #16360 from [BEAM-13546][Playground] Update nginx [noreply] Merge pull request #16192 from [BEAM-13395] [Playground] Tag katas [noreply] Merge pull request #16254 from [BEAM-13249][Playground] Security – Mock [ningkang0957] [BEAM-12879] Prevented missing permission from failing GCS I/O [noreply] Merge pull request #16347: fix: move connector to use v1 BigQuery [noreply] [BEAM-13603] Fix bug in apache_beam.utils.Shared (#16437) [noreply] [BEAM-10345] Add an import guard to support recent google-cloud-spanner [Heejong Lee] [BEAM-13091] Generate missing staged names from hash for Dataflow runner [Heejong Lee] add test [arietis27] [BEAM-13604] NPE while getting null from BigDecimal column [noreply] Fixed empty labels treated as wildcard when matching cache files [noreply] [BEAM-13570] Remove erroneous compileClasspath dependency. (#16438) [noreply] [BEAM-13015] Plumb through process wide and bundle cache through the [noreply] [BEAM-13015] Cache the state backed iterable used for large GBK results. [noreply] Fix formatting/alignment (#16443) [noreply] Merge pull request #16183 from [BEAM-13427] [Playground] show logs for [noreply] [BEAM-10277] re-write encoding position tests to declare schema protos [noreply] Update local_env_tests.yml (#16444) [noreply] [BEAM-13574] Filesystem abstraction Rename support (#16428) [noreply] [BEAM-13597] Setup Go in github actions (#16446) [ningkang0957] [BEAM-13602] Prevented metrics gathering from failing bigtable io [noreply] Merge pull request #16161 from [BEAM-12164] Add Spanner Partition [noreply] Merge pull request #16203 from [BEAM-12164] Add Spanner Change Stream [Alexey Romanenko] [adhoc] Fix BigTableIO description [Alexey Romanenko] [BEAM-8727] Bump software.amazon.awssdk to 2.17.106 [ningkang0957] make the code more pythonic [noreply] [BEAM-13015] Remove dead code now that all instances have migrated to [noreply] [BEAM-13386] Add RLock support for cloudpickle (#16250) [danthev] Fix overflow [relax] don't close streams in finalize [noreply] Loosen typing extensions bound [Alexey Romanenko] [BEAM-4868] Bump com.amazonaws to 1.12.135 [mmack] [BEAM-13243][BEAM-8374] Add support for missing PublishResponse fields [noreply] [BEAM-12092] Bump jedis to version 4.0.1 (#16287) [noreply] [BEAM-13534] Add automated port polling to expansion service runner if [noreply] Merge pull request #16344 from [BEAM-13536][Playground][Bugfix] CI step [noreply] Merge pull request #16359 from [BEAM-13545][Playground] Add [noreply] Merge pull request #16384 from [BEAM-13308] [Playground] Getting [noreply] Merge pull request #16306 from [BEAM-13447] [Playground] Add filling of [noreply] Merge pull request #16361 from [BEAM-13543][Playground] Add logic of [noreply] [BEAM-12562] Dataframe pipe implementation (#16256) [noreply] Merge pull request #16338 from [BEAM-13528][Playground] Add liveness [noreply] [BEAM-13626] Remap expanded outputs after merging. (#16471) [noreply] Merge pull request #16147 from [BEAM-13359] [Playground] Tag existing [noreply] [BEAM-3221] Improve documentation in model pipeline protos (#16474) [noreply] [BEAM-13614] Add OnWindowExpiration support to the Java SDK harness and [Robert Bradshaw] Optional args and kwargs for named external transforms. [noreply] Merge pull request #16156 from [BEAM-13391] Fix temporary file format in [mmack] [adhoc] Run spotlessApply on java examples to fix master [mmack] [BEAM-8806] Integration test for SqsIO [mmack] [BEAM-13631] Add deterministic SQS message coder to fix reading from SQS [noreply] [BEAM-13628] Update SideInputCache to use full Transform and [noreply] [BEAM-13432] Skip ExpansionService creation in Job Server (#16222) [noreply] [BEAM-13616] Initial files for vendored gRPC 1.43.2 (#16460) [mmack] [BEAM-13638] Datatype of timestamp fields in SqsMessage for AWS IOs for [aydar.zaynutdinov] [BEAM-13641][Playground] [noreply] Merge pull request #16469 from [BEAM-13623][Playground] [Bugfix] During [noreply] Merge pull request #16149 from [BEAM-13113] [Playground] playground [noreply] Merge pull request #16363 from [BEAM-13557] [Playground] show code [noreply] Merge pull request #16374 from [BEAM-13398][Playground] Split LifeCycle [noreply] [BEAM-13616][BEAM-13646] Update vendored calcite 1.28.0 with protobuf [chamikaramj] Adds several example multi-language Python pipelines [noreply] Merge pull request #16325 from [BEAM-13471] [Playground] Tag existing [noreply] [BEAM-13399] Move service liveness polling to Runner type (#16487) [daria.malkova] Split builder into several builder for each step of pipeline execution [mmack] [BEAM-8806] Integration test for SqsIO using Localstack [Valentyn Tymofieiev] Provide API to check whether a hint is known. [noreply] [BEAM-13480] Increase pipeline timeout for [noreply] Stronger typing inference for CoGBK. (#16465) [noreply] [BEAM-12464] Change ProtoSchemaTranslator beam schema creation to match [noreply] Introduce the notion of a JoinIndex for fewer shuffles. (#16101) [noreply] Merge pull request #16467 from [BEAM-12164]: SpannerIO [Valentyn Tymofieiev] [BEAM-12558] Fix doc typo. [noreply] Merge pull request #16385 from [BEAM-13535] [Playground] add cancel [noreply] Merge pull request #16485 from [BEAM-13486] [Playground] For unit tests [Heejong Lee] [BEAM-13455] Remove duplicated artifacts when using multiple [thiagotnunes] [BEAM-12164]: Add SDF for reading change stream records [tuyarer] [BEAM-13577] Beam Select's uniquifyNames function loses nullability of [arietis27] [BEAM-13400] JDBC IO does not support UUID and JSONB PostgreSQL types [noreply] [BEAM-12572] Run java examples on multiple runners (#16450) [jrmccluskey] [BEAM-10206] Resolve go vet errors in protox package [noreply] Merge pull request #16482 from [BEAM-13429][Playground] Add builder for [noreply] [BEAM-13590] Fix abc imports from collections (#15850) [jrmccluskey] Fix staticcheck errors in transforms directory [jrmccluskey] Remove unnecessary fmt.Sprintf() in partition.go [jrmccluskey] Replace bytes.Compare() with bytes.Equal() in test cases [jrmccluskey] Replace string(buf.Bytes()) with buf.String() in coder_test.go [jrmccluskey] Remove unnecessary blank identifier assignment in harness.go [jrmccluskey] fix capitalized error strings in expansionx [jrmccluskey] Clean up string cast of bytes in vet.go and corresponding tests [jrmccluskey] Remove unnecessary fmt call in universal.go [Robert Bradshaw] Remove tab from source. [noreply] Redirecting cross-language transforms content (#16504) [noreply] doc tweaks (#16498) [noreply] [BEAM-12621] - Update Jenkins VMs to modern Ubuntu version (#16457) [Robert Bradshaw] Bump beam container version. [noreply] [BEAM-13664] Fix Primitives hashing benchmark (#16523) [noreply] Merge pull request #16507: [BEAM-13137] Fixes ES utest size flakiness [noreply] Remove jcenter repositories from gradle configuration. (#16532) [sergey.kalinin] Update GH Actions to use proper variables names and proper triggers [noreply] [BEAM-13430] Remove jcenter which will no longer contain any updates. [noreply] [BEAM-13616] Update com.google.cloud:libraries-bom to 24.2.0 (#16509) [alexander.zhuravlev] [BEAM-13680] Fixed code_repository (added pipelineUuid to RunCodeResult [Robert Bradshaw] Also bump FnAPI container. [noreply] [BEAM-13616][BEAM-13645] Switch to vendored grpc 1.43.2 (#16543) [noreply] [BEAM-13616][BEAM-13646] Upgrade vendored calcite to 1.28.0:0.2 (#16544) [noreply] Merge pull request #16486 from [BEAM-13544][Playground] Add logs to [noreply] [BEAM-13683] Correct SQL transform schema, fix expansion address [noreply] Update walkthrough.md (#16512) [noreply] [BEAM-11808][BEAM-9879] Support aggregate functions with two arguments [noreply] Merge pull request #16506 from [BEAM-13652][Playground] Send examples' [noreply] Merge pull request #16322 from [BEAM-13407] [Playground] Preload fonts [noreply] [BEAM-13665] Make SpannerIO projectID optional again (#16547) [Carl Yeksigian] Cache bucket matcher regex in GcsPath [noreply] [BEAM-13015] Add state caching capability to be used as hint for runners [noreply] Merge pull request #16309: [BEAM-13503] Set a default value to [noreply] [BEAM-13015] Provide caching statistics in the status client. (#16495) [noreply] [BEAM-13611] Skip test_xlang_jdbc_write (#16554) [noreply] Merge pull request #16370 from [BEAM-13556] playground - color and [noreply] Merge pull request #16531 from [BEAM-13567] [playground] Handle run code [noreply] Merge pull request #16533 from [BEAM-13548] [Playground] Add example [noreply] Merge pull request #16519 from [BEAM-13639] [Playground] Add [noreply] Merge pull request #16518 from [BEAM-13619] [Playground] Add loading [benjamin.gonzalez] [BEAM-12572] Add DistinctExample test [benjamin.gonzalez] [BEAM-12572] Add DistinctExample test [dhuntsperger] edited README and comments in Python multi-lang pipes examples [noreply] Merge pull request #16243 from [Kiley Sok] Allow Java 17 to be used in SDK [Kiley Sok] add testing support [Kiley Sok] Add more testing support for java 17 [noreply] [BEAM-13683] Make cross-language SQL example pipeline (#16567) [noreply] [BEAM-13688] fixed type in BPG 4.5.3 window section (#16560) [noreply] Remove obsolete commands from Inventory job. (#16564) [Kiley Sok] workaround for jamm [Pablo Estrada] Revert "Merge pull request #15863 from [BEAM-13184] Autosharding for [Pablo Estrada] BEAM-13611 reactivating jdbcio xlang test [noreply] Disable logging for memoization test. (#16556) [Steve Niemitz] [BEAM-13689] Output token elements when BQ batch writes complete. [noreply] Merge pull request #16472: [BEAM-13697] Add SchemaFieldNumber annotation [noreply] Merge pull request #16373 from [BEAM-13515] [Playground] Hiding lines in [Heejong Lee] [BEAM-13093] Enable JavaUsingPython CrossLanguageValidateRunner test for [noreply] Merge pull request #16371 from [BEAM-13518][Playground] Beam Playground [noreply] Update Java FnAPI beam master (#16572) [noreply] [BEAM-13699] Replace fnv with maphash. (#16573) [noreply] [BEAM-13693] Bump [noreply] [BEAM-10206] Remove Fatalf calls in non-test goroutines for [ningkang0957] [BEAM-13687] Improved Spanner IO request count metrics [noreply] [BEAM-13430] Re-add provided configuration (#16552) [noreply] Merge pull request #16540 from [BEAM-13678][Playground]Update Github [noreply] Merge pull request #16546 from [BEAM-13661] [BEAM-13704] [Playground] [noreply] Merge pull request #16369 from [BEAM-13558] [Playground] Hide the Graph [noreply] [BEAM-10206] Add key for fields in wrapper (#16583) [noreply] Merge pull request #16530 from Adding JSON support in SpannerIO and [noreply] [BEAM-13685] Enable users to specify cache directory under Interactive [Kiley Sok] Add Jenkins test for Java 17 [Kiley Sok] Fix jvm hex and skip errorprone [Kiley Sok] Fix display data for anonymous classes [Kiley Sok] fix jpms tests [Heejong Lee] [BEAM-13716] Clear before creating a new virtual environment in [mmack] [BEAM-13653] Make SnsIO.write topicArn optional. If provided, validate [artur.khanin] Privacy policy update regarding Apache Beam Playground [noreply] [BEAM-10897] Update the fastavro lower bound due to an issue on Windows [noreply] [BEAM-13605] Update pandas_doctests_test denylists in preparation for [noreply] Merge pull request #16538 from [BEAM-13676][Playground][Bugfix]Build Of [noreply] Merge pull request #16582 from [BEAM-13711] [Playground] [Bugfix] Add [noreply] Merge pull request #16515 from [BEAM-13636] [Playground] Checking the [ningkang0957] [BEAM-13275] Removed the explicit selenium dependency from setup [Daniel Oliveira] [BEAM-13321] Fix exception with BigQuery StreamWriter TraceID. [noreply] [BEAM-10206] Deprecate unused shallow cloning functions (#16600) [noreply] Bump Dataflow container versions (#16602) [noreply] Improved multi-language pipelines section of the programming guide [Kiley Sok] skip zetasql [marcin.kuthan] Get rid of unnessecary logs for BigQuery streaming writes in [mmack] [BEAM-13510] Don't retry on invalid SQS receipt handles. [daria.malkova] Support SCIO SDK via sbt projects [mmack] [BEAM-8807] Add integration test for SnsIO.write (Sdk v1 & v2) [benjamin.gonzalez] [BEAM-12572] Add java examples tests [noreply] [BEAM-13736] Make lifting cache exact. (#16603) [noreply] Merge pull request #16565 from [BEAM-13692][Playground] Implement [noreply] Merge pull request #16502 from [BEAM-13650][Playground] Add link for [marco.robles] add pending jobs to readme [noreply] Update README.md [noreply] [BEAM-13310] remove call to get offset consumer config, which was rep… [benjamin.gonzalez] [BEAM-12572] Change tests collected by runner [samuelw] [BEAM-11648] Share thread pool across RetryManager instances. [aydar.zaynutdinov] [BEAM-13737][Playground] [dhuntsperger] added GitHub example references to Python multilang quickstart [mmack] [BEAM-13746] Fix deserialization of SSECustomerKey for AWS Sdk v2 [noreply] [BEAM-7928] Allow users to specify worker disk type for Dataflow runner [Pablo Estrada] Exclude per-key order tests on Twister2 runner [noreply] Merge pull request #16534 from [BEAM-13671][Playground] Add backend [noreply] [BEAM-13271] Bump errorprone to 2.10.0 (#16231) [noreply] [BEAM-13595] Don't load main session when cloudpickle is used. (#16589) [benjamin.gonzalez] [BEAM-12572] Fix PipelineOptions, skip tests for specific runners [Heejong Lee] Update readme for XVR tests [benjamin.gonzalez] [BEAM-12572] Skip failing tests [benjamin.gonzalez] [BEAM-12572] Fix checkstyle warnings [mmack] [adhoc] Test S3Options and AwsOptions for Sdk v2 [noreply] [BEAM-13537] Fix NPE in kafkatopubsub example (#16625) [noreply] [BEAM-13740] update java_tests.yml to remove setup-go, which is [Heejong Lee] Fix google3 import error [benjamin.gonzalez] [BEAM-12572] Skip tfidf tests on direct runner [rogelio.hernandez] [BEAM-13051] Pylint misplaced-bare-raise warning enabled [benjamin.gonzalez] [BEAM-12572] Fix PipelineOptions [noreply] [BEAM-12976] Implement Java projection pushdown optimizer. (#16513) [Robert Bradshaw] Make num-stages counter into an internal counter. [Kyle Weaver] [BEAM-13751] Don't block on gcloud when attempting to get default GCP [benjamin.gonzalez] [BEAM-12572] Fix checkstyle warning [Kyle Weaver] [BEAM-13751] Parameterize wait timeout so test doesn't waste 2s. [Kyle Weaver] [BEAM-13751] Add comment explaining sleep. [noreply] Merge pull request #16579 from Revert "Revert "Merge pull request #15863 [noreply] Merge pull request #16606 from [BEAM-13247] [Playground] Embedding [noreply] Update Python SDK beam-master tags (#16630) [noreply] Merge pull request #16592 from [BEAM-13722][Playground] Add precompiling [noreply] Merge pull request #16505 from [BEAM-13527] [Playground] Pipeline [Kiley Sok] spotless [Kiley Sok] spotless [mrudary] Generalize S3FileSystem to support multiple URI schemes. [Kenneth Knowles] [BEAM-13768] Fix NullPointerException in BigQueryStorageSourceBase [Heejong Lee] Fix Java SDK container image name for load-tests and nexmark [noreply] [BEAM-13293] XLang Jdbc IO for Go SDK (#16111) [noreply] [BEAM-10206] Add Go Vet to Github Actions (#16612) [daria.malkova] Change executable name fo go tests [avilovpavel6] Fix java test [noreply] [BEAM-13769] Skip test_main_session_not_staged_when_using_cloudpickle [noreply] [BEAM-6744] Support implicitly setting project id in Go Dataflow runner [noreply] Merge pull request #16493 from [BEAM-13632][Playground] Save catalog [noreply] Exclude jul-to-slf4j from Spark runner in quickstart POM templates [career] [BEAM-13734] Support cache directories that use GCS buckets [noreply] [BEAM-11936] Enable a few errorprone checks that were broken by pinned [noreply] [BEAM-13780] Add CONTRIBUTING.md pointing to main guide (#16666) [noreply] [BEAM-13777] Accept cache capacity as input parameter instead of default [noreply] [BEAM-13051][A] Enable pylint warnings [noreply] [BEAM-13779] Fix pr labeling (#16665) [noreply] Merge pull request #16581 from [BEAM-12164]: Add [noreply] Fix labeler trigger (#16674) [noreply] [BEAM-13781] Exclude grpc-netty-shaded from gax-grpc's dependency [noreply] [BEAM-13051] Fixed pylint warnings : raising-non-exception (E0710), [noreply] [BEAM-13740] Correctly install go before running tests (#16673) [noreply] [BEAM-12830] Update local Docker env Go version. (#16670) [noreply] [BEAM-13051][B] Enable pylint warnings [noreply] [BEAM-13430] Revert Spark libraries in Spark runner to provided (#16675) [noreply] [BEAM-12240] Add Java 17 support (#16568) [noreply] [BEAM-13760] Add random component to default python dataflow job name [Kiley Sok] Fix trigger [noreply] Merge pull request #16655 from [BEAM-12164]: Add retry protection to [noreply] Merge pull request #16586 from [BEAM-13731] FhirIO: Add support for [Kiley Sok] skip checker framework [Kiley Sok] fix app name [noreply] [BEAM-13011] Adds a link to Multi-language Pipelines Tips wiki page [Kiley Sok] remove duplicate property check [noreply] [BEAM-12572] Run python examples on multiple runners (#16154) [noreply] [BEAM-13574] Large Wordcount (#16455) [Robert Bradshaw] Avoid packaging avro in the java harness jar. [noreply] [BEAM-13293] Refactor JDBC IO Go Wrapper (#16686) [noreply] Edit license script for Java, add manual licenses for xz (#16692) [mmack] [BEAM-13563] Restructure Kinesis Source for Aws 2 internally to prepare [noreply] [BEAM-4665] Allow joining a running dataflow pipeline without throwing [noreply] [BEAM-13801] Add standard coder tests for state backed iterable. [noreply] [BEAM-13430] Fix provided configuration by removing extendsFrom for [noreply] [BEAM-12830] Print clearer go version fail message (#16693) [Jan Lukavský] Add reference to Books to Learning Resources in website [noreply] Use ThreadLocal for DESERIALIZATION_CONTEXT (#16680) [noreply] Minor: Add apt update after adding deadsnakes repository in dev env [noreply] [BEAM-13807] Regenerate container images to get TF 2.8.0 (#16707) [noreply] [BEAM-13399, BEAM-13683] Eagerly materialize artifacts for automated [noreply] Merge pull request #16617 from [BEAM-13743] [Playground] Add context [noreply] Merge pull request #16618 from [BEAM-13744] [Playground] Add context [noreply] Merge pull request #16698 from [BEAM-13802][Playground] [Bugfix] Clean [noreply] [BEAM-13293][BEAM-13806] Pipe a SchemaIO flag through Go integration [noreply] [BEAM-13605] Modify groupby.apply implementation in preparation for [Kyle Weaver] [BEAM-13796] projection pushdown in BQ IO [noreply] Merge pull request #16436 from [BEAM-1330] - DatastoreIO Writes should [Heejong Lee] [BEAM-13813] Add support for URL artifact to extractStagingToPath [mmack] [BEAM-13663] Remove unused duplicate option for AWS client configuration [avilovpavel6] Remove Python SQL Test example from catalog [mmack] [BEAM-13246] Add support for S3 Bucket Key at the object level (AWS Sdk [relax] Fix timer consistency in direct runner [noreply] [BEAM-13757] adds pane observation in DoFn (#16629) [Jan Lukavský] Change links to Books from Amazon to Publisher [benjamin.gonzalez] [BEAM-12672] Retry flaky tests [noreply] [BEAM-13605] Add support for pandas 1.4.0 (#16590) [benjamin.gonzalez] [BEAM-12672] Fix spotlessApply [noreply] [BEAM-13761] adds Debezium IO wrapper for Go SDK (#16642) [Kyle Weaver] [BEAM-12976] Log projection pushdown optimizations. [noreply] [BEAM-13024] Unify PipelineOptions behavior (#16719) [noreply] Update sdks/go/pkg/beam/artifact/materialize_test.go [noreply] Merge pull request #16605 from [BEAM-13634][Playground] Create a [noreply] Merge pull request #16593 from [BEAM-13725][Playground] Add graph to the [noreply] Merge pull request #16699 from [BEAM-13789][Playground] Change logic of [alexander.chermenin] Fixed CSS for Case study page [mmack] [BEAM-13203] Deprecate SnsIO.writeAsync for AWS Sdk v2 due to risk of [Jeff Tapper] Update Java LTS roadmap info on website for Java 17 [noreply] [BEAM-13828] Fix stale bot (#16734) [noreply] Merge pull request #16364 from [BEAM-13182] Add diagrams to backend [noreply] [BEAM-13811] Fix save_main_session arg in tests examples (#16709) [Kiley Sok] Update beam-master version [egalpin] Use default context output rather than outputWithTimestamp for [noreply] [BEAM-13015] Calculate exception for closing BeamFnDataInboundObserver2 [benjamin.gonzalez] [BEAM-12572] Change jobs to run as cron jobs [noreply] Minor doc tweaks for validating vendoring. (#16747) [noreply] [BEAM-13686] OOM while logging a large pipeline even when logging level [noreply] [BEAM-13629] Update URL artifact type for Dataflow Go (#16490) [noreply] [BEAM-13832] Add automated expansion service start-up to JDBCio (#16739) [noreply] [BEAM-13831] Add automated expansion service infra into Debezium Read() [noreply] [BEAM-13821] Add automated expansion service start-up to KafkaIO [noreply] [BEAM-13799] Created a Dataproc cluster manager for Interactive Beam [noreply] Merge pull request #16727: [BEAM-11971] remove unsafe Concurrent data [noreply] Merge pull request #16726 from [BEAM-12164]: Parses change streams [mmack] [BEAM-13147] Avoid nullness issue during init of AwsModule (AWS Sdk v2) [Ismaël Mejía] [BEAM-13839] Upgrade zstd-jni to version 1.5.2-1 [mmack] [BEAM-13840] Fix usage of legacy rawtypes in AWS modules [alexander.zhuravlev] [BEAM-13820] Changed color of delete icon in pipeline options dropdown, [david.prieto.rivera] Missing contribution [noreply] [BEAM-11971] Revert "Fix timer consistency in direct runner" (#16748) [noreply] [BEAM-13193] Aggregates fn api outbound data/timers of different [noreply] [BEAM-13767] Migrate a bundle of grade tasks to use configuration [noreply] Merge pull request #16653 from [BEAM-12164]: Add integration tests for [noreply] Merge pull request #16728 from [BEAM-13823] Update docs for SnowflakeIO [noreply] Merge pull request #16660 from [BEAM-13771][Playground] Send multifile [noreply] Merge pull request #16646 from [BEAM-13643][Playground] Setup running [noreply] [BEAM-13015] Add state caching benchmark and move benchmarks to their [noreply] [BEAM-13419] Check for initialization in dataflow runner (#16765) [noreply] Merge pull request #16701 from [BEAM-13786] [Playground] [Bugfix] Update [noreply] Merge pull request #16754 from [BEAM-13838][Playground] Add logs in case [noreply] [BEAM-13293] consistent naming for expansion service address and flag [noreply] Merge pull request #16700 from [BEAM-13790][Playground] Change logic of [Pablo Estrada] Output successful rows from BQ Streaming Inserts [noreply] [BEAM-13830] update dependency for debeziumio expansion service (#16743) [noreply] [BEAM-13761] consistent namings for expansion address in Debezium IO [noreply] [BEAM-13806] Shutting down SchemaIO expansion services from Go VR [noreply] [release-2.36.0] Update website/changelog for release 2.36.0 (#16627) [noreply] [BEAM-13848] Update numpy intersphinx link (#16767) [noreply] [release-23.6.0] Fix JIRA link for 2.36 blog (#16771) [Kyle Weaver] [BEAM-12712] Spark: Exclude looping timer tests. [noreply] [BEAM-13647] Use role for Go worker binary. (#16729) [schapman] BEAM-13439 Type annotation for ptransform_fn [noreply] [BEAM-13606] Fail bundles with failed BigTable mutations (#16751) [mmack] [adhoc] Remove remaining usage of Powermock from aws2. [marco.robles] fix broken links in jobs & remove the invalid ones [Kyle Weaver] Update Dataflow Python dev container images. [Kiley Sok] Add java 17 to changes [noreply] [BEAM-12914] Add missing 3.9 opcodes to type inference. (#16761) [noreply] [BEAM-13321] Initial BigQueryIO externalization. (#16489) [noreply] [BEAM-13193] Enable process bundle response elements embedding in Java [noreply] [BEAM-13830] added a debeziumio_expansion_addr flag to GoSDK (#16780) [noreply] Apply spotless. (#16783) [Daniel Oliveira] [BEAM-13732] Switch x-lang BigQueryIO expansion service to GCP one. [noreply] [BEAM-13858] Fix broken github action on :sdks:go:examples:wordCount [Kiley Sok] add jira for runner v2 [noreply] [BEAM-13732] Go SDK BigQuery IO wrapper. Initial implementation. [noreply] [BEAM-13732] Add example for Go BigQuery IO wrapper. (#16786) [noreply] Update CHANGES.md with Go SDK milestones. (#16787) [noreply] [BEAM-13193] Allow BeamFnDataOutboundObserver to flush elements. [akustov] fix name project id from secreton scio deploy action [noreply] [BEAM-13803] Add support for native iterable side inputs to the Go SDK [noreply] [BEAM-11095] Better error handling for illegal emit functions (#16776) [noreply] Merge pull request #16613 from Supporting JdbcIO driver in classpath for [laraschmidt] Fixing the log line to properly handle a large allowed skew. [noreply] Merge pull request #15848 from [BEAM-13835] An any-type implementation [Valentyn Tymofieiev] [BEAM-12920] Assume that bare generators types define simple generators. [Valentyn Tymofieiev] Add a container for Python 3.9. [Valentyn Tymofieiev] Allow job submission with Python 3.9 on Dataflow runner [Valentyn Tymofieiev] Add Python 3.9 test suites. Keep Dataflow V1 suites unchanged for now. [Valentyn Tymofieiev] Add py3.9 Github actions suites. [Valentyn Tymofieiev] Py39 Doc updates. [Valentyn Tymofieiev] [BEAM-9980] Simplify run_validates_container.sh to avoid branching. [Valentyn Tymofieiev] Update Cython to a new version that has py39 wheels. [Valentyn Tymofieiev] [BEAM-13845] Fix comparison with potentially incomparable default [Valentyn Tymofieiev] [BEAM-12920] Assume that bare generators types define simple generators. [Valentyn Tymofieiev] Mark Python 3.9 as supported version. [noreply] [release-2.36.0][website] Fix github release notes script, header for [noreply] Use shell to run python for setupVirtualenv (#16796) [n] BEAM-13159 Update embedded-redis dependency [Daniel Oliveira] [BEAM-13830] Properly shut down Debezium expansion service in IT script. [noreply] Merge pull request #16659 from [BEAM-13774][Playground] Add user to [Valentyn Tymofieiev] [BEAM-13868] Remove gsutil dep from hdfs IT test. [noreply] [BEAM-13776][Playground] (#16731) [noreply] [BEAM-13867] Drop NaNs returned by nlargest in flight_delays example [noreply] Announce Python 3.9 in CHANGES.md (#16802) [Brian Hulette] Moving to 2.38.0-SNAPSHOT on master branch. [noreply] [BEAM-11095] Better error handling for iter/reiter/multimap (#16794) [n] address comments [noreply] Minor: Add 2.38.0 section to CHANGES.md (#16804) [Kyle Weaver] [BEAM-13919] Annotate PerKeyOrderingTest with UsesStatefulParDo. [noreply] [BEAM-12000] Fix typo in portable Python job definition (#16812) [Pablo Estrada] Simplify README for new users [noreply] [BEAM-12164]: Fixes SpannerChangeStreamIT (#16806) [noreply] [BEAM-12572] Fix failures in python examples tests (#16781) [noreply] [BEAM-13921] filter out debeziumIO test for spark runner (#16815) [noreply] [BEAM-13855] Skip SpannerChangeStreamOrderedWithinKeyIT and [noreply] [BEAM-13679] playground - move quick start category to the top (#16808) [noreply] Update license_script.sh (#16789) [noreply] [BEAM-13908] [Coverage] Better testing coverage for gcpopts (#16816) [noreply] Merge pull request #16809 from [BEAM-12164] Added integration test for [alexander.zhuravlev] [BEAM-13775] Fixed bug with run button [ihr] [BEAM-13836] Fix the answers placeholders locations in the Python katas [ihr] [BEAM-13923] Fix the answers placeholders locations in the Java katas [noreply] [BEAM-4032]Support staging binary distributions of dependency packages [noreply] [BEAM-13834] Increase influxDB persistent storage. (#16817) [noreply] Minor: Fix link to nexmark benchmarks (#16803) [noreply] Regenerate python container base_image_requirements.txt (#16832) [noreply] Update 2.36.0 blog post to mention ARM64 support [randomstep] [BEAM-9195] Bump org.testcontainers to 1.16.3 [noreply] Merge pull request #16703 from [BEAM-13804][Playground][Bugfix] Add [noreply] Merge pull request #16611 from [BEAM-13712][Playground] Add graph for [noreply] Merge pull request #16757 from [BEAM-13655] [Playground] Persist the [stranniknm] [BEAM-13785] playground - enable scio sdk [noreply] Minor: Disable checker framework in nightly snapshot (#16829) [artur.khanin] Updated example link [noreply] [BEAM-13860] Make `DoFn.infer_output_type` return element type (#16788) [noreply] [BEAM-13894] Unit test utilities in the ptest package (#16830) [Kenneth Knowles] Add test for processing time continuation trigger [Kyle Weaver] [BEAM-13106] Support Flink 1.14. [Kyle Weaver] [BEAM-13106] Reuse executor instead of shutting it down mid-test. [Kyle Weaver] [BEAM-13106] Prevent infinite wait in Flink savepoint test. [noreply] [BEAM-13922] [Coverage] Make boot.go more testable and add tests [noreply] Exclude SpannerChangeStream IT from Dataflow V1 postcommit (#16851) [noreply] [BEAM-13930] Address StateSpec consistency issue between Runner and Fn [Ismaël Mejía] [BEAM-13202] Fix typos on tests names for VarianceFnTest [mattcasters] [BEAM-13854] Document casting trick for Avro value serializer in KafkaIO [Kenneth Knowles] Disable AfterSynchronizedProcessingTime test on Dataflow [Ismaël Mejía] [BEAM-13202] Add Coder to CountIfFn.Accum [Ismaël Mejía] [BEAM-13202] Reuse Count transform code since CountIf is a specific case [noreply] Merge pull request #16838 from [BEAM-13931] - make sure large rows cause [Kenneth Knowles] Add test category UsesProcessingTimeTimers [Kenneth Knowles] Label tests that need UsesProcessingTimeTimers [Kenneth Knowles] Exclude UsesProcessingTimeTimers from SamzaRunner tests [noreply] Seznam Case Study (#16825) [noreply] [Website] Apache Hop Case Study (#16824) [noreply] [BEAM-13694] Force hadoop-hdfs-client in hadoopVersion tests for hdfs [noreply] [Website] Ricardo - added case study feedback (#16807) [noreply] Merge pull request #16735 from [BEAM-13827] - fix medium file size [Kyle Weaver] [BEAM-13106] A couple additional fixes to FlinkSavepointTest. [noreply] Merge pull request #16753 from [BEAM-13837] [Playground] show graph on [mmack] [adhoc] Migrate KinesisIOIT to use ITEnvironment for Localstack based IT [rogelio.hernandez] [BEAM-13051] Added descriptions to Kinesis and PortableRunner exceptions [noreply] [BEAM-13955] Fix pylint breakage from #16836 (#16867) [relax] Fix TableRow conversion for the case of fields named "f" [noreply] Bump dataflow.fnapi_container_version (#16874) [mmack] [BEAM-13563] Introducing common AWS ClientBuilderFactory to unify [laraschmidt] Fix final allowskew error to properly handle a large allowedSkew [noreply] Case studies page improvements (#16702) [noreply] [BEAM-13946] Add get_dummies(), a non-deferred column operation on [Kyle Weaver] [BEAM-13796] Move test to ReadTest class and correct javadoc for [noreply] [release-2.36.0] Fix pickler argument for 2.36 blog (#16774) [thiagotnunes] fix: fix bug when retrieving either string or json [noreply] [adhoc] Avoid using SerializablePipelineOptions for testing to minimize [noreply] [BEAM-13812] Integrate DataprocClusterManager into Interactive [Kyle Weaver] [BEAM-13796] Pushdown is not supported on TypedRead#fromQuery. [noreply] [BEAM-12572] Fix failing python examples tests in Dataflow runner [Ankur Goenka] [BEAM-13952] Sickbaying [noreply] Remove build status from PR (#16902) [noreply] [BEAM-13738] Reenable ignored SQS test after bumping elasticmq for fixed [noreply] fix build status link (#16907) [noreply] Merge pull request #16549 from [BEAM-13681][Playground] Refactoring [noreply] Merge pull request #16732 from [BEAM-13825] [Playground] updated [noreply] Merge pull request #16683 from [BEAM-13713][Playground] Java graph [noreply] case study pages - improvements and fixes (#16896) [noreply] Palo Alto case study (#16915) [noreply] [BEAM-12645] Fix code-cov flakes due to monorepo. (#16925) [noreply] [BEAM-13969] Deprecate stringx package (#16884) [noreply] Add Go badge to ReadMe (#16897) [noreply] [BEAM-13980] Re-add method gone missing in af2f8ee6 (#16918) [benjamin.gonzalez] Revert PR#16253 due errors with plugin flaky-test-handler [noreply] [BEAM-13884] Improve mtime package (#16924) [noreply] Minor: Update Go API doc links (#16932) [noreply] [BEAM-13218] Re-enable [noreply] Merge pull request #16857 from [BEAM-13662] [Playground] Support [noreply] Merge pull request #16826 from [BEAM-13870] [Playground] Increase test [stranniknm] Palo Alto case study - fix link [bulat.safiullin] [BEAM-13976] [Website] update homepage [bulat.safiullin] [BEAM-13976] [Website] update homepage, add logo [noreply] Fix BoundedQueueExecutor and StreamingDataflowWorker to actually limit [noreply] [BEAM-1857] Add Neo4jIO (#15916) [noreply] [BEAM-13767] Migrate serveral portable runner tasks to use configuration [noreply] [BEAM-13996] Removing 'No cluster_manager is associated with the [noreply] [BEAM-13906] Improve coverage of errors package (#16934) [noreply] [BEAM-13886] unit tests for trigger package (#16935) [noreply] [BEAM-4767] Remove beam- prefix from release script tags (#16899) [noreply] [BEAM-13866] Add small unit tests to errorx, make boolean assignment [noreply] [BEAM-13925] Add most of the supporting files for the pr management [noreply] Merge pull request #16846 from [BEAM-12164]: Add sad path tests for [bulat.safiullin] [BEAM-13976] [Website] update text [bulat.safiullin] [BEAM-13976] [Website] Update Community landing page [rogelio.hernandez] [BEAM-12777] Removed current docs version redirect [bulat.safiullin] [BEAM-13979] [Website] Update Community/Contact us page [noreply] Memoize some objects for timer processing to reduce overhead. (#16207) [noreply] [BEAM-13965] Use TypeDeserializer if type information is available to [noreply] [BEAM-13912] Add more coverage for dataflow.go (#16903) [Luke Cwik] [BEAM-10212] Clean-up comments, remove rawtypes usage. [noreply] [BEAM-12563] swaplevel general function for dataframe and series [noreply] [BEAM-14001] Update coder.go unit tests (#16952) [noreply] [BEAM-13910] Improve coverage of gcsx package (#16942) [noreply] [BEAM-13015] Use a DirectExecutor for state since we are just completing [bulat.safiullin] [BEAM-13979] [Website] update title [stranniknm] [BEAM-13999] playground - support vertical orientation for graph [bulat.safiullin] [BEAM-13979] [Website] delete space [noreply] Build wheels for Python 3.9 [noreply] Merge pull request #16892 from [BEAM-13755] [Playground] Scroll the [noreply] Merge pull request #16880 from [BEAM-13963][Playground] Get bucket name [noreply] Merge pull request #16870 from [BEAM-13874][Playground] Tag multifile [bulat.safiullin] [BEAM-13979] [Website] add Beam Playground [noreply] Merge pull request #16910 from [BEAM-13724] [Playground] Get the default [noreply] [BEAM-14008] Fix incorrect guava import (#16966) [noreply] Fix ignored exception in BatchSpannerRead. (#16960) [noreply] [BEAM-13917] Improve coverage of databaseio package (#16956) [noreply] [BEAM-13925] Add entry files to process new prs and pr updates for PR [noreply] [BEAM-13899] Improve coverage of debug package (#16951) [noreply] [BEAM-13907] Improve coverage of textio package (#16937) [noreply] [BEAM-9150] Fix beam_PostRelease_Python_Candidate (python RC validation [bulat.safiullin] [BEAM-13976] [Website] delete Beam Playground [benjamin.gonzalez] [BEAM-12572] Enable test with Pipeline serialization error fixed, link [noreply] Merge pull request #16850: [BEAM-11205] Upgrade Libraries BOM [noreply] Merge pull request #16484 from [BEAM-13633] [Playground] Implement [noreply] Add 2022 events blog post (#16975) [benjamin.gonzalez] [BEAM-12572] Sickbay TfIdf test for DirectRunner [noreply] Clean up Go formatter suggestions (#16973) [noreply] [BEAM-14012] Add go fmt to Github Actions (#16978) [noreply] [BEAM-13911] Add basic tests to Go direct runner. (#16979) [noreply] [BEAM-13960] Add support for more types when converting from between row [bulat.safiullin] [BEAM-13976] [Website] change navbar css links rules, delete links from [bulat.safiullin] [BEAM-13977] [Website] delete available-contact-channels on mobile [bulat.safiullin] [BEAM-13976] [Website] change padding size between the sections [Alexey Romanenko] Bump org.mongodb:mongo-java-driver to 3.12.10 [noreply] [BEAM-13973] Link Dataproc Flink master URLs to the InteractiveRunner [noreply] [BEAM-13925] Turn pr bot on for go prs (#16984) [Pablo Estrada] Skipping flaky sad-path tests for Spanner changestreams [noreply] [BEAM-13964] Bump kotlin to 1.6.x (#16882) [noreply] Merge pull request #16906: [BEAM-13974] Handle idle Storage Api streams [noreply] Merge pull request #16562 from [BEAM-13051][D] Enable pylint warnings [noreply] [BEAM-13925] A couple small pr-bot bug fixes (#16996) [noreply] [BEAM-14029] Add getter, setter for target maven repo (#16995) [noreply] [BEAM-13903] Improve coverage of metricsx package (#16994) [noreply] [BEAM-13892] Improve coverage of avroio package (#16990) [rahuliyer573] py: Import beam plugins before starting SdkHarness [jakub.kukul] [BEAM-14039] Propagate ignore_unknown_columns parameter. [noreply] [adhoc] Prepare aws2 ClientConfiguration for json serialization and [bulat.safiullin] [BEAM-13976] [Website] change title to capital letters [noreply] Merge pull request #16879 from [BEAM-12164] Add javadocs to [stephen.patel] BEAM-14011 fix s3 filesystem multipart copy [noreply] [Cleanup] Update pre-v2 go package references (#17002) [noreply] [BEAM-13885] Add unit tests to window package (#16971) [noreply] Merge pull request #16891 from [BEAM-13872] [Playground] Increase test [noreply] Merge pull request #16912 from [BEAM-13878] [Playground] Increase test [noreply] Merge pull request #16946 from [BEAM-13873] [Playground] Increase test [noreply] [BEAM-13951] Update mass_comment.py list of Run commands (#16889) [dannymccormick] [BEAM-11085] Test that windows are correctly observed in DoFns [noreply] [BEAM-10652] Allow Clustering without Partition in BigQuery (#16578) [noreply] [BEAM-13857] Add K:V flags for expansion service jars and addresses to [Kiley Sok] Add Java 17 Nexmark metrics to Grafana [yiru] . [Chamikara Madhusanka Jayalath] Updates x-lang release validation to use staged jars [Valentyn Tymofieiev] Bump numpy bound to include 1.22 and regenerate container deps. [github-actions] [BEAM-13925] months in date constructor are 0 indexed [bulat.safiullin] [BEAM-13976] [Website] change title [noreply] Merge pull request #16842 from [BEAM-13932][Playground] Container's user [noreply] Doc updates and blog post for 2.37.0 (#16887) [jrmccluskey] [BEAM-14050] Update taxi.go example instructions [yiru] . [noreply] Remove resolved issue in docs + update class path on sample (#17018) [noreply] [BEAM-14016] Fixed flaky postcommit test (#17009) [noreply] Remove resolved issue in notebook [noreply] [BEAM-13947] Add split() and rsplit(), non-deferred column operations on [noreply] BEAM-14026 - Fixes bug related to Unnesting nested rows in an array [hengfeng] [BEAM-12164]: display the metadata table's name on UI [noreply] [BEAM-13925] Add ability to get metrics on pr-bot performance (#16985) [noreply] Give pr bot write permissions on pr update [dhuntsperger] documented maven-to-gradle conversion for Dataflow; refactored java [dhuntsperger] adding a list of example pipelines [benjamin.gonzalez] [BEAM-12572] Refactor naming for BigQueryClient [noreply] Adding a logical type for Schemas using proto serialization. (#16940) [noreply] BEAM-13765 missing PAssert methods (#16668) [yiru] . [Robert Bradshaw] Update dataflow API client. [Robert Bradshaw] Instructions for updating apitools generated files. [dhuntsperger] removing unnecessary `ls` command from maven project generation [noreply] [BEAM-13909] improve coverage of Provision package (#17014) [bulat.safiullin] [BEAM-14040] [Website] create new page, add link [yiru] format fix [yiru] . [noreply] Merge pull request #17027: [BEAM-11205] Upgrade GCP Libraries BOM [noreply] [BEAM-13709] Inconsistent behavior when parsing boolean flags across [yiru] make DoFn into a separate class [noreply] [BEAM-10976] Bundle finalization: Harness and some exec changes (#16980) [noreply] Revert "[BEAM-13993] [BEAM-10039] Import beam plugins before starting [noreply] Merge pull request #16976 from [BEAM-14010] [Website] Add Playground [noreply] [BEAM-12447] Upgrade cloud build client and add/cleanup options (#17032) [bulat.safiullin] [BEAM-13977] [Website] change title [bulat.safiullin] [BEAM-13979] [Website] change text [bulat.safiullin] [BEAM-13976] [Website] change text [bulat.safiullin] [BEAM-13977] [Website] change text, add capital letters [noreply] Merge pull request #17036 from [BEAM-12164] Convert all static instances [bulat.safiullin] [BEAM-13976] [Website] add playground sass, change text-align [noreply] fix variable reference (#16991) [yiru] . [noreply] Mapped JOB_STATE_RESOURCE_CLEANING_UP to State.RUNNING. [noreply] Merge pull request #16844 from [BEAM-12164]: allow for nanosecond [ryanthompson591] fixed typo in typehints [noreply] [BEAM-13904] Increase unit testing in the reflectx package (#17024) [bulat.safiullin] [BEAM-14040] [Website] add io connectors table [bulat.safiullin] [BEAM-13976] [Website] add playground section, add empty line [stranniknm] [BEAM-14079] playground - improve accessibility [noreply] [BEAM-13925] Find and address prs that havent been reviewed in a week [noreply] Fix import path [noreply] [BEAM-13925] Fix one more import path [noreply] Add a StatefulDoFn test that sets event time timer within allowed [bulat.safiullin] [BEAM-14040] [Website] add overflow to css, add table content [noreply] Merge pull request #17056 from [BEAM-14076] [SnowflakeIO] Add support [Ismaël Mejía] [BEAM-13981] Remove Spark Runner specific code for event logging [bulat.safiullin] [BEAM-14040] [Website] change ✘ for ✔, add license, add br [bulat.safiullin] [BEAM-14040] [Website] add empty line [vitaly.terentyev] [BEAM-2766] Support null key/values in HadoopFormatIO [bulat.safiullin] [BEAM-14040] [Website] change td [vitaly.terentyev] [BEAM-2766] Fix checkstyle [bulat.safiullin] [BEAM-14041] [Website] update built io transforms [noreply] [BEAM-14072] [BEAM-13993] [BEAM-10039] Import beam plugins before [mmack] [BEAM-13175] Add KinesisIO.write for AWS SDK v2. [bulat.safiullin] [BEAM-14041] [Website] move connectors from Miscellaneous to Database [bulat.safiullin] [BEAM-14040] [Website] change links color [mmack] [adhoc] Update owners for aws modules [zyichi] Remove unused prebuild_sdk_container_base_iamge option from validate [noreply] [BEAM-14051] Add a warning to gradle.properties that many Beam modules [noreply] Merge pull request #17000 from [BEAM-13881] [Playground] Increase test [noreply] Add deterministic dict coding via key sorting. (#17037) [noreply] Merge pull request #17054 from [BEAM-14075] [SnowflakeIO] Change a [danielamartinmtz] Updated metrics' CronJob API to use the latest batch version. [dhuntsperger] fixing filename formatting in response to feedback [dhuntsperger] adding extra step emphasizing runner setupt [dhuntsperger] reorganized instructions to emphasize setup steps for runners [mmack] [BEAM-13175] Document changes to IOs in amazon-web-services2 for AWS [bulat.safiullin] [BEAM-14041] [Website] change IO from go to java [bulat.safiullin] [BEAM-14040] [Website] change links, change specific version to current [noreply] [BEAM-13767] Move a bunch of python tasks to use gradle configuration… [danielamartinmtz] Updated cluster to test in metrics-upgrade-clone in BeamMetrics_Publish [yiru] fix setting [noreply] Merge pull request #17052 from [BEAM-13818] [SnowflakeIO] Add support [noreply] Adding pydoc for StateHandler (#17091) [noreply] BEAM-3165 Bypass split if numSplit is zero (#17084) [hengfeng] feat: add more custom metrics [aydar.zaynutdinov] [BEAM-13976][Website] [aydar.zaynutdinov] [BEAM-14040][Website] [aydar.zaynutdinov] [BEAM-14041][Website] [noreply] [BEAM-14103][Playgrounf][Bugfix] Fix google analytics id (#17092) [noreply] Minor: Make ScopedReadStateSupplier final (#16992) [noreply] [BEAM-14113] Improve SamzaJobServerDriver extensibility (#17099) [noreply] [BEAM-14116] Chunk commit requests dynamically (#17004) [noreply] Merge pull request #17079 from [BEAM-13660] Add types and queries in [noreply] [BEAM-13888] Add unit testing to ioutilx (#17058) [noreply] Merge pull request #16822 from [BEAM-13841][Playground] Add Application [noreply] Minor: Make serializableCoder warning gramatically correct english [noreply] [BEAM-14091] Fixing Interactive Beam show/collect for remote runners [noreply] [BEAM-11934] Add enable_file_dynamic_sharding to allow DataflowRunner [noreply] [BEAM-12777] Create symlink for `current` directory (#17105) [noreply] [BEAM-14020] Adding SchemaTransform, SchemaTransformProvider, [noreply] [BEAM-13015] Modify metrics to begin and reset to a non-dirty state. [noreply] [BEAM-14112] Avoid storing a generator in _CustomBigQuerySource (#17100) [noreply] Populate environment capabilities in v1beta3 protos. (#17042) [Kyle Weaver] [BEAM-12976] Test a whole pipeline using projection pushdown in BQ IO. [Kyle Weaver] [BEAM-12976] Enable projection pushdown for Java pipelines on Dataflow, [noreply] [BEAM-14038] Auto-startup for Python expansion service. (#17035) [Kyle Weaver] [BEAM-14123] Fix typo in hdfsIntegrationTest task name. [Kyle Weaver] [BEAM-14124] Add display data to BQ storage reads. [noreply] [BEAM-13893] improved coverage of jobopts package (#17003) [noreply] Merge pull request #16977 from [BEAM-12164] Added integration test for [mmack] [adhoc] Minor cleanup for aws2 tests [mmack] [BEAM-14125] Update website IO matrix to recommend aws2 IOs [danielamartinmtz] Updated StateFulSet k8s obejct in cassandra-svc-statefulset.yaml file in [danielamartinmtz] Updated documentation including cluster specs. [noreply] Beam 13058 k8s apis upgrade - elasticsearch (#18) [noreply] [BEAM-14128] Eliminating quadratic behavior of [danielamartinmtz] Removed code used for testing. [noreply] [BEAM-13972] Add RunInference interface (#16917) [danielamartinmtz] Removed code used for testing in job_PostCommit_BeamMetrics_Publish [noreply] Beam 13058 k8s apis upgrade - Adding Basic Auth details in documentation [noreply] Merge pull request #17116 from [BEAM-12164] Remove change_stream in [yiru] fix checkstyle [yiru] spotlessapply [noreply] Deprecate tags.go (#17025) [noreply] [BEAM-12753] and [BEAM-12815] Fix Flink Integration Tests (#17067) [noreply] Merge pull request #16895 from [BEAM-13882][Playground] More tests for [noreply] [BEAM-13925] Add weekly automation to update our reviewer config [noreply] Merge pull request #17076 from Beam 14082 update payground for mobile [noreply] [BEAM-13925] Assign committers in the scheduled action (#17062) [noreply] Pin setup-gcloud to v0 instead of master (#17123) [noreply] [BEAM-3304] documentation for PaneInfo in BPG (#17047) [noreply] Merge pull request #17016 from [BEAM-14049][Playground] Add new API [noreply] Merge pull request #17077 from [BEAM-14078] [Website] change link [noreply] Merge pull request #17085 from [BEAM-14077] [Website] add beam [noreply] Update Changes.md w/Go pipeline pre-process fix. [noreply] [BEAM-14098] wrapper for postgres on JDBC IO GO SDK (#17088) [noreply] Merge pull request #17023 from [BEAM-12164]: Remove child partition [noreply] [BEAM-14122] Upgrade pip-licenses dependency (#17132) [mmack] [adhoc] Move aws IT tests to testing package according to best practices [noreply] fixes static checks and go lint issues (#17138) [Kyle Weaver] Don't print in task configuration. [noreply] [BEAM-14136] Clean up staticcheck and linter warnings in the Go SDK [noreply] Merge pull request #17063 from [BEAM-12164] Fix flaky tests [noreply] Revert "[BEAM-14112] Avoid storing a generator in _CustomBigQuerySource [Kyle Weaver] [BEAM-4106] Remove filesToStage from Flink pipeline option list. [noreply] [BEAM-14071] Enabling Flink on Dataproc for Interactive Beam (#17044) [noreply] Minor: Bypass schema registry in schemas_test.py (#17108) [mmack] [BEAM-13232] Close clients properly in KinesisSource. Also use lazy init [noreply] [BEAM-14141] Set Interactive Beam to use the default Dataproc image [noreply] BEAM-14115 - Update find criteria limited to _id (#17102) [chamikaramj] Disable BigQueryIOStorageWriteIT for general Java post-commit [Pablo Estrada] [BEAM-14151] Excluding Spanner CDC tests from Dataflow V1 suite [noreply] Revert "[BEAM-14038] Auto-startup for Python expansion service. [noreply] Minor: Bump timeout for Java PreCommit (#17157) [noreply] [BEAM-14152] Disable flaky [danielamartinmtz] Added comments in initContainers and remove unused code in elasticsearch [noreply] Fixing a small bug in TypedSchemaTransformTest that caused it to flake. [noreply] [BEAM-14116] Catch MonitoringInfoMetricName null keys or values in the [noreply] [BEAM-14129] Restructure SubscriptionPartitionLoader to use a manual SDF [noreply] [BEAM-13015] Avoid repeated weighing of StateKey in [noreply] Add option to add modules to JDK add-open (#17110) [noreply] [BEAM-13015] Clarify ownership of the list for state caching across [noreply] [BEAM-14134] Optimize memory allocations for various core coders [noreply] [BEAM-14129] Restructure PubsubLiteIO Read side to produce smaller [noreply] [BEAM-12697] Add primitive field generation from IR to SBE extension [noreply] [BEAM-13889] Add test cases to jsonx package (#17124) [noreply] Remove unreachable code in container.go (#17166) [noreply] Add ability to handle streaming input to AvroSchemaIOProvider (#17126) [noreply] [BEAM-12898] Flink Load Tests failure- UncheckedExecutionException - [Daniel Oliveira] Moving to 2.39.0-SNAPSHOT on master branch. ------------------------------------------ [...truncated 36.28 KB...] Collecting requests<3.0.0,>=2.24.0 Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB) Collecting typing-extensions>=3.7.0 Using cached typing_extensions-4.1.1-py3-none-any.whl (26 kB) Collecting pandas<1.5,>=1.0 Using cached pandas-1.3.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB) Collecting cachetools<5,>=3.1.0 Using cached cachetools-4.2.4-py3-none-any.whl (10 kB) Collecting google-apitools<0.5.32,>=0.5.31 Using cached google_apitools-0.5.31-py3-none-any.whl Collecting google-auth<3,>=1.18.0 Using cached google_auth-2.6.2-py2.py3-none-any.whl (156 kB) Collecting google-cloud-datastore<2,>=1.8.0 Using cached google_cloud_datastore-1.15.3-py2.py3-none-any.whl (134 kB) Collecting google-cloud-pubsub<3,>=2.1.0 Using cached google_cloud_pubsub-2.11.0-py2.py3-none-any.whl (232 kB) Collecting google-cloud-pubsublite<2,>=1.2.0 Using cached google_cloud_pubsublite-1.4.1-py2.py3-none-any.whl (265 kB) Collecting google-cloud-bigquery<3,>=1.6.0 Using cached google_cloud_bigquery-2.34.2-py2.py3-none-any.whl (206 kB) Collecting google-cloud-bigquery-storage>=2.6.3 Using cached google_cloud_bigquery_storage-2.13.0-py2.py3-none-any.whl (179 kB) Collecting google-cloud-core<2,>=0.28.1 Using cached google_cloud_core-1.7.2-py2.py3-none-any.whl (28 kB) Collecting google-cloud-bigtable<2,>=0.31.1 Using cached google_cloud_bigtable-1.7.0-py2.py3-none-any.whl (267 kB) Collecting google-cloud-spanner<2,>=1.13.0 Using cached google_cloud_spanner-1.19.1-py2.py3-none-any.whl (255 kB) Collecting grpcio-gcp<1,>=0.2.2 Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB) Collecting google-cloud-dlp<4,>=3.0.0 Using cached google_cloud_dlp-3.6.2-py2.py3-none-any.whl (113 kB) Collecting google-cloud-language<2,>=1.3.0 Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB) Collecting google-cloud-videointelligence<2,>=1.8.0 Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB) Collecting google-cloud-vision<2,>=0.38.0 Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB) Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0 Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB) Collecting boto3>=1.9 Using cached boto3-1.21.25-py3-none-any.whl (132 kB) Collecting azure-storage-blob>=12.3.2 Using cached azure_storage_blob-12.10.0-py3-none-any.whl (346 kB) Collecting azure-core>=1.7.0 Using cached azure_core-1.23.0-py3-none-any.whl (179 kB) Collecting freezegun>=0.3.12 Using cached freezegun-1.2.1-py3-none-any.whl (16 kB) Collecting mock<3.0.0,>=1.0.1 Using cached mock-2.0.0-py2.py3-none-any.whl (56 kB) Collecting parameterized<0.8.0,>=0.7.1 Using cached parameterized-0.7.5-py2.py3-none-any.whl (17 kB) Collecting pyhamcrest!=1.10.0,<2.0.0,>=1.9 Using cached PyHamcrest-1.10.1-py3-none-any.whl (48 kB) Collecting pyyaml<7.0.0,>=3.12 Using cached PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB) Collecting requests_mock<2.0,>=1.7 Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB) Collecting tenacity<6.0,>=5.0.2 Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB) Collecting pytest<5.0,>=4.4.0 Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB) Collecting pytest-xdist<2,>=1.29.0 Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB) Collecting pytest-timeout<2,>=1.3.3 Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB) Collecting sqlalchemy<2.0,>=1.3 Using cached SQLAlchemy-1.4.32-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB) Collecting psycopg2-binary<3.0.0,>=2.8.5 Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB) Collecting testcontainers[mysql]<4.0.0,>=3.0.3 Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB) Collecting cryptography>=36.0.0 Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB) Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Flink_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0) Collecting msrest>=0.6.21 Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB) Collecting s3transfer<0.6.0,>=0.5.0 Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB) Collecting jmespath<2.0.0,>=0.7.1 Using cached jmespath-1.0.0-py3-none-any.whl (23 kB) Collecting botocore<1.25.0,>=1.24.25 Using cached botocore-1.24.25-py3-none-any.whl (8.6 MB) Collecting cffi>=1.12 Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB) Collecting fasteners>=0.14 Using cached fasteners-0.17.3-py3-none-any.whl (18 kB) Collecting pyasn1-modules>=0.2.1 Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB) Collecting rsa<5,>=3.1.4 Using cached rsa-4.8-py3-none-any.whl (39 kB) Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Flink_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3) Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5 Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB) Collecting google-resumable-media<3.0dev,>=0.6.0 Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB) Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5 Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB) Collecting grpc-google-iam-v1<0.13dev,>=0.12.3 Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl Collecting google-auth<3,>=1.18.0 Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB) Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Flink_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (60.10.0) Collecting grpcio-status>=1.16.0 Using cached grpcio_status-1.45.0-py3-none-any.whl (10.0 kB) Collecting overrides<7.0.0,>=6.0.1 Using cached overrides-6.1.0-py3-none-any.whl (14 kB) Collecting docopt Using cached docopt-0.6.2-py2.py3-none-any.whl Collecting pyparsing<3,>=2.4.2 Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) Collecting pbr>=0.11 Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB) Collecting pyasn1>=0.1.7 Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB) Collecting attrs>=17.4.0 Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB) Collecting wcwidth Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB) Collecting atomicwrites>=1.0 Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB) Collecting more-itertools>=4.0.0 Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB) Collecting pluggy<1.0,>=0.12 Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB) Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Flink_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3) Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Flink_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0) Collecting pytest-forked Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB) Collecting execnet>=1.1 Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB) Collecting certifi>=2017.4.17 Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB) Collecting charset-normalizer~=2.0.0 Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB) Collecting idna<4,>=2.5 Using cached idna-3.3-py3-none-any.whl (61 kB) Collecting urllib3<1.27,>=1.21.1 Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB) Collecting greenlet!=0.4.17 Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB) Collecting docker Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB) Collecting wrapt Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB) Collecting deprecation Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB) Collecting pymysql Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB) Collecting pycparser Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB) Collecting googleapis-common-protos<2.0dev,>=1.6.0 Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB) Collecting google-crc32c<2.0dev,>=1.0 Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB) Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Flink_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.7.0) Collecting isodate>=0.6.0 Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB) Collecting requests-oauthlib>=0.5.0 Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB) Collecting typing-utils>=0.0.3 Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB) Collecting websocket-client>=0.32.0 Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB) Collecting oauthlib>=3.0.0 Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB) Building wheels for collected packages: apache-beam Building wheel for apache-beam (setup.py): started Building wheel for apache-beam (setup.py): finished with status 'done' Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2695955 sha256=c23eb36686ce5370e9332d543a631cd86d27ef7d851422969a2829e71698048c Stored in directory: /home/jenkins/.cache/pip/wheels/bc/10/43/2e6f08d57c0f9fc9e1f1ee753673c40f9e03ed660bcae39b4a Successfully built apache-beam Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite Attempting uninstall: pyparsing Found existing installation: pyparsing 3.0.7 Uninstalling pyparsing-3.0.7: Successfully uninstalled pyparsing-3.0.7 Attempting uninstall: pluggy Found existing installation: pluggy 1.0.0 Uninstalling pluggy-1.0.0: Successfully uninstalled pluggy-1.0.0 Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.25 botocore-1.24.25 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.45.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0 > Task :sdks:python:apache_beam:testing:load_tests:run WARNING:apache_beam.transforms.core:Key coder FastPrimitivesCoder for transform <ParDo(PTransform) label=[Step: 0]> with stateful DoFn may not be deterministic. This may cause incorrect behavior for complex key types. Consider adding an input type hint for this transform. WARNING:apache_beam.transforms.core:Key coder FastPrimitivesCoder for transform <ParDo(PTransform) label=[Step: 1]> with stateful DoFn may not be deterministic. This may cause incorrect behavior for complex key types. Consider adding an input type hint for this transform. WARNING:apache_beam.transforms.core:Key coder FastPrimitivesCoder for transform <ParDo(PTransform) label=[Step: 2]> with stateful DoFn may not be deterministic. This may cause incorrect behavior for complex key types. Consider adding an input type hint for this transform. WARNING:apache_beam.transforms.core:Key coder FastPrimitivesCoder for transform <ParDo(PTransform) label=[Step: 3]> with stateful DoFn may not be deterministic. This may cause incorrect behavior for complex key types. Consider adding an input type hint for this transform. WARNING:apache_beam.transforms.core:Key coder FastPrimitivesCoder for transform <ParDo(PTransform) label=[Step: 4]> with stateful DoFn may not be deterministic. This may cause incorrect behavior for complex key types. Consider adding an input type hint for this transform. WARNING:apache_beam.transforms.core:Key coder FastPrimitivesCoder for transform <ParDo(PTransform) label=[Step: 5]> with stateful DoFn may not be deterministic. This may cause incorrect behavior for complex key types. Consider adding an input type hint for this transform. WARNING:apache_beam.transforms.core:Key coder FastPrimitivesCoder for transform <ParDo(PTransform) label=[Step: 6]> with stateful DoFn may not be deterministic. This may cause incorrect behavior for complex key types. Consider adding an input type hint for this transform. WARNING:apache_beam.transforms.core:Key coder FastPrimitivesCoder for transform <ParDo(PTransform) label=[Step: 7]> with stateful DoFn may not be deterministic. This may cause incorrect behavior for complex key types. Consider adding an input type hint for this transform. WARNING:apache_beam.transforms.core:Key coder FastPrimitivesCoder for transform <ParDo(PTransform) label=[Step: 8]> with stateful DoFn may not be deterministic. This may cause incorrect behavior for complex key types. Consider adding an input type hint for this transform. WARNING:apache_beam.transforms.core:Key coder FastPrimitivesCoder for transform <ParDo(PTransform) label=[Step: 9]> with stateful DoFn may not be deterministic. This may cause incorrect behavior for complex key types. Consider adding an input type hint for this transform. WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter. INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev INFO:root:Using provided Python SDK container image: gcr.io/apache-beam-testing/beam_portability/beam_python3.7_sdk:latest INFO:root:Python SDK container image set to "gcr.io/apache-beam-testing/beam_portability/beam_python3.7_sdk:latest" for Docker environment WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--iterations=10', '--number_of_counter_operations=0', '--number_of_counters=0', '--stateful', '--use_stateful_load_generator'] INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING ERROR:root:java.lang.NoClassDefFoundError: Could not initialize class org.apache.beam.runners.core.construction.SerializablePipelineOptions INFO:apache_beam.runners.portability.portable_runner:Job state changed to FAILED Traceback (most recent call last): File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Flink_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/pardo_test.py";,> line 167, in <module> ParDoTest().run() File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Flink_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py";,> line 151, in run self.result.wait_until_finish(duration=self.timeout_ms) File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Flink_Streaming/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py";,> line 600, in wait_until_finish raise self._runtime_exception RuntimeError: Pipeline load-tests-python-flink-streaming-pardo-1-0324100339_00162c87-7535-43ba-aa69-f4ead1febf7e failed in state FAILED: java.lang.NoClassDefFoundError: Could not initialize class org.apache.beam.runners.core.construction.SerializablePipelineOptions > Task :sdks:python:apache_beam:testing:load_tests:run FAILED FAILURE: Build failed with an exception. * Where: Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Flink_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58 * What went wrong: Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'. > error occurred * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 1m 28s 15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/dspnvhtbctrhq Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
