See <https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Direct_Java17/2/display/redirect?page=changes>
Changes: [heejong] [BEAM-13092] Adding dummy external transform translators for Dataflow [mmack] [BEAM-11494][BEAM-11821] FileIO stops overwriting files on retries (AWS [Robert Bradshaw] Revert "Revert "Allow wildcards for java class lookup transform [Daniel Oliveira] Minor Gradle Fixups: Removing unneeded dependency in expansion service. [benjamin.gonzalez] [BEAM-13088] Add parameter tempWriteDataset to BigQueryIO to store temp [Robert Bradshaw] Move stand-alone expansion service jar into its own project. [Daniel Oliveira] [BEAM-13321] Pass TempLocation as pipeline option to Dataflow Go for [melissapa] [BEAM-11758] Update basics page: Trigger, State and timers [Pablo Estrada] [py] Supporting ignore_unknown_values for WriteToBigQuery [noreply] Merge pull request #16072 from [BEAM-13135][Playground] Add function to [noreply] Merge pull request #16084 from [BEAM-13347][Playground] [Bugfix] Backend [noreply] Merge pull request #15987 from [BEAM-13242] Allow values with smaller [noreply] [BEAM-11936] Fix errorprone UnusedVariable in Extensions (#16033) [noreply] Merge pull request #15936 from [BEAM-13351] FhirIO GetPatientEverything [avilovpavel6] Add fmt preparator for go sdk [mmack] [BEAM-13233] Replace AWS API used to list shards from DescribeStream to [noreply] Merge pull request #16073 from [BEAM-13267][Playground] Implement [ningkang0957] Updated screen diff test for Interactive Beam [noreply] [BEAM-13282][Playground] Create go dockerfile #16049 [noreply] Merge pull request #16085 from [Beam-13336][Playground] Refactor [noreply] [BEAM-11936] Fix errorprone UnusedVariable in Extensions-sql (#16034) [noreply] [BEAM-13193] Support [Robert Bradshaw] Require explicit designation of proxy partitioning specification. [Robert Bradshaw] Elementwise transforms preserve all partitionings. [noreply] Merge pull request #16090: [BEAM-13352] Add transient qualifier to [aydar.zaynutdinov] [BEAM-13329][Playground] [aydar.zaynutdinov] [BEAM-13329][Playground] [aydar.zaynutdinov] [BEAM-13360][Playground] [noreply] Merge pull request #16011 from [BEAM-12164] Add models for Spanner [noreply] [BEAM-13335] Use signed int64 range for DataFrame read unique indexes [iyi] Add test case for untriggered expansion with temp tables. [zyichi] Fix small typo in ProcessBundleHandlerTest.java [noreply] Merge pull request #16079 from [BEAM-13241] [Playground] Frontend [noreply] Ricardo Case Study (#16087) [noreply] Merge pull request #16086 from [BEAM-13258][Playground] Get and store [noreply] [BEAM-11936] Fix errorprone UnusedVariable in Runners (#16035) [noreply] [BEAM-8123] Add cloudpickle as optional library (#15472) [noreply] [BEAM-12572] Beam python examples continuously exercised on at least 2 [alexander.zhuravlev] [BEAM-13370] Deleted unused prints & strings [ilya.kozyrev] Fix pylint issues and apply yapf with Beam config [stranniknm] [BEAM-13112]: playground embedded version [aydar.zaynutdinov] [BEAM-13363][Playground] [ilya.kozyrev] fix white spaces [noreply] Don't pin a particular version of Tensorflow. (#16102) [noreply] [BEAM-12733] Fix failing integration tests for Java Recommendation AI [noreply] [BEAM-13288] improve logging for no rows present error (#16096) [aydar.zaynutdinov] [BEAM-13357][Playground] [thiagotnunes] [BEAM-12164] Add Spanner Change Stream DAOs [aydar.zaynutdinov] [BEAM-13357][Playground] [noreply] Merge pull request #16117 from [BEAM-13368][Playground][Bugfix] Fix CI [noreply] Run python SpannerIO IT with python 3.7 only to avoid overload spanner [noreply] Merge pull request #15378 from [RFC] Define and document per-key [noreply] Bump python containers to beam-master-20211202 (#16129) [Luke Cwik] [BEAM-13015] Simplify fake state API testing client and make it [noreply] [BEAM-13354, BEAM-13015, BEAM-12802, BEAM-12588] Support prefetch for [zyichi] Fix failing RecommendationAICatalogItemIT [Valentyn Tymofieiev] Clarify instructions on how to get contributor list. [noreply] [BEAM-11936] Fix errorprone UnusedVariable in IO (#16036) [zyichi] [BEAM-13380] Fix broken seedjob. [noreply] Merge pull request #16118 from [BEAM-13362][Playground] python ci cd [slebedev] [BEAM-13384] Re-exported the metrics subpackage in [aydar.zaynutdinov] [BEAM-13357][Playground] [noreply] Merge pull request #16137 from [BEAM-13381][Playground][Bugfix] Update [noreply] Merge pull request #16107 from [BEAM-13332] [Playground] Add logs output [noreply] Merge pull request #16109 from [BEAM-13330][Playground] Save Java logs [noreply] Merge pull request #16125 from [BEAM-13322][Playground] Support Java [zyichi] [BEAM-13388] Fix broken google cloud dlp test. [Robert Bradshaw] Better type hints for Count combiners. [melissapa] [BEAM-11758] Update basics page: Window, Watermark [relax] don't store unserializable DatasetService as a member variable [noreply] [BEAM-13236] Properly close kinesis producer on teardown (#15955) [noreply] Merge pull request #16150 from [BEAM-13396][Playground][Bugfix] Issues [noreply] Merge pull request #16148 from [BEAM-13394][Playground] [Bugfix] Fix [noreply] Avoid overriding explicit portable job submission disabling. (#16143) [noreply] Merge pull request #16151 from [BEAM-13350][Playground] Support running [zyichi] [BEAM-13373] Increase python post commit timeout to reduce chance of [zyichi] [BEAM-13401] Wait for worker start to reduce flakiness in [melissapa] [BEAM-11758] Final cleanup for Beam Basics doc content [msbukal] Exclude FhirIOPatientEverything from v2 dataflow runner intg test. [noreply] [BEAM-13371] Fix bug where DataFrame overview snippets don't show up [dpcollins] Add a workaround for https://github.com/googleapis/gax-java/issues/1577 [noreply] [BEAM-12976] Implement pipeline visitor to get global field access in… [noreply] [BEAM-13388] Use 3.0.0 as lower bound for google-cloud-dlp (#16164) [noreply] Merge pull request #16127 from [BEAM-13366] [Playground] Add support [stranniknm] fix playground frontend licences [noreply] Merge pull request #16167 from [BEAM-13409][Playground] [Bugfix] Change [noreply] Merge pull request #16136 from [BEAM-13365] [Playground] Add Pipelines [noreply] [BEAM-13244] Support STS Assume role credentials provider for AWS SDK v2 [noreply] [BEAM-11936] Fix errorprone UnusedVariable in core,examples,harness.. [noreply] [BEAM-11936] Remove suppressUnusedVariable flag (#16171) [zyichi] Fix pytest unknown markers warning. [noreply] [BEAM-13090] Adding SDK harness container overrides option to Java SDK [noreply] [BEAM-11936] Fix errorprone warnings (#15890) [Kyle Weaver] Include name of missing tag in error message. [noreply] [BEAM-13015] Start integrating a process wide cache. (#16130) [aydar.zaynutdinov] [BEAM-13424][Playground] [aydar.zaynutdinov] [BEAM-13424][Playground] [stranniknm] [BEAM-13426] clear log and output panels on example change [aydar.zaynutdinov] [BEAM-13428][Playground] [aydar.zaynutdinov] [BEAM-13424][Playground] [daria.malkova] add type of precompiled object [noreply] [BEAM-11936] Remove suppression in ReadFromKafkaDoFn (#16174) [noreply] [BEAM-12561] method truncate on series and dataframe (#15833) [noreply] [BEAM-13294] Widen key schema for all keys before use (#16158) [noreply] [BEAM-13171] Support for stopReadTime on KafkaIO SDF (#15951) [noreply] [BEAM-13399] Add functionality to download Beam JARs from Maven [noreply] [BEAM-13402] Add another workaround for [stranniknm] [BEAM-13423]: fix frontend failure if no examples [Alexey Romanenko] [BEAM-13171] Fix Duration ambiguity [daria.malkova] change return type of 2 methods [mmack] [BEAM-13441] Use quiet delete for S3 batch deletes. In quiet mode only [mmack] [BEAM-13410][BEAM-11788] Integration test for S3Filesystem using [Alexey Romanenko] [BEAM-13434] Bump up Apache log4j2 vulnerability to 2.15.0 [noreply] Merge pull request #16138 from [BEAM-13343][Playground] Support go unit [noreply] [BEAM-12560] Dataframe idxmin and idxmax implementation (#15827) [noreply] [BEAM-12565] Dataframe compare implementation (#16027) [noreply] [BEAM-12976] Use a map to pass all pushdown requests at once. (#16189) [noreply] Updating Grafana from v8.1.2 to v8.1.6 [noreply] [BEAM-12683] Fix failing integration tests for Python Recommendation AI [noreply] [BEAM-13015] Plumb the cache through contexts and transform executors. [dpcollins] [BEAM-13402] Simplify PubsubLiteSink [daria.malkova] Docs for validators tests [daria.malkova] change context type [noreply] Merge pull request #16140 from [BEAM-13377][Playground] Update CI/CD [noreply] Merge pull request #16120 from [BEAM-13333][Playground] Save Python logs [noreply] Merge pull request #16185 from [BEAM-13425][Playground][Bugfix] Support [mmack] [BEAM-13445] Correctly set data limit when flushing S3 upload buffer and [noreply] Merge pull request #16121 from [BEAM-13334][Playground] Save Go logs to [aydar.zaynutdinov] [BEAM-13438][Playground] [noreply] Merge pull request #16179 from [BEAM-13344][Playground] support python [noreply] Merge pull request #16208 from [BEAM-13442][Playground] Filepath to log [stranniknm] [BEAM-13446]: fix incorrect tab symbol [noreply] [BEAM-13276] bump jackson-core to 2.13.0 for .test-infra (#16062) [noreply] Change Pub/Sub Lite PollResult to set explicit watermark (#16216) [noreply] Removes the comment that seems no longer relevant. [noreply] [BEAM-13454] Fix and test dataframe read_fwf. (#16064) [Valentyn Tymofieiev] Update a few dependencies that may depend on log4j transitively. [noreply] [BEAM-12976] Pipeline visitor to discover pushdown opportunities. [noreply] [BEAM-13015] Allow decoding a set of elements until we hit the block [noreply] [BEAM-13159] Add Redis Stream (XADD) Write Support (#15858) [noreply] Merge pull request #15994: [BEAM-13263] Support OnWindowExpiration in [noreply] Bump dataflow container version to beam-master-20211213 (#16213) [noreply] Merge pull request #16198 from [BEAM-13437][Playground] Add [noreply] Merge pull request #16195 from a[BEAM-13436][Playground] Add [noreply] [BEAM-13355] add Big Query parameter to enable users to specify load_… [noreply] Updated the base images to use debian:bullseye (#16221) [noreply] [BEAM-13434] Bump log4j to 2.16.0. (#16237) [mmack] [BEAM-13209] Fix DynamoDBIO.write to properly handle partial success [mmack] [BEAM-13209] Fix DynamoDBIO.write to properly handle partial success [stranniknm] [BEAM-13466]: sort categories and examples by name [noreply] [BEAM-13341] Serialize reflection parameter in AvroCoder cloud object [noreply] Update grafana from 8.1.6 to 8.1.8 [noreply] [BEAM-13015] Update FakeBeamFnStateClient to generate elements that stop [noreply] [BEAM-13218] Sickbay [noreply] [BEAM-13399] Add infrastructure to start JARs from Go functions (#16214) [noreply] Merge pull request #16061 from [BEAM-13428] [Playground] Integrate [noreply] Clarify CoGroupByKey creates Iterable, not list. (#16099) [noreply] [BEAM-12931] Allow for DoFn#getAllowedTimestampSkew() when checking the [noreply] [BEAM-13467] Properly handle null argument types for logical types. [noreply] [BEAM-10277] Initial implementation for encoding position in Python [noreply] [BEAM-11545] State & timer for batched RPC calls pattern (#13643) [noreply] Automatically prune local images before building an RC. (#16238) [noreply] Add verbose error messages to container-related scripts. (#16056) [noreply] [BEAM-13456] Rollback #15890 to fix timeout in Java PostCommit (#16257) [noreply] [BEAM-13015] Add a state backed iterable that can be mutated under [noreply] [BEAM-13388] Update Cloud DLP after breaking changes. (#16236) [rsl4] [BEAM-13397] Bump numpy to 1.21 for M1 chip [noreply] [BEAM-13434] Bump google pubsublite on master. (#16265) [mmack] [adhoc] Fix guava imports in tests [aydar.zaynutdinov] [BEAM-13485][Playground] [alexander.zhuravlev] [BEAM-13476] Changed Timeout Error Text [rsl4] re-generate requirements.txt [alexander.zhuravlev] [BEAM-13474] Changed 'Playground' logo text color [mmack] [BEAM-13443] Avoid blocking put to Kinesis records queue to shutdown [noreply] Merge pull request #16278 from [BEAM-13479] [Playground] Change logic [noreply] Merge pull request #16281 from [BEAM-13475] [Playground] [Bugfix] Error [noreply] Merge pull request #16279 from [BEAM-13473] [Playground] [Bugfix] Reset [noreply] Merge pull request #16232 from [BEAM-13285][Playground] Add steps to [noreply] [BEAM-13399] Add check for dev versions of JARs to download code [noreply] Merge pull request #16122 from [BEAM-13345] [Playground] add resizable [benjamin.gonzalez] [BEAM-13088] Make tempDataset final [noreply] Merge pull request #16170 from [BEAM-13411][Playground] Add getting of [noreply] Merge pull request #16172 from [BEAM-13417] [Playground] Add java [noreply] Merge pull request #16240 from [BEAM-13465][Playground] Change error [noreply] Merge pull request #16234 from [BEAM-13461][Playground] [Bugfix] Error [noreply] Merge pull request #15489 from [BEAM-12865] Allow custom batch_duration [noreply] [BEAM-13483] Increase timeout of Java Examples Dataflow suite. (#16226) [Kyle Weaver] [BEAM-13496] Upgrade Flink runner to include log4j patches. [Kyle Weaver] [BEAM-13497] Correct class name in Flink tests. [noreply] Pin transitive log4j dependencies to 2.17.0 in :sdks:java:io:hcatalog [noreply] [BEAM-13434] Bump solr to 8.11.1 (#16296) [Kyle Weaver] [BEAM-13498] Sickbay portable Flink testProcessElementSkew. [noreply] Use a patched shadow 6.1.0 plugin using Log4j 2.16.0 (#16269) [noreply] [BEAM-12830] Replace GoGradle plugin with Shell Scripts. (#16291) [stranniknm] [BEAM-13502]: fix loading example on embedded version [noreply] [BEAM-13430] Start the process of upgrading the Gradle 7. (#16292) [noreply] [BEAM-13430] Introduce new testRuntimeMigration configuration that [noreply] Update flink cluster to use a supported dataproc version (1.2 -> 1.5) [Robert Bradshaw] Add microbenchmark for row coder. [noreply] [BEAM-13499] Add warning about hcatalog to release notes and javadoc [Robert Bradshaw] Compile RowCoderImpl with Cython. [Robert Bradshaw] RowCoder optimizations. [Robert Bradshaw] Further optimizations for common case of components in order. [noreply] [BEAM-13430] Remove propdeps and replace with compileOnly (#16308) [Robert Bradshaw] Optimize rows with null fields. [noreply] Merge pull request #16304 from [BEAM-13491] [Playground] Examples' [noreply] Merge pull request #16283 from [BEAM-13448] [Playground] track run code [noreply] Merge pull request #16244 from [BEAM-13463] [Playground] add retries to [noreply] Merge pull request #16241 from [BEAM-13440] [Playground] Implement [noreply] [BEAM-13399] Add integration test for Go SDK expansion service JAR [noreply] [BEAM-13421] Fix bug with xs called with non-tuple key (#16258) [Robert Bradshaw] RowCoder lint. [mmack] [BEAM-13494] Avoid draining Kinesis API limits & CPU when reading from [alexander.zhuravlev] [BEAM-13489] Updated manifest.json and index.html to follow best [arietis27] [BEAM-13517] Unable to write nulls to columns with logical types [stranniknm] [BEAM-13501] playground - add log about processing and precompiled run [noreply] Make S3 streaming more efficient (#15931) [noreply] [Playground][BEAM-13358]Deploy Go service via Ci/CD (#16210) [dpcollins] [BEAM-13430] Clean up tests that override the time of the JVM [noreply] [BEAM-12830] Install go version w/host platform. (#16330) [noreply] [BEAM-13399] Add extra time buffer to reduce likelihood of flake in [noreply] [BEAM-12865] Fix triggering_frequency validation (#16315) [noreply] [BEAM-13525] Sickbay new PardoTests for Dataflow V2 Java Streaming VR [noreply] [BEAM-12830] Avoid flock and prepare task once. (OSX fix.) (#16333) [noreply] Bump Python beam-master containers (#16332) [arietis27] [BEAM-13517] Unable to write nulls to columns with logical types [stranniknm] [BEAM-13531] playground - fetch example info only once [mmack] [BEAM-7559] Add integration test for DynamoDBIO (SDK v2) [mmack] [BEAM-7559] Add integration test for DynamoDBIO (SDK v1) [mmack] [adhoc] Forbid to import guava and others from org.testcontainers.shaded [alexander.zhuravlev] [BEAM-13530] Fixed scrollbar in the Output section [noreply] [BEAM-13399] Disable expansion service integration test on Samza and [noreply] [BEAM-13430] Upgrade beam to gradle 7.3.2 (#16319) [Kyle Weaver] [BEAM-13521] flink pvr batch precommit [noreply] Merge pull request #16320 from [BEAM-13512] [Playground] send request to [noreply] Merge pull request #16209 from [Playground][BEAM-13304]Deploy Python [noreply] Merge pull request #16105 from [BEAM-13328] [Playground] Add [noreply] Merge pull request #16103 from [BEAM-13252] [Playground] Feedback Thumbs [noreply] Better type inference for GroupBy. (#16318) [noreply] [BEAM-13534] Update TestAutomatedExpansionService liveness check to not [noreply] [BEAM-13490] Bump log4j version to 2.17.0 (#16276) [noreply] Merge pull request #16355 from [BEAM-13529] [Playground] [Bugfix] Fix [noreply] Fixed grammar error in Go ParDo function comments. (#16329) [noreply] Merge pull request #16285 from [BEAM-13492][Playground] Update backend [noreply] [BEAM-13526] Kafka.IO: make DeserializerProvider a public interface [Kyle Weaver] [BEAM-13571] Fix ClassNotFound exception in Flink tests [Kyle Weaver] [BEAM-13498] [BEAM-13573] exclude new tests on Flink [noreply] Exclude UsesOnWindowExpiration by category from Dataflow v2 streaming [noreply] [BEAM-13052] Increment pubsub python version and fix breakages. (#16126) [noreply] [BEAM-13052] Add Pub/Sub Lite xlang transforms in python (#15727) [noreply] [BEAM-13402] Version bump Pub/Sub Lite and implement changes to ensure ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on apache-beam-jenkins-7 (beam) in workspace <https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Direct_Java17/ws/> The recommended git tool is: NONE No credentials specified Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/apache/beam.git > git init > <https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Direct_Java17/ws/src> > # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git --version # timeout=10 > git --version # 'git version 2.7.4' > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # > timeout=10 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # > timeout=10 > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision b33bebcda54749bc93cc18cb6c3ea03862656b13 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f b33bebcda54749bc93cc18cb6c3ea03862656b13 # timeout=10 Commit message: "Merge pull request #16215 from dpcollins-google/publish-no-desync" > git rev-list --no-walk 33bcdf6f7e7adffdddc193ea52f9551d5f526b33 # timeout=10 No emails were triggered. [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content SPARK_LOCAL_IP=127.0.0.1 SETUPTOOLS_USE_DISTUTILS=stdlib [EnvInject] - Variables injected successfully. [Gradle] - Launching build. [src] $ <https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Direct_Java17/ws/src/gradlew> :runners:direct-java:setJavaSDK8AsCompiler :runners:direct-java:shadowJar :runners:direct-java:shadowTestJar Starting a Gradle Daemon, 1 busy Daemon could not be reused, use --status for details Configuration on demand is an incubating feature. > Task :buildSrc:compileJava NO-SOURCE > Task :buildSrc:compileGroovy FROM-CACHE > Task :buildSrc:pluginDescriptors > Task :buildSrc:processResources > Task :buildSrc:classes > Task :buildSrc:jar > Task :buildSrc:assemble > Task :buildSrc:spotlessGroovy FROM-CACHE > Task :buildSrc:spotlessGroovyCheck UP-TO-DATE > Task :buildSrc:spotlessGroovyGradle FROM-CACHE > Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE > Task :buildSrc:spotlessCheck UP-TO-DATE > Task :buildSrc:pluginUnderTestMetadata > Task :buildSrc:compileTestJava NO-SOURCE > Task :buildSrc:compileTestGroovy NO-SOURCE > Task :buildSrc:processTestResources NO-SOURCE > Task :buildSrc:testClasses UP-TO-DATE > Task :buildSrc:test NO-SOURCE > Task :buildSrc:validatePlugins FROM-CACHE > Task :buildSrc:check UP-TO-DATE > Task :buildSrc:build FAILURE: Build failed with an exception. * What went wrong: Task 'setJavaSDK8AsCompiler' not found in project ':runners:direct-java'. * Try: > Run gradlew tasks to get a list of available tasks. > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 37s 10 actionable tasks: 4 executed, 4 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/y2eq3nmimmtng Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
