See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1335/display/redirect?page=changes>
Changes: [noreply] Update WriteToBigQuery multiple destinations doc [Ismaël Mejía] [BEAM-11948] Drop support for Flink 1.8 and 1.9 [randomstep] [BEAM-11900] Bump libthrift to 0.14.0 [randomstep] [BEAM-11900] Bump libthrift to 0.14.1 [Kyle Weaver] [BEAM-10925] Java UDF type tests for input refs. [randomstep] [BEAM-12066] Bump classgraph to 4.8.104 [Kyle Weaver] [BEAM-12102] Catch and rethrow Calcite CannotPlanException. [kawaigin] [BEAM-10708] Read/Write Intermediate PCollections [kawaigin] Fix lint [kawaigin] Fix based on comments [kawaigin] Added clear method to InMemoryCache because tests might be flaky when a [noreply] [BEAM-11961] InfluxDBIOIT failing with unauthorized error (#14215) [noreply] Add DataFrame API changes to CHANGES.md (#14454) [noreply] Fix: Allow BigQuery tableIds with hyphens (#14125) [noreply] Merge pull request #14394 from [BEAM-11277] Add method to check if two [kawaigin] Avoid using interactive_environment module in the test because [noreply] [BEAM-449] Support PCollectionList in PAssert (#14322) [kawaigin] [BEAM-11045] Avoid broken deps [kawaigin] Added back the setUp as additional cleanup routine before each test. [noreply] [BEAM-11742] Use pyarrow schema instead column names when creating [noreply] [BEAM-7372] remove usage of future package and unnecessary builtins [noreply] [BEAM-7372] cleanup codes for py2 compatibility from ------------------------------------------ [...truncated 4.41 MB...] INFO: 2021-04-08T11:07:22.033Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow Apr 08, 2021 11:07:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:07:22.064Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign Apr 08, 2021 11:07:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:07:22.100Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles Apr 08, 2021 11:07:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:07:22.142Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify Apr 08, 2021 11:07:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:07:22.230Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read Apr 08, 2021 11:07:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:07:22.284Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow Apr 08, 2021 11:07:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:07:22.369Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten Apr 08, 2021 11:07:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:07:22.787Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create Apr 08, 2021 11:07:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:07:22.862Z: Starting 1 workers in us-central1-f... Apr 08, 2021 11:07:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:07:23.132Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create Apr 08, 2021 11:07:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:07:23.290Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write Apr 08, 2021 11:07:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:07:43.375Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Apr 08, 2021 11:08:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:08:08.348Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s). Apr 08, 2021 11:08:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:08:36.255Z: Workers have started successfully. Apr 08, 2021 11:08:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-08T11:08:36.286Z: Workers have started successfully. [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project word-count-beam: An exception occured while executing the Java class. unable to create new native thread -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] Failed command > Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom FAILED > Task :runners:google-cloud-dataflow-java:runQuickstartJavaDataflow Repository URL: https://repository.apache.org/content/repositories/snapshots Version: 2.30.0-SNAPSHOT GCS Project: apache-beam-testing GCS Region: us-central1 GCS Storage bucket: temp-storage-for-release-validation-tests/nightly-snapshot-validation ************************************** * Scenario: Run Apache Beam Java SDK Quickstart - Dataflow ************************************** ************************************** * Test: Gets the WordCount Example Code ************************************** mvn archetype:generate --update-snapshots -DarchetypeGroupId=org.apache.beam -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples -DarchetypeVersion=2.30.0-SNAPSHOT -DgroupId=org.example -DartifactId=word-count-beam -Dversion="0.1" -Dpackage=org.apache.beam.examples -DinteractiveMode=false Using maven /home/jenkins/tools/maven/apache-maven-3.5.4 Exception in thread "main" java.io.IOException: Cannot run program "sh" (in directory "/tmp/groovy-generated-7779555504940169856-tmpdir"): error=11, Resource temporarily unavailable at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) at java_lang_ProcessBuilder$start$1.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117) at TestScripts._execute(TestScripts.groovy:168) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrap.invoke(PogoMetaMethodSite.java:190) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166) at TestScripts._mvn(TestScripts.groovy:221) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166) at TestScripts.run(TestScripts.groovy:126) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrap.invoke(PogoMetaMethodSite.java:190) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) at QuickstartArchetype.generate(QuickstartArchetype.groovy:27) at QuickstartArchetype$generate.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) at quickstart-java-dataflow.run(quickstart-java-dataflow.groovy:29) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:925) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:908) at org.codehaus.groovy.runtime.InvokerHelper.runScript(InvokerHelper.java:412) at org.codehaus.groovy.runtime.InvokerHelper$runScript.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133) at quickstart-java-dataflow.main(quickstart-java-dataflow.groovy) Caused by: java.io.IOException: error=11, Resource temporarily unavailable at java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:247) at java.lang.ProcessImpl.start(ProcessImpl.java:134) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ... 56 more > Task :runners:google-cloud-dataflow-java:runQuickstartJavaDataflow FAILED > Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow Repository URL: https://repository.apache.org/content/repositories/snapshots Version: 2.30.0-SNAPSHOT GCS Project: apache-beam-testing GCS Region: us-central1 GCS Storage bucket: temp-storage-for-release-validation-tests/nightly-snapshot-validation BigQuery Dataset: beam_postrelease_mobile_gaming PubSub Topic: java_mobile_gaming_topic ************************************** * Scenario: Run Apache Beam Java SDK Mobile Gaming Examples - Dataflow ************************************** mvn archetype:generate --update-snapshots -DarchetypeGroupId=org.apache.beam -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples -DarchetypeVersion=2.30.0-SNAPSHOT -DgroupId=org.example -DartifactId=word-count-beam -Dversion="0.1" -Dpackage=org.apache.beam.examples -DinteractiveMode=false Using maven /home/jenkins/tools/maven/apache-maven-3.5.4 Exception in thread "main" java.io.IOException: Cannot run program "sh" (in directory "/tmp/groovy-generated-4563028141628783584-tmpdir"): error=11, Resource temporarily unavailable at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) at java_lang_ProcessBuilder$start$1.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117) at TestScripts._execute(TestScripts.groovy:168) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrap.invoke(PogoMetaMethodSite.java:190) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166) at TestScripts._mvn(TestScripts.groovy:221) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166) at TestScripts.run(TestScripts.groovy:126) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrap.invoke(PogoMetaMethodSite.java:190) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) at QuickstartArchetype.generate(QuickstartArchetype.groovy:27) at QuickstartArchetype$generate.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) at mobilegaming-java-dataflow.run(mobilegaming-java-dataflow.groovy:30) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:925) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:908) at org.codehaus.groovy.runtime.InvokerHelper.runScript(InvokerHelper.java:412) at org.codehaus.groovy.runtime.InvokerHelper$runScript.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133) at mobilegaming-java-dataflow.main(mobilegaming-java-dataflow.groovy) Caused by: java.io.IOException: error=11, Resource temporarily unavailable at java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:247) at java.lang.ProcessImpl.start(ProcessImpl.java:134) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ... 56 more > Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED FAILURE: Build completed with 3 failures. 1: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 2: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 3: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 56m 47s 9 actionable tasks: 8 executed, 1 from cache Publishing build scan... https://gradle.com/s/qpnepsuxemooe Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
