See
<https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/76/display/redirect>
------------------------------------------
[...truncated 308.59 KB...]
Mar 05, 2018 11:38:44 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
removeTemporaryFiles
INFO: Will remove known temporary file
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/.temp-beam-2018-03-05_23-38-33-1/5e2ba93e-ea06-4eab-b80b-62d62efdc505
Mar 05, 2018 11:38:44 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
removeTemporaryFiles
INFO: Will remove known temporary file
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/.temp-beam-2018-03-05_23-38-33-1/90ca0791-456b-4845-b925-382ddb758c20
Mar 05, 2018 11:38:44 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
removeTemporaryFiles
INFO: Will remove known temporary file
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/.temp-beam-2018-03-05_23-38-33-1/118e4787-7769-4f0a-808d-b6c1433c5630
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:17 min
[INFO] Finished at: 2018-03-05T23:38:44Z
[INFO] Final Memory: 44M/966M
[INFO] ------------------------------------------------------------------------
grep user19_BananaWallaby java-userscore-result.txt*
java-userscore-result.txt
-00002-of-00003:total_score: 231, user: user19_BananaWallaby
Verified total_score: 231, user: user19_BananaWallaby
*****
* Test: SUCCEED: UserScore successfully run on DirectRunners.
*****
*****
* Test: Running: HourlyTeamScore example on DirectRunner
*****
mvn compile exec:java
-Dexec.mainClass=org.apache.beam.examples.complete.game.HourlyTeamScore
-Dexec.args="--tempLocation=gs://temp-storage-for-release-validation-tests/mobile-gaming/tmp
--output=java-hourlyteamscore-result.txt
--input=gs://temp-storage-for-release-validation-tests/mobile-gaming/5000_gaming_data.csv"
-Pdirect-runner
Using maven /home/jenkins/tools/maven/apache-maven-3.5.2
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building word-count-beam 0.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
word-count-beam ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources,
i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @
word-count-beam ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ word-count-beam ---
Mar 05, 2018 11:39:00 PM org.apache.beam.sdk.io.FileBasedSource
getEstimatedSizeBytes
INFO: Filepattern
gs://temp-storage-for-release-validation-tests/mobile-gaming/5000_gaming_data.csv
matched 1 files with total size 386400
Mar 05, 2018 11:39:00 PM org.apache.beam.sdk.io.FileBasedSource split
INFO: Splitting filepattern
gs://temp-storage-for-release-validation-tests/mobile-gaming/5000_gaming_data.csv
into bundles of size 96600 took 86 ms and produced 1 files and 4 bundles
Mar 05, 2018 11:39:07 PM
org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer e3c5af47-8643-4494-ba1a-fcb5de085505 for window
[2015-11-17T00:00:00.000Z..2015-11-17T01:00:00.000Z) pane
PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0}
destination null
Mar 05, 2018 11:39:07 PM
org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer 7a8f1282-67a7-4431-8732-2c97af22e990 for window
[2015-11-17T00:00:00.000Z..2015-11-17T01:00:00.000Z) pane
PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0}
destination null
Mar 05, 2018 11:39:07 PM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/.temp-beam-2018-03-05_23-38-56-0/e3c5af47-8643-4494-ba1a-fcb5de085505
Mar 05, 2018 11:39:07 PM
org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer 19a9a28a-b3d1-40a5-a210-fe932bb257c6 for window
[2015-11-17T00:00:00.000Z..2015-11-17T01:00:00.000Z) pane
PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0}
destination null
Mar 05, 2018 11:39:07 PM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/.temp-beam-2018-03-05_23-38-56-0/7a8f1282-67a7-4431-8732-2c97af22e990
Mar 05, 2018 11:39:07 PM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/.temp-beam-2018-03-05_23-38-56-0/19a9a28a-b3d1-40a5-a210-fe932bb257c6
Mar 05, 2018 11:39:07 PM
org.apache.beam.sdk.io.WriteFiles$FinalizeTempFileBundles$FinalizeFn process
INFO: Finalizing 1 file results
Mar 05, 2018 11:39:07 PM
org.apache.beam.sdk.io.WriteFiles$FinalizeTempFileBundles$FinalizeFn process
INFO: Finalizing 1 file results
Mar 05, 2018 11:39:07 PM
org.apache.beam.sdk.io.WriteFiles$FinalizeTempFileBundles$FinalizeFn process
INFO: Finalizing 1 file results
Mar 05, 2018 11:39:07 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
moveToOutputFiles
INFO: Will copy temporary file
FileResult{tempFilename=/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/.temp-beam-2018-03-05_23-38-56-0/19a9a28a-b3d1-40a5-a210-fe932bb257c6,
shard=0, window=[2015-11-17T00:00:00.000Z..2015-11-17T01:00:00.000Z),
paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0,
onTimeIndex=0}} to final location
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/java-hourlyteamscore-result.txt
-2015-11-16 16:00:00.000-2015-11-16 17:00:00.000-0-of-3
Mar 05, 2018 11:39:07 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
moveToOutputFiles
INFO: Will copy temporary file
FileResult{tempFilename=/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/.temp-beam-2018-03-05_23-38-56-0/e3c5af47-8643-4494-ba1a-fcb5de085505,
shard=1, window=[2015-11-17T00:00:00.000Z..2015-11-17T01:00:00.000Z),
paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0,
onTimeIndex=0}} to final location
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/java-hourlyteamscore-result.txt
-2015-11-16 16:00:00.000-2015-11-16 17:00:00.000-1-of-3
Mar 05, 2018 11:39:07 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
moveToOutputFiles
INFO: Will copy temporary file
FileResult{tempFilename=/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/.temp-beam-2018-03-05_23-38-56-0/7a8f1282-67a7-4431-8732-2c97af22e990,
shard=2, window=[2015-11-17T00:00:00.000Z..2015-11-17T01:00:00.000Z),
paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0,
onTimeIndex=0}} to final location
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/java-hourlyteamscore-result.txt
-2015-11-16 16:00:00.000-2015-11-16 17:00:00.000-2-of-3
Mar 05, 2018 11:39:07 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
removeTemporaryFiles
INFO: Will remove known temporary file
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/.temp-beam-2018-03-05_23-38-56-0/7a8f1282-67a7-4431-8732-2c97af22e990
Mar 05, 2018 11:39:07 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
removeTemporaryFiles
INFO: Will remove known temporary file
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/.temp-beam-2018-03-05_23-38-56-0/19a9a28a-b3d1-40a5-a210-fe932bb257c6
Mar 05, 2018 11:39:07 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
removeTemporaryFiles
INFO: Will remove known temporary file
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/.temp-beam-2018-03-05_23-38-56-0/e3c5af47-8643-4494-ba1a-fcb5de085505
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 19.016 s
[INFO] Finished at: 2018-03-05T23:39:07Z
[INFO] Final Memory: 31M/659M
[INFO] ------------------------------------------------------------------------
grep AzureBilby java-hourlyteamscore-result.txt*
java-hourlyteamscore-result.txt
-2015-11-16 16:00:00.000-2015-11-16 17:00:00.000-0-of-3:window_start:
2015-11-16 16:00:00.000, total_score: 2788, team: AzureBilby
Verified total_score: 2788, team: AzureBilby
*****
* Test: SUCCEED: HourlyTeamScore successfully run on DirectRunners.
*****
*****
* Test: Running: LeaderBoard example on DirectRunner
*****
bq rm -f -t beam_postrelease_mobile_gaming.leaderboard_user
bq rm -f -t beam_postrelease_mobile_gaming.leaderboard_team
sleep 20
mvn compile exec:java
-Dexec.mainClass=org.apache.beam.examples.complete.game.injector.Injector
-Dexec.args="apache-beam-testing python-leaderboard-topic none"
Using maven /home/jenkins/tools/maven/apache-maven-3.5.2
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building word-count-beam 0.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
word-count-beam ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources,
i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @
word-count-beam ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ word-count-beam ---
Topic projects/apache-beam-testing/topics/python-leaderboard-topic was created.
Injecting to topic: projects/apache-beam-testing/topics/python-leaderboard-topic
Starting Injector
[+(MagentaKoala, num members: 12, starting at: 1520293161480, expires in: 34,
robot: Robot-1)]
[+(BarnRedBandicoot, num members: 7, starting at: 1520293161480, expires in:
20, robot: null)]
[+(AuburnCockatoo, num members: 14, starting at: 1520293161480, expires in: 25,
robot: null)]
[+(AuburnWombat, num members: 15, starting at: 1520293161480, expires in: 31,
robot: null)]
[+(FuchsiaKangaroo, num members: 16, starting at: 1520293161480, expires in:
29, robot: null)]
[+(BarnRedBandicoot, num members: 8, starting at: 1520293161480, expires in:
21, robot: null)]
[+(AliceBlueEmu, num members: 11, starting at: 1520293161480, expires in: 36,
robot: null)]
[+(FuchsiaQuokka, num members: 5, starting at: 1520293161480, expires in: 27,
robot: null)]
[+(AmaranthBilby, num members: 11, starting at: 1520293161480, expires in: 21,
robot: Robot-2)]
[+(AzureKookaburra, num members: 15, starting at: 1520293161480, expires in:
34, robot: null)]
[+(AmberEchidna, num members: 10, starting at: 1520293161480, expires in: 34,
robot: Robot-0)]
[+(AppleGreenPossum, num members: 10, starting at: 1520293161480, expires in:
20, robot: Robot-1)]
[+(AppleGreenEchidna, num members: 14, starting at: 1520293161481, expires in:
26, robot: null)]
[+(AlmondCockatoo, num members: 10, starting at: 1520293161481, expires in: 32,
robot: Robot-14)]
[+(BananaCockatoo, num members: 10, starting at: 1520293161481, expires in: 24,
robot: null)]
DELAY(423562, 1)
{timestamp_ms=1520292737000}
late data for:
user2_BarnRedBandicoot,BarnRedBandicoot,13,1520292737000,2018-03-05 15:39:21.495
mvn compile exec:java
-Dexec.mainClass=org.apache.beam.examples.complete.game.LeaderBoard
-Dexec.args="--project=apache-beam-testing
--tempLocation=gs://temp-storage-for-release-validation-tests/mobile-gaming/tmp
--dataset=beam_postrelease_mobile_gaming
--topic=projects/apache-beam-testing/topics/java_mobile_gaming_topic
--output=gs://gs://temp-storage-for-release-validation-tests/mobile-gaming/java-leaderboard-result.txt"
-Pdirect-runner
Using maven /home/jenkins/tools/maven/apache-maven-3.5.2
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building word-count-beam 0.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
word-count-beam ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources,
i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @
word-count-beam ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ word-count-beam ---
[WARNING]
java.lang.IllegalArgumentException: Pubsub topic is not in
projects/<project_id>/topics/<topic_name> format:
projects/apache-beam-testing/topics/java_mobile_gaming_topic
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$PubsubTopic.fromPath
(PubsubIO.java:377)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$Read.fromTopic
(PubsubIO.java:635)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$Read.fromTopic
(PubsubIO.java:626)
at org.apache.beam.examples.complete.game.LeaderBoard.main
(LeaderBoard.java:207)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
at java.lang.Thread.run (Thread.java:748)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.911 s
[INFO] Finished at: 2018-03-05T23:39:42Z
[INFO] Final Memory: 21M/297M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java
(default-cli) on project word-count-beam: An exception occured while executing
the Java class. Pubsub topic is not in
projects/<project_id>/topics/<topic_name> format:
projects/apache-beam-testing/topics/java_mobile_gaming_topic -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building word-count-beam 0.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
word-count-beam ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources,
i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory
/tmp/groovy-generated-5435141615528769469-tmpdir/word-count-beam/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @
word-count-beam ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ word-count-beam ---
[WARNING]
java.lang.IllegalArgumentException: Pubsub topic is not in
projects/<project_id>/topics/<topic_name> format:
projects/apache-beam-testing/topics/java_mobile_gaming_topic
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$PubsubTopic.fromPath
(PubsubIO.java:377)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$Read.fromTopic
(PubsubIO.java:635)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$Read.fromTopic
(PubsubIO.java:626)
at org.apache.beam.examples.complete.game.LeaderBoard.main
(LeaderBoard.java:207)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
at java.lang.Thread.run (Thread.java:748)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.911 s
[INFO] Finished at: 2018-03-05T23:39:42Z
[INFO] Final Memory: 21M/297M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java
(default-cli) on project word-count-beam: An exception occured while executing
the Java class. Pubsub topic is not in
projects/<project_id>/topics/<topic_name> format:
projects/apache-beam-testing/topics/java_mobile_gaming_topic -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] Failed command
:runners:direct-java:runMobileGamingJavaDirect FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:direct-java:runMobileGamingJavaDirect'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java''
> finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 2m 56s
2 actionable tasks: 2 executed
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
