See
<https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/2339/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-2939] Add split_and_size method for SDFs.
[robertwb] Also test registration of standard coders.
------------------------------------------
[...truncated 146.43 KB...]
Apr 08, 2019 1:21:35 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 144 files cached, 15 files newly uploaded in 8
seconds
Apr 08, 2019 1:21:35 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 144 files cached, 15 files newly uploaded in 8
seconds
Apr 08, 2019 1:21:35 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 144 files cached, 15 files newly uploaded in 8
seconds
Apr 08, 2019 1:21:35 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 144 files cached, 15 files newly uploaded in 8
seconds
Apr 08, 2019 1:21:35 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 144 files cached, 15 files newly uploaded in 8
seconds
Apr 08, 2019 1:21:35 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 144 files cached, 15 files newly uploaded in 8
seconds
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query11.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query9.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query10.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query6.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query8.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query1.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BoundedSideInputJoin.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query0.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query12.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query5.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query9/Query9.Monitor as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query8/Query8.Monitor as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query5/Query5.Monitor as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query12/Query12.Monitor as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Monitor as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query11/Query11.Monitor as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/MapElements/Map as step s3
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query10/Query10.Monitor as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query1/Query1.Monitor as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query0/Query0.Monitor as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query6/Query6.Monitor as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query11/Query11.Snoop as step s3
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Snoop as step s3
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query12/Query12.Snoop as step s3
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query5/Query5.Snoop as step s3
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query6/Query6.Snoop as step s3
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query0/Query0.Snoop as step s3
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query1/Query1.Snoop as step s3
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query9/Query9.Snoop as step s3
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BoundedSideInputJoin/BoundedSideInputJoin.Monitor as step s4
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query8/Query8.Snoop as step s3
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query10/Query10.Snoop as step s3
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BoundedSideInputJoin/BoundedSideInputJoin.Snoop as step s5
==========================================================================================
Run started 2019-04-08T13:21:25.780Z and ran for PT10.379S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
Exception in thread "main" 0008 query:MONITOR_NEW_USERS;
exportSummaryToBigQuery:true; numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 *** not run ***
0001 *** not run ***
0002 *** not run ***
0003 *** not run ***
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
==========================================================================================
java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:127)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:414)
Caused by: java.lang.NullPointerException
at
org.apache.beam.runners.core.construction.SchemaTranslation.toProto(SchemaTranslation.java:57)
at
org.apache.beam.runners.core.construction.SchemaTranslation.toProto(SchemaTranslation.java:85)
at
org.apache.beam.runners.core.construction.SchemaTranslation.toProto(SchemaTranslation.java:73)
at
org.apache.beam.runners.core.construction.SchemaTranslation.toProto(SchemaTranslation.java:60)
at
org.apache.beam.runners.dataflow.util.SchemaCoderCloudObjectTranslator.toCloudObject(SchemaCoderCloudObjectTranslator.java:55)
at
org.apache.beam.runners.dataflow.util.SchemaCoderCloudObjectTranslator.toCloudObject(SchemaCoderCloudObjectTranslator.java:31)
at
org.apache.beam.runners.dataflow.util.CloudObjects.asCloudObject(CloudObjects.java:69)
at
org.apache.beam.runners.dataflow.util.CloudObjectTranslators.addComponents(CloudObjectTranslators.java:59)
at
org.apache.beam.runners.dataflow.util.CloudObjectTranslators.access$000(CloudObjectTranslators.java:51)
at
org.apache.beam.runners.dataflow.util.CloudObjectTranslators$6.toCloudObject(CloudObjectTranslators.java:250)
at
org.apache.beam.runners.dataflow.util.CloudObjectTranslators$6.toCloudObject(CloudObjectTranslators.java:245)
at
org.apache.beam.runners.dataflow.util.CloudObjects.asCloudObject(CloudObjects.java:69)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator.translateCoder(DataflowPipelineTranslator.java:1180)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator.access$600(DataflowPipelineTranslator.java:120)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$StepTranslator.addOutput(DataflowPipelineTranslator.java:707)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$StepTranslator.addOutput(DataflowPipelineTranslator.java:653)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$8.translateSingleHelper(DataflowPipelineTranslator.java:986)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$8.translate(DataflowPipelineTranslator.java:967)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$8.translate(DataflowPipelineTranslator.java:964)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.visitPrimitiveTransform(DataflowPipelineTranslator.java:476)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:665)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:317)
at
org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:251)
at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.translate(DataflowPipelineTranslator.java:415)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator.translate(DataflowPipelineTranslator.java:174)
at
org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:740)
at
org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:179)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
at
org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1178)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:89)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:78)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 08, 2019 1:21:36 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 144 files cached, 15 files newly uploaded in 8
seconds
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Apr 08, 2019 1:21:36 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 144 files cached, 15 files newly uploaded in 8
seconds
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.Monitor as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.Snoop as step s3
Apr 08, 2019 1:21:36 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 144 files cached, 15 files newly uploaded in 8
seconds
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BoundedSideInputJoin.ReadBounded as step s1
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
as step s2
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/MapElements/Map as step s3
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BoundedSideInputJoin/BoundedSideInputJoin.Monitor as step s4
Apr 08, 2019 1:21:36 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BoundedSideInputJoin/BoundedSideInputJoin.Snoop as step s5
Apr 08, 2019 1:21:37 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 144 files cached, 15 files newly uploaded in 9
seconds
Apr 08, 2019 1:21:37 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query2.ReadBounded as step s1
Apr 08, 2019 1:21:37 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query2/Query2.Monitor as step s2
Apr 08, 2019 1:21:37 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query2/Query2.Snoop as step s3
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java''
> finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 49s
81 actionable tasks: 55 executed, 26 from cache
Publishing build scan...
Publishing failed.
The response from https://scans-in.gradle.com/in/5.2.1/2.1 was not from the
build scan server.
Your network environment may be interfering, or the service may be unavailable.
If you believe this to be in error, please report this problem via
https://gradle.com/scans/help/plugin and include the following via copy/paste:
----------
Gradle version: 5.2.1
Plugin version: 2.1
Request URL: https://scans-in.gradle.com/in/5.2.1/2.1
Request ID: c74ebc27-45e6-4b66-a51d-db170efadefa
Response status code: 503
Response content type: text/html; charset=utf-8
Response server type: cloudflare
----------
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
