See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow_V2_Java17/1904/display/redirect?page=changes>
Changes:
[noreply] update documentation (#26201)
------------------------------------------
[...truncated 561.00 KB...]
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:RESHUFFLE; numEvents:10000000
0015 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0016 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 *** not run ***
0001 19.0 525817.6 9200000
0002 18.8 530898.3 83527
0003 40.4 247279.9 59507
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
0015 *** not run ***
0016 *** not run ***
==========================================================================================
2023-04-12T15:48:33.653Z DONE Query8
2023-04-12T15:48:33.680Z 1 samples not enough to calculate steady-state event
rate
2023-04-12T15:48:33.680Z streaming query appears to have finished waiting for
completion.
Apr 12, 2023 3:48:33 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query8
==========================================================================================
Run started 2023-04-12T15:42:54.369Z and ran for PT339.312S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:RESHUFFLE; numEvents:10000000
0015 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0016 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 *** not run ***
0001 19.0 525817.6 9200000
0002 18.8 530898.3 83527
0003 40.4 247279.9 59507
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 59.3 168679.6 92833
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
0015 *** not run ***
0016 *** not run ***
==========================================================================================
2023-04-12T15:48:33.875Z 1 samples not enough to calculate steady-state event
rate
2023-04-12T15:48:33.914Z RUNNING Query5
2023-04-12T15:48:33.914Z streaming query appears to have finished waiting for
completion.
2023-04-12T15:48:33.914Z no activity
Apr 12, 2023 3:48:33 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query0
2023-04-12T15:48:33.957Z RUNNING Query7
2023-04-12T15:48:33.984Z 1 samples not enough to calculate steady-state event
rate
2023-04-12T15:48:33.988Z 1 samples not enough to calculate steady-state event
rate
Apr 12, 2023 3:48:34 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query11
Apr 12, 2023 3:48:34 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query12
2023-04-12T15:48:34.043Z streaming query appears to have finished waiting for
completion.
2023-04-12T15:48:34.043Z no activity
2023-04-12T15:48:34.043Z new perf
{"snapshots":null,"runtimeSec":159.554,"numEvents":10000000,"eventsPerSec":62674.70574225654,"eventBytesPerSec":8044705.222056482,"numResults":202346,"resultsPerSec":1268.1976008122642,"resultBytesPerSec":20291.161612996228,"startupDelaySec":-102.958,"processingDelaySec":150.714,"shutdownDelaySec":5.829,"timeDilation":6.267157200696943,"errors":null,"jobId":null}
2023-04-12T15:48:34.063Z 1 samples not enough to calculate steady-state event
rate
2023-04-12T15:48:34.097Z 5 samples not enough to calculate steady-state event
rate
Apr 12, 2023 3:48:34 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
2023-04-12T15:48:34.123ZSEVERE: Failed to get metric fatal, from namespace
Query10
no activity
Apr 12, 2023 3:48:34 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query14
2023-04-12T15:48:34.140Z new perf
{"snapshots":null,"runtimeSec":152.293,"numEvents":9117081,"eventsPerSec":59865.397621689764,"eventBytesPerSec":7684292.396892832,"numResults":8734998,"resultsPerSec":57356.529847071106,"resultBytesPerSec":7362309.48237936,"startupDelaySec":-98.132,"processingDelaySec":17.884,"shutdownDelaySec":8.344,"timeDilation":6.566224317598314,"errors":null,"jobId":null}
2023-04-12T15:48:34.514Z 1 samples not enough to calculate steady-state event
rate
2023-04-12T15:48:34.533Z 2 samples not enough to calculate steady-state event
rate
Apr 12, 2023 3:48:34 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query5
2023-04-12T15:48:34.575Z new perf
{"snapshots":null,"runtimeSec":156.916,"numEvents":10000000,"eventsPerSec":63728.36422034719,"eventBytesPerSec":8179949.125646843,"numResults":435,"resultsPerSec":2.7721838435851027,"resultBytesPerSec":44.354941497361644,"startupDelaySec":-111.472,"processingDelaySec":156.631,"shutdownDelaySec":16.947,"timeDilation":0.5735552779831248,"errors":null,"jobId":null}
Apr 12, 2023 3:48:34 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
2023-04-12T15:48:34.587Z no activity
SEVERE: Failed to get metric fatal, from namespace Query7
2023-04-12T15:48:48.953Z DONE Query0 (waiting for shutdown)
2023-04-12T15:48:48.974Z 1 samples not enough to calculate steady-state event
rate
==========================================================================================
Run started 2023-04-12T15:42:54.369Z and ran for PT354.605S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:RESHUFFLE; numEvents:10000000
0015 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0016 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 37.3 268449.2 10000000
0001 19.0 525817.6 9200000
0002 18.8 530898.3 83527
0003 40.4 247279.9 59507
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 59.3 168679.6 92833
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 *** not run ***
0014 *** not run ***
0015 *** not run ***
0016 *** not run ***
==========================================================================================
2023-04-12T15:48:49.088Z RUNNING Query11
2023-04-12T15:48:49.092Z RUNNING Query12 (waiting for shutdown)
2023-04-12T15:48:49.165Z RUNNING Query10
2023-04-12T15:48:49.205Z RUNNING Query14
2023-04-12T15:48:49.614Z RUNNING Query5
2023-04-12T15:48:49.628Z RUNNING Query7
2023-04-12T15:48:49.671Z 2 samples not enough to calculate steady-state event
rate
2023-04-12T15:48:49.687Z 1 samples not enough to calculate steady-state event
rate
2023-04-12T15:48:49.688Z no activity
Apr 12, 2023 3:48:49 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query11
2023-04-12T15:48:49.719Z new perf
{"snapshots":null,"runtimeSec":174.563,"numEvents":10000000,"eventsPerSec":57285.90823943219,"eventBytesPerSec":7353018.090889823,"numResults":1385378,"resultsPerSec":7936.2636984928085,"resultBytesPerSec":126980.21917588494,"startupDelaySec":-102.958,"processingDelaySec":150.714,"shutdownDelaySec":6.447,"timeDilation":5.728304394402022,"errors":null,"jobId":null}
2023-04-12T15:48:49.734Z 6 samples not enough to calculate steady-state event
rate
2023-04-12T15:48:49.760Z 1 samples not enough to calculate steady-state event
rate
Apr 12, 2023 3:48:49 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query14
2023-04-12T15:48:49.807Z new perf
{"snapshots":null,"runtimeSec":166.387,"numEvents":9630490,"eventsPerSec":57880.06274528659,"eventBytesPerSec":7429389.627795441,"numResults":9303593,"resultsPerSec":55915.384014376126,"resultBytesPerSec":7177271.084880429,"startupDelaySec":-98.132,"processingDelaySec":17.884,"shutdownDelaySec":9.899,"timeDilation":6.01002482165073,"errors":null,"jobId":null}
Apr 12, 2023 3:48:49 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query10
2023-04-12T15:48:49.842Z streaming query appears to have finished waiting for
completion.
2023-04-12T15:48:49.842Z no activity
2023-04-12T15:48:50.341Z 2 samples not enough to calculate steady-state event
rate
2023-04-12T15:48:50.376Z 2 samples not enough to calculate steady-state event
rate
Apr 12, 2023 3:48:50 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
2023-04-12T15:48:50.403Z no activity
SEVERE: Failed to get metric fatal, from namespace Query7
Apr 12, 2023 3:48:50 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query5
2023-04-12T15:48:50.436Z new perf
{"snapshots":null,"runtimeSec":175.332,"numEvents":10000000,"eventsPerSec":57034.6542559259,"eventBytesPerSec":7320768.011543814,"numResults":1473,"resultsPerSec":8.401204571897885,"resultBytesPerSec":134.41927315036617,"startupDelaySec":-111.472,"processingDelaySec":156.631,"shutdownDelaySec":14.224,"timeDilation":1.9106609175735176,"errors":null,"jobId":null}
FATAL: command execution failed
hudson.remoting.ChannelClosedException: Channel
"hudson.remoting.Channel@1e856394:apache-beam-jenkins-13": Remote call on
apache-beam-jenkins-13 failed. The channel is closing down or has closed down
at hudson.remoting.Channel.call(Channel.java:993)
at
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
at com.sun.proxy.$Proxy139.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1215)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1207)
at hudson.Launcher$ProcStarter.join(Launcher.java:524)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:814)
at hudson.model.Build$BuildExecution.build(Build.java:199)
at hudson.model.Build$BuildExecution.doRun(Build.java:164)
at
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:522)
at hudson.model.Run.execute(Run.java:1896)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:442)
Caused by: java.io.IOException: Pipe closed after 0 cycles
at
org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:126)
at
org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:105)
at
hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:94)
at
hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:75)
at
hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:105)
at
hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
at
hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
FATAL: Channel "hudson.remoting.Channel@1e856394:apache-beam-jenkins-13":
Remote call on apache-beam-jenkins-13 failed. The channel is closing down or
has closed down
java.io.IOException: Pipe closed after 0 cycles
at
org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:126)
at
org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:105)
at
hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:94)
at
hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:75)
at
hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:105)
at
hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
at
hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Caused: hudson.remoting.ChannelClosedException: Channel
"hudson.remoting.Channel@1e856394:apache-beam-jenkins-13": Remote call on
apache-beam-jenkins-13 failed. The channel is closing down or has closed down
at hudson.remoting.Channel.call(Channel.java:993)
at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1150)
at
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
at hudson.model.Run.execute(Run.java:1896)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:442)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
