See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/7043/display/redirect?page=changes>
Changes:
[noreply] Update WriteToBigQuery multiple destinations doc
[randomstep] [BEAM-11900] Bump libthrift to 0.14.0
[randomstep] [BEAM-11900] Bump libthrift to 0.14.1
[kawaigin] [BEAM-10708] Read/Write Intermediate PCollections
[kawaigin] Fix lint
[kawaigin] Fix based on comments
[kawaigin] Added clear method to InMemoryCache because tests might be flaky
when a
[noreply] Fix: Allow BigQuery tableIds with hyphens (#14125)
[noreply] Merge pull request #14394 from [BEAM-11277] Add method to check if two
[kawaigin] Avoid using interactive_environment module in the test because
[noreply] [BEAM-449] Support PCollectionList in PAssert (#14322)
[kawaigin] [BEAM-11045] Avoid broken deps
[kawaigin] Added back the setUp as additional cleanup routine before each test.
[noreply] [BEAM-11742] Use pyarrow schema instead column names when creating
------------------------------------------
[...truncated 38.52 MB...]
"NAME": "event.bytes",
"PTRANSFORM":
"SessionSideInputJoin/SessionSideInputJoin.Monitor/ParMultiDo(Monitor)"
}
}, {
"urn": "beam:metric:user:sum_int64:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "oI0G",
"labels": {
"NAMESPACE": "SessionSideInputJoin.Events",
"NAME": "event.elements",
"PTRANSFORM":
"SessionSideInputJoin/SessionSideInputJoin.Monitor/ParMultiDo(Monitor)"
}
}, {
"urn": "beam:metric:user:distribution_int64:v1",
"type": "beam:metrics:distribution_int64:v1",
"payload": "oI0Gs7XX05K5sZ8CzvPc9oovnvnc9oov",
"labels": {
"NAMESPACE": "SessionSideInputJoin.Events",
"NAME": "event.processingTime",
"PTRANSFORM":
"SessionSideInputJoin/SessionSideInputJoin.Monitor/ParMultiDo(Monitor)"
}
}, {
"urn": "beam:metric:user:distribution_int64:v1",
"type": "beam:metrics:distribution_int64:v1",
"payload": "oI0G4IfhsPven/8BgIiL+OgphtaL+Ogp",
"labels": {
"NAMESPACE": "SessionSideInputJoin.Events",
"NAME": "event.eventTimestamp",
"PTRANSFORM":
"SessionSideInputJoin/SessionSideInputJoin.Monitor/ParMultiDo(Monitor)"
}
}, {
"urn": "beam:metric:user:sum_int64:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "ueLgAw==",
"labels": {
"NAMESPACE": "SessionSideInputJoin.Results",
"NAME": "result.bytes",
"PTRANSFORM":
"SessionSideInputJoin/SessionSideInputJoin.Debug/ParMultiDo(Monitor)"
}
}, {
"urn": "beam:metric:user:sum_int64:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "4M4F",
2021-04-08T00:17:25.648Z Waiting for main pipeline to 'finish'
"labels": {
"NAMESPACE": "SessionSideInputJoin.Results",
"NAME": "result.elements",
"PTRANSFORM":
"SessionSideInputJoin/SessionSideInputJoin.Debug/ParMultiDo(Monitor)"
}
}, {
"urn": "beam:metric:user:distribution_int64:v1",
"type": "beam:metrics:distribution_int64:v1",
"payload": "4M4FgN6Dj/mB6uoBo9iv+OgpxaWw+Ogp",
"labels": {
"NAMESPACE": "SessionSideInputJoin.Results",
"NAME": "result.eventTimestamp",
"PTRANSFORM":
"SessionSideInputJoin/SessionSideInputJoin.Debug/ParMultiDo(Monitor)"
}
}, {
"urn": "beam:metric:user:distribution_int64:v1",
"type": "beam:metrics:distribution_int64:v1",
"payload": "4M4FtNLW0pbOsogCufrc9oov/oDd9oov",
"labels": {
"NAMESPACE": "SessionSideInputJoin.Results",
"NAME": "result.processingTime",
"PTRANSFORM":
"SessionSideInputJoin/SessionSideInputJoin.Debug/ParMultiDo(Monitor)"
}
}, {
"urn": "beam:metric:user:sum_int64:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "4M4F",
"labels": {
"NAMESPACE": "SessionSideInputJoin",
"NAME": "discarded",
"PTRANSFORM": "SessionSideInputJoin.DevNull/ParMultiDo(Anonymous)"
}
}, {
"urn": "beam:metric:user:sum_int64:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "0A8=",
"labels": {
"NAMESPACE": "SessionSideInputJoin",
"NAME": "newPersons",
"PTRANSFORM":
"SessionSideInputJoin/SessionSideInputJoin.Snoop/ParMultiDo(Anonymous)"
}
2021-04-08T00:17:25.648Z }, {
"urn": "beam:metric:user:sum_int64:v1",
DONE SessionSideInputJoin
"type": "beam:metrics:sum_int64:v1",
"payload": "oI0G",
"labels": {
"NAMESPACE": "SessionSideInputJoin",
"NAME": "events",
"PTRANSFORM":
"SessionSideInputJoin/SessionSideInputJoin.Snoop/ParMultiDo(Anonymous)"
}
}, {
"urn": "beam:metric:user:sum_int64:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "8C4=",
"labels": {
"NAMESPACE": "SessionSideInputJoin",
"NAME": "newAuctions",
"PTRANSFORM":
"SessionSideInputJoin/SessionSideInputJoin.Snoop/ParMultiDo(Anonymous)"
}
}, {
"urn": "beam:metric:user:sum_int64:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "4M4F",
"labels": {
"NAMESPACE": "SessionSideInputJoin",
"NAME": "bids",
"PTRANSFORM":
"SessionSideInputJoin/SessionSideInputJoin.Snoop/ParMultiDo(Anonymous)"
}
}, {
"urn": "beam:metric:user:sum_int64:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "9AM=",
"labels": {
"NAMESPACE": "source",
"NAME": "elements_read",
"PTRANSFORM":
"NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)/ProcessKeyedElements/SplittableParDoViaKeyedWorkItems.ProcessElements"
}
}, {
"urn": "beam:metric:user:sum_int64:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "4M4F",
"labels": {
"NAMESPACE": "SessionSideInputJoin",
"NAME": "records",
"PTRANSFORM": "SessionSideInputJoin.Format/ParMultiDo(Anonymous)"
}
}]
}
}
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:11, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: ad5d689a6190e8038f5a62118f3125af, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:10, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 115cc52b0933bfdd880f85be05cc7547, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.TaskExecutor$JobLeaderListenerImpl
jobManagerLostLeadership
INFO: JobManager for job 9c2b4dd2df7bdf9e1177a05f8dede877 with leader id
875a109e7cb6b554c461972f59cb423d lost leadership.
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:5, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 01fa0fddb3a310adb5b9b2df2a98dfb3, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
2021-04-08T00:17:25.654Z 1 samples not enough to calculate steady-state event
rate
Apr 08, 2021 12:17:25 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace SessionSideInputJoin
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:9, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: f8dea08786ec39d5be7b48ebf1f92b0d, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:14, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 44a8ff739896f3670402be082efe3b19, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:0, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 373fafa23512b525ac58c104f74c9416, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.webmonitor.WebMonitorEndpoint
lambda$shutDownInternal$5
INFO: Removing cache directory /tmp/flink-web-ui
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:3, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 7c5201b256f290ee6518f67b045ae594, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.rest.RestServerEndpoint
lambda$closeAsync$1
INFO: Shut down complete.
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:2, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 598e3d2dbdf5b9d9afd57892d6c5d7be, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.resourcemanager.ResourceManager deregisterApplication
INFO: Shut down cluster because application is in CANCELED, diagnostics
DispatcherResourceManagerComponent has been closed..
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:1, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: e6ba8d083de5dcd0ace69b583bc89b05, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent
closeAsyncInternal
INFO: Closing components.
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:8, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: bbe70e6345b977708cf516d471cfb3d6, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.dispatcher.runner.AbstractDispatcherLeaderProcess
closeInternal
INFO: Stopping SessionDispatcherLeaderProcess.
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:7, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 7445e02ca143b842e666daaaf2aa6938, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl close
INFO: Closing the SlotManager.
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.dispatcher.Dispatcher onStop
INFO: Stopping dispatcher akka://flink/user/rpc/dispatcher_58.
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl suspend
INFO: Suspending the SlotManager.
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.dispatcher.Dispatcher
terminateRunningJobs
INFO: Stopping all currently running jobs of dispatcher
akka://flink/user/rpc/dispatcher_58.
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:4, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 26d0d0dbf67abbe706005e5a28506bcb, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:6, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: c1ad2d5c412a437aeb5d3841b3b9d3f8, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:15, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 52c6ca613be8a5e67f1f18c11cfd0436, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.rest.handler.legacy.backpressure.BackPressureRequestCoordinator
shutDown
INFO: Shutting down back pressure request coordinator.
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.dispatcher.Dispatcher
lambda$onStop$0
INFO: Stopped dispatcher akka://flink/user/rpc/dispatcher_58.
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:13, state:ALLOCATED, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 2ccd2e50c402d05cba4a5c70f3548fb0, jobId:
9c2b4dd2df7bdf9e1177a05f8dede877).
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService stop
INFO: Stop job leader service.
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager shutdown
INFO: Shutting down TaskExecutorLocalStateStoresManager.
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.io.disk.FileChannelManagerImpl lambda$getFileCloser$0
INFO: FileChannelManager removed spill file directory
/tmp/flink-io-a1bb55d3-676f-442d-9c91-86ccbc256f65
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.io.network.NettyShuffleEnvironment close
INFO: Shutting down the network environment and its components.
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.io.disk.FileChannelManagerImpl lambda$getFileCloser$0
INFO: FileChannelManager removed spill file directory
/tmp/flink-netty-shuffle-558b58e9-cdb1-4255-9da5-eaa7d72defd7
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.taskexecutor.KvStateService
shutdown
INFO: Shutting down the kvState service and its components.
Apr 08, 2021 12:17:25 AM
org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService stop
INFO: Stop job leader service.
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.filecache.FileCache shutdown
INFO: removed file cache directory
/tmp/flink-dist-cache-a306b38b-c3b9-4910-a5ef-1179edac74b6
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.taskexecutor.TaskExecutor
handleOnStopException
INFO: Stopped TaskExecutor akka://flink/user/rpc/taskmanager_56.
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.rpc.akka.AkkaRpcService
stopService
INFO: Stopping Akka RPC service.
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.rpc.akka.AkkaRpcService
stopService
INFO: Stopping Akka RPC service.
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.rpc.akka.AkkaRpcService
lambda$stopService$7
INFO: Stopped Akka RPC service.
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.blob.BlobServer close
INFO: Stopped BLOB server at 0.0.0.0:43493
Apr 08, 2021 12:17:25 AM org.apache.flink.runtime.rpc.akka.AkkaRpcService
lambda$stopService$7
INFO: Stopped Akka RPC service.
> Task :sdks:java:testing:nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 29s
112 actionable tasks: 2 executed, 110 up-to-date
Publishing build scan...
https://gradle.com/s/utshzhnln6hfm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
