MartijnVisser commented on PR #21128:
URL: https://github.com/apache/flink/pull/21128#issuecomment-1323541403
@gaborgsomogyi I did some debugging locally. If you correct the location to
the directory, the test still fails. For example, the `perJobYarnCluster` fails
because the job is being cancelled instead of finishing successfully. For
example, see:
```
2022-11-22 12:30:04,606 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Starting execution of job 'WordCount Example'
(56e7231d71adead5c4c3409b1e8f295c) under job master id
00000000000000000000000000000000.
2022-11-22 12:30:04,612 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Starting scheduling with scheduling strategy
[org.apache.flink.runtime.scheduler.strategy.PipelinedRegionSchedulingStrategy]
2022-11-22 12:30:04,612 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job WordCount
Example (56e7231d71adead5c4c3409b1e8f295c) switched from state CREATED to
RUNNING.
2022-11-22 12:30:04,616 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - CHAIN
DataSource (at main(WordCount.java:69)
(org.apache.flink.api.java.io.TextInputFormat)) -> FlatMap (FlatMap at
main(WordCount.java:84)) -> Combine (SUM(1), at main(WordCount.java:87) (1/2)
(d70d4be96afa4bcefa3275ef9b89a439_1b58034f691643610d089711d8321d03_0_0)
switched from CREATED to SCHEDULED.
2022-11-22 12:30:04,616 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - CHAIN
DataSource (at main(WordCount.java:69)
(org.apache.flink.api.java.io.TextInputFormat)) -> FlatMap (FlatMap at
main(WordCount.java:84)) -> Combine (SUM(1), at main(WordCount.java:87) (2/2)
(d70d4be96afa4bcefa3275ef9b89a439_1b58034f691643610d089711d8321d03_1_0)
switched from CREATED to SCHEDULED.
2022-11-22 12:30:04,616 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Reduce
(SUM(1), at main(WordCount.java:87) (1/2)
(d70d4be96afa4bcefa3275ef9b89a439_64a7a9e5ab13db3bb7c7ac4cec59879e_0_0)
switched from CREATED to SCHEDULED.
2022-11-22 12:30:04,616 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Reduce
(SUM(1), at main(WordCount.java:87) (2/2)
(d70d4be96afa4bcefa3275ef9b89a439_64a7a9e5ab13db3bb7c7ac4cec59879e_1_0)
switched from CREATED to SCHEDULED.
2022-11-22 12:30:04,616 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - DataSink
(CsvOutputFormat (path:
/var/folders/5d/k7d0ltfn3hgbwlvyc5l9_mvc0000gn/T/junit14650925366352419605,
delimiter: )) (1/2)
(d70d4be96afa4bcefa3275ef9b89a439_3040bb2235154f828371bc8c04c5a973_0_0)
switched from CREATED to SCHEDULED.
2022-11-22 12:30:04,616 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - DataSink
(CsvOutputFormat (path:
/var/folders/5d/k7d0ltfn3hgbwlvyc5l9_mvc0000gn/T/junit14650925366352419605,
delimiter: )) (2/2)
(d70d4be96afa4bcefa3275ef9b89a439_3040bb2235154f828371bc8c04c5a973_1_0)
switched from CREATED to SCHEDULED.
2022-11-22 12:30:04,631 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Connecting to ResourceManager
akka.tcp://flink@martijnsmbpm15:58267/user/rpc/resourcemanager_*(00000000000000000000000000000000)
2022-11-22 12:30:04,871 INFO
org.apache.flink.yarn.YarnResourceManagerDriver [] - Recovered 0
containers from previous attempts ([]).
2022-11-22 12:30:04,872 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Recovered 0 workers from previous attempt.
2022-11-22 12:30:04,921 INFO org.apache.hadoop.conf.Configuration
[] - resource-types.xml not found
2022-11-22 12:30:04,921 INFO
org.apache.hadoop.yarn.util.resource.ResourceUtils [] - Unable to
find 'resource-types.xml'.
2022-11-22 12:30:04,928 INFO
org.apache.hadoop.yarn.util.resource.ResourceUtils [] - Adding
resource type - name = memory-mb, units = Mi, type = COUNTABLE
2022-11-22 12:30:04,928 INFO
org.apache.hadoop.yarn.util.resource.ResourceUtils [] - Adding
resource type - name = vcores, units = , type = COUNTABLE
2022-11-22 12:30:04,933 INFO
org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled
external resources: []
2022-11-22 12:30:04,938 INFO
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl [] - Upper bound
of the thread pool size is 500
2022-11-22 12:30:04,946 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Resolved ResourceManager address, beginning registration
2022-11-22 12:30:04,964 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Registering job manager
[email protected]://flink@martijnsmbpm15:58267/user/rpc/jobmanager_1
for job 56e7231d71adead5c4c3409b1e8f295c.
2022-11-22 12:30:04,983 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Registered job manager
[email protected]://flink@martijnsmbpm15:58267/user/rpc/jobmanager_1
for job 56e7231d71adead5c4c3409b1e8f295c.
2022-11-22 12:30:04,991 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - JobManager successfully registered at ResourceManager,
leader id: 00000000000000000000000000000000.
2022-11-22 12:30:04,994 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager []
- Received resource requirements from job 56e7231d71adead5c4c3409b1e8f295c:
[ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN},
numberOfRequiredSlots=2}]
2022-11-22 12:30:05,062 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Requesting new worker with resource spec WorkerResourceSpec {cpuCores=2.0,
taskHeapSize=25.600mb (26843542 bytes), taskOffHeapSize=0 bytes,
networkMemSize=64.000mb (67108864 bytes), managedMemSize=230.400mb (241591914
bytes), numSlots=2}, current pending count: 1.
2022-11-22 12:30:05,074 INFO
org.apache.flink.yarn.YarnResourceManagerDriver [] - Requesting
new TaskExecutor container with resource TaskExecutorProcessSpec {cpuCores=2.0,
frameworkHeapSize=128.000mb (134217728 bytes), frameworkOffHeapSize=128.000mb
(134217728 bytes), taskHeapSize=25.600mb (26843542 bytes), taskOffHeapSize=0
bytes, networkMemSize=64.000mb (67108864 bytes), managedMemorySize=230.400mb
(241591914 bytes), jvmMetaspaceSize=256.000mb (268435456 bytes),
jvmOverheadSize=192.000mb (201326592 bytes), numSlots=2}, priority 1.
2022-11-22 12:30:10,496 INFO
org.apache.flink.yarn.YarnResourceManagerDriver [] - Received 1
containers.
2022-11-22 12:30:10,497 INFO
org.apache.flink.yarn.YarnResourceManagerDriver [] - Received 1
containers with priority 1, 1 pending container requests.
2022-11-22 12:30:10,502 INFO
org.apache.flink.yarn.YarnResourceManagerDriver [] - Removing
container request Capability[<memory:1024,
vCores:2>]Priority[1]AllocationRequestId[0]ExecutionTypeRequest[{Execution
Type: GUARANTEED, Enforce Execution Type: false}].
2022-11-22 12:30:10,503 INFO
org.apache.flink.yarn.YarnResourceManagerDriver [] - Accepted 1
requested containers, returned 0 excess containers, 0 pending container
requests of resource <memory:1024, vCores:2>.
2022-11-22 12:30:10,503 INFO
org.apache.flink.yarn.YarnResourceManagerDriver [] - TaskExecutor
container_1669116565071_0002_01_000002(martijnsmbpm15:58229) will be started on
martijnsmbpm15 with TaskExecutorProcessSpec {cpuCores=2.0,
frameworkHeapSize=128.000mb (134217728 bytes), frameworkOffHeapSize=128.000mb
(134217728 bytes), taskHeapSize=25.600mb (26843542 bytes), taskOffHeapSize=0
bytes, networkMemSize=64.000mb (67108864 bytes), managedMemorySize=230.400mb
(241591914 bytes), jvmMetaspaceSize=256.000mb (268435456 bytes),
jvmOverheadSize=192.000mb (201326592 bytes), numSlots=2}.
2022-11-22 12:30:10,507 INFO
org.apache.flink.yarn.YarnResourceManagerDriver [] - TM:Adding
remoteYarnConfPath
file:/var/folders/5d/k7d0ltfn3hgbwlvyc5l9_mvc0000gn/T/junit15361768760757807396/.flink/application_1669116565071_0002/yarn-site.xml
to the container local resource bucket
2022-11-22 12:30:10,747 INFO
org.apache.flink.yarn.YarnResourceManagerDriver [] - Creating
container launch context for TaskManagers
2022-11-22 12:30:10,748 INFO
org.apache.flink.yarn.YarnResourceManagerDriver [] - Starting
TaskManagers
2022-11-22 12:30:10,757 WARN org.apache.flink.runtime.util.HadoopUtils
[] - Could not find Hadoop configuration via any of the supported
methods (Flink configuration, environment variables).
2022-11-22 12:30:10,768 INFO
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl [] - Processing
Event EventType: START_CONTAINER for Container
container_1669116565071_0002_01_000002
2022-11-22 12:30:10,768 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Requested worker container_1669116565071_0002_01_000002(martijnsmbpm15:58229)
with resource spec WorkerResourceSpec {cpuCores=2.0, taskHeapSize=25.600mb
(26843542 bytes), taskOffHeapSize=0 bytes, networkMemSize=64.000mb (67108864
bytes), managedMemSize=230.400mb (241591914 bytes), numSlots=2}.
2022-11-22 12:30:14,006 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - RECEIVED
SIGNAL 2: SIGINT. Shutting down as requested.
2022-11-22 12:30:14,008 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Shutting
YarnJobClusterEntrypoint down with application status UNKNOWN. Diagnostics
Cluster entrypoint has been closed externally..
2022-11-22 12:30:14,015 INFO org.apache.flink.runtime.blob.BlobServer
[] - Stopped BLOB server at 0.0.0.0:58268
2022-11-22 12:30:14,023 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Shutting
down rest endpoint.
2022-11-22 12:30:14,084 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Removing
cache directory
/var/folders/5d/k7d0ltfn3hgbwlvyc5l9_mvc0000gn/T/flink-web-9ae99cf9-dc8f-4eb1-85cc-dc57cd6b50f0/flink-web-ui
2022-11-22 12:30:14,087 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] -
http://192.168.2.217:58270 lost leadership
2022-11-22 12:30:14,087 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Shut down
complete.
2022-11-22 12:30:14,087 INFO
org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent
[] - Closing components.
2022-11-22 12:30:14,087 INFO
org.apache.flink.runtime.dispatcher.runner.DefaultDispatcherRunner [] -
DefaultDispatcherRunner was revoked the leadership with leader id
00000000-0000-0000-0000-000000000000. Stopping the DispatcherLeaderProcess.
2022-11-22 12:30:14,088 INFO
org.apache.flink.runtime.dispatcher.runner.JobDispatcherLeaderProcess [] -
Stopping JobDispatcherLeaderProcess.
2022-11-22 12:30:14,089 INFO
org.apache.flink.runtime.resourcemanager.ResourceManagerServiceImpl [] -
Stopping resource manager service.
2022-11-22 12:30:14,090 INFO
org.apache.flink.runtime.resourcemanager.ResourceManagerServiceImpl [] -
Resource manager service is not running. Ignore revoking leadership.
2022-11-22 12:30:14,090 INFO
org.apache.flink.runtime.dispatcher.MiniDispatcher [] - Stopping
dispatcher akka.tcp://flink@martijnsmbpm15:58267/user/rpc/dispatcher_0.
2022-11-22 12:30:14,092 INFO
org.apache.flink.runtime.dispatcher.MiniDispatcher [] - Stopping all
currently running jobs of dispatcher
akka.tcp://flink@martijnsmbpm15:58267/user/rpc/dispatcher_0.
2022-11-22 12:30:14,097 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Stopping the JobMaster for job 'WordCount Example'
(56e7231d71adead5c4c3409b1e8f295c).
2022-11-22 12:30:14,100 INFO
org.apache.flink.runtime.dispatcher.MiniDispatcher [] - Job
56e7231d71adead5c4c3409b1e8f295c reached terminal state SUSPENDED.
2022-11-22 12:30:14,104 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job WordCount
Example (56e7231d71adead5c4c3409b1e8f295c) switched from state RUNNING to
SUSPENDED.
org.apache.flink.util.FlinkException: Scheduler is being stopped.
at
org.apache.flink.runtime.scheduler.SchedulerBase.closeAsync(SchedulerBase.java:637)
~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.jobmaster.JobMaster.stopScheduling(JobMaster.java:1056)
~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.jobmaster.JobMaster.stopJobExecution(JobMaster.java:1019)
~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:442)
~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:239)
~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.lambda$terminate$0(AkkaRpcActor.java:578)
~[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83)
~[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:577)
~[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:196)
~[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.actor.Actor.aroundReceive(Actor.scala:537)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.actor.Actor.aroundReceive$(Actor.scala:535)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.actor.ActorCell.invoke(ActorCell.scala:547)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.dispatch.Mailbox.run(Mailbox.scala:231)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) [?:?]
at
java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
[?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) [?:?]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
[?:?]
at
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
[?:?]
2022-11-22 12:30:14,120 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - CHAIN
DataSource (at main(WordCount.java:69)
(org.apache.flink.api.java.io.TextInputFormat)) -> FlatMap (FlatMap at
main(WordCount.java:84)) -> Combine (SUM(1), at main(WordCount.java:87) (1/2)
(d70d4be96afa4bcefa3275ef9b89a439_1b58034f691643610d089711d8321d03_0_0)
switched from SCHEDULED to CANCELING.
2022-11-22 12:30:14,121 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - CHAIN
DataSource (at main(WordCount.java:69)
(org.apache.flink.api.java.io.TextInputFormat)) -> FlatMap (FlatMap at
main(WordCount.java:84)) -> Combine (SUM(1), at main(WordCount.java:87) (1/2)
(d70d4be96afa4bcefa3275ef9b89a439_1b58034f691643610d089711d8321d03_0_0)
switched from CANCELING to CANCELED.
2022-11-22 12:30:14,123 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
d70d4be96afa4bcefa3275ef9b89a439_1b58034f691643610d089711d8321d03_0_0.
2022-11-22 12:30:14,125 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - CHAIN
DataSource (at main(WordCount.java:69)
(org.apache.flink.api.java.io.TextInputFormat)) -> FlatMap (FlatMap at
main(WordCount.java:84)) -> Combine (SUM(1), at main(WordCount.java:87) (2/2)
(d70d4be96afa4bcefa3275ef9b89a439_1b58034f691643610d089711d8321d03_1_0)
switched from SCHEDULED to CANCELING.
2022-11-22 12:30:14,125 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - CHAIN
DataSource (at main(WordCount.java:69)
(org.apache.flink.api.java.io.TextInputFormat)) -> FlatMap (FlatMap at
main(WordCount.java:84)) -> Combine (SUM(1), at main(WordCount.java:87) (2/2)
(d70d4be96afa4bcefa3275ef9b89a439_1b58034f691643610d089711d8321d03_1_0)
switched from CANCELING to CANCELED.
2022-11-22 12:30:14,125 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
d70d4be96afa4bcefa3275ef9b89a439_1b58034f691643610d089711d8321d03_1_0.
2022-11-22 12:30:14,125 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Reduce
(SUM(1), at main(WordCount.java:87) (1/2)
(d70d4be96afa4bcefa3275ef9b89a439_64a7a9e5ab13db3bb7c7ac4cec59879e_0_0)
switched from SCHEDULED to CANCELING.
2022-11-22 12:30:14,125 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Reduce
(SUM(1), at main(WordCount.java:87) (1/2)
(d70d4be96afa4bcefa3275ef9b89a439_64a7a9e5ab13db3bb7c7ac4cec59879e_0_0)
switched from CANCELING to CANCELED.
2022-11-22 12:30:14,125 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
d70d4be96afa4bcefa3275ef9b89a439_64a7a9e5ab13db3bb7c7ac4cec59879e_0_0.
2022-11-22 12:30:14,125 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Reduce
(SUM(1), at main(WordCount.java:87) (2/2)
(d70d4be96afa4bcefa3275ef9b89a439_64a7a9e5ab13db3bb7c7ac4cec59879e_1_0)
switched from SCHEDULED to CANCELING.
2022-11-22 12:30:14,125 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Reduce
(SUM(1), at main(WordCount.java:87) (2/2)
(d70d4be96afa4bcefa3275ef9b89a439_64a7a9e5ab13db3bb7c7ac4cec59879e_1_0)
switched from CANCELING to CANCELED.
2022-11-22 12:30:14,125 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
d70d4be96afa4bcefa3275ef9b89a439_64a7a9e5ab13db3bb7c7ac4cec59879e_1_0.
2022-11-22 12:30:14,125 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - DataSink
(CsvOutputFormat (path:
/var/folders/5d/k7d0ltfn3hgbwlvyc5l9_mvc0000gn/T/junit14650925366352419605,
delimiter: )) (1/2)
(d70d4be96afa4bcefa3275ef9b89a439_3040bb2235154f828371bc8c04c5a973_0_0)
switched from SCHEDULED to CANCELING.
2022-11-22 12:30:14,125 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - DataSink
(CsvOutputFormat (path:
/var/folders/5d/k7d0ltfn3hgbwlvyc5l9_mvc0000gn/T/junit14650925366352419605,
delimiter: )) (1/2)
(d70d4be96afa4bcefa3275ef9b89a439_3040bb2235154f828371bc8c04c5a973_0_0)
switched from CANCELING to CANCELED.
2022-11-22 12:30:14,125 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
d70d4be96afa4bcefa3275ef9b89a439_3040bb2235154f828371bc8c04c5a973_0_0.
2022-11-22 12:30:14,126 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - DataSink
(CsvOutputFormat (path:
/var/folders/5d/k7d0ltfn3hgbwlvyc5l9_mvc0000gn/T/junit14650925366352419605,
delimiter: )) (2/2)
(d70d4be96afa4bcefa3275ef9b89a439_3040bb2235154f828371bc8c04c5a973_1_0)
switched from SCHEDULED to CANCELING.
2022-11-22 12:30:14,127 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - DataSink
(CsvOutputFormat (path:
/var/folders/5d/k7d0ltfn3hgbwlvyc5l9_mvc0000gn/T/junit14650925366352419605,
delimiter: )) (2/2)
(d70d4be96afa4bcefa3275ef9b89a439_3040bb2235154f828371bc8c04c5a973_1_0)
switched from CANCELING to CANCELED.
2022-11-22 12:30:14,129 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
d70d4be96afa4bcefa3275ef9b89a439_3040bb2235154f828371bc8c04c5a973_1_0.
2022-11-22 12:30:14,130 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job
56e7231d71adead5c4c3409b1e8f295c has been suspended.
2022-11-22 12:30:14,131 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Close ResourceManager connection
bf303538bb378c3126b9d7396fb542f8: Stopping JobMaster for job 'WordCount
Example' (56e7231d71adead5c4c3409b1e8f295c).
2022-11-22 12:30:14,146 INFO
org.apache.flink.runtime.dispatcher.MiniDispatcher [] - Stopped
dispatcher akka.tcp://flink@martijnsmbpm15:58267/user/rpc/dispatcher_0.
2022-11-22 12:30:14,449 WARN org.apache.hadoop.ipc.Client
[] - Exception encountered while connecting to the server
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[?:?]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[?:?]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276) ~[?:?]
at sun.nio.ch.IOUtil.read(IOUtil.java:245) ~[?:?]
at sun.nio.ch.IOUtil.read(IOUtil.java:223) ~[?:?]
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:356) ~[?:?]
at
org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
~[hadoop-common-2.10.2.jar:?]
at java.io.FilterInputStream.read(FilterInputStream.java:133) ~[?:?]
at java.io.BufferedInputStream.fill(BufferedInputStream.java:252) ~[?:?]
at java.io.BufferedInputStream.read(BufferedInputStream.java:271) ~[?:?]
at java.io.DataInputStream.readInt(DataInputStream.java:392) ~[?:?]
at
org.apache.hadoop.ipc.Client$IpcStreams.readResponse(Client.java:1865)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:365)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:629)
~[hadoop-common-2.10.2.jar:?]
at org.apache.hadoop.ipc.Client$Connection.access$2200(Client.java:423)
~[hadoop-common-2.10.2.jar:?]
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:833)
~[hadoop-common-2.10.2.jar:?]
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:829)
~[hadoop-common-2.10.2.jar:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
at javax.security.auth.Subject.doAs(Subject.java:423) ~[?:?]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1938)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:828)
~[hadoop-common-2.10.2.jar:?]
at org.apache.hadoop.ipc.Client$Connection.access$3700(Client.java:423)
~[hadoop-common-2.10.2.jar:?]
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1621)
~[hadoop-common-2.10.2.jar:?]
at org.apache.hadoop.ipc.Client.call(Client.java:1450)
~[hadoop-common-2.10.2.jar:?]
at org.apache.hadoop.ipc.Client.call(Client.java:1403)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
~[hadoop-common-2.10.2.jar:?]
at com.sun.proxy.$Proxy39.stopContainers(Unknown Source) ~[?:?]
at
org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.stopContainers(ContainerManagementProtocolPBClientImpl.java:142)
~[hadoop-yarn-common-2.10.2.jar:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:?]
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:?]
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:433)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:166)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:158)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:96)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:362)
~[hadoop-common-2.10.2.jar:?]
at com.sun.proxy.$Proxy40.stopContainers(Unknown Source) ~[?:?]
at
org.apache.hadoop.yarn.client.api.impl.NMClientImpl.stopContainerInternal(NMClientImpl.java:426)
~[hadoop-yarn-client-2.10.2.jar:?]
at
org.apache.hadoop.yarn.client.api.impl.NMClientImpl.stopContainer(NMClientImpl.java:303)
~[hadoop-yarn-client-2.10.2.jar:?]
at
org.apache.hadoop.yarn.client.api.impl.NMClientImpl.cleanupRunningContainers(NMClientImpl.java:121)
~[hadoop-yarn-client-2.10.2.jar:?]
at
org.apache.hadoop.yarn.client.api.impl.NMClientImpl.serviceStop(NMClientImpl.java:112)
~[hadoop-yarn-client-2.10.2.jar:?]
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:222)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl.serviceStop(NMClientAsyncImpl.java:240)
~[hadoop-yarn-client-2.10.2.jar:?]
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:222)
~[hadoop-common-2.10.2.jar:?]
at
org.apache.flink.yarn.YarnResourceManagerDriver.terminate(YarnResourceManagerDriver.java:223)
~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager.terminate(ActiveResourceManager.java:184)
~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.resourcemanager.ResourceManager.stopResourceManagerServices(ResourceManager.java:315)
~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.resourcemanager.ResourceManager.onStop(ResourceManager.java:301)
~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:239)
~[flink-dist-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.lambda$terminate$0(AkkaRpcActor.java:578)
~[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83)
~[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:577)
~[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:196)
~[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.actor.Actor.aroundReceive(Actor.scala:537)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.actor.Actor.aroundReceive$(Actor.scala:535)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.actor.ActorCell.invoke(ActorCell.scala:547)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.dispatch.Mailbox.run(Mailbox.scala:231)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
[flink-rpc-akka_3f8b2750-5919-429e-97ab-a0077948520e.jar:1.17-SNAPSHOT]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) [?:?]
at
java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
[?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) [?:?]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
[?:?]
at
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
[?:?]
2022-11-22 12:30:14,463 ERROR
org.apache.hadoop.yarn.client.api.impl.NMClientImpl [] - Failed to
stop Container container_1669116565071_0002_01_000002when stopping NMClientImpl
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
