[
https://issues.apache.org/jira/browse/FLINK-21329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312962#comment-17312962
]
Matthias commented on FLINK-21329:
----------------------------------
For [build
#13326|https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=13326&view=logs&j=6caf31d6-847a-526e-9624-468e053467d6&t=0b23652f-b18b-5b6e-6eb6-a11070364610]
I had a look at the timestamps. First of all, in all cases the test doesn't
fail for the {{HashMapStateBackend}}. Every failure is caused by the {{4 10
rocks false false}} setup (i.e. {{parallism=4}}, {{maxAttempts=10}},
{{EmbeddedRocksDBStateBackend}} used, {{incrementalCheckpoints=false}},
{{killJvm=false}}).
It looks like the checkpoint takes up quite some time for checkpoints later in
the test:
{code}
$ grep "Received ack\|attempt #\|Triggering checkpoin"
flink-vsts-standalonesession-0-fv-az127-111.log
2021-02-15 00:52:20,703 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (1/4) (attempt #0) with attempt id
4f4ad74ee089d4dd970ec49c30c04bd9 to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:52:20,720 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (2/4) (attempt #0) with attempt id
82981af733cb7b98a6a5887afe6ddd9f to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:52:20,725 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (3/4) (attempt #0) with attempt id
449a2c47d0757dcea7dd388e48f85e45 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:52:20,726 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (4/4) (attempt #0) with attempt id
7973e2cbeb1c92794d732674a4b61416 to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:52:20,743 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (1/4) (attempt #0) with attempt id
880f6ba3112ff89dc036928542878ce2 to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:52:20,763 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (2/4) (attempt #0) with attempt id
55bd0597a2f983d09dfc97ff7bd9e825 to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:52:20,767 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (3/4) (attempt #0) with attempt id
82f8ea6e74f36cfd316a288372811f68 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:52:20,769 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (4/4) (attempt #0) with attempt id
0c95f9805ca3eb643e9dcac59c43f0e7 to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:52:22,930 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 1 (type=CHECKPOINT) @ 1613350342913 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:52:23,900 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 1 from task 7973e2cbeb1c92794d732674a4b61416
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:52:23,908 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 1 from task 449a2c47d0757dcea7dd388e48f85e45
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:52:23,949 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 1 from task 4f4ad74ee089d4dd970ec49c30c04bd9
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:52:24,009 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 1 from task 82981af733cb7b98a6a5887afe6ddd9f
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:52:25,593 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 1 from task 880f6ba3112ff89dc036928542878ce2
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:52:25,706 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 1 from task 55bd0597a2f983d09dfc97ff7bd9e825
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:52:25,734 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 1 from task 0c95f9805ca3eb643e9dcac59c43f0e7
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:52:25,859 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 1 from task 82f8ea6e74f36cfd316a288372811f68
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:52:26,027 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 2 (type=CHECKPOINT) @ 1613350346023 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:52:26,076 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 2 from task 82981af733cb7b98a6a5887afe6ddd9f
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:52:26,109 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 2 from task 7973e2cbeb1c92794d732674a4b61416
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:52:26,326 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 2 from task 449a2c47d0757dcea7dd388e48f85e45
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:52:27,388 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (1/4) (attempt #1) with attempt id
788bb45fc691f093c8fa38aedf4c7c5e to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:52:27,389 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (2/4) (attempt #1) with attempt id
a484233ad0362559effe8505022cd7bb to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:52:27,389 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (3/4) (attempt #1) with attempt id
405efa0b99caf483b2ff3fbde9626142 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:52:27,412 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (4/4) (attempt #1) with attempt id
ceaba47bd65a109d5ae70453ee6d55d3 to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:52:27,426 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (1/4) (attempt #1) with attempt id
9eef2db0894ded24070d711200cef00b to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:52:27,430 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (2/4) (attempt #1) with attempt id
11a87877544346f024394efdf94f61a5 to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:52:27,434 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (3/4) (attempt #1) with attempt id
bfeb96193c8b4b1a7720523d934ae148 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:52:27,436 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (4/4) (attempt #1) with attempt id
b8789dbe8cb8f86c2a34cbcb1994937e to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:52:28,312 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 3 (type=CHECKPOINT) @ 1613350348309 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:52:29,130 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 3 from task 405efa0b99caf483b2ff3fbde9626142
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:52:29,230 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 3 from task 788bb45fc691f093c8fa38aedf4c7c5e
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:52:30,378 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 3 from task a484233ad0362559effe8505022cd7bb
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:52:30,969 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 3 from task ceaba47bd65a109d5ae70453ee6d55d3
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:52:31,783 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 3 from task 11a87877544346f024394efdf94f61a5
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:52:31,928 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 3 from task 9eef2db0894ded24070d711200cef00b
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:52:31,985 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 3 from task b8789dbe8cb8f86c2a34cbcb1994937e
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:52:32,078 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 3 from task bfeb96193c8b4b1a7720523d934ae148
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:52:32,229 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 4 (type=CHECKPOINT) @ 1613350352189 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:52:32,297 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 4 from task ceaba47bd65a109d5ae70453ee6d55d3
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:52:32,304 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 4 from task 405efa0b99caf483b2ff3fbde9626142
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:52:32,376 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 4 from task a484233ad0362559effe8505022cd7bb
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:52:35,478 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (1/4) (attempt #2) with attempt id
048a1a9d6f2a992a191a13daccb1948c to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:52:35,479 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (2/4) (attempt #2) with attempt id
b0c96b67fcb504a094ad8d0bcbe3e77e to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:52:35,494 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (3/4) (attempt #2) with attempt id
3b6dfaaa1ca850325860f3862ee79c16 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:52:35,495 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (4/4) (attempt #2) with attempt id
ecfb6e5e16de6b8bc8ebcb4123c01eb0 to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:52:35,502 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (1/4) (attempt #2) with attempt id
a98bdd545328575af264df54ec5c26a5 to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:52:35,503 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (2/4) (attempt #2) with attempt id
8b41c40b397ce1c3c4dedf229e317810 to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:52:35,504 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (3/4) (attempt #2) with attempt id
efe3eb687b17f610e031d1f30b4703e9 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:52:35,510 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (4/4) (attempt #2) with attempt id
59fd2b7377356729a937d5a7f1765256 to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:52:36,409 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 5 (type=CHECKPOINT) @ 1613350356406 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:52:41,052 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 5 from task 048a1a9d6f2a992a191a13daccb1948c
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:52:42,306 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 5 from task ecfb6e5e16de6b8bc8ebcb4123c01eb0
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:52:44,067 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 5 from task b0c96b67fcb504a094ad8d0bcbe3e77e
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:52:44,937 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 5 from task 3b6dfaaa1ca850325860f3862ee79c16
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:52:46,971 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 5 from task efe3eb687b17f610e031d1f30b4703e9
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:52:47,053 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 5 from task 59fd2b7377356729a937d5a7f1765256
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:52:47,081 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 5 from task 8b41c40b397ce1c3c4dedf229e317810
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:52:47,359 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 5 from task a98bdd545328575af264df54ec5c26a5
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:52:47,404 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 6 (type=CHECKPOINT) @ 1613350367400 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:52:47,463 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 6 from task 3b6dfaaa1ca850325860f3862ee79c16
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:52:47,551 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 6 from task 048a1a9d6f2a992a191a13daccb1948c
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:52:50,769 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (1/4) (attempt #3) with attempt id
42505d412d2fdbfd25b2d2f96185c8cc to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:52:50,769 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (2/4) (attempt #3) with attempt id
691e4d66555626abcd504ae697a97617 to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:52:50,769 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (3/4) (attempt #3) with attempt id
383562c09b5e1bfd12434ed7e2a17110 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:52:50,779 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (4/4) (attempt #3) with attempt id
702f75f4ee6a8fbd626272ee4ee69b4d to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:52:50,779 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (1/4) (attempt #3) with attempt id
016fdf521f8ab31ddbf825098358738b to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:52:50,780 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (2/4) (attempt #3) with attempt id
73add6ac94057d25dd721ceec2edd0fb to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:52:50,783 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (3/4) (attempt #3) with attempt id
02018111834d3bc0ba89bf4a180b6f4e to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:52:50,783 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (4/4) (attempt #3) with attempt id
144e393b9e369541eeb7bed89b79952a to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:52:51,070 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 7 (type=CHECKPOINT) @ 1613350371067 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:53:06,270 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 7 from task 691e4d66555626abcd504ae697a97617
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:53:06,331 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 7 from task 42505d412d2fdbfd25b2d2f96185c8cc
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:53:06,959 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 7 from task 702f75f4ee6a8fbd626272ee4ee69b4d
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:53:07,213 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 7 from task 383562c09b5e1bfd12434ed7e2a17110
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:53:08,936 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 7 from task 016fdf521f8ab31ddbf825098358738b
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:53:09,125 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 7 from task 73add6ac94057d25dd721ceec2edd0fb
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:53:09,243 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 7 from task 02018111834d3bc0ba89bf4a180b6f4e
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:53:09,602 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 7 from task 144e393b9e369541eeb7bed89b79952a
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:53:09,644 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 8 (type=CHECKPOINT) @ 1613350389632 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:53:14,233 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (1/4) (attempt #4) with attempt id
6f78f2945af7ac21eb1dd3fcf2689ebc to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:53:14,234 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (2/4) (attempt #4) with attempt id
f026bd2bbea6227dc80d89b5c6eda695 to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:53:14,235 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (3/4) (attempt #4) with attempt id
86a2c8ba4a03b89a2c591cc5d3cd9c6f to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:53:14,235 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (4/4) (attempt #4) with attempt id
58a49f79cc22df7897aed0bbcf83cbf8 to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:53:14,235 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (1/4) (attempt #4) with attempt id
78d3ce338fb1e2c27a67fe5eec73fd9a to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:53:14,235 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (2/4) (attempt #4) with attempt id
7900b3067e9211096fa2b9b2ad4db1e3 to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:53:14,235 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (3/4) (attempt #4) with attempt id
2beb71300bcfaed2ec16c88312b2bb24 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:53:14,235 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (4/4) (attempt #4) with attempt id
941eb8f47142721812ec335fdac77e66 to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:53:15,262 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 9 (type=CHECKPOINT) @ 1613350395260 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:53:32,027 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 9 from task f026bd2bbea6227dc80d89b5c6eda695
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:53:32,276 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 9 from task 6f78f2945af7ac21eb1dd3fcf2689ebc
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:53:37,030 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 9 from task 58a49f79cc22df7897aed0bbcf83cbf8
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:53:42,168 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 9 from task 86a2c8ba4a03b89a2c591cc5d3cd9c6f
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:53:47,214 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 9 from task 941eb8f47142721812ec335fdac77e66
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:53:47,530 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 9 from task 2beb71300bcfaed2ec16c88312b2bb24
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:53:47,641 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 9 from task 7900b3067e9211096fa2b9b2ad4db1e3
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:53:47,944 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 9 from task 78d3ce338fb1e2c27a67fe5eec73fd9a
of job dfa837311853a69cdca82db082270dff at 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:53:48,914 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 10 (type=CHECKPOINT) @ 1613350428898 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:53:48,964 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 10 from task
f026bd2bbea6227dc80d89b5c6eda695 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:53:49,135 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 10 from task
6f78f2945af7ac21eb1dd3fcf2689ebc of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:53:51,678 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (1/4) (attempt #5) with attempt id
910ca61f7469c8ec7949ec14e2c0e239 to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:53:51,678 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (2/4) (attempt #5) with attempt id
dfe8fff4ae0da75df98be7d9d6590e45 to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:53:51,678 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (3/4) (attempt #5) with attempt id
c50f803c83979afd49ea479f653ba8af to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:53:51,678 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (4/4) (attempt #5) with attempt id
8c50f0659c3e6541784fdd4c2474333e to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:53:51,678 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (1/4) (attempt #5) with attempt id
029e69c47b7824deb136eec7728441ef to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:53:51,678 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (2/4) (attempt #5) with attempt id
cb23e8fdefac39a50d3bd5ffb3bc3e86 to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:53:51,678 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (3/4) (attempt #5) with attempt id
95ac0f1a6bf2c2281db6972028d3f957 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:53:51,678 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (4/4) (attempt #5) with attempt id
1d1805b290593d54313a8a7e33d69ab7 to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:53:52,142 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 11 (type=CHECKPOINT) @ 1613350432135 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:54:33,436 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 11 from task
c50f803c83979afd49ea479f653ba8af of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:54:33,540 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 11 from task
dfe8fff4ae0da75df98be7d9d6590e45 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:54:34,314 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 11 from task
910ca61f7469c8ec7949ec14e2c0e239 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:54:35,783 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 11 from task
8c50f0659c3e6541784fdd4c2474333e of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:54:41,022 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 11 from task
1d1805b290593d54313a8a7e33d69ab7 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:54:41,137 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 11 from task
cb23e8fdefac39a50d3bd5ffb3bc3e86 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:54:41,179 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 11 from task
029e69c47b7824deb136eec7728441ef of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:54:42,511 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 11 from task
95ac0f1a6bf2c2281db6972028d3f957 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:54:43,941 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 12 (type=CHECKPOINT) @ 1613350483847 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:54:43,959 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 12 from task
dfe8fff4ae0da75df98be7d9d6590e45 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:54:44,017 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 12 from task
910ca61f7469c8ec7949ec14e2c0e239 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:54:44,149 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 12 from task
c50f803c83979afd49ea479f653ba8af of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:54:48,998 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (1/4) (attempt #6) with attempt id
8f89a8ab1e618df1b940f7b47dfaf55f to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:54:48,998 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (2/4) (attempt #6) with attempt id
e427bc378c8a4ccef08266f513ba04b3 to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:54:48,998 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (3/4) (attempt #6) with attempt id
3019aa338dcb1c5da07b68f3d2053e8d to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:54:48,998 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (4/4) (attempt #6) with attempt id
fbc19f708a93a3ddcbb4448037af25eb to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:54:48,998 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (1/4) (attempt #6) with attempt id
1e4fa709d4cae44602d2d0f269f663e6 to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:54:48,998 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (2/4) (attempt #6) with attempt id
5413a5bf4812076e4bcf91a660b2c8df to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:54:48,998 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (3/4) (attempt #6) with attempt id
7f3cf7692ebb98e99a485fe1c4ca0616 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:54:48,999 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (4/4) (attempt #6) with attempt id
c541f3d355a456d9d68456b3393e3109 to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:54:49,354 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 13 (type=CHECKPOINT) @ 1613350489351 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:55:34,111 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 13 from task
8f89a8ab1e618df1b940f7b47dfaf55f of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:55:37,156 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 13 from task
fbc19f708a93a3ddcbb4448037af25eb of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:55:37,666 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 13 from task
3019aa338dcb1c5da07b68f3d2053e8d of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:55:39,720 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 13 from task
e427bc378c8a4ccef08266f513ba04b3 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:55:45,909 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 13 from task
c541f3d355a456d9d68456b3393e3109 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:55:46,107 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 13 from task
7f3cf7692ebb98e99a485fe1c4ca0616 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:55:46,798 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 13 from task
1e4fa709d4cae44602d2d0f269f663e6 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:55:47,181 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 13 from task
5413a5bf4812076e4bcf91a660b2c8df of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:55:49,045 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 14 (type=CHECKPOINT) @ 1613350549043 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:55:49,245 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 14 from task
3019aa338dcb1c5da07b68f3d2053e8d of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:55:50,396 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 14 from task
8f89a8ab1e618df1b940f7b47dfaf55f of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:55:51,134 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 14 from task
fbc19f708a93a3ddcbb4448037af25eb of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:55:54,998 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (1/4) (attempt #7) with attempt id
ec5ea040bc73c307ab416a8a1fcb208c to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:55:54,998 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (2/4) (attempt #7) with attempt id
f3f009ae9a8b62967d9cfbe5b3b5f3ad to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:55:54,998 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (3/4) (attempt #7) with attempt id
c67144c6d3b24a2ca98959981dca7507 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:55:54,998 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (4/4) (attempt #7) with attempt id
9b4d2495d74966513b62f57682ec1b0c to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:55:54,998 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (1/4) (attempt #7) with attempt id
225fc04f7a84880fc83266d12e457d1d to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:55:54,999 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (2/4) (attempt #7) with attempt id
a90fc3b1d8f9cf189c769bb82aa9212e to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:55:54,999 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (3/4) (attempt #7) with attempt id
37c71ff6a787d67a389372c90a3ed7cd to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:55:54,999 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (4/4) (attempt #7) with attempt id
ed1ee0ebefc90ecc5d49f5bf93cd928b to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:55:55,270 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 15 (type=CHECKPOINT) @ 1613350555266 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:56:47,653 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 15 from task
ec5ea040bc73c307ab416a8a1fcb208c of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:56:50,392 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 15 from task
9b4d2495d74966513b62f57682ec1b0c of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:56:50,866 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 15 from task
c67144c6d3b24a2ca98959981dca7507 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:56:52,394 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 15 from task
f3f009ae9a8b62967d9cfbe5b3b5f3ad of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:57:01,563 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 15 from task
37c71ff6a787d67a389372c90a3ed7cd of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:57:01,813 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 15 from task
ed1ee0ebefc90ecc5d49f5bf93cd928b of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:57:02,145 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 15 from task
225fc04f7a84880fc83266d12e457d1d of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:57:03,166 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 15 from task
a90fc3b1d8f9cf189c769bb82aa9212e of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:57:03,840 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 16 (type=CHECKPOINT) @ 1613350623794 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:57:03,897 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 16 from task
ec5ea040bc73c307ab416a8a1fcb208c of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:57:03,916 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 16 from task
9b4d2495d74966513b62f57682ec1b0c of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:57:04,000 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 16 from task
c67144c6d3b24a2ca98959981dca7507 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:57:06,478 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 16 from task
f3f009ae9a8b62967d9cfbe5b3b5f3ad of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:57:07,038 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (1/4) (attempt #8) with attempt id
e9d9b82afa186c411e6bdf881873e25d to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:57:07,038 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (2/4) (attempt #8) with attempt id
b35b8153a9f2c0ba60b51ecf243ea73f to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:57:07,038 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (3/4) (attempt #8) with attempt id
39993b609c57b6226c58c199bd722159 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:57:07,038 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (4/4) (attempt #8) with attempt id
f979490d52ab8fe2ff77b2932a0bfbef to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:57:07,038 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (1/4) (attempt #8) with attempt id
7e12451b9faa438f6f06e995e474ed71 to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:57:07,038 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (2/4) (attempt #8) with attempt id
edc0da7ea739f999e8a3c2850d89e5ed to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:57:07,038 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (3/4) (attempt #8) with attempt id
8f96e7dc7d586e7ea101cb16e3facf29 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:57:07,045 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (4/4) (attempt #8) with attempt id
5d7bc3d6c1df759348a120a702e76aa9 to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:57:07,423 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 17 (type=CHECKPOINT) @ 1613350627420 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:58:07,870 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 17 from task
39993b609c57b6226c58c199bd722159 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:58:07,871 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 17 from task
b35b8153a9f2c0ba60b51ecf243ea73f of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:58:07,886 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 17 from task
e9d9b82afa186c411e6bdf881873e25d of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:58:08,957 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 17 from task
f979490d52ab8fe2ff77b2932a0bfbef of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:58:19,243 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 17 from task
5d7bc3d6c1df759348a120a702e76aa9 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:58:19,685 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 17 from task
7e12451b9faa438f6f06e995e474ed71 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:58:19,962 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 17 from task
8f96e7dc7d586e7ea101cb16e3facf29 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:58:20,276 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 17 from task
edc0da7ea739f999e8a3c2850d89e5ed of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:58:21,869 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 18 (type=CHECKPOINT) @ 1613350701857 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:58:21,897 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 18 from task
e9d9b82afa186c411e6bdf881873e25d of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:58:21,936 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 18 from task
f979490d52ab8fe2ff77b2932a0bfbef of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:58:23,333 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (1/4) (attempt #9) with attempt id
3e8bb0f0ec3899f62304a8dffa103faf to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:58:23,333 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (2/4) (attempt #9) with attempt id
c6864d5e4339977fec5e5b10b8cb9db7 to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:58:23,333 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (3/4) (attempt #9) with attempt id
e49c76a882344632c0d9a84d456e2700 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:58:23,333 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (4/4) (attempt #9) with attempt id
7b95374e5523ec14be73a17668467420 to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:58:23,333 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (1/4) (attempt #9) with attempt id
4bbd2a3aac0de6593f8199b0ef8eaaff to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:58:23,333 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (2/4) (attempt #9) with attempt id
22514f0ff054d425950b839e92aad3da to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:58:23,333 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (3/4) (attempt #9) with attempt id
adc4277261c074dc802e9f4c5ff80717 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:58:23,363 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (4/4) (attempt #9) with attempt id
b3f43cadfe0d2bd2eeff7ba8d2210f10 to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:58:23,846 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 19 (type=CHECKPOINT) @ 1613350703843 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:59:26,678 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 19 from task
3e8bb0f0ec3899f62304a8dffa103faf of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:59:27,346 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 19 from task
c6864d5e4339977fec5e5b10b8cb9db7 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:59:35,173 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 19 from task
7b95374e5523ec14be73a17668467420 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:59:36,477 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 19 from task
e49c76a882344632c0d9a84d456e2700 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:59:44,876 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 19 from task
b3f43cadfe0d2bd2eeff7ba8d2210f10 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 00:59:45,223 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 19 from task
22514f0ff054d425950b839e92aad3da of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 00:59:45,824 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 19 from task
adc4277261c074dc802e9f4c5ff80717 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 00:59:46,549 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 19 from task
4bbd2a3aac0de6593f8199b0ef8eaaff of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 00:59:48,874 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 20 (type=CHECKPOINT) @ 1613350788857 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 00:59:50,794 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (1/4) (attempt #10) with attempt id
ad1ef05bb02038bab84960023be2139f to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:59:50,795 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (2/4) (attempt #10) with attempt id
1eb939470ab93f6a50c6df4e0d5757a1 to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:59:50,796 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (3/4) (attempt #10) with attempt id
41b6ec89308b04922c9f043faa83c7b9 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:59:50,796 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (4/4) (attempt #10) with attempt id
89878b144856fead9161dafeed1b416d to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:59:50,796 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (1/4) (attempt #10) with attempt id
cc83d9b2bb53ac446aad4ed344259223 to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 00:59:50,797 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (2/4) (attempt #10) with attempt id
957a638e36686032cb9f9bd7144915f3 to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 00:59:50,797 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (3/4) (attempt #10) with attempt id
728e40516877edb97748f12a56b9a4c4 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 00:59:50,809 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (4/4) (attempt #10) with attempt id
072aa4c8afc4326e05d926468125cfac to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 00:59:51,077 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 21 (type=CHECKPOINT) @ 1613350791074 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 01:01:02,644 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 21 from task
1eb939470ab93f6a50c6df4e0d5757a1 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 01:01:02,652 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 21 from task
ad1ef05bb02038bab84960023be2139f of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 01:01:11,154 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 21 from task
89878b144856fead9161dafeed1b416d of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 01:01:12,482 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 21 from task
41b6ec89308b04922c9f043faa83c7b9 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 01:01:21,515 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 21 from task
cc83d9b2bb53ac446aad4ed344259223 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 01:01:22,282 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 21 from task
728e40516877edb97748f12a56b9a4c4 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:42443-4be16a @ fv-az127-111.internal.cloudapp.net (dataPort=40525).
2021-02-15 01:01:23,136 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 21 from task
957a638e36686032cb9f9bd7144915f3 of job dfa837311853a69cdca82db082270dff at
10.1.0.4:46341-c100f5 @ fv-az127-111.internal.cloudapp.net (dataPort=40557).
2021-02-15 01:01:23,385 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 21 from task
072aa4c8afc4326e05d926468125cfac of job dfa837311853a69cdca82db082270dff at
10.1.0.4:44619-9da455 @ fv-az127-111.internal.cloudapp.net (dataPort=40803).
2021-02-15 01:01:25,380 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 22 (type=CHECKPOINT) @ 1613350885367 for job
dfa837311853a69cdca82db082270dff.
2021-02-15 01:01:25,631 DEBUG
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Received
acknowledge message for checkpoint 22 from task
ad1ef05bb02038bab84960023be2139f of job dfa837311853a69cdca82db082270dff at
10.1.0.4:36079-9ecc1c @ fv-az127-111.internal.cloudapp.net (dataPort=43283).
2021-02-15 01:01:28,495 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (1/4) (attempt #11) with attempt id
19a29174acd3e16276da75a060a45f89 to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 01:01:28,498 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (2/4) (attempt #11) with attempt id
8ac7c1c87b1809407eec0d64bd20a7cb to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 01:01:28,502 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (3/4) (attempt #11) with attempt id
97f2b0292ba8efb63aea4b44e217f382 to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 01:01:28,504 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (4/4) (attempt #11) with attempt id
31d061561664d3b35d4ffb51060783e3 to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
2021-02-15 01:01:28,510 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (1/4) (attempt #11) with attempt id
a68642cf1f5c0d64625c07ee07dbb57c to 10.1.0.4:36079-9ecc1c @
fv-az127-111.internal.cloudapp.net (dataPort=43283) with allocation id
34ba1021819900f400145f4eeb68cf98
2021-02-15 01:01:28,511 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (2/4) (attempt #11) with attempt id
73854761d80223752d9e973e49899574 to 10.1.0.4:46341-c100f5 @
fv-az127-111.internal.cloudapp.net (dataPort=40557) with allocation id
7dbde52bef472bf82dfe6b9c51030bbe
2021-02-15 01:01:28,513 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (3/4) (attempt #11) with attempt id
cf6c2d52b4929d79db4a1db09ae7279d to 10.1.0.4:42443-4be16a @
fv-az127-111.internal.cloudapp.net (dataPort=40525) with allocation id
6bd434b58457e3ede2b354f5ec514bda
2021-02-15 01:01:28,514 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Unnamed (4/4) (attempt #11) with attempt id
ebb41ec1aaa5ba8ddf134b3f9ade46ac to 10.1.0.4:44619-9da455 @
fv-az127-111.internal.cloudapp.net (dataPort=40803) with allocation id
9e93afdbf0a591fe019a790882e49405
{code}
> "Local recovery and sticky scheduling end-to-end test" does not finish within
> 600 seconds
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-21329
> URL: https://issues.apache.org/jira/browse/FLINK-21329
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Affects Versions: 1.13.0
> Reporter: Robert Metzger
> Assignee: Matthias
> Priority: Critical
> Labels: test-stability
> Fix For: 1.13.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=13118&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee&t=ff888d9b-cd34-53cc-d90f-3e446d355529&l=38515
> {code}
> Feb 08 22:25:46
> ==============================================================================
> Feb 08 22:25:46 Running 'Local recovery and sticky scheduling end-to-end test'
> Feb 08 22:25:46
> ==============================================================================
> Feb 08 22:25:46 TEST_DATA_DIR:
> /home/vsts/work/1/s/flink-end-to-end-tests/test-scripts/temp-test-directory-46881214821
> Feb 08 22:25:47 Flink dist directory:
> /home/vsts/work/1/s/flink-dist/target/flink-1.13-SNAPSHOT-bin/flink-1.13-SNAPSHOT
> Feb 08 22:25:47 Running local recovery test with configuration:
> Feb 08 22:25:47 parallelism: 4
> Feb 08 22:25:47 max attempts: 10
> Feb 08 22:25:47 backend: rocks
> Feb 08 22:25:47 incremental checkpoints: false
> Feb 08 22:25:47 kill JVM: false
> Feb 08 22:25:47 Starting zookeeper daemon on host fv-az127-394.
> Feb 08 22:25:47 Starting HA cluster with 1 masters.
> Feb 08 22:25:48 Starting standalonesession daemon on host fv-az127-394.
> Feb 08 22:25:49 Starting taskexecutor daemon on host fv-az127-394.
> Feb 08 22:25:49 Waiting for Dispatcher REST endpoint to come up...
> Feb 08 22:25:50 Waiting for Dispatcher REST endpoint to come up...
> Feb 08 22:25:51 Waiting for Dispatcher REST endpoint to come up...
> Feb 08 22:25:53 Waiting for Dispatcher REST endpoint to come up...
> Feb 08 22:25:54 Dispatcher REST endpoint is up.
> Feb 08 22:25:54 Started TM watchdog with PID 28961.
> Feb 08 22:25:58 Job has been submitted with JobID
> e790e85a39040539f9386c0df7ca4812
> Feb 08 22:35:47 Test (pid: 27970) did not finish after 600 seconds.
> Feb 08 22:35:47 Printing Flink logs and killing it:
> {code}
> and
> {code}
> at
> org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalDriver.unhandledError(ZooKeeperLeaderRetrievalDriver.java:184)
> at
> org.apache.flink.shaded.curator4.org.apache.curator.framework.imps.CuratorFrameworkImpl$6.apply(CuratorFrameworkImpl.java:713)
> at
> org.apache.flink.shaded.curator4.org.apache.curator.framework.imps.CuratorFrameworkImpl$6.apply(CuratorFrameworkImpl.java:709)
> at
> org.apache.flink.shaded.curator4.org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:100)
> at
> org.apache.flink.shaded.curator4.org.apache.curator.shaded.com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
> at
> org.apache.flink.shaded.curator4.org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:92)
> at
> org.apache.flink.shaded.curator4.org.apache.curator.framework.imps.CuratorFrameworkImpl.logError(CuratorFrameworkImpl.java:708)
> at
> org.apache.flink.shaded.curator4.org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:874)
> at
> org.apache.flink.shaded.curator4.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:990)
> at
> org.apache.flink.shaded.curator4.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:943)
> at
> org.apache.flink.shaded.curator4.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:66)
> at
> org.apache.flink.shaded.curator4.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:346)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by:
> org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at
> org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
> at
> org.apache.flink.shaded.curator4.org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:862)
> ... 10 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
