rmetzger commented on pull request #13999: URL: https://github.com/apache/flink/pull/13999#issuecomment-724710253
Thanks a lot! The TPC-DS test looks fine, but it still seems there's something horribly wrong. From the logs you posted: ``` 2020-11-10T12:24:37.8306408Z Nov 10 12:24:37 ============================================================================== 2020-11-10T12:24:37.8307360Z Nov 10 12:24:37 Running 'Local recovery and sticky scheduling end-to-end test' 2020-11-10T12:24:37.8307813Z Nov 10 12:24:37 ============================================================================== 2020-11-10T12:24:37.8318695Z Nov 10 12:24:37 TEST_DATA_DIR: /home/vsts/work/1/s/flink-end-to-end-tests/test-scripts/temp-test-directory-37831251962 2020-11-10T12:24:38.0041905Z Nov 10 12:24:38 Flink dist directory: /home/vsts/work/1/s/flink-dist/target/flink-1.12-SNAPSHOT-bin/flink-1.12-SNAPSHOT 2020-11-10T12:24:38.0122238Z Nov 10 12:24:38 Running local recovery test with configuration: 2020-11-10T12:24:38.0123147Z Nov 10 12:24:38 parallelism: 4 2020-11-10T12:24:38.0124059Z Nov 10 12:24:38 max attempts: 10 2020-11-10T12:24:38.0124638Z Nov 10 12:24:38 backend: rocks 2020-11-10T12:24:38.0126122Z Nov 10 12:24:38 incremental checkpoints: true 2020-11-10T12:24:38.0126724Z Nov 10 12:24:38 kill JVM: true 2020-11-10T12:24:38.1930829Z Nov 10 12:24:38 Starting zookeeper daemon on host fv-az679-899. 2020-11-10T12:24:38.3802448Z Nov 10 12:24:38 Starting HA cluster with 1 masters. 2020-11-10T12:24:39.5396496Z Nov 10 12:24:39 Starting standalonesession daemon on host fv-az679-899. 2020-11-10T12:24:41.0508365Z Nov 10 12:24:41 Starting taskexecutor daemon on host fv-az679-899. 2020-11-10T12:24:41.1024434Z Nov 10 12:24:41 Waiting for Dispatcher REST endpoint to come up... 2020-11-10T12:24:42.1556438Z Nov 10 12:24:42 Waiting for Dispatcher REST endpoint to come up... 2020-11-10T12:24:43.4788593Z Nov 10 12:24:43 Waiting for Dispatcher REST endpoint to come up... 2020-11-10T12:24:44.5372485Z Nov 10 12:24:44 Waiting for Dispatcher REST endpoint to come up... 2020-11-10T12:24:45.5882111Z Nov 10 12:24:45 Dispatcher REST endpoint is up. 2020-11-10T12:24:45.5887480Z Nov 10 12:24:45 Started TM watchdog with PID 5352. 2020-11-10T12:24:53.0836657Z Nov 10 12:24:53 Job has been submitted with JobID ec86992d33ad0ee7e2662c5f8a5fe413 2020-11-10T12:26:53.2080580Z Nov 10 12:26:53 Test (pid: 23346) did not finish after 600 seconds. 2020-11-10T12:26:53.2081678Z Nov 10 12:26:53 Printing Flink logs and killing it: ``` It seems that the test gets killed way too early (earlier than 600 seconds), and that the test tries to stop a lot more TaskManagers than it started. I hope that https://github.com/apache/flink/pull/13996 will help detect such problems. I will merge your PR now. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
