wangyang0918 commented on issue #10143: [FLINK-13184]Starting a TaskExecutor blocks the YarnResourceManager's main thread URL: https://github.com/apache/flink/pull/10143#issuecomment-553452755 @walterddr Thanks for your comments. I want to clarify again how the optimization works. First, all the `-yD` arguments and the `flink-conf.yaml` under flink lib will be merged to a new file `application_xxxx-flink-conf.yamlxxxx.tmp` in flink client. Then it will be uploaded to hdfs and added to local resource for jobmanager. Also it is added to envShipFileList so that the taskmanager could have the same `flink-conf.yaml` with jobmanager through Yarn distributed cache. Since we will update some config options in jobmanager(such as `taskmanager.memory.size`), so i suggest to pass this differences to taskmanager through dynamic properties. I think the updated config options in jobmanager are limited. So the dynamic properties will not have too many and the launch command will not be huge. @walterddr @TisonKun If you both insist that we should upload only one taskmanager config to hdfs instead of using dynamic properties, i will update the PR to use this way. However, i still think using the dynamic properties is a better solution. In addition, the launch_container.sh is not localized from hdfs. It is generated by Yarn NodeManager according to the `ContainerLaunchContext` through `NMClient#startContainer`.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
