kl0u commented on issue #9974: [FLINK-14501][FLINK-14502] Decouple ClusterDescriptor/ClusterSpecification from CommandLine URL: https://github.com/apache/flink/pull/9974#issuecomment-546876357 Hi @TisonKun ! Thanks for the comments, I integrated them. Now for your question, I understand that there may be arguments on both sides but here are my thoughts on why using Java SPI: 1) Java-based service discovery is already used in some parts of the code e.g. `FileSystem` and Table API 2) We do not have to hardcode the available ClusterClientFactories and always update the list whenever a new one is added. I understand that we have to add it to a file, but I believe this is more maintainable as it is a predefined location and not somewhere in the code. In addition, IDEs have better integration for java-based SPI. Finally, I have not thoroughly checked but we may be able to add a "completeness" test in the future where we check for all production implementations of the ClusterClientFactory and if there is one that is not included in any META-INF, then fail the test. 2) In the future this discovery service is probably going to be the same that will also give the Executor, so we wanted it to be compatible with the isCompatibleWith method. On the Executor side, we do not want to have an FQN-based discovery strategy because we want to be flexible to change implementations, without having to ask users to change their configurations. For example, it seems that for now we will have one executor for every sessionCluster execution. In the future we may want to split Standalone from Yarn. If we had an FQN based strategy, and the user uses directly the configuration to ask for an executor, then we would have to ask to change the class name provided there. Please have another look at this PR, and if you are ok with it, I will go on and merge.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
