Hi all, I wanted to discuss with the community the direction custom executor support will take in Aurora. I think it's a good time to start a discussion on the vision everyone has in order to determine the best direction to take.
I hope we can settle on something concrete vision sooner than later in order to deliver this feature in a timely manner (hopefully before Mesoscon!). Summary of what's been discussed so far: What's been settled: * Configuration for executor should be be loaded by the scheduler for security reasons. (At least for the time being) (https://issues.apache.org/jira/browse/AURORA-1288?focusedCommentId=14601470&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14601470) * It would be better to create this feature in such a way that it's prepared to adapt the future REST API. Therefore, it doesn't make sense to extend the DSL to support custom executors. The consequence of this is that a custom client will be necessary to run custom executors (for now). (http://mail-archives.apache.org/mod_mbox/aurora-dev/201507.mbox/%3CCAGRA8uNPsaVJud1HObGFfq9jN43TxJK3CUe-SzfWwP8by_gn%2Bg%40mail.gmail.com%3E) * We will be using a config file (as discussed earlier in the mailing list) and it will be in JSON Things up in the air: * What should the config file look like? E.g.: Should it mimic ExecutorSettings? Should it mimic the Mesos TaskInfo? If we mimic TaskInfo, do we ditch ExecutorSettings? * Do we provide an interface to allow users to build the Mesos tasks themselves? Or do we have a generic Mesos task with some required fields and some not required? - If we take the route of using a generic Mesos tasks, how do we take this direction with minimal effect on thermos specific settings? - If we take the route of allowing users to build the tasks themselves, what would be the best way to accomplish this? For example, how should we call the correct task builder? Do we make it dynamic based on the executor name per job or static for the entire scheduler? - Some discussion about this has been taking place here (https://reviews.apache.org/r/36289/) Looking forward to hearing some thoughts on this. -Renan
