[
https://issues.apache.org/jira/browse/AURORA-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531123#comment-14531123
]
brian wickman commented on AURORA-1288:
---------------------------------------
The rough outline would be to create the concept of a pystachio Unstructured
type -- this would be the equivalent of a scala Any type -- which means it
takes anything that is a Struct subclass. We'd then change the Job struct in
src/main/python/apache/aurora/config/schema/base.py to change task =
Required(Task) to task = Required(Unstructured). This means that we could take
a thermos-defined Task or YourCustomExecutorTask and plumb it through the
client via AuroraConfigLoader instead of pulling Thermos schemas in directly
from base.py.
There are not *that* many touch-points for job.task throughout the client, but
the ones that exist are significant, and abstracting them to be not
thermos-specific will require some effort. The biggest one that comes to mind
is moving resources=Resources from Task to Job, where it belongs, which, while
a tiny code change, is a huge deprecation cycle on our end. If you look in
src/main/python/apache/aurora/config/thrift.py, most of that is generalizable
to any task type. It's only the last few lines where we do Thermos-specific
validation and generate a Thermos-specific ExecutorConfig that will need to be
pluggable based upon the task type.
> Design for supporting custom executor
> -------------------------------------
>
> Key: AURORA-1288
> URL: https://issues.apache.org/jira/browse/AURORA-1288
> Project: Aurora
> Issue Type: Task
> Reporter: Meghdoot Bhattacharya
>
> The goal is to capture the list of changes in the client and the scheduler
> required to support any executor other than thermos. This will help non
> thermos use cases to adopt aurora easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
