I recently thought about the same idea. Use case for us would be to scale a job 0 instances. While this sounds useless at first, it can be quite powerful when trying to implement a feature like socket activation.
________________________________________ From: Maxim Khutornenko <ma...@apache.org> Sent: Wednesday, June 29, 2016 22:43 To: dev@aurora.apache.org Subject: [PROPOSAL] Job as a first-class citizen TL;DR - I am proposing we store and maintain job-level data (JobConfiguration [1]) instead of relying on storing everything in a TaskConfig [2]. Aurora storage currently does not have a concept of a "job" when it comes to services and adhoc jobs. Instead, it relies on a collection of TaskConfigs that represent a view of what the job state is. This is in stark contrast to cron jobs, which are already represented by the JobConfiguration struct. This lack of representation limits our ability to deliver richer features and may result in suboptimal design and storage utilization. Specifically, the following is currently impossible: - storing normalized job-level data without repeating it in every task (e.g. contactEmail, isService); - maintaining job-level data that may be different for every instance (SLA requirements, topology specs for stateful services and etc.); - knowing what the job instance count is without pulling all ACTIVE tasks and iterating over them. To address the above, I propose we start treating Aurora job as a tangible entity in the storage and specifically use JobConfiguration wherever applicable. As a welcome side effect, this will let us: - allow instantaneous job updates when job-level fields are updated (e.g. those that don't require instance restarts); - finally get rid of the deprecated Identity struct [3]; - reduce or completely eliminate DB garbage collection of abandoned job keys [4] Any thoughts, suggestions, objections? Thanks, Maxim [1] - https://github.com/apache/aurora/blob/4e28b9c8b29b66f2f10b0a6cafdec1f8e2c1bd7b/api/src/main/thrift/org/apache/aurora/gen/api.thrift#L316-L338 [2] - https://github.com/apache/aurora/blob/4e28b9c8b29b66f2f10b0a6cafdec1f8e2c1bd7b/api/src/main/thrift/org/apache/aurora/gen/api.thrift#L240-L284 [3] - https://issues.apache.org/jira/browse/AURORA-84 [4] - RowGarbageCollector: https://github.com/apache/aurora/blob/b24619b28c4dbb35188871bacd0091a9e01218e3/src/main/java/org/apache/aurora/scheduler/storage/db/RowGarbageCollector.java
