huge +1, socket activation is our exact use case for this type of action also
-Jake On Wed, Jun 29, 2016 at 5:18 PM, Erb, Stephan <stephan....@blue-yonder.com> wrote: > I recently thought about the same idea. Use case for us would be to scale > a job 0 instances. While this sounds useless at first, it can be quite > powerful when trying to implement a feature like socket activation. > > ________________________________________ > From: Maxim Khutornenko <ma...@apache.org> > Sent: Wednesday, June 29, 2016 22:43 > To: dev@aurora.apache.org > Subject: [PROPOSAL] Job as a first-class citizen > > TL;DR - I am proposing we store and maintain job-level data > (JobConfiguration [1]) instead of relying on storing everything in a > TaskConfig [2]. > > > Aurora storage currently does not have a concept of a "job" when it > comes to services and adhoc jobs. Instead, it relies on a collection > of TaskConfigs that represent a view of what the job state is. This is > in stark contrast to cron jobs, which are already represented by the > JobConfiguration struct. > > This lack of representation limits our ability to deliver richer > features and may result in suboptimal design and storage utilization. > Specifically, the following is currently impossible: > > - storing normalized job-level data without repeating it in every task > (e.g. contactEmail, isService); > > - maintaining job-level data that may be different for every instance > (SLA requirements, topology specs for stateful services and etc.); > > - knowing what the job instance count is without pulling all ACTIVE > tasks and iterating over them. > > To address the above, I propose we start treating Aurora job as a > tangible entity in the storage and specifically use JobConfiguration > wherever applicable. As a welcome side effect, this will let us: > > - allow instantaneous job updates when job-level fields are updated > (e.g. those that don't require instance restarts); > - finally get rid of the deprecated Identity struct [3]; > - reduce or completely eliminate DB garbage collection of abandoned job > keys [4] > > Any thoughts, suggestions, objections? > > Thanks, > Maxim > > > [1] - > https://github.com/apache/aurora/blob/4e28b9c8b29b66f2f10b0a6cafdec1f8e2c1bd7b/api/src/main/thrift/org/apache/aurora/gen/api.thrift#L316-L338 > > [2] - > https://github.com/apache/aurora/blob/4e28b9c8b29b66f2f10b0a6cafdec1f8e2c1bd7b/api/src/main/thrift/org/apache/aurora/gen/api.thrift#L240-L284 > > [3] - https://issues.apache.org/jira/browse/AURORA-84 > > [4] - RowGarbageCollector: > > https://github.com/apache/aurora/blob/b24619b28c4dbb35188871bacd0091a9e01218e3/src/main/java/org/apache/aurora/scheduler/storage/db/RowGarbageCollector.java >
