[
https://issues.apache.org/jira/browse/YARN-117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated YARN-117:
--------------------------------
Attachment: YARN-117-2.patch
This is an aggregate patch for jenkins and testing; YARN-530 is the subset for
real review
* contains a code-review of all {{innerInit()}}, {{innerStart()}} ops
* contain a code review of all {{innerStop()}} operations to harden against
null values (MAPREDUCE-3502).
* incorporates YARN-535 to fix a race condition in one test
> Enhance YARN service model
> --------------------------
>
> Key: YARN-117
> URL: https://issues.apache.org/jira/browse/YARN-117
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: YARN-117-2.patch, YARN-117.patch
>
>
> Having played the YARN service model, there are some issues
> that I've identified based on past work and initial use.
> This JIRA issue is an overall one to cover the issues, with solutions pushed
> out to separate JIRAs.
> h2. state model prevents stopped state being entered if you could not
> successfully start the service.
> In the current lifecycle you cannot stop a service unless it was successfully
> started, but
> * {{init()}} may acquire resources that need to be explicitly released
> * if the {{start()}} operation fails partway through, the {{stop()}}
> operation may be needed to release resources.
> *Fix:* make {{stop()}} a valid state transition from all states and require
> the implementations to be able to stop safely without requiring all fields to
> be non null.
> Before anyone points out that the {{stop()}} operations assume that all
> fields are valid; and if called before a {{start()}} they will NPE;
> MAPREDUCE-3431 shows that this problem arises today, MAPREDUCE-3502 is a fix
> for this. It is independent of the rest of the issues in this doc but it will
> aid making {{stop()}} execute from all states other than "stopped".
> MAPREDUCE-3502 is too big a patch and needs to be broken down for easier
> review and take up; this can be done with issues linked to this one.
> h2. AbstractService doesn't prevent duplicate state change requests.
> The {{ensureState()}} checks to verify whether or not a state transition is
> allowed from the current state are performed in the base {{AbstractService}}
> class -yet subclasses tend to call this *after* their own {{init()}},
> {{start()}} & {{stop()}} operations. This means that these operations can be
> performed out of order, and even if the outcome of the call is an exception,
> all actions performed by the subclasses will have taken place. MAPREDUCE-3877
> demonstrates this.
> This is a tricky one to address. In HADOOP-3128 I used a base class instead
> of an interface and made the {{init()}}, {{start()}} & {{stop()}} methods
> {{final}}. These methods would do the checks, and then invoke protected inner
> methods, {{innerStart()}}, {{innerStop()}}, etc. It should be possible to
> retrofit the same behaviour to everything that extends {{AbstractService}}
> -something that must be done before the class is considered stable (because
> once the lifecycle methods are declared final, all subclasses that are out of
> the source tree will need fixing by the respective developers.
> h2. AbstractService state change doesn't defend against race conditions.
> There's no concurrency locks on the state transitions. Whatever fix for wrong
> state calls is added should correct this to prevent re-entrancy, such as
> {{stop()}} being called from two threads.
> h2. Static methods to choreograph of lifecycle operations
> Helper methods to move things through lifecycles. init->start is common,
> stop-if-service!=null another. Some static methods can execute these, and
> even call {{stop()}} if {{init()}} raises an exception. These could go into a
> class {{ServiceOps}} in the same package. These can be used by those services
> that wrap other services, and help manage more robust shutdowns.
> h2. state transition failures are something that registered service listeners
> may wish to be informed of.
> When a state transition fails a {{RuntimeException}} can be thrown -and the
> service listeners are not informed as the notification point isn't reached.
> They may wish to know this, especially for management and diagnostics.
> *Fix:* extend {{ServiceStateChangeListener}} with a callback such as
> {{stateChangeFailed(Service service,Service.State targeted-state,
> RuntimeException e)}} that is invoked from the (final) state change methods
> in the {{AbstractService}} class (once they delegate to their inner
> {{innerStart()}}, {{innerStop()}} methods; make a no-op on the existing
> implementations of the interface.
> h2. Service listener failures not handled
> Is this an error an error or not? Log and ignore may not be what is desired.
> *Proposed:* during {{stop()}} any exception by a listener is caught and
> discarded, to increase the likelihood of a better shutdown, but do not add
> try-catch clauses to the other state changes.
> h2. Support static listeners for all AbstractServices
> Add support to {{AbstractService}} that allow callers to register listeners
> for all instances. The existing listener interface could be used. This allows
> management tools to hook into the events.
> The static listeners would be invoked for all state changes except creation
> (base class shouldn't be handing out references to itself at this point).
> These static events could all be async, pushed through a shared
> {{ConcurrentLinkedQueue}}; failures logged at warn and the rest of the
> listeners invoked.
> h2. Add some example listeners for management/diagnostics
> * event to commons log for humans.
> * events for machines hooked up to the JSON logger.
> * for testing: something that be told to fail.
> h2. Services should support signal interruptibility
> The services would benefit from a way of shutting them down on a kill signal;
> this can be done via a runtime hook. It should not be automatic though, as
> composite services will get into a very complex state during shutdown. Better
> to provide a hook that lets you register/unregister services to terminate,
> and have the relevant {{main()}} entry points tell their root services to
> register themselves.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
