Steve Loughran created SPARK-19978:
--------------------------------------
Summary: spark thrift server to switch to normative hadoop 2.2+
service lifecycle
Key: SPARK-19978
URL: https://issues.apache.org/jira/browse/SPARK-19978
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.2.0
Reporter: Steve Loughran
Priority: Minor
Spark's hive-thriftserver module (for which there isn't a JIRA component yet,
BTW), contains, in {{org.apache.hive.service}}, a copy and paste of Hive's
service lifecyle classes.
Hive's classes are in fact a copy and paste of the Hadoop 0.23 service
lifecycle classes. Those were reworked, "fixed" for Hadoop 2.2 by all the
changes in YARN-117
in particular
* restructuring of subclass point means that the state checks happen at the
right place to stop re-entrancy
* no race conditions on the methods
* resilient stop
* composite service handles failure of child services properly
* very handy diagnostics logging of lifecycles
* robust listener API.
+ various other fixes
The hive thriftserver services can and should move to the hadoop 2.2 ones,
which will take a couple of hours, testing included.
Essentially
# change parent
# move subclass operations from init/start/stop to serviceInit(), serviceStart
and serviceStop() methods
# test
The only troublespot is hive's calls take a HiveConf in init; I'd fix that by
still taking a Hadoop Configuration (the superclass) but having a {[HiveService
extends AbstractService}} that always makes sure the stored conf is a HiveConf,
and add a getHiveConf() method for subclasses to access it. The subclasses will
also get a guarantee that their {{serviceInit(Configuration)}} call will always
be supplied a config which can be cast to HiveConf. I'll need to add a
CompositeService.
If people want this, I can do it. If they don't, I wont bother.
As to why Hive have an out-of-date subclass of the original lifecycle stuff?
That topic has been raise in the past, it always came down to "we wanted to run
on older versions of Hadoop". They can revisit that discussion...doesn't mean
other projects can't make their own choices
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
