[
https://issues.apache.org/jira/browse/SPARK-19978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-19978.
----------------------------------
Resolution: Incomplete
> spark thrift server to switch to normative hadoop 2.2+ service lifecycle
> ------------------------------------------------------------------------
>
> Key: SPARK-19978
> URL: https://issues.apache.org/jira/browse/SPARK-19978
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.2.0
> Reporter: Steve Loughran
> Priority: Minor
> Labels: bulk-closed
>
> Spark's hive-thriftserver module (for which there isn't a JIRA component yet,
> BTW), contains, in {{org.apache.hive.service}}, a copy and paste of Hive's
> service lifecyle classes.
> Hive's classes are in fact a copy and paste of the Hadoop 0.23 service
> lifecycle classes. Those were reworked, "fixed" for Hadoop 2.2 by all the
> changes in YARN-117
> in particular
> * restructuring of subclass point means that the state checks happen at the
> right place to stop re-entrancy
> * no race conditions on the methods
> * resilient stop
> * composite service handles failure of child services properly
> * very handy diagnostics logging of lifecycles
> * robust listener API.
> + various other fixes
> The hive thriftserver services can and should move to the hadoop 2.2 ones,
> which will take a couple of hours, testing included.
> Essentially
> # change parent
> # move subclass operations from init/start/stop to serviceInit(),
> serviceStart and serviceStop() methods
> # test
> The only troublespot is hive's calls take a HiveConf in init; I'd fix that by
> still taking a Hadoop Configuration (the superclass) but having a
> {[HiveService extends AbstractService}} that always makes sure the stored
> conf is a HiveConf, and add a getHiveConf() method for subclasses to access
> it. The subclasses will also get a guarantee that their
> {{serviceInit(Configuration)}} call will always be supplied a config which
> can be cast to HiveConf. I'll need to add a CompositeService.
> If people want this, I can do it. If they don't, I wont bother.
> As to why Hive have an out-of-date subclass of the original lifecycle stuff?
> That topic has been raise in the past, it always came down to "we wanted to
> run on older versions of Hadoop". They can revisit that discussion...doesn't
> mean other projects can't make their own choices
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
