Just to clear up this thread. Thanks to Kos we tracked the the condition, where Drill was setting its config before the whole quorum was available.
For reference I ended up implementing something a bit like: https://github.com/juju-solutions/layer-apache-spark/blob/master/reactive/spark.py#L159 Thanks guys, I now have a working basic Apache Drill cluster \o/ Tom -------------- Director Meteorite.bi - Saiku Analytics Founder Tel: +44(0)5603641316 (Thanks to the Saiku community we reached our Kickstart <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> goal, but you can always help by sponsoring the project <http://www.meteorite.bi/products/saiku/sponsorship>) On 31 May 2016 at 13:34, Konstantinos Tsakalozos < [email protected]> wrote: > Hm... I do not see anything wrong with this part of the code. Looking at > [0] you should be getting all three Zookeeper units/conversations. > > Would it be possible to share the charm you are building so that I can > test it locally or ping us on IRC channel #juju at freenode for a pair > debugging session? > > > [0] > https://github.com/juju-solutions/interface-zookeeper/blob/master/requires.py#L40 > > On Tue, May 31, 2016 at 3:19 PM, Tom Barber <[email protected]> > wrote: > >> Okay, slightly different question: >> >> zookeeper/0 active idle 2.0-beta7 8 >> 2181/tcp,9998/tcp 10.106.143.126 Ready (3 zk units) >> zookeeper/1 active idle 2.0-beta7 9 >> 2181/tcp,9998/tcp 10.106.143.234 Ready (3 zk units) >> zookeeper/2 active idle 2.0-beta7 10 >> 2181/tcp,9998/tcp 10.106.143.133 Ready (3 zk units) >> >> I have 3 ZK units and I used the example here: >> https://github.com/juju-solutions/interface-zookeeper for the client >> config so my function looks like: >> >> @when('zookeeper.ready') >> @when_not('charm.configured') >> def configure(zookeeper): >> zklist = '' >> for zk_unit in zookeeper.zookeepers(): >> zklist += add_zookeeper(zk_unit['host'], zk_unit['port']) >> zklist = zklist[:-1] >> >> But it only iterates once even though 3 are listed: >> >> a) did I do something wrong >> b) is this due to some leader election thing and is actually correct? >> c) another answer? >> >> Thanks >> >> Tom >> >> -------------- >> >> Director Meteorite.bi - Saiku Analytics Founder >> Tel: +44(0)5603641316 >> >> (Thanks to the Saiku community we reached our Kickstart >> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >> goal, but you can always help by sponsoring the project >> <http://www.meteorite.bi/products/saiku/sponsorship>) >> >> On 31 May 2016 at 12:37, Tom Barber <[email protected]> wrote: >> >>> Fair enough, that resolves that then! >>> >>> Thanks Kos. >>> >>> Tom >>> >>> -------------- >>> >>> Director Meteorite.bi - Saiku Analytics Founder >>> Tel: +44(0)5603641316 >>> >>> (Thanks to the Saiku community we reached our Kickstart >>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >>> goal, but you can always help by sponsoring the project >>> <http://www.meteorite.bi/products/saiku/sponsorship>) >>> >>> On 31 May 2016 at 12:25, Konstantinos Tsakalozos < >>> [email protected]> wrote: >>> >>>> Ah, it is clear now, thanks. >>>> >>>> The Apache Hadoop charms use Zookeeper when in HA mode. The namenode >>>> is the charm that relates to Zookeeper [0,1]. So to answer your question >>>> the Hadoop charms use ZK but they do not carry ZK under the hood. ZK >>>> is treated as separate service/charm. >>>> >>>> >>>> [0] https://github.com/juju-solutions/layer-apache-hadoop-namenode >>>> /blob/master/metadata.yaml >>>> [1] https://github.com/juju-solutions/layer-apache-hadoop-namenode >>>> /blob/master/reactive/namenode_failover.py >>>> >>>> On Tue, May 31, 2016 at 1:50 PM, Tom Barber <[email protected]> >>>> wrote: >>>> >>>>> No worries Kos >>>>> >>>>> I'm rubbish at explaining stuff properly! >>>>> >>>>> My query was more along the lines of "I can see the ZK charms >>>>> available, but surely the Hadoop charms use ZK under the hood somewhere, >>>>> do >>>>> any of them expose the ZK relation so you can hook into that", because it >>>>> seems like overkill running a second ZK setup if one already exists inside >>>>> an existing Juju Hadoop deployment. >>>>> >>>>> Tom >>>>> >>>>> >>>>> >>>>> -------------- >>>>> >>>>> Director Meteorite.bi - Saiku Analytics Founder >>>>> Tel: +44(0)5603641316 >>>>> >>>>> (Thanks to the Saiku community we reached our Kickstart >>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >>>>> goal, but you can always help by sponsoring the project >>>>> <http://www.meteorite.bi/products/saiku/sponsorship>) >>>>> >>>>> On 31 May 2016 at 11:47, Konstantinos Tsakalozos < >>>>> [email protected]> wrote: >>>>> >>>>>> I am sorry Tom, I replied to your first email. >>>>>> >>>>>> Not sure I fully get your question. Not sure what "this" refers to in >>>>>> "Is there a way to hook this". >>>>>> >>>>>> Charms interacting with a Zookeeper quorum need to use this >>>>>> interface: http://interfaces.juju.solutions/interface/zookeeper/ . >>>>>> Through the zookeeper interface you can be informed about zookeeper units >>>>>> joining and departing the quorum. >>>>>> >>>>>> You can have a single Zookeeper service that you can scale to an odd >>>>>> number of units. Then you can have any charm "requiring" the above >>>>>> relation >>>>>> use the zookeeper service. Typically you would need to construct a >>>>>> connection string or edit some configuration file within your charm in >>>>>> order to reference the zookeeper units. Have a look at https:// >>>>>> github.com/juju-solutions/layer-apache >>>>>> -spark/blob/master/lib/charms/layer/apache_spark.py#L175 >>>>>> >>>>>> Thanks, >>>>>> Konstantinos >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, May 31, 2016 at 1:28 PM, Konstantinos Tsakalozos < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Tom, >>>>>>> >>>>>>> Have you looked at: https://jujucharms.com/apache-zookeeper/trusty/3 >>>>>>> The apache-zookeeper provides the zookeeper interface, see >>>>>>> https://api.jujucharms.com/charmstore/v5/trusty/apache >>>>>>> -zookeeper-3/archive/metadata.yaml >>>>>>> >>>>>>> Thanks, >>>>>>> Konstantinos >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, May 31, 2016 at 1:13 PM, Tom Barber <[email protected] >>>>>>> > wrote: >>>>>>> >>>>>>>> Morning >>>>>>>> >>>>>>>> There is a zookeeper interface and a bucket load of Big Data stuff >>>>>>>> which all must run ZK under the hood. Is there anything that provides >>>>>>>> the >>>>>>>> relationit so my charm can hook into ZK? I can't see anything obvious >>>>>>>> in >>>>>>>> the Charm Store. >>>>>>>> >>>>>>>> Tom >>>>>>>> -------------- >>>>>>>> >>>>>>>> Director Meteorite.bi - Saiku Analytics Founder >>>>>>>> Tel: +44(0)5603641316 >>>>>>>> >>>>>>>> (Thanks to the Saiku community we reached our Kickstart >>>>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >>>>>>>> goal, but you can always help by sponsoring the project >>>>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>) >>>>>>>> >>>>>>>> -- >>>>>>>> Juju mailing list >>>>>>>> [email protected] >>>>>>>> Modify settings or unsubscribe at: >>>>>>>> https://lists.ubuntu.com/mailman/listinfo/juju >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
-- Juju mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
