Re: Is there a universal interface I can use?
On 28.11.2017 00:38, Stuart Bishop wrote: > On 23 November 2017 at 21:37, Tilman Baumann > <tilman.baum...@canonical.com> wrote: >> I didn't want to talk about it before it's usable. I think I might be >> working on something similar. >> >> https://github.com/tbaumann/jujucharm-layer-cassandra-backup >> >> It seems to only use "nodetool snapshot" >> I'm integrating this for a 3rd party so I don't quite know what is going >> on there. But looks like the intent is pretty much the same. > > I think this charm needs to remain a subordinate, because 'nodetool > snapshot' requires a JMX connection and that should be blocked > (because it is not secured). > > I'd be happy to have actions on the main Cassandra charm to manage > snapshots, and cronned snapshots would also be a feature suitable for > the main charm. But you would still need some way to ship the > snapshots to your backup host which should be a subordinate. That would be quite nice actually. Backup and snapshot could be two different actions even. Snapshot is a little low-level as it is per-node. But it makes for fast recovery if a node hickups. Full backup like Haw Leoeung implemented in https://jujucharms.com/u/hloeung/cassandra-backup/ is probably even more useful for many scenarios. I'm quite stumped right now with the odd combination of needing to be a subordinate and needing to connect to the database relation. I just can't get it to work. And I'm out of ideas. I would love to just finish it "to get it to work" but I don't know what else I could try at this point. Generally I like the idea of just plonking on subordinates to extend features. But backup is a quite central one. Makes sense in the main charm. The percona charm has for example a backup action too. I just checked. It also just places a file in the file-system. (No cron scheduled one. But that would be useful there too) Very nice would be a unified backup plugin thing that lives in charmhelpers and could interact with some backup server. Wishful thinking... :D Might eve be a nice use of the not yet supported shared storage thing https://jujucharms.com/docs/stable/charms-storage I'm not very familiar with the coding style used in the cassandra charm. But I think I could help you with adding those functionalities even. I will have a lot of distractions the next two weeks. But I can see what I can do... Thanks Tilman -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Is there a universal interface I can use?
Hey John, apart from those fields missing. (BTW check usage section of https://jujucharms.com/cassandra/36 for a list of fields) interface.yaml calls this interface elasticsearch. I was able to use this after I changed that. ;-) Cheers Tilman On 23.11.2017 11:39, Tilman Baumann wrote: > Cool. Thanks > > The two fields I was interested in, username and password are mising > though. :D > But I'm thinking right now if I even want to go that route... > > On 23.11.2017 04:01, John Meinel wrote: >> I did start working on a Cassandra interface for something I was working >> on. I don't know that it is complete but >> https://github.com/jameinel/interface-cassandra >> >> Was my attempt at it. >> >> John >> =:-> -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Is there a universal interface I can use?
On 22.11.2017 18:51, Cory Johns wrote: > However, I see that you also want to retrieve relation data that the > cassandra charm provides using the "cassandra" interface protocol. Here > it becomes important to note that whether a relation is subordinate or > not is independent of the interface protocol the relation uses. Thus, > you could mark the endpoint as "scope: container" and still use the > "cassandra" interface protocol. You would then want to drop the > "host-system" endpoint from your charm, mark the "database" endpoint as > "interface: cassandra", and connect as you were trying to do. However, > I seem to recall that Stuart (the maintainer of the cassandra charm) > raised a possible issue with subordinates using relations that the other > end doesn't expect to be subordinate, so there might be some concerns > there. Fascinatingly this works. I always thought the implicit juju-info relation is something that is only implicit in the provides section. Very useful. So I tried this requires: database: interface: cassandra scope: container And indeed cassandra:database connects with cassandra-backup:database and creates a subordinate charm. Wonderful. But it doesn't actually work. :/ $ juju run --unit cassandra-backup/7 -- relaton-ids database /tmp/juju-exec940792089/script.sh: line 1: relaton-ids: command not found $ juju run --unit cassandra-backup/7 -- relation-ids database database:499 $ juju run --unit cassandra-backup/7 -- relation-list -r database:499 cassandra-analytics/4 $ juju run --unit cassandra-backup/7 -- relation-get -r database:499 - cassandra-analytics/4 private-address: 100.107.66.2 There is basically nothing to read in that relation. If I compare that with a properly connected relation to the same cassandra instance: $ juju run --unit contrail-analytics/1 -- relation-get -r cassandra:277 - cassandra-analytics/4 cluster_name: juju datacenter: juju host: XX.YY.0.119 native_transport_port: "9042" password: XX private-address: 100.107.66.2 rpc_port: "9160" username: juju_contrail-analytics So something, perhaps the scope, makes this not work. -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Is there a universal interface I can use?
On 22.11.2017 23:26, Haw Loeung wrote: > Hi Tilman, > > On Wed, Nov 22, 2017 at 04:02:08PM +0100, Tilman Baumann wrote: >> However, that doesn't seem to work. Juju complains the relation doesn't >> exist. >> $ juju add-relation cassandra-backup:database cassandra:database >> ERROR no relations found >> >> So, is there a interface that I can (ab-)use in a similar way? >> >> I don't want to build a full blown cassandra interface and at it to the >> list. > > Not sure if you've seen this, but I did some work recently with > something similar to backup Cassandra DBs: > > | https://jujucharms.com/u/hloeung/cassandra-backup/ > > It's currently still experimental and uses the CQL COPY > TO... commands. The Cassandra charm already ships out the required > credentials so tools such as cqlsh should just work. In light of this, I should perhaps consider renaming my charm cassandra-snapshots. End goal might be to combine the two. signature.asc Description: OpenPGP digital signature -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Is there a universal interface I can use?
Cool. Thanks The two fields I was interested in, username and password are mising though. :D But I'm thinking right now if I even want to go that route... On 23.11.2017 04:01, John Meinel wrote: > I did start working on a Cassandra interface for something I was working > on. I don't know that it is complete but > https://github.com/jameinel/interface-cassandra > > Was my attempt at it. > > John > =:-> > > On Nov 23, 2017 02:26, "Haw Loeung" <haw.loe...@canonical.com > <mailto:haw.loe...@canonical.com>> wrote: > > Hi Tilman, > > On Wed, Nov 22, 2017 at 04:02:08PM +0100, Tilman Baumann wrote: > > However, that doesn't seem to work. Juju complains the relation > doesn't > > exist. > > $ juju add-relation cassandra-backup:database cassandra:database > > ERROR no relations found > > > > So, is there a interface that I can (ab-)use in a similar way? > > > > I don't want to build a full blown cassandra interface and at it > to the > > list. > > Not sure if you've seen this, but I did some work recently with > something similar to backup Cassandra DBs: > > | https://jujucharms.com/u/hloeung/cassandra-backup/ > <https://jujucharms.com/u/hloeung/cassandra-backup/> > > It's currently still experimental and uses the CQL COPY > TO... commands. The Cassandra charm already ships out the required > credentials so tools such as cqlsh should just work. > > > Regards, > > Haw > > -- > Juju mailing list > Juju@lists.ubuntu.com <mailto:Juju@lists.ubuntu.com> > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > <https://lists.ubuntu.com/mailman/listinfo/juju> > > > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Is there a universal interface I can use?
On 22.11.2017 23:26, Haw Loeung wrote: > Hi Tilman, > > On Wed, Nov 22, 2017 at 04:02:08PM +0100, Tilman Baumann wrote: >> However, that doesn't seem to work. Juju complains the relation doesn't >> exist. >> $ juju add-relation cassandra-backup:database cassandra:database >> ERROR no relations found >> >> So, is there a interface that I can (ab-)use in a similar way? >> >> I don't want to build a full blown cassandra interface and at it to the >> list. > > Not sure if you've seen this, but I did some work recently with > something similar to backup Cassandra DBs: > > | https://jujucharms.com/u/hloeung/cassandra-backup/ I didn't want to talk about it before it's usable. I think I might be working on something similar. https://github.com/tbaumann/jujucharm-layer-cassandra-backup It seems to only use "nodetool snapshot" I'm integrating this for a 3rd party so I don't quite know what is going on there. But looks like the intent is pretty much the same. > It's currently still experimental and uses the CQL COPY > TO... commands. The Cassandra charm already ships out the required > credentials so tools such as cqlsh should just work. Good point. And worst case I can read the yaml file for cassandra myself. Saves me a cumbersome additional relation. I was undecided about this. But this is shaping up to be the cleaner solution... signature.asc Description: OpenPGP digital signature -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Is there a universal interface I can use?
I'm writing a reactive subordinate charm for cassandra.
I can not find a interface for cassandra. But that's ok, since I don't
really need a full blown database connection client.
Easy I thought and just re-used the juju-info interface for fun and profit.
requires:
host-system:
interface: juju-info
scope: container
database:
interface: juju-info
And in the code
@when('database.available')
def db_changed(cassandra):
for conv in cassandra.conversations():
username = conv.get_remote('username')
password = conv.get_remote('password')
...
However, that doesn't seem to work. Juju complains the relation doesn't
exist.
$ juju add-relation cassandra-backup:database cassandra:database
ERROR no relations found
So, is there a interface that I can (ab-)use in a similar way?
I don't want to build a full blown cassandra interface and at it to the
list.
Cheers and thanks
Tilman
--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
Kafka support for filebeat, topbeat and packetbeat
I just added Kafka output to filebeat, topbeat and packetbeat. If anyone is interested... "charm test" is broken for me, apologies if I committed a broken test. I manually tested the filebeat changes, the rest is just copy-pasta. (same code everywhere) https://github.com/juju-solutions/layer-beats-base/pull/15 https://github.com/juju-solutions/layer-filebeat/pull/19 https://github.com/juju-solutions/layer-topbeat/pull/13 https://github.com/juju-solutions/layer-packetbeat/pull/5 Cheers Tilman Baumann -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Is a set state called multiple times?
The confusion comes from the expectations that states are somehow events
and @when decorators are event handlers.
@when are predicates on states. When predicates are true the decorated
code is executed at every hook invocation.
If you want to trigger on edges (state changes) you need to build flipflops.
@when("state.wanted")
@when_not("state.reached")
do stuff here when entering state
set_state("state.reached")
@when_not("state.wanted")
@when("state.reached")
do stuff here when leaving state
remove_state("state.reached")
Reactive is still quite nice after you wrapped your head around that.
// Rant follows
Shame though that ALL interfaces are broken and thus unusable.
Interfaces need to be fixed on a fundamental level IMHO.
I have a workaround proposal
https://github.com/tbaumann/charm-interface-peer-discovery/blob/master/peers.py
But in truth, that is not a great solution, it's just a workaround.
I know I'm derailing the conversation here completely, but the
brokenness of interfaces has been a personal annoyance for a while and I
wish I could do something to change that.
One of the issues is that one can not react on "relation.connected"
states because the context they are supposed to have (argument to
decorated function) only works in the right hook context.
I propose a @interface decorator which will create the right context and
instantiate a interface object.
That way I can use interfaces in all sorts of functions and not only in
hook context and the kludges around to avoid being called outside of
hook context.
Problems with interfaces:
- *.connected state gets dropped when one unit leaves. Not when the
last one leaves.
- self.* only works when code runs in the right hook context
- interface class reference is only passed in decorated functions if the
hook inside the interface code ran
- Only the first occurrence of a unit joining a relation can be caught
with handler
- Interfaces have 90% identical code which is just boilerplate
Just my 0.02€
On 27.07.2017 03:37, fengxia wrote:
> Hi Juju,
>
> Once I set a state, set_state("here"), I want to make sure its @when
> will only be executed ONCE (when "here" from False->True).
>
> So my thought is to remove_state("here") in its @when("here") code
> block. If I don't, will this @when be called multiple times if I don't
> reset this state? What's the good practice here?
>
--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
Re: Is base64 really the best way to use complex config values
On 08.06.2017 13:39, Stuart Bishop wrote: > It is only popular because people keep cargo culting it > into their charms when it is unnecessary. I always call it out in > reviews and get people to switch to unencoded text. On the topic of cargo-culting. I'm re-writing a charm right now. Partly because I must. But also partly because I slightly disagree with the approach taken. (Simplicity vs. future proofing) Question. In upgrade-charm, can I see from which version I'm coming? I need to do cleanup when coming from a older version. I have other ways of detecting that, but would be cool if it had a similar logic to dpkg postinst where I get told where I'm coming from. And how about changing config options? I like to split one option into multiple. I could parse out the old values and transform them into the new config fields. I don't expect that to be possible though. What is the policy there. OK to break API? Can I remove options? How will the user notice? Cheers Tilman -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Is base64 really the best way to use complex config values
I see a lot of charms use base64 values in config parameters. Especially
when the values are stuff like custom templates.
Is this really the way to go? It may avoid shell quoting hell for
parameters set via command line. (Usually trivial)
But when set via --file option (which is clearly the better way with
complex fields) then I have to say the 'here document' features of YAML
are actually quite good.
The problem I see with bas64 is that nobody can read it without decoding
it every time.
Opinions?
Just as a example of what I mean:
application: logstash-supportcloud
charm: logstash-conf-d
settings:
config:
value: |
filter {
if [message] == "" {
drop { }
}
}
output {
gelf {
host => "foobar"
port => "5002"
}
}
Cheers
Tilman Baumann
--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
Re: juju test doesn't work
On 31.05.2017 14:39, Tim Van Steenburgh wrote: > But, I couldn't. The review form at https://review.jujucharms.com/ will > respond with a empty response (Firefox says https protocol error. Chrome > says empty response.) > Where would I report that as a bug? > > > There's a "report bug" link at the bottom of the review queue web pages. > I went ahead and filed a bug for the problem you're > hitting: https://github.com/juju-solutions/review-queue/issues/86 Cool thanks. I see you already found out what the issue is. Great. Cheers Tilman -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: juju test doesn't work
On 31.05.2017 10:45, John Meinel wrote: > I'm pretty sure 'charm' tools have moved over to using 'snap install > charm' as the preferred method for getting the charm tools. I'm not sure > that there is a way to deprecate/remove the versions that are in the > archives. For something like Zesty, it probably would have been possible > to just remove it, I think it was just an oversight. Yea, I found references for that. But we can't just abandon the dpkg package without letting the users know. :) Replace the binary with 'echo use snap you fool' would be fine with me. However, I should have mentioned. I did see that and tried that. The charm command provided by the snap does not have a 'build' option. $ charm build ERROR unrecognized command: charm build > I'm not sure about 'juju test' itself. It is actually a plugin supplied > by the charm tools (I believe) rather than something that is part of the > core 'juju' packaging. (The binary is actually /usr/bin/juju-test, and a > while back people asked that anything we find in $PATH as 'juju-foo' can > be called as just 'juju foo') Yes, you are right. That did come from the charm-tools package. And it is also not there with the charm snap. > I *do* believe that there was a push to actually split "tools for Charm > developers" away from "tools for people operating software with Juju". > So likely there is something like "charm test" that you should actually > be using, rather than an old, deprecated 'juju-test' that we shouldn't > be installing anymore. test as a charm command makes a lot of sense. And IIRC it already exists, I think I used juju test mostly because of lingering docu references to it. But again, only the packeted version does that. The snap charm command has no test command. > On Wed, May 31, 2017 at 11:57 AM, Tilman Baumann > <tilman.baum...@canonical.com <mailto:tilman.baum...@canonical.com>> wrote: > [...] > > During this development I found a number of bugs that made it quite hard > to be productive. -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
juju test doesn't work
I prepared two min charms which I needed for a customer. cs:~tilmanbaumann/logstash-conf-d-1 https://github.com/tbaumann/charm-logstash-conf-d cs:~tilmanbaumann/logstash-input-elasticsearch-1 https://github.com/tbaumann/charm-logstash-input-elasticsearch Those are subordninate charms that can be used like plugins to load custom configurations in logstash (/etc/logstash/conf.d/) Or to connect logstash to elasticsearch via relations. During this development I found a number of bugs that made it quite hard to be productive. Charm tools is broken. Thank god there is a custom ppa for it. https://bugs.launchpad.net/ubuntu/+source/charm-tools/+bug/1660004 And juju test doesn't work. That is why my charms don't have tests yet. https://bugs.launchpad.net/juju/+bug/1608723 I felt quite confident about my manual tests and I see no reason to believe #1608723 will be gone soon, so I decided to submit my charms for review anyway. Probably wasting your time. But I was eager to see some feedback. But, I couldn't. The review form at https://review.jujucharms.com/ will respond with a empty response (Firefox says https protocol error. Chrome says empty response.) Where would I report that as a bug? So you see, right now quite a lot is broken for charm developers. At least those running Zesty. Please excuse my mini rant about infrastructure. I suppose this will get better again. But perhaps someone can offer some opinions about my charms? :D Cheers Tilman Baumann PS: https://github.com/tbaumann/charm-logstash-conf-d has a tests branch with a initial testing idea. But I could never test it... The layer code is slightly more current than the published charms. I found a few typos I think. -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: normal charm to subordinate charm and now peer relation does not work
OMG, I went way deep down the rabbit hole.
I had to find charms.reactive.join() to find what the difference of the
scopes really means.
This kept me busy and questioning my sanity for two days:
$ git diff
diff --git a/peers.py b/peers.py
index 976c0ad..93cda29 100644
--- a/peers.py
+++ b/peers.py
@@ -12,7 +12,7 @@ from charms.reactive import scopes
class PeerDiscovery(RelationBase):
-scope = scopes.UNIT
+scope = scopes.SERVICE
class states(StateList):
connected = State('{relation_name}.connected')
Sometimes software engineering feels just like a farce. :)
Cheers guys, I hope you where entertained.
Tilman
On 25.01.2017 16:54, Tilman Baumann wrote:
> On 25.01.2017 15:49, Tilman Baumann wrote:
>> On 25.01.2017 14:24, Tilman Baumann wrote:
>>> On 25.01.2017 13:16, Stuart Bishop wrote:
>>>> On 25 January 2017 at 18:43, Tilman Baumann
>>>
>>>> I don't know why your peer relation (with global scope) starts
>>>> misbehaving after you add the container scoped juju-info relation to
>>>> turn your charm into a subordinate. It might be helpful to inspect the
>>>> peer relation with the hook environment tools to try to narrow down if
>>>> the problem is with Juju, charms.reactive, or something else. Using
>>>> debug-hooks, or 'juju run --unit foo/0 "relation-ids ssh-peers"' and
>>>> 'juju run --unit foo/0 "relation-list -r ssh-peers:64"' if you haven't
>>>> done this before.
>>>
>>> Thanks. That was the breakthrough hint.
>>> [...]
>>> It is either reactive or my charm code.
>>
>> Reactive.
>>
>> Shortened version of my debug interface code:
>>
>> @hook('{peers:peer-discovery}-relation-{joined,changed}')
>> def changed(self):
>> for conv in self.conversations():
>> log("JujuInfoClient Conversation.serialize():
>> {}".format(conv.__class__.serialize(conv)))
>>
>> I get:
>> Conversation.serialize(): {'scope': 'iptables-peer-ssh/102',
>> 'namespace': 'ssh-peers:105', 'units': ['iptables-peer-ssh/102']}
>>
>> I have to go deeper down the rabbit hole. But somehow the Conversations
>> get initialised with bullshit.
>>
>> This would be a example of a correctly initialised conversion.
>> {'scope': 'global', 'units': ['dokuwiki/25'], 'namespace': 'host-system'}
>>
>> I will go bare-metal with charmhelpers and see what I get there...
>>
>
> charmhelpers.core.hookenv tools seem to perform as expected
>
> units = related_units()
> for unit in units:
> relation_get('private-address', unit)
>
> I will bodge up a workaround with that now in order to be able to
> deploy. But surely there is something wrong in the way RelationBase gets
> initialised...
>
--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
Re: normal charm to subordinate charm and now peer relation does not work
On 25.01.2017 15:49, Tilman Baumann wrote:
> On 25.01.2017 14:24, Tilman Baumann wrote:
>> On 25.01.2017 13:16, Stuart Bishop wrote:
>>> On 25 January 2017 at 18:43, Tilman Baumann
>>
>>> I don't know why your peer relation (with global scope) starts
>>> misbehaving after you add the container scoped juju-info relation to
>>> turn your charm into a subordinate. It might be helpful to inspect the
>>> peer relation with the hook environment tools to try to narrow down if
>>> the problem is with Juju, charms.reactive, or something else. Using
>>> debug-hooks, or 'juju run --unit foo/0 "relation-ids ssh-peers"' and
>>> 'juju run --unit foo/0 "relation-list -r ssh-peers:64"' if you haven't
>>> done this before.
>>
>> Thanks. That was the breakthrough hint.
>> [...]
>> It is either reactive or my charm code.
>
> Reactive.
>
> Shortened version of my debug interface code:
>
> @hook('{peers:peer-discovery}-relation-{joined,changed}')
> def changed(self):
> for conv in self.conversations():
> log("JujuInfoClient Conversation.serialize():
> {}".format(conv.__class__.serialize(conv)))
>
> I get:
> Conversation.serialize(): {'scope': 'iptables-peer-ssh/102',
> 'namespace': 'ssh-peers:105', 'units': ['iptables-peer-ssh/102']}
>
> I have to go deeper down the rabbit hole. But somehow the Conversations
> get initialised with bullshit.
>
> This would be a example of a correctly initialised conversion.
> {'scope': 'global', 'units': ['dokuwiki/25'], 'namespace': 'host-system'}
>
> I will go bare-metal with charmhelpers and see what I get there...
>
charmhelpers.core.hookenv tools seem to perform as expected
units = related_units()
for unit in units:
relation_get('private-address', unit)
I will bodge up a workaround with that now in order to be able to
deploy. But surely there is something wrong in the way RelationBase gets
initialised...
--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
Re: normal charm to subordinate charm and now peer relation does not work
On 25.01.2017 14:24, Tilman Baumann wrote:
> On 25.01.2017 13:16, Stuart Bishop wrote:
>> On 25 January 2017 at 18:43, Tilman Baumann
>
>> I don't know why your peer relation (with global scope) starts
>> misbehaving after you add the container scoped juju-info relation to
>> turn your charm into a subordinate. It might be helpful to inspect the
>> peer relation with the hook environment tools to try to narrow down if
>> the problem is with Juju, charms.reactive, or something else. Using
>> debug-hooks, or 'juju run --unit foo/0 "relation-ids ssh-peers"' and
>> 'juju run --unit foo/0 "relation-list -r ssh-peers:64"' if you haven't
>> done this before.
>
> Thanks. That was the breakthrough hint.
> [...]
> It is either reactive or my charm code.
Reactive.
Shortened version of my debug interface code:
@hook('{peers:peer-discovery}-relation-{joined,changed}')
def changed(self):
for conv in self.conversations():
log("JujuInfoClient Conversation.serialize():
{}".format(conv.__class__.serialize(conv)))
I get:
Conversation.serialize(): {'scope': 'iptables-peer-ssh/102',
'namespace': 'ssh-peers:105', 'units': ['iptables-peer-ssh/102']}
I have to go deeper down the rabbit hole. But somehow the Conversations
get initialised with bullshit.
This would be a example of a correctly initialised conversion.
{'scope': 'global', 'units': ['dokuwiki/25'], 'namespace': 'host-system'}
I will go bare-metal with charmhelpers and see what I get there...
--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
Re: normal charm to subordinate charm and now peer relation does not work
On 25.01.2017 13:16, Stuart Bishop wrote: > On 25 January 2017 at 18:43, Tilman Baumann > I don't know why your peer relation (with global scope) starts > misbehaving after you add the container scoped juju-info relation to > turn your charm into a subordinate. It might be helpful to inspect the > peer relation with the hook environment tools to try to narrow down if > the problem is with Juju, charms.reactive, or something else. Using > debug-hooks, or 'juju run --unit foo/0 "relation-ids ssh-peers"' and > 'juju run --unit foo/0 "relation-list -r ssh-peers:64"' if you haven't > done this before. Thanks. That was the breakthrough hint. $ juju run --unit iptables-peer-ssh/97 "relation-list -r ssh-peers:103" iptables-peer-ssh/95 iptables-peer-ssh/98 iptables-peer-ssh/99 $ juju run --unit iptables-peer-ssh/97 "relation-get -r ssh-peers:103 private-address iptables-peer-ssh/95" 172.16.254.217 $ juju run --unit iptables-peer-ssh/97 "relation-get -r ssh-peers:103 private-address iptables-peer-ssh/98" 172.16.254.232 $ juju run --unit iptables-peer-ssh/97 "relation-get -r ssh-peers:103 private-address iptables-peer-ssh/99" 172.16.254.43 So clearly, the problem is not juju. I can see all relations. It is either reactive or my charm code. > One thing to remember is that units join the peer relation one at a > time. So in your peer relation-joined hook, you will only see a single > unit. Then a relation-changed, again with a single unit. And maybe a few > more times with a single unit. Eventually the rest of the units will > join, each time triggering relation-changed one or more times. Maybe > your problem is just that you are looking too soon :) Hm, perhaps that is the case. Though I should get the joined event multiple times until the last unit has joined. But I think I can keep working on this now, once the problem is isolated a little more. -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: normal charm to subordinate charm and now peer relation does not work
On 25.01.2017 13:06, Alex Kavanagh wrote: > Hi Tilman > > On Wed, Jan 25, 2017 at 11:43 AM, Tilman Baumann > <tilman.baum...@canonical.com <mailto:tilman.baum...@canonical.com>> wrote: > > At this point I'm pretty sure that this is a bug or undocumented > feature. > > > The peer relation of a subordinate charm only has one conversation. > Despite scope being 'global' in metadata.yaml and the RelationBase class > being scope = scope.UNIT. > > > Shouldn't the metadata.yaml ALSO be 'unit' scope if you want individual > conversations without sharing the data between all the subordinate > units? global would imply that all of the relations have the same > data? The subordinate to principal relation will be unit (anyway), but > the peer is just like any other relation, except here it is between the > subordinate's on multiple machines. The naming is quite confusing here. There is a bug for that. https://bugs.launchpad.net/juju/+bug/1499900 Container in this case means it 'converses' with the other services on the same unit. Global means, it can talk to everything. This should really be unified. The naming in reactive seems more intuitive. But in the end reactive is a layer on-top of charms, I guess reactive should have stayed with the established nomenclature here. -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: normal charm to subordinate charm and now peer relation does not work
At this point I'm pretty sure that this is a bug or undocumented feature. The peer relation of a subordinate charm only has one conversation. Despite scope being 'global' in metadata.yaml and the RelationBase class being scope = scope.UNIT. Either I'm wrong to expect this to work and subordinates are only supposed to have container scopes. Then it is a dokufix and should be caugt by charm proof. Or, it is a bug. I could not find the point in the code which could be wrong here. If anyone points me to the right place I would not mind working on a fix. In go or python. Thanks Tilman PS: File as bug in LP? On 25.01.2017 11:00, Tilman Baumann wrote: > On 24.01.2017 16:56, Alex Kavanagh wrote: >> Hi Tilman >> >> (I'm not an expert here, but was staring at the docs) >> >> I suspect that your peers relationship should be unit if each peer needs >> to have it's own conversation? Otherwise, with a global scope, every >> peer will overwrite the other's information? At least I'm wondering if >> that what the scopes mean: see >> here: https://jujucharms.com/docs/2.0/developer-layers-interfaces >> >> If that's completely wrong, then a) sorry for the noise, and b) do tell, >> as it will help me in my understanding of juju scopes. > > No I think this must generally be the direction I need to think towards. > Any thought impulse in that direction can unlock the knot in my head. :) > > Thing is, my interface class is scope UNIT already. (GLOBAL and SERVICE > definitely would be wrong I think) > > What hasn't really connected in my brain is how this relates to the > scopes in metadata.yaml > They have different values. I could only find documentation for global > and container. Global apparently is the default. So that is what I set > it to explicitly. I'm quite sure that I had tried it with 'container' > too in one of the iterations of testing. > > I can't really see how the interface class scope could have changed. It > was always scope.UNIT. So I'm going to dig through some code to > understand the implications of scope in metadata.yaml... > > > Thanks Ales > Tilman > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: normal charm to subordinate charm and now peer relation does not work
On 24.01.2017 16:56, Alex Kavanagh wrote: > Hi Tilman > > (I'm not an expert here, but was staring at the docs) > > I suspect that your peers relationship should be unit if each peer needs > to have it's own conversation? Otherwise, with a global scope, every > peer will overwrite the other's information? At least I'm wondering if > that what the scopes mean: see > here: https://jujucharms.com/docs/2.0/developer-layers-interfaces > > If that's completely wrong, then a) sorry for the noise, and b) do tell, > as it will help me in my understanding of juju scopes. No I think this must generally be the direction I need to think towards. Any thought impulse in that direction can unlock the knot in my head. :) Thing is, my interface class is scope UNIT already. (GLOBAL and SERVICE definitely would be wrong I think) What hasn't really connected in my brain is how this relates to the scopes in metadata.yaml They have different values. I could only find documentation for global and container. Global apparently is the default. So that is what I set it to explicitly. I'm quite sure that I had tried it with 'container' too in one of the iterations of testing. I can't really see how the interface class scope could have changed. It was always scope.UNIT. So I'm going to dig through some code to understand the implications of scope in metadata.yaml... Thanks Ales Tilman -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: normal charm to subordinate charm and now peer relation does not work
Any comments are appreciated. I'm stuck with this right now and I'm out of ideas. Thanks Tilman On 24.01.2017 14:44, Tilman Baumann wrote: > [...] > Any idea what could be happening here. > I suspect something with the scopes. But they are now all set > explicitly. Any changes in implicit behavior should not affect this. > Am I expecting something from subordinate charms which they can not provide? > > The ssh-peers-relation-* hooks are still called when a unit joins or > leaves. But somehow it's information is not added to the conversation. > Or rather, I think it replaces actually the currently visible single > entry in there. -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
normal charm to subordinate charm and now peer relation does not work
Hi,
it became clear that my charm which was so far standalone (to be
installed together with other charms on the same machine) should better
be subordinate.
A few obvious changes where required regarding the life-cycle of the
charm. No big deal.
https://github.com/tbaumann/iptables-peer-ssh
However, suddenly the peer relation is behaving very oddly.
I'm only seeing one (arbitrary) peer unit in the relation.
This method only does one loop iteration and returns only one value.
def units(self):
hosts = []
for conv in self.conversations():
hosts.append(conv.get_remote('private-address'))
return hosts
The class has scope.UNIT scope.
class PeerDiscovery(RelationBase):
scope = scopes.UNIT
metadata.yaml specifies global scope for the peer relation.
peers:
ssh-peers:
interface: peer-discovery
scope: global
requires:
host-system:
interface: juju-info
scope: container
Any idea what could be happening here.
I suspect something with the scopes. But they are now all set
explicitly. Any changes in implicit behavior should not affect this.
Am I expecting something from subordinate charms which they can not provide?
The ssh-peers-relation-* hooks are still called when a unit joins or
leaves. But somehow it's information is not added to the conversation.
Or rather, I think it replaces actually the currently visible single
entry in there.
Here is the as of yet unpublished interface code.
https://github.com/tbaumann/charm-interface-peer-discovery
--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
Re: peer relation without public interface
On 13.01.2017 17:44, Tilman Baumann wrote: > I thought about it a little and I do think a no-op interface purely for > peer discovery could actually help others with similar requiremrnts too. > Hence this https://github.com/tbaumann/charm-interface-peer-discovery > > If there are no objections, I will try to get this published. I implemented this atexit() cleanup behavior. I think it makes things a lot cleaner for the user of the interface. As long as the implications are well documented this causes fewer surprises than without I find. So, how do I publish interfaces? Is there a review process? Cheers Tilman -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: peer relation without public interface
I thought about it a little and I do think a no-op interface purely for peer discovery could actually help others with similar requiremrnts too. Hence this https://github.com/tbaumann/charm-interface-peer-discovery If there are no objections, I will try to get this published. This is the charm that uses it https://github.com/tbaumann/iptables-peer-ssh Any comments? There is one thing that I don't like a lot, but might be normal. Right now I need to unset the state at the end of my handler function. I find that is bleeding dependencies of the interface into the main code. I really like this idea http://blog.ajkavanagh.me/2016/12/30/better-charm-interfaces/ hookenv.atexit() with a closure to clean up the state at the end of the hook invocation seems like a very sensible thing to do. But I also understand the resistance towards magic. But then, this is something even the basic layer does... I shall look into this next week... Cheers and thanks Tilman PS: github is down ATM. If you wonder why the links don't work. I guess I should have used launchpad. Old habits die hard... On 12.01.2017 21:17, Merlijn Sebrechts wrote: > Hi Tilman > > > I think it's best that you create your own interface layer to do this. > Then include that interface layer in `layer.yaml` and the hooks should > be created at build time. > > More info on developing interface > layers: > https://jujucharms.com/docs/2.0/developer-layers-interfaces#writing-an-interface-layer > > > > Regards > Merlijn > > > > 2017-01-11 14:23 GMT+01:00 Tilman Baumann <tilman.baum...@canonical.com > <mailto:tilman.baum...@canonical.com>>: > > Hi, > > I'm writing a layered reactive-python charm which uses a peer relation > to know all units of the same application. > > However I don't seem to find a way to convince charm build to create the > ./hook/ files for this relation for me. > > If I do my metadata.yaml like this and include the interface in > layers.yaml then I get the hook files and the interface layer code gets > pulled in. > metadata.yaml > peers: > foobar: > interface: ceph > > > If I don't specify a interface or use my own name, then the hooks won't > be created. So my @hook handlers in the reactive code never see it. > > What is the general best practice for providing peer relations of a own > type in a layered charm? > > I suppose there is no reason why I can't put the code in a class based > off RelationBase like interface layers usually do? The code entry point > seems to the @hook decorators around the methods. I don't need to create > instances or anything like that, right? > > Cheers and Thanks > Tilman > > -- > Juju mailing list > Juju@lists.ubuntu.com <mailto:Juju@lists.ubuntu.com> > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > <https://lists.ubuntu.com/mailman/listinfo/juju> > > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
peer relation without public interface
Hi, I'm writing a layered reactive-python charm which uses a peer relation to know all units of the same application. However I don't seem to find a way to convince charm build to create the ./hook/ files for this relation for me. If I do my metadata.yaml like this and include the interface in layers.yaml then I get the hook files and the interface layer code gets pulled in. metadata.yaml peers: foobar: interface: ceph If I don't specify a interface or use my own name, then the hooks won't be created. So my @hook handlers in the reactive code never see it. What is the general best practice for providing peer relations of a own type in a layered charm? I suppose there is no reason why I can't put the code in a class based off RelationBase like interface layers usually do? The code entry point seems to the @hook decorators around the methods. I don't need to create instances or anything like that, right? Cheers and Thanks Tilman -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
