Weekly Development Summary

[Menno Smits]

Hi everyone,

It's time for another update on what the Juju team has been up to. There's
been lots of great progress. Here are the highlights...

*Cross Model Relations*
It is now possible to establish relations between controllers! Here's a
basic example of how this looks on the command line:

$ juju bootstrap aws foo
$ juju deploy mysql
$ juju offer mysql:db

$ juju bootstrap aws bar
$ juju deploy mediawiki
$ juju expose mediawiki
$ juju relate mediawki:db foo:admin/default.mysql

This even works when the controllers are running in different clouds,
opening up all a number of exciting possibilities.

*Upgrades from Juju 1.25 to 2.x*
There's been lots of progress this week with the tooling to allow 1.25
deployments to be upgraded to 2.x. More aspects of Juju's model are now
covered by the export tools and work is now underway to support upgrading
of agent binaries. Work will start soon towards converting LXC containers
to LXD.

*Local Resources in Bundles*
Just as bundles can reference local charms, they can now reference
local resources.
As well as a referencing a revision of a resource in the charm store, you
can now specify a path to a local file when creating a bundle with charm
resources included. Here's a (fake) example of how this looks:

services:
  ubuntu:
charm: "/path/to/ubuntu/charm"
num_units: 1
resources:
  software: "/path/to/bundle/simple-bundle.zip"

Previously, the "software" field above could have only referred to a
resource revision number.

This feature will land in the "develop" branch in the next day or so and
will be released as part of Juju 2.3.

*Operating System Upgrade Support*
Work is underway on a new "update-series" command which allows the operator
to tell Juju that the operating system version for an application or
machine has changed. The idea is that the operator can perform an operating
system upgrade of one or more Juju managed machines and then tell Juju
about the change.

*Juju 2.2.3*
There will be another Juju 2.2 release out soon. It'll have the following
important changes:

   - Fixed a recently introduced upgrade issue
    affecting the Azure and
   vSphere providers.
   - Fixed model watching API to correctly report
    instance status changes.
   - Added  "primary-network"
   configuration attribute for the vSphere provider.
   - Jitter added to metrics collection to spread out load on Juju
   controllers.
   - Fixed to charm resource cleanup.
   - Fixed potential race  when
   completing model migrations.


*Quick Links*
  Work pending: https://github.com/juju/juju/pulls
  Recent commits: https://github.com/juju/juju/commits/develop

Have a great weekend.

Cheers,
Menno
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Is a set state called multiple times?

[fengxia]

Now thinking of it, I just realized that this concept of True state 
executing repeatedly is actually a good thing.



In many cases, there is a need to query external resource for its 
status. This is often implemented as a polling loop. So in charm, I can 
implement it as



@when("prep")

def prep():

# are we there yet?

is_ready = prep_test()

if is_ready:

remove_state("prep")

set_state("do.sth")

@when("do.sth")

def do_sth():

# do this

pass


# move on

remove_state("do.sth")

set_state("next")


On 07/27/2017 03:56 PM, Cory Johns wrote:

fengxia,

It's probably more enlightening to think of them as "flags" rather 
than states.  (Indeed, the next release of charms.reactive will 
deprecate calling them states and will provide set_flag, remove_flag, 
etc. functions instead.)


On Thu, Jul 27, 2017 at 3:29 PM, fengxia > wrote:


Alex,

Thank you for the detailed explanations and examples.

After reading Tilman's and Cory's replies, I think the confusion
is at continuous evaluation (thus execution) of a True state. So a
pair of @when and @when_not will result in one of them being
executed over and over despite adding a remove_state("myself") in
the @when block.

I'm still trying to grasp the idea of this "state" instead of
treating it as an event handler.

So for states, I usually draw a state machine diagram. In this
case, it feels rather unnatural that all True states will
inherently loop to themselves.

But I don't what alternative is in charm's context.


On 07/27/2017 04:13 AM, Alex Kavanagh wrote:

Hi

On Thu, Jul 27, 2017 at 2:37 AM, fengxia > wrote:

Hi Juju,

Once I set a state, set_state("here"), I want to make sure
its @when will only be executed ONCE (when "here" from
False->True).

So my thought is to remove_state("here") in its @when("here")
code block. If I don't, will this @when be called multiple
times if I don't reset this state? What's the good practice here?


You have a couple of options here depending on the nature of the
handler.

 1. If, in the lifetime of the unit's existence, the handler only
has to execute ONCE.  (and I mean EVER), then there is a
@only_once decorator that can be used.  It can be used in
combination with other decorators to set up a condition, but
it guarantees that the handler will only be called once.
However, what you probably want is ...
 2. Use a @when_not('flag') and then set it the 'flag' in the
body of the handler.

The first would look something like:

@when('some-condition-flag')
@only_once
def
do_something_only_once_when_some_condition_flag_is_set_for_the_first_time():
 ... do something once ...

The second treats a flag as a 'have I done this yet' condition,
and allows you to reset the flag at some other point in the
charm's life cycle so that you can do it again.  'installed' is a
good example of this:

@when_not('installed-something')
def do_install_of_something():
... do the installation ...
# when it is fully successful, set the installed-something
flag.  Don't set it early as
# if it errors, a future handler invocation may be able to
continue the installation.
set_state('installed-something')


@when(some other conditions indicating do an upgrade)
def do_upgrade():
 ... set upgrade sources, or other pre upgrade actions
 remove_state('installed-something')

In this situation, hopefully you can see that we can re-use
'do_install_of_something()' when we do upgrades.

I think it's useful to think about states (flags) as being a
'memory' that something has happened, and use them to either gate
on not doing things again, or to trigger the next action is a
graph of actions that need to take place to get the charm's
payload to the desired operational state.  I tend to name them,
and use them, to indicate when something has happened, rather
than when it hasn't, and so tend to use @when_not('some-flag') on
the handler that eventually sets that flag.

Hope that this helps.
Alex.



-- 
Feng xia

Engineer
Lenovo USA

Phone: 5088011794 
fx...@lenovo.com 

Lenovo.com
Twitter | Facebook | Instagram | Blogs | Forums


-- 
Juju mailing list

Juju@lists.ubuntu.com 
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju





-- 
Alex Kavanagh - Software Engineer

Cloud Dev Ops - Solutions & Product Engineering - Canonical Ltd


   

Re: Is a set state called multiple times?

[Cory Johns]

fengxia,

It's probably more enlightening to think of them as "flags" rather than
states.  (Indeed, the next release of charms.reactive will deprecate
calling them states and will provide set_flag, remove_flag, etc. functions
instead.)

On Thu, Jul 27, 2017 at 3:29 PM, fengxia  wrote:

> Alex,
>
> Thank you for the detailed explanations and examples.
>
> After reading Tilman's and Cory's replies, I think the confusion is at
> continuous evaluation (thus execution) of a True state. So a pair of @when
> and @when_not will result in one of them being executed over and over
> despite adding a remove_state("myself") in the @when block.
>
> I'm still trying to grasp the idea of this "state" instead of treating it
> as an event handler.
>
> So for states, I usually draw a state machine diagram. In this case, it
> feels rather unnatural that all True states will inherently loop to
> themselves.
>
> But I don't what alternative is in charm's context.
>
> On 07/27/2017 04:13 AM, Alex Kavanagh wrote:
>
> Hi
>
> On Thu, Jul 27, 2017 at 2:37 AM, fengxia  wrote:
>
>> Hi Juju,
>>
>> Once I set a state, set_state("here"), I want to make sure its @when will
>> only be executed ONCE (when "here" from False->True).
>>
>> So my thought is to remove_state("here") in its @when("here") code block.
>> If I don't, will this @when be called multiple times if I don't reset this
>> state? What's the good practice here?
>
>
> You have a couple of options here depending on the nature of the handler.
>
>
>1. If, in the lifetime of the unit's existence, the handler only has
>to execute ONCE.  (and I mean EVER), then there is a @only_once decorator
>that can be used.  It can be used in combination with other decorators to
>set up a condition, but it guarantees that the handler will only be called
>once.  However, what you probably want is ...
>2. Use a @when_not('flag') and then set it the 'flag' in the body of
>the handler.
>
> The first would look something like:
>
> @when('some-condition-flag')
> @only_once
> def do_something_only_once_when_some_condition_flag_is_set_
> for_the_first_time():
>  ... do something once ...
>
> The second treats a flag as a 'have I done this yet' condition, and allows
> you to reset the flag at some other point in the charm's life cycle so that
> you can do it again.  'installed' is a good example of this:
>
> @when_not('installed-something')
> def do_install_of_something():
> ... do the installation ...
> # when it is fully successful, set the installed-something flag.
> Don't set it early as
> # if it errors, a future handler invocation may be able to continue
> the installation.
> set_state('installed-something')
>
>
> @when(some other conditions indicating do an upgrade)
> def do_upgrade():
>  ... set upgrade sources, or other pre upgrade actions
>  remove_state('installed-something')
>
> In this situation, hopefully you can see that we can re-use
> 'do_install_of_something()' when we do upgrades.
>
> I think it's useful to think about states (flags) as being a 'memory' that
> something has happened, and use them to either gate on not doing things
> again, or to trigger the next action is a graph of actions that need to
> take place to get the charm's payload to the desired operational state.  I
> tend to name them, and use them, to indicate when something has happened,
> rather than when it hasn't, and so tend to use @when_not('some-flag') on
> the handler that eventually sets that flag.
>
> Hope that this helps.
> Alex.
>
>>
>>
>> --
>> Feng xia
>> Engineer
>> Lenovo USA
>>
>> Phone: 5088011794
>> fx...@lenovo.com
>>
>> Lenovo.com
>> Twitter | Facebook | Instagram | Blogs | Forums
>>
>>
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm
>> an/listinfo/juju
>>
>
>
>
> --
> Alex Kavanagh - Software Engineer
> Cloud Dev Ops - Solutions & Product Engineering - Canonical Ltd
>
>
> --
> Feng xia
> Engineer
> Lenovo USA
>
> Phone: 5088011794 <(508)%20801-1794>fx...@lenovo.com
>   
> Lenovo.com
> Twitter | Facebook | Instagram | Blogs | Forums
>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Is a set state called multiple times?

[Cory Johns]

On Thu, Jul 27, 2017 at 12:29 PM, Tilman Baumann <
tilman.baum...@canonical.com> wrote:

> Problems with interfaces:
> - *.connected state gets dropped when one  unit leaves. Not when the
> last one leaves.
>

If this is happening, then it's because the particular interface layer is
doing something wrong in how it's setting or removing the state with the
Conversation API.  However, this is made easy to get wrong by the confusing
nature of the Conversations API, which we are working on addressing.


> - self.* only works when code runs in the right hook context
>

The purpose of the Conversations API (self.conversations()) was to address
this so that the handlers wouldn't need to keep track of the specific
relation IDs and unit names that the hook context supplies.  So, if the
interface layer is written correctly within that metaphor, the actual hook
context is irrelevant.  However, we're working on an improved API that
solves the same issue while being more explicit about what it's doing to
reduce the confusion.


> - interface class reference is only passed in decorated functions if the
> hook inside the interface code ran
>

The class reference is passed to any handler decorated with a @when
decorator with a state that was set by an interface layer class using
Conversation.set_state() (or the self.set_state() helper on the interface
class, which is essentially self.conversations()[0].set_state()).  This
will happen regardless of what hook context the charm is currently
executing in.

Part of the API improvements will remove the instances from handler params
and make them available via a global context which will always provide a
consistent way to access them.


> - Only the first occurrence of a unit joining a relation can be caught
> with handler
>

All joining units can be handled by interface layers.  The joined hook will
run for all joining units, and those units will be added to the interface
instance's conversations if the joined state is applied within the the
@hook decorated handler.  Those units can then all be accessed by iterating
over self.conversations() in a interface instance method called from a
@when decorated handler (but not from the @hook decorated handlers in the
interface class; those will only see the one unit :/).

Part of the API improvements for this will be to have some of the flags
(states), such as joined, managed automatically by the framework (which
should remove the need for @hook handlers entirely and also reduce
boilerplate), and then to provide explicit methods for associating specific
units with flags as well as seeing what units are associated with a given
flag.

If you are interested in learning more about how we're looking to improve
charms.reactive and contributing to the discussion, I invite you to take a
look at https://github.com/juju-solutions/charms.reactive/pull/101  That PR
contains documentation for the proposed API changes, and a lot of
discussion hashing out how best those API should look to the user.  In
fact, it might even be a bit too much discussion for a GitHub PR comment
stream to organize well.  :)
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Is a set state called multiple times?

[fengxia]

Alex,

Thank you for the detailed explanations and examples.

After reading Tilman's and Cory's replies, I think the confusion is at 
continuous evaluation (thus execution) of a True state. So a pair of 
@when and @when_not will result in one of them being executed over and 
over despite adding a remove_state("myself") in the @when block.


I'm still trying to grasp the idea of this "state" instead of treating 
it as an event handler.


So for states, I usually draw a state machine diagram. In this case, it 
feels rather unnatural that all True states will inherently loop to 
themselves.


But I don't what alternative is in charm's context.


On 07/27/2017 04:13 AM, Alex Kavanagh wrote:

Hi

On Thu, Jul 27, 2017 at 2:37 AM, fengxia > wrote:


Hi Juju,

Once I set a state, set_state("here"), I want to make sure its
@when will only be executed ONCE (when "here" from False->True).

So my thought is to remove_state("here") in its @when("here") code
block. If I don't, will this @when be called multiple times if I
don't reset this state? What's the good practice here?


You have a couple of options here depending on the nature of the handler.

 1. If, in the lifetime of the unit's existence, the handler only has
to execute ONCE.  (and I mean EVER), then there is a @only_once
decorator that can be used.  It can be used in combination with
other decorators to set up a condition, but it guarantees that the
handler will only be called once.  However, what you probably want
is ...
 2. Use a @when_not('flag') and then set it the 'flag' in the body of
the handler.

The first would look something like:

@when('some-condition-flag')
@only_once
def 
do_something_only_once_when_some_condition_flag_is_set_for_the_first_time():

 ... do something once ...

The second treats a flag as a 'have I done this yet' condition, and 
allows you to reset the flag at some other point in the charm's life 
cycle so that you can do it again.  'installed' is a good example of this:


@when_not('installed-something')
def do_install_of_something():
... do the installation ...
# when it is fully successful, set the installed-something flag.  
Don't set it early as
# if it errors, a future handler invocation may be able to 
continue the installation.

set_state('installed-something')


@when(some other conditions indicating do an upgrade)
def do_upgrade():
 ... set upgrade sources, or other pre upgrade actions
 remove_state('installed-something')

In this situation, hopefully you can see that we can re-use 
'do_install_of_something()' when we do upgrades.


I think it's useful to think about states (flags) as being a 'memory' 
that something has happened, and use them to either gate on not doing 
things again, or to trigger the next action is a graph of actions that 
need to take place to get the charm's payload to the desired 
operational state.  I tend to name them, and use them, to indicate 
when something has happened, rather than when it hasn't, and so tend 
to use @when_not('some-flag') on the handler that eventually sets that 
flag.


Hope that this helps.
Alex.



-- 
Feng xia

Engineer
Lenovo USA

Phone: 5088011794 
fx...@lenovo.com 

Lenovo.com
Twitter | Facebook | Instagram | Blogs | Forums


-- 
Juju mailing list

Juju@lists.ubuntu.com 
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju





--
Alex Kavanagh - Software Engineer
Cloud Dev Ops - Solutions & Product Engineering - Canonical Ltd


--
Feng xia
Engineer
Lenovo USA

Phone: 5088011794
fx...@lenovo.com

Lenovo.com
Twitter | Facebook | Instagram | Blogs | Forums

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Is a set state called multiple times?

[fengxia]

Tilman,

This is quite interesting, and I think that's exactly what confused me 
-- I took @when as event handler, and expected to be executed only on 
False->True transition, but not being continuously executed if I don't 
"remove_state" inside itself.



On 07/27/2017 12:29 PM, Tilman Baumann wrote:

The confusion comes from the expectations at states are somehow events
and @when decorators are event handlers.
@when are predicates on states. When predicates are true the decorated
code is executed at every hook invocation.


If you want to trigger on edges (state changes) you need to build flipflops.


@when("state.wanted")
@when_not("state.reached")

do stuff here when entering state
set_state("state.reached")


@when_not("state.wanted")
@when("state.reached")

do stuff here when leaving state
remove_state("state.reached")


Reactive is still quite nice after you wrapped your head around that.


// Rant follows

Shame though that ALL interfaces are broken and thus unusable.

Interfaces need to be fixed on a fundamental level IMHO.
I have a workaround proposal
https://github.com/tbaumann/charm-interface-peer-discovery/blob/master/peers.py
But in truth, that is not a great solution, it's just a workaround.

I know I'm derailing the conversation here completely, but the
brokenness of interfaces has been a personal annoyance for a while and I
wish I could do something to change that.

One of the issues is that one can not react on "relation.connected"
states because the context they are supposed to have (argument to
decorated function) only works in the right hook context.
I propose a @interface decorator which will create the right context and
instantiate a interface object.
That way I can use interfaces in all sorts of functions and not only in
hook context and the kludges around to avoid being called outside of
hook context.


Problems with interfaces:
- *.connected state gets dropped when one  unit leaves. Not when the
last one leaves.
- self.* only works when code runs in the right hook context
- interface class reference is only passed in decorated functions if the
hook inside the interface code ran
- Only the first occurrence of a unit joining a relation can be caught
with handler
- Interfaces have 90% identical code which is just boilerplate



Just my 0.02€


On 27.07.2017 03:37, fengxia wrote:

Hi Juju,

Once I set a state, set_state("here"), I want to make sure its @when
will only be executed ONCE (when "here" from False->True).

So my thought is to remove_state("here") in its @when("here") code
block. If I don't, will this @when be called multiple times if I don't
reset this state? What's the good practice here?



--
Feng xia
Engineer
Lenovo USA

Phone: 5088011794
fx...@lenovo.com

Lenovo.com
Twitter | Facebook | Instagram | Blogs | Forums


--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: juju bug (?) when allocating floating ips to machines

[Heather Lanigan]

Hi Patrizio,

Judging by the code in develop, we do not check the tenant_id when choosing
a FIP.  There is an attempt to ensure the FIP is in the provided external
network, if specified.  So that may be another work around.

Please file a bug.  I'm wondering if there are more places the provider
should be checking the tenant as well.

-Heather



On Thu, Jul 27, 2017 at 9:04 AM, Patrizio Bassi 
wrote:

> Hi all
>
> i'm using juju 2.1.2.1 (i didn't upgrade to 2.2 yet, that's why i didn't
> open a bug on launchpad) with openstack as cloud provider.
>
> When i use as credentials an Admin user (but a specific tenant) i have
> issues with floating ip assignment: the admin user can see all the floating
> ips in the openstack region.
> So, if another tenant allocates an IP without assigning to a VM (so,
> unused) juju tries to use it and attach to the VM it just deployed.
>
> i.e.
>
> user test1 is Admin and has primary project "tenant-one"
> user test2 is member of project "tenant-two"
>
> credentials given to juju are test1, test1_password, tenant-one and
> RegionOne.
>
> # source novarc_test1
>
> # neutron floatingip-list
> +--+--+-
> +--+
> | id   | fixed_ip_address |
> floating_ip_address | port_id  |
> +--+--+-
> +--+
> | 03d1a8e8-fd55-4d6e-ab7e-b62061ea6206 | 192.168.0.10 | 10.1.2.19
> | b6ac7caf-0c6e-4d81-b055-ecb8b4bdeebd |
> | 2b4e48ba-aad6-4d78-aff6-88b912f89bf5 | 192.168.0.20 | 10.1.2.9
>  | 17f69b3b-97d0-4cec-8208-e4d2ac2f1034 |
> | 3144b683-2cf5-43cf-bddd-b06cb5662430 |  | 10.1.2.22
> |  |
> | 55145d85-58ea-4f15-8a0c-96a719c0fa8d | 192.168.0.22 | 10.1.2.4
>  | 6eeaa12b-0971-496c-bd38-89e9b9d71818 |
> +--+--+-
> +--+
>
> the third line shows and ip address assigned to tenant-two by test2.
>
> User test1 has admin role so he has permission to see the ip.
> Using a command like "neutron floatingip-show
> 3144b683-2cf5-43cf-bddd-b06cb5662430" correctly shows the project_id uuid
> related to tenant-two and not tenant-one.
>
> juju model is configured with
> use-default-secgroup  model   true
> use-floating-ip   model   true
>
> When trying to deploy any application juju spawns a VM, but it never ends
> and logs:
>
> Unable to associate floating IP 10.1.2.22 to fixed IP 192.168.0.9 for
> instance 3d95283c-69f2-4cf1-8980-99462a5904a2.
>
> Removing the unused floating ip address or using a member-only (not admin
> user) bypass the problem: juju will allocate a new ip and associate with
> the new VM.
>
> I didn't try but i do think that if an user is member of two different
> tenants it may try to mis-use the addresses and mess with them, failing to
> deploy.
>
> Desiderata: juju should check if the allocated ip address is in the same
> tenant view of the given credentials.
>
> Regards
>
> Patrizio
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Application of Membership for Charmers

[Billy Olsen]

Thanks Tim, Alex, David, Jorge, James and the rest of the charmers!


On Thu, Jul 27, 2017 at 10:07 AM Tim Van Steenburgh <
tim.van.steenbu...@canonical.com> wrote:

> +1 and approved, thanks Billy.
>
> On Fri, Jul 21, 2017 at 8:41 AM, Billy Olsen 
> wrote:
>
>> Hello Charmers,
>>
>> My name is Billy Olsen and I've been a long time contributor to the
>> OpenStack Charms, going back 3 years now. I'm currently a core member of
>> the OpenStack charming community and have additionally made contributions
>> of code and review effort to the charm-helpers library.
>>
>> I believe that I can provide a positive contribution in the overall
>> charming ecosystem.
>>
>> Thanks,
>>
>> Billy
>>
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Application of Membership for Charmers

[Tim Van Steenburgh]

+1 and approved, thanks Billy.

On Fri, Jul 21, 2017 at 8:41 AM, Billy Olsen 
wrote:

> Hello Charmers,
>
> My name is Billy Olsen and I've been a long time contributor to the
> OpenStack Charms, going back 3 years now. I'm currently a core member of
> the OpenStack charming community and have additionally made contributions
> of code and review effort to the charm-helpers library.
>
> I believe that I can provide a positive contribution in the overall
> charming ecosystem.
>
> Thanks,
>
> Billy
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Is a set state called multiple times?

[Tilman Baumann]

The confusion comes from the expectations that states are somehow events
and @when decorators are event handlers.
@when are predicates on states. When predicates are true the decorated
code is executed at every hook invocation.


If you want to trigger on edges (state changes) you need to build flipflops.


@when("state.wanted")
@when_not("state.reached")

do stuff here when entering state
set_state("state.reached")


@when_not("state.wanted")
@when("state.reached")

do stuff here when leaving state
remove_state("state.reached")


Reactive is still quite nice after you wrapped your head around that.


// Rant follows

Shame though that ALL interfaces are broken and thus unusable.

Interfaces need to be fixed on a fundamental level IMHO.
I have a workaround proposal
https://github.com/tbaumann/charm-interface-peer-discovery/blob/master/peers.py
But in truth, that is not a great solution, it's just a workaround.

I know I'm derailing the conversation here completely, but the
brokenness of interfaces has been a personal annoyance for a while and I
wish I could do something to change that.

One of the issues is that one can not react on "relation.connected"
states because the context they are supposed to have (argument to
decorated function) only works in the right hook context.
I propose a @interface decorator which will create the right context and
instantiate a interface object.
That way I can use interfaces in all sorts of functions and not only in
hook context and the kludges around to avoid being called outside of
hook context.


Problems with interfaces:
- *.connected state gets dropped when one  unit leaves. Not when the
last one leaves.
- self.* only works when code runs in the right hook context
- interface class reference is only passed in decorated functions if the
hook inside the interface code ran
- Only the first occurrence of a unit joining a relation can be caught
with handler
- Interfaces have 90% identical code which is just boilerplate



Just my 0.02€


On 27.07.2017 03:37, fengxia wrote:
> Hi Juju,
> 
> Once I set a state, set_state("here"), I want to make sure its @when
> will only be executed ONCE (when "here" from False->True).
> 
> So my thought is to remove_state("here") in its @when("here") code
> block. If I don't, will this @when be called multiple times if I don't
> reset this state? What's the good practice here?
> 

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: CephFS Backend for Hadoop

[James Beedy]

Dmitrii,

Many thanks for your insight here.

~James

On Wed, Jul 26, 2017 at 11:34 PM, Patrizio Bassi 
wrote:

>
> 2017-07-26 9:17 GMT+02:00 Mark Shuttleworth :
>
>> On 26/07/17 07:14, Patrizio Bassi wrote:
>>
>> Deploying hadoop via juju in an openstack tenant requires a separate
>> model (as far as i could design it).
>> So we may use the new juju 2.2 cross model relation to relate the hadoop
>> charms to the openstack ceph units.
>>
>> does it sound feasible?
>>
>>
>> Yes, that sounds feasible. I'm not sure how Ceph identity / permissions
>> will work in that case (i.e. who has access to which data, how Ceph will
>> correlate tenants in OpenStack both through Cinder and through a direct
>> relationship). In principle though, as long as the networking is arranged
>> so that IP addresses and routes enable traffic to flow between your tenant
>> network and your Ceph network, and as long as both sets of machines can see
>> the Juju controller, they can exchange messages and traffic.
>>
>> Mark
>>
>
>
> Dear Mark,
>
> On relation join event we may create a new ceph storage pool dedicated to
> the incoming unit (i.e. prefixed with the controller/model/unit/charm name
> by default). Can cephx proto
>
> Regarding networking openstack neutron by default block traffic from
> tenant VM to the admin network which it required to access the same ceph
> mon/osd. It requires changing neutron or implement an external nat for
> instance (our solution at the moment)
>
> Patrizio
>
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Is a set state called multiple times?

[Cory Johns]

It does seem that the current behavior of re-evaluating handlers despite
none of their flags (states) having changed is confusing, particularly for
new users, and we are planning on changing this behavior.  Unfortunately,
there are some charms that have come to rely on the current behavior, so we
will need to provide some mechanism for getting that behavior, at least for
a transition period.  We will also need to provide a mechanism to "force" a
flag to be considered "re-activated" even if that's as basic as requiring
it be removed and then re-added.

On Thu, Jul 27, 2017 at 4:13 AM, Alex Kavanagh 
wrote:

> Hi
>
> On Thu, Jul 27, 2017 at 2:37 AM, fengxia  wrote:
>
>> Hi Juju,
>>
>> Once I set a state, set_state("here"), I want to make sure its @when will
>> only be executed ONCE (when "here" from False->True).
>>
>> So my thought is to remove_state("here") in its @when("here") code block.
>> If I don't, will this @when be called multiple times if I don't reset this
>> state? What's the good practice here?
>
>
> You have a couple of options here depending on the nature of the handler.
>
>
>1. If, in the lifetime of the unit's existence, the handler only has
>to execute ONCE.  (and I mean EVER), then there is a @only_once decorator
>that can be used.  It can be used in combination with other decorators to
>set up a condition, but it guarantees that the handler will only be called
>once.  However, what you probably want is ...
>2. Use a @when_not('flag') and then set it the 'flag' in the body of
>the handler.
>
> The first would look something like:
>
> @when('some-condition-flag')
> @only_once
> def do_something_only_once_when_some_condition_flag_is_set_
> for_the_first_time():
>  ... do something once ...
>
> The second treats a flag as a 'have I done this yet' condition, and allows
> you to reset the flag at some other point in the charm's life cycle so that
> you can do it again.  'installed' is a good example of this:
>
> @when_not('installed-something')
> def do_install_of_something():
> ... do the installation ...
> # when it is fully successful, set the installed-something flag.
> Don't set it early as
> # if it errors, a future handler invocation may be able to continue
> the installation.
> set_state('installed-something')
>
>
> @when(some other conditions indicating do an upgrade)
> def do_upgrade():
>  ... set upgrade sources, or other pre upgrade actions
>  remove_state('installed-something')
>
> In this situation, hopefully you can see that we can re-use
> 'do_install_of_something()' when we do upgrades.
>
> I think it's useful to think about states (flags) as being a 'memory' that
> something has happened, and use them to either gate on not doing things
> again, or to trigger the next action is a graph of actions that need to
> take place to get the charm's payload to the desired operational state.  I
> tend to name them, and use them, to indicate when something has happened,
> rather than when it hasn't, and so tend to use @when_not('some-flag') on
> the handler that eventually sets that flag.
>
> Hope that this helps.
> Alex.
>
>>
>>
>> --
>> Feng xia
>> Engineer
>> Lenovo USA
>>
>> Phone: 5088011794
>> fx...@lenovo.com
>>
>> Lenovo.com
>> Twitter | Facebook | Instagram | Blogs | Forums
>>
>>
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm
>> an/listinfo/juju
>>
>
>
>
> --
> Alex Kavanagh - Software Engineer
> Cloud Dev Ops - Solutions & Product Engineering - Canonical Ltd
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

juju bug (?) when allocating floating ips to machines

[Patrizio Bassi]

Hi all

i'm using juju 2.1.2.1 (i didn't upgrade to 2.2 yet, that's why i didn't
open a bug on launchpad) with openstack as cloud provider.

When i use as credentials an Admin user (but a specific tenant) i have
issues with floating ip assignment: the admin user can see all the floating
ips in the openstack region.
So, if another tenant allocates an IP without assigning to a VM (so,
unused) juju tries to use it and attach to the VM it just deployed.

i.e.

user test1 is Admin and has primary project "tenant-one"
user test2 is member of project "tenant-two"

credentials given to juju are test1, test1_password, tenant-one and
RegionOne.

# source novarc_test1

# neutron floatingip-list
+--+--+-
+--+
| id   | fixed_ip_address |
floating_ip_address | port_id  |
+--+--+-
+--+
| 03d1a8e8-fd55-4d6e-ab7e-b62061ea6206 | 192.168.0.10 | 10.1.2.19
  | b6ac7caf-0c6e-4d81-b055-ecb8b4bdeebd |
| 2b4e48ba-aad6-4d78-aff6-88b912f89bf5 | 192.168.0.20 | 10.1.2.9
   | 17f69b3b-97d0-4cec-8208-e4d2ac2f1034 |
| 3144b683-2cf5-43cf-bddd-b06cb5662430 |  | 10.1.2.22
  |  |
| 55145d85-58ea-4f15-8a0c-96a719c0fa8d | 192.168.0.22 | 10.1.2.4
   | 6eeaa12b-0971-496c-bd38-89e9b9d71818 |
+--+--+-
+--+

the third line shows and ip address assigned to tenant-two by test2.

User test1 has admin role so he has permission to see the ip.
Using a command like "neutron floatingip-show
3144b683-2cf5-43cf-bddd-b06cb5662430"
correctly shows the project_id uuid related to tenant-two and not
tenant-one.

juju model is configured with
use-default-secgroup  model   true
use-floating-ip   model   true

When trying to deploy any application juju spawns a VM, but it never ends
and logs:

Unable to associate floating IP 10.1.2.22 to fixed IP 192.168.0.9 for
instance 3d95283c-69f2-4cf1-8980-99462a5904a2.

Removing the unused floating ip address or using a member-only (not admin
user) bypass the problem: juju will allocate a new ip and associate with
the new VM.

I didn't try but i do think that if an user is member of two different
tenants it may try to mis-use the addresses and mess with them, failing to
deploy.

Desiderata: juju should check if the allocated ip address is in the same
tenant view of the given credentials.

Regards

Patrizio
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Is a set state called multiple times?

[Alex Kavanagh]

Hi

On Thu, Jul 27, 2017 at 2:37 AM, fengxia  wrote:

> Hi Juju,
>
> Once I set a state, set_state("here"), I want to make sure its @when will
> only be executed ONCE (when "here" from False->True).
>
> So my thought is to remove_state("here") in its @when("here") code block.
> If I don't, will this @when be called multiple times if I don't reset this
> state? What's the good practice here?


You have a couple of options here depending on the nature of the handler.


   1. If, in the lifetime of the unit's existence, the handler only has to
   execute ONCE.  (and I mean EVER), then there is a @only_once decorator that
   can be used.  It can be used in combination with other decorators to set up
   a condition, but it guarantees that the handler will only be called once.
   However, what you probably want is ...
   2. Use a @when_not('flag') and then set it the 'flag' in the body of the
   handler.

The first would look something like:

@when('some-condition-flag')
@only_once
def
do_something_only_once_when_some_condition_flag_is_set_for_the_first_time():
 ... do something once ...

The second treats a flag as a 'have I done this yet' condition, and allows
you to reset the flag at some other point in the charm's life cycle so that
you can do it again.  'installed' is a good example of this:

@when_not('installed-something')
def do_install_of_something():
... do the installation ...
# when it is fully successful, set the installed-something flag.  Don't
set it early as
# if it errors, a future handler invocation may be able to continue the
installation.
set_state('installed-something')


@when(some other conditions indicating do an upgrade)
def do_upgrade():
 ... set upgrade sources, or other pre upgrade actions
 remove_state('installed-something')

In this situation, hopefully you can see that we can re-use
'do_install_of_something()' when we do upgrades.

I think it's useful to think about states (flags) as being a 'memory' that
something has happened, and use them to either gate on not doing things
again, or to trigger the next action is a graph of actions that need to
take place to get the charm's payload to the desired operational state.  I
tend to name them, and use them, to indicate when something has happened,
rather than when it hasn't, and so tend to use @when_not('some-flag') on
the handler that eventually sets that flag.

Hope that this helps.
Alex.

>
>
> --
> Feng xia
> Engineer
> Lenovo USA
>
> Phone: 5088011794
> fx...@lenovo.com
>
> Lenovo.com
> Twitter | Facebook | Instagram | Blogs | Forums
>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm
> an/listinfo/juju
>



-- 
Alex Kavanagh - Software Engineer
Cloud Dev Ops - Solutions & Product Engineering - Canonical Ltd
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: CephFS Backend for Hadoop

[Patrizio Bassi]

2017-07-26 9:17 GMT+02:00 Mark Shuttleworth :

> On 26/07/17 07:14, Patrizio Bassi wrote:
>
> Deploying hadoop via juju in an openstack tenant requires a separate model
> (as far as i could design it).
> So we may use the new juju 2.2 cross model relation to relate the hadoop
> charms to the openstack ceph units.
>
> does it sound feasible?
>
>
> Yes, that sounds feasible. I'm not sure how Ceph identity / permissions
> will work in that case (i.e. who has access to which data, how Ceph will
> correlate tenants in OpenStack both through Cinder and through a direct
> relationship). In principle though, as long as the networking is arranged
> so that IP addresses and routes enable traffic to flow between your tenant
> network and your Ceph network, and as long as both sets of machines can see
> the Juju controller, they can exchange messages and traffic.
>
> Mark
>


Dear Mark,

On relation join event we may create a new ceph storage pool dedicated to
the incoming unit (i.e. prefixed with the controller/model/unit/charm name
by default). Can cephx proto

Regarding networking openstack neutron by default block traffic from tenant
VM to the admin network which it required to access the same ceph mon/osd.
It requires changing neutron or implement an external nat for instance (our
solution at the moment)

Patrizio
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju