Re: Juju GUI handling of empty config breaks promulgated charms

[John Meinel]

>
> ...
>
> The ideal solution is of course to fix it in Juju, and have it stop
> throwing away perfectly valid configuration data :)
>
>
> AFAICT we fixed this in Juju 1.15, but there are 2 places you can pass
config. The old field uses the compatibility of "" means nil, the new field
handles nil vs "" correctly. The Juju CLI should work correctly, it sounds
like the GUI was missed to use the new field.

John
=:->
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: PostgreSQL Use Case/Issues

[Mark Shuttleworth]

On 22/02/17 19:46, James Beedy wrote:
> A client can 'accept the defaults' by not setting any properties on
>
> the db relation when it joins (dating back to the original protocol
> with pyjuju). When the PostgreSQL charm runs its relation-joined and
> relation-changed hooks, it has no way of telling if the client just
> wants to 'accept the defaults', or if the client has not yet run its
> relation-joined or relation-changed hooks yet. So if it sees an empty
> relation, it assumes 'accept the defaults' and provides a database
> named after the client service.
>

IIRC we agreed that the full state of a unit would be exposed to it from
the beginning, if we know that.

We have had ample time to introduce changes in behaviour since pyjuju,
so I suspect this is just something that slipped through the cracks, not
something we especially want to preserve. Could you file a bug with the
proposed change in behaviour that would enable charmers to be more
definitive in their approach?

Mark
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju GUI handling of empty config breaks promulgated charms

[Stuart Bishop]

On 22 February 2017 at 22:45, Merlijn Sebrechts
 wrote:
> This - is - awesome !
>
> I guess this hack will still require a rebuild for the affected Charms?

Other affected charms could use the same hack (patching
charmhelpers.core.hookenv.config to convert nulls to ''), and would
need to be rebuilt and published.

I considered submitting it to charm-helpers, but its not completely
backwards compatible and could cause problems for charms that really
do expect null config settings.

The ideal solution is of course to fix it in Juju, and have it stop
throwing away perfectly valid configuration data :)


-- 
Stuart Bishop 

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Juju 2.1.0, and Conjure-up, are here!

[Curtis Hovey-Canonical]

A new release of Juju, 2.1.0, and Conjure-up, are here!


## What's new in 2.1.0

- Model migration
- Interactive `add-cloud`
- Networking changes
- Conjure-up
- LXD credential changes
- Changes to the GUI
- Instrumentation of Juju via Prometheus endpoints
- Improved OpenStack keystone v3 authentication
- New cloud-regions supported
- Additional improvements


### Model migration

Model migration allows you to easily move a live model from one
controller to another. The same configuration of machines, units and
their relationships will be replicated on a secondary controller, while
your applications continue uninterrupted.

Migration is a useful alternative to upgrading a controller in place,
and for moving models off a busy controller. When upgrading a
controller, you can bootstrap a new controller running a newer version
of Juju and then migrate each model across one at a time. This is safer
than upgrading a controller while it is running many applications.

Currently there are some restrictions:

  - The source and destination controllers need to be in the same cloud
environment.
  - The destination controller needs to be running on the same cloud
substrate as the source controller.
  - Destination controllers on different regions or VPCs need direct
connectivity to the source controller.
  - The version of Juju running on the destination controller needs to
be the same or newer than the version on the source controller.
  - The controller model cannot be migrated.

To migrate a model on the current controller to a model on another
controller, you simply name the model as the first argument followed by
the target controller (a model with the same name cannot already exist
on the target controller):

juju migrate  

This will initiate the migration with output similar to the following:

Migration started with ID "d1924666-1b00-4805-89b5-5ed5a6744426:0"

You can monitor the migration progress from the output of the juju
status command run against the source model. The juju show-model command
also shows migration progress.

If the migration fails at any point, the model will be reactivated on
the original controller in the same state it was in before the migration
process was started. The duration of a migration will depend on the
complexity of the model, the resources it uses and the capabilities of
the hosted environment. Most migrations will take minutes, and even
large deployments are unlikely to take hours.

When complete, the model will no longer exist on the source controller,
and the model, all its applications, machines and units will be running
from the target controller.

Use `juju switch` to select the migrated model in the destination
controller:

juju switch :
juju status

There is more information on model migration in the Juju documentation
online at



### Interactive `add-cloud`

With previous versions of Juju, the `add-cloud` command would need to be
fed a specifically formatted YAML file if your cloud of choice wasn't
directly supported by Juju. You can still do this, but from version 2.1,
you can also step through a simple interactive process that will create
a working configuration for you.

Typing `juju add-cloud` starts the process and produces the following
output:

Cloud Types
  maas
  manual
  openstack
  vsphere

Select cloud type:

Simply answer the three or four questions for your new cloud and Juju
will do the rest. The next step is to add credentials for this new
cloud, which can be done with the similarly interactive command:

juju add-credentials

Again, follow the prompts to add the requested information.

A more detailed walkthrough of the process is published in the online
Juju documentation here:



### Networking changes

A number of changes have been introduced to make the use of networks,
particularly networking of containers, more efficient and consistent in
Juju.

Juju models networks using the primitive of "spaces". A space is made up
of one or more routable subnets with common ingress and egress rules.
The operator can model this topology in such a way that applications
have the required network connectivity without generating network IP
maps of overwhelming complexity that are not portable.

The default behaviour in Juju 2.0 was that all machines might host
containers and so all interfaces were bridged by default, even if a
container was never placed on the machine. If a container was placed on
the machine all of the network devices of that machine were made
available to each container. This led to issues where the operator
wanted a much cleaner model where the containers only had access to
networks that the model required. Starting from Juju 2.1 this will no
longer be true. Juju will only create the bridges which are necessary
for a container to operate in the model.

The changes in 2.1

Re: PostgreSQL Use Case/Issues

[James Beedy]

On Wed, Feb 22, 2017 at 8:55 AM, Stuart Bishop 
wrote:

> On 22 February 2017 at 21:46, James Beedy  wrote:
>
> > Experiencing some varying results with the PostgreSQL charm, hoping to
> get
> > some validation on my use case.
>
> This all seems the pgsql interface. Expected use is at
> http://interface-pgsql.readthedocs.io/en/stable/
> requires.html#example-usage
> if you haven't seen it already. The protocol had a fairly large
> (backwards compatible) change recently, so sorry about the teething
> troubles.
>
> > 1. Sometimes (feeling like 1/5ish deploys) a database is created with the
> > name of the application, instead of the database name the application
> > requested.
>
> Unfortunately this seems possible.
>
> A client can 'accept the defaults' by not setting any properties on
> the db relation when it joins (dating back to the original protocol
> with pyjuju). When the PostgreSQL charm runs its relation-joined and
> relation-changed hooks, it has no way of telling if the client just
> wants to 'accept the defaults', or if the client has not yet run its
> relation-joined or relation-changed hooks yet. So if it sees an empty
> relation, it assumes 'accept the defaults' and provides a database
> named after the client service. If your client then runs its
> relation-joined hook, the 'db.connected' state will be set (because
> there is a relation), and the 'db.*.available' states will also be set
> because there is a database available (not the one you want, but you
> haven't had a change to say otherwise yet). At which point your
> handlers kick in on *.available and get valid connection strings to a
> different database to the one you want. And maybe the handler that
> calls set_database() runs too, but it is too late.
>
> I think I can fix this, but I'll need to make a corresponding
> adjustment in the PostgreSQL charm and the fix will only take effect
> with updated services.
>
> +1 for a fix. Thanks.



> pre-reactive, the burden was on the charm to wait until the database
> name provided matches the one requested. That is still what you can do
> here if you need an immediate work around, although the goal of the
> interface is to remove that sort of annoying implementation detail
> from your charm so I certainly should try and sort this out.
>
>
> > 2. Every ~ 1/5 deploys (odd how this keeps surfacing), I get a 'NoneType'
> > error when trying to access 'master.host', or 'master.uri' via relation
> to
> > the PostgreSQL charm on the firing of 'master.available'. See bug created
> > here -> https://bugs.launchpad.net/interface-pgsql/+bug/1666337
>
> Its related to the above issue. Your charm connects and gets the
> db.master.available state set. But you want to specify the database
> name, so a handler runs calling set_database(). At this point the
> .master and .standbys properties start correctly returning None, but
> set_database() neglected to remove the *.available states so handlers
> got kicked in that shouldn't have.
>
> Ok, so a fix coming for this too in that case? This one is borking on my
devs who are deploying my bundles, in turn causing me grief, but also
borking on me too, making me question my own sanity :(


> > 3. Users seem to have different access privs.
> >
> > On this specific deploy, everything seems to have initialized correctly,
> but
> > my applications don't have consistent access to the database across the
> > board. See -> http://paste.ubuntu.com/24046732/
>
> PostgreSQL tries to be secure by default, so users do not implicitly
> get access to each others tables. The charm that creates the tables
> needs to also grant permissions to them. Commonly people just 'GRANT
> ALL ON TABLE foo TO PUBLIC' to give full permissions to all other
> users on the system (all related services in the Juju case).


Applying this GRANT to just the other user got me around the permissions
issue as well (I was previously only granting to the database, didn't
realize I also needed to grant table explicitly), thanks!


> If you
> need more control, you can use the set_roles() method on the interface
> to have PostgreSQL grant some roles to your user, and then grant
> permissions explicitly to those roles. But this doesn't really help
> much from a security POV, so I've been toying with the idea of just
> having clients all connect as the same user for the common case where
> people don't want granular permissions (even if it does make security
> minded people wince).
>

Will the "common use case" be the only use case?



> --
> Stuart Bishop 
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: PostgreSQL Use Case/Issues

[Stuart Bishop]

On 22 February 2017 at 21:46, James Beedy  wrote:

> Experiencing some varying results with the PostgreSQL charm, hoping to get
> some validation on my use case.

This all seems the pgsql interface. Expected use is at
http://interface-pgsql.readthedocs.io/en/stable/requires.html#example-usage
if you haven't seen it already. The protocol had a fairly large
(backwards compatible) change recently, so sorry about the teething
troubles.

> 1. Sometimes (feeling like 1/5ish deploys) a database is created with the
> name of the application, instead of the database name the application
> requested.

Unfortunately this seems possible.

A client can 'accept the defaults' by not setting any properties on
the db relation when it joins (dating back to the original protocol
with pyjuju). When the PostgreSQL charm runs its relation-joined and
relation-changed hooks, it has no way of telling if the client just
wants to 'accept the defaults', or if the client has not yet run its
relation-joined or relation-changed hooks yet. So if it sees an empty
relation, it assumes 'accept the defaults' and provides a database
named after the client service. If your client then runs its
relation-joined hook, the 'db.connected' state will be set (because
there is a relation), and the 'db.*.available' states will also be set
because there is a database available (not the one you want, but you
haven't had a change to say otherwise yet). At which point your
handlers kick in on *.available and get valid connection strings to a
different database to the one you want. And maybe the handler that
calls set_database() runs too, but it is too late.

I think I can fix this, but I'll need to make a corresponding
adjustment in the PostgreSQL charm and the fix will only take effect
with updated services.

pre-reactive, the burden was on the charm to wait until the database
name provided matches the one requested. That is still what you can do
here if you need an immediate work around, although the goal of the
interface is to remove that sort of annoying implementation detail
from your charm so I certainly should try and sort this out.


> 2. Every ~ 1/5 deploys (odd how this keeps surfacing), I get a 'NoneType'
> error when trying to access 'master.host', or 'master.uri' via relation to
> the PostgreSQL charm on the firing of 'master.available'. See bug created
> here -> https://bugs.launchpad.net/interface-pgsql/+bug/1666337

Its related to the above issue. Your charm connects and gets the
db.master.available state set. But you want to specify the database
name, so a handler runs calling set_database(). At this point the
.master and .standbys properties start correctly returning None, but
set_database() neglected to remove the *.available states so handlers
got kicked in that shouldn't have.


> 3. Users seem to have different access privs.
>
> On this specific deploy, everything seems to have initialized correctly, but
> my applications don't have consistent access to the database across the
> board. See -> http://paste.ubuntu.com/24046732/

PostgreSQL tries to be secure by default, so users do not implicitly
get access to each others tables. The charm that creates the tables
needs to also grant permissions to them. Commonly people just 'GRANT
ALL ON TABLE foo TO PUBLIC' to give full permissions to all other
users on the system (all related services in the Juju case). If you
need more control, you can use the set_roles() method on the interface
to have PostgreSQL grant some roles to your user, and then grant
permissions explicitly to those roles. But this doesn't really help
much from a security POV, so I've been toying with the idea of just
having clients all connect as the same user for the common case where
people don't want granular permissions (even if it does make security
minded people wince).

-- 
Stuart Bishop 

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju GUI handling of empty config breaks promulgated charms

[Merlijn Sebrechts]

This - is - awesome !

I guess this hack will still require a rebuild for the affected Charms?

2017-02-22 16:41 GMT+01:00 Stuart Bishop :

> On 21 February 2017 at 16:30, Merlijn Sebrechts
>  wrote:
> > Thanks, Stuart!
> >
> > Now I get a different error. It seems that the charm really can't handle
> > null as a string value.
>
> Rather than comb through the charm updating all the call sites, I've
> made a hack to ensure config values never get returned as null. I also
> didn't want to change the style, as the alternative (using
> config.get()) can hide other bugs.
>
> Other charms will have the same problem, so this should certainly be
> fixed in Juju.
>
> --
> Stuart Bishop 
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju GUI handling of empty config breaks promulgated charms

[Stuart Bishop]

On 21 February 2017 at 16:30, Merlijn Sebrechts
 wrote:
> Thanks, Stuart!
>
> Now I get a different error. It seems that the charm really can't handle
> null as a string value.

Rather than comb through the charm updating all the call sites, I've
made a hack to ensure config values never get returned as null. I also
didn't want to change the style, as the alternative (using
config.get()) can hide other bugs.

Other charms will have the same problem, so this should certainly be
fixed in Juju.

-- 
Stuart Bishop 

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

PostgreSQL Use Case/Issues

[James Beedy]

Hello,

Experiencing some varying results with the PostgreSQL charm, hoping to get
some validation on my use case.

I'm trying to connect different charms to the PostgreSQL charm, with each
charm requesting the same database from PostgreSQL. What I'm expecting to
happen, is for the PostgreSQL charm to create a user for each requesting
application, each user being granted access on the database that the charm
requested. This is mostly happening as I expect, minus a few gotchas.
Hopefully someone can verify that this is expected behavior.

Basically, I'm getting some inconsistent results across the board. Let me
enumerate.

1. Sometimes (feeling like 1/5ish deploys) a database is created with the
name of the application, instead of the database name the application
requested.

Here is an example of a bundle that gets deployed where each 'feed-web',
'feed-worker', and 'copy' charms all request the same database.

http://paste.ubuntu.com/24046778/

Looking into the database tables and users that have been created/granted
access shows:

http://paste.ubuntu.com/24046795/.

On line #5, you will see the rouge db. Unfortunately, that db is the one
that my 'feed-web' charm ends up getting from the pgsql interface, and ends
up in its config. This is bad, because it then uses a different database
then the rest of the apps in the stack. The rest of the charms get the
database that they requested.

2. Every ~ 1/5 deploys (odd how this keeps surfacing), I get a 'NoneType'
error when trying to access 'master.host', or 'master.uri' via relation to
the PostgreSQL charm on the firing of 'master.available'. See bug created
here -> https://bugs.launchpad.net/interface-pgsql/+bug/1666337

3. Users seem to have different access privs.

On this specific deploy, everything seems to have initialized correctly,
but my applications don't have consistent access to the database across the
board. See -> http://paste.ubuntu.com/24046732/


I'm hoping to get some verification on whether what I'm doing, and
expecting to happen is legitimate or not #3, as well as some feedback as to
what the deal is with the inconsistencies in #2 and #1.

Thanks
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Generic VNF Configuration and Management talk from Config Management Camp

[Merlijn Sebrechts]

Thanks for posting,  Adam! I wanted to see the talk, but I forgot.. Very
good presentation, answers a bunch of questions I had.

2017-02-22 11:00 GMT+01:00 Adam Israel :

> Hey everyone,
>
> As part of the recent CfgMgmtCamp in Ghent, Belgium, I gave a talk on the
> main track titled Generic VNF Configuration and Management.
>
> Much to my introverted horror ;) , it was filmed and is now available for
> everyone to see. If you have any interest in NFV, VNFs, or the
> telecommunication space, you may find it useful.
>
> https://www.youtube.com/watch?v=en6P3WawUBA&index=6&list=
> PLBZBIkixHEidnVKFIxogd3Qf-ai3RKTJ_
>
> -Adam
> --
> Adam Israel, Software Engineer
> Canonical // Cloud DevOps // Juju // Ecosystem
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Generic VNF Configuration and Management talk from Config Management Camp

[Adam Israel]

Hey everyone,

As part of the recent CfgMgmtCamp in Ghent, Belgium, I gave a talk on the
main track titled Generic VNF Configuration and Management.

Much to my introverted horror ;) , it was filmed and is now available for
everyone to see. If you have any interest in NFV, VNFs, or the
telecommunication space, you may find it useful.

https://www.youtube.com/watch?v=en6P3WawUBA&index=6&list=PLBZBIkixHEidnVKFIxogd3Qf-ai3RKTJ_

-Adam
-- 
Adam Israel, Software Engineer
Canonical // Cloud DevOps // Juju // Ecosystem
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju