Re: upgrade-charm and default configuration values

[Michael Nelson]

Hi there Marco,

On Wed, Mar 9, 2016 at 4:18 AM Marco Ceppi 
wrote:

> Hello everyone!
>
> tl;dr: upgrade-charm also updates default configuration which can break
> running deployments. Should we be updating charms anyways esp since
> defaults are now broken for fresh deploys? Discuss.
>

I think the updating of default config options is a good discussion to
have, but more specifically, the question I feel needs answering is: Can a
charm not only update the default config options (to point to a new major
version, for eg.) but also update it's support to only handle that new
default option. This is, afaiui, what seems to happen. More details below...


>
> We've had a problem pop up a few times and I wanted to start a discussion
> about it with the community. Default configuration values and upgrade-charm.
>
> We have a few examples where the charm by default is broken, but we're
> getting resistance on updating these because of the default action in Juju.
> On upgrade-charm, if the default configuration is still set on the charm,
> then it will be updated on charm upgrade. I understand why this seems like
> a good idea but I have a few examples where this really hurts.
>
> MySQL
>
> There is a long standing issue with MySQL on local/LXC where because of a
> faulty configuration option default value which has existed since the charm
> was created, it attempts to consume 80% of available ram for a cache
> instead of just a sane default (300M for example) which MySQL experts
> recommend. However, because of the implication of the change for running
> deployments the merge request is essentially on-hold:
>
> [0] https://bugs.launchpad.net/charms/+source/mysql/+bug/1373862
>
> ELK
>
> Recently, a few configuration options were changed for Kibana[1] that
> aligned it with the latest release of that software.
>

The issue isn't just that the default config options were changed - if you
check that MP [1] you'll see kibana3 used a javascript configuration file
on the unit itself (search for files/charm/config.js), whereas kibana4 uses
a yml file (search for files/charm/config.yml).

So once I upgrade-charm to this version, even if I've explicitly set my
kibana_source and kibana_source_checksum options so it won't unexpectedly
get new defaults of kibana4, it still won't work because my kibana3 won't
know what to do with the yml config file - it'll still expect the js one
(or perhaps it'd just keep using the old file - but the init script which
runs /srv/kibana4 would fail). In that case, it may be better if I'd not
set kibana_source so I would upgrade to the new version - not sure.

That is, the change here to upgrade-charm would break an existing
deployment. I'm not saying it was the wrong thing to land those changes,
just that we need to be aware (and make sure people who use the charms for
more than demo's are aware).



> This, and changes to logstash mean the elasticsearch charm need to be
> updated too. As it stands today, with the defaults in the charms, there is
> no working ELK stack[2].
>

I don't think that's true - we're using elasticsearch2, kibana4 and
logstash2 for an ELK deployment currently (since late last year). The
change in the kibana charm (to kibana4) meant we needed to use ES2.x and
Logstash2.x. I explicitly didn't land the es2 and logstash2 charm changes
in the upstream stable charms for the above reason, instead creating an
elasticsearch2 branch:

https://code.launchpad.net/~onlineservices-charmers/charms/trusty/elasticsearch/elasticsearch2

Not ideal - but it won't break existing deployments. If you think it's
better, I can propose the elasticsearch2 charm (and the related logstash2)
 for merging into the main "stable" trusty charms, but we just need to be
aware that we may break existing deployments if people upgrade.

I don't know that you can generally update a major version of a service
automatically on upgrade-charm without affecting related services. We have
an existing project which currently uses ES1.x which we're upgrading to 2.x
(not via upgrade-charm). If you want more details about why we can't simply
upgrade-charm, feel free to reach out to Wes Mason, who's doing the upgrade.

Cheers,
Michael


> There's a merge[3] to address this but again is being met with resistance
> because this implicit issue of existing deployments get the latest
> configuration if it's still default.
>
> [1]
> https://code.launchpad.net/~chris.macnaughton/charms/trusty/kibana/version_bump/+merge/276709
> [2] https://bugs.launchpad.net/charms/+source/elasticsearch/+bug/1553256
> [3]
> https://code.launchpad.net/~lazypower/charms/trusty/elasticsearch/20-bump/+merge/288423
>
> I'm inclined to merge these and properly educate users on configuration
> pinning and bundles to make sure the latest software works as expected
>


> but am interested in feedback and discussing if we need to potentially
> change the default behavior of Juju to if not allow the user to address
> this at 

httpbakery .Client overly restrictive on body - io.ReaderSeeker

[Tim Penhey]

Hi folks,

I came across this recently while looking at sending tools and charms
between two controllers.

The local storage interface gives us an io.ReaderCloser. Unfortunately,
we can't give that to the http client for the POST method as it needs an
io.ReaderSeeker.

I'd like to challenge the need for the seek method.

Is there any way we could change the method to take an io.Reader?

Perhaps with a documentation caveat that if it implements
io.ReaderSeeker, that Seek(0,0) will be called.

Right now, I'm going through a local temp file, but I'd rather not have
to if I don't need to.

Tim

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: admin is dead, long live $USER

[Andrew Wilkins]

I think I did remove some of that code, because it wasn't being used and
wasn't clear why it was there. Easy enough to put back in.

Cheers,
Andrew

On Mon, Mar 7, 2016 at 8:40 AM Tim Penhey  wrote:

> Ah yeah... it is mostly coming back to me now.
>
> I actually have a branch lying around that does most of this.
>
> Perhaps I should look at it again :-)
>
> Tim
>
> On 04/03/16 20:08, Ian Booth wrote:
> > Hey Tim
> >
> > The new bootstrap UX has not removed any --admin-user flag.
> > I can see that the server jujud bootstrap command has an --admin-user
> argument
> > but it appears this is never set anywhere in the cloud init scripts. Or
> not that
> > I can see. I've checked older version of the relevant files and can't
> see where
> > we've ever used this.
> >
> > So maybe we have a capability to bootstrap the controller agent with a
> specified
> > admin-user but have not hooked it up yet?
> >
> > On 04/03/16 08:11, Tim Penhey wrote:
> >> Ah... it used to be there :-) At least it is on my feature branch, but I
> >> don't think I have merged the most recent master updates that has the
> >> work to re-work bootstrap for the new cloud credentials stuff.
> >>
> >> Tim
> >>
> >> On 04/03/16 10:09, Rick Harding wrote:
> >>> If we do that we need to also make it configurable on bootstrap as an
> >>> option.
> >>>
> >>> +1 overall
> >>>
> >>>
> >>> On Thu, Mar 3, 2016, 4:07 PM Tim Penhey  >>> > wrote:
> >>>
> >>> Hi folks,
> >>>
> >>> I was thinking that with the upcoming big changes with 2.0, we
> should
> >>> tackle a long held issue where we have the initial user called
> "admin".
> >>>
> >>> There was a request some time back that we should use the current
> user's
> >>> name. The reason it wasn't implemented at that time was due to
> logging
> >>> into the GUI issues. These have been resolved some time back with
> the
> >>> multiple user support that was added.
> >>>
> >>> All the server side code handles the ability to define the initial
> user
> >>> for the controller model, and we do this in all the tests, so the
> >>> default test user is actually called "test-admin".
> >>>
> >>> I *think* that all we need to do is change the default value we
> use in
> >>> the bootstrap command for the AdminUserName (--admin-user flag)
> from
> >>> "admin" to something we derive from the current user.
> >>>
> >>> Probably worth doing now.
> >>>
> >>> Thoughts?
> >>>
> >>> Tim
> >>>
> >>> --
> >>> Juju-dev mailing list
> >>> Juju-dev@lists.ubuntu.com 
> >>> Modify settings or unsubscribe at:
> >>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
> >>>
> >>
> >>
>
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Planning for Juju 2.2 (16.10 timeframe)

[Mark Shuttleworth]

On 08/03/16 16:04, Tom Barber wrote:
> Also, for stuff like monitoring,  being able to position a charm service on
> a different service provider to bolster resiliency.

That comes implicitly with cross-model relations, since the different
models can be on different clouds.

This enables pretty amazing hybrid operations in general - place what
you want where you want it, then just connect it all up.

Mark

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Planning for Juju 2.2 (16.10 timeframe)

[Mark Shuttleworth]

On 08/03/16 15:59, Tom Barber wrote:
> From my perspective relationship joins that can span models would be great.
> I know I brought it up before, but being able to create, for example a
> central monitoring model, or central Gitlab model that charms in my various
> other models could tap into without them being merged into a "super model"
> or maybe be in different regions on different controllers, would be great.

Yes, this is a great request, and the good news is it is on the list for
Juju 2.1 :)

Mark

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: upgrade-charm and default configuration values

[Rick Harding]

Thanks Marco, if I understand what we're dealing with is there a pattern
here where we can get a little mix of both worlds?

What I'm wondering is that, if the charm author needs to make these default
changes for a good reason, that we can bring it to user's attention using
the blocked status? Can we upgrade, but mark the charm as 'blocked on
accepting new config default' and teach users that if they run something,
such as an unset command, or some next config changed event, that the charm
hooks can respond to in a standard fashion?

This allows the charm author to move forward, but also makes sure that
significant changes are made much more apparent to end users.

On Tue, Mar 8, 2016 at 12:18 PM Marco Ceppi 
wrote:

> Hello everyone!
>
> tl;dr: upgrade-charm also updates default configuration which can break
> running deployments. Should we be updating charms anyways esp since
> defaults are now broken for fresh deploys? Discuss.
>
> We've had a problem pop up a few times and I wanted to start a discussion
> about it with the community. Default configuration values and upgrade-charm.
>
> We have a few examples where the charm by default is broken, but we're
> getting resistance on updating these because of the default action in Juju.
> On upgrade-charm, if the default configuration is still set on the charm,
> then it will be updated on charm upgrade. I understand why this seems like
> a good idea but I have a few examples where this really hurts.
>
> MySQL
>
> There is a long standing issue with MySQL on local/LXC where because of a
> faulty configuration option default value which has existed since the charm
> was created, it attempts to consume 80% of available ram for a cache
> instead of just a sane default (300M for example) which MySQL experts
> recommend. However, because of the implication of the change for running
> deployments the merge request is essentially on-hold:
>
> [0] https://bugs.launchpad.net/charms/+source/mysql/+bug/1373862
>
> ELK
>
> Recently, a few configuration options were changed for Kibana[1] that
> aligned it with the latest release of that software. This, and changes to
> logstash mean the elasticsearch charm need to be updated too. As it stands
> today, with the defaults in the charms, there is no working ELK stack[2].
> There's a merge[3] to address this but again is being met with resistance
> because this implicit issue of existing deployments get the latest
> configuration if it's still default.
>
> [1]
> https://code.launchpad.net/~chris.macnaughton/charms/trusty/kibana/version_bump/+merge/276709
> [2] https://bugs.launchpad.net/charms/+source/elasticsearch/+bug/1553256
> [3]
> https://code.launchpad.net/~lazypower/charms/trusty/elasticsearch/20-bump/+merge/288423
>
> I'm inclined to merge these and properly educate users on configuration
> pinning and bundles to make sure the latest software works as expected but
> am interested in feedback and discussing if we need to potentially change
> the default behavior of Juju to if not allow the user to address this at
> least provide feedback that not only will this new charm be added but these
> configuration values updated.
>
> I understand a lot of the ELK problems go away with resources in 2.0, but
> the MySQL is an example that we run into a bit of times when addressing
> problems in configuration.
>
> Thanks,
> Marco Ceppi
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Planning for Juju 2.2 (16.10 timeframe)

[Tom Barber]

Also, for stuff like monitoring,  being able to position a charm service on
a different service provider to bolster resiliency.

Tom
On 8 Mar 2016 23:59, "Tom Barber"  wrote:

> Hi Mark
>
> From my perspective relationship joins that can span models would be
> great. I know I brought it up before, but being able to create, for example
> a central monitoring model, or central Gitlab model that charms in my
> various other models could tap into without them being merged into a "super
> model" or maybe be in different regions on different controllers, would be
> great.
>
> Cheers
>
> Tom
>
> --
>
> Director Meteorite.bi - Saiku Analytics Founder
> Tel: +44(0)5603641316
>
> (Thanks to the Saiku community we reached our Kickstart
> 
> goal, but you can always help by sponsoring the project
> )
>
> On 8 March 2016 at 23:51, Mark Shuttleworth  wrote:
>
>> Hi folks
>>
>> We're starting to think about the next development cycle, and gathering
>> priorities and requests from users of Juju. I'm writing to outline some
>> current topics and also to invite requests or thoughts on relative
>> priorities - feel free to reply on-list or to me privately.
>>
>> An early cut of topics of interest is below.
>>
>>
>>
>> *Operational concerns ** LDAP integration for Juju controllers now we
>> have multi-user controllers
>> * Support for read-only config
>> * Support for things like passwords being disclosed to a subset of
>> user/operators
>> * LXD container migration
>> * Shared uncommitted state - enable people to collaborate around changes
>> they want to make in a model
>>
>> There has also been quite a lot of interest in log control - debug
>> settings for logging, verbosity control, and log redirection as a systemic
>> property. This might be a good area for someone new to the project to lead
>> design and implementation. Another similar area is the idea of modelling
>> machine properties - things like apt / yum repositories, cache settings
>> etc, and having the machine agent setup the machine / vm / container
>> according to those properties.
>>
>>
>>
>> *Core Model * * modelling individual services (i.e. each database
>> exported by the db application)
>>  * rich status (properties of those services and the application itself)
>>  * config schemas and validation
>>  * relation config
>>
>> There is also interest in being able to invoke actions across a relation
>> when the relation interface declares them. This would allow, for example, a
>> benchmark operator charm to trigger benchmarks through a relation rather
>> than having the operator do it manually.
>>
>> *Storage*
>>
>>  * shared filesystems (NFS, GlusterFS, CephFS, LXD bind-mounts)
>>  * object storage abstraction (probably just mapping to S3-compatible
>> APIS)
>>
>> I'm interested in feedback on the operations aspects of storage. For
>> example, whether it would be helpful to provide lifecycle management for
>> storage being re-assigned (e.g. launch a new database application but reuse
>> block devices previously bound to an old database  instance). Also, I think
>> the intersection of storage modelling and MAAS hasn't really been explored,
>> and since we see a lot of interest in the use of charms to deploy
>> software-defined storage solutions, this probably will need thinking and
>> work.
>>
>>
>>
>> *Clouds and providers *
>>  * System Z and LinuxONE
>>  * Oracle Cloud
>>
>> There is also a general desire to revisit and refactor the provider
>> interface. Now we have seen many cloud providers get done, we are in a
>> better position to design the best provider interface. This would be a
>> welcome area of contribution for someone new to the project who wants to
>> make it easier for folks creating new cloud providers. We also see constant
>> requests for a Linode provider that would be a good target for a refactored
>> interface.
>>
>>
>>
>>
>> *Usability * * expanding the set of known clouds and regions
>>  * improving the handling of credentials across clouds
>>
>> Mark
>>
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Planning for Juju 2.2 (16.10 timeframe)

[Tom Barber]

Hi Mark

>From my perspective relationship joins that can span models would be great.
I know I brought it up before, but being able to create, for example a
central monitoring model, or central Gitlab model that charms in my various
other models could tap into without them being merged into a "super model"
or maybe be in different regions on different controllers, would be great.

Cheers

Tom

--

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316

(Thanks to the Saiku community we reached our Kickstart

goal, but you can always help by sponsoring the project
)

On 8 March 2016 at 23:51, Mark Shuttleworth  wrote:

> Hi folks
>
> We're starting to think about the next development cycle, and gathering
> priorities and requests from users of Juju. I'm writing to outline some
> current topics and also to invite requests or thoughts on relative
> priorities - feel free to reply on-list or to me privately.
>
> An early cut of topics of interest is below.
>
>
>
> *Operational concerns ** LDAP integration for Juju controllers now we
> have multi-user controllers
> * Support for read-only config
> * Support for things like passwords being disclosed to a subset of
> user/operators
> * LXD container migration
> * Shared uncommitted state - enable people to collaborate around changes
> they want to make in a model
>
> There has also been quite a lot of interest in log control - debug
> settings for logging, verbosity control, and log redirection as a systemic
> property. This might be a good area for someone new to the project to lead
> design and implementation. Another similar area is the idea of modelling
> machine properties - things like apt / yum repositories, cache settings
> etc, and having the machine agent setup the machine / vm / container
> according to those properties.
>
>
>
> *Core Model * * modelling individual services (i.e. each database
> exported by the db application)
>  * rich status (properties of those services and the application itself)
>  * config schemas and validation
>  * relation config
>
> There is also interest in being able to invoke actions across a relation
> when the relation interface declares them. This would allow, for example, a
> benchmark operator charm to trigger benchmarks through a relation rather
> than having the operator do it manually.
>
> *Storage*
>
>  * shared filesystems (NFS, GlusterFS, CephFS, LXD bind-mounts)
>  * object storage abstraction (probably just mapping to S3-compatible APIS)
>
> I'm interested in feedback on the operations aspects of storage. For
> example, whether it would be helpful to provide lifecycle management for
> storage being re-assigned (e.g. launch a new database application but reuse
> block devices previously bound to an old database  instance). Also, I think
> the intersection of storage modelling and MAAS hasn't really been explored,
> and since we see a lot of interest in the use of charms to deploy
> software-defined storage solutions, this probably will need thinking and
> work.
>
>
>
> *Clouds and providers *
>  * System Z and LinuxONE
>  * Oracle Cloud
>
> There is also a general desire to revisit and refactor the provider
> interface. Now we have seen many cloud providers get done, we are in a
> better position to design the best provider interface. This would be a
> welcome area of contribution for someone new to the project who wants to
> make it easier for folks creating new cloud providers. We also see constant
> requests for a Linode provider that would be a good target for a refactored
> interface.
>
>
>
>
> *Usability * * expanding the set of known clouds and regions
>  * improving the handling of credentials across clouds
>
> Mark
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju 2.0 and local charm deployment

[Ian Booth]

So to clarify what we'll do. We'll support the same syntax in bundle files as we
do for deploy.

Deploys charm store charms:

$ juju deploy cs:wordpress
$ juju deploy wordpress

Deploys a local charm from a directory:

$ juju deploy ./charms/wordpress
$ juju deploy ./wordpress

So below deploys a local nova-compute charm in a directory co-located with the
bundle.yaml file.

 series: trusty
 services:
   nova-compute:
 charm: ./nova-compute
 num_units: 2

This one deploys a charm store charm:

 series: trusty
 services:
   nova-compute:
   charm: nova-compute
   num_units: 2



On 09/03/16 03:59, Rick Harding wrote:
> Long term we want to have a pattern when the bundle is a directory with
> local charms in a directory next to the bundles.yaml file. We could not do
> this cleanly before the multi-series charms that are just getting out the
> door. I think that bundles with local charms will be suboptimal until we
> can get those bits to line up.
> 
> I don't think we want to be doing the file based urls, but to build a
> pattern that's reusable and makes sense across systems. Creating a standard
> pattern I think is the best path forward.
> 
> On Tue, Mar 8, 2016 at 12:26 PM Martin Packman 
> wrote:
> 
>> On 05/03/2016, Ian Booth  wrote:

 How will bundles work which reference local charms? Will this work as
 expected where nova-compute is a directory at the same level as a bundle
 file?

 ```
 series: trusty
 services:
   nova-compute:
 charm: ./nova-compute
 num_units: 2
 ```

>>>
>>> The above will work but not until a tweak is made to bundle deployment to
>>> interpret a path on disk rather than a url. It's a small change. This
>> would
>>> be done as part of the work to remove the local repo support.
>>
>> Can we keep interpreting the the reference in the bundle as a url, but
>> start supporting file urls? That seems neater than treating the cs:
>> prefix as magic not-a-filename.
>>
>> The catch is that there's no sane way of referencing locations outside
>> a base url.
>>
>> charm: file:nova-compute
>>
>> Works as a reference to a dir inside the base location, but:
>>
>> charm: file:../nova-compute
>>
>> Will not work as a reference to a sibling directory. And absolute file
>> paths are pretty useless across machines.
>>
>> Martin
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>
> 
> 
> 

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Planning for Juju 2.2 (16.10 timeframe)

[Mark Shuttleworth]

Hi folks

We're starting to think about the next development cycle, and gathering
priorities and requests from users of Juju. I'm writing to outline some
current topics and also to invite requests or thoughts on relative
priorities - feel free to reply on-list or to me privately.

An early cut of topics of interest is below.

*Operational concerns

** LDAP integration for Juju controllers now we have multi-user controllers
* Support for read-only config
* Support for things like passwords being disclosed to a subset of
user/operators
* LXD container migration
* Shared uncommitted state - enable people to collaborate around changes
they want to make in a model

There has also been quite a lot of interest in log control - debug
settings for logging, verbosity control, and log redirection as a
systemic property. This might be a good area for someone new to the
project to lead design and implementation. Another similar area is the
idea of modelling machine properties - things like apt / yum
repositories, cache settings etc, and having the machine agent setup the
machine / vm / container according to those properties.

*Core Model

* * modelling individual services (i.e. each database exported by the db
application)
 * rich status (properties of those services and the application itself)
 * config schemas and validation
 * relation config

There is also interest in being able to invoke actions across a relation
when the relation interface declares them. This would allow, for
example, a benchmark operator charm to trigger benchmarks through a
relation rather than having the operator do it manually.

*Storage*

 * shared filesystems (NFS, GlusterFS, CephFS, LXD bind-mounts)
 * object storage abstraction (probably just mapping to S3-compatible APIS)

I'm interested in feedback on the operations aspects of storage. For
example, whether it would be helpful to provide lifecycle management for
storage being re-assigned (e.g. launch a new database application but
reuse block devices previously bound to an old database  instance).
Also, I think the intersection of storage modelling and MAAS hasn't
really been explored, and since we see a lot of interest in the use of
charms to deploy software-defined storage solutions, this probably will
need thinking and work.


*Clouds and providers
*
 * System Z and LinuxONE
 * Oracle Cloud

There is also a general desire to revisit and refactor the provider
interface. Now we have seen many cloud providers get done, we are in a
better position to design the best provider interface. This would be a
welcome area of contribution for someone new to the project who wants to
make it easier for folks creating new cloud providers. We also see
constant requests for a Linode provider that would be a good target for
a refactored interface.


*Usability

* * expanding the set of known clouds and regions
 * improving the handling of credentials across clouds

Mark
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju stable 1.25.4 is proposed for release

[Alexis Bruemmer]

WOOT!  Thank you to the teams for pushing hard to get this out!

On Tue, Mar 8, 2016 at 2:17 PM, Curtis Hovey-Canonical  wrote:

> # juju-core 1.25.4
>
> A new proposed stable release of Juju, juju-core 1.25.4, is now available.
> This release may replace version 1.25.3 on Tuesday March 15.
>
>
> ## Getting Juju
>
> juju-core 1.25.4 is available for Xenial and backported to earlier
> series in the following PPA:
>
> https://launchpad.net/~juju/+archive/proposed
>
> Windows, Centos, and OS X users will find installers at:
>
> https://launchpad.net/juju-core/+milestone/1.25.4
>
> Proposed releases use the "proposed" simple-streams. You must configure
> the 'agent-stream' option in your environments.yaml to use the matching
> juju agents.
>
>
> ## Notable Changes
>
> This releases addresses stability and performance issues.
>
>
> ## Resolved issues
>
>   * Jujud offers poodle vulnerable sslv3 on 17070
> Lp 1536269
>
>   * Leadership dependency failures in ci deploy tests
> Lp 1539656
>
>   * Upgrade 1.24.7 -> 1.25.3 fails
> Lp 1546100
>
>   * Destroyed leader, new leader not elected.
> Lp 1511659
>
>   * New ec2 korea region
> Lp 1530957
>
>   * Maas bridge script handles vlan nics incorrectly
> Lp 1532167
>
>   * 1.25.0: bootstrap failure - warning discarding api open error: eof
> Lp 1538303
>
>   * Bootstrap node does not use the proxy to fetch tools from
> streams.c.c
> Lp 1515289
>
>   * Payload/persistence intermittent failure
> Lp 1519061
>
>   * 1.25.3 can't bootstrap xenial environments
> Lp 1550306
>
>   * Enable daily image stream in juju local provider
> Lp 1554073
>
>   * Juju's maas bridge script is echoed to the console by cloud-init
> during bootstrap
> Lp 1536587
>
>   * Cookies file locked for too long
> Lp 1534643
>
>
> Finally
>
> We encourage everyone to subscribe the mailing list at
> juju-...@lists.canonical.com, or join us on #juju-dev on freenode.
>
>
> --
> Curtis Hovey
> Canonical Cloud Development and Operations
> http://launchpad.net/~sinzui
>
> --
> Juju-dev mailing list
> juju-...@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>



-- 
Alexis Bruemmer
Juju Core Manager, Canonical Ltd.
(503) 686-5018
alexis.bruem...@canonical.com
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju stable 1.25.4 is proposed for release

[Alexis Bruemmer]

WOOT!  Thank you to the teams for pushing hard to get this out!

On Tue, Mar 8, 2016 at 2:17 PM, Curtis Hovey-Canonical  wrote:

> # juju-core 1.25.4
>
> A new proposed stable release of Juju, juju-core 1.25.4, is now available.
> This release may replace version 1.25.3 on Tuesday March 15.
>
>
> ## Getting Juju
>
> juju-core 1.25.4 is available for Xenial and backported to earlier
> series in the following PPA:
>
> https://launchpad.net/~juju/+archive/proposed
>
> Windows, Centos, and OS X users will find installers at:
>
> https://launchpad.net/juju-core/+milestone/1.25.4
>
> Proposed releases use the "proposed" simple-streams. You must configure
> the 'agent-stream' option in your environments.yaml to use the matching
> juju agents.
>
>
> ## Notable Changes
>
> This releases addresses stability and performance issues.
>
>
> ## Resolved issues
>
>   * Jujud offers poodle vulnerable sslv3 on 17070
> Lp 1536269
>
>   * Leadership dependency failures in ci deploy tests
> Lp 1539656
>
>   * Upgrade 1.24.7 -> 1.25.3 fails
> Lp 1546100
>
>   * Destroyed leader, new leader not elected.
> Lp 1511659
>
>   * New ec2 korea region
> Lp 1530957
>
>   * Maas bridge script handles vlan nics incorrectly
> Lp 1532167
>
>   * 1.25.0: bootstrap failure - warning discarding api open error: eof
> Lp 1538303
>
>   * Bootstrap node does not use the proxy to fetch tools from
> streams.c.c
> Lp 1515289
>
>   * Payload/persistence intermittent failure
> Lp 1519061
>
>   * 1.25.3 can't bootstrap xenial environments
> Lp 1550306
>
>   * Enable daily image stream in juju local provider
> Lp 1554073
>
>   * Juju's maas bridge script is echoed to the console by cloud-init
> during bootstrap
> Lp 1536587
>
>   * Cookies file locked for too long
> Lp 1534643
>
>
> Finally
>
> We encourage everyone to subscribe the mailing list at
> juju-...@lists.canonical.com, or join us on #juju-dev on freenode.
>
>
> --
> Curtis Hovey
> Canonical Cloud Development and Operations
> http://launchpad.net/~sinzui
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>



-- 
Alexis Bruemmer
Juju Core Manager, Canonical Ltd.
(503) 686-5018
alexis.bruem...@canonical.com
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Juju stable 1.25.4 is proposed for release

[Curtis Hovey-Canonical]

# juju-core 1.25.4

A new proposed stable release of Juju, juju-core 1.25.4, is now available.
This release may replace version 1.25.3 on Tuesday March 15.


## Getting Juju

juju-core 1.25.4 is available for Xenial and backported to earlier
series in the following PPA:

https://launchpad.net/~juju/+archive/proposed

Windows, Centos, and OS X users will find installers at:

https://launchpad.net/juju-core/+milestone/1.25.4

Proposed releases use the "proposed" simple-streams. You must configure
the 'agent-stream' option in your environments.yaml to use the matching
juju agents.


## Notable Changes

This releases addresses stability and performance issues.


## Resolved issues

  * Jujud offers poodle vulnerable sslv3 on 17070
Lp 1536269

  * Leadership dependency failures in ci deploy tests
Lp 1539656

  * Upgrade 1.24.7 -> 1.25.3 fails
Lp 1546100

  * Destroyed leader, new leader not elected.
Lp 1511659

  * New ec2 korea region
Lp 1530957

  * Maas bridge script handles vlan nics incorrectly
Lp 1532167

  * 1.25.0: bootstrap failure - warning discarding api open error: eof
Lp 1538303

  * Bootstrap node does not use the proxy to fetch tools from
streams.c.c
Lp 1515289

  * Payload/persistence intermittent failure
Lp 1519061

  * 1.25.3 can't bootstrap xenial environments
Lp 1550306

  * Enable daily image stream in juju local provider
Lp 1554073

  * Juju's maas bridge script is echoed to the console by cloud-init
during bootstrap
Lp 1536587

  * Cookies file locked for too long
Lp 1534643


Finally

We encourage everyone to subscribe the mailing list at
juju-...@lists.canonical.com, or join us on #juju-dev on freenode.


-- 
Curtis Hovey
Canonical Cloud Development and Operations
http://launchpad.net/~sinzui

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Juju stable 1.25.4 is proposed for release

[Curtis Hovey-Canonical]

# juju-core 1.25.4

A new proposed stable release of Juju, juju-core 1.25.4, is now available.
This release may replace version 1.25.3 on Tuesday March 15.


## Getting Juju

juju-core 1.25.4 is available for Xenial and backported to earlier
series in the following PPA:

https://launchpad.net/~juju/+archive/proposed

Windows, Centos, and OS X users will find installers at:

https://launchpad.net/juju-core/+milestone/1.25.4

Proposed releases use the "proposed" simple-streams. You must configure
the 'agent-stream' option in your environments.yaml to use the matching
juju agents.


## Notable Changes

This releases addresses stability and performance issues.


## Resolved issues

  * Jujud offers poodle vulnerable sslv3 on 17070
Lp 1536269

  * Leadership dependency failures in ci deploy tests
Lp 1539656

  * Upgrade 1.24.7 -> 1.25.3 fails
Lp 1546100

  * Destroyed leader, new leader not elected.
Lp 1511659

  * New ec2 korea region
Lp 1530957

  * Maas bridge script handles vlan nics incorrectly
Lp 1532167

  * 1.25.0: bootstrap failure - warning discarding api open error: eof
Lp 1538303

  * Bootstrap node does not use the proxy to fetch tools from
streams.c.c
Lp 1515289

  * Payload/persistence intermittent failure
Lp 1519061

  * 1.25.3 can't bootstrap xenial environments
Lp 1550306

  * Enable daily image stream in juju local provider
Lp 1554073

  * Juju's maas bridge script is echoed to the console by cloud-init
during bootstrap
Lp 1536587

  * Cookies file locked for too long
Lp 1534643


Finally

We encourage everyone to subscribe the mailing list at
juju-...@lists.canonical.com, or join us on #juju-dev on freenode.


-- 
Curtis Hovey
Canonical Cloud Development and Operations
http://launchpad.net/~sinzui

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Reactive roadmap

[Casey Marshall]

Simon,
You raise some very interesting questions and good points.

I've also been developing charms with layers, run into some of these same
issues, and developed a pragmatic approach that has helped me manage it
pretty well so far. In a more mature layer/interface ecosystem, it might
not be the optimal approach. I look forward to adapting my tactics as the
ecosystem evolves.

On Tue, Mar 8, 2016 at 8:19 AM, Simon Davy  wrote:

> Hi all
>
> My team (Online Services at Canonical) maintains >20 private charms
> for our services, plus a few charmstore charms.
>
> Most of these charms are written with charmhelpers ansible support, or
> with the Services framework. We would like to move towards
> consolidating these approaches (as both have issues), and so have been
> experimenting with reactive.
>
> We like the ideas in reactive, especially the composition part, as
> sharing common code between our charms has been very painful. Also,
> the higher-level user-defined events that reactive provides is a
> definite improvement over having to implement the lower level relation
> dance semantics every time.
>
> However, it's a fast moving target, and we have encountered some
> issues.  So we have a couple of questions, that we haven't been able
> to locate answers for in reactive docs (we may have missed some?).
>
> 1) Is there a roadmap for reactive? A target for a stable 1.0 release,
> or similar? We'd ideally like a stable base to build from before
> committing to use a new framework, having been (re)writing/maintaining
> charms for 4+ years now :)
>

I too would like to see and help the ecosystem mature.


> 2) Layer pinning. Right now, layers are evolving fast, and the lack of
> pinning to layer versions has caused charm builds to break from day to
> day. Is this a planned feature?
>

For the time being, I've been avoiding upstream churn by vendoring all the
layers and interfaces that my deployment uses. These are all subdirectories
of a project that builds a bundle consisting of several charms, and also
contains supporting end-to-end test data and CI scripts. With the most
trivial of configuration changes, I can build, deploy and test these
services by compiling source from development branches, installing from
release tarballs or debian packages. This has been amazing for end-to-end
testing, demos of features in development, as well as complex feature
development where changes need to be coordinated across several services at
once.

Combined with the LXD provider, which is just amazingly fast on a xenial
host, this project makes me very confident about researching "what if"-type
scenarios about our deployment -- it doesn't take much at all to make minor
changes to a charm and just go try something. That could be a configuration
change, a different charm that provides the same interface, a feature
branch or set of feature branches for the services -- anything, really.

I manage the vendored layers and interfaces with "git subtree", which is
really the only way I'd even consider vendoring. Subtree allows me to pull
upstream layer & interface projects into mine, while retaining all the
upstream revision history, and retaining the ability to continue to
contribute to and pull from these upstream git repositories. Vendoring
without this capability would be fraught with peril, requiring too much
manual, tedious, error-prone work to work safely in my opinion.

An interface or layer that starts as a project subdirectory out of a
short-term need, can be easily refactored into something generally useful,
and then pushed to a separate git repo with a "git subtree push", which
then becomes the new upstream where I can collaborate with other charmers.
This is how I started the local-monitors and nrpe-external-master
interfaces. They began as an internal migration from an older charm example
(which was ansible-based). I published them with 'git subtree push' once I
felt they were suitable for general reuse.

I'll admit there are some disadvantages to the vendoring approach. Unless I
really pay attention to those upstreams, I might miss a useful bugfix or
security improvement. That's also a challenge with pinning btw (which is
effectively a cheap copy form of vendoring with references). It'd be nice
if I could rely on a branch name to be stable but provide bugfixes.

Completely vendoring upstreams with subtree could become unnecessary,
needless overhead, if stable upstreams were common practice. I'd still
probably want to pin upstream revisions though, just to have reproducible
builds.


> 3) Downloading from the internet. This issue has been common in
> charmstore charms, and is discouraged, AIUI. But the same issue
> applies for layers, and possibly with more effect, due to a layer's
> composibility.  We simply can not utilise any layer that downloads
> things from github or similar, and I'm sure others are in a similar
> situation.  We're aware of resources, but not convinced this is a
> 

Re: Juju 2.0 and local charm deployment

[Rick Harding]

Long term we want to have a pattern when the bundle is a directory with
local charms in a directory next to the bundles.yaml file. We could not do
this cleanly before the multi-series charms that are just getting out the
door. I think that bundles with local charms will be suboptimal until we
can get those bits to line up.

I don't think we want to be doing the file based urls, but to build a
pattern that's reusable and makes sense across systems. Creating a standard
pattern I think is the best path forward.

On Tue, Mar 8, 2016 at 12:26 PM Martin Packman 
wrote:

> On 05/03/2016, Ian Booth  wrote:
> >>
> >> How will bundles work which reference local charms? Will this work as
> >> expected where nova-compute is a directory at the same level as a bundle
> >> file?
> >>
> >> ```
> >> series: trusty
> >> services:
> >>   nova-compute:
> >> charm: ./nova-compute
> >> num_units: 2
> >> ```
> >>
> >
> > The above will work but not until a tweak is made to bundle deployment to
> > interpret a path on disk rather than a url. It's a small change. This
> would
> > be done as part of the work to remove the local repo support.
>
> Can we keep interpreting the the reference in the bundle as a url, but
> start supporting file urls? That seems neater than treating the cs:
> prefix as magic not-a-filename.
>
> The catch is that there's no sane way of referencing locations outside
> a base url.
>
> charm: file:nova-compute
>
> Works as a reference to a dir inside the base location, but:
>
> charm: file:../nova-compute
>
> Will not work as a reference to a sibling directory. And absolute file
> paths are pretty useless across machines.
>
> Martin
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

upgrade-charm and default configuration values

[Marco Ceppi]

Hello everyone!

tl;dr: upgrade-charm also updates default configuration which can break
running deployments. Should we be updating charms anyways esp since
defaults are now broken for fresh deploys? Discuss.

We've had a problem pop up a few times and I wanted to start a discussion
about it with the community. Default configuration values and upgrade-charm.

We have a few examples where the charm by default is broken, but we're
getting resistance on updating these because of the default action in Juju.
On upgrade-charm, if the default configuration is still set on the charm,
then it will be updated on charm upgrade. I understand why this seems like
a good idea but I have a few examples where this really hurts.

MySQL

There is a long standing issue with MySQL on local/LXC where because of a
faulty configuration option default value which has existed since the charm
was created, it attempts to consume 80% of available ram for a cache
instead of just a sane default (300M for example) which MySQL experts
recommend. However, because of the implication of the change for running
deployments the merge request is essentially on-hold:

[0] https://bugs.launchpad.net/charms/+source/mysql/+bug/1373862

ELK

Recently, a few configuration options were changed for Kibana[1] that
aligned it with the latest release of that software. This, and changes to
logstash mean the elasticsearch charm need to be updated too. As it stands
today, with the defaults in the charms, there is no working ELK stack[2].
There's a merge[3] to address this but again is being met with resistance
because this implicit issue of existing deployments get the latest
configuration if it's still default.

[1]
https://code.launchpad.net/~chris.macnaughton/charms/trusty/kibana/version_bump/+merge/276709
[2] https://bugs.launchpad.net/charms/+source/elasticsearch/+bug/1553256
[3]
https://code.launchpad.net/~lazypower/charms/trusty/elasticsearch/20-bump/+merge/288423

I'm inclined to merge these and properly educate users on configuration
pinning and bundles to make sure the latest software works as expected but
am interested in feedback and discussing if we need to potentially change
the default behavior of Juju to if not allow the user to address this at
least provide feedback that not only will this new charm be added but these
configuration values updated.

I understand a lot of the ELK problems go away with resources in 2.0, but
the MySQL is an example that we run into a bit of times when addressing
problems in configuration.

Thanks,
Marco Ceppi
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Reactive roadmap

[Simon Davy]

Hi all

My team (Online Services at Canonical) maintains >20 private charms
for our services, plus a few charmstore charms.

Most of these charms are written with charmhelpers ansible support, or
with the Services framework. We would like to move towards
consolidating these approaches (as both have issues), and so have been
experimenting with reactive.

We like the ideas in reactive, especially the composition part, as
sharing common code between our charms has been very painful. Also,
the higher-level user-defined events that reactive provides is a
definite improvement over having to implement the lower level relation
dance semantics every time.

However, it's a fast moving target, and we have encountered some
issues.  So we have a couple of questions, that we haven't been able
to locate answers for in reactive docs (we may have missed some?).

1) Is there a roadmap for reactive? A target for a stable 1.0 release,
or similar? We'd ideally like a stable base to build from before
committing to use a new framework, having been (re)writing/maintaining
charms for 4+ years now :)

2) Layer pinning. Right now, layers are evolving fast, and the lack of
pinning to layer versions has caused charm builds to break from day to
day. Is this a planned feature?

3) Downloading from the internet. This issue has been common in
charmstore charms, and is discouraged, AIUI. But the same issue
applies for layers, and possibly with more effect, due to a layer's
composibility.  We simply can not utilise any layer that downloads
things from github or similar, and I'm sure others are in a similar
situation.  We're aware of resources, but not convinced this is a
scalable solution for layers, as it makes using a charm that has
layers that require resources much more complex. So, some clarity in
this area would be helpful.

Thanks for all the work on making charming better.

-- 
Simon

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Automatic periodic upgrades as part of the base layer

[Stuart Bishop]

On 8 March 2016 at 05:24, Marco Ceppi  wrote:
> This is definitely more an operator decision than a charm decision. There
> are two existing charms to address this. An unattended-upgrades charm and
> landscape-client. Check those out first to see if the fit your needs.
>
> Marco

I've been toying with the idea of a package upgrade action in the apt
layer. It would unhold held packages if necessary, set state giving
your handlers the opportunity for pre/post upgrade hooks (like
shutting daemons down and restarting them), run apt-get dist-upgrade,
and rehold packages if necessary. This would require implementing
action support into charms.reactive, which has been discussed a bit on
github.

Landscape takes care of most day to day updates, but the most
important packages tend to get held to ensure unattended upgrades
don't take down the service.



> On Mon, Mar 7, 2016, 5:16 PM Mark Shuttleworth  wrote:
>>
>> On 07/03/16 13:29, Merlijn Sebrechts wrote:
>> > What is your experience with upgrades. Do they have a tendency to break
>> > things? Should this be enabled by default, added in as a configurable
>> > switch or not added at all?
>>
>> In 16.04, if unattended-upgrades is installed you will by default get
>> security updates automatically and can opt in to additional updates.
>> Common practice is just to turn them on, with some percentage of
>> machines also enabling the "proposed" pocket (where stuff goes before it
>> gets to the updates pocket). Machines with "proposed" act as canaries
>> for incoming updates. Security tends to land hard and fast because,
>> well, security, but then it gets a lot more QA and the changes are
>> generally tiny.
>>
>> Mark

-- 
Stuart Bishop 

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju