Re: Reactive roadmap

[Simon Davy]

On Mon, Mar 14, 2016 at 2:28 PM, Cory Johns  wrote:
> On Tue, Mar 8, 2016 at 9:19 AM, Simon Davy  wrote:
>>
>> 1) Is there a roadmap for reactive? A target for a stable 1.0 release,
>> or similar? We'd ideally like a stable base to build from before
>> committing to use a new framework, having been (re)writing/maintaining
>> charms for 4+ years now :)
>
>
> The layered & reactive approach saw adoption more quickly than I expected,
> which is both great and unfortunate for working out the kinks.  That said,
> the core concepts of both aspects of this approach have been fairly stable,
> with additions rather than breakages.  There is a 2.0 release of charm-tools
> coming, to coincide with Juju 2.0, but again, the build process should be
> backwards compatible.  There are some issues with charms.reactive that may
> require incompatible changes to fix, but that would be for 2.0 and the
> semantic version range that the base layer uses gives us flexibility there,
> though that leads in to your next point.

That's helpful to know, thanks.

>> 2) Layer pinning. Right now, layers are evolving fast, and the lack of
>> pinning to layer versions has caused charm builds to break from day to
>> day. Is this a planned feature?
>
>
> There has been quite a bit of discussion about this and I don't think either
> side has swayed the other.  On the one hand, as you note, there is a valid
> argument for needing to avoid breakage and in these early days, layers are
> evolving quickly.
>
> On the other hand, we want to strongly encourage charm authors to always
> sync the layers and interfaces they use to take advantage of the
> improvements and fixes from upstream, lest charms end up stagnating.  And
> that means encouraging backward-compatibility in the layers.  To that end,
> it has been suggested that layers be handled like interface protocols in
> that, if you need to make an incompatible change, you can fork the layer
> with a different name and both can coexist until one supplants the other.

That is a reasonable policy, feels like a similar discussion in the
other thread about upgrading elasticsearch/kibana charms. But, as a
policy alone, it's not enough to trust for backwards compatibility. We
have that policy in Ubuntu, particularly for SRUs and security
updates. But it's backed up with significant testing, and thus we
trust it not to break stuff (or rather, not to break stuff with a high
enough frequency to matter). Layers/interfaces, as of yet don't have
that assurance.

So, regards the trade-off between the two approaches, we would always
come down on the former approach. This is because our experience
across the board with all dependency management tools we use is that
pinning to versions is essential for reliability.  We pin our
dependent code branches, we pin our python packages, we pin our npm
packages, we pin our charms, we pin/hold key apt packages. Because if
we don't an upstream change can break the CD pipeline and then we
can't deploy.

This 'vendor/pin all the things your app depends on' approach is part
of the use case for snappy, and docker. Also, a comparison to golang
tooling seems apt. Go only needs build time dependencies too, and yet
the standard approach is to vendor all the things.

Yes, there's a downside, it's painful to keep things in sync. If you
want to 'strongly encourage' folks to keep up to date, some good
tooling around vendoring/syncing would be my recommendation..

I suspect the reality is that if you went along the lines of the
second argument, then people would just work round it anyway (like we
are doing right now).

A thought that's been brewing for a while: what if layers/interfaces
were available as python wheels, and interfaces.juju.solutions
provided an additional PyPI index? Then you could leverage the
existing tooling of pip and requirements.txt to provide your
dependency management?


> Additionally, as Stuart pointed out with tongue-in-cheek, the local repo can
> serve that purpose, and with charm-tools 2.0 that will become easier with
> the pull-source command (https://github.com/juju/charm-tools/pull/125).

OK, so a standard tool as a part of charm-tools to vendor/sync layers
would work for us I think.

>> 3) Downloading from the internet. This issue has been common in
>> charmstore charms, and is discouraged, AIUI. But the same issue
>> applies for layers, and possibly with more effect, due to a layer's
>> composibility.  We simply can not utilise any layer that downloads
>> things from github or similar, and I'm sure others are in a similar
>> situation.
>
>
> Again, as Stuart mentioned, this is actually *better* with layers than it
> has been in the past, because layers encourage charm dependencies to be
> bundled in the charm's wheelhouse,

Right, but that only works for python dependencies...

> and layers can provide more consistent
> adoption of best practices (it only needs to be done right once, instead of
> 

Re: Reactive roadmap

[Mark Shuttleworth]

On 14/03/16 13:55, Simon Davy wrote:
> Right, downloading at build time is a different problem. The issue is
> that the layer might do something on the install hook, for example,
> which downloads from the internet at run time, on the units. Such
> things work fine in dev or for demos, but will fail in many production
> environments. 

Resources, which are assets that can be published through the charm
store for charms (like a specific JVM tarball, or JAR file) are a
significant improvement in this.

Also, during the Juju 2.x timeline we will separate the "build" phase
from the "install" phase, enabling all the downloads to happen once only
and be shared across multiple units on clouds that support snapshots.

Mark

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Reactive roadmap

[Cory Johns]

On Tue, Mar 8, 2016 at 9:19 AM, Simon Davy  wrote:

> 1) Is there a roadmap for reactive? A target for a stable 1.0 release,
> or similar? We'd ideally like a stable base to build from before
> committing to use a new framework, having been (re)writing/maintaining
> charms for 4+ years now :)
>

The layered & reactive approach saw adoption more quickly than I expected,
which is both great and unfortunate for working out the kinks.  That said,
the core concepts of both aspects of this approach have been fairly stable,
with additions rather than breakages.  There is a 2.0 release of
charm-tools coming, to coincide with Juju 2.0, but again, the build process
should be backwards compatible.  There are some issues with charms.reactive
that may require incompatible changes to fix, but that would be for 2.0 and
the semantic version range that the base layer uses gives us flexibility
there, though that leads in to your next point.


> 2) Layer pinning. Right now, layers are evolving fast, and the lack of
> pinning to layer versions has caused charm builds to break from day to
> day. Is this a planned feature?
>

There has been quite a bit of discussion about this and I don't think
either side has swayed the other.  On the one hand, as you note, there is a
valid argument for needing to avoid breakage and in these early days,
layers are evolving quickly.

On the other hand, we want to strongly encourage charm authors to always
sync the layers and interfaces they use to take advantage of the
improvements and fixes from upstream, lest charms end up stagnating.  And
that means encouraging backward-compatibility in the layers.  To that end,
it has been suggested that layers be handled like interface protocols in
that, if you need to make an incompatible change, you can fork the layer
with a different name and both can coexist until one supplants the other.

Additionally, as Stuart pointed out with tongue-in-cheek, the local repo
can serve that purpose, and with charm-tools 2.0 that will become easier
with the pull-source command (https://github.com/juju/charm-tools/pull/125).


> 3) Downloading from the internet. This issue has been common in
> charmstore charms, and is discouraged, AIUI. But the same issue
> applies for layers, and possibly with more effect, due to a layer's
> composibility.  We simply can not utilise any layer that downloads
> things from github or similar, and I'm sure others are in a similar
> situation.


Again, as Stuart mentioned, this is actually *better* with layers than it
has been in the past, because layers encourage charm dependencies to be
bundled in the charm's wheelhouse, and layers can provide more consistent
adoption of best practices (it only needs to be done right once, instead of
every time).

(Note, though, that's there's an open issue with regards to some few Python
modules and network-restricted deployments:
https://github.com/juju/charm-tools/issues/117)


> We're aware of resources, but not convinced this is a
> scalable solution for layers, as it makes using a charm that has
> layers that require resources much more complex. So, some clarity in
> this area would be helpful.
>

I can't say that I understand why you think resources and layers would make
things more complicated.  They seem like they will work well together and
would solve the other half of charms in network-restricted environments
quite nicely.
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Reactive roadmap

[Simon Davy]

On Mon, Mar 14, 2016 at 1:05 PM, John Meinel  wrote:
> ...
>
>>
>>
>> > 3) Downloading from the internet. This issue has been common in
>> > charmstore charms, and is discouraged, AIUI. But the same issue
>> > applies for layers, and possibly with more effect, due to a layer's
>> > composibility.  We simply can not utilise any layer that downloads
>> > things from github or similar, and I'm sure others are in a similar
>> > situation.  We're aware of resources, but not convinced this is a
>> > scalable solution for layers, as it makes using a charm that has
>> > layers that require resources much more complex. So, some clarity in
>> > this area would be helpful.
>>
>> Yes, layers that do not work in network restricted environments are
>> not useful to many people. I think you will find layers will improve
>> things here. Layers only need to be written correctly once. And if
>> they are broken, only fixed once. A big improvement over cargo culted
>> code, where you could end up fixing essentially the same bug or adding
>> the same feature several times.
>>
>
> Layers themselves are a compile time thing, not a runtime thing, right? So
> while the code in the layer might say "download some resource from github",
> you the layer itself is only downloaded from github before it is published
> into the charm store. Am I understanding this wrong?
>
> John
> =:->

Right, downloading at build time is a different problem.

The issue is that the layer might do something on the install hook,
for example, which downloads from the internet at run time, on the
units.

Such things work fine in dev or for demos, but will fail in many
production environments.

-- 
Simon

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Reactive roadmap

[John Meinel]

...


>
> > 3) Downloading from the internet. This issue has been common in
> > charmstore charms, and is discouraged, AIUI. But the same issue
> > applies for layers, and possibly with more effect, due to a layer's
> > composibility.  We simply can not utilise any layer that downloads
> > things from github or similar, and I'm sure others are in a similar
> > situation.  We're aware of resources, but not convinced this is a
> > scalable solution for layers, as it makes using a charm that has
> > layers that require resources much more complex. So, some clarity in
> > this area would be helpful.
>
> Yes, layers that do not work in network restricted environments are
> not useful to many people. I think you will find layers will improve
> things here. Layers only need to be written correctly once. And if
> they are broken, only fixed once. A big improvement over cargo culted
> code, where you could end up fixing essentially the same bug or adding
> the same feature several times.
>
>
Layers themselves are a compile time thing, not a runtime thing, right? So
while the code in the layer might say "download some resource from github",
you the layer itself is only downloaded from github before it is published
into the charm store. Am I understanding this wrong?

John
=:->
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Reactive roadmap

[Stuart Bishop]

On 9 March 2016 at 01:19, Simon Davy  wrote:

> 2) Layer pinning. Right now, layers are evolving fast, and the lack of
> pinning to layer versions has caused charm builds to break from day to
> day. Is this a planned feature?

You can pin the layers in your build environment - they are only
unpinned if you are pulling them direct from the upstream branch. A
tool like codetree could be used to checkout your dependencies with
pinned versions allowing repeatable builds if you need that. Allowing
you to deal with the accumulated breakage and incompatibilities in one
huge indigestible chunk later, far too late to affect upstream changes
;)


> 3) Downloading from the internet. This issue has been common in
> charmstore charms, and is discouraged, AIUI. But the same issue
> applies for layers, and possibly with more effect, due to a layer's
> composibility.  We simply can not utilise any layer that downloads
> things from github or similar, and I'm sure others are in a similar
> situation.  We're aware of resources, but not convinced this is a
> scalable solution for layers, as it makes using a charm that has
> layers that require resources much more complex. So, some clarity in
> this area would be helpful.

Yes, layers that do not work in network restricted environments are
not useful to many people. I think you will find layers will improve
things here. Layers only need to be written correctly once. And if
they are broken, only fixed once. A big improvement over cargo culted
code, where you could end up fixing essentially the same bug or adding
the same feature several times.


-- 
Stuart Bishop 

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Reactive roadmap

[Casey Marshall]

Simon,
You raise some very interesting questions and good points.

I've also been developing charms with layers, run into some of these same
issues, and developed a pragmatic approach that has helped me manage it
pretty well so far. In a more mature layer/interface ecosystem, it might
not be the optimal approach. I look forward to adapting my tactics as the
ecosystem evolves.

On Tue, Mar 8, 2016 at 8:19 AM, Simon Davy  wrote:

> Hi all
>
> My team (Online Services at Canonical) maintains >20 private charms
> for our services, plus a few charmstore charms.
>
> Most of these charms are written with charmhelpers ansible support, or
> with the Services framework. We would like to move towards
> consolidating these approaches (as both have issues), and so have been
> experimenting with reactive.
>
> We like the ideas in reactive, especially the composition part, as
> sharing common code between our charms has been very painful. Also,
> the higher-level user-defined events that reactive provides is a
> definite improvement over having to implement the lower level relation
> dance semantics every time.
>
> However, it's a fast moving target, and we have encountered some
> issues.  So we have a couple of questions, that we haven't been able
> to locate answers for in reactive docs (we may have missed some?).
>
> 1) Is there a roadmap for reactive? A target for a stable 1.0 release,
> or similar? We'd ideally like a stable base to build from before
> committing to use a new framework, having been (re)writing/maintaining
> charms for 4+ years now :)
>

I too would like to see and help the ecosystem mature.


> 2) Layer pinning. Right now, layers are evolving fast, and the lack of
> pinning to layer versions has caused charm builds to break from day to
> day. Is this a planned feature?
>

For the time being, I've been avoiding upstream churn by vendoring all the
layers and interfaces that my deployment uses. These are all subdirectories
of a project that builds a bundle consisting of several charms, and also
contains supporting end-to-end test data and CI scripts. With the most
trivial of configuration changes, I can build, deploy and test these
services by compiling source from development branches, installing from
release tarballs or debian packages. This has been amazing for end-to-end
testing, demos of features in development, as well as complex feature
development where changes need to be coordinated across several services at
once.

Combined with the LXD provider, which is just amazingly fast on a xenial
host, this project makes me very confident about researching "what if"-type
scenarios about our deployment -- it doesn't take much at all to make minor
changes to a charm and just go try something. That could be a configuration
change, a different charm that provides the same interface, a feature
branch or set of feature branches for the services -- anything, really.

I manage the vendored layers and interfaces with "git subtree", which is
really the only way I'd even consider vendoring. Subtree allows me to pull
upstream layer & interface projects into mine, while retaining all the
upstream revision history, and retaining the ability to continue to
contribute to and pull from these upstream git repositories. Vendoring
without this capability would be fraught with peril, requiring too much
manual, tedious, error-prone work to work safely in my opinion.

An interface or layer that starts as a project subdirectory out of a
short-term need, can be easily refactored into something generally useful,
and then pushed to a separate git repo with a "git subtree push", which
then becomes the new upstream where I can collaborate with other charmers.
This is how I started the local-monitors and nrpe-external-master
interfaces. They began as an internal migration from an older charm example
(which was ansible-based). I published them with 'git subtree push' once I
felt they were suitable for general reuse.

I'll admit there are some disadvantages to the vendoring approach. Unless I
really pay attention to those upstreams, I might miss a useful bugfix or
security improvement. That's also a challenge with pinning btw (which is
effectively a cheap copy form of vendoring with references). It'd be nice
if I could rely on a branch name to be stable but provide bugfixes.

Completely vendoring upstreams with subtree could become unnecessary,
needless overhead, if stable upstreams were common practice. I'd still
probably want to pin upstream revisions though, just to have reproducible
builds.


> 3) Downloading from the internet. This issue has been common in
> charmstore charms, and is discouraged, AIUI. But the same issue
> applies for layers, and possibly with more effect, due to a layer's
> composibility.  We simply can not utilise any layer that downloads
> things from github or similar, and I'm sure others are in a similar
> situation.  We're aware of resources, but not convinced this is a
> 

Reactive roadmap

[Simon Davy]

Hi all

My team (Online Services at Canonical) maintains >20 private charms
for our services, plus a few charmstore charms.

Most of these charms are written with charmhelpers ansible support, or
with the Services framework. We would like to move towards
consolidating these approaches (as both have issues), and so have been
experimenting with reactive.

We like the ideas in reactive, especially the composition part, as
sharing common code between our charms has been very painful. Also,
the higher-level user-defined events that reactive provides is a
definite improvement over having to implement the lower level relation
dance semantics every time.

However, it's a fast moving target, and we have encountered some
issues.  So we have a couple of questions, that we haven't been able
to locate answers for in reactive docs (we may have missed some?).

1) Is there a roadmap for reactive? A target for a stable 1.0 release,
or similar? We'd ideally like a stable base to build from before
committing to use a new framework, having been (re)writing/maintaining
charms for 4+ years now :)

2) Layer pinning. Right now, layers are evolving fast, and the lack of
pinning to layer versions has caused charm builds to break from day to
day. Is this a planned feature?

3) Downloading from the internet. This issue has been common in
charmstore charms, and is discouraged, AIUI. But the same issue
applies for layers, and possibly with more effect, due to a layer's
composibility.  We simply can not utilise any layer that downloads
things from github or similar, and I'm sure others are in a similar
situation.  We're aware of resources, but not convinced this is a
scalable solution for layers, as it makes using a charm that has
layers that require resources much more complex. So, some clarity in
this area would be helpful.

Thanks for all the work on making charming better.

-- 
Simon

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju