Re: A (Very) Minimal Charm

[Free Ekanayaka]

On 1 December 2016 at 13:53, Marco Ceppi  wrote:

> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
> wrote:
>
>> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>>
>> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
>> to deploy now, because it has been updated to exercise more of Juju's
>> features.  My response was - just make a minimal charm, it's easy.  And
>> then of course, I had to figure out how minimal you can get.  Here it is:
>>
>> It's just a directory with a metadata.yaml in it with these contents:
>>
>> name: min
>> summary: nope
>> description: nope
>> series:
>>   - xenial
>>
>> (obviously you can set the series to whatever you want)
>> No other files or directories are needed.
>>
>>
>> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>>
>
> I'm happy to work though changes to the Ubuntu charm to decrease "bloat".
>
>
>> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
>> but the switch to reactive plus conflicts in layer-base wanting to a)
>> support lots of toolchains to allow layers above it to be slimmer and b) be
>> a suitable base for "just deploy me" ubuntu.
>>
>
> But it is to support the reactive framework, where we utilize newer Juju
> features, like status and application-version to make the charm rich
> despite it's minimal goal set.
>

Yeah, and I think this is a good thing.


> Honestly, a handful of cached wheelhouses and some apt packages don't
> strike me as bloat
>

No it's not per-se. However I think this highlights a more general issue
with the current implementation of the reactive stack. It's not only the
ubuntu charm that has slowed done, it's any reactive-based charm, because
the steps required to "setup" reactive take longer than they used to.

I see a couple of (possibly alternative) ways to improve the situation:

1) Make sure the dependencies of the base reactive layer are packaged, that
should be much faster than pip install, and fall back to pip only for
what's not there (i.e. dependencies added by the consumers of the base
layer). Also, package the base layer itself.

2) Add support for images, so when you deploy some vanilla charm there's an
associated "pre-built" image that will be very fast. I guess this is in the
juju road map anyways.

We always need to keep in mind that this experience will be compared with
things like Kubernetes and Docker, and speed-y deployments really unlock
velocity when iterating on charm development (think for instance running
integration tests).
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Free Ekanayaka]

On 1 December 2016 at 14:39, Marco Ceppi  wrote:

> On Thu, Dec 1, 2016 at 8:28 AM Free Ekanayaka <
> free.ekanay...@canonical.com> wrote:
>
>> On 1 December 2016 at 13:53, Marco Ceppi 
>> wrote:
>>
>> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
>> wrote:
>>
>> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>>
>> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
>> to deploy now, because it has been updated to exercise more of Juju's
>> features.  My response was - just make a minimal charm, it's easy.  And
>> then of course, I had to figure out how minimal you can get.  Here it is:
>>
>>
>> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>>
>>
>> I'm happy to work though changes to the Ubuntu charm to decrease "bloat".
>>
>>
>> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
>> but the switch to reactive plus conflicts in layer-base wanting to a)
>> support lots of toolchains to allow layers above it to be slimmer and b) be
>> a suitable base for "just deploy me" ubuntu.
>>
>>
>> But it is to support the reactive framework, where we utilize newer Juju
>> features, like status and application-version to make the charm rich
>> despite it's minimal goal set.
>>
>>
>> Yeah, and I think this is a good thing.
>>
>>
>> Honestly, a handful of cached wheelhouses and some apt packages don't
>> strike me as bloat
>>
>>
>> No it's not per-se. However I think this highlights a more general issue
>> with the current implementation of the reactive stack. It's not only the
>> ubuntu charm that has slowed done, it's any reactive-based charm, because
>> the steps required to "setup" reactive take longer than they used to.
>>
>
> The problem we're hitting with wheelhouses is exactly the one that snaps
> aim to solve:
>
> - apt packages are not cross distro, and we have reactive centos charms
> - apt packages lag a bit meaning we can't get consistent features even
> between trusty and xenial, let alone xenial and tip
>
> I see a couple of (possibly alternative) ways to improve the situation:
>>
>> 1) Make sure the dependencies of the base reactive layer are packaged,
>> that should be much faster than pip install, and fall back to pip only for
>> what's not there (i.e. dependencies added by the consumers of the base
>> layer). Also, package the base layer itself.
>>
>
> I'm very keen on a development in the snap world, where an unconfined
> "classic" style package would be available. This means we could snap up all
> the dependencies of the basic layer for every architecture and skip the
> setup phase for reactive. I think this is probably our best bet going
> forward.
>

Sounds good.
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Marco Ceppi]

On Thu, Dec 1, 2016 at 8:28 AM Free Ekanayaka 
wrote:

> On 1 December 2016 at 13:53, Marco Ceppi 
> wrote:
>
> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
> wrote:
>
> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>
> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
> to deploy now, because it has been updated to exercise more of Juju's
> features.  My response was - just make a minimal charm, it's easy.  And
> then of course, I had to figure out how minimal you can get.  Here it is:
>
>
> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>
>
> I'm happy to work though changes to the Ubuntu charm to decrease "bloat".
>
>
> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
> but the switch to reactive plus conflicts in layer-base wanting to a)
> support lots of toolchains to allow layers above it to be slimmer and b) be
> a suitable base for "just deploy me" ubuntu.
>
>
> But it is to support the reactive framework, where we utilize newer Juju
> features, like status and application-version to make the charm rich
> despite it's minimal goal set.
>
>
> Yeah, and I think this is a good thing.
>
>
> Honestly, a handful of cached wheelhouses and some apt packages don't
> strike me as bloat
>
>
> No it's not per-se. However I think this highlights a more general issue
> with the current implementation of the reactive stack. It's not only the
> ubuntu charm that has slowed done, it's any reactive-based charm, because
> the steps required to "setup" reactive take longer than they used to.
>

The problem we're hitting with wheelhouses is exactly the one that snaps
aim to solve:

- apt packages are not cross distro, and we have reactive centos charms
- apt packages lag a bit meaning we can't get consistent features even
between trusty and xenial, let alone xenial and tip

I see a couple of (possibly alternative) ways to improve the situation:
>
> 1) Make sure the dependencies of the base reactive layer are packaged,
> that should be much faster than pip install, and fall back to pip only for
> what's not there (i.e. dependencies added by the consumers of the base
> layer). Also, package the base layer itself.
>

I'm very keen on a development in the snap world, where an unconfined
"classic" style package would be available. This means we could snap up all
the dependencies of the basic layer for every architecture and skip the
setup phase for reactive. I think this is probably our best bet going
forward.


> 2) Add support for images, so when you deploy some vanilla charm there's
> an associated "pre-built" image that will be very fast. I guess this is in
> the juju road map anyways.
>

Sure, a build step is an interesting one, but it still incurs the same wait
for a setup, you just don't feel it as much.


> We always need to keep in mind that this experience will be compared with
> things like Kubernetes and Docker, and speed-y deployments really unlock
> velocity when iterating on charm development (think for instance running
> integration tests).
>

Speed is just one facet of the experience, though an important one. For the
speed of Kubernetes we win, hands down: 10 mins to deploy a full k8s
cluster (with Juju) is only really outpaced by SAAS. I know that's not the
point you were making, but it's the true speed comparison of what we're
doing.

That being said, we're very keen on making charm development, and
deployment, faster. Reactive 1.0 was a first leap in that direction, as we
round off work in 2.0 and leveraging other technologies like unconfined
snaps, we can start to speed up the bootstrap process of reactive charms.

I'll file bugs to track these items

Marco
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Aaron Bentley]

I hope that as you implement this, you avoid making fat charms.  Can you
use "resources" for this?

Aaron

On 2016-12-01 08:39 AM, Marco Ceppi wrote:
> On Thu, Dec 1, 2016 at 8:28 AM Free Ekanayaka
> > wrote:
> 
> On 1 December 2016 at 13:53, Marco Ceppi  > wrote:
> 
> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard
> >
> wrote:
> 
> On Thu, 1 Dec 2016 at 04:02 Nate Finch
> >
> wrote:
> 
> On IRC, someone was lamenting the fact that the Ubuntu
> charm takes longer to deploy now, because it has been
> updated to exercise more of Juju's features.  My
> response was - just make a minimal charm, it's easy. 
> And then of course, I had to figure out how minimal you
> can get.  Here it is:
> 
> 
> This is neat, but doesn't detract from the bloat in the
> ubuntu charm.
> 
> 
> I'm happy to work though changes to the Ubuntu charm to decrease
> "bloat".
>  
> 
> IMHO the bloat in the ubuntu charm isn't from support for
> Juju features, but the switch to reactive plus conflicts in
> layer-base wanting to a) support lots of toolchains to allow
> layers above it to be slimmer and b) be a suitable base for
> "just deploy me" ubuntu.
> 
> 
> But it is to support the reactive framework, where we utilize
> newer Juju features, like status and application-version to make
> the charm rich despite it's minimal goal set.
> 
> 
> Yeah, and I think this is a good thing.
>  
> 
> Honestly, a handful of cached wheelhouses and some apt packages
> don't strike me as bloat
> 
> 
> No it's not per-se. However I think this highlights a more general
> issue with the current implementation of the reactive stack. It's
> not only the ubuntu charm that has slowed done, it's any
> reactive-based charm, because the steps required to "setup" reactive
> take longer than they used to.
> 
> 
> The problem we're hitting with wheelhouses is exactly the one that snaps
> aim to solve:
> 
> - apt packages are not cross distro, and we have reactive centos charms
> - apt packages lag a bit meaning we can't get consistent features even
> between trusty and xenial, let alone xenial and tip
> 
> I see a couple of (possibly alternative) ways to improve the situation:
> 
> 1) Make sure the dependencies of the base reactive layer are
> packaged, that should be much faster than pip install, and fall back
> to pip only for what's not there (i.e. dependencies added by the
> consumers of the base layer). Also, package the base layer itself.
> 
> 
> I'm very keen on a development in the snap world, where an unconfined
> "classic" style package would be available. This means we could snap up
> all the dependencies of the basic layer for every architecture and skip
> the setup phase for reactive. I think this is probably our best bet
> going forward.
>  
> 
> 2) Add support for images, so when you deploy some vanilla charm
> there's an associated "pre-built" image that will be very fast. I
> guess this is in the juju road map anyways.
> 
> 
> Sure, a build step is an interesting one, but it still incurs the same
> wait for a setup, you just don't feel it as much.
>  
> 
> We always need to keep in mind that this experience will be compared
> with things like Kubernetes and Docker, and speed-y deployments
> really unlock velocity when iterating on charm development (think
> for instance running integration tests).
> 
> 
> Speed is just one facet of the experience, though an important one. For
> the speed of Kubernetes we win, hands down: 10 mins to deploy a full k8s
> cluster (with Juju) is only really outpaced by SAAS. I know that's not
> the point you were making, but it's the true speed comparison of what
> we're doing.
> 
> That being said, we're very keen on making charm development, and
> deployment, faster. Reactive 1.0 was a first leap in that direction, as
> we round off work in 2.0 and leveraging other technologies like
> unconfined snaps, we can start to speed up the bootstrap process of
> reactive charms.
> 
> I'll file bugs to track these items
> 
> Marco
> 
> 



signature.asc
Description: OpenPGP digital signature
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Casey Marshall]

On Thu, Dec 1, 2016 at 6:53 AM, Marco Ceppi 
wrote:

> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
> wrote:
>
>> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>>
>> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
>> to deploy now, because it has been updated to exercise more of Juju's
>> features.  My response was - just make a minimal charm, it's easy.  And
>> then of course, I had to figure out how minimal you can get.  Here it is:
>>
>> It's just a directory with a metadata.yaml in it with these contents:
>>
>> name: min
>> summary: nope
>> description: nope
>> series:
>>   - xenial
>>
>> (obviously you can set the series to whatever you want)
>> No other files or directories are needed.
>>
>>
>> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>>
>
> I'm happy to work though changes to the Ubuntu charm to decrease "bloat".
>
>
>> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
>> but the switch to reactive plus conflicts in layer-base wanting to a)
>> support lots of toolchains to allow layers above it to be slimmer and b) be
>> a suitable base for "just deploy me" ubuntu.
>>
>
> But it is to support the reactive framework, where we utilize newer Juju
> features, like status and application-version to make the charm rich
> despite it's minimal goal set. Honestly, a handful of cached wheelhouses
> and some apt packages don't strike me as bloat, but I do want to make sure
> the Ubuntu charm works for those using it. So,
>

I think a minimal wheelhouse to provide a consistent charm hook runtime is
very reasonable and definitely not the problem here.

There are too many packages that get installed by default with the reactive
framework that most charms don't need. When I deploy the ubuntu charm (but
this applies to any charm built with reactive and layer:basic), I also get:

2016-12-01 17:45:47 INFO install The following NEW packages will be
installed:
2016-12-01 17:45:47 INFO install   binutils build-essential cpp cpp-5
dpkg-dev fakeroot g++ g++-5 gc
c gcc-5
2016-12-01 17:45:47 INFO install   libalgorithm-diff-perl
libalgorithm-diff-xs-perl libalgorithm-mer
ge-perl
2016-12-01 17:45:47 INFO install   libasan2 libatomic1 libc-dev-bin
libc6-dev libcc1-0 libcilkrts5 l
ibdpkg-perl
2016-12-01 17:45:47 INFO install   libexpat1-dev libfakeroot
libfile-fcntllock-perl libgcc-5-dev libgomp1
2016-12-01 17:45:47 INFO install   libisl15 libitm1 liblsan0 libmpc3
libmpx0 libpython3-dev libpython3.5-dev
2016-12-01 17:45:47 INFO install   libquadmath0 libstdc++-5-dev libtsan0
libubsan0 linux-libc-dev make
2016-12-01 17:45:47 INFO install   manpages-dev python-pip-whl python3-dev
python3-pip python3-setuptools
2016-12-01 17:45:47 INFO install   python3-wheel python3.5-dev

None of my charms need build-essential or a g++ compiler, that's a lot of
unnecessary dependencies! Can we get rid of most of these? Would installing
the bare minimum with --no-install-recommends help?


> What's the real problem with the Ubuntu charm today?
> How does it not achieve it's goal of providing a relatively blank Ubuntu
> machine? What are people using the Ubuntu charm for?
>
> Other than demos, hacks/workarounds, and testing I'm not clear on the
> purpose of an Ubuntu charm in a model serves.
>
> Marco
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju-dev
>
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Bundle SVG Generator updated

[Marco Ceppi]

A quick update: You can now put in a bundle URL from the charm store in
addition to a raw HTTP address to get an SVG/PNG/PDF/XML render.

Examples on the homepage have been updated, they're repeated here for your
convenience:

http://svg.juju.solutions/?bundle=openstack-telemetry
http://svg.juju.solutions/?bundle=cs:~bigdata-dev/bundle/apache-analytics-sql-15=png

Thanks,
Marco Ceppi

On Thu, Dec 1, 2016 at 9:48 AM James Donner 
wrote:

> This is a godsend.
>
> It used to be incredibly difficult to gather images of Juju bundles, but
> this tool removes so many steps from the process I used to have to take.
>
> Awesome work Marco!
>
> On Wed, Nov 30, 2016 at 9:37 PM, Marco Ceppi 
> wrote:
>
> Hey everyone,
>
> A while ago I built a small web app that you could feed raw bundle.yaml
> files to and get back SVG images of the deployment like you would in the
> Charm Store or Juju GUI. This was built for things like Juju Benchmarking
> which shows an image representing the Juju deployment under test. Now it's
> become a resource for those building slides, presentations, blog posts, and
> readme files. However, SVG is not the most approachable format for some of
> these new use cases.
>
> As a result of this, I've added a raster converter as part of the
> application making bundle.yaml and transform them into transparent PNGs,
> PDF, or XML. The service will continue to default to SVG.
>
> Here's a few live examples:
>
> PNG: svg.juju.solutions/?bundle-file=https://[snip]/bundle.yaml=png
> 
> PDF: svg.juju.solutions/?bundle-file=https://[snip]/bundle.yaml=pdf
> 
>
> The landing page has been updated to include a quick submission form, no
> more need to hack URLs manually. https://svg.juju.solutions
>
> Finally, this is an open source project with an accompanying charm:
>
> * https://github.com/marcoceppi/svg.juju.solutions
> * https://github.com/marcoceppi/layer-charmsvg
> * https://jujucharms.com/u/marcoceppi/charm-svg
>
> The site is, of course, deployed and managed by Juju.
>
> Thanks,
> Marco Ceppi
>
> --
> Juju mailing list
> j...@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Cory Johns]

Marco,

What is the issue you mentioned with using snaps where you mentioned
needing an "unconfined classic snap"?

On Thu, Dec 1, 2016 at 1:13 PM, Marco Ceppi 
wrote:

> On Thu, Dec 1, 2016 at 12:56 PM Casey Marshall <
> casey.marsh...@canonical.com> wrote:
>
> On Thu, Dec 1, 2016 at 6:53 AM, Marco Ceppi 
> wrote:
>
> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
> wrote:
>
> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>
> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
> to deploy now, because it has been updated to exercise more of Juju's
> features.  My response was - just make a minimal charm, it's easy.  And
> then of course, I had to figure out how minimal you can get.  Here it is:
>
> It's just a directory with a metadata.yaml in it with these contents:
>
> name: min
> summary: nope
> description: nope
> series:
>   - xenial
>
> (obviously you can set the series to whatever you want)
> No other files or directories are needed.
>
>
> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>
>
> I'm happy to work though changes to the Ubuntu charm to decrease "bloat".
>
>
> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
> but the switch to reactive plus conflicts in layer-base wanting to a)
> support lots of toolchains to allow layers above it to be slimmer and b) be
> a suitable base for "just deploy me" ubuntu.
>
>
> But it is to support the reactive framework, where we utilize newer Juju
> features, like status and application-version to make the charm rich
> despite it's minimal goal set. Honestly, a handful of cached wheelhouses
> and some apt packages don't strike me as bloat, but I do want to make sure
> the Ubuntu charm works for those using it. So,
>
>
> I think a minimal wheelhouse to provide a consistent charm hook runtime is
> very reasonable and definitely not the problem here.
>
> There are too many packages that get installed by default with the
> reactive framework that most charms don't need. When I deploy the ubuntu
> charm (but this applies to any charm built with reactive and layer:basic),
> I also get:
>
> 2016-12-01 17:45:47 INFO install The following NEW packages will be
> installed:
> 2016-12-01 17:45:47 INFO install   binutils build-essential cpp cpp-5
> dpkg-dev fakeroot g++ g++-5 gc
> c gcc-5
> 2016-12-01 17:45:47 INFO install   libalgorithm-diff-perl
> libalgorithm-diff-xs-perl libalgorithm-mer
> ge-perl
> 2016-12-01 17:45:47 INFO install   libasan2 libatomic1 libc-dev-bin
> libc6-dev libcc1-0 libcilkrts5 l
> ibdpkg-perl
> 2016-12-01 17:45:47 INFO install   libexpat1-dev libfakeroot
> libfile-fcntllock-perl libgcc-5-dev libgomp1
> 2016-12-01 17:45:47 INFO install   libisl15 libitm1 liblsan0 libmpc3
> libmpx0 libpython3-dev libpython3.5-dev
> 2016-12-01 17:45:47 INFO install   libquadmath0 libstdc++-5-dev libtsan0
> libubsan0 linux-libc-dev make
> 2016-12-01 17:45:47 INFO install   manpages-dev python-pip-whl python3-dev
> python3-pip python3-setuptools
> 2016-12-01 17:45:47 INFO install   python3-wheel python3.5-dev
>
> None of my charms need build-essential or a g++ compiler, that's a lot of
> unnecessary dependencies! Can we get rid of most of these? Would installing
> the bare minimum with --no-install-recommends help?
>
>
> This comes up a bit, and I'm really eager to figure this out. Allow me to
> explain the catch-22. It's name is pyyaml.
>
> So the wheelhouse, by default, is 3.8M in size, this is the stock
> wheelhouse we vendor in:
>
> 312K charmhelpers-0.10.0.tar.gz
> 21K  charms.reactive-0.4.5.tar.gz
> 349K Jinja2-2.8.tar.gz
> 14K  MarkupSafe-0.23.tar.gz
> 1.7M netaddr-0.7.18.tar.gz
> 1.1M pip-8.1.2.tar.gz
> 19K  pyaml-16.11.4.tar.gz
> 248K PyYAML-3.12.tar.gz
> 29K  six-1.10.0.tar.gz
> 13K  Tempita-0.5.2.tar.gz
>
> The problem child is PyYAML, which is a compiled cpyhton module, which
> requires build-essential. The latest version is 3.12, trusty is 3.10,
> xenial 3.11, and zesty (finally) 3.12 http://packages.ubuntu.
> com/search?keywords=python-yaml, CentOS 6 & 7 are 3.10
>
> So, the easy path here is to just make sure charms.reactive works with
> PyYAML 3.10 and do a `--no-install-recommends` but that's half the problem.
> There will inevitably be python modules that require build-essential to
> compile on install.
>
> We can't cache the compiled wheelhouse because of architectures (same way
> resources complicate this) so we must compile at deploy time.
>
> One path forward, after dropping PyYAML 3.12 (if feasible), would be to
> see if we can detect when a python module needs to be compiled and setting
> a flag in the rendered charm to install the build-essentials, etc.
>
> I'll file issues and spend some time seeing if we can actually detect when
> you need to compile a wheelhouse vs a straight python module that and
> lowering the requirement for PyYAML (using packaged version 

Fwd: A (Very) Minimal Charm

[Adam Collard]

Oops, forgot to reply to the list.

-- Forwarded message -
From: Adam Collard 
Date: Thu, 1 Dec 2016 at 13:43
Subject: Re: A (Very) Minimal Charm
To: Marco Ceppi 


On Thu, 1 Dec 2016 at 12:53 Marco Ceppi  wrote:

On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
wrote:

On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:

On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
to deploy now, because it has been updated to exercise more of Juju's
features.  My response was - just make a minimal charm, it's easy.  And
then of course, I had to figure out how minimal you can get.  Here it is:

It's just a directory with a metadata.yaml in it with these contents:

name: min
summary: nope
description: nope
series:
  - xenial

(obviously you can set the series to whatever you want)
No other files or directories are needed.


This is neat, but doesn't detract from the bloat in the ubuntu charm.


I'm happy to work though changes to the Ubuntu charm to decrease "bloat".


Great!




IMHO the bloat in the ubuntu charm isn't from support for Juju features,
but the switch to reactive plus conflicts in layer-base wanting to a)
support lots of toolchains to allow layers above it to be slimmer and b) be
a suitable base for "just deploy me" ubuntu.


But it is to support the reactive framework, where we utilize newer Juju
features, like status and application-version to make the charm rich
despite it's minimal goal set. Honestly, a handful of cached wheelhouses
and some apt packages don't strike me as bloat, but I do want to make sure
the Ubuntu charm works for those using it. So,

What's the real problem with the Ubuntu charm today?


It apt-get installs something
It pip installs something

Both of these take non-trivial amounts of time.

How does it not achieve it's goal of providing a relatively blank Ubuntu
machine?


It does, but an additional goal (IMO) is to do so quickly and with minimal
(zero?) network requirements.


What are people using the Ubuntu charm for?


Demonstrating/finding Juju bugs whilst ruling out lots of other potential
sources of confusion.


Other than demos, hacks/workarounds, and testing I'm not clear on the
purpose of an Ubuntu charm in a model serves.


Hope this sheds some light on what we were using the Ubuntu charm for.


Marco
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Nate Finch]

So, the whole point of my original email was to say - if you want a minimal
charm, just make one, it's easy.
I just published my above mentioned minimal charm as cs:~natefinch/nop

It's not showing up on jujucharms.com, maybe because charm proof is failing
because it's missing everything except the metadata?  I don't know, but it
still works with juju deploy.

On Thu, Dec 1, 2016 at 10:07 PM José Antonio Rey  wrote:

> Wouldn't it be possible for us to implement a configuration flag, and
> have it as default? Going back to the general point, the idea behind the
> ubuntu charm is to have a vainilla Ubuntu where you can work on
> anything. I understand we're mostly using it for testing, and reactive
> is now a big part of the ecosystem. I find two simple approaches for this:
>
>   * Create a ubuntu-vainilla charm which doesn't install any of the
> required packages OR
>   * Implement a 'vainilla' boolean configuration flag, where you can
> specify True for the vainilla Ubuntu install, False to install all of
> the reactive/other dependencies.
>
> If we get to work around the pyyaml issue and implement a better
> solution in the long term, I think that would be amazing. However, we
> can't let one dependency drag us down, and in the meanwhile, we have to
> implement a workaround.
>
> On 12/01/2016 01:56 PM, Cory Johns wrote:
> > Marco,
> >
> > What is the issue you mentioned with using snaps where you mentioned
> > needing an "unconfined classic snap"?
> >
> > On Thu, Dec 1, 2016 at 1:13 PM, Marco Ceppi  > > wrote:
> >
> > On Thu, Dec 1, 2016 at 12:56 PM Casey Marshall
> > >
> > wrote:
> >
> > On Thu, Dec 1, 2016 at 6:53 AM, Marco Ceppi
> > >
> > wrote:
> >
> > On Thu, Dec 1, 2016 at 5:00 AM Adam Collard
> >  > > wrote:
> >
> > On Thu, 1 Dec 2016 at 04:02 Nate Finch
> >  > > wrote:
> >
> > On IRC, someone was lamenting the fact that the
> > Ubuntu charm takes longer to deploy now, because it
> > has been updated to exercise more of Juju's
> > features.  My response was - just make a minimal
> > charm, it's easy.  And then of course, I had to
> > figure out how minimal you can get.  Here it is:
> >
> > It's just a directory with a metadata.yaml in it
> > with these contents:
> >
> > name: min
> > summary: nope
> > description: nope
> > series:
> >   - xenial
> >
> > (obviously you can set the series to whatever you
> want)
> > No other files or directories are needed.
> >
> >
> > This is neat, but doesn't detract from the bloat in the
> > ubuntu charm.
> >
> >
> > I'm happy to work though changes to the Ubuntu charm to
> > decrease "bloat".
> >
> >
> > IMHO the bloat in the ubuntu charm isn't from support
> > for Juju features, but the switch to reactive plus
> > conflicts in layer-base wanting to a) support lots of
> > toolchains to allow layers above it to be slimmer and b)
> > be a suitable base for "just deploy me" ubuntu.
> >
> >
> > But it is to support the reactive framework, where we
> > utilize newer Juju features, like status and
> > application-version to make the charm rich despite it's
> > minimal goal set. Honestly, a handful of cached wheelhouses
> > and some apt packages don't strike me as bloat, but I do
> > want to make sure the Ubuntu charm works for those using it.
> So,
> >
> >
> > I think a minimal wheelhouse to provide a consistent charm hook
> > runtime is very reasonable and definitely not the problem here.
> >
> > There are too many packages that get installed by default with
> > the reactive framework that most charms don't need. When I
> > deploy the ubuntu charm (but this applies to any charm built
> > with reactive and layer:basic), I also get:
> >
> > 2016-12-01 17:45:47 INFO install The following NEW packages will
> > be installed:
> > 2016-12-01 17:45:47 INFO install   binutils build-essential cpp
> > cpp-5 dpkg-dev fakeroot g++ g++-5 gc
> > c gcc-5
> > 2016-12-01 17:45:47 INFO install   libalgorithm-diff-perl
> > 

Re: A (Very) Minimal Charm

[Adam Collard]

On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:

On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
to deploy now, because it has been updated to exercise more of Juju's
features.  My response was - just make a minimal charm, it's easy.  And
then of course, I had to figure out how minimal you can get.  Here it is:

It's just a directory with a metadata.yaml in it with these contents:

name: min
summary: nope
description: nope
series:
  - xenial

(obviously you can set the series to whatever you want)
No other files or directories are needed.


This is neat, but doesn't detract from the bloat in the ubuntu charm.

IMHO the bloat in the ubuntu charm isn't from support for Juju features,
but the switch to reactive plus conflicts in layer-base wanting to a)
support lots of toolchains to allow layers above it to be slimmer and b) be
a suitable base for "just deploy me" ubuntu.
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Marco Ceppi]

On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
wrote:

> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>
> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
> to deploy now, because it has been updated to exercise more of Juju's
> features.  My response was - just make a minimal charm, it's easy.  And
> then of course, I had to figure out how minimal you can get.  Here it is:
>
> It's just a directory with a metadata.yaml in it with these contents:
>
> name: min
> summary: nope
> description: nope
> series:
>   - xenial
>
> (obviously you can set the series to whatever you want)
> No other files or directories are needed.
>
>
> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>

I'm happy to work though changes to the Ubuntu charm to decrease "bloat".


> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
> but the switch to reactive plus conflicts in layer-base wanting to a)
> support lots of toolchains to allow layers above it to be slimmer and b) be
> a suitable base for "just deploy me" ubuntu.
>

But it is to support the reactive framework, where we utilize newer Juju
features, like status and application-version to make the charm rich
despite it's minimal goal set. Honestly, a handful of cached wheelhouses
and some apt packages don't strike me as bloat, but I do want to make sure
the Ubuntu charm works for those using it. So,

What's the real problem with the Ubuntu charm today?
How does it not achieve it's goal of providing a relatively blank Ubuntu
machine? What are people using the Ubuntu charm for?

Other than demos, hacks/workarounds, and testing I'm not clear on the
purpose of an Ubuntu charm in a model serves.

Marco
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Stuart Bishop]

On 1 December 2016 at 19:53, Marco Ceppi  wrote:

> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
> wrote:
>
>> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>>
>> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
>> to deploy now, because it has been updated to exercise more of Juju's
>> features.  My response was - just make a minimal charm, it's easy.  And
>> then of course, I had to figure out how minimal you can get.  Here it is:
>>
>> It's just a directory with a metadata.yaml in it with these contents:
>>
>> name: min
>> summary: nope
>> description: nope
>> series:
>>   - xenial
>>
>> (obviously you can set the series to whatever you want)
>> No other files or directories are needed.
>>
>>
>> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>>
>
> I'm happy to work though changes to the Ubuntu charm to decrease "bloat".
>
>
>> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
>> but the switch to reactive plus conflicts in layer-base wanting to a)
>> support lots of toolchains to allow layers above it to be slimmer and b) be
>> a suitable base for "just deploy me" ubuntu.
>>
>
> But it is to support the reactive framework, where we utilize newer Juju
> features, like status and application-version to make the charm rich
> despite it's minimal goal set. Honestly, a handful of cached wheelhouses
> and some apt packages don't strike me as bloat, but I do want to make sure
> the Ubuntu charm works for those using it. So,
>
> What's the real problem with the Ubuntu charm today?
> How does it not achieve it's goal of providing a relatively blank Ubuntu
> machine? What are people using the Ubuntu charm for?
>
> Other than demos, hacks/workarounds, and testing I'm not clear on the
> purpose of an Ubuntu charm in a model serves.
>

The cs:ubuntu charm gets used on production to attach subordinates too. For
example, we install cs:ubuntu onto our controller nodes so we can install
subordinates like cs:ntp, cs:nrpe, cs:~telegraf-chamers/telegraf and
others. Its also used in test suites for these sort of subordinates.

The 'problem' is, like all reactive charms, the first thing it does is pull
down approximately 160MB of packages and installs them (installing pip
pulls in build-essentials, or at least a big chunk of it). Its very
noticeable when working locally, and maybe in CI environments.

If I knew how to solve this for all reactive charms, I would have suggested
it already. It could be fixed in cs:ubuntu by making it non-reactive, if
people think it is worth it (its not like it actually needs any reactive
features. A minimal metadata.yaml and an install or start hook to set the
status is all it needs).

Maybe reactive is entrenched enough as the new world order that we can get
specific cloud images spun for it, where a pile of packages are
preinstalled so we don't need to wait for cloud-init or the charm to
install them. We might be able to lower deployment times from minutes to
seconds, since often this step is the main time sink.

-- 
Stuart Bishop 
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev