Wouldn't it be possible for us to implement a configuration flag, and have it as default? Going back to the general point, the idea behind the ubuntu charm is to have a vainilla Ubuntu where you can work on anything. I understand we're mostly using it for testing, and reactive is now a big part of the ecosystem. I find two simple approaches for this:

* Create a ubuntu-vainilla charm which doesn't install any of the required packages OR * Implement a 'vainilla' boolean configuration flag, where you can specify True for the vainilla Ubuntu install, False to install all of the reactive/other dependencies.


If we get to work around the pyyaml issue and implement a better solution in the long term, I think that would be amazing. However, we can't let one dependency drag us down, and in the meanwhile, we have to implement a workaround.

On 12/01/2016 01:56 PM, Cory Johns wrote:
Marco,

What is the issue you mentioned with using snaps where you mentioned
needing an "unconfined classic snap"?

On Thu, Dec 1, 2016 at 1:13 PM, Marco Ceppi <marco.ce...@canonical.com
<mailto:marco.ce...@canonical.com>> wrote:

    On Thu, Dec 1, 2016 at 12:56 PM Casey Marshall
    <casey.marsh...@canonical.com <mailto:casey.marsh...@canonical.com>>
    wrote:

        On Thu, Dec 1, 2016 at 6:53 AM, Marco Ceppi
        <marco.ce...@canonical.com <mailto:marco.ce...@canonical.com>>
        wrote:

            On Thu, Dec 1, 2016 at 5:00 AM Adam Collard
            <adam.coll...@canonical.com
            <mailto:adam.coll...@canonical.com>> wrote:

                On Thu, 1 Dec 2016 at 04:02 Nate Finch
                <nate.fi...@canonical.com
                <mailto:nate.fi...@canonical.com>> wrote:

                    On IRC, someone was lamenting the fact that the
                    Ubuntu charm takes longer to deploy now, because it
                    has been updated to exercise more of Juju's
                    features.  My response was - just make a minimal
                    charm, it's easy.  And then of course, I had to
                    figure out how minimal you can get.  Here it is:

                    It's just a directory with a metadata.yaml in it
                    with these contents:

                    name: min
                    summary: nope
                    description: nope
                    series:
                      - xenial

                    (obviously you can set the series to whatever you want)
                    No other files or directories are needed.


                This is neat, but doesn't detract from the bloat in the
                ubuntu charm.


            I'm happy to work though changes to the Ubuntu charm to
            decrease "bloat".


                IMHO the bloat in the ubuntu charm isn't from support
                for Juju features, but the switch to reactive plus
                conflicts in layer-base wanting to a) support lots of
                toolchains to allow layers above it to be slimmer and b)
                be a suitable base for "just deploy me" ubuntu.


            But it is to support the reactive framework, where we
            utilize newer Juju features, like status and
            application-version to make the charm rich despite it's
            minimal goal set. Honestly, a handful of cached wheelhouses
            and some apt packages don't strike me as bloat, but I do
            want to make sure the Ubuntu charm works for those using it. So,


        I think a minimal wheelhouse to provide a consistent charm hook
        runtime is very reasonable and definitely not the problem here.

        There are too many packages that get installed by default with
        the reactive framework that most charms don't need. When I
        deploy the ubuntu charm (but this applies to any charm built
        with reactive and layer:basic), I also get:

        2016-12-01 17:45:47 INFO install The following NEW packages will
        be installed:
        2016-12-01 17:45:47 INFO install   binutils build-essential cpp
        cpp-5 dpkg-dev fakeroot g++ g++-5 gc
        c gcc-5
        2016-12-01 17:45:47 INFO install   libalgorithm-diff-perl
        libalgorithm-diff-xs-perl libalgorithm-mer
        ge-perl
        2016-12-01 17:45:47 INFO install   libasan2 libatomic1
        libc-dev-bin libc6-dev libcc1-0 libcilkrts5 l
        ibdpkg-perl
        2016-12-01 17:45:47 INFO install   libexpat1-dev libfakeroot
        libfile-fcntllock-perl libgcc-5-dev libgomp1
        2016-12-01 17:45:47 INFO install   libisl15 libitm1 liblsan0
        libmpc3 libmpx0 libpython3-dev libpython3.5-dev
        2016-12-01 17:45:47 INFO install   libquadmath0 libstdc++-5-dev
        libtsan0 libubsan0 linux-libc-dev make
        2016-12-01 17:45:47 INFO install   manpages-dev python-pip-whl
        python3-dev python3-pip python3-setuptools
        2016-12-01 17:45:47 INFO install   python3-wheel python3.5-dev

        None of my charms need build-essential or a g++ compiler, that's
        a lot of unnecessary dependencies! Can we get rid of most of
        these? Would installing the bare minimum with
        --no-install-recommends help?


    This comes up a bit, and I'm really eager to figure this out. Allow
    me to explain the catch-22. It's name is pyyaml.

    So the wheelhouse, by default, is 3.8M in size, this is the stock
    wheelhouse we vendor in:

    312K charmhelpers-0.10.0.tar.gz
    21K  charms.reactive-0.4.5.tar.gz
    349K Jinja2-2.8.tar.gz
    14K  MarkupSafe-0.23.tar.gz
    1.7M netaddr-0.7.18.tar.gz
    1.1M pip-8.1.2.tar.gz
    19K  pyaml-16.11.4.tar.gz
    248K PyYAML-3.12.tar.gz
    29K  six-1.10.0.tar.gz
    13K  Tempita-0.5.2.tar.gz

    The problem child is PyYAML, which is a compiled cpyhton module,
    which requires build-essential. The latest version is 3.12, trusty
    is 3.10, xenial 3.11, and zesty (finally)
    3.12 http://packages.ubuntu.com/search?keywords=python-yaml
    <http://packages.ubuntu.com/search?keywords=python-yaml>, CentOS 6 &
    7 are 3.10

    So, the easy path here is to just make sure charms.reactive works
    with PyYAML 3.10 and do a `--no-install-recommends` but that's half
    the problem. There will inevitably be python modules that require
    build-essential to compile on install.

    We can't cache the compiled wheelhouse because of architectures
    (same way resources complicate this) so we must compile at deploy time.

    One path forward, after dropping PyYAML 3.12 (if feasible), would be
    to see if we can detect when a python module needs to be compiled
    and setting a flag in the rendered charm to install the
    build-essentials, etc.

    I'll file issues and spend some time seeing if we can actually
    detect when you need to compile a wheelhouse vs a straight python
    module that and lowering the requirement for PyYAML (using packaged
    version instead) will probably remove a lot of the reactive
    bootstrap time.

    Marco

    --
    Juju-dev mailing list
    juju-...@lists.ubuntu.com <mailto:juju-...@lists.ubuntu.com>
    Modify settings or unsubscribe at:
    https://lists.ubuntu.com/mailman/listinfo/juju-dev
    <https://lists.ubuntu.com/mailman/listinfo/juju-dev>





--
José Antonio Rey

--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Reply via email to