Re: [ Docs ] - Charming with Docker

[Kapil Thangavelu]

On Tue, Sep 15, 2015 at 9:25 AM, Charles Butler <
charles.but...@canonical.com> wrote:

> I see your concerns about layering, reactive, et-al. for introductory/new
> charmers. I however raise you a value proposition, and will speak to
> the point we are generating the 'best practice' method, that we want to
> guide charmers down. The common feedback we've received from
> authors is "I'm not as concerned with the overhead of learning, I just
> want you to tell me the best way to do it." While this represents a small
> number from the overall community of people charming, we've delivered on
> this asked behavior. We're taking away some maintenance
> burdens, and exposing a scaffolded framework to deliver your app
> containerized workload.
>

That's fair, it feels pretty good as an intro to best practices guide for
writing reactive style python charms working in an ecosystem with reuse and
extensibility.


>
> While its arguably more complex than `curl get.docker.com | sh` - there
> are a few things happening here that should concern you already.
>
> - You're not verifying the payload, and blindly executing it via a shell.
> This violates charm store policy, and is a black hole into whats happening
> without tailing the unit logs.
>

> While the script is a replica of that installation method, (credit is
> given in the layer repository readme) it was deemed important to mimic
> their recommended install - when you look at the ansible based docker
> charm and its divergent history + issue log.
>
> The script has been modified to surface what the unit is doing via the
> newer `juju status-set` routines  for clear visibility. And we get the
> added benefit of supporting  cent/debian out of the box with this method.
> The ownership burden is on  us, the ~charmers that are maintaining
> the docker layer.
>

> I appreciate the feedback though Kapil, and thanks for acknowledging the
> raise in knowledge to 'get started' - you're right its not as straight
> forward as curling a bash script and piping to sudo. But we don't
> encourage that anyway :) We feel that you can learn and get productive with
> this
> fairly quickly, with minimal knowledge of whats actually happening layers
> below. Just look at what states become exposed and take action.
>
>
i think you misunderstood the download analogy, apt-get install docker is
equivalent in this context or downloading over ssl with checksum and cert
verification and pipe into shell. the concern and question was the best way
to introduce folks to using juju with docker when the alternative is two
lines of shell script.  ie. the comparison for the nginx example an install
hook.

apt-get install docker.io

and a start hook

docker run -p 80:80 dockerfile/nginx -v  html_dir:/var/www/html



> Give it a spin and let us know if you've changed your opinion at all, and
> I'll be happy to talk/work through the concerns.
>


as far as adoption goes, an easy on ramp up the learning curve matters.
docker, terraform, packer, ansible, all great examples of that, and for
onboarding people into writing charms that utilize docker i think that's
its important as well to show the simplest thing that works. as a framework
or layer for docker charms, i'd at least expect it handles some set of the
common concerns germane to using docker across multiple hosts, swarm,
networking, image gc, container gc, registry config, proxies, etc. as it is
the docker example isn't really about docker, its an intro to reactive
style python charm dev, but a docker guide of its own has value. if the
target audience is those working on integration with an ecosystem of charms
with lots of optional integrations and relations (openstack, bigdata,
container eco) then the reactive style scales in complexity well but it
doesn't really address atm anything wrt to docker usage concerns.

cheers,

-kapil



>
> - Charles
>
>
>
>
> Charles Butler  - Juju Charmer
> Come see the future of datacenter orchestration: http://jujucharms.com
>
> On Tue, Sep 15, 2015 at 7:46 AM, Kapil Thangavelu 
> wrote:
>
>> i think participating in the burgeoning docker ecosystem is a worthwhile
>> goal by making it easier to write charms that utilize docker. I do have
>> some concerns though about the complexity of the layering that's taking
>> place in the charm ecosystem. I've found that juju has been fairly hard to
>> teach and adapt to real world usage and the layering of frameworks at the
>> charm authoring level makes learning and productivity for new users even
>> harder to achieve (ie those docs reference, charm helpers, reactive charm
>> helpers, docker reactive 'layer', charm composition), all of which are
>> fairly advanced concepts to a new user, and frankly is all of that
>> nescessary to understand running a shell script
>> https://github.com/juju-solutions/layer-docker/blob/master/scripts/install_docker.sh
>> which is effectively the jujuized version of (curl get.docker.com | sh)
>> .. now say i want to configure an insecure regis

Re: [ANN] Amulet 1.11.1 released

[Marco Ceppi]

As a follow up we've just released 1.11.2 due to an issue in the initial
python packaging. There is no difference between 1.11.1 and 1.11.2.

Thanks,
Marco Ceppi

On Tue, Sep 15, 2015 at 12:57 PM Marco Ceppi 
wrote:

> Hello,
>
> I'm happy to announce the Amulet 1.11.1 release. You can install and
> upgrade amulet from the juju stable ppa:
>
> sudo add-apt-repository ppa:juju/stable
> sudo apt-get update
> sudo apt-get install amulet
>
> For all other users, amulet is available in pip
>
> pip install amulet
>
> # Highlights
>
> Some highlights in this patch release
>
> - Support for v4 bundle format
> - Changed sentry commands to use ssh instead of run
> - Remove cli module and entry point
>
> See the full log below for all changes
>
> # Release notes
>
> These are the commits included in this release:
>
> c545b09 Use services instead of series, and support series being left out
> of bundle
> 5a68375 Support v4 format bundles
> b6da1c3 Changed sentry commands to use ssh instead of run to avoid
> blocking with hooks
> a11573e Don't configure logging in the library code
> e4e13c4 Remove cli module and entry point
>
> Special thanks to all contributors for this release:
>
> - Tim Van Steenburgh
> - Cory Johns
> - Skyler Berg
>
> For any questions and support you can mail this list (
> juju@lists.ubuntu.com), find us on freenode in #juju, or
> http://askubuntu.com with questions tagged amulet. Any issues or bugs can
> be reported against the launchpad project: https://launchpad.net/amulet
>
> Thanks,
> Marco Ceppi
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

[ANN] Amulet 1.11.1 released

[Marco Ceppi]

Hello,

I'm happy to announce the Amulet 1.11.1 release. You can install and
upgrade amulet from the juju stable ppa:

sudo add-apt-repository ppa:juju/stable
sudo apt-get update
sudo apt-get install amulet

For all other users, amulet is available in pip

pip install amulet

# Highlights

Some highlights in this patch release

- Support for v4 bundle format
- Changed sentry commands to use ssh instead of run
- Remove cli module and entry point

See the full log below for all changes

# Release notes

These are the commits included in this release:

c545b09 Use services instead of series, and support series being left out
of bundle
5a68375 Support v4 format bundles
b6da1c3 Changed sentry commands to use ssh instead of run to avoid blocking
with hooks
a11573e Don't configure logging in the library code
e4e13c4 Remove cli module and entry point

Special thanks to all contributors for this release:

- Tim Van Steenburgh
- Cory Johns
- Skyler Berg

For any questions and support you can mail this list (juju@lists.ubuntu.com),
find us on freenode in #juju, or http://askubuntu.com with questions tagged
amulet. Any issues or bugs can be reported against the launchpad project:
https://launchpad.net/amulet

Thanks,
Marco Ceppi
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: [ Docs ] - Charming with Docker

[Charles Butler]

I see your concerns about layering, reactive, et-al. for introductory/new
charmers. I however raise you a value proposition, and will speak to
the point we are generating the 'best practice' method, that we want to
guide charmers down. The common feedback we've received from
authors is "I'm not as concerned with the overhead of learning, I just want
you to tell me the best way to do it." While this represents a small
number from the overall community of people charming, we've delivered on
this asked behavior. We're taking away some maintenance
burdens, and exposing a scaffolded framework to deliver your app
containerized workload.

While its arguably more complex than `curl get.docker.com | sh` - there are
a few things happening here that should concern you already.

- You're not verifying the payload, and blindly executing it via a shell.
This violates charm store policy, and is a black hole into whats happening
without tailing the unit logs.

While the script is a replica of that installation method, (credit is given
in the layer repository readme) it was deemed important to mimic
their recommended install - when you look at the ansible based docker charm
and its divergent history + issue log.

The script has been modified to surface what the unit is doing via the
newer `juju status-set` routines  for clear visibility. And we get the
added benefit of supporting  cent/debian out of the box with this method.
The ownership burden is on  us, the ~charmers that are maintaining
the docker layer.

I appreciate the feedback though Kapil, and thanks for acknowledging the
raise in knowledge to 'get started' - you're right its not as straight
forward as curling a bash script and piping to sudo. But we don't encourage
that anyway :) We feel that you can learn and get productive with this
fairly quickly, with minimal knowledge of whats actually happening layers
below. Just look at what states become exposed and take action.

Give it a spin and let us know if you've changed your opinion at all, and
I'll be happy to talk/work through the concerns.

- Charles




Charles Butler  - Juju Charmer
Come see the future of datacenter orchestration: http://jujucharms.com

On Tue, Sep 15, 2015 at 7:46 AM, Kapil Thangavelu  wrote:

> i think participating in the burgeoning docker ecosystem is a worthwhile
> goal by making it easier to write charms that utilize docker. I do have
> some concerns though about the complexity of the layering that's taking
> place in the charm ecosystem. I've found that juju has been fairly hard to
> teach and adapt to real world usage and the layering of frameworks at the
> charm authoring level makes learning and productivity for new users even
> harder to achieve (ie those docs reference, charm helpers, reactive charm
> helpers, docker reactive 'layer', charm composition), all of which are
> fairly advanced concepts to a new user, and frankly is all of that
> nescessary to understand running a shell script
> https://github.com/juju-solutions/layer-docker/blob/master/scripts/install_docker.sh
> which is effectively the jujuized version of (curl get.docker.com | sh)
> .. now say i want to configure an insecure registry or any other docker cli
> param i have to break apart the layer abstraction anyways. It does seem
> like a useful intro to some of the advanced/additional concepts in charm
> authoring ecosystem, but at the same time the burden seems high (ie. kiss
> violation) for how to run a docker container.
>
> cheers,
>
> Kapil
>
>
> On Fri, Sep 11, 2015 at 11:40 AM, Charles Butler <
> charles.but...@canonical.com> wrote:
>
>> If anyone here is interested in delivering Docker App Containers with
>> Juju, mbruzek and I have put together some documents around a hot new
>> process using composer layers and the reactive framework.
>>
>> This is interesting because you the charm author will only be concerned
>> with how to deliver your application layers logic. You don't need to worry
>> about installing docker, or how to scaffold out a full charm boilerplate.
>> This entire process reduces the total cost of ownership of the author to
>> just managing their app layer + container.
>>
>>  https://github.com/juju/docs/pull/672
>>
>> And we've constructed/linked to an example charm using this process.
>> There's probably holes in this document, and welcome feedback directly on
>> the pull request to suss them out.
>>
>> We're fully interested in receiving your feedback about this, as its
>> important that we are properly engaging our users that are interested in
>> delivering their dockerized app with Juju, and that we've given you the
>> proper lessons to do so easily, and made the right decisions in tooling.
>>
>> All the best,
>>
>>
>> Charles Butler  - Juju Charmer
>> Come see the future of datacenter orchestration: http://jujucharms.com
>>
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>>
>
-- 
Juju mailing

Re: [ Docs ] - Charming with Docker

[Kapil Thangavelu]

i think participating in the burgeoning docker ecosystem is a worthwhile
goal by making it easier to write charms that utilize docker. I do have
some concerns though about the complexity of the layering that's taking
place in the charm ecosystem. I've found that juju has been fairly hard to
teach and adapt to real world usage and the layering of frameworks at the
charm authoring level makes learning and productivity for new users even
harder to achieve (ie those docs reference, charm helpers, reactive charm
helpers, docker reactive 'layer', charm composition), all of which are
fairly advanced concepts to a new user, and frankly is all of that
nescessary to understand running a shell script
https://github.com/juju-solutions/layer-docker/blob/master/scripts/install_docker.sh
which is effectively the jujuized version of (curl get.docker.com | sh) ..
now say i want to configure an insecure registry or any other docker cli
param i have to break apart the layer abstraction anyways. It does seem
like a useful intro to some of the advanced/additional concepts in charm
authoring ecosystem, but at the same time the burden seems high (ie. kiss
violation) for how to run a docker container.

cheers,

Kapil


On Fri, Sep 11, 2015 at 11:40 AM, Charles Butler <
charles.but...@canonical.com> wrote:

> If anyone here is interested in delivering Docker App Containers with
> Juju, mbruzek and I have put together some documents around a hot new
> process using composer layers and the reactive framework.
>
> This is interesting because you the charm author will only be concerned
> with how to deliver your application layers logic. You don't need to worry
> about installing docker, or how to scaffold out a full charm boilerplate.
> This entire process reduces the total cost of ownership of the author to
> just managing their app layer + container.
>
>  https://github.com/juju/docs/pull/672
>
> And we've constructed/linked to an example charm using this process.
> There's probably holes in this document, and welcome feedback directly on
> the pull request to suss them out.
>
> We're fully interested in receiving your feedback about this, as its
> important that we are properly engaging our users that are interested in
> delivering their dockerized app with Juju, and that we've given you the
> proper lessons to do so easily, and made the right decisions in tooling.
>
> All the best,
>
>
> Charles Butler  - Juju Charmer
> Come see the future of datacenter orchestration: http://jujucharms.com
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: state of networking?

[Kapil Thangavelu]

Its been hard to see much progress on this and i wanted to checkin wrt to
the current state.

The requirement of public ip on for the subnets sort of defeats the purpose
of supporting non default vpcs. The use of vpc is typically around network
segmentation and isolation semantics, ie db and app tier subnets don't have
public ips by design. In fact at larger orgs, its not typical that an app
deployment team would even have access to rearrange the network topology on
demand. The learned model of inform juju of the network topology for a
given env by defining/importing netspace/zone from extant subnets is more
typical.

cheers,
kapil


On Fri, Jul 24, 2015 at 3:44 PM, Dimiter Naydenov <
dimiter.nayde...@canonical.com> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 23.07.2015 22:57, Kapil Thangavelu wrote:
> > I've talked to a few folk at some conferences, but i'm curious
> > what's been happening in networking?
> >
> > it feels like its been fairly long time w/ little visible progress
> > on end user features. particularly i'm curious about aws (ie. the
> > worlds biggest cloud :-).. more concretely - can i use existing
> > (non default) vpcs? - can i create/use extant subnets across zones
> > and specify them for services? - can i control routing between
> > subnets or alternatively control/enforce iptables for a service
> > based on extant relations (optional)?
> >
> > afaics most of the network progress was in various client libs
> > afaics over the last year (and a maas centric core network
> > model)... are there any plans to switch out to the aws api sdk
> > instead of maintaining a separate client lib?
> >
> > thanks,
> >
> > Kapil
> >
> >
> >
> Hey Kapil,
>
> I can report some progress on the points you've asked about:
> 1) non-default VPC support is mostly done - see bug
> https://bugs.launchpad.net/juju-core/+bug/1321442, which I have mostly
> finished fixing. In brief, there will be a "vpc-id" environ setting
> that can be used to specify a non-default (but compatible) VPC to use.
> By compatible at this stage I mean 2 things: at least one subnet per
> AZ, all subnets in the VPC have MapPublicIPOnLaunch set.
> 2) the AWS VPC support is ongoing in a feature branch, the MVP
> proposal will include: add existing subnet to juju (make juju aware of
> it); create a space including one or more subnets; deploy a service
> within a space.
> 3) it's on the roadmap to do more sophisticated
> routing/ACL/firewalling between spaces, but it won't happen until the
> 16.04 time frame most likely.
>
> HTH,
> Dimiter
> - --
> Dimiter Naydenov 
> Juju Core Sapphire team 
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2.0.22 (GNU/Linux)
>
> iQEcBAEBAgAGBQJVspWBAAoJENzxV2TbLzHwoxkH/RM5JcXSNtL3wyLxafGbaCos
> XMNEQAnMSE/EtQerDEfuu2GFA+Un1Rc0ng6gN6322uc0Ey3OSY9IQ2s8fGhaKFJh
> NiBVHgLtlC77lKoIMGDyGf6OXXTRqZC/T/kM2Z2xrdWNcyVMySi2jH1+2kab+Ljr
> 3hwKc546DjVpaigqLx/Tq66G2yoyrS8ITdudgK8K6LmPf7hUWLMPCbYam/Dw+yuC
> PSiD2J8VNklOsg8U7zDPAyMcL+3ymyyIbp6aZRn5o6Hmkgfo64P+9J6waqFCfGTz
> qtEE3PTHneajXES9ewOiTbY0NKn7joT2T5qlxJjrMVXeIYqzGId0iaIVwIgi1DI=
> =p2Df
> -END PGP SIGNATURE-
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: millions of warnings per day, state DB grows by 4GB/day

[Kapil Thangavelu]

fwiw in the past, i always found it useful to run
https://github.com/kapilt/juju-dbinspect (specifically the history
function) on pre-releases to check the txn collection for bad system
behavior wrt to extraneous writes and file bugs. txn cleanup is great, but
avoiding extraneous txns in the first place is probably better.

cheers,
kapil



On Thu, Aug 27, 2015 at 12:02 AM, Menno Smits 
wrote:

> On 27 August 2015 at 03:55, John Meinel  wrote:
>
>> Juju 1.23 has some known issues with Lease operations. All of those have
>> been fixed in the 1.24.4 (5?) release. It is good practice to take a backup
>> before upgrading, and if you want we can try to help sort out the specific
>> growth to make sure it gets cleaned out after the upgrade. (Even just
>> getting mongo stats on the collections as a whole is likely to help us
>> understand where the size is coming from.)
>>
>> If it is what I think it is, then I would expect the txns collection to
>> be growing rapidly. IIRC there is already code in the 1.24 series to prune
>> old transactions after they've been successfully applied. But that *might*
>> be a 1.25 thing. I do believe we have a program that can do the work
>> directly if it isn't integrated already.
>>
>
> The txns pruning functionality has been there since 1.24.0 so the
> excessive txns collection size should get resolved automatically following
> an upgrade to any 1.24 version (upgrading to the latest 1.24 is
> recommended).
>
> You may find you have a problem upgrading away from 1.23 (again, due to
> problems with the new lease feature). I created a Juju plugin to help work
> around this. See this post to the juju list:
> https://www.mail-archive.com/juju@lists.ubuntu.com/msg02658.html
>
> It's probably best read in context of the whole thread starting here:
> https://www.mail-archive.com/juju@lists.ubuntu.com/msg02574.html
>
> - Menno
>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju