Re: Apt layer released
This is so cool! I hope we can integrate this in the bigdata Charms.
Letting users install additional dependencies is something that would come
in handy!
2016-01-28 16:48 GMT+01:00 Stuart Bishop :
> The Apt layer is more code broken out from my PostgreSQL charm work
> that will be used by future Cassandra charm work.
>
> It provides a simple, charms.reactive friendly interface for dealing
> with apt sources and deb packages. But I think its main utility is
> providing consistent configuration for those extra features required
> for real world production deploys without charmers needing to add a
> single line of code; specifying custom apt sources, package signing
> keys, package version pinning, and a list of arbitrary additional
> packages to install. It also finally documents these options,
> previously only found in the bowels of charmhelpers.core.fetch (which
> provides the underlying implementation).
>
>
>
> # Apt layer
>
> The Apt layer for Juju enables layered charms to more easily deal with
> deb packages and apt sources in a simple and efficient manner. It
> provides consistent configuration for operators, allowing them to
> easily specify custom apt sources and additional debs required for
> their particular installations.
>
> ## Configuration
>
> The charm may provide defaults for these service configuration
> (config.yaml) options, and the operator may override them as required.
>
> * `extra_packages`
>
> A space separated list of additional deb packages to install on
> each unit.
>
> * `package_status`
>
> 'install' or 'hold'. When set to hold, packages installed using
> the Apt layer API will be pinned, so that they will not be
> automatically upgraded when package updates are performed. 'hold'
> is particularly useful for allowing a service such as Landscape
> to automatically apply security updates to most of the system,
> whilst holding back any potentially service affecting updates.
>
> * `install_sources`
>
> A list of apt sources containing the packages that need to be
> installed.
> Each source may be either a line that can be added directly to
> sources.list(5), or in the form ppa:/ for adding
> Personal Package Archives, or a distribution component to enable.
> The list is a yaml list, encoded as a string. The nicest way of
> declaring this in a yaml file looks like the following (in
> particular,
> the | character indicates that the value is a multiline string):
>
> ```
> install_sources: |
> - ppa:stub/cassandra
> - deb http://www.apache.org/dist/cassandra/debian 21x main
> ```
>
> * `install_keys`
>
> A list of GPG signing keys to accept. There needs to be one entry
> per entry in install_sources. null may be used if no keep is
> needed, which is the case for PPAs and for the standard Ubuntu
> archives. Keys should be full ASCII armoured GPG public keys.
> GPG key ids are also accepted, but in most environments this
> mechanism is not secure. The install_keys list, like
> install_sources, must also be a yaml formatted list encoded as
> a string:
>
> ```
> install_keys: |
> - null
> - |
> -BEGIN PGP PUBLIC KEY BLOCK-
> Version: GnuPG v1
>
>
> mQINBFQJvgUBEAC0KcYCTj0hd15p4fiXBsbob0sKgsvN5Lm7N9jzJWlGshJ0peMi
>
> kH8YhDXw5Lh+mPEHksL7t1L8CIr1a+ntns/Opt65ZPO38ENVkOqEVAn9Z5sIoZsb
>
> AUeLlJzSeRLTKhcOugK7UcsQD2FHnMBJz50bxis9X7pjmnc/tWpjAGJfaWdjDIo=
> =yiQ4
> -END PGP PUBLIC KEY BLOCK-
> ```
>
> ## Usage
>
> Queue packages for installation, and have handlers waiting for
> these packages to finish being installed:
>
> ```
> from reactive import apt
>
> @hook('install')
> def install():
> apt.queue_install(['git'])
>
> @when_not('apt.installed.gnupg')
> def install_gnupg():
> apt.queue_install(['gnupg'])
>
> @when('apt.installed.git')
> @when('apt.installed.gnupg')
> def grabit():
> clone_repo()
> validate_repo()
> ```
>
> ### API
>
> Several methods are exposed in the reactive.apt Python package.
>
> * `add_source(source, key=None)`
>
> Add an apt source.
>
> A source may be either a line that can be added directly to
> sources.list(5), or in the form ppa:/ for adding
> Personal Package Archives, or a distribution component to enable.
>
> The package signing key should be an ASCII armoured GPG key. While
> GPG key ids are also supported, the retrieval mechanism is insecure.
> There is no need to specify the package signing key for PPAs or for
> the main Ubuntu archives.
>
> It is preferable if charms do not call this directly to hard
> coded apt sources, but instead have these sources listed
> as defaults in the install_sources config option. This allows
> operators to mirror your packages
Re: Apt layer released
Cool thanks for the additional info, I'm going to integrate this into my layers now On Thu, Jan 28, 2016 at 11:46 AM, Stuart Bishop wrote: > On 28 January 2016 at 23:01, Adam Stokes > wrote: > > Why would someone want to use this instead of what's provided in > > charmhelpers? > > This wraps what is provided in charmhelpers. > > To use raw charmhelpers, you need to write the hooks and handlers to > call its features at the right time and in the right order. You need > to call fetch.configure_sources, whenever the relevant config items > change. You need to install the list of extra packages, whenever the > relevant config item changes. You need to repin held packages, if the > option is set, whenever new packages are installed. To use the layer, > you just add an entry to layers.yaml and get it all done for you, and > it is all done consistently across all charms using the layer. You > don't have to worry about inconsistencies between your > reimplementation of the wheel with the next charms reimplementation of > the wheel causing confusion to users. > > It also removes all the boilerplate. You don't need to include all > that gumph in your config.yaml. > > It sets reactive states your handlers can wait on so you don't have to > set reactive states your handlers can wait on. > > By requesting a package install and having handlers wait on the > install to complete, you get to write highly decoupled code without > losing efficiency. Several different handlers in different parts of > the code base and even in different layers can all schedule the > packages they care about to be installed, and have a single apt-get > update run, and only if necessary. > > Improvements to the layer improve all charms using the layer. When > Juju charm configuration gets richer data structures, we can write all > the migration, compatibility and deprecation stuff once and all charms > get it next time they are built. Fixes in the charmhelpers codebase > can't fix its callsites or improve the documentation in your > config.yaml. > > So you want to use it to save initial work and future maintenance :) > > > > -- > Stuart Bishop > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Apt layer released
On 28 January 2016 at 23:01, Adam Stokes wrote: > Why would someone want to use this instead of what's provided in > charmhelpers? This wraps what is provided in charmhelpers. To use raw charmhelpers, you need to write the hooks and handlers to call its features at the right time and in the right order. You need to call fetch.configure_sources, whenever the relevant config items change. You need to install the list of extra packages, whenever the relevant config item changes. You need to repin held packages, if the option is set, whenever new packages are installed. To use the layer, you just add an entry to layers.yaml and get it all done for you, and it is all done consistently across all charms using the layer. You don't have to worry about inconsistencies between your reimplementation of the wheel with the next charms reimplementation of the wheel causing confusion to users. It also removes all the boilerplate. You don't need to include all that gumph in your config.yaml. It sets reactive states your handlers can wait on so you don't have to set reactive states your handlers can wait on. By requesting a package install and having handlers wait on the install to complete, you get to write highly decoupled code without losing efficiency. Several different handlers in different parts of the code base and even in different layers can all schedule the packages they care about to be installed, and have a single apt-get update run, and only if necessary. Improvements to the layer improve all charms using the layer. When Juju charm configuration gets richer data structures, we can write all the migration, compatibility and deprecation stuff once and all charms get it next time they are built. Fixes in the charmhelpers codebase can't fix its callsites or improve the documentation in your config.yaml. So you want to use it to save initial work and future maintenance :) -- Stuart Bishop -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Apt layer released
Why would someone want to use this instead of what's provided in
charmhelpers?
On Thu, Jan 28, 2016 at 10:48 AM, Stuart Bishop wrote:
> The Apt layer is more code broken out from my PostgreSQL charm work
> that will be used by future Cassandra charm work.
>
> It provides a simple, charms.reactive friendly interface for dealing
> with apt sources and deb packages. But I think its main utility is
> providing consistent configuration for those extra features required
> for real world production deploys without charmers needing to add a
> single line of code; specifying custom apt sources, package signing
> keys, package version pinning, and a list of arbitrary additional
> packages to install. It also finally documents these options,
> previously only found in the bowels of charmhelpers.core.fetch (which
> provides the underlying implementation).
>
>
>
> # Apt layer
>
> The Apt layer for Juju enables layered charms to more easily deal with
> deb packages and apt sources in a simple and efficient manner. It
> provides consistent configuration for operators, allowing them to
> easily specify custom apt sources and additional debs required for
> their particular installations.
>
> ## Configuration
>
> The charm may provide defaults for these service configuration
> (config.yaml) options, and the operator may override them as required.
>
> * `extra_packages`
>
> A space separated list of additional deb packages to install on
> each unit.
>
> * `package_status`
>
> 'install' or 'hold'. When set to hold, packages installed using
> the Apt layer API will be pinned, so that they will not be
> automatically upgraded when package updates are performed. 'hold'
> is particularly useful for allowing a service such as Landscape
> to automatically apply security updates to most of the system,
> whilst holding back any potentially service affecting updates.
>
> * `install_sources`
>
> A list of apt sources containing the packages that need to be
> installed.
> Each source may be either a line that can be added directly to
> sources.list(5), or in the form ppa:/ for adding
> Personal Package Archives, or a distribution component to enable.
> The list is a yaml list, encoded as a string. The nicest way of
> declaring this in a yaml file looks like the following (in
> particular,
> the | character indicates that the value is a multiline string):
>
> ```
> install_sources: |
> - ppa:stub/cassandra
> - deb http://www.apache.org/dist/cassandra/debian 21x main
> ```
>
> * `install_keys`
>
> A list of GPG signing keys to accept. There needs to be one entry
> per entry in install_sources. null may be used if no keep is
> needed, which is the case for PPAs and for the standard Ubuntu
> archives. Keys should be full ASCII armoured GPG public keys.
> GPG key ids are also accepted, but in most environments this
> mechanism is not secure. The install_keys list, like
> install_sources, must also be a yaml formatted list encoded as
> a string:
>
> ```
> install_keys: |
> - null
> - |
> -BEGIN PGP PUBLIC KEY BLOCK-
> Version: GnuPG v1
>
>
> mQINBFQJvgUBEAC0KcYCTj0hd15p4fiXBsbob0sKgsvN5Lm7N9jzJWlGshJ0peMi
>
> kH8YhDXw5Lh+mPEHksL7t1L8CIr1a+ntns/Opt65ZPO38ENVkOqEVAn9Z5sIoZsb
>
> AUeLlJzSeRLTKhcOugK7UcsQD2FHnMBJz50bxis9X7pjmnc/tWpjAGJfaWdjDIo=
> =yiQ4
> -END PGP PUBLIC KEY BLOCK-
> ```
>
> ## Usage
>
> Queue packages for installation, and have handlers waiting for
> these packages to finish being installed:
>
> ```
> from reactive import apt
>
> @hook('install')
> def install():
> apt.queue_install(['git'])
>
> @when_not('apt.installed.gnupg')
> def install_gnupg():
> apt.queue_install(['gnupg'])
>
> @when('apt.installed.git')
> @when('apt.installed.gnupg')
> def grabit():
> clone_repo()
> validate_repo()
> ```
>
> ### API
>
> Several methods are exposed in the reactive.apt Python package.
>
> * `add_source(source, key=None)`
>
> Add an apt source.
>
> A source may be either a line that can be added directly to
> sources.list(5), or in the form ppa:/ for adding
> Personal Package Archives, or a distribution component to enable.
>
> The package signing key should be an ASCII armoured GPG key. While
> GPG key ids are also supported, the retrieval mechanism is insecure.
> There is no need to specify the package signing key for PPAs or for
> the main Ubuntu archives.
>
> It is preferable if charms do not call this directly to hard
> coded apt sources, but instead have these sources listed
> as defaults in the install_sources config option. This allows
> operators to mirror your packages to internal archives and
> deploy your charm in environm
Apt layer released
The Apt layer is more code broken out from my PostgreSQL charm work
that will be used by future Cassandra charm work.
It provides a simple, charms.reactive friendly interface for dealing
with apt sources and deb packages. But I think its main utility is
providing consistent configuration for those extra features required
for real world production deploys without charmers needing to add a
single line of code; specifying custom apt sources, package signing
keys, package version pinning, and a list of arbitrary additional
packages to install. It also finally documents these options,
previously only found in the bowels of charmhelpers.core.fetch (which
provides the underlying implementation).
# Apt layer
The Apt layer for Juju enables layered charms to more easily deal with
deb packages and apt sources in a simple and efficient manner. It
provides consistent configuration for operators, allowing them to
easily specify custom apt sources and additional debs required for
their particular installations.
## Configuration
The charm may provide defaults for these service configuration
(config.yaml) options, and the operator may override them as required.
* `extra_packages`
A space separated list of additional deb packages to install on
each unit.
* `package_status`
'install' or 'hold'. When set to hold, packages installed using
the Apt layer API will be pinned, so that they will not be
automatically upgraded when package updates are performed. 'hold'
is particularly useful for allowing a service such as Landscape
to automatically apply security updates to most of the system,
whilst holding back any potentially service affecting updates.
* `install_sources`
A list of apt sources containing the packages that need to be installed.
Each source may be either a line that can be added directly to
sources.list(5), or in the form ppa:/ for adding
Personal Package Archives, or a distribution component to enable.
The list is a yaml list, encoded as a string. The nicest way of
declaring this in a yaml file looks like the following (in particular,
the | character indicates that the value is a multiline string):
```
install_sources: |
- ppa:stub/cassandra
- deb http://www.apache.org/dist/cassandra/debian 21x main
```
* `install_keys`
A list of GPG signing keys to accept. There needs to be one entry
per entry in install_sources. null may be used if no keep is
needed, which is the case for PPAs and for the standard Ubuntu
archives. Keys should be full ASCII armoured GPG public keys.
GPG key ids are also accepted, but in most environments this
mechanism is not secure. The install_keys list, like
install_sources, must also be a yaml formatted list encoded as
a string:
```
install_keys: |
- null
- |
-BEGIN PGP PUBLIC KEY BLOCK-
Version: GnuPG v1
mQINBFQJvgUBEAC0KcYCTj0hd15p4fiXBsbob0sKgsvN5Lm7N9jzJWlGshJ0peMi
kH8YhDXw5Lh+mPEHksL7t1L8CIr1a+ntns/Opt65ZPO38ENVkOqEVAn9Z5sIoZsb
AUeLlJzSeRLTKhcOugK7UcsQD2FHnMBJz50bxis9X7pjmnc/tWpjAGJfaWdjDIo=
=yiQ4
-END PGP PUBLIC KEY BLOCK-
```
## Usage
Queue packages for installation, and have handlers waiting for
these packages to finish being installed:
```
from reactive import apt
@hook('install')
def install():
apt.queue_install(['git'])
@when_not('apt.installed.gnupg')
def install_gnupg():
apt.queue_install(['gnupg'])
@when('apt.installed.git')
@when('apt.installed.gnupg')
def grabit():
clone_repo()
validate_repo()
```
### API
Several methods are exposed in the reactive.apt Python package.
* `add_source(source, key=None)`
Add an apt source.
A source may be either a line that can be added directly to
sources.list(5), or in the form ppa:/ for adding
Personal Package Archives, or a distribution component to enable.
The package signing key should be an ASCII armoured GPG key. While
GPG key ids are also supported, the retrieval mechanism is insecure.
There is no need to specify the package signing key for PPAs or for
the main Ubuntu archives.
It is preferable if charms do not call this directly to hard
coded apt sources, but instead have these sources listed
as defaults in the install_sources config option. This allows
operators to mirror your packages to internal archives and
deploy your charm in environments without network access.
Sets the `apt.needs_update` reactive state.
* `queue_install(packages, options=None)`
Queue one or more deb packages for install. The actual package
installation will be performed later by a handler in the
apt layer. The `apt.installed.{name}` state will be set once
