Re: Planning for Juju 2.2 (16.10 timeframe)

[roger peppe]

On 16 March 2016 at 15:04, Kapil Thangavelu  wrote:
> Relations have associated config schemas that can be set by the user
> creating the relation. I.e. I could run one autoscaling service and
> associate with relation config for autoscale options to the relation with a
> given consumer service.

Great, I hoped that's what you meant.
I'm also +1 on this feature - it would enable all kinds of useful flexibility.

One recent example I've come across that could use this feature
is that we've got a service that can hand out credentials to services
that are related to it. At the moment the only way to state that
certain services should be handed certain classes of credential
is to have a config value that holds a map of service name to
credential info, which doesn't seem great - it's awkward, easy
to get wrong, and when a service goes away, its associated info
hangs around.

Having the credential info associated with the relation itself would be perfect.

>
> On Wed, Mar 16, 2016 at 9:17 AM roger peppe 
> wrote:
>>
>> On 16 March 2016 at 12:31, Kapil Thangavelu  wrote:
>> >
>> >
>> > On Tue, Mar 8, 2016 at 6:51 PM, Mark Shuttleworth 
>> > wrote:
>> >>
>> >> Hi folks
>> >>
>> >> We're starting to think about the next development cycle, and gathering
>> >> priorities and requests from users of Juju. I'm writing to outline some
>> >> current topics and also to invite requests or thoughts on relative
>> >> priorities - feel free to reply on-list or to me privately.
>> >>
>> >> An early cut of topics of interest is below.
>> >>
>> >> Operational concerns
>> >>
>> >> * LDAP integration for Juju controllers now we have multi-user
>> >> controllers
>> >> * Support for read-only config
>> >> * Support for things like passwords being disclosed to a subset of
>> >> user/operators
>> >> * LXD  container migration
>> >> * Shared uncommitted state - enable people to collaborate around
>> >> changes
>> >> they want to make in a model
>> >>
>> >> There has also been quite a lot of interest in log control - debug
>> >> settings for logging, verbosity control, and log redirection as a
>> >> systemic
>> >> property. This might be a good area for someone new to the project to
>> >> lead
>> >> design and implementation. Another similar area is the idea of
>> >> modelling
>> >> machine properties - things like apt / yum repositories, cache settings
>> >> etc,
>> >> and having the machine agent setup the machine / vm / container
>> >> according to
>> >> those properties.
>> >>
>> >
>> > ldap++. as brought up in the user list better support for aws best
>> > practice
>> > credential management, ie. bootstrapping with transient credentials (sts
>> > role assume, needs AWS_SECURITY_TOKEN support), and instance role for
>> > state
>> > servers.
>> >
>> >
>> >>
>> >> Core Model
>> >>
>> >>  * modelling individual services (i.e. each database exported by the db
>> >> application)
>> >>  * rich status (properties of those services and the application
>> >> itself)
>> >>  * config schemas and validation
>> >>  * relation config
>> >>
>> >> There is also interest in being able to invoke actions across a
>> >> relation
>> >> when the relation interface declares them. This would allow, for
>> >> example, a
>> >> benchmark operator charm to trigger benchmarks through a relation
>> >> rather
>> >> than having the operator do it manually.
>> >>
>> >
>> > in priority order, relation config
>>
>> What do you understand by the term "relation config"?

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Usability issues with status-history

[Ian Booth]

On 17/03/16 19:51, William Reade wrote:
> I see this as a combination of two problems:
> 
> 1) We're spamming the end user with "whatever's in the status-history
> collection" rather than presenting a digest tuned for their needs.
> 2) Important messages get thrown away way too early, because we don't know
> which messages are important.
> 
> I think the pocket/transient/expiry solutions boil down to "let's make the
> charmer decide what's important", and I don't think that will help. The
> charmer is only sending those messages *because she believes they're
> important*; even if we had "perfect" trimming heuristics for the end user,
> we do the *charmer* a disservice by leaving them no record of what their
> charm actually did.
> 
> And, more generally: *every* message we throw away makes it hard to
> correctly analyse any older message. This applies within a given entity's
> domain, but also across entities: if you're trying to understand the
> interactions between 2 units, but one of those units is generating many
> more messages, you'll have 200 messages to inspect; but the 100 for the
> faster unit will only cover (say) the last 30 for the slower one, leaving
> 70 slow-unit messages that can't be correlated with the other unit's
> actions. At best, those messages are redundant; at worst, they're actively
> misleading.
> 
> So: I do not believe that any approach that can be summed up as "let's
> throw away *more* messages" is going to help either. We need to fix (2) so
> that we have raw status data that extends reasonably far back in time; and
> then we need to fix (1) so that we usefully precis that data for the user
> (...and! leave a path that makes the raw data observable, for the cases
> where our heuristics are unhelpful).
> 

I mostly agree but still believe there's a case for transient messages. The case
where Juju is downloading an image and emits progress updates which go into
status history is to me clearly a case where we needn't persist every single one
(or any). In that case, it's not a charmer deciding but Juju. And with status
updates like X% complete, as soon as a new message arrives, the old one is
superseded anyway. The user is surely just interested to know the current status
and when it completes they don't care anymore. And Juju agent can still decide
to say make every 10% of download progress messages non-transient to they go to
history for future reference.

> Cheers
> William
> 
> PS re: UX of asking for N entries... I can see end-user stories for
> timespans, and for "the last N *significant* changes". What's the scenario
> where a user wants to see exactly 50 message atoms?
> 

No one would say they want to see exactly 50 - it's an estimate. It's like when
you git log and you ask for the last 20 commits. If that's not enough to see
what you want, you just run again with an increased number.

I do think allowing for a timespan to be specified may be useful.

John's suggestion for adding a lifetime does sounds more complicated than we
want right now.

Would this work s an initial improvement for 2.0:

1. Increase limit of stored messages per entity so say 500 (from 100)
2. Allow messages emitted from Juju to be marked as transient
eg for download progress
3. Do smarter filtering of what is displayed with status-history
eg if we see the same tuple of messages over and over, consolidate

TIMETYPESTATUS  MESSAGE
26 Dec 2015 13:51:59Z   agent   executing   running config-changed hook
26 Dec 2015 13:51:59Z   agent   idle
26 Dec 2015 13:56:57Z   agent   executing   running update-status hook
26 Dec 2015 13:56:59Z   agent   idle
26 Dec 2015 14:01:57Z   agent   executing   running update-status hook
26 Dec 2015 14:01:59Z   agent   idle
26 Dec 2015 14:01:57Z   agent   executing   running update-status hook
26 Dec 2015 14:01:59Z   agent   idle

becomes

TIME TYPE STATUS MESSAGE
26 Dec 2015 13:51:59Z agent executing running config-changed hook
26 Dec 2015 13:51:59Z agent idle
>> Repeated 3 times, last occurence:
26 Dec 2015 14:01:57Z agent executing running update-status hook
26 Dec 2015 14:01:59Z agent idle





> On Thu, Mar 17, 2016 at 6:30 AM, John Meinel  wrote:
> 
>>
>>
>> On Thu, Mar 17, 2016 at 8:41 AM, Ian Booth 
>> wrote:
>>
>>>
>>> Machines, services and units all now support recording status history. Two
>>> issues have come up:
>>>
>>> 1. https://bugs.launchpad.net/juju-core/+bug/1530840
>>>
>>> For units, especially in steady state, status history is spammed with
>>> update-status hook invocations which can obscure the hooks we really care
>>> about
>>>
>>> 2. https://bugs.launchpad.net/juju-core/+bug/1557918
>>>
>>> We now have the concept of recording a machine provisioning status. This
>>> is
>>> great because it gives observability to what is happening as a node is
>>> being
>>> allocated in the cloud. With LXD, this feature has been used to give
>>> visibility
>>> to 

Re: Planning for Juju 2.2 (16.10 timeframe)

[Stuart Bishop]

On 9 March 2016 at 10:51, Mark Shuttleworth  wrote:

> Operational concerns

I still want 'juju-wait' as a supported, builtin command rather than
as a fragile plugin I maintain and as code embedded in Amulet that the
ecosystem team maintain. A thoughtless change to Juju's status
reporting would break all our CI systems.

> Core Model

At the moment logging, monitoring (alerts) and metrics involve
customizing your charm to work with a specific subordinate. And at
deploy time, you of course need to deploy and configure the
subordinate, relate it etc. and things can get quite cluttered.

Could logging, monitoring and metrics be brought into the core model somehow?

eg. I attach a monitoring service such as nagios to the model, and all
services implicitly join the monitoring relation. Rather than talk
bespoke protocols, units use the 'monitoring-alert' tool send a JSON
dict to the monitoring service (for push alerts). There is some
mechanism for the monitoring service to trigger checks remotely.
Requests and alerts go via a separate SSL channel rather than the
relation, as relations are too heavy weight to trigger several times a
second and may end up blocked by eg. other hooks running on the unit
or jujud having been killed by OOM.

Similarly, we currently handle logging by installing a subordinate
that knows how to push rotated logs to Swift. It would be much nicer
to set this at the model level, and have tools available for the charm
to push rotated logs or stream live logs to the desired logging
service. syslog would be a common approach, as would streaming stdout
or stderr.

And metrics, where a charm installs a cronjob or daemon to spit out
performance metrics as JSON dicts to a charm tool which sends them to
the desired data store and graphing systems, maybe once a day or maybe
several times a second. Rather than the current approach of assuming
statsd as the protocol and spitting out packages to an IP address
pulled from the service configuration.

>  * modelling individual services (i.e. each database exported by the db
> application)
>  * rich status (properties of those services and the application itself)
>  * config schemas and validation
>  * relation config
>
> There is also interest in being able to invoke actions across a relation
> when the relation interface declares them. This would allow, for example, a
> benchmark operator charm to trigger benchmarks through a relation rather
> than having the operator do it manually.

This is interesting. You can sort of do this already if you setup ssh
so units can run commands on each other, but network partitions are an
issue. Triggering an action and waiting on the result works around
this problem.

For failover in the PostgreSQL charm, I currently need to leave
requests in the leader settings and wait for units to perform the
requested tasks and report their results using the peer relation. It
might be easier to coordinate if the leader was able to trigger these
tasks directly on the other units.

Similarly, most use cases for charmhelpers.coordinator or the
coordinator layer would become easier. Rather than using several
rounds of leadership and peer relation hooks to perform a rolling
restart or rolling upgrade, the leader could trigger the operations
remotely one at a time via a peer relation.


> Storage
>
>  * shared filesystems (NFS, GlusterFS, CephFS, LXD bind-mounts)
>  * object storage abstraction (probably just mapping to S3-compatible APIS)
>
> I'm interested in feedback on the operations aspects of storage. For
> example, whether it would be helpful to provide lifecycle management for
> storage being re-assigned (e.g. launch a new database application but reuse
> block devices previously bound to an old database  instance). Also, I think
> the intersection of storage modelling and MAAS hasn't really been explored,
> and since we see a lot of interest in the use of charms to deploy
> software-defined storage solutions, this probably will need thinking and
> work.

Reusing an old mount on a new unit is a common use case. Single unit
PostgreSQL is simplest here - it detects an existing database is on
the mount, and rather than recreate it fixes permissions (uids and
gids will often not match), mounts it and recreates any resources the
charm needs (such as the 'nagios' user so the monitoring checks work).
But if you deploy multiple PostgreSQL units reusing old mounts, what
do you do? At the moment, the one lucky enough to be elected master
gets used and the others destroyed.

Cassandra is problematic, as the newly provisioned units will have
different positions and ranges in the replication ring and the
existing data will usually actually belong to other units in the
service. It would be simpler to create a new cluster, then attach the
old data as an 'import' mount and have the storage hook load it into
the cluster. Which requires twice the disk space, but means you could
migrate a 10 unit Cassandra cluster to a new 5 unit 

Re: Planning for Juju 2.2 (16.10 timeframe)

[Andrew Wilkins]

On Sat, Mar 19, 2016 at 12:53 AM Jacek Nykis 
wrote:

> On 08/03/16 23:51, Mark Shuttleworth wrote:
> > *Storage*
> >
> >  * shared filesystems (NFS, GlusterFS, CephFS, LXD bind-mounts)
> >  * object storage abstraction (probably just mapping to S3-compatible
> APIS)
> >
> > I'm interested in feedback on the operations aspects of storage. For
> > example, whether it would be helpful to provide lifecycle management for
> > storage being re-assigned (e.g. launch a new database application but
> > reuse block devices previously bound to an old database  instance).
> > Also, I think the intersection of storage modelling and MAAS hasn't
> > really been explored, and since we see a lot of interest in the use of
> > charms to deploy software-defined storage solutions, this probably will
> > need thinking and work.
>
> Hi Mark,
>
> I took juju storage for a spin a few weeks ago. It is a great idea and
> I'm sure it will simplify our models (no more need for
> block-storage-broker and storage charms). It will also improve security
> because block-storage-broker needs nova credentials to work
>
> I only played with storage briefly but I hope my feedback and ideas will
> be useful
>
> * IMO it would be incredibly useful to have storage lifecycle
> management. Deploying a new database using pre-existing block device you
> mentioned would certainly be nice. Another scenario could be users who
> deploy to local disk and decide to migrate to block storage later
> without redeploying and manual data migration
>
>

> One day we may even be able to connect storage with actions. I'm
> thinking "storage snapshot" action followed by juju deploy to create up
> to date database clone for testing/staging/dev
>
> * I found documentation confusing. It's difficult for me to say exactly
> what is wrong but I had to read it a few times before things became
> clear. I raised some specific points on github:
> https://github.com/juju/docs/issues/889
>
> * cli for storage is not as nice as other juju commands. For example we
> have the in the docs:
>
> juju deploy cs:~axwalk/postgresql --storage data=ebs-ssd,10G pg-ssd
>
> I suspect most charms will use single storage device so it may be
> possible to optimize for that use case. For example we could have:
>
> juju deploy cs:~axwalk/postgresql --storage-type=ebs-ssd --storage-size=10G
>

It seems like the issues you've noted below are all documentation issues,
rather than limitations in the implementation. Please correct me if I'm
wrong.


> If we come up with sensible defaults for different providers we could
> make end users' experience even better by making --storage-type optional
>

Storage type is already optional. If you omit it, you'll get the provider
default. e.g. for AWS, that's EBS magnetic disks.


> * it would be good to have ability to use single storage stanza in
> metadata.yaml that supports all types of storage. They way it is done
> now [0] means I can't test block storage hooks in my local dev
> environment. It also forces end users to look for storage labels that
> are supported
>
> [0] http://paste.ubuntu.com./15414289/


Not quite sure what you mean here. If you have a "filesystem" type, you can
use any storage provider that supports natively creating filesystems (e.g.
"tmpfs") or block devices (e.g. "ebs"). If you specify the latter, Juju
will manage the filesystem on the block device.

* the way things are now hooks are responsible for creating filesystem
> on block devices. I feel that as a charmer I shouldn't need to know that
> much about storage internals. I would like to ask juju and get
> preconfigured path back. Whether it's formatted and mounted block
> device, GlusterFS or local filesystem it should not matter


That is exactly what it does, so again, I think this is an issue of
documentation clarity. If you're using the "filesystem" type, Juju will
create the filesystem; if you use "block", it won't.

If you could provide more details on what you're doing (off list, I think
would be best), I can try and help. We can then feed back into the docs to
make it clearer.

Cheers,
Andrew

* finally I hit 2 small bugs:
>
> https://bugs.launchpad.net/juju-core/+bug/1539684
> https://bugs.launchpad.net/juju-core/+bug/1546492



>
>
> If anybody is interested in more details just ask, I'm happy to discuss
> or try things out just note that I will be off next week so will most
> likely reply on 29th
>
>
> Regards,
> Jacek
>
> --
> Juju mailing list
> j...@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Planning for Juju 2.2 (16.10 timeframe)

[Eric Snow]

On Fri, Mar 18, 2016 at 8:57 AM, Tom Barber  wrote:
> c) upload files with actions. Currently for some things I need to pass in
> some files then trigger an action on the unit upon that file. It would be
> good to say path=/tmp/myfile.xyz and have the action upload that to a place
> you define.

Have you taken a look at resources in the upcoming 2.0?  You define
resources in your charm metadata and use "juju attach" to upload them
to the controller (e.g. "juju attach my-service/0
my-resource=/tmp/myfile.xyz"). *  Then charms can use the
"resource-get" hook command to download the resource file from the
controller.  "resource-get" returns the path where the downloaded file
was saved.

-eric


* You will also upload the resources to the charm store for charm store charms.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev