Re: Juju 2.3 beta2 is here!

2017-11-10 Thread Dmitrii Shcherbakov 
There is an edge case to that: when you remove a machine and add a new one
an ID cannot be reused.

I believe it's just auto-increment in the database: one does not reuse
auto-incremented IDs for efficiency (otherwise you have to implement "find
first available unused ID" functionality).

So, if there is a machine "1" in the model which gets deleted and a --to 1
placement directive in a new bundle with --use-existing-machines flag
passed, this new machine will get a new ID (depending on how many placement
directives like that already exist and implementation it may not be
 + 1).

I think this edge case can stay as is because you've already hacked an
initial model further at this point - just modify your add-on bundle to
reflect the model state.

But a clean deployment is an important case to consider in my view.

Best Regards,
Dmitrii Shcherbakov

Field Software Engineer
IRC (freenode): Dmitrii-Sh

On Fri, Nov 10, 2017 at 4:13 PM, roger peppe 
wrote:

> On 10 November 2017 at 10:40, Dmitrii Shcherbakov
>  wrote:
> > This might not be an ideal example after all. However, I encountered
> > something else in this case - final model machine IDs are not the same
> as I
> > would expect while looking at the bundle.
>
> I'd've thought that --use-existing-machines might solve that case, but...
> I guess it depends if --use-existing-machines also guarantees that any new
> machines created will have machine ids that match the ones in the bundle.
>
> If not, I guess it probably should.
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-10 Thread roger peppe 
On 10 November 2017 at 10:40, Dmitrii Shcherbakov
 wrote:
> This might not be an ideal example after all. However, I encountered
> something else in this case - final model machine IDs are not the same as I
> would expect while looking at the bundle.

I'd've thought that --use-existing-machines might solve that case, but...
I guess it depends if --use-existing-machines also guarantees that any new
machines created will have machine ids that match the ones in the bundle.

If not, I guess it probably should.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-10 Thread Dmitrii Shcherbakov 
This might not be an ideal example after all. However, I encountered
something else in this case - final model machine IDs are not the same as I
would expect while looking at the bundle. This is Juju 2.2.6, MAAS 2.2.2. I
am not sure there can be any guarantees about that due to parallelization
of provisioner and az spread code - I expected that during an initial
deployment on a *clean model* (no prior machine number allocation), bundle
and model machine numbers would match but apparently they don't.

If model machine numbers matched the bundle ones after a clean model
deployment, my example would be about slicing a huge bundle into multiple
ones and deploying in steps (rabbitmq is a bad example of something to
deploy like that).

So:

bundle.yaml machine definitions:

http://paste.ubuntu.com/25930362/
...
  '0':
series: xenial
constraints: tags=gateway
zone: z01
  '1':
series: xenial
constraints: tags=gateway
zone: z02
  '2':
series: xenial
constraints: tags=gateway
zone: z03
  '3': 
series: xenial
constraints: tags=compute
zone: z01
  '4': 
series: xenial
constraints: tags=compute
zone: z02
  '5': 
series: xenial
constraints: tags=compute
zone: z03
  '6': *compute-z1
  '7': *compute-z2
  '8': *compute-z3
  '9': *compute-z1
  '10': *compute-z2
  '11': *compute-z3
  '12': *compute-z1
...

They were eventually enumerated as follows:

http://paste.ubuntu.com/25930364/

0 started  198.51.105.129  7nc46sxenial  z01
Deployed
0/lxd/0   started  198.51.105.152  juju-ada6ad-0-lxd-0   xenial  z01
Container started
...
1 started  198.51.105.130  hxgx7cxenial  z02
Deployed
1/lxd/0   started  198.51.105.150  juju-ada6ad-1-lxd-0   xenial  z02
Container started
...
2 started  198.51.105.131  wdskcyxenial  z03
Deployed
...
4 started  198.51.105.134  f3e4gmxenial  z02
Deployed
5 started  10.30.21.22 k8spqwxenial  default
Deployed
6 started  10.30.21.47 feethexenial  default
Deployed
7 started  10.30.21.77 gesdy6xenial  default
Deployed
8 started  10.30.21.45 46rp4sxenial  default
Deployed
9 started  10.30.21.55 ce6e38xenial  default
Deployed
10started  10.30.21.46 yqew8fxenial  default
Deployed
11started  10.30.21.50 bdfn4yxenial  default
Deployed
12started  198.51.105.132  prbbg4xenial  z03
Deployed
12/lxd/0  started  198.51.105.142  juju-ada6ad-12-lxd-0  xenial  z03
Container started
...
13started  10.30.21.88 xrmf76xenial  default
Deployed
14started  10.30.21.81 sbqwexxenial  default
Deployed
15started  10.30.21.17 hkgcf6xenial  default
Deployed
16started  10.30.21.62 sdtfmmxenial  default
Deployed
17started  10.30.21.48 ghnxwwxenial  default
Deployed
18started  10.30.21.30 xe8da4xenial  default
Deployed
19started  10.30.21.65 nhd773xenial  default
Deployed
20started  198.51.105.135  433afexenial  z01
Deployed
...

Note how machines in the "default" AZ in MAAS took precedence in integer ID
allocation after machines 4 and 12 and, more importantly, how node 2 in the
bundle became node 12 in the model.

So, instead of 0, 1, 2 in the bundle we have 0, 1, 12 and have to use  --to
lxd:0,lxd:1,lxd:12 for placement.

I understand that there is AZ spread code in Juju https://git.io/vF2nu
https://git.io/vF2nD so I can see why the original bundle numbering does
not match.

The end result is that it requires an additional step in the big bundle
separation use-case:

1. deploy a core.yaml bundle part;
2. get resulting machine numbers and apply to bundle-add-ons.yaml (no juju
export-bundle?)
2. deploy bundle-add-ons.yaml.

Manual processing in step 2 is something to consider because we have no
reliable means of addressing machines in dependent bundles then.

Would reusing existing unit names be possible (the feature is about
existing machine definitions so it's good to clarify)?

bundle-add-ons.yaml:
...
to:
  - old-app/0
  - old-app/1
  - old-app/2

This is different from = mapping which isn't
considered as you mentioned. It's rather a reference to symbolic machine
name ideas.


Best Regards,
Dmitrii Shcherbakov

Field Software Engineer
IRC (freenode): Dmitrii-Sh

On Fri, Nov 10, 2017 at 2:20 AM, Tim Penhey 
wrote:

> On 10/11/17 12:12, Dmitrii Shcherbakov wrote:
> > It's situations like the following that I am trying to avoid:
> >
> >   rabbitmq-server:
> > charm: cs:xenial/rabbitmq-server
> > bindings:
> >   "": *oam-space
> >   amqp: *internal-space
> >   cluster: *internal-space
> >

Re: Juju 2.3 beta2 is here!

2017-11-10 Thread roger peppe 
On 9 November 2017 at 22:49, Tim Penhey  wrote:
> No we aren't going to reuse --to.
>
> The --to directive is just for placement directives. Trying to use that
> to overload machine mappings for bundles adds complexity for no real value.

As I see it, machine mapping *is* placement, as applied to a bundle rather
than the first unit of a deployed application.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-09 Thread Tim Penhey 
On 10/11/17 12:12, Dmitrii Shcherbakov wrote:
> It's situations like the following that I am trying to avoid:
> 
>   rabbitmq-server:
>     charm: cs:xenial/rabbitmq-server
>     bindings:
>       "": *oam-space
>       amqp: *internal-space
>       cluster: *internal-space
>     options:
>       source: *openstack-origin
>       min-cluster-size: 3
>       cluster-partition-handling: pause_minority
>     num_units: 3
>     to:
>     - lxd:0
>     - lxd:1
>     - lxd:2
> 
> serialized by hand to: juju deploy rabbitmq-server --config
> ../rabbitmq.yaml -n 3 --to lxd:0,lxd:1,lxd:12 --bind "space-oam
> amqp=space-oam cluster=space-oam"
> 
> cat ../rabbitmq.yaml 
> rabbitmq-server:
>   source: cloud:xenial-ocata
>   min-cluster-size: 3
>   cluster-partition-handling: pause_minority
> 
> Which includes brain-parsing the definition, serializing it to
> .yaml + -n  --to  --bind 

I don't understand what it is you are trying to avoid here?

What is it you are trying to do?

Tim

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-09 Thread Dmitrii Shcherbakov 
Tim,

Whichever works best in terms of code-base clarity and stability - it's
hard to debug spaghetti code with lots of overrides so fully agreed on --to.

"existing" sounds good to me. I only have to do it once and it's easy to
remember. Extra 3 characters to type are irrelevant when you type 300
characters per minute.

It's situations like the following that I am trying to avoid:

  rabbitmq-server:
charm: cs:xenial/rabbitmq-server
bindings:
  "": *oam-space
  amqp: *internal-space
  cluster: *internal-space
options:
  source: *openstack-origin
  min-cluster-size: 3
  cluster-partition-handling: pause_minority
num_units: 3
to:
- lxd:0
- lxd:1
- lxd:2

serialized by hand to: juju deploy rabbitmq-server --config
../rabbitmq.yaml -n 3 --to lxd:0,lxd:1,lxd:12 --bind "space-oam
amqp=space-oam cluster=space-oam"

cat ../rabbitmq.yaml
rabbitmq-server:
  source: cloud:xenial-ocata
  min-cluster-size: 3
  cluster-partition-handling: pause_minority

Which includes brain-parsing the definition, serializing it to
.yaml + -n  --to  --bind 

Machine to unit mapping: not that important in my view.

Best Regards,
Dmitrii Shcherbakov

Field Software Engineer
IRC (freenode): Dmitrii-Sh

On Fri, Nov 10, 2017 at 1:49 AM, Tim Penhey 
wrote:

> No we aren't going to reuse --to.
>
> The --to directive is just for placement directives. Trying to use that
> to overload machine mappings for bundles adds complexity for no real value.
>
> We will use --map-machines. I'm a big proponent of explicit is better
> than implicit.
>
> While I'm not 100% fixed on using the term "existing", I don't think
> either '...' nor ':' make a lot of sense. They are not used elsewhere in
> juju anywhere, and aren't really intuitive.
>
> We also aren't looking to map machine placement directives to other
> bundle placement directive types.
>
> Tim
>
> On 10/11/17 10:39, Dmitrii Shcherbakov wrote:
> > I think it's nice to reuse --to because we don't use it with bundles on
> > juju deploy. A unified --map[-machines] would also be fine to me.
> >
> >  := ( |
> > )=( | )
> > --to ([,]+[,:] ) | :
> >
> > Remap two, otherwise use existing:
> >
> > --to 1=2,3=5,:
> >
> > The same with app names, but have to error out on lxd:1 or kvm:5 in a
> > bundle if exapp/2 or exapp/5 are themselves in lxd or kvm "containers"
> > (surely nested containers or VMs are possible to do by hand but insane
> > in this case):
> >
> > --to 1=exapp/2,3=exapp/5,:
> >
> > This is kind of complex as there *may* be a conflicting placement
> > directive in a bundle for exappplugin/2 or it may be a subordinate (in
> > which case we ignore the mapping altogether)
> >
> > --to exappplugin/2=exapp/2,3=exappother/5,:
> >
> > FWIW it may be a bundle without machine definitions and placement
> > directives, so, still a valid case.
> >
> > Remap two, otherwise allocate new:
> >
> > --to 1=2,3=5
> >
> > Use existing:
> >
> > --to :
> >
> > Allocate new:
> > 
> >
> > Another idea, albeit more complex, is to introduce ranges to that:
> >
> > 3:5=41:43
> >
> > 3:5=41:43,:
> >
> > 3:5=neutron-gateway/0:neutron-gateway/2
> >
> > Also have to be careful about the following (I haven't seen the feature
> > code yet):
> >
> > 1. odd charm or app names during machine spec parsing:
> > https://launchpad.net/charm-6wind-virtual-accelerator
> > 
> >
> > 2. endpoint -> network space bindings and storage bindings for existing
> > apps in a given model that may be present in a newly deployed bundle.
> >
> > Really appreciate the major changes coming to Juju 2.3 - thanks a lot!
> >
> >
> > Best Regards,
> > Dmitrii Shcherbakov
> >
> > Field Software Engineer
> > IRC (freenode): Dmitrii-Sh
> >
> > On Thu, Nov 9, 2017 at 1:20 PM, roger peppe  > > wrote:
> >
> > I still like the idea of overloading the --to flag rather than having
> > a new --map-machines flag. It's concise and fits well, I think - we
> > want the machines in this bundle to mapped *to* the machines we're
> > specifying here.
> >
> > I like the thrust of Tim's suggestion for "existing" but I'm not
> > entirely sure about that word - it's quite long and it doesn't quite
> > express the identity relationship that I see there. How about "same"?
> >
> > For example:
> >
> >  juju deploy --to 1=2,2=3,same some-bundle
> >
> > Another possibility: use ellipses to imply the rest of the mapping:
> >
> > juju deploy --to 1=2,2=3,... some-bundle
> > juju deploy --to ... some-bundle
> >
> >
> >
> > On 9 November 2017 at 02:43, Tim Penhey  > > wrote:
> > >
> > >
> > > On 09/11/17 13:06, Mark Shuttleworth wrote:
> > >> On 11/07/2017 03:11 PM, John Meinel wrote:
> > >>> ...
> > >>>
> > >>>
> > >>> >

Re: Juju 2.3 beta2 is here!

2017-11-09 Thread Tim Penhey 
No we aren't going to reuse --to.

The --to directive is just for placement directives. Trying to use that
to overload machine mappings for bundles adds complexity for no real value.

We will use --map-machines. I'm a big proponent of explicit is better
than implicit.

While I'm not 100% fixed on using the term "existing", I don't think
either '...' nor ':' make a lot of sense. They are not used elsewhere in
juju anywhere, and aren't really intuitive.

We also aren't looking to map machine placement directives to other
bundle placement directive types.

Tim

On 10/11/17 10:39, Dmitrii Shcherbakov wrote:
> I think it's nice to reuse --to because we don't use it with bundles on
> juju deploy. A unified --map[-machines] would also be fine to me.
> 
>  := ( |
> )=( | )
> --to ([,]+[,:] ) | :
> 
> Remap two, otherwise use existing:
> 
> --to 1=2,3=5,:
> 
> The same with app names, but have to error out on lxd:1 or kvm:5 in a
> bundle if exapp/2 or exapp/5 are themselves in lxd or kvm "containers"
> (surely nested containers or VMs are possible to do by hand but insane
> in this case):
> 
> --to 1=exapp/2,3=exapp/5,:
> 
> This is kind of complex as there *may* be a conflicting placement
> directive in a bundle for exappplugin/2 or it may be a subordinate (in
> which case we ignore the mapping altogether)
> 
> --to exappplugin/2=exapp/2,3=exappother/5,:
> 
> FWIW it may be a bundle without machine definitions and placement
> directives, so, still a valid case.
> 
> Remap two, otherwise allocate new:
> 
> --to 1=2,3=5
> 
> Use existing:
> 
> --to :
> 
> Allocate new:
> 
> 
> Another idea, albeit more complex, is to introduce ranges to that:
> 
> 3:5=41:43
> 
> 3:5=41:43,:
> 
> 3:5=neutron-gateway/0:neutron-gateway/2
> 
> Also have to be careful about the following (I haven't seen the feature
> code yet):
> 
> 1. odd charm or app names during machine spec parsing:
> https://launchpad.net/charm-6wind-virtual-accelerator
> 
> 
> 2. endpoint -> network space bindings and storage bindings for existing
> apps in a given model that may be present in a newly deployed bundle.
> 
> Really appreciate the major changes coming to Juju 2.3 - thanks a lot!
> 
> 
> Best Regards,
> Dmitrii Shcherbakov
> 
> Field Software Engineer
> IRC (freenode): Dmitrii-Sh
> 
> On Thu, Nov 9, 2017 at 1:20 PM, roger peppe  > wrote:
> 
> I still like the idea of overloading the --to flag rather than having
> a new --map-machines flag. It's concise and fits well, I think - we
> want the machines in this bundle to mapped *to* the machines we're
> specifying here.
> 
> I like the thrust of Tim's suggestion for "existing" but I'm not
> entirely sure about that word - it's quite long and it doesn't quite
> express the identity relationship that I see there. How about "same"?
> 
> For example:
> 
>      juju deploy --to 1=2,2=3,same some-bundle
> 
> Another possibility: use ellipses to imply the rest of the mapping:
> 
>     juju deploy --to 1=2,2=3,... some-bundle
>     juju deploy --to ... some-bundle
> 
> 
> 
> On 9 November 2017 at 02:43, Tim Penhey  > wrote:
> >
> >
> > On 09/11/17 13:06, Mark Shuttleworth wrote:
> >> On 11/07/2017 03:11 PM, John Meinel wrote:
> >>> ...
> >>>
> >>>
> >>>     > Perhaps just:
> >>>     >
> >>>     >   juju deploy --map-machines A=B,C=D
> >>>     >
> >>>     > ... or some variant of that?
> >>>     >
> >>>     > Let's use the betas to refine and condense and clarify.
> >>>
> >>>     +1 to that. I'm wondering if use-existing-machines is ever
> appropriate
> >>>     on its own, as the machine numbers in a model are ephemeral but
> >>>     machine numbers in a bundle are static.
> >>>
> >>>
> >>> Feedback from Admins that one of their big use case really is for
> >>> bundle-a to lay down a definition/base charm across everything, and
> >>> bundle-b to be meant as an exact overlay, and all of the machine-ids
> >>> are exact matches. And having to specify 0=0,...50=50 is a lot
> of ugly
> >>> boilerplate.
> >>
> >> I would expect that --map-machines means that machine numbers
> correspond
> >> UNLESS remapped. So your ugly boilerplate is not needed.
> >
> > Been thinking more... how about this as a proposal:
> >
> > I think we can combine both the --use-existing-machines and the
> > --bundle-machine into the single --map-machines:
> >
> > So...
> >
> > To use the existing machines as is:
> >   --map-machines existing
> >
> > To only map two machines,
> >   --map-machines 1=2,2=3
> >
> > To use existing, and map two machines
> >   --map-machines existing,1=2,2=3
> >
> > Thoughts?
> >
> > Tim
>

Re: Juju 2.3 beta2 is here!

2017-11-09 Thread Dmitrii Shcherbakov 
I think it's nice to reuse --to because we don't use it with bundles on
juju deploy. A unified --map[-machines] would also be fine to me.

 := ( |
)=(
| )
--to ([,]+[,:] ) | :

Remap two, otherwise use existing:

--to 1=2,3=5,:

The same with app names, but have to error out on lxd:1 or kvm:5 in a
bundle if exapp/2 or exapp/5 are themselves in lxd or kvm "containers"
(surely nested containers or VMs are possible to do by hand but insane in
this case):

--to 1=exapp/2,3=exapp/5,:

This is kind of complex as there *may* be a conflicting placement directive
in a bundle for exappplugin/2 or it may be a subordinate (in which case we
ignore the mapping altogether)

--to exappplugin/2=exapp/2,3=exappother/5,:

FWIW it may be a bundle without machine definitions and placement
directives, so, still a valid case.

Remap two, otherwise allocate new:

--to 1=2,3=5

Use existing:

--to :

Allocate new:


Another idea, albeit more complex, is to introduce ranges to that:

3:5=41:43

3:5=41:43,:

3:5=neutron-gateway/0:neutron-gateway/2

Also have to be careful about the following (I haven't seen the feature
code yet):

1. odd charm or app names during machine spec parsing:
https://launchpad.net/charm-6wind-virtual-accelerator

2. endpoint -> network space bindings and storage bindings for existing
apps in a given model that may be present in a newly deployed bundle.

Really appreciate the major changes coming to Juju 2.3 - thanks a lot!


Best Regards,
Dmitrii Shcherbakov

Field Software Engineer
IRC (freenode): Dmitrii-Sh

On Thu, Nov 9, 2017 at 1:20 PM, roger peppe 
wrote:

> I still like the idea of overloading the --to flag rather than having
> a new --map-machines flag. It's concise and fits well, I think - we
> want the machines in this bundle to mapped *to* the machines we're
> specifying here.
>
> I like the thrust of Tim's suggestion for "existing" but I'm not
> entirely sure about that word - it's quite long and it doesn't quite
> express the identity relationship that I see there. How about "same"?
>
> For example:
>
>  juju deploy --to 1=2,2=3,same some-bundle
>
> Another possibility: use ellipses to imply the rest of the mapping:
>
> juju deploy --to 1=2,2=3,... some-bundle
> juju deploy --to ... some-bundle
>
>
>
> On 9 November 2017 at 02:43, Tim Penhey  wrote:
> >
> >
> > On 09/11/17 13:06, Mark Shuttleworth wrote:
> >> On 11/07/2017 03:11 PM, John Meinel wrote:
> >>> ...
> >>>
> >>>
> >>> > Perhaps just:
> >>> >
> >>> >   juju deploy --map-machines A=B,C=D
> >>> >
> >>> > ... or some variant of that?
> >>> >
> >>> > Let's use the betas to refine and condense and clarify.
> >>>
> >>> +1 to that. I'm wondering if use-existing-machines is ever
> appropriate
> >>> on its own, as the machine numbers in a model are ephemeral but
> >>> machine numbers in a bundle are static.
> >>>
> >>>
> >>> Feedback from Admins that one of their big use case really is for
> >>> bundle-a to lay down a definition/base charm across everything, and
> >>> bundle-b to be meant as an exact overlay, and all of the machine-ids
> >>> are exact matches. And having to specify 0=0,...50=50 is a lot of ugly
> >>> boilerplate.
> >>
> >> I would expect that --map-machines means that machine numbers correspond
> >> UNLESS remapped. So your ugly boilerplate is not needed.
> >
> > Been thinking more... how about this as a proposal:
> >
> > I think we can combine both the --use-existing-machines and the
> > --bundle-machine into the single --map-machines:
> >
> > So...
> >
> > To use the existing machines as is:
> >   --map-machines existing
> >
> > To only map two machines,
> >   --map-machines 1=2,2=3
> >
> > To use existing, and map two machines
> >   --map-machines existing,1=2,2=3
> >
> > Thoughts?
> >
> > Tim
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm
> an/listinfo/juju-dev
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-09 Thread roger peppe 
I still like the idea of overloading the --to flag rather than having
a new --map-machines flag. It's concise and fits well, I think - we
want the machines in this bundle to mapped *to* the machines we're
specifying here.

I like the thrust of Tim's suggestion for "existing" but I'm not
entirely sure about that word - it's quite long and it doesn't quite
express the identity relationship that I see there. How about "same"?

For example:

 juju deploy --to 1=2,2=3,same some-bundle

Another possibility: use ellipses to imply the rest of the mapping:

juju deploy --to 1=2,2=3,... some-bundle
juju deploy --to ... some-bundle



On 9 November 2017 at 02:43, Tim Penhey  wrote:
>
>
> On 09/11/17 13:06, Mark Shuttleworth wrote:
>> On 11/07/2017 03:11 PM, John Meinel wrote:
>>> ...
>>>
>>>
>>> > Perhaps just:
>>> >
>>> >   juju deploy --map-machines A=B,C=D
>>> >
>>> > ... or some variant of that?
>>> >
>>> > Let's use the betas to refine and condense and clarify.
>>>
>>> +1 to that. I'm wondering if use-existing-machines is ever appropriate
>>> on its own, as the machine numbers in a model are ephemeral but
>>> machine numbers in a bundle are static.
>>>
>>>
>>> Feedback from Admins that one of their big use case really is for
>>> bundle-a to lay down a definition/base charm across everything, and
>>> bundle-b to be meant as an exact overlay, and all of the machine-ids
>>> are exact matches. And having to specify 0=0,...50=50 is a lot of ugly
>>> boilerplate.
>>
>> I would expect that --map-machines means that machine numbers correspond
>> UNLESS remapped. So your ugly boilerplate is not needed.
>
> Been thinking more... how about this as a proposal:
>
> I think we can combine both the --use-existing-machines and the
> --bundle-machine into the single --map-machines:
>
> So...
>
> To use the existing machines as is:
>   --map-machines existing
>
> To only map two machines,
>   --map-machines 1=2,2=3
>
> To use existing, and map two machines
>   --map-machines existing,1=2,2=3
>
> Thoughts?
>
> Tim

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-08 Thread Tim Penhey 
On 09/11/17 13:06, Mark Shuttleworth wrote:
> On 11/07/2017 03:11 PM, John Meinel wrote:
>> ...
>>  
>>
>> > Perhaps just:
>> >
>> >   juju deploy --map-machines A=B,C=D
>> >
>> > ... or some variant of that?
>> >
>> > Let's use the betas to refine and condense and clarify.
>>
>> +1 to that. I'm wondering if use-existing-machines is ever appropriate
>> on its own, as the machine numbers in a model are ephemeral but
>> machine numbers in a bundle are static.
>>
>>
>> Feedback from Admins that one of their big use case really is for
>> bundle-a to lay down a definition/base charm across everything, and
>> bundle-b to be meant as an exact overlay, and all of the machine-ids
>> are exact matches. And having to specify 0=0,...50=50 is a lot of ugly
>> boilerplate.
> 
> I would expect that --map-machines means that machine numbers correspond
> UNLESS remapped. So your ugly boilerplate is not needed.

Part of the problem with this is that this isn't how any of the flags
work just now.

All flags are either bool flags, which take no args, or other types that
*always* take args.

Now this isn't to say that we couldn't force something, but it isn't as
intuitive.

We could make
  --map-machines
work by itself, and we could make
  --map-machines=1=2,3=4
work, but the underlying flag parsing library wouldn't like:
  --map-machines 1=2,3=4

Because if you are a "bool" type flag, you can work without any args, or
work with an explicit arg, the second case, but it doesn't work with the
third.

Tim

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-08 Thread Mark Shuttleworth 
On 11/07/2017 03:11 PM, John Meinel wrote:
> ...
>  
>
> > Perhaps just:
> >
> >   juju deploy --map-machines A=B,C=D
> >
> > ... or some variant of that?
> >
> > Let's use the betas to refine and condense and clarify.
>
> +1 to that. I'm wondering if use-existing-machines is ever appropriate
> on its own, as the machine numbers in a model are ephemeral but
> machine numbers in a bundle are static.
>
>
> Feedback from Admins that one of their big use case really is for
> bundle-a to lay down a definition/base charm across everything, and
> bundle-b to be meant as an exact overlay, and all of the machine-ids
> are exact matches. And having to specify 0=0,...50=50 is a lot of ugly
> boilerplate.

I would expect that --map-machines means that machine numbers correspond
UNLESS remapped. So your ugly boilerplate is not needed.

Mark
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-06 Thread John Meinel 
...


> > Perhaps just:
> >
> >   juju deploy --map-machines A=B,C=D
> >
> > ... or some variant of that?
> >
> > Let's use the betas to refine and condense and clarify.
>
> +1 to that. I'm wondering if use-existing-machines is ever appropriate
> on its own, as the machine numbers in a model are ephemeral but
> machine numbers in a bundle are static.
>
>
Feedback from Admins that one of their big use case really is for bundle-a
to lay down a definition/base charm across everything, and bundle-b to be
meant as an exact overlay, and all of the machine-ids are exact matches.
And having to specify 0=0,...50=50 is a lot of ugly boilerplate.
While a plethora of options isn't great, addressing a significant use case
is.

I think we need to keep in mind that there really are 2 distinct use cases
for bundles:
1) Sharable snapshots of a particular application
2) Descriptions of an entire deployment, that people want to be able to
reproduce. (Teardown, redo, understand the description of an existing
deployment, etc.)

For things like the charm store, (1) is common, and the use cases around
machines being an abstract idea is common.
For many customer engagements, we end up with bundles that are very
specific to the site.

The '--use-existing-machines' is meant to flip the logic around
machine-identities are purely abstract, to machine identities are explicit
unless otherwise changed.

I agree that it would be good to expand upon the ability to have abstract
identifiers for machines in bundles, so that you don't end up with
1=2,2=3,3=1 sort of scenarios.

John
=:->
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-06 Thread roger peppe 
On 6 November 2017 at 17:24, roger peppe  wrote:
> juju deploy --to 3=0,4=3

Also, looking further forward, I'd like to see more informative
machine names in bundles, because with a command line like the above,
it's not clear what the purpose of the assigned machines might be.
It'd be nicer, I think, to be able to say:

juju deploy --to database=0,backend=1

The main awkwardness with that is that it's already possible to use
application names in the "to" field of a bundle's application
specification, so there's an ambiguity. That could be resolved by
using some disambiguating syntax in the bundle (e.g. a leading '#' on
machine names).

So a bundle might look like:

   machines:
   database:
   backend:
   applications:
   wordpress:
   charm: wordpress
   num_units: 1
   to: '#database'
   mysql:
   charm: mysql
   num_units: 1
   to: '#backend'
   relations:
   - ["wordpress:db", "mysql:server"]


  rog.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-06 Thread roger peppe 
On 2 November 2017 at 07:16, Mark Shuttleworth  wrote:
> On 11/02/2017 04:56 AM, Chris Lee wrote:
>
> A new development release of Juju is here, 2.3-beta2.
>
>
> 2.3 is looking great, and is worth a test run for those of you with larger
> models and an interest in cross-model relations.
>
>
> ## New and Improved
>
> * Deploying bundles can now target existing machines
>
>juju deploy some-bundle --use-existing-machines --bundle-machine 3=4
>
>
> From a D-R-Y perspective, we --dont --need --all --those --options :)
>
> Perhaps just:
>
>   juju deploy --map-machines A=B,C=D
>
> ... or some variant of that?
>
> Let's use the betas to refine and condense and clarify.

+1 to that. I'm wondering if use-existing-machines is ever appropriate
on its own, as the machine numbers in a model are ephemeral but
machine numbers in a bundle are static.

ISTM that there's a strong connection between specifying the machines that
a bundle is to be run on and specifying the machine that a newly deployed
service will run on. Going with that thought, how about

juju deploy --to 3=0,4=3

and losing the --use-existing-machines flag? That way there are
no new flags at all for this feature and the intent of existing flags
is still preserved.

  rog.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-02 Thread Ian Booth 

> * Parallelization of the Machine Provisioner
>>
>> Provisioning of machines is now faster!  Groups of machines will now be
>> provisioned in parallel reducing deployment time, especially on large
>> bundles.  Please give it a try and let us know what you think.
>>
>> Benchmarks for time to deploy 16 machines on different clouds:
>>
>> AWS:
>>
>> juju 2.2.5 4m36s
>>
>> juju 2.3-beta2 3m17s
>>
>> LXD:
>>
>> juju 2.2.5 3m57s
>>
>> juju 2.3-beta2 2m57s
>>
>> Google:
>>
>> juju 2.2.5 5m21s
>>
>> juju 2.3-beta2 2m10s
>>
>> OpenStack:
>>
>> juju 2.2.5 12m40s
>>
>> juju 2.3-beta2 4m52s
>>
>>
>>
> Oh heck yes this is a great improvement! I don't see MAAS numbers here, but
> I imagine palatalization has been implemented there too? Bare metal can be
> so slow to boot sometimes ;)
>

Works for all clouds. The provisioning code is generic and has been extracted
from each provider and moved up a layer. It got complicated because of the need
to still ensure even spread of distribution groups across availability zones in
the parallel case. There just wasn't time to get any MAAS numbers prior to
cutting the beta, but empirically, there's improvement across the board.
Positive deployment stories to share would be welcome :-)




-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-02 Thread Marco Ceppi 
On Thu, Nov 2, 2017 at 12:57 AM Chris Lee  wrote:

> A new development release of Juju is here, 2.3-beta2.
>

Woop woop!


> * Autoconfiguration of FAN networking for EC2 and GCE providers
>
> When creating a model in a VPC environment on EC2 or on GCE FAN settings
> (model-config fan-config and container-networking-method) will be
> autoconfigured, and container networking will work out-of-the-box.
>

This means we can get a hyper converged Kuberentes on public cloud.
Lowering our 9 node HA deployment to just three! I plan on banging on this
for a the beta to make sure there aren't any oddities.

* Parallelization of the Machine Provisioner
>
> Provisioning of machines is now faster!  Groups of machines will now be
> provisioned in parallel reducing deployment time, especially on large
> bundles.  Please give it a try and let us know what you think.
>
> Benchmarks for time to deploy 16 machines on different clouds:
>
> AWS:
>
> juju 2.2.5 4m36s
>
> juju 2.3-beta2 3m17s
>
> LXD:
>
> juju 2.2.5 3m57s
>
> juju 2.3-beta2 2m57s
>
> Google:
>
> juju 2.2.5 5m21s
>
> juju 2.3-beta2 2m10s
>
> OpenStack:
>
> juju 2.2.5 12m40s
>
> juju 2.3-beta2 4m52s
>
>
>
Oh heck yes this is a great improvement! I don't see MAAS numbers here, but
I imagine palatalization has been implemented there too? Bare metal can be
so slow to boot sometimes ;)
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-02 Thread Ian Booth 


>>
>> * Parallelization of the Machine Provisioner
>>
>>
>> Provisioning of machines is now faster!  Groups of machines will now
>> be provisioned in parallel reducing deployment time, especially on
>> large bundles.  Please give it a try and let us know what you think.
>>
> 
> This is great. Did we also add support for automatic provisioning
> retries to handle sporadic cloud failures?
>

Some providers do have some such retries built in. eg Azure, Openstack,
Rackspace handle rate limit exceeded errors and Do The Right Thing. We're still
progressively addressing robustness concerns elsewhere.



-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 2.3 beta2 is here!

2017-11-02 Thread Mark Shuttleworth 
On 11/02/2017 04:56 AM, Chris Lee wrote:
>
> A new development release of Juju is here, 2.3-beta2.
>

2.3 is looking great, and is worth a test run for those of you with
larger models and an interest in cross-model relations.


> ## New and Improved
>
> * Deploying bundles can now target existing machines
>
>    juju deploy some-bundle --use-existing-machines --bundle-machine 3=4
>

From a D-R-Y perspective, we --dont --need --all --those --options :)

Perhaps just:

  juju deploy --map-machines A=B,C=D

... or some variant of that?

Let's use the betas to refine and condense and clarify.


>
> * Cross model relations now supports nagios in its own [controller:]model
>
>
> The nagios and nrpe charms are updated to be cross model relations aware.
>

\o/ that's going to be very popular very quickly :)

> This means they work in either a single model or cross model scenario.
> For now, the charms are available at cs:~wallyworld but will be
> published to the store soon.
>

This reminds me, let's action the move from the LP-style branches
('~wallyworld/') to snap-style branches ('foo/stable/wally-test')


> * Autoconfiguration of FAN networking for EC2 and GCE providers
>
>
> When creating a model in a VPC environment on EC2 or on GCE FAN
> settings (model-config fan-config and container-networking-method)
> will be autoconfigured, and container networking will work
> out-of-the-box.
>
>
> * Parallelization of the Machine Provisioner
>
>
> Provisioning of machines is now faster!  Groups of machines will now
> be provisioned in parallel reducing deployment time, especially on
> large bundles.  Please give it a try and let us know what you think.
>

This is great. Did we also add support for automatic provisioning
retries to handle sporadic cloud failures?

Mark
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev