Re: New juju in ubuntu

[Martin Packman]

On 06/04/2016, Stuart Bishop  wrote:
>
> How do our plugins know what version of juju is in play? Can they
> assume that the 'juju' binary found on the path is the juju that
> invoked the plugin, or is there some other way to tell using
> environment variables or such? Or will all the juju plugins just fail
> if they are invoked from the non-default juju version?

The new packaging uses wrapping scripts for 'juju-1' and 'juju-2.0'
that prepend PATH with the version-specific directories. The default
/usr/bin/juju is just a symlink though, so invoking outside the
context of the wrappers will still be ambiguous. I think we do need a
better solution here.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New juju in ubuntu

[Stuart Bishop]

On 7 April 2016 at 16:46, roger peppe  wrote:
> On 7 April 2016 at 10:17, Stuart Bishop  wrote:
>> On 7 April 2016 at 16:03, roger peppe  wrote:
>>> On 7 April 2016 at 09:38, Tim Penhey  wrote:
 We could probably set an environment variable for the plugin called
 JUJU_BIN that is the juju that invoked it.

 Wouldn't be too hard.
>>>
>>> How does that stop old plugins failing because the new juju is trying
>>> to use them?
>>>
>>> An alternative possibility: name all new plugins with the prefix "juju2-" 
>>> rather
>>> than "juju".
>>
>> I've opened https://bugs.launchpad.net/juju-core/+bug/1567296 to track this.
>>
>> Prepending the $PATH is not hard either - just override the
>> environment in the exec() call.
>>
>> The nicest approach may be to not use 'juju1', 'juju2' and 'juju' but
>> instead just 'juju'. It would be a thin wrapper that sets the $PATH
>> and invokes the correct binary based on some configuration such as an
>> environment variable. This would fix plugins, and lots of other stuff
>> that are about to break too such as deployment scripts, test suites
>> etc.
>
> There are actually two problems here. One is the fact that plugins
> use the Juju binary. For that, setting the PATH might well be the right thing.
>
> But there's also a problem with other plugins that use the Juju API
> directly (they might be written in Go, for example) and therefore
> implicitly assume the that they're talking to a juju 1 or juju 2 environment.
> Since local configuration files have changed and the API has changed, it's
> important that a plugin written for Go 1 won't be invoked by a juju 2
> binary.

If juju 2.x changed the plugin prefix from juju- to juju2-, that would
also solve the issue of juju 2.x specific plugins showing up in juju
1.x's command line help and vice versa.

-- 
Stuart Bishop 

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New juju in ubuntu

[roger peppe]

On 7 April 2016 at 10:17, Stuart Bishop  wrote:
> On 7 April 2016 at 16:03, roger peppe  wrote:
>> On 7 April 2016 at 09:38, Tim Penhey  wrote:
>>> We could probably set an environment variable for the plugin called
>>> JUJU_BIN that is the juju that invoked it.
>>>
>>> Wouldn't be too hard.
>>
>> How does that stop old plugins failing because the new juju is trying
>> to use them?
>>
>> An alternative possibility: name all new plugins with the prefix "juju2-" 
>> rather
>> than "juju".
>
> I've opened https://bugs.launchpad.net/juju-core/+bug/1567296 to track this.
>
> Prepending the $PATH is not hard either - just override the
> environment in the exec() call.
>
> The nicest approach may be to not use 'juju1', 'juju2' and 'juju' but
> instead just 'juju'. It would be a thin wrapper that sets the $PATH
> and invokes the correct binary based on some configuration such as an
> environment variable. This would fix plugins, and lots of other stuff
> that are about to break too such as deployment scripts, test suites
> etc.

There are actually two problems here. One is the fact that plugins
use the Juju binary. For that, setting the PATH might well be the right thing.

But there's also a problem with other plugins that use the Juju API
directly (they might be written in Go, for example) and therefore
implicitly assume the that they're talking to a juju 1 or juju 2 environment.
Since local configuration files have changed and the API has changed, it's
important that a plugin written for Go 1 won't be invoked by a juju 2
binary.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New juju in ubuntu

[Stuart Bishop]

On 7 April 2016 at 16:03, roger peppe  wrote:
> On 7 April 2016 at 09:38, Tim Penhey  wrote:
>> We could probably set an environment variable for the plugin called
>> JUJU_BIN that is the juju that invoked it.
>>
>> Wouldn't be too hard.
>
> How does that stop old plugins failing because the new juju is trying
> to use them?
>
> An alternative possibility: name all new plugins with the prefix "juju2-" 
> rather
> than "juju".

I've opened https://bugs.launchpad.net/juju-core/+bug/1567296 to track this.

Prepending the $PATH is not hard either - just override the
environment in the exec() call.

The nicest approach may be to not use 'juju1', 'juju2' and 'juju' but
instead just 'juju'. It would be a thin wrapper that sets the $PATH
and invokes the correct binary based on some configuration such as an
environment variable. This would fix plugins, and lots of other stuff
that are about to break too such as deployment scripts, test suites
etc.

-- 
Stuart Bishop 

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New juju in ubuntu

[roger peppe]

On 7 April 2016 at 09:38, Tim Penhey  wrote:
> We could probably set an environment variable for the plugin called
> JUJU_BIN that is the juju that invoked it.
>
> Wouldn't be too hard.

How does that stop old plugins failing because the new juju is trying
to use them?

An alternative possibility: name all new plugins with the prefix "juju2-" rather
than "juju".

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New juju in ubuntu

[Tim Penhey]

We could probably set an environment variable for the plugin called
JUJU_BIN that is the juju that invoked it.

Wouldn't be too hard.

Tim

On 07/04/16 17:25, Stuart Bishop wrote:
> On 7 April 2016 at 03:55, Marco Ceppi  wrote:
>>
>> On Wed, Apr 6, 2016 at 10:07 AM Stuart Bishop 
>> wrote:
>>>
>>> On 5 April 2016 at 23:35, Martin Packman 
>>> wrote:
>>>
 The challenge here is we want Juju 2.0 and all the new functionality
 to be the default on release, but not break our existing users who
 have working Juju 1.X environments and no deployment upgrade path yet.
 So, versions 1 and 2 have to be co-installable, and when upgrading to
 xenial users should get the new version without their existing working
 juju being removed.

 There are several ways to accomplish that, but based on feedback from
 the release team, we switched from using update-alternatives to having
 'juju' on xenial always be 2.0, and exposing the 1.X client via a
 'juju-1' binary wrapper. Existing scripts can either be changed to use
 the new name, or add the version-specific binaries directory
 '/var/lib/juju-1.25/bin' to the path.
>>>
>>> How do our plugins know what version of juju is in play? Can they
>>> assume that the 'juju' binary found on the path is the juju that
>>> invoked the plugin, or is there some other way to tell using
>>> environment variables or such? Or will all the juju plugins just fail
>>> if they are invoked from the non-default juju version?
>>
>>
>> You can invoke `juju version` from within the plugin and parse the output.
>> That's what I've been doing when I need to distinguish functionality.
> 
> That seems fine if you are invoking the plugin from the default
> unnumbered 'juju'. But running 'juju2 wait' will mean that juju-wait
> will be executing juju 1.x commands and fail. And conversely running
> 'juju1 wait' will invoke juju 2.x and probably fail.
> 
> I think the plugin API needs to be extended to support allowing
> multiple juju versions to coexist. An environment variable would do
> the trick but require every plugin to be fixed. Altering $PATH so
> 'juju' runs the correct juju would allow existing plugins to run
> unmodified (the bulk of them will work with both juju1 and juju2,
> since the cli is similar enough that many plugins will work
> unmodified.
> 


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New juju in ubuntu

[Stuart Bishop]

On 7 April 2016 at 03:55, Marco Ceppi  wrote:
>
> On Wed, Apr 6, 2016 at 10:07 AM Stuart Bishop 
> wrote:
>>
>> On 5 April 2016 at 23:35, Martin Packman 
>> wrote:
>>
>> > The challenge here is we want Juju 2.0 and all the new functionality
>> > to be the default on release, but not break our existing users who
>> > have working Juju 1.X environments and no deployment upgrade path yet.
>> > So, versions 1 and 2 have to be co-installable, and when upgrading to
>> > xenial users should get the new version without their existing working
>> > juju being removed.
>> >
>> > There are several ways to accomplish that, but based on feedback from
>> > the release team, we switched from using update-alternatives to having
>> > 'juju' on xenial always be 2.0, and exposing the 1.X client via a
>> > 'juju-1' binary wrapper. Existing scripts can either be changed to use
>> > the new name, or add the version-specific binaries directory
>> > '/var/lib/juju-1.25/bin' to the path.
>>
>> How do our plugins know what version of juju is in play? Can they
>> assume that the 'juju' binary found on the path is the juju that
>> invoked the plugin, or is there some other way to tell using
>> environment variables or such? Or will all the juju plugins just fail
>> if they are invoked from the non-default juju version?
>
>
> You can invoke `juju version` from within the plugin and parse the output.
> That's what I've been doing when I need to distinguish functionality.

That seems fine if you are invoking the plugin from the default
unnumbered 'juju'. But running 'juju2 wait' will mean that juju-wait
will be executing juju 1.x commands and fail. And conversely running
'juju1 wait' will invoke juju 2.x and probably fail.

I think the plugin API needs to be extended to support allowing
multiple juju versions to coexist. An environment variable would do
the trick but require every plugin to be fixed. Altering $PATH so
'juju' runs the correct juju would allow existing plugins to run
unmodified (the bulk of them will work with both juju1 and juju2,
since the cli is similar enough that many plugins will work
unmodified.

-- 
Stuart Bishop 

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New juju in ubuntu

[Marco Ceppi]

On Wed, Apr 6, 2016 at 10:07 AM Stuart Bishop 
wrote:

> On 5 April 2016 at 23:35, Martin Packman 
> wrote:
>
> > The challenge here is we want Juju 2.0 and all the new functionality
> > to be the default on release, but not break our existing users who
> > have working Juju 1.X environments and no deployment upgrade path yet.
> > So, versions 1 and 2 have to be co-installable, and when upgrading to
> > xenial users should get the new version without their existing working
> > juju being removed.
> >
> > There are several ways to accomplish that, but based on feedback from
> > the release team, we switched from using update-alternatives to having
> > 'juju' on xenial always be 2.0, and exposing the 1.X client via a
> > 'juju-1' binary wrapper. Existing scripts can either be changed to use
> > the new name, or add the version-specific binaries directory
> > '/var/lib/juju-1.25/bin' to the path.
>
> How do our plugins know what version of juju is in play? Can they
> assume that the 'juju' binary found on the path is the juju that
> invoked the plugin, or is there some other way to tell using
> environment variables or such? Or will all the juju plugins just fail
> if they are invoked from the non-default juju version?
>

You can invoke `juju version` from within the plugin and parse the output.
That's what I've been doing when I need to distinguish functionality.
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New juju in ubuntu

[Stuart Bishop]

On 5 April 2016 at 23:35, Martin Packman  wrote:

> The challenge here is we want Juju 2.0 and all the new functionality
> to be the default on release, but not break our existing users who
> have working Juju 1.X environments and no deployment upgrade path yet.
> So, versions 1 and 2 have to be co-installable, and when upgrading to
> xenial users should get the new version without their existing working
> juju being removed.
>
> There are several ways to accomplish that, but based on feedback from
> the release team, we switched from using update-alternatives to having
> 'juju' on xenial always be 2.0, and exposing the 1.X client via a
> 'juju-1' binary wrapper. Existing scripts can either be changed to use
> the new name, or add the version-specific binaries directory
> '/var/lib/juju-1.25/bin' to the path.

How do our plugins know what version of juju is in play? Can they
assume that the 'juju' binary found on the path is the juju that
invoked the plugin, or is there some other way to tell using
environment variables or such? Or will all the juju plugins just fail
if they are invoked from the non-default juju version?

-- 
Stuart Bishop 

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New juju in ubuntu

[Marco Ceppi]

On Tue, Apr 5, 2016 at 1:37 PM Martin Packman 
wrote:

> On 05/04/2016, Adam Collard  wrote:
> > Will there be a transitional release of 1.25.x that also gives us a juju1
> > binary to facilitate this?
>
> Yes, the plan entails both an update to the juju in main for the 2.0
> release and a new juju 1.25 in universe that cooperates with the
> changes.
>
> For the debian packaging nitty gritty, what we had before was:
>
> Archive:
>   juju (depends juju-core)
> juju-core (/usr/bin/juju etc)
>   juju-local (depends juju-core, juju-mongodb, lxc bits)
>   juju-local-kvm (depends juju-core, juju-mongodb, libvirt bits)
>
> Devel PPA:
>   juju2 (/usr/bin/juju2 etc)
>
> We have the following proposed:
>
> Main:
>   juju (depends juju-2.0, suggests juju-core)
> juju-2.0 (/usr/lib/juju-2.0/bin/juju, linked as /usr/bin/juju and
> wrapped as /usr/bin/juju-2.0)
>
> (The lxd provider is bundled, replacing the local packaging bits for 2.0).
>
> Universe:
>   juju-core (transitional depends juju-1.25)
>   juju-1 (depends juju-1.25)
> juju-1.25 (/usr/lib/juju-1.25/bin/juju, wrapped as /usr/bin/juju-1)
>   juju-local (depends juju-1.25, juju-mongodb, lxc bits)
>   juju-local-kvm (depends juju-1.25, juju-mongodb, libvirt bits)
>
> When upgrading, you keep your juju package and upgrade juju-core to
> the new 1.X packaging, and pick up the new 2.0 package. With a fresh
> install, you just get 2.0 unless you ask for juju-1 as well or take
> the suggests.
>

A funny side-effect of this is `juju 1` and `juju 2.0` are now valid
commands since they follow the plugin format. Which means `juju 1 2.0
status` would invoke the juju-2.0 bin and run the `juju-1` bin which would
then run the `juju-2.0` bin again before finally querying status :)
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New juju in ubuntu

[Martin Packman]

On 05/04/2016, Adam Collard  wrote:
> Will there be a transitional release of 1.25.x that also gives us a juju1
> binary to facilitate this?

Yes, the plan entails both an update to the juju in main for the 2.0
release and a new juju 1.25 in universe that cooperates with the
changes.

For the debian packaging nitty gritty, what we had before was:

Archive:
  juju (depends juju-core)
juju-core (/usr/bin/juju etc)
  juju-local (depends juju-core, juju-mongodb, lxc bits)
  juju-local-kvm (depends juju-core, juju-mongodb, libvirt bits)

Devel PPA:
  juju2 (/usr/bin/juju2 etc)

We have the following proposed:

Main:
  juju (depends juju-2.0, suggests juju-core)
juju-2.0 (/usr/lib/juju-2.0/bin/juju, linked as /usr/bin/juju and
wrapped as /usr/bin/juju-2.0)

(The lxd provider is bundled, replacing the local packaging bits for 2.0).

Universe:
  juju-core (transitional depends juju-1.25)
  juju-1 (depends juju-1.25)
juju-1.25 (/usr/lib/juju-1.25/bin/juju, wrapped as /usr/bin/juju-1)
  juju-local (depends juju-1.25, juju-mongodb, lxc bits)
  juju-local-kvm (depends juju-1.25, juju-mongodb, libvirt bits)

When upgrading, you keep your juju package and upgrade juju-core to
the new 1.X packaging, and pick up the new 2.0 package. With a fresh
install, you just get 2.0 unless you ask for juju-1 as well or take
the suggests.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New juju in ubuntu

[Adam Collard]

Will there be a transitional release of 1.25.x that also gives us a juju1
binary to facilitate this?
On Tue, 5 Apr 2016 at 19:36, Martin Packman 
wrote:

> Some of you will be aware of the excitement over the last week getting
> Juju 2.0 into Xenial, this is an update mail so everyone knows what
> the goal is with the user experience.
>
> The challenge here is we want Juju 2.0 and all the new functionality
> to be the default on release, but not break our existing users who
> have working Juju 1.X environments and no deployment upgrade path yet.
> So, versions 1 and 2 have to be co-installable, and when upgrading to
> xenial users should get the new version without their existing working
> juju being removed.
>
> There are several ways to accomplish that, but based on feedback from
> the release team, we switched from using update-alternatives to having
> 'juju' on xenial always be 2.0, and exposing the 1.X client via a
> 'juju-1' binary wrapper. Existing scripts can either be changed to use
> the new name, or add the version-specific binaries directory
> '/var/lib/juju-1.25/bin' to the path.
>
> Because the meaning of 'juju' is changing, we want to give a
> on-first-run message about how to work with any existing environments
> the user may have. That work is covered by:
> 
> The best way to detect seems to be, when creating the new
> configuration for 2.0, if ~/.juju or JUJU_HOME is set, use the series
> to suggest ways to access or install the 1.X client.
>
> There were some other interesting points from review.
>
> The way we do plugins both leads to large binaries with a lot of
> duplication, and means the only sane way to deal with multiple juju
> versions is through path manipulation. If anyone is wants to
> distribute juju plugins, they'll need to watch out for this. See this
> bug on the binary sizes:
> 
>
> Also we've had to avoid stripping the juju binaries, which is
> otherwise standard distro practice, as it was unsupported and go 1.2
> breaks. With the switch to go 1.6 it seems like this may now work as
> expected, but needs to go through testing:
> 
>
> We also have some ongoing questions over golang packaging in the
> distro as a whole, but have demonstrated we an split out and create
> seperate golang-*-dev packages for nearly all the juju dependencies as
> needed.
>
> We hope to avoid most of the pain from this packaging process next
> cycle by putting our alpha releases into the devel disto (Y 16.10),
> rather than just adding them to our PPA. This will not need to replace
> the 2.0 version if we have compatibility concerns with early versions
> given the packaging changes from this cycle, but we will also be back
> to no-breakage mode as well.
>
> Martin
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev