Re: Upgrading minimum Go version?

[Tim Penhey]

On 27/11/15 08:43, Michael Hudson-Doyle wrote:
> On 27 November 2015 at 02:24, Martin Packman
>  wrote:
>> On 26/11/2015, Andrew Wilkins  wrote:
>>> Hi (mostly Curtis),
>>>
>>> Is there a plan to bump the minimum Go version? Some of our dependencies do
>>> not build with Go 1.2. The LXD provider only builds with Go 1.3 (I think?),
>>> and I've got a PR up that updates the azure-sdk-for-go dependency, but it's
>>> blocked because the newer doesn't build with Go 1.2.
> 
> Is this something we've done to ourselves or is there a third-party
> library we're depending on that doesn't work with Go 1.2?

The two main ones I know about are lxd and the new azure go bindings.

We have worked around other issues in other library code, like some of
the x/net stuff.

Tim


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Upgrading minimum Go version?

[David Cheney]

Martin: Can you setup a unit test job that uses Go 1.5 ? I will apply
the same process I did for the -race job, to get the build voting,
then reenable the tests that were skipped.

On Fri, Nov 27, 2015 at 12:24 AM, Martin Packman
 wrote:
> On 26/11/2015, Andrew Wilkins  wrote:
>> Hi (mostly Curtis),
>>
>> Is there a plan to bump the minimum Go version? Some of our dependencies do
>> not build with Go 1.2. The LXD provider only builds with Go 1.3 (I think?),
>> and I've got a PR up that updates the azure-sdk-for-go dependency, but it's
>> blocked because the newer doesn't build with Go 1.2.
>
> There are two issues with requiring a newer go version:
>
> * The test suite needs to actually pass reliably with go 1.5
> * How we get a version of juju that doesn't build with go 1.2 into
> trusty needs resolving
>
> With putting the newest juju into older Ubuntu releases, our strategy
> so far has been to SRU back the same source as we put in the current
> release, which we have a special allowance for. We cannot SRU a newer
> go toolchain over the existing working 1.2 version, for hopefully
> obvious reasons. Someone else may have better ideas, but I think that
> leaves us basically three options:
>
> * Stop putting new jujus in older ubuntus
> * Use backports rather than -updates (this is what lxd does I believe)
> * SRU a newer go toolchain to some alternative, juju only location
>
> All of that is pretty big process changes that we can't just go ahead and do.
>
> Test failures have been gradually tackled, we're down to a small-ish
> set of issues causing reliable red results when we run unit tests on
> vivid(1.3) and wily/xenial(1.5).
>
> "TestContainerProvisionerStarted did not start"
> http://reports.vapour.ws/releases/issue/5614b0e7749a5613c5aff2e0
>
> "TestAPI2ResultError fails"
> http://reports.vapour.ws/releases/issue/5640b28f749a562a3f656b39
>
> "unexpected message panic"
> http://reports.vapour.ws/releases/issue/55f34dfd749a561a11823b0b
>
> Seem to be the ones I see most in recent test failures.
>
> Martin
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Upgrading minimum Go version?

[Martin Packman]

On 26/11/2015, David Cheney  wrote:
> Martin: Can you setup a unit test job that uses Go 1.5 ? I will apply
> the same process I did for the -race job, to get the build voting,
> then reenable the tests that were skipped.

We've had that since near the start of the wily development cycle, now
we have two:

http://juju-ci.vapour.ws/job/run-unit-tests-wily-amd64/
http://juju-ci.vapour.ws/job/run-unit-tests-xenial-amd64/

This is basically inherited pain from never making the tests work on
vivid (go 1.3) so the job could never become voting.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Upgrading minimum Go version?

[Michael Hudson-Doyle]

On 27 November 2015 at 09:39, Tim Penhey  wrote:
> On 27/11/15 08:43, Michael Hudson-Doyle wrote:
>> On 27 November 2015 at 02:24, Martin Packman
>>  wrote:
>>> On 26/11/2015, Andrew Wilkins  wrote:
 Hi (mostly Curtis),

 Is there a plan to bump the minimum Go version? Some of our dependencies do
 not build with Go 1.2. The LXD provider only builds with Go 1.3 (I think?),
 and I've got a PR up that updates the azure-sdk-for-go dependency, but it's
 blocked because the newer doesn't build with Go 1.2.
>>
>> Is this something we've done to ourselves or is there a third-party
>> library we're depending on that doesn't work with Go 1.2?
>
> The two main ones I know about are lxd and the new azure go bindings.

By the azure go bindings you mean something Canonical didn't write,
like https://github.com/Azure/azure-sdk-for-go? That sort of thing
sounds like a good argument for the 1.5-in-trusty SRU thing.

> We have worked around other issues in other library code, like some of
> the x/net stuff.

Do you have a nice list of these workarounds? :-)

Cheers,
mwh

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Upgrading minimum Go version?

[Andrew Wilkins]

On Fri, Nov 27, 2015 at 7:49 AM Michael Hudson-Doyle <
michael.hud...@canonical.com> wrote:

> On 27 November 2015 at 09:39, Tim Penhey  wrote:
> > On 27/11/15 08:43, Michael Hudson-Doyle wrote:
> >> On 27 November 2015 at 02:24, Martin Packman
> >>  wrote:
> >>> On 26/11/2015, Andrew Wilkins  wrote:
>  Hi (mostly Curtis),
> 
>  Is there a plan to bump the minimum Go version? Some of our
> dependencies do
>  not build with Go 1.2. The LXD provider only builds with Go 1.3 (I
> think?),
>  and I've got a PR up that updates the azure-sdk-for-go dependency,
> but it's
>  blocked because the newer doesn't build with Go 1.2.
> >>
> >> Is this something we've done to ourselves or is there a third-party
> >> library we're depending on that doesn't work with Go 1.2?
> >
> > The two main ones I know about are lxd and the new azure go bindings.
>
> By the azure go bindings you mean something Canonical didn't write,
> like https://github.com/Azure/azure-sdk-for-go? That sort of thing
> sounds like a good argument for the 1.5-in-trusty SRU thing.
>

Yes, that's the one. They depend on, and are vendoring, a version of
go.crypto that doesn't build with Go 1.2.

> We have worked around other issues in other library code, like some of
> > the x/net stuff.
>
> Do you have a nice list of these workarounds? :-)
>
> Cheers,
> mwh
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

[ANN] charm-tools 1.9.3

[Simon Davy]

On Thursday, 26 November 2015, Marco Ceppi 
wrote:
> On Wed, Nov 25, 2015 at 4:08 PM Simon Davy  wrote:
>>
>> On 25 November 2015 at 16:02, Marco Ceppi 
wrote:
>> > ## Wheel House for layer dependencies
>> >
>> > Going forward we recommend all dependencies for layers and charms be
>> > packaged in a wheelhouse.txt file. This perform the installation of
pypi
>> > packages on the unit instead of first on the local machine meaning
Python
>> > libraries that require architecture specific builds will do it on the
units
>> > architecture.
>>
>> If I'm understanding the above correctly, this approach is a blocker for
us.
>>
>> We would not want to install direct from pypi on a production service
>>
>>  1) pypi packages are not signed (or when they are, pip doesn't verify
>> the signature)
>>  2) pypi is an external dependency and thus unreliable (although not
>> as bad these days)
>>  3) old versions can disappear from pypi at an authors whim.
>>  4) installing c packages involves installing a c toolchain on your prod
machine
>>
>> Additionally, our policy (Canonical's, that is), does not allow access
>> to the internet on production machines, for very good reasons. This is
>> the default policy in many (probably most) production environments.
>>
>> Any layer or charm that consumes a layer that uses this new approach
>> for dependencies would thus be unusable to us :(
>>
>> It also harms repeatability, and I would not want to use it even if
>> our access policy allowed access to pypi.
>>
>> For python charm dependencies, we use system python packages as much
>> as possible, or if we need any wheels, we ship that wheel in the
>> charm, and pip install it directly from the there. No external
>> network, completely repeatable.
>
> So, allow me to clarify. If you review the pastebin outputs from the
original announcement email, what this shift does is previously `charm
build` would create and embed installed dependencies into the charm under
lib/ much like charm-helper-sync did for instead for any arbitrary Pypi
dependency. Issues there are for PyYAML it will build a yaml.so file which
would be built based on the architecture of your machine and not the cloud.

Right. This was the bit which confused me, I think.

Can we not just use python-yaml, as its installed by default on cloud
images anyway?

We use virtualenv with --system-site-packages, and use system packages for
python libs with c packages where possible, leaving wheels for things which
aren't packaged or we want newer versions of.

> This new method builds source wheels and embeds the wheel in the charm.
There's a bootstrap process on deploy that will unpackage and install the
dependencies on the system when deployed. The deps are still bundled in the
charm just the output of the charm is much more sane and easier to read
>
>>
>> Another option is to require/configure a local pypi to pull the
>> packages from, but  again, an external dependency and spof.
>>
>> I much prefer what the current tool seems to do, bundle deps as wheels
>> into a wheels/ dir as part of the charm build process.  Then that
>> charm is self-contained, and requires no external access, and is more
>> reliable/repeatable.
>>
>> > This also provides the added bonus of making `charm layers` a
>> > much cleaner experience.
>> >
>> > Here's an example of side-by-side output of a charm build of the basic
layer
>> > before and after converting to Wheelhouse.
>> >
>> > Previous: http://paste.ubuntu.com/13502779/ (53 directories, 402 files)
>> > Wheelhouse:  http://paste.ubuntu.com/13502779// (3 directories, 21
files)
>>
>> These are the same link?
>>
>> But looking at the link, I much prefer that version - everything is
>> bundled with the charm, as I suggest above.
>
> Sorry, meant to send two links.
> The first: http://paste.ubuntu.com/13502779/
> The Second: http://paste.ubuntu.com/13511384/
> Now which one would you prefer :)

Great :) Problem solved, sorry for the noise.



-- 
Simon
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju