Re: Automatic commit squashing

[David Cheney]

I thought feature branches, like communism, sounded good but had
failed in practice.

On Thu, Jun 16, 2016 at 7:06 PM, Horacio Duran
<horacio.du...@canonical.com> wrote:
> On second thought, this might be a problem for feature branches but we can
> device a way to tell the bot that something is a fb
>
>
> On Thursday, 16 June 2016, Horacio Duran <horacio.du...@canonical.com>
> wrote:
>>
>> +1 on Dave's suggestion
>>
>> On Thursday, 16 June 2016, David Cheney <david.che...@canonical.com>
>> wrote:
>>>
>>> Counter suggestion: the bot refuses to accept PR's that contain more
>>> than one commit, then it's up to the submitter to prepare it in any
>>> way that they feel appropriate.
>>>
>>> On Thu, Jun 16, 2016 at 6:44 PM, roger peppe <roger.pe...@canonical.com>
>>> wrote:
>>> > Squashed commits are nice, but there's something worth watching
>>> > out for: currently the merge commit is committed with the text
>>> > that's in the github PR, but when a squashed commit is made, this
>>> > text is ignored and only the text in the actual proposed commit ends up
>>> > in the history. This surprised me (I often edit the PR description
>>> > as the review continues) so worth being aware of, I think.
>>> >
>>> >   cheers,
>>> > rog.
>>> >
>>> > On 16 June 2016 at 02:12, Menno Smits <menno.sm...@canonical.com>
>>> > wrote:
>>> >> Hi everyone,
>>> >>
>>> >> Following on from the recent thread about commit squashing and commit
>>> >> message quality, the idea of automatically squashing commit at merge
>>> >> time
>>> >> has been raised. The idea is that the merge bot would automatically
>>> >> squash
>>> >> commits for a pull request into a single commit, using the PR
>>> >> description as
>>> >> the commit message.
>>> >>
>>> >> With this in place, developers can commit locally using any approach
>>> >> they
>>> >> prefer. The smaller commits they make as they work won't be part of
>>> >> the
>>> >> history the team interacts with in master.
>>> >>
>>> >> When using autosquashing the quality of pull request descriptions
>>> >> should get
>>> >> even more scrutiny during reviews. The quality of PR descriptions is
>>> >> already
>>> >> important as they are used for merge commits but with autosquashing in
>>> >> place
>>> >> they will be the *only* commit message.
>>> >>
>>> >> Autosquashing can be achieved technically by either having the merge
>>> >> bot do
>>> >> the squashing itself, or by taking advantage of Github's feature to do
>>> >> this
>>> >> (currently in preview mode):
>>> >>
>>> >> https://developer.github.com/changes/2016-04-01-squash-api-preview/
>>> >>
>>> >> We need to ensure that the squashed commits are attributed to the
>>> >> correct
>>> >> author (i.e. not jujubot). I'm not sure what we do with pull requests
>>> >> which
>>> >> contain work from multiple authors. There doesn't seem to be an
>>> >> established
>>> >> approach for this.
>>> >>
>>> >> Thoughts?
>>> >>
>>> >> - Menno
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Juju-dev mailing list
>>> >> Juju-dev@lists.ubuntu.com
>>> >> Modify settings or unsubscribe at:
>>> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>> >>
>>> >
>>> > --
>>> > Juju-dev mailing list
>>> > Juju-dev@lists.ubuntu.com
>>> > Modify settings or unsubscribe at:
>>> > https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>>
>>> --
>>> Juju-dev mailing list
>>> Juju-dev@lists.ubuntu.com
>>> Modify settings or unsubscribe at:
>>> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Automatic commit squashing

[David Cheney]

Counter suggestion: the bot refuses to accept PR's that contain more
than one commit, then it's up to the submitter to prepare it in any
way that they feel appropriate.

On Thu, Jun 16, 2016 at 6:44 PM, roger peppe  wrote:
> Squashed commits are nice, but there's something worth watching
> out for: currently the merge commit is committed with the text
> that's in the github PR, but when a squashed commit is made, this
> text is ignored and only the text in the actual proposed commit ends up
> in the history. This surprised me (I often edit the PR description
> as the review continues) so worth being aware of, I think.
>
>   cheers,
> rog.
>
> On 16 June 2016 at 02:12, Menno Smits  wrote:
>> Hi everyone,
>>
>> Following on from the recent thread about commit squashing and commit
>> message quality, the idea of automatically squashing commit at merge time
>> has been raised. The idea is that the merge bot would automatically squash
>> commits for a pull request into a single commit, using the PR description as
>> the commit message.
>>
>> With this in place, developers can commit locally using any approach they
>> prefer. The smaller commits they make as they work won't be part of the
>> history the team interacts with in master.
>>
>> When using autosquashing the quality of pull request descriptions should get
>> even more scrutiny during reviews. The quality of PR descriptions is already
>> important as they are used for merge commits but with autosquashing in place
>> they will be the *only* commit message.
>>
>> Autosquashing can be achieved technically by either having the merge bot do
>> the squashing itself, or by taking advantage of Github's feature to do this
>> (currently in preview mode):
>>
>> https://developer.github.com/changes/2016-04-01-squash-api-preview/
>>
>> We need to ensure that the squashed commits are attributed to the correct
>> author (i.e. not jujubot). I'm not sure what we do with pull requests which
>> contain work from multiple authors. There doesn't seem to be an established
>> approach for this.
>>
>> Thoughts?
>>
>> - Menno
>>
>>
>>
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A cautionary tale - mgo asserts

[David Cheney]

You had me at "ruins mongodb", actually just "ruins'.

On Wed, Jun 8, 2016 at 9:04 PM, roger peppe  wrote:
> This is also relevant (but probably only for larger documents):
>
> https://jeremywsherman.com/blog/2013/04/23/key-reordering-ruins-mongodb/
>
> Another reason to avoid entire-subdocument matches.
>
> On 8 June 2016 at 10:42, Menno Smits  wrote:
>>
>>
>> On 8 June 2016 at 21:05, Tim Penhey  wrote:
>>>
>>> Hi folks,
>>>
>>> tl;dr: not use structs in transaction asserts
>>>
>>> ...
>>>
>>> The solution is to not use a field struct equality, even though it is easy
>>> to write, but to use the dotted field notation to check the embedded field
>>> values.
>>
>>
>>
>> To give a more concrete example, asserting on a embedded document field like
>> this is problematic:
>>
>>   ops := []txn.Op{{
>>   C: "collection",
>>   Id: ...,
>>   Assert: bson.D{{"some-field", Thing{A: "foo", B: 99}}},
>>   Update: ...
>>   }
>>
>> Due to the way mgo works[1], the document the transaction operation is
>> asserting against may have been written with A and B in reverse order, or
>> the Thing struct in the Assert may have A and B swapped by the time it's
>> used. Either way, the assertion will fail randomly.
>>
>> The correct approach is to express the assertion like this:
>>
>>   ops := []txn.Op{{
>>   C: "collection",
>>   Id: ...,
>>   Assert: bson.D{
>>   {"some-field.A", "foo"},
>>   {"some-field.B", 99},
>>   },
>>   Update: ...
>>   }
>>
>> or this:
>>
>>   ops := []txn.Op{{
>>   C: "collection",
>>   Id: ...,
>>   Assert: bson.M{
>>   "some-field.A": "foo",
>>   "some-field.B": 99,
>>   },
>>   Update: ...
>>   }
>>
>>>
>>> Yet another thing to add to the list of things to check when doing
>>> reviews.
>>
>>
>> I think we can go a bit further and error on attempts to use structs for
>> comparison in txn.Op asserts in Juju's txn layers in state. Just as we
>> already do some munging and checking of database operations to ensure
>> correct multi-model behaviour, we should be able to do this same for this
>> issue and prevent it from happening again.
>>
>> - Menno
>>
>> [1] If transaction operations are loaded and used from the DB (more likely
>> under load when multiple runners are acting concurrently), the Insert,
>> Update and Assert fields are loaded as bson.M (this is what the bson
>> Unmarshaller does for interface{} typed fields). Once this happens field
>> ordering is lost.
>>
>>
>>
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

The CI build time continue to rise alarmingly

[David Cheney]

CI build times are now an average of 36 minutes. That means in a
typical 8 hour work day, assuming doing nothing other than landing
branches, less than 16 commits can be landed.

While bugs can be worked on and reviewed in parallel, landing is a
sequential action that blocks everyone, and given the landing bot is
batting less than 0.500, this limits the practical number of changes
that can be landed in a day, a sprint, a iteration, or a development
cycle.

I cannot make it any clearer than this, the speed of CI limits the
velocity of this team.

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Tech board agenda

[David Cheney]

On Wed, May 25, 2016 at 4:14 PM, Andrew Wilkins
<andrew.wilk...@canonical.com> wrote:
> On Wed, May 25, 2016 at 1:43 PM David Cheney <david.che...@canonical.com>
> wrote:
>>
>> Hello,
>>
>> Several times over this week there has been a call to do X to bring an
>> issue to the tech board's attention.
>>
>> Sadly X has generally be "email", which in my mind is a shitty way of
>> tabling issues for discussion.
>>
>> - Is there an agenda for the tech board?
>
>
> Usually.

Is there an agenda document that can be shared ?

>> - Is there a way to add items for the tech board to discuss?
>
>
> Currently email, or message one of us on IRC? Do you have alternative
> suggestions?

I'd like to be able to add them to the tech board agenda.

>
> If someone were going to put forward a significant proposal, it might be
> worth writing up a google doc and sending it to juju-dev. The less
> bike-shedding the better, though.

This is contradictory to the advice I've revived from others who say
things like "make sure you email juju-dev so the tech board see it". I
think there is some confusion how items are to be communicated.

>
>> - Is there a mechanism for the tech board to communicate the results
>>
>> of their deliberations on said items?
>
>
> So far we haven't needed to, because we've been focused on sprint-related
> things only. I imagine that would be an email to juju-dev, though. Do you
> have alternative suggestions?

I'd love it if you would take meeting minutes on your agenda.

>
>> Thank you for your time
>>
>> Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Tech board agenda

[David Cheney]

Hello,

Several times over this week there has been a call to do X to bring an
issue to the tech board's attention.

Sadly X has generally be "email", which in my mind is a shitty way of
tabling issues for discussion.

- Is there an agenda for the tech board?
- Is there a way to add items for the tech board to discuss?
- Is there a mechanism for the tech board to communicate the results
of their deliberations on said items?

Thank you for your time

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

natefinch/npipe/v2 does not support the POSIX read/close semantics juju needs

[David Cheney]

TL;DR read issue https://bugs.launchpad.net/bugs/1581337

Thumper asked me to forward this to juju-dev for discussion

http://reports.vapour.ws/releases/3964/job/run-unit-tests-win2012-amd64/attempt/2384

Fails because, npipe.Accept is blocking on accept

goroutine 41 [syscall, locked to thread]:
syscall.Syscall(0x7f8139f102c, 0x2, 0x1a8, 0x, 0x0,
0xc082183de8, 0x3e5, 0x1c0050)
 C:/Go/src/runtime/syscall_windows.go:163 +0x5c
syscall.WaitForSingleObject(0x1a8, 0xc0, 0xc082183e10, 0x0, 0x0)
 C:/Go/src/syscall/zsyscall_windows.go:693 +0x6f
gopkg.in/natefinch/npipe%2ev2.waitForCompletion(0x15c, 0xc0822a42a0,
0xc08202b880, 0x0, 0x0)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/gopkg.in/natefinch/npipe.v2/npipe_windows.go:181
+0x45
gopkg.in/natefinch/npipe%2ev2.(*PipeListener).AcceptPipe(0xc08202b880,
0x0, 0x0, 0x0)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/gopkg.in/natefinch/npipe.v2/npipe_windows.go:307
+0x397
gopkg.in/natefinch/npipe%2ev2.(*PipeListener).Accept(0xc08202b880,
0x0, 0x0, 0x0, 0x0)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/gopkg.in/natefinch/npipe.v2/npipe_windows.go:261
+0x44
github.com/juju/juju/worker/metrics/spool.(*socketListener).loop(0xc08227e780,
0x0, 0x0)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/github.com/juju/juju/worker/metrics/spool/listener.go:71
+0xaa
github.com/juju/juju/worker/metrics/spool.NewSocketListener.func1(0xc08227e780)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/github.com/juju/juju/worker/metrics/spool/listener.go:43
+0x65
created by github.com/juju/juju/worker/metrics/spool.NewSocketListener
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/github.com/juju/juju/worker/metrics/spool/listener.go:44
+0x140

and

socketListener is waiting on the tomb to hit done.

goroutine 43 [chan receive]:
launchpad.net/tomb.(*Tomb).Wait(0xc08227e790, 0x0, 0x0)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/launchpad.net/tomb/tomb.go:108
+0x5f
github.com/juju/juju/worker/metrics/spool.(*socketListener).Stop(0xc08227e780)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/github.com/juju/juju/worker/metrics/spool/listener.go:56
+0x18c
github.com/juju/juju/worker/metrics/sender.(*sender).stop(0xc082269770)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/github.com/juju/juju/worker/metrics/sender/sender.go:104
+0x45
github.com/juju/juju/worker/metrics/sender.(*sender).(github.com/juju/juju/worker/metrics/sender.stop)-fm()
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/github.com/juju/juju/worker/metrics/sender/manifold.go:77
+0x27
github.com/juju/juju/worker/metrics/spool.(*periodicWorker).Kill(0xc0822a40e0)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/github.com/juju/juju/worker/metrics/spool/listener.go:103
+0x28
github.com/juju/juju/worker/metrics/sender_test.(*ManifoldSuite).setupWorkerTest.func1(0xc0822791d0)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/github.com/juju/juju/worker/metrics/sender/manifold_test.go:101
+0x3d
github.com/juju/testing.(*CleanupSuite).callStack(0xc0821a9a28,
0xc0822791d0, 0xc08226bdd0, 0x2, 0x2)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/github.com/juju/testing/cleanup.go:52
+0x51
github.com/juju/testing.(*CleanupSuite).TearDownTest(0xc0821a9a28, 0xc0822791d0)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/github.com/juju/testing/cleanup.go:46
+0x4d
github.com/juju/testing.(*IsolationSuite).TearDownTest(0xc0821a9a20,
0xc0822791d0)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/github.com/juju/testing/isolation.go:38
+0x44
reflect.Value.call(0xcf2800, 0xc0821a9a20, 0x2213, 0xd242c0, 0x4,
0xc0822adf10, 0x1, 0x1, 0x0, 0x0, ...)
 C:/Go/src/reflect/value.go:435 +0x1214
reflect.Value.Call(0xcf2800, 0xc0821a9a20, 0x2213, 0xc0822adf10, 0x1,
0x1, 0x0, 0x0, 0x0)
 C:/Go/src/reflect/value.go:303 +0xb8
gopkg.in/check%2ev1.(*suiteRunner).runFixture.func1(0xc0822791d0)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/gopkg.in/check.v1/check.go:721
+0x16a
gopkg.in/check%2ev1.(*suiteRunner).forkCall.func1(0xc0821eac00,
0xc0822791d0, 0xea4fb0)
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/gopkg.in/check.v1/check.go:666
+0x76
created by gopkg.in/check%2ev1.(*suiteRunner).forkCall
 
c:/users/admini~1/appdata/local/temp/tmpyrijbo/gogo/src/gopkg.in/check.v1/check.go:667
+0x2a0
FAIL github.com/juju/juju/worker/metrics/sender 1200.101s

But the wait on listener.go:56 only happens after the listener is
closed on listener.go:52. But if the listener is closed, then the
Accept on listener.go:71 will have returned an error and the goroutine
will be in the process of tearing itself down. But we see that isn't
the case, the signal for accept to return with an error has not
happened.

This is usually referred to as concurrent read/close semantics. POSIX
_does_not_ define this behaviour for file descriptors. ie one thread
may be blocked on read, another may issue close 

Re: Awful dependency problem caused by romulus

[David Cheney]

Thanks Matty!

On Thu, May 19, 2016 at 6:51 PM, Matthew Williams
<matthew.willi...@canonical.com> wrote:
> Yeah - the mistake we made was starting with pure intentions but over time
> starting to think of it as just another part of core. I'll have to discuss
> it with Casey when he's up
>
> On Thu, May 19, 2016 at 9:47 AM, David Cheney <david.che...@canonical.com>
> wrote:
>>
>> I think that would be the best solution, I don't see how we can undo
>> the dependencies between cmd/juju and romulus -- they're so tightly
>> coupled they should probably live in the same repository.
>>
>> On Thu, May 19, 2016 at 6:45 PM, Matthew Williams
>> <matthew.willi...@canonical.com> wrote:
>> > Really sorry about this Dave, I'd not realised just how much they relied
>> > on
>> > each other. Surely there's an argument for romulus being merged into
>> > core?
>> >
>> > On Thu, May 19, 2016 at 8:55 AM, David Cheney
>> > <david.che...@canonical.com>
>> > wrote:
>> >>
>> >> On Thu, May 19, 2016 at 5:04 PM, roger peppe
>> >> <roger.pe...@canonical.com>
>> >> wrote:
>> >> > On 19 May 2016 at 07:02, David Cheney <david.che...@canonical.com>
>> >> > wrote:
>> >> >> Hello,
>> >> >>
>> >> >> github.com/juju/juju/cmd/juju/commands:
>> >> >>   github.com/juju/romulus/cmd/commands:
>> >> >> github.com/juju/romulus/cmd/setplan: <<<<<<<<<<<<<<<<<<<<<
>> >> >>   github.com/juju/juju/api/service:
>> >> >>   github.com/juju/juju/cmd/modelcmd:
>> >> >>
>> >> >> cmd/juju depends on the romulus repository, and the romulus
>> >> >> repository
>> >> >> depends on juju.
>> >> >>
>> >> >> This is terrible. It means we _cannot_ change the public api of the
>> >> >> juju that romulus depends on because then juju won't compile, and we
>> >> >> cannot land the fix to romulus without breaking juju.
>> >> >
>> >> > I agree that this is unfortunate, but "cannot" is a strong word. I
>> >> > believe
>> >> > that there is a (somewhat painful) workaround for this - we've been
>> >> > in
>> >> > similar situations
>> >> > before.
>> >> >
>> >> > Say you want to change the public API of juju in a backwardly
>> >> > incompatible
>> >> > way. Here's how you can do it.
>> >> >
>> >> > First change the API and fix romulus to work with the new API,
>> >> > without
>> >> > merging either change into their repos.
>> >> >
>> >> > Then push the romulus change to the romulus repo in a *feature
>> >> > branch*
>> >> > rather onto master. Tests will not pass in this branch because it
>> >> > depends
>> >> > on as-yet-to-be-landed changes in juju, but the code is now available
>> >> > in
>> >> > the romulus repo.
>> >> >
>> >> > Then propose the Juju changes with the feature-branch revision
>> >> > of romulus as a dependency. Tests should pass OK because godeps
>> >> > doesn't care which branch its dependencies are pulled from.
>> >> >
>> >> > Once that's landed, land the romulus changes in romulus master
>> >> > depending on the just-landed changes in juju.
>> >> >
>> >> > Then update juju to use the latest romulus dependency.
>> >>
>> >> Or I could just land the commits directly. I guess it depends if we
>> >> want to play the CI game or not. My point is creating loops like this
>> >> means we have to reach for even more creative measures to mitigate
>> >> them.
>> >>
>> >> To be clear, this is a big mistake, it's fine for juju to depend on a
>> >> project, we currently depend on 72 projects. What is not ok is for
>> >> that project to then depend back on juju, that is poor software
>> >> engineering.
>> >>
>> >> > As for the cyclic dependency itself, perhaps there's an argument for
>> >> > moving the main juju command into a separate repo (or everything
>> >> > *but*
>> >> > the juju main command into a separate repo) so that it's possible
>> >> > to include externally implemented commands without creating a cycle.
>> >>
>> >> I'd very much like to see this. It's clear that the juju command is
>> >> going to have to serve multiple masters, and by breaking it off into a
>> >> separate project this would force us (juju) to create a supported
>> >> public API which we currently do not have.
>> >>
>> >> >
>> >> >   cheers,
>> >> > rog.
>> >> >
>> >> >> Casey, please fix this immediately. Either juju depends on romulus,
>> >> >> or
>> >> >> romulus depends on juju, but at the moment they both depend on each
>> >> >> other and that is a showstopper,
>> >> >>
>> >> >> Thanks
>> >> >>
>> >> >> Dave
>> >> >>
>> >> >> --
>> >> >> Juju-dev mailing list
>> >> >> Juju-dev@lists.ubuntu.com
>> >> >> Modify settings or unsubscribe at:
>> >> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >>
>> >> --
>> >> Juju-dev mailing list
>> >> Juju-dev@lists.ubuntu.com
>> >> Modify settings or unsubscribe at:
>> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >
>> >
>
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Awful dependency problem caused by romulus

[David Cheney]

I think that would be the best solution, I don't see how we can undo
the dependencies between cmd/juju and romulus -- they're so tightly
coupled they should probably live in the same repository.

On Thu, May 19, 2016 at 6:45 PM, Matthew Williams
<matthew.willi...@canonical.com> wrote:
> Really sorry about this Dave, I'd not realised just how much they relied on
> each other. Surely there's an argument for romulus being merged into core?
>
> On Thu, May 19, 2016 at 8:55 AM, David Cheney <david.che...@canonical.com>
> wrote:
>>
>> On Thu, May 19, 2016 at 5:04 PM, roger peppe <roger.pe...@canonical.com>
>> wrote:
>> > On 19 May 2016 at 07:02, David Cheney <david.che...@canonical.com>
>> > wrote:
>> >> Hello,
>> >>
>> >> github.com/juju/juju/cmd/juju/commands:
>> >>   github.com/juju/romulus/cmd/commands:
>> >> github.com/juju/romulus/cmd/setplan: <<<<<<<<<<<<<<<<<<<<<
>> >>   github.com/juju/juju/api/service:
>> >>   github.com/juju/juju/cmd/modelcmd:
>> >>
>> >> cmd/juju depends on the romulus repository, and the romulus repository
>> >> depends on juju.
>> >>
>> >> This is terrible. It means we _cannot_ change the public api of the
>> >> juju that romulus depends on because then juju won't compile, and we
>> >> cannot land the fix to romulus without breaking juju.
>> >
>> > I agree that this is unfortunate, but "cannot" is a strong word. I
>> > believe
>> > that there is a (somewhat painful) workaround for this - we've been in
>> > similar situations
>> > before.
>> >
>> > Say you want to change the public API of juju in a backwardly
>> > incompatible
>> > way. Here's how you can do it.
>> >
>> > First change the API and fix romulus to work with the new API, without
>> > merging either change into their repos.
>> >
>> > Then push the romulus change to the romulus repo in a *feature branch*
>> > rather onto master. Tests will not pass in this branch because it
>> > depends
>> > on as-yet-to-be-landed changes in juju, but the code is now available in
>> > the romulus repo.
>> >
>> > Then propose the Juju changes with the feature-branch revision
>> > of romulus as a dependency. Tests should pass OK because godeps
>> > doesn't care which branch its dependencies are pulled from.
>> >
>> > Once that's landed, land the romulus changes in romulus master
>> > depending on the just-landed changes in juju.
>> >
>> > Then update juju to use the latest romulus dependency.
>>
>> Or I could just land the commits directly. I guess it depends if we
>> want to play the CI game or not. My point is creating loops like this
>> means we have to reach for even more creative measures to mitigate
>> them.
>>
>> To be clear, this is a big mistake, it's fine for juju to depend on a
>> project, we currently depend on 72 projects. What is not ok is for
>> that project to then depend back on juju, that is poor software
>> engineering.
>>
>> > As for the cyclic dependency itself, perhaps there's an argument for
>> > moving the main juju command into a separate repo (or everything *but*
>> > the juju main command into a separate repo) so that it's possible
>> > to include externally implemented commands without creating a cycle.
>>
>> I'd very much like to see this. It's clear that the juju command is
>> going to have to serve multiple masters, and by breaking it off into a
>> separate project this would force us (juju) to create a supported
>> public API which we currently do not have.
>>
>> >
>> >   cheers,
>> > rog.
>> >
>> >> Casey, please fix this immediately. Either juju depends on romulus, or
>> >> romulus depends on juju, but at the moment they both depend on each
>> >> other and that is a showstopper,
>> >>
>> >> Thanks
>> >>
>> >> Dave
>> >>
>> >> --
>> >> Juju-dev mailing list
>> >> Juju-dev@lists.ubuntu.com
>> >> Modify settings or unsubscribe at:
>> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Awful dependency problem caused by romulus

[David Cheney]

On Thu, May 19, 2016 at 5:04 PM, roger peppe <roger.pe...@canonical.com> wrote:
> On 19 May 2016 at 07:02, David Cheney <david.che...@canonical.com> wrote:
>> Hello,
>>
>> github.com/juju/juju/cmd/juju/commands:
>>   github.com/juju/romulus/cmd/commands:
>> github.com/juju/romulus/cmd/setplan: <<<<<<<<<<<<<<<<<<<<<
>>   github.com/juju/juju/api/service:
>>   github.com/juju/juju/cmd/modelcmd:
>>
>> cmd/juju depends on the romulus repository, and the romulus repository
>> depends on juju.
>>
>> This is terrible. It means we _cannot_ change the public api of the
>> juju that romulus depends on because then juju won't compile, and we
>> cannot land the fix to romulus without breaking juju.
>
> I agree that this is unfortunate, but "cannot" is a strong word. I believe
> that there is a (somewhat painful) workaround for this - we've been in
> similar situations
> before.
>
> Say you want to change the public API of juju in a backwardly incompatible
> way. Here's how you can do it.
>
> First change the API and fix romulus to work with the new API, without
> merging either change into their repos.
>
> Then push the romulus change to the romulus repo in a *feature branch*
> rather onto master. Tests will not pass in this branch because it depends
> on as-yet-to-be-landed changes in juju, but the code is now available in
> the romulus repo.
>
> Then propose the Juju changes with the feature-branch revision
> of romulus as a dependency. Tests should pass OK because godeps
> doesn't care which branch its dependencies are pulled from.
>
> Once that's landed, land the romulus changes in romulus master
> depending on the just-landed changes in juju.
>
> Then update juju to use the latest romulus dependency.

Or I could just land the commits directly. I guess it depends if we
want to play the CI game or not. My point is creating loops like this
means we have to reach for even more creative measures to mitigate
them.

To be clear, this is a big mistake, it's fine for juju to depend on a
project, we currently depend on 72 projects. What is not ok is for
that project to then depend back on juju, that is poor software
engineering.

> As for the cyclic dependency itself, perhaps there's an argument for
> moving the main juju command into a separate repo (or everything *but*
> the juju main command into a separate repo) so that it's possible
> to include externally implemented commands without creating a cycle.

I'd very much like to see this. It's clear that the juju command is
going to have to serve multiple masters, and by breaking it off into a
separate project this would force us (juju) to create a supported
public API which we currently do not have.

>
>   cheers,
> rog.
>
>> Casey, please fix this immediately. Either juju depends on romulus, or
>> romulus depends on juju, but at the moment they both depend on each
>> other and that is a showstopper,
>>
>> Thanks
>>
>> Dave
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at: 
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Awful dependency problem caused by romulus

[David Cheney]

On Thu, May 19, 2016 at 4:30 PM, Casey Marshall
<casey.marsh...@canonical.com> wrote:
> On Wed, May 18, 2016 at 11:02 PM, David Cheney <david.che...@canonical.com>
> wrote:
>>
>> Hello,
>>
>> github.com/juju/juju/cmd/juju/commands:
>>   github.com/juju/romulus/cmd/commands:
>> github.com/juju/romulus/cmd/setplan: <<<<<<<<<<<<<<<<<<<<<
>>   github.com/juju/juju/api/service:
>>   github.com/juju/juju/cmd/modelcmd:
>>
>> cmd/juju depends on the romulus repository, and the romulus repository
>> depends on juju.
>>
>> This is terrible. It means we _cannot_ change the public api of the
>> juju that romulus depends on because then juju won't compile, and we
>> cannot land the fix to romulus without breaking juju.
>
>
> Why do you want to introduce breaking changes in the API?

Juju does not have a stable API. I'm sorry if this has not been
communicated clearly.

>
>>
>>
>> Casey, please fix this immediately. Either juju depends on romulus, or
>> romulus depends on juju, but at the moment they both depend on each
>> other and that is a showstopper,
>>
>> Thanks
>>
>> Dave
>
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Awful dependency problem caused by romulus

[David Cheney]

Hello,

github.com/juju/juju/cmd/juju/commands:
  github.com/juju/romulus/cmd/commands:
github.com/juju/romulus/cmd/setplan: <
  github.com/juju/juju/api/service:
  github.com/juju/juju/cmd/modelcmd:

cmd/juju depends on the romulus repository, and the romulus repository
depends on juju.

This is terrible. It means we _cannot_ change the public api of the
juju that romulus depends on because then juju won't compile, and we
cannot land the fix to romulus without breaking juju.

Casey, please fix this immediately. Either juju depends on romulus, or
romulus depends on juju, but at the moment they both depend on each
other and that is a showstopper,

Thanks

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Three CI builds in the last 48 hours have failed because launchpad had a lie down

[David Cheney]

This is a huge waste of wall time and bandwidth, and increases the
chance of failure significantly. This is something I would like to see
changed.

On Thu, May 19, 2016 at 12:06 PM, John Meinel <j...@arbash-meinel.com> wrote:
> CI does a build from scratch in a pristine VM. There is some discussion
> about changing that in a variety of ways (in a container in a long running
> VM, etc), but it doesn't have a local copy to work from so it has to pull it
> from upstream each time.
>
> John
> =:->
>
>
> On Wed, May 18, 2016 at 8:05 PM, David Cheney <david.che...@canonical.com>
> wrote:
>>
>> We already have godeps which can take a set of vcs repos and flick
>> them to the right revisions.
>>
>> Why does CI check out every single dependency from upstream every
>> single time we do a build ? That introduces wc -l dependencies.tsv
>> points of failure to every single CI run -- not to mention the several
>> minutes it spends laboriously checking out the code.
>>
>> Thanks
>>
>> Dave
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Three CI builds in the last 48 hours have failed because launchpad had a lie down

[David Cheney]

We already have godeps which can take a set of vcs repos and flick
them to the right revisions.

Why does CI check out every single dependency from upstream every
single time we do a build ? That introduces wc -l dependencies.tsv
points of failure to every single CI run -- not to mention the several
minutes it spends laboriously checking out the code.

Thanks

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Today I submitted 5 PR's to be merged, 3 failed because mongo shat itself

[David Cheney]

This feels like a bug in mongodb. We store approximately zero data in
mongodb during test runs -- seriously, one machine, a charm at most,
that's it. It mongodb has such amazingly high overheads just start to
store data that sounds like a serious problem that is being ignored.

On Wed, May 18, 2016 at 7:32 PM, Christian Muirhead
<christian.muirh...@canonical.com> wrote:
> WiredTiger is *much* slower at creating and dropping indexes and
> collections. I haven't worked out why that is, other than doing some
> stracing and seeing that a lot of time is spent in fdatasync - I haven't dug
> into the mongo source code.
> There's a bit more detail in these bugs:
> https://bugs.launchpad.net/juju-core/+bug/1573294
> https://jira.mongodb.org/browse/SERVER-21198
>
> I've changed the tests so that instead of dropping and recreating databases
> in teardown and setup we clear out all of the collections (except the
> transaction collections) between tests. Obviously that's worse from the
> perspective of test isolation, but it seems to work well - better than I was
> expecting, to be honest.
>
> Cheers,
> Christian
>
> On Wed, May 18, 2016 at 9:58 AM roger peppe <roger.pe...@canonical.com>
> wrote:
>>
>> Out of interest, what's causing the 3.2 slowdown and what's the hack to
>> speed it up again?
>>
>> On 18 May 2016 09:51, "Christian Muirhead"
>> <christian.muirh...@canonical.com> wrote:
>>>
>>>
>>>
>>> On Wed, May 18, 2016 at 2:04 AM David Cheney <david.che...@canonical.com>
>>> wrote:
>>>>
>>>> 100x more webscale
>>>
>>> Ha!
>>>
>>> I'm *just about* finished the hack to make the state tests on 3.2 run in
>>> about the same time as on 2.4. On my machine the state tests take 6m24s on
>>> 3.2 and the old version took 4m56s. Which is still worse, unfortunately, but
>>> at least it isn't 100x worse. So if there are stability benefits to running
>>> the tests on 3.2 it's still a win, I guess?
>>>
>>>
>>>>
>>>>
>>>> On Wed, May 18, 2016 at 11:02 AM, Horacio Duran
>>>> <horacio.du...@canonical.com> wrote:
>>>> > For now we are trying to go around mongo issues that make the tests
>>>> > 100x
>>>> > slower (yes one hundred) once this is fixed we should start using
>>>> > mongo 3.2
>>>> > exclusively since 2.4 iirc is EOL or near. The issue lies in the new
>>>> > storage
>>>> > engine, which we could skip if mmapv1 ( the old one) wasn't also
>>>> > nearing EOL
>>>> > I am currently on the phone but if You want more details I can dig up
>>>> > the
>>>> > bug with details of what I am talking about.
>>>> >
>>>> >
>>>> > On Tuesday, 17 May 2016, David Cheney <david.che...@canonical.com>
>>>> > wrote:
>>>> >>
>>>> >> What's the plan for mongo 3.2 ? Will we be required to support 2.x
>>>> >> versions for the foreseeable future, or is there a possibility to
>>>> >> make
>>>> >> it a build or run time failure if mongo < 3.2 is installed on the
>>>> >> host
>>>> >> ?
>>>> >>
>>>> >> On Wed, May 18, 2016 at 9:01 AM, Martin Packman
>>>> >> <martin.pack...@canonical.com> wrote:
>>>> >> > On 17/05/2016, Curtis Hovey-Canonical <cur...@canonical.com> wrote:
>>>> >> >>
>>>> >> >> The juju-mongo2.6 package will be be preferred by juju 1.2.5 in
>>>> >> >> xenial
>>>> >> >> and without other changes, 2.4 will be used by all other 1.25
>>>> >> >> series.
>>>> >> >
>>>> >> > This isn't yet true, there's a bug open for it:
>>>> >> >
>>>> >> > "Use juju-mongodb2.6 for 1.25 on xenial"
>>>> >> > <https://bugs.launchpad.net/ubuntu/+source/juju-core/+bug/1570650>
>>>> >> >
>>>> >> > I had made the packaging change, but without juju code changes as
>>>> >> > well
>>>> >> > it just went and installed the old (2.4) juju-mongodb anyway when
>>>> >> > setting up a state server.
>>>> >> >
>>>> >> > Martin
>>>> >> >
>>>> >> > --
>>>> >> > Juju-dev mailing list
>>>> >> > Juju-dev@lists.ubuntu.com
>>>> >> > Modify settings or unsubscribe at:
>>>> >> > https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>>> >>
>>>> >> --
>>>> >> Juju-dev mailing list
>>>> >> Juju-dev@lists.ubuntu.com
>>>> >> Modify settings or unsubscribe at:
>>>> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>>>
>>>> --
>>>> Juju-dev mailing list
>>>> Juju-dev@lists.ubuntu.com
>>>> Modify settings or unsubscribe at:
>>>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>>
>>>
>>> --
>>> Juju-dev mailing list
>>> Juju-dev@lists.ubuntu.com
>>> Modify settings or unsubscribe at:
>>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>>
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Today I submitted 5 PR's to be merged, 3 failed because mongo shat itself

[David Cheney]

100x more webscale

On Wed, May 18, 2016 at 11:02 AM, Horacio Duran
<horacio.du...@canonical.com> wrote:
> For now we are trying to go around mongo issues that make the tests 100x
> slower (yes one hundred) once this is fixed we should start using mongo 3.2
> exclusively since 2.4 iirc is EOL or near. The issue lies in the new storage
> engine, which we could skip if mmapv1 ( the old one) wasn't also nearing EOL
> I am currently on the phone but if You want more details I can dig up the
> bug with details of what I am talking about.
>
>
> On Tuesday, 17 May 2016, David Cheney <david.che...@canonical.com> wrote:
>>
>> What's the plan for mongo 3.2 ? Will we be required to support 2.x
>> versions for the foreseeable future, or is there a possibility to make
>> it a build or run time failure if mongo < 3.2 is installed on the host
>> ?
>>
>> On Wed, May 18, 2016 at 9:01 AM, Martin Packman
>> <martin.pack...@canonical.com> wrote:
>> > On 17/05/2016, Curtis Hovey-Canonical <cur...@canonical.com> wrote:
>> >>
>> >> The juju-mongo2.6 package will be be preferred by juju 1.2.5 in xenial
>> >> and without other changes, 2.4 will be used by all other 1.25 series.
>> >
>> > This isn't yet true, there's a bug open for it:
>> >
>> > "Use juju-mongodb2.6 for 1.25 on xenial"
>> > <https://bugs.launchpad.net/ubuntu/+source/juju-core/+bug/1570650>
>> >
>> > I had made the packaging change, but without juju code changes as well
>> > it just went and installed the old (2.4) juju-mongodb anyway when
>> > setting up a state server.
>> >
>> > Martin
>> >
>> > --
>> > Juju-dev mailing list
>> > Juju-dev@lists.ubuntu.com
>> > Modify settings or unsubscribe at:
>> > https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Today I submitted 5 PR's to be merged, 3 failed because mongo shat itself

[David Cheney]

What's the plan for mongo 3.2 ? Will we be required to support 2.x
versions for the foreseeable future, or is there a possibility to make
it a build or run time failure if mongo < 3.2 is installed on the host
?

On Wed, May 18, 2016 at 9:01 AM, Martin Packman
 wrote:
> On 17/05/2016, Curtis Hovey-Canonical  wrote:
>>
>> The juju-mongo2.6 package will be be preferred by juju 1.2.5 in xenial
>> and without other changes, 2.4 will be used by all other 1.25 series.
>
> This isn't yet true, there's a bug open for it:
>
> "Use juju-mongodb2.6 for 1.25 on xenial"
> 
>
> I had made the packaging change, but without juju code changes as well
> it just went and installed the old (2.4) juju-mongodb anyway when
> setting up a state server.
>
> Martin
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: The mean time for a CI run has risen to 33 minutes

[David Cheney]

The change to Go 1.6 will have increased the _compile_ time for the
tests by 3x, but compared to the time of running them, this shouldn't
be the main contributor.

Go 1.7 will bring significantly improved compile and link times (the
linking times are much better than any previous version of Go)

On Wed, May 18, 2016 at 9:06 AM, Martin Packman
<martin.pack...@canonical.com> wrote:
> On 16/05/2016, David Cheney <david.che...@canonical.com> wrote:
>> This got significantly worse in the last 6 weeks. What happened ?
>
> Either the juju tests are slower, or trusty on aws is slower. It's a
> fresh cloud instance each run, and still trusty because we switched to
> xenial and the lxd tests failed due to lack of isolation. Could change
> back to xenial now Curtis added some lxd setup support the the
> makefile I think, but that is unlikely to help speed at all.
>
> Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Today I submitted 5 PR's to be merged, 3 failed because mongo shat itself

[David Cheney]

What is the story with mongo ? It's constantly causing builds to fail
CI because of it's complete shitness.

I've heard that some people have moved to mongo 3.2 which fixes the
problem, but as CI clearly is running the old rubbish version, this
clearly isn't a problem which can be called fixed.

What is the solution ? Will mongo 2.6 be deprecated entirely and the
build fail it the mongo version installed is < 3.2 ?

Thanks

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

The mean time for a CI run has risen to 33 minutes

[David Cheney]

This got significantly worse in the last 6 weeks. What happened ?
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: go test github.com/juju/juju/state takes > 10 minutes to run

[David Cheney]

My environment has not changed since 14.04, so I'm probably not
running mongo 3.2.

On Tue, May 17, 2016 at 4:13 PM, Cheryl Jennings
<cheryl.jenni...@canonical.com> wrote:
> Are you using mongo 3.2?  (see bug
> https://bugs.launchpad.net/juju-core/+bug/1573294)
>
> On Mon, May 16, 2016 at 9:52 PM, David Cheney <david.che...@canonical.com>
> wrote:
>>
>> Testing this package takes 16 minutes on my machine*; it sure didn't
>> use to take this long.
>>
>> What happened ?
>>
>> * yes, you have to raise the _10 minute_ timeout to make this test run.
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

go test github.com/juju/juju/state takes > 10 minutes to run

[David Cheney]

Testing this package takes 16 minutes on my machine*; it sure didn't
use to take this long.

What happened ?

* yes, you have to raise the _10 minute_ timeout to make this test run.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Please run go test -race before pushing code

[David Cheney]

Is the race detector build currently running and will it block landing ?

On Wed, May 11, 2016 at 6:48 AM, Cheryl Jennings
 wrote:
> Hey everyone,
>
> There have been some new races showing up in the CI race detector test.  As
> you're making your bug fixes, please make sure you run go test -race before
> pushing your code up for review to make sure you have not introduced new
> races.
>
> Thanks!
> -Cheryl
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: I think master is leaking

[David Cheney]

If the test suite panics, especially during a tear down from a failure
set up, you leak mongos and data in /tmp. I have a cron job that runs
a few times a day to keep this leaking below a few gig.

On Sat, Apr 9, 2016 at 8:13 AM, Horacio Duran
 wrote:
> Hey, this is more of an open question/sharing.
> I was a good half hour trying to find why reboot test was timeouting and
> found that I had a couple of lxcs that I dont recall creating (at least one
> of them was a juju api server) and a good 5-8 mongos running.
> I killed the test, the mongos and the lxcs and magic, the reboot tests ran
> well.
> Did anyone else detect this kind of behavior? I believe either destroy (I
> deployed a couple of times earlier and destroyed) or the tests are leaking,
> at least, lxcs Ill try to repro later but so far I would like some one
> else's input.
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Go 1.6 is now in trusty-proposed

[David Cheney]

On Tue, Mar 29, 2016 at 10:42 AM, Mark Ramm-Christensen
(Canonical.com)  wrote:
> Never a good time to stop feature work entirely and fix what amounts to a
> race prone set of tests.
>
>
> But I would advocate building in some practices to improve the situation
> incrementally:
>
> fixing one major issue per team per week

SGTM. How do we know which of the millions of private lists of bugs
are the critical ones? Which of the hundred "critical", "papercut",
"urgent" LP tags are the critical ones?

> promoting all issues which fail CI more than x times per week to Critical
> blocking all branches on fridays except for fixes for bugs on the top issues
> list

Which timezone is friday ?

> or some other similar policy
>
> Over time over time any of the above policies will bring the total number of
> test failures down significantly, and would still allow progress on feature
> work.
>
> On Mon, Mar 28, 2016 at 1:05 PM, Nate Finch 
> wrote:
>>
>> I'll just note that we've had flaky tests for as long as I've been working
>> on Juju, and there's never a "good" time to fix them. :)
>>
>> On Mon, Mar 28, 2016 at 11:48 AM Aaron Bentley
>>  wrote:
>>>
>>> -BEGIN PGP SIGNED MESSAGE-
>>> Hash: SHA256
>>>
>>> On 2016-03-28 09:03 AM, Katherine Cox-Buday wrote:
>>> > Generally +1 on this, but I'm also intrigued by Martin's
>>> > statistic... do we currently weight test failures by how likely
>>> > they are to fail (i.e. how likely they are flaky)? That seems like
>>> > it would be a great metric to use to decide which to fix first.
>>>
>>> We don't do it on the likelihood of failure, but we do it on the
>>> frequency of failure.
>>>
>>> http://reports.vapour.ws/releases/top-issues
>>>
>>> I report on these on the cross-team call, and once the 2.0 settles
>>> down, I'll be reporting them on the release call again.
>>>
>>> Aaron
>>> -BEGIN PGP SIGNATURE-
>>> Version: GnuPG v2
>>>
>>> iQEcBAEBCAAGBQJW+VJcAAoJEK84cMOcf+9hWrwH/0JradfscIE0wnt+yCW9nNCR
>>> 9hTHI2U19v1VuP6pWI4UiC7srfojPI8EXXEXrrAhF9rT8tpVK4EcJRJK9RvWvvz5
>>> BEquHMS0+eROFOqDJFavEB8hU7BKHErzkSwSG8uKq7JuwHs9gNtQO9z9fIhVKjnr
>>> aP4z2IliCqbYfXbupfSTD8TmqhI0AipQymTg3QB4C3sJdXzc5GjzIIckUo/X7aJj
>>> zH1tEtlwOdP0c9F+8ZVs1j6AAkb+uDGc/1Qr4MT1kInqGkli2UNF4TOX/AihNPyH
>>> iwYgq6O7uOkijFTrL9obRfbXxIFw1WCc9cYzxbRYnGfQff47Dyj7/BUStPPH0i0=
>>> =8FQ6
>>> -END PGP SIGNATURE-
>>>
>>> --
>>> Juju-dev mailing list
>>> Juju-dev@lists.ubuntu.com
>>> Modify settings or unsubscribe at:
>>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>
>
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Go 1.6 is now in trusty-proposed

[David Cheney]

I know if we didn't retry constantly, the Juju tests'd never pass. But
by retrying, there is no impetus to fix them.

How about we stop retrying flaky tests? The blocked build get's the grease.

On Mon, Mar 28, 2016 at 5:23 PM, Martin Packman
<martin.pack...@canonical.com> wrote:
> On 27/03/2016, David Cheney <david.che...@canonical.com> wrote:
>> Hi Martin,
>>
>> I was told that the Go 1.6 tests were voting, so these bugs should be
>> blocking bugs. Is this not the case ?
>
> The tests are voting, and giving blesses, so no blocking bugs, but a
> lot of the remaining issues are low-occurrence failures. Basically the
> unit tests pass generally given the three attempts, but overall fail a
> lot from a number of issues that all happen only occasionally.
>
> For instance, bug 1553292:
>
> <http://reports.vapour.ws/releases/issue/56c36c5b749a567aa9496178>
>
> This is maybe ~5% chance of failing, but given the number of jobs now
> using go 1.5+ that's still six failures in the last week.
>
> We have enough issues like this that CI spends a lot more time
> retesting on go 1.6 than we do on go 1.2 with the same unit tests.
>
> Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Go 1.6 is now in trusty-proposed

[David Cheney]

Hi Martin,

I was told that the Go 1.6 tests were voting, so these bugs should be
blocking bugs. Is this not the case ?

Thanks

Dave

On Fri, Mar 25, 2016 at 12:51 AM, Martin Packman
 wrote:
> On 24/03/2016, Ian Booth  wrote:
>>
>> Not yet. The builders and test infrastructure all need to be updated, and
>> the package needs a week to transition out of proposed.
>
> I'd also encourage people to look again at the go 1.5/1.6 unit test
> intermittent failures, as we're still often doing multiple runs in
> order to get a pass.
>
> 
>
> Martin
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Go 1.6 is now in trusty-proposed

[David Cheney]

THIS IS FANTASTIC NEWS MICHAEL!

On Thu, Mar 24, 2016 at 5:03 PM, Michael Hudson-Doyle
 wrote:
> Hi,
>
> As of a few minutes ago, there is now a golang-1.6 package in
> trusty-proposed:
> https://launchpad.net/ubuntu/trusty/+source/golang-1.6 (thanks for the
> review and copy, Steve).
>
> One difference between this and the package I prepared earlier is that
> it does not install /usr/bin/go but rather /usr/lib/go-1.6/bin/go so
> Makefiles and such will need to be adjusted to invoke that directly or
> put /usr/lib/go-1.6/bin on $PATH or whatever. (This also means it can
> be installed alongside the golang packages that are already in
> trusty).
>
> Cheers,
> mwh
> (Hoping that we can now really properly ignore gccgo-4.9 ppc64el bugs!)
>
> On 17 February 2016 at 07:58, Michael Hudson-Doyle
>  wrote:
>> I have approval for the idea but also decided to wait for 1.6 and upload
>> that instead. I'm also on leave currently so hopefully this can all happen
>> in early March.
>>
>> Cheers,
>> mwh
>>
>> On 17/02/2016 1:17 am, "John Meinel"  wrote:
>>>
>>> To start with, thanks for working on this. However, doesn't this also
>>> require changing the CI builds to use your ppa?
>>>
>>> What is the current state of this? I was just looking around and noticed
>>> golang1.5-go isn't in anything specific for Trusty that I can see. I realize
>>> if its going into an SRU it requires a fair amount of negotiation with other
>>> teams, so I'm not  surprised to see it take a while. I just wanted to check
>>> how it was going.
>>>
>>> Thanks,
>>>
>>> John
>>> =:->
>>>
>>> On Mon, Jan 18, 2016 at 7:32 AM, Michael Hudson-Doyle
>>>  wrote:

 Hi all,

 As part of the plan for getting Go 1.5 into trusty (see here
 https://wiki.ubuntu.com/MichaelHudsonDoyle/Go15InTrusty) I've built
 packages (called golang1.5-go rather than golang-go) for trusty in my
 ppa:

 https://launchpad.net/~mwhudson/+archive/ubuntu/go15-trusty/+packages

 (assuming 3:1.5.3-0ubuntu4 actually builds... I seem to be having a
 "make stupid packaging mistakes" day)

 I'll write up a SRU bug to start the process of getting this into
 trusty tomorrow but before it does end up in trusty it would seem like
 a good idea to run the CI tests using juju-core packages built with
 this version of the go compiler. Is that something that's feasible to
 arrange

 The only packaging requirement should be to change the build-depends
 to be on golang1.5-go rather than golang-go or gccgo.

 Cheers,
 mwh

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>>
>>>
>>
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: juju 2.0-alpha2 with lxd

[David Cheney]

Unless you are building Juju from source (which it doesn't look like
you are) you don't need a Go compiler installed.

On Mon, Feb 15, 2016 at 11:20 AM, Marco Ceppi  wrote:
> Right, it's not going to work:
>
>> It's not so much that you don't have golang installed, it's about the
>> version of golang juju was compiled with at build time. If you take the
>> juju-core deb from Xenial and install it on a machine that has LXD working
>> it'll work. However, since the builders that run to build the packages
>> install golang from the main archive (1.2 in trusty, 1.3 in vivid, 1.5 in
>> wily, and 1.6 in xenial) and because the LXD bindings require a pretty
>> recent version of golang to work - even if you have LXD installed, etc
>> you'll still need to grab the compiled Juju client from a newer version to
>> use it.
>
> On Sun, Feb 14, 2016 at 7:17 PM Daniel Bidwell  wrote:
>>
>> Let's try this again.  I am running wiley 15.10 with golang 2:1.5.1 and
>> lxd 0.20.  Adding golang didn't do anything for me.
>>
>> On Sun, 2016-02-14 at 23:55 +, Marco Ceppi wrote:
>> > What version of Ubuntu are you using? If it's anything other than
>> > Xenial or Wily the LXD provider won't work. LXD bindings require a
>> > golang version that's too new for Trusty.
>> >
>> > On Sun, Feb 14, 2016 at 6:14 PM Daniel Bidwell 
>> > wrote:
>> > > I am trying to bootstrap lxd.
>> > >
>> > > I started with 'juju init' and then 'juju -v --debug bootstrap -m
>> > > lxd'
>> > > It allocated a container and deployed to it and ended with the
>> > > following lines of debug output:
>> > >
>> > > 2016-02-14 23:04:46 INFO juju.cmd supercommand.go:59 running jujud
>> > > [2.0
>> > > -alpha2 gc go1.2.1]
>> > > 2016-02-14 23:04:46 DEBUG juju.agent agent.go:506 read agent
>> > > config,
>> > > format "1.18"
>> > > 2016-02-14 23:04:46 INFO juju.network network.go:248 setting prefer
>> > > -ipv6 to false
>> > > 2016-02-14 23:04:46 ERROR cmd supercommand.go:448 no registered
>> > > provider for "lxd"
>> > > 2016-02-14 23:04:46 ERROR cmd supercommand.go:448 failed to
>> > > bootstrap
>> > > model: subprocess encountered error code 1
>> > >
>> > > What do I have to do to "register the lxd provider"?  Is this done
>> > > before the boostrap command, during, or after?
>> > >
>> > > A 'juju status' command hangs for ever.  Another 'juju bootstrap'
>> > > command starts out happy, but then tells me that lxd is already
>> > > deployed.  I can 'ssh ubuntu@10.0.3.x' and get into the container
>> > > just
>> > > fine, but not 'juju ssh 0'.
>> > >
>> > > What am I missing?
>> > > --
>> > > Daniel Bidwell 
>> > >
>> > >
>> > > --
>> > > Juju-dev mailing list
>> > > Juju-dev@lists.ubuntu.com
>> > > Modify settings or unsubscribe at:
>> > > https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> > >
>> --
>> Daniel Bidwell 
>>
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: utils/fslock needs to DIAF

[David Cheney]

On Tue, Dec 1, 2015 at 8:10 PM, roger peppe <roger.pe...@canonical.com> wrote:
> So I'm with axw on this one - flock seems like it is a reasonable tool for
> the job here. FWIW a Unix domain socket also suffers from the "won't
> work across NFS" problem. Abstract unix domain sockets also
> have the problem that they won't work with long file paths (what
> were they thinking?)

This is false, abstract unix domains sockets have nothing to do with
nfs, they don't live on a filesystem. wrt to a long path, don't think
of it as a path, think of it as a per machine key.

> We have been using github.com/camlistore/lock and although it's not
> totally ideal (see https://github.com/camlistore/lock/issues/10 )
> it does the job. Note that it's non-blocking, so a blocking
> layer above it is necessary, for example see the lockFile
> function in 
> https://github.com/juju/persistent-cookiejar/blob/master/serialize.go
>
> The Windows named mutex thing does sound interesting because
> it's a blocking API, which is actually what we need. On the other
> hand, under Windows, files opened for writing are locked by default
> anyway, so it might be easier just to leverage that property.
> The camlistore lock code could use some improvement for the
> Windows platform - we could either fork it, or bradfitz would
> probably be amenable to a PR.
>
>   cheers,
> rog.
>
> On 1 December 2015 at 04:12, Nate Finch <nate.fi...@canonical.com> wrote:
>> I'm not a linux expert, but definitely a named mutex is exactly the correct
>> thing to use for Windows.  Using something else for this purpose would be
>> very surprising to a Windows dev and much more likely to be buggy.  I don't
>> see any reason we need to use the exact same implementation on all OSes if
>> there is something that does exactly the right thing without any finagling.
>> FWIW, mutexes do die with the process:
>> https://msdn.microsoft.com/en-us/library/windows/desktop/ms682411(v=vs.85).aspx
>>
>> On Mon, Nov 30, 2015 at 8:29 PM Andrew Wilkins
>> <andrew.wilk...@canonical.com> wrote:
>>>
>>> On Tue, Dec 1, 2015 at 9:07 AM David Cheney <david.che...@canonical.com>
>>> wrote:
>>>>
>>>> http://0pointer.de/blog/projects/locking.html
>>>>
>>>> In short, opening the same file twice and asserting a lock on it will
>>>> succeed.
>>>
>>>
>>> Thanks. The article is a bit exasperated. Yes, there are problems to be
>>> aware of, but it doesn't make them unusable in all cases.
>>>  - Juju agents don't get installed onto NFS file systems, so doesn't
>>> matter for the agents.
>>>  - We're in full control of the files we're locking, we're not locking
>>> some file like /etc/passwd where some other random bit of code in the
>>> process is going to open/close it and release the lock by accident.
>>>  - We don't need byte-range locking.
>>>
>>> So only the original uncertainty remains: do we care about clients running
>>> their home directory on an NFS share, where the NFS *server* is too old to
>>> support flock?
>>>
>>> Maybe a named mutex on Windows and a domain socket on *NIX is the way to
>>> go. I'm not dead set on flock; I just want something that is simple, robust
>>> and portable.
>>>
>>> Cheers,
>>> Andrew
>>>
>>>> On Tue, Dec 1, 2015 at 12:04 PM, Andrew Wilkins
>>>> <andrew.wilk...@canonical.com> wrote:
>>>> > On Tue, Dec 1, 2015 at 8:57 AM David Cheney
>>>> > <david.che...@canonical.com>
>>>> > wrote:
>>>> >>
>>>> >> fcntl won't work in threaded go applications, it barely works in non
>>>> >> threaded applications.
>>>> >
>>>> >
>>>> > This is news to me. I've used it plenty in the past, in multi-threaded
>>>> > programs. Please point me at relevant literature that explains where it
>>>> > doesn't work.
>>>> >
>>>> >>
>>>> >> I'm not interested in "doesn't work on windows" arguments. Yes, we
>>>> >> have to support windows, but that doesn't mean we have to be dragged
>>>> >> down to it's lowest common denominator.
>>>> >
>>>> >
>>>> > Agreed, if we're actually crippling anything.
>>>> >
>>>> >>
>>>> >> I think it's fine to develop a lock type that does the best available
>>>> >> for each platform using conditional co

Re: utils/fslock needs to DIAF

[David Cheney]

fcntl won't work in threaded go applications, it barely works in non
threaded applications.

I'm not interested in "doesn't work on windows" arguments. Yes, we
have to support windows, but that doesn't mean we have to be dragged
down to it's lowest common denominator.

I think it's fine to develop a lock type that does the best available
for each platform using conditional compilation.

On Tue, Dec 1, 2015 at 11:47 AM, Andrew Wilkins
<andrew.wilk...@canonical.com> wrote:
> On Tue, Dec 1, 2015 at 8:03 AM David Cheney <david.che...@canonical.com>
> wrote:
>>
>> On Tue, Dec 1, 2015 at 11:00 AM, Andrew Wilkins
>> <andrew.wilk...@canonical.com> wrote:
>> > On Tue, Dec 1, 2015 at 7:57 AM David Cheney <david.che...@canonical.com>
>> > wrote:
>> >>
>> >> Doesn't look like there is windows support, and it uses fcntl (flock)
>> >> under the hood, which is what we have now.
>> >
>> >
>> > flock isn't the problematic thing Tim is talking about. utils/fslock
>> > attempts to create a directory in a known location, and if it succeeds,
>> > it
>> > "holds the lock". Unlocking means removing the directory.
>>
>> The problem is if the process dies/exits/goes mental while holding the
>> lock we get into this existential gridlock where we're not sure if the
>> process _is_ alive, so we shouldn't remove the lock, or the process is
>> dead, so we should remove the lock.
>>
>> abstract unix domain sockets do not have this problem on windows; kill
>> the process, file is removed from the abstract namespace.
>
>
> POSIX advisory file locks (flock) do not have this problem either. See:
> man(2) fcntl, section "Advisory record locking". When the file descriptor is
> closed, the lock is released; file descriptors are closed when the process
> dies.
>
> We could build a mutex on top of a unix domain socket, but then we'll have
> something completely separate for Windows. Shared named mutex? I'm not
> convinced the overall solution would be any more robust, and I'm pretty sure
> it's going to be more complicated. Happy to be proven wrong though.
>
>> >
>> > We would have to contribute Windows support, but it's not hard, I've
>> > done it
>> > before.
>> >
>> >>
>> >> On Tue, Dec 1, 2015 at 10:55 AM, Casey Marshall
>> >> <casey.marsh...@canonical.com> wrote:
>> >> > How about github.com/camlistore/lock ?
>> >> >
>> >> > On Mon, Nov 30, 2015 at 5:43 PM, Tim Penhey
>> >> > <tim.pen...@canonical.com>
>> >> > wrote:
>> >> >>
>> >> >> Hi folks,
>> >> >>
>> >> >> The fslock was a mistake that I added to the codebase some time
>> >> >> back.
>> >> >> It
>> >> >> provided an overly simplistic solution to a more complex problem.
>> >> >>
>> >> >> Really the filesystem shouldn't be used as a locking mechanism.
>> >> >>
>> >> >> Most of the code that exists for the fslock now is working around
>> >> >> its
>> >> >> deficiencies. Instead we should be looking for a better replacement.
>> >> >>
>> >> >> Some "features" that were added to fslock were added to work around
>> >> >> the
>> >> >> issue that the lock did not die with the process that created it, so
>> >> >> some mechanism was needed to determine whether the lock should be
>> >> >> broken
>> >> >> or not.
>> >> >>
>> >> >> What we really need is a good OS agnostic abstraction that provides
>> >> >> the
>> >> >> ability to create a "named" lock, acquire the lock, release the
>> >> >> lock,
>> >> >> and make sure that the lock dies when the process dies, so another
>> >> >> process that is waiting can acquire the lock. This way no
>> >> >> "BreakLock"
>> >> >> functionality is required, nor do we need to try and do think like
>> >> >> remember which process owns the lock.
>> >> >>
>> >> >> So...
>> >> >>
>> >> >> We have three current operating systems we need to support:
>> >> >>
>> >> >> Linux - Ubuntu and CentOS
>> >> >> MacOS - client only - bit the CLI uses a lock for th

Re: utils/fslock needs to DIAF

[David Cheney]

Doesn't look like there is windows support, and it uses fcntl (flock)
under the hood, which is what we have now.

On Tue, Dec 1, 2015 at 10:55 AM, Casey Marshall
 wrote:
> How about github.com/camlistore/lock ?
>
> On Mon, Nov 30, 2015 at 5:43 PM, Tim Penhey 
> wrote:
>>
>> Hi folks,
>>
>> The fslock was a mistake that I added to the codebase some time back. It
>> provided an overly simplistic solution to a more complex problem.
>>
>> Really the filesystem shouldn't be used as a locking mechanism.
>>
>> Most of the code that exists for the fslock now is working around its
>> deficiencies. Instead we should be looking for a better replacement.
>>
>> Some "features" that were added to fslock were added to work around the
>> issue that the lock did not die with the process that created it, so
>> some mechanism was needed to determine whether the lock should be broken
>> or not.
>>
>> What we really need is a good OS agnostic abstraction that provides the
>> ability to create a "named" lock, acquire the lock, release the lock,
>> and make sure that the lock dies when the process dies, so another
>> process that is waiting can acquire the lock. This way no "BreakLock"
>> functionality is required, nor do we need to try and do think like
>> remember which process owns the lock.
>>
>> So...
>>
>> We have three current operating systems we need to support:
>>
>> Linux - Ubuntu and CentOS
>> MacOS - client only - bit the CLI uses a lock for the local cache
>> Windows
>>
>> For Linux, and possibly MacOS, flock is a possibility, but can we do
>> better? Is there something that is better suited?
>>
>> For Windows, while you can create global semaphores or mutex instances,
>> I'm not sure of entities that die with the process. Can people recommend
>> solutions?
>>
>> Cheers,
>> Tim
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Reviews for merges _from_ master to feature branches

[David Cheney]

Hello,

Why are reviewers being created for merges from master to feature
branches ? What purpose does this serve ?

Thanks

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: utils/fslock needs to DIAF

[David Cheney]

On Tue, Dec 1, 2015 at 10:43 AM, Tim Penhey  wrote:
> Hi folks,
>
> The fslock was a mistake that I added to the codebase some time back. It
> provided an overly simplistic solution to a more complex problem.
>
> Really the filesystem shouldn't be used as a locking mechanism.
>
> Most of the code that exists for the fslock now is working around its
> deficiencies. Instead we should be looking for a better replacement.
>
> Some "features" that were added to fslock were added to work around the
> issue that the lock did not die with the process that created it, so
> some mechanism was needed to determine whether the lock should be broken
> or not.
>
> What we really need is a good OS agnostic abstraction that provides the
> ability to create a "named" lock, acquire the lock, release the lock,
> and make sure that the lock dies when the process dies, so another
> process that is waiting can acquire the lock. This way no "BreakLock"
> functionality is required, nor do we need to try and do think like
> remember which process owns the lock.
>
> So...
>
> We have three current operating systems we need to support:
>
> Linux - Ubuntu and CentOS
> MacOS - client only - bit the CLI uses a lock for the local cache
> Windows
>
> For Linux, and possibly MacOS, flock is a possibility, but can we do
> better? Is there something that is better suited?

For linux the best alternative I know of for a per process mutex is to
use a unix domain socket in the abstract namespace. These are
automatically removed when the listening socket is closed, or the
process dies, there is no need to cleanup stale pids.

For other posix systems a pidfile, or on disk unix domain socket is
probably the best we can do.

>
> For Windows, while you can create global semaphores or mutex instances,
> I'm not sure of entities that die with the process. Can people recommend
> solutions?
>
> Cheers,
> Tim
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: utils/fslock needs to DIAF

[David Cheney]

Please no. The better way is to use an abstract unix domain socket to
create a mutex.

On Tue, Dec 1, 2015 at 10:56 AM, Andrew Wilkins
 wrote:
> On Tue, Dec 1, 2015 at 7:43 AM Tim Penhey  wrote:
>>
>> Hi folks,
>>
>> The fslock was a mistake that I added to the codebase some time back. It
>> provided an overly simplistic solution to a more complex problem.
>>
>> Really the filesystem shouldn't be used as a locking mechanism.
>>
>> Most of the code that exists for the fslock now is working around its
>> deficiencies. Instead we should be looking for a better replacement.
>>
>> Some "features" that were added to fslock were added to work around the
>> issue that the lock did not die with the process that created it, so
>> some mechanism was needed to determine whether the lock should be broken
>> or not.
>>
>> What we really need is a good OS agnostic abstraction that provides the
>> ability to create a "named" lock, acquire the lock, release the lock,
>> and make sure that the lock dies when the process dies, so another
>> process that is waiting can acquire the lock. This way no "BreakLock"
>> functionality is required, nor do we need to try and do think like
>> remember which process owns the lock.
>>
>> So...
>>
>> We have three current operating systems we need to support:
>>
>> Linux - Ubuntu and CentOS
>> MacOS - client only - bit the CLI uses a lock for the local cache
>> Windows
>>
>> For Linux, and possibly MacOS, flock is a possibility, but can we do
>> better? Is there something that is better suited?
>>
>> For Windows, while you can create global semaphores or mutex instances,
>> I'm not sure of entities that die with the process. Can people recommend
>> solutions?
>
>
> I've been wanting to do this for a long time (I think I've whinged in your
> vicinity about it before), but I've held off because of uncertainty about
> compatibility with NFS (which is probably a rare scenario, and only for the
> client).
>
> I think it was jam that brought up the heritage of directory locking in bzr,
> where flock doesn't work reliably over NFS. I think that's old news though.
> The manpage for flock discusses the NFS/kernel limitations of flock; since
> we don't go back past precise, we should be fine.
>
> I think flock (fcntl F_SETLK/F_GETLK) is an appropriate option for Linux and
> OS X. Windows has FileLock
> (https://msdn.microsoft.com/en-us/library/windows/desktop/aa365202(v=vs.85).aspx),
> and LockFileEx (for more control). Just as with F_SETLK, if the process
> dies, the lock is released.
>
> Cheers,
> Andrew
>
>> Cheers,
>> Tim
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Reviews for merges _from_ master to feature branches

[David Cheney]

It's not gonna get through the merge bot if that's the case. To
clarify, using the landing bot, thumbs up, pointless reviews that
nobody reviews and have poor descriptions, thumbs down.

On Tue, Dec 1, 2015 at 11:20 AM, Rick Harding
<rick.hard...@canonical.com> wrote:
> Sanity check on merge conflicts resolved?
>
>
> On Mon, Nov 30, 2015, 7:18 PM David Cheney <david.che...@canonical.com>
> wrote:
>>
>> Hello,
>>
>> Why are reviewers being created for merges from master to feature
>> branches ? What purpose does this serve ?
>>
>> Thanks
>>
>> Dave
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Upgrading minimum Go version?

[David Cheney]

As of this morning, the unit tests pass on xenial (thanks to Andrew
for getting them over the hump) using Go 1.5

On Tue, Dec 1, 2015 at 3:21 AM, Curtis Hovey-Canonical
 wrote:
> On Mon, Nov 30, 2015 at 10:36 AM, John Meinel  wrote:
>> Given how often people still use "--upload-tools" for things like private
>> clouds (and is definitely the one used for local provider), I'd really worry
>> about having a jujud on your local machine that wasn't built with the same
>> toolchain as the one you get from "juju bootstrap" in other cases. Very easy
>> to end up with hard to understand/reproduce bugs.
>
> I share your concerns John.
>
> We have seen several bugs marked invalid because the agent uploads is
> essentially unknown origin because Juju is more than willing to fake a
> version when uploading. There are also reports of Juju attempting to
> build agents from source:
> https://bugs.launchpad.net/juju-core/+bug/1399606
>
> We do want everyone using the agent in streams, which have a lot more
> testing behind them. The agent (jujud) on users of the Juju stable PPA
> is the same that was used to make streams. This is not true for users
> of Ubuntu's packages, and we know Ubuntu can change its tool chain
> between the time we create official agents and it builds its packages.
> We do testing to certify they Ubuntu clients and agents are
> functionally equivalent to those in the PPA and streams. As Ubuntu
> Wily+ prefers system Go dev packages to the embedded go packages we
> provide, Ubuntu's packages will contain more divergence than Trusty.
> We are obligated to do on-demand testing to certify a change to a
> system Go dev package.
>
> Ideally, We would separate the jujud from juju-core package that
> provides the client. Users don't get a jujud when they install a
> client. Instead, Juju can get the desired agents from streams.
>
> --
> Curtis Hovey
> Canonical Cloud Development and Operations
> http://launchpad.net/~sinzui
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: utils/fslock needs to DIAF

[David Cheney]

On Tue, Dec 1, 2015 at 11:00 AM, Andrew Wilkins
<andrew.wilk...@canonical.com> wrote:
> On Tue, Dec 1, 2015 at 7:57 AM David Cheney <david.che...@canonical.com>
> wrote:
>>
>> Doesn't look like there is windows support, and it uses fcntl (flock)
>> under the hood, which is what we have now.
>
>
> flock isn't the problematic thing Tim is talking about. utils/fslock
> attempts to create a directory in a known location, and if it succeeds, it
> "holds the lock". Unlocking means removing the directory.

The problem is if the process dies/exits/goes mental while holding the
lock we get into this existential gridlock where we're not sure if the
process _is_ alive, so we shouldn't remove the lock, or the process is
dead, so we should remove the lock.

abstract unix domain sockets do not have this problem on windows; kill
the process, file is removed from the abstract namespace.

>
> We would have to contribute Windows support, but it's not hard, I've done it
> before.
>
>>
>> On Tue, Dec 1, 2015 at 10:55 AM, Casey Marshall
>> <casey.marsh...@canonical.com> wrote:
>> > How about github.com/camlistore/lock ?
>> >
>> > On Mon, Nov 30, 2015 at 5:43 PM, Tim Penhey <tim.pen...@canonical.com>
>> > wrote:
>> >>
>> >> Hi folks,
>> >>
>> >> The fslock was a mistake that I added to the codebase some time back.
>> >> It
>> >> provided an overly simplistic solution to a more complex problem.
>> >>
>> >> Really the filesystem shouldn't be used as a locking mechanism.
>> >>
>> >> Most of the code that exists for the fslock now is working around its
>> >> deficiencies. Instead we should be looking for a better replacement.
>> >>
>> >> Some "features" that were added to fslock were added to work around the
>> >> issue that the lock did not die with the process that created it, so
>> >> some mechanism was needed to determine whether the lock should be
>> >> broken
>> >> or not.
>> >>
>> >> What we really need is a good OS agnostic abstraction that provides the
>> >> ability to create a "named" lock, acquire the lock, release the lock,
>> >> and make sure that the lock dies when the process dies, so another
>> >> process that is waiting can acquire the lock. This way no "BreakLock"
>> >> functionality is required, nor do we need to try and do think like
>> >> remember which process owns the lock.
>> >>
>> >> So...
>> >>
>> >> We have three current operating systems we need to support:
>> >>
>> >> Linux - Ubuntu and CentOS
>> >> MacOS - client only - bit the CLI uses a lock for the local cache
>> >> Windows
>> >>
>> >> For Linux, and possibly MacOS, flock is a possibility, but can we do
>> >> better? Is there something that is better suited?
>> >>
>> >> For Windows, while you can create global semaphores or mutex instances,
>> >> I'm not sure of entities that die with the process. Can people
>> >> recommend
>> >> solutions?
>> >>
>> >> Cheers,
>> >> Tim
>> >>
>> >> --
>> >> Juju-dev mailing list
>> >> Juju-dev@lists.ubuntu.com
>> >> Modify settings or unsubscribe at:
>> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >
>> >
>> >
>> > --
>> > Juju-dev mailing list
>> > Juju-dev@lists.ubuntu.com
>> > Modify settings or unsubscribe at:
>> > https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: utils/fslock needs to DIAF

[David Cheney]

http://0pointer.de/blog/projects/locking.html

In short, opening the same file twice and asserting a lock on it will succeed.

On Tue, Dec 1, 2015 at 12:04 PM, Andrew Wilkins
<andrew.wilk...@canonical.com> wrote:
> On Tue, Dec 1, 2015 at 8:57 AM David Cheney <david.che...@canonical.com>
> wrote:
>>
>> fcntl won't work in threaded go applications, it barely works in non
>> threaded applications.
>
>
> This is news to me. I've used it plenty in the past, in multi-threaded
> programs. Please point me at relevant literature that explains where it
> doesn't work.
>
>>
>> I'm not interested in "doesn't work on windows" arguments. Yes, we
>> have to support windows, but that doesn't mean we have to be dragged
>> down to it's lowest common denominator.
>
>
> Agreed, if we're actually crippling anything.
>
>>
>> I think it's fine to develop a lock type that does the best available
>> for each platform using conditional compilation.
>
>
> Again, agreed, but only if there's something to be gained by doing this. I'm
> still not convinced there is.
>
>> On Tue, Dec 1, 2015 at 11:47 AM, Andrew Wilkins
>> <andrew.wilk...@canonical.com> wrote:
>> > On Tue, Dec 1, 2015 at 8:03 AM David Cheney <david.che...@canonical.com>
>> > wrote:
>> >>
>> >> On Tue, Dec 1, 2015 at 11:00 AM, Andrew Wilkins
>> >> <andrew.wilk...@canonical.com> wrote:
>> >> > On Tue, Dec 1, 2015 at 7:57 AM David Cheney
>> >> > <david.che...@canonical.com>
>> >> > wrote:
>> >> >>
>> >> >> Doesn't look like there is windows support, and it uses fcntl
>> >> >> (flock)
>> >> >> under the hood, which is what we have now.
>> >> >
>> >> >
>> >> > flock isn't the problematic thing Tim is talking about. utils/fslock
>> >> > attempts to create a directory in a known location, and if it
>> >> > succeeds,
>> >> > it
>> >> > "holds the lock". Unlocking means removing the directory.
>> >>
>> >> The problem is if the process dies/exits/goes mental while holding the
>> >> lock we get into this existential gridlock where we're not sure if the
>> >> process _is_ alive, so we shouldn't remove the lock, or the process is
>> >> dead, so we should remove the lock.
>> >>
>> >> abstract unix domain sockets do not have this problem on windows; kill
>> >> the process, file is removed from the abstract namespace.
>> >
>> >
>> > POSIX advisory file locks (flock) do not have this problem either. See:
>> > man(2) fcntl, section "Advisory record locking". When the file
>> > descriptor is
>> > closed, the lock is released; file descriptors are closed when the
>> > process
>> > dies.
>> >
>> > We could build a mutex on top of a unix domain socket, but then we'll
>> > have
>> > something completely separate for Windows. Shared named mutex? I'm not
>> > convinced the overall solution would be any more robust, and I'm pretty
>> > sure
>> > it's going to be more complicated. Happy to be proven wrong though.
>> >
>> >> >
>> >> > We would have to contribute Windows support, but it's not hard, I've
>> >> > done it
>> >> > before.
>> >> >
>> >> >>
>> >> >> On Tue, Dec 1, 2015 at 10:55 AM, Casey Marshall
>> >> >> <casey.marsh...@canonical.com> wrote:
>> >> >> > How about github.com/camlistore/lock ?
>> >> >> >
>> >> >> > On Mon, Nov 30, 2015 at 5:43 PM, Tim Penhey
>> >> >> > <tim.pen...@canonical.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Hi folks,
>> >> >> >>
>> >> >> >> The fslock was a mistake that I added to the codebase some time
>> >> >> >> back.
>> >> >> >> It
>> >> >> >> provided an overly simplistic solution to a more complex problem.
>> >> >> >>
>> >> >> >> Really the filesystem shouldn't be used as a locking mechanism.
>> >> >> >>
>> >> >> >> Most of the code that exists for the fslock now is working around
>> >> >> >> its
>> >> >> >> def

Re: Upgrading minimum Go version?

[David Cheney]

Martin: Can you setup a unit test job that uses Go 1.5 ? I will apply
the same process I did for the -race job, to get the build voting,
then reenable the tests that were skipped.

On Fri, Nov 27, 2015 at 12:24 AM, Martin Packman
 wrote:
> On 26/11/2015, Andrew Wilkins  wrote:
>> Hi (mostly Curtis),
>>
>> Is there a plan to bump the minimum Go version? Some of our dependencies do
>> not build with Go 1.2. The LXD provider only builds with Go 1.3 (I think?),
>> and I've got a PR up that updates the azure-sdk-for-go dependency, but it's
>> blocked because the newer doesn't build with Go 1.2.
>
> There are two issues with requiring a newer go version:
>
> * The test suite needs to actually pass reliably with go 1.5
> * How we get a version of juju that doesn't build with go 1.2 into
> trusty needs resolving
>
> With putting the newest juju into older Ubuntu releases, our strategy
> so far has been to SRU back the same source as we put in the current
> release, which we have a special allowance for. We cannot SRU a newer
> go toolchain over the existing working 1.2 version, for hopefully
> obvious reasons. Someone else may have better ideas, but I think that
> leaves us basically three options:
>
> * Stop putting new jujus in older ubuntus
> * Use backports rather than -updates (this is what lxd does I believe)
> * SRU a newer go toolchain to some alternative, juju only location
>
> All of that is pretty big process changes that we can't just go ahead and do.
>
> Test failures have been gradually tackled, we're down to a small-ish
> set of issues causing reliable red results when we run unit tests on
> vivid(1.3) and wily/xenial(1.5).
>
> "TestContainerProvisionerStarted did not start"
> http://reports.vapour.ws/releases/issue/5614b0e7749a5613c5aff2e0
>
> "TestAPI2ResultError fails"
> http://reports.vapour.ws/releases/issue/5640b28f749a562a3f656b39
>
> "unexpected message panic"
> http://reports.vapour.ws/releases/issue/55f34dfd749a561a11823b0b
>
> Seem to be the ones I see most in recent test failures.
>
> Martin
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

version.Current changes

[David Cheney]

Hello,

This is an informational message to explain the recent changes to
version.Current.

version.Current used to be a version.Binary value consisting of the
version number, an architecture and a series value.

As of this afternoon, version.Current is now a version.Number value;
series and arch have been removed.

Why was this done ?

Over the years version.Current has been a fantastically useful set of
values to have around, and as such has been called into service for a
wide range of uses, many of which were unrelated to its original
purpose of tracking the current version of the juju binary.

To counteract this, the uses of version.Current have been teased apart
and in many places replaced with more appropriate abstractions.
version.Current now exists to track the version number of this version
of Juju -- that's it.

Where has version.Current.Number gone ?

version.Current _is_ the version.Number now.

Where has version.Current.Arch gone ?

The arch of the machine the code is running on is available from
github.com/juju/utils/arch.HostArch()

You can patch it with

s.PatchValue(, func() string { "new arch" / arch.Arm64
or other constant })

Please be mindful to only use arch.HostArch when you want the
architecture of the machine you are running on. In many cases, if you
want the architecture of the processor you are running on
runtime.GOARCH may be more appropriate.

Where has version.Current.Series gone ?

The series of the machine the code is running on is available from
github.com/juju/utils/series.HostSeries()

You can patch it with

s.PatchValue(, func() string { "new series" })

Please be mindful to only use series.HostSeries() when you want the
_series_ that matches the machine you are running on. In many cases,
if you want the operating system, runtime.GOOS may be more
appropriate.

Thanks

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Cheryl graduating

[David Cheney]

Congratulations Cheryl!

On Tue, Sep 1, 2015 at 11:46 AM, Tim Penhey  wrote:
> Hi folks,
>
> I have been very happy with the quality of Cheryl's reviews and the time
> is right for her to be considered a fully graduated reviewer.
>
> Thanks for all the reviews Cheryl.
>
> Tim
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Pointer Receivers on Error() and String() methods

[David Cheney]

Yup, I agree with Rog. This is fixing the problem in the wrong place.

On Mon, Aug 24, 2015 at 5:14 PM, roger peppe roger.pe...@canonical.com wrote:
 I'm afraid I'm not convinced. Declaring the Error method on the
 pointer receiver is idiomatic (just grep for ' Error\(' in the Go source)
 and is a useful indicator that the error value is always intended
 to be a pointer.

 There's a much simpler fix for this: let the errors package
 recover from this itself. We can just make Err.Error call fmt.Sprint
 to get the error message (a one line change)

 Then a wrapped nil error will print nil just like normal nil
 errors.


 On 20 August 2015 at 03:45, Nate Finch nate.fi...@canonical.com wrote:
 tl;dr:  Don't.  Use a value receiver.  99% of the time you can just delete
 the * on the receiver and it'll still work.  If you really must use a
 pointer, please handle the case where you're called with a nil receiver.

 I spent most of today trying to understand why my new hook command was
 producing this output:

 error: %!v(PANIC=runtime error: invalid memory address or nil pointer
 dereference)

 It took me a while to figure out that this is what fmt.Printf(error: %v\n,
 err) outputs when err's Error() method panics.  If you're using %s or %v to
 print a value (or use Println which implicitly uses %v), then fmt will look
 for a String() method or Error() method on the value to call, and use the
 output of that for the value's string output.  If that method panics, fmt
 prints the panic in the way you see above (everything after the PANIC=).

 Of course, the problem here is that there's no type being written, and since
 it was an error interface, it could almost anything.  Using %#v skips
 calling the Error/String methods and prints out the values in a go format,
 which told me this was a juju/errors.Err value, wrapping an API params Error
 value which was a nil pointer.  When we call Error() on an errors.Err, we
 call Error() on the cause explicitly, which was panicking.

 Here's a minimal reproduction http://play.golang.org/p/ncNVrza-hn (you'll
 have to copy it to a local file and go run it, since the playground won't
 run code external to the stdlib).

 So what's sort of interesting is that printing the error before it gets
 Traced works fine, but after the trace it is not fine.  The errors.Err's
 Error() function looks like it's explicitly calling the Error() method on
 the wrapped Cause error, which is probably the problem.  fmt.Printf must use
 some reflection magic to avoid doing that.

 Now, the root cause of this particular bug is actually my own mistake.  Line
 21 should check if orig is nil and then assign nil explicitly to err if it
 is.  Then errors.Trace would be able to tell that the error is nil and would
 just return nil itself, instead of thinking it's a valid error and wrapping
 it.

 However, you can sidestep this entirely by doing one of two things:  either
 just make the Error() (or String()) method use a value receiver.. in which
 case this code would produce this output:

 %!v(PANIC=value method main.MyError.Error called using nil *MyError pointer)

 (you can try it with the repro code I linked to)

 This printout is a lot more helpful and useful and obvious than the other
 nil pointer printout.

 OR

 Just handle a nil receiver:

 func (e *MyError) Error() string {
 if e == nil {
 return nil MyError
 }
 return e.Message
 }

 (note that it is dereferencing the pointer to e to access the Message field
 which causes the panic. Calling a method on a nil pointer is totally fine
 and will not panic if the code inside does not try to derefence the pointer
 to get to a field).

 Grepping through our code, I see a lot of pointer receivers on Error and
 String methods (45 and 77 respectively).  I think we should at least change
 all of these to be value methods (unless that's not possible.   That's a
 trivial change, and gives a much more useful printout when the pointer is
 nil.

 -Nate

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: ppc64le timeouts cursing things

[David Cheney]

On Fri, Jul 17, 2015 at 5:08 AM, Dimiter Naydenov
dimiter.nayde...@canonical.com wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 On 17.07.2015 12:07, James Tunnicliffe wrote:
 /me opens can of worms
 Thanks for starting the discussion :)


 Having spent perhaps too long trying to parallelise the running of
 the unit test suite over multiple machines using various bat guano
 crazy ideas, I know too much about this but haven't got an easy
 fix. I do know the right fix is to re-write the very long tests
 that we have.

 If you want to find long running tests, go test -v ./... -check.v
 is the command to run at top level. You will get a lot of output,
 but it isn't difficult to find tests that take longer than 10
 seconds with grep and I am sure I could dig the script out that I
 wrote that examines the output and tells you all tests over a
 certain runtime.

 When you run go test ./... at the top of juju/juju it runs suites
 in parallel. If you have multiple long tests in a suite then it has
 a significant impact on the total runtime. We have no way with the
 current tools to exclude single tests without modifying the tests
 themselves;

 How about GOMAXPROCS=1 go test ./... ? Won't that force the runtime to
 run all suites sequentially?

When you glob packages, ./..., each _package_ is run in parallel, up
to the number of cores on the machine. That is, each package is
compiled in test scope, producing a $PKG.test and that is run in
parallel. Inside each package suites are run in sequence,
alphabetically.

You can adjust the parallelism of the former with the -P flag --
GOMAXPROCS is the wrong hammer for this job.


 if we did we could run all the tests that take less than a few
 seconds by maintaining a list of long tests, and run those long
 tests as a separate, parallel task. The real fix is to put some
 effort into making the long running tests more unit test and less
 full stack test. 30+ seconds is not what we want. The least worst
 idea I have is making a sub-suite for tests that take  10 seconds,
 one test per suite, so the standard tools will run them in parallel
 with everything else. Providing you have many CPUs there is a
 reasonable chance this will help. It is not remotely nice though.

 Using go tool pprof can also help figuring out why certain tests take
 a long time and/or memory. I'm planning to experiment with it and come
 up with some feedback.


 0 ✓ dooferlad@homework2
 ~/dev/go/src/github.com/juju/juju/worker/uniter $ go test -check.v

 Shorter tests deleted from this list. The longest are: PASS:
 uniter_test.go:1508: UniterSuite.TestActionEvents 39.711s PASS:
 uniter_test.go:1114: UniterSuite.TestUniterRelations 16.276s PASS:
 uniter_test.go:970: UniterSuite.TestUniterUpgradeGitConflicts
 11.354s

 These are a worth a look: PASS: uniter_test.go:2053:
 UniterSuite.TestLeadership 5.146s PASS: util_unix_test.go:103:
 UniterSuite.TestRunCommand 6.946s PASS: uniter_test.go:2104:
 UniterSuite.TestStorage 4.593s PASS: uniter_test.go:1367:
 UniterSuite.TestUniterCollectMetrics 4.102s PASS:
 uniter_test.go:774: UniterSuite.TestUniterDeployerConversion
 6.904s PASS: uniter_test.go:427:
 UniterSuite.TestUniterDyingReaction 5.772s PASS:
 uniter_test.go:393: UniterSuite.TestUniterHookSynchronisation
 4.546s PASS: uniter_test.go:1274:
 UniterSuite.TestUniterRelationErrors 4.536s PASS:
 uniter_test.go:476: UniterSuite.TestUniterSteadyStateUpgrade
 6.405s PASS: uniter_test.go:895:
 UniterSuite.TestUniterUpgradeConflicts 6.430s

 ok   github.com/juju/juju/worker/uniter 175.014s

 James

 On 17 July 2015 at 04:59, Tim Penhey tim.pen...@canonical.com
 wrote:
 Hi Curtis,

 I have been looking at some of the recent cursings from ppc64le,
 and the last two included timeouts for the worker/uniter tests.

 On my machine, amd64, i7, 16 gig ram, I get the following:

 $ time go test 2015-07-17 03:53:03 WARNING juju.worker.uniter
 upgrade123.go:26 no uniter state file found for unit
 unit-mysql-0, skipping uniter upgrade step OK: 51 passed PASS ok
 github.com/juju/juju/worker/uniter  433.256s

 real7m24.270s user3m18.647s sys 1m2.472s

 Now lets ignore the the logging output that someone should fix,
 we can see how long it takes here. Given that gccgo on power is
 slower, we are going to do two things:

 1) increase the timeouts for the uniter

 2) change the uniter tests

 WRT to point 2, most of the uniter tests are actually fully
 functional end to end tests, and should not be run every time we
 land code.

 They should be moved into the featuretest package.

 Thanks, Tim

 -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify
 settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev



 - --
 Dimiter Naydenov dimiter.nayde...@canonical.com
 Juju Core Sapphire team http://juju.ubuntu.com
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v2.0.22 (GNU/Linux)

 iQEcBAEBAgAGBQJVqPAiAAoJENzxV2TbLzHwIHYIAKLXI2F4V/Jp+3rFLqbOCrgx
 

Re: Handling user manual intervention in juju

[David Cheney]

You could try expect(1).

Is the product java by any chance ?

On Mon, Jun 22, 2015 at 7:44 PM, Shilpa Kaul shilk...@in.ibm.com wrote:
 Hi Team,

 Can you please help me in finding a solution for a scenario where product
 configuration or installation involves manual intervention and product does
 not support silent method for that step.How can this be handled in a charm.

 For example , consider a scenario where a product is installed and
 configured using ./install command. The command takes care of all install
 and configuration but it stops at Accept license step. The user has to
 either enter 1 for accepting license or 2 for not accepting license. Till
 now I have not got any equivalent command line option to (ie waiting for
 license to be accepted) accept the license by default so that install
 process does not stop for any manual user intervention.

 How can charm handle such type of scenarios , any insights on this please.

 Thanks and Regards,
 Shilpa Kaul


 --
 Juju mailing list
 Juju@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju


-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: I'm concerned

[David Cheney]

This should be achievable. go test sends SIGQUIT on timeout, we can
setup a SIGQUIT handler in the topmost suite (or import it as a side
effect package), do whatever cleanup is needed, then os.Exit, unhandle
the signal and try to send SIGQUIT to ourselves, or just panic.

On Wed, Jun 17, 2015 at 3:25 PM, Tim Penhey tim.pen...@canonical.com wrote:
 Hey team,

 I am getting more and more concerned about the length of time that
 master has been cursed.

 It seems that sometime recently we have introduced serious instability
 in cmd/jujud/agent, and it is often getting wedged and killed by the
 test timeout.

 I have spent some time looking, but I have not yet found a definitive
 cause.  At least some of the time the agent is failing to stop and is
 deadlocked.

 This is an intermittent failure, but intermittent enough that often at
 least one of the unit test runs fails with this problem cursing the
 entire run.

 One think I have considered to aid in the debugging is to add some code
 to the juju base suites somewhere (or in testing) that adds a goroutine
 that will dump the gocheck log just before the test gets killed due to
 timeout - perhaps a minute before. Not sure if we have access to the
 timeout or not, but we can at least make a sensible guess.

 This would give us at least some logging to work through on these
 situations where the test is getting killed due to running too long.

 If no one looks at this and fixes it overnight, I'll start poking it
 with a long stick tomorrow.

 Cheers,
 Tim

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Please run your tests with -race

[David Cheney]

Here is a report of the current races. http://paste.ubuntu.com/11256308/

On Thu, May 21, 2015 at 11:10 AM, Tim Penhey tim.pen...@canonical.com wrote:
 Thanks Dave for this great write up.

 In order to drive the data races to zero, and make a soon to be added CI
 test as voting, I propose that as folks are writing tests in packages,
 races in at least that file, an possibly that package should be fixed.

 Tim

 On 21/05/15 12:12, David Cheney wrote:
 Hello,

 TL;DR - juju has lots of problems with data races, please test your
 code with the -race flag to ensure it doesn't get worse while we try
 to fix the problem.

 Longer version:

 In debugging https://bugs.launchpad.net/bugs/1456398 I found that
 there are multiple data races in the Juju code base. It's been long
 suspected that the test's are racy, PatchValue is super easy to
 introduce a race if all the workers started by a test have not exited
 before the suite's tear down function runs.

 However, more serious races have been discovered, such as
 https://bugs.launchpad.net/bugs/1456857 which affects code all the way
 back to 1.22.

 Why is a data race bad ?

 Ok, so you're looking at https://bugs.launchpad.net/bugs/1456857 and
 you're thinking, so maybe the tls code accidentally uses the wrong
 certificate for a little bit, how bad is that?

 The problem is data races affect the integrity of the structures that
 the garbage collector uses. In the example above replacing the
 certificate means one CPU can see the new value, and another
 potentially the old value. When it comes to to run the gc, depending
 on which CPU walks that chain of pointers it may think that the old
 certificate is still live, or the new certificate is unreachable --
 and boom that memory is marked as free and the certificate corrupted.

 The short version is: there are no safe data races, and Juju is not
 reliable until they have all been fixed.

 How to run the race detector ?

 The race detector comes with Go and is available by adding the -race
 flag to invocations of go test, so what was

 go test github.com/juju/juju/...

 becomes

 go test -race github.com/juju/juju/...

 The downside of this is the race detector has significant overhead, at
 least 2x, so tests will be even slower.

 Thanks

 Dave



-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Please run your tests with -race

[David Cheney]

Hello,

TL;DR - juju has lots of problems with data races, please test your
code with the -race flag to ensure it doesn't get worse while we try
to fix the problem.

Longer version:

In debugging https://bugs.launchpad.net/bugs/1456398 I found that
there are multiple data races in the Juju code base. It's been long
suspected that the test's are racy, PatchValue is super easy to
introduce a race if all the workers started by a test have not exited
before the suite's tear down function runs.

However, more serious races have been discovered, such as
https://bugs.launchpad.net/bugs/1456857 which affects code all the way
back to 1.22.

Why is a data race bad ?

Ok, so you're looking at https://bugs.launchpad.net/bugs/1456857 and
you're thinking, so maybe the tls code accidentally uses the wrong
certificate for a little bit, how bad is that?

The problem is data races affect the integrity of the structures that
the garbage collector uses. In the example above replacing the
certificate means one CPU can see the new value, and another
potentially the old value. When it comes to to run the gc, depending
on which CPU walks that chain of pointers it may think that the old
certificate is still live, or the new certificate is unreachable --
and boom that memory is marked as free and the certificate corrupted.

The short version is: there are no safe data races, and Juju is not
reliable until they have all been fixed.

How to run the race detector ?

The race detector comes with Go and is available by adding the -race
flag to invocations of go test, so what was

go test github.com/juju/juju/...

becomes

go test -race github.com/juju/juju/...

The downside of this is the race detector has significant overhead, at
least 2x, so tests will be even slower.

Thanks

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Public Service Annoucement

[David Cheney]

Why doesn't the bot run this hook ?

On Fri, Apr 24, 2015 at 9:54 PM, Matthew Williams
matthew.willi...@canonical.com wrote:
 There's a very useful pre-push hook available for juju that does useful
 things like run gofmt, go vet, checks we can build etc.

 If you don't have it setup there's some lovely instructions in
 Contributing.md:

 https://github.com/juju/juju/blob/master/CONTRIBUTING.md#local-clone

 Thanks for listening

 Matty

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Public Service Annoucement

[David Cheney]

Sorry. I was not clear.

Why isn't the bot running the pre-commit hook as part of the CI test ?

What we have now is a warning, gleefully ignored.

What we need is an error, which rejects the PR if it doesn't pass the
pre-commit hook.

On Fri, Apr 24, 2015 at 10:28 PM, John Meinel j...@arbash-meinel.com wrote:
 The bot cannot make changes to the tree and commit it back to trunk, because
 it doesn't have direct push rights to github. Instead it has API rights to
 accept this merge. So even if it did run the changes, it wouldn't end up
 in trunk, and we could only reject the merge if it changed things. (As I
 understand it)

 John
 =:-

 On Fri, Apr 24, 2015 at 3:56 PM, David Cheney david.che...@canonical.com
 wrote:

 Why doesn't the bot run this hook ?

 On Fri, Apr 24, 2015 at 9:54 PM, Matthew Williams
 matthew.willi...@canonical.com wrote:
  There's a very useful pre-push hook available for juju that does useful
  things like run gofmt, go vet, checks we can build etc.
 
  If you don't have it setup there's some lovely instructions in
  Contributing.md:
 
  https://github.com/juju/juju/blob/master/CONTRIBUTING.md#local-clone
 
  Thanks for listening
 
  Matty
 
  --
  Juju-dev mailing list
  Juju-dev@lists.ubuntu.com
  Modify settings or unsubscribe at:
  https://lists.ubuntu.com/mailman/listinfo/juju-dev
 

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev



-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

linux/arm64 benchmark improvements

[David Cheney]

Hello,

Here are the results of the work Aram and I did last week on improving
the performance of our arm64 port of Go. This is comparing the build
from April 06 (before we started) to today (April 16th).

http://paste.ubuntu.com/10831430/

Cheers

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Proposal: move package replicaset to its own GitHub repo

[David Cheney]

Do it!

On Wed, Mar 25, 2015 at 9:22 PM, Nate Finch nate.fi...@canonical.com wrote:
 +100, thanks for doing this, Andrew!

 On Wed, Mar 25, 2015 at 6:16 AM, roger peppe roger.pe...@canonical.com
 wrote:

 This seems like a grand plan to me.

 On 25 March 2015 at 08:12, Andrew Wilkins andrew.wilk...@canonical.com
 wrote:
  I'd like to move the replicaset package to github.com/juju/replicaset.
  For
  rationale and preparatory changes, see: http://reviews.vapour.ws/r/1260/
 
  Please debate here. Silence == tacit approval.
 
  Thanks,
  Andrew
 
  --
  Juju-dev mailing list
  Juju-dev@lists.ubuntu.com
  Modify settings or unsubscribe at:
  https://lists.ubuntu.com/mailman/listinfo/juju-dev
 

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev



 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Testing on windows

[David Cheney]

It is clear that we need blocking CI jobs for Windows and CentOS --
has this been added to the Nuremberg agenda ?

On Thu, Mar 19, 2015 at 10:24 PM, Gabriel Samfira
gsamf...@cloudbasesolutions.com wrote:
 On 19.03.2015 12:07, Michael Foord wrote:

 Do all tests pass (or skip) on Windows now? Do we have a CI job
 running them?
 There is a PR under review, and one other fix pending that will make all
 tests pass in juju core again. There is no job yet in the CI that blocks
 master if tests fail on Windows. We should be able to turn it on once:
 http://reviews.vapour.ws/r/1119/ merges and one other fix lands in
 github.com/juju/testing.

 Until then, it is still useful to test on Windows before pushing, to
 limit the risk of breaking anything else. Its also good practice, as
 windows does offer some differences from Linux, and we need to be aware
 of them when writing code. We also need to get accustomed to writing new
 features with multiple platforms in mind. It would really be bad to
 create a bigger gap between the platforms then already exists (playing
 catch up is never a good thing).

 A set of branches for CentOS support will soon be proposed, and soon
 after that Debian (Jessie), so testing on windows should be good
 practice for when that happens :).

 Cheers,
 Gabriel





 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: FOSDEM 2015 - Juju Talk is now online!

[David Cheney]

Nice work Chuck!

On Thu, Feb 26, 2015 at 4:47 AM, Charles Butler
charles.but...@canonical.com wrote:
 If you missed out on FOSDEM 2015 and the awesomeness of the Configuration
 Management room talks, don't fret! The video has been posted! Big thanks to
 the organizers of FOSDEM for making this video available under a Creative
 Commons 2.0 Attribution license.

 2 sources:

 https://www.youtube.com/watch?v=JOq8YrRUFFc


 http://mirror.as35701.net/video.fosdem.org//2015/devroom-configuration_management/juju_orchestration.mp4



 --
 All the best,

 Charles Butler charles.but...@canonical.com - Juju Charmer
 Come see the future of datacenter orchestration: http://jujucharms.com

 --
 Juju mailing list
 Juju@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju


-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

What are blocks ?

[David Cheney]

Hello,

Can someone direct me to the design documents for the enigmatic
'block' functionality which is showing up in the review queue.

Thanks

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: separating out golang juju client from

[David Cheney]

There is no reason for the 130 (at last count) packages that
constitute juju-core (not counting the dozens of other packages we
bring in as dependencies) to live in the same repository.

If licensing is the lever that we use to break up this monolithic
repository, consider me +1

On Fri, Dec 19, 2014 at 11:05 PM, Kapil Thangavelu
kapil.thangav...@canonical.com wrote:


 On Fri, Dec 19, 2014 at 7:02 AM, Nate Finch nate.fi...@canonical.com
 wrote:

 While I am generally for using more permissive licenses, I'm not sure how
 useful that might be... most significant changes require modifications to
 both the client and the server, or at least to libraries used by both.


 That sort of misses the point of building apps that use juju apis. Yes the
 two packages need to be updated together for new changes same as today.


 There's not that much code under cmd/juju compared to the whole rest of
 the repo.


 Again its not about that code, its about building other applications and
 facilitating integrations.


 cheers,
 Kapil


 On Fri, Dec 19, 2014 at 6:03 AM, Kapil Thangavelu
 kapil.thangav...@canonical.com wrote:

 one of the issues with having it in tree, means client usage falls under
 the AGPL. We want to have the client used widely under a more permissive
 license. I've already had contributions to other projects n'acked due to
 license on our libraries. I'd like to see it moved to a separate repo so
 that's possible. Thoughts?

 cheers,
 Kapil



 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: git clients vulnerability found

[David Cheney]

Fortunately all of us run an operating system with a case sensitive
file system, right ?

On Fri, Dec 19, 2014 at 8:32 AM, Horacio Duran
horacio.du...@canonical.com wrote:
 Heads up people
 https://github.com/blog/1938-vulnerability-announced-update-your-git-clients
 apparently there is a vulnerability in git client that might be exploited
 via pushing certain snippets to the repos.

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Too space, or not two space

[David Cheney]

 I think you guys have more things to worry about than the spacing style for
 comments.

I'm happy to let this discussion die at this point.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Too space, or not two space

[David Cheney]

To quote Butterick's
http://practicaltypography.com/one-space-between-sentences.html

 I think two spaces look bet­ter so that’s what I’m go­ing to use.

I’m telling you the rule. If you want to put per­sonal taste ahead of
the rule, I can’t stop you. But per­sonal taste doesn’t make the rule
evap­o­rate. It’s like say­ing “I don’t like how sub­junc­tive-mood
verbs sound, so I’m never go­ing to use them.”

On Wed, Nov 19, 2014 at 11:10 AM, Jesse Meek jesse.m...@canonical.com wrote:
 I'm coming across double spaces after a full stop in comments. Some consider
 this an error, others an intentional style that makes the comment cleaner to
 read.

 Yes, it's a minor point but in the name of consistency and avoiding future
 review ping pong on the issue, I'm calling for a vote:

 Should we have a single space after full stops in comments?
 http://xkcd.com/1285/

 Please reply +1 for yes to one space, -1 for no to one space. I'll tally up
 votes on Friday and update our style guide accordingly.

 My vote: +1

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Please stop adding dependencies to the state package

[David Cheney]

This is https://bugs.launchpad.net/juju-core/+bug/1392520

I have assigned it to myself.

On Fri, Nov 14, 2014 at 10:33 AM, Eric Snow eric.s...@canonical.com wrote:
 On Thu, Nov 13, 2014 at 4:05 PM, David Cheney
 david.che...@canonical.com wrote:
 Hello,

 Please stop adding dependencies to the state package.

 The dependency tree for state must go

 mgo - state - things that depend on state

 At the moment state depends on things like backups, multiwatchers,
 presence, toolstorage.

 This is wrong.

 Eric, i'm not picking on you here, it's just that b sorts first, and
 this is the one I saw today. There are many other examples, including
 the multiwatcher which also break this model

 If you add functionality to state, then the code has to go ***INSIDE**
 state, or be written to take a *state.State.

 Please help me out by not adding more dependencies to state.

 Thanks for pointing this out, Dave.  I'll fix this for backups.

 -eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: how to update dependencies.tsv

[David Cheney]

Does your $GOPATH include more than one entry ?

The make target should probably just expect godeps to be in your $PATH.

On Wed, Oct 29, 2014 at 6:37 PM, Dimiter Naydenov 
dimiter.nayde...@canonical.com wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 +1, in fact there is already a godeps target in the Makefile, but I
 never managed to get it working. It's looking for $(GOPATH)/bin/godeps
 binary and I have it right there on my machine, but make still claims
  File `godeps' does not exist. (when running $ make -d godeps).

 Somebody with better make-fu knowledge should probably give a hand
 with this :)

 On 29.10.2014 05:58, John Meinel wrote:
  can we please just have make dependencies.tsv do the right thing
  so we don't have to remember which set of flags and env vars we
  need to use this time?
 
  I'm also not 100% sure that we'll have even downloaded all the
  windows dependencies if they are a strict superset given that you
  are running go get on a Linux machine to start with.
 
  John =:-
 
 
  On Tue, Oct 28, 2014 at 11:41 PM, Nate Finch
  nate.fi...@canonical.com mailto:nate.fi...@canonical.com
  wrote:
 
  We have a few windows dependencies that are not (and in some cases
  can not be) imported for the linux build.  Luckily, the windows
  dependencies are a strict super set of the linux dependencies, so
  we can still use godeps to create dependencies.tsv, just by
  setting GOOS first, thusly (from the root of github.com/juju/juju
  http://github.com/juju/juju):
 
  GOOS=windows godeps -t ./...  dependencies.tsv
 
  Please use this command line when updating dependencies.tsv so
  that is keeps a standard format which will minimize conflicts in
  the file.
 
  -Nate
 
  -- Juju-dev mailing list Juju-dev@lists.ubuntu.com
  mailto:Juju-dev@lists.ubuntu.com Modify settings or unsubscribe
  at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
 
 
 
 


 - --
 Dimiter Naydenov dimiter.nayde...@canonical.com
 juju-core team
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1

 iQEcBAEBAgAGBQJUUJlIAAoJENzxV2TbLzHwND0IAMS3c79LeWwTDnf6sEyMwTcm
 HdvBxaxEeJLNqv8Sq+mK4jWyMAtmoQ/9Azup7g9DZmb1aJXD94n25nf19NNdBkxi
 wzLBMqq3iEkKY+09js8r5S4ScRMOErBrhL7wd4+PDFTL1y5SeDAacBDCzZQKeQlY
 IRkIfOXjfSpQuALONZNUlToMAuTQzC80gf914QX7DOpKMFhsiS6cMIb2nYlAScEZ
 UFjJwRrTEgjiGoouq/KnmahhYK4i4dsdQDy9a44lTgIb+igoAi6OBzTJ8ktqac+G
 +e+sS25SHJ+T2R+iPtlJnExW3raa9vHxIJQBazB8C/QjLDqrOtzt1XSe5WxlQxs=
 =akp+
 -END PGP SIGNATURE-

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: wordpress charm internal deps

[David Cheney]

At a guess these dependencies come in from the image processing libraries
that wordpress needs.

On Fri, Oct 17, 2014 at 10:50 PM, Vasiliy Tolstov v.tols...@selfip.ru
wrote:

 Hi. Can somebody explain me, why wordpress charm try to install so
 many unneeded packages:

 2014-10-17 11:34:16 INFO db-relation-changed   bzr libapr1 libaprutil1
 libassuan0 libdrm-intel1 libdrm-nouveau2
 2014-10-17 11:34:16 INFO db-relation-changed   libdrm-radeon1
 libfontenc1 libgl1-mesa-dri libgl1-mesa-glx libglapi-mesa
 2014-10-17 11:34:16 INFO db-relation-changed   libgpgme11 libice6
 libjs-jquery libllvm3.4 libpciaccess0 libserf-1-1 libsm6
 2014-10-17 11:34:16 INFO db-relation-changed   libsvn1 libtcl8.6
 libtk8.6 libtxc-dxtn-s2tc0 libutempter0 libx11-xcb1
 2014-10-17 11:34:16 INFO db-relation-changed   libxaw7 libxcb-dri2-0
 libxcb-dri3-0 libxcb-glx0 libxcb-present0
 2014-10-17 11:34:16 INFO db-relation-changed   libxcb-shape0
 libxcb-sync1 libxcomposite1 libxdamage1 libxfixes3 libxft2
 2014-10-17 11:34:16 INFO db-relation-changed   libxi6 libxinerama1
 libxmu6 libxrandr2 libxrender1 libxshmfence1 libxss1
 2014-10-17 11:34:16 INFO db-relation-changed   libxt6 libxtst6 libxv1
 libxxf86dga1 libxxf86vm1 mercurial mercurial-common
 2014-10-17 11:34:16 INFO db-relation-changed   python-bzrlib
 python-crypto python-dbus python-dbus-dev python-gi
 2014-10-17 11:34:16 INFO db-relation-changed   python-gpgme
 python-httplib2 python-keyring python-launchpadlib
 2014-10-17 11:34:16 INFO db-relation-changed
 python-lazr.restfulclient python-lazr.uri python-paramiko
 2014-10-17 11:34:16 INFO db-relation-changed   python-secretstorage
 python-simplejson python-wadllib subversion tcl tcl8.6
 2014-10-17 11:34:16 INFO db-relation-changed   tk tk8.6 x11-common
 x11-utils xbitmaps xterm

 Why in server drm, tk, xterm , libx libraries?

 --
 Vasiliy Tolstov,
 e-mail: v.tols...@selfip.ru
 jabber: v...@selfip.ru

 --
 Juju mailing list
 Juju@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Is there a setting in ReviewBoard to let me see *both* the comments and the diffs

[David Cheney]

Hi

Reviewboard is driving me nuts. Either it can't do this, or I have put
it in some mode where it does this, but it can only show me the review
comments *OR* the diff, but not both.

Is there a setting that shows both diffs and the stream of review
comments _on_the_same_screen ?

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Versioned packages and testing dependencies

[David Cheney]

John, Roger,

This situation right here is why I am strongly opposed to the gopkg.in
versioning model. I recommend reverting to the original launchpad
location.

Dave

On Mon, Sep 22, 2014 at 9:56 PM, roger peppe roger.pe...@canonical.com wrote:
 That's an interesting observation and I think I agree.

 The general rule is probably something like this:

 - If a type is part of the exported API of a versioned package and the
 package is changed to import that type from somewhere
 else, the package's version must be incremented.

 Given that versions usually apply to repositories, not packages,
 that does unfortunately mean that all the repositories which
 export testing functionality using gocheck need to have
 their versions incremented.

 I don't think it all has to happen at the same time, because APIs
 remain stable. It does mean that juju-core can only be updated
 after all of its dependencies have been updated though.

   cheers,
 rog.

 On 22 September 2014 12:14, John Meinel j...@arbash-meinel.com wrote:
 So Bogdan Teleaga was kind enough to put in the effort to move all of our
 source trees to start importing from gopkg.in/check.v1 rather than
 depending on labix.org/gocheck.

 However, this means that if we land those changes, code that depends on the
 testing infrastructure provided by those packages will break because a
 labix.org/gocheck.Suite is not a gopkg.in/check.v1.Suite.
 And even further, Checkers are different (so we have to update
 github.com/juju/testing/checkers/ before other code.)

 Since we're doing versioned stable APIs I'm wondering if this means things
 need a version bump. I *think* the answer is yes, because if you just did:
   go get gopkg.in/juju/charm.v3

 Before the change, you could run cd github.com/juju/juju; go test ./...
 but if we change just charm.v3 but not juju/juju then go test ./... just
 breaks.
 Thus, changing your testing infrastructure while it doesn't change your API
 it changes your *test interface* which makes it an unstable API break.

 And we have the unfortunate process that *all* of our code is tested via
 'gocheck' which means that we have to update-the-world to rev its import.

 Certainly I'd rather not do the work if we don't have to, so I'd like other
 people's input if we feel we have to.

 John
 =:-


 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: psa pls don't use transactions for single doc unobserved atomic updates

[David Cheney]

Hi Kapil,

I'm sure that the 'put everything in a transaction' pattern is applied
unilaterally.

Do you have some specific operations in mind ?

Dave

On Mon, Sep 22, 2014 at 7:46 AM, Kapil Thangavelu
kapil.thangav...@canonical.com wrote:
 Its sort of misses the point on why we're doing client side transactions.
 Mongodb has builtin atomic operations on an individual document. We use
 client side txns (multiple order of magnitude slower) for multi-document
 txns *and/or* things we want to observe for watches.

 -k


 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Is ReviewBoard a good thing?

[David Cheney]

On Sat, Sep 20, 2014 at 1:02 AM, Eric Snow eric.s...@canonical.com wrote:
 On Thu, Sep 18, 2014 at 6:32 PM, David Cheney
 david.che...@canonical.com wrote:
 There were three problem reviewboard was supposed to address, review
 comments are sent immediately, no side by side diffs, and no way to to
 chained proposals.

 It will be worth being extra clear on ReviewBoard's pros and cons.
 I'll open a new thread.


 I think that over the last few months we've all learnt to live with
 the first issue

 I for one haven't.  On several occasions I have made a comment that I
 later realized was not valid, but am not able to simply remove since
 it already sent.


 On the second, github now does nice side by side diffs.

 Agreed.


 On the third, it appears reviewboard leaves you solidly to your own
 devices to implement chained proposals.

 rbt supports it just fine and reviewboard itself supports it (see the
 Depends on field on every review request).  What support did you
 have in mind?

Ian and Tim want whatever they had with launchpad. I never used lp
like that so I never felt the loss; I just propose one branch at a
time and have others waiting in the wings that I can propose as soon
as the predecessor lands.


 -eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Doing chained diffs w/ Reviewboard

[David Cheney]

+1 to that

On Thu, Sep 18, 2014 at 7:53 PM, Adam Collard
adam.coll...@canonical.com wrote:
 On 18 September 2014 10:49, John Meinel j...@arbash-meinel.com wrote:

 Has anyone succeeded in getting this to work?

 The steps I tried to do were:

  git co master
  git pull upstream master
  git co base-branch
  git diff master...  base.diff
  git co dependent-branch
  git diff master...  dependent.diff
  git merge-base master HEAD  remember-this-rev

 And then put the dependent.diff into the Diff: *, and then the
 base.diff into Parent Diff: and then 'remember-this-rev' into the Base
 Commit ID.

 I also tried putting git merge-base master base-branch as the Base
 Commit ID.


 This makes me think you're using the UI to do this.

 Let me repeat my Public Safety Announcement: Do NOT use ReviewBoard's UI for
 uploading diffs. Please for $deity's sake use rbt post.

 https://www.reviewboard.org/docs/rbtools/0.6/rbt/commands/post/#distributed-version-control-systems

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Doing chained diffs w/ Reviewboard

[David Cheney]

Also, be watchful for the other reviewboard footgun, paged diffs.

Reviewboard pages large reviews, so if you're used to thinking 'phew,
i've gotten to the end of the page, i'm done, check again, there
maybe a surprise waiting for you at the bottom of the page.

On Thu, Sep 18, 2014 at 9:03 PM, David Cheney
david.che...@canonical.com wrote:
 +1 to that

 On Thu, Sep 18, 2014 at 7:53 PM, Adam Collard
 adam.coll...@canonical.com wrote:
 On 18 September 2014 10:49, John Meinel j...@arbash-meinel.com wrote:

 Has anyone succeeded in getting this to work?

 The steps I tried to do were:

  git co master
  git pull upstream master
  git co base-branch
  git diff master...  base.diff
  git co dependent-branch
  git diff master...  dependent.diff
  git merge-base master HEAD  remember-this-rev

 And then put the dependent.diff into the Diff: *, and then the
 base.diff into Parent Diff: and then 'remember-this-rev' into the Base
 Commit ID.

 I also tried putting git merge-base master base-branch as the Base
 Commit ID.


 This makes me think you're using the UI to do this.

 Let me repeat my Public Safety Announcement: Do NOT use ReviewBoard's UI for
 uploading diffs. Please for $deity's sake use rbt post.

 https://www.reviewboard.org/docs/rbtools/0.6/rbt/commands/post/#distributed-version-control-systems

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Is ReviewBoard a good thing?

[David Cheney]

There were three problem reviewboard was supposed to address, review
comments are sent immediately, no side by side diffs, and no way to to
chained proposals.

I think that over the last few months we've all learnt to live with
the first issue

On the second, github now does nice side by side diffs.

On the third, it appears reviewboard leaves you solidly to your own
devices to implement chained proposals.

I'm with Jesse, I vote to stop using reviewboard, I don't think it's
paying it's way.


On Fri, Sep 19, 2014 at 10:19 AM, Jesse Meek jesse.m...@canonical.com wrote:
 We moved to GitHub in the hope of lowering the bar to contributors outside
 the team. GitHub is *the* platform and process for open source software.
 This was the logic behind the move. It was deemed to have the most mindshare
 and we sacrificed our prefered platform and process to be part of that
 mindshare.

 We are now leaving that 'main stream' process to something that suits the
 tastes of our team - ReviewBoard. This adds friction for new contributors
 (friction everyone has experienced this week). If we value our preferred
 methods of reviewing over keeping to a well known process for outside
 contributors, the best process was launchpad + rietveld. Shouldn't we simply
 return to that.

 Considering we have been successfully using GitHub for several months now,
 using reviewboard is not a necessity. Obviously, I will go with whatever the
 team decides, but I'm concerned that we have moved to reviewboard without
 considering that it undermines (as far as I can see) our primary reason for
 using GitHub.

 Jess

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: State should not depend on the API server

[David Cheney]

On Tue, Sep 16, 2014 at 7:59 PM, William Reade
william.re...@canonical.com wrote:
 On Thu, Sep 11, 2014 at 3:35 PM, Nate Finch nate.fi...@canonical.com wrote:
 I don't see how having different identical structs that are updated
 simultaneously in any way prevents any problems with compatibility.

 If we're updating those structs simultaneously, we're completely
 missing the point. Once we've defined a pure-data struct that might be
 persisted or sent over the wire we *must not change that struct* -- if
 we want to send or store different data, we need a new struct.

 Maybe I'm missing something from the proposal, but it seems like this just
 adds busywork without actually solving any problems that weren't caused by
 incorrect use of the code in the first place.

 Isn't that tautological? AFAICS, storing a charm.Meta (or a
 params.anything) directly in the DB *is* incorrect use of the code,
 but nobody realises that until it's too late: that is the problem, and
 that's what we're addressing.

For example, apiserver/params.StateServingInfo was being stored
directly in the database. Woe to the person that updates the api only
to discover that data was silently deleted from the state database :(


 Separation of logic is absolutely a good thing.  Separation of data is not
 nearly so useful.

 It's harder than you might think to separate the data from the logic
 that acts on it ;).

 Cheers
 William

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: ReviewBoard is now the official review tool for juju

[David Cheney]

On Tue, Sep 16, 2014 at 7:27 PM, roger peppe roger.pe...@canonical.com wrote:
 On 16 September 2014 09:22, Jonathan Aquilina jaquil...@eagleeyet.net wrote:
 If i am not mistaken if you have multiple commits in a branch git has
 something built in called git squash. This obviously eliminates the 5 step
 process into one merge and one push.

 I don't see that command. Are you thinking of the squash
 functionality of rebase -i?

 FWIW, I never run those five steps in sequence together.
 Usually I just get to a situation where I know that I have all tests
 passing and I'm up to date with master (for example I've done a merge
 some time ago, probably before fixing a bunch of tests).

 Then it's just:

 $ git reset upstream/master
 $ git commit -am 'my commit message'

Nice trick, so that resets the underlying branch to master, leaving
everything you have comitted or merged uncomitted ?


 which is usually quicker than running rebase -i, and much
 quicker if the rebase replay leads to conflicts.

   cheers,
 rog.

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: ReviewBoard is now the official review tool for juju

[David Cheney]

Is there a setting to make reviewboard show the diff in a review by
the default ? For some reason for me it always requires me to hit the
'view diff' link.

On Tue, Sep 16, 2014 at 6:39 AM, Ian Booth ian.bo...@canonical.com wrote:


 On 16/09/14 00:50, Eric Snow wrote:
 On Mon, Sep 15, 2014 at 8:09 AM, Eric Snow eric.s...@canonical.com wrote:
 Yeah, those steps are a lot, though keep in mind that effectively it's
 only 2 steps more than before if you use the -p flag to rbt post and
 were already keeping your local master up to date.

 Just to be clear, here are the steps again, slightly reformatted:

 (0). Rebase relative to upstream master.
   - if origin is different than upstream, sync and push it

 Before I create a new branch, I ensure my local and origin (forked copy) 
 master
 branches are up to date. However, once the branch is created, thereafter I do
 not rebase because it has caused nothing but trouble, with lots of manual 
 effort
 required to fix things up wrt conflicts etc. And it should not be necessary -
 the code repository should be able to track the state of play such that when 
 you
 request a merge, it knows how to create the correct diff against the current
 official trunk which will be merged into.


 Step (0) is also pretty easy and I'll argue that people should be
 doing it anyway.


 Disagree :-)
 I never (or very, very rarely) had to do this with Launchpad and bzr and 
 things
 Just Worked. I don't do it now with github and pull requests. I'd like to 
 think
 we'd be able to avoid the burden moving forward also.

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A beginner's adventure in Charm authoring

[David Cheney]

On Fri, Sep 5, 2014 at 5:04 AM, Nate Finch nate.fi...@canonical.com wrote:
 given that we currently use the path, you can't have one charm for multiple
 series anyway.

For deploying local: charms, symlinks work fine here.

 This would at least be better than what we have right now,
 and would be backwards compatible (older jujus would just require the old
 style local deploy and would ignore the extra series specification in the
 metadata).


 On Thu, Sep 4, 2014 at 2:58 PM, Curtis Hovey-Canonical
 cur...@canonical.com wrote:

 On Thu, Sep 4, 2014 at 3:26 AM, John Meinel j...@arbash-meinel.com
 wrote:
 ...
  At the very least we need to know what OS Series the charm is targeting.
  Which is currently only inferred from the path. I don't particularly
  like
  it, and I think the code that searches your whole repository and then
  picks
  the best one is bad, as it confuses people far more often than it is
  helpful.
  (If you have $REPO, and have $REPO/precise/charm and
  $REPO/precise/charm-backup but the 'revision' number in charm-backup is
  higher for whatever reason, juju deploy --repository=$REPO charm will
  actually deploy charm-backup)

 I thought we agreed in Burlington that the charm can declare the
 series in the metadata.yaml
 series: trusty

 A list of series was rejected I recall. There was a plan to change
 charm store ingestion to read the metadata.yaml. Maybe this fell apart
 because without support for a list of series/oses, you cannot have one
 charm supporting more than one series.

 --
 Curtis Hovey
 Canonical Cloud Development and Operations
 http://launchpad.net/~sinzui



 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Copyright information in headers

[David Cheney]

I think this is needless busy work, I vote that as long as there is a
copyright header with _a_ year, that is sufficient.

On Thu, Sep 4, 2014 at 7:50 AM, Ian Booth ian.bo...@canonical.com wrote:
 Hi folks

 The question recently came up in reviews as to whether we should be updating 
 the
 date in the copyright statement in the file header when we make a change to 
 the
 code in that file. I sought clarification from Robie Basak, who previously had
 provided input on licensing issues and compliance for getting Juju included in
 trusty. Below is what he said.

 TL;DR;
 It doesn't really matter, we just need to agree on a policy. It is suggested
 though that we do update the date when we make a change. Agree?

 snip

 What's our policy for dates in copyright headers?

 // Copyright 2012, 2013 Canonical Ltd.
 // Licensed under the AGPLv3, see LICENCE file for details.

 From the point of view of acceptability for Ubuntu, it doesn't
 particularly matter, and I don't believe it'll cause any issue for us
 whatever you do here. I'll certainly be happy to upload whether or not
 you update the date.

 I'll try to explain my perspective on this, but I'm not entirely
 confident that there isn't something I'm missing for the broader
 picture, so note that I Am Not A Lawyer, etc.

 For the above, do we need to add 2014 if we modify the file this year?
 Or is the date just meant to be the year the file was first published?

 I think it's meant to be the sum of all the copyright claims on the
 file. So if you add some new code, you have a copyright claim on the new
 code in the newer year in which you made it.

 AIUI, the purpose of the date is that since copyright expires
 (theoretically, anyway), updating the date updates the copyright claim,
 which would give us more control in the (eventual) event that copyright
 expires.

 In practice, IMHO this is never going to matter since nobody is going to
 care about the copyright on a piece of software that is that old anyway.
 But I suppose laws could change, so the right thing to do would be to
 add a new year whenever you make a change in a new year on a per-file
 file basis. BTW, it's common to fold 2012, 2013, 2014 to just
 2012-2014.

 But I don't particularly care for upload purposes.



 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Logger prefixes for api/apiserver are changing

[David Cheney]

Wow. I'm sorry I broke that, it didn't even occur to me.

I wonder if there is a better way to handle this default case of the
name of the logger follows the package path), ie

var logger = loggo.Logger() // or something, it could be a new
method, logger.Default(), or something.

which will inspect the call tree and determine its logger name automagically.

On Tue, Sep 2, 2014 at 8:00 PM, Dimiter Naydenov
dimiter.nayde...@canonical.com wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 Hi,

 As you may already know, this pull request just landed today, moving
 state/api/ - api/ and state/apiserver/ - apiserver/ :
 https://github.com/juju/juju/pull/655

 It didn't fix the logger prefixes in some places, which I'm fixing
 with this follow-up: https://github.com/juju/juju/pull/659

 So be aware anything in the moved to top-level packages api and
 apiserver will use juju.api.* and juju.apiserver.* logger
 prefixes from now on (existing code will be converted as soon as #659
 lands, but for new code please follow the same pattern).

 Cheers!
 - --
 Dimiter Naydenov dimiter.nayde...@canonical.com
 juju-core team
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1

 iQEcBAEBAgAGBQJUBZUhAAoJENzxV2TbLzHwewkH/AhtPOv1h3w+9LkyQdmH7Pbl
 28cucj422TOXIek65+tB4fGmeCglQTPJkaLcsImWxKhYU7oAP9iT52lKjHiV377t
 l7sSVprKWp1AfF307lT6xI8derNnoSJtEHTwbXZKc4LaQt60Zv/++yuc4xLu/zYJ
 z2rkypveSTfGrL7AqdoSsgmj1HbbO3uw9dlz17Mw96FuHsPZbOm28P+g3Y8FYjLR
 4EBkZ+qHoUGl0dUV4hQyezFiY5H3SXPhqEIpm2ImPpj5MfNHPd1w7PHlPzXVcrkC
 WxWSzpdPsNhOi3AWGT9lwLaLkMA4emt/24exHHVifKFnzOy/Ob4Dr/QnYaB8zfM=
 =Mkd/
 -END PGP SIGNATURE-

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Logger prefixes for api/apiserver are changing

[David Cheney]

I'll take thumper's temperature for this feature at the standup and
see what he thinks.

On Wed, Sep 3, 2014 at 12:30 AM, Eric Snow eric.s...@canonical.com wrote:
 On Tue, Sep 2, 2014 at 5:07 AM, David Cheney david.che...@canonical.com 
 wrote:
 I wonder if there is a better way to handle this default case of the
 name of the logger follows the package path), ie

 var logger = loggo.Logger() // or something, it could be a new
 method, logger.Default(), or something.

 which will inspect the call tree and determine its logger name automagically.

 +1

 -eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: State should not depend on the API server

[David Cheney]

The goal I have been tasked with is breaking the interdependency
between the state package and the _client_ api types in
state/api/params.

I don't have any opinion on adding additional layers assuming that
their dependancies flow downwards, ie

labix.org/mgo - juju/state - juju/apiserver

On Mon, Sep 1, 2014 at 4:03 PM, John Meinel j...@arbash-meinel.com wrote:
 FWIW I'd favor 3 layers, though it does mean you have to do copying between
 structs that would likely otherwise be almost identical. A State layer for
 saving in the DB, an API layer for communication, and a Model layer for use
 in the Agents/Clients.

 John
 =:-



 On Mon, Sep 1, 2014 at 9:40 AM, David Cheney david.che...@canonical.com
 wrote:

 Hello,

 This is an introductory email to explain my upcoming series of pull
 requests.

 The theme is simple: state must not depend on the api server.

 This is currently not true, state depends on api/params because some
 of the api types have leaked into the state package and are being
 directly stored in mongo.

 Please speak up if you disagree with this proposal.

 Dave

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev



-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

State should not depend on the API server

[David Cheney]

Hello,

This is an introductory email to explain my upcoming series of pull requests.

The theme is simple: state must not depend on the api server.

This is currently not true, state depends on api/params because some
of the api types have leaked into the state package and are being
directly stored in mongo.

Please speak up if you disagree with this proposal.

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: gccgo internal compiler errors

[David Cheney]

nah, we have a fix upstream, we just need to get that backported to
trusty then this becomes a non issue.

On Fri, Aug 29, 2014 at 7:44 PM, Matthew Williams
matthew.willi...@canonical.com wrote:
 As it's something we need to be doing for a while yet is there value in
 adding this as a task that gets run by the landing bot?

 Thanks

 Matty


 On Thu, Aug 28, 2014 at 11:48 PM, Tim Penhey tim.pen...@canonical.com
 wrote:

 Hi folks,

 I spent some time this morning looking at
 https://bugs.launchpad.net/juju-core/+bug/1362636

 A critical regression that was breaking CI on power.

 There is a bug in gccgo where we hit an internal compiler error when
 comparing an interface to a concrete type that implements the interface
 (as opposed to a pointer to the concrete type implementing the interface).

 This impacts some of the names.Tag rework that is going on.

 If you try to compare:
var tag names.Tag = names.NewMachineTag(1)

if names.NewUnitTag(1) == tag {
   // BOOM!!!
}

 This is entirely valid Go, and works fine with gc, but gccgo barfs
 horribly.

 My fix is here: https://github.com/juju/juju/pull/633

 This is just a warning.

 Remember folks that we need to support gccgo still (for at least another
 year until we have power and arm64 using gc).

 You can test locally by doing this:
   go test -compiler gccgo

 If you install the gccgo packages, which I don't remember, but hopefully
 someone will follow up with.

 Cheers,
 Tim

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev



 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: getting rid of all-machines.log

[David Cheney]

Hi Horatio,

I don't see a way to resolve the very disparate set of opinions you've
highlighted below. It's also not clear from your email who is
responsible for making a decision.

I suggest reframing the discussion as user stories. ie

* As a Juju user with a Windows workstation I want to use juju
debug-log (this should work today)
* As a Juju user who has deployed a mixed environment (to the best of
my knowledge there is no requirement for the state servers to run on
Windows, this seems contrary to Canonical's goal of Free Software)
containing a windows workload charm I want to view the logs from that
charm.

Dave

On Wed, Aug 27, 2014 at 5:35 AM, Horacio Duran
horacio.du...@canonical.com wrote:
 Hey, In an effort to move forward with juju's windows integration I have
 summarized what seems to be the core points of this discussion to the best
 of my ability (please excuse me if I missed or misunderstood something).
 The two core points of discussion on this thread are:
 * should we remove all-machines.log: which has been voted against, at least
 for the moment, since it is used for debug-log.
 * how do we support logging in windows: The strongest suggestions here are a
 syslog package by gabriel and logging into MongoDB by Gustavo.

 We do require some decision on the front of windows logging to have a
 complete windows support. Ideally we need senior citizens of juju dev
 community to weight into this in order to get a clear path to follow.

 Here is a summary I made to help myself while following this discussion:

 Nate original suggestion:
 * Remove all-machines.log: Claiming it takes a lot of space and it is not a
 multi platform solution

 Tim, John, Aaaron, etc:
 * all-machines.log is required for debug-log
 * makes it big and it would be nice to rotate it.

 Nate, gabriel:
 * keep all-machines.log
 * use a go-only solution (syslog package with ports from gabriel for
 windows)
 John
 * agrees.

 Nate, gabriel:
 * remove rsyslog from al OSes in favor of one solution that fits all OSes
 * Replace with go only solution.

 Dave:
 * Dont mind about the logs, make it just output and let external tools
 handle logging and rotation.
 * all-machines.log might be a bit bloated and it could contain less data
 that is more useful.
 (Here is the reference to 12factor that will later be attibuted to nate)
 Ian:
 * Agrees with dave, yet we should provide a rolling mechanism.

 Gabriel:
 * Windows does not support capturing stdout as a logging mechanism, it
 requires to explicitly log into the event log.
 * Thinks that using rsyslog to stream logs from agents to state server is
 too much overhead on external tools.
 * Proposes replacing external rsyslog with in app solution for the case of
 streaming logs.
 * Alternative solution, he does not recommend it, to create (and bundle with
 jujud.exe) a wrapper for windows only.

 Gustavo:
 * Present a possible alternative by using a MongoDB capped collection
 which will suit our use cases but does not recommend it because of the idea
 needs maturing on some details.

 Matt:
 * We should provide the option to log to stdout or syslog.

 Kapil:
 * Supports Gustavo's idea of logging in a structured form into Mongo as it
 makes sense to dump structured data with structure instead of serializing it
 to be de-serialized later.
 * We can send also messages to syslog and let OPS people collec them
 themselves.

 Gabriel (summarizing)
 * I will be looking into event log for local windows logging. This will
 probably require writing a package.
 * the syslog change will solve in the sort term, the aggregation issue from
 Windows nodes (when something better comes along, I will personally send a
 case of beer or ice-cream...or both, to whomever removes syslog as a
 dependency)
 * lumberjack works *now* for local logging on both Windows and Ubuntu. It
 simply removes 2 dependencies (for logging) with just a few lines of code...

 --
 Horacio


 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: getting rid of all-machines.log

[David Cheney]

On Wed, Aug 27, 2014 at 3:04 PM, Ian Booth ian.bo...@canonical.com wrote:


 On 27/08/14 14:56, John Meinel wrote:
 My personal vote would be:
 a) Use something that can write directly to multiple syslog receivers over
 a TLS encrypted connection from inside the jujud binary (e.g. don't use
 rsyslog to read the log files and forward them on to the state servers, but
 just write directly)

 I may be misremembering, but at the time that was the preferred approach. But
 then someone said Go's inbuilt syslog APIs were broke, so the compromise was 
 to
 use rsyslog forwarding.

 Does anyone else recall why it may have been said that Go's syslog APIs are 
 broken?

The reconnect logic is broken in all the version's of the syslog api.
The general consensus is that package is a mistake and should not be
used.


 I'd actually like to continue keeping a log file on local disk, as it can
 help diagnose when the problem is that you are failing to connect to
 somewhere else.

 Big +1 IMHO. I've always been a fan of logging to a text file readable in vi,
 emacs, whatever. Lowest common denominator is often the only option for
 debugging. I have no issue with logging elsewhere also, but can we please keep
 the log file as well.


 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: First customer pain point pull request - default-hook

[David Cheney]

Nope, still not receiving you clearly

On Mon, Aug 18, 2014 at 7:20 AM, Tim Penhey tim.pen...@canonical.com wrote:
 -BEGIN PGP MESSAGE-
 Charset: windows-1252
 Version: GnuPG v1

 hQIMA233D38ktbXXAQ/+OxwEOmqX/SS+lyJGh7WBYOZge3P8G9+bdu0u8v/sGioD
 xbPcpRAp4D3GaCyaMS231JTdvWjMgegw9f6AnnaeLiEdcmFi3YBEzvnDCR5+Lfpb
 KNMZFUsL9CYtsBy+tWBFkowsFwiKy66UiOQDRGW6G6V+5v/L2v4TUoDN+LE74/h2
 jbLuGCUX4od4UYyWDLfj5NPN0DlgYrt5kiv2NCC2QUmYqVrpuoFKl+dFkbt8WIti
 WXV6TjoFIbeRd83m3A4++f0jNlsUg9WWVnBIYVgomL/ndbBC8DDwWijAJfqh660P
 gUspAWx9/QxzoMXU/nUAfPoGpim0tRsd41FfrncgPGV1P15MBNcsLrR20QOhiwU5
 C/QO7M34VbdTySw5bK6xPX6SpVtIS7fNsqtJbZuhzpR1e9IaIERYPfKwZz20YGqa
 WBnzr80FUcFrq98BqcAGbQfgkHhMRBoS22mPCSrTkh8ki5lKkckc4UGrr3/JhJuD
 B2sTxfIQosbHbGBHhFn/gED/lGKeMMxA7lZdsMcJqgeou1/ggrbqc3vrqWsGs4mq
 9hKm2blZJEMGrhqpAEJ0cd5b6a76u5AKsr+F7G468XHWLrs1wMpyFiXURG9Hzp3N
 PmbzNomZmG8412QEEua8XDP3YT+3g54yakNr/vXisST44rbaZCCA0qhmj09jGf6F
 AQ4DdFn2rLtx3aMQA/9vSXW55V6QVufALE7cMXzYxhliBUnihnXRJRkjxUo54DY1
 HoeJwfHAoxQcvAlf5HXO/rfSe+CFKqZlP4kzgHXKHimXHCiukmqUIrEIiDCFQ2pm
 lvLJXccOya5F/wzPoaBzRWFmHoSZm790kvFM7Uy6Ko6V/5gRaKmCsdXOMBjmTAQA
 gjYVJqdifYIFRAdcH3IypC2lq1Ck1ndghv2r9T9QMlI6zZ1qJ6e/puwL6TKUxAqY
 FAqCj/rqm0qkirVPdGh0rPmfjV4PSnOLaTgbvGkul5wh3iJjddhZFyfaxxVTjRBY
 SFjKk5U3yAvym5L5XdaICWZDMOuZjKNj01VDeKTQKwXSwLABvWDovh0Sd9D/cf8a
 KHcW+CMHh89I1NyiW+6elJAU0HAnUtAHvlmoHzxLNsHtodQcdHX2sFU2UNYH+bj3
 6N4blppY0wFhIOQ5AzehdKef5fhAKQL+SZ+IMY5o5oZHxsMxCVg4K7jwhfjtA9UH
 Omf5mffBCF6dCTpg6nTIk/Fc92LNPxeX9ougfudFKMNCd9LHt2YMn9Q9CBgMEwxP
 nBAPcQhSCqtNg/QYNSWlBQS0M2d5oVvrG/YgzDWHsG3jcpKwS/S5R8QNP/U12U9W
 jKxvIV8F37ORzROO1NSeftWP0LrYagVPgZoFriXgEOP3cCEEILeKqNPtsGBvDFHO
 NgActMmYhMuND67rnuFjrqc410WEzIQ70wjw4BEitzLTvgw1fvViriXbqT8G8tfq
 wXTCRAj85GErd4BMG1Ve60PA7jLuf95XORftyLQQBIkaZksfcH2JXg4Dr1FaVQXV
 oiLWzuO5ezCh/Bhf4j7Zc1dhFg==
 =3J0G
 -END PGP MESSAGE-

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju trunk now is compatible with Mongo 2.6

[David Cheney]

Will CI failures using mongodb-2.6 be considered critical and block the build ?

On Fri, Aug 15, 2014 at 10:07 AM, Ian Booth ian.bo...@canonical.com wrote:
 A little while back we started work as a background task to port Juju to work
 with Mongo 2.6. After a final push from Andrew to get everything finalised, 
 the
 current Juju trunk now has Mongo 2.6 support finished/working.

 Note that Mongo 2.4.x (as shipped in cloud archive and/or trusty) is still the
 officially supported version for Juju.

 The reason for this work was due to the fact that Mongo 2.6 offers various 
 fixes
 and improvements over 2.4; some of these fixes are for issues Juju encounters
 but which will not be fixed for 2.4
 eg https://jira.mongodb.org/browse/SERVER-11807
 Also as time goes on, we need to have a viable pathway to move to Mongo 2.6 
 (and
 later versions) since that's where most of the development focus will be. Plus
 Mongo 2.6 is already in proposed for Utopic, so we now have the option to
 package juju-mongodb with a consistent version of Mongo as ships with the 
 distro.

 What we will do now is set up regular test runs using Mongo 2.6 on CI so we'll
 have data available about that combination's reliability etc.


 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: getting rid of all-machines.log

[David Cheney]

Ian asked me to post my thoughts here.

I am not in favour of applications rolling their own logs, I believe
that applications should not know anything about their log output,
they should just dump it all to stdout and another process should take
care of shuttling the data, logging it, culling it, whatever.

This is summarised here, http://12factor.net/logs

I think our current system with rsyslog is working fine and there is
no reason to remove it.

The problems with all-machines.log being to large is independent of
log rolling or any of these other arguments. We simply log too much.
There would be no request for log rolling it all-machines.log
contained only useful data.

Dave

On Sat, Aug 9, 2014 at 2:58 AM, Nate Finch nate.fi...@canonical.com wrote:
 So, rsyslog rotation works fine on Linux, but we can't do that on Windows.
 If we have to do something different for Windows, I'd rather just do one
 thing which is cross platform compatible for all our OSes, and not have to
 support a different configuration for each OS.  Doing it all in-application
 also insulates us from external dependencies... if some future or past
 Ubuntu series (or CentOS) has a different version of rsyslog, it could
 behave differently / require a different configuration, etc.



 On Fri, Aug 8, 2014 at 12:40 PM, David Britton david.brit...@canonical.com
 wrote:

 On Fri, Aug 08, 2014 at 12:03:21PM -0400, Nate Finch wrote:
 [...]
  remote syslog and to the local file log, we wouldn't need to worry about
  log rotation of the local log screwing up what gets sent to the remote

 Do the standard rsyslog log rotation mechanisms not function well?

 On Windows, what about the event log (which has remote
 viewing/aggregation capabilities built in)?

 --
 David Britton david.brit...@canonical.com



 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Label: Ready For Review

[David Cheney]

What if the review needs the author to rework ?

On Tue, Aug 12, 2014 at 7:55 AM, Nate Finch nate.fi...@canonical.com wrote:
 Merge it and it'll get closed and out of the list of open PRs.  I presume
 the submitter is paying enough attention to merge their own stuff.

 On Aug 11, 2014 5:44 PM, David Cheney david.che...@canonical.com wrote:

 How can we remove the label once the review has been done ?

 On Tue, Aug 12, 2014 at 4:33 AM, Nate Finch nate.fi...@canonical.com
 wrote:
  I made a label on github.com/juju/juju (and coincidentally
  github.com/juju/utils) called Ready For Review. The reason for the label
  is
  that it is often difficult to figure out what branches are actually
  ready to
  be reviewed and which ones are really WIP and therefore aren't waiting
  to be
  reviewed.  It's simple to filter by labels to see what's assigned to
  Ready
  For Review, so the on-call reviewers (or anyone else) can find stuff to
  review.
 
  I did this because some people had mentioned to me that they had
  branches
  that were waiting for reviews, but no one was reviewing them.  Pinging
  people who are online works, but it's hard to ping people who aren't
  online so I figured this was easier and gives everyone somewhere to
  go
  to find what PR's are languishing.
 
  I know we have the WIP: prefix for branches that aren't ready to be
  generally reviewed but that's opt-out, which means it's easy to
  forget
  to put that on your branch and have people think it's ready for review
  when
  it's not  which means people tend to err on the side of just not
  reviewing stuff.  The Ready For Review label is opt-in, so there's no
  doubt
  that the submitter thinks it's ready.
 
  It currently requires someone on this list to add the label (at least
  for
  github.com/juju/juju), which is somewhat unfortunate, but it's really
  only
  needed if you think your code won't get reviewed otherwise... and maybe
  just
  asking someone to add that label will encourage them to review your
  code.
 
  -Nate
 
  --
  Juju-dev mailing list
  Juju-dev@lists.ubuntu.com
  Modify settings or unsubscribe at:
  https://lists.ubuntu.com/mailman/listinfo/juju-dev
 

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Port ranges - restricting opening and closing ranges

[David Cheney]

Yes, absolutely.

On Tue, Aug 5, 2014 at 8:33 PM, Domas Monkus domas.mon...@canonical.com wrote:
 A follow-up question: should closing a port that was not opened previous to
 that result in an error?

 Domas


 On Fri, Jun 27, 2014 at 2:13 PM, Matthew Williams
 matthew.willi...@canonical.com wrote:

 +1 on an opened-ports hook tool, I've added it to the task list


 On Fri, Jun 27, 2014 at 9:41 AM, William Reade
 william.re...@canonical.com wrote:

 Agreed. Note, though, that we'll want to give charms a way to know what
 ports they have already opened: I think this is a case where
 look-before-you-leap maybe beats easier-ask-forgiveness-than-permission (and
 the consequent requirement that error messages be parsed...). An
 opened-ports hook tool should do the trick.


 On Thu, Jun 26, 2014 at 9:18 PM, Gustavo Niemeyer gust...@niemeyer.net
 wrote:

 +1 to Mark's point. Handling exact matches is much easier, and does
 not prevent a fancier feature later, if there's ever the need.

 On Thu, Jun 26, 2014 at 3:38 PM, Mark Ramm-Christensen (Canonical.com)
 mark.ramm-christen...@canonical.com wrote:
  My belief is that as long as the error messages are clear, and it is
  easy to
  close 8000-9000 and then open 8000-8499 and 8600-9000, we are fine.
  Of
  course it is nicer if we can do that automatically for you, but I
  don't
  see why we can't add that later, and I think there is a value in
  keeping a
  port-range as an atomic data-object either way.
 
  --Mark Ramm
 
 
  On Thu, Jun 26, 2014 at 2:11 PM, Domas Monkus
  domas.mon...@canonical.com
  wrote:
 
  Hi,
  me and Matthew Williams are working on support for port ranges in
  juju.
  There is one question that the networking model document does not
  answer
  explicitly and the simplicity (or complexity) of the implementation
  depends
  greatly on that.
 
  Should we only allow units to close exactly the same port ranges that
  they
  have opened? That is, if a unit opens the port range [8000-9000], can
  it
  later close ports [8500-8600], effectively splitting the previously
  opened
  port range in half?
 
  Domas
 
  --
  Juju-dev mailing list
  Juju-dev@lists.ubuntu.com
  Modify settings or unsubscribe at:
  https://lists.ubuntu.com/mailman/listinfo/juju-dev
 
 
 
  --
  Juju-dev mailing list
  Juju-dev@lists.ubuntu.com
  Modify settings or unsubscribe at:
  https://lists.ubuntu.com/mailman/listinfo/juju-dev
 



 --

 gustavo @ http://niemeyer.net

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev



 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev



 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev



 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: avoiding unnecessarily entering upgrade mode

[David Cheney]

SGTM.

On Wed, Aug 6, 2014 at 11:10 AM, Menno Smits menno.sm...@canonical.com wrote:
 Right now, a Juju machine agent is in upgrade mode from the moment it
 starts until the upgrade-steps worker is finished. During this period API
 logins are heavily restricted and most of the agent's workers don't start
 until upgrade mode stops.

 This happens even when there is no upgrade to perform. The upgrade-steps
 worker always runs at machine agent startup and upgrade mode is in force
 until it finishes.

 Upgrade mode is typically short-lived (say 10 seconds) but if something is
 wrong (e.g. mongo issues) the upgrade-steps worker may take longer or not
 finish resulting in the user seeing lots of upgrade in progress messages
 from the client and in the logs.
 This is particularly confusing when a user hasn't even requested an upgrade
 themselves.

 I would like to change the machine agent so that upgrade mode is only
 entered if the version in agent.conf is different from the running software
 version. This would mean that upgrade mode is only entered if there is an
 actual upgrade to perform.

 The version in agent.conf is only updated after a successful upgrade so it
 is the right thing to use to determine if upgrade mode should be entered.

 The current behaviour means that the (idempotent) upgrade steps for the
 current version are always run each time the machine agent starts. If the
 change I'm proposing is implemented this will no longer happen. Does this
 seem like a problem to anyone? For instance, do we rely on the upgrade steps
 for the current version being run after bootstrap?

 The ticket for this work is at: https://bugs.launchpad.net/bugs/1350111

 Cheers,
 Menno



 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: apiserver/testing.FakeAuthoriser is not a good mock

[David Cheney]

On Thu, Jul 31, 2014 at 5:12 PM, William Reade
william.re...@canonical.com wrote:
 On Thu, Jul 31, 2014 at 7:30 AM, David Cheney david.che...@canonical.com
 wrote:

 Proposal: remove Authoriser.GetAuthEntity() and replace all calls with
 Authoriser.GetAuthTag(). I have a branch for this, it's  30 lines as
 there are only 3 places in the apiserver where we do this and they
 usually call the .Tag method on the entity they get back from
 GetAuthEntity.


 SGTM.

 2. This extends from point 1, while the svrRoot derives everything
 from svrRoot.entity, the FakeAuthoriser allows the caller to pass a
 unique value for the tag and the entity of the authorisee. This leads
 to impossible situations where the FakeAuthorizer returns nil for
 AuthGetEntity but a non nil value for AuthGetTag. Other permutations
 exist in our tests and are expected by the test logic.

 Propsal: Once step 1 is fixed the difference between svrRoot and
 FakeAuthoriser is the former starts from a state.Entity and derives a
 tag from which other values are derived, the latter skips the initial
 step and starts from a tag. This work falls out of the solution to
 steps 1 and 3.

 3. The helper methods, AuthMachineAgent(), AuthUnitAgent() on the
 Authoriser interface are implemented differently in svrRoot and
 FakeAuthoriser. In tests it is quite common to take the FakeAuthoriser
 from the test suite, copy it and change some of these values leading
 to impossible situations, ie, the tag or entity of the FakeAuthoriser
 pointing to a Unit, but AuthMachineAgent() returning true.

 Proposal: The simplest solution is to copy the implementation of these
 helper methods from svrRoot to FakeAuthoriser. A more involved
 solution would be to factor these methods out to be functions in the
 apiserver/common package that take an Authorizer. This second step may
 not pay for itself.


 I would strongly favour the latter solution. The benefit of having a fake
 authorizer whose behaviour diverges is that (at least in theory) it allows
 for comprehensive testing against arbitrary authorizer behaviour;
 constraining that behaviour so we're only testing realistic situations will
 be great, but I fear that doing so by copy-pasting code will only lead to
 more divergences in future. Let's make sure we're using the same logic
 across the board.

You got it boss.


 Cheers
 William

 These steps resolves the majority of the discontinuity between the two
 implementations and will resolve a set of blocking issues I've hit
 converting the apiserver and state packages to speak tags natively.

 Thanks for your time

 Dave

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev



-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

CI tests are installing Juju on the CI machine ...

[David Cheney]

Creating config file /etc/mercurial/hgrc.d/hgext.rc with new version
Setting up zip (3.0-4) ...
Setting up juju-core (1.20.1-0ubuntu1~12.04.1~juju1) ...
update-alternatives: using /usr/lib/juju-1.20.1/bin/juju to provide
/usr/bin/juju (juju) in auto mode.
Setting up rsyslog-gnutls (5.8.6-1ubuntu8.6) ...

^ it's leaking in from a dependency, this is risky to say the least.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Possible landing bot issues

[David Cheney]

Hello,

I think there are two problems with the current landing bot.

1. the bot doesn't appear to NACK changes that are not gofmt'd, this
means changes land which others cannot fix because their .gitconfig
prevents them.

2. the bot doesn't appear to be enforcing it's lockdown rules as the
comment it places on the PR contains the magic string that the bot
looks for when it see's another $$ instruction.

Thank you for your time.

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Possible landing bot issues

[David Cheney]

I managed to sneak one in an hour ago, is the fix live ?

On Thu, Jul 31, 2014 at 10:18 AM, Ian Booth ian.bo...@canonical.com wrote:

 I think there are two problems with the current landing bot.


 Thanks for raising the issues.


 1. the bot doesn't appear to NACK changes that are not gofmt'd, this
 means changes land which others cannot fix because their .gitconfig
 prevents them.


 Still on the todo list, sorry. Will be fixed asap.

 2. the bot doesn't appear to be enforcing it's lockdown rules as the
 comment it places on the PR contains the magic string that the bot
 looks for when it see's another $$ instruction.


 Curtis has already fixed this plus landed some other imporvements (see other
 emails).

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

apiserver/testing.FakeAuthoriser is not a good mock

[David Cheney]

Hello,

This message serves as background to a series of changes coming soon.

The goal of this message, and the changes that follow, is to describe
the problems that I see with the FakeAuthoriser and a describe in
general terms the steps I propose to address those shortcomings.

Background:

The FakeAuthoriser implements the apiserver/common.Authoriser
interface and is used heavily in the apiserver tests. The other
implementation of this interface is the apiserver itself,
specifically, apiserver.srvRoot.

tf. The goal of the FakeAuthoriser is to imitate apiserver.svrRoot.

I assert that this is currently not true and that the tests are
structured to expect the behaviour of the FakeAuthorizer rather than
the contract of apiserver/common.Authoriser as defined by
apiserver.srvRoot.

Issues:

This section describes the ways that the FakeAuthoriser diverges from
the behaviour of the srvRoot authoriser.

1. In srvRoot, all authorisation stems ultimately from the
state.Entity object referenced from the srvRoot instance. This is a
problem as our authorisation system is based on the _tag_ passed over
the api, not the state.Entity, although for practical purposes the tag
of the authenticated entity and the entity itself are interchangeable.

Proposal: remove Authoriser.GetAuthEntity() and replace all calls with
Authoriser.GetAuthTag(). I have a branch for this, it's  30 lines as
there are only 3 places in the apiserver where we do this and they
usually call the .Tag method on the entity they get back from
GetAuthEntity.

2. This extends from point 1, while the svrRoot derives everything
from svrRoot.entity, the FakeAuthoriser allows the caller to pass a
unique value for the tag and the entity of the authorisee. This leads
to impossible situations where the FakeAuthorizer returns nil for
AuthGetEntity but a non nil value for AuthGetTag. Other permutations
exist in our tests and are expected by the test logic.

Propsal: Once step 1 is fixed the difference between svrRoot and
FakeAuthoriser is the former starts from a state.Entity and derives a
tag from which other values are derived, the latter skips the initial
step and starts from a tag. This work falls out of the solution to
steps 1 and 3.

3. The helper methods, AuthMachineAgent(), AuthUnitAgent() on the
Authoriser interface are implemented differently in svrRoot and
FakeAuthoriser. In tests it is quite common to take the FakeAuthoriser
from the test suite, copy it and change some of these values leading
to impossible situations, ie, the tag or entity of the FakeAuthoriser
pointing to a Unit, but AuthMachineAgent() returning true.

Proposal: The simplest solution is to copy the implementation of these
helper methods from svrRoot to FakeAuthoriser. A more involved
solution would be to factor these methods out to be functions in the
apiserver/common package that take an Authorizer. This second step may
not pay for itself.

These steps resolves the majority of the discontinuity between the two
implementations and will resolve a set of blocking issues I've hit
converting the apiserver and state packages to speak tags natively.

Thanks for your time

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Conditional compilation primer

[David Cheney]

Hello,

Tim asked me to circulate this to aide those who have to review code
that deals with conditional compilation.

http://dave.cheney.net/2013/10/12/how-to-use-conditional-compilation-with-the-go-build-tool

Cheers

Dave

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Can we get rid of the hash(password) dance?

[David Cheney]

Yes, please!

On Thu, Jul 17, 2014 at 2:49 PM, John Meinel j...@arbash-meinel.com wrote:
 Michael is working on changing how we handle sessions with Mongo, and
 noticed that his first attempt started running into Auth failures.
 It turned out that this was because of the hash(password) dance. (For those
 who don't know, in certain circumstances we used to seed the password for
 the database with the hash(password) and then once we had a secure channel
 we would replace it with the real password.)

 I believe all of our production bootstrap code has gotten rid of the
 password dance, because we now just use cloud-init to bring up a machine and
 then SSH into that machine to finish provisioning. Thus all the information
 is already over the secure SSH channel, instead of the insecure cloud-init
 user data.

 From what I can tell poking around the code base, the only place that still
 uses the hash(password) is actually in the Dummy provider.

 I feel like we're at a point where we can safely remove that from the Dummy
 provider, and also remove the fallback code in our 'connect to the database'
 code. (If we leave it in, then I think after changing the password just
 reconnecting to the database is fine, because it should happen infrequently.

 Thoughts?

 John
 =:-


 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Poor performance on Juju local with lxc containers.

[David Cheney]

Excellent news. I'm glad you got to the bottom of it.

Do you know why php5-xdebug was installed the lxc container, but not
your vagrant vm ?

On Wed, Jul 9, 2014 at 5:24 PM, Sebastian sebas5...@gmail.com wrote:
 ha!! gotcha!! finally I know why, the php5-xdebug was installed and enabled
 by default, didn't know was such a big impact of performance.

 So this is going to the charm's configuration, so can be an option.

 Happy Sebas now.

 2014-07-09 2:40 GMT-03:00 Sebastian sebas5...@gmail.com:

 Hey there!

 After I did the Drupal charm, part of my plan was that the developers now
 can use the juju-local with lxc containers to deploy every project
 environment using bundles.

 But! we notice a BIG difference of performance in the containers, so we
 run a PHP benchmarking script:

 Host (vbox):
 vagrant@vagrant-ubuntu-trusty-64:~$ php bench.php
 --
 test_math : 3.190 sec.
 test_stringmanipulation   : 2.859 sec.
 test_loops: 1.702 sec.
 test_ifelse   : 1.319 sec.
 --
 Total time:   : 9.07 sec.

 Lxc container (into the vbox):
 ubuntu@vagrant-local-machine-4:~$ php bench.php
 --
 test_math : 37.062 sec.
 test_stringmanipulation   : 30.826 sec.
 test_loops: 5.749 sec.
 test_ifelse   : 6.149 sec.
 --
 Total time:   : 79.786 sec.


 As you can see, the containers created by Juju took 70 seconds more than
 the host.
 The versions of the php where the same, and this was also tested in bare
 metal, to ensure that the virtualization of the host wasn't the issue.

 We understand how this impact the implementation of Juju as a complete
 solution for devs and ops. Is not everyone that can buy a super computer to
 virtualize each charm, in his local notebook. So that's why I'm here,
 telling our story.

 Maybe this is an issue that belongs to the LXC group, but I thought
 interesting to share this.

 If someone can do the same benchmarkings and share the results, or any
 help to make things better, please!! it's more than welcome.

 Cheers!,
 Sebas.



 --
 Juju mailing list
 Juju@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju


-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Landing bot job changed

[David Cheney]

Additionally, the bot is no longer reporting back to the PR that the
build failed to merge.

On Mon, Jul 7, 2014 at 11:11 AM, David Cheney
david.che...@canonical.com wrote:
 The lander is broken

 + exit 0
 All passed, sending merge
 /tmp/hudson6791049747065303228.sh: line 44:
 /home/ubuntu/jenkins-github-lander/bin/lander-merge-result: No such
 file or directory
 /tmp/hudson6791049747065303228.sh: line 44:
 /home/ubuntu/jenkins-github-lander/bin/lander-merge-result: No such
 file or directory
 Build step 'Execute shell' marked build as failure
 Description set: davecheney 128-update-ssh-terminal

 On Sat, Jul 5, 2014 at 9:56 PM, Martin Packman
 martin.pack...@canonical.com wrote:
 I have switched the jenkins landing bot to a new job, as part of the
 work on getting it faster and more reliable. If you want to check back
 on old results, they are preserved for now under a different job name:

 http://juju-ci.vapour.ws:8080/job/github-merge-juju-old/

 I'm actually having second thoughts in writing this, maybe just
 renaming the job to keep the links in existing prs correct would have
 been better. I can still do that if anyone has strong opinions.

 Martin

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Splitting out state/api into its own repo

[David Cheney]

On Fri, Jun 27, 2014 at 4:21 PM, John Meinel j...@arbash-meinel.com wrote:
 Just my quick thought, I think moving it out from state/api into just a
 top level api would  be reasonable and a lot less clumsy than trying to
 pull it out into an entirely separate repository.

+1

I don't think the api package is useful outside Juju (at this time)
and splitting it into another repo just doubles the amount of work.


 I'm not sure if Gustavo realized that state/apiserver is the HTTP(-ish,
 its really just JSON rpc over a websocket) interface to state. state/api
 is all go client code (agents or user client code). And it would at least be
 really nice for all those things to have an import test that says does not
 depend on anything under state.)

 It certainly would make it clearer that nothing that is using the api should
 import something from, say, apiserver, or state directly.

 John
 =:-


 On Thu, Jun 26, 2014 at 9:23 PM, roger peppe rogpe...@gmail.com wrote:

 I have a slightly different proposal, inspired by the recent
 Go proposal for internal imports (http://golang.org/s/go14internal),
 which currently looks like it will actually be implemented.

 We move all public facing APIs into top level packages within
 the juju repo, and move everything else under
 github.com/juju/juju/internal

 Packages I would suggest should be exported:

 github.com/juju/juju/api// moved from state/api
 github.com/juju/juju/cmd/juju
 github.com/juju/juju/cmd/jujud
 ... (all the other commands, but not the cmd package itself)
 github.com/juju/
 github.com/juju/juju/cmd/internal/jujucmd// containing
 supercommand.go from current cmd
 github.com/juju/juju/juju  // but only the API-related pieces

 Some packages would want moving, because they're user-facing
 but currently inside packages that we would probably not want to
 export:

 github.com/juju/environs/configstore

 is the only one I can think of right now.

 Others would want splitting. For example, the environs package
 is a mix between user-facing and internal stuff right now.
 It would be great to take out all the user-facing config file stuff
 (that might sit
 well inside the juju package).

 Still others would benefit from being made available externally.
 I think the rpc package is probably one of those that would
 sit well inside its own repo.

 Some packages could benefit from having their own internal
 directory - the uniter is one of those, for example. The
 apiserver too has many sub-packages that should not really
 be visible to the rest of juju.

 In the end, we should end up with an API that it might actually be
 feasible to stabilise. I'd like juju to move under gopkg.in at some
 point, providing useful stability guarantees for external users
 that might want to build Go programs based on our code.

   cheers,
 rog.

 On 26 June 2014 17:41, Eric Snow eric.s...@canonical.com wrote:
  (I've put a more structured proposal below, but here's some context.)
 
  Over the last couple weeks I've been spinning up on the juju code
  base, which is large enough to dissipate any hope of understanding it
  all quickly. wink Most of what I've focused on is relative to the
  juju tools and the remote API, both for the sake of backup/restore.
 
  One thing that threw me off is that it has not been obvious that the
  code under `state/api` is the public-facing API for juju's state (as
  someone recently explained to me).  For a while I thought `state/api`
  held the state API client code, but from what I understand now it
  actually contains all the public facing code for the juju state API
  (and [consequently?] for juju as a whole?).  In some regard I would
  expect a public API to be sit in a top-level package (rather than
  nested down like it is).
 
  A couple of other things got in the way.  For one, we basically don't
  have any documentation for the API.  I expect that it will mirror the
  documentation for the juju subcommands/tools pretty closely, but
  regardless the doc doesn't exist.  Setting up godoc for the api
  package would be great and so would a new page in the juju
  documentation.  I realize there has been some discussion on this point
  of late, from which I expect a doc will take shape in the short term.
  In the meantime, we've actually been telling people to wrap calls to
  the CLI tools rather than using the API.  The documentation side of
  things is a somewhat orthogonal, though topically related, issue.
 
  For another, while there has been a pretty good effort to keep the
  `api` package relatively un-entwined from the rest of juju, there were
  a few times when I found it hard to follow what was going on.  This
  was particularly true of the underlying state RPC implementation,
  though at this point things make a lot more sense.  Having a separate
  repo would help delineate the boundary between the API and juju
  itself, which should in turn help make the API code easier to follow.
 
  So...
 

Re: End Of Review marker

[David Cheney]

Rietveld also supports git

On Thu, Jun 12, 2014 at 8:46 PM, Ian Booth ian.bo...@canonical.com wrote:
 It's also the same when you are responding to review comments. You want to 
 mark
 them all as Done (or whatever) and have those go out in a batch to let the
 reviewer know they can come back and +1.

 Surely we're not the only people annoyed by this? I wonder what more 
 experienced
 github users do. Or maybe people know that github sucks for code reviews and 
 use
 gerrit or something else?

 On 12/06/14 20:38, Horacio Duran wrote:
 Hey, I don't know if this bugs everyone or just me but it happens very
 often that I am working while people are reviewing my code on gh. While
 people is reviewing and commenting on the code I keep getting mails and the
 diff page from the pr keeps changing. To know when its all done and I can
 finally try to answer/fix all the comments I usually wait until my phone
 stops ringing madly with mails but I think that we could do better. At the
 end of the diff page there is a comment box where you can add comments
 (where you usually add your $$merge$$ or LGTM) We could add something
 there, like Done just to let the author know we are done with the review
 and not just reading a big confusing chunk of code.
 What do you people think?




 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: moving bzr wip branch to git

[David Cheney]

Nice

On Wed, Jun 4, 2014 at 2:08 PM, Jesse Meek jesse.m...@canonical.com wrote:
 So the command that worked for me (thanks for the help Tim) was:

 ~/go/src/github.com/juju/juju$ (cd ~/go/src/launchpad.net/juju-core  bzr
 diff --color=never -r ancestor::parent) | patch  -p0 --merge

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju is on github

[David Cheney]

Make it so

On Thu, Jun 5, 2014 at 6:59 AM, Menno Smits menno.sm...@canonical.com wrote:
 On 5 June 2014 06:58, Jorge Niedbalski jorge.niedbal...@canonical.com
 wrote:


 2cents: Why not rename the CONTRIBUTING file to CONTRIBUTING.md
 so it will be rendered as a markdown file by github?


 That seems sensible. Any objections?



 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev