Re: Machine agents uninstall themselves upon worker.ErrTerminateAgent.

[Eric Snow]

On Mon, May 9, 2016 at 10:50 AM, Eric Snow <eric.s...@canonical.com> wrote:
> I'll check it with 1.25 too, though I expect that the result will be the same.

Under 1.25, using the local provider, I was able to reproduce the
behavior.  On the  and even got the same confusing log entry:

  2016-05-09 16:48:51 DEBUG juju.cmd.jujud machine.go:1741 uninstall
file "/var/lib/juju/uninstall-agent" does not exist

So it's not readily apparent what is deleting the files and removing
the init services.  However, it *is* happening.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Machine agents uninstall themselves upon worker.ErrTerminateAgent.

[Eric Snow]

On Mon, May 9, 2016 at 1:56 AM, Andrew Wilkins
 wrote:
> But... I've just looked at the 1.25 branch again, and the fix *was* made
> there. And from Jorge's comment
> https://bugs.launchpad.net/juju-core/+bug/1514874/comments/4, we can see
> that the uninstall logic isn't actually running (see `uninstall file
> "/var/lib/juju/uninstall-agent" does not exist`
> https://github.com/juju/juju/blob/1.25/cmd/jujud/agent/machine.go#L1741)
>
> I'm not sure what to make of that. Eric, have you confirmed that that code
> is what's causing the issue? Are we sure we're not barking up the wrong
> tree?

Those logs certainly don't line up with the reported bug. :/  It's
possible that they aren't the right logs.  Regardless, my initial
investigation was based on inspecting code that "uninstalls" the
agent.  The only such code I found was machineAgent.uninstallAgent()
(and in the manual provder's Destroy()).

To be sure about it, I verified the issue using the LXD provider under
2.0, following the steps Jorge specified in the bug:

1) Modify the apipassword on the /var/lib/juju/agents/machine-8/agent.conf
2) Restart jujud-machine-8

The agent is uninstalled and the the uninstall file is there:

  2016-05-09 16:21:48 INFO juju.agent uninstall.go:47 agent already
marked ready for uninstall

However, the machine is still there:

  $ juju status
  ...
  0  started 10.235.227.113
juju-af8e08f2-3953-4bcb-84de-767a15f46b4f-machine-0 xenial

I'll check it with 1.25 too, though I expect that the result will be the same.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Machine agents uninstall themselves upon worker.ErrTerminateAgent.

[Eric Snow]

On Fri, May 6, 2016 at 11:37 AM, William Reade
<william.re...@canonical.com> wrote:
> On Fri, May 6, 2016 at 5:50 PM, Eric Snow <eric.s...@canonical.com> wrote:
>>
>> See https://bugs.launchpad.net/juju-core/+bug/1514874.
>
>
> Mainly, we just shouldn't do it. The only *actual reason* for us to do this
> is to support the manual provider; any other machine that happens to be dead
> will be cleaned up by the responsible provisioner in good time. There's a
> file we write to enable the suicidal behaviour when we enlist a manual
> machine, and we *shouldn't* have ever written it for any other reason.

So how about, other than the container bits, we drop the "uninstall"
code entirely.  To address the manual provider case, during that
provider's StartInstance() we add a "clean-up-juju" script somewhere
on the machine?  Then folks can use that to confidently clean up after
the fact.

> So, narrowly, fixing this involves relocating the more-widely-useful
> (in-container loop device detach IIRC?) code inside MachineAgent.uninstall;

Yep.

> I don't think we want any of this -- it's all a consequence of
> inappropriately triggering the uninstall stuff in the first place.

Yeah, if we simply did no automatic cleanup and (on manual provider)
give users a script that would let them clean up afterward, then they
could happily(?) manually repair their machines with much less
trouble.

>> * Could this lead to collisions if the instance is re-purposed as a
>> different machine?  I suppose it could also expose sensitive data when
>> likewise re-purposed, since it won't necessarily be in the same model
>> or controller.  However, given the need for admin access that probably
>> isn't a likely problem.
>
> If substrates are leaking data between separate instances, the substrate is
> screwed up, and there's not much we can do about it. The manual provider has
> no such guarantees, and that's the only justification for trying to clean up
> so drastically in the first place.

Fair enough.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Machine agents uninstall themselves upon worker.ErrTerminateAgent.

[Eric Snow]

See https://bugs.launchpad.net/juju-core/+bug/1514874.

When starting, the machine agent starts up several workers in a runner
(more or less the same in 1.25 and 2.0).  Then it waits for the runner
to stop.  If one of the workers has a "fatal" error then the runner
stops and the machine agent handles the error.  If the error is
worker.ErrTerminateAgent then the machine agent uninstalls (cleans up)
itself.  This amounts to deleting its data dir and uninstalling its
jujud and mongo init services.  However, all other files are left in
place* and the machine/instance is not removed from Juju (agent stays
"lost").

Notably, the unit agent handles fatal errors a bit differently, with
some retry logic (e.g. for API connections) and never cleans up after
itself.

The problem here is that a fatal error *may* be recoverable with
manual intervention or with retries.  Cleaning up like we do makes it
harder to manually recover.  Such errors are theoretically extremely
uncommon so we don't worry about doing anything other than stop and
"uninstall".  However, as seen with the above-referenced bug report,
bugs can lead to fatal errors happening with enough frequency that the
agent's clean-up becomes a pain point.

The history of this behavior isn't particularly clear so I'd first
like to hear what the original rationale was for cleaning up the agent
when "terminating" it.  Then I'd like to know if perhaps that
rationale has changed.  Finally, I'd like us to consider alternatives
that allow for better recoverability.

Regarding that third part, we have a number of options.  I introduced
several in the above-referenced bug report and will expand on them
here:

1. do nothing
This would be easy :) but does not help with the pain point.
2. be smarter about retrying (e.g. retry connecting to API) when
running into fatal errors
This would probably be good to do but the effort might not pay for itself.
3. do not clean up (data dir, init service, or either)
Leaving the init service installed really isn't an option because
we don't want
the init service to try restarting the agent over and over.
Leaving the data dir
in place isn't good because it will conflict with any new agent
dir the controller
tries to put on the instance in the future.
4. add a DO_NOT_UNINSTALL or DO_NOT_CLEAN_UP flag (e.g. in the
machine/model config or as a sentinel file on instances) and do not
clean up if it is set to true (default?)
This would provide a reasonable quick fix for the above bug, even if
temporary and even if it defaults to false.
5. disable (instead of uninstall) the init services and move the data
dir to some obvious but out-of-the-way place (e.g.
/home/ubuntu/agent-data-dir-XXMODELUUIDXX-machine-X) instead of
deleting it
This is a reasonable longer term solution as the concerns
described for 3 are
addressed and manual intervention becomes more feasible.
6. same as 5 but also provide an "enable-juju" (or "revive-juju",
"repair-juju", etc.) command *on the machine* that would re-enable the
init services and restore (or rebuild) the agent's data dir
This would make it even easier to manually recover.
7. first try to automatically recover (from machine or controller)
This would require a sizable effort for a problem that shouldn't
normally happen.
8. do what the unit agent does
I haven't looked closely enough to see if this is a good fit.

I'd consider 4 with a default of false to be an acceptable quick fix.
Additionally, I'll advocate for 6 (or 5) as the most appropriate
solution in support of manual recovery from "fatal" errors.

-eric


* Could this lead to collisions if the instance is re-purposed as a
different machine?  I suppose it could also expose sensitive data when
likewise re-purposed, since it won't necessarily be in the same model
or controller.  However, given the need for admin access that probably
isn't a likely problem.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Static Analysis tests

[Eric Snow]

On Wed, Apr 27, 2016 at 9:14 PM, Nate Finch  wrote:
> So, I don't really think the method of testing should determine where a test
> lives or how it is run.  I could test the exact same things with a more
> common unit test - check the tls config we use when dialing the API is using
> tls 1.2, that it only uses these specific ciphersuites, etc.  In fact, we
> have some unit tests that do just that, to verify that SSL is disabled.
> However, then we'd need to remember to write those same tests for every
> place we make a tls.Config.

The distinction here is between testing the code (execution) and
verifying that the code base follows proscribed constraints, whether
technical or organizational.  Perhaps you could call the latter
"meta-testing". :)  For testing the code, it is really useful to have
the tests next to the code.  For meta-tests, e.g. static-analysis
against the code base as a whole, the relationship with any particular
package is tenuous at best.

So I agree with the others that "tests" which focus on the code-base
as a whole, via static analysis or otherwise, should be grouped
together in the directory tree.  There's prior art with the
featuretests package, though it's not quite the same thing.

I also agree that Lingo would be an effective tool in cases like this
where we are enforcing a code-base-wide policy, e.g. "don't import
testing packages in non-test files" or "use the helper function when
creating a tls.Config".  However, that does not mean we can't wrap
calls to Lingo in test methods (more on this below).

>
> The thing I like about having this as part of the unit tests is that it's
> zero friction.  They already gate landings.  We can write them and run them
> them just like we write and run go tests 1000 times a day.  They're not
> special.  There's no other commands I need to remember to run, scripts I
> need to remember to set up.  It's go test, end of story.

I agree that is useful, particularly if we can isolate them, e.g. with
"go test -short" or the testing tags that I proposed.  I don't think
the objection is necessarily about running these "meta" tests via "go
test".  Personally I kind of like that approach.  It wouldn't be hard
to write test suites with methods which call out to the tools.  That
would thereby allow us to meet all our testing needs through "go test"
while allowing us to continue using the other tools independently to
meet additional specific needs.

For me, the objection is more about *where* such tests live.

>
> The comment about Lingo is valid, though I think we have room for both in
> our processes.  Lingo, in my mind, is more appropriate at review-time, which
> allows us to write lingo rules that may not have 100% confidence.  They can
> be strong suggestions rather than gating rules.  The type of test I wrote
> should be a gating rule - there are no false positives.

Again, having granular testing tags would allow us to make that
distinction in a way that the test tooling could easily distinguish.

>
> To give a little more context, I wrote the test as a suite, where you can
> add tests to hook into the code parsing, so we can trivially add more tests
> that use the full parsed code, while only incurring the 16.5 second parsing
> hit once for the entire suite.  That doesn't really affect this discussion
> at all, but I figured people might appreciate that this could be extended
> for more than my one specific test.

Fair enough.

>  I certainly wouldn't advocate people
> writing new 17 seconds tests all over the place.

Oh good! :)

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: auto-upgrading models

[Eric Snow]

On Mon, Apr 25, 2016 at 5:24 PM, Menno Smits  wrote:
> A per-model flag which indicates that the model should automatically upgrade
> when the controller does might be nice too (this what #7 means I think?).

I meant a CLI flag for "juju upgrade-juju".  If I understood right,
you are suggesting a per-model config option.  I like that.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: auto-upgrading models

[Eric Snow]

On Mon, Apr 25, 2016 at 6:38 PM, Andrew Wilkins
 wrote:
> Whatever we do, #2 should never be optional.

Agreed.

>
> I would like us to have all of #2, #3, #4, #7.

This is my opinion too.  Menno suggested another option (call it #8):
a model config setting equivalent to the flag suggested by #7.  I
support that one too.

> For #3, we could say "upgrade
> each model ... or run `juju upgrade-juju --all-models
> `".

+1

>
> I don't think this should be restricted to "--upload-tools", but rather just
> upgrading the admin model in general.

Agreed.  I was focused on --upload-tools because that's where I had
actually seen a problem with the upgrade story.  This is partly
because you can only upload tools to the admin model now.  However, I
agree that this should not be limited to --upload-tools.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

auto-upgrading models

[Eric Snow]

In a recent bug I was working on the issue of auto-upgrading models
came up.  I also ran into this personally not too long ago.
Currently, running "juju upgrade-juju -m admin --upload-tools"[1]
upgrades the admin model to a new version.  To set the version of any
other model to the uploaded one, you must do so separately afterward,
e.g. "juju upgrade-juju -m default 2.0-rc1.3". [2]

The fact that you must upgrade the non-admin model separately is
something new with multi-model support.  Perhaps it is only something
that will throw off folks that have relied on --upload-tools
previously and perhaps it is something that we'll just adjust to.
However, I wanted to bring the matter up here and identify potential
courses of action (not all mutually exclusive):

1. do nothing (rely on users to know to always upgrade juju per-model)
2. clearly point this out in the documentation
3. add a note in the output of "juju upgrade-juju --upload-tools"
reminding admins to manually upgrade each model
4. make the "juju is out-of-date" warnings also show up for models
relative to the controller
5. auto-upgrade models when the controller is upgraded
6. auto-upgrade but have a flag to not auto-upgrade
7. have a flag to auto-upgrade

FWIW, I don't consider #1 or #5 to be acceptable options.  I'm on the
fence about #6; it aligns with what I expect would be a better default
experience but hesitate to make unrequested changes of that sort by
default.  So #7 might be more appropriate if the consequences of #6
would be too risky.  Regardless, I do think the user experience of
upgrade-juju could be improved.

Thoughts?

-eric


[1] You can no longer upload tools to any other model than admin.
[2] Thankfully, due to some recent work by axw the new version is
*available* to all models.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Move provider implementations under their related projects.

[Eric Snow]

Perhaps we should move the implementation of some providers over to
the repos for the projects with which the providers interact (and then
just import them in providers/all).  Then those providers would be
more likely to stay up-to-date in the face of changes in the project,
particularly backward-incompatible ones.  For example:

* provider/maas -> github.com/juju/gomaasapi/maasprovider
* provider/lxd -> github.com/lxc/lxd/lxdprovider
* ...

or something like that.  It's not a perfect solution nor a complete
one, but it should help.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Planning for Juju 2.2 (16.10 timeframe)

[Eric Snow]

On Fri, Mar 18, 2016 at 8:57 AM, Tom Barber  wrote:
> c) upload files with actions. Currently for some things I need to pass in
> some files then trigger an action on the unit upon that file. It would be
> good to say path=/tmp/myfile.xyz and have the action upload that to a place
> you define.

Have you taken a look at resources in the upcoming 2.0?  You define
resources in your charm metadata and use "juju attach" to upload them
to the controller (e.g. "juju attach my-service/0
my-resource=/tmp/myfile.xyz"). *  Then charms can use the
"resource-get" hook command to download the resource file from the
controller.  "resource-get" returns the path where the downloaded file
was saved.

-eric


* You will also upload the resources to the charm store for charm store charms.

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Planning for Juju 2.2 (16.10 timeframe)

[Eric Snow]

On Fri, Mar 18, 2016 at 8:57 AM, Tom Barber  wrote:
> c) upload files with actions. Currently for some things I need to pass in
> some files then trigger an action on the unit upon that file. It would be
> good to say path=/tmp/myfile.xyz and have the action upload that to a place
> you define.

Have you taken a look at resources in the upcoming 2.0?  You define
resources in your charm metadata and use "juju attach" to upload them
to the controller (e.g. "juju attach my-service/0
my-resource=/tmp/myfile.xyz"). *  Then charms can use the
"resource-get" hook command to download the resource file from the
controller.  "resource-get" returns the path where the downloaded file
was saved.

-eric


* You will also upload the resources to the charm store for charm store charms.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: LXD support (maybe)

[Eric Snow]

I also left  a review.  Mostly LGTM.  I just wanted a bit of feedback
before giving it a ship-it. :)

-eric

On Thu, Feb 25, 2016 at 8:11 AM, Andrew McDermott
 wrote:
> I took a quick look through it as I was OCR - comments in RB.
>
> On 25 February 2016 at 15:00, John Meinel  wrote:
>>
>> So I have a patch up http://reviews.vapour.ws/r/3973
>>
>> Which with the help from Tycho does look like it allows you to "juju
>> bootstrap lxd lxd" again, but there are a fair number of caveats.
>>
>> The big one is that it is only compatible with lxd-2.0.0~beta4 (because
>> the API has changed since beta3 and very different from 0.21/0.26). And
>> currently 0.26 is the version that is in Trusty-backports. (I do see
>> 2.0.0~beta4 in Xenial main, though.)
>>
>> For people that want to try it out, you can use ppa:ubuntu-lxc/lxd-stable
>> as it has 2.0.0~beta4 in it. I'm not sure if CI will let it end up in master
>> or the next Juju release because of not having 2.0.0~beta4 easily available
>> in Trusty.
>>
>> I'll try to respond to any reviews or feedback that people give, though
>> I'm technically off for the weekend. I know its one of the blockers for the
>> next release, though, so I'll try to help.
>>
>> John
>> =:->
>>
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>
>
>
>
> --
> Andrew McDermott 
> Juju Core Sapphire team 
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju devel 1.26-alpha2 is available for testing

[Eric Snow]

On Fri, Nov 27, 2015 at 9:00 AM, Marco Ceppi  wrote:
> On Fri, Nov 27, 2015 at 9:37 AM Aaron Bentley 
> wrote:
>>  Requirements
>>
>> - Running Wily (LXD is installed by default)
>>
>
> For the LXD provider, I have the latest LXD installed on trusty, will that
> work or is it hard-coded to wily+ ?

It will work fine if using a juju built with Go 1.3+ (e.g. the juju
package on wily or building locally and using --upload-tools).  The
provider uses the LXD Go bindings which require Go 1.3+.  It is a
compile-time issue rather than a run-time check.

Until trusty ships with Go 1.3+ we cannot ship a Juju that depends on
the LXD Go bindings, thus the LXD provider is disabled if Juju is
compiled with anything older than Go 1.3.  As Aaron said, there's a
separate thread discussing how to solve the problem of getting the
latest full-featured Juju on trusty.

-eric

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju devel 1.26-alpha2 is available for testing

[Eric Snow]

On Fri, Nov 27, 2015 at 9:00 AM, Marco Ceppi  wrote:
> On Fri, Nov 27, 2015 at 9:37 AM Aaron Bentley 
> wrote:
>>  Requirements
>>
>> - Running Wily (LXD is installed by default)
>>
>
> For the LXD provider, I have the latest LXD installed on trusty, will that
> work or is it hard-coded to wily+ ?

It will work fine if using a juju built with Go 1.3+ (e.g. the juju
package on wily or building locally and using --upload-tools).  The
provider uses the LXD Go bindings which require Go 1.3+.  It is a
compile-time issue rather than a run-time check.

Until trusty ships with Go 1.3+ we cannot ship a Juju that depends on
the LXD Go bindings, thus the LXD provider is disabled if Juju is
compiled with anything older than Go 1.3.  As Aaron said, there's a
separate thread discussing how to solve the problem of getting the
latest full-featured Juju on trusty.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: trouble with dependent repos

[Eric Snow]

On Mon, Nov 23, 2015 at 9:32 PM, Nate Finch  wrote:
> Last week, I had trouble landing code in gopkg.in/juju/charm.v6-unstable,

I had a similar experience last week with backward-incompatible
changes in the utils repo. :(

> and using that code from master of github.com/juju/juju.  I was hoping
> someone could help me figure out a way to avoid problems I've encountered.
>
> [snip]
>
> Anyone have ideas on how can we prevent incompatible changes landing in a
> dependent repo without the corresponding fixes landing in juju soon
> thereafter?

I'd recommend avoiding backward-incompatible changes if at all
possible.  Often the temptation is to "fix" some signature while
you're working on related code.  Sometimes you really need a different
signature.  Either way, almost every time you can instead introduce a
new function with the new signature and deprecate the old one.  Using
that approach you do not break backward compatibility and thus do not
cause the problem for dependent repos that you've described.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: lxd provider in master

[Eric Snow]

On Sun, Nov 22, 2015 at 8:50 PM, Tim Penhey  wrote:
> Hi all,
>
> I noticed that the lxd provider was now in master and attempted to use
> it to help debug an HA problem.
>
> In environments.yaml file I added a very simple clause:
>
>lxd:
>   type: lxd

This is what I've been using.  It works fine.

>
> But when I try to bootstrap, I get the following:
>
> $ juju bootstrap --debug
> 2015-11-23 03:26:14 INFO juju.cmd supercommand.go:58 running juju
> [1.26-alpha2 gc]
> 2015-11-23 03:26:14 DEBUG juju.environs.config config.go:1473 coercion
> failed attributes: map[string]interface {}{"namespace":"tim-lxd",
> "container":"lxd"}, checker:

This helps identify the problem.  The environment you tried to
bootstrap has "container: lxd" and there is no "lxd" container type.
Perhaps it's an old environment, set up back when the LXD provider was
first prototyped (with its own container type)?  It does not appear to
be the simple "lxd" environment you showed above.

Also, FYI, "namespace" is still around but is not necessary.  It is
useful when you want to have multiple LXD providers running on your
local LXD.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: use --upload-tools when bootstrapping the LXD provider

[Eric Snow]

On Fri, Nov 6, 2015 at 8:41 AM, Aaron Bentley
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> That may be true, but the LXD provider branch has a stale version
> number.  Once the version number is updated to 1.26-alpha2, I imagine
> the lxd provider will no longer try to use the released streams.
>
> The stale version number is also causing builds to fail:
> http://reports.vapour.ws/releases/3272
> http://reports.vapour.ws/releases/3277

Ah.  Thanks.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Hardware version on the OVA file in the vmware provider

[Eric Snow]

@Sergey, do you know how this works out?

-eric

On Thu, Oct 1, 2015 at 2:06 PM, Trond Viggo Håpnes
 wrote:
> Hi all
>
> The documentation for the vmware provider [1] states that "you will need to
> have an existing vSphere
> installation, which supports VMWare's Hardware Version 8 or better", but it
> looks like the OVA file
> is version 10?
>
> Is there a reason for this? This excludes all ESX version older than 5.5.
>
> And is there a workaround to get the vmware provider working on ESX 5.1?
>
> [1] https://jujucharms.com/docs/devel/config-vmware
>
> --
> Trond Viggo Håpnes 
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

link to review request in launchpad

[Eric Snow]

All,

After posting a review request (i.e. a PR), please be sure to add a
link to the review request/PR (as a comment) to the lp issue the patch
addresses.  Otherwise it's a pain trying to track down which commits
actually relate to the issue.

Similarly, be sure to include a link to the related issue in the PR description.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Reminder: on June 15th, Go 1.4 will stop compiling things from code.google.com.

[Eric Snow]

On Tue, Jun 2, 2015 at 5:57 AM, Katherine Cox-Buday
katherine.cox-bu...@canonical.com wrote:
 I think there was an effort to remove this from our code, but I thought I'd
 send out a reminder to check and for personal projects. May the source be
 with you...

code.google.com/p/winsvc is the only one left and there's a patch up
for review that switches that one over.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: possible approach for test tags

[Eric Snow]

On Wed, May 13, 2015 at 9:59 PM, Andrew Wilkins
andrew.wilk...@canonical.com wrote:
 Main thing that concerns me is that it's all opt-in.

Same here.

 I guess it doesn't
 matter
 too much, as long as CI continues to run all the tests. There's nothing
 stopping
 people from skipping running the tests and proposing junk today.

Right.

 A couple of things I'd like to see added to the proposal:
  - the ability to have negated tags, e.g. -tags=!long or -tags=^long or
 whatever.

I already have that (- instead of !), just not well documented. :)

  - a meta all tag, or command line flag to set all the tags (e.g. for CI)

Great idea.  I added base, but that's something else.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

possible approach for test tags

[Eric Snow]

We've had discussions here a couple of times (since I joined the team)
about classifying tests in our suite so they could be run more
flexibly.  We've also already added explicit handling for a specific
kind of test along those same lines: featuretests.  Additionally you
can think of the CI tests as another test classification.  So we're
already taking the approach in a limited fashion.

I'd like to see something more generic that we could use in the juju
test suite.  I've put up a proof of concept patch that demonstrates
what I think would fit the bill:

  http://reviews.vapour.ws/r/1647/

If the approach seems reasonable then we could start using it for new
tests and as appropriate when touching existing code.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Running Juju in Google compute engine

[Eric Snow]

We have info in the release notes [1], but looks like we have a gap in
the docs (I'll follow up on that).  Also, it looks like the release
notes did not get updated with the auth-file config option that
Andrew mentioned.

Feel free to ping me if Andrew's and Curtis' comments don't take care
of it for you.

-eric

[1] https://launchpad.net/juju-core/1.23/1.23.0#release-notes

On Fri, May 1, 2015 at 6:26 AM, Jorge O. Castro jo...@ubuntu.com wrote:
 I was able to find the project name, client email, and client id in
 the GCE UI, but I wasn't able to find where I'm supposed to find the
 private key, can someone help me out?

 --
 Jorge Castro
 Canonical Ltd.
 http://juju.ubuntu.com/ - Automate your Cloud Infrastructure

 --
 Juju mailing list
 Juju@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: backporting changes to packages that use gopkg.in

[Eric Snow]

On Thu, Mar 26, 2015 at 7:53 PM, Nate Finch nate.fi...@canonical.com wrote:
 tl;dr: godeps overrides gopkg.in, so you can have godeps pin a commit from a
 different branch than gopkg.in is retrieving (i.e. make a release-number
 branch, like 1.22 and use godeps to pin commits from there, even if go get
  gopkg.in is getting a different branch).

So basically, gopkg.in is just a means for a project to explicitly
control, in a sane way, the git revision that `go get` uses.  If your
project does *not* use something like gopkg.in then either tip of the
master branch must always be correct (messy) or folks that import your
code must manually manage the revision (as we do with godeps).

There is a lot less magic going on with gopkg.in than you might think.
In the context of `go get`, gopkg.in just proxies the github URL to
the corresponding named revision (branch or tag).  It does not cache a
copy of the branch or repo or otherwise restrict the revisions
available to the git fetch that `go get` does.  So non-ancestor
revisions still get downloaded.  That's why could still take the
godeps/1.22-branch approach without a lot of complexity.

In the context of juju core, using gopkg.in for dependencies doesn't
buy us as much since we use godeps.  It's still fine though since the
import URL indicates the intended target and it doesn't impede our
ability to adjust via godeps (as I originally thought it did).

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Migrating imports away from code.google.com/p to golang.org/x

[Eric Snow]

On Fri, Mar 27, 2015 at 9:59 AM, Nate Finch nate.fi...@canonical.com wrote:
 Just an FYI:

 As you've probably heard, code.google.com is going away.  It'll be read-only
 for a year IIRC and then poof.

 Most of the packages from the go authors lived there (crypto, etc), but now
 will be maintained at golang.org/x.  We'll be changing our imports to point
 at the new repos, so our code doesn't break when google code goes away.  The
 only outlier is the GCE wrapper, which now lives at github.com/google.

I'm working up a patch right now for juju core (master-only).  Some of
our other dependencies will need to be updated to use the new
locations:

gopkg.in/macaroon-bakery.v0
code.google.com/p/go.net - golang.org/x/net
bitbucket.org/kardianos/service
code.google.com/p/winsvc - ???

Also, the GCE provider relies on code.google.com/p/goauth2 which is
deprecated and the replacement is not compatible, so I'll post a
separate patch to address that for 1.23 and master.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: vCloud Director Support

[Eric Snow]

On Thu, Mar 26, 2015 at 11:07 AM, Vahric Muhtaryan vah...@doruk.net.tr wrote:
 Hello ,

 I saw that google cloud engine is added as an supported provider , still
 there is no any date about integration of vmware ?

I'm not sure of specific release plans, Alexis can fill you in more
specifically.  Work on a vmware (vSphere) provider has been
progressing well for a while now and I expect it will be finished in
time for one of juju's next few bi-monthly releases.

-eric

 Also do you know jelastic ? Any implementation request arrived for jealstic
 ?

 Regards
 VM

 From: Alexis Bruemmer alexis.bruem...@canonical.com
 Date: Thursday 18 December 2014 19:55
 To: Kapil Thangavelu kapil.thangav...@canonical.com
 Cc: Vahric Muhtaryan vah...@doruk.net.tr, juju juju@lists.ubuntu.com
 Subject: Re: vCloud Director Support

 Actually we are currently working on a plan for VMWare provider support in
 juju, but this is very recent and we have no confirmed dates or detail
 deliveries at this stage.

 Vahric, I would be happy to chat with you further about your use case and
 see how our current plans align.

 --Alexis

 On Thu, Dec 18, 2014 at 9:45 AM, Kapil Thangavelu
 kapil.thangav...@canonical.com wrote:



 On Wed, Dec 17, 2014 at 7:23 AM, Vahric Muhtaryan vah...@doruk.net.tr
 wrote:

 Hello All

 MAAS and JUJU is looks like very good product , I would like to ask as an
 infrastructure will you add vCloud Director , we are vmwre vcloud air
 network partner and I m thinking to integrate the juju with our platform ,
 and dev or product manage in this list ?



 There are dev and product managers on this list. a vmware provider is a
 substantial development and maintenance effort. Its definitely of interest,
 but its not currently in scope on the core roadmap. Contributions for it
 would be welcome. Vmware has been working on go api bindings which will make
 that work easier https://github.com/vmware/govcloudair and
 https://github.com/vmware/govmomi

 cheers,

 Kapil

 --
 Juju mailing list
 Juju@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju



 --
 Alexis Bruemmer
 Juju Core Manager, Canonical Ltd.
 (503) 686-5018
 alexis.bruem...@canonical.com

 --
 Juju mailing list
 Juju@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju


-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: restoring juju state

[Eric Snow]

On Tue, Mar 24, 2015 at 9:25 PM, Tim Penhey tim.pen...@canonical.com wrote:
 I'm seriously not in favour of calling it recover, and not just
 because it has a particular meaning in Go.

I'm glad you said so then. :)


 Perhaps rebuild or recreate, but recover feels wrong.

Got it.  I'm not married to recover.  It was just the first
reasonable (to me) alternative that popped into my head.

rebuild and recreate could work, though they don't seem quite
right either.  Perhaps something more like replace or reset?  So
far it's all re* names, so maybe wipe? :)  The catch is that it will
be a subcommand of juju backups so it also needs to be something
that implies applying a backup rather than affecting a backup.  Hmm.
apply?

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: GCE provider

[Eric Snow]

On Wed, Mar 25, 2015 at 1:04 AM, Andrew Wilkins
andrew.wilk...@canonical.com wrote:
 Hey folks,

 I just signed up for a GCE trial, and tested out the provider on master.
 Found a couple of issues.

 I've filed https://bugs.launchpad.net/juju-core/+bug/1436191. I suspect the
 same issue would exist on 1.23, but haven't tested. I haven't set the
 status/importance, because I don't know what the importance is (there's no
 CI or docs?), and I'd like for someone to confirm the bug.

I'll take a look.


 Second, a UX thing. It was mostly clear how to generate keys, but it wasn't
 super clear what format to put them into environments.yaml. Obviously
 this'll be documented, but I think it'd be nice if in environments.yaml we
 could just specify the location of the JSON file that GCE spits out, and
 have Juju extract the info it needs, like we do with certs and SSH keys.

That's a good idea.  I'll check it out.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: restoring juju state

[Eric Snow]

On Wed, Mar 25, 2015 at 4:14 AM, roger peppe roger.pe...@canonical.com wrote:
 Ooh, a proper bikeshed!

:)


 If this command is still doing the same thing it was when
 I first wrote the original hack version, then it's more like
 restore-state-servers-and-hope-that-nothing-much-changed-since-we-dumped.

Pretty much.


 I'm guessing that Eric's issue with restore is that the word implies that
 it restores the entire environment, which it does not.

That's actually the problem I have with calling the whole thing
backups when it's really
state-DB-backups-plus-a-couple-other-key-state-artifacts.  But that is
a separate issue. *  :)

 But recover
 has a similar issue.

The problem I have with the name restore is that it implies we are
going to fix up the state server (and possibly the environment) to
match some earlier backup.  Instead, we replace it with a new one and
don't do anything about fixing the replicaset (if any existed).
Furthermore, as you indicated, the machines in the restored state
might not line up properly with the current set of instances in the
environment.  Restore doesn't clean up that situation either.


 Perhaps dump-state and restore-state might convey more
 accurately what's being done here?

You're probably right, though restore in the name still carries the
same implications, in my mind though.

-eric

*  I almost think the juju backups super command would be better
named juju state.  backups would then be just a part of that.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: GCE provider

[Eric Snow]

On Wed, Mar 25, 2015 at 8:00 PM, Dimiter Naydenov
dimiter.nayde...@canonical.com wrote:
 One thing to bear in mind. It seems the europe-west1-a AZ is
 considered deprecated and will be shut down soon (end of this month
 I believe).
 It will be nice to skip this AZ when trying to place instances.

Would you mind opening a bug on that?  Thanks.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

restoring juju state

[Eric Snow]

juju 1.23 adds the new restore, all written in Go and integrated into
core.  This is a great thing and the result of a lot of effort by
Horacio.  In discussions with him about it, there are two things that
came up that I wanted to bring up here.

First, the name restore is misleading.  It gives the impression of
capability that it does not actually provide.  I propose that we
rename it to recover in 1.23 and forward.

Second, I am now under the impression that after restore is complete,
the environment is left with instances on the provider that juju
should be cleaning up but isn't.  In the case of a single state
server, the old state server instance is left dead but still active.
In the case of HA, in addition to the state server that handles the
restore API request, the other state servers are also left in that
same situation.

So for a 3 server replicaset you are left with 3 instances that juju
is not using but that are still active on the provider side.  Users
may not realize they need to manually remove those instances to avoid
further costs (depending on the provider, I suppose).  Please explain
if I've misunderstood this or if juju has some other mechanism by
which such instances are cleaned up.  If I haven't missed something
then I think we need to fix restore to clean up those state machines.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: restoring juju state

[Eric Snow]

I've opened https://bugs.launchpad.net/juju-core/+bug/1435413 for
renaming restore to recover.

-eric

On Mon, Mar 23, 2015 at 10:00 AM, Horacio Duran
horacio.du...@canonical.com wrote:
 Nope this is not very accurate I believe that you got some parts wrong out
 of our conversation. I am currently on vacation but upon return I'll submit
 a writeup with some graphic flows explaining how this all works (the recover
 part I agree with, in any case what is being recovered/restored is the state
 server so that is what we should be trying to convey with the name)

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Testing on windows

[Eric Snow]

That would we awesome!  I was thinking along the same lines.

-eric

On Thu, Mar 19, 2015 at 8:51 AM, Gabriel Samfira
gsamf...@cloudbasesolutions.com wrote:
 That is actually a great idea :). We can try and do this in Nuremberg next 
 month ;). We have some tooling to do an unattended install. We can use that 
 to create the base qcow2 image, and install dependencies.

 for testing we can simply use the base image as a qcow2 backing file, and use 
 WinRM to run commands inside it from a simple python script.

 We can implement this during the sprint :D.
 
 From: Eric Snow [eric.s...@canonical.com]
 Sent: Thursday, March 19, 2015 4:33 PM
 To: Gabriel Samfira
 Cc: juju-dev@lists.ubuntu.com
 Subject: Re: Testing on windows

 Thanks for bringing this up.  And thanks to Bogdan (and anyone else
 involved) for writing up that great walkthrough!  I had expected it to
 be more complicated.  I'm going to be running the unit tests on
 windows from now on. :)  I'm sure I've broken then a bunch in the last
 couple weeks.

 What would be nice is a script that does the following:

 * start up a KVM with windows
 * pull a specified branch from a github juju repo (and run godeps)
 * run the test suite (and show me the results)
 * shut down the KVM

 Bonus points for optionally setting up a fresh KVM each time and for
 optionally running other provided commands (e.g. to test bootstrap,
 deploy).

 If I get a chance I'll write something up.  (I have so much free time. wink)

 -eric

 On Wed, Mar 18, 2015 at 7:40 PM, Gabriel Samfira
 gsamf...@cloudbasesolutions.com wrote:
 Hello folks,

 It has come to my attention that there may be some confusion in regards
 to some Windows testing. There have been a couple of branches that have
 merged which break windows tests in juju-core.

 I would like to remind everyone that there is a guide available at:

 http://wiki.cloudbase.it/juju-testing

 that will help you set up a testing environment on Windows. Also, if
 there are any questions regarding Windows weirdness, please feel free to
 contact me on irc (gsamfira) or bogdanteleaga. We will be more then
 happy to help you navigate any Windows issues you might have.


 Kind regards,
 Gabriel
 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Testing on windows

[Eric Snow]

Thanks for bringing this up.  And thanks to Bogdan (and anyone else
involved) for writing up that great walkthrough!  I had expected it to
be more complicated.  I'm going to be running the unit tests on
windows from now on. :)  I'm sure I've broken then a bunch in the last
couple weeks.

What would be nice is a script that does the following:

* start up a KVM with windows
* pull a specified branch from a github juju repo (and run godeps)
* run the test suite (and show me the results)
* shut down the KVM

Bonus points for optionally setting up a fresh KVM each time and for
optionally running other provided commands (e.g. to test bootstrap,
deploy).

If I get a chance I'll write something up.  (I have so much free time. wink)

-eric

On Wed, Mar 18, 2015 at 7:40 PM, Gabriel Samfira
gsamf...@cloudbasesolutions.com wrote:
 Hello folks,

 It has come to my attention that there may be some confusion in regards
 to some Windows testing. There have been a couple of branches that have
 merged which break windows tests in juju-core.

 I would like to remind everyone that there is a guide available at:

 http://wiki.cloudbase.it/juju-testing

 that will help you set up a testing environment on Windows. Also, if
 there are any questions regarding Windows weirdness, please feel free to
 contact me on irc (gsamfira) or bogdanteleaga. We will be more then
 happy to help you navigate any Windows issues you might have.


 Kind regards,
 Gabriel
 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

using the reviewboard dashboard effectively

[Eric Snow]

I realized today that some folks may not be aware of some of the
helpful columns in the reviewboard dashboard.  You can see the full
list of available columns by clicking on the pencil icon on the right.
There are a handful of columns that I've found really helpful to me as
a reviewer.  Most notably:

* Ship It! (the green check mark) - indicates how many ship-its a
review request has. It also shows how many open issues there are.
This helps me know what point the request is at in the review process.
* Reviews - shows how many reviews the request has received.  This
allows me to quickly know if a request hasn't gotten enough attention.
* New Updates (the ... bubble) - shows whether or not there has been
any activity on the review request since you last looked at it. That
way you can quickly see when you may need to follow up on a review you
gave.
* My Comments (the green boxy thing) - indicates whether or not you
have commented on a review request.  This allows me to visually filter
the requests on which I should follow up.

FYI, here's the list of columns I have in my dashboard, ordered the
way I like it:

ID
Ship It!
Reviews
New Updates
My Comments
Diff Size
Summary
Submitter
Repository
Last Updated (this is my sort column)

Hope this helps all the reviewers be more productive. :)

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Feedback on a base fake type in the testing repo

[Eric Snow]

On Wed, Feb 11, 2015 at 6:43 PM, Andrew Wilkins
andrew.wilk...@canonical.com wrote:
 Looks okay in principal, though I'm not sure how much benefit it would
 add to the existing, tailored fakes in the codebase. It seems a little bit
 weird that error results are encapsulated, but not non-error results.

Yeah, I've tried to make that work several different ways to no avail.
It's messy since return types vary greatly and a mechanism to
accommodate them would require extra boilerplate in faked methods to
accommodate type conversions.  So it didn't seem worth bothering to
me.*  Error returns are a different story since there is only one type
to deal with (error).

-eric

* I suppose I could use reflection to make it all work out, but I'd
rather avoid the complexity (at least in this initial patch).

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A cautionary tale of names

[Eric Snow]

On Sun, Jan 11, 2015 at 8:17 PM, Tim Penhey tim.pen...@canonical.com wrote:
 This is because the EC2 provider tags the machines with the environment
 name, and not environment UUID.  I think we should change this ASAP.

Tim, I'm glad you brought this up.  We have followed the lead of the
EC2 provider in this regard, so we'll need to fix this.

Perhaps other providers need fixing too?

On Mon, Jan 12, 2015 at 8:43 AM, Gustavo Niemeyer gust...@niemeyer.net wrote:
 A few quick notes:

 - Having an understandable name in a resource useful

Agreed.  For GCE we may have both the env name and UUID.


 - Being a tag means it's possible to have a name *and* a UUID

What do you mean by this?  Do tags incorporate the env UUID somehow?

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Things which may be of interest in Go v1.4

[Eric Snow]

Thanks for writing this up, Katherine!

On Thu, Dec 11, 2014 at 9:21 AM, Katherine Cox-Buday
katherine.cox-bu...@canonical.com wrote:
 go generate
 https://golang.org/doc/go1.4#gogenerate
 This is *very* powerful and could reduce the number of copy/paste snippets,
 or unsafe reflection code we have to write. For those of you who are
 familiar with the lisps, this is essentially macros. You write code that
 examines Go expressions and does something. This code is triggered by
 //go:generate command arg comments in the code. As a quick example, our
 StringSet type could be generated and even expanded to encompass any type.

Cool.  So generics via macros?

 https://golang.org/doc/go1.4#internalpackages
 Just what it sounds like. We can now hide implementation details from
 importers. Probably more useful for the ancillary Juju packages as Juju Core
 is probably not imported very much.

Nice.  This would be good for our various testing sub-packages, among
other things.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: juju vs. Puppet/Chef/Salt/Ansible/etc.

[Eric Snow]

On Wed, Dec 10, 2014 at 8:23 AM, Kapil Thangavelu
kapil.thangav...@canonical.com wrote:
 On Wed, Dec 10, 2014 at 10:16 AM, Eric Snow eric.s...@canonical.com wrote:
 It would be great if we could leverage what you've done to be
 more visible to customers.  Having similar support for Puppet, Chef,
 and Salt would be awesome too.  It's just a matter of justifying the
 resources to get it done...

 michael's done most of the legwork on both ansible and saltstack in
 charmhelpers [0], although i'm not aware of any public charms that use the
 saltstack support. For chef, the rails charm is a good example [1].

Sweet.  I'd missed that.

-eric

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

juju vs. Puppet/Chef/Salt/Ansible/etc.

[Eric Snow]

The reaction I get most often from folks that aren't familiar with
juju and skim through the juju site is that it looks like a competitor
to the various configuration management tools out there like Puppet or
Salt.  However, my experience is that while they have some overlap,
they sit at different layers.

Have I grown out of touch?  Conceivably those projects have or are
working on juju-like functionality that I'm not aware of.  If not (or
even if so), what's the best way to educate people on what juju is and
how it will help them when they're already steeped in the lower-layer
config. management world?

Related to that, how can we help those same folks wrap all their
existing recipes, etc. in charms?  It's got to be easy enough that
they can justify the effort.

-eric

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Too space, or not two space

[Eric Snow]

On Tue, Nov 18, 2014 at 5:13 PM, David Cheney
david.che...@canonical.com wrote:
 To quote Butterick's
 http://practicaltypography.com/one-space-between-sentences.html

And to quote that same page:

If you have to use a typewriter-style font, you can use two spaces
after sentences. Otherwise, don’t.

I use a mono-space font.

Furthermore, I disagree with the statement In the second paragraph,
the extra spaces disrupt the balance of white space.  I find that
having each sentence be visually distinct to help with readability.
Apparently I'm not the only one.  From what I can tell, studies on the
matter are inconclusive. [1]

Regardless, I don't think it matters all that much, particularly since
(as Roger pointed out) godoc automatically converts to single-space
for you.  Like Nate, I think there are more significant things to
worry about, though perhaps not as controversial. :)

It seems like the overall consensus is to use single space.  If that
means it becomes a rule for our code then naturally I'll go along.
Otherwise I'd rather not bother.  To be honest, this all seems like a
waste of time.

-eric


[1] 
http://en.wikipedia.org/wiki/Sentence_spacing#Effects_on_readability_and_legibility

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard update

[Eric Snow]

On Mon, Nov 17, 2014 at 5:10 AM, Dimiter Naydenov
dimiter.nayde...@canonical.com wrote:
 I can confirm RB diffs for backports to 1.21 get generated correctly
 now, and the PR description is updated to include a link to the RB diff.

Thanks for reporting on this.


 There's one issue however -- the link in the PR is
 https://reviews.vapour.ws/r/id, rather than http://..;, which
 cause some problems:

I've switched it to HTTP.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

reviewboard update

[Eric Snow]

FYI, I was able to solve 3 reviewboard-github integration issues today:

1. pull requests for branches other than master now work (e.g. 1.21 backports)
2. no more hitting rate limits (5000 requests/hour limit instead of 60)
3. pull request bodies now get updated with a link to the new review request

If you have any trouble with any of these please let me know.  Thanks!

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Please stop adding dependencies to the state package

[Eric Snow]

On Thu, Nov 13, 2014 at 4:05 PM, David Cheney
david.che...@canonical.com wrote:
 Hello,

 Please stop adding dependencies to the state package.

 The dependency tree for state must go

 mgo - state - things that depend on state

 At the moment state depends on things like backups, multiwatchers,
 presence, toolstorage.

 This is wrong.

 Eric, i'm not picking on you here, it's just that b sorts first, and
 this is the one I saw today. There are many other examples, including
 the multiwatcher which also break this model

 If you add functionality to state, then the code has to go ***INSIDE**
 state, or be written to take a *state.State.

 Please help me out by not adding more dependencies to state.

Thanks for pointing this out, Dave.  I'll fix this for backups.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: review board not syncing?

[Eric Snow]

On Tue, Nov 11, 2014 at 12:28 PM, Jesse Meek jesse.m...@canonical.com wrote:
 The latest three reviews on GitHub (#1103,#1102,#1101) I cannot see in
 Review Board. Do we have a loose wire?

For 1101 at least, it's in reviewboard (402).

I'll look into it.  I'm pretty sure the github integration struggles
with large patches (perhaps due to GH API throttling).

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: deprecating the juju-backup (and juju-restore) CLI plugin

[Eric Snow]

On Fri, Oct 31, 2014 at 8:35 AM, Curtis Hovey-Canonical
cur...@canonical.com wrote:
 On Thu, Oct 30, 2014 at 2:10 PM, Eric Snow eric.s...@canonical.com wrote:
 What would be the best approach for deprecating the old backup plugin?
  I was going to simply gut it (it's a bash script) and stick in a
 deprecation warning message and a call to juju backups create
 --download.  However, jam rightly pointed out to me that this may be
 a sticky situation (as described above).  I'd really like to find a
 way to eliminate the old backup implementation.  Thoughts?

 Look before you leap. Call juju status and check that the version is
 1.21.+ and use new command, else old command.

Good idea.

 We expect new clients to
 manage old state-servers during transitions. The transition my be long
 if the enterprise is not upgrading their envs. I think we are stuck
 with the old implementation for as long as we support trusty.

Yeah, that's what I figured but hoped wasn't the case.


 Users do not know that backup is a bash script...
 juju backup
 is the command, so we can write an alternate plugin that checks the
 state-server version, then either calls to new command or the old
 script which is renamed, If we rename/change juju-backup.bash, we need
 a simultaneous packaging change.

Since it is transparent to users and otherwise mostly irrelevant, I'll
leave the filename alone and add the conditional operation to the
existing script.

Am I okay with adding a deprecation warning message (to stderr) that
instructs juju backup users to use juju backups create instead?
I'm not sure what precedent there is for doing so with our CLI.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

deprecating the juju-backup (and juju-restore) CLI plugin

[Eric Snow]

The new juju backups create can be used instead of juju backup (a
CLI plugin).  So you would think we could deprecate and later remove
the old plugin. Unfortunately, juju backups create won't work with
1.20 or earlier API servers.  So it's not quite as simple as I'd hoped
as long as newer clients will be connecting to older servers.

What would be the best approach for deprecating the old backup plugin?
 I was going to simply gut it (it's a bash script) and stick in a
deprecation warning message and a call to juju backups create
--download.  However, jam rightly pointed out to me that this may be
a sticky situation (as described above).  I'd really like to find a
way to eliminate the old backup implementation.  Thoughts?

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Menno's reviewer graduation

[Eric Snow]

On Wed, Oct 29, 2014 at 7:13 PM, Tim Penhey tim.pen...@canonical.com wrote:
 Hey there fellow Juju developers,

 I'd like to announce Menno's reviewer graduation.  I have been very
 happy with the quality and understanding shown in Menno's reviews.

 Congratulations.

Good job!

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard - most recent diff by default?

[Eric Snow]

On Mon, Oct 27, 2014 at 9:05 PM, Jesse Meek jesse.m...@canonical.com wrote:
 Is possible and preferable to show the most recent diff by default?

If you mean instead of showing the reviews page by default,
ReviewBoard doesn't support that out of the box.  Certainly we could
customize RB to do so, and for all I know it would be easy.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: per-database mongodump

[Eric Snow]

On Mon, Oct 27, 2014 at 9:26 PM, John Meinel j...@arbash-meinel.com wrote:
 What are the tables we're wanting to filter? (charm storage because of its
 size?)

The presence DB and a new one for backups.  We actually do want to
back up the blobstore DB (charms/tools).

 Certainly 3 and 4 aren't reasonable if the point is to avoid having 4GB of
 charm data in the backup.

Yeah, I'd rather avoid that if possible.  It would also make the
restore situation messier.  We should be able to avoid including
unnecessary (even invalid) data from the backup archive.


 If we went with (1) for consistency sake, we should really go back to stop
 the db, dump to a backup.

Yep.  Alternatively we could lock the databases.  Either way it would
mean juju would be unresponsive during backup.  I suppose when we have
a replicaset we could take one member offline to make the backup, but
I'd rather avoid that sort of complication.

 It is also sensitive to us adding a collection and forgetting to include it
 in the dump. (since it is a whitelist, right?)

I've already implemented option 1 (http://reviews.vapour.ws/r/268/).
The patch actually uses a blacklist, taking advantage of
mgo.Session.DatabaseNames().


 I'd really like (2) as a blacklist option, but I don't know how hard that is
 to do, and how much we expend to dump GB of charm data just to filter them
 out again.

mongodump actually dumps each database to its own directory.  If
necessary, the trick would be to hack the dumped oplog to remove
references to the ignored databases.  I just need to verify that
mongorestore --oplogReplay will be happy with that, or even if
mongorestore offers us any other route.  I'll be taking some time
today to look into it.

Thanks for the feedback.  After hearing your thoughts and taking
another look with fresh eyes, option 2 seems much more feasible and
palatable.  My only concern is that it relies on what amount to
implementation details of mongodump (and mongorestore).

-eric


 John
 =:-

 On Tue, Oct 28, 2014 at 3:07 AM, Eric Snow eric.s...@canonical.com wrote:

 For backup we currently call mongodump with the --oplog option.  This
 dumps all databases at the same moment in time, which gives us
 assurances about the consistency of each database.  However, we want
 to exclude some databases.  Unfortunately --oplog is not compatible
 with the --db option.

 I'm looking for options here.  I'm already considering, but am not
 satisfied with, the following:

 1. (mongodump --db ...) Forgo the moment-in-time guarantee of --oplog
 (seems risky).
 2. (mongodump --oplog ...) Manually remove the undesired databases
 from the dumped data (possible? risky!).
 3. (mongodump --oplog ...) Skip the undesired databases when calling
 mongorestore (possible?).
 4. (mongodump --oplog ...) Clear the undesired databases after calling
 mongorestore.

 The problem with 3 and 4 is that the backup archive will contain a
 bunch of erroneous data (the dumps of the databases we want to
 ignore).

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard-github integration

[Eric Snow]

On Tue, Oct 21, 2014 at 11:40 AM, Michael Foord
michael.fo...@canonical.com wrote:

 On 20/10/14 22:38, Eric Snow wrote:

 This should be resolved now.  I've verified it works for me.  If it
 still impacts anyone, just let me know.


 I still have the issue I'm afraid. No reviewer set, no diff.

 http://reviews.vapour.ws/r/211/

 Michael

:(

The frustrating part is that I haven't been impacted by this.  I'll
hack a solution in the immediate term and go from there.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard-github integration

[Eric Snow]

On Tue, Oct 21, 2014 at 3:40 AM, Michael Foord
michael.fo...@canonical.com wrote:

 On 20/10/14 22:38, Eric Snow wrote:

 This should be resolved now.  I've verified it works for me.  If it
 still impacts anyone, just let me know.


 I still have the issue I'm afraid. No reviewer set, no diff.

 http://reviews.vapour.ws/r/211/

For now I've hard-coded adding juju-team.  If anyone still has trouble
with this please let me know.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard-github integration

[Eric Snow]

On Tue, Oct 21, 2014 at 10:12 AM, Eric Snow eric.s...@canonical.com wrote:
 For now I've hard-coded adding juju-team.  If anyone still has trouble
 with this please let me know.

The issue with not finding files in the repo should be fixed now.
This means that auto-generated review requests should have diffs and
be published automatically.  If you have any trouble with it, let me
know.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard-github integration

[Eric Snow]

Yeah, this is the same issue that Ian brought up.  I'm looking into
it.  Sorry for the pain.

-eric

On Mon, Oct 20, 2014 at 5:31 PM, Dimiter Naydenov
dimiter.nayde...@canonical.com wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 Hey Eric,

 Today I tried proposing a PR and the RB issue (#202) was created, but
 it didn't have Reviewers field set (as described below), it wasn't
 published (due to the former), but MOST importantly didn't have a diff
 uploaded. After fiddling around with rbt I managed to do:
 $ rbt diff  ~/patch
 (while on the proposed feature branch)

 And then went to the RB issue page and manually uploaded the generated
 diff and published it.

 So most definitely the hook generating RB issues have to upload the
 diff as well :)

 It's coming together, keep up the good work!

 Cheers,
 Dimiter

 On 20.10.2014 16:53, Eric Snow wrote:
 On Mon, Oct 20, 2014 at 6:06 AM, Ian Booth
 ian.bo...@canonical.com wrote:
 Hey Eric

 This is awesome, thank you.

 I did run into a gotcha - I created a PR and then looked at the
 Incoming review queue and there was nothing new there. I then
 clicked on All in the Outgoing review queue and saw that the
 review was unpublished. I then went to publish it and it
 complained at least one reviewer was needed. So I had to fill in
 juju-team and all was good.

 1. Can we make it so that the review is published automatically?
 2. Can we pre-fill juju-team as the reviewer?

 Good catch.  The two are actually related.  The review is
 published, but that fails because no reviewer got set.  I'll get
 that fixed.

 -eric



 - --
 Dimiter Naydenov dimiter.nayde...@canonical.com
 juju-core team
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1

 iQEcBAEBAgAGBQJURSrnAAoJENzxV2TbLzHw0BQH/16P4qPDI28kkGs398qRKY5s
 eUtcHBpYs+JuLV2ZA0LjCpTds89RBDW6cKsxcfXxaAmawIb0KHh920VzKb1Wl2OT
 z/iMOF2q91LnV58dqPf7mZjHaT1LPRdSRxg6aAZW/mjexwVRtRDT4Asd5w6JpKrH
 9Tkqfy86OilJ70X8qNbegvjJrBAttwoLLI4jwJq4dNWUbWCBbuumryh0k6+GlmNH
 NiKbpi45pPy/RIFVA7ewbLIOpUXleHm5NIGlA/liZOMHpz0w5QHK3FYGLuGMNzQC
 fq4qW6rfb1ITdr7XWsA3gooV6FUndw3mbNsod3QgSv82RDA6GGECHeYimGG94/g=
 =POJ4
 -END PGP SIGNATURE-

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard-github integration

[Eric Snow]

This should be resolved now.  I've verified it works for me.  If it
still impacts anyone, just let me know.

-eric

On Mon, Oct 20, 2014 at 7:34 PM, Eric Snow eric.s...@canonical.com wrote:
 Yeah, this is the same issue that Ian brought up.  I'm looking into
 it.  Sorry for the pain.

 -eric

 On Mon, Oct 20, 2014 at 5:31 PM, Dimiter Naydenov
 dimiter.nayde...@canonical.com wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 Hey Eric,

 Today I tried proposing a PR and the RB issue (#202) was created, but
 it didn't have Reviewers field set (as described below), it wasn't
 published (due to the former), but MOST importantly didn't have a diff
 uploaded. After fiddling around with rbt I managed to do:
 $ rbt diff  ~/patch
 (while on the proposed feature branch)

 And then went to the RB issue page and manually uploaded the generated
 diff and published it.

 So most definitely the hook generating RB issues have to upload the
 diff as well :)

 It's coming together, keep up the good work!

 Cheers,
 Dimiter

 On 20.10.2014 16:53, Eric Snow wrote:
 On Mon, Oct 20, 2014 at 6:06 AM, Ian Booth
 ian.bo...@canonical.com wrote:
 Hey Eric

 This is awesome, thank you.

 I did run into a gotcha - I created a PR and then looked at the
 Incoming review queue and there was nothing new there. I then
 clicked on All in the Outgoing review queue and saw that the
 review was unpublished. I then went to publish it and it
 complained at least one reviewer was needed. So I had to fill in
 juju-team and all was good.

 1. Can we make it so that the review is published automatically?
 2. Can we pre-fill juju-team as the reviewer?

 Good catch.  The two are actually related.  The review is
 published, but that fails because no reviewer got set.  I'll get
 that fixed.

 -eric



 - --
 Dimiter Naydenov dimiter.nayde...@canonical.com
 juju-core team
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1

 iQEcBAEBAgAGBQJURSrnAAoJENzxV2TbLzHw0BQH/16P4qPDI28kkGs398qRKY5s
 eUtcHBpYs+JuLV2ZA0LjCpTds89RBDW6cKsxcfXxaAmawIb0KHh920VzKb1Wl2OT
 z/iMOF2q91LnV58dqPf7mZjHaT1LPRdSRxg6aAZW/mjexwVRtRDT4Asd5w6JpKrH
 9Tkqfy86OilJ70X8qNbegvjJrBAttwoLLI4jwJq4dNWUbWCBbuumryh0k6+GlmNH
 NiKbpi45pPy/RIFVA7ewbLIOpUXleHm5NIGlA/liZOMHpz0w5QHK3FYGLuGMNzQC
 fq4qW6rfb1ITdr7XWsA3gooV6FUndw3mbNsod3QgSv82RDA6GGECHeYimGG94/g=
 =POJ4
 -END PGP SIGNATURE-

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at: 
 https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard-github integration

[Eric Snow]

And of course if you see any problems please let me know. :)

-eric

On Sat, Oct 18, 2014 at 7:38 AM, Eric Snow eric.s...@canonical.com wrote:
 With the switch to Reviewboard we introduced extra steps to our
 workflow (mostly involving rbt).  This in turn made things more
 difficult for new/existing contributors.  I've been able to take some
 time in the last couple weeks to improve the situation by adding some
 integration between github and reviewboard.

 As of tonight that integration has reached an initial milestone.  The
 barriers to contribution introduced by Reviewboard are essentially
 gone.  Furthermore, the automation means the review requests should
 stay in sync with the pull requests.  So I'm happy to report that,
 unless you are chaining branches (which github PRs don't support
 anyway), you shouldn't need to use rbt anymore.

 Currently:

 * a new PR automatically triggers the creation of a new review request
 * the review request has a link back to the pull request
 * updates to the PR (i.e. pushes to your branch) automatically trigger
 an update to the review request
 * closing (discard/merge) a PR automatically triggers closing the
 corresponding review request
 * re-opening a PR automatically triggers re-opening the corresponding
 review request
 * a reviewboard user gets created if there wasn't one already

 Nearly working:

 * after the review request is created, a link to it is added to a pull
 request comment

 Future work:

 * support patch queues/chained branches/etc. (via trigger in PR summary)
 * add reviewboard support to the merge bot (check for ship-it before
 doing anything)

 Will not happen:

 * automatically merge PR when given ship-it
 * PR comments (including review comments) will not be pushed to the
 corresponding review request
 * likewise reviewboard comments won't be pushed up to the corresponding PR

 I can't promise that the future work will happen in the short term,
 but I'll post any updates as they come.  Enjoy!

 -eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

reviewboard-github integration

[Eric Snow]

With the switch to Reviewboard we introduced extra steps to our
workflow (mostly involving rbt).  This in turn made things more
difficult for new/existing contributors.  I've been able to take some
time in the last couple weeks to improve the situation by adding some
integration between github and reviewboard.

As of tonight that integration has reached an initial milestone.  The
barriers to contribution introduced by Reviewboard are essentially
gone.  Furthermore, the automation means the review requests should
stay in sync with the pull requests.  So I'm happy to report that,
unless you are chaining branches (which github PRs don't support
anyway), you shouldn't need to use rbt anymore.

Currently:

* a new PR automatically triggers the creation of a new review request
* the review request has a link back to the pull request
* updates to the PR (i.e. pushes to your branch) automatically trigger
an update to the review request
* closing (discard/merge) a PR automatically triggers closing the
corresponding review request
* re-opening a PR automatically triggers re-opening the corresponding
review request
* a reviewboard user gets created if there wasn't one already

Nearly working:

* after the review request is created, a link to it is added to a pull
request comment

Future work:

* support patch queues/chained branches/etc. (via trigger in PR summary)
* add reviewboard support to the merge bot (check for ship-it before
doing anything)

Will not happen:

* automatically merge PR when given ship-it
* PR comments (including review comments) will not be pushed to the
corresponding review request
* likewise reviewboard comments won't be pushed up to the corresponding PR

I can't promise that the future work will happen in the short term,
but I'll post any updates as they come.  Enjoy!

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard-github integration

[Eric Snow]

Also, I did this as a reviewboard extension.  All the code is online
(BSD license): https://bitbucket.org/ericsnowcurrently/rb_webhooks_extension.
I haven't charmed it up, but it should be pretty easy to adapt the
charm I wrote for rb_oauth.

-eric

On Sat, Oct 18, 2014 at 7:38 AM, Eric Snow eric.s...@canonical.com wrote:
 With the switch to Reviewboard we introduced extra steps to our
 workflow (mostly involving rbt).  This in turn made things more
 difficult for new/existing contributors.  I've been able to take some
 time in the last couple weeks to improve the situation by adding some
 integration between github and reviewboard.

 As of tonight that integration has reached an initial milestone.  The
 barriers to contribution introduced by Reviewboard are essentially
 gone.  Furthermore, the automation means the review requests should
 stay in sync with the pull requests.  So I'm happy to report that,
 unless you are chaining branches (which github PRs don't support
 anyway), you shouldn't need to use rbt anymore.

 Currently:

 * a new PR automatically triggers the creation of a new review request
 * the review request has a link back to the pull request
 * updates to the PR (i.e. pushes to your branch) automatically trigger
 an update to the review request
 * closing (discard/merge) a PR automatically triggers closing the
 corresponding review request
 * re-opening a PR automatically triggers re-opening the corresponding
 review request
 * a reviewboard user gets created if there wasn't one already

 Nearly working:

 * after the review request is created, a link to it is added to a pull
 request comment

 Future work:

 * support patch queues/chained branches/etc. (via trigger in PR summary)
 * add reviewboard support to the merge bot (check for ship-it before
 doing anything)

 Will not happen:

 * automatically merge PR when given ship-it
 * PR comments (including review comments) will not be pushed to the
 corresponding review request
 * likewise reviewboard comments won't be pushed up to the corresponding PR

 I can't promise that the future work will happen in the short term,
 but I'll post any updates as they come.  Enjoy!

 -eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Is ReviewBoard a good thing?

[Eric Snow]

On Sat, Sep 20, 2014 at 12:01 AM, Jesse Meek jesse.m...@canonical.com wrote:
 On 20/09/14 02:34, Eric Snow wrote:
 I was not seriously suggesting we return to lp. Using ReviewBoard
 reintroduces what we gave up with lp: both the good (tooling that addresses
 pain points) and the bad (not a well known/widely adopted process of
 contributing). In this regard, using ReviewBoard is akin to returning to lp.

Sorry, I misread.  Thanks for clarifying.


 It is not a question of does ReviewBoard address our pain points nor is it
 a question of is this just teething?. Both valid questions in their own
 right, but they miss the point. Is raising the bar to outside contributors
 necessary and justified?

What do you mean by raising the bar?  If you mean the extra steps
we've introduced for reviewboard, I agree to the extent that we do not
yet have any automation between github and reviewboard, and we must
take the steps manually.  However, once we have automated those steps
it will be a non-issue.

Furthermore, I will argue that reviewboard provides a better code
review experience than github.  That's a relatively subjective
assessment, but there are also concrete benefits that I hope I've
outlined well in the pros and cons thread.

FWIW, I do not plan on updating the CONTRIBUTING doc until after we
have github-reviewboard automation in place.  Until then outside
contributers won't have to worry about reviewboard.  And afterward
they still won't have to worry about more than simply following the
link to the review request for their PR.


 Is it necessary? Many of us have addressed our own pain points with GitHub
 already. I use browser plugins, git hooks and scripts to augment my
 workflow.

I'd be interested to know what folks are using to work around the
deficiencies in github (reviews or otherwise).  I expect such things
would be generally useful.  Ideally no one would need to worry about
such workarounds, which is what we're trying to address via
reviewboard, but I expect that any such tools would be useful,
reviewboard or not.

 Yet I can work along side the first time contributor that has
 nothing more than git and a GitHub account. What pain point necessitates
 raising the bar to contributors?

I agree that, without the github-reviewboard automation, any
requirements to use reviewboard are more roadblocks to contribution.


 Is it justified? Given such a pain point exists, does solving it justify
 *forcing* a new tool on a developer? This is the decision we are making and
 it is not just for 'us' the team. It is for our would-be external
 contributors. The ones that we moved to GitHub for.

I'm glad you spoke up on this.  It's important we keep this firmly in
mind when making any changes to workflow.  FYI, I had a patch for
CONTRIBUTING.md that updated the workflow relative to the new
reviewboard steps/tooling.  However, you've convinced me to abandon it
in favor of simply waiting until we have automation in place, to avoid
adding barriers to entry.  Thanks. :)

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Is ReviewBoard a good thing?

[Eric Snow]

On Fri, Sep 19, 2014 at 7:55 AM, Dimiter Naydenov
dimiter.nayde...@canonical.com wrote:
 +1, In addition you can always check
 https://github.com/juju/juju/pulls to see what's in the queue. For
 sub-repositories it's the same, like
 https://github.com/juju/names/pulls. While I agree it's not all in one
 place, I tend to work on the main repo mostly, and alternatively you
 can check the Notifications page (https://github.com/notifications) to
 see all activity.

The problem is when your PR in another repo sits there because
everyone tends to work on the main repo mostly.  A unified review
queue solves that quite nicely. :)

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Is ReviewBoard a good thing?

[Eric Snow]

On Fri, Sep 19, 2014 at 8:34 AM, Eric Snow eric.s...@canonical.com wrote:
 I agree that reviewboard as we currently have it now adds extra work
 to our workflow. Not only does this impact the juju team, but it does
 add a stumbling block to more community involvement.  However, my firm
 belief is that the real pain points are addressable in the short term.
 Let's give it a chance.

Just to be clear, while I have spent a lot of time on setting up
ReviewBoard and believe it's our best option, I appreciate this
discussion and gladly accept both positive and negative feedback.  If
we decide that ReviewBoard isn't worth it, I'll support the decision
and move on. :)  Furthermore, I recognize that the future of
reviewboard for our team is mostly up to the team leads but ultimately
is driven by the whole team.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Is ReviewBoard a good thing?

[Eric Snow]

On Fri, Sep 19, 2014 at 6:11 AM, Gabriel Samfira
gsamf...@cloudbasesolutions.com wrote:
 Just a suggestion:

 A git plugin similar to what Gerrit has would simplify things. For example,
 Gerrit has a nice little plugin called Review. Simply doing:

 git review

 In your current branch would push the patchest to gerrit. Something similar
 for RB, would probably simplify things a lot. Chained PR's could probably be
 done by specifying in the commit message something like:

 depends on #PR ID

That would definitely ease the pain.  Even if we have automation of
the main workflow, a git plugin would still help with chained branches
(which github does not support).

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Is ReviewBoard a good thing?

[Eric Snow]

On Thu, Sep 18, 2014 at 6:32 PM, David Cheney
david.che...@canonical.com wrote:
 There were three problem reviewboard was supposed to address, review
 comments are sent immediately, no side by side diffs, and no way to to
 chained proposals.

It will be worth being extra clear on ReviewBoard's pros and cons.
I'll open a new thread.


 I think that over the last few months we've all learnt to live with
 the first issue

I for one haven't.  On several occasions I have made a comment that I
later realized was not valid, but am not able to simply remove since
it already sent.


 On the second, github now does nice side by side diffs.

Agreed.


 On the third, it appears reviewboard leaves you solidly to your own
 devices to implement chained proposals.

rbt supports it just fine and reviewboard itself supports it (see the
Depends on field on every review request).  What support did you
have in mind?

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

The Pros and Cons of ReviewBoard.

[Eric Snow]

Given that I've in some part driven the switch to ReviewBoard, I want
to make sure we are all on the same page and any decision on its
future can be made objectively.  This is an outgrowth of the current
discussion on whether or not we should ditch reviewboard.

Let's look at the pros and cons of using it (at least relative to
github).  Feel free to expand on any point here or add to them.

-eric

ReviewBoard Pros:

* self-hosted (flexibility, ownership)
* unified review queue with detailed info
* reviews are composed of multiple comments, not just one
* reviews have worklow-supporting metadata (ship-it, issues)
* reviews can be edited as a whole before publishing
* review comments are threaded (provides context)
* customizable (3rd party and custom extensions)
* extensive remote API
* some github integration
* supports chained branches (anti-pattern?)
* allows you to look at new changes in context of old comments
* allows you to look at changes between review request updates
* does not require a PR to exist

ReviewBoard Cons:

* self-hosted (hosting, maintenance, etc.)
* adds manual steps to our workflow
* extra steps increase the barrier to contributing
* not a part of the mainstream github workflow
* requires adjusting to a new tool for most people
* web UI has some usability issues (list?)
* emails formatting is complicated (subjective)

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: The Pros and Cons of ReviewBoard.

[Eric Snow]

On Fri, Sep 19, 2014 at 9:37 AM, Eric Snow eric.s...@canonical.com wrote:
 Given that I've in some part driven the switch to ReviewBoard, I want
 to make sure we are all on the same page and any decision on its
 future can be made objectively.  This is an outgrowth of the current
 discussion on whether or not we should ditch reviewboard.

 Let's look at the pros and cons of using it (at least relative to
 github).  Feel free to expand on any point here or add to them.

 -eric

 ReviewBoard Pros:

 * self-hosted (flexibility, ownership)
 * unified review queue with detailed info
 * reviews are composed of multiple comments, not just one
 * reviews have worklow-supporting metadata (ship-it, issues)
 * reviews can be edited as a whole before publishing
 * review comments are threaded (provides context)
 * customizable (3rd party and custom extensions)
 * extensive remote API
 * some github integration
 * supports chained branches (anti-pattern?)
 * allows you to look at new changes in context of old comments
 * allows you to look at changes between review request updates
 * does not require a PR to exist

 ReviewBoard Cons:

 * self-hosted (hosting, maintenance, etc.)
 * adds manual steps to our workflow
 * extra steps increase the barrier to contributing
 * not a part of the mainstream github workflow
 * requires adjusting to a new tool for most people
 * web UI has some usability issues (list?)
 * emails formatting is complicated (subjective)

Solutions:

* add integration between github and reviewboard (github webhooks)
  - addresses manual steps (i.e. barrier-to-entry/workflow concerns)
* provide a git plugin that wraps rbt and better supports our workflow
  - addresses complex workflow concerns
* (unlikely) Modify and add to the web UI (via an extension)
  - addresses web UI concerns
* (unlikely) Modify and add to the email formatting (via an extension)
  - addresses email formatting concerns

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: ReviewBoard is now the official review tool for juju

[Eric Snow]

On Sun, Sep 14, 2014 at 10:28 PM, Menno Smits menno.sm...@canonical.com wrote:
 Eric,

 Thanks for setting this up.

 Firstly, in your email you mention rbt pull several times. I think you
 mean rbt post right? I don't see a pull command documented anywhere.

Correct.  I meant rbt post. :)


 I've run in to one issue so far. When I tried to get my first review in to
 Reviewboard today it took me a long time to figure out how to get it to
 generate the correct diff. After much gnashing of teeth I figured out that
 rbt post generates a diff by comparing origin/master against the current
 branch. This means that if you haven't updated your local master branch
 recently and pushed your local master branch to your personal fork on Github
 (this is the part I missed) you'll end up with diffs that include lots of
 changes that have already been merged and have nothing to do with your
 branch.

 As things stand the workflow actually needs to be:

 1. Ensure your feature branch is rebased against upstream/master
 2. Create a pull request like normal via github.
 3. Switch to your local master branch.
 4. git pull to update master
 5. git push origin master to update your personal master on github.
 5. Switch back to your feature branch (git checkout - helps here)
 6. Run rbt post while at your branch to create a review request.
 7. open the review request in your browser and publish it.
   - alternately use the rbt --open (-o) and/or --publish (-p) flags.
 8. add a comment to the PR with a link to the review request.
 9. address reviews until you get a Ship It! (like normal, with LGTM).
 10. add a $$merge$$ comment to the PR (like normal).

 This is a bit confusing and inconvenient. I can see us all forgetting to
 keep our personal master branches on GH updated.

Yeah, those steps are a lot, though keep in mind that effectively it's
only 2 steps more than before if you use the -p flag to rbt post and
were already keeping your local master up to date.

However, I'll look into a cleaner approach.  If rbt cannot handle it,
it may make sense to write a quick git review command that wraps all
that.  FYI, I've been using a git refresh command for a while that
wraps the pull/rebase steps, so writing one that does the extra review
steps shouldn't be too onerous.

All this is part of why I had considered waiting until we could roll
out proper github-reviewboard integration.  However, I felt like
ReviewBoard was enough of a benefit on its own that waiting for the
integration was unwarranted.


 It looks like the TRACKING_BRANCH option in .reviewboardrc could be helpful.
 It defaults to origin/master but if we changed it to upstream/master I
 suspect Reviewboard will then generate diffs against the shared master
 branch instead of what we might happen to have in master in our personal
 forks. The of course relies on every developer having a remote called
 upstream that points to the right place (which isn't necessarily true).

I'll take a look at that.


 If TRACKING_BRANCH isn't going to work then whatever automation we come up
 with to streamline RB/GH integration is probably going to have to sort this
 out.

Ultimately I think it's worth getting the tooling right in the very
short term. :)

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: ReviewBoard is now the official review tool for juju

[Eric Snow]

On Mon, Sep 15, 2014 at 8:09 AM, Eric Snow eric.s...@canonical.com wrote:
 Yeah, those steps are a lot, though keep in mind that effectively it's
 only 2 steps more than before if you use the -p flag to rbt post and
 were already keeping your local master up to date.

Just to be clear, here are the steps again, slightly reformatted:

(0). Rebase relative to upstream master.
  - if origin is different than upstream, sync and push it
1. Create a pull request via github.
2. Run rbt pull -p while at your branch to create a review request.
3. add a comment to the PR with a link to the review request.
4. address reviews until you get a Ship It! (like normal, with LGTM).
5. add a $$merge$$ comment to the PR (like normal).
6. mark the review request as submitted.

So, steps 2, 3, and 6 are completely new.  They don't add a lot of
work and I plan on automating all 3 of those new steps.

Step (0) is also pretty easy and I'll argue that people should be
doing it anyway.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: ReviewBoard is now the official review tool for juju

[Eric Snow]

On Mon, Sep 15, 2014 at 11:05 AM, Katherine Cox-Buday
katherine.cox-bu...@canonical.com wrote:
 Let me preface this by saying I like the Reviewboard style of reviewing
 changes.

 It's somewhat concerning to me that our reviews are now disconnected from
 what will actually be landed into trunk. In Github, you were reviewing the
 actual diff which would be landed. In reviewboard, we're now reviewing a
 diff manually uploaded by the developer. There's a lot of room for error in
 terms of what diff we review vs. what diff we land.

 Any thoughts on how to couple these things once again?

I'm working on integration between github and reviewboard such that
creation of a PR creates a new review request automatically.  The same
goes for updating a PR.  Not only will that address what you are
talking about, it will remove the extra steps of creating/updating the
review request yourself and of closing a review request as submitted.
Ergo, rbt would not be needed for the typical workflow.  You would
still use it for pipedlined branches, though that could probably be
automated as well.

There is the possibility of pushing info from ReviewBoard back to
github (e.g. ship-it - LGTM comment), but I don't think it buys
us enough to make it worth it (it's notably trickier).

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: ReviewBoard is now the official review tool for juju

[Eric Snow]

On Mon, Sep 15, 2014 at 12:54 PM, Dimiter Naydenov
dimiter.nayde...@canonical.com wrote:
 - From my meager experience with writing git plugins (any executable in
 $PATH with git- prefix), what are you describing can be easily
 achieved. If you write a git plugin, named e.g. git-rbpropose, using
 the GitHub CLI (https://github.com/github/hub) and rbt, you can
 automate the process:
  1. Pushing the branch to origin (checking for uncommitted changes).
  2. Creating a GitHub PR for the branch, which includes launching the
 default editor to fill-in the PR description (using hub).
  3. Creating a RB review (using rbt).
  4. Optionally opening the default browser with it.

 I have a couple of handy scripts that do this, which I've shared before:
 git-sync (), and git-propose (), along with a few aliases (). git-sync
 fits especially well with the RB workflow, because it pull
 upstream/master into your local repo's master, then pushes it back to
 origin/master, and finally (when you're on a branch other than master)
 rebases all branch commits over origin/master, interactively. What
 usually do is run git propose after the finaly git sync to create
 a PR - the only thing missing is the RB steps.

Cool.  I'll take a look.


 Cheers and thanks for all the hard work around putting RB workflow in
 place,

Glad to do it. :)

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

repos in reviewboard

[Eric Snow]

Which of the repositories listed at https://github.com/juju should be
set up in reviewboard?  I'm pretty sure on most of them, but a more
authoritative list would help me out.  In the meantime I'm adding the
ones I'm sure on.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: ReviewBoard and our workflow

[Eric Snow]

On Thu, Sep 11, 2014 at 1:34 AM, Frank Mueller
frank.muel...@canonical.com wrote:
 For switching to a new tool and a new workflow I would like to not simply
 discuss it in a somehow undefined way together with subjunctive terms
 (Everybody should now ...) here via mail. Please lets fix the workflow in
 a how-to-contribute.md in juju/doc/contributions, so that we easily can
 point any new internal and external contributor to it. And also let's have a
 well defined date for switching and communicate it early enough.

 If somebode is already writing this doc mentioned above please let me know.
 Otherwise I'll do it.

Hi Frank,

I was planning on updating
https://github.com/juju/juju/blob/master/CONTRIBUTING.md once I felt
comfortable with how reviewboard fit into our workflow.  Do you think
something in juju/doc/contributions, which perhaps elaborates on the
info in CONTRIBUTING.md, would be a worthy addition?  Thanks for
bringing this up, by the way. :)

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: ReviewBoard and our workflow

[Eric Snow]

On Wed, Sep 10, 2014 at 3:34 PM, Menno Smits menno.sm...@canonical.com wrote:
 Thanks Eric! I've used Reviewboard at a previous job and I'm fairly sure
 that it aligns better with the way the Juju Core team likes to work than
 Github's review features.

 Two questions:

 1. Is this what we're supposed to be doing from right now?

Nope.  I'm hopeful that we will switch over to reviewboard in a week
or two.  Until then github is still our review tool.  However, in the
meantime feel free to try out the Reviewboard-oriented workflow as
well. :)


 2. I'm pretty sure some configuration of the rbt tool is required so that it
 knows how to talk to the Reviewboard server. Is there a config file
 available?

The first time in a repo you run rbt setup-repo which generates a
user-agnostic, repo-specific .reviewboardrc file.  I expect that
before long we will commit that file in each of our repos.  Once that
happens, no one will have to do anything special (e.g. run rbt
setup-repo) any longer.

See https://www.reviewboard.org/docs/rbtools/dev/rbt/configuration/

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard

[Eric Snow]

On Mon, Sep 8, 2014 at 8:05 PM, Tim Penhey tim.pen...@canonical.com wrote:
 On 09/09/14 04:32, Eric Snow wrote:
 To install rbt:

 sudo pip install --allow-unverified rbtools --allow-external rbtools rbtools

 Ah... no.

 Perhaps in a virtual env.

Is it the sudo or the flags to which you object? :)  Using a
virtualenv would indeed be a good idea!  So, as a correction to my
previous instructions:

  virtualenv ~/.venvs/reviewboard
  ~/.venvs/reviewboard/bin/pip install --allow-unverified rbtools
--allow-external rbtools rbtools
  alias rbt='~/.venvs/reviewboard/bin/rbt'

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard

[Eric Snow]

Also, I've written up all the steps for deploying, as well as config
changes, a to-do list, and other reviewboard-related details:

https://docs.google.com/a/canonical.com/spreadsheets/d/1OLg-AEGHasnDgUUaK5mVhK8FOyvsPPJrWds6JtUhZb8/edit?usp=sharing

Not all the config settings I've listed have been done yet.  I'll roll
those out when I can.  Also note that for some of the config settings
I'm using values other than what I've listed in the spreadsheet.  Each
of those will be changed to the correct value before we switch over.

-eric

On Fri, Sep 5, 2014 at 7:09 PM, Eric Snow eric.s...@canonical.com wrote:
 I'm pleased to announce that we now have a working demo of ReviewBoard
 available:

 http://reviews.vapour.ws/

 Feel free to take it for a spin!

 There's no need to register for an account.  Just click the github
 OAuth button on the login page.  The first time you do so you'll be
 redirected to a github page asking if you approve.  Then you'll be
 redirected back to ReviewBoard.  At that point it will automatically
 create an account for you and log you in.  Due to session cookies, you
 shouldn't need to log in all that often.

 I've set up two of the repos to start us off.  I can set up more if it
 would help, but the two should be enough to get us a feel for things.

 Let me know any feedback you have.  There are a number of
 configuration options we can tweak.

 You can do just about everything through the web interface, but I
 recommend using the reviewboard client CLI rbt (pip install
 RBTools).

 Keep in mind that this is only a demo.  Though it's mostly complete
 and the  final release will be at the same URL (except SSL-only),
 there are still settings we need to tweak and workflow considerations
 to resolve.

 Consequently, the site will probably be wiped at least once before any
 final release.  So try it out but don't do anything important on
 there.

 -eric

 p.s. Dog food was served.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard

[Eric Snow]

On Mon, Sep 8, 2014 at 4:37 AM, Ian Booth ian.bo...@canonical.com wrote:
 Hi Eric

 Fantastic, thank you.

 Quick question - can we set up a Juju team group and have that group
 automatically be assigned as a reviewer for newly created review requests? I
 tried to create a new request using the web ui and had to manually enter the
 reviewer. Except there was no group set up yet for the Juju team.

I've added a review group (juju-team) and a default reviewer
(juju-tream) with that group (and all repos) assigned to it.  As we
add repos and users I'll make sure that stays up to date.


 Also, when creating a new review request, it shows a list of commits to choose
 from, whereas I would be wanting to see a list of branches since that's how we
 create the PRs on Github and the branch is what the review is based on. Can we
 fix this?

Unfortunately reviewboard doesn't support github PRs and they don't
plan on supporting it.  At some point I'd like to add support for
automatically generating review requests from pull requests and
automatically notifying the CI bot when a review request gets a
ship-it.

In the meantime I recommend using rbt (as echoed by Adam).  I know
As a bonus, rbt allows you to create review requests relative to a
parent revision (ergo branch), so you can chain patches.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard

[Eric Snow]

On Mon, Sep 8, 2014 at 10:21 AM, Eric Snow eric.s...@canonical.com wrote:
 In the meantime I recommend using rbt (as echoed by Adam).  I know
 As a bonus, rbt allows you to create review requests relative to a
 parent revision (ergo branch), so you can chain patches.

Wayne just pointed out to me that rbt + our OAuth-based users aren't a
great mix.  This is because the auto-registered reviewboard users are
not set up with passwords.  I'll figure out a solution for us.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard

[Eric Snow]

On Mon, Sep 8, 2014 at 10:21 AM, Eric Snow eric.s...@canonical.com wrote:
 In the meantime I recommend using rbt (as echoed by Adam).  I know
 As a bonus, rbt allows you to create review requests relative to a
 parent revision (ergo branch), so you can chain patches.

To install rbt:

sudo pip install --allow-unverified rbtools --allow-external rbtools rbtools

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: reviewboard

[Eric Snow]

On Mon, Sep 8, 2014 at 10:23 AM, Eric Snow eric.s...@canonical.com wrote:
 Wayne just pointed out to me that rbt + our OAuth-based users aren't a
 great mix.  This is because the auto-registered reviewboard users are
 not set up with passwords.  I'll figure out a solution for us.

The OAuth stuff should work fine if you use your github username for
the username and oauth:username@github for your password
(replacing username with your github username).

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Review Request 15: juju-run: allow commands to specifiy a relation context

[eric . snow]

---
This is an automatically generated e-mail. To reply, visit:
http://reviews.vapour.ws/r/15/#review4
---


A general comment.

- ericsnowcurrently


On None, wwitzel3 wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 http://reviews.vapour.ws/r/15/
 ---
 
 Review request for juju team.
 
 
 Repository: juju (core)
 
 
 Description
 ---
 
 This adds the ability to specify a --relation context with juju run.
 
  juju run --unit=haproxy/0 --relation=cache “hooks/cache-relation-changed”
 
 
 Diffs
 -
 
   worker/uniter/uniter_test.go 1c438c0459b7bfba1983ec3862f0829ddc4f0e31 
   worker/uniter/uniter.go a12383401c39c5c00dd318408565a6478f085f35 
   worker/uniter/runlistener_test.go 6f6bf77e5946e663eac5743dd79920bc57b18fa2 
   worker/uniter/runlistener.go 97cb078a04a749d30c77fc965e5d4f5322baaf3e 
   worker/uniter/context.go 8f7c3313d15a1d2ef3408f1b267127803d91b056 
   cmd/jujud/run_test.go 2d0d890bfde58d87ed5840c8b6f4f4f67ae69092 
   cmd/jujud/run.go a6e9073275ee0338b8fe03217879aed6dcc9e6c6 
   cmd/juju/run_test.go dce99485e93b22eff418cd3d5f00bedb8653508b 
   cmd/juju/run.go ecd625cd15754f4c87b50ff27a3b642ae5a5621c 
   apiserver/params/internal.go b3f281f2f3f0ac2791d18f4dda30defb8821d235 
   apiserver/client/run.go c39e8f9e185a2562bf8be7537d0e2c3930cd13c3 
 
 Diff: http://reviews.vapour.ws/r/15/diff/
 
 
 Testing
 ---
 
 All tests passing.
 Manual testing and verification of run commands against a mongodb cluster 
 setup.
 Bootstrapping of local, ec2, and digitalocean.
 
 
 Thanks,
 
 wwitzel3
 


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: my first charm

[Eric Snow]

On Wed, Sep 3, 2014 at 11:37 PM, Andrew Wilkins
andrew.wilk...@canonical.com wrote:
 On Thu, Sep 4, 2014 at 3:46 AM, Eric Snow eric.s...@canonical.com wrote:
 Here's a write-up on my experience writing a charm for the first time.

 Thanks for writing this up.

Blame Nate!  He had the sense to ask me to do it. :)

 * there is no unexpose command

 ???

 andrew@blank:~$ juju help unexpose
 usage: juju unexpose [options] service
 purpose: unexpose a service

 options:
 -e, --environment (= )
 juju environment to operate in
 andrew@blank:~$


Wow.  I could swear I did not see it when I tried, yet I see it there
too.  I'm glad because if there hadn't been one it would have made me
question my understanding of what expose does.

 [1] I haven't had a chance to look at haproxy, but I'd expect that
 interface to be dependent on services that support multiple units.

 What are you after, if not something which load balances across multiple
 units?

If that's what haproxy does (where the units have no knowledge of each
other) then it's the right tool for the job.  I'm already planning on
using a reverse-proxy for SSL and quickly switching to a fail-over
instance, but I'm guessing that won't change (at least for the SSL
needs).

For redundancy I'm also planning on setting up replication on the
postgresql charm, which I'm guessing would have to happen
independently of haproxy.  However, that will be easy (the charm
already supports it).

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Logger prefixes for api/apiserver are changing

[Eric Snow]

On Tue, Sep 2, 2014 at 5:07 AM, David Cheney david.che...@canonical.com wrote:
 I wonder if there is a better way to handle this default case of the
 name of the logger follows the package path), ie

 var logger = loggo.Logger() // or something, it could be a new
 method, logger.Default(), or something.

 which will inspect the call tree and determine its logger name automagically.

+1

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: State should not depend on the API server

[Eric Snow]

On Mon, Sep 1, 2014 at 12:03 AM, John Meinel j...@arbash-meinel.com wrote:
 FWIW I'd favor 3 layers, though it does mean you have to do copying between
 structs that would likely otherwise be almost identical. A State layer for
 saving in the DB, an API layer for communication, and a Model layer for use
 in the Agents/Clients.

This is pretty much what I've done for the new backups in response to
conversations with Menno and William.  It's worked out fine.  The 3
types are similar, but in each case have been slightly different.  So
the choice of having them different has proved helpful.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: backup API in 1.20

[Eric Snow]

On Wed, Jul 30, 2014 at 12:25 AM, John Meinel j...@arbash-meinel.com wrote:
 So as discussed, I think we still want to support a HTTP GET based API for
 actually downloading the content to the client. And the CLI can do steps
 like:
  RPC to a request to create a backup (either this is synchronous and returns
 a URL, or the next step is to wait on the next request for when it is done
 to get the URL)
  HTTP GET to download the backup .tar (.xz,,bz2,gz whatever)
 put those bytes somewhere locally and then exit.

That's what I understood as well.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

backup API in 1.20

[Eric Snow]

The API server side of backup made it into 1.20 (the client-side and
CLI did not).  However, the API is exposed via HTTP POST rather than
through websockets RPC.  We are correcting this right now.  The
question is, are there any objections to removing backup from the
state API in 1.20 (or, less desirably, backporting the
POST-websockets change)?

Proposal
===
Remove state/backup/ and state/apiserver/backup.go from 1.20.

Alternative
=
Backport POST-websockets change to 1.20.

Motivation

* eliminate POST-base backup API
* juju 1.20 in trusty

Impact
==
The impact on users should be vanishingly small.  The client and CLI
sides of backup did not make it into 1.20, and we have not documented
the state API.  I expect that there is no one using the backup API
currently.  The situation would be more problematic if 1.20 makes it
into trusty; we would have to continue supporting the POST-based
backup API (along with the newer websockets one).

The impact on juju should be positive.  We will not need to support
both a POST-based backup API and a websockets-based one.  Removing the
backup API will be a much less complicated change than backporting the
websockets-based API.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: full functional tests for API methods

[Eric Snow]

Cool.  I'll have a look. :)

-eric

On Tue, Jul 22, 2014 at 10:30 PM, John Meinel j...@arbash-meinel.com wrote:
 https://docs.google.com/a/canonical.com/document/d/1hXAOBVpM8o--9L7GOb4r6NUaloZQ8BsWj8W-Bqpznpw/edit?usp=sharing
 is about how CI is set up today, and how to add more tests there.

 John
 =:-


 On Tue, Jul 22, 2014 at 7:10 PM, Eric Snow eric.s...@canonical.com wrote:

 On Tue, Jul 22, 2014 at 8:17 AM, Eric Snow eric.s...@canonical.com
 wrote:
  Yep.  For my case (backup/restore) we have a good enough solution
  already.

 Incidentally, do we have documentation somewhere on CI tests (e.g.
 backup/restore)?  Specifically, where are they, how do they work, and
 how do you install them?

 -eric

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev



-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: full functional tests for API methods

[Eric Snow]

On Tue, Jul 22, 2014 at 8:17 AM, Eric Snow eric.s...@canonical.com wrote:
 Yep.  For my case (backup/restore) we have a good enough solution already.

Incidentally, do we have documentation somewhere on CI tests (e.g.
backup/restore)?  Specifically, where are they, how do they work, and
how do you install them?

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Splitting out state/api into its own repo

[Eric Snow]

On Fri, Jun 27, 2014 at 3:05 AM, William Reade
william.re...@canonical.com wrote:
 I think one of the biggest problems is the naming: state/api is a hackish
 and minimal api client implementation, while state/apiserver is where the
 actual api is defined... except the params package, which for some reason
 lives under state/api.

 I think the most important actions are:

   * move state/api/params under state/apiserver
   * move state/apiserver to the top level, and make sure it's clearly
 documented

 Then I'd be keen to separate the internal api client code from the external
 one; and at that point I'd be happy to move the external api client code
 into its own repo. There's no disadvantage to having that code external,
 because we can't afford to break our external api clients regardless; for
 the internal ones we have more power and control, because we're the only
 ones who have to deal with the impact of change.

 (This would then involve separating the protocol-level code out somewhere
 *else*, so that we could reuse it both internally and externally; and we'd
 probably want both the server and client protocol parts together; but I
 think that the point where we can reasonably move a package outside the main
 repo is some way away regardless, so I'm not keen to focus on it at the
 moment.)

This all sounds great.  It's basically what I was trying to suggest.
:)  I just don't have as clear a picture of the distinction between
internal/external client, which is where I went awry.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Splitting out state/api into its own repo

[Eric Snow]

On Fri, Jun 27, 2014 at 5:36 AM, Nate Finch nate.fi...@canonical.com wrote:
 have the code somewhere easily accessible in some repo, preferably without
 needing to download the entire juju codebase.

Good point.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Splitting out state/api into its own repo

[Eric Snow]

On Thu, Jun 26, 2014 at 11:04 AM, Gustavo Niemeyer gust...@niemeyer.net wrote:
 Hey Eric,

 Some comments below, offering a slightly different perspective to be
 used as a data point in your quest.

That was totally helpful.  And with the further discussion (thanks
everyone!), I'm still hopeful that something good will come of this
(besides the benefit to my understanding). :)

In the end I'd argue that anything we can do (within reason) to make
the code easier to understand is a win.  That benefits not only
newcomers to the code like myself, but the project as a whole; if done
thoughtfully the code often ends up with better boundaries between
components and between internal-/external-facing roles.

Regarding understandability, I can't speak to what effort has gone
into a complex system like juju (though I suspect it has been
significant), but my experience thus far has been that the code
(specifically the API code) in its current state hasn't been the
easiest to follow.  I expect the complexity of the system has a lot to
do with that.  From the discussion so far it sounds like there are
things we can do about it, which is great!

 For 3, splitting it off not only seems to needlessly increase the
 maintenance burden, but also feels incorrect from the orthogonality
 standpoint: state/api maps state into a public API, and is a critical
 dependency for juju to work at all.

Yeah, as William pointed out, splitting the API client out into its
own repo would only make sense after distilling it down to the actual
public-facing part.

Anyway, thanks again for the insight!  Everyone's been really helpful
as I've been spinning up.

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Splitting out state/api into its own repo

[Eric Snow]

Thanks to all the responses thus far.  While the discussion has been
insightful, I'm also hopeful that it will still be fruitful. :)

Understandably the responses have been focused on the main point of my
original post, splitting out the API client into its own repo.
However, there were a few other points/questions that I think would be
worth revisiting (or are effectively still on the table) that were
otherwise tucked away here.  Following is the original email with the
everything but those points snipped away (and edits in square
brackets).

-eric

 For a while I thought `state/api`
 held the state API client code, but from what I understand now it
 actually contains all the public facing code for the juju state API
 (and [consequently?] for juju as a whole?).  In some regard I would
 expect a public API to be sit in a top-level package (rather than
 nested down like it is).

 * introduce a juju API client interface type

 * is `state/api` just for the state client/RPC, juju state in general,
 or juju as a whole?
From what I understand, it is the middle one (though I originally
 thought it was the first).
 * is there other public juju API other than the state-related API?

 * should the `api` [package] contains *all* public-facing juju API
 (state-related or not), regardless of whether or not `state/API`
 currently does?

 * would it be worth restructuring the `api` package to group the
 different methods/constants/types by component (e.g. charms/state
 archive/tools/services/units/etc.)?
This would particularly impact `api/params`.

 * how much should dependencies in juju proper on the `api` package be
 dialed back?
I'd think it would be as much as possible.  More encapsulation
 around the API/RPC part of juju would be good.

 * should the underlying state client RPC implementation move over with
 `state/api` (or even into its own repo)?

 Alternatives
 =
 * apply some of the ideas here to `state/api` rather than in a new repo
 * move some or all of `state/api` into a new top-level `api` package

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Splitting out state/api into its own repo

[Eric Snow]

(I've put a more structured proposal below, but here's some context.)

Over the last couple weeks I've been spinning up on the juju code
base, which is large enough to dissipate any hope of understanding it
all quickly. wink Most of what I've focused on is relative to the
juju tools and the remote API, both for the sake of backup/restore.

One thing that threw me off is that it has not been obvious that the
code under `state/api` is the public-facing API for juju's state (as
someone recently explained to me).  For a while I thought `state/api`
held the state API client code, but from what I understand now it
actually contains all the public facing code for the juju state API
(and [consequently?] for juju as a whole?).  In some regard I would
expect a public API to be sit in a top-level package (rather than
nested down like it is).

A couple of other things got in the way.  For one, we basically don't
have any documentation for the API.  I expect that it will mirror the
documentation for the juju subcommands/tools pretty closely, but
regardless the doc doesn't exist.  Setting up godoc for the api
package would be great and so would a new page in the juju
documentation.  I realize there has been some discussion on this point
of late, from which I expect a doc will take shape in the short term.
In the meantime, we've actually been telling people to wrap calls to
the CLI tools rather than using the API.  The documentation side of
things is a somewhat orthogonal, though topically related, issue.

For another, while there has been a pretty good effort to keep the
`api` package relatively un-entwined from the rest of juju, there were
a few times when I found it hard to follow what was going on.  This
was particularly true of the underlying state RPC implementation,
though at this point things make a lot more sense.  Having a separate
repo would help delineate the boundary between the API and juju
itself, which should in turn help make the API code easier to follow.

So...

In the interest of understanding juju better and of making the API
more accessible, I took a little time to investigate possible
improvements.  One of the first ones to come to mind was to split
`state/api` into its own repo.  That smelled like a heavy lift
especially considering the many interdependencies between `state/api`
and juju proper (though apparently `go` mitigates that somewhat).

Undaunted, I gave it an initial stab (see `Implementation` below).
The bulk of this effort was fixing all the imports, which I ended up
writing a script to solve.  All the tests pass on both sides.  I
wouldn't be comfortable with the split as-is (see `Left to do` below)
but it demonstrated to me that it is possible.

That said, *possible* should not imply *advisable*, and given my
inexperience with the juju project I don't presume to do much more
about splitting out an api repo without feedback.  At the very least
this will be a chance for you to educate me (and presumably others)
about the juju code base. :)  If the code split is a bad idea then
this is your change to officially put the idea to rest.  (It would not
surprise me if I've misunderstood something important here!)

If illumination and a public rejection of the idea are all that come
of this I'll still be satisfied.  However, I'm hopeful that it's a
good enough idea to warrant the effort and that I understand enough
about juju at this point to have at least a vague sense of that. :)

Thoughts?

-eric

++

Proposal
==
* split `github.com/juju/juju/state/api` into its own repo
* place the new api repo at `github.com/juju/api`
* reduce dependencies in the new repo on the code in the main juju repo
* introduce a juju API client interface type
* godocs for the `api` package (should happen regardless)
* more? (see `Open questions below`)

Pros
===
* helps bless the API as a supported first-class feature of juju
* makes the API more discoverable
* encourages a stronger separation between the public API (and
implementation?) and juju proper, including on the server side
* reduces the chance that we inadvertently change/break the public API

Cons

* touches a lot of code
   + churn
   + requires that a lot of (most?) outstanding patches be updated
* risk of introducing new bugs
* effort to make the split
* effort to reduce dependencies in the new repo
   + the API *implementation* is tightly entwined with juju proper?

Open questions
===
Here are some questions that have bearing on this proposal.  Some of
them would directly impact the scope of the proposal.  Others could be
addressed separately (to spread the scope out in manageable chunks).

* is `state/api` just for the state client/RPC, juju state in general,
or juju as a whole?
   From what I understand, it is the middle one (though I originally
thought it was the first).
* is there other public juju API other than the state-related API?
* should the `api` repo contains *all* public-facing juju API

git tutorial

[Eric Snow]

Here's a link to a nice git tutorial:

  https://www.atlassian.com/git/tutorial/

Another nice resource:

  http://git-scm.com/doc

...particularly the getting-started section in the book:

  http://git-scm.com/book/en/Getting-Started-Git-Basics

-eric

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev