Re: apiserver/testing.FakeAuthoriser is not a good mock
On Thu, Jul 31, 2014 at 7:30 AM, David Cheney david.che...@canonical.com wrote: Proposal: remove Authoriser.GetAuthEntity() and replace all calls with Authoriser.GetAuthTag(). I have a branch for this, it's 30 lines as there are only 3 places in the apiserver where we do this and they usually call the .Tag method on the entity they get back from GetAuthEntity. SGTM. 2. This extends from point 1, while the svrRoot derives everything from svrRoot.entity, the FakeAuthoriser allows the caller to pass a unique value for the tag and the entity of the authorisee. This leads to impossible situations where the FakeAuthorizer returns nil for AuthGetEntity but a non nil value for AuthGetTag. Other permutations exist in our tests and are expected by the test logic. Propsal: Once step 1 is fixed the difference between svrRoot and FakeAuthoriser is the former starts from a state.Entity and derives a tag from which other values are derived, the latter skips the initial step and starts from a tag. This work falls out of the solution to steps 1 and 3. 3. The helper methods, AuthMachineAgent(), AuthUnitAgent() on the Authoriser interface are implemented differently in svrRoot and FakeAuthoriser. In tests it is quite common to take the FakeAuthoriser from the test suite, copy it and change some of these values leading to impossible situations, ie, the tag or entity of the FakeAuthoriser pointing to a Unit, but AuthMachineAgent() returning true. Proposal: The simplest solution is to copy the implementation of these helper methods from svrRoot to FakeAuthoriser. A more involved solution would be to factor these methods out to be functions in the apiserver/common package that take an Authorizer. This second step may not pay for itself. I would strongly favour the latter solution. The benefit of having a fake authorizer whose behaviour diverges is that (at least in theory) it allows for comprehensive testing against arbitrary authorizer behaviour; constraining that behaviour so we're only testing realistic situations will be great, but I fear that doing so by copy-pasting code will only lead to more divergences in future. Let's make sure we're using the same logic across the board. Cheers William These steps resolves the majority of the discontinuity between the two implementations and will resolve a set of blocking issues I've hit converting the apiserver and state packages to speak tags natively. Thanks for your time Dave -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: apiserver/testing.FakeAuthoriser is not a good mock
On Thu, Jul 31, 2014 at 5:12 PM, William Reade william.re...@canonical.com wrote: On Thu, Jul 31, 2014 at 7:30 AM, David Cheney david.che...@canonical.com wrote: Proposal: remove Authoriser.GetAuthEntity() and replace all calls with Authoriser.GetAuthTag(). I have a branch for this, it's 30 lines as there are only 3 places in the apiserver where we do this and they usually call the .Tag method on the entity they get back from GetAuthEntity. SGTM. 2. This extends from point 1, while the svrRoot derives everything from svrRoot.entity, the FakeAuthoriser allows the caller to pass a unique value for the tag and the entity of the authorisee. This leads to impossible situations where the FakeAuthorizer returns nil for AuthGetEntity but a non nil value for AuthGetTag. Other permutations exist in our tests and are expected by the test logic. Propsal: Once step 1 is fixed the difference between svrRoot and FakeAuthoriser is the former starts from a state.Entity and derives a tag from which other values are derived, the latter skips the initial step and starts from a tag. This work falls out of the solution to steps 1 and 3. 3. The helper methods, AuthMachineAgent(), AuthUnitAgent() on the Authoriser interface are implemented differently in svrRoot and FakeAuthoriser. In tests it is quite common to take the FakeAuthoriser from the test suite, copy it and change some of these values leading to impossible situations, ie, the tag or entity of the FakeAuthoriser pointing to a Unit, but AuthMachineAgent() returning true. Proposal: The simplest solution is to copy the implementation of these helper methods from svrRoot to FakeAuthoriser. A more involved solution would be to factor these methods out to be functions in the apiserver/common package that take an Authorizer. This second step may not pay for itself. I would strongly favour the latter solution. The benefit of having a fake authorizer whose behaviour diverges is that (at least in theory) it allows for comprehensive testing against arbitrary authorizer behaviour; constraining that behaviour so we're only testing realistic situations will be great, but I fear that doing so by copy-pasting code will only lead to more divergences in future. Let's make sure we're using the same logic across the board. You got it boss. Cheers William These steps resolves the majority of the discontinuity between the two implementations and will resolve a set of blocking issues I've hit converting the apiserver and state packages to speak tags natively. Thanks for your time Dave -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
OCR Updates
William just updated the Juju-core Reviewers spreadsheet to reflect a couple of people leaving. That means that with Mod(N) the N has now changed, so you're expected OCR days are going to be different. For example, Frank and William are on for tomorrow (I was on for tomorrow, but I'm now on for Tuesday), etc. John =:- -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
LXC OS Updates
Hey all! Ian and I were discussing lp:1350493 https://bugs.launchpad.net/juju-core/+bug/1350493 (1.20.x local provider not running apt-get update) and thought it might be good to raise a few ideas here. It is my understanding that currently we do LXC cloning in the interest of saving time -- the thinking being: if I have an image set up, I want to use it as a starting-point when provisioning machines. Pursuant to this performance goal, Juju will not run the OS's upgrades if a clone is performed. Of course, this is what is causing the aforementioned bug. I am doing some work for another issue lp:1322302 https://bugs.launchpad.net/ubuntu/+source/juju-core/+bug/1322302 which involves attempting to speed up provisioning of local providers. One of the things we're introducing are config values (with sane, backwards-compatible defaults) that allow the user to specify whether or not they'd like Juju to refresh the available updates (in dpkg systems, apt-get update), and whether or not they'd like Juju to execute any available updates (in dpkg systems, apt-get upgrade). We'd like to remove the assumption embedded in the code that if we're cloning an LXC container, then we never want to perform OS upgrades. Instead, the logic based on the config variable will take over, and the user can have whatever behavior they desire. Local installations will be snappy, and production installations will be up to date. We feel this empowers both developers and end-users, but wanted to raise this for discussion. Feedback welcome! - Katherine -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Parameter validation Juju
Hey all, What is the general consensus on parameter validation in Juju? As a general practice, I always like to see code that validates parameters passed into methods so that it fails fast, and gives future developers a contract to work off of. Just yesterday I experienced a nil-reference panic which occurred after a lot of heavy operations. Parameter validation would have kicked me out much higher in the stack and given me some indication of what was wrong. Always interested in discussion :) - Katherine -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Question about unprivileged lxc containers
Hello, While working on a bug assignment related to LXC templates, i noticed that the golxc driver is performing the following subprocess invocation on the Create method: ``` lxc-create -n juju-trusty-template -t ubuntu-cloud -f /var/lib/juju/containers/juju-trusty-template/lxc.conf -- --debug --userdata /var/lib/juju/containers/juju-trusty-template/cloud-init --hostid juju-trusty-template -r trusty ``` The problem with this command is that is forcing the usage of /var/lib/juju/containers/juju-trusty-template/lxc.conf as the default and this file doesn't includes any configuration directive regarding to id_maps , which is a requirement to run unprivileged containers, also using the (-f) flag has preference over my locally defined ~/.config/lxc/default.conf. Do we need to add id_maps options for unprivileged containers to golxc? Any other idea? (More information: https://www.stgraber.org/2014/01/17/lxc-1-0-unprivileged-containers/ ) -- Jorge Niedbalski R. Software Sustaining Engineer @ Canonical Canonical Technical Services Engineering Team -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Please use gopkg.in for importing mgo
Trunk is currently broken if building using a clean GOPATH because revision 03e56dcd was recently merged which imports mgo from labix.org instead of gopkg.in. We no longer use mgo from labix.org and godeps no longer installs it from that location. The following import paths should be used instead: gopkg.in/mgo.v2 gopkg.in/mgo.v2/bson gopkg.in/mgo.v2/txn This was perhaps not publicised well enough but the switch was made a couple of weeks ago. Right now juju will only build on machines that incidentally have a labix.org mgo install. If the machine doesn't already have it, godeps won't install it and builds fail. I imagine the problem revision got past the landing bot because our test hosts still have the labix.org mgo installed. If so, this should be cleaned up. I will fix the problem imports in the upgrades package now. - Menno -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
CI tests are installing Juju on the CI machine ...
Creating config file /etc/mercurial/hgrc.d/hgext.rc with new version Setting up zip (3.0-4) ... Setting up juju-core (1.20.1-0ubuntu1~12.04.1~juju1) ... update-alternatives: using /usr/lib/juju-1.20.1/bin/juju to provide /usr/bin/juju (juju) in auto mode. Setting up rsyslog-gnutls (5.8.6-1ubuntu8.6) ... ^ it's leaking in from a dependency, this is risky to say the least. -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Please use gopkg.in for importing mgo
On 1 August 2014 12:09, Ian Booth ian.bo...@canonical.com wrote: The following import paths should be used instead: gopkg.in/mgo.v2 gopkg.in/mgo.v2/bson gopkg.in/mgo.v2/txn This was perhaps not publicised well enough but the switch was made a couple of weeks ago. FWIW, the switch was only done yesterday in order to pick up the new mgo driver version needed to fix a mongo panic, and the changes were only done on Github and not Launchpad, hence the import path change. Not quite. Trunk has been using mgo from gopkg.in since July 28. A similar change was done on the 1.20 branch yesterday so that we could get the recent mgo fix there. -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
