Re: [Launchpad-dev] Permissions for creating private objects
On Tue, 1 May 2012 11:00:26 +0100, Jonathan Lange j...@mumak.net wrote: 2. zope.security has no way to declare that certain values of certain parameters require certain permissions The big example here is 'private=True'. There's no way to declare this that I can see. You *have* to write code[1] that looks like: if private == True: check_permission(SOME_PERMISSION, object) 3. Even were this solved, we need a different permission. Say we had a new_private method that set private to True, then we'd need a permission other than launchpad.Append to govern it. The required permissions are defined a bit like this: $OBJECT.$ATTRIBUTE needs $PERMISSION And people are assigned permissions like this this: $PERSON has $PERMISSION on $OBJECT So for our hypothetical object representing a person's PPAs: PPAs.new needs launchpad.Append PPAs.new_private needs ??? ~jrandom has launchpad.Append on his collection of PPAs, so they can call 'new' and I can't. What permission should 'new_private' require? Say we used launchpad.Moderate or a new permission launchpad.AppendPrivate, then that would address the problem, but it would assume that we only ever have one level of entitlement per collection. I don't have any answers, conclusions or recommendations I'm afraid, so I welcome your thoughts even more than usual. Would this be solved by having multiple collections? Person.PublicPPAs and Person.PrivatePPAs, each of which can have a separate Launchpad.Append permission. A collection that glued them together could perhaps be used to limit the impact of this to users and code that doesn't care about the distinction. This would seem to fall down if you need two orthoganal attributes controlled by elevated permissions though. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Help please - launchpadlib collections
Hi Ian,
Firstly I wanted to note that lp.projects is a heterogenous collection
isn't it? You can access both projects and project groups with it, and
don't they have a different interface?
Secondly I'm wondering why this is trying to be a collection at all. You
wouldn't even want to iterate them, and if they are heterogenous then
putting them in a collection seems odd. Would it be better for them to
be accessed as top level objects (lp.myservice)?
Thanks,
James
On Wed, 11 Apr 2012 10:27:39 +1000, Ian Booth ian.bo...@canonical.com wrote:
Hi
I'd like some input with an issue concerning collections in launchpadlib.
The background is that I want to have launchpadlib recognise a new top
level collection called 'services'. Services are looked up by name. The
collection is heterogeneous in that each named service provides
different capabilities via different exported methods. All services
extend an IService interface and this is how the collection is currently
defined:
class IServiceFactory(Interface):
export_as_webservice_collection(IService)
@operation_parameters(name=TextLine(required=True))
@operation_returns_entry(IService)
@export_read_operation()
@operation_for_version(beta)
def getByName(name):
Lookup a service by name.
I want to allow launchpadlib to be used like so:
lp = Launchpad()
service = lp.services['myservice']
service.foo()
(As an aside, the current syntax required to access a named service is:
# XXX 2012-02-23 wallyworld bug 681767
# Launchpadlib can't do relative url's
service = launchpad.load(
'%s/+services/sharing' % self.launchpad._root_uri)
Even if bug 681767 were fixed, the client would still need to know the
URL to traverse whereas having a named 'services' collection hides this
implementation detail. Perhaps this is ok though?)
Because the services collection is heterogeneous, there needs to be some
way which allows launchpadlib to know what each service instance in the
collection provides. My initial solution was to define a new ServiceSet
in launchpadlib (a solution already used for projects, bugs,
distributions etc):
class ServiceSet(CollectionWithKeyBasedLookup):
A custom subclass capable of service lookup by service name.
def _get_url_from_id(self, key):
Transform a service name into the URL to a service.
return str(
self._root._root_uri.ensureSlash()) +
'+services/' + str(key)
# All services are different so we need to ensure each service
# representation is retrieved when needed.
collection_of = None
This works well and correctly deals with the fact that each named
service instance has different capabilities. However, concerns were
raised that it adds to LOC count and could be done generically without
the helper class.
My understanding is that an alternative solution would require changes
and/or enhancements to the WADL generation rules, and for additional
information to be added to the WADL. At the moment, lazr.restful (which
is used by launchpadlib) doesn't really know what constitutes a top
level collection and uses heuristics to determine this.
eg from ServiceRootResource
# XXX sinzui 2008-09-29 bug=276079:
# Top-level collections need a marker interface
# so that so top-level utilities are explicit.
if (provided.isOrExtends(ICollection)
and ICollection.implementedBy(registration.factory)):
This issue would need to be addressed along with the ability to define
more specifically what's in each collection.
I would love to get input from the smart folks on this list as to what
the best way forward is. I think Gary, Francis, (and Leonard) have
worked on this specific stuff in the past. Do we want to attempt to
tackle the higher level issues highlighted above? Apart from the bug
concerning the definition of top level collections (276079), I couldn't
find a bug specifically related to heterogeneous collections. The LOC
count for an alternative solution would be (far?) greater than what my
solution has, but may allow launchpadlib to in the future handle new
heterogeneous collections without additional code changes. I am wary of
diverting too much time away from the disclosure project for a nice to
have but non-core disclosure item. I see the options as:
1. Adopt my current solution which defines a ServiceSet in launchpadlib
and allows the syntax lp.services['myservice']
2. Fix bug 681767 (not sure of the effort required) and use syntax
lp.load('/+services/myservice')
3. Address the WADL/lazr.restful issues to support syntax
lp.services['myservice'] and make launchpadlib 'futureproof'
Thoughts?
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help :
Re: [Launchpad-dev] announcing pybars - handlebars for Python
On Mon, 20 Feb 2012 17:17:22 +1300, Robert Collins robe...@robertcollins.net wrote: bin/py -m timeit -s 'import pystache' -s 'source = file(/home/robertc/source/launchpad/lp-branches/working/lib/lp/bugs/templates/buglisting.mustache, rb).read().decode(utf8)' -s 'import json' -s 'data = json.loads(file(profdata, rb).read().decode(utf8))' -s 'render = pystache.render' -s 'import pdb' 'render(source, data)' It looks like this is importing pdb, which could be skewing your numbers? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] [Joerg Jaspert] Description-less packages file
Hi, This change will likely affect some people in Ubuntu. We may want to make the change for Ubuntu itself, but first we should make sure that all our systems pulling from Debian still work. Thanks, James ---BeginMessage--- Hello world, I don't know if I really got everyone who should have a copy of this mail in my CC list, so please forward it to wherever you think I am missing. Thanks. I just merged a patch from Ansgar to generate the Packages files without the English description embedded inside them. Instead they are now written into a new file, the English Translation file in main/i18n/Translation-en.bz2. They thus appear alongside all other translated descriptions as just another language. apt co will (or should) just download those Translation files to show the description, as they do already for all other languages. This lets us save quite a bit of space on our mirrors by not repeating them as many times as we have architectures - and also enables non-English-speaking users (and eventually multi-arch enabled APT) to save on download size, as they no longer need to download a language that is of no use to them or is already there. As this is is a larger changeset we did not switch the official dists/ tree directly. Instead we are providing files generated in this way over at [1] for you to test with and report bugs you find in this implementation or in the handling of it in whatever tool breaks down. That location is updated right during dinstall so should always contain current information. Provided there are no showstoppers turning up we intend to switch the official dists/ tree one month from now. [1] http://ftp-master.debian.org/newdists/ -- bye, Joerg It’s not easy to juggle a pregnant wife and a troubled child, but somehow I managed to fit in eight hours of TV a day. pgpclvnzfycSK.pgp Description: PGP signature ---End Message--- ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Should team membership requests expire?
On Wed, 12 Oct 2011 17:07:49 +0700, Jeroen Vermeulen j...@canonical.com wrote: And I wonder: if a team membership request has not been approved in a year, say, doesn't that amount to a denial of the request? Shouldn't we treat it as one? It sounds reasonable to me, provided you can send the requeust again later if it expired. If someone misses the mail and comes back a year and a day later to ask about it not allowing them in the team would be annonying. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Lessons from a weird bug
Hi Brad, Thanks for writing this up. It reminded of another bug to do with validators and dict ordering: https://bugs.launchpad.net/lazr.restful/+bug/561521 I'm not sure if there are changes that could be made to avoid these sorts of problems in future, but in that bug Michael makes a statement that someone may be able to confirm or deny, which could eliminate these problems (at the risk of introducing them elsewhere of course.) Thanks, James On Fri, 30 Sep 2011 16:41:00 -0400, Brad Crittenden brad.critten...@canonical.com wrote: Hi, I just spent a long time trying to figure out a very frustrating bug so I thought I'd share, even though I'm a bit dubious of any general lessons. Some rather innocuous changes resulted in test failures dealing with conjoined bugtasks. Poring over diffs of the changes didn't reveal anything obvious that could be causing the problem. It turns out that BugTask attributes use Storm validators extensively. (Really these are transformers as they don't set errors, return True or False, or throw exceptions. Instead they return transformed values.) Further these validators were implicitly depending on the order of attributes are set when a new BugTask is constructed. One section of 'validate_conjoined_attribute' has this snippet: # If this bugtask has no bug yet, then we are probably being # instantiated. if self.bug is None: return value BugTask has never been updated to a Storm class, and is instantiated in BugTaskSet 'createTask' with: bugtask = BugTask(**create_params) Those kwargs eventually get passed down to the Storm SQLObjectBase class and are used like: def set(self, **kwargs): for attr, value in kwargs.iteritems(): setattr(self, attr, value) The problem is the order the attributes are set depends on the ordering returned by iteritems(), which for a dict is arbitrary and should never be relied upon. However the code depended upon self.bug being None, i.e. not yet set, as a check for whether a new object was being constructed or a mature object was having an attribute changed. The code I was working on simply changed the name of one attribute from 'status' to '_status' and this upset the order of the attributes returned by .iteritems(). Before my change 'bug' was consistently being set as the last attribute. After my change it was set before '_status' and the behavior of the validator changed. The minimal fix is easy: replace the test for 'self.bug is None' with 'self._SO_creating' an attribute SQLObjectBase sets during object construction. In retrospect the comment # If this bugtask has no bug yet, then we are probably being # instantiated. should have been a red flag to me trying to debug my code, to the reviewer who originally looked at the branch where it was introduced, and to the developer who wrote it. Probably is a weasel-word that shouldn't be used to describe code behavior. Thanks to Danilo helping me to figure out the change in behavior. --bac ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Failed attempt to contribute to lazr.restfulclient
On Tue, 13 Sep 2011 11:43:23 +0100, Jonathan Lange j...@canonical.com wrote: Once I figured out how to get past that, and buildout ran successfully, I tried running the tests. They failed[3]. I believe there are some tests that are only supposed to be run within launchpad's test harness. Perhaps I'm misremembering and it's launchpadlib that is in that state, but I remember not being able to get a clean test run on one of them without doing it in that way. Such a dependency certainly makes it harder to contribute, and makes it less likely that lazr.restfulclient will be reused by another project. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Removing ubuntu-branches and ubuntu-techboard celebrities
On Mon, 30 May 2011 17:28:14 -0400, Francis J. Lacoste francis.laco...@canonical.com wrote: For official package branch, this change would affect James Westby as he's the only member of ubuntu branches that is not part of the technical board. Maybe what's needed is to fix https://bugs.launchpad.net/launchpad/+bug/365098 and allow anyone who can upload the package to set the official link. The question then becomes, do we need to fix that bug before proceeding (in other word, would there be unwanted fall-outs from restricting this to the current distribution owners.) Yes, that bug needs to be fixed before removing the celebrity, or some other way of keeping the importer working needs to be found. The importer currently runs with my credentials in order to be able to do all of the things that it needs to do (handily I am a core-dev and a member of ~ubuntu-branches, which equates to full permission to do everything that it needs to do.) We want to change this anyway, but haven't ever got to doing it. The importer frequently sets official branches (e.g. when a new package is uploaded to Ubuntu,) and so this needs to continue to work, but I don't feel strongly about the exact mechanics. To put it another way the importer needs to have * ability to push to every official branch for Debian and Ubuntu (to keep them up to date) * ability to set the official branches for Debian and Ubuntu (to add them when something new appears, e.g. new package, first security update for a package to a particular release) and it uses all of those permissions regularly, so any temporary removal would disrupt its operation. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] simplifying the bug model -a little-
On Fri, 15 Apr 2011 09:55:02 +0100, Jonathan Lange j...@canonical.com wrote: Ubuntu would be the biggest hit by this change, so it's worth talking directly to some key people there. I think Kate Stewart (the RM) and key people in QA (e.g. Brian Murray, Jeremy Foshee) would be a good start. This would be important to do, as currently all combinations of (series targeted) and (milestoned) are used in release planning. Taking away some of those combinations will at the very least require changing the release processes/documentation, and may lead to a change to the propsal to accomodate those needs. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] brainstorm: cheaper API collection iteration
On Fri, 25 Mar 2011 15:11:19 +1300, Robert Collins robert.coll...@canonical.com wrote: But how do we teach launchpadlib to do this? And how do we encode this in the wadl? My understanding, may be way off. Pages in collections are just links, so you can generate what you like. You can generate the bare link to the collection to get the first page, and then when that page is returned it would say next_page_link: https://api.launchpad.net/devel/some_collection?start_key=endkey_of_this_collection then when the next page is needed launchpadlib simply hits that URL. Therefore I think this would be fairly straightforward to change in the current webservice. It also looks as though just changing lazr.batchnavigator will get most of the way there, so it may be that fixing the web UI fixes the API too. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Fwd: Kernel team scripts ...
On Thu, 17 Mar 2011 12:33:34 +, Jonathan Lange j...@canonical.com wrote: If you are interested in bug workflow, or in the Launchpad API, you should take a look at these scripts. They represent things that people need to do with Launchpad, and it's always good to know that. Also, there are probably ways to make them faster that are obvious to folk familiar with Launchpad's internals. If you make the scripts faster, you make the Kernel team more productive and thus make my network driver less likely to crash. I can see two things that would help, one from LP, and one from lpltk: 1) Adding a CVE search API call (substring match on display_name) - the cve script has a part that is currently O(num CVEs) 2) Don't save the tag list after every tag addition, batch them until the end. - This would save num_tags-1 round trips in each script Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Recipe should be out of beta
On Wed, 2 Mar 2011 15:10:08 +0200, Francis J. Lacoste francis.laco...@canonical.com wrote: Hi Tim, Since your team has now completed recipe, shouldn't we remove the beta flag and make them available to everyone? We should probably blog about this new feature also. May I humbly suggest a series of blog posts? It's a great new feature that should be introduced and explained in some depth so that it gets the uptake that it deserves. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [Francis J. Lacoste] Announcing Launchpad Squads
Sorry, I meant to forward this to linaro-dev. James On Mon, 31 Jan 2011 15:50:00 -0500, James Westby james.wes...@linaro.org wrote: Hi, Since we use Launchpad so much within Linaro this may be of interest to some. I am the Launchpad stakeholder representative for Linaro, so if you wish to request any features or escalate any bugs then please feel free to talk to me. Thanks, James We've made some changes to how we organise the Launchpad team! We're no longer divided into application-based teams (Bugs, Code, Foundations, Registry, Soyuz and Translations). Instead, we now have five cross-domain engineering squads: three focused on features and two on maintenance. How does this affect you? -- When you want to speak to someone about a specific part of Launchpad, whether for help, to escalate something or about an operational issue, things have changed a little. Specifically: * For help and any other operational issues related to Launchpad, you should either send an email to the launchpad-users mailing list[1], or file a question on the Launchpad project[2]. (Reminder: these forums are public.) * If your request would benefit from interactive discussions, drop by in #launchpad on Freenode where one of the people working on the maintenance squads will be able to help you. If your request isn't suitable for public consumption, you can reach the same people on the #launchpad-ops channel on the private Canonical IRC server. * To prioritize feature requests and to escalate other bugs, you should contact your stakeholders representative. See [3] for the list of people. * You can also always bounce ideas to the developers list [4]. [1] https://launchpad.net/~launchpad-users [2] https://answers.launchpad.net/launchpad/+addquestion [3] https://wiki.canonical.com/Launchpad/Strategy/StakeholderMeeting [4] https://lists.ubuntu.com/mailman/listinfo/canonical-launchpad Short-term, we also expect some churn as people are exposed to areas they weren't used to before. But down-the-line, we'll have much more distributed knowledge coverage across the whole application. How the squads work --- The squads will alternate between development project and maintenance modes. The three development squads will work on longer term projects, usually resulting in new functionality. Once such a squad has finished a project they'll swap places with one of the two maintenance squads. Bugs, operational issues and so on will be taken care of by the maintenance squads. Deciding which development projects to take on will remain the responsibility of our strategist (Jonathan Lange) in collaboration with the Launchapd Stakeholders group. You can find a list of who is in each squad on our dev wiki [5]. [5] https://dev.launchpad.net/Squads Why are we doing this? -- Our app-based teams served us well, but were becoming a liability: * Many parts of Launchapd, such as Blueprint, didn't have a dedicated team and were basically unmaintained. * Each team was responsible for both new features within their app as well as maintenance. That slowed both of these. * Projects that required cross-app integration suffered from hand-off and coordination problem. With this new structure, we expect to see: * Better cohesion across the application, as one squad will be responsible for the implementation of new aspects across the whole application. * Reduced cycle-time both for bug fixes and for new features as the context-switching will be removed. It will be my pleasure to answer any questions you might have regarding this reorg. -- Francis J. Lacoste francis.laco...@canonical.com Non-text part: application/pgp-signature -- ubuntu-platform mailing list ubuntu-platf...@lists.canonical.com https://lists.canonical.com/mailman/listinfo/ubuntu-platform ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] notes towards async api clients
On Tue, 25 Jan 2011 12:10:43 +1100, Martin Pool m...@canonical.com wrote:
I had a bit of a go at this over the weekend. It is gratifyingly fast
compared to what I expect to see with launchpadlib clients, just
through doing fewer requests and not unnecessarily blocking on them.
I like that a lot.
The code is in http://launchpad.net/wrested and lp:wrested.
For instance:
./wrestler.py https://api.launchpad.net/devel/bugs/1
To be fair I believe lplib would perform approximately as well if you
just did lp.load('https://api.launchpad.net/devel/bugs/1').
Thanks,
James
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] notes towards async api clients
On Tue, 25 Jan 2011 12:10:43 +1100, Martin Pool m...@canonical.com wrote: So it's quite fun and I intend to continue. If someone wants to talk about it I'd like that too. Thanks for this, it's good to see things progressing on this front. I think that this would make a great bottom layer for a twisted LP client library. I think that if we were to merge this and txrestfulclient then we would have the best of both worlds. I think this is a better building-block way to go, but most developers won't want to hardcode URLs everywhere, so putting it together with the higher-level stuff in txrestfulclient (which deals with WADL etc.) would allow developers to work at the level that they want. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Webservice performance
Hi, First off, let me say thanks for the webservice. Having it there has enabled a lot of things, and it is very much a positive in my eyes. However, I have a short story to tell you. Ubuntu has used the work items tracker for a few releases now. This scrapes the blueprints tracker, and pulls out workitems that are in the whiteboards, as well as bug links etc. and graphs them over time. http://people.canonical.com/~platform/workitems/natty/all-ubuntu-11.04-beta.html Due to the fact that blueprints weren't exported on the API, it has always done this by screenscraping, using a set of regexes to pull out what it wants from each page. This has made it hard to do certain things (such as look at dependencies, which are only represented in graphical form on the blueprint page), so recently Guilherme and I exported an API for blueprints. I then ported the work items tracker to use this API, which was quick to do. However, when testing it we noticed that it was about 10% slower than the old code. This suprised me, as I was expecting a performance increase, due to the batching that the webservice gives you when dealing with collections, and possible small increases from the appservers having to do less work (rendering HTML etc) and not using regexes as much due to having attributes already defined. When looking in to why, I turned on tracing of the HTTP requests that were being made, and it was quickly apparent what the problem was. The code wants to associate blueprints with people, so for each of (drafter, assignee, approver) it gets their user id. drafter = bp.drafter.name assignee = bp.assignee.name approver = bp.approver.name What happens here is that there is an extra round trip on each of these lines, to follow the link in the blueprint representation, get the person representation, and to pull out their name. These round trips quickly negate the benefit of the batching, as rather than 1 round trip per blueprint that the old code does, we now do 3 (plus one for every page of blueprints). Even with caching, the code still does the round trips to ensure that the cache is up to date (when we don't really care in this case, we're not realtime, and changes while the code is running can actually be harmful as it is tricky to code defensively against that) I was able to confirm this by writing a hacky function that given a link to a person tells you there user id by parsing the URL, and the code then showed the expected speedup over the old code. While this isn't a huge issue, I'm disappointed that the API is slower than screen scraping, and it isn't much of an advert for the API. Please do something about this. Having a much better performing API would open up the possiblilities of a whole new swathe of applications using it, much like adding the API in the first place did. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Fwd: failure
On Wed, 1 Dec 2010 14:45:11 +1100, Martin Pool m...@canonical.com wrote: james_w said on irc that an error like this http://paste.ubuntu.com/538491/ was fixed by installing libpq-dev. I was remiss in not pasting the full output of make. Confusingly buildout will intersperse normal progress messages in between error messages, in such a fashion that time seems not to be flowing continuously. This led me to miss the compile error that was fixed by installing that package. The lesson: always read further back in buildout failure output than you think that you need to. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] pushing db-devel branches *is* slower, we do need a cronjob fixing things up.
On Sun, 28 Nov 2010 07:01:47 +1300, Robert Collins robe...@robertcollins.net
wrote:
Something like this, running as launchpad-pqm - I'm on leave, so I
leave this up to whomever gets annoyed at the push times first.
import bzrlib.plugin
bzrlib.plugin.load_plugins()
import bzrlib.repository
with bzrlib.initialize():
db = bzrlib.repository.Repository.open('lp:launchpad/db-devel')
devel = bzrlib.repository.Repository.open('lp:launchpad')
devel.fetch(db)
Does PQM handle the target branch being write-locked gracefully?
Thanks,
James
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Strange error from web service
On Fri, 19 Nov 2010 09:36:34 +, Jonathan Lange j...@canonical.com wrote: Hello, I just got this in my cron mail. I've never seen this error before. Is this a bug in launchpadlib, or is this something that launchpadlib scripts should be written defensively against? My guess is that the response was truncated. This is not unusual in launchpadlib interactions, but I don't remember seeing that particular error previously. I would recommend defensive coding, but unfortunately between each of the different layers involved there are many exceptions to catch. lazr.restfulclient now has some retry support, so you may also want to contribute a branch to that to also retry on this error. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] UDS session?
On Mon, 18 Oct 2010 09:41:20 +1300, Michael Hudson michael.hud...@canonical.com wrote: As someone who spans the worlds a bit, I'd also be happy to be there, but am probably too familiar with bzr launchpad to actually be able to run the session :-) It sounds like we have enough people to run a session, so I will register it. I don't think we are looking for anything particularly specific. When I ran one of these a couple of months ago I gave a quick demonstration of the bug tracker, and then bzr and merge proposals, as I think that's what most people will interact with most frequently. Blueprints are something they will work with as well, so maybe briefly covering that would be good. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] stories for external test helpers (librarian, memcache, etc)
On Wed, 29 Sep 2010 18:50:35 +0700, Stuart Bishop stuart.bis...@canonical.com wrote: On Wed, Sep 29, 2010 at 5:54 PM, Julian Edwards julian.edwa...@canonical.com wrote: * Rabbit MQ (why is this installed when we're not even using it yet?) * Memcached You can turn off memcached - LP starts this as necessary. And RabbitMQ since no code is using it yet (waiting on staging install). I think the 'preferred' method is still the incredibly ugly: update-rc.d -f memcached remove update-rc.d memcached stop 20 2 3 4 5 . (or is there some way of making service(8) do the right thing? Nothing in the manpage) This is not the preferred way. Unfortunately there isn't yet a way to consistently do this across services, but there is a convention to have an /etc/default/name file where it can be configured. Therefore to stop memcached running you can edit /etc/default/memcached and change ENABLE_MEMCACHED to no. RabbitMQ unfortunately doesn't have as nice a way of doing it, but you could set DAEMON=true in /etc/default/rabbitmq if you wanted to do it. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Results: derived distro mock-up testing round 2
On Wed, 1 Sep 2010 08:59:59 +0200, Michael Nelson michael.nel...@canonical.com wrote: I've not heard it discussed... but that could be a great idea. I think the soyuz diff generation was assumed because its what has always been used for package diffs (pre-dating package branches)... and I'd wrongly assumed we'd need diffs between the lucid and derilucid version). Note that there may be some tweaks required to loggerhead to get the features we need. The unknown factors for me so far are: 1) When a new version of a src package is published, are we guaranteed that the version is already committed to a bzr branch (for derived distros also) (for build from branch of course it is, but normal uploads?) Guaranteed? No. The importer is quicker than the publisher, so it should generally be true for distributions that it imports. 2) If I have a source publishing record in Launchpad, can I get directly to the bzr branch and rev number even if it hasn't been built via a recipe but is a standard upload (I'll follow that up if no-one beats me to it). No, Launchpad has no facility to store that data as far as I know. It's a simple mapping from (package, series, version) to (branch, tag) though. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Results: derived distro mock-up testing round 2
On Tue, 31 Aug 2010 17:16:18 +0200, Michael Nelson michael.nel...@canonical.com wrote: Thanks Matthew for transcribing and summarising all the info - it's invaluable. And thanks Peter, Colin and Loïc for taking the time to give such feedback. and thank you for these great notes. Consider which diffs are useful and should be available This is hard, at least given the column restriction. That is, giving the different types of diffs an appropriate textual link in such a small space. That said, this could be another example of why a drop-down more-info section for each row could be very useful. This would allow us to present the various diffs without the restriction of the column width. Regarding the auto-generation of diffs, Julian's main concern was that we'll fill the librarian unnecessarily, but based on Colin's feedback, it sounds like we could be sensible about the diff generation. Would the following work? 1) For packages that have changed in derilucid, but not lucid (ie. the lucid version is the last common version), we list only one diff (last-common-version to derilucid version), and we don't auto-generate as its not necessary for making a decision. (?) Sounds sensible. 2) For packages that have changed in lucid but not in derilucid (ie. the derilucid version is the last common version), we list only one diff (last-common-version to lucid version), and we *do* auto-generate it, as it will be required to make the decision to sync. Agreed. 3) For packages that have diverged (changes in both), we list two diffs: * last-common-version to derilucid version * last-common-version to lucid version and we auto-generate both so that Obviously we need to ensure that we remove references to diffs once package differences are resolved (so the librarian isn't filled up) Colin mentioned: You never really want a diff from the current version in Lucid to the current version in Derilucid unless one of those is also the common version. Otherwise you have this unreadable diff.. I'd assumed this diff would be necessary in the following scenario: 1) New version uploaded to Derilucid 2) Derilucid changes merged upstream and uploaded, resulting in a new Lucid version, at this point, although the currently published version numbers are different, the diff of current version in Lucid to current version in Derilucid would show you that you can now sync wouldn't it? (ie. there would just be small changelog differences?). Let me know anyway. What we do want to see often is the result of merging one in to the other (usually lucid to derilucid). Has there been any discussion of not using soyuz's diff generations stuff and instead linking out to bazaar.launchpad.net? It's not the most stable thing in Launchpad, but it does allow for dynamic diff generation, and so everything can be supplied without delay, without affecting the librarian at all. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Implementation plan for derived distros UI
On Tue, 24 Aug 2010 12:23:16 +0200, Michael Nelson michael.nel...@canonical.com wrote: Great, so we can instead use a status field with values like: NEEDS_ATTENTION, CURRENTLY_IGNORED, ALWAYS_IGNORED. Would that work for you? Yes. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Implementation plan for derived distros UI
On Tue, 24 Aug 2010 12:33:21 +0200, Michael Nelson michael.nel...@canonical.com wrote: On a related note, I'm assuming that comments added for a row (in the UI), such as We're waiting for version 1.2.5b would be comments per source package name rather than per version... is that correct? (ie. when Foo 1.2.1 / Foo 1.2.0 is already displayed in the UI, and Foo 1.2.2 is uploaded to the parent, would you expect to see the existing row updated to Foo 1.2.2 / Foo 1.2.0 or the existing row obsoleted (with its comments) and a new record for Foo 1.2.2 / Foo 1.2.0 created? I'm assuming the former). Both could be true. For prior art we have merge-o-matic. That persists the comments, until the merge disappears from the list for some reason. Adding the option to state whether the comment applies to one version or more would make for a cluttered UI I think. I would tend towards keeping the comments around, but note which version they were first added too. That will avoid confusion if people say this version. We may want to have similar rules to merge-o-matic in terms of removing comments when a merge is dealt with, unless the comment was added when a package was marked ignore always or something. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Implementation plan for derived distros UI
On Mon, 23 Aug 2010 09:02:57 +0200, Michael Nelson michael.nel...@canonical.com wrote: Sure - we can store whatever is needed for the workflow... I must have missed the use-case for an 'ignored_always'. Can you explain it to me? (are you expecting the normal 'ignored' boolean to be reset automatically when a new version is uploaded, where as this one would never be reset automatically?) Colin pointed out that both are useful. We currently have 'ignored_always' as the sync-blacklist.txt file. We would like 'ignored' to save duplicated work if a particular version doesn't need to be merged, but future ones will. AFAIK, it is a cost consideration, but IMO to consider that cost we'd want to take into account whether the diff is always going to be consulted (or if in 90% etc.). If it was nearly always used, or if we new it was always used in certain situations, it would simplify the workflow to just make sure it was there and didn't need to be requested. I'm not sure if it was already discussed before I was involved... Julian? StevenK? Having the button there is fine. Could we measure usage to see if we can find conditions in which pregenerating would be useful? My only concern is that jobs on request aren't very nice for writing API clients against. We have to trigger the job, then poll for the presence of the result. If failure states aren't exposed then we need a timeout too, all of which does not aid usability. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Implementation plan for derived distros UI
On Mon, 23 Aug 2010 11:00:57 +0100, Julian Edwards julian.edwa...@canonical.com wrote: It's also slightly future proof in that we might want to show component, pocket etc on that page at some point. This reminded me that once this is available you will have requests for filtering by packageset, and possibly other requirements. Do you have an idea of how you would want to accomplish that? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Making migrating to storm easier.
On Sun, 22 Aug 2010 20:17:40 +1200, Robert Collins robert.coll...@canonical.com wrote: Amongst other migration issues, __nonzero__ on SQLObjectResultSet makes converting SQLObject to Storm code tricky, because its hard to grep for, silently fails (bool(now-storm-resultset) - always true). Just to note that since the last storm release you can now use .is_empty() on either result set, with the same behaviour. I would suggest that no new code does bool(SQLObjectResultSet), and people convert any instances that they see to .is_empty(). That will allow a smoother migration, without having to port every use by changing the base class. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] The start of the Matchers invasion
On Wed, 04 Aug 2010 16:30:13 -0400, James Westby james.wes...@canonical.com
wrote:
Hi,
testtools and assertThat have been available for a while now, but I
just landed what I think are the first uses of them in the launchpad
tree.
Yes, these are fairly similar to custom assertion methods, and while
they may seem a little more heavyweight, they do have some advantages:
* They can be composed in a more pleasant fashion.
* They can be used outside of testing.
* They are easier to test.
* They can provide better failure messages.
* They are outside the TestCase class, so you don't have to load your
base TestCase class, or have many different ones and then have
difficulty when you want to combine two or more.
Oh, I forgot one of the coolest things about matchers.
Rob just landed this matcher:
class HasQueryCount(Matcher):
Adapt a Binary Matcher to the query count on a QueryCollector.
If there is a mismatch, the queries from the collector are provided
as a
test attachment.
def __init__(self, count_matcher):
Create a HasQueryCount that will match using
count_matcher.
self.count_matcher = count_matcher
def __str__(self):
return HasQueryCount(%s) % self.count_matcher
def match(self, something):
mismatch = self.count_matcher.match(something.count)
if mismatch is None:
return None
return _MismatchedQueryCount(mismatch, something)
class _MismatchedQueryCount(Mismatch):
The Mismatch for a HasQueryCount matcher.
def __init__(self, mismatch, query_collector):
self.count_mismatch = mismatch
self.query_collector = query_collector
def describe(self):
return queries do not match: %s %
(self.count_mismatch.describe(),)
def get_details(self):
result = []
for query in self.query_collector.queries:
result.append(unicode(query).encode('utf8'))
return {'queries': Content(ContentType('text', 'plain',
{'charset': 'utf8'}), lambda:['\n'.join(result)])}
The interesting thing is the last method, Mismatch.get_details().
If this mismatch causes an assertion failure then testtools will call
that method on it, and if it returns anything it will automatically call
addDetail() on the TestCase, to add the information to the test result.
What this means is that you can write a small bit of code on your
Mismatches that will ensure that everytime a test fails because of the
mismatch, the user can access certain information. Here it is listing
all of the queries, but it could tell you the state of all packages in
an archive, the state of a bug, the content of a HTTP response, whatever
you like.
This means that you don't have to dump everything in the assertion
message, making them easier to read, while still allowing developers to
access information about the state of the system at the time of failure.
Thanks,
James
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] warning: we will soon have much noise in the test results...
On Thu, 05 Aug 2010 10:14:03 -0400, Aaron Bentley aa...@canonical.com wrote: It's still nagging us to fix something we haven't agreed needs fixing, though. Personally, I think we probably should use proxified objects in tests, but I don't think nagging is the right way to go about it. Agreed. Certainly if we don't want to do this we shouldn't nag, and the current output isn't helpful either way. However, assuming that we want proxied object in tests... My solution is here: https://code.edge.launchpad.net/~abentley/launchpad/no-proxy-warnings/+merge/31781 Those who want the warnings can have them, and everyone else doesn't have to put up with them. this is a fine stopgap solution if we have people using the environment variable. jml's solution is better once we have fixed all known cases though, as it doesn't require you to have an environment variable set to know that you are adding a new problem. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Archive deletion strategy
On Thu, 5 Aug 2010 15:30:09 +0100, Julian Edwards julian.edwa...@canonical.com wrote: On Thursday 05 August 2010 15:06:35 James Westby wrote: On Thu, 5 Aug 2010 11:33:40 +0100, Julian Edwards julian.edwa...@canonical.com wrote: On Wednesday 04 August 2010 21:16:33 James Westby wrote: #bugpackageinfestation - SPR ??? Only main archives have bugs so it should not matter for COPY/PPA archives. Deryck will have to fix this whenever they do PPA bugs :) Well, I am adding code to delete PRIMARY archives, so I have to consider what to do. Why's that? It's out of scope I thought - aren't you just deleting COPY/PPA archives? Then I misunderstood this: On Tuesday 03 August 2010 13:34:30 James Westby wrote: Just to be clear, you are advocating deleting every trace of an archive (except records referenced elsewhere), as soon as the publisher runs after you delete an archive, for all archive types? For all archive types where we support deletion, yes. This is the only way that people can re-use the same name for PPAs after they delete them, unless they resurrect the same PPA again and are aware of its upload history. where in messages earlier in the thread you hadn't corrected me when I said that PARTNER/PRIMARY/DEBUG were all deleteable. Should the user be told on +delete that they can't delete as they have outstanding builds? I think that would be bad, as they won't care, though we could warn them, as they may not know. I think it would be better for the publisher to leave archives that are DELETING in that state if they have outstanding builds and come back to them next time. Possibly, although I'm worried that will slow down the publisher with the extra work. Archive deletion will be fairly infrequent, such that there will only be a few archive deletion requests at most outstanding at any one time. For each there will be a query as to whether it has any builds outstanding? Is that query going to take a long time? Other references that I missed before: * packagecopyrequest - already CASCADE I guess we need to check to see if there's a job active for the PCR, and either kill it or prevent deletion. Well, I'd just like to kill packagecopyrequest, but I'm hesitant to make this work depend on that. As for that particular case, there is a race between +delete and the publisher running. Therefore we make the publisher skip and try again next time, and have +delete remove any packagecopyrequest to prevent any starting in the meantime, to reduce the load. I'm not sure how we would kill a running job? * archivejob - should CASCADE? Yes. This is basically the same issue? Aaron, what happens on Branch.delete() with BranchJobs? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Archive deletion strategy
On Thu, 05 Aug 2010 10:38:53 -0400, Aaron Bentley aa...@canonical.com wrote: On 08/05/2010 10:30 AM, Julian Edwards wrote: On Thursday 05 August 2010 15:06:35 James Westby wrote: There is also SourcePackageRecipeBuild that I missed before. This is CASCADE on Archive, but there are several references to it that need to be taken care of. - SourcePackageRecipeBuildUpload - CASCADE? Not familiar to me. May not actually be used outside of archiveuploader/tests/test_recipeuploads.py Yeah, I'm having trouble finding reference to it. - SPR - already NULLable, so again it's just that we could have an SPR that we don't know the origin of any more. - SourcePackageRecipeBuildJob - prevent deletion? I would probably warn, the way we do on branch deletion, and then delete if the user decided to proceed. I assumed this was parallel to PackageBuild that Julian was saying should prevent deletion. Can we go ahead and just delete SourcePackageRecipeBuildJob? - SourcePackageRecipeData - CASCADE I'm not as familiar with that data model. Aaron? * SourcePackageRecipe.daily_build_archive - already NULLable. Again a warning here would be appropriate when deleting. Aaron would have to comment there. I would NULL the field, but that's conservative. It's quite likely the user has no use for the recipe if they're deleting the archive that was its target. My instinct would be to be conservative as well. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Archive deletion strategy
On Thu, 5 Aug 2010 16:22:17 +0100, Julian Edwards julian.edwa...@canonical.com wrote: It seems odd that someone would initiate one of these and then immediately want to delete the archive. It's most likely to be a mistaken deletion surely? In which case it would make more sense to refuse the deletion. If it becomes a problem later then we can do the extra work to kill the Job, but I'd cover the simple cases first to reduce the work here. That's true. It may be that they try to delete to undo a mistaken copy request, but I assume that disable would work there? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Archive deletion strategy - nulling upload_archive
On Thu, 05 Aug 2010 20:13:12 +0100, Max Bowsher m...@f2s.com wrote: OOI, what's the UI going to look here? For example +sourcepub/id/+listing-archive-extra = Publishing details *Published* on -mm-dd *Copied from* an archive which has been deleted Changelog foo (1.0-1) lucid; urgency=low = That's what I was going to go for if we delete the archives. The loss of information for copied packages troubles me a bit, but I can see there's no nice answer here - people will hate it if copied builds block PPA deletion. I think the information loss won't be too bad. You will still be able to get to the build etc., you just won't know the name of the archive where it came from. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Archive deletion strategy
On Tue, 3 Aug 2010 11:08:03 +0100, Julian Edwards julian.edwa...@canonical.com wrote: I think that's pretty much it, although we need to examine exactly which database rows need to be deleted and under what conditions. I've added some ON DELETE CASCADEs in the past where it's a no brainer but it won't delete everything because of the multiple publication referencing package files issue. Ok, I've been through the schema and I think this is the set of tables chained off archive: # Archive - signing_key - KEEP #signing_key (GPGKey) - ? # SPPH - Archive and SPR - DELETE #SPR - Archive - DELETE IF ONLY REFERENCED BY SPPH AND # BUILD ATTACHED TO THIS ARCHIVE. what # about packageuploadsource? #bugpackageinfestation - SPR ??? #SPRF - SPR - CASCADE # BPPH - Archive and BPR - DELETE #BPR - build (delete the build and it will cascade) - DELETE # IF ONLY REFERENCED BY # BPPH AND BUILD ATTACHED # TO THIS ARCHIVE. #BPF - BPR - CASCADE # archivearch - Archive - DELETE # archiveauthtoken - Archive - DELETE # archivedependency - Archive (twice) - DELETE WHEN archive, ? # when dependency # archivepermission - Archive - DELETE # archivesubscriber - Archive - DELETE # build - Archive, SPR #buildqueue - build # distributionsourcepackagecache - Archive - DELETE # distroseriespackagecache - Archive - DELETE # packageupload - Archive - WHO KNOWS? #packageuploadcustom - packageupload #packageuploadbuild - packageupload, build #packageuploadsource - packageupload, SPR Firstly we have to keep archive, as you can do a build targetting your PPA, and then copy that to another PPA. We won't delete those builds, so we either have to remove the NOT NULL on the build-archive reference, or keep the Archive around. Either way we need to test what happens if you go to a build page where the build references a deleted archive. Archive references signing key, we could clean this up, or leave it around such that a resurrected archive would get the same key. I'll skip to some easy ones. archivearch can be deleted, we don't need to record which architectures the archive supports. archiveauthtoken, archivepermission and archivesubscriber can all be deleted. distributionsourcepackagecace and distroseriespackagecache just cache data for the archive, so they can be deleted. We want to remove all PublishingHistory records that reference the archive, as they are Archive specific. These reference SPR and BPR which can be referred to from more than one PublishingHistory, and also from multiple builds. Therefore we would check if all the references were ones we were about to delete, and if so remove them too. I'm not sure whether it would cause problems to delete buildqueue entries for the builds, or whether they should be left dangling. This is not a particularly easy thing to test either. Each of SPR and BPR have file tables associated with them, and they should be set to CASCADE, which will clean them up for us. bugpackageinfestation also references SPR: I'm not sure what it is, but my guess is that CASCADE is appropriate there too. I'm not sure what to do with packageupload, but my guess is that deleting them all is appropriate. archivedependency is a little tricky as it has two FKs to archive. Where the dependency is specifying what the archive being deleted depends on it should be deleted, but what about where the archive is depended on by another? Given that we are removing all the packages anyway, I think it makes sense to delete them too, but it may lead to some surprising behaviour for people. I'm still not entirely sure what the logic is for detecting which PackageReleases and Builds can be deleted, but that's what tests are for, right? Any suggestions are welcome. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] The start of the Matchers invasion
Hi, testtools and assertThat have been available for a while now, but I just landed what I think are the first uses of them in the launchpad tree. For those that know what they are: lp.testing.matchers has a couple of basic matchers, please use and extend. I would suggest lp.app.testing.matchers or similar for domain specific ones so that the file doesn't grow too large. You will find tests in lp.testing.tests.test_matchers, please also add tests there (or in equivalent places in your app) for matchers that you add. The current users of the matchers are in lp.testing.tests.test_factory. For those that aren't familiar with matchers: testtools (which the Launchpad TestCase classes are derived from) provides the basic infrastructure for matchers. http://bazaar.launchpad.net/~testtools-dev/testtools/trunk/annotate/head:/MANUAL#L114 The use for them is self.assertThat(something, SomeMatcher()) such as self.assertThat(an_object, Provides(IAnInterface)) The matchers that you pass have a small interface, in testtools.matchers, where they define __str__ to explain what they will match, and match(matchee) to do the matching. Their constructors can take whatever they like, which will normally be parameters to influence the expected value. If the value passed to match() matches, then None is returned, otherwise an instance of a Mismatch is returned. These have an even simpler interface, just defining describe() to explain what went wrong. assertThat() deals with these objects for you, so you just pass in the value and the matcher, and it will do the check and then print out an explanation if needed. Yes, these are fairly similar to custom assertion methods, and while they may seem a little more heavyweight, they do have some advantages: * They can be composed in a more pleasant fashion. * They can be used outside of testing. * They are easier to test. * They can provide better failure messages. * They are outside the TestCase class, so you don't have to load your base TestCase class, or have many different ones and then have difficulty when you want to combine two or more. There's no requirement to use them, but they are there if you want to make use of them, see above for where to find the existing ones and to add more. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] warning: we will soon have much noise in the test results...
On Wed, 04 Aug 2010 16:25:31 -0400, Curtis Hovey curtis.ho...@canonical.com wrote: I landed a few changes because of this. I think everyone should be aware of two discoveries that concerned me. 1. The while looking at one report on a naked object I saw a test description disagreed with a test result. I was really puzzled. Really, because I wrote all the code and I knew re-reading it that it was wrong. There was a typo in the attr assignment. Login with the wrapped object showed that that attr did not exist. Using the real attribute did make the test work as I expected: items orders from 'hot' to 'cold' 2. While wrapping the milestone I saw factory.makeMilestone() rarely made a valid milestone. A milestone must have a product or distro series. No test with a distro milestone could render a milestone, release, or distribution view. We did not have a test that exercised these conditions with the factory made milestone, but lp engineers have tried to help users be creating milestones using SQL, and these same milestones definitely cause exceptions last year. 3. While adding tests to avoid regressions for the ones I was fixing I found 5 classes which did not correctly implement the interfaces they claimed to. I assumed it was standard practice to verifyObject() an instance of each new class you wrote, but there seems to be a hole here. The cases I found were minor, and I'm not sure how a major bug would squeeze in here, but it still seems like a hole. It certainly stole a day from me. I think the registry team have branches that fixed or fix all the issues reported running registry tests. Soyuz also has the SoyuzTestPublisher which was creating some objects that weren't proxied. Fixing that is proving to be more time consuming than any I have fixed in the factory. I am fixing these each time I see a warning in the tests I run (when I run a subset), so taking the warning away would mean that I would stop doing that. From a complete test run earlier today we still have: test_publishing.py complaints about removeSecurityProxy (now ShouldThisBeUsingRemoveSecurityProxy). This can just be replaced with removeSecurityProxy, and I have that in an in-progress branch here, but it will be a while before that lands. Someone with a branch ready to submit could fold that in. This is by far and away the most number of lines in the test output. makeSeries makeBranchMergeProposal makeCodeReviewVoteReference makeSourcePackageRecipeBuildJob makeTranslationMessage makePOTMsgSet makeTranslationImportQueueEntry So there are not too many left to fix. An alternative to turning it off would be to only produce one warning per method, per run. That would still provide the warning but also be much less output. If we are going to ditch the proxy, or turn it off in tests, or something, then could we please do it soon? Otherwise I will waste even more hours of my time. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Python rebuilds (was: Archive deletion strategy)
On Tue, 3 Aug 2010 11:12:30 +0100, Julian Edwards julian.edwa...@canonical.com wrote: On Tuesday 03 August 2010 01:21:36 James Westby wrote: On Mon, 2 Aug 2010 18:00:40 -0400, Barry Warsaw ba...@canonical.com wrote: Yep. I think PPAs are *usually* used to provide packages to users. In my case, I'm using them primarily as a kind of test build farm of a subset of the archive. This has other problems though (e.g. overwhelming the build machines) so while it's convenient, it might not be exactly the right (or typical) use of the resources, even though you and Julian keep insisting it's okay :). Julian was saying the other day that he doesn't think it is the correct strategy, and that you should be using a COPY archive to do these tests. Perhaps you should discuss it with him again and consider switching if that would make more sense. I spoke to doko and he said that they weren't using COPY archives because it's harder to select which packages they want to build. This isn't a problem for me provided we know _in advance_ of doing this so that we can score the PPA's builds down globally. Otherwise, the rebuild ends up DOSing the build farm. We could make change COPY archives to allow copying a specific list of packages fairly easily couldn't we? (COPY archives use the source archive as an additional source at build time?) Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Archive deletion strategy
On Tue, 3 Aug 2010 11:08:03 +0100, Julian Edwards julian.edwa...@canonical.com wrote: What actions do we have? As I see it we currently have the following: 1. Move creation of DELETED publishing records to the publisher. We don't need to do that at all because we're going to delete those rows entirely. 2. Modify the publisher to remove COPY archives from disk as well as PPAs. 3. Modify the publisher to also remove the librarian files of COPY archives. (Alternatively we could extend the garbage collection approach used for PPAs to COPY archives.) You don't need to do this at all, the GC will DTRT for unreferenced files. We just need to make sure we only delete the SPRF/BPF rows if there's no more publications referencing them. 4. Modify the publisher to delete db rows of publication records, unique sources and binaries, etc. in COPY archies. (Less important than deleteting the librarian files.) There will be alternative or further actions depending on what we want to achieve. I think that's pretty much it, although we need to examine exactly which database rows need to be deleted and under what conditions. I've added some ON DELETE CASCADEs in the past where it's a no brainer but it won't delete everything because of the multiple publication referencing package files issue. Just to be clear, you are advocating deleting every trace of an archive (except records referenced elsewhere), as soon as the publisher runs after you delete an archive, for all archive types? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Archive deletion strategy
On Mon, 02 Aug 2010 23:14:55 +0100, Max Bowsher m...@f2s.com wrote: On 02/08/10 22:09, James Westby wrote: (Note that uploading the same version as before with different contents isn't a problem with apt. Apt will do the right thing and get you the new version, but there may be other tools that break.) Mmm, I disagree. I've observed Apt doing the following: 1) Aha! The package in the archive has changed its metadata! You need to upgrade from version 1.0-1 to version 1.0-1. 2) But, I already have version 1.0-1 in /var/cache/apt/archives/ so I don't need to download it. 3) Oh noes! The installed package is still different from the archive! You need to upgrade from version 1.0-1 to version 1.0-1. ... repeat infinitely, until you manually purge the old version from /var/cache/apt/archives/, so that Apt actually downloads the package, rather than reinstalling the old incarnation of the version number. Hah, that's mighty silly of it, and is a bug in apt that should be fixed. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Python rebuilds (was: Archive deletion strategy)
On Mon, 2 Aug 2010 18:00:40 -0400, Barry Warsaw ba...@canonical.com wrote: Yep. I think PPAs are *usually* used to provide packages to users. In my case, I'm using them primarily as a kind of test build farm of a subset of the archive. This has other problems though (e.g. overwhelming the build machines) so while it's convenient, it might not be exactly the right (or typical) use of the resources, even though you and Julian keep insisting it's okay :). Julian was saying the other day that he doesn't think it is the correct strategy, and that you should be using a COPY archive to do these tests. Perhaps you should discuss it with him again and consider switching if that would make more sense. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Generic Colletion?
On Tue, 20 Jul 2010 11:41:21 +0200, Jeroen Vermeulen j...@canonical.com wrote: This is a generic base for that kind of collection. It wouldn't slot comfortably underneath the existing BranchCollection API because the latter exposes Storm's Select objects. Storm doesn't really expose those, but you can obtain them through a private method. My Collection class doesn't go that deep. A simple but quite usable concrete collection class based on my Collection takes 4 lines (that's including import docstring). I hope it will facilitate lots of future instances of the collection pattern. Hi, Thanks for doing this Jeroen, it is silly easy to create these very useful objects now. To make it even easier for others I created https://dev.launchpad.net/Collections documenting the couple of steps required to do this. If someone could fill in the couple of sections there when they implement the features that I haven't added yet, that would be great. Our own Collection is much simpler but pushes optimization problems higher up: you maintain some special knowledge in your concrete collection, or you just use the collections differently. We already have an instance of optimization through specialized knowledge in our pilot implementation, and it works well without complicating the normal case. I think that it would be valuable to put some performance things to watch out for on that page too. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] warning: we will soon have much noise in the test results...
On Wed, 21 Jul 2010 18:54:50 +0200, Abel Deuring abel.deur...@canonical.com wrote: The branch lp:~adeuring/launchpad/security-guarded-test-object-factory-1 is at present in ec2 and will land soon. Its main change affects LaunchpadObjectFactory: This class has at present ca 20 makeWhatever() methods which return objects without a security proxy. If this happens, LaunchpadObjectFactory will now print a warning. I intend to land further branches where the affected methods will changed so that they return security proxied objects. This will in turn cause a larger number of test failures. As a simple fix/workaround, I added a function remove_security_proxy_and_shout_at_engineer(obj) which just returns removeSecurityProxy(obj) but it also prints a warning to stderr. Properly fixing all tests would take far too much time -- and after all, the test behaviour will be the same as before. The only difference is that we will clearly see where our tests work with bad objects. Hi, Just a couple of observations as I got some of this in my tests. I realise things may be changing, but I wanted to record them as I found them. PLEASE FIX: LaunchpadObjectFactory.getAnyPocket returns an unproxied object. - This returns an enum value, which I don't think can be proxied? PLEASE FIX: LaunchpadObjectFactory.makeSourcePackagePublishingHistory returns an unproxied object. - This does # SPPH and SSPPH IDs are the same, since they are SPPH is an SQLVIEW # of SSPPH and other useful attributes. return SourcePackagePublishingHistory.get(sspph.id) what would be the way to get a proxied object if that is what we want? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Permission checks in the model (was: Re: warning: we will soon have much noise in the test results...)
On Mon, 26 Jul 2010 11:15:01 +0100, Jonathan Lange j...@canonical.com wrote:
(Poking around will also find model code that checks for
permissions or even duplicates the logic found in
canonical/launchpad/security.py).
Could someone provide some guidance as to how to do this well. I was
asked to make the Specification model object safe to export by moving
some code out of the view in to the model.
One of the pieces of code was in proposeGoal(), in the view:
# do the stuff to propose the goal
specification.proposeGoal(goal, user)
if check_permission('launchpad.Driver', goal):
# approve the proposition automatically
...
As this should happen if you propose the goal over the API I wanted to
move this down to the model.
The signature of proposeGoal() is
def proposeGoal(self, goal, proposer):
...
Therefore I want to check in there whether the proposer has
launchpad.Driver on the goal.
check_permission doesn't work for that, and using the code in
security.py is tricky as it requires getting an interaction for
proposer, which seems wrong somehow.
What I found, by looking at bug nomination acceptence, which is similar,
was
is_driver = False
for driver in goal.drivers:
if proposer.inTeam(driver):
is_driver = True
break
if is_driver:
self.acceptBy(proposer)
which also seems wrong.
What should I be doing?
- What I did?
- Don't do this in the model?
- Change the API so that check_permission works?
- Use the security.py code, possibly writing a helper to get an
interaction for a person without logging them in, and passing this
to checkAuthenticated().
- Something else?
Thanks,
James
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] architecture review progress
On Mon, 26 Jul 2010 06:46:59 +0200, Robert Collins robe...@robertcollins.net wrote: I'm not entirely sure myself, however there appear to be quite a few frequency based things around that are intrinsically event driven, and they aren't all 5 years old :). There isn't - sadly - a single bug / design aspect that I can point to and say 'if that were better, everyone would reach for this tool first'. If I were to make some guesses - the system (AIUI) requires a bunch of manual work to provide a robust debuggable 'do things later' mechanism, on a case by case basis. So adding a table is pretty high friction; writing new UI's for the same table is pretty high friction. To get a feel for what it is like to use the job system when there isn't an appropriate job type take a look at https://dev.launchpad.net/Foundations/JobSystem All that copy and paste code gets you to the point where you have a job type (like BranchJob), and an implementation of a single job of that type. It's missing things such as writing the cronjob and getting it deployed, and writing a UI to start/monitor/cancel etc. jobs of that type. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] How do I support older API calls when changing the underlying model?
On Mon, 26 Jul 2010 14:45:39 +1200, Tim Penhey tim.pen...@canonical.com wrote: Hi Leonard, I'm in the situation where I'm wanting to split up a status field on the branch merge proposal into two. However there is an existing API that is used to fetch merge proposals. How do we continue to support the older methods when changing the underlying model? Should we even try? I'm wanting to break the queue_status member into merge_status and review_status. I was also considering re-using the BranchMergeProposalStatus enum for the merge_status with changed values. However I feel that this is more likely to break something. I'm at a bit of a loss to know where to start with supporting the old method where the method directly gets values from an enumerated type I'm changing. Should I instead of reusing the BranchMergeProposalStatus enum, create two new ones (instead of just one new one)? How do I have a method that is only supported in an old API version? I can't answer all your questions, but I can make sure you have seen eggs/lazr.restful-0.9.29-py2.6.egg/lazr/restful/docs/multiversion.txt which documents the bare lazr.restful way of doing this. It may give you some clues for at least some of your questions. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] New feature in Launchpad TestCase base class
On Mon, 26 Jul 2010 11:32:38 -0400, Aaron Bentley aa...@canonical.com wrote: Are you handling cases where the test reconfigures the oops handler or uses its own instead of the global? I don't know, Tim? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] How do I support older API calls when changing the underlying model?
On Mon, 26 Jul 2010 12:16:48 -0400, Leonard Richardson leonard.richard...@canonical.com wrote: @operation_removed_in(devel) @operation_for_version(devel) Do these need to be updated next time devel is snapshotted, to create 2.0 or whatever it will be? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] New feature in Launchpad TestCase base class
Hi, I've just landed a change to use the great testtools feature of result attachments (addDetails()) to attach any generated oops to the result. I did this after I broke XMLRPC and had several tests that just failed with an OOPS id on the xmlrpc client side. On Tue, 4 May 2010 10:38:11 +0100, Jonathan Lange j...@canonical.com wrote: Perhaps we should go a step further and fail tests if they generate any oopses? It would be easy enough to provide a method that flushes out the stored oopses list. I think it would be good to do this as well. I'm unsure about a method to clear self.oopses though, as it could mask issues. How about def clearOopses(self, count=1): self.assertEqual( count, len(self.oopses), There were an unexpected number of oops generated by this test: expected %d, got %d. % (count, len(self.oopses))) self.oopses = [] This will ensure that if you manage to trigger two oops when you expect only one you will get a failure, and the code I just landed will make sure that you can see them. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Adapters API
On Thu, 01 Jul 2010 15:40:18 +0100, Michael Hudson michael.hud...@canonical.com wrote: Here's a thought experiment which possibly gets to the point Rob was making: We want to make Launchpad more of a 'choose your set of apps' thing, so that you can run, say, just codehosting (and registry, of course) or just bugtracking. It would be nice if being able to run different 'mixes' *in the same process* wasn't completely, theoretically, impossible -- so for example, having something in the bugs code that actually added things to the lp.registry.interfaces.product.IProduct interface object would fall into this category, as opposed to having the process of building the api root consider which things to add to *this* api root's product interface depending on config[1] values. For a start, testing is a lot easier in the second approach. That is a good point. I'm not sure it's entirely feasible with lazr.restful right now, but having something that mediates what it does with the decorators that it finds would be possible. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Adapters API
On Mon, 29 Mar 2010 14:59:12 -0400, Leonard Richardson leonard.richard...@canonical.com wrote: Assuming I understand what you're trying to do, the simplest way to represent this would be to define an annotation like @import_from_adapter() and put @import_from_adapter(IBranchTarget) in IProduct and ISourcePackage, while leaving it out of IPerson and IProductSeries. In IBranchTarget you would have an annotation like @only_used_in_adaptations that would stop it from being exported as a separate entry type. You would publish IBranchTarget.name as branch_target_name (or whatever you wanted) to avoid colliding with a 'name' declared in IProduct or ISourcePackage. The same would hold true for IBranchTargetName's named operations. You can choose whatever name you want, but if that name conflicts with a name found in the original interface, the original interface takes precedence. The end result would be 'product' and 'source_package' looking like they had everything that's @exported in IBranchTarget. Behind the scenes, a product or source package would be adapted to IBranchTarget to retrieve values like 'branch_target_name'. A PATCH that modified (eg.) both name and branch_target_name would make both object modifications within a single database transaction. This just came up again in a different situation, with a couple of additional requirements. What we would like to do is avoid registry importing from so many of the other apps, to avoid the circular dependencies. To do this we are currently looking at having ISomething interfaces in the apps (bugs, code, soyuz etc.) that define an interface relevant to them. This is standard practice for a lot of things, see IBugTarget, IHasBranches etc. The implementations of these are then adapters, which isn't usually the case. class ProductToBugTarget(): adapts(IProduct) implements(IBugTarget) This means that all the bugs-specific code can live in lp.bugs. This means that we want something similar to before, where some IProduct methods should be made available over the webservice by adapting the IProduct and calling the methods on the result. In order to break the circular dependency we need to stop IProduct from deriving from IBugTarget as well. In our experimental branch we define a marker interface in lp.registry for things that can be adapted to bug targets, which helps with registration etc. class IProduct(ICanBeBugTarget): The wrinkle that this has it that it doesn't accomodate the webservice. Currently in order for IBugTarget methods to be available on IProduct they have to be available in the interface that IProduct derives from, currently IBugTarget, which means that bugs code has to be imported in to the registry. Using ICanBeBugTarget means that IBugTarget can't add methods to IProduct right now. Therefore we want a way to annotate that extra methods should be added from the webservice point of view, but crucially, we don't want to import the bugs interface in order to do that. class ICanBeBugTarget(Interface): @import_from_adapter(IBugTarget) requires importing lp.bugs.interfaces.bugtarget.IBugTarget, which gives us the circular dependencies again. Would it be possible for us to instead have something like: class ProductToBugTarget(): @only_used_in_adaptations @add_things_to(IProduct) such that the arrow is drawn the other way, and apps can add things to registry objects without the registry having to import them? I don't see why this can't be done, except that it means that finding out the methods on an object is now a global operation, requiring importing everything and considering the result, rather than just importing the interface and scanning it. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Adapters API
On Wed, 30 Jun 2010 07:27:53 +1000, Robert Collins robert.coll...@canonical.com wrote: A few small thoughts if I may. Yay to looser coupling. Sadness at more global state (vs directly expressed dependencies, directly injected connections). I'm not sure what you mean here. Using 'import everything' for your dependency injection seems risky (but I haven't quantified it) to me. Everything isn't a core part of the design, just the easiest thing to express in that one mail. We can vary this part as needed. Lastly, I think that you'd want to check the performance of the global lookup you're adding. Globals are generally evil, and adding more doesn't really feel like a solution to me. I agree that it should be considered, but for the root wadl you have to load everything in anyway, so I don't think there will be a significant impact there. If there are times when we don't have everything loaded and want to list the methods, or find whether a particlar method is valid/where it should be dispatched to then there will be a cost. It would be possible to avoid this my having references to other interfaces to look for webservice declarations on that didn't use symbols, and used strings or something instead. That would avoid the global nature, but at a cost. In addition, having additive behaviour, where features are enabled and disabled as you install different parts of Launchpad, while something that is a long way from actually being possible, would be tougher with that approach. As a parallel, it might be worth seeing it as wanting to provide hook points, similar to bzr, where plugins can extend/modify behaviour, but the core doesn't know anything about the details of the plugins. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] API issue moving branches
On Thu, 20 May 2010 10:13:13 -0400, Leonard Richardson leonard.richard...@canonical.com wrote: I don't know what launchpadlib setup you used to get your 13 seconds number, but it was either a new setup or an old setup. Let's say it was a new setup. You made two HTTP requests and it took 13 seconds (due to latency, that's longer than it took me to make the same two requests). I think you may be going a little far in assuming that he would have been making 2 http requests in a new setup. That requires using a shared cache, which is unacceptable to some applications without writing their own cache implementation right now. (The alternative is fixing the bug in httplib2 or lazr.restfulclient and waiting for it to be widely deployed.) Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] API issue moving branches
On Thu, 20 May 2010 11:46:50 -0400, Leonard Richardson leonard.richard...@canonical.com wrote: I think you may be going a little far in assuming that he would have been making 2 http requests in a new setup. That requires using a shared cache, which is unacceptable to some applications without writing their own cache implementation right now. (The alternative is fixing the bug in httplib2 or lazr.restfulclient and waiting for it to be widely deployed.) Can you elaborate? I'm not sure which bug you're talking about. I know that shared caches are problematic when you're running launchpadlib in a multithreaded environment. Multi-threaded and multi-process. If there is a chance that you will have two clients running at the same time then you don't want them to share caches. For some applications this is highly likely, and so using a shared cache is a bad idea, and so you either have to implement a new cache, or pay the startup cost every time. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Building non-native packages from branches
Hi all, Another short spec from me on building non-native packages from branches, required for Ubuntu adoption, https://dev.launchpad.net/BuildBranchToArchive/NonNativePackages Comments appreciated. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Launchpad's service WADL clarification
On Sun, 25 Apr 2010 19:13:57 +0530, Manish Sinha m...@manishsinha.net wrote: On 4/25/2010 6:25 PM, James Westby wrote: No, it means that the method takes any/all of the union of the parameters defined directly with param or as part of a representation. Meaning in this case the method signature would be methodname(foo, bar, baz) From WADL submission 1. Zero or more |representation| elements - see section 2.11 http://www.w3.org/Submission/wadl/#x3-210002.11. Note that use of |representation| elements is confined to HTTP methods that accept an entity body in the request (e.g., PUT or POST). Sibling |representation| elements represent logically equivalent alternatives, e.g., a particular resource might support multiple XML grammars for a particular request. Ok, so it seems you have a grasp on what is legal in the WADL and not now? Taking in to account that rule and the previous discussion would suggest a complete description of what is allowed by the spec. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] bzr-builder woes - and where to from here
On Fri, 23 Apr 2010 07:18:50 +1000, William Grant wgr...@ubuntu.com wrote: On Thu, 2010-04-22 at 16:01 -0400, Francis J. Lacoste wrote: Hi Tim, That plan doesn't work. By policy, we only install stuff within the data centre from IS-controlled archives. I might point out that these machines run code from arbitrary users on the Internet. But the point does sort of apply a bit. So once, the updated package is available into the Launchpad PPA, you will need to file an RT # to have it deployed to the builders. lamont is responsible for maintaining the builders, so you can ping him. He'll take the Launchpad package, review it and then make the necessary magic to have it deployed to our builders. That will replace your #3 step below. We would need an IS-controlled PPA, then. The builders are virtualised and cleaned before every build, so we need this new bzr-builder *inside* the chroot, so it must be installed for each build. Another alternative is to install it on the host somehow, and then teach it how to chroot. It's a bunch more code to write though, and I'm not sure it's an important distinction from a code point of view. If the operation viewpoint requires it then we can do it. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Branching Ubuntu, again
On Mon, 19 Apr 2010 13:32:24 +1200, Michael Hudson
michael.hud...@canonical.com wrote:
It's on https://wiki.ubuntu.com/NewReleaseCycleProcess, but only as:
# Notify James Westby/Jonathan Lange to initialize the branches for the
new series.
We'll need to run:
./scripts/branch-distro.py -v ubuntu lucid maverick
And at the end of the run we need to run something like:
update branch set next_mirror_time = CURRENT_TIMESTAMP AT TIME ZONE
'UTC' where id in (select branch from seriessourcepackagebranch where
distroseries in (select distroseries.id from distroseries, distribution
where distroseries.name in ('lucid', 'maverick') and distribution.name =
'ubuntu'))
to trigger the puller to run. Last time this clogged up the branch
puller queue for several hours, so maybe we should run something like:
update branch set next_mirror_time = CURRENT_TIMESTAMP AT TIME ZONE
'UTC' where id in (select branch from seriessourcepackagebranch where
distroseries in (select distroseries.id from distroseries, distribution
where distroseries.name in ('lucid', 'maverick') and distribution.name =
'ubuntu' limit 500))
every so often until it doesn't update any rows, or something that
spreads next_mirror_time for these branches evenly over the next 10 hours.
Great, thanks for the information. Should I file an RT or something so
that the LOSAs are aware that this will need to be done?
* Anything I have forgotten?
The timing hasn't really worked out, but I'm working on removing the
hosted/mirrored split for Launchpad branches, so next time the script
will be simpler and faster, as well as not needing to do the
next_mirror_time frobbing.
Well, it's not like this is going to be a one-time event.
Thanks,
James
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Test failures on some platforms due to lazr.restful bug
[ Resending to the correct address, sorry Leonard ] Hi, Some of you are seeing Failure in test lib/lp/registry/tests/../stories/webservice/xx-distribution.txt Failed doctest test for xx-distribution.txt File lib/lp/registry/tests/../stories/webservice/xx-distribution.txt, line 0 -- File lib/lp/registry/tests/../stories/webservice/xx-distribution.txt, line 193, in xx-distribution.txt Failed example: pprint_entry(antarctica_country_mirror_archive) Exception raised: Traceback (most recent call last): File /home/abentley/launchpad/stable/eggs/zope.testing-3.8.1-py2.5.egg/zope/testing/doctest.py, line 1361, in __run compileflags, 1) in test.globs File doctest xx-distribution.txt[line 193, example 42], line 1, in module File /home/abentley/launchpad/stable/eggs/lazr.restful-0.9.24-py2.5.egg/lazr/restful/testing/webservice.py, line 168, in pprint_entry for key, value in sorted(json_body.items()): AttributeError: 'NoneType' object has no attribute 'items' where the failure is actually in the patch command before (checking response codes of all webservice codes can give more useful error messages.) I tracked this down to https://bugs.edge.launchpad.net/lazr.restful/+bug/561521 It only affects some platforms, and not ec2/buildbot apparently, so you'll probably only see it locally. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Build from recipes to multiple series
On Fri, 09 Apr 2010 13:26:57 -0400, Aaron Bentley aa...@canonical.com wrote: On 04/09/2010 12:18 PM, Jonathan Lange wrote: Has the model been updated yet to accomodate this feature? I don't know. Best to ask someone from the Code team. These messages don't have enough context to tell me what the feature is. Launchpad's model allows a recipe to have multiple distroseries, and these are used as the defaults when a user requests builds. When the user requests builds, a separate build is generated for each distroseries the user selects. The builds just vary in target distroseries? That's not going to work. The change just added to bzr-builder allows you to make it work by ensuring that the version numbers don't collide and give the correct upgrade path. The launchpad branch I pointed to shows how to make use of this. I'll be happy to help you get the feature implemented correctly. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Build from recipes to multiple series
On Wed, 10 Mar 2010 15:03:10 +, Jonathan Lange j...@canonical.com wrote: I need to know that the proposals here allow for the desired UI to be implemented, and we need a decision on the two options in the multi-series proposal. Tim's made one already. :) Ok, revision 78 of lp:bzr-builder adds --append-version to allow this to be done. lp:~james-w/launchpad/recipe-append-version shows how it would be integrated in to the build process. Someone can take this and integrate it properly once the above is updated in sourcecode. Has the model been updated yet to accomodate this feature? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Heads Up: Possible errors from exporting Choice
Hi, Just a heads up that might save you a few minutes of head-scratching. I have a branch playing in ec2 right now to catch problems with exporting Choices. If you do this then you can't actually use the attributes through the API, but the normal way that webservice tests are written doesn't catch it. If you are doing something like: some_person = exported(Choice(vocabulary=SomeGroupOfPeople)) then you will now see AssertionError: You exported some_person as an IChoice based on an SQLObjectVocabularyBase, you should use lazr.restful.fields.ReferenceChoice instead. which is trying to be helpful and telling you that you instead need to do from lazr.restful.fields import ReferenceChoice some_person = exported(ReferenceChoice(vocabulary=SomeGroupOfPeople, schem=IPerson)) All users of the broken way in the current tree should be fixed (anything that isn't is really broken), but you may find this as you export something new. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Code Import Reviews
On Tue, 06 Apr 2010 14:03:26 +0100, James Westby jw+deb...@jameswestby.net wrote: Hi, I'd like to discuss reviews of new code imports. Currently SVN and CVS code imports require review before working. What are the reasons for this? I've been working on exposing code imports over the API, including creation, and should have this done this release. That provides the functionality I need to start importing Debian branches using a script. This will obviously create a lot of new imports though, and there will be plenty of SVN ones, if not many CVS ones. Reviewing all of those will not be pleasant though. If reviews are required, then lets discuss how to limit the pain. Otherwise we should look at automatically approving, either all new imports, or ones created for this purpose. Another thing that came out of a conversation that Michael and I had yesterday was that if the Debian maintainer uses a branch of upstream this won't stack on the upstream project. Auto-stacking will be difficult anyway given the existing rules for auto-stacking package branches. Could we explore explicit stacking of code imports? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Documentation on landing changes
Hi, After getting PQM access I struggled to find information on how to use it without asking lots of people. To try and fix this for someone in the future I wrote down what I learnt: https://dev.launchpad.net/LandingChanges Please correct, amend, extend, link, replace or merge as appropriate. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Code Import Reviews
Hi, I'd like to discuss reviews of new code imports. Currently SVN and CVS code imports require review before working. What are the reasons for this? I've been working on exposing code imports over the API, including creation, and should have this done this release. That provides the functionality I need to start importing Debian branches using a script. This will obviously create a lot of new imports though, and there will be plenty of SVN ones, if not many CVS ones. Reviewing all of those will not be pleasant though. If reviews are required, then lets discuss how to limit the pain. Otherwise we should look at automatically approving, either all new imports, or ones created for this purpose. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Adapters API (was Re: Exposing newCodeImport)
On Tue, 30 Mar 2010 10:32:01 +0100, Jonathan Lange j...@canonical.com wrote: And, since my IRC backlog indicates I didn't make it clear enough here, I'm not saying Go forth and implement this solution, I'm discussing possible solutions to an interesting problem. It would be wise to seek Gary or Leonard's advice on this topic. Yes, I'm keen to, as I'm blocked on this, and pretty soon I will have to timeout and drop my LP hacking activities for some time to progress other things. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Adapters API (was Re: Exposing newCodeImport)
On Mon, 29 Mar 2010 17:21:56 +0100, Jonathan Lange j...@canonical.com wrote: I think ultimately you'll need to be explicit. For example, IBranchTarget(sourcepackage).name != sourcepackage.name for any ISourcePackage. Right, that's true, which pretty much rules out the invisible to clients approach, unless a prefix is added. branch_target_name would be ok in my eyes, but branchTargetNewCodeImport() or similar would not be. There are interfaces which are adapted to that have writeable properties which change the database when written to. Even if there weren't any that existed now, I think it would be misguided to constrain our object model to make the webservice implementation easier. Agreed. If there are writeable attributes in any adapted objects then you have the choice between a PATCH to the object and it's adapted version that would have to infer which attributes are associated with which object, or two PATCH requests with the inherent atomicity concerns. I'd lean to the second – to being explicit. The atomicity concerns would not be unique to adapted objects. True. I think we are in agreement then. There's pretty much nothing that would stop you implementing this approach now, just without any sugar to make it easy to do so. Should I have a look at doing it this way for IBranchTarget.newCodeImport()? Actually, perhaps I don't want to volunteer for that, as I'm not sure how I would implement the canonical url for IBranchTarget, given that it would change based on the target, and IBranchTarget doesn't actually have any attributes for gettting the original object. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Adapters API (was Re: Exposing newCodeImport)
On Mon, 29 Mar 2010 18:11:26 +0100, Jonathan Lange j...@canonical.com wrote: On Mon, Mar 29, 2010 at 5:55 PM, James Westby jw+deb...@jameswestby.net wrote: On Mon, 29 Mar 2010 17:21:56 +0100, Jonathan Lange j...@canonical.com wrote: I think ultimately you'll need to be explicit. For example, IBranchTarget(sourcepackage).name != sourcepackage.name for any ISourcePackage. Right, that's true, which pretty much rules out the invisible to clients approach, unless a prefix is added. branch_target_name would be ok in my eyes, but branchTargetNewCodeImport() or similar would not be. I guess you could implement IBranchTarget(obj).name and IBranchTarget(obj).newCodeImport() on the client-side too. As sugar for this case, or are you speaking about something else? This would obviously mirror the LP internals more closely, but I'm not sure how it could be communicated in the WADL that these were adapters, if we wanted to go for an approach most similar to the current LP code. If there are writeable attributes in any adapted objects then you have the choice between a PATCH to the object and it's adapted version that would have to infer which attributes are associated with which object, or two PATCH requests with the inherent atomicity concerns. I'd lean to the second – to being explicit. The atomicity concerns would not be unique to adapted objects. True. I think we are in agreement then. There's pretty much nothing that would stop you implementing this approach now, just without any sugar to make it easy to do so. Should I have a look at doing it this way for IBranchTarget.newCodeImport()? Actually, perhaps I don't want to volunteer for that, as I'm not sure how I would implement the canonical url for IBranchTarget, given that it would change based on the target, and IBranchTarget doesn't actually have any attributes for gettting the original object. It's not on the interface, but all of the implementations define a 'context' property. Ah, thanks. James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Adapters API (was Re: Exposing newCodeImport)
On Mon, 29 Mar 2010 14:59:12 -0400, Leonard Richardson leonard.richard...@canonical.com wrote: IBranchTarget has several implementations, one for each of product, sourcepackage, person, and productseries. I want the method to be only available on the first two if possible, but it's fine if it has to be all four, as it is part of the interface after all. Assuming I understand what you're trying to do, the simplest way to represent this would be to define an annotation like @import_from_adapter() and put @import_from_adapter(IBranchTarget) in IProduct and ISourcePackage, while leaving it out of IPerson and IProductSeries. In IBranchTarget you would have an annotation like @only_used_in_adaptations that would stop it from being exported as a separate entry type. You would publish IBranchTarget.name as branch_target_name (or whatever you wanted) to avoid colliding with a 'name' declared in IProduct or ISourcePackage. The same would hold true for IBranchTargetName's named operations. You can choose whatever name you want, but if that name conflicts with a name found in the original interface, the original interface takes precedence. The end result would be 'product' and 'source_package' looking like they had everything that's @exported in IBranchTarget. Behind the scenes, a product or source package would be adapted to IBranchTarget to retrieve values like 'branch_target_name'. A PATCH that modified (eg.) both name and branch_target_name would make both object modifications within a single database transaction. Would this work? (This is not something I can get to anytime soon, BTW.) That sounds like it would work fine for me. It's clearly more work than a small mixin, so is it worth the effort to do this? Are there other adapters that we may want to expose? Is it a pattern that will grow in usage in the LP codebase? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Exposing newCodeImport
On Mon, 29 Mar 2010 08:05:30 +1300, Michael Hudson michael.hud...@canonical.com wrote: I've got it to the stage where I have IBranchTarget.newCodeImport() and I want to expose this method over the API. I have two questions about how to do that though. However, I can't simply expose it, as it isn't on the actual branch targets. I'd like to know what the best way to do this is. IBranchTarget has several implementations, one for each of product, sourcepackage, person, and productseries. I want the method to be only available on the first two if possible, but it's fine if it has to be all four, as it is part of the interface after all. I can add newCodeImport methods to IProduct and ISourcePackage that just do def newCodeImport(...): return IBranchTarget(self).newCodeImport(...) but that seems like unnecessary duplication. Is there some way that I can tell lazr.restful to consider IBranchTarget when looking at what to export on IProduct etc? I *think* but without checking the code or really knowing for sure that you have to make IProduct inherit from IBranchTarget to do this. But IIRC products are not IBranchTargets, they are adaptable to IBranchTargets, and I don't think adaptation is (or really could be) a concept that translates into the API. Right, so it sounds like I still need a small interface and mixin to do the adaptation and call newCodeImport on the result? Does anyone know of a similar situation elsewhere that I can look at for inspiration? The second issue is with the registrant argument to newCodeImport. This eventually becomes the owner of the created branch. For my use case I want to specify this, as I don't want my user to own all the branches. However, for API simplicity it shouldn't be required. If I use @call_with(registrant=REQUEST_USER) will that prevent it from being specified by the caller? Is the correct solution to use @call_with() as above, but allow and optional owner parameter that defaults to registrant? That would solve the issue, and also prevent the registrant of the code import being recorded as a team, so it sounds like the best solution. Yes, that all sounds sane. Good, thinking about it further over the weekend it sounds like registrant=REQUEST_USER and owner=None is the right approach. In addition, I presume I need to add in access control, so that I can't create a code import owned by any user, only teams that I am part of. Is there an idiom for that? I'm pretty sure you can describe a parameter to a named operation as a schema field, so in this case you'd set it to be a ReferenceChoice for the appropriate vocabulary -- can't remember what it's called, but it certainly exists, the Branch +edit page uses it. There would be a vocabulary for IPersons that registrant is a member of? I was assuming I would need to add some sort of manual check? For these things is checking membership the right thing, or is there some sort of permission check so that e.g. admins can set anyone as the owner? Hope this was helpful, please don't be *too* surprised if its a bit inaccurate in places. No problem. Thanks for the feedback. James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Exposing newCodeImport
On Mon, 29 Mar 2010 11:02:53 +1300, Michael Hudson michael.hud...@canonical.com wrote: No, but there's a vocabulary for IPersons that the logged in user is a member of -- UserTeamsParticipationPlusSelfVocabulary -- and that's the same thing here. Great. I was assuming I would need to add some sort of manual check? Not really. For these things is checking membership the right thing, or is there some sort of permission check so that e.g. admins can set anyone as the owner? I thought the vocabulary did this, but it seems that it's not, it's done in the +edit form setup code (I guess because the widgets are different depending on whether the user is an admin or not). I'll start by using the vocabulary and writing a few tests for the simple cases. Once we have that then we can look at what is required for the special cases. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Test failure in webservice-marshallers.txt
Hi, I'm seeing a test failure in webservice-marshallers.txt: File lib/canonical/launchpad/ftests/../doc/webservice-marshallers.txt, line 86, in webservice-marshallers.txt Failed example: request.publication.endRequest(request, None) Exception raised: Traceback (most recent call last): File /home/jw2328/devel/lp-sourcedeps/eggs/zope.testing-3.8.1-py2.5.egg/zope/testing/doctest.py, line 1361, in __run compileflags, 1) in test.globs File doctest webservice-marshallers.txt[line 86, example 31], line 1, in module File /home/jw2328/devel/launchpad/devel/lib/canonical/launchpad/webapp/publication.py, line 634, in endRequest self.endProfilingHook(request) File /home/jw2328/devel/launchpad/devel/lib/canonical/launchpad/webapp/publication.py, line 706, in endProfilingHook now = datetime.fromtimestamp(da.get_request_start_time()) TypeError: a float is required Michael confirms that he sees it in devel, db-devel and production-devel. It looks to me like canonical.launchpad.webapp.adapter.set_request_started() is never being called, but the test explicitly ends the request with request.publication.endRequest(request, None) which causes the failure. The set_request_started() function appears to be usually called by LaunchpadBrowserPublication.beforeTraversal(request) which isn't called by this test, but is by a couple of others. Adding a call to that method fails as login() has already been done. Perhaps it would work to add it earlier though. buildbot isn't showing any failures for this, so Michael suspects it to be environmental changes. We are both running lucid. I haven't upgraded any packages recently, but I'm not sure I've run this test locally before. If we assume that it is lucid, what package is used from the base system that could be involved here? I would assume that all the zope stuff which would affect the publication comes from eggs. Is it correct that this test should have started the request? If so, does it need to do it explicitly? How? If it should be done by something else, what should take care of it? Is there a hole in the test coverage that means this isn't being caught by the automated tests of buildbot? I'm going to leave this alone now, and hope that someone with more zope knowledge can fix it. It's eaten enough of my time already. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Python2.6
On Thu, 4 Mar 2010 18:45:26 -0300, Sidnei da Silva sidnei.da.si...@gmail.com wrote: On Thu, Feb 4, 2010 at 4:40 PM, Jamu Kakar jka...@kakar.ca wrote: Hi, On Tue, Feb 2, 2010 at 10:12 AM, Barry Warsaw ba...@canonical.com wrote: James checked with Jamu about Landscape. They're still on 2.5 but he apparently does not think getting to 2.6 is going to be difficult for them. I'm not sure about U1. Our client runs on python2.6 now. I believe the only package we have on the server side that will need attention before we move to python2.6 is Zope 3 (specifically, lp:~landscape/zope3/trunk). We're currently running it on python2.5. Ideally, instead of fixing our branch, we'd move to a shared Zope package (or a collection of eggs, or ...). I haven't actually tried to make the branch work with python2.6, and maybe it's harder than I think it will be. So yeah, I'm optimistic. :) FWIW, Landscape trunk now works with *both* Python 2.5 and 2.6, with just a make variable switch. Our plan is now to get our Staging servers moved to Python 2.6 ASAP. Speaking of which, where is this magical PPA containing backports of Python 2.6 to Hardy? https://edge.launchpad.net/~python-dev/+archive/ppa I think, Matthias can confirm. That may not be all you need depending on whether you use any packages with binary extensions, and how the packaging helper tools behave given the new release, but it's a base to build on. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Build from recipes to multiple series
On Tue, 2 Mar 2010 12:36:28 +, Jonathan Lange j...@canonical.com wrote: There are open questions about how it matches with some UI issues, and how we store the recipes given the fact that they aren't necessarily tied to a single series. Storage should be guided by the UI, not the other way around. Indeed, what I meant was: traversal needs to be considered, as the current plan falls down to some extent if a recipe doesn't have a single target series. In addition there are open UI questions about which recipe to present to the user given that the current mockups lead them to choose the recipe before selecting target distroseries. If they are looking at a packaging branch then we can key of that, but even then do you show all recipes that have been used to build for that series? Only those that had a successful build? What do you show if none have been built for the series yet (e.g. the day after we release lucid and open m***) Looks good. * The Error handling section should mention the case where insert-nested refers to a merge line. Indeed. * I don't think namespacing is too much Ok. * There's no way to change the base branch Yes, unless we have the implicit base name associated with it. * It looks like it has the potential to be very confusing Could you be a little more verbose? * Perhaps one way to help with this is to have a tool that displays the recipe that results from the derivation Certainly. * Why not remove? If you aint gonna need it... Probably worth adding and shouldn't be too hard though. What's the next step? I need to know that the proposals here allow for the desired UI to be implemented, and we need a decision on the two options in the multi-series proposal. Once we have something that works well for the user experience I can implement it or help someone else through the process. I will be waiting on confirmation from the LP team on what they would like to have, preferably with (updated) mockups that outline how they will map to the new features. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Build from recipes to multiple series
On Tue, 2 Mar 2010 13:16:11 +, Jonathan Lange j...@canonical.com wrote: A derived recipe isn't designed for readability. To understand what it means, you need to be able to look at the parent recipes. The difference between insert-after and insert-nest causes me to pause and think when I read them. All of these combined make me think that derived recipes can confuse newbies. I think it's pretty much inherent to derivation though. Agreed. I plan to have a way to see the complete recipe, and you may want to implement that in the web ui too. Do you think that the difference in the insert-* methods are warranted? Just having insert-after means you can't do one thing (modify a nested branch that is currently unmodified in the parent), so I put it there for completeness. * Why not remove? If you aint gonna need it... OK, you've convinced me. Let's wait for a use case :) I guess I'll see whether the implementation means remove becomes trivial. What's the next step? I need to know that the proposals here allow for the desired UI to be implemented, and we need a decision on the two options in the multi-series proposal. Once we have something that works well for the user experience I can implement it or help someone else through the process. I will be waiting on confirmation from the LP team on what they would like to have, preferably with (updated) mockups that outline how they will map to the new features. OK. I don't have an opinion now, but will cultivate one carefully over the next few hours. I just spoke to Michael and he is happy that the current mockups work well with the proposals, so that's one step closer. Do we want input from the design team/Mark on the mockups/changes at some stage? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Bug heat and the API
Hi, With the addition of bug heat there's a danger of a little trap being inadvertently set for those using the API. If bug_heat is exported as an attribute over the API then it goes in to the calculation of the etag for a bug. That means that bug_heat, like status, importance, etc. can't change between GETting the bug over the API and PATCHing new values to it. However, as bug_heat is decayed using cron it will race with all modifications done over the API, meaning that you will occaisionally get 412 errors from your scripts. There would be a couple of ways of fixing this, such as exporting bug_heat as a method not as an attribute. We already have something like this with, at least, code, where you race with the scanner when pushing up a branch and then modifying an attribute of the branch over the API. You could argue that script authors should deal with 412 errors, as they can happen with concurrent user edits and the like. While that is true, I'm not sure we need to be making it more likely to happen, and racing with automated tasks is less or a reason to conflict anyway. Plus, the facilities for handling conflicts in the API are lacking from where they should be if all API users should be dealing with them. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Bug heat and the API
On Tue, 2 Mar 2010 16:59:03 +, Gavin Panella gavin.pane...@canonical.com wrote: There was another similar problem recently, but I can't recall the details. task_age being exported on bug_task, which I fixed :-) This one was worse as it changed every second (it was a @property that calcuated derived data), which meant you couldn't take more than a second to GET a bug_task and PATCH it, quite difficult with most people's RTTs. I agree that making it a method would be the quickest fix, but it feels more natural as an attribute. That's true. A way to mark attributes as volatile - and thus not included in etag calculations - could be one solution (i.e. an export decorator). I guess this would need support in lazr.restfulclient though. That would open the door to exporting other aggregate or derived model properties as attributes, though that might not always be a good idea. I don't think it needs support in lazr.restfulclient, etags are opaque strings to the client, it just remembers what the server told it last. Leonard could confirm whether only using writeable attributes when deciding whether to change the etag would be a good idea. The only issue with doing it is that a 412 may be the correct thing to do in certain circumstances, but I think they would be rare. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Bug heat and the API
On Tue, 2 Mar 2010 11:23:46 -0600, Deryck Hodge deryck.ho...@canonical.com wrote: IBug.heat is already exported. Ah, but not on edge according to the apidoc. It's there for both lpnet and staging. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Bug heat and the API
On Tue, 2 Mar 2010 18:28:54 +, Tom Berger tom.ber...@canonical.com wrote: On 2 March 2010 17:42, James Westby jw+deb...@jameswestby.net wrote: On Tue, 2 Mar 2010 11:23:46 -0600, Deryck Hodge deryck.ho...@canonical.com wrote: IBug.heat is already exported. Ah, but not on edge according to the apidoc. It's there for both lpnet and staging. It is on the edge documentation too. Could it be that you're looking at an old version of that page from the browser cache? Indeed, thanks. James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] RFC: UI for +filebug while waiting-for-blob-to-be-processed
On Tue, 16 Feb 2010 11:56:12 +, Graham Binns gra...@canonical.com wrote: I like this idea, but we rejected it early on because there's some data in Apport blobs that can be used to populate the +filebug form. Of course, we could add it after the fact, for example updating the bug description to be apport description + user description and so on. I'm ambivalent about doing that though. That said, it wouldn't be hard to have the job class tied to the bug once it's been filed and then, once the job's complete, updating the bug with the data from the blob. I'll look into how the data from the blob is used again (I can't remember why we rejected this approach before; I'm assuming there's a good reason that I've forgotten). If it's possible to do it this way then maybe that's how we should proceeed. A surprising number of people delete the apport info from the bug description, so appending that afterwards would reduce the incidence of that (at the cost of an extra bugmail I assume). However, the title of the bug is what worries me. We have a tough enough time with people deleting the apport-suggested title and putting in Crash!!! or similar, and that is all we would have if apport didn't suggest a title at all (I'm not sure how you would join the apport and user set titles). Also, usually people have no clue as to what caused the bug, so asking them to fill in blank boxes will stop some people filing. At least having them prepopulated allows people to write I don't know what happened, I was just asked to submit this without feeling like they aren't conveying any information whatsoever. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [RFC] Launchpad Enhancement Proposals
On Thu, 11 Feb 2010 12:31:27 +1000, Ian Clatworthy ian.clatwor...@canonical.com wrote: Also, one trick that's appropriate now and then is to write documents that have ongoing value (e.g. end user documentation) instead of documents that become obsolete. Indeed, I like combining specs and documentation, but I don't do enough of it. You do the UI design by writing the documentation for the feature despite it not existing, and where you would put a screenshot you put a mockup. You can then add another page that explains more about what is going on under the hood, which can act as your architectural specification for the feature. This means that you get the benefits of a specification, but it keeps its value after the implementation is done, and doesn't grow out of date like a spec will as you keep the documentation up to date as things change over their lifetime. It also includes a user experience focused approach as it forces you to think about how people will use it first, and the goal of a simple to describe interface nicely meshes with your desire to write less documentation. There are clearly some problems though, it adds a fairly large amount to the cost of changing direction in the middle of implementation, good documentation doesn't always match up with what you need to think about when designing something, we don't all make great documenters, and rolling out a half-finished feature becomes trickier as you have to edit the documentation so people don't get confused. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Python2.6
On Mon, 01 Feb 2010 18:32:34 -0800, James Westby jw+deb...@jameswestby.net wrote: Hi, I'm at the Canonical Ubuntu sprint and we were just discussing python versions. The current plan is that there will only be 2.6 and 3.1 in lucid. python2.5 in lucid is in its final days. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] RFC on build from branch UI
On Wed, 3 Feb 2010 16:12:10 +0100, Michael Nelson michael.nel...@canonical.com wrote: I have some comments that are quite fundamental, but let me say first that I like the design for interaction, and I'm confident that whatever you end up going with will be a pleasure to use. The main point that underlies all my comemnts is that your focus is on building a branch using a recipe, rather than building a recipe. Yes, definitely. I had thought that the recipe is just a means to the user goal of building a branch. Sorry, I wasn't trying to minimise their importance or flexibility, but rather ensure that the desire I want to build this branch stays in focus. I agree that is an important aim. Why wouldn't this be possible? Given that if you click Build this branch (or whatever it'll be), the overlay will allow you to select the recipe you want (ie. including those recipes that just refer to this branch). That wasn't clear, sorry. I think the base branch being immutable in the overlay indicated to me that it was all that would be offered. but for daily builds that won't be the base branch. Yes, so if, from the packaging branch, you click on Build this branch and non of the available recipes meet your requirements, so you need to create a new recipe - then if you do want a base-branch different from the current one you're stuck at this point (with the current design, as the base-branch is non-editable). So I talked with jml about this the other day when thinking through the workflow, and from memory there were three thoughts: 1. The base-branch seems like the logical place to build your base-branch (and specifying the packaging branch from there). That's not to say it is the only way, just the most natural way for the 90% case. Do you have other thoughts? I think it may be skewed as I am an Ubuntu developer, but I don't think packaging branches will just be implementation details. For instance, you may decide you want to set up a daily build for a new project, which isn't packaged yet. To do this you branch, create the packaging and push it. They you may go to the packaging branch as that was just what you were working with. I'm not sure it's a good enough argument that it should change your approach, but perhaps something like giving them a message if they choose daily build while on a package branch. 2. we couldn't see how we could have a consistent UI if users specified the base-branch (as you could create a recipe that has nothing to do with the current branch at all. [1]) 3. The case where it *does* make sense to initiate the recipe creation from a packaging branch is when the packaging branch contains the source already, and hence it is the base_branch (ie. no other merges required to have both the debian directory and the source code toghether). I guess you are saying that there are other cases? Well, creating a recipe from a packaging branch is how you will build a package in a non-daily fashion, so that has to be supported, but it will be the base branch in that case. Given that you understand the workflows much better than anyone, I'd be keen to have a call sometime to clarify the above :) I'd be happy to do that. Next week is probably better as I guess we will be in closer timezones again. This is exactly what's displayed in the overlay isn't it (or the non-js version)? It would be difficult to display this information (the recipes) on the branch page itself, as there is no single bit of the recipe that helps users identify whether it's the one they're after. You'd need the recipe name, owner, (distroseries?), at the very least (hence that's what's displayed in the overlay dropdown, but even then you might need more information (revnos, packaging branch etc.), which is why the overlay displays that in-line. Makes sense. [ snip more convincing comments ] Some further unstructured comments: When viewing a source package recipe build within their PPA, users can: easily go to the recipe that was used and from there to the branches. They should also be able to go to the manifest and so see the revisions that were used. I'm keen for these links to be more common in Launchpad, from e.g. bugs to the revisions that fix them etc. Being able to go binary package-source package-manifest- branches will be useful. Yes, that's great. I've updated the wiki with: When viewing a source package recipe build within their PPA, users can: ... * View the manifest (and from there the related branches and recipe) Although it'd be worth a link from the manifest to the recipe, I wouldn't link directly to the recipe as it could be very confusing if the recipe has been edited since? Good point. Yes, the mockup wasn't meant to communicate that it was editable (just that you could create a new recipe) - my mistake. I've updated it to show the non-editable recipe selection/display. See what you think. Looks great.
Re: [Launchpad-dev] Python2.6
On Wed, 3 Feb 2010 06:27:02 +0700, Stuart Bishop stuart.bis...@canonical.com wrote: We are not on PostgreSQL 8.4 yet (need Slony-I backported to Hardy PG 8.3 + PG 8.4). I wouldn't worry about postgresql-plpython-8.3 either as it doesn't really matter to us what version of Python is embedded (there is not much Python to fix if it is an issue). Yeah, sorry, this was a list with alternative packages substituted for comparison with my installed system. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] LCA mini-conf schedule - bridging the gap
On Fri, 8 Jan 2010 13:19:11 +1100, Jonathan Lange j...@canonical.com wrote: Each of these talks look like good ideas, but the titles, descriptions and (to a certain extent) the content all need to be aimed toward persuading people that UDD, soyuz code imports matter. I don't think using UDD in a talk title is a good idea. I haven't come up with one I like yet though, I'm working on it. For soyuz / build from recipes, it's easy -- get your trunk into the hands of a hojillion Ubuntu users. For code imports, it's less obvious -- remember that most people at the conference don't actually use Bazaar. One thing I want to be careful of is not giving people the impression we will be auto-building Ubuntu, so I'd like to steer more towards making available than getting into the hands of. Obviously the talk will make it clear, but the majority of attendees are more likely to just read the title/abstract. Basically I don't want it to be read as Ubuntu will be full of crap that no-one ever tested (any more than it already is). Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Build engineer report, Dec 2009
On Wed, 6 Jan 2010 09:42:04 +1100, Jonathan Lange j...@canonical.com wrote: Interestingly, I showed the recipe for bzr-tools-grep at UDS-L, and a git fan in the audience sniggered about Bazaar's vaunted usability because, bzr ls -VR --kind=file --null | xargs -0 grep -In %s becomes git grep %s Perhaps someone would like to add the grep command to bzr-core, or a plugin. You mean like lp:~vila/bzr/grep (confusingly a plugin under the bzr project?) I agree that it would be great to have it in core. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] First cut at recipe db-schema patch
On Wed Dec 02 06:53:49 -0500 2009 Jonathan Lange wrote: 3. Branches have the owner in the URL, not doing that here would seem to be inconsistent. This for me is the convinving argument. Branches have the owner as part of the name, this was decided early on, perhaps with knowledge of issues such as the URLs breaking (happens fairly frequently when e.g. your project matures so it's no longer controlled by ~you, but by ~your-project-dev). As we used this system we learnt more about how it worked and what the edge cases are, such as the interaction with bzr where it stores the expanded URL as the parent, so even if you update the dev focus people still end up at a dead end. Now we have a very similar situation crop up, where we have the same choice to make again, but with slightly differening concerns. It may be that this requires a different answer, but I am yet to see any argument that convinces me of that. It seems that most of the desire to change it is from wanting to avoid the issues we have seen with branches. Therefore I think consistency should win out. As Jono said, if they are different then to the user it can look like another weird LP inconsistency. Also, taking the less-trodden path is tempting because we avoid a pile of known issues, but it does open us up to several new ones, and we will know less how to tackle them. For instance we want inheritence in recipes. Therefore one needs to reference another. In text form this may be by URI. Bazaar has stacked branches that reference each other by relative URI, which has some interesting characteristics, for instance look at the dance that is required to upgrade branches on LP to 2a. How will we store the inheritence in the DB, how will it be presented? I don't know right now, but having the experience with bzr URIs might help us design that. In addition though, consistency has it's own benefits. What if we decide the fix the issues with changing ownership changing the URLs by, say, allowing redirects from the old name. If we had chosen not to follow branches for recipes then we would still have to solve the issues of having one namespace, we wouldn't be able to re-use the work. Therefore I think consistency should be the default, and only if the differing circumstances require a different answer should another approach be chosen. If the existing solution is so bad that we don't want to use it then don't, but also consider fixing up or changing the existing solution. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] First cut at recipe db-schema patch
On Wed Dec 02 08:03:22 -0500 2009 Barry Warsaw wrote: Isn't that where teams come in? Would it be totally insane to restrict tilde names in the URLs to teams only? Even when it's super trivial to create new teams? wink That's an interesting approach. It still has the issue that the owner of the team needs to be active and willing to allow you to work on it, though an LP admin could fix it for you given evidence. That's more palatable than them fixing it for you now by giving you ownership of the other person's account at least. I do think making it super trivial to create new teams would be a pre-requisite though. That's currently a barrier that may put some people off from experimenting. Though knowing that you will leave artifacts can also put people off. If just deleting your recipe is not enough to remove any trace of you doing the experiment then some people won't try, because they will want to avoid evidence of their possible upcoming mistakes. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] First cut at recipe db-schema patch
On Wed Dec 02 13:06:03 -0500 2009 Julian Edwards wrote: On Wednesday 02 December 2009 03:50:50 Michael Hudson wrote: which we don't have an analogue for yet in the build from recipe world -- I don't know if we need one though as SourcePackageRelease - Build is 1-to-many in a way that doesn't apply to recipes. Also, if they're automated daily builds, who is the requester? I guess the person who set it up? Or a celebrity? I dunno, I'm just throwing it out there. Do we have any idea yet how notifications will work? If the requester is subscribed to get notifications then the person that set it up would probably be appropriate. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] First cut at recipe db-schema patch
On Wed Dec 02 23:31:10 -0500 2009 Martin Pool wrote: That was a very insightful mail. I don't really want to agree with your conclusion, but I don't want to for the reasons you said - that we know the problems we've seen in the past with branches. Perhaps the best argument to do something differently is to see if it is in fact better, and then we could come back and apply that approach to branches. Structuring it as an experiment is interesting. It does give you the inconsistency for at least some time, and has some issues of it's own, such as inertia to changing either side once the experiment has commenced. My other concern would be that it soon became “the way it works,” rather than an experiment; that no-one would actually evaluate after, say, six months which approach would be better for each. There is also a danger than the inertia of changing the way branches worked would mean that they remained the same, even if the recipe approach proved better. Perhaps spending 10 minutes discussing how branches could be transitioned would help if it was positioned as an experiment. Perhaps there is a middle ground - at the moment you can have project-scope branches but only for series. We could have some kind of concept of a team allowed to create recipes in the project namespace. That could be very useful, c.f. official PPAs. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] First cut at recipe db-schema patch
On Mon Nov 30 06:22:13 -0500 2009 Jonathan Lange wrote: * Putting owner in the URL and conflating ownership with write permissions leads to broken URLs, which is bad. * Not having the owner in the URL disallows recipes for the same package with the same name. What other considerations are there? * Branches have the owner in the URL, not doing that here would seem to be inconsistent. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Immediate plan for Build Farm generic jobs
On Sun Nov 22 17:05:54 -0600 2009 Ian Clatworthy wrote: By that, I assume you mean I've packaged this branch. Upload it.? Or did you mean Package this branch for me and upload it.? Please clarify. I think the second is far more important wrt connecting upstreams to user communities via bzr+lp+ubuntu. You want a command that automatically does all the packaging for an arbitrary upstream project? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Immediate plan for Build Farm generic jobs
On Thu Nov 19 22:28:45 -0600 2009 Michael Hudson wrote: I still don't understand at all how this specific part works. If bzr-builder supports cross recipe references today, it doesn't document them. It does not. We will be working on a proposal for doing this over the next few months, and will involve LP developers in the discussion so that LP will be able to support them. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Immediate plan for Build Farm generic jobs
On Thu Nov 19 22:15:02 -0600 2009 Michael Hudson wrote: Recipes would definitely have an owner for access control. Whether that would be part of the URL/namespace is a separate decision. I was thinking of a parallel with branches, but you are better placed to know what will work well. Hm. At least making the manifest doesn't require running arbitrary code :-) Now you say this, I need to look again. The 0.2 format supports arbitrary code execution, but I'm not sure how it interacts with manifest creation. Also, there has been some discussion of handling private branches. After some discussions yesterday it is now clear that this is important for Canonical, as well as supporting private recipes. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Immediate plan for Build Farm generic jobs
Hi, Thanks for working on this everyone. Apologies if I duplicate comments already made elsewhere in the thread. On Wed Nov 18 00:15:27 -0600 2009 Michael Hudson wrote: Separately, we need to decide where a recipe lives. The current thinking is https://launchpad.net/ubuntu/karmic/+source/some-package/+recipe/recipe-name; which seems OK to me (we'd have to trust a bit that this recipe would build a recipe for some-package in karmic, but that doesn't seem any different to say branches today). Why not under a person? How do you do access control? Finally, we could stick an archive on the recipe, but maybe we don't want to. I'll talk about this a bit more later in the mail. I don't think we want to. They are not archive-specific, and you could put the recipe in to any archive. However, knowing which recipe was used to build a package would be useful. One of the things bzr-builder does when it creates the debianised source tree is create a manifest, which is a sort of frozen version of a recipe -- it references particular revisions of the branches so as to allow a repeat of exactly this build. We could use a manifest like this to actually run the recipe: at the point where the build is requested, we make the manifest and stuff it into the database. This seems like a neat idea, but isn't how bzr-builder works now as far as I can tell. It's not. However, I think it's quite a good idea, otherwise you have to collect the manifest too, or have the collector extract it from the source package. The difficulty would be resolving revision ids, as that requires bzr code, rather than just SQL (I assume). I think the current plan is to use bzr-builder to make the debianized source tree and bzr-builddeb to then make the source package. That currently isn't possible, but we plan to fix that. This may be the preferred way, but we need to decide how they will work together to be sure. I'm presume the process for getting the source package off the builder and into the process of being built will follow that of the existing builders: the builder will tell the buildd-manager where to get the .dsc, the manager will parse this to find the other parts of the package and then grab them, shove all of the files into the librarian and trigger the existing parts of soyuz to look at them somehow[1]. Will tell it where the .changes is, no? Also, as I said, you may have to collect a manifest as well. In case the above wasn't enough, here's some things I haven't thought hard about: - do people want to subscribe to a recipe? - does this mean getting notified when the recipe builds or fails to build? - does this mean getting notified when the recipe is changed? This is very important to get right. I don't know whether subscription is the right approach, but notifications on problems need to be communicated to the right people in the right way. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] branch-distro.py run for Ubuntu release
Hi, Thanks to great work from Michael we now have a tested branch-distro.py script to create Lucid branches. I have just been notified as part of the release process to stop the importers so that they don't pick up lucid and naïvely create branches. This means that in probably the next 2-24 hours I will get a notification that lucid is created and we should create the branches for it. I think this means that we will need some LOSA time scheduled to do this? Is someone willing to take this job on? Michael knows more about the requirements than I do, but I believe it to be fairly straightforward to run, and we expect the runtime to be ~6 hours. I will update the list when I get the second notification. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Debian archive almost ready to accept v3 source packages
Hi, I just read this http://blog.ganneff.de/blog/2009/10/27/debian-ftpmaster-meeting.html The summary is that dak has just had the changes merged to support the v3 source format. It doesn't say when they will be accepted, but it may be very soon. Is Launchpad ready for this? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Bug page polish
On Thu Oct 22 09:58:32 +0100 2009 Jonathan Lange wrote: On Tue, Oct 20, 2009 at 8:57 PM, Martin Albisetti martin.albise...@canonical.com wrote: - Being on a roll here, how about highlighting the reporters' comments? His comments will most likely be very important. I think you only want to do this subtly. I know Bryce values his greasemonkey script that does this. It makes it easier for him to deal with the person that reported the my video flickers bug, while telling the others who are commenting on it, but who have completely different symptoms, to file a new bug. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Bug page polish
On Wed Oct 21 16:27:49 +0100 2009 Julian Edwards wrote: On Tuesday 20 October 2009 20:57:49 Martin Albisetti wrote: Hello Launchpaderos and Launchpaderas, Hello Duderino. - Being on a roll here, how about highlighting the reporters' comments? His comments will most likely be very important. On a similar note, how about adding the poster's icon next to the comment? Again similarly, adding the user id of the user would be useful to many of us. In Ubuntu development we have quite a few processes where some action is taken because a developer said so in a bug. This is fairly fragile currently as anyone could change their display name to James Westby and make requests posing as me. Some of us that process such requests use a greasemonkey script to display the user id next to the comment, to try and alleviate this (though ~jamesw might still be able to pose as me without being caught). In addition it displays the emblems of some Ubuntu related teams when the person is in them, which is even more robust, and their karma (not really useful here, but can be useful in other cases). It would be good to have this built-in. I realise that working out which team emblems to show could be complex. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Build From Branch, or BFB
Hi Muharem, Thanks for the information. On Wed Oct 07 07:12:03 +0100 2009 Muharem Hrnjadovic wrote: Currently, we basically use anonymous ftp and rely on the key the source package is signed with and the .changes file to determine who the uploader/maintainer is. This is the same for binary and source uploads? I'm familiar with how source uploads work from doing them myself, but not the binary uploads from the buildds, which seems to have more in common with build-from-branch. This is problematic in quite a few regards and we planned to switch over to an authenticated (ssh based?) upload mechanism since July of last year. That's interesting, because... This would make it possible to upload unsigned packages That changes the security assurances that we have for packages, you are now relying on SSH keys rather than GPG keys. Are they believed to give us the same assurances? or for the uploader to be different from the person who signed the package. I don't see that is different from what we can do now? I can already sign a .changes with someone else in Changed-By (sponsoring), and can even give them the .changes back to upload directly if I like. Is this change driven by concerns over the current process for binary uploads from the buildds? Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
