Re: [Launchpad-dev] ec2 test failures
On Thu, Nov 17, 2011 at 09:03:24AM -0500, Aaron Bentley wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11-11-16 09:12 PM, Robert Collins wrote: One way to fix it would be to allocate each thread a range, monotonically increasing, and giving each range say 1M integers. Another would be to use the same series across all threads. Thats probably simpler. Bjorn implemented this to improve parallelism, IIRC. Not quite. I wanted to use LaunchpadObjectFactory to create example data that I could play with. E.g. a script create-20-persons.py, which would create 20 person records. Before you could run it once to create 20 persons, but not twice to create 40. My implementation used uuids, though. Someone else changed it to integers, since they thought that the generated names were too long. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Landing lazr-js upgrade blocked by Windmill, A Proposal
On Mon, Nov 22, 2010 at 01:00:30PM -0600, Deryck Hodge wrote: On Mon, Nov 22, 2010 at 11:39 AM, Robert Collins robert.coll...@canonical.com wrote: Man, it sucks that our infrastructure here is so broken [No one needs more than 640K]. This plan has some risk in it. Have you considered fixing Windmill? It may be unmaintained upstream, but we're still using it. Yes, I did consider fixing Windmill. There's an assumption there that I *can* fix Windmill. Okay, so I'll give myself the benefit of the doubt. Let's say I can. :-) Then there's the assumption that fixing the 512k bug will unblock this landing. We don't know what other problems might be lying around with so much JavaScript. We know the 512k bug is the current problem. I could spend a couple days getting that fixed and still not be able to land this branch. You could at least spend 5-10 minutes trying. The original bug about this is this one: https://bugs.launchpad.net/launchpad-foundations/+bug/519744 (Not sure why it's marked fixed, since the issue was never fixed, only worked around). Comment #1 has a suggestion on a possible fix/workaround in Windmill. I would suggest trying to change that constant and see what happens. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Landing lazr-js upgrade blocked by Windmill, A Proposal
On Wed, Nov 24, 2010 at 08:08:59AM -0600, Deryck Hodge wrote: Also, I think your you could at least spend 5-10 minutes is a bit unfair. I spent greater than 3 weeks on this branch, trying to get the new lazr-js and Windmill to place nice together. Yes, I meant spend another 5-10 minutes, after I give you this pointer :) Yes, now that I have new information, a simple fix does seem possible. But even then, come on, it's not 5-10 minutes. It's more like 8 hours, by the time you edit code, make a new windmill tarball, run a layer or two locally, and then run it through ec2 to see if it works. Well, I didn't say the whole process would take 5-10 minutes (let's extned it to 30 minutes, just to be on the safe side). I just suggested you should spend so much trying to fix it. I.e., apply the fix in your local egg, run the Windmill tests locally (and/or ec2). If the tests pass, it might be worth spending time going through the whole process you described. If they don't work, it might be better to go with a different solution. Still, as you say, the .js file shouldn't be 1.5Mb to begin with. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] How to run a few unit tests a bit faster
On Thu, Aug 26, 2010 at 03:15:26PM +1200, Michael Hudson wrote: On Wed, 25 Aug 2010 21:58:55 -0400, Gary Poster gary.pos...@canonical.com wrote: I think most people use the -t option to bin/test to find tests to run. This finds all possible tests, and then filters. If you are only running some relatively svelte little tests without layer set up, this can be very noticeable. You might like -m and -p. -m runs the tests in a module. -p runs the tests in a package. For me on Launchpad, -t is about 3 times slower than -m or -p. What's the difference between passing -m or -p and not passing any option at all? The following all run the same number of tests at the same speed afaict: ./bin/test -p lp.codehosting.vfs.tests -u ./bin/test -m lp.codehosting.vfs.tests -u ./bin/test lp.codehosting.vfs.tests -u Also, I don't understand the difference between -m and -p, given that you pass a package to -m... oh, it seems that -m takes a regexp whereas -p takes a package path. That suggests that -p ought to be fractionally faster, but it really doesn't seem to make any difference to me. In any case, yeah, don't use -t if you can get away without it! Well, unless -t is used together with -m or -p. That way you can filter down to specific tests within the package or module, and it's still as fast as -m or -p alone. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] performance tuesday - a few notes
On Thu, Aug 12, 2010 at 09:19:35AM -0300, Guilherme Salgado wrote: We used to avoid cachedproperties on model classes as debugging things when you have a (sometimes unexpected) caching at that layer is not fun, but we seem to have lots of them now. I remember that Bjorn used to advocate strongly for us not using them. FWIW, the reason I usually advocate against cachedproperty on model classes is basically what Salgado said. Especially since usually you can solve the problem in some other way, which makes it more clear that caching (that isn't being automatically invalidated when needed) is used. What I don't like about cachedproperty is that it easily becomes a tool that simply replaces @property. People just just it, just in case it's needed, without any thought about what the implications are. That said, I'm not against that kind of caching on the model object. Most of our request are read transactions, and for such transactions it's safe to use. It might be intresting experimenting with automatically caching properties in read transactions. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] memcache, responsiveness and load {short story, lets turn memcache off}
On Fri, Aug 06, 2010 at 11:10:35AM +1000, Martin Pool wrote: It would be useful to establish that memcached does in fact give us effective slashdot protection. If we used a web load generator to send (oh I don't know, what's typical?) 10 requests/second to a particular bug page, what happens? It seems like it ought to help, but it would be nice to test it on edge. On the other hand a lot of random casual visitor traffic may be anonymous, and might be served from squid anyhow? I'd be interested in knowing this as well. I've always wondered whether the caching on the milestone list actually helps, and how it helps. Considering it's cache:private, it means that each user has its own cache, right? So it doesn't help when multiple people look at the page the first time. It only helps when the same person reloads the page multiple times. Now, which use cases exist for that, except for checking if something changed, usually after having modified bugs himself, or someone on IRC told him that they changed bugs? For the use case when the user knows something has changed, he will try to reload until he actually sees the changes, and is a really bad user experience. It might be a good idea to do a UI review for memcache uses, to see how those will affect users' existing workflows, and some justification for whether it's worth the tradeoff, or not. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] HandOff to new Technical Architect
As per https://wiki.canonical.com/Launchpad/PolicyandProcess/HandOff, here's a list of projects I currently have in progress. I'm sending this to the list, since we don't know who will be the new Technical Architect. There are basically two projects in progress. = Architect Overview = The first one is the Architect Overview. It has been stalled for quite a while. There are discussions about this on this mailing list. The current status is that there's a branch, lp:~bjornt/launchpad/architect-vision that contains the start of this. It has the few first chapters mostly written. What's left to do is: * Land the branch * Make sure that tests are run, so that the examples are correct * Write the rest. There's a lot of cover, so it will have to be a team effort. The goal with this project is to provide a vision for the architecture, a list of what currently is there, how the different components of the architecture are used, and also to make some of our existing doctests visible. We have a few good doctests, but they aren't that useful at the moment, since they are hidden away. = ACLs = The second project is currently in progress, but there's still quite a lot to do; making projects private. I've tried to capture most of our thoughts around this in the References section of https://dev.launchpad.net/LEP/BetterPrivacy. There's also a branch up for review, lp:~bjornt/launchpad/acl-adapter, which contains the first part of the ACL system that can be landed. It also contains documentation for how to add new ACL classes. Looking at the MP is useful as well, since there was a useful discussion between me and Robert in there. The next step is to do https://dev.launchpad.net/LEP/BetterPrivacy/ACLImplementationForBugs. Abel will do that work, hopefully starting next week. I'll be around to mentor him doing this (although I'm away Monday and Tuesday next week). In the meantime, performance testing will need to be done. I tried to do that today, but due to forgetting granting access to the new table, and no LOSA at hand, I didn't get too far. I'll try to fit in at least some testing before I leave; I'll send notes about it later. For the testing, I did write a DB patch for BugTaskACL in lp:~bjornt/launchpad/bugtask-acl-db. Note that it won't apply cleanly, before bug 600934 is fixed. Before that, you need to delete duplicate entries in the BugSubscription table first. Also note that lp:~bjornt/launchpad/acl-adapter contains only what's needed for the bugtask ACLs. It's very basic, and handles only ACLs on one level. The lp:~bjornt/launchpad/privacy-spike has a proof-of-concept implemenation of multiple-level ACLs, with inheritance, and that needs to be merged to trunk at a later stage. I didn't do that now, since I'm a strong believer in doing things in small steps, and not land things that won't be used right away. Finally I should add that the ACL systems need to be landed with great care. It has the potential of slowing down LP, so each step requires performance testing to be made. It also has the potential of exposing private date, so care needs to be made in that aspect as well. Well, that's all for now, I won't be far away, so will be available for chats later on! -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Fwd: Diffamation release
On Tue, Jun 22, 2010 at 09:28:27AM +1000, Martin Pool wrote: Some of you might have seen the cool Diffamation demo at UDS. The code's now available, under a GPL-compatible licence. I don't know if that means it's AGPL compatible. Can you summarize what Diffamation is, for those of us who didn't see the demo? A get a 500 error looking at the project page. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Moving on... but not too far!
So, it's been more than 5 years... Launchpad has been, and still is, a great project to work on. But 5 years is a very long time, and I think it's time to finally move on, and get a change of environment. On July 5 I will be joining the Landscape team. I'm doing this change for two major reasons. The first is, as I said, to get a change of environment, to see how it's like to work in another group. The second reason is that I feel that I want to make a switch back to engineering, to get my hands dirty more often :) I won't write a long letter. I'm sure that we'll meet each other often enough :) I might even head down to the Epic for a few days! -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] [tech] ACLs: performance tests acceptable - moving on to implementing ACLs for bugs
To summarize the work on ACL so far, the performance tests that I did were acceptable, and the plan is to now take the proof-of-concept implementation and actually implement ACLs for bugs. We're still waiting on UI mock-ups. First, a few notes on performance. It's very important that the performance doesn't get degraded severely when we use ACLs. I've tried test the performance as good as I can locally, by creating a lot of bugs. My tests showed that we need to keep the 'private' attribue to keep the common case (public object) fast, and that using ACLs instead of BugSubscriptions was a bit slower. I do think we can get it equally fast on real data, though. My tests were a bit flawed, since I couldn't of course replicate real data and hardware. I've put up some notes on https://dev.launchpad.net/LEP/BetterPrivacy/ImplementationNotes. That's the notes from the pre-imp discussions we had, and it also contains notes about the performance tests I did. The next step is to use an ACL table to track bug privacy, instead of using the BugSubscription table. Abel will be helping me to get some knowledge transfered to the Bugs team. It will be done in small and careful steps, and no functionality will change. It's done so that we get something into the codebase, and so that it gets easier to do performance testing no real data. You can find out more about the implementation plans at https://dev.launchpad.net/LEP/BetterPrivacy/ACLImplementationForBugs. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] RFC: One True Way of addressing notification emails.
On Fri, Jun 11, 2010 at 10:51:43AM +0100, Graham Binns wrote: Agree, the statement is basically bug 47 says that Jane Doe changed me to confirmed/medium. This also works better if you eventually want to batch up multiple minor actions by multiple people. This would also give us a consistent fix for the problems of the reporter's name being absent on the first mail in a thread, or when the bug is retargeted into your subscription. I like this solution, and I'm not sure why I didn't think of it in the first place, frankly. So, the question is: is this something that we're happy to standardise on? So, to give my thoughts on this issue. We should certainly standardise, so that all e-mail notifications from Launchpad follow the same style. Extending BaseMailer to do this would be good. As to what to standardise on, that's yet to decide. I wouldn't change the way e-mail notifications look drastically, before talking to people that deal with a lot of bugmail. I suspect that they might not be too happy about us breaking their current workflow, so we should only do it if it makes a whole lot of sense. As to my personal preferences, then I think having the name of the person triggering the notification in the From header is really useful. I would hate seeing us remove that useful information and replace it with useless information. I don't care much whether the user's e-mail address is used, or the bug address. I don't think being able to easily reply to the user out-of-bug, or messing up people's address books are strong enough reasons not to do something. They are minor issues. If we go for the bug address, adding something like (via Bug #XXX) would be a good idea, to make it more clear. I would like to see the bug address in the To header, though, since it makes it more clear that the notification is sent to all bug subscribers, not only to me. As for the additional e-mail header you were planning to add, I think it's a good idea, but I agree with Robert, that it shouldn't be called -Sender. That gives the impression that the person did indeed send the mail himself. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] RFC: One True Way of addressing notification emails.
On Thu, Jun 10, 2010 at 11:25:05AM -0400, Curtis Hovey wrote: Answers does this: add_notification['From'] 'Sample Person question...@answers.launchpad.net' add_notification['Reply-To'] 'question...@answers.launchpad.net' The intent was to make it clear that the message was initiated by a user, but the context is the question. Well, not that important, but the original reason we (I) did this, was because the idea was that Answers should be used for commercial support, so we collected design criteria from that point of one. One of those were that they didn't want people to contact support staff out-of-band, and instead force it to go through the tickets. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [tech] ACL system
On Wed, May 26, 2010 at 10:26:33AM +1200, Tim Penhey wrote: Hi Bjorn (and other interested parties), I have a number of concerns, most of them around the fundamentals of how it will all be implemented, efficiency of queries, how this will work with collections and so on. Firstly though, I was a little confused by one of the first examples, and it certainly seemed counter-intuitive. If you need to limit who can view the project, you can grant permissions to specific people or teams:: project_acl.grant(VIEW, bob) project_acl.has(VIEW, alice) False project_acl.has(VIEW, bob) True project_acl.grant(VIEW, alice) project_acl.has(VIEW, alice) True Before this we had shown that everyone had view access to the project, including alice and bob. Granting view permission for bob should not effect alice. In order for alice to not see it, we should have either revoked alice's view permission, or revoked view for everyone. I actually had an explicit revoke(VIEW, EVERYONE) in there before, but since that is something that you have to do with the first grant, I removed it. It seemed better to consider VIEW, EVERYONE as a special case. Which brings me to another question. What happens if we have: view for everyone view revoked for alice Can alice see it because she is one of everyone? How is exclusion going to work. Is it that if the user gets visibility through any team, they get view permission? Yes. The current system tracks only who has permissions. It doesn't track who hasn't. So if you have VIEW for EVERYONE, you can't revoke permissions for specific users. You have to revoke the permission for everyone, and then add the users that should have access. It's a limitation, but it makes things easier, and none of our current use cases required granting to a group of people, except for X. What permissions are we talking about here? Just VIEW, and EDIT, or are we going for CRUD (create, read, update, delete)? What about admin and other special permissions? That hasn't been decided yet. The plan is to start with VIEW and MODIFY_ACL in the first step, and then add more as is needed. There's no built-in restriction on which permissions can be in there. (VIEW is used for controlling who can see an object, and MODIFY_ACL is used for controlling who can modify the ACLs.) Is this going to be a replacement for our security adapters or an enhancement to it? If it is an enhancement, do we know how yet? The first step is to have this to be on top (or part) of the security adapters. So we first check to see if the user is allowed by the ACL, and then the reqular adapters will check everything else, like who can write to the object, and so on. It was mentioned that subsystems (or applications) of Launchpad can have different ACLs, but no example is given. I'm curious as to how that'd be implemented too. I guess the best example here is to have a public project, but private branches. As I mentioned before, an object's ACLs are inherited by its children. For example, a project's branches can be considered to be part of the project itself, so they have the same ACLs as the project. I.e., we have a chain, Branch - Project. We we create a branch, we look at the branch's ACL parent, and use the same ACL as the parent. To have private branches, we need to override the ACLs for all the branches that will be created. IN order to do this, we insert a virtual object Branches in the chain, Branch - Branches - Project. We now override the ACL for Branches, to say that only a limited set of people have VIEW permissions. When we create a branch now, it will look at its ACL parent Branches, and use the same permission. Also, the example shows privacy on a bug task. How does this relate to the but itself? What if a bug has multiple tasks, but some are private? The bug itself doesn't have any ACL defined, since it doesn't have a single parent. If you have a private bug task, and add another bug task, that ones that will be able to view the new bug task, will also be able to see the bug description. They won't necessarily have access to see the old bug task, though, meaning that when they look at the bug, they will either not see the old bug task at all, or they will see that there is a bug task, but it won't see any information about it. Or they will be given permission to see the old bug task explicitly. It's up to the programmer to decide how to handle this. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [tech] ACL system
On Tue, May 25, 2010 at 10:05:28AM -0400, Gary Poster wrote: - Cacheing should be an increasingly important aspect of our performance concerns, particularly of fundamental design decisions like this one. I'd like to have some thoughts and discussion on how this plan affects cacheing (via something like memcache). The approach to searching you discuss, for instance, should be driven from, or at the least informed by, both user interface goals and performance concerns, IMO. Sure, I'd be happy to have a chat about this. - From an implementation perspective, is the intent that this will continue to be enforced with the current security proxy mechanism? (I would expect so, but I didn't see confirmation in the document.) Yes, we will still use the security proxies, and the proxies will query the ACL system. - You have this comment: Note that in order for this to work in practice, giving someone access to a bug only, means that they will also be allowed to see basic information about the project, for example the project name. That's a very good point. What's your expected approach to deal with this? I don't know yet. For this one I'm waiting for the UI mockups, and the requirements to be clarified. It depends on whether we just want to expose the name, so that the user can traverse to the object, expose a small subset of project data, or give access to most data in the project. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Which Postgres version to use for development?
Is there any reason why we shouldn't recommend people to use Postgres 8.4 when developing Launchpad? We currently have the situation that some people use 8.3, and some 8.4. This causes the problem that the diff for current.sql gets really huge when sampledata is re-generated. We should either decide which Postgres version people should use when generating current.sql, or make sure that the generation is the same, no matter which version is used. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] API issue moving branches
On Wed, May 12, 2010 at 08:55:51AM +0100, Julian Edwards wrote: On Monday 10 May 2010 11:39:24 Leonard Richardson wrote: I take Francis's general point that we should think about the API that will feel most natural to the end-user, rather than publishing our internal API directly. I find this quite hard to do since the API exposes our model objects. So, does our interal model API feel natural to us, but not to our users? I doubt it. If our model API isn't good, we should take the opportunity to improve it. When Francis said that distribution and product attributes are implementation details, I'd interpret that as they are *storage* implementation details. Even our internal API shouldn't expose those attributes as being writable. These should be written only inside the model object itself. If something outside the model object sets it, it should use the target attribute/method, which takes care of all the logic. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [tech] Converging the build and job systems (slowly and carefully)
On Thu, May 06, 2010 at 11:34:03AM -0400, Aaron Bentley wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, This is my first Architecture Team email, so if I'm doing it wrong, please let me know. I think you're doing great. Francis suggested bringing these concerns to this forum. We have two systems that, broadly speaking, have the same function: the job system and the build system. They both exist to provide a way for Launchpad to schedule and run processes, and gather their output. You might think that the build system's purpose is to build packages, but Soyuz already has put in a bunch of work to generalize it, and it's now able to perform builds for Rosetta that don't output a package. This is why it's under lp.buildmaster, not lp.soyuz. (Though shouldn't it be lp.services.buildmaster?) If I had to characterize the differences between the build system and the job system... 1. The build system always runs its code in a chroot, while the job system runs it natively. This means that the job system can have lower startup costs, but cannot build packages for other distros. 2. The build system can run virtualized, which means it can safely run untrusted code. 3. The build system distributes its work across multiple machines, using a central buildmaster. The Job system hasn't been tested on multiple machines, but is intended to use the database to allow multiple machines to coordinate. I think that we should be working to converge these systems, so that we can take advantage of their commonalities. This matches my impressions of the two systems as well, and I'm all for unifying the two systems. I don't know how much we can unify, but we'll find out. Both are proven, production systems that should be modified carefully. But we can start by converging their data models. Yes, that's a start. Another start would be to unify the APIs. Personally I find it more useful to have consistent APIs, than consistend data models, although they go a bit hand in hand. But I guess my point is that it's ok for them to work a bit differently, as long as the external APIs are consistent. Both the job system and the build system are big and complicated systems. I think it will be quite hard to just suggest things to be modified one step at a time, without looking at how we want it to look like in the end. At least for me :) If Code and Soyuz people have time at UDS, I would love to see a document creating, summarizing the important parts of the system, explain how they can be unified (for example, giving example of common methods), and list all the steps that need to happen. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [Branch ~launchpad-pqm/launchpad/db-devel] Rev 9322: [release-critical=bjornt][r=gary][bug=572077] Revert sendmail.py part
On Mon, May 03, 2010 at 09:46:44AM +1200, Tim Penhey wrote: On Sat, 01 May 2010 08:16:24 you wrote: Merge authors: Björn Tillenius (bjornt) revno: 9322 [merge] committer: Launchpad Patch Queue Manager launch...@pqm.canonical.com branch nick: launchpad timestamp: Fri 2010-04-30 21:13:26 +0100 message: [release-critical=bjornt][r=gary][bug=572077] Revert sendmail.py part of db-devel r9205, so that scripts don't use the zope mailers. Scripts need to be configured, and possibly rewritten, before we can use the zope mailers there. modified: lib/lp/services/mail/sendmail.py Hi Bjorn, As you probably know already, it was me who did this change. Yes. I forgot to send an e-mail about it, since it was Friday evening, and I had better things to do :) You might want to read a blog post of mine, although I hope you don't take it personal ;) (this was just an excellent real-world example) http://tillenius.me/blog/2010/05/03/problems-with-drive-by-fixes/ The primary reason for this was local testing. I wasn't aware of the full consequences of the change, and was trying to QA locally. How should we QA changes locally when dealing with script email? Unfortunately, it's quite difficult. You could log the mail, or the important parts of the mail. Another thing would be to do something special for the development config. I think that mailman has a test smtp server. It might be possible to look into using that one to store the sent mails somewhere. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] PQM closing at 2200 UTC Thursday
Hi, I'm your (not so) friendly release manager for this cycle. We're going to close PQM one day earlier this cycle, on Thursday, at 2200 UTC. The reason for this is that the rollout will be on Tuesday at 0900 UTC, so we need to close PQM earlier to get time to get things in shape in time. If you have something that needs to get in, that you think will not make it until the PQM deadline, please let me now as soon as possible. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Automatically setting the bug task status from the branch scanner
On Mon, Apr 12, 2010 at 08:01:24AM +0100, Graham Binns wrote: On 12 April 2010 06:04, Tim Penhey tim.pen...@canonical.com wrote: Hi Deryck, As discussed before I'm tackling the JFDI approach and I'm going to get the scanner to update the bug-task status [1]. I'd like the bug-task to be set to in-progress when it is linked to the branch, and fix-committed if it is linked to a branch that was marked as merged through a merge proposal. Since the scanner is a script, what do you think the best approach is so that the appropriate subscribers get notified? Â I had a brief look at the bug code but it was not clear to me exactly how to proceed. The simplest way I can think of, though not a particularly elegant one, is to do something like this: from lp.bugs.adapters.bugchange import BugTaskStatusChange change = BugTaskStatusChange( bug_task=bug_task, when=UTC_NOW, person=current_person, what_changed='status', old_value=old_bugtask_status.title, new_value=new_bugtask_status.title) bug_task.bug.addChange(change) However, I'd be inclined to say use whatever the API uses, but I don't actually know what that is (just from a bit of looking around at pre-8am I can't find it, anyway). The easiest way is probably to use IBug.setStatus(). It will do the right thing (including notifying people about changes), even if you pass in an IProductSeries. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [tech] Help creating the Architectural Vision!
On Wed, Mar 31, 2010 at 12:02:19PM -0300, Guilherme Salgado wrote: On Wed, 2010-03-31 at 12:59 +0300, Bjorn Tillenius wrote: Hi guys, so, it's time for the Technical Architect team to tackle its first task. Create the Launchpad Architectural Vision. First of all, what is it? It's aimed to be documents that describe the different parts of Launchpad. It will explain how Launchpad works, which moving parts are involved, and how it will fit together. It will also explain the various sub-systems, and explain how to do various things in Launchpad code. For example, how do I send and receive e-mail? How do I add an html page? How do I write a script? Etc. An important goal of the Vision is to avoid people doing duplicate work, so it should document that canonical way of doing various things in Launchpad. If you implement a new feature, and for example don't know how the implementation should look like, the Vision should be the place to look, to see what existing code you can re-use. That's why I want to focus on things that are shared (or supposed to be shared) across applications to start with. I need your help, though! It's a lot of work, and some of you have a lot of expertise of certain areas of Launchpad. I need help figuring out what should be included in the Vision. I also need volunteers describing the different parts. For example, someone from the Code team would be a good candidate writing about the job system. Although, not necessarily. If you're interested in how a certain part works, trying to document it is a great way of learning! Also, you don't have to volunteer yourself, finding other volunteers is also acceptable :) These are the things I thought of so far that should be included: * Overview of Launchpad and its moving parts * job system * e-mail (sending/receiving) * Database garbage collection (garbo) * expose API methods/attributes * Web app * Browser views * TAL formatters * Breadcrumbs * canonical_url * AJAX * Librarian * Comments * DB Schema Master/slave DBs and when/how to use them Vocabularies, maybe? Yes, all of these should be in there, thanks. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [tech] Help creating the Architectural Vision!
On Wed, Mar 31, 2010 at 04:48:54PM +0100, Jonathan Lange wrote: On Wed, Mar 31, 2010 at 4:25 PM, Henning Eggers henning.egg...@canonical.com wrote: Am 31.03.2010 11:59, schrieb Bjorn Tillenius: Hi guys, so, it's time for the Technical Architect team to tackle its first task. Create the Launchpad Architectural Vision. Without wanting to bikeshed here but when I first read Vision I thought this would be about how Launchpad will/should be in the future, not about how it is at the moment. Don't get me wrong, I think documenting the current architecture of Launchpad from a high-level perspective is a great idea but I just would not call it vision. Maybe view was intended? Overview? Launchpad Architectural Overview? That was my thought. We should have both :) Indeed, and that was the plan :) And yes, overview would be a great word. Sure, let's call it overview, and have the vision be a part of it. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [tech] Help creating the Architectural Vision!
On Thu, Apr 01, 2010 at 04:21:08PM +1300, Michael Hudson wrote: These are the things I thought of so far that should be included: * Overview of Launchpad and its moving parts * job system * e-mail (sending/receiving) * Database garbage collection (garbo) * expose API methods/attributes * Web app * Browser views * TAL formatters * Librarian * Comments * DB Schema * LaunchpadScript * LaunchpadCronScript * Security policy * SSH server While there is only one SSH server now, there will hopefully be two soon (a poppy replacement) so here I guess we should be clear and say the codehosting ssh server. I put the ssh server there, since jml mentioned on IRC that he had a branch that extracted the SSH server from code hosting :) * Code layout (what's in lp.code/lp.services/lp.registry/etc.) * Testing * Where to place tests * How to write tests * Layers Anything missing? Some bits of codehosting: * the puller * the scanner * the virtual file system used to access branches * code imports Also Soyuz seems to be missing entirely... Right. I didn't put anything application-specific in there. I wanted to concentrate on the things that are (or should be) shared across applications. I want it to list the things that are shared, and give guidelines on what else should be shared. Documenting the applications themselves would be good, but it's another project. Of course, some documention will be there, since an overview of how Launchpad works requires it. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [tech] Help creating the Architectural Vision!
On Thu, Apr 01, 2010 at 08:32:28AM -0600, Edwin Grubbs wrote: On Wed, Mar 31, 2010 at 3:59 AM, Bjorn Tillenius bj...@canonical.com wrote: Hi guys, so, it's time for the Technical Architect team to tackle its first task. Create the Launchpad Architectural Vision. First of all, what is it? It's aimed to be documents that describe the different parts of Launchpad. It will explain how Launchpad works, which moving parts are involved, and how it will fit together. It will also explain the various sub-systems, and explain how to do various things in Launchpad code. For example, how do I send and receive e-mail? How do I add an html page? How do I write a script? Etc. A summary of the moving parts and how they fit together is definitely something that is missing currently, however, there is a lot of existing documentation for the various sub-systems. Are we just trying to point the reader to that documentation, trying to explain things more tersely, or planning to move some of that existing documentation into the architect-vision directory? I really think we should avoid duplicating documentation. If existing doc tests are too verbose, we can break them into pieces. At this point, I'm not sure how much of the existing doctest we will move (or have the master document point to). I agree that we should avoid duplicating documentation, though. However, most of our doctests are not good documentation. If we can make them into good documentation, than that's fine. Let's see how things work out, and decide what to do later. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] [tech] Help creating the Architectural Vision!
Hi guys, so, it's time for the Technical Architect team to tackle its first task. Create the Launchpad Architectural Vision. First of all, what is it? It's aimed to be documents that describe the different parts of Launchpad. It will explain how Launchpad works, which moving parts are involved, and how it will fit together. It will also explain the various sub-systems, and explain how to do various things in Launchpad code. For example, how do I send and receive e-mail? How do I add an html page? How do I write a script? Etc. An important goal of the Vision is to avoid people doing duplicate work, so it should document that canonical way of doing various things in Launchpad. If you implement a new feature, and for example don't know how the implementation should look like, the Vision should be the place to look, to see what existing code you can re-use. That's why I want to focus on things that are shared (or supposed to be shared) across applications to start with. I need your help, though! It's a lot of work, and some of you have a lot of expertise of certain areas of Launchpad. I need help figuring out what should be included in the Vision. I also need volunteers describing the different parts. For example, someone from the Code team would be a good candidate writing about the job system. Although, not necessarily. If you're interested in how a certain part works, trying to document it is a great way of learning! Also, you don't have to volunteer yourself, finding other volunteers is also acceptable :) These are the things I thought of so far that should be included: * Overview of Launchpad and its moving parts * job system * e-mail (sending/receiving) * Database garbage collection (garbo) * expose API methods/attributes * Web app * Browser views * TAL formatters * Librarian * Comments * DB Schema * LaunchpadScript * LaunchpadCronScript * Security policy * SSH server * Code layout (what's in lp.code/lp.services/lp.registry/etc.) * Testing * Where to place tests * How to write tests * Layers Anything missing? So, how to go about writing the documentation? Well, to start with, you can check out lp:~bjornt/launchpad/architect-vision. In there, there is an 'architect-vision' dir in the root. If you install python-sphinx and run 'make html' in there, you get html files in .build/html that you can view in a browser. A couple of things to think about: * I don't want to give too many guidelines how to write. Let's do a set-based design, look at how different people format their documentations, and consilidate later * Please use doctests! It's important that the documentation is up to date. Doctests is a great way of doing this. The doctests shouldn't aim to test the feature, though. We have other tests for this. The aim for the doctest is to show examples, and to help keeping the documentation up to date. Running the doctests isn't hooked up yet, but I'll work on it next. * The generated HTML will be published somewhere for easy access. Volunteers for this task is also appreciated. * Documentation for Sphinx can be found at http://sphinx.pocoo.org/contents.html -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] [tech] Technical Architect Team Meeting Guidelines
Hi, We discussed earlier about how the Technical Architect team should discuss and coordinate things, and decided to try out an e-mail based approach. In short, e-mails with a subject tag of [tech], are topics that the team should pay extra attention too. I've written up some guidelines here: https://dev.launchpad.net/TechnicalArchitect/Team The most important things are to keep discussions short (switch to phone early), and to at the very least reply privately to me, confirming that you've read the topic. The latter is like the roll-call in a traditional meeting, to show that you don't ignore the mails. Expect a [tech] mail shortly. For simplicity, here's a copy of the section from the wiki page: == Meeting Guidelines == The team doesn't have scheduled meetings. Instead discussion and coordination happen on the `launchpad-dev` mailing list, and through phone calls. When a topic is meant for the team, the e-mail subject has a `[tech]` tag. Members of the team should pay special attention to such mail, and '''should reply promptly, even if it's just to say that they have no comments.''' It's important to answer, to acknowledge that you have read the mail. If all you want to do is to acknowledge, reply directly to the Technical Architect, to reduce noise on the list. E-mail discussion tend to be long-drawn and have high latency. To try to avoid this, please consider the following tips: * If the discussion has already gone on too long, or you feel that a subject will need a lot of discussion (more than 1-2 e-mails), '''arrange a phone call with interested parties''', and report back with the outcome of the discussion. * '''Predict which questions other people will ask you when reading your mail.''' Try to answer the questions before they are asked. * '''When asking questions, predict which follow-up questions you might ask when you get your answer.''' Ask these questions directly, together with the first question. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Blocked on upgrading subunit
On Mon, Mar 15, 2010 at 01:23:33PM +, Jonathan Lange wrote: On Sun, Mar 14, 2010 at 9:45 PM, Michael Hudson michael.hud...@canonical.com wrote: Jonathan Lange wrote: Hello, ... I'd like to upgrade zope.testing and take advantage of their better subunit support, but that requires upgrading subunit too. We currently maintain subunit as a sourcecode dependency. We manage it in the branch lp:~launchpad-pqm/subunit/trunk. That branch is a KnitPackRepository. subunit trunk is a 2a repository. If I naively change sourcedeps.conf like so: @@ -12,7 +12,7 @@ -subunit lp:~launchpad-pqm/subunit/trunk;revno=61 +subunit lp:~subunit/subunit/trunk;revno=120 I get incompatible repository errors. subunit is not a Python package. It's built with autotools, and thus making an egg for it is beyond my ken and maybe inappropriate. Which leaves me with a bunch of questions: 1. Changing sourcedeps.conf to point to a branch that's not managed by our PQM is OK, isn't it? After all, we still have to pass the tests to change the revno of the branch, so we aren't losing any safety afaict. As Max said, we are losing some safety. Not sure how much or how much we care. I think we don't care. I think we should care, since this opens up the possibility for people to change what is being run on our production systems, without us knowing about it. Sure, for this particular branch it should be safe. But it's easier to have a policy of point only to branches owned by ~launchpad-pqm, rather than point only to branches owned by ~launchpad-pqm, unless we trust the branch. The latters leaves an example that it's ok to point to foreign branches that we don't have any control of. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Remove spidermonkey-bin from launchpad-developer-dependencies?
On Wed, Mar 10, 2010 at 02:47:41AM +, Max Bowsher wrote: launchpad-developer-dependencies depends on spidermonkey-bin. This package has been removed from lucid. It provided one binary, smjs, a javascript shell. Grepping the Launchpad source finds nothing using this command. The relevant changelog entry in which this was added was: launchpad-dependencies (0.35) intrepid; urgency=low * New deps: python-cherrypy3 and spidermonkey-bin -- Launchpad package maintainers launch...@lists.canonical.com Wed, 15 Oct 2008 11:35:06 -0300 Does anyone know why it was added, and if we may remove it? Lazr-js uses it for jslint. It uses it indirectly, though. It uses /usr/bin/js, which points to /etc/alternatives/js, which points to /usr/bin/smjs. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Meeting time for Technical Architect Team
Hi guys, so, everyone in the To list is listed as being part of the Technical Architect team, and now we have to do the hard part and decide when to have a weekly meeting :) It's tricky, since we're spread across in a lot of different time zones. You can see the most extreme ones listed here: http://bit.ly/cgSpXq The most troublesome one is Michael :) So, Michael, during which hours would you be willing to have a meeting? I'm currently considering having the meeting on either UTC Tuesday or Thursday, since I don't have many meetings those days. Looking at the different times, it's impossible to have the meeting in a time that is convenient for everyone. I propose a couple of options. 1. Have the meeting at European evening/New Zealand morning every week. 2. Alternate the meeting times between having it like 1. one week, and European afternoon/US morning another week. 3. Experiment with e-mail based meeting/discussions. FWIW, I'm proposing to alternate the meeting times, since I'm not too keen on having it in the evening, after core hours. Better to have it only once every two week if necessary. This means that Michael wouldn't be able to participate in every meeting, but that's ok, since Code has two representatives. Let me clarify what I mean with 3. as well. The idea here is to send out the meeting agenda via e-mail once a week, at a fixed time. Each team representative would respond to information requests in a timely manner. The danger here is that there will be long, low-bandwidth, discussions. To address that, we would be strict at scheduling phone calls, so that interested parties (since often, it's sufficient for only a few people to be involved in the discussion) can discuss things in a higher bandwidth medium, reporting back the results. I have no idea how this would work in practice, but maybe worth an experiment. What do you think? -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Bug heat and the API
On Wed, Mar 03, 2010 at 08:02:27AM +1100, Robert Collins wrote: On Tue, 2010-03-02 at 11:23 -0600, Deryck Hodge wrote: I also think it would be nice to be able to mark an attribute as ignorable for the sake of calculating the etag when we decorate attributes for exporting. If status on a bug changes, that could matter to a script. If heat changes, not so much. Heat can't be set by an external script anyway. I wonder if read_only does this or could be made to do this? I asked Leonard on IRC and he wasn't sure. He's also very busy at the moment to dig further into the question. This seems a nice solution. But it doesn't solve it for attributes that can be set. Speaking from an HTTP point of view... derived aspects of an object that have *semantic* value should be included in the ETag; because the ETag controls caching. Read only has nothing to do with whether a change to a field should invalidate caches. This is a good point. The ETag should probably stay as it is, as it's used for caching. For PATCH commands, they supply If-Match to only patch the object they think they are starting from. Its possible server side to decide that: - the Etag is a recent one - only readonly fields have been changed since that etag was issued - so we can accept the patch Another solution, that might be simpler, is to have two tags. One ETag for caching, and another tag used for avoiding mid-air collisions. This might mean that we can't use the standard HTTP headers for mid-air-collision checking, but I don't see a big problem with that. I suggested a new tag, since I though Robert's suggestion would be quite hard to implement, without storing the generated Etags somewhere. But I guess it's possible to do what Robert suggested without keeping track of the Etags. For example, we could calculate to hashes, one for writable fields (let's call it WHASH) and one for read-only fields (let's call it RHASH). We would then concatenate WHASH and RHASH to get the Etag. When the server checks the If-Match Etag, it would only check the first part of the Etag, the WHASH. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Memcached API
Hi, I took a look at the current Memcached work, and saw the merge proposal for the TAL API. I took a look at the current API, and while it looks nice and easy to use, I can't really tell whether it's a good or bad API. I can't tell, since I don't really know what we want to use Memcached for. I know that we want to cache parts of the page, but that's pretty vague. I do know that one thing that we want to cache is the rendering of bug comments, since that's pretty safe to cache, and they are expensive to render. What else are we wanting to cache? When can we consider this done, from a Foundations point of view? Personally I'd like to at least see bug comments cached. There are a different ways this can be done, and I'd be interested to know how the current API design was decided upon. What other options were considered, and why were they abandoned? -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Formation of a Technical Architect team
Hi, to help me do my tasks properly, and to get more buy-in from Launchpad developers for designs and tech debt reduction, I'd like to form a Technical Architect team. This team will work similar to the UI/AJAX team that we have already. There should be one (maybe two where appropriate) member from each sub-team in Launchpad; Bugs, Code, Foundations, Registry, Soyuz, Translations. If you want to want to represent your team, please talk to your team lead. And team leads, please make sure to select one representative of your team until Thursday. The purpose of this team will include bringing attention to code that could need some love and refactoring, providing domain knowledge for design discussions, coordinating implementation within your team, and probably more as it gets discovered by time. This team will serve as a good point of coordination, but discussions will still happen with people outside the team as necessary, of course. As an example of what the team will be doing, the first task will be to look at the job system that the Code team built. The Bugs team has already started to move some of their tasks to the job system, and we'll see if it's feasible to have other teams doing the same. It's not yet decided exactly how the team will be run. We should have a weekly meeting of some sort, but I'd like to see who wants to participate, before deciding how and when the meetings will be, since it might be difficult due to time zones. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] using PermissiveSecurityPolicy when serving private xmlrpc requests
On Mon, Mar 01, 2010 at 07:17:25AM -0500, Jonathan Lange wrote: On Thu, Feb 25, 2010 at 3:50 PM, Michael Hudson michael.hud...@canonical.com wrote: Bjorn Tillenius wrote: ... So, I thought a bit more about this. I think it makes sense for the private XML-RPC server to work in a similar manner as scripts work. However, I do want us to get rid of the PermissiveSecurityPolicy, since it's a bit too permissive. Instead, I want the scrips (and private XML-RPC server) to set up an interaction using a special system user principal, or a principal more specific to the current task. The latter is better, but the former might be enough, at least to start with. The normal security policy can now look for this special principal, and give appropiate access. This all sounds reasonable to me. Being a bit more sophisticated about principals for the private XML-RPC server also sounds nice because there are some methods (translatePath, createBranch) which act on behalf of a Launchpad user and currently use the test helpers 'login_person' and 'logout', which is a bit ick. I guess this is a bit orthogonal, but I've been thinking about whether we could use the API for our internal services (such as codehosting). If we did, what would the security model look like? It would look pretty much how I described it. We would keep our existing web app security policy, and would have to define one or more system users, as which the internal systems would authenticate. It's a bit trickier here, since our web service API users don't have the same freedom as our internal API users. For example, scripts can do things on behalf of other users. When the link a branch, they can say which person should be registered as linking the branch. I think we would have to rework how the API works, in order for our internal systems to use it, but maybe not. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Faster TDD cycle: persist test services between test runs
On Mon, Mar 01, 2010 at 09:46:03PM +0700, Stuart Bishop wrote: On Mon, Mar 1, 2010 at 7:59 PM, Bjorn Tillenius bj...@canonical.com wrote: If you're into TDD, you're probably a bit annoyed about Librarian and Memcached taking quite a while to set up and tear down when running the tests, right? Well, these are things that we usually don't change, so there's not much point in keep starting and stopping them every time you run a test. I understand the Librarian being slow, but the memcached layer should be taking very little time to startup or shut down. If it is, there might be a problem. Could be. For me setting up MemcachedLayer is really quick, only 0.122 seconds, but tearing it down takes about 5 seconds. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Script for seeing which revisions have been QAed
Hi Ursula, could I ask you to write a script that tracks which revisions that have already been QAed OK? The script would be used in the new merge workflow that I would propose, but it's something that we could use even for the next rollout, to see which revision that would be safe to roll out. So having something ready until then would be great :) Let me explain how it should work. We need to know which revisions of devel/db-devel that have been marked as OK in the QA process. But more importantly, we want to avoid rolling out revisions that haven't been QAed, so we need to know which consecutive revions thave have been QAed OK. This is a bit hard to explain, so let me give you an example. Let say we have the following revisions and their QA statuses: r41: OK r42: OK r43: NEEDS QA r44: OK r45: NEEDS QA The script should tell us that r42 should be rolled out. The same is true if some of the revisions are bad: r41: OK r42: OK r43: BAD r44: OK r45: NEEDS QA We will only roll out r44 after r43 has been marked as OK. so if we land a fix for r43 in r46, we might have something like this: r41: OK r42: OK r43: RCFIXED (r46) r44: OK r45: NEEDS QA r46: OK In the example above, r42 is still the revision to roll out, since r43 should only be rolled out together with r46, which can't be rolled out wince r45 hasn't been QA yet. Do you understand more or less how it should work? We could have a call, if you need more clarification. The first step would be to simply tell which revision that is safe to roll out. The next step is to clearly show which revisions are blocking other revision from being rolled out. For example, in the last example, the QA for r45 blocks r43-r46 from being rolled out. A more common situation will be something like: r51: OK r52: NEEDS QA r53: OK r54: OK r55: OK r56: OK r57: OK r58: NEEDS QA r59: NEEDS QA r60: NEEDS QA In this scenario we need to draw attention to r52 and get it QA as soon as possible. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Script for seeing which revisions have been QAed
On Wed, Feb 24, 2010 at 05:06:23PM +0100, Danilo Šegan wrote: Hi Bjorn, Ursula, У сре, 24. 02 2010. у 15:28 +0200, Bjorn Tillenius пише: The script should tell us that r42 should be rolled out. The same is true if some of the revisions are bad: r41: OK r42: OK r43: BAD r44: OK r45: NEEDS QA We will only roll out r44 after r43 has been marked as OK. so if we land a fix for r43 in r46, we might have something like this: r41: OK r42: OK r43: RCFIXED (r46) r44: OK r45: NEEDS QA r46: OK In the example above, r42 is still the revision to roll out, since r43 should only be rolled out together with r46, which can't be rolled out wince r45 hasn't been QA yet. I think this is needlessly complicated. What we need to achieve can be simply achieved by landing everything up to the first 'NEEDS QA'/'BAD'. Otherwise, we'd be asking of Ursula to implement a complicated graph traversal algorithm that's very easy to break. Everything else we can solve with one of several policies: 1. use 'needs QA' until it's been QAd as OK (no use for 'BAD' then) 2. switch from 'BAD' to 'RCFIXED' only once the RC revision and all previous revisions have been QAd as OK I should have added that I don't care much how this is implemented, I just gave an example. Both of these options provide a very simple algorithm, with very minimal disadvantages compared to your proposal. However, the more important question here is how do we keep track of revisions and their QA statuses? Remember that we are in the process of switching to using tags for QA. FWIW, Ursula is busy with implementing bits and pieces of tags-for-QA, Yes, I agree that Ursula should finish the tags-for-QA. It's not worth implementing a system that works for both wiki and tags, when we're in the process of getting rid of the wiki. and I'd have to ask you to add your request to the backlog (on Translations kanban board). Sure, I could do that. Also, considering this is part of a bigger feature (continuous QA), I'd prefer if it was marked as such, and if people like Ursula, Diogo, Gary and me were all engaged in the design (because we are going to make sure it's delivered), or otherwise, you are going to be randomly asking people to implement something for what there might not be enough buy-in. I will certainly discuss this in more detail. This was just one thing that I was quite sure was needed, so I though it would be better to get things going already. I'm going to spend the next few days going over my MergeWorklowDraft proposal and start defining things in more detail. This will involve discussions with interested parties. This got me thinking, how do we keep track of features involving multiple teams using the Kanban boards? -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] using PermissiveSecurityPolicy when serving private xmlrpc requests
On Tue, Feb 23, 2010 at 05:56:47PM +1300, Michael Hudson wrote: Hi there. Today's hacking has involved working on methods implemented by the private xml-rpc server. If you've done this before, you'll know that it can be a bit annoying as calls to this server are not authenticated, so you end up having to weaken the security declarations a whole lot or use removeSecurityProxy liberally, neither of which feels very nice (I have complained about this before, I think). Can you give a list of all the methods that the private XML-RPC server currently implements? -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] uploading to the librarian without a database connection?
On Thu, Feb 18, 2010 at 05:54:58PM +1300, Michael Hudson wrote: It's all easy enough, apart from one thing: the need to upload the log file of the import to the librarian. These log files can get pretty large and staging them through the internal xml-rpc server just seems wasteful silly. Hence the question in the subject: is it possible to upload to the librarian without a database connection? It seems this is supported at the protocol level, but if there's a librarian client that works in this way I can't find it. Any advice is welcome! Even if it's don't do that :-) At the moment it looks like I'll have to write some more client code, but it's not like it's a very complicated protocol. Have you tried using ILibrarianClient.remoteAddFile()? In theory, that one should work. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] New merge workflow to keep features on edge until they have been QAed
On Fri, Feb 12, 2010 at 11:57:12AM -0500, Francis J. Lacoste wrote:
On February 12, 2010, Bjorn Tillenius wrote:
Hi,
at the team lead sprint we discussed how we could solve the problem of
unfinished features being pushed out to launchpad.net every time we do a
release.
I have a proposal how we can keep the development cycle separate from
our montly push out db changes cycle. I think using branches to handle
this would work well, so I'm proposing how we could change our merge
worfklow to make things better. There are still a few unknowns, but
let's start a discussion of what I have so far, before I try to refine
it.
I've created a LPE, to see how that works, which describes why we're
doing this:
https://dev.launchpad.net/LEP/ReleaseFeaturesWhenTheyAreDone
That link doesn't work for me.
Oops, seems I forget to actually save the page. It should work now.
To summarize, branches that contain db changes always get landed to the
`db` branch. Other changes get landed to the `edge` branch, although
branches that are part of a larger features (consisting of multiple
branches) get landed to edge through an integration branch for that
feature.
The landed revision stays in the `edge` branch until it has been marked
as being QAed. When the revision is marked as good, it automatically
gets merged into the `production` branch, which is rolled out to
`launchpad.net` daily.
I don't understand how you can land changes both bug fixes and feature
integration on the edge branch and how to deploy the QA revisions to
production. Bazaar doesn't support cherry-picking.
Right. I initially thought it would be ok, since usually bzr handles
that situation quite well. However, since we want to see which revisions
that haven't been pulled over yet, it won't work that well (unless we
keep track of it elsewhere).
I think we need an intermediate branch (devel) on which we land simple bug
fixes, and any stretch of QA revisions on that branch can be merged to
production.
Edge would be the result of devel + features integration branches.
Yeah. I was hoping to avoid adding a new branch, since it complicates
things, but it looks like we have to.
{{attachment:merge-workflow.png}}
== The QA step ==
A revision gets merged from `edge` to `production` only after it has
been marked as good in the QA step. This helps ensure that we don't
rollout feature to `launchpad.net` that haven't been QAed.
XXX: How the QA step works regarding to marking the revision as good/bad
is still undefined.
A revision property?
I don't think this would work, since AFAIK, you can't really change
revision properties after they have been committed. Or can you?
== Feature branches ==
Features that will involve multiple branches should have an integration
branch for that feature. Instead of landing branches related to the
feature to `edge`, they land their branches to the integration branch.
New revisions in the integration branch will automatically be merge into
the `edge` branch.
The reason for having an integration branch branch is to keep track of
all the revisions that belong to the feature, so that when the feature
has been marked as good in the QA process, the whole feature can be
rolled out at once. Parts of the feature can still be rolled out by
QAing specific revisions.
XXX: How to register new feature branches, and how to merge to them, is
still undefined. Maybe a better way would be to use revision properties
to tag revisions as being part of a bigger feature, or use bug links?
Again, since bazaar doesn't support revision cherry-picking, we probably need
separate branch. I heard that bzr-builder could be use to merge a set of
branches automatically.
Yes, might be possible to use bzr-builder, I haven't looked into that
yet.
== What about `staging.launchpad.net` ==
Staging takes the `edge` branch and merges in the `db` branch.
I don't think this works because edge contains feature branches that are not
ready for roll-out. I think we want staging to be exactly what will be rolled-
out next month. So it should be the db branch + all the QA changes.
Ideally we should have both (one staging for lpnet and one for edge).
Staging is good for QAing the feature without modifying production data.
--
Björn Tillenius | https://launchpad.net/~bjornt
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] New merge workflow to keep features on edge until they have been QAed
Hi,
at the team lead sprint we discussed how we could solve the problem of
unfinished features being pushed out to launchpad.net every time we do a
release.
I have a proposal how we can keep the development cycle separate from
our montly push out db changes cycle. I think using branches to handle
this would work well, so I'm proposing how we could change our merge
worfklow to make things better. There are still a few unknowns, but
let's start a discussion of what I have so far, before I try to refine
it.
I've created a LPE, to see how that works, which describes why we're
doing this:
https://dev.launchpad.net/LEP/ReleaseFeaturesWhenTheyAreDone
I've also started to document how the new process would work, which is
what I want comments on:
https://dev.launchpad.net/MergeWorkflowDraft
There is a picture there, so I would advise you to look at the wiki
page, but I will include the text in this mail, to make it easier to
discuss.
I've chosen to include both the addition of a QA step before revisions
can be rolled out to production, and continuous rollouts to
launchpad.net. I think the QA step should remain, but we could choose
not to do continuous rollouts to begin with.
I'm away on Monday and Tuesday, but will be able to participate in the
discussion on Wednesday again. Francis might be able to clarify things,
even though I think this version of the proposal is a bit different from
what we have discussed. I'm very open to changes to the proposal that
reduces the complexity for the developers landing features.
Here's the text of MergeWorkflowDraft:
This is a proposal of how a developer branch flows through the different
production branches we have, after the developer branch is ready for
landing.
To summarize, branches that contain db changes always get landed to the
`db` branch. Other changes get landed to the `edge` branch, although
branches that are part of a larger features (consisting of multiple
branches) get landed to edge through an integration branch for that
feature.
The landed revision stays in the `edge` branch until it has been marked
as being QAed. When the revision is marked as good, it automatically
gets merged into the `production` branch, which is rolled out to
`launchpad.net` daily.
{{attachment:merge-workflow.png}}
== The QA step ==
A revision gets merged from `edge` to `production` only after it has
been marked as good in the QA step. This helps ensure that we don't
rollout feature to `launchpad.net` that haven't been QAed.
XXX: How the QA step works regarding to marking the revision as good/bad
is still undefined.
== Feature branches ==
Features that will involve multiple branches should have an integration
branch for that feature. Instead of landing branches related to the
feature to `edge`, they land their branches to the integration branch.
New revisions in the integration branch will automatically be merge into
the `edge` branch.
The reason for having an integration branch branch is to keep track of
all the revisions that belong to the feature, so that when the feature
has been marked as good in the QA process, the whole feature can be
rolled out at once. Parts of the feature can still be rolled out by
QAing specific revisions.
XXX: How to register new feature branches, and how to merge to them, is
still undefined. Maybe a better way would be to use revision properties
to tag revisions as being part of a bigger feature, or use bug links?
=== Resolving conflicts ===
If there's a conflict when automatically merging the feature branch into
the `edge` branch, someone has to manually merge the feature branch into
`edge` and resolve the conflicts.
== Bug fixes ==
Bug fixes for features that already exposed on `launchpad.net` can go
directly to the `edge` branch. After the revision has been QAed, it gets
merged into the `production` branch.
== Continuous rollouts to `launchpad.net` ==
The `production` branch gets rolled out to the web app servers daily, to
expose our users to bug fixes not too long after the fix has been
verified to work on `edge.launchpad.net`.
We only do this for the web app servers, since we can update them
without any downtime.
XXX: If we're not ready for this, things could get merged into the `db`
branch after being QAed, waiting to get rolled out with the next
release.
== What about `staging.launchpad.net` ==
Staging takes the `edge` branch and merges in the `db` branch.
--
Björn Tillenius | https://launchpad.net/~bjornt
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] The with statement
On Wed, Jan 27, 2010 at 12:31:04PM -0500, Barry Warsaw wrote:
Now that we're on Python 2.5, we have a few new language features at our
disposal. In today's AMEU reviewer meeting, I brought up the 'with' statement
and was asked to write something up about it. The with statement will be okay
to use in Launchpad code, so you'll see more of it crop up, and it's good to
know how and when to use it.
First off, in Python 2.5 you need a future import to enable the with
statement. 'with' is a keyword in Python 2.6, so when we move to that version
we can get rid of all the future imports:
Thanks for the write-up, Barry. I think the with statement is good, but
I'd like to re-iterate one of Jeroen's concern (at least part of it, I
think).
For example, if you wanted to print every line in /etc/password, and you were
anal like me, you'd do something like:
fp = open('/etc/passwd')
try:
... for line in fp:
... sys.stdout.write(line)
... finally:
... # Don't wait for gc to free the file descriptor.
... fp.close()
Look at how much nicer this is using the with statement. This works because
built-in file objects already support the context manager protocol:
with open('/etc/passwd') as fp:
... for line in fp:
... sys.stdout.write(line)
I think this is an excellent use of with. It's well-known that files
work like this, so it's easy to understand, and the code gets cleaner.
It's also very easy to write your own context managers to work with the
with-statement. The documentation has all the gory details, but essentially,
you just need to implement an __enter__() method and an __exit__() method.
The former gets called when the with-statement starts, and the latter gets
called when it's done. The signature of __exit__() is a little tricky to
allow it to handle exceptions in the body of the with-statement, but usually
you don't care and will just let those percolate up.
think we should be a bit careful using it for other things than files.
I'm sure there are good use cases for using with with custom
implementations, but we should be careful not to over-use it. Just
because it's easy to write, doesn't mean that we should do it. It's very
easy to get carried away, wanting to use new features just because. It's
important to think about the end result.
So, when writing context manager implementations, you (and the reviewer)
should ask youself the question: how easy is it to see what's going on
when the with statement is used? The with statement does add a level of
magic. If you need to chase down the implementation to figure out what's
going on, it's not a good design. As I see it, the with statement is
mainly useful wehre you have a well-known behaviour, where it's easy to
find out what's going on (for example from the name), or when the
clean-up step isn't important. This doesn't mean that we should avoid
writing our own context managers at all cost, just that we should
evaluate the benefit. The more an object is used in the code, the more
likely it is that the behaviour will be well-known.
--
Björn Tillenius | https://launchpad.net/~bjornt
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Using HTTP basic auth on windmill tests
On Wed, Feb 10, 2010 at 04:09:52PM -0500, Gary Poster wrote: Nobody replied yet. +1 on using basic auth for me. I like having the test openid provider on a separate domain, and this limitation doesn't seem too bad, at least for now.. I'm also +1 on using basic auth. I don't think any of our Windmill tests include the login step as part of the workflow that is tested. The login is mostly test setup, so how we do it isn't important. As long as we have non-Windmill tests that make sure logging in without Javascript works, we should be fine. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] doctest size, refactoring, moving corner cases to unittests, etc
On Wed, Feb 10, 2010 at 09:17:36PM +, Gavin Panella wrote: Long doctests can accumulate large amounts of state. Object, test data and variable reuse causes headaches for developers and reviewers. This bothers me, so I brought it up at a reviewer's meeting a long long time ago: Because of the accumulation of state in doctests I think it would be good to limit new doctests to 200 lines or less. gmb wrote a new doctest last week of just under 200 lines and it was still perfectly comprehensible. That's where the number comes from. The consensus was that a 200 line limit wasn't the answer, but there were lots of other ideas, so let's discuss it. This is an excellent thing to discuss. In fact, this is one of the things that is on my plate to deal with during this cycle, so hearing what people think is interesting. I have some ideas on how to improve the situate, and reading the IRC log, it looks like at least some people. I don't know what the right thing to do is. I think this requires some experimentation, testing a few different strategies and see which one looks the best, and is most useful. My basic ideas so far is to make the doctests into actual documentation. For each major part of Launchpad, there should be a doctest describing the part. That doctest can link to other doctests which describes the part in more detail, or describes sub-parts. The doctest should be high-level, and not describe too much detail, preventing them from breaking due to trivial changes. More detailed tests could go into Python tests. This is the basic idea that I'm going to experiment with first. It might turn out it's not feasible. For example, good documentation is hard to write, and it's only feasible to write good documentation if someone actually reads it. So another challenge will be to actually make use of the documentation, providing in a way that people will find it useful, and actually read it. The less useful documentation proves to be, the less we should spend time writing it. That said, documentation can actually be useful for the one writing it. This should not be under-estimated. Explaining how things work, and most importantly explaining why things work the way they do, forces you to think about how easy things are to understand. If you have trouble explaining it, maybe your design is not so good. I know that I have realised this a few times, and I'm sure others would as well, if they made the effort. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Odd Windmill messages in buildbot logs
On Tue, Jan 26, 2010 at 04:04:04PM -0500, Gary Poster wrote:
(I am not regarding this as a release-critical problem, but I thought I
should raise this.)
Looking through the full buildbot results, I saw odd Windmill
errors-that-were-not-errors. For instance, look at this output
https://lpbuildbot.canonical.com/builders/db_lp/builds/456/steps/shell_7/logs/stdio
and look for the string test_results: ERROR.
This happens four times in the output, each time at the beginning of a
windmill suite. Here's an example from Registry, that I asked Curtis to
verify was not a problem as a spot check:
test_adding_milestone_on_addrelease_page
(lp.registry.windmill.tests.test_add_milestone.TestAddMilestone)test_results:
ERRORTest Failure in test {'version': '0.1', 'suite_name':
'AddMilestone', 'result': False, 'starttime': '2010-0-26T15:50:39.752Z',
'output': None, 'debug': 'Looking up name loginpage_submit_login, failed.',
'params': {'name': 'loginpage_submit_login', 'uuid':
'732522b4-0a64-11df-9cbc-12313b015491'}, 'endtime':
'2010-0-26T15:50:39.753Z', 'method': 'asserts.assertNode'}
The testrunner does not count this as an error.
The testrunner does not count this as an error, because it's not an
error. It's just Windmill logging stuff as ERROR, when it really
shouldn't. This ahappens when you use asserts to check something, for
example if a node is present. There is a parameter you can pass in to
make the assert not fail, which is fine, except that you get those log
messages that look like errors.
The fix is simply to tell Windmill not to log anything. If Paul is build
engineer next month, I believe this issue will be fixed :)
--
Björn Tillenius | https://launchpad.net/~bjornt
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Backing out the release and close bugs feature
On Wed, Jan 27, 2010 at 06:09:43PM -0500, Curtis Hovey wrote: The oops on edge indicate that we cannot make mass changes to bugs because launchpad cannot build the list subscribers who should be notified of the changes. While notify() is sending the signals to update bug history, user karma, and bug subscribers, all this work must happen in the time of the request. Could you show some sample oopses? We've had similar issues in the past, which we managed to resolve without doing things asynchronous. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] make check is broken
On Thu, Jan 21, 2010 at 12:31:13PM +, Julian Edwards wrote: Do you think it's acceptable that some parts of the UI might be totally un-usable? If the JS doesn't work then it should fall back to the plain pages, so I wouldn't call that unusable. It may not fall back automatically of course, but at least there *are* fallbacks. Just like broken broken should fallback to something that works? I'm talking about the case where we don't know that that JS is broken. If we know it's broken, it's easy to fall back on something. But we certainly don't have anything in place that catches any JS errors, ensure that everything got hooked up properly, and undoes all the JS. I don't think such a foolproof system is possible. How do you propose we find all these issues that needs to be fixed, before the tests are aready? The only issues so far is this issue, which you are the only one to see, and the issue that Paul and Tim reported. I'm looking at the latter now (which I hadn't seen before, even though I ran the Windmill tests quite a lot), and I have no way of reproducing your issue. How about we all ship and and make the tests ready, if they aren't already? My problem is that most of Soyuz has nothing to do with UI. It's extremely frustrating to be unable to land a change to the buildd manager, for example, because some JS is failing. I understand this is not exactly part of the stop the line ethos, but while we have these distinct parts that are not in the same line, it's bad for my blood pressure. Well, sure I can see how that is frustrating. Just as frustrating for non-Soyuz people that we're currently in testfix mode due to a failing Soyuz test. I thought that having a separate JS buildbot was far better. The only thing wrong was that we didn't have a process in place for someone to fix the test failures when they happened. And we didn't have a process that stops the edge rollout (and production rollouts of course) when this happens, to avoid having broken UI being shipped to our users. I don't think a process that is non-blocking will work that well. And to be far, how many times have Windmill stopped the process? So far it's only stopped people from landing JS changes, which itself is bad, but there's only two people so far, and we kind of have a workaround (until we fix it). I'm not really counting your problem, since you're the only experiencing it, and you have the option to either use EC2 or not use xvfb, exluding the Windmill tests (if you're working on code that certainly isn't used in UI, which excludes Soyuz model code). Perhaps I should consider splitting some of the non-UI Soyuz stuff into a separate code base? This might be an interesting thing to do. Not only for this issue, but to make the test suite faster to run in the common case. If it's easy, I'd like to see an example of how you would do it. Please note that your work may be thrown away, though, so don't do it unless it's easy. If it's harder, try to explain how things work instead. What would be included in the new package, and what depends on what, for example. In general it's a lot of work doing all this, so it might not be worth it. It requires changes in our buildout setup. For example, I assume that some of the non-UI still would depend on other things in LP? That means that whenever the normal code base changes, the tests for the new code base would have to be run and possibly put us into testfix mode. Having a separate code base also means it's harder to move code between the code bases. I.e., having a separate causes quite a lot of complications, so we'll only do it if there are major gains. I'm still interested in a proposal how the new code base would look like. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] make check is broken
On Wed, Jan 20, 2010 at 03:16:42PM +, Julian Edwards wrote:
On Wednesday 20 January 2010 14:32:48 Maris Fogels wrote:
Ok, that is really strange. xvfb is used to run Firefox in a headless
environment, and should only by run by buildbot.
Looks like moving the the JS continuous integration onto mainline pulled
those builder-specific changes along with them.
One option: check for a DISPLAY environment variable. If it exists, then
you are not the buildbot! And you don't need xvfb. If it does exist,
then use the xvfb harness. Sound good?
Actually I would prefer it always uses xvfb by default - that way I don't get
firefox popping up and down and getting in my way. It's rather annoying!
You
probably want a switch so it can run like that though, if you want to watch
it.
Agreed. My plan is to use xvfb be default, but allowing you to not using
it, in case you need to debug what's going on. Having Firefox pop up
suddenly is really annoying.
Another thing I just noticed, I am running the tests on a separate box of
mine
that I am ssh'ed into. It had this error when I was running with bin/test:
{{{
Set up canonical.testing.layers.BaseWindmillLayer in 0.000 seconds.
X11 connection rejected because of wrong authentication.
Set up lp.bugs.windmill.testing.BugsWindmillLayerError: cannot open
display:
localhost:10.0
I would say that this is a setup issue on your box. But then you said
that a new realease fixed it. Can you point to the changelog?
And it seems stuck there. :(
I second a another email that said js tests are not ready to block landings
yet.
Do you think it's acceptable that some parts of the UI might be totally
un-usable?
How do you propose we find all these issues that needs to be
fixed, before the tests are aready? The only issues so far is this
issue, which you are the only one to see, and the issue that Paul and
Tim reported. I'm looking at the latter now (which I hadn't seen before,
even though I ran the Windmill tests quite a lot), and I have no way of
reproducing your issue. How about we all ship and and make the tests
ready, if they aren't already?
--
Björn Tillenius | https://launchpad.net/~bjornt
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] OK, I'm stumped
On Wed, Jan 20, 2010 at 06:21:37PM +1300, Bjorn Tillenius wrote: On Tue, Jan 19, 2010 at 10:44:16AM -0700, Paul Hummer wrote: On Mon, 18 Jan 2010 22:03:16 +1300 Tim Penhey tim.pen...@canonical.com wrote: I have a branch that just modifies some javascript. lp:~thumper/launchpad/js-play I'm trying to land this on devel, but it fails 42 windmill tests running through ec2 (I'm pretty sure this is all of them). However the tests pass locally. I'm willing to bet they are the same ones that are failing on me in ec2. It fails on all sorts of commands, including client.asserts.assertProperty and client.waits.forElement. They also fail locally, unless I run the tests with bin/test -D, and then they don't fail. That's the real pisser. It indicates to me that there's a timing issue on tests, and my change just changed the timing issue. I'm tempted to just land this branch. None of the tests are tests that touch code that I've touched and I'm also stumped. I took a look at this today. I will continue tomorrow, unless someone else takes figures out what's going on here while I'm asleep. I haven't found the real issue, but I did find some interesting results. If I run the test in latest devel (r10197), everything is fine. If I then apply the patch of the changes in Tim's branch, the test runner suddenly takes 100% CPU on my machine while running the Windmill tests. This is odd, since the patch is JS only. I've done a profiling run of the tests with the patch applied. I haven't had time to analyze the results, but if someone is up for it, you can find the .prof file here: https://devpad.canonical.com/~bjorn/r10197-with-js-play-patch.prof The patch that I applied is here: https://devpad.canonical.com/~bjorn/js-play.patch So, I still haven't figured out what's wrong yet; I'm still investigating. I've noticed one thing, though. Without Tim's patch, the generated launchpad.js is 511 870 bytes. Slightly less then 500 kb. With Tim's patch, launchpad.js is now 512625. Slightly more than 500 kb. In fact, I ran some test just now to confirm my theory. I took a plain launchpad.js and added whitespace to it. When it's 512 000 bytes, everything is thing. If it's 512 001 bytes, the test runner uses 100% CPU. So something is becoming very inefficient when dealing with files larger than 512 000 (500*1024) bytes. Does this magic number ring any bells? BTW, if you want to get some JS patch landed, a workaround could be to find JS code that we don't use, and exclude it when generating launchpad.js to make it smaller. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Tired of running utilities/link-external-sourcecode manually?
On Wed, Jan 13, 2010 at 04:20:08AM +, Julian Edwards wrote: On Monday 11 January 2010 22:34:08 Bjorn Tillenius wrote: I'd say that if you need a script to make a branch, something is broken. Why do we have rocketfuel-branch then? (Genuine question) I don't know. I've never used it. I think it used to be more complicated, but now it only does: bzr branch trunk new-branch cd new-branch utilities/link-external-sourcecode make With my fix, the call to link-external-sourcecode isn't needed anymore. Assuming that 'bzr branch' is fast enough, which is should be, and is for me, then this script isn't needed. Could be that 'bzr branch' should cd to the created branch be default (or at least have an option to do it.) I don't want to run make, unless I need to. Sometimes I don't run make for new branches, so I rather do that manually (and I use 'make tags', not 'make') I would argue for that this script should be deleted, but if others still think it's useful, I'm not going to stop them from using it. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Tired of running utilities/link-external-sourcecode manually?
Are you also tired of always having to run utilities/link-external-sourcecode manually when creating new branch? Well, you don't have to do it anymore. If you make sure that the LP_SOURCEDEPS_PATH environment variable is set to something like ~/canonical/launchpad/trunk/sourcecode (where sourcecode contains populated download-cache and eggs directories), 'make' will automatically run utilities/link-external-sourcecode $LP_SOURCEDEPS_PATH/..' for you. I chose to use LP_SOURCEDEPS_PATH which points to the sourcecode/ directory, since that's what rocketfuel-get uses, so it should be possible to share the variable between the scripts. If something doesn't work for you, feel free to fix it! This landed in r10138. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Windmill tests enabled again (maybe)
On Thu, Jan 07, 2010 at 10:06:26PM +0100, Bjorn Tillenius wrote: Hi, just a heads up that Windmill hopefully will run in the buildbot slaves soon again. I have a branch playing in ec2 that enables them, and disables the tests that are currently playing. Hopefully it will pass and land. The test suite did pass before, except for some Windmill tests that now have been disabled. This has now landed, and it has even passed a buildbot run, so it's now in stable! -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Windmill tests enabled again (maybe)
Hi, just a heads up that Windmill hopefully will run in the buildbot slaves soon again. I have a branch playing in ec2 that enables them, and disables the tests that are currently playing. Hopefully it will pass and land. The test suite did pass before, except for some Windmill tests that now have been disabled. I won't be around to deal with any possible fallouts. I'm boarding a plane now, and will arrive at some point on Saturday. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Windmill test failures will now put us into testfix mode!
So, finally, when a Windmill test fail, it will cause us to enter testfix mode, and someone will have to fix it! I'm sure that we will see some spuriuos problems, so that we'll enter testfix mode, even though we shouldn't. Let's work together to fix those issues. I'm sure that these issues will be less painful than it was to make sure that the jscheck builder was green, and a lot less painful then when we started to use buildbot! I wrote a blog post to mark the occasion, with some thoughts about why blocking on test failures is a really good idea: http://tillenius.me/blog/2009/12/21/block-on-test-failures/ -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Who's not following planet.launchpad.net?
Who here doesn't read planet.launchpad.net on a regular basis? I'm guessing quite a few, since we haven't really advertised it. It's starting to get some interesting posts, and I think everyone involved with Launchpad development should read it. If you don't plan to start reading it, I would like to know why. The reason I'm asking is that I'm planning to write regular blog posts to channel my thoughts, and keep you guys updated with what I'm working on. I want to try to use blog posts for this, rather than e-mails to the list, since I want to try to medium, to avoid it getting lost, and to get a wider audience than the core Launchpad developers. I would suggest that you subscribed planet.launchpad.net to follow my (and other's) posts. If you want a more focused view, you can subscribe to only my posts: http://tillenius.me/feeds/atom/tag/launchpad/. If you for some reason have something against reading blog posts, please let me know, so I can try to accomodate you somehow. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Where to put security-related code?
On Wed, Dec 16, 2009 at 10:29:53AM +0100, Henning Eggers wrote: Please consider a method called setStatus(new_status) on a model class. This is a fairly common pattern as many objects in the model have some sort of status attribute and you want to control access to it [1]. Using the zcml and permission checkers in security.py you can control who can access (i.e. call) setStatus on the class but you cannot control which parameters may be passed into setStatus, i.e. which statuses the current user may set or not. This is a known limitation of the zope security model, I guess. Well, kind of. The zope security machinery handles only attribute getting and setting. The current practice in Launchpad code is to pass the requesting user into the method, i.e. setStatus(new_status, user) and let the model method do the checking. I found that this results in a lot of repetitive code between the model and security.py. The checks are similar, mostly testing user.inTeam(...) and require boiler plate code like getUtility(ILaunchpadCelebrities). I tried to reduce some of that code duplication by introducing security_helpers.py which has simple methods like is_admin_or_rosetta_expert(user) which can both be used by the checkers in security.py and the model code for parameter value enforcement. The file security_helpers.py is currently located in lp/translation/utilities but I already have a branch here that moves it to lp/services for more general use. That location is still up to debate, though. Adi suggested today that it would be less confusing if we had all the security related code in security.py and use the checkers from model code, i.e. calling something like OnlyRosettaExpertsAndAdmins(self).checkAuthenticated(user). I think you're on a slipperly slope here. Basically what you want to do is to replace things like (using pseudo code here): if user_is_admin() or user_is_rosetta_expert(): do_things() with: if user_is_admin_or_rosetta_expert(): do_things() I.e., you want to take an if condition and replace it with a function. Instead of using Python's 'or' key word, you want to construct text strings. Your example is simple, but what happens when you need to add a condition? You have to add another function, which has the same name, but with '_or_new_condition' appended to it. It quickly leads to names like EditDistributionMirrorByOwnerOrDistroOwnerOrMirrorAdminsOrAdmins, which is hard to read, and you can't wrap the line, if it gets too long. You also have the issue that some checkers need a specific context object, and will check the properties of that object. Let's keep the canonical.launchpad.security separate, and let's not depend on it. It's the security policy that is used by the web app mainly. For example, in scripts, we don't use it, and shouldn't have to import it really. I don't think it provides a nice API for the checks that you want to do. Sometimes it makes sense to share code. What we usally do is to put a method on the model class, which the security policy can use. See PublicToAllOrPrivateToExplicitSubscribersForBugTask for an example. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Can we drop 'ssh' from launchpad-developer-dependencies?
On Mon, Dec 14, 2009 at 11:36:35AM -0200, Guilherme Salgado wrote: On Mon, 2009-12-14 at 16:46 +1300, Michael Hudson wrote: Tim Penhey wrote: On Sat, 12 Dec 2009 02:33:43 Guilherme Salgado wrote: https://bugs.edge.launchpad.net/launchpad/+bug/495454 If not, can we just replace it with openssh-client? I defer to Michael. Michael, can we remove the dependency? The dependency was added in this change: http://bazaar.launchpad.net/~launchpad-committers/meta-lp-deps/trunk/revision/17 Thanks for reminding me I was the one to add that dependency; I just wish I could remember why I did so. I vaguely remember that it was added since rocketfuel-setup used ssh to connect to devpad, so ssh was considered a developer dependency. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Regarding Email Notifications.
On Wed, Dec 09, 2009 at 10:21:33AM -0300, Martin Albisetti wrote: On Wed, Dec 9, 2009 at 10:11 AM, Deryck Hodge deryck.ho...@canonical.com wrote: I was under the impression that there wasn't consensus about this I'm not saying there is, it was mostly an opinion really. At this point, I'm sure you've talked to more people about this than I have. Sorry for being to absolutist about a hand-wavy thing :) Actually, there kind of was consensus about this. At least at one point we decided to JFDI, and fix it. We did talk to several people who did find the notifications to be of somewhat use, but it was more important to stop the e-mail floods. There's even a test for this, TestBugChanges.test_marked_as_duplicate, what should make sure that only the people subscribed to the bug that is being marked as a duplicate is notified. If this isn't working, we need to take a look at that test, and find out why it's not working. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [Branch ~launchpad-pqm/launchpad/devel] Rev 10012: [r=mwhudson][ui=none] Allow listing sftp directories on
On Wed, Dec 09, 2009 at 02:33:19PM -, nore...@launchpad.net wrote:
Merge authors:
Aaron Bentley (abentley)
Related merge proposals:
https://code.launchpad.net/~abentley/launchpad/sftp-ls/+merge/15580
proposed by: Aaron Bentley (abentley)
review: Approve - Michael Hudson (mwhudson)
revno: 10012 [merge]
committer: Launchpad Patch Queue Manager launch...@pqm.canonical.com
branch nick: launchpad
timestamp: Wed 2009-12-09 14:30:52 +
message:
[r=mwhudson][ui=none] Allow listing sftp directories on
MemoryTransport
modified:
lib/lp/codehosting/sftp.py
lib/lp/codehosting/tests/test_sftp.py
=== modified file 'lib/lp/codehosting/sftp.py'
--- lib/lp/codehosting/sftp.py2009-10-29 00:51:48 +
+++ lib/lp/codehosting/sftp.py2009-11-27 21:59:26 +
@@ -20,6 +20,7 @@
]
+from copy import copy
import errno
import os
import stat
@@ -319,6 +320,10 @@
for stat_result, filename in zip(stat_results, filenames):
shortname = urlutils.unescape(filename).encode('utf-8')
+stat_result = copy(stat_result)
+for attribute in ['st_uid', 'st_gid', 'st_mtime', 'st_nlink']:
+if getattr(stat_result, attribute, None) is None:
+setattr(stat_result, attribute, 0)
This code is very unobvious to me. Why do you have to replace None with
0? A comment would be useful here.
longname = lsLine(shortname, stat_result)
attr_dict = self._translate_stat(stat_result)
yield (shortname, longname, attr_dict)
=== modified file 'lib/lp/codehosting/tests/test_sftp.py'
--- lib/lp/codehosting/tests/test_sftp.py 2009-10-31 12:07:16 +
+++ lib/lp/codehosting/tests/test_sftp.py 2009-12-08 16:24:12 +
@@ -3,12 +3,15 @@
Tests for the transport-backed SFTP server implementation.
+from __future__ import with_statement
+from contextlib import closing
import os
import unittest
from bzrlib.tests import TestCaseInTempDir
from bzrlib import errors as bzr_errors
from bzrlib.transport import get_transport
+from bzrlib.transport.memory import MemoryTransport
from bzrlib import urlutils
from twisted.conch.ssh import filetransfer
@@ -579,6 +582,39 @@
deferred = self.sftp_server.openDirectory(nonexistent)
return self.assertFailure(deferred, filetransfer.SFTPError)
+def test_openDirectoryMemory(self):
+openDirectory works on MemoryTransport.
+transport = MemoryTransport()
+transport.put_bytes('hello', 'hello')
+sftp_server = TransportSFTPServer(AsyncTransport(transport))
+deferred = sftp_server.openDirectory('.')
+def check_directory(directory):
+with closing(directory):
+names = [entry[0] for entry in directory]
+self.assertEqual(['hello'], names)
+return deferred.addCallback(check_directory)
+
+def test__format_directory_entries_with_MemoryStat(self):
+format_directory_entries works with MemoryStat.
+
+MemoryStat lacks many fields, but format_directory_entries works
+around that.
+
+t = MemoryTransport()
+stat_result = t.stat('.')
+entries = self.sftp_server._format_directory_entries(
+[stat_result], ['filename'])
+self.assertEqual(list(entries), [
+('filename', 'drwxr-xr-x0 00 0 '
+ 'Jan 01 1970 filename',
+ {'atime': 0,
+ 'gid': 0,
+ 'mtime': 0,
+ 'permissions': 16877,
+ 'size': 0,
+ 'uid': 0})])
+self.assertIs(None, getattr(stat_result, 'st_mtime', None))
What's this last assert for? Why don't you care whether stat_result has
an st_mtime attribute?
--
Björn Tillenius | https://launchpad.net/~bjornt
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [Branch ~launchpad-pqm/launchpad/devel] Rev 10012: [r=mwhudson][ui=none] Allow listing sftp directories on
On Wed, Dec 09, 2009 at 09:57:51AM -0500, Aaron Bentley wrote:
Bjorn Tillenius wrote:
+for attribute in ['st_uid', 'st_gid', 'st_mtime', 'st_nlink']:
+if getattr(stat_result, attribute, None) is None:
+setattr(stat_result, attribute, 0)
This code is very unobvious to me. Why do you have to replace None with
0?
I don't. None is not a valid value for any of the named attributes, so
it's a useful sentry. If getattr(stat_result, attribute, None) returns
None, the attribute did not exist at all.
Well, reading the code, it looks like it can be either None or not
exist. How do I know that it can't be None? It would be much clearer if
you either had a comment explaining it, or did:
missing = object()
if if getattr(stat_result, attribute, missing) is missing:
When the input is a MemoryStat, I have to add these attributes because
lsLine requires them to be set.
+def test__format_directory_entries_with_MemoryStat(self):
+format_directory_entries works with MemoryStat.
+
+MemoryStat lacks many fields, but format_directory_entries works
+around that.
+
+t = MemoryTransport()
+stat_result = t.stat('.')
+entries = self.sftp_server._format_directory_entries(
+[stat_result], ['filename'])
+self.assertEqual(list(entries), [
+('filename', 'drwxr-xr-x0 00 0 '
+ 'Jan 01 1970 filename',
+ {'atime': 0,
+ 'gid': 0,
+ 'mtime': 0,
+ 'permissions': 16877,
+ 'size': 0,
+ 'uid': 0})])
+self.assertIs(None, getattr(stat_result, 'st_mtime', None))
What's this last assert for? Why don't you care whether stat_result has
an st_mtime attribute?
I do care, very much. I want to to *not* have an st_mtime attribute,
because it didn't have one before it was passed into
_format_directory_entries, and _format_directory_entries should not
modify its input.
Same comment as above. It reads to me that you don't care whether it has
an attribute, or if it's None, either is fine. Again, either a comment,
or using the 'missing' pattern would make things much easier to
understand.
Also, why do you have this assert as the last check? I'm asking, since
somehow it feels natural to me to first check the pre-conditions, and
then make sure the code works, rather then first showing that the code
works, and then check the pre-conditions. For example, what happens if
the pre-condition fails? Where is the error reported? Of course, there's
also the question, do you need to check that that the attribute is
missing? What happens if stat_result suddenly would gain that attribute?
Does _format_directory_entries() start to fail?
--
Björn Tillenius | https://launchpad.net/~bjornt
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] process experiment proposal - move all our bugs to launchpad.net/launchpad
On Mon, Dec 07, 2009 at 10:02:11AM -0500, Barry Warsaw wrote: On Dec 07, 2009, at 11:55 AM, Tim Penhey wrote: My hunch is that people are already using tags in an almost rigorous way to model this. Yes we do, but I think we should still look to add components at some stage. I agree. I don't think tags are actually the right fit for this, though they're the best we have right now. Some things you could do once you have proper components: * auto-assign bugs in that component to a person/team for triage It depends on what you mean with auto-assign. I don't think the bug should actually be assigned to someone, rather a team/person would be responsible for a component, and would easily see which bugs he needs to triage. * get an overview of the general health or velocity of a component * manage structural subscriptions on a component level All what you say can be done using tags. For example, one thing we want to do is to offer structural subscriptions for tags. Seeing the general health or velocity for a tag would also be useful, and whan you have tag subscriptions, the auto-assign kind of work (without actual setting the assignee). So, again, what value does adding yet another data structure add, rather than to improve the way tags work? One of the reasons why I think project groups and projects doesn't work is because Launchpad isn't agile enough to let you narrow and widen your focus easily based on your task. Components can help this if done right. For example, sometimes I care about things project-wide, other times I care just about a particular component. Sometimes I care about my own personal artifacts, other times I want to see a community-defined slice through them (e.g. tags). You could substitute component with tag in the paragraph above, no? Or rather official tag, which aren't community defined. Using tags mixes too many orthogonal concepts together. For example, it's kind of silly to have tags like mailing-lists, ui, and trivial. They're great for letting community-driven organization emerge organically though. Why is it silly? 'ui' I can understand, it's way way too broad, or at least the way we've used it is silly, since it basically meant that a bug needed some ui changes, which a lot of bugs do. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] process experiment proposal - move all our bugs to launchpad.net/launchpad
On Tue, Dec 08, 2009 at 09:54:39AM +, Julian Edwards wrote: On Tuesday 08 December 2009 08:17:57 Bjorn Tillenius wrote: Why is it silly? 'ui' I can understand, it's way way too broad, or at least the way we've used it is silly, since it basically meant that a bug needed some ui changes, which a lot of bugs do. From the wiki page https://dev.launchpad.net/LaunchpadBugTags ui - Involves only template and/or browser code I find this a useful distinction, but maybe that's because Soyuz doesn't have many ui bugs compared to the rest. Yeah, it could be that it makes more sense for Soyuz. But still, you usually don't know that it involves only template and/or browser code, before you actually start coding. It often happens that something that is done in browser code today, is better to have in model code, so you end up moving the code, and the tag is now incorrect. Also, what do you use the tag for? Do you ever look at look at the list of ui bugs? Why? In what way does the ui bugs make sense as a group of bugs? -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] process experiment proposal - move all our bugs to launchpad.net/launchpad
On Tue, Dec 08, 2009 at 10:12:01AM +, Julian Edwards wrote: On Tuesday 08 December 2009 10:02:33 Bjorn Tillenius wrote: Yeah, it could be that it makes more sense for Soyuz. But still, you usually don't know that it involves only template and/or browser code, before you actually start coding. It often happens that something that is done in browser code today, is better to have in model code, so you end up moving the code, and the tag is now incorrect. I disagree there. I don't usually slap on a ui tag unless I've looked at the problem and analysed what needs to be done. Of course there may be exceptions as you describe but these are very rare. I think ui is actually more appropriate for template-only changes. Right. The issue here is that you use the tag in one way, and others use it in another way. The main issue is probably the name here, if it should be about template-only changes, template-only would be a better name. Also, what do you use the tag for? Do you ever look at look at the list of ui bugs? Why? In what way does the ui bugs make sense as a group of bugs? I only occasionally look at lists - but as it happens I am using them at the moment to find a variety of bug types for Jelmer. Its main use to me is as a quick indicator of the kind of work involved to fix the bug. This is a good use case. Sometimes we can also group the same kind of fixes into the same branch. This is less of a good use case. I'd say please don't group ui fixes together, unless they are highly related, it makes it harder to review the changes. Don't save work for yourself, by creating more work for the reviewers (including post-merge reviewers). -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] process experiment proposal - move all our bugs to launchpad.net/launchpad
On Tue, Dec 08, 2009 at 05:40:34PM +0700, Stuart Bishop wrote: On Tue, Dec 8, 2009 at 3:17 PM, Bjorn Tillenius bj...@canonical.com wrote: On Mon, Dec 07, 2009 at 10:02:11AM -0500, Barry Warsaw wrote: On Dec 07, 2009, at 11:55 AM, Tim Penhey wrote: My hunch is that people are already using tags in an almost rigorous way to model this. Yes we do, but I think we should still look to add components at some stage. I agree. I don't think tags are actually the right fit for this, though they're the best we have right now. Some things you could do once you have proper components: * auto-assign bugs in that component to a person/team for triage It depends on what you mean with auto-assign. I don't think the bug should actually be assigned to someone, rather a team/person would be responsible for a component, and would easily see which bugs he needs to triage. If you assign NEW bugs meeting criteria X to team Y, it is visible to everyone that team Y has chosen to be responsible to triage these bugs. If you just provide views so that people in team Y can easily see NEW bugs meeting criteria X, it is not visible to other people. We have a blank assignee despite the fact there are actually people responsible for moving the bugtask forward. Why are we hiding this rather important information? I'm not saying that we should hide the information. Today we do a terrible job at highlighting the triaging process. I'm just not convinced that overloading the assignee is the way to go here, though. Having a bug assigned to someone gives the message this bug belong to the assignee, don't you touch it!. I don't think that's a good message, since we want to encourage people to help triaging the bugs. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Running Windmill tests in buildbot and ec2
On Tue, Dec 08, 2009 at 02:54:43PM +1300, Michael Hudson wrote: Michael Hudson wrote: Bjorn Tillenius wrote: If you can get launchpad-developer-dependencies updated, I can do the EC2 test images on my Monday -- drop me a mail. You wouldn't happen to know which extra packages were installed to get it to work? I'm assuming xvfb needs to be there. But what about firefox, which packages where installed to get it to run? Plain 'firefox', or something else? My IRC logs suggest this was the magic line: apt-get -y install xvfb firefox xfonts-base Cool. The launchpad-developer-dependencies 0.59 now depends on the extra packages. If you could create new ec2 images, it would be great. If you could explain how to update the images, even better :) I first run a command like this: ./utilities/ec2 update-image --public launchpad-ec2test112 -m based-on:ami-e6f6158f Then test it and use ec2-modify-image-attributes to make it public. This is a bit inefficient as it builds an image completely from scratch rather than updating our existing one, but the code that does that doesn't actually upgrade the installed packages currently (I finally got around to filing a bug about this: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/493774 ). Anyone can run the above command of course, but only images created by myself, Gary or Francis are considered by ./utilities/ec2 for running currently. The new image is building. And is now public as machine version 112. I tried to run the windmill tests and it looked ok to me, although two failed; you can see my command and the output at http://pastebin.ubuntu.com/336965/ . Thanks! The two failures are kind of ok. They shouldn't fail, but the same two tests are failing in the jscheck builder currently. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [Branch ~launchpad-pqm/launchpad/devel] Rev 9989: [r=sinzui][ui=none][bug=297833] Improved error message when trying to
On Tue, Dec 08, 2009 at 09:44:09AM -0600, Edwin Grubbs wrote: @@ -779,16 +781,36 @@ return False -class PublicPersonChoice(Choice): +class PrivateTeamNotAllowed(ConstraintNotSatisfied): + __doc__ = _(A private team is not allowed.) + + +class PrivateMembershipTeamNotAllowed(ConstraintNotSatisfied): + __doc__ = _(A private-membership team is not allowed.) + + +class PersonChoice(Choice): + A person or team. + + This is useful as a superclass and provides a clearer error message than + Constraint not satisfied. + + implements(IReferenceChoice) + schema = IObject # Will be set to IPerson once IPerson is defined. + + +class PublicPersonChoice(PersonChoice): A person or team who is public. - implements(IReferenceChoice) - schema = IObject # Will be set to IPerson once IPerson is defined. def constraint(self, value): - return is_valid_public_person(value) - - -class ParticipatingPersonChoice(Choice): + if is_valid_public_person(value): + return True + else: + # The vocabulary prevents the revealing of private team names. + raise PrivateTeamNotAllowed(value) I was going to ask for some explicit tests for this field, and ask you what happens if you pass in an invalid public person to the validator. However, I did took a look as is_valid_public_person(), and it looks like the name isn't correct, it should be named is_public_person, no? Renaming it to is_public_person() would be fine. Cool, that makes things a bit cleaner. Do we really need functions to check the visibility attribute? The is_valid_public_person() and is_private_membership() are also used by the storm field validators validate_public_person() and validate_person_not_private_membership() found in lp/registry/interfaces/person.py. Although it is unlikely that these tests will ever change, it does make it possible to change the form validation and the model validation simultaneously. If it is worthwhile to keep, we should definitely add some comments so that we don't add code to those functions that is inappropriate for either level of validation. Do you think it is worthwhile to remove these functions? Hmm. Probably not. Ideally you should, or maybe move them to the model code instead, but it's probably not worth the trouble. It's also not clear to me, whether we need to check that the passed in value is an IPerson. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Running Windmill tests in buildbot and ec2
On Sat, Dec 05, 2009 at 11:13:06AM +1300, Michael Hudson wrote: Bjorn Tillenius wrote: On Fri, Dec 04, 2009 at 08:35:38AM +1300, Michael Hudson wrote: Bjorn Tillenius wrote: Hi, next week I'd like us to get rid of the jscheck buildbot builder, and instead run the Windmill tests in the normal devel and db-devel (and later also production-devel) builders. In order to do this, we need to update all the images. Michael, you did set up the jscheck builder, right? Well, the LOSAs set it up at my instruction. Can you make sure that the LOSAs know what needs to be added to the devel and db-devel images? The only difference is installing a few more packages, I think. Also, someone needs to updated the images we use to run tests on EC2. Who's responsible for that? Again I ask Michael, since you know what needs to be done, and have updated those images in the past. But please feel free to delegate to someone else :) If the extra dependencies become part of launchpad-developer-dependencies in the ~launchpad PPA, then updating the EC2 image is really really easy. And I think it would probably make updating the buildbot images easier too... Yeah, I was a bit hesitant about adding it to launchpad-developer-dependencies, since it's only used for testing on non-developer machines, but it's worth adding it anyway, to make it easier to update the images. And actually, after playing around with it a bit, I definitly want to run the Windmill tests headless locally as well. It's so much nicer not having Firefox pop up now and then when running 'make jscheck'. Well, we could have the launchpad-dependencies source package build another 'launchpad-headless-tests' package or something, but I don't think it's much extra on a developer system really. Right, the xvfb package is very small. Oh, and I guess PQM will have to be updated as well, since we currently run tests there for cherry picks, right? Ideally the images should be updated on Monday, when PQM is open again. I don't want to do it earlier, to avoid disrupting any possible rollout, and I don't want it done much later, since we need to get started running the tests early, so that we can fix any issues before the 3.1.12 rollout. Monday sounds ambitious but doable. Yeah, that's why I said 'ideally' :) I suspect it will take a bit longer to get everything updated. If you can get launchpad-developer-dependencies updated, I can do the EC2 test images on my Monday -- drop me a mail. You wouldn't happen to know which extra packages were installed to get it to work? I'm assuming xvfb needs to be there. But what about firefox, which packages where installed to get it to run? Plain 'firefox', or something else? My IRC logs suggest this was the magic line: apt-get -y install xvfb firefox xfonts-base Cool. The launchpad-developer-dependencies 0.59 now depends on the extra packages. If you could create new ec2 images, it would be great. If you could explain how to update the images, even better :) -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [Branch ~launchpad-pqm/launchpad/devel] Rev 9987: [r=noodles][ui=none][bug=230801] Fixed oops when renewing membership.
On Tue, Dec 08, 2009 at 02:46:13AM -, nore...@launchpad.net wrote:
Merge authors:
Edwin Grubbs (edwin-grubbs)
revno: 9987 [merge]
committer: Launchpad Patch Queue Manager launch...@pqm.canonical.com
branch nick: launchpad
timestamp: Tue 2009-12-08 02:44:20 +
message:
[r=noodles][ui=none][bug=230801] Fixed oops when renewing membership.
modified:
lib/lp/registry/browser/person.py
--
lp:launchpad/devel
https://code.launchpad.net/~launchpad-pqm/launchpad/devel
You are subscribed to branch lp:launchpad/devel.
To unsubscribe from this branch go to
https://code.launchpad.net/~launchpad-pqm/launchpad/devel/+edit-subscription.
=== modified file 'lib/lp/registry/browser/person.py'
--- lib/lp/registry/browser/person.py 2009-11-21 14:45:26 +
+++ lib/lp/registry/browser/person.py 2009-12-03 02:50:03 +
@@ -548,7 +548,10 @@
@action(_(Renew), name=renew)
def renew_action(self, action, data):
member = self.context.person
-member.renewTeamMembership(self.context.team)
+# This if-statement prevents an exception if the user
+# double clicks on the submit button.
+if self.context.canBeRenewedByMember():
+member.renewTeamMembership(self.context.team)
self.request.response.addInfoNotification(
_(Membership renewed until ${date}., mapping=dict(
date=self.context.dateexpires.strftime('%Y-%m-%d'
Why isn't there a test for this change?
--
Björn Tillenius | https://launchpad.net/~bjornt
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [Branch ~launchpad-pqm/launchpad/devel] Rev 9989: [r=sinzui][ui=none][bug=297833] Improved error message when trying to
On Tue, Dec 08, 2009 at 05:01:16AM -, nore...@launchpad.net wrote: Merge authors: Edwin Grubbs (edwin-grubbs) revno: 9989 [merge] committer: Launchpad Patch Queue Manager launch...@pqm.canonical.com branch nick: launchpad timestamp: Tue 2009-12-08 04:58:31 + message: [r=sinzui][ui=none][bug=297833] Improved error message when trying to add a private team as a member of another team. modified: lib/canonical/launchpad/fields/__init__.py lib/lp/registry/browser/person.py lib/lp/registry/browser/tests/team-views.txt lib/lp/registry/interfaces/person.py lib/lp/registry/stories/teammembership/xx-private-membership.txt lib/lp/registry/templates/team-add-my-teams.pt -- lp:launchpad/devel https://code.launchpad.net/~launchpad-pqm/launchpad/devel You are subscribed to branch lp:launchpad/devel. To unsubscribe from this branch go to https://code.launchpad.net/~launchpad-pqm/launchpad/devel/+edit-subscription. === modified file 'lib/canonical/launchpad/fields/__init__.py' --- lib/canonical/launchpad/fields/__init__.py2009-11-23 03:43:05 + +++ lib/canonical/launchpad/fields/__init__.py2009-12-01 22:09:05 + @@ -37,6 +37,8 @@ 'PasswordField', 'PillarAliases', 'PillarNameField', +'PrivateMembershipTeamNotAllowed', +'PrivateTeamNotAllowed', 'ProductBugTracker', 'ProductNameField', 'PublicPersonChoice', @@ -758,7 +760,7 @@ Return True if the person is public. from canonical.launchpad.interfaces import IPerson, PersonVisibility if not IPerson.providedBy(person): -raise ConstraintNotSatisfied(Expected a person.) +return False if person.visibility == PersonVisibility.PUBLIC: return True else: Not your code, but you could have updated the last lines to read: return person.visibility == PersonVisibility.PUBLIC @@ -770,7 +772,7 @@ True if the person/team has private membership visibility. from canonical.launchpad.interfaces import IPerson, PersonVisibility if not IPerson.providedBy(person): -raise ConstraintNotSatisfied(Expected a person.) +return False if person.visibility == PersonVisibility.PRIVATE_MEMBERSHIP: # PRIVATE_MEMBERSHIP. return True Same here. @@ -779,16 +781,36 @@ return False -class PublicPersonChoice(Choice): +class PrivateTeamNotAllowed(ConstraintNotSatisfied): +__doc__ = _(A private team is not allowed.) + + +class PrivateMembershipTeamNotAllowed(ConstraintNotSatisfied): +__doc__ = _(A private-membership team is not allowed.) + + +class PersonChoice(Choice): +A person or team. + +This is useful as a superclass and provides a clearer error message than +Constraint not satisfied. + +implements(IReferenceChoice) +schema = IObject# Will be set to IPerson once IPerson is defined. + + +class PublicPersonChoice(PersonChoice): A person or team who is public. -implements(IReferenceChoice) -schema = IObject# Will be set to IPerson once IPerson is defined. def constraint(self, value): -return is_valid_public_person(value) - - -class ParticipatingPersonChoice(Choice): +if is_valid_public_person(value): +return True +else: +# The vocabulary prevents the revealing of private team names. +raise PrivateTeamNotAllowed(value) I was going to ask for some explicit tests for this field, and ask you what happens if you pass in an invalid public person to the validator. However, I did took a look as is_valid_public_person(), and it looks like the name isn't correct, it should be named is_public_person, no? Do we really need functions to check the visibility attribute? + + +class ParticipatingPersonChoice(PersonChoice): A person or team who is not a private membership team. A person can participate in all contexts. A PRIVATE team can participate @@ -796,8 +818,10 @@ user. A PRIVATE MEMBERSHIP team is severely limited in the roles in which it can participate. -implements(IReferenceChoice) -schema = IObject# Will be set to IPerson once IPerson is defined. def constraint(self, value): -return not is_private_membership(value) +if not is_private_membership(value): +return True +else: +# The vocabulary prevents the revealing of private team names. +raise PrivateMembershipTeamNotAllowed(value) -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Reminder: Add outcome of IRC discussions to reviews
Hi all, when you're reviewing a branch and have a discussion with the coder on IRC, or on the phone, don't forget to add details about that discussion to the actual merge proposal. This could be a paragraph explaining what issues where brought up, and what the outcome was, or simply a paste of the IRC log. This is important, since when looking at merged revisions, I might find something that I don't understand. Sometimes that issue is mentioned in the review, and then it's easy for me to understand. But sometimes nothing is mentioned, or it's mentioned that the discussion happened on IRC. This means that I have no idea what was discussed, and I have to ask the coder the same question that the reviewer might have asked already. I know that some people already are good at capturing the conversation in the merge proposal; this is reminder to the rest to do the same -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Running Windmill tests in buildbot and ec2
Hi, next week I'd like us to get rid of the jscheck buildbot builder, and instead run the Windmill tests in the normal devel and db-devel (and later also production-devel) builders. In order to do this, we need to update all the images. Michael, you did set up the jscheck builder, right? Can you make sure that the LOSAs know what needs to be added to the devel and db-devel images? Also, someone needs to updated the images we use to run tests on EC2. Who's responsible for that? Again I ask Michael, since you know what needs to be done, and have updated those images in the past. But please feel free to delegate to someone else :) Oh, and I guess PQM will have to be updated as well, since we currently run tests there for cherry picks, right? Ideally the images should be updated on Monday, when PQM is open again. I don't want to do it earlier, to avoid disrupting any possible rollout, and I don't want it done much later, since we need to get started running the tests early, so that we can fix any issues before the 3.1.12 rollout. BTW, this might cause some pain in the beginning, but I'm hoping it will be less painful then the switch to buildbot was :) The Windmill suite is actually reasonably stable at the moment. There's one test that is consistently failing, that we'll have to either fix, or disable until we figure out why it's failing. Apart from that there are some spurious test failures from time to time, but putting us into testfix mode when that happens hopefully encourages someone to actually fix it! -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Launchpad Design Specs
On Wed, Dec 02, 2009 at 12:00:33PM -0700, Brian Fromme wrote: Hi, It's my opinion that we should have a more visible model for Launchpad design specifications. As a new person (I won't say developer, even though I could be one) to LP, I'd like to learn more about the system from a high-level before I dive into reading code. At the moment, I can't find any easy way to do this (yes, I know about Blueprints) and I can't find any LP process documents on the subject. I think we should be capturing our specifications in chunks, much like the Blueprint notion. We also should be writing enough detail and drawing diagrams to better explain the whole system to others. Then, I think we need to keep these specs in a visible location so people who don't know as much as we do can find them. What do you think? We have to be a bit careful here. We've tried this many times in the past, and they have failed. The thing is that people design things, implement them, and then change it. The design might even change during implementation, since you find out more about the problem then. It's actually a high cost of changing the design document when you change the system, since it's a context switch. This leads to stale documentation that is out of date, and confuses people more than it helps. The only way I see something like this working is if the documents live in the source tree, and are doctests so that it's natural (and you're forced to, since it will start to fail) to change the specification when doing changes to the system. Actually, it doesn't even have to be doctests. It should be documentation with tests. It would probably be possible to have a normal unittest test, exctract the comments, and produce a suitable document. We also have to come up with a good level of details. IMHO, these should be very high level, so that they won't need to be updated that often. The more detail you include, the harder it will be to write, and the harder it will be to write, the lower the quality will be. I think we've had enough experience in this field to know that it's better to have this very lightweight. If the process isn't lightweight it will fail. That's why I think that diagrams are pretty much out of the question, except for very simple ascii ones. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] bugtasks, context
On Wed, Dec 02, 2009 at 06:34:15PM -0300, Martin Albisetti wrote: So, bugtasks. I'd like to bring forward a problem that seems to be popping up more and more as we focus on breaching the gap: bugtasks and context. As it stands today, if you're looking at an upstream bug that also has a bugtask for a package in Ubuntu, and you want to nominate that bug, you can't. You need to start hacking around to get to the bug from the package context (if you manage to figure out that's what you need to do). A quick hack^fix would be to let people click on bugtasks and the context gets automagically changed. I wouldn't call this a quick hack, since it will require UI changes, and it will probably confuse users. My quick hack would be to display all the bugtasks' series when nominating a bug. That's a really simple hack, and should be quick. No UI changes are needed, it's only some Python code to maybe create a new vocabulary and make use of it. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] process experiment proposal - move all our bugs to launchpad.net/launchpad
On Fri, Nov 27, 2009 at 08:20:45AM +1100, Robert Collins wrote: Executive summary - Run an experiment where we work with our bugs on a single project rather than spread across 20-odd projects. If we don't like it, put the bug tasks back where they came from after the experiment. If we do like it, keep them on the single project :). The experiment would be a success if it becomes easier to see the overall launchpad bug priorities / more awareness of other subteam priorities and issues. Just a quick question before I go to sleep. Why do we have to move the bugs? That's rather disruptive, and not easily undone, and will cause some confusion. Why not simply use /launchpad-project instead? The bugs are already there, providing a view across all projects. From your success criteria it sounds like focusing on /launchpad-project would be enough, making the experiment more lightweight to execute. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] process experiment proposal - move all our bugs to launchpad.net/launchpad
On Fri, Nov 27, 2009 at 09:11:13AM +1100, Robert Collins wrote: On Thu, 2009-11-26 at 23:55 +0200, Bjorn Tillenius wrote: The experiment would be a success if it becomes easier to see the overall launchpad bug priorities / more awareness of other subteam priorities and issues. Just a quick question before I go to sleep. Why do we have to move the bugs? That's rather disruptive, and not easily undone, and will cause some confusion. Its easy to do and undo! Seriously, it is. for project in projects: for bug in project.bugs: if bug not in launchpadproject.bugs: output_bug() move_bug() How do you deal with bugs that are targeted to multiple LP projects? What about e-mail notifications? Anway, that's not so important. It's the disruptive part that is. I've spent a lot of time fine-tuning my e-mail filters for bug notifications. If you do this experiment, you're going to force me to do that again. I won't do it at once, I know that from experience. There will be a period where I basically ignore bug notifications, since it's not possible to follow them. Maybe we need to make everyone's life a pain to make progress, but the question is, is it worth it? I'd like us to refine the purpose of this experiment first. This is a really expensive experiment, since it will totally disrupt people's current workflow; they have to come up with a new one. People have already found the need to get a more focused view of the project's bugs. I don't think we have a good alternative at the moment. We currently use projects to do the filtering, but by doing this experiment, we problably would switch to filtering on tags instead. Practically that doesn't make any difference. People would probably do what they did previously; it would just take a while to get there, and some other (like milestone) tracking for a team would be lost. Personally, I think a more useful experiment would be to try out different hypothesis, which doesn't make the whole team unproductive for a few cycles. One hypothesis is that we need to get views for a single team. Let's implement views for teams for milestones, assigned bugs, subscribed bugs, and so on. Then have one team try it out and see how it works. We could do this in /launchpad-project, since this feature should work for both projects and project groups. Why not simply use /launchpad-project instead? The bugs are already there, providing a view across all projects. From your success criteria it sounds like focusing on /launchpad-project would be enough, making the experiment more lightweight to execute. Two reasons - Launchpad isn't equally functional at the project-group scope, (e.g. series are not visible there). Any other examples of lack of functionality? We don't currently use series anyway, so I don't think that would matter. Secondly, we wouldn't get as much feedback if folk can individually opt-out rather than saying 'hey it really isn't working for me'. Right. You will get much more feedback if force people to do something. However, that makes this experiment very expensive. Not implementation-wise, but in developer time. It will make people unproductive in the beginning. So unless we have a really good goal, and committment to get there, I think we should try with cheaper experiments instead. We've done that in the past, where a team has committed to trying something else. I think that has worked out quite well, without forcing everyone to do it. Getting 30 developers to try something out is hard. Getting a sub team to do it is easier. Why don't we make an experiment, to see whether having an opt-out actually doesn't work? :) -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Getting upstream's trunk code for a package, and data and so forth
On Sun, Nov 15, 2009 at 11:56:47PM -0600, Jonathan Lange wrote: Hello Launchpadders UDDers, Since we want people to be able to easily build packages out of the trunk of their upstreams, we want to have working Bazaar branches for the trunks of the upstreams of all our source packages, starting with the ones in 'main'. To me, this is a quantifiable goal, so I think we should have a graph that shows the following: - Number of packages (maybe in an interesting subset) - Number of these packages with upstream links - Number of packages that have upstream links that have trunk branches - Number of packages that have upstream links that have trunk branches that work When all of the lines converge, we win, and get to eat cake. I've written some queries that measure these things, which I shall ask to be put onto a graph by-and-by. However, I'd like you to have a look at these queries and review them for correctness. For those who are interested, the numbers for Lucid are: - Packages in main: 16143 - Packages in main w/ upstream links: 16 Is an 'upstream link' a link from the package to the upstream project in Launchpad? If so, this measurement doesn't make much sense, given how packaging links currently work. It's designed so that if no link exist for the current series, it will look for a link in previous series. Therefore, the user doesn't have an incentive to create links for Lucid, if the package already has a valid link for a previous series. Things work just as well with a link for a previous series as it works with a link for the current series. What should probably happen here is that we should automatically carry over links to new series, either by explicitly copying the links, or implicitly by having the Python API take care of things. The only reason at the moment to create links for Lucid, is if the package doesn't have any link for a previous series, or if you want the project page to say that a package exists for the project in Lucid. The latter shouldn't be necessary, we should do it for the user. In other words, we should use this graph to drive people to register links for Lucid; it's unproductive use of their and our time. It would be better use of our time to actually fix how packaging works. I know some work was done before, so I don't know how much is remaining. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Getting upstream's trunk code for a package, and data and so forth
On Tue, Nov 24, 2009 at 09:02:40AM -0500, Aaron Bentley wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bjorn Tillenius wrote: In other words, we should use this graph to drive people to register links for Lucid; it's unproductive use of their and our time. Shouldn't? Right, shouldn't. Thanks for the correction. However, if all the packaging links now have been copied over, I guess we should use it after all :) -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Dogfooding Launchpad for real (or, new tag: lp-dogfood)
On Mon, Nov 02, 2009 at 04:36:17PM +, Jonathan Lange wrote: Hello Launchpadders, One thing we talked about at the team lead sprint a few weeks back was what would it take to dogfood Launchpad for real. The result of the discussion was that it would be a fair chunk of work and wouldn't deliver much of value to most of our users. With my Product Strategist hat on, I agree. But let's just say that I didn't agree. Well, then I'd write an email an awful lot like this: = The problem = Launchpad has a project group called 'launchpad-project' and many subprojects, one for each team or component (depending on how you want to slice it). Examples include malone, soyuz and launchpad-code. However, there's really only one project, 'launchpad', which has its code in one place. The reasons we don't just have one project, AIUI, are: * too hard for teams to manage their bugs -- too much clutter * u... But there are all sorts of negative consequences from us going against the grain of Launchpad's design: * the dupe finder works poorly * milestones and series are harder to manage overall, since they have to be duplicated I wouldn't say these are problems that will be solved by abandoning the use of a project group. Sure, it might make the problem go away for us, but will it make the problem go away for other people using project groups? I'd say we are already dogfooding. We're dogfooding how to use project groups. Sure, it might not be the best way of using project groups, but what is the best way? I though that projects within a group is usually quite closely related, no? Thus you want the dupe finder to work across project groups, no? And you want to be able to create milestone across project groups, no? I think these are issue that we should really fix, not work around them. Or is the grand plan to get rid of project groups altogether? * confusing for users and new developers * it's somewhat inauthentic I don't know what you have in mind here. In what way is it confusing, and is that also something that we maybe should fix, rather than work around? -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Packageset URLs (Was: Re: pre-implementation call re. package set traversal change)
On Thu, Oct 29, 2009 at 02:44:40PM +0100, Muharem Hrnjadovic wrote: Hello Bjoern, the URIs we currently use for package sets are like '../package-sets/mozilla' Because package set names are only unique in combination with a distro series now the URIs will need to change to something like: '../package-sets/karmic/mozilla' or '../package-sets/mozilla/karmic' Let's wait to discuss the implementation details. I'm a bit interested in how a URL for a package set looks like. You said on IRC that you decided to give package sets it's own top-level space. And looking at the code, a URL path looks like: /package-set/desktop And with a distro series, it would be either /package-set/desktop/hoary or /package-set/hoary/desktop Now, the obvious question is, what about the distribution? Are we not going to have package sets for non-Ubuntu distributions ever? Why doesn't the package set hang of the distribution in the first place? Given the current URL structure, how will the UI navigation look like? -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Packageset URLs (Was: Re: pre-implementation call re. package set traversal change)
On Fri, Oct 30, 2009 at 11:45:19AM +0100, Muharem Hrnjadovic wrote: Bjorn Tillenius wrote: On Thu, Oct 29, 2009 at 02:44:40PM +0100, Muharem Hrnjadovic wrote: Hello Bjoern, the URIs we currently use for package sets are like '../package-sets/mozilla' Because package set names are only unique in combination with a distro series now the URIs will need to change to something like: '../package-sets/karmic/mozilla' or '../package-sets/mozilla/karmic' Let's wait to discuss the implementation details. I'm a bit interested in how a URL for a package set looks like. You said on IRC that you With the database schema changes that already landed on db-devel package set names cease to be unique globally i.e. a package set names are now only unique in combination with a distro series. That breaks the existing package set naming/traversal scheme. decided to give package sets it's own top-level space. And looking at the code, a URL path looks like: /package-set/desktop And with a distro series, it would be either /package-set/desktop/hoary or /package-set/hoary/desktop In order to fix the currently broken traversal scheme for package sets and have a working LP API after the roll-out on 04-Nov-2009 I will need to land a branch that adopts the URI structure above (e.g. /package-sets/karmic/kernel-packages) today. Having said that, I am wide open to discussing package set URI/traversal changes for the forthcoming LP developement cycle. Right, it makes sense to fix this to keep it working. However, it's quite important to think about the URL structure, considering a few basic use cases, before deciding what it should be. It sounds like we definitely want to change it, and now you have the problem that people will rely on the initial URL. Which means that you either have to make sure that the old URL still works, or say sorry guys, you're script will break, and you will have to adjust them to work with the new URL structure. The latter is quite hard to do, since you don't know who have alrady written scripts that deal with package sets. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Packageset URLs (Was: Re: pre-implementation call re. package set traversal change)
On Fri, Oct 30, 2009 at 05:36:40PM +0100, Muharem Hrnjadovic wrote: I guess we are a bit lucky this time. As far as I am aware only Colin Watson has been using the package sets until now. Well, that's the point, you said As far as I am aware, which means that you don't know who are using them :) That's the thing when you expose something via the API, you get people to use it in ways you didn't expect. Sure, for package sets this might not be the case. It seems we are all in violent agreement regarding the (new) package set URL :) it shall be as follows: /$distribution/$distroseries/+packageset/$packageset e.g. /ubuntu/karmic/+packages/mozilla One question, though. From IRC: jml al-maisan, AIUI, DistributionPackageSets have names too, right? jml al-maisan, where are you putting them in the traversal? al-maisan jml: no al-maisan we did away with the names for DistributionPackageSets We did away with the explicit name, but a DistributionPackageSets still has an implicit name. How do you navigate to a package set from /ubuntu? I would assume that you would stay in the Distribution context, and have something like /ubuntu/+packageset/mozilla. This is similare to how you navigate to source packages. If you are on /ubuntu and go to the mozilla package, you end up at /ubuntu/+source/mozilla, not at /ubuntu/karmic/+source/mozilla -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Lazr-js now managed through buildout
Lazr-js has been removed from sourcecode/, and is now managed through buildout. This means that if you want to do a change to Lazr-js and use it in Launchpad, you have to do like you do with other buildout-managed packages: 1) Make the change in Lazr-js 2) Make a new sdist, with a higher version number 3) Upload sdist to lp-source-dependencies 4) Update versions.cfg in LP to use the new version This also means that if you share sourcecode/ between the branches you're working on, you might have to merge in latest devel in the branch you're currently working on. A rocketfuel-get in devel will remove lazr-js from sourcecode/, so older branches that uses lazr-js from sourcecode/ will stop working. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] JS broken in devel
Please don't land any changes to lazr-js yet. It appears that my changes to lazr-js broke the generation of launchpad.js, since Javascript is broken in latest devel if devmode is turned off. I might have to revert the changes to lazr-js temporarily, and if you land something in lazr-js, I would have to revert those as well. I've filed bug 462502 to keep track of this issue. I've also asked for the edge updates to be disabled until this issue is resolved. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] JS broken in devel
On Wed, Oct 28, 2009 at 11:47:17AM +0200, Bjorn Tillenius wrote: Please don't land any changes to lazr-js yet. It appears that my changes to lazr-js broke the generation of launchpad.js, since Javascript is broken in latest devel if devmode is turned off. I might have to revert the changes to lazr-js temporarily, and if you land something in lazr-js, I would have to revert those as well. I've filed bug 462502 to keep track of this issue. I've also asked for the edge updates to be disabled until this issue is resolved. This has now been fixed in r9793. When it hits stable, edge updates can be enabled again (well, maybe, it seemed like edge updates were already disabled for some other reason) -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [Branch ~launchpad-pqm/launchpad/devel] Rev 9762: [r=intellectronica][ui=none][bug=452070] Optimise
On Fri, Oct 23, 2009 at 10:13:14PM -, nore...@launchpad.net wrote: Merge authors: Muharem Hrnjadovic (al-maisan) Related merge proposals: https://code.launchpad.net/~al-maisan/launchpad/clog-oops-452070/+merge/13789 proposed by: Muharem Hrnjadovic (al-maisan) review: Approve - Tom Berger (intellectronica) revno: 9762 [merge] committer: Launchpad Patch Queue Manager launch...@pqm.canonical.com branch nick: launchpad timestamp: Fri 2009-10-23 23:09:49 +0100 message: [r=intellectronica][ui=none][bug=452070] Optimise distro/+source/package/+changelog pages in order to prevent OOPSs removed: lib/lp/soyuz/templates/sourcepackagerelease-change-summary.pt modified: lib/canonical/launchpad/webapp/tales.py lib/lp/bugs/doc/bug.txt lib/lp/bugs/interfaces/bug.py lib/lp/bugs/model/bug.py lib/lp/registry/browser/distributionsourcepackage.py lib/lp/registry/configure.zcml lib/lp/registry/doc/sourcepackage.txt lib/lp/registry/model/distributionsourcepackage.py lib/lp/soyuz/browser/configure.zcml lib/lp/soyuz/browser/sourcepackagerelease.py lib/lp/soyuz/stories/soyuz/xx-distributionsourcepackagerelease-pages.txt lib/lp/soyuz/templates/distributionsourcepackagerelease-changes.pt === modified file 'lib/lp/bugs/doc/bug.txt' --- lib/lp/bugs/doc/bug.txt 2009-07-08 03:14:59 + +++ lib/lp/bugs/doc/bug.txt 2009-10-22 15:39:19 + @@ -37,6 +37,34 @@ ... NotFoundError: 'Unable to locate bug with nickname +bugs.' +It is also possible to retrieve a number of bugs by specifying the bug numbers +of interest. + + result_set = bugset.getByNumbers([6, 1234]) + print result_set.count() +1 What's a bug number? Don't we usually talk about bug ids? Also, it's highly confusing that you specify two ids/numbers, yet you get only one result! + + [the_bug_found] = result_set + print the_bug_found.title +Firefox crashes when Save As dialog for a nonexistent window is closed + + result_set = bugset.getByNumbers([6, 4321, 1]) + print result_set.count() +2 Again, you get one bug less then you asked for. Does it always return one result less than you ask for? + + second_bug_found = result_set[1] Where do the results get ordered? Are you guaranteed that you will get this bug as the second result? + print second_bug_found.title +Firefox does not support SVG How about showing the actual ids of the bugs returned? It's quite hard to associating a title with an id. === modified file 'lib/lp/bugs/model/bug.py' --- lib/lp/bugs/model/bug.py 2009-08-26 01:54:39 + +++ lib/lp/bugs/model/bug.py 2009-10-22 10:33:00 + @@ -33,7 +33,7 @@ from sqlobject import SQLMultipleJoin, SQLRelatedJoin from sqlobject import SQLObjectNotFound from storm.expr import And, Count, In, LeftJoin, Select, SQLRaw, Func -from storm.store import Store +from storm.store import EmptyResultSet, Store from lazr.lifecycle.event import ( ObjectCreatedEvent, ObjectDeletedEvent, ObjectModifiedEvent) @@ -49,6 +49,7 @@ from canonical.launchpad.interfaces.hwdb import IHWSubmissionBugSet from canonical.launchpad.interfaces.launchpad import ILaunchpadCelebrities from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet +from canonical.launchpad.interfaces.lpstorm import IStore from canonical.launchpad.interfaces.message import ( IMessage, IndexedMessage) from canonical.launchpad.interfaces.structuralsubscription import ( @@ -1627,6 +1628,14 @@ return bugs +def getByNumbers(self, bug_numbers): +see `IBugSet`. +if bug_numbers is None or len(bug_numbers) 1: Can len(bug_numbers) ever be less than 0? +return EmptyResultSet() +store = IStore(Bug) +result_set = store.find(Bug, In(Bug.id, bug_numbers)) +return result_set + class BugAffectsPerson(SQLBase): A bug is marked as affecting a user. === modified file 'lib/lp/registry/model/distributionsourcepackage.py' --- lib/lp/registry/model/distributionsourcepackage.py2009-09-16 04:31:39 + +++ lib/lp/registry/model/distributionsourcepackage.py2009-10-22 15:04:42 + @@ -408,6 +412,23 @@ 'BugTask.distribution = %s AND BugTask.sourcepackagename = %s' % sqlvalues(self.distribution, self.sourcepackagename)) +@staticmethod +def getPersonsByEmail(email_addresses): +[(EmailAddress,Person), ..] iterable for given email addresses. +if email_addresses is None or len(email_addresses) 1: +return EmptyResultSet() +# Perform basic sanitization of email addresses. +email_addresses = [ +address.lower().strip() for address in email_addresses] +store = IStore(Person) +origin = [ +Person, Join(EmailAddress,
Re: [Launchpad-dev] [Branch ~launchpad-pqm/launchpad/devel] Rev 9757: [r=intellectronica][ui=intellectronica][bug=455340][js] When a branch
On Fri, Oct 23, 2009 at 05:45:30AM -, nore...@launchpad.net wrote:
Merge authors:
Graham Binns (gmb)
revno: 9757 [merge]
committer: Launchpad Patch Queue Manager launch...@pqm.canonical.com
branch nick: launchpad
timestamp: Fri 2009-10-23 06:43:47 +0100
message:
[r=intellectronica][ui=intellectronica][bug=455340][js] When a branch
is linked to a bug using the inline widget on the bug page the
bug's branch list will be automatically updated.
added:
lib/lp/bugs/templates/bug-branch.pt
modified:
lib/canonical/launchpad/javascript/bugs/bugtask-index.js
lib/lp/bugs/browser/bugbranch.py
lib/lp/bugs/browser/configure.zcml
lib/lp/bugs/templates/bugtask-index.pt
=== modified file 'lib/canonical/launchpad/javascript/bugs/bugtask-index.js'
--- lib/canonical/launchpad/javascript/bugs/bugtask-index.js 2009-10-16
08:25:31 +
+++ lib/canonical/launchpad/javascript/bugs/bugtask-index.js 2009-10-22
15:03:15 +
@@ -633,22 +633,69 @@
// Call linkBranch() on the bug.
config = {
on: {
-success: function(client) {
+success: function(bug_branch_entry) {
link_branch_link.toggleClass(
'update-in-progress-message');
-anim.run();
+
+// Grab the XHTML representation of the branch and add
+// it to the list of branches.
+config = {
+on: {
+success: function(branch_html) {
+add_branch_to_linked_branches(branch_html);
+}
+},
+accept: LP.client.XHTML
+}
+lp_client.get(bug_branch_entry.get('self_link'), config);
},
-failure: error_handler.getFailureHandler()
+failure: error_handler.getFailureHandler(),
},
You didn't lint this file after doing this change on the last line, did
you?
(Having a trailing comma in dictionaries aren't allowed in JS)
--
Björn Tillenius | https://launchpad.net/~bjornt
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [Branch ~launchpad-pqm/launchpad/devel] Rev 9769: [r=bac][ui=none][bug=455812,
On Sat, Oct 24, 2009 at 09:40:29AM -, nore...@launchpad.net wrote:
Merge authors:
Edwin Grubbs (edwin-grubbs)
Related merge proposals:
https://code.launchpad.net/~edwin-grubbs/launchpad/bug-458169-distroseries-timeout/+merge/13851
proposed by: Edwin Grubbs (edwin-grubbs)
review: Approve - Brad Crittenden (bac)
revno: 9769 [merge]
committer: Launchpad Patch Queue Manager launch...@pqm.canonical.com
branch nick: launchpad
timestamp: Sat 2009-10-24 10:37:08 +0100
message:
[r=bac][ui=none][bug=455812,
458169] Fixed timeouts on distroseries page and distro milestone page.
modified:
lib/lp/registry/browser/productseries.py
lib/lp/registry/model/distribution.py
=== modified file 'lib/lp/registry/model/distribution.py'
--- lib/lp/registry/model/distribution.py 2009-10-16 15:00:55 +
+++ lib/lp/registry/model/distribution.py 2009-10-23 16:20:14 +
@@ -325,10 +325,7 @@
return (2, self.name)
return (3, self.name)
-# XXX: 2008-01-29 kiko: This is used in a number of places and given it's
-# already listified, why not spare the trouble of regenerating this as a
-# cachedproperty? Answer: because it breaks tests.
-@property
+@cachedproperty('_cached_serieses')
def serieses(self):
See `IDistribution`.
ret = DistroSeries.selectBy(distribution=self)
@@ -1497,6 +1494,9 @@
if owner.inTeam(self.driver) and not owner.inTeam(self.owner):
# This driver is a release manager.
series.driver = owner
+
+if safe_hasattr(self, '_cached_serieses'):
+del self._cached_serieses
return series
A change like this deserves an explicit test. Having @cachedproperty on
the model itself is dangerous, since you need to be careful to
invalidate the cache when needed. Therefore you should have a test
showing that the cache is invalided whenever the series information
changes.
--
Björn Tillenius | https://launchpad.net/~bjornt
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] [Branch ~launchpad-pqm/launchpad/devel] Rev 9767: [r=abel][ui=none][bug=438985] Setting yourself as bug supervisor will
On Sat, Oct 24, 2009 at 09:03:14AM -, nore...@launchpad.net wrote: Merge authors: Graham Binns (gmb) Related merge proposals: https://code.launchpad.net/~gmb/launchpad/make-me-super-bug-438985/+merge/13840 proposed by: Graham Binns (gmb) review: Approve - Abel Deuring (adeuring) revno: 9767 [merge] committer: Launchpad Patch Queue Manager launch...@pqm.canonical.com branch nick: launchpad timestamp: Sat 2009-10-24 10:00:34 +0100 message: [r=abel][ui=none][bug=438985] Setting yourself as bug supervisor will no longer OOPS. modified: lib/canonical/launchpad/database/structuralsubscription.py lib/lp/registry/tests/test_product.py lib/lp/testing/factory.py === modified file 'lib/lp/registry/tests/test_product.py' --- lib/lp/registry/tests/test_product.py 2009-08-13 19:03:36 + +++ lib/lp/registry/tests/test_product.py 2009-10-23 13:48:28 + @@ -145,5 +150,31 @@ 'new') +class BugSupervisorTestCase(TestCaseWithFactory): +A TestCase for bug supervisor management. + +layer = LaunchpadFunctionalLayer + +def setUp(self): +super(BugSupervisorTestCase, self).setUp() +self.person = self.factory.makePerson() +self.product = self.factory.makeProduct(owner=self.person) +login(self.person.preferredemail.email) + +def testPersonCanSetSelfAsSupervisor(self): +# A person can set themselves as bug supervisor for a product. +# This is a regression test for bug 438985. +user = getUtility(IPersonSet).getByName(self.person.name) Did you try to change makePerson() not to strip the security proxy as we discussed? Or add an XXX if it broke too many tests? -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Problems with make run on Karmic
On Wed, Oct 07, 2009 at 07:11:12AM +0100, Graham Binns wrote: ImportError: No module named psycopg2 make: *** [initscript-stop] Error 1 Any ideas? I've reinstalled launchpad-developer-dependencies and so far, no dice. Do you have launchpad-database-dependencies installed? -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Converting old Windmill tests and make them pass
When 3.0 is out of the way, the next priority will be to get the currently failing Windmill test suite to pass. I have made some changes to the way Windmill tests are run, but old tests need to be converted in order to have them run in the new way. So the way you will make sure that a Windmill test passes will be: 1. Convert the test to a TestCase test 2. Run 'make jscheck' 'make jscheck' runs only the new-style Windmill tests, and that test suite currently passes. There's a buildbot running 'make jscheck' daily: https://lpbuildbot.canonical.com/builders/jscheck The old Windmill tests are run using the functest test runner. We want to get rid of that test runner, and instead use our own bin/test test runner. In order to do this, we need to convert the old tests to be layered TestCase tests. You can see examples of a new-style Windmill tests in lib/lp/bugs/windmill/tests/test_bug_commenting.py and lib/lp/code/windmill/tests/test_branch_links.py The process of converting the tests is basically: 1. Make sure that YourApplicationWindmillLayer exists. If not, create one in lp.yourapplication.windmill.testing. See lp.bugs.windmill.testing.py for an example. 2. Make sure that bin/test --layer=YourApplicationWindmillLayer is run by 'make jscheck' (Check in Makefile) 3. Move the python file to lp.yourapplication.windmill.tests. The test_foo.py file has to be directly under a tests/ directory, otherwise it won't be picked up. 4. Create a subclass of TestCaseWithFactory, setting layer = YourApplicationWindmillLayer. 5. Create a test_foo() method, moving the contents of the old Windmill test function inside it. And that's it really. You can now run the test using bin/test --layer=YourApplicationWindmillLayer -t test_foo. Specifying the layer is important, since by default bin/test won't run tests in a WindmillLayer. As a bonus, you can now use the LaunchpadObjectFactory to do test setup. Don't forget to commit transaction after the setup, since otherwise the app server won't see the changes. The db also gets reset between tests, so you don't have to clean up after yourselves in the test. Please let me know if you run into any problems. Known issues: * If don't create a new branch, you might have to re-generate bin/test, since there were changes in test.in, and bin/test doesn't get re-genereated automatically. Removing bin/buildout and running 'make' fixes it. The symptom here is that bin/test --layer=WindmillLayer will run all the tests. * Running 'bin/test --layer=WindmillLayer' or running two tests in different WindmillLayer Hangs when the second layer is set up. * Sometimes an exception is raised (complaining about invalid thread ids) during teardown. I have submitted fixes for the two latter issues upstream, and hopefully it will be included in Windmill 1.3, so it will be fixes when we upgrade. If it causes too much of a problem, I'll look into patching our version locally. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] BuildEngineer and ReleaseManagerRotation
On Fri, Sep 18, 2009 at 11:06:48AM +1200, Michael Hudson wrote: Michael Hudson wrote: Christian Robottom Reis wrote: I finally got around to reviewing https://dev.launchpad.net/PolicyAndProcess/ReleaseManagerRotation and https://dev.launchpad.net/BuildEngineer They both look good (after some minor edits by me wink). I just wanted to ask two things: a) Why does one page end in Rotation and fall under PolicyAndProcess, and the other not? Dunno, it's a wiki, so it would be unnatural if it wasn't random and inconsistent? b) How is the BuildEngineer rotation going this cycle? I think it's going pretty well. The role is quite stressful in some ways, with lots of things that take far longer than you'd think and fighting with systems that no-one really understands any more. Oh, I guess I should say that I find this part of the role: * Monitor the buildbot and ensure smooth operation of builds. o This means monitoring the builders and making sure that somebody is assigned to fixing any build failures or errors as they arise. to be perhaps not be the greatest idea, mostly because build failures need to be fixed *now*, not when the build engineer wakes up, When reading the point above, I don't think it says that the build engineer should fix build failures. It's more like monitoring the build failures, and make sure that all issues we run into are picked up by someone. For example, what's up with the error about the slaves running out of disk space? Has it been reported? Is someone looking into it? and also because I think part of the point of being the BE is that it allows you to get away from being distracted by the 1001 things we all have pressing at us constantly. This might be true. However, I do think that build errors that aren't caused by real test failures (like the example above) should get really high priority to get fixed. Systemic issues that affect the reliable functioning of buildbot should of course be very high priority build engineer issues -- but that's not the same thing. ... right, you agree with my, I guess :) So we need to define what the responsibilites really are. I would say it's the build engineer's responsibility to go through the build failures, and pick up any failure that isn't a real test failure (e.g. intermittently failing tests don't fall into this category). He doesn't necessarily have to fix it himself, but he should make sure that a bug is filed about it, and that someone is looking into it. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Windmill and uuid listening on the ports we're using
Hi, I remember when we started using Windmill, we have this problem that after a test run, uuid had somehow started listening on a port we used (either the app or smtp server, I can't remember). We don't seem to have this problem anymore, so can anyone remember how we fixed it? Well, actually we do have this problem. I'm making the Windmill tests run in a subclass of AppServerLayer, which starts an SMTP server. After windmill and the SMTP server is stopped, libuuid has started listening on the port the SMTP server used. I've worked around it for now by stopping the server before starting Windmill, since I don't think any tests use it, but I'd like to solve this properly. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Windmill and uuid listening on the ports we're using
On Wed, Sep 16, 2009 at 09:15:05AM -0400, Francis J. Lacoste wrote: On September 15, 2009, Bjorn Tillenius wrote: Hi, I remember when we started using Windmill, we have this problem that after a test run, uuid had somehow started listening on a port we used (either the app or smtp server, I can't remember). We don't seem to have this problem anymore, so can anyone remember how we fixed it? It was fixed by a new upstream version. I think they catch the error and change ports automatically. Ah, right. Well, actually we do have this problem. I'm making the Windmill tests run in a subclass of AppServerLayer, which starts an SMTP server. After windmill and the SMTP server is stopped, libuuid has started listening on the port the SMTP server used. I've worked around it for now by stopping the server before starting Windmill, since I don't think any tests use it, but I'd like to solve this properly. Since that's in our code, we should probably do the same thing. Catch the port already in use and find another port. Yeah, I'll look into it. BTW, I did find out why this is happening. Windmill uses libuuid, which forks a process that inherits the parent process' file descriptors, including the open ports. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Windmill and uuid listening on the ports we're using
On Wed, Sep 16, 2009 at 09:31:50AM -0400, Francis J. Lacoste wrote: On September 16, 2009, Bjorn Tillenius wrote: On Wed, Sep 16, 2009 at 09:15:05AM -0400, Francis J. Lacoste wrote: On September 15, 2009, Bjorn Tillenius wrote: Well, actually we do have this problem. I'm making the Windmill tests run in a subclass of AppServerLayer, which starts an SMTP server. After windmill and the SMTP server is stopped, libuuid has started listening on the port the SMTP server used. I've worked around it for now by stopping the server before starting Windmill, since I don't think any tests use it, but I'd like to solve this properly. Since that's in our code, we should probably do the same thing. Catch the port already in use and find another port. Yeah, I'll look into it. BTW, I did find out why this is happening. Windmill uses libuuid, which forks a process that inherits the parent process' file descriptors, including the open ports. Ah, then this is a bug in libuuid I guess. It should closes all unused open filedescriptor after forking. I guess it doesn't do an exec after the fork, otherwise they would automatically be closed. Maybe it's not a bug in libuuid. I think it does an exec, but FD_CLOEXEC isn't set on the file descriptor. I'll see if I can fix it. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Using reStructured Text instead of moin syntax in the wiki?
On Wed, Sep 16, 2009 at 12:33:34PM -0400, Karl Fogel wrote:
Barry Warsaw ba...@canonical.com writes:
As Francis points out, and perhaps as icky as it is, I think #2 is the
most practical solution. So maybe my encouragement was too strong,
but I still think it should be possible and allowed to write moin and
rest in the public Launchpad wikis.
Some people who might edit our wiki will never need reST syntax for
anything else -- they won't get to the point of ever writing doc tests,
because maybe they only got as far as getting Launchpad up and running,
or they only worked on configuration issues, or whatever.
I think your comfort with reST syntax is interfering with your ability
to see the inherent editor-unfriendliness of having a two syntaxes in
the same wiki :-).
I could maybe see using reST syntax for wiki pages that are direct
copy-and-paste from a Python doctest. I tried to get some stats on
which pages are in reST right now, but ran into this:
$ cat PageHits
https://dev.launchpad.net/FrontPage?action=edit
https://dev.launchpad.net/Getting?action=edit
https://dev.launchpad.net/OpenSourcing?action=edit
[...]
$ mkdir results
$ cd results
$ for page in `cat ../PageHits`
do wget -v --no-check-certificate --user kfogel --password ${page}
done
--2009-09-16 12:29:43-- https://dev.launchpad.net/FrontPage?action=edit
Resolving dev.launchpad.net... 91.189.90.233
Connecting to dev.launchpad.net|91.189.90.233|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2009-09-16 12:29:44 ERROR 403: Forbidden.
[...]
$
All pages got that error. Any idea how to pull the wiki source of pages
automatically?
Use action=raw instead of acion=edit. If there's still something on the
edit page you need, you need to send a cookie header.
--
Björn Tillenius | https://launchpad.net/~bjornt
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] RFC: Bug page 3.0 redesign
On Thu, Sep 10, 2009 at 08:08:05AM +1000, Martin Pool wrote: 2009/9/10 Tom Berger tom.ber...@canonical.com: Merge proposals are definitely a better object to link to, if they exist. And if not? I think what we should do is display both. Link to the branch, and if it's proposed for merging show that and link to the merge proposal. Yes, I think you need the branch and the merge proposal, and we also really need to show the state of the merge proposal, because that gives you a more precise view of what 'in progress' means. This has to mesh with what the code team will be changing there but if you showed the overall state (approved, needs review, etc) and the list of votes as in https://code.edge.launchpad.net/bzr/+activereviews you couldn't go too far wrong. On the other hand the overall branch state (as opposed to the mp state) is generally boring and could probably be removed - but it may be worth checking how many people use it to verify that. I think the code team plan to drop it entirely. I think the branch status should go, but I would like to see it replaced by a combination the MP status, and the last commit date. The date of the last (preferably non-mainline) commit date helps indicate any progress made. -- Björn Tillenius | https://launchpad.net/~bjornt ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
