Re: [Launchpad-dev] Launchpad critical bugs fall to 100

[Julian Edwards]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 14/03/13 00:37, Curtis Hovey wrote:
> All Hail William and Steve for bringing Lp to its lowest number of 
> oopses and timeouts in 6 years.

Fixing these criticals is often hard, boring and a right slog.  Well
done guys.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlFBFk8ACgkQWhGlTF8G/HcwVgCgkReYk6yuY7GwNArSv3clncHf
N/QAoIVX4J5zTSXbjyiaQT31Ee7f77wC
=EyYJ
-END PGP SIGNATURE-

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] need help for a bug in the ui

[Julian Edwards]

On Thursday 21 Jun 2012 18:19:15 Ivo Kracht wrote:
> Hi all,
> I could need help with the following:
> 
> I saw this bug ( https://bugs.launchpad.net/launchpad/+bug/8242929 ) and
> wondered what should be edited out. On the example page in the comments
> you can see that on the left hand side under "Architectures and builds
> for Oneiric" and also further up on the page this little "i" simbol with
> a text saying "Show ..." is used a lot but further down this link on the
> right side of the actual text saying "All ..." is being used. It seems
> rather inconsistent to use both forms and just deleting one of them at
> "Latest uploads" does not really fix the inconsistency of the UI.
> Could I get some opinions about how it should be fixed? It seems to be a
> matter of personal taste mostly.

For people confused by the type I am guessing he meant 
https://bugs.launchpad.net/launchpad/+bug/824292

Anyway Ivo, I think we need to do two things:

1. Get rid of the "All uploads" on the right.
2. Change the link other text from "Show uploads" to "Show upload queue"

I've commented on the bug to this effect.

Cheers.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Maintainable UI modifications

[Julian Edwards]

On Friday 08 June 2012 15:17:41 Huw Wilkins wrote:
> Hi Launchpad,
> 
> I've been looking at how we can make some changes to our UI, but I'm
> hitting a bit of a problem.
> 
> If we want to do anything significant to our UI we want to do it under a
> feature flag. However, once I use a feature flag I very quickly end up
> having to maintain two sets of HTML and CSS or I end up hacking extra
> layers of code on top of existing HTML/CSS instead of being able to
> write it cleanly from the start.
> 
> This becomes very apparent if we want to re-skin Launchpad were there
> are potentially hundreds of places that would need to be feature
> flagged. That means a lot of duplication and the new skin would be
> written on top of the existing HTML/CSS structure.
> 
> Does anyone have any suggestions about how we could approach this problem?

I don't think there's an easy solution to your problem.

Last time we re-skinned we just froze regular development for a while and 
everyone mucked in to do it quickly as possible before we released the re-
skinned site.  Urgent maintenance fixes were just cowboyed IIRC.



___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Please be very lenient in your test timeouts--or don't write them!

[Julian Edwards]

On Monday 04 June 2012 16:11:42 Gary Poster wrote:
>  * Using wall-clock time to verify something is fragile anyway.  Test
> something else related instead (e.g., the number of queries performed).

Totally agree.

As far as I am concerned, if a test relies on wall clock time it's already 
broken.

FWIW, Twisted has a fake clock that you can wind forwards/backwards at will 
which is really handy for this sort of thing.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] LEP: Improve our Launchpad setup scripts

[Julian Edwards]

On Thursday 31 May 2012 14:32:23 Gary Poster wrote:
>  * Robert encouraged us to only support lightweight checkouts, and not
> branches (for colocated use or otherwise), again for a tighter focus.
> We felt we mostly had the code for branches anyway, and we knew some
> people (Aaron, for instance) liked and used colocated branches, so we
> wanted to keep them.  That said, this is perhaps a maintenance burden,
> and perhaps it would be appropriate to push back on us here, so we'll
> certainly listen.

On the subject of lightweight checkouts, I think if you guys don't use them 
you should give it a try in a colo tree.  When I made the switch I went from 
waiting minutes to seconds to start working on a new branch because you don't 
need to rebuild everything in a new directory.

I also use no-tree branching, so my normal workflow is this:

$ bzr branch trunk new-branch
(instant operation)
$ cd 
$ bzr switch new-branch
(seconds)

And done, you're ready to hack.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

[Launchpad-dev] Quality of commit messages has declined

[Julian Edwards]

Hi folks

I've noticed that the quality of some of our commit messages has been pretty 
bad for a while now.  Taking some of the last few, not picking on anyone in 
particular:

  [r=gary][no-qa] use a private fork of bzr
  [r=gmb][no-qa] Launchpad now depends on zope.testing-3.9.4-p7.

Most of them don't tell me *why* the change is being made; it merely reports 
fact.  Why is a private fork of bzr being used?  Why do we need that version 
of zope.testing?

Can I urge people to please (re-)read: 
https://dev.launchpad.net/PQMCommitMessages
and refresh your memories on good commit messages.  Also if you are a 
reviewer, you might want to check the commit message on the MP (or ask the 
developer to set it).

Your future selves will thank you!  And so will I!

Cheers
J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] rabbitfixture on pypi

[Julian Edwards]

On Tuesday 15 May 2012 20:48:55 Gavin Panella wrote:
> On 15 May 2012 19:25, Brad Crittenden  wrote:
> > Hi Julian,
> > 
> > It appears you are the only one with upload privileges for
> > rabbitfixture on pypi.  Could you either 1) grant me
> > (bradcrittenden) upload rights or 2) upload rabbitfixture 0.3.3 [1]
> > to pypi yourself?
> 
> Julian, please can you also grant me access?
> 
> > I've created a new release because we're seeing *very* rare startup
> > failures that look like they should be fixed by Martin Pools fix
> > from r 26.
> 
> Brad, can you give any details of the failures? I've had experience of
> weird failures in rabbitfixture and spent a lot of time getting it
> mostly reliable, so even the smallest bit of information might bring
> back some suppressed memories.
> 
> Gavin.

You're both in the owner role now.

Cheers.___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Removing IArchive.commercial

[Julian Edwards]

On Thursday 03 May 2012 15:22:56 Jonathan Lange wrote:
> > Enums are nearly always better than a bool. I've lost count of the number
> > of times I've seen loads of mutually exclusive bools because people
> > didn't plan ahead.
> 
> Or because people didn't update existing bools to enums when they
> decided they needed a new mutually existing one. It's not that we
> don't plan ahead, it's that we often don't update code when the
> situation or our understanding changes.

Updating a bool to an enum is not trivial and requires a database migration, 
potentially of millions of rows.  The path of least resistance at that point 
is to add more bools.

> If we add this as an enum, it will only have two fields. Some time
> from now, someone unfamiliar with this conversation will look and ask
> "Why is this an enum? It only has two fields" and then they'll switch
> it to be a boolean.

I really doubt that.  Especially when you add a comment to explain the 
potential use of the enum.  You comment your code, right? :)

I really don't understand the reticence to use enums instead of bools, there 
is zero harm in doing so.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Removing IArchive.commercial

[Julian Edwards]

On Thursday 03 May 2012 11:26:11 Jonathan Lange wrote:
> Actually, that wasn't entirely honest. Sorry. Here's what I was
> actually thinking:
> 
> I want to continue doing this work incrementally. Before the 'enum'
> suggestion, the way I planned to do it was this:
> 
>  1. make a property with the new name;  make that a simple
> getter/setter for 'commercial'
>  2. change the permission model for setting both it & commercial
>  3. a series of patches to rename the column in the db
> 
> I still think this is a good plan.  We could do the enum bit as a step 4.
> 
> However, I'm not 100% convinced about the enum. It is more flexible,
> but without an actual use case it's a YAGNI. I also suspect that it

Enums are nearly always better than a bool. I've lost count of the number of 
times I've seen loads of mutually exclusive bools because people didn't plan 
ahead.

If you have an enum you can even mask it as a bool property.

> will drive me into a positive line count delta. If we decide that
> that's the way it should be, I'll happily do the work. I'd only note
> that we could postpone that work until there's a use case, at no cost.

One of my concerns about this whole line delta thing is now coming to bear.  
If we let it get in the way of good design it's a disaster.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Removing IArchive.commercial

[Julian Edwards]

On Thursday 03 May 2012 10:27:50 Julian Edwards wrote:
> Call it "block_notifications" and we get a few possibilities:

s/block_notifications/suppress_notifications/

for obvious reasons.  Caffeine was not quite at sufficient levels when I wrote 
that.


___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Removing IArchive.commercial

[Julian Edwards]

On Wednesday 02 May 2012 12:04:45 Jonathan Lange wrote:
> On Wed, May 2, 2012 at 11:36 AM, Julian Edwards
> 
>  wrote:
> > On Tuesday 01 May 2012 15:05:30 Jonathan Lange wrote:
> >> On Mon, Apr 30, 2012 at 5:28 PM, Jonathan Lange  wrote:
> >> ...
> >> 
> >> >  * add a boolean parameter to the subscription that controls whether
> >> > emails are sent
> >> 
> >> This should have read "constructor" not "subscription".
> >> 
> >> The corollary is that it would be stored as a field on Archive, and
> >> that anyone would be able to set it on construction. Presumably only
> >> PPA owners could change it. William jokingly suggested 'shut_up' as a
> >> name for this field.
> > 
> > So basically s/commercial/shut_up/
> 
> Yeah. Probably spelled as "suppress_subscription_notifications" or
> something of that ilk. Would be simpler than 'commercial' in that it
> would need fewer special permissions and wouldn't be so easy to
> confuse with commercial_admin and things like that.
> 
> jml

I'd go for an enum instead of a bool, far more flexible.

Call it "block_notifications" and we get a few possibilities:

NONE
SUBSCRIPTIONS
ALL

etc.

Otherwise, looks good!

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Removing IArchive.commercial

[Julian Edwards]

On Tuesday 01 May 2012 15:05:30 Jonathan Lange wrote:
> On Mon, Apr 30, 2012 at 5:28 PM, Jonathan Lange  wrote:
> ...
> 
> >  * add a boolean parameter to the subscription that controls whether
> > emails are sent
> 
> This should have read "constructor" not "subscription".
> 
> The corollary is that it would be stored as a field on Archive, and
> that anyone would be able to set it on construction. Presumably only
> PPA owners could change it. William jokingly suggested 'shut_up' as a
> name for this field.

So basically s/commercial/shut_up/

?


___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Removing IArchive.commercial

[Julian Edwards]

On Tuesday 01 May 2012 09:12:28 you wrote:
> On Tue, May 1, 2012 at 3:27 AM, Julian Edwards
> 
>  wrote:
> > On Monday 30 Apr 2012 17:28:27 Jonathan Lange wrote:
> >> We think that removing 'commercial' would make Launchpad simpler and
> >> reduce its maintenance costs. However, we would want to still be able
> >> to subscribe people without sending them emails. It's on this that we
> >> want your input.
> > 
> > Can you explain a bit more why you think it would make LP simpler?
> >  Bearing in
> > mind you still want to do the same thing elsewhere:
> Because it would have less stuff in it. 

I thought you said you wanted to remove it and implement it differently, which 
doesn't say to me that it would have less stuff in it.  I'm sorry if I 
misunderstood you?

> What do you mean by the same
> thing elsewhere? Do you mean preventing other "normal" PPA emails from
> going out?

Yes, you want to prevent PPA emails going to SC users, right?

> If that's the behaviour we need (mails sent to uploaders who aren't
> archive subscribers or if the archive is used for the Software
> Centre), then that sounds like intrinsic complexity. All decoupling
> will do is take the 'if' statements about mail in Soyuz and turn them
> into 'if' statements about events in a mail dispatcher.

Yes, that's true to some extent, but if you'd seen the code I am talking about 
you'd be fairly horrified. :)

> > Ideally we need a separate subscription mechanism that clearly delineates
> > email sending from the Soyuz innards.  I think Rob had some ideas for that
> > to use Rabbit as part of the SOA push.  Decoupling email entirely from LP
> > would be marvellous.
> 
> Well, that's rather paralyzing. Is there no middle ground?

There almost certainly is.  I was just pointing out where Rob wants to go with 
emails in the hope that we/you can do something that is heading in that 
direction.  I'd hate for you to write something that is thrown away at some 
point!

Sorry, I can't really spend a great deal of time helping since I am not really 
working on LP these days, although I'm very happy to at least try and help 
someone else improve this area as it's one that I know well and have spent far 
too long getting frustrated about.

Cheers
J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Removing IArchive.commercial

[Julian Edwards]

On Monday 30 Apr 2012 17:28:27 Jonathan Lange wrote:
> We think that removing 'commercial' would make Launchpad simpler and
> reduce its maintenance costs. However, we would want to still be able
> to subscribe people without sending them emails. It's on this that we
> want your input.

Can you explain a bit more why you think it would make LP simpler?  Bearing in 
mind you still want to do the same thing elsewhere:

> Here are some options that we thought of:
> 
>  * add a boolean parameter to the subscription that controls whether
> emails are sent
>  * don't send subscription emails when the request came from the API
>  * don't send subscription emails at all
> 
> Are any of these preferable? Are there better options?

Emails from PPA actions are a total mess right now and unfortunately None of 
these suggestions will improve it IMHO.  One of the main problems is that 
everything is too tightly coupled which makes it far too easy to break 
unrelated events.  For example, emails are only normally sent to uploaders, 
except if you are an archive subscriber, and except again if it's commercial 
:/

Ideally we need a separate subscription mechanism that clearly delineates 
email sending from the Soyuz innards.  I think Rob had some ideas for that to 
use Rabbit as part of the SOA push.  Decoupling email entirely from LP would 
be marvellous.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Is CopyArchiveJob in use?

[Julian Edwards]

On Thursday 26 April 2012 22:37:51 Aaron Bentley wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> On 12-04-26 09:53 PM, Julian Edwards wrote:
> > I'm quite excited to see an off the shelf solution being used to
> > run our jobs. Have you documented its use in LP anywhere on the dev
> > wiki?
> 
> No, I've been more focused on getting it implemented.  What kind of
> information would you like to see up on the dev wiki?

Off the top of my head:
 * How it's architected and data flow
 * Example job explaining how to set up new ones
 * Production setup details
 * QA setup details___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Is CopyArchiveJob in use?

[Julian Edwards]

> We have Celery deployed on production (ackee) now.  Running garbo
> tasks via Celery is probably simple, but we have been focusing on jobs
> which provide the IRunnableJob interface, so I don't think our work is
> directly applicable.

I'm quite excited to see an off the shelf solution being used to run our jobs.  
Have you documented its use in LP anywhere on the dev wiki?  It's a bit more 
of a challenge keeping up with LP developments while we've been working on 
MAAS!

Cheers.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Is CopyArchiveJob in use?

[Julian Edwards]

On Tuesday 24 Apr 2012 09:21:55 Aaron Bentley wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> On 12-04-23 10:52 PM, Julian Edwards wrote:
> > On Monday 23 April 2012 14:25:22 Aaron Bentley wrote:
> >> I'm working to convert all Jobs so they can run under Celery
> >> (not including build farm jobs).
> > 
> > Good, we need to remove all trace of Job from BFJs. They are not
> > Jobs and it was a mistake to use the Job code originally.
> 
> To clarify, I'm not planning to do anything to BFJs.  They will still
> share the code with Jobs that they shared before I began (which is
> mostly model definition and some state transition rules).

Don't worry I wasn't expecting you to change them :)

> I think that BFJs could *become* Jobs, but that's a task for another
> time, if we decide we want that.

I want the exact opposite, they are totally unlike Jobs and have caused no end 
of pain trying to integrate them as such.

Cheers.___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Is CopyArchiveJob in use?

[Julian Edwards]

On Monday 23 April 2012 14:25:22 Aaron Bentley wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> I'm working to convert all Jobs so they can run under Celery (not
> including build farm jobs).

Good, we need to remove all trace of Job from BFJs. They are not Jobs and it 
was a mistake to use the Job code originally.

> I started to convert CopyArchiveJob, but
> it appears to be unused; it does not have a script, it's not mentioned
> in the process-job-source-groups, and it does not have a config
> section.  It appears the last serious work done on it was in 2010.
> 
> Is this code in use?

I don't think it is.  The PackageCopyJob stuff has superseded it, so feel free 
to rip it out.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Submitting a patch docs missing info about QA process

[Julian Edwards]

On Friday 13 April 2012 14:28:55 Jonathan Lange wrote:
> Hello,
> 
> The documentation on submitting a patch to Launchpad[1] is incomplete.
> I've been told by Steven K that contributors are expected to "QA"
> their own patch, but that's not listed in the process, and I couldn't
> find a description of how to QA on the wiki page.
> 
> Further, I was told by someone on IRC that I had to create & link a
> bug report so the QA system would work.
> 
> I found the QA process docs[2], but they aren't exactly targeted at
> the casual contributor. I would be grateful if someone who understands
> the process and the intent behind it would write a document aimed at
> folks like these.
> 
> I have updated the PatchSubmission page, adding instructions to
> _always_ link a bug and to do QA. I haven't been able to be clear on
> what "doing QA" is though. I am personally unclear on how to tell when
> something is ready to QA, and on how to flag the bug when the change
> doesn't work as expected but doesn't introduce any significant
> problems.
> 
> Also, at the risk of derailing this thread, I don't understand why
> contributors are trusted to do their own QA but not trusted to do
> their own code review.

I do not think casual contributors should be doing their own QA unless they 
actually want to do it, or if it's a complex change that requires special 
knowledge. *cough* soyuz *cough*

QA can be a very painful part of the process in developing for LP and we 
should be helping contributors, not alienating them.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Hit list for maintenance costs reduction

[Julian Edwards]

On Friday 13 April 2012 21:18:39 Graham Binns wrote:
> On 13 April 2012 16:37, Jonathan Lange  wrote:
> > I'd rather raise ideas here in case because they probably need refinement:
> > 
> >  * reduce test suite run time without reducing coverage
> 
> We (~Yellow) are working on that - or at least on spreading out the
> load. There are some roadblocks in our way yet, but we're making good
> progress.

While I think that the parallel test project is a great piece of worthwhile 
work, I still think there is a lot of room for improvement when people write 
tests.  I don't remember the last time I saw a review talk about test 
performance.

So, any change to speed up an individual test is an excellent change. (I know 
of many in the Soyuz world that we can remove tens of seconds from, minutes in 
some cases.)



___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] imperatives in bugs considered harmful - even for short lived workitems

[Julian Edwards]

On Friday 13 April 2012 09:26:02 Martin Pool wrote:
> I agree.  I think the important thing is that the bug clearly describe
> a testable assertion about the system and, if it's not obvious, the
> reason why the reporter thinks this is bad.
>
> For instance, scanning the criticals, 341927 "launchpad needs bounce
> handling of email" is perfectly clear and concise, and the description
> removes any doubt.  "Launchpad doesn't disable sending mail to
> destinations that bounce" wouldn't help anybody.

The key to this particular one being a successful bug report is that the 
description describes *why* something is problem.  Where I get irritated with 
bug reports is where they say something "should" happen but fail to explain 
anything about why or give me any background.

I still think the title should summarise the problem though, it *really* helps 
when scanning bug listings where I don't see the description.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Launchpad on Precise

[Julian Edwards]

On Friday 09 Mar 2012 08:13:58 Deryck Hodge wrote:
> On Fri, Mar 9, 2012 at 1:05 AM, Martin Pool  wrote:
> > On 9 March 2012 13:02, Julian Edwards  
wrote:
> >> IMO it's not a bug; I have explained on the bug you filed.
> > 
> > Let's talk on the bug or on irc.
> > 
> >> Copying should work (with binaries), what problem do you experience?
> > 
> > I tried to copy sources and that failed; copying binaries did work.
> 
> Thanks for weighing in here guys. Unfortunately, I think this bug you
> guys are discussing is really unrelated to the core issue. I shouldn't
> have included it in the paste, but I thought it might reveal something
> I wasn't aware of.
> 
> I think the real heart of the issue is here:
> 
> 
> The following packages have unmet dependencies:
>  launchpad-developer-dependencies : Depends:
> launchpad-database-dependencies (= 0.106~precise1) but it is not going
> to be installed
> Depends:
> launchpad-messagequeue-dependencies (= 0.106~precise1) but it is not
> going to be installed
> Depends: postgresql-doc-8.4 but it
> is not installable
> Recommends: pyflakes but it is not
> going to be installed
> Recommends: subunit but it is not
> going to be installed
> Recommends: bzr-lpreview-body but
> it is not going to be installed
> E: Unable to correct problems, you have held broken packages.
> 
> 
> 
> It looks to me like launchpad-messagequeue-dependencies wants
> postgresql-doc-8.4 which is not installable. I have no idea what "not
> installable" means or how to fix it. I'm sorry to be so dumb about
> packaging, but if anyone could help me sort this out and get me going
> again, I'd be grateful.

I mentioned this to wgrant last week - the actual problem is that our 
rabbitmq-management plugin needs an older version of rabbitmq-server which is 
not in the archive for precise any more.

 rabbitmq-management : Depends: rabbitmq-server (< 2.6.2) but 2.7.1-0ubuntu1 
is to be installed

Since wgrant made the plugin in the first place I was kinda hoping he'd do 
another ;)

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Launchpad on Precise

[Julian Edwards]

On Friday 09 Mar 2012 11:24:09 Martin Pool wrote:
> On 9 March 2012 11:18, Martin Pool  wrote:
> > This is (imo) a bug in Launchpad, that the metadata files for a ppa
> > are 404 until you upload to that distro series:
> > https://bugs.launchpad.net/launchpad/+bug/950433
> > 
> > I'm going to kludge around it now by copying a package into the bzr
> > precise ppa.
> Actually, I can't, unless I have something newer than what's in
> Precise.  So, it would be really nice if the lp bug can be fixed.
> 
> For now I guess you should just comment out the ppa from your sources list.

IMO it's not a bug; I have explained on the bug you filed.

Copying should work (with binaries), what problem do you experience?___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] auto staging buildd deployments (rt 49358)

[Julian Edwards]

On Friday 02 Mar 2012 17:33:10 Martin Pool wrote:
> Carrying on with the work we did on splitting out the buildds from
> last year, with the help of cinerama and others from IS, we're soon
> going to start automatically deploying from tip to the staging
> buildds.  So if you need to change that machinery (which shouldn't be
> all that often), the flow will be:
> 
>  - write, propose, review, and merge a change to
>  runk> -
> 
> will automatically build it (you might want to nudge it after your commit)
>  - package appears in
>  runk> - puppet will pull it from that ppa on to the staging buildds
> 
> You can then do some kind of build on staging.l.n to try it in a
> realisticish environment.
> 
> To deploy to lpnet, you'll need to also ask IS to verify and rebuild
> the changes and then manually deploy them.
> 
> This isn't actually live yet but should be soon.
> 
> hth,

This is fantastic!  Thanks Martin.
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Derived distributions of Ubuntu development milestones

[Julian Edwards]

On Monday 27 Feb 2012 18:53:25 Timothy Chavez wrote:
> Hello,
> 
> The CE Infrastructure team would like to use derived distributions to
> create snapshots of each Ubuntu development milestone starting with
> Precise Beta1 (if possible).  These snapshots would be used by our
> engineering teams to build "stable" project images while the release,
> itself, is still being developed.
> 
> As I understand it, our team would be one of the users of derived
> distributions (the feature is currently being used to derive Ubuntu from
> Debian, no?) > and that carries a bit of risk with it, but I think I'm
> okay with that.  Even if we can't use it for Beta1, this seems to be a
> good long term solution.

It is derived indeed and there's not a great deal of risk.  We always intended 
derivations to be snapshot-capable.

> If you think this is a legitimate and even plausible use of this
> feature, let us know, we'd be happy to beta test :)  Thanks!
> 
> -tim

It sounds like a great usage to me.  Some questions:

 * Are you snapshotting all of Ubuntu or just main?  This will affect 
librarian space which is tight at the moment until the new storage comes in.
 * Are you wanting to rebuild?  It'd be a cool rebuild test as I've wanted to 
use DDs for that for a while, but you'd need to coordinate with Distro to 
avoid swamping the build farm with two at the same time.
 * Do you need a published archive? (we currently publish derivations on a 
separate machine but nobody used this in anger yet)

J___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] disabling longpoll

[Julian Edwards]

On Saturday 18 Feb 2012 22:27:21 Ian Booth wrote:
> My initial reply to the list on this topic has been much better
> reflected in many of the subsequent comments. I am strongly in favour of
> keeping the feature turned on and polished. I benefit hugely from it and
> do not personally encounter any of the limitations mentioned. If it is
> decided to turn it off, then I'll request that we keep the feature flag
> enabled for me only if it comes to that. I do not want to lose this feature.

Or as someone said earlier in the thread, make it opt-in with an open team you 
can join.

But I'd much rather see the wildcard work done :)

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] disabling longpoll

[Julian Edwards]

On Friday 17 Feb 2012 18:36:54 Jonathan Lange wrote:
> Using those three sites, I rarely want to have more than a small
> number of tabs open at once. Perhaps their UI model encourages such
> behaviour at some fundamental level.

I am strongly in agreement here.


J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] disabling longpoll

[Julian Edwards]

On Saturday 18 Feb 2012 07:32:06 Robert Collins wrote:
> The wildcard approach would indeed support up to 23 *total* tabs in
> the browser : but note that on the 24th tab, the site will halt. This

Don't you mean 23 tabs that have merge proposals open?  No other page holds a 
long connection to LP.  Also, someone said this only affects Firefox I think?

This is why I think this will be virtually a non-issue once we fix the 
wildcard domain.

> cap is shared amongst all pages, so twitter, facebook, identi,ca,
> gmail, google plus - each of those counts towards the limit; you don't
> actually need /all that/ many things active to run into a limit.
> 
> I think the wildcard approach + docs is a reasonable first step, but I
> think we need to make sure we can tell how many concurrent polls users
> have (because that will cost *us* as well) - one way would be to
> capture the LP username in the long poll request and log it : even if
> its not trusted data (e.g. not secure) we'd be able to get indicative
> statistics ('total concurrency was no higher than N for X users').

If we knew how many long polls were open we could just stop providing pages 
that have long polls once the limit is reached.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] disabling longpoll

[Julian Edwards]

On Friday 17 February 2012 14:49:11 Jonathan Lange wrote:
> On Thu, Feb 16, 2012 at 11:34 PM, Robert Collins
> 
>  wrote:
> > I think we should disable longpoll; its not finished, noone is working
> > on it, and it causes semi-regular confusion for folk in the LP team
> > when they cannot use code.launchpad.net.
> > 
> > Any objections? [I'm not proposing removing the code yet].
> 
> I have a mild objection. As a frequent user of the website, I would
> really like to see this feature completed.

Hear hear.

We spent so much time and effort getting it this far that my team and I would 
be *very* frustrated if it were turned off.

Personally, I don't care about people who have 20+ tabs open on Launchpad, I 
think they're freaks and edge cases.  The majority of users will not be doing 
that and will get extreme satisfaction from the working feature.

Let's get the wildcard domain thing sorted out which will alleviate the 
initial concerns and for those using Firefox with its global connection limit 
- tough.

> Also, on IRC, stub suggested that perhaps this could help me as an API
> user, reducing my need to poll. If that were the case, then surely
> that would lead to reduced load on Launchpad, perhaps leading to a
> reduced maintenance cost.

+1000

That's exactly what this was designed for.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] disabling longpoll

[Julian Edwards]

On Friday 17 February 2012 12:34:22 Robert Collins wrote:
> I think we should disable longpoll; its not finished, noone is working
> on it, and it causes semi-regular confusion for folk in the LP team
> when they cannot use code.launchpad.net.
> 
> Any objections? [I'm not proposing removing the code yet].

Can you enumerate the "semi-regular confusion for folk in the LP team" please?___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] RFC: either remove 'Ubuntu CoC signing required' for PPA use, or enforce it consistently

[Julian Edwards]

On Sunday 12 February 2012 13:09:39 Robert Collins wrote:
> At the moment, we require the Ubuntu CoC to be signed to create and
> use a PPA as an individual. As a member of a team, there is no such
> requirement.
> 
> (Note that private PPAs w/commercial entitlements are arguably
> different, so I'm not talking about them at all here).
> 
> I believe jml has in the past said that the CoC requirement wasn't all
> that useful - it occurs to me that we can make LP PPA's easier to get
> going with if we drop it entirely; if we are not willing to do that,
> then I think we really have to fix the use-via-team hole, or we'll be
> tangling ourselves up in knots to justify keeping this
> partial-enforced rule.
> 
> Any strong objections or concerns any which way?

Fixing the hole should be fairly trivial, so let's do it (remembering to not 
enforce for private/commercial PPAs).

I would also look at the original reasons for having this requirement before 
removing it.

Cheers
J___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Getting rid of Ubuntu archive admin shell access

[Julian Edwards]

On 01/09/12 02:52, Colin Watson wrote:
> For instance (without meaning to disparage Matthias,
>who's probably the main user of this), if populate-archive.py were
>the only remaining thing we needed direct ftpmaster access for, then
>it might be a reasonable tradeoff to terminate our shell access and
>handle that by means of webops requests.

We eventually intend to use the derived distros feature to replace the
copy archive hack. It would be useful if you could evaluate it as such
and give some feedback on any shortcomings.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Need help and ideas for a weird Poppy problem

[Julian Edwards]

FYI I figured this out in the end.

It turns out that writing OOPSes is deferred to a thread pool.  However, the 
AMQP port was not open on the firewall, so once 10 OOPSes were queued up, it 
was hanging.

D'oh ...

On Tuesday 20 December 2011 15:30:28 Julian Edwards wrote:
> Hi folks
> 
> As you may know, Poppy is the Twisted-based FTP/SFTP server for uploading
> packages to Soyuz. I recently landed a change to fix its logging (along with
> a few other Twisted-based services such as the librarian, branch-puller
> etc) so that it uses the python-oops stuff correctly.
> 
> It was released last Friday and within 10 minutes the instance on the PPA
> machine (germanium) went into a weird state where it was unable to contact
> the xmlrpc-private auth service running on the appservers, and hence all
> SFTP requests fail.
> 
> Here is an example from the log of an unsuccessful XMLRPC request:
> 
> 2011-12-16 10:27:38+ [SSHService ssh-userauth on
> KeepAliveSettingSSHServerTransport (TimeoutProtocol)] Starting factory
> 
> ... wait ...
> 2011-12-16 10:28:08+ [-] [Failure instance: Traceback (failure with no
> frames): : User timeout caused
> connection failure.
> ]
> 2011-12-16 10:28:08+ [-] udienz failed auth publickey
> 2011-12-16 10:28:08+ [-] unauthorized login: unable to get avatar id
> 2011-12-16 10:28:08+ [-] Stopping factory
> 
> 
> And here is one that works:
> 
> 2011-12-16 10:15:11+ [SSHService ssh-userauth on
> KeepAliveSettingSSHServerTransport (TimeoutProtocol)] Starting factory
> 
> 2011-12-16 10:15:11+ [QueryProtocol,client] Stopping factory
> 
> 
> Because it's a 30 second timeout, this timeout error message is indicative
> in my experience of the TCP SYN package not being ACKed (timeouts for open
> connections are much, much longer).  However, restarting the Poppy instance
> will make things work again, so I'm not sure whether it's a code problem or
> an infrastructure problem.
> 
> We are currently running a very old revision of code on germanium so it's
> blocking further rollouts on there.  Oddly, this only affects the PPA
> machine, not the Poppy on cocoplum (the Ubuntu machine).  I've also blasted
> hundreds of connections at the dogfood box to try and make it fail, and it
> doesn't.  It's also worth noting that the instance on germanium also
> occasionally gets problems contacting the keyserver when it's trying to
> verify GPG signatures, which requires a restart to fix.
> 
> Since I'm at a total loss as to what to do next, I am going to put the
> latest code back on germanium tomorrow and run it in production again so I
> can gather more data when it goes wrong.
> 
> In the meantime, if anyone can come up with any ideas on how to figure out
> what's going on here I'd really appreciate it!
> 
> Cheers
> J
> 
> ___
> Mailing list: https://launchpad.net/~launchpad-dev
> Post to : launchpad-dev@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~launchpad-dev
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

[Launchpad-dev] Need help and ideas for a weird Poppy problem

[Julian Edwards]

Hi folks

As you may know, Poppy is the Twisted-based FTP/SFTP server for uploading 
packages to Soyuz. I recently landed a change to fix its logging (along with a 
few other Twisted-based services such as the librarian, branch-puller etc) so 
that it uses the python-oops stuff correctly.

It was released last Friday and within 10 minutes the instance on the PPA 
machine (germanium) went into a weird state where it was unable to contact the 
xmlrpc-private auth service running on the appservers, and hence all SFTP 
requests fail.

Here is an example from the log of an unsuccessful XMLRPC request:

2011-12-16 10:27:38+ [SSHService ssh-userauth on 
KeepAliveSettingSSHServerTransport (TimeoutProtocol)] Starting factory 

... wait ...
2011-12-16 10:28:08+ [-] [Failure instance: Traceback (failure with no 
frames): : User timeout caused 
connection failure.
]
2011-12-16 10:28:08+ [-] udienz failed auth publickey
2011-12-16 10:28:08+ [-] unauthorized login: unable to get avatar id
2011-12-16 10:28:08+ [-] Stopping factory 


And here is one that works:

2011-12-16 10:15:11+ [SSHService ssh-userauth on 
KeepAliveSettingSSHServerTransport (TimeoutProtocol)] Starting factory 

2011-12-16 10:15:11+ [QueryProtocol,client] Stopping factory 


Because it's a 30 second timeout, this timeout error message is indicative in 
my experience of the TCP SYN package not being ACKed (timeouts for open 
connections are much, much longer).  However, restarting the Poppy instance 
will make things work again, so I'm not sure whether it's a code problem or an 
infrastructure problem.

We are currently running a very old revision of code on germanium so it's 
blocking further rollouts on there.  Oddly, this only affects the PPA machine, 
not the Poppy on cocoplum (the Ubuntu machine).  I've also blasted hundreds of 
connections at the dogfood box to try and make it fail, and it doesn't.  It's 
also worth noting that the instance on germanium also occasionally gets 
problems contacting the keyserver when it's trying to verify GPG signatures, 
which requires a restart to fix.

Since I'm at a total loss as to what to do next, I am going to put the latest 
code back on germanium tomorrow and run it in production again so I can gather 
more data when it goes wrong.

In the meantime, if anyone can come up with any ideas on how to figure out 
what's going on here I'd really appreciate it!

Cheers
J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] confusion about status of launchpad-buildd, plus buildbot updates

[Julian Edwards]

On Friday 09 December 2011 12:34:09 Gary Poster wrote:
> My tentative conclusions:
> - launchpad-buildd is supposed to be installed everywhere, but it is not.
> - relatedly, it is not part of the launchpad-dependencies metapackage
> (confirmed
> http://bazaar.launchpad.net/~launchpad-committers/meta-lp-deps/trunk/view/he
> ad:/debian/control
>  w/head:/debian/control> ) but it should be.
> - if the tests don't find launchpad-buildd, they cope, either by hiding
> the tests or by going down some code path.

The tests rely on the python-lpbuildd package, launchpad-buildd is what gets 
installed on the actual slaves IIRC.

> Julian could not confirm these conclusions.  Could Martin, William, or
> someone else confirm?  If we know what needs to be done, we can find
> someone to do it.

launchpad-buildd does not need to be installed, python-lpbuildd does.  Does 
this fit with your observations at all?

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] testfix mode in progress, and other related excitement

[Julian Edwards]

On Thursday 08 December 2011 13:55:22 Gary Poster wrote:
> From what I hear, this happens repeatedly for Benji.  I'll see if we can get
> bigjools to take a look.

wgrant commented on the bug that it's unrelated and has been fixed in 
launchpad-buildd 111

https://bugs.launchpad.net/launchpad/+bug/779367/comments/5



___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Changes to Launchpad PPA for refactor-cron-germinate

[Julian Edwards]

On Tuesday 06 December 2011 22:00:39 Colin Watson wrote:
> Hi,
> 
> My PPA:
> 
>   https://launchpad.net/~cjwatson/+archive/launchpad
> 
> ... contains changes to the germinate package (you can ignore everything
> else in that PPA) which will be required in order to land this branch:
> 
>   https://code.launchpad.net/~cjwatson/launchpad/refactor-cron-germinate
> 
> Could a Launchpad developer review this lot and copy into the Launchpad
> PPA?  germinate 2.0 is in precise (or will be once the librarian is
> fixed), so no change is needed for that suite.

All copied.  The ec2 and buildbot images will need updating as well if you are 
using the new package in tests?

> I think it's OK to accept a possible small temporary slowdown in the
> cause of making things a lot faster later.

We're happy if you're happy :)

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Can we get rid of per-script table permissions

[Julian Edwards]

On Tuesday 06 December 2011 14:41:48 Francis J. Lacoste wrote:
> Hi Stuart,
> 
> Can we get rid of per-script table permissions? Is there a significant
> benefit to this policy in terms of isolation. The bulk of our code runs
> in the app server with almost unlimited access to the DB.

+1, I have long advocated doing this.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] All hail bikeshedders: looking for a tag to represent critical-in-new-feature

[Julian Edwards]

On Thursday 01 December 2011 10:02:31 Matthew Revell wrote:
> On 1 December 2011 01:46, Francis J. Lacoste
> > I like self-inflicted. You also suggested friendly-fire which has a
> > weird poetic tone to it.
> 
> self-inflicted is great: it sums up how the Critical came about and
> has a tint of humility :) friendly-fire would autocomplete in fewer
> keystrokes, though.

Aren't all bugs self-inflicted? :)

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] launchpad-buildd split out

[Julian Edwards]

On Tuesday 29 November 2011 09:39:13 Martin Pool wrote:
> Eventually it would be perhaps good to have a fake implementation that
> talks the right protocol (to test lp without buildd), a driver that
> talks that protocol to the buildd to test it alone, and then a harness
> that tests an entire running system.  I think the last is the most
> urgent: just answering "did that deployment to staging work or not" is
> super complicated and manual and is the place we had a lot of stalls
> in deployment.  Without that, any introduction of fakes is going to be
> risky.

There are already fake buildd objects in the manager tests. This may be good 
or bad :)

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Hiding comments from most Launchpad users

[Julian Edwards]

On Tuesday 29 November 2011 08:02:05 Robert Collins wrote:
> On Tue, Nov 29, 2011 at 3:26 AM, Julian Edwards
> 
>  wrote:
> > On Wednesday 16 November 2011 13:06:47 curtis Hovey wrote:
> >> 4. We do not know who hid the comment, or why.
> > 
> > This is a more general problem across Launchpad and I see requests for
> > "who did this?" all the time.  We should start recording this data in a
> > generic fashion as soon as possible and worry about adding an interface
> > to view it later.  It probably needs a new audit table where we can
> > store generic actions
>  s/table/service/
> 
> > and link them to a Person.
> 
> Why a service? Because if its generic there is no need for it to be
> part of LP core; being separate will allow reuse in e.g. SSO, Software
> Centre Agent, Ratings and Reviews,  Being a separate service also
> allows for restricted restricted access to its DB, if we choose - to
> allow a higher standard of security around the audit trail[s].

Rabbit!

Yes, this is a much better way than a table.  Do you fancy knocking up a LEP?

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Hiding comments from most Launchpad users

[Julian Edwards]

On Wednesday 16 November 2011 13:06:47 curtis Hovey wrote:
> 4. We do not know who hid the comment, or why.

This is a more general problem across Launchpad and I see requests for "who 
did this?" all the time.  We should start recording this data in a generic 
fashion as soon as possible and worry about adding an interface to view it 
later.  It probably needs a new audit table where we can store generic actions 
and link them to a Person.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] capturing backtraces on every 'expensive' operation

[Julian Edwards]

On Sunday 20 November 2011 09:15:28 Robert Collins wrote:
> Some low hanging fruit to get a longer gc period, if we want one:
>  - we can compress the oopses - they compress very very well.
>- this just needs a patch to amqp2disk -  oops-tools already
> handles compressed oopses.
>  - we can move oops-tools to a dedicated server (already planned)

Also don't bother capturing the first 10-20 queries, they are always pretty 
much the same boilerplate ones.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Shaping Launchpad's next 18 months

[Julian Edwards]

On Tuesday 15 November 2011 14:52:26 Aaron Bentley wrote:
> Event-based jobs.  Currently, we poll to see what jobs to run.
> Instead, we should use an event system, such as RabbitMQ, to run the
> jobs as soon as they're requested.

We are closer to this than you might think :)

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] launchpad-buildd split out

[Julian Edwards]

On Thursday 10 November 2011 09:46:31 Martin Pool wrote:
> I've just copied what was lib/canonical/buildd into
> lp:launchpad-buildd  and soon
> I'm going to propose a merge to launchpad that makes this a dependency
> and deletes the code from the Launchpad tree.  I've already (I think)
> removed all the code-level coupling.

\o/

Thanks for doing this Martin!

> Still to be done:
>  * launchpad needs to treat this as a dependency; i'm thinking
> probably to make it a dpkg dependency because it's not really a lot
> like a python egg

I'm not entirely sure it needs to be a run-time dependency; if it is it should 
be an optional one since the vast majority of LP runs without it.  It should 
definitely be a testing dependency though.  I don't know how to do that with 
dpkg dependencies.

>  * launchpad needs to call in to it to run its tests

We need to be *super* careful that we don't break buildd now it's separate.  
The coupling between the buildd-manager and the slave code is far too high and 
it's easily broken :( 

Cheers
J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

[Launchpad-dev] PPA job notifications

[Julian Edwards]

Hello Huw,

I just had a conversation with Matthew and a few others on IRC about adding 
some sort of notification area to the PPA pages and we'd like to solicit your 
design input please!

The basic issue is that copying packages to a PPA frequently times out, so we 
want to switch PPA copying to a background job.  However, we currently have no 
way of telling the user that a copy is in progress or indeed if one failed.

So what we'd like is some sort of notification area that we can display 
somewhere on all a PPA's related pages, as long as you are an uploader, that 
shows something like this:

| Package Name | Version  | Copied From | Copied By  | Target Series | Status
| coolpackage  | 1.1~ppa1 | PPA for ... | cooluser   | oneiric   | FAILED
| cooltool | 2.3~ppa4 | PPA for ... | cooluser   | oneiric   | COPYING
| 

When a copy completes successfully we expect the relevant line to disappear.
A failed copy should remain displayed forever until someone acks it.

We also want to eventually link this with the longpoll infrastructure so 
everything appears and disappears automatically.

Do you have any thoughts on how this can slot into the PPA pages?

Cheers
J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] ddeb for PPA and derived distributions

[Julian Edwards]

Francis, here's the summary then:

 * We have a switch so that DDEBs are automatically built in PPAs, and it 
   works right now.  Let me know which PPAs need it.
   (I didn't know about that when I spoke to you guys in the session earlier
this week!)

 * The same switch can be used for primary archives (includes derived distros)
   but the implementation for primary archives is broken.  We'd need to spend
   ~4 days fixing it.  Once that's done derived distros can have DDEBs.

Cheers.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] ddeb for PPA and derived distributions

[Julian Edwards]

On Thursday 03 November 2011 11:43:04 Francis J. Lacoste wrote:
> Hi,
> 
> Linaro is interested in having an archive available with ddeb for all of
> the packages.
> 
> We don't support ddeb in derived archives, and the way it's implemented
> for PPA doesn't really work for them either, as it requires
> modifications to the packaging of all packages for which they want ddeb.
> 
> They could work-around this by uploading a modified debhelper to a PPA
> that automatically builds the ddeb. But that would mean they'd have to
> make sure that their customized debhelper is always ahead of the one in
> the distro and that they need to upload packages to both the ddeb PPA as
> well as their main one. Plus they want to move to derived distro.
> 
> So the question is how can we support enabling ddeb building
> automatically in PPA and derived distros.
> 
> Cheers

We're not really that close to enabling ddebs in primary archives (which 
includes derived distros).  William thinks that the approach taken so far is 
wrong and I am inclined to agree with him.

The original requirement was that the ddebs for Ubuntu were published in a 
separate archive, otherwise the main archive would start bulging to three 
times its size and there was the worry that this would cause mirrors to drop 
Ubuntu.

However, keeping deb and ddeb publications in lockstep across two archives is 
proving to be pretty nasty for various reasons so we are looking at mashing 
them in the same archive again, but getting the primary mirroring script to 
split them out at that point.  We'd not have to do that for derived distros 
since we don't mirror them.

I need to talk to William some more about this to see where the code is at, 
since he has done the vast bulk of it.  Hopefully we can offer a strategy 
tomorrow.

Cheers.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

[Launchpad-dev] Unicode and Launchpad

[Julian Edwards]

Dear hackers,

There have been a few incidents lately where the code has blown up with 
UnicodeDecodeError.  We were talking about this on the Team Leads weekly call 
yesterday, and I have been tasked with starting a thread to discuss how we can 
mitigate these errors in the future.

The things we discussed on the call were fairly simple:

 * Keep all strings as unicode internally (with the exception of plain ASCII 
strings which are easily coerced to unicode automatically)
 * Convert to/from unicode only when necessary (e.g. utf8 byte string or MIME) 
at the point the string *exits or enters* Launchpad.
 * Never use str()
 * Whenever someone is dealing with strings in a branch, please review 
accordingly.

I'm going to throw this open to you guys now to see if anyone else has 
anything valuable to contribute.  I'll then summarise everything into our 
coding guidelines after the discussion ends.

Cheers.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Hiding or disabling non-available actions

[Julian Edwards]

On Wednesday 26 October 2011 14:17:04 Huw Wilkins wrote:
> On 25/10/11 22:38, Julian Edwards wrote:
> > Personally, I hate hate hate magically appearing and disappearing links.
> > I'd vote for making them disabled rather than invisible.
> 
> Right, the key thing there is "magically appearing and disappearing
> links ". I dare say you use sites all the time that hide controls and
> don't even notice because the context is clear.

Well my problem is that I assume different roles.  Sometimes I forget that 
something is (buildd) admin only, for example and point it out to other people 
and we get confused when they cannot see it.

So as you say, context is everything.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Hiding or disabling non-available actions

[Julian Edwards]

On Monday 24 October 2011 16:20:29 Martin Pool wrote:
> > A control should be hidden if:
> > - It is never available to the user in this context (the user can not do
> > anything to enable it)
> I agree with the principle but it depends a bit on what you mean by 'never'.
> 
> Some of the cases that have come up in lp (and been handled inconsistently):
> 
>  * this control would be available if you logged in
>  * ... if you had some greater privilege in this context (eg if you
> were a bug supervisor, )
>  * ... if you changed some configuration of this context (eg turning
> on/off use of an application by a project)
>  * ... you had some superhuman privilege (~admin etc)
> 
> It could be good to explain this in the style guide.

Personally, I hate hate hate magically appearing and disappearing links. I'd 
vote for making them disabled rather than invisible.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

[Launchpad-dev] Landing and deployment strategies for SOA components

[Julian Edwards]

Hey Rob,

After working on Long Poll, people in my team have been asking questions about 
how we are going to manage deployments of SOA components when we have 
dependencies between them.

My initial suggestion is that we continue to simply roll out the latest of 
everything, everywhere, during regular NDT unless we know of a good reason not 
to.  What do you think?

Also, have you thought about whether we should setting up PQM or Tarmac on all 
of these components? We're going to need something to manage landings to 
ensure tests are run.

Cheers.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Bug tagging

[Julian Edwards]

On Saturday 08 October 2011 09:20:58 Robert Collins wrote:
> So I don't think we should nest things that are only historically
> within one subteams code base: our long term sanity, breaking things
> into services and reducing the size of the code base - getting more
> bang for our buck - depend on breaking down these artifical walls.

"breaking things into services"

I would have thought this was a good reason to be more granular with tags, not 
less.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] To wiki or not to wiki

[Julian Edwards]

On Friday 07 October 2011 20:35:40 Robert Collins wrote:
> On Fri, Oct 7, 2011 at 8:19 PM, John Arbash Meinel
> 
>  wrote:
> > The only thing I want to point out is that if docs are in-tree, you
> > can do "grep -rnI OldAPI" to update the callers of your api, and to
> > notice that there is some documentation about the old api that should
> > be updated.
> > 
> > It is quite a bit harder to do that if the docs are in the wiki, but
> 
> > that might fall under:
> How is it harder?

IMO it's not harder, but very much more convenient.

If you're grepping the tree for invocations of "foo()" then it'll bring up 
documentation at the same time, which reminds you to update it.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] To wiki or not to wiki

[Julian Edwards]

On Thursday 06 October 2011 14:47:31 Jonathan Lange wrote:
> On Thu, Oct 6, 2011 at 2:41 PM, Julian Edwards
> 
>  wrote:
> > On Thursday 06 October 2011 09:03:26 Benji York wrote:
> >> I have never gotten much out of API documentation, especially
> >> generated
> >> API documentation.  I can read the code about as quickly as I can read
> >> API docs and reading code makes it easier to drill down into the
> >> contents of a function/method when the need arises.
> > 
> > I think we're conflating two types of documentation here, api docs and
> > conceptual docs.  I think we should continue with the current way that
> > we use for api docs; that is docstrings and pydoc.
> 
> Count me out of that "we". I know they're different, but I reckon we
> need a fairly small amount of the "conceptual" kind.

Ok.

I am mostly referring to all the stuff we're writing about LongPoll at the 
moment, which most people will find alien along with RabbitMQ.  Also, Soyuz, 
not many people know about how that works, so I'm writing a huge amount of 
docs (at Francis' request) about it.



___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] To wiki or not to wiki

[Julian Edwards]

On Thursday 06 October 2011 09:03:26 Benji York wrote:
> I have never gotten much out of API documentation, especially generated
> API documentation.  I can read the code about as quickly as I can read
> API docs and reading code makes it easier to drill down into the
> contents of a function/method when the need arises.

I think we're conflating two types of documentation here, api docs and 
conceptual docs.  I think we should continue with the current way that we use 
for api docs; that is docstrings and pydoc.

J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] To wiki or not to wiki

[Julian Edwards]

On Thursday 06 October 2011 11:34:59 Matthew Revell wrote:
> On 6 October 2011 11:27, Julian Edwards  
wrote:
> > On Thursday 06 October 2011 10:14:10 Matthew Revell wrote:
> >> Now, as for whether we favour Sphinx docs or doctests, I'm a little
> >> less clear on. I prefer the Sphinx stuff because anyone can access
> >> them via rtfd. However, my experience with doctests is pretty limited.
> > 
> > Talking of R[F]TD, I'd much rather we had docs.launchpad.net and host it
> > ourselves.
> 
> Would a redirect from docs.launchpad.net to launchpad.readthedocs.org be
> enough?
> >From a branding and general feel-good point of view, I get the idea
> 
> that hosting it ourselves and branding it nicely would be a good thing
> but it depends how much other stuff would have to wait while we did
> that.

I'd think it was ok temporarily but I don't think we should be relying on an 
external service when pushing out static pages is utterly trivial (relatively 
speaking).

> 
> >  * MOIN SUCKS
> >  * Doctests blow (I personally think they have a place, just a limited
> > one) * rST is portable
> >  * Docs in the tree are desirable, using Sphinx.
> >   * Caveat - landing to LP is heavyweight, but Matthew thinks this is a
> > benefit to have it reviewed. (I agree)
> >  * Wikkid is nice. (having docs in a separate tree for it might be a
> > compromise between LP's heavy landing process and reviewability)
> 
> I am pretty heavily in favour of having the same review process and
> commit rights for landing docs. Losing the four hour (soon to be
> shorter!) test suite just for a doc change might be nice, though.

Yes, that was my point :)  Docs don't need testing.

J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] To wiki or not to wiki

[Julian Edwards]

On Thursday 06 October 2011 10:14:10 Matthew Revell wrote:
> Now, as for whether we favour Sphinx docs or doctests, I'm a little
> less clear on. I prefer the Sphinx stuff because anyone can access
> them via rtfd. However, my experience with doctests is pretty limited.

Talking of R[F]TD, I'd much rather we had docs.launchpad.net and host it 
ourselves.

We've got some good points so far, thanks Matthew and Gavin.  Summary:

 * MOIN SUCKS
 * Doctests blow (I personally think they have a place, just a limited one)
 * rST is portable
 * Docs in the tree are desirable, using Sphinx.
   * Caveat - landing to LP is heavyweight, but Matthew thinks this is a 
benefit to have it reviewed. (I agree)
 * Wikkid is nice. (having docs in a separate tree for it might be a 
compromise between LP's heavy landing process and reviewability)

J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] To wiki or not to wiki

[Julian Edwards]

On Wednesday 05 October 2011 10:34:24 Julian Edwards wrote:
> On Tuesday 04 October 2011 17:38:01 Francis J. Lacoste wrote:
> > Hi there,
> > 
> > The Red squad is progressing rapidly on the productionization of the
> > txlongpoll infrastructure. As part of that, they are documenting this
> > system (yeah!)
> > 
> >  https://dev.launchpad.net/LongPoll
> > 
> > Seeing that document though, it begs the question if the wiki is where
> > that documentation should be authored and maintained.
> > 
> > The alternative would be to maintain-it as sphinx documentation in the
> > tree. We have an-up-to-date rendering of this at
> > http://launchpad.readthedocs.org/en/latest/
> > 
> > What do you guys think?
> 
> I think we are all pulling in different directions and nobody is steering
> the rudderless documentation ship.
> 
> There are now three places to maintain documentation: the dev wiki, RTD
> (Sphnix) and doctests.  I'd like to start a rational discussion on how we
> should be managing this sort of thing because quite frankly it's getting
> annoying not knowing where to look, even if I already know something is on
> the wiki.
> 
> Stub's "wikis are where documentation goes to die" is a bit of an emotional
> comment but reminds us that documentation is able to become out of date
> very, very quickly unless it's maintained. Doctests, while able to help a
> little here, are not a panacea.  So, I differ from him in that I believe
> this applies no matter where documentation lives.
> 
> I do like the general idea of keeping documentation in our tree and
> generating HTML from it.  It's partly why I started the /LongPoll wiki
> pages in rst format so that we can drop the same raw text anywhere.  Doing
> this seems to have several advantages:
> 
>  * it's more easily searchable
>  * it's available offline
>  * it's easier to update the documentation along with the code
> 
> Perhaps the dev wiki should only contain information that is more likely to
> remain static, such as "getting the code", "debugging tips" etc.
> 
> I am happy to collate opinions on this and drive it forwards.

I'm pretty sure you guys are more opinionated than this. C'mon, let us have it 
:)

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

[Launchpad-dev] Bug tagging

[Julian Edwards]

Hi bug triagers

This is just a polite reminder to please tag bugs that you triage.  The tags 
are listed here https://dev.launchpad.net/LaunchpadBugTags
in case you forgot what they are for.

Tagging bugs is important as it makes it much easier to search and filter them 
later.

Thanks!

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] To wiki or not to wiki

[Julian Edwards]

On Tuesday 04 October 2011 17:38:01 Francis J. Lacoste wrote:
> Hi there,
> 
> The Red squad is progressing rapidly on the productionization of the
> txlongpoll infrastructure. As part of that, they are documenting this
> system (yeah!)
> 
>  https://dev.launchpad.net/LongPoll
> 
> Seeing that document though, it begs the question if the wiki is where
> that documentation should be authored and maintained.
> 
> The alternative would be to maintain-it as sphinx documentation in the
> tree. We have an-up-to-date rendering of this at
> http://launchpad.readthedocs.org/en/latest/
> 
> What do you guys think?

I think we are all pulling in different directions and nobody is steering the 
rudderless documentation ship.

There are now three places to maintain documentation: the dev wiki, RTD 
(Sphnix) and doctests.  I'd like to start a rational discussion on how we 
should be managing this sort of thing because quite frankly it's getting 
annoying not knowing where to look, even if I already know something is on the 
wiki.

Stub's "wikis are where documentation goes to die" is a bit of an emotional 
comment but reminds us that documentation is able to become out of date very, 
very quickly unless it's maintained. Doctests, while able to help a little 
here, are not a panacea.  So, I differ from him in that I believe this applies 
no matter where documentation lives.

I do like the general idea of keeping documentation in our tree and generating 
HTML from it.  It's partly why I started the /LongPoll wiki pages in rst 
format so that we can drop the same raw text anywhere.  Doing this seems to 
have several advantages:

 * it's more easily searchable
 * it's available offline
 * it's easier to update the documentation along with the code

Perhaps the dev wiki should only contain information that is more likely to 
remain static, such as "getting the code", "debugging tips" etc.

I am happy to collate opinions on this and drive it forwards.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Request for brainjuice: bug 194558

[Julian Edwards]

On Tuesday 04 October 2011 07:50:31 Robert Collins wrote:
> B: Have the librarian do it: include the object to link to as a form
> header, have the upload service run in the same domain as the
> appserver (so auth cookies are available) and make an XMLRPC call to
> the appserver before returning the HTTP status code to the user. In
> fact the XMLRPC call to the appservers could return the next_url to
> redirect to, giving a seamless experience.
[snip] 
> my recommendation is B.

Graham, the poppy server may give you some inspiration.  It declares FTP and 
SFTP services in a twistd server.

I can't help thinking we should unify all these uploads somehow, but Soyuz 
uploads have concurrency race conditions that need fixing first.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Debian importer status?

[Julian Edwards]

On Monday 26 September 2011 02:26:49 Robert Collins wrote:
> On Mon, Sep 26, 2011 at 11:38 AM, Stuart Bishop
> 
>  wrote:
> > Hi.
> > 
> > "Imports of Debian are currently suspended due to a problem with the
> > importer. It should be working again Monday, we will notify at that
> > time"

That was me.

> > 
> > That was the last message on our status identi.ca timeline, from 9
> > days ago. Anyone in the know want to post an update?
> 
> I asked in -ops, status is that its running 48-hours a run so things
> are working but extraordinarily slow. fix is in devel blocked on DF
> restore to qa.

Right.  I am waiting for jtv's final changes to land before I issue another 
notice.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] New tricks for StormStatementRecorder

[Julian Edwards]

On Friday 23 September 2011 18:22:11 Gary Poster wrote:
> from lp.testing import StormStatementRecorder
> 
> with StormStatementRecorder() as recorder:
> ...code that touches the DB goes here...
> 
> print recorder

This is cool!

> You can conditionally get tracebacks by passing a callable that receives a
> SQL query string and returns a boolean True if a traceback should be
> collected, and False if it should not. The SQL will be normalized to
> capitalization and space normalized. For example,
> StormStatementRecorder(lambda sql: 'STRUCTURALSUBSCRIPTION' in sql) would
> get you tracebacks when the SQL has something to do with structural
> subscriptons.

But this is awesomely unbelieveably useful, thanks Gary!

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] DB process change: new users must be set up on production before landing

[Julian Edwards]

On Friday 23 September 2011 13:17:20 Robert Collins wrote:
> On Fri, Sep 23, 2011 at 10:50 PM, Julian Edwards
> 
>  wrote:
> > Was this change discussed with the TA or anyone else?
> 
> Yah, with me. Like all such things, if its a bad idea we can change it.
> 
> However, adding new DB users is time consuming, and we've had to delay
> rollouts twice now because of new DB users : we have a critical
> section around deploys once something lands in trunk, so it seems
> better to avoid potentially blocking things being in that section
> where we can easily do so. (And its straight forward to file an RT and
> get users added).
> 
> Lastly there is a High bug noting that this is an issue for us and we
> should do something about it.
> 
> I hope that makes sense.

Great, thanks.  I just wanted to make sure it was discussed, I am happy with 
whatever you guys decided.

Cheers.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] DB process change: new users must be set up on production before landing

[Julian Edwards]

Was this change discussed with the TA or anyone else?

On Friday 23 September 2011 02:29:55 William Grant wrote:
> Hi everyone,
> 
> I've just revised the DB user section of the DB change policy[1] to
> require that new users, or access to existing users from new machines,
> must be set up on production before a branch requiring the change is
> landed. Previously this was only required at some point before the
> branch was deployed.
> 
> This is similar to our rules around the sequencing of model and DB
> patches, where we now require prerequisite changes to be entirely
> deployed before anything dependent can land.
> 
> Moving these deployment sequence points to pre-landing makes deployments
> faster, easier and safer. I believe the slight delays brought by the
> policies to some landings are a good trade-off here.
> 
> Thanks,
> William
> 
> [1]
>  tes_on_Changing_security.cfg>

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] ec2 land not playing nice

[Julian Edwards]

On Thursday 22 September 2011 10:08:05 Jonathan Lange wrote:
> I don't know for sure. If it's a file, I'd guess maybe somewhere in
> ~/.launchpadlib or ~/.config/launchpadlib. However, that stack trace
> seems to indicate that a StringIO(value) is being passed around and
> causing the corruption. In which case, you might want to remove keys
> from your keyring. In GNOME Passwords and Keys, you want to look for
> "network password" and delete that.

I've had this before. Deleting the password under "network password" fixes it 
as jml says.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] performance tuesday: better batches are -go-

[Julian Edwards]

On Wednesday 21 September 2011 09:23:17 Jeroen Vermeulen wrote:
> On 2011-09-21 05:23, Robert Collins wrote:
> > https://code.launchpad.net/~adeuring/launchpad/bug-739052-9/+merge/76241
> > 
> > That is all.
> 
> What does this mean?  What does the "better" refer to?  I read the MP
> and the relevant change seems to be that you replaced ListRangeFactory
> with StormRangeFactory.  But the reason for that change is not
> explained, so now I have to find out or more or accept that I wasted my
> time trying to decypher your email.  :)

I had the same reaction so I just did a search in the dev wiki for 
StormRangeFactory, but it came up empty.

It would be super useful if you could document this there, with an example, 
and it'll lower the entry barrier and hopefully encourage people to use it.

Cheers.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] making it easier to determine if an instance has what it needs to work

[Julian Edwards]

On Thursday 15 September 2011 12:27:57 Robert Collins wrote:
> This is something we have much more direct control over, and can test
> effectively already. I think its reasonable to debate it, but its a
> totally different problem to the one I've raised ;)

I'm not sure it's any different.  It's a still a requirement of the process to 
have something set up to make it work.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] making it easier to determine if an instance has what it needs to work

[Julian Edwards]

On Thursday 15 September 2011 10:09:09 Jonathan Lange wrote:
> On Thu, Sep 15, 2011 at 5:01 AM, Robert Collins
>  wrote:
> ...
> 
> > Bug 760079 talks about a app server specific checker, but thinking
> > about this more perhaps we should bake into all our services and
> > scripts a 'dependency checker'.
> > 
> > E.g.
> > oops-prune --check-environment
> > would:
> >  - check it has the needed DB access
> >  - and $any $other $needed firewall holes
> > in a non-destructive way.
> 
> ...
> 
> > What do you think?
> 
> Seems a fairly sensible idea. I'd be worried a little bit about skew
> between what an app thinks it needs to run and what it actually needs.
> Perhaps it's best phrased as a dry-run start up, rather than a deps
> check?

Agree, both with the idea and these concerns.

One of the biggest skews we have that *consistently* hurts us is setting up DB 
users with the right permissions on tables.  This is largely because the 
security.cfg is not action-based, but user-based.  That is - we should be 
composing DB users' permissions out of a set of actions/roles (I think there's 
one example if this in the file but it's an exception), not setting up each 
user discretely.

However, I want to re-examine the requirement to set up explicit permissions 
on each user at all.  Why do we need this?  What is it helping to protect?  Is 
any of this worth the hassle and rollout problems?

This permissioning is also sometimes extremely hard to catch in testing. Quite 
often a particular set of permissions are only required in a tiny corner case 
that is not always relevant to the test intentions.

We already know that having separate users is useful to identify rogue 
connections etc, but I fail to understand why they need permissions.  This is 
particularly interesting given that we only do this with scripts - the webapp 
has carte blanche.

I would like do away with the discrete permissions and make all the users 
inherit from "main".  This would also bring the benefits of quicker tests 
since they don't need to commit and switch db users.

I'd love to hear opinions on this.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Build service thoughts

[Julian Edwards]

On Thursday 15 September 2011 01:34:04 Martin Pool wrote:
> > One source package is built for many distros and distro versions.
> > This would be nice to have in PPAs, again removing a minor hassle from
> > packagers of having to build and upload source packages multiple
> > times.
> 
> Right, this is somewhat harder to individually script around, and I
> think one of the bigger annoyances of PPAs today.  From the
> implementation point of view you do need different version numbers for
> each rebuild, but for the user it seems unreasonable.
> 
> We could do either of a couple of kludges:
> 
>  * some kind of easy flow around copying the binary between
> distroseries, for cases where it's appropriate
>  * a purely-automated tool to copy the package, add a changelog entry,
> reupload - like a subset of autoppa, but without needing any
> configuration aside from the choice of destination series.
> 
> I kind of like the latter: it would cope ok in the fairly common
> (cite?) case of one source package building adequately well across all
> active series.

There's bugs about this:
https://bugs.launchpad.net/launchpad/+bug/235064
https://bugs.launchpad.net/launchpad/+bug/225013

The former being more appropriate, but it's quite hard to fix - the bug 
explains why.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Ian Booth graduates to full reviewer

[Julian Edwards]

On Friday 02 September 2011 05:05:01 Jeroen Vermeulen wrote:
> We no longer have a team structure or regular meetings to pass this
> through, nor any standing procedures that I can find, so I'm winging
> this.  If I'm skipping a step: too bad, we'll patch it up later but the
> decision stands.
> 
> As Ian "wallyworld" Booth's review mentor, I'm satisfied that he needs
> no further mentoring.  In perhaps the world's only promotion of this
> kind, he loses a star.

Congrats Ian!

> Ian's recent review of Henning's JavaScript branch shows that Ian has
> grown from a shy, good-natured, if-that's-alright-with-you commenter to
> a vicious shark who will go as deep as it takes to find out what's wrong
> with your branch.  And that's how I like my reviewers.  Congratulations
> Ian!

I'm going to remember that next time I review one of your branches ;)

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Matthew Revell is the new Launchpad Product Manager

[Julian Edwards]

Congrats Matthew!

On Wednesday 24 August 2011 17:26:56 Francis J. Lacoste wrote:
> Hello Launchpadders,
> 
> It's my pleasure to announce that as of today, Matthew Revell is the new
> Launchpad Product Manager. It was a close race as we got a lot of very
> good candidates, but in the end, his bold vision for Launchpad along
> with his data-centric approach that he intends to bring to the role won
> our hearths.

He'll be setting a few fires in the hearths no doubt.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] New profiling/analysis features

[Julian Edwards]

\o/

Awesome!  Thanks Gary.  Did you put this on the dev wiki anywhere?

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] test rebuilds

[Julian Edwards]

On Monday 22 August 2011 15:23:49 Matthias Klose wrote:
> it would be nice if getBuildRecords would accept as parameters
> 
>  - the component. it takes a long time to walk through the list of build
>records, and usually I first want to start such a script for the main
>component.
> 

https://bugs.launchpad.net/launchpad/+bug/831862

>  - the architecture. same here, avoid build records I'm not interested in.
>that would speed up the bug filing (svammel project)/

It already takes an arch_tag.

> 
>  - maybe give a regexp which package names to skip, e.g. language-* or
>kde-l10n-*, which should not be rescored.

https://bugs.launchpad.net/launchpad/+bug/831864

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] test rebuilds

[Julian Edwards]

(CC to launchpad-dev for the attention of the maintenance squads)

On Friday 19 August 2011 23:07:15 Matthias Klose wrote:
> Hi,
> 
> started another test build today on the non-virtualized armel buildds.
> https://launchpad.net/ubuntu/+archive/test-rebuild-20110819/
> would be nice to have a ftbfs page for this rebuild too.
> 
> for the x86 rebuild, I'm currently running a script to score up builds on a
> regular basis. Looking at the currently running builds in the test archive,
> I'm rescoring builds up to ten builds on amd64 and i386. That means, that
> I'm using roughly half of the buildds, minus/plus some errors. Poor man's
> scheduling, but with the current scheduling, the builds just didn't happen
> (zero, or maybe two test rebuilds).
> 
> I'm currently screen-scraping the web pages for the test rebuilds, but get
> some inconsistent results. calling wget --no-cache seems to help, but not
> always. Would it be possible to expose some more things for the test
> rebuilds in the launchpad API?
> 
>   Matthias

What sort of API things do you need?  Builds are already fairly well exposed.

J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Help! Knowing what packages are in a distribution

[Julian Edwards]

On Thursday 18 August 2011 18:24:32 curtis Hovey wrote:
> A user can report a bug or ask a question about any package installed on
> their system via apport/ubuntu-bug/Get Help.
> 
> /me tries
> 
>  is deleted. It was published
> in Dapper and I cannot report a bug on it now since it was deleted. Okay
> maybe we do not need to support deleted packages since Lp does not
> currently support it
>
> Lp lets me ask questions, but I think we could consider desupporting
> this case to be consistent with bugs.
> 
> In the case of a branch, I am still learning the rules. Francis
> qualifies this as an official branch. What is official though? I think
> it means the branch linked to the package such as lp:principia/mysql
> which is actually shorthand for lp:principia//mysql (AKA
> SeriesSourcePackageBranch)

Based on the thread title you started - the package is not in the distribution 
if it's not published.  This is why I think unpublished source package 
branches are an odd case and IMHO are not "in the distribution".

> No it does not count, though that process is what created source package
> names that Lp current uses for bug and question targets. ubuntu-bug does
> not let me report bugs against  python-html5-browser since it is only in
> PPAs.

Based on the bridging-the-gap work I expect we'll need to support bugs on PPAs 
at some point.  Think of the situation where a developer needs bug reports on 
daily builds (which is one of the main reasons of doing daily builds).

> This issue also came up when working on bridging the gap, where DSP is a
> fulcrum for package, branch, upstream project, etc... We decided to do
> the minimum work do move on to something different after 13 months. I
> think the performance gain as well as the certainty of knowing valid
> DSPs makes this effort worth doing.

Also - nothing apart from Soyuz should ever be looking at publishing records, 
it's a gross violation of the interface if it does.  Soyuz definitely should 
be creating this summary info if other apps need it.

> I am inclined to study the removal of the DSP virtual object. I believe
> there are instances where Lp works with the DSP as virtual before
> deciding that it needs to me in the DB.

I think some places create one just so they can call canonical_url() on it.

> Ah. Yes we do need to know if the status is deleted. I think this means
> that an official package branch that was deleted and never published
> makes a DSP also deleted/obsolete.
> 
> Maybe obsolete is a better term to describe a DSP that is not used any
> more. I am not proposing that we delete it when the package/branch is
> deleted.

"Obsolete" is a specific publishing status for when a package was completely 
removed from the archive in a particular distroseries.  "Deleted" just refers 
to a version that was manually removed.

Everything else that's auto-removed has the "superseded" status.

We should definitely simplify any statuses for DSP, most things don't need to 
know this level of detail.

> So while I was thinking about this as a DSP issue there are cases where
> the bugtarget is (DistroSeries)SourcePackages. I think this happens via
> bug nominations; ubuntu-bug reports the bug on the DSP.  . We have no
> mechanism to ensure that source package branches are represented in the
> DSP table.

Right.  We could go for a DSSP table to replace the existing fake as well if 
required.  Again, I think some of the Soyuz pages would benefit.

(It's bizarrely named just "SourcePackage" in the current code)

> > What needs to happen if there's a branch for an unpublished source?
> 
> I am certain we know the source package name and the distro, which is
> enough to create the DSP entry.

See my earlier comment about whether this is really desirable.

Cheers.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Help! Knowing what packages are in a distribution

[Julian Edwards]

On Thursday 18 August 2011 16:11:00 Curtis Hovey wrote:
> This is a long email. If you have knowledge of packages, source package
> branches, and distros, please attempt to reply.

I'm attempting.  It's been a long day...

> A DSP is a valid choice if:
> * The source was published by the distro.
>   Once published, it is forever a valid choice.
>   
>  (still published)
>    (deleted,
>   yet still valid)

How did you decide on the validity rules?  Particularly the last one. (I am 
curious)

> Packages that were never published nor had a branch are not valid.
> 
> Was never published in Ubuntu. It was never published in any
> debian-based distro. No distro has a branch for it. It is invalid.

That page exists probably because the package exists in a PPA.  Does presence 
in a PPA count for anything?

(sorry, more questions than answers so far)

> Though there is work going on to demoralise the data to make the DSP

This is one of the best typos I've ever seen you make.  If it's deliberate to 
see if we were paying attention, then kudos.  If it wasn't, you still get 
kudos for giving me a massive belly laugh.

> queries fast, I doubt the vocab is a viable path to reach the goal shown
> in the demo. We need to search pillars, packages and branches quickly.
> 
> The rules imply that once a DSP determined to be valid, it is always
> valid, Lp needs to recheck invalid DSPs that may have been added.

As I mentioned above, why do you need deleted publications to be valid?

> I believe we need a resource that represents every DSP that is a valid
> choice. Lp does have a DSP table, and parts of Lp does treat it as a
> definitive resource, but it is flawed as William Grant pointed out. The
> table has two purposes, Store DSP information that is beyond the package
> and branch, like bug supervisor or bug reporting guidelines. It also
> stores facts like PO message count and bug heat. We want this data for
> every valid DSP, but not every DSP is in the table, and there are more
> than a thousand impossible DSPs in the table.

So there's two DSPs in the model code, as I suspect you know.  There's a fake 
one and a real one, which is in the DB.

We thought a while ago that getting rid of the fake one is a seriously good 
idea but there was never enough impetus to do the work.  It would solve a raft 
of problems with performance on some of the Soyuz pages.

The idea is that it would be a cache of the latest publication data and other 
bits required from other tables.  Keeping it up to date is pretty easy since 
we create all publications in once place in the code.

I'd be very happy to see this work done and the old DSP fake removed.

(I forgot the name of this data pattern but it's a common one I've seen and 
used before)

> 
> In the case if impossible DSPs in the table, they are mostly historic
> entries created by users to targeted a bug to a distro that does not use
> Lp to manage bugs. There is only one bad entry for Ubuntu, there are
> many for Debian that need investigation. There are about 1000 rows for
> distros that never used Lp to track bugs, do not have publishing
> history, do not have branches.
> 
> There are two reason for missing rows. Changes were made in the last
> year or two to ensure that every uploaded source package has a DSP
> entry. Packages that were in older Ubuntu releases are not present. I
> image we could make the missing DSP rows by mining the source package
> publishing history table.

Presumably this is only packages that are in older Ubuntu series but not in 
the latest?

In that case it needs entries for those packages to record that the latest 
status is "deleted".

Yes, we need to get some hard-hats on and go mining to fill these gaps.
 
> We have no mechanism to ensure that source package branches are
> represented in the DSP table.

What needs to happen if there's a branch for an unpublished source?

> Maybe the branch scanner could do this.

If it did, we'd need to be careful about what publishing data it has as per my 
idea above.

> Principia, a distro with just branches does have rows in the DSP table
> for the packages the owners have configured or have bugs. Most are not
> represented, so bugs cannot be retargeted to them. How do we ensure that
> source package branches are in the table?

It seems like the table needs to be written to from two different places, the 
branch scanner and the Soyuz publishing code.  Obviously that has the caveat 
of locking, data integrity and data overlap.  Maybe there needs to be another 
table?

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHe

[Launchpad-dev] For people doing support, "My upload hangs!" complaints

[Julian Edwards]

I created this FAQ for this, well, frequently asked question.
https://answers.launchpad.net/launchpad/+faq/1738

People uploading packages sometimes see their uploads hang with 1k to go.  As 
far as we can tell this is not a Launchpad bug, because it's now happened with 
two totally separate FTP server implementations.  It's most likely to be a 
router bug since FTP needs some packet inspection to make it NAT properly, but 
really we have no idea what causes it.

You can work around the problem using SFTP.  The above FAQ explains how to do 
it.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Trying to add storm tracing to scripts

[Julian Edwards]

> This tracer is installed by the act of importing its module, so the
> install_tracer() call is a no-op.
> 
> Next, you'll need to use set_request_started() before
> LaunchpadStatementTracer will start recording, then you'll need to
> trigger an oops before calling clear_request_started() at the end of
> the interesting bit (or just letting the script end). At least, this
> is how checkwatches does it... which may or may not be a good
> recommendation ;)

Sorted, thanks Gavin.

> See the WorkingBase class in lib/lp/bugs/scripts/checkwatches/base.py
> for some convenience stuff that should be easy to borrow.

My eyes - help, my eyes.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

[Launchpad-dev] Trying to add storm tracing to scripts

[Julian Edwards]

Dear hackers

I need to add storm statement tracing to one of the scripts we've developed 
for Derived Distros so I can see if there's a performance improvement to be 
made.  I figured it would be a simple case of adding these three lines to the 
script (as advised by Rob):

from canonical.launchpad.webapp.adapter import LaunchpadStatementTracer
from storm.tracer import install_tracer 
install_tracer(LaunchpadStatementTracer())

And then at the end of the script I should be able to do this:

import sys
import logging
from zope.error.interfaces import IErrorReportingUtility
oops = getUtility(IErrorReportingUtility).handling(sys.exc_info())
logging.info("LOGGED OOPS: %s", oops.id)

However the oops it produces has no statement log and interestingly no stack 
trace.

http://pastebin.ubuntu.com/664318/

I don't have much experience with tinkering with the OOPS tools and stuff, am 
I doing something wrong or is there something else that's broken?

Thanks guys.
J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] rocketfuel-setup and recommended packages

[Julian Edwards]

On Wednesday 03 August 2011 04:31:27 Robert Collins wrote:
> Is there any reason we shouldn't set '--no-install-recommends' for the
> rocketfuel-setup apt-get install lines? Makes VM setup a bit
> cheaper...

JFDI.  I also have this in my apt config.  I never, ever want recommended 
packages installed, I hate that default.

APT::Install-Recommends "false";

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Changes to Launchpad PPA for dpkg-xz-support and multiarch-translations

[Julian Edwards]

On Friday 29 July 2011 09:40:53 Colin Watson wrote:
> On Mon, Jul 25, 2011 at 01:26:09PM +0100, Colin Watson wrote:
> > My PPA:
> >   https://launchpad.net/~cjwatson/+archive/launchpad
> > 
> > ... contains changes to apt and python-apt which are required in order
> > 
> > to land these two branches:
> >   https://code.launchpad.net/~cjwatson/launchpad/dpkg-xz-support-619152
> >   https://code.launchpad.net/~cjwatson/launchpad/multiarch-translations
> > 
> > (Some of the changes were already in lucid-updates, and all of them are
> > in oneiric.)
> > 
> > Julian Edwards told me that I should stage them in a PPA and then they
> > could be reviewed and copied into the Launchpad PPA.  Could a Launchpad
> > developer who knows how to do this take care of it?
> 
> Anyone?  I'd really like to finish off this landing saga if possible.

The maintenance squads should be dealing with these sorts of requests.  I'll 
do it this time and I'm speaking to Colin directly about it.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] An open invitation to do a no-downtime-deploy

[Julian Edwards]

On Wednesday 20 July 2011 14:15:34 curtis Hovey wrote:
> You may have noticed that releases are most often done by our beloved
> antipodeans. While the work to request a deployment is trivial, the QA
> is not. Engineers may sacrifice a considerable amount of time checking
> that QA Staging is deployable. They had this time when most of them were
> working on maintenance, but all of them are now on feature work.
> 
> I am sure we all appreciate the work done by William, Steven, Ian, and
> Robert. Well done. I suspect that no-downtime-deploys would stop if
> these gentlemen took a one week holiday, or another large meteor strikes
> Au. I think it is time that others arrange for a  no-downtime-deploy.

Heartily agree.

Apart from said Antipodeans getting all the strife, it's actually a problem 
for everyone else losing out on a valuable experience.

Anyhow, I thought we'd already said that only maintenance squads would deal 
with NDT rollouts?

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

[Launchpad-dev] Reminder about a great Testtools matcher

[Julian Edwards]

I hope you all have the documents for Testtools[1] matchers on speed dial 
because they are awesome.  I thought I'd remind you about the MatchesStructure 
matcher because I used it a while ago, forgot about it, and re-discovered it 
recently.

Check out this diff snippet:

-self.assertEqual(source_name, copy_job.package_name)   
-self.assertEqual(version, copy_job.package_version)
-self.assertEqual(target_archive, copy_job.target_archive)  
-self.assertEqual(source_archive, copy_job.source_archive)  
-self.assertEqual(to_series, copy_job.target_distroseries)  
-self.assertEqual(to_pocket, copy_job.target_pocket)
-self.assertFalse(copy_job.include_binaries)
-self.assertEquals(PackageCopyPolicy.INSECURE, copy_job.copy_policy)
+self.assertThat(copy_job, MatchesStructure(
+package_name=Equals(source_name),  
+package_version=Equals(version),   
+target_archive=Equals(target_archive),
+source_archive=Equals(source_archive),
+target_distroseries=Equals(to_series),
+target_pocket=Equals(to_pocket),
+include_binaries=Equals(False),
+copy_policy=Equals(PackageCopyPolicy.INSECURE)))

I replaced a load of serially-processed assertions with a single assertion.  
This is good because:

 * It checks all of the conditions, not just up until the first one fails.
 * It prints out all the values if any of them fail to match.

I hope to see more use of this pattern!

Cheers
J.

[1]
http://readthedocs.org/docs/testtools/en/latest/for-test-authors.html#matchers

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Launchpad's tone of voice

[Julian Edwards]

On Tuesday 12 July 2011 12:00:10 Matthew Revell wrote:
> Sort of; by which I mean "yes". For at least two years now the company
> style guide has required British English. As we're aiming to bring
> Launchpad more into line with the Canonical brand, and we're
> undertaking a rebrand project, it seems like the right time for
> Launchpad to also switch to British English.

Spelling differences are probably only 50% of the story here since there's 
many phrases specific to British vs US English, and some of them will grate on 
the foreign ear.  For example, when Americans say "likely will", it makes me 
wince!

I've always advocated that we should strive to be as country/culture agnostic 
as possible.  This means we need input from native speakers of both types to 
make sure it's not awkward English for them.  In my above example, I would 
translate this to "is likely to be".

Hopefully at that point we'll only be left with unavoidable clashes of 
spelling with common words like colour/color where the company style dictates 
precedence.

Cheers.  ;)

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] story tests - any objects to a single mass conversion to regular pagetests

[Julian Edwards]

On Wednesday 06 July 2011 14:43:34 Francis J. Lacoste wrote:
> Hi Robert,
> 
> I'm sorry to disappoint you, but that won't work. Most of our story
> tests are now isolated. Only stories starting by a number are actually
> part of a unisolated sequence. A naive grep finds 68 such files that are
> part of a sequence. If it's only those sequence you wanted to collapse,
> and it will require a shell script a little bit more clever than your
> example one.

Was about to have a hernia and then you wrote this.

The idea of one massive .txt file is not only scary it would not work, in 
particular the old crappy soyuz page tests that depend on sample data.

As Francis says, cat the NN-*.txt together to fix the isolation, the others 
are fine AFAIK.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Etsy graphing using StatsD and Graphite

[Julian Edwards]

On Wednesday 06 July 2011 03:11:28 Martin Pool wrote:
> On 5 July 2011 19:30, Martin Pool  wrote:
> > I think I (or anyone) can easily change txstatsd to run the tuolumne
> > loader as a pipe subprocess. Aside from anything else, statsd gives the
> > apps nice isolation from the back end. We could even use two or more at
> > once.
> 
> ... I meant, two or more backends.  For instance statsd could send
> something to a lossy/summarizing rrd-like tool that is suited to
> drawing real time graphs, and also to a lossless warehouse that will
> record everything for ever, and also to a plain-text log file for
> ad-hoc analysis through custom scripts.  I don't know if we'd actually
> want to deploy that but it's a neat idea.

This stuff is all new to me but I really liked your demo last week.

Gavin and I were talking earlier about how to set up graphing for Rabbit, 
since our tuolumne approach of "write a huge query on the database" won't work 
for figuring out queue lengths and other Rabbity stats.

Apologies if I missed it earlier in the thread, but how do you think we could 
do that with current or future tools?

Cheers.
J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] ec2 seems horked (html5browser not importable)

[Julian Edwards]

On Tuesday 05 July 2011 12:38:43 John Arbash Meinel wrote:
> For example, the python redirect script that is loaded into memory
> whenever Apache starts loads enough of Launchpad's codebase, that it is
> about 130MB on a 32-bit machine, and 230MB or so on a 64-bit machine.
> I'm pretty sure that most of it could page out to swap since it doesn't
> get actively touched. But that is still almost 1/16 of my RAM for
> something that isn't being used. (Firefox can end up taking more RAM
> sometimes, but at least I was using it during that time. :)

I just filed https://bugs.launchpad.net/launchpad/+bug/806361 about this 
because it annoys me a lot too.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Ubuntu feedback for Distro Derivatives LEP

[Julian Edwards]

On Friday 17 June 2011 22:26:56 Bryce Harrington wrote:
> Thanks again to Julian and everyone working on the Distro Derivatives
> effort.  Can't wait to see the sync button make its debut.
> 
> Here is a summary of issues/ideas raised by ubuntu users.  I've entered
> bug reports for each item brought up.  I know we're not a stakeholder
> and so our feedback may not be considered as relevant in the immediate
> term, but perhaps you'll consider doing follow up work later.

Thanks very much for collating these bugs Bryce, we'll do what we can to fix 
as many as possible.

Cheers
J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Changing Lp to not conflate public, private, and securty visibility

[Julian Edwards]

On Wednesday 22 June 2011 03:43:38 Curtis Hovey wrote:
> The "managing disclosure" feature of the "disclosure" feature  will
> permit project/distro maintainers to see all subordinate public and
> private artefacts. By assigning a person or team to the roles or
> maintainer, driver, bug supervisor, or security contact, you will be
> granting knowledge/access to the projects private artefacts. This does
> not convey access to security artefacts like bugs (and branches). The
> person or team in the security role will have access to public, private,
> and security artefacts.

Are you going to differentiate at all between "knowledge of" and "access" in 
this feature?

If "knowledge of" is important, then I would have thought that to make a top-
level object like a distro or project private then you need all of its 
components to be private.  For example, below I've listed all the tables that 
have a FK to distribution - wouldn't you need to make all of these objects 
private before you can have a truly private distro?

Apologies if I am way off track here, I am happy to be corrected on what your 
aim is.

Cheers.



TABLE "announcement" CONSTRAINT "announcement_distribution_fkey" FOREIGN 
KEY
 (distribution) REFERENCES distribution(id)
TABLE "answercontact" CONSTRAINT "answercontact__distribution__fkey" 
FOREIGN
 KEY (distribution) REFERENCES distribution(id)
TABLE "archive" CONSTRAINT "archive__distribution__fk" FOREIGN KEY 
(distribu
tion) REFERENCES distribution(id)
TABLE "bugsummary" CONSTRAINT "bugsummary_distribution_fkey" FOREIGN KEY 
(di
stribution) REFERENCES distribution(id) ON DELETE CASCADE
TABLE "bugsummaryjournal" CONSTRAINT "bugsummaryjournal_distribution_fkey" 
F
OREIGN KEY (distribution) REFERENCES distribution(id) ON DELETE CASCADE
TABLE "bugtask" CONSTRAINT "bugtask__distribution__fk" FOREIGN KEY 
(distribution) REFERENCES distribution(id)
TABLE "bugtrackercomponent" CONSTRAINT 
"bugtrackercomponent_distribution_fkey" FOREIGN KEY (distribution) REFERENCES 
distribution(id)
TABLE "customlanguagecode" CONSTRAINT 
"customlanguagecode_distribution_fkey" FOREIGN KEY (distribution) REFERENCES 
distribution(id)
TABLE "distributionbounty" CONSTRAINT "distributionbounty_distribution_fk" 
FOREIGN KEY (distribution) REFERENCES distribution(id)
TABLE "distributionjob" CONSTRAINT "distributionjob_distribution_fkey" 
FOREIGN KEY (distribution) REFERENCES distribution(id)
TABLE "distributionmirror" CONSTRAINT 
"distributionmirror_distribution_fkey" FOREIGN KEY (distribution) REFERENCES 
distribution(id)
TABLE "distributionsourcepackage" CONSTRAINT 
"distributionpackage__distribution__fk" FOREIGN KEY (distribution) REFERENCES 
distribution(id)
TABLE "distributionsourcepackagecache" CONSTRAINT 
"distributionsourcepackagecache_distribution_fk" FOREIGN KEY (distribution) 
REFERENCES distribution(id)
TABLE "distrocomponentuploader" CONSTRAINT 
"distrocomponentuploader_distribution_fk" FOREIGN KEY (distribution) 
REFERENCES distribution(id)
TABLE "distroseries" CONSTRAINT "distroseries__distribution__fk" FOREIGN 
KEY (distribution) REFERENCES distribution(id)
TABLE "entitlement" CONSTRAINT "entitlement_distribution_fkey" FOREIGN KEY 
(distribution) REFERENCES distribution(id)
TABLE "faq" CONSTRAINT "faq_distribution_fkey" FOREIGN KEY (distribution) 
REFERENCES distribution(id)
TABLE "karma" CONSTRAINT "karma_distribution_fkey" FOREIGN KEY 
(distribution) REFERENCES distribution(id)
TABLE "karmacache" CONSTRAINT "karmacache_distribution_fkey" FOREIGN KEY 
(distribution) REFERENCES distribution(id)
TABLE "message" CONSTRAINT "message_distribution_fk" FOREIGN KEY 
(distribution) REFERENCES distribution(id)
TABLE "milestone" CONSTRAINT "milestone_distribution_fk" FOREIGN KEY 
(distribution) REFERENCES distribution(id)
TABLE "oauthaccesstoken" CONSTRAINT "oauthaccesstoken_distribution_fkey" 
FOREIGN KEY (distribution) REFERENCES distribution(id)
TABLE "oauthrequesttoken" CONSTRAINT "oauthrequesttoken_distribution_fkey" 
FOREIGN KEY (distribution) REFERENCES distribution(id)
TABLE "officialbugtag" CONSTRAINT "officialbugtag_distribution_fkey" 
FOREIGN KEY (distribution) REFERENCES distribution(id)
TABLE "pillarname" CONSTRAINT "pillarname_distribution_fkey" FOREIGN KEY 
(distribution) REFERENCES distribution(id) ON DELETE CASCADE
TABLE "publisherconfig" CONSTRAINT "publisherconfig__distribution__fk" 
FOREIGN KEY (distribution) REFERENCES distribution(id)
TABLE "question" CONSTRAINT "question__distribution__fk" FOREIGN KEY 
(distribution) REFERENCES distribution(id)
TABLE "specification" CONSTRAINT "specification_distribution_fk" FOREIGN 
KEY (distribution) REFERENCES distribution(id)
TABLE "structuralsubscription" CONSTRAINT 
"structuralsubscription_distribution_fkey" FOREIGN KEY (distribution) 
REFERENCES distribution(id)

___
Mailing list: https://lau

Re: [Launchpad-dev] [RFC] LEP: Build from branch into archive

[Julian Edwards]

On Tuesday 21 June 2011 14:15:58 Curtis Hovey wrote:
> Is the goal just to rebuild a package in an archive to ensure deps are
> met and that build links to changed binaries?

No, we already have a way of doing that (which will in fact be eventualy 
superseded by the derived distros work).

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

[Launchpad-dev] Bad coding day

[Julian Edwards]

Next time you make a code mistake, don't feel too bad.  This one takes the 
biscuit:

https://github.com/MrMEEE/bumblebee/commit/a047be85247755cdbe0acce6#diff-1

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Derived Distros / Version Merge Worksheet

[Julian Edwards]

On Thursday 16 June 2011 10:35:04 Bryce Harrington wrote:
> Sorry, yeah I've been aware of the effort but the elevator pitch seemed
> geared towards OEMs so I never really took a good look at it until
> lifeless showed it.
> 
> I'm mindful of Jono's point about feedback at the tail end of a LEP can
> do more harm than good...

Heh.  We've actually had good feedback all the way through the development 
cycle from various OEM, Linaro and Ubuntu guys.

> Fair enough.  I'm sharing them more for you to suck useful ideas out of
> more than anything else.  I suspect both GNOME and X guys are pretty set
> on our own reports.  But the functionality in them might be of some more
> general interest.

Absolutely, thanks for pointing them out.

> Not quite that elaborate, it just indicates difference vs. debian and
> difference vs. upstream.  So, green if everything's in sync with both
> upstream and debian, red if it's not tracking either upstream or debian,
> and an intermediate color if it's something in between.  So, if you
> ignore upstream, a simplified version of this might go something like:
> 
> * Given a version scheme A.B.C-X[ubuntuY] and two versions of a package
> 
>   1.  Color green if derived distro's package has same or greater A, B,
>   C, X numbers.
>   2.  Color yellow if derived distro has a lower X value
>   3.  Color red if derived distro has lower A, A.B, or A.B.C values

As per my previous email, do the filtering options and other pages get 
anywhere close to useful?

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Derived Distros / Version Merge Worksheet

[Julian Edwards]

On Thursday 16 June 2011 10:11:45 Julian Edwards wrote:
> The colouration is interesting.  What is it based on, the package version
> divergence or the debdiff size?

Acutally I just saw the legend at the bottom of the page:

{{{
Ubuntu package is latest upstream
New unstable version available
Newer version available in Debian
Synchronised with Debian but newer version available upstream
Unsynchronised and newer version available upstream
Newer upstream and Debian versions available
}}}

The different filtering options and the two other pages we have go some way to 
addressing this.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] Derived Distros / Version Merge Worksheet

[Julian Edwards]

On Thursday 16 June 2011 09:20:38 Bryce Harrington wrote:
> Hi all,
> 
> lifeless today pointed me at
> https://launchpad.net/ubuntu/oneiric/+localpackagediffs which is exciting
> stuff.

We only started on it 12 months ago :)

See all the other similar pages at
https://launchpad.net/ubuntu/oneiric linked from the "Derived from Sid" 
portlet.

> You may already be aware of this, but several of us on the distro have
> "worksheet reports" to help us track packages needing merges, somewhat
> similar to +localpackagediffs:
> 
>  
> http://www.bryceharrington.org/X/Reports/ubuntu-x-swat/versions-current.ht
> ml http://people.canonical.com/~platform/desktop/versions.html

I had not seen those before, just the merges.ubuntu.com pages.

> Probably the two main things worth noting are the inclusion of links to
> the ultimate upstream package, and the heavy use of color to help show
> the degree of difference between the two distros, and to upstream.

The page links to the upstream package inside LP and the feature is designed 
to only work inside LP for fully-hosted distros.  Debian is a bit of a special 
case because we import its source packages into LP so that we can generate 
that page.  I can't see us doing anything special like those reports any time 
soon I'm afraid but I'm more than happy to take suggestions on how to improve 
the existing page in a generic fashion.

The colouration is interesting.  What is it based on, the package version 
divergence or the debdiff size?

J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] RFC: change from using lint to using a pretty printer

[Julian Edwards]

On Tuesday 14 June 2011 10:13:06 Robert Collins wrote:
> Can you expand on the value you get? What does it do for you / us to
> be told about whitespace and indentation variation?

Well like I said, pyflakes catches most of them for me.  If someone else 
doesn't fix their lint then it shows up in my editor.

I appreciate the aesthtics of neat and tidy code, formatted as the coder 
intended.

> I've had very good experience with them on every project that got them
> up and running (and smashed on a few times to sort teething-issues
> out) in the past.

Maybe my experience is more tainted than yours :)

> So, to me, just about all of the format-and-layout warnings are
> useless: they don't make our code less buggy; my argument here is to
> free our brains to focus on the stuff that matters; lower the bar for
> casual contributors [by lowering the workload for all developers].
> Many folk have internalised the rules, but many more haven't.

Good formatting is part of good coding practice IMO, and helps prevent the 
bugs appearing in the first place in my experience.  But I agree that this 
should not be a barrier to entry for casual contributors.

I'm still -1 but I'll wait for other opinions and go with the flow.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] RFC: Is readonly mode fixable, or should we ditch it entirely?

[Julian Edwards]

[snip]

I think this is all a no-brainer.  Let's JFDI.

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] RFC: change from using lint to using a pretty printer

[Julian Edwards]

On Tuesday 14 June 2011 01:55:28 Robert Collins wrote:
> Currently we get two (IMO) distinct benefits from running linters:
>  * we find out about functional issues:
>- repeated definitions
>- unsafe constructs
>  * We find out about cosmetic issues:
>   - whitespace
>   - indentation
> 
> I think we get great value from the former, and limited if any value
> from the second.

I get good value from both, quite frankly, excepting the stupid warnings like 
the one about having a blank line at the end of the file.

> So, I'd like to propose we do something other projects do: restrict
> lint to reporting on functional issues, and use an idempotent pretty
> printer to reformat our code automatically.
> 
> We would add a post-commit hook in pqm that would reformat the changed
> files using the pretty printer, and we'd no longer need to spend any
> time thinking or worrying about formatting.
> 
> What do you think?

-1

I don't trust pretty printers any further than I can throw 'em.  I also want 
to indent and format stuff as *I* like within the rules.

As Gavin said, let's just suppress the useless warnings in the linter and 
carry on as we were.  I suspect most of us have pyflakes in our editors 
anyway, which means we get very few genuine bits of lint.

Cheers
J

___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Launchpad-dev] how should we make sure new pages meet our performance goals?

[Julian Edwards]

On Wednesday 08 June 2011 11:06:54 Robert Collins wrote:
> If anyone has ideas on how we can sensibly meet the third requirement
> during feature development, please share!

We've added tests on our new pages that demonstrate a constant query count no 
matter what data is there.


___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help   : https://help.launchpad.net/ListHelp