Hi Graham, tl;dr Thanks for looking in this issue. I think the let's upload to twisted is a big hand wave at the moment and we should probably investigate why apache is failing.
I don't understand how allowing direct upload to the librarian would
solve this bug. It might, but I think there are quite a few loose ends
that need to be connected first.
For reminder, the problem is that the file upload form fails at the
apache level in some nebulous but - not so uncommon - cases. That form
handles upload both the content (which ends up in the librarian) and the
metadata (creating the release and linking it to the file content).
So loose ends:
* As a general deployment aren't we always running apache in front of
twisted? So uploading to the librarian would fail similarly than
now.
* It's interesting that users report that creating the release using
the API script doesn't suffer from this bug. In which case, this
might point to a apache <-> browser bad interaction.
* How are we going to handle the cross domain issue (loads from
launchpad.net, posts to launchpadlibrarian.net - without going
through apache?)
* Are we going to write a "specific" view in twisted to take care of
all the data handled by this form. Or more specifically, devise a
generic protocol where once the content is stored, the librarian
redirect to the real view with the file alias reference + the
other metadata originally passed in)?
And I'm pretty sure that "solutions" to any of these problems will open
up a bunch of other issues. Making that path really too much for what we
are trying to achieve (work-around a weird apache bug).
Simpler fix:
* See if playing if timeouts on apache change the behavior.
* Change (or make an alternative yui-based upload form) that uses the
yui flash-based upload widget (maybe the flash http doesn't
trigger the apache bug, like the launchpadlib script does).
* Just add warnings to users that they use the launchpadlib script if
they encounter repeated upload errors.
Cheers
On 11-10-03 01:13 PM, Graham Binns wrote:
> Calling all folks, hello folks...
>
> I've been staring at bug 194558 (Project file uploads time-out but don't
> OOPS)[1] for the last few days. The problem is twofold:
>
> 1. Uploads occasionally fail at the Apache level - i.e. Apache has the
> upload but can't pass it on to the appserver.
> 2. We don't get OOPSes when this happens.
>
> I've been looking at problem #1 (even though #2 is probably the easier
> to fix). Our fearless TA made the following suggestion, complete with
> hand-waving, in the bug comments [2]:
>
>> It may be easiest to fix this by moving uploads out of the zope
>> appservers, which our infrastructure is not geared around supporting
>> big uploads too. E.g. upload directly to the librarian, then register
>> the content with LP (/handwave)
>
> And I'm interested in investigating this further. As far as I can tell,
> we have the following issues to solve:
>
> 1. How do we upload directly to the librarian from the browser? (AIUI
> the librarian uses an almost-but-not-quite-HTTP-like protocol, which
> makes things awkward, but if I've misunderstood then please feel
> free to tell me and so make me very happy).
> 2. How do we register the content with LP once the upload has
> completed? (The answer that both Gary and I have arrived at involves
> the words "hey, look at this RabbitMQ thing over here...")
>
> Really, I'm looking for any information that anyone has on how the
> Librarian works, since I don't have a great understanding of it. If I'm
> wrong about the protocol then this is a relatively simple bug to fix. If
> I'm right then I suspect I'll need your help to work it out.
>
> [1] https://bugs.launchpad.net/launchpad/+bug/194558
> [2] https://bugs.launchpad.net/launchpad/+bug/194558/comments/6
>
--
Francis J. Lacoste
francis.laco...@canonical.com
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
