-----Original Message-----
From: Jürgen Schmidt [mailto:[email protected]]
Sent: Thursday, 8 December 2011 6:40 PM
To: [email protected]
Subject: Re: Extensions and templates
On 12/8/11 12:50 AM, Gavin McDonald wrote:
-----Original Message-----
From: Andrea Pescetti [mailto:[email protected]]
Sent: Thursday, 8 December 2011 8:27 AM
To: [email protected]
Subject: Re: Extensions and templates
On 29/11/2011 Rob Weir wrote:
==Option 1: Remain at OSUOSL==
We could remain with OSUOSL hosting. However, the existing site is
very unstable.
This would be best both for short and long term. In the short term,
it provides continuity of service and it doesn't break existing
links. In the long term, the Drupal instance can be updated and
extended (it's not easy, but it is something that would have fairly
high chances of
success) to get it to be something like the new model (distributed
repositories) you describe in step 5.
Sorry, OSUOSL don’t want anything to do with these any longer, I
thought folks got the hint when they turned off monitoring and no
longer look at the issues of the host, but rather restart it only when
prompted. The hosts themselves cannot cope with all the memory and cpu
these are consuming all the time, let alone the bandwidth.
This is no longer an option.
An important point that everybody seems to miss is that the
instability of the current Extensions site
http://extensions.services.openoffice.org/ is, very likely, unrelated
to Drupal. The underlying Drupal instance is rather sound (and it
perfectly managed to sustain the traffic in 2010, which should not be
significantly different from 2011); from the behavior of the site, it
definitely seems to me that the instability is due to other components,
like the caching server (Varnish) in front of it or other caching mechanisms.
I very much disagree. Caching can be tweaked for sure, but I've had a
look around the drupal sites and it is not optimal to say the least.
A second very important point is that we need to get the Extensions
and Templates source code (two different codebases) under the SGA;
while Drupal itself is GPL, Thorsten Bosbach while working at Oracle
created a lot of custom PHP code for the two sites. This code, as far
as I know, has never been released. Access to the source code is a
prerequisite for any possible analysis/improvement of the website.
==Option 2: Move Critical extensions to stable host==
Indeed, as you write, this would be an extreme option.
More extreme would be to do nothing, as you'll end up with nothing.
==Option 3: Clone OSUOSL repositories to another host==
This is not significantly different from Option 1; i.e., if there are
other hosting options available the mere cloning of the site would
not take long, but again the problem is not with the site but with caching.
Do not blame caches for poor performance. the caches are improving a
bad situation, they can be tweaked to improve further.
Note that, since the Templates site has already been ported to Drupal
6 using the so called "code-driven development" technique, that
source code would allow to install an empty pre-configured clone of
the Templates site anywhere. This would be extremely useful for testing.
==Option 4: Host repositories elsewhere, using new UI==
As I used to say, everybody who thinks that the Extensions or
Templates sites can be replaced easily has never tried submitting a
template!
Thorsten did a lot of customization work on the two sites; any
replacement would provide a largely inferior user experience.
I think you don’t think very highly of other peoples abilities, a poor outlook.
==Option 5: Re-architect the Repositories== This is the option I
personally favor for long term. ...
This would allow multiple
repositories to look and behave identically from the data perspective.
This is an interesting long term solution indeed, but I see it
feasible as a
(complex) version of Option 1-3; i.e., we obtain the current codebase
with the aim of updating it and extending it in this direction.
The other thing this approach does is separate the extension
metadata from the actual licensed extension. If we wanted to have a
canonical repository of registered extensions, but without actually
hosting or storing the extensions, then that should be OK. We're
hosting URL's to resources. We're not distributing code.
This would offer some advantages, but I see advantages in offering
hosting for extensions too. The current Extensions site offers both
options (host there or externally), but if I recall correctly some
automatic mechanisms, like autogeneration of the update URL, only work
if the local hosting is used.
Having spoken to OSUOSL, having looked around the machines and
services in question and having looked at and been told of the
excessive bandwidth (and that is MUST stop), here is the route I intend to
take:
1. Move the services to a newer more modern host at the ASF
(temporary) 2. BandAid the installation to stabilise it for the short
term (this is still more work than it sounds) 3. Stick Apache TrafficServer in
front (not varnish) to improve response times / caching.
4. Go with the choice of Option 5. that is, to allow the hosting and
downloading of the templates
and extensions to be with the 3rd party authors. We will hold master
copies, and provide metadata
and links to the download locations / master sites, but we will not allow
downloading directly.
This will solve the excessive bandwidth issues longer term. I intend to
start the work of this sometime
in January.
we should keep in mind that not every extension or template developer has
the opportunity to host the extension or template themselves.
And then we have potentially the problem that the third party sides are not
available when users try to download. Very poor user experience. Ok the
moment it is also bad but we are looking for a long term solution.
If Apache is not able to host such a repository, we can of course think of
multiple repositories in the future.
The Apache one would be the default. And here we can host extension that
are Apache conform and potentially hosted on our svn or apache-extras.
If you or anyone else here has any complaints or issues or further
idea, please bring them to the infra team now as I intend to get cracking in
this very soon, the status quo can not continue, for benefit of all.
Help welcomed at any step of the way.
(Note that moving services from OSUOSL hosts to the ASF hosts does
nothing to solved the bandwidth issues because the ASF servers are
also OSUOSL hosted!)
ups, that is not really promising when i think of
- a future download of OOo binaries.
- the svn performance
- the wiki performance (confluence wiki)
