Thanks for comments. Rob. I was hoping to get your and others thoughts
on this TLD structure issue: Where do we plug the wiki, the forums, the
other website services into our svn hierarchy (where XXXX=the relevant
service):
* incubator/ooo/site/trunk/XXXX
or
* incubator/ooo/site/XXXX/trunk
or where?
There's no clear slot in our current TLD structure. I've put my
responses on the rest below.
On 29/08/11 14:59, Rob Weir wrote:
On Mon, Aug 29, 2011 at 7:15 AM, Terry Ellison<[email protected]> wrote:
I have tracked the various svn discussions on the DL so far. This note is
to initiate a discussion on the specific topic of how we should organise the
top-level-directory (TLD) structure within svn. As I have mentioned
elsewhere, family circumstances have underlined that any of use could
suddenly leave this project either as a conscious decision or due
circumstances quite outside our control, and this really underlines the
issue that all of our work should at some level be brought under proper
configuration management (CM) and revision control.
IMHO, the current TLD split branches, site, tags, trunk is missing an
aspect. The branches and trunk are the main roots for the OOo code-base its
self, but the OOo also spans a wide range of services which should also be
under proper CM. At the moment "site/trunk" superficially has the
organisation of the "OOo website" but we don't have just one single
homogeneous website; we have a range website services which we are migrating
over to Apache and surely the work on *all* of these need to be brought
under CM.
Let me discuss one example. We run 10 national language (NL) forums based
on the "off-the-shelf" FLOSS package phpBB. However:
* We also support 16 different language variants, each with its own
language pack. (For example we have EN and EN-US so which reflects the
different spelling practice either side of the "pond".) And multiple
languages are supported on each forum (for example I always use English
(plus occasional google-translate), even when working on the Japanese forum.
* An out-of the-box phpBB installation is one code-base per (NL) forum,
which simply doesn't scale across 10 NL instances, so in our configuration
all 10 instances share the same code-base.
* We have a number of forum customisations which have been requested /
negotiated by our community groups.
* phpBB is regularly upgraded by its maintainers (~ every 6 months), on
each upgrade all of the above needs to be regressed across the new version
of the package.
So my issue on the "forums" service is what should we keep under CM and
where. So for example:
* The underlying platform, that is the VM which hosts the service, is
maintained under CM within /infra/infrastructure/trunk/machines/vms/ and
/infra/infrastructure/trunk/docs/machines. However, these only relate to
the "root stuff" done by and maintained by the infrastructure team.
* There is the application, which is maintained by one (or hopefully
more than) one application maintainer(s). The application maintainers do
this work within their normal interactive log-in and don't use root
privileges. Here we *don't* need to keep under CM the base phpBB + 16 NL
packs, because in this case the phpBB developers already do this. However,
I have a standard scripted process for unpacking this tarballs into correct
destination directories, applying patches, and any customisation add-ons,
and version upgrades, etc. It is this meta-application stuff -- these
unpack and build scripts, and the OOo site-specific custom add-ons, and
build documentation which needs to be under CM within the /incubator/ooo
tree.
* There is the application content, which is huge and changes on a
minute-by-minute basis, but which is not within CM (in the svn sense), but
must be backed up properly through a documented backup / disaster-recovery
policy.
* And lastly we have one-off task related items, for example in the case
of the forums I am developing a process and scripts which specifically
relate to the migration of the content and cut-over of the service from
Oracle to Apache.
I have exactly the same hierarchy for the ooo-wiki and I would expect the
same to be happening for Kay's and Raphael's work also. So getting down to
hard-tacks,
* Where do we home the *ooo-forums* node in this tree (or any of the
other service sub-systems)?
* Within the *ooo-forums* node, should we have a standard next-level
sub-structure and if so, then what?
* What do we do about existing nodes which do not follow this standard.
For example /incubator/ooo/site/trunk/tools seems to really be a small
subset of the kenai migration scripts so (if we accept my argument) should
really be in /incubator/ooo/site/trunk/kenai/tools or equiv.
Could we do something like this:
1) Check in the source for whatever base phpBB we're customizing. So
the "official" phpBB release becomes the trunk.
2) Our customizations occur in a branch
Tried this before on projects. It really doesn't work. There are
~2,500 files of which we update about 20-30 with a single patch file.
If we do it the way you suggest, we would have a huge bulk of revisions
every phpBB release. It's a lot easier to keep the build script and the
patch file under CM and then we only have two files to update every
release.
3) We try, over time, to push relevant patches upstream to phpBB, to
reduce future merge work.
Already do this where I can and have retired a couple of patches this
way, but the phpBB team are only interested introducing truly general
and desirable changes into phpBB core. phpBB includes a concept of
plug-in modules and hooks, but the current architecture doesn't really
facilitate the sort of mods that we do. However, the mods architecture
is being overhauled in the future phpBB v3.2 and this might enable me to
move far more of the patch content into a proper ooo-site module. But
this is futures, not the current phpBB version.
4) We have a script to push our customized phpBB branch into
production (or INFRA has a script to pull it)
Already do this.
5) When we get new patches, revisions, etc., from phpBB we check that
into trunk and then do a merge on our branch
In other words, we keep the unaltered phpBB in SVN, and do our
customization work in a branch.
I am really not sure if there any advantages at all in doing this and it
involves a LOT more work. Given your next point, I think it sensible to
avoid nugatory work.
Backups of content, I think, are unrelated.
Another thing to consider is asking, longer term, who is going to
maintain phpBB @ Apache? Are we the only project that will use it?
Or do we think that others will adopt it as well? If not at a project
level, maybe Apache would find it interesting at a foundation-level
for interactions with the public? If we can encourage broader use at
Apache, and get a couple more people able and willing to support it at
the INFRA level, then it would make more sense of the base code to be
at the INFRA level.
-Rob
The whole issue of my being a SPoF for the phpBB application is a
concern, though Drew already has a pretty good understanding and could
step in in extremis. Having another (hopefully younger generation)
developer tracking all this would be a good thing, but perhaps the
easiest step-one would be to get everything properly baselined in SVN,
then look for volunteers.
Regards
Terry
