On Sun, Feb 20, 2011 at 8:35 AM, Pascal Bleser <[email protected]> wrote: > FOSDEM is finally done, now I have time for Packman again ;) > > As you might already have noticed, our current OBS instance is > broken, it has severe issues with noarch packages, amongst other > things. > Fixing it is very difficult, I tried to diagnose the problem but > it partly comes from the OBS itself, and partly from my script > that inserts incoming packages into our website's database. > And it's a very old OBS instance (1.7) anyway... > > Detlef already took care of almost everything (*huge* thanks to > Detlef!): > * we have an OBS 2.1 instance that is properly configured, and > up and running on one of the worker hosts > * we discussed the new repository layout a while ago, and came > to an agreement (see below) > * the new OBS already exports repositories to our FTP server and > on mirrors > > What is merely missing now is > 1) implement the glue between the repositories/packages and our > website > 2) move all our packages from the old OBS to the new OBS > > THE NEW OBS INSTANCE > ~~~~~~~~~~~~~~~~~~~~ > It is currently running in an openSUSE 11.3 chroot on one of the > worker hosts, which is only temporary. Once everything is > properly set up, we will scrap the old OBS on > pmbs.links2linux.org and move it there. > > It already implements part of the new repository layout (see > below), namely the "Essentials" repository, which is exported to > our FTP site: > ftp://ftp.links2linux.de/pub/packman/suse/ > > The difference here is that it exports to the subdirectories > "openSUSE_11.2", "openSUSE_11.3" and "Factory" (instead of > "11.1", "11.2", "11.3" and "factory), to adhere to the > conventions of build.opensuse.org. > > Those directories are also on our mirrors. > > THE NEW REPOSITORY LAYOUT > ~~~~~~~~~~~~~~~~~~~~~~~~~ > We discussed several options on this mailing-list in December > [1], and came up with the following solution: > > [1]http://lists.links2linux.de/pipermail/packman/2010-December/009213.html > > * we will split Packman into four repositories: > * Essentials > * Multimedia > * Games > * Extra (or "Other", I don't remember :)) > > * we will also, using a little trick, provide one "big" > repository that aggregates the content of all that > > The idea is that by having more fine-grained repositories, > people can decide to only add the packages that are only > provided on Packman and nowhere else (Essentials), without > having duplicates. It makes it also easier for us, because we > currently cannot build everything for Factory and SLE11 (and > Tumbleweed, which is slowly starting to get ready), so we can > limit it to building Essentials only. > > The "big repository" will not cause any duplicate efforts, as we > simply use "createrepo" to generate the RPM-MD metadata for all > the packages in all the subdirectories (Essentials, Multimedia, > Games, Extra) into the top-level directory. > > The directory layout on the FTP server and our mirrors will > hence be as follows: > > /packman > /suse > /openSUSE_11.3 > packman.repo ............ (1) > /repodata ............... (2) > /Essentials > /i586 > /x86_64 > /noarch > /src > /repodata ............. (3) > Essentials.repo ....... (4) > /Multimedia > /i586 > ... > /repodata > > (1) add this .repo file for everything > (2) contains the RPM-MD metadata for everything > (3) contains the RPM-MD metadata for Essentials > (4) add this .repo for Essentials > > THE WEBSITE > ~~~~~~~~~~~ > I managed to hack all night and implement a script that stores > the repository information into our database. > It does have some major differences with the old approach, which > also makes it slightly simpler: > * the source is the repository on the filesystem itself, merely > using the RPM-MD metadata (repomd.xml and primary.xml.gz) > without having to analyse the RPM files > * it inserts missing "packages", "releases" and "files_bin" into > the database > * it removes "packages", "releases" and "files_bin" that don't > exist in the metadata from the database > > In order for the website to work without modifications, it will > always run on the RPM-MD of "everything", without taking care of > the different repositories (Essentials, ...). > If desired, that's still something we could implement in the > future as well (e.g. show in which Packman repositories the > package is present), but that will require changes to the > database layout as well as to the website itself. > > The script is in Perl, on gitorious: > http://gitorious.org/packman/rpmmd2pmdb > > TODO > ~~~~ > So, now is the time to assess what we still need to do. > Off the top of my head: > * minor details in the above mentioned script, such as updating > the description of "packages" as well as their "last_update" > timestamp (will be done today) > * migrate our packages into the new OBS instance > * kill the old OBS instance > * move the new OBS instance to pmbs.links2linux.org > * migrate our packager accounts > > MIGRATION > ~~~~~~~~~ > Migrating the packages is obviously the most tedious part of the > job and, as far as I know, Detlef already did everything for the > "Essentials" repository (yay!). > > Detlef, can you give us an update on what is already done and > what still needs to be done ? > > cheers,
Just out of curiosity, what is happening with the opensuse 11.4 repos besides essentials? It looks like they are building successfully and are being published, but I don't see them anywhere. Are they available somewhere? -Todd _______________________________________________ Packman mailing list [email protected] http://lists.links2linux.de/cgi-bin/mailman/listinfo/packman
