On Mon, 9 Mar 2009 09:54:00 -0700
Shawn Willden <shawn-ta...@willden.org> wrote:
> On Monday 09 March 2009 10:24:58 am zooko wrote:
> > The Debian package builder on Shawn's intrepid box is successfully
> > building .debs, but has nowhere to upload them. Brian, Shawn: could
> > we arrange it so that it uploads the .debs to a directory on a Tahoe
> > grid? :-)
>
> There's a Tahoe node that usually, but not always, runs on the intrepid
> box, connected to the test grid. Probably a better choice is the test grid
> node that is running on a neighboring machine (IP 10.0.1.1, port 3456).
For this purpose, we might want to upload them to the allmydata prodgrid
instead.. might be more reliable and less stall-sensitive. I dunno.
For the record, here's our overall plan:
* create a master tahoe directory structure, in the shape of a regular APT
repository
(dists/{gutsy,hardy,intrepid}/tahoe/{source,binary-amd64,binary-i386}/*.deb)
* extract the writecap for dists/intrepid/tahoe/binary-amd64 , give it to
Shawn, he writes it into a file accessible by his buildslave (maybe by doing
'tahoe set-alias tahoe-intrepid-amd64-debs WRITECAP')
* thanks to tahoe's simple capability-based access control, Shawn (and
his buildslave) get the ability to publish intrepid-amd64 debs but not
the ability to tamper with anything else. Magic!
* configure the build process to use 'tahoe put *.deb
tahoe-intrepid-amd64-debs:' as its upload step, then invoke some
as-yet-undefined mechanism to notify the "APT repo manager" to update the
index
* on the server side:
* build an APT repo manager: a daemon that sits around waiting to uploaders
to tell it that a new file has been added to the Tahoe directory
structure. When that happens:
* download the structure to local disk. This could just use "tahoe cp -r
repo: ./repo", but we'd prefer it to have a sort of inverse backupdb:
something that lets us refrain from doing a download when the local file
looks sufficiently up-to-date.
* run the APT index-building scripts to create Packages.gz, Contents, etc
* mirror everything back into tahoe: 'tahoe cp -r ./repo repo:' . Again,
this would be better if 'tahoe cp' used the backupdb, but this is an
easier feature addition than the downloaddb ('restoredb'?) described
above
* mirror everything over to http://allmydata.org/debian
* publish the tahoe readcap for that 'repo:' directory
With that in place, there will be two ways to make your debian box have
bleeding-edge tahoe packages: either point APT at allmydata.org, or point APT
at a tahoe webapi server and the published readcap. The latter is slightly
more self-hosting and cooler, but if the testgrid isn't being reliable, you
might prefer to use a normal HTTP site instead.
The "APT repo manager" notification step might be a tiny Foolscap client..
there's a demo of something similar in the foolscap docs/ directory already.
This week is looking kind of busy, I'm not going to have much time to work on
this project, but maybe next week. I'll let you know when/if I build any of
the pieces listed above. Note that we could set up the buildslave to upload
.debs to a directory independently of me getting the time to change the APT
repo manager scheme around (the debs just wouldn't be APT-gettable for a
while, people could still fetch them manually from the published readcap).
cheers,
-Brian
_______________________________________________
tahoe-dev mailing list
tahoe-dev@allmydata.org
http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
