Re: New mirror, ftp.acc.umu.se
On 31 Jul 2000, Philip Hands wrote: Mattias Wadenstein [EMAIL PROTECTED] writes: We would also be happy to run a mirror for the potato ISOs provided there is a someone we could rsync towards, since we have much bandwidth and not that much CPU power we could probably rsync them a couple of times over before the pseudo-image-kit would finish doing its stuff. ;) Er, no -- the pseudo-image-kit doesn't take much CPU -- it's just sticking FTP/HTTP files together. The server has 2 40MHz supersparcs. I've tried the pseudo-image-kit on another of our servers, with 2 60MHz supersparcs, and it took much more time than downloading the isos would have taken. (This was slink, but it took me over an hour making those isos, downloading them would probably have taken less than 20 minutes.) In fact, you presumably have a local mirror of the debian archive, if you're hosting ftp.se.debian.org, in which case the pseudo-image-kit would be working at the bandwidth of your disks, which I imagine is rather higher than the bandwidth you have to your nearest cd mirror. Well, that depends on the other end. :) I don't think I'll find any other ends that are that fast though. We do have more potential bandwidth to network than disk.. So, be a good chap and use the pseudo-image-kit. Once you've done that, if you cannot find a closer mirror to do the final rsync from, feel free to point it at cdimage.debian.org Well, the one at tut.fi seems pretty close. A few tests didn't get that great bandwidth to it though, only 250 KB/s or so. But that is probably going to be faster than setting up the pseudo-image-kit anyway. We'll see. Perhaps we have a mirror up and running tomorrow. If that is so, we'll let you know. :) /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: New mirror, ftp.acc.umu.se
On Tue, 1 Aug 2000, J.A. Bezemer wrote: On Mon, 31 Jul 2000, Mattias Wadenstein wrote: On 31 Jul 2000, Philip Hands wrote: Mattias Wadenstein [EMAIL PROTECTED] writes: We would also be happy to run a mirror for the potato ISOs provided there is a someone we could rsync towards, since we have much bandwidth and not that much CPU power we could probably rsync them a couple of times over before the pseudo-image-kit would finish doing its stuff. ;) Er, no -- the pseudo-image-kit doesn't take much CPU -- it's just sticking FTP/HTTP files together. The server has 2 40MHz supersparcs. I've tried the pseudo-image-kit on another of our servers, with 2 60MHz supersparcs, and it took much more time than downloading the isos would have taken. Sparcs have a known issue with the `wc' command, see http://cdimage.debian.org/~costar/pseudo-image-kit/ (at the bottom). ftp-deb@churchill:~ wc --version wc (GNU textutils) 2.0 I hope it was a problem with the solaric wc and not the sol/sparc version of GNU wc. It takes no noticable time to 'wc -c' an cd image, so I'm guessing that is the case. Also, if (many) other processes are running, things may slow down quite much. This is because make-pseudo-image is a shell script which calls many external programs, and each call gives priority to other running programs (even if the called program would terminate immediately). Well, the machine behaves pretty good during load, and it doesn't have much else to do really. It usually has a processor or two idle waiting for something to do. It is just a matter of the processors and disks being slow. The rsync part takes a bunch of CPU power, probably the time for calculating a bunch of checksums. And that rsync takes aobut 10 minutes too. Loadgraph: http://www.acc.umu.se/technical/statistics/loadavg/view.html.en?name=churchill I started the multigrab at 14 CET, which should be noticable. :) Time: 12460.08u 24367.33s 11:49:09.45 -14.-3% It did take some time. Probably 3/5th make-pseudo-image and 2/5th rsync, or something like that. We do have more potential bandwidth to network than disk.. Hmm, in that case setting up a simple FTP/HTTP/rsync redir to e.g. ftp.sunet.se would be faster than using your own disks for the mirroring, wouldn't it? ;-) Well, a couple of megs per second is no problem. And if you spread the accesses out to many different disks and different controllers, you can get a pretty good disk bandwidth. :) But the pseudo-imake-kit reads from just one disk at a time and writes to one other disk. Also, ftp.sunet.se doesn't have the potato isos, otherwise I would just have rsync:ed them from there, just as we do with the official ones. I also made some modifications to the multigrab script (so that you can run "multigrab */*.list"), and some other modifications here and there (I don't remember if all was changing the shell to bash instead of /bin/sh which is really slow on solaris, or if I made more modifications :/ ). Oh, well. The modified pseudo-image-kit is in the mirror/potato_test-cycle-3/pseudo-image-kit-2.0 directory, the actual images are in the mirror/potato_test-cycle-3 dir. That is: http://ftp.se.debian.org/mirror/potato_test-cycle-3/ ftp://ftp.se.debian.org/mirror/potato_test-cycle-3/ ftp.se.debian.org::potato_test-cycle-3 /Mattias Wadenstein - writing way too long emails and staying up too long. :) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: 2.2 CD images appearing
On 14 Aug 2000, Philip Hands wrote: If you are running a mirror, then please use the pseudo-image-kit if at all possible, and if it's not possible then use a mirror that has more bandwidth (like sunsite.org.uk for example, who already have the binary-i386-1 and 1_NONUS images available) We should have both of those over here at ftp.se.debian.org when NONUS syncs in about 1 minute or so. If people could mention when they've got copies of the images, so that others can start mirroring from them, it might help spread the load more (cdimage.d.o is only a P166, so cannot stand too many rsyncs). I'm running an rsync for the source dir. Unfortunately I did some mv magic instead of waiting syncing with hard links, so if you want to see if they have synced you have to check the date or size. If things get out of hand, I may switch the password checks back on until the excitement dies down, so if that happens, and you're running a top level mirror, and you've not got a password, mail me. I have no password, if ftp.se.debian.org is considerd a top level mirror you can reply in private. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: rsync - binary-i386-3.iso
On 16 Aug 2000, BorX wrote: Hi. I used pseudo-image method on ftp.fr.debian.org to d/l all files : 4 pseudo-images (3 "normals" + non-us). I was about to rsync them to official CD isos, but the files binary-i386-3.iso and .list disappeared... I was working on sunsite.org.uk, so I jumped to another rsync server : I still haven't found this binary-i386-3.iso. I'm sure it exists as I got the .list one time, and it is mentioned in MDSUMS so... You understood it, I talk about brand-new 2.2. Please send me an rsync adress in which I'm sure to get this 3rd image : I'm impatient to burn it :]... We at ftp.se.debian.org have all i386 and source images. As do trumpetti.atm.tut.fi. Our machine (ftp.se.debian.org) is rather slow, so we only allow 8 rsyncs, but there shouldn't be that much of a problem of getting in now. I'm currently working on the other architectures, trumpetti.atm.tut.fi has most but not the NONUS images among the other architectures. And neither has any mirror I have found in the rsync-mirror-listing. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: rsync - binary-i386-3.iso
On 16 Aug 2000, Philip Hands wrote: Actually, sunsite.org.uk never got binary-i386-3.iso, because they've run out of disk space. (No, I'm not joking) BTW please make sure to check the md5sums of any images before making them public on mirrors --- I have a feeling open.hands.com is occasionally introducing errors as it reads its disks. If you have to re-rsync an almost right image, using a block size of 65536 seems to be reasonably quick. Is there no way to give trumpetti.atm.tut.fi and perhaps a couple of others exclusive access? Trumpetti has almost a complete set, and if a couple of mirrors could get all of the images most of the other mirrors can follow. Trumpetti is also pretty fast. Which is good. :) I have been through all mirrors listed on the rsync-mirrors page several times, and I can't find the images anywhere accessable. /Mattias Wadenstein - a bit frustrated at not being able to get the images -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
New fast, soon complete mirror.
The mirror: rsync-debcd.acc.umu.se It will probably have all images within an hour. (Currently lacking a arm, alpha and powerpc NONUS-images.) It is fast in both cpu and bandwidth. It will only be fast until 26/8, after that it will be the regular slow ftp.se.debian.org. There might be reason to mention it in cdimage.debian.org's motd after we have managed to get all the images. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: New fast, soon complete mirror.
On Thu, 17 Aug 2000, Mattias Wadenstein wrote: The mirror: rsync-debcd.acc.umu.se It will probably have all images within an hour. (Currently lacking a arm, alpha and powerpc NONUS-images.) We now have all the images. The md5sums have been verified. We only allow rsync access, but that should not be a problem. Especially those within nordunet should seriously consider syncing from us now. There seems to be some problem with the connection between cdimage.debian.org and nordunet, some of the NONUS isos we downloaded from ftp.gigabell.net and some took an extra roundtrip via another system with other connectivity to the world. /Mattias Wadenstein - finally with a complete mirror -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: blind student
On Fri, 27 Oct 2000, Saqib Shaikh wrote: Hello, I am a blind student in the UK, and I am finding the pseudo image kit hard to use wtih my speech software. I have downloaded the iso images for binary1, binary2 source1 and source2 from sunsite.org.uk, but they do not have binary-i386-3.iso or source-3.iso. from where can i get these cds for 2.2rev0? Our mirror ftp.se.debian.org has binary-i386-3.iso and source-3.iso. It should be resonably fast from UK academic networks. You can find it at: http://ftp.se.debian.org/debian-iso/ ftp://ftp.se.debian.org/debian-iso/ /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: out of date info on http://cdimage.debian.org/ch32.html
On Thu, 14 Dec 2000, J.A. Bezemer wrote: On Tue, 12 Dec 2000, Othmar Pasteka wrote: btw. is the web content also mirror able? so that http://cdimage.at.debian.org/ also has the webcontent? (other than redirecting the url). wget --mirror (And since it's 200kB you can also just wget all of it again each night ;-) (That of course also means that not much bandwidth is saved by mirroring, which seems to make it quite pointless.) Removing a single point of failure? I know this was a problem when potato was released, cdimage.debian.org was hard to get webpages from. Btw, our mirror is up-to-date again. We had some problems with rsync, it seemed to want to mirror two sets of images, then making hardlinks. And we didn't have enough space for that. After some manual syncing and linking it seems to work just fine. (We are syncing from trumpetti.atm.tut.fi.) /Mattias Wadenstein - admin of ftp.se.debian.org -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: New CD image creation tool
On Tue, 19 Dec 2000, J.A. Bezemer wrote: On Sun, 17 Dec 2000, Richard Atterer wrote: On Sun, Dec 17, 2000 at 01:14:06AM +0100, J.A. Bezemer wrote: - Maybe 'diff' is the wrong name here, but I can't thing of a better one at the moment. I don't like it, either - hmm... maybe "image template" sounds better? While we're at it, I'll call the human-readable file "location list" from now on, until someone comes up with something better. ;) How about the concept of "cooking" a CD image? We have "special ingredients" (i.e. the literal binary data), a "recipe" (image template) and a "grocery list" of places (i.e. FTP/HTTP sites) where to get the "standard ingredients". Should be understandable even to people who didn't ever touch a computer before ;-) I like the naming, it makes sense. It can even be extended into the domain of downloading pre-cooked CD images. [Advantages] - By querying servers before it starts to download, the tool can determine whether all files are actually available. You mean, downloading an ls-lR? Or querying for each individual file? (The latter wouldn't be advantageous since _if_ they exist we'll be downloading them later anyway.) I was thinking of individual queries, although directory scans are probably better. Why would an initial check not be an advantage? It allows you to abort straight away, instead of downloading, say, a hundred packages before encountering one which doesn't exist. I agree that checking would be an advantage, but it should not cost too much. ls-lR.gz is quite big (1.3M on http://ftp.us.debian.org/debian/) (and some mirrors have US and non-US combined in one ls-lR, others don't); scanning directories doesn't work with HTTP servers and with FTP (using package pools) it will transfer about the same size as the UNzipped ls-lR (~10M). Checking every single file is only easy with HTTP and will still cost ~500 bytes(?) * 2000 files/CD = ~1MB per CD. Latency would be a big issue, especially for the future. As bandwidth increases, we'll still have latency. For me, with a good network connection, first checking and then downloading would probably take much more time (my guess is 30% or so). It will also put greater load on the mirrors, checking if a file exists is almost as hard as delivering the file. But then, perhaps our mirror is kind of special in being CPU-bound, not bandwidth-bound. :) Furthermore, when using pools checking is not a really big issue since everything should (at least _can_) still be available. Don't concentrate on it now, you can very well add this "new feature!" later (and I guess making it optional and "off-by-default" would be best for most users). That makes sense, it is a feature that will be useful for some users, but it is harmful for other users. For the rest, the recipie approach makes sense. Now all that is left is finishing a good design and implementation. ;) /Mattias Wadenstein (admin of ftp.se.debian.org) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Woody is online again!
On Sun, 8 Oct 2000, Attila Nagy wrote: Hello, After a short pause in the production of the fsn.hu Woody unofficial CDs I restarted the process this weekend. And now we have started to mirror them, the mirror should be up and running tomorrow. ftp://ftp.se.debian.org/pub/cd-images/debian-unofficial/ http://ftp.se.debian.org/pub/cd-images/debian-unofficial/ rsync: ftp.se.debian.org::pub/cd-images/debian-unofficial/ The set will be upgraded every Sunday at around 16-17h CET (European meantime), so it is advised to not download from the archive on weekends, because you will have inconsistent files. We will rsync them early on monday then. If you don't have any objections to a mirror that is. :) Btw ftp.fsn.hu is alot slower these days, only around 30-50K/s unlike the 300K/s of a year ago or so. Perhaps a mirror for this will put a bit less load on it? If somebody wants to test these images he/she can use rsync to upgrade the local copies every week, so it won't eat up all his/her bandwidth. The bandwidth problem is on your side. ;) Btw, regarding the rsync-mirrors.html: the rsync-debcd.acc.umu.se isn't faster anymore, but it won't go away. (Same machine as ftp.se.debian.org now.) But we might use that if we would ever put up a front-end machine for rsync again. /Mattias Wadenstein - admin of ftp.se.debian.org (aka ftp.acc.umu.se) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Woody is online again!
On Mon, 15 Jan 2001, Attila Nagy wrote: We will rsync them early on monday then. If you don't have any objections to a mirror that is. :) You can rsync every morning :) Hehe, we'll add that then. Btw ftp.fsn.hu is alot slower these days, only around 30-50K/s unlike the 300K/s of a year ago or so. Perhaps a mirror for this will put a bit less load on it? ftp.fsn.hu is on the hungarian academic network (HBONE, operated by IIF) which has good connections to the outside world (STM-1 to Europe and a smaller connection to the US). Due to the enormous traffic (nearly 10 terabytes per month) they have set the switch port to 10 Mbps (quite good decision! :( so the site is very slow at this time. Well, we are on the swedish university network (SUNET), which also runs ftp.sunet.se. That one has managed close to 7TB for one week ( http://ftp.sunet.se/statistik/ ). I don't think there would be any problems even if we managed to fill that fast ethernet interface up (which our current server can't do, it is too slow CPU-wise). `woody-i386-3.raw' at 62695764 (9%) 17.3K/s eta:8h43m [Receiving data] This also means it will take some more time than expected to get the mirror up and running. My new guess is tomorrow. :) I am currently before a move to a commercial ISP who has donated an unlimited connection to the internet with a FastEthernet connection to their switch and maybe a GigaEthernet in the future (that would be very good, although I have no funds to buy a GE card, and operating the server is also from my salary :). So it seems the site will be as fast as it was in few weeks. Hope it works out for you. Until then you could try and redirect some traffic to us for the debian part anyway. :) Sorry for the above, in Hungary it seems a public, free software FTP site is not a good business :) Is it ever? :) It is a nice service to the world though. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: I'm interested!... where are they? :)
On Tue, 30 Jan 2001, jason andrade wrote: On Mon, 29 Jan 2001, Mark wrote: "If you are interested, there are some unofficial CD images of "woody" (the current Debian development version, to be released as 2.3 or maybe 3.0 in due time) for linux-i386 binary and of "sid" (the even more unstable development version) for hurd-i386 binary (new/2/3). Also available there is an unofficial "potato" linux-i386 "extra" CD with security updates, non-free, KDE, KDE2, Helix Gnome and some other programs. " I'm very interested in finding these files, where are they to be found? these are primarily available from hungary hosted by a guy called attila nagy (who built all of them except the hurd iso i believe). ftp://ftp.fsn.hu/ you can also find a mirror in sweden at sunet (sorry mattias, i can't remeber the site exactly :-) http://ftp.se.debian.org/pub/cd-images/debian-unofficial/ ftp://ftp.se.debian.org/pub/cd-images/debian-unofficial/ rsync ftp.se.debian.org::pub/cd-images/debian-unofficial/ It is a full mirror of ftp.fsn.hu's directory with unofficial cd-images updated daily. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Unofficial Debian SID CDs
On Fri, 9 Mar 2001, Attila Nagy wrote: Hello, I started to produce Debian SID install CDs on ftp.fsn.hu and also updated the woody ones. BTW, due to disk problems I have to stop updating the potato extra CDs which hold non-free and other user requested software (but they remain available). Ah, that is why yesterday's cron job said someting about "opendir(sid): Permission denied" :) The current sid set stands of 6 CDs. The last one is the non-free CD, so it is necessary only if you want to install "non-free" software. This is the case with the woody sets too, so the last CD contains Debian non-free packages only. ps: these files will be (hopefully) appear soon on the mirrors below: ftp://ftp.planetmirror.com/pub/debian-cd/unoffical [Australia] ftp://omega.elte.hu/mirror/debian-unofficial [Hungary] ftp://ftp.acc.umu.se/pub/cd-images/debian-unofficial [Sweden] A couple of things regarding that: I think we would have greater availability if you would do the kind of hardlink-trick that the official debian setup uses. That way the rsync won't start by deleting the woody isos and then downloading them in full to another name. Since we seem to get about 70K/s it will take most of today to sync everything. We will also run out of space after the sid images have managed to sync. (We only had about 4 gig left or so.) We are contacting our disk sponsor about this, and hopefully we'll get another nice disk. We might manage to find some temporary storage (borrow a couple of disks), I'm looking into that right now. Hopefully it will be solved today. This would mean some small downtime of course, but it shouldn't be more than 15 minutes. Ah, the potato images was moved and the symlink doesn't point to a place that exists over here, would it be safe in the future to copy symlinks as the files/directories they point towards? Since we are going to run out of space soon, I don't think that a full mirror of ftp.fsn.hu would be a good idea. ;) I hope I don't come across as complaining or anything, it is great that these cd-images exists. I'm just trying to make them more available. :) /Mattias Wadenstein - ftp.se.debian.org -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Unofficial Debian SID CDs
On Sat, 10 Mar 2001, Attila Nagy wrote: I think we would have greater availability if you would do the kind of hardlink-trick that the official debian setup uses. That way the rsync won't start by deleting the woody isos and then downloading them in full to another name. Since we seem to get about 70K/s it will take most of today to sync everything. I don't really understand you. There was a change in the directory structure, but this won't happen again. If there will be another update all the filenames will be the same. Ah, ok. Of they won't change again there is no need. We will also run out of space after the sid images have managed to sync. (We only had about 4 gig left or so.) We are contacting our disk sponsor about this, and hopefully we'll get another nice disk. Wow, you have a disk sponsor! I would need one too ;) The very nice people at southpole (.se) that used to run ftp.se.debian.org are still involved in providing this service to the public, even if they aren't running it directly any more. I wonder a bit if some of the bigger linux supporters (IBM springs to mind) can't be convinced to donate some disk to these important services. You might want to try and contact them. The hardest part would probably to get in touch with the right person(s). Ah, the potato images was moved and the symlink doesn't point to a place that exists over here, would it be safe in the future to copy symlinks as the files/directories they point towards? The potato images won't change in the future. If you need them download with ftp and exclude them from the rsync process. Ok, we will do that when we have diskspace for it, I think. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Unofficial Debian SID CDs
On Sat, 10 Mar 2001, Attila Nagy wrote: Hello, I wonder a bit if some of the bigger linux supporters (IBM springs to mind) can't be convinced to donate some disk to these important services. These companies donate stuff only for big sites, not for small ones like mine (ours?)... Most of them don't, most of the time. Which is kind of silly, setting up a disk pool or something, that nice sites like yours (ours?) could ask for storage from. It would be a fairly low cost operation, and enable more mirrors/sites to stay around and have more content. But anyway, keep on asking. Just because you are most likely to get a no, doesn't mean you will always get a no. But if you never ask, you will never get anything. But a smaller, local sponsor would probably be easier to convince, if you can find them. /Mattias Wadenstein - with some general rants -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Debian woody and sid unofficial CDs
On Fri, 13 Apr 2001, jason andrade wrote: On Fri, 13 Apr 2001, Attila Nagy wrote: Hello, FSN.hu's Debian woody and sid unofficial CDs have been updated. planetmirror.com is updating the woody and sid mirrors. And our nightly cronjob has started updating, it is currently at sid-i386-4.raw and is currently going about 20K/s. /Mattias Wadenstein - going back to work at the nice new hardware that is going to be a much faster ftp server soon. :) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: New SID and WOODY
On Wed, 18 Apr 2001, Attila Nagy wrote: Hello, I've made an error last week so here are the new SID and WOODY CD images. ftp://ftp.fsn.hu/pub/CDROM-Images/debian-unofficial rsync is disabled currently, because Redhat fans are taking all my resources. FTP should work, although I don't know how a PIII-450 will handle more than 1500 concurrent FTP users (I've messed up my ftpd's config and I don't want to kill them)... The mirrors (at least ftp.se.debian.org and ftp.planetmirror.com) will catch up soon, I think they have a stronger machine ;) ftp-deb@churchill:/export/ftp/mirror/ftp.fsn.hu/pub/CDROM-Images/debian-unoffial/sid rsync --progress -av --block-size=8192 193.224.40.96::ftp/cds/sid\* . receiving file list ... done sid-i386-1.raw 23774034 (3%) Well, it will be a while, since we don't have all that much hardware yet. 2x40MHz supersparc with 256 megs of ram. :) ftpup 34 days, 20:10,load average: 4.67, 5.32, 8.67 I'll update woody after sid is updated. Check the dates to see if they are updated yet. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: New SID and WOODY
On Wed, 18 Apr 2001, Attila Nagy wrote: Hello, I've made an error last week so here are the new SID and WOODY CD images. [snip] The mirrors (at least ftp.se.debian.org and ftp.planetmirror.com) will catch up soon, I think they have a stronger machine ;) We now have the woody and sid images. http://ftp.se.debian.org/mirror/ftp.fsn.hu/pub/CDROM-Images/debian-unofficial/ We also have rsync access, but that might be a bit crowded with people trying to sync their debian mirrors. /Mattias Wadenstein - hoping that installing the new hardware won't take too much time -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: official 2.2_rev3 CDs available
On 30 Apr 2001, Philip Hands wrote: Attila Nagy [EMAIL PROTECTED] writes: Hello, everyone --- it's now down to 15Mbit/s, so maybe you'll have more luck. If not, I'll switch the passwords on for the weekend. BTW, wouldn't be wiser to figure out a layout which is compatible with the rsync way of mirroring? This would help mirrors to catch up automatically without the need for moving the old rev2 isos to rev3 and sync them. It already is done in an rsync friendly way. The file names in the potato_test directory tree do not change between point revisions, so if you also mirror that part of the hierarchy, and use the --hard-links switch to ensure that the hard links are maintained, then the images will be downloaded over the top of the potato_test files, and then the hard links will be installed into the new versioned directory. Well, just an rsync over the entire debian-cd directory doesn't work that way for me, unfortunately. Because it handles the 2.x dir first, before it comes to the potato_test dir. And if you only rsync the potato_test dir (and make links manually or scripted later) you need addtional storage, because then you will end up with the new set in potato_test and the old set in 2.x. This might work fine for some people, but unfortunately we do not have that much storage free (we use that to mirror stuff instead). I wonder if rsync could be convinced to operate with checksums and so for an entire fileset.. That would mean considerably more work for the server though, so the caching checksums feature is probably something that must be implemented first. /Mattias Wadenstein - random ramblings on the issue of rsync -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: official 2.2_rev3 CDs available
On 30 Apr 2001, Philip Hands wrote: Mattias Wadenstein [EMAIL PROTECTED] writes: Well, just an rsync over the entire debian-cd directory doesn't work that way for me, unfortunately. Because it handles the 2.x dir first, before it comes to the potato_test dir. Is this starting from a point of not having the old images in the potato_test dir? If so, I can see why you'd get what you describe, otherwise I'd expect it to work properly. I have the hardlinks and potato_test in place. When I have everything up I do rsyncs against full mirrors fine and nothing gets downloaded and so. You can do a rsync -rvn ftp.se.debian.org::debian-iso to see the file structure. And those are hardlinks, we do not have the space for two sets of debian cd-images. I expected it to work properly the first time too, but then I noticed that I ran out of disk somewhere around i386. :) And if you only rsync the potato_test dir (and make links manually or scripted later) you need addtional storage, because then you will end up with the new set in potato_test and the old set in 2.x. This might work fine for some people, but unfortunately we do not have that much storage free (we use that to mirror stuff instead). I think you just need to put the links into potato_test in first, and all will work as you wish --- you cannot expect rsync to guess that the files in a directory next door, are a bit like the ones you're trying to download. Yeah, it can't guess that. But 2.2_rev3 is listed before potato_test, thus it tries to sync 2.2_rev3 first, then recognizes that those are the same as in potato_test. I don't see how I could put potato_test first. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: woody ISOs for non-i386?
On Tue, 24 Jul 2001, jason andrade wrote: On Tue, 24 Jul 2001, Anthony Towns wrote: [0] i386 isos are at ftp://ftp.fsn.hu/pub/CDROM-Images/debian-unofficial/woody/ apparently. they're also in AU at http://planetmirror.com/pub/debian-cd/unofficial/woody/ however since the unofficial ISO's for i386 woody take around 3G, it'll need to be someone with between 15-20G of disk available to generate this is my guess. unfortunately i don't have the space required to host something like this in AU :-/ And with the growth of the debian archive they aren't on our mirror[1] anymore, but that is a temporary thing, we hope to get them back when we finally manage to move our server to the new hardware we have been working on for several months now (not the hardware, but the software involved). [1] ftp.acc.umu.se aka ftp.se.debian.org and ftp.gnome.org /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Want to mirror Debian CDs
On Fri, 9 Nov 2001, jason andrade wrote: On Thu, 8 Nov 2001, David Gray wrote: The IBM LTC at Beaverton, Oregon, USA uses Debian quite a bit and have asked that I include the Debian ISO images along with the other distributions I currently mirror. The machine I would use for this is a UNIX variant (Sequent (now IBM) DYNIX/ptx). After reading the debcdmirror script and what it takes for maintenance, I think the route I want to take is just to rsync the iso images straight across - I'm not sure I want to build ISO images on my end and am unclear why I would want to unless the distribution is not fixed, but changes. I thought changes of this nature would go in an update directory. The debcdmirror script has some issues, the main point of it and the pseudo-image-kit that it uses is to limit the bandwidth usage on the cd image mirror sites (taking most of the data from a regular debian mirror). Several of us mirror admins have raised opnions that it isn't really needed except to save bandwidth at the master site at release-times. Could you please tell me how to use rsync to grab the iso's, and what is needed to maintain the mirrors when the distribution changes? Hopefully cron is all I would need. One problem with rsync and cron is that the directory name changes when there is a new revision or release. Just running an rsync over the debian-cd directory in cron would mean that when there is a release, rsync would start out by deleting the existing images and then start downloading the others. If you have enough storage to maintain two copies of the cd images, there is a way around this for new revisions using the potato_test directory that is present on the master site. It contains hard links to the images and has names that doesn't change between revisions. Then you can use the fact that between revisions alot of the cd image does not change, rsync takes care of that. Also there will not be the day or two of interrupted service when you have no images and are downloading the 15 gigs from a mirror. A good script would then: 1. rsync the potato_test directory 2. rsync the entire cd-image directory with --hard-links Of course, if you don't mind being a couple of days late with new images, a plain rsync of the debian-cd (or debian-iso in my case) directory would work fine. Just remember to include --hard-links so that you won't get two copies of the images, if there is both a potato_test directory and the 2.2rev4 one. And a --delete-after would probably be a good thing to include too, so the rsync won't start out with deleting everything. The just a plain rsync has the added feature of also working with future releases, where one can expect a change in name from potato_test at least. however, if you just want to mirror all the images (for all architectures?) then if you wait for a few days, there will be a number of sites you can rsync or ftp them from. please keep in mind that there are 6 architectures and source, so you need about 12-15G of disk space for a full debian-cd 2.2_rev4 mirror. rsync has no particular advantage for you to use over ftp if you aren't building images over previous images (some people have reported quite good results with that) and it's probably best to leave rsync slots open to mirrors/users doing this, if you are fetching fresh images. Well, if one keeps a mirror for a long time, one usually have previous images to rsync over. But it is additional work to set that up and get it to work. Rsync is also pretty good at mirroring a directory structure. some of the usual sites that carry debian iso image archives planetmirror.com/pub/debian-cd/2.2_rev4/ (our site. free plug :-) ftp.fsn.hu ftp.sunet.se Mattias Wadenstein [EMAIL PROTECTED] who i think runs ftp.acc.umu.se Yes, I'm almost done with getting the images. I'm running a final rsync to get all the permissions, links and so right, I should have all the images correctly already. cdimage.debian.org (master site - busiest, and preferably use a mirror before hitting it) And to save bandwidth at the master site, please don't hit it with just a plain rsync. Use the pseudo-image-kit if you need to use that one. Also there have been no mirror listed in the US here (planetmirror.com is in Australia and cdimage.debian.org is in the UK), perhaps someone in that part of the world can contribute the names of a couple of working mirrors? On the other hand, we don't really have problems with transatlantic capacity anymore. So if you feel you can trust our IBM hardware and software ( http://ftp.acc.umu.se/mirror/ftp-about.html ), feel free to mirror from us. :) /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Want to mirror Debian CDs
On Fri, 9 Nov 2001, Mattias Wadenstein wrote: Also there have been no mirror listed in the US here (planetmirror.com is in Australia and cdimage.debian.org is in the UK), perhaps someone in that part of the world can contribute the names of a couple of working mirrors? According to sources http://ftp-mirror.internap.com/pub/debian-cd/ should be fairly close to you, but I don't really know what access methods and so they have. (I can't connec to them with rsync right now, but http works fine). If you want to update quickly, it would probably be a good idea to check around to see what speeds you get from different servers. If you don't really care, just pick one that seems to be committed to stay updated and is resonably close. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Redesign of cdimage website
On Thu, 15 Nov 2001, Richard Atterer wrote: I've finally pulled myself together and started on a redesigned website - different graphics and site structure, but mostly the same content. I've uploaded the first results to http://cdimage.debian.org/~atterer/. Very nice. You'll notice that most of the links don't work yet - a lot still remains to be done. How do people like it? Does it stand any chance of eventually replacing http://cdimage.debian.org/? It looks much better than it already. One thing: Reorder the list so that Install directly from the network. is the first point, since it is the one we should point most users towards? Otherwise it looks clear and usable. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Redesign of cdimage website
On Fri, 16 Nov 2001, Richard Atterer wrote: On Fri, Nov 16, 2001 at 08:24:06AM +0100, Mattias Wadenstein wrote: One thing: Reorder the list so that Install directly from the network. is the first point, since it is the one we should point most users towards? I've already been asked to do this by someone else. But I'm not sure whether it should really be the first option: Assuming that people who want to use Debian are not dummies ;-), if they go to a site called cdimage, I expect they'll want to get CD images, and probably have reasons for not wanting to install via the network. They might have, and then the will continue to scan the list of options. Thing is that you stop when you see something that matches what you want. If you know you don't want a network install for some reason, you move on quickly. If you are a newbie that is unsure, they might very well not read the part about network installs being better, since they might already have followed a link (to http downloads or something). But it is a minor point anyway, the important thing is that the webpage is much easier to find stuff on. Myself, I'd want to download the images even if I had a permanent net connection, because I might want to install Debian on a friend's machine at one point, or use it for rescue booting when my system is broken, etc. etc... Same here. I download (over nfs) and burn the image just to have a boot media for a network install. It is easier and less trouble than making floppies. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian ISO (Was: Re: [PLUG] RedHat 7.1 iso?)
On Fri, 12 Oct 2001, Wookey wrote: On Wed 10 Oct, Ben Collins wrote: I've brought this up before. I went to go get an ISO, and was presented with 50 questions about what I intended to do with it, and what type of person I was. It's rediculous, regardless of the issue it is trying to solve. I sympathise, but any suggestion of change brings us back to the original question - do we have the bandwidth to just serve people iso images? Speaking as a cd image mirror admin: Yes. Checking the list of mirrors europe seems to have enough mirrors to provide easy http/ftp downloads. I don't know enough about the bandwidth situation in US or Australia though. But I don't think it would be so bad, at worst the user could discover that downloading cd-images is really, really slow and chose another method. If we do then, yes lets simplify the whole damn thing. Yeah, I think I'm not the only one that thinks booting from a cd is easier than booting from several floppies for a network install. Add some other factors (good bandwidth, easy access to cd writer and blank cds, dislike for floppies and so on) and getting a cd from a mirror is the easiest solution. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian ISO (Was: Re: [PLUG] RedHat 7.1 iso?)
On Sat, 13 Oct 2001, Philip Charles wrote: On Fri, 12 Oct 2001, Ted Cabeen wrote: It doesn't save bandwidth on the clients. It saves bandwith on the servers. There are a lot more debian mirrors than there are debian-cd mirrors, and they're a lot better distributed as well. The rsync just blends the downloaded packages into a iso-compatible image. Most of that process is the iso munging, not null blocks. The major problems occure at release time when everyone and their dog are trying to get the images. The PIK enables Debian mirrors to build their own images and check them against cd-image.debian.org, this means faster propogation of the images. When Mandrake is released everything locks up for a few days, Red Hat mirrors build up non-public archives for a few weeks and then make them public when the release is announced. At the moment there is no problem with downloading Debian iso images, but when woody is released ... Well, that is usually solvable quite good with a primary site that only allows other mirrors to connect. After those 10-20 mirrors or so have managed to get the images, there are usually enough bandwidth around for the other mirrors to update resonably fast. Sure, the servers (especially those that updates first) are going to be slow for a day or two, but I'm pretty optimistic given the master server is pretty much unaffected by the release. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Want to mirror Debian CDs
On 16 Nov 2001, Philip Hands wrote: Mattias Wadenstein [EMAIL PROTECTED] writes: A good script would then: 1. rsync the potato_test directory 2. rsync the entire cd-image directory with --hard-links Of course, if you don't mind being a couple of days late with new images, a plain rsync of the debian-cd (or debian-iso in my case) directory would work fine. Just remember to include --hard-links so that you won't get two copies of the images, if there is both a potato_test directory and the 2.2rev4 one. And a --delete-after would probably be a good thing to include too, so the rsync won't start out with deleting everything. This last bit is about right, but I'm not sure why you suggested the 2 stage approach, or why you say that you need enough storage for 2 sets. Well, it is because my experience with rsync is that it starts by downloading the new files (no speedup) and _then_ realize that there are hardlinks around and update those. By doing the 2-step approach, it updates the .raw images first and then can use it to update the hardlinks. I am responding now (rather late) because I wasn't sure this was the case, or I just made a mistake. But assuming that the rev4.1 ppc images aren't totally different from the rev4, I should get some matched data in the rsync. (The images grow about 650K/s which is pretty close to the network traffic, I'll have the --stats output later.) /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Our mirror of ftp.fsn.hu's unofficial cd-images
We are happy to announce that we once again keep an updated and complete mirror of the unofficial cd-images from ftp.fsn.hu, they were a bit pruned and not updated for a while when we were having space issues. The access methods are the same ones that are listed in the MIRRORS file: ftp://ftp.acc.umu.se/pub/cd-images/debian-unofficial/ http://ftp.acc.umu.se/pub/cd-images/debian-unofficial/ rsync://ftp.acc.umu.se/pub/cd-images/debian-unofficial/ I'm sorry for the long delay in getting it updated, we always thought the space problems to be solved shortly, but delays just kept adding up. Now it should take a while until the 40 gigs currently free are eaten up by the ever growing debian/ :) /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: cdimage-unofficial.debian.net now has a mirror
On Mon, 21 Jan 2002, jason andrade wrote: On Mon, 21 Jan 2002, Santiago Garcia Mantinan wrote: Hi! I suppose you all know that through http://cdimage-unofficial.debian.net we are offering unofficial Woody cd images of all the architectures targetted to be released. can you give us an idea of the total disk space used? i am assuming that you are offering cd images of all the woody and sid architectures ? stalin:/export/ftp/mirror/trasno.net du -Hs unofficial-debian-cds/ 39G unofficial-debian-cds stalin:/export/ftp/mirror/trasno.net/unofficial-debian-cds ls woody-alpha/ woody-i386/ woody-mips/ woody-s390/ woody-arm/woody-ia64/ woody-mipsel/ woody-sparc/ woody-hppa/ woody-m68k/ woody-powerpc/ If you want to mirror them, you can just add a daily rsync from us. They will go away when official woody isos are released, we will need the space by then. is there any collaboration between this effort and the fsn.hu one ? I don't know, I mirror both now. :) /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: permission problems for unofficial sid .list files ?
On Mon, 21 Jan 2002, jason andrade wrote: Hi, i'm trying to sync the list files for the unofficial sid images at fsn.hu but i get: send_files failed to open debian-unofficial/sid/sid-i386-6.raw.list: Permission denied send_files failed to open debian-unofficial/sid/sid-i386-7.raw.list: Permission denied send_files failed to open debian-unofficial/sid/sid-i386-8.raw.list: Permission denied has anyone else got these files ? mattias ? I have them, which is kind of odd, since they are mirrored with -rw--- permissions. I'm not sure if this is intentional or not, but I have fixed the permissions on my mirror. The permissions will go back by the time we sync next time, so hopefully ftp.fsn.hu will have fixed this by then. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 2.2_r6 CDs at cdimage.d.o
On 5 Apr 2002, Philip Hands wrote: Hi Folks, Sorry about the delay, I'm afraid the release slipped by without my noticing it. In future, feel free to Cc: me in on the Where are the new CD images message you send here, just to make sure that (as happened in this case) I'd not been desubscribed from the list, and been busy enough not to take too much notice of that fact. Anyway, the new images are up for grabs in the normal place: rsync://cdimage.debian.org/debian-cd/ Mirrored in full on ftp.se.debian.org (aka ftp.acc.umu.se). Most other mirrors I found that have already gotten some of this lacks the potato_test link dir and so on. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 2.2_r6 jigdo files cdimage.d.o
On Sun, 7 Apr 2002, Richard Atterer wrote: On Sat, Apr 06, 2002 at 10:37:27PM +0200, Mattias Wadenstein wrote: Hmm.. This might be a very good idea if someone would write the jigdo-mirror script (or point me towards it if it is already done). It doesn't exist yet, but I'm willing to write it (later this week). Basically, what I'm thinking of is a script that can be run immediately after the rsyncs that update the FTP archive and the jigdo dir with all its contents. jigdo-mirror would walk through the jigdo dir, look for files like $jigdoDir/foo/bar/image.jigdo, and check whether any of those files is newer than the corresponding $imageDir/foo/bar/image.iso. If yes (the jigdo file was updated), it will (re)generate the image, using only the files on the local mirror. It will never try to download any missing files, instead it'll just abort and wait to be re-run after the next archive update. Hm, there'll also have to be an extra run over $imageDir to detect and delete any images whose jigdo files were removed. Is this what you want? If that would work, sure. If you have the time to write it. Otherwise a simple script that just makes all the images given a couple of directories ($jigdoDir, $imageDir and $archiveDir are the ones I can think of right now) would do fine. This should be trivial, but I haven't figured out how the arguments to jigdo-port works yet. (Yes, jigdo-port is the only one that compiles on AIX for us. C++ isn't exactly portable. :/) I'm currently updating the cd-image mirror by hand anyway. The important part is that the result looks exactly like the one on the master site so I can do an inexpensive rsync to get everything right. Hmm.. That brings another problem, the filesystem metadata has to look exactly right for that to work, otherwise rsync will assume that it has change and make the server calculate checksums. That could be solved with --size-only to rsync though, we just have to remember it. For extra fun ftp.se.debian.org == ftp.gnome.org and the gnome desktop 2.0 release is currently scheduled for May 1st. :) LOL, hopefully your server won't go up in smoke... We are working on finding a temporary home for it on borrowed hardware. That is also why I'm pointing out rsync-debcd.acc.umu.se now, the main ftp server might be very busy even with much bigger hardware than we currently have. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 2.2_r6 jigdo files cdimage.d.o
On Mon, 8 Apr 2002, Richard Atterer wrote: On Sun, Apr 07, 2002 at 01:04:56PM +0200, Mattias Wadenstein wrote: (Yes, jigdo-port is the only one that compiles on AIX for us. C++ isn't exactly portable. :/) Oh - but the file format has changed, and the latest jigdo files are no longer readable by jigdo-port! :-( I *really* want to avoid having to update two different programs whenever I make a change, so I have no plans to fix jigdo-port. [Standard rant: C++ *is* portable, and the standard has been out for 3 years - it's about time everybody caught up with it! Maybe gcc is boot-strappable on AIX... But I realize that would be quite a lot of work just to compile jigdo-file. :-( ] 3 years is a very short time. We have a fairly updated system, but 3 years is well below the time a system might be around without any updates other than security upgrades. The version is: VisualAge C++ Professional / C for AIX Compiler, Version 5 And that is a new compiler, not a very old one. I know of a few sites that run much older ones. Some configure output: checking whether the C++ compiler is recent enough... no * Your compiler failed to recognize some advanced C++ * constructs. This means that it is probably too old. * In case compilation fails, try upgrading to a newer * compiler, e.g. GCC 2.95 or later. It then procedes to fail to find db_create in -ldb[lots of versions] despite that being installed. --without-libdb gives a failure later on at: checking whether dgettext works... no * Make sure gettext is installed, or use * --disable-nls to switch off gettext support. configure: error: dgettext() call could not be linked. Also some interesting features turns up, like: checking size of unsigned long... 0 checking size of unsigned long long... 0 With g++ (and a bunch of hacking), it compiles but fails to link. I haven't had time to take a better look at it yet, or turn the software over to someone else around here. It is still painful, especially compared to jigdo-port which compiled cleanly and without problems. The important part is that the result looks exactly like the one on the master site so I can do an inexpensive rsync to get everything right. Should be possible. However, the template files include an md5sum of the images, which is checked by jigdo; if jigdo says that the file has been regenerated OK, it *is* OK and no extra rsync run is necessary. OTOH, it might make sense to fall back to rsync if not all of the parts in an image could be retrieved. I could add that to the script. It isn't for the images, it is for having all the other files and directories looking exactly like the master site, a true mirror. That is why I want the same directory structure, so that it will find the images with the correct size and not bother with them. If rsync believes it has to sync the images, it will put a great deal of load on both sides, even if the data is still the same. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 2.2_r6 jigdo files cdimage.d.o
On Wed, 10 Apr 2002, Richard Atterer wrote: On Wed, Apr 10, 2002 at 01:35:00PM +0200, Mattias Wadenstein wrote: ld: 0711-317 ERROR: Undefined symbol: Base64OutBase64StringOut::code Hmm, /very/ interesting. I'm completely unable to tell whether my code was valid C++. :) Anyway, try the attached patch. Thanks, now it compiles. And now for the next problem: /src/jigdo/jigdo-0.6.4/aix43# gmake install /bin/sh: 0403-057 Syntax error at line 1 : `;' is not expected. gmake: *** [install-po] Error 2 (Yes, that is all of the output. No, /bin/sh isn't bash.) Do you have a good documentation for the change in file format, so that I could perhaps hack jigdo-port into working again? Yes, check doc/TechDetails.txt in the jigdo source tarball, search for obsolete. The change is simple; add support for reading the new entry types 5 and 6. Ok, we'll see. I'm actually quite busy, despite my activity here. :) /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 2.2_r6 jigdo files cdimage.d.o
On Wed, 10 Apr 2002, Mattias Wadenstein wrote: On Wed, 10 Apr 2002, Richard Atterer wrote: On Wed, Apr 10, 2002 at 01:35:00PM +0200, Mattias Wadenstein wrote: ld: 0711-317 ERROR: Undefined symbol: Base64OutBase64StringOut::code Hmm, /very/ interesting. I'm completely unable to tell whether my code was valid C++. :) Anyway, try the attached patch. Thanks, now it compiles. And now for the next problem: /src/jigdo/jigdo-0.6.4/aix43# gmake install /bin/sh: 0403-057 Syntax error at line 1 : `;' is not expected. gmake: *** [install-po] Error 2 (Yes, that is all of the output. No, /bin/sh isn't bash.) And that has the simple explanation of this line in the install-po part: @for file in ; do \ Which I commented out and now even make install works. :) Sorry to bother you with such a simple problem, but it is a bug. Now it runs, and except for a few issues like segfaults when prompted with the wrong options and things like that, I now get: jigdo-file make-image: `woody-i386-1.raw.template' is not a template file For all values of templates and other options I have tried. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: pre-release Jigdo files appearing at cdimage.d.o, please test
On 9 Apr 2002, Philip Hands wrote: Hi folks, I'm just running off some images, which are appearing here: http://cdimage.debian.org/pre-release-jigdo/ i386 is available m68k is appearing as I type I'll check them out once I have a working way of turning them into images. Thoughts about the upcomming release: = Because of the vast amount of space that's going to be needed for a full set of CDs this time round, I cannot see it being possible to carry the actual ISO images in full, and I don't think attempting to mirror them would be a very productive experience for anyone. Well, we carry a full set of woody images today, and I think we will carry a full set of woody images after the release too. Sure, it will take a while to mirror them, but it should be done in a couple of days, depending on bandwidth, server load and so on. Of course, for the major mirrors after the release, this will be way too long and put too much load on the master server. That being the case, it would be good if anyone that expects to want the released images in a hurry after they become available, gives Jigdo a try to make sure that they are actually capable of grabbing images this way. Yes, this is why I'm trying to get a working jigdo around here. Once we do release, and once the mirrors have got hold of the jigdo files, I'll probably generate a selection of images so people that get trouble with jigdo can finish off with rsync. I'll probably do something like CDs 1 1_NONUS for all archs, and maybe all of the i386 and source sets. If people have better ideas, I'm open to suggestions. Sounds fine. Just define the official directory tree and naming and put up the additional files in the correct places, so this is known in advance. That way the jigdo-mirror script (or some additional script called that moves stuff around) can be written in advance and know where it should put the images. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Announcing jigdo-easy 2.1 - please test
On Fri, 12 Apr 2002, J.A. Bezemer wrote: Hi all! I'm pleased to announce the availability of jigdo-easy 2.1 for both Linux/*UX and Windows. Download it at: http://cdimage.debian.org/~costar/jigdo/ (actually, any OS with C compiler) - well, not all C compilers think that -Wall is a correct switch, but that is easily fixed. Is there a good way to set CFLAGS only if it isn't set? Since the official woody images will be available primarily (at least the first days/weeks) in jigdo format, we should make sure that the download tools are working properly. I'm quite confident that jigdo-easy/jigdo-port will do a fine job, but I'd still appreciate it if people (yes, that means you) could give it a try on as many systems as possible (which includes all available variants of M$Win). Please report your experiences to the debian-cd list, or to me privately. Well, it kind of works with jigdo-mirror, it makes alot of progress (on some .jigdo files from the pre-release set mentioned earlier). Then I get: --- Checking file `/export/ftp/mirror/debian-non-US/pool/non-US/main/z/zope-popyda/zope-popyda_2.0.7-1_all.deb'... Pasting... Done. 2002-04-14 16:39:24: 0 parts still missing from image 2002-04-14 16:39:24: Image creation failed --- for various values of Checking file. Which doesn't really tell me why it failed, but I guess something went wrong. This might very well be the fault of jigdo-mirror though, I'll probably have some time to look at it later this week unless someone else figures out why. Just jigdo-easy on a 2.2rev6 image worked fine though. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Announcing jigdo-easy 2.1 - please test
On Sun, 14 Apr 2002, Mattias Wadenstein wrote: Well, it kind of works with jigdo-mirror, it makes alot of progress (on some .jigdo files from the pre-release set mentioned earlier). Then I get: --- Checking file `/export/ftp/mirror/debian-non-US/pool/non-US/main/z/zope-popyda/zope-popyda_2.0.7-1_all.deb'... Pasting... Done. 2002-04-14 16:39:24: 0 parts still missing from image 2002-04-14 16:39:24: Image creation failed --- for various values of Checking file. Which doesn't really tell me why it failed, but I guess something went wrong. This might very well be the fault of jigdo-mirror though, I'll probably have some time to look at it later this week unless someone else figures out why. Ok, it is probably a missing file. I have verified this behaviour with jigdo-easy, which also hides the fact of what file was missing by clearing the screen too soon. This is what I get (thanks to logging in screen): ---8--- CONSTRUCTING IMAGE -- Created `debian-30pre1-i386-binary-1_NONUS.iso.list' Merging parts from `file:' URIs, if any... Checking file `/export/ftp/mirror/debian-non-US/dists/woody/non-US/Contents-i386.gz'... Not needed. ESC[HESC[J Jigsaw Download Easy Copyright (C) 2001-2002 R. Atterer, J.A. Bezemer GPL2 File : debian-30pre1-i386-binary-1_NONUS.iso Label: 'Debian GNU/Linux 3.0 pre1 Woody - Pre-release i386 Binary-1_NONUS CD' Info : 'Generated on Sat, 13 Apr 2002 03:39:47 +0100' PROBLEM --- Oops - not all files could be downloaded... ---8--- And this is pretty much the same error I see when trying to do the same thing with the jigdo-mirror script. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: woody CDs...
On Fri, 26 Apr 2002, Attila Nagy wrote: Hello, That's 650 MB*84= 54600 MB= 53.32 GBs. I will probably manage to have that online. If we drop the 40-45 gigs we currently have in unofficial woody images and bring some more disk online that we recently got our hands on. After woody there will be another testing release. So the debian unofficial saga will continue :) Yes, unfortunately we might not have the space to have that online other than for i386 (most of it are Manty's unofficial woody images for non-1386 architectures). But then, we might get lucky and have some more disk donated to the computer club. (And some faster servers, and a bigger uplink, and ... :) ) Yes, that is a hint. Got any wide scsi or (ibm) SSA disks? :) /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: New jigdo test on cdimage.d.o
On 27 Apr 2002, Philip Hands wrote: Hi, I've mostly finished off my publish_cds script, and the results are available here: http://cdimage.debian.org/jigdo-area/ The structure is: The *.jigdo, *.template and MD5SUMS for each architecture are here: jigdo-area/version/jigdo/arch/ And the final images should go into /debian-cd/version/arch/ ? and this is a hard-link farm built from the TDIR link farms for the architectures, and so should have every file required for the jigdo files: jigdo-area/version/snapshot/ The jigdo files should have the right URLs in for the template, and the snapshot. Well, at lest the snapshot seems to work. BTW does the fact that the template is pointing at cdimage in this file mean that the whole world are going to try to pick up the templates from there, rather than the mirror the grabbed the jigdo file from? Well, for the mirror situation, it (jigdo-mirror) seems to find them locally when you have mirrored it. Mirroring the snapshot is probably harder though. Anyway, have a play and tell me what breaks. The alpha and sparc images are still not right, but I thought getting this out for testing was a higher priority. Trying to build them now with (woody) jigdo-file and jigdo-mirror gives: Found 1322 of the 1356 files required by the template Copied input files to temporary file `image.tmp' - repeat command and supply more files to continue 2002-05-01 04:10:19: 34 parts still missing from image 2002-05-01 04:10:19: Too many files missing in local mirror On more than one image. With a higher $maxMissing (250) it hopefully will build all of them. (I'm running woody for this on the temporary server that we found just in time for the May 1st release that didn't happen, and might be gone by the time it eventually happens.) And some times on this machine: 2002-05-01 04:28:18: Found `/export/ftp/jigdo-test/cdimage.debian.org/jigdo-area/3.0-pre2/jigdo/arm/woody-arm-3.jigdo' 2002-05-01 04:28:24: Found `debian-30p2-arm-binary-3.iso', template `http://cdimage.debian.org/jigdo-area/3.0-pre2/jigdo/arm/woody-arm-3.template' 2002-05-01 04:28:24: Will try to create `/export/ftp/jigdo-test/debian-cd/arm/debian-30p2-arm-binary-3.iso' 2002-05-01 04:28:24: Found template `/export/ftp/jigdo-test/cdimage.debian.org/jigdo-area/3.0-pre2/jigdo/arm/woody-arm-3.template' 2002-05-01 04:29:49: 53 parts still missing from image 2002-05-01 04:30:54: Image checksum is correct, moving image into place 2002-05-01 04:30:54: Found `/export/ftp/jigdo-test/cdimage.debian.org/jigdo-area/3.0-pre2/jigdo/arm/woody-arm-4.jigdo' It will still take some time to generate them all, but then, I could probably break it up per arch and run lots of them in parallell, the cpu is idle most of the time. /Mattias Wadenstein - should be asleep by now -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: New jigdo test on cdimage.d.o
On 2 May 2002, Philip Hands wrote: On Wed, 2002-05-01 at 03:38, Mattias Wadenstein wrote: On 27 Apr 2002, Philip Hands wrote: The structure is: The *.jigdo, *.template and MD5SUMS for each architecture are here: jigdo-area/version/jigdo/arch/ And the final images should go into /debian-cd/version/arch/ ? I was thinking of putting them in a subdirectory: /debian-cd/version/images/arch/ Ok. Mainly because on open, the images will need to be on a different partition from the snapshot (which has to live on /home because of the hard-links to the archive). It would also make it more obvious for people that want to mirror only the images, or perhaps all bar the images. Yeah, that sounds good. Also, to start with at least, the images will only be a partial set on open, because leting people rsync 90-odd CDs off open would be a mistake. Yeah, but if we manage to borrow a nice temporary server for release, we should be able to have all online within 4 hours of getting the jigdo files. Note the if though. AJ's latest suggestion, which seems like a good plan, is to point cdimage.debian.org at raff.d.o instead of open, and have all the jigdo files (which should have no non-US stuff in them, so raff being in the USA os fine) and the main part of the This seems to be a truncated paragraph. Then we could just have the 1_NONUS images on open (as well as all the jigdo files) and call it something like nonus-cds.d.o Ok, because what I need to sync is the jigdo files and templates and then I could probably manage to build them fine (there should be no need for fallback if I build them the same day). BTW does the fact that the template is pointing at cdimage in this file mean that the whole world are going to try to pick up the templates from there, rather than the mirror the grabbed the jigdo file from? Well, for the mirror situation, it (jigdo-mirror) seems to find them locally when you have mirrored it. Mirroring the snapshot is probably harder though. Having raff as the home of the main snapshot seems to be the way to go here. Yeah, that seems resonable. I'd like to say that we could have a snapshot mirror over here, but we have scalability concerns with the number of inodes on our ftp server. Is there a script around to build a snapshot given a release? (If you are a bit late and miss a few files, that is a simple rsync to an existing snapshot location to fix..) Anyway, have a play and tell me what breaks. The alpha and sparc images are still not right, but I thought getting this out for testing was a higher priority. Trying to build them now with (woody) jigdo-file and jigdo-mirror gives: Found 1322 of the 1356 files required by the template Copied input files to temporary file `image.tmp' - repeat command and supply more files to continue 2002-05-01 04:10:19: 34 parts still missing from image 2002-05-01 04:10:19: Too many files missing in local mirror Shouldn't it be getting the missing bits from the snapshot? Yes, the default parameter maxMissing=25 was too small in jigdo-mirror. When I increased it, it worked fine. I managed to build a full set of the pre2 images (except for alpha that is). Richard: Have you made sure that it will never happen that non-US/Contents-i386.gz and main/Contents-i386.gz will be downloaded in the same fetch batch in jigdo-mirror? They have the same name, so wget might overwrite or something. Just tell me that you have thought about it and solved the problem. :) /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: About jigdo test on cdimage.d.o
On 2 May 2002, Heikki Vatiainen wrote: I built three i386 images with jigdo-mirror using jigdo and template files from http://cdimage.debian.org/jigdo-area/ The images were built correctly once maxMissing was raised above 25, just as Mattias noted. It looks like about 80 files were fetched from the snapshot archive for e.g. i386 disk 3. Yeah, the most I saw was about 120-150 for one cd. 250 was big enough for all of them to build without incident. Of course, given time this number would become even larger. This was the reason why I only built three CDs since downloading hundreds of files for just testing seemed like waste of bandwidth when an up-to-date mirror is sitting on the same disk. Phil, are you planning to update the snapshot soon? Yeah. The reason I built all of them was to make sure that it could be done and there was no issues anywhere. Would it be of any interest to make these publicly available? Since the release was postponed and we probably won't have this machine around to when the release actually happens, we never bothered to install rsync/httpd/ftpd and so. There is space enough on the ordinary server if I drop manty's images that haven't updated for a month now. Manty: Any objections? /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: About jigdo test on cdimage.d.o
On Fri, 3 May 2002, Santiago Garcia Mantinan wrote: rsync/httpd/ftpd and so. There is space enough on the ordinary server if I drop manty's images that haven't updated for a month now. Manty: Any objections? Not at all. Well, actually they didn't need to go away. And stuff is up now at http://ftp.acc.umu.se/mirror/debian-cd/ (ftp and rsync at the same place). I thought this would be a good opportunity to migrate to the debian-cd naming instead of debian-iso too, to fit in with everyone else. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Burning a .raw file...
On Fri, 19 Apr 2002, Attila Nagy wrote: Hello, (Um, Attila probably doesn't like the idea of seeing fsn.hu melt down, with 12 mirrors each rsyncing 5GB of data from it... :-/ Does the rsync mirror script use rsync --hard-links? In that case, the complete rsync could be avoided by using hard links.) That machine is already dying. I would need some slot 1 PIII processors to speed that up, but they are very hard to find... Well, in an effort to get a bit less load on it, I'm offering push-triggered mirroring of the directory: ftp.fsn.hu/pub/CDROM-Images/debian-unofficial/ Hmm.. Regarding that, when is the best time for me to run that rsync script? If I have others that mirror from me, it is probably good if I manage to avoid mirroring just when the archive is being updated. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Burning a .raw file...
On Fri, 10 May 2002, Attila Nagy wrote: Hello, That machine is already dying. I would need some slot 1 PIII processors to speed that up, but they are very hard to find... Well, in an effort to get a bit less load on it, I'm offering push-triggered mirroring of the directory: ftp.fsn.hu/pub/CDROM-Images/debian-unofficial/ Wow, that's great! Toffer at ftp.no.d.o that I already trigger for debian[-non-US] asked for it. Since it takes so long to sync the dataset, without trigger you run quite a large risk of trying to update during updates. So please, if you have a mirror of the ftp.fsn.hu unofficial debian images and some familarity with the debian push-sync system, consider sending me an email to set it up. A quick check reveales that: Starting at Fri May 10 14:05:02 MET_DST 2002 [update snipped] Done at Sat May 11 05:14:46 MET_DST 2002 Perhaps I should move it up earlier in the day, that would make it easier for my server (which has heavy load from about midnight to 07). But then, even at the highest load it should be fast enoguh to keep up. We are before a major reconstruction of the site, so I hope in the future we can handle the load better (currently we are capped at 3000 concurrent FTP clients :). Well, either something broke or you are having some kind of scheduled maintenece right now, I can't get to it either. BTW, as turned out we have some problems with our ISP too, because their international pipes went saturated, so that's why we are so slow from outside Hungary. Ok, that could explain the 11kB/s rates I see in my rsyncs. Or that could be explained by the really stupid routing (going via .us). We are working on this, but first we have to legalise this service, so we decided to set up a foundation around that. This way we hope that we can get an agreement from our current ISP (with the service parameters laid down on paper) and we will try to get more hardware (AKA donations, so money will also count) to build a system where we can put several frontends to different ISPs in Hungary. That sounds great! Hope you manage to find stuff and links to build an even better server. The first, we will try have multiple 2.5 Gbps connections to GEANT, the European research network, but that race will be hard to win. :( Yeah, you aren't really a university. But if you could do the multiple frontend thing, you could perhaps put one on the research net? If it only uses that capacity for geant/i2 routes, it shouldn't be much of a problem except that of setting the links and routing up. Also, perhpas you can point to the fact that other projects like this exists on educational links like ftp.sunet.se (avg bw used 200-500Mbit/s) run and paid for by the swedish university network itself. I hope that they can recognize the importance of this work, and they decide to support us. We'll see... Yeah. Hmm.. Regarding that, when is the best time for me to run that rsync script? If I have others that mirror from me, it is probably good if I manage to avoid mirroring just when the archive is being updated. Currently, the images are made daily on a separate machine and synced with the public archive nearly every week, when I think it is time to do that (I occassionally can test the images and if they are bad, I don't upload them, but that depends very much on my spare time). In the future I plan to do this differently, but that needs the current upgrade first. Ok, I was asking because of what time of day I should do the syncs. I put it at the arbitrary time 14:05 (local time here), because we have much more night (mirroring) load than day (end-user) load. Please tell me if you have any preferences. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: heads up for fsn debian-unofficial mirrors
On Thu, 16 May 2002, Attila Nagy wrote: Please issue a for i in *.raw*; do mv $i `echo $i | sed s/raw/iso/g`; done command in those directories (or something like this, which do you prefer and can use under your shell :). And also a sed s/raw/iso/g md5sums md5sums.new mv md5sums.new \ md5sums to have the md5sums files reflect the change. I've already done that. If you don't make these changes, you will transfer 9.3 GBs of the same data, you have already on disk. I read this email just before this: ftp-deb@churchill:~ date Thu May 16 14:13:05 MET_DST 2002 ftp-deb@churchill:~ head fsn-cd.log Starting at Thu May 16 14:05:01 MET_DST 2002 [...] receiving file list ... done deleting woody/woody-i386-8.raw.list deleting woody/woody-i386-8.raw [...] So I'm currently in the process of downloading everything (on the second cd right now)... I think it is a good change, but I would have appriciated knowing about it a bit earlier. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: heads up for fsn debian-unofficial mirrors
On Thu, 16 May 2002, Attila Nagy wrote: Hello, [...] receiving file list ... done deleting woody/woody-i386-8.raw.list deleting woody/woody-i386-8.raw [...] Ooops, sorry. BTW, the rsync switch --delete-after delete after transferring, not before is quite useful, if you have enough free space. Yes, I know. I use that normally, but i missed it this time around. I'll change that now. I think it is a good change, but I would have appriciated knowing about it a bit earlier. Sorry, I planned to send that message yesterday, but had to go earlier, so I couldn't. I hope you can find a fast mirror. Yeah, I'm mirroring from ftp.no.debian.org that hasn't gotten a trigger from me yet. Shouldn't take that long, I get a couple of megs/s from there :) /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: .raw extension is misleading
On Fri, 17 May 2002, Richard Atterer wrote: On Wed, May 15, 2002 at 04:29:20PM +0100, Philip Hands wrote: On Tue, 2002-05-14 at 19:14, Richard Atterer wrote: jigdo/, not jigdo-area/. (Not that it really matters...) I wasn't overly happy about using jigdo-area, but couldn't think of anything more appropriate. The reason not to call it simply jigdo is that the jigdo-area contains one or more versioned directories, each of which contains a jigdo and a snapshot directory, so having jigdo appear at two levels in the hierarchy seemed likely to give rise to confusion. That's true. Why not call the topmost dir jigdo and leave out the second one, i.e. snapshot in jigdo/2.2r6/snapshot/ jigdo/template files in jigdo/2.2r6/i386 Well, it isn't jigdo that is distributed there. It is jigdo files for use with jigdo. Or what about cd-images? snapshot in cd-images/2.2r6/snapshot/ jigdo/template files in cd-images/2.2r6/jigdo/i386 This would seem better. I'm fine with jigdo-area/ though. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Are PIK mirrors obsolete?
On Sun, 26 May 2002, H. Peter Anvin wrote: On mirrors.kernel.org, we recently suffered a major disk failure, which among other things ate our PIK CD mirror setup. I went to look for the software again, and find that it has mostly been superceded by jigdo, and that is the only way to download the Debian DVD images. PIK was kind of inefficient and required a final rsync to get the full image, which requires (at least) the master site to carry a full set of images. Jigdo is kind of a better PIK in that it builds an actual image, not just a pseudo-image from a jigdo file and a template (containing the literal data not found in packages on your local debian mirror). Since no mirror carries the Debian DVD images jigdo is the only way to get it. The downsides if jigdo is that it is a fairly complex C++ program which means some portability issues when dealing with older systems. There exists a C port, but that is not actively maintained currenlty. It is also a fairly new system, which means just the obvious bugs have been noticed and taken care of. I believe there are some usability issues to take care of after The Release. It makes me reluctant to spend the work to resurrect the debian-cd mirror on mirrors.kernel.org. Perhaps someone could explain the situation as well as this jigdo business to me... Basically, what you need to do is install the jigdo program and then this script (with a few modifications) should do the job of building images from a mirror of the jigdo files: http://www.acc.umu.se/~maswan/debian-push/jigdo-mirror Richard: Perhaps the jigdo-mirror script should be in a more official place than here and in the list archives? /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Are PIK mirrors obsolete?
On Sun, 26 May 2002, H. Peter Anvin wrote: Mattias Wadenstein wrote: Basically, what you need to do is install the jigdo program and then this script (with a few modifications) should do the job of building images from a mirror of the jigdo files: http://www.acc.umu.se/~maswan/debian-push/jigdo-mirror Richard: Perhaps the jigdo-mirror script should be in a more official place than here and in the list archives? Okay... we had the same issue with PIK before the debcdmirror script came out (and there is a similar issue with the RedHat-apt scripts, but that's another issue.) The jigdo-mirror script would be the equivalent of debcdmirror, it could probably use some work since I think I am the only one that has reported usage of it (and I found some issues). Seriously, I won't be setting up anything until there is a standard piece of software I can download, configure, make (whatever), point it at a configuration file and have it operate autonomously, creating the whole directory hierarchy. It's too much work for me to operate anything else. This is what jigdo-mirror is supposed to do. It does depend on the jigdo binary, which should be a simple configure, make, install (if it compiles). But you could always do plain old rsync-mirroring. Sure, you'll download quite a bit more data than strictly needed if you also have a debian mirror, but I woulnd't mind if you did that to my mirror (ftp.acc.umu.se, aka ftp.se.debian.org aka ftp.gnome.org). Or you could wait until after the woody release, jigdo is new and really put into place with the distribution of woody images. I'm sure most issues will be dealt with a couple of weeks after that. Actually, that would be my suggestion if you don't want to deal with this now. Run a plain rsync-mirror from us and come back to this issue later when you have the time and the software might have matured a bit. /Mttias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Are PIK mirrors obsolete?
On Sun, 26 May 2002, H. Peter Anvin wrote: Mattias Wadenstein wrote: But you could always do plain old rsync-mirroring. Sure, you'll download quite a bit more data than strictly needed if you also have a debian mirror, but I woulnd't mind if you did that to my mirror (ftp.acc.umu.se, aka ftp.se.debian.org aka ftp.gnome.org). Or you could wait until after the woody release, jigdo is new and really put into place with the distribution of woody images. I'm sure most issues will be dealt with a couple of weeks after that. Actually, that would be my suggestion if you don't want to deal with this now. Run a plain rsync-mirror from us and come back to this issue later when you have the time and the software might have matured a bit. Works for me... which module should I mirror? ftp.acc.umu.se::debian-iso If you just want a mirror of the official released images. I have some other images around too, but that module is the place where official images will end up. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Are PIK mirrors obsolete?
On Mon, 27 May 2002, Richard Atterer wrote: On Mon, May 27, 2002 at 12:57:01AM +0200, Mattias Wadenstein wrote: The downsides if jigdo is that it is a fairly complex C++ program which means some portability issues when dealing with older systems. FWIW, I fixed a few issues lately. I suspect it still won't work for you, but if you have the time you can try it out again and tell me where it fails now. I'll try it when I get time, hopefully next week, I'll also check how it performs when both source and destination are nfs-mounted directories. But since we do need a temporary server anyway to be able to handle the release... There exists a C port, but that is not actively maintained currenlty. It is also a fairly new system, which means just the obvious bugs have been noticed and taken care of. I believe there are some usability issues to take care of after The Release. What usability issues? :) I'm making slow progress with the jigdo GUI app if that's what you mean... Mostly the general feeling of the programs and scripts. There are some rough edges, nothing that isn't easily fixed when you actually notice exactly what they are. But this usually needs new users, once you are used to handling the program and scripts, you don't notice them. That is why I was refering to this as a thing for the future, I just have this general feeling, but I can't think of any particular issues. http://www.acc.umu.se/~maswan/debian-push/jigdo-mirror Richard: Perhaps the jigdo-mirror script should be in a more official place than here and in the list archives? jigdo-mirror has been part of the last two jigdo releases. I always had the feeling that nobody bothers to read my release announcements. ;-P Sorry, i haven't had the time to read them in detail. I just checked the webpage now when responding to this. On Mon, May 27, 2002 at 01:31:10AM +0200, Mattias Wadenstein wrote: The jigdo-mirror script would be the equivalent of debcdmirror, it could probably use some work since I think I am the only one that has reported usage of it (and I found some issues). Yes, so far yours was the only feedback. Or you could wait until after the woody release, jigdo is new and really put into place with the distribution of woody images. I'm sure most issues will be dealt with a couple of weeks after that. Are you thinking of any particular issues? None that I can think of, but then I'm the only feedback so far. It would be a resonable guess that there will be some more, especially when it comes to usability. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: CD release announcement
On 30 May 2002, Philip Hands wrote: On Thu, 2002-05-30 at 13:16, Richard Atterer wrote: Where will the official 3.0r0 jigdo files be placed? I think it would be wise *not* to use cdimage/jigdo-area for that, because does this work if you point it at: http://cdimage.debian.org/jigdo-area/current/ ? If so, there's no problem (as long as I remember to update the symlink :-) Just a small mention, I mirror jigdo-area daily now (excluding snapshot) on http://ftp.se.debian.org/mirror/debian-cd/jigdo-area/ - after the release I'll make the symlink (taking out mirror/ in that url) and stuff work so there isn't both a debian-iso and debian-cd with slightly different things (debian-iso is a pure mirror of the potato release images) Right now I made the decision that it is better to leave debian-iso as the mirror it is. The other thing that is likely to happen (I should be able to sort it out soon) is that cdimage will actually be a CNAME for raff.d.o and non-us.cdimage.debian.org will point at open. Where will the jigdo files turn up? open is a bit closer, but I get good rates to raff too. Just wondering if I should have that cron job that mirrors jigdo-area point at cdimage or open. I'll probably do alot of manual work at release time anyway, but it would be good to know where stuff will turn up. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Woody Mini-cd results
On Sat, 1 Jun 2002, Chris Lawrence wrote: On Jun 01, D E Radel wrote: Your 185MB CD was a life-saver! It's a keeper! On the same topic, I've added a CD for Alpha and updated the CDs to woody as of a couple of days ago (that makes this revision 15 for those of you keeping score at home). The Alpha CD is fairly important to test because I used a isomarkboot compiled for i386 to make it bootable; IIRC nobody has tested whether that works yet (there was some discussion on the debian-cd list about it though). Anyway, the URL: http://www.phy.olemiss.edu/debian-minicd/ For rsync locations, mirrors, and md5sums, see: http://www.phy.olemiss.edu/debian-minicd/README.txt Perhaps http://ftp.acc.umu.se/pub/cd-images/debian-minicd/ would be a better link to our mirror, that doesn't reflect the old place they were in. I just created that symlink. On the other hand, since it is a symlink rsync doesn't seem to be happy taking just that directory as an arguemnt (files in it is fine) without -L. For end users this shouldn't be an issue though, only for those mirroring from us. Both are there, you decide which one you want in the mirror listing. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: jigdo-easy threw a firewall ?
On Fri, 7 Jun 2002, Richard Atterer wrote: Hi Mickaël, On Fri, Jun 07, 2002 at 03:52:14PM +0200, Mickaël Canévet wrote: Or is there any mirror with a full dvd iso, not only the jigdo file ? No, sorry - these files are simply far too big to be distributed conventionally. That sounds like a fun challenge. I'll take a look at it after I manage to get a stable jigdo mirroring solution somehow. :) /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: pre-release 3.0 p11 published, p12 coming soon
On 28 Jun 2002, Philip Hands wrote: On Thu, 2002-06-27 at 20:25, Philip Hands wrote: pre12 is now mostly out. And I just built the set on ftp.se.debian.org/mirror/debian-cd/ with jigdo. Some experiences: Having tmpdir on a fast local file system helps alot compared to having it on the same nfs-mounted directory, even though it is mv:ed remotely in the end. Total runtime: 12 hours. I expect I can do some nfs tweaking to bring that time down an hour or two (I forgot to mount the rw-version of the mirror fs with udp, so the mv at the end took longer time than reading all the packages in some cases (peak rates at 6M/s vs 3M/s)). /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian 3.0 released. Jigdo files available.
On Mon, 22 Jul 2002, Attila Nagy wrote: Yes, if you don't know how to use jigdo, you won't be able to get isos from the master sites (at least not yet for a while). This requires manual intervention, that's my only problem. Yeah, for the first days manual intervention. The isos are available on rsync-debcd.acc.umu.se though, but routing from there to your site is rather bad. It seems that not the routing, but something else. I can't connect to that machine, using rsync. (the port is open, but I don't get anything back) ftp.acc.umu.se works fine. Hmm.. Seems to be broken then. It worked fine when I tested it.tm :/ I'm moving this to another server now anyway, it should be up in a matter of hours. I am not aware of any other mirror with them online. The case is trivial :) The two behaviours I've seen so far: - they just don't care and are waiting for updates from cdimage.debian.org, where they've used to mirror from - they did a search and found jigdo, but don't know what is that (they are now learning :) - they already got the files with some manual magic (jigdo, download from another site, etc) This release method saves lot of internet pipes from being saturated. :) The point being giving a fair chance of a bunch of mirrors getting the images before cdimage.d.o gets saturated. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
All woody isos up on rsync-debcd.acc.umu.se
This time on a working temporary host. rsync: rsync-debcd.acc.umu.se::debian-cd http://rsync-debcd.acc.umu.se/debian-cd Benchmarking has given the bottleneck to be the slow ata raid card delivering about 20MByte/s. This machine is dedicated for this purpose for the time being. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: jigdo
On 24 Jul 2002, Philip Hands wrote: On Wed, 2002-07-24 at 03:34, Bradley Glonka wrote: any chance we'll see official images on the cdimage rsync server? You mean .iso's? On current evidence, I don't think it would be productive, because it would clog up cdimage.d.o to a point that the people that are currently successfully using jigdo, might not be able to grab snapshot files they might need, without really improving things. There are a few mirrors offering .iso files, so if you need them, they are out there. If you tell me that those mirrors are too busy to be usable, then you'll be making my point for me. :-) Well, ftp.se.debian.org are offering them, but is a bit too busy to be nicely usable. That is why http://rsync-debcd.acc.umu.se (also with rsync access) offers them with a very fast connection and has pretty much no usage. Philip: A pointer to that host on http://cdimage.d.o perhaps? A link in the README.html or something like that? /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: I am stupid...
On Mon, 19 Aug 2002, Sergio Restrepo wrote: The topic may very well hold true, but here goes my question: I have browsed thru practically all of the links in: http://www.debian.org/CD/http-ftp/ And I am yet to find the iso images for 3.0 It would be great if you could clarufy that for me or just plain call me stupid. The problem is that most of them does not have the 3.0 images, in favour of the jigdo distribution method. I would suggest that the mirrors having 3.0 images will be flagged as such in the mirror listing. The ones I know about: ftp.se.debian.org ftp.du.se ftp.de.debian.org I think there is one in australia too. Then there are a couple with just some architectures. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: No US sites with 3.0 CD images
On Mon, 26 Aug 2002, Richard Atterer wrote: On Mon, Aug 26, 2002 at 10:30:58AM -0700, [EMAIL PROTECTED] wrote: There appear to be no US mirror sites with 3.0 CD images. The only site that lists 3.0, mirror.csit.fsu.edu/debian-cd, has disabled access due to too much traffic. Hm, maybe rsync access to the images on raff and/or open should be enabled? I originally said in the announcement sent to mirror admins that this was going to happen, so many of them are probably still waiting for it... :-/ Or just have the standard mirroring way of setting up an rsync and forgetting about it. Of course, that means getting the images on debian.hands.com or changing the dns too, I rather doubt that any forgotten rsync has updated to use raff. If they follow this list, they should know about a bunch of sites that has the images already. And if the mirror listing could be updated to reflect the status of the mirrors, perhaps it would be even easier to find. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Seeking advice w/r to debcdmirror
On Fri, 30 Aug 2002, Richard Atterer wrote: On Thu, Aug 29, 2002 at 10:47:00AM +0200, Daniel Lang wrote: Ok, thanks. I followed the instructions on: http://www.debian.org/CD/mirroring/ which does not mention jigdo but the debcdmirror/pik. I've cc:'d [EMAIL PROTECTED], so I hope they will update the page. I've had plans to do that for a while, but I'm not sure what the new contents should be. Should jigdo-mirror be the new standard method for mirroring CDs? Unfortunately, I have received next to no feedback about jigdo-mirror, for all I know it might still contain some bugs. Lots of debug info (like wget output) that for the most part isn't needed, what I would like is a summary of what cds weren't generated successfully. Otherwise I think that the only modification I did was up the number of max files do be downloaded from fallback server to enough. The script worked. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: [Roland Mas lolando@debian.org] Re: debian-installer status-- 2002-10-14
On Fri, 1 Nov 2002, Richard Atterer wrote: On Wed, Oct 30, 2002 at 02:02:20AM +0100, Tollef Fog Heen wrote: would you make the appropriate files (*.jigdo and *.template) available through anonymous rsync? I had to wget -r to get them, and that gives me lots of unneeded files (HTML stuff mostly). And it's hard to keep up-to-date. Well, rsync isn't really /that/ useful for jigdo/template files because the data of those files is compressed in an rsync-unfriendly way with zlib... But rsync is still a convenient way to mirror a directory structure. wget's mirror functionality isn't great. You could try lftp instead, it can omit the HTML files and also delete old files: lftp -c 'open http://foo.com/bar/; mirror --verbose --delete . /tmp/bar' Sure, that works too. Not as convenient as just adding yet another rsync if you run a largish mirror at least. But that might not be the case here. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Too few up-to-date CD image mirrors
On Fri, 29 Nov 2002, Joey Hess wrote: Richard Atterer wrote: Basically, the current situation WRT CD image mirrors is getting, er, irritating: Far too few of them carry the 3.0r0 images, many only have the jigdo files or 2.2r6. BTW the situation seems to be most pathetic for the US mirrors for some reason, I have much better luck finding isos on the other mirrors. Cannot find any debian iso's of 3.0 in the US at all. Yes, I have gotten the question on can I find it anywhere closer than Sweden from people in the US, and not being able to answer. I can name 2 or 3 mirrors in Sweden and a bunch more in other european countries though. Originally, the plan for 3.0r0 was that ISOs would be made available via jigdo and that mirrors would update using the jigdo-mirror script which I wrote especially for that purpose. (And I mailed all mirror admins with the details.) Later, the images would also become available normally via rsync (AFAIR), but somehow that never happened. But few admins seem to have set up jigdo-mirror... I think for most of them, mirroring something is not an option if it can't be fetched with http, ftp or rsync. shrug Well, even for secondary mirrors one of our mirrors found a problem with some version of a debcdmirror script that requires an ls-lR file that we do not have. So let's do it their way: I propose that we offer rsync (and maybe HTTP?) access to the images again starting with 3.0r1. raff has enough disc space, so the size should not be a problem. Well, when it comes to larger mirrors (as in number of diffrent archives they carry), compiling/installing new software for a specific archive is probably considered a bit too much work. I see jigdo as a temporary way to get the jigdo files from the main server before the isos are made available a week later or so. To avoid the huge congestion at release time. I would guess at about 10-20 mirrors in total, but that should be enough to get the isos out in resonable time after a release. Also this kind of change takes time. If you get no user inquiries about why hasn't $foo updated for the last couple of months, chances are that you'll never notice unless the mirroring script starts logging errors. I offered the web team to write a program to check the cd image mirror list, and output some kind of table of what ftp mirrors: - are down or broken - have jigdo - have current isos With the idea being this would be used to prune the list shown to users on the web site or perhaps generate a seperate list for people who get to the mirror list wanting jugdo files, vs. those who get to the list looking for an iso mirror. I think it's important to automate this. Yes, this would be very good. I have been thinking of writing such a script myself, but never found the time for it. I would be very grateful if you could write such a script. I will not try to check the http mirrors though. Indeed, I think the http mirrors are a bad idea, since (as is seen in my mistake reading one of them in the gentoo thread), they introduce a bunch of third party web sites of varying quality that users must navigate to find isos. I hope the http stuff is not a necessary evil. For us http is just a better ftp, without the logging in as anonymous part. The standard apache file listing is good enough. Once the mirror list is whipped into shape, it'll also be possible to do things like a cgi that uses it to pick a mirror based on a user's address, for one-click iso downloading. Could be nice, but the most important thing is to be able to separate the list into chunks with the top one being the mirrors carrying the latest release. Or course it would be best if I had a canoical site to run the checking program against, then it could just do a straightforward comparison. Unfortunatly, it seems that cdimage.debian.org no longer carries isos, and so there is no canoical site? Is that right? That has been a main issue for getting other mirrors to mirror the images too. They aren't on cdimage.debian.org, so why should we bother with some kind of not as official ones. The fact that we carry isos with released md5sums seems not as important. Or just a convenient excuse to be a bit more lazy. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Too few up-to-date CD image mirrors
On Fri, 29 Nov 2002, Joey Hess wrote: One interesting thing that I'm running into is that some mirrors have a cd with size 614780752, and some have 612171776 for debian-30r0-i386-binary-1.iso. I'm assuming the more prevalant 612171776 is correct, but lack the facilities to check md5sums or anything. Some mirrors with the bigger sized cd's include: ftp.tu-clausthal.de ftp.cvt.stuba.sk tantale.fifi.org ftp.tku.edu.tw ftp.belnet.be ftp.funet.fi Of course my script throws these bad sized ones out but I wonder what is going on. Where do you get those sizes from? My ftp and rsync clients shows 612171776 for debian-30r0-i386-binary-1.iso on those mirrors. lftp ftp.funet.fi:/pub/Linux/mirrors/debian-cdimage/3.0_r0/i386 ls debian-30r0-i386-binary-1.iso -rw-r--r-- 1 mirrormirror 612171776 Jul 20 16:31 debian-30r0-i386-binary-1.iso This is also the size of the downloaded file, which has a correct md5sum. I also noticed that I have this structure: debian-cd/3.0_r0/images/i386 Should I remove the images dir? I am fairly certain that it ended up there during the discussions on jigdo-mirror and a standard structure for the release, but since there is no official master site, I can't be sure. Other mirrors seems to not have the images dir. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Too few up-to-date CD image mirrors
On Fri, 29 Nov 2002, Joey Hess wrote: Mattias Wadenstein wrote: Where do you get those sizes from? My ftp and rsync clients shows 612171776 for debian-30r0-i386-binary-1.iso on those mirrors. lftp ftp.funet.fi:/pub/Linux/mirrors/debian-cdimage/3.0_r0/i386 ls debian-30r0-i386-binary-1.iso -rw-r--r-- 1 mirrormirror 612171776 Jul 20 16:31 debian-30r0-i386-binary-1.iso This is also the size of the downloaded file, which has a correct md5sum. Hmm, my script gets the size with the ftp SIZE command. For example, on ftp.tu-clausthal.de: Net::FTP=GLOB(0x8327c38) SIZE debian-30r0-i386-binary-1.iso Net::FTP=GLOB(0x8327c38) 213 614780752 It too shows up as 612171776 in the ls command, so I am not sure what is going on there. Ok. If the SIZE command says that on ftp.funet.fi, it is lying. That is the image I downloaded and checked the size and md5sum on. I also noticed that I have this structure: debian-cd/3.0_r0/images/i386 Should I remove the images dir? I am fairly certain that it ended up there during the discussions on jigdo-mirror and a standard structure for the release, but since there is no official master site, I can't be sure. Other mirrors seems to not have the images dir. I only find one or two with the images directory. IMHO it should be removed for consistency. My script shall not default to looking for it. Ok. I'll wait a day or two to see if someone shouts that it should be left as it is, then I'll move it. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: state of the cdrom mirrors report
On Fri, 29 Nov 2002, Joey Hess wrote: Joey Hess wrote: Here is the current state of the cdrom mirror network. I decided not to try to track yada files since the web site only links to 2 mirrors for yada stuff. I added source cd checking. It only checks the first image for US and non-US. Correction, it checks for the first image binary and source. It does not check for non-us stuff at all, though I could add it. A check for i386 (and perhaps source) or a full set could also be interesting for those users that aren't on i386. I think non-us is less important. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: state of the cdrom mirrors report
On Sat, 30 Nov 2002, jason andrade wrote: On Fri, 29 Nov 2002, Joey Hess wrote: Many sites have old isos, or no isos, or a nonstandard directory structure or filenames, earning a 0 in the binary column. And of course a lot of mirrors don't have source. hmm, i have been skipping most of this thread. what new isos ? i was under the impression (as of the last few years) that isos are generated once for a release and unless there are some exceptional circumstances are never regenerated without an increment of some kind. New isos as in 3.0. Many sites just carry 2.2_r[67]. at ftp.au.debian.org, we generated the images using jigdo and then did a final sync against the authoritative images at cdimage.debian.org. we never check after that for various reasons o images are never supposed to change o there could be some problem at cdimage which would wipe out our local archive also Yes, this is the good method of handling cdimage mirroring. so has there been some fundamental shift in the way debian is producing iso images - as evidenced by the very high number of sites below which get a 0 in the binary column ? Just that cdimage.debian.org does not carry the images, just the jigdo files. That is enough to get most sites not updated. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Too few up-to-date CD image mirrors
On Mon, 2 Dec 2002, Josip Rodin wrote: On Fri, Nov 29, 2002 at 09:23:07PM +0100, Richard Atterer wrote: Originally, the plan for 3.0r0 was that ISOs would be made available via jigdo and that mirrors would update using the jigdo-mirror script which I wrote especially for that purpose. (And I mailed all mirror admins with the details.) Later, the images would also become available normally via rsync (AFAIR), but somehow that never happened. So let's do it their way: I propose that we offer rsync (and maybe HTTP?) access to the images again starting with 3.0r1. raff has enough disc space, so the size should not be a problem. Of course, we'd better implement some kind of access control. And what about cdimage.d.o? - it would be nice to have another primary mirror in Europe to distribute the load. I asked Ryan (of DSA) about this and he told me that he'd prefer it if we just had people use jigdo since the debian/ mirrors are plenty and likely much faster than mirrors overloaded with heaps of rampant image downloaders. Frankly I agree with that sentiment, there sure are plenty of better ways to waste bandwidth. Yes, but jigdo is more hassle for the end users. Sure, it is a waste of bandwidth, but if we didn't have bandwidth to waste, we wouldn't run a debian[-cd] mirror do begin with. You have a tradeoff there between what is convenient for users and what is an effective use of our mirroring resources. But then, the mirror admins that voice their opinions here are those that have already stated that they would rather have isos available. But then, we already have a bunch of mirrors that do have images, so _something_ needs to be done to bring back the consistency. Yes. A restricted rsync access to main mirrors would be good enough if we don't want cdimage.d.o (or cdimage.us) to waste too much bandwidth. Just so that once we have built the images with jigdo we can rsync the directory structure against an official structure. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Too few up-to-date CD image mirrors
On Mon, 2 Dec 2002, Josip Rodin wrote: On Fri, Nov 29, 2002 at 11:45:54PM +0100, Mattias Wadenstein wrote: Or course it would be best if I had a canoical site to run the checking program against, then it could just do a straightforward comparison. Unfortunatly, it seems that cdimage.debian.org no longer carries isos, and so there is no canoical site? Is that right? That has been a main issue for getting other mirrors to mirror the images too. They aren't on cdimage.debian.org, so why should we bother with some kind of not as official ones. The fact that we carry isos with released md5sums seems not as important. Or just a convenient excuse to be a bit more lazy. Hey, cut them some slack -- AFAICT, we just left cdimage.d.o lingering there with the old images for months, and nobody informed the less involved of the mirror admins about any changes. Heck, just a few weeks ago I mentioned some of the broken links to Phil and his response was more like oops, better fix that. I know for a fact that mirror admins are a relatively clueless bunch, but I also know that they can be educated given enough effort. Well, usually the bigger they are the more clue they have but less effort to put into a single archive. Of course, this excuse was only part of the problem, of course they were (as usual on mirrors) almost out of space and haven't quite got the next generation server online. I think it is almost something mythical about the next server that will have good enough capacity unlike the current one that is way too slow. It always takes much, much longer to get running than one could hope. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Too few up-to-date CD image mirrors
On Mon, 2 Dec 2002, Anthony Towns wrote: On Fri, Nov 29, 2002 at 09:23:07PM +0100, Richard Atterer wrote: So let's do it their way: I propose that we offer rsync (and maybe HTTP?) access to the images again starting with 3.0r1. raff has enough disc space, so the size should not be a problem. Yay. Also, it would be very nice to have the netinst image become official sooner rather than later. Pointing to images we don't control is bad, recommending them above what we do provide smacks of incompetence. Yes, this is probably even more important. Who makes that decision and how do we make it happen? Because netinst images are important ones and it has been the consensus of the debian-cd list that users should be pointed to that one before the full isos. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Too few up-to-date CD image mirrors
On Mon, 2 Dec 2002, Josip Rodin wrote: On Mon, Dec 02, 2002 at 01:08:51PM +0100, Mattias Wadenstein wrote: Yes, but jigdo is more hassle for the end users. Sure, it is a waste of bandwidth, but if we didn't have bandwidth to waste, we wouldn't run a debian[-cd] mirror do begin with. You have a tradeoff there between what is convenient for users and what is an effective use of our mirroring resources. But then, the mirror admins that voice their opinions here are those that have already stated that they would rather have isos available. Yep, that's quite a valid argument. I also have images on my mirror because I do have the bandwidth, and if it'll help a bunch of users, it's probably worth it, so I keep them. The thing is, it appears that bandwidth is not such a commodity in the US ;) and all of our servers that have the disk space -- saens, raff, gluck -- are there. debian.hands.com, which is more-or-less our server (it's listed at machines.cgi but it's still Phil's machine) barely has the space for the stuff that's already on it, and none of the others seem to have space either. Well, then the US users will just have to download them from over here? :) We should probably think about getting another few drives into the new syncproxy.eu for this... Well, just one more 120 gig ide drive would be enough to add to that raid5 set, I think. I'll take a look at it. Need to get that machine installed and so on too. But then, we already have a bunch of mirrors that do have images, so _something_ needs to be done to bring back the consistency. Yes. A restricted rsync access to main mirrors would be good enough if we don't want cdimage.d.o (or cdimage.us) to waste too much bandwidth. Just so that once we have built the images with jigdo we can rsync the directory structure against an official structure. Hmm. One thing, though, why keep the images locked on that machine just for a sync that happens every few months? Plus the administrative overhead of giving out accounts. Because it is hell to try and mirror the images at release time when the master machine is overloaded with a gazillion end users trying to download it. And since other mirrors can't get to the images to mirror, more end users end up at the master site. Having the root of the tree hidden from public access is a good thing when it comes to mirroring structures. The administrative overhead isn't that bad, I'd guess at 10-30 accounts that change very seldom. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Too few up-to-date CD image mirrors
On Mon, 2 Dec 2002, Richard Atterer wrote: On Mon, Dec 02, 2002 at 01:08:51PM +0100, Mattias Wadenstein wrote: A restricted rsync access to main mirrors would be good enough if we don't want cdimage.d.o (or cdimage.us) to waste too much bandwidth. Just so that once we have built the images with jigdo we can rsync the directory structure against an official structure. Hm, but rsync doesn't work well if files move, does it? You'll have to modify the dir structure yourself. Yes, that is true. But you do get a verification on that you got it right, also md5 files, readmes and other things like that. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: dvd-isos (was: jigdo files for sid-dvd)
On Wed, 28 May 2003, Attila Nagy wrote: [snip] Maybe you are running a Linux kernel which is unable to handle bigger than 2/4 GB files... Heh, good place to put in a reminder to the kfki.hu guys that perhaps there is such an issue there too. ftp: get: Access failed: 550 Can't open sarge-i386-1.iso: File too large http: 403 Forbidden rsync: write failed /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: dvd-isos (was: jigdo files for sid-dvd)
On Wed, 28 May 2003, Kadlecsik Jozsi wrote: On Wed, 28 May 2003, Mattias Wadenstein wrote: On Wed, 28 May 2003, Attila Nagy wrote: [snip] Maybe you are running a Linux kernel which is unable to handle bigger than 2/4 GB files... Heh, good place to put in a reminder to the kfki.hu guys that perhaps there is such an issue there too. ftp: get: Access failed: 550 Can't open sarge-i386-1.iso: File too large Yes, ftpd lacked the largefile support. The daemon is recompiled and restarted, now it servers large files well. Ok. http: 403 Forbidden That's strange: I have just downloaded sarge-i386-1.iso without any problem. (Galeon does not display numbers properly, but the donwloaded file is OK.) Maybe I typoed it while testing. Rsync was the one that bothered me most since that is what I'm using to mirror that archive. And ftp had the nice error message. rsync: write failed rsync upgraded, now it seems to work fine. Hmm.. Oops. I forgot the softlimit at 2 gig we had, that was the rsync issue. Now we're updating fine. /Mattias Wadenstein - have been a bit too busy to debug this -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Some thoughts
On Tue, 15 Jul 2003, Richard Atterer wrote: On Mon, Jul 14, 2003 at 04:57:43PM +0200, Mattias Wadenstein wrote: On Sun, 13 Jul 2003, Mattias Wadenstein wrote: One suggestion for being able to have testing images that are just a simple repoint away from actual release would be to have a similar setup in the debian-cd dir as the main archive: debian-cd/ woody/ sarge/ stable - woody testing - sarge And then just repoint the stable link at release time. And in each of these directories: images/3.0_r1/hppa/ .../arm/ ... jigdo/3.0_r0/i386/ ../arm/ ... /3.0_r1/i386/ While we're at it, what about distribution of weekly snapshots? With new-style long filenames (say, debian-sarge-030711-alpha-binary-1.iso), rsyncing them will be very inefficient, and I doubt the mirrors will want to download all the data again every week. Possibilities: 1) Continue to use old-style names (woody-i386-1.iso) 2) Only release as .jigdo files 3) Only release on one server, don't bother mirroring My preference is 2), but as usual I'm biased... :-) Another issue is to distinguish testing-but-just-about-to-be-released images from weekly testing snapshot images in a clear way - how? Or were you thinking of distributing the weekly snapshots in sarge/? If so, the images would have to be renamed on release... :-/ Yes, and problem is that rsync can't handle file renaming. I'd say that we just keep the snapshots and testing stuff as jigdos, and then eventually make isos when it is released. That way mirrors will have alreday rsynced the jigdos and can make the isos themselves if they know how. The point of having a sarge directory with a testing link (that is then changed to a stable link) is to have the testing stuff out and close to the final data when the release comes. Perhaps we should go for the rsync hardlink tricks anyway? Just make a standard release and mirroring script that handles a .latest/arm/1.iso (and all the other numbers/arches). /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
cdimage push mirroring
I'm setting up cdimage push mirroring now. For this I need the script tested and I need a couple of more hosts that can do push-triggered mirroring for the new distribution tree. Current offers are from: ftp.fi.debian.org ftp.eu.uu.net ftp.de.debian.org So europe seems to be covered decently, US hosts would be very interesting. The script and stuff is here: http://www.acc.umu.se/~maswan/debian-push/cdimage/ It requires jigdo with jigdo-mirror installed. Read the REAME. The outline of the script is: * Delete the files in ::debian-cd/images/ that aren't on the master * Mirror ::debian-cd/jigdo/ * Generate images from the dir pointed to by current with jigdo-mirror * rsync all of ::debian-cd to get symlinks and dates and stuff and be sure everything got there correctly Questions, comments, suggestions, patches, rants, flames, donations to me and/or the list. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Status of cdimage mirroring
It has now become clear that farbror.acc.umu.se should take on the name
cdimage.debian.org soon so if you'd like to bugtest the machine (serving
rsync, ftp http) or have any objections, now would be a good time to
poke me about them.
Since there might be uncertanties about filenames, these seem like a
sensible suggestion:
debian-30r1-alpha-binary-1.iso
that is debian-{$release|snapshot}-{$arch-binary|source}-$n.iso
Do we really need to include the -binary part or wouldn't just $arch be
enough? Of course, just using this would mean that renaming and jigdo
hacking wouldn't be necessary. :)
Also a few more push-triggered mirrors would be really nice. I'll write an
email about this to the mirrors list soon too.
/Mattias Wadenstein - the cdimage mirroring guy
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Accessibility of official jigdo files for Sarge
On Thu, 24 Jul 2003, John Winters wrote: Hi all, I sell (amongst lots of other things) CDs of Debian Woody and testing snapshots of Sarge. Currently it appears that the only way one can download the official .jigdo files for unofficial Sarge CDs (!) is by means of http from gluck.debian.org. This is inefficient, particularly if one wants to keep doing it. Is there any way they could be made available by rsync? This is in the plans for the new cdimage.debian.org so you could mirror from there. But then I too need an efficient method of getting them to that host. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: cdimage push mirroring
On Thu, 17 Jul 2003, Mattias Wadenstein wrote: The script and stuff is here: http://www.acc.umu.se/~maswan/debian-push/cdimage/ It requires jigdo with jigdo-mirror installed. Read the REAME. The outline of the script is: * Delete the files in ::debian-cd/images/ that aren't on the master * Mirror ::debian-cd/jigdo/ * Generate images from the dir pointed to by current with jigdo-mirror * rsync all of ::debian-cd to get symlinks and dates and stuff and be sure everything got there correctly Should this current link be changed into a stable and a testing link for the current stable and testing versions? Keeping a few revisions of old jigdos around is probably considered a good idea, at least for stable revisions. This is reflected in the current state of farbror.acc.umu.se. If this is interesting I could add this to the script with the option of generating isos based on the stable and/or testing links. The downside is that with options it won't be an exact mirror of the data on cdimage.d.o. And regarding getting data to farbror, perhaps it would be valuable to get testing snapshots out and circulating among mirrors? If someone could give me a private (or public) rsync module on gluck I could start mirroring a bit. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Accessibility of official jigdo files for Sarge
On Fri, 25 Jul 2003, Attila Nagy wrote: Richard Atterer wrote: The changes from week to week can only be small and it would make sense to rsync the differences and re-build the files rather than re-download what can be a very big file. (Some of the later ones from fsn.hu have been 90M for just one (gzipped) .template file.) I wonder why they're getting so big - if all files inside an .iso are found by jigdo-file, the resulting .template file should be 5MB. :-/ The largest ones at ftp.fsn.hu are: 35M./sarge-dvd/jigdo/sarge-ia64-1.template 35M./sarge-dvd/jigdo/sarge-mips-1.template 35M./sarge-dvd/jigdo/sarge-mipsel-1.template 35M./sarge-dvd/jigdo/sarge-s390-1.template 35M./sid-dvd/jigdo/sid-i386-1.template 38M./sarge-dvd/jigdo/sarge-arm-1.template 38M./sarge-dvd/jigdo/sarge-m68k-1.template 38M./sarge-dvd/jigdo/sarge-powerpc-1.template 38M./sarge-dvd/jigdo/sarge-sparc-1.template 39M./sarge-dvd/jigdo/sarge-alpha-1.template 47M./sarge-dvd/jigdo/sarge-i386-1.template The jigdo directories went up to 10+ gigs and we had to stop mirroring for a while (the debian main archive is kind of more important to us). Is this problem fixed now? Which I guess makes sense, because the following files are left out from the .jigdos: egrep -v '/Contents|/Packages|/README|INDEX$|/Maintainers|/Release$|/debian-ke yring\.tar\.gz$|/ls-lR|//doc/[^/]+/?[^/]*\.(txt|html)$' \ Can the d-i images be found by jigdo or are they still downloaded from ~tfheen somewhere? /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Accessibility of official jigdo files for Sarge
On Fri, 25 Jul 2003, Attila Nagy wrote: Mattias Wadenstein wrote: And mirrors would like to use rsync even if the rsync algorithm doesn't matter. It is a convenient and well-known tool for mirroring a directory structure, including all the metadata and stuff. If we wouldn't talk about Linux I would suggest forget rsync. It's very bad in handling many thousands of files, both on the client and the server side. We switched to cvsup where we could (for example FreeBSD mirror). It's a lot more efficient. A full FreeBSD mirror can be done in about 4 MBs of memory (about 600-800,000 files as far as I can remember). And also it is much faster in the start of the process. Ah, this seems like a good tool in that regard then. Not that it matters for cdimages and jigdos but it could be relevant for the man debian archive where the rsync file list building dominates sync time in some cases. Thanks for the tip, I'll take a look at it. It still doesn't change the fact that rsync is the well-known tool by mirror admins though. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Accessibility of official jigdo files for Sarge
On Fri, 25 Jul 2003, Attila Nagy wrote: Mattias Wadenstein wrote: Thanks for the tip, I'll take a look at it. It still doesn't change the fact that rsync is the well-known tool by mirror admins though. The only problem is that cvsup is written in Modula-3, so it's not quite in perl or C :) http://www.cvsup.org/ Ah, seems like quite an unfriendly language from a sysadmins PoV with a very limited set of ports from what I could find. Oh, well. There exists a debian package. So for some of our mirrors it shouldn't be much of a problem. But I guess that rsync will be the standard way for some more years. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Accessibility of official jigdo files for Sarge
On Tue, 29 Jul 2003, Josip Rodin wrote: On Tue, Jul 29, 2003 at 10:00:16AM +0200, Attila Nagy wrote: For Debian mirrors, we have practically no use in features like compression, RCS, deltas and whatever else. To eliminate the initial delay in rsyncing, now that would be a useful thing. Did you try that, is that improved with cvsup? No, I didn't try it with Debian mirrors, but I did with FreeBSD. It contains much more files than Debian, because of the CVS repositories checked out. It starts the synchronization much earlier than rsync debian/ mirrors don't contain CVS stuff, and most probably never will. When you look at it, they are quite trivial, because just a handful of files are ever updated -- only new files are added. Hence, results with updating, diffing files, they probably don't matter to us. We need an experiment with our stuff. What needs to be done to test this? 1. Run cvsupd on a machine. 2. Run cvsup to update a mirror from that first machine. Just look at top and so on during this time. :) The big expected difference is due to a local cache of filenames that is kept on the client side and threaded approach to start sending needed files as soon as they are discovered, before doing a full directory tree traversal and so on. and which is even better, the memory requirement is very low (maximum 4-6 MBs). I guess only the latter worth the change. We had many rsync mirrors and our machine ran out of memory very soon. Just imagine a mirror of four-five bigger projects (like NetBSD, FreeBSD, OpenBSD and Debian) and voila, you need minimum 1 GBs of RAM just for the rsync in memory file list. I didn't notice any noteworthy problems with a machine with 96 MB RAM, although I mass-mirror stuff less than half the size of debian/ (linux, gnu, php etc). The memory usage for the rsync process is about 40 megs for debian/ and increases with every file/dir/link/whatever added. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bittorrent distribution of cdimages
I suspect I'm not the only one that thinks that this might be a good idea. I'm actually planning on offering this, what I'd like feedback on is how official it should be. Should I (or whoever is doing them) place the .torrents in the debian-cd directory on cdimage.debian.org so that they are mirrored all over the place and look official or should I place them in some unofficial/test- directory? When it comes to tracker I'm planning on using a dedicated dns name with a short ttl so we can repoint it at need if the machine can't handle it (I should be able to find a fast one though). /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug in the FAQ
http://www.debian.org/CD/faq/ links to jigdo files of official releases and similar things at http://cdimage.debian.org/jigdo-area/current/jigdo/ This does not exist. http://cdimage.debian.org/debian-cd/jigdo/ is the place where they are and they are distributed to all the http/ftp servers that have adopted the new structure too. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Distribution via Internet2
On Thu, 9 Oct 2003, Scott Atchley wrote: Hi, We have been asked to distribute your ISOs for Internet2-connected users. We run a research testbed for sharable, wide-area, distributed storage. We offer tools that allow users to store and retrieve data stored on this testbed. Because these storage servers are on Internet2, users connected to Internet2-connected campuses can download the content at speeds from 30-90 Mbps (or faster if they have gigabit connections to the backbone and all the way through to their machine). You can read more about it at: http://linux.dsi.internet2.edu/ Looks like a fun project. From the looks of it you include european educational sites too, is this correct? Please add the above link to your mirror list for the United States and add (for Internet2 users only) after the link. Ok, we'll get around to that in the general mirror listing overhaul that we're going do do soon. /Mattias Wadenstein - on a european acadmic link -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Bittorrent distribution of cdimages
On Thu, 9 Oct 2003, Patrick Strasser wrote: Mattias Wadenstein wrote: [about offering Debian CD-images via BitTorrent] I suspect I'm not the only one that thinks that this might be a good idea. I'm actually planning on offering this, what I'd like feedback on is how official it should be. I know that people like BitTorrent, and it that it is a nice P2P-System. But what is the point of a second distribution system besides a perfectly working distribution system that uses already existing staging points? Because the cdimage distribution form is one that bittorrent does good. And that it already has some mindshare. I don't see an advantage in BitTorrent. It needs new software, users to spread data, extra administration, support. Moreover jigdo is IMO the best way to spread Debian CDs, thanks to the nature of ISO9660. Yes, but there is a fairly large install base that already has the software. The other point is that I'm willing to maintain this. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Enlighten me about jigdo ..
On Fri, 14 Nov 2003, Delian Krustev wrote: On Friday 14 November 2003 12:41, Richard Atterer wrote: An rsync-based scheme (whose design inspired that of jigdo) was used in the old days, the pseudo image kit (PIK). The use of rsync turned out to be problematic (too much load on the server), so one of the aims of jigdo was to get rid of rsync. All in all, the PIK worked well, but it didn't scale to large numbers of parallel downloads. This is a big step backward. Debian has 300 mirrors and push mirroring already running. Loadbalancing should not be problematic in that case. If a particular server gets too busy it might limit the number of concurrent connections. Priority connections might also be considered(e.g. always allow connection from other mirrors while keep the regular downloaders number in certain amount). And the cpu power certainly gets cheaper faster than the bandwith .. Something more. It's proven to work. Look at the rsync servers at let's say mirrors.kernel.org or planetmirror.com. The load that PIK caused was that it did an image that was close to the official one (thus pseudo image) that needed to be rsynced against the real image. The issue was not the load on the debian mirrors but on the debian iso mirrors that kept the real isos available for rsyncing. This rsync ate a lot of cputime. But the real issue is the one later on. Anyone can build a debian cd, but what people want is the official debian cd and that has to be an exact copy of the officially generated and tested one. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
3.0r2 images
Philip: Are you going to generate the 3.0r2 cd-images? If so, do you know how, where and when? If not, manty said that he can do them, he just suspects that you'll do them better. Or perhaps someone else on this list feels both qualified and willing do do this? Once I get a set of jigdos and then get them onto cdimage.d.o, I'll do the isos and push-trigger stuff. Just need someone to do the steps before that. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 3.0_r2 jigdos finally signed off --- please check MD5SUMs
On Sun, 7 Dec 2003, bob parker wrote: On Sat, 6 Dec 2003 12:17, Steve McIntyre wrote: On Fri, Dec 05, 2003 at 09:50:35PM +, Philip Hands wrote: On the whole, this could have gone smoother than it did, and would probably have gone much better if I'd just left it to Steve, who we have to thank for producing the vast majority of the images that we're actually using for 3.0_r2 --- Thanks Steve :-) Hmmm. Smoother, maybe. I just made a mistake copying the update files into place and overwrote the original MD5SUMS files for most of the architectures. Bugger. As I (obviously) can't recreate them with Phil's key, I've regenerated replacements that are now signed by me. And the update jigdos are in place now. Use them wisely... :-) In what place please, my search of debian still comes up with 3.0_r1. http://cdimage.debian.org/debian-cd/ has the new jigdos now, and isos for that matter. They should be turning up on the push-triggered mirrors in the next few hours. I'm kind of not assuming enough load to break the mirror run just for a point release iso, so I'm just allowing public access now and telling people about it. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Woody r2 ISO / Jigdo files
On Mon, 8 Dec 2003, Steve McIntyre wrote: On Fri, Dec 05, 2003 at 12:39:56PM +0200, Lior Kaplan wrote: Hello, I saw the announcement about r2 in the main site, but got surprised to see there are no ISO files of r2. I tried to find a jigdo files as a replacement, but these are not available either. Can you clear the case? Is it related to the security problem which occurred recently? A mixture of that and various other issues. The r2 jigdo files went up at the end of Friday, so should by now hopefully have propagated out through the mirrors and be available all over. There are actually isos too all over too. Unfortunately the mirror listings could need some auto-updating goodies to say which mirrors have updated, but if you can't find a local mirror http://cdimage.debian.org/debian-cd/ has the stuff. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 3.0_r2 update CDs broken?
On Wed, 10 Dec 2003, Steve McIntyre wrote: On Tue, Dec 09, 2003 at 07:30:06PM +, Steve McIntyre wrote: On Tue, Dec 09, 2003 at 05:02:40PM +, Steve McIntyre wrote: On Tue, Dec 09, 2003 at 11:35:57AM +, Steve McIntyre wrote: OK, found it. Local bug in how update-cd and md5sum were communicating. Now fixed, and I'm building new images and jigdos now. They'll be available later today, I hope. I have new images and jigdos ready to go. To avoid confusion between the first set and these, I want to name them something different. Suggestions? I'm thinking of debian-30r2.01-update-* atm... No better suggestions; I'll go with these now, and a README to explain the name in the 3.0_r2 directory. I've checked the new images, jigdos and templates, and they seem fine now. They're in place on non-us.cdimage.debian.org, along with the README and a Packages.tar.gz which contains the missing Packages files in case people want to try and fix/work around the broken initial set of images. Do you want me to update this on cdimage.debian.org and mirrors? And where is the README? /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 3.0r2.01 Update CDs?
On Sun, 15 Feb 2004, Jan Kesten wrote: Hi all! After my rsync today I noticed that there were some new jigdo-Templates for the update CDs. Looked around a bit but I didn't find whats the difference? http://cdimage.debian.org/debian-cd/jigdo/3.0_r2/README.update And they aren't exactly new, I just forgot to put them (the 3.2 update cds) up on cdimage.debian.org until this evening, mirrors seem to be catching up now. I got reminded by a fellow mirror admin today, I'll be looking into procedures for not forgetting (or having the publishing part available to Steve McIntyre too (interested?)). /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Mirror updates
From the push-trigger jigdo+iso mirror update project, here are the push-triggered mirrors that should be on the mirror list: Primary: http://cdimage.debian.org/debian-cd/ Push-triggered ones: http://ftp.se.debian.org/debian-cd/ http://ftp.fi.debian.org/debian-cd/ http://ftp.de.debian.org/debian-cd/ http://ftp.nl.debian.org/debian-cd/ http://ftp.eu.uu.net/debian-cd/ http://ftp.leo.org/debian-cd/ http://debian.lcs.mit.edu/pub/debian-cd/ http://debian.essentkabel.com/debian-cd/ http://debian.oregonstate.edu/debian-cdimage/ http://ftp.nluug.nl/os/Linux/distr/debian-CD/ These all have plenty of bandwidth (even if they might be occasionally slow due to load of course), and even if most of them are in Europe (especially .NL), there are a couple in north america and should be plenty of bandwidth available for users in total. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mirror updates
On Wed, 18 Feb 2004, jason andrade wrote: On Wed, 18 Feb 2004, Mattias Wadenstein wrote: From the push-trigger jigdo+iso mirror update project, here are the push-triggered mirrors that should be on the mirror list: Primary: http://cdimage.debian.org/debian-cd/ Push-triggered ones: http://ftp.se.debian.org/debian-cd/ http://ftp.fi.debian.org/debian-cd/ http://ftp.de.debian.org/debian-cd/ http://ftp.nl.debian.org/debian-cd/ http://ftp.eu.uu.net/debian-cd/ http://ftp.leo.org/debian-cd/ http://debian.lcs.mit.edu/pub/debian-cd/ http://debian.essentkabel.com/debian-cd/ http://debian.oregonstate.edu/debian-cdimage/ http://ftp.nluug.nl/os/Linux/distr/debian-CD/ how do i get ftp.au.debian.org added to this list ? Scripts, keys, etc: http://www.acc.umu.se/~maswan/debian-push/cdimage/ [*] It is a multiple-stage jigdo mirror thingie, all wrapped up in a script. Then contact me for an ssh trigger setup, test and addition to the cd-runmirrors script on farbror.acc.umu.se aka cdimage.debian.org. [*]: For existing users, there are some small updates. Mostly cosmetic though (an empty MD5SUMS-UPDATE/ etc directory turns up in images/current/ during the jigdo-mirror run, it is then removed). /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: 2.2 iso of jigdo files still available?
On Tue, 24 Feb 2004, Steve McIntyre wrote: On Tue, Feb 24, 2004 at 04:42:27PM +, Wookey wrote: We still get people who want to buy 2.2 arm/source CD images from time to time. For reasons I won't bore you with I've not got any 2.2r anything source images right now, so I went to download them again and failed to find any jigdo files or isos. Has anyone still got some I could use? Presumably jigdo will fail miserably as many of the original files will be gone from the archive now, and my archive image is somewhat out of date too, so I need isos at least to rsync against if not just download? In fact there are still some jigdo files for 2.2r7 (the last potato release) on open. I don't know if they're official or not at this point; they're not publically accessible atm. We removed them from cdimage.d.o when potato was removed from the archive, thinking that they wouldn't be very usable. Perhaps a snapshot of isos somewhere would be nice too, there might be one on planetmirror? Jason? Of course, given a decent storage donation, I could set an archive section up on cdimage.d.o. :) /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bittorrents of current release available
I created some. This is a beta location, do we want this included in the real debian-cd directory tree? http://cdimage.debian.org/pub/test/deb-bt/debian-cd/torrents/3.0_r2/ The service is currently in beta stage, so please test it and tell me about any strange stuff. I'll be adding startup scripts etc so that it will start at boot without manual intervention, then it might be ready for the mirror pages if it works otherwise. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
