Hello, >From the feature list of urlgrabber, I thought maybe someone would be interested in Metalink. These features in particular:
# byte ranges - fetch only a portion of the file # reget - for a urlgrab, resume a partial download # batched downloads using threads - download multiple files simultaneously (feature still in progress) # retries - automatically retry a download if it fails. The number of retries and failure types are configurable # mirror groups - treat a list of mirrors as a single source, automatically switching mirrors if there is a failure Metalink lists mirrors, P2P, checksums, and other metadata in an XML format for programs that download. (There are five programs for making these Metalinks). There are also eight clients for Mac, Unix, and Windows. The clients can use the multiple URLs for higher reliability (in case a server goes offline, the transfer automatically uses another mirror) or segmented/multi-threaded downloads which are much faster. There can be full file, and segment/partial file checksums listed. The segment checksums allow segments with errors to be re-transmitted, instead of the whole file. This makes it useful for distribution of large files like ISO disk images. Downloads can be filtered by location, OS, or language, if that information is provided. >From using yum, it seems like it can already switch mirrors, but it doesn't do segmented downloads. You already have the byte range fetching, I guess you are avoiding segmented downloads. Metalink is used by OpenOffice.org, openSUSE, around ten Linux and BSD distributions, and other open source and commercial organizations that use mirrors and/or P2P. Here's what the format looks like: <?xml version="1.0" encoding="UTF-8"?> <metalink version="3.0" xmlns="http://www.metalinker.org/";> <files> <file name="FC-6-i386-DVD.iso"> <os>Linux-x86</os> <size>3525195776</size> <verification> <hash type="md5">89d337f5543474532027950b44ecbdd5</hash> </verification> <resources> <url type="http" location="gb" preference="10"> http://zeniiia.linux.org.uk/pub/distributions/fedora/linux/core/6/i386/iso/F C-6-i386-DVD.iso </url> <url type="ftp" location="ua" preference="10"> ftp://fedora.org.ua/pub/linux/fedora/core/6/i386/iso/FC-6-i386-DVD.iso </url> <url type="http" location="tr" preference="10"> http://mirrors.cs.hacettepe.edu.tr/fedora/linux/core/6/i386/iso/FC-6-i386-DV D.iso </url> </resources> </file> </files> </metalink> There's is more information available at http://en.wikipedia.org/wiki/Metalink http://www.linux.com/article.pl?sid=06/11/01/1641247 http://en.opensuse.org/Released_Version#Metalink (( Anthony Bryan )) Metalink [ http://www.metalinker.org ] _______________________________________________ Yum-devel mailing list [email protected] https://lists.dulug.duke.edu/mailman/listinfo/yum-devel
