Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
On Sat, 2013-04-27 at 10:13 +0200, Laszlo Kajan wrote: Dear Ben! On 27/04/13 00:46, Ben Hutchings wrote: [...] However, I would expect the vast majority of installations to be on amd64, so if you always generate a 64-bit little-endian database and avoid duplicating when installing on such a machine then it would be better for most users (not so nice for others). (Incidentally, arch:all packages generating arch-specific data have interesting interactions with multi-arch. I doubt many people with multi-arch systems would want this package to generate multiple versions of the database, but you never know...) I see. According to [1], Arch:all with Multi-Arch:same is an error. [1] https://wiki.ubuntu.com/MultiarchSpec So at this point I see one way forward: 1: Move the postinst script into a new Arch:any package that depends on 'metastudent-data'. This Arch:any package would build the native database in postinst (with no multiarch support for now). What do you think? This might work, but be careful. Consider a multiarch system with armhf as primary and armel as additional architecture. Both architectures are little-endian, 32-bit. If you install metastudent-data-native/armel and metastudent-data-native/armhf, they should only generate one native copy of the data (right?). But if you remove one and leave the other, the native copy should stay. Ben. -- Ben Hutchings Knowledge is power. France is bacon. signature.asc Description: This is a digitally signed message part
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
Dear Ben! On 27/04/13 00:46, Ben Hutchings wrote: On Fri, Apr 26, 2013 at 03:21:43PM +0200, Laszlo Kajan wrote: Dear FTP Masters! On 23/04/13 15:13, Benjamin Drung wrote: [...] You can use xz for the source and binary package to reduce the size. The default compression level for xz reduces the size of the source tarball from 415 MB to 272 MB: $ ls -1s --si metastudent-data_1.0.0.tar* 823M metastudent-data_1.0.0.tar 381M metastudent-data_1.0.0.tar.bz2 415M metastudent-data_1.0.0.tar.gz 272M metastudent-data_1.0.0.tar.xz $ ls -1sh metastudent-data_1.0.0.tar* 784M metastudent-data_1.0.0.tar 363M metastudent-data_1.0.0.tar.bz2 396M metastudent-data_1.0.0.tar.gz 259M metastudent-data_1.0.0.tar.xz Following Benjamin's suggestion and the data.debian.org document [1], we have prepared a 'metastudent-data' arch:all package that is ~130MB (xz compressed). The package builds required architecture dependent databases in the postinst script. The purpose of this is to save space in the archive that each architecture dependent version would take up. [...] Does this mean that installing the package results in having two uncompressed copies of the data on disk? If so, wouldn't it be better to do: Indeed, the original arch:all version, and the native one. The arch:all version is not needed any more after conversion, and could be removed. Thanks for drawing my attention to this. 1. Compress the database (with xz). 2. Build the package without compression (contents are already compressed so re-compressing would be a waste of time). 3. In postinst, decompress and convert the database to native. However, I would expect the vast majority of installations to be on amd64, so if you always generate a 64-bit little-endian database and avoid duplicating when installing on such a machine then it would be better for most users (not so nice for others). (Incidentally, arch:all packages generating arch-specific data have interesting interactions with multi-arch. I doubt many people with multi-arch systems would want this package to generate multiple versions of the database, but you never know...) I see. According to [1], Arch:all with Multi-Arch:same is an error. [1] https://wiki.ubuntu.com/MultiarchSpec So at this point I see one way forward: 1: Move the postinst script into a new Arch:any package that depends on 'metastudent-data'. This Arch:any package would build the native database in postinst (with no multiarch support for now). What do you think? Best regards, Laszlo -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/517b88b9.8000...@rostlab.org
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
On Fri, Apr 26, 2013 at 11:46:44PM +0100, Ben Hutchings wrote: On Fri, Apr 26, 2013 at 03:21:43PM +0200, Laszlo Kajan wrote: On 23/04/13 15:13, Benjamin Drung wrote: [...] You can use xz for the source and binary package to reduce the size. The default compression level for xz reduces the size of the source tarball from 415 MB to 272 MB: Following Benjamin's suggestion and the data.debian.org document [1], we have prepared a 'metastudent-data' arch:all package that is ~130MB (xz compressed). The package builds required architecture dependent databases in the postinst script. The purpose of this is to save space in the archive that each architecture dependent version would take up. [...] Does this mean that installing the package results in having two uncompressed copies of the data on disk? If so, wouldn't it be better to do: 1. Compress the database (with xz). 2. Build the package without compression (contents are already compressed so re-compressing would be a waste of time). 3. In postinst, decompress and convert the database to native. If it's never going to be recompressed, you really want to compress it up the wazoo: | compression | decompression xz | size |amd64 | armhf | armhf -0 | 407076744 | 1:49.77 |6:14.47 | 1:23.31 -6 | 271088012 | 14:56.38 | 47:40.23 | 1:02.37 -9e | 195223672 | 19:38.15 | 1:06:50| 48.01 Far less space taken, _and_ it decompresses faster. However, I would expect the vast majority of installations to be on amd64, so if you always generate a 64-bit little-endian database and avoid duplicating when installing on such a machine then it would be better for most users (not so nice for others). Looks like we're getting 539375329859372 cores on one blade armhf machines, but you have a point. I find it quite strange why would the on-disk format ever care about word width, though: if the data fits in 32 bits, there's lots of waste for no gain -- mmap or not. -- ᛊᚨᚾᛁᛏᚣ᛫ᛁᛊ᛫ᚠᛟᚱ᛫ᚦᛖ᛫ᚹᛖᚨᚲ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130427140916.ga24...@angband.pl
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
Dear FTP Masters! On 23/04/13 15:13, Benjamin Drung wrote: [...] You can use xz for the source and binary package to reduce the size. The default compression level for xz reduces the size of the source tarball from 415 MB to 272 MB: $ ls -1s --si metastudent-data_1.0.0.tar* 823M metastudent-data_1.0.0.tar 381M metastudent-data_1.0.0.tar.bz2 415M metastudent-data_1.0.0.tar.gz 272M metastudent-data_1.0.0.tar.xz $ ls -1sh metastudent-data_1.0.0.tar* 784M metastudent-data_1.0.0.tar 363M metastudent-data_1.0.0.tar.bz2 396M metastudent-data_1.0.0.tar.gz 259M metastudent-data_1.0.0.tar.xz Following Benjamin's suggestion and the data.debian.org document [1], we have prepared a 'metastudent-data' arch:all package that is ~130MB (xz compressed). The package builds required architecture dependent databases in the postinst script. The purpose of this is to save space in the archive that each architecture dependent version would take up. The arch:all package is almost identical to the source package. * Please comment on this solution. If you like it, we will upload it (targeting the 'main' area), and have 'metastudent' (also in main) depend on it. [1] http://ftp-master.debian.org/wiki/projects/data/ Thank you for commenting. Best regards, Laszlo -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/517a7f67.3070...@rostlab.org
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
On Fri, Apr 26, 2013 at 03:21:43PM +0200, Laszlo Kajan wrote: Dear FTP Masters! On 23/04/13 15:13, Benjamin Drung wrote: [...] You can use xz for the source and binary package to reduce the size. The default compression level for xz reduces the size of the source tarball from 415 MB to 272 MB: $ ls -1s --si metastudent-data_1.0.0.tar* 823M metastudent-data_1.0.0.tar 381M metastudent-data_1.0.0.tar.bz2 415M metastudent-data_1.0.0.tar.gz 272M metastudent-data_1.0.0.tar.xz $ ls -1sh metastudent-data_1.0.0.tar* 784M metastudent-data_1.0.0.tar 363M metastudent-data_1.0.0.tar.bz2 396M metastudent-data_1.0.0.tar.gz 259M metastudent-data_1.0.0.tar.xz Following Benjamin's suggestion and the data.debian.org document [1], we have prepared a 'metastudent-data' arch:all package that is ~130MB (xz compressed). The package builds required architecture dependent databases in the postinst script. The purpose of this is to save space in the archive that each architecture dependent version would take up. [...] Does this mean that installing the package results in having two uncompressed copies of the data on disk? If so, wouldn't it be better to do: 1. Compress the database (with xz). 2. Build the package without compression (contents are already compressed so re-compressing would be a waste of time). 3. In postinst, decompress and convert the database to native. However, I would expect the vast majority of installations to be on amd64, so if you always generate a 64-bit little-endian database and avoid duplicating when installing on such a machine then it would be better for most users (not so nice for others). (Incidentally, arch:all packages generating arch-specific data have interesting interactions with multi-arch. I doubt many people with multi-arch systems would want this package to generate multiple versions of the database, but you never know...) Ben. -- Ben Hutchings We get into the habit of living before acquiring the habit of thinking. - Albert Camus -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130426224644.gf2...@decadent.org.uk
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
On 04/23/2013 11:48 AM, Laszlo Kajan wrote: Dear Russ, Debian Med Team, Charles! (Please keep Tobias Hamp in replies.) @Russ: Please allow me to include you in a discussion about a few bioinformatics packages that depend on big, but free data [2]. I have cited your opinion [3] in this discussion before. You are on the technical committee and on the policy team, so you, together with Charles, can help substantially here. [2] http://lists.alioth.debian.org/pipermail/debian-med-packaging/2013-April/thread.html [3] https://lists.debian.org/debian-vote/2013/03/msg00279.html This email is to continue the discussion about free packages that depend on big (e.g. 400MB) free data outside 'main'. These packages apparently violate policy 2.2.1 [0] for inclusion in 'main' because they require software outside the 'main' area to function. They do not violate point #1 of the social contract [1], which requires non-dependency on non-free components. For these big data packages, policy seems to be overly restrictive compared to the social contract, leading to seemingly unfounded rejection from 'main'. Indeed, many bioinformatics programs relies on external data. But I am afraid that if we start to add some data packages, we will open an endless open door BioInformatics datasets are large, and becoming huge and numerous. This size will be an issue for Debian mirrors (mainly if some indexed data are system dependent) but will also be a pain for the user if, when installing a program (to have a look), it downloads GBs of dependent packaged data. It may be really slow and fill the user disk (and I do not talk of package updates). Should not those data dependency clearly stated somewhere with the software package, with a script to get them ? Olivier [0] http://www.debian.org/doc/debian-policy/ch-archive.html [1] http://www.debian.org/social_contract * In case the social contract indeed allows such packages to be in 'main' (and policy is overly restrictive), how could it be ensured that the packages are accepted? * What is the procedure within Debian to elicit a decision about the handling of such packages in terms of archive area? Discussion on d-devel, followed by policy change? Asking the policy team to clarify policy for such packages? Technical committee? + Charles suggested such packages could go into 'main' [4], with a clear indication of the large data dependency of the package in the long description. When possible, providing the scripts for generating the large data as well. [4] http://lists.alioth.debian.org/pipermail/debian-med-packaging/2013-April/019292.html My goal as a Debian Developer and a packager is to get packages into Debian (so 'main') that are allowed in there, in reasonably short time. I would like to resolve this issue properly, because I believe it may pop up more often in bioinformatics software. For example, imagine a protein folding tool that would require a very large database to search for homologues for contact prediction, and using the contacts it would predict protein three-dimensional structure. This has been done before [5], and such a tool would be (is) immensely useful for bioinformatics. This tool would depend on gigabytes of data we would not package. Yet, by all means, I would want the tool to be part of the distribution. [5] http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0028766 Thank you for your opinion and advice. Best regards, Laszlo ___ Debian-med-packaging mailing list debian-med-packag...@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/debian-med-packaging -- Olivier Sallou IRISA / University of Rennes 1 Campus de Beaulieu, 35000 RENNES - FRANCE Tel: 02.99.84.71.95 gpg key id: 4096R/326D8438 (keyring.debian.org) Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51777990.60...@irisa.fr
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
Olivier Sallou olivier.sal...@irisa.fr writes: Indeed, many bioinformatics programs relies on external data. But I am afraid that if we start to add some data packages, we will open an endless open door BioInformatics datasets are large, and becoming huge and numerous. This size will be an issue for Debian mirrors (mainly if some indexed data are system dependent) but will also be a pain for the user if, when installing a program (to have a look), it downloads GBs of dependent packaged data. It may be really slow and fill the user disk (and I do not talk of package updates). Without having any solution, I'd like to mention that this is not a med specific problem. I am working on a number of packages for astrophysics that require/suggest up to some hundreds of megabytes calibration data http://www.eso.org/sci/software/pipelines/. Given that these packages are of quite specific interest, it makes IMO no sense to pollute all Debian mirrors with these files. Best regards Ole -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/ytzobd49zcp@news.ole.ath.cx
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
Hi Olivier! On 24/04/13 08:20, Olivier Sallou wrote: On 04/23/2013 11:48 AM, Laszlo Kajan wrote: Dear Russ, Debian Med Team, Charles! (Please keep Tobias Hamp in replies.) @Russ: Please allow me to include you in a discussion about a few bioinformatics packages that depend on big, but free data [2]. I have cited your opinion [3] in this discussion before. You are on the technical committee and on the policy team, so you, together with Charles, can help substantially here. [2] http://lists.alioth.debian.org/pipermail/debian-med-packaging/2013-April/thread.html [3] https://lists.debian.org/debian-vote/2013/03/msg00279.html This email is to continue the discussion about free packages that depend on big (e.g. 400MB) free data outside 'main'. These packages apparently violate policy 2.2.1 [0] for inclusion in 'main' because they require software outside the 'main' area to function. They do not violate point #1 of the social contract [1], which requires non-dependency on non-free components. For these big data packages, policy seems to be overly restrictive compared to the social contract, leading to seemingly unfounded rejection from 'main'. Indeed, many bioinformatics programs relies on external data. But I am afraid that if we start to add some data packages, we will open an endless open door BioInformatics datasets are large, and becoming huge and numerous. This size will be an issue for Debian mirrors (mainly if some indexed data are system dependent) but will also be a pain for the user if, when installing a program (to have a look), it downloads GBs of dependent packaged data. It may be really slow and fill the user disk (and I do not talk of package updates). Should not those data dependency clearly stated somewhere with the software package, with a script to get them ? Yes, the former (clearly state large external data dependency in the long package description) is exactly what Charles Plessy recommended. And your idea with the script to get the data is exactly what we implemented for this 'metastudent' package. So we clearly think along the same lines... Now we just have to discuss it with the FTP master team as well, so we see if this is acceptable for them (or they prefer to have the data in the archive). Laszlo -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/5177e18b.4010...@rostlab.org
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
Hello Didier! On 24/04/13 09:32, Didier 'OdyX' Raboud wrote: Le mardi, 23 avril 2013 12.23:23, Andreas Tille a écrit : I would even go that far that it might make sense to package these data and upload it to demonstrate that we should *really* create a solution for such cases if they will increase in the number and size of data packages. Isn't that what data.debian.org is supposed to be(come) ? * http://ftp-master.debian.org/wiki/projects/data/ * http://lists.debian.org/debian-devel/2010/09/msg00692.html Thanks for pointing this out, I didn't know about this. This would work very well for the 'metastudent' (and other of the same kind) data. A policy change (point 'We need to change policy.' of [1]) could be initiated, as Russ Allbery noted before [2]. But - data.debian.org does not (yet) exist, does it? [1] http://ftp-master.debian.org/wiki/projects/data/ [2] http://lists.alioth.debian.org/pipermail/debian-med-packaging/2013-April/019320.html Laszlo -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/5177e604.50...@rostlab.org
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
On 04/24/2013 04:02 PM, Laszlo Kajan wrote: Hello Didier! On 24/04/13 09:32, Didier 'OdyX' Raboud wrote: Le mardi, 23 avril 2013 12.23:23, Andreas Tille a écrit : I would even go that far that it might make sense to package these data and upload it to demonstrate that we should *really* create a solution for such cases if they will increase in the number and size of data packages. Isn't that what data.debian.org is supposed to be(come) ? * http://ftp-master.debian.org/wiki/projects/data/ * http://lists.debian.org/debian-devel/2010/09/msg00692.html Thanks for pointing this out, I didn't know about this. This would work very well for the 'metastudent' (and other of the same kind) data. A policy change (point 'We need to change policy.' of [1]) could be initiated, as Russ Allbery noted before [2]. But - data.debian.org does not (yet) exist, does it? sounds idea is quite old (2009) but did not progress. Could be the opportunity to relaunch it. [1] http://ftp-master.debian.org/wiki/projects/data/ [2] http://lists.alioth.debian.org/pipermail/debian-med-packaging/2013-April/019320.html Laszlo ___ Debian-med-packaging mailing list debian-med-packag...@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/debian-med-packaging -- Olivier Sallou IRISA / University of Rennes 1 Campus de Beaulieu, 35000 RENNES - FRANCE Tel: 02.99.84.71.95 gpg key id: 4096R/326D8438 (keyring.debian.org) Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/5177ec98.5040...@irisa.fr
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
Hello Andreas! On 23/04/13 12:23, Andreas Tille wrote: On Tue, Apr 23, 2013 at 11:48:05AM +0200, Laszlo Kajan wrote: This email is to continue the discussion about free packages that depend on big (e.g. 400MB) free data outside 'main'. In your practical case is this data say 500MB? Are we talking about compressed or uncompressed data (= 400MB on users harddisk or on all Debian mirrors world-wide)? It is around 404MB, gzip compressed [1]. I think it is not arch independent. I think BLAST databases (the main bulk in the tar.gz) are sensitive to the size of int, and endian-ness. [1] ftp://rostlab.org/metastudent/metastudent-data_1.0.0.tar.gz We do actually have examples of 500MB binary packages: udd@ullmann:/srv/mirrors/debian$ find . -type f -size +500M -name *.deb ./pool/main/f/freefoam/freefoam-dev-doc_0.1.0+dfsg-1_all.deb ./pool/main/libr/libreoffice/libreoffice-dbg_4.0.3~rc1-3_amd64.deb ./pool/main/libr/libreoffice/libreoffice-dbg_4.0.3~rc1-3_kfreebsd-amd64.deb ./pool/main/libr/libreoffice/libreoffice-dbg_4.0.3~rc1-2_amd64.deb ./pool/main/libr/libreoffice/libreoffice-dbg_4.0.3~rc1-2_kfreebsd-amd64.deb ./pool/main/n/ns3/ns3-doc_3.16+dfsg1-1_all.deb ./pool/main/n/ns3/ns3-doc_3.15+dfsg-1_all.deb ./pool/main/w/webkitgtk/libwebkit2gtk-3.0-0-dbg_1.11.91-1_amd64.deb ./pool/non-free/r/redeclipse-data/redeclipse-data_1.4-1_all.deb Even if the topic should be clarified in general because we will certainly have larger data sets than this in the future I could imagine that packaging this very data in your case should not be the main problem under the current circumstances as long there is no better solution found. I would even go that far that it might make sense to package these data and upload it to demonstrate that we should *really* create a solution for such cases if they will increase in the number and size of data packages. All right, we will package and upload the big data in case no one thinks of a better solution and discussion dies in, say, a week. Laszlo -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/517675d3.3030...@rostlab.org
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
Am Dienstag, den 23.04.2013, 13:51 +0200 schrieb Laszlo Kajan: Hello Andreas! On 23/04/13 12:23, Andreas Tille wrote: On Tue, Apr 23, 2013 at 11:48:05AM +0200, Laszlo Kajan wrote: This email is to continue the discussion about free packages that depend on big (e.g. 400MB) free data outside 'main'. In your practical case is this data say 500MB? Are we talking about compressed or uncompressed data (= 400MB on users harddisk or on all Debian mirrors world-wide)? It is around 404MB, gzip compressed [1]. I think it is not arch independent. I think BLAST databases (the main bulk in the tar.gz) are sensitive to the size of int, and endian-ness. [1] ftp://rostlab.org/metastudent/metastudent-data_1.0.0.tar.gz You can use xz for the source and binary package to reduce the size. The default compression level for xz reduces the size of the source tarball from 415 MB to 272 MB: $ ls -1s --si metastudent-data_1.0.0.tar* 823M metastudent-data_1.0.0.tar 381M metastudent-data_1.0.0.tar.bz2 415M metastudent-data_1.0.0.tar.gz 272M metastudent-data_1.0.0.tar.xz $ ls -1sh metastudent-data_1.0.0.tar* 784M metastudent-data_1.0.0.tar 363M metastudent-data_1.0.0.tar.bz2 396M metastudent-data_1.0.0.tar.gz 259M metastudent-data_1.0.0.tar.xz -- Benjamin Drung Debian Ubuntu Developer -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1366722785.3022.4.camel@deep-thought
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
Hello Benjamin! On 23/04/13 15:13, Benjamin Drung wrote: Am Dienstag, den 23.04.2013, 13:51 +0200 schrieb Laszlo Kajan: Hello Andreas! On 23/04/13 12:23, Andreas Tille wrote: On Tue, Apr 23, 2013 at 11:48:05AM +0200, Laszlo Kajan wrote: This email is to continue the discussion about free packages that depend on big (e.g. 400MB) free data outside 'main'. In your practical case is this data say 500MB? Are we talking about compressed or uncompressed data (= 400MB on users harddisk or on all Debian mirrors world-wide)? It is around 404MB, gzip compressed [1]. I think it is not arch independent. I think BLAST databases (the main bulk in the tar.gz) are sensitive to the size of int, and endian-ness. [1] ftp://rostlab.org/metastudent/metastudent-data_1.0.0.tar.gz You can use xz for the source and binary package to reduce the size. The default compression level for xz reduces the size of the source tarball from 415 MB to 272 MB: $ ls -1s --si metastudent-data_1.0.0.tar* 823M metastudent-data_1.0.0.tar 381M metastudent-data_1.0.0.tar.bz2 415M metastudent-data_1.0.0.tar.gz 272M metastudent-data_1.0.0.tar.xz $ ls -1sh metastudent-data_1.0.0.tar* 784M metastudent-data_1.0.0.tar 363M metastudent-data_1.0.0.tar.bz2 396M metastudent-data_1.0.0.tar.gz 259M metastudent-data_1.0.0.tar.xz Ah great! Thanks for checking this. A lesson for the future. We will switch to xz. Best regards, Laszlo -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/5176f4e6.2000...@rostlab.org
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
Dear Russ! Thank you for getting back to me. On 23/04/13 18:48, Russ Allbery wrote: Laszlo Kajan lka...@debian.org writes: This email is to continue the discussion about free packages that depend on big (e.g. 400MB) free data outside 'main'. These packages apparently violate policy 2.2.1 [0] for inclusion in 'main' because they require software outside the 'main' area to function. They do not violate point #1 of the social contract [1], which requires non-dependency on non-free components. For these big data packages, policy seems to be overly restrictive compared to the social contract, leading to seemingly unfounded rejection from 'main'. * In case the social contract indeed allows such packages to be in 'main' (and policy is overly restrictive), how could it be ensured that the packages are accepted? Yes, I agree. Although we should probably talk with ftp-master about whether they would like the data to just be packaged and uploaded as a regular package. Ftp-master was included in the initial thread [1], but they did not (yet) respond, and I started to feel that it may be impolite to flood their inbox with an issue like this. Since perhaps they alone can not decide about it. So yes, ftp-master is included in the mail once again. [1] http://lists.alioth.debian.org/pipermail/debian-med-packaging/2013-April/019282.html * What is the procedure within Debian to elicit a decision about the handling of such packages in terms of archive area? Discussion on d-devel, followed by policy change? Asking the policy team to clarify policy for such packages? Technical committee? Discussing it on debian-devel seems right, but I would also draw it to ftp-master's attention, since they're the people who have to worry about archive size). We can easily move on to modifying Policy if there's a consensus to let packages like that pull the data down from some external source. How to gauge that consensus? Laszlo -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/5176f5f5.5040...@debian.org
Re: [Debian-med-packaging] Question about proper archive area for packages that require big data for operation
Laszlo Kajan lka...@debian.org writes: On 23/04/13 18:48, Russ Allbery wrote: Discussing it on debian-devel seems right, but I would also draw it to ftp-master's attention, since they're the people who have to worry about archive size). We can easily move on to modifying Policy if there's a consensus to let packages like that pull the data down from some external source. How to gauge that consensus? Generally the way it works is that if no one objects to the idea, we bring it up on the Policy list, where we have a somewhat more formal process that involves seconds or objections. I think the ideal from a usability standpoint would be to just upload the data directly to the Debian archive, though. It's just a question of how big of packages we want to handle through the mirror network, or whether it's worth the effort to create a separate archive of huge data packages. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/878v487vrc@windlord.stanford.edu
