Re: [gentoo-dev] Adding large files to the mirrors?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hiya, First off, thanks everybody for the suggestions. It seems like people aren't keen to mirror the large tables on our mirrors, but no one's mentioned a reason for not doing so. Is there an infra reason behind that or did everyone just take my caution and run with it? On 16/10/13 02:40, Chí-Thanh Christopher Nguyễn wrote: I suggest that you add a large USE flag, and if it is enabled, download and install the whole thing. If disabled, just print the URLs in a log message, so the user can download themselves if they wish. So downloading them manually is a pain (the larger tables aren't in a single zip, they're split amongst 12 files for each table), and the ebuild to do the downloading is already built. I'll include a postinst note indicating that the tables are getting stored twice, and I should even be able to give them an indication of how much space they're wasting. I'll also provide a URL they can visit if they want to download manually instead. The question then becomes, do I split the ebuild into two (giving us three ebuilds for what is otherwise a tiny package) just so some SRCs are mirror-restricted and others aren't? All of that hinges on whether our servers can handle the extra size... Mike 5:) -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlJeWR4ACgkQu7rWomwgFXq8bQCfRde/Tg7sAirqT05d5spckC+s fUIAoIH1SlXvLmM3CqM0x1vQN0oPdiKi =qqsa -END PGP SIGNATURE-
Re: [gentoo-dev] Adding large files to the mirrors?
Il 16/10/2013 11:15, Mike Auty ha scritto: Hiya, [snip] So downloading them manually is a pain (the larger tables aren't in a single zip, they're split amongst 12 files for each table), and the ebuild to do the downloading is already built. I'll include a postinst note indicating that the tables are getting stored twice, and I should even be able to give them an indication of how much space they're wasting. I'll also provide a URL they can visit if they want to download manually instead. The question then becomes, do I split the ebuild into two (giving us three ebuilds for what is otherwise a tiny package) just so some SRCs are mirror-restricted and others aren't? All of that hinges on whether our servers can handle the extra size... Mike 5:) Add a bash script that can download all the files and mention it in postinst. the script will download all the files in current directory and put them in the correct place.
Re: [gentoo-dev] Adding large files to the mirrors?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hiya, On 16/10/13 12:54, Francesco R. wrote: Il 16/10/2013 11:15, Mike Auty ha scritto: Add a bash script that can download all the files and mention it in postinst. the script will download all the files in current directory and put them in the correct place. That's an absolutely awesome suggestion, I'll do that! Thanks! 5:) Mike 5:) -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlJfGK8ACgkQu7rWomwgFXrUIACfTTUcmqbApW4CqMlNd3YLC5yp pLwAn0hUHOEoG+noOMtgHqhBtKFb6tbT =E9tY -END PGP SIGNATURE-
[gentoo-dev] Adding large files to the mirrors?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi there, I'm updating the app-crypt/ophcrack-tables package to include the new tables available from their site. These are basically just additional data packages that can be useful with the app-crypt/ophcrack package, but they're very large. Including all of them will come just short of 30Gb. I don't know whether it's ok to add that to our mirrors, or add RESTRICT=mirror? Also, these take up a lot of space in distfiles. Does anyone know of a clever way to allow them to be installed without also stashing a copy in distfiles (symlinking to the distfiles directory is a no go, because each set needs its own files to have the same name as the other sets)? I guess it's a bit of an infra question, but thought I'd ask here in case anyone else has found themselves in a similar situation. For more specifics about what the package is like/for see the test ebuild at [1]... Mike 5:) [1] http://git.overlays.gentoo.org/gitweb/?p=dev/ikelos.git;a=blob;f=app-crypt/ophcrack-tables/ophcrack-tables-1.1.ebuild -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlJdzaQACgkQu7rWomwgFXqq1wCePSA0S03pVVxeesqAbd2faO4h 2wgAn0P0y5dvhsArRnFdFajrApM5C4YV =fim1 -END PGP SIGNATURE-
Re: [gentoo-dev] Adding large files to the mirrors?
Can't the users download and install those tables manually from the official ophcrack website/server? If so, maybe adding the instructions in the pkg_postinst() phase of app-crypt/ophcrack ebuild would be enough. In that case, it wouldn't make sense having app-crypt/ophcrack-tables in the portage tree and should be treecleaned. ** Vicente Olivert Riera Gentoo Linux Developer ID GnuPG: 5AE9E7B2E9BBCBA8 ** On 10/16/2013 01:20 AM, Mike Auty wrote: Hi there, I'm updating the app-crypt/ophcrack-tables package to include the new tables available from their site. These are basically just additional data packages that can be useful with the app-crypt/ophcrack package, but they're very large. Including all of them will come just short of 30Gb. I don't know whether it's ok to add that to our mirrors, or add RESTRICT=mirror? Also, these take up a lot of space in distfiles. Does anyone know of a clever way to allow them to be installed without also stashing a copy in distfiles (symlinking to the distfiles directory is a no go, because each set needs its own files to have the same name as the other sets)? I guess it's a bit of an infra question, but thought I'd ask here in case anyone else has found themselves in a similar situation. For more specifics about what the package is like/for see the test ebuild at [1]... Mike 5:) [1] http://git.overlays.gentoo.org/gitweb/?p=dev/ikelos.git;a=blob;f=app-crypt/ophcrack-tables/ophcrack-tables-1.1.ebuild signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Adding large files to the mirrors?
On 10/16/2013 07:20 AM, Mike Auty wrote: Hi there, I'm updating the app-crypt/ophcrack-tables package to include the new tables available from their site. These are basically just additional data packages that can be useful with the app-crypt/ophcrack package, but they're very large. Including all of them will come just short of 30Gb. I don't know whether it's ok to add that to our mirrors, or add RESTRICT=mirror? Add a ophcrack-tables-big package that is mirror-restricted (that way the rest of ophcrack-tables is unaffected) Pull it in with a useflag (big-tables ?) The double size problem with a copy in distfiles and livefs might be unavoidable - I have no good idea how to fix that
Re: [gentoo-dev] Adding large files to the mirrors?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/15/2013 07:29 PM, Vicente Olivert Riera wrote: Can't the users download and install those tables manually from the official ophcrack website/server? If so, maybe adding the instructions in the pkg_postinst() phase of app-crypt/ophcrack ebuild would be enough. In that case, it wouldn't make sense having app-crypt/ophcrack-tables in the portage tree and should be treecleaned. I think this may be the sanest option *for the large tables*. The small tables are great, and you can einfo a message about the large ones. - -Zero ** Vicente Olivert Riera Gentoo Linux Developer ID GnuPG: 5AE9E7B2E9BBCBA8 ** On 10/16/2013 01:20 AM, Mike Auty wrote: Hi there, I'm updating the app-crypt/ophcrack-tables package to include the new tables available from their site. These are basically just additional data packages that can be useful with the app-crypt/ophcrack package, but they're very large. Including all of them will come just short of 30Gb. I don't know whether it's ok to add that to our mirrors, or add RESTRICT=mirror? Also, these take up a lot of space in distfiles. Does anyone know of a clever way to allow them to be installed without also stashing a copy in distfiles (symlinking to the distfiles directory is a no go, because each set needs its own files to have the same name as the other sets)? I guess it's a bit of an infra question, but thought I'd ask here in case anyone else has found themselves in a similar situation. For more specifics about what the package is like/for see the test ebuild at [1]... Mike 5:) [1] http://git.overlays.gentoo.org/gitweb/?p=dev/ikelos.git;a=blob;f=app-crypt/ophcrack-tables/ophcrack-tables-1.1.ebuild -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSXertAAoJEKXdFCfdEflKICcP/2I+L61IWFk2Ackp+SqGgCHV NKuqCAFeoGywXIIgoFI/eWrOfHBQwyNKrNL6lRcBcYQXJy96yR/caRSPS+zPsEoU B4pJ13KJg0ipCriZUXHMtAOGcBOJRr40ItW0L3qT8ICl8TIFHeaDuqx5Sbl7GuHQ Ewqt5CdN4g7lpp5XROmz/LKHr3fHi+KNMPj7GrKd/guoALqmJvclmyWmmrovzr91 1ueSPghfSF78AHjEj8X7H7702/waLeOpww2MZkXfAY+H3Chw2AuTR0JC7xiSpMvX Dq4rtLUmdeobozBwNDkczoVGcfxoR+AguNlZUIz2Vij2A3XxVznqEYHY+VIbwZVm o5ut+P58GJBBct5TSi1IMdq2Rfrg7xP16e757Ki1FDSbQamaDhl8SETPbPQ0YrjV J9m8JQ2tlvgMi20Z+UA5QVWT9jZq8s8/26eDkeZNzYilmNZ8sSWUAlpNOgC0xsSG n4RN1COEVcpyxhyFjjMLB+GjxVa14fCgpZ+mcd/VwRmxU7scnw8YweWALamnAEbB FmcBB6tedeeA6TWcXtr3HGbfwt37KdHHo8xDJpJVl42dk08FKtV5sSY4vA2MDkL9 pyI4r4qo076KT9N4ZIsljPf3pGAYe3ERnIdNFgEci8UJBapoHh2GnLuTAN+EFjDt GcEiQLGWWMK9TuVL/9Jj =kOqk -END PGP SIGNATURE-
Re: [gentoo-dev] Adding large files to the mirrors?
Mike Auty schrieb: Hi there, I'm updating the app-crypt/ophcrack-tables package to include the new tables available from their site. These are basically just additional data packages that can be useful with the app-crypt/ophcrack package, but they're very large. Including all of them will come just short of 30Gb. I don't know whether it's ok to add that to our mirrors, or add RESTRICT=mirror? Currently they are on sourceforge mirrors AFAICT, so I expect that upstream will not mind if Gentoo users download from there. Also, these take up a lot of space in distfiles. Does anyone know of a clever way to allow them to be installed without also stashing a copy in distfiles (symlinking to the distfiles directory is a no go, because each set needs its own files to have the same name as the other sets)? I suggest that you add a large USE flag, and if it is enabled, download and install the whole thing. If disabled, just print the URLs in a log message, so the user can download themselves if they wish. Best regards, Chí-Thanh Christopher Nguyễn
