Re: [gentoo-user] accelerate emerge
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
There is some examples in /etc/make.conf.example, but not for prozilla ...
If you are using the latest prozilla:
RESUMECOMMAND='/usr/local/bin/proz -r --no-curses --no-getch -s ${URI}
- -P ${DISTDIR} --min-size=2048'
This will invoke an ftp mirror search if the filesize if above 2MB (I do
this because the time taken for a 2MB is less than the search for
faster mirrors). Adjust to your likings ;-)
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2 (GNU/Linux)
iD8DBQFEA3KTCt0ZF9kLPvYRAgalAJsFip3T5V7XWRFNed2f0LH2rXUKCACgpBYV
hFsSjn1c/mHztaYOjDdidSw=
=A5u8
-END PGP SIGNATURE-
--
gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
On Monday 27 February 2006 10:39, El Nino wrote: if you want to use prozilla, just change the FETCHCOMMAND and RESUMECOMMAND in /etc/make.conf There is some examples in /etc/make.conf.example, but not for prozilla how can i get to know all these portage features? Read two lines above this one for the answer to your question. The other way is to read man portage, man emerge, man make.conf. The example file stays updated with new features (it may lag some) and so do the man pages. -- Zac Slade [EMAIL PROTECTED] ICQ:1415282 YM:krakrjak AIM:ttyp99 -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
On Mon, Feb 27, 2006 at 05:05:39PM +0100, Alexander Skwar wrote El Nino wrote: is there a way to accelerate the fetching part of emerge by using prozilla or some other tool? I never quite understood the sense in those tools. Why should prozilla or some other tool make the download be faster? When I download something with wget, or watch emerge invoking wget, it's always maxing out the saturation of the line. I have a different interpretation. Assume I'm doing an emerge --deep --update --world I've set my ADSL router-modem to log off after 15 minutes. The sequence of events is something like... 1) emerge wants to download a package, so it attempts to connect to a server, taking several seconds to wake up my ADSL router-modem 2) emerge downloads a big package, taking a few minutes to do so 3) emerge spends the next half hour building the big package, during which time the modem-router logs off 4) emerge finally finishes building the package, and wants to work on the next one... GOTO 1 I could run a short script #!/bin/bash emerge --deep --update --world --fetchonly emerge --deep --update --world ...but I'd like to get an emerge going on the 1st package as soon as it's finished downloading, whilst having the downloads of all the other packages continue in a separate thread. When the 1st build is finished, check whether the 2nd package has been downloaded. If not, wait. Then build the 2nd package... etc, etc. The best way to describe it is as a --fetchonly emerge that launches a separate emerge as each individual package is finished downloading. The build emerges should be serialized, i.e. only one build running at a time, because a package may depend on the immediately preceding package. -- Walter Dnes [EMAIL PROTECTED] In linux /sbin/init is Job #1 My musings on technology and security at http://tech_sec.blog.ca -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
On Tue, 2006-02-28 at 17:27 -0500, Walter Dnes wrote: I have a different interpretation. I assume you know about FEATURES=parallel-fetch? -- Iain Buchanan iain at netspace dot net dot au Elves and Dragons! I says to him. Cabbages and potatoes are better for you and me. -- J. R. R. Tolkien -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
On 2/28/06, Iain Buchanan [EMAIL PROTECTED] wrote: On Tue, 2006-02-28 at 17:27 -0500, Walter Dnes wrote: I have a different interpretation. I assume you know about FEATURES=parallel-fetch? It's probably not the best idea to recommend ~arch versions of portage. Mike -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
On Tuesday 28 February 2006 16:27, Walter Dnes [EMAIL PROTECTED] wrote about 'Re: [gentoo-user] accelerate emerge': #!/bin/bash emerge --deep --update --world --fetchonly emerge --deep --update --world ...but I'd like to get an emerge going on the 1st package as soon as it's finished downloading, whilst having the downloads of all the other packages continue in a separate thread. Try adding '' to your first line to start it as a background process. I did this for months. MAKE SURE emerge --info SHOWS THAT YOU ARE USING THE DISTLOCKS FEATURE. It's a separate process instead of a separate thread, but it should do what you want. You may also want to throw a ' /dev/null 2-' before the '', but that depends on your sensibilities. -- If there's one thing we've established over the years, it's that the vast majority of our users don't have the slightest clue what's best for them in terms of package stability. -- Gentoo Developer Ciaran McCreesh -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
On Tuesday 28 February 2006 17:25, Mike Owen [EMAIL PROTECTED] wrote about 'Re: [gentoo-user] accelerate emerge': On 2/28/06, Iain Buchanan [EMAIL PROTECTED] wrote: On Tue, 2006-02-28 at 17:27 -0500, Walter Dnes wrote: I have a different interpretation. I assume you know about FEATURES=parallel-fetch? It's probably not the best idea to recommend ~arch versions of portage. I will recommend going ~arch for portage, I've done so for over a year. That said, there's no need to go ~arch just for this feature. See my other post. -- If there's one thing we've established over the years, it's that the vast majority of our users don't have the slightest clue what's best for them in terms of package stability. -- Gentoo Developer Ciaran McCreesh -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
On Tue, 2006-02-28 at 15:25 -0800, Mike Owen wrote: On 2/28/06, Iain Buchanan [EMAIL PROTECTED] wrote: On Tue, 2006-02-28 at 17:27 -0500, Walter Dnes wrote: I have a different interpretation. I assume you know about FEATURES=parallel-fetch? It's probably not the best idea to recommend ~arch versions of portage. Why not? Its a great feature. My (two) systems have been running entirely ~x86 ever since I installed them (years ago), and I've never had any real problems. Only rarely do I have to recompile, or downgrade, but usually it's all handled by portage anyway. Standard disclaimer applies: just because it works now, doesn't mean it will work in the future; not for production / isolated servers, etc. Note that from the comments before you modify ~arch: ... '~arch' is a superset of 'arch' which includes the unstable, in testing, packages ... 'Broken' packages will not be added to testing ... IF YOU ARE UNSURE OF YOUR ARCH, OR THE IMPLICATIONS, DO NOT MODIFY [ACCEPT_KEYWORDS]. :) -- Iain Buchanan iain at netspace dot net dot au The use of money is all the advantage there is to having money. -- B. Franklin -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
El Nino wrote: AybOwan! is there a way to accelerate the fetching part of emerge by using prozilla or some other tool? Someone else answered that question directly, so I'll try the indirect approach. Another way to speed up emerges is to download while you compile: emerge -fq foo emerge foo Of course it only makes sense if 'foo' has at least one dependency, but if you use this command frequently it makes sense to alias it. Concurrent downloads used to be difficult, but now that Portage has locking it's reasonably safe. Still, every optimization comes with an increase in risk, and concurrent downloading is no different. Peace, Mike -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
Iain Buchanan wrote: On Tue, 2006-02-28 at 17:27 -0500, Walter Dnes wrote: I have a different interpretation. I assume you know about FEATURES=parallel-fetch? How long has this been around? I couldn't find any mention of it in the man files. Where is it documented? Thanks, Mike -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
It's new in the latest testing version of emerge. --- Vladimir On Tue, 2006-02-28 at 22:17 -0500, Michael A. Smith wrote: Iain Buchanan wrote: On Tue, 2006-02-28 at 17:27 -0500, Walter Dnes wrote: I have a different interpretation. I assume you know about FEATURES=parallel-fetch? How long has this been around? I couldn't find any mention of it in the man files. Where is it documented? Thanks, Mike -- Vladimir G. Ivanovic [EMAIL PROTECTED] -- gentoo-user@gentoo.org mailing list
[gentoo-user] accelerate emerge
AybOwan! is there a way to accelerate the fetching part of emerge by using prozilla or some other tool? -- ... The future lies ahead. ___ Have you mooed today? \^__^ \ (oo) \___ (__) \ )\/\ | |-w | | || | 2.6.15-gentoo-r1-sinhalese-jan201 (((o)))~--~--~-- Proud to be a Sinhalese. SINHALESE ARE GENIUSES OF IRRIGATION http://easyweb.easynet.co.uk/~sydney/sinhales.htm -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
2006/2/27, El Nino [EMAIL PROTECTED]: AybOwan! is there a way to accelerate the fetching part of emerge by using prozilla or some other tool? Hi, if you want to use prozilla, just change the FETCHCOMMAND and RESUMECOMMAND in /etc/make.conf There is some examples in /etc/make.conf.example, but not for prozilla ... HTH. Boris PS: With portage-2.1_pre* you can use the parallel fetch, just add parallel-fetch in the FEATURES variable ... ... The future lies ahead. ___ Have you mooed today? \^__^ \ (oo) \___ (__) \ )\/\ | |-w | | || | 2.6.15-gentoo-r1-sinhalese-jan201 (((o)))~--~--~-- Proud to be a Sinhalese. SINHALESE ARE GENIUSES OF IRRIGATION http://easyweb.easynet.co.uk/~sydney/sinhales.htm -- gentoo-user@gentoo.org mailing list -- Quiconque me parle de Dieu en veut à ma bourse ou à ma liberté. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
El Nino wrote: is there a way to accelerate the fetching part of emerge by using prozilla or some other tool? I never quite understood the sense in those tools. Why should prozilla or some other tool make the download be faster? When I download something with wget, or watch emerge invoking wget, it's always maxing out the saturation of the line. Alexander Skwar -- Doubt isn't the opposite of faith; it is an element of faith. -- Paul Tillich, German theologian. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
On 2/27/06, Boris Fersing [EMAIL PROTECTED] wrote: 2006/2/27, El Nino [EMAIL PROTECTED]: AybOwan! is there a way to accelerate the fetching part of emerge by using prozilla or some other tool? Hi, if you want to use prozilla, just change the FETCHCOMMAND and RESUMECOMMAND in /etc/make.conf There is some examples in /etc/make.conf.example, but not for prozilla ... HTH. Boris PS: With portage-2.1_pre* you can use the parallel fetch, just add parallel-fetch in the FEATURES variable ... how can i get to know all these portage features? ... The future lies ahead. ___ Have you mooed today? \^__^ \ (oo) \___ (__) \ )\/\ | |-w | | || | 2.6.15-gentoo-r1-sinhalese-jan201 (((o)))~--~--~-- Proud to be a Sinhalese. SINHALESE ARE GENIUSES OF IRRIGATION http://easyweb.easynet.co.uk/~sydney/sinhales.htm -- gentoo-user@gentoo.org mailing list -- Quiconque me parle de Dieu en veut à ma bourse ou à ma liberté. -- gentoo-user@gentoo.org mailing list -- ... The future lies ahead. ___ Have you mooed today? \^__^ \ (oo) \___ (__) \ )\/\ | |-w | | || | 2.6.15-gentoo-r1-sinhalese-jan201 (((o)))~--~--~-- Proud to be a Sinhalese. SINHALESE ARE GENIUSES OF IRRIGATION http://easyweb.easynet.co.uk/~sydney/sinhales.htm -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
On 2/27/06, Alexander Skwar [EMAIL PROTECTED] wrote: El Nino wrote: is there a way to accelerate the fetching part of emerge by using prozilla or some other tool? I never quite understood the sense in those tools. Why should prozilla or some other tool make the download be faster? When I download something with wget, or watch emerge invoking wget, it's always maxing out the saturation of the line. Well, if you have a high bandwidth line, like 8MB ADSL for example, the host providing the download usually limit your download speed reserving some band to other requests. If you could teach portage to fetch pieces (like some other tools do) from different mirrors till you saturate your line, your download would be considerably faster (all your band would be used). So, lets say I can use a program and tell it to use the GENTOO_MIRRORS variable to download various pieces of the same file (reaching the mirror servers limit and after a while your band limit), this would ensure you would always use the max band you can... I think I saw some tool over the net that could do that... Maybe wget itself /me go read the man pages... -- Daniel da Veiga Computer Operator - RS - Brazil -BEGIN GEEK CODE BLOCK- Version: 3.1 GCM/IT/P/O d-? s:- a? C++$ UBLA++ P+ L++ E--- W+++$ N o+ K- w O M- V- PS PE Y PGP- t+ 5 X+++ R+* tv b+ DI+++ D+ G+ e h+ r+ y++ --END GEEK CODE BLOCK-- -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
Hi, On Mon, 27 Feb 2006 17:05:39 +0100 Alexander Skwar [EMAIL PROTECTED] wrote: Why should prozilla or some other tool make the download be faster? When I download something with wget, or watch emerge invoking wget, it's always maxing out the saturation of the line. On my 1Gig line on my workstation at work it's usually _not_ saturizing the line. But I decided that it's not very polite to use a parallel fetching tool under these circumstances... -hwh -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] accelerate emerge
Hans-Werner Hilse wrote: Why should prozilla or some other tool make the download be faster? When I download something with wget, or watch emerge invoking wget, it's always maxing out the saturation of the line. On my 1Gig line on my workstation at work it's usually _not_ saturizing the line. But I decided that it's not very polite to use a parallel fetching tool under these circumstances... I would bet that has more to do with traffic shaping on your connection to the external world than anything to do with the local bandwidth, in which case you could probably parallel all you want w/o improving download performance. -- gentoo-user@gentoo.org mailing list
