Re: [gentoo-user] Calculating dependencies...: Any way to make it faster?
Neil Bothwick wrote: On Sat, 24 Jan 2015 07:03:30 +0100, Tomas Mozes wrote: Binary packages: http://thread.gmane.org/gmane.linux.gentoo.devel/94176 In my experience, using the -k or -K option makes emerge take longer to calculate the package list, which makes sense as it also needs to check the availability of packages. You could try a lower --backtrack value, but the time saved may be more than lost again when it causes problems. Setting the backtrack to a lower value was all I could think of too. Other than that, I think it just needs horsepower under the hood. This topic reminds me of when I was installing Gentoo on a 133 MHz machine with around 256MBs of ram. It's been a while but still, it was slow. Dale :-) :-)
Re: [gentoo-user] Calculating dependencies...: Any way to make it faster?
Am 24.01.2015 um 05:20 schrieb meino.cra...@gmx.de: Is there any way to make it faster or (in other words): Are there different ways to Calculating dependencies... and have only chossen the slowest one...? What can I do to spped it up? Portage is written in Python, normally running on CPython. While CPython is the standard, it isn't the fastest way to run Python. You could try switching over to PyPy, which uses a JIT-compiler that CPython doesn't have. This should get quite a big performance boost, if portage is being able to run under PyPy, that is. Alternatively you could try a portage replacement like Paludis, which is being written completely in C++.
Re: [gentoo-user] Calculating dependencies...: Any way to make it faster?
If the bottleneck is reading the information from disk you might upgrade the SD card or use a USB drive instead, which may have better random access performance. You could also store the portage tree on another machine with faster storage and access it over the network. If the bottleneck is actually calculating the dependencies, you are probably out of luck for the immediate future. For calculating dependencies I suspect the larger bottleneck is reading everything from disk, seeing as the machines seem fast enough you didn't complain too much about actually compiling. In either case you should try to revisit distcc and cross compiling as that is the only reliable way to speed everything up. You do not necessarily need to use distcc with a cross compiler (the configuration most likely to cause problems, though the wiki does address this).
Re: [gentoo-user] Calculating dependencies...: Any way to make it faster?
On Saturday 24 January 2015 06:56:16 meino.cra...@gmx.de wrote: I experimented with kinds of not compiling it natively like distcc, crosscompiling and such. May be me of may be a problem with the tools/ the environment/the setup or whatever: The results were corrupted systems every time. This costed me even more time than waiting for Calculationg dpeendencies So I am back to natively compiling that stuff... Have you tried exporting your packages directory over NFS to a chroot on another box? I do this for my little Atom box and it works a treat; I know that at some others here do the same. You just have to make sure that /etc/portage/... files are identical between the two machines - apart from those few things you actually want to differ, like MAKEOPTS and buildpkg. Before discovering this method I tried distcc and cross-compiling and had nothing but pain, but the nfs-packages method is straightforward and easily understood. -- Rgds Peter.
Re: [gentoo-user] Calculating dependencies...: Any way to make it faster?
On Sat, 24 Jan 2015 07:03:30 +0100, Tomas Mozes wrote: Binary packages: http://thread.gmane.org/gmane.linux.gentoo.devel/94176 In my experience, using the -k or -K option makes emerge take longer to calculate the package list, which makes sense as it also needs to check the availability of packages. You could try a lower --backtrack value, but the time saved may be more than lost again when it causes problems. -- Neil Bothwick Hors d'oeuvres: 3 sandwiches cut into 40 pieces. pgp9OXirksYKp.pgp Description: OpenPGP digital signature
[gentoo-user] Calculating dependencies...: Any way to make it faster?
Hi, for my embedded systems I use gentoo. Their harddisks are simple microSDcards. When updateing or emerging especially the Calculation dependencies... is a step which needs a lot of patience of the user (me ;). Is there any way to make it faster or (in other words): Are there different ways to Calculating dependencies... and have only chossen the slowest one...? What can I do to spped it up? Best regards, Meino PS: This is ***NO*** complain against Gentoo, the emerge-process or any other implizit or explizit critism!!! I *love* Gentoo -- thats why I am using it even for my embedded systems. PPS: Yes, I know of crosscompiling and other way to do the hard work on another machine. I screwed up my systems more than once in the past with my attempts to get that working. I want to natively compile on the embedded systems therefore.
Re: [gentoo-user] Calculating dependencies...: Any way to make it faster?
On 24/01/15 13:23, Daniel Frey wrote: On 01/23/2015 08:20 PM, meino.cra...@gmx.de wrote: Hi, for my embedded systems I use gentoo. Their harddisks are simple microSDcards. When updateing or emerging especially the Calculation dependencies... is a step which needs a lot of patience of the user (me ;). I have a QX9650 and it can be a few minutes on mine, especially on a world update. My slower CPUs (Celerons) can take more than five minutes, I don't even want to think about embedded. Is there any way to make it faster or (in other words): Are there different ways to Calculating dependencies... and have only chossen the slowest one...? I'd be interested as well to know as well. It used to be it did a simple dependency check and installed packages - then revdep-rebuild could check for packages that need rebuilds. It's not really an issue if you only run emerge once, but if you have to do it several times in one session it gets old really quick. It reminds me of waiting for Windows XP checking for updates. Just give it a half hour, it'll figure it out. :-( Dan distcc can make a big difference on slow machines where you have 3 or so hosts to throw jobs at. ccache in particular speeds up multiple passes at an emerge. Downside is a few packages cant use ccache and exhibit seemingly random failures to compile but if known they can be excluded using a portage setting. BillK
Re: [gentoo-user] Calculating dependencies...: Any way to make it faster?
Bill Kenworthy bi...@iinet.net.au [15-01-24 06:48]: On 24/01/15 13:23, Daniel Frey wrote: On 01/23/2015 08:20 PM, meino.cra...@gmx.de wrote: Hi, for my embedded systems I use gentoo. Their harddisks are simple microSDcards. When updateing or emerging especially the Calculation dependencies... is a step which needs a lot of patience of the user (me ;). I have a QX9650 and it can be a few minutes on mine, especially on a world update. My slower CPUs (Celerons) can take more than five minutes, I don't even want to think about embedded. Is there any way to make it faster or (in other words): Are there different ways to Calculating dependencies... and have only chossen the slowest one...? I'd be interested as well to know as well. It used to be it did a simple dependency check and installed packages - then revdep-rebuild could check for packages that need rebuilds. It's not really an issue if you only run emerge once, but if you have to do it several times in one session it gets old really quick. It reminds me of waiting for Windows XP checking for updates. Just give it a half hour, it'll figure it out. :-( Dan distcc can make a big difference on slow machines where you have 3 or so hosts to throw jobs at. ccache in particular speeds up multiple passes at an emerge. Downside is a few packages cant use ccache and exhibit seemingly random failures to compile but if known they can be excluded using a portage setting. BillK Hi Bill, thank you for your reply! :) I experimented with kinds of not compiling it natively like distcc, crosscompiling and such. May be me of may be a problem with the tools/ the environment/the setup or whatever: The results were corrupted systems every time. This costed me even more time than waiting for Calculationg dpeendencies So I am back to natively compiling that stuff... Best regards, Meino
Re: [gentoo-user] Calculating dependencies...: Any way to make it faster?
On 01/23/2015 08:20 PM, meino.cra...@gmx.de wrote: Hi, for my embedded systems I use gentoo. Their harddisks are simple microSDcards. When updateing or emerging especially the Calculation dependencies... is a step which needs a lot of patience of the user (me ;). I have a QX9650 and it can be a few minutes on mine, especially on a world update. My slower CPUs (Celerons) can take more than five minutes, I don't even want to think about embedded. Is there any way to make it faster or (in other words): Are there different ways to Calculating dependencies... and have only chossen the slowest one...? I'd be interested as well to know as well. It used to be it did a simple dependency check and installed packages - then revdep-rebuild could check for packages that need rebuilds. It's not really an issue if you only run emerge once, but if you have to do it several times in one session it gets old really quick. It reminds me of waiting for Windows XP checking for updates. Just give it a half hour, it'll figure it out. :-( Dan
Re: [gentoo-user] Calculating dependencies...: Any way to make it faster?
On 2015-01-24 05:20, meino.cra...@gmx.de wrote: Hi, for my embedded systems I use gentoo. Their harddisks are simple microSDcards. When updateing or emerging especially the Calculation dependencies... is a step which needs a lot of patience of the user (me ;). Is there any way to make it faster or (in other words): Are there different ways to Calculating dependencies... and have only chossen the slowest one...? What can I do to spped it up? Best regards, Meino PS: This is ***NO*** complain against Gentoo, the emerge-process or any other implizit or explizit critism!!! I *love* Gentoo -- thats why I am using it even for my embedded systems. PPS: Yes, I know of crosscompiling and other way to do the hard work on another machine. I screwed up my systems more than once in the past with my attempts to get that working. I want to natively compile on the embedded systems therefore. Recent change to make portage a bit faster (or to fail faster): https://bugs.gentoo.org/show_bug.cgi?id=536926 Binary packages: http://thread.gmane.org/gmane.linux.gentoo.devel/94176 As far as I know, there is no magical tuning that would speed up portage (it would be enabled by default I believe ;)).
