Re: [gentoo-user] Re: Emerge Problems...
From: Peter Humphrey pe...@humphrey.ukfsn.org On Tuesday 01 February 2011 20:43:43 BRM wrote: And you're doing a typically manual process for updating all the systems - update your server first, then any rsync clients. Fine dandy if that is your process - but it's not mine. I may update my laptop twice as often as the other two, especially if I want to play with some software or try something out, or fix a bug, or get a later version of KDE. The server gets updated may be once a month, while the laptop is either once a month or at whim when I want something that just came out. It's not harder to do it this way, just a different method. The original rsync script worked perfectly fine; the broken update I did when I lost it is what started this whole thread. What's wrong with keeping your server's portage cache up to date? You don't have to update the server from it if you don't want to, but if the cache is out of date it isn't being much of a server. I recommend Occam's Razor. -- Here's the problem with the Server's /usr/portage being hosted by rsync: - Server sync's its portage against gentoo mirrors (emerge --sync) - Update Server (emerge world -vuDN) - Client sync's its portage against server portage mirror (emerge --sync) - Update Client (emerge world -vudN) So if you are manually updating the server, then no problem - you control the timing. Now all that seems to work fine until you introduce the automatic updates of the server's portage, e.g. via cron. Suppose the Server Update doesn't complete due to a build error. If the server automatically updated its portage during the build time then when you go to redo the build you may end up with another set of updates to push in, meanwhile you haven't finished the last round. Sure, the clients will still update just fine - it's not a problem for _them_, it's a problem for the server. So, Occam's Razor - store the rsync hosted portage mirror separately from the server's /usr/portage copy, and sync the server against the local rsync just like all the clients. The rsync hosted mirror can now be updated at will without any repercussions to any install, and the server works just like any of the clients; so now you end up with: - sync server portage mirror against gentoo mirrors at scheduled intervals, e.g. every day at midnight - Server sync's its portage against server portage mirror (emerge --sync) - Update Server (emerge world -vuDN) - Client sync's its portage against server portage mirror (emerge --sync) - Update Client (emerge world -vudN) The server is now completely 100% independent of the portage it is hosting for everyone else on the internal network, and you can get through a full update - resolving all issues, etc. - before any re-syncing. So then the question becomes why run the night cron to update the server's portage mirror? B/c I am not updating or installing software on my server as frequently as my other systems; so it doesn't need to be in sync itself as frequently. Ben
Re: [gentoo-user] Re: Emerge Problems...
On 15:41 Tue 01 Feb , Joshua Murphy wrote: The trick I've been using for... a couple years now, across various machines (no cron involved), is syncing one box that shares portage *and* my distfiles on nfs, portage R/O, distfiles R/W, then when it's done syncing and starts its own metadata update, hop across all the others and do an emerge --metadata. From the emerge man page: In versions of portage =2.1.5 the --metadata action is totally unnecessary unless the user has enabled FEATURES=metadata-trasfer in make.conf(5). Could mean that you can skip this --metadata step on your other machines? Greetings, Nils -- Nils Holland * Ti Systems, Wunsorf-Luthe (Germany) Powered by GNU/Linux since 1998
Re: [gentoo-user] Re: Emerge Problems...
- Original Message From: Dale rdalek1...@gmail.com Nils Holland wrote: On 21:35 Mon 31 Jan , Francesco Talamona wrote: On Monday 31 January 2011, BRM wrote: I just wrote a new script last night, but I'm still not sure that all of the parameters are correct Why not something proven and reliable like emerge --sync? In fact, what I always do is sync one of my machines with an official Gentoo mirror via emerge --sync, and then I just use rsync to distribute the updated tree to all my other local machines as in: rsync --delete -trmv /usr/portage/user@dest_host:/usr/portage One might want to ask rsync to exclude the distfiles directory, but I always include it as it oftentimes saves me the download of a file I've already downloaded during an emerge on another machine. In any case, locally updating my tree via rsync has always worked fine for me. Leaving the --delete option to rsync out, however, immediately leads to problems, with various ebuild-related error messages on subsequent emerges. I can imagine that the OP did, in fact, update his tree in such an inconsistent manner, but that can certainly be fixed, with the surest way being a emerge --sync using an official mirror. Definitely missed the delete option on the new script. Maybe I am missing something but I have two machines here. I sync to the Gentoo servers with the main rig and then sync the second rig from the main rig. All you have to do is start the rsync service and set the IP address in the SYNC line in make.conf on the second rig. This is my rsyncd.conf on the main rig: # Simple example for enabling your own local rsync server [gentoo-portage] path = /usr/portage comment = Gentoo Portage tree exclude = /distfiles /packages If you want to include distfiles, just remove it from the exclude line. For my distfiles, I run http-replicator to fetch those. It works pretty well. If the machine you are hosting portage on (via rsync) is fast enough to complete all updates within the update cycle (e.g. sync'ing 1 time a day, so it has 23:59:59 to complete all builds) then it is likely not a problem to do as that. If the machine is not fast enough - mine is a PII 233 w/160 MB RAM, takes a while do to updates - then you really have to separate out what you are hosting from what you are using. Otherwise you end up in the situation that you have started one system update (or software install), have a build failure for whatever reason, and then can't complete the same one due to changes in the local copy of portage. So, even if your system fell into the first situation - where it is fast enough - then I would still recommend doing the little extra to run as the second situation. It's just far easier to maintain. I'm actually surprised the Gentoo Mirror documentation doesn't recommend doing this to start with, but then again - the machine they recommend are magnitudes faster than what I'm running so it's not likely an issue. (Either that or everyone figures it out on their own and then just doesn't say anything.) Why? The local portage copy is always up-to-date, or reasonably so. No - I don't sync every 1/2 hour (like the official mirrors do), but I could force it to sync when I need to if that was an issue; typically once a day is sufficient and that's run by a cron job. But I also keep my server system relatively stable - I don't install a lot of software on it, and I don't necessarily update it frequently. So now I can update my laptop and desktop as well without having to first update the server itself since the rsync hosted portage is independent of the server. Ben
Re: [gentoo-user] Re: Emerge Problems...
On Tue, 1 Feb 2011 05:48:32 -0800 (PST), BRM wrote: If the machine is not fast enough - mine is a PII 233 w/160 MB RAM, takes a while do to updates - then you really have to separate out what you are hosting from what you are using. Otherwise you end up in the situation that you have started one system update (or software install), have a build failure for whatever reason, and then can't complete the same one due to changes in the local copy of portage. You can still use emerge -sync instead of a home brewed script. In make conf, set SYNC to localhost, then in your cron job, do SYNC=some gentoo rsync mirror emerge --sync So, even if your system fell into the first situation - where it is fast enough - then I would still recommend doing the little extra to run as the second situation. It's just far easier to maintain. I've been using a single portage tree to serve a LAN and for use by the host for years with no hint of any of the problems you suggest. I just make sure the cron job on the server syncs earlier than the rest of the LAN and everything is up to date. -- Neil Bothwick Politics: Poli (many) - tics (blood sucking parasites) signature.asc Description: PGP signature
Re: [gentoo-user] Re: Emerge Problems...
Neil Bothwick wrote: On Tue, 1 Feb 2011 05:48:32 -0800 (PST), BRM wrote: If the machine is not fast enough - mine is a PII 233 w/160 MB RAM, takes a while do to updates - then you really have to separate out what you are hosting from what you are using. Otherwise you end up in the situation that you have started one system update (or software install), have a build failure for whatever reason, and then can't complete the same one due to changes in the local copy of portage. You can still use emerge -sync instead of a home brewed script. In make conf, set SYNC to localhost, then in your cron job, do SYNC=some gentoo rsync mirror emerge --sync So, even if your system fell into the first situation - where it is fast enough - then I would still recommend doing the little extra to run as the second situation. It's just far easier to maintain. I've been using a single portage tree to serve a LAN and for use by the host for years with no hint of any of the problems you suggest. I just make sure the cron job on the server syncs earlier than the rest of the LAN and everything is up to date. I used to have four computers a good while back. Back then, I synced my main rig then synced the others off it. This was several years ago. I don't use a cron job or anything to do this, just some old fashioned typing. I don't recall ever having trouble with it syncing to my main rig. Did I mention it was a very old Compaq 200MHz CPU machine with a whopping 128MBs of ram? Thing looks like a filing cabinet. To me, it seems the OP is making something complicated when it is just not needed. If you want to use cron jobs, set the main rig to sync a hour before the others would be set to sync against it. If the rig that syncs to Gentoo servers is to slow, set them two hours apart. From my understanding, you get the same tree all the way around. Giving some more thought, I once put /usr/portage on nfs. I sync once and all the systems used the same copy of the tree. The other way worked out to be easier tho. I seem to recall the need for running emerge --metadata too. That took a while on the old Compaq. lol Dale :-) :-)
Re: [gentoo-user] Re: Emerge Problems...
On Tue, Feb 1, 2011 at 12:20 PM, Dale rdalek1...@gmail.com wrote: Neil Bothwick wrote: On Tue, 1 Feb 2011 05:48:32 -0800 (PST), BRM wrote: If the machine is not fast enough - mine is a PII 233 w/160 MB RAM, takes a while do to updates - then you really have to separate out what you are hosting from what you are using. Otherwise you end up in the situation that you have started one system update (or software install), have a build failure for whatever reason, and then can't complete the same one due to changes in the local copy of portage. You can still use emerge -sync instead of a home brewed script. In make conf, set SYNC to localhost, then in your cron job, do SYNC=some gentoo rsync mirror emerge --sync So, even if your system fell into the first situation - where it is fast enough - then I would still recommend doing the little extra to run as the second situation. It's just far easier to maintain. I've been using a single portage tree to serve a LAN and for use by the host for years with no hint of any of the problems you suggest. I just make sure the cron job on the server syncs earlier than the rest of the LAN and everything is up to date. I used to have four computers a good while back. Back then, I synced my main rig then synced the others off it. This was several years ago. I don't use a cron job or anything to do this, just some old fashioned typing. I don't recall ever having trouble with it syncing to my main rig. Did I mention it was a very old Compaq 200MHz CPU machine with a whopping 128MBs of ram? Thing looks like a filing cabinet. To me, it seems the OP is making something complicated when it is just not needed. If you want to use cron jobs, set the main rig to sync a hour before the others would be set to sync against it. If the rig that syncs to Gentoo servers is to slow, set them two hours apart. From my understanding, you get the same tree all the way around. Giving some more thought, I once put /usr/portage on nfs. I sync once and all the systems used the same copy of the tree. The other way worked out to be easier tho. I seem to recall the need for running emerge --metadata too. That took a while on the old Compaq. lol Dale :-) :-) The trick I've been using for... a couple years now, across various machines (no cron involved), is syncing one box that shares portage *and* my distfiles on nfs, portage R/O, distfiles R/W, then when it's done syncing and starts its own metadata update, hop across all the others and do an emerge --metadata. Once each one finishes, run through their individual updates. Because distfiles is shared, and portage's distfile locking is done right... I download each tarball of sources exactly once, even when 5-6 machines might share the same one. I've been quite pleased by that... even more handy is the shared git pull of wine that I build against on 3 different boxes (I tend to stagger those rebuilds though, haven't risked finding out if that would clash). -- Poison [BLX] Joshua M. Murphy
Re: [gentoo-user] Re: Emerge Problems...
- Original Message From: Dale rdalek1...@gmail.com To: gentoo-user@lists.gentoo.org Sent: Tue, February 1, 2011 12:20:56 PM Subject: Re: [gentoo-user] Re: Emerge Problems... Neil Bothwick wrote: On Tue, 1 Feb 2011 05:48:32 -0800 (PST), BRM wrote: If the machine is not fast enough - mine is a PII 233 w/160 MB RAM, takes a while do to updates - then you really have to separate out what you are hosting from what you are using. Otherwise you end up in the situation that you have started one system update (or software install), have a build failure for whatever reason, and then can't complete the same one due to changes in the local copy of portage. You can still use emerge -sync instead of a home brewed script. In make conf, set SYNC to localhost, then in your cron job, do SYNC=some gentoo rsync mirror emerge --sync So, even if your system fell into the first situation - where it is fast enough - then I would still recommend doing the little extra to run as the second situation. It's just far easier to maintain. I've been using a single portage tree to serve a LAN and for use by the host for years with no hint of any of the problems you suggest. I just make sure the cron job on the server syncs earlier than the rest of the LAN and everything is up to date. I used to have four computers a good while back. Back then, I synced my main rig then synced the others off it. This was several years ago. I don't use a cron job or anything to do this, just some old fashioned typing. I don't recall ever having trouble with it syncing to my main rig. Did I mention it was a very old Compaq 200MHz CPU machine with a whopping 128MBs of ram? Thing looks like a filing cabinet. To me, it seems the OP is making something complicated when it is just not needed. If you want to use cron jobs, set the main rig to sync a hour before the others would be set to sync against it. If the rig that syncs to Gentoo servers is to slow, set them two hours apart. From my understanding, you get the same tree all the way around. Giving some more thought, I once put /usr/portage on nfs. I sync once and all the systems used the same copy of the tree. The other way worked out to be easier tho. I seem to recall the need for running emerge --metadata too. That took a while on the old Compaq. lol And you're doing a typically manual process for updating all the systems - update your server first, then any rsync clients. Fine dandy if that is your process - but it's not mine. I may update my laptop twice as often as the other two, especially if I want to play with some software or try something out, or fix a bug, or get a later version of KDE. The server gets updated may be once a month, while the laptop is either once a month or at whim when I want something that just came out. It's not harder to do it this way, just a different method. The original rsync script worked perfectly fine; the broken update I did when I lost it is what started this whole thread. As the old saying goes - Different Strokes for Different Folks. Ben
Re: [gentoo-user] Re: Emerge Problems...
BRM wrote: And you're doing a typically manual process for updating all the systems - update your server first, then any rsync clients. Fine dandy if that is your process - but it's not mine. I may update my laptop twice as often as the other two, especially if I want to play with some software or try something out, or fix a bug, or get a later version of KDE. The server gets updated may be once a month, while the laptop is either once a month or at whim when I want something that just came out. It's not harder to do it this way, just a different method. The original rsync script worked perfectly fine; the broken update I did when I lost it is what started this whole thread. As the old saying goes - Different Strokes for Different Folks. Ben Again, maybe I am missing something but it doesn't really matter how often you update. Some people sync their main server and test packages, upgrade some stuff figure out a few workarounds then later on sync the other machines against the main server. The portage tree may be days old on the main server by that time but at least you know what you are up against if you are updating a LOT of systems. As you say tho, different strokes. Dale :-) :-)
Re: [gentoo-user] Re: Emerge Problems...
On 18:13 Mon 31 Jan , Dale wrote: Nils Holland wrote: In fact, what I always do is sync one of my machines with an official Gentoo mirror via emerge --sync, and then I just use rsync to distribute the updated tree to all my other local machines as in: rsync --delete -trmv /usr/portage/ user@dest_host:/usr/portage One might want to ask rsync to exclude the distfiles directory, but I always include it as it oftentimes saves me the download of a file I've already downloaded during an emerge on another machine. Maybe I am missing something but I have two machines here. I sync to the Gentoo servers with the main rig and then sync the second rig from the main rig. All you have to do is start the rsync service and set the IP address in the SYNC line in make.conf on the second rig. This is my rsyncd.conf on the main rig: [...] That actually makes sense, it would mean that rsyncd would have to be running and appropriately configured on the local master machine, and then it would provide the advantage of being able to sync all other local machines with the local master via a standard emerge --sync instead of a relatively long rsync command. Indeed, I guess I'll start doing this here as well. However, I have a server hosted at some hosting company as well, and I prefer not to sync it with an official Gentoo mirror, but with my local portage tree, in order to be sure that I have the exact same version of the portage tree on the server that I also use locally. For that case, NAT would prevent my server from contacting an rsync daemon on a local machine, so I'm actually using a locally invoked rsync to shove the tree to the server (vs. having the server fetch it). I guess that can't easily be changed, but it's not a problem anyway, as my current mode of operation works well. Greetings, Nils -- Nils Holland * Ti Systems, Wunsorf-Luthe (Germany) Powered by GNU/Linux since 1998
Re: [gentoo-user] Re: Emerge Problems...
On Monday 31 January 2011 22:26:20 BRM wrote: emerge --sync works fine for your _normal_ portage tree. But if you are running a mirror on a gentoo system that also needs its own copy of portage, then you really need to have two portage trees on the system. One portage tree is hosted by rsync for all - it can be synch'd at will with the official portage trees. The second portage tree is the system's portage tree, and is only sync'd when you update it - just like any other gentoo system. I don't understand any of this. Why should any two systems require different versions of the portage tree? -- Rgds Peter. Linux Counter 5290, 1994-04-23.
Re: [gentoo-user] Re: Emerge Problems...
On Tuesday 01 February 2011 20:43:43 BRM wrote: And you're doing a typically manual process for updating all the systems - update your server first, then any rsync clients. Fine dandy if that is your process - but it's not mine. I may update my laptop twice as often as the other two, especially if I want to play with some software or try something out, or fix a bug, or get a later version of KDE. The server gets updated may be once a month, while the laptop is either once a month or at whim when I want something that just came out. It's not harder to do it this way, just a different method. The original rsync script worked perfectly fine; the broken update I did when I lost it is what started this whole thread. What's wrong with keeping your server's portage cache up to date? You don't have to update the server from it if you don't want to, but if the cache is out of date it isn't being much of a server. I recommend Occam's Razor. -- Rgds Peter. Linux Counter 5290, 1994-04-23.
[gentoo-user] Re: Emerge Problems...
On Monday 31 January 2011, BRM wrote: I just wrote a new script last night, but I'm still not sure that all of the parameters are correct Why not something proven and reliable like emerge --sync? Ciao Francesco -- Linux Version 2.6.36-gentoo-r6, Compiled #2 SMP PREEMPT Mon Jan 3 11:54:58 CET 2011 Two 1GHz AMD Athlon 64 Processors, 4GB RAM, 4021.84 Bogomips Total aemaeth
Re: [gentoo-user] Re: Emerge Problems...
- Original Message From: Francesco Talamona francesco.talam...@know.eu On Monday 31 January 2011, BRM wrote: I just wrote a new script last night, but I'm still not sure that all of the parameters are correct Why not something proven and reliable like emerge --sync? emerge --sync works fine for your _normal_ portage tree. But if you are running a mirror on a gentoo system that also needs its own copy of portage, then you really need to have two portage trees on the system. One portage tree is hosted by rsync for all - it can be synch'd at will with the official portage trees. The second portage tree is the system's portage tree, and is only sync'd when you update it - just like any other gentoo system. Why? I originally ran the server with rsync hosting its portage tree, with daily synchronizations. However, when I forgot and let the server fall behind a little in updates, it became quickly clear that it needed its own separate copy of portage so I can install software without synchronizing portage - or rather, install software without having to update the whole system, etc. Now, may be there are options for emerge --sync that I'm not aware of to handle just this case - but it works very well, and I ran it for quite a while. Sadly, I did not have that script backed up or anything; so I will have to recreate it again. Ben
Re: [gentoo-user] Re: Emerge Problems...
On 21:35 Mon 31 Jan , Francesco Talamona wrote: On Monday 31 January 2011, BRM wrote: I just wrote a new script last night, but I'm still not sure that all of the parameters are correct Why not something proven and reliable like emerge --sync? In fact, what I always do is sync one of my machines with an official Gentoo mirror via emerge --sync, and then I just use rsync to distribute the updated tree to all my other local machines as in: rsync --delete -trmv /usr/portage/ user@dest_host:/usr/portage One might want to ask rsync to exclude the distfiles directory, but I always include it as it oftentimes saves me the download of a file I've already downloaded during an emerge on another machine. In any case, locally updating my tree via rsync has always worked fine for me. Leaving the --delete option to rsync out, however, immediately leads to problems, with various ebuild-related error messages on subsequent emerges. I can imagine that the OP did, in fact, update his tree in such an inconsistent manner, but that can certainly be fixed, with the surest way being a emerge --sync using an official mirror. Greetings, Nils -- Nils Holland * Ti Systems, Wunsorf-Luthe (Germany) Powered by GNU/Linux since 1998
Re: [gentoo-user] Re: Emerge Problems...
Nils Holland wrote: On 21:35 Mon 31 Jan , Francesco Talamona wrote: On Monday 31 January 2011, BRM wrote: I just wrote a new script last night, but I'm still not sure that all of the parameters are correct Why not something proven and reliable like emerge --sync? In fact, what I always do is sync one of my machines with an official Gentoo mirror via emerge --sync, and then I just use rsync to distribute the updated tree to all my other local machines as in: rsync --delete -trmv /usr/portage/user@dest_host:/usr/portage One might want to ask rsync to exclude the distfiles directory, but I always include it as it oftentimes saves me the download of a file I've already downloaded during an emerge on another machine. In any case, locally updating my tree via rsync has always worked fine for me. Leaving the --delete option to rsync out, however, immediately leads to problems, with various ebuild-related error messages on subsequent emerges. I can imagine that the OP did, in fact, update his tree in such an inconsistent manner, but that can certainly be fixed, with the surest way being a emerge --sync using an official mirror. Greetings, Nils Maybe I am missing something but I have two machines here. I sync to the Gentoo servers with the main rig and then sync the second rig from the main rig. All you have to do is start the rsync service and set the IP address in the SYNC line in make.conf on the second rig. This is my rsyncd.conf on the main rig: # Simple example for enabling your own local rsync server [gentoo-portage] path = /usr/portage comment = Gentoo Portage tree exclude = /distfiles /packages If you want to include distfiles, just remove it from the exclude line. For my distfiles, I run http-replicator to fetch those. It works pretty well. Dale :-) :-)
[gentoo-user] Re: Emerge Problems...
On Sunday 30 January 2011, BRM wrote: A little while back my server ran out of hard disk space (due to a failed hard drive) and as a result my local portage mirror got destroyed. Well, I fixed there server - initially by just grabbing a new copy of portage like a new install since it was just completely hosed, and the server is back up and working. However, now my desktop and laptop are both having problems. They sync just fine against the server, but I get a series of errors about not having various ebuilds in the manifest files - so many that I can't emerge anything (even portage). It seems your three systems share a broken portage tree, try with the latest portage snapshot, for example from http://distro.ibiblio.org/pub/linux/distributions/gentoo/snapshots/ You can also skip the sync and put it directly on the clients to see if the rsync service on server is broken... Once you stabilize the root cause, it's time to focus on the other issues (for example run a non-X runlevel on the laptop to fix the login issue, use nano until vim is ok, and so on). HTH Francesco -- Linux Version 2.6.36-gentoo-r6, Compiled #2 SMP PREEMPT Mon Jan 3 11:54:58 CET 2011 Two 1GHz AMD Athlon 64 Processors, 4GB RAM, 4021.84 Bogomips Total aemaeth
Re: [gentoo-user] Re: emerge problems
Will do, thanks for your assistance! On Sun, Jan 4, 2009 at 7:57 PM, Nikos Chantziaras rea...@arcor.de wrote: Nikos Chantziaras wrote: /etc/portage/packages.use: dev-java/sun-jdk -doc Typo. It's /etc/portage/package.use. You need to emerge -N world after that.
[gentoo-user] Re: emerge problems
James Stull wrote:
What is the best way to block this package?
On Sun, Jan 4, 2009 at 6:35 AM, pk pete...@coolmail.se wrote:
James Stull wrote:
I switched my eselect profile from a generic gentoo system to desktop.
Unfortunately I keep getting this error when I run emerge -uDNav world
I
tried it with the --skipfirst flag but I continue to have this problem.
* One or more packages are either masked or have missing dependencies:
*
* =dev-java/java-sdk-docs-1.6.0* pulled in by:
* ('installed', '/', 'dev-java/sun-jdk-1.6.0.11', 'nomerge')
*
Hi,
The docs are fetch-restricted; you need to download the documentation
from http://java.sun.com/javase/6/docs/; and follow the instructions in
the ebuild.
Don't use the doc USE flag of sun-jdk. That is, put this in
/etc/portage/packages.use:
dev-java/sun-jdk -doc
[gentoo-user] Re: emerge problems
Nikos Chantziaras wrote: /etc/portage/packages.use: dev-java/sun-jdk -doc Typo. It's /etc/portage/package.use. You need to emerge -N world after that.
