Re: [gentoo-dev] Gentoo Enterprise deployment tools
Omkhar Arasaratnam wrote: I think most of the assumptions that you're making involve giving your user population root access. Don't ?? The assumptions I am making are clearly not involving giving a user population root access. I just point to the lack of tools to maintain semi-frozen trees and to automate software updates on a large enterprise desktop deployment, and try to see if this gathers interest. I fail to see where I need to give wheel to user. My position would rather be that workstations don't need a portage tree or an emerge command. A software deployment server could push package installation on workstations and keep track of what's installed where and with which configuration files. -- Thierry Carrez (Koon) Gentoo Linux Security -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Gentoo Enterprise deployment tools
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Thierry Carrez wrote: Omkhar Arasaratnam wrote: I think most of the assumptions that you're making involve giving your user population root access. Don't ?? The assumptions I am making are clearly not involving giving a user population root access. I just point to the lack of tools to maintain semi-frozen trees and to automate software updates on a large enterprise desktop deployment, and try to see if this gathers interest. I fail to see where I need to give wheel to user. My position would rather be that workstations don't need a portage tree or an emerge command. A software deployment server could push package installation on workstations and keep track of what's installed where and with which configuration files. Would the semi-frozen tree you're looking for be embodied by GLEP 14? (I think it would for my needs.) Imagine if you could do something like this: # emerge --securityupdates world And you would get only the updates required by GLSA's that affect /your/ machine. http://www.gentoo.org/proj/en/glep/glep-0014.html This is similar to what Enterprises do with Windows installations, and I honestly think GLEP 14 could become portage's killer feature benefiting 'normal' users and enterprise users alike. The ability to push updates from a central location is, of course, a separate matter. :) Nathan -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCuVgR2QTTR4CNEQARAqr3AJ9YqU09o9ZzeAVOapVCu5DnfyXCJwCgnDT6 RzGQ8mNP9qxH6RePsnIpK6M= =a99r -END PGP SIGNATURE- -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Gentoo Enterprise deployment tools
M. Edward (Ed) Borasky wrote: Thierry Carrez wrote: Hi folks, I would like to get your opinion on Enterprise-oriented desktop deployment tools for Gentoo Linux (or the lack of). As a small company CIO, I deployed Gentoo on a small scale here but quickly ran into scaling problems and the lack of tools to help. I have deployed gentoo on openmosix for the reason of scale, where only one machine was the subject of configuration and point of access, while all neighbor nodes were less specific in configuration but all shared kernel. openmosix is more secure than plain old IP, given its custom marked packets (or something keen like that) and is well suited to LAMP which can occasionally expect a cgi process to take a dump or hang or freeze, upon the loss of a node holding its virtual process. open source databases have lousy replicatoin, or at least did when i was undertaking this. its unlikely the IO-bound processes will migrate, but the linked-list traversals deservedly take a hike(ie migrate) when the app design falls short. so the question of how to begin locking down packages, etc for HA, that's where staging and testing plans are of key importance. this is where my own syadmin handywork started with a cloning system: the mothership on one side, and a rescue disc, gig-e hub, and a nc/gnutar combo to clone and stage mutations/development. -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Gentoo Enterprise deployment tools
Thierry Carrez wrote: Hi folks, I would like to get your opinion on Enterprise-oriented desktop deployment tools for Gentoo Linux (or the lack of). As a small company CIO, I deployed Gentoo on a small scale here but quickly ran into scaling problems and the lack of tools to help. There is no obvious way to freeze a Portage tree (or to design a specific profile) for testing on a golden workstation, to build a set of update packages (ServicePack) and push it to the workstations, or to have centralized accountability of what's installed where. There is no easy way to avoid having to keep a synchronized copy of the portage tree on all systems, even when using yourown-binaries. Network mountable trees seem to work well enough although an emerge --metadata is still required on clients. I would disagree also on the profile argument. Profiles are very powerful, very details, and have decent manpages as well as literally tons of examples. What specifically is stopping you from rolling your own profile? The rest of that stuff is a generally well known about issue ( at least in the portage community ). Many features of portage-2.1 will be helpful in this type of situation. With automatic deployments, would we run into difficult-to-solve etc-update problems ? Should/could the ServicePack system take care of that ? I wouldn't use etc-update for this on a enterprise rollout personally. If you need config cfengine does a nice job, as well as using cvs/rcs/something-else Even in a simpler setup (preprod production) we don't have the tools to push a software configuration change from a test machine to a production one. What exactly are you looking for here? What tools are missing ? Is it our job to provide them ? Can it reasonably be done ? Am I just wrong to want to use Gentoo in that direction ? Portage needs work; I know the devs are working on it, I know there are other people who are doing there own things. I see a lot of portage-2.1 features that greatly simplify what you are trying to do ( repositories, config rewrite..etc.. ). I think portage and what it covers is a big part of this. Recollecting a conversation with jstubbs about portage he mentioned that he wouldn't want the portage-team to maintain a Enterprise-like distribution program, but that the new API would be great to write one against ;) I also know Chotchki was looking at doing his senior thesis on a network aware portage that did some cool things. A lot of this is just waiting ( and helping :) :) ) the portage devs get the work done that needs to get done. I know Cardoe and genstef? are working on a seperate package manager that just handles binaries but uses all the current portage stuff, so you might want to talk to them as well. Next week: Gentoo-as-a-metadistribution tools :) -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Gentoo Enterprise deployment tools
Alec Warner wrote: There is no obvious way to freeze a Portage tree (or to design a specific profile) for testing on a golden workstation, to build a set of update packages (ServicePack) and push it to the workstations, or to have centralized accountability of what's installed where. There is no easy way to avoid having to keep a synchronized copy of the portage tree on all systems, even when using yourown-binaries. Network mountable trees seem to work well enough although an emerge --metadata is still required on clients. I would disagree also on the profile argument. Profiles are very powerful, very details, and have decent manpages as well as literally tons of examples. What specifically is stopping you from rolling your own profile? The rest of that stuff is a generally well known about issue ( at least in the portage community ). Many features of portage-2.1 will be helpful in this type of situation. I probably wasn't clear :) I don't say that it cannot be done, and I don't ask what's the best way to do it. I just ask *if* we should try to provide higher-level tools (and/or doc) to help in doing so. It's not obvious (especially for non-developers) how to proceed in that situation, even if a lot of people have designed their own solution in their corner. With automatic deployments, would we run into difficult-to-solve etc-update problems ? Should/could the ServicePack system take care of that ? I wouldn't use etc-update for this on a enterprise rollout personally. If you need config cfengine does a nice job, as well as using cvs/rcs/something-else Again, the technology is out there, it's just not tightly integrated. Should we leave it as-is and let everyone design his own tools to connect the dots or should we ? Even in a simpler setup (preprod production) we don't have the tools to push a software configuration change from a test machine to a production one. What exactly are you looking for here? Should we provide high-level software that defines an update pack (new binaries + configuration changes), allows to test it on a preproduction system and (once tested) to push it to registered production systems ? Or let everyone write his own treefreezing + network mounts + shell scripts + cfengine / CVS magic combo to do it ? Portage needs work; I know the devs are working on it, I know there are other people who are doing there own things. I see a lot of portage-2.1 features that greatly simplify what you are trying to do ( repositories, config rewrite..etc.. ). I think portage and what it covers is a big part of this. Recollecting a conversation with jstubbs about portage he mentioned that he wouldn't want the portage-team to maintain a Enterprise-like distribution program, but that the new API would be great to write one against ;) I don't think it should be the role of the portage-team either. I also know Chotchki was looking at doing his senior thesis on a network aware portage that did some cool things. A lot of this is just waiting ( and helping :) :) ) the portage devs get the work done that needs to get done. I know Cardoe and genstef? are working on a seperate package manager that just handles binaries but uses all the current portage stuff, so you might want to talk to them as well. I sure hope they will comment on that thread :) -- Koon -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Gentoo Enterprise deployment tools
Thierry Carrez wrote: Hi folks, I would like to get your opinion on Enterprise-oriented desktop deployment tools for Gentoo Linux (or the lack of). As a small company CIO, I deployed Gentoo on a small scale here but quickly ran into scaling problems and the lack of tools to help. There is no obvious way to freeze a Portage tree (or to design a specific profile) for testing on a golden workstation, Why not? Just don't update the portage tree to build a set of update packages (ServicePack) and push it to the workstations, or to have centralized accountability of what's installed where. set make.conf to point to a static tree stored somewhere which you update as time permits. You could even create an /etc/init.d script that would check for updates (emerge --sync) against your QA'ed frozen tree at each boot. There is no easy way to avoid having to keep a synchronized copy of the portage tree on all systems, even when using yourown-binaries. see above With automatic deployments, would we run into difficult-to-solve etc-update problems ? Should/could the ServicePack system take care of that ? etc-update would continue to be a problem Even in a simpler setup (preprod production) we don't have the tools to push a software configuration change from a test machine to a production one. What tools are missing ? Is it our job to provide them ? Can it reasonably be done ? Am I just wrong to want to use Gentoo in that direction ? Next week: Gentoo-as-a-metadistribution tools :) I think most of the assumptions that you're making involve giving your user population root access. Don't Lock down your make.conf and only roll up update when you deem is neccassary using your internal tree. etc-update may still be an issue -- Omkhar Arasaratnam - Gentoo PPC64 Developer [EMAIL PROTECTED] - http://dev.gentoo.org/~omkhar Gentoo Linux / PPC64 Linux: http://ppc64.gentoo.org -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Gentoo Enterprise deployment tools
On Tue, Jun 21, 2005 at 08:35:52PM +0200, Thierry Carrez wrote: I don't say that it cannot be done, and I don't ask what's the best way to do it. I just ask *if* we should try to provide higher-level tools (and/or doc) to help in doing so. It's not obvious (especially for non-developers) how to proceed in that situation, even if a lot of people have designed their own solution in their corner. Best way to do it? Scary notion, not the way we're doing it currently. Push mode is preferable imo, 'cept no code exists to support that. Someone could write the necessary client/server code, but that would have issues when bound into existing portage apis... With automatic deployments, would we run into difficult-to-solve etc-update problems ? Should/could the ServicePack system take care of that ? I wouldn't use etc-update for this on a enterprise rollout personally. If you need config cfengine does a nice job, as well as using cvs/rcs/something-else Again, the technology is out there, it's just not tightly integrated. Should we leave it as-is and let everyone design his own tools to connect the dots or should we ? Not sure if the technology persay is out there honestly. If it were a cluster, cloned boxes, I'd say minimalize CONFIG_PROTECT, and (assuming you write the client/server cruft above) slip in config pkgs that get installed alongside... or, just jam the config changes into the pkg (not clean but it's possible). Or just trigger staggered reboot's on the boxes if you've got a fast network and pxe boot + imaging setup (I like the other method a bit more however :) If you're managing a half dozen servers, each server running it's own customized httpd.conf, I don't see an easy way to handle that (would love to hear any ideas people have on that one). Basically, kind of curious of how one could easily handle config management of multiple boxes, with config's potentially being wildly different from system to system (talking about a bit more then just /etc/conf.d/net.* and /etc/hostname differences here). I suspect just wrapping the config changes into a bingpkg, and sliding them out alongside on a push would suffice, but that's just one possible method. Even in a simpler setup (preprod production) we don't have the tools to push a software configuration change from a test machine to a production one. What exactly are you looking for here? Should we provide high-level software that defines an update pack (new binaries + configuration changes), allows to test it on a preproduction system and (once tested) to push it to registered production systems ? Or let everyone write his own treefreezing + network mounts + shell scripts + cfengine / CVS magic combo to do it ? How do you push it? I don't mean, what protocol/underlying, I'm asking how do you actually push _portage_ to do what you want? Either you try and abuse the craptastic api in stable to pull it off, or you probably resort to a catalyst akin trick of calling emerge via system. Neither is a proper solution. Api is required, further, preferably portage innards being designed such that you can swap in your own remote subsystem (whether cache tree or config) so it's a matter of pushing commands down the client/server pipes, with the portage config/installation on that box pulling what it needs (remote tree == having to pull all relevant files if building, binpkg is easier however). Portage needs work; I know the devs are working on it, I know there are other people who are doing there own things. I see a lot of portage-2.1 features that greatly simplify what you are trying to do ( repositories, config rewrite..etc.. ). I think portage and what it covers is a big part of this. Recollecting a conversation with jstubbs about portage he mentioned that he wouldn't want the portage-team to maintain a Enterprise-like distribution program, but that the new API would be great to write one against ;) I don't think it should be the role of the portage-team either. I draw a slightly finer line... portaged, some hypothetical client/server ap, not our business to implement, just provide an api for them to use. Thing is, if they're going remote, they'll either need to be able to trigger sync's on the boxes local tree (innefficient as all hell), or the tree is remote. If the tree is remote, that falls on portage devs head to provide a framework so the tree can be remote, in other words abstraction/framework design. Further... if you're pushing updates out, you need some method to query the vdb from the target- even if you're dealing with pushing updates down to a set of identical installations, you need to identify (easily/cleanly) what needs to be built, and what needs to be pushed down the line. Dancing around it, but you need access to the vdb for that system definition, which probably would be stored locally... in which case, the system targets
