Hi, On Tue, 16 May 2006, David Bustos wrote:
> Quoth Milan Jurik on Tue, May 16, 2006 at 11:30:00PM +0200: >>> It's true that the repository is like the registry in that they are >>> both opaque. This project is not changing that, though. Like I said, >>> this project will in fact make it easier for the administrator to reduce >>> the cost of repository corruption. >> >> I'm still trying to understand how it reduce the cost of repository >> corruption. E.g. if repository (one file) will be corrupted then admin >> will lost every change for every service after the latest backup. > > If the customizations were not loaded from a profile, then it will not > reduce the cost. > >>> The API is a necessary part of the repository because it allows us to >>> add features under the covers, like this project. >> >> But API can cover also standard Unix way - files and filesystem structure. > > The standard Unix way is insufficient and not extensible. > I'm sorry, but you can cover text files with some API, if you need it (and it is done in many ways today). And check of consistency must be done in both situations, binary blob and text files are for API similar. >>>> And only for services now, but why not to extend to applications also. Add >>>> some rights management... >>> >>> Now that's starting to sound like the registry. We'll add functionality >>> as it's called for, but that's a separate discussion. >> >> Yes, but this project will start it. >> >> I understand why SMF needs its "temporary" binary blob. But it is not so >> big problem, in the situation when system is in some troubles you can >> bypass SMF and work with services manually. But if configuration will be >> in repository, you cannot bypass it so easily. > > Services critical to repository recovery must be manually startable. > Since this project only creates the infrastructure for services to move > their configuration into the repository, without actually doing so for > any services, it is up to those service authors to ensure that this > criterion is met. > Yes, I understand that this project will prepare only engine and the decision is on service authors. I'm not against some consistent API for config management. I'm against one big binary blob. >>>>>> In this >>>>>> case some SQLite files probably etc.? >>>>> >>>>> Like the rest of it, in-repository profiles will be kept in a database, >>>>> which is currently SQLite. Profiles will have external representations, >>>>> however. Most likely, this will be some sort of collection of manifest >>>>> files, which are XML. >>>> >>>> Yes, some default settings will be in XML files, I understand. But >>>> settings made by admins will be only in SQLite (or in XML file if exported >>>> by hand). >>> >>> Is your concern that there is only one copy of the customization, or >>> that it is not visible in a text editor? >> >> What is more admin friendly, bunch of svccfg commands or one text file? In >> my case, I like more to edit some /etc files than e.g. to run ldapclient >> command. > > I agree. But keeping configuration in editable text files of varying > syntax precludes transactionality, low-level consistency, and > system-wide configuration coordination, like profiles. If you know of > a way around this, please show us a prototype. > Every time you will load configuration from registry you should check consistency of loaded data. It is very similar to situation today. And system-wide configuration, what do you mean? E.g. nsswitch.conf, passwd, group, user_attr, resolv.conf, /var/ldap/ldap_cred_file (not editable :-/) etc. They are consistent on many, really many systems today. OK, with text files you don't have transactions. But do you need them today? Today you prepare everything and then you upload all config to service in one time. OK, with binary blob you can do many checks and you can hope that it is consistent later. But you shouldn't rely on it in this critical part. You should check that all properties are consistent when you are starting to use them. Every time. Like in text files. >>>>>> No text files for easy editing with >>>>>> vi-like editors? >>>>> >>>>> Outside of the repository, profiles will almost certainly be >>>>> human-editable. Inside the repository, they won't be, just like the >>>>> rest of the repository. >>>> >>>> But main store will be SQLite. Not temporary but main. >>> >>> We will not make the datastore an editable text file. That would >>> encourage services to bypass the API and prevent us from implementing >>> projects like this. >> >> What is the difference between: >> >> svccfg setprop someprop = value >> >> vs. >> >> vi /etc/some_directories/service.conf >> svcadm reload service >> >> OK, in the first situation you will set only one property, so service will >> not need to parse everything. But many times you will run svccfg more than >> one, it can lead up to more work for service, many repetitive things. > > This assertion is incorrect, since services only need to reread their > configuration on svcadm refresh, not each property change. > OK, does it mean that you will upload all config with svccfg and then use svcadm for reloading? No events during changes of properties with svccfg? > The larger issue of administrative ease-of-use is important to us, but > when it stands in the way of features like transactionality, snapshots, > roll-back, and profiles, which we believe also contribute to > administrative ease-of-use, we must forge ahead. You could argue that > one could implement snapshots and profiles with plain files, but doing > so would be enormously complex and would only introduce more points of > failure and misunderstanding into the system. > Yes, I know. I was working with many RCS for /etc during last years. I was even supervisor for one Master Thesis for RCS FUSE module. So that, I understand what you are trying to do. But I don't agree with the way. >>>> Can I make 'cp /etc/svc/repository.db backups' safely on running system? >>>> Probably not because svc.configd has O_RDWR for it. >>> >>> I think you can if you pstop svc.configd first. This does seem like >> >> Is it save to pstop it? > > Yes. Clients will hang in the meantime, but after startup, clients > rarely access the repository. > Now, but with properties in the repository you cannot rely on it. Many services could load their data from the repository later. You can hang some important service during it. >>> a deficiency, though. I've filed >>> >>> 6425993 Manual repository backups >>> >>> for you. >> >> OK, thank you (I added to interest list for this bug myself). Small side >> effect :-) But still how frequently will be done automatic backup, or that >> will be only during boot as today? > > More frequent automatic backups sounds like a reasonable request. Is > that what you're asking for? > Yes, if you want to have one big file and very different way of work, you should try to do more things automatically. The problem is "When?". I don't have answer here. >>>> And what about distribution of config "files" to other machines? With >>>> 'svccfg'? >>> >>> That's one of the goals of this project. You should be able to export >>> customizations you've made into a profile so that you can distribute it >>> to other machines. >> >> Some dedicated daemon/service or through XML files? > > Almost certainly XML files. > >>>> SMF is nice for service management. Not very different interface to init.d >>>> structure in reality and brings enhancements. After system troubles I know >>>> from my memory which services I want to start on that machine. But moving >>>> some "/etc file" configs to "through API accessible" blob? >>>> >>>> What will be benefits for admins/users/customers? >>> >>> 1. Each service doesn't have to write its own parser. >> >> There could be small library which can parse files with structure e.g. >> "name: value". No binary blob needed. Many present configs can be >> recycled. > > Ok, go code it. > Does it mean if I will show you code of the library which will save configs in directory/file structure in this way, will you start to think about the alternative way than one big binary blob? >>> 2. We can implement profiles without rewriting all services. >> >> Through some wrappers for every service, for every service different one. > > I'm not sure what you mean. > If I understood you well, you need to write some wrapper for every service which you don't want to modify but you want to use SMF for it. So that there is no code which could be used for it now. No argument for binary blob. >>> 3. We can implement a distributed repository without rewriting all >>> services. >> >> Distributed? You mean reimplementation of scp command? > > No. I mean that if a service requests a property which is not on the > local machine, it would fall through to some network machine, so you > could configure multiple machines from one store. > So that you will need to have some network daemon dedicated to this. >>> See also >>> http://blogs.sun.com/roller/page/lianep?entry=smf_repository_design_choices >>> . >> >> Interesting (I think I read it some months ago): >> >> 3. Single point of access. >> Single point of failure. I'm not against binary blobs. E.g. postfix is >> using them also. And sendmail has also its config in recomputed files. But >> both are only temporary, not main store. Main store is human readable, >> many files, not one big. > > Human-readable configuration files are also a single point of failure. > Like I said above, administrative convenience is valued, but it was > standing in the way of progress. > With human-readable config you can edit store in many ways. With binary blob you will rely on SMF and also on situation that all services will be correct and not corrupt properties which don't belong to them. >> 4. Access control. >> It is not in SMF now, I think. And this will lead to more MSWin registry >> way. > > Write access control is present. See smf_security(5). > My fault, I'm sorry. >> All others could be solved also in the files way, I think. > > How would you make text files transactional? Making only certain parts > of a configuration file changeable by certain people would require > breaking the file up. Once you did that, how would you take atomic > snapshots? How would you implement profiles which could change some > parts of the configuration without others? Wouldn't you have to put > each line into a separate file? > I cannot make text files transactional. But do I really need it? True, if you want to give rights to some users, you need to break the file up. Atomic snapshosts for configuration? How frequently will it change? You will change profiles 2-3 per day as typical maximum. Or are you changing config of you system more frequently? I'm doing "snapshosts" of many config file very often and really, they are consistent. Is it so hard to change only some lines in text files? > Certainly, you could find ways to do all of these things through the > traditional read(2)/write(2) interface, but the way we've chosen is > much, much simpler. > Yes, it is. For us, developers. But it is very strange for all others. Are we working for them or are they working for us? Still searching for benefits... >> Why should >> Solaris lost Unix "nearly everything is file" way? Who was discussing this >> with admins/customers? I tried it today with 2 my friends, both are >> working with Solaris quite a lot and are very knowledgeable and for them >> even SMF for services is strange. But they can accept it for its benefits. >> But repository of configs? >> I asked for admin's benefits. Not developer's. > > It's true that manipulating configuration in the repository isn't as > easy as it could be today. We're working on that (see, for example, > http://opensolaris.org/os/project/vpanels ). We think the benefits > resulting from the repository, present and future, justify the current > inconvenience. And the lack of loud complaints seems to support that. > I see. Simple Unix goes out. Binary repository, java configuration utils... OK, I also see that I'm the only one that isn't happy from it here. Maybe I meet wrong people, maybe admins aren't on this list, maybe not enough amount of admins switched to S10 already. And maybe I am wrong :-) > > David > Best regards, Milan Jurik
