Chris, I understand the temptation to write off unexpected information that conflicts with long-held beliefs. Of course, I take issue with your homophobia and blind vitriol, but the general impetus behind your objections is certainly a universal one.
> You are actually justifying �stressing� the registry? "Stress-testing" is a term from the systems world that refers to seeing how much simulated load a particular component, or chain of components, can handle while still providing adequate performance. The level of applied load in stress-testing is expected to be in excess of best-practices production conditions; it simply provides a far horizon, like ratings such as MTBF. > So now your love for the ease of registry programming has caused it > to come to life. I don't love or hate the Registry, and this certainly has nothing to do with "programming." I'm speaking of benchmarked performance. I know that it's markedly faster than SQL under any load I've ever given it. Scanning the rest of your e-mail, I note that you STILL can't find the time to run some benchmarks. I really think we'd have a much fairer debate if you had something to back you up. Seriously: is there something you can run to prove your points? > The registry is a simply binary tree that is memory mapped as > needed. This is as untrue as it was when you first said it. "The configuration manager doesn't access a hive's image on disk every time a Registry access occurs. Instead, it keeps a version of every hive in the kernel's memory. When a hive initializes, the Configuration Manager determines the size of the hive file, and reads the hive file into memory." (MSDN) > There is nothing *smart * about it. You also just said that the CPU > was conservative with the ODBC calls rather than the registry. The ability of a process to utilize all available CPU for brief periods without compromising pre-emption is not a sign of bad design. Yes, staying off the CPU a little bit is generally *better* design, but since ODBC also ran it up to 100% under load, there's no substantive design flaw. > ODBC is a client mechanism, but only to the extent of language > parsing and data transfer. It DOES NOT handle the computations > necessary for searching a large data store. Since ODBC does not > interact with anything other than SQL queries, the only impact on > other processes is the same general impact that any 2 programs have > running on the same box. The registry on the other hand requires all > of the search computations to occur on the client. This takes a toll > on the client. All of your theoretical backgrounders are very instructive, and generally well-reasoned. But they don't change the fact that ODBC and MDAC are much slower than Registry API. I don't see why you're pushing theory, when practice proves you wrong. > When designing software to be the most efficient and scalable, you > implement elements that specialize in the individual tasks needed. Well said. And if the data you need to store is domains, usernames, and passwords, the perfect way to implement is in a local database with blazing-fast indexing and memory management. Your comment about tree depth is off-topic; Imail never goes deeper than 7-8 levels, no matter how many users there are on the system. > I have to repeat this line. �And it clearly cannot handle the > concurrent calls to the ODBC and MDAC layers with any sort of > efficiency.� Umm� yea. We should send this line to Oracle and > Microsoft, because it obvious that databases are no longer > efficient. Cracking wise doesn't become you here, because your mocking conclusion is basically right. Surprise, SQL Server and Oracle are not always the most efficient ways to store and access data. SQL, especially with its proprietary extensions, is a language that offers unparalled data manipulation and query complexity. But have you ever used databases like VB/ISAM? Apollo? They all blow "enterprise" SQL databases out of the water, as does the Registry. Try one. Your jaw will drop. >Every load of the registry would possibly cause an immediate write to >virtual memory. Only if your machine is memory-starved, which means it isn't ready to be a mail server. >Since there is no guarantee that the entire registry can fit into >memory, the file calls were memory-mapped to avoid the possibly >overhead of constant copying of the same data. Memory-mapped >functions work in a similar fashion to virtual memory without the >copy. The function then assumes that the current location is virtual >memory and works from there. All of that off the top of your head? Cool. Anyway, when you get a chance, go to MSDN and see how it's really architected. Or even faster--do a registry API call and watch your HD access light. Oops! It was in RAM! > Ok. So let me get this straight. I am wishing that registry were a > hierarchical database? Are you saying it isn�t? Do you think that it > is a relational database? It is neither a relational database nor the hobbled hierarchical database you think it is. Of course, it is hierarchical. >What is interesting is that I misstated saying that the registry is >copied into memory and you state that this is �flatly untrue�. What >makes this odd is that just a few statements ago you argued that the >entire registry WAS loaded into memory. Which is it? There is no conflict between my statements whatsoever. You may want to read my quote from your last post again: >> So for every [n] user/process registry action you have [n] copies of >> the registry being loaded from the disk and being processed in >> memory. > > This is flatly untrue. Like I said, this statement is untrue--"[n] copies" and all that junk makes it untrue. > This is getting ugly. Please research the underlying file API used > to handle the file accesses. RegFlushKey() utilizes > FlushViewOfFile(LPCVOID lpBaseAddress, DWORD dwNumberOfBytesToFlush) > to handle the updating of changes. I don't care what deeper API the call uses to do its job; that's not germane. You claimed that a flush is always necessary after *every* API call. It most certainly is not. > What happened? You get your first copy of Windows ME and Visual > Basic and you think you have a grasp on Windows architecture? The > total size of the registry DOES affect every process that accesses > it. How can you possibly deny this. Sure, the total size of any object, on disk and/or in memory has some impact on its processing. But this is irrelevant to your performance hangups. (I have been using Windows since version 3.0, and began programming in ASM in 1984 for Apple. And you?) > So you are asking a high volume enterprise level application to > routinely use the Emergency Recovery Disk to maintain performance? No, I'm pointing out that another of your positions is misguided. You can back up the Registry from within the OS very simply. "Routinely"? No, because you'd only need to do that if you had a huge amount of user turnover. > Really, are you serious? SQL fragmentation absolutely does happen� > ON A SEPERATE SERVER THAN IMAIL, and is easily remedied by routine > scheduled database maintenance jobs. You can do the same thing with routine backups of the Registry, or via API calls. And again, if you can give me any benchmarks, please, ANY, that show that even a fragmented Registry can't compete with SQL Server, please send them along. > Y�know, SQL Server Agent? Here is a very interesting point. SQL > Server has mechanisms built in to preserve performance and > integrity. That's true, SQL Server is easier to maintain. And our discussion (didn't I say this in my last message?) is about using the Registry as a high-speed cache. Its administration hasn't ever been the issue! > The registry, since it was not designed to be an enterprise > database, does not. Are you aware that NT domains run from the Registry (SAM hive)? That's an enterprise application for you, though perhaps not one that means anything to you. But you're harping on the "enterprise" buzzword, when the Imail server doesn't *need* an enterprise database. It needs a desktop database that accepts multiple simultaneous connections and gives back data very, very quickly. The Registry is that database. > 1. You can back it up and restore it. You can't do that with the > registry, at least not from within the OS. You've never backed up a Registry file--of course, it can't be done, then. And yet MS, CA, Legato, UltraBac and Seagate seem to think it can be done. Funny, that. NTBACKUP.EXE, which is definitely "within the OS," can back up the Registry just fine. > 2. A database has a transaction back out scheme to ensure that after a > crash only complete series of related transactions are included in the > database. The registry is transactional, but I guess you didn't look hard enough to find that. It's not as flexible or user-controllable as SQL, certainly, but it does have rollback. > 3. Crucial database information may be stored redundantly. There is no > way to do this with the registry. Yes, that's true. But, um, we're talking about the Registry as a cache providing high-speed read access to SQL data, so...this is irrelevant. > 4. Wild apps can't access a database. Any app can access and modify > the registry. You were getting at this before, as well. I don't really see why, though. Are people running other apps on an Imail server that hit the registry thousands of times a day? Well, then they need this lesson from you. But I don't. If you want to get more specific on this point, as regards Imail, the ODBC connection string is freely available in the Registry, so if I were a black hat who wanted to *wanted* to hit Imail's database from a trojan or something, no problem. And even without Imail, if you support NT integrated authentication on SQL, any desktop app can abuse that with ease. > 5. Databases may be stored in separate files. The registry is one > giant file. If anything goes wrong with any of it, you lose it all. Actually, the Registry is several different files. But you might have to go to a machine to verify that, so don't go out of your way. > �Yes, we did lose our entire user base, but Sandy said it would be > fast.� I guess you can write a book while collecting unemployment. Good one, that was rich. With Registry backups, the chances of that happening are no greater than with SQL. > Which would you rather lose, an MBX file or your registry. I would rather lose a Registry that I had last night's backup of than an MBX file that was always open whenever I tried to back it up. > Ever had a corrupt registry? Not due to anything you're worried about. > Ever had a corrupt MBX? The two aren�t even comparable. That depends heavily on whose MBX you're talking about. I run MSCS with mirrored Registries on both servers. I've never lost a Registry setting. I've had to restore giant MBXs from disk after corruption (see a related thread about MBX size), though. > No. What I�m saying is that the registry can be used by any application > you install on the server. ANY application. This is without a doubt a > compromise of the integrity of your data store. If you run other apps that hit the Registry constantly, you don't know how to dedicate a system. > Imagine the scenario: You install a program to, let�s say, schedule > running the Emergency Repair Disk to defrag your registry. You don�t > like the program, so you try to uninstall it. Uh oh, the dufus who > wrote it didn�t thoroughly test the registry calls (it�s freeware, > what do you expect). Bye bye registry. Bye bye users. A chilling scenario. You really had me going. I don't install software in production without thoroughly testing it. And you don't need a third-party tool to back up the Registry, as pointed out above. > Oh. So the only way that the registry can become corrupt and > unusable is if there is file corruption. Hmmm� Wrong again. We are > not talking about a text file on Win98 here. You corrupt a node of > the hierarchy that affects the system, and your hosed. Remember, > this is an NTFS drive with a binary file, so you can�t just drop to > dos and edit the node. Sure, your thinking: �Well geez Chris, I have > an Emergency Repair Disk. I can just pop that in and I�ll be fine.�. > Oh really. > Ever tried to make a repair disk with a couple of hundred THOUSAND > users in your registry? Try it and benchmark that for me. I just did it. 100,000 users, 92 MB Registry. The backup took 2 minutes, 31 seconds. > If you can even complete it, the time to create it is completely > unmanageable and can no way be done in a reliable, routine fashion. I have an extra workstation you can borrow, since you seem to be low on testing equipment. > I was just trying to share some of the pitfalls associated with the > design decision you were about to make in regards to high volume > systems. And I thank you for your theoretical warnings. They led me to do more research and benchmarking. I conclude that the pitfalls aren't really there. > I think you have a great idea. It is sound in theory and I > truly wish you all the luck. Thanks, but it would be better if you would actually look at my stats and understand that I'm not defending the Registry just for the fun of it. I think that SQL and the Registry each have their place, whereas you (like many who are taken aback when they find that Imail uses the Registry by default) get your back up because you expect, both due to marketing and your own real experience, that the complexity and flexibility of enterprise RDBMS does not have any effect on their performance. > I just know from first hand experience the utter mistake the > registry as a database is for large systems. I've gotta guess that this first-hand experience must've been with a product other than Imail. Imail and the Registry are an extremely good match. > Sorry if you don�t agree. Aftre your first post, I was willing to believe that there's a break-even point and that after that point the Registry offers diminishing returns, but after doing the research, the facts don't back this up, so to agree would mean flying in the face of the evidence. And again, remember, we're dealing with Imail's particular querying needs and ODBC habits. > Put yourself in a scenario with a high volume system with a > gazillion active users using the registry and then get back to me. The highest-volume and highest-user-count system uses the Registry, per Len Conrad. Sandy Please visit http://www.ipswitch.com/support/mailing-lists.html to be removed from this list. An Archive of this list is available at: http://www.mail-archive.com/imail_forum%40list.ipswitch.com/ Please visit the Knowledge Base for answers to frequently asked questions: http://www.ipswitch.com/support/IMail/
