From: "Dustin Puryear" <[EMAIL PROTECTED]> Date: Sun, 29 Apr 2007 13:09:38 -0500
Hi Howard. On the slide "The Road Ahead..", you mention a scale-out need to "allow multi-terabyte DBs to be served without requiring a single giant server". Can you discuss what you mean a bit farther, especially in comparison to a distributed directory? I guess I don't see the specific advantage.. The only thing that really caught my attention on this slide was "cluster-friendly optimizations", which makes sense, but how powerful is this need really?
Hi Dustin. Good question. I think we're just acknowledging the fact that there are lots of folks who need large-scale directories, who have already invested in clustering setups. In the aggregate they have plenty of memory and CPU cycles available, but because it's not present as a single-system-image it's harder to make optimal use of it. In the cases where the site can deploy sufficiently large individual machines, clustering is totally a non-issue.
One problem with distributing large databases over separate servers is the continual need to rebalance the data distribution. I.e., the query load and the data volume load are almost never equivalent. As the database grows, the continual rebalancing gets to be a significant portion of the directory's runtime. Also if a participating server dies, all of the data needs to be redistributed again.
OpenLDAP already handles distribution using a variety of ways (back-ldap, back-meta, glue, etc...) and that's fine as a starting point, but it's pretty far from ideal. With clustering you can easily attach multiple terabytes of storage to a SAN and provide equally high speed disk access to all of the cluster nodes, but the normal BerkeleyDB doesn't support shared access in such an environment. (shared access to remote storage, remote access to shared locks, etc...)
Also from the performance aspect, distribution at the LDAP level more than doubles the response latency of all queries. Even with OpenLDAP we can only optimize away part of this latency; we can fully stream reads/decoding but BER prevents us from fully streaming writes/encoding.
The goal of the work I'm talking about is to allow multiple slapd processes to operate out of a single shared database, and distribute the query load across these servers with minimal overhead. There should be no visible overhead from cluster rebalancing due to server failures, and distributed data access should avoid redundant BER encode/decode passes between nodes.
It's always possible that hardware costs will come down enough to make this work a moot point. Now that AMD Opterons can address up to 256TB of RAM, and Sun has announced a line of Sparc servers that can also address up to 256TB of RAM, we may find that there are more affordable options for large SSI systems. I started laying out the design of this backend in early 2006 because the biggest system I could benchmark from AMD had well over a terabyte of RAM available, but only in a cluster configuration with only 32GB of RAM per node. Obviously the hardware landscape has changed since then.
-- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ --- You are currently subscribed to [EMAIL PROTECTED] as: [EMAIL PROTECTED] To unsubscribe send email to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the SUBJECT of the message.
