LDAP scalability
Hi , I am evaluating LDAP in general (Apache DS as possible LDAP) for use as a user repository to be used for authentication /authorization for a large scale web application. The choice is RDBMS vs LDAP vs NoSQL. I am well aware that LDAP is used by large enterprises. These enterprises have typically thousands of users. But have not seen it referenced in large scale web application architectures - such as those at google , facebook, linkedin which deal with millions of users. ( They might be using ldap but I have not seen anything on the web that says they do) Can LDAP in general and Apache DS in particular scale to millions of users ? Are there any blogs/articles on web that talk of LDAP use in architectures of very large scale. thanks -- http://khangaonkar.blogspot.com/
Re: LDAP scalability
On 5/17/11 11:54 PM, Manoj Khangaonkar wrote: Hi , Hi, I am evaluating LDAP in general (Apache DS as possible LDAP) for use as a user repository to be used for authentication /authorization for a large scale web application. The choice is RDBMS vs LDAP vs NoSQL. It's not really a choice. LDAP is the only pristine solution when it comes to manage authentication and authorization. Using a RDBMS or a NoSQL system implies you build a authn/authz solution on top of it (I'm not talking about LDAP over a RDBMS) I am well aware that LDAP is used by large enterprises. These enterprises have typically thousands of users. Hundred of thousands, and I have seen big telco companies using LDAP for more than 70 000 000 users... But have not seen it referenced in large scale web application architectures - such as those at google , facebook, linkedin which deal with millions of users. ( They might be using ldap but I have not seen anything on the web that says they do) Probably because they don't necessarily want to expose such a sensitive part of their IT, but most certainly because they need a highly replicated system. Can LDAP in general and Apache DS in particular scale to millions of users ? Base line, yes. Dealing with millions of users is not really an issue. What is important here is not the number of users, but much more the operation per second you want to process on the LDAP server. On a laptop, OpenLDAP currently deal with up to 10 000 authentication *per second*, and with ApacheDS, last time I conducted a test (last year), it was around 4 500 authentication per second. Are there any blogs/articles on web that talk of LDAP use in architectures of very large scale. Not that I know of. But the next LDAP conference (in Germany, http://www.daasi.de/ldapcon2011/) might see some talks about such a thing. Hope it helps. thanks -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: LDAP scalability
Here's one for you guys. Blizzard, the entertainment guru's had the following appear on their website which is ample evidence to suggest they are using LDAP for their member infrastructure. http://us.battle.net/sc2/en/forum/topic/2325095821 What emmanual has said is correct, sensitive infrastructures aren't openly discussed. That reflects an infrastructure of users which includes all the players of Starcraft II, World of Warcraft (16.7 million users) So there you go, now in the multiples of millions and outside the hundreds of thousands with a subscription value of over $334 million USD. ;) On Wed, May 18, 2011 at 8:57 AM, Emmanuel Lecharny elecha...@gmail.comwrote: On 5/17/11 11:54 PM, Manoj Khangaonkar wrote: Hi , Hi, I am evaluating LDAP in general (Apache DS as possible LDAP) for use as a user repository to be used for authentication /authorization for a large scale web application. The choice is RDBMS vs LDAP vs NoSQL. It's not really a choice. LDAP is the only pristine solution when it comes to manage authentication and authorization. Using a RDBMS or a NoSQL system implies you build a authn/authz solution on top of it (I'm not talking about LDAP over a RDBMS) I am well aware that LDAP is used by large enterprises. These enterprises have typically thousands of users. Hundred of thousands, and I have seen big telco companies using LDAP for more than 70 000 000 users... But have not seen it referenced in large scale web application architectures - such as those at google , facebook, linkedin which deal with millions of users. ( They might be using ldap but I have not seen anything on the web that says they do) Probably because they don't necessarily want to expose such a sensitive part of their IT, but most certainly because they need a highly replicated system. Can LDAP in general and Apache DS in particular scale to millions of users ? Base line, yes. Dealing with millions of users is not really an issue. What is important here is not the number of users, but much more the operation per second you want to process on the LDAP server. On a laptop, OpenLDAP currently deal with up to 10 000 authentication *per second*, and with ApacheDS, last time I conducted a test (last year), it was around 4 500 authentication per second. Are there any blogs/articles on web that talk of LDAP use in architectures of very large scale. Not that I know of. But the next LDAP conference (in Germany, http://www.daasi.de/ldapcon2011/) might see some talks about such a thing. Hope it helps. thanks -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: LDAP scalability
Oh and on a side note, never have I seen facebook directly disclose its infrastructure. Yes there are rumours it uses a mySQL database however I say that (a) it is a rumour and (b) it would be highly modified. On Wed, May 18, 2011 at 9:09 AM, Bren Norris bren.nor...@measanctum.comwrote: Here's one for you guys. Blizzard, the entertainment guru's had the following appear on their website which is ample evidence to suggest they are using LDAP for their member infrastructure. http://us.battle.net/sc2/en/forum/topic/2325095821 What emmanual has said is correct, sensitive infrastructures aren't openly discussed. That reflects an infrastructure of users which includes all the players of Starcraft II, World of Warcraft (16.7 million users) So there you go, now in the multiples of millions and outside the hundreds of thousands with a subscription value of over $334 million USD. ;) On Wed, May 18, 2011 at 8:57 AM, Emmanuel Lecharny elecha...@gmail.comwrote: On 5/17/11 11:54 PM, Manoj Khangaonkar wrote: Hi , Hi, I am evaluating LDAP in general (Apache DS as possible LDAP) for use as a user repository to be used for authentication /authorization for a large scale web application. The choice is RDBMS vs LDAP vs NoSQL. It's not really a choice. LDAP is the only pristine solution when it comes to manage authentication and authorization. Using a RDBMS or a NoSQL system implies you build a authn/authz solution on top of it (I'm not talking about LDAP over a RDBMS) I am well aware that LDAP is used by large enterprises. These enterprises have typically thousands of users. Hundred of thousands, and I have seen big telco companies using LDAP for more than 70 000 000 users... But have not seen it referenced in large scale web application architectures - such as those at google , facebook, linkedin which deal with millions of users. ( They might be using ldap but I have not seen anything on the web that says they do) Probably because they don't necessarily want to expose such a sensitive part of their IT, but most certainly because they need a highly replicated system. Can LDAP in general and Apache DS in particular scale to millions of users ? Base line, yes. Dealing with millions of users is not really an issue. What is important here is not the number of users, but much more the operation per second you want to process on the LDAP server. On a laptop, OpenLDAP currently deal with up to 10 000 authentication *per second*, and with ApacheDS, last time I conducted a test (last year), it was around 4 500 authentication per second. Are there any blogs/articles on web that talk of LDAP use in architectures of very large scale. Not that I know of. But the next LDAP conference (in Germany, http://www.daasi.de/ldapcon2011/) might see some talks about such a thing. Hope it helps. thanks -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: LDAP scalability
On 5/18/11 1:13 AM, Bren Norris wrote: Oh and on a side note, never have I seen facebook directly disclose its infrastructure. http://www.infoq.com/presentations/Facebook-Software-Stack But it was back in 2009... Yes there are rumours it uses a mySQL database however I say that (a) it is a rumour and (b) it would be highly modified. (a) is not a rumor :) and (b) is true... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: LDAP scalability
Thank you very much sir! :) On Wed, May 18, 2011 at 9:34 AM, Emmanuel Lécharny elecha...@apache.orgwrote: On 5/18/11 1:13 AM, Bren Norris wrote: Oh and on a side note, never have I seen facebook directly disclose its infrastructure. http://www.infoq.com/presentations/Facebook-Software-Stack But it was back in 2009... Yes there are rumours it uses a mySQL database however I say that (a) it is a rumour and (b) it would be highly modified. (a) is not a rumor :) and (b) is true... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: LDAP scalability
Emmanuel Bren, Thanks for the responses. They were helpful. Manoj On Tue, May 17, 2011 at 4:35 PM, Bren Norris bren.nor...@measanctum.com wrote: Thank you very much sir! :) On Wed, May 18, 2011 at 9:34 AM, Emmanuel Lécharny elecha...@apache.orgwrote: On 5/18/11 1:13 AM, Bren Norris wrote: Oh and on a side note, never have I seen facebook directly disclose its infrastructure. http://www.infoq.com/presentations/Facebook-Software-Stack But it was back in 2009... Yes there are rumours it uses a mySQL database however I say that (a) it is a rumour and (b) it would be highly modified. (a) is not a rumor :) and (b) is true... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- http://khangaonkar.blogspot.com/
