Abhinav, Let me reply to you inline.
On Mon, Feb 1, 2010 at 10:51 PM, Abhinav Singh <[email protected]>wrote: > Hi, > > I am looking to deploy pub sub for a website build to display live sports > scorecard to the users. > At peak traffic, there are about 200K concurrent users. Since the load is > probably too much, I am doubting if pub sub can really be a good solution > here. > Just like a website can scale to 200,000 concurrent users, Pubsub and XMPP can, it's just a matter of having an effective strategy to do so. > > Here is an overview of how I am looking to deploy pub sub: > > 1. User visits http://localhost/match-id-23000 > 2. As soon as user loads the match page, he is subscribed to the match > specific node > > Do these users have accounts or are they anonymous? Temporary subscriptions while web browsing requires a pubsub service that supports temporary subscriptions. > > 1. There can be 1 or more (<5) publishers, who actually are vendors > sitting in remote locations > 2. Vendors publish the live scorecard as and when data is available > with them > 3. Subscribed users gets the live scorecard update in almost real-time > > Here is the technical flow of the same: > > 1. On page load, an ajax is fired subscribing the user for match node > 2. Target url for all ajax requests in a PHP connection manager (say > bosh.class.php) > > I assume you're talking about JAXL here. Also consider client-side for browser updates with strophe.js. > > 1. PHP connection manager parse incoming request params, generated > appropriate xml's as specified in various XEP's and communicate with the > jabber server > 2. PHP conn. manager communicate by curling the jabber server > 3. Jabber server respond back whenever there is new data available for > the subscribers > > Now the scalability doubts which comes to my mind are: > > 1. With PHP connection manager holding every single incoming request > (timeout 60 sec) and waiting for a response from the jabber server, I > assume > under 200K concurrent users the web server will soon stop accepting the > incoming ajax requests. > > This is why I suggest client-side strophe.js -- though I'm sure you *could* make the server-side management work if you threw enough hardware at the problem. > > 1. Also are ejabberd, openfire and the likes capable of handling such a > load. > > Ejabberd, configured correctly can handle the load. Now, I'm not talking about ejabberd's publish-subscribe interface. I would suggest using Ralph's Idavoll or my SleekPubsub which I'm very actively developing to solve this problem. I also suggest a small cluster or some load balancing. I do not recommend Openfire for this kind of load. You can also take a look at Tigase w/ their built in Publish-Subscribe. > > 1. > 2. If yes, how much is the estimated throughput of these jabber server. > (need to access amount of infrastructure required) > > A well tuned XMPP Server generally shoots for 100,000 messages per second on a single core CPU. With extra cores, clustering, etc, you can easily get higher. > > 1. > 2. Overall I think PHP connection manager will be the main bottleneck > here. So will it be advisable to proxy all incoming ajax requests directly > to the jabber server and put the login between the vendor and the jabber > server. > > I agree. Again, consider client-side connection to BOSH in JS with strophe.js. > > 1. > > Thanks in advance for your helpful advise and insight. > > Abhinav Singh, > Bangalore, > India > http://abhinavsingh.com > > Thanks. I'm a fulltime XMPP consultant, and happy to help with these kinds of projects. I'm happy to give free advice, however, when it comes to actually writing code or helping you install and configure services, I have reasonable rates. You should know that what you're proposing it completely doable. It was a lot harder a couple of years ago, as there weren't very many good implementations of Publish-Subscribe (although Idavoll was around). Some people had bad experience trying to do this with ejabberd's earlier implementations of publish-subscribe, though I hear it has improved. Thanks, Nathan Fritz 509.396.6306
