Re: Scalability
I've had excellent results with Glassfish. The login server for the MMO I worked on (swtor.com) has been benchmarked at 32,000 logins per second on a 8 CPU machine. The maximum throughput on a simple servlet (nothing dynamic though, just static data), was benchmarked at around 70k. Obviously, obtaining these rates depends on a lot of things, and we noticed that (unsurprisingly) the number of cores has a big impact on maximum scalability (32 / 8 = 4k per core). If you use an antiquated server that still processes requests on a thread per connection model, then your throughput will suffer dramatically. Any modern app server or webserver that has a good non-blocking I/O implementation of the reqest/response handler will do. Servers that come to my mind are tomcat 6, glassfish, grizzly... Hope this helps happy hacking! On 11 Apr 2010, at 16:48, François Meillet wrote: Hi Wicketers, 1) I need to build a system which can perform -6000 different users per day and -200 concurent users The software used: wicket postgresql (mainly read operations) hibernate jms At the beginning there will be 2 computers (I prefer horizontal scalability). What would be your advice for the hardware (hard drive will be 10.000 RPM). -processor ? -ram 2) Also I would appreciate your advice for the server: I have the choice for the server: Tomcat + a JMS implementation like HornetQ, or Jboss Application Server. I hesitate between the 2 solutions, as I don't need EJBs, but we never know !!! . Have you any advice ? Thanks François - To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org For additional commands, e-mail: users-h...@wicket.apache.org -- npg - To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org For additional commands, e-mail: users-h...@wicket.apache.org
Re: Scalability
2010/4/11 François Meillet fm...@meillet.com: Hi Wicketers, 1) I need to build a system which can perform -6000 different users per day and -200 concurent users I can very easily run that on my laptop. The software used: wicket postgresql (mainly read operations) hibernate jms At the beginning there will be 2 computers (I prefer horizontal scalability). What would be your advice for the hardware (hard drive will be 10.000 RPM). -processor ? -ram For 200 concurrent sessions, 100 MB is probably already enough (20MB would be for sessions if you average 100K per session). But give your VM 1GB or more and you should be able to handle a much larger peak (600?) easily 2) Also I would appreciate your advice for the server: I have the choice for the server: Tomcat + a JMS implementation like HornetQ, or Jboss Application Server. I hesitate between the 2 solutions, as I don't need EJBs, but we never know !!! . If you don't need EJBs (and who would, I've always steered clear from them and never regretted it), than don't worry about it. In the unlikely case you ever do, just address it then by swapping out your app server. Write your software generically enough so that you don't depend on a single one. If you know you need MQ and you want management of it integrated in your application server, JBoss is worth considering. Also look at Glassfish while you're at it. Eelco - To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org For additional commands, e-mail: users-h...@wicket.apache.org
RE: Scalability
I hesitate between the 2 solutions, as I don't need EJBs, but we never know !!! . If you have an app that currently doesn't need EJBs then keep it that way!! ;) - To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org For additional commands, e-mail: users-h...@wicket.apache.org
RE: Scalability
On Mon, 2010-04-12 at 08:31 +1000, Chris Colman wrote: I hesitate between the 2 solutions, as I don't need EJBs, but we never know !!! . If you have an app that currently doesn't need EJBs then keep it that way!! ;) More EJB FUD yay - To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org For additional commands, e-mail: users-h...@wicket.apache.org
Re: Scalability PageMaps
James Carman schrieb: hi james Do you think that modulus number should be customizable? So, for now, you mod your number of users by 100. But, for larger sites, they might want to mod it by 1000 or 256 or whatever. actually, the first level dir is created from hash%1 and the second from hash/1%1, while the third level dir is the sessionId itself. i do not see a need for more, do you? why should anyone care of the inner structure of the filestore, as long as it cannot lead to problems? cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
On Thu, Sep 18, 2008 at 3:35 AM, Uwe Schäfer [EMAIL PROTECTED] wrote: James Carman schrieb: hi james Do you think that modulus number should be customizable? So, for now, you mod your number of users by 100. But, for larger sites, they might want to mod it by 1000 or 256 or whatever. actually, the first level dir is created from hash%1 and the second from hash/1%1, while the third level dir is the sessionId itself. i do not see a need for more, do you? why should anyone care of the inner structure of the filestore, as long as it cannot lead to problems? So, why are we dividing it up in the first place if it can't lead to problems? Why are we picking arbitrary divisors? Are they the optimal setting? cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
James Carman schrieb: So, why are we dividing it up in the first place if it can't lead to problems? before, there was just the sessionId, which - as we all know - could lead to problems with 32k concurrent sessions. what you now have is 8 numbers (like in 100.000.000 minus one) * 32k. i would be surprised to see that many sessions without having an almost infinite session timeout. Why are we picking arbitrary divisors? Are they the optimal setting? one more char per directory can wreck the 32k limit, one less would give you a 32million (with optimal distribution of course) which i found to be an 'unnecessary limitation'. i´d say reasonable, but i really would not mind to see this configurable. i just didn´t because a) i cannot imagine a valid usecase for more (could be wrong here of course, and would like to know) and - more importantly - b) this is a very small, local patch with no API change with and thus less opportunities for a rookie like me to screw things up. if it were configurable... you know... ;) cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
yes having this configurable looks ridiculous to me. The only thing that i can think of is is the 32K limit for all the filesystems? or does another filesystem has another limit? Because if that is the case we could make it so that for example we do 3 (1000 dirs) or 4 (1 dirs) numbers per dir hopefully the hash is a bit equally divided so the first dir will make 999 dirs before it complete fills up for example 1 dir hope it balances On Thu, Sep 18, 2008 at 1:32 PM, Uwe Schäfer [EMAIL PROTECTED]wrote: James Carman schrieb: So, why are we dividing it up in the first place if it can't lead to problems? before, there was just the sessionId, which - as we all know - could lead to problems with 32k concurrent sessions. what you now have is 8 numbers (like in 100.000.000 minus one) * 32k. i would be surprised to see that many sessions without having an almost infinite session timeout. Why are we picking arbitrary divisors? Are they the optimal setting? one more char per directory can wreck the 32k limit, one less would give you a 32million (with optimal distribution of course) which i found to be an 'unnecessary limitation'. i´d say reasonable, but i really would not mind to see this configurable. i just didn´t because a) i cannot imagine a valid usecase for more (could be wrong here of course, and would like to know) and - more importantly - b) this is a very small, local patch with no API change with and thus less opportunities for a rookie like me to screw things up. if it were configurable... you know... ;) cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
Johan Compagner schrieb: hi Johan or does another filesystem has another limit? from what i´ve googled, all FS i could think of either dont care, or limit to (almost) 32k or more. (could be wrong here, anyone?) hopefully the hash is a bit equally divided so the first dir will make 999 dirs before it complete fills up for example 1 dir looking at String.hashCode i would say yes, especially for small Strings (32chars) distribution is as good as it gets : cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
I don't you you want to come anywhere near the amount of 32k files in single folder. I wouldn't assume that every filesystem can handle such amount without performance problems. -Matej On Thu, Sep 18, 2008 at 1:49 PM, Johan Compagner [EMAIL PROTECTED]wrote: yes having this configurable looks ridiculous to me. The only thing that i can think of is is the 32K limit for all the filesystems? or does another filesystem has another limit? Because if that is the case we could make it so that for example we do 3 (1000 dirs) or 4 (1 dirs) numbers per dir hopefully the hash is a bit equally divided so the first dir will make 999 dirs before it complete fills up for example 1 dir hope it balances On Thu, Sep 18, 2008 at 1:32 PM, Uwe Schäfer [EMAIL PROTECTED] wrote: James Carman schrieb: So, why are we dividing it up in the first place if it can't lead to problems? before, there was just the sessionId, which - as we all know - could lead to problems with 32k concurrent sessions. what you now have is 8 numbers (like in 100.000.000 minus one) * 32k. i would be surprised to see that many sessions without having an almost infinite session timeout. Why are we picking arbitrary divisors? Are they the optimal setting? one more char per directory can wreck the 32k limit, one less would give you a 32million (with optimal distribution of course) which i found to be an 'unnecessary limitation'. i´d say reasonable, but i really would not mind to see this configurable. i just didn´t because a) i cannot imagine a valid usecase for more (could be wrong here of course, and would like to know) and - more importantly - b) this is a very small, local patch with no API change with and thus less opportunities for a rookie like me to screw things up. if it were configurable... you know... ;) cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
Also, looking at this whole thing. 32000 concurrent sessions on one machine? What are you trying to achieve? I have yet to see a machine that can handle 32k concurrent users... -Matej On Thu, Sep 18, 2008 at 4:12 PM, Matej Knopp [EMAIL PROTECTED] wrote: I don't you you want to come anywhere near the amount of 32k files in single folder. I wouldn't assume that every filesystem can handle such amount without performance problems. -Matej On Thu, Sep 18, 2008 at 1:49 PM, Johan Compagner [EMAIL PROTECTED]wrote: yes having this configurable looks ridiculous to me. The only thing that i can think of is is the 32K limit for all the filesystems? or does another filesystem has another limit? Because if that is the case we could make it so that for example we do 3 (1000 dirs) or 4 (1 dirs) numbers per dir hopefully the hash is a bit equally divided so the first dir will make 999 dirs before it complete fills up for example 1 dir hope it balances On Thu, Sep 18, 2008 at 1:32 PM, Uwe Schäfer [EMAIL PROTECTED] wrote: James Carman schrieb: So, why are we dividing it up in the first place if it can't lead to problems? before, there was just the sessionId, which - as we all know - could lead to problems with 32k concurrent sessions. what you now have is 8 numbers (like in 100.000.000 minus one) * 32k. i would be surprised to see that many sessions without having an almost infinite session timeout. Why are we picking arbitrary divisors? Are they the optimal setting? one more char per directory can wreck the 32k limit, one less would give you a 32million (with optimal distribution of course) which i found to be an 'unnecessary limitation'. i´d say reasonable, but i really would not mind to see this configurable. i just didn´t because a) i cannot imagine a valid usecase for more (could be wrong here of course, and would like to know) and - more importantly - b) this is a very small, local patch with no API change with and thus less opportunities for a rookie like me to screw things up. if it were configurable... you know... ;) cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
From Matej's previous email: e.g. int hash = computeHash(sessionId) % 100; String folder = + hash + / + sessionId; That's where I got the 100 thing. So, what we're talking about here is potentially 10k buckets for session store folders. There would be the potential for 10k top-level folders. Perhaps smaller sites would wish to not have the potential for 10k folders? I don't know. A smallish site might just want to make the divisor 1, if they don't have many users. The folders won't be removed even after all of their subfolders are removed, right? On Thu, Sep 18, 2008 at 3:35 AM, Uwe Schäfer [EMAIL PROTECTED] wrote: James Carman schrieb: hi james Do you think that modulus number should be customizable? So, for now, you mod your number of users by 100. But, for larger sites, they might want to mod it by 1000 or 256 or whatever. actually, the first level dir is created from hash%1 and the second from hash/1%1, while the third level dir is the sessionId itself. i do not see a need for more, do you? why should anyone care of the inner structure of the filestore, as long as it cannot lead to problems? cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
Also, looking at this whole thing. 32000 concurrent sessions on one machine? What are you trying to achieve? I have yet to see a machine that can handle 32k concurrent users... hehe - the new 'Office Cray' CX1 might be up for the challenge. :D - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
I saw that yesterday. I wonder if that thing will run linux? When I get rich and infamous, I'm buying one. On Thu, Sep 18, 2008 at 10:32 AM, Jan Kriesten [EMAIL PROTECTED] wrote: Also, looking at this whole thing. 32000 concurrent sessions on one machine? What are you trying to achieve? I have yet to see a machine that can handle 32k concurrent users... hehe - the new 'Office Cray' CX1 might be up for the challenge. :D - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
you can order it with windows hpc _or_ red hat linux :-) I saw that yesterday. I wonder if that thing will run linux? When I get rich and infamous, I'm buying one. Also, looking at this whole thing. 32000 concurrent sessions on one machine? What are you trying to achieve? I have yet to see a machine that can handle 32k concurrent users... hehe - the new 'Office Cray' CX1 might be up for the challenge. :D - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
James Carman schrieb: hi James That's where I got the 100 thing. ah, i lost track of that. did not want to sound harsh or anything. The folders won't be removed even after all of their subfolders are removed, right? no empty folder should be left behind (if you stay away from just killing the appserver, of course ;) cu uwe -- THOMAS DAILY GmbH Adlerstraße 19 79098 Freiburg Deutschland T + 49 761 3 85 59 0 F + 49 761 3 85 59 550 E [EMAIL PROTECTED] www.thomas-daily.de Geschäftsführer/Managing Directors: Wendy Thomas, Susanne Larbig Handelsregister Freiburg i.Br., HRB 3947 Registrieren Sie sich unter http://morningnews.thomas-daily.de für die kostenfreien TD Morning News, eine Auswahl aktueller Themen des Tages morgens um 9:00 in Ihrer Mailbox. Hinweis: Der Redaktionsschluss für unsere TD Morning News ist täglich um 8:30 Uhr. Es werden vorrangig Informationen berücksichtigt, die nach 16:00 Uhr des Vortages eingegangen sind. Die Email-Adresse unserer Redaktion lautet [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
Do you think that modulus number should be customizable? So, for now, you mod your number of users by 100. But, for larger sites, they might want to mod it by 1000 or 256 or whatever. On Wed, Sep 17, 2008 at 3:25 PM, Uwe Schäfer [EMAIL PROTECTED] wrote: Igor Vaynberg schrieb: you can prefix the file with the session name... appended a patch (to the issue) that uses two directory-levels (from the hashcode) as a prefix to the sessionId and removes them on unbind(). thx for your suggestions. cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
Matej Knopp schrieb: can you create an enhancement jira issue for this? will do. thx cu uwe -- THOMAS DAILY GmbH Adlerstraße 19 79098 Freiburg Deutschland T + 49 761 3 85 59 0 F + 49 761 3 85 59 550 E [EMAIL PROTECTED] www.thomas-daily.de Geschäftsführer/Managing Directors: Wendy Thomas, Susanne Larbig Handelsregister Freiburg i.Br., HRB 3947 Registrieren Sie sich unter http://morningnews.thomas-daily.de für die kostenfreien TD Morning News, eine Auswahl aktueller Themen des Tages morgens um 9:00 in Ihrer Mailbox. Hinweis: Der Redaktionsschluss für unsere TD Morning News ist täglich um 8:30 Uhr. Es werden vorrangig Informationen berücksichtigt, die nach 16:00 Uhr des Vortages eingegangen sind. Die Email-Adresse unserer Redaktion lautet [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
Uwe Schäfer schrieb: can you create an enhancement jira issue for this? filed as WICKET-1837, patch against 1.4m3 included. thx uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
I had a quick glance at the patch. The folder hierarchy it creates seems to be quite deep, i'm not sure if that is necessary. Perhaps 3-4 levels would be enough? Also I'm afraid that this approach would leave lot of empty folders. -Matej On Tue, Sep 16, 2008 at 3:55 PM, Uwe Schäfer [EMAIL PROTECTED]wrote: Uwe Schäfer schrieb: can you create an enhancement jira issue for this? filed as WICKET-1837, patch against 1.4m3 included. thx uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
Matej Knopp schrieb: I had a quick glance at the patch. The folder hierarchy it creates seems to be quite deep, i'm not sure if that is necessary. Perhaps 3-4 levels would be enough? Also I'm afraid that this approach would leave lot of empty folders. well, that depends on the number of different chars the container uses in a session-id. i honestly don´t know if this is part of the spec. of course making every fourth char a / might be enough. it is just that i´ve seen this strategy in squid. yeah, i did not take a look at removing yet. i´ll do in the evening. cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
Another approach would be just counting hash from session id and mod it. -Matej On Tue, Sep 16, 2008 at 6:00 PM, Uwe Schäfer [EMAIL PROTECTED]wrote: Matej Knopp schrieb: I had a quick glance at the patch. The folder hierarchy it creates seems to be quite deep, i'm not sure if that is necessary. Perhaps 3-4 levels would be enough? Also I'm afraid that this approach would leave lot of empty folders. well, that depends on the number of different chars the container uses in a session-id. i honestly don´t know if this is part of the spec. of course making every fourth char a / might be enough. it is just that i´ve seen this strategy in squid. yeah, i did not take a look at removing yet. i´ll do in the evening. cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
Matej Knopp schrieb: Another approach would be just counting hash from session id and mod it. that´s much better for sure. i´ll do it asap. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
A hash doesnt have to be unique for the same string :) So when will the first be that has session leakage because of that On 9/16/08, Matej Knopp [EMAIL PROTECTED] wrote: Another approach would be just counting hash from session id and mod it. -Matej On Tue, Sep 16, 2008 at 6:00 PM, Uwe Schäfer [EMAIL PROTECTED]wrote: Matej Knopp schrieb: I had a quick glance at the patch. The folder hierarchy it creates seems to be quite deep, i'm not sure if that is necessary. Perhaps 3-4 levels would be enough? Also I'm afraid that this approach would leave lot of empty folders. well, that depends on the number of different chars the container uses in a session-id. i honestly don´t know if this is part of the spec. of course making every fourth char a / might be enough. it is just that i´ve seen this strategy in squid. yeah, i did not take a look at removing yet. i´ll do in the evening. cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
you can prefix the file with the session name... -igor On Tue, Sep 16, 2008 at 2:17 PM, Johan Compagner [EMAIL PROTECTED] wrote: A hash doesnt have to be unique for the same string :) So when will the first be that has session leakage because of that On 9/16/08, Matej Knopp [EMAIL PROTECTED] wrote: Another approach would be just counting hash from session id and mod it. -Matej On Tue, Sep 16, 2008 at 6:00 PM, Uwe Schäfer [EMAIL PROTECTED]wrote: Matej Knopp schrieb: I had a quick glance at the patch. The folder hierarchy it creates seems to be quite deep, i'm not sure if that is necessary. Perhaps 3-4 levels would be enough? Also I'm afraid that this approach would leave lot of empty folders. well, that depends on the number of different chars the container uses in a session-id. i honestly don´t know if this is part of the spec. of course making every fourth char a / might be enough. it is just that i´ve seen this strategy in squid. yeah, i did not take a look at removing yet. i´ll do in the evening. cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scalability PageMaps
Hash will only be there for the folder name. e.g. int hash = computeHash(sessionId) % 100; String folder = + hash + / + sessionId; this way you will have 100 top level folders in which you'll have session folders. This was it's much less likely to have 32k session in one folder. Or you can also go multiple levels, compute hash from part of the session id, etc. -Matej On Tue, Sep 16, 2008 at 11:17 PM, Johan Compagner [EMAIL PROTECTED]wrote: A hash doesnt have to be unique for the same string :) So when will the first be that has session leakage because of that On 9/16/08, Matej Knopp [EMAIL PROTECTED] wrote: Another approach would be just counting hash from session id and mod it. -Matej On Tue, Sep 16, 2008 at 6:00 PM, Uwe Schäfer [EMAIL PROTECTED]wrote: Matej Knopp schrieb: I had a quick glance at the patch. The folder hierarchy it creates seems to be quite deep, i'm not sure if that is necessary. Perhaps 3-4 levels would be enough? Also I'm afraid that this approach would leave lot of empty folders. well, that depends on the number of different chars the container uses in a session-id. i honestly don´t know if this is part of the spec. of course making every fourth char a / might be enough. it is just that i´ve seen this strategy in squid. yeah, i did not take a look at removing yet. i´ll do in the evening. cu uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
