Paragraphs. Please! :) -----Original Message----- From: Steel City Phantom <[email protected]> Sender: [email protected] Date: Thu, 26 Jan 2012 13:07:55 To: <[email protected]> Reply-To: [email protected] Subject: Re: [The Java Posse] Re: Experience to scaling up many cores and server clusters. Java and other technologies.
ive done a few projects where i had some massive memcache servers to handle session storage and just basic jboss clusters. it worked pretty well, in all cases when we found a bottle neck it was either poor code or something else should have been stored in memcache. stuff like authentication token, configuration tables, user profile tables, etc was in memcache. you certainly do not need to write the system in C++ to get a scalable system. as you said, its all about resources and how much time you have to finish. but i have yet to come across something that JEE couldn't handle very nicely. one mistake (in my opinion) i see a lot of people do is they will write their app into a single war file and deploy that to clusters. if you are dealing with a very heavy load, you get better performance by breaking the application out into specifc tasks. say for example, you have an application that uploads pictures (flickr for example) i would break that apart as follows subproject that would handle the user interface and run that on a cluster with general purpose servers. subproject that handles image processing, resizing, cropping, anything of that sort and deploy that to a group of servers with huge processors and memory subproject that handles file storage and retrieval. deploy that on servers that have say an ISCSI array or some other storage system of your choosing. subproject that handles business rules and database storage, deploy that on servers that can skimp on the cpu, but have lots of memory. tie those subsystems together with an efficient messaging system and high end infrastructure hardware and you are set. using that method allows you to really zero in on what aspect of the system is not performing as well as others and deal with it. that will certainly get you a system that can handle millions of requests an hour with no problem. also, take a very close look at the libraries you use. like in the above system, i wouldn't dream of using hibernate, JDBC all the way. you would be amazed how much of a performance gain you get by only getting the fields you need back from the DB at a time instead of the whole record like hibernate is tuned for. and yes, i know hibernate can do it, but i find just using JDBC is much quicker to code. if you are working with an existing system, you can do things in stages. first order, break the app into subprojects and deploy the various subprojects on different servers. then analyze, figure out which part is the slowest and work on that piece. keep doing that cycle and you will end up with a great system. PS, on multi-server systems like this, you would be shocked to see how many times i gained 5 - 10% performance just by changing the patch cords with high quality cords. take nothing for granted. On Thu, Jan 26, 2012 at 11:48 AM, James Ward <[email protected]> wrote: > Most modern messaging systems have a built-in way to do message > throttling. But I believe that would only be on the message server receive > side. If you need it on the send side then sticking a value in memcached > might be the most performant place to put that global state. > > -James > > > > On 01/26/2012 08:01 AM, Carl Jokl wrote: > >> I suppose that the whole Stateless push is also encouraged by >> Stateless Session Beans too. >> I suppose certain problems can be complicated by a need for some kind >> of common state and can get in the way of parallelism. One example in >> the system I currently work with is the need to have a limit on the >> number of messages sent that are of a given category. Sending vast >> numbers of messages can be scaled in parallel but it gets complicated >> if the system must guarantee that no more than a given number of >> messages of that category are sent because then a central count needs >> to be kept and updated in a synchronized manner. >> >> > -- > You received this message because you are subscribed to the Google Groups > "The Java Posse" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to javaposse+unsubscribe@** > googlegroups.com <javaposse%[email protected]>. > For more options, visit this group at http://groups.google.com/** > group/javaposse?hl=en <http://groups.google.com/group/javaposse?hl=en>. > > -- You want it fast, cheap, or right. Pick two!! -- You received this message because you are subscribed to the Google Groups "The Java Posse" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/javaposse?hl=en. -- You received this message because you are subscribed to the Google Groups "The Java Posse" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.
