Sorry for missing this out, I meant JPA+TopLink instead of hibernate in the previous email --- Regards, Saptarshi PURKAYASTHA Director R & D, HISP India Health Information Systems Programme
My Tech Blog: http://sunnytalkstech.blogspot.com You Live by CHOICE, Not by CHANCE 2009/4/11 Saptarshi Purkayastha <sun...@gmail.com> > From all the experiments on the state servers and our office server, we > have learnt a few lessons, some of which may be useful to other implementers > and developers: > > We have started different tomcats on different ports and each tomcat uses > 1GB max memory. The advantage of such a setup is that different state > applications can be started/stopped/different memory > allocations/threadpools, etc and one state application is not affecting the > other state application... The disadvantage is that the port number in the > URL can be discomforting to the user. So we created a domain and then each > state has a subdomain mapped to the application. i.e. hp.nhsrc-hmis.org or > bihar.nhsrc-hmis.org... We can play and move the state apps off > servers/tomcats and no one realizes that we moved. Can be used to move apps > when we know a load may be coming. > > In our office I was able to deploy 14 state apps on 6GB max memory to one > tomcat ... but only when one app was deployed at a time from the manager > application. or else the c3p0 bean would suggest a memory leak and the JDBC > connection to MySQL would just get lost. This means whenever after 14 apps > were deployed and tomcat was restarted, the apps would fail to deploy > because all would deploy together. > > To find what caused the error, I tried to do a profiler run and found out > some points of optimization. Memory areas where hibernate sessions were > active went through frequent paging... something of a bottleneck of sorts. > The c3p0 bean was creating Datasource pools even when we were not using them > and probably would only use later after the application was fully running > and doing something useful. Being able to remove hibernate and its mates > altogether can make the application very much lighter... > > That brings me to an empirical question: Can JPA solve some of these memory > issues?? Since it has learnt lessons from all the ORMs including hibernate > and improvised a lot on EJB... I talked to abyot that we may want to try JPA > in the CBS project and try to find an answer to that question. > > BTW, thanks bob for the suggestion. I tried to put the libs out to tomcat, > but they only work when all are put... I couldn't make tomcat selective. > That means we may have translations lost or all apps using the same > language, since they are in the jars. Other than that, one loaded app can > steal other apps resources... and when there is load on multiple apps > running in this configuration, it still gives the JDBC connection lost > exception... only difference is not during deployment, but when using the > apps, which is more dangerous. > --- > Regards, > Saptarshi PURKAYASTHA > Director R & D, HISP India > Health Information Systems Programme > > My Tech Blog: http://sunnytalkstech.blogspot.com > You Live by CHOICE, Not by CHANCE > > > 2009/4/10 Bob Jolliffe <bobjolli...@gmail.com> > >> On tomcat I put ALL jars in shared/lib. And I have this at the bottom >> of my server.xml: >> >> <Context path= "/state1" >> docBase="/home/bobj/dhis2/dhis.war"></Context> >> <!-- <Context path= "/state2" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state3" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state4" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state5" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state6" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state7" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state8" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state9" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state10" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state11" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state12" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state13" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state14" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state15" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state16" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state17" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state18" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state19" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> <!-- <Context path= "/state20" >> docBase="/home/bobj/dhis2/dhis.war"></Context> --> >> </Host> >> </Engine> >> </Service> >> </Server> >> >> And yes, if I uncomment, I can start all 20 of these instances on my >> 2M machine without any particular stress on my available Perm space. >> They are all sharing the same jar so the difference between 1 and 20 >> is pretty marginal. It does take a LONG time to load them all up >> though. Caveats: >> 1. I'm not saying this is a good idea! Just a proof of concept. One >> badly behaved app could bring down all the others. >> 2. The big issue now becomes heap usage. One of the areas we need to >> think about optimizing some of the DHIS2 routines is in the use of >> heap. I haven't looked in detail yet, but for example, the reason why >> we have sometimes had memory issues with import is that we seem to be >> effectively pulling all the data into the heap before flushing the >> hibernate cache. This can be kind of a waste seeing that we are using >> a SAX parser for the very reason that we might scale to very large >> input streams. Probably we can look at ways of chunking these sort of >> operations to increase scalability. In fact I'd be very surprised if >> the Hibernate cache manager isn't smart enough to do this chunking >> already. Either way this kind of issue is solvable. >> 3. Obviously, with the above described arrangement, my database >> connections are all messed up. Still have to sort out the dhis2_home >> context init_parameter. >> >> Cheers >> Bob >> >> 2009/4/10 Knut Staring <knu...@gmail.com>: >> > On Tue, Apr 7, 2009 at 9:47 AM, Bob Jolliffe <bobjolli...@gmail.com> >> wrote: >> >> >> >> If you pull out all the WEB-INF/lib/*.jar files from the war and place >> >> those into tomcat's lib directory you are left with a dhis.war which >> >> is only 1.8M. (I did it on jetty, but same should apply to tomcat). >> > >> > For some reason, it seems it is not as straightforward as that with >> tomcat - >> > if I move some .jar files from WEB-INF/lib Tomcat doesn't seem to find >> them >> > anymore. >> > Knut >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~dhis2-devs >> Post to : dhis2-devs@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~dhis2-devs >> More help : https://help.launchpad.net/ListHelp >> > >
_______________________________________________ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
