Howdy, Where's the EJB running? If it's in the same VM as tomcat, one thing you can do is junk this design and redesign with EJBs at all.
Another idea is to retrieve data in batches from the DB, extract only the information you need for graphing into a temporary object, get the next batch (not keeping the previous in memory), etc, until the object holds all and only the data you need to graph. Then graph it and discard the object. The idea here is to keep as little in memory as possible. Execution speed may be sacrificed a little bit, but if you design it right and use a connection pool, batch querying (use a PreparedStatement as well maybe) can be fast enough for your needs. Yet another idea is that there are frameworks out there to implement iterating over a database query result for you, e.g. TICL. Finally, what's wrong with increasing -Xmx? If you've tuned everything you can in your app, and it still needs some memory, then give it the memory ;) Your environment may dictate otherwise, but usually physical memory is cheap and can only benefit you. Yoav Shapira Millennium ChemInformatics >-----Original Message----- >From: Andrew Latham [mailto:[EMAIL PROTECTED] >Sent: Thursday, March 06, 2003 10:34 PM >To: [EMAIL PROTECTED] >Subject: Memory issues with large query storage > >I hope someone can assist here. > >Basically I have been writing a 3 Tier Web app using Tomcat and trying >to adhere to the Struts framework. The application connects to an >existing MSSQL database and creates graphs of the data. The way I've >built it is I define an entity bean to store a row of returned data from >the select query. Each entity bean is appended to a Collection and then >made available to the JSP for me to access using <logic:iterate>. This >all works fine, until the Collection.size() reaches numbers like 600,000 >and then I hit memory issues. I know there is not much I can do but >increase -Xmx and page it or buy more physical memory. > >The question I have relates to displaying the data. Some of the output I >want in a table. Showing all results returned is not good if its more >than 100, so I created a "google" type <<previous 10, next 10>> to cycle >through them, so for this to happen I need the Collection of what can be >100,000 beans to sit in memory as having to go back and get the next 10 >from the SQL database is about 2 minutes each time. When the Collection >is returned it has a scope of "Request". To make this available to >another JSP page (for the next 10) I rescoped the Collection to >"session". This introduces a memory problem where it sits around until >the session ends and the GC picks it up. Is there a better way to pass >the Collection from one JSP to another without re-scoping it or what I >can see happen is that if a user runs many queries they all sit around >until the session ends utilising memory. > >I've thought I've setting the Collection to NULL when a new query is >run. Is this an acceptable method? > >Appreciate any advice > >Thanks > >Andrew Latham > > > >*---------- >This message and any attachment(s) is intended only for the use of the >addressee(s) and may contain information that is PRIVILEGED and >CONFIDENTIAL. If you are not the intended addressee(s), you are hereby >notified that any use, distribution, disclosure or copying of this >communication is strictly prohibited. If you have received this >communication in error, please erase all copies of the message and its >attachment(s) and notify the sender or Kanbay postmaster immediately. > >Any views expressed in this message are those of the individual sender and >not of Kanbay. > >Although we have taken steps to ensure that this e-mail and any >attachment(s) are free from any virus, we advise that in keeping with good >computing practice the recipient should ensure they are actually virus >free. This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
