On 24 October 2011 18:29, Paul Joseph <[email protected]> wrote: > Actually the real reason is that the user is using a query that required > crafting to return 25K results. He is seeking to show the system "does not > scale." >
Then set a limit when building the query > > Actually I am using in my XML template, a cocoon flowscript repeater widget > that does paging. But this is well after the Java logic has built the large > array of results (the array results and the vector secureResults below) > The actual paging should be done on DB level, not after you've fetched the enormous amount of results. > > On 10/24/2011 12:22 PM, Jasha Joachimsthal wrote: > > The first question that comes to my mind: why would you need 25000 objects > as query result? Can't you do 1 query that only returns the total possible > amount of objects and another query that returns e.g. the first 10 results > and build in some paging logic for the next 10. > > Jasha Joachimsthal > > Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466 > US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll free) > > www.onehippo.com > > > On 24 October 2011 18:13, Paul Joseph <[email protected]> wrote: > >> I think you hit the nail on the head. >> >> I assumed that the memory is freed at session time out. I did not realize >> this was not the case. >> >> I know what the culprit is in this case--the array that I use to store the >> objects returned from the query. >> >> I do this in two places--once in the business logic (java code) and once >> in the flow script. >> >> The code in the java business logic is as follows--the item of interest is >> a "Task". How would I optimize it in terms of memory collection? >> >> public Object[] getSearchResults() throws Exception { >> >> Object[] results = getSearchResultBeans(sql_query); >> >> Vector secureResults = new Vector(); >> >> for (int n=0; n < results.length; n++) { >> >> Task t = (Task)results[n]; >> >> if (securityManager.canIViewTask(t)) secureResults.add(t); >> >> } >> >> return secureResults.toArray(); >> >> } >> >> On 10/24/2011 11:50 AM, Nathaniel, Alfred wrote: >> >>> Hi Paul, >>> >>> I don't think that it is a database issue. >>> It is rather the question where the application places the large amounts >>> of data and how it is cleaned up. >>> >>> I interprete your statements that it is placed in the session object >>> assuming that the memory is freed at session timeout. >>> Unfortunately that is not the case. >>> >>> The session timeout is a security feature to force a new login if the >>> same user comes back after a longish idle time. >>> There is no guarantee that the container will actually delete the session >>> object at the session timeout. >>> As long as there is a reference to the session object GC cannot free the >>> attached memory. >>> >>> You will have to find a way that the application keeps the data only for >>> the duration of a request, or use another mechanism to limit the memory >>> requirements. >>> >>> HTH, Alfred. >>> >>> -----Original Message----- >>> From: Paul Joseph [mailto:[email protected]] >>> Sent: Montag, 24. Oktober 2011 17:16 >>> To: [email protected] >>> Subject: thought I had fixe it >>> >>> Hi there, >>> >>> I thought I had fixed this memory issue but... >>> >>> I am using Tomcat 6 in a Windows 32 bit environment (Windows 2003) with >>> Cocoon 2.1.11 and Java 1.6, agains Postgresql 8.4 with the latest >>> Postgres 9.1, JDBC 4 driver. >>> >>> >>> The behavior is this: >>> >>> The user fires of a LARGE query that returns 25,000 large objects. >>> >>> The user repeats this 20 times within 5 minutes to show me he can freeze >>> the app. >>> >>> On the 20th time, it says out of memory (heap space). >>> >>> >>> The JVM indicates that it is maxed to the limit specified in Xms and >>> that there is only about 2MB of memory free. >>> >>> I then ask her to log off. >>> >>> The session time out is set to 20 minutes. >>> >>> But even after an hour, the memory is not reclaimed by the JVM--it still >>> reports that only about 2MB is still free. >>> >>> Is the fact that it is not reclaiming memory an indication of a memory >>> leak? >>> >>> I am using the following settings in my repository.database: >>> >>> <jdbc-connection-descriptor >>> jcd-alias="WebApp" >>> default-connection="true" >>> platform="PostgreSQL" >>> jdbc-level="4.0" >>> driver="org.postgresql.Driver" >>> protocol="jdbc" >>> subprotocol="postgresql" >>> dbalias="//localhost:5432/WebApp" >>> username="******" >>> password="******" >>> eager-release="false" >>> batch-mode="false"> >>> >>> <connection-pool maxActive="200" validationQuery="" /> >>> <sequence-manager >>> >>> className="org.apache.ojb.broker.util.sequence.SequenceManagerNextValImpl" >>> /> >>> </jdbc-connection-descriptor> >>> >>> >>> Thanks much! >>> Paul >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> The content of this e-mail is intended only for the confidential use of >>> the person addressed. >>> If you are not the intended recipient, please notify the sender and >>> delete this e-mail immediately. >>> Thank you. >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >>> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> >
