RE: Apache Derby - Locks up.
Hi Brett, Many thanks for your help, VisualVM showed up the issue and you were indeed correct.. It was ignoring the commands of increasing the heap. I have now got a fresh version of the startNetworkService and made the necessary changes. It now runs well.. One question from this experience, using VisualVM I can see over multiple hours the HeapSpace continues to grow and doesn't garbage collect. Is there a setting or timer which can garbage collect the heap more frequently? Kind regards, Chris Olver -Original Message- From: Bergquist, Brett [mailto:bbergqu...@canoga.com] Sent: 06 June 2014 23:21 To: Derby Discussion Subject: RE: Apache Derby - Locks up. Maybe connect up with jvisualvm to the process and see what it is reporting. Jvisualvm will show the environment of the process.Since you are specifying both a -Xmx and -Xms, the heap should be initially allocated to your 6G and never grow or shrink. Jvisualvm will show if that is it is doing so. It seems strange that it is requiring so much memory. Yes your row count is high but I have tables in my database that are regularly 10M records that are queried every 5 minutes with no issue and I can do so with a much smaller memory setting (4G is what I am using in testing). Derby can work in an environment like this as one system that we have has tables that are inserted into about 10M records/day with no issues. -Original Message- From: Chris Olver [mailto:chris.ol...@3plearning.com] Sent: Thursday, June 05, 2014 11:42 PM To: Derby Discussion Subject: RE: Apache Derby - Locks up. Just before it locked up, I managed to get this: --- Derby Network Server Runtime Information --- -- Session Information --- Session # :4 Database :etl User :a # Statements:1 Prepared Statement Information: Stmt ID SQLText - --- SYSLH0001 SELECT * FROM APP.PROFILES Session # :19 - # Connection Threads : 2 # Active Sessions : 2 # Waiting Sessions : 0 Total Memory : 3042443264 Free Memory : 153934376 So I'm not sure what heap size it is reaching. Regards, -Chris -Original Message- From: Bryan Pendleton [mailto:bpendleton.de...@gmail.com] Sent: 06 June 2014 12:32 To: Derby Discussion Subject: Re: Apache Derby - Locks up. It's interesting that you are trying to set it to use 6 G, and yet in your other message there was the line: Total Memory : 1756889088 Free Memory : 306272128 which seems to indicate that it's only using 1.7 GB. Like maybe you're running a 32 bit JVM, not a 64 bit JVM, somehow? bryan
RE: Apache Derby - Locks up.
Maybe connect up with jvisualvm to the process and see what it is reporting. Jvisualvm will show the environment of the process.Since you are specifying both a -Xmx and -Xms, the heap should be initially allocated to your 6G and never grow or shrink. Jvisualvm will show if that is it is doing so. It seems strange that it is requiring so much memory. Yes your row count is high but I have tables in my database that are regularly 10M records that are queried every 5 minutes with no issue and I can do so with a much smaller memory setting (4G is what I am using in testing). Derby can work in an environment like this as one system that we have has tables that are inserted into about 10M records/day with no issues. -Original Message- From: Chris Olver [mailto:chris.ol...@3plearning.com] Sent: Thursday, June 05, 2014 11:42 PM To: Derby Discussion Subject: RE: Apache Derby - Locks up. Just before it locked up, I managed to get this: --- Derby Network Server Runtime Information --- -- Session Information --- Session # :4 Database :etl User :a # Statements:1 Prepared Statement Information: Stmt ID SQLText - --- SYSLH0001 SELECT * FROM APP.PROFILES Session # :19 - # Connection Threads : 2 # Active Sessions : 2 # Waiting Sessions : 0 Total Memory : 3042443264 Free Memory : 153934376 So I'm not sure what heap size it is reaching. Regards, -Chris -Original Message- From: Bryan Pendleton [mailto:bpendleton.de...@gmail.com] Sent: 06 June 2014 12:32 To: Derby Discussion Subject: Re: Apache Derby - Locks up. It's interesting that you are trying to set it to use 6 G, and yet in your other message there was the line: Total Memory : 1756889088 Free Memory : 306272128 which seems to indicate that it's only using 1.7 GB. Like maybe you're running a 32 bit JVM, not a 64 bit JVM, somehow? bryan
RE: Apache Derby - Locks up.
Chris, can you get a stack trace from the Network Server process when this occurs? From: Chris Olver [mailto:chris.ol...@3plearning.com] Sent: Wednesday, June 04, 2014 4:42 AM To: derby-user@db.apache.org Subject: Apache Derby - Locks up. Hi, We are looking to use Apache Derby (Network Server) as a Caching Layer for our ETL process. Unfortunately we are finding that it locks up quite frequently. We have two tables which have around 10 million rows, two indexes in each. We can be reading (straight SELECT * FROM) or writing updates when it happens. CPU will spike to 100% (its on a rather powerful box) and then all existing and new JDBC clients are unable to connect. Running runtimeinfo (when it locks up, issuing this command can take a few minutes to get a response): --- Derby Network Server Runtime Information --- -- Session Information --- Session # :116 Database :etl User : abc # Statements:1 Prepared Statement Information: Stmt ID SQLText - --- SYSLH0001 SELECT * FROM APP.USERS Session # :117 - # Connection Threads : 4 # Active Sessions : 2 # Waiting Sessions : 0 Total Memory : 1756889088 Free Memory : 306272128 No errors can be seen in the log. I am rather confused as it seems like the perfect solution Derby just locks up. Thoughts or advise appreciated. OS: Windows 8.1 Java Runtime: 1.8.0_05-b13 Derby: 10.10.2.0 Regards, -Chris
Re: Apache Derby - Locks up.
java.lang.OutOfMemoryError: Java heap space Have you experimented with giving more memory to your JVM? thanks, bryan
RE: Apache Derby - Locks up.
When it crashed, the command line for starting the server was: %_JAVACMD% -d64 -Xms6072M -Xmx6072M -XX:+AggressiveHeap -XX:MaxPermSize=1024M -XX:+CMSClassUnloadingEnabled %DERBY_OPTS% -classpath %CLASSPATH%;%LOCALCLASSPATH% org.apache.derby.drda.NetworkServerControl start %DERBY_CMD_LINE_ARGS% Would there be a better place to set it? I can set to 8G and see what happpens. -Chris -Original Message- From: Bryan Pendleton [mailto:bpendleton.de...@gmail.com] Sent: 06 June 2014 12:22 To: Derby Discussion Subject: Re: Apache Derby - Locks up. java.lang.OutOfMemoryError: Java heap space Have you experimented with giving more memory to your JVM? thanks, bryan
Re: Apache Derby - Locks up.
It's interesting that you are trying to set it to use 6 G, and yet in your other message there was the line: Total Memory : 1756889088 Free Memory : 306272128 which seems to indicate that it's only using 1.7 GB. Like maybe you're running a 32 bit JVM, not a 64 bit JVM, somehow? bryan
RE: Apache Derby - Locks up.
It got the roughly the same point and has stopped with 6GB heap size. Looking at task manager, the max commit size is 4.1GB.. Is there any other places that could be overruling the command line setting? Regards, Chris Olver Technical Business Analyst, 3P Learning www.3plearning.com Follow us on Facebook and Twitter. -Original Message- From: Bryan Pendleton [mailto:bpendleton.de...@gmail.com] Sent: 06 June 2014 12:22 To: Derby Discussion Subject: Re: Apache Derby - Locks up. java.lang.OutOfMemoryError: Java heap space Have you experimented with giving more memory to your JVM? thanks, bryan
RE: Apache Derby - Locks up.
Just before it locked up, I managed to get this: --- Derby Network Server Runtime Information --- -- Session Information --- Session # :4 Database :etl User :a # Statements:1 Prepared Statement Information: Stmt ID SQLText - --- SYSLH0001 SELECT * FROM APP.PROFILES Session # :19 - # Connection Threads : 2 # Active Sessions : 2 # Waiting Sessions : 0 Total Memory : 3042443264 Free Memory : 153934376 So I'm not sure what heap size it is reaching. Regards, -Chris -Original Message- From: Bryan Pendleton [mailto:bpendleton.de...@gmail.com] Sent: 06 June 2014 12:32 To: Derby Discussion Subject: Re: Apache Derby - Locks up. It's interesting that you are trying to set it to use 6 G, and yet in your other message there was the line: Total Memory : 1756889088 Free Memory : 306272128 which seems to indicate that it's only using 1.7 GB. Like maybe you're running a 32 bit JVM, not a 64 bit JVM, somehow? bryan
Apache Derby - Locks up.
Hi, We are looking to use Apache Derby (Network Server) as a Caching Layer for our ETL process. Unfortunately we are finding that it locks up quite frequently. We have two tables which have around 10 million rows, two indexes in each. We can be reading (straight SELECT * FROM) or writing updates when it happens. CPU will spike to 100% (its on a rather powerful box) and then all existing and new JDBC clients are unable to connect. Running runtimeinfo (when it locks up, issuing this command can take a few minutes to get a response): --- Derby Network Server Runtime Information --- -- Session Information --- Session # :116 Database :etl User : abc # Statements:1 Prepared Statement Information: Stmt ID SQLText - --- SYSLH0001 SELECT * FROM APP.USERS Session # :117 - # Connection Threads : 4 # Active Sessions : 2 # Waiting Sessions : 0 Total Memory : 1756889088 Free Memory : 306272128 No errors can be seen in the log. I am rather confused as it seems like the perfect solution Derby just locks up. Thoughts or advise appreciated. OS: Windows 8.1 Java Runtime: 1.8.0_05-b13 Derby: 10.10.2.0 Regards, -Chris
