when assigning 12GB as Xmx, the JVM went to 9.1G, and then the out of memory. when assigning 15GB on the 16GB machine, the JVM went to 10.6GB and then the exception came. I am not sure how many GB does such a query need in H2? Is there a possibility to enhance such queries to use
Cheers, Kais On Mon, Nov 5, 2012 at 11:02 AM, Kais Haddadin <[email protected]>wrote: > > Hi, > > sorry about this, but I think there was some confusions about the versions > of the H2 used in my test. It seems that there was older versions involved. > I redid the test completely with version 169 and I could connect to the > database after the load is finished. In my program, am trying to make a big > query and am getting out of memory exception, my query looks like this, on > 6 million records: > > select "Col1",...,"Col50","Measures",SUM("Measure51") as > "Measure51",...,SUM("Measure250") as "Measure250" from > "sampleNorm_1"."NormalizedData" group by "Col1",...,"Col50","Measures" > > the result should not be a lot, since each column in the group by clause > include small number of distinct records. > > I redid the test with H2 console, I assigned 10GB as Xmx, I still get out > of memory exception,the weird part,the java goes only up to 7.5 G, I have > 16G on the machine and there still more RAM free for the JVM. I'm not sure > if this is something that H2 can do something about it. But For me I can > simply not perform this query regardless how much Memory I assign for the > JVM. Do you think H2 team can help me in this? I can provide the database > for testing if needed. > > Thanks a lot, > > Kais > > > On Fri, Nov 2, 2012 at 4:54 PM, Thomas Mueller < > [email protected]> wrote: > >> Hi, >> >> What database URL do you use? >> >> What happens during those 10-15 minutes, could you get a few full thread >> dumps? >> >> How exactly did you load the database? You wrote "I am trying to load big >> table" but when I read your mail I'm not sure: did you finish loading the >> database, and now try to access it (read from it)? >> >> By the way, according to the error code, you are actually using H2 >> version 1.3.169 (but that's no a problem). >> >> Regards, >> Thomas >> >> >> On Fri, Nov 2, 2012 at 4:42 PM, kais haddadin <[email protected]>wrote: >> >>> Hello! >>> >>> I am trying to load big table: 250 columns and 6 million lines in H2. It >>> loads (or I think at least it is because the db is now 11.5GB), but It >>> gives me always out of memory as soon as I try to make a query on it. I >>> stopped my program andI tried to access the H2 database outside my own >>> program and through the commandline tool and after 10-15minutes of waiting, >>> I get this error: >>> >>> SQL Exception: Allgemeiner Fehler: "java.lang.RuntimeException: >>> old!=record pos: >>> 516 old:page[516] data leaf table:0 SYS entries:1 parent:514 keys:[5] >>> offsets:[1 >>> 947] new:page[516] data leaf table:8 LOBS entries:0 parent:0 keys:null >>> offsets:n >>> ull" >>> General error: "java.lang.RuntimeException: old!=record pos:516 >>> old:page[516] da >>> ta leaf table:0 SYS entries:1 parent:514 keys:[5] offsets:[1947] >>> new:page[516] d >>> ata leaf table:8 LOBS entries:0 parent:0 keys:null offsets:null"; SQL >>> statement: >>> >>> CREATE CACHED TABLE IF NOT EXISTS INFORMATION_SCHEMA.LOBS(ID BIGINT >>> PRIMARY KEY, >>> BYTE_COUNT BIGINT, TABLE INT) HIDDEN [50000-169] >>> [Enter] jdbc:h2:... >>> >>> I am using version 1.3.168. I can't attach the database since it is very >>> big. >>> Is this a result of the "out of memory" exception I am getting when >>> doing the big query? how can I make this work? >>> >>> Thanks in advance, >>> Kais >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "H2 Database" group. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msg/h2-database/-/6NYu5xiRtO8J. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group at >>> http://groups.google.com/group/h2-database?hl=en. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "H2 Database" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/h2-database?hl=en. >> > > -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/h2-database?hl=en.
