Matthew McCawley wrote:
I've run into the same issue as Adriano when running on a single, large table
about 1.4 GB in size. I enable autocommit before the compress statement and
disable it after. I have encountered the error when deleting portions of the
data as well as all of it. I also found that the compression would succeed
if I used a stack size of 2 MB and a maximum heap size of 1 GB (-Xss2048k
-Xmx1g). I'll be working with this more next week, so I'll see if anything
changes when working with a larger dataset.
You see the same kind of repeated stack in the error? This loop looks
strange to me, and I don't think should be related to size of the tables:
at
org.apache.derby.iapi.store.raw.xact.RawTransaction.notifyObservers(Unknown
Source)
at org.apache.derby.impl.store.raw.data.DropOnCommit.update(Unknown Source)
at java.util.Observable.notifyObservers(Observable.java:142)
at
org.apache.derby.iapi.store.raw.xact.RawTransaction.notifyObservers(Unknown
Source)
at org.apache.derby.impl.store.raw.data.DropOnCommit.update(Unknown Source)
at java.util.Observable.notifyObservers(Observable.java:142)
at
org.apache.derby.iapi.store.raw.xact.RawTransaction.notifyObservers(Unknown
Source)
at org.apache.derby.impl.store.raw.data.DropOnCommit.update(Unknown Source)
at java.util.Observable.notifyObservers(Observable.java:142)
at
org.apache.derby.iapi.store.raw.xact.RawTransaction.notifyObservers(Unknown
Source)
at org.apache.derby.impl.store.raw.data.DropOnCommit.update(Unknown Source)
at java.util.Observable.notifyObservers(Observable.java:142)
at
org.apache.derby.iapi.store.raw.xact.RawTransaction.notifyObservers(Unknown
Source)
There are some reported problems with the amount of memory in general
that compres table uses, which are likely to be a different issue. For
these memory issues it is helpful to post exactly what jvm you are
using, what OS, and what flags you are giving the jvm. And how much
memory is on your machine.
Derby was not originally created with vldb in mind, so multi-gigabyte
tables could very well be exercising new code paths. Derby definitely
has the ability to perform index creations/sorts on tables bigger than
memory size, but there are some reported problems in its estimates of
how much memory it should use to do so. These estimates can definitely
be affected by jvm startup flags.
