Is there a bug open related to this that I can watch? As our customers' databases grow, I'm concerned we'll run into the same issue. We normally run with a thread stack size of 128k, because our application can have in excess of 200 threads, so requiring a stack size of 2mb for compression is not desirable.
Sent from my iPhone On Feb 18, 2012, at 8:11, Mike Matrigali <[email protected]> wrote: > Matthew McCawley wrote: >> I've run into the same issue as Adriano when running on a single, large table >> about 1.4 GB in size. I enable autocommit before the compress statement and >> disable it after. I have encountered the error when deleting portions of the >> data as well as all of it. I also found that the compression would succeed >> if I used a stack size of 2 MB and a maximum heap size of 1 GB (-Xss2048k >> -Xmx1g). I'll be working with this more next week, so I'll see if anything >> changes when working with a larger dataset. > You see the same kind of repeated stack in the error? This loop looks > strange to me, and I don't think should be related to size of the tables: > at > org.apache.derby.iapi.store.raw.xact.RawTransaction.notifyObservers(Unknown > Source) > at org.apache.derby.impl.store.raw.data.DropOnCommit.update(Unknown Source) > at java.util.Observable.notifyObservers(Observable.java:142) > at > org.apache.derby.iapi.store.raw.xact.RawTransaction.notifyObservers(Unknown > Source) > at org.apache.derby.impl.store.raw.data.DropOnCommit.update(Unknown Source) > at java.util.Observable.notifyObservers(Observable.java:142) > at > org.apache.derby.iapi.store.raw.xact.RawTransaction.notifyObservers(Unknown > Source) > at org.apache.derby.impl.store.raw.data.DropOnCommit.update(Unknown Source) > at java.util.Observable.notifyObservers(Observable.java:142) > at > org.apache.derby.iapi.store.raw.xact.RawTransaction.notifyObservers(Unknown > Source) > at org.apache.derby.impl.store.raw.data.DropOnCommit.update(Unknown Source) > at java.util.Observable.notifyObservers(Observable.java:142) > at > org.apache.derby.iapi.store.raw.xact.RawTransaction.notifyObservers(Unknown > Source) > > > There are some reported problems with the amount of memory in general that > compres table uses, which are likely to be a different issue. For > these memory issues it is helpful to post exactly what jvm you are using, > what OS, and what flags you are giving the jvm. And how much memory is on > your machine. > > Derby was not originally created with vldb in mind, so multi-gigabyte tables > could very well be exercising new code paths. Derby definitely > has the ability to perform index creations/sorts on tables bigger than > memory size, but there are some reported problems in its estimates of > how much memory it should use to do so. These estimates can definitely > be affected by jvm startup flags.
