Re: Loading dbpedia.nq: exception in thread main gc overhead limit exceeded

[Andy Seaborne]

On 21/12/13 03:19, Rose Beck wrote:
I tried setting using
export JAVA_OPTS ="-Xms32768m -Xmx32768m"

Try JVM_ARGS

Also, don't set it to 32G - try about 2-4G.

export JVM_ARGS="-Xmx2048m"

A lot of the space TDB loading needs is not in the heap, it's memory 
mapped files which are outside the java heap.  If you set the heap 
large, it will slow down because not much space is left for OS file caching.

        Andy


the maximum heap size to 32 GB. Still I am getting the same error.  I am
not able to understand as to where am I making the error?


On Sat, Dec 21, 2013 at 6:09 AM, Rose Beck <rosebeck...@gmail.com> wrote:

Hi,

I am trying to load dbpedia in n-quads form (from BTC-2012 dataset). I am
using apache-jena-2.10.0. And for loading I used the following command (in
ubuntu 12.04 server, with :
$ cd apache-jena-2.10.0
$
./bin/tdbloader
--loc=/home/
jenaDbpedia -v /home/jyotil/dbpedia.nq

However, after loading a significant number of quads I am getting the
following error:
exception in thread main gc overhead limit exceeded?
          at
com.hp.hpl.jena.tdb.index.bplustree.BPTreeNode.get(BPTreeNode.java:154)
         at
com.hp.hpl.jena.tdb.index.bplustree.BPTreeNode.internalInsert(BPTreeNode.java:447)
         at
com.hp.hpl.jena.tdb.index.bplustree.BPTreeNode.insert(BPTreeNode.java:212)
         at
com.hp.hpl.jena.tdb.index.bplustree.BPlusTree.addAndReturnOld(BPlusTree.java:328)
         at
com.hp.hpl.jena.tdb.index.bplustree.BPlusTree.add(BPlusTree.java:320)
         at
com.hp.hpl.jena.tdb.index.TupleIndexRecord.performAdd(TupleIndexRecord.java:60)
         at
com.hp.hpl.jena.tdb.index.TupleIndexBase.add(TupleIndexBase.java:64)
         at com.hp.hpl.jena.tdb.index.TupleTable.add(TupleTable.java:96)
         at
com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.addRow(NodeTupleTableConcrete.java:87)
         at
com.hp.hpl.jena.tdb.store.bulkloader.LoaderNodeTupleTable.load(LoaderNodeTupleTable.java:110)
         at
com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader$DestinationDSG.process(BulkLoader.java:265)
         at
com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader$DestinationDSG.quad(BulkLoader.java:257)
         at
org.apache.jena.riot.lang.LangNQuads.runParser(LangNQuads.java:56)
         at org.apache.jena.riot.lang.LangBase.parse(LangBase.java:42)
         at org.apache.jena.riot.RiotReader.parse(RiotReader.java:116)
         at org.apache.jena.riot.RiotReader.parse(RiotReader.java:93)
         at org.apache.jena.riot.RiotReader.parse(RiotReader.java:66)
         at
com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadQuads$(BulkLoader.java:160)
         at
com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadDataset(BulkLoader.java:121)
         at com.hp.hpl.jena.tdb.TDBLoader.loadDataset$(TDBLoader.java:284)
         at com.hp.hpl.jena.tdb.TDBLoader.loadDataset(TDBLoader.java:197)
         at com.hp.hpl.jena.tdb.TDBLoader.load(TDBLoader.java:76)


Can someone please help me as to where am I going wrong?