On 08/03/11 18:45, Frank Budinsky wrote:
Hi Andy,
I never actually tried to run it past 100,000 graphs, so I don't know what
will happen.
I just finished running it with -xmx1200m, and as you thought it should, it
did run (with 100,000 graphs) to completion. It took just under 4 hours to
run on my Thinkpad T61 laptop running Windows XP and with 3G of RAM. I
noticed that it was loading about 700 graphs/minute at the start, around
550 graphs/minute though most of the run (up till about 90,000 graphs), but
then got significantly slower towards the end (i.e., only about 55
graphs/minute for the last 1000). You mentioned that your JVM was 1.6 G. Is
that something you can config? I noticed that the total memory of my java
process peaked at about 1.2 G.
No - it might just be an artifact of the using a 64 bit JVM (openjdk).
The heap size (max memory) was 1044M in main() I think.
I'll try running the test again with 500,000 graphs to see if it gets all
the way through. Since it runs so slow, I'll let it run overnight and see
what happens.
It was fatser my my machine but I was using a desktop and disk speeds on
portables are notoriously slow for database use.
How long does it take to load the 100,000 graphs for you? I assume this
runs much faster on your hardware.
573.87 seconds and one cup of coffee running inside Eclipse.
My desktop is a quad core i5 CPU 760 @ 2.80GHz, ext4 filing system in
Ubuntu 10.10. SATA IDE disk; 7200 RPM.
I'm wondering if there's a minimum
hardware requirement to run TDB, especially if it's being used to load tens
or hundreds of millions of triples? It would be nice to set expectations
for what kind of hardware is needed for this.
Yes but it's very hard to set expectations. If its doing subject ->
properties lookup, it scales much better than GROUP BY and BI queries.
SSDs are the next new thing.
The bulk loader is faster; and you can load on one machine and run
queries on another.
Andy
Thanks,
Frank.
Andy Seaborne<[email protected]> wrote on 03/08/2011 11:53:06
AM:
[image removed]
Re: OutOfMemoryError while loading datagraphs
Andy Seaborne
to:
jena-users
03/08/2011 11:53 AM
Please respond to jena-users
On 08/03/11 15:05, Frank Budinsky wrote:
I tried increasing the amount of memory, but that just increased the
number
of calls that succeed (e.g., 10000 vs 2000) before getting the
exception.
What sizes of heap are you using?
I've been experimenting with various heap sizes, noticing that the
bigger I
make it the longer it runs
before crashing. When using the the default (small) heap size (-xmx64m)
the
test program fails at about 5800 graphs. If I bump it all the way up to
-xmx1200m, like you did, I suspect I will also be able to run the test
to
completion (100,000 graphs), but it takes very long to run (more than 2
hours on my machine). I'm guessing this is also running much faster for
you?
Extrapolating from what you're saying it looks like I would need a heap
of
6G, or so, to hit my original target of 500,000 graphs (about 50M
triples
total). Does that sound right? That is, needing to run with such a huge
heap?
No - the caches are bounded. Once they reach steady state, there is no
further growth and no fixed scale limit. I'mve run a trial with 500,000
with -Mx1200M and it works. for me. JVM is 1.6G. The caches were still
filling up in in your test case.
The caches are LRU by slots, which is a bit crude for the node cache as
nodes vary in size. Index files have fixed used units (blocks - they
are 8Kbytes).
The default settings are supposed to work for heap of 1.2G -- it's what
the scripts set the heap to.
The caches were still filling up in in your test case.
Andy
Thanks,
Frank.
Andy Seaborne<[email protected]> wrote on 03/08/2011
09:23:50
AM:
[image removed]
Re: OutOfMemoryError while loading datagraphs
Andy Seaborne
to:
jena-users
03/08/2011 09:24 AM
Please respond to jena-users
(Frank sent me the detached file)
Frank,
I'm on a 64 bit machine, but I'm settign direct mode and limiting the
Java heap size with -Xmx
With a heap of 1200M, java reports 1066M max memory, and the test
runs.
With a heap of 500M, java reports 444M max memory, and the test stops
at
11800.
Things will be a little different for 32 bit but should be
approximately
the same. TDB is doing the same things.
Tweaking the block cache sizes (sorry, magic needed) down to 5000
(read,
default 10000) and 1000 (write, default 2000), it runs at 500M, but
slower.
There are quite a few files for named graphs so small changes in cache
size get multiplied (x12, I think).
I tried increasing the amount of memory, but that just increased the
number
of calls that succeed (e.g., 10000 vs 2000) before getting the
exception.
What sizes of heap are you using?
Andy
On 07/03/11 18:34, Frank Budinsky wrote:
Hi Andy,
I created a simple standalone test program that roughly simulates
what
my application is doing and it also crashes with the same
OutOfMemoryError exception. I've attached it here. Would it be
possible
for you to give it a try?
/(See attached file: TDBOutOfMemoryTest.java)/
Just change TDB_DIR to some new empty database location and run. It
get's the OutOfMemoryError at around 5800 graphs when I run it with
default VM params.
Thanks,
Frank.
Andy Seaborne<[email protected]> wrote on 03/02/2011
09:38:51 AM:
> [image removed]
>
> Re: OutOfMemoryError while loading datagraphs
>
> Andy Seaborne
>
> to:
>
> jena-users
>
> 03/02/2011 09:41 AM
>
> Please respond to jena-users
>
> Hi Frank,
>
> On 28/02/11 14:48, Frank Budinsky wrote:
> >
> > Hi Andy,
> >
> > I did some further analysis of my OutOfMemeoryError problem,
and
this is
> > what I've discovered. The problem seems to be that there is
one
instance of
> > class NodeTupleTableConcrete that contains an ever growing
set of
tuples
> > which eventually uses up all the available heap space and
then
crashes.
> >
> > To be more specific, this field in class TupleTable:
> >
> > private final TupleIndex[] indexes ;
> >
> > seems to contain 6 continually growing TupleIndexRecord
instances
> > (BPlusTrees). From my measurements, this seems to eat up
approximately 1G
> > of heap for every 1M triples in the Dataset (i.e., about 1K
per
datagraph).
> > So, to load my 100K datagraphs (~10M total triples) it would
seem
to need
> > 10G of heap space.
>
> There are 6 indexes for named graphs (see the files GSPO etc).
TDB
uses
> total indexing which puts a lot of work at load time but means
any
> lookup needed is always done with an index scan. The code can
run
with
> less indexes - the minimum is one - but that is no exposed in
the
> configuration.
>
> Each index holds quads (4 NodeIds, a NodeId is 64 bits on disk).
As
the
> index grows the data goes to disk. There is a finite LRU cache
in
front
> of each index.
>
> Does your dataset have a location? If has no location, it's all
> in-memory with a RAM-disk like structure. This is for
small-scale
> testing only - it really does read and write blocks out of the
RAM
disk
> by copy to give strict disk-like semantics.
>
> There is also a NodeTable mapping between NodeId and Node
(Jena's
> graph-level RDF Term class). This has a cache in front of it .
> readPropertiesFile
> The long-ish literals maybe the problem. The node table cache is
> fixed-number, not bounded by size.
>
> The sizeof the caches are controlled by:
>
> SystemTDB.Node2NodeIdCacheSize
> SystemTDB.NodeId2NodeCacheSize
>
> These are not easy to control but either (1) get the source code
and
> alter the default values (2) see the code in SystemTDB that uses
a
> properties file.
>
> If you can end me a copy of the data, I can try loading it here.
>
> > Does this make sense? How is it supposed to work? Shouldn't
the
triples
> > from previously loaded named graphs be eligable for GC when
I'm
loading the
> > next named graph? Could it be that I'm holding onto something
that's
> > preventing GC in the TupleTable?
> >
> > Also, after looking more carefully at the resources being
indexed,
I
> > noticed that many of them do have relatively large literals
(100s
of
> > characters). I also noticed that when using Fuseki to load
the
resources I
> > get lots of warning messages like this, on the console:
> >
> > Lexical form 'We are currently doing
> > this:<br></br><br></br>workspaceConnection.replaceComponents
> > (replaceComponents, replaceSource, falses, false,
> > monitor);<br></br><br></br>the new way of doing it would be
something
> >
like:<br></br><br></br><br></br>
> > ArrayList<IComponentOp> replaceOps = new
> > ArrayList<IComponentOp>();<br></
> br>
> > for (Iterator iComponents = components.iterator();
iComponents.hasNext();)
> >
{<br></br>
> > IComponentHandle componentHandle = (IComponentHandle)
iComponents.next
> > ();<br></
>
br>
> > replaceOps.add(promotionTargetConnection.componentOpFactory
> > ().replaceComponent
> > (componentHandle,<br></
>
br>
> > buildWorkspaceConnection,
> > false));<br></br> }
> <br></br><br></br>
> > promotionTargetConnection.applyComponentOperations
(replaceOps,
monitor);'
> > not valid for datatype
> > http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral
> >
> > Could this be part of the problem?
>
> No - it's a different issue. This is something coming from the
parser.
>
> RDF XMLLiterals have special rules - they must follow
> exclusive canonical XML, which means, amongst a lot of other
thigs,
they
> have to be a single XML node. The rules for exclusive Canonical
XML
are
> really quite strict (e.g. attributes in alphabetical order).
>
> http://www.w3.org/TR/rdf-concepts/#section-XMLLiteral
>
> If you want to store XML or HTML fragments, you can't use RDF
> XMLLiterals very easily - you have to mangle them to conform to
the
> rules. I suggest either as strings or invent your own datatype.
>
> You can run the parser on it's own using
> "riotcmd.riot --validate FILE ..."
>
>
> Andy
>
> >
> > Thanks,
> > Frank.
