On 25/04/15 15:05, Daniel Hernández wrote:
El 2015-04-25 10:50, Andy Seaborne escribió:
...
how far did it get?
I was at 12 millions of triples.
$ tail tdb-03-dout.log
12:21:19 INFO loader :: Add: 11,950,000 triples (Batch:
2,033 / Avg: 23,940)
12:21:50 INFO loader :: Add: 12,000,000 triples (Batch:
1,619 / Avg: 22,640)
12:21:50 INFO loader :: Elapsed: 530.03 seconds
[2015/04/25 12:21:50 UTC]
12:22:14 INFO loader :: Add: 12,050,000 triples (Batch:
2,040 / Avg: 21,729)
12:22:48 INFO loader :: Add: 12,100,000 triples (Batch:
1,485 / Avg: 20,571)
12:23:24 INFO loader :: Add: 12,150,000 triples (Batch:
1,367 / Avg: 19,447)
12:24:03 INFO loader :: Add: 12,200,000 triples (Batch:
1,289 / Avg: 18,386)
12:24:56 INFO loader :: Add: 12,250,000 triples (Batch:
940 / Avg: 17,092)
12:26:07 INFO loader :: Add: 12,300,000 triples (Batch:
708 / Avg: 15,624)
12:27:50 INFO loader :: Add: 12,350,000 triples (Batch:
482 / Avg: 13,864)
Daniel,
Than I'm baffled as to where the space is going. While the 15e6
predicates is unusual, a 10G should be wildly too large and I don't see
how it affects the loading at the point shown.
For tdbloader2, it seems to be the stats space - that would be affected
by 15e6 predciates. A very large heap should merely slow down loading,
not still run out of space.(I'll look at adding a "--no-stats" flag anyway)
Could I get a copy of your data to try up in my development setup with a
profiler?
Thanks
Andy
