[
https://issues.apache.org/jira/browse/JENA-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222045#comment-13222045
]
Sarven Capadisli commented on JENA-217:
---------------------------------------
Thanks Andy.
Just to confirm that the filesystem is ext4.
16 GB of RAM, however I'd say that the available free is from 4-6GB. The swap
is 16GB.
$ java -version
java version "1.6.0_20"
OpenJDK Runtime Environment (IcedTea6 1.9.13) (6b20-1.9.13-0ubuntu1~10.04.1)
OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
I'll give tdbloader one more go because this time around it was going at a
pretty decent speed (in comparison to my previous imports - I don't have a
reason for this either). It took about 13 hours before breaking. I think there
was another hour to go before finishing up.
The only issue I have with switching over to tdbloader2 or 3 is that I have to
add the context URI to my N-Triples for N-Quads.
> tdbloader : Unsafe memory access operation
> ------------------------------------------
>
> Key: JENA-217
> URL: https://issues.apache.org/jira/browse/JENA-217
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB
> Affects Versions: Fuseki 0.2.1
> Environment: Ubuntu 10.04
> Reporter: Sarven Capadisli
>
> First of all, I'm not sure if this is necessarily a bug or whether my system
> encountered some other issue. Here it is:
> I'm trying to import world-development-indicators.nt which is 68 GB into a
> TDB store.
> I've performed the following operation:
> java tdb.tdbloader --loc /usr/lib/fuseki/DB/DB.worldbank2
> world-development-indicators
> --graph=http://worldbank.270a.info/graph/world-bank-indicators
> /data/worldbank.270a.info/indicators/en/indicator/world-bank-indicators.nt
> ..
> 360,393,432 triples loaded in 15,956.80 seconds [Rate: 22,585.57 per second]
> ..
> ** Index GSPO->GPOS: 360,202,756 slots indexed in 18,194.61 seconds [Rate:
> 19,797.23 per second]
> ..
> ** Index GSPO->GOSP: 360,202,756 slots indexed in 4,410.64 seconds [Rate:
> 81,666.85 per second]
> ..
> ** Index GSPO->POSG: 360,202,756 slots indexed in 4,354.58 seconds [Rate:
> 82,718.05 per second]
> ..
> ** Index GSPO->OSPG: 360,202,756 slots indexed in 4,084.50 seconds [Rate:
> 88,187.73 per second]
> ..
> Index GSPO->SPOG: 113,900,000 slots (Batch: 213,675 slots/s / Avg: 113,695
> slots/s)
> Exception in thread "main" java.lang.InternalError: a fault occurred in a
> recent unsafe memory access operation in compiled Java code
> at com.hp.hpl.jena.tdb.lib.TupleLib.tuple(TupleLib.java:208)
> at
> com.hp.hpl.jena.tdb.index.TupleIndexRecord$1.convert(TupleIndexRecord.java:195)
> at
> com.hp.hpl.jena.tdb.index.TupleIndexRecord$1.convert(TupleIndexRecord.java:191)
> at org.openjena.atlas.iterator.Iter$4.next(Iter.java:293)
> at
> com.hp.hpl.jena.tdb.store.bulkloader.LoaderNodeTupleTable.copyIndex(LoaderNodeTupleTable.java:209)
> at
> com.hp.hpl.jena.tdb.store.bulkloader.BuilderSecondaryIndexesSequential.createSecondaryIndexes(BuilderSecondaryIndexesSequential.java:43)
> at
> com.hp.hpl.jena.tdb.store.bulkloader.LoaderNodeTupleTable.createSecondaryIndexes(LoaderNodeTupleTable.java:192)
> at
> com.hp.hpl.jena.tdb.store.bulkloader.LoaderNodeTupleTable.loadSecondaryIndexes(LoaderNodeTupleTable.java:91)
> at
> com.hp.hpl.jena.tdb.store.bulkloader.LoaderNodeTupleTable.loadIndexStart(LoaderNodeTupleTable.java:144)
> at
> com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader$1.finish(BulkLoader.java:232)
> at
> com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadTriples$(BulkLoader.java:141)
> at
> com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadNamedGraph(BulkLoader.java:107)
> at com.hp.hpl.jena.tdb.TDBLoader.loadNamedGraph$(TDBLoader.java:271)
> at com.hp.hpl.jena.tdb.TDBLoader.loadGraph$(TDBLoader.java:246)
> at com.hp.hpl.jena.tdb.TDBLoader.loadGraph(TDBLoader.java:177)
> at com.hp.hpl.jena.tdb.TDBLoader.load(TDBLoader.java:112)
> at tdb.tdbloader.loadNamedGraph(tdbloader.java:157)
> at tdb.tdbloader.exec(tdbloader.java:142)
> at arq.cmdline.CmdMain.mainMethod(CmdMain.java:97)
> at arq.cmdline.CmdMain.mainRun(CmdMain.java:59)
> at arq.cmdline.CmdMain.mainRun(CmdMain.java:46)
> at tdb.tdbloader.main(tdbloader.java:53)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
