Hi, My machine paramters:
Memory: MemTotal: 33029940 kB SwapTotal: 2096472 kB Operating System: Linux 2.6.18-164.15.1.el5 #1 SMP Mon Mar 1 10:56:08 EST 2010 x86_64 GNU/Linux 8 Processors: vendor_id : AuthenticAMD cpu family : 15 model : 65 model name : Dual-Core AMD Opteron(tm) Processor 8212 stepping : 2 cpu MHz : 2000.190 cache size : 1024 KB cpu cores : 2 apicid : 7 fpu : yes fpu_exception : yes cpuid level : 1 wp : yes ..... I am using 1.1-SNAPSHOT of neo4j kernel java -version java version "1.6.0_17" Java(TM) SE Runtime Environment (build 1.6.0_17-b04-248-9M3125) Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01-101, mixed mode) I set parameter: ((LuceneIndexService) indexService).enableCache( "uri", 500000 ); and run it with MAVEN_OPTS=-Xmx2048m size of the db files: -rw-r--r-- 1 ludab users 11 May 6 12:47 active_tx_log drwxr-xr-x 4 ludab users 4096 May 11 16:22 lucene drwxr-xr-x 2 ludab users 4096 May 11 16:22 lucene-fulltext -rw-r--r-- 1 ludab users 42 May 11 16:22 neostore -rw-r--r-- 1 ludab users 9 May 11 16:22 neostore.id -rw-r--r-- 1 ludab users 1302400087 May 11 16:22 neostore.nodestore.db -rw-r--r-- 1 ludab users 9 May 11 16:22 neostore.nodestore.db.id -rw-r--r-- 1 ludab users 1798672920 May 11 16:22 neostore.propertystore.db -rw-r--r-- 1 ludab users 158 May 11 16:22 neostore.propertystore.db.arrays -rw-r--r-- 1 ludab users 9 May 11 16:22 neostore.propertystore.db.arrays.id -rw-r--r-- 1 ludab users 9 May 11 16:22 neostore.propertystore.db.id -rw-r--r-- 1 ludab users 74 May 11 16:22 neostore.propertystore.db.index -rw-r--r-- 1 ludab users 9 May 11 16:22 neostore.propertystore.db.index.id -rw-r--r-- 1 ludab users 370 May 11 16:22 neostore.propertystore.db.index.keys -rw-r--r-- 1 ludab users 9 May 11 16:22 neostore.propertystore.db.index.keys.id -rw-r--r-- 1 ludab users 15728535205 May 11 16:22 neostore.propertystore.db.strings -rw-r--r-- 1 ludab users 9 May 11 16:22 neostore.propertystore.db.strings.id -rw-r--r-- 1 ludab users 13276183857 May 11 16:22 neostore.relationshipstore.db -rw-r--r-- 1 ludab users 9 May 11 16:22 neostore.relationshipstore.db.id -rw-r--r-- 1 ludab users 9323 May 11 16:22 neostore.relationshiptypestore.db -rw-r--r-- 1 ludab users 9 May 11 16:22 neostore.relationshiptypestore.db.id -rw-r--r-- 1 ludab users 275570 May 11 16:22 neostore.relationshiptypestore.db.names -rw-r--r-- 1 ludab users 9 May 11 16:22 neostore.relationshiptypestore.db.names.id -rw-r--r-- 1 ludab users 4 May 11 16:22 nioneo_logical.log.active -rw-r--r-- 1 ludab users 29827 May 6 12:47 tm_tx_log.1 -rw-r--r-- 1 ludab users 0 May 11 16:19 tm_tx_log.2 > I've seen the same kind of performance. Adding a KB of ~6.5m triples took > about 21 hours. > > This was using Java 1.6, w/ 8G of RAM to the heap, no other JVM settings. > > I ran it on a new iMac, OSX 10.6.2, 2.8 Ghz Intel i7, 16G RAM using the > 1.1 > SNAPSHOT version of the kernel using code copied from your site. I don't > have sizes on the files created after the data was inserted. The inserts > were done through the Sesame Sail interface to the database. > > Cheers, > > Mike > > On Fri, May 7, 2010 at 5:07 AM, Johan Svensson > <[email protected]>wrote: > >> Hi, >> >> Adding a literal (with average size around ~400 bytes if the numbers >> are correct) should not result in such a big difference of injection >> times. >> >> Could you give some more information regarding setup so we can track >> down the cause of this. Good things to know would be: >> >> o java version and jvm switches/configuration (heap size, gc settings >> etc) >> o amount of RAM on the machine >> o neo4j kernel version (1.1-SNAPSHOT or 1.0?) >> o configuration passed into the neo4j kernel if any >> o size of the different store files in the neo4j db directory (after >> injection of data) >> >> Regards, >> Johan >> >> On Fri, May 7, 2010 at 10:26 AM, Mattias Persson >> <[email protected]> wrote: >> > I'm not sure why it's taking such a long time, it shouldn't be that >> slow. >> > I'll see if I can get time to look into that... but I don't know when >> > exactly. Have you ran it through a profiler yourself? I think that's a >> good >> > idea since I'm a little locked up right now :) >> > >> > Maybe the answer is obvious once I can look at such an output from a >> > profiler! >> > >> > 2010/5/6 Lyudmila L. Balakireva <[email protected]> >> > >> >> Hi, >> >> >> >> I am testing the neo4j with dbpedia data. When I am loading >> triples >> in >> >> the form (URI,URI,URI) >> >> the speed is good ( 10 mln in 2 min). The loading URI,URI,LITERAL is >> very >> >> slow. The Literal is mainly chunks of text. >> >> For example shortabstract_en.nt with 2943434 records took 6.4 hrs >> to >> >> load. The file size is 1194900107. >> >> >> >> Is it possible to optimize the loading of literals or it is better to >> >> store abstracts somewhere else and keep pointer to some verbose text >> in >> >> neo4j? >> >> What is casing the dramatic difference in loading of the verbose >> >> literals? >> >> >> >> Thanks, Luda >> _______________________________________________ >> Neo mailing list >> [email protected] >> https://lists.neo4j.org/mailman/listinfo/user >> > _______________________________________________ > Neo mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ Neo mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
