And the Virtuoso config is a single instance: OpenLink Virtuoso version 06.01.3127, on Linux (x86_64-unknown-linux-gnu), Single Edition
There are currently about 300M triples in the store. I read that 500M per 16GB or RAM should still run ok, so I don't think a clustered version should be necessary? Performing Sparql queries still is very fast, but adding / deleting triples seems to be very slow. On Fri, Sep 16, 2011 at 2:26 PM, karel braeckman <[email protected]> wrote: > > Forgot to mention the machine details: > > 24GB RAM > 2x quadcore Xeon E5540 2.5GHz > Virtuoso data is on SSD disks > > Regards, > Karel > > On Fri, Sep 16, 2011 at 1:57 PM, karel braeckman <[email protected]> > wrote: >> >> Hi all, >> We tried setting up DBpedia Live on one of our local servers and keep it in >> sync with the official DBpedia live, using the dbpintegrator tool. Our setup >> is documented here: http://medialoep.vrtmedialab.be/blog/?p=96. >> However, adding / deleting triples from / to Virtuoso has become extremely >> slow. I was pointed in the direction >> of http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtRDFPerformanceTuning by @kidehen >> and @pkleef. I changed the following parameters in Virtuoso's .ini file >> from their default value: >> [Database] >> MaxCheckpointRemap = 1000000 >> [Parameters] >> NumberOfBuffers = 1360000 >> MaxDirtyBuffers = 1000000 >> I can see no real difference in performance though, Virtuoso gets stuck on a >> DELETE query (it is busy with it for the last 125 minutes at 100% CPU :-S). >> As I am not at all an expert on Virtuoso tuning, all suggestions are welcome. >> For completeness, I have included the output of status('rhck'); in isql >> below. >> Best regards, >> Karel >> ---------------------------------------------- >> >> OpenLink Virtuoso Server >> >> Version 06.01.3127-pthreads for Linux as of Mar 16 2011 >> >> Started on: 2011/09/16 11:35 GMT+120 >> >> >> >> Database Status: >> >> File size 56727961600, 6924800 pages, 2771949 free. >> >> 1360000 buffers, 47404 used, 2 dirty 0 wired down, repl age 0 0 w. io 40 >> w/crsr. >> >> Disk Usage: 48335 reads avg 0 msec, 0% r 0% w last 254 s, 10448 writes, >> >> 136 read ahead, batch = 230. Autocompact 22 in 12 out, 43% saved. >> >> Gate: 391 2nd in reads, 0 gate write waits, 0 in while read 0 busy scrap. >> >> Log = virtuoso.trx, 4003 bytes >> >> 4151261 pages have been changed since last backup (in checkpoint state) >> >> Current backup timestamp: 0x0000-0x00-0x00 >> >> Last backup date: unknown >> >> Clients: 1 connects, max 1 concurrent >> >> RPC: 65 calls, 1 pending, 1 max until now, 0 queued, 0 burst reads (0%), 0 >> second brk=11304103936 >> >> Checkpoint Remap 1159 pages, 0 mapped back. 125 s atomic time. >> >> DB master 6924800 total 2771949 free 1159 remap 0 mapped back >> >> temp 768 total 763 free >> >> >> >> Lock Status: 0 deadlocks of which 0 2r1w, 0 waits, >> >> Currently 2 threads running 0 threads waiting 0 threads in vdb. >> >> Pending: >> >> >> >> Client 1111:1: Account: dba, 361734 bytes in, 3237 bytes out, 17 stmts. >> >> Transaction status: PENDING, 1 threads. >> >> Locks: >> >> >> >> >> >> Running Statements: >> >> Time (msec) Text >> >> 7797190 sparql DELETE FROM <http://live.dbpedia.org> { >> >> <http://dbpedia.org/resource/T >> >> >> >> >> >> Replication Status: Server db-UBUNTU. >> >> db-MUSSORGSKY db-MUSSORGSKY 0 OFF. >> >> db-UBUNTU db-UBUNTU 0 OFF. >> >> >> >> >> >> Index Usage: >> >> Table Index Touches Reads %Miss Locks Waits %W n-dead >> >> DB.DBA.RDF_PREFIX RDF_PREFIX 3 95 2375% 6 0 0% 0 >> >> DB.DBA.RDF_PREFIX DB_DBA_RDF_PREFIX_UNQC_RP_ID 2193 4 0% 3 0 0% 0 >> >> DB.DBA.RDF_IRI RDF_IRI 40 4670 11390% 80 0 0% 0 >> >> DB.DBA.RDF_IRI DB_DBA_RDF_IRI_UNQC_RI_ID 353 8 2% 40 0 0% 0 >> >> DB.DBA.RDF_QUAD RDF_QUAD 953949641 13253 0% 2445 0 0% 0 >> >> DB.DBA.RDF_QUAD RDF_QUAD_SP 5596 447 7% 465 0 0% 0 >> >> DB.DBA.RDF_QUAD RDF_QUAD_POGS 1512 13514 893% 829 0 0% 0 >> >> DB.DBA.RDF_QUAD RDF_QUAD_GS 564 500 88% 80 0 0% 0 >> >> DB.DBA.RDF_QUAD RDF_QUAD_OP 152 942 615% 152 0 0% 0 >> >> DB.DBA.RDF_OBJ RDF_OBJ 3151 4134 131% 325 0 0% 0 >> >> DB.DBA.RDF_OBJ RO_VAL 954 2319 242% 137 0 0% 0 >> >> DB.DBA.RO_START RO_START 35 64 177% 35 0 0% 0 >> >> DB.DBA.RDF_DATATYPE RDF_DATATYPE 230 1 0% 238 0 0% 0 >> >> DB.DBA.RDF_LANGUAGE RDF_LANGUAGE 840 1 0% 0 0 0% 0 >> >> DB.DBA.RDF_LANGUAGE DB_DBA_RDF_LANGUAGE_UNQC_RL_TWOBYTE 334 1 0% 344 0 0% 0 >> >> DB.DBA.RDF_OBJ_FT_RULES RDF_OBJ_FT_RULES 11 1 8% 14 0 0% 0 >> >> DB.DBA.RDF_GRAPH_GROUP RDF_GRAPH_GROUP_IRI 2 1 33% 1 0 0% 0 >> >> WS.WS.SYS_DAV_COL SYS_DAV_COL 13862 1 0% 20890 0 0% 0 >> >> WS.WS.SYS_DAV_COL SYS_DAV_COL_ID 221 1 0% 46 0 0% 0 >> >> WS.WS.SYS_DAV_RES SYS_DAV_RES 17368 164 0% 4409 0 0% 0 >> >> WS.WS.SYS_DAV_RES SYS_DAV_RES_COL 3242 4 0% 3609 0 0% 0 >> >> WS.WS.SYS_DAV_RES SYS_DAV_RES_FULL_PATH 7484 4 0% 4961 0 0% 0 >> >> WS.WS.SYS_DAV_RES SYS_DAV_RES_IID 3 1 25% 3 0 0% 0 >> >> WS.WS.SYS_DAV_RES_TYPES SYS_DAV_RES_TYPES 5 1 16% 10 0 0% 0 >> >> DB.DBA.HTTP_PATH HTTP_PATH 10 1 9% 11 0 0% 0 >> >> DB.DBA.VSPX_SESSION VSPX_SESSION 484 1 0% 522 0 0% 0 >> >> VAD.DBA.VAD_REGISTRY VAD_REGISTRY 1664 3 0% 356 0 0% 0 >> >> VAD.DBA.VAD_REGISTRY VAD_REGISTRY_CHDIR 928 1 0% 357 0 0% 0 >> >> VAD.DBA.VAD_REGISTRY VAD_REGISTRY_KEY 1 2 100% 1 0 0% 0 >> >> VAD.DBA.VAD_HELP VAD_HELP 23 1 4% 46 0 0% 0 >> >> >> >> >> >> Hash indexes >> >> ----------------------------------- ------------------------------------------------------------------------------ BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA http://p.sf.net/sfu/rim-devcon-copy2 _______________________________________________ Dbpedia-discussion mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
