Hi Karel, First of all try these settings which we use for our DBpedia-Live virtuoso instance: NumberOfBuffers = 800000 MaxDirtyBuffers = 600000 and please don't forget to restart you Virtuoso server after changing those settings, in order for them to take effect.
Secondly, in the website you sent, you have mentioned that it took two days to fill your Virtuoso instance with an initial dump of DBpedia-Live, which seems too long for me. Thirdly, my question is "How long is Virtuoso running till now?" -- Kind Regards Mohamed Morsey Department of Computer Science University of Leipzig On 09/16/2011 02:26 PM, karel braeckman wrote: > Forgot to mention the machine details: > > 24GB RAM > 2x quadcore Xeon E5540 2.5GHz > Virtuoso data is on SSD disks > > > Regards, > Karel > > > On Fri, Sep 16, 2011 at 1:57 PM, karel braeckman > <[email protected] > <mailto:[email protected]>> wrote: > > Hi all, > > We tried setting up DBpedia Live on one of our local servers and > keep it in sync with the official DBpedia live, using the > dbpintegrator tool. Our setup is documented here: > http://medialoep.vrtmedialab.be/blog/?p=96. > > However, adding / deleting triples from / to Virtuoso has become > extremely slow. I was pointed in the direction of > > http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtRDFPerformanceTuning > by @kidehen > and @pkleef. I changed the following parameters in Virtuoso's .ini > file from their default value: > > [Database] > MaxCheckpointRemap= 1000000 > > [Parameters] > NumberOfBuffers= 1360000 > MaxDirtyBuffers= 1000000 > > I can see no real difference in performance though, Virtuoso gets > stuck on a DELETE query (it is busy with it for the last 125 minutes > at 100% CPU :-S). As I am not at all an expert on Virtuoso tuning, > all suggestions are welcome. > > For completeness, I have included the output of status('rhck'); in > isql below. > > Best regards, > Karel > > ---------------------------------------------- > > > OpenLink Virtuoso Server > > Version 06.01.3127-pthreads for Linux as of Mar 16 2011 > > Started on: 2011/09/16 11:35 GMT+120 > > Database Status: > > File size 56727961600, 6924800 pages, 2771949 free. > > 1360000 buffers, 47404 used, 2 dirty 0 wired down, repl > age 0 0 w. io 40 w/crsr. > > Disk Usage: 48335 reads avg 0 msec, 0% r 0% w last 254 s, > 10448 writes, > > 136 read ahead, batch = 230. Autocompact 22 in 12 out, > 43% saved. > > Gate: 391 2nd in reads, 0 gate write waits, 0 in while > read 0 busy scrap. > > Log = virtuoso.trx, 4003 bytes > > 4151261 pages have been changed since last backup (in > checkpoint state) > > Current backup timestamp: 0x0000-0x00-0x00 > > Last backup date: unknown > > Clients: 1 connects, max 1 concurrent > > RPC: 65 calls, 1 pending, 1 max until now, 0 queued, 0 > burst reads (0%), 0 second brk=11304103936 > > Checkpoint Remap 1159 pages, 0 mapped back. 125 s atomic time. > > DB master 6924800 total 2771949 free 1159 remap 0 mapped back > > temp 768 total 763 free > > Lock Status: 0 deadlocks of which 0 2r1w, 0 waits, > > Currently 2 threads running 0 threads waiting 0 threads > in vdb. > > Pending: > > Client 1111:1: Account: dba, 361734 bytes in, 3237 bytes > out, 17 stmts. > > Transaction status: PENDING, 1 threads. > > Locks: > > Running Statements: > > Time (msec) Text > > 7797190 sparql DELETE FROM <http://live.dbpedia.org> { > > <http://dbpedia.org/resource/T > > Replication Status: Server db-UBUNTU. > > db-MUSSORGSKY db-MUSSORGSKY 0 OFF. > > db-UBUNTU db-UBUNTU 0 OFF. > > Index Usage: > > TableIndexTouches Reads %Miss Locks Waits %W n-dead > > DB.DBA.RDF_PREFIX RDF_PREFIX 3 95 2375% 6 0 0% 0 > > DB.DBA.RDF_PREFIX DB_DBA_RDF_PREFIX_UNQC_RP_ID 2193 4 0% 3 > 0 0% 0 > > DB.DBA.RDF_IRI RDF_IRI 40 4670 11390% 80 0 0% 0 > > DB.DBA.RDF_IRI DB_DBA_RDF_IRI_UNQC_RI_ID 353 8 2% 40 0 0% 0 > > DB.DBA.RDF_QUAD RDF_QUAD 953949641 13253 0% 2445 0 0% 0 > > DB.DBA.RDF_QUAD RDF_QUAD_SP 5596 447 7% 465 0 0% 0 > > DB.DBA.RDF_QUAD RDF_QUAD_POGS 1512 13514 893% 829 0 0% 0 > > DB.DBA.RDF_QUAD RDF_QUAD_GS 564 500 88% 80 0 0% 0 > > DB.DBA.RDF_QUAD RDF_QUAD_OP 152 942 615% 152 0 0% 0 > > DB.DBA.RDF_OBJ RDF_OBJ 3151 4134 131% 325 0 0% 0 > > DB.DBA.RDF_OBJ RO_VAL 954 2319 242 <tel:954%202319%20242>% > 137 0 0% 0 > > DB.DBA.RO_START RO_START 35 64 177% 35 0 0% 0 > > DB.DBA.RDF_DATATYPE RDF_DATATYPE 230 1 0% 238 0 0% 0 > > DB.DBA.RDF_LANGUAGE RDF_LANGUAGE 840 1 0% 0 0 0% 0 > > DB.DBA.RDF_LANGUAGE DB_DBA_RDF_LANGUAGE_UNQC_RL_TWOBYTE > 334 1 0% 344 0 0% 0 > > DB.DBA.RDF_OBJ_FT_RULES RDF_OBJ_FT_RULES 11 1 8% 14 0 0% 0 > > DB.DBA.RDF_GRAPH_GROUP RDF_GRAPH_GROUP_IRI 2 1 33% 1 0 0% 0 > > WS.WS.SYS_DAV_COL SYS_DAV_COL 13862 1 0% 20890 0 0% 0 > > WS.WS.SYS_DAV_COL SYS_DAV_COL_ID 221 1 0% 46 0 0% 0 > > WS.WS.SYS_DAV_RES SYS_DAV_RES 17368 164 0% 4409 0 0% 0 > > WS.WS.SYS_DAV_RES SYS_DAV_RES_COL 3242 4 0% 3609 0 0% 0 > > WS.WS.SYS_DAV_RES SYS_DAV_RES_FULL_PATH 7484 4 0% 4961 0 0% 0 > > WS.WS.SYS_DAV_RES SYS_DAV_RES_IID 3 1 25% 3 0 0% 0 > > WS.WS.SYS_DAV_RES_TYPES SYS_DAV_RES_TYPES 5 1 16% 10 0 0% 0 > > DB.DBA.HTTP_PATH HTTP_PATH 10 1 9% 11 0 0% 0 > > DB.DBA.VSPX_SESSION VSPX_SESSION 484 1 0% 522 0 0% 0 > > VAD.DBA.VAD_REGISTRY VAD_REGISTRY 1664 3 0% 356 0 0% 0 > > VAD.DBA.VAD_REGISTRY VAD_REGISTRY_CHDIR 928 1 0% 357 0 0% 0 > > VAD.DBA.VAD_REGISTRY VAD_REGISTRY_KEY 1 2 100% 1 0 0% 0 > > VAD.DBA.VAD_HELP VAD_HELP 23 1 4% 46 0 0% 0 > > Hash indexes > > ----------------------------------- > > > > > ------------------------------------------------------------------------------ > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA > http://p.sf.net/sfu/rim-devcon-copy2 > ------------------------------------------------------------------------------ BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA http://p.sf.net/sfu/rim-devcon-copy2 _______________________________________________ Dbpedia-discussion mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
