Using virtuoso 6-TP1, several weeks ago, I imported approx 500,000,000
triples from several bio2rdf graphs and built the additional bitmap indices
RDF_QUAD_GPOS, RDF_QUAD_OGPS, RDF_QUAD_OPGS, RDF_QUAD_POGS
I also created a free text index using:
DB.DBA.RDF_OBJ_FT_RULE_ADD(NULL, NULL, 'bio-ft-index')
None of the RDF data have been changed since then.
I have since upgraded to the full Virtuoso 6.0 release (running on 64 bit
Ubuntu Linux).
Left alone the database transaction log grows rapidly (300MB+ in 6 hours or
so) and after a while it stops responding to (formerly working) odbc
queries, though isql and the web interface continue to work fine. In this
state, there are often thousands of entries in the output of status()
consisting of IEP, IER, ISP etc:
OpenLink Virtuoso Server
> Version 06.00.3123-pthreads for Linux as of Oct 24 2009
> Started on: 2009/10/25 16:04 GMT+00
>
> Database Status:
> File size 102588481536, 12523008 pages, 4875617 free.
> 400000 buffers, 399225 used, 341076 dirty 3 wired down, repl age 1055029
> 22129 w. io 2 w/crsr.
> Disk Usage: 7468535 reads avg 10 msec, 98% r 0% w last 6 s, 9281852
> writes,
> 1531 read ahead, batch = 319. Autocompact 1301469 in 1071895 out, 17%
> saved.
> Gate: 43356 2nd in reads, 0 gate write waits, 0 in while read 0 busy
> scrap.
> Log = utopia-database.trx, 326890480 bytes
> 1151167 pages have been changed since last backup (in checkpoint state)
> Current backup timestamp: 0x8223-0x20-0xEE
> Last backup date: Thu Oct 22 11:59:12 2009
>
> Clients: 143 connects, max 5 concurrent
> RPC: 2032 calls, -246 pending, 1 max until now, 0 queued, 0 burst reads
> (0%), 9 second brk=548294656
> Checkpoint Remap 278243 pages, 0 mapped back. 2176 s atomic time.
> DB master 12523008 total 4875616 free 278243 remap 73949 mapped back
> temp 5376 total 5370 free
>
> Lock Status: 5 deadlocks of which 5 2r1w, 2454 waits,
> Currently 2 threads running 0 threads waiting 0 threads in vdb.
> Pending:
> 6324267: ISR 0
> 0: ISR 0
> 10292114: IEP 0
> 11794176: ISR 0
> 2: IER 0
> 11066624: ISR 0
> 44: IER 0
> 11259136: IER 0
> 13: IER 0
> 12105985: ISR 0
> 134: IER 0
>
>
> *[... skip a lot more ISR and IER entries ... ]*
>
> Client 1111:209:-210: Account: kend, 1010 bytes in, 1404 bytes out, 1
> stmts.
> PID: 5046, OS: unix, Application: unknown, IP#: 192.168.122.253
> Transaction status: PENDING, 0 threads.
> Locks: 9227654: IS, 159214: IS, 6324267: IS, 10771394: IS,
>
> Client 1111:123:-124: Account: dba, 506 bytes in, 1878114 bytes out, 1
> stmts.
> PID: 1708, OS: unix, Application: unknown, IP#: 127.0.0.1
> Transaction status: PENDING, 0 threads.
> Locks:
>
> Running Statements:
> Time (msec) Text
>
> Replication Status: Server anonymous.
> anonymous anonymous 0 OFF.
>
> Hash indexes
>
> No. of rows in result: *2357 *
>
With tracing enabled and no external connections to the DB, and nothing
listed by status() under "Running Statements" , the following messages are
continually produced:
*trace_on('user_log', 'ddl_log', 'client_sql', 'errors', 'transact', 'exec',
'thread');
*
>
> 16:26:31 LTRS_2 0 Internal Internal Restart transact 0x7f66bca67790
> 16:26:33 LTRS_1 0 Internal Internal Commit transact 0x7f66bca67790
> 140733193388032
> 16:26:33 LTRS_2 0 Internal Internal Restart transact 0x7f66bca67790
> 16:26:35 LTRS_1 0 Internal Internal Commit transact 0x7f66bca67790
> 140733193388032
> 16:26:35 LTRS_2 0 Internal Internal Restart transact 0x7f66bca67790
> 16:26:41 LTRS_1 0 Internal Internal Commit transact 0x7f66bca67790
> 140733193388032
> 16:26:41 LTRS_2 0 Internal Internal Restart transact 0x7f66bca67790
> 16:26:44 LTRS_1 0 Internal Internal Commit transact 0x7f66bca67790
> 140733193388032
> 16:26:44 LTRS_2 0 Internal Internal Restart transact 0x7f66bca67790
> 16:26:48 LTRS_1 0 Internal Internal Commit transact 0x7f66bca67790
> 140733193388032
> 16:26:48 LTRS_2 0 Internal Internal Restart transact 0x7f66bca67790
>
What might be causing this (updating free text search? upgrading database
format from TP1 to release?) and what would be the best remedy?*
*Unfortunately I have no way of knowing if this same behaviour was exhibited
before switching to the release version from TP1.
Many thanks,
James*
*
--
Dr. James Marsh. Research Fellow, Advanced Interfaces Group.
School of Computer Science, The University of Manchester,
Oxford Road, Manchester, UK. M13 9PL.*
*
