Thank you Hugh for your answer.
Virtuoso runs on a VM with 32GBytes of RAM. This VM is dedicated to the
specific virtuoso instance and hosted in cloud infrastructure.
I have left my transaction to run until Tuesday (about 4 days !!!) and
still had no response. The VM reported 95% usage of memory but still no
response. The response in the status() command was the following:
*OpenLink Virtuoso Server*
*Version 07.20.3214-pthreads for Linux as of Oct 14 2015 *
*Started on: 2016-07-02 16:13 GMT+3*
*Database Status:*
* File size 0, 10407936 pages, 3793820 free.*
* 2720000 buffers, 130665 used, 4 dirty 0 wired down, repl age 0 0 w. io 0
w/crsr.*
* Disk Usage: 131941 reads avg 0 msec, 0% r 0% w last 0 s, 31533 writes
flush 4.255 MB,*
* 562 read ahead, batch = 224. Autocompact 0 in 0 out, 0% saved.*
*Gate: 388 2nd in reads, 0 gate write waits, 0 in while read 0 busy
scrap. *
*Log = /usr/local/var/lib/virtuoso/db/virtuoso.trx, 4146 bytes*
*6613693 pages have been changed since last backup (in checkpoint state)*
*Current backup timestamp: 0x0000-0x00-0x00*
*Last backup date: unknown*
*Clients: 1 connects, max 1 concurrent*
*RPC: 8 calls, 1 pending, 1 max until now, 0 queued, 0 burst reads (0%), 0
second 0M large, 22M max*
*Checkpoint Remap 0 pages, 0 mapped back. 35 s atomic time.*
* DB master 10407936 total 3793820 free 0 remap 0 mapped back*
* temp 256 total 251 free*
*Lock Status: 0 deadlocks of which 0 2r1w, 39 waits,*
* Currently 1 threads running 0 threads waiting 0 threads in vdb.*
*Pending:*
*Client 1111:1:-3: Account: dba, 215 bytes in, 289 bytes out, 1 stmts.*
*PID: 30445, OS: unix, Application: unknown, IP#: 127.0.0.1*
*Transaction status: PENDING, 1 threads.*
*Locks: *
*Running Statements:*
* Time (msec) Text*
* 567 status()*
*Hash indexes*
On Wednesday, I restarted the VM in order to make sure that the transaction
was cleared, and executed the query through the isql-v while enabling log.
The usage of memory is currently about 60% and I got plenty of log messages
that I cannot really interpret. I attach the log file in the current email.
Perhaps, you could provide me with some guidance on what I should do, based
on the log file.
Please note the following:
The values in the virtuoso.ini configuration file are:
*NumberOfBuffers = 2720000*
*MaxDirtyBuffers = 2000000*
The response in the status(); command in the SQL is the following:
*OpenLink Virtuoso Server*
*Version 07.20.3214-pthreads for Linux as of Oct 14 2015 *
*Started on: 2016-07-06 20:17 GMT+3*
*Database Status:*
* File size 0, 11027968 pages, 3947693 free.*
* 2720000 buffers, 483125 used, 31 dirty 0 wired down, repl age 0 0 w. io
0 w/crsr.*
* Disk Usage: 472040 reads avg 0 msec, 0% r 0% w last 0 s, 936646 writes
flush 10.35 MB,*
* 2533 read ahead, batch = 168. Autocompact 530282 in 446316 out, 15%
saved col ac: 45742 in 12% saved.*
*Gate: 446 2nd in reads, 0 gate write waits, 0 in while read 0 busy
scrap. *
*Log = /usr/local/var/lib/virtuoso/db/virtuoso.trx, 1403 bytes*
*7079218 pages have been changed since last backup (in checkpoint state)*
*Current backup timestamp: 0x0000-0x00-0x00*
*Last backup date: unknown*
*Clients: 1 connects, max 1 concurrent*
*RPC: 6 calls, 1 pending, 1 max until now, 0 queued, 0 burst reads (0%), 0
second 0M large, 152M max*
*Checkpoint Remap 589 pages, 0 mapped back. 464 s atomic time.*
* DB master 11027968 total 3947693 free 589 remap 2 mapped back*
* temp 256 total 251 free*
*Lock Status: 0 deadlocks of which 0 2r1w, 20 waits,*
* Currently 1 threads running 0 threads waiting 0 threads in vdb.*
*Pending:*
*Client 1111:1:-2: Account: dba, 215 bytes in, 289 bytes out, 1 stmts.*
*PID: 10273, OS: unix, Application: unknown, IP#: 127.0.0.1*
*Transaction status: PENDING, 1 threads.*
*Locks: *
*Running Statements:*
* Time (msec) Text*
* 673 status()*
*Hash indexes*
Thank you in advance for your help.
Kind regards,
Pantelis Natsiavas
2016-07-02 23:22 GMT+03:00 Hugh Williams <hwilli...@openlinksw.com>:
> Hi Pantelis,
>
> SPARUL operations are performed as part of a transaction, which is stored
> in memory until the operation is completed and committed to the database,
> and for such a large graph would consume a significant amount system
> memory, possibly all available memory and not being able to complete. Thus
> how much memory is available on the system and how much is allocated for
> Virtuoso data buffers ie NumberOfBuffers INI file param and how much memory
> is left ? You can run the “status();” command from isql to see the Buffers
> setting and other Virtuoso server status metrics.
>
> You can also try prepending the query with the "DEFINE sql:log-enable 3”
> pragma which sets transaction log mode to autocommit during the insert
> significantly reducing the memory requirement which may then allow the
> operation to complete, as is required when deleting large graphs as
> documented at:
>
>
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksGuideDeleteLargeGraphs
>
> I also note you are running a 3214 build which is from June 2015 and would
> recommend updating to the latest 3217 build from April 2016 …
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc. // http://www.openlinksw.com/
> Weblog -- http://www.openlinksw.com/blogs/
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> Twitter -- http://twitter.com/OpenLink
> Google+ -- http://plus.google.com/100570109519069333827/
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> Universal Data Access, Integration, and Management Technology Providers
>
> > On 2 Jul 2016, at 15:06, Pantelis Natsiavas <natsia...@gmail.com> wrote:
> >
> > Hi everybody.
> >
> > I am trying to execute the following query
> >
> > INSERT INTO GRAPH <targetGraph> {
> > ?a <someProperty> ?b
> > }
> > WHERE {
> > GRAPH <sourceGraph> {
> > ?a <someOtherProperty> ?c .
> > BIND(REPLACE(STR(?a),"something:", "", "i") AS ?b) }
> > }
> >
> > The sourceGraph contains about 94 million triplets. When I execute the
> above SPARQL, I get no error, however, I get no response either. I check
> after some minutes and the <targetGraph> has not yet been created. The
> system acts as if I had never executed the statement.
> >
> > I have tried executing the above through /sparql and through the
> /conductor web interfaces.
> >
> > Is there a way to see what is really going on? Am I doing something
> wrong?
> >
> > I have also already tried to debug through the isql-v localhost:8890
> dba <password> -D but I get a totally uncomprehensible error:
> >
> > isql-v() [0x44986a]
> > isql-v() [0x4498c8]
> > isql-v() [0x44b00e]
> > isql-v() [0x42fc65]
> > isql-v() [0x4108b0]
> > isql-v() [0x4043ee]
> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fc06a645ec5]
> > isql-v() [0x40441e]
> > GPF: Dkernel.c:3080 can't free if in served sessions
> > Segmentation fault (core dumped)
> >
> > Please note that I am using Version: 07.20.3214, Build: Oct 14 2015 on
> ubuntu server.
> >
> > Kind regards,
> > Pantelis Natsiavas
> >
> ------------------------------------------------------------------------------
> > Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
> > Francisco, CA to explore cutting-edge tech and listen to tech luminaries
> > present their vision of the future. This family event has something for
> > everyone, including kids. Get more information and register today.
> > http://sdm.link/attshape_______________________________________________
> > Virtuoso-users mailing list
> > Virtuoso-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/virtuoso-users
>
>
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
