Re: [Virtuoso-users] Loading ttl files halts
Hi Dong,
Also, run the “status(‘’);” command run from isql-vt to see the status of the
machine at this point, which will confirm if the buffers set in the INI have
taken effect and if any locks/deadlocks etc might be occurring. The output of
the top command to see the overall memory usage on the system would be useful
to see. Finally what is the actual full Virtuoso version being used, which can
be obtained by typing the command:
virtuoso-t -?
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
On 27 Nov 2012, at 17:55, Thomas Michaux wrote:
> same trick for deleting graph :
> http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtTipsAndTricksGuideDeleteLargeGraphs
>
> Such large graphs can be cleared by changing the transaction log mode to
> autocommit on each operation, deleting the graph(s) or triples, and then
> setting the log mode back to its original setting. This is easily done using
> the Virtuoso log_enable function, with the settings log_enable(3,1).
>
> This function may be called on its own, prior to the delete operation, via
> iSQL (either command-line or the Conductor variant), as shown:
>
> log_enable(3,1);
> SPARQL CLEAR GRAPH ;
>
>
>
> Le 27/11/2012 18:52, Thomas Michaux a écrit :
>> Hi,
>>
>> Maybe it's related to the logging system stacking info in RAM, take a look
>> at this :
>>
>> "One of the side effects of the default log_enable = 2 setting is that
>> triggers are not enabled to speed the loading of data. If triggers are
>> required for RDF Graph replication between nodes etc. then the log_enable
>> mode should be set to 3 when calling the rdf_loader_run() function as
>> follows:
>> rdf_loader_run (log_enable=>3);
>>
>> "
>> in http://www.openlinksw.com/dataspace/dav/wiki/Main/VirtBulkRDFLoader
>>
>> Le 27/11/2012 18:16, Y. Dong a écrit :
>>> Hi there,
>>>
>>>
>>> Currently I am trying to load yago2s into the virtuoso database. There are
>>> 15 ttl(turtle) files, totalling 8.5GB in size. The server is running on an
>>> amazon instance with 64bit Ubuntu 12.04.01 LTS, 8GB Memory, and more than
>>> enough disk space. Virtuoso Server Version is 06.01.3127. I use ld_dir()
>>> and rdf_loader_run() to load those files. virtuoso.ini is set to utilise
>>> the 8GB memory.
>>>
>>> At first, the loading process ran smoothly, but after half way, the
>>> virtuoso server stops responding. I can see 100% CPU usage by virtuoso and
>>> I can enter isql-v to start the virtuoso command line. But the server
>>> doesn't respond to any command I type in isql-v, it just stalls and I have
>>> to use ctrl+c to exit. Now I can't even use status() to see basic stats of
>>> the server. The http virtuoso server also stops responding. So I believe
>>> the whole virtuoso process stalls somehow. There isn't anything unusual in
>>> the log file virtuoso.log (attached below).
>>>
>>> Has anyone encountered similar problems before?
>>>
>>> Tue Nov 27 2012
>>> 15:20:34 INFO: { Loading plugin 1: Type `plain', file `wikiv' in
>>> `/usr/local//lib/virtuoso/hosting'
>>> 15:20:34 INFO: WikiV version 0.6 from OpenLink Software
>>> 15:20:34 INFO: Support functions for WikiV collaboration tool
>>> 15:20:34 INFO: SUCCESS plugin 1: loaded from
>>> /usr/local//lib/virtuoso/hosting/wikiv.so }
>>> 15:20:34 INFO: { Loading plugin 2: Type `plain', file `mediawiki' in
>>> `/usr/local//lib/virtuoso/hosting'
>>> 15:20:34 INFO: MediaWiki version 0.1 from OpenLink Software
>>> 15:20:34 INFO: Support functions for MediaWiki collaboration tool
>>> 15:20:34 INFO: SUCCESS plugin 2: loaded from
>>> /usr/local//lib/virtuoso/hosting/mediawiki.so }
>>> 15:20:34 INFO: { Loading plugin 3: Type `plain', file `creolewiki' in
>>> `/usr/local//lib/virtuoso/hosting'
>>> 15:20:34 INFO: CreoleWiki version 0.1 from OpenLink Software
>>> 15:20:34 INFO: Support functions for CreoleWiki collaboration tool
>>> 15:20:34 INFO: SUCCESS plugin 3: loaded from
>>> /usr/local//lib/virtuoso/hosting/creolewiki.so }
>>> 15:20:34 INFO: { Loading plugin 4: Type `plain', file `im' in
>>> `/usr/local//lib/virtuoso/hosting'
>>> 15:20:35 INFO: IM version 0.6 from OpenLink Software
>>> 15:20:35 INFO: Support functions for Image Magick 6.6.9
>>> 15:20:35 INFO: SUCCESS plugin 4: loaded from
>>> /usr/local//lib/virtuoso/hosting/im.so }
>>> 15:20:35 INFO: OpenLink Virtuoso Universal Server
>>> 15:20:35 INFO: Version 06.01.3127-pthreads for Linux as of Sep 14 2012
>>> 15:20:35 INFO: uses parts of OpenSSL, PCRE, Html Tidy
>>> 15:20:41 INFO: Database version 3126
>>> 15:20:41 INFO: SQL Opti
Re: [Virtuoso-users] Loading ttl files halts
same trick for deleting graph :
http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtTipsAndTricksGuideDeleteLargeGraphs
Such large graphs can be cleared by changing the transaction log mode to
autocommit on each operation, deleting the graph(s) or triples, and then
setting the log mode back to its original setting. This is easily done
using the Virtuoso log_enable function, with the settings log_enable(3,1).
This function may be called on its own, prior to the delete operation,
via iSQL (either command-line or the Conductor variant), as shown:
log_enable(3,1);
SPARQL CLEAR GRAPH ;
Le 27/11/2012 18:52, Thomas Michaux a écrit :
Hi,
Maybe it's related to the logging system stacking info in RAM, take a
look at this :
"One of the side effects of the default |log_enable = 2| setting is
that triggers are not enabled to speed the loading of data. If
triggers are required for RDF Graph replication between nodes etc.
then the log_enable mode should be set to *3* when calling the
rdf_loader_run() function as follows:
rdf_loader_run (log_enable=>3);
"
in http://www.openlinksw.com/dataspace/dav/wiki/Main/VirtBulkRDFLoader
Le 27/11/2012 18:16, Y. Dong a écrit :
Hi there,
Currently I am trying to load yago2s into the virtuoso database. There are 15
ttl(turtle) files, totalling 8.5GB in size. The server is running on an amazon
instance with 64bit Ubuntu 12.04.01 LTS, 8GB Memory, and more than enough disk
space. Virtuoso Server Version is 06.01.3127. I use ld_dir() and
rdf_loader_run() to load those files. virtuoso.ini is set to utilise the 8GB
memory.
At first, the loading process ran smoothly, but after half way, the virtuoso
server stops responding. I can see 100% CPU usage by virtuoso and I can enter
isql-v to start the virtuoso command line. But the server doesn't respond to
any command I type in isql-v, it just stalls and I have to use ctrl+c to exit.
Now I can't even use status() to see basic stats of the server. The http
virtuoso server also stops responding. So I believe the whole virtuoso process
stalls somehow. There isn't anything unusual in the log file virtuoso.log
(attached below).
Has anyone encountered similar problems before?
Tue Nov 27 2012
15:20:34 INFO: { Loading plugin 1: Type `plain', file `wikiv' in
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO: WikiV version 0.6 from OpenLink Software
15:20:34 INFO: Support functions for WikiV collaboration tool
15:20:34 INFO: SUCCESS plugin 1: loaded from
/usr/local//lib/virtuoso/hosting/wikiv.so }
15:20:34 INFO: { Loading plugin 2: Type `plain', file `mediawiki' in
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO: MediaWiki version 0.1 from OpenLink Software
15:20:34 INFO: Support functions for MediaWiki collaboration tool
15:20:34 INFO: SUCCESS plugin 2: loaded from
/usr/local//lib/virtuoso/hosting/mediawiki.so }
15:20:34 INFO: { Loading plugin 3: Type `plain', file `creolewiki' in
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO: CreoleWiki version 0.1 from OpenLink Software
15:20:34 INFO: Support functions for CreoleWiki collaboration tool
15:20:34 INFO: SUCCESS plugin 3: loaded from
/usr/local//lib/virtuoso/hosting/creolewiki.so }
15:20:34 INFO: { Loading plugin 4: Type `plain', file `im' in
`/usr/local//lib/virtuoso/hosting'
15:20:35 INFO: IM version 0.6 from OpenLink Software
15:20:35 INFO: Support functions for Image Magick 6.6.9
15:20:35 INFO: SUCCESS plugin 4: loaded from
/usr/local//lib/virtuoso/hosting/im.so }
15:20:35 INFO: OpenLink Virtuoso Universal Server
15:20:35 INFO: Version 06.01.3127-pthreads for Linux as of Sep 14 2012
15:20:35 INFO: uses parts of OpenSSL, PCRE, Html Tidy
15:20:41 INFO: Database version 3126
15:20:41 INFO: SQL Optimizer enabled (max 1000 layouts)
15:20:43 INFO: Compiler unit is timed at 0.001027 msec
15:21:03 DEBUG: built-in procedure "repl_undot_name" overruled by the RDBMS
15:21:03 DEBUG: built-in procedure "REPL_FQNAME" overruled by the RDBMS
15:21:03 DEBUG: built-in procedure "REPL_COLTYPE_PS" overruled by the RDBMS
15:21:03 DEBUG: built-in procedure "REPL_COLTYPE" overruled by the RDBMS
15:21:04 INFO: Roll forward started
15:21:42 INFO: 1000 transactions, 7274496 bytes replayed (13 %)
15:22:13 INFO: 2000 transactions, 14548992 bytes replayed (27 %)
15:22:42 INFO: 3000 transactions, 21856256 bytes replayed (40 %)
15:23:11 INFO: 4000 transactions, 29163520 bytes replayed (54 %)
15:23:38 INFO: 5000 transactions, 36405248 bytes replayed (68 %)
15:23:54 INFO: 6000 transactions, 42500096 bytes replayed (79 %)
15:24:00 INFO: 7000 transactions, 47677440 bytes replayed (89 %)
15:24:06 INFO: 8000 transactions, 52396032 bytes replayed (97 %)
15:24:08 INFO: 8250 transactions, 53526661 bytes replayed (100 %)
15:24:08 INFO: Roll forward complete
15:24:12 INFO: Checkpoint started
15:24:22 INFO: Checkpoint finished, log reused
15:24:22 INFO: HTTP/WebDAV server online at 8890
15:24:22 INFO: Server online at (p
Re: [Virtuoso-users] Loading ttl files halts
Hi,
Maybe it's related to the logging system stacking info in RAM, take a
look at this :
"One of the side effects of the default |log_enable = 2| setting is that
triggers are not enabled to speed the loading of data. If triggers are
required for RDF Graph replication between nodes etc. then the
log_enable mode should be set to *3* when calling the rdf_loader_run()
function as follows:
rdf_loader_run (log_enable=>3);
"
in http://www.openlinksw.com/dataspace/dav/wiki/Main/VirtBulkRDFLoader
Le 27/11/2012 18:16, Y. Dong a écrit :
Hi there,
Currently I am trying to load yago2s into the virtuoso database. There are 15
ttl(turtle) files, totalling 8.5GB in size. The server is running on an amazon
instance with 64bit Ubuntu 12.04.01 LTS, 8GB Memory, and more than enough disk
space. Virtuoso Server Version is 06.01.3127. I use ld_dir() and
rdf_loader_run() to load those files. virtuoso.ini is set to utilise the 8GB
memory.
At first, the loading process ran smoothly, but after half way, the virtuoso
server stops responding. I can see 100% CPU usage by virtuoso and I can enter
isql-v to start the virtuoso command line. But the server doesn't respond to
any command I type in isql-v, it just stalls and I have to use ctrl+c to exit.
Now I can't even use status() to see basic stats of the server. The http
virtuoso server also stops responding. So I believe the whole virtuoso process
stalls somehow. There isn't anything unusual in the log file virtuoso.log
(attached below).
Has anyone encountered similar problems before?
Tue Nov 27 2012
15:20:34 INFO: { Loading plugin 1: Type `plain', file `wikiv' in
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO: WikiV version 0.6 from OpenLink Software
15:20:34 INFO: Support functions for WikiV collaboration tool
15:20:34 INFO: SUCCESS plugin 1: loaded from
/usr/local//lib/virtuoso/hosting/wikiv.so }
15:20:34 INFO: { Loading plugin 2: Type `plain', file `mediawiki' in
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO: MediaWiki version 0.1 from OpenLink Software
15:20:34 INFO: Support functions for MediaWiki collaboration tool
15:20:34 INFO: SUCCESS plugin 2: loaded from
/usr/local//lib/virtuoso/hosting/mediawiki.so }
15:20:34 INFO: { Loading plugin 3: Type `plain', file `creolewiki' in
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO: CreoleWiki version 0.1 from OpenLink Software
15:20:34 INFO: Support functions for CreoleWiki collaboration tool
15:20:34 INFO: SUCCESS plugin 3: loaded from
/usr/local//lib/virtuoso/hosting/creolewiki.so }
15:20:34 INFO: { Loading plugin 4: Type `plain', file `im' in
`/usr/local//lib/virtuoso/hosting'
15:20:35 INFO: IM version 0.6 from OpenLink Software
15:20:35 INFO: Support functions for Image Magick 6.6.9
15:20:35 INFO: SUCCESS plugin 4: loaded from
/usr/local//lib/virtuoso/hosting/im.so }
15:20:35 INFO: OpenLink Virtuoso Universal Server
15:20:35 INFO: Version 06.01.3127-pthreads for Linux as of Sep 14 2012
15:20:35 INFO: uses parts of OpenSSL, PCRE, Html Tidy
15:20:41 INFO: Database version 3126
15:20:41 INFO: SQL Optimizer enabled (max 1000 layouts)
15:20:43 INFO: Compiler unit is timed at 0.001027 msec
15:21:03 DEBUG: built-in procedure "repl_undot_name" overruled by the RDBMS
15:21:03 DEBUG: built-in procedure "REPL_FQNAME" overruled by the RDBMS
15:21:03 DEBUG: built-in procedure "REPL_COLTYPE_PS" overruled by the RDBMS
15:21:03 DEBUG: built-in procedure "REPL_COLTYPE" overruled by the RDBMS
15:21:04 INFO: Roll forward started
15:21:42 INFO: 1000 transactions, 7274496 bytes replayed (13 %)
15:22:13 INFO: 2000 transactions, 14548992 bytes replayed (27 %)
15:22:42 INFO: 3000 transactions, 21856256 bytes replayed (40 %)
15:23:11 INFO: 4000 transactions, 29163520 bytes replayed (54 %)
15:23:38 INFO: 5000 transactions, 36405248 bytes replayed (68 %)
15:23:54 INFO: 6000 transactions, 42500096 bytes replayed (79 %)
15:24:00 INFO: 7000 transactions, 47677440 bytes replayed (89 %)
15:24:06 INFO: 8000 transactions, 52396032 bytes replayed (97 %)
15:24:08 INFO: 8250 transactions, 53526661 bytes replayed (100 %)
15:24:08 INFO: Roll forward complete
15:24:12 INFO: Checkpoint started
15:24:22 INFO: Checkpoint finished, log reused
15:24:22 INFO: HTTP/WebDAV server online at 8890
15:24:22 INFO: Server online at (pid 28660)
15:28:59 INFO: PL LOG: Loader started
15:29:20 INFO: PL LOG: Loader started
15:30:16 INFO: PL LOG: Loader started
--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/li
[Virtuoso-users] Loading ttl files halts
Hi there,
Currently I am trying to load yago2s into the virtuoso database. There are 15
ttl(turtle) files, totalling 8.5GB in size. The server is running on an amazon
instance with 64bit Ubuntu 12.04.01 LTS, 8GB Memory, and more than enough disk
space. Virtuoso Server Version is 06.01.3127. I use ld_dir() and
rdf_loader_run() to load those files. virtuoso.ini is set to utilise the 8GB
memory.
At first, the loading process ran smoothly, but after half way, the virtuoso
server stops responding. I can see 100% CPU usage by virtuoso and I can enter
isql-v to start the virtuoso command line. But the server doesn't respond to
any command I type in isql-v, it just stalls and I have to use ctrl+c to exit.
Now I can't even use status() to see basic stats of the server. The http
virtuoso server also stops responding. So I believe the whole virtuoso process
stalls somehow. There isn't anything unusual in the log file virtuoso.log
(attached below).
Has anyone encountered similar problems before?
Tue Nov 27 2012
15:20:34 INFO: { Loading plugin 1: Type `plain', file `wikiv' in
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO: WikiV version 0.6 from OpenLink Software
15:20:34 INFO: Support functions for WikiV collaboration tool
15:20:34 INFO: SUCCESS plugin 1: loaded from
/usr/local//lib/virtuoso/hosting/wikiv.so }
15:20:34 INFO: { Loading plugin 2: Type `plain', file `mediawiki' in
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO: MediaWiki version 0.1 from OpenLink Software
15:20:34 INFO: Support functions for MediaWiki collaboration tool
15:20:34 INFO: SUCCESS plugin 2: loaded from
/usr/local//lib/virtuoso/hosting/mediawiki.so }
15:20:34 INFO: { Loading plugin 3: Type `plain', file `creolewiki' in
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO: CreoleWiki version 0.1 from OpenLink Software
15:20:34 INFO: Support functions for CreoleWiki collaboration tool
15:20:34 INFO: SUCCESS plugin 3: loaded from
/usr/local//lib/virtuoso/hosting/creolewiki.so }
15:20:34 INFO: { Loading plugin 4: Type `plain', file `im' in
`/usr/local//lib/virtuoso/hosting'
15:20:35 INFO: IM version 0.6 from OpenLink Software
15:20:35 INFO: Support functions for Image Magick 6.6.9
15:20:35 INFO: SUCCESS plugin 4: loaded from
/usr/local//lib/virtuoso/hosting/im.so }
15:20:35 INFO: OpenLink Virtuoso Universal Server
15:20:35 INFO: Version 06.01.3127-pthreads for Linux as of Sep 14 2012
15:20:35 INFO: uses parts of OpenSSL, PCRE, Html Tidy
15:20:41 INFO: Database version 3126
15:20:41 INFO: SQL Optimizer enabled (max 1000 layouts)
15:20:43 INFO: Compiler unit is timed at 0.001027 msec
15:21:03 DEBUG: built-in procedure "repl_undot_name" overruled by the RDBMS
15:21:03 DEBUG: built-in procedure "REPL_FQNAME" overruled by the RDBMS
15:21:03 DEBUG: built-in procedure "REPL_COLTYPE_PS" overruled by the RDBMS
15:21:03 DEBUG: built-in procedure "REPL_COLTYPE" overruled by the RDBMS
15:21:04 INFO: Roll forward started
15:21:42 INFO: 1000 transactions, 7274496 bytes replayed (13 %)
15:22:13 INFO: 2000 transactions, 14548992 bytes replayed (27 %)
15:22:42 INFO: 3000 transactions, 21856256 bytes replayed (40 %)
15:23:11 INFO: 4000 transactions, 29163520 bytes replayed (54 %)
15:23:38 INFO: 5000 transactions, 36405248 bytes replayed (68 %)
15:23:54 INFO: 6000 transactions, 42500096 bytes replayed (79 %)
15:24:00 INFO: 7000 transactions, 47677440 bytes replayed (89 %)
15:24:06 INFO: 8000 transactions, 52396032 bytes replayed (97 %)
15:24:08 INFO: 8250 transactions, 53526661 bytes replayed (100 %)
15:24:08 INFO: Roll forward complete
15:24:12 INFO: Checkpoint started
15:24:22 INFO: Checkpoint finished, log reused
15:24:22 INFO: HTTP/WebDAV server online at 8890
15:24:22 INFO: Server online at (pid 28660)
15:28:59 INFO: PL LOG: Loader started
15:29:20 INFO: PL LOG: Loader started
15:30:16 INFO: PL LOG: Loader started
--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
