Hi Hugh,
I am calling TTLP_MT() on the isql command line at the moment. I keep
the file size under 1 Gb and limit the number of lines (or triples) to 5
million. My system has a modest memory of 4Gb.
I have just had a hang situation again, this time with a really small
file: 44 Mb and 300,000 lines, so I had a chance to look into some
details. The isql session hangs, but I could open another session and
execute status(). It showed that the TTLP_MT() statement was still
active. At that moment virtuoso-t did not feature in the list made by
the 'top' command. I could also count the number of triples imported,
that being some 80 or 90 percent of the file contents. I restarted
Virtuoso and reran the statement. That time it did quickly succeed. I
was happy to notice that despite me not having removed the partial
result of the first attempt TTLP_MT() did not create any duplicate
triples: the triple count was equal to the line count.
I have not tried the bulk loader, although I did notice its existence
when looking for help on the web. I assumed that when TTLP_MT() hangs
when I call it from isql it would also hang when I use rdf_loader_run().
But maybe I should just give it a try.
Regards,
Frans
On 2014-04-18 12:43, Hugh Williams wrote:
Hi Frans,
How are you calling the TTLP_MT() function to load multiple files and
when you says these file are small what is there typical size or
number of lines/triples ? For loading of large number of files I would
expect you to be using the rdf_loader_run() bulk loader function which
calls ttlp() under the covers for loading as detailed at:
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtBulkRDFLoader
Using this method you can then use the LDMeter functions to monitor
the load rates as detailed at:
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksGuideLDMeterUtility
Also, at the time the TTLP load appear to hang I presume you can still
access the server in which case what does the "status();" command
reports as to the status of the server at that point ? How much memory
is on the system and how much does the "top" command report as being
used by Virtuoso ?
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
On 18 Apr 2014, at 10:12, Frans Knibbe | Geodan
<frans.kni...@geodan.nl <mailto:frans.kni...@geodan.nl>> wrote:
Hello,
I am using the function TTLP_MT to load ntriple files in Virtuoso
7.1. Even though I take care to have reasonably small files
(splitting the files by line numbers if necessary) TTLP_MT still
frequently hangs. I have to restart Virtuoso, remove the partial load
results and start over. This is quite bothersome, because I have a
lot of data to load. I would like to be able to load the files from a
script without constantly having to check if Virtuoso hangs and
making repairs.
Are there any tips on getting TTLP_MT to work without hanging? I
think the memory configuration is OK, available system memory is used
without disk swapping.
Regards,
Frans
------------------------------------------------------------------------
Frans Knibbe
Geodan
President Kennedylaan 1
1079 MB Amsterdam (NL)
T +31 (0)20 - 5711 347
E frans.kni...@geodan.nl
www.geodan.nl <http://www.geodan.nl/> | disclaimer
<http://www.geodan.nl/disclaimer>
------------------------------------------------------------------------
------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and
their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
------------------------------------------------------------------------
Frans Knibbe
Geodan
President Kennedylaan 1
1079 MB Amsterdam (NL)
T +31 (0)20 - 5711 347
E frans.kni...@geodan.nl
www.geodan.nl <http://www.geodan.nl> | disclaimer
<http://www.geodan.nl/disclaimer>
------------------------------------------------------------------------
------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech
_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
