Re: [Virtuoso-users] strange error when bulk-loading Turtle files

2018-12-20 Thread Peter F. Patel-Schneider
Changing ttlpv.sql to use insert soft (and also making sure to use the stored id) appears to fix the problem I am encountering. I prepared a modified ttlpv.sql (attached) and reinstalled Virtuoso as follows: cd /home/virtuoso/test killall virtuoso-t rm -rf vos cp ttlpv.sql ./virtuoso-opensource/l

Re: [Virtuoso-users] strange error when bulk-loading Turtle files

2018-12-19 Thread Peter F. Patel-Schneider
Aah. It's likely then that something in MacOS doesn't switch between threads within ttlpv.sql. It is also possible that there is a similar issue in ttlpv.sql having to do with type ids (rdf_rl_type_id). That code is very similar to the code where I found the bug. peter On 12/19/18 11:05 AM, H

Re: [Virtuoso-users] strange error when bulk-loading Turtle files

2018-12-19 Thread Hugh Williams
Hi Peter, My original testing was on Darwin (macOS), I have switched to CentOS 7 and have been able to recreate the issue on first run: [hwilliams@localhost database]$ sh test.sh Wed 19 Dec 15:43:33 GMT 2018: 3/ Start up Virtuoso with an empty database OpenLink Virtuoso Interactive SQL (Virtuos

Re: [Virtuoso-users] strange error when bulk-loading Turtle files

2018-12-19 Thread Peter F. Patel-Schneider
That's not my experience on several machines. I've experienced this bug on two servers, one running CentOS release 6.9 (Final) and one running Ubuntu 16.04.5 LTS. The bug depends at least on loading in parallel *and* encountering new language tags. Loading the files a second time will almost cer

Re: [Virtuoso-users] strange error when bulk-loading Turtle files

2018-12-18 Thread Hugh Williams
Hi Peter, I generated the datasets from your python script and loaded them into a local Virtuoso open source multiple times but did not see any occurrences of the error: SQL> select * from load_list; ll_file ll_graph

Re: [Virtuoso-users] strange error when bulk-loading Turtle files

2018-12-18 Thread Peter F. Patel-Schneider
I created some synthetic data that tickles the bug reliably on my machine with a standard virtuoso.ini (just adding the directory for the files to the allowed list). I'm attaching the generator program for the files and a loading script. peter On 12/18/18 9:46 AM, Peter F. Patel-Schneider wrote

Re: [Virtuoso-users] strange error when bulk-loading Turtle files

2018-12-18 Thread Peter F. Patel-Schneider
I did a bit of digging and it sure looks as if there is a race condition in rdf_rl_lang_id in ttlpv.sql. This code appears to check to see if the language tag is already in DB.DBA.RDF_LANGUAGE and adds it if not. But another thread could do the same insert between the check and the insert, as fa

Re: [Virtuoso-users] strange error when bulk-loading Turtle files

2018-12-18 Thread Peter F. Patel-Schneider
I'm loading the Turtle Wikidata RDF complete dump, split into pieces and loaded with 10 active readers. About half the time the load fails with one or more of these errors. The errors are always near the beginning of the load---in the first group of 10 files to be loaded and near the beginning o

Re: [Virtuoso-users] strange error when bulk-loading Turtle files

2018-12-11 Thread Hugh Williams
Hi Peter, The triple value do indeed appear to be valid, but the problem could be somewhere else in the dataset file and not necessarily on the reported line or line before it. Is it a public dataset you are loading and if so can you provide a copy for local testing ? Best Regards Hugh Willia