Re: [Virtuoso-users] remove graph limitations on /describe/ service in FCT ?

[Thomas Michaux]

sorry you just have to remove "-dev" from the hostname, so the correct 
(public available) ones are :



1) https://data.idref.fr/describe/?url=http://www.idref.fr/169814289/id

2) https://data.idref.fr/sparql?query=DESCRIBE 
<http://www.idref.fr/169814289/id>=text/plain



Le 27/09/2018 à 12:46, Hugh Williams a écrit :

Hi Thomas,

Your URLs are not accessible external and just hang trying to load, 
thus is there some firewall restriction in place ?


Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.      // http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



On 27 Sep 2018, at 11:26, Thomas Michaux <mailto:mich...@abes.fr>> wrote:


in bold are the missing triples

1) result is

<http://www.idref.fr/169814289/id><http://www.w3.org/2004/02/skos/core#prefLabel> 
  "Dubois-Brissonnet, Florence" .
<http://www.idref.fr/169814289/id><http://purl.org/dc/terms/language>   
<http://lexvo.org/id/iso639-3/fra>  .
<http://www.idref.fr/169814289/id>
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person>  
.


2) result is

<http://www.idref.fr/169814289>   <http://xmlns.com/foaf/0.1/primaryTopic>  
<http://www.idref.fr/169814289/id>  .
*<http://www.sudoc.fr/169816842/id> 
<http://purl.org/dc/terms/contributor> 
<http://www.idref.fr/169814289/id> .*

<http://www.idref.fr/169814289/id><http://www.w3.org/2004/02/skos/core#prefLabel> 
  "Dubois-Brissonnet, Florence" .
*<http://www.sudoc.fr/178738921/id> 
<http://id.loc.gov/vocabulary/relators/ths> 
<http://www.idref.fr/169814289/id> . 
<http://www.sudoc.fr/201857987/id> 
<http://purl.org/dc/terms/contributor> 
<http://www.idref.fr/169814289/id> . *<http://www.idref.fr/169814289/id>	<http://purl.org/dc/terms/language>	<http://lexvo.org/id/iso639-3/fra>  .
*<http://www.sudoc.fr/199182817/id> 
<http://id.loc.gov/vocabulary/relators/pbd> 
<http://www.idref.fr/169814289/id> . 
<http://www.sudoc.fr/221554505/id> 
<http://id.loc.gov/vocabulary/relators/pbd> 
<http://www.idref.fr/169814289/id> . *<http://www.idref.fr/169814289/id>	<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>	<http://xmlns.com/foaf/0.1/Person>  .


Le 27/09/2018 à 12:12, Thomas Michaux a écrit :

Hello,

we need to find a solution to remove limitations the way  FCT code 
is "restricting" results of describe (Description TAB) results to 
one unique graph dataset, while the SPARQL describe operator allow 
to query ALL graphs by default :


you can compare the 2 results :

1) FCT/describe

https://data-dev.idref.fr/describe/?url=http://www.idref.fr/169814289/id 



2) SPARQL query

https://data-dev.idref.fr/sparql?query=DESCRIBE 
<http://www.idref.fr/169814289/id>=text/plain



Thanks four your help and suggestion to change this behaviour,


Thomas



___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net 
<mailto:Virtuoso-users@lists.sourceforge.net>

https://lists.sourceforge.net/lists/listinfo/virtuoso-users




___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

[Virtuoso-users] remove graph limitations on /describe/ service in FCT ?

[Thomas Michaux]

Hello,

we need to find a solution to remove limitations the way  FCT code is 
"restricting" results of describe (Description TAB) results to one 
unique graph dataset, while the SPARQL describe operator allow to query 
ALL graphs by default :


you can compare the 2 results :

1) FCT/describe

https://data-dev.idref.fr/describe/?url=http://www.idref.fr/169814289/id

2) SPARQL query

https://data-dev.idref.fr/sparql?query=DESCRIBE 
=text/plain



Thanks four your help and suggestion to change this behaviour,


Thomas



___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] remove graph limitations on /describe/ service in FCT ?

[Thomas Michaux]

in bold are the missing triples

1) result is

<http://www.idref.fr/169814289/id><http://www.w3.org/2004/02/skos/core#prefLabel> 
  "Dubois-Brissonnet, Florence" .
<http://www.idref.fr/169814289/id><http://purl.org/dc/terms/language>   
<http://lexvo.org/id/iso639-3/fra> .
<http://www.idref.fr/169814289/id>
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> .


2) result is

<http://www.idref.fr/169814289>   <http://xmlns.com/foaf/0.1/primaryTopic>  
<http://www.idref.fr/169814289/id> .
*<http://www.sudoc.fr/169816842/id> 
<http://purl.org/dc/terms/contributor> <http://www.idref.fr/169814289/id> .*

<http://www.idref.fr/169814289/id><http://www.w3.org/2004/02/skos/core#prefLabel> 
  "Dubois-Brissonnet, Florence" .
*<http://www.sudoc.fr/178738921/id> 
<http://id.loc.gov/vocabulary/relators/ths> 
<http://www.idref.fr/169814289/id> . <http://www.sudoc.fr/201857987/id> 
<http://purl.org/dc/terms/contributor> 
<http://www.idref.fr/169814289/id> . *<http://www.idref.fr/169814289/id>	<http://purl.org/dc/terms/language>	<http://lexvo.org/id/iso639-3/fra> .
*<http://www.sudoc.fr/199182817/id> 
<http://id.loc.gov/vocabulary/relators/pbd> 
<http://www.idref.fr/169814289/id> . <http://www.sudoc.fr/221554505/id> 
<http://id.loc.gov/vocabulary/relators/pbd> 
<http://www.idref.fr/169814289/id> . *<http://www.idref.fr/169814289/id>	<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>	<http://xmlns.com/foaf/0.1/Person> .



Le 27/09/2018 à 12:12, Thomas Michaux a écrit :

Hello,

we need to find a solution to remove limitations the way  FCT code is 
"restricting" results of describe (Description TAB) results to one 
unique graph dataset, while the SPARQL describe operator allow to 
query ALL graphs by default :


you can compare the 2 results :

1) FCT/describe

https://data-dev.idref.fr/describe/?url=http://www.idref.fr/169814289/id

2) SPARQL query

https://data-dev.idref.fr/sparql?query=DESCRIBE 
<http://www.idref.fr/169814289/id>=text/plain



Thanks four your help and suggestion to change this behaviour,


Thomas



___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] backup-dump impossible after loading an "invalid" IRI

[Thomas Michaux]

.3217-pthreads for Darwin as of Oct 31 2017
17:17:05 uses parts of OpenSSL, PCRE, Html Tidy
17:17:05 SQL Optimizer enabled (max 1000 layouts)
17:17:05 Roll forward started
17:17:06 Roll forward complete
17:17:06 Checkpoint started
17:17:06 Checkpoint finished, log reused
17:17:06 Checkpoint started
17:17:06 Checkpoint finished, log reused
17:17:06 Server exiting
De-iMac-232:database hwilliams$ 
~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t

De-iMac-232:database hwilliams$ tail -f virtuoso.log
17:17:14 uses parts of OpenSSL, PCRE, Html Tidy
17:17:14 Database version 3126
17:17:14 SQL Optimizer enabled (max 1000 layouts)
17:17:15 Compiler unit is timed at 0.000266 msec
17:17:16 Roll forward started
17:17:16     3 transactions, 185 bytes replayed (100 %)
17:17:16 Roll forward complete
17:17:16 PL LOG: Can't get list of vad packages in 
/Users/hwilliams/vos-7-install/vad/

17:17:16 Checkpoint started
17:17:16 Checkpoint finished, log reused
17:17:18 HTTP/WebDAV server online at 8890
17:17:18 Server online at  (pid 10513)
^C
De-iMac-232:database hwilliams$ ~/Downloads/isql 
Connected to OpenLink Virtuoso
Driver: 07.20.3214 OpenLink Virtuoso ODBC Driver
OpenLink Interactive SQL (Virtuoso), version 0.9849b.
Type HELP; for help and EXIT; to exit.
SQL> sparql select * from 
<http://hub.abes.fr/journals/wiley2011/20170921> where {?s ?p ?o};

s                     p                                           o
LONG VARCHAR                 LONG VARCHAR                             
          LONG VARCHAR

___

http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana c g a 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type 
http://xmlns.com/foaf/0.1/Person


1 Rows. -- 1 msec.
SQL>

De-iMac-232:database hwilliams$ 
~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t -?

Virtuoso Open Source Edition (Column Store) (multi threaded)
Version 7.2.5-dev.3217-pthreads as of Oct 31 2017
Compiled for Darwin (x86_64-apple-darwin15.6.0)
Copyright (C) 1998-2017 OpenLink Softwar

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.      // http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



On 22 Dec 2017, at 15:56, Thomas Michaux <mich...@abes.fr 
<mailto:mich...@abes.fr>> wrote:


Hello,

we may have found a case where bulk load doesn't report error but 
invalid data has a side effect on the "backup-dump" process, step to 
reproduce :


1) - in an empty database : use

ld_dir('e:/virtuosoBACKUP/test', 
'badiritest.nt','http://hub.abes.fr/journals/wiley2011/20170921');


with these data in file badiritest.nt

<http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana\u00A0c\u00A0g\u00A0a> 
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type> 
<http://xmlns.com/foaf/0.1/Person> .


rdf_loader_run();

checkpoint;

shutdown;

2) then you can't success to process the "backup-dump" procedure :

virtuoso-t +foreground +configfile "c:\Program Files\OpenLink 
Software\VOS7\virtuoso-opensource\database\virtuoso.ini" +backup-dump



even if data is really in store :

SQL> SPARQL select ?s ?p ?o from 
<http://hub.abes.fr/journals/wiley2011/20170921> {?s ?p ?o};

s p o
LONG VARCHAR LONG VARCHAR LONG VARCHAR
___

http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana c g a 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type 
http://xmlns.com/foaf/0.1/Person


1 Rows. -- 0 msec.


tested on windows , need to reproduce under linux.


Thanks

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org <http://Slashdot.org>! 
http://sdm.link/slashdot___

Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net 
<mailto:Virtuoso-users@lists.sourceforge.net>

https://lists.sourceforge.net/lists/listinfo/virtuoso-users




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] backup-dump impossible after loading an "invalid" IRI

[Thomas Michaux]

Unable to locate file }
17:17:05 { Loading plugin 3: Type `plain', file `creolewiki' in 
`/Users/hwilliams/vos-7-install/hosting'

17:17:05   FAILED  plugin 3: Unable to locate file }
17:17:05 OpenLink Virtuoso Universal Server
17:17:05 Version 07.20.3217-pthreads for Darwin as of Oct 31 2017
17:17:05 uses parts of OpenSSL, PCRE, Html Tidy
17:17:05 SQL Optimizer enabled (max 1000 layouts)
17:17:05 Roll forward started
17:17:06 Roll forward complete
17:17:06 Checkpoint started
17:17:06 Checkpoint finished, log reused
17:17:06 Checkpoint started
17:17:06 Checkpoint finished, log reused
17:17:06 Server exiting
De-iMac-232:database hwilliams$ 
~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t

De-iMac-232:database hwilliams$ tail -f virtuoso.log
17:17:14 uses parts of OpenSSL, PCRE, Html Tidy
17:17:14 Database version 3126
17:17:14 SQL Optimizer enabled (max 1000 layouts)
17:17:15 Compiler unit is timed at 0.000266 msec
17:17:16 Roll forward started
17:17:16     3 transactions, 185 bytes replayed (100 %)
17:17:16 Roll forward complete
17:17:16 PL LOG: Can't get list of vad packages in 
/Users/hwilliams/vos-7-install/vad/

17:17:16 Checkpoint started
17:17:16 Checkpoint finished, log reused
17:17:18 HTTP/WebDAV server online at 8890
17:17:18 Server online at  (pid 10513)
^C
De-iMac-232:database hwilliams$ ~/Downloads/isql 
Connected to OpenLink Virtuoso
Driver: 07.20.3214 OpenLink Virtuoso ODBC Driver
OpenLink Interactive SQL (Virtuoso), version 0.9849b.
Type HELP; for help and EXIT; to exit.
SQL> sparql select * from 
<http://hub.abes.fr/journals/wiley2011/20170921> where {?s ?p ?o};

s                     p                                           o
LONG VARCHAR                 LONG VARCHAR                             
          LONG VARCHAR

___

http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana c g a 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type 
http://xmlns.com/foaf/0.1/Person


1 Rows. -- 1 msec.
SQL>

De-iMac-232:database hwilliams$ 
~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t -?

Virtuoso Open Source Edition (Column Store) (multi threaded)
Version 7.2.5-dev.3217-pthreads as of Oct 31 2017
Compiled for Darwin (x86_64-apple-darwin15.6.0)
Copyright (C) 1998-2017 OpenLink Softwar

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.      // http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



On 22 Dec 2017, at 15:56, Thomas Michaux <mich...@abes.fr 
<mailto:mich...@abes.fr>> wrote:


Hello,

we may have found a case where bulk load doesn't report error but 
invalid data has a side effect on the "backup-dump" process, step to 
reproduce :


1) - in an empty database : use

ld_dir('e:/virtuosoBACKUP/test', 
'badiritest.nt','http://hub.abes.fr/journals/wiley2011/20170921');


with these data in file badiritest.nt

<http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana\u00A0c\u00A0g\u00A0a> 
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type> 
<http://xmlns.com/foaf/0.1/Person> .


rdf_loader_run();

checkpoint;

shutdown;

2) then you can't success to process the "backup-dump" procedure :

virtuoso-t +foreground +configfile "c:\Program Files\OpenLink 
Software\VOS7\virtuoso-opensource\database\virtuoso.ini" +backup-dump



even if data is really in store :

SQL> SPARQL select ?s ?p ?o from 
<http://hub.abes.fr/journals/wiley2011/20170921> {?s ?p ?o};

s p o
LONG VARCHAR LONG VARCHAR LONG VARCHAR
___

http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana c g a 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type 
http://xmlns.com/foaf/0.1/Person


1 Rows. -- 0 msec.


tested on windows , need to reproduce under linux.


Thanks

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org <http://Slashdot.org>! 
http://sdm.link/slashdot___

Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net 
<mailto:Virtuoso-users@lists.sourceforge.net>

https://lists.sourceforge.net/lists/listinfo/virtuoso-users




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___

[Virtuoso-users] backup-dump impossible after loading an "invalid" IRI

[Thomas Michaux]

Hello,

we may have found a case where bulk load doesn't report error but 
invalid data has a side effect on the "backup-dump" process, step to 
reproduce :


1) - in an empty database : use

ld_dir('e:/virtuosoBACKUP/test', 
'badiritest.nt','http://hub.abes.fr/journals/wiley2011/20170921');


with these data in file badiritest.nt

 
 
 .


rdf_loader_run();

checkpoint;

shutdown;

2) then you can't success to process the "backup-dump" procedure :

virtuoso-t +foreground +configfile "c:\Program Files\OpenLink 
Software\VOS7\virtuoso-opensource\database\virtuoso.ini" +backup-dump



even if data is really in store :

SQL> SPARQL select ?s ?p ?o from 
 {?s ?p ?o};

s p o
LONG VARCHAR LONG VARCHAR LONG VARCHAR
___

http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana c g a 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type 
http://xmlns.com/foaf/0.1/Person


1 Rows. -- 0 msec.


tested on windows , need to reproduce under linux.


Thanks

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] possible recover for graph names corruption in V 07.20.3217 Database version 3126 ?

[Thomas Michaux]

ok, i understand, i'll give it a try, thanks


Le 13/12/2017 à 18:49, Kingsley Idehen a écrit :

On 12/12/17 9:52 AM, Thomas Michaux wrote:

Hello,

we made a dump (+backup-dump) after we found messages in virtuoso.log 
like these :


" There are pages in the backup set that are actually free. Should do 
backup_context_clear () and thus get a full backup. This can indicate 
corruption around page 73 81024."


Okay, but you are making a backup of data in a database where the 
aforementioned index is corrupt, as the system message indicates.





the dump was ok, no errors, after we saved the .db and delete it we 
recover by reading the .trx from previous step (+restore-crash-dump) 
and it was also successfull.


Yes, but you are using +restore-crash-dump against a backup of a 
corrupted database. You should have used +crash-dump as per:


./virtuoso -?

Usage:
  virtuoso [-fcnCbDARwMKrBd] [+foreground] [+configfile arg] 
[+no-checkpoint]

   [+checkpoint-only] [+backup-dump] [+crash-dump]
   [+crash-dump-data-ini arg] [+restore-crash-dump] [+wait]
   [+mode arg] [+dumpkeys arg] [+restore-backup arg]
   [+backup-dirs arg] [+debug] [+pwdold arg] [+pwddba arg]
   [+pwddav arg]
  +foreground    run in the foreground
  +configfile    use alternate configuration file
  +no-checkpoint do not checkpoint on startup
  +checkpoint-only   exit as soon as checkpoint on startup is complete
  +backup-dump   dump database into the transaction log, then exit
*  +crash-dump    dump inconsistent database into the 
transaction log, then exit*
  +crash-dump-data-ini   specify the DB ini to use for reading the 
data to dump

*+restore-crash-dump    restore from a crash-dump*
  +wait  wait for background initialization to complete
  +mode  specify mode options for server startup (onbalr)
  +dumpkeys  specify key id(s) to dump on crash dump 
(default : all)

  +restore-backup    restore from online backup
  +backup-dirs   default backup directories
  +debug Show additional debugging info
  +pwdold    Old DBA password
  +pwddba    New DBA password
  +pwddav    New DAV password


Kingsley



Now the problem is we have graphs but only with IRI_ID showed, no 
names ? is there an internal data structure to update to have the 
graph names back or is it definitely corrupted ?


Thanks

PS:

15:21:41 OpenLink Virtuoso Universal Server
15:21:41 Version 07.20.3217-pthreads for Linux as of Feb 10 2017
15:21:41 uses parts of OpenSSL, PCRE, Html Tidy
15:21:47 Database version 3126


graph list :

SQL> sparql select distinct ?g where {GRAPH ?g{?s a ?t}};
g
LONG VARCHAR
___ 



http://www.openlinksw.com/schemas/virtrdf#
iri_id_179939769_with_no_name_entry
b3sifp
http://www.w3.org/2002/07/owl#
iri_id_101440794_with_no_name_entry
iri_id_138171152_with_no_name_entry
iri_id_109064725_with_no_name_entry
iri_id_120447918_with_no_name_entry
iri_id_122475737_with_no_name_entry
iri_id_126542051_with_no_name_entry
iri_id_134423479_with_no_name_entry
iri_id_126235267_with_no_name_entry
iri_id_130855391_with_no_name_entry
iri_id_182388251_with_no_name_entry
iri_id_115082563_with_no_name_entry
iri_id_107233893_with_no_name_entry
iri_id_176955935_with_no_name_entry
iri_id_138170213_with_no_name_entry
http://localhost:8890/sparql
iri_id_120447899_with_no_name_entry
iri_id_126235266_with_no_name_entry
iri_id_111093750_with_no_name_entry
iri_id_111317575_with_no_name_entry
iri_id_115156110_with_no_name_entry
iri_id_169252958_with_no_name_entry
iri_id_184144716_with_no_name_entry
iri_id_176961926_with_no_name_entry
iri_id_104573215_with_no_name_entry
iri_id_126542040_with_no_name_entry
iri_id_173680485_with_no_name_entry
iri_id_169407489_with_no_name_entry
virtrdf-label
iri_id_102730058_with_no_name_entry
iri_id_113150701_with_no_name_entry
iri_id_126920998_with_no_name_entry

35 Rows. -- 3384 msec.


-- 


Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users



--
Regards,

Kingsley Idehen 
Founder & CEO
OpenLink Software   (Home Page:http://www.openlinksw.com)

Weblogs (Blogs):
Legacy Blog:http://www.openlinksw.com/blog/~kidehen/
Blogspot Blog:http://kidehen.blogspot.com
Medium Blog:https://medium.com/@kidehen

Profile Pages:
Pinterest:https://www.pinterest.com/kidehen/
Quora:https://www.quora.com/profile/Kingsley-Uyi-Idehen
Twitter:https://twitter.com/kidehen
Google+:https://plus.google.com/+KingsleyIdehen/about
LinkedIn:http://www.linkedin.com/in/kidehen

Web 

[Virtuoso-users] possible recover for graph names corruption in V 07.20.3217 Database version 3126 ?

[Thomas Michaux]

Hello,

we made a dump (+backup-dump) after we found messages in virtuoso.log 
like these :


" There are pages in the backup set that are actually free. Should do 
backup_context_clear () and thus get a full backup. This can indicate 
corruption around page 73 81024."



the dump was ok, no errors, after we saved the .db and delete it we 
recover by reading the .trx from previous step (+restore-crash-dump) and 
it was also successfull.


Now the problem is we have graphs but only with IRI_ID showed, no names 
? is there an internal data structure to update to have the graph names 
back or is it definitely corrupted ?


Thanks

PS:

15:21:41 OpenLink Virtuoso Universal Server
15:21:41 Version 07.20.3217-pthreads for Linux as of Feb 10 2017
15:21:41 uses parts of OpenSSL, PCRE, Html Tidy
15:21:47 Database version 3126


graph list :

SQL> sparql select distinct ?g where {GRAPH ?g{?s a ?t}};
g
LONG VARCHAR
___

http://www.openlinksw.com/schemas/virtrdf#
iri_id_179939769_with_no_name_entry
b3sifp
http://www.w3.org/2002/07/owl#
iri_id_101440794_with_no_name_entry
iri_id_138171152_with_no_name_entry
iri_id_109064725_with_no_name_entry
iri_id_120447918_with_no_name_entry
iri_id_122475737_with_no_name_entry
iri_id_126542051_with_no_name_entry
iri_id_134423479_with_no_name_entry
iri_id_126235267_with_no_name_entry
iri_id_130855391_with_no_name_entry
iri_id_182388251_with_no_name_entry
iri_id_115082563_with_no_name_entry
iri_id_107233893_with_no_name_entry
iri_id_176955935_with_no_name_entry
iri_id_138170213_with_no_name_entry
http://localhost:8890/sparql
iri_id_120447899_with_no_name_entry
iri_id_126235266_with_no_name_entry
iri_id_111093750_with_no_name_entry
iri_id_111317575_with_no_name_entry
iri_id_115156110_with_no_name_entry
iri_id_169252958_with_no_name_entry
iri_id_184144716_with_no_name_entry
iri_id_176961926_with_no_name_entry
iri_id_104573215_with_no_name_entry
iri_id_126542040_with_no_name_entry
iri_id_173680485_with_no_name_entry
iri_id_169407489_with_no_name_entry
virtrdf-label
iri_id_102730058_with_no_name_entry
iri_id_113150701_with_no_name_entry
iri_id_126920998_with_no_name_entry

35 Rows. -- 3384 msec.


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] loader corrupting XMLLiteral in V7.2

[Thomas Michaux]

faster way to reproduce (see 
https://github.com/openlink/virtuoso-opensource/issues/662 last comment ) :


|SPARQL INSERT INTO <http://hub.abes.fr/wiley/journals2011subst>
{
   
<http://hub.abes.fr/wiley/periodical/rta/1997/volume_10/issue_3/101002/tjcj10982418199705103353ajdrta530cp2x/w>
   <http://purl.org/dc/terms/abstract>
   
"\\begin{eqnarray*}r(G)\\ge{1\\"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral>
  }
|

then SPARQL query

|select ?o DATATYPE(?o) {
<http://hub.abes.fr/wiley/periodical/rta/1997/volume_10/issue_3/101002/tjcj10982418199705103353ajdrta530cp2x/w>
 <http://purl.org/dc/terms/abstract> ?o }
|

will return

|"\begin{eqnarray*}r(G)\ge&{1\"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral>
|

which is NOT well formed XML :(

Thomas


Le 31/05/2017 11:36, Thomas Michaux a écrit :

Hello,

I can reproduce a problem while loading 
http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral thru loader , 
original triple literal is well formed XML (see around " 
\\ge{1\\over 2 ") parsing is OK.


but once loaded, the resulting data in triplestore is NO MORE WELL 
FORMED XML with a standalone & symbol  :


\ge&{1\over2}


our release is
OpenLink Virtuoso  Server
Version 07.20.3217-pthreads for Linux as of Feb 10 2017


to replay the bug :

load test triple
then query :

select ?o  DATATYPE(?o) {
<http://hub.abes.fr/wiley/periodical/rta/1997/volume_10/issue_3/101002/tjcj10982418199705103353ajdrta530cp2x/w> 
<http://purl.org/dc/terms/abstract> ?o }



Thanks for your help !

Thomas

test triple :

<http://hub.abes.fr/wiley/periodical/rta/1997/volume_10/issue_3/101002/tjcj10982418199705103353ajdrta530cp2x/w> 
<http://purl.org/dc/terms/abstract> "We analyze a randomized 
greedy\nmatching algorithm (RGA) aimed at producing a 
matching with a large number of edges in a\ngiven weighted 
graph. RGA was first introduced and studied by Dyer and Frieze in [3] 
for\nunweighted graphs. In the weighted version, at each 
step a new edge is chosen from the\n remaining graph with probability 
proportional to its weight, and is added to the\nmatching. 
The two vertices of the chosen edge are removed, and the step is 
repeated\nuntil there are no edges in the remaining graph. 
We analyze the expected size \u00CE\u00BC(G) of\nthe 
number of edges in the output matching produced by RGA, when RGA is 
repeatedly\n applied to the same graph G. Let 
r(G)=\u00CE\u00BC(G)/m(G), where m(G) is the maximum number 
of\nedges in a matching in G. For a class 
\u00EF\u00BF\u00BD of graphs, let \u00CF\uFFFD(\u00EF\u00BF\u00BD) be 
the infimum values\n r(G) over all graphs G in \u00EF\u00BF\u00BD 
(i.e., \u00CF\uFFFD is the \u00E2\u20AC\u0153worst\u00E2\u20AC\uFFFD 
performance ratio of RGA\nrestricted to the class 
\u00EF\u00BF\u00BD). Our main results are bound for \u00CE\u00BC, r, 
and \u00CF\uFFFD. For example,\nthe following results 
improve or generalize similar results obtained in [3] for 
the\nunweighted version of RGA; 
\\begin{eqnarray*}r(G)\\ge{1\\over 
2\u00E2\u20AC\uFFFD|V|/2|E|}\\quad\n\\mbox{(if $G$ has a 
perfect matching)} {\\sqrt{26}\u00E2\u20AC\uFFFD4\\over\n 
2}\\le\\rho(\\hbox{\\sf SIMPLE PLANAR 
GRAPHS})\\le.68436349 \\rho(\\hbox{SIMPLE\n 
$\\Delta$\u00E2\u20AC\uFFFDGRAPHS})\\ge{1\\over2}+{\\sqrt{(\\Delta\u00E2\u20AC\uFFFD1)^2+1}\u00E2\u20AC\uFFFD(\\Delta\u00E2\u20AC\uFFFD1)\\over2}\\end{eqnarray*}(where\n 
the class $\\Delta$\\hbox{\\sf-GRAPHS}$ is the set of graphs of 
maximum degree at most \u00CE\u201D).\n\u00C2\u00A9 1997 
John Wiley  Sons, Inc.\u00E2\u20AC\u0192Random Struct. Alg., 10: 
353\u00E2\u20AC\u201C383,\n 
1997"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral>



-- 


Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] Scheduler interval not respected ?

[Thomas Michaux]

ok, thanks for the information

Le 11/04/2017 21:04, Hugh Williams a écrit :

Hi Thomas,

This is a known issue in the Conductor Scheduler UI which displays the 
“Last time”  column lapsed time in “Next”  column, for which a issue 
report exists for development to fix ...


Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  // http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



On 11 Apr 2017, at 10:46, Thomas Michaux <mich...@abes.fr 
<mailto:mich...@abes.fr>> wrote:


Hello,

We need to delay full text indexing, using

DB.DBA.VT_BATCH_UPDATE ( 'DB.DBA.RDF_OBJ', 'ON', 8640 );

just after virtuoso startup the scheduler view in conductor 
mentionned the job will run six days later, which was what we 
expect (set yesterday)


but it seems to have a pb in display or not respecting the 6d delay 
as today the view says "Next" run in 17h !!! not six days :(


Event Name <javascript:void(0)> 	Status <javascript:void(0)> 	Start 
<javascript:void(0)> 	Last time <javascript:void(0)> 	Next 
<javascript:void(0)> 	Interval <javascript:void(0)>


VT_INC_INDEX_DB_DBA_RDF_OBJ() 	OK 	Yesterday at 17:52 	Yesterday at 
17:58 	17hrs,43min 	6d,0hrs,0min



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org <http://slashdot.org>! 
http://sdm.link/slashdot___

Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net 
<mailto:Virtuoso-users@lists.sourceforge.net>

https://lists.sourceforge.net/lists/listinfo/virtuoso-users




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

[Virtuoso-users] Scheduler interval not respected ?

[Thomas Michaux]

Hello,

We need to delay full text indexing, using

DB.DBA.VT_BATCH_UPDATE ( 'DB.DBA.RDF_OBJ', 'ON', 8640 );

just after virtuoso startup the scheduler view in conductor mentionned 
the job will run six days later, which was what we expect (set 
yesterday)


but it seems to have a pb in display or not respecting the 6d delay as 
today the view says "Next" run in 17h !!! not six days :(


Event Name  	Status  	Start 
 	Last time  	Next 
 	Interval 


VT_INC_INDEX_DB_DBA_RDF_OBJ() 	OK 	Yesterday at 17:52 	Yesterday at 
17:58 	17hrs,43min 	6d,0hrs,0min



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] Write load high relative to disk write throughput / intensive JDBC sparql INSERT DATA INTO GRAPH

[Thomas Michaux]

...sorry, sent the INDEX information tab instead of "SPACE" info tab :

Table   Index name  RowsPages   Row pages   Blob pages  Size
DB.DBA.RDF_OBJ  DB.DBA.RDF_OBJ  30443064255445  255445  0   1833.25 
MB
DB.DBA.RDF_IRI  DB.DBA.RDF_IRI  59063426206412  206412  0   1247.19 
MB
DB.DBA.RDF_IRI 	DB_DBA_RDF_IRI_UNQC_RI_ID 	59063426 	183541 	183541 	0 
1282.95 MB
DB.DBA.RDF_OBJ_RO_FLAGS_WORDS 	DB.DBA.RDF_OBJ_RO_FLAGS_WORDS 	8341023 
179875 	179875 	0 	1332.22 MB

DB.DBA.RDF_OBJ  RO_VAL  30443064117863  117863  0   773.81 MB
DB.DBA.RO_START DB.DBA.RO_START 3044306485319   85319   
0   475.31 MB


Le 20/03/2017 à 11:34, Thomas Michaux a écrit :




Le 19/03/2017 à 16:15, Hugh Williams a écrit :

Hi Thomas,

Hi,

Is the loading of the dataset now complete or it is still in progress as you 
opening statement is not clear ?
You should not need 40GB RAM for inserting and hosting 240 million triples, 
which should require less then 10GB depending on how well they can be 
compressed for storage in the database.
loading is complete, we finished at 243 188 427  triples  , hosting 
now requires 25GB ram, 15Gb disk, details :


void:triples 243188427 ;
 void:classes 13 ;
 void:entities 58523487 ;
 void:distinctSubjects 58523514 ;
 void:properties 32 ;
 void:distinctObjects 73171603 .

Total pages 1925120
Free pages  607377
Buffers 272
Buffers used244554
Dirty buffers   3
Wired down buffers  0

Table   Index name  Touches Reads   Read %
DB.DBA.RDF_QUAD RDF_QUAD1562356553  36371   0
DB.DBA.RDF_QUAD RDF_QUAD_POGS   609423455   16989   0
DB.DBA.RDF_QUAD RDF_QUAD_SP 378769255   35822   0
DB.DBA.RDF_QUAD RDF_QUAD_GS 340377017   16340


I assume you have set the swappiness as suggested previously ?

yes, done, $ sysctl vm.swappiness
vm.swappiness = 10


When you recompiled your Virtuoso was this done from the git stable/7 or 
develop/7 branch , as I latter has a number of memory consumption fixes that 
would not be in stable/7, thus I would suggest building from develop/7.

will investigate.

The two main problems we encountered while loading were :

- logs messages indicating "Flushing at 5.7 MB/s while application is 
making dirty pages at 1.7 MB/s." which we interpreted as not enough 
write speed while receiving lots of JDBC INSERTs (disk issue ? buffer 
issue ? ...)


- high memory consumption (40GB RAM), virtuoso process never releasing 
memory while loading, free RAM always going down...



Have you provided a copy of your INI file previously,  if not can you provide a 
copy ?

see attached (FYI QueryLog= was not active while loading)

Do ensure the following params are set to 1 in order to clean up unused 
threads/resources and reduce memory consumption of the Virtuoso server, which 
can otherwise be construed as memory leaks.:

ThreadCleanupInterval= 1
ResourcesCleanupInterval = 1

we have theses settings right.

Thanks for your help,

Thomas

if needed we model ORCID 2016 dataset using :
c1  c2
http://xmlns.com/foaf/0.1/Person
28021451
http://purl.org/ontology/bibo/Document  
14283692
http://purl.org/ontology/bibo/Journal   
9104659
http://xmlns.com/foaf/0.1/PersonalProfileDocument   
2527333
http://xmlns.com/foaf/0.1/Article   
974945
http://www.w3.org/ns/org#Membership 
807465
http://www.w3.org/2006/vcard/ns#Address 
807423
http://www.w3.org/ns/org#Organization   
807418
http://purl.org/ontology/bibo/Conference
769451
http://www.w3.org/ns/org#OrganizationalUnit 
649291
http://www.w3.org/2004/02/skos/core#Concept 
371731
http://purl.org/ontology/bibo/Book  
205493
http://www.w3.org/ns/org#Role   
168423
http://www.w3.org/1999/02/22-rdf-syntax-ns#Property 
170
http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat 
130
http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat
98
http://www.w3.org/2000/01/rdf-schema#Class  
56
http://www.openlinksw.com/schemas/virtrdf#QuadMapValue  
8
http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapColumn
8
http://www.openlinksw.com/schemas/virtrdf#QuadMapColumn 
8





Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //http://www.openlinksw.com/
Weblog   --http://www.openlinksw.com/blogs/
LinkedIn --http://www.linkedin.com/company/openlink-software/
Twitter  --http://twitter.com/OpenLink
Google+  --http://plus.google.com/100570109519069333827/
Facebook --http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers




On 15 Mar 2017, at 17:08, Thomas Michaux<mich...@abes.fr>  wrote:

Hello,

FYI, virtuoso still loading but we needed to increase memory ressources,
now the process use almost 40GB of ram :

[devel@tulipe-test2 ~]$ ./memcheck-virtuoso.sh
2017-03-15T17:54 VmSize: 41273424kB

Re: [Virtuoso-users] Write load high relative to disk write throughput / intensive JDBC sparql INSERT DATA INTO GRAPH

[Thomas Michaux]

Le 19/03/2017 à 16:15, Hugh Williams a écrit :

Hi Thomas,

Hi,


Is the loading of the dataset now complete or it is still in progress as you 
opening statement is not clear ?

You should not need 40GB RAM for inserting and hosting 240 million triples, 
which should require less then 10GB depending on how well they can be 
compressed for storage in the database.
loading is complete, we finished at 243 188 427  triples  , hosting now 
requires 25GB ram, 15Gb disk, details :


void:triples 243188427 ;
 void:classes 13 ;
 void:entities 58523487 ;
 void:distinctSubjects 58523514 ;
 void:properties 32 ;
 void:distinctObjects 73171603 .

Total pages 1925120
Free pages  607377
Buffers 272
Buffers used244554
Dirty buffers   3
Wired down buffers  0

Table   Index name  Touches Reads   Read %
DB.DBA.RDF_QUAD RDF_QUAD1562356553  36371   0
DB.DBA.RDF_QUAD RDF_QUAD_POGS   609423455   16989   0
DB.DBA.RDF_QUAD RDF_QUAD_SP 378769255   35822   0
DB.DBA.RDF_QUAD RDF_QUAD_GS 340377017   16340



I assume you have set the swappiness as suggested previously ?

yes, done, $ sysctl vm.swappiness
vm.swappiness = 10



When you recompiled your Virtuoso was this done from the git stable/7 or 
develop/7 branch , as I latter has a number of memory consumption fixes that 
would not be in stable/7, thus I would suggest building from develop/7.

will investigate.

The two main problems we encountered while loading were :

- logs messages indicating "Flushing at 5.7 MB/s while application is 
making dirty pages at 1.7 MB/s." which we interpreted as not enough 
write speed while receiving lots of JDBC INSERTs (disk issue ? buffer 
issue ? ...)


- high memory consumption (40GB RAM), virtuoso process never releasing 
memory while loading, free RAM always going down...




Have you provided a copy of your INI file previously,  if not can you provide a 
copy ?

see attached (FYI QueryLog= was not active while loading)


Do ensure the following params are set to 1 in order to clean up unused 
threads/resources and reduce memory consumption of the Virtuoso server, which 
can otherwise be construed as memory leaks.:

ThreadCleanupInterval= 1
ResourcesCleanupInterval = 1

we have theses settings right.

Thanks for your help,

Thomas

if needed we model ORCID 2016 dataset using :
c1  c2
http://xmlns.com/foaf/0.1/Person

28021451

http://purl.org/ontology/bibo/Document  

14283692

http://purl.org/ontology/bibo/Journal   

9104659

http://xmlns.com/foaf/0.1/PersonalProfileDocument   

2527333

http://xmlns.com/foaf/0.1/Article   

974945

http://www.w3.org/ns/org#Membership 

807465

http://www.w3.org/2006/vcard/ns#Address 

807423

http://www.w3.org/ns/org#Organization   

807418

http://purl.org/ontology/bibo/Conference

769451

http://www.w3.org/ns/org#OrganizationalUnit 

649291

http://www.w3.org/2004/02/skos/core#Concept 

371731

http://purl.org/ontology/bibo/Book  

205493

http://www.w3.org/ns/org#Role   

168423

http://www.w3.org/1999/02/22-rdf-syntax-ns#Property 

170

http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat 

130

http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat

98

http://www.w3.org/2000/01/rdf-schema#Class  

56

http://www.openlinksw.com/schemas/virtrdf#QuadMapValue  

8

http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapColumn

8

http://www.openlinksw.com/schemas/virtrdf#QuadMapColumn 

8






Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers




On 15 Mar 2017, at 17:08, Thomas Michaux <mich...@abes.fr> wrote:

Hello,

FYI, virtuoso still loading but we needed to increase memory ressources,
now the process use almost 40GB of ram :

[devel@tulipe-test2 ~]$ ./memcheck-virtuoso.sh
2017-03-15T17:54 VmSize: 41273424kB 5883

stats for the graph <http://hub.abes.fr/referentiel/ORCID/2016> (forget
to mention, it's the only graph in db) :

239 451 028 triples


this:Dataset a void:Dataset ;
rdfs:seeAlso <http://hub.abes.fr/referentiel/ORCID/2016> ;
rdfs:label "" ;
void:sparqlEndpoint <http://idrefplus.v102.abes.fr:8890/sparql> ;
void:triples 239451028 ;
void:classes 13 ;
void:entities 57692917 ;
void:distinctSubjects 57650847 ;
void:properties 32 ;
void:distinctObjects 72219514 .

this:sameAsLinks a void:Linkset ;
void:inDataset this:Dataset ;
void:triples 997389 ;
void:linkPredicate owl:sameAs .


Le 14/03/2017 à 10:05, Thomas Michaux a écrit :

Re: [Virtuoso-users] Write load high relative to disk write throughput / intensive JDBC sparql INSERT DATA INTO GRAPH

[Thomas Michaux]

Hello,

FYI, virtuoso still loading but we needed to increase memory ressources, 
now the process use almost 40GB of ram :

[devel@tulipe-test2 ~]$ ./memcheck-virtuoso.sh
2017-03-15T17:54 VmSize: 41273424kB 5883

stats for the graph <http://hub.abes.fr/referentiel/ORCID/2016> (forget 
to mention, it's the only graph in db) :

239 451 028 triples


this:Dataset a void:Dataset ;
  rdfs:seeAlso <http://hub.abes.fr/referentiel/ORCID/2016> ;
  rdfs:label "" ;
  void:sparqlEndpoint <http://idrefplus.v102.abes.fr:8890/sparql> ;
  void:triples 239451028 ;
  void:classes 13 ;
  void:entities 57692917 ;
  void:distinctSubjects 57650847 ;
  void:properties 32 ;
  void:distinctObjects 72219514 .

this:sameAsLinks a void:Linkset ;
  void:inDataset this:Dataset ;
  void:triples 997389 ;
  void:linkPredicate owl:sameAs .


Le 14/03/2017 à 10:05, Thomas Michaux a écrit :
> Hi Hugh,
>
> Le 10/03/2017 à 14:01, Hugh Williams a écrit :
>> Hi Thomas,
>>
>> Is the ORCID dataset the only RDF datasets in the Virtuoso RDF Quad Store 
>> currently, or are there others ?
>>
>> What is the size of the ORCID dataset ie triple count ?
> I gave you wrong informations, because I misundertstood the process.
> Below are the correct details of our INSERT procedure from ORACLE db :
>
> - dataset is from ORCID 2016 XML download available on this page
> https://orcid.org/content/download-file ("The file contains the public
> information associated with each user's ORCID record. Each record is
> included as a separate file in both JSON and XML. "
> https://figshare.com/articles/ORCID_Public_Data_File_2016/4134027).
>
> They are uploaded inside ORACLE as XML records.
>
> - then in a ORACLE PL/SQL procedure we apply "on the fly" an XSLT
> stylesheet (using Oracle XMLTRANSFORM efficient XSLT transform engine)
> to have an RDX/XML file for each ORCID XML record in the ORACLE table
>
> - next in the process we use Jena tools to generate also "on the fly"
> TRIPLES from this RDF/XML results
>
> - these are the  triples we're finally inserting via a JDBC "SPARQL
> INSERT DATA INTO GRAPH..." call to virtuoso from the PL/SQL Oracle
> procedure via virtuoso JDBC driver (and not ORACLE jdbc driver, my
> mistake, as you guessed)
>
> -checking release of JDBC driver is > java -cp virtjdbc3.jar
> virtuoso.jdbc3.Driver
> OpenLink Virtuoso(TM) Driver for JDBC(TM) Version 3.x [Build 3.62]
>
> (the driver is embedded  inside ORACLE java JVM)
>
> Thanks in advance if you have suggestions.
>
> Last "statistics" on the graph size give  : 182 405 784 triples
>
>
> this:Dataset a void:Dataset ;
>rdfs:seeAlso <http://hub.abes.fr/referentiel/ORCID/2016> ;
>rdfs:label "" ;
>void:sparqlEndpoint <http://idrefplus.v102.abes.fr:8890/sparql> ;
>void:triples 182405784 ;
>void:classes 13 ;
>void:entities 43946633 ;
>void:distinctSubjects 43922470 ;
>void:properties 32 ;
>void:distinctObjects 56509541 .
>
> this:sameAsLinks a void:Linkset ;
>void:inDataset this:Dataset ;
>void:triples 759462 ;
>
>
>
>> I would definitely suggest setting swappiness to 10 to reduce swapping to 
>> disk which should speed inserts rates.
> done
>> Looking at you status() command output I see "Clients: 4177045 connects, max 
>> 3 concurrent”  indicating more than 4 million SQL connections have been made 
>> to Virtuoso since it was started on 9th Mar . What is making that many 
>> connections, it is this insertion process
> yes, it is
>> or are there other clients reading from the instance also ?
> none for the moment, instance is private
>>Apart from that the status() output looks fine with please of unused 
>> Buffers for database working set size to be increased and still fit in 
>> memory ,
> don't really understand the point about buffers, but also noticed the
> use is not "maximized", because there are no other clients reading from
> the instance I suppose ?
>> no deadlock and only one pending transaction which is one of your inserts.
>>
>> You talk about the Oracle JDBC Driver but I still don’t see its relevance as 
>> ultimately your insertions to Virtuoso must be done one of its client 
>> interfaces / services ie either the /sparql endpoint or the Virtuoso JDBC 
>> driver I would presume, thus which is it ?
> my mistake, as I said driver is > java -cp virtjdbc3.jar
> virtuoso.jdbc3.Driver
> OpenLink Virtuoso(TM) Driver for JDBC(TM) Version 3.x [Build 3.62]
>> The "DEFINE sql:log-enable 2” pragma being passed in the SPARQL insert 
>> queries does set row by row auto-commit and turn o

Re: [Virtuoso-users] Write load high relative to disk write throughput / intensive JDBC sparql INSERT DATA INTO GRAPH

[Thomas Michaux]

Hi Hugh,

Le 10/03/2017 à 14:01, Hugh Williams a écrit :
> Hi Thomas,
>
> Is the ORCID dataset the only RDF datasets in the Virtuoso RDF Quad Store 
> currently, or are there others ?
>
> What is the size of the ORCID dataset ie triple count ?

I gave you wrong informations, because I misundertstood the process. 
Below are the correct details of our INSERT procedure from ORACLE db :

- dataset is from ORCID 2016 XML download available on this page 
https://orcid.org/content/download-file ("The file contains the public 
information associated with each user's ORCID record. Each record is 
included as a separate file in both JSON and XML. " 
https://figshare.com/articles/ORCID_Public_Data_File_2016/4134027).

They are uploaded inside ORACLE as XML records.

- then in a ORACLE PL/SQL procedure we apply "on the fly" an XSLT 
stylesheet (using Oracle XMLTRANSFORM efficient XSLT transform engine) 
to have an RDX/XML file for each ORCID XML record in the ORACLE table

- next in the process we use Jena tools to generate also "on the fly" 
TRIPLES from this RDF/XML results

- these are the  triples we're finally inserting via a JDBC "SPARQL 
INSERT DATA INTO GRAPH..." call to virtuoso from the PL/SQL Oracle 
procedure via virtuoso JDBC driver (and not ORACLE jdbc driver, my 
mistake, as you guessed)

-checking release of JDBC driver is > java -cp virtjdbc3.jar 
virtuoso.jdbc3.Driver
OpenLink Virtuoso(TM) Driver for JDBC(TM) Version 3.x [Build 3.62]

(the driver is embedded  inside ORACLE java JVM)

Thanks in advance if you have suggestions.

Last "statistics" on the graph size give  : 182 405 784 triples


this:Dataset a void:Dataset ;
  rdfs:seeAlso <http://hub.abes.fr/referentiel/ORCID/2016> ;
  rdfs:label "" ;
  void:sparqlEndpoint <http://idrefplus.v102.abes.fr:8890/sparql> ;
  void:triples 182405784 ;
  void:classes 13 ;
  void:entities 43946633 ;
  void:distinctSubjects 43922470 ;
  void:properties 32 ;
  void:distinctObjects 56509541 .

this:sameAsLinks a void:Linkset ;
  void:inDataset this:Dataset ;
  void:triples 759462 ;



>
> I would definitely suggest setting swappiness to 10 to reduce swapping to 
> disk which should speed inserts rates.
done
>
> Looking at you status() command output I see "Clients: 4177045 connects, max 
> 3 concurrent”  indicating more than 4 million SQL connections have been made 
> to Virtuoso since it was started on 9th Mar . What is making that many 
> connections, it is this insertion process
yes, it is
> or are there other clients reading from the instance also ?
none for the moment, instance is private
>   Apart from that the status() output looks fine with please of unused 
> Buffers for database working set size to be increased and still fit in memory 
> ,
don't really understand the point about buffers, but also noticed the 
use is not "maximized", because there are no other clients reading from 
the instance I suppose ?
> no deadlock and only one pending transaction which is one of your inserts.
>
> You talk about the Oracle JDBC Driver but I still don’t see its relevance as 
> ultimately your insertions to Virtuoso must be done one of its client 
> interfaces / services ie either the /sparql endpoint or the Virtuoso JDBC 
> driver I would presume, thus which is it ?
my mistake, as I said driver is > java -cp virtjdbc3.jar 
virtuoso.jdbc3.Driver
OpenLink Virtuoso(TM) Driver for JDBC(TM) Version 3.x [Build 3.62]
>
> The "DEFINE sql:log-enable 2” pragma being passed in the SPARQL insert 
> queries does set row by row auto-commit and turn off transaction logging, 
> which is the fastest transaction mode for write operations, see:
>
>   http://docs.openlinksw.com/virtuoso/fn_log_enable/
ok, thanks, a good point

Thomas
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc.  //  http://www.openlinksw.com/
> Weblog   -- http://www.openlinksw.com/blogs/
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> Twitter  -- http://twitter.com/OpenLink
> Google+  -- http://plus.google.com/100570109519069333827/
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> Universal Data Access, Integration, and Management Technology Providers
>
>
>
>> On 10 Mar 2017, at 10:54, Thomas Michaux <mich...@abes.fr> wrote:
>>
>> Hi,
>>
>> thanks Hugh, we reached 110 932 303 triples loaded from our ORCID dataset 
>> since yesterday, and still loading...
>>
>>
>>
>> Virtuoso process use VmSize: 32227664kB 32708 of memory of :
>>
>> KiB Mem : 32780296 total,   243972 free, 29985320 used,  2551004 buff/cache
>> KiB Swap:  2097148 total,  1734244 free,   362904 used.  2241196 avail Mem
>>
>> previous 4h logs :
>>

Re: [Virtuoso-users] Write load high relative to disk write throughput / intensive JDBC sparql INSERT DATA INTO GRAPH

[Thomas Michaux]

nnects, max 3 concurrent
RPC: 25061533 calls, -4177308 pending, 2 max until now, 0 queued, 37 
burst reads (0%), 0 second 5M large, 298M max

Checkpoint Remap 132107 pages, 0 mapped back. 554 s atomic time.
DB master 1000960 total 247030 free 132107 remap 44169 mapped back
   temp  165120 total 160375 free

Lock Status: 0 deadlocks of which 0 2r1w, 28 waits,
   Currently 2 threads running 0 threads waiting 0 threads in vdb.
Pending:
  1100: IER 10.34.10.171
  1: IER 10.34.10.171

Client :4175445:  Account: dba, 364 bytes in, 359 bytes out, 1 stmts.
PID: 25646, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 1 threads.
Locks:

Client :4177046:  Account: ABES, 2728 bytes in, 361 bytes out, 2 stmts.
Transaction status: PENDING, 0 threads.
Locks:


Running Statements:
 Time (msec) Text
   8 sparql DEFINE sql:log-enable 2 INSERT DATA INTO GRAPH 
<http://hub.abes.fr/refere

  76 status()


Hash indexes


44 Rows. -- 77 msec.


Le 10/03/2017 à 02:03, Hugh Williams a écrit :

Hi Thomas,

What is this JDBC Connector from Oracle that is being used for the 
inserts in RDF/XML form ?
Oracle 12.1 brings it's own jdk  1.6.0_37, so if i'm right ojdbc6.jar 
Thin Driver or OCI Driver :


"Oracle JDBC Drivers release 12.1.0.1.0 production Readme.txt :
Driver Versions
---

These are the driver versions in the 12R1 release:

  - JDBC Thin Driver 12R1
100% Java client-side JDBC driver for use in client applications,
middle-tier servers and applets.

  - JDBC OCI Driver 12R1
Client-side JDBC driver for use on a machine where OCI 12R1
is installed.

  - JDBC Thin Server-side Driver 12R1
JDBC driver for use in Java program in the database to access
remote Oracle databases.

  - JDBC Server-side Internal Driver 12R1
Server-side JDBC driver for use by Java Stored procedures.  This
driver used to be called the "JDBC Kprb Driver".





What is the ORCID dataset being used as the only one I see is in 
N-Triple format from 2014 at:


https://datahub.io/dataset/orcid_2014_dataset

will ask for this


Performing inserts with transaction would consume more memory 
maintaining the transaction than with log_enable(2) which auto commits 
without transaction logging in memory.
is it possible to have autocommit enabled the way we perform sparql 
INSERTs ? we used *DEFINE sql:log-enable 2* in the query


The  O_DIRECT param set in your INI file is an old param for which no 
real benefit has been seen on current OS’es and on a Linux system 
setting swappiness as detailed at:


https://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning#Linux-only%20--%20”swappiness 
<https://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning#Linux-only%20--%20%E2%80%9Dswappiness>"


Would give better results.
ok, known this, thought it was done but raised back to 30 after check, 
will find a way to fix it @ 10.




There is also no real need to set ColumnStore = 1 as for as the 
RDF_QUAD tables is column store by default in Virtuoso 7 , so that 
setting would only have effect on default SQL table creation


If you still have problems, can you provide a copy of your 
virtuoso.log file and the output of the “status();” command for review ...


Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  // http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



On 9 Mar 2017, at 17:28, Thomas Michaux <mich...@abes.fr 
<mailto:mich...@abes.fr>> wrote:


Hello,

We are loading ORCID 2016 in a V7 instance (Version 
07.20.3217-pthreads for Linux as of Feb 10 2017), we DO NOT want to 
use the bulk loader, instead we are providing SPARQL inserts of 
RDF/XML files via JDBC connector from Oracle.


Virtuoso is hosted on 8 cores, 32Gb platform.

We successfully inserted 75 633 079 triples until virtuoso.log 
signals performances problems on "disk write throughput", is there 
something else to optimize in the virtuoso.ini while we are in this 
"loading" phase (no SPARQL "read" query from clients at the moment ) ?


We've already done :

- full text indexation has been delayed ( DB.DBA.VT_BATCH_UPDATE ( 
'DB.DBA.RDF_OBJ', 'ON', 8640 ); )

- MaxCheckpointRemap = 505856 ( it's larger than 25% of total pages)
- UnremapQuota   = 0
- DefaultIsolation   = 2
- O_DIRECT = 1 (we are on XFS filesystem)
- ColumnStore  = 1 (we started from a new, fresh .db, 
deleted all previous existing .db, .trx)


Can we do something at transaction level ? We commit each JDBC insert 
as short as possible (1 insert-> 1 co

[Virtuoso-users] Write load high relative to disk write throughput / intensive JDBC sparql INSERT DATA INTO GRAPH

[Thomas Michaux]

Hello,

We are loading ORCID 2016 in a V7 instance (Version 07.20.3217-pthreads 
for Linux as of Feb 10 2017), we DO NOT want to use the bulk loader, 
instead we are providing SPARQL inserts of RDF/XML files via JDBC 
connector from Oracle.


Virtuoso is hosted on 8 cores, 32Gb platform.

We successfully inserted 75 633 079 triples until virtuoso.log signals 
performances problems on "disk write throughput", is there something 
else to optimize in the virtuoso.ini while we are in this "loading" 
phase (no SPARQL "read" query from clients at the moment ) ?


We've already done :

- full text indexation has been delayed ( DB.DBA.VT_BATCH_UPDATE ( 
'DB.DBA.RDF_OBJ', 'ON', 8640 ); )

- MaxCheckpointRemap = 505856 ( it's larger than 25% of total pages)
- UnremapQuota   = 0
- DefaultIsolation   = 2
- O_DIRECT = 1 (we are on XFS filesystem)
- ColumnStore  = 1 (we started from a new, fresh .db, 
deleted all previous existing .db, .trx)


Can we do something at transaction level ? We commit each JDBC insert as 
short as possible (1 insert-> 1 commit), query is :


"'sparql *DEFINE sql:log-enable 2* INSERT DATA INTO GRAPH '||graphe ||' 
{ '|| var_clob_line|| ' }'"


I can see that free memory slowly decrease, and finally the server hang.

Thanks for your help ! (Attached is virtuoso.ini)

Thomas
;
;  virtuoso.ini
;
;  Configuration file for the OpenLink Virtuoso VDBMS Server
;
;  To learn more about this product, or any other product in our
;  portfolio, please check out our web site at:
;
;  http://virtuoso.openlinksw.com/
;
;  or contact us at:
;
;  general.informat...@openlinksw.com
;
;  If you have any technical questions, please contact our support
;  staff at:
;
;  technical.supp...@openlinksw.com
;
;
;  Database setup
;
[Database]
DatabaseFile   = 
/usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso.db
ErrorLogFile   = 
/usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso.log
LockFile   = 
/usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso.lck
TransactionFile= 
/usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso20170309162914.trx
;TransactionFile= /LN_Hupe/virtuoso20151207171442.trx
xa_persistent_file = 
/usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso.pxa
ErrorLogLevel  = 7
FileExtend = 200
MaxCheckpointRemap = 505856
UnremapQuota   = 0
DefaultIsolation   = 2
Striping   = 0
TempStorage= TempDatabase

[TempDatabase]
DatabaseFile   = 
/usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso-temp.db
TransactionFile= 
/usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso-temp.trx
MaxCheckpointRemap = 2000
Striping   = 0

;
;  Server parameters
;
[Parameters]
ServerPort   = 
LiteMode = 0
DisableUnixSocket= 1
DisableTcpSocket = 0
;SSLServerPort  = 2111
;SSLCertificate = cert.pem
;SSLPrivateKey  = pk.pem
;X509ClientVerify   = 0
;X509ClientVerifyDepth  = 0
;X509ClientVerifyCAFile = ca.pem
MaxClientConnections = 10
CheckpointInterval   = 20
O_DIRECT = 1
CaseMode = 2
MaxStaticCursorRows  = 5000
CheckpointAuditTrail = 1
AllowOSCalls = 0
SchedulerInterval= 10
;DirsAllowed  = ., 
/usr/local/virtuoso-opensource/share/virtuoso/vad, /home/devel, /LN_Hupe, 
/LN_Hupe/dumpviaf
;production
DirsAllowed  = ., 
/usr/local/virtuoso-opensource/share/virtuoso/vad, /home/devel/logs
ThreadCleanupInterval= 1
ThreadThreshold  = 10
ResourcesCleanupInterval = 1
FreeTextBatchSize= 10
SingleCPU= 0
VADInstallDir= /usr/local/virtuoso-opensource/share/virtuoso/vad/
PrefixResultNames= 0
RdfFreeTextRulesSize = 100
IndexTreeMaps= 256
MaxMemPoolSize   = 2
PrefixResultNames= 0
MacSpotlight = 0
IndexTreeMaps= 64
MaxQueryMem  = 3G   ; memory allocated to query processor
VectorSize   = 1000 ; initial parallel query vector (array of query 
operations) size
MaxVectorSize= 100  ; query vector size threshold.
AdjustVectorSize = 0
ThreadsPerQuery  = 8
AsyncQueueMaxThreads = 10
ColumnStore  = 1
;server side query logging
;At run time, this may be enabled or disabled with prof_enable (), overriding 
the specification of the ini file
;QueryLog = virtuoso.qrl
;;
;; When running with large data sets, one should configure the Virtuoso
;; process to use between 2/3 to 3/5 of free system memory and to stripe
;; storage on all available disks.
;;
;; Uncomment next two lines if there is 2 GB system memory free
;NumberOfBuffers  = 17
;MaxDirtyBuffers  = 13
;; Uncomment next two lines if there is 4 GB system memory free
;NumberOfBuffers  = 34

Re: [Virtuoso-users] virtuoso.qrl (query logs) has a special format ?

[Thomas Michaux]

tuoso-opensource/bin/virtuoso-t() [0x5e52d3]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x591998]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x599e77]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c0442]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c934d]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x592026]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x59a2e7]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c0442]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c934d]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5e197f]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5e52d3]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x591998]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x599e77]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c0442]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c934d]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5e197f]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5e52d3]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x591998]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x599e77]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c0442]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c934d]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x592026]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x59a2e7]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c0442]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c934d]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5e197f]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5e52d3]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x591998]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x599e77]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c0442]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c934d]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5e197f]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5e52d3]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x591998]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x599e77]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c0442]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c934d]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x592026]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x59a2e7]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c0442]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c934d]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5e197f]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5e52d3]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x591998]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x599e77]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c0442]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c934d]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5e197f]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5e52d3]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x591998]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x599e77]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c0442]
16:13:30 /usr/local/virtuoso-opensource/bin/virtuoso-t() [0x5c934d]
16:13:30 GPF: insert.c:162 page read with no key defd





Le 21/04/2016 à 19:04, Hugh Williams a écrit :

Hi Thomas,

You analyse the Query log by querying the special “sys_query_log” view 
in Virtuoso as detailed at:


http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#querylogging

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  // http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers

On 21 Apr 2016, at 17:39, Thomas Michaux <mich...@abes.fr 
<mailto:mich...@abes.fr>> wrote:


Hello,

I activated  QueryLog = virtuoso.qrl in my 
virtuoso.ini to collect and analyze logs for queries performed 
against our Virtuoso instance but i've problems reading special chars 
in the results :


- is the file format simple text ?
- does it use compression ?
- proprietary format ? encoding ?

Thanks,

Thomas

here is an excerpt when the file is opended in Notepad++ :

After code:
  0: __ro2sq := Call __ro2sq (.O)
  5: DB.DBA.RDF_LANGUAGE_OF_OBJ := Call 
DB.DBA.RDF_LANGUAGE_OF_OBJ (__ro2sq)

  12: BReturn 0
time 0% fanout 0 inpu

[Virtuoso-users] virtuoso.qrl (query logs) has a special format ?

[Thomas Michaux]

Hello,

I activated  QueryLog = virtuoso.qrl in my virtuoso.ini 
to collect and analyze logs for queries performed against our Virtuoso 
instance but i've problems reading special chars in the results :


- is the file format simple text ?
- does it use compression ?
- proprietary format ? encoding ?

Thanks,

Thomas

here is an excerpt when the file is opended in Notepad++ :

After code:
  0: __ro2sq := Call __ro2sq (.O)
  5: DB.DBA.RDF_LANGUAGE_OF_OBJ := Call DB.DBA.RDF_LANGUAGE_OF_OBJ 
(__ro2sq)

  12: BReturn 0
time 0% fanout 0 input 0 rows
Select (__ro2sq, DB.DBA.RDF_LANGUAGE_OF_OBJ, _cvt)
}
*̽ 
@ ¼ Á¼+½  ‚Ó;L–HÒ‘   ¼
½ @@µ10.34.11.31µSPARQLÌ̼ ½ 纽  
9ö¶*  
 select __ro2sq (O), DB.DBA.RDF_LANGUAGE_OF_OBJ (__ro2sq (O)) , 
cast (b3s_lbl_order (P, 0) as int) from RDF_QUAD table option (with 
'facets')
where S = __i2id (?) and P = __i2id 
('http://www.openlinksw.com/schemas/virtrdf#label', 0) and not 
is_bnode_iri_id (O) order by 3 option (same_as)*Ì÷|„¥aHå½ ¤„¼
¼ ¼ 
¼ ¼ ¼ ¼B¼ÿ¼3¼¼½ >.¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¶  ^{ *

Params: <$29 dtp 242 (0, 0) >
time  0.95% fanout 1 input 1 rows
time20% fanout 1 input 1 rows

Precode:
  0: __i2id := Call __i2id (:0)
  5: BReturn 0
{ fork
time32% fanout 1 input 1 rows
  Multistate transitive canned,  input (__i2id) output (i-__i2id)
time16% fanout63 input 1 rows
RDF Inference subproperty iterates inferred  o=p= ##label
time30% fanout 0 input63 rows
RDF_QUAD  0.47 rows(.O, .P)
 inlined  P = inferred ,  S = k_i-__i2id
time 0% fanout 0 input 0 rows
END Node


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] list graph query performance in V7.2.2 ?

[Thomas Michaux]

ok, glad to know you're aware of these kind of issues...

don't know if it can help you but before your suggestion (when i had 
AdjustVectorSize = 1) i was experimenting "waits" issues (message " * 
Monitor: Locks are held for a long time" in the logs), i tried to 
investigate with stats query (select top 10 * from sys_l_stat order by 
waits desc;) and was only able to notice a lot of wait time was on 
VSPX_SESSION index at the top of stats locks results.

...and after your suggestion (AdjustVectorSize = 0), i dont' know if 
it's linked, but i noticed these messages in the logs :

12:56:52 OpenLink Virtuoso Universal Server
12:56:52 Version 07.20.3215-pthreads for Linux as of Mar 29 2016
12:56:52 uses parts of OpenSSL, PCRE, Html Tidy
12:57:02 Database version 3126
12:57:02 SQL Optimizer enabled (max 1000 layouts)
12:57:04 Compiler unit is timed at 0.000205 msec
12:57:21 Roll forward started
12:57:21 SQL Error: 23000 : SR175: Uniqueness violation : Violating 
unique index VSPX_SESSION on table DB.DBA.VSPX_SESSION. Transaction killed.
12:57:21 In roll forward batch commit failed code 5
12:57:21 Rfwd error: 23000: SR175: Uniqueness violation : Violating 
unique index VSPX_SESSION on table DB.DBA.VSPX_SESSION. Transaction killed.
12:57:21 SQL Error: 23000 : SR175: Uniqueness violation : Violating 
unique index VSPX_SESSION on table DB.DBA.VSPX_SESSION. Transaction killed.
12:57:21 101 transactions, 10072 bytes replayed (100 %)
12:57:21 Roll forward complete
12:57:22 Checkpoint started
12:57:36 Checkpoint finished, new log is 
/usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso20160405125721.trx
12:57:38 HTTP/WebDAV server online at 8890
12:57:38 Server online at  (pid 20868)
13:57:40 Checkpoint started
13:57:40 Checkpoint finished, new log is 
/usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso20160405135739.trx
14:57:42 Checkpoint started
14:57:42 Checkpoint finished, new log is 
/usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso20160405145741.trx
15:57:44 Checkpoint started
15:57:45 Checkpoint finished, new log is 
/usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso20160405155743.trx

Thomas


Le 05/04/2016 13:20, Hugh Williams a écrit :
> Hi Thomas,
>
> Yes, "AdjustVectorSize = 0” as it is the new default in 7.2.2 both in the INI 
> files and if not set,  following similar problems display/counting of graphs 
> and queries with long execution times …
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc.  //  http://www.openlinksw.com/
> Weblog   -- http://www.openlinksw.com/blogs/
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> Twitter  -- http://twitter.com/OpenLink
> Google+  -- http://plus.google.com/100570109519069333827/
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> Universal Data Access, Integration, and Management Technology Providers
>
>
>
>> On 5 Apr 2016, at 12:03, Thomas Michaux <mich...@abes.fr> wrote:
>>
>> yes it's improving (i added some grpah meanwhile so 46 rows is the correct 
>> number) :
>>
>> SPARQL SELECT DISTINCT ?g WHERE { GRAPH ?g { ?s ?p ?o } };
>> 46 Rows. -- 1337 msec.
>>
>> DB.DBA.SPARQL_SELECT_KNOWN_GRAPHS();
>> 46 Rows. -- 434 msec.
>>
>> I had already VectorSize   = 1000 in virtuoso.ini but I had 
>> AdjustVectorSize = 1 so changing to 0 has an effect
>>
>> We have 232 973 825, read only use case mainly, is it ok to keep 
>> AdjustVectorSize = 0 has documentation says it will be better to 
>> improve performance with = 1 ?
>>
>> Thanks,
>>
>> Thomas
>>
>>
>> Le 05/04/2016 12:52, Hugh Williams a écrit :
>>> Hi Thomas,
>>>
>>> Can you please set the following INI file params in the “[Parameters]” 
>>> section, restart Virtuoso and rerun the queries to see if the response time 
>>> for the second query and DB.DBA.SPARQL_SELECT_KNOWN_GRAPHS() improves :
>>>
>>> VectorSize   = 1000
>>> AdjustVectorSize = 0
>>>
>>>
>>> Best Regards
>>> Hugh Williams
>>> Professional Services
>>> OpenLink Software, Inc.  //  http://www.openlinksw.com/
>>> Weblog   -- http://www.openlinksw.com/blogs/
>>> LinkedIn -- http://www.linkedin.com/company/openlink-software/
>>> Twitter  -- http://twitter.com/OpenLink
>>> Google+  -- http://plus.google.com/100570109519069333827/
>>> Facebook -- http://www.facebook.com/OpenLinkSoftware
>>> Universal Data Access, Integration, and Management Technology Providers
>>>
>>>
>>>
>>>> On 5 Apr 2016, at 10:24, Thomas Michaux <mich...@abes.fr>

Re: [Virtuoso-users] list graph query performance in V7.2.2 ?

[Thomas Michaux]

yes it's improving (i added some grpah meanwhile so 46 rows is the 
correct number) :

SPARQL SELECT DISTINCT ?g WHERE { GRAPH ?g { ?s ?p ?o } };
46 Rows. -- 1337 msec.

DB.DBA.SPARQL_SELECT_KNOWN_GRAPHS();
46 Rows. -- 434 msec.

I had already VectorSize   = 1000 in virtuoso.ini but I had 
AdjustVectorSize = 1 so changing to 0 has an effect

We have 232 973 825, read only use case mainly, is it ok to keep 
AdjustVectorSize = 0 has documentation says it will be better to 
improve performance with = 1 ?

Thanks,

Thomas


Le 05/04/2016 12:52, Hugh Williams a écrit :
> Hi Thomas,
>
> Can you please set the following INI file params in the “[Parameters]” 
> section, restart Virtuoso and rerun the queries to see if the response time 
> for the second query and DB.DBA.SPARQL_SELECT_KNOWN_GRAPHS() improves :
>
> VectorSize   = 1000
> AdjustVectorSize = 0
>
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc.  //  http://www.openlinksw.com/
> Weblog   -- http://www.openlinksw.com/blogs/
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> Twitter  -- http://twitter.com/OpenLink
> Google+  -- http://plus.google.com/100570109519069333827/
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> Universal Data Access, Integration, and Management Technology Providers
>
>
>
>> On 5 Apr 2016, at 10:24, Thomas Michaux <mich...@abes.fr> wrote:
>>
>> Hello,
>>
>> I found strange that V7.2.2 is faster to answer :
>>
>>
>> SQL> sparql SELECT ?g COUNT(*) AS ?c { GRAPH ?g {?s ?p ?o.} } GROUP BY ?g 
>> ORDER BY DESC(?c);
>> 37 Rows. -- 2447 msec.
>> (use all 8 cores and CPU up to 800%)
>>
>>
>> than
>>
>>
>> SPARQL SELECT DISTINCT ?g WHERE { GRAPH ?g { ?s ?p ?o } };
>> 37 Rows. -- 522493 msec.
>> (seem to use only 1 core and CPU up to 100%)
>>
>> any idea why not should use the first query in the Conductor "graph" tab 
>> (DB.DBA.SPARQL_SELECT_KNOWN_GRAPHS)  to list the graphs ??
>>
>> Thanks,
>>
>> Thomas
>> --
>> ___
>> Virtuoso-users mailing list
>> Virtuoso-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users


--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

[Virtuoso-users] list graph query performance in V7.2.2 ?

[Thomas Michaux]

Hello,

I found strange that V7.2.2 is faster to answer :


SQL> sparql SELECT ?g COUNT(*) AS ?c { GRAPH ?g {?s ?p ?o.} } GROUP BY 
?g ORDER BY DESC(?c);

37 Rows. -- *2447 msec.*
(use all 8 cores and CPU up to 800%)


than


SPARQL SELECT DISTINCT ?g WHERE { GRAPH ?g { ?s ?p ?o } };
37 Rows. -- *522493 msec.*
(seem to use only 1 core and CPU up to 100%)

any idea why not should use the first query in the Conductor "graph" tab 
(DB.DBA.SPARQL_SELECT_KNOWN_GRAPHS)  to list the graphs ??


Thanks,

Thomas
--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] Fwd: Re: Fwd: Re: Problems using +backup-dump from Virtuoso 6 to Virtuoso 7 with +restore-crash-dump

[Thomas Michaux]

Hello,

Do you have any news ? or do you have an issue number I can follow ?

Thanks for your help.

Thomas

Le 09/12/2015 12:21, Hugh Williams a écrit :

Hi Thomas,

These issues have been reported to development to look into …

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  // http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers

On 9 Dec 2015, at 11:02, Thomas Michaux <mich...@abes.fr 
<mailto:mich...@abes.fr>> wrote:


so far I can state two problems :

1/ in a fresh empty V7 build Version 7.2.2-rc1.3215-pthreads as of 
Nov 23 2015,

using bulkloader,

ld_dir('/LN_Hupe', '*.nt', 
'http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV');

rdf_loader_run();

on a simple file :

<http://www.hub.abes.fr/springerB/ebook/3540192379/chapter/37/w> 
<http://dbpedia.org/ontology/subtitle> "Part III: a sharp result on 
the lack of uniform 
stabilization"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral>


will store the object part of the triple as a 
http://www.w3.org/2001/XMLSchema#string literal instead of
http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral (but the string 
is displayed in a correct manner)




--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] Fwd: Re: Problems using +backup-dump from Virtuoso 6 to Virtuoso 7 with +restore-crash-dump

[Thomas Michaux]

one more bad news :(

this time I tried with the dump_one_graph_nt procedure from V6 (because 
in V6 dump_nquads() calls the http_nquad() function which is not available)


I then checked with grep that the  dump file for the graph (72Mo) 
contained the XMLLiteral literals, and it has, here are few examples lines :


<http://www.hub.abes.fr/springerB/ebook/3540192379/chapter/37/w> 
<http://dbpedia.org/ontology/subtitle>  "Part III: a sharp result on the 
lack of uniform 
stabilization"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral> .
<http://www.hub.abes.fr/springerB/ebook/3540192379/chapter/86/w> 
<http://dbpedia.org/ontology/subtitle>  "The existence and unicity 
problem for 
bases"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral> .
<http://www.hub.abes.fr/springerB/ebook/3540544089/chapter/22/w> 
<http://dbpedia.org/ontology/subtitle>  "System of non linear ordinary 
differential 
equations"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral> .
<http://www.hub.abes.fr/springerB/ebook/3540761691/chapter/11/w> 
<http://dbpedia.org/ontology/subtitle>  "Experiences in LAN and WAN 
environments"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral> .


Then, I imported in V7 from this file using bulkloader :

ld_dir('/LN_Hupe', '*.nt', 
'http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV');

rdf_loader_run();


but still got the problem when querying in V7 with DATATYPE 
returning http://www.w3.org/2001/XMLSchema#string (instead of 
http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral ), for example 
same "Subject" uri and same predicate in query from the first example 
above  :


 select ?graph ?titre (DATATYPE(?titre ))from 
<http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV> where {

 GRAPH ?graph {
<http://www.hub.abes.fr/springerB/ebook/3540192379/chapter/37/w> 
<http://dbpedia.org/ontology/subtitle> ?titre

}
}


graph   titre   callret-2
http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV 
"???� rootzPart III: a sharp result on the lack of uniform 
stabilization" 	http://www.w3.org/2001/XMLSchema#string



...hope it will help to investigate

Thomas







Le 08/12/2015 16:13, Thomas Michaux a écrit :
Hugh, new information : same type error when copying V6 virtuoso.db in 
a clean "row store original" V7 , we lost XMLLiteral present in V6 datas :


select ?titre (DATATYPE(?titre ))from 
<http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV> 
where 
{<http://www.hub.abes.fr/springerB/ebook/1-85233-134-8/chapter/9/w> 
dcterms:title ?titre}


returns

titre   callret-1
"???� rootbVariants of nonlinear normal form observer design" 
http://www.w3.org/2001/XMLSchema#string



so now we can exclude error during the row to column store procedure.

Thomas


Le 08/12/2015 15:34, Thomas Michaux a écrit :

Le 08/12/2015 14:03, Hugh Williams a écrit :

Hi Thomas,

Don’t know what you mean by " pbs on datas” ???

I assume this spurious chars at the beginning of the strings ?


yes (triple count is ok in V7/V7, no RDF data loss)



Are you saying this string values where XMLLiteral in vs but are now 
XMLSchema#string in v7 , which may be the cause ?
yes XMLLiteral in V6 (via DATATYPE function call) and 
XMLSchema#string in V7




Did you convert the RDF_QUAD table to column store mode also ?


yes

As would be interesting to know if this occurred when it was in v7 
row store original mode also …


i will check this and provide information



Have you tried dumping and reloading the RDF Quad data as also 
suggested as an alternative previously ? ie


not at the moment but i will do as "plan B" if we have the same 
missing types in V7 row store test



Best Regards

Hugh Williams
Professional Services
OpenLink Software, Inc.  // http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers


On 8 Dec 2015, at 11:55, Thomas Michaux <mich...@abes.fr> wrote:

in SPARQL : select ?titre (DATATYPE(?titre ))from 
<http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV> 
where 
{<http://www.hub.abes.fr/springerB/ebook/1-85233-134-8/chapter/9/w> 
dcterms:title ?titre}


V7 result :

titre   callret-1
"???� rootbVariants of nonlinear normal form observer design" 
http://www.w3.org/2001/XMLSchema#string




V6 result:

titre   callret-1
Variants of nonlinear normal form observer design 
http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral




Thomas

Le 08/12/2015 12:42, Thomas Michaux a écrit :
when 

[Virtuoso-users] Fwd: Re: Fwd: Re: Problems using +backup-dump from Virtuoso 6 to Virtuoso 7 with +restore-crash-dump

[Thomas Michaux]

 Message transféré 
Sujet : 	Re: [Virtuoso-users] Fwd: Re: Problems using +backup-dump from 
Virtuoso 6 to Virtuoso 7 with +restore-crash-dump

Date :  Tue, 8 Dec 2015 19:16:27 +0100
De :Thomas Michaux <mich...@abes.fr>
Pour :  Hugh Williams <hwilli...@openlinksw.com>
Copie à : 	virtuoso-users@lists.sourceforge.net 
<virtuoso-users@lists.sourceforge.net>, Yann NICOLAS <nico...@abes.fr>




ok last one for today,

I found my last test wasn't really "unit testing", because i did not 
start from an empty V7 db !
I suspect I mixed a V6 conductor (present in virtuoso.db copy) my V7 RDF 
instance !


So I've done another test importing from a graph dump V6 in a clean, 
empty V7 and this time things change a bit as we do not have spurious 
chars at the beginning of the strings when querying ! (but still problem 
with the return of DATATYPE() function anyway) :


select ?graph ?titre (DATATYPE(?titre ))from 
<http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV> where {

 GRAPH ?graph {
<http://www.hub.abes.fr/springerB/ebook/3540192379/chapter/37/w> 
<http://dbpedia.org/ontology/subtitle> ?titre

}
}

graph   titre   callret-2
http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV 	"Part 
III: a sharp result on the lack of uniform stabilization" 
http://www.w3.org/2001/XMLSchema#string



Thomas



Le 08/12/2015 17:47, Thomas Michaux a écrit :

one more bad news :(

this time I tried with the dump_one_graph_nt procedure from V6 
(because in V6 dump_nquads() calls the http_nquad() function which is 
not available)


I then checked with grep that the  dump file for the graph (72Mo) 
contained the XMLLiteral literals, and it has, here are few examples 
lines :


<http://www.hub.abes.fr/springerB/ebook/3540192379/chapter/37/w> 
<http://dbpedia.org/ontology/subtitle> "Part III: a sharp result on 
the lack of uniform 
stabilization"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral> .
<http://www.hub.abes.fr/springerB/ebook/3540192379/chapter/86/w> 
<http://dbpedia.org/ontology/subtitle> "The existence and unicity 
problem for 
bases"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral> .
<http://www.hub.abes.fr/springerB/ebook/3540544089/chapter/22/w> 
<http://dbpedia.org/ontology/subtitle> "System of non linear ordinary 
differential 
equations"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral> .
<http://www.hub.abes.fr/springerB/ebook/3540761691/chapter/11/w> 
<http://dbpedia.org/ontology/subtitle> "Experiences in LAN and WAN 
environments"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral> .


Then, I imported in V7 from this file using bulkloader :

ld_dir('/LN_Hupe', '*.nt', 
'http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV');

rdf_loader_run();


but still got the problem when querying in V7 with DATATYPE 
returning http://www.w3.org/2001/XMLSchema#string (instead of 
http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral ), for example 
same "Subject" uri and same predicate in query from the first example 
above  :


 select ?graph ?titre (DATATYPE(?titre ))from 
<http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV> where {

 GRAPH ?graph {
<http://www.hub.abes.fr/springerB/ebook/3540192379/chapter/37/w> 
<http://dbpedia.org/ontology/subtitle> ?titre

}
}


graph   titre   callret-2
http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV 
"???� rootzPart III: a sharp result on the lack of uniform 
stabilization" 	http://www.w3.org/2001/XMLSchema#string



...hope it will help to investigate

Thomas








--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] Fwd: Re: Fwd: Re: Problems using +backup-dump from Virtuoso 6 to Virtuoso 7 with +restore-crash-dump

[Thomas Michaux]

so far I can state two problems :

1/ in a fresh empty V7 build Version 7.2.2-rc1.3215-pthreads as of Nov 
23 2015,

using bulkloader,

ld_dir('/LN_Hupe', '*.nt', 
'http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV');

rdf_loader_run();

on a simple file :

<http://www.hub.abes.fr/springerB/ebook/3540192379/chapter/37/w> 
<http://dbpedia.org/ontology/subtitle> "Part III: a sharp result on the 
lack of uniform 
stabilization"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral>


will store the object part of the triple as a 
http://www.w3.org/2001/XMLSchema#string literal instead of
http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral (but the string is 
displayed in a correct manner)




2/ something in my virtuoso.db V6 Version 6.1.6.3127-pthreads as of Oct 
19 2012  is causing the same problem when i try to use it as the 
starting point of my V7 installation,
+ it adds misinterpretation of the literal string and decodes it with 
spurious chars.



Thomas Michaux


Le 09/12/2015 10:15, Thomas Michaux a écrit :


 Message transféré 
Sujet : 	Re: [Virtuoso-users] Fwd: Re: Problems using +backup-dump 
from Virtuoso 6 to Virtuoso 7 with +restore-crash-dump

Date :  Tue, 8 Dec 2015 19:16:27 +0100
De :Thomas Michaux <mich...@abes.fr>
Pour :  Hugh Williams <hwilli...@openlinksw.com>
Copie à : 	virtuoso-users@lists.sourceforge.net 
<virtuoso-users@lists.sourceforge.net>, Yann NICOLAS <nico...@abes.fr>




ok last one for today,

I found my last test wasn't really "unit testing", because i did not 
start from an empty V7 db !
I suspect I mixed a V6 conductor (present in virtuoso.db copy) my V7 
RDF instance !


So I've done another test importing from a graph dump V6 in a clean, 
empty V7 and this time things change a bit as we do not have spurious 
chars at the beginning of the strings when querying ! (but still 
problem with the return of DATATYPE() function anyway) :


select ?graph ?titre (DATATYPE(?titre ))from 
<http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV> where {

 GRAPH ?graph {
<http://www.hub.abes.fr/springerB/ebook/3540192379/chapter/37/w> 
<http://dbpedia.org/ontology/subtitle> ?titre

}
}

graph   titre   callret-2
http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV 
"Part III: a sharp result on the lack of uniform stabilization" 
http://www.w3.org/2001/XMLSchema#string



Thomas



Le 08/12/2015 17:47, Thomas Michaux a écrit :

one more bad news :(

this time I tried with the dump_one_graph_nt procedure from V6 
(because in V6 dump_nquads() calls the http_nquad() function which is 
not available)


I then checked with grep that the  dump file for the graph (72Mo) 
contained the XMLLiteral literals, and it has, here are few examples 
lines :


<http://www.hub.abes.fr/springerB/ebook/3540192379/chapter/37/w> 
<http://dbpedia.org/ontology/subtitle> "Part III: a sharp result on 
the lack of uniform 
stabilization"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral> .
<http://www.hub.abes.fr/springerB/ebook/3540192379/chapter/86/w> 
<http://dbpedia.org/ontology/subtitle> "The existence and unicity 
problem for 
bases"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral> .
<http://www.hub.abes.fr/springerB/ebook/3540544089/chapter/22/w> 
<http://dbpedia.org/ontology/subtitle> "System of non linear ordinary 
differential 
equations"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral> .
<http://www.hub.abes.fr/springerB/ebook/3540761691/chapter/11/w> 
<http://dbpedia.org/ontology/subtitle> "Experiences in LAN and WAN 
environments"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral> .


Then, I imported in V7 from this file using bulkloader :

ld_dir('/LN_Hupe', '*.nt', 
'http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV');

rdf_loader_run();


but still got the problem when querying in V7 with DATATYPE 
returning http://www.w3.org/2001/XMLSchema#string (instead of 
http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral ), for example 
same "Subject" uri and same predicate in query from the first example 
above  :


 select ?graph ?titre (DATATYPE(?titre ))from 
<http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV> 
where {

 GRAPH ?graph {
<http://www.hub.abes.fr/springerB/ebook/3540192379/chapter/37/w> 
<http://dbpedia.org/ontology/subtitle> ?titre

}
}


graph   titre   callret-2
http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV 	"???� 
rootzPart III: a sharp result on the lack of uniform stabilization" 
http://www.w3.org/2001/XMLSchema#string



...hope it will help to investigate

Thomas










-

Re: [Virtuoso-users] Problems using +backup-dump from Virtuoso 6 to Virtuoso 7 with +restore-crash-dump

[Thomas Michaux]

Hello,

Thanks Hugh,

we performed as explained in 
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/UpgradingToVOS610#Upgrading%20from%20Release%206.x%20to%20Release%207.x 
( convert the RDF_QUAD table to Column Store )


and it worked... except now we have pbs on datas (which was correct in 
V6), for exemple :


select ?titre from 
<http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV> where 
{?s dcterms:title ?titre}

 return in SPARQL query results of the conductor interface in V6 :

titre
Variants of nonlinear normal form observer design
Input/State linearization
On achieving Lp (ℓp) performance with global internal stability for 
linear plants with saturating actuators
Design criteria for uncertain models with structured and unstructured 
uncertainties

Sliding mode control of systems with delayed states and controls
On global stabilization of nonlinear dynamical systems
On sampled data variable structure control systems
Stochastic optimal control problems
Modeling, Estimation, and Their Applications for Distributed Parameter 
Systems

Optimal input design

...

the same SPARQL query on V7 now returns in the conductor :

titre
"???� rootbVariants of nonlinear normal form observer design"
"???� root2Input/State linearization"
"???� root??On achieving Lp (ℓp) performance with global internal 
stability for linear plants with saturating actuators"
"???� root??Design criteria for uncertain models with structured and 
unstructured uncertainties"
"???� root??Sliding mode control of systems with delayed states and 
controls"

"???� rootlOn global stabilization of nonlinear dynamical systems"
"???� rootdOn sampled data variable structure control systems"
"???� rootFStochastic optimal control problems"
"Modeling, Estimation, and Their Applications for Distributed Parameter 
Systems"

"???� root(Optimal input design"


...

Also tried directly SPARQL query in isql client and same divergences, 
(so it's excluding presentation problems in the conductor i suppose) see 
attached screenshots.


Do you think the faulty part resides in the reindexing  process or in 
the driver accessing the datas in V7 ?


Thanks,

Thomas Michaux.


PS: I'm planning a test to see the datas from V6 into V7 (just copying 
virtusoso.db) without converting to column store indexes.



Le 05/12/2015 03:22, Hugh Williams a écrit :

Hi Thomas,

That won’t work as the +backup-dump is not compatible even between 6.x 
engine numbers which are thus included in the trx file to indicate 
this should you attempt to restore / replay with a different database 
engine number hence the error you indicating the server version don’t 
match.


So you have to perform the upgrade by copying the database as is  and 
starting with a v7 binary and then convert the RDF_QUAD table to 
Column Store mode as indicated at:


http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/UpgradingToVOS610#Upgrading%20from%20Release%206.x%20to%20Release%207.x

If you just have RDF Datasets then you could also dump them to file 
datasets in nquad form and then reloading into an empty/virtuoso v7 
database as detailed at:


http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFDumpNQuad

Then reload with the Virtuoso RDF Bulk loader into the empty v7 database:

http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtBulkRDFLoader


Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  // http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers

On 4 Dec 2015, at 08:46, Thomas Michaux <mich...@abes.fr 
<mailto:mich...@abes.fr>> wrote:


Hello,

I have a problem trying to build a new .db in Virtuoso 7 from a dump of
Virtuoso 6  (i.e. we do not want to just copy virtuoso.db to have
benefits of Column Store mode) :

The dump is built from Virtuoso Open Source Edition (multi threaded)
Version 6.1.6.3127-pthreads as of Oct 19 2012
Compiled for Linux (x86_64-unknown-linux-gnu)
Copyright (C) 1998-2012 OpenLink Software

after a checkpoint; shutdown; and /usr/local/bin/virtuoso-t +configfile
/usr/local/var/lib/virtuoso/db/virtuoso.ini +backup-dump

Then we are trying to read this dump transaction file on another host
with Virtuoso Open Source Edition (Column Store) (multi threaded)
Version 7.2.2-rc1.3215-pthreads as of Nov 23 2015
Compiled for Linux (x86_64-unknown-linux-gnu)
Copyright (C) 1998-2015 OpenLink Software

with ./virtuoso-t +configfile
/usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso.ini
+restore-crash-dump +foreground

but...
19:11:18 Roll forward started
19:11:1

Re: [Virtuoso-users] Problems using +backup-dump from Virtuoso 6 to Virtuoso 7 with +restore-crash-dump

[Thomas Michaux]

Important precision,

we have focused this problems on a specific type of literals :

Je dois préciser que le pb concerne des données de type Littéraux XML 
(http://www.w3.org/TR/REC-rdf-syntax/#section-Syntax-XML-literals) :



<http://www.hub.abes.fr/springerB/ebook/1-85233-134-8/chapter/9/w>
<http://purl.org/dc/terms/title>  "Variants of nonlinear normal form observer 
design"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral>  .



Le 08/12/2015 10:39, Thomas Michaux a écrit :

Hello,

Thanks Hugh,

we performed as explained in 
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/UpgradingToVOS610#Upgrading%20from%20Release%206.x%20to%20Release%207.x 
( convert the RDF_QUAD table to Column Store )


and it worked... except now we have pbs on datas (which was correct in 
V6), for exemple :


select ?titre from 
<http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV> 
where {?s dcterms:title ?titre}

 return in SPARQL query results of the conductor interface in V6 :

titre
Variants of nonlinear normal form observer design
Input/State linearization
On achieving Lp (ℓp) performance with global internal stability for 
linear plants with saturating actuators
Design criteria for uncertain models with structured and unstructured 
uncertainties

Sliding mode control of systems with delayed states and controls
On global stabilization of nonlinear dynamical systems
On sampled data variable structure control systems
Stochastic optimal control problems
Modeling, Estimation, and Their Applications for Distributed Parameter 
Systems

Optimal input design

...

the same SPARQL query on V7 now returns in the conductor :

titre
"???� rootbVariants of nonlinear normal form observer design"
"???� root2Input/State linearization"
"???� root??On achieving Lp (ℓp) performance with global internal 
stability for linear plants with saturating actuators"
"???� root??Design criteria for uncertain models with structured and 
unstructured uncertainties"
"???� root??Sliding mode control of systems with delayed states and 
controls"

"???� rootlOn global stabilization of nonlinear dynamical systems"
"???� rootdOn sampled data variable structure control systems"
"???� rootFStochastic optimal control problems"
"Modeling, Estimation, and Their Applications for Distributed 
Parameter Systems"

"???� root(Optimal input design"


...

Also tried directly SPARQL query in isql client and same divergences, 
(so it's excluding presentation problems in the conductor i suppose) 
see attached screenshots.


Do you think the faulty part resides in the reindexing  process or in 
the driver accessing the datas in V7 ?


Thanks,

Thomas Michaux.


PS: I'm planning a test to see the datas from V6 into V7 (just copying 
virtusoso.db) without converting to column store indexes.



Le 05/12/2015 03:22, Hugh Williams a écrit :

Hi Thomas,

That won’t work as the +backup-dump is not compatible even between 
6.x engine numbers which are thus included in the trx file to 
indicate this should you attempt to restore / replay with a different 
database engine number hence the error you indicating the server 
version don’t match.


So you have to perform the upgrade by copying the database as is  and 
starting with a v7 binary and then convert the RDF_QUAD table to 
Column Store mode as indicated at:


http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/UpgradingToVOS610#Upgrading%20from%20Release%206.x%20to%20Release%207.x

If you just have RDF Datasets then you could also dump them to file 
datasets in nquad form and then reloading into an empty/virtuoso v7 
database as detailed at:


http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFDumpNQuad

Then reload with the Virtuoso RDF Bulk loader into the empty v7 database:

http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtBulkRDFLoader


Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  // http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers


On 4 Dec 2015, at 08:46, Thomas Michaux <mich...@abes.fr> wrote:

Hello,

I have a problem trying to build a new .db in Virtuoso 7 from a dump of
Virtuoso 6  (i.e. we do not want to just copy virtuoso.db to have
benefits of Column Store mode) :

The dump is built from Virtuoso Open Source Edition (multi threaded)
Version 6.1.6.3127-pthreads as of Oct 19 2012
Compiled for Linux (x86_64-unknown-linux-gnu)
Copyright (C) 1998-2012 OpenLink Software

after a checkpoint; shutdown; and /usr/local/bin/virtuoso-t +configfile
/usr/local/va

[Virtuoso-users] Fwd: Re: Problems using +backup-dump from Virtuoso 6 to Virtuoso 7 with +restore-crash-dump

[Thomas Michaux]

when we compare the datatypes from V6 and v7 it seems we'd lost the 
"strong" typing in V6 to a "generic" V7 type 
http://www.w3.org/2001/XMLSchema#string :



SELECT DB.DBA.RDF_DATATYPE_OF_OBJ( "s_1_1_t0"."O") AS "titre_datatype", 
DB.DBA.RDF_LANGUAGE_OF_OBJ( "s_1_1_t0"."O") AS "titre_langage"

FROM DB.DBA.RDF_QUAD AS "s_1_1_t0"
WHERE
  "s_1_1_t0"."G" = __i2idn ( __bft( 
'http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV' , 1))

  AND
  "s_1_1_t0"."S" = __i2idn ( __bft( 
'http://www.hub.abes.fr/springerB/ebook/1-85233-134-8/chapter/9/w' , 1))

  AND
  "s_1_1_t0"."P" = __i2idn ( __bft( 'http://purl.org/dc/terms/title' , 1))
OPTION (QUIETCAST)


V7 :
titre_datatype titre_langage
LONG VARCHAR LONG VARCHAR
___

http://www.w3.org/2001/XMLSchema#string BLOB 0 chars

1 Rows. -- 2 msec.

V6 :

titre_datatype titre_langage
LONG VARCHAR LONG VARCHAR
___

#i232 BLOB 0 chars

1 Rows. -- 1 msec.


Thomas

 Message transféré 
Sujet : 	Re: [Virtuoso-users] Problems using +backup-dump from Virtuoso 
6 to Virtuoso 7 with +restore-crash-dump

Date :  Tue, 8 Dec 2015 11:16:32 +0100
De :Thomas Michaux <mich...@abes.fr>
Pour : 	virtuoso-users@lists.sourceforge.net 
<virtuoso-users@lists.sourceforge.net>

Copie à :   Hugh Williams <hwilli...@openlinksw.com>



Important precision,

we have focused this problems on a specific type of literals :

Je dois préciser que le pb concerne des données de type Littéraux XML 
(http://www.w3.org/TR/REC-rdf-syntax/#section-Syntax-XML-literals) :



<http://www.hub.abes.fr/springerB/ebook/1-85233-134-8/chapter/9/w>    
<http://purl.org/dc/terms/title>  "Variants of nonlinear normal form observer 
design"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral>  .



Le 08/12/2015 10:39, Thomas Michaux a écrit :

Hello,

Thanks Hugh,

we performed as explained in 
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/UpgradingToVOS610#Upgrading%20from%20Release%206.x%20to%20Release%207.x 
( convert the RDF_QUAD table to Column Store )


and it worked... except now we have pbs on datas (which was correct in 
V6), for exemple :


select ?titre from 
<http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV> 
where {?s dcterms:title ?titre}

 return in SPARQL query results of the conductor interface in V6 :

titre
Variants of nonlinear normal form observer design
Input/State linearization
On achieving Lp (ℓp) performance with global internal stability for 
linear plants with saturating actuators
Design criteria for uncertain models with structured and unstructured 
uncertainties

Sliding mode control of systems with delayed states and controls
On global stabilization of nonlinear dynamical systems
On sampled data variable structure control systems
Stochastic optimal control problems
Modeling, Estimation, and Their Applications for Distributed Parameter 
Systems

Optimal input design

...

the same SPARQL query on V7 now returns in the conductor :

titre
"???� rootbVariants of nonlinear normal form observer design"
"???� root2Input/State linearization"
"???� root??On achieving Lp (ℓp) performance with global 
internal stability for linear plants with saturating actuators"
"???� root??Design criteria for uncertain models with structured 
and unstructured uncertainties"
"???� root??Sliding mode control of systems with delayed states 
and controls"

"???� rootlOn global stabilization of nonlinear dynamical systems"
"???� rootdOn sampled data variable structure control systems"
"???� rootFStochastic optimal control problems"
"Modeling, Estimation, and Their Applications for Distributed 
Parameter Systems"

"???� root(Optimal input design"


...

Also tried directly SPARQL query in isql client and same divergences, 
(so it's excluding presentation problems in the conductor i suppose) 
see attached screenshots.


Do you think the faulty part resides in the reindexing  process or in 
the driver accessing the datas in V7 ?


Thanks,

Thomas Michaux.


PS: I'm planning a test to see the datas from V6 into V7 (just copying 
virtusoso.db) without converting to column store indexes.



Le 05/12/2015 03:22, Hugh Williams a écrit :

Hi Thomas,

That won’t work as the +backup-dump is not compatible even between 
6.x engine numbers which are thus included in the trx file to 
indicate this should you attempt to restore / replay with a different 
database engine number hence the error you indicating the server 
version don’t match.


So you have to perfor

Re: [Virtuoso-users] Fwd: Re: Problems using +backup-dump from Virtuoso 6 to Virtuoso 7 with +restore-crash-dump

[Thomas Michaux]

Le 08/12/2015 14:03, Hugh Williams a écrit :

Hi Thomas,

Don’t know what you mean by " pbs on datas” ???

I assume this spurious chars at the beginning of the strings ?


yes (triple count is ok in V7/V7, no RDF data loss)



Are you saying this string values where XMLLiteral in vs but are now 
XMLSchema#string in v7 , which may be the cause ?

yes XMLLiteral in V6 (via DATATYPE function call) and XMLSchema#string in V7



Did you convert the RDF_QUAD table to column store mode also ?


yes

As would be interesting to know if this occurred when it was in v7 row 
store original mode also …


i will check this and provide information



Have you tried dumping and reloading the RDF Quad data as also 
suggested as an alternative previously ? ie


not at the moment but i will do as "plan B" if we have the same missing 
types in V7 row store test



Best Regards

Hugh Williams
Professional Services
OpenLink Software, Inc.  // http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers

On 8 Dec 2015, at 11:55, Thomas Michaux <mich...@abes.fr 
<mailto:mich...@abes.fr>> wrote:


in SPARQL : select ?titre (DATATYPE(?titre ))from 
<http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV> 
where 
{<http://www.hub.abes.fr/springerB/ebook/1-85233-134-8/chapter/9/w> 
dcterms:title ?titre}


V7 result :

titre   callret-1
"???� rootbVariants of nonlinear normal form observer design" 
http://www.w3.org/2001/XMLSchema#string




V6 result:

titre   callret-1
Variants of nonlinear normal form observer design 
http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral




Thomas

Le 08/12/2015 12:42, Thomas Michaux a écrit :
when we compare the datatypes from V6 and v7 it seems we'd lost the 
"strong" typing in V6 to a "generic" V7 type 
http://www.w3.org/2001/XMLSchema#string :



SELECT DB.DBA.RDF_DATATYPE_OF_OBJ( "s_1_1_t0"."O") AS 
"titre_datatype", DB.DBA.RDF_LANGUAGE_OF_OBJ( "s_1_1_t0"."O") AS 
"titre_langage"

FROM DB.DBA.RDF_QUAD AS "s_1_1_t0"
WHERE
  "s_1_1_t0"."G" = __i2idn ( __bft( 
'http://www.hub.abes.fr/springer/ebooksLN2011/SPR_EBOOK_642_19NOV' , 1))

  AND
  "s_1_1_t0"."S" = __i2idn ( __bft( 
'http://www.hub.abes.fr/springerB/ebook/1-85233-134-8/chapter/9/w' , 1))

  AND
  "s_1_1_t0"."P" = __i2idn ( __bft( 'http://purl.org/dc/terms/title' 
, 1))

OPTION (QUIETCAST)


V7 :
titre_datatype titre_langage
LONG VARCHAR LONG VARCHAR
___

http://www.w3.org/2001/XMLSchema#string BLOB 0 chars

1 Rows. -- 2 msec.

V6 :

titre_datatype titre_langage
LONG VARCHAR LONG VARCHAR
___

#i232 BLOB 0 chars

1 Rows. -- 1 msec.


Thomas

---- Message transféré 
Sujet : 	Re: [Virtuoso-users] Problems using +backup-dump from 
Virtuoso 6 to Virtuoso 7 with +restore-crash-dump

Date :  Tue, 8 Dec 2015 11:16:32 +0100
De :Thomas Michaux <mich...@abes.fr>
Pour : 	virtuoso-users@lists.sourceforge.net 
<virtuoso-users@lists.sourceforge.net>

Copie à :   Hugh Williams <hwilli...@openlinksw.com>



Important precision,

we have focused this problems on a specific type of literals :

Je dois préciser que le pb concerne des données de type Littéraux 
XML (http://www.w3.org/TR/REC-rdf-syntax/#section-Syntax-XML-literals) :



<http://www.hub.abes.fr/springerB/ebook/1-85233-134-8/chapter/9/w>
<http://purl.org/dc/terms/title>  "Variants of nonlinear normal form observer 
design"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral>  .




--
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

[Virtuoso-users] Problems using +backup-dump from Virtuoso 6 to Virtuoso 7 with +restore-crash-dump

[Thomas Michaux]

Hello,

I have a problem trying to build a new .db in Virtuoso 7 from a dump of 
Virtuoso 6  (i.e. we do not want to just copy virtuoso.db to have 
benefits of Column Store mode) :

The dump is built from Virtuoso Open Source Edition (multi threaded)
Version 6.1.6.3127-pthreads as of Oct 19 2012
Compiled for Linux (x86_64-unknown-linux-gnu)
Copyright (C) 1998-2012 OpenLink Software

after a checkpoint; shutdown; and /usr/local/bin/virtuoso-t +configfile 
/usr/local/var/lib/virtuoso/db/virtuoso.ini +backup-dump

Then we are trying to read this dump transaction file on another host 
with Virtuoso Open Source Edition (Column Store) (multi threaded)
Version 7.2.2-rc1.3215-pthreads as of Nov 23 2015
Compiled for Linux (x86_64-unknown-linux-gnu)
Copyright (C) 1998-2015 OpenLink Software

with ./virtuoso-t +configfile 
/usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso.ini 
+restore-crash-dump +foreground

but...
19:11:18 Roll forward started
19:11:18 The transaction log file has been produced by server version 
'06.01.3127'. The version of this server is '07.20.3215'. If the 
transaction log is empty or you do not want to replay it then delete it 
and start the server again. Otherwise replay the log using the server of 
version '06.01.3127' and make checkpoint and shutdown to ensure that the 
log is empty, then delete it and start using new version.
19:11:18 Server exiting


Any advices ?

Thanks,

Thomas Michaux






--
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911=/4140
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

[Virtuoso-users] pb with xslt/xpath ?

[Thomas Michaux]

Hello,

Could you give precision on the xslt transformer used in Virtuoso ? 
because I'm having different results when transforming same XML+XSLT  
with Saxon 9 or with Virtuoso Open Source Edition

  (multi threaded)
Version 6.1.6.3127-pthreads as of Oct 19 2012
Compiled for Linux (x86_64-unknown-linux-gnu)
Copyright (C) 1998-2012 OpenLink Softwar

Thanks

--
Try New Relic Now  We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app,  servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

[Virtuoso-users] caching problems with sparql endpoint + on the fly XSLT transform using DAV directory and xslt-uri parameter?

[Thomas Michaux]

Hello,

I'm querying a Virtuoso endpoint via HTTP 
http://localhost:8890/sparql/?query=define...

i'm expecting RDF/XML + using a XSLT stylesheet served by webDAV 
xslt-uri=http://localhost:8890/DAV/VAD/springerRDF2MarcXML.xsl

I granted read permisson to other users on the stylesheet ressource so 
http://localhost:8890/DAV/VAD/springerRDF2MarcXML.xsl is available by 
HTTP to everyone.

Everything is ok, transformation on the fly is a success having 
uncommented ExternalXsltSource=1 in virtuoso.ini,

...but once I edited my stylesheet in the DAV directory, the changes I 
made were NOT effective in following transformations :(
( I tested my stylesheet outside in a  standalone transformer and it's 
ok, no the source of  this problem)

I also tried by deleting old XSLT, then uploading the new one, same 
behaviour : always the old fashioned transform results from old 
stylesheet, even if directly requesting XSLT by HTTP is now serving the 
correct new one at http://localhost:8890/DAV/VAD/springerRDF2MarcXML.xsl

Is there any caching system I need to force transform processor for 
refreshing XSLT stylesheet ?
stopping and restarting Virtuoso will do the job but it's not really a 
solution  :)

Thanks in advance,

Thomas






--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_mar
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] Loading ttl files halts

[Thomas Michaux]

Hi,

Maybe it's related to the logging system stacking info in RAM, take a 
look at this :


One of the side effects of the default |log_enable = 2| setting is that 
triggers are not enabled to speed the loading of data. If triggers are 
required for RDF Graph replication between nodes etc. then the 
log_enable mode should be set to *3* when calling the rdf_loader_run() 
function as follows:


rdf_loader_run (log_enable=3);



in http://www.openlinksw.com/dataspace/dav/wiki/Main/VirtBulkRDFLoader

Le 27/11/2012 18:16, Y. Dong a écrit :

Hi there,


Currently I am trying to load yago2s into the virtuoso database. There are 15 
ttl(turtle) files, totalling 8.5GB in size. The server is running on an amazon 
instance with 64bit Ubuntu 12.04.01 LTS, 8GB Memory, and more than enough disk 
space.  Virtuoso  Server Version is 06.01.3127. I use ld_dir() and 
rdf_loader_run() to load those files. virtuoso.ini is set to utilise the 8GB 
memory.

At first, the loading process ran smoothly, but after half way, the virtuoso 
server stops responding. I can see 100% CPU usage by virtuoso and I can enter 
isql-v to start the virtuoso command line. But the server doesn't respond to 
any command I type in isql-v, it just stalls and I have to use ctrl+c to exit.  
Now I can't even use status() to see basic stats of the server. The http 
virtuoso server also stops responding. So I believe the whole virtuoso process 
stalls somehow. There isn't anything unusual in the log file virtuoso.log 
(attached below).

Has anyone encountered similar problems before?

Tue Nov 27 2012
15:20:34 INFO: { Loading plugin 1: Type `plain', file `wikiv' in 
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO:   WikiV version 0.6 from OpenLink Software
15:20:34 INFO:   Support functions for WikiV collaboration tool
15:20:34 INFO:   SUCCESS plugin 1: loaded from 
/usr/local//lib/virtuoso/hosting/wikiv.so }
15:20:34 INFO: { Loading plugin 2: Type `plain', file `mediawiki' in 
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO:   MediaWiki version 0.1 from OpenLink Software
15:20:34 INFO:   Support functions for MediaWiki collaboration tool
15:20:34 INFO:   SUCCESS plugin 2: loaded from 
/usr/local//lib/virtuoso/hosting/mediawiki.so }
15:20:34 INFO: { Loading plugin 3: Type `plain', file `creolewiki' in 
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO:   CreoleWiki version 0.1 from OpenLink Software
15:20:34 INFO:   Support functions for CreoleWiki collaboration tool
15:20:34 INFO:   SUCCESS plugin 3: loaded from 
/usr/local//lib/virtuoso/hosting/creolewiki.so }
15:20:34 INFO: { Loading plugin 4: Type `plain', file `im' in 
`/usr/local//lib/virtuoso/hosting'
15:20:35 INFO:   IM version 0.6 from OpenLink Software
15:20:35 INFO:   Support functions for Image Magick 6.6.9
15:20:35 INFO:   SUCCESS plugin 4: loaded from 
/usr/local//lib/virtuoso/hosting/im.so }
15:20:35 INFO: OpenLink Virtuoso Universal Server
15:20:35 INFO: Version 06.01.3127-pthreads for Linux as of Sep 14 2012
15:20:35 INFO: uses parts of OpenSSL, PCRE, Html Tidy
15:20:41 INFO: Database version 3126
15:20:41 INFO: SQL Optimizer enabled (max 1000 layouts)
15:20:43 INFO: Compiler unit is timed at 0.001027 msec
15:21:03 DEBUG: built-in procedure repl_undot_name overruled by the RDBMS
15:21:03 DEBUG: built-in procedure REPL_FQNAME overruled by the RDBMS
15:21:03 DEBUG: built-in procedure REPL_COLTYPE_PS overruled by the RDBMS
15:21:03 DEBUG: built-in procedure REPL_COLTYPE overruled by the RDBMS
15:21:04 INFO: Roll forward started
15:21:42 INFO: 1000 transactions, 7274496 bytes replayed (13 %)
15:22:13 INFO: 2000 transactions, 14548992 bytes replayed (27 %)
15:22:42 INFO: 3000 transactions, 21856256 bytes replayed (40 %)
15:23:11 INFO: 4000 transactions, 29163520 bytes replayed (54 %)
15:23:38 INFO: 5000 transactions, 36405248 bytes replayed (68 %)
15:23:54 INFO: 6000 transactions, 42500096 bytes replayed (79 %)
15:24:00 INFO: 7000 transactions, 47677440 bytes replayed (89 %)
15:24:06 INFO: 8000 transactions, 52396032 bytes replayed (97 %)
15:24:08 INFO: 8250 transactions, 53526661 bytes replayed (100 %)
15:24:08 INFO: Roll forward complete
15:24:12 INFO: Checkpoint started
15:24:22 INFO: Checkpoint finished, log reused
15:24:22 INFO: HTTP/WebDAV server online at 8890
15:24:22 INFO: Server online at  (pid 28660)
15:28:59 INFO: PL LOG: Loader started
15:29:20 INFO: PL LOG: Loader started
15:30:16 INFO: PL LOG: Loader started


--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net

Re: [Virtuoso-users] Loading ttl files halts

[Thomas Michaux]

same trick for deleting graph : 
http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtTipsAndTricksGuideDeleteLargeGraphs


Such large graphs can be cleared by changing the transaction log mode to 
autocommit on each operation, deleting the graph(s) or triples, and then 
setting the log mode back to its original setting. This is easily done 
using the Virtuoso log_enable function, with the settings log_enable(3,1).


This function may be called on its own, prior to the delete operation, 
via iSQL (either command-line or the Conductor variant), as shown:


log_enable(3,1);
SPARQL CLEAR GRAPH graph-name;



Le 27/11/2012 18:52, Thomas Michaux a écrit :

Hi,

Maybe it's related to the logging system stacking info in RAM, take a 
look at this :


One of the side effects of the default |log_enable = 2| setting is 
that triggers are not enabled to speed the loading of data. If 
triggers are required for RDF Graph replication between nodes etc. 
then the log_enable mode should be set to *3* when calling the 
rdf_loader_run() function as follows:

rdf_loader_run (log_enable=3);


in http://www.openlinksw.com/dataspace/dav/wiki/Main/VirtBulkRDFLoader

Le 27/11/2012 18:16, Y. Dong a écrit :

Hi there,


Currently I am trying to load yago2s into the virtuoso database. There are 15 
ttl(turtle) files, totalling 8.5GB in size. The server is running on an amazon 
instance with 64bit Ubuntu 12.04.01 LTS, 8GB Memory, and more than enough disk 
space.  Virtuoso  Server Version is 06.01.3127. I use ld_dir() and 
rdf_loader_run() to load those files. virtuoso.ini is set to utilise the 8GB 
memory.

At first, the loading process ran smoothly, but after half way, the virtuoso 
server stops responding. I can see 100% CPU usage by virtuoso and I can enter 
isql-v to start the virtuoso command line. But the server doesn't respond to 
any command I type in isql-v, it just stalls and I have to use ctrl+c to exit.  
Now I can't even use status() to see basic stats of the server. The http 
virtuoso server also stops responding. So I believe the whole virtuoso process 
stalls somehow. There isn't anything unusual in the log file virtuoso.log 
(attached below).

Has anyone encountered similar problems before?

Tue Nov 27 2012
15:20:34 INFO: { Loading plugin 1: Type `plain', file `wikiv' in 
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO:   WikiV version 0.6 from OpenLink Software
15:20:34 INFO:   Support functions for WikiV collaboration tool
15:20:34 INFO:   SUCCESS plugin 1: loaded from 
/usr/local//lib/virtuoso/hosting/wikiv.so }
15:20:34 INFO: { Loading plugin 2: Type `plain', file `mediawiki' in 
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO:   MediaWiki version 0.1 from OpenLink Software
15:20:34 INFO:   Support functions for MediaWiki collaboration tool
15:20:34 INFO:   SUCCESS plugin 2: loaded from 
/usr/local//lib/virtuoso/hosting/mediawiki.so }
15:20:34 INFO: { Loading plugin 3: Type `plain', file `creolewiki' in 
`/usr/local//lib/virtuoso/hosting'
15:20:34 INFO:   CreoleWiki version 0.1 from OpenLink Software
15:20:34 INFO:   Support functions for CreoleWiki collaboration tool
15:20:34 INFO:   SUCCESS plugin 3: loaded from 
/usr/local//lib/virtuoso/hosting/creolewiki.so }
15:20:34 INFO: { Loading plugin 4: Type `plain', file `im' in 
`/usr/local//lib/virtuoso/hosting'
15:20:35 INFO:   IM version 0.6 from OpenLink Software
15:20:35 INFO:   Support functions for Image Magick 6.6.9
15:20:35 INFO:   SUCCESS plugin 4: loaded from 
/usr/local//lib/virtuoso/hosting/im.so }
15:20:35 INFO: OpenLink Virtuoso Universal Server
15:20:35 INFO: Version 06.01.3127-pthreads for Linux as of Sep 14 2012
15:20:35 INFO: uses parts of OpenSSL, PCRE, Html Tidy
15:20:41 INFO: Database version 3126
15:20:41 INFO: SQL Optimizer enabled (max 1000 layouts)
15:20:43 INFO: Compiler unit is timed at 0.001027 msec
15:21:03 DEBUG: built-in procedure repl_undot_name overruled by the RDBMS
15:21:03 DEBUG: built-in procedure REPL_FQNAME overruled by the RDBMS
15:21:03 DEBUG: built-in procedure REPL_COLTYPE_PS overruled by the RDBMS
15:21:03 DEBUG: built-in procedure REPL_COLTYPE overruled by the RDBMS
15:21:04 INFO: Roll forward started
15:21:42 INFO: 1000 transactions, 7274496 bytes replayed (13 %)
15:22:13 INFO: 2000 transactions, 14548992 bytes replayed (27 %)
15:22:42 INFO: 3000 transactions, 21856256 bytes replayed (40 %)
15:23:11 INFO: 4000 transactions, 29163520 bytes replayed (54 %)
15:23:38 INFO: 5000 transactions, 36405248 bytes replayed (68 %)
15:23:54 INFO: 6000 transactions, 42500096 bytes replayed (79 %)
15:24:00 INFO: 7000 transactions, 47677440 bytes replayed (89 %)
15:24:06 INFO: 8000 transactions, 52396032 bytes replayed (97 %)
15:24:08 INFO: 8250 transactions, 53526661 bytes replayed (100 %)
15:24:08 INFO: Roll forward complete
15:24:12 INFO: Checkpoint started
15:24:22 INFO: Checkpoint finished, log reused
15:24:22 INFO: HTTP/WebDAV server online at 8890
15:24:22 INFO: Server online at 

Re: [Virtuoso-users] Loading RDF into Virtuoso

[Thomas Michaux]

Hello,

for

HTTP/1.1 409 Conflict: can not create resource if a collection with
same name exists (when WebDav checkbox is on and the physical path is
set to the rdf_sink folder of a user)
 - Q: why is there a message that a collection already exists if there
are no files in DAV directories and there are no graphs in the RDF
store with same graph URI?


don't know if it can help you, but for another problem I had to 
investigate around the management of resources in DAV file system, you 
can lsit resources of a collection if you know it's full path name using 
API procedures :


select DB.DBA.DAV_PLAIN_SUBCOLS_P (30,'/DAV/VAD/theses/',1,'dav','dav');

for instance

30 is the corresponding ID of the collection '/DAV/VAD/theses/' you can 
use DAV_SEARCH_ID to switch from path to ID :


SQL *select  DAV_SEARCH_ID('/DAV/VAD/theses/','c');*
DAV_SEARCH_ID
LONG VARCHAR
___

*30*

1 Rows. -- 0 msec.

you can also get details on collection with :

select WS.WS.COL_PATH (COL_ID), 'C', 0, COL_MOD_TIME,
COL_ID, COL_PERMS, COL_GROUP, COL_OWNER, COL_CR_TIME, 
'dav/unix-directory', COL_NAME

*  from WS.WS.SYS_DAV_COL*
*where**
**COL_ID = 30;

*Thomas


Le 22/10/2012 11:12, Uldis Bojars a écrit :

Could you tell me what configuration options you have for this SPARQL
endpoint? My config is included in the screenshot attached.

In the list of endpoints the sparql-graph-crud/ has type DAV (for me)
which might explain DAV-related messages. Tried different settings in
the SPARQL endpoint configuration page (e.g., unchecking is a WebDav
directory). That did not fix the problem, but lead to other types of
HTTP errors:

HTTP/1.1 501 Method Not Implemented (when unchecked is a WebDav directory)

HTTP/1.1 409 Conflict: can not create resource if a collection with
same name exists (when WebDav checkbox is on and the physical path is
set to the rdf_sink folder of a user)
  - Q: why is there a message that a collection already exists if there
are no files in DAV directories and there are no graphs in the RDF
store with same graph URI?

It seems that HTTP error messages are the same regardless of if CURL
is instructed to authenticate or if the password was even entered
correctly.

With trace enabled the only information recorded in the virtuoso.log
file is internal transaction commit / restart.

Uldis

On Mon, Oct 22, 2012 at 11:28 AM, Uldis Bojars capts...@gmail.com wrote:

On Mon, Oct 22, 2012 at 3:23 AM, Hugh Williams hwilli...@openlinksw.com wrote:

That query format works for me also:

$ curl -v -T example.rdf 
http://localhost:8890/sparql-graph-crud?graph-uri=http://test.org/my -u dba

Can you please set the Virtusoso  CallstackOnException option as detailed in 
the following documentation which may provide more details as to the source of the error 
on the client:

Set both CallstackOnException and TraceOn in the .ini file.
However, no additional information is reported about this error.

[Parameters]
CallstackOnException = 1
TraceOn  = user_log, failed_log, compile, ddl_log, errors, exec

When HTTPLogFile is also set, the HTTP request itself is reported
there (in the Apache access.log format) but that's about it.

Any idea why is this error not being reported?

Since the same HTTP PUT command works for you and David perhaps there
are some configuration parameters that you have set up and I have not?
Could it be that HTTP PUT uses, e.g., DAV directory for temporary
files uploaded? If I have not set it up might it yield Invalid path
?

Uldis


On 21 Oct 2012, at 23:25, David Brooks wrote:


This all works for me

curl -v -T t.rdf
http://localhost:8890/sparql-graph-crud?graph-uri=http://test.org/my -u dba
Enter host password for user 'dba':
* About to connect() to localhost port 8890 (#0)
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8890 (#0)
* Server auth using Basic with user 'dba'

PUT /sparql-graph-crud?graph-uri=http://test.org/my HTTP/1.1
Authorization: Basic ZGJhOmRiYQ==
User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7

OpenSSL/0.9.8r zlib/1.2.3

Host: localhost:8890
Accept: */*
Content-Length: 25978
Expect: 100-continue


 HTTP/1.1 100 Continue
 HTTP/1.1 201 Created
 Server: Virtuoso/06.01.3127 (Darwin) x86_64-apple-darwin10.8.0
 Connection: Keep-Alive
 Content-Type: text/html; charset=UTF-8
 Date: Sun, 21 Oct 2012 22:17:42 GMT
 Accept-Ranges: bytes
 Content-Length: 0

* Connection #0 to host localhost left intact
* Closing connection #0

And if the graph already exists the response is  HTTP/1.1 200 OK.

You may have to dig into Virtuoso's sources to see what triggers a 409
response.



Regards,
Dave

On 22/10/12 10:54 AM, Uldis Bojars wrote:

HTTP PUT implies DELETE-ing the target graph before INSERT-ing new triples:
http://www.w3.org/TR/sparql11-http-rdf-update/#http-put

That must be why the error message mentioned deleting.

However, that problem was 

Re: [Virtuoso-users] Loading RDF into Virtuoso

[Thomas Michaux]

once you got a specific directory ID you can also list files in it using 
RES_NAME like '%' filename pattern matching :



SQL select   RES_FULL_PATH, 'R', length (RES_CONTENT), 
RES_MOD_TIME,  RES_ID, RES_PERMS, RES_GROUP, RES_OWNER, 
RES_CR_TIME, RES_TYPE, RES_NAME *from WS.WS.SYS_DAV_RES * *where 
RES_NAME like '%' and RES_COL = 8009;*
RES_FULL_PATH unnamed  length   RES_MOD_TIME RES_ID RES_PERMS
RES_GROUP   RES_OWNER   RES_CR_TIME RES_TYPE RES_NAME
VARCHAR VARCHAR  INTEGER  TIMESTAMPINTEGER NOT NULL 
VARCHAR  INTEGER INTEGER TIMESTAMP VARCHAR VARCHAR

___

/DAV/VAD/theses/2012TOUL2002/document R8434 2012.10.27 
15:23.47 0  138305 110110110NM  0   0   2012.10.27 
15:23.47 0 application/rdf+xml document


1 Rows. -- 1 msec.


and if you want the content of files, using conductor web interface :

select*RES_CONTENT from WS.WS.SYS_DAV_RES  where  RES_NAME like '%' and 
RES_COL = 8009*;



Thomas

Le 07/11/2012 12:19, Thomas Michaux a écrit :

Hello,

for
HTTP/1.1 409 Conflict: can not create resource if a collection with
same name exists (when WebDav checkbox is on and the physical path is
set to the rdf_sink folder of a user)
  - Q: why is there a message that a collection already exists if there
are no files in DAV directories and there are no graphs in the RDF
store with same graph URI?

don't know if it can help you, but for another problem I had to 
investigate around the management of resources in DAV file system, 
you can lsit resources of a collection if you know it's full path name 
using API procedures :


select DB.DBA.DAV_PLAIN_SUBCOLS_P (30,'/DAV/VAD/theses/',1,'dav','dav');

for instance

30 is the corresponding ID of the collection '/DAV/VAD/theses/' you 
can use DAV_SEARCH_ID to switch from path to ID :


SQL *select  DAV_SEARCH_ID('/DAV/VAD/theses/','c');*
DAV_SEARCH_ID
LONG VARCHAR
___

*30*

1 Rows. -- 0 msec.

you can also get details on collection with :

select WS.WS.COL_PATH (COL_ID), 'C', 0, COL_MOD_TIME,
COL_ID, COL_PERMS, COL_GROUP, COL_OWNER, COL_CR_TIME, 
'dav/unix-directory', COL_NAME

*  from WS.WS.SYS_DAV_COL*
*where**
**COL_ID = 30;

*Thomas


Le 22/10/2012 11:12, Uldis Bojars a écrit :

Could you tell me what configuration options you have for this SPARQL
endpoint? My config is included in the screenshot attached.

In the list of endpoints the sparql-graph-crud/ has type DAV (for me)
which might explain DAV-related messages. Tried different settings in
the SPARQL endpoint configuration page (e.g., unchecking is a WebDav
directory). That did not fix the problem, but lead to other types of
HTTP errors:

HTTP/1.1 501 Method Not Implemented (when unchecked is a WebDav directory)

HTTP/1.1 409 Conflict: can not create resource if a collection with
same name exists (when WebDav checkbox is on and the physical path is
set to the rdf_sink folder of a user)
  - Q: why is there a message that a collection already exists if there
are no files in DAV directories and there are no graphs in the RDF
store with same graph URI?

It seems that HTTP error messages are the same regardless of if CURL
is instructed to authenticate or if the password was even entered
correctly.

With trace enabled the only information recorded in the virtuoso.log
file is internal transaction commit / restart.

Uldis

On Mon, Oct 22, 2012 at 11:28 AM, Uldis Bojarscapts...@gmail.com  wrote:

On Mon, Oct 22, 2012 at 3:23 AM, Hugh Williamshwilli...@openlinksw.com  wrote:

That query format works for me also:

$ curl -v -T 
example.rdfhttp://localhost:8890/sparql-graph-crud?graph-uri=http://test.org/my 
 -u dba

Can you please set the Virtusoso  CallstackOnException option as detailed in 
the following documentation which may provide more details as to the source of the error 
on the client:

Set both CallstackOnException and TraceOn in the .ini file.
However, no additional information is reported about this error.

[Parameters]
CallstackOnException = 1
TraceOn  = user_log, failed_log, compile, ddl_log, errors, exec

When HTTPLogFile is also set, the HTTP request itself is reported
there (in the Apache access.log format) but that's about it.

Any idea why is this error not being reported?

Since the same HTTP PUT command works for you and David perhaps there
are some configuration parameters that you have set up and I have not?
Could it be that HTTP PUT uses, e.g., DAV directory for temporary
files uploaded? If I have not set it up might it yield Invalid path
?

Uldis


On 21 Oct 2012, at 23:25, David Brooks wrote:


This all works for me

curl -v -T t.rdf
http://localhost:8890/sparql-graph-crud?graph-uri=http://test.org/my  -u dba
Enter host password for user 'dba':
* About to connect() to localhost port 8890 (#0)
*   Trying