Re: [Virtuoso-users] backup-dump impossible after loading an "invalid" IRI
hello Hugh,
some news : we can now reproduce the issue with a significant amount of
triples !
I uploaded a sample file to reproduce the "Broken index
DB_DBA_RDF_IRI_UNQC_RI_ID and Broken index RDF_IRI" on +backup-dump, here
https://github.com/openlink/virtuoso-opensource/files/1625829/gsl2017100301.zip
It seems to much "IRI is not in Unicode Normal Form" triples will
silently break the RDF_IRI and DB_DBA_RDF_IRI_UNQC_RI_ID indices.
Thanks
Le 22/12/2017 à 18:21, Hugh Williams a écrit :
Hi Thomas,
This works of me with Virtuoso open source and commercial builds I
have on Mac OS X. What is the version of the Virtuoso Windows open
source binary you are using ( output of "virtuoso-t -?” command ) ?
Below is the output from my Open source test:
De-iMac-232:database hwilliams$ ~/Downloads/isql
Connected to OpenLink Virtuoso
Driver: 07.20.3214 OpenLink Virtuoso ODBC Driver
OpenLink Interactive SQL (Virtuoso), version 0.9849b.
Type HELP; for help and EXIT; to exit.
SQL> ld_dir('.',
'badiritest.nt','http://hub.abes.fr/journals/wiley2011/20170921');
Done. -- 5 msec.
SQL> select * from load_list;
ll_file ll_graph
ll_state ll_started ll_done ll_host
ll_work_time ll_error
VARCHAR NOT NULL VARCHAR
INTEGER TIMESTAMP TIMESTAMP INTEGER
INTEGER VARCHAR
___
./badiritest.nt http://hub.abes.fr/journals/wiley2011/20170921
0 NULL NULL
NULL NULL NULL
1 Rows. -- 1 msec.
SQL> rdf_loader_run();
Done. -- 8 msec.
SQL> select * from load_list; ll_file
ll_graph ll_state ll_started ll_done
ll_host ll_work_time ll_error
VARCHAR NOT NULL VARCHAR
INTEGER TIMESTAMP TIMESTAMP INTEGER
INTEGER VARCHAR
___
./badiritest.nt http://hub.abes.fr/journals/wiley2011/20170921
2 2017.12.22 17:16.5 309717000
2017.12.22 17:16.5 313513000 0 NULL NULL
1 Rows. -- 1 msec.
SQL> sparql select * from
where {?s ?p ?o};
s p o
LONG VARCHAR LONG VARCHAR
LONG VARCHAR
___
http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana c g a
http://www.w3.org/1999/02/22-rdf-syntax-ns#type
http://xmlns.com/foaf/0.1/Person
1 Rows. -- 1 msec.
SQL> checkpoint;
Done. -- 44 msec.
SQL> shutdown;
De-iMac-232:database hwilliams$
~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t +backup-dump
Fri Dec 22 2017
17:16:44 { Loading plugin 1: Type `plain', file `wikiv' in
`/Users/hwilliams/vos-7-install/hosting'
17:16:44 FAILED plugin 1: Unable to locate file }
17:16:44 { Loading plugin 2: Type `plain', file `mediawiki' in
`/Users/hwilliams/vos-7-install/hosting'
17:16:44 FAILED plugin 2: Unable to locate file }
17:16:44 { Loading plugin 3: Type `plain', file `creolewiki' in
`/Users/hwilliams/vos-7-install/hosting'
17:16:44 FAILED plugin 3: Unable to locate file }
17:16:44 OpenLink Virtuoso Universal Server
17:16:44 Version 07.20.3217-pthreads for Darwin as of Oct 31 2017
17:16:44 uses parts of OpenSSL, PCRE, Html Tidy
17:16:44 Database version 3126
17:16:44 SQL Optimizer enabled (max 1000 layouts)
17:16:45 Compiler unit is timed at 0.000284 msec
17:16:46 Roll forward started
17:16:46 Roll forward complete
17:16:47 PL LOG: Can't get list of vad packages in
/Users/hwilliams/vos-7-install/vad/
17:16:47 Database dump started
17:16:47 Checkpoint finished, log reused
17:16:47 Dumping the schema tables
17:16:47 Dumping the registry
17:16:47 Dumping the schema done
17:16:47 Database dump complete
17:16:47 Server exiting
De-iMac-232:database hwilliams$ rm virtuoso.db
De-iMac-232:database hwilliams$
~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t -R -f
Fri Dec 22 2017
17:17:05 { Loading plugin 1: Type `plain', file `wikiv' in
`/Users/hwilliams/vos-7-install/hosting'
17:17:05 FAILED plugin 1: Unable to locate file }
17:17:05 { Loading plugin 2: Type `plain', file `mediawiki' in
`/Users/hwilliams/vos-7-install/hosting'
17:17:05 FAILED plugin 2: Unable to locate file }
17:17:05 { Loading plugin 3: Type `plain', file `creolewiki' in
`/Users/hwilliams/vos-7-install/hosting'
17:17:05 FAILED plugin 3: Unable to locate file }
17:17:05 OpenLink Virtuoso Universal Server
17:17:05 Version 07.20.3217-pthreads for Darwin as of Oct 31 2017
17:17:05
Re: [Virtuoso-users] backup-dump impossible after loading an "invalid" IRI
Hello Hugh, and happy new year !
Sorry for the inconvenience, get it working correctly this morning !
i don't understand the point i missed last time :(
11:19:08 OpenLink Virtuoso Universal Server
11:19:08 Version 07.20.3217-threads for Win64 as of Apr 25 2016
11:19:08 uses parts of OpenSSL, PCRE, Html Tidy
11:19:08 Database version 3126
11:19:08 SQL Optimizer enabled (max 1000 layouts)
11:19:09 Compiler unit is timed at 0.000127 msec
11:19:09 Roll forward started
11:19:09 Roll forward complete
11:19:09 Database dump started
11:19:09 Checkpoint finished, log reused
11:19:09 Dumping the schema tables
11:19:09 Dumping the registry
11:19:09 Dumping the schema done
11:19:09 Database dump complete
under Windows 10 PRO
Le 22/12/2017 à 18:21, Hugh Williams a écrit :
Hi Thomas,
This works of me with Virtuoso open source and commercial builds I
have on Mac OS X. What is the version of the Virtuoso Windows open
source binary you are using ( output of "virtuoso-t -?” command ) ?
Below is the output from my Open source test:
De-iMac-232:database hwilliams$ ~/Downloads/isql
Connected to OpenLink Virtuoso
Driver: 07.20.3214 OpenLink Virtuoso ODBC Driver
OpenLink Interactive SQL (Virtuoso), version 0.9849b.
Type HELP; for help and EXIT; to exit.
SQL> ld_dir('.',
'badiritest.nt','http://hub.abes.fr/journals/wiley2011/20170921');
Done. -- 5 msec.
SQL> select * from load_list;
ll_file ll_graph
ll_state ll_started ll_done ll_host
ll_work_time ll_error
VARCHAR NOT NULL VARCHAR
INTEGER TIMESTAMP TIMESTAMP INTEGER
INTEGER VARCHAR
___
./badiritest.nt http://hub.abes.fr/journals/wiley2011/20170921
0 NULL NULL
NULL NULL NULL
1 Rows. -- 1 msec.
SQL> rdf_loader_run();
Done. -- 8 msec.
SQL> select * from load_list; ll_file
ll_graph ll_state ll_started ll_done
ll_host ll_work_time ll_error
VARCHAR NOT NULL VARCHAR
INTEGER TIMESTAMP TIMESTAMP INTEGER
INTEGER VARCHAR
___
./badiritest.nt http://hub.abes.fr/journals/wiley2011/20170921
2 2017.12.22 17:16.5 309717000
2017.12.22 17:16.5 313513000 0 NULL NULL
1 Rows. -- 1 msec.
SQL> sparql select * from
where {?s ?p ?o};
s p o
LONG VARCHAR LONG VARCHAR
LONG VARCHAR
___
http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana c g a
http://www.w3.org/1999/02/22-rdf-syntax-ns#type
http://xmlns.com/foaf/0.1/Person
1 Rows. -- 1 msec.
SQL> checkpoint;
Done. -- 44 msec.
SQL> shutdown;
De-iMac-232:database hwilliams$
~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t +backup-dump
Fri Dec 22 2017
17:16:44 { Loading plugin 1: Type `plain', file `wikiv' in
`/Users/hwilliams/vos-7-install/hosting'
17:16:44 FAILED plugin 1: Unable to locate file }
17:16:44 { Loading plugin 2: Type `plain', file `mediawiki' in
`/Users/hwilliams/vos-7-install/hosting'
17:16:44 FAILED plugin 2: Unable to locate file }
17:16:44 { Loading plugin 3: Type `plain', file `creolewiki' in
`/Users/hwilliams/vos-7-install/hosting'
17:16:44 FAILED plugin 3: Unable to locate file }
17:16:44 OpenLink Virtuoso Universal Server
17:16:44 Version 07.20.3217-pthreads for Darwin as of Oct 31 2017
17:16:44 uses parts of OpenSSL, PCRE, Html Tidy
17:16:44 Database version 3126
17:16:44 SQL Optimizer enabled (max 1000 layouts)
17:16:45 Compiler unit is timed at 0.000284 msec
17:16:46 Roll forward started
17:16:46 Roll forward complete
17:16:47 PL LOG: Can't get list of vad packages in
/Users/hwilliams/vos-7-install/vad/
17:16:47 Database dump started
17:16:47 Checkpoint finished, log reused
17:16:47 Dumping the schema tables
17:16:47 Dumping the registry
17:16:47 Dumping the schema done
17:16:47 Database dump complete
17:16:47 Server exiting
De-iMac-232:database hwilliams$ rm virtuoso.db
De-iMac-232:database hwilliams$
~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t -R -f
Fri Dec 22 2017
17:17:05 { Loading plugin 1: Type `plain', file `wikiv' in
`/Users/hwilliams/vos-7-install/hosting'
17:17:05 FAILED plugin 1: Unable to locate file }
17:17:05 { Loading plugin 2: Type `plain', file `mediawiki' in
`/Users/hwilliams/vos-7-install/hosting'
17:17:05 FAILED plugin 2: Unable to locate file }
17:17:05 {
Re: [Virtuoso-users] backup-dump impossible after loading an "invalid" IRI
Hi Thomas,
This works of me with Virtuoso open source and commercial builds I have on Mac
OS X. What is the version of the Virtuoso Windows open source binary you are
using ( output of "virtuoso-t -?” command ) ?
Below is the output from my Open source test:
De-iMac-232:database hwilliams$ ~/Downloads/isql
Connected to OpenLink Virtuoso
Driver: 07.20.3214 OpenLink Virtuoso ODBC Driver
OpenLink Interactive SQL (Virtuoso), version 0.9849b.
Type HELP; for help and EXIT; to exit.
SQL> ld_dir('.',
'badiritest.nt','http://hub.abes.fr/journals/wiley2011/20170921');
Done. -- 5 msec.
SQL> select * from load_list;
ll_file
ll_graph
ll_statell_started ll_done ll_host
ll_work_time ll_error
VARCHAR NOT NULL
VARCHAR
INTEGER TIMESTAMPTIMESTAMPINTEGER INTEGER
VARCHAR
___
./badiritest.nt
http://hub.abes.fr/journals/wiley2011/20170921
0 NULL NULL NULLNULL
NULL
1 Rows. -- 1 msec.
SQL> rdf_loader_run();
Done. -- 8 msec.
SQL> select * from load_list;
ll_file
ll_graph
ll_statell_started ll_done ll_host
ll_work_time ll_error
VARCHAR NOT NULL
VARCHAR
INTEGER TIMESTAMPTIMESTAMPINTEGER INTEGER
VARCHAR
___
./badiritest.nt
http://hub.abes.fr/journals/wiley2011/20170921
2 2017.12.22 17:16.5 309717000 2017.12.22 17:16.5 313513000 0
NULLNULL
1 Rows. -- 1 msec.
SQL> sparql select * from
where {?s ?p ?o};
s
p
o
LONG VARCHAR
LONG VARCHAR
LONG VARCHAR
___
http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana
c g a http://www.w3.org/1999/02/22-rdf-syntax-ns#type
http://xmlns.com/foaf/0.1/Person
1 Rows. -- 1 msec.
SQL> checkpoint;
Done. -- 44 msec.
SQL> shutdown;
De-iMac-232:database hwilliams$
~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t +backup-dump
Fri Dec 22 2017
17:16:44 { Loading plugin 1: Type `plain', file `wikiv' in
`/Users/hwilliams/vos-7-install/hosting'
17:16:44 FAILED plugin 1: Unable to locate file }
17:16:44 { Loading plugin 2: Type `plain', file `mediawiki' in
`/Users/hwilliams/vos-7-install/hosting'
17:16:44 FAILED plugin 2: Unable to locate file }
17:16:44 { Loading plugin 3: Type `plain', file `creolewiki' in
`/Users/hwilliams/vos-7-install/hosting'
17:16:44 FAILED plugin 3: Unable to locate file }
17:16:44 OpenLink Virtuoso Universal Server
17:16:44 Version 07.20.3217-pthreads for Darwin as of Oct 31 2017
17:16:44 uses parts of OpenSSL, PCRE, Html Tidy
17:16:44 Database version 3126
17:16:44 SQL Optimizer enabled (max 1000 layouts)
17:16:45 Compiler unit is timed at 0.000284 msec
17:16:46 Roll forward started
17:16:46 Roll forward complete
17:16:47 PL LOG: Can't get list of vad packages in
/Users/hwilliams/vos-7-install/vad/
17:16:47 Database dump started
17:16:47 Checkpoint finished, log reused
17:16:47 Dumping the schema tables
17:16:47 Dumping the registry
17:16:47 Dumping the schema done
17:16:47 Database dump complete
17:16:47 Server exiting
De-iMac-232:database hwilliams$ rm virtuoso.db
De-iMac-232:database hwilliams$
~/src/git/vos-7-develop/binsrc/virtuoso/virtuoso-t -R -f
Fri Dec 22 2017
17:17:05 { Loading plugin 1: Type `plain', file `wikiv' in
`/Users/hwilliams/vos-7-install/hosting'
17:17:05 FAILED plugin 1: Unable to locate file }
17:17:05 { Loading plugin 2: Type `plain', file `mediawiki' in
`/Users/hwilliams/vos-7-install/hosting'
[Virtuoso-users] backup-dump impossible after loading an "invalid" IRI
Hello,
we may have found a case where bulk load doesn't report error but
invalid data has a side effect on the "backup-dump" process, step to
reproduce :
1) - in an empty database : use
ld_dir('e:/virtuosoBACKUP/test',
'badiritest.nt','http://hub.abes.fr/journals/wiley2011/20170921');
with these data in file badiritest.nt
.
rdf_loader_run();
checkpoint;
shutdown;
2) then you can't success to process the "backup-dump" procedure :
virtuoso-t +foreground +configfile "c:\Program Files\OpenLink
Software\VOS7\virtuoso-opensource\database\virtuoso.ini" +backup-dump
even if data is really in store :
SQL> SPARQL select ?s ?p ?o from
{?s ?p ?o};
s p o
LONG VARCHAR LONG VARCHAR LONG VARCHAR
___
http://hub.abes.fr/wiley/periodical/ejoc/2004/volume_2004/issue_21/101002/ejoc200400407/pintodiana c g a
http://www.w3.org/1999/02/22-rdf-syntax-ns#type
http://xmlns.com/foaf/0.1/Person
1 Rows. -- 0 msec.
tested on windows , need to reproduce under linux.
Thanks
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
