Re: [Virtuoso-users] infrequent errors on parallel querying

[Hugh Williams]

Hi Andreas,

On the 8891 & 8895 server instances can you connect with the isql PL Debugger ( 
isql  dba  -D ) and provide the output of running the 
command “info threads” , which will show the current running threads and 
procedures associated with them as detailed at:

http://docs.openlinksw.com/virtuoso/pldebugger.html#pldebugger

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



> On 5 Mar 2016, at 21:31, Hugh Williams  wrote:
> 
> Hi Andrea,
> 
> I haven’t been able reproduce this issue as steps have not been provided to 
> recreate, just  information to try and analyse the behaviour being 
> encountered. If you can provide a copy of your data /databases and HTTP 
> recordings of they queries being run,  then that would be easiest to 
> reproduce the issue ? Failing that if the data is proprietary/secure and 
> cannot be provided then there is an option to gather database statistics 
> which can be imported locally  and the analysed by development  to facilitate 
> reproducing the effect, as detailed at:
>   
>   
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtQueryOptDiagnostic
> 
> With regards to the latest sys_l_stat queries detailing locks/waits on the 
> database in both sets of queries all the locks/waits are occurring on the  
> RO_VAL index of the DB.DBA.RDF_OBJ table:
> 
> KEY_TABLE 
> INDEX_NAME
> LOCKSWAITSWAIT_PCT  DEADLOCKS  LOCK_ESC  WAIT_MSECS
> VARCHAR NOT NULL  
> VARCHAR   
> INTEGER  INTEGER  INTEGER  INTEGER  INTEGER  INTEGER
> ___
> 
> DB.DBA.RDF_OBJ
> RO_VAL
> 3431249  1577 01141 95   501193384
> 
> which is used for storing long “O” (Object) values in the RDF_QUAD table or 
> if they have a Free-Text index,see:
> 
>   
> http://docs.openlinksw.com/virtuoso/rdfdatarepresentation.html#rdfquadtables 
> 
> Thus do you have many long O ie object/literal values in your datasets or 
> have Free-text indexing enabled ? Although I don’t see any use of 
> bif:contains which would use the FT index in the sample queries you have 
> provided ?
> 
> Will have to check with development as to possible cause or prevention of 
> these locks on the RO_VAL index …
> 
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc.  //  http://www.openlinksw.com/
> Weblog   -- http://www.openlinksw.com/blogs/
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> Twitter  -- http://twitter.com/OpenLink
> Google+  -- http://plus.google.com/100570109519069333827/
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> Universal Data Access, Integration, and Management Technology Providers
> 
> 
> 
>> On 5 Mar 2016, at 10:34, Nolle, Andreas  wrote:
>> 
>> Dear Hugh,
>> 
>> I've again restarted my application and created the top 50 information as 
>> well as the corresponding log files at two different times. Please find 
>> these files at 
>> https://www.dropbox.com/s/6xd1yl6jy8x9pv6/virtuoso_logs.zip?dl=0
>> 
>> Please notice that the errors received at the client (my application) vary 
>> between:
>> - Virtuoso 22026 Error SR477: Error serializing the value into an ANY column
>> - Virtuoso RDFZZ Error DB.DBA.SPARQL_REXEC('http://141.87.4.8:/sparql', 
>> ...) returned unsupported Content-Type '(NULL)'
>> - Virtuoso 40001 Error SR172: Transaction deadlocked (most of the errors!)
>> and occur still infrequently. So if a query ended with an error but is 
>> evaluated some seconds or minutes later, it may happened that the evaluation 
>> even works fine.
>> 
>> Were you already able to reproduce the described issue regarding the long 
>> evaluation times and/or the mentioned errors?
>> 
>> Best regards
>> Andy
>> 
>> 
>> -Ursprüngliche Nachricht-
>> Von: Nolle, Andreas 
>> Gesendet: Freitag, 4. März 2016 09:10
>> An: 'Hugh Williams' 
>> Cc: Kingsley Idehen ; 
>> virtuoso-users@lists.sourceforge.net
>> Betreff: AW: [Virtuoso-users] infrequent errors on parallel querying
>> 
>> Dear Hugh,
>> 
>> now 

Re: [Virtuoso-users] infrequent errors on parallel querying

[Hugh Williams]

Hi Andrea,

I haven’t been able reproduce this issue as steps have not been provided to 
recreate, just  information to try and analyse the behaviour being encountered. 
If you can provide a copy of your data /databases and HTTP recordings of they 
queries being run,  then that would be easiest to reproduce the issue ? Failing 
that if the data is proprietary/secure and cannot be provided then there is an 
option to gather database statistics which can be imported locally  and the 
analysed by development  to facilitate reproducing the effect, as detailed at:


http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtQueryOptDiagnostic

With regards to the latest sys_l_stat queries detailing locks/waits on the 
database in both sets of queries all the locks/waits are occurring on the  
RO_VAL index of the DB.DBA.RDF_OBJ table:

KEY_TABLE   
  INDEX_NAME
LOCKSWAITSWAIT_PCT  DEADLOCKS  LOCK_ESC  WAIT_MSECS
VARCHAR NOT NULL
  VARCHAR   
INTEGER  INTEGER  INTEGER  INTEGER  INTEGER  INTEGER
___

DB.DBA.RDF_OBJ  
  RO_VAL
3431249  1577 01141 95   501193384

 which is used for storing long “O” (Object) values in the RDF_QUAD table or if 
they have a Free-Text index,see:


http://docs.openlinksw.com/virtuoso/rdfdatarepresentation.html#rdfquadtables 

Thus do you have many long O ie object/literal values in your datasets or have 
Free-text indexing enabled ? Although I don’t see any use of bif:contains which 
would use the FT index in the sample queries you have provided ?

Will have to check with development as to possible cause or prevention of these 
locks on the RO_VAL index …

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



> On 5 Mar 2016, at 10:34, Nolle, Andreas <no...@hs-albsig.de> wrote:
> 
> Dear Hugh,
> 
> I've again restarted my application and created the top 50 information as 
> well as the corresponding log files at two different times. Please find these 
> files at https://www.dropbox.com/s/6xd1yl6jy8x9pv6/virtuoso_logs.zip?dl=0
> 
> Please notice that the errors received at the client (my application) vary 
> between:
> - Virtuoso 22026 Error SR477: Error serializing the value into an ANY column
> - Virtuoso RDFZZ Error DB.DBA.SPARQL_REXEC('http://141.87.4.8:/sparql', 
> ...) returned unsupported Content-Type '(NULL)'
> - Virtuoso 40001 Error SR172: Transaction deadlocked (most of the errors!)
> and occur still infrequently. So if a query ended with an error but is 
> evaluated some seconds or minutes later, it may happened that the evaluation 
> even works fine.
> 
> Were you already able to reproduce the described issue regarding the long 
> evaluation times and/or the mentioned errors?
> 
> Best regards
> Andy
> 
> 
> -Ursprüngliche Nachricht-
> Von: Nolle, Andreas 
> Gesendet: Freitag, 4. März 2016 09:10
> An: 'Hugh Williams' <hwilli...@openlinksw.com>
> Cc: Kingsley Idehen <kide...@openlinksw.com>; 
> virtuoso-users@lists.sourceforge.net
> Betreff: AW: [Virtuoso-users] infrequent errors on parallel querying
> 
> Dear Hugh,
> 
> now I've run a database integrity check and perform a +crash-dump and restore 
> on Virtuoso instance running at port 8895.
> After that I've restarted my application and executed the top50 select after 
> some transaction deadlock errors occurred.
> 
> Please find the top50 information as well as the corresponding log files at 
> https://www.dropbox.com/s/qflp4xtq34o1ioh/locks.zip?dl=0
> 
> The strange thing is that errors regarding "No ext map for ... in uncommitted 
> blob cpt" are still be there...
> 
> Please notice that 24 queries are evaluated in parallel via the /sparql end 
> point, i.e. http
> 
> Best Regards
> Andy
> 
> -Ursprüngliche Nachricht-
> Von: Hugh Williams [mailto:hwilli...@openlinksw.com]
> Gesendet: Donnerstag, 3. März 2016 02:09
> An: Nolle, Andreas <no...@hs-albsig.de>
> Cc: Kings

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Dear Hugh,

I've again restarted my application and created the top 50 information as well 
as the corresponding log files at two different times. Please find these files 
at https://www.dropbox.com/s/6xd1yl6jy8x9pv6/virtuoso_logs.zip?dl=0

Please notice that the errors received at the client (my application) vary 
between:
- Virtuoso 22026 Error SR477: Error serializing the value into an ANY column
- Virtuoso RDFZZ Error DB.DBA.SPARQL_REXEC('http://141.87.4.8:/sparql', 
...) returned unsupported Content-Type '(NULL)'
- Virtuoso 40001 Error SR172: Transaction deadlocked (most of the errors!)
and occur still infrequently. So if a query ended with an error but is 
evaluated some seconds or minutes later, it may happened that the evaluation 
even works fine.

Were you already able to reproduce the described issue regarding the long 
evaluation times and/or the mentioned errors?

Best regards
Andy


-Ursprüngliche Nachricht-
Von: Nolle, Andreas 
Gesendet: Freitag, 4. März 2016 09:10
An: 'Hugh Williams' <hwilli...@openlinksw.com>
Cc: Kingsley Idehen <kide...@openlinksw.com>; 
virtuoso-users@lists.sourceforge.net
Betreff: AW: [Virtuoso-users] infrequent errors on parallel querying

Dear Hugh,

now I've run a database integrity check and perform a +crash-dump and restore 
on Virtuoso instance running at port 8895.
After that I've restarted my application and executed the top50 select after 
some transaction deadlock errors occurred.

Please find the top50 information as well as the corresponding log files at 
https://www.dropbox.com/s/qflp4xtq34o1ioh/locks.zip?dl=0

The strange thing is that errors regarding "No ext map for ... in uncommitted 
blob cpt" are still be there...

Please notice that 24 queries are evaluated in parallel via the /sparql end 
point, i.e. http

Best Regards
Andy

-Ursprüngliche Nachricht-
Von: Hugh Williams [mailto:hwilli...@openlinksw.com]
Gesendet: Donnerstag, 3. März 2016 02:09
An: Nolle, Andreas <no...@hs-albsig.de>
Cc: Kingsley Idehen <kide...@openlinksw.com>; 
virtuoso-users@lists.sourceforge.net
Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying

Hi Andreas,

The select top 10 * from sys_l_stat order by waits desc; queries from the 8891 
& 8895 instances do not show an excessive number of waits, only about 1990+ 
waits on the SYS_DAV_QUEUE index. When you ran those queries was the system 
under  typical work load ie  was the long running query being run at the time ? 
Note you should also run the query a number of time to get a sample of the 
waits as indicated at:

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#perfdiaglocking

I certainly would have expected a larger number of waits on the 8891 instance 
where the "WARNING: * Monitor: Many lock waits”  errors where occurring 
extensively in the log …

You indicate the "transaction deadlock and No ext map for dp 3193852 in 
uncommitted blob cpt” on the 8895 instance only occur when running several 
queries in parallel, so this would be the time to be running the query to check 
for locks/waits. How many of these queries are typically being run in parallel 
? Also are these queries being executed via the /sparql end point ie HTTP  or 
SQL interface ?

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



> On 2 Mar 2016, at 17:31, Nolle, Andreas <no...@hs-albsig.de> wrote:
> 
> Dear Hugh,
> 
> please find the requested information about top50 
> athttps://www.dropbox.com/s/ylrb20ylze38cj6/top50.zip?dl=0
> 
> Please notice that both errors (transaction deadlock and No ext map for dp 
> 3193852 in uncommitted blob cpt) only occur if I run several queries (of the 
> type already mentioned) in parallel.
> Since my assumption is that especially the transaction deadlock error 
> is probably related to the long evaluation time, I haven’t restarted my 
> application so far… However, I will run a database integrity check and 
> perform a +crash-dump and restore to eliminate the “uncommitted blob” errors 
> in the next run of my application.
> 
> Best regards
> Andy

--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Kingsley Idehen]

On 3/4/16 3:12 AM, Nolle, Andreas wrote:
>
> Dear Kingsley,
>
>  
>
> were you able to reproduce the mentioned behavior regarding the long
> evaluation time?
>
> Did you find any reason for that?
>
>  
>
> Best regards
>
> Andy
>
>  
>
>  

It is being looked into by the development and support teams, as
indicated by responses from Hugh.

-- 
Regards,

Kingsley Idehen   
Founder & CEO 
OpenLink Software 
Company Web: http://www.openlinksw.com
Personal Weblog 1: http://kidehen.blogspot.com
Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen
Twitter Profile: https://twitter.com/kidehen
Google+ Profile: https://plus.google.com/+KingsleyIdehen/about
LinkedIn Profile: http://www.linkedin.com/in/kidehen
Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this



smime.p7s
Description: S/MIME Cryptographic Signature
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Dear Kingsley,

were you able to reproduce the mentioned behavior regarding the long evaluation 
time?
Did you find any reason for that?

Best regards
Andy



--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Dear Hugh,

now I've run a database integrity check and perform a +crash-dump and restore 
on Virtuoso instance running at port 8895.
After that I've restarted my application and executed the top50 select after 
some transaction deadlock errors occurred.

Please find the top50 information as well as the corresponding log files at 
https://www.dropbox.com/s/qflp4xtq34o1ioh/locks.zip?dl=0

The strange thing is that errors regarding "No ext map for ... in uncommitted 
blob cpt" are still be there...

Please notice that 24 queries are evaluated in parallel via the /sparql end 
point, i.e. http

Best Regards
Andy

-Ursprüngliche Nachricht-
Von: Hugh Williams [mailto:hwilli...@openlinksw.com] 
Gesendet: Donnerstag, 3. März 2016 02:09
An: Nolle, Andreas <no...@hs-albsig.de>
Cc: Kingsley Idehen <kide...@openlinksw.com>; 
virtuoso-users@lists.sourceforge.net
Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying

Hi Andreas,

The select top 10 * from sys_l_stat order by waits desc; queries from the 8891 
& 8895 instances do not show an excessive number of waits, only about 1990+ 
waits on the SYS_DAV_QUEUE index. When you ran those queries was the system 
under  typical work load ie  was the long running query being run at the time ? 
Note you should also run the query a number of time to get a sample of the 
waits as indicated at:

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#perfdiaglocking

I certainly would have expected a larger number of waits on the 8891 instance 
where the "WARNING: * Monitor: Many lock waits”  errors where occurring 
extensively in the log …

You indicate the "transaction deadlock and No ext map for dp 3193852 in 
uncommitted blob cpt” on the 8895 instance only occur when running several 
queries in parallel, so this would be the time to be running the query to check 
for locks/waits. How many of these queries are typically being run in parallel 
? Also are these queries being executed via the /sparql end point ie HTTP  or 
SQL interface ?

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



> On 2 Mar 2016, at 17:31, Nolle, Andreas <no...@hs-albsig.de> wrote:
> 
> Dear Hugh,
> 
> please find the requested information about top50 
> athttps://www.dropbox.com/s/ylrb20ylze38cj6/top50.zip?dl=0
> 
> Please notice that both errors (transaction deadlock and No ext map for dp 
> 3193852 in uncommitted blob cpt) only occur if I run several queries (of the 
> type already mentioned) in parallel.
> Since my assumption is that especially the transaction deadlock error is 
> probably related to the long evaluation time, I haven’t restarted my 
> application so far…
> However, I will run a database integrity check and perform a +crash-dump and 
> restore to eliminate the “uncommitted blob” errors in the next run of my 
> application.
> 
> Best regards
> Andy

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Dear Hugh,

please find the requested information about top50 at 
https://www.dropbox.com/s/ylrb20ylze38cj6/top50.zip?dl=0

Please notice that both errors (transaction deadlock and No ext map for dp 
3193852 in uncommitted blob cpt) only occur if I run several queries (of the 
type already mentioned) in parallel.
Since my assumption is that especially the transaction deadlock error is 
probably related to the long evaluation time, I haven’t restarted my 
application so far…
However, I will run a database integrity check and perform a +crash-dump and 
restore to eliminate the “uncommitted blob” errors in the next run of my 
application.

Best regards
Andy



Von: Hugh Williams [mailto:hwilli...@openlinksw.com]
Gesendet: Mittwoch, 2. März 2016 15:55
An: Nolle, Andreas <no...@hs-albsig.de>
Cc: Kingsley Idehen <kide...@openlinksw.com>; 
virtuoso-users@lists.sourceforge.net
Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying

Hi Andreas,

Analysing  your logs further I see:

1. On the remote (8891) instance the SERVICE keyword references I  see many 
occurrences of the following in there log:

07:11:34 WARNING: * Monitor: Many lock waits
07:13:34 WARNING: * Monitor: Many lock waits
07:15:43 WARNING: * Monitor: Many lock waits
07:17:45 WARNING: * Monitor: Many lock waits
07:19:34 WARNING: * Monitor: Should read for update because lock escalation 
from shared to exclusive fails frequently (1)
07:19:34 WARNING: * Monitor: Locks are held for a long time
07:19:46 WARNING: * Monitor: Many lock waits

and status shows 12 deadlocks and thousands of occurrences of the following 
messaged in the output:

Lock Status: 12 deadlocks of which 0 2r1w, 107 waits,
   Currently 10 threads running 7 threads waiting 0 threads in vdb.
Pending:
  82435: IER 141.87.4.9
  212: IER 141.87.4.9
  208: IER 141.87.4.9
  213: IER 141.87.4.9
  138756: IER 141.87.4.9
  276: IER 141.87.4.9
  137220: IER 141.87.4.9
.
.
.

The deadlocks are not a problem per say , it is more the many pending waits. 
Thus as indicated in the performance tuning link 
(http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#ptune) sent previously 
please provide the output of running the query:

select top 50 *  sys_l_stat order by wait_msecs desc

To see in which table all the waits are pending on …

2. On the local instance (8895) the queries are being run from I see many 
occurrences of these errors in the logs:

01:13:01 ERROR: No ext map for dp 3193852 in uncommitted blob cpt
01:13:01 ERROR: No ext map for dp 3193853 in uncommitted blob cpt
01:13:01 ERROR: No ext map for dp 3193854 in uncommitted blob cpt
01:13:01 ERROR: No ext map for dp 3193855 in uncommitted blob cpt
01:13:04 INFO: Checkpoint finished, log reused
01:45:34 WARNING: * Monitor: Locks are held for a long time
01:45:35 INFO: Free blob page refd start = 2319104 L=2319104
01:45:35 ERROR: Blob starting L=2319104 inconsistent before delete.  Not deleted
01:45:35 INFO: Free blob page refd start = 2319105 L=2319105
01:45:35 ERROR: Blob starting L=2319105 inconsistent before delete.  Not deleted
01:45:35 INFO: Free blob page refd start = 2319106 L=2319106
01:45:35 ERROR: Blob starting L=2319106 inconsistent before delete.  Not deleted
01:45:35 INFO: Free blob page refd start = 2319107 L=2319107

Thus you should first running a database integrity check on the database:

backup ‘/dev/null’

and perform a +crash-dump and restore of the database to eliminate these errors 
as detailed at:


http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#diagnosingrepairing

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



On 2 Mar 2016, at 14:30, Nolle, Andreas 
<no...@hs-albsig.de<mailto:no...@hs-albsig.de>> wrote:

Dear Kingsley,

thanks for your reply.

The overall triple counts of each instance is
  - Virtuoso instance running at port 8891: 17765873 triples
  - Virtuoso instance running at port 8893: 27897291 triples
  - Virtuoso instance running at port 8895: 16956 triples
  - Virtuoso instance running at port 8899: 72372256 triples

If you are interested in the ini files, you can find the current versions 
athttps://www.dropbox.com/s/awdx744vomsb5vr/virtuoso_ini.zip?dl=0

For profiling I just used simple the test queries such that it is probably 
easier to find out the problem. By doing so, I came to the assumption that 
especially the size of the result set causes really long query evaluation time:
If I run q1:
   SELECT ?x ?b
   WHERE {
  SERVICE <http://141.87.4.8:8891/sparql> {
 ?x <

Re: [Virtuoso-users] infrequent errors on parallel querying

[Hugh Williams]

Hi Andreas,

The select top 10 * from sys_l_stat order by waits desc; queries from the 8891 
& 8895 instances do not show an excessive number of waits, only about 1990+ 
waits on the SYS_DAV_QUEUE index. When you ran those queries was the system 
under  typical work load ie  was the long running query being run at the time ? 
Note you should also run the query a number of time to get a sample of the 
waits as indicated at:

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#perfdiaglocking

I certainly would have expected a larger number of waits on the 8891 instance 
where the "WARNING: * Monitor: Many lock waits”  errors where occurring 
extensively in the log …

You indicate the "transaction deadlock and No ext map for dp 3193852 in 
uncommitted blob cpt” on the 8895 instance only occur when running several 
queries in parallel, so this would be the time to be running the query to check 
for locks/waits. How many of these queries are typically being run in parallel 
? Also are these queries being executed via the /sparql end point ie HTTP  or 
SQL interface ?

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



> On 2 Mar 2016, at 17:31, Nolle, Andreas  wrote:
> 
> Dear Hugh,
> 
> please find the requested information about top50 
> athttps://www.dropbox.com/s/ylrb20ylze38cj6/top50.zip?dl=0
> 
> Please notice that both errors (transaction deadlock and No ext map for dp 
> 3193852 in uncommitted blob cpt) only occur if I run several queries (of the 
> type already mentioned) in parallel.
> Since my assumption is that especially the transaction deadlock error is 
> probably related to the long evaluation time, I haven’t restarted my 
> application so far…
> However, I will run a database integrity check and perform a +crash-dump and 
> restore to eliminate the “uncommitted blob” errors in the next run of my 
> application.
> 
> Best regards
> Andy



smime.p7s
Description: S/MIME cryptographic signature
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Dear Hugh,

please find the requested information about top50 at 
https://www.dropbox.com/s/ylrb20ylze38cj6/top50.zip?dl=0

Please notice that both errors (transaction deadlock and No ext map for dp 
3193852 in uncommitted blob cpt) only occur if I run several queries (of the 
type already mentioned) in parallel.
Since my assumption is that especially the transaction deadlock error is 
probably related to the long evaluation time, I haven’t restarted my 
application so far…
However, I will run a database integrity check and perform a +crash-dump and 
restore to eliminate the “uncommitted blob” errors in the next run of my 
application.

Best regards
Andy


--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Kingsley Idehen]

On 3/2/16 9:30 AM, Nolle, Andreas wrote:
>
> Dear Kingsley,
>
>  
>
> thanks for your reply.
>
>  
>
> The overall triple counts of each instance is
>
>   - Virtuoso instance running at port 8891: 17765873 triples
>
>   - Virtuoso instance running at port 8893: 27897291 triples
>
>   - Virtuoso instance running at port 8895: 16956 triples
>
>   - Virtuoso instance running at port 8899: 72372256 triples
>
>  
>
> If you are interested in the ini files, you can find the current
> versions at
> https://www.dropbox.com/s/awdx744vomsb5vr/virtuoso_ini.zip?dl=0
>
>  
>
> For profiling I just used simple the test queries such that it is
> probably easier to find out the problem. By doing so, I came to the
> assumption that especially the size of the result set causes really
> long query evaluation time:
>
> If I run q1:
>
>SELECT ?x ?b
>
>WHERE {
>
>   SERVICE  {
>
>  ?x  ?b .
>
>   } .
>
>}
>
> at lets say Virtuoso instance running at port 8895 it will end up
> within 2 minutes. This is because the service call will return only
> 32562 results.
>
>  
>
> But if I run q2:
>
>SELECT ?x ?b
>
>WHERE {
>
>   SERVICE  {
>
>  ?x  ?b .
>
>   } .
>
>}
>
> where the service call returns 313421 results, the query evaluation on
> the same instance (running at port 8895) will take something around 1
> hour!
>
>  
>
> The corresponding explain and profile plans of those queries can be
> found at https://www.dropbox.com/s/ein0hxhtgk85ci9/analysis_files.zip?dl=0
>
> Please notice that *_federated means the queries are like above and
> are evaluated at the Virtuoso instance running at port 8895, and
> *_local means the evaluation of these queries (without SERVICE) at the
> Virtuoso instance running at port 8891.
>
>  
>
> On analyzing the long evaluation times, especially for q2 in the
> federated case, I found out that if q2 is for example evaluated at
> Virtuoso instance running at port 8895 and the Virtuoso instance
> running at port 8891 is shut downed after 5 minutes, the query
> evaluation still proceeds and ended after more than one hour with the
> same number of results (313421) as e.g. in the local evaluation.
>
>  
>
> Please let me know if you need any other information.
>
>  
>
> Best regards
>
> Andy
>

Andy,

Okay I think we have enough information to work with.



-- 
Regards,

Kingsley Idehen   
Founder & CEO 
OpenLink Software 
Company Web: http://www.openlinksw.com
Personal Weblog 1: http://kidehen.blogspot.com
Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen
Twitter Profile: https://twitter.com/kidehen
Google+ Profile: https://plus.google.com/+KingsleyIdehen/about
LinkedIn Profile: http://www.linkedin.com/in/kidehen
Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this



smime.p7s
Description: S/MIME Cryptographic Signature
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Hugh Williams]

.
>}
> where the service call returns 313421 results, the query evaluation on the 
> same instance (running at port 8895) will take something around 1 hour!
>  
> The corresponding explain and profile plans of those queries can be found 
> athttps://www.dropbox.com/s/ein0hxhtgk85ci9/analysis_files.zip?dl=0 
> <https://www.dropbox.com/s/ein0hxhtgk85ci9/analysis_files.zip?dl=0>
> Please notice that *_federated means the queries are like above and are 
> evaluated at the Virtuoso instance running at port 8895, and *_local means 
> the evaluation of these queries (without SERVICE) at the Virtuoso instance 
> running at port 8891.
>  
> On analyzing the long evaluation times, especially for q2 in the federated 
> case, I found out that if q2 is for example evaluated at Virtuoso instance 
> running at port 8895 and the Virtuoso instance running at port 8891 is shut 
> downed after 5 minutes, the query evaluation still proceeds and ended after 
> more than one hour with the same number of results (313421) as e.g. in the 
> local evaluation.
>  
> Please let me know if you need any other information.
>  
> Best regards
> Andy
>  
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
>  
> <http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___>
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net 
> <mailto:Virtuoso-users@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users 
> <https://lists.sourceforge.net/lists/listinfo/virtuoso-users>


Begin forwarded message:

From: Kingsley Idehen <kide...@openlinksw.com>
Subject: Re: [Virtuoso-users] infrequent errors on parallel querying
Date: 2 March 2016 at 13:28:22 GMT
To: virtuoso-users@lists.sourceforge.net

On 3/2/16 4:24 AM, Nolle, Andreas wrote:
> Hi Hugh,
> 
>  
> 
> this was also my first guess, but the network is not the problem. This is 
> definitively the case because I’ve evaluated again q2
> 
>SELECT ?x ?b
> 
>WHERE {
> 
>   SERVICE < <http://141.87.4.8:8891/sparql>http://141.87.4.8:8891/sparql 
> <http://141.87.4.8:8891/sparql>> {
> 
>  ?x < 
> <http://swrc.ontoware.org/ontology#pages>http://swrc.ontoware.org/ontology#pages
>  <http://swrc.ontoware.org/ontology#pages>> ?b .
> 
>   } .
> 
>}
> 
> at Virtuoso instance running at port 8895 and SHUTDOWN the Virtuoso instance 
> running at port 8891 after 5 minutes.
> 
> The query evaluation hat still proceed and ended after more than one hour 
> with the same number of results (313421).
> 
> At that it doesn’t matter if the query is executed via isql or via the SPARQL 
> interface (locally AND remote).
> 
>  
> 
> So for me it seems like a bug, because the Virtuoso at 8895 takes one hour 
> despite the fact that no additional join or any other operation has to be 
> done.
> 
>  
> 
> Best
> 
> Andy
> 

Andy, 

There are a lot of factors that could come into play here. If you haven't 
already done so, please provide query and result times for queries against each 
of the individual instances in this SPARQL-FED query. The goal would be to at 
least eliminate any local issues. 

Also shed some light on the triple count per each of these instances. 

Ultimately we have a network or query optimizer issue in play, we just need a 
productive route to determining which of these it is..

-- 
Regards,

Kingsley Idehen   
Founder & CEO 
OpenLink Software 
Company Web: http://www.openlinksw.com <http://www.openlinksw.com/>
Personal Weblog 1: http://kidehen.blogspot.com <http://kidehen.blogspot.com/>
Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen 
<http://www.openlinksw.com/blog/~kidehen>
Twitter Profile: https://twitter.com/kidehen <https://twitter.com/kidehen>
Google+ Profile: https://plus.google.com/+KingsleyIdehen/about 
<https://plus.google.com/+KingsleyIdehen/about>
LinkedIn Profile: http://www.linkedin.com/in/kidehen 
<http://www.linkedin.com/in/kidehen>
Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this 
<http://kingsley.idehen.net/dataspace/person/kidehen#this>
--
Site24x7 APM Insi

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Dear Kingsley,

thanks for your reply.

The overall triple counts of each instance is
  - Virtuoso instance running at port 8891: 17765873 triples
  - Virtuoso instance running at port 8893: 27897291 triples
  - Virtuoso instance running at port 8895: 16956 triples
  - Virtuoso instance running at port 8899: 72372256 triples

If you are interested in the ini files, you can find the current versions at 
https://www.dropbox.com/s/awdx744vomsb5vr/virtuoso_ini.zip?dl=0



For profiling I just used simple the test queries such that it is probably 
easier to find out the problem. By doing so, I came to the assumption that 
especially the size of the result set causes really long query evaluation time:

If I run q1:

   SELECT ?x ?b

   WHERE {

  SERVICE  {

 ?x  ?b .

  } .

   }

at lets say Virtuoso instance running at port 8895 it will end up within 2 
minutes. This is because the service call will return only 32562 results.



But if I run q2:

   SELECT ?x ?b

   WHERE {

  SERVICE  {

 ?x  ?b .

  } .

   }

where the service call returns 313421 results, the query evaluation on the same 
instance (running at port 8895) will take something around 1 hour!



The corresponding explain and profile plans of those queries can be found at 
https://www.dropbox.com/s/ein0hxhtgk85ci9/analysis_files.zip?dl=0

Please notice that *_federated means the queries are like above and are 
evaluated at the Virtuoso instance running at port 8895, and *_local means the 
evaluation of these queries (without SERVICE) at the Virtuoso instance running 
at port 8891.



On analyzing the long evaluation times, especially for q2 in the federated 
case, I found out that if q2 is for example evaluated at Virtuoso instance 
running at port 8895 and the Virtuoso instance running at port 8891 is shut 
downed after 5 minutes, the query evaluation still proceeds and ended after 
more than one hour with the same number of results (313421) as e.g. in the 
local evaluation.



Please let me know if you need any other information.



Best regards

Andy


--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Kingsley Idehen]

On 3/2/16 4:24 AM, Nolle, Andreas wrote:
>
> Hi Hugh,
>
>  
>
> this was also my first guess, but the network is not the problem. This
> is definitively the case because I’ve evaluated again q2
>
>SELECT ?x ?b
>
>WHERE {
>
>   SERVICE  {
>
>  ?x  ?b .
>
>   } .
>
>}
>
> at Virtuoso instance running at port 8895 and SHUTDOWN the Virtuoso
> instance running at port 8891 after 5 minutes.
>
> The query evaluation hat still proceed and ended after more than one
> hour with the same number of results (313421).
>
> At that it doesn’t matter if the query is executed via isql or via the
> SPARQL interface (locally AND remote).
>
>  
>
> So for me it seems like a bug, because the Virtuoso at 8895 takes one
> hour despite the fact that no additional join or any other operation
> has to be done.
>
>  
>
> Best
>
> Andy
>

Andy,

There are a lot of factors that could come into play here. If you
haven't already done so, please provide query and result times for
queries against each of the individual instances in this SPARQL-FED
query. The goal would be to at least eliminate any local issues.

Also shed some light on the triple count per each of these instances.

Ultimately we have a network or query optimizer issue in play, we just
need a productive route to determining which of these it is..

-- 
Regards,

Kingsley Idehen   
Founder & CEO 
OpenLink Software 
Company Web: http://www.openlinksw.com
Personal Weblog 1: http://kidehen.blogspot.com
Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen
Twitter Profile: https://twitter.com/kidehen
Google+ Profile: https://plus.google.com/+KingsleyIdehen/about
LinkedIn Profile: http://www.linkedin.com/in/kidehen
Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this



smime.p7s
Description: S/MIME Cryptographic Signature
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Hi Hugh,

this was also my first guess, but the network is not the problem. This is 
definitively the case because I’ve evaluated again q2

   SELECT ?x ?b

   WHERE {

  SERVICE  {

 ?x  ?b .

  } .

   }
at Virtuoso instance running at port 8895 and SHUTDOWN the Virtuoso instance 
running at port 8891 after 5 minutes.
The query evaluation hat still proceed and ended after more than one hour with 
the same number of results (313421).
At that it doesn’t matter if the query is executed via isql or via the SPARQL 
interface (locally AND remote).

So for me it seems like a bug, because the Virtuoso at 8895 takes one hour 
despite the fact that no additional join or any other operation has to be done.

Best
Andy

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Sorry for sending the mail again, but my previous message to Virtuoso-users 
awaits moderator approval…
So I cut out some parts:


Hi Hugh,

if I open the provided files with an editor, there is no error.

For example, profile_q2_federated contain:

Connected to OpenLink Virtuoso
Driver: 07.20.3215 OpenLink Virtuoso ODBC Driver
result
LONG VARCHAR
___

http://www.bibsonomy.org/bibtex/20001ccbac4a192c385dd1a813c33e4a4 
130--147
http://www.bibsonomy.org/bibtex/2001726ad493878da37ce74c327b64142
451--464
http://www.bibsonomy.org/bibtex/2002a5c36b038d703388ea3133d1c1d64
125--144
http://www.bibsonomy.org/bibtex/2002b342f8686b42ca9386b7101e57d03 
1966--1972
http://www.bibsonomy.org/bibtex/2002f844c77738f07d6aed27377358948  
36--50
http://www.bibsonomy.org/bibtex/200583305f841e25137f18dc67f63e83a  
571-573
http://www.bibsonomy.org/bibtex/200825babcdaa38b3fa7afef00705b74e  
1553-1558
http://www.bibsonomy.org/bibtex/2008eea7c2709538ca3824513b797dd5f 
319--355
http://www.bibsonomy.org/bibtex/20091a9f693bb98983ac76724ecfaa344  
195--205
http://www.bibsonomy.org/bibtex/200b84d4fae9f972bd06d1648119d1ac8  
203--223
http://www.bibsonomy.org/bibtex/200d0f3fa9f6a4ab1fc84973c9f6d03f7 
10--12
http://www.bibsonomy.org/bibtex/200d246b67269ee8de5bdb900b766f229 92-97
http://www.bibsonomy.org/bibtex/200ecc41577170f3a59cd67882c309d0a  77-86
http://www.bibsonomy.org/bibtex/200ecc846b53f52079c34af53e8badcfa   
4--10
http://www.bibsonomy.org/bibtex/200ef9f672b4e826261dcd9219d5d0507  82-83
http://www.bibsonomy.org/bibtex/200f4976a86726f22802bd39e93386926 
238--246
http://www.bibsonomy.org/bibtex/2010ebc7f46486e290469eb950a20d04d 
319--325
http://www.bibsonomy.org/bibtex/2011380714119b1a98da3addca3fe0e50 18
http://www.bibsonomy.org/bibtex/2012c49708658e54a9e61214be4bdc4b027--36
http://www.bibsonomy.org/bibtex/2012c49708658e54a9e61214be4bdc4b027–36

{
time   1.8e-07% fanout 1 input 1 rows

Precode:
  0: vector := Call vector ()
  5: vector := Call vector (, )
  10: BReturn 0
Subquery 31
{
time   1.5e-07% fanout 1 input 1 rows

Precode:
  0: __stub :=  := artm  1
  4: BReturn 0
END Node
time   6.5e-08% fanout 0 input 1 rows
Subquery Select(__stub)
}
time 1e+02% fanout  1000 input 1 rows

Precode:
  0: ws_endpoint :=  := artm http://141.87.4.8:8891/sparql>
  4: ws_params :=  := artm vector
  8: qtext_template :=  := artm http://swrc.ontoware.org/ontology#pages> ?b . }>
  12: qtext_posmap :=  := artm 
  16: param_row :=  := artm vector
  20: expected_vars :=  := artm vector
  24: proc_ctr :=  := artm  0
  28::= Call __reset_temp ( 140051501624464 )
  33::= Call DB.DBA.SPARQL_SINV_IMP (http://141.87.4.8:8891/sparql>, vector, http://swrc.ontoware.org/ontology#pages> ?b . }>, , 
vector, vector)
  40: BReturn 0
Key from temp (proc_ctr, RSET)


After code:
  0: aref := Call aref (RSET,  1 )
  5: x := Call __id2in (aref)
  10: aref := Call aref (RSET,  0 )
  15: b := Call __ro2sq (aref)
  20: BReturn 0
time   2.7e-08% fanout 0 input  1000 rows
Select (x, b)
}


4987453 msec 99% cpu, 1.40551e+06 rnd200089 seq 0% same seg 
0.0686584% same pg
144592 disk reads, 143623 read ahead,  0.445211% wait
Compilation: 2 msec 0 reads 0% read 0 messages 0% clw

5 Rows. -- 4987467 msec.


Best
Andy

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

e 8893: 27897291

-Virtuoso instance 8895: 16956

-    Virtuoso instance 8899: 72372256

Best
Andy



Von: Hugh Williams [mailto:hwilli...@openlinksw.com]
Gesendet: Sonntag, 21. Februar 2016 02:51
An: Nolle, Andreas <no...@hs-albsig.de>
Cc: virtuoso-users@lists.sourceforge.net
Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying

Hi Andy,

What is a typical query being executing that is result in a HTTP error and what 
is the actual error ?

When the error occurs what is the status of the server at that point , please 
provide the output of running the command:

status();

Are any errors reported in the “virtuoos.log” file, please provide a copy for 
review ?

You indicate having a 128GB RAM machine but memory allocated to Virtuoso is  
that for a 16GB RAM machine ie NumberOfBuffer = 136 in the INI file, was 
there a specific reason for this ?

How many triples are hosted in the Virtuoso instance , please provide the 
output of the count query:

select count(*) where {?s ?p ?o}

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



On 20 Feb 2016, at 15:47, Nolle, Andreas 
<no...@hs-albsig.de<mailto:no...@hs-albsig.de>> wrote:

Hi folks,

I’ve set up four Virtuoso instances (Version 7.2.2.3215) on an Ubuntu 14.04.4 
LTS server with 12 cores and 128 GB memory.
If I run several (federated) queries in parallel (maximal 16 at a time), where 
some of the queries may take several minutes, it may happened that a SPARQL 
endpoint return some HTTP error. If I rerun the same query some seconds later, 
it will ordinarily be evaluated correct and the requested results are returned.
I’ve already tried to solve this infrequent issue but without any success. Can 
you please give me some recommendations?
Maybe my configuration may comprise some not well-chosen parameter?


Please find below my virtuoso.ini:

[Database]
DatabaseFile   = ../var/lib/virtuoso/db/virtuoso.db
ErrorLogFile   = ../var/lib/virtuoso/db/virtuoso.log
LockFile   = ../var/lib/virtuoso/db/virtuoso.lck
TransactionFile= ../var/lib/virtuoso/db/virtuoso.trx
xa_persistent_file = ../var/lib/virtuoso/db/virtuoso.pxa
ErrorLogLevel  = 7
FileExtend = 200
MaxCheckpointRemap = 15
Striping   = 0
TempStorage= TempDatabase

[TempDatabase]
DatabaseFile   = ../var/lib/virtuoso/db/virtuoso-temp.db
TransactionFile= ../var/lib/virtuoso/db/virtuoso-temp.trx
MaxCheckpointRemap = 9000
Striping   = 0

[Parameters]
ServerPort   = 1112
LiteMode = 0
DisableUnixSocket= 1
DisableTcpSocket = 0
MaxClientConnections = 1
ServerThreads= 1000
CheckpointInterval   = 60
O_DIRECT = 0
CaseMode = 2
MaxStaticCursorRows  = 5000
CheckpointAuditTrail = 0
AllowOSCalls = 0
SchedulerInterval= 10
DirsAllowed  = ., ../share/virtuoso/vad,../saleem/dumps, 
/opt/benchset/bibsonomy
ThreadCleanupInterval= 1
ThreadThreshold  = 1000
ResourcesCleanupInterval = 1
FreeTextBatchSize= 10
SingleCPU= 0
VADInstallDir= ../share/virtuoso/vad/
PrefixResultNames= 0
RdfFreeTextRulesSize = 100
IndexTreeMaps= 512
MaxMemPoolSize   = 2
PrefixResultNames= 0
MacSpotlight = 0
MaxQueryMem  = 8G; memory allocated to query processor
VectorSize   = 1000 ; initial parallel query vector 
(array of query operations) size
MaxVectorSize= 200 ; query vector size threshold.
AdjustVectorSize = 0
ThreadsPerQuery  = 24
AsyncQueueMaxThreads = 36
NumberOfBuffers  = 136
MaxDirtyBuffers  = 100
TraceOn  = errors
CallstackOnException = 1
MaxSortedTopRows = 1000
DefaultIsolation   = 2

[HTTPServer]
ServerPort  = 8891
ServerRoot  = ../var/lib/virtuoso/vsp
MaxClientConnections= 1
DavRoot = DAV
EnabledDavVSP   = 0
HTTPProxyEnabled= 0
TempASPXDir = 0
DefaultMailServer   = localhost:25
ServerThreads   = 1000
MaxKeepAlives   = 100
KeepAliveTimeout= 10
MaxCachedProxyConnections   = 10
ProxyConnectionCacheTimeout = 15
HTTPThreadSize  = 28
HttpPrintWarningsInOutput   = 0
Charset = UTF-8
MaintenancePage = atomic.html
EnabledGzipContent  = 1

[Auto

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Hi Hugh,

the query workload is almost equally distributed.

The errors in the log regarding the lock file and “missed delete of name id 
cache” occurred both not before updating to Version 7.2.2.1. Maybe some indexes 
aren’t updated…

The status of the Virtuoso instance running at Port 8895 seems to remain 
indefinitely.

Only the issue regarding memory leaks doesn’t occurred (until now) since I 
updated to the new version. Don’t know if this is solved…

By the way, I have an additional question:
Since I have a lot of queries comprising parts like
 ?x <http://purl.org/dc/terms/partOf> ?a .
 FILTER ( ?x >= 
<http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp> ) .
 FILTER ( ?x < 
<http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp> ) .
Is it possible to speed up the evaluation time of those queries, e.g. by enable 
some additional indexing?

Cheers
Andy



Von: Hugh Williams [mailto:hwilli...@openlinksw.com]
Gesendet: Dienstag, 23. Februar 2016 01:05
An: Nolle, Andreas <no...@hs-albsig.de>
Cc: virtuoso-users@lists.sourceforge.net
Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying

Hi Andreas,

Missed that you were running 4 Virtuoso instances, how is the query workload 
split across the 4 instances ?

I noticed the following errors in all the logs during done attempt to restart:

09:23:02 ERROR: Unable to lock file ../var/lib/virtuoso/db/virtuoso.lck 
(Resource temporarily unavailable).
09:23:02 ERROR: Virtuoso is already runnning (pid 1264)
09:23:02 ERROR: This probably means you either do not have permission to start
09:23:02 ERROR: this server, or that virtuoso-t is already running.
09:23:02 ERROR: If you are absolutely sure that this is not the case, please try
09:23:02 ERROR: to remove the file ../var/lib/virtuoso/db/virtuoso.lck and 
start again.
09:26:41 INFO: ERRS_0 01V01 QW004 :2891: 
WS.WS.SPARQL_ENDPOINT_GENERATE_FORM: Incompatible types INTEGER (189) and 
VARCHAR (182) in = for debug and 
09:34:13 DEBUG: missed delete of name id cache 
benchset/d4e1a90786ac524b9f96f9d2f0506a12 0 (0x20e975b )
09:34:16 DEBUG: missed delete of name id cache 
benchset/265b6be68f2030063c4c17e3778dbbe1 0 (0x2122444 )
09:35:50 DEBUG: missed delete of name id cache 
benchset/8d0299f2a49cf965ac7c98101f55afaa 0 (0x2181ac6 )

But then did not occur on the following startup, do you know what happened here 
as it seem there was an attempt to start an already running server, but I have 
not seen the "DEBUG: missed delete of name id cache …” errors before …

In the status_8895 output I see many occurrences of:

Pending:
  1126400: IER 141.87.4.9
  156: IER 141.87.4.9
  152: IER 141.87.4.9
  148: IER 141.87.4.9
  144: IER 141.87.4.9
  .
  .
  .

Do these remain indefinitely or do they get cleanup after a while ?

Also you indicated that this errors had not been seen with the 3215 build 
upgrade, is this still the case ?


Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



On 21 Feb 2016, at 13:46, Nolle, Andreas 
<no...@hs-albsig.de<mailto:no...@hs-albsig.de>> wrote:

Hi Hugh,

typical queries that are executed (each at any Virtuoso instance) are like the 
following:

SELECT ?x ?P0src ?P1src
WHERE {
   {
  SERVICE <http://141.87.4.8:8899/sparql> {
 ?x rdf:type <http://purl.org/ontology/bibo/Collection> .
  } .
  BIND (<http://141.87.4.8:8899/sparql> AS ?P0src)
   }
   UNION
   {
  ?x rdf:type <http://purl.org/ontology/bibo/Collection> .
  BIND (<http://141.87.4.8:8891/sparql> AS ?P0src)
   }
   UNION
   {
  SERVICE <http://141.87.4.8:8893/sparql> {
 ?x rdf:type <http://purl.org/ontology/bibo/Collection> .
  } .
  BIND (<http://141.87.4.8:8893/sparql> AS ?P0src)
   }
   UNION
   {
  SERVICE <http://141.87.4.8:8895/sparql> {
 ?x rdf:type <http://purl.org/ontology/bibo/Collection> .
  } .
  BIND (<http://141.87.4.8:8895/sparql> AS ?P0src)
   } .
   ?x rdf:type <http://swrc.ontoware.org/ontology#Article> .
   BIND (<http://141.87.4.8:8891/sparql> AS ?P1src)
}

Since it is not possible to use the keyword OFFSET for queries, i.e. query 
parts, that would return more than 1048576 results, there are also queries like:

SELECT ?x ?a ?b ?P0src ?P1src
WHERE {
   {
  SERVICE <http://141.87.4.8:8899/sparql> {
 ?x <http://purl.org/dc/terms/partOf> ?a .
 FILTER ( ?x >= 
<http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOI

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Hi Hugh,

if I open the provided files with an editor, there is no error.

For example, profile_q2_federated contain:

Connected to OpenLink Virtuoso
Driver: 07.20.3215 OpenLink Virtuoso ODBC Driver
result
LONG VARCHAR
___

http://www.bibsonomy.org/bibtex/20001ccbac4a192c385dd1a813c33e4a4 
130--147
http://www.bibsonomy.org/bibtex/2001726ad493878da37ce74c327b64142
451--464
http://www.bibsonomy.org/bibtex/2002a5c36b038d703388ea3133d1c1d64
125--144
http://www.bibsonomy.org/bibtex/2002b342f8686b42ca9386b7101e57d03 
1966--1972
http://www.bibsonomy.org/bibtex/2002f844c77738f07d6aed27377358948  
36--50
http://www.bibsonomy.org/bibtex/200583305f841e25137f18dc67f63e83a  
571-573
http://www.bibsonomy.org/bibtex/200825babcdaa38b3fa7afef00705b74e  
1553-1558
http://www.bibsonomy.org/bibtex/2008eea7c2709538ca3824513b797dd5f 
319--355
http://www.bibsonomy.org/bibtex/20091a9f693bb98983ac76724ecfaa344  
195--205
http://www.bibsonomy.org/bibtex/200b84d4fae9f972bd06d1648119d1ac8  
203--223
http://www.bibsonomy.org/bibtex/200d0f3fa9f6a4ab1fc84973c9f6d03f7 
10--12
http://www.bibsonomy.org/bibtex/200d246b67269ee8de5bdb900b766f229 92-97
http://www.bibsonomy.org/bibtex/200ecc41577170f3a59cd67882c309d0a  77-86
http://www.bibsonomy.org/bibtex/200ecc846b53f52079c34af53e8badcfa   
4--10
http://www.bibsonomy.org/bibtex/200ef9f672b4e826261dcd9219d5d0507  82-83
http://www.bibsonomy.org/bibtex/200f4976a86726f22802bd39e93386926 
238--246
http://www.bibsonomy.org/bibtex/2010ebc7f46486e290469eb950a20d04d 
319--325
http://www.bibsonomy.org/bibtex/2011380714119b1a98da3addca3fe0e50 18
http://www.bibsonomy.org/bibtex/2012c49708658e54a9e61214be4bdc4b027--36
http://www.bibsonomy.org/bibtex/2012c49708658e54a9e61214be4bdc4b027–36

{
time   1.8e-07% fanout 1 input 1 rows
Precode:
  0: vector := Call vector ()
  5: vector := Call vector (, )
  10: BReturn 0
Subquery 31
{
time   1.5e-07% fanout 1 input 1 rows
Precode:
  0: __stub :=  := artm  1
  4: BReturn 0
END Node
time   6.5e-08% fanout 0 input 1 rows
Subquery Select(__stub)
}
time 1e+02% fanout  1000 input 1 rows
Precode:
  0: ws_endpoint :=  := artm http://141.87.4.8:8891/sparql>
  4: ws_params :=  := artm vector
  8: qtext_template :=  := artm http://swrc.ontoware.org/ontology#pages> ?b . }>
  12: qtext_posmap :=  := artm 
  16: param_row :=  := artm vector
  20: expected_vars :=  := artm vector
  24: proc_ctr :=  := artm  0
  28::= Call __reset_temp ( 140051501624464 )
  33::= Call DB.DBA.SPARQL_SINV_IMP (http://141.87.4.8:8891/sparql>, vector, http://swrc.ontoware.org/ontology#pages> ?b . }>, , 
vector, vector)
  40: BReturn 0
Key from temp (proc_ctr, RSET)

After code:
  0: aref := Call aref (RSET,  1 )
  5: x := Call __id2in (aref)
  10: aref := Call aref (RSET,  0 )
  15: b := Call __ro2sq (aref)
  20: BReturn 0
time   2.7e-08% fanout 0 input  1000 rows
Select (x, b)
}


4987453 msec 99% cpu, 1.40551e+06 rnd200089 seq 0% same seg 
0.0686584% same pg
144592 disk reads, 143623 read ahead,  0.445211% wait
Compilation: 2 msec 0 reads 0% read 0 messages 0% clw

5 Rows. -- 4987467 msec.


Best
Andy


Von: Hugh Williams [mailto:hwilli...@openlinksw.com]
Gesendet: Montag, 29. Februar 2016 17:05
An: Nolle, Andreas <no...@hs-albsig.de>
Cc: virtuoso-users@lists.sourceforge.net
Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying

Hi Andreas,

How are you running the script for the query profiles as all scripts provided 
contain errors of the form:

Last login: Mon Feb 29 16:01:14 on ttys006
/Users/hwilliams/Downloads/analysis_files/profile_q2_federated ; exit;
Hughs-MBP:~ hwilliams$ 
/Users/hwilliams/Downloads/analysis_files/profile_q2_federated ; exit;
/Users/hwilliams/Downloads/analysis_files/profile_q2_federated: line 1: 
Connected: command not found
/Users/hwilliams/Downloads/analysis_files/profile_q2_federated: line 2: 
Driver:: command not found
/Users/hwilliams/Downloads/analysis_files/profile_q2_federated: line 3: result: 
command not found
/Users/hwilliams/Downloads/analysis_files/profile_q2_federated: line 4: LONG: 
command not found
/Users/hwilliams/Downloads/analysis_files/profile_q2_federated: line 5: 
___:
 command not found
/Users/hwilliams/Downloads/analysis_files/profile_q2_federated: line 7: 
http://www.bibsonomy.org/bibtex/20001ccbac4a192c385dd1a813c33e4a4: No such file 
or directory
/Users/hwilliams/Downloads/analysis_files/profile_q2_federated: line 8: 
http://www.bibsonomy.org/bibtex/2001726ad493878da37ce74c327b64142: No such file 
or di

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Dear Hugh,

could you already open and inspect the files?

Best
Andy



Von: Nolle, Andreas
Gesendet: Montag, 29. Februar 2016 17:43
An: 'Hugh Williams' 
Cc: 'virtuoso-users@lists.sourceforge.net' 

Betreff: AW: [Virtuoso-users] infrequent errors on parallel querying

Sorry for sending the mail again, but my previous message to Virtuoso-users 
awaits moderator approval…
So I cut out some parts:


Hi Hugh,

if I open the provided files with an editor, there is no error.

For example, profile_q2_federated contain:

Connected to OpenLink Virtuoso
Driver: 07.20.3215 OpenLink Virtuoso ODBC Driver
result
LONG VARCHAR
___

http://www.bibsonomy.org/bibtex/20001ccbac4a192c385dd1a813c33e4a4 
130--147
http://www.bibsonomy.org/bibtex/2001726ad493878da37ce74c327b64142
451--464
http://www.bibsonomy.org/bibtex/2002a5c36b038d703388ea3133d1c1d64
125--144
http://www.bibsonomy.org/bibtex/2002b342f8686b42ca9386b7101e57d03 
1966--1972
…
{
time   1.8e-07% fanout 1 input 1 rows

Precode:
  0: vector := Call vector ()
  5: vector := Call vector (, )
  10: BReturn 0
Subquery 31
{
time   1.5e-07% fanout 1 input 1 rows

Precode:
  0: __stub :=  := artm  1
  4: BReturn 0
END Node
time   6.5e-08% fanout 0 input 1 rows
Subquery Select(__stub)
}
time 1e+02% fanout  1000 input 1 rows
…

4987453 msec 99% cpu, 1.40551e+06 rnd200089 seq 0% same seg 
0.0686584% same pg
144592 disk reads, 143623 read ahead,  0.445211% wait
Compilation: 2 msec 0 reads 0% read 0 messages 0% clw

5 Rows. -- 4987467 msec.

Best
Andy

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Sorry for sending the mail again, but my previous message to Virtuoso-users 
awaits moderator approval…
So I cut out some parts:


Hi Hugh,

if I open the provided files with an editor, there is no error.

For example, profile_q2_federated contain:

Connected to OpenLink Virtuoso
Driver: 07.20.3215 OpenLink Virtuoso ODBC Driver
result
LONG VARCHAR
___

http://www.bibsonomy.org/bibtex/20001ccbac4a192c385dd1a813c33e4a4 
130--147
http://www.bibsonomy.org/bibtex/2001726ad493878da37ce74c327b64142
451--464
http://www.bibsonomy.org/bibtex/2002a5c36b038d703388ea3133d1c1d64
125--144
http://www.bibsonomy.org/bibtex/2002b342f8686b42ca9386b7101e57d03 
1966--1972
…
{
time   1.8e-07% fanout 1 input 1 rows

Precode:
  0: vector := Call vector ()
  5: vector := Call vector (, )
  10: BReturn 0
Subquery 31
{
time   1.5e-07% fanout 1 input 1 rows

Precode:
  0: __stub :=  := artm  1
  4: BReturn 0
END Node
time   6.5e-08% fanout 0 input 1 rows
Subquery Select(__stub)
}
time 1e+02% fanout  1000 input 1 rows
…

4987453 msec 99% cpu, 1.40551e+06 rnd200089 seq 0% same seg 
0.0686584% same pg
144592 disk reads, 143623 read ahead,  0.445211% wait
Compilation: 2 msec 0 reads 0% read 0 messages 0% clw

5 Rows. -- 4987467 msec.

Best
Andy

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Hugh Williams]

t-
> Von: Hugh Williams [mailto:hwilli...@openlinksw.com] 
> Gesendet: Sonntag, 28. Februar 2016 22:21
> An: Nolle, Andreas <no...@hs-albsig.de>
> Cc: virtuoso-users@lists.sourceforge.net
> Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying
> 
> Hi Andreas,
> 
> Generally there should be no create additional indexes as the 2 Full & 3 
> partial indexes have been found to be sufficient for most use cases, with 
> only one use case encountered where an addtional partial index was required 
> as detailed at:
> 
>   
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning#Index%20Scheme%20Selection
> 
> Thus what indexes (STATISTICS DB.DBA.RDF_QUAD;) are created on these various 
> instances you have are they all the same or do they have different indexes as 
> the more indexes the more large the database and hence more memory required 
> for hosting it. Also are the triple counts the same on all these instances ?
> 
> Did you make any of the other INI file changes in my previous email, 
> certainly “AdjustVectorSize” should be set to 0 ?
> 
> You should also consider profiling the queries to determine if the best quey 
> plan is being used, which can be done with the “profile()” function as 
> detailed at:
> 
>   
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksAanalyzingSPARQLQuery
>   http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#querylogging  - 
> Query Logging 
> 
> Note there is are also some INI file params that can be set to control query 
> optimisation ie plans as if a bad plan is being chosen then these options can 
> in some cases enable better plans to be chosen, see:
> 
>   
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtQueryOptDiagnostic
> 
> If you can provide query plans, database statistics then these can be 
> analysed to trying and determine the cause of long running queries …
> 
> It certainly would be interesting to see how the query plans and database 
> stats vary being the instance where the query runs in msecs and the other 
> were it runs in 40+mins ...
> 
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc.  //  http://www.openlinksw.com/
> Weblog   -- http://www.openlinksw.com/blogs/
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> Twitter  -- http://twitter.com/OpenLink
> Google+  -- http://plus.google.com/100570109519069333827/
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> Universal Data Access, Integration, and Management Technology Providers
> 
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
> ___
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users



smime.p7s
Description: S/MIME cryptographic signature
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Hi Hugh,

again many thanks for your reply and working at the weekend!

I've created the index just on a copy of the original database for some tests...
So all indexes (STATISTICS DB.DBA.RDF_QUAD;) created on these various instances 
are the same, since these are all default indexes.
If you are interested on the output of STATISTICS DB.DBA.RDF_QUAD; for each 
Virtuoso instance, please see the zip file referenced below.
The triple counts of each instance is
  - Virtuoso instance running at port 8891: 17765873 triples
  - Virtuoso instance running at port 8893: 27897291 triples
  - Virtuoso instance running at port 8895: 16956 triples
  - Virtuoso instance running at port 8899: 72372256 triples

Right, I've already changed the ini files according to your suggestions:
- removing ServerThreads
- reducing MaxClientConnections to 200
- reducing MaxQueryMem
- setting AdjustVectorSize to 0

For profiling I shrinked the test queries such that it is probably easier to 
find out the problem. By doing so, I came to the assumption that not the 
FILTERS causes the long query evaluation time, but the size of the result set.
If I run q1:
   SELECT ?x ?b
   WHERE {
  SERVICE <http://141.87.4.8:8891/sparql> {
 ?x <http://swrc.ontoware.org/ontology#edition> ?b .
  } .
   }
at lets say Virtuoso instance running at port 8895 it will end up within 2 
minutes. This is because the service call will return only 32562 results.

But if I run q2:
   SELECT ?x ?b
   WHERE {
  SERVICE <http://141.87.4.8:8891/sparql> {
 ?x <http://swrc.ontoware.org/ontology#pages> ?b .
  } .
   }
where the service call returns 313421 results, the query evaluation on the same 
instance (running at port 8895) will take something around 1 hour!

Do you have any explanation for that or any suggestion how such long evaluation 
times can be fixed?

If you are still interested in explain and profile plans of those queries, 
please find it at 
https://www.dropbox.com/s/ein0hxhtgk85ci9/analysis_files.zip?dl=0.
Please notice that *_federated means the queries are like above and are 
evaluated at the Virtuoso instance running at port 8895, and *_local means the 
evaluation of these queries (without SERVICE) at the Virtuoso instance running 
at port 8891.

Best regards
Andy



-Ursprüngliche Nachricht-
Von: Hugh Williams [mailto:hwilli...@openlinksw.com] 
Gesendet: Sonntag, 28. Februar 2016 22:21
An: Nolle, Andreas <no...@hs-albsig.de>
Cc: virtuoso-users@lists.sourceforge.net
Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying

Hi Andreas,

Generally there should be no create additional indexes as the 2 Full & 3 
partial indexes have been found to be sufficient for most use cases, with only 
one use case encountered where an addtional partial index was required as 
detailed at:


http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning#Index%20Scheme%20Selection

Thus what indexes (STATISTICS DB.DBA.RDF_QUAD;) are created on these various 
instances you have are they all the same or do they have different indexes as 
the more indexes the more large the database and hence more memory required for 
hosting it. Also are the triple counts the same on all these instances ?

Did you make any of the other INI file changes in my previous email, certainly 
“AdjustVectorSize” should be set to 0 ?

You should also consider profiling the queries to determine if the best quey 
plan is being used, which can be done with the “profile()” function as detailed 
at:


http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksAanalyzingSPARQLQuery
http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#querylogging  - 
Query Logging 

Note there is are also some INI file params that can be set to control query 
optimisation ie plans as if a bad plan is being chosen then these options can 
in some cases enable better plans to be chosen, see:


http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtQueryOptDiagnostic

If you can provide query plans, database statistics then these can be analysed 
to trying and determine the cause of long running queries …

It certainly would be interesting to see how the query plans and database stats 
vary being the instance where the query runs in msecs and the other were it 
runs in 40+mins ...

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers

--
Site24x7 APM Insight: Get Deep Visibility into Applic

Re: [Virtuoso-users] infrequent errors on parallel querying

[Hugh Williams]

Hi Andreas,

Generally there should be no create additional indexes as the 2 Full & 3 
partial indexes have been found to be sufficient for most use cases, with only 
one use case encountered where an addtional partial index was required as 
detailed at:


http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning#Index%20Scheme%20Selection

Thus what indexes (STATISTICS DB.DBA.RDF_QUAD;) are created on these various 
instances you have are they all the same or do they have different indexes as 
the more indexes the more large the database and hence more memory required for 
hosting it. Also are the triple counts the same on all these instances ?

Did you make any of the other INI file changes in my previous email, certainly 
“AdjustVectorSize” should be set to 0 ?

You should also consider profiling the queries to determine if the best quey 
plan is being used, which can be done with the “profile()” function as detailed 
at:


http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksAanalyzingSPARQLQuery
http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#querylogging  - 
Query Logging 

Note there is are also some INI file params that can be set to control query 
optimisation ie plans as if a bad plan is being chosen then these options can 
in some cases enable better plans to be chosen, see:


http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtQueryOptDiagnostic

If you can provide query plans, database statistics then these can be analysed 
to trying and determine the cause of long running queries …

It certainly would be interesting to see how the query plans and database stats 
vary being the instance where the query runs in msecs and the other were it 
runs in 40+mins ...

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



> On 28 Feb 2016, at 14:58, Nolle, Andreas <no...@hs-albsig.de> wrote:
> 
> Dear Hugh,
> 
> many thanks for your reply!
> Sorry, I forgot to mentioned that I increased up to 24 core and 228 GB memory.
> 
> I'm still on trying to optimize the performance and to avoid transaction 
> deadlocks... but unfortunately without any success so far...
> 
> I've already tried to create some additional indexes, like PSOG, but the 
> evaluation of some filter parts take in some cases really long (more than 40 
> minutes)...
> 
> For me it is also very strange why the evaluation of query
> 
> define input:default-graph-uri <http://clashsniffer.hs-albsig.de/benchset>
> PREFIX xsd:  <http://www.w3.org/2001/XMLSchema#>
> PREFIX owl: <http://www.w3.org/2002/07/owl#>
> PREFIX quest:   <http://obda.org/quest#>
> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
> PREFIX rdfs:<http://www.w3.org/2000/01/rdf-schema#>
> 
> SELECT ?x ?b
> WHERE {
> ?x <http://purl.org/ontology/bibo/volume> ?b .
> FILTER ( ?x >= <http://eprints.wmin.ac.uk/id/eprint/1620> ) .
> FILTER ( ?x < <http://pub.epsilon.slu.se/id/eprint/11557> ) .
> }
> 
> at instance running at port 8895 will take only some ms, but executing query
> 
> define input:default-graph-uri <http://clashsniffer.hs-albsig.de/benchset> 
> PREFIX xsd:  <http://www.w3.org/2001/XMLSchema#>
> PREFIX owl: <http://www.w3.org/2002/07/owl#>
> PREFIX quest:   <http://obda.org/quest#>
> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
> PREFIX rdfs:<http://www.w3.org/2000/01/rdf-schema#>
> 
> SELECT ?x ?b
> WHERE {
>  SERVICE <http://141.87.4.8:8895/sparql> {
> ?x <http://purl.org/ontology/bibo/volume> ?b .
> FILTER ( ?x >= <http://eprints.wmin.ac.uk/id/eprint/1620> ) .
> FILTER ( ?x < <http://pub.epsilon.slu.se/id/eprint/11557> ) .
>  } .
> }
> 
> at another instance is running possibly indefinitely... (stopped the 
> evaluation after 45 minutes).
> 
> Best
> Andy
> 
> 
> 
> -Ursprüngliche Nachricht-
> Von: Hugh Williams [mailto:hwilli...@openlinksw.com] 
> Gesendet: Sonntag, 28. Februar 2016 04:14
> An: Nolle, Andreas <no...@hs-albsig.de>
> Cc: virtuoso-users@lists.sourceforge.net
> Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying
> 
> Hi Andreas,
> 
> In you INI file “MaxClientConnections” and “ServerThreads” are the same thing 
> the latter be

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Dear Hugh,

many thanks for your reply!
Sorry, I forgot to mentioned that I increased up to 24 core and 228 GB memory.

I'm still on trying to optimize the performance and to avoid transaction 
deadlocks... but unfortunately without any success so far...

I've already tried to create some additional indexes, like PSOG, but the 
evaluation of some filter parts take in some cases really long (more than 40 
minutes)...

For me it is also very strange why the evaluation of query

define input:default-graph-uri <http://clashsniffer.hs-albsig.de/benchset>
PREFIX xsd:  <http://www.w3.org/2001/XMLSchema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX quest:   <http://obda.org/quest#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs:<http://www.w3.org/2000/01/rdf-schema#>

SELECT ?x ?b
WHERE {
  ?x <http://purl.org/ontology/bibo/volume> ?b .
  FILTER ( ?x >= <http://eprints.wmin.ac.uk/id/eprint/1620> ) .
  FILTER ( ?x < <http://pub.epsilon.slu.se/id/eprint/11557> ) .
}

at instance running at port 8895 will take only some ms, but executing query

define input:default-graph-uri <http://clashsniffer.hs-albsig.de/benchset> 
PREFIX xsd:  <http://www.w3.org/2001/XMLSchema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX quest:   <http://obda.org/quest#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs:<http://www.w3.org/2000/01/rdf-schema#>

SELECT ?x ?b
WHERE {
   SERVICE <http://141.87.4.8:8895/sparql> {
  ?x <http://purl.org/ontology/bibo/volume> ?b .
  FILTER ( ?x >= <http://eprints.wmin.ac.uk/id/eprint/1620> ) .
  FILTER ( ?x < <http://pub.epsilon.slu.se/id/eprint/11557> ) .
   } .
}

at another instance is running possibly indefinitely... (stopped the evaluation 
after 45 minutes).

Best
Andy



-Ursprüngliche Nachricht-
Von: Hugh Williams [mailto:hwilli...@openlinksw.com] 
Gesendet: Sonntag, 28. Februar 2016 04:14
An: Nolle, Andreas <no...@hs-albsig.de>
Cc: virtuoso-users@lists.sourceforge.net
Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying

Hi Andreas,

In you INI file “MaxClientConnections” and “ServerThreads” are the same thing 
the latter being the original name which is retained for compatibility reasons. 
Why do you set these params to 1000 on both sections of the INI file ? I would 
reduce them to some thing like 200 as especially the HTTP Server ones are 
pre-allocated on server startup which will consume significant server resources 
for these threads.

Looking at the Vectored execution INI file params you have set:

MaxQueryMem  = 32G  ; memory allocated to query processor
VectorSize   = 1000 ; initial parallel query vector (array of query 
operations) size
MaxVectorSize= 100  ; query vector size threshold.
AdjustVectorSize = 1
ThreadsPerQuery  = 24

Why is MaxQueryMem set so high ie 32G as this will all consume significant 
system memory when multiple queries are being run, thus I would set it to 
something like the 2G default to start off with. How much memory is available 
on the system ?

AdjustVectorSize should be set to 0 in most use cases …

I assume you machine has 24 cores which is why you set "ThreadsPerQuery 
 = 24” ?

See the following documentation on vectored query execution:

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#confvectexec - 
Configuring Vectored Execution

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#tunparamsmworkload  - 
Tuning Parameters for Multiuser Workloads

and the following on Transaction deadlocks and how to determine and avoid them:

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#perfdiag - 
Performance diagnostics 

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#TRANSACTION_ISOLATION_LEVELS
 - Transaction Metrics, Diagnostics and Optimization

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



> On 26 Feb 2016, at 08:03, Nolle, Andreas <no...@hs-albsig.de> wrote:
> 
> The critical error is now that after some query evaluations I get a lot of 
> "Virtuoso 40001 Error SR172: Transaction deadlocked" errors. For me this is 
> really strange because I've set the DefaultIsolation = 2 and only SELECT 
> queries are executed. Please notice that I've slightly changed some 
> parameters in the ini files.
> 
> It would be really nice i

Re: [Virtuoso-users] infrequent errors on parallel querying

[Hugh Williams]

Hi Andreas,

In you INI file “MaxClientConnections” and “ServerThreads” are the same thing 
the latter being the original name which is retained for compatibility reasons. 
Why do you set these params to 1000 on both sections of the INI file ? I would 
reduce them to some thing like 200 as especially the HTTP Server ones are 
pre-allocated on server startup which will consume significant server resources 
for these threads.

Looking at the Vectored execution INI file params you have set:

MaxQueryMem  = 32G  ; memory allocated to query processor
VectorSize   = 1000 ; initial parallel query vector (array of query 
operations) size
MaxVectorSize= 100  ; query vector size threshold.
AdjustVectorSize = 1
ThreadsPerQuery  = 24

Why is MaxQueryMem set so high ie 32G as this will all consume significant 
system memory when multiple queries are being run, thus I would set it to 
something like the 2G default to start off with. How much memory is available 
on the system ?

AdjustVectorSize should be set to 0 in most use cases …

I assume you machine has 24 cores which is why you set "ThreadsPerQuery 
 = 24” ?

See the following documentation on vectored query execution:

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#confvectexec - 
Configuring Vectored Execution

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#tunparamsmworkload  - 
Tuning Parameters for Multiuser Workloads

and the following on Transaction deadlocks and how to determine and avoid them:

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#perfdiag - 
Performance diagnostics 

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#TRANSACTION_ISOLATION_LEVELS
 - Transaction Metrics, Diagnostics and Optimization

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



> On 26 Feb 2016, at 08:03, Nolle, Andreas <no...@hs-albsig.de> wrote:
> 
> The critical error is now that after some query evaluations I get a lot of 
> "Virtuoso 40001 Error SR172: Transaction deadlocked" errors. For me this is 
> really strange because I've set the DefaultIsolation = 2 and only SELECT 
> queries are executed. Please notice that I've slightly changed some 
> parameters in the ini files.
> 
> It would be really nice if you could help me on that and give me some 
> suggestions how transaction errors can be avoided on evaluating SELECT 
> queries. Please find the current log and ini files of each Virtuoso instance 
> as well as the corresponding output of status() at 
> https://www.dropbox.com/s/v6hvw7j7uz9r9uq/virtuoso_files.zip?dl=0
> 
> Best
> Andy
> 
> 
> -Ursprüngliche Nachricht-
> Von: Nolle, Andreas [mailto:no...@hs-albsig.de] 
> Gesendet: Freitag, 26. Februar 2016 09:00
> An: Hugh Williams <hwilli...@openlinksw.com>
> Cc: virtuoso-users@lists.sourceforge.net
> Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying
> 
> Hi Hugh,
> 
> in the meantime I solved some of the errors. Assuming the following query is 
> evaluated at endpoint running at port 8893:
> 
> SELECT ?x ?a ?b ?P0src ?P1src
> WHERE {
>  {
> SERVICE <http://141.87.4.8:8899/sparql> {
>?x <http://purl.org/dc/terms/partOf> ?a .
>FILTER ( ?x >= 
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp> ) .
>FILTER ( ?x < 
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp> ) .
> } .
> BIND (<http://141.87.4.8:8899/sparql> AS ?P0src)
>  }
>  UNION
>  {
> SERVICE <http://141.87.4.8:8891/sparql> {
>?x <http://purl.org/dc/terms/partOf> ?a .
>FILTER ( ?x >= 
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp> ) .
>FILTER ( ?x < 
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp> ) .
> } .
> BIND (<http://141.87.4.8:8891/sparql> AS ?P0src)
>  }
>  UNION
>  {
> ?x <http://purl.org/dc/terms/partOf> ?a .
> FILTER ( ?x >= 
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp> ) .
> FILTER ( ?x < 
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp> ) .
> BIND (<http://141.87.4.8:8893/sparql> AS ?P0src)
>  }
>  UNION
>  {
> SERVICE <http://141.87.4.8:8895/sparql> {
>?x <

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

The critical error is now that after some query evaluations I get a lot of 
"Virtuoso 40001 Error SR172: Transaction deadlocked" errors. For me this is 
really strange because I've set the DefaultIsolation = 2 and only SELECT 
queries are executed. Please notice that I've slightly changed some parameters 
in the ini files.

It would be really nice if you could help me on that and give me some 
suggestions how transaction errors can be avoided on evaluating SELECT queries. 
Please find the current log and ini files of each Virtuoso instance as well as 
the corresponding output of status() at 
https://www.dropbox.com/s/v6hvw7j7uz9r9uq/virtuoso_files.zip?dl=0

Best
Andy


-Ursprüngliche Nachricht-
Von: Nolle, Andreas [mailto:no...@hs-albsig.de] 
Gesendet: Freitag, 26. Februar 2016 09:00
An: Hugh Williams <hwilli...@openlinksw.com>
Cc: virtuoso-users@lists.sourceforge.net
Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying

Hi Hugh,

in the meantime I solved some of the errors. Assuming the following query is 
evaluated at endpoint running at port 8893:

SELECT ?x ?a ?b ?P0src ?P1src
WHERE {
   {
  SERVICE <http://141.87.4.8:8899/sparql> {
 ?x <http://purl.org/dc/terms/partOf> ?a .
 FILTER ( ?x >= 
<http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp> ) .
 FILTER ( ?x < 
<http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp> ) .
  } .
  BIND (<http://141.87.4.8:8899/sparql> AS ?P0src)
   }
   UNION
   {
  SERVICE <http://141.87.4.8:8891/sparql> {
 ?x <http://purl.org/dc/terms/partOf> ?a .
 FILTER ( ?x >= 
<http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp> ) .
 FILTER ( ?x < 
<http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp> ) .
  } .
  BIND (<http://141.87.4.8:8891/sparql> AS ?P0src)
   }
   UNION
   {
  ?x <http://purl.org/dc/terms/partOf> ?a .
  FILTER ( ?x >= 
<http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp> ) .
  FILTER ( ?x < 
<http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp> ) .
  BIND (<http://141.87.4.8:8893/sparql> AS ?P0src)
   }
   UNION
   {
  SERVICE <http://141.87.4.8:8895/sparql> {
 ?x <http://purl.org/dc/terms/partOf> ?a .
 FILTER ( ?x >= 
<http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp> ) .
 FILTER ( ?x < 
<http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp> ) .
  } .
  BIND (<http://141.87.4.8:8895/sparql> AS ?P0src)
  BIND (<http://141.87.4.8:8895/sparql> AS ?P0src)
   } .
   ?b <http://www.aktors.org/ontology/portal#article-of-journal> ?x .
   FILTER ( ?x >= 
<http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp> ) .
   FILTER ( ?x < <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp> ) 
.
   BIND (<http://141.87.4.8:8893/sparql> AS ?P1src) }

If this endpoint has no ?x matching pattern at line 22 of this query, an 
exception is thrown. It is possible to avoid that by replacing line 20 to 26 by

UNION
   {
  {
 ?x <http://purl.org/dc/terms/partOf> ?a .
 FILTER ( ?x >= 
<http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp> ) .
 FILTER ( ?x < 
<http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp> ) .
  }
  BIND (<http://141.87.4.8:8893/sparql> AS ?P0src)
   }

Best
Andy

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + 
Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end 
web transactions and take corrective actions now Troubleshoot faster and 
improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Hi Hugh,

in the meantime I solved some of the errors. Assuming the following query is 
evaluated at endpoint running at port 8893:

SELECT ?x ?a ?b ?P0src ?P1src
WHERE {
   {
  SERVICE  {
 ?x  ?a .
 FILTER ( ?x >= 
 ) .
 FILTER ( ?x < 
 ) .
  } .
  BIND ( AS ?P0src)
   }
   UNION
   {
  SERVICE  {
 ?x  ?a .
 FILTER ( ?x >= 
 ) .
 FILTER ( ?x < 
 ) .
  } .
  BIND ( AS ?P0src)
   }
   UNION
   {
  ?x  ?a .
  FILTER ( ?x >= 
 ) .
  FILTER ( ?x < 
 ) .
  BIND ( AS ?P0src)
   }
   UNION
   {
  SERVICE  {
 ?x  ?a .
 FILTER ( ?x >= 
 ) .
 FILTER ( ?x < 
 ) .
  } .
  BIND ( AS ?P0src)
  BIND ( AS ?P0src)
   } .
   ?b  ?x .
   FILTER ( ?x >= 
 ) .
   FILTER ( ?x <  ) 
.
   BIND ( AS ?P1src)
}

If this endpoint has no ?x matching pattern at line 22 of this query, an 
exception is thrown. It is possible to avoid that by replacing line 20 to 26 by

UNION
   {
  {
 ?x  ?a .
 FILTER ( ?x >= 
 ) .
 FILTER ( ?x < 
 ) .
  }
  BIND ( AS ?P0src)
   }

Best
Andy

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Hugh Williams]

HI Andreas,

The  “missed delete of name id cache…”  messages can be ignored as they are 
more informative and should not be in the default log, so will be moved to a 
higher or more verbose log level …

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



> On 24 Feb 2016, at 15:05, Nolle, Andreas  wrote:
> 
> Hi Hugh,
>  
> in addition to my last mail I have the following question:
> Do the logs “missed delete of name id cache…” have any influence on the 
> completeness of query results?
>  
> Best
> Andy
>  
>  
>  
> Von: Nolle, Andreas 
> Gesendet: Dienstag, 23. Februar 2016 14:42
> An: 'Hugh Williams' 
> Cc: virtuoso-users@lists.sourceforge.net
> Betreff: AW: [Virtuoso-users] infrequent errors on parallel querying
>  
> Hi Hugh,
>  
> the query workload is almost equally distributed.
>  
> The errors in the log regarding the lock file and “missed delete of name id 
> cache” occurred both not before updating to Version 7.2.2.1. Maybe some 
> indexes aren’t updated…
>  
> The status of the Virtuoso instance running at Port 8895 seems to remain 
> indefinitely.
>  
> Only the issue regarding memory leaks doesn’t occurred (until now) since I 
> updated to the new version. Don’t know if this is solved…
>  
> By the way, I have an additional question:
> Since I have a lot of queries comprising parts like
>  ?x  ?a .
>  FILTER ( ?x >= 
>  ) .
>  FILTER ( ?x < 
>  ) .
> Is it possible to speed up the evaluation time of those queries, e.g. by 
> enable some additional indexing?
>  
> Cheers
> Andy



smime.p7s
Description: S/MIME cryptographic signature
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Nolle, Andreas]

Hi Hugh,

in addition to my last mail I have the following question:
Do the logs “missed delete of name id cache…” have any influence on the 
completeness of query results?

Best
Andy



Von: Nolle, Andreas
Gesendet: Dienstag, 23. Februar 2016 14:42
An: 'Hugh Williams' 
Cc: virtuoso-users@lists.sourceforge.net
Betreff: AW: [Virtuoso-users] infrequent errors on parallel querying

Hi Hugh,

the query workload is almost equally distributed.

The errors in the log regarding the lock file and “missed delete of name id 
cache” occurred both not before updating to Version 7.2.2.1. Maybe some indexes 
aren’t updated…

The status of the Virtuoso instance running at Port 8895 seems to remain 
indefinitely.

Only the issue regarding memory leaks doesn’t occurred (until now) since I 
updated to the new version. Don’t know if this is solved…

By the way, I have an additional question:
Since I have a lot of queries comprising parts like
 ?x  ?a .
 FILTER ( ?x >= 
 ) .
 FILTER ( ?x < 
 ) .
Is it possible to speed up the evaluation time of those queries, e.g. by enable 
some additional indexing?

Cheers
Andy


--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] infrequent errors on parallel querying

[Hugh Williams]

t;= 
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp 
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) .
>  FILTER ( ?x < 
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp 
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) .
>   } .
>   BIND (<http://141.87.4.8:8899/sparql <http://141.87.4.8:8899/sparql>> 
> AS ?P0src)
>}
>UNION
>{
>   SERVICE <http://141.87.4.8:8891/sparql <http://141.87.4.8:8891/sparql>> 
> {
>  ?x <http://purl.org/dc/terms/partOf 
> <http://purl.org/dc/terms/partOf>> ?a .
>  FILTER ( ?x >= 
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp 
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) .
>  FILTER ( ?x < 
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp 
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) .
>   } .
>   BIND (<http://141.87.4.8:8891/sparql <http://141.87.4.8:8891/sparql>> 
> AS ?P0src)
>}
>UNION
>{
>   ?x <http://purl.org/dc/terms/partOf <http://purl.org/dc/terms/partOf>> 
> ?a .
>   FILTER ( ?x >= 
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp 
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) .
>   FILTER ( ?x < 
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp 
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) .
>   BIND (<http://141.87.4.8:8893/sparql <http://141.87.4.8:8893/sparql>> 
> AS ?P0src)
>}
>UNION
>{
>   SERVICE <http://141.87.4.8:8895/sparql <http://141.87.4.8:8895/sparql>> 
> {
>  ?x <http://purl.org/dc/terms/partOf 
> <http://purl.org/dc/terms/partOf>> ?a .
>  FILTER ( ?x >= 
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp 
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) .
>  FILTER ( ?x < 
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp 
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) .
>   } .
>   BIND (<http://141.87.4.8:8895/sparql <http://141.87.4.8:8895/sparql>> 
> AS ?P0src)
>   BIND (<http://141.87.4.8:8895/sparql <http://141.87.4.8:8895/sparql>> 
> AS ?P0src)
>} .
>?b <http://www.aktors.org/ontology/portal#article-of-journal 
> <http://www.aktors.org/ontology/portal#article-of-journal>> ?x .
>FILTER ( ?x >= 
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp 
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) .
>FILTER ( ?x < <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp 
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) .
>BIND (<http://141.87.4.8:8893/sparql <http://141.87.4.8:8893/sparql>> AS 
> ?P1src)
> }
>  
> where the filter parts are generated beforehand by querying all individuals 
> of a specific concept piece by piece for each 1048576 results.
>  
> The error that occur is usually:
> Virtuoso HTCLI Error HC001: Read Error in HTTP Client
>  
> But as I remember other errors that occurred during several tests over the 
> last weeks are:
> -No Http Response
> -Transaction aborted
> -Timeout
> -target server failed to respond
> -…
>  
> Please find the output of status() for each Virtuoso instance (running at 
> port 8891, 8893, 8895, 8899) as well as the corresponding log in the zip file 
> at https://www.dropbox.com/s/yzmrybj7jf8m8az/files.zip?dl=0 
> <https://www.dropbox.com/s/yzmrybj7jf8m8az/files.zip?dl=0>. The call of 
> status() was done after I received a http error from Virtuoso instance 8893. 
> For the same Virtuoso instance (running at port 8893) I got a timeout on 
> connecting by isql. Only after a restart it was possible to connect via isql.
>  
> The reason why I assign only 16 GB per Virtuoso instance (4 x 16 GB in total) 
> is that each instance is take still more than 16 GB after some hours or days. 
> In my case I run up to 25000 queries (like the one above) at each Virtuoso 
> instance, but maximal 16 at one time over all instances. If you look at the 
> memory usage it starts normally but it increases over time as long as the 
> swap space is overcrowded… If this will be the case my code (running on 
> another server) restarts the whole server hosting the Virtuoso instances 

Re: [Virtuoso-users] infrequent errors on parallel querying

[Hugh Williams]

Hi Andy,

What is a typical query being executing that is result in a HTTP error and what 
is the actual error ?

When the error occurs what is the status of the server at that point , please 
provide the output of running the command:

status();

Are any errors reported in the “virtuoos.log” file, please provide a copy for 
review ?

You indicate having a 128GB RAM machine but memory allocated to Virtuoso is  
that for a 16GB RAM machine ie NumberOfBuffer = 136 in the INI file, was 
there a specific reason for this ?

How many triples are hosted in the Virtuoso instance , please provide the 
output of the count query:

select count(*) where {?s ?p ?o}

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



> On 20 Feb 2016, at 15:47, Nolle, Andreas  wrote:
> 
> Hi folks,
>  
> I’ve set up four Virtuoso instances (Version 7.2.2.3215) on an Ubuntu 14.04.4 
> LTS server with 12 cores and 128 GB memory.
> If I run several (federated) queries in parallel (maximal 16 at a time), 
> where some of the queries may take several minutes, it may happened that a 
> SPARQL endpoint return some HTTP error. If I rerun the same query some 
> seconds later, it will ordinarily be evaluated correct and the requested 
> results are returned.
> I’ve already tried to solve this infrequent issue but without any success. 
> Can you please give me some recommendations?
> Maybe my configuration may comprise some not well-chosen parameter?
>  
>  
> Please find below my virtuoso.ini:
>  
> [Database]
> DatabaseFile   = ../var/lib/virtuoso/db/virtuoso.db
> ErrorLogFile   = ../var/lib/virtuoso/db/virtuoso.log
> LockFile   = ../var/lib/virtuoso/db/virtuoso.lck
> TransactionFile= ../var/lib/virtuoso/db/virtuoso.trx
> xa_persistent_file = ../var/lib/virtuoso/db/virtuoso.pxa
> ErrorLogLevel  = 7
> FileExtend = 200
> MaxCheckpointRemap = 15
> Striping   = 0
> TempStorage= TempDatabase
>  
> [TempDatabase]
> DatabaseFile   = ../var/lib/virtuoso/db/virtuoso-temp.db
> TransactionFile= ../var/lib/virtuoso/db/virtuoso-temp.trx
> MaxCheckpointRemap = 9000
> Striping   = 0
>  
> [Parameters]
> ServerPort   = 1112
> LiteMode = 0
> DisableUnixSocket= 1
> DisableTcpSocket = 0
> MaxClientConnections = 1
> ServerThreads= 1000
> CheckpointInterval   = 60
> O_DIRECT = 0
> CaseMode = 2
> MaxStaticCursorRows  = 5000
> CheckpointAuditTrail = 0
> AllowOSCalls = 0
> SchedulerInterval= 10
> DirsAllowed  = ., ../share/virtuoso/vad,../saleem/dumps, 
> /opt/benchset/bibsonomy
> ThreadCleanupInterval= 1
> ThreadThreshold  = 1000
> ResourcesCleanupInterval = 1
> FreeTextBatchSize= 10
> SingleCPU= 0
> VADInstallDir= ../share/virtuoso/vad/
> PrefixResultNames= 0
> RdfFreeTextRulesSize = 100
> IndexTreeMaps= 512
> MaxMemPoolSize   = 2
> PrefixResultNames= 0
> MacSpotlight = 0
> MaxQueryMem  = 8G; memory allocated to query processor
> VectorSize   = 1000 ; initial parallel query vector 
> (array of query operations) size
> MaxVectorSize= 200 ; query vector size threshold.
> AdjustVectorSize = 0
> ThreadsPerQuery  = 24
> AsyncQueueMaxThreads = 36
> NumberOfBuffers  = 136
> MaxDirtyBuffers  = 100
> TraceOn  = errors
> CallstackOnException = 1
> MaxSortedTopRows = 1000
> DefaultIsolation   = 2
>  
> [HTTPServer]
> ServerPort  = 8891
> ServerRoot  = ../var/lib/virtuoso/vsp
> MaxClientConnections= 1
> DavRoot = DAV
> EnabledDavVSP   = 0
> HTTPProxyEnabled= 0
> TempASPXDir = 0
> DefaultMailServer   = localhost:25
> ServerThreads   = 1000
> MaxKeepAlives   = 100
> KeepAliveTimeout= 10
> MaxCachedProxyConnections   = 10
> ProxyConnectionCacheTimeout = 15
> HTTPThreadSize  = 28
> HttpPrintWarningsInOutput   = 0
> Charset = UTF-8
> MaintenancePage = atomic.html
> EnabledGzipContent  = 1
>  
> [AutoRepair]
> BadParentLinks = 0
>  
> [Client]
> SQL_PREFETCH_ROWS  = 100
> SQL_PREFETCH_BYTES = 16000
> SQL_QUERY_TIMEOUT  = 0
> SQL_TXN_TIMEOUT= 0
>  
> [VDB]
> ArrayOptimization   = 0
>