On 4/19/13 3:49 AM, Jerven Bolleman wrote:
Forgot reply all-------- Original Message --------Subject: Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.Date: Thu, 18 Apr 2013 23:21:46 +0200 From: Jerven Bolleman <[email protected]> To: Rob Warren <[email protected]> Hi Rob, There is a fundamental problem with HTTP status codes. Lets say a user submits a complex but small sparql request. My server sees the syntax is good and starts to reply in good faith. This means the server starts the http response and sends an 200 OK Some results are being send.... However, during the evaluation the server gets an exception. What to do? I can't change the status code anymore...
Depends.If you have OFFSET and LIMIT in use, you can reflect the new state of affairs when the next GET is performed i.e, lets say you have OFFSET 20 and LIMIT 20, the URL with OFFSET 40 is the request for the next batch of results from the solution and the one that would reflect the new state of affairs.
Not really, not if the configuration granularity is there. Also remember, parsing, solution preparation, and actual data retrieval are distinct tasks. Also when dealing with aggregates, if you have key compression and actual horizontal partitioning of aggregates across a cluster, the time to solution shrinks, as we've demonstrated across both versions 7 and much more so with with version 7 where column storage enables much more compactness of data in general.Waiting until server know the query can be answered is not feasible because that would mean the server can't start giving replies as soon as possible. Which likely leads to connection timeouts.
Using HTTP status codes when responses are likely to be larger than 1 MB works badly in practice.
See my earlier comments about retrieval being distinct from solution preparation. Fetching the data via OFFSET and LIMIT based sparql-protocol URLs does enable you address this issue. The same issue used to exist with SQL RDBMS based data access via ODBC, JDBC etc.. each of those APIs separate query solution preparation from actual data retrieval.
When we separate components the right way a lot can be achieved. Links: 1. http://bit.ly/WteWYI -- Virtuoso 7.0 Column Store 2. http://bit.ly/17oSWk9 -- VLDB 2009 Tutorial on Column Stores3. http://bit.ly/14ULX2F -- LOD2 benchmark report for BSBM as 50 & 150 Billion Triples scales (achieved as a result of the use of Column Storage, Key Compressed, and Vectored query execution).
Kingsley
Regards, Jerven On Apr 18, 2013, at 10:53 PM, Rob Warren wrote:On 18-Apr-13, at 8:53 AM, Jerven Bolleman wrote:Many of the current public SPARQL endpoints limit all their users to queries of limited CPU time. But this is not enough to really manage (mis) use of an endpoint. Also the SPARQL api being http based suffers from the problem that we first send the status code and may only find out later that we can'tanswer the query after all. Leading to a 200 not OK problem :(Jerven,I agree that a 200 reply to 'query too complex', 'query too big' or 'query timeout' is not acceptable. However, limits on queries are a tool to keep dumb clients from pounding on the server too hard.A standardized reply / error would be something that I would like to see in that it allows the client to modify its approach to querying the server. It would also be an opportunity to have the server signal to the client what trade-off it is willing to make between sending more triples and increasing the query complexity.Could '413 Request Entity Too Large', '429 Too Many Requests' and '453 Not Enough Bandwidth' be abused here for Sparql endpoints?rhw------------------------------------------------------------------- Jerven Bolleman [email protected] SIB Swiss Institute of Bioinformatics Tel: +41 (0)22 379 58 85 CMU, rue Michel Servet 1 Fax: +41 (0)22 379 58 58 1211 Geneve 4, Switzerland www.isb-sib.ch - www.uniprot.org Follow us at https://twitter.com/#!/uniprot -------------------------------------------------------------------
-- Regards, Kingsley Idehen Founder & CEO OpenLink Software Company Web: http://www.openlinksw.com Personal Weblog: http://www.openlinksw.com/blog/~kidehen Twitter/Identi.ca handle: @kidehen Google+ Profile: https://plus.google.com/112399767740508618350/about LinkedIn Profile: http://www.linkedin.com/in/kidehen
smime.p7s
Description: S/MIME Cryptographic Signature
