Setting the timeout is a good idea but there are a couple of things in
Rajiv's message that suggest that is not all that is going on.
We have observed that thread count for the Fuseki process is increasing
with time.
> I know there might be code issue and resources should be
> closed properly
Are we talking about SELECT queries?
How many concurrent users of the application side are there?
And are the results quite large (larger than TCP buffering)?
The only way I can think of, at the moment, is if the client is keeping
the TCP connection open and the TCP connection filling up, causing
backup at the server. Given, you can calling it from Tomcat, is the
Tomcat application code long lived?
If you can attach a java debugger to the server, could you tell me what
the threads are doing (or not doing)?
If the client is consuming the results very slowly, or jumping out of
the ResultSet loop prematurely, then the server has no way of knowing
whether the client application code has really finished or is going to
come back later.
The pattern:
QueryExecution exec = QueryExecutionFactory.create(q,ds) ;
try {
ResultSet rs = exec.execSelect() ;
while(rs.hasNext()) {
return ...
}
} finally {
exec.close();
}
catches that.
Another approach is
ResultSet rs = exec.execSelect() ;
rs = ResultSetFactory.copyResults(rs) ;
which forces the ARQ code to pull in all the results, which than
releases the connection back tot he local connection pool.
It does not matter whether we're talking streaming or if Fuseki gets all
the query results, then sent the results (which it doesn't do - it
streams when possible) but it does make this vector more likely.
To defend the server, Brian's suggestion of a reverse proxy gives you a
control point. Rather than put every mechanism needed into Fuseki,
keeping it focues on beign a SPARQL engine and using well-known,
well-engineered other components.
The default configuration of Fuseki is to use Jetty's
BlockingChannelConnector but it is possible to complete take control of
the jetty configuration with --jetty-config.
(SelectChannelConnector might have been better but it has been reported
to be unstable on OS/X for Fuseki's usage patterns)
It looks like an (unintentional) "Slow HTTP" DOS attack but on the
response consumption side, not the request sending.
https://community.qualys.com/blogs/securitylabs/2011/11/02/how-to-protect-against-slow-http-attacks
The client OS will still be ack'ing the TCP connection and may do so
forever from a server-style app like from Tomcat.
Setting the query time may reduce the general effect but it will not
free up threads. A query timeout is a graceful abort of the query - the
query gets a chance to clean up and that needs the query execution in
the server getting called.
Actually killing threads in Java without the threads involvement is bad.
http://docs.oracle.com/javase/1.5.0/docs/guide/misc/threadPrimitiveDeprecation.html
Andy
On 12/12/13 09:33, Brian McBride wrote:
Hi Rajiv,
You can find how to set query timeouts in the Fuseki documentation at [1]
To set server wide timeout:
[] rdf:type fuseki:Server ;
#Server-wide context parameters can be given here.
#For example, to set query timeouts: on a server-wide basis:
#Format 1: "1000"-- 1second timeout
#Format 2: "10000,60000"-- 10s timeout to first result,
then 60s timeout to for rest of query.
#See java doc for ARQ.queryTimeout
#ja:context [ ja:cxtName "arq:queryTimeout"; ja:cxtValue
"10000"] ;
Uncomment the last line and set the value you require per the comments
above.
Do you have a reverse proxy configured to limit the rate at which
requests are going to Fuseki? Can you confirm that all your requests
are going through that proxy?
Brian
[1]
http://jena.apache.org/documentation/serving_data/#running-a-fuseki-server
On 11/12/2013 19:47, Chaudhuri, Rajiv wrote:
Hi,
Our application which is running at Tomcat server is making query to
Fuseki
TDB store using HTTP protocol.
We have observed that thread count for the Fuseki process is increasing
with time.
I know there might be code issue and resources should be closed properly.
*But we would like to achieve the same using configuration at Fuseki
server- i.e. If the query takes too much time (which means that
resource is
not closed by application server) then there should be time out and
Fuseki
server should implicitly kill the thread.*
The impact of this thread count getting increased is Fuseki server stop
responding and process gets kill automatically after sometimes and we
have
to restart the Fuseki server again.
