[
https://issues.apache.org/jira/browse/JENA-515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742366#comment-13742366
]
Andy Seaborne commented on JENA-515:
------------------------------------
You say it's a batch of 1000 resources - that does not sound very big but
Fuseki is very cautious about validating data before (more memory consumed).
Are you running with the standard heap size configured? Is this a 32 bit or 64
bit setup? How many triples is a batch?
Out of memory is a global condition that isn't caused by one operation
specifically, it's the outcome of all the operations currently, and recently,
executed.
The worst case is that the writing of data from the journal to the main
database may be affected. While that process itself is not a consumer of
memory to any significant degree (over and above the material already
in-memory), some other combination of request at the same time can cause the
JVM to run out of memory. The server is compromised at this point.
Some special cases could be recovered from; for example, if there are no
writers and no readers hanging on to old versions of the database, then the
OOME can be dealt with by aborting all read ransactions and hoping enough
memory is freed up.
Reducing the transitory memory usage in Fuseki is probably a more beneficial
route to go. Better connecting the parsing of incoming data to the transaction
system would be a natural, immediate step.
> Incorrectly error message "Currently in a transaction" when querying SPARQL
> ---------------------------------------------------------------------------
>
> Key: JENA-515
> URL: https://issues.apache.org/jira/browse/JENA-515
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB
> Affects Versions: TDB 0.10.1
> Reporter: Knut-Olav Hoven
>
> This has happened after a OutOfMemoryError during import (batched import of
> 1000 resources per request).
> Fetching entire database with curl works OK:
> http://localhost:3030/ds/data?graph=sift-tprogram
> When querying with SPARQL I get an 500 error "Currently in a transaction".
> No other clients are active.
> Log output:
> {code}
> 15:27:01 INFO Fuseki :: [73] GET
> http://localhost:3030/ds/query?query=PREFIX+sift%3A+%3Chttp%3A%2F%2Fgluon.nrk.no%2F2013%2Fsift%23%3E%0D%0APREFIX+prog%3A+%3Chttp%3A%2F%2Fgluon.nrk.no%2F2013%2Fsift-program%23%3E%0D%0A%0D%0ASELECT+*%0D%0AFROM+%3Chttp%3A%2F%2Flocalhost%3A3030%2Fds%2Fdata%2Fsift-tprogram%3E%0D%0A%7B%0D%0A%3Fs+sift%3Akeys+%222005%2F02952%22+.%0D%0A%3Fs+%3Fp+%3Fo%0D%0A%7D%0D%0ALIMIT+1000&output=text&stylesheet=%2Fxml-to-html.xsl&force-accept=text%2Fplain
> 15:27:01 INFO Fuseki :: [73] Query = PREFIX sift:
> <http://gluon.nrk.no/2013/sift#> PREFIX prog:
> <http://gluon.nrk.no/2013/sift-program#> SELECT * FROM
> <http://localhost:3030/ds/data/sift-tprogram> { ?s sift:keys "2005/02952" .
> ?s ?p ?o } LIMIT 1000
> 15:27:02 WARN Fuseki :: [73] RC = 500 : Currently in a
> transaction (location:/home/n21759/var/jena-fuseki/db/)
> com.hp.hpl.jena.sparql.JenaTransactionException: Currently in a transaction
> (location:/home/n21759/var/jena-fuseki/db/)
> at
> com.hp.hpl.jena.tdb.transaction.DatasetGraphTransaction.checkNotActive(DatasetGraphTransaction.java:125)
> at
> com.hp.hpl.jena.sparql.core.DatasetGraphTrackActive.begin(DatasetGraphTrackActive.java:44)
> at
> org.apache.jena.fuseki.servlets.HttpAction.beginRead(HttpAction.java:118)
> at
> org.apache.jena.fuseki.servlets.SPARQL_Query.execute(SPARQL_Query.java:234)
> at
> org.apache.jena.fuseki.servlets.SPARQL_Query.executeWithParameter(SPARQL_Query.java:193)
> at
> org.apache.jena.fuseki.servlets.SPARQL_Query.perform(SPARQL_Query.java:78)
> at
> org.apache.jena.fuseki.servlets.SPARQL_ServletBase.executeLifecycle(SPARQL_ServletBase.java:185)
> at
> org.apache.jena.fuseki.servlets.SPARQL_ServletBase.executeAction(SPARQL_ServletBase.java:166)
> at
> org.apache.jena.fuseki.servlets.SPARQL_ServletBase.execCommonWorker(SPARQL_ServletBase.java:154)
> at
> org.apache.jena.fuseki.servlets.SPARQL_ServletBase.doCommon(SPARQL_ServletBase.java:73)
> at
> org.apache.jena.fuseki.servlets.SPARQL_Query.doGet(SPARQL_Query.java:59)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
> at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1448)
> at
> org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82)
> at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:294)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
> at org.eclipse.jetty.server.Server.handle(Server.java:370)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:949)
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1011)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644)
> at
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
> at
> org.eclipse.jetty.server.nio.BlockingChannelConnector$BlockingChannelEndPoint.run(BlockingChannelConnector.java:298)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> at java.lang.Thread.run(Thread.java:722)
> 15:27:02 INFO Fuseki :: [73] 500 Currently in a transaction
> (location:/home/n21759/var/jena-fuseki/db/) (601 ms)
> {code}
> Log message of the OutOfMemoryError:
> {code}
> java.lang.OutOfMemoryError: Java heap space
> Dumping heap to java_pid28296.hprof ...
> Heap dump file created [280365745 bytes in 2.986 secs]
> 15:10:27 WARN HttpAction :: Transaction still active in endWriter
> - no commit or abort seen (forced abort)
> 15:10:28 WARN HttpAction :: Exception in forced abort (trying to
> continue)
> 15:10:40 WARN Fuseki :: [66] RC = 500 : Java heap space
> java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:2367)
> at
> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
> at
> java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
> at
> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
> at java.lang.StringBuffer.append(StringBuffer.java:237)
> at java.io.StringWriter.write(StringWriter.java:112)
> at java.io.PrintWriter.write(PrintWriter.java:456)
> at java.io.PrintWriter.write(PrintWriter.java:473)
> at java.io.PrintWriter.print(PrintWriter.java:603)
> at java.io.PrintWriter.println(PrintWriter.java:756)
> at java.lang.Throwable$WrappedPrintWriter.println(Throwable.java:763)
> at java.lang.Throwable.printStackTrace(Throwable.java:657)
> at java.lang.Throwable.printStackTrace(Throwable.java:720)
> at
> org.apache.log4j.DefaultThrowableRenderer.render(DefaultThrowableRenderer.java:60)
> at
> org.apache.log4j.spi.ThrowableInformation.getThrowableStrRep(ThrowableInformation.java:87)
> at
> org.apache.log4j.spi.LoggingEvent.getThrowableStrRep(LoggingEvent.java:413)
> at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:313)
> at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
> at
> org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
> at
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> at org.apache.log4j.Category.callAppenders(Category.java:206)
> at org.apache.log4j.Category.forcedLog(Category.java:391)
> at org.apache.log4j.Category.log(Category.java:856)
> at org.slf4j.impl.Log4jLoggerAdapter.warn(Log4jLoggerAdapter.java:479)
> at org.apache.jena.atlas.logging.Log.warn(Log.java:160)
> at org.apache.jena.atlas.logging.Log.warn(Log.java:155)
> at
> org.apache.jena.fuseki.servlets.HttpAction.endWrite(HttpAction.java:156)
> at
> org.apache.jena.fuseki.servlets.SPARQL_REST_RW.doPost(SPARQL_REST_RW.java:97)
> at
> org.apache.jena.fuseki.servlets.SPARQL_REST.doPost$(SPARQL_REST.java:246)
> at
> org.apache.jena.fuseki.servlets.SPARQL_REST.dispatch(SPARQL_REST.java:203)
> at
> org.apache.jena.fuseki.servlets.SPARQL_REST.perform(SPARQL_REST.java:190)
> at
> org.apache.jena.fuseki.servlets.SPARQL_ServletBase.executeLifecycle(SPARQL_ServletBase.java:185)
> 15:10:41 INFO Fuseki :: [66] 500 Java heap space (73.238 s)
> 15:10:41 INFO Fuseki :: [67] POST
> http://localhost:3030/ds/data?graph=sift-tprogram
> 15:11:06 WARN Fuseki :: [67] RC = 500 : Java heap space
> java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:2694)
> at java.lang.String.<init>(String.java:203)
> at java.lang.StringBuilder.toString(StringBuilder.java:405)
> at
> org.apache.jena.riot.tokens.TokenizerText.readIRI(TokenizerText.java:477)
> at
> org.apache.jena.riot.tokens.TokenizerText.parseToken(TokenizerText.java:192)
> at
> org.apache.jena.riot.tokens.TokenizerText.hasNext(TokenizerText.java:89)
> at
> org.apache.jena.atlas.iterator.PeekIterator.fill(PeekIterator.java:50)
> at
> org.apache.jena.atlas.iterator.PeekIterator.next(PeekIterator.java:92)
> at org.apache.jena.riot.lang.LangEngine.nextToken(LangEngine.java:99)
> at org.apache.jena.riot.lang.LangEngine.expect(LangEngine.java:143)
> at
> org.apache.jena.riot.lang.LangEngine.expectOrEOF(LangEngine.java:128)
> at
> org.apache.jena.riot.lang.LangTurtleBase.expectEndOfTriplesTurtle(LangTurtleBase.java:258)
> at
> org.apache.jena.riot.lang.LangTurtle.expectEndOfTriples(LangTurtle.java:49)
> at
> org.apache.jena.riot.lang.LangTurtleBase.triples(LangTurtleBase.java:243)
> at
> org.apache.jena.riot.lang.LangTurtleBase.triplesSameSubject(LangTurtleBase.java:186)
> at
> org.apache.jena.riot.lang.LangTurtle.oneTopLevelElement(LangTurtle.java:44)
> at
> org.apache.jena.riot.lang.LangTurtleBase.runParser(LangTurtleBase.java:90)
> at org.apache.jena.riot.lang.LangBase.parse(LangBase.java:42)
> at
> org.apache.jena.fuseki.servlets.SPARQL_REST.parse(SPARQL_REST.java:417)
> at
> org.apache.jena.fuseki.servlets.SPARQL_REST.parseBody(SPARQL_REST.java:406)
> at
> org.apache.jena.fuseki.servlets.SPARQL_REST_RW.doPost(SPARQL_REST_RW.java:87)
> at
> org.apache.jena.fuseki.servlets.SPARQL_REST.doPost$(SPARQL_REST.java:246)
> at
> org.apache.jena.fuseki.servlets.SPARQL_REST.dispatch(SPARQL_REST.java:203)
> at
> org.apache.jena.fuseki.servlets.SPARQL_REST.perform(SPARQL_REST.java:190)
> at
> org.apache.jena.fuseki.servlets.SPARQL_ServletBase.executeLifecycle(SPARQL_ServletBase.java:185)
> at
> org.apache.jena.fuseki.servlets.SPARQL_ServletBase.executeAction(SPARQL_ServletBase.java:166)
> at
> org.apache.jena.fuseki.servlets.SPARQL_ServletBase.execCommonWorker(SPARQL_ServletBase.java:154)
> at
> org.apache.jena.fuseki.servlets.SPARQL_ServletBase.doCommon(SPARQL_ServletBase.java:73)
> at
> org.apache.jena.fuseki.servlets.SPARQL_REST.service(SPARQL_REST.java:180)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
> at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1448)
> 15:11:06 INFO Fuseki :: [67] 500 Java heap space (25.723 s)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
