Hi Marco, A little update on the status quo:
• I noticed that the bug you observed was basically exposed by the previous bug fix (it existed before, but it didn’t occur in your particular setting). • It happens only in certain cases (i.e., for specific document sizes) and if the added/replaced document has namespaces. • Another good thing: It happens only if your database is empty. It’s not exactly an elegant proposal, but as long as we haven’t fixed the bug, you could add an initial <dummy/> document to your database. We are doing our best, though. Christian On Wed, Jul 31, 2019 at 10:20 AM Christian Grün <christian.gr...@gmail.com> wrote: > > Marco, thanks for the attachment. I have created a script that > triggers the error [1]. Most probably, the bug is related to the > previous bug issue. We’ll have a look at this soon. – Christian > > [1] https://github.com/BaseXdb/basex/issues/1711 > > > > On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere <m.lett...@gmail.com> wrote: > > > > Thanks Christian, > > I haven't been able to reproduce the bug with your SSCE. > > Nevertheless I spent some time in tracing the operations with the actual > > data I'm currently storing into the DB and a sequence of > > db:add/db:replace close enough to actual app flow. > > I attach the query script which should be rather self explaining. > > You may run the script changing always $f with $f + 1 starting (from 0). > > By uncommenting the different configurations of the variable $input you > > can see how the misbehaviour depends somehow on the data because only > > $d8 is causing the crash. > > I stared at the diffs from $d8 to the other but there really isn't any > > significant difference so now it's very hard for me to understand. > > > > Thank you in advance for all support that you can provide as usual. > > Regards, > > Marco. > > > > On 26/07/19 18:41, Christian Grün wrote: > > > Hi Marco, > > > > > > Back in February, a user reported a database inconsistency [1] – which > > > happened, too, if a database was nearly empty – and we managed to > > > build a little text case to get this reproduced [2]. After that, 9.2 > > > was released. Maybe this fix introduced another irregularity for this > > > special case? > > > > > > Maybe we can get this reproduced by taking the script from issue 1662 > > > and modify it a little? That would be great. > > > > > > Sorry to the inconvenience, > > > Christian > > > > > > PS: You can safely ignore the "Creating database..." output. It may > > > just indicate that an XML document is parsed, and that an internal > > > main-memory database instance is created, possibly for your users.xml > > > file in the data directory. > > > > > > [1] > > > https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg11459.html > > > [2] https://github.com/BaseXdb/basex/issues/1662 > > > > > > > > > > > > On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere <m.lett...@gmail.com> wrote: > > >> Hi all, > > >> > > >> starting with 9.2.1 we experience a strange error with a RESTXQ API of > > >> ours that we have been using for years. > > >> > > >> The typical pattern is lookup a document update it and store it back. > > >> > > >> We have done this milions of time and also all the tests work neatly. > > >> > > >> But when using it from inside the web application, at around the tenth > > >> operation we get the DB corrupted with the stack trace [1]. This seems > > >> to happen only when the database is nearly empty and, for sure, it > > >> doesn't appear on 8.x series. > > >> > > >> It feels like some concurrency flaw is introduced somewhere but it's > > >> hard to say because the API is eight years in age and it never changed > > >> significantly. > > >> > > >> I know it's hard without an SSCE... but while working to insulate this > > >> byzantine behaviour (I've been trying for days now) maybe someone in the > > >> list has a clue or is able to interpret the stack trace? > > >> > > >> Thanks and have a nice weekend, > > >> > > >> Marco. > > >> > > >> > > >> [1] > > >> > > >> Improper use? Potential bug? Your feedback is welcome: > > >> Contact: basex-talk@mailman.uni-konstanz.de > > >> Version: BaseX 9.3 beta > > >> Java: Ubuntu, 13 > > >> OS: Linux, amd64 > > >> Stack Trace: > > >> java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for > > >> length 1 > > >> at > > >> org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) > > >> at > > >> org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) > > >> at > > >> org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) > > >> at org.basex.data.Data.kind(Data.java:304) > > >> at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) > > >> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) > > >> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) > > >> at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) > > >> at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) > > >> at org.basex.query.expr.Arr.inline(Arr.java:69) > > >> at org.basex.query.expr.path.Path.inline(Path.java:962) > > >> at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) > > >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) > > >> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > > >> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > > >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) > > >> at org.basex.query.expr.Try.inline(Try.java:106) > > >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) > > >> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > > >> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > > >> at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) > > >> at > > >> org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) > > >> at org.basex.query.scope.MainModule.comp(MainModule.java:81) > > >> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) > > >> at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) > > >> at org.basex.query.QueryContext.compile(QueryContext.java:312) > > >> at org.basex.query.QueryContext.iter(QueryContext.java:331) > > >> at > > >> org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) > > >> at org.basex.http.web.WebResponse.create(WebResponse.java:63) > > >> at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) > > >> at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) > > >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) > > >> at > > >> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) > > >> at > > >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) > > >> at > > >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > > >> at > > >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > > >> at > > >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > >> at > > >> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > > >> at > > >> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) > > >> at > > >> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > > >> at > > >> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) > > >> at > > >> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > > >> at > > >> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) > > >> at > > >> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) > > >> at > > >> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > > >> at > > >> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) > > >> at > > >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > > >> at > > >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > >> at org.eclipse.jetty.server.Server.handle(Server.java:505) > > >> at > > >> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) > > >> at > > >> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) > > >> at > > >> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) > > >> at > > >> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > > >> at > > >> org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > > >> at > > >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > > >> at > > >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > > >> at > > >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > > >> at > > >> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > > >> at > > >> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > > >> at > > >> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) > > >> at > > >> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) > > >> at java.base/java.lang.Thread.run(Thread.java:835) > > >> > >
