Re: [basex-talk] Database corrupted on updating with RestXQ
Yes!
I can confirm that the bug disappeared on the scenario used to reproduce it!
Thank again.
Marco.
On 26/09/19 14:37, Christian Grün wrote:
Is it automatically available shipped in docker?
I guess so ("Updated 38 minutes ago"):
https://hub.docker.com/u/basex
The current scenario where to put it immediately at test is based
on the
docker distribution and it's taking too much time at the moment to
setup
a different test env.
I've received your answer about concurrent XQuery execution. I'm
currently elaborating my observations on this. Hop to be able to
formulate them during WE.
Cheers,
Marco.
On 26/09/19 13:55, Christian Grün wrote:
> Hi Marco,
>
> Finally some news on the storage bug you reported: It is fixed with
> the latest snapshot [1].
>
> Looking forward to your tests (and I hope you received my
response on
> parallel querying),
> Christian
>
> [1] http://files.basex.org/releases/latest/
>
>
>
> On Tue, Aug 6, 2019 at 9:38 AM Christian Grün
mailto:christian.gr...@gmail.com>> wrote:
>> Hi Marco,
>>
>> I think that the bug fix (which is still on my todo list) will
be made
>> available with BaseX 9.3; so, for now, it’s probably better to
choose
>> the workaround.
>>
>> Cheers,
>> Christian
>>
>>
>>
>> On Sat, Aug 3, 2019 at 10:02 AM Marco Lettere
mailto:m.lett...@gmail.com>> wrote:
>>> Hi Christian,
>>> I'm currently preparing a deployment based on Docker for one
of our
>>> customers. Here in Italy it's holiday time in August so I have
a bit of
>>> time and I can coope with your suggested workaround for the
moment.
>>> Just a question to be prepared ... are intermediate bug-fix
releases
>>> also available in the form of docker containers?
>>> Thanks for your support as usual.
>>> Regards,
>>> Marco.
>>>
>>> On 02/08/19 15:04, Christian Grün wrote:
Hi Marco,
A little update on the status quo:
• I noticed that the bug you observed was basically exposed
by the
previous bug fix (it existed before, but it didn’t occur in your
particular setting).
• It happens only in certain cases (i.e., for specific
document sizes)
and if the added/replaced document has namespaces.
• Another good thing: It happens only if your database is empty.
It’s not exactly an elegant proposal, but as long as we
haven’t fixed
the bug, you could add an initial document to your
database.
We are doing our best, though.
Christian
On Wed, Jul 31, 2019 at 10:20 AM Christian Grün
mailto:christian.gr...@gmail.com>> wrote:
> Marco, thanks for the attachment. I have created a script that
> triggers the error [1]. Most probably, the bug is related to the
> previous bug issue. We’ll have a look at this soon. – Christian
>
> [1] https://github.com/BaseXdb/basex/issues/1711
>
>
>
> On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere
mailto:m.lett...@gmail.com>> wrote:
>> Thanks Christian,
>> I haven't been able to reproduce the bug with your SSCE.
>> Nevertheless I spent some time in tracing the operations
with the actual
>> data I'm currently storing into the DB and a sequence of
>> db:add/db:replace close enough to actual app flow.
>> I attach the query script which should be rather self
explaining.
>> You may run the script changing always $f with $f + 1
starting (from 0).
>> By uncommenting the different configurations of the
variable $input you
>> can see how the misbehaviour depends somehow on the data
because only
>> $d8 is causing the crash.
>> I stared at the diffs from $d8 to the other but there
really isn't any
>> significant difference so now it's very hard for me to
understand.
>>
>> Thank you in advance for all support that you can provide
as usual.
>> Regards,
>> Marco.
>>
>> On 26/07/19 18:41, Christian Grün wrote:
>>> Hi Marco,
>>>
>>> Back in February, a user reported a database inconsistency
[1] – which
>>> happened, too, if a database was nearly empty – and we
managed to
>>> build a little text case to get this reproduced [2]. After
that, 9.2
>>> was released. Maybe this fix introduced another
irregularity for this
>>> special case?
>>>
>>> Maybe we can get this reproduced by taking the script from
issue 1662
>>> and modify it a little? That would be great.
>>>
>>> Sorry to the inconvenience,
>>> Christian
>>>
Re: [basex-talk] Database corrupted on updating with RestXQ
>
> Is it automatically available shipped in docker?
>
I guess so ("Updated 38 minutes ago"):
https://hub.docker.com/u/basex
The current scenario where to put it immediately at test is based on the
> docker distribution and it's taking too much time at the moment to setup
> a different test env.
> I've received your answer about concurrent XQuery execution. I'm
> currently elaborating my observations on this. Hop to be able to
> formulate them during WE.
> Cheers,
> Marco.
>
> On 26/09/19 13:55, Christian Grün wrote:
> > Hi Marco,
> >
> > Finally some news on the storage bug you reported: It is fixed with
> > the latest snapshot [1].
> >
> > Looking forward to your tests (and I hope you received my response on
> > parallel querying),
> > Christian
> >
> > [1] http://files.basex.org/releases/latest/
> >
> >
> >
> > On Tue, Aug 6, 2019 at 9:38 AM Christian Grün
> wrote:
> >> Hi Marco,
> >>
> >> I think that the bug fix (which is still on my todo list) will be made
> >> available with BaseX 9.3; so, for now, it’s probably better to choose
> >> the workaround.
> >>
> >> Cheers,
> >> Christian
> >>
> >>
> >>
> >> On Sat, Aug 3, 2019 at 10:02 AM Marco Lettere
> wrote:
> >>> Hi Christian,
> >>> I'm currently preparing a deployment based on Docker for one of our
> >>> customers. Here in Italy it's holiday time in August so I have a bit of
> >>> time and I can coope with your suggested workaround for the moment.
> >>> Just a question to be prepared ... are intermediate bug-fix releases
> >>> also available in the form of docker containers?
> >>> Thanks for your support as usual.
> >>> Regards,
> >>> Marco.
> >>>
> >>> On 02/08/19 15:04, Christian Grün wrote:
> Hi Marco,
>
> A little update on the status quo:
>
> • I noticed that the bug you observed was basically exposed by the
> previous bug fix (it existed before, but it didn’t occur in your
> particular setting).
> • It happens only in certain cases (i.e., for specific document sizes)
> and if the added/replaced document has namespaces.
> • Another good thing: It happens only if your database is empty.
>
> It’s not exactly an elegant proposal, but as long as we haven’t fixed
> the bug, you could add an initial document to your database.
>
> We are doing our best, though.
> Christian
>
>
>
> On Wed, Jul 31, 2019 at 10:20 AM Christian Grün
> wrote:
> > Marco, thanks for the attachment. I have created a script that
> > triggers the error [1]. Most probably, the bug is related to the
> > previous bug issue. We’ll have a look at this soon. – Christian
> >
> > [1] https://github.com/BaseXdb/basex/issues/1711
> >
> >
> >
> > On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere
> wrote:
> >> Thanks Christian,
> >> I haven't been able to reproduce the bug with your SSCE.
> >> Nevertheless I spent some time in tracing the operations with the
> actual
> >> data I'm currently storing into the DB and a sequence of
> >> db:add/db:replace close enough to actual app flow.
> >> I attach the query script which should be rather self explaining.
> >> You may run the script changing always $f with $f + 1 starting
> (from 0).
> >> By uncommenting the different configurations of the variable $input
> you
> >> can see how the misbehaviour depends somehow on the data because
> only
> >> $d8 is causing the crash.
> >> I stared at the diffs from $d8 to the other but there really isn't
> any
> >> significant difference so now it's very hard for me to understand.
> >>
> >> Thank you in advance for all support that you can provide as usual.
> >> Regards,
> >> Marco.
> >>
> >> On 26/07/19 18:41, Christian Grün wrote:
> >>> Hi Marco,
> >>>
> >>> Back in February, a user reported a database inconsistency [1] –
> which
> >>> happened, too, if a database was nearly empty – and we managed to
> >>> build a little text case to get this reproduced [2]. After that,
> 9.2
> >>> was released. Maybe this fix introduced another irregularity for
> this
> >>> special case?
> >>>
> >>> Maybe we can get this reproduced by taking the script from issue
> 1662
> >>> and modify it a little? That would be great.
> >>>
> >>> Sorry to the inconvenience,
> >>> Christian
> >>>
> >>> PS: You can safely ignore the "Creating database..." output. It may
> >>> just indicate that an XML document is parsed, and that an internal
> >>> main-memory database instance is created, possibly for your
> users.xml
> >>> file in the data directory.
> >>>
> >>> [1]
> https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg11459.html
> >>> [2] https://github.com/BaseXdb/basex/issues/1662
> >>>
> >>>
> >>>
> >>> On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere
> wrote:
> Hi all,
>
> starting
Re: [basex-talk] Database corrupted on updating with RestXQ
Thanks Christian. Is it automatically available shipped in docker? The current scenario where to put it immediately at test is based on the docker distribution and it's taking too much time at the moment to setup a different test env. I've received your answer about concurrent XQuery execution. I'm currently elaborating my observations on this. Hop to be able to formulate them during WE. Cheers, Marco. On 26/09/19 13:55, Christian Grün wrote: Hi Marco, Finally some news on the storage bug you reported: It is fixed with the latest snapshot [1]. Looking forward to your tests (and I hope you received my response on parallel querying), Christian [1] http://files.basex.org/releases/latest/ On Tue, Aug 6, 2019 at 9:38 AM Christian Grün wrote: Hi Marco, I think that the bug fix (which is still on my todo list) will be made available with BaseX 9.3; so, for now, it’s probably better to choose the workaround. Cheers, Christian On Sat, Aug 3, 2019 at 10:02 AM Marco Lettere wrote: Hi Christian, I'm currently preparing a deployment based on Docker for one of our customers. Here in Italy it's holiday time in August so I have a bit of time and I can coope with your suggested workaround for the moment. Just a question to be prepared ... are intermediate bug-fix releases also available in the form of docker containers? Thanks for your support as usual. Regards, Marco. On 02/08/19 15:04, Christian Grün wrote: Hi Marco, A little update on the status quo: • I noticed that the bug you observed was basically exposed by the previous bug fix (it existed before, but it didn’t occur in your particular setting). • It happens only in certain cases (i.e., for specific document sizes) and if the added/replaced document has namespaces. • Another good thing: It happens only if your database is empty. It’s not exactly an elegant proposal, but as long as we haven’t fixed the bug, you could add an initial document to your database. We are doing our best, though. Christian On Wed, Jul 31, 2019 at 10:20 AM Christian Grün wrote: Marco, thanks for the attachment. I have created a script that triggers the error [1]. Most probably, the bug is related to the previous bug issue. We’ll have a look at this soon. – Christian [1] https://github.com/BaseXdb/basex/issues/1711 On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere wrote: Thanks Christian, I haven't been able to reproduce the bug with your SSCE. Nevertheless I spent some time in tracing the operations with the actual data I'm currently storing into the DB and a sequence of db:add/db:replace close enough to actual app flow. I attach the query script which should be rather self explaining. You may run the script changing always $f with $f + 1 starting (from 0). By uncommenting the different configurations of the variable $input you can see how the misbehaviour depends somehow on the data because only $d8 is causing the crash. I stared at the diffs from $d8 to the other but there really isn't any significant difference so now it's very hard for me to understand. Thank you in advance for all support that you can provide as usual. Regards, Marco. On 26/07/19 18:41, Christian Grün wrote: Hi Marco, Back in February, a user reported a database inconsistency [1] – which happened, too, if a database was nearly empty – and we managed to build a little text case to get this reproduced [2]. After that, 9.2 was released. Maybe this fix introduced another irregularity for this special case? Maybe we can get this reproduced by taking the script from issue 1662 and modify it a little? That would be great. Sorry to the inconvenience, Christian PS: You can safely ignore the "Creating database..." output. It may just indicate that an XML document is parsed, and that an internal main-memory database instance is created, possibly for your users.xml file in the data directory. [1] https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg11459.html [2] https://github.com/BaseXdb/basex/issues/1662 On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere wrote: Hi all, starting with 9.2.1 we experience a strange error with a RESTXQ API of ours that we have been using for years. The typical pattern is lookup a document update it and store it back. We have done this milions of time and also all the tests work neatly. But when using it from inside the web application, at around the tenth operation we get the DB corrupted with the stack trace [1]. This seems to happen only when the database is nearly empty and, for sure, it doesn't appear on 8.x series. It feels like some concurrency flaw is introduced somewhere but it's hard to say because the API is eight years in age and it never changed significantly. I know it's hard without an SSCE... but while working to insulate this byzantine behaviour (I've been trying for days now) maybe someone in the list has a clue or is able to interpret the stack trace? Thanks and have a nice weekend, Marco. [1] Improper use?
Re: [basex-talk] Database corrupted on updating with RestXQ
Hi Marco, Finally some news on the storage bug you reported: It is fixed with the latest snapshot [1]. Looking forward to your tests (and I hope you received my response on parallel querying), Christian [1] http://files.basex.org/releases/latest/ On Tue, Aug 6, 2019 at 9:38 AM Christian Grün wrote: > > Hi Marco, > > I think that the bug fix (which is still on my todo list) will be made > available with BaseX 9.3; so, for now, it’s probably better to choose > the workaround. > > Cheers, > Christian > > > > On Sat, Aug 3, 2019 at 10:02 AM Marco Lettere wrote: > > > > Hi Christian, > > I'm currently preparing a deployment based on Docker for one of our > > customers. Here in Italy it's holiday time in August so I have a bit of > > time and I can coope with your suggested workaround for the moment. > > Just a question to be prepared ... are intermediate bug-fix releases > > also available in the form of docker containers? > > Thanks for your support as usual. > > Regards, > > Marco. > > > > On 02/08/19 15:04, Christian Grün wrote: > > > Hi Marco, > > > > > > A little update on the status quo: > > > > > > • I noticed that the bug you observed was basically exposed by the > > > previous bug fix (it existed before, but it didn’t occur in your > > > particular setting). > > > • It happens only in certain cases (i.e., for specific document sizes) > > > and if the added/replaced document has namespaces. > > > • Another good thing: It happens only if your database is empty. > > > > > > It’s not exactly an elegant proposal, but as long as we haven’t fixed > > > the bug, you could add an initial document to your database. > > > > > > We are doing our best, though. > > > Christian > > > > > > > > > > > > On Wed, Jul 31, 2019 at 10:20 AM Christian Grün > > > wrote: > > >> Marco, thanks for the attachment. I have created a script that > > >> triggers the error [1]. Most probably, the bug is related to the > > >> previous bug issue. We’ll have a look at this soon. – Christian > > >> > > >> [1] https://github.com/BaseXdb/basex/issues/1711 > > >> > > >> > > >> > > >> On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere > > >> wrote: > > >>> Thanks Christian, > > >>> I haven't been able to reproduce the bug with your SSCE. > > >>> Nevertheless I spent some time in tracing the operations with the actual > > >>> data I'm currently storing into the DB and a sequence of > > >>> db:add/db:replace close enough to actual app flow. > > >>> I attach the query script which should be rather self explaining. > > >>> You may run the script changing always $f with $f + 1 starting (from 0). > > >>> By uncommenting the different configurations of the variable $input you > > >>> can see how the misbehaviour depends somehow on the data because only > > >>> $d8 is causing the crash. > > >>> I stared at the diffs from $d8 to the other but there really isn't any > > >>> significant difference so now it's very hard for me to understand. > > >>> > > >>> Thank you in advance for all support that you can provide as usual. > > >>> Regards, > > >>> Marco. > > >>> > > >>> On 26/07/19 18:41, Christian Grün wrote: > > Hi Marco, > > > > Back in February, a user reported a database inconsistency [1] – which > > happened, too, if a database was nearly empty – and we managed to > > build a little text case to get this reproduced [2]. After that, 9.2 > > was released. Maybe this fix introduced another irregularity for this > > special case? > > > > Maybe we can get this reproduced by taking the script from issue 1662 > > and modify it a little? That would be great. > > > > Sorry to the inconvenience, > > Christian > > > > PS: You can safely ignore the "Creating database..." output. It may > > just indicate that an XML document is parsed, and that an internal > > main-memory database instance is created, possibly for your users.xml > > file in the data directory. > > > > [1] > > https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg11459.html > > [2] https://github.com/BaseXdb/basex/issues/1662 > > > > > > > > On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere > > wrote: > > > Hi all, > > > > > > starting with 9.2.1 we experience a strange error with a RESTXQ API > > > of > > > ours that we have been using for years. > > > > > > The typical pattern is lookup a document update it and store it back. > > > > > > We have done this milions of time and also all the tests work neatly. > > > > > > But when using it from inside the web application, at around the tenth > > > operation we get the DB corrupted with the stack trace [1]. This seems > > > to happen only when the database is nearly empty and, for sure, it > > > doesn't appear on 8.x series. > > > > > > It feels like some concurrency flaw is introduced somewhere but it's > > > hard to say
Re: [basex-talk] Database corrupted on updating with RestXQ
Hi Marco, I think that the bug fix (which is still on my todo list) will be made available with BaseX 9.3; so, for now, it’s probably better to choose the workaround. Cheers, Christian On Sat, Aug 3, 2019 at 10:02 AM Marco Lettere wrote: > > Hi Christian, > I'm currently preparing a deployment based on Docker for one of our > customers. Here in Italy it's holiday time in August so I have a bit of > time and I can coope with your suggested workaround for the moment. > Just a question to be prepared ... are intermediate bug-fix releases > also available in the form of docker containers? > Thanks for your support as usual. > Regards, > Marco. > > On 02/08/19 15:04, Christian Grün wrote: > > Hi Marco, > > > > A little update on the status quo: > > > > • I noticed that the bug you observed was basically exposed by the > > previous bug fix (it existed before, but it didn’t occur in your > > particular setting). > > • It happens only in certain cases (i.e., for specific document sizes) > > and if the added/replaced document has namespaces. > > • Another good thing: It happens only if your database is empty. > > > > It’s not exactly an elegant proposal, but as long as we haven’t fixed > > the bug, you could add an initial document to your database. > > > > We are doing our best, though. > > Christian > > > > > > > > On Wed, Jul 31, 2019 at 10:20 AM Christian Grün > > wrote: > >> Marco, thanks for the attachment. I have created a script that > >> triggers the error [1]. Most probably, the bug is related to the > >> previous bug issue. We’ll have a look at this soon. – Christian > >> > >> [1] https://github.com/BaseXdb/basex/issues/1711 > >> > >> > >> > >> On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere wrote: > >>> Thanks Christian, > >>> I haven't been able to reproduce the bug with your SSCE. > >>> Nevertheless I spent some time in tracing the operations with the actual > >>> data I'm currently storing into the DB and a sequence of > >>> db:add/db:replace close enough to actual app flow. > >>> I attach the query script which should be rather self explaining. > >>> You may run the script changing always $f with $f + 1 starting (from 0). > >>> By uncommenting the different configurations of the variable $input you > >>> can see how the misbehaviour depends somehow on the data because only > >>> $d8 is causing the crash. > >>> I stared at the diffs from $d8 to the other but there really isn't any > >>> significant difference so now it's very hard for me to understand. > >>> > >>> Thank you in advance for all support that you can provide as usual. > >>> Regards, > >>> Marco. > >>> > >>> On 26/07/19 18:41, Christian Grün wrote: > Hi Marco, > > Back in February, a user reported a database inconsistency [1] – which > happened, too, if a database was nearly empty – and we managed to > build a little text case to get this reproduced [2]. After that, 9.2 > was released. Maybe this fix introduced another irregularity for this > special case? > > Maybe we can get this reproduced by taking the script from issue 1662 > and modify it a little? That would be great. > > Sorry to the inconvenience, > Christian > > PS: You can safely ignore the "Creating database..." output. It may > just indicate that an XML document is parsed, and that an internal > main-memory database instance is created, possibly for your users.xml > file in the data directory. > > [1] > https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg11459.html > [2] https://github.com/BaseXdb/basex/issues/1662 > > > > On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere > wrote: > > Hi all, > > > > starting with 9.2.1 we experience a strange error with a RESTXQ API of > > ours that we have been using for years. > > > > The typical pattern is lookup a document update it and store it back. > > > > We have done this milions of time and also all the tests work neatly. > > > > But when using it from inside the web application, at around the tenth > > operation we get the DB corrupted with the stack trace [1]. This seems > > to happen only when the database is nearly empty and, for sure, it > > doesn't appear on 8.x series. > > > > It feels like some concurrency flaw is introduced somewhere but it's > > hard to say because the API is eight years in age and it never changed > > significantly. > > > > I know it's hard without an SSCE... but while working to insulate this > > byzantine behaviour (I've been trying for days now) maybe someone in the > > list has a clue or is able to interpret the stack trace? > > > > Thanks and have a nice weekend, > > > > Marco. > > > > > > [1] > > > > Improper use? Potential bug? Your feedback is welcome: > > Contact: basex-talk@mailman.uni-konstanz.de > > Version:
Re: [basex-talk] Database corrupted on updating with RestXQ
Hi Christian, I'm currently preparing a deployment based on Docker for one of our customers. Here in Italy it's holiday time in August so I have a bit of time and I can coope with your suggested workaround for the moment. Just a question to be prepared ... are intermediate bug-fix releases also available in the form of docker containers? Thanks for your support as usual. Regards, Marco. On 02/08/19 15:04, Christian Grün wrote: Hi Marco, A little update on the status quo: • I noticed that the bug you observed was basically exposed by the previous bug fix (it existed before, but it didn’t occur in your particular setting). • It happens only in certain cases (i.e., for specific document sizes) and if the added/replaced document has namespaces. • Another good thing: It happens only if your database is empty. It’s not exactly an elegant proposal, but as long as we haven’t fixed the bug, you could add an initial document to your database. We are doing our best, though. Christian On Wed, Jul 31, 2019 at 10:20 AM Christian Grün wrote: Marco, thanks for the attachment. I have created a script that triggers the error [1]. Most probably, the bug is related to the previous bug issue. We’ll have a look at this soon. – Christian [1] https://github.com/BaseXdb/basex/issues/1711 On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere wrote: Thanks Christian, I haven't been able to reproduce the bug with your SSCE. Nevertheless I spent some time in tracing the operations with the actual data I'm currently storing into the DB and a sequence of db:add/db:replace close enough to actual app flow. I attach the query script which should be rather self explaining. You may run the script changing always $f with $f + 1 starting (from 0). By uncommenting the different configurations of the variable $input you can see how the misbehaviour depends somehow on the data because only $d8 is causing the crash. I stared at the diffs from $d8 to the other but there really isn't any significant difference so now it's very hard for me to understand. Thank you in advance for all support that you can provide as usual. Regards, Marco. On 26/07/19 18:41, Christian Grün wrote: Hi Marco, Back in February, a user reported a database inconsistency [1] – which happened, too, if a database was nearly empty – and we managed to build a little text case to get this reproduced [2]. After that, 9.2 was released. Maybe this fix introduced another irregularity for this special case? Maybe we can get this reproduced by taking the script from issue 1662 and modify it a little? That would be great. Sorry to the inconvenience, Christian PS: You can safely ignore the "Creating database..." output. It may just indicate that an XML document is parsed, and that an internal main-memory database instance is created, possibly for your users.xml file in the data directory. [1] https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg11459.html [2] https://github.com/BaseXdb/basex/issues/1662 On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere wrote: Hi all, starting with 9.2.1 we experience a strange error with a RESTXQ API of ours that we have been using for years. The typical pattern is lookup a document update it and store it back. We have done this milions of time and also all the tests work neatly. But when using it from inside the web application, at around the tenth operation we get the DB corrupted with the stack trace [1]. This seems to happen only when the database is nearly empty and, for sure, it doesn't appear on 8.x series. It feels like some concurrency flaw is introduced somewhere but it's hard to say because the API is eight years in age and it never changed significantly. I know it's hard without an SSCE... but while working to insulate this byzantine behaviour (I've been trying for days now) maybe someone in the list has a clue or is able to interpret the stack trace? Thanks and have a nice weekend, Marco. [1] Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.3 beta Java: Ubuntu, 13 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) at org.basex.data.Data.kind(Data.java:304) at org.basex.query.value.node.DBNode.(DBNode.java:50) at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) at org.basex.query.expr.Arr.inline(Arr.java:69) at org.basex.query.expr.path.Path.inline(Path.java:962) at
Re: [basex-talk] Database corrupted on updating with RestXQ
Hi Marco, A little update on the status quo: • I noticed that the bug you observed was basically exposed by the previous bug fix (it existed before, but it didn’t occur in your particular setting). • It happens only in certain cases (i.e., for specific document sizes) and if the added/replaced document has namespaces. • Another good thing: It happens only if your database is empty. It’s not exactly an elegant proposal, but as long as we haven’t fixed the bug, you could add an initial document to your database. We are doing our best, though. Christian On Wed, Jul 31, 2019 at 10:20 AM Christian Grün wrote: > > Marco, thanks for the attachment. I have created a script that > triggers the error [1]. Most probably, the bug is related to the > previous bug issue. We’ll have a look at this soon. – Christian > > [1] https://github.com/BaseXdb/basex/issues/1711 > > > > On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere wrote: > > > > Thanks Christian, > > I haven't been able to reproduce the bug with your SSCE. > > Nevertheless I spent some time in tracing the operations with the actual > > data I'm currently storing into the DB and a sequence of > > db:add/db:replace close enough to actual app flow. > > I attach the query script which should be rather self explaining. > > You may run the script changing always $f with $f + 1 starting (from 0). > > By uncommenting the different configurations of the variable $input you > > can see how the misbehaviour depends somehow on the data because only > > $d8 is causing the crash. > > I stared at the diffs from $d8 to the other but there really isn't any > > significant difference so now it's very hard for me to understand. > > > > Thank you in advance for all support that you can provide as usual. > > Regards, > > Marco. > > > > On 26/07/19 18:41, Christian Grün wrote: > > > Hi Marco, > > > > > > Back in February, a user reported a database inconsistency [1] – which > > > happened, too, if a database was nearly empty – and we managed to > > > build a little text case to get this reproduced [2]. After that, 9.2 > > > was released. Maybe this fix introduced another irregularity for this > > > special case? > > > > > > Maybe we can get this reproduced by taking the script from issue 1662 > > > and modify it a little? That would be great. > > > > > > Sorry to the inconvenience, > > > Christian > > > > > > PS: You can safely ignore the "Creating database..." output. It may > > > just indicate that an XML document is parsed, and that an internal > > > main-memory database instance is created, possibly for your users.xml > > > file in the data directory. > > > > > > [1] > > > https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg11459.html > > > [2] https://github.com/BaseXdb/basex/issues/1662 > > > > > > > > > > > > On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere wrote: > > >> Hi all, > > >> > > >> starting with 9.2.1 we experience a strange error with a RESTXQ API of > > >> ours that we have been using for years. > > >> > > >> The typical pattern is lookup a document update it and store it back. > > >> > > >> We have done this milions of time and also all the tests work neatly. > > >> > > >> But when using it from inside the web application, at around the tenth > > >> operation we get the DB corrupted with the stack trace [1]. This seems > > >> to happen only when the database is nearly empty and, for sure, it > > >> doesn't appear on 8.x series. > > >> > > >> It feels like some concurrency flaw is introduced somewhere but it's > > >> hard to say because the API is eight years in age and it never changed > > >> significantly. > > >> > > >> I know it's hard without an SSCE... but while working to insulate this > > >> byzantine behaviour (I've been trying for days now) maybe someone in the > > >> list has a clue or is able to interpret the stack trace? > > >> > > >> Thanks and have a nice weekend, > > >> > > >> Marco. > > >> > > >> > > >> [1] > > >> > > >> Improper use? Potential bug? Your feedback is welcome: > > >> Contact: basex-talk@mailman.uni-konstanz.de > > >> Version: BaseX 9.3 beta > > >> Java: Ubuntu, 13 > > >> OS: Linux, amd64 > > >> Stack Trace: > > >> java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for > > >> length 1 > > >> at > > >> org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) > > >> at > > >> org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) > > >> at > > >> org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) > > >> at org.basex.data.Data.kind(Data.java:304) > > >> at org.basex.query.value.node.DBNode.(DBNode.java:50) > > >> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) > > >> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) > > >> at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) > > >> at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) > > >> at
Re: [basex-talk] Database corrupted on updating with RestXQ
Marco, thanks for the attachment. I have created a script that triggers the error [1]. Most probably, the bug is related to the previous bug issue. We’ll have a look at this soon. – Christian [1] https://github.com/BaseXdb/basex/issues/1711 On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere wrote: > > Thanks Christian, > I haven't been able to reproduce the bug with your SSCE. > Nevertheless I spent some time in tracing the operations with the actual > data I'm currently storing into the DB and a sequence of > db:add/db:replace close enough to actual app flow. > I attach the query script which should be rather self explaining. > You may run the script changing always $f with $f + 1 starting (from 0). > By uncommenting the different configurations of the variable $input you > can see how the misbehaviour depends somehow on the data because only > $d8 is causing the crash. > I stared at the diffs from $d8 to the other but there really isn't any > significant difference so now it's very hard for me to understand. > > Thank you in advance for all support that you can provide as usual. > Regards, > Marco. > > On 26/07/19 18:41, Christian Grün wrote: > > Hi Marco, > > > > Back in February, a user reported a database inconsistency [1] – which > > happened, too, if a database was nearly empty – and we managed to > > build a little text case to get this reproduced [2]. After that, 9.2 > > was released. Maybe this fix introduced another irregularity for this > > special case? > > > > Maybe we can get this reproduced by taking the script from issue 1662 > > and modify it a little? That would be great. > > > > Sorry to the inconvenience, > > Christian > > > > PS: You can safely ignore the "Creating database..." output. It may > > just indicate that an XML document is parsed, and that an internal > > main-memory database instance is created, possibly for your users.xml > > file in the data directory. > > > > [1] > > https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg11459.html > > [2] https://github.com/BaseXdb/basex/issues/1662 > > > > > > > > On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere wrote: > >> Hi all, > >> > >> starting with 9.2.1 we experience a strange error with a RESTXQ API of > >> ours that we have been using for years. > >> > >> The typical pattern is lookup a document update it and store it back. > >> > >> We have done this milions of time and also all the tests work neatly. > >> > >> But when using it from inside the web application, at around the tenth > >> operation we get the DB corrupted with the stack trace [1]. This seems > >> to happen only when the database is nearly empty and, for sure, it > >> doesn't appear on 8.x series. > >> > >> It feels like some concurrency flaw is introduced somewhere but it's > >> hard to say because the API is eight years in age and it never changed > >> significantly. > >> > >> I know it's hard without an SSCE... but while working to insulate this > >> byzantine behaviour (I've been trying for days now) maybe someone in the > >> list has a clue or is able to interpret the stack trace? > >> > >> Thanks and have a nice weekend, > >> > >> Marco. > >> > >> > >> [1] > >> > >> Improper use? Potential bug? Your feedback is welcome: > >> Contact: basex-talk@mailman.uni-konstanz.de > >> Version: BaseX 9.3 beta > >> Java: Ubuntu, 13 > >> OS: Linux, amd64 > >> Stack Trace: > >> java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length > >> 1 > >> at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) > >> at > >> org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) > >> at > >> org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) > >> at org.basex.data.Data.kind(Data.java:304) > >> at org.basex.query.value.node.DBNode.(DBNode.java:50) > >> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) > >> at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) > >> at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) > >> at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) > >> at org.basex.query.expr.Arr.inline(Arr.java:69) > >> at org.basex.query.expr.path.Path.inline(Path.java:962) > >> at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) > >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) > >> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > >> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) > >> at org.basex.query.expr.Try.inline(Try.java:106) > >> at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) > >> at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > >> at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > >> at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) >
Re: [basex-talk] Database corrupted on updating with RestXQ
Thanks Christian, I haven't been able to reproduce the bug with your SSCE. Nevertheless I spent some time in tracing the operations with the actual data I'm currently storing into the DB and a sequence of db:add/db:replace close enough to actual app flow. I attach the query script which should be rather self explaining. You may run the script changing always $f with $f + 1 starting (from 0). By uncommenting the different configurations of the variable $input you can see how the misbehaviour depends somehow on the data because only $d8 is causing the crash. I stared at the diffs from $d8 to the other but there really isn't any significant difference so now it's very hard for me to understand. Thank you in advance for all support that you can provide as usual. Regards, Marco. On 26/07/19 18:41, Christian Grün wrote: Hi Marco, Back in February, a user reported a database inconsistency [1] – which happened, too, if a database was nearly empty – and we managed to build a little text case to get this reproduced [2]. After that, 9.2 was released. Maybe this fix introduced another irregularity for this special case? Maybe we can get this reproduced by taking the script from issue 1662 and modify it a little? That would be great. Sorry to the inconvenience, Christian PS: You can safely ignore the "Creating database..." output. It may just indicate that an XML document is parsed, and that an internal main-memory database instance is created, possibly for your users.xml file in the data directory. [1] https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg11459.html [2] https://github.com/BaseXdb/basex/issues/1662 On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere wrote: Hi all, starting with 9.2.1 we experience a strange error with a RESTXQ API of ours that we have been using for years. The typical pattern is lookup a document update it and store it back. We have done this milions of time and also all the tests work neatly. But when using it from inside the web application, at around the tenth operation we get the DB corrupted with the stack trace [1]. This seems to happen only when the database is nearly empty and, for sure, it doesn't appear on 8.x series. It feels like some concurrency flaw is introduced somewhere but it's hard to say because the API is eight years in age and it never changed significantly. I know it's hard without an SSCE... but while working to insulate this byzantine behaviour (I've been trying for days now) maybe someone in the list has a clue or is able to interpret the stack trace? Thanks and have a nice weekend, Marco. [1] Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.3 beta Java: Ubuntu, 13 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) at org.basex.data.Data.kind(Data.java:304) at org.basex.query.value.node.DBNode.(DBNode.java:50) at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) at org.basex.query.expr.Arr.inline(Arr.java:69) at org.basex.query.expr.path.Path.inline(Path.java:962) at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) at org.basex.query.expr.Try.inline(Try.java:106) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) at org.basex.query.scope.MainModule.comp(MainModule.java:81) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) at org.basex.query.QueryContext.compile(QueryContext.java:312) at org.basex.query.QueryContext.iter(QueryContext.java:331) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) at org.basex.http.web.WebResponse.create(WebResponse.java:63) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) at org.basex.http.BaseXServlet.service(BaseXServlet.java:65)
Re: [basex-talk] Database corrupted on updating with RestXQ
Hi Marco, Back in February, a user reported a database inconsistency [1] – which happened, too, if a database was nearly empty – and we managed to build a little text case to get this reproduced [2]. After that, 9.2 was released. Maybe this fix introduced another irregularity for this special case? Maybe we can get this reproduced by taking the script from issue 1662 and modify it a little? That would be great. Sorry to the inconvenience, Christian PS: You can safely ignore the "Creating database..." output. It may just indicate that an XML document is parsed, and that an internal main-memory database instance is created, possibly for your users.xml file in the data directory. [1] https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg11459.html [2] https://github.com/BaseXdb/basex/issues/1662 On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere wrote: > > Hi all, > > starting with 9.2.1 we experience a strange error with a RESTXQ API of > ours that we have been using for years. > > The typical pattern is lookup a document update it and store it back. > > We have done this milions of time and also all the tests work neatly. > > But when using it from inside the web application, at around the tenth > operation we get the DB corrupted with the stack trace [1]. This seems > to happen only when the database is nearly empty and, for sure, it > doesn't appear on 8.x series. > > It feels like some concurrency flaw is introduced somewhere but it's > hard to say because the API is eight years in age and it never changed > significantly. > > I know it's hard without an SSCE... but while working to insulate this > byzantine behaviour (I've been trying for days now) maybe someone in the > list has a clue or is able to interpret the stack trace? > > Thanks and have a nice weekend, > > Marco. > > > [1] > > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk@mailman.uni-konstanz.de > Version: BaseX 9.3 beta > Java: Ubuntu, 13 > OS: Linux, amd64 > Stack Trace: > java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 > at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) > at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) > at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) > at org.basex.data.Data.kind(Data.java:304) > at org.basex.query.value.node.DBNode.(DBNode.java:50) > at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) > at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) > at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) > at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) > at org.basex.query.expr.Arr.inline(Arr.java:69) > at org.basex.query.expr.path.Path.inline(Path.java:962) > at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) > at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) > at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) > at org.basex.query.expr.Try.inline(Try.java:106) > at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) > at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) > at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) > at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) > at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) > at org.basex.query.scope.MainModule.comp(MainModule.java:81) > at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) > at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) > at org.basex.query.QueryContext.compile(QueryContext.java:312) > at org.basex.query.QueryContext.iter(QueryContext.java:331) > at > org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) > at org.basex.http.web.WebResponse.create(WebResponse.java:63) > at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) > at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) > at > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) > at >
Re: [basex-talk] Database corrupted on updating with RestXQ
As an additional remark I set the MIXUPDATES = true option. Moreover I notice the wording "Creating database..." among the request logs that are output when running with -d [1]. This sounds strange (the only point I see the string in the code is the Builder class [2]) but it happens also with 8.6.7 which doesn't show this misbehaviour. M. [1] _ REQUEST _ (GET /hlcm/xdwapi/v2/xdw/2.25.8046764881955849894)@912634350 org.eclipse.jetty.server.Request@3665b1ee - Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 - Connection: keep-alive - User-Agent: Java/13 - Host: 10.17.217.1:8984 _ RESPONSE HTTP/1.1 200 Set-Cookie: JSESSIONID=a2ryzgdm4t941ra9jmes9mj87;Path=/ Expires: Thu, 01 Jan 1970 00:00:00 GMT Content-Type: application/xml; charset=UTF-8 * Creating Database...* . 5.58 ms (100 MB) _ REQUEST _ (GET /hlcm/taskviewer/showworkflow?workflowid=2.25.8046764881955849894)@1286045891 org.eclipse.jetty.server.Request@4ca780c3 - Accept: */* - Connection: keep-alive - Referer: http://localhost:8984/ui/hlcm/main.cameo?config=HLCM_1=test1=rossi - Accept-Encoding: gzip, deflate, br - User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/75.0.3770.90 Chrome/75.0.3770.90 Safari/537.36 - Accept-Language: it,en-US;q=0.9,en;q=0.8 - Cookie: COOKIE_SUPPORT=true; GUEST_LANGUAGE_ID=en_US; pnctest=1; JSESSIONID=2g5ep7mzq0ah1ktre4b398hfc - Host: localhost:8984 _ RESPONSE HTTP/1.1 200 [2] https://github.com/BaseXdb/basex/blob/f7a5492c46d55e1c1f58df24b8ed9567c176e8c1/basex-core/src/main/java/org/basex/build/Builder.java On 26/07/19 16:29, Marco Lettere wrote: Hi all, starting with 9.2.1 we experience a strange error with a RESTXQ API of ours that we have been using for years. The typical pattern is lookup a document update it and store it back. We have done this milions of time and also all the tests work neatly. But when using it from inside the web application, at around the tenth operation we get the DB corrupted with the stack trace [1]. This seems to happen only when the database is nearly empty and, for sure, it doesn't appear on 8.x series. It feels like some concurrency flaw is introduced somewhere but it's hard to say because the API is eight years in age and it never changed significantly. I know it's hard without an SSCE... but while working to insulate this byzantine behaviour (I've been trying for days now) maybe someone in the list has a clue or is able to interpret the stack trace? Thanks and have a nice weekend, Marco. [1] Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.3 beta Java: Ubuntu, 13 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) at org.basex.data.Data.kind(Data.java:304) at org.basex.query.value.node.DBNode.(DBNode.java:50) at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) at org.basex.query.expr.Arr.inline(Arr.java:69) at org.basex.query.expr.path.Path.inline(Path.java:962) at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) at org.basex.query.expr.Try.inline(Try.java:106) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) at org.basex.query.scope.MainModule.comp(MainModule.java:81) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) at org.basex.query.QueryContext.compile(QueryContext.java:312) at org.basex.query.QueryContext.iter(QueryContext.java:331) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) at org.basex.http.web.WebResponse.create(WebResponse.java:63) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) at org.basex.http.BaseXServlet.service(BaseXServlet.java:65)
[basex-talk] Database corrupted on updating with RestXQ
Hi all, starting with 9.2.1 we experience a strange error with a RESTXQ API of ours that we have been using for years. The typical pattern is lookup a document update it and store it back. We have done this milions of time and also all the tests work neatly. But when using it from inside the web application, at around the tenth operation we get the DB corrupted with the stack trace [1]. This seems to happen only when the database is nearly empty and, for sure, it doesn't appear on 8.x series. It feels like some concurrency flaw is introduced somewhere but it's hard to say because the API is eight years in age and it never changed significantly. I know it's hard without an SSCE... but while working to insulate this byzantine behaviour (I've been trying for days now) maybe someone in the list has a clue or is able to interpret the stack trace? Thanks and have a nice weekend, Marco. [1] Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.3 beta Java: Ubuntu, 13 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) at org.basex.data.Data.kind(Data.java:304) at org.basex.query.value.node.DBNode.(DBNode.java:50) at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) at org.basex.query.expr.Arr.inline(Arr.java:69) at org.basex.query.expr.path.Path.inline(Path.java:962) at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) at org.basex.query.expr.Try.inline(Try.java:106) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) at org.basex.query.scope.MainModule.comp(MainModule.java:81) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) at org.basex.query.QueryContext.compile(QueryContext.java:312) at org.basex.query.QueryContext.iter(QueryContext.java:331) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) at org.basex.http.web.WebResponse.create(WebResponse.java:63) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:505) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) at
