Hi,
we have a rather strange and hard to track problem with corrupted databases.
Our setup is:
* Docker container with a Tomcat that hosts BaseX with some custom
RESTXQ services
* BaseX 9.2.4
* Java 14
* Docker runs on a Linux VM
Workflow
* Create database with RESTXQ service call
* Import JSON document with RESTXQ service call
o call this multiple times.
After some Import calls, the import fails and the database is corrupt
from this point on.
We first thought that it has something to do with the content of the
document. But we found no pattern. Sometimes it works, but sometime it
does not.
There is no concurrency involved. There are no other clients that read
or write to the database.
We also tried to deactivate the UPDINDEX setting. But it had no effect
and we could reproduce the error with and without the automatic index
update.
The logs in case of errors look like this:
06:58:00.683 172.18.0.2:33728 admin REQUEST [PUT]
/c42-core/api/v1/restxq/user/documents/c42-index/metadata%40document06:58:00.700
172.18.0.2:33728 admin 500 Unexpected error: Improper use? Potential
bug? Your feedback is welcome: Contact:
basex-talk@mailman.uni-konstanz.de Version: BaseX 9.2.4 Java: Oracle
Corporation, 14.0.1 OS: Linux, amd64 Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: Index 4 out of bounds for
length 1 at
org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) at
org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:467) at
org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) at
org.basex.data.Data.kind(Data.java:304) at
org.basex.query.up.DataUpdates.prepare(DataUpdates.java:133) at
org.basex.query.up.ContextModifier.prepare(ContextModifier.java:90) at
org.basex.query.up.Updates.prepare(Updates.java:168) at
org.basex.query.QueryContext.update(QueryContext.java:678) at
org.basex.query.QueryContext.iter(QueryContext.java:332) at
org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73)
at org.basex.http.web.WebResponse.create(WebResponse.java:63) at
org.basex.http... 16.59 ms Our service does not much. It just calls
db:replace(). declare variable $documents:IMPORT_OPTS := map {'chop':
fn:false(), 'stripns': fn:false(), 'intparse':
fn:true()};declare%rest:PUT("{$xml}")%rest:consumes("application/xml")%rest:produces("application/json")%rest:path("/user/documents/{$databaseId}/{$documentId}")%updatingfunction
documents:create($databaseId as xs:string, $documentId as xs:string,
$xml as document-node()){if (db:exists($databaseId)) then
(update:output(response:empty(204, ())),db:replace($databaseId,
documents:decode($documentId), $xml, $documents:IMPORT_OPTS))else
(update:output(response:json(errors:error('C42UDO002', map
{'databaseId': $databaseId}), 404)), ())}; I've attached the example
input document. One addition: We could not reproduce this error running
the Docker container on a Windows host. Any feedback or hints to solve
this are greatly appreciated. Best regards Johannes
[
{
"databaseid": "c42-content",
"documentid": "doc_26424521995_de-DE",
"metadata": [
{
"id": "document",
"name": "Document ID",
"values": [
"doc_26424521995_de-DE"
]
},
{
"id": "media",
"name": "Media Document ID",
"values": [
"media_26424521995_de-DE"
]
},
{
"id": "projectId",
"name": "Project ID",
"values": [
"26424521995"
]
},
{
"id": "lang",
"name": "Language",
"values": [
"de-DE"
]
},
{
"id": "sysTitle",
"name": "System Title",
"values": [
"Kompaktleistungsschalter 3VA mit IEC-Zertifikat"
]
},
{
"id": "type",
"name": "Type",
"values": [
"Gerätehandbuch"
]
},
{
"id": "system",
"name": "System",
"values": [
"SENTRON"
]
},
{
"id": "productGroup",
"name": "Product Group",
"values": [
"Schutzgeräte"
]
},
{
"id": "importDate",
"name": "Import Date",
"values": [
"2020-07-16T06:10:50.405Z"
]
}
]
},
{
"databaseid": "c42-content",
"documentid": "doc_26424521995_en-US",
"metadata": [
{
"id": "document",
"name": "Document ID",
"values": [
"doc_26424521995_en-US"
]
},
{
"id": "media",
"name": "Media Document ID",
"values": [
"media_26424521995_en-US"
]
},
{
"id": "projectId",
"name": "Project ID",
"values": [
"26424521995"
]
},
{
"id": "lang",
"name": "Language",
"values": [
"en-US"
]
},
{
"id": "sysTitle",
"name": "System Title",
"values": [
"3VA molded case circuit breakers with IEC certificate"
]
},
{
"id": "type",
"name": "Type",
"values": [
"Manual"
]
},
{
"id": "system",
"name": "System",
"values": [
""
]
},
{
"id": "productGroup",
"name": "Product Group",
"values": [
""
]
},
{
"id": "importDate",
"name": "Import Date",
"values": [
"2020-06-26T10:06:16.589Z"
]
}
]
},
{
"databaseid": "c42-content",
"documentid": "doc_118588327179_de-DE",
"metadata": [
{
"id": "document",
"name": "Document ID",
"values": [
"doc_118588327179_de-DE"
]
},
{
"id": "media",
"name": "Media Document ID",
"values": [
"media_118588327179_de-DE"
]
},
{
"id": "projectId",
"name": "Project ID",
"values": [
"118588327179"
]
},
{
"id": "lang",
"name": "Language",
"values": [
"de-DE"
]
},
{
"id": "sysTitle",
"name": "System Title",
"values": [
"Offener Leistungsschalter 3WA"
]
},
{
"id": "type",
"name": "Type",
"values": [
"Systemhandbuch"
]
},
{
"id": "system",
"name": "System",
"values": [
"SENTRON"
]
},
{
"id": "productGroup",
"name": "Product Group",
"values": [
"Schutzgeräte"
]
},
{
"id": "importDate",
"name": "Import Date",
"values": [
"2020-07-16T06:11:42.026Z"
]
}
]
},
{
"databaseid": "c42-content",
"documentid": "doc_10211322123_de-DE",
"metadata": [
{
"id": "document",
"name": "Document ID",
"values": [
"doc_10211322123_de-DE"
]
},
{
"id": "media",
"name": "Media Document ID",
"values": [
"media_10211322123_de-DE"
]
},
{
"id": "projectId",
"name": "Project ID",
"values": [
"10211322123"
]
},
{
"id": "lang",
"name": "Language",
"values": [
"de-DE"
]
},
{
"id": "sysTitle",
"name": "System Title",
"values": [
"PAC4200"
]
},
{
"id": "type",
"name": "Type",
"values": [
"Systemhandbuch"
]
},
{
"id": "system",
"name": "System",
"values": [
"SENTRON"
]
},
{
"id": "productGroup",
"name": "Product Group",
"values": [
"Multifunktionsmessgerät"
]
},
{
"id": "importDate",
"name": "Import Date",
"values": [
"2020-07-16T06:32:17.543Z"
]
}
]
},
{
"databaseid": "c42-content",
"documentid": "doc_10211322123_en-US",
"metadata": [
{
"id": "document",
"name": "Document ID",
"values": [
"doc_10211322123_en-US"
]
},
{
"id": "media",
"name": "Media Document ID",
"values": [
"media_10211322123_en-US"
]
},
{
"id": "projectId",
"name": "Project ID",
"values": [
"10211322123"
]
},
{
"id": "lang",
"name": "Language",
"values": [
"en-US"
]
},
{
"id": "sysTitle",
"name": "System Title",
"values": [
"PAC4200"
]
},
{
"id": "type",
"name": "Type",
"values": [
"System Manual"
]
},
{
"id": "system",
"name": "System",
"values": [
"SENTRON"
]
},
{
"id": "productGroup",
"name": "Product Group",
"values": [
"Power Monitoring Device"
]
},
{
"id": "importDate",
"name": "Import Date",
"values": [
"2020-07-16T06:32:26.085Z"
]
}
]
},
{
"databaseid": "c42-content",
"documentid": "doc_10211322123_es-MX",
"metadata": [
{
"id": "document",
"name": "Document ID",
"values": [
"doc_10211322123_es-MX"
]
},
{
"id": "media",
"name": "Media Document ID",
"values": [
"media_10211322123_es-MX"
]
},
{
"id": "projectId",
"name": "Project ID",
"values": [
"10211322123"
]
},
{
"id": "lang",
"name": "Language",
"values": [
"es-MX"
]
},
{
"id": "sysTitle",
"name": "System Title",
"values": [
"PAC4200"
]
},
{
"id": "type",
"name": "Type",
"values": [
"Manual de sistema"
]
},
{
"id": "system",
"name": "System",
"values": [
"SENTRON"
]
},
{
"id": "productGroup",
"name": "Product Group",
"values": [
"Multímetro"
]
},
{
"id": "importDate",
"name": "Import Date",
"values": [
"2020-07-16T06:32:35.532Z"
]
}
]
},
{
"databaseid": "c42-content",
"documentid": "doc_10211322123_fr-FR",
"metadata": [
{
"id": "document",
"name": "Document ID",
"values": [
"doc_10211322123_fr-FR"
]
},
{
"id": "media",
"name": "Media Document ID",
"values": [
"media_10211322123_fr-FR"
]
},
{
"id": "projectId",
"name": "Project ID",
"values": [
"10211322123"
]
},
{
"id": "lang",
"name": "Language",
"values": [
"fr-FR"
]
},
{
"id": "sysTitle",
"name": "System Title",
"values": [
"PAC4200"
]
},
{
"id": "type",
"name": "Type",
"values": [
"Manuel système"
]
},
{
"id": "system",
"name": "System",
"values": [
"SENTRON"
]
},
{
"id": "productGroup",
"name": "Product Group",
"values": [
"Multimètre"
]
},
{
"id": "importDate",
"name": "Import Date",
"values": [
"2020-07-16T06:32:41.861Z"
]
}
]
},
{
"databaseid": "c42-content",
"documentid": "doc_10211322123_it-IT",
"metadata": [
{
"id": "document",
"name": "Document ID",
"values": [
"doc_10211322123_it-IT"
]
},
{
"id": "media",
"name": "Media Document ID",
"values": [
"media_10211322123_it-IT"
]
},
{
"id": "projectId",
"name": "Project ID",
"values": [
"10211322123"
]
},
{
"id": "lang",
"name": "Language",
"values": [
"it-IT"
]
},
{
"id": "sysTitle",
"name": "System Title",
"values": [
"PAC4200"
]
},
{
"id": "type",
"name": "Type",
"values": [
"Manuale di sistema"
]
},
{
"id": "system",
"name": "System",
"values": [
"SENTRON"
]
},
{
"id": "productGroup",
"name": "Product Group",
"values": [
"Multimetro"
]
},
{
"id": "importDate",
"name": "Import Date",
"values": [
"2020-07-16T06:32:48.277Z"
]
}
]
},
{
"databaseid": "c42-content",
"documentid": "doc_10211322123_pt-BR",
"metadata": [
{
"id": "document",
"name": "Document ID",
"values": [
"doc_10211322123_pt-BR"
]
},
{
"id": "media",
"name": "Media Document ID",
"values": [
"media_10211322123_pt-BR"
]
},
{
"id": "projectId",
"name": "Project ID",
"values": [
"10211322123"
]
},
{
"id": "lang",
"name": "Language",
"values": [
"pt-BR"
]
},
{
"id": "sysTitle",
"name": "System Title",
"values": [
"PAC4200"
]
},
{
"id": "type",
"name": "Type",
"values": [
"Manual do sistema"
]
},
{
"id": "system",
"name": "System",
"values": [
"SENTRON"
]
},
{
"id": "productGroup",
"name": "Product Group",
"values": [
"Multimedidor"
]
},
{
"id": "importDate",
"name": "Import Date",
"values": [
"2020-07-16T06:32:54.732Z"
]
}
]
},
{
"databaseid": "c42-content",
"documentid": "doc_10211322123_ru-RU",
"metadata": [
{
"id": "document",
"name": "Document ID",
"values": [
"doc_10211322123_ru-RU"
]
},
{
"id": "media",
"name": "Media Document ID",
"values": [
"media_10211322123_ru-RU"
]
},
{
"id": "projectId",
"name": "Project ID",
"values": [
"10211322123"
]
},
{
"id": "lang",
"name": "Language",
"values": [
"ru-RU"
]
},
{
"id": "sysTitle",
"name": "System Title",
"values": [
"PAC4200"
]
},
{
"id": "type",
"name": "Type",
"values": [
"Системное руководство"
]
},
{
"id": "system",
"name": "System",
"values": [
"SENTRON"
]
},
{
"id": "productGroup",
"name": "Product Group",
"values": [
"Многофункциональное измерительное устройство"
]
},
{
"id": "importDate",
"name": "Import Date",
"values": [
"2020-07-16T06:33:03.412Z"
]
}
]
},
{
"databaseid": "c42-content",
"documentid": "doc_10211322123_tr-TR",
"metadata": [
{
"id": "document",
"name": "Document ID",
"values": [
"doc_10211322123_tr-TR"
]
},
{
"id": "media",
"name": "Media Document ID",
"values": [
"media_10211322123_tr-TR"
]
},
{
"id": "projectId",
"name": "Project ID",
"values": [
"10211322123"
]
},
{
"id": "lang",
"name": "Language",
"values": [
"tr-TR"
]
},
{
"id": "sysTitle",
"name": "System Title",
"values": [
"PAC4200"
]
},
{
"id": "type",
"name": "Type",
"values": [
"Sistem kitabı"
]
},
{
"id": "system",
"name": "System",
"values": [
"SENTRON"
]
},
{
"id": "productGroup",
"name": "Product Group",
"values": [
"Enerji analizörü"
]
},
{
"id": "importDate",
"name": "Import Date",
"values": [
"2020-07-16T06:33:06.624Z"
]
}
]
},
{
"databaseid": "c42-content",
"documentid": "doc_10211322123_zh-CHS",
"metadata": [
{
"id": "document",
"name": "Document ID",
"values": [
"doc_10211322123_zh-CHS"
]
},
{
"id": "media",
"name": "Media Document ID",
"values": [
"media_10211322123_zh-CHS"
]
},
{
"id": "projectId",
"name": "Project ID",
"values": [
"10211322123"
]
},
{
"id": "lang",
"name": "Language",
"values": [
"zh-CHS"
]
},
{
"id": "sysTitle",
"name": "System Title",
"values": [
"PAC4200"
]
},
{
"id": "type",
"name": "Type",
"values": [
"系统手册"
]
},
{
"id": "system",
"name": "System",
"values": [
"SENTRON"
]
},
{
"id": "productGroup",
"name": "Product Group",
"values": [
"电力监测设备"
]
},
{
"id": "importDate",
"name": "Import Date",
"values": [
"2020-07-16T06:33:09.842Z"
]
}
]
}
]
