Re: [basex-talk] Overwrite databases via REST interface
Hi Björn, The BaseX admin user can be compared with the Unix root user. In productive environments, you should avoid the admin account and create additional, task-specific users. For example, you may want to create a user, which only has read/update permissions for specific databases [1]. Hope this helps, Christian [1] http://docs.basex.org/wiki/User_Management On Wed, Jan 15, 2020 at 3:31 PM Braunschweig, Björn wrote: > > Dear list, > > I have a use case where multiple distributed processes make use of a shared > basex instance using the REST interface. Unfortunately executing PUT > /rest/mydb creates an empty database regardless of whether it contains files > or not. Is there a config option or something else to not overwrite existing > databases? Otherwise I have to implement a locking mechanism. > > Regards, > Björn
[basex-talk] Overwrite databases via REST interface
Dear list, I have a use case where multiple distributed processes make use of a shared basex instance using the REST interface. Unfortunately executing PUT /rest/mydb creates an empty database regardless of whether it contains files or not. Is there a config option or something else to not overwrite existing databases? Otherwise I have to implement a locking mechanism. Regards, Björn smime.p7s Description: S/MIME cryptographic signature
Re: [basex-talk] slow search performance across multiple databases
Hi Björn, hi Omar,
All filter expressions that yield document nodes in distinct document
order will now be rewritten to path expressions. Feel free to check
out the latest stable snapshot [1].
Christian
[1] http://files.basex.org/releases/latest/
On Wed, Jan 15, 2020 at 9:22 AM Christian Grün
wrote:
>
> Hi Björn,
>
> > [2] Does not use index
> > declare namespace lmerFile = "http://www.ddb.de/LMERfile;;
> > count(
> > (# db:enforceindex #) {
> > let $dbs := ("00", "01")
> > for $db in $dbs
> > for $doc in db:open($db)
> > where $doc//lmerFile:format/text() = 'urn:123'
> > return $doc
> > })
>
> The index rewriting should take effect if you e.g. rewrite
> db:open($db)/* instead of db:open($db). The background:
>
> * The where clause will be attached to the for expression as predicate
> * As thefor expression is a simple function call, the result will be a
> filter expression: db:open($db)[.//lmerFile:format/text() = 'urn:123']
> * If the for expression is a path expression, the where clause will be
> attached to the last predicate of this path. The result will be a path
> again.
> * Only path expressions are rewritten to index access, no filters.
>
> But thanks for your observation. We will check if we can implicitly
> rewrite your function call to a path expression if it’s embedded in an
> iteration like yours.
>
> Best,
> Christian
>
>
> > Von: BaseX-Talk Im Auftrag von
> > Omar Siam
> > Gesendet: Dienstag, 14. Januar 2020 17:06
> > An: basex-talk@mailman.uni-konstanz.de
> > Betreff: Re: [basex-talk] slow search performance across multiple databases
> >
> > Hi,
> >
> > I have trouble seeing the original second query.
> >
> > But if I got it correct then the problem is that to have BaseX
> > automatically rewrite for indexing you have to supply the DB you want to
> > search in as string literal. That is db:open("DB") and not let $db := "DB"
> > db:open($db). Or for with multiple dbs for that matter.
> >
> > Maybe I am wrong and the enforceindex pragma solves that now. I have to
> > admit my code predates the pragma and always uses string literals when
> > accessing a database.
> >
> > Best regards
> >
> > Omar
> >
> > Am 14.01.2020 um 13:33 schrieb Braunschweig, Björn:
> > > Dear Christian,
> > >
> > > Thank you for your quick response.
> > >
> > > Could you tell me why the following two queries have a difference in
> > > behaviour so that the first one apparently uses the text index [1] and
> > > the second one [2] does not?
> > >
> > > Your second suggestion with db:text is nice but I cannot figure out how
> > > to efficiently extract other informations like something from the root
> > > element without crawling the whole database and performing a full table
> > > scan so to say.
> > >
> > > Regards,
> > > Björn
> > >
> > > [1]
> > > declare namespace lmerFile = "http://www.ddb.de/LMERfile;; count( (#
> > > db:enforceindex #) {
> > >let $dbs := ("00")
> > >for $db in $dbs
> > >for $doc in db:open($db)
> > >where $doc//lmerFile:format/text() =
> > > 'urn:diasid:fty:kopal:00'
> > >return $doc
> > > })
> > >
> > > Compiling:
> > > - merge steps: descendant::element(lmerFile:format)
> > > - rewrite for to let: for $db_1 in $dbs_0
> > > - inline $dbs_0
> > > - inline $db_1
> > > - pre-evaluate db:open(database[,path]) to document-node() sequence:
> > > db:open("00") -> (db:open-pre("00", 0), ...)
> > > - apply text index for "urn:123"
> > > - rewrite cached filter to cached path: ((db:open-pre("00", 0),
> > > ...))[(descendant::element(lmerFile:format)/text() = "urn:123... ->
> > > db:text("00", "urn:123")/parent::element(lmerFile:format)/ancest...
> > > - rewrite where clause(s)
> > > - simplify FLWOR expression: db:text("00",
> > > "urn:123")/parent::element(lmerFile:format)/ancest...
> > > Optimized Query:
> > > count((# Q{http://basex.org/modules/db}enforceindex #) { db:text("00",
> > > "urn:123")/parent::element(lmerFile:format)/ancestor::node()[self::doc
> > > ument-node()] })
> > > Query:
> > > declare namespace lmerFile = "http://www.ddb.de/LMERfile;; count( (#
> > > db:enforceindex #) { let $dbs := ("00") for $db in $dbs for $doc in
> > > db:open($db) where $doc//lmerFile:format/text() = 'urn:123' return
> > > $doc })
> > > Result:
> > > - Hit(s): 1 Item
> > > - Updated: 0 Items
> > > - Printed: 4 b
> > > - Read Locking: (global)
> > > - Write Locking: (none)
> > > Timing:
> > > - Parsing: 0.26 ms
> > > - Compiling: 55.69 ms
> > > - Evaluating: 60.66 ms
> > > - Printing: 0.37 ms
> > > - Total Time: 116.98 ms
> > > Query Plan:
> > >
> > >
> > >
> > >
> > > db:enforceindex
> > >
> > >
> > >
> > >
> > >urn:123
> > >
> > > > > type="element()?"/>
> > >
> > >
> > > > > type="document-node()?"/>
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > [2]
> > >
> > >
Re: [basex-talk] slow search performance across multiple databases
Hi Björn,
> [2] Does not use index
> declare namespace lmerFile = "http://www.ddb.de/LMERfile;;
> count(
> (# db:enforceindex #) {
> let $dbs := ("00", "01")
> for $db in $dbs
> for $doc in db:open($db)
> where $doc//lmerFile:format/text() = 'urn:123'
> return $doc
> })
The index rewriting should take effect if you e.g. rewrite
db:open($db)/* instead of db:open($db). The background:
* The where clause will be attached to the for expression as predicate
* As thefor expression is a simple function call, the result will be a
filter expression: db:open($db)[.//lmerFile:format/text() = 'urn:123']
* If the for expression is a path expression, the where clause will be
attached to the last predicate of this path. The result will be a path
again.
* Only path expressions are rewritten to index access, no filters.
But thanks for your observation. We will check if we can implicitly
rewrite your function call to a path expression if it’s embedded in an
iteration like yours.
Best,
Christian
> Von: BaseX-Talk Im Auftrag von
> Omar Siam
> Gesendet: Dienstag, 14. Januar 2020 17:06
> An: basex-talk@mailman.uni-konstanz.de
> Betreff: Re: [basex-talk] slow search performance across multiple databases
>
> Hi,
>
> I have trouble seeing the original second query.
>
> But if I got it correct then the problem is that to have BaseX automatically
> rewrite for indexing you have to supply the DB you want to search in as
> string literal. That is db:open("DB") and not let $db := "DB" db:open($db).
> Or for with multiple dbs for that matter.
>
> Maybe I am wrong and the enforceindex pragma solves that now. I have to admit
> my code predates the pragma and always uses string literals when accessing a
> database.
>
> Best regards
>
> Omar
>
> Am 14.01.2020 um 13:33 schrieb Braunschweig, Björn:
> > Dear Christian,
> >
> > Thank you for your quick response.
> >
> > Could you tell me why the following two queries have a difference in
> > behaviour so that the first one apparently uses the text index [1] and the
> > second one [2] does not?
> >
> > Your second suggestion with db:text is nice but I cannot figure out how to
> > efficiently extract other informations like something from the root element
> > without crawling the whole database and performing a full table scan so to
> > say.
> >
> > Regards,
> > Björn
> >
> > [1]
> > declare namespace lmerFile = "http://www.ddb.de/LMERfile;; count( (#
> > db:enforceindex #) {
> >let $dbs := ("00")
> >for $db in $dbs
> >for $doc in db:open($db)
> >where $doc//lmerFile:format/text() =
> > 'urn:diasid:fty:kopal:00'
> >return $doc
> > })
> >
> > Compiling:
> > - merge steps: descendant::element(lmerFile:format)
> > - rewrite for to let: for $db_1 in $dbs_0
> > - inline $dbs_0
> > - inline $db_1
> > - pre-evaluate db:open(database[,path]) to document-node() sequence:
> > db:open("00") -> (db:open-pre("00", 0), ...)
> > - apply text index for "urn:123"
> > - rewrite cached filter to cached path: ((db:open-pre("00", 0),
> > ...))[(descendant::element(lmerFile:format)/text() = "urn:123... ->
> > db:text("00", "urn:123")/parent::element(lmerFile:format)/ancest...
> > - rewrite where clause(s)
> > - simplify FLWOR expression: db:text("00",
> > "urn:123")/parent::element(lmerFile:format)/ancest...
> > Optimized Query:
> > count((# Q{http://basex.org/modules/db}enforceindex #) { db:text("00",
> > "urn:123")/parent::element(lmerFile:format)/ancestor::node()[self::doc
> > ument-node()] })
> > Query:
> > declare namespace lmerFile = "http://www.ddb.de/LMERfile;; count( (#
> > db:enforceindex #) { let $dbs := ("00") for $db in $dbs for $doc in
> > db:open($db) where $doc//lmerFile:format/text() = 'urn:123' return
> > $doc })
> > Result:
> > - Hit(s): 1 Item
> > - Updated: 0 Items
> > - Printed: 4 b
> > - Read Locking: (global)
> > - Write Locking: (none)
> > Timing:
> > - Parsing: 0.26 ms
> > - Compiling: 55.69 ms
> > - Evaluating: 60.66 ms
> > - Printing: 0.37 ms
> > - Total Time: 116.98 ms
> > Query Plan:
> >
> >
> >
> >
> > db:enforceindex
> >
> >
> >
> >
> >urn:123
> >
> > > type="element()?"/>
> >
> >
> > > type="document-node()?"/>
> >
> >
> >
> >
> >
> >
> >
> > [2]
> >
> > Compiling:
> > - pre-evaluate expression list to xs:string sequence: ("00", "01")
> > - merge steps: descendant::element(lmerFile:format)
> > - inline $dbs_0
> > - rewrite where clause(s)
> > Optimized Query:
> > count((# Q{http://basex.org/modules/db}enforceindex #) { for $db_1 in
> > ("00", "01") return
> > (db:open($db_1))[(descendant::element(lmerFile:format)/text() =
> > "urn:123")] })
> > Query:
> > declare namespace lmerFile = "http://www.ddb.de/LMERfile;; count( (#
> > db:enforceindex #) { let $dbs := ("00", "01") for $db in $dbs for $doc
> > in db:open($db) where
