Hello Johannes, our best practice for similar scenarios is to have two instances of basex running. One that is mainly collecting data from various sources (http api, ftp, filesystem) and another instance which then gets the collected infos all at once.
BaseX really shines at processing larger chunks (and querying big documents) - think batching. But it is not well suited for hundreds of updating requests at the same time. Especially not if you have a user interface that should stay responsive at all times. Greetings from Lindau, Max Gärber Am Di., 19. Feb. 2019 um 14:53 Uhr schrieb Johannes Bauer <johannes.ba...@tanner.de>: > > Hi, > > I need some input on how best to implement an updating database operation. > > I use BaseX within a Web Application and have RESTXQ service endpoints. > > One such service collects information from multiple databases and > creates a new document (with the collected infos) in another database. > The list of affected databases is dynamic and will be determined by the > RESTXQ module. > > I'm afraid that, due to the missing static database information in the > query, the service will lock *all* databases and because I have an > updating operation the locks will be write locks. > Depending on the amount of databases and their size, the operation might > take some time (maybe even up to some minutes). And I don't want to have > write locks for such a long period of time. > > I've also tried another approach: > > I use a non updating service that collects the data and then calls > another RESTXQ service per HTTP request that makes the db update. > This way I only have read locks on *all* databases and then a short > single write lock on the modified db. > > But this did not work. The write call deadlocked. The first service > probably has not released the read locks yet when the second service is > called. > > I also tried to set the QUERYINFO option to see some lock information > for my module, but I didn't get any output. > > Do you have any best practice on how to approach such a task? > > > Best regards > Johannes >
