Hi Anastasiou, :-)
> Am 12.09.2017 um 10:54 schrieb Anastasiou A. <a.anastas...@swansea.ac.uk>: > > Hello everyone > > I am trying to load BaseX with a large number of XML files (~500), each one a > few hundreds of MBs big. > BaseX fails with a message along the lines “This is too big for one database”. => might be that you hit one of our database limits. See the first line here for general limits: http://docs.basex.org/wiki/Statistics <http://docs.basex.org/wiki/Statistics> > > Can I please ask: > > 1) Are there any logs, beyond the DB logs? If yes, where can I find them? > a. The reason I am asking is because once basexgui gives the message, > there is no indication about the error. > Ideally, I would like to know if this is a limitation on memory amount or > number of items (?). The database logs are all there is. You can use the admin module to add your own messages to these logs, http://docs.basex.org/wiki/Admin_Module#admin:write-log <http://docs.basex.org/wiki/Admin_Module#admin:write-log> So for example in a try / catch block you could log more explicitly whats happening :) > > 2) The parser options include reading XML files from archives, which is > very convenient, but once the file has been > parsed, does BaseX require the “originals” for queries / returning results? If you created a database from your archive, i.e. db:create(‚foo.zip‘,…) the original files are no longer needed. http://docs.basex.org/wiki/Database_Module#db:create <http://docs.basex.org/wiki/Database_Module#db:create> > > 3) Is it possible to do federation with BaseX? In other words, let’s say > I split a database in two large parts (as per #1), > is it possible to launch two baseX servers and then have them talk to each > other so that ultimately I just query one of > them and get back unified results? > Yes, it is possible, but only if you are willing to suffer a little :-) You can use the http://docs.basex.org/wiki/Client_Module <http://docs.basex.org/wiki/Client_Module> to connect to a running basex server and you could use the http://docs.basex.org/wiki/XQuery_Module#xquery:fork-join <http://docs.basex.org/wiki/XQuery_Module#xquery:fork-join> to connect to multiple servers at once, pose a query and return the results. But expect to do some stuff by hand: i.e.: - maintain a list of servers you currently have - handle storing, deleting and updating documents - handle document retrieval, … So a probably large bit of manual work is absolutely required, but depending on what you need to do with your documents it might work "well enough“ :-) If you need more information let me know! Best Michael > All the best
