Re: [basex-talk] How to set DBPath in BaseX version 8.6.7 as in version 7.3.1?
I've found the answer for it from this page http://docs.basex.org/wiki/Options
so for the java code in version 8.6.7, it needs to use
System.setProperty("org.basex.dbpath", "/home/xxx/test_basex/");
to set this global path and then, BaseX will read/create collection to this
dbpath correctly.
Bang, Pham Huu
- Research Associate/PhD Student in Computer Science-
Address: Room 96, 2nd Floor, Campus Ring 1, Jacobs University, 28759 Bremen,
Germany.
Email: b.pham...@jacobs-university.de
Tel: +49 174 7096020
"Your best teacher is your last mistake"
Jacobs University,
Address:
Campus Ring 1, 28759 Bremen, Germany
Phone:
+49 421 20040
"Our Focus in research and education"
From: basex-talk-boun...@mailman.uni-konstanz.de
on behalf of Bang Pham Huu
Sent: Friday, January 5, 2018 12:26:01 PM
To: basex-talk@mailman.uni-konstanz.de
Subject: [basex-talk] How to set DBPath in BaseX version 8.6.7 as in version
7.3.1?
Hello,
I'm using BaseX version 7.3.1 and I can set dbpath where the collections
are created with this code:
new Set("dbpath", "/home//basex-database").execute(context);
However, when changing to BaseX version 8.6.7, this statement will throw
exception "org.basex.core.BaseXException: Global option 'DBPATH' cannot
be set."
and instead it tries to create new collections to a default folder:
/home//BaseXData
How can I set BaseX to work on existing dbpath folder
(/home//basex-database) as in version 7.3.1? Because I have existing
data here and not the newly created one.
Thanks,
Re: [basex-talk] BaseX Sharding with ActiveMQ & Docker
Hi James, as to my experience the approach works. I've tried to use N Slaves BaseX (DB only) instances coordinated by one Master BaseX including HTTP server. On the latter I implemented a RestXQ [1] receiving a HTTP request which was then turned into a XQuery and executed on the different backend instances through the client module [2]. I have been able to achieve a scalability of nearly 80% on some particular queries. The results were encouraging indeed but a few details related to parallel computing in general have to be considered. 1) If the query is memory only then you can exploit multi-cores to actually achieve parallelism. If the query has to heavily access DBs on disk (most of the cases actually are io-bound) then you should ensure that your distributed Slaves access different disks or reside on different PCs otherwise all the gain of parallel computation is lost by overhead on disk access. 2) Uploading of documents from a single source usually doesn't exploit data-parallelism because all the data has to be sequentialized through a distributor node first. So if you are able to generate the data from the beginning in a distributed way, this could improve the distribution step too. 3) Another thing to keep in mind is that you should try to perform as much of the work on the Slaves (for instance query and transformation) and limit the work on the coordinator to a bare minimum of computing and memory usage in order to avoid the coordinator to be flooded with the results produced by all your slaves. I achieved this by injecting functions as external variables in the XQ to be executed on the Slaves. Hope this is useful for your work and I'm really looking forward to know how it will proceed! Regards, Marco. [1] http://docs.basex.org/wiki/RESTXQ [2] http://docs.basex.org/wiki/Client_Module On 03/01/2018 19:50, James Sears wrote: In the past I hit a scalability limit with BaseX - a billion+ nodes kind of a made querying it a bit slower than I liked. I thought I'd try and address this, so I’ve written some code and placed it in GitHub: https://github.com/jameshnsears/xqa-documentation What I've done is proof of concept, that's all - no way "finished". I'm emailing the list in the hope that what I've done so far might generate some constructive criticism. Maybe my approach has potential, maybe it doesn't? There are only four components so far, the first three are Docker containers: * an ActiveMQ instance * a load balancer * a shard * a command line client exists to load the XML, from file, into an ActiveMQ queue. The software requires close to zero configuration. For example, each shard you start will automatically receive XML from the load balancer. And the load balancer distributes XML so that each shard holds the same # of documents. There's a Travis project associated with the above link - it shows how easy it is to run the software end to end. So far my effort is all about ingesting the XML, before I move further I thought I'd canvass some feedback - so if anyone has any then please give it :-) Thanks.
[basex-talk] How to set DBPath in BaseX version 8.6.7 as in version 7.3.1?
Hello,
I'm using BaseX version 7.3.1 and I can set dbpath where the collections
are created with this code:
new Set("dbpath", "/home//basex-database").execute(context);
However, when changing to BaseX version 8.6.7, this statement will throw
exception "org.basex.core.BaseXException: Global option 'DBPATH' cannot
be set."
and instead it tries to create new collections to a default folder:
/home//BaseXData
How can I set BaseX to work on existing dbpath folder
(/home//basex-database) as in version 7.3.1? Because I have existing
data here and not the newly created one.
Thanks,
Re: [basex-talk] Hundreds of thousands of java.io.IOException: Stream Closed in BaseX 7.3.1 from only catalina.out of Tomcat's log?
Hello Bang, I am going to assume that you use the quite outdated version 7.3.1 because it was the only one in the Maven Central repository at the time. Since a few months we improved the build process and new release are now also available in Maven Central (but also in the BaseX internal Maven Repository). So the first thing I would recommend is for you to upgrade to a recent 8.6.x version. Maybe this bug is already solved in the meantime. Cheers Dirk Senacor Technologies Aktiengesellschaft - Sitz: Eschborn - Amtsgericht Frankfurt am Main - Reg.-Nr.: HRB 105546 Vorstand: Matthias Tomann, Marcus Purzer - Aufsichtsratsvorsitzender: Daniel Gr?zinger -Urspr?ngliche Nachricht- Von: basex-talk-boun...@mailman.uni-konstanz.de [mailto:basex-talk-boun...@mailman.uni-konstanz.de] Im Auftrag von Bang Pham Huu Gesendet: Donnerstag, 4. Januar 2018 10:30 An: basex-talk@mailman.uni-konstanz.de Betreff: [basex-talk] Hundreds of thousands of java.io.IOException: Stream Closed in BaseX 7.3.1 from only catalina.out of Tomcat's log? Hello, I've been using BaseX (version 7.3.1 with Tomcat 8) to store an XML database from XML files and using XQuery to query the database. It works great before with simple Java servlet as I recall. After that, I updated to use Spring Boot (version 1.5.2) for this web application (only update from Java servlet to Spring Controller, the core to query BaseX doesn't change), and it seems this weird exception java.io.IOException: Stream Closed appears randomly by 10 times in Tomcat's log (catalina.out). I tried to catch this exception in my web application but it is not able to catch. My web application's log (secore.log) also couldn't catch this error, only Tomcat's log (catalina.out) logs this error but with 10 times (the log file is huge). Any idea to solve this problem will help me a lot? Here is the stack trace in catalina.out for 1 exception: https://pastebin.com/XGu7Vr2f And here is the configuration for log4j to log to both console and file (secore.log) and only console (catalina.out) contains this exception: https://pastebin.com/09ma3eyw Thanks,
