Re: [basex-talk] Performance issue with BaseX CLI

-konstanz.de Objet : Re: [basex-talk] Performance issue with BaseX CLI Hi again, I had a quick look into the monitoring code, and I noticed two things: 1. It looks to me (correct me if I’m wrong) as if the code of the project was initially written for Saxon and then ported to BaseX. If you

Re: [basex-talk] Performance issue with BaseX CLI

>> *Stop* >> >> *Elapse time* >> >> *Start* >> >> *Stop* >> >> *Elapse time* >> >> Check Monitoring 2022 FRH >> >> 06:16:54 >> >> 06:19:30 >> >> 00:02:36 >> >> 06:44:06 >> >>

Re: [basex-talk] Performance issue with BaseX CLI

:31:47 > > 00:06:01 > > 10:05:55 > > 11:39:07 > > 01:33:12 > > > > > > *De :* Liam R. E. Quin > *Envoyé :* samedi 20 avril 2024 05:00 > *À :* ANDRADE Antonio ; > basex-talk@mailman.uni-konstanz.de > *Objet :* Re: [basex-talk] Performance issue wi

Re: [basex-talk] Performance issue with BaseX CLI

On Mon, 2024-04-22 at 08:54 +0200, ANDRADE Antonio wrote: > At this moment, the XML document is not intended to be stored. This > is why it is not loaded into a database before processing. BaseX is designed to operate primarily on documents in the database, which is why i suggest trying that.

Re: [basex-talk] Performance issue with BaseX CLI

an.uni-konstanz.de Objet : Re: [basex-talk] Performance issue with BaseX CLI On Fri, 2024-04-19 at 10:45 +0200, ANDRADE Antonio wrote: Hie, For the purposes of European Water Framework Directive reporting, I compared the performances of the Saxon and BaseX XQuery engines. First, you should consider

Re: [basex-talk] Performance issue with BaseX CLI

On Fri, 2024-04-19 at 10:45 +0200, ANDRADE Antonio wrote: > Hie, >   > For the purposes of European Water Framework Directive reporting, I > compared the performances of the Saxon and BaseX XQuery engines. First, you should consider (as i think Martin said) the Java runtime startup time,

Re: [basex-talk] Performance issue with BaseX CLI

. The Python/bash difference for the calling script does not seem to explain the observed performance differences. De : Hans-Juergen Rennau Envoyé : vendredi 19 avril 2024 11:25 À : basex-talk@mailman.uni-konstanz.de; ANDRADE Antonio Objet : Re: [basex-talk] Performance issue with BaseX CLI Hi

Re: [basex-talk] Performance issue with BaseX CLI

Am 19.04.2024 um 10:45 schrieb ANDRADE Antonio: Hie, For the purposes of European Water Framework Directive reporting, I compared the performances of the Saxon and BaseX XQuery engines. I observe a performance gap of a factor of 100 to 200 depending on the use case (see functions

Re: [basex-talk] Performance issue with BaseX CLI

Hi Antonio, my experience is very different - quite comparable performance, except for very specific cases, e.g. massive use of fn:idref(). Furthermore, the performance of BaseX is often so stupendous that an improvement by an order of magnitude (not to mention two) appears to me very

[basex-talk] Performance issue with BaseX CLI

Hie, For the purposes of European Water Framework Directive reporting, I compared the performances of the Saxon and BaseX XQuery engines. I observe a performance gap of a factor of 100 to 200 depending on the use case (see functions test_xquery_monitoring() and test_xquery_multischema_2022()

Re: [basex-talk] Performance of db:get-value() for large maps

AM To: Eliot Kimber Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Performance of db:get-value() for large maps [External Email] Hi Eliot, When stored XQuery values are requested, they are always fully materialized in main-memory. Depending on the size, that may take a while. T

Re: [basex-talk] Performance of db:get-value() for large maps

Hi Eliot, When stored XQuery values are requested, they are always fully materialized in main-memory. Depending on the size, that may take a while. The following query can be used to create a map with 1 million entries and store it in a database. It takes around 1200 ms on my machine: let $data

[basex-talk] Performance of db:get-value() for large maps

I’m working on constructing DITA key spaces for our content. My current implementation builds an XQuery map that contains the key space data as well as the XML data from which the key space was constructed, which can be quite a bit (1-2 megabytes of XML all told—key space construction

Re: [basex-talk] Performance of ft:search function

Exactly: The longer you run a BaseX instance, the faster it gets. That’s particularly noticeable when using the client/server or HTTP architecture. There are various reasons for that: BaseX caches, OS & main-memory caching, JIT optimizations, … Tim Thompson schrieb am Fr., 29. Apr. 2022,

Re: [basex-talk] Performance of ft:search function

Oh, I see--thanks for the tip; I wasn't aware of the SET RUNS feature, which is really helpful! With 1000 runs, the average execution time is more in line with expectations: 38.96ms for expression #1 and 12.44ms for #2. But I notice that with successive executions, #1 gets faster: 38.96ms,

Re: [basex-talk] Performance of ft:search function

> > 2. Direct lookup against subindex > Time: 3.3ms > Expression: ft:search($index, $text)/../.. > > 3. Lookup against subindex file with reference to large index > Time: 2.9ms > Expression: > let $s := > ft:search($index, $text)/../.. > return db:open-id($db, $s/id)/../.. > > My question is:

[basex-talk] Performance of ft:search function

Hello, I have a largish (5.4G) file with a full-text index that I am using to reconcile names in a local dataset. I've been experimenting with splitting the file into many smaller index files to improve performance. I group the entries by initial character and create a new index file for each

Re: [basex-talk] Performance issue on copy-modify-return when too many items are being replaced

Forwarding to the mailing list in order to share knowledge. On Fri, Nov 12, 2021 at 1:41 PM BaseX Support wrote: > Hi France, > > I’d need to get my hands on your code to tell you exactly where it’s > best used, but I can give you some more details on the XQuery > specification: > > When

Re: [basex-talk] Performance: first and second xQuery execution / local PC and server

Hi Christian, thank you very much for your answer! You were totally right: After some tests with lower Xmx value (3700m) I can confirm that the xquery executes much faster on the virtual server (about 15 minutes). I have no control over the VMware config ... I just was provided with the

[basex-talk] Performance: first and second xQuery execution / local PC and server

Hello BaseX-Community, I have a question concerning performance ... once again ... sorry ;-) First thing I noticed is that when I run a query that involves quite a lot of data (more information below), the first execution of the query takes much more time as the second execution. In my

Re: [basex-talk] Performance loss between version 9.2.4 and 9.3.2 when executing specific xQuery

ps://arbeiterkammer.at/100><https://arbeiterkammer.at/100><https://w.ak.at/zukunftsprogramm> Von: Christian Grün Gesendet: Montag, 11. Mai 2020 13:02 An: BIRKNER Michael Cc: basex-talk@mailman.uni-konstanz.de Betreff: Re: [basex-talk] Performance loss

Re: [basex-talk] Performance loss between version 9.2.4 and 9.3.2 when executing specific xQuery

Hi Michael, I checked your use case in greater depth, and I found the change in our code that caused the slowdown [1]. A) The nutshell answer : Just use the attached query! B) The extensive technical answer: • In previous versions of BaseX, most paths in FLWOR expressions were »inlined« in the

Re: [basex-talk] Performance loss between version 9.2.4 and 9.3.2 when executing specific xQuery

------ > *Von:* Christian Grün > *Gesendet:* Freitag, 8. Mai 2020 14:24 > *An:* BIRKNER Michael > *Cc:* basex-talk@mailman.uni-konstanz.de > *Betreff:* Re: [basex-talk] Performance loss between version 9.2.4 and > 9.3.2 when executing specific xQuery > &

Re: [basex-talk] Performance loss between version 9.2.4 and 9.3.2 when executing specific xQuery

__ Von: Christian Grün Gesendet: Freitag, 8. Mai 2020 14:24 An: BIRKNER Michael Cc: basex-talk@mailman.uni-konstanz.de Betreff: Re: [basex-talk] Performance loss between version 9.2.4 and 9.3.2 when executing specific xQuery And I’m always delighted to be confronted with library use case. BaseX

Re: [basex-talk] Performance loss between version 9.2.4 and 9.3.2 when executing specific xQuery

- > > *Die AK setzt sich seit 100 Jahren für Gerechtigkeit ein. Damals. Heute. > Für immer.* > > *arbeiterkammer.at/100 <https://arbeiterkammer.at/100>** > <https://arbeiterkammer.at/100>* <https://w.ak.at/zukunftsprogramm> > > >

Re: [basex-talk] Performance loss between version 9.2.4 and 9.3.2 when executing specific xQuery

And even more curiously, we are also working in this very same time on handling fetches from OAI-PMH sources! :-D M. On 08/05/20 13:37, Imsieke, Gerrit, le-tex wrote: Just saying that I find it sooo interesting to learn at which places and for which purposes BaseX is being employed. Have a

Re: [basex-talk] Performance loss between version 9.2.4 and 9.3.2 when executing specific xQuery

Just saying that I find it sooo interesting to learn at which places and for which purposes BaseX is being employed. Have a nice weekend! On 08.05.2020 13:31, BIRKNER Michael wrote: Hi Christian, thank you for your answers. As you can guess the queries I sent in my original email are just

Re: [basex-talk] Performance loss between version 9.2.4 and 9.3.2 when executing specific xQuery

n für Gerechtigkeit ein. Damals. Heute. Für immer. arbeiterkammer.at/100<https://arbeiterkammer.at/100><https://arbeiterkammer.at/100><https://w.ak.at/zukunftsprogramm> Von: Christian Grün Gesendet: Freitag, 8. Mai 2020 12:37 An: BIRKNER Michael Cc: basex-

Re: [basex-talk] Performance loss between version 9.2.4 and 9.3.2 when executing specific xQuery

I tried to reproduce your use case by creating some sample data (with a few millions of entries), but both the query plan and the performance were similar in 9.2.4 and the current 9.3.3 beta version. And I am still trying to understand your example query. Is it correct that the attribute of your

Re: [basex-talk] Performance loss between version 9.2.4 and 9.3.2 when executing specific xQuery

Thanks, Michael, for the valuable observation. It might be that another newly integrated optimization proves to be detrimental to your existing query. I’ll try to find the culprit. Just a minor question: In db2, a single document seems to be stored. Does this mean that only one record is assigned

[basex-talk] Performance loss between version 9.2.4 and 9.3.2 when executing specific xQuery

Hi, I am observing a performance loss between BaseX versions 9.2.4 (which I was using so far) and 9.3.2 (to which I updated recently) when executing an xQuery like this: --- (: Open 2 databases and get all s :) let $recsFromDb1 := db:open('db1')/record let $recsFromDb2 :=

Re: [basex-talk] Performance (index?) on large intermediate structures

Dear Sebastian, In fact XQuery maps are very fast, and they are a great data structure for speeding up repeated lookups if you encounter cases in which this is not done automatically by the optimizer. If you manage to send us a little example query that demonstrates the pattern you are working

[basex-talk] Performance (index?) on large intermediate structures

Dear BaseX community I am currently working with BaseX to write a rather long XQuery statement with quite large intermediate structures (XML documents / fragments) bound to variables (in FLWOR expressions), which I query by XPath using attributes etc. further down. Having nested loops, this is

Re: [basex-talk] Performance of Query joining two documents

Thank you very much! Using your advice about performance I solved not only this, but some other ugly queries here. Cheers, WILLIAM DAVID VELÁSQUEZ CREATIVO DE SOFTWARE Creativos Digitales S.A.S. Calle 30A # 83 - 53 Local 1033 Tel: 322 1730 - 311 709 8421 Medellín, Colombia

Re: [basex-talk] Performance of Query joining two documents

Hi William, Your query will be evaluated in appr. 100-200 ms if you do some little rewritings. Here is one variant: declare function local:compare($actual, $prev) { for $c in $actual/portfolio/asset return { $c/* }, { $prev/portfolio/asset[assetid/text() =

Re: [basex-talk] Performance issues and query optimizer

Hi France, > 1. (For each //element without @id, add @id) then db:optimise and redirect > to next step > 2. For each /*, process content to XLIFF (uses the new ids so requires > preceding redirect to commit changes). > > Now, is it suggested to work with updindex/autooptimize to true or as shown

Re: [basex-talk] Performance issues and query optimizer

​The approach will definitely help with optimizing individual queries. The other level I'd like to know about is if content is indeed indexed when I think it should be and which indexing approach is best. We have batch processes in multiple layers and I'm not sure that our approach to them is

Re: [basex-talk] Performance issues and query optimizer

Just as an example: On our database with attribtue index the following query db:open("lookup")//entry[@zip = "53040" and @city ="BETTOLLE"] returns the following output in the info view: Compiling: - pre-evaluate db:open("lookup") to document-node() - atomic evaluation of (@*:zip = "53040")

Re: [basex-talk] Performance issues and query optimizer

Can I avoid doing that by waiting on version 9.0 and enforcing rewriting? On Fri, Oct 20, 2017 at 2:35 PM, Marco Lettere wrote: > Hi France, > check out the info window of the GUI in a test-run it will tell. > Anyway I think that if the database name is in a declared

Re: [basex-talk] Performance issues and query optimizer

Hi France, check out the info window of the GUI in a test-run it will tell. Anyway I think that if the database name is in a declared variable it should defintely be able to grasp the index. If $dbname is passed into another function as a function parameter then it could be that the info for

[basex-talk] Performance issues and query optimizer

Hi, We are working hard right now on performance issues. I just read this about an upcoming release: Enforce Rewritings In various cases, existing index structures will not be utilized by the query optimizer. This is usually the case if the name of the database is not a static string (e.g.,

Re: [basex-talk] Performance analysis in web DBA

Hi Erik, The DBA editor is limited to the execution of queries. The query plan can be retrieved via XQuery, though: xquery:parse('1') Best, Christian On Thu, Aug 3, 2017 at 6:57 AM, Erik Peterson wrote: > Is there a way to get the query plan and stats in the web dba...like

[basex-talk] Performance analysis in web DBA

Is there a way to get the query plan and stats in the web dba...like the native client has with the info view?

Re: [basex-talk] Performance: associating one with many, or many with one?

Dear Jay, > And what I'm wondering is that, in a situation where there's one of > something (a musician), and a bunch of something else they're associated > with (song writting or performance credits), which node should get the > reference? I would go with the first approach (listing the track

Re: [basex-talk] Performance: associating one with many, or many with one?

...@mailman.uni-konstanz.de] De la part de Jay Straw Envoyé : mercredi 29 mars 2017 03:16 À : basex-talk@mailman.uni-konstanz.de Objet : [basex-talk] Performance: associating one with many, or many with one? Hi List! I'm building an application in base-x that's sort of like a local music wiki: bands

[basex-talk] Performance: associating one with many, or many with one?

Hi List! I'm building an application in base-x that's sort of like a local music wiki: bands, members, venues, promoters, and albums and songs are some of what I'm representing in XML. And what I'm wondering is that, in a situation where there's one of something (a musician), and a bunch of

Re: [basex-talk] Performance and heavy load

Hi, I would like to know more about keep the session opened as you state it -- I am using Java/Groovy client populating a large database (over half a million resources) and if I keep the session opened, so it could be reused within the thread, after a while it starts to cause problems. The only

Re: [basex-talk] Performance and heavy load

{sessionEntry.BaseXSession.Timeout = timeout;} return sessionEntry;} Cheers, Martín. Date: Wed, 19 Aug 2015 13:41:34 +0200 To: basex-talk@mailman.uni-konstanz.de From: m...@centrum.cz Subject: Re: [basex-talk] Performance and heavy load Hi, I would like to know more about

Re: [basex-talk] Performance with joins?

Adding a text() step to the predicate does dramatically reduce the execution time. This query: declare namespace marc=http://www.loc.gov/MARC21/slim;; for $m in collection( latin_hold_20150730 )/marc:collection/marc:record, $r in collection( latin_hold_20150730 )/root/row[BIB_ID/text() =

Re: [basex-talk] Performance with joins?

Dear Tim, The query plan indicates that no index is applied. Your query may be evaluated faster when rewriting BIB_ID to BIB_ID/text(). I will see if this can automatically be done by the query compiler. Best, Christian On Tue, Aug 4, 2015 at 4:42 AM, Tim Thompson timat...@gmail.com wrote:

Re: [basex-talk] Performance with joins?

Executes in 257.13 ms. But it shouldn't be necessary to explicitly specify the text() node, right (or would even be bad practice to do so[1])? The results of both variants should be equivalent indeed (unless a BIB_ID does not have more than one text node). I wouldn't call it bad practice to

Re: [basex-talk] Performance with joins?

I was mistaken; I hadn't enabled indexes on this database. However, after creating attribute and text indexes, the query actually seems to take longer to execute (483622.95 ms on last run), although the query plan itself doesn't seem to have changed: Compiling: - pre-evaluating

Re: [basex-talk] Performance and heavy load

OK Resource(s) added in 123.96 ms. 125.52 ms Thanks! Martín. From: christian.gr...@gmail.com Date: Tue, 28 Jul 2015 15:12:48 +0200 Subject: Re: [basex-talk] Performance and heavy load To: ferrari_mar...@hotmail.com CC: basex-talk@mailman.uni-konstanz.de Out of interest: Do you

Re: [basex-talk] Performance and heavy load

ms Thanks! Martín. From: christian.gr...@gmail.com Date: Tue, 28 Jul 2015 15:12:48 +0200 Subject: Re: [basex-talk] Performance and heavy load To: ferrari_mar...@hotmail.com CC: basex-talk@mailman.uni-konstanz.de Out of interest: Do you use a recent version of BaseX? On Tue, Jul 28

Re: [basex-talk] Performance and heavy load

. From: ferrari_mar...@hotmail.com To: christian.gr...@gmail.com Date: Thu, 30 Jul 2015 00:46:17 + CC: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Performance and heavy load Hi Christian, I've dug more into this problem. We've installed BaseX 8.2.3 on our Linux box

Re: [basex-talk] Performance and heavy load

d'origine- De : basex-talk-boun...@mailman.uni-konstanz.de [mailto:basex-talk-boun...@mailman.uni-konstanz.de] De la part de Fabrice Etanchaud Envoyé : mardi 28 juillet 2015 11:36 À : Maximilian Gärber; Martín Ferrari Cc : basex-talk@mailman.uni-konstanz.de Objet : Re: [basex-talk] Performance

Re: [basex-talk] Performance and heavy load

Hi Martin, how do you spread the log files? All into one db or do you create new dbs? If you keep on adding all files to the same database, the add times will slow down over time. Please keep in mind that you can query multiple databases at once, so I would rather have more databases. With 8.3

Re: [basex-talk] Performance and heavy load

@mailman.uni-konstanz.de Objet : Re: [basex-talk] Performance and heavy load Hi Martin, how do you spread the log files? All into one db or do you create new dbs? If you keep on adding all files to the same database, the add times will slow down over time. Please keep in mind that you can query

Re: [basex-talk] Performance and heavy load

work. We still have a lot of load, so I'll let you know how it goes when we enable it again in production. Thanks, Martín. From: christian.gr...@gmail.com Date: Tue, 28 Jul 2015 15:12:48 +0200 Subject: Re: [basex-talk] Performance and heavy load To: ferrari_mar...@hotmail.com CC: basex

Re: [basex-talk] Performance and heavy load

CC: basex-talk@mailman.uni-konstanz.de Subject: RE: [basex-talk] Performance and heavy load Date: Tue, 28 Jul 2015 09:40:19 + An another idea : If you never replace a file, You may expect better performance setting up a REST-XQ function simply calling db:add. The documentation

[basex-talk] Performance and heavy load

Hi guys,I'm quite new to BaseX. I've read a bit already, but perhaps you can help so I can investigate further. We are having a performance problem with our BaseX server. We're running it on a VM, and hitting it from around 5 web servers. Under no stress, I get this timing from the log for

Re: [basex-talk] performance of preceding/following axis

Hi Daniel, //w[@type = NN][(subsequence(preceding::w, 1, 3), subsequence(following::w, 1, 3))/@type = ADJA] The preceding axis can be quite costly. You could try to use preceding-sibling and following-sibling instead (if it makes sense in your scenario). Another option could be to replace the

[basex-talk] performance of preceding/following axis

Hi, I'm trying to use BaseX for linguistic queries on a TEI document containing annotated tokens (i.e. tei:w-elements with attributes). I'm specifically interested in distance queries that allow to search for combinations of token features within a given window (e.g. all nouns that have an

Re: [basex-talk] performance of preceding/following axis

Hello Daniel, I don't have much time right now, but maybe a few pointers to get you started. I didn't test any of this, so take it with a grain of salt. However, I guess your subsequence solution is not performing optimal, as I would guess that there really is a new sequence created. So for

Re: [basex-talk] Performance issue related with Casting.

Hi Ankit, Now the problem is that, It is working fine with this, but when I declare variable $ids as xs:string* sequence. The reason is the existence of the Java function: We cannot always determine statically what types the results will have. As a result, we cannot statically detect that

[basex-talk] Performance issue related with Casting.

Hi, I am invoking a java method inside My XQuery script, My java methods returns a string array which contains the Id of elements. Now I am taking that list in a xquery variable. It looks like this. let $ids := javacontext:getIds(); // Java Method return /sample_xml/category/product[@id = $ids]

Re: [basex-talk] Performance related query.

Hi Ankit, have you already compared the query info outoput? Best, Christian On Fri, Mar 27, 2015 at 10:35 AM, ankit kumar anky4b...@gmail.com wrote: Hi, I am getting performance issue while using my own xquery library. I have written an xquery module which contains a single function which

Re: [basex-talk] Performance related query.

Hi, Optimized query without module import count(db:attribute(large_products, distinct-values(db:open-pre(large_products,0)/products/*[@catid]/@catid))/self::id/parent::p:category[parent::products/parent::document-node()]) optimized query with module import count(let $catRefs_4 :=

[basex-talk] Performance related query.

Hi, I am getting performance issue while using my own xquery library. I have written an xquery module which contains a single function which return all the categories belong to a set of products as given below. Also there is one constrain that i cannot pass all the category to the

Re: [basex-talk] Performance related query.

Thanks Christian, I am able to solve the issue. Thanks, Ankit On 27 March 2015 at 17:10, Christian Grün christian.gr...@gmail.com wrote: Hi Ankit, The query info output indicates that only the first query is rewritten for index access (→ db:attribute). If you always work with the same

Re: [basex-talk] Performance related query.

Hi Ankit, The query info output indicates that only the first query is rewritten for index access (→ db:attribute). If you always work with the same database instance, the following version of your query should be evaluated faster: declare function lib:getCategory($products) { let

Re: [basex-talk] Performance [former 'of PUL commit']

A cold one which is growing (with scheduled batch ops) but indexed and UPDINDEX=true (for rare update operations that still might occur) and a hot one for realtime data which has no index but whose size will be reduced and kept constant. This makes perfect sense. You could also think about

Re: [basex-talk] Performance [former 'of PUL commit']

On 25/02/2015 14:51, Christian Grün wrote: A cold one which is growing (with scheduled batch ops) but indexed and UPDINDEX=true (for rare update operations that still might occur) and a hot one for realtime data which has no index but whose size will be reduced and kept constant. This makes

Re: [basex-talk] Performance of PUL commit

Hi Marco, There are two clients which read and write through RESTXQ to/from 2 different logical DBs on the same server. Is it possible that behaving concurrently we cause some sort of race condition that serializes operations for a long time period? Yes, this is absolutely possible. I guess

Re: [basex-talk] Performance of PUL commit

On 22/02/2015 19:50, Christian Grün wrote: Hi Marco, db:add or db:replace (BTW does this make a difference from a performance point of view?). It's faster to add documents, as this operation won't check if a document already exists. In future versions of BaseX, however, we will possibly

Re: [basex-talk] Performance of PUL commit

I'd like just to understand to what relates the huge difference (d) between querying the db and scheduling an update to the PUL which is what the internal function does and the time logged by the basexhttp server as the completion time of the whole RestXQ call which is T + d. Hm, difficult to

[basex-talk] Performance of PUL commit

It's me again sorry. ;-) One question that puzzles me in these last hours .. This is the scenario on 2 DBs (same server) with the following sizes: 136746 docs ~1 Gb 194608 docs~0.5 Gb declare %rest:path('{$project}/some/url/') %rest:POST('{$body}') %rest:consumes(application/xml)

Re: [basex-talk] Performance of PUL commit

Just one more hint, if we parallelly to the ongoing insertions post one directly with basexclient, we don't experience this lag ... M. On 20/02/2015 15:53, Marco Lettere wrote: It's me again sorry. ;-) One question that puzzles me in these last hours .. This is the scenario on 2 DBs

Re: [basex-talk] Performance on batch processes with http requests.

Hi, here is an example: A process that aggregates a few 100 topics and transforms the aggregated content to a large HTML file for reviewers to see all content together works fine. Try to do it for 32 languages, and you run out of memory. I'm trying to build a small sample. Our real processes also

Re: [basex-talk] Performance on batch processes with http requests.

Hi France, I guess there is no simple answer to your question; it mostly depends on the architecture of your approach what would be the best solution and further steps. And I'm not quite sure what's the major challenge? Is it performance, is it technical restrictions, is it the overall concept?

[basex-talk] Performance on batch processes with http requests.

Hi, I have an item that I would like to bring to attention. We have developed a web controller to let users manages translation processes for BaseX content. Our process is something like this: - Users select content to translation (1 to 500 small files) + languages to translation to (1 to

Re: [basex-talk] Performance issue due to evaluation during function composition ?

Hi Jean-Marc, It's not that trivial to optimize your query, but I'd like to let you know that pre-evaluation of function literals works if function declarations are swapped [1]. Christian [1] https://github.com/BaseXdb/basex/issues/1052 On Fri, Jan 2, 2015 at 10:19 AM, jean-marc Mercier

Re: [basex-talk] Performance issue due to evaluation during function composition ?

Hi Jean-Marc, Dirk has already outlined well what this is about. I have added a new GitHub request [1]. Cheers, Christian [1] https://github.com/BaseXdb/basex/issues/1052 On Fri, Jan 2, 2015 at 11:30 PM, Dirk Kirsten d...@basex.org wrote: Hello Jean-Marc, yeah, I can reproduce that.

Re: [basex-talk] Performance issue due to evaluation during function composition ?

errata : * declare variable $BigDb := db:open('MyBigDataBase'); 2015-01-02 21:54 GMT+01:00 jean-marc Mercier jeanmarc.merc...@gmail.com: Dirk, Hello, sorry, but a very similar behavior seems to appear also in the latest version BaseX80-20141225.174535. Here is a new code to test : declare

Re: [basex-talk] Performance issue due to evaluation during function composition ?

Dirk, Hello, sorry, but a very similar behavior seems to appear also in the latest version BaseX80-20141225.174535. Here is a new code to test : declare variable $BigDb := AppParam:DBInputOpen($ConfigDoc); declare function local:dummy($db) {local:call(local:count_elements#1,$db)}; declare

Re: [basex-talk] Performance issue due to evaluation during function composition ?

Hi Dirk, You are right, I was using an old 8.0 BaseX version (seems to be the one from 23/09/2014). The problem does not appear in the latest release. Thank you ! 2015-01-02 19:29 GMT+01:00 Dirk Kirsten d...@basex.org: Hello Jean-Marc, what version of BaseX did you use? I tried with the

Re: [basex-talk] Performance issue due to evaluation during function composition ?

Hello Jean-Marc, what version of BaseX did you use? I tried with the latest snapshot and used the XMark dataset (with factor 80, which produces around the same number of elements (133653910 to be exact) you had) and got the following output with an up-to-date index: - 0.01 ms - 0.01 ms - 0.0

[basex-talk] Performance issue due to evaluation during function composition ?

Hello (and happy new year 2015 !) I am facing some performance issues opening big databases due to an (unexpected for me) argument evaluation during function composition. Is this a normal behavior ? Cheers Here is a code description : declare variable $db := db:open(MyBigDataBase); declare

Re: [basex-talk] Performance issue due to evaluation during function composition ?

Hello Jean-Marc, yeah, I can reproduce that. However, I would not consider it a bug but rather a not-so-clever optimizer. The optimized query looks something like this: declare function local:dummy($db_0) { let $fun_8 := local:count_elements#1 return $fun_8($db_0) }; (prof:time(22383),

Re: [basex-talk] Performance and benchmark

Very good news ! Since I know how to do my request with XPath and not use direct path to file anymore, the performance are excellent ! database size | direct path | XPath 2.4 Go| 1.5 sec 160% CPU | 0.25 sec 6% CPU 224 Ko| 0.24 sec 2% CPU | 0.24 sec 2% CPU It looks that path

Re: [basex-talk] Performance and benchmark

On Thu, Dec 18, 2014 at 2:14 AM, Christian Grün christian.gr...@gmail.com wrote: 4) With my database, on a t2.micro EC2 instance (1 VCPU + 1 Go RAM), BaseX is unusable. That's a good hint. What does unusable mean? Did you encounter problems to create the database, or are your queries running

Re: [basex-talk] Performance and benchmark

database size | shared CentOS| t2.micro EC2 CoreOS 2.4 Go| 1.5 sec 160% CPU | 30 sec3% CPU ??!! 254 Mo| 0.35 sec 35% CPU | 0.21 sec 30% CPU 224 Ko| 0.24 sec 2% CPU | 0.14 sec 3% CPU The CPU usage on the t2.micro is not what it should be for the 2.4Go database and

Re: [basex-talk] Performance and benchmark

Florent: We run BaseX on EC2 micro and small instances without significant issue (though AWS servers, particularly micro/small, are not known for their high performance). Have you tried on a different instance type? Not sure how heavily loaded your instance is but understand that the t2 instance

Re: [basex-talk] Performance and benchmark

On Thu, Dec 18, 2014 at 3:35 PM, Pascal Heus pascal.h...@gmail.com wrote: We run BaseX on EC2 micro and small instances without significant issue (though AWS servers, particularly micro/small, are not known for their high performance). Have you tried on a different instance type? Not sure how

[basex-talk] Performance and benchmark

Hello, I have evaluated XML databases for an important project, and I'm very happy with BaseX which has great fonctionnalities and is very easy to use. I'm now about deploying my project in production, so I have new questions. My database has 400 000 XML resource files for a total 2.4 Go size,

Re: [basex-talk] Performance and benchmark

Salut Florent, Welcome to our list. 1) What are the CPU/RAM quantityies recommanded to use BaseX in production ? ... I'm giving you a single answer to your first three questions: It's difficult to give you general advice here, as it mostly depends on what you plan to do with the stored

[basex-talk] Performance of matches() vs castable as xs:integer

Hello all, I'm scanning millions of XML records imported from CSV looking for instances of 'bib_rec_id' which are non-numeric. Which of these two is if-statements likely to complete earlier? for $a in (/csv/record/bib_rec_id) return if ($a castable as xs:integer) then ... blah or if

[basex-talk] Performance

Hello to everyone, while trying to trace the performance I could expect from my RestXQ app I created a DB with something like 300/400k documents. For the test the documents are rather homogenoeus and stored plainly in the DB root collection. I had two major objectives. 1) understand the

Re: [basex-talk] Performance

collection('db')/a[@att = 'xxx' and b/text() = 'zzz'] on my laptop (Intel I7, 8 GB Ram and SSD) takes some 5 to 7 seconds. How does your query plan look like? What I noticed and this is feeling strange is that the time is often spent in rather similar proportion between query compilation and

Re: [basex-talk] Performance Degraded In function-lookup

Dear Christian, Thanks for the suggestion, the map is certainly cleaner looking than the switch statement. Is support for updating statements in an eval something that would be considered for the future, for example by postponing the application of the updates to after the calling function

  1   2   >