Re: [basex-talk] Xquery recursion and db:add() - stack overflow

Thanks for pointer! Code is rewritten using hof:until() and tested towards a particular set at our national provider of library data. The script still accumulates data, so it will probably still run into memory troubles with larger datasets, but the stack-overflow should be taken care of. For

Re: [basex-talk] Xquery recursion and db:add() - stack overflow

Hello Lars, just a thought (and really just a pointer, I am neither a purely functional guy and also I feel like I am missing something obious...): Maybe you could rewrite the recursive approach using higher order functions. Consider a query like the following hof:scan-left(1 to 100, map {

Re: [basex-talk] Xquery recursion and db:add() - stack overflow

Thanks Johan and Matti for useful suggestions. Cutting down on the chunks seems to be a viable alternative. It would have been nice, though, to have a robust harvester in XQuery that could take on anything, although the recursive version works fine as long as the dataset consist of a couple of

Re: [basex-talk] Xquery recursion and db:add() - stack overflow

Hello, If your case allows using external tools for harvesting, I can highly recommend metha (https://github.com/miku/metha) which is a fairly full featured command line OAI-PMH harvester. Best regards, Matti L. On 11/05/16 18:31 , "basex-talk-boun...@mailman.uni-konstanz.de on behalf of Johan

Re: [basex-talk] Xquery recursion and db:add() - stack overflow

Hi Lars I have done some OAI-PMH fetches but never got into stack-overflow issues. I guess one workaround you can do on your part is to partition your query with date-ranges using the query parameters "from" and "until" on your initial call to the endpoint. Regards, Johan Mörén On Wed, May 11,

Re: [basex-talk] Xquery recursion and db:add() - stack overflow

The basexgui startup file now contains: BASEX_JVM="-Xmx8g -Xss4m $BASEX_JVM" It helped the script a long way, but eventually it had to kneel. It works fine though, on smaller datasets. Maybe there is some other way to get the data over. I'll have a talk with the guys providing the

Re: [basex-talk] Xquery recursion and db:add() - stack overflow

Hello Lars, if you have a deep recursion Java will at some point hit its stack size limit. Have you already tried to simply increase the Java stack size, e.g. by passing the parameter -Xss2m to the JVM? Cheers Dirk On 05/11/2016 01:43 PM, Lars Johnsen wrote: > The following code generates the

[basex-talk] Xquery recursion and db:add() - stack overflow

The following code generates the error "Stack Overflow: try tail recursion?" The code reads in bibliographic data using OAI-PMH and updates a database for each chunk of data. With OAI-PMH, only part of the data is available for each request, so the server returns a resumption token if there are