Re: [basex-talk] Weird behaviour with sequence and random:integer

[Christian Grün]

Hi Marco,

> ("a", "b", "c")[trace(1 + random:integer(3))]

The filter expression is defined in such a way that the predicate is
evaluated anew for every item of the sequence. If you want
random:integer(3) to be evaluated only once, you can either bind it to a
variable…

let $r := random:integer(3) + 1
return ("a", "b", "c")]$r]

…or use functions like fn:subsequence:

subsequence(("a", "b", "c"), random:integer(3) + 1, 1)

Hope this helps,
Christian



Marco Lettere  schrieb am Mi., 22. Mai 2024, 15:34:

> Dear all,
>
> I have this strange behavior when running in BaseX 10.4:
>
> ["a","b","c"](1 + random:integer(3))
>
> I got something empty result, sometimes (correctly) one out of the
> three, sometimes two results...
>
> I report the results and the trace in [1] and [2].
>
> What is the reason for this?
>
> Thank you.
>
> Marco.
>
> [1] Two results
> a
> c
>
> Evaluating:
> 1
> 3
> 3
> Compiling:
> - rewrite list to xs:string sequence: ("a", "b", "c")
> - swap operands: (1 + random:integer(3))
> Optimized Query:
> ("a", "b", "c")[trace((random:integer(3) + 1))]
> Query:
> ("a", "b", "c")[trace(1 + random:integer(3))]
> Result:
> - Hit(s): 2 Items
> - Updated: 0 Items
> - Printed: 3 b
> - Read Locking: (none)
> - Write Locking: (none)
> Timing:
> - Parsing: 0.21 ms
> - Compiling: 0.4 ms
> - Optimizing: 0.1 ms
> - Evaluating: 0.57 ms
> - Printing: 0.02 ms
> - Total Time: 1.31 ms
>
>
> [2] Empty result
> ()
>
> Evaluating:
> 3
> 3
> 1
> Compiling:
> - rewrite list to xs:string sequence: ("a", "b", "c")
> - swap operands: (1 + random:integer(3))
> Optimized Query:
> ("a", "b", "c")[trace((random:integer(3) + 1))]
> Query:
> ("a", "b", "c")[trace(1 + random:integer(3))]
> Result:
> - Hit(s): 0 Items
> - Updated: 0 Items
> - Printed: 0 b
> - Read Locking: (none)
> - Write Locking: (none)
> Timing:
> - Parsing: 0.16 ms
> - Compiling: 0.41 ms
> - Optimizing: 0.15 ms
> - Evaluating: 0.56 ms
> - Printing: 0.02 ms
> - Total Time: 1.3 ms
> Query Plan:
> 
>
>  
>a
>b
>c
>  
>  
>
>  
>3
>  
>  1
>
>  
>
> 
>
>

Re: [basex-talk] Making store reactive

[Christian Grün]

Hi Marco,

Thanks for your suggestion. Some thoughts:

• Function items can depend on the currently evaluated code and its static
and dynamic context, but we could possibly design something similar as for
the Job Module, in which the query is passed on as a string or a URI
reference and evaluated completely independently.
• Registered observers could be handled similarly as „services”, i.e., made
persistent, end up in the same query pool, discarded by user requests,
gracefully shut down when a server stops, etc. [1].
• The feature request reminds me of triggers what we envisioned for
databases (but that were eventually discarded [2]).

Having said this, it could take a while to make this happen as it’s a
non-trivial request :) I’d like to hear about suggestions of other readers.

Apart from that, we are always interested in feedback on the Store Module;
it’s still fairly new, but more and more people seem to discover it.

Ciao,
Christian

[1] https://docs.basex.org/wiki/Job_Module#Services
[2] https://github.com/BaseXdb/basex/issues/1082



On Wed, May 22, 2024 at 9:49 AM Marco Lettere  wrote:

> Dear Christian and BaseX developers,
>
> just wondering if adding something like the following would be hard to
> implement.
>
> *store:observe($key as xs:string, $observers as function(*)*) *
>
> with $observers being something like
>
> *function($key as xs:string).*
>
> The semantics is to call the registered observers whenever a value
> associated with the key in the store changes (put, remove, clear, ..).
>
> This would allow for nicely decoupled observer - notification pattern.
>
> Does it make sense?
>
> Regards,
>
> Marco.
>

Re: [basex-talk] Session problems with two BaseX installations

[Christian Grün]

Hi Jack,

It seems that the browser ignores different ports when handling session
data (just a guess). Does it work if you change the session key of your
second DBA instance [1]?

Best,
Christian

[1]
https://github.com/BaseXdb/basex/blob/df83d80238a27f3a168e3d5e88f984c819a37fb8/basex-api/src/main/webapp/dba/lib/config.xqm#L8-L9


On Wed, May 22, 2024 at 5:44 AM Jack Steyn  wrote:

> Hi,
>
> I have two separate BaseX installations on the same machine. Each has its
> own set of databases and users. I run their HTTP servers simultaneously,
> configured to their own sets of ports.
>
> When I authenticate to the DBA webapp of one, I am logged out of the DBA
> webapp of the other.
>
> What might I do to prevent this?
>
> Best regards,
>
> Jack
>

Re: [basex-talk] string:levenshtein producing incorrect results?

[Christian Grün]

Thanks for the hint, the DBA code has been updated (the function is being
replaced with the new fn:char('\n') function). A new snapshot is online.


On Mon, May 13, 2024 at 8:12 AM Jack Steyn  wrote:

> Thanks, Christian.
>
> When I download and unzip the latest version, start the HTTP server and
> navigate to localhost:8080, I'm given the following error:
>
> Stopped at [...]/basex/webapp/dba/jobs/job-result.xqm, 34/49:
> [XPST0017] Unknown function: string:nl.
>
> It's easy enough to work around by editing job-result.xqm (and jobs.xqm in
> which string:nl also appears), but wanted to bring it to your attention in
> case you weren't aware.
>
> Cheers,
>
> Jack
>
>
> On Fri, 10 May 2024, 7:37 pm Christian Grün, 
> wrote:
>
>> Hi Hack,
>>
>> That’s been helpful, thanks. We’ve aligned our Damerau/Levenshtein
>> algorithms, the latest version should behave as expected [1, 2].
>>
>> Best,
>> Christian
>>
>> [1] https://files.basex.org/releases/latest/
>> [2]
>> https://github.com/BaseXdb/basex/commit/6889ac108c6b32d448d640d53ec098bbb8938f06
>>
>>
>> On Thu, May 9, 2024 at 8:29 AM Jack Steyn  wrote:
>>
>>> Hi,
>>>
>>> According to my copy of BaseX 10.7,
>>>
>>> string:levenshtein('oil field', 'oilfield')
>>>
>>> and
>>>
>>> string:levenshtein('oil field', 'coalfield')
>>>
>>> both return the same value, 0.7778.
>>>
>>> My understanding is that the Levenshtein-Damerau distance between 'oil
>>> field' and 'oilfield' is 1 and between 'oil field' and 'coalfield' is 3, so
>>> following the formula from
>>> https://docs.basex.org/wiki/String_Module#string:levenshtein
>>>
>>> 1.0 – distance / max(length of strings)
>>>
>>> should give 0.888... and 0.666... respectively.
>>>
>>> Am I off-base here or is there something awry with string:levenshtein?
>>>
>>> Cheers,
>>>
>>> Jack
>>>
>>

Re: [basex-talk] string:levenshtein producing incorrect results?

[Christian Grün]

Hi Hack,

That’s been helpful, thanks. We’ve aligned our Damerau/Levenshtein
algorithms, the latest version should behave as expected [1, 2].

Best,
Christian

[1] https://files.basex.org/releases/latest/
[2]
https://github.com/BaseXdb/basex/commit/6889ac108c6b32d448d640d53ec098bbb8938f06


On Thu, May 9, 2024 at 8:29 AM Jack Steyn  wrote:

> Hi,
>
> According to my copy of BaseX 10.7,
>
> string:levenshtein('oil field', 'oilfield')
>
> and
>
> string:levenshtein('oil field', 'coalfield')
>
> both return the same value, 0.7778.
>
> My understanding is that the Levenshtein-Damerau distance between 'oil
> field' and 'oilfield' is 1 and between 'oil field' and 'coalfield' is 3, so
> following the formula from
> https://docs.basex.org/wiki/String_Module#string:levenshtein
>
> 1.0 – distance / max(length of strings)
>
> should give 0.888... and 0.666... respectively.
>
> Am I off-base here or is there something awry with string:levenshtein?
>
> Cheers,
>
> Jack
>

Re: [basex-talk] BaseX installation cannot see Java

[Christian Grün]

Thanks for the insight. We’ll need to ensure that the check runs with all
versions of Windows that we support, but if a similar problem is reported
back to us again, we’ll think about a multi-step version check.

Best,
Christian



On Thu, May 9, 2024 at 11:09 PM  wrote:

> Mr. Grun,
>
> It seems that the Windows Management Instrumentation Command-line (WMIC)
> command line utility gives a reliable result in windows.
>
> https://www.techtarget.com/searchenterprisedesktop/definition/Windows-Manage
> ment-Instrumentation-Command-line-WMIC
> 
>
> It is deprecated, but working on any Windows OS, including Windows Server
> 2022.
>
> On my home machine with Windows 10 OS the following command
> c:\>wmic product where "Name like '%JRE%' or name Like'%Java%' or name
> Like'%JDK%' " get name,Vendor,Version
>
> emits the following 4 (four) Java installations:
>
> Name  Vendor
> Version
> Java(TM) SE Development Kit 18.0.2 (64-bit)   Oracle Corporation
> 18.0.2.0
> Java 8 Update 391 (64-bit)Oracle Corporation
> 8.0.3910.13
> Eclipse Temurin JDK with Hotspot 21.0.2+13 (x64)  Eclipse Adoptium
> 21.0.2.13
> Java Auto Updater Oracle Corporation
> 2.8.391.13
>
> To see all available entries:
> c:\>wmic product where "Name like '%JRE%' or name Like'%Java%' or name
> Like'%JDK%' " get * /format:textvaluelist
>
> So, you can try to integrate wmic based method in the BaseX Windows
> installer.
>
> Regards,
> Yitzhak Khabinsky
>
>
>

Re: [basex-talk] BaseX installation cannot see Java

[Christian Grün]

>
> Where and how BaseX installer got into old Java v1.6 and missed a
> legitimate
> Eclipse Temurin JRE with Hotspot 17.0.8.1+1?
>

Good question. We would need to better understand the NSIS installer code
[1]. The installer provided dedicated Java checks, but those did not work
with newer JDK versions anymore, which is why we switched to the basic
'java -version' call.

If someone has worked with NSIS in the past, feel free to share your
knowledge.

[1] https://nsis.sourceforge.io/Main_Page



>
> c:\>java -version shows the following
> openjdk version "17.0.8.1" 2023-08-24
> OpenJDK Runtime Environment Temurin-17.0.8.1+1 (build 17.0.8.1+1)
> OpenJDK 64-Bit Server VM Temurin-17.0.8.1+1 (build 17.0.8.1+1, mixed mode,
> sharing)
>
> c:\>path shows the following:
> PATH=C:\Program Files\Eclipse
>
> Adoptium\jre-17.0.8.101-hotspot\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOW
> S\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WI
> NDOWS\System32\OpenSSH\;C:\Windows\CCM;C:\Program
> Files\Eclypsium\;C:\Program Files (x86)\Microsoft SQL
> Server\160\DTS\Binn\;C:\Program
> Files\Azure Data
> Studio\bin;C:\Users\i300179\AppData\Local\Microsoft\WindowsApps;
>
> Overall, that's a lot for such a great support of the product.
>
> Regards,
> Yitzhak Khabinsky
>
>
>

Re: [basex-talk] BaseX installation cannot see Java

[Christian Grün]

Hi Yitzhak,

We have revised the Windows installation script and improved the debugging
output. If it fails, it gives more information on the analyzed version
string. Please check it out and tell us if it works for you [1].

Hope this helps,
Christian

[1] https://files.basex.org/releases/latest/


On Tue, May 7, 2024 at 10:40 PM Christian Grün 
wrote:

> Thanks for reporting this, Yitzhak.
>
> It seems something is going wrong in the Java version check of the
> installer:
>
> *https://github.com/BaseXdb/basex-dist/blob/20c565a4b8b79c27d276a191b35cb224ce537ef3/win/BaseX.nsi#L50
> <https://github.com/BaseXdb/basex-dist/blob/20c565a4b8b79c27d276a191b35cb224ce537ef3/win/BaseX.nsi#L50>*
>
>
>
>
>  schrieb am Di., 7. Mai 2024, 19:27:
>
>> C:\>java -version
>> openjdk version "17.0.8.1" 2023-08-24
>> OpenJDK Runtime Environment Temurin-17.0.8.1+1 (build 17.0.8.1+1)
>> OpenJDK 64-Bit Server VM Temurin-17.0.8.1+1 (build 17.0.8.1+1, mixed mode,
>> sharing)
>>
>>
>>
>> Regards,
>> Yitzhak Khabinsky
>>
>>
>>

Re: [basex-talk] BaseX installation cannot see Java

[Christian Grün]

Thanks for reporting this, Yitzhak.

It seems something is going wrong in the Java version check of the
installer:

*https://github.com/BaseXdb/basex-dist/blob/20c565a4b8b79c27d276a191b35cb224ce537ef3/win/BaseX.nsi#L50
*




 schrieb am Di., 7. Mai 2024, 19:27:

> C:\>java -version
> openjdk version "17.0.8.1" 2023-08-24
> OpenJDK Runtime Environment Temurin-17.0.8.1+1 (build 17.0.8.1+1)
> OpenJDK 64-Bit Server VM Temurin-17.0.8.1+1 (build 17.0.8.1+1, mixed mode,
> sharing)
>
>
>
> Regards,
> Yitzhak Khabinsky
>
>
>

Re: [basex-talk] minor issue in documentation

[Christian Grün]

Hi Rob,

Thanks. The current documentation will soon be replaced by a new version
(generated with BaseX) [1].

Best,
Christian

[1] https://help.basex.org/main/Database_Functions#db:type



On Fri, May 3, 2024 at 11:16 AM Rob Stapper  wrote:

> Hi Christian,
>
> I stumbled on this minor issue in the documentation:
>
> @ docs.basex.org/wiki/Database_Module#db:type
>
> *Examples*
>
>- *db:type("DB", "factbook.xml")* returns *true* if the specified
>resource is an XML document.
>
>
> *true* should be 'xml'.
>
>
>
> mvgr.
>
> Rob Stapper
>
>

Re: [basex-talk] Request and JSON parsing

[Christian Grün]

Thanks. Here’s one way to do it:

let $body := document {
  

  <_ type="object">
ba03177
0.83175087

  ba
  1810-40
  Heinrich von Kleist
  Die heilige Cäcilie
  https://kleist-digital.de/...
  Die Aebtissinn, ...

  

  
}
return 
  {
for $result in $body/json/result/_
return {
  for $field in ('Score', 'Blatt', 'Autor', 'Titel', 'Text', 'Link')
  return 
{ $field }: 
{ data($result//*[name() = lower-case($field)]) }
  
}
  }


Another one is…

return 
  {
for $result in $body/json/result/_
return {
  for $text in $result//text()
  let $field := name($text/..)
  return 
{ $field ! (upper-case(substring(., 1, 1)) || substring(.,
2))  }: 
{ $text }
  
}
  }


…but of course you can also output only the relevant fields.

Hope this helps,
Christian

Re: [basex-talk] Request and JSON parsing

[Christian Grün]

Hi Günter,

2. I'm getting a full Object (but I dont know, if its the raw JSON-Object
> like above), but I am not able, to parse it, to get a list in html.
>

Could you share the result of your response with us (possibly shortened)?

It would be interesting to learn something about the type of the response
body. What do you get if you inspect:type($response[2]) ?

let $data := json:parse($input, map { 'format': 'xquery' })
>
> return map:for-each($data, function($k, $v) {
>
>   $k || ': ' || string-join($v, ', ')
>
> })
>

If you use json:parse($input) without a specific format, you’ll get an XML
representation of the JSON data, which is usually simpler to postprocess.

Hope this helps
Christian

Re: [basex-talk] Performance issue with BaseX CLI

[Christian Grün]

Hi again,

I had a quick look into the monitoring code, and I noticed two things:

1. It looks to me (correct me if I’m wrong) as if the code of the project
was initially written for Saxon and then ported to BaseX. If you are
interested in using BaseX, you could focus on the slow functions, try
alternative writings and (if you want to run the code with both processors
in the future) ensure that Saxon still gives delivers good performance.

2. Some functions can be noticeably sped up (for both BaseX and Saxon) if
you use XQuery 3.1 features such as maps or group by. For example, the
runtime of #131014 could possibly be reduced with something similar to…

  for $ms in $Monitoring/*:MonitoringSite
  let $emsc := $ms/*:euMonitoringSiteCode
  for $ceqm in $ms/*:ChemicalEcologicalQuantitativeMonitoring
  let $V_rech := $ceqm/*:parameterCode || '/' || $ceqm/*:parameterOther ||
'/' || $ceqm/*:chemicalMatrix
  group by $group := $emsc || ': ' || $V_rech
  where count($ceqm) > 1
  return $V_rech

If BaseX turns out to be the way to go, it’s definitely worth taking
advantage of the database aspect. In BaseX, databases are fairly
light-weight, which means you can simply create them before running the
queries (e.g., with a single 'CREATE DB poc
/path/to/poc_rapportage_controle-main/xml' command) and use db:get('poc',
'your-doc.xml') in the queries to access a document (or even stick with
doc('your-doc.xml') if you enable DEFAULTDB [1]).

Hope this helps,
Christian

[1] https://docs.basex.org/wiki/Options#DEFAULTDB


On Mon, Apr 22, 2024 at 9:32 AM Christian Grün 
wrote:

> Hi Antonio,
>
> As Liam indicated, you may get better performance when adding your
> documents to a database.
>
> In general, though, the runtimes of BaseX and Saxon have aligned pretty
> much over the years, and I assume there’ll be a trivial reason behind the
> drastic difference in the runtime.
>
> Your test setup is probably too complex for us readers to spend more time
> with it. Could you possibly share a more basic example with us, ideally
> with a single document and query file?
>
> Thanks in advance,
> Christian
>
>
>
> On Mon, Apr 22, 2024 at 8:54 AM ANDRADE Antonio <
> antonio.andr...@ofb.gouv.fr> wrote:
>
>> @Liam R. E. Quin  : Thanks for your feedback. The
>> processing time is between 2 minutes and more than 11 hours (see table
>> below). Thus, the loading time of the Java virtual machine has little
>> impact. The main XQuery script loads the XML document once at the start of
>> processing. It is then requested several times as part of more or less
>> complex quality controls. At this moment, the XML document is not intended
>> to be stored. This is why it is not loaded into a database before
>> processing.
>>
>>
>>
>>
>>
>> *Saxon*
>>
>> *BaseX*
>>
>>
>>
>> *Start*
>>
>> *Stop*
>>
>> *Elapse time*
>>
>> *Start*
>>
>> *Stop*
>>
>> *Elapse time*
>>
>> Check Monitoring 2022 FRH
>>
>> 06:16:54
>>
>> 06:19:30
>>
>> 00:02:36
>>
>> 06:44:06
>>
>> 10:05:21
>>
>> 03:21:15
>>
>> Check Multi schéma 2022 FRH
>>
>> 06:25:46
>>
>> 06:31:47
>>
>> 00:06:01
>>
>> 10:05:55
>>
>> 11:39:07
>>
>> 01:33:12
>>
>>
>>
>>
>>
>> *De :* Liam R. E. Quin 
>> *Envoyé :* samedi 20 avril 2024 05:00
>> *À :* ANDRADE Antonio ;
>> basex-talk@mailman.uni-konstanz.de
>> *Objet :* Re: [basex-talk] Performance issue with BaseX CLI
>>
>>
>>
>> On Fri, 2024-04-19 at 10:45 +0200, ANDRADE Antonio wrote:
>>
>> Hie,
>>
>>
>>
>> For the purposes of European Water Framework Directive reporting, I
>> compared the performances of the Saxon and BaseX XQuery engines.
>>
>>
>>
>> First, you should consider (as i think Martin said) the Java runtime
>> startup time, typically a second or so.
>>
>>
>>
>> Second, BaseX is a database. If you will process the same document many
>> times, first load it into a database and then use the Python BaseX client.
>> This will avoid startup time, and, more importantly, will allow BaseX to
>> make use of database indexes.
>>
>>
>>
>> If you will only process any given document once, then Saxon may well be
>> the appropriate tool.
>>
>>
>>
>> liam
>>
>>
>>
>>
>>
>> --
>>
>> Liam Quin, https://www.delightfulcomputing.com/
>> <https://antiphishing.vadesecure.com/v4?f=SnpNUUNxek1BTWh6ZFZjaWCyrlumiLHtHmHGdEVdgTGAg0gyDE-v9PTNgKgfV2Nw=cHp0TkJv

Re: [basex-talk] file:path-to-native() throws an error if its argument does not exist

[Christian Grün]

Hi Gerrit,

If you don’t need the canonical path to a file resource on the file system,
file:resolve-path may be the better choice. It can be used for both file
URIs and local (relative or absolute) paths.

Hope this helps,
Christian


On Mon, Apr 22, 2024 at 8:00 AM Imsieke, Gerrit, le-tex <
gerrit.imsi...@le-tex.de> wrote:

> I have a file:// URI that corresponds to a directory that I need to create
> (using svn mkdir, therefore file:create-dir() is not an option here) if it
> doesn’t exist. Calling file:path-to-native() on it results in a
> file:not-found error. Is there a fundamental reason why the file needs to
> exist before transforming its URI into the OS-native representation? Using
> BaseX 10.7.
>
> Gerrit
>

Re: [basex-talk] Performance issue with BaseX CLI

[Christian Grün]

Hi Antonio,

As Liam indicated, you may get better performance when adding your
documents to a database.

In general, though, the runtimes of BaseX and Saxon have aligned pretty
much over the years, and I assume there’ll be a trivial reason behind the
drastic difference in the runtime.

Your test setup is probably too complex for us readers to spend more time
with it. Could you possibly share a more basic example with us, ideally
with a single document and query file?

Thanks in advance,
Christian



On Mon, Apr 22, 2024 at 8:54 AM ANDRADE Antonio 
wrote:

> @Liam R. E. Quin  : Thanks for your feedback. The
> processing time is between 2 minutes and more than 11 hours (see table
> below). Thus, the loading time of the Java virtual machine has little
> impact. The main XQuery script loads the XML document once at the start of
> processing. It is then requested several times as part of more or less
> complex quality controls. At this moment, the XML document is not intended
> to be stored. This is why it is not loaded into a database before
> processing.
>
>
>
>
>
> *Saxon*
>
> *BaseX*
>
>
>
> *Start*
>
> *Stop*
>
> *Elapse time*
>
> *Start*
>
> *Stop*
>
> *Elapse time*
>
> Check Monitoring 2022 FRH
>
> 06:16:54
>
> 06:19:30
>
> 00:02:36
>
> 06:44:06
>
> 10:05:21
>
> 03:21:15
>
> Check Multi schéma 2022 FRH
>
> 06:25:46
>
> 06:31:47
>
> 00:06:01
>
> 10:05:55
>
> 11:39:07
>
> 01:33:12
>
>
>
>
>
> *De :* Liam R. E. Quin 
> *Envoyé :* samedi 20 avril 2024 05:00
> *À :* ANDRADE Antonio ;
> basex-talk@mailman.uni-konstanz.de
> *Objet :* Re: [basex-talk] Performance issue with BaseX CLI
>
>
>
> On Fri, 2024-04-19 at 10:45 +0200, ANDRADE Antonio wrote:
>
> Hie,
>
>
>
> For the purposes of European Water Framework Directive reporting, I
> compared the performances of the Saxon and BaseX XQuery engines.
>
>
>
> First, you should consider (as i think Martin said) the Java runtime
> startup time, typically a second or so.
>
>
>
> Second, BaseX is a database. If you will process the same document many
> times, first load it into a database and then use the Python BaseX client.
> This will avoid startup time, and, more importantly, will allow BaseX to
> make use of database indexes.
>
>
>
> If you will only process any given document once, then Saxon may well be
> the appropriate tool.
>
>
>
> liam
>
>
>
>
>
> --
>
> Liam Quin, https://www.delightfulcomputing.com/
> 
>
> Available for XML/Document/Information Architecture/XSLT/
>
> XSL/XQuery/Web/Text Processing/A11Y training, work & consulting.
>
> Barefoot Web-slave, antique illustrations:  http://www.fromoldbooks.org
> 
>

Re: [basex-talk] .xar deployment

[Christian Grün]

…I agree with Andy: This looks like a Java issue. You could try enabling
IGNORECERT in your .basex configuration [1] and executing REPO INSTALL to
see if that changes anything.

[1] https://docs.basex.org/wiki/Options#IGNORECERT



On Wed, Apr 10, 2024 at 1:57 PM Andy Bunce  wrote:

> Hi Régis,
>
> I am pretty sure your issue comes from the SSL certificates available to
> your Java VM, and is not a problem with your xar package.
> Running Basex with -d shows more information [2]
>
> Something like [1] may fix it for one machine, but it is a slow process
> and far from ideal.
>
> /Andy
>
> [1]
> https://stackoverflow.com/questions/21076179/pkix-path-building-failed-and-unable-to-find-valid-certification-path-to-requ/36427118#36427118
> [2] basex -d -c ""
> ...
> javax.net.ssl.SSLHandshakeException: PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target
> at
> java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:578)
> at
> java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:123)
> at org.basex.io.IOUrl.response(IOUrl.java:145)
> at org.basex.io.IOUrl.inputStream(IOUrl.java:127)
> at org.basex.io.in.BufferInput.get(BufferInput.java:49)
> at org.basex.io.IOUrl.read(IOUrl.java:112)
> at
> org.basex.query.util.pkg.RepoManager.install(RepoManager.java:64)
> at org.basex.core.cmd.RepoInstall.run(RepoInstall.java:36)
> at org.basex.core.Command.run(Command.java:233)
> at org.basex.core.Command.execute(Command.java:93)
> at org.basex.api.client.LocalSession.execute(LocalSession.java:131)
> at org.basex.api.client.Session.execute(Session.java:36)
> at org.basex.core.CLI.execute(CLI.java:94)
> at org.basex.core.CLI.execute(CLI.java:78)
> at org.basex.core.CLI.execute(CLI.java:65)
> at org.basex.BaseX.(BaseX.java:83)
> at org.basex.BaseX.main(BaseX.java:45)
> Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target
>
>
> On Wed, 10 Apr 2024 at 11:16, Régis WITZ  wrote:
>
>> Hi all,
>> I'm kind of a XQuery noob, but I'm guilty of the creation of a little
>> BaseX module, Heimdall (10.5281/zenodo.10638084
>> ).
>> It's automatically built, unit tested and documented using GitLab CI.
>> All build results, including the .xar file, are hosted using GitLab Pages.
>> Thus, the module .xar archive is available here :
>> |https://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar|
>> .
>> As you see, all this is hosted in Huma-Num's self-hosted GitLab instance
>> (Huma-Num is a french research infrastructure).
>>
>> However, this mail is /not/ (just) a shameless plug.
>> My issue is that I cannot install my .xar directly with |REPO INSTALL| :
>>
>>  1. I cannot install it with the CLI:
>>
>> basex -c"> path='
>> https://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar'/>"
>> [repo:not-found] Package
>> 'https://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar
>> '
>> not found.
>>
>> Things are perfectly fine with for example the functx package :
>> |basex -c"> path='https://files.basex.org/modules/expath/functx-1.0.xar'/>"|
>>  2. For gigs, I tried to change the protocol from HTTPS to HTTP, and
>> here is the error that I get :
>>
>> basex -c"> path='
>> http://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar'/>"
>> [repo:parse] heimdall.xar: Resource "expath-pkg.xml" not found..
>>
>> So, maybe there is a problem with my expath-pkg.xml file ?
>> I didn't find any -but, as I said, I'm no pro. If needed, here is
>> the file that is wrapped in the .xar :
>>
>> https://gitlab.huma-num.fr/datasphere/heimdall/xquery/-/raw/main/expath-pkg.xml?ref_type=heads
>> <
>> https://gitlab.huma-num.fr/datasphere/heimdall/xquery/-/raw/main/expath-pkg.xml?ref_type=heads
>> >
>>  3. However, when I download my .xar file from the URL, and then REPO
>> INSTALL it using the local path, stuff works fine :
>> wget --no-check-certificate
>> https://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar
>> basex -c""
>> basex -c""
>>
>> Name Version  Type Path
>>
>> ---
>> http://heimdall.huma-num.fr  2.1  EXPath
>> http-heimdall.huma-num.fr-2.1
>>
>> 1 package(s).
>>
>> So I suppose the problem /might/ not be within the .xar itself, but the
>> way it is hosted ?
>> Maybe REPO INSTALL doesn't like my disturbing lack of SSL certificate ?
>> I tried to search for some clue, but 

Re: [basex-talk] some eq versus =

[Christian Grün]

Hi Leo,

I came across this question because I needed to know whether there are city
> elements twice in the file. For that I wrote version 2 and the result was
> wrong. Then I wrote version 1 with
> … satisfies . => deep-equal($city)
> that gave the correct answer. I noticed that I do not fully understand the
> cast behavior of the = operator…
>

Thanks. deep-equal() is probably what you want. If you use generalized
comparisons (=, !=, etc.), or if you use “data(.)” or “string(.)”, the
descendant text nodes of the referenced node will be concatenated and
returned as single string. As a result, queries like the following one…

  X = X

…will return “true” because the atomized value of both operands is“X”.

Sibling node traversals are often slow, as the same nodes are repatedly
processed.

  for $city-group in //city
  group by $string := serialize($city-group)
  where count($city-group) > 1
  return head($city-group)

With the latest BaseX 11 snapshot and the upcoming XQuery 4 features, it
could be:

  for value $v in map:build(//city, serialize#1)
  where count($v) > 1
  return head($v)

Hope this helps,
Christian

Re: [basex-talk] some eq versus =

[Christian Grün]

Hi Leo,

Yes, they are equivalent. Version 2 is a bit faster because "." will only
need to be atomized once.

We’ll examine if BaseX can automatically rewrite version 1 to version 2.

Best,
Christian


On Mon, Apr 8, 2024 at 11:59 PM Leo Studer  wrote:

> Hello
>
> are the following queries equivalent?
>
> 1. //*city*[*some* *$city* *in* following::*city* *satisfies* *string*(.)
> eq *string*(*$city*)]
>
> 2. //*city*[. = following::*city* ]
>
>
>
> Thanks in advance,
> Leo
>
>
>

Re: [basex-talk] Single command expected

[Christian Grün]

Hi Vladimir,

There’s currently no such function available. Even if we had a user:kill
function, we could not ensure that a user logs in a millisecond later when
user:drop is going to be executed. We could enrich user:drop et al. with an
“enforce” option to kill users, but the challenge is that a user may
currently execute a long-running update query that needs to be finalized
before the user can be dropped, thus delaying the execution of the
user:drop operation and possible other subsequent requests.

Have you already observed the presented pattern?

Best,
Christian


On Fri, Apr 5, 2024 at 10:36 AM Ветошкин Владимир 
wrote:

> Hi,
>
> How can I kill session and then user:grant or user:drop permissions in a
> single command?
> When I make it in two different commands - user logins between them and
> the second command gets error, that the user is currently logged in.
>
> --
> С уважением,
> Ветошкин Владимир Владимирович
>
>

Re: [basex-talk] hof:until is gone?

[Christian Grün]

Hi Graydon,

Folks tell us it’s time to stop delaying BaseX 11… We’re trying hard.

The good news: The only difference between hof:until [1] and fn:do-until
[2] is the order of parameters. The following queries will do the same
thing:

hof:until(function($a) { $a > 16 }, function($a) { $a * 2 }, 1)
do-until(1, function($a) { $a * 2 }, function($a) { $a > 16 })

With fn:do-until, the input has moved to the first position, the action
comes second and the predicate comes last.

Hope this helps,
Christian

[1] https://files.basex.org/releases/10.0/BaseX100.pdf
[2]
https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-do-until



On Thu, Mar 28, 2024 at 5:41 PM Graydon  wrote:

> Hi Andy --
>
> On Thu, Mar 28, 2024 at 01:06:10PM +, Andy Bunce scripsit:
> > hof:until is not gone, it is just hiding  [1]
> > - In 10.7 it is there but undocumented in Wiki
>
> Which is useful to know -- thank you! -- but makes me think a bunch of
> my production code is going to break hard when 11 is released. I would
> really like to avoid that. (In particular, the conversation with
> management about how much refactoring work will be required.)
>
> > - In BaseX 11 you need to use XQuery 4  fn:while-do⁺, fn:do-until⁺ [2]
>
> It is very probably my brain, but I'm having trouble transposing
> hof:until to those functions. I can an implement what I want to do with
> hof:until like so:
>
> declare function xc:dropTableLinesFancy($in as node()*) as node()* {
>   let $result as map(*) :=
> hof:until(
>   (: are we done? (= we've run off the end of our list of nodes ) :)
>   function($m) {
>  empty($m?oldNodes)
> },
>   (: create new list of line elements :)
>   function($m) {
>   map {
> 'oldNodes': tail($m?oldNodes),
> 'newNodes': ($m?newNodes,
>   switch (true())
> case starts-with(head($m?oldNodes),':stab') return 
> case starts-with(head($m?oldNodes),':rtab') return 
> case $m?toggle return 
> default return head($m?oldNodes)
>   ),
> 'toggle': (
>   switch (true())
> case starts-with(head($m?oldNodes),':stab') return true()
> case starts-with(head($m?oldNodes),':rtab') return false()
> default return $m?toggle
>   )
>   }
>   },
>   (: initial input = we start with no new nodes :)
>   map { 'oldNodes': $in, 'newNodes': (), 'toggle': false() }
> )
>   return $result?newNodes
> };
>
> Anyone willing to provide an example of what that would look like in
> fn:while-do or fn:do-until?
>
> Thanks!
> Graydon
>
>
> --
> Graydon Saunders  | graydon...@fastmail.com
> Þæs oferéode, ðisses swá mæg.
> -- Deor  ("That passed, so may this.")
>

Re: [basex-talk] BaseX and NFS

[Christian Grün]

Hi Marco,

I’m sorry, I can’t give any explanation on why BaseX performs worse with
NFS. In principal, we rely on standard Java IO/NIO.

Does it make a noticeable difference if you store copy many small files a)
locally and b) to the NFS exported disk?

Ciao,
Christian



On Thu, Mar 14, 2024 at 5:28 PM Marco Lettere  wrote:

> Dear all,
>
> we have an instance of BaseX running on top of an NFS exported disk. Yes
> yes I know it's not the best possible scenario thus I was expecting a
> slight performance decrease.
>
> Anyway, when comparing local disk to NFS disk, for a tiny operation like
> storing a very small document into a database (without optimize or
> indexes) we get orders of magnitude of difference. And I'm saying from
> few hundreds of ms to several seconds in some cases.
>
> Does anyone have experience with cases like these? Or a solid motivation
> that explains this degradation?
>
> Thank you very much as usual.
>
> Marco.
>
>

Re: [basex-talk] Out of Main Memory

[Christian Grün]

Hi Greg,

I would have guessed that 12 GB is enough for 4.7 GB; but it sometimes
depends on the input. If you like, you can share a single typical document
with us, and we can have a look at it. 61 GB will be too large for a
complete full-text index, though. However, it’s always possible to
distribute documets across multiple databases and access them with a single
query [1].

The full-text index is not incremental (in opposition to the other index
structures), which means it must be re-created it after updates. However,
it’s possible to re-index updated database instances and query fully
indexed databases at the same time.

Hope this helps,
Christian

[1] https://docs.basex.org/wiki/Databases


On Thu, Mar 14, 2024 at 10:58 PM Murray, Gregory 
wrote:

> Thanks, Christian. I don’t think selective indexing is applicable in my
> use case, because I need to perform full-text searches on the entirety of
> each document. Each XML document represents a physical book that was
> digitized, and the structure of each document is essentially a header with
> metadata and a body with the OCR text of the book. The OCR text is split
> into pages, where one  element contains all the words from one
> corresponding printed page from the physical book. Obviously the number of
> words in each  varies widely based on the physical dimensions of the
> book and the typeface.
>
>
>
> So far, I have loaded 12,331 documents, containing a total of 2,196,771
> pages. The total size of those XML documents on disk is 4.7GB. But that is
> only a fraction of the total number of documents I want to load into BaseX.
> The total number is more like 160,000 documents. Assuming that the
> documents I’ve loaded so far are a representative sample, and I believe
> that’s true, then the total size of the XML documents on disk, prior to
> loading them into BaseX, would be about 4.7GB * 13 = 61.1GB.
>
>
>
> Normally the OCR text, once loaded, almost never changes. But the metadata
> fields do change as corrections are made. Also we add more XML documents
> routinely as we digitize more books over time. Therefore updates and
> additions are commonplace, such that keeping indexes up to date is
> important, to allow full-text searches to stay performant. I’m wondering if
> there are techniques for optimizing such quantities of text.
>
>
>
> Thanks,
>
> Greg
>
>
>
> *From: *Christian Grün 
> *Date: *Thursday, March 14, 2024 at 8:48 AM
> *To: *Murray, Gregory 
> *Cc: *basex-talk@mailman.uni-konstanz.de <
> basex-talk@mailman.uni-konstanz.de>
> *Subject: *Re: [basex-talk] Out of Main Memory
>
> Hi Greg,
>
>
>
> A quick reply: If only parts of your documents are relevant for full-text
> queries, you can restrict the selection with the FTINDEX option (see [1]
> for more information).
>
>
>
> How large is the total size of your input documents?
>
>
>
> Best,
>
> Christian
>
>
>
> [1] https://docs.basex.org/wiki/Indexes#Selective_Indexing
>
>
>
>
>
>
>
> On Tue, Mar 12, 2024 at 8:34 PM Murray, Gregory 
> wrote:
>
> Hello,
>
>
>
> I’m working with a database that has a full-text index. I have found that
> if I iteratively add XML documents, then optimize, add more documents,
> optimize again, and so on, eventually the “optimize” command will fail with
> “Out of Main Memory.” I edited the basex startup script to change the
> memory allocation from -Xmx2g to -Xmx12g. My computer has 16 GB of memory,
> but of course the OS uses up some of it. I have found that if I exit
> memory-hungry programs (web browser, Oxygen), start basex, and then run the
> “optimize” command, I still get “Out of Main Memory.” I’m wondering if
> there are any known workarounds or strategies for this situation. If I
> understand the documentation about indexes correctly, index data is
> periodically written to disk during optimization. Does this mean that
> running optimize again will pick up where the previous attempt left off,
> such that running optimize repeatedly will eventually succeed?
>
>
>
> Thanks,
>
> Greg
>
>
>
>
>
> Gregory Murray
>
> Director of Digital Initiatives
>
> Wright Library
>
> Princeton Theological Seminary
>
>
>
>
>
>

Re: [basex-talk] Out of Main Memory

[Christian Grün]

Hi Greg,

A quick reply: If only parts of your documents are relevant for full-text
queries, you can restrict the selection with the FTINDEX option (see [1]
for more information).

How large is the total size of your input documents?

Best,
Christian

[1] https://docs.basex.org/wiki/Indexes#Selective_Indexing



On Tue, Mar 12, 2024 at 8:34 PM Murray, Gregory 
wrote:

> Hello,
>
>
>
> I’m working with a database that has a full-text index. I have found that
> if I iteratively add XML documents, then optimize, add more documents,
> optimize again, and so on, eventually the “optimize” command will fail with
> “Out of Main Memory.” I edited the basex startup script to change the
> memory allocation from -Xmx2g to -Xmx12g. My computer has 16 GB of memory,
> but of course the OS uses up some of it. I have found that if I exit
> memory-hungry programs (web browser, Oxygen), start basex, and then run the
> “optimize” command, I still get “Out of Main Memory.” I’m wondering if
> there are any known workarounds or strategies for this situation. If I
> understand the documentation about indexes correctly, index data is
> periodically written to disk during optimization. Does this mean that
> running optimize again will pick up where the previous attempt left off,
> such that running optimize repeatedly will eventually succeed?
>
>
>
> Thanks,
>
> Greg
>
>
>
>
>
> Gregory Murray
>
> Director of Digital Initiatives
>
> Wright Library
>
> Princeton Theological Seminary
>
>
>
>
>

Re: [basex-talk] validate:xsd() questions

[Christian Grün]

Hi Greg,

A helpful observation. With XQuery 4, parameters can directly be addressed
by their name, and optional arguments can be omitted:

  validate:xsd($doc, options := $options)


The changes that were required to make this work led to a bug that is now
fixed in the latest snapshot [1].

Thanks,
Christian

[1] https://files.basex.org/releases/latest/



On Mon, Mar 11, 2024 at 2:42 PM Murray, Gregory 
wrote:

> Hello,
>
>
>
> What is the default value of the “cache” option for the validate:xsd()
> function? The default doesn’t seem to be indicated in the documentation for
> that function.
>
>
>
> Also, the following code raises an error message saying “Item expected,
> empty sequence found” but that seems odd since the second argument to
> validate:xsd() is supposed to be optional, thereby telling the validation
> process to fall back to using the schema indicated in the document itself.
>
>
>
> let $options := map {'cache' : true()}
>
> for $doc in collection()
>
> return validate:xsd($doc, (), $options)
>
>
>
> Is this a bug or have I misunderstood something?
>
>
>
> Thanks,
>
> Greg
>
>
>
>
>
> Gregory Murray
>
> Director of Digital Initiatives
>
> Wright Library
>
> Princeton Theological Seminary
>
>
>
>
>

Re: [basex-talk] xslt:transform-report result

[Christian Grün]

…very appreciated, Greg! Your edits look completely fine. Thanks.


On Mon, Mar 11, 2024 at 5:49 PM Murray, Gregory 
wrote:

> Hi Christian,
>
>
>
> I made two tweaks in the documentation for xslt:transform-report. First,
> the function returns a map, but the documentation indicated xs:string as
> the return type, so I changed it to map(*).  Second, the function seems to
> work fine as long as the stylesheet doesn’t rely on , so
> indicating “Requires Saxon 10” at the beginning of the “Summary” for the
> entire function seems misleading. Based on what you’ve said below, only the
> “messages” value of the map doesn’t work without Saxon 10. So, I moved
> “Requires Saxon 10” to the “messages” description. (Using Requires Saxon
> 10 causes a display oddity, so I omitted the  wrapper.)
>
>
>
> Please revert or fix my changes as you deem best.
>
>
>
> Thanks,
>
> Greg
>
>
>
> *From: *BaseX-Talk  on behalf
> of Christian Grün 
> *Date: *Monday, March 11, 2024 at 3:12 AM
> *To: *Andy Bunce 
> *Cc: *BaseX 
> *Subject: *Re: [basex-talk] xslt:transform-report result
>
> Hi Andy,
>
>
>
> Thanks for your edits, which I’ve just revised: It turns out that Saxon 10
> is required for xslt:transform-report for both BaseX 10 and 11 beta. As
> you’ve indicated, the Saxon API seems to change with new versions. We
> haven’t checked yet what exactly has changed, and whether the API has
> changed again from version 11 to 12 .
>
>
>
> Best,
>
> Christian
>
>
>
>
>
> On Fri, Mar 8, 2024 at 6:57 PM Andy Bunce  wrote:
>
> Hi,
>
>
>
> I have recently been using xslt:transform-report to get xsl:message
> reports.
> I am using BaseX 10.7 with various versions of Saxon-he. I have run
> example 4 from[1]
> with
>
>- saxon-he-10.9.jar
>- saxon-he-11.6.jar
>- saxon-he-12.4.jar
>
> Only saxon-he-10 captures the messages for me. (although when using BaseX
> 11 I seem to recall only Saxon 12 captured them)
>
> Is this correct and expected?
>
>
>
> I understand the Saxon API may be a moving target.
>
> Perhaps the documentation could indicate a recommended Saxon version for a
> given BaseX version, rather than "For the moment, messages can only be
> returned with recent versions of Saxon."
>
> I have made minor updates to [2] to this effect. Please edit or revert if
> my understanding is incorrect.
>
>
>
> /Andy
>
> .
>
> [1] https://docs.basex.org/wiki/XSLT_Module#Examples
>
> [2] https://help.basex.org/main/XSLT_Module
>
>
>
> On Wed, 4 May 2022 at 22:16, Christian Grün 
> wrote:
>
> I agree it's somewhat unexpected. As we are working on the string result
> that is returned by Saxon, it's currently not that easy indeed to decide
> how to interpret the character stream.
>
>
>
>
>
>
>
> Andy Bunce  schrieb am Mi., 4. Mai 2022, 23:11:
>
> Ok thanks. I thought this might have been unintended behavior.
>
> if($report?result instance of document-node()+) then
> document{$report?result} else $report?result
>
> seems to give me what I expected here.
>
> /Andy
>
>
>
>
>
> On Wed, 4 May 2022 at 16:58, Christian Grün 
> wrote:
>
> Thanks, Andy, I’ve updated the documentation.
>
>
>
> On Wed, May 4, 2022 at 3:12 PM Andy Bunce  wrote:
>
> Hi,
>
>
>
> Using BaseX 9.7.1 and saxon9he-9.9.1.jar
>
> The documentation suggests the ?result from xslt:transform-report should
> be *a* document-node where possible [1]
>
> This seems not quite to be the case when there are processing instructions
> or comments at the top level. In these cases a sequence of document-nodes
> is returned.
>
>
>
> /Andy
>
> [1] https://docs.basex.org/wiki/XSLT_Module#xslt:transform-report
>
>
>
> let $xslt:=http://www.w3.org/1999/XSL/Transform;
> version="3.0">
> 
> 
>
>
>
> let $xml:=document{ , }
> return xslt:transform-report($xml,$xslt)
>
>
>
> Returns
>
> map {
>   "messages": (),
>   "result": (, )
> }
>
>
>
>
>
>
>
>
>
>

Re: [basex-talk] xslt:transform-report result

[Christian Grün]

Hi Andy,

Thanks for your edits, which I’ve just revised: It turns out that Saxon 10
is required for xslt:transform-report for both BaseX 10 and 11 beta. As
you’ve indicated, the Saxon API seems to change with new versions. We
haven’t checked yet what exactly has changed, and whether the API has
changed again from version 11 to 12 .

Best,
Christian


On Fri, Mar 8, 2024 at 6:57 PM Andy Bunce  wrote:

> Hi,
>
> I have recently been using xslt:transform-report to get xsl:message
> reports.
> I am using BaseX 10.7 with various versions of Saxon-he. I have run
> example 4 from[1]
> with
>
>- saxon-he-10.9.jar
>- saxon-he-11.6.jar
>- saxon-he-12.4.jar
>
> Only saxon-he-10 captures the messages for me. (although when using BaseX
> 11 I seem to recall only Saxon 12 captured them)
> Is this correct and expected?
>
> I understand the Saxon API may be a moving target.
> Perhaps the documentation could indicate a recommended Saxon version for a
> given BaseX version, rather than "For the moment, messages can only be
> returned with recent versions of Saxon."
> I have made minor updates to [2] to this effect. Please edit or revert if
> my understanding is incorrect.
>
> /Andy
> .
> [1] https://docs.basex.org/wiki/XSLT_Module#Examples
> [2] https://help.basex.org/main/XSLT_Module
>
>
> On Wed, 4 May 2022 at 22:16, Christian Grün 
> wrote:
>
>> I agree it's somewhat unexpected. As we are working on the string result
>> that is returned by Saxon, it's currently not that easy indeed to decide
>> how to interpret the character stream.
>>
>>
>>
>> Andy Bunce  schrieb am Mi., 4. Mai 2022, 23:11:
>>
>>> Ok thanks. I thought this might have been unintended behavior.
>>> if($report?result instance of document-node()+) then
>>> document{$report?result} else $report?result
>>> seems to give me what I expected here.
>>> /Andy
>>>
>>>
>>> On Wed, 4 May 2022 at 16:58, Christian Grün 
>>> wrote:
>>>
>>>> Thanks, Andy, I’ve updated the documentation.
>>>>
>>>> On Wed, May 4, 2022 at 3:12 PM Andy Bunce  wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Using BaseX 9.7.1 and saxon9he-9.9.1.jar
>>>>> The documentation suggests the ?result from xslt:transform-report
>>>>> should be *a* document-node where possible [1]
>>>>> This seems not quite to be the case when there are processing
>>>>> instructions or comments at the top level. In these cases a sequence of
>>>>> document-nodes is returned.
>>>>>
>>>>> /Andy
>>>>> [1] https://docs.basex.org/wiki/XSLT_Module#xslt:transform-report
>>>>>
>>>>> let $xslt:=http://www.w3.org/1999/XSL/Transform; version="3.0">
>>>>> 
>>>>> 
>>>>>
>>>>> let $xml:=document{ , }
>>>>> return xslt:transform-report($xml,$xslt)
>>>>>
>>>>> Returns
>>>>> map {
>>>>>   "messages": (),
>>>>>   "result": (, )
>>>>> }
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>

Re: [basex-talk] Troubles with ft:mark

[Christian Grün]

Hi Jack,

In general, it should always be possible to get ft:mark working somehow;
it’s just difficult to give general advice how to do it ;)

If you like, you can provide us with a stripped-down version of your code
that you can’t get to work.

Best,
Christian


On Wed, Mar 6, 2024 at 5:25 AM Jack Steyn  wrote:

> Hi Christian,
>
> Thank you very much for your explanation and variant example.
>
> In my use case, the local:search function is itself being called (as a
> named function reference) from within another function that is the endpoint
> of a RESTXQ API. This containing function handles a number of things e.g.
> pagination, deduplication, transformation of XML into HTML.
>
> Even when I rewrite this local:search to match your variant example,
> incorrect results are still returned. But when I then add %rest:GET
> annotations to turn the local:search function into its own endpoint, the
> correct results are returned only when I use that endpoint directly.
>
> Thus I assume the containing function makes things again too complicated
> for metadata to be propagated.
>
> Does that sound plausible to you? And can you suggest any simple ways
> around it? I'm afraid applying %basex:inline hasn't helped.
>
> Very best,
>
> Jack
>
> On Fri, 1 Mar 2024, 8:12 pm Christian Grün, 
> wrote:
>
>> Hi Jack,
>>
>> > When you say you can't reproduce it, do you mean you get 14 results
>> from running this script?
>>
>> Yes, that’s what I meant.
>>
>> The upcoming information will be very technical and specific. You are
>> welcome to focus on the examples.
>>
>> Your updated example was helpful, and I noticed it’s a bunch of issues
>> that lead to the unexpected results. The core challenge is that ft:mark and
>> ft:extract only yield expected results if the internally collected
>> full-text metadata is not lost at some stage during the internal processing
>> – which can happen at many places hidden to the writer of the query.
>>
>> In your specific example, the full-text information gets lost because the 
>> local:search
>> function is too complex to be inlined by the compiler (which enables
>> further optimizations that eventually allow metadata propagation). You can
>> tackle this by forcing the compiler to inline your function:
>>
>>   declare %basex:inline function local:search(...)
>>
>> Using '(ethnicgroups, languages)' instead of 'name() = (...)' is another
>> practical advice; it helps the optimizer to detect at compile time that
>> metadata will be available at runtime. Another solution is to use
>> 'local-name()' instead of 'name()' (local-name does not rely on namespace
>> that may possibly occur in a database, which also affects the way how
>> full-text queries are evaluated).
>>
>> Here’s a variant that should work:
>>
>> declare function local:search(
>>   $database  as xs:string,
>>   $query as xs:string
>> ) {
>>   let $country := ft:search($database, $query)/ancestor::country
>>   let $search := function($node) { $node/text() contains text { $query } }
>>   return (
>> ft:mark($country[.//name[$search(.)]]),
>> ft:mark($country[.//city[$search(.)]]),
>> ft:mark($country[.//(ethnicgroups, languages)[$search(.)]])
>>   )
>> };
>> local:search('factbook', 'German')
>>
>> …or…
>>
>>   let $search := function($nodes) { $nodes[text() contains text { $query
>> }] }
>>   return (ft:mark($country[$search(.//name)]), ...
>>
>> From today’s perspective, we would certainly design ft:mark and
>> ft:extract in a way that the results are always correct. The consequences,
>> however, would be a much more restricted syntax.
>>
>> Hope this helps,
>> Christian
>>
>>
>> On Thu, Feb 29, 2024 at 12:13 AM Jack Steyn  wrote:
>>
>>> Hi Christian,
>>>
>>> When I run your script, I do get 14 elements.
>>>
>>> When I run the following script I just get 12.
>>>
>>> 
>>>   true
>>>   https://files.basex.org/xml/factbook.xml
>>> 
>>>   
>>> 
>>>
>>> When you say you can't reproduce it, do you mean you get 14 results from
>>> running this script?
>>>
>>> Cheers,
>>>
>>> Jack
>>>
>>> On Thu, 29 Feb 2024, 1:02 am Christian Grün, 
>>> wrote:
>>>
>>>> Hi Jack,
>>>>
>>>> Thanks for your observation.
>>>>
>>>>
>>>>> The first result of this query is the entry for Austria. I would
>>>>> expect both of the instances of the word 'German' in that entry to be
>>>>> surrounded by  tags. However only the first instance is.
>>>>>
>>>>
>>>> I couldn’t reproduce this yet. Here’s a command script that returns 14
>>>> German elements:
>>>>
>>>> 
>>>>   true
>>>>   https://files.basex.org/xml/factbook.xml
>>>> 
>>>>   >>> )
>>>> return $marked//*[text() = 'German']
>>>>   ]]>
>>>> 
>>>>
>>>> Could you check if you get the same result?
>>>>
>>>> Thanks in advance
>>>> Christian
>>>>
>>>>

Re: [basex-talk] Inconsistent behavior for root context in BaseX GUI

[Christian Grün]

Hi Tim,

Thanks for spending so much time on this, and the visual “proof”.

A mysterious one! The query seems to be correctly evaluated on my system,
whatever I do.


>- There’s no error that’s thrown, so I’m not certain about how to
>share a stack trace?
>
>
Good to know. If no stack trace is output on command line, it’s at least no
obvious bug.


>- I’ve tried to reproduce the issue with other XML input and queries,
>but nothing else seems to trigger it.
>
>
Good to know, too ;)

We’ll keep on trying. If someone else in our group manages to trigger the
bug, I’ll give you an update. Feel free to share more information with us
if you should manage to further isolate the bug.

Best,
Christian

Re: [basex-talk] DBA Editor behavior in v11 not as expected

[Christian Grün]

…thanks; we’ve added some usability tweaks in the editor.


On Mon, Mar 4, 2024 at 11:15 AM Eliot Kimber 
wrote:

> Now that I’m using an expected extension, a bit more feedback on the
> Editor UI.
>
>
>
> Having saved a file and then not modified what’s in the editor, opening a
> new file should not prompt for confirmation as the editor isn’t “dirty” and
> there can be no data loss (the contents of the editor were just saved).
>
>
>
> In my case I’m switching between two little test scripts: one to store
> docs and one to operate on them, so having to respond to an unnecessary
> confirmation every time I switch is annoying.
>
>
>
> Cheers,
>
>
>
> E.
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com <https://www.servicenow.com>
>
> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
> <https://twitter.com/servicenow> | YouTube
> <https://www.youtube.com/user/servicenowinc> | Facebook
> <https://www.facebook.com/servicenow>
>
>
>
> *From: *Christian Grün 
> *Date: *Friday, March 1, 2024 at 2:58 PM
> *To: *Eliot Kimber 
> *Cc: *BaseX 
> *Subject: *Re: [basex-talk] DBA Editor behavior in v11 not as expected
> *[External Email]*
>
>
> --
>
> Good to know, it'll be easy to fix that [1]; we should do so anyway [2].
>
>
>
> [1]
> https://github.com/BaseXdb/basex/blob/main/basex-api/src/main/webapp/dba/static/editor.js#L56
>
> [2] https://help.basex.org/main/XQuery_Extensions#suffixes
>
>
>
> Eliot Kimber  schrieb am Fr., 1. März 2024,
> 21:41:
>
> I use “.xqy” for full XQuery scripts and “.xqm” for XQuery modules.
>
>
>
> Cheers,
>
>
>
> E.
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com <https://www.servicenow.com>
>
> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
> <https://twitter.com/servicenow> | YouTube
> <https://www.youtube.com/user/servicenowinc> | Facebook
> <https://www.facebook.com/servicenow>
>
>
>
> *From: *Christian Grün 
> *Date: *Friday, March 1, 2024 at 10:57 AM
> *To: *Eliot Kimber 
> *Cc: *BaseX 
> *Subject: *Re: [basex-talk] DBA Editor behavior in v11 not as expected
> *[External Email]*
>
>
> --
>
> Hi Eliot,
>
>
>
> We have aligned the behavior of the DBA editor to the BaseX GUI. Saved
> files are only executable if they are detected as XQuery file.
>
>
>
> But maybe we need to support some more file suffixes (.xq is the default
> extension). How do you name your files?
>
>
>
> Best,
>
> Christian
>
>
>
>
>
> Eliot Kimber  schrieb am Fr., 1. März 2024,
> 17:52:
>
> Using the 27-02-2024 build of v11, if I create a query in the Editor panel
> and then save it, the Run button is disabled, which is unexpected.
>
>
>
> If I then open the file, the Run button remains disabled.
>
>
>
> If I cut the contents of the editing panel, select “close” to clear the
> panel, then paste back into the panel, the Run button is enabled.
>
>
>
> I think this behavior is incorrect—as long as there is some content in the
> editor the Run button should be enabled.
>
>
>
> Cheers,
>
>
>
> E.
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com <https://www.servicenow.com>
>
> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
> <https://twitter.com/servicenow> | YouTube
> <https://www.youtube.com/user/servicenowinc> | Facebook
> <https://www.facebook.com/servicenow>
>
>

Re: [basex-talk] Inconsistency in base-uri()

[Christian Grün]

…just a quick reply: That’s probably related to [1], an ancient issue, in
which I tended to recommend the usage of db:path. I wish we’d finally find
time and ressources to tackle this.


[1] https://github.com/BaseXdb/basex/issues/1172

On Mon, Mar 4, 2024 at 11:49 AM Eliot Kimber 
wrote:

> Using BaseX 11 (but I think the code is the same in BaseX 10).
>
>
>
> I’m trying to understand how base-uri() behaves relative to how it should
> behave when the database path of a document is not a valid URI, i.e., it
> has a space in it.
>
>
>
> First I have this test:
>
>
>
> let $doc as document-node() := document {  xml:base="temp/child-uri%20with%20space.xml">child }
>
> return $doc/*/child ! base-uri(.)
>
>
>
> Which produces:
>
> file:///data/basex/data/.dba/temp/child-uri%20with%20space.xml
>
>
>
> Which is the correct result: it’s the value of @xml:base and the escaped
> spaces make it a valid URI.
>
>
>
> Replacing %20 with “ “ in the @xml:base value results in this error:
>
> * Invalid URI: Illegal character in path at index 14: temp/child-uri with
> space.xml.*
>
>
>
> Also correct as the spaces have to be escaped.
>
>
>
> This verifies that base-uri() applied to nodes with explicit @xml:base
> attributes work per the spec. But this test does not involve database paths.
>
>
>
> To try to test things with database paths I then created this pair of test
> scripts:
>
> Script to put docs in a database:
>
> let $db := 'temp'
>
> let $filename as xs:string := 'with space.xml'
>
> let $doc1 as document-node() := document {No
> xml:base}
>
> let $doc2 as document-node() := document {  xml:base="{'/temp/xmlbase/doc2_' || $filename}">With xml:base
> unescaped }
>
> let $doc3 as document-node() := document {  xml:base="{iri-to-uri( '/temp/xmlbase/doc3_' || $filename)}">With xml:base
> escaped }
>
> return (()
>
> ,db:put($db, $doc1, 'doc1_' || $filename)
>
> ,db:put($db, $doc2, 'doc2_' || $filename)
>
> ,db:put($db, $doc3, 'doc3_' || $filename)
>
> )
>
>
>
> Script to report on them:
>
> let $db := 'temp'
>
> let $filenameBase as xs:string := 'with space.xml'
>
> return
>
> for $i in 1 to 3
>
>   let $filename := 'doc' || $i  || '_' || $filenameBase
>
>   let $doc := db:get($db, $filename)
>
>   let $child as element() := $doc/*/child
>
>   let $dbPath := db:path($doc)
>
>   let $baseUriDoc := base-uri($doc)
>
>   let $baseUriChild :=
>
>   try {
>
> base-uri($child)
>
>   } catch * {
>
> $err:description
>
>   }
>
>   return (()
>
>,``[
>
> Doc "`{$dbPath}`":]``
>
>,$doc
>
>,``[xml:base att:  "`{$child/@xml:base}`"]``
>
>,``[base URI of doc:  "`{$baseUriDoc}`"]``
>
>,``[base URI of child: "`{$baseUriChild}`"]``
>
>   )
>
>
>
> Which returns this result:
>
> Doc "doc1_with space.xml":
>
> 
>
>   No xml:base
>
> 
>
> xml:base att:  ""
>
> base URI of doc:  "/temp/doc1_with space.xml"
>
> base URI of child: "/temp/doc1_with space.xml"
>
>
>
> Doc "doc2_with space.xml":
>
> 
>
>   With xml:base
> unescaped
>
> 
>
> xml:base att:  "/temp/xmlbase/doc2_with space.xml"
>
> base URI of doc:  "/temp/doc2_with space.xml"
>
> base URI of child: "Invalid URI: Illegal character in path at index 23:
> /temp/xmlbase/doc2_with space.xml."
>
>
>
> Doc "doc3_with space.xml":
>
> 
>
>   With xml:base
> escaped
>
> 
>
> xml:base att:  "/temp/xmlbase/doc3_with%20space.xml"
>
> base URI of doc:  "/temp/doc3_with space.xml"
>
> base URI of child: "Invalid URI: Illegal character in path at index 15:
> /temp/doc3_with space.xml."
>
>
>
> Note the result for doc3: It’s reporting the base URI of the document
> (/temp/doc3_with space.xml), not the base URI of the child
> (/temp/xmlbase/doc_with%20space.xml). Why? I think the answer is that under
> the covers it’s doing resolve-uri(), which also checks the validity of both
> the base and relative parts.
>
>
>
> One observation is that base-uri() is treating the db-provided base URI
> differently from an xml:base-provided base URI, but only when there is no
> @xml:base attribute.
>
>
>
> In doc 1, the database path has a space but base-uri() does not fail when
> returning it even though it’s not a valid URI. Why not?
>
>
>
> In doc 2, the xml:base-supplied base URI is correctly reported as invalid,
> but the database-supplied base URI of the root is not reported as invalid.
>
>
>
> My expectation would be that the behavior is consistent: Either all URIs
> must be valid, including those coming from database paths or all are
> automatically escaped (as though iri-to-uri() had been applied).
>
>
>
> Finally, why do I get the result for doc 3, where it’s reporting the
> database path as the base URI of the child rather than the
> @xml:base-defined base URI (which is correctly escaped).
>
>
>
> In my code, which depends on the use of @xml:base to do DITA link
> resolution for “resolved” DITA maps, I’ve adjusted my code to escape URIs
> in @xml:base values and as far as I can tell everything works as it should.
> But I’m still concerned about the 

Re: [basex-talk] Inconsistent behavior for root context in BaseX GUI

[Christian Grün]

Hi Tim,

Thanks for your observation. We need more help to reproduce this: Could you
describe in detail, and step by step, how to trigger this issue?

In addition, you could…
• tell us more about your OS and JDK version
• start BaseX on command line and share a possible stack trace with us
• verify if the error also occurs with other XML input (such as ) and
other queries (such as .)

The simpler the use case, the better…

Thanks
Christian



On Fri, Mar 1, 2024 at 6:11 PM Thompson, Timothy 
wrote:

> I’ve been noticing an inconsistent behavior in the GUI with BaseX 11 beta
> (build 17d8426).
>
>
>
> I have a simple query on a database created from a CSV file. Sometimes it
> works, but sometimes the root context is evaluated as an empty sequence:
>
>
>
> - rewrite context value: . -> ()
>
> - rewrite util:root(nodes): util:root(()) -> ()
>
>
>
> I notice this when I open the database as context and try to execute the
> query. After the query fails, if I try to click on the “home” icon in the
> result panel, it also returns an empty sequence, even though the database
> is open.
>
>
>
> If I reopen the database and run the query, it works again.
>
>
>
> Sample data:
>
>
>
> 
>
>   
>
> http://id.loc.gov/authorities/names/n0121
>
> http://id.loc.gov/rwo/agents/n0121
>
> 0.11861849
>
>   
>
>   
>
> http://id.loc.gov/authorities/names/n0122
>
> http://id.loc.gov/rwo/agents/n0122
>
> 0.11699477
>
>   
>
>   
>
> http://id.loc.gov/authorities/names/n0267
>
> http://id.loc.gov/rwo/agents/n0267
>
> 0.10811427
>
>   
>
> 
>
>
>
> Query (with DB open as context):
>
>
>
> count(distinct-values(
>
>   for $rec in /csv/record
>
>   where contains($rec/entry[1], "id.loc.gov")
>
>   return $rec/entry[1]
>
> ))
>
>
>
> Thanks in advance,
>
> Tim
>
>
>

Re: [basex-talk] DBA Editor behavior in v11 not as expected

[Christian Grün]

Good to know, it'll be easy to fix that [1]; we should do so anyway [2].

[1]
https://github.com/BaseXdb/basex/blob/main/basex-api/src/main/webapp/dba/static/editor.js#L56
[2] https://help.basex.org/main/XQuery_Extensions#suffixes


Eliot Kimber  schrieb am Fr., 1. März 2024,
21:41:

> I use “.xqy” for full XQuery scripts and “.xqm” for XQuery modules.
>
>
>
> Cheers,
>
>
>
> E.
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com <https://www.servicenow.com>
>
> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
> <https://twitter.com/servicenow> | YouTube
> <https://www.youtube.com/user/servicenowinc> | Facebook
> <https://www.facebook.com/servicenow>
>
>
>
> *From: *Christian Grün 
> *Date: *Friday, March 1, 2024 at 10:57 AM
> *To: *Eliot Kimber 
> *Cc: *BaseX 
> *Subject: *Re: [basex-talk] DBA Editor behavior in v11 not as expected
> *[External Email]*
>
>
> --
>
> Hi Eliot,
>
>
>
> We have aligned the behavior of the DBA editor to the BaseX GUI. Saved
> files are only executable if they are detected as XQuery file.
>
>
>
> But maybe we need to support some more file suffixes (.xq is the default
> extension). How do you name your files?
>
>
>
> Best,
>
> Christian
>
>
>
>
>
> Eliot Kimber  schrieb am Fr., 1. März 2024,
> 17:52:
>
> Using the 27-02-2024 build of v11, if I create a query in the Editor panel
> and then save it, the Run button is disabled, which is unexpected.
>
>
>
> If I then open the file, the Run button remains disabled.
>
>
>
> If I cut the contents of the editing panel, select “close” to clear the
> panel, then paste back into the panel, the Run button is enabled.
>
>
>
> I think this behavior is incorrect—as long as there is some content in the
> editor the Run button should be enabled.
>
>
>
> Cheers,
>
>
>
> E.
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com <https://www.servicenow.com>
>
> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
> <https://twitter.com/servicenow> | YouTube
> <https://www.youtube.com/user/servicenowinc> | Facebook
> <https://www.facebook.com/servicenow>
>
>

Re: [basex-talk] DBA Editor behavior in v11 not as expected

[Christian Grün]

Hi Eliot,

We have aligned the behavior of the DBA editor to the BaseX GUI. Saved
files are only executable if they are detected as XQuery file.

But maybe we need to support some more file suffixes (.xq is the default
extension). How do you name your files?

Best,
Christian



Eliot Kimber  schrieb am Fr., 1. März 2024,
17:52:

> Using the 27-02-2024 build of v11, if I create a query in the Editor panel
> and then save it, the Run button is disabled, which is unexpected.
>
>
>
> If I then open the file, the Run button remains disabled.
>
>
>
> If I cut the contents of the editing panel, select “close” to clear the
> panel, then paste back into the panel, the Run button is enabled.
>
>
>
> I think this behavior is incorrect—as long as there is some content in the
> editor the Run button should be enabled.
>
>
>
> Cheers,
>
>
>
> E.
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com 
>
> LinkedIn  | Twitter
>  | YouTube
>  | Facebook
> 
>

Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state

[Christian Grün]

Glad to hear it, thanks Eliot.


Eliot Kimber  schrieb am Fr., 1. März 2024,
16:21:

> Using the 27-02-2024 build I have confirmed that the optimize database
> deadlock seems to be resolved.
>
>
>
> I was able to easily upgrade my code to replace prof:dump() with message()
> and db:open() with db:get() and everything else seems to be working as it
> should.
>
>
>
> I like the new Editor replacement for the old Query feature in the DBA app.
>
>
>
> Cheers,
>
>
>
> E.
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com <https://www.servicenow.com>
>
> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
> <https://twitter.com/servicenow> | YouTube
> <https://www.youtube.com/user/servicenowinc> | Facebook
> <https://www.facebook.com/servicenow>
>
>
>
> *From: *Christian Grün 
> *Date: *Wednesday, February 28, 2024 at 9:36 AM
> *To: *Eliot Kimber 
> *Cc: *basex-talk@mailman.uni-konstanz.de <
> basex-talk@mailman.uni-konstanz.de>
> *Subject: *Re: [basex-talk] Optimize database never returns, leaves
> database in "opened by another process" state
> *[External Email]*
>
>
> --
>
> …this one could be related to a bug that was recently fixed in the latest
> snapshot [1]. About time to get BaseX 11 finished…
>
>
>
> [1]
> https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496
>
>
>
>
>
>
>
> On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber 
> wrote:
>
> Using Basex 10.7 on Linux.
>
>
> I’m running a sequence of jobs to update and optimize a set of databases
> following loading a number of documents created dynamically (as opposed to
> being read from disk).
>
>
>
> I’m seeing a new behavior, which is that the optimization step never
> completes but also doesn’t show any error in the log. The database shows no
> items and is in the locked by another process state if I try to drop it.
> This behavior seems to be consistently repeatable with my current code base
> (I’m working on some code updates, so it’s possible I’ve introduced
> something that would cause this behavior but I haven’t changed the code
> that leads up to the failing optimize). The server has plenty of disk
> space, etc.
>
>
>
> Optimization code is:
>
>try {
>
>  if (db:exists($database))
>
>  then
>
>  (
>
>util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing
> database `{$database}`]``),
>
>db:optimize($database, true(), $dbadmin:dbOptimizeOptions)
>
>  )
>
>  else util:logToConsole('dbadmin:optimizeDatabase', ``[Database
> '`{$database}`' does not exist. Nothing to optimize.]``)
>
>} catch * {
>
>  util:logToConsole(
>
>'dbadmin:optimizeDatabase',
>
>``[Exception optimizing database '`{$database}`': `{$err:code}` -
> `{$err:description}`]``,
>
>'error')
>
>}
>
>
>
> And the optimization options are:
>
> declare variable $dbadmin:dbOptimizeOptions as map(*) :=
>
> (: Turn on all the indexes :)
>
>   map {
>
> 'attrindex' : true(),
>
> 'tokenindex' : true(),
>
> 'textindex' : true(),
>
> 'ftindex' : true()
>
>   };
>
> This code has been working fine for a long time and I’ve been running 10.7
> for a least a couple of months, so I’m wondering:
>
> A) Would would cause this behavior?
> B) How can I diagnose it short of debugging the Java code (which I can do
> but it’s non-trivial for me to set up).
>
>
>
> Thanks,
>
>
>
> Eliot
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com <https://www.servicenow.com>
>
> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
> <https://twitter.com/servicenow> | YouTube
> <https://www.youtube.com/user/servicenowinc> | Facebook
> <https://www.facebook.com/servicenow>
>
>

Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state

[Christian Grün]

…exactly; thanks, Andy. Time works against us, so we’ve added a link to the
new documentation on the start page of docs.basex.org.


On Thu, Feb 29, 2024 at 11:15 AM Andy Bunce  wrote:

> Maybe: https://help.basex.org/main/Profiling_Module
>
> /Andy
>
> On Thu, 29 Feb 2024 at 09:40, Eliot Kimber 
> wrote:
>
>> Using the latest build, 11.0 beta 17d8426, the prof:dump() function is
>> reported as an unknown function.
>>
>>
>>
>> What replaces it (or where can I find the V11 docs)?
>>
>>
>>
>> Thanks,
>>
>>
>>
>> E.
>>
>>
>>
>>
>>
>> _
>>
>> *Eliot Kimber*
>>
>> Sr Staff Content Engineer
>>
>> O: 512 554 9368
>>
>> M: 512 554 9368
>>
>> servicenow.com <https://www.servicenow.com>
>>
>> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
>> <https://twitter.com/servicenow> | YouTube
>> <https://www.youtube.com/user/servicenowinc> | Facebook
>> <https://www.facebook.com/servicenow>
>>
>>
>>
>> *From: *Eliot Kimber 
>> *Date: *Thursday, February 29, 2024 at 3:15 AM
>> *To: *Christian Grün 
>> *Cc: *basex-talk@mailman.uni-konstanz.de <
>> basex-talk@mailman.uni-konstanz.de>
>> *Subject: *Re: [basex-talk] Optimize database never returns, leaves
>> database in "opened by another process" state
>>
>> Found the latest build at https://files.basex.org/releases/latest/
>>
>>
>>
>> Cheers,
>>
>>
>>
>> E.
>>
>>
>>
>> _
>>
>> *Eliot Kimber*
>>
>> Sr Staff Content Engineer
>>
>> O: 512 554 9368
>>
>> M: 512 554 9368
>>
>> servicenow.com <https://www.servicenow.com>
>>
>> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
>> <https://twitter.com/servicenow> | YouTube
>> <https://www.youtube.com/user/servicenowinc> | Facebook
>> <https://www.facebook.com/servicenow>
>>
>>
>>
>> *From: *Eliot Kimber 
>> *Date: *Thursday, February 29, 2024 at 2:45 AM
>> *To: *Christian Grün 
>> *Cc: *basex-talk@mailman.uni-konstanz.de <
>> basex-talk@mailman.uni-konstanz.de>
>> *Subject: *Re: [basex-talk] Optimize database never returns, leaves
>> database in "opened by another process" state
>>
>> I’m trying to compile the latest code but “mvn clean install” fails on
>> failure to download some dependencies:
>>
>> [*INFO*] *--< *org.basex:basex*
>> >---*
>>
>> [*INFO*] *Building BaseX Core 11.0-SNAPSHOT*
>>
>> [*INFO*]   from pom.xml
>>
>> [*INFO*] *[ jar
>> ]-*
>>
>> Downloading from devsnc-mirror:
>> http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.pom
>>
>> [*WARNING*] The POM for jp.sourceforge.igo:igo:jar:0.4.3 is missing, no
>> dependency information available
>>
>> Downloading from devsnc-mirror:
>> http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.pom
>>
>> [*WARNING*] The POM for org.apache:lucene-stemmers:jar:3.4.0 is missing,
>> no dependency information available
>>
>> Downloading from devsnc-mirror:
>> http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.jar
>>
>> Downloading from devsnc-mirror:
>> http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.jar
>>
>> [*INFO*]
>> **
>>
>> [*INFO*] *BUILD FAILURE*
>>
>> [*INFO*]
>> **
>>
>> [*INFO*] Total time:  5.589 s
>>
>> [*INFO*] Finished at: 2024-02-29T02:34:11-06:00
>>
>> [*INFO*]
>> **
>>
>> [*ERROR*] Failed to execute goal on project basex: *Could not resolve
>> dependencies for project org.basex:basex:jar:11.0-SNAPSHOT: The following
>> artifacts could not be resolved: jp.sourceforge.igo:igo:jar:0.4.3 (absent),
>> org.apache:lucene-stemmers:jar:3.4.0 (absent): Could not find artifact
>> jp.sourceforge.igo:igo:jar:0.4.3 in devsnc-mirror

Re: [basex-talk] Troubles with ft:mark

[Christian Grün]

Hi Jack,

> When you say you can't reproduce it, do you mean you get 14 results from
running this script?

Yes, that’s what I meant.

The upcoming information will be very technical and specific. You are
welcome to focus on the examples.

Your updated example was helpful, and I noticed it’s a bunch of issues that
lead to the unexpected results. The core challenge is that ft:mark and
ft:extract only yield expected results if the internally collected
full-text metadata is not lost at some stage during the internal processing
– which can happen at many places hidden to the writer of the query.

In your specific example, the full-text information gets lost because
the local:search
function is too complex to be inlined by the compiler (which enables
further optimizations that eventually allow metadata propagation). You can
tackle this by forcing the compiler to inline your function:

  declare %basex:inline function local:search(...)

Using '(ethnicgroups, languages)' instead of 'name() = (...)' is another
practical advice; it helps the optimizer to detect at compile time that
metadata will be available at runtime. Another solution is to use
'local-name()' instead of 'name()' (local-name does not rely on namespace
that may possibly occur in a database, which also affects the way how
full-text queries are evaluated).

Here’s a variant that should work:

declare function local:search(
  $database  as xs:string,
  $query as xs:string
) {
  let $country := ft:search($database, $query)/ancestor::country
  let $search := function($node) { $node/text() contains text { $query } }
  return (
ft:mark($country[.//name[$search(.)]]),
ft:mark($country[.//city[$search(.)]]),
ft:mark($country[.//(ethnicgroups, languages)[$search(.)]])
  )
};
local:search('factbook', 'German')

…or…

  let $search := function($nodes) { $nodes[text() contains text { $query }]
}
  return (ft:mark($country[$search(.//name)]), ...

>From today’s perspective, we would certainly design ft:mark and ft:extract
in a way that the results are always correct. The consequences, however,
would be a much more restricted syntax.

Hope this helps,
Christian


On Thu, Feb 29, 2024 at 12:13 AM Jack Steyn  wrote:

> Hi Christian,
>
> When I run your script, I do get 14 elements.
>
> When I run the following script I just get 12.
>
> 
>   true
>   https://files.basex.org/xml/factbook.xml
> 
>   
> 
>
> When you say you can't reproduce it, do you mean you get 14 results from
> running this script?
>
> Cheers,
>
> Jack
>
> On Thu, 29 Feb 2024, 1:02 am Christian Grün, 
> wrote:
>
>> Hi Jack,
>>
>> Thanks for your observation.
>>
>>
>>> The first result of this query is the entry for Austria. I would expect
>>> both of the instances of the word 'German' in that entry to be surrounded
>>> by  tags. However only the first instance is.
>>>
>>
>> I couldn’t reproduce this yet. Here’s a command script that returns 14
>> German elements:
>>
>> 
>>   true
>>   https://files.basex.org/xml/factbook.xml
>> 
>>   > )
>> return $marked//*[text() = 'German']
>>   ]]>
>> 
>>
>> Could you check if you get the same result?
>>
>> Thanks in advance
>> Christian
>>
>>

Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state

[Christian Grün]

…this one could be related to a bug that was recently fixed in the latest
snapshot [1]. About time to get BaseX 11 finished…

[1]
https://github.com/BaseXdb/basex/commit/45d97f8065615fb734b712bc4c77c39899e9d496



On Mon, Feb 26, 2024 at 5:25 PM Eliot Kimber 
wrote:

> Using Basex 10.7 on Linux.
>
>
> I’m running a sequence of jobs to update and optimize a set of databases
> following loading a number of documents created dynamically (as opposed to
> being read from disk).
>
>
>
> I’m seeing a new behavior, which is that the optimization step never
> completes but also doesn’t show any error in the log. The database shows no
> items and is in the locked by another process state if I try to drop it.
> This behavior seems to be consistently repeatable with my current code base
> (I’m working on some code updates, so it’s possible I’ve introduced
> something that would cause this behavior but I haven’t changed the code
> that leads up to the failing optimize). The server has plenty of disk
> space, etc.
>
>
>
> Optimization code is:
>
>try {
>
>  if (db:exists($database))
>
>  then
>
>  (
>
>util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing
> database `{$database}`]``),
>
>db:optimize($database, true(), $dbadmin:dbOptimizeOptions)
>
>  )
>
>  else util:logToConsole('dbadmin:optimizeDatabase', ``[Database
> '`{$database}`' does not exist. Nothing to optimize.]``)
>
>} catch * {
>
>  util:logToConsole(
>
>'dbadmin:optimizeDatabase',
>
>``[Exception optimizing database '`{$database}`': `{$err:code}` -
> `{$err:description}`]``,
>
>'error')
>
>}
>
>
>
> And the optimization options are:
>
> declare variable $dbadmin:dbOptimizeOptions as map(*) :=
>
> (: Turn on all the indexes :)
>
>   map {
>
> 'attrindex' : true(),
>
> 'tokenindex' : true(),
>
> 'textindex' : true(),
>
> 'ftindex' : true()
>
>   };
>
> This code has been working fine for a long time and I’ve been running 10.7
> for a least a couple of months, so I’m wondering:
>
> A) Would would cause this behavior?
> B) How can I diagnose it short of debugging the Java code (which I can do
> but it’s non-trivial for me to set up).
>
>
>
> Thanks,
>
>
>
> Eliot
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com 
>
> LinkedIn  | Twitter
>  | YouTube
>  | Facebook
> 
>

Re: [basex-talk] Odd "Access denied" messages in the log

[Christian Grün]

Hi Eliot,

It’s difficult to tell which requests are sent to BaseX. Maybe you can use
a networking monitoring tool such as Wireshark to get more hints?

Best,
Christian


On Sun, Feb 25, 2024 at 11:59 PM Eliot Kimber 
wrote:

> I have an application (our Mirabel system) running on a server inside our
> firewall (so not visible to the open Internet).
>
>
>
> I’ve recently started seeing messages like this in the log:
>
> Access denied: .
>
> Access denied: PRI * HTTP/2.0 SM .
>
>
>
> Where the value reported can be quite varied, but is often unrenderable
> characters or other stuff. (In this case the characters are all \uFFFD).
>
> The log messages all report the same IP address.
>
>
>
> This server does not use named users, so there’s no authentication
> required to access it.
>
>
>
> The IP address is not one of my own servers, so I don’t think it’s
> something generated by my own code.
>
>
>
> Any idea what this might be? It’s started relatively recently, which makes
> me think it might be some sort of penetration test.
>
>
>
> Cheers,
>
>
>
> E.
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com 
>
> LinkedIn  | Twitter
>  | YouTube
>  | Facebook
> 
>

Re: [basex-talk] Troubles with ft:mark

[Christian Grün]

Hi Jack,

Thanks for your observation.


> The first result of this query is the entry for Austria. I would expect
> both of the instances of the word 'German' in that entry to be surrounded
> by  tags. However only the first instance is.
>

I couldn’t reproduce this yet. Here’s a command script that returns 14
German elements:


  true
  https://files.basex.org/xml/factbook.xml

  


Could you check if you get the same result?

Thanks in advance
Christian

Re: [basex-talk] basex failed to cast large numbers as xs:integer

[Christian Grün]

Hi Max,

Thanks for your report.

The behavior is correct, but I agree it’s surprising:

• //* returns two results:  and 
• the atomized value of  is '14588204311438466813'. This value
exceeds the BaseX limit for integers (2^64), which is why [not(. castable
as xs:integer)] returns true for this element.
• the atomized value of  is 1458820431. This value can be
represented as xs:integer, so – as expected – [not(. castable as
xs:integer)] reurns false.

If you want to want to check for numbers that exceed the 2^64 limit, you
can use xs:decimal tests. If you want to check if the single text nodes are
integers, you can use:

//text()[not(. castable as xs:integer)]

Hope this helps,
Christian

Re: [basex-talk] Whitespace

[Christian Grün]

Hi Owen,

Do you have specific problems with whitespace in your query service? If
yes, which version of BaseX are you using?

Best,
Christian


On Wed, Feb 14, 2024 at 6:22 PM Owen Ambur  wrote:

> Lack of capability to deal appropriately with whitespaces (and
> punctuation) results in false positives in our StratML-enabled query
> service at https://search.aboutthem.info/
>
> Will look forward to learning if anything can be done about it.
>
> Owen Ambur
> https://www.linkedin.com/in/owenambur/
>
>

Re: [basex-talk] Found problems with map:for-each

[Christian Grün]

Hi Vincenzo,

Thanks for your observation and the easily reproducible test case; we’ve
uploaded a new stable snapshot with a bug fix.

Please note that it’s generally risky to explicitly return empty sequences,
as the optimizer will always try to get rid of code that may not contribute
to the final result (we try hard, though, to keep code alive that has side
effects, such as file:write-binary). In the given case, you could simply
rewrite your code as follows:

let $files := map {"hello1.txt" : ... }
return map:for-each($files, function($filename, $content) {
  file:write-binary($filename, $content, 0)
})

Ciao,
Christian

[1] https://files.basex.org/releases/latest/



On Mon, Feb 19, 2024 at 4:51 PM Vincenzo Cestone 
wrote:

> Hi all,
>
> with the basex 10.7 version (but even in 9.6 version I have the same
> issue) I found that the following code wont work as expected:
> (: It wont work :)
> let $files := map {"hello1.txt" : xs:base64Binary('SGVsbG8gd29ybGQ='),
> "hello2.txt" : xs:base64Binary('SGVsbG8gd29ybGQ=')}
> let $w := map:for-each($files, function($filename, $content) {
>   file:write-binary($filename, $content, 0)
> })
> return ()
>
> that is, it will not write the two files hello1.txt and hello2.txt in the
> basex_home/bin folder.
>
> But, if you return $w instead:
> (: It works :)
> let $files := map {"hello1.txt" : xs:base64Binary('SGVsbG8gd29ybGQ='),
> "hello2.txt" : xs:base64Binary('SGVsbG8gd29ybGQ=')}
> let $w := map:for-each($files, function($filename, $content) {
>   file:write-binary($filename, $content, 0)
> })
> return $w
>
> With a similar implementation, with the classic FLWOR, the issue does not
> arise, even if I return the empty sequence:
> (: It works :)
> let $files := map {"hello1.txt" : xs:base64Binary('SGVsbG8gd29ybGQ='),
> "hello2.txt" : xs:base64Binary('SGVsbG8gd29ybGQ=')}
> let $w := for $filename in map:keys($files)
>   return file:write-binary($filename, map:get($files, $filename), 0)
> return ()
>
> that is it will write two files hello1.txt and hello2.txt in
> basex_home/bin folder.
>
> Note that the files in the examples above are for demonstration purposes,
> however in my code they are actually binary files.
> My java version is a Oracle JDK 17
>
> This problem also occur to others?
>
> Thanks,
> Vincenzo
>

Re: [basex-talk] Slow full-text querying

[Christian Grün]

Dear Greg,

In BaseX, it’s the text nodes that are indexed. Here are some query that
take advantage of the full-text index:

db:get("theocom")[.//text() contains text 'apple']
db:get("theocom")//page[text() contains text 'apple']
db:get("theocom")//text()[. contains text 'apple']
...

If you check out the output of the GUI’s info panel, you’ll see whether the
full-text index is applied.

There are several ways to compute scores for documents; here are two
variants:

let $db := 'theocom'
let $keywords := 'apple'
let $search := function($doc) { $doc//text() contains text { $keywords } }
for $doc in db:get($db)[$search(.)]
order by ft:score($search($doc)) descending
return db:path($doc)

let $keywords := "apple"
for $texts score $score in db:get("theocom")//text()[. contains text {
$keywords }]
group by $uri := db:path($texts)
let $scores := sum($score)
order by $scores descending
return ($scores, $uri, $texts)

By evaluating specific score values for text nodes, you have more freedom
to decide how to interpret the scores. For example, you can rank scores of
title elements higher than those of paragraphs.

I invite you to have a look at our documentation for more information and
examples [1,2].

Hope this helps,
Christian

[1] https://docs.basex.org/wiki/Full-Text
[2] https://docs.basex.org/wiki/Indexes#Full-Text_Index



On Sat, Feb 17, 2024 at 1:23 PM Murray, Gregory 
wrote:

> Hello,
>
>
>
> I have a database with several thousand XML documents, although I have
> tens of thousands I’d like to add. Each XML document contains a book — both
> the bibliographic metadata such as title, author, etc. (each in its own
> element) and the complete OCR text of all pages of the book. Each page of
> text from each book is in a  element with a single text node
> containing all words from that page in the book, resulting in large blocks
> of text.
>
>
>
> I’ve added a full-text index and optimized it. I am finding that full-text
> searching is very slow. The query shown below consistently takes about 20
> seconds to run, even though there are only about 7400 documents. Obviously
> that’s far too slow to use the query in a web application, where the user
> expects a quick response.
>
>
>
> My first thought is whether the query is actually using the full-text
> index. Is there a way for me to determine that?
>
>
>
> I’m also wondering if my query is crude or is missing something. I don’t
> need the text nodes containing the search words; I only need to know which
> documents contain the words.
>
>
>
> let $keywords := "apple"
>
> for $doc in collection("theocom")
>
> let score $score := $doc contains text {$keywords}
>
> order by $score descending
>
> where $score > 0
>
> return concat($score, " ", base-uri($doc))
>
>
>
> As you can see, I’m searching all text in the entirety of each book. Is
> there a way to rewrite such a query for faster performance?
>
>
>
> Also, I’m wondering if the structure of the XML documents is such that the
> documents themselves need to have smaller blocks of text. For example, if
> the OCR text were contained in  elements, each containing only a
> single line of text, as printed in the original physical book, would
> full-text searching be noticeably faster, since each text node is much
> smaller?
>
>
>
> Thanks,
>
> Greg
>
>
>

Re: [basex-talk] Help with loading of 9 million documents

[Christian Grün]

Thanks for the addition, Liam; I should have mentioned that.

If your input has mixed content, and if the relevant sections have
xml:space='preserve' attributes…

The very tc34q.

…whitespace stripping will be safe.

Similarly, it may be helpful to know that the whitspace gets lost if XML
strings…

The very tc34q.

…are evaluated as XQuery. To prevent that, you can add a statement to the
prolog of the query:

declare boundary-space preserve;
The very tc34q.

Whitespace handling is generally a tricky issue in XML.

Best,
Christian


On Wed, Feb 14, 2024 at 10:38 AM Liam R. E. Quin 
wrote:

> On Tue, 2024-02-13 at 20:29 +0100, Christian Grün wrote:
>
>
> If your XML input has been properly indented to improve readibility, you
> can reduce the size of your database by dropping superfluous whitespace
> during the import:
>
> SET STRIPWS ON; CREATE DB ...
> db:create('db', '/path/to/documents', (), map { 'stripws': true() })
>
>
> Beware that this is not schema-based, and can remove whitespace nodes in
> mixed content -
> The very tc34q.
> may become (as i understand it)
> The verytc34q.
> (i have seen this, with different software, cause potentially catastrophic
> problems in aircraft manuals!)
>
> liam
>
> --
>
> Liam Quin, https://www.delightfulcomputing.com/
> Available for XML/Document/Information Architecture/XSLT/
> XSL/XQuery/Web/Text Processing/A11Y training, work & consulting.
> Barefoot Web-slave, antique illustrations:  http://www.fromoldbooks.org
>

Re: [basex-talk] Help with loading of 9 million documents

[Christian Grün]

Hi Dietmar,

> Or is there a way to speed this up?
The fastest solution is to import all documents during database creation,
either with the CREATE DB command or the corresponding XQuery function:

CREATE DB name-of-db /path/to/documents
db:create('db', '/path/to/documents')

The database command ADD, or db:add, can be used as well to import more
than one document at a time.

If your XML input has been properly indented to improve readibility, you
can reduce the size of your database by dropping superfluous whitespace
during the import:

SET STRIPWS ON; CREATE DB ...
db:create('db', '/path/to/documents', (), map { 'stripws': true() })

> Have you ever loaded 9 million documents into a basex database?

What’s the approximate size of the 32,000 documents?

In principle, it’s no problem to add 10 million documents or more to a
database as long as the input doesn’t exceed specific limits [1]. If you
exceed the limits, you can create multiple databases and access them with a
single query [2].

> I load the documents using the BaseXClient and the ADD method.

Are you using the Java implementation of the client? Feel free to share
some code with us.

Hope this helps,
Christian

[1] https://docs.basex.org/wiki/Statistics
[2] https://docs.basex.org/wiki/Databases

Re: [basex-talk] Help with usage of Regex while deleting resources

[Christian Grün]

Hi Deepak,

For deletions, you can write:

let $db := 'db'
for $path in db:list($db, '2023')[matches(., '/\d\d')]
return db:delete($db, $path)

When accessing documents, it’s faster to iterate over the resources:

for $doc in db:get('db', '2023')
where matches(db:path($doc), '/\d\d')
return ...

Hope this helps,
Christian



Deepak Dinakara  schrieb am Mo., 12. Feb. 2024,
18:04:

> Hi,
>
> I wanted to know if it's possible to give a regex while deleting a
> resource.
> I have documents stored in a hierarchy of collections like
> {year}{month}/doc.xml.
> Eg: 202301/abc.xml, 202302/def.xml.
> If I want to delete a resource "abc.xml", Is it possible to issue commands
> like "*db:delete("db-name", '/*/abc.xml')*" ? Right now, I can do a
> XQuery with db:list and endsWith and get the complete path of "abc.xml".
> But regex would have been very handy.
>
> Similarly I also want to execute queries against a list of collections
> using regex.
> Something like "*for $document in collection('db-name/20230*')*" (First 9
> months of 2023)
> Right now, I am doing something like
> "for $i in ('01', '02', '03', '04', ... '09')
> for $document in collection('test-collection/2023' || $i)"
> But if there are better ways, kindly let me know.
>
> Thank you,
> Deepak
>
>

Re: [basex-talk] basexhttp out of memory where basexgui suceeds

[Christian Grün]

…great to hear, thanks.

On Thu, Feb 8, 2024 at 7:30 AM Jack Steyn  wrote:

> Hi Christian,
>
> Sorry, I should have provided a self-contained example to begin with.
>
> In any case, I was running BaseX 10.0; after noticing that 10.6 boasts
> 'Much more memory-efficient representation of XML fragments', I upgraded to
> 10.7 and the problem appears to be resolved (and wow, there is a big
> difference in performance – kudos!).
>
> Many thanks,
>
> Jack
>
> On Wed, 7 Feb 2024, 6:09 pm Christian Grün, 
> wrote:
>
>> Hi Jack,
>>
>> If you run the query via basexhttp, how do you retrieve the results,
>> i.e., which client do you use?
>>
>> Can you possibly provide us with a self-contained example, something like…
>>
>> for $i in 1 to 50
>> return  update {
>>   insert node  into .
>> }
>>
>> …and some steps to reproduce the behavior?
>>
>> Thanks in advance,
>> Christian
>>
>>
>> On Wed, Feb 7, 2024 at 6:04 AM Jack Steyn  wrote:
>>
>>> Hi,
>>>
>>> I have a database about 200 MB in size made up of approximately 150 000
>>> documents of similar size and structure as children of the root node.
>>>
>>> When I run the following script in basexgui a significant amount of
>>> memory is consumed (over 1 GB if I'm reading the display correctly), but I
>>> do get a result:
>>>
>>> for $doc in db:get('docs')
>>> return $doc update {
>>> delete node .//*[local-name() = ('A', 'B', 'C', 'D')]
>>> }
>>>
>>> When I run it over basexhttp I get a java.lang.OutOfMemoryError: Java
>>> heap space. I have increased the memory available to the JVM to 4 GB but
>>> this has not affected the failure of the script.
>>>
>>> How can I resolve this? Is there some rewriting of the script that would
>>> help, or is it more specific to basexhttp?
>>>
>>> Many thanks,
>>>
>>> Jack
>>>
>>>

Re: [basex-talk] basexhttp out of memory where basexgui suceeds

[Christian Grün]

Hi Jack,

If you run the query via basexhttp, how do you retrieve the results, i.e.,
which client do you use?

Can you possibly provide us with a self-contained example, something like…

for $i in 1 to 50
return  update {
  insert node  into .
}

…and some steps to reproduce the behavior?

Thanks in advance,
Christian


On Wed, Feb 7, 2024 at 6:04 AM Jack Steyn  wrote:

> Hi,
>
> I have a database about 200 MB in size made up of approximately 150 000
> documents of similar size and structure as children of the root node.
>
> When I run the following script in basexgui a significant amount of memory
> is consumed (over 1 GB if I'm reading the display correctly), but I do get
> a result:
>
> for $doc in db:get('docs')
> return $doc update {
> delete node .//*[local-name() = ('A', 'B', 'C', 'D')]
> }
>
> When I run it over basexhttp I get a java.lang.OutOfMemoryError: Java heap
> space. I have increased the memory available to the JVM to 4 GB but this
> has not affected the failure of the script.
>
> How can I resolve this? Is there some rewriting of the script that would
> help, or is it more specific to basexhttp?
>
> Many thanks,
>
> Jack
>
>

Re: [basex-talk] Query regarding delete documents

[Christian Grün]

>
> I avoided "OPTIMIZE" since it caused OOM issues and I was fine without the
> index which looks like a double edged sword : ).
>

If you don’t need the indexes, you can disable them when running
db:optimize, and setting textindex and attrindex to false.

QQ, Is there some stat on how much RAM is needed for maintaining basex DB
> of size 'X'GB (with regular inserts and delete)  so that "optimize" could
> be called without worrying about OOM?
>

Hardly possible to say in general; it depends a lot on the “regular inserts
and deletes” ;) If you cannot solve the optimization problem, feel free to
share the OOM stack trace with us.

Re: [basex-talk] A question about (non standard) BaseX stopping

[Christian Grün]

Salut Pierre-Yves,

Sorry for not giving a helpful answer so far. Indeed it’s the sophisticated
setup that makes it difficult to give quick advice. Can you successfully
stop your server when you run a single basexhttp instance, or does this
already fail? Is it reproducible with 2 instances?

In general, explicit stops are not that important any more today: When the
system sends an ordinary stop/kill signal, the instance will be gracefully
shut down as well.

Hope this helps (a little), feel free to ask more questions,
Christian


On Tue, Jan 30, 2024 at 11:39 AM Pierre-Yves Jallud <
pierre-yves.jal...@ens-lyon.fr> wrote:

> Hi all,
>
> after having configuring a BaseX hub (|cluster|farm|...?), I have a last
> problem (I hope 爛). When I want to stop a BaseX instance with
> basexhttpstop (or basexhttp stop), I have different results, but none stop
> the instance.
>
> The context:
>
>- I'm using the following java version: *OpenJDK 64-Bit Server VM
>(Red_Hat-11.0.21.0.9-1.el7_9) (build 11.0.21+9-LTS, mixed mode, sharing)*
>- in the .bash_profile of the basexusr1 (and for each (system LINUX)
>BaseX user), I precise the BASEX_JVM variable (*export
>BASEX_JVM="-Dorg.basex.path=/basex-ferme/sites/basex1"*) to customize
>the webapp and data repository
>- */basex-ferme/sites/basex1/basex* is a symbolic link to the BaseX
>reference (every site has this symbolic link) located here:
>*/basex-ferme/basex* (hum... this is also a symbolic link pointing to
>the last version of basex: */basex-ferme/basex-10.7*)
>- the ports are different for each instance and are (only) precised
>when I launch the BaseX process
>- I launch each process with this command (adapting basex1, basexusr1
>and ports for each BaseX):
> *su - -s /bin/bash basexusr1 -c
>"/basex-ferme/sites/basex1/basex/bin/basexhttp -S -h11204 -p11205 -s11206" 
> *
>- for each command below, the code returned is ($?) 0... but the
>process is still running 樂 (as the web site). I only test the access of
>the DBA service (logging as admin) and I've got no database (at this
>time).
>
> The result of the stop commands:
>
>1. > su - -s /bin/bash basexusr1 -c
>"/basex-ferme/sites/basex1/basex/bin/basexhttpstop -h11204 -p11205 -s11206"
>BaseX 10.7 [HTTP Server]
>[main] INFO org.eclipse.jetty.util.log - Logging initialized @1523ms
>to org.eclipse.jetty.util.log.Slf4jLog
>HTTP Server was stopped (port: 11204).
>Server was stopped (port: 11205).
>
>2. > su - -s /bin/bash basexusr1 -c
>"/basex-ferme/sites/basex1/basex/bin/basexhttp stop -h11204 -p11205 
> -s11206"
>BaseX 10.7 [HTTP Server]
>[main] INFO org.eclipse.jetty.util.log - Logging initialized @1441ms
>to org.eclipse.jetty.util.log.Slf4jLog
>HTTP Server was stopped (port: 11204).
>Server was stopped (port: 11205).
>
>3. > su - -s /bin/bash basexusr1 -c
>"/basex-ferme/sites/basex1/basex/bin/basexhttp stop -s11206"
>BaseX 10.7 [HTTP Server]: 11205).
>[main] INFO org.eclipse.jetty.util.log - Logging initialized @1819ms
>to org.eclipse.jetty.util.log.Slf4jLog
>HTTP Server was stopped (port: 8080).
>Server was stopped (port: 1984).
>
>4. > su - -s /bin/bash basexusr1 -c
>"/basex-ferme/sites/basex1/basex/bin/basexhttp stop -h11204 -s11206"
>BaseX 10.7 [HTTP Server]
>[main] INFO org.eclipse.jetty.util.log - Logging initialized @1366ms
>to org.eclipse.jetty.util.log.Slf4jLog
>HTTP Server was stopped (port: 11204).
>Server was stopped (port: 1984).
>
>5. > su - -s /bin/bash basexusr1 -c 
> "/basex-ferme/sites/basex1/basex/bin/basexhttp
>stop -p11205 -s11206"
>BaseX 10.7 [HTTP Server]
>[main] INFO org.eclipse.jetty.util.log - Logging initialized @1006ms
>to org.eclipse.jetty.util.log.Slf4jLog
>HTTP Server was stopped (port: 8080).
>Server was stopped (port: 11205).
>
>
> ... and there is no track of the stop request in the log file (but there
> are the tracks of the admin access). The only solution I have found is to
> kill the process. That's my plan B, but I hope there is another cleaner
> solution 爛
>
> Has anyone a suggestion to resolve my problem?... and sorry for the
> complexity of the context 
>
> Greetings
> Pierre-Yves
> --
> *Pierre-Yves Jallud*
> *Pôle Humanité Numériques / Ingénieur en ingénierie logicielle*
>
> --
> IHRIM - UMR 5317 
> CNRS 
> ENS de Lyon 
> 15 Parvis René Descartes - BP7000 - 69342 Lyon CEDEX 07
> +33 (0)4 37 37 63 83 - pierre-yves.jal...@ens-lyon.fr
> --
>

Re: [basex-talk] Query regarding delete documents

[Christian Grün]

Hi Deepak,

Some more time would have to be spent to explain this in full detail.

In a nutshell, deleted space is not immediately reduced to a minimum size
because inserts are more regular. If you want to minimize your database
structure after extensive updates, you can call OPTIMIZE (ALL).

Hope this helps,
Christian


On Mon, Feb 5, 2024 at 1:09 PM Deepak Dinakara 
wrote:

> Hi,
> I wanted to understand more on how deleting documents impact the storage
> layout.
> I did go through https://docs.basex.org/wiki/Storage_Layout but didn't
> find all the required information.
> I observed that size of files like "txt.basex" and "atv.basex" doesn't
> shrink on deleting documents.
> Is that expected?
>
> PS: Thanks for the great product.
>
> Thank you,
> Deepak
>
>

Re: [basex-talk] hof:until qt4cg drops?

[Christian Grün]

I've sent you an e-mail in private.

The Clark notation for QNames with URIs was dropped. Instead, the Expanded
QName syntax (which is also used in XPath and XQuery) is the one that is
supported exclusively in the future:

-Q{URI}name




Andy Bunce  schrieb am Sa., 3. Feb. 2024, 16:11:

> Hi Christian,
>
> > Hope this helps
>
> Sounds like a good way forward. Sign me up, please.
>
> Digression:
> I made a tiny change to the wiki recently concerning the "Q" in expanded
> QNames [1] because I thought it was required.
> On reflection, this may be because I was using v9 at the time.
>
> I now think the "Q" is required in BaseX  v9, but optional in all V10+.
> Is this correct?
>
> /Andy
>
> [1]
> https://docs.basex.org/index.php?title=Command-Line_Options=146=16921=16694
>
>
>
>
> On Sat, 3 Feb 2024 at 11:15, Christian Grün 
> wrote:
>
>> Hi Andy,
>>
>> Thanks for the raised finger. Indeed it’s the specification of XQuery 4
>> that takes much more time than anticipated. The process is highly dynamic,
>> with confirmed features being revised again. Some weeks ago, we have
>> stopped updating the old documentation, and we are working on a new one
>> (completely based on XQuery, with the contents and history stored in
>> databases). It will replace the old one and will be version-based, i.e.,
>> we’ll freeze the documentation of BaseX 11 (12, etc.) once it’s released
>> and maintain the documentation for the upcoming version in parallel.
>>
>> As BaseX 11 won’t be finalized until end of this month as originally
>> planned, here’s a link to the new documentation, which we’re regularly
>> updating:
>>
>> https://help.basex.org/
>>
>> The contents will stay, but the rendering is still work in progress.
>> We’ve converted the contents to a mixture of XML and contemporary Markdown
>> syntax;.editing works similar as before. I invite you (and everyone else)
>> to ask for an account to be able to help us keeping the contents up-to-date
>> and bug free.
>>
>> We know ithe current state is a bit unfortunate. For those who are
>> looking for a reliable version of the BaseX 10 documentation, we point to
>> the archived PDF version:
>>
>> https://docs.basex.org/wiki/Documentation
>>
>> Hope this helps,
>> Christian
>>
>>
>> Andy Bunce  schrieb am Fr., 2. Feb. 2024, 19:25:
>>
>>>
>>> I wanted to look up the hof:until[1] syntax in the wiki. It tells me
>>> that it has been replaced with  fn:iterate-while. Well, I hope, when the
>>> dust settles, there is something similar to this in the spec, but it looks
>>> like it may not be iterate-while .
>>>
>>> Today a user of the current BaseX release(10.7) is going to struggle to
>>> find what hof:until does and wont find any fn:iterate-while . I think
>>> deprecating functions and moving forward is great. I do like to see what
>>> 4.0 might bring with the BaseX 11 betas but  if I catch myself writing
>>> anything substantial I stop. The ground is unstable.
>>>
>>> I think the wiki should center on the functionality of the current
>>> release, with just teasers about the future.
>>> Christian, I think you hinted that the future may hold some
>>> documentation API built into the product, or did I imagine that?
>>>
>>> /Andy
>>>
>>> [1] https://docs.basex.org/wiki/Higher-Order_Functions_Module,
>>>
>>

Re: [basex-talk] hof:until qt4cg drops?

[Christian Grün]

Hi Andy,

Thanks for the raised finger. Indeed it’s the specification of XQuery 4
that takes much more time than anticipated. The process is highly dynamic,
with confirmed features being revised again. Some weeks ago, we have
stopped updating the old documentation, and we are working on a new one
(completely based on XQuery, with the contents and history stored in
databases). It will replace the old one and will be version-based, i.e.,
we’ll freeze the documentation of BaseX 11 (12, etc.) once it’s released
and maintain the documentation for the upcoming version in parallel.

As BaseX 11 won’t be finalized until end of this month as originally
planned, here’s a link to the new documentation, which we’re regularly
updating:

https://help.basex.org/

The contents will stay, but the rendering is still work in progress. We’ve
converted the contents to a mixture of XML and contemporary Markdown
syntax;.editing works similar as before. I invite you (and everyone else)
to ask for an account to be able to help us keeping the contents up-to-date
and bug free.

We know ithe current state is a bit unfortunate. For those who are looking
for a reliable version of the BaseX 10 documentation, we point to the
archived PDF version:

https://docs.basex.org/wiki/Documentation

Hope this helps,
Christian


Andy Bunce  schrieb am Fr., 2. Feb. 2024, 19:25:

>
> I wanted to look up the hof:until[1] syntax in the wiki. It tells me that
> it has been replaced with  fn:iterate-while. Well, I hope, when the dust
> settles, there is something similar to this in the spec, but it looks like
> it may not be iterate-while .
>
> Today a user of the current BaseX release(10.7) is going to struggle to
> find what hof:until does and wont find any fn:iterate-while . I think
> deprecating functions and moving forward is great. I do like to see what
> 4.0 might bring with the BaseX 11 betas but  if I catch myself writing
> anything substantial I stop. The ground is unstable.
>
> I think the wiki should center on the functionality of the current
> release, with just teasers about the future.
> Christian, I think you hinted that the future may hold some documentation
> API built into the product, or did I imagine that?
>
> /Andy
>
> [1] https://docs.basex.org/wiki/Higher-Order_Functions_Module,
>

Re: [basex-talk] How to configure org.basex.path

[Christian Grün]

Salut Pierre-Yves,

The home directory can be specified via the Java system property
“org.basex.path”. As it’s required before BaseX is even started, it must be
passed on to the JDK, e.g. via “-Dorg.basex.path=/path/to/my/basex”. This
string can be…

• statically added to the BASEX_JVM property in your “basexhttp” start
script, or
• dynamically assigned to JDK_JAVA_OPTIONS on command-line before running
“basexhttp” (export JDK_JAVA_OPTIONS=...).

Hope this helps. If not, just keep on asking ;)

Merci et salutations,
Christian


On Wed, Jan 24, 2024 at 5:09 PM Pierre-Yves Jallud <
pierre-yves.jal...@ens-lyon.fr> wrote:

> Hi all,
>
> I would like to precise the home directory of BaseX using org.basex.path
> (cf. https://docs.basex.org/wiki/Configuration#Home_Directory). Is it
> possible to configure this when I launch the basexhttp? For exemple,
> something like (befor launching basexhttp):
>
> export org.basex.path=/path/to/my/basex
>
> Or maybe:
>
> basexhttp -S org.basex.path=/path/to/my/basex
>
> ... or what else?
>
> Many thanks in advance to enlighten me 
>
> Pierre-Yves
>
> PS: as you can see, I didn't use Java for a long time 
> --
> *Pierre-Yves Jallud*
> *Pôle Humanité Numériques / Ingénieur en ingénierie logicielle*
>
> --
> IHRIM - UMR 5317 
> CNRS 
> ENS de Lyon 
> 15 Parvis René Descartes - BP7000 - 69342 Lyon CEDEX 07
> +33 (0)4 37 37 63 83 - pierre-yves.jal...@ens-lyon.fr
> --
>

Re: [basex-talk] Consistent NPE loading content on one server, other server works

[Christian Grün]

…spending much time on a problem rarely helps. It’s often better to write
an e-mail, and discover the solution right after that.

A new stable snapshot with the bug fix is online [1,2].

Cheers,
Christian

[1] https://github.com/BaseXdb/basex/issues/2272
[2] https://files.basex.org/releases/latest/



On Wed, Jan 24, 2024 at 4:14 PM Christian Grün 
wrote:

> Hi Eliot,
>
> Thanks for your observations, and some first debugging.
>
> The idmap file is only created and required if UPDINDEX is enabled. Well,
> let’s say it should: Maybe this file is not created in due time.
>
> I haven’t managed to reproduce the bug yet. If you are more successful,
> please let us know.
>
> Cheers,
> Christian
>
>
> On Wed, Jan 24, 2024 at 12:22 AM Eliot Kimber 
> wrote:
>
>> I found the proximate cause: Not having UPDINDEX set to true() on the
>> failing server.
>>
>> When I set UPDINDEX to true on the target database, then the load
>> succeeded.
>>
>> Tracking through the Java code it looks like a failure to initialize the
>> idmap member of the DiskData object. This code looks like it could be the
>> issue:
>>
>> // open data and indexes
>> init();
>> if(meta.updindex) {
>>   idmap = new IdPreMap(meta.dbFile(DATAIDP));
>>   if(meta.textindex) textIndex = new UpdatableDiskValues(this,
>> IndexType.TEXT);
>>   if(meta.attrindex) attrIndex = new UpdatableDiskValues(this,
>> IndexType.ATTRIBUTE);
>>   if(meta.tokenindex) tokenIndex = new UpdatableDiskValues(this,
>> IndexType.TOKEN);
>> } else {
>>   if(meta.textindex) textIndex = new DiskValues(this, IndexType.TEXT);
>>   if(meta.attrindex) attrIndex = new DiskValues(this,
>> IndexType.ATTRIBUTE);
>>   if(meta.tokenindex) tokenIndex = new DiskValues(this, IndexType.TOKEN);
>> }
>> if(meta.ftindex) ftIndex = new FTIndex(this);
>>
>>
>>
>> The idmap is only set of meta.updIndex is true.
>>
>>
>>
>> Cheers,
>>
>>
>>
>> E.
>>
>> _
>>
>> *Eliot Kimber*
>>
>> Sr Staff Content Engineer
>>
>> O: 512 554 9368
>>
>> M: 512 554 9368
>>
>> servicenow.com <https://www.servicenow.com>
>>
>> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
>> <https://twitter.com/servicenow> | YouTube
>> <https://www.youtube.com/user/servicenowinc> | Facebook
>> <https://www.facebook.com/servicenow>
>>
>>
>>
>> *From: *Eliot Kimber 
>> *Date: *Tuesday, January 23, 2024 at 11:51 AM
>> *To: *basex-talk@mailman.uni-konstanz.de <
>> basex-talk@mailman.uni-konstanz.de>
>> *Subject: *Consistent NPE loading content on one server, other server
>> works
>>
>> I have two servers running the same code, both on 10.7. I have a REST API
>> handler that takes data as input and stores it in a database, applying some
>> preprocessing to the data first.
>>
>> The data successfully loads on my dev server and consistently fails on
>> the production server, with this Java failure:
>>
>> [INFO] Posting
>> "/Users/eliot.kimber/git-basex/product-content-analytics/analytics/adobe/adobe-analytics-Utah_Pages_Viewed_Aug2023-clean.csv"
>>
>> [INFO]   as filename "adobe-analytics-Utah_Pages_Viewed_Aug2023-clean.csv"
>>
>> [INFO]   to Mirabel server "http://mirabel.corp.service-now.com:9984;...
>>
>> Unexpected error: Improper use? Potential bug? Your feedback is welcome:
>>
>> Contact: basex-talk@mailman.uni-konstanz.de
>>
>> Version: BaseX 10.7
>>
>> Java: Oracle Corporation, 17.0.8
>>
>> OS: Linux, amd64
>>
>> Stack Trace:
>>
>> java.lang.NullPointerException: Cannot invoke
>> "org.basex.index.IdPreMap.write(org.basex.io.IOFile)" because "this.idmap"
>> is null
>>
>> at org.basex.data.DiskData.write(DiskData.java:151)
>>
>> at org.basex.data.DiskData.close(DiskData.java:160)
>>
>> at org.basex.core.Datas.unpin(Datas.java:52)
>>
>> at org.basex.core.cmd.Close.close(Close.java:45)
>>
>> at
>> org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:124)
>>
>> at
>> org.basex.query.up.primitives.db.DBOptimize.apply(DBOptimize.java:119)
>>
>> at
>> org.basex.query.up.DataUpdates.applyDbUpdates(DataUpdates.java:213)
>>
>> at
>> org.base

Re: [basex-talk] Consistent NPE loading content on one server, other server works

[Christian Grün]

Hi Eliot,

Thanks for your observations, and some first debugging.

The idmap file is only created and required if UPDINDEX is enabled. Well,
let’s say it should: Maybe this file is not created in due time.

I haven’t managed to reproduce the bug yet. If you are more successful,
please let us know.

Cheers,
Christian


On Wed, Jan 24, 2024 at 12:22 AM Eliot Kimber 
wrote:

> I found the proximate cause: Not having UPDINDEX set to true() on the
> failing server.
>
> When I set UPDINDEX to true on the target database, then the load
> succeeded.
>
> Tracking through the Java code it looks like a failure to initialize the
> idmap member of the DiskData object. This code looks like it could be the
> issue:
>
> // open data and indexes
> init();
> if(meta.updindex) {
>   idmap = new IdPreMap(meta.dbFile(DATAIDP));
>   if(meta.textindex) textIndex = new UpdatableDiskValues(this,
> IndexType.TEXT);
>   if(meta.attrindex) attrIndex = new UpdatableDiskValues(this,
> IndexType.ATTRIBUTE);
>   if(meta.tokenindex) tokenIndex = new UpdatableDiskValues(this,
> IndexType.TOKEN);
> } else {
>   if(meta.textindex) textIndex = new DiskValues(this, IndexType.TEXT);
>   if(meta.attrindex) attrIndex = new DiskValues(this,
> IndexType.ATTRIBUTE);
>   if(meta.tokenindex) tokenIndex = new DiskValues(this, IndexType.TOKEN);
> }
> if(meta.ftindex) ftIndex = new FTIndex(this);
>
>
>
> The idmap is only set of meta.updIndex is true.
>
>
>
> Cheers,
>
>
>
> E.
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com 
>
> LinkedIn  | Twitter
>  | YouTube
>  | Facebook
> 
>
>
>
> *From: *Eliot Kimber 
> *Date: *Tuesday, January 23, 2024 at 11:51 AM
> *To: *basex-talk@mailman.uni-konstanz.de <
> basex-talk@mailman.uni-konstanz.de>
> *Subject: *Consistent NPE loading content on one server, other server
> works
>
> I have two servers running the same code, both on 10.7. I have a REST API
> handler that takes data as input and stores it in a database, applying some
> preprocessing to the data first.
>
> The data successfully loads on my dev server and consistently fails on the
> production server, with this Java failure:
>
> [INFO] Posting
> "/Users/eliot.kimber/git-basex/product-content-analytics/analytics/adobe/adobe-analytics-Utah_Pages_Viewed_Aug2023-clean.csv"
>
> [INFO]   as filename "adobe-analytics-Utah_Pages_Viewed_Aug2023-clean.csv"
>
> [INFO]   to Mirabel server "http://mirabel.corp.service-now.com:9984;...
>
> Unexpected error: Improper use? Potential bug? Your feedback is welcome:
>
> Contact: basex-talk@mailman.uni-konstanz.de
>
> Version: BaseX 10.7
>
> Java: Oracle Corporation, 17.0.8
>
> OS: Linux, amd64
>
> Stack Trace:
>
> java.lang.NullPointerException: Cannot invoke
> "org.basex.index.IdPreMap.write(org.basex.io.IOFile)" because "this.idmap"
> is null
>
> at org.basex.data.DiskData.write(DiskData.java:151)
>
> at org.basex.data.DiskData.close(DiskData.java:160)
>
> at org.basex.core.Datas.unpin(Datas.java:52)
>
> at org.basex.core.cmd.Close.close(Close.java:45)
>
> at
> org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:124)
>
> at
> org.basex.query.up.primitives.db.DBOptimize.apply(DBOptimize.java:119)
>
> at
> org.basex.query.up.DataUpdates.applyDbUpdates(DataUpdates.java:213)
>
> at
> org.basex.query.up.DataUpdates.apply(DataUpdates.java:172)
>
> at
> org.basex.query.up.ContextModifier.apply(ContextModifier.java:120)
>
> at org.basex.query.up.Updates.apply(Updates.java:179)
>
> at
> org.basex.query.QueryContext.update(QueryContext.java:663)
>
> at
> org.basex.query.QueryContext.lambda$iter$4(QueryContext.java:357)
>
> at org.basex.query.QueryContext.run(QueryContext.java:766)
>
> at org.basex.query.QueryContext.iter(QueryContext.java:357)
>
> at
> org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:78)
>
> at
> org.basex.http.web.WebResponse.create(WebResponse.java:58)
>
> at
> org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:72)
>
> at
> org.basex.http.BaseXServlet.service(BaseXServlet.java:69)
>
> at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>
> at
> org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459)
>
> at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
>
> at
> org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
>
> at
> 

Re: [basex-talk] function prof:dump not working

[Christian Grün]

Hi Dharmendra,

I invite you to share a little self-contained example with us.

Can you reproduce it with the latest version of BaseX?

Thanks in advance,
Christian



DK Singh  schrieb am Mo., 22. Jan. 2024, 12:44:

> Hi,
> I am using BaseX 9.5 and we are using  the prof:dump() function for
> debugging code and data. When a query is run in the BaseX GUI, the output
> of prof:dump() is shown in the info view.
>
> When we are calling this function in XQUERY and try to print output of
> prof:dump() in the log then it is not printing anything.
>
> As per BaseX documentation, if LOGTRACE is 'ON' and BaseX is being used as
> a web-application, then the output of prof:dump() function gets logged in
> database logs. Checked on the server, LOGTRACE is 'ON' but the output is
> not getting logged in log file..
>
> Also we try it to store output prof:dump() in a variable and print it into
> log file but not working
>
> let $_Logs := prof:dump($work/@id || ' (' || $work/*:titleSet/*:title[1]
> || ')', 'Excluded from output, as no media found: ')
> return admin:write-log('output of function prof:dump is: '|| $_Logs)
>
> Can you please suggest how we can capture output of function prof:dump()
> in a XML format or in the BaseX log file?
>
> Thank You
> Dharmendra
>
>

Re: [basex-talk] Corrupt data base not recognized by inspect

[Christian Grün]

Dear Jürgen,

The safest/easiest option for now may be to consider a database corrupt if
the output does not contain the string “No inconsistencies found”. For more
details, feel free to check out the source code [1].

Hope this helps,
Christian

[1]
https://github.com/BaseXdb/basex/blob/main/basex-core/src/main/java/org/basex/core/cmd/Inspect.java


On Mon, Jan 15, 2024 at 2:44 PM Jürgen Schwenk Büro <
juergen.schw...@gruendler.de> wrote:

> Dear Christian,
>
> thanks a lot for your work. I tried it out and got the answer "Access to
> storage failed (java.lang.RuntimeException: java.io.EOFException)." That is
> fine.
>
> One more, perhaps last :-), question: To avoid a crash when opening
> database in our application I send the "inspect" command via C-Adapter
> basexdbc.c and check the resulting string. This is because the query
> command itself is OK, so I have to check the content. Up to now I check the
> resulting string if it contains either "Warning: Database is inconsistent."
> or "RuntimeException". I hope, these are the only possible failure
> responses. Or are there any other?
>
> Regards,
>
> Jürgen
> Am 15.01.2024 um 11:04 schrieb Christian Grün:
>
> Dear Jürgen,
>
> It turned out that the invocation of the INSPECT command raised an
> exception that was not caught by the command itself, but was only sent to
> stderr. We’ve revised this with the latest snapshot [1].
>
> Best (and good luck for your power connection in 2024)
> Christian
>
> [1] https://files.basex.org/releases/latest/
>
>
>
> On Thu, Jan 11, 2024 at 6:07 PM Jürgen Schwenk Büro <
> juergen.schw...@gruendler.de> wrote:
>
>> Hello,
>>
>> in our application we "produced" a corrupt data base caused by a power
>> fail. On restart we check data base using the "inspect" command to detect
>> failures, creating new empty data base if found one. The attached data base
>> seems OK using "inspect", but accessing the data base we get an exception.
>> Perhaps this may help to make "inspect" more reliable.
>>
>> Thanks a lot,
>>
>> Jürgen Schwenk
>>
>>
>> Client output:
>>
>> > list
>> Name  Resources  Size Input Path
>>
>> ---
>> VentilutionOnlineDataBaseCorrupt  1  1170
>> VentilutionOnlineDataBase.xml
>>
>> 1 database(s).
>>
>> > open VentilutionOnlineDataBaseCorrupt
>> Database 'VentilutionOnlineDataBaseCorrupt' was opened in 0.03 ms.
>>
>> > inspect
>> Checking main table (152 nodes):
>> - 0 invalid node kinds
>> - 0 invalid parent references
>> - 0 wrong parent/descendant relationships
>> No inconsistencies found.
>> 'VentilutionOnlineDataBaseCorrupt' inspected in 0.06 ms.
>>
>> > xquery //*
>> Improper use? Potential bug? Your feedback is welcome:
>> Contact: basex-talk@mailman.uni-konstanz.de
>> Version: BaseX 10.1
>> Java: Eclipse Adoptium, 17.0.4
>> OS: Windows 10, amd64
>> Stack Trace:
>> java.lang.RuntimeException: Data Access out of bounds:
>> - pre value: 152
>> - table size: 152
>> - first/next pre value: 0/152
>> - #total/used pages: 1/1
>> - accessed page: 0 (1 > 0]
>> at org.basex.util.Util.notExpected(Util.java:64)
>> at
>> org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:477)
>> at
>> org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:158)
>> at org.basex.data.Data.kind(Data.java:312)
>> at org.basex.query.value.node.DBNode$6.next(DBNode.java:402)
>> at org.basex.query.value.node.DBNode$6.next(DBNode.java:1)
>> at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38)
>> at org.basex.query.expr.path.IterStep$1.next(IterStep.java:1)
>> at org.basex.query.QueryContext.next(QueryContext.java:375)
>> at org.basex.query.expr.path.IterPath$1.next(IterPath.java:48)
>> at org.basex.query.scope.MainModule$1.next(MainModule.java:55)
>> at org.basex.core.cmd.AQuery.run(AQuery.java:88)
>> at org.basex.core.Command.run(Command.java:233)
>> at org.basex.core.Command.execute(Command.java:93)
>> at org.basex.server.ClientListener.run(ClientListener.java:141)
>>
>> Query "softwareProjectVentilution" executed in 0.31 ms.
>>
>>
>> --
>>
>>
>> --
>>
>> Jürgen Schwenk
>> - R Engineer -
>>
>>
>>
>>
>> Gründler GmbH Tel: +49 (0)7441 40761-14
>> Jaspisstrasse 23 Fax: +49 (0)7441 40761-99
>> 72250 Freudenstadt E-Mail: juergen.schw...@gruendler.de
>> --
>>
>>
>> --
>>
>> Jürgen Schwenk
>> - R Engineer -
>>
>>
>>
>>
>> Gründler GmbH Tel: +49 (0)7441 40761-14
>> Jaspisstrasse 23 Fax: +49 (0)7441 40761-99
>> 72250 Freudenstadt E-Mail: juergen.schw...@gruendler.de
>>
> --
>
>
> --
>
> Jürgen Schwenk
> - R Engineer -
>
>
>
>
> Gründler GmbH Tel: +49 (0)7441 40761-14
> Jaspisstrasse 23 Fax: +49 (0)7441 40761-99
> 72250 Freudenstadt E-Mail: juergen.schw...@gruendler.de
>

Re: [basex-talk] Processing stack traces in XQuery code

[Christian Grün]

Hi Omar,

we’ve re-added the stack trace to the latest 11.0 snapshot [1], and will
document this properly soon. I’ve slightly changed the behavior: Instead of
a single descriptive string, the stack trace is now returned as a string
sequence (the first entry points to the position in the code that failed,
the following entries point to the function calls).

I would like to see $err:stack-trace added to the standard. I’ll give you
an update if the QT4 group confirms that this is a good idea (might take a
while).

Best,
Christian

[1] https://files.basex.org/releases/latest/



On Thu, Jan 11, 2024 at 12:15 PM Omar Siam  wrote:

> Hi,
>
> Until 10.6 one could find the stack trace of an error in
> $err:additional. This feature is now gone from BaseX and the documentation.
>
> I made my life much easier with reporting stack traces on errors in
> RestXQ endpoints. Now I don't see how to get that information anymore.
>
> Any advice how to get this functionality back?
>
> Best regards
>
> --
> Mag. Ing. Omar Siam
> Austrian Center for Digital Humanities and Cultural Heritage
> Österreichische Akademie der Wissenschaften | Austrian Academy of Sciences
> Stellvertretende Behindertenvertrauensperson | Deputy representative for
> disabled persons
> Bäckerstraße 13, 1010 Wien, Österreich | Vienna, Austria
> T: +43 1 51581-7295
> omar.s...@oeaw.ac.at | www.oeaw.ac.at/acdh
>
>

Re: [basex-talk] Corrupt data base not recognized by inspect

[Christian Grün]

Dear Jürgen,

It turned out that the invocation of the INSPECT command raised an
exception that was not caught by the command itself, but was only sent to
stderr. We’ve revised this with the latest snapshot [1].

Best (and good luck for your power connection in 2024)
Christian

[1] https://files.basex.org/releases/latest/



On Thu, Jan 11, 2024 at 6:07 PM Jürgen Schwenk Büro <
juergen.schw...@gruendler.de> wrote:

> Hello,
>
> in our application we "produced" a corrupt data base caused by a power
> fail. On restart we check data base using the "inspect" command to detect
> failures, creating new empty data base if found one. The attached data base
> seems OK using "inspect", but accessing the data base we get an exception.
> Perhaps this may help to make "inspect" more reliable.
>
> Thanks a lot,
>
> Jürgen Schwenk
>
>
> Client output:
>
> > list
> Name  Resources  Size Input Path
>
> ---
> VentilutionOnlineDataBaseCorrupt  1  1170
> VentilutionOnlineDataBase.xml
>
> 1 database(s).
>
> > open VentilutionOnlineDataBaseCorrupt
> Database 'VentilutionOnlineDataBaseCorrupt' was opened in 0.03 ms.
>
> > inspect
> Checking main table (152 nodes):
> - 0 invalid node kinds
> - 0 invalid parent references
> - 0 wrong parent/descendant relationships
> No inconsistencies found.
> 'VentilutionOnlineDataBaseCorrupt' inspected in 0.06 ms.
>
> > xquery //*
> Improper use? Potential bug? Your feedback is welcome:
> Contact: basex-talk@mailman.uni-konstanz.de
> Version: BaseX 10.1
> Java: Eclipse Adoptium, 17.0.4
> OS: Windows 10, amd64
> Stack Trace:
> java.lang.RuntimeException: Data Access out of bounds:
> - pre value: 152
> - table size: 152
> - first/next pre value: 0/152
> - #total/used pages: 1/1
> - accessed page: 0 (1 > 0]
> at org.basex.util.Util.notExpected(Util.java:64)
> at
> org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:477)
> at
> org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:158)
> at org.basex.data.Data.kind(Data.java:312)
> at org.basex.query.value.node.DBNode$6.next(DBNode.java:402)
> at org.basex.query.value.node.DBNode$6.next(DBNode.java:1)
> at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38)
> at org.basex.query.expr.path.IterStep$1.next(IterStep.java:1)
> at org.basex.query.QueryContext.next(QueryContext.java:375)
> at org.basex.query.expr.path.IterPath$1.next(IterPath.java:48)
> at org.basex.query.scope.MainModule$1.next(MainModule.java:55)
> at org.basex.core.cmd.AQuery.run(AQuery.java:88)
> at org.basex.core.Command.run(Command.java:233)
> at org.basex.core.Command.execute(Command.java:93)
> at org.basex.server.ClientListener.run(ClientListener.java:141)
>
> Query "softwareProjectVentilution" executed in 0.31 ms.
>
>
> --
>
>
> --
>
> Jürgen Schwenk
> - R Engineer -
>
>
>
>
> Gründler GmbH Tel: +49 (0)7441 40761-14
> Jaspisstrasse 23 Fax: +49 (0)7441 40761-99
> 72250 Freudenstadt E-Mail: juergen.schw...@gruendler.de
> --
>
>
> --
>
> Jürgen Schwenk
> - R Engineer -
>
>
>
>
> Gründler GmbH Tel: +49 (0)7441 40761-14
> Jaspisstrasse 23 Fax: +49 (0)7441 40761-99
> 72250 Freudenstadt E-Mail: juergen.schw...@gruendler.de
>

Re: [basex-talk] Improper use error

[Christian Grün]

Hi Dharmendra,

We can only give limited free support for older versions of BaseX. Please
check if the behavior can be reproduced with the latest release.

Thanks in advance
Christian




DK Singh  schrieb am Fr., 12. Jan. 2024, 17:11:

> Hi i am executing the simple
> command  uri-collection('SYS_CONTENT_METADATA') and i am getting the below
> error:
>
> Error:
> Improper use? Potential bug? Your feedback is welcome:
> Contact: basex-talk@mailman.uni-konstanz.de
> Version: BaseX 9.5
> Java: Oracle Corporation, 1.8.0_151
> OS: Windows Server 2012 R2, amd64
> Stack Trace:
> java.lang.ArrayIndexOutOfBoundsException
>
> Can you please suggest how to resolve this issue?
>
> Thanks
> Dharmendra Kumar Singh
>
>
>

Re: [basex-talk] Corrupt data base not recognized by inspect

[Christian Grün]

Dear Jürgen,

Thanks for the observation.

In fact, the existing consistency checks won’t allow you to completely rule
out inconsistencies in databases. If your data is not strictly
confidential, we invite you to share the corrupt database instance with us,
it may help us to improve our checks. In addition, it could be interesting
to learn more about the update operation that was running during the power
fail (provided it was logged, e.g. in the data/.logs directory).

All the best,
Christian



On Thu, Jan 11, 2024 at 6:07 PM Jürgen Schwenk Büro <
juergen.schw...@gruendler.de> wrote:

> Hello,
>
> in our application we "produced" a corrupt data base caused by a power
> fail. On restart we check data base using the "inspect" command to detect
> failures, creating new empty data base if found one. The attached data base
> seems OK using "inspect", but accessing the data base we get an exception.
> Perhaps this may help to make "inspect" more reliable.
>
> Thanks a lot,
>
> Jürgen Schwenk
>
>
> Client output:
>
> > list
> Name  Resources  Size Input Path
>
> ---
> VentilutionOnlineDataBaseCorrupt  1  1170
> VentilutionOnlineDataBase.xml
>
> 1 database(s).
>
> > open VentilutionOnlineDataBaseCorrupt
> Database 'VentilutionOnlineDataBaseCorrupt' was opened in 0.03 ms.
>
> > inspect
> Checking main table (152 nodes):
> - 0 invalid node kinds
> - 0 invalid parent references
> - 0 wrong parent/descendant relationships
> No inconsistencies found.
> 'VentilutionOnlineDataBaseCorrupt' inspected in 0.06 ms.
>
> > xquery //*
> Improper use? Potential bug? Your feedback is welcome:
> Contact: basex-talk@mailman.uni-konstanz.de
> Version: BaseX 10.1
> Java: Eclipse Adoptium, 17.0.4
> OS: Windows 10, amd64
> Stack Trace:
> java.lang.RuntimeException: Data Access out of bounds:
> - pre value: 152
> - table size: 152
> - first/next pre value: 0/152
> - #total/used pages: 1/1
> - accessed page: 0 (1 > 0]
> at org.basex.util.Util.notExpected(Util.java:64)
> at
> org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:477)
> at
> org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:158)
> at org.basex.data.Data.kind(Data.java:312)
> at org.basex.query.value.node.DBNode$6.next(DBNode.java:402)
> at org.basex.query.value.node.DBNode$6.next(DBNode.java:1)
> at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38)
> at org.basex.query.expr.path.IterStep$1.next(IterStep.java:1)
> at org.basex.query.QueryContext.next(QueryContext.java:375)
> at org.basex.query.expr.path.IterPath$1.next(IterPath.java:48)
> at org.basex.query.scope.MainModule$1.next(MainModule.java:55)
> at org.basex.core.cmd.AQuery.run(AQuery.java:88)
> at org.basex.core.Command.run(Command.java:233)
> at org.basex.core.Command.execute(Command.java:93)
> at org.basex.server.ClientListener.run(ClientListener.java:141)
>
> Query "softwareProjectVentilution" executed in 0.31 ms.
>
>
> --
>
>
> --
>
> Jürgen Schwenk
> - R Engineer -
>
>
>
>
> Gründler GmbH Tel: +49 (0)7441 40761-14
> Jaspisstrasse 23 Fax: +49 (0)7441 40761-99
> 72250 Freudenstadt E-Mail: juergen.schw...@gruendler.de
> --
>
>
> --
>
> Jürgen Schwenk
> - R Engineer -
>
>
>
>
> Gründler GmbH Tel: +49 (0)7441 40761-14
> Jaspisstrasse 23 Fax: +49 (0)7441 40761-99
> 72250 Freudenstadt E-Mail: juergen.schw...@gruendler.de
>

Re: [basex-talk] Processing stack traces in XQuery code

[Christian Grün]

Hi Omar,

Thanks for your observation.

We have removed the BaseX-specific string output, as it was redundant. You
should be able to reproduce the old output as follows (using the new XQuery
4 string template syntax):

`Stopped at { $err:module }, { $err:line-number }/{ $err:column-number }:
{ $err:description }`

As you’ve already discovered the responsible commit, you may have
discovered the new XQuery 4 $err:map variable: It bundles all other error
values into a single variable.

And @Martin Honnen thanks for mentioning fn:stack-trace. The function is
still under discussion and might be dropped again: The specification gives
processors much freedom in optimizing its expressions, so it’s hardly
possible to define the function in a way that’s not completely
implementation-defined.

Hope this helps,
Christian


On Thu, Jan 11, 2024 at 12:52 PM Omar Siam  wrote:

> I think stack traces were removed in this commit from
> basex-core/src/main/java/org/basex/query/expr/Catch.java
>
> Revision: fc2376f6c9c250babfac7a7161b8a61740878b2b
> Author: Christian Grün  
> Date: 17.05.2023 11:25:01
> Message:
> [XQ4] try/catch, $err:map
>
> 
> Modified: basex-api/src/main/java/org/basex/http/restxq/RestXqFunction.java
> Modified: basex-core/src/main/java/org/basex/query/QueryParser.java
> Modified: basex-core/src/main/java/org/basex/query/expr/Catch.java
> Modified: basex-core/src/test/java/org/basex/query/expr/XQuery4Test.java
>
> line 141 ->  151
> qe.getMessage() is now replaced with an Empty.VALUE const.
>
> Am 11.01.2024 um 12:39 schrieb Martin Honnen:
>
>
>
> Am 11.01.2024 um 12:37 schrieb Omar Siam 
> :
>
> I get: Unknown function: fn:stack-trace() when I use stack-trace().
>
>
> Ah, sorry, it looks like it is not yet implemented. Wait for Christian to
> give you hopefully some better info than my ill guess.
>
>
> --
> Mag. Ing. Omar Siam
> Austrian Center for Digital Humanities and Cultural Heritage
> Österreichische Akademie der Wissenschaften | Austrian Academy of Sciences
> Stellvertretende Behindertenvertrauensperson | Deputy representative for 
> disabled persons
> Bäckerstraße 13, 1010 Wien, Österreich | Vienna, Austria
> T: +43 1 51581-7295omar.s...@oeaw.ac.at | www.oeaw.ac.at/acdh
>
>

Re: [basex-talk] basexgui is crashing for me ...

[Christian Grün]

Maybe you could try an older version of the JDK or JRE?



Jonathan Robie  schrieb am Mi., 10. Jan. 2024,
19:41:

> I should have included this:
>
>  % java --version
> openjdk 21.0.1 2023-10-17 LTS
> OpenJDK Runtime Environment Temurin-21.0.1+12 (build 21.0.1+12-LTS)
> OpenJDK 64-Bit Server VM Temurin-21.0.1+12 (build 21.0.1+12-LTS, mixed
> mode)
>
> % basexgui
> 2024-01-10 13:40:21.624 java[56086:5869820] WARNING: Secure coding is
> automatically enabled for restorable state! However, not on all supported
> macOS versions of this application. Opt-in to secure coding explicitly by
> implementing
> NSApplicationDelegate.applicationSupportsSecureRestorableState:.
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGILL (0x4) at pc=0x7ff80e07cb73, pid=56086, tid=259
> #
> # JRE version: OpenJDK Runtime Environment Homebrew (21.0.1) (build 21.0.1)
> # Java VM: OpenJDK 64-Bit Server VM Homebrew (21.0.1, mixed mode, sharing,
> tiered, compressed oops, compressed class ptrs, g1 gc, bsd-amd64)
> # Problematic frame:
> # C  [AppKit+0xc8fb73]  _NSCarbonMenuCrashIfNeeded+0x258
> #
> # Core dump will be written. Default location: /cores/core.56086
> #
> # An error report file with more information is saved as:
> # /Users/jonathan/github/Clear/hs_err_pid56086.log
> #
> # If you would like to submit a bug report, please visit:
> #   https://github.com/Homebrew/homebrew-core/issues
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
> zsh: abort  basexgui
>
> On Wed, Jan 10, 2024 at 1:30 PM Jonathan Robie 
> wrote:
>
>> Sigh - I just did that, it still crashes ...
>>
>> Jonathan
>>
>> On Wed, Jan 10, 2024 at 1:22 PM Jonathan Robie 
>> wrote:
>>
>>> Thanks for the prompt, helpful response!
>>>
>>> Jonathan
>>>
>>> On Wed, Jan 10, 2024 at 5:54 AM Christian Grün <
>>> christian.gr...@gmail.com> wrote:
>>>
>>>> Hi Jonathan,
>>>>
>>>> This problem is related to Homebrew and Apple [1], and does not seem to
>>>> be solved yet.
>>>>
>>>> There are alternatives, though. One solution that has been suggested by
>>>> my colleague Michael is to:
>>>>
>>>> 1. Uninstall the homebrew openjdk:
>>>>brew uninstall openjdk --ignore-dependencies
>>>> 2. Manual install Adoptium Open JDK (11 or later):
>>>> https://adoptium.net:
>>>>
>>>> See [2] for further information.
>>>>
>>>> Hope this helps,
>>>> Christian
>>>>
>>>> [1] https://github.com/Homebrew/homebrew-core/issues/150824
>>>> [2]
>>>> https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg15457.html
>>>>
>>>>
>>>>
>>>> On Tue, Jan 9, 2024 at 9:17 PM Jonathan Robie 
>>>> wrote:
>>>>
>>>>> Basexgui has been crashing for me.
>>>>>
>>>>> I thought my environment might be messed up, so I uninstalled brew,
>>>>> reinstalled it, and checked my environment.
>>>>>
>>>>> Then I ran basexgui, and got another crash.
>>>>>
>>>>> % basexgui
>>>>>> 2024-01-09 15:09:58.791 java[18502:5023580] WARNING: Secure coding is
>>>>>> automatically enabled for restorable state! However, not on all supported
>>>>>> macOS versions of this application. Opt-in to secure coding explicitly by
>>>>>> implementing
>>>>>> NSApplicationDelegate.applicationSupportsSecureRestorableState:.
>>>>>> #
>>>>>> # A fatal error has been detected by the Java Runtime Environment:
>>>>>> #
>>>>>> #  SIGILL (0x4) at pc=0x7ff80e07cb73, pid=18502, tid=259
>>>>>> #
>>>>>> # JRE version: OpenJDK Runtime Environment Homebrew (21.0.1) (build
>>>>>> 21.0.1)
>>>>>> # Java VM: OpenJDK 64-Bit Server VM Homebrew (21.0.1, mixed mode,
>>>>>> sharing, tiered, compressed oops, compressed class ptrs, g1 gc, 
>>>>>> bsd-amd64)
>>>>>> # Problematic frame:
>>>>>> # C  [AppKit+0xc8fb73]  _NSCarbonMenuCrashIfNeeded+0x258
>>>>>> #
>>>>>> # Core dump will be written. Default location: /cores/core.18502
>>>>>> #
>>>>>> # An error report file with more information is saved as:
>>>>>> # /Users/jonathan/github/Clear/hs_err_pid18502.log
>>>>>> #
>>>>>> # If you would like to submit a bug report, please visit:
>>>>>> #   https://github.com/Homebrew/homebrew-core/issues
>>>>>> # The crash happened outside the Java Virtual Machine in native code.
>>>>>> # See problematic frame for where to report the bug.
>>>>>> #
>>>>>> zsh: abort  basexgui
>>>>>>
>>>>>
>>>>> What should I do?
>>>>>
>>>>> Jonathan
>>>>>
>>>>

Re: [basex-talk] How to invalidate XSD cache without stopping and restarting Jetty Web server?

[Christian Grün]

;) Right, thanks. An updated version is online.


On Wed, Jan 10, 2024 at 4:23 PM  wrote:

> Mr. Grun,
>
> It is probably better to fix the actual source code:
> _VALIDATE_XSD_INIT(ValidateXsdInit::new, "init()",
>
> As follows:
> _VALIDATE_XSD_INIT(ValidateXsdInit::new, "xsd-init()",
>
> Regards,
> Yitzhak Khabinsky
>
> -Original Message-
> From: ykhab...@bellsouth.net 
> Sent: Wednesday, January 10, 2024 10:16 AM
> To: 'Christian Grün' 
> Cc: 'BaseX' 
> Subject: RE: [basex-talk] How to invalidate XSD cache without stopping and
> restarting Jetty Web server?
>
> Hi Mr. Grun,
>
> Thanks for such super quick turnaround!
>
> It seems that the first link [1]
> https://github.com/BaseXdb/basex/issues/2267
> shows incorrect function signature as validate:xsd-init().
> Shouldn't it  be validate:init()?
>
> Regards,
> Yitzhak Khabinsky
>
> From: Christian Grün 
> Sent: Wednesday, January 10, 2024 6:19 AM
> To: ykhab...@bellsouth.net
> Cc: BaseX 
> Subject: Re: [basex-talk] How to invalidate XSD cache without stopping and
> restarting Jetty Web server?
>
> Hi Yitzhak,
>
> Thanks for the suggestion. Feel free to check out the latest stable
> snapshot, it contains two new functions for discarding cached XSD schemas
> and XSL transformers [1,2].
>
> Best,
> Christian
>
> [1] https://github.com/BaseXdb/basex/issues/2267
> [2] https://files.basex.org/releases/latest/
>
>
> On Tue, Jan 9, 2024 at 11:00 PM <mailto:ykhab...@bellsouth.net> wrote:
> Hello,
>
> We are using BaseX 10.8 beta for XML files validation against the XSD.
> Our XSDs are conformant with the XSD 1.1 that why all necessary Apache
> Xerces 12.2.2 files are in the lib/custom directory.
> On top of that we are using Jetty Web server by launching c:\Program Files
> (x86)\BaseX\bin\basexhttp.bat
>
> Eventually, our REST HTTP call is as follow:
> string REQUESTURL =
> $"http://{HOST}:{PORT}/rest?run={XQUERY}&$xml={xmlFile}&$xsd={xsdFile};;
>
> Our XQuery validation call is as follows:
> let $result := validate:xsd-report($xml, $xsd, map {
> 'http://apache.org/xml/features/validation/cta-full-xpath-checking':
> true(),
> 'cache': true() })
>
> Everything is working as expected. :)
>
> Here is a question.
> ===
> The validate:xsd-report(...) call is using caching via the 'cache': true()
> parameter.
> We are updating XSDs from time to time.
> Unfortunately, the validation is not picking up a new version of the XSD
> file on the file system.
> It is unaware that the *.xsd file was updated.
> It is still using an old XSD version that is cached in memory.
>
> Is it possible to invalidate the XSD cache without stopping and restarting
> Jetty Web server?
> IMHO, it is too intrusive.
> Maybe you can introduce a manual command for XSD cache invalidation in the
> BaseX GUI?
> Or some other mechanism to handle the scenario.
>
> Regards,
> Yitzhak Khabinsky
>
>
>
>

Re: [basex-talk] XSLT version

[Christian Grün]

Hi Martin,

BaseX 8.6.7 is pretty old now. Could you please if the behavior can be
reproduced with the latest version of BaseX?

Thanks in advance,
Christian


On Wed, Jan 10, 2024 at 6:09 PM Martin Dowling <
martin.dowl...@bloomsbury.com> wrote:

> Hi
>
>
>
> I’m trying to run a query in BaseX (version 8.6.7) which calls an XSLT 2.0
> stylesheet using xslt:transform(). I’m getting errors that clearly imply
> the system does not recognise XSLT 2.0 functions, and is expecting an XSLT
> 1.0 stylesheet.
>
>
>
> However, when I run xslt:version(), it returns “2.0” – which implies the
> system is set up for XSLT 2.0 stylesheets.
>
>
>
> Is there some other setting somewhere that might need to be updated, to
> enable the system to run an XSLT 2.0 transform?
>
>
>
> Thank you,
>
>
>
> Martin
>
> --
>
> This email and its contents are confidential and may also be privileged
> and protected by copyright. If you are not the named recipient please
> notify the sender immediately, delete the email and all attachments
> completely from your system and do not use, print, copy, disclose or
> distribute any part of its contents. Bloomsbury Publishing Plc and its
> subsidiaries may monitor email traffic. Any views expressed in this email
> are those of the author, and do not necessarily represent those of
> Bloomsbury. This email does not conclude a binding agreement. Bloomsbury
> Publishing Plc 50 Bedford Square, London WC1B 3DP. Company registered in
> England and Wales company no. 1984336. Bloomsbury Publishing Inc. 1385
> Broadway, Fifth floor, New York, 10018. Bloomsbury Publishing Pty Ltd,
> Level 4, 387 George St, Sydney, NSW 2000. Bloomsbury Publishing India Pvt.
> Ltd. DDA Complex, LSC , Building No.4, Second Floor, Pocket C-6&7, Vasant
> Kunj, New Delhi 110070. ABC-CLIO | Bloomsbury, ABC-CLIO is an imprint of
> Bloomsbury, 147 Castilian Drive, Santa Barbara, CA 93117.
>

Re: [basex-talk] How to invalidate XSD cache without stopping and restarting Jetty Web server?

[Christian Grün]

Hi Yitzhak,

Thanks for the suggestion. Feel free to check out the latest stable
snapshot, it contains two new functions for discarding cached XSD schemas
and XSL transformers [1,2].

Best,
Christian

[1] https://github.com/BaseXdb/basex/issues/2267
[2] https://files.basex.org/releases/latest/


On Tue, Jan 9, 2024 at 11:00 PM  wrote:

> Hello,
>
> We are using BaseX 10.8 beta for XML files validation against the XSD.
> Our XSDs are conformant with the XSD 1.1 that why all necessary Apache
> Xerces 12.2.2 files are in the lib/custom directory.
> On top of that we are using Jetty Web server by launching c:\Program Files
> (x86)\BaseX\bin\basexhttp.bat
>
> Eventually, our REST HTTP call is as follow:
> string REQUESTURL =
> $"http://{HOST}:{PORT}/rest?run={XQUERY}&$xml={xmlFile}&$xsd={xsdFile};;
>
> Our XQuery validation call is as follows:
> let $result := validate:xsd-report($xml, $xsd, map {
> 'http://apache.org/xml/features/validation/cta-full-xpath-checking':
> true(),
> 'cache': true() })
>
> Everything is working as expected. :)
>
> Here is a question.
> ===
> The validate:xsd-report(...) call is using caching via the 'cache': true()
> parameter.
> We are updating XSDs from time to time.
> Unfortunately, the validation is not picking up a new version of the XSD
> file on the file system.
> It is unaware that the *.xsd file was updated.
> It is still using an old XSD version that is cached in memory.
>
> Is it possible to invalidate the XSD cache without stopping and restarting
> Jetty Web server?
> IMHO, it is too intrusive.
> Maybe you can introduce a manual command for XSD cache invalidation in the
> BaseX GUI?
> Or some other mechanism to handle the scenario.
>
> Regards,
> Yitzhak Khabinsky
>
>
>
>

Re: [basex-talk] basexgui is crashing for me ...

[Christian Grün]

Hi Jonathan,

This problem is related to Homebrew and Apple [1], and does not seem to be
solved yet.

There are alternatives, though. One solution that has been suggested by my
colleague Michael is to:

1. Uninstall the homebrew openjdk:
   brew uninstall openjdk --ignore-dependencies
2. Manual install Adoptium Open JDK (11 or later):
https://adoptium.net:

See [2] for further information.

Hope this helps,
Christian

[1] https://github.com/Homebrew/homebrew-core/issues/150824
[2]
https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg15457.html



On Tue, Jan 9, 2024 at 9:17 PM Jonathan Robie 
wrote:

> Basexgui has been crashing for me.
>
> I thought my environment might be messed up, so I uninstalled brew,
> reinstalled it, and checked my environment.
>
> Then I ran basexgui, and got another crash.
>
> % basexgui
>> 2024-01-09 15:09:58.791 java[18502:5023580] WARNING: Secure coding is
>> automatically enabled for restorable state! However, not on all supported
>> macOS versions of this application. Opt-in to secure coding explicitly by
>> implementing
>> NSApplicationDelegate.applicationSupportsSecureRestorableState:.
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  SIGILL (0x4) at pc=0x7ff80e07cb73, pid=18502, tid=259
>> #
>> # JRE version: OpenJDK Runtime Environment Homebrew (21.0.1) (build
>> 21.0.1)
>> # Java VM: OpenJDK 64-Bit Server VM Homebrew (21.0.1, mixed mode,
>> sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-amd64)
>> # Problematic frame:
>> # C  [AppKit+0xc8fb73]  _NSCarbonMenuCrashIfNeeded+0x258
>> #
>> # Core dump will be written. Default location: /cores/core.18502
>> #
>> # An error report file with more information is saved as:
>> # /Users/jonathan/github/Clear/hs_err_pid18502.log
>> #
>> # If you would like to submit a bug report, please visit:
>> #   https://github.com/Homebrew/homebrew-core/issues
>> # The crash happened outside the Java Virtual Machine in native code.
>> # See problematic frame for where to report the bug.
>> #
>> zsh: abort  basexgui
>>
>
> What should I do?
>
> Jonathan
>

Re: [basex-talk] Creating backups manually

[Christian Grün]

Hi Matt,

Your assumption is correct: Database backups are nothing else than zipped
archives of the corresponding database subdirectory located in the 'data'
directory.

If you create backups on your own, you should ensure that no updates are
currently running during your update operation. If you use CREATE BACKUP or
db:create-backup, BaseX will take care of that.

Hope this helps,
Christian



I'm interested in creating backups manually so I can use a different
> compression algorithm. Based on the source code[1], it looks like backups
> are just created by adding each file (excluding upd.basex) in the
> database directory to a .zip file, so I could do the same using tar and
> my compression algorithm of choice. Is my understanding correct or am I
> missing some other logic?
>
> Thanks,
> Matt
>
> [1]
> https://github.com/BaseXdb/basex/blob/main/basex-core/src/main/java/org/basex/core/cmd/CreateBackup.java#L90-L119
>

Re: [basex-talk] DBs corrupted; Unparseable date.

[Christian Grün]

Hi France,

I’m sorry to hear that.

Times are busy, especially with the end of the year approaching. As the
number of our clients has increased, and as we want to do justice to
everyone, our consulting is now based on SLAs that give you guaranteed
response times. Are you interested in a quote?

All the best,
Christian



On Tue, Dec 12, 2023 at 11:28 AM France Baril 
wrote:

> I just noticed that I didn't include the full trace as I was trying to not
> list all our DBs. Here it is:
>
> ... more unparseable DB names
> bx_1  | Unparseable date: "zh-tw"
> bx_1  | Unparseable date: "zu-za"
> bx_1  | [qtp289639718-34] INFO com.bradmcevoy.http.HttpManager
> - PROPFIND :: http://localhost:8972/webdav/2023-06-14T07:37:56.294Z/ -
> http://localhost:8972/webdav/2023-06-14T07:37:56.294Z/
> bx_1  | org.basex.core.BaseXException: Stopped at /srv/basex/,
> 1/41:
> bx_1  | [db:name] Invalid name: 2023-06-14T07:37:56.294Z.
> bx_1  | at org.basex.core.Command.execute(Command.java:94)
> bx_1  | at
> org.basex.api.client.LocalSession.execute(LocalSession.java:131)
> bx_1  | at
> org.basex.api.client.Session.execute(Session.java:36)
> bx_1  | at
> org.basex.http.webdav.WebDAVQuery.execute(WebDAVQuery.java:70)
> bx_1  | at
> org.basex.http.webdav.WebDAVService.dbExists(WebDAVService.java:84)
> bx_1  | at
> org.basex.http.webdav.WebDAVFactory.getResource(WebDAVFactory.java:51)
> bx_1  | at
> com.bradmcevoy.http.ResourceHandlerHelper.process(ResourceHandlerHelper.java:86)
> bx_1  | at
> com.bradmcevoy.http.webdav.PropFindHandler.process(PropFindHandler.java:97)
> bx_1  | at
> com.bradmcevoy.http.StandardFilter.process(StandardFilter.java:52)
> bx_1  | at
> com.bradmcevoy.http.FilterChain.process(FilterChain.java:40)
> bx_1  | at
> com.bradmcevoy.http.HttpManager.process(HttpManager.java:228)
> bx_1  | at
> org.basex.http.webdav.WebDAVServlet.run(WebDAVServlet.java:36)
> bx_1  | at
> org.basex.http.BaseXServlet.service(BaseXServlet.java:69)
> bx_1  | at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> bx_1  | at
> org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459)
> bx_1  | at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
> bx_1  | at
> org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
> bx_1  | at
> org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:319)
> bx_1  | at
> org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:273)
> bx_1  | at
> org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201)
> bx_1  | at
> org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
> bx_1  | at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
> bx_1  | at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> bx_1  | at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
> bx_1  | at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> bx_1  | at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> bx_1  | at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
> bx_1  | at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
> bx_1  | at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
> bx_1  | at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
> bx_1  | at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
> bx_1  | at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
> bx_1  | at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
> bx_1  | at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
> bx_1  | at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> bx_1  | at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> bx_1  | at
> org.eclipse.jetty.server.Server.handle(Server.java:516)
> bx_1  | at
> org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
> bx_1  | at
> org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
> bx_1  | at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
> bx_1  | 

Re: [basex-talk] Code highlighting not working on Wiki

[Christian Grün]

Hi Johan,

we've disabled the highlighting in the Wiki. We're working on a new
documentation based on RESTXQ, which will hopefully deliver a better user
experience.

Best,
Christian



Christian Grün  schrieb am Do., 30. Nov. 2023,
13:35:

> Thanks, Johan, for the reminder. We’re working on it and will hopefully be
> able to fix it soon. All the best, Christian
>
>
>
> Johan Mörén  schrieb am Do., 30. Nov. 2023, 12:37:
>
>> Hi,
>>
>> Just discovered that the code samples on the basex wiki doesn't seem to
>> be working fully. Noticed it a couple of days ago and thought it was
>> temporary, but the problem is still there.
>>
>> Regards,
>> Johan
>>
>

Re: [basex-talk] Generate CSV with literal "&", not ""

[Christian Grün]

>
> Hi Eliot,

> let $doWrite := file:write('/Users/eliot.kimber/temp/apps-to-topics.csv',
> $report)
>

file:write-text should do the job. An alternative is to use the
serialization method 'text':

file:write($path, $value, map { 'method': 'text })

Hope this helps,
Christian

Re: [basex-talk] Code highlighting not working on Wiki

[Christian Grün]

Thanks, Johan, for the reminder. We’re working on it and will hopefully be
able to fix it soon. All the best, Christian



Johan Mörén  schrieb am Do., 30. Nov. 2023, 12:37:

> Hi,
>
> Just discovered that the code samples on the basex wiki doesn't seem to be
> working fully. Noticed it a couple of days ago and thought it was
> temporary, but the problem is still there.
>
> Regards,
> Johan
>

Re: [basex-talk] BaseX-Talk Digest, Vol 167, Issue 9

[Christian Grün]

>
> Hi Rob,

> Just a thought: shouldn’t util:if not just be part of the regular BaseX
> syntax? Together with fn:for-each, the fn:fold functions and
> hof:fold-left1, they represent the group of procesflow-controll-functions.
>
>From a syntactical view, util:if is just a function (which simulates the
behavior of the standard “if expression”).

Does this help?
Christian

<#m_929262194324372041_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>

Re: [basex-talk] BaseX 10.7 and util:if

[Christian Grün]

Hi Andy, hi Rob,

util:if is back again.

Just give me a note if there are other functions in the Utility Module (or
in the HOF Module) that you rely on.

Best,
Christian

On Mon, Nov 20, 2023 at 1:15 PM Andy Bunce  wrote:

> Hi,
> The documentation for the utility module [1] says certain functions will
> be removed in version 11 because they will be in XPath 4.0.
> However, util:if has been removed from the documentation and I think there
> is no equivalent function in this case.
> util:if is an alternative  syntax for the BaseX Ternary operator  [2].
>
> ( I have used it because it was easier than trying to get my XQuery parser
> EBNF to accept  $ok ?? 1 !! 0 etc)
>
> Is util:if to be removed?
>
> /Andy
> [1] https://docs.basex.org/wiki/Utility_Module#util:if
> [2] https://docs.basex.org/wiki/XQuery_Extensions#Ternary_If
>

Re: [basex-talk] Issue in BaseX 11.0 beta

[Christian Grün]

Thanks, Tim,

That was helpful: You came across a hidden bug that was introduced three
years ago (and which was probably triggered by some newer optimizations).
The bug has been fixed [1,2].

Best,
Christian

[1] https://github.com/BaseXdb/basex/issues/2261
[2] https://files.basex.org/releases/latest



On Wed, Nov 22, 2023 at 4:10 PM Thompson, Timothy 
wrote:

> Hello,
>
>
>
> In BaseX 11.0 beta, I’m encountering a strange error. I’ve created a gist
> that illustrates the behavior:
> https://gist.github.com/timathom/7fd44b43de1fa72162cd59cf755c3d91.
>
>
>
> I have a series of maps that I want to check against a lookup index and
> then filter the results. I’m setting a Boolean variable that only seems to
> get applied if I reference it within the *then* statement. I’ve tested
> the query in an earlier BaseX version (10.6), and it produces the expected
> result (without referencing the Boolean variable again).
>
>
>
> Thanks in advance,
>
> Tim
>
>
>
>
>
> --
> Tim A. Thompson (*he, him*)
> Librarian for Applied Metadata Research
> Yale University Library
> www.linkedin.com/in/timathompson
>
> timat...@protonmail.com 
>
>
>

Re: [basex-talk] BaseX 10.7 and util:if (Christian Gr?n)

[Christian Grün]

…thanks, Rob, thanks Andy, for the inquiry. I think we can re-add the
function, it shouldn’t cause problems.

I learned over time that the best (and sometimes only) way to find out
which features are used is to simply remove them…


On Wed, Nov 22, 2023 at 1:07 PM Rob Stapper  wrote:

> Christian,
>
>
>
> I really hope you will change your mind about this. In BaseX, as a
> function development environment, all structural constructs such as: “if
> then else” should be available as functions. Personally: as a
> functiondevelopmentjunky, I took a lot of effort replacing all
> ifThenElse-constructs into util:if-functions. I really would be
> disappointed if structural constructs aren’t available as functions in
> BaseX.
>
>
>
> Hope you’ll change your mind about this point.
>
>
>
> Regards,
>
> Rob Stapper
>
>
>
> Message: 4
>
> Date: Tue, 21 Nov 2023 16:45:31 +0100
>
> From: Christian Gr?n 
>
> To: Andy Bunce 
>
> Cc: BaseX 
>
> Subject: Re: [basex-talk] BaseX 10.7 and util:if
>
> Message-ID:
>
>   <
> cap94bnmjeexxsfbyvyjtjvjcsbgkzbxonr99xnkpcv_oq9g...@mail.gmail.com>
>
> Content-Type: text/plain; charset="utf-8"
>
>
>
> ?thanks for the pointer. I?ve revised the documentation.
>
>
>
> One property of util:if was that you could omit the argument for the else
>
> branch. We decided to drop util:if, because the specification now provides
>
> a new syntax for if expressions where the else branch is optional:
>
>
>
>   if ($test) { $then }
>
>   if ($test) { $then } else { $else }
>
>
>
> The ternary operator was temporarily added to the spec as an official
>
> feature, but we eventually dropped it again to reduce the number of
>
> different notations for existing features (unfortunately, the popular A ? B
>
> : C syntax was no option, as ? is already used for lookups).
>
>
>
>
>
>
>
> Sent from Mail  for
> Windows
>
>
>
> *From: *basex-talk-requ...@mailman.uni-konstanz.de
> *Sent: *Wednesday, November 22, 2023 12:00 PM
> *To: *basex-talk@mailman.uni-konstanz.de
> *Subject: *BaseX-Talk Digest, Vol 167, Issue 7
>
>
>
>
>
>
> 
> Virus-free.www.avast.com
> 
> <#m_592359606519990792_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>

Re: [basex-talk] Bug in parse-xml-fragment() and ampersand entity?

[Christian Grün]

<pre>You could try this (at least for basic use cases):

declare function local:replaceText(
  $inputas text(),
  $search   as xs:string*,
  $replace  as xs:string*
) as node()* {
  let $pattern := string-join($search ! ('\b' || . || '\b'), '|')
  for $m in analyze-string($input, $pattern, 'j')/*
  return if($m/self::fn:match) then (
element { $replace[index-of($search, $m)] } {}
  ) else (
text { data($m) }
  )
};
local:replaceText(
  text { 'a String1 b String2 c' },
  ("String1", "String2", "String3"),
  ("Replace1", "Replace2", "Replace3")
)


On Wed, Nov 22, 2023 at 12:15 PM Zimmel, Daniel <d.zim...@esvmedien.de>
wrote:

> Thanks for the example, this makes sense.
>
> In my case, I think I can safely add a function to ensure that any “&amp;”
> string will be passed to parse-xml-fragment() as “&amp;amp;” because the
> source of the unparsed xml will always be well-formed.
>
>
>
> For background:
>
>
>
> I need to wrap a given list of string matches in a textnode to elements
> and because I need to to this recursively (eg the string is found three
> times in the textnode) and also need to pass a list of different matches
> with different replacements, I find it easier to do it with string
> replacements and then parsing the thenfragmented text node back to XML
> rather than ending up splitting the text node to multiple nodes directly.
>
>
>
> declare function my:replaceText(
>
>   $word as xs:string,
>
>   $search as xs:string*,
>
>   $replace as xs:string*) as xs:string {
>
>   (: $search = ("String1", "String2", "String3")
>
>  $replace = ("&lt;Replace1/&gt;", "&lt;Replace2/&gt;",
> "&lt;Replace3/&gt;")
>
>  :)
>
>  if (empty($search)) then $word
>
>   else
>
>   replace(my:replaceText($word, tail($search), tail($replace)),
> '(?<![">])'||my:escapeText(head($search)), head($replace),'j')
>
> };
>
>
>
> perhaps there is a better way to do this... how would one do this sort of
> replacement on mixed content without having to parse this back to XML?
>
> But this works for me when being careful to add some handling for &amp,
> &gt and &lt.
>
>
>
> +1 for internal parser velocity!
>
>
>
> *Von:* Christian Grün <christian.gr...@gmail.com>
> *Gesendet:* Dienstag, 21. November 2023 18:54
> *An:* Zimmel, Daniel <d.zim...@esvmedien.de>
> *Cc:* BaseX <basex-talk@mailman.uni-konstanz.de>
> *Betreff:* Re: [basex-talk] Bug in parse-xml-fragment() and ampersand
> entity?
>
>
>
> Yes, I can see the problem: &DUMMY; ist interpreted as unknown entity and
> thus replaced with a question mark (a better choice would be the Unicode
> Replacement Character xFFFD anyway, from today's perspective). We'll keep
> that in mind and think about alternatives.
>
>
>
> If your input is supposed to be interpreted as a single text fragment, one
> fallback solution (for now) would be
>
>
>
> data(parse-xml('<x>' || $string || '</x>'))
>
>
>
>
>
>
>
>
>
>
>
> Zimmel, Daniel <d.zim...@esvmedien.de> schrieb am Di., 21. Nov. 2023,
> 18:34:
>
> Thanks for the insight!
>
>
>
> I can see the benefit with your example – if you look at my example, it is
> clearly eating the text (“DUMMY”) which might be an edge case, but is
> obviously a problem when you think the function will give you an error in
> case of non-wellformedness – some text has silently been deleted.
>
>
>
> Daniel
>
>
>
> *Von:* Christian Grün <christian.gr...@gmail.com>
> *Gesendet:* Dienstag, 21. November 2023 16:59
> *An:* Zimmel, Daniel <d.zim...@esvmedien.de>
> *Cc:* basex-talk@mailman.uni-konstanz.de
> *Betreff:* Re: [basex-talk] Bug in parse-xml-fragment() and ampersand
> entity?
>
>
>
> Hi Daniel,
>
>
>
> Yes, I assume we’ll need to call it a bug… Although what BaseX is
> currently doing is known to us to be out of spec behavior. The function
> fn:parse-xml-fragments is based on our internal XML parser, which is much
> faster than the standard XML parser (in particular for small input), and it
> tolerates input that’s not perfectly well-formed. In addition, it accepts
> HTML entities without a linked DTD:
>
>
>
>parse-xml-fragment(`&auml;`)
>
>
>
> We should at least document the behavior or (better) introduce a custom
> BaseX function for it.
>
>
>
> Hope this helps (for now),
>
> Christian
>
>
>
>
>
>
>
> On Tue, Nov 21, 2023 at 3:17 PM Zimmel, Daniel <d.zim...@esvmedien.de>
> wrote:
>
> Hi,
>
> is this a bug?
>
> Query:
> parse-xml-fragment('Tom &amp; Jerry')
>
> Result:
> Tom ? Jerry
>
> Same result with:
> parse-xml-fragment('Tom &amp;DUMMY; Jerry')
>
> BaseX 10.7
>
> Saxon complains correctly that the resulting document node is not
> well-formed.
> BaseX should also return an error, shouldn't it?
>
> Best, Daniel
>
>
</pre>

Re: [basex-talk] Bug in parse-xml-fragment() and ampersand entity?

[Christian Grün]

Yes, I can see the problem:  ist interpreted as unknown entity and
thus replaced with a question mark (a better choice would be the Unicode
Replacement Character xFFFD anyway, from today's perspective). We'll keep
that in mind and think about alternatives.

If your input is supposed to be interpreted as a single text fragment, one
fallback solution (for now) would be

data(parse-xml('' || $string || ''))






Zimmel, Daniel  schrieb am Di., 21. Nov. 2023, 18:34:

> Thanks for the insight!
>
>
>
> I can see the benefit with your example – if you look at my example, it is
> clearly eating the text (“DUMMY”) which might be an edge case, but is
> obviously a problem when you think the function will give you an error in
> case of non-wellformedness – some text has silently been deleted.
>
>
>
> Daniel
>
>
>
> *Von:* Christian Grün 
> *Gesendet:* Dienstag, 21. November 2023 16:59
> *An:* Zimmel, Daniel 
> *Cc:* basex-talk@mailman.uni-konstanz.de
> *Betreff:* Re: [basex-talk] Bug in parse-xml-fragment() and ampersand
> entity?
>
>
>
> Hi Daniel,
>
>
>
> Yes, I assume we’ll need to call it a bug… Although what BaseX is
> currently doing is known to us to be out of spec behavior. The function
> fn:parse-xml-fragments is based on our internal XML parser, which is much
> faster than the standard XML parser (in particular for small input), and it
> tolerates input that’s not perfectly well-formed. In addition, it accepts
> HTML entities without a linked DTD:
>
>
>
>parse-xml-fragment(``)
>
>
>
> We should at least document the behavior or (better) introduce a custom
> BaseX function for it.
>
>
>
> Hope this helps (for now),
>
> Christian
>
>
>
>
>
>
>
> On Tue, Nov 21, 2023 at 3:17 PM Zimmel, Daniel 
> wrote:
>
> Hi,
>
> is this a bug?
>
> Query:
> parse-xml-fragment('Tom  Jerry')
>
> Result:
> Tom ? Jerry
>
> Same result with:
> parse-xml-fragment('Tom DUMMY; Jerry')
>
> BaseX 10.7
>
> Saxon complains correctly that the resulting document node is not
> well-formed.
> BaseX should also return an error, shouldn't it?
>
> Best, Daniel
>
>

Re: [basex-talk] Bug in parse-xml-fragment() and ampersand entity?

[Christian Grün]

Hi Daniel,

Yes, I assume we’ll need to call it a bug… Although what BaseX is currently
doing is known to us to be out of spec behavior. The function
fn:parse-xml-fragments is based on our internal XML parser, which is much
faster than the standard XML parser (in particular for small input), and it
tolerates input that’s not perfectly well-formed. In addition, it accepts
HTML entities without a linked DTD:

   parse-xml-fragment(``)

We should at least document the behavior or (better) introduce a custom
BaseX function for it.

Hope this helps (for now),
Christian



On Tue, Nov 21, 2023 at 3:17 PM Zimmel, Daniel 
wrote:

> Hi,
>
> is this a bug?
>
> Query:
> parse-xml-fragment('Tom  Jerry')
>
> Result:
> Tom ? Jerry
>
> Same result with:
> parse-xml-fragment('Tom DUMMY; Jerry')
>
> BaseX 10.7
>
> Saxon complains correctly that the resulting document node is not
> well-formed.
> BaseX should also return an error, shouldn't it?
>
> Best, Daniel
>

Re: [basex-talk] Bug in fn:fold-right?

[Christian Grün]

Hi Tim,

Thanks for the pointer. It seems that the reported bug has been fixed just
recently (at least I couldn’t reproduce it anymore). Could you try the
latest snapshot? [1]

Thanks in advance,
Christian

[1] https://files.basex.org/releases/latest/



On Mon, Nov 20, 2023 at 9:00 PM Thompson, Timothy 
wrote:

> Greetings,
>
>
>
> I noticed that there seems to be an issue with fn:fold-right in BaseX
> (10.8 beta). When I pass it a sequence of elements, they do not get
> atomized/accumulated. Example:
>
>
>
> let $data := 
>
>   a
>
>   b
>
>   c
>
> 
>
> return
>
>   {
>
> fold-right(
>
>   $data/*,
>
>   "",
>
>   concat(?, ".", ?)
>
> )
>
>   }
>
>
>
> returns: 
>
>
>
> The same query in Saxon-HE 11.4 returns: a.b.c.
>
>
>
> Best regards,
>
> Tim
>
>
>
>
>
> --
> Tim A. Thompson (*he, him*)
> Librarian for Applied Metadata Research
> Yale University Library
>
>
>

Re: [basex-talk] BaseX 10.7 and util:if

[Christian Grün]

…thanks for the pointer. I’ve revised the documentation.

One property of util:if was that you could omit the argument for the else
branch. We decided to drop util:if, because the specification now provides
a new syntax for if expressions where the else branch is optional:

  if ($test) { $then }
  if ($test) { $then } else { $else }

The ternary operator was temporarily added to the spec as an official
feature, but we eventually dropped it again to reduce the number of
different notations for existing features (unfortunately, the popular A ? B
: C syntax was no option, as ? is already used for lookups).


On Mon, Nov 20, 2023 at 1:15 PM Andy Bunce  wrote:

> Hi,
> The documentation for the utility module [1] says certain functions will
> be removed in version 11 because they will be in XPath 4.0.
> However, util:if has been removed from the documentation and I think there
> is no equivalent function in this case.
> util:if is an alternative  syntax for the BaseX Ternary operator  [2].
>
> ( I have used it because it was easier than trying to get my XQuery parser
> EBNF to accept  $ok ?? 1 !! 0 etc)
>
> Is util:if to be removed?
>
> /Andy
> [1] https://docs.basex.org/wiki/Utility_Module#util:if
> [2] https://docs.basex.org/wiki/XQuery_Extensions#Ternary_If
>

Re: [basex-talk] The duration of a cached job

[Christian Grün]

Hi Andy,

Thanks for the observation. I’ve fixed the problem (backed by an additional
JUnit test), and a new snapshot is online [1].

Best,
Christian

[1] https://files.basex.org/releases/latest/


On Mon, Nov 20, 2023 at 12:36 PM Andy Bunce  wrote:

> Hi,
>
> Back in the winter of 2017, BaseX 8.6 , I reported an issue with job
> duration [1]
>
> A question about jobs:list-details[2]
> I had expected the attribute duration: evaluation time (for running and
> cached jobs) to be frozen once the job is cached.
>
>
> Using BaseX 10.7 it seems this issue is back
> ```
>  job:eval("42",(),map{"cache":true()})
> job24
>
>job:list-details("job24")
>  duration="PT39.031S"
>  reads="(none)" writes="(none)" time="2023-11-20T11:23:44.133Z">42
>
>job:list-details("job24")
>  duration="PT2M35.894S"
> reads="(none)" writes="(none)" time="2023-11-20T11:23:44.133Z">42
> ```
> /Andy
>
> [1]
> https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg09036.html
> [2] https://docs.basex.org/wiki/Job_Module#job:list-details
> 
>

Re: [basex-talk] WG: JAVA Performance Issue under RH8 by using BASEX with NFS system

[Christian Grün]

Hi Andre,

It’s probably out of scope to give a reasonable response to your questions
via the public mailing list (feel free to ask for our commercial services),
but here’s a quick shot:

>
>- why is the whole database read even for a "small" query ? (looks
>like a missing index or bad execution plan.. ?)
>
> It may depend a lot on what the query does. Could you give us an example
for a typical “small query”?

>
>- The UNIX related question (maybe it's worth opening a ticket with
>   redhat)
>
> I guess so. I’m sorry I have no in-depth knowledge on why RHL 6 and 8
behaves differently here.

Have you already tried different JDK versions/distributions?

Best,
Christian

Re: [basex-talk] db:copy

[Christian Grün]

>
> Are they use different methods to copy databases? Or that jdk bug only?
>

It’s exactly the same; when db:copy is called, the function isn’t aware of
whether it has been called by a server or standalone/client instance.

Are you sure that both the server and the standalone version of BaseX use
the same JRE? You can e.g. check this with the following query:

for $p in proc:property-names()[starts-with(., 'java.vm.')]
return $p || ': ' || proc:property($p)




> 16.11.2023, 15:00, "Christian Grün" :
>
> Hi, this definitely looks like a bug in the JDK context. What happens if
> you upgrade your JDK to a newer version?
> Best, Christian
>
>
> On Thu, Nov 16, 2023 at 12:56 PM Ветошкин Владимир 
> wrote:
>
>
> Hi everyone.
>
> Since I have changed jdk version to 12 (azul) - I get error, when I copy
> database (db:copy).
> It's only in basexserver. When I run basex - everything is ok.
>
> Error:
> Improper use? Potential bug? Your feedback is welcome: Contact:
> basex-talk@mailman.uni-konstanz.de Version: BaseX 9.1.2 Java: Azul
> Systems, Inc., 12.0.2 OS: Linux, amd64 Stack Trace:
> java.lang.NoClassDefFoundError: Could not initialize class
> sun.nio.fs.UnixCopyFile at
> java.base/sun.nio.fs.UnixFileSystemProvider.copy(UnixFileSystemProvider.java:258)
> at java.base/java.nio.file.Files.copy(Files.java:1297) at
> org.basex.io.IOFile.copyTo(IOFile.java:319) at
> org.basex.core.cmd.Copy.copy(Copy.java:83) at
> org.basex.query.up.primitives.name.DBCopy.apply(DBCopy.java:43) at
> org.basex.query.up.NameUpdates.apply(NameUpdates.java:58) at
> org.basex.query.up.ContextModifier.apply(ContextModifier.java:132) at
> org.basex.query.up.Updates.apply(Updates.java:156) at
> org.basex.query.QueryContext.iter(QueryContext.java:351) at
> org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) at
> org.basex.core.cmd.AQuery.query(AQuery.java:109) at
> org.basex.core.cmd.XQuery.run(XQuery.java:22) at
> org.basex.core.Command.run(Command.java:257) at
> org.basex.core.Command.execute(Command.java:93) at
> org.basex.server.ClientListener.run(ClientListener.java:140)
> --
> С уважением,
> Ветошкин Владимир Владимирович
>
>
>
>
> --
> С уважением,
> Ветошкин Владимир Владимирович
>
>

Re: [basex-talk] db:copy

[Christian Grün]

Hi, this definitely looks like a bug in the JDK context. What happens if
you upgrade your JDK to a newer version?
Best, Christian


On Thu, Nov 16, 2023 at 12:56 PM Ветошкин Владимир 
wrote:

>
> Hi everyone.
>
> Since I have changed jdk version to 12 (azul) - I get error, when I copy
> database (db:copy).
> It's only in basexserver. When I run basex - everything is ok.
>
> Error:
> Improper use? Potential bug? Your feedback is welcome: Contact:
> basex-talk@mailman.uni-konstanz.de Version: BaseX 9.1.2 Java: Azul
> Systems, Inc., 12.0.2 OS: Linux, amd64 Stack Trace:
> java.lang.NoClassDefFoundError: Could not initialize class
> sun.nio.fs.UnixCopyFile at
> java.base/sun.nio.fs.UnixFileSystemProvider.copy(UnixFileSystemProvider.java:258)
> at java.base/java.nio.file.Files.copy(Files.java:1297) at
> org.basex.io.IOFile.copyTo(IOFile.java:319) at
> org.basex.core.cmd.Copy.copy(Copy.java:83) at
> org.basex.query.up.primitives.name.DBCopy.apply(DBCopy.java:43) at
> org.basex.query.up.NameUpdates.apply(NameUpdates.java:58) at
> org.basex.query.up.ContextModifier.apply(ContextModifier.java:132) at
> org.basex.query.up.Updates.apply(Updates.java:156) at
> org.basex.query.QueryContext.iter(QueryContext.java:351) at
> org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) at
> org.basex.core.cmd.AQuery.query(AQuery.java:109) at
> org.basex.core.cmd.XQuery.run(XQuery.java:22) at
> org.basex.core.Command.run(Command.java:257) at
> org.basex.core.Command.execute(Command.java:93) at
> org.basex.server.ClientListener.run(ClientListener.java:140)
> --
> С уважением,
> Ветошкин Владимир Владимирович
>
>

Re: [basex-talk] WG: JAVA Performance Issue under RH8 by using BASEX with NFS system

[Christian Grün]

…difficult to tell what might be the reason; we didn’t get similar reports
so far. Most of the BaseX caching algorithms rely on the file system, but
we don’t differ between Windows- and UNIX-based systems.

Does it make a difference if you change another JDK distribution?

Best,
Christian


On Mon, Nov 13, 2023 at 12:32 PM Otto, Michael (Layout Analysis) <
michael.o...@globalfoundries.com> wrote:

>
>
> --
> *Von:* Otto, Michael (Layout Analysis) 
> *Gesendet:* Sonntag, 12. November 2023 17:16
> *An:* basex-talk-requ...@mailman.uni-konstanz.de <
> basex-talk-requ...@mailman.uni-konstanz.de>
> *Betreff:* AW: JAVA Performance Issue under RH8 by using BASEX with NFS
> system
>
> Hello.
>
> Since switching basex (853) application from RH6 to RH8 I do see a
> performance issue.
>
>
> internal tests:
> RH6/NFS --> runtime < 1s
> RH8/local storage on server --> runtime < 1s
>
> RH8/NFS --> runtime > 15s
>
> It seams that we see a caching issue in RH8 context (1st read of RH6 taes
> longer too).
>
> I did several tests with several JAVA and BASEX(V10.7) versions (including
> latest one).
> IT tried several mounting options for RH8 server to NFS too, but that did
> not solve that issue.
>
>
> I that a known issue? I guess this is a caching issue of RH8 (actual only
> seen by using BaseX).
> Is there any known solution for this?
>
> BR
>
> GLOBALFOUNDRIES Dresden Module One LLC & Co. KG:
>
>
>
> *Michael Otto*
>
> *Tapeout Engn*
>
> *FAB 1 TAPE OUT INTEGRATION*
>
>
>
> T +49 (0) 351 277-86013
>
> F +49 (0) 351 277-86013
>
> M/S: I21-MPI
> michael.o...@globalfoundries.com
>
>
>
>
>
>
> http://www.globalfoundries.com/
>
>
>
>
>
> GLOBALFOUNDRIES Dresden Module One Limited Liability Company & Co. KG
>
> Wilschdorfer Landstr. 101, 01109 Dresden, Deutschland
>
> Registered Office/ Sitz Dresden I Registergericht Dresden HRA 5255
>
> General Partner/ Vertretungsberechtigter Komplementär: GLOBALFOUNDRIES
> Dresden Module One LLC (Sitz Wilmington, Delaware, USA)
>
> General Manager/ Geschäftsführer der GLOBALFOUNDRIES Dresden Module One
> LLC: Thomas Caulfield
>
> --
> *Von:* Otto, Michael (Layout Analysis) 
> *Gesendet:* Mittwoch, 8. November 2023 12:01
> *An:* basex-talk-requ...@mailman.uni-konstanz.de <
> basex-talk-requ...@mailman.uni-konstanz.de>
> *Betreff:* JAVA Performance Issue under RH8 by using BASEX with NFS system
>
> Hello.
>
> Since switching basex (853) application from RH6 to RH8 I do see a
> performance issue.
>
>
> internal tests:
> RH6/NFS --> runtime < 1s
> RH8/local storage on server --> runtime < 1s
>
> RH8/NFS --> runtime > 15s
>
> It seams that we see a caching issue in RH8 context (1st read of RH6 taes
> longer too).
>
> I did several tests with several JAVA and BASEX(V10.7) versions (including
> latest one).
> IT tried several mounting options for RH8 server to NFS too, but that did
> not solve that issue.
>
>
> I that a known issue? I guess this is a caching issue of RH8 (actual only
> seen by using BaseX).
> Is there any known solution for this?
>
> BR
>
> GLOBALFOUNDRIES Dresden Module One LLC & Co. KG:
>
>
>
> *Michael Otto*
>
> *Tapeout Engn*
>
> *FAB 1 TAPE OUT INTEGRATION*
>
>
>
> T +49 (0) 351 277-86013
>
> F +49 (0) 351 277-86013
>
> M/S: I21-MPI
> michael.o...@globalfoundries.com
>
>
>
>
>
> http://www.globalfoundries.com/
>
>
>
>
>
> GLOBALFOUNDRIES Dresden Module One Limited Liability Company & Co. KG
>
> Wilschdorfer Landstr. 101, 01109 Dresden, Deutschland
>
> Registered Office/ Sitz Dresden I Registergericht Dresden HRA 5255
>
> General Partner/ Vertretungsberechtigter Komplementär: GLOBALFOUNDRIES
> Dresden Module One LLC (Sitz Wilmington, Delaware, USA)
>
> General Manager/ Geschäftsführer der GLOBALFOUNDRIES Dresden Module One
> LLC: Thomas Caulfield
>
>
>

Re: [basex-talk] Crash on ADD file.xml

[Christian Grün]

Hi Csaba,

If you want to perform parallel/concurrent read and write operations on
your databases, you must exclusively use the client/server architecture.
See [1] for more details.

Best,
Christian

[1] https://docs.basex.org/wiki/Startup#Concurrent_Operations



Csaba Fekete  schrieb am So., 12. Nov. 2023, 14:20:

> Hi all
>
> Scenario: I am trying to add a file *while a query is being run through
> the rest api*.
> Everything is working fine if no query is being run.
> Am I trying to do something that isn't supported?
>
> The command is:
>
> /opt/basex/bin/basex -v -c"OPEN dbname;ADD $fname.xml"
>
> The command exits with the message:
>
> Improper use? Potential bug? Your feedback is welcome:
> Contact: basex-talk@mailman.uni-konstanz.de
> Version: BaseX 11.0 beta
> Java: Ubuntu, 11.0.20.1
> OS: Linux, amd64
> Stack Trace:
> java.lang.RuntimeException: dbname: lock file does not exist.
> at org.basex.util.Util.notExpected(Util.java:68)
> at org.basex.data.DiskData.finishUpdate(DiskData.java:246)
> at org.basex.core.cmd.ACreate.update(ACreate.java:97)
> at org.basex.core.cmd.Add.run(Add.java:56)
> at org.basex.core.Command.run(Command.java:233)
> at org.basex.core.Command.execute(Command.java:93)
> at org.basex.api.client.LocalSession.execute(LocalSession.java:131)
> at org.basex.api.client.Session.execute(Session.java:36)
> at org.basex.core.CLI.execute(CLI.java:94)
> at org.basex.core.CLI.execute(CLI.java:78)
> at org.basex.core.CLI.execute(CLI.java:65)
> at org.basex.BaseX.(BaseX.java:82)
> at org.basex.BaseX.main(BaseX.java:44)
>
> Thanks as always
> Csaba
>

Re: [basex-talk] Passing an array to REST HTTP service

[Christian Grün]

Hi Csaba,

The variable types of the REST API are limited to atomic values. With BaseX
11, however, it will be possible to bind multiple values [1]:

http://localhost:8080/rest?run=query=3=4=5

(: XQuery file:  query.xq :)
declare variable $n as xs:integer* external;
array { $n }

You can try the latest stable snapshot [2].

Best,
Christian

[1] https://docs.basex.org/wiki/REST#GET_Method_2
[2] https://files.basex.org/releases/latest/


On Fri, Nov 3, 2023 at 8:30 AM Csaba Fekete  wrote:

> Hi
>
> I'm wondering if this is at all possible...
> I defined an array variable in my XQUERY:
>
> declare variable $ca as array(xs:integer) external := [];
>
>
> How can I assign a value (e.g [1,2]) to this variable through the URL? I
> have already tried:
> http://host:port/rest/dbname?run=query.xq&$ca[]=1&$ca[]=2
> That didn't work. Neither did this:
> http://host:port/rest/dbname?run=query.xq&$ca=1,2
>
> Any hints are much appreciated.
> Thanks
> Csaba
>

Re: [basex-talk] Text index on uncommon node

[Christian Grün]

Hi Matt,

I assume the culprit is the common string you're looking up. It probably
occurs very often in your database. You can e.g. verify this via
index:texts('MyDatabase', 'false') or count(db:get('MyDatabase')//text()[.
= 'false']).

If you don't need to perform exact queries on arbitrary elements, you could
think about restricting the text index to specific element names to reduce
the number of intermediate hits [1].

Hope this helps,
Christian

[1] https://docs.basex.org/wiki/Indexes#Selective_Indexing


Matthew Dziuban  schrieb am Mo., 30. Okt. 2023, 22:09:

> Hi Christian,
>
> Thanks for the quick response, and sure thing. It does look like the text
> index is applied in both cases. While I was writing up an example of the
> slow query, I realized that the way I'll actually be querying it is by
> wrapping the condition in not(...). After doing so, it now only takes 6
> seconds to run -- still slower but better. The slow query looks like this:
>
> for $x in db:open('MyDatabase')/data/element
> where not($x/child1/child2/valid = 'false')
> return $x
>
> And the fast query looks like this:
>
> for $x in db:open('MyDatabase')/data/element
> where $x/child3/child4/id = '123'
> return $x
>
> Let me know if this is helpful -- if not, I could share more info in a
> direct email about the actual structure of the database and the Info output.
>
> Thanks,
> Matt
>
> On Mon, Oct 30, 2023 at 4:40 PM Christian Grün 
> wrote:
>
>> Hi Matt,
>>
>> In general, all nodes are treated identically, no matter what the
>> hierarchy is or regular the target path is.
>>
>> Could you share some more information with us? How do the queries look
>> like (the slow and the fast one)? Is the text index applied in both cases?
>>
>> Thanks in advance,
>> Christian
>>
>>
>>
>> Matthew Dziuban  schrieb am Mo., 30. Okt. 2023,
>> 21:33:
>>
>>> Hi all,
>>>
>>> I'm working with a database structured like so:
>>>
>>> 
>>>   ...
>>>   ...
>>>   ...
>>> 
>>>
>>> There are a total of about 1.5 million  nodes in the database.
>>> Each  has many child nodes, one of which is uncommon -- it only
>>> appears in 727 s.
>>>
>>> I'm writing a query that has a condition on this uncommon field, but the
>>> query takes about 20 seconds to run, whereas another with a condition on a
>>> child node that appears in every  only takes about 20 milliseconds
>>> to run.
>>>
>>> Based on the Info in the GUI, it does appear that the text index is
>>> being used -- I see 'apply text index for "..."'. Is it expected that the
>>> query time would be this much longer? Is the text index somehow built
>>> differently for nodes that don't appear often in the database?
>>>
>>> Thanks in advance,
>>> Matt
>>>
>>
>
> --
> Matthew R. Dziuban
> mattdziuban.com
> 703-973-6717
> mrdziu...@gmail.com
>

Re: [basex-talk] Text index on uncommon node

[Christian Grün]

Hi Matt,

In general, all nodes are treated identically, no matter what the hierarchy
is or regular the target path is.

Could you share some more information with us? How do the queries look like
(the slow and the fast one)? Is the text index applied in both cases?

Thanks in advance,
Christian



Matthew Dziuban  schrieb am Mo., 30. Okt. 2023, 21:33:

> Hi all,
>
> I'm working with a database structured like so:
>
> 
>   ...
>   ...
>   ...
> 
>
> There are a total of about 1.5 million  nodes in the database.
> Each  has many child nodes, one of which is uncommon -- it only
> appears in 727 s.
>
> I'm writing a query that has a condition on this uncommon field, but the
> query takes about 20 seconds to run, whereas another with a condition on a
> child node that appears in every  only takes about 20 milliseconds
> to run.
>
> Based on the Info in the GUI, it does appear that the text index is being
> used -- I see 'apply text index for "..."'. Is it expected that the query
> time would be this much longer? Is the text index somehow built differently
> for nodes that don't appear often in the database?
>
> Thanks in advance,
> Matt
>

Re: [basex-talk] RESTXQ Custom Multipart Response

[Christian Grün]

Hi Jean-Marc,

For the time being, I’m sorry multipart support is only available for
requests (multipart/mixed [1], multipart/form-data [2]). Responses are
limited to single parts (multipart data has not been considered in the
RESTXQ specification).

I have added a clarifying comment to our documentation.

Best,
Christian

[1] https://docs.basex.org/wiki/RESTXQ#Multipart_Types
[2] https://docs.basex.org/wiki/RESTXQ#HTML_Form_Fields


On Sun, Oct 29, 2023 at 8:21 AM Jean-Marc ANSEL  wrote:

> Hello,
>
> I'm trying to do a custom multipart response with basex RESTXQ module.
>
> The Custom Response paragraph of https://docs.basex.org/wiki/RESTXQ
> indicates :
>
> Custom responses can be generated in XQuery by returning an
> |rest:response| element, an |http:response| child node that matches the
> syntax of the EXPath HTTP Client Module
>  specification, and optional child
> nodes that will be serialized as usual.
>
> So, the §4.1 of http://expath.org/spec/http-client   (EXPath HTTP Client
> Module ) indicates :
>
> message = string>
> (http:header*,
>   (http:multipart |
>http:body)?)
> 
>
>
> But when I try :
>
> declare %rest:path('media4') function page:testjm4() {
> 
> 
>
>
>  TEST
>
>
> };
>
>
> The answer is irremediably :
>
> Stopped at /root/basex/webapp/test.xqm, 46/39:
> [basex:restxq] Unexpected node: http://expath.org/ns/http-client; media-type="text">TEST
> 
>
> Neither hhtp:body or http:multipart element can be placed as a child of
> http:response, that seems contradictory with the expath.org spec...
>
> Any help would be appreciated...
>
> Thanks o lot!
>
> JM
>
> Using :
>
> BaseX Version 10.7
> httpserver : basexhttp
> Operating System: Ubuntu 18.04.6 LTS
> Kernel: Linux 4.15.0-197-generic
> Architecture: x86-64
>
>

Re: [basex-talk] Crash when starting basexgui

[Christian Grün]

Thanks. It seems we have to wait for Apple to fix this. Unfortunately,
their bug tracker doesn’t seem to be public [2].

[1]
https://github.com/Homebrew/homebrew-core/issues/150824#issuecomment-1776139189


On Wed, Oct 25, 2023 at 1:01 PM Johan Mörén  wrote:

> Running MacOS Sonoma Apple Silicon M2 processor and ran into the same
> problem when homebrew updated my JDK to 21.0.1 today.
>
> According to this article it seems like applications like jmeter and
> IntelliJ needs to fix this by implementing a specific interface. I guess
> this applies to Basexgui as well?
>
>
> https://stackoverflow.com/questions/77283578/sonoma-and-nsapplicationdelegate-applicationsupportssecurerestorablestate
>
> Regards,
> Johan
>
>

Re: [basex-talk] Crash when starting basexgui

[Christian Grün]

Hi Joe, thanks Michael,

I’m pretty sure this is something that needs to be fixed by Apple or the
OpenJDK guys. A Homebrew issue seems to point to a similar failure [1].

Maybe BaseX can be run with an older version of the JRE?

Best,
Christian

[1] https://github.com/Homebrew/homebrew-core/issues/150965



On Tue, Oct 24, 2023 at 2:28 PM Michael Seiferle  wrote:

> Dear Joe,
>
> Thanks for the report,I haven’t upgraded to Sonoma yet, but will this
> afternoon :)
>
> I will try to reproduce the error and come back to the list. :-)
>
>
> Best
> Michael
>
> Am 24.10.2023 um 13:40 schrieb Joe Wicentowski :
>
> 
> Hi all,
>
> I am experiencing a crash when starting basexgui. The splash screen
> appears briefly, and then the message pasted in below appears, and I've
> also attached the referenced error report file and the crash report that
> appears in the Mac crash dialog. I'm using BaseX 10.7, installed via
> Homebrew (which installed BaseX's openjdk dependency and thus is using
> openjdk 21), on macOS Sonoma (14.0).
>
> I'd be happy to provide any other info that would help in identifying the
> cause of the crash.
>
> Joe
>
>
> % basexgui
> 2023-10-24 07:34:22.308 java[69449:3862142] WARNING: Secure coding is
> automatically enabled for restorable state! However, not on all supported
> macOS versions of this application. Opt-in to secure coding explicitly by
> implementing
> NSApplicationDelegate.applicationSupportsSecureRestorableState:.
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGILL (0x4) at pc=0x7ff80d2b1b73, pid=69449, tid=259
> #
> # JRE version: OpenJDK Runtime Environment Homebrew (21.0) (build 21)
> # Java VM: OpenJDK 64-Bit Server VM Homebrew (21, mixed mode, sharing,
> tiered, compressed oops, compressed class ptrs, g1 gc, bsd-amd64)
> # Problematic frame:
> # C  [AppKit+0xc8fb73]  _NSCarbonMenuCrashIfNeeded+0x258
> #
> # No core dump will be written. Core dumps have been disabled. To enable
> core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /Users/joe/workspace/hsg-project/repos/hsg-shell/hs_err_pid69449.log
> #
> # If you would like to submit a bug report, please visit:
> #   https://github.com/Homebrew/homebrew-core/issues
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
> zsh: abort  basexgui
>
>
> 
> 
>
>

Re: [basex-talk] New escape-solidus option

[Christian Grün]

Hi Tim,

A new snapshot is available [1]. Please note that you’ll need to write…




  Test

 => serialize(map {

  "escape-solidus": "no",

  "method": "json",

  "json": map {"format": "basic" }

})

…because “escape-solidus” is defined as a standard serialization parameter
(similar to “json-node-output-method”). It seems that will be the final
solution [2].

Hope this helps,
Christian

[1] https://files.basex.org/releases/latest/
[2] https://github.com/qt4cg/qtspecs/issues/759


On Fri, Oct 20, 2023 at 10:07 PM Thompson, Timothy <
timothy.thomp...@yale.edu> wrote:

> Hello,
>
>
>
> I was very happy to see that a new *escape-solidus* option had been
> defined for XQuery/XPath 4.0. Is this available in the latest BaseX
> snapshot?
>
>
>
> I tried this:
>
> 
>
>   Test
>
>  => serialize(map {"method": "json", "json": map {"format":
> "basic", "escape-solidus": "no"}})
>
>
>
> But it returns:
>
> {"Escape\/Solidus":"Test"}
>
>
>
>
>
> Thank you,
>
> Tim
>
>
>
>
>
> --
> Tim A. Thompson (*he, him*)
> Librarian for Applied Metadata Research
> Yale University Library
>
>
>

Re: [basex-talk] New escape-solidus option

[Christian Grün]

Hi Tim,

You are really on the cutting edge :) I’ll upload a new snapshot with the
new option in a few days; I will keep you posted.

Best,
Christian



Thompson, Timothy  schrieb am Fr., 20. Okt.
2023, 22:07:

> Hello,
>
>
>
> I was very happy to see that a new *escape-solidus* option had been
> defined for XQuery/XPath 4.0. Is this available in the latest BaseX
> snapshot?
>
>
>
> I tried this:
>
> 
>
>   Test
>
>  => serialize(map {"method": "json", "json": map {"format":
> "basic", "escape-solidus": "no"}})
>
>
>
> But it returns:
>
> {"Escape\/Solidus":"Test"}
>
>
>
>
>
> Thank you,
>
> Tim
>
>
>
>
>
> --
> Tim A. Thompson (*he, him*)
> Librarian for Applied Metadata Research
> Yale University Library
>
>
>

Re: [basex-talk] Access denied using client

[Christian Grün]

Szia Csaba,

Thanks for your observation. You may need to set the password first, and
start the server after that.

Does this help?
Christian



Csaba Fekete  schrieb am Fr., 20. Okt. 2023, 21:40:

> Hello,
>
> A total newbie here, I'm a first time user.
> I have installed basex on Ubuntu and I'm running the server attached to
> 127.0.0.1, on port 1984. It looks like this is working and the server is
> listening.
> Now I create a password for admin:
>
> basex@cac5ba21e248:/opt/basex/bin$ ./basex
> BaseX 10.8 beta [Standalone]
> Try 'help' to get more information.
> > PASSWORD xxx
> Password of user 'admin' changed.
>
> Now I'm trying to connect using the client:
>
> basex@cac5ba21e248:/opt/basex/bin$ ./basexclient -n127.0.0.1 -p1984
> -Uadmin -Pxxx
> Access denied: admin.
>
> The server log confirms the same:
>
> basex@cac5ba21e248:~/basex/.logs$ cat 2023-10-20.log
> 18:51:30.575 SERVER admin OK Server was started (port: 1984).
> 18:51:42.316 172.22.0.1:41314 admin ERROR Access denied: admin. 30.7 ms
> 18:53:04.079 172.22.0.1:59892 admin ERROR Access denied: admin. 23.59 ms
>
> What am I missing?
> Thanks and regards
>
> PS. I'm doing this inside a docker container if that matters.
>
>
>

Re: [basex-talk] Advice on accepting UTF-16 XML (and others) from RESTXQ?

[Christian Grün]

Hi Ash,

The easiest way to accept an XML document, no matter which encoding it has,
looks as follows:

(: your RESTXQ endpoint :)
declare
  %rest:POST('{$xml}')
  %rest:path('/test')
function local:xml($xml) {
  $xml//text()
};

# a command-line call to send the request
curl -H"Content-Type:application/xml" -XPOST -Ttest.xml "
http://localhost:8080/test;

Here is a slightly more complex example to upload the document via an HTML
form [1]:

declare
  %rest:GET
  %rest:path('/test')
  %output:method('html')
function local:f() {
  


  
};

declare
  %rest:POST
  %rest:path("/test")
  %rest:form-param("files", "{$files}")
function local:upload($files) {
  {
for $namein map:keys($files)
let $content := $files($name)
return {
  try {
fetch:binary-doc($content)
  } catch * {
attribute error { $err:description }
  }
}
  }
};

The uploaded binary is converted to an XML document with fetch:binary-doc
[2]. An error is output if the input cannot be parsed as XML. Again, the
encoding doesn’t matter: It will dynamically be derived from either the
byte order mask, the first bytes or the embedded XML declaration.

It seems that’s the solution that would also help you to get the unit test
running, right? Maybe the standard function will be enhanced to also accept
xs:base64Binary in XQuery 4, but that’s still discussed [3].

Hope this helps,
Christian

[1] https://docs.basex.org/wiki/RESTXQ#File_Uploads
[2] https://docs.basex.org/wiki/Fetch_Module
[3] https://github.com/qt4cg/qtspecs/issues/748

Re: [basex-talk] Db:token() vs db:attribute(): Attribute is much slower -- why

[Christian Grün]

Hi Eliot,

If the length of a search strings exceeds MAXLEN [1], a sequential search
will be performed instead. You can achieve index-based lookups by
increasing the limit when creating/optimizing a database.

Hope this helps,
Christian

[1] https://docs.basex.org/wiki/Options#MAXLEN


On Wed, Oct 11, 2023 at 1:16 AM Eliot Kimber 
wrote:

> Sorry, my times are off—I was doing a nonsense multiplication of the
> result of prof:track.
>
> Reporting the raw times returned by db:token(), I get:
>
> "lookup by docPath took 0.255696ms"
>
> "lookup by docPath took 0.054183ms"
>
> "lookup by docPath took 0.211373ms"
>
> "lookup by docPath took 0.175889ms"
>
> "lookup by docPath took 0.220833ms"
>
> "lookup by docPath took 0.156875ms"
>
> "lookup by docPath took 0.139612ms"
>
> "lookup by docPath took 0.007018ms"
>
> "lookup by docPath took 0.179297ms"
>
> "lookup by docPath took 0.00964ms"
>
> "lookup by docPath took 0.093668ms"
>
>
>
> Which adds up to about 400ms for 1300 topics looked up.
>
>
>
> Using db:attribute() I get:
>
> "lookup by docPath took 3.794676ms"
>
> "lookup by docPath took 0.032425ms"
>
> "lookup by docPath took 0.092237ms"
>
> "lookup by docPath took 0.097815ms"
>
> "lookup by docPath took 1.246617ms"
>
> "lookup by docPath took 0.050855ms"
>
> "lookup by docPath took 0.065685ms"
>
> "lookup by docPath took 0.00511ms"
>
> "lookup by docPath took 0.057833ms"
>
>
>
> Cheers,
>
>
>
> E.
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com 
>
> LinkedIn  | Twitter
>  | YouTube
>  | Facebook
> 
>
>
>
> *From: *Eliot Kimber 
> *Date: *Tuesday, October 10, 2023 at 6:02 PM
> *To: *basex-talk@mailman.uni-konstanz.de <
> basex-talk@mailman.uni-konstanz.de>
> *Subject: *Db:token() vs db:attribute(): Attribute is much slower -- why
>
> Using 10.6 on macOS:
>
>
>
> I have a data file with elements like:
>
>bundlename="bundle-appportman-application-portfolio-management"
> topicpath="product/application-portfolio-management/concept/application-portfolio-management-landing-page.dita"
> notviewed="viewed" viewcount="13579">
>
>
>
> One such entry for each topic in our public docs server that is accessed
> at least once (Adobe page views data).
>
>
>
> I generate a report for all the topics used from a root map that includes
> the page views data for each topic for which there is an entry.
>
>
>
> The database with this data set is indexed with an attribute and a token
> index.
>
>
>
> In my testing I’m finding that lookup using db:attribute() on the
> @topicpath attribute (which is always a single value) is about 10x slower
> than using db:token().
>
>
>
> That is, I have this code:
>
> $records := db:token($analyticsmgmt:analyticsDatabase, $docPath,
> 'topicpath')/..
>
>
>
> When I run this for a set of about 1200 topics, the db:token version takes
> about 400ms but the db:attribute version takes 9 seconds!
>
>
>
> I’m surprised at this result—I would expect the two forms of index to
> perform with the same speed: the attribute is a single blank-delimited
> token and I would expect the string match under the covers to be using the
> same code path.
>
>
>
> But using the db:token database is clearly faster by a lot.
>
>
>
> If I report the time each lookup uses, the times vary quite a lot, but for
> the token lookup they tend to be in the 10-60ms range, but for the
> attribute lookup, many are in the 1 or 2 second range, which explains the
> much longer run time.
>
>
>
> I’m wondering if this result is expected due to implementation details of
> the attribute index as distinct from the token index or if this indicates
> some logic bug in the attribute index implementation.
>
>
>
> Cheers,
>
>
>
> E.
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com 
>
> LinkedIn  | Twitter
>  | YouTube
>  | Facebook
> 
>

Re: [basex-talk] More options for http-client?

[Christian Grün]

Hi Andreas,

are there any plans for the http client to become even more configurable?
>

Currently no, but it would certainly be possible, given that we’ve recently
added some BaseX-specific options to handle CSV, JSON and HTML input.
Finalizing version 2 of the spec would still be a better option [1], but we
haven’t found enough time for it yet.

Best,
Christian

[1] https://expath.github.io/expath-cg/specs/http-client-2/



>
>
> I have seen in the BaseX sources that the HttpClient.Builder
> (org.basex.io.IOUrl:173) is mostly served with standards. (except
> sslContext, connectTimeout, followRedirects).
>
>
>
> For example, I have a special case where I need to force the version to
> HTTP/1.1.
>
>
>
> Tried as a workaround to create my own instance via HttpClient.Builder
> using the XQuery Java binding, but failed on the interfaces.
>
>
>
>
>
> Thanks a lot
>
> Andreas
>