Re: [basex-talk] .xar deployment

[Andy Bunce]

Hi Régis,

I am pretty sure your issue comes from the SSL certificates available to
your Java VM, and is not a problem with your xar package.
Running Basex with -d shows more information [2]

Something like [1] may fix it for one machine, but it is a slow process and
far from ideal.

/Andy

[1]
https://stackoverflow.com/questions/21076179/pkix-path-building-failed-and-unable-to-find-valid-certification-path-to-requ/36427118#36427118
[2] basex -d -c ""
...
javax.net.ssl.SSLHandshakeException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target
at
java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:578)
at
java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:123)
at org.basex.io.IOUrl.response(IOUrl.java:145)
at org.basex.io.IOUrl.inputStream(IOUrl.java:127)
at org.basex.io.in.BufferInput.get(BufferInput.java:49)
at org.basex.io.IOUrl.read(IOUrl.java:112)
at org.basex.query.util.pkg.RepoManager.install(RepoManager.java:64)
at org.basex.core.cmd.RepoInstall.run(RepoInstall.java:36)
at org.basex.core.Command.run(Command.java:233)
at org.basex.core.Command.execute(Command.java:93)
at org.basex.api.client.LocalSession.execute(LocalSession.java:131)
at org.basex.api.client.Session.execute(Session.java:36)
at org.basex.core.CLI.execute(CLI.java:94)
at org.basex.core.CLI.execute(CLI.java:78)
at org.basex.core.CLI.execute(CLI.java:65)
at org.basex.BaseX.(BaseX.java:83)
at org.basex.BaseX.main(BaseX.java:45)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target


On Wed, 10 Apr 2024 at 11:16, Régis WITZ  wrote:

> Hi all,
> I'm kind of a XQuery noob, but I'm guilty of the creation of a little
> BaseX module, Heimdall (10.5281/zenodo.10638084
> ).
> It's automatically built, unit tested and documented using GitLab CI.
> All build results, including the .xar file, are hosted using GitLab Pages.
> Thus, the module .xar archive is available here :
> |https://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar|
> .
> As you see, all this is hosted in Huma-Num's self-hosted GitLab instance
> (Huma-Num is a french research infrastructure).
>
> However, this mail is /not/ (just) a shameless plug.
> My issue is that I cannot install my .xar directly with |REPO INSTALL| :
>
>  1. I cannot install it with the CLI:
>
> basex -c" path='
> https://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar'/>"
> [repo:not-found] Package
> 'https://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar'
> not found.
>
> Things are perfectly fine with for example the functx package :
> |basex -c" path='https://files.basex.org/modules/expath/functx-1.0.xar'/>"|
>  2. For gigs, I tried to change the protocol from HTTPS to HTTP, and
> here is the error that I get :
>
> basex -c" path='
> http://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar'/>"
> [repo:parse] heimdall.xar: Resource "expath-pkg.xml" not found..
>
> So, maybe there is a problem with my expath-pkg.xml file ?
> I didn't find any -but, as I said, I'm no pro. If needed, here is
> the file that is wrapped in the .xar :
>
> https://gitlab.huma-num.fr/datasphere/heimdall/xquery/-/raw/main/expath-pkg.xml?ref_type=heads
> <
> https://gitlab.huma-num.fr/datasphere/heimdall/xquery/-/raw/main/expath-pkg.xml?ref_type=heads
> >
>  3. However, when I download my .xar file from the URL, and then REPO
> INSTALL it using the local path, stuff works fine :
> wget --no-check-certificate
> https://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar
> basex -c""
> basex -c""
>
> Name Version  Type Path
>
> ---
> http://heimdall.huma-num.fr  2.1  EXPath
> http-heimdall.huma-num.fr-2.1
>
> 1 package(s).
>
> So I suppose the problem /might/ not be within the .xar itself, but the
> way it is hosted ?
> Maybe REPO INSTALL doesn't like my disturbing lack of SSL certificate ?
> I tried to search for some clue, but didn't find any clue in the
> documentation . Maybe I'm blind,
> maybe that's not the problem at all.
>
> It's not exactly a blocking issue, but my noobiness regarding the basex
> ecosystem (and xquery in general) prevents me to understanding what
> happens, my README contains false info, and I'm not at ease with that.
> If someone had any intel, I'd be thankful.
>
> If needed, here is the 

Re: [basex-talk] hof:until is gone?

[Andy Bunce]

Hi Christian,

That hof:until update recipe is very helpful.

>From my occasional reads of qt4 feeds, I guess Graydon's function might
look something  like the below
I like the look of the upcoming V4.0 features.

/Andy

declare function local:dropTableLinesFancy($in as node()*) as node()* {
  do-until(
(: initial input = we start with no new nodes :)
map { 'oldNodes': $in, 'newNodes': (), 'toggle': false()
},
fn($m){ map {
'oldNodes': tail($m?oldNodes),
'newNodes': ($m?newNodes,
switch {
case starts-with(head($m?oldNodes),':stab')
return 
case starts-with(head($m?oldNodes),':rtab')
return 
case $m?toggle return 
default return head($m?oldNodes)
}),
'toggle': switch {
case starts-with(head($m?oldNodes),':stab') return
true()
case starts-with(head($m?oldNodes),':rtab') return
false()
default return $m?toggle
}
}
  },
   (: are we done? (= we've run off the end of our list of nodes ) :)
  fn($m){ empty($m?oldNodes) }
)?newNodes
};

On Thu, 28 Mar 2024 at 18:34, Christian Grün 
wrote:

> Hi Graydon,
>
> Folks tell us it’s time to stop delaying BaseX 11… We’re trying hard.
>
> The good news: The only difference between hof:until [1] and fn:do-until
> [2] is the order of parameters. The following queries will do the same
> thing:
>
> hof:until(function($a) { $a > 16 }, function($a) { $a * 2 }, 1)
> do-until(1, function($a) { $a * 2 }, function($a) { $a > 16 })
>
> With fn:do-until, the input has moved to the first position, the action
> comes second and the predicate comes last.
>
> Hope this helps,
> Christian
>
> [1] https://files.basex.org/releases/10.0/BaseX100.pdf
> [2]
> https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-do-until
>
>
>
> On Thu, Mar 28, 2024 at 5:41 PM Graydon  wrote:
>
>> Hi Andy --
>>
>> On Thu, Mar 28, 2024 at 01:06:10PM +, Andy Bunce scripsit:
>> > hof:until is not gone, it is just hiding  [1]
>> > - In 10.7 it is there but undocumented in Wiki
>>
>> Which is useful to know -- thank you! -- but makes me think a bunch of
>> my production code is going to break hard when 11 is released. I would
>> really like to avoid that. (In particular, the conversation with
>> management about how much refactoring work will be required.)
>>
>> > - In BaseX 11 you need to use XQuery 4  fn:while-do⁺, fn:do-until⁺ [2]
>>
>> It is very probably my brain, but I'm having trouble transposing
>> hof:until to those functions. I can an implement what I want to do with
>> hof:until like so:
>>
>> declare function xc:dropTableLinesFancy($in as node()*) as node()* {
>>   let $result as map(*) :=
>> hof:until(
>>   (: are we done? (= we've run off the end of our list of nodes ) :)
>>   function($m) {
>>  empty($m?oldNodes)
>> },
>>   (: create new list of line elements :)
>>   function($m) {
>>   map {
>> 'oldNodes': tail($m?oldNodes),
>> 'newNodes': ($m?newNodes,
>>   switch (true())
>> case starts-with(head($m?oldNodes),':stab') return 
>> case starts-with(head($m?oldNodes),':rtab') return 
>> case $m?toggle return 
>> default return head($m?oldNodes)
>>   ),
>> 'toggle': (
>>   switch (true())
>> case starts-with(head($m?oldNodes),':stab') return true()
>> case starts-with(head($m?oldNodes),':rtab') return false()
>> default return $m?toggle
>>   )
>>   }
>>   },
>>   (: initial input = we start with no new nodes :)
>>   map { 'oldNodes': $in, 'newNodes': (), 'toggle': false() }
>> )
>>   return $result?newNodes
>> };
>>
>> Anyone willing to provide an example of what that would look like in
>> fn:while-do or fn:do-until?
>>
>> Thanks!
>> Graydon
>>
>>
>> --
>> Graydon Saunders  | graydon...@fastmail.com
>> Þæs oferéode, ðisses swá mæg.
>> -- Deor  ("That passed, so may this.")
>>
>

Re: [basex-talk] hof:until is gone?

[Andy Bunce]

hof:until is not gone, it is just hiding  [1]

   - In 10.7 it is there but undocumented in Wiki
   - In BaseX 11 you need to use XQuery 4  fn:while-do⁺, fn:do-until⁺ [2]

Hope this helps
/Andy

[1]
https://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg15596.html
[2] https://help.basex.org/main/Higher-Order_Functions_Module


On Thu, 28 Mar 2024 at 12:44, Eliot Kimber 
wrote:

> This feels like an ideal use case for XSLT for-each-group—something that
> is challenging to implement in XQuery.
>
>
>
> Cheers,
>
>
>
> E.
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> Digital Content & Design
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com 
>
> LinkedIn  | Twitter
>  | YouTube
>  | Facebook
> 
>
>
>
> *From: *BaseX-Talk  on behalf
> of Graydon Saunders 
> *Date: *Thursday, March 28, 2024 at 7:43 AM
> *To: *BaseX 
> *Subject: *[basex-talk] hof:until is gone?
> *[External Email]*
>
>
> --
>
> Hello --
>
>
>
> So I'm trying to do this:
>
>
>
> import module namespace xcs = "http://www.xcential.com/xquery/utils/script;
> at "same-words-same-order-script.xqm";
> (: we don't need xc computationally but there are external variables in
> that namespace in scope :)
> import module namespace xc = "http://www.xcential.com/xquery/utils; at
> 'same-words-same-order.xqm';
>
> declare function xc:dropTableLines($in as node()*,$toggle as xs:boolean)
> as node()* {
>   switch (true())
> case empty($in) return ()
> case starts-with(head($in),':stab') return
> (,xc:dropTableLines(tail($in),true()))
> case starts-with(head($in),':rtab') return
> (,xc:dropTableLines(tail($in),false()))
> case $toggle return (,xc:dropTableLines(tail($in),$toggle))
> default return (head($in),xc:dropTableLines(tail($in),$toggle))
> };
>
> let $test as element() := 
>   
>   
>   :stab
>   weasels
>   :stab
>   weasels
>   
>   
>   :rtab.
>   
>   asparagus
>   
>   
>   :stab
>   
>   weasels
>   :rtab.
>   asparagus
>   asparagus
>   asparagus
>   asparagus
>   asparagus
>   :stab
>   :stab
>   
>   
>   :stab
>   
>   weasels
>   
>   :rtab.
>   
>   asparagus
>   
>   
>   
> 
>
> return element {'text'} {
> xc:dropTableLines($test/line,false())
>   }
>
>
>
> Only at scale the stack blows up and I get the "try tail recursion?"
> suggestion. I would have tried hof:until for that, since I have to pass the
> current state of "are we dropping or not dropping intervening content?",
> but it looks like it's been removed? And the available hof functions in 4
> look like they're strictly positional which is actively unhelpful in this
> case. (At least with whatever brain cells I currently have.)
>
>
>
> What's the appropriate pattern for "process a sequence, toggling an action
> on or off based on the last member of the sequence we looked at?"
>
>
>
> Thanks!
>
> Graydon
>

Re: [basex-talk] xslt:transform-report result

[Andy Bunce]

Hi,

I have recently been using xslt:transform-report to get xsl:message reports.
I am using BaseX 10.7 with various versions of Saxon-he. I have run example
4 from[1]
with

   - saxon-he-10.9.jar
   - saxon-he-11.6.jar
   - saxon-he-12.4.jar

Only saxon-he-10 captures the messages for me. (although when using BaseX
11 I seem to recall only Saxon 12 captured them)
Is this correct and expected?

I understand the Saxon API may be a moving target.
Perhaps the documentation could indicate a recommended Saxon version for a
given BaseX version, rather than "For the moment, messages can only be
returned with recent versions of Saxon."
I have made minor updates to [2] to this effect. Please edit or revert if
my understanding is incorrect.

/Andy
.
[1] https://docs.basex.org/wiki/XSLT_Module#Examples
[2] https://help.basex.org/main/XSLT_Module


On Wed, 4 May 2022 at 22:16, Christian Grün 
wrote:

> I agree it's somewhat unexpected. As we are working on the string result
> that is returned by Saxon, it's currently not that easy indeed to decide
> how to interpret the character stream.
>
>
>
> Andy Bunce  schrieb am Mi., 4. Mai 2022, 23:11:
>
>> Ok thanks. I thought this might have been unintended behavior.
>> if($report?result instance of document-node()+) then
>> document{$report?result} else $report?result
>> seems to give me what I expected here.
>> /Andy
>>
>>
>> On Wed, 4 May 2022 at 16:58, Christian Grün 
>> wrote:
>>
>>> Thanks, Andy, I’ve updated the documentation.
>>>
>>> On Wed, May 4, 2022 at 3:12 PM Andy Bunce  wrote:
>>>
>>>> Hi,
>>>>
>>>> Using BaseX 9.7.1 and saxon9he-9.9.1.jar
>>>> The documentation suggests the ?result from xslt:transform-report
>>>> should be *a* document-node where possible [1]
>>>> This seems not quite to be the case when there are processing
>>>> instructions or comments at the top level. In these cases a sequence of
>>>> document-nodes is returned.
>>>>
>>>> /Andy
>>>> [1] https://docs.basex.org/wiki/XSLT_Module#xslt:transform-report
>>>>
>>>> let $xslt:=http://www.w3.org/1999/XSL/Transform; version="3.0">
>>>> 
>>>> 
>>>>
>>>> let $xml:=document{ , }
>>>> return xslt:transform-report($xml,$xslt)
>>>>
>>>> Returns
>>>> map {
>>>>   "messages": (),
>>>>   "result": (, )
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>

Re: [basex-talk] Optimize database never returns, leaves database in "opened by another process" state

[Andy Bunce]

Maybe: https://help.basex.org/main/Profiling_Module

/Andy

On Thu, 29 Feb 2024 at 09:40, Eliot Kimber 
wrote:

> Using the latest build, 11.0 beta 17d8426, the prof:dump() function is
> reported as an unknown function.
>
>
>
> What replaces it (or where can I find the V11 docs)?
>
>
>
> Thanks,
>
>
>
> E.
>
>
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com 
>
> LinkedIn  | Twitter
>  | YouTube
>  | Facebook
> 
>
>
>
> *From: *Eliot Kimber 
> *Date: *Thursday, February 29, 2024 at 3:15 AM
> *To: *Christian Grün 
> *Cc: *basex-talk@mailman.uni-konstanz.de <
> basex-talk@mailman.uni-konstanz.de>
> *Subject: *Re: [basex-talk] Optimize database never returns, leaves
> database in "opened by another process" state
>
> Found the latest build at https://files.basex.org/releases/latest/
>
>
>
> Cheers,
>
>
>
> E.
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com 
>
> LinkedIn  | Twitter
>  | YouTube
>  | Facebook
> 
>
>
>
> *From: *Eliot Kimber 
> *Date: *Thursday, February 29, 2024 at 2:45 AM
> *To: *Christian Grün 
> *Cc: *basex-talk@mailman.uni-konstanz.de <
> basex-talk@mailman.uni-konstanz.de>
> *Subject: *Re: [basex-talk] Optimize database never returns, leaves
> database in "opened by another process" state
>
> I’m trying to compile the latest code but “mvn clean install” fails on
> failure to download some dependencies:
>
> [*INFO*] *--< *org.basex:basex*
> >---*
>
> [*INFO*] *Building BaseX Core 11.0-SNAPSHOT*
>
> [*INFO*]   from pom.xml
>
> [*INFO*] *[ jar
> ]-*
>
> Downloading from devsnc-mirror:
> http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.pom
>
> [*WARNING*] The POM for jp.sourceforge.igo:igo:jar:0.4.3 is missing, no
> dependency information available
>
> Downloading from devsnc-mirror:
> http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.pom
>
> [*WARNING*] The POM for org.apache:lucene-stemmers:jar:3.4.0 is missing,
> no dependency information available
>
> Downloading from devsnc-mirror:
> http://nexus.proxy.devsnc.com/content/groups/public/jp/sourceforge/igo/igo/0.4.3/igo-0.4.3.jar
>
> Downloading from devsnc-mirror:
> http://nexus.proxy.devsnc.com/content/groups/public/org/apache/lucene-stemmers/3.4.0/lucene-stemmers-3.4.0.jar
>
> [*INFO*]
> **
>
> [*INFO*] *BUILD FAILURE*
>
> [*INFO*]
> **
>
> [*INFO*] Total time:  5.589 s
>
> [*INFO*] Finished at: 2024-02-29T02:34:11-06:00
>
> [*INFO*]
> **
>
> [*ERROR*] Failed to execute goal on project basex: *Could not resolve
> dependencies for project org.basex:basex:jar:11.0-SNAPSHOT: The following
> artifacts could not be resolved: jp.sourceforge.igo:igo:jar:0.4.3 (absent),
> org.apache:lucene-stemmers:jar:3.4.0 (absent): Could not find artifact
> jp.sourceforge.igo:igo:jar:0.4.3 in devsnc-mirror
> (http://nexus.proxy.devsnc.com/content/groups/public
> )* -> *[Help 1]*
>
>
>
> And then:
>
> [*ERROR*] Failed to execute goal on project basex: *Could not resolve
> dependencies for project org.basex:basex:jar:11.0-SNAPSHOT: The following
> artifacts could not be resolved: jp.sourceforge.igo:igo:jar:0.4.3 (absent),
> org.apache:lucene-stemmers:jar:3.4.0 (absent):
> jp.sourceforge.igo:igo:jar:0.4.3 was not found in
> http://nexus.proxy.devsnc.com/content/groups/public
>  during a previous
> attempt. This failure was cached in the local repository and resolution is
> not reattempted until the update interval of devsnc-mirror has elapsed or
> updates are forced* -> *[Help 1]*
>
>
>
> Where ‘nexus.proxy.devsnc.com’ is our internal Maven proxy.
>
> This is after removing my local maven repository cache.
>
> Before doing that, I got the “present but not available” failure on these
> libraries. The jars are present in my original .m2/repository but not in a
> newly-created repository (having moved the original one out of the way).
>
>
>
> I’m using maven 3.9.6.
>
>
>
> As far as I can tell from my research, this is a change in behavior from
> Maven 2 

Re: [basex-talk] file:write, formatting, and maps in select attributes

[Andy Bunce]

 Hi Graydon,

>I'm trying to regularize the formatting of some XSLT files

I don't think this can or should be done with BaseX serialization. I would
use a dedicated formater/ pretty printer.
For me the sign of a good formatter  is that it tidies random indentation
while preserving what looks intentional.

Phil Fearon has done a lot of great work on formatting XML/XSLT [1]. I use
his Deltaxml Vscode extension [2]
Works for me [3]
If Vscode is not a route you want to take, maybe the older standalone
xmlspectrum [4] would also work (untested).

/Andy

[1] https://github.com/pgfearo
[2] https://marketplace.visualstudio.com/items?itemName=deltaxml.xslt-xpath
[3] https://imgur.com/a/E0388Si
[3] https://github.com/pgfearo/xmlspectrum



On Fri, 23 Feb 2024 at 23:37,  wrote:

> Hi Graydon,
>
> This might not be a serialization but a parsing problem  The XML parser
> performs attribute value normalization as per
> https://www.w3.org/TR/xml/#AVNormalize
>
> I have often wished that there were a parser option to preserve newlines
> in attribute values, in particular for keeping the indentation of multiline
> XPath expressions in XProc or XSLT in a future XProc 1.0 → 3.0 migration
> tool.
>
> For such a migration, the XPath expressions may need modifications, too,
> so I thought they need to be parsed and transformed anyway, and then they
> can be re-serialized with indentation (I haven't tried b/c I'm  writing
> this on the phone, but I strongly believe I can have newlines serialized in
> attribute values).
>
> But in your case a parser option would be useful.
>
> An alternative could  be to apply some text preprocessing to the document.
> For example, read it with unparsed-text, use analyze-string to match
> attribute values (difficult even in moderately complex cases), replace each
> newline with a PUA character and serialize this character as newline using
> a character map later on.
>
> Gerrit
>
> Sent from MailDroid 
>
> -Original Message-
> From: Graydon Saunders 
> To: BaseX 
> Sent: Fri, 23 Feb 2024 22:50
> Subject: [basex-talk] file:write, formatting, and maps in select attributes
>
> Hello --
>
> I'm trying to regularize the formatting of some XSLT files by loading them
> (via doc()) and writing them back out via file:write()
>
> I know about the *indent* and *indent-attributes* serialization
> parameters, and mostly they're doing what I want.
>
> What I'd like is to preserve a one-line-per-entry format for a map in a
> select attribute. For example, I'd like to keep
>
> 
>
> instead of
>
>  name="hexNum"
> select="   map { 1: '0', 2: '1',
>   3: '2', 4: '3', 5: '4', 6: '5', 7: '6',
>   8: '7', 9: '8', 10: '9', 11: 'a', 12:
> 'b', 13: 'c', 14: 'd', 15: 'e', 16: 'f'
>   }"
> static="yes"/>
>
> I don't think there's a way to do this with the serialization parameters,
> but is there a way to do this?
>
> Thanks!
> Graydon
>
> --
> Graydon Saunders  | graydon...@fastmail.com
> Þæs oferéode, ðisses swá mæg.
> -- Deor  ("That passed, so may this.")
>

Re: [basex-talk] hof:until qt4cg drops?

[Andy Bunce]

Hi Christian,

> Hope this helps

Sounds like a good way forward. Sign me up, please.

Digression:
I made a tiny change to the wiki recently concerning the "Q" in expanded
QNames [1] because I thought it was required.
On reflection, this may be because I was using v9 at the time.

I now think the "Q" is required in BaseX  v9, but optional in all V10+.
Is this correct?

/Andy

[1]
https://docs.basex.org/index.php?title=Command-Line_Options=146=16921=16694




On Sat, 3 Feb 2024 at 11:15, Christian Grün 
wrote:

> Hi Andy,
>
> Thanks for the raised finger. Indeed it’s the specification of XQuery 4
> that takes much more time than anticipated. The process is highly dynamic,
> with confirmed features being revised again. Some weeks ago, we have
> stopped updating the old documentation, and we are working on a new one
> (completely based on XQuery, with the contents and history stored in
> databases). It will replace the old one and will be version-based, i.e.,
> we’ll freeze the documentation of BaseX 11 (12, etc.) once it’s released
> and maintain the documentation for the upcoming version in parallel.
>
> As BaseX 11 won’t be finalized until end of this month as originally
> planned, here’s a link to the new documentation, which we’re regularly
> updating:
>
> https://help.basex.org/
>
> The contents will stay, but the rendering is still work in progress. We’ve
> converted the contents to a mixture of XML and contemporary Markdown
> syntax;.editing works similar as before. I invite you (and everyone else)
> to ask for an account to be able to help us keeping the contents up-to-date
> and bug free.
>
> We know ithe current state is a bit unfortunate. For those who are looking
> for a reliable version of the BaseX 10 documentation, we point to the
> archived PDF version:
>
> https://docs.basex.org/wiki/Documentation
>
> Hope this helps,
> Christian
>
>
> Andy Bunce  schrieb am Fr., 2. Feb. 2024, 19:25:
>
>>
>> I wanted to look up the hof:until[1] syntax in the wiki. It tells me that
>> it has been replaced with  fn:iterate-while. Well, I hope, when the dust
>> settles, there is something similar to this in the spec, but it looks like
>> it may not be iterate-while .
>>
>> Today a user of the current BaseX release(10.7) is going to struggle to
>> find what hof:until does and wont find any fn:iterate-while . I think
>> deprecating functions and moving forward is great. I do like to see what
>> 4.0 might bring with the BaseX 11 betas but  if I catch myself writing
>> anything substantial I stop. The ground is unstable.
>>
>> I think the wiki should center on the functionality of the current
>> release, with just teasers about the future.
>> Christian, I think you hinted that the future may hold some documentation
>> API built into the product, or did I imagine that?
>>
>> /Andy
>>
>> [1] https://docs.basex.org/wiki/Higher-Order_Functions_Module,
>>
>

[basex-talk] hof:until qt4cg drops?

[Andy Bunce]

I wanted to look up the hof:until[1] syntax in the wiki. It tells me that
it has been replaced with  fn:iterate-while. Well, I hope, when the dust
settles, there is something similar to this in the spec, but it looks like
it may not be iterate-while .

Today a user of the current BaseX release(10.7) is going to struggle to
find what hof:until does and wont find any fn:iterate-while . I think
deprecating functions and moving forward is great. I do like to see what
4.0 might bring with the BaseX 11 betas but  if I catch myself writing
anything substantial I stop. The ground is unstable.

I think the wiki should center on the functionality of the current release,
with just teasers about the future.
Christian, I think you hinted that the future may hold some documentation
API built into the product, or did I imagine that?

/Andy

[1] https://docs.basex.org/wiki/Higher-Order_Functions_Module,

Re: [basex-talk] RESTXQ oddity?

[Andy Bunce]

In addition to Gerrit's comments...
This line jumped out as problematic  to me.
%rest:path('theocom/static/{$dir=.+}/{$file=.+}')

The pattern =.+ matches the rest of the path. You only need this once at
the end of the path.
Maybe just:
%rest:path('theocom/static/{$dir}/{$file=.+}')

/Andy


On Mon, 29 Jan 2024 at 14:49, Murray, Gregory 
wrote:

> Hello,
>
>
>
> I’m trying to build my first BaseX web application. I’ve got a RESTXQ
> function for returning static files to the browser, and other RESTXQ
> functions corresponding to URLs in the web application. The function for
> static files is based on the one in DBA:
>
>
>
> (:~ Returns the specified static file to the browser as binary data. :)
>
> declare
>
>   %rest:GET
>
>   %rest:path('theocom/static/{$dir=.+}/{$file=.+}')
>
> function c:file($dir as xs:string, $file  as xs:string)
>
> {
>
>   let $path := file:base-dir() || 'static/' || $dir || '/' || $file
>
>   return
>
> (
>
>   web:response-header(
>
> map { 'media-type': web:content-type($path) },
>
> map { 'Content-Length': file:size($path) }
>
>   ),
>
>   file:read-binary($path)
>
> )
>
> };
>
>
>
> Another function returns the home page for the web application. That page
> works just fine. The HTML document gets returned to the browser, and the
> static CSS and image files are sent to the browser successfully.
>
>
>
> (:~ Returns the home page for the entire web application. :)
>
> declare
>
>   %rest:GET
>
>   %rest:path('theocom')
>
>   %output:method('html')
>
>   %output:html-version('5')
>
> function c:home()
>
> {
>
>   let $params := c:http-params()
>
>   return
>
> (
>
>   c:extra-response-headers(),
>
>   home:get-home-page($params)
>
> )
>
> };
>
>
>
> I have another RESTXQ function that corresponds to a URL that goes deeper
> into the web application:
>
>
>
> (:~ Returns a page listing contributors (libraries whose content we've
> imported), for browsing. :)
> declare
>   %rest:GET
>   %rest:path('theocom/contributors/{$alpha=[a-z]}')
>   %output:method('html')
>   %output:html-version('5')
> function *c:browse-contributors*($alpha as xs:string)
> {
>   let $params :=
> *map:merge*(
>   (
> *map:entry*("browse", "contributor"),
> *map:entry*("alpha", *upper-case*($alpha)),
> *c:http-params*()
>   )
> )
>   return
> (
>   *c:extra-response-headers*(),
>   *browse:contributors*($params)
> )
> };
>
>
>
> The problem I’m having is that on this page, the browser returns 404 for
> the static files, even though the code for the HTML  where the CSS
> files are linked is identical to the home page. Am I doing something wrong,
> or is the BaseX RESTXQ implementation doing something unexpected?
>
>
>
> Thanks,
>
> Greg
>
>
>
>
>
> Gregory Murray
>
> Director of Digital Initiatives
>
> Wright Library
>
> Princeton Theological Seminary
>
>
>
>
>

Re: [basex-talk] GUI feature request: paste into search boxes

[Andy Bunce]

Hi Tim,

You don't say what OS or version but on Windows it is true the search boxes
have no right click context menu, but they do support Ctl-V.

/Andy



On Wed, 17 Jan 2024 at 17:03, Thompson, Timothy 
wrote:

> Hello,
>
>
>
> In the BaseX GUI, it’s not possible (as far as I can tell) to paste a
> copied value into search boxes. Not being able to do so can be
> inconvenient, for example, when searching for a string like a UUID in
> result output, etc.
>
>
>
> Is lack of support for pasting intentional, or am I missing something?
>
>
>
> Thanks in advance,
>
> Tim
>
>
>
>
>
> --
> Tim A. Thompson (*he, him*)
> Librarian for Applied Metadata Research
> Yale University Library
> www.linkedin.com/in/timathompson
>
>
>

[basex-talk] Debugging XML catalog with BaseX 10.7

[Andy Bunce]

Hi,

I am following the guide at [1] with the aim of loading some XML with a
variety of DTD doctypes.

I have a catalog.xml set (which works in another app) and in lib/custom I
have

   - saxon-he-10.9.jar
   - xmlresolver-5.2.2-data.jar
   - xmlresolver-5.2.2.jar
   - xmlresolver.properties

It is not working as I hoped. Is there a way to get debug info from
xmlresolver to a file without going to the Java level?

The guide says "You must configure your environment with an appropriate
logging backend." [2]
I guess this is about SLF4J  but I have no
experience here.

/Andy

[1] https://docs.basex.org/wiki/Catalog_Resolver
[2] https://xmlresolver.org/ch06#xml.catalog.defaultLoggerLogLevel

Re: [basex-talk] The duration of a cached job

[Andy Bunce]

 Works for me. Thanks.
/Andy

On Tue, 21 Nov 2023 at 15:33, Christian Grün 
wrote:

> Hi Andy,
>
> Thanks for the observation. I’ve fixed the problem (backed by an
> additional JUnit test), and a new snapshot is online [1].
>
> Best,
> Christian
>
> [1] https://files.basex.org/releases/latest/
>
>
> On Mon, Nov 20, 2023 at 12:36 PM Andy Bunce  wrote:
>
>> Hi,
>>
>> Back in the winter of 2017, BaseX 8.6 , I reported an issue with job
>> duration [1]
>>
>> A question about jobs:list-details[2]
>> I had expected the attribute duration: evaluation time (for running and
>> cached jobs) to be frozen once the job is cached.
>>
>>
>> Using BaseX 10.7 it seems this issue is back
>> ```
>>  job:eval("42",(),map{"cache":true()})
>> job24
>>
>>job:list-details("job24")
>> > duration="PT39.031S"
>>  reads="(none)" writes="(none)" time="2023-11-20T11:23:44.133Z">42
>>
>>job:list-details("job24")
>> > duration="PT2M35.894S"
>> reads="(none)" writes="(none)" time="2023-11-20T11:23:44.133Z">42
>> ```
>> /Andy
>>
>> [1]
>> https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg09036.html
>> [2] https://docs.basex.org/wiki/Job_Module#job:list-details
>> <https://docs.basex.org/wiki/Job_Module#job:list-details>
>>
>

[basex-talk] BaseX 10.7 and util:if

[Andy Bunce]

Hi,
The documentation for the utility module [1] says certain functions will be
removed in version 11 because they will be in XPath 4.0.
However, util:if has been removed from the documentation and I think there
is no equivalent function in this case.
util:if is an alternative  syntax for the BaseX Ternary operator  [2].

( I have used it because it was easier than trying to get my XQuery parser
EBNF to accept  $ok ?? 1 !! 0 etc)

Is util:if to be removed?

/Andy
[1] https://docs.basex.org/wiki/Utility_Module#util:if
[2] https://docs.basex.org/wiki/XQuery_Extensions#Ternary_If

[basex-talk] The duration of a cached job

[Andy Bunce]

Hi,

Back in the winter of 2017, BaseX 8.6 , I reported an issue with job
duration [1]

A question about jobs:list-details[2]
I had expected the attribute duration: evaluation time (for running and
cached jobs) to be frozen once the job is cached.


Using BaseX 10.7 it seems this issue is back
```
 job:eval("42",(),map{"cache":true()})
job24

   job:list-details("job24")
42

   job:list-details("job24")
42
```
/Andy

[1]
https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg09036.html
[2] https://docs.basex.org/wiki/Job_Module#job:list-details

Re: [basex-talk] Accessing BaseX server on remote machine

[Andy Bunce]

 >I saw no way to connect remotely using that .bat file
Option -n for example
-nserver.basex.org
See table athttps://docs.basex.org/wiki/Command-Line_Options#Client

/Andy

On Thu, 17 Aug 2023, 14:07 Furst, Thomas,  wrote:

> I can ping the remote machine, and RDP into the machine and use BaseX
> locally there.
>
>
>
> I connected using basexclient.bat on the remote machine. I saw no way to
> connect remotely using that .bat file.
>
>
>
> Version of BaseX is 10.6
>
>
>
> *Tom Furst*
>
> Developer, S1000D SME
>
>
>
> *From:* Andy Bunce 
> *Sent:* Wednesday, August 16, 2023 7:21 PM
> *To:* Furst, Thomas 
> *Cc:* BaseX 
> *Subject:* Re: [basex-talk] Accessing BaseX server on remote machine
>
>
>
> Hi Tom,
>
>
>
> Can you ping the remote machine?
>
> Can you connect using  basexclient.bat?
>
> Which version of basex?
>
>
>
> /Andy
>
>
>
> On Wed, 16 Aug 2023, 22:00 Furst, Thomas,  wrote:
>
> Never got any response on this. Does anyone have any help they can provide?
>
>
>
> *From:* BaseX-Talk  *On
> Behalf Of *Furst, Thomas
> *Sent:* Tuesday, July 18, 2023 7:36 AM
> *To:* basex-talk@mailman.uni-konstanz.de
> *Subject:* [basex-talk] Accessing BaseX server on remote machine
>
>
>
> Hello,
>
>
>
> I am attempting to use BaseX from a central location on our network so
> that all team members can access it. I am using the Python Client to
> achieve this. Everything works fine when the host is entered as
> 'localhost'. Once I change it to the remote host name, I am getting a time
> out error reported back:
>
> TimeoutError: [WinError 10060] A connection attempt failed because the
> connected party did not properly respond after a period of time, or
> established connection failed because connected host has failed to respond
>
>
>
> On the remote machine (Windows 10 Pro for Workstations) I have opened port
> 1984 in the firewall on both the inbound and outbound. Can someone help me
> understand what I may be doing incorrectly? Or what I may have overlooked
> doing that is also required?
>
>
>
> Thank you,
>
>
>
> Tom Furst
>
>

Re: [basex-talk] Accessing BaseX server on remote machine

[Andy Bunce]

Hi Tom,

Can you ping the remote machine?
Can you connect using  basexclient.bat?
Which version of basex?

/Andy

On Wed, 16 Aug 2023, 22:00 Furst, Thomas,  wrote:

> Never got any response on this. Does anyone have any help they can provide?
>
>
>
> *From:* BaseX-Talk  *On
> Behalf Of *Furst, Thomas
> *Sent:* Tuesday, July 18, 2023 7:36 AM
> *To:* basex-talk@mailman.uni-konstanz.de
> *Subject:* [basex-talk] Accessing BaseX server on remote machine
>
>
>
> Hello,
>
>
>
> I am attempting to use BaseX from a central location on our network so
> that all team members can access it. I am using the Python Client to
> achieve this. Everything works fine when the host is entered as
> 'localhost'. Once I change it to the remote host name, I am getting a time
> out error reported back:
>
> TimeoutError: [WinError 10060] A connection attempt failed because the
> connected party did not properly respond after a period of time, or
> established connection failed because connected host has failed to respond
>
>
>
> On the remote machine (Windows 10 Pro for Workstations) I have opened port
> 1984 in the firewall on both the inbound and outbound. Can someone help me
> understand what I may be doing incorrectly? Or what I may have overlooked
> doing that is also required?
>
>
>
> Thank you,
>
>
>
> Tom Furst
>

Re: [basex-talk] Whitespaces not preserved in TEI documents.

[Andy Bunce]

Perhaps this addition would be more findable if moved to the STRIPWS
description?

/Andy


On Wed, 5 Jul 2023 at 16:54, Andy Bunce  wrote:

> Hi Christian,
>
> You are right about Saxon. The documentation appeared, to me, to suggest
> the default was preserve, and I didn't check!
> The documentation is now very clear[1].
>
> "The default is strip, in accordance with Appendix C.1 of the XQuery
> specification."
>
> I have added a few words to the Wiki [2]
>
> Thanks
>
> /Andy
> [1]
> https://www.saxonica.com/documentation12/index.html#!conformance/xquery31
> [2] https://docs.basex.org/wiki/BaseX_10#Whitespaces
>
> On Sat, 1 Jul 2023 at 07:41, Christian Grün 
> wrote:
>
>> Thanks for the suggestions. I agree that XML whitespace handling will
>> always be a challenge and challenging to grasp… No matter what defaults are
>> used in an implementation. It’s true, our documentation could provide more
>> detail about this; maybe we can spend more time on that in the future. All
>> edits are welcome ;)
>>
>> I had a look at what Saxon does [2] (item 32). It looks like the Saxon
>>> boundary-space default is preserve.
>>>
>>
>> I believe that Saxon strips boundary space by default as well. At least
>> that’s my command-line experience (I didn’t check how that correlates to
>> the information given in their documentation):
>>
>> (: query.xq :)
>>  
>>
>> (: call :)
>> java -cp "saxon-he-12.2.jar;xmlresolver-5.1.1.jar" net.sf.saxon.Query
>> query.xq
>>
>> (: result :)
>> 
>>
>> Best,
>> Christian
>>
>>

Re: [basex-talk] Whitespaces not preserved in TEI documents.

[Andy Bunce]

Hi Christian,

You are right about Saxon. The documentation appeared, to me, to suggest
the default was preserve, and I didn't check!
The documentation is now very clear[1].

"The default is strip, in accordance with Appendix C.1 of the XQuery
specification."

I have added a few words to the Wiki [2]

Thanks

/Andy
[1]
https://www.saxonica.com/documentation12/index.html#!conformance/xquery31
[2] https://docs.basex.org/wiki/BaseX_10#Whitespaces

On Sat, 1 Jul 2023 at 07:41, Christian Grün 
wrote:

> Thanks for the suggestions. I agree that XML whitespace handling will
> always be a challenge and challenging to grasp… No matter what defaults are
> used in an implementation. It’s true, our documentation could provide more
> detail about this; maybe we can spend more time on that in the future. All
> edits are welcome ;)
>
> I had a look at what Saxon does [2] (item 32). It looks like the Saxon
>> boundary-space default is preserve.
>>
>
> I believe that Saxon strips boundary space by default as well. At least
> that’s my command-line experience (I didn’t check how that correlates to
> the information given in their documentation):
>
> (: query.xq :)
>  
>
> (: call :)
> java -cp "saxon-he-12.2.jar;xmlresolver-5.1.1.jar" net.sf.saxon.Query
> query.xq
>
> (: result :)
> 
>
> Best,
> Christian
>
>

Re: [basex-talk] Whitespaces not preserved in TEI documents.

[Andy Bunce]

Glad to have helped. Whitespace is important
...and hard.

I think the Wiki (https://docs.basex.org) could help more here. Maybe with
a "whitespace "page or/and a page listing implementation-defined [1]
settings.
I had a look at what Saxon does [2] (item 32). It looks like the Saxon
boundary-space default is preserve.

Perhaps that default would be more in line with user expectations
especially now the notorious CHOP has gone ;-)

/Andy

[1] https://www.w3.org/TR/xquery-31/#dt-implementation-defined
[2]
https://www.saxonica.com/documentation12/index.html#!conformance/xquery31

On Fri, 30 Jun 2023 at 12:31, Montserrat Matías  wrote:

> Hi Andy.
>
>
>
> It was not a bug!
>
> I was totally wrong.
>
>
>
> I didn't know about that feature.
>
> I have a lot to learn...
>
> Your proposed solution works fine.
>
>
>
> Thank you very much for your quick response.
>
>
>
> Regards,
>
> Montse.
>
>
>
> *De:* Andy Bunce 
> *Enviado el:* jueves, 29 de junio de 2023 12:07
> *Para:* Montserrat Matías 
> *CC:* basex-talk@mailman.uni-konstanz.de
> *Asunto:* Re: [basex-talk] Whitespaces not preserved in TEI documents.
>
>
>
> *ATENCIÓN*: Este es un mensaje externo originado fuera de la RAE. Por
> favor, no haga clic en enlaces ni abra archivos adjuntos a menos que
> reconozca al remitente y sepa que el contenido es seguro.
> --
>
> Hi Montse,
>
>
>
> I believe the current behavior is correct. The important feature of your
> example is that the XML is constructed in the XQuery source.
>
> In this case the, somewhat obscure, boundary-space setting applies [1].
> This has been discussed on the list [2]
>
>
>
> Your example works for me if the boundary-space preserve setting is
> added...
>
> ```
>
> declare boundary-space preserve;
> let $doc := Gente grosera o
> vulgar.La
> chusma.
>
> return $doc/entry//sense[@id = '1']
>
> ```
>
> Best regards
>
> /Andy
>
>
>
> [1] https://www.w3.org/TR/xquery-31/#id-boundary-space-decls
>
> [2]
> https://mailman.uni-konstanz.de/pipermail/basex-talk/2016-September/011251.html
>
>
>
>
>
> On Thu, 29 Jun 2023 at 09:38, Montserrat Matías  wrote:
>
> Hello.
>
>
>
> I have a bug with BaseX version 10 and higher related to handling
> whitespaces in documents with mixed content elements.
>
>
>
> I'm working with narrative documents (TEI).
>
> Following the instructions in the documentation about this type of
> documents (containing mixed elements) I have disabled the indentation and
> the STRIPWS option.
>
>
>
> When I execute queries with connection to the database, the result is as
> explained in the documentation (blanks are preserved).
>
>
>
> However, when I use the BaseX libraries to execute queries in a local
> context (without a connection to the database) the spaces are sometimes
> removed.
>
> Same error if I run the query in the Database Administration interface.
>
>
>
> Query:
>
> db:option('stripws')
>
>
>
> Result:
>
> false
>
>
>
> Query:
>
> db:option('serializer')
>
>
>
> Result:
>
>
> map{"omit-xml-declaration":"yes","binary":"yes","method":"basex","use-character-maps":"","tabulator":"no","allow-duplicate-names":"no","media-type":"","doctype-public":"","escape-uri-attributes":"no","standalone":"omit","csv":map{"lax":true(),"backslashes":false(),"separator":"comma","allow":"","header":false(),"quotes":true(),"format":"direct"},"indents":2,"json-node-output-method":"xml","json":map{"escape":true(),"strings":false(),"lax":false(),"indent":(),"format":"direct","merge":false()},"doctype-system":"","item-separator":(),"indent":"no","suppress-indentation":"","byte-order-mark":"no","include-content-type":"yes","encoding":"UTF-8","newline":"\n","normalization-form":"none","html-version":"","version":"","limit":-1,"undeclare-prefixes":"no","cdata-section-elements":"","parameter-document":""}
>
>
>
> Query:
>
> let $doc := Gente grosera o
> 

Re: [basex-talk] Whitespaces not preserved in TEI documents.

[Andy Bunce]

Hi Montse,

I believe the current behavior is correct. The important feature of your
example is that the XML is constructed in the XQuery source.
In this case the, somewhat obscure, boundary-space setting applies [1].
This has been discussed on the list [2]

Your example works for me if the boundary-space preserve setting is added...
```
declare boundary-space preserve;
let $doc := Gente grosera o
vulgar.La
chusma.

return $doc/entry//sense[@id = '1']
```
Best regards
/Andy

[1] https://www.w3.org/TR/xquery-31/#id-boundary-space-decls
[2]
https://mailman.uni-konstanz.de/pipermail/basex-talk/2016-September/011251.html


On Thu, 29 Jun 2023 at 09:38, Montserrat Matías  wrote:

> Hello.
>
>
>
> I have a bug with BaseX version 10 and higher related to handling
> whitespaces in documents with mixed content elements.
>
>
>
> I'm working with narrative documents (TEI).
>
> Following the instructions in the documentation about this type of
> documents (containing mixed elements) I have disabled the indentation and
> the STRIPWS option.
>
>
>
> When I execute queries with connection to the database, the result is as
> explained in the documentation (blanks are preserved).
>
>
>
> However, when I use the BaseX libraries to execute queries in a local
> context (without a connection to the database) the spaces are sometimes
> removed.
>
> Same error if I run the query in the Database Administration interface.
>
>
>
> Query:
>
> db:option('stripws')
>
>
>
> Result:
>
> false
>
>
>
> Query:
>
> db:option('serializer')
>
>
>
> Result:
>
>
> map{"omit-xml-declaration":"yes","binary":"yes","method":"basex","use-character-maps":"","tabulator":"no","allow-duplicate-names":"no","media-type":"","doctype-public":"","escape-uri-attributes":"no","standalone":"omit","csv":map{"lax":true(),"backslashes":false(),"separator":"comma","allow":"","header":false(),"quotes":true(),"format":"direct"},"indents":2,"json-node-output-method":"xml","json":map{"escape":true(),"strings":false(),"lax":false(),"indent":(),"format":"direct","merge":false()},"doctype-system":"","item-separator":(),"indent":"no","suppress-indentation":"","byte-order-mark":"no","include-content-type":"yes","encoding":"UTF-8","newline":"\n","normalization-form":"none","html-version":"","version":"","limit":-1,"undeclare-prefixes":"no","cdata-section-elements":"","parameter-document":""}
>
>
>
> Query:
>
> let $doc := Gente grosera o
> vulgar.La
> chusma.
>
> return $doc/entry//sense[@id = '1']
>
>
>
> Result:
>
> Gente grosera o
> vulgar.Lachusma.
>
>
>
> The whitespace contained between the  and  tags has been
> stripped.
>
>
>
> In version 9.3.5 this problem did not occur.
>
>
>
> Can you please have a look at this? In a TEI document whitespaces are
> important.
>
>
>
> Thanx in advance,
>
> Montse.
>
> ___
>
> NOTA LEGAL
>
> El contenido de este mensaje de correo electrónico, incluidos los ficheros
> adjuntos, es confidencial y está protegido por el artículo 18.3 de la
> Constitución española, que garantiza el secreto de las comunicaciones. Si
> usted recibe este mensaje por error, le rogamos que se ponga en contacto
> con el remitente para informarle de este hecho y no difunda su contenido ni
> haga copias. La *Real Academia Española* informa de que los datos que en
> esta comunicación figuran, así como los que mantiene de usted y/o de su
> empresa, son tratados con la finalidad de mantener el contacto, así como de
> realizar las gestiones que en esta aparecen, y son utilizados de forma
> autorizada por las partes y sin cederse a terceros ajenos. Puede ejercer
> sus derechos a través de proteccion_de_da...@rae.es. Puede obtener más
> información sobre protección de datos en nuestra página web
>  o contactando directamente con nosotros
> (Reglamento UE 2016/679).
>

[basex-talk] Minor GUI irritation

[Andy Bunce]

Hi,
In the GUI when viewing an open database with many namespaces via database
-> properties->information.
The properties window is excessively tall, often larger than the screen
height.
There seems to be extra blank space below the "Resource Properties" section
and perhaps the namespaces section that could have a scroll bar if required.

db:create("nstest",(1 to 60)!``[]``,(1 to 60)!(. ||
".xml"))

Creates a database that shows the behavior on my screen.
Seen with BaseXGUI 10.6 on windows, but I think I also saw it with v9.

Regards
/Andy
[image: image.png]

Re: [basex-talk] Fwd: FW: java binding passing variables

[Andy Bunce]

Glad it works for you. Regarding the 4 arguments I think this is a version
thing. I downloaded v1.6 which can take 1, 3 or 4 arguments [1]

public String 
<https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/String.html?is-external=true>
toUnicode(String
<https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/String.html?is-external=true>
str,
List
<https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/List.html?is-external=true>https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/String.html?is-external=true>>
warns,
boolean sloppy,
boolean lenient)

Below is a slightly cleaner version of the code. The BaseX wiki has a lot
of good information on Java bindings [2]

import module namespace bdrc = "java:io.bdrc.ewtsconverter.EwtsConverter";
declare namespace list = "java:java.util.ArrayList";

let $roman := "bk ra shi se bde hh legs"

let $warns:=list:new()
let $result:=bdrc:toUnicode($roman,$warns,false())
let $warns:=list:toArray($warns )
return map{"result": $result,
   "warn": $warns
 }

/Andy

[1]
https://github.com/buda-base/ewts-converter/blob/master/src/main/java/io/bdrc/ewtsconverter/EwtsConverter.java#L1539
[2] https://docs.basex.org/wiki/Java_Bindings

On Wed, 12 Apr 2023 at 09:41, Burkhard Quessel  wrote:

> Hi Andy,
>
> When I tried to run the code, Basex complained about the 3rd argument in
> bdrc:toUnicode($roman,$warn,true(),false())
> “4 arguments supplied, 1 or 3 expected”
>
> With the 3rd argument removed it works a treat and is going to save me a
> lot of time. I have thousands of records to check and your reply really
> saved my bacon.
>
> Thank you ever so much,
>
>
>
> Burkhard
>
>
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows
>
>
>
> *From: *Andy Bunce 
> *Sent: *11 April 2023 19:54
> *To: *Burkhard Quessel 
> *Cc: *basex-talk@mailman.uni-konstanz.de
> *Subject: *Re: [basex-talk] Fwd: FW: java binding passing variables
>
>
>
> Hi Burkhard,
>
>
>
> The thing about the warn argument is that it can be updated by the Java
> call. So I think you will need to pass in a suitable Java object.
>
> For this case I tried an  ArrayList
>
>
>
> import module namespace bdrc = "io.bdrc.ewtsconverter.EwtsConverter";
> let $warn:=Q{java.util.ArrayList}new()
> let $roman := "bk ra shise bde legs"
> let $result:=bdrc:toUnicode($roman,$warn,true(),false())
> return map{"result": $result,
>"warn": Q{java.util.ArrayList}toString($warn)
>  }
>
>
>
> Results in
>
> map {
>   "warn": "[line 1: ""bk"": Expected vowel after ""k"".]",
>   "result": "བཀ་ར་ཤིསེ་བདེ་ལེགས"
> }
>
>
>
> I don't know if this is right or not. There are better ways to pick apart
> $warn.
>
>
>
> /Andy
>
>
>
>
>
>
>
>
>
> On Tue, 11 Apr 2023 at 13:16, Burkhard Quessel  wrote:
>
> *Subject:* java binding passing variables
>
>
>
> I am trying to use xquery to transform Tibetan which is presented in Roman
> script to Unicode, i.e. into original Tibetan script. For this I use
> existing java code from here:
>
>
> https://github.com/buda-base/ewts-converter/blob/master/src/main/java/io/bdrc/ewtsconverter/EwtsConverter.java
>
>
>
> I add the jar of this code (
> https://jar-download.com/artifact-search/ewts-converter)  to my basex
> library and import like this:
>
> import module namespace bdrc = "io.bdrc.ewtsconverter.EwtsConverter";
>
> I can then use a java method to convert my Romanised Tibetan text to
> Unicode Original Tibetan script like so:
>
>
>
> let $roman := "bkra shis bde legs"
>
> return
>
> bdrc:toUnicode($roman)
>
>
>
> so far it works perfectly:
>
> བཀྲ་ཤིས་བདེ་ལེགས
>
>
>
> My  problems begin when I try to add additional parameters to the
> conversion.
>
>
>
>  The java documentation lists a number of possible parameters:
>
> *toUnicode(String str, List warns, boolean sloppy, boolean
> lenient)*
>
> (I want the “warns” one) but I am just too stupid to figure out how
> exactly to include this in in my  xquery code. Can anyone help out?
>
>
>
>  Thanks Burkhard
>
>
>
>
>
>

Re: [basex-talk] Fwd: FW: java binding passing variables

[Andy Bunce]

Hi Burkhard,

The thing about the warn argument is that it can be updated by the Java
call. So I think you will need to pass in a suitable Java object.
For this case I tried an  ArrayList

import module namespace bdrc = "io.bdrc.ewtsconverter.EwtsConverter";
let $warn:=Q{java.util.ArrayList}new()
let $roman := "bk ra shise bde legs"
let $result:=bdrc:toUnicode($roman,$warn,true(),false())
return map{"result": $result,
   "warn": Q{java.util.ArrayList}toString($warn)
 }

Results in
map {
  "warn": "[line 1: ""bk"": Expected vowel after ""k"".]",
  "result": "བཀ་ར་ཤིསེ་བདེ་ལེགས"
}

I don't know if this is right or not. There are better ways to pick apart
$warn.

/Andy




On Tue, 11 Apr 2023 at 13:16, Burkhard Quessel  wrote:

> *Subject:* java binding passing variables
>
>
>
> I am trying to use xquery to transform Tibetan which is presented in Roman
> script to Unicode, i.e. into original Tibetan script. For this I use
> existing java code from here:
>
>
> https://github.com/buda-base/ewts-converter/blob/master/src/main/java/io/bdrc/ewtsconverter/EwtsConverter.java
>
>
>
> I add the jar of this code (
> https://jar-download.com/artifact-search/ewts-converter)  to my basex
> library and import like this:
>
> import module namespace bdrc = "io.bdrc.ewtsconverter.EwtsConverter";
>
> I can then use a java method to convert my Romanised Tibetan text to
> Unicode Original Tibetan script like so:
>
>
>
> let $roman := "bkra shis bde legs"
>
> return
>
> bdrc:toUnicode($roman)
>
>
>
> so far it works perfectly:
>
> བཀྲ་ཤིས་བདེ་ལེགས
>
>
>
> My  problems begin when I try to add additional parameters to the
> conversion.
>
>
>
>  The java documentation lists a number of possible parameters:
>
> *toUnicode(String str, List warns, boolean sloppy, boolean
> lenient)*
>
> (I want the “warns” one) but I am just too stupid to figure out how
> exactly to include this in in my  xquery code. Can anyone help out?
>
>
>
>  Thanks Burkhard
>
>
>

Re: [basex-talk] Automated testing for XML databases

[Andy Bunce]

Hi Shuxin Li,

It certainly is very impressive work and I am also keen to know more.

I see you have also been working with eXist [1]
I assume it builds on "Query Plan Guidance (QPG), a test case generation
method with the guidance of query plans."?  [2]

Best wishes
/Andy

[1] https://github.com/eXist-db/exist/issues/created_by/Twilight-Shuxin
[2] https://zenodo.org/record/7623847



On Thu, 30 Mar 2023 at 10:06, Christian Grün 
wrote:

> Dear Shuxin Li,
>
> I’m impressed by how you repeatedly manage to discover new bugs in
> BaseX. How do you proceed to uncover the errors? Is it a
> trial-and-error approach, or are the tests generated automatically? Do
> you/does your group plan to write an article that describes your
> approach?
>
> All the best from Germany,
> Christian
>
>
>
> On Fri, Mar 17, 2023 at 10:22 AM Christian Grün
>  wrote:
> >
> > Dear Shuxin Li,
> >
> > Thanks for sharing the bug reports with us. I appreciate that, they’re
> definitely helpful.
> >
> > You could as well think about creating and sending pull requests for
> your test cases to the official W3 test suite [1]. This way, all
> implementors of the standard who run the tests may benefit from your
> observations.
> >
> > I'm looking forward to the upcoming reports.
> >
> > Thanks again,
> > Christian
> >
> > [1] https://github.com/w3c/qt3tests
> >
> >
> >
> >
> >
> > Shuxin Li  schrieb am Fr., 17. März 2023, 10:01:
> >>
> >> Greetings!
> >>
> >> I'm Shuxin, an undergraduate CS student, who is currently an intern at
> NUS Test lab. I’ve been working on an automated testing tool for XML
> processors and databases. My goal is to find bugs in important systems, in
> order to improve these systems’ reliability. In the past few days, I
> submitted a few bug reports on Github (Github id: Twilight-Shuxin). Thanks
> for having already fixed these!
> >>
> >> I would like to ask whether you find these reports useful and would
> also appreciate future reports if I find more bugs? Any feedback for us is
> highly appreciated.  We also plan to open-source this tool for public usage
> after it's fully realized, and truly hope that it might be of help to the
> community. Thanks!
> >>
> >> Best Regards,
> >> Shuxin Li
>

[basex-talk] DBA: upload of database backups

[Andy Bunce]

Hi,

Since around 9.7 the DBA has an option to upload backup zip files.
However it appears the backups are uploaded to data/.dba rather than data/
so can not be restored via the DBA UI.
Maybe the problem is here [1]

/Andy
[1]
https://github.com/BaseXdb/basex/blob/main/basex-api/src/main/webapp/dba/databases/backups/backup-upload.xqm#L27-L28

Re: [basex-talk] xqDoc - potential updates

[Andy Bunce]

Hi Loren,

>I am looking at maybe doing some updates to the xqDoc spec and the
codebase.

I have some suggestions for additions to the xqDoc schema:

1. Add optional boolean attribute 'external' to xqDoc:function and xqDoc:
variable. I see the main use for this as allowing documentation to show
clearly which variables can be passed in when an XQuery main module is
invoked.

2.  Add additional optional children to the variable element.
Namely: body, invoked and ref-variable.
These children are defined for functions. Declared variables can have
complex code which references other elements. These changes allow the cross
reference information to be complete.

3. Add optional attribute 'at' to import element. Maybe controversial but
if it is in the source I think the doc should be able to capture it.

 I think I could get a pull request for these schema changes together in a
couple of weeks if you like?

/Andy

Re: [basex-talk] Very niche question about BaseX 9.5.1 and java bindings

[Andy Bunce]

Got it I think...
props:store·java.io.Writer·*java.lang.String*(
Q{java:java.io.FileWriter}new($source), "The saucer has landed")

On Mon, 22 Aug 2022 at 15:11, Andy Bunce  wrote:

> Hi,
>
> I am trying to write a Java property file using v9.5.1 and no new Java
> code.
> The code below runs in 9.7.3. Although it only uses 9.5 features in 9.5.1
> it produces the error message
>
> props:store·java.io.Writer·String cannot be called with (Writer, String).
> but
> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store-java.io.Writer-java.lang.String-
>
> I assume this is a 9.5.1 java binding bug?
> Any workaround suggestions or details of what the problem is?
> /Andy
>
> ---
> (:~  Using Basex 9.5.1
> and assuming a file  "test.properties" in folder containing this module:)
> import module namespace props = "java:java.util.Properties";
> declare variable $source:=
> file:resolve-path("test.properties",file:base-dir())
>
>  !(prof:void(props:load·java.io.Reader(Q{java:java.io.FileReader}new(.))),.)
>
> ;
>
> props:store·java.io.Writer·String( Q{java:java.io.FileWriter}new($source),
> "The saucer has landed")
>

[basex-talk] Very niche question about BaseX 9.5.1 and java bindings

[Andy Bunce]

Hi,

I am trying to write a Java property file using v9.5.1 and no new Java code.
The code below runs in 9.7.3. Although it only uses 9.5 features in 9.5.1
it produces the error message

props:store·java.io.Writer·String cannot be called with (Writer, String).
but
https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store-java.io.Writer-java.lang.String-

I assume this is a 9.5.1 java binding bug?
Any workaround suggestions or details of what the problem is?
/Andy

---
(:~  Using Basex 9.5.1
and assuming a file  "test.properties" in folder containing this module:)
import module namespace props = "java:java.util.Properties";
declare variable $source:=
file:resolve-path("test.properties",file:base-dir())

 !(prof:void(props:load·java.io.Reader(Q{java:java.io.FileReader}new(.))),.)

;

props:store·java.io.Writer·String( Q{java:java.io.FileWriter}new($source),
"The saucer has landed")

Re: [basex-talk] changes in BaseX-10.0 w.r.t. http:send-request() / fetch:doc() functions?

[Andy Bunce]

Great!
I just spotted that the home page was failing in the same way. It has no
redirect.
/Andy

  let $uri := 'http://vocab.getty.edu/'
  return http:send-request()

On Fri, 5 Aug 2022 at 10:13, Ron Van den Branden 
wrote:

> I'm stunned, thanks so much!
>
> Best,
>
> Ron
> On 5/08/2022 11:05, Christian Grün wrote:
>
> This is what we found out (by the help of Wireshark, and some online
> resources):
>
> • The new JDK HTTP Client does not attach a default "Accept" header to the
> HTTP Request.
> • The getty.edu web server (Tomcat?) returns a syntax error when this
> header is missing in the request.
> • We also had a look at the 303 redirection. It works fine; with BaseX 10,
> redirection could even be improved, as protocol changes (http → https) are
> now supported, too.
>
> A new snapshot with a workaround is online [1,2].
>
> Thanks for the observation.
> Christian
>
> [1] https://github.com/BaseXdb/basex/issues/2133
> [2] https://files.basex.org/releases/latest/
>
>
>
> On Fri, Aug 5, 2022 at 9:22 AM Ron Van den Branden <
> ron.vdbran...@gmail.com> wrote:
>
>> Hi,
>>
>> Thanks for chiming in, Andy! I realized yesterday that I should have
>> added how some URLs can be retrieved without problem in BaseX-10.0, e.g.:
>>
>>   let $uri := 'https://www.w3.org'
>>   return http:send-request(> href="{$uri}"/>)
>>
>> ...which is well-formed (to rule out non-XML parser issues), and indeed
>> has no redirection, which seems consistent with Andy's observation. Yet,
>> https://w3.org also is retrieved successfully, which has an initial 301
>> response (instead of 303).
>>
>> Best,
>>
>> Ron
>> On 4/08/2022 18:34, Andy Bunce wrote:
>>
>> There seems to be a 303 redirect. Maybe this is relevant
>> https://stackoverflow.com/a/66325588/3210344
>> /Andy
>> [image: image.png]
>>
>>
>>
>> On Thu, 4 Aug 2022 at 16:19, Christian Grün 
>> wrote:
>>
>>> What I have assessed so far is that it’s the Java Client that fails to
>>> retrieve the result. It’s the same response that’s returned by BaseX.
>>>
>>> String uri = "http://vocab.getty.edu/aat/300027473.rdf;;
>>> HttpClient client = HttpClient.newBuilder().build();
>>> HttpRequest request = HttpRequest.newBuilder(URI.create(uri)).build();
>>> BodyHandler handler = HttpResponse.BodyHandlers.ofString();
>>> HttpResponse result = client.send(request, handler);
>>> System.out.println(result.statusCode());
>>> System.out.println(result.body());
>>>
>>> 400
>>> Apache Tomcat/7.0.42 - Error
>>> report<!--H1
>>>
>>> {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
>>> H2
>>> {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;}
>>> H3
>>> {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}
>>> BODY
>>> {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;}
>>> B
>>> {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;}
>>> P
>>> {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A
>>> {color : black;}A.name {color : black;}HR {color :
>>> #525D76;}--> HTTP Status 400 - >> size="1" noshade="noshade">type Status
>>> reportmessage description The
>>> request sent by the client was syntactically incorrect.>> size="1" noshade="noshade">Apache Tomcat/7.0.42
>>>
>>> So we need to find out why the server thinks the Java request is
>>> »syntactically incorrect«. Maybe we can compare the low-level
>>> representation of the requests with Java 9 and 10 (?).
>>>
>>

Re: [basex-talk] changes in BaseX-10.0 w.r.t. http:send-request() / fetch:doc() functions?

[Andy Bunce]

There seems to be a 303 redirect. Maybe this is relevant
https://stackoverflow.com/a/66325588/3210344
/Andy
[image: image.png]



On Thu, 4 Aug 2022 at 16:19, Christian Grün 
wrote:

> What I have assessed so far is that it’s the Java Client that fails to
> retrieve the result. It’s the same response that’s returned by BaseX.
>
> String uri = "http://vocab.getty.edu/aat/300027473.rdf;;
> HttpClient client = HttpClient.newBuilder().build();
> HttpRequest request = HttpRequest.newBuilder(URI.create(uri)).build();
> BodyHandler handler = HttpResponse.BodyHandlers.ofString();
> HttpResponse result = client.send(request, handler);
> System.out.println(result.statusCode());
> System.out.println(result.body());
>
> 400
> Apache Tomcat/7.0.42 - Error
> report HTTP Status 400 -  size="1" noshade="noshade">type Status
> reportmessage description The
> request sent by the client was syntactically incorrect. size="1" noshade="noshade">Apache Tomcat/7.0.42
>
> So we need to find out why the server thinks the Java request is
> »syntactically incorrect«. Maybe we can compare the low-level
> representation of the requests with Java 9 and 10 (?).
>

Re: [basex-talk] BaseX 10 • The Double-Digit Summer Edition

[Andy Bunce]

Looks good to me.
I like the renamings and I look forward to taking a look at how the locking
changes play out

One minor issue I have is: in the GUI editor Mouse-down+ drag is not
selecting text. This is on Win10 with a trackpad and Java version as below.

/Andy

IMPLEMENTOR="Eclipse Adoptium"
IMPLEMENTOR_VERSION="Temurin-11.0.15+10"
JAVA_VERSION="11.0.15"
JAVA_VERSION_DATE="2022-04-19"


On Tue, 2 Aug 2022 at 08:14, Marco Lettere  wrote:

> THANKS!
>
> M.
>
> On 01/08/22 15:27, Christian Grün wrote:
> > Dear all,
> >
> > It’s been around 15 years ago when the first Open-Source version of
> > BaseX was released. Thanks to the continuous support of all of you, we
> > are thrilled today to announce the first double-digit version of our
> > XML framework:
> >
> >https://basex.org/
> >
> > BaseX 10 assists you with your everyday data challenges with some
> > great new features:
> >
> > STORAGE
> >
> > • Beside XML Documents and Binary Data, you can use the new »XQuery
> > Values« resource type to store arbitrary items and sequences in
> > databases.
> > • General data (registered users, scheduled services, and key-value
> > stores) can now be backed up as well.
> > • Mixed-content developers will be delighted: Whitespaces are now
> > preserved by default.
> >
> > XQUERY
> >
> > • Database locking has been improved by splitting up XQuery
> > compilation into a static and dynamic step.
> > • A new Store Module allows you to organize values in a persistent
> > main-memory key-value store.
> > • You can store and retrieve XQuery values with db:put-value and
> db:get-value.
> > • When serializing XML, indentation is now turned off by default.
> >
> > GENERAL
> >
> > • BaseX 10 requires Java 11 to run.
> > • The GUI comes with improved HiDPI support, new icons and improved
> consistency.
> > • All HTTP requests are now based on the new Java HTTP client.
> > • Catalog support is now universal and based on either the new XML
> > Catalog API or Norman Walsh’s excellent XML Resolver.
> > • The default ports for web applications were changed from 8984/8985
> > to 8080/8081.
> > • The default admin password was removed to improve security.
> > • Results of the REST API are now wrapped in the default REST namespace.
> >
> > We have taken the version jump as an opportunity to rename various
> > existing functions and commands. The database storage has not changed:
> > Databases created with BaseX 10 are backward compatible and can still
> > be opened with BaseX 9.
> >
> > Visit our documentation for more details, and information on how to
> > upgrade your existing applications:
> >
> >https://docs.basex.org/wiki/BaseX_10
> >
> > Have fun!
> > Your BaseX Team
>

Re: [basex-talk] Multi-valued values in maps - sequences vs arrays

[Andy Bunce]

Hi Bridger,

> is even possible; e.g. map{ "key": [1,2,3,4,5] }
Well that works fine for me, so yes.

For your other examples, I think the answer is that map:merge with map{
"duplicates":"combine"} always generates value *sequences *on duplicate
keys.
So, maybe, generate the array you want, then put it in the map

let $a:=(1 to 5)!array{.}=>array:join()
return map:entry("key",$a)

/Andy

On Wed, 3 Aug 2022 at 09:37, Martin Honnen  wrote:

>
> Am 03.08.2022 um 04:27 schrieb Bridger Dyson-Smith:
> >
> >
> > I would appreciate some help understanding how I might go about having
> > a multi-valued array as the value of a map key, or if this is even
> > possible; e.g.
> >
> > map{ "key": [1,2,3,4,5] }
>
>
> map { 'key' : array { 1 to 5 } }
>
>
> Or have I missed the point?
>
>

Re: [basex-talk] Lock avoidance using inspect:functions?

[Andy Bunce]

Hi Christian,

>I’m somewhat sorry this bug is fixed now
Me too :)

> As we can’t assess at compile time if locking is necessary.
I guess there are two compile times at play here, as with xquery-eval.
It would be nice to find a way to downgrade locks as well as upgrade them.
But easier said than done.

/Andy





On Mon, 13 Jun 2022 at 12:17, Christian Grün 
wrote:

> Hi Andy,
>
> > I am pretty sure this is a bug/oversight in inspect:functions. I suspect
> running it risks data integrity, so it is not a good idea.
>
> Good catch; and I’m somewhat sorry this bug is fixed now [1]. As we
> can’t assess at compile time if locking is necessary, a global read
> lock is now applied as soon as the function is called. The read lock
> is upgraded to a write lock if one of the returned functions is
> invoked as an updating function.
>
> Cheers,
> Christian
>
> [1]
> https://github.com/BaseXdb/basex/commit/8e6ae0d02188348b7b8adb872657fc700518675b
>

[basex-talk] Lock avoidance using inspect:functions?

[Andy Bunce]

 Following Omar's  message on lock management [1]
I thought I would share a technique that I have tried. It makes use of the
function inspect:functions [2].

The code below shows two versions of an eval-update function. After running
each version the info.result view reports:

   - local:eval-update => a global write lock.
   - local:eval-update2 => no locks at all!

I am pretty sure this is a bug/oversight in inspect:functions. I suspect
running it risks data integrity, so it is not a good idea.

However, I do make use of a non-updating version of this technique, as a
way to do string evaluation without triggering a global read lock.
If I know that no lockable resources are referenced then it seems safe to
me?
Note: inspect:functions is similar to the unimplemented standard XQuery 3.1
function fn:load-xquery-module[3]

/Andy

[1]
https://mailman.uni-konstanz.de/pipermail/basex-talk/2022-May/017207.html
[2] https://docs.basex.org/wiki/Inspection_Module#inspect:functions
[3] https://www.w3.org/TR/xpath-functions-31/#func-load-xquery-module


(:~ evaluate XQuery string, string is enriched with external variable
definitions for convenience :)
declare %updating function local:eval-update($exp as xs:string,$binds as
map(*)){
  let $decl:= map:keys($binds)!``[declare variable $`{ . }`
external;]``=>string-join("")
   return xquery:eval-update($decl || $exp, $binds)
};

declare %updating function local:eval-update2($exp as xs:string,$binds as
map(*)){
  let $decl:= map:keys($binds)!``[let $`{ . }` :=$binds("`{ . }`")
]``=>string-join("")
  let $mod:=("module namespace _='abuse'; declare %updating function
_:foo($binds){" , $decl, "return " , $exp , "};")=>string-join("")
  let $p:=file:create-temp-file("APB", ".xqm")
  let$_:=file:write-text($p,$mod)
  return updating inspect:functions($p)($binds)
};

(: note: the bind functionality is not used in this example)
let $binds:=map{"p1":42}
let $exp:="db:replace('factbook', 'abuse.xml', '')"
return  local:eval-update2($exp,$binds)

Re: [basex-talk] Executing optimized XQuery in RestXQ without having to deal with global lock situations

[Andy Bunce]

> a bottleneck shows in the way BaseX handles file access.

I wonder if this issue is relevant?
https://github.com/BaseXdb/basex/issues/1574
/Andy


On Mon, 2 May 2022 at 16:11, Omar Siam  wrote:

> What I like about BaseX is that it is very good at optimizing
> self-contained queries about the size a user can read and understand [1]
> [2] and that it has a DB locking system for transaction management [3]
> that is robust and easy to understand.
>
> What I don’t like so much about BaseX is that these two mechanisms don’t
> work very well with complex code that is split into various modules. I
> use modules for code that may be shared among projects or just as a
> means of grouping common concerns in one module.
> That I don’t like this behaviour does not mean I know (or have any hope)
> that this can be solved in a better way without at least make unpleasant
> sacrifices elsewhere. It is just the setting I have to deal with.
>
> When BaseX cannot determine anymore which DBs are used in a query and
> which are not, it falls back to assuming there are no indexes, so
> automatic optimization in this regard is stopped, and it assumes that
> just all DBs known to BaseX are used in that query so it acquires a
> global lock. [4]
>
> When doing only reading queries this is not much of a problem. Using
> indexes in queries can be forced with functions or with the
> db:enforceindex pragma [5].
>
> Problems start showing when trying to implement a CRUD RestXQ
> application. Create, update and delete can be implemented using the
> XQuery update standard but of course now this will get slow and
> cumbersome when for many read operations it cannot be determined which
> DBs they use and so a global read lock is held. That of course means
> that no global write lock can be acquired until all read operations are
> finished on all DBs known to a BaseX instance.
>
> This is especially problematic when one instance of BaseX with a RestXQ
> application is used to serve data from independent databases. Say one
> instance of BaseX has a RestXQ API that servers a lot of different
> dictionaries for different natural languages. This is my use case.
> Although the content of dictionary entries is different, the parts in
> the TEI/XML I try to manipulate, that are created, read, updated or
> deleted, are the same. So, a common API should handle many independent
> dictionaries, edited by many users, using one instance of BaseX.
>
> Also, when working with my biggest XML database of several GB I ran into
> problems when reindexing after an update. Reindexing all those GB of
> data takes too long and makes small updates in there impossible.
>
> Why not multiple instances of BaseX? Well because for better or worse
> BaseX runs in a JVM and even after I tried to minimize the memory
> footprint of an idle BaseX it is still a little less than 300 MB and we
> run a lot of services here on shared servers so RAM usage matters. Also,
> RAM usage is a part of the costs when using commercial cloud services.
> But of course, not running BaseX at all if not used is best if you pay
> per minute. And also: as recently discussed on the list: BaseX as any
> Java program gets optimized while running by the JVM and then those
> optimizations as well as caching will benefit all the data hosted in one
> instance but would be less efficient with multiple instances I assume.
>
> So how do I achieve four goals:
> *  Keep the XQuery short and concise because that is what the optimizer
> can handle best?
> * Keep the code separated into Modules that deal with one particular
> aspect?
> * Use RestXQ and not another technique to actually implement the RESTful
> API?
> * All this while being able to split GB of XML data into portions that
> can be reindexed in a reasonable amount of time?
>
> The two thing that help here a lot is eval functions like xquery:eval
> [6] and String Constructors [7].
> Say, I want to run a query but on different collections (databases). I
> can do this by having a list of collections and executing the actual
> query in a for loop with the concrete collection as a variable.
> If I just write the XQuery code down like this the problem is that the
> optimizer would need to evaluate the query to find out which databases
> to lock and what indexes can be used. BaseX is not built to do this
> (yet). It does not mock run the query. So, it decides that a global lock
> needs to be used. Depending on the use of XQuery Update either a global
> write lock or a global read lock is acquired. Easy to understand but
> does not help with performance here.
> If I want to make the situation worse for the optimizer I can use
> xquery:eval. That of course makes the XQuery code totally opaque to the
> optimizer. A global lock is guaranteed.
>
> Still another eval function is a solution here. There is the jobs module
> jobs:eval [8].
> If I break up my code into jobs only these jobs hold locks for as long
> as they run. This can be a much shorter period of 

Re: [basex-talk] Adding another document from XQuery evaluation

[Andy Bunce]

Hi Markus,

I don't think the messages are misleading, however the rules here can be
hard to follow at first.

My understanding of the updating rules is:
All the updating must be done together as the very last thing to happen.
This means ONLY in the final  'return' statement of the main expression or
defined in the final 'return' statement in functions.
These updating function definitions can only be *called *in the final
statement of the main expression.

The other XQuery rule that applies to your examples is that:
 if there is only one expression in main or a function body then the
keyword "return" is omitted

So your first example becomes:

declare %updating function local:add_document($items as item()*)
as empty-sequence()
{
 db:add("Test_DB", $items, "Test.xml")
};

let $my_data as element() := My document
return local:add_document($my_data)

In your second example, you can *define *the updating function with let,
but *calling *it requires the use of the *updating *keyword as described in
the link you give.

let $my_data as element() := My document,
$add_document := %updating function($items) {db:add("Test_DB", $items,
"Test.xml")}
return (*updating *$add_document($my_data))

Regards
/Andy

On Wed, 18 May 2022 at 16:05, Markus Elfring  wrote:

> > This aspect of XQuery syntax (when to use commas, when to construct
> explicit
> > sequences with wrapping parens) can be a bit challenging to internalize.
>
> Thanks for your constructive feedback.
>
> I hope that further clarifications will adjust known construction
> challenges.
>
>
> >This rework runs for me in the BaseX GUI:
>
> I observed also software behaviour for this adjusted script variant
> which is acceptable finally.
>
> * How much were the error messages (which I mentioned for
>   the discussed use case) misleading?
>
> * Should the desired document addition work also by the application of
>   a corresponding inline function expression?
>   https://docs.basex.org/wiki/XQuery_Update#User-Defined_Functions
>
> Regards,
> Markus
>

Re: [basex-talk] Repeating data based on varying items in an array

[Andy Bunce]

Hi Markus,
Possibly the below is helpful to you
/Andy

declare function local:foo($item as array(*)){
  let $f:=$item(1)
  let $v:=$item(2)
  return array:for-each($v, function($q){ [$f,$q] }) ?*
};

(["f", [1, 2, 3]], ["g", [4, 5]]) ! local:foo(.)

On Fri, 13 May 2022 at 11:47, Markus Elfring  wrote:

> Hello,
>
> It seems that I am still not familiar enough with the programming language
> “XQuery”
> for the quick specification of a bit of code for a general data processing
> task.
> Thus I hope that the following use case can be clarified better.
>
>
> A sequence like “(["f" [1 2 3]], ["g" [4, 5]])” might be an interesting
> test example.
> It contains two nested arrays.
>
> * One part refers to fixed data.
>
> * Another part refers to some items with a varying number.
>
>
> I would like to convert the varying items into a data representation like
> the following.
>
> f,1
> f,2
> f,3
> g,4
> g,5
>
>
> Can any customised functions help for the desired transformation?
>
> Regards,
> Markus
>

Re: [basex-talk] Feature Preview: Caching Module

[Andy Bunce]

Maybe keep it simple and focused on the minimum required for XQuery use :)
>From a quick test Jedis seem to work fine from custom/lib [1]

/Andy
[1] https://gist.github.com/apb2006/9563707df4d8f7dd536d9cd3ea70046f



On Tue, 10 May 2022 at 12:26, Christian Grün 
wrote:

> ;·) About time for BohemiaX.
>
> Similar to Redis, we could either work with expiry dates or limit the
> cache to a maximum number of entries (and drop the ones with the
> oldest access time).
>
>
> On Sat, May 7, 2022 at 5:30 PM Andy Bunce  wrote:
> >
> > "There are only two hard things in Computer Science: cache invalidation
> and naming things." [1]
> >
> > This topic combines the two ;)
> > As for alternative names I offer: the exotic: entrepot, the industrial:
> kvstore, the bohemian: stash. Personally I think cache is fine.
> >
> > I have sometimes thought of using BaseX and Redis together managed by
> docker-compose.
> > I like the data structures [3] and the concept of 'variables' that self
> destruct after some time [4]
> >
> > /Andy
> >
> > [1] https://www.martinfowler.com/bliki/TwoHardThings.html
> > [2] https://en.wikipedia.org/wiki/Entrep%C3%B4t
> > [3] https://redis.io/docs/about/
> > [4] https://redis.io/commands/expire/
> >
> > On Fri, 6 May 2022 at 15:26, Christian Grün 
> wrote:
> >>
> >> Hi Liam,
> >>
> >> > What happens on a crash (e.g. power failure)?
> >>
> >> If BaseX is shut down gracefully, the data will be stored; otherwise,
> >> it might get lost indeed. If the cached data is important, it’s
> >> advisable to call cache:write after each update.
> >>
> >> In the documentation, I mentioned that the cache will automatically be
> >> written to disk at shutdown time. Based on some more feedback I got, I
> >> imagine there can be cases in which you simply want to create a
> >> temporary cache without making it persistent. I think I’ll change
> >> this, and I will only serialize the cache if a cache file already
> >> exists on disk (as a result of a previous explicit cache:write call).
> >>
> >> > E/g/ for the listtle teszt/experiment site i have at
> >> > www.fromoldbooks.org (and www.fromoldbooks.org/Search/) there's a
> >> > framework i wrote that calls out to BaseX and keeps a cached result in
> >> > a separate file, one per query. […]
> >>
> >> By reading your reply and the one from Omar, I wonder if the »Cache«
> >> is really the best term to describe what the module offers. It’s
> >> basically a Main-Memory Key-Value Store that can be made persistent,
> >> similar to e.g. Redis. Suggestions for a better name are welcome.
> >>
> >> All the best,
> >> Christian
>

Re: [basex-talk] Feature Preview: Caching Module

[Andy Bunce]

 "There are only two hard things in Computer Science: cache invalidation
and naming things." [1]

This topic combines the two ;)
As for alternative names I offer: the exotic: entrepot, the industrial:
kvstore, the bohemian: stash. Personally I think cache is fine.

I have sometimes thought of using BaseX and Redis together managed by
docker-compose.
I like the data structures [3] and the concept of 'variables' that self
destruct after some time [4]

/Andy

[1] https://www.martinfowler.com/bliki/TwoHardThings.html
[2] https://en.wikipedia.org/wiki/Entrep%C3%B4t
[3] https://redis.io/docs/about/
[4] https://redis.io/commands/expire/

On Fri, 6 May 2022 at 15:26, Christian Grün 
wrote:

> Hi Liam,
>
> > What happens on a crash (e.g. power failure)?
>
> If BaseX is shut down gracefully, the data will be stored; otherwise,
> it might get lost indeed. If the cached data is important, it’s
> advisable to call cache:write after each update.
>
> In the documentation, I mentioned that the cache will automatically be
> written to disk at shutdown time. Based on some more feedback I got, I
> imagine there can be cases in which you simply want to create a
> temporary cache without making it persistent. I think I’ll change
> this, and I will only serialize the cache if a cache file already
> exists on disk (as a result of a previous explicit cache:write call).
>
> > E/g/ for the listtle teszt/experiment site i have at
> > www.fromoldbooks.org (and www.fromoldbooks.org/Search/) there's a
> > framework i wrote that calls out to BaseX and keeps a cached result in
> > a separate file, one per query. […]
>
> By reading your reply and the one from Omar, I wonder if the »Cache«
> is really the best term to describe what the module offers. It’s
> basically a Main-Memory Key-Value Store that can be made persistent,
> similar to e.g. Redis. Suggestions for a better name are welcome.
>
> All the best,
> Christian
>

Re: [basex-talk] xslt:transform-report result

[Andy Bunce]

Ok thanks. I thought this might have been unintended behavior.
if($report?result instance of document-node()+) then
document{$report?result} else $report?result
seems to give me what I expected here.
/Andy


On Wed, 4 May 2022 at 16:58, Christian Grün 
wrote:

> Thanks, Andy, I’ve updated the documentation.
>
> On Wed, May 4, 2022 at 3:12 PM Andy Bunce  wrote:
>
>> Hi,
>>
>> Using BaseX 9.7.1 and saxon9he-9.9.1.jar
>> The documentation suggests the ?result from xslt:transform-report should
>> be *a* document-node where possible [1]
>> This seems not quite to be the case when there are processing
>> instructions or comments at the top level. In these cases a sequence of
>> document-nodes is returned.
>>
>> /Andy
>> [1] https://docs.basex.org/wiki/XSLT_Module#xslt:transform-report
>>
>> let $xslt:=http://www.w3.org/1999/XSL/Transform; version="3.0">
>> 
>> 
>>
>> let $xml:=document{ , }
>> return xslt:transform-report($xml,$xslt)
>>
>> Returns
>> map {
>>   "messages": (),
>>   "result": (, )
>> }
>>
>>
>>
>>
>>

[basex-talk] xslt:transform-report result

[Andy Bunce]

Hi,

Using BaseX 9.7.1 and saxon9he-9.9.1.jar
The documentation suggests the ?result from xslt:transform-report should be
*a* document-node where possible [1]
This seems not quite to be the case when there are processing instructions
or comments at the top level. In these cases a sequence of document-nodes
is returned.

/Andy
[1] https://docs.basex.org/wiki/XSLT_Module#xslt:transform-report

let $xslt:=http://www.w3.org/1999/XSL/Transform;
version="3.0">



let $xml:=document{ , }
return xslt:transform-report($xml,$xslt)

Returns
map {
  "messages": (),
  "result": (, )
}

Re: [basex-talk] xsl:transform-report message truncation

[Andy Bunce]

Ah, yes I see now. I never noticed this before. Looks good now I realise
what is happening.
/Andy

On Fri, 29 Apr 2022 at 19:11, Christian Grün 
wrote:

> Hi Andy,
>
> It’s the BaseX standard serializer that truncates maps and arrays.
> Some more examples:
>
> [ string-join(1 to 1000) ],
> map { 1: string-join(1 to 1000) }
>
> You can get the full string by attaching a ?* lookup step to your query.
>
> Maybe we can remove the truncation of values in function items; I’ll
> have some more thoughts on that.
>
> Thanks,
> Christian
>

[basex-talk] xsl:transform-report message truncation

[Andy Bunce]

Hi,
Using 9.7.1
(: test transform :)
let $xslt:=http://www.w3.org/1999/XSL/Transform;
   version="3.0">

I want to see all of the very long message
a
a bbb  

gg  gg
aaa important
bit


return xslt:transform-report(,$xslt)?messages

Returns
["I want to see all of the very long message a
 a bbb  
gg  gg
aaa..."]

Is it BaseX truncating this? Can it be turned off for this case?

/Andy

Re: [basex-talk] Using error and catch for error paths in REST API endpoint code

[Andy Bunce]

 I imagine reworking the scopes will be very much non-trivial.
(I am just happy I found that "simple" example - XQST0049 has been
troubling me for years)

Thanks, for this and all your great work on BaseX.

 /Andy

On Mon, 25 Apr 2022 at 15:13, Christian Grün 
wrote:

> Although this can run into issues like #[1194]. For example trying to do
>> all modules...
>>
>> (: hack to scrape  module names from github :)
>> let $mods:=fetch:xml("
>> https://github.com/acdh-oeaw/vleserver_basex/tree/main/vleserver
>> ",map{"parser":"html"})
>> //a[@data-pjax="#repo-content-pjax-container"][ends-with(.,".xqm")]
>> /concat("https://raw.githubusercontent.com",replace(@href,"/blob/","/"))
>>
>> (: get namespace prefixes in use :)
>> return $mods!( try{ inspect:xqdoc(.)/*:namespaces } catch *
>> {$err:description})
>>
>
> The upper code snippet can be evaluated with the latest snapshot, as all
> URLs will now be normalized similar to local file paths [1,2]. Work on the
> scope chain gang still in progress ;)
>
> [1] https://github.com/BaseXdb/basex/issues/2093
> [2] https://files.basex.org/releases/latest/
>
>
>

Re: [basex-talk] Using error and catch for error paths in REST API endpoint code

[Andy Bunce]

It is hard...Working on the scope chain gang :)

https://www.songlyrics.com/sam-cooke/working-on-the-chain-gang-lyrics/

On Sun, 24 Apr 2022 at 13:26, Christian Grün 
wrote:

> XQuery Compiler Life would be so easy if there was only one module per URI
> …
> Thanks for your analysis and the minimized example.
>
>
> On Sun, Apr 24, 2022 at 2:05 PM Andy Bunce  wrote:
>
>> Thanks for the resolution hint and the clue that there is no error when
>> running locally.
>>
>> I dug a bit deeper. It seems running modules via http shines a light on
>> some less common codepaths #[2091]
>>
>>  I get an error from just:
>> inspect:xqdoc("
>> https://raw.githubusercontent.com/acdh-oeaw/vleserver_basex/main/vleserver/changes.xqm
>> ")
>> [XQST0049] Duplicate declaration of static variable $_:basePath.
>>
>> I note that:
>>
>>1. "changes.xqm" imports "utils.xqm" and "data/changes.xqm"
>>2. "data/changes.xqm" imports "../utils.xqm"
>>
>>
>> So utils.xqm is imported with two different paths and BaseX fails to spot
>> it is the same file when the transport is https
>> but it does spot they are the same when these are local files. I think
>> this is correct, BUT it is not the true problem.
>>
>> I believe the true problem is the raising of error XQST0049, regardless
>> of whether the imports of utils.xqm are the same file or not.
>> Because: Module imports are not transitive #[2048]  and in particular:
>> "module A does not have access to the *variables *declared in module C"
>>
>> There is no "Duplicate declaration of static variable" in this case
>> because  "changes.xqm" should not be able to see what is imported by
>> "data/changes.xqm"
>> The variables are defined in different scopes. Saxon XQuery runs with no
>> error.
>>
>> /Andy
>>
>> [2091] https://github.com/BaseXdb/basex/issues/2091
>> [2048]  https://github.com/BaseXdb/basex/issues/2048
>>
>>
>>
>>
>>
>>
>>
>>
>> On Sat, 23 Apr 2022 at 17:11, Christian Grün 
>> wrote:
>>
>>> Hi Andy,
>>>
>>>
>>>> (: hack to scrape  module names from github :)
>>>> let $mods:=fetch:xml("
>>>> https://github.com/acdh-oeaw/vleserver_basex/tree/main/vleserver
>>>> ",map{"parser":"html"})
>>>> //a[@data-pjax="#repo-content-pjax-container"][ends-with(.,".xqm")]
>>>> /concat("https://raw.githubusercontent.com
>>>> ",replace(@href,"/blob/","/"))
>>>>
>>>> (: get namespace prefixes in use :)
>>>> return $mods!( try{ inspect:xqdoc(.)/*:namespaces } catch *
>>>> {$err:description})
>>>>
>>>
>>> I was surprised to see that this expression even triggers an exception
>>> if only the first module is inspected. If the repository is cloned and if
>>> it’s invoked on the local files, it runs smoothly, so maybe it’s a bug in
>>> the resolution of the remote URIs that a module that’s imported multiple
>>> times is not detected as already parsed.
>>>
>>> I didn’t have time yet to track this down.
>>>
>>>

Re: [basex-talk] Using error and catch for error paths in REST API endpoint code

[Andy Bunce]

Thanks for the resolution hint and the clue that there is no error when
running locally.

I dug a bit deeper. It seems running modules via http shines a light on
some less common codepaths #[2091]

 I get an error from just:
inspect:xqdoc("
https://raw.githubusercontent.com/acdh-oeaw/vleserver_basex/main/vleserver/changes.xqm
")
[XQST0049] Duplicate declaration of static variable $_:basePath.

I note that:

   1. "changes.xqm" imports "utils.xqm" and "data/changes.xqm"
   2. "data/changes.xqm" imports "../utils.xqm"


So utils.xqm is imported with two different paths and BaseX fails to spot
it is the same file when the transport is https
but it does spot they are the same when these are local files. I think this
is correct, BUT it is not the true problem.

I believe the true problem is the raising of error XQST0049, regardless of
whether the imports of utils.xqm are the same file or not.
Because: Module imports are not transitive #[2048]  and in particular:
"module A does not have access to the *variables *declared in module C"

There is no "Duplicate declaration of static variable" in this case
because  "changes.xqm" should not be able to see what is imported by
"data/changes.xqm"
The variables are defined in different scopes. Saxon XQuery runs with no
error.

/Andy

[2091] https://github.com/BaseXdb/basex/issues/2091
[2048]  https://github.com/BaseXdb/basex/issues/2048








On Sat, 23 Apr 2022 at 17:11, Christian Grün 
wrote:

> Hi Andy,
>
>
>> (: hack to scrape  module names from github :)
>> let $mods:=fetch:xml("
>> https://github.com/acdh-oeaw/vleserver_basex/tree/main/vleserver
>> ",map{"parser":"html"})
>> //a[@data-pjax="#repo-content-pjax-container"][ends-with(.,".xqm")]
>> /concat("https://raw.githubusercontent.com",replace(@href,"/blob/","/"))
>>
>> (: get namespace prefixes in use :)
>> return $mods!( try{ inspect:xqdoc(.)/*:namespaces } catch *
>> {$err:description})
>>
>
> I was surprised to see that this expression even triggers an exception if
> only the first module is inspected. If the repository is cloned and if it’s
> invoked on the local files, it runs smoothly, so maybe it’s a bug in the
> resolution of the remote URIs that a module that’s imported multiple times
> is not detected as already parsed.
>
> I didn’t have time yet to track this down.
>
>

Re: [basex-talk] Using error and catch for error paths in REST API endpoint code

[Andy Bunce]

Hi Omar,

Thanks for this.
I also enjoyed your earlier message re:
>I tried to get creative at finding a way to optimize querying the data
without having many long lasting global lock situations
I have been in a similar situation and discovered how alternative code
approaches using different builtin functions can have very different lock
profiles.
I am hopeful that  #[2063] will greatly reduce locking issues.

>Is there any sane way to get a QName with an unknown prefix of an error as
a string like in the catch all handler and resolve it against
all prefix-URI mappings known in some XQuery program?

Maybe using inspect:xqdoc can help to get the prefix to namespace map. e.g.

inspect:xqdoc("
https://raw.githubusercontent.com/acdh-oeaw/vicav-app/master/http.xqm
")/*:namespaces
http://www.xqdoc.org/1.0;>
  https://tools.ietf.org/html/rfc7807"/>
  http://exquery.org/ns/request"/>
  https://www.oeaw.ac.at/acdh/tools/vle/cors"/>
  http://basex.org/modules/admin"/>
  
  https://tools.ietf.org/html/rfc7231#section-6"/>
  http://exquery.org/ns/restxq"/>
  http://expath.org/ns/http-client"/>
  http://www.w3.org/2012/xquery"/>


Although this can run into issues like #[1194]. For example trying to do
all modules...

(: hack to scrape  module names from github :)
let $mods:=fetch:xml("
https://github.com/acdh-oeaw/vleserver_basex/tree/main/vleserver
",map{"parser":"html"})
//a[@data-pjax="#repo-content-pjax-container"][ends-with(.,".xqm")]
/concat("https://raw.githubusercontent.com",replace(@href,"/blob/","/"))

(: get namespace prefixes in use :)
return $mods!( try{ inspect:xqdoc(.)/*:namespaces } catch *
{$err:description})

I feel the solution to this must be related to the scoping of names, and in
some way depend on #[2048]

/Andy

[2063] https://github.com/BaseXdb/basex/issues/2063
[1194] https://github.com/BaseXdb/basex/issues/1194
[2048] https://github.com/BaseXdb/basex/issues/2048



On Tue, 12 Apr 2022 at 18:58, Omar Siam  wrote:

> Way too often I saw myself and my colleagues write huge, very hard to
> understand deeply nested if () then else code to handle any control path
> but the complete success one in RestXQ code. [1] is an example of such
> code even though it uses the module I want to introduce here. It seems I
> didn’t have time to refactor it yet.
> To add insult to injury the code producing a 500, 404, 403, 401, 302 or
> maybe even 201 response code was
> * neither short nor very uniform and
> * would respond to the browser with XML/XHTML or JSON or text but in
> most situations the format the browser side had the hardest time to handle
>
> For example [2] delivers an XHTML message no matter what the Accept
> header says and although it is an accurate message to a user, very
> similar XML snippets are found throughout that unit and maybe need to be
> adapted each separately if the HTML needs to updated.
>
> My XML snippets CRUD API started as a port of an apigility (now laminas
> api-tools [3]) API working with a relational backend to store XML
> snippets to something that does the same but just uses BaseX for any
> data storage and is much better at querying using sophisticated XPaths.
> So from that previous API I learned about RFC 7807. [4]
>
> RFC 7807 “Problem Details for HTTP APIs” [5] is one of several
> specification now available for reporting errors from REST APIs. This
> specification explicitly states how the errors should look like in XML
> as well as in JSON.
>
> Maybe this is a bit bold, but I used the URL of that very RFC 7807 as
> the resolved URI for my module. [6]
> Please note: I have to admit my modules have something unusual in
> common: I use the namespace prefix “_” within the module and only use a
> more talking prefix when I use a module elsewhere. So, the “_” prefix
> can map to numerous URIs in my code. I am not 100% sure if there are
> down sides to this style but I use it for a while now and no problems
> come to mind.
>
> I really dislike to get an error, especially during development of some
> service, without any indication of where that actually occurred. That is
> to say: I like stack traces in my errors.
> It also would be great if any runtime error in my code would be reported
> as XML or JSON, depending on the format the browser asked for, just as
> errors I explicitly raise.
>
> A few parts provided by BaseX greatly help in getting all of this
> packaged in some xqm-file.
> * A stack trace is always available in “$err:additional” when catching
> errors [7]
> * One catch-all error handler can be installed (“declare
> %rest:error('*') function”, although there are minor downsides to this
> catch-all handler) [8]
> * The XML based direct format BaseX uses to store JSON by default makes
> it very easy to transform RFC 7807 XML to JSON [9]
> * It is easy to query the request header anywhere in RestXQ XQuery code
> running on BaseX [10]
>
> I tried to have easy to remember function names that make the code
> readable as if it was a sentence. Therefore for example, 

Re: [basex-talk] Issue with HTTP client and authentication

[Andy Bunce]

I find  httpbin.org a useful resource to test this kind of thing [1] and [2]
/Andy

[1] http://httpbin.org/#/Auth/get_basic_auth__user___passwd_
[2] http://httpbin.org/basic-auth/user1/mypass

On Wed, 2 Feb 2022 at 16:58, Christian Grün 
wrote:

> Good to hear; so at least Digest is detected at this stage ;)
>
>
> Tim Thompson  schrieb am Mi., 2. Feb. 2022, 17:54:
>
>> With "digest" I get: "[experr:HC0004] Invalid authentication method:
>> digest."
>>
>> --
>> Tim A. Thompson
>> Metadata Librarian
>> Yale University Library
>>
>> On Wed, Feb 2, 2022 at 11:52 AM Christian Grün 
>> wrote:
>>
>>> Yes, it should still be supported. I assume it doesn't make a difference
>>> if you use "Digest" or "digest"?
>>>
>>>
>>>
>>> Tim Thompson  schrieb am Mi., 2. Feb. 2022, 17:48:
>>>
 Thanks, Christian. Right, that wasn't a working example. The server I'm
 querying is behind a firewall, so I can't really reproduce the issue
 unfortunately.

 The BaseX documentation states that the HTTP client does support Digest
 authentication. However, I did some poking around using Wireshark and
 limited knowledge of network traffic. When I run the request in curl, the
 expected protocol is played out, and I see the evidence in Wireshark:

 (1) The server responds with "401 Unauthorized" and provides the nonce
 value.
 (2) The client (curl) then does its business (supplies the username and
 password, resends the request, etc.)

 But when I submit the request via BaseX, it never gets past step (1).
 It never seems to send the username and password at all. Is Digest
 authentication still supported?

 Best,
 Tim


 --
 Tim A. Thompson
 Metadata Librarian
 Yale University Library


 On Wed, Feb 2, 2022 at 10:13 AM Christian Grün <
 christian.gr...@gmail.com> wrote:

> Hi Tim,
>
> Difficult to tell; both the curl and the XQuery variants give me 404.
> Do you think you could provide us with an example that works out of
> the box?
>
> Best,
> Christian
>
>
>
> On Wed, Feb 2, 2022 at 12:54 AM Tim Thompson 
> wrote:
> >
> > Hello,
> >
> > I'm trying to post a SPARQL query to an endpoint using Digest
> authentication with the HTTP client.
> >
> > The query works fine using curl:
> >
> > curl --digest --user user:pass -X POST -d@'test.rq' \
> > -H "Content-type: application/sparql-query" \
> > 'http://example.org/sparql'
> >
> > But the equivalent request in BaseX fails with 401 Unauthorized:
> >
> > let $endpoint := "http://example.org/sparql;
> > let $user := "user"
> > let $pass := "pass"
> > let $type := "application/sparql-query"
> >
> > let $response := (
> >   http:send-request(
> >  > method="POST"
> > href="{$endpoint}"
> > username="{$user}"
> > password="{$pass}"
> > auth-method="Digest"
> > send-authorization="true">
> >  > name="Content-Type"
> > value="{$type}; charset=utf-8"/>
> >  > media-type="{$type}">{
> > ``[
> >   select * where {?s ?p ?o} limit 1
> > ]``
> >   }
> > 
> >   )
> > )
> > return
> >   $response
> >
> > Any ideas about what might be causing the BaseX HTTP client to be
> denied here?
> >
> > Thanks in advance,
> > Tim
> >
> >
> > --
> > Tim A. Thompson
> > Metadata Librarian
> > Yale University Library
> >
>

Re: [basex-talk] Unable to Make My Custom EXPath Module Work

[Andy Bunce]

 Hi,

I have used EXPath packaging, often to package some Java code and a calling
xquery module [1].
>the URL of the expath-pkg.xml library module which will be imported from
outside must match the name of the package
In my experience this is not required, see [2] for example.
I think the problem may be that only the first xquery referenced in
expath-pkg works.
I believe your example should work.

One problem I had when trying to use the repository feature for more
complex packaging  occurs when the package exposes some xqm and that xqm
imports a utility library xqm that is in the package, but not exposed in
the expath-pkg.xml,  and the Xquery that imports the packaged xqm also
imports a different version of the utility library.
Conflicts can arise between the two versions. They should be isolated [3]

/Andy

[1] https://github.com/expkg-zone58
[2]
https://github.com/expkg-zone58/ex-thumbnailator/blob/master/src/main/expath-pkg.xml
[3] https://github.com/BaseXdb/basex/issues/2048

On Mon, 24 Jan 2022 at 17:43, Christian Grün 
wrote:

> > How best to manage deployment of XQuery modules and top-level scripts
> from a source environment to a running remote server so that using scripts
> can import modules using just the module namespace?
>
> If you don’t care about version conflicts and other potential
> dependency errors, you could use the following expression (the
> argument of fetch:binary can be an arbitrary URI):
>
> basex "archive:extract-to(db:option('repopath'), fetch:binary('zip'))"
>
> > I looked quickly at the code for EXPath package processing but I didn’t
> spent much time in it, but it probably wouldn’t be too hard to enhance it
> to behave as I expected. I could possibly pursue that if anyone else would
> use it.
>
> Thanks for the offer, that would certainly be appreciated.
>

[basex-talk] Module imports are not transitive?

[Andy Bunce]

Hi,

"Module imports are not transitive—that is, importing a module provides
access only to function and variable declarations contained directly in the
imported module. For example, if module A imports module B, and module B
imports module C, module A does not have access to the functions and
variables declared in module C."
https://www.w3.org/TR/xquery-30/#id-module-import

I have noticed before that BaseX does not appear to implement this see [1]
My case was a conflict in a module that was imported internally within an
EXpath Package and the same namespace used as a module in an application. I
have just seen another example today where the current behavior was
regarded as a "good" feature.

I don't remember ever seeing any discussion on this. Is this something that
might be implemented at some point?
/Andy

[1]
https://gist.github.com/apb2006/9702104456e5ee2c48f4ff4f9eba5c1b#file-main-xq

Re: [basex-talk] java.lang.RuntimeException: Data Access out of bounds

[Andy Bunce]

I remember seeing similar stacktraces with an old BaseX version. Could even
have been 9.1.
I would inspect[1] the database and run against latest version
/Andy

[1] .https://docs.basex.org/wiki/Commands#INSPECT

On Thu, 29 Jul 2021 at 20:54, Christian Grün 
wrote:

> Maybe you could try to find out how the behavior can be triggered from
> scratch; I’ll then be happy to look at it.
>
>
> On Thu, Jul 29, 2021 at 9:27 PM Jonathan Robie 
> wrote:
> >
> > The easiest way might be to make someone from BaseX a member of a
> private GitHub repo that has queries, .bxs files, and datasets.
> >
> > The behavior seems to vary depending on the indexes that have been
> defined.
> >
> > Jonathan
> >
> > On Thu, Jul 29, 2021 at 2:12 PM Christian Grün <
> christian.gr...@gmail.com> wrote:
> >>
> >> How can we reproduce this?
> >>
> >> On Thu, Jul 29, 2021 at 7:59 PM Jonathan Robie <
> jonathan.ro...@gmail.com> wrote:
> >> >
> >> > Any ideas what this might mean?  What should I do?
> >> >
> >> > Jonathan
>

Re: [basex-talk] getting request's headers inside a Web Application

[Andy Bunce]

 Hi BR,

When called from within a WebApp this should give the result you are
looking for..
*request:header*( 'User-Agent')

See https://docs.basex.org/wiki/Request_Module#request:header

/Andy

On Tue, 15 Jun 2021 at 15:33, dudz...@yahoo.com  wrote:

> Dear all,
>
> I do not see an option to obtain the headers from the HTTP request from
> inside webapp function. Just like in the example:
>
> declare
>   %rest:GET
>   %rest:path('tests-html')
>   ???%rest:headers('User-Agent')???
>   %output:method('html')
> function page-main:start-empty(
>   ???$user-agent as xs:string???
> ) as element(html) {
> ...
> }
>
> Is this possible?
>
>
> BR
>

Re: [basex-talk] FOP module

[Andy Bunce]

I am sure it is already on your list but just in case...
There are some map:new references that need to be map:merge in
https://files.basex.org/modules/org/basex/modules/map-extras.xqm

/Andy

On Fri, 11 Dec 2020 at 15:50, Christian Grün 
wrote:

> Hi Jérôme,
>
> I just learned there are some more modules (such as the FTP Module)
> that need to be aligned with the latest version of BaseX. I hope we’ll
> manage to look at that next week.
>
> Merci et salutations
> Christian
>
>
>
> On Tue, Dec 8, 2020 at 3:09 PM Jerome Chauveau
>  wrote:
> >
> > Hi,
> >
> > I am trying to use the (too old ?) FOP module, but unsuccessfully.
> > I copied that example :
> >
> > import module namespace fop = 'org.basex.modules.fop.FOP';
> > let $xsl := xslt:transform('fop.xml', 'fop.xsl')
> > let $pdf := fop:transform($xsl)
> > return file:write-binary('fop.pdf', $pdf)
> >
> >
> > from this thead :
> https://mailman.uni-konstanz.de/pipermail/basex-talk/2014-February/006326.html
> >
> > which generates that error (in a BaseX 9.3) :
> >
> > [XPTY0004] class org.basex.modules.fop.FOP tried to access private
> method 'void org.basex.query.value.item.B64.(byte[])'
> > (org.basex.modules.fop.FOP and org.basex.query.value.item.B64 are in
> unnamed module of loader 'app'). Caused by: FOP:transform(document-node()).
> >
> > Any suggestions on how to fix this ?
> > Thanks
> >
> > Regards
> >
> >
> > Jérôme
> >
> >
> >
>

Re: [basex-talk] Help with a function to return paths

[Andy Bunce]

If you really don't want the leading /

 declare function local:pathbuild($result as xs:string,$this as node()){
   concat(
$result, '/'[$result], name($this),
$this/@*! concat("[@" , name(.) , "='" , data(.),
"']")=>string-join('')
  )
};

On Fri, 16 Oct 2020 at 17:48, Andy Bunce  wrote:

> Hi Bridger,
>
> How about..
>
> declare function local:pathbuild($result as xs:string,$this as node()){
>  concat(
> $result, "/" , name($this),
> $this/@*! concat("[@" , name(.) , "='" , data(.),
> "']")=>string-join('')
>   )
> };
>
> innermost($input//*)!fold-left(ancestor-or-self::*,'',local:pathbuild#2)
>
> /Andy
>
> On Fri, 16 Oct 2020 at 16:29, Bridger Dyson-Smith 
> wrote:
>
>> Hi all -
>>
>> I hope you don't mind a question about serializing distinct XPaths. I'm
>> trying to generate some reports for documentation and the available
>> built-in and library functions[1] aren't quite what I need.
>>
>> The output I'm after is:
>> test/aaa/bbb[@type="foo"]
>> test/aaa/bbb[@type="foo"][@enc="bar"]
>> test/aaa/bbb[@type="bzz"][@enc="bar"]
>> test/aaa/bbb[@type="qux"][@enc="bar"][@key="yes"][@point="start"]
>> test/aaa/bbb[@type="qux"][@enc="bar"][@key="yes"][@point="end"]
>>
>> I have a couple of functions that are getting me close, but I can't quite
>> manage the output strings, and multiple children are causing me trouble
>> (entirely too much like real life). Any help or suggestions would be
>> greatly appreciated. I've created a gist[2] with the following example:
>>
>> Thanks very much for your time and trouble.
>> Best,
>> Bridger
>>
>> ```xquery
>>
>> xquery version "3.1";
>>
>> declare variable $input :=
>>   
>> 
>>   bbb content
>> 
>> 
>>   bbb content
>> 
>> 
>>   bbb content
>>   bbb content
>>   bbb content
>> 
>>   ;
>>
>> declare function local:elem(
>>   $nodes as node()*
>> ) as xs:string* {
>>   for $node in $nodes
>>   return(
>> string-join(
>>   if ($node/@*)
>>   then (string-join((name($node) || string-join(for $att in $node/@* 
>> return local:atty($att))), "/"), local:elem($node/child::*))
>>   else if ($node/child::*)
>>then (for $child in $node/child::* return local:elem($child), 
>> local:elem($node/child::*))
>>else (name($node) || "/" || local:elem($node/child::*))
>> )
>>   )
>> };
>>
>> declare function local:atty(
>>   $att as attribute()
>> ) as xs:string* {
>>   "[@" || name($att) || "='" || data($att) || "']"
>> };
>>
>>
>> local:e2($input
>> (:
>>   this currently returns
>>   test/aaa/bbb[@type='foo']/
>>   /
>>   /aaa/bbb[@type='foo'][@enc='bar']/
>>   /
>>   /aaa/bbb[@type='bzz'][@enc='bar']/
>>   /bbb[@type='qux'][@enc='bar'][@key='yes'][@point='start']/
>>   /bbb[@type='qux'][@enc='bar'][@key='yes'][@point='end']/
>>   /
>>
>> :)
>>
>> ```
>>
>> [1] fn:path, and the related functions from functx
>> (functx:distinct-element-paths, functx:path-to-node, and
>> functx:path-to-node-with-pos). The functx functions are really close and
>> awesome, but I need to incorporate attributes into my output.
>> [2] https://gist.github.com/CanOfBees/8cfb435ac06986c9b0b0c215a786f4d7
>>
>>

Re: [basex-talk] Help with a function to return paths

[Andy Bunce]

 Hi Bridger,

How about..

declare function local:pathbuild($result as xs:string,$this as node()){
 concat(
$result, "/" , name($this),
$this/@*! concat("[@" , name(.) , "='" , data(.),
"']")=>string-join('')
  )
};

innermost($input//*)!fold-left(ancestor-or-self::*,'',local:pathbuild#2)

/Andy

On Fri, 16 Oct 2020 at 16:29, Bridger Dyson-Smith 
wrote:

> Hi all -
>
> I hope you don't mind a question about serializing distinct XPaths. I'm
> trying to generate some reports for documentation and the available
> built-in and library functions[1] aren't quite what I need.
>
> The output I'm after is:
> test/aaa/bbb[@type="foo"]
> test/aaa/bbb[@type="foo"][@enc="bar"]
> test/aaa/bbb[@type="bzz"][@enc="bar"]
> test/aaa/bbb[@type="qux"][@enc="bar"][@key="yes"][@point="start"]
> test/aaa/bbb[@type="qux"][@enc="bar"][@key="yes"][@point="end"]
>
> I have a couple of functions that are getting me close, but I can't quite
> manage the output strings, and multiple children are causing me trouble
> (entirely too much like real life). Any help or suggestions would be
> greatly appreciated. I've created a gist[2] with the following example:
>
> Thanks very much for your time and trouble.
> Best,
> Bridger
>
> ```xquery
>
> xquery version "3.1";
>
> declare variable $input :=
>   
> 
>   bbb content
> 
> 
>   bbb content
> 
> 
>   bbb content
>   bbb content
>   bbb content
> 
>   ;
>
> declare function local:elem(
>   $nodes as node()*
> ) as xs:string* {
>   for $node in $nodes
>   return(
> string-join(
>   if ($node/@*)
>   then (string-join((name($node) || string-join(for $att in $node/@* 
> return local:atty($att))), "/"), local:elem($node/child::*))
>   else if ($node/child::*)
>then (for $child in $node/child::* return local:elem($child), 
> local:elem($node/child::*))
>else (name($node) || "/" || local:elem($node/child::*))
> )
>   )
> };
>
> declare function local:atty(
>   $att as attribute()
> ) as xs:string* {
>   "[@" || name($att) || "='" || data($att) || "']"
> };
>
>
> local:e2($input
> (:
>   this currently returns
>   test/aaa/bbb[@type='foo']/
>   /
>   /aaa/bbb[@type='foo'][@enc='bar']/
>   /
>   /aaa/bbb[@type='bzz'][@enc='bar']/
>   /bbb[@type='qux'][@enc='bar'][@key='yes'][@point='start']/
>   /bbb[@type='qux'][@enc='bar'][@key='yes'][@point='end']/
>   /
>
> :)
>
> ```
>
> [1] fn:path, and the related functions from functx
> (functx:distinct-element-paths, functx:path-to-node, and
> functx:path-to-node-with-pos). The functx functions are really close and
> awesome, but I need to incorporate attributes into my output.
> [2] https://gist.github.com/CanOfBees/8cfb435ac06986c9b0b0c215a786f4d7
>
>

Re: [basex-talk] Accessing Flexmark (Java) from Query

[Andy Bunce]

Hi Andreas,

I think Martin has it right, I have some old code that does it like that
[1]. You can try it via[2]

/Andy

[1]
https://github.com/expkg-zone58/ex-markdown/blob/master/src/main/content/markdown.xqm#L12
[2] https://github.com/expkg-zone58/ex-markdown/releases


On Sun, 20 Sep 2020 at 15:30, Martin Honnen  wrote:

>
> On 20.09.2020 01:41, Andreas Mixich wrote:
> > Am 19.09.2020 um 23:30 schrieb Martin Honnen:
> >> import com.vladsch.flexmark.parser.Parser.Builder;
> >>
> >> with
> >>
> >>$builder => Parser.Builder:build()
> > Hello Martin,
> >
> > thanks for taking the time!
> >
> > I am not quite sure I understand. Do you want me to:
> >
> >import module namespace Parser.Builder =
> > "java:com.vladsch.flexmark.parser.Parser.Builder";
> >
>
> I am not sure I have completely understood the reflection and namespace
> conventions but after some reading and testing it seems the inner class
> Builder should be accessible by
>
>
> declare namespace Builder =
> 'java:com.vladsch.flexmark.parser.Parser$Builder';
>
>
> and then
>
> $builder => Builder:build()
>
>
> should work for the variable
>
> let $builder := Parser:builder($mds)
>
> you have, you should be able to chain
>
> MutableDataSet:new() => Parser:builder() => Builder:build()
>
>

Re: [basex-talk] missing web.xml from docker image

[Andy Bunce]

If some updating of the docker image is undertaken it would be good to try
to enable multi-architecture support.
Currently the BaseX docker image fails on ARM machines.  It is far from
alone in this.

It is a shame as BaseX seems to run surprisingly well on ARM64 machines
such as the Odroid N2 and Raspberry Pi 4.

Recent docker developments claim to make ARM support much easier [1]
/Andy
[1]
https://developer.arm.com/common-tasks/getting-started-with-docker/single-page


On Tue, 15 Sep 2020 at 09:06, Michael Seiferle  wrote:

> Hi Nicholas,
>
> I think this might be indeed a good chance to re-engineer our docker
> image.
> We had one user that maintained the image long time ago, but maybe it
> would be good to have it updated.
> We don’t tend to use docker that often, as you might have noticed ;-)
>
> best
> Michael
>
>
> > Am 05.09.2020 um 02:24 schrieb Nicholas Saunders <
> saunders.nicho...@gmail.com>:
> >
> > There was an additional response:
> >
> >
> > https://devops.stackexchange.com/q/12338/23443
> >
> > with remarks about the basex docker image which are a bit over my head
> > but should prove useful for the maintainer(s).
> >
> > On 9/4/20, Nicholas Saunders  wrote:
> >> Certainly, I was able to get the container running.
> >>
> >> The web.xml threw me off, and then the docs said to use a deprecated
> >> command.
> >>
> >> On 9/4/20, Markus Wittenberg  wrote:
> >>> Hi there,
> >>>
> >>> in fact the web.xml is not missing. You should be able to connect to
> >>> basex using the exposed ports. This shouldn't be a problem on
> localhost.
> >>> Depending on your system configuration on a server you might have to
> >>> provide access to the ports to the web explicitly.
> >>>
> >>> The REST endpoints /rest/* over port 8984 are also activated by
> default.
> >>>
> >>> If you follow the example Running your own Application/DBA in the
> >>> documentaton you can also use RESTXQ applications.
> >>>
> >>> Best regards,
> >>>
> >>>   Markus
> >>>
> >>> Am 04.09.2020 um 12:11 schrieb Christian Grün:
>  …anyone else around here who uses Docker?
> 
>  On Fri, Sep 4, 2020 at 11:32 AM Nicholas Saunders
>   wrote:
> > thanks.  I was able to create the container okay, but how do I
> connect
> > to the database?
> >
> > What I've done so far is gain root on the container itself and
> > manually copy files in to add to the db.  Can I perhaps connect to
> the
> > db on the container through basex on the host?  Or would that be an
> > odd approach?
> >
> > On 9/3/20, Christian Grün  wrote:
> >> Thanks for the observation. I’ll try to get into contact with the
> guys
> >> who
> >> wrote the article and see if this can be updated.
> >>
> >>
> >>
> >> Nicholas Saunders  schrieb am Fr., 4.
> >> Sep.
> >> 2020, 01:19:
> >>
> >>> from what I gather, the link command is deprecated:
> >>>
> >>>  it sounds like you are using the link feature. This has been
> >>> deprecated for years. instead, use a   v2 or v3 docker-compose.yml
> >>> file (if using compose) or if using a different system, simply put
> >>> both containers on the same│
> >>>
> >>> so the documentation is out of date.
> >>>
> >>> On 9/3/20, Christian Grün  wrote:
>  Hi Nicholas,
> 
>  Have you already worked through the Docker documentation in our
> Wiki
>  [1]?
> 
>  Best,
>  Christian
> 
>  [1] https://docs.basex.org/wiki/Docker
> 
> 
> 
> 
>  On Thu, Sep 3, 2020 at 4:23 PM Nicholas Saunders
>   wrote:
> > I've just opened an issue:
> >
> > https://github.com/BaseXdb/basex/issues/1936
> >
> > based on an inability to run the docker image:
> >
> > https://devops.stackexchange.com/q/12338/23443
> >
> >
> > but thought I'd ask here if there might be something obvious.
> I'm
> > creating the container from the hub image with:
> >
> > docker run -d \
> > --name basexhttp \
> > --publish 1984:1984 \
> > --publish 8984:8984 \
> > --volume "$HOME/basex/data":/srv/basex/data \
> > --volume "$HOME/basex/repo":/srv/basex/repo \
> > --volume "$HOME/basex/webapp":/srv/basex/webapp \
> > basex/basexhttp:latest
> >
> >
> > and another person reported the missing file.  Is the file
> missing?
> > I'm not sure.
> >>>
> >>> --
> >>> Markus Wittenberg
> >>>
> >>> Tel +49 (0)341 248 475 36
> >>> Mail wittenb...@axxepta.de
> >>>
> >>> 
> >>>
> >>> axxepta solutions GmbH
> >>> Lehmgrubenweg 17, 88131 Lindau
> >>>
> >>> Amtsgericht Berlin HRB 97544B
> >>> Geschäftsführer: Karsten Becke, Maximilian Gärber
> >>>
> >>>
> >>
>
>

Re: [basex-talk] BaseX - CORS

[Andy Bunce]

Hi Elina,  (cc: basex-talk)

Happy to help. That email correspondence is a bit old, and the code
referenced is ancient. I suspect your problems creating the headers in
XQuery restxq are due to default namespace prefix changes, rest vs restxq,
way back when.

You are correct that enabling the supplied CORS filter in web.xml works for
GET and POST but not PUT.
The reason is that PUT is not in the default AllowedMethods [1]

As an exercise I have created a little sample that uses only RESTXQ
facilities. [2] It assumes the web.xml CORS filter is not enabled.

As supplied it has:
A cors.html file, to be opened directly in the browser, i.e. using the file
protocol.
A cors.xqm intended to be put in the webapp folder of a local basexhttp
running on 8984

Only the PUT request is allowed as I have commented out the Xquery
generation of the CORS headers for GET and POST.
Note PUT is a non-simple CORS request and will result in the browser
sending an additional request using the OPTIONS method [3]
This can be seen in the screenshot and is handled in the cors.xqm code.

Hope this helps
/Andy

[1]
https://www.eclipse.org/jetty/documentation/current/cross-origin-filter.html
[2] https://gist.github.com/apb2006/ea6a901a570b04f94e0e0e21098a5402
[3]
https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#Preflighted_requests

[image: image.png]

On Fri, 7 Aug 2020 at 13:53, Elina Koskentalo  wrote:

> Hello Andy,
>
> I hope you don't mind me contacting you, I found this email corresponding
> between you and another person about BaseX and adding
> the Access-Control-Allow-Origin header to the response.
> https://mailman.uni-konstanz.de/pipermail/basex-talk/2017-February/011782.html
>
> I have a very similar issue than what you are talking about there. I don't
> manage to define a rule in the restxq file that would simply add
> Access-Control-Allow-Origin header to or requests. I already manipulated
> the filters in the web.xml file which now works for GET but for PUT I still
> get the CORS issue back.
>
> Would you have any tips for me? Thank you very much in advance!
>
> Best regards,
> Elina Koskentalo
> ekosk...@gmail.com
>

Re: [basex-talk] BaseX GUI language settings

[Andy Bunce]

Hi Ben,

GUI Menu... Options>Preferences>Language
As it says, a GUI restart is required.

Cheers
/Andy


On Wed, 26 Feb 2020 at 08:46, Ben Engbers  wrote:

> Hi,
>
> My default language for basexgui is Dutch but I want to create
> screenshots from a GUI that uses English.
>
> How can I switch the language temporarily?
>
> Cheers,
> Ben
>

Re: [basex-talk] Processing instruction constructors: problem with output

[Andy Bunce]

Reminds me of a similar issue I had with inline script blocks containing <
characters..

xhtml sets the serialization but the content type will stay as text/html
unless you explicitly set it.
%output:media-type('application/xhtml+xml')

/Andy

On Thu, 22 Aug 2019 at 16:47, Tim Thompson  wrote:

> Thanks for testing, Bridger! You should see the "3" if XSLTForms is
> working. I got this to work correctly by changing the BaseX output
> serialization method to "xml" instead of "xhtml". I'm not sure why "xhtml"
> doesn't work, but XSLTForms converts the document to HTML in the browser
> anyway, so this should be okay.
>
> Best,
> Tim
>
> --
> Tim A. Thompson
> Discovery Metadata Librarian
> Yale University Library
>
> On Thu, Aug 22, 2019 at 11:35 AM Bridger Dyson-Smith <
> bdysonsm...@gmail.com> wrote:
>
>> Hi Tim -
>> A quick test and this is working correctly(?) for me;
>> 1. I cloned Alain's repo to `.../webapp/static/`
>> 2. I copy-pasta'd your example into `.../webapp/test.xqm`
>> 3. After starting the BaseX HTTP server, I hit `localhost:8984` and
>> `localhost:8984/test` and had good responses from both; i.e. at `/test`,
>> I'm seeing a "Test", a "Placeholder", and a fun little gray-ish ball (but
>> no "3" in either browser).
>>
>> This was with Firefox 60.8.0esr and Chromium 76.0.3809.87.
>>
>> Is there anything specific I can try to tell you? I'm not very familiar
>> with XSLTforms.
>> Best,
>> Bridger
>>
>> On Thu, Aug 22, 2019 at 11:16 AM Christian Grün <
>> christian.gr...@gmail.com> wrote:
>>
>>> I can’t promise if I can further dive into this. Maybe Alain has some
>>> hints?
>>>
>>>
>>> On Thu, Aug 22, 2019 at 5:09 PM Tim Thompson  wrote:
>>> >
>>> > Okay, so the response panel shows the correct output, but for some
>>> reason the stylesheet is not being called. If it's not too much trouble,
>>> would you be able to try this with XSLTForms (
>>> https://github.com/AlainCouthures/xphoneforms.git) placed in the BaseX
>>> webapp static dir?
>>> >
>>> > Here is an expanded demo query:
>>> >
>>> >
>>> > xquery version "3.1";
>>> >
>>> > module namespace test = "test";
>>> >
>>> > declare
>>> >   %rest:path("/test")
>>> >   %rest:GET
>>> >   %output:method("xhtml")
>>> > function test:home(
>>> >
>>> > ) as document-node() {
>>> >
>>> >   document {
>>> > processing-instruction { "xml-stylesheet" } {
>>> > 'type="text/xsl"
>>> href="/static/xphoneforms/build/xsl/xsltforms.xsl"'
>>> > },
>>> > processing-instruction { "xsltforms-options" } { 'debug="no"' },
>>> > processing-instruction { "css-conversion" } { 'no' }
>>> > ,
>>> > http://www.w3.org/1999/xhtml; xmlns:xf="
>>> http://www.w3.org/2002/xforms;
>>> >   xmlns:ev="http://www.w3.org/2001/xml-events; xmlns:my="test"
>>> >   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>>> xmlns:xsd="http://www.w3.org/2001/XMLSchema;>
>>> >
>>> >   
>>> > TALLER BIBFRAME
>>> > http://www.w3.org/2002/xforms; id="main">
>>> >   
>>> > 
>>> >   Test
>>> > 
>>> >   
>>> > 
>>> >
>>> >
>>> >   
>>> >   
>>> > Placeholder
>>> > 
>>> > 
>>> >   
>>> > 
>>> >   
>>> > 
>>> >   }
>>> >
>>> > };
>>> >
>>> > --
>>> > Tim A. Thompson
>>> > Discovery Metadata Librarian
>>> > Yale University Library
>>> >
>>> > On Thu, Aug 22, 2019 at 10:57 AM Christian Grün <
>>> christian.gr...@gmail.com> wrote:
>>> >>
>>> >> Hm… You could check the Response panel in the Network View of your
>>> >> browser. The latest version of Chrome, Firefox, Opera, Edge and IE
>>> >> should give you the correct result (those are the ones that I have
>>> >> tried).
>>> >>
>>> >>
>>> >> > I'm getting the comments in the browser (web console). When I
>>> dereference the endpoint with curl, the processing instructions are correct.
>>> >> >
>>> >> > In the browser, the XForms instructions are not being processed. I
>>> don't think XSLTForms would be the culprit, because the stylesheet is not
>>> being called.
>>> >> >
>>> >> > --
>>> >> > Tim A. Thompson
>>> >> > Discovery Metadata Librarian
>>> >> > Yale University Library
>>> >> >
>>> >> > On Thu, Aug 22, 2019 at 10:37 AM Christian Grün <
>>> christian.gr...@gmail.com> wrote:
>>> >> >>
>>> >> >> Hi Tim,
>>> >> >>
>>> >> >> I have just tested your endpoint code (with the latest version of
>>> >> >> BaseX), and I receive processing instructions as expected:
>>> >> >>
>>> >> >> >> >> >> href="static/xphoneforms/build/xsl/xsltforms.xsl"?>
>>> >> >> ...
>>> >> >>
>>> >> >> Which version of BaseX are you using? Are you also getting
>>> comments in
>>> >> >> your response when calling the function from the browser or via
>>> other
>>> >> >> tools (such as curl), and could it have to do with the way how
>>> >> >> XSLTForms processes the result?
>>> >> >>
>>> >> >> Best,
>>> >> >> Christian
>>> >> >>
>>> >> >>
>>> >> >> On Thu, Aug 22, 2019 at 4:31 PM Tim Thompson 
>>> wrote:
>>> 

Re: [basex-talk] problem: accessing inner java class methods with bases java bindings

[Andy Bunce]

Hi Florian,

It can be done, but it is perhaps a bit fragile.

You need to see now the class you want to reference is named after
compilation(e.g in the jar). Often it will be of the form
outerclass$innerclass.class
A BaseX example  accessing a inner class Builder from a Parser class [1].
Some general Java discussion [2]

/Andy

[1]
https://github.com/expkg-zone58/ex-markdown/blob/master/src/main/content/markdown.xqm#L12
[2]
https://stackoverflow.com/questions/2883181/why-is-an-anonymous-inner-class-containing-nothing-generated-from-this-code

On Sun, 7 Jul 2019 at 12:47, Florian Eckey 
wrote:

> Hi,
>
>
>
> I use basex java bindings in my project. It works fine but when I try to
> access inner java classes i cant find a way to get it work.
>
> I will use the SimpleJavaMail libs and there is a class called
> “MailerRegularBuilder” which is an inner class of “MailerBuilder”,
> especially with the method “buildMailer()”.
>
>
>
> Here is a sample snippet of the based java class source code:
>
> public class MailerBuilder {
>
>
>
> ...
>
>
>
>private MailerBuilder() {
>
> }
>
>
>
> public static class MailerRegularBuilder extends 
> MailerGenericBuilder {
>
>
>
> ...
>
>
>
>public Mailer buildMailer() {
>
>return new Mailer(this);
>
> }
>
>
>
> ...
>
>
>
> }
>
> }
>
>
>
> In java I can easily access this method in the inner class with the
> following code (in red):
>
>
>
> MailerRegularBuilder mailRegularBuilder = MailerBuilder.*withSMTPServer*("
> smtp.gmail.com", 465, "username1", "password1");
>
> mailRegularBuilder = mailRegularBuilder.withProperty(
> "mail.smtp.starttls.enable", "true");
>
> mailRegularBuilder = mailRegularBuilder.withProperty(
> "mail.smtp.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
>
>
>
> Mailer inhouseMailer = mailRegularBuilder.buildMailer();
>
>
>
> inhouseMailer.sendMail(email);
>
>
>
> That works fine (remember it is only a snipped).
>
> With XQuery I always get a *class not found exception* or a *unknown
> method exception* with the following code:
>
>
>
> declare namespace Email = "java:org.simplejavamail.email.Email";
>
> declare namespace EmailBuilder =
> "java:org.simplejavamail.email.EmailBuilder";
>
> declare namespace Mailer = "java:org.simplejavamail.mailer.Mailer";
>
> declare namespace MailerBuilder =
> "java:org.simplejavamail.mailer.MailerBuilder";
>
> declare namespace EmailPopulatingBuilder =
> "java:org.simplejavamail.email.EmailPopulatingBuilder";
>
> declare namespace MailerRegularBuilder =
> "java:org.simplejavamail.mailer.MailerBuilder.MailerRegularBuilder";
>
> declare namespace MailerGenericBuilder =
> "java:org.simplejavamail.mailer.MailerGenericBuilder";
>
>
>
> declare variable $local:fromName := "Florian Eckey";
>
> declare variable $local:fromEmailAdress := "a...@gmail.com";
>
> declare variable $local:username := "username1"
>
> declare variable $local:password := "password1";
>
> declare variable $local:smtpServer := "smtp.gmail.com";
>
> declare variable $local:smtpPort := 465;
>
>
>
> declare function local:sendMail($toName, $toEmailAdress, $subject,
> $plainText, $attachementName, $attachement, $attachementType) {
>
>   let $emailPopulatingBuilder := EmailBuilder:starting-blank()
>
>   let $emailPopulatingBuilder :=
> EmailPopulatingBuilder:from($emailPopulatingBuilder,$local:fromName,
> $local:fromEmailAdress)
>
>   let $emailPopulatingBuilder :=
> EmailPopulatingBuilder:to($emailPopulatingBuilder,$toName, $toEmailAdress)
>
>   let $emailPopulatingBuilder :=
> EmailPopulatingBuilder:withSubject($emailPopulatingBuilder,$subject)
>
>   let $emailPopulatingBuilder := if($attachement) then
> EmailPopulatingBuilder:withAttachment($attachementName, $attachement,
> $attachementType) else $emailPopulatingBuilder
>
>   let $emailPopulatingBuilder :=
> EmailPopulatingBuilder:withPlainText($emailPopulatingBuilder,$plainText)
>
>
>
>   let $email := EmailPopulatingBuilder:buildEmail($emailPopulatingBuilder)
>
>
>
>   let $mailerRegularBuilder :=
> MailerBuilder:withSMTPServer($local:smtpServer, xs:int($local:smtpPort),
> $local:username, $local:password)
>
>   let $mailerRegularBuilder :=
> MailerGenericBuilder:withProperty($mailerRegularBuilder,"mail.smtp.starttls.enable",
> "true")
>
>   let $mailerRegularBuilder :=
> MailerGenericBuilder:withProperty($mailerRegularBuilder,"mail.smtp.socketFactory.class",
> "javax.net.ssl.SSLSocketFactory")
>
>
>
>   *let $mailer := MailerRegularBuilder:buildMailer($mailerRegularBuilder)*
>
>
>
>   return (Mailer:sendMail($mailer,$email),"email send")
>
> };
>
>
>
> The red line will always throw the error: *[XPST0017] Unknown class:
> org.simplejavamail.mailer.MailerBuilder.MailerRegularBuilder.*
>
>
>
> Beside: I copied all the used java libs into the lib folder of my basex
> instance. All other classes were found.
>
>
>
> Does anyone have an idea how to access this inner class methods? Or is it
> 

Re: [basex-talk] BaseX 9.2: The Spring Edition

[Andy Bunce]

Thanks for this.

I particularly like db:dir, which I think will help with creating
performant collection-based views of databases.

I am not clear what is meant by:
> - no need to import basex-api modules anymore!
Does this just mean all modules listed at
http://docs.basex.org/wiki/Module_Library are statically bound?

/Andy




On Wed, 17 Apr 2019 at 08:40, Marco Lettere  wrote:

> Thanks! Great work as usual!
> M.
>
> On 16/04/19 16:10, Christian Grün wrote:
> > Dear all,
> >
> > We are more than pleased to announce version 9.2 of BaseX, our XML
> > framework, database system and XQuery 3.1 processor:
> >
> >http://basex.org/
> >
> > This time, we have put a special focus on performance improvements:
> >
> > XQUERY PERFORMANCE
> > - comparisons will more often be rewritten to hash joins
> > - optimized access to subsequences and single sequence items
> > - consistent propagation of static types to evaluation results
> > - improved static typing of function items (maps, arrays)
> > - more aggressive pre-evaluation at compile time
> >
> > STORAGE PERFORMANCE
> > - large databases will be opened faster
> > - regular storage offset will be computed at runtime
> >
> > GENERAL XQUERY FEATURES
> > - no need to import basex-api modules anymore!
> > - Archive Module: archive:create-from, new options
> > - Database Module: directory-based access via db:dir
> > - Profiling Module: new functions (prof:gc, prof:runtime)
> > - Validation Module: assign XSD validation features
> > - WebSocket Module: query scheduling via ws:eval
> > - Utility Module: various new helper functions
> > - XSLT Module: support for XML Catalogs (thank you Liam)
> >
> > NEW OPTIONS
> > - RESTXQERRORS: Suppress parsing errors at runtime
> > - FULLPLAN: comprehensive query plan information
> >
> > FULLTEXT FEATURES
> > - stemmer for Arabic texts added
> >
> > USER INTERFACES
> > - GUI: better support for latest JDK versions (incl. OpenJDK)
> > - DBA: revised search features in log panel
> >
> > For a more comprehensive list of added and updated features. please
> > check out our documentation: http://docs.basex.org/.
> >
> > We are looking forward to your feedback.
> >
> > Have fun with the new release,
> > Your BaseX Team
>
>
>

Re: [basex-talk] A Tail Call, that runs (not) if different numeric values are being used as parameter

[Andy Bunce]

Hi Andreas,

Ignoring for a moment the mysteries of tail recursion, I wonder if a simple
regex can give you the wrapping you are looking for. See [1]

declare function local:word-wrap($string,$width){
  $string=>normalize-space()=>concat(" ")=>replace(``[(.{0,`{ $width }`})
]``,'$1')
};
"This is a very long line of text. It goes well beyond 80 (eighty)
characters in width. Therefore we need a word wrap function, which we test
in here. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad
minim ve iam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex
ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate
velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat
cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id
est laborum."
=>util:replicate(1)=>string-join("")

=>local:word-wrap(80)

/Andy
[1] https://stackoverflow.com/a/3892909/3210344

On Sat, 13 Apr 2019 at 06:06, Andreas Mixich 
wrote:

> Hi,
>
> as far as I have understood, a tail call function must end with the call
> to itself, after it has calculated all the values and, upon recursive call,
> no operation must be open in the calling instance of the function. This is
> why the following case confuses me.
>
> The function, as shown, produces a stack overflow and I get the hint to do
> tail recursion. However, if I swap just the expressions (after 'then' and
> 'else') in lines 13 and 14, the function computes. Now I would have
> thought, that it would break, because I recurse within another function,
> that is still open ('concat()') and thus must return to the calling
> function after recursion, but, interestingly, the swap of the expressions
> after the 'then' and 'else' statements on lines 13 and 14 do not error out
> with a stack overflow, in spite 'concat()' still being open. Both cases
> produce an integer. Also, advice would be welcome, on how I can solve this.
> Thank you very much.
>
> 1 declare function local:word-wrap(
> 2   $string as xs:string,
> 3   $width as xs:integer)
> 4
> 5 {
> 6   let $t := normalize-space($string)
> 7   return
> 8 if (string-length($t) < $width or string-length($t) = $width)
> 9 then $t
> 10else
> 11  let $idx :=
> 12if (matches($t, "\s"))
> 13then functx:index-of-string-last(substring($t, $width), " ")
> 14else $width
> 15  return
> 16concat(
> 17  string:trim-right(substring($t, 1, $idx)),
> 18  out:nl(),
> 19  local:word-wrap(string:trim-left(substring($t, $idx)),$width)
> 20)
> 21 };
> 22
> 23 local:word-wrap("This is a very long line of text. It goes well beyond
> 80 (eighty) characters in width. Therefore we need a word wrap function,
> which we test in here. Lorem ipsum dolor sit amet, consectetur adipisicing
> elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut
> enim ad minim ve iam, quis nostrud exercitation ullamco laboris nisi ut
> aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in
> voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint
> occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit
> anim id est laborum.",80)
>
> --
> Minden jót, all the best, Alles Gute,
> Andreas Mixich
>

Re: [basex-talk] Global locks

[Andy Bunce]

I look forward to the longer term solution :)
But in the meantime these pragmas, although not pretty, would allow
solutions to specific multitasking performance issues to be explored where
currently they can be difficult/impossible. I think there is a certain
elegance in advisory nature of pragmas.

>If manual locking is enabled, an error will be raised if a database is
accessed that has not been specified in a lock pragma.

Is it required to specify all databases accessed within the manualock scope
or could it work just specifying specific databases as readonly?
And the execution would error if these were violated.

/Andy

/Andy


On Wed, 13 Feb 2019 at 16:56, Christian Grün 
wrote:

> > Would this work where the database names are only known at run time?
>
> Right now, our locking is based on static code analysis; runtime
> information won’t be included. In the longer term, we would like to
> extend our compiler to perform multiple steps:
>
> 1. Static parsing
> 2. Static optimizations (pre-evaluate 1+2 to 3, etc.)
> 3. Dynamic optimizations (reoptimize the query with query parameters
> and other context information)
> 4. Physical optimizations (open databases, utilize available index
> structures)
>
> Queries that have been parsed and statically compiled (Step 1+2) could
> be stored as compiled queries, or kept in main-memory and duplicated
> before dynamic optimizations take place. This could further reduce
> access times in RESTXQ applications – and it would allow us to improve
> locking, as we could do the lock check after Step 3.
>

Re: [basex-talk] jobs:list-details o/p

[Andy Bunce]

Looks good. Thanks.
/Andy

On Wed, 13 Feb 2019 at 16:14, Christian Grün 
wrote:

> Hi Andy,
>
> I completely agree, the inclusion of the dummy file name doesn’t make
> any sense here. Job handling has been revised in Version 9, but I
> think I have found a solution for the job listing that should work
> similar to the original one.
>
> A new snapshot is online [1], your feedback is welcome.
> Christian
>
> [1] http://files.basex.org/releases/latest/
>
>
>
> On Tue, Feb 12, 2019 at 7:05 PM Andy Bunce  wrote:
> >
> > Hi,
> >
> > Before BaseX 9 jobs:list-details would return the query string used in
> the jobs:eval (or the uri if that was used) - as per the current docs [1]
> > Versions 9+ returns a "dummy" file name in this case.
> >
> >   jobs:eval("2+4",(),map{"cache":true()})
> >
> >   jobs:list-details("job2")
> >  duration="PT0.001S" reads="(none)" writes="(none)"
> time="2019-02-12T17:53:41.772Z">file:///C:/Users/andy/git/vue-poc/src/vue-poc/features/tasks/model.build/file
> >
> > I found the old behavior useful, (although it missed recording the
> static-base-uri).
> >
> > /Andy
> >
> > [1] http://docs.basex.org/wiki/Jobs_Module#jobs:list-details
> >
> >
>

[basex-talk] jobs:list-details o/p

[Andy Bunce]

Hi,

Before BaseX 9 jobs:list-details would return the query string used in the
jobs:eval (or the uri if that was used) - as per the current docs [1]
Versions 9+ returns a "dummy" file name in this case.

  jobs:eval("2+4",(),map{"cache":true()})

  jobs:list-details("job2")
file:///C:/Users/andy/git/vue-poc/src/vue-poc/features/tasks/model.build/file

I found the old behavior useful, (although it missed recording the
static-base-uri).

/Andy

[1] http://docs.basex.org/wiki/Jobs_Module#jobs:list-details

Re: [basex-talk] Global locks

[Andy Bunce]

sorry typo there, should be:
I expected *BEP *to appear only in the Read locking, as it does if I remove
the db:create call



On Fri, 8 Feb 2019 at 16:03, Andy Bunce  wrote:

> Using 9.1.2
>
> let $d:=db:open(``[BEP]``)
> return db:create(``[BEP-staging]``,$d,$d!base-uri(.))
>
> The GUI query info reports:
> - Read Locking: (none)
> - Write Locking: BEP,BEP-staging
>
> I expected BEP-staging to appear only in the Read locking, as it does if I
> remove the db:create call
> Is this correct?
>
> /Andy
>
>
> On Thu, 21 Sep 2017 at 16:26, Christian Grün 
> wrote:
>
>> > The use of String Constructor does not avoid the global read lock
>>
>> Good to know, thanks. It now does [1], provided that the constructor
>> consists of a single static strings.
>>
>> [1] http://files.basex.org/releases/latest/
>>
>>
>>
>> > > > duration="PT0.007S" reads="(global)"
>> > writes="(none)">count(collection(``[BEP]``))
>> >
>> > On 20 September 2017 at 23:28, Christian Grün <
>> christian.gr...@gmail.com>
>> > wrote:
>> >>
>> >> > bxcode:inline-variables($xqcode-uri) as xs:string
>> >> > @returns xquery code that is the source code from $xqcode-uri but
>> with
>> >> > inlining "constant" variables and operations applied
>> >> >
>> >> > This could be used as a preprocessor.
>> >> > Is this something that could be created from the bits and pieces in
>> the
>> >> > current BaseX Java code?
>> >>
>> >> This reminds me of the discussion with Omar [1]: There is currently no
>> >> way to generate a correct string representation for compiled code. In
>> >> the GUI Info View, we output a string representation of the optimized
>> >> query. In simple cases, this string is a valid and equivalent to the
>> >> original query string, but it’s fairly easy to generate invalid
>> >> results.
>> >>
>> >> If it’s an option for you to generate code in XQuery, something like
>> >> the following could be done:
>> >>
>> >> _ bxcode.xqm ___
>> >>
>> >>   module namespace bxcode = 'bxcode';
>> >>   declare function bxcode:bind-external-variables(
>> >> $uri   as xs:string,
>> >> $vars  as map(xs:string, xs:string)
>> >>   ) as xs:string {
>> >> let $xqcode := unparsed-text($uri)
>> >> return fold-left(
>> >>   map:keys($vars), $xqcode, function($string, $name) {
>> >> let $value := replace($vars($name), ']``', '`{"]``"}`', 'q')
>> >> return $string
>> >> => replace('declare variable $' || $name || ' external;', '',
>> 'q')
>> >> => replace('$' || $name, '``[' || $value || ']``', 'q')
>> >>   }
>> >> )
>> >>   };
>> >>
>> >> _ example.xq ___
>> >>
>> >>  declare variable $db external;
>> >>  count(collection($db))
>> >>
>> >> _ run-bxcode.xq ___
>> >>
>> >>   import module namespace bxcode = 'bxcode' at 'bxcode.xqm';
>> >>   bxcode:bind-external-variables('query.xq', map { 'db': 'BEP' })
>> >>
>> >> _ output ___
>> >>
>> >>   count(collection(``[BEP]``))
>> >>
>> >> Cheers,
>> >> Christian
>> >>
>> >> [1]
>> >>
>> https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg09863.html
>> >
>> >
>>
>

Re: [basex-talk] Global locks

[Andy Bunce]

Using 9.1.2

let $d:=db:open(``[BEP]``)
return db:create(``[BEP-staging]``,$d,$d!base-uri(.))

The GUI query info reports:
- Read Locking: (none)
- Write Locking: BEP,BEP-staging

I expected BEP-staging to appear only in the Read locking, as it does if I
remove the db:create call
Is this correct?

/Andy


On Thu, 21 Sep 2017 at 16:26, Christian Grün 
wrote:

> > The use of String Constructor does not avoid the global read lock
>
> Good to know, thanks. It now does [1], provided that the constructor
> consists of a single static strings.
>
> [1] http://files.basex.org/releases/latest/
>
>
>
> >  > duration="PT0.007S" reads="(global)"
> > writes="(none)">count(collection(``[BEP]``))
> >
> > On 20 September 2017 at 23:28, Christian Grün  >
> > wrote:
> >>
> >> > bxcode:inline-variables($xqcode-uri) as xs:string
> >> > @returns xquery code that is the source code from $xqcode-uri but with
> >> > inlining "constant" variables and operations applied
> >> >
> >> > This could be used as a preprocessor.
> >> > Is this something that could be created from the bits and pieces in
> the
> >> > current BaseX Java code?
> >>
> >> This reminds me of the discussion with Omar [1]: There is currently no
> >> way to generate a correct string representation for compiled code. In
> >> the GUI Info View, we output a string representation of the optimized
> >> query. In simple cases, this string is a valid and equivalent to the
> >> original query string, but it’s fairly easy to generate invalid
> >> results.
> >>
> >> If it’s an option for you to generate code in XQuery, something like
> >> the following could be done:
> >>
> >> _ bxcode.xqm ___
> >>
> >>   module namespace bxcode = 'bxcode';
> >>   declare function bxcode:bind-external-variables(
> >> $uri   as xs:string,
> >> $vars  as map(xs:string, xs:string)
> >>   ) as xs:string {
> >> let $xqcode := unparsed-text($uri)
> >> return fold-left(
> >>   map:keys($vars), $xqcode, function($string, $name) {
> >> let $value := replace($vars($name), ']``', '`{"]``"}`', 'q')
> >> return $string
> >> => replace('declare variable $' || $name || ' external;', '',
> 'q')
> >> => replace('$' || $name, '``[' || $value || ']``', 'q')
> >>   }
> >> )
> >>   };
> >>
> >> _ example.xq ___
> >>
> >>  declare variable $db external;
> >>  count(collection($db))
> >>
> >> _ run-bxcode.xq ___
> >>
> >>   import module namespace bxcode = 'bxcode' at 'bxcode.xqm';
> >>   bxcode:bind-external-variables('query.xq', map { 'db': 'BEP' })
> >>
> >> _ output ___
> >>
> >>   count(collection(``[BEP]``))
> >>
> >> Cheers,
> >> Christian
> >>
> >> [1]
> >>
> https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg09863.html
> >
> >
>

Re: [basex-talk] restxq

[Andy Bunce]

Hi Giuseppe,

You can use the dba app to see what settings are in use. Goto url
/dba/settings and check WEBPATH.

/Andy


On Fri, 9 Nov 2018 at 13:10, Giuseppe G. A. Celano <
cel...@informatik.uni-leipzig.de> wrote:

> Hi Marco,
>
> I think you are right, but I cannot find the webapp folder which is used
> when I launch basexhttp. I am sure the basexhttp I launch is the one I want
> to launch, but the webapp folder it checks for .xqm files is not the one
> within the "basex" folder containing both of them.
>
>
> On Nov 9, 2018, at 1:34 PM, Marco Lettere  wrote:
>
> It looks like your basexhttp server is pointing to a different directory
> than the one you expect.
> This might happen from time to time depending on type of installation env
> variables and things like that.
> M.
>
> On 09/11/18 13:30, Christian Grün wrote:
>
> …difficult to tell. Could you please provide us with a minimized
> version and tell us the exact steps how to proceed (1. download
> basex91.zip, 2. unzip, etc.)?
> On Fri, Nov 9, 2018 at 1:03 PM Giuseppe G. A. Celano
>  wrote:
>
> I am trying to make a RESTXQ webservice I created with BaseX 8.3 available
> in Basex 9.2. I simply copied my "file.xqm" into the webapp folder but,
> when I type the path of a function contained in it, it does not work ("No
> function found that matches the request."). Strangely enough, when I
> minimally modify restxq.xqm, the modifications do not apply either. Any
> idea about what the problem might be? Thanks.
>
>
>
>
>

Re: [basex-talk] Converting a Date

[Andy Bunce]

Hi Ron,

I took a quick look at Joe's module and made the following changes:

   1. Change datetime:format-date(..,"-MM-dd") to
   format-date(..,"[]-[MM]-[DD]")
   2. *Change xdt*:dayTimeDuration("P1D") -> *xs*:dayTimeDuration
   3. Change function in namespace *local:* to *dates:*

And saved as a gist [1]. Then in BaseX

import module namespace dates = "http://xqdev.com/dateparser; at "
https://gist.githubusercontent.com/apb2006/ee0effdd53ca80daf4f1b3e99794ed89/raw/7eedb8343c3cbf0e4ef0063be5af532e8bbfe485/date-parser.xqm
";

dates:parseDate("March 2017")

Returns

  
2017-03-01
2017-03-31
  
  2017-03-01


Joe,
I think all these changes are compatible with eXist too? [2]

Hope this helps.
/Andy

[1] https://gist.github.com/apb2006/ee0effdd53ca80daf4f1b3e99794ed89
[2] https://sourceforge.net/p/exist/mailman/message/34988159/

On Mon, 5 Nov 2018 at 02:03, Ron Katriel  wrote:

> Hi Joe,
>
> Thanks for sharing that. I tried adapting your eXist port to BaseX and ran
> into issues with namespaces. At this point I don’t have the time or
> expertise to complete this but hopefully someone else will take up the
> challenge.
>
> Best,
> Ron
>
> On November 3, 2018 at 12:19:49 AM, Joe Wicentowski (joe...@gmail.com)
> wrote:
>
> Hi Ron,
>
> You might find Ryan Grimm's date-parser library module useful if you have
> a larger range of date formats to handle:
>
>
> https://github.com/marklogic-community/commons/blob/master/dates/date-parser.xqy
> 
>
> While it was written with some MarkLogic-specific code, I adapted it for
> use with eXist (but haven't tested it with BaseX):
>
>
> https://github.com/HistoryAtState/twitter/blob/master/modules/date-parser.xqm
> 
>
> Best,
> Joe
>
> On Fri, Nov 2, 2018 at 6:48 PM Ron Katriel  wrote:
>
>> Hi Christian,
>>
>> Much appreciated! I hardened the code (see below) since the dates (from
>> CT.gov
>> )
>> occasionally also have the day of the month (e.g., “March 21, 2014”).
>> Currently the function is dropping the day in such cases but I will look
>> into capturing it in a future iteration.
>>
>> Best,
>> Ron
>>
>>
>> declare function local:to-date($string) {
>>   if (fn:matches($string, '[A-Za-z]+ [0-9]+') or fn:matches($string,
>> '[A-Za-z]+ [0-9]+, [0-9]+'))
>>   then
>>   let $m := index-of($MONTHS, substring-before($string, ' '))
>>   let $y := xs:integer(functx:substring-after-last($string, ' '))
>>   return xs:date(string-join(
>> (
>>   format-number($y, ''),
>>   format-number($m, '00'),
>>   '01'
>> ),
>> '-')
>>   )
>>   else
>>   ()
>> };
>>
>> On November 2, 2018 at 4:20:41 PM, Christian Grün (
>> christian.gr...@gmail.com) wrote:
>>
>> Hi Ron,
>>
>> If your timestamp is available in IETF format, you can use
>> fn:parse-ietf-date [1]. Otherwise, you’ll need to write a simple
>> function by yourself:
>>
>> declare variable $MONTHS := (
>> 'January', 'February', 'March', 'April', 'May', 'June',
>> 'July', 'August', 'September', 'October', 'November', 'December'
>> );
>>
>> declare function local:to-date($string) {
>> let $m := index-of($MONTHS, substring-before($string, ' '))
>> let $y := xs:integer(substring-after($string, ' '))
>> return xs:date(string-join((
>> format-number($y, ''),
>> format-number($m, '00'),
>> '01'
>> ), '-'))
>> };
>> local:to-date('March 2017')
>>
>> Best,
>> Christian
>>
>> [1]
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.w3.org_TR_xpath-2Dfunctions-2D31_-23func-2Dparse-2Dietf-2Ddate=DwIFaQ=fi2D4-9xMzmjyjREwHYlAw=44jDQvzmnB_-ovfO6Iusj0ItciJrcWMOQQwd2peEBBE=Yy9sC1xS_Ix-pGSVDp-Lbmz8BOft0S1WdVpEM-qzRw4=uQ3NqCv8FpHulP4q1arjItJX3-gCHwi_06WN4znRz48=
>>
>>
>>
>> On Fri, Nov 2, 2018 at 9:09 PM Ron Katriel  wrote:
>> >
>> > Hi,
>> >
>> > Is there a BaseX function for converting a string date in the form of
>> “March 2017” to xs:date or xs:dateTime?
>> >
>> > Thanks,
>> > Ron
>>
>>

Re: [basex-talk] XML catalog with fetch:xml

[Andy Bunce]

Ok, Thanks for the info it is what I guessed, but I note the "currently" :)

It would be good, long term, to look to matching the Saxon capabilties here
[1], part of which (xslt) are covered by this issue [2].

And related: I wonder sometimes about a "xmldb:" uri scheme and a custom
resolver that could be passed to 3rd party validation and transform
libraries to allow them to access BaseX db resources. I seem to remember a
number of issues on the list that could be related to the saving temp
copies of db resources to the file system as seems to occur currently.

No doubt this would be a big task, if it is even possible :)

/Andy

[1] http://www.saxonica.com/documentation9.5/sourcedocs/xml-catalogs.html
[2] https://github.com/BaseXdb/basex/issues/1405

On 25 September 2018 at 09:28, Christian Grün 
wrote:

> Hi Andy,
>
> Currently, XML catalog files will only be evaluated by the XML parser,
> which will be called after the addressed resource has been located and
> retrieved. In other words: The specified catalog file will be
> evaluated indeed, but only for URIs that occur within the specified
> XML document:
>
>   fetch:xml('file.xml', map {
> 'catfile': '/path/to/catalog.xml',
> 'dtd': true()
>   })
>
> file.xml:
>   
>   
>
> catalog.xml:
>   
>rewritePrefix="file:///path/to/dtd/" />
>   
>
> Hope this helps (a little),
> Christian
>
>
> On Tue, Sep 18, 2018 at 11:13 PM Andy Bunce  wrote:
> >
> > Hi,
> >
> > Using a recent 9.1 Beta on windows.
> > I create an XML catalog at c:\tmp\mycatalog.xml
> >
> > 
> >>   name="http://nowhere.com/doc;
> >   uri="file:///c:/tmp/doc.xml" />
> > 
> >
> > Then in the GUI
> >
> > fetch:xml("http://nowhere.com/doc;,
> > map{"catfile": "C:\tmp\mycatalog.xml"}
> > )
> >
> > It goes to nowhere, I was hoping to get back c:/tmp/doc.xml
> > Should this work?
> >
> > /Andy
> >
>

[basex-talk] XML catalog with fetch:xml

[Andy Bunce]

Hi,

Using a recent 9.1 Beta on windows.
I create an XML catalog at c:\tmp\mycatalog.xml


  http://nowhere.com/doc;
  uri="file:///c:/tmp/doc.xml" />


Then in the GUI

fetch:xml("http://nowhere.com/doc;,
map{"catfile": "C:\tmp\mycatalog.xml"}
)

It goes to nowhere, I was hoping to get back c:/tmp/doc.xml
Should this work?

/Andy

Re: [basex-talk] Programmatically extracting function signatures

[Andy Bunce]

Sounds good. Of course I meant the etc/ folder not lib/.
/Andy

On 17 September 2018 at 17:28, Bridger Dyson-Smith 
wrote:

> Andy -
> thank you for that suggestion!
>
> On Mon, Sep 17, 2018 at 12:13 PM Andy Bunce  wrote:
>
>> If you look in lib/ folder the file modules.zip contains the BaseX
>> modules as .xqm stub files with functions marked as external.
>> The entry for archive:delete is shown below. At one point I am sure these
>> *.xqm could be parsed by functions like inspect:module and inspect:xqdoc
>> Currently they raise errors like [XQST0045] Function '...' is in reserved
>> namespace.
>>
>> I believe were generated by scraping the wiki :)
>>
>> However:
>> * some of these are a little out of date (at least in 9.02)
>> * modules.zip is not present in recent 9.1 betas
>>
>> This is a shame as I hoped to use to one day use these in some kind of
>> IDE for content assist.
>> (although I have hoped that for a long time without actually doing it.)
>>
>> This is exactly the use case I have. I'm attempting to help work on an
> IntelliJ plugin that will leverage the signatures (for things I don't quite
> understand :).
>
> Regards
>> /Andy
>>
>> (:~
>>  : Deletes entries from an $archive .
>>  : The format of $entries is the same as for > href="#archive:create">archive:create .
>>  :
>>  : @error bxerr:ARCH0005 the entries of the given archive cannot be
>> modified.
>>  : @error bxerr:ARCH archive creation failed for some other reason.
>>  :)
>> declare function archive:delete($archive as xs:base64Binary, $entries as
>> item()*) as xs:base64Binary external;
>>
>> Thanks again for the suggestion.
> Best,
> Bridger
>
>
>> On 17 September 2018 at 16:43, Bridger Dyson-Smith > > wrote:
>>
>>> Hi all -
>>>
>>> I was curious if there was a way to extract function signatures from
>>> BaseX. I'm currently reading through the (excellent) wiki but was curious
>>> if there was a more automated process I might be able to use. I've been
>>> trying to think of something clever with function-lookup, but I can't seem
>>> to make it dynamic based on module namespace; e.g.
>>>
>>> ```
>>> for $fun in function-lookup(xs:QName("archive:delete"), 1)
>>> return
>>>   $fun
>>> ```
>>>
>>> I may just grab the wiki and parse it :), but I'm also interested in
>>> changes between versions of the processor.
>>>
>>> Any thoughts or suggestions would be greatly appreciated!
>>> Best,
>>> Bridger
>>>
>>
>>

Re: [basex-talk] Programmatically extracting function signatures

[Andy Bunce]

If you look in lib/ folder the file modules.zip contains the BaseX modules
as .xqm stub files with functions marked as external.
The entry for archive:delete is shown below. At one point I am sure these
*.xqm could be parsed by functions like inspect:module and inspect:xqdoc
Currently they raise errors like [XQST0045] Function '...' is in reserved
namespace.

I believe were generated by scraping the wiki :)

However:
* some of these are a little out of date (at least in 9.02)
* modules.zip is not present in recent 9.1 betas

This is a shame as I hoped to use to one day use these in some kind of IDE
for content assist.
(although I have hoped that for a long time without actually doing it.)

Regards
/Andy

(:~
 : Deletes entries from an $archive .
 : The format of $entries is the same as for archive:create .
 :
 : @error bxerr:ARCH0005 the entries of the given archive cannot be
modified.
 : @error bxerr:ARCH archive creation failed for some other reason.
 :)
declare function archive:delete($archive as xs:base64Binary, $entries as
item()*) as xs:base64Binary external;

On 17 September 2018 at 16:43, Bridger Dyson-Smith 
wrote:

> Hi all -
>
> I was curious if there was a way to extract function signatures from
> BaseX. I'm currently reading through the (excellent) wiki but was curious
> if there was a more automated process I might be able to use. I've been
> trying to think of something clever with function-lookup, but I can't seem
> to make it dynamic based on module namespace; e.g.
>
> ```
> for $fun in function-lookup(xs:QName("archive:delete"), 1)
> return
>   $fun
> ```
>
> I may just grab the wiki and parse it :), but I'm also interested in
> changes between versions of the processor.
>
> Any thoughts or suggestions would be greatly appreciated!
> Best,
> Bridger
>

Re: [basex-talk] First non-null value?

[Andy Bunce]

 If you put your "possibles" in an array rather than a sequence  then the
index of the first non-empty item
identifies the match.

let $results := [$possible1,$possible2,$possible3,$possible4,$
possible5,'FAILED']

let $index:= array:fold-left($results,
   -1,
   function($acc,$this){
  if($acc gt 0)then $acc else
  if (exists(array:get($results,-$acc))) then
-$acc else $acc -1
  })

let $foundIt:= array:get($results,$index)

This seems a bit tricksy, using the BaseX specific higher order function
hof:until [1] is cleaner

let $index:= hof:until(
  function($index){ exists(array:get($results,$index)) },
  function($index){ $index+1 },
  1)

/Andy

[1] http://docs.basex.org/wiki/Higher-Order_Functions_Module#hof:until


On 14 September 2018 at 07:38, Liam R. E. Quin 
wrote:

> On Thu, 2018-09-13 at 16:18 -0400, Graydon Saunders wrote:
> > let $possible1 as xs:string* := (: go looking for a value via one
> > route :)
> > let $possible2  (: all the other routes in preference order :)
> > 
> > let $foundIt as xs:string :=
> > ($possible1,$possible2,$possible3,$possible4,$possible5,'FAILED')[1]
> >
> > This works nicely in terms of "I got the value by the least-bad
> > route".
> > What I'm blanking on is "how do I tell which was the first
> > possibility to
> > match?" without resorting to sprawl of if-then-else statements.  I
> > have the
> > idea that there must be a compact way but I have no idea what it is
> > if
> > there is.
>
> Don't use variables - just construct a sequence,
> let $possibles as xs:string := (
>stuff to make possible1,
>stuff to make possible 2,
>. . .
>'fallback value'
>)[1]
> >
>
> Liam
>
>
> --
> Liam Quin, https://www.holoweb.net/liam/cv/
> Web slave for vintage clipart http://www.fromoldbooks.org/
> Available for XML/Document/Information Architecture/
> XSL/XQuery/Web/Text Processing/A11Y work/training/consulting.
>
>

Re: [basex-talk] Example xquery needed to write output to an local xml file

[Andy Bunce]

Hi Marco,

BaseX sounds like a good fit for your requirements to me :)

You dont say what your error is, but you have declared local:process-data
as returning an element named  whereas the code is returning an
element named .
Perhaps it should be:

declare function local:process-data ($nodes as item()*) as element (
RMCollectionItemsDATA )
or
... as element ( * )

Also I suspect you want an additional \ in your o/p path
declare variable $output := 'C:\DATAQUERY\BaseX\Q001\outputPROD*\*';

Regards
/Andy
---
declare variable $data := collection(PRODRMCollectionItems);
declare variable $output := 'C:\DATAQUERY\BaseX\Q001\outputPROD*\*';

(:This is where all the work is happening:)
declare function local:process-data ($nodes as item()*) as element (
*RMCollectionItemsDATA*)
{
  
  {
 for $a in $nodes//artObjectGetResponse/artObject[objectNumber eq
'NG-NM-1395-A']
   return
{$a}
  }
  
};

(:here we store stuff:)
let $filename := concat($output, "marco.xml")
return
  file:write($filename, local:process-data($data))


On 6 September 2018 at 16:13, Marco Roling  wrote:

> Dear all,
>
> I am new on the block here, and trying to find my way around BaseX, which
> seems to meet my needs for my research on digital museum collection data
> analysis.
>
> I have a very large collection of xml files (> 60 files) that I am
> querying, and I want to write the returned result immediately in a file. The
> following xquery (see below) is not working, which probably comes as no
> surprise to you all.
>
> But to get me started, I would like to have a simple example that works.
> Any example would do.
>
> Hope anyone can help me out with this.
>
> Thanks very much, best, Marco.
>
>
> =
>
> declare variable $data := collection(PRODRMCollectionItems);
> declare variable $output := 'C:\DATAQUERY\BaseX\Q001\outputPROD';
>
> (:This is where all the work is happening:)
> declare function local:process-data ($nodes as item()*) as element (root)
> {
>   
>   {
>  for $a in $nodes//artObjectGetResponse/artObject[objectNumber eq
> 'NG-NM-1395-A']
>return
> {$a}
>   }
>   
> };
>
> (:here we store stuff:)
> let $filename := concat($output, "marco.xml")
> return
>   file:write($filename, local:process-data($data))
> =
>
>

Re: [basex-talk] Different interpretation of regex in eXist, Saxon and BaseX

[Andy Bunce]

Great!
I believe the "!" option is best ignored...:)

>Note: On the Java platform, this can also be achieved using the flag "!";
this was never formally supported and is likely to be withdrawn in a future
Saxon version. [1]

/Andy
[1] https://www.saxonica.com/html/documentation/functions/fn/matches.html[1]

On 9 August 2018 at 18:54, Christian Grün  wrote:

> > A new snapshot will be available later tonight.
>
> …which is now.
>
> On Thu, Aug 9, 2018 at 7:02 PM Christian Grün 
> wrote:
> >
> > > +1 for the Java flag as this enables \b for word boundaries as
> mentioned here [1]
> >
> > True, I missed that one as well more than once.
> >
> > I’ve just support for Java’s default parser [1,2]. Apart from 'j'
> > (which doesn’t need to be prefixed with a semicolon, as in Saxon), '!'
> > is available as alternative. As it’s not officially documented in
> > Saxon, just keep this one as a secret :)
> >
> > A new snapshot will be available later tonight.
> >
> > [1] https://github.com/BaseXdb/basex/issues/1608
> > [2] http://docs.basex.org/wiki/XQuery_Extensions#Regular_expressions
>

Re: [basex-talk] Schematron package error

[Andy Bunce]

 Hi Dharmendra,

Thanks for sending me the files. They run without error for me.
I was using BaseX 9.0.2 with Saxon saxon9he-9.7.0.20.jar

Using the GUI on windows produced an out of memory until I increased xms in
the bat file:
set BASEX_JVM=-Xmx2400m %BASEX_JVM%

The schematron you are using requires xslt2 so having Saxon in the path is
important.

I suggest you create a new BaseX 9.0.2 installation. Install saxon and the
schematron package
and try running in that.

Regards
/Andy


On 22 July 2018 at 19:03, Imsieke, Gerrit, le-tex 
wrote:

> I’m afraid we cannot help you without seeing the actual Schematron file
> that you are trying to compile/apply.
>
> On 22.07.2018 19:59, DK Singh wrote:
>
>> Thank U Andy for your response, I have tried but it is giving schematron
>> compilation error, but when I validate on Oxygen editor xmls got validated,
>> as github documentation I am doing first schematron compilation then
>> validating the XML, SO how can I resolve this error Now
>>
>> On Sun 22 Jul, 2018, 9:39 PM James Ball, > m...@jamesball.co.uk>> wrote:
>>
>> Hello Dharmendra,
>>
>> Have you made sure that the Schematron is compiled before using it
>> to validate? I think this is the error you get if you try to use an
>> un-compiled Schematron file to perform a validation.
>>
>> The necessary steps are outlined with the module documentation on
>> GitHub and I had it working successfully just a week ago.
>>
>> Regards, James
>>
>>  > Date: Fri, 20 Jul 2018 17:07:30 +0530
>>  > From: DK Singh > dharam.m...@gmail.com>>
>>  > To: BaseX > >
>>  > Subject: [basex-talk] Schematron package error
>>  > Message-ID:
>>  >   > twju9kw...@mail.gmail.com >
>>  > Content-Type: text/plain; charset="utf-8"
>>  >
>>  > Hi All,
>>  >
>>  > I am doing the schematron validation against XML docuemnt but i
>> am getting
>>  > these errors.
>>  >
>>  > Bad name element: XPath error. No XPath. Bad name element: XPath
>> error. No
>>  > XPath. Error on line 1409 column 61 of iso_dsdl_include.xsl: Too
>> many
>>  > nested template or function calls. The stylesheet may be looping.
>> at
>>  > xsl:call-template name="sch-check:strip-strings"
>>  >
>> (file:///C:/Program%20Files%20(x86)/BaseX/repo/http-github.
>> com-Schematron-schematron-basex-1.2/content/iso-
>> schematron/iso_dsdl_include.xsl#1409)
>>  >
>>  >
>>  > it looks like basex schematron pacakge throwing the error to run
>> schematron
>>  > validation.
>>  > can anyone suggest how can i resolve this errors.
>>  >
>>  > Regards
>>  > Dharmendra Kumar Singh
>>  > -- next part --
>>  > An HTML attachment was scrubbed...
>>  > URL:
>> > ents/20180720/3801dbc8/attachment-0001.html>
>>  >
>>
>>
> --
> Gerrit Imsieke
> Geschäftsführer / Managing Director
> le-tex publishing services GmbH
> Weissenfelser Str. 84, 04229 Leipzig, Germany
> Phone +49 341 355356 110, Fax +49 341 355356 510
> gerrit.imsi...@le-tex.de, http://www.le-tex.de
>
> Registergericht / Commercial Register: Amtsgericht Leipzig
> Registernummer / Registration Number: HRB 24930
>
> Geschäftsführer / Managing Directors:
> Gerrit Imsieke, Svea Jelonek, Thomas Schmidt
>

Re: [basex-talk] Schematron package error

[Andy Bunce]

Hi DK,

This runs ok for me..

import module namespace schematron = "
http://github.com/Schematron/schematron-basex;;
let $sch := schematron:compile(doc('
https://raw.githubusercontent.com/apb2006/vue-poc/master/src/vue-poc/static/resources/schematron/ark.sch'
))
let $xml := fn:doc('
https://raw.githubusercontent.com/apb2006/vue-poc/master/src/vue-poc/static/resources/schematron/ark.xml
')
let $validation := schematron:validate($xml, $sch)

return $validation

Maybe there is an issue with your source files?

Regards
/Andy

On 20 July 2018 at 13:25, DK Singh  wrote:

> HI  Christial,
>
> i am doing validation like this:
>
> import module namespace schematron = "http://github.com/Schematron/
> schematron-basex";
>
> let $sch := schematron:compile(doc('D:\2018\schemas\docbook-mods.sch'))
> let $xml := fn:doc('D:\2018\InthisIssue66.xml')
> let $validation := schematron:validate($xml, $sch)
>
> return $validation
>
>
>
> then it is giving thease kind of errors.
>
> Error:
> Stopped at C:/Program Files (x86)/BaseX/repo/http-github.
> com-Schematron-schematron-basex-1.2/content/schematron.xqm, 39/31:
> [bxerr:BXSL0001] Bad name element: XPath error. No XPath.
> Bad name element: XPath error. No XPath.
> Bad name element: XPath error. No XPath.
> Error on line 1409 column 61 of iso_dsdl_include.xsl:
>   Too many nested template or function calls. The stylesheet may be
> looping.
>   at xsl:call-template name="sch-check:strip-strings"
> (file:///C:/Program%20Files%20(x86)/BaseX/repo/http-github.com-Schematron-
> schematron-basex-1.2/content/iso-schematron/iso_dsdl_include.xsl#1409)
>   at xsl:call-template name="sch-check:strip-strings"
> (file:///C:/Program%20Files%20(x86)/BaseX/repo/http-github.com-Schematron-
> schematron-basex-1.2/content/iso-schematron/iso_dsdl_include.xsl#1409)
>   at xsl:call-template name="sch-check:strip-strings"
> (file:///C:/Program%20Files%20(x86)/BaseX/repo/http-github.com-Schematron-
> schematron-basex-1.2/content/iso-schematron/iso_dsdl_include.xsl#1409)
>   at xsl:call-template name="sch-check:strip-strings"
> (file:///C:/Program%20Files%20(x86)/BaseX/repo/http-github.com-Schematron-
> schematron-basex-1.2/content/iso-schematron/iso_dsdl_include.xsl#1409)
>   at xsl:call-template name="sch-check:strip-strings"
> (file:///C:/Program%20Files%20(x86)/BaseX/repo/http-github.com-Schematron-
> schematron-basex-1.2/content/iso-schematron/iso_dsdl_include.xsl#1409)
>
> Regards
> Dharmendra Kumar Singh
>
>
>
>
> On Fri, Jul 20, 2018 at 5:12 PM Christian Grün 
> wrote:
>
>> As usual, send us a little reproducible example. Thanks.
>>
>> DK Singh  schrieb am Fr., 20. Juli 2018, 13:39:
>>
>>> Hi All,
>>>
>>> I am doing the schematron validation against XML docuemnt but i am
>>> getting these errors.
>>>
>>> Bad name element: XPath error. No XPath. Bad name element: XPath error.
>>> No XPath. Error on line 1409 column 61 of iso_dsdl_include.xsl: Too many
>>> nested template or function calls. The stylesheet may be looping. at
>>> xsl:call-template name="sch-check:strip-strings"
>>> (file:///C:/Program%20Files%20(x86)/BaseX/repo/http-
>>> github.com-Schematron-schematron-basex-1.2/content/
>>> iso-schematron/iso_dsdl_include.xsl#1409)
>>>
>>> it looks like basex schematron pacakge throwing the error to run
>>> schematron validation.
>>> can anyone suggest how can i resolve this errors.
>>>
>>> Regards
>>> Dharmendra Kumar Singh
>>>
>>

Re: [basex-talk] DBA session timeout

[Andy Bunce]

Hi Omar,

I think you want idleTimeout in jetty.xml [1]

Regards
/Andy

https://github.com/BaseXdb/basex/blob/master/basex-api/src/main/webapp/WEB-INF/jetty.xml#L14

On 26 June 2018 at 14:50, Omar Siam  wrote:

> Hi list!
>
> I just got asked if I can change the session timeout for DBA. I actually
> can't. I don't know where to look. Can anybody can give me a hint?
>
> Best regards
>
> Omar Siam
>
>

Re: [basex-talk] XML well-formed error

[Andy Bunce]

Hi  Dharmendra Kumar,

When I run this in Oxygen, loading the result into an editor window, I get
the same message that BaseX  gives:

Description: The markup in the document following the root element must be
well-formed.
Start location: 1:68199
URL:
http://www.saxonica.com/html/documentation/javadoc/net/sf/saxon/trans/SaxonErrorCode.html#SXXP0003


Your transform is not generating a valid XML document.

Regards
/Andy

On 6 June 2018 at 11:07, DK Singh  wrote:

> Hi Christian for your reference i am attaching the XSLT and XML which i am
> trying to transfom using BaseX GUI
>
> QUERY
> =
>
> let $xml := fn:doc('D:\2018\VCLP-A2 - admin.xml')
> let $xsl := fn:doc('D:\2018\DBK2BloomsburyDbk.xslt')
> return  xslt:transform($xml,$xsl)
>
> Error
> ==
> Error:
> Stopped at D:/2017/BLOOMSBURRY/testing-code/eBloomsbury/file, 3/23:
> [FODC0002] "" (Line 1): The markup in the document following the root
> element must be well-formed.
>
> you can try out to simply run the query on BaseX GUI , i have attached the
> XML and  XSLt
>
> Regards
> Dharmendra Kumar Sngh
>
>
>
> On Wed, Jun 6, 2018 at 3:24 PM, Christian Grün 
> wrote:
>
>> Hi
>> ​
>> Dharmendra Kumar.
>>
>> P
>> lease provide us with a little self-contained example (see
>> http://sscce.org/ for more information).
>>
>> ​
>> Thanks in advance ;)
>> Christian
>>
>>
>>
>> On Wed, Jun 6, 2018 at 11:53 AM, DK Singh  wrote:
>>
>>> Hi Christian,
>>> i am running the simple query on Basex GUI
>>>
>>> let $xml := fn:doc('D:\2018\VCLP-A2 - admin.xml')
>>> let $xsl := fn:doc('D:\2018\DBK2BloomsburyDbk.xslt')
>>> return  xslt:transform($xml,$xsl)
>>> Regards
>>> ​​
>>> Dharmendra Kumar Singh
>>>
>>> On Wed, Jun 6, 2018 at 2:49 PM, Christian Grün <
>>> christian.gr...@gmail.com> wrote:
>>>
 Hi ​Dhamendra Kumar,​

 As usual,
 ​​
 please provide us with a little example (see http://sscce.org/ for
 more information).

 ​​
 Thanks in advance,
 Christian


 On Wed, Jun 6, 2018 at 11:16 AM, DK Singh 
 wrote:

> Hi All,
> I have a XML file when i apply xslt using xslt:transform function it
> is giving well formed error, even i tried on BaseX UI  using
> xslt:transform function
>
> [FODC0002] "" (Line 1): The markup in the document following the root
> element must be well-formed.
>
> but when the same file i transform on the oxygeng editor it getting
> transformed, can anyone suggest what going wrong.
> Regards
> Dharmendra Kumar Singh
>


>>>
>>
>

Re: [basex-talk] Serialize csv xquery map

[Andy Bunce]

Ah, I thought I had tried that, but it is working now.

Thanks
/Andy

On 29 May 2018 at 16:05, Christian Grün  wrote:

> Hi Andy,
>
> You can serialize the map by specifying the 'xquery' format as option:
>
>   map {
> "records": ["Huber", "Sepp", "Hauptstraße 13", "93547 Hintertupfing"],
> "names": ["Name", "First_Name", "Address", "City"]
>   }
>   => csv:serialize(map { 'format': 'xquery', 'header': true() })
>
> The result:
>
>   ​Name,First_Name,Address,City
>   Huber,Sepp,Hauptstraße 13,93547 Hintertupfing
> ​
> ​Cheers,
> Christian
> ​
>
>
>
>
> On Tue, May 29, 2018 at 4:43 PM, Andy Bunce  wrote:
>
>> The xqueryformat of CSV records as a sequence of arrays [1] is very
>> useful when parsing
>> but there seems to be no easy way to serialize from this format. Is this
>> correct?
>>
>> map {
>>   "records": ["Huber", "Sepp", "Hauptstraße 13", "93547 Hintertupfing"],
>>   "names": ["Name", "First_Name", "Address", "City"]
>> }
>> =>csv:serialize()
>>
>> [SENR0001] Items of type map(xs:string, array(xs:string)) cannot be
>> serialized.
>>
>> ... would be nice if it could.
>> /Andy
>>
>> [1] http://docs.basex.org/wiki/CSV_Module#XQuery
>>
>
>

[basex-talk] Serialize csv xquery map

[Andy Bunce]

The xqueryformat of CSV records as a sequence of arrays [1] is very useful
when parsing
but there seems to be no easy way to serialize from this format. Is this
correct?

map {
  "records": ["Huber", "Sepp", "Hauptstraße 13", "93547 Hintertupfing"],
  "names": ["Name", "First_Name", "Address", "City"]
}
=>csv:serialize()

[SENR0001] Items of type map(xs:string, array(xs:string)) cannot be
serialized.

... would be nice if it could.
/Andy

[1] http://docs.basex.org/wiki/CSV_Module#XQuery

Re: [basex-talk] GUI

[Andy Bunce]

 Hi Giuseppe,

It has been moved to be the 1st button on the toolbar "New"(or ctl-T)

/Andy

On 16 April 2018 at 09:55, Giuseppe Celano  wrote:

> I see that in the 9.0 version the "+ button" to add a new tab is missing.
> I think it was very useful: can it be re-introduced in the following
> releases?
>
> Best,
> Giuseppe
>
>
> Universität Leipzig
> Institute of Computer Science
> Augustusplatz 10
> 04109 Leipzig
> Deutschland
> E-mail: cel...@informatik.uni-leipzig.de
> E-mail: giuseppegacel...@gmail.com
> Web site 1: http://www.dh.uni-leipzig.de/wo/team/
> Web site 2: https://sites.google.com/site/giuseppegacelano/
>
>

Re: [basex-talk] SSL support for BaseX REST API

[Andy Bunce]

I have been trying this out recently, in part to look at service workers
[1]
I am using  BaseX 9 betas for this. This has a newer jetty version.
I have put a jetty.xml that is working for me as a gist [2]

You will need to change the keystore location [3] to something that points
to your keystore.
It is based on the jetty9 article [4]

/Andy

https://w3c.github.io/ServiceWorker/
https://gist.github.com/apb2006/b24e92f84c42838ec7ef7de2cf937835
https://gist.github.com/apb2006/b24e92f84c42838ec7ef7de2cf937835#file-jetty-xml-L57
https://www.blackpepper.co.uk/what-we-think/blog/jetty-runner-https-xml-configuration


On 14 March 2018 at 20:59, Liam R. E. Quin  wrote:

> On Wed, 2018-03-14 at 14:18 -0500, Giavanna J Richards wrote:
> > I'm trying to determine how to enable SSL communications with the
> > BaseX server
>
> I don't know if this helps, but I run BaseX listening only to
> "localhost" so that SSL isn't an issue (as a connection to localhost
> doesn't normally go over a network), and connect (on the same system)
> from PHP or Perl (!) or you can proxy via apache.
>
> If BaseX is running on a different computer, you could also proxy on
> the system running BaseX e.g. with apache and .htaccess or the server
> conf & mod_rewrite. That way you'd use SSL to get to apache and then an
> in-memory connection from there to BaseX.
>
> Liam
>
> --
> Liam Quin, W3C, http://www.w3.org/People/Quin/
> Staff contact for Verifiable Claims WG, SVG WG, XQuery WG
> Improving Web Advertising: https://www.w3.org/community/web-adv/
> Personal: awesome vintage art: http://www.fromoldbooks.org/
>

Re: [basex-talk] bug in XSLT

[Andy Bunce]

 Your stylesheet returns html. The html serialization adds the meta tag
without a closing tag.
 xslt:transform must return node() so fails trying to extract XML from that.

Maybe xslt:transform-text[1] can help, changing your last line to

return xslt:transform-text(/, $s)
Gives
 
   
  
   
   


/Andy

[1] http://docs.basex.org/wiki/XSLT_Module#xslt:transform-text

On 23 February 2018 at 09:54, Kristian Kankainen 
wrote:

> You could use the html module to convert the html into xml. Read about the
> html module here: http://docs.basex.org/wiki/HTML_Module
>
> Br
> Kristian K
> 23. veebr 2018 11:31 AM kirjutas kuupäeval nikos dimitrakas <
> ni...@dsv.su.se>:
>
> There seems to be a problem when using baseX to create html with XSLT.
> Saxon seems to add a META-tag if the head-tag is present. But it is not
> . It is instead  (without closing tag, which is ok in html)
> and this causes baseX to give an error:
> The element type "META" must be terminated by the matching end-tag
> "". With the newer version of Saxon the error is the same but the
> case has been changed to  instead of .
>
> This can be easily reproduced with the following query:
>
> let $s :=
> http://www.w3.org/
> 1999/XSL/Transform">
>   
>   
> 
>   
> 
> return xslt:transform(/, $s)
>
> In the new XSLT version there is an option to set the output to xhtml and
> that works, but the html output (which is the only available option in XSLT
> 1) fails. The generated meta tag seem to be this
>
> 
>
> I have even tried to override this by specifying that meta tag myself with
> closing tag but that does not work.
> I even tried setting the attribute include-content-type in the xsl:output,
> but it had no effect.
> Setting the output method to xml would work if the html genererated is in
> fact correct xml (like in the example above).
> Output metod "text" seems to also behave strangely. Maybe BaseX ignores
> the output method and tries to treat everything as xml?
>
> /nikos
>
>

Re: [basex-talk] An illegal reflective access operation has occurred

[Andy Bunce]

Hi Massimo,
Seems to be a common problem running with Java 9.
This link [1] suggests some command line work arounds (untested by me)

/Andy

[1]
https://blog.codefx.org/java/java-9-migration-guide/#Illegal-Access-To-Internal-APIs

On 29 November 2017 at 11:25, Massimo Franceschet <
massimo.francesc...@gmail.com> wrote:

> No, it is not running.
>
> Best,
>
> Massimo
>
> Il 28 Nov 2017 17:18, "Christian Grün"  ha
> scritto:
>
>> Hi Massimo,
>>
>> It seems Mac OSX support is getting more and more difficult with each
>> Java release.
>> Could you still run BaseX in spite of the returned warnings?
>>
>> Best,
>> Christian
>>
>>
>>
>> On Tue, Nov 28, 2017 at 7:52 AM, Massimo Franceschet
>>  wrote:
>> > Hi.
>> >
>> > I am trying BaseX version 8.6.7 with java version “9.0.1” on a macOS
>> Sierra,
>> > planning to show it in classroom. When running the GUI I am getting the
>> > following error:
>> >
>> > WARNING: An illegal reflective access operation has occurred
>> > WARNING: Illegal reflective access by org.basex.gui.GUIMacOSX
>> > (file:/Applications/basex/basex/BaseX.jar) to method
>> > com.apple.eawt.Application.getApplication()
>> > WARNING: Please consider reporting this to the maintainers of
>> > org.basex.gui.GUIMacOSX
>> > WARNING: Use --illegal-access=warn to enable warnings of further illegal
>> > reflective access operations
>> > WARNING: All illegal access operations will be denied in a future
>> release
>> > Failed to initialize native Mac OS X interface
>> >
>> > Thanks for help.
>> >
>> > Massimo
>>
>

Re: [basex-talk] bat problem with latest builds v9

[Andy Bunce]

Also seems to be a scambling of xquery and sql modules.
Code and doc http://docs.basex.org/wiki/XQuery_Module
/Andy

On 22 November 2017 at 18:45, Andy Bunce <bunce.a...@gmail.com> wrote:

>
> Hi,
>
> Bat files for reecent builds are not running for me and contain a line
> like:
> set CP=%MAIN%\target\classes;%MAIN%\lib\*;%MAIN%\lib\custom\*
>
> Looks like a substition is not happening somewhere. I think should be:
>
> set CP=%MAIN%\BaseX.jar;%MAIN%\lib\*;%MAIN%\lib\custom\*
>
> /Andy
>

[basex-talk] bat problem with latest builds v9

[Andy Bunce]

Hi,

Bat files for reecent builds are not running for me and contain a line like:
set CP=%MAIN%\target\classes;%MAIN%\lib\*;%MAIN%\lib\custom\*

Looks like a substition is not happening somewhere. I think should be:

set CP=%MAIN%\BaseX.jar;%MAIN%\lib\*;%MAIN%\lib\custom\*

/Andy

Re: [basex-talk] basex-node binding objects

[Andy Bunce]

Hi E. Wray,

1. basex-node results are always strings but you can use JSON.parse on that
string, if that is appropriate because for example of the serialization
used.
See [1],[2] for examples
2. Do you mean you would like query.bind [3] to automatically convert
"undefined" to ()?
If so, or indeed if you mean something different, best to create an issue
in the basex-node project on github.

Regards
/Andy

[0] https://github.com/apb2006/basex-node
[1] https://github.com/apb2006/basex-node/blob/master/examples/json.js
[2] https://github.com/apb2006/basex-node/blob/master/examples/json2.js
[3]
https://github.com/apb2006/basex-node/blob/master/docs/commands.md#the-query-object


On 15 November 2017 at 16:17, E. Wray Johnson 
wrote:

> Andy,
>
>
>1. I am pelase that JSON result objects are mapped to JavaScript
>objects.  Are undefined values handled (e.g. XQuery () vs JavaScript
>undefined)?
>2. I am using basex-node and having trouble binding objects in a
>query.  I would like to have the object result in an XQUERY map.  It would
>be nice if undefined values as defined above are handled as well.  For now
>I have to do something like this'$x := map ' 
> +JSON.stringify(obj).replace('undefined',
>'()').
>
> I hope this make sense.
>

Re: [basex-talk] BaseXdb/basex] Packaging: Java and XQuery code (#1413)

[Andy Bunce]

Hi Tim,

Not sure if this is your problem here, but I think the xslt element uses
 rather than . e.g

 
  http://www.functx.com/functx.xsl
  functx.xsl
   

/Andy


On 24 October 2017 at 15:16, Tim Thompson  wrote:

> Thanks, Christian. I had a quick question about the expath-pkg.xml file
> (attached). I have two XSLT files that I want to list separately, along
> with listing Saxon-HE (version 9.8.x) as a dependency. However, when I add
> a second dependency element and try to import the package, I get an error:
>
> [bxerr:BXRE0009] Package version is not supported.
>
> Looking at the EXPath package schema[1], it seems as though this should be
> valid (the dependency element should be repeatable).
>
> Tim
>
> [1] http://expath.org/spec/pkg#structure
>
> --
> Tim A. Thompson
> Discovery Metadata Librarian
> Yale University Library
>
>
> On Fri, Oct 20, 2017 at 2:25 PM, Christian Grün  > wrote:
>
>> Hi Tim,
>>
>> Thanks for the link to your RDF library.
>>
>> I’m glad to hear that the new packaging mechanism has already found
>> watchers. I have just uploaded a new snapshot, and I would be glad to
>> have your feedback as first tester ;)
>>
>> > but I wanted to be sure I was going about it correctly because it still
>> > seems a little like magic to me. I am also wondering whether this new
>> > feature is something I might want to take advantage of.
>>
>> I haven’t run any queries yet with your module, but the package
>> structure looks fine.
>>
>> Our experience over the years is that the EXPath packaging mechanism
>> is a bit difficult to understand, and not as portable as we believe it
>> could be. It would be nice, for example, if a package could be
>> distributed without implementation-specific details inside. But it’s
>> always easy to complain, and it takes a lot of time to get things
>> going.
>>
>> For our own use cases, the simple packaging mechanism is just good
>> enough. It would be nice if we could add some versioning support in
>> the future, though.
>>
>> Cheers
>> Christian
>>
>>
>>
>> > The module (https://github.com/timathom/basex-rdf) uses Günther
>> Rademacher's
>> > REx Parser Generator[1] to create a Java parser for the RDF TriG/Turtle
>> > syntax[2], using the BaseX extension function option.
>> >
>> > Following the BaseX documentation[3], I have created a XAR package that
>> > includes the JAR file and an XQuery wrapper. The wrapper calls the
>> function
>> > exposed by the Java module (import module namespace trig =
>> > "org.basex.modules.rdf.TriG";) and then processes the raw parse tree,
>> > overlaying some additional abstractions (I confess that I cheated and
>> used
>> > XSLT for most of the processing functionality).
>> >
>> > Because of the built-in BaseX extension function, this approach seems
>> > different from a straightforward "combined" module approach, but,
>> again, I
>> > wanted to ask whether the way I'm doing this seems correct.
>> >
>> > Thanks in advance.
>> >
>> > Tim
>> >
>> > [1] http://www.bottlecaps.de/rex/
>> > [2] https://www.w3.org/TR/trig/
>> > [3] docs.basex.org/wiki/Repository
>> >
>> > --
>> > Tim A. Thompson
>> > Discovery Metadata Librarian
>> > Yale University Library
>> >
>>
>
>

Re: [basex-talk] proc:system ?

[Andy Bunce]

Hi Jean-Marc,

I have hit this kind of issue before and it is not BaseX specific. The
problem is Windows(tm) limits [1]
This is compounded by the Java call from proc:system passing in a copy of
the environment of the current process [2].
Mine has built up a large class path over time with my lib/custom
experiements. My  `proc:property("java.class.path")` is nearly 4k

You could try to "clean" your classpath and switch your file locations to
shorter paths

/Andy

[1] https://support.microsoft.com/en-gb/help/830473/command-prom
pt-cmd--exe-command-line-string-limitation
[2] https://docs.oracle.com/javase/7/docs/api/java/lang/ProcessBuilder.html#
environment()

On 12 October 2017 at 11:19, jean-marc Mercier 
wrote:

> Hello BaseX team,
>
> I am an extensive user of the proc:system BaseX command (and looking
> forward for proc:fork !).
>  I am using this command with in-memory arguments, it means that my way of
> using it is :
> proc:system("my_exe", $my_arguments_for_exe)
>
> $my_arguments_for_exe is an xml node serialized to string by BaseX. As my
> test are getting more complex, the size of $my_arguments_for_exe are
> increasing. It seems that we are reaching a limit now, since the compiler
> refuses to pass arguments :
>
> "Stopped at CoDeFi_Interface.xqm, 44/22:
> [BXPR] Cannot run program "C:\Jiheme\scientifique\
> recherche\CoDeFi\CoDeFi\CoDeFi.exe": CreateProcess error=206, Nom de
> fichier ou extension trop long
> "
> Is there a way to bypass this ?
>
> Thank you for your answer,
>
> Cheers,
>
> Jean-Marc
>

[basex-talk] put with relative uri

[Andy Bunce]

Hi,

I notice fn:put() appears to be resolving uris relative to the working
directory, as the file module does, rather than static-base-uri() as
fn:doc() does.
I dont think it was always like this and it seems to against the intent of
the spec[1]?

/Andy


[1] The external effects of upd:put are implementation-defined, since they
occur outside the domain of XQuery. The intent is that, if upd:put is
invoked on a document node and no error is raised, a subsequent query can
access the stored document by invoking fn:doc with the same URI.
https://www.w3.org/TR/xquery-update-10/#id-upd-put

fn:put("test.xml")
---
fn:doc("test.xml")  (: [FODC0002] Resource 'C:/.../test.xml' does not
exist. :)

Re: [basex-talk] Global locks

[Andy Bunce]

The use of String Constructor does not avoid the global read lock, but I
can see with some changes and a suitable set of conventions this could be
made to work.

Cheers
/Andy

count(
collection(``[BEP]``))

On 20 September 2017 at 23:28, Christian Grün 
wrote:

> > bxcode:inline-variables($xqcode-uri) as xs:string
> > @returns xquery code that is the source code from $xqcode-uri but with
> > inlining "constant" variables and operations applied
> >
> > This could be used as a preprocessor.
> > Is this something that could be created from the bits and pieces in the
> > current BaseX Java code?
>
> This reminds me of the discussion with Omar [1]: There is currently no
> way to generate a correct string representation for compiled code. In
> the GUI Info View, we output a string representation of the optimized
> query. In simple cases, this string is a valid and equivalent to the
> original query string, but it’s fairly easy to generate invalid
> results.
>
> If it’s an option for you to generate code in XQuery, something like
> the following could be done:
>
> _ bxcode.xqm ___
>
>   module namespace bxcode = 'bxcode';
>   declare function bxcode:bind-external-variables(
> $uri   as xs:string,
> $vars  as map(xs:string, xs:string)
>   ) as xs:string {
> let $xqcode := unparsed-text($uri)
> return fold-left(
>   map:keys($vars), $xqcode, function($string, $name) {
> let $value := replace($vars($name), ']``', '`{"]``"}`', 'q')
> return $string
> => replace('declare variable $' || $name || ' external;', '', 'q')
> => replace('$' || $name, '``[' || $value || ']``', 'q')
>   }
> )
>   };
>
> _ example.xq ___
>
>  declare variable $db external;
>  count(collection($db))
>
> _ run-bxcode.xq ___
>
>   import module namespace bxcode = 'bxcode' at 'bxcode.xqm';
>   bxcode:bind-external-variables('query.xq', map { 'db': 'BEP' })
>
> _ output ___
>
>   count(collection(``[BEP]``))
>
> Cheers,
> Christian
>
> [1] https://www.mail-archive.com/basex-talk@mailman.uni-
> konstanz.de/msg09863.html
>

Re: [basex-talk] Global locks

[Andy Bunce]

Thanks for the explanation.

Wanting to parameterise the input collections, in  fairly simple ways,
seems to me a natural and common requirement.
but the consequence of a global read lock is all other tasks are unable to
access any database. So great care required ;-)

>Variables will only be inlined at compile time, i.e., after the locks have
already been assigned.

Consider a function:

bxcode:inline-variables($xqcode-uri) as xs:string
@returns xquery code that is the source code from $xqcode-uri but with
inlining "constant" variables and operations applied

This could be used as a preprocessor.
Is this something that could be created from the bits and pieces in the
current BaseX Java code?

/Andy



On 20 September 2017 at 18:27, Christian Grün 
wrote:

> Hi Andy,
>
> > Note @reads="BEP"
> >
> > Is this correct and/or of any consequence?
>
> This is correct: If the argument of fn:collection, fn:doc, etc. is
> directly supplied as string argument, it is possible to statically
> detect (at parse time) which database needs to be locked. Variables
> will only be inlined at compile time, i.e., after the locks have
> already been assigned. As a result, your first query will cause a
> global lock.
>
> Currently, we only have one compilation step. In an ideal BaseX world,
> we would have at least 2 steps:
>
> 1. Static compilation, in which a query would be typed and rewritten
> without considering dynamic input (external variables, databases, …).
> The result could be reused several times, and in your case, the
> database name would have been inlined.
>
> 2. Dynamic compilation, which binds external variables (from clients,
> RESTXQ calls, etc.), checks databases for index structures, etc.
>
> It might turn out that Step 1 is too expensive in order to be
> evaluated before locking a query. On the other hand, static
> compilation needs not be locked, it can always run in parallel with
> queries that are currently evaluated.
>
> Does this answer your question?
> Christian
>

[basex-talk] Global locks

[Andy Bunce]

Running the following in the GUI (8.6.5)
```
let $db:="BEP" return count(collection($db))
```
The info windows shows $db is inlined

Compiling:
- inline $db_0
- pre-evaluate collection("BEP") to document-node() sequence
- pre-evaluate count((db:open-pre("BEP",0), ...)) to xs:integer
- simplify gflwor
Optimized Query:
9193

All good.

Submitting a job
```
let $q:='let $db:="BEP" return count(collection($db))'
return
jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.))
```
returns:
let $db:="BEP" return
count(collection($db))

Note @reads="(global)"

But
```
let $q:='count(collection("BEP"))'
return
jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.))
```
count(collection("BEP"))

Note @reads="BEP"

Is this correct and/or of any consequence?

/Andy

Re: [basex-talk] How do create XML for a one byte hex field that is used as a flag byte

[Andy Bunce]

Hi Dave,

Sounds like  the Binary_Module[1] will help you. This is the BaseX
implementation of the EXPath standard [2]
E.g.

```
let $byte:=bin:from-octets(4)
let $DTFLSYSS:=bin:from-octets(8)
let $new:=  bin:or($byte,$DTFLSYSS)
return bin:to-octets($new)
```

Regards
/Andy

[1] http://docs.basex.org/wiki/Binary_Module
[2] http://expath.org/spec/binary

On 7 September 2017 at 21:01, Dave Day  wrote:

> Greetings list,
>
>I don't know if I am asking this question correctly.  I'm new to the
> XML world.  I can google this, get some hits, but when I read the posts, it
> is not really clear to me.  Any help is appreciated.
>
> On an IBM mainframe,  I've got a one byte hex field defined as follows:
>
>   PCDTFLAG DSXL1 FLAG TO INDICATE ORIGIN OF INFO
>
>
> DTFLENTP EQU   B'1000' INFO CAME FROM MODULE ENTRY POINT
> DTFLMEPS EQU   B'0100' DB2 MEPL LIST HAS BEEN SEARCHED
> DTFLMEPL EQU   B'0010' INFO CAME FROM DB2 MEPL LIST
> DTFLDB2S EQU   B'0001' DSNMODS HAS BEEN SEARHCED
> DTFLDSNM EQU   B'1000' INFO CAME FROM DSNMODS SEARCH
> DTFLSYST EQU   B'0100' INFO CAME FROM SYSTEM TABLE
> DTFLSYSS EQU   B'0010' SYSTEM TABLE HAS BEEN SEARCHED
> DTFLCMNC EQU   B'0001' COMMON AND NUC HAS BEEN SEARCHED
>
> I need to be able to create XML for the field PCDTFLAG.  OK, define an
> element as hexBinary. But then how do I define the bits within the byte.
>
> On the mainframe side, with the equated values,  I can OR the flag
> byte with the equated value, and this will set the correct bit on in the
> byte.
>
> OI   PCDTFLAG,DTFLENTPwill turn on bit 0 in the 1 byte field if it
> is not on already.
>
> I can test for the bit on or off.
>
> TMPCDTFLAG,DTFLENTP
>
> JZ  some-other-area-of-code   will branch if the bit is off
>
> JOanother-area-of-code  will branch if the bit is on
>
> So, I guess I am asking two questions.  One, how to give definition to
> bits within a byte, and two, how to manipulate and test those defined bits.
>
> Regards,
>
> -- Dave Day
>
>
>
>

Re: [basex-talk] HTTP module

[Andy Bunce]

I have submiited an issue https://github.com/BaseXdb/basex/issues/1487

/Andy

On 15 August 2017 at 00:06, Kendall Shaw  wrote:

> Does it work to use Andy’s multipart version with encoding=”UTF-8” added
> to body elements?
>
> On 8/14/17, 3:59 PM, "Giuseppe Celano" 
> wrote:
>
> Thanks, Andy. I have also tried to invoke curl via proc:execute():
>
> proc:execute("curl",("-F", "data=@example.txt",  "-F", "tagger=",
> "-F", "parser=",  "https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__lindat.mff.cuni.cz_services_udpipe_api_process=DwIFAg=DS6PUFBBr_
> KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc=JgwnBEpN1c-DDmq-
> Up2QMq9rrGyfWK0KtSpT7dxRglA=qRCvzBXhWnhavXBh6P8EXvNUf4YQi3
> 2_9ryqeOGjFo0=eR_QvfgwPhEJZno15HPXKZ7T6_aiGenJQ-NEFhJyCyU= " ))
>
> The function works, but unfortunately the text inside the file is not
> recognized as UTF-8, and so I get al lot of gibberish in the result. At the
> beginning I though it was due to
> my MacOS configuration, but I experimented a lot, and the problem
> seems to depend on BaseX.
>
> I run the basexgui (and basex) commands of the bin folder from my
> Terminal window and they should inherit the environment variables (and
> indeed proc:execute("locale") also shows the right UTF-8 values).
>
> I will open a Github issue, unless I am missing something here.
>
>
>
>
>
>
>

Re: [basex-talk] HTTP module

[Andy Bunce]

Hi Giuseppe,
.
You could use web:create-url to create your url [1] , although it requires
parameters to have explicit values. E.g

let $target:="http://lindat.mff.cuni.cz/services/udpipe/api/process;
let $params:=map{
   "tokenizer":0,  "tagger":0, "parser":0,
   "data":"Děti pojedou k babičce. Už se těší"
}
return http:send-request(,
web:create-url($target,$params))

You could change this to  read  data from a file
"data":file:read-text("example.txt"),

Regards
/Andy

[1] http://docs.basex.org/wiki/Web_Module#web:create-url

On 14 August 2017 at 13:11, Giuseppe Celano <
cel...@informatik.uni-leipzig.de> wrote:

> Hi,
>
> I am accessing a RESTful API via the following command:
>
> curl -F data=@example.txt  -F tokenizer= -F tagger= -F parser=
> http://lindat.mff.cuni.cz/services/udpipe/api/process > example2.txt
>
> I am wondering what the best way is to do that in BaseX. The service also
> has a URL syntax, as shown in the following example:
>
> http://lindat.mff.cuni.cz/services/udpipe/api/process?
> tokenizer=D%C4%9Bti%20pojedou%20k%
> 20babi%C4%8Dce.%20U%C5%BE%20se%20t%C4%9B%C5%A1%C3%AD.
>
> I have tried:
>
> http:send-request( href='http://lindat.mff.cuni.cz/services/udpipe/api/
> process?tokenizertaggerparserdata=D%C4%9Bti%
> 20pojedou%20k%20babi%C4%8Dce.%20U%C5%BE%20se%20t%C4%9B%C5%A1%C3%AD'/>)
>
> and works perfectly;  but I was trying to put the body of the request in
> http:body, but it does not work:
>
> http:send-request( href='http://lindat.mff.cuni.cz/services/udpipe/api/process'>
> 
> tokenizertaggerparserdata=Děti pojedou k babičce. Už se
> těší.
> 
> )
>
> I could invoke the curl command in BaseX, but maybe there is a more
> elegant way to send the content of the file (than adding it to the URL).
> Thanks.
>
> Ciao,
> Giuseppe
>
>
> Universität Leipzig
> Institute of Computer Science, Digital Humanities
> Augustusplatz 10
> 04109 Leipzig
> Deutschland
> E-mail: cel...@informatik.uni-leipzig.de
> E-mail: giuseppegacel...@gmail.com
> Web site 1: http://www.dh.uni-leipzig.de/wo/team/
> Web site 2: https://sites.google.com/site/giuseppegacelano/
>
>

Re: [basex-talk] Shouldn't CHOP = false make xml:space="preserve" the default behavior?

[Andy Bunce]

It seems globally setting `indent=no` gets applied to WebDAV (and
everywhere else where serialization is not explicit specified). This would
be my preference for WebDAV, as it means documents can be round-tripped
without any changes being introduced. The only side effect from this
setting I have seen is view-source on generated html source is harder to
read, but this is not a real issue.

I have not tried setting them in web.xml yet. I wondered if you would
expect it to work :-).
I will try...

Cheers
/Andy

On 10 August 2017 at 18:40, Christian Grün 
wrote:

> Hi Andy,
>
> > Can the WebDAV serialization be set independently of the default, in
> > web.xml?
>
> The defaults for whitespace chopping and serialization can only be
> assigned globally for all features of BaseX. Did you try to set both
> 'org.basex.chop' and 'org.basex.serializer' in web.xml / does it
> introduce other unwanted side effects?
>
> Cheers,
> Christian
>
>
> > /Andy
> >
> > On 7 August 2017 at 09:57, Christian Grün 
> wrote:
> >>
> >> Dear Ottid,
> >>
> >> Thanks for providing us with the helpful example, which helped me to
> >> understand the problem.
> >>
> >> >> replace /a foo bar
> >> > "a.xml" (Line 1): Open quote is expected for attribute "xml:space"
> >> > associated with an  element type  "root".
> >>
> >> Just a side note: Command-line parsing is restrictive when it comes to
> >> replacing XML. The reason is that is possible to send multiple
> >> commands in a single line, as shown in the following example:
> >>
> >>   create db db; replace /a ; xquery .
> >>
> >>
> >> >> xquery /root
> >> > foo bar
> >>
> >> You may be surprised to hear that whitespaces in your document were
> >> actually chopped, and that the whitespaces are  added by the
> >> serializer, because the "indent" serialization parameter is by default
> >> set to "yes".
> >>
> >> It was surprised to see that no one else pointed at this so far, and
> >> that was not mentioned in our documentation, so I have just added some
> >> explanatory lines [1,2].
> >>
> >> Some more technical background:
> >>
> >> If you call BaseX the "info storage" command, you will see which XML
> >> nodes are stored in the document:
> >>
> >> > set chop on;create db db  ; info storage
> >> CHOP: true
> >> Database 'db' created in 11.0 ms.
> >> PRE  DIS  SIZ  ATS  ID  NS  KIND  CONTENT
> >> -
> >>   0121   0   0  DOC   db.xml
> >>   1111   1   0  ELEM  aIf
> >>
> >> > set chop off;create db db  ; info storage
> >> CHOP: false
> >> Database 'db' created in 20.12 ms.
> >> PRE  DIS  SIZ  ATS  ID  NS  KIND  CONTENT
> >> -
> >>   0131   0   0  DOC   db.xml
> >>   1121   1   0  ELEM  a
> >>   2111   2   0  TEXT
> >>
> >> Serialization indentation was a chosen as default because it goes hand
> >> in hand with the CHOP option. It even works fine if CHOP is disabled
> >> if a document has whitespaces included (in which case no whitespaces
> >> will be added by the serialized). But it may definitely cause
> >> undesirable output if a document contains no superfluous whitespaces,
> >> such as in your case.
> >>
> >> Hope this helps,
> >> Christian
> >>
> >> [1] http://docs.basex.org/wiki/Options#CHOP
> >> [2] http://docs.basex.org/wiki/Full-Text#Mixed_Content
> >
> >
>