Re: [basex-talk] Weird behaviour with sequence and random:integer

[Marco Lettere]

Everytime one learns something new.

Thanks for the explanation!

M.

On 22/05/24 15:41, Christian Grün wrote:

Hi Marco,

> ("a", "b", "c")[trace(1 + random:integer(3))]

The filter expression is defined in such a way that the predicate is 
evaluated anew for every item of the sequence. If you want 
random:integer(3) to be evaluated only once, you can either bind it to 
a variable…


let $r := random:integer(3) + 1
return ("a", "b", "c")]$r]

…or use functions like fn:subsequence:

subsequence(("a", "b", "c"), random:integer(3) + 1, 1)

Hope this helps,
Christian



Marco Lettere  schrieb am Mi., 22. Mai 2024, 15:34:

Dear all,

I have this strange behavior when running in BaseX 10.4:

["a","b","c"](1 + random:integer(3))

I got something empty result, sometimes (correctly) one out of the
three, sometimes two results...

I report the results and the trace in [1] and [2].

What is the reason for this?

Thank you.

Marco.

[1] Two results
a
c

Evaluating:
1
3
3
Compiling:
- rewrite list to xs:string sequence: ("a", "b", "c")
- swap operands: (1 + random:integer(3))
Optimized Query:
("a", "b", "c")[trace((random:integer(3) + 1))]
Query:
("a", "b", "c")[trace(1 + random:integer(3))]
Result:
- Hit(s): 2 Items
- Updated: 0 Items
- Printed: 3 b
- Read Locking: (none)
- Write Locking: (none)
Timing:
- Parsing: 0.21 ms
- Compiling: 0.4 ms
- Optimizing: 0.1 ms
- Evaluating: 0.57 ms
- Printing: 0.02 ms
- Total Time: 1.31 ms


[2] Empty result
()

Evaluating:
3
3
1
Compiling:
- rewrite list to xs:string sequence: ("a", "b", "c")
- swap operands: (1 + random:integer(3))
Optimized Query:
("a", "b", "c")[trace((random:integer(3) + 1))]
Query:
("a", "b", "c")[trace(1 + random:integer(3))]
Result:
- Hit(s): 0 Items
- Updated: 0 Items
- Printed: 0 b
- Read Locking: (none)
- Write Locking: (none)
Timing:
- Parsing: 0.16 ms
- Compiling: 0.41 ms
- Optimizing: 0.15 ms
- Evaluating: 0.56 ms
- Printing: 0.02 ms
- Total Time: 1.3 ms
Query Plan:

   
 
   a
   b
   c
 
 
   
 
   3
 
 1
   
 
   

[basex-talk] Weird behaviour with sequence and random:integer

[Marco Lettere]

Dear all,

I have this strange behavior when running in BaseX 10.4:

["a","b","c"](1 + random:integer(3))

I got something empty result, sometimes (correctly) one out of the 
three, sometimes two results...


I report the results and the trace in [1] and [2].

What is the reason for this?

Thank you.

Marco.

[1] Two results
a
c

Evaluating:
1
3
3
Compiling:
- rewrite list to xs:string sequence: ("a", "b", "c")
- swap operands: (1 + random:integer(3))
Optimized Query:
("a", "b", "c")[trace((random:integer(3) + 1))]
Query:
("a", "b", "c")[trace(1 + random:integer(3))]
Result:
- Hit(s): 2 Items
- Updated: 0 Items
- Printed: 3 b
- Read Locking: (none)
- Write Locking: (none)
Timing:
- Parsing: 0.21 ms
- Compiling: 0.4 ms
- Optimizing: 0.1 ms
- Evaluating: 0.57 ms
- Printing: 0.02 ms
- Total Time: 1.31 ms


[2] Empty result
()

Evaluating:
3
3
1
Compiling:
- rewrite list to xs:string sequence: ("a", "b", "c")
- swap operands: (1 + random:integer(3))
Optimized Query:
("a", "b", "c")[trace((random:integer(3) + 1))]
Query:
("a", "b", "c")[trace(1 + random:integer(3))]
Result:
- Hit(s): 0 Items
- Updated: 0 Items
- Printed: 0 b
- Read Locking: (none)
- Write Locking: (none)
Timing:
- Parsing: 0.16 ms
- Compiling: 0.41 ms
- Optimizing: 0.15 ms
- Evaluating: 0.56 ms
- Printing: 0.02 ms
- Total Time: 1.3 ms
Query Plan:

  
    
  a
  b
  c
    
    
  
    
  3
    
    1
  
    
  

Re: [basex-talk] Making store reactive

[Marco Lettere]

Inlined ...

On 22/05/24 13:00, Christian Grün wrote:

Hi Marco,

Thanks for your suggestion. Some thoughts:

• Function items can depend on the currently evaluated code and its 
static and dynamic context, but we could possibly design something 
similar as for the Job Module, in which the query is passed on as a 
string or a URI reference and evaluated completely independently.


Yes!


• Registered observers could be handled similarly as „services”, i.e., 
made persistent, end up in the same query pool, discarded by user 
requests, gracefully shut down when a server stops, etc. [1].


Yes!


• The feature request reminds me of triggers what we envisioned for 
databases (but that were eventually discarded [2]).


I remember. But IMO this could have much less implications and hidden 
complexity compared to a full-blown DB trigger mechanism.





Having said this, it could take a while to make this happen as it’s a 
non-trivial request :) I’d like to hear about suggestions of other 
readers.


I was not even expecting for this "loud thought" to be promoted so 
quickly to "feature request" :-) ... Thus thank you!





Apart from that, we are always interested in feedback on the Store 
Module; it’s still fairly new, but more and more people seem to 
discover it.


Ciao,
Christian

[1] https://docs.basex.org/wiki/Job_Module#Services
[2] https://github.com/BaseXdb/basex/issues/1082



On Wed, May 22, 2024 at 9:49 AM Marco Lettere  wrote:

Dear Christian and BaseX developers,

just wondering if adding something like the following would be
hard to implement.

/store:observe($key as xs:string, $observers as function(*)*) /

with $observers being something like

/function($key as xs:string)./

The semantics is to call the registered observers whenever a value
associated with the key in the store changes (put, remove, clear, ..).

This would allow for nicely decoupled observer - notification pattern.

Does it make sense?

Regards,

Marco.

[basex-talk] Making store reactive

[Marco Lettere]

Dear Christian and BaseX developers,

just wondering if adding something like the following would be hard to 
implement.


/store:observe($key as xs:string, $observers as function(*)*) /

with $observers being something like

/function($key as xs:string)./

The semantics is to call the registered observers whenever a value 
associated with the key in the store changes (put, remove, clear, ..).


This would allow for nicely decoupled observer - notification pattern.

Does it make sense?

Regards,

Marco.

[basex-talk] BaseX and NFS

[Marco Lettere]

Dear all,

we have an instance of BaseX running on top of an NFS exported disk. Yes 
yes I know it's not the best possible scenario thus I was expecting a 
slight performance decrease.


Anyway, when comparing local disk to NFS disk, for a tiny operation like 
storing a very small document into a database (without optimize or 
indexes) we get orders of magnitude of difference. And I'm saying from 
few hundreds of ms to several seconds in some cases.


Does anyone have experience with cases like these? Or a solid motivation 
that explains this degradation?


Thank you very much as usual.

Marco.

Re: [basex-talk] BaseX 10.7: The Summer Edition

[Marco Lettere]

GREAT

Big thanks!

On 04/08/23 12:54, Christian Grün wrote:

Dear all,

We’re pleased to announce version 10.7 of BaseX, of our XML framework: 
https://basex.org.


The new release is a big step forward towards BaseX 11:

• We have added numerous new operators, functions and features of 
XQuery 4.
• The GUI editor and result view now provide full support for Unicode 
characters.

• Font rendering has been improved (you can tweak it in the Font Dialog)
• More BaseX 11 preview features available (see docs.basex.org 
)

• Various bug fixes (web:forward, job:eval, main-memory documents)

Have fun,
Your BaseX Team

Re: [basex-talk] Dynamic path segment for websockets

[Marco Lettere]

Dear Christian,

we are currently thinking of a solution based on adding attributes to 
the websocket with the dedicated function. Then filtering out the right 
websocket to send a message to based on the attribute value.


as a prerequisite it has to be possible to pass query-parameters to the 
ws:connect endpoint url?


From the JS point of view "ws://myserver.com/path?param=1" is a valid 
ws url. Is the server side code able to handle this?


Thanks a lot,

Marco.

On 23/06/23 16:46, Christian Grün wrote:

Hi Marco,

As most web applications open only one or a few websockets at the same
time, we decided against supporting wildcards.

Feel free to share background information on your use case with us,
maybe you can convince us to get active ;·)

Ciao,
Christian


On Fri, Jun 23, 2023 at 11:21 AM Marco Lettere  wrote:

Dear all,

so I just found out that it is not possible to create websockets bound
to dynamic paths.

If I try a websocket connection to

declare
%ws:connect('/notification/subject/{$subject}')
function log:subject() {};

I get an error (without significant logs). Whereas

declare
%ws:connect('/notification/subject/one')
function log:subject() {};

works well.

Does this have a motivation? Is there another way of meeting the
requirement of having dynamic websocket channels?

Thanks in advance,

Marco.

Re: [basex-talk] Dynamic path segment for websockets

[Marco Lettere]

I'll try to explain my use-case.

I have a system that acts remotely. Jobs are requested from a GUI and 
the logs produced by the jobs have to be streamed back to the GUI via 
websocket after being treated by a service of mine developed as a RestXQ 
in BaseX10.


In the current solution all the GUIs connect to the same websocket URL. 
Sending something to that websocket makes everything available to 
everyone and it's up to the javascript code to filter out what is of 
interest for the user.


Instead it would be nice to have isolated channels end-to-end for 
instance by using the id of the logged user.


Does this make enough sense?

Danke schön Christian.

M.

On 23/06/23 16:46, Christian Grün wrote:

Hi Marco,

As most web applications open only one or a few websockets at the same
time, we decided against supporting wildcards.

Feel free to share background information on your use case with us,
maybe you can convince us to get active ;·)

Ciao,
Christian


On Fri, Jun 23, 2023 at 11:21 AM Marco Lettere  wrote:

Dear all,

so I just found out that it is not possible to create websockets bound
to dynamic paths.

If I try a websocket connection to

declare
%ws:connect('/notification/subject/{$subject}')
function log:subject() {};

I get an error (without significant logs). Whereas

declare
%ws:connect('/notification/subject/one')
function log:subject() {};

works well.

Does this have a motivation? Is there another way of meeting the
requirement of having dynamic websocket channels?

Thanks in advance,

Marco.

Re: [basex-talk] Queuing order for db:add

[Marco Lettere]

Hi Christian,

sorry for not being clear enough.

I was just wondering whether I can rely on insertion order being kept 
when I perform a corresponding db:get("db", "a.xml"). In other words 
given your slightly modified example:


for $i in 1 to 100
return db:add('db', , 'a.xml')

Will

db:get('db','a.xml')

always return

(, , ...)

M.

On 11/04/23 09:03, Christian Grün wrote:

Hi Marco,


Does db:add multiple resources to the same path guarantee sorting based
on insertion time?

Sorry for the late reply, but I’m not sure if I understand your
question. It’s correct, it’s possible to add multiple resources to the
same path, e.g. as follows:

for $i in 1 to 100
return db:add('db', , 'a.xml')

There’s no need to do any sorts, as the target path is identical. All
resources will simply be attached to the database storage.

Or did you question yourself if sorting is some necessary when
resources are requested in a later step?

Ciao,
Christian

[basex-talk] Queuing order for db:add

[Marco Lettere]

Dear all,

I've probably overlooked this in the docs but asking for confirmation. 
Does db:add multiple resources to the same path guarantee sorting based 
on insertion time?


If yes, then maybe I could add a note about this in the docs?

Thanks,

Marco.

Re: [basex-talk] Unable to set admin password

[Marco Lettere]

Yes, confirmed.

Need to ensure that users.xml is in the prorper data folder. Thus need 
to set admin password after change to .basex configuration file I suppose.


Sorry again for bothering.

M.

On 20/02/23 10:50, Marco Lettere wrote:

Hm, sorry. When downloading BaseX and using it barely the steps work.

So it has something to do with our specific layout we change the 
location of the data, repo and webapp folder


Maybe this is the issue ...

M.


On 20/02/23 10:40, Marco Lettere wrote:

Dear all,

I'm trying to setup a new instance of BaseX 10 in order to start 
migrating some applications of ours.


Anyway I'm unable to set the admin password.

Followin https://docs.basex.org/wiki/BaseX_10#Migrating_Applications, 
I'm currently trying with [1] and [2] which both ask me to enter a 
new password. But afterwards I keep getting [3] and I'm unable to 
access dba too.


I cannot understand what I'm doing wrongly ...

Any help? Thanks!

[1] java -cp BaseX.jar org.basex.BaseX -c PASSWORD
Password: newpassword

[2] bin/basex -c PASSWORD
Password: newpassword

[3] bin/basexclient
Username: admin
Password: newpassword
Access denied: admin

Re: [basex-talk] Unable to set admin password

[Marco Lettere]

Hm, sorry. When downloading BaseX and using it barely the steps work.

So it has something to do with our specific layout we change the 
location of the data, repo and webapp folder


Maybe this is the issue ...

M.


On 20/02/23 10:40, Marco Lettere wrote:

Dear all,

I'm trying to setup a new instance of BaseX 10 in order to start 
migrating some applications of ours.


Anyway I'm unable to set the admin password.

Followin https://docs.basex.org/wiki/BaseX_10#Migrating_Applications, 
I'm currently trying with [1] and [2] which both ask me to enter a new 
password. But afterwards I keep getting [3] and I'm unable to 
access dba too.


I cannot understand what I'm doing wrongly ...

Any help? Thanks!

[1] java -cp BaseX.jar org.basex.BaseX -c PASSWORD
Password: newpassword

[2] bin/basex -c PASSWORD
Password: newpassword

[3] bin/basexclient
Username: admin
Password: newpassword
Access denied: admin

[basex-talk] Unable to set admin password

[Marco Lettere]

Dear all,

I'm trying to setup a new instance of BaseX 10 in order to start 
migrating some applications of ours.


Anyway I'm unable to set the admin password.

Followin https://docs.basex.org/wiki/BaseX_10#Migrating_Applications, 
I'm currently trying with [1] and [2] which both ask me to enter a new 
password. But afterwards I keep getting [3] and I'm unable to access 
dba too.


I cannot understand what I'm doing wrongly ...

Any help? Thanks!

[1] java -cp BaseX.jar org.basex.BaseX -c PASSWORD
Password: newpassword

[2] bin/basex -c PASSWORD
Password: newpassword

[3] bin/basexclient
Username: admin
Password: newpassword
Access denied: admin

Re: [basex-talk] Too aggressive optimizations?

[Marco Lettere]

Dear Christian,

I have to admit that all what you say makes sense and I was even digging 
in my swapped long term memory for something like the annotation you 
mention.


The background information you share makes me feel more comfortable and 
I'll take your advice.


Thanks!

Marco.

On 07/02/23 19:22, Christian Grün wrote:

Hi Marco,

let $ops := (
for $i in (1 to 5)
let $url := "http://www.google.com;
return function(){
(file:write(file:create-temp-file("ttt",string($i)),fetch:content-type($url)))
}
)
let $download := xquery:fork-join($ops)
return count($ops)


I've noticed often the archive arrives empty. So after investigation I've found that 
query [1] isnon predictable. It is often optimized to "count(0)".

That’s surprising, and I didn’t manage to reproduce it. count($pos)
should consistently yield 5, as the number of (non-executed) function
items attached to $ops is 5, regardless of what is supposed to happen
before or after the variable declaration. Maybe you wrote
"count($download)" or something similar? Is there any other way to get
it reproduced?

However, I can confirm that xquery:fork-join($ops) is not evaluated,
as its result, which is bound to $download, is never referenced.
Here’s a better way to write it:

let $ops := (... for $i in ...)
return (
   xquery:fork-join($ops),
   count($pos)
)

Another solution to enforce the evaluation of the function call is the
basex:non-deterministic pragma [1]…

let $ops := (... for $i in ...)
let $download := (# basex:non-deterministic #) { xquery:fork-join($ops) }
return count($ops)

…but in general, it’s better to get rid of unreferenced variables in
the code whenever possible.

Some background noise: Non-deterministic and side-effecting functions
carve out a niche existence in the official W3 standards, as they
contradict the nature of functional languages. It’s tricky for the
optimizer to treat them properly: Function items are deterministic,
but when they are evaluated, they may trigger side effects.
Deterministic code that seems irrelevant is removed from the original
query whenever possible, so solutions to circumvent this are to either
wrap the expression with the pragma (thus, annotating it as
non-deterministic), or by moving it to a result sequence.

And a monologic side note: Maybe we should internally annotate
xquery:fork-join as non-deterministic. Even if it may contain purely
deterministic code, it’s almost always used for non-deterministic
operations in practice.

Hope this helps,
Christian

[1] https://docs.basex.org/wiki/XQuery_Extensions

[basex-talk] Too aggressive optimizations?

[Marco Lettere]

Dear all,

my scenario is a RestXQ:

- download resources and store them in temporary directory.

- do it with fork-join in order to obtain smaller latency

- compress to zip archive and return the archive data.

I've noticed often the archive arrives empty. So after investigation 
I've found that query [1] isnon predictable. It is often optimized to 
"count(0)".


I can manage to produce results from time to time but not consistently 
with [2].


[3] Seems the safer solution.

The behavior is the same with 9.x and 10.

Since I do not feel very comfortable, is there someone who can tell me 
if I'm doing it wrong or if there is a secure solution or if I should 
abandon fork-join tout-court?


Thanks a lot.

Regards,

Marco.

[1]

let $ops := (
  for $i in (1 to 5)
  let $url := "http://www.google.com;
  return function(){
(file:write(file:create-temp-file("ttt",string($i)),fetch:content-type($url)))
  }
)
let $download := xquery:fork-join($ops)
return count($ops)

[2]

let $ops := (
  for $i in (1 to 5)
  let $url := "http://www.google.com;
  return function(){
(file:write(file:create-temp-file("ttt",string($i)),fetch:content-type($url)),1)
  }
)
let $download := xquery:fork-join($ops)
return count($ops)

[3]

let $ops := xquery:fork-join(
  for $i in (1 to 5)
  let $url := "http://www.google.com;
  return function(){
    (1, 
file:write(file:create-temp-file("ttt",string($i)),fetch:content-type($url)))

  }
)
return count($ops)

Re: [basex-talk] Reducing logs

[Marco Lettere]

Eliot' s point is ok for us too. We also mine info about HTTP traffic 
out of the log files.


In contradiction to some comments on [1] there is a good motivation for 
writing to stdout/err and it's related to the execution of servers 
inside docker containers.


"docker logs [my_container_with_basex] -f" would be of great value. 
Particularly when running a service on tools such as Portainer that do 
not allow users to "exec into" a container. This is currently not 
exploitable because the only thing you get is the logs of the server 
which is less useful when developing a RESTXQ service for instance


"docker logs" intercepts stdout/stderr. There are some workarounds to 
map /dev/stdout to a file. But again, BaseX switches the file every day 
so that would also not be feasible.


So, trying to resume:

- put code such as [2] to a trace level reducing impact on standard log 
to a on or two liner as for REST calls


- have a standardized logging system (log4j? slf4j?) that allows to 
differentiate logs (HTTP, Server) and levels redirecting to file or 
console and configuring rotations different from changing the file.


- make the dba GUI not trying to open by default the last log file 
because if it is large this causes an OOM that a user cannot avoid


- Bonus: logging options changeable through the dba UI would be 
incredibly useful but I can understand possible difficulties here.


O2C :-)

Thanks a lot Christian for paying usual high level attention to your users'!

Regards,

M.

[1] https://github.com/BaseXdb/basex/issues/940

[2]


12:22:27.849 10.0.4.15:50424  admin OK 0.02 CLOSE[0]

> 12:22:27.848 10.0.4.15:50434 admin OK 1.40 FULL[0]
> 12:22:27.848 10.0.4.15:50424 admin OK 0.83 FULL[0]
> 12:22:27.847 10.0.4.15:50420 admin OK 0.05 CLOSE[0]
> 12:22:27.847 10.0.4.15:50412 admin OK 0.04 CLOSE[0]
> 12:22:27.847 10.0.4.15:50434 admin OK 0.05 BIND[0]

On 02/02/23 16:36, Eliot Kimber wrote:


For me, being able to log HTTP requests separately (different log 
file) from XQuery-generated would be very helpful. For our Mirabel 
server we’ll want to be able to mine usage statistics from the HTTP 
request logs but for general administration and monitoring, it’s the 
messages generated by the XQuery code serving the pages that are useful.


Cheers,

E.

_

*Eliot Kimber*

Sr Staff Content Engineer

O: 512 554 9368

M: 512 554 9368

servicenow.com <https://www.servicenow.com>

LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter 
<https://twitter.com/servicenow> | YouTube 
<https://www.youtube.com/user/servicenowinc> | Facebook 
<https://www.facebook.com/servicenow>


*From: *BaseX-Talk  on 
behalf of Christian Grün 

*Date: *Thursday, February 2, 2023 at 7:42 AM
*To: *Marco Lettere 
*Cc: *basex-talk@mailman.uni-konstanz.de 


*Subject: *Re: [basex-talk] Reducing logs

[External Email]

Hi Marco,

As this requirement has been reported back to us repeatedly, it’s time
to create an issue for it [1].

I think we’ll replace the general LOG option with a more flexible
variant. We’ll still need to define which levels to offers; any
feedback is welcome. The new option will probably be made available
with BaseX 11 (to be expected around spring).

Ciao,
Christian

[1] https://github.com/BaseXdb/basex/issues/2168




On Thu, Feb 2, 2023 at 1:37 PM Marco Lettere  wrote:
>
> Dear all,
>
> when using the Basex Xquery server (9.7.3) we see that logs such as [1]
> are produced for every query execution.
>
> Since we have a scenario which involves polling and thus we have a lot
> of query executions the log files increase to hundreds om MBs during 
a day.

>
> This makes them unmanageable and even cause dba UI to crash with OOM
> exceptions when opening the logs page.
>
> Is there a way to disable this logging (without affecting other logs
> such as HTTP)?
>
> Is it really necessary to log server side query executions at this grain
> by default? Maybe it could be made optional?
>
> Thanks for any support.
>
> Regards,
>
> Marco.
>
> [1]
> 12:22:27.849 10.0.4.15:50424 admin OK 0.02 CLOSE[0]
> 12:22:27.848 10.0.4.15:50434 admin OK 1.40 FULL[0]
> 12:22:27.848 10.0.4.15:50424 admin OK 0.83 FULL[0]
> 12:22:27.847 10.0.4.15:50420 admin OK 0.05 CLOSE[0]
> 12:22:27.847 10.0.4.15:50412 admin OK 0.04 CLOSE[0]
> 12:22:27.847 10.0.4.15:50434 admin OK 0.05 BIND[0]
> db=infrastructures as xs:string
> 12:22:27.846 10.0.4.15:50410 admin OK 0.22 CLOSE[0]
> 12:22:27.846 10.0.4.15:50434 admin OK 0.06 BIND[0]
> id=ontheroad-lxd as xs:string
> 12:22:27.846 10.0.4.15:50424 admin OK 0.05 BIND[0]
> db=infrastructures as xs:string
> 12:22:27.846 10.0.4.15:50420 admin OK 0.62 FULL[0]
> 12:22:27.846 10.0.4.15:50412 admin OK 0.90 FULL[0]
> 12:22:27.846 10.0.4.15:50424 admin OK 0.06 BIND[0]
> id=onthe

[basex-talk] Reducing logs

[Marco Lettere]

Dear all,

when using the Basex Xquery server (9.7.3) we see that logs such as [1] 
are produced for every query execution.


Since we have a scenario which involves polling and thus we have a lot 
of query executions the log files increase to hundreds om MBs during a day.


This makes them unmanageable and even cause dba UI to crash with OOM 
exceptions when opening the logs page.


Is there a way to disable this logging (without affecting other logs 
such as HTTP)?


Is it really necessary to log server side query executions at this grain 
by default? Maybe it could be made optional?


Thanks for any support.

Regards,

Marco.

[1]
12:22:27.849    10.0.4.15:50424    admin OK    0.02    CLOSE[0]
12:22:27.848    10.0.4.15:50434    admin    OK    1.40 FULL[0]
12:22:27.848    10.0.4.15:50424    admin    OK    0.83 FULL[0]
12:22:27.847    10.0.4.15:50420    admin    OK    0.05 CLOSE[0]
12:22:27.847    10.0.4.15:50412    admin    OK    0.04 CLOSE[0]
12:22:27.847    10.0.4.15:50434    admin    OK    0.05 BIND[0] 
db=infrastructures as xs:string

12:22:27.846    10.0.4.15:50410    admin    OK    0.22 CLOSE[0]
12:22:27.846    10.0.4.15:50434    admin    OK    0.06 BIND[0] 
id=ontheroad-lxd as xs:string
12:22:27.846    10.0.4.15:50424    admin    OK    0.05 BIND[0] 
db=infrastructures as xs:string

12:22:27.846    10.0.4.15:50420    admin    OK    0.62 FULL[0]
12:22:27.846    10.0.4.15:50412    admin    OK    0.90 FULL[0]
12:22:27.846    10.0.4.15:50424    admin    OK    0.06 BIND[0] 
id=ontheroad-lxd as xs:string

12:22:27.845    10.0.4.15:50380    admin    OK    0.02 CLOSE[0]
12:22:27.845    10.0.4.15:50366    admin    OK    0.01 CLOSE[0]
12:22:27.845    10.0.4.15:50402    admin    OK    0.02 CLOSE[0]
12:22:27.845    10.0.4.15:50420    admin    OK    0.04 BIND[0] 
db=infrastructures as xs:string

12:22:27.845    10.0.4.15:50386    admin    OK    0.02 CLOSE[0]
12:22:27.845    10.0.4.15:50410    admin    OK    0.88 FULL[0]
12:22:27.845    10.0.4.15:50420    admin    OK    0.06 BIND[0] 
id=ontheroad-lxd as xs:string
12:22:27.845    10.0.4.15:50412    admin    OK    0.04 BIND[0] 
db=infrastructures as xs:string
12:22:27.845    10.0.4.15:50434    admin    OK    0.04 QUERY[0] declare 
variable $id external; declare variable $db external; 
db:open($db)[json/id = $id]

Re: [basex-talk] BaseX unreachable behind load balancer

[Marco Lettere]

Hi Harry,

one thing that hits me frequently on cloud machines provisioned by 
others is SELinux. If your host is running Linux of course...


In this case there is some documentation around how to check it out by 
looking into the logs of your proxy service (the error should something 
like "not permitted").


Check the Selinux property for http proxies with:

> sudo getsebool -a | grep http

Whereas to disable selinux enforcement on http proxy permanently (-P flag):

> sudo setsebool -P httpd_verify_dns 0

Regards,

Marco.


On 18/11/22 13:18, Harry King wrote:

Hi,

I’m wondering if someone might be able to offer a hint or two.  I’m attempting 
to run BaseX in AWS behind a Network Load Balancer (NLB) using the 9.5.2 docker 
image with the default config to start with.  I’ve setup a TCP target group and 
the healthcheck appears happy on port 1984.Reachability analyser suggests 
firewall is good to go.  From the local box I can telnet localhost 1984:

telnet localhost 1984
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
BaseX:3334891053316136

I get a similar response if I use the server’s private IP address:

telnet 10.x.x.x 1984
Trying 10.x.x.x...
Connected to 10.x.x.x.
Escape character is '^]'.
BaseX:3335242111998298


So far, so good.

If I attempt the same via the NLB I get a timeout, which suggests to me either 
firewall issue or the application is refusing to listen for some reason.  The 
documentation, seems to suggest by default, basex should respond to requests 
from any IP or hostname, do I understand that correctly, or do I need to alter 
the default config?

Thanks in advance!

Re: [basex-talk] BaseX 10 • The Double-Digit Summer Edition

[Marco Lettere]

THANKS!

M.

On 01/08/22 15:27, Christian Grün wrote:

Dear all,

It’s been around 15 years ago when the first Open-Source version of
BaseX was released. Thanks to the continuous support of all of you, we
are thrilled today to announce the first double-digit version of our
XML framework:

   https://basex.org/

BaseX 10 assists you with your everyday data challenges with some
great new features:

STORAGE

• Beside XML Documents and Binary Data, you can use the new »XQuery
Values« resource type to store arbitrary items and sequences in
databases.
• General data (registered users, scheduled services, and key-value
stores) can now be backed up as well.
• Mixed-content developers will be delighted: Whitespaces are now
preserved by default.

XQUERY

• Database locking has been improved by splitting up XQuery
compilation into a static and dynamic step.
• A new Store Module allows you to organize values in a persistent
main-memory key-value store.
• You can store and retrieve XQuery values with db:put-value and db:get-value.
• When serializing XML, indentation is now turned off by default.

GENERAL

• BaseX 10 requires Java 11 to run.
• The GUI comes with improved HiDPI support, new icons and improved consistency.
• All HTTP requests are now based on the new Java HTTP client.
• Catalog support is now universal and based on either the new XML
Catalog API or Norman Walsh’s excellent XML Resolver.
• The default ports for web applications were changed from 8984/8985
to 8080/8081.
• The default admin password was removed to improve security.
• Results of the REST API are now wrapped in the default REST namespace.

We have taken the version jump as an opportunity to rename various
existing functions and commands. The database storage has not changed:
Databases created with BaseX 10 are backward compatible and can still
be opened with BaseX 9.

Visit our documentation for more details, and information on how to
upgrade your existing applications:

   https://docs.basex.org/wiki/BaseX_10

Have fun!
Your BaseX Team

Re: [basex-talk] BaseX YAML Parser?

[Marco Lettere]

Hi Elliot,

I remember having asked info about a possible YAML serialization format 
on the Slack Xml.com channel one year ago.


They don't really have a nice opinion about Yaml and no-one looked 
really interested in supporting it.


Thus I think there are philosophy matters here :-)

M.

On 26/07/22 15:11, Eliot Kimber wrote:


I found myself needing to operate on some YAML documents and tried to 
find an XQuery YAML parser and did not find one, at least not one of 
sufficient completeness.


It seems like kind of an obvious missing thing in the XQuery world so 
just curious if I missed something?


I don’t have time myself to write one—it can’t be that hard but it’s 
not that critical of a need for me. I ended up using the Python PyYAML 
library, which let me quickly convert YAML to JSON, which then I could 
of course process just fine with XQuery.


Thanks,

E.

_

*Eliot Kimber*

Sr Staff Content Engineer

O: 512 554 9368

M: 512 554 9368

servicenow.com 

LinkedIn  | Twitter 
 | YouTube 
 | Facebook 

Re: [basex-talk] RESTXQ: Serve images that are not within the web app dir

[Marco Lettere]

Hi Eliot,

the content in the webap/static folder is served statically at the 
.../static/myresource path.


Symlinking your folder in there should do the trick. At least up to 
previous versions of Basex. You could just ad src="/static/myresoiurce"/> in your page...


I think I've read in tne ML about some issues recently with symlinking 
but I have not investigated in depth the issue ...


Check it out.

M.

On 06/07/22 21:08, Eliot Kimber wrote:


In my RESTXQ application, I have the XML content in a database but the 
images referenced from that content is only in a local clone of the 
git repository from which the XML content is loaded into BaseX.


I would like to serve the images from RESTXQ-generated HTML Pages but 
I’m not sure how to do that other than to use URLs that include the 
base64 image data directly.


What I think I need to do is implement a RESTXQ end point that handles 
image URLs that I know how to convert into image files on the file 
system but I don’t see how to then return the raw bytes as the return 
result from the RESTXQ handler function.


I’m trying to avoid copying the images because there’s no other reason 
to have them in a database or otherwise copy them somewhere else on 
the file system.


I suppose I could symlink the git directories under the webapp 
directory but that seems kind of icky too.


Thanks,

E.

_

*Eliot Kimber*

Sr Staff Content Engineer

O: 512 554 9368

M: 512 554 9368

servicenow.com 

LinkedIn  | Twitter 
 | YouTube 
 | Facebook 

Re: [basex-talk] BaseX 9.7.3: Tweaks, Fixes, Previews

[Marco Lettere]

Yeah!

So do we!

Very excited!

On 30/06/22 16:18, Bridger Dyson-Smith wrote:

Christian and team -

On Thu, Jun 30, 2022 at 10:04 AM Christian Grün 
 wrote:


Hi all,

A new BaseX maintenance release is available. This is what you get:

• DBA: Improved support for database backups
• Performance tweaks and bug fixes

…and some more BaseX 10 previews:

• Store Module (before: Caching Module)
• Backup of general data (users, jobs, stores)

We spend most of our time on BaseX 10. Stay tuned!


Thank you for your efforts on our behalf!
I won't speak for others, but I'm on the edge of my seat! Looking 
forward to v10.


Yours,
Christian
BaseX Team


Best wishes,
Bridger

Re: [basex-talk] Encoding hassle ....

[Marco Lettere]

Thank you very much.

We'll give it a try ASAP.

M.

On 13/06/22 16:45, Christian Grün wrote:

If the content of a file is written to another file without
intermediate steps, it is streamed and consumes constant memory. The
implementation for streaming the data was deficient.

Re: [basex-talk] Encoding hassle ....

[Marco Lettere]

Oh yes thanks. Forgot to mention this. Forcing utf8 doesn't help.

Il ven 27 mag 2022, 19:26 Bridger Dyson-Smith  ha
scritto:

> Marco -
> I'm sorry but I can only corroborate your findings, and that trying to
> force UTF-8 by adding the encoding parameter to the functions doesn't seem
> to help; e.g.
>
> ) ./bin/basex
> BaseX 9.7.1 [Standalone]
> Try 'help' to get more information.
> > xquery file:current-dir()
> /usr/home/bridger/bin/basex/
> Query executed in 886.62 ms.
> > xquery file:write-text("a1.txt", "°" || out:nl(), "UTF-8")
>
> Query executed in 4.32 ms.
> > xquery file:read-text("a1.txt")
> °
>
> Query executed in 1.99 ms.
> > xquery file:write-text("a2.txt", file:read-text("a1.txt", "UTF-8"),
> "UTF-8")
>
> Query executed in 1.83 ms.
> > xquery file:read-text("a2.txt")
> [file:io-error] Decoding error: xb0
> > xquery file:read-text("a2.txt", "UTF-8")
> [file:io-error] Decoding error: xb0
> > xquery file:read-text("a2.txt", "ISO-8859-1")
> °
>
> Query executed in 2.01 ms.
>
> On Fri, May 27, 2022 at 1:00 PM Marco Lettere  wrote:
>
>> Dear all,
>>
>> after wrapping our heads around this for hours today, we don't know how
>> to get rid of this inconsistency. Thus I ask for help ...
>>
>> SSCE:
>>
>> BaseX 9.6.4 [Standalone]
>> Try 'help' to get more information.
>>  > xquery file:write-text("a1.txt", "°" || out:nl()) (: Same with
>> codepoints-to-string(176) instead of "°" :)
>>
>> Query executed in 183.94 ms.
>>  > xquery file:read-text("a1.txt")
>> °
>>
>> Query executed in 1.49 ms.
>>  > xquery file:write-text("a2.txt", file:read-text("a1.txt"))
>> Query executed in 3.4 ms.
>>
>>  > xquery file:read-text("a2.txt")
>> [file:io-error] Decoding error: xb0
>>
>> Testing the files with linux command-line tool "file", this is the output:
>>
>>  > file a1.txt
>> a1.txt: Unicode text, UTF-8 text
>>
>>  > file a2.txt
>> a2.txt: ISO-8859 text
>>
>> Reading the file after "copying" it seems to change the encoding. How is
>> this supposed to be handled?
>>
>> Regards,
>>
>> Marco.
>>
>>

[basex-talk] Encoding hassle ....

[Marco Lettere]

Dear all,

after wrapping our heads around this for hours today, we don't know how 
to get rid of this inconsistency. Thus I ask for help ...


SSCE:

BaseX 9.6.4 [Standalone]
Try 'help' to get more information.
> xquery file:write-text("a1.txt", "°" || out:nl()) (: Same with 
codepoints-to-string(176) instead of "°" :)


Query executed in 183.94 ms.
> xquery file:read-text("a1.txt")
°

Query executed in 1.49 ms.
> xquery file:write-text("a2.txt", file:read-text("a1.txt"))
Query executed in 3.4 ms.

> xquery file:read-text("a2.txt")
[file:io-error] Decoding error: xb0

Testing the files with linux command-line tool "file", this is the output:

> file a1.txt
a1.txt: Unicode text, UTF-8 text

> file a2.txt
a2.txt: ISO-8859 text

Reading the file after "copying" it seems to change the encoding. How is 
this supposed to be handled?


Regards,

Marco.

Re: [basex-talk] BaseX, RESTXQ, and React?

[Marco Lettere]

Hi Eliot,

just my 2 cents here

I am not a fan of bulky JS frameworks and most of the things they allow 
to do is perfectly doable with vanilla JS anyway. So I'm not a very 
expert of React.


That being said I think that usage of frameworks or not in the browser 
is orthogonal to the use of BaseX for all the server side stuff. BaseX 
is ok and you can decide at which architectural layer you want to employ it.


We at Nubisware have walked literally all the roads that take you from 
frontend to backend (and back). All of them involve BaseX + RestXQ to a 
large extent.


Some solutions use a slim layer between our REST APIs (done with RestXQ) 
and the Browser. This layer is server side and (not even worth to 
mention) done in RestXQ too. It is in charge of handling security, 
generate HTML and serve static CSS, JS and images.


Nowadays we are experimenting again with slightly thicker artifacts in 
the browser since we are in a flirt with vanilla web-components [1] 
which introduce a lot of advantages of component based development 
without being tied up hardly to a vendor biased framework. This scenario 
looks closer to what you are going for ...  In this case the security is 
managed in specific web components and the JS code makes authorized (be 
aware that security is a bit trickier here since browsers are untrusted 
endpoints) calls directly to the REST APIs. In this pattern, since all 
the frontend code is static, we can use for instance BaseX to serve the 
content in dev environment while resolving to Nginx for serving it in 
production.


To make a long story short, go for the way you like and according to 
your team's preferences but keep using Basex+RestXQ for the backend 
because it's great! :-D


Marco.

[1] https://www.webcomponents.org/


On 20/05/22 00:17, Eliot Kimber wrote:


I’m starting to learn React in the context of enhancing a React-based 
site created by others (lightstep.com/incident-reponse/docs).


I’m also developing a non-trivial web application in BaseX (the 
Project Mirabel system I’ve mentioned before).


For Mirabel I just hacked some pages using RESTXQ and the least CSS 
and JavaScript I could get away with, cribbing liberally from the 
out-of-the-box BaseX web apps.


But as I learn about React and its companion, Material UI, I’m 
thinking that there’s probably a lot of power to combining React with 
RESTXQ as a way to build a top-notch site with lots of interactivity 
at relatively low cost (React and MUI provide all the interaction 
implementation details you might want). But I’m not sure what this 
would mean in practice.


I’m wondering if anyone else has done this or at least thought about 
it enough to have some ideas about what to do or not do?


In addition, at the most recent DITA conference, Convex 2022, George 
Bina presented a very interesting experiment in generating React 
components for DITA topics where the components did the dynamic 
content resolution at delivery time rather than at HTML building time, 
enabling practical incremental update of DITA-based content through a 
React-based site. The same technique could be applied to RESTXQ by 
generating XQuery modules for individual topics. But an advantage of a 
React approach is that it’s more general and could be used with any 
React-based site. But in any case it’s gotten me thinking more about 
the generation of information


I see React being used mostly in statically-generated sites that are 
published through something like Netlify but my Mirabel application is 
very much almost entirely dynamic in that most of the content is the 
result of queries.


One could treat BaseX as a supplier of microservices via a REST API 
and that’s probably how most users of BaseX with React do it, if for 
no other reason than separate groups are probably responsible for the 
web stuff and the BaseX stuff.


But in the case of my Mirabel application it’s a single team doing 
everything with limited web development knowledge but access to 
talented web designers. So having all the web stuff also in BaseX 
makes development and maintenance easier with the team we have. We’re 
not JavaScript programmers but we can manage composing React 
components together and doing the minimal programming needed to make 
them behave appropriately.


Clearly I can just have RESTXQ-generated pages import the React 
libraries and supporting components directly and I can generate JSX 
code as inline scripting and maybe that’s the best approach for an 
all-RESTXQ site but maybe there’s something better?


Or does stepping up to a proper React site that uses BaseX as a source 
of microservices make more sense, even for this small project?


Thanks,

Eliot

_

*Eliot Kimber*

Sr Staff Content Engineer

O: 512 554 9368

M: 512 554 9368

servicenow.com 

LinkedIn  | Twitter 
 | YouTube 

Re: [basex-talk] BaseX 9.7 • Ukraine Edition

[Marco Lettere]

Ten years of a great Software, a Great Company, a great Community and 
great Persons as you are Christian.


And your email demonstrates exactly this!

Happy Birthday!

PEACE

M.

On 23/03/22 13:15, Christian Grün wrote:

Dear all,

Just next to us, millions of people are currently being torn out of
their home country, and we desperately hope that the situation in
Ukraine will calm down soon. For those of you who are planning to
donate money to us in the upcoming weeks, we kindly ask you this time
to redirect it to Médecins Sans Frontières or any NGO of your choice.

– We are back to provide you with a new and fresh version of BaseX. It
comes with a big bunch of advanced performance tweaks and
optimizations and some new XQuery features:

• Support for xsl:messages via xslt:transform-report:
• Support for formatting integers and dates in French
• XSD validation: Improved support for XML catalogs
• Remove namespaces from elements via util:strip-namespaces

For a more comprehensive list of added and updated features, look into
our documentation (docs.basex.org) and check out the GitHub issues
(github.com/BaseXdb/basex/issues).

And one more pleasant announcement: Today, our company BaseX GmbH Our
celebrates its 10th anniversary! Thanks to all of you who have keeping
us busy with challenging projects. Rest assured we’ll be happy to
continue to give you advice and support for the upcoming decades.

All the best,
Your BaseX Team

Re: [basex-talk] BaseX 9.6.4: Tweaks, Fixes & Thank Yous

[Marco Lettere]

Thanks again Christian & BaseX team! Never enough of saying it!

Whenever trying to explaining why OSS is the way to go, I cite your 
product along with Blender as most valuable examples!


Happy 2021 ending and a normal 2022 to all of you!

M.

On 17/12/21 10:17, Christian Grün wrote:

Dear all,

We conclude this year with yet another release of BaseX, our XML
framework, database and XQuery processor [1]. The new version contains
new performance tweaks and minor bug fixes, the details of which can
be examined on our GitHub page [2].

As we’ve got numerous inquiries regarding Log4j: I can comfort you
that BaseX is not affected by the vulnerabilities. We have our own
logging mechanism, which has no built-in code execution features. The
same applies to the Jetty web server.

Log4j once again demonstrates that Open Source is everywhere, and that
it takes valuable time and resources to build professional and secure
software. We send out a big thank you to those of you who are (and
have been) supporting us by paying for maintenance, sponsoring new
features and giving donations. It’s you who is keeping BaseX both
alive and up to date!

Have a good time,
Christian

[1] https://basex.org
[2] https://github.com/BaseXdb/basex/commits/master

Re: [basex-talk] Didn't know ....

[Marco Lettere]

I'll wait for the Winter Edition since I easily could re-arrange for the 
specific point.


But your usual effort is well appreciated!

Thanks Chrisitan.

M.

On 07/12/21 11:40, Christian Grün wrote:

…fixed and deployed [1]. BaseX 9.6.4 will be released in December.

[1] https://files.basex.org/releases/latest/


On Tue, Dec 7, 2021 at 11:04 AM Christian Grün
 wrote:

…definitely a (fairly new) bug, and one that should be fixed soon [1].
I have opened an issue to track this down.

Thanks,
Christian

[1] https://github.com/BaseXdb/basex/issues/2054


On Tue, Dec 7, 2021 at 10:31 AM Martin Honnen  wrote:


Am 07.12.2021 um 10:16 schrieb Marco Lettere:

Hi all,

Just because "abc" || () returns "abc", it felt natural to me that  should return  but I get 
instead.

This is on a 9.6.1.

I can't recall whether this has always been like that thus it's one of
my xquery shortcomings or if actually something strange is going on
here ...



I would see that as a BaseX bug.

[basex-talk] Didn't know ....

[Marco Lettere]

Hi all,

Just because "abc" || () returns "abc", it felt natural to me that attr="abc{()}"/> should return  but I get  
instead.


This is on a 9.6.1.

I can't recall whether this has always been like that thus it's one of 
my xquery shortcomings or if actually something strange is going on here ...


Thanks for clarifications.

Marco.

[basex-talk] Custom map combinator

[Marco Lettere]

Dear all,

just thinking out loudly ... Whether it could be interesting to specify 
a function with proper signature as a custom combinator for map:merge.


Currently "duplicates" : "combine" results in the entry value being a 
sequence of the originating values. This could cause loss of information 
in the case that one of the merged maps does not contain the key. It 
could be useful to use defaults or placeholders instead ...


Maybe even combine the values as strings or numbers 

Too exotic?

M.

Re: [basex-talk] BaseX 9.6: The Summer Edition

[Marco Lettere]

Thank you very much!

M.

On 19/08/21 14:22, Christian Grün wrote:

Dear all,

We provide you with a new and fresh version of BaseX, our open source
XML framework, database system and XQuery 3.1 processor:

   https://basex.org/

Apart from our main focus (query rewritings and optimizations), we
have added the following enhancements:

XQUERY: MODULES, FEATURES
- Archive Module, archive:write: stream large archives to file
- SQL Module: support for more SQL types
- Full-Text Module, ft:thesaurus: perform Thesaurus queries
- Fulltext, fuzzy search: specify Levenshtein limit
- UNROLLLIMIT option: control limit for unrolling loops

XQUERY: JAVA BINDINGS
- Java objects of unknown type are wrapped into function items
- results of constructor calls are returned as function items
- the standard package "java.lang." has become optional
- array arguments can be specified with the middle dot notation
- conversion can be controlled with the WRAPJAVA option
- better support for XQuery arrays and maps

WEB APPLICATIONS
- RESTXQ: Server-Timing HTTP headers are attached to the response

For a more comprehensive list of added and updated features, look into
our documentation (docs.basex.org) and check out the GitHub issues
(github.com/BaseXdb/basex/issues).

Have fun,
Your BaseX Team

Re: [basex-talk] Can XQuery return unique values present across multiple databases?

[Marco Lettere]

Welcome Tamara and yes, Gerrit is completely right about BaseX + RestXQ. 
Most of the time you don't need more! :-D


M.

On 13/08/21 22:07, Imsieke, Gerrit, le-tex wrote:

Tamara! Welcome to another proselyte in the Church of Xquery.

This church is open to people from all walks of life, for example, 
from XSLT or XForms backgrounds.


But not from PHP.

Just kidding, everyone is free to use the tools and languages they got 
accustomed to (until they are made aware of the X stack, that is).


Do more with BaseX and RESTXQ, it is quite rewarding! The community is 
nice, inclusive, and welcoming.


Gerrit

On 13.08.2021 20:50, Tamara Marnell wrote:
Thank you, Martin! I'm new to XQuery and didn't know about 
higher-order functions like fold-left. With a little tweaking, this 
is perfect for my application:


declare variable $d as xs:string external;
declare variable $f as xs:string external;
declare function local:get_eads($facet as xs:string, $db_ids as 
item()+) as item()* {

   let $split := tokenize($facet, ':')
   return db:open('facet-' || $split[1])/terms/term[@text=$split[2] 
and @db=$db_ids]/ead

};
let $db_ids := tokenize($d, '\|')
let $facets := tokenize($f, '\|')
let $eads :=
   fold-left(
     $facets,
     local:get_eads(head($facets), $db_ids),
     function($all_eads, $facet) {
       let $facet_eads := local:get_eads($facet, $db_ids)
       let $eads_in_both := distinct-values($all_eads[.=$facet_eads])
       return $eads_in_both
     }
   )
return {
   for $ead in $eads
     return {$ead}
}

On Thu, Aug 12, 2021 at 11:27 PM Martin Honnen > wrote:



    Am 13.08.2021 um 00:12 schrieb Tamara Marnell:

    Short question: Is it possible to write an XQuery FLWOR statement
    that can return a set of unique values present across multiple
    databases?

    Long question: Our new website in development displays EAD finding
    aids stored across 45 databases in BaseX. I've built "facet"
    databases that index terms in the EADs from controlled
    vocabularies like subjects, places, personal names, etc. The
    indexes follow this structure, where each EAD node contains a
    unique identifier:

    
      
        12345
        67890
      
      
        abcde
      
      {etc.}
    

    In the search interface, users can select multiple facets to apply
    to one search. For example, they could browse database 12 for EADs
    with the subject "Literature" /and/ the place "Oregon," etc.

    I currently use the REST server to run an XQuery file that loops
    through each selected facet and prints /all/ EAD IDs for each
    submitted term and database. Then after results are returned, I
    use PHP to count occurences of each EAD and print them only if the
    total count matches the count of facets used.

    declare variable $d as xs:string external;
    declare variable $f as xs:string external;
    let $db_ids := tokenize($d, '\|')
    return {
    for $facet in tokenize($f, '\|')
      let $split := tokenize($facet, ':')
      let $facet_type := $split[1]
      let $facet_term := $split[2]
      let $facet_db := 'facet-' || $facet_type
      return {
        for $ead in db:open($facet_db)/terms/term[@text=$facet_term
    and @db=$db_ids]/ead
          return $ead
      }
    }

    So in the hypothetical example above, I'd pass "12" as d (or
    multiple selected databases separated by bars) and
    "subject:Literature|geogname:Oregon" as f, and I'd get back a
    document like:

    
      
        12345
        67890
      
      
        12345
      
    

    The count of "12345" will equal the count of the user's selected
    facets, so that result will be printed, but 67890 will not.

    Is there a more efficient way to do this? I'd prefer the XQuery to
    return only the EADs that meet all criteria, so only 12345 would
    be returned because it's in facet-subject under Literature /and/
    in facet-geogname under "Oregon," and then I don't have to do any
    post-processing.



    I think you can use fold-left to reduce the found eas while
    selecting them:



    let $db_ids := tokenize($d, '\|')
    return
     {
     let $facet-maps :=
   fold-left(
     for $facet in tokenize($f, '\|')
   let $split := tokenize($facet, ':')
   let $facet_type := $split[1]
   let $facet_term := $split[2]
   let $facet_db := 'facet-' || $facet_type
     return
   map:merge(
   for $ead in
    db:open($facet_db)/terms/term[@text=$facet_term and @db=$db_ids]/ead
   return map:entry(string($ead), map { 'node' :
    $ead, 'type' : $facet_type, 'term' : $facet_term })
   ,
   map { 'duplicates' : 'combine' }
   )
    ,
    map{},
    function($ams, $m) {
    for $m1 in $ams
    

[basex-talk] Solidarity and closeness

[Marco Lettere]

We at Nubisware would like to express all our solidarity to all the 
members of the list who are currently affected by the flood disaster in 
Central EU.


In particular, our closeness with Christian, Michael and the German 
BaseX team who has been backing our daily work for years.


Looking forward to sustain whatever initiative will be organized to help 
out.


Stay strong.

Marco & the Nubisware team.

Re: [basex-talk] Increase automatic logout time

[Marco Lettere]

Hi Josselin,

what we usually do is to tune the solution on the client and for 
specific page in order to not impact the overal session mechanism for 
all the apps and services hosted by the service.


Whenever it's needed we add a small JS snippet like [1] to keep the page 
alive. Just provide an endpoint responding with 200 OK as 
"/application/ajax/heartbeat" in the example and you are ready to go.


Hope it's useful.

M.

[1]
  window.setInterval(()=>{
    fetch("/application/ajax/heartbeat").then(
  console.log("keep alive")
    )
  }, 6)


On 02/07/21 18:02, Christian Grün wrote:

Dear Josselin,


For the management of the users and the permissions, we relied on the BaseX doc 
(https://docs.basex.org/wiki/Permissions).

If you are storing users in sessions, you can increase the session
timeout by adding the following entry in the web.xml file:


   
   
 480
   
   ...


Hope this helps, more questions are welcome,
Christian



  Globally everything works fine, except that we encounter a kind of
automatic logout of the users maybe about 1 hour after their login,
maybe even less, it’s hard to say… In fact, our forms are quite big
and it often takes more than an hour to finish processing a single
historical resource, and when the user save his work, if he is logged
out, he loses all what he’ve done.

Would you know a way to increase this logout time? On our local BaseX instance, 
we tried changing the database options, including Keepalive 
(https://docs.basex.org/wiki/Options#KEEPALIVE). We changed the value 600 to 
60, just to make sure it was this option that needed to be changed, but it 
didn't seem to have any effect…

For the moment we set up an automatic backup system, but this solution 
introduces other issues and is therefore not ideal.

Thank you in advance for your help!!
Take good care,
Josselin.

Re: [basex-talk] HTTP Client DELETE with body

[Marco Lettere]

Thank you! I didn't want to disturb your weekend though Top quality
BaseX service! 朗

Il sab 12 giu 2021, 16:37 Christian Grün  ha
scritto:

> Grazie, Marco.
>
> You convinced me there is no benefit in keeping the restriction (and
> it we didn’t add it to the draft of version 2 of the spec either [1]).
> I’ve just uploaded a new snapshot [2]; I hope it keeps you going.
>
> Ciao, Christian.
>
> [1] https://expath.github.io/expath-cg/specs/http-client-2/
> [2] https://files.basex.org/releases/latest/
>
>
> > So I guess you’ve stumbled upon an API that expects a payload with
> DELETE?
> >
> > Yes Christian, trying to delete an application role association to a
> user on the Keycloak REST APIs requires the role to be passed as JSON
> object in the DELETE body [1].
> >
> > So at the moment I'm able to add roles to a user but I cannot change
> them because I'm unabale to remove one.
> >
> > Telling just because you asked ;-)
> >
> > Thanks a lot anyway for your attention.
> >
> > [1]
> https://www.keycloak.org/docs-api/11.0/rest-api/index.html#_client_role_mappings_resource
> >
> > Cheers
> > Christian
> >
> > [1] http://expath.org/spec/http-client#d2e104
> >
> >
> >
> >
> > Marco Lettere  schrieb am Sa., 12. Juni 2021,
> 11:48:
> >>
> >> Dear all,
> >>
> >> I have to call an API which model an HTTP DELETE operation based on the
> >> body of the request.
> >>
> >> It looks like currently http:send-request is not allowing body to be
> >> present ("Body not expected for method DELETE.").
> >>
> >> I feel that this is bit too restrictive interpretation of the spec which
> >> states (as far as I recall) that there is no defined semantic for the
> >> operation but not that it is forbidden.
> >>
> >> Is there any possibility to relax this behaviour and allow for bodies in
> >> DELETE calls? [1]
> >>
> >> Thank you very much.
> >>
> >> Marco.
> >>
> >>
> >> [1]
> >>
> https://github.com/BaseXdb/basex/blob/382c8097afb1ec5bac1e9511a2287686c33f669d/basex-core/src/main/java/org/basex/util/http/HttpRequestParser.java
> >>
>

Re: [basex-talk] HTTP Client DELETE with body

[Marco Lettere]

So I guess you’ve stumbled upon an API that expects a payload with DELETE?

Yes Christian, trying to delete an application role association to a 
user on the Keycloak REST APIs requires the role to be passed as JSON 
object in the DELETE body [1].


So at the moment I'm able to add roles to a user but I cannot change 
them because I'm unabale to remove one.


Telling just because you asked ;-)

Thanks a lot anyway for your attention.

[1] 
https://www.keycloak.org/docs-api/11.0/rest-api/index.html#_client_role_mappings_resource



Cheers
Christian

[1] http://expath.org/spec/http-client#d2e104 
<http://expath.org/spec/http-client#d2e104>





Marco Lettere mailto:m.lett...@gmail.com>> 
schrieb am Sa., 12. Juni 2021, 11:48:


Dear all,

I have to call an API which model an HTTP DELETE operation based
on the
body of the request.

It looks like currently http:send-request is not allowing body to be
present ("Body not expected for method DELETE.").

I feel that this is bit too restrictive interpretation of the spec
which
states (as far as I recall) that there is no defined semantic for the
operation but not that it is forbidden.

Is there any possibility to relax this behaviour and allow for
bodies in
DELETE calls? [1]

Thank you very much.

Marco.


[1]

https://github.com/BaseXdb/basex/blob/382c8097afb1ec5bac1e9511a2287686c33f669d/basex-core/src/main/java/org/basex/util/http/HttpRequestParser.java

<https://github.com/BaseXdb/basex/blob/382c8097afb1ec5bac1e9511a2287686c33f669d/basex-core/src/main/java/org/basex/util/http/HttpRequestParser.java>

[basex-talk] HTTP Client DELETE with body

[Marco Lettere]

Dear all,

I have to call an API which model an HTTP DELETE operation based on the 
body of the request.


It looks like currently http:send-request is not allowing body to be 
present ("Body not expected for method DELETE.").


I feel that this is bit too restrictive interpretation of the spec which 
states (as far as I recall) that there is no defined semantic for the 
operation but not that it is forbidden.


Is there any possibility to relax this behaviour and allow for bodies in 
DELETE calls? [1]


Thank you very much.

Marco.


[1] 
https://github.com/BaseXdb/basex/blob/382c8097afb1ec5bac1e9511a2287686c33f669d/basex-core/src/main/java/org/basex/util/http/HttpRequestParser.java

Re: [basex-talk] Make use of external Identity Management for BaseX

[Marco Lettere]

Forgot to mention that for decoding the JWT we needed to hook up Java's 
internal B64 decoder because I don't remember why the one provided with 
basex was not able to handle that encoding.


On 04/05/21 09:51, Marco Lettere wrote:

Hi Rob and all,
I attach here an example of an OIDC code grant flow implemented with 
RestXQ, BaseX permission and error handler.
The file includes a sort of library for performing the steps of the 
OIDC flow plus a minimal application that is registered as public 
client inside keycloak and which is what you should access from your 
browser by calling http://localhost:8984/authtest or 
http://localhost:8984/authtest/internal.
I've put into it also the logout procedure for performing the 
back-channel logout which closes the SSO session.


This is only a resume of a more generic and complex module but it 
should be useful as a howto and it should be as simple to install as 
copying the file to your BaseX' webapp folder. Use it as you like.

Regards,
Marco.

On 03/05/21 12:50, Rob wrote:

Hi Marco,

Got it!
Looking forward to the code! :)

With kind regards,
Rob

Op 3 mei 2021, om 09:23 heeft Marco Lettere <mailto:m.lett...@gmail.com>> het volgende geschreven:


Hi Rob,
no the code in the repository the link refers to is more like 
administrative tooling for batch importing configuration into 
Keycloak for administration purposes.
I'll work on creating an excerpt of the code that realizes Oauth2 
code grant flow with RestXQ and Keycloak and I'll share by tonight 
hopefully.

Regards,
Marco.

On 01/05/21 17:49, Rob wrote:

Hi all!

Sorry for the late response!
I’m reading all the emails now, haha.

I’m very happy to hear that it’s possible, I see you shared the 
following:
https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery 
<https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery>


Thanks for sharing!

Based on the following quote from Marco’s mail:
"We have also a RestXQ module that in combination with basex:perm 
annotations protects access to a GUI implementing the Oauth2 
Code-grant flow. If Rob is interested we could share.”


The link above, is that the RESTXQ module Marco’s been talking about?
Or is there another RESTXQ module?

With kind regards,
Rob

Op 1 mei 2021, om 05:29 heeft Adam Law <mailto:adamjames...@gmail.com>> het volgende geschreven:


If Rob isn't interested, I certain am... Yes please share

>>We have also a RestXQ module that in combination with basex:perm
annotations protects access to a GUI implementing the Oauth2 
Code-grant

flow. If Rob is interested we could share.

Many thanks for sharing 
https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery 
<https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery>


This is a bit forward of me, but would you consider changing the 
license away from European Union Public Licence which appears to 
be 'copyleft' to say BSD 3-Clause, MIT, ...


Adam

On Fri, Apr 30, 2021 at 6:00 PM 
<mailto:basex-talk-requ...@mailman.uni-konstanz.de>> wrote:


Send BaseX-Talk mailing list submissions to
basex-talk@mailman.uni-konstanz.de
<mailto:basex-talk@mailman.uni-konstanz.de>

To subscribe or unsubscribe via the World Wide Web, visit
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
<https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk>
or, via email, send a message with subject or body 'help' to
basex-talk-requ...@mailman.uni-konstanz.de
<mailto:basex-talk-requ...@mailman.uni-konstanz.de>

You can reach the person managing the list at
basex-talk-ow...@mailman.uni-konstanz.de
<mailto:basex-talk-ow...@mailman.uni-konstanz.de>

When replying, please edit your Subject line so it is more
specific
than "Re: Contents of BaseX-Talk digest..."


Today's Topics:

   1. Make use of external Identity Management for BaseX (Rob)
   2. Re: Reloading jars on a running http server. (Reece Dunn)
   3. Getting profile information in server responses. (Reece
Dunn)
   4. Re: Make use of external Identity Management for BaseX
      (Christian Gr?n)
   5. Re: Reloading jars on a running http server. (Christian
Gr?n)
   6. Re: Getting profile information in server responses.
      (Christian Gr?n)
   7. Re: Reloading jars on a running http server. (Reece Dunn)
   8. Re: Reloading jars on a running http server. (Christian
    Gr?n)
   9. Re: Make use of external Identity Management for BaseX
      (Marco Lettere)


--

Message: 1
Date: Thu, 29 Apr 2021 15:54:08 +0200
From: Rob mailto:priv...@tjalma.com>>
To: BaseX mailto:basex-talk@ma

Re: [basex-talk] Make use of external Identity Management for BaseX

[Marco Lettere]

Hi Rob and all,
I attach here an example of an OIDC code grant flow implemented with 
RestXQ, BaseX permission and error handler.
The file includes a sort of library for performing the steps of the OIDC 
flow plus a minimal application that is registered as public client 
inside keycloak and which is what you should access from your browser by 
calling http://localhost:8984/authtest or 
http://localhost:8984/authtest/internal.
I've put into it also the logout procedure for performing the 
back-channel logout which closes the SSO session.


This is only a resume of a more generic and complex module but it should 
be useful as a howto and it should be as simple to install as copying 
the file to your BaseX' webapp folder. Use it as you like.

Regards,
Marco.

On 03/05/21 12:50, Rob wrote:

Hi Marco,

Got it!
Looking forward to the code! :)

With kind regards,
Rob

Op 3 mei 2021, om 09:23 heeft Marco Lettere <mailto:m.lett...@gmail.com>> het volgende geschreven:


Hi Rob,
no the code in the repository the link refers to is more like 
administrative tooling for batch importing configuration into 
Keycloak for administration purposes.
I'll work on creating an excerpt of the code that realizes Oauth2 
code grant flow with RestXQ and Keycloak and I'll share by tonight 
hopefully.

Regards,
Marco.

On 01/05/21 17:49, Rob wrote:

Hi all!

Sorry for the late response!
I’m reading all the emails now, haha.

I’m very happy to hear that it’s possible, I see you shared the 
following:
https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery 
<https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery>


Thanks for sharing!

Based on the following quote from Marco’s mail:
"We have also a RestXQ module that in combination with basex:perm 
annotations protects access to a GUI implementing the Oauth2 
Code-grant flow. If Rob is interested we could share.”


The link above, is that the RESTXQ module Marco’s been talking about?
Or is there another RESTXQ module?

With kind regards,
Rob

Op 1 mei 2021, om 05:29 heeft Adam Law <mailto:adamjames...@gmail.com>> het volgende geschreven:


If Rob isn't interested, I certain am... Yes please share

>>We have also a RestXQ module that in combination with basex:perm
annotations protects access to a GUI implementing the Oauth2 
Code-grant

flow. If Rob is interested we could share.

Many thanks for sharing 
https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery 
<https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery>


This is a bit forward of me, but would you consider changing the 
license away from European Union Public Licence which appears to be 
'copyleft' to say BSD 3-Clause, MIT, ...


Adam

On Fri, Apr 30, 2021 at 6:00 PM 
<mailto:basex-talk-requ...@mailman.uni-konstanz.de>> wrote:


Send BaseX-Talk mailing list submissions to
basex-talk@mailman.uni-konstanz.de
<mailto:basex-talk@mailman.uni-konstanz.de>

To subscribe or unsubscribe via the World Wide Web, visit
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
<https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk>
or, via email, send a message with subject or body 'help' to
basex-talk-requ...@mailman.uni-konstanz.de
<mailto:basex-talk-requ...@mailman.uni-konstanz.de>

You can reach the person managing the list at
basex-talk-ow...@mailman.uni-konstanz.de
<mailto:basex-talk-ow...@mailman.uni-konstanz.de>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of BaseX-Talk digest..."


Today's Topics:

   1. Make use of external Identity Management for BaseX (Rob)
   2. Re: Reloading jars on a running http server. (Reece Dunn)
   3. Getting profile information in server responses. (Reece Dunn)
   4. Re: Make use of external Identity Management for BaseX
      (Christian Gr?n)
   5. Re: Reloading jars on a running http server. (Christian Gr?n)
   6. Re: Getting profile information in server responses.
      (Christian Gr?n)
   7. Re: Reloading jars on a running http server. (Reece Dunn)
   8. Re: Reloading jars on a running http server. (Christian Gr?n)
   9. Re: Make use of external Identity Management for BaseX
      (Marco Lettere)


--

Message: 1
Date: Thu, 29 Apr 2021 15:54:08 +0200
From: Rob mailto:priv...@tjalma.com>>
To: BaseX mailto:basex-talk@mailman.uni-konstanz.de>>
Subject: [basex-talk] Make use of external Identity Management for
        BaseX
Message-ID: <39036ef3-e67d-470d-b2f4-e6fbffdf4...@tjalma.com
<mailto:39036ef3-e67d-470d-b2f4-e6fbffdf4...@tjalma.com&

Re: [basex-talk] Make use of external Identity Management for BaseX

[Marco Lettere]

Hi Rob,
no the code in the repository the link refers to is more like 
administrative tooling for batch importing configuration into Keycloak 
for administration purposes.
I'll work on creating an excerpt of the code that realizes Oauth2 code 
grant flow with RestXQ and Keycloak and I'll share by tonight hopefully.

Regards,
Marco.

On 01/05/21 17:49, Rob wrote:

Hi all!

Sorry for the late response!
I’m reading all the emails now, haha.

I’m very happy to hear that it’s possible, I see you shared the following:
https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery 
<https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery>


Thanks for sharing!

Based on the following quote from Marco’s mail:
"We have also a RestXQ module that in combination with basex:perm 
annotations protects access to a GUI implementing the Oauth2 
Code-grant flow. If Rob is interested we could share.”


The link above, is that the RESTXQ module Marco’s been talking about?
Or is there another RESTXQ module?

With kind regards,
Rob

Op 1 mei 2021, om 05:29 heeft Adam Law <mailto:adamjames...@gmail.com>> het volgende geschreven:


If Rob isn't interested, I certain am... Yes please share

>>We have also a RestXQ module that in combination with basex:perm
annotations protects access to a GUI implementing the Oauth2 Code-grant
flow. If Rob is interested we could share.

Many thanks for sharing 
https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery 
<https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery>


This is a bit forward of me, but would you consider changing the 
license away from European Union Public Licence which appears to be 
'copyleft' to say BSD 3-Clause, MIT, ...


Adam

On Fri, Apr 30, 2021 at 6:00 PM 
<mailto:basex-talk-requ...@mailman.uni-konstanz.de>> wrote:


Send BaseX-Talk mailing list submissions to
basex-talk@mailman.uni-konstanz.de
<mailto:basex-talk@mailman.uni-konstanz.de>

To subscribe or unsubscribe via the World Wide Web, visit
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
<https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk>
or, via email, send a message with subject or body 'help' to
basex-talk-requ...@mailman.uni-konstanz.de
<mailto:basex-talk-requ...@mailman.uni-konstanz.de>

You can reach the person managing the list at
basex-talk-ow...@mailman.uni-konstanz.de
<mailto:basex-talk-ow...@mailman.uni-konstanz.de>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of BaseX-Talk digest..."


Today's Topics:

   1. Make use of external Identity Management for BaseX (Rob)
   2. Re: Reloading jars on a running http server. (Reece Dunn)
   3. Getting profile information in server responses. (Reece Dunn)
   4. Re: Make use of external Identity Management for BaseX
      (Christian Gr?n)
   5. Re: Reloading jars on a running http server. (Christian Gr?n)
   6. Re: Getting profile information in server responses.
      (Christian Gr?n)
   7. Re: Reloading jars on a running http server. (Reece Dunn)
   8. Re: Reloading jars on a running http server. (Christian Gr?n)
       9. Re: Make use of external Identity Management for BaseX
      (Marco Lettere)


--

Message: 1
Date: Thu, 29 Apr 2021 15:54:08 +0200
From: Rob mailto:priv...@tjalma.com>>
To: BaseX mailto:basex-talk@mailman.uni-konstanz.de>>
Subject: [basex-talk] Make use of external Identity Management for
        BaseX
Message-ID: <39036ef3-e67d-470d-b2f4-e6fbffdf4...@tjalma.com
<mailto:39036ef3-e67d-470d-b2f4-e6fbffdf4...@tjalma.com>>
Content-Type: text/plain;       charset=utf-8

Hi,

I have a question.
Can BaseX make use of an Identity Management application like
Keycloak for Users?

I can?t find anything about it :)

With kind regards,
Rob Tjalma

--

Message: 2
Date: Thu, 29 Apr 2021 20:48:09 +0100
From: Reece Dunn mailto:mscl...@googlemail.com>>
To: Christian Gr?n mailto:christian.gr...@gmail.com>>
Cc: BaseX mailto:basex-talk@mailman.uni-konstanz.de>>
Subject: Re: [basex-talk] Reloading jars on a running http server.
Message-ID:
       
mailto:nebya1je4hcrzvagi_serccrxstxmzcwc...@mail.gmail.com>>
Content-Type: text/plain; charset="utf-8"

Hi Christian,

Thanks for the response. Unfortunately, I've not been able to get the
reloading working.

Kind regards,
Reece

On Wed, 21 Apr 2021 at 18:49, Christian Gr?n
mailto:christian.gr...@gm

Re: [basex-talk] Make use of external Identity Management for BaseX

[Marco Lettere]

Hi Adam,
sorry but licensing terms are out of our choice.
We've developed that code in the framework of EU H2020 funded projects 
[1,2] thus we have to adhere to the policies of the Institution we 
cooperate with (Italian Council of Research).


Anyway as I told this code is more a proof of concept for administrative 
activities related to batch creation of clients ... Feel free to use and 
adapt the code as you prefer.


Regards,
Marco.

[1] https://www.blue-cloud.org/
[2] http://project.sobigdata.eu/

On 01/05/21 05:29, Adam Law wrote:

If Rob isn't interested, I certain am... Yes please share

>>We have also a RestXQ module that in combination with basex:perm
annotations protects access to a GUI implementing the Oauth2 Code-grant
flow. If Rob is interested we could share.

Many thanks for sharing 
https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery 
<https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery>


This is a bit forward of me, but would you consider changing the 
license away from European Union Public Licence which appears to be 
'copyleft' to say BSD 3-Clause, MIT, ...


Adam

On Fri, Apr 30, 2021 at 6:00 PM 
<mailto:basex-talk-requ...@mailman.uni-konstanz.de>> wrote:


Send BaseX-Talk mailing list submissions to
basex-talk@mailman.uni-konstanz.de
<mailto:basex-talk@mailman.uni-konstanz.de>

To subscribe or unsubscribe via the World Wide Web, visit
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
<https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk>
or, via email, send a message with subject or body 'help' to
basex-talk-requ...@mailman.uni-konstanz.de
<mailto:basex-talk-requ...@mailman.uni-konstanz.de>

You can reach the person managing the list at
basex-talk-ow...@mailman.uni-konstanz.de
<mailto:basex-talk-ow...@mailman.uni-konstanz.de>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of BaseX-Talk digest..."


Today's Topics:

   1. Make use of external Identity Management for BaseX (Rob)
   2. Re: Reloading jars on a running http server. (Reece Dunn)
   3. Getting profile information in server responses. (Reece Dunn)
   4. Re: Make use of external Identity Management for BaseX
      (Christian Gr?n)
   5. Re: Reloading jars on a running http server. (Christian Gr?n)
   6. Re: Getting profile information in server responses.
      (Christian Gr?n)
   7. Re: Reloading jars on a running http server. (Reece Dunn)
   8. Re: Reloading jars on a running http server. (Christian Gr?n)
   9. Re: Make use of external Identity Management for BaseX
      (Marco Lettere)


--

Message: 1
Date: Thu, 29 Apr 2021 15:54:08 +0200
From: Rob mailto:priv...@tjalma.com>>
To: BaseX mailto:basex-talk@mailman.uni-konstanz.de>>
Subject: [basex-talk] Make use of external Identity Management for
        BaseX
Message-ID: <39036ef3-e67d-470d-b2f4-e6fbffdf4...@tjalma.com
<mailto:39036ef3-e67d-470d-b2f4-e6fbffdf4...@tjalma.com>>
Content-Type: text/plain;       charset=utf-8

Hi,

I have a question.
Can BaseX make use of an Identity Management application like
Keycloak for Users?

I can?t find anything about it :)

With kind regards,
Rob Tjalma

--

Message: 2
Date: Thu, 29 Apr 2021 20:48:09 +0100
From: Reece Dunn mailto:mscl...@googlemail.com>>
To: Christian Gr?n mailto:christian.gr...@gmail.com>>
Cc: BaseX mailto:basex-talk@mailman.uni-konstanz.de>>
Subject: Re: [basex-talk] Reloading jars on a running http server.
Message-ID:
       
mailto:nebya1je4hcrzvagi_serccrxstxmzcwc...@mail.gmail.com>>
Content-Type: text/plain; charset="utf-8"

Hi Christian,

Thanks for the response. Unfortunately, I've not been able to get the
reloading working.

Kind regards,
Reece

On Wed, 21 Apr 2021 at 18:49, Christian Gr?n
mailto:christian.gr...@gmail.com>>
wrote:

> Hi Reece,
>
> If you install your Java code as JAR file in the repository [1], the
> code will be loaded and unloaded every time when your query is
> executed. If you get an error message?
>
>   java.lang.reflect.InaccessibleObjectException: Unable to make
field
> private final jdk.internal.loader.URLClassPath
> java.net.URLClassLoader.ucp accessible: module java.base does not
> "opens java.net <http://java.net>" to unnamed module @79e2c065
>
> ?unloading fails [2], as you?re probably using a more recent version
> of the JDK, which restricts 

Re: [basex-talk] Make use of external Identity Management for BaseX

[Marco Lettere]

Hi Christian and Rob,
as the authors of the code you point to we confirm that interacting with 
Keycloak in XQuery and in particular RestXQ is rather comfortable.
That example is mostly a demonstration code for a tool that scrapes a 
site index and registers sites named gateways as public clients (in 
Oauth2 sense) into a Keycloak instance.
It is a proof of concept (as you can see from the credentials  :-D) that 
has been used as a batch one-time migration in a particular setting.


We have also a RestXQ module that in combination with basex:perm 
annotations protects access to a GUI implementing the Oauth2 Code-grant 
flow. If Rob is interested we could share.


If there should be any plan of integrating Oauth2/OIDC flows more 
tightly into BaseX we could also be available for helping out.


Regards,
Marco.

On 30/04/21 08:47, Christian Grün wrote:

Hi Rob,

There is no prepackaged module for Keycloak (at least I’m aware of
none), but you can write your own code with XQuery. See [1] for an
example how this might look like.

Are you using RESTXQ?

Best,
Christian

[1] 
https://code-repo.d4science.org/gCubeSystem/d4science-keycloak-themes/src/branch/master/src/utils/xquery



On Thu, Apr 29, 2021 at 3:54 PM Rob  wrote:

Hi,

I have a question.
Can BaseX make use of an Identity Management application like Keycloak for 
Users?

I can’t find anything about it :)

With kind regards,
Rob Tjalma

Re: [basex-talk] BaseX 9.5: The Spring Edition

[Marco Lettere]

Really excited!
Thank you very much for everything! As usual!
M.

On 25/02/21 13:01, Christian Grün wrote:

Dear all,

we are more than happy to provide you with a new and fresh version of
BaseX, our XML framework, database system and XQuery 3.1 processor:

   https://basex.org/


...

Re: [basex-talk] Nasty trap with RESTXQ content negotiation

[Marco Lettere]

Hi Christian,

"negotiation code (which got fairly complex, due to all the quality
factor checks)."

I had no doubt about this ... and it was confirmed after looking at the 
code referenceyou attached. I think that server software such as Jetty, 
Tomcat a.s.o. should provide tools and APIs for managing Content 
Negotiation OOTB.
Nevertheless I installed the new snapshot and now my use case is 
correctly supported!

Thanks a lot for the prompt support as usual!
Bis bald,
Marco.

On 18/02/21 21:22, Christian Grün wrote:

Hi Marco,


If I specify an explicit %rest:consumes("*/*") in the annotations of 
fallback(), specific() correctly intercepts the request.

That was misleading: It shouldn’t make a difference if the generic
annotation is specified or omitted. This has been fixed.


Unfortunately at that point calling with "Content-Type: text/plain" which 
should slip through the functions and be catched by fallback() yields instead:

Your observations (and similar ones that we came across in one of our
projects) motivated me to have another intense look at the content
negotiation code (which got fairly complex, due to all the quality
factor checks). I believe I found a solution that matches the
requirements without compromising backward compatibility. Details (for
those who are interested) can be found in [1].

A new snapshot is available [2], I would be happy if you could test it for us.

Ciao,
Christian

[1] 
https://github.com/BaseXdb/basex/commit/bde52a5ef88e327a7e3f6ad35036cec92767b0be
[2] https://files.basex.org/releases/latest/




declare
   %rest:path("test")
   %rest:POST("{$body}")
   %rest:consumes("text/csv", "application/xml", "application/json")
   %output:method("text")
function s:specific($body as document-node()) {
"Treated as document"
};

declare
   %rest:path("test")
   %rest:POST("{$body}")
   %output:method("text")
function s:fallback($body as item()) {
"Treated as binary"
};

[basex-talk] Nasty trap with RESTXQ content negotiation

[Marco Lettere]

Dear all,

First of all, I hope everything is fine and all of you are staying healthy!

When I read again the docs about Content Negotiation in RestXQ I noticed 
the sentence:


Functions can be restricted to specific Media Types. The default type 
is|*/*|.


So I wrote the two RestXQ signaures [1] with the expectation of 
*fallback()* behaving as a catch-all for mime-types not declared in 
*specific(**)*.


At that point, when calling the service with something stating 
"Content-Type: application/xml" BaseX yields the following error:


Stopped at /:
[basex:restxq] Several functions found for path "test":
- ccp_ui:test [text/plain]
- ccp_ui:test2 [text/plain]

If I specify an explicit %rest:consumes("*/*") in the annotations of 
fallback(), specific() correctly intercepts the request.


Unfortunately at that point calling with "Content-Type: text/plain" 
which should slip through the functions and be catched by fallback() 
yields instead:


No function found that matches the request.

So there is no combination, to my knowledge, to implement my use-case 
correctly ... besides handling mime-types explicitly in code (which is 
rather uncomfortable).


I'm using BaseX 9.4.5 at the moment.

Do you have any hint on this?

Thank you very much.

Marco.

[1]

declare
  %rest:path("test")
  %rest:POST("{$body}")
  %rest:consumes("text/csv", "application/xml", "application/json")
  %output:method("text")
function s:specific($body as document-node()) {
   "Treated as document"
};

declare
  %rest:path("test")
  %rest:POST("{$body}")
  %output:method("text")
function s:fallback($body as item()) {
   "Treated as binary"
};

Re: [basex-talk] Dr. Mike Kay's proposals for XPath and XQuery 4.0

[Marco Lettere]

Couple of things directly out of my mind even if more library than core 
language oriented are YAML support and the new proposal for convenient 
HTTP client functions. I remember a proposal by you, Christian, together 
with Adam maybe?


M.

On 16/11/20 15:30, Loren Cahlander wrote:

Please make sure to keep me in the loop.  I will need to keep xqDoc up to date.


On Nov 16, 2020, at 6:28 AM, Christian Grün  wrote:


I don't know how many implementers we have on this list, outside of Christian 
and co., but thought you all might find it interesting.

Everyone: raise your hands and get involved ;)

Michael’s initiative to push for a new version of XQuery, XPath & co.
is very promising indeed. If you are interested in sharing your ideas
which features you would like to see in a new version: Now may still
be a good time for it!




PS I found this announcement on the XML.com Slack channel, fyi.

[1] Mike Kay's post from Slack:
I've been working on fleshing out my ideas for XSLT/XPath 4.0, based on my XML 
Prague paper earlier this year. I'm hoping to gather together a community group 
of some kind to take this forward; meanwhile I've published a very preliminary 
set of drafts:
XSLT: https://www.saxonica.com/qt4specs/XT/Overview-diff.html
F: https://www.saxonica.com/qt4specs/FO/Overview-diff.html
XPath: https://www.saxonica.com/qt4specs/XP/xpath-40-diff.html
XQuery: https://www.saxonica.com/qt4specs/XQ/xquery-40-diff.html
Preparing for the rotten tomatoes: let me know what you think and whether you 
want to participate.
NOTE: each document has a summary of changes at the end. It's probably best to 
go there first.

Re: [basex-talk] BaseX GUI and remote editing

[Marco Lettere]

Thanks for the hint Bridger.
We would like to stay lightweight thus I'd prefer to not introduce big 
IDEs only for pushing code.
So for the moment (at least for me having Linux) I'll use Nautilus 
(Gnome file browser) to connect to the remote folder containing the code 
and opening files locally with BaseX Gui...  we have to be much more 
careful with coworking and conflict management. But with no build 
process (thanks XQ) and developing in rather isolated sections (thanks 
microservices) may be it could work.

We'll see...
Thanks again.
M.

On 13/11/20 17:33, Bridger Dyson-Smith wrote:

Hi Marco,

I'm sure others will have better thoughts on this, so please take this 
with a grain of salt. Jetbrains IntelliJ (and other IDEs) provide 
Remote Deployment options, and I would think Eclipse would offer 
something similar. I.e. write locally, and then push changes to the 
remote server with a keyboard chord. However, these types of 
connections require SSH or something similar.


I don't know if that's a helpful thought or not.
Best,
Bridger

PS Shameless plug from a happy user: the Jetbrains IDEs can leverage 
Reece Dunn's awesome XQuery plugin.


On Fri, Nov 13, 2020 at 9:50 AM Marco Lettere <mailto:m.lett...@gmail.com>> wrote:


Hello all!

We are in the slightly uncomfortable condition to do some remote
coding.
Means write XQ code that has to be kept on a server different from
our PCs.

We don't want to slow down our very agile development workflow by
passing everytime from a Git repo ...

Has anyone here some experience with remote editing of this kind?
Using
BaseX Gui is out of discussion, isn't it?

Thanks,

Marco.

[basex-talk] BaseX GUI and remote editing

[Marco Lettere]

Hello all!

We are in the slightly uncomfortable condition to do some remote coding. 
Means write XQ code that has to be kept on a server different from our PCs.


We don't want to slow down our very agile development workflow by 
passing everytime from a Git repo ...


Has anyone here some experience with remote editing of this kind? Using 
BaseX Gui is out of discussion, isn't it?


Thanks,

Marco.

Re: [basex-talk] Paging results of an XQuery-Search

[Marco Lettere]

Xquery native windowing instructions despite a slightly complex syntax 
are extremely powerfull and you can rely on the optimizer being able to 
optimize where ever possible.

M.

https://www.w3.org/TR/xquery-30/#id-windows

On 25/09/20 15:55, Omar Siam wrote:

Perhaps you want to use fn:subsequence($hits, $start, $num)?
But of course if you have millions of hits that is not efficient enough.

Am 25.09.2020 um 13:23 schrieb Günter Dunz-Wolff:

Hi all,
is it possible to present the results of an XQuery search in parts of 
50? In order to speed up the search, only the first 50 results should 
be displayed and only if the user is interested in further results, 
50 new results each should be displayed. Is this even possible? I did 
not find anything about it in the documentation.


Example:
let $collection := collection("data")
let $hits := $collection//*:s[.//text() contains text {$query_string}]
let $count_hits := count($hits)
return

{
if $count_hits <= 50 then

{for $hit in $hits
return
{$hit}

else
???
}


Thanks for any advice.
Best regards
Guenter

Re: [basex-talk] Accessing Flexmark (Java) from Query

[Marco Lettere]

On 20/09/20 22:42, Andreas Mixich wrote:

Currently I get the error:

   [XPTY0004] com.vladsch.flexmark.parser.Parser$Builder:build: Multiple
functions with 1.


Hi Andreas,

this is because of Java distinguishes method signatures not only by 
number of parameters but also by type. You have to use the middle-dot 
notation to clearly define which method you need as in [1].


Regards,

Marco.

[1] https://docs.basex.org/wiki/Java_Bindings#Functions_and_Variables

Re: [basex-talk] missing web.xml from docker image

[Marco Lettere]

On 15/09/20 10:35, Andy Bunce wrote:
... as BaseX seems to run surprisingly well on ARM64 machines such as 
the Odroid N2 and Raspberry Pi 4.


Absolutely!

I have been running a demo/development site of a customer's web 
application (fully developed in BaseX) on my home Raspberry PI 3 for 
weeks. :-)


Until the cloud service provider of choice resolved some issues they had 
on their side ;-).


M.

Re: [basex-talk] Yaml serialization

[Marco Lettere]

@Marco: Do you know which conversions are provided by FasterXML:
arbitrary XML → YAML, arbitrary YAML → XML, or both? If the underlying
conversion rules are promising, we could include them a 'fasterxml'
format in BaseX (similar to the 'jsonml' format).

As far as I can understand from the docs and the code on github there 
Jackson is a sort of hub with several dataformat modules which are able 
to parse and serialize from the various formats.


I don't know whether, with the term arbitrary, you mean to be able to 
provide your own syntax for the XML serialization. In this case I am not 
sure.


M.

Re: [basex-talk] Yaml serialization

[Marco Lettere]

Hm, not sure but maybe FasterXML's Jackson? It supports alternative non 
JSON data formats among which YAML, XML.

But maybe that one is a huge dependency.
M.

On 12/08/20 12:38, Christian Grün wrote:

Hi Marco, hi Fabrice,

YAML serialization would be a nice feature indeed.

Are you aware of existing (ideally bidirectional) mappings that would 
be appropriate for general use?


Cheers
Christian





Marco Lettere mailto:m.lett...@gmail.com>> 
schrieb am Mi., 12. Aug. 2020, 12:04:


Dear BaseX team,

just out of curiosity, could you imagine what kind of effort it
would be
to provide YAML as another serialization method for BaseX in
addition to
XML, HTML, JSON, CSV?

The datamodel seems pretty close to the JSON one so maybe a very
similar
transformation to XML could be implemented ...

Ciao,

Marco.

[basex-talk] Yaml serialization

[Marco Lettere]

Dear BaseX team,

just out of curiosity, could you imagine what kind of effort it would be 
to provide YAML as another serialization method for BaseX in addition to 
XML, HTML, JSON, CSV?


The datamodel seems pretty close to the JSON one so maybe a very similar 
transformation to XML could be implemented ...


Ciao,

Marco.

Re: [basex-talk] Transaction Support

[Marco Lettere]

I'm not the maximum expert on this because I'm only a user. So maybe 
Christian could correct me.
The PUL is actually an implementation detail. It's not something you 
access explicitly. Thus no. There is no explicit mechanism.
The behaviour (START TRANS, do smthg, COMMIT) is implicit in each and 
every XQuery script that the compiler recognizes to be "updating" 
(a.k.a. performing database modifications).
Even if it accesses multiple databases. So to me this looks as being 
transactional in nature.
For instance when I call a RestXQ service which stores a document in one 
DB and then updates a record in another one and this second operation 
raises an error causing the RestXQ to quit then both the operations will 
be canceled automatically. This is very different from, for example, 
MongoDB.
Of course this means that there are complex locking strategies which you 
have to pay attention to when performing particularly complex operations 
but this is another point.


I hoffe alles goht guet i de Schwiz (hm ... Schwitzerduetsch is hard to 
write)

M.

On 16/07/20 17:55, Reto Peter wrote:


Hi Marco

Thanks for answering

But that means Basex is NOT supporting database transactions

Or is it possible to implement real transactions (START TRANS, do 
something, COMMIT trans) with that PUL or something?


Reto

*From:*BaseX-Talk [mailto:basex-talk-boun...@mailman.uni-konstanz.de] 
*On Behalf Of *Marco Lettere

*Sent:* 16 July 2020 17:46
*To:* basex-talk@mailman.uni-konstanz.de
*Subject:* Re: [basex-talk] Transaction Support

Hi Reto,

AFAIK Basex is transactional in the sense that whenever you start a 
sequence of commands or an XQuery script, all the "updating 
operations" that modify the database are always stored in a PUL (a 
list of potential updates).


Only when the script terminates all the operations on the DB are 
effectively committed.


There is no explicit rollback operation.

Regards,

Marco.

On 16/07/20 17:40, Reto Peter wrote:

I am evaluating BaseX for my XML project.

I need transaction support like

-Start transaction

-Run queries (read, write, update)

-Commit or rollback transaction

When I see the documentation, it lists Transaction Manager. But
when I look at the details, I cannot find anything like that.

Anyone can explain me how is the support, or is there an add-on or
planned something?

Best regards

Reto, Frauenfeld, Schweiz

Re: [basex-talk] Transaction Support

[Marco Lettere]

Hi Reto,
AFAIK Basex is transactional in the sense that whenever you start a 
sequence of commands or an XQuery script, all the "updating operations" 
that modify the database are always stored in a PUL (a list of potential 
updates).
Only when the script terminates all the operations on the DB are 
effectively committed.

There is no explicit rollback operation.
Regards,
Marco.

On 16/07/20 17:40, Reto Peter wrote:


I am evaluating BaseX for my XML project.

I need transaction support like

-Start transaction

-Run queries (read, write, update)

-Commit or rollback transaction

When I see the documentation, it lists Transaction Manager. But when I 
look at the details, I cannot find anything like that.


Anyone can explain me how is the support, or is there an add-on or 
planned something?


Best regards

Reto, Frauenfeld, Schweiz

Re: [basex-talk] BaseX 9.4: The Summer Edition

[Marco Lettere]

Even in this strange 2020 Summertime can now definitely begin with the
Summer edition which looks awesome!!!
Thanks to all of you!
Marco.

Il mar 14 lug 2020, 13:40 Christian Grün  ha
scritto:

> We are glad to give you Version 9.4 of BaseX, our XML framework,
> database system and XQuery 3.1 processor:
>
>   https://basex.org/
>
> We have focused on rewriting and optimizing complex XQuery code and
> speeding up your RESTXQ apps:
>
> WEB APPLICATIONS
> - DBA: support for millions of log entries
> - LOGTRACE option: write trace output to logs or STDERR
> - rest:init: Full or partial initialization of RESTXQ cache
> - basexhttp: register job services
>
> GUI
> - result view: show number of results and result size
> - Shift F4-F7: toggle search options (case, regeex, ..)
> - Ctrl-Shift-B: jump to matching bracket
>
> XQUERY LOCKING
> - single lock option for reading and updating queries
> - Java bindings: annotation added for updating functions
>
> XQUERY FUNCTIONS
> - convenience functions: json:doc, csv:doc, html:doc
> - element names: convert:encode-key, convert:decode-key
> - jobs:eval: option added for writing log entries
> - session module: only create new sessions if required
>
> XQUERY PERFORMANCE
> - boolean comparisons: flatten nested expressions
> - boolean expressions: merge conjunctions
> - comparisons: simplify operands with if expression
> - database functions: always open at compile time
> - databases: propagate to more expressions at compile time
> - filters: inline context for single items
> - filters: rewrite to simple maps
> - FLWOR: inline where clauses into let clauses
> - FLWOR: merge last with return clause, rewrite to simple map
> - FLWOR: rewrite "return if ..." to "where ... return"
> - fn:for-each, fn:filter: rewrite to FLWORs and filters
> - if expression, EBV tests: simplify boolean tests
> - lists, sets, logical expressions: flatten expressions
> - lists: rewrite to union expressions
> - logical expressions: apply more boolean algebra rules
> - paths: remove redundant predicates
> - predicates with name functions: rewrite to name tests
> - predicates: merge expressions, discard redundant tests
> - predicates: remove of superfluous and redundant tests
> - set expressions: merge operands
> - simple map: inline cast expressions
> - simple maps: faster evaluation
> - simple maps: merge operands
> - simple maps: rewrite to paths, optimize for index access
> - switch: rewrite to if expression
> - types: skip redundant checks, promotions  and conversions
>
> For a more comprehensive list of added and updated features, check out
> our documentation (docs.basex.org) and check out the GitHub issues
> (github.com/BaseXdb/basex/issues).
>
> Have fun,
> Your BaseX Team
>

Re: [basex-talk] Websites that use BaseX

[Marco Lettere]

Hi Jonathan,
we as a company have used Nubisware for several "front-facing" 
applications. They are not exactly ecommerce but full blown enterprise 
applications with complex UIs and/or based on APIs.


 * A rating service (with dedicated UI) based on contract ontologies;
 * a system for remote realtime monitoring and failure prediction in
   industrial settings with a websocket push based interface [1,2]
 * An API and viewer/simulator for IHE XDW based clinical workflow
   tracking integrated with the Camunda BPMN Engine
 * A Psychological Patient Record with web based frontend
 * A frontend plus service orchestrator for the daily management of a
   Coworking site.
 * ...

We are very happy of the results and our customers too (which is much 
more important). What our customers like the most, about XQuery/BaseX 
based solutions, is the very low footprint and impact on their 
infrastructures and deployments.

M.

[1] 
https://www.linkedin.com/posts/global-solutions-%26-services_shortendistances-savecosts-customersatisfaction-activity-6678200998059683840-bedJ 

[2] 
https://www.linkedin.com/posts/global-solutions-%26-services_shortendistances-savecosts-customersatisfaction-activity-6673859515550715905-21GS


On 23/06/20 22:56, Liam R. E. Quin wrote:

On Tue, 2020-06-23 at 16:34 -0400, Joe Wicentowski wrote:

Hi all,

I'd welcome any and all contributions to my list of XQuery-powered
resources:

   https://github.com/joewiz/xquery-power

Not a major ecommerce site :) but...

Most Web pages on From Old Books _dot_ Org -
https://www.fromoldbooks.org/ - incorporate some content generated by
XQuery, and many of the pages are entirely generated at runtime by
XQuery, using BaseX.

There may still be some pages using qizx/open.

Please don't add this to your site, but, It's possible to add, ;show-
query=text/plain to the pages under /Search/ to see the XQuery itself.

Liam

Re: [basex-talk] Performance loss between version 9.2.4 and 9.3.2 when executing specific xQuery

[Marco Lettere]

And even more curiously, we are also working in this very same time on 
handling fetches from OAI-PMH sources! :-D

M.

On 08/05/20 13:37, Imsieke, Gerrit, le-tex wrote:
Just saying that I find it sooo interesting to learn at which places 
and for which purposes BaseX is being employed. Have a nice weekend!


On 08.05.2020 13:31, BIRKNER Michael wrote:

Hi Christian,


thank you for your answers. As you can guess the queries I sent in my 
original email are just simplified  examples.



The real XML structure is like the following (its library data in 
format "MarcXML", here you see an example: 
https://www.loc.gov/standards/marcxml/Sandburg/sandburg.xml)




… … …



Mag. Michael Birkner
AK Wien - Bibliothek
1040, Prinz Eugen Straße 20-22
T: +43 1 501 65 12455
F: +43 1 501 65 142455
M: +43 664 88957669

Re: [basex-talk] A question about RESTXQ rest:form-param()

[Marco Lettere]

I set up this restxq [1].
And call it with something like [2].
Got an output of [3] which looks like what I expect.
M.

[1]
module namespace t = "urn:test";

declare
  %rest:path("test")
  %rest:POST
  %rest:consumes("application/x-www-form-urlencoded")
  %output:method("html")
function t:test() {
 
   
 {request:parameter-names() ! (. || "=" || 
request:parameter(.))}

   
 
};

[2]
curl --location --request POST 'http://localhost:8984/test' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'param1=Hello' \
--data-urlencode 'param2=World'

[3]


param1=Hello param2=World



On 22/04/20 16:03, Zimmel, Daniel wrote:

the mime type seems correct, all params are being sent. I do not have problems 
getting the body, only request:parameter-names() does not give any values. 
(request:header works, for example)

-----Ursprüngliche Nachricht-
Von: Marco Lettere 
Gesendet: Mittwoch, 22. April 2020 15:31
An: Zimmel, Daniel ; basex-talk@mailman.uni-konstanz.de
Betreff: Re: AW: [basex-talk] A question about RESTXQ rest:form-param()

Are you setting the mime type to "application/x-www-form-urlencoded"?
This is the prerequisite in order to get the body of a post parsed to form 
parameters.
M.

On 22/04/20 15:22, Zimmel, Daniel wrote:

Thanks for the hint Marco,

I just tried request:parameter-names(): that one works with GET but with POST I 
get an empty-sequence, I am not sure what the problem is here.
I can access the full (unparsed) body, though (I can use that).

Daniel

-Ursprüngliche Nachricht-
Von: Marco Lettere 
Gesendet: Mittwoch, 22. April 2020 14:24
An: basex-talk@mailman.uni-konstanz.de
Betreff: Re: [basex-talk] A question about RESTXQ rest:form-param()

Hi Daniel,
another way is to use request:* functions [1] available for programmatic 
inspection of the request.
Cheers,
M.

[1] https://docs.basex.org/wiki/Request_Module


On 22/04/20 12:18, Zimmel, Daniel wrote:

Hi Daniel,

RTFM. Simply pass the body in %rest:POST("{$body}").

Sorry for disturbing!

Daniel

-Ursprüngliche Nachricht-
Von: Zimmel, Daniel 
Gesendet: Mittwoch, 22. April 2020 11:27
An: BaseX 
Betreff: [basex-talk] A question about RESTXQ rest:form-param()

Hi,

in an HTML form with a RESTXQ POST action, do I really have to declare 
%rest:form-param() for every param, or is there a way to pass the full query 
component in one parameter?

Best, Daniel

Re: [basex-talk] New Slack workspace for XML discussions: xmlcom.slack.com

[Marco Lettere]

Yeah! I'm in it!
M.

On 17/04/20 15:35, Christian Grün wrote:

Dear XML aficionados,

For interactive discussions on XML, XQuery and BaseX, feel free to
join the new XML workspace on Slack:

https://join.slack.com/t/xmlcom/shared_invite/zt-dkmwrygj-kdviJUKoXb7_ERdCoJSAfQ

Thanks to Adam Retter for opening the workspace.
Thanks to Steven Majewski for forwarding the link.

Have fun,
Christian
__ _  _   _ _

On Thu, Apr 16, 2020 at 7:34 PM Majewski, Steven Dennis (sdm7g)
 wrote:

FYI:

Begin forwarded message:

From: "Adam Retter adam.ret...@googlemail.com" 

Subject: [xsl] [OT] New Slack workspace for XML discussions
Date: April 16, 2020 at 11:59:25 AM EDT
To: xsl-l...@lists.mulberrytech.com
Reply-To: 

Hopefully the moderators won't mind me posting this? I think it has
value for readers of this list...

Just a quick heads up in-case anyone is interested, there is now a new
vendor neutral Slack Workspace for discussing all things XML related -
https://www.xml.com/news/2020-04-slack-workspace-for-the-xml-community/

Cheers Adam.
--
Adam Retter

skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk
--~
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
EasyUnsubscribe: http://lists.mulberrytech.com/unsub/xsl-list/504751
or by email: xsl-list-un...@lists.mulberrytech.com
--~--

Re: [basex-talk] BaseX 9.3.2: Performance Tweaks, Bug Fixes

[Marco Lettere]

Currently we are not very used to good news ... So, thanks Christian for 
this one who definitely sounds like that!


M.

On 10/03/20 14:18, Christian Grün wrote:

Dear all,

We are glad to provide you with a new BaseX 9.3.2 maintenance release:

http://basex.org

In preparation to version 9.4, we have introduced numerous small query
optimizations that speed up path, predicate and simple map operations
and will rewrite your queries for index access more often than before.
Feel free to check out the full list of added rewritings and
enhancements on Github:

   https://github.com/BaseXdb/basex/issues?q=is%3Aissue+is%3Aclosed

Have fun with the new release,

Christian
BaseX Team

Re: [basex-talk] BaseX 9.3: The Winter Edition

[Marco Lettere]

Thank you all so much!
The list of improvements is really impressive!
Great job as usual.
M.

On 29/11/19 17:14, Christian Grün wrote:

Hello to our loyal readers of our list,

We are glad to announce version 9.3 of BaseX, our XML framework,
database system and XQuery 3.1 processor:

   http://basex.org/

The focus has been put on in-depth optimizations of complex XQuery
code and faster processing of RESTXQ web applications:

RESTXQ
- seamless streaming of custom responses to the client
- Request Module: bind attributes to current request
- better support for the OPTIONS and HEAD methods
- trace output is now stored in the database logs
- enforce premature HTTP response with web:error
- optional GZIP compression of HTTP responses
- server-side forwarding with web:forward

USER INTERFACES
- GUI: project view: skip directories with .ignore file
- GUI: project view: do now follow symbolic links
- DBA: user panel updated to show user info
- DBA: faster processing of log entries

STORAGE
- hot fix for replacing single documents with namespaces
- improved XQuery processing of binary resources
- ADDCACHE: faster caching of single documents
- WITHDB: less locking when using fn:doc

XQUERY FUNCTIONS
- user:info/user:update-info: process user-specific data
- db:open-id, db:open-pre: support for multiple ids
- file:descendants: return descendant file paths
- inspect:type: return type of a value as string
- db:alter-backup: rename database backups
- xslt:transform: support for catalog files

XQUERY PERFORMANCE
- merge of adjacent/redundant steps in paths and predicates
- removal of redundant switch, typeswitch and catch clauses
- interruption of side-effecting code (Files Modul etc.)
- fn:count: pre-evaluation of zero-or-one arguments
- faster processing of alternative steps: a/(b,c)
- rewriting of paths to simple map expressions
- inlining of values in simple map expressions
- flattening of nested simple map expressions
- rewriting of lists to union expressions
- removal of superfluous predicates

For a more comprehensive list of added and updated features. please
check out our documentation: http://docs.basex.org/.

Your feedback is welcome as usual.

Have fun with the new release,
Your BaseX Team

Re: [basex-talk] file:list and symbolic links

[Marco Lettere]

I see your point Christian. Thank you.
Of course recursively following the dir structure may work.
Passing a boolean to switch the behaviour with a developer choice might be
viable? Or is the file:list function standardized by xquery or expath spec?
Have a nice weekend!
M.

Il sab 9 nov 2019, 18:36 Christian Grün  ha
scritto:

> Hi Marco,
>
> The rationale is that symbolic links can lead to cycles if you have
> circular dependencies in your directory structure. What I learnt is
> that this happens quite frequently in practice.
>
> You can write an XQuery function with file:list and file:children,
> which recursively calls itself for each directory entry. The following
> function returns paths to all descendant files, excluding directories;
> it should follow all symbolic links:
>
>   declare function file:traverse($dir) {
> for $child in file:children($dir)
> return if (file:is-dir($child)) then (
>   file:traverse($child)
> ) else (
>   $child
> )
>   };
>
> Hope this helps,
> Christian
>
>
> On Wed, Nov 6, 2019 at 11:41 AM Marco Lettere  wrote:
> >
> > Dear all,
> >
> > what is the rationale behind the fact that file:list has been rewritten
> > to not following symbolic links [1]?
> >
> > We had some jobs related code that relied on symlinks and stopped
> > working after moving to new version of BaseX.
> >
> > Is there any work around? By using children or the new descendants
> > functions maybe?
> >
> > Thanks for your support.
> >
> > Marco.
> >
> > [1]
> >
> https://github.com/BaseXdb/basex/blob/bbb62d1ec3e10b4fa1cc48bd4e16911fa73e46df/basex-core/src/main/java/org/basex/query/func/file/FileList.java#L103
> >
>

[basex-talk] file:list and symbolic links

[Marco Lettere]

Dear all,

what is the rationale behind the fact that file:list has been rewritten 
to not following symbolic links [1]?


We had some jobs related code that relied on symlinks and stopped 
working after moving to new version of BaseX.


Is there any work around? By using children or the new descendants 
functions maybe?


Thanks for your support.

Marco.

[1] 
https://github.com/BaseXdb/basex/blob/bbb62d1ec3e10b4fa1cc48bd4e16911fa73e46df/basex-core/src/main/java/org/basex/query/func/file/FileList.java#L103

Re: [basex-talk] Database corrupted on updating with RestXQ

[Marco Lettere]

Yes!
I can confirm that the bug disappeared on the scenario used to reproduce it!
Thank again.
Marco.

On 26/09/19 14:37, Christian Grün wrote:


Is it automatically available shipped in docker?


I guess so ("Updated 38 minutes ago"):

https://hub.docker.com/u/basex



The current scenario where to put it immediately at test is based
on the
docker distribution and it's taking too much time at the moment to
setup
a different test env.
I've received your answer about concurrent XQuery execution. I'm
currently elaborating my observations on this. Hop to be able to
formulate them during WE.
Cheers,
Marco.

On 26/09/19 13:55, Christian Grün wrote:
> Hi Marco,
>
> Finally some news on the storage bug you reported: It is fixed with
> the latest snapshot [1].
>
> Looking forward to your tests (and I hope you received my
response on
> parallel querying),
> Christian
>
> [1] http://files.basex.org/releases/latest/
>
>
>
> On Tue, Aug 6, 2019 at 9:38 AM Christian Grün
mailto:christian.gr...@gmail.com>> wrote:
>> Hi Marco,
>>
>> I think that the bug fix (which is still on my todo list) will
be made
>> available with BaseX 9.3; so, for now, it’s probably better to
choose
>> the workaround.
>>
    >> Cheers,
>> Christian
>>
>>
>>
>> On Sat, Aug 3, 2019 at 10:02 AM Marco Lettere
mailto:m.lett...@gmail.com>> wrote:
>>> Hi Christian,
>>> I'm currently preparing a deployment based on Docker for one
of our
>>> customers. Here in Italy it's holiday time in August so I have
a bit of
>>> time and I can coope with your suggested workaround for the
moment.
>>> Just a question to be prepared ... are intermediate bug-fix
releases
>>> also available in the form of docker containers?
>>> Thanks for your support as usual.
>>> Regards,
>>> Marco.
>>>
>>> On 02/08/19 15:04, Christian Grün wrote:
>>>> Hi Marco,
>>>>
>>>> A little update on the status quo:
>>>>
>>>> • I noticed that the bug you observed was basically exposed
by the
>>>> previous bug fix (it existed before, but it didn’t occur in your
>>>> particular setting).
>>>> • It happens only in certain cases (i.e., for specific
document sizes)
>>>> and if the added/replaced document has namespaces.
>>>> • Another good thing: It happens only if your database is empty.
>>>>
>>>> It’s not exactly an elegant proposal, but as long as we
haven’t fixed
>>>> the bug, you could add an initial  document to your
database.
>>>>
>>>> We are doing our best, though.
>>>> Christian
>>>>
>>>>
>>>>
>>>> On Wed, Jul 31, 2019 at 10:20 AM Christian Grün
>>>> mailto:christian.gr...@gmail.com>> wrote:
>>>>> Marco, thanks for the attachment. I have created a script that
>>>>> triggers the error [1]. Most probably, the bug is related to the
>>>>> previous bug issue. We’ll have a look at this soon. – Christian
>>>>>
>>>>> [1] https://github.com/BaseXdb/basex/issues/1711
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere
mailto:m.lett...@gmail.com>> wrote:
>>>>>> Thanks Christian,
>>>>>> I haven't been able to reproduce the bug with your SSCE.
>>>>>> Nevertheless I spent some time in tracing the operations
with the actual
>>>>>> data I'm currently storing into the DB and a sequence of
>>>>>> db:add/db:replace close enough to actual app flow.
>>>>>> I attach the query script which should be rather self
explaining.
>>>>>> You may run the script changing always $f with $f + 1
starting (from 0).
>>>>>> By uncommenting the different configurations of the
variable $input you
>>>>>> can see how the misbehaviour depends somehow on the data
because only
>>>>>> $d8 is causing the crash.
>>>>>> I stared at the diffs from $d8 to the other but there
really isn't any
>>>>>> significant difference so now it's very hard

Re: [basex-talk] Database corrupted on updating with RestXQ

[Marco Lettere]

Thanks Christian.
Is it automatically available shipped in docker?
The current scenario where to put it immediately at test is based on the 
docker distribution and it's taking too much time at the moment to setup 
a different test env.
I've received your answer about concurrent XQuery execution. I'm 
currently elaborating my observations on this. Hop to be able to 
formulate them during WE.

Cheers,
Marco.

On 26/09/19 13:55, Christian Grün wrote:

Hi Marco,

Finally some news on the storage bug you reported: It is fixed with
the latest snapshot [1].

Looking forward to your tests (and I hope you received my response on
parallel querying),
Christian

[1] http://files.basex.org/releases/latest/



On Tue, Aug 6, 2019 at 9:38 AM Christian Grün  wrote:

Hi Marco,

I think that the bug fix (which is still on my todo list) will be made
available with BaseX 9.3; so, for now, it’s probably better to choose
the workaround.

Cheers,
Christian



On Sat, Aug 3, 2019 at 10:02 AM Marco Lettere  wrote:

Hi Christian,
I'm currently preparing a deployment based on Docker for one of our
customers. Here in Italy it's holiday time in August so I have a bit of
time and I can coope with your suggested workaround for the moment.
Just a question to be prepared ... are intermediate bug-fix releases
also available in the form of docker containers?
Thanks for your support as usual.
Regards,
Marco.

On 02/08/19 15:04, Christian Grün wrote:

Hi Marco,

A little update on the status quo:

• I noticed that the bug you observed was basically exposed by the
previous bug fix (it existed before, but it didn’t occur in your
particular setting).
• It happens only in certain cases (i.e., for specific document sizes)
and if the added/replaced document has namespaces.
• Another good thing: It happens only if your database is empty.

It’s not exactly an elegant proposal, but as long as we haven’t fixed
the bug, you could add an initial  document to your database.

We are doing our best, though.
Christian



On Wed, Jul 31, 2019 at 10:20 AM Christian Grün
 wrote:

Marco, thanks for the attachment. I have created a script that
triggers the error [1]. Most probably, the bug is related to the
previous bug issue. We’ll have a look at this soon. – Christian

[1] https://github.com/BaseXdb/basex/issues/1711



On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere  wrote:

Thanks Christian,
I haven't been able to reproduce the bug with your SSCE.
Nevertheless I spent some time in tracing the operations with the actual
data I'm currently storing into the DB and a sequence of
db:add/db:replace close enough to actual app flow.
I attach the query script which should be rather self explaining.
You may run the script changing always $f with $f + 1 starting (from 0).
By uncommenting the different configurations of the variable $input you
can see how the misbehaviour depends somehow on the data because only
$d8 is causing the crash.
I stared at the diffs from $d8 to the other but there really isn't any
significant difference so now it's very hard for me to understand.

Thank you in advance for all support that you can provide as usual.
Regards,
Marco.

On 26/07/19 18:41, Christian Grün wrote:

Hi Marco,

Back in February, a user reported a database inconsistency [1] – which
happened, too, if a database was nearly empty – and we managed to
build a little text case to get this reproduced [2]. After that, 9.2
was released. Maybe this fix introduced another irregularity for this
special case?

Maybe we can get this reproduced by taking the script from issue 1662
and modify it a little? That would be great.

Sorry to the inconvenience,
Christian

PS: You can safely ignore the "Creating database..." output. It may
just indicate that an XML document is parsed, and that an internal
main-memory database instance is created, possibly for your users.xml
file in the data directory.

[1] 
https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg11459.html
[2] https://github.com/BaseXdb/basex/issues/1662



On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere  wrote:

Hi all,

starting with 9.2.1 we experience a strange error with a RESTXQ API  of
ours that we have been using for years.

The typical pattern is lookup a document update it and store it back.

We have done this milions of time and also all the tests work neatly.

But when using it from inside the web application, at around the tenth
operation we get the DB corrupted with the stack trace [1]. This seems
to happen only when the database is nearly empty and, for sure, it
doesn't appear on 8.x series.

It feels like some concurrency flaw is introduced somewhere but it's
hard to say because the API is eight years in age and it never changed
significantly.

I know it's hard without an SSCE... but while working to insulate this
byzantine behaviour (I've been trying for days now) maybe someone in the
list has a clue or is able to interpret the stack trace?

Thanks and have a nice weekend,

Marco.


[1]

Im

[basex-talk] Suggestion for concurrency of RestXQ

[Marco Lettere]

Dear all,

on one of our applications based on RestXQ services running on 9.0.2 we 
experience a strange behaviour related to concurrency.


It is a stateless service running on a server with http-local set to 
true. So no database access. We removed all accesses to sessions and the 
code doesn't even write to files or other external resources. It's 
really only computation.


When I run multiple requests, I can clearly see that they are accepted 
simultaneously (up to 8 which matches PARALLEL). They even complete out 
of order according to the session id reported in the log. [1]


The strange thing is: performance seems to be as if calls were executed 
serially. If T is the time to complete one call then the pattern is T + 
T + T + T + T.


I have to add that the algorithm is rather complex. Lots of functions 
(possibly inlined, private and the main one even recursive) which filter 
and transform a large XML request into a possibly large XML output.


If I simplify the called RestXQ to something like [2] then I get a log 
like [3] which shows a pattern like the one I'm expecting T+d+d+d+d 
where d is just a slight overhead (orders of magnitude smaller).


My question is what do I have to check in order to understand where the 
hard synchronization of the calls occurs? Might this be related to 
non-deterministic functions? What else?


I've tried to isolate parts of the code but I'm unable to get 
predictable results.


Thank you very much for all the hints you will be able to provide!

Regards,

Marco.

[1]

17:10:19.204    127.0.0.1:58038    admin    REQUEST    [POST] 
http://localhost:8081/rate-selector/select-matching-rates
17:10:19.204    127.0.0.1:58040    admin    REQUEST [POST] 
http://localhost:8081/rate-selector/select-matching-rates
17:10:19.205    127.0.0.1:58042    admin    REQUEST [POST] 
http://localhost:8081/rate-selector/select-matching-rates
17:10:19.205    127.0.0.1:58044    admin    REQUEST [POST] 
http://localhost:8081/rate-selector/select-matching-rates
17:10:19.208    127.0.0.1:58046    admin    REQUEST [POST] 
http://localhost:8081/rate-selector/select-matching-rates

17:10:21.325    127.0.0.1:58040    admin    200 2121.01 ms
17:10:23.174    127.0.0.1:58046    admin    200 3965.74 ms
17:10:25.035    127.0.0.1:58038    admin    200 5830.63 ms
17:10:26.889    127.0.0.1:58044    admin    200 7683.76 ms
17:10:28.739    127.0.0.1:58042    admin    200 9534.46 ms


[2]

(:~
 : Test concurrency
 :)
declare
  %rest:path("/test")
  %rest:POST("{$body}")
function rs:test($body as node()) {
   {$body}{fold-left(1 to 1000, 0, function($acc, $next){ $acc + 
math:sqrt($next)})}

};


[3]

17:48:02.782    127.0.0.1:48262    admin    REQUEST    [POST] 
http://localhost:8081/rate-selector/test
17:48:02.783    127.0.0.1:48252    admin    REQUEST    [POST] 
http://localhost:8081/rate-selector/test
17:48:02.783    127.0.0.1:48258    admin    REQUEST    [POST] 
http://localhost:8081/rate-selector/test
17:48:02.784    127.0.0.1:48260    admin    REQUEST    [POST] 
http://localhost:8081/rate-selector/test
17:48:02.784    127.0.0.1:48254    admin    REQUEST    [POST] 
http://localhost:8081/rate-selector/test

17:48:04.428    127.0.0.1:48262    admin    200        1646.03 ms
17:48:04.509    127.0.0.1:48260    admin    200        1725.45 ms
17:48:04.710    127.0.0.1:48254    admin    200        1926.27 ms
17:48:04.772    127.0.0.1:48258    admin    200        1988.41 ms
17:48:04.904    127.0.0.1:48252    admin    200        2120.33 ms

Re: [basex-talk] Database corrupted on updating with RestXQ

[Marco Lettere]

Hi Christian,
I'm currently preparing a deployment based on Docker for one of our 
customers. Here in Italy it's holiday time in August so I have a bit of 
time and I can coope with your suggested workaround for the moment.
Just a question to be prepared ... are intermediate bug-fix releases 
also available in the form of docker containers?

Thanks for your support as usual.
Regards,
Marco.

On 02/08/19 15:04, Christian Grün wrote:

Hi Marco,

A little update on the status quo:

• I noticed that the bug you observed was basically exposed by the
previous bug fix (it existed before, but it didn’t occur in your
particular setting).
• It happens only in certain cases (i.e., for specific document sizes)
and if the added/replaced document has namespaces.
• Another good thing: It happens only if your database is empty.

It’s not exactly an elegant proposal, but as long as we haven’t fixed
the bug, you could add an initial  document to your database.

We are doing our best, though.
Christian



On Wed, Jul 31, 2019 at 10:20 AM Christian Grün
 wrote:

Marco, thanks for the attachment. I have created a script that
triggers the error [1]. Most probably, the bug is related to the
previous bug issue. We’ll have a look at this soon. – Christian

[1] https://github.com/BaseXdb/basex/issues/1711



On Sat, Jul 27, 2019 at 4:02 PM Marco Lettere  wrote:

Thanks Christian,
I haven't been able to reproduce the bug with your SSCE.
Nevertheless I spent some time in tracing the operations with the actual
data I'm currently storing into the DB and a sequence of
db:add/db:replace close enough to actual app flow.
I attach the query script which should be rather self explaining.
You may run the script changing always $f with $f + 1 starting (from 0).
By uncommenting the different configurations of the variable $input you
can see how the misbehaviour depends somehow on the data because only
$d8 is causing the crash.
I stared at the diffs from $d8 to the other but there really isn't any
significant difference so now it's very hard for me to understand.

Thank you in advance for all support that you can provide as usual.
Regards,
Marco.

On 26/07/19 18:41, Christian Grün wrote:

Hi Marco,

Back in February, a user reported a database inconsistency [1] – which
happened, too, if a database was nearly empty – and we managed to
build a little text case to get this reproduced [2]. After that, 9.2
was released. Maybe this fix introduced another irregularity for this
special case?

Maybe we can get this reproduced by taking the script from issue 1662
and modify it a little? That would be great.

Sorry to the inconvenience,
Christian

PS: You can safely ignore the "Creating database..." output. It may
just indicate that an XML document is parsed, and that an internal
main-memory database instance is created, possibly for your users.xml
file in the data directory.

[1] 
https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg11459.html
[2] https://github.com/BaseXdb/basex/issues/1662



On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere  wrote:

Hi all,

starting with 9.2.1 we experience a strange error with a RESTXQ API  of
ours that we have been using for years.

The typical pattern is lookup a document update it and store it back.

We have done this milions of time and also all the tests work neatly.

But when using it from inside the web application, at around the tenth
operation we get the DB corrupted with the stack trace [1]. This seems
to happen only when the database is nearly empty and, for sure, it
doesn't appear on 8.x series.

It feels like some concurrency flaw is introduced somewhere but it's
hard to say because the API is eight years in age and it never changed
significantly.

I know it's hard without an SSCE... but while working to insulate this
byzantine behaviour (I've been trying for days now) maybe someone in the
list has a clue or is able to interpret the stack trace?

Thanks and have a nice weekend,

Marco.


[1]

Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 9.3 beta
Java: Ubuntu, 13
OS: Linux, amd64
Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
   at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507)
   at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468)
   at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156)
   at org.basex.data.Data.kind(Data.java:304)
   at org.basex.query.value.node.DBNode.(DBNode.java:50)
   at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139)
   at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164)
   at org.basex.query.func.db.DbOpen.value(DbOpen.java:19)
   at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82)
   at org.basex.query.expr.Arr.inline(Arr.java:69)
   at org.basex.query.expr.path.Path.inline(Pat

Re: [basex-talk] Forcing download of JSON doc

[Marco Lettere]

Maybe using json:serialize() rather than serialize?

In general if output:method is set to Json it should be sufficient to 
return map{} or array{} or alternatively the XML version of a json 
document without having to explicitly serialize ...


M.

On 01/08/19 14:36, Tim Thompson wrote:
Thank you, Marco. So, the download issue seems to be with XForms 
(XSLTForms) rather than BaseX. If I do a file upload from a plain HTML 
form, the file download is triggered correctly. However, even though I 
have set "application/json" as the Content-Type, the contents of the 
downloaded file are an escaped string rather than a serialized JSON 
object: e.g.,


"{\n  \"key\": \"value\"\n}"
Is there a way to produce a properly serialized JSON file for download?

--
Tim A. Thompson
Discovery Metadata Librarian
Yale University Library

On Thu, Aug 1, 2019 at 3:03 AM Marco Lettere <mailto:m.lett...@gmail.com>> wrote:


Hi Tim,
downloading from a browser when posting a form is usually a bit
tricky. For me it always ended up with the creation of an
artificial  element with an href crafted from the form
parameters and with an attribute download="filename.ext" added to it.
Anyway you could try with returning a "forced"
application/octet-stream Content-Type header 

M.

On 01/08/19 05:00, Tim Thompson wrote:

Hello,

I am posting a JSON doc to a RESTXQ function from XForms and
trying to set a Content-Disposition header to trigger a file
download. Is this possible to do without first saving the JSON
doc as a file? Right now, the RESTXQ function returns the correct
headers, but does not force a file download dialog.

Thanks in advance!
Tim

--
Query:

xquery version "3.1";

module namespace wrap = "http://example.org/wrap;;

declare
  %rest:path("/wrap")
  %rest:POST("{$data}")
  %output:method("json")
function wrap:process(
  $data as node()
) as item()* {
  let $name := data($data/data/upload/file/@name)
  let $json := json-to-xml($data/data/upload/file)
  let $header :=
    ``[attachment; filename="`{$name}`";]``
  return
    (
    
      
        
        
      
    ,

    http://www.w3.org/2005/xpath-functions;>
        
          {$json}
          Wrapper_Profile
          Wrapper:Profile:BF2
          Profile wrapper for resource
templates
          Wrapper
          https://ld4p.github.io/sinopia/schemas/0.0.9/profile.json
          {current-dateTime()}
        
     => xml-to-json() => parse-json() => serialize(map
{'method': 'json', 'use-character-maps' : map {'/' : '/'}})
    )
};


--
Tim A. Thompson
Discovery Metadata Librarian
Yale University Library

Re: [basex-talk] Forcing download of JSON doc

[Marco Lettere]

Hi Tim,
downloading from a browser when posting a form is usually a bit tricky. 
For me it always ended up with the creation of an artificial  
element with an href crafted from the form parameters and with an 
attribute download="filename.ext" added to it.
Anyway you could try with returning a "forced" application/octet-stream 
Content-Type header 


M.

On 01/08/19 05:00, Tim Thompson wrote:

Hello,

I am posting a JSON doc to a RESTXQ function from XForms and trying to 
set a Content-Disposition header to trigger a file download. Is this 
possible to do without first saving the JSON doc as a file? Right now, 
the RESTXQ function returns the correct headers, but does not force a 
file download dialog.


Thanks in advance!
Tim

--
Query:

xquery version "3.1";

module namespace wrap = "http://example.org/wrap;;

declare
  %rest:path("/wrap")
  %rest:POST("{$data}")
  %output:method("json")
function wrap:process(
  $data as node()
) as item()* {
  let $name := data($data/data/upload/file/@name)
  let $json := json-to-xml($data/data/upload/file)
  let $header :=
    ``[attachment; filename="`{$name}`";]``
  return
    (
    
      
        
        
      
    ,

    http://www.w3.org/2005/xpath-functions;>
        
          {$json}
          Wrapper_Profile
          Wrapper:Profile:BF2
                      key="description">Profile wrapper for resource 
templates

          Wrapper
                      
key="schema">https://ld4p.github.io/sinopia/schemas/0.0.9/profile.json

          {current-dateTime()}
        
     => xml-to-json() => parse-json() => serialize(map 
{'method': 'json', 'use-character-maps' : map {'/' : '/'}})

    )
};


--
Tim A. Thompson
Discovery Metadata Librarian
Yale University Library

Re: [basex-talk] Database corrupted on updating with RestXQ

[Marco Lettere]

Thanks Christian,
I haven't been able to reproduce the bug with your SSCE.
Nevertheless I spent some time in tracing the operations with the actual 
data I'm currently storing into the DB and a sequence of 
db:add/db:replace close enough to actual app flow.

I attach the query script which should be rather self explaining.
You may run the script changing always $f with $f + 1 starting (from 0).
By uncommenting the different configurations of the variable $input you 
can see how the misbehaviour depends somehow on the data because only 
$d8 is causing the crash.
I stared at the diffs from $d8 to the other but there really isn't any 
significant difference so now it's very hard for me to understand.


Thank you in advance for all support that you can provide as usual.
Regards,
Marco.

On 26/07/19 18:41, Christian Grün wrote:

Hi Marco,

Back in February, a user reported a database inconsistency [1] – which
happened, too, if a database was nearly empty – and we managed to
build a little text case to get this reproduced [2]. After that, 9.2
was released. Maybe this fix introduced another irregularity for this
special case?

Maybe we can get this reproduced by taking the script from issue 1662
and modify it a little? That would be great.

Sorry to the inconvenience,
Christian

PS: You can safely ignore the "Creating database..." output. It may
just indicate that an XML document is parsed, and that an internal
main-memory database instance is created, possibly for your users.xml
file in the data directory.

[1] 
https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg11459.html
[2] https://github.com/BaseXdb/basex/issues/1662



On Fri, Jul 26, 2019 at 4:29 PM Marco Lettere  wrote:

Hi all,

starting with 9.2.1 we experience a strange error with a RESTXQ API  of
ours that we have been using for years.

The typical pattern is lookup a document update it and store it back.

We have done this milions of time and also all the tests work neatly.

But when using it from inside the web application, at around the tenth
operation we get the DB corrupted with the stack trace [1]. This seems
to happen only when the database is nearly empty and, for sure, it
doesn't appear on 8.x series.

It feels like some concurrency flaw is introduced somewhere but it's
hard to say because the API is eight years in age and it never changed
significantly.

I know it's hard without an SSCE... but while working to insulate this
byzantine behaviour (I've been trying for days now) maybe someone in the
list has a clue or is able to interpret the stack trace?

Thanks and have a nice weekend,

Marco.


[1]

Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 9.3 beta
Java: Ubuntu, 13
OS: Linux, amd64
Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
  at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507)
  at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468)
  at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156)
  at org.basex.data.Data.kind(Data.java:304)
  at org.basex.query.value.node.DBNode.(DBNode.java:50)
  at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139)
  at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164)
  at org.basex.query.func.db.DbOpen.value(DbOpen.java:19)
  at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82)
  at org.basex.query.expr.Arr.inline(Arr.java:69)
  at org.basex.query.expr.path.Path.inline(Path.java:962)
  at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66)
  at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804)
  at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339)
  at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106)
  at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785)
  at org.basex.query.expr.Try.inline(Try.java:106)
  at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816)
  at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339)
  at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106)
  at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294)
  at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67)
  at org.basex.query.scope.MainModule.comp(MainModule.java:81)
  at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114)
  at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105)
  at org.basex.query.QueryContext.compile(QueryContext.java:312)
  at org.basex.query.QueryContext.iter(QueryContext.java:331)
  at
org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73)
  at org.basex.http.web.WebResponse.create(WebResponse.java:63)
  at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53)
  at org.basex.http.BaseXServlet.service(BaseXServl

Re: [basex-talk] Database corrupted on updating with RestXQ

[Marco Lettere]

As an additional remark I set the MIXUPDATES = true option.
Moreover I notice the wording "Creating database..." among the request 
logs that are output when running with -d [1].
This sounds strange (the only point I see the string in the code is the 
Builder class [2]) but it happens also with 8.6.7 which doesn't show 
this misbehaviour.


M.

[1]
_ REQUEST _
(GET /hlcm/xdwapi/v2/xdw/2.25.8046764881955849894)@912634350 
org.eclipse.jetty.server.Request@3665b1ee

- Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
- Connection: keep-alive
- User-Agent: Java/13
- Host: 10.17.217.1:8984
_ RESPONSE 
HTTP/1.1 200
Set-Cookie: JSESSIONID=a2ryzgdm4t941ra9jmes9mj87;Path=/
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Type: application/xml; charset=UTF-8
*
Creating Database...*
. 5.58 ms (100 MB)
_ REQUEST _
(GET 
/hlcm/taskviewer/showworkflow?workflowid=2.25.8046764881955849894)@1286045891 
org.eclipse.jetty.server.Request@4ca780c3

- Accept: */*
- Connection: keep-alive
- Referer: 
http://localhost:8984/ui/hlcm/main.cameo?config=HLCM_1=test1=rossi

- Accept-Encoding: gzip, deflate, br
- User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, 
like Gecko) Ubuntu Chromium/75.0.3770.90 Chrome/75.0.3770.90 Safari/537.36

- Accept-Language: it,en-US;q=0.9,en;q=0.8
- Cookie: COOKIE_SUPPORT=true; GUEST_LANGUAGE_ID=en_US; pnctest=1; 
JSESSIONID=2g5ep7mzq0ah1ktre4b398hfc

- Host: localhost:8984
_ RESPONSE 
HTTP/1.1 200

[2] 
https://github.com/BaseXdb/basex/blob/f7a5492c46d55e1c1f58df24b8ed9567c176e8c1/basex-core/src/main/java/org/basex/build/Builder.java


On 26/07/19 16:29, Marco Lettere wrote:

Hi all,

starting with 9.2.1 we experience a strange error with a RESTXQ API  
of ours that we have been using for years.


The typical pattern is lookup a document update it and store it back.

We have done this milions of time and also all the tests work neatly.

But when using it from inside the web application, at around the tenth 
operation we get the DB corrupted with the stack trace [1]. This seems 
to happen only when the database is nearly empty and, for sure, it 
doesn't appear on 8.x series.


It feels like some concurrency flaw is introduced somewhere but it's 
hard to say because the API is eight years in age and it never changed 
significantly.


I know it's hard without an SSCE... but while working to insulate this 
byzantine behaviour (I've been trying for days now) maybe someone in 
the list has a clue or is able to interpret the stack trace?


Thanks and have a nice weekend,

Marco.


[1]

Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 9.3 beta
Java: Ubuntu, 13
OS: Linux, amd64
Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for 
length 1

    at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507)
    at 
org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468)
    at 
org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156)

    at org.basex.data.Data.kind(Data.java:304)
    at org.basex.query.value.node.DBNode.(DBNode.java:50)
    at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139)
    at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164)
    at org.basex.query.func.db.DbOpen.value(DbOpen.java:19)
    at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82)
    at org.basex.query.expr.Arr.inline(Arr.java:69)
    at org.basex.query.expr.path.Path.inline(Path.java:962)
    at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66)
    at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804)
    at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339)
    at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106)
    at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785)
    at org.basex.query.expr.Try.inline(Try.java:106)
    at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816)
    at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339)
    at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106)
    at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294)
    at 
org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67)

    at org.basex.query.scope.MainModule.comp(MainModule.java:81)
    at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114)
    at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105)
    at org.basex.query.QueryContext.compile(QueryContext.java:312)
    at org.basex.query.QueryContext.iter(QueryContext.java:331)
    at 
org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73)

    at org.basex.http.web.WebResponse.create(WebResponse.java:63)
    at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53)
    at org.basex.http.BaseXServlet.service(BaseXServl

[basex-talk] Database corrupted on updating with RestXQ

[Marco Lettere]

Hi all,

starting with 9.2.1 we experience a strange error with a RESTXQ API  of 
ours that we have been using for years.


The typical pattern is lookup a document update it and store it back.

We have done this milions of time and also all the tests work neatly.

But when using it from inside the web application, at around the tenth 
operation we get the DB corrupted with the stack trace [1]. This seems 
to happen only when the database is nearly empty and, for sure, it 
doesn't appear on 8.x series.


It feels like some concurrency flaw is introduced somewhere but it's 
hard to say because the API is eight years in age and it never changed 
significantly.


I know it's hard without an SSCE... but while working to insulate this 
byzantine behaviour (I've been trying for days now) maybe someone in the 
list has a clue or is able to interpret the stack trace?


Thanks and have a nice weekend,

Marco.


[1]

Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 9.3 beta
Java: Ubuntu, 13
OS: Linux, amd64
Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
    at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507)
    at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468)
    at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156)
    at org.basex.data.Data.kind(Data.java:304)
    at org.basex.query.value.node.DBNode.(DBNode.java:50)
    at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139)
    at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164)
    at org.basex.query.func.db.DbOpen.value(DbOpen.java:19)
    at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82)
    at org.basex.query.expr.Arr.inline(Arr.java:69)
    at org.basex.query.expr.path.Path.inline(Path.java:962)
    at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66)
    at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804)
    at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339)
    at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106)
    at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785)
    at org.basex.query.expr.Try.inline(Try.java:106)
    at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816)
    at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339)
    at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106)
    at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294)
    at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67)
    at org.basex.query.scope.MainModule.comp(MainModule.java:81)
    at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114)
    at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105)
    at org.basex.query.QueryContext.compile(QueryContext.java:312)
    at org.basex.query.QueryContext.iter(QueryContext.java:331)
    at 
org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73)

    at org.basex.http.web.WebResponse.create(WebResponse.java:63)
    at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53)
    at org.basex.http.BaseXServlet.service(BaseXServlet.java:65)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
    at 
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
    at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542)
    at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
    at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
    at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
    at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700)
    at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
    at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
    at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
    at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
    at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667)
    at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
    at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
    at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
    at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)

    at org.eclipse.jetty.server.Server.handle(Server.java:505)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
    at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
    at 

Re: [basex-talk] Quickest way to parse a DomSource

[Marco Lettere]

Thank you Martin.
It's worth experimenting

In the meanwhile (thanks to my colleague who had a closer look at the 
DOMSource API) I got to understand that there's a getNode() method 
returning the underlying DOM Node.

This allows for minimizing the module import statements.

Thus, fyi,  at the moment this is my work-around:

(: Import DOMSource API:)
declare namespace source = "javax.xml.transform.dom.DOMSource";

(:... code to get to the custom Document representation stored in a 
variable called $document... :)


(: get the DOMSource through custom API:)
let $source := bpmndom:getDomSource($document)

(:DOMSource API to get underlying Document Node :)
let $doc := source:getNode($source)

(:Now this Xpath expression works :)
return $doc//*:userTask

Cheers,
Marco.


On 22/07/19 12:53, Martin Honnen wrote:

Am 22.07.2019 um 11:32 schrieb Marco Lettere:



I have to access portions of the XML representation of the BPMN file
by binding the engine's API.

The closest that I can get to a standard XML representation is
currently a javax.xml.transform.dom.DOMSource.

What is the shortest way to transform (in XQuery) this DomSource to a
BaseX representation of the Document in order to query it?



The XQJ API seems to have a method

http://xqj.net/javadoc/javax/xml/xquery/XQDynamicContext.html#bindDocument(javax.xml.namespace.QName,%20javax.xml.transform.Source,%20javax.xml.xquery.XQItemType) 



 to allow passing in a java.xml.transform.Source so that seems at least
one way to deal with a DOMSource. But it happens on the Java side and is
tied to that particular XQJ API.

[basex-talk] Quickest way to parse a DomSource

[Marco Lettere]

Hi all,

I'm currently working on the usage of BaseX Xquery scripts inside a BPMN 
engine.


I have to access portions of the XML representation of the BPMN file by 
binding the engine's API.


The closest that I can get to a standard XML representation is currently 
a javax.xml.transform.dom.DOMSource.


What is the shortest way to transform (in XQuery) this DomSource to a 
BaseX representation of the Document in order to query it?


Thanks,

Marco.

Re: [basex-talk] PATCH verb in RestXQ implementation

[Marco Lettere]

Hi Daniel,
I'm pretty sure that PATCH is supported in recent versions of BaseX. 
Even if I don't remember exactly when it started since it's not traced 
in the Changelog of the documentation page for RestXQ.


From the docs at [1]:

The HTTP method annotations are equivalent to allHTTP request methods 
except TRACE and 
CONNECT. Zero or more methods may be used on a function; if none is 
specified, the function will be invoked for each method.


[1] http://docs.basex.org/wiki/RESTXQ

Ciao,
Marco.

On 16/07/19 19:22, Schopper, Daniel wrote:

Dear list,
I just came across the fact that BaseX does not recognize PATCH as a 
known HTTP verb (at least in my admittedly not bleading-edge version). 
Is this a feature or a bug?
And, in order to work around that: is there a way of passing payload 
to a custom %rest:method (without having an additional PUT or POST 
annotation on the same XQuery function)?

Thanks in advance!
Best
Daniel

[basex-talk] [ANN.] JSR223 wrapper

[Marco Lettere]

Ok then ... since I haven't got any feedback on this and because 
crawling the web myself hasn't produced any results either, I decided to 
invest a couple of weekends re-studying the the JSR223 (Java Scripting 
Framework) and implementing a wrapper for BaseX by myself.


For those who are not aware, JSR223 is a specification standardizing the 
embedding scripting languages into Java based applications. Was BSF once 
upon a time
As a use case at Nubisware, in particular, we wanted to continue using 
BaseX powered XQuery 3.1 scripts inside BPMN workflow engines for 
ScriptTasks and Event Handlers.


One plan that we have for the future is to create an equivalent engine 
that is able to execute scripts through REST on a remote BaseX server 
... but still reasoning whether this is backed by any real use case.


You can access the code and readme at [1] or download the current v0.1 
jar from [2].


Of course every 2 cents of feedback will be warmly appreciated.

Enjoy,

Marco.

[1] https://github.com/nubisware/basexjsr223
[2] 
https://github.com/nubisware/basexjsr223/releases/download/v0.1/BaseXJSR223.jar


On 11/06/19 13:00, Marco Lettere wrote:

Hi all,

is anyone on the list aware of a JSR223 compliant wrapper for BaseX in 
order to include XQuery scripting "seamlessly" into scriptable 
applications?


Thanks.

Marco.

[basex-talk] JS223 wrapper

[Marco Lettere]

Hi all,

is anyone on the list aware of a JS223 compliant wrapper for BaseX in 
order to include XQuery scripting "seamlessly" into scriptable applications?


Thanks.

Marco.

Re: [basex-talk] Update of arbitrarily selected elements?

[Marco Lettere]

Yes. In addition, if you don't have IDs in place and elements are all in 
the same container, you could maybe use their position. Or a combination 
of an XPAT selector for the contianer root and then a sequence of 
positions for the elements to be updated.

M.

On 03/06/19 09:46, Michael Seiferle wrote:

Hi Josh,

I think the most widely used approach would be addressing items via ID 
— just as you would in a traditional database approach — this might 
involve adding IDs to all elements.



(as in, not selectable with a query)
With XPath you can select arbitrary parts of your document, maybe 
using XPath expressions would be helpful?



Hope this helps a bit, feel free to ask for more specific information  :)

Best

Michael







Am 31.05.2019 um 20:14 schrieb Joshua Kordani 
>:


Greetings all.
I am curious about how basex supports the idea of modifying any given 
(or set of given) elements, say a user makes a query to select a 
bunch of elements for editing and wishes to add an attribute to an 
arbitrary subset (as in, not selectable with a query), how do I 
reference a particular element for later updating?

Sincerely,
Josh

Re: [basex-talk] Optimizer and JSON/XML response

[Marco Lettere]

Hi Christian,
I'll try to answer your questions inline ...

On 16/05/19 19:38, Christian Grün wrote:

Hi Marco,

We haven’t experienced such a behavior by ourselves, even in very
complex applications. Is the faulty behavior reproducible?

Only with the exact configuration of our current applciation...

Does it
make a difference if you change the value of the PARSERESTXQ option
[1]?
No, it doesn't. we tried higher values (wrt the default 3) and also 0 
and -1 ...

And is it already the http:send-request call that returns the
wrong result, or is it the actual RESTXQ function in which the
http:send-request call is contained?
The http:send-request returns the second element of the response in map 
format. We discovered it actually because a following XPath expression 
raised an error because it hit a map instead of a document as expected...

You could additionally output the
result of the function call to standard error (via fn:trace) or to a
file (via file:write) and compare the output in both of your setups.
We did all sort of tests and it's the http:send-request without doubts 
that already returns a map...


If there’s any chance to reproduce this (e.g. by creating hundreds or
thousands of dummy RESTXQ functions), that’d be fine.


I've tried to generate #FILES * #FUNCT  number of RESTXQS with 
increasing number of files and functions (up to OOM exceptions with 
default -Xms=2G) but I'm unable to reproduce... [A][B]


So I suspect that the problem might be in he file structure  we use 
softlinks to hook up restxq folders, a REPO full of implementation 
modules so may be the resolution of all this relations might be the issue?


Clueless at the moment ...

M.

[A] python script to generate extra RESTXQS

#!/bin/python3
FILES = 100
RESTXQS = 1000

qtempl = open("template.txt").read()

for f in range(FILES):
    outf = open("file" + str(f) + ".xqm", "w")
    qout = 'module namespace _ = "urn:dummy:' + str(f) + '";\n\n';
    for q in range(RESTXQS):
    qout += "\n" + qtempl.format(path=(str(f)+"_"+str(q)))
    outf.write(qout)

[B] template query stored in template.txt

declare
  %rest:path("test/{path}")
  %rest:GET
function _:f{path}() {{
   "{path}"
}};



Cheers,
Christian

[1] http://docs.basex.org/wiki/Options#PARSERESTXQ




On Thu, May 16, 2019 at 1:27 PM Marco Lettere  wrote:

Hello everyone,

we came accross an inconsistency which we are not able to find an
explanation for and giving up after having spent on it the whole morning...

  From a restxq we call an external REST service (InfluxDB) like shown in
[1]. This usually works as expected returning a response forged as in [2].

When the same RESTXQ is deployed in a much more complex application
(with several RESTXQ modules and REPO modules) the first call (which
takes a lot of time because it incurs the initial compiling delay)
returns [3] instead of [2]. From the second call onwards the result
stabilizes correctly to [2].

If I call any other RESTXQ function as a sort of warm-up (spending there
all the initial delay) [1] behaves correctly returning always [2].

Do you please have a clue on why this happens?

We noticed it for the first time because we started to use service jobs
which cause this access pattern ...

Thanks for any help.

[1]

module namespace test = 'test';

declare
%rest:path("test")
%rest:GET
function test:test() {
let $url :=
"http://localhost:8086/query?pretty=falsechunked=falsedb=nestwayq=SELECT+*+FROM+%22frontoffice%22+WHERE+time+%3E%3D+%272019-05-01T00%3A00%3A00Z%27+AND+time+%3C%3D+%272019-05-31T23%3A59%3A50Z%27;
let $results := http:send-request(, $url)
return $results
};

[2]

http://expath.org/ns/http-client;
status="200" message="OK">
  
  
  
  
  
  
  


  
  <_ type="object">
  
  <_ type="object">
  frontoffice
  
  <_>time
  <_>consumable
  <_>customer
  <_>guests
  <_>type
  <_>user
  
  
  <_ type="array">
<_>2019-05-10T21:55:00Z
  <_>access
<_>c9a83ce3-c938-4677-a0df-72bc96354509
  <_ type="number">0
  <_>starttimer
<_>d50de945-23ed-4b33-84e8-8ef533a96fb7
  
  
  
  
  
  



[3]

http://expath.org/ns/http-client;
status="200" message="OK">
  
  
  
  
  
  
  

map {
"results": [map {
  "series": [map {
"values": [["2019-05-10T21:55:00Z", "access",
"c9a83ce3-c938-4677-a0df-72bc96354509", 0.0e0, "starttimer",
"d50de945-23ed-4b33-84e8-8ef533a96fb7"]],
"name": "frontoffice",
"columns": ["time", "consumable", "customer", "guests", "type",
"user"]
  }]
}]
}

Re: [basex-talk] Optimizer and JSON/XML response

[Marco Lettere]

Just to add up ...

BaseX version is 9.0.2.

Moreover when running from a schedule service job the only way to avoid 
the issue described is to always first call another RESTXQ and then [1].
Which is somehow what I expect if services are run in their own 
process/jvm...
Despite the subject of the mail it looks like an issue in the RESTXQ 
parser/compiler rather than an optimizer issue...

M.


On 16/05/19 13:27, Marco Lettere wrote:

Hello everyone,

we came accross an inconsistency which we are not able to find an 
explanation for and giving up after having spent on it the whole 
morning...


From a restxq we call an external REST service (InfluxDB) like shown 
in [1]. This usually works as expected returning a response forged as 
in [2].


When the same RESTXQ is deployed in a much more complex application 
(with several RESTXQ modules and REPO modules) the first call (which 
takes a lot of time because it incurs the initial compiling delay) 
returns [3] instead of [2]. From the second call onwards the result 
stabilizes correctly to [2].


If I call any other RESTXQ function as a sort of warm-up (spending 
there all the initial delay) [1] behaves correctly returning always [2].


Do you please have a clue on why this happens?

We noticed it for the first time because we started to use service 
jobs which cause this access pattern ...


Thanks for any help.

[1]

module namespace test = 'test';

declare
  %rest:path("test")
  %rest:GET
function test:test() {
  let $url := 
"http://localhost:8086/query?pretty=falsechunked=falsedb=nestwayq=SELECT+*+FROM+%22frontoffice%22+WHERE+time+%3E%3D+%272019-05-01T00%3A00%3A00Z%27+AND+time+%3C%3D+%272019-05-31T23%3A59%3A50Z%27;

  let $results := http:send-request(, $url)
  return $results
};

[2]

http://expath.org/ns/http-client; 
status="200" message="OK">

    
    
    
    
    value="95b729c7-77ca-11e9-8065-"/>

    
    


    
    <_ type="object">
    
    <_ type="object">
    frontoffice
    
    <_>time
    <_>consumable
    <_>customer
    <_>guests
    <_>type
    <_>user
    
    
    <_ type="array">
<_>2019-05-10T21:55:00Z
    <_>access
<_>c9a83ce3-c938-4677-a0df-72bc96354509
    <_ type="number">0
    <_>starttimer
<_>d50de945-23ed-4b33-84e8-8ef533a96fb7
    
    
    
    
    
    



[3]

http://expath.org/ns/http-client; 
status="200" message="OK">

    
    
    
    
    value="7957030a-77cc-11e9-8066-"/>

    
    

map {
  "results": [map {
    "series": [map {
  "values": [["2019-05-10T21:55:00Z", "access", 
"c9a83ce3-c938-4677-a0df-72bc96354509", 0.0e0, "starttimer", 
"d50de945-23ed-4b33-84e8-8ef533a96fb7"]],

  "name": "frontoffice",
  "columns": ["time", "consumable", "customer", "guests", "type", 
"user"]

    }]
  }]
}

[basex-talk] Optimizer and JSON/XML response

[Marco Lettere]

Hello everyone,

we came accross an inconsistency which we are not able to find an 
explanation for and giving up after having spent on it the whole morning...


From a restxq we call an external REST service (InfluxDB) like shown in 
[1]. This usually works as expected returning a response forged as in [2].


When the same RESTXQ is deployed in a much more complex application 
(with several RESTXQ modules and REPO modules) the first call (which 
takes a lot of time because it incurs the initial compiling delay) 
returns [3] instead of [2]. From the second call onwards the result 
stabilizes correctly to [2].


If I call any other RESTXQ function as a sort of warm-up (spending there 
all the initial delay) [1] behaves correctly returning always [2].


Do you please have a clue on why this happens?

We noticed it for the first time because we started to use service jobs 
which cause this access pattern ...


Thanks for any help.

[1]

module namespace test = 'test';

declare
  %rest:path("test")
  %rest:GET
function test:test() {
  let $url := 
"http://localhost:8086/query?pretty=falsechunked=falsedb=nestwayq=SELECT+*+FROM+%22frontoffice%22+WHERE+time+%3E%3D+%272019-05-01T00%3A00%3A00Z%27+AND+time+%3C%3D+%272019-05-31T23%3A59%3A50Z%27;

  let $results := http:send-request(, $url)
  return $results
};

[2]

http://expath.org/ns/http-client; 
status="200" message="OK">

    
    
    
    
    value="95b729c7-77ca-11e9-8065-"/>

    
    


    
    <_ type="object">
    
    <_ type="object">
    frontoffice
    
    <_>time
    <_>consumable
    <_>customer
    <_>guests
    <_>type
    <_>user
    
    
    <_ type="array">
<_>2019-05-10T21:55:00Z
    <_>access
<_>c9a83ce3-c938-4677-a0df-72bc96354509
    <_ type="number">0
    <_>starttimer
<_>d50de945-23ed-4b33-84e8-8ef533a96fb7
    
    
    
    
    
    



[3]

http://expath.org/ns/http-client; 
status="200" message="OK">

    
    
    
    
    value="7957030a-77cc-11e9-8066-"/>

    
    

map {
  "results": [map {
    "series": [map {
  "values": [["2019-05-10T21:55:00Z", "access", 
"c9a83ce3-c938-4677-a0df-72bc96354509", 0.0e0, "starttimer", 
"d50de945-23ed-4b33-84e8-8ef533a96fb7"]],

  "name": "frontoffice",
  "columns": ["time", "consumable", "customer", "guests", "type", 
"user"]

    }]
  }]
}

Re: [basex-talk] BaseX 9.2: The Spring Edition

[Marco Lettere]

Yes, great question. We were asking this ourselves too.
M.

On 17/04/19 11:33, Andy Bunce wrote:


I am not clear what is meant by:
> - no need to import basex-api modules anymore!
Does this just mean all modules listed at 
http://docs.basex.org/wiki/Module_Library are statically bound?


/Andy

Re: [basex-talk] BaseX 9.2: The Spring Edition

[Marco Lettere]

Thanks! Great work as usual!
M.

On 16/04/19 16:10, Christian Grün wrote:

Dear all,

We are more than pleased to announce version 9.2 of BaseX, our XML
framework, database system and XQuery 3.1 processor:

   http://basex.org/

This time, we have put a special focus on performance improvements:

XQUERY PERFORMANCE
- comparisons will more often be rewritten to hash joins
- optimized access to subsequences and single sequence items
- consistent propagation of static types to evaluation results
- improved static typing of function items (maps, arrays)
- more aggressive pre-evaluation at compile time

STORAGE PERFORMANCE
- large databases will be opened faster
- regular storage offset will be computed at runtime

GENERAL XQUERY FEATURES
- no need to import basex-api modules anymore!
- Archive Module: archive:create-from, new options
- Database Module: directory-based access via db:dir
- Profiling Module: new functions (prof:gc, prof:runtime)
- Validation Module: assign XSD validation features
- WebSocket Module: query scheduling via ws:eval
- Utility Module: various new helper functions
- XSLT Module: support for XML Catalogs (thank you Liam)

NEW OPTIONS
- RESTXQERRORS: Suppress parsing errors at runtime
- FULLPLAN: comprehensive query plan information

FULLTEXT FEATURES
- stemmer for Arabic texts added

USER INTERFACES
- GUI: better support for latest JDK versions (incl. OpenJDK)
- DBA: revised search features in log panel

For a more comprehensive list of added and updated features. please
check out our documentation: http://docs.basex.org/.

We are looking forward to your feedback.

Have fun with the new release,
Your BaseX Team

Re: [basex-talk] concurrent performance issues?

[Marco Lettere]

Hi Michael,
there has been a lot of discussion related to this issue. The most 
significant result that has alway clearly emerged and that Christian is 
always evangelizing us about is that beeing DB -access strictly serial, 
querying in paralllel is causing only trouble unless the data is 
actually split onto different disks. accessing in parallel just causes 
the OS/HW TO jump around on the disk missing al the cache efficiency and 
optimization.
So, unless you have hardware that really supports parallel access, just 
prefer sequential disk access.


Hoffe dies hilft.

M.


On 08/04/19 16:05, Michael Perkonigg wrote:


Hallo,

ich versuche gerade, BaseX hier für eine zentrale XML-DB zu evaluieren.

Die DB ist read-only. Wir haben also keinerlei Updates.

Wir haben verschiedene XQuery-Queries(?), die abgesetzt werden, dabei 
ist aufgefallen, dass wenn ich mehrere Queries parallel starte die 
Antwortzeiten unverhätnismässig steigen.


Einzeln:

Query1: 500ms

Query2: 1200ms

Parallel brauchen beide zusammen dann zwischen 3 und 5 Sekunden.

Erstaunlicherweise sind die Antwortzeiten mit der Option PARALLEL=1 am 
geringsten, d.h. bei der Defaulteinstellung PARALLEL=8 (und auch schon 
bei PARALLEL=2) steigen die Antwortzeiten.


Ich frage mich nun, ob BaseX als Datenbank für eine verteilte 
Applikation mit zentraler DB überhaupt geeignet ist und ob ich 
vielleicht mit irgendwelchen Optionen BaseX für meine Zwecke 
optimieren kann.


Ganz theoretisch könnte die DB ja mit hundert parallelen Anfragen 
belastet werden.



--
mit freundlichen Grüßen

Re: [basex-talk] Hitting the Tail Recursion wall

[Marco Lettere]

If you need to join the resulting strings just use  well ... 
string-join ...


declare function local:topath($path){
  let $pathseg := tokenize($path, "/")
  let $pathsequence :=
    fold-left($pathseg, (), function($out, $segment){
  if($segment = "." or $segment = "") then $out
  else if($segment = "..") then $out[position() lt count($out)]
  else ($out, $segment)
    })
  return string-join($pathsequence, "/")
};

local:topath("/a/b/c/../../../g")

On 01/04/19 22:02, Andreas Mixich wrote:

Marco Lettere wrote onm 01.04.2019 at 18:01:


declare function local:topath($path){  let $pathseg := tokenize($path,
"/")  return    fold-left($pathseg, (), function($out, $segment){
if($segment = "." or $segment = "") then $out  else if($segment =
"..") then $out[position() lt count($out)]  else ($out,
$segment)    })}; local:topath("/a/b/c/../../../g")

Beautiful and very close, except for a minor caveat: the "/" are needed,
to reconstruct the absolute path-part from the relative.
I played around and tried to place some "/" to your function, but all
variants placed some "/" wrong or twice.

Last but not least: You asked for the use case, which is described on
https://tools.ietf.org/html/rfc3986#section-5.2.4 (note also the
sequences shown after the description of the steps)


 5.2.4 <https://tools.ietf.org/html/rfc3986#section-5.2.4>.
 Remove Dot Segments



The pseudocode also refers to a "remove_dot_segments" routine for
interpreting and removing the special "." and ".." complete path
segments from a referenced path.  This is done after the path is
extracted from a reference, whether or not the path was relative, in
order to remove any invalid or extraneous dot-segments prior to
forming the target URI.  Although there are many ways to accomplish
this removal process, we describe a simple method using two string
buffers.

1.  The input buffer is initialized with the now-appended path
components and the output buffer is initialized to the empty
string.

2.  While the input buffer is not empty, loop as follows:

A.  If the input buffer begins with a prefix of "../" or "./",
then remove that prefix from the input buffer; otherwise,

B.  if the input buffer begins with a prefix of "/./" or "/.",
where "." is a complete path segment, then replace that
prefix with "/" in the input buffer; otherwise,

C.  if the input buffer begins with a prefix of "/../" or "/..",
where ".." is a complete path segment, then replace that
prefix with "/" in the input buffer and remove the last
segment and its preceding "/" (if any) from the output
buffer; otherwise,

D.  if the input buffer consists only of "." or "..", then remove
that from the input buffer; otherwise,

E.  move the first path segment in the input buffer to the end of
the output buffer, including the initial "/" character (if
any) and any subsequent characters up to, but not including,
the next "/" character or the end of the input buffer.

3.  Finally, the output buffer is returned as the result of
remove_dot_segments.

Note that dot-segments are intended for use in URI references to
express an identifier relative to the hierarchy of names in the base
URI.  The remove_dot_segments algorithm respects that hierarchy by
removing extra dot-segments rather than treat them as an error or
leaving them to be misinterpreted by dereference implementations.

The following illustrates how the above steps are applied for two
examples of merged paths, showing the state of the two buffers after
each step.

   STEP   OUTPUT BUFFER INPUT BUFFER

1 : /a/b/c/./../../g
2E:   /a/b/c/./../../g
2E:   /a/b  /c/./../../g
2E:   /a/b/c/./../../g
2B:   /a/b/c/../../g
2C:   /a/b  /../g
2C:   /a/g
2E:   /a/g

   STEP   OUTPUT BUFFER INPUT BUFFER

1 <https://tools.ietf.org/html/rfc3986#section-1> : 
mid/content=5/../6
2E:   mid   /content=5/../6
2E:   mid/content=5 /../6
2C:   mid   /6
2E:   mid/6

Re: [basex-talk] Hitting the Tail Recursion wall

[Marco Lettere]

Hi Andreas,
I don't know whether I correctly understood you use-case but what about 
going with hof functions [1]?

Maybe your code could turn to something as simple as

declare function local:topath($path){
  let $pathseg := tokenize($path, "/")
  return
    fold-left($pathseg, (), function($out, $segment){
  if($segment = "." or $segment = "") then $out
  else if($segment = "..") then $out[position() lt count($out)]
  else ($out, $segment)
    })
};
local:topath("/a/b/c/../../../g")

Regards,
Marco.

[1] http://docs.basex.org/wiki/Higher-Order_Functions

On 01/04/19 15:49, Andreas Mixich wrote:

I think, I am finding it...

I replaced the first expression with

`prof:dump(concat("in: ", $path, " out: ",
string-join(array:flatten($out,`

and this gives me more info. So I may be able to solve this alone.

Re: [basex-talk] Quick Windows issue: Invalid reply on port 1984, works fine on port 8984

[Marco Lettere]

Hi Jim,

I think port 1984 is not meant to be connected to over HTTP. It's a pure 
socket connection that ships BaseX internal client-server protocol.
When you connect to that port you probably have to use the Python client 
API in order to get your BaseX COMMANDS (not HTTP requests) serialized 
over the wire.


Hope this helps [cit.].

Regards,
M.

On 28/03/19 23:28, Jim Salmons wrote:

Hello BaseX folk,

I have just installed version 9.1.2 and am starting to learn BaseX in
support of my post-cancer #PayItForward Bonus Round #CitizenScientist
research in #DigitalHumanities and #MachineLearning. My goal is to "cut out
the middleman" of writing intermediate JSON and XML fragment files in order
to update the #MAGAZINEgts ground-truth storage file that I am developing
for a digital collection of Softalk Apple magazine at the Internet Archive.
With BaseX and its Python interface, I want to write new and updated
document structure and content depiction metadata directly to a local copy
of this ground-truth master file which will then be easily popped out of the
database as an XML file for uploading to the Softalk Apple collection at the
Internet Archive.

I hit a bump, reported to the Github Issue queue, where my Windows 10
installation can not open the BaseX GUI exe launcher. The work-around, as
noted in my issue report, is to use the Getting Started advice to execute
the jar directly. By creating a desktop icon that launches the execuable jar
in the installation folder, the GUI launches as expected.

With that minor issue behind me, I am wondering why my attempts to connect
to the server via localhost:1984 port does not work while everything is fine
and dandy when hitting port 8984. Port 1984 returns an invalid HTTP
response. (Note: Other ports that are not assigned to anything else return a
connection refused, so this invalid response is not the same as the server
not being there.)

If I keep a Powershell window open with the server running, every once in a
while (after several minutes or more) the server throws this error:

java.lang.NullPointerException
 at org.basex.core.Context.perm(Context.java:252)
 at org.basex.core.Command.run(Command.java:250)
 at org.basex.core.Command.run(Command.java:143)
 at org.basex.core.cmd.Close.close(Close.java:54)
 at org.basex.server.ClientListener.close(ClientListener.java:232)
 at org.basex.BaseXServer$1.run(BaseXServer.java:157)
 at java.base/java.util.TimerThread.mainLoop(Timer.java:556)
 at java.base/java.util.TimerThread.run(Timer.java:506)

which may be related to port 1984 not responding, a firewall issue perhaps?
Honestly, I don't see anything that looks like a BaseX process or service
running in Task Manager (other than a generic 'Java(TM) Platform SE binary'
process, and there is nothing that looks related in the Windows Firewall
entries. I'm still investigating this here, but if anyone knows what is
akimbo, I would greatly appreciate your advice.

Thank you... and looking forward to being a new BaseX community member. :-)

Happy-Healthy Vibes,
-: Jim Salmons :-

 Jim Salmons
     Twitter: @Jim_Salmons

     https://www.researchgate.net/profile/Jim_Salmons (my #CitizenScientist
profile)
 www.medium.com/@Jim_Salmons/ (my mostly #DigitalHumanities writing)

Re: [basex-talk] Serialization issue with HTTP response

[Marco Lettere]

Great! Thanks!

On 08/03/19 13:16, Christian Grün wrote:

Hi Marco,

Sorry for letting you wait. Specializations of the json mimetype will
now be detected as well [1,2].

Cheers,
Christian

[1] 
https://github.com/BaseXdb/basex/commit/a18a092b11a5240075b081ffd301014f0dc00edf
[2] http://files.basex.org/releases/latest/


On Tue, Jan 15, 2019 at 11:35 AM Marco Lettere  wrote:

Hi all,
we have to deal with a third-party REST service which in case of error 
conditions returns this mime type: application/problem+json; charset=utf-8.

I wrote this RestXQ [1] to mock it. Just copy it into restxq.xqm...

When I call it like [2] I get [XPTY0004] Cannot convert xs:base64Binary to xs:string: 
"IntcImFcIjpcImJcIn0i". This is because the response is seen as a binary and 
this can be double checked calling with [3].
I felt (maybe wrongly) that specializations of the mimetype had been faced in 
BaseX implementation of http:request somewhere back in time but I couldn't find 
anything in the mail archives.

The strange thing however is that I get the same error when calling like [4] 
with override-mime-type set to text/json or application/json which instead I 
was expecting to work.

Is the behaviour in this scenario expected or does this hide some inconsistency?

Thanks, as usual, for the support.

Regards,

Marco.

[1]
declare%rest:path("mc/dummy")
%rest:GET
%output:method("json")
function page:dummy() {
   
 
   
 
   ,
   '{"a":"b"}'
};

[2]
json:parse(
   http:send-request(,"http://localhost:8984/mc/dummy;)[2]
)

[3]
json:parse(
   convert:binary-to-string(
 http:send-request(,"http://localhost:8984/mc/dummy;)[2]
   )
)

[4]
json:parse(
   http:send-request(,"http://localhost:8984/mc/dummy;)[2]
)

Re: [basex-talk] Serialization issue with HTTP response

[Marco Lettere]

Hi Christian,
I'd like to kindly ask whether there has been some follow-up on this 
matter...

Thanks,
M.

On 16/01/19 16:27, Christian Grün wrote:

I hope I’ll be able to test this tomorrow. And your mail is yet
another motivation to get the upcoming version 2 of the HTTP Client
finished (http://expath.github.io/expath-cg/specs/http-client-2).



On Tue, Jan 15, 2019 at 11:35 AM Marco Lettere  wrote:

Hi all,
we have to deal with a third-party REST service which in case of error 
conditions returns this mime type: application/problem+json; charset=utf-8.

I wrote this RestXQ [1] to mock it. Just copy it into restxq.xqm...

When I call it like [2] I get [XPTY0004] Cannot convert xs:base64Binary to xs:string: 
"IntcImFcIjpcImJcIn0i". This is because the response is seen as a binary and 
this can be double checked calling with [3].
I felt (maybe wrongly) that specializations of the mimetype had been faced in 
BaseX implementation of http:request somewhere back in time but I couldn't find 
anything in the mail archives.

The strange thing however is that I get the same error when calling like [4] 
with override-mime-type set to text/json or application/json which instead I 
was expecting to work.

Is the behaviour in this scenario expected or does this hide some inconsistency?

Thanks, as usual, for the support.

Regards,

Marco.

[1]
declare%rest:path("mc/dummy")
%rest:GET
%output:method("json")
function page:dummy() {
   
 
   
 
   ,
   '{"a":"b"}'
};

[2]
json:parse(
   http:send-request(,"http://localhost:8984/mc/dummy;)[2]
)

[3]
json:parse(
   convert:binary-to-string(
 http:send-request(,"http://localhost:8984/mc/dummy;)[2]
   )
)

[4]
json:parse(
   http:send-request(,"http://localhost:8984/mc/dummy;)[2]
)

Re: [basex-talk] RESTXQ and regexp

[Marco Lettere]

Thanks for your suggestion Liam!
We've tried this out on the current deployment which involves version 
8.6.3 and it didn't fix our specific use case.
My idea here (I'm not sure to make the correct assumption though) is 
that the proposition "if multiple candidate functions ar eleft over" 
stated in the docs is not holding here.
It rather looks as if BaseX is considering only the more generic one to 
match.

Regards,
Marco.

On 23/02/19 23:33, Liam R. E. Quin wrote:

On Fri, 2019-02-22 at 12:05 +0100, Marco Lettere wrote:


(: Matches anything followed by /input :)
declare %rest:path("app/{$path=.+}/input")
function page:inputs($path) { ... };

(: Matches all other all paths starting with "app/" :)
declare %rest:path("app/{$path=.+}")
function page:others($path) { ... };

Can you use content negotiation/quality to say the /input one is
preferred when both match? E.g. %rest:produces("*/*;qs=0.8") on the
page:others function and */*;qs=1.0 on the input ont?

Liam

Re: [basex-talk] BaseX-Talk Digest, Vol 110, Issue 29

[Marco Lettere]

Hi Shaun,
I agree on your suggestions related to the regexp.
The thing is we _have_ tried to achieve a more fine grained filter by 
matching in the $path related segment app/"anything but the word 
input"/input.
Unfortunately according to my knowledge this requires fiddling with 
positive lookahead expressions. We haven't been able to make them work 
in RESTXQ.
We hoped that being /app/"anything"/input/ more specific than 
/app/"anything"/, the former would ha been preferred. But it doesn't 
seem to work like that.

Regards,
Marco.


On 23/02/19 22:11, Shaun Flynn wrote:

Hello there Marco,

This might be down to the choice of regex.

Assume $path = "path"

app/{$path=.+}

Will match

"app/path=1234"

And

"app/path=1234/input"

Which I think would be matched by ".+" part of the expression, as 
"/input" would match ".+".


Kind regards

On Sat, 23 Feb 2019, 11:00 , 
<mailto:basex-talk-requ...@mailman.uni-konstanz.de>> wrote:


Send BaseX-Talk mailing list submissions to
basex-talk@mailman.uni-konstanz.de
<mailto:basex-talk@mailman.uni-konstanz.de>

To subscribe or unsubscribe via the World Wide Web, visit
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
or, via email, send a message with subject or body 'help' to
basex-talk-requ...@mailman.uni-konstanz.de
<mailto:basex-talk-requ...@mailman.uni-konstanz.de>

You can reach the person managing the list at
basex-talk-ow...@mailman.uni-konstanz.de
<mailto:basex-talk-ow...@mailman.uni-konstanz.de>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of BaseX-Talk digest..."


Today's Topics:

   1. RESTXQ and regexp (Marco Lettere)


------

Message: 1
Date: Fri, 22 Feb 2019 12:05:29 +0100
From: Marco Lettere mailto:m.lett...@gmail.com>>
To: BaseX mailto:basex-talk@mailman.uni-konstanz.de>>
Subject: [basex-talk] RESTXQ and regexp
Message-ID: <3f71e0bc-e16d-6c4d-6ee8-246a7cc75...@gmail.com
<mailto:3f71e0bc-e16d-6c4d-6ee8-246a7cc75...@gmail.com>>
Content-Type: text/plain; charset=utf-8; format=flowed

Hello all,

according to the docs this should be possible:

(: Matches all paths with "app" as first, a number as second, and
"order" as third segment :)
declare %rest:path("app/{$code=[0-9]+}/order")
 ? function page:order($full-path) { ... };

(: Matches all other all paths starting with "app/" :)
declare %rest:path("app/{$path=.+}")
 ? function page:others($path) { ... };

And it actually is... We are currently in the very unfortunate
situation
where we need the following:

(: Matches anything followed by /input :)
declare %rest:path("app/{$path=.+}/input")
 ? function page:inputs($path) { ... };

(: Matches all other all paths starting with "app/" :)
declare %rest:path("app/{$path=.+}")
 ? function page:others($path) { ... };

Unfortunately this makes all the calls to be redirected to the
page:others function ... is there a motivation behind it or could
it be
possible to work around that?

Thanks,

Marco.



End of BaseX-Talk Digest, Vol 110, Issue 29
***

[basex-talk] RESTXQ and regexp

[Marco Lettere]

Hello all,

according to the docs this should be possible:

(: Matches all paths with "app" as first, a number as second, and 
"order" as third segment :)

declare %rest:path("app/{$code=[0-9]+}/order")
  function page:order($full-path) { ... };

(: Matches all other all paths starting with "app/" :)
declare %rest:path("app/{$path=.+}")
  function page:others($path) { ... };

And it actually is... We are currently in the very unfortunate situation 
where we need the following:


(: Matches anything followed by /input :)
declare %rest:path("app/{$path=.+}/input")
  function page:inputs($path) { ... };

(: Matches all other all paths starting with "app/" :)
declare %rest:path("app/{$path=.+}")
  function page:others($path) { ... };

Unfortunately this makes all the calls to be redirected to the 
page:others function ... is there a motivation behind it or could it be 
possible to work around that?


Thanks,

Marco.

Re: [basex-talk] Global lock = false and parallel update processes to different DBs

[Marco Lettere]

Hi France,
I recall once I've been successful in generating xquery strings by patching
the database name into it and then processing it with xquery:eval.
Might this be anche approach for you?
M.

Il giorno mer 6 feb 2019, 09:12 France Baril 
ha scritto:

> Irsh, we have 52 languages and all our system is based on being able to
> work with any language and let clients add/remove languages without having
> to call developers. I can't imagine the domino effect of having to build a
> shell function per language per process that access the DB.
>
> Plus as we are running batch processes, I think we'll just run out of
> memory.
>
> I'm thinking one function like this per language is what you propose :
>
> rest-path /base/filter-es-us()
> function filter-es-us {
> let $src-db = db:open(es-us)
> let $results := apply-non-updating-processes($src-db)...  where result is
> a map of (filename, xml)
> return
>   for $result in $results
>   return  db:replace('staging-es-us', $results)
> };
>
> apply-non-updating-processes($src-db){
> map:merge(
>  for $file in $src-db/*
>  res= do x
>  return map:entry ($file/base-uri, res)
> };
>
>
> Since we run batch processes I'm also thinking we'll run out of memory
> with processes like that... or maybe we need to split also small functions
> so each tiny update is in its own function... then maintaiing functions for
> 52 languages becomes even harder... or I add an extra layer of abstraction
> and build the .xqm functions dynamically based on a central code base and
> the dynamic language names... hmmm
>
> I'm thinking out loud here trying to find my way outside of dynamic
> names... but static naming of databases doesn't sound like a good idea in
> our case. Dynamic naming is at the core of our approach... or maybe I'm so
> laced in it that I can't see the easy way in?
>
>
>
>
>
> On Mon, Feb 4, 2019 at 11:46 AM Christian Grün 
> wrote:
>
>> Hi France,
>>
>> > I noticed that the latest version of BaseX lost this feature and
>> nothing seems to replace it. I'm trying to improve performance of batch
>> processes and I was counting on that feature a lot. Any change it will come
>> back or that something equivalent will come?
>>
>> With BaseX 9, we removed the classical GLOBALLOCK option (i.e.,
>> GLOBALLOCK = false is standard now).
>>
>> > get db:open($lang)/*
>> > process
>> > save to db:open('staging-' || $lang)
>>
>> The name of your database may be specified as static string in your
>> query (no matter if you use BaseX 8 or 9):
>>
>>   get db:open('de')/*
>>   process
>>   save to db:open('staging-de')
>>
>> Did you try this already?
>> Christian
>>
>
>
> --
> France Baril
> Architecte documentaire / Documentation architect
> france.ba...@architextus.com
>

Re: [basex-talk] BaseX/GUI v9.1.2 memory use

[Marco Lettere]

Same for us over here. The ability to follow symlinks is a very powerful 
feature that we use to externalize folders (data, restxq for instance). 
So please don't remove it altogether!

M.

On 22/01/19 23:50, Graydon Saunders wrote:
I've been handling updates by making data/ a symbolic link to a data 
directory that's a sibling of the basex directory.  (Move the old, 
unpack the new, go into new and replace data/ with a symbolic link up 
and over.)


Would hate to see that stop working.

On Tue, Jan 22, 2019, 17:36 Christian Grün > wrote:


Good to hear that! I can’t recollect that something particular has
changed in version 9.1.2, regarding the scanning of project files,
but I’ll have some thoughts how we can trace and interrupt such
loops (or ignore symbolic links instead).



Am Di., 22. Jan. 2019, 23:22 hat Bridger Dyson-Smith
mailto:bdysonsm...@gmail.com>> geschrieben:

Glad that helped :)

I see this when I start from a fresh install vs expanding the
ZIP into the same directory.

On Tue, Jan 22, 2019, 5:17 PM Rick Graham
mailto:rickhg1...@gmail.com> wrote:

Thanks Bridger!

Indeed, I quit basexgui and manually edited .basexgui to
set the project directory to a newly created empty
directory.  basexgui seems normal/stable after that.

I rarely, as in almost never, use wine but I didn't have
this issue with previous versions of BaseX.  Something
seems unexpected here.


On Tue, Jan 22, 2019 at 11:04 PM Bridger Dyson-Smith
mailto:bdysonsm...@gmail.com>> wrote:

Hi Rick, et al,
I think (but am not 100% sure) that the GUI defaults
to looking through your home directory on startup. So,
somewhere in `~/rick/.wine/dosdevices/...` you have
symbolic links that are looped.

I think you might be able to circumvent this problem
by finding `.basexgui` - it would probably be close to
wherever you started the GUI from on your filesystem.
I think you can edit some of the PATHS there and that
may help?

Again, I'm not sure. HTH!
Best,
Bridger

On Tue, Jan 22, 2019 at 4:56 PM Rick Graham
mailto:rickhg1...@gmail.com>>
wrote:

The command-line seemed to be operating normally.

What exactly is/are my project directories?

I attached to the running GUI instance `strace -f
-e trace=stat -p 13368` and it has infinite
repetitions of:

[pid 13436]

stat("/home/rick/.wine/dosdevices/z:/sys/class/thermal/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone2/subsystem/thermal_zone0/device/subsystem/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/device:75/subsystem/devices/PNP0C0A:02",
0x7f7beb2796e0) = -1 ELOOP (Too many levels of
symbolic links)


What's going on here?

On Tue, Jan 22, 2019 at 10:21 PM Christian Grün
mailto:christian.gr...@gmail.com>> wrote:

at

org.basex.gui.view.project.ProjectFiles.add(ProjectFiles.java:167)
at

org.basex.gui.view.project.ProjectFiles.add(ProjectFiles.java:173)
at

org.basex.gui.view.project.ProjectFiles.add(ProjectFiles.java:173)
at

org.basex.gui.view.project.ProjectFiles.add(ProjectFiles.java:173)


Looks like a endless loop that is caused by
parsing the files in your project directory.
Do you possibly have any symbolic links?

Can you reproduce the problem with a
completely fresh BaseX zip archive?

Re: [basex-talk] Documentation suggestion

[Marco Lettere]

One more ... the signature of hmac function at [1] looks messed up...

[1] http://docs.basex.org/wiki/Cryptographic_Module#crypto:hmac

On 16/01/19 16:42, Christian Grün wrote:

I had one but lost it like 99% of my account credentials ... :-(

Sorry to hear that. – I’ll be happy to enable another one for you.



M.

On 16/01/19 16:25, Christian Grün wrote:

Hi Marco,


I suggest to add an explicit statement of which is the default value of
the combine option of map:merge.

Done. If you are interested, feel free to get a little account for our Wiki!

Christian

Re: [basex-talk] Documentation suggestion

[Marco Lettere]

Thanks.
I had one but lost it like 99% of my account credentials ... :-(
M.

On 16/01/19 16:25, Christian Grün wrote:

Hi Marco,


I suggest to add an explicit statement of which is the default value of
the combine option of map:merge.

Done. If you are interested, feel free to get a little account for our Wiki!

Christian

Re: [basex-talk] Serialization issue with HTTP response

[Marco Lettere]

Just WoW! Looks great!
M.

On 16/01/19 16:27, Christian Grün wrote:

I hope I’ll be able to test this tomorrow. And your mail is yet
another motivation to get the upcoming version 2 of the HTTP Client
finished (http://expath.github.io/expath-cg/specs/http-client-2).



On Tue, Jan 15, 2019 at 11:35 AM Marco Lettere  wrote:

Hi all,
we have to deal with a third-party REST service which in case of error 
conditions returns this mime type: application/problem+json; charset=utf-8.

I wrote this RestXQ [1] to mock it. Just copy it into restxq.xqm...

When I call it like [2] I get [XPTY0004] Cannot convert xs:base64Binary to xs:string: 
"IntcImFcIjpcImJcIn0i". This is because the response is seen as a binary and 
this can be double checked calling with [3].
I felt (maybe wrongly) that specializations of the mimetype had been faced in 
BaseX implementation of http:request somewhere back in time but I couldn't find 
anything in the mail archives.

The strange thing however is that I get the same error when calling like [4] 
with override-mime-type set to text/json or application/json which instead I 
was expecting to work.

Is the behaviour in this scenario expected or does this hide some inconsistency?

Thanks, as usual, for the support.

Regards,

Marco.

[1]
declare%rest:path("mc/dummy")
%rest:GET
%output:method("json")
function page:dummy() {
   
 
   
 
   ,
   '{"a":"b"}'
};

[2]
json:parse(
   http:send-request(,"http://localhost:8984/mc/dummy;)[2]
)

[3]
json:parse(
   convert:binary-to-string(
 http:send-request(,"http://localhost:8984/mc/dummy;)[2]
   )
)

[4]
json:parse(
   http:send-request(,"http://localhost:8984/mc/dummy;)[2]
)