from:"Zimmel, Daniel"

Re: [basex-talk] Bug in parse-xml-fragment() and ampersand entity?

2023-11-22 Thread Zimmel, Daniel

<pre>Cool – this looks smart with using index-of on the $replace sequence. I got a 
feeling that analyze-string might help out with it returning text mixed with 
elements, but never thought of this pattern.

Thanks for the XQuery enlightenment, I think I can very much build on this 
example and avoid the entity problems.

Von: Christian Grün <christian.gr...@gmail.com>
Gesendet: Mittwoch, 22. November 2023 12:41
An: Zimmel, Daniel <d.zim...@esvmedien.de>
Cc: BaseX <basex-talk@mailman.uni-konstanz.de>
Betreff: Re: [basex-talk] Bug in parse-xml-fragment() and ampersand entity?

You could try this (at least for basic use cases):

declare function local:replaceText(
  $inputas text(),
  $search   as xs:string*,
  $replace  as xs:string*
) as node()* {
  let $pattern := string-join($search ! ('\b' || . || '\b'), '|')
  for $m in analyze-string($input, $pattern, 'j')/*
  return if($m/self::fn:match) then (
element { $replace[index-of($search, $m)] } {}
  ) else (
text { data($m) }
  )
};
local:replaceText(
  text { 'a String1 b String2 c' },
  ("String1", "String2", "String3"),
  ("Replace1", "Replace2", "Replace3")
)


On Wed, Nov 22, 2023 at 12:15 PM Zimmel, Daniel 
<d.zim...@esvmedien.de<<a  rel="nofollow" href="mailto:d.zim...@esvmedien.de">mailto:d.zim...@esvmedien.de</a>>> wrote:
Thanks for the example, this makes sense.
In my case, I think I can safely add a function to ensure that any “&amp;” 
string will be passed to parse-xml-fragment() as “&amp;amp;” because the source 
of the unparsed xml will always be well-formed.

For background:

I need to wrap a given list of string matches in a textnode to elements and 
because I need to to this recursively (eg the string is found three times in 
the textnode) and also need to pass a list of different matches with different 
replacements, I find it easier to do it with string replacements and then 
parsing the thenfragmented text node back to XML rather than ending up 
splitting the text node to multiple nodes directly.

declare function my:replaceText(
  $word as xs:string,
  $search as xs:string*,
  $replace as xs:string*) as xs:string {
  (: $search = ("String1", "String2", "String3")
 $replace = ("&lt;Replace1/&gt;", "&lt;Replace2/&gt;", "&lt;Replace3/&gt;")
 :)
 if (empty($search)) then $word
  else
  replace(my:replaceText($word, tail($search), tail($replace)), 
'(?<![">])'||my:escapeText(head($search)), head($replace),'j')
};

perhaps there is a better way to do this... how would one do this sort of 
replacement on mixed content without having to parse this back to XML?
But this works for me when being careful to add some handling for &amp, &gt and 
&lt.

+1 for internal parser velocity!

Von: Christian Grün 
<christian.gr...@gmail.com<<a  rel="nofollow" href="mailto:christian.gr...@gmail.com">mailto:christian.gr...@gmail.com</a>>>
Gesendet: Dienstag, 21. November 2023 18:54
An: Zimmel, Daniel <d.zim...@esvmedien.de<<a  rel="nofollow" href="mailto:d.zim...@esvmedien.de">mailto:d.zim...@esvmedien.de</a>>>
Cc: BaseX 
<basex-talk@mailman.uni-konstanz.de<<a  rel="nofollow" href="mailto:basex-talk@mailman.uni-konstanz.de">mailto:basex-talk@mailman.uni-konstanz.de</a>>>
Betreff: Re: [basex-talk] Bug in parse-xml-fragment() and ampersand entity?

Yes, I can see the problem: &DUMMY; ist interpreted as unknown entity and thus 
replaced with a question mark (a better choice would be the Unicode Replacement 
Character xFFFD anyway, from today's perspective). We'll keep that in mind and 
think about alternatives.

If your input is supposed to be interpreted as a single text fragment, one 
fallback solution (for now) would be

data(parse-xml('<x>' || $string || '</x>'))





Zimmel, Daniel <d.zim...@esvmedien.de<<a  rel="nofollow" href="mailto:d.zim...@esvmedien.de">mailto:d.zim...@esvmedien.de</a>>> schrieb am 
Di., 21. Nov. 2023, 18:34:
Thanks for the insight!

I can see the benefit with your example – if you look at my example, it is 
clearly eating the text (“DUMMY”) which might be an edge case, but is obviously 
a problem when you think the function will give you an error in case of 
non-wellformedness – some text has silently been deleted.

Daniel

Von: Christian Grün 
<christian.gr...@gmail.com<<a  rel="nofollow" href="mailto:christian.gr...@gmail.com">mailto:christian.gr...@gmail.com</a>>>
Gesendet: Dienstag, 21. November 2023 16:59
An: Zimmel, Daniel <d.zim...@esvmedien.de<<a  rel="nofollow" href="mailto:d.zim...@esvmedien.de&quo

Re: [basex-talk] Bug in parse-xml-fragment() and ampersand entity?

2023-11-22 Thread Zimmel, Daniel

<pre>Thanks for the example, this makes sense.
In my case, I think I can safely add a function to ensure that any “&amp;” 
string will be passed to parse-xml-fragment() as “&amp;amp;” because the source 
of the unparsed xml will always be well-formed.

For background:

I need to wrap a given list of string matches in a textnode to elements and 
because I need to to this recursively (eg the string is found three times in 
the textnode) and also need to pass a list of different matches with different 
replacements, I find it easier to do it with string replacements and then 
parsing the thenfragmented text node back to XML rather than ending up 
splitting the text node to multiple nodes directly.

declare function my:replaceText(
  $word as xs:string,
  $search as xs:string*,
  $replace as xs:string*) as xs:string {
  (: $search = ("String1", "String2", "String3")
 $replace = ("&lt;Replace1/&gt;", "&lt;Replace2/&gt;", "&lt;Replace3/&gt;")
 :)
 if (empty($search)) then $word
  else
  replace(my:replaceText($word, tail($search), tail($replace)), 
'(?<![">])'||my:escapeText(head($search)), head($replace),'j')
};

perhaps there is a better way to do this... how would one do this sort of 
replacement on mixed content without having to parse this back to XML?
But this works for me when being careful to add some handling for &amp, &gt and 
&lt.

+1 for internal parser velocity!

Von: Christian Grün <christian.gr...@gmail.com>
Gesendet: Dienstag, 21. November 2023 18:54
An: Zimmel, Daniel <d.zim...@esvmedien.de>
Cc: BaseX <basex-talk@mailman.uni-konstanz.de>
Betreff: Re: [basex-talk] Bug in parse-xml-fragment() and ampersand entity?

Yes, I can see the problem: &DUMMY; ist interpreted as unknown entity and thus 
replaced with a question mark (a better choice would be the Unicode Replacement 
Character xFFFD anyway, from today's perspective). We'll keep that in mind and 
think about alternatives.

If your input is supposed to be interpreted as a single text fragment, one 
fallback solution (for now) would be

data(parse-xml('<x>' || $string || '</x>'))





Zimmel, Daniel <d.zim...@esvmedien.de<<a  rel="nofollow" href="mailto:d.zim...@esvmedien.de">mailto:d.zim...@esvmedien.de</a>>> schrieb am 
Di., 21. Nov. 2023, 18:34:
Thanks for the insight!

I can see the benefit with your example – if you look at my example, it is 
clearly eating the text (“DUMMY”) which might be an edge case, but is obviously 
a problem when you think the function will give you an error in case of 
non-wellformedness – some text has silently been deleted.

Daniel

Von: Christian Grün 
<christian.gr...@gmail.com<<a  rel="nofollow" href="mailto:christian.gr...@gmail.com">mailto:christian.gr...@gmail.com</a>>>
Gesendet: Dienstag, 21. November 2023 16:59
An: Zimmel, Daniel <d.zim...@esvmedien.de<<a  rel="nofollow" href="mailto:d.zim...@esvmedien.de">mailto:d.zim...@esvmedien.de</a>>>
Cc: 
basex-talk@mailman.uni-konstanz.de<<a  rel="nofollow" href="mailto:basex-talk@mailman.uni-konstanz.de">mailto:basex-talk@mailman.uni-konstanz.de</a>>
Betreff: Re: [basex-talk] Bug in parse-xml-fragment() and ampersand entity?

Hi Daniel,

Yes, I assume we’ll need to call it a bug… Although what BaseX is currently 
doing is known to us to be out of spec behavior. The function 
fn:parse-xml-fragments is based on our internal XML parser, which is much 
faster than the standard XML parser (in particular for small input), and it 
tolerates input that’s not perfectly well-formed. In addition, it accepts HTML 
entities without a linked DTD:

   parse-xml-fragment(`&auml;`)

We should at least document the behavior or (better) introduce a custom BaseX 
function for it.

Hope this helps (for now),
Christian



On Tue, Nov 21, 2023 at 3:17 PM Zimmel, Daniel 
<d.zim...@esvmedien.de<<a  rel="nofollow" href="mailto:d.zim...@esvmedien.de">mailto:d.zim...@esvmedien.de</a>>> wrote:
Hi,

is this a bug?

Query:
parse-xml-fragment('Tom &amp; Jerry')

Result:
Tom ? Jerry

Same result with:
parse-xml-fragment('Tom &amp;DUMMY; Jerry')

BaseX 10.7

Saxon complains correctly that the resulting document node is not well-formed.
BaseX should also return an error, shouldn't it?

Best, Daniel
</pre>

Re: [basex-talk] Bug in parse-xml-fragment() and ampersand entity?

2023-11-21 Thread Zimmel, Daniel

Thanks for the insight!

I can see the benefit with your example – if you look at my example, it is 
clearly eating the text (“DUMMY”) which might be an edge case, but is obviously 
a problem when you think the function will give you an error in case of 
non-wellformedness – some text has silently been deleted.

Daniel

Von: Christian Grün 
Gesendet: Dienstag, 21. November 2023 16:59
An: Zimmel, Daniel 
Cc: basex-talk@mailman.uni-konstanz.de
Betreff: Re: [basex-talk] Bug in parse-xml-fragment() and ampersand entity?

Hi Daniel,

Yes, I assume we’ll need to call it a bug… Although what BaseX is currently 
doing is known to us to be out of spec behavior. The function 
fn:parse-xml-fragments is based on our internal XML parser, which is much 
faster than the standard XML parser (in particular for small input), and it 
tolerates input that’s not perfectly well-formed. In addition, it accepts HTML 
entities without a linked DTD:

   parse-xml-fragment(``)

We should at least document the behavior or (better) introduce a custom BaseX 
function for it.

Hope this helps (for now),
Christian



On Tue, Nov 21, 2023 at 3:17 PM Zimmel, Daniel 
mailto:d.zim...@esvmedien.de>> wrote:
Hi,

is this a bug?

Query:
parse-xml-fragment('Tom  Jerry')

Result:
Tom ? Jerry

Same result with:
parse-xml-fragment('Tom DUMMY; Jerry')

BaseX 10.7

Saxon complains correctly that the resulting document node is not well-formed.
BaseX should also return an error, shouldn't it?

Best, Daniel

[basex-talk] Bug in parse-xml-fragment() and ampersand entity?

2023-11-21 Thread Zimmel, Daniel

Hi,

is this a bug?

Query:
parse-xml-fragment('Tom  Jerry')

Result:
Tom ? Jerry

Same result with:
parse-xml-fragment('Tom DUMMY; Jerry')

BaseX 10.7

Saxon complains correctly that the resulting document node is not well-formed.
BaseX should also return an error, shouldn't it?

Best, Daniel

Re: [basex-talk] use-character-maps vs parameter-document with db:export()

2023-10-05 Thread Zimmel, Daniel

Thanks for the hint.
So this would be a parameter for the implementation option that is described in 
detail here:
https://www.w3.org/TR/xslt-xquery-serialization-31/#serparams-in-xdm-instance

I was not sure, because it is declared a standard (BaseX) parameter here 
https://docs.basex.org/wiki/Serialization#Standard_Parameters but not getting 
respected in db:export() (https://docs.basex.org/wiki/Database_Module#db:export)
It only works when is is declared in the prolog, as in your example on the wiki 
page (declare option output:parameter-document "map.xml";)

Daniel

Von: Christian Grün 
Gesendet: Dienstag, 3. Oktober 2023 14:38
An: Zimmel, Daniel 
Cc: BaseX 
Betreff: Re: [basex-talk] use-character-maps vs parameter-document with 
db:export()

Hi Daniel,

 Is it?

It is. Maybe the parameter could be introduced with the 4.0 version of the 
spec. I invite you to add a proposal for it in the qtspecs repository [1].

Thanks,
Christian

[1] https://github.com/qt4cg/qtspecs/issues

[basex-talk] use-character-maps vs parameter-document with db:export()

2023-09-29 Thread Zimmel, Daniel

Hi,

when using db:export, the serialization parameter use-character-maps works just 
fine.
But can someone explain why it is not possible to use parameter-document? At 
least, it does not work here with 10.3.

As in:

db:export($dbname,$path, map{
  'indent':'no',
  'omit-xml-declaration':'no',
  'parameter-document':'map.xml'
})

OK I can see that there is no parameter-document option in the table I can find 
here defined for fn:serialize():
https://www.w3.org/TR/xpath-functions-31/#func-serialize

When not using db:export(), all is well as described:
https://docs.basex.org/wiki/Serialization#Character_mappings

So in short, I think the answer is the spec. Is it?

Have a nice weekend, Daniel

Re: [basex-talk] Timeframe for supporting fn:load-xquery-module()?

2023-06-08 Thread Zimmel, Daniel

XQS looks like an interesting approach, thanks for the pointer.

I guess you do know about it, but if you do not, for "an implementation of 
Schematron that would work well with BaseX" I use SchXslt quite heavily, which 
can be installed with your BaseX installation with the REPO command, see 
https://github.com/schxslt/schxslt/releases

Daniel

Von: BaseX-Talk  Im Auftrag von 
Eliot Kimber
Gesendet: Donnerstag, 8. Juni 2023 15:05
An: BaseX 
Betreff: Re: [basex-talk] Timeframe for supporting fn:load-xquery-module()?

I am very interested in an implementation of Schematron that would work well 
with BaseX-we use Schematron very heavily at ServiceNow and I have been trying 
to implement some Schematron features in our Mirabel BaseX application, 
although I've had very little time to work on it. We are trying to implement a 
"Schematron tester" that lets you explore existing Schematron rules and see how 
they function against the content managed in the BaseX databases and also 
create and test new rules. My current implementation is very crude and doesn't 
implement the full range of what you can do in a Schematron.

If there's anything I can do to assist or test a Schematron implementation, 
please don't hesitate to reach out.

Cheers,

Eliot
_
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com
LinkedIn | 
Twitter | 
YouTube | 
Facebook

From: BaseX-Talk 
mailto:basex-talk-boun...@mailman.uni-konstanz.de>>
 on behalf of Christian Grün 
mailto:christian.gr...@gmail.com>>
Date: Thursday, June 8, 2023 at 2:46 AM
To: Andrew Sales mailto:and...@andrewsales.com>>
Cc: BaseX 
mailto:basex-talk@mailman.uni-konstanz.de>>
Subject: Re: [basex-talk] Timeframe for supporting fn:load-xquery-module()?
[External Email]


Hi Andrew,

Thanks for your mail. I've seen your and Adam's comment, but I didn't have time 
yet to give a reply. We'll add an implementation of fn:load-xquery-module in 
BaseX 11, which is planned for Summer/Fall this year. A proof-of-concept 
implementation will be made available with BaseX 10.7 or 10.8.

All the best
Christian



Andrew Sales mailto:and...@andrewsales.com>> schrieb am 
Mi., 7. Juni 2023, 23:15:
Hello,

The docs say:

"The three functions fn:transform, fn:load-xquery-module and fn:collation-key 
may be added in a future version of BaseX as their implementation might require 
the use of additional external libraries."

Is there a timeframe for this? I'm interested in particular in the 
load-xquery-module() to enable an implementation of Schematron written in pure 
XQuery[1] to be portable between engines[2].

Many thanks,
Andrew

[1] https://github.com/AndrewSales/XQS
[2] https://github.com/AndrewSales/XQS/issues/8

Re: [basex-talk] Odd behaviour in the GUI (XPath with XQuery window)

2023-04-19 Thread Zimmel, Daniel


> Just to be sure: You are pointing to the (as we call it) input bar, 
> containing the dropdown menu for the input mode and the text field, right [1]?

right

>>   * hit Strg-A to mark the line so I can start with a new query

> And just a note: When placing the cursor in this field, previously entered 
> should automatically be highlighted. Maybe that depends on the OS, I’m not 
> sure.

ah yes, it is automatically highlighted directly after placing the cursor - no 
problem there. It is NOT highlighted when the cursor is still in place and I 
click again (at least on Windows). Sorry I was not specific about this - and 
not really a problem here.

>>   * enter a slash to my XPath, it jumps back to a random entry in the 
>> history, and does appends my slash to that entry. This does not make sense 
>> to me at all.

> That doesn’t sounds reasonable. It seems to work on my system, though, and I 
> cannot remember something has been changed.

Thanks for the feedback. Now this is strange, I just restarted my GUI and the 
problem vanishes :-o
(Usually the GUI will remain open for several days on my machine so I do not 
restart very often)
Seems I have to look closer on what causes this behaviour. Glad to know it is 
not intended, though.
I hope I can reproduce this soon.

Maybe I accidentally hit some shortcut to prompt this behaviour? It is strange, 
because it happens so often I thought this had changed.
I am on Windows 10 with Java 19.0.1 (plus JTattoo-1.6.13)- this might be some 
odd Windows thing I accidentally trigger regularly, it seems. For the moment, I 
just cannot tell how.

Thanks, Daniel

[basex-talk] Odd behaviour in the GUI (XPath with XQuery window)

2023-04-19 Thread Zimmel, Daniel

Hi,

somewhere between BaseX 9 and 10 there was a change in the behaviour of the 
Find/XQuery window (which I use quite a lot for quick access to data):

* with 9.5, I can enter any valid XPath, and then delete it for a fresh start 
with another expression...
* with 10.3, whenever I do the following:
  * place the cursor in the XQuery window
  * hit Strg-A to mark the line so I can start with a new query
  * enter a slash to my XPath, it jumps back to a random entry in the history, 
and does appends my slash to that entry. This does not make sense to me at all.

Is this a feature or a bug? Am I using it wrong? I find it quite irritating, 
because it takes away the possibility to fire a quick XPath (as a valid XQuery 
expression) on the currently opened database without having to open a fresh 
query tab.

Sorry if i am missing something obvious.

Thanks, Daniel

Re: [basex-talk] Potential bug in GUI, Unit Tests

2023-03-31 Thread Zimmel, Daniel


>> 1. changes that are not saved to disk do not get respected - as 
>> opposed to other queries

> True, this feels confusing. Maybe it would be better to disable the test icon 
> if the current file has not been saved?

Yes that would be less confusing (to me).

> If you prefer to, you can enable the option »Save before executing files« in 
> the Preferences Dialog. This way, opened files will automatically be saved 
> before a query is run.

Thanks, I did not know that.

> 2. pressing the default button for running queries executes the most 
> recent query which is open in another tab

> This is actually meant to be a feature: If lots of library modules are opened 
> in the editor, it can save a lot of time if you can execute the main module 
> that has been executed most recently.

Ah - I can see the value - I always thought there is no other context for the 
play button than the current tab. 

Thanks, Daniel

[basex-talk] Potential bug in GUI, Unit Tests

2023-03-23 Thread Zimmel, Daniel

Hi,

while I am trying to find my way around the Unit Module, I noticed the 
following unexpected behavior.

0. Using the test example from https://docs.basex.org/wiki/Unit_Module#Query in 
the GUI
1. changes that are not saved to disk do not get respected - as opposed to 
other queries
2. pressing the default button for running queries executes the most recent 
query which is open in another tab

While 1 might not be a bug , 2 feels like one (a query fires that is not 
displayed on the current tab)

BaseX 10.3

Bye, Daniel

Re: [basex-talk] Progress bar (again)

2023-03-16 Thread Zimmel, Daniel

Yes, this works great in the console!
However I can see why it is better to stick to the standard functions. In my 
case, this does not work when running via an Ant apply task, since Ant has its 
own loggers and is doing something internally which prevents jumping to the 
start of the line. Perhaps there is a way to do this with Ant, but I do not 
think this is a good idea for portability...

Thanks for the insight, Daniel

-Ursprüngliche Nachricht-
Von: Christian Grün  
Gesendet: Mittwoch, 15. März 2023 17:37
An: Zimmel, Daniel 
Cc: basex-talk@mailman.uni-konstanz.de
Betreff: Re: [basex-talk] Progress bar (again)

> Thanks, I was hoping there might be some way to update the output result 
> instead of simply adding to output.
> I just ran a 420 minutes regular expressions query, which I might be 
> going to optimize now because there is not much visible progress 
> anymore :-P

Sorry, now I got it. You could try something like this:

for $i in 1 to 100
let $string := string-join((
  (: yields , and jumps to the beginning of the current line :)
  string:cr(),
  'FORTSCHRITT: ',
  (1 to $i idiv 5) ! '.',
  $i, '%'
))
return (
  prof:sleep(50),
  (: calls System.err.println($string) :)
  Q{java.lang.System}err() => Q{java.io.PrintStream}print($string)
)

I think we won’t offer a built-in function for that, as the result of functions 
like fn:trace, prof:dump, etc. will be sent to different output channels, some 
of which won’t allow the overwriting of output that has previously been sent.

Hope this helps,
Christian

Re: [basex-talk] Progress bar (again)

2023-03-15 Thread Zimmel, Daniel

Thanks, I was hoping there might be some way to update the output result 
instead of simply adding to output.
I just ran a 420 minutes regular expressions query, which I might be going to 
optimize now because there is not much visible progress anymore :-P

-Ursprüngliche Nachricht-
Von: Christian Grün  
Gesendet: Donnerstag, 9. März 2023 17:25
An: Zimmel, Daniel 
Cc: basex-talk@mailman.uni-konstanz.de
Betreff: Re: [basex-talk] Progress bar (again)

Hi Daniel,

> can anyone say what might be needed to implement this as a module?

Looks interesting! I hope our Wiki article on creating and installing modules 
is helpful [1].

Cheers,
Christian

[1] https://docs.basex.org/wiki/Repository

>
> Daniel
>

Re: [basex-talk] Problem with using XInclude in BaseX

2023-03-15 Thread Zimmel, Daniel

While this may not solve your problem, I can confirm that your files import 
just fine with BaseX 10.3 GUI on my Windows machine.
There is a checkbox in the import dialog "Use XInclude", but it works 
regardless whether the I check or uncheck the box.

-Ursprüngliche Nachricht-
Von: BaseX-Talk  Im Auftrag von 
Virgile Reignier
Gesendet: Dienstag, 14. März 2023 18:55
An: basex-talk@mailman.uni-konstanz.de
Betreff: [basex-talk] Problem with using XInclude in BaseX

Bonjour,

Je me permets de vous contacter dans le cadre de mon doctorat en histoire 
médiévale. Je souhaiterai en effet organiser le dépouillement de mes archives à 
l'aide d'une base de données en XML en utilisant le logiciel BaseX, mais je 
suis face à une difficulté que je n'arrive pas à surmonter dans son 
utilisation. Cette difficulté est liée à l'utilisation de la balise 
.

J'ai tenté de produire un exemple minimaliste avec deux fichiers dans un même 
répertoire pour voir ce qui ne marche pas :

file music.xml :

http://www.w3.org/2001/XInclude;>
 Smoke on the water
 Deep Purple
 


file label.xml :

Purple Records

Alors que ces deux fichiers sont décrits comme valides par Oxygen, 
l'importation du fichier music.xml avec BaseX (que ce soit avec l'interface DBA 
ou l'interface GUI) génère systématiquement l'erreur suivante : "An include 
with href 'label.xml'failed, and no fallback element was found". Je peux en 
revanche importer le fichier dans la base de données via Oxygen et 
l'utilisation du WebDAV de BaseX, mais le fichier est alors indiqué comme étant 
de type "binary" et est absolument inemployable pour réaliser des requêtes.

J'ai pourtant tenté de réaliser la requête suivante au sein de ma DB dans BaseX 
GUI, ce qui n'a pas renvoyé d'erreurs :

declare namespace xi = "http://www.w3.org/2001/XInclude;;

xi:include

J'ai également tenté de charger ces mêmes fichiers dans eXist DB, et 
l'inclusion s'est réalisée sans problèmes. Je ne comprends donc pas ce qui ne 
va pas avec l'utilisation de XInclude dans BaseX. Est-ce qu'il y a un problème 
avec la résolution des URI ?

Je travaille sous Windows 10 (j'ai aussi fait des tests sur Ubuntu et obtenu le 
même résultat), BaseX 10.4 et Java 1.8.0_361.

Est-ce que c'est quelque chose qui vous est déjà arrivé ? J'ai essayé de 
chercher dans les archives mail de basex-talk et trouvé quelques problèmes 
similaires, mais qui ont été résolus par l'emploi de BaseX GUI. Or chez moi 
cela marche aussi mal avec GUI qu'avec DBA. Je peux transférer mon travail sur 
eXist DB, mais j'avoue m'être assez familiarisé avec BaseX et souhaiterai 
rester sur ce logiciel tant que possible. Je suis donc preneur de toutes vos 
propositions.

En vous remerciant par avance pour vos réponses,

Bien cordialement,

Virgile Reignier

Hello,

I am contacting you as part of my PhD in medieval history. I would like to 
organise the analysis of my archives with the help of an XML database using the 
BaseX software, but I am encountering a difficulty that I cannot overcome in 
its use. This difficulty is related to the use of the  tag.

I tried to produce a minimalist example with two files in the same directory to 
see what goes wrong:

file music.xml :

http://www.w3.org/2001/XInclude;>
 Smoke on the water
 Deep Purple
 


file label.xml :

Purple Records

While both files are described as valid by Oxygen, importing the music.xml file 
with BaseX (either with the DBA interface or the GUI
interface) consistently generates the following error: "An include with href 
'label.xml'failed, and no fallback element was found". I can import the file 
into the database via Oxygen and the use of BaseX WebDAV, but the file is then 
indicated as being of type "binary" and is absolutely unusable for queries.

However, I tried to perform the following query within my DB in BaseX GUI, 
which did not return any errors:

declare namespace xi = "http://www.w3.org/2001/XInclude;;

xi:include

I also tried to load these same files into eXist DB, and the inclusion was 
successful. So I don't understand what is wrong with using XInclude in BaseX. 
Is there a problem with URI resolution?

I'm working on Windows 10 (I also tested on Ubuntu and got the same result), 
BaseX 10.4 and Java 1.8.0_361.

Is this something that has happened to you before? I tried to search the 
basex-talk mail archives and found some similar problems, but they were solved 
by using BaseX GUI. However, in my case it works just as badly with GUI as with 
DBA. I can transfer my work to eXist DB, but I have become quite familiar with 
BaseX and would like to stay with it as long as possible. I am therefore 
interested in any suggestions you may have.

Thank you in advance for your answers,

Virgile Reignier

[basex-talk] Progress bar (again)

2023-03-09 Thread Zimmel, Daniel

Hey,

I have a task that may take longer to execute (several minutes), so I felt I 
got to have a fancy progress bar on the command line to entertain me.

No progress bar though, found this in the archives: 
https://mailman.uni-konstanz.de/pipermail/basex-talk/2020-November/015845.html

So this is the closest I got (running BaseX from an Ant task):

xquery_update_db:
 [echo] XQuery Update (Database) wird ausgeführt: xquery/basex.reduce.xquery
[apply] EXPORTER: omit-xml-declaration=no
[apply] Database 'DEBUGX' was opened in 113.62 ms.
[apply] "FORTSCHRITT: 0%"
[apply] "FORTSCHRITT: *5%"
[apply] "FORTSCHRITT: **10%"
[apply] "FORTSCHRITT: ***15%"
[apply] "FORTSCHRITT: 20%"
[apply] "FORTSCHRITT: *25%"

While I am happy enough with this (it is quite accurate and I like the 
dramatically mounting effect), can anyone say what might be needed to implement 
this as a module?

Daniel

Re: [basex-talk] Possible to get line number of node?

2022-12-22 Thread Zimmel, Daniel

If you are allowed to include macros, there should be a way to use some VBA 
scripting for providing a launcher for Oxygen... but you don’t want to ask me 
about VBA :-)

Over here we run a self-built NodeJS-webapp, where we can display an “Open in 
Oxygen” button next to the message.

Von: Eliot Kimber 
Gesendet: Donnerstag, 22. Dezember 2022 15:01
An: Zimmel, Daniel ; basex-talk@mailman.uni-konstanz.de
Betreff: Re: Possible to get line number of node?

Interesting—I did not know Oxygen provided this option.

I’m wondering what I could put in an Excel spreadsheet that would make this 
work on a user’s machine…

Cheers,

E.

_
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> | 
Twitter<https://twitter.com/servicenow> | 
YouTube<https://www.youtube.com/user/servicenowinc> | 
Facebook<https://www.facebook.com/servicenow>

From: Zimmel, Daniel mailto:d.zim...@esvmedien.de>>
Date: Thursday, December 22, 2022 at 4:21 AM
To: Eliot Kimber 
mailto:eliot.kim...@servicenow.com>>, 
basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> 
mailto:basex-talk@mailman.uni-konstanz.de>>
Subject: AW: Possible to get line number of node?
[External Email]


I can’t answer anything helpful for the reliability of line numbers, but can 
tell you how we are doing this in a similar use case:
we make use of what Oxygen calls “simplified XPath index path” 
(https://www.oxygenxml.com/doc/versions/25.0/ug-editor/topics/opening-document-from-cli.html)

Roughly:
* we write out that path (“#element(1/2/6/15)”) to a Schematron diagnostics 
element
* in our frontend to Schemaron we generate a button which will take you 
directly to Oxygen

So, if you don’t have to care about other editors than Oxygen, this might be a 
simple and reliable way. At least to us, works like a charm.

Best, Daniel

Von: BaseX-Talk 
mailto:basex-talk-boun...@mailman.uni-konstanz.de>>
 Im Auftrag von Eliot Kimber
Gesendet: Mittwoch, 21. Dezember 2022 23:14
An: 
basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de>
Betreff: [basex-talk] Possible to get line number of node?

I’m pretty sure the answer is “no”, but is there any way from BaseX 9.6 to get 
the source line number a given node starts on?

My use case is I’m looking for specific elements and making a report of 
documents that contain them and identifying information about the elements to 
help with finding them outside of BaseX (i.e., in Oxygen where the authoring 
happens). I’m parsing with the “trim whitespace” option turned off and I’m not 
doing any DTD-aware parsing, so the elements as loaded into the database will 
match the source (no additional attributes or anything). So at least in theory 
BaseX could maintain accurate line numbers.

These elements don’t have @id attributes so no natural way to identify them, so 
it would be useful to be able to report the source line number as a guide to 
people fixing these elements.

I didn’t see anything in the docs so I thought I’d ask just in case I’ve missed 
some trick.

Thanks,

E.
_
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> | 
Twitter<https://twitter.com/servicenow> | 
YouTube<https://www.youtube.com/user/servicenowinc> | 
Facebook<https://www.facebook.com/servicenow>

Re: [basex-talk] Possible to get line number of node?

2022-12-22 Thread Zimmel, Daniel

I can’t answer anything helpful for the reliability of line numbers, but can 
tell you how we are doing this in a similar use case:
we make use of what Oxygen calls “simplified XPath index path” 
(https://www.oxygenxml.com/doc/versions/25.0/ug-editor/topics/opening-document-from-cli.html)

Roughly:
* we write out that path (“#element(1/2/6/15)”) to a Schematron diagnostics 
element
* in our frontend to Schemaron we generate a button which will take you 
directly to Oxygen

So, if you don’t have to care about other editors than Oxygen, this might be a 
simple and reliable way. At least to us, works like a charm.

Best, Daniel

Von: BaseX-Talk  Im Auftrag von 
Eliot Kimber
Gesendet: Mittwoch, 21. Dezember 2022 23:14
An: basex-talk@mailman.uni-konstanz.de
Betreff: [basex-talk] Possible to get line number of node?

I’m pretty sure the answer is “no”, but is there any way from BaseX 9.6 to get 
the source line number a given node starts on?

My use case is I’m looking for specific elements and making a report of 
documents that contain them and identifying information about the elements to 
help with finding them outside of BaseX (i.e., in Oxygen where the authoring 
happens). I’m parsing with the “trim whitespace” option turned off and I’m not 
doing any DTD-aware parsing, so the elements as loaded into the database will 
match the source (no additional attributes or anything). So at least in theory 
BaseX could maintain accurate line numbers.

These elements don’t have @id attributes so no natural way to identify them, so 
it would be useful to be able to report the source line number as a guide to 
people fixing these elements.

I didn’t see anything in the docs so I thought I’d ask just in case I’ve missed 
some trick.

Thanks,

E.
_
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com
LinkedIn | 
Twitter | 
YouTube | 
Facebook

Re: [basex-talk] Problem with validate:xsd-report

2022-09-01 Thread Zimmel, Daniel

Not sure why you have the doctype declaration in there... perhaps your parser 
is missing XMLSchema.dtd?
In my xml.xsd there is no such declaration, I fetched it from 
https://www.w3.org/2001/xml.xsd

Best, Daniel

Von: BaseX-Talk  Im Auftrag von 
Hans-Juergen Rennau
Gesendet: Mittwoch, 31. August 2022 21:21
An: BaseX 
Betreff: [basex-talk] Problem with validate:xsd-report

Dear BaseX people,

I encounter a problem with validate:xsd-report(), more particular with using a 
schema importing the well-known xml.xsd. The imported attribute declaration 
(xml:base) is not found.

I am pretty sure that it is my fault, but I just can't find my error - could 
you take a look?

(1) Document: doc.xml

http://www.w3.org/2001/XMLSchema-instance; 
xsi:noNamespaceSchemaLocation="schema.xsd">
xyz


(2) Schema: schema.xsd

http://www.w3.org/2001/XMLSchema; 
elementFormDefault="qualified">
http://www.w3.org/XML/1998/namespace; 
schemaLocation="xml.xsd"/>










(In the same folder, a downloaded copy of xml.xsd is stored - for a snippet, 
see PS)

(3) Query: validate.xq

let $doc := 'doc.xml'
let $xsd := 'schema.xsd'
return validate:xsd-report($doc, $xsd)

(4) Result:

basex validate.xq
=>

  invalid
  src-resolve: 
Cannot resolve the name 'xml:base' to a(n) 'attribute declaration' component.
  


I do not understand why the import seems to be ignored. Validating with oXygen 
works fine, but I suppose that oXygen ignores the @schemaLocation.

If you see the reason, please let me know.

Cheers,
Hans-Jürgen

PS: Snippet from xml.xsd, stored in the same folder:



http://www.w3.org/XML/1998/namespace; 
xmlns:xs="http://www.w3.org/2001/XMLSchema;  xml:lang="en">
  ...
  

  See http://www.w3.org/TR/xmlbase/ for
   information about this attribute.

  
  ...

Re: [basex-talk] XML Catalog and xslt:transform()

2022-06-02 Thread Zimmel, Daniel

I see, thanks Gerrit and Christian for the insight. This *does* sound wickedly 
unfunny.

OK if I actually do not need to be able to parse the DTD wouldn't the simple 
workaround be:

fetch:xml('file:///C:/temp/catalog/dokument.xml')
=> xslt:transform('transform.xsl')

At least this is what works here, resulting in a new document node and trashing 
the DTD declaration.

Daniel

-Ursprüngliche Nachricht-
Von: BaseX-Talk  Im Auftrag von 
Imsieke, Gerrit, le-tex
Gesendet: Donnerstag, 2. Juni 2022 16:40
An: basex-talk@mailman.uni-konstanz.de
Betreff: Re: [basex-talk] XML Catalog and xslt:transform()

As a workaround, you might be able to read the documents using doc() in XQuery 
(this might work with the help of the catalog, in contrast to
doc() from within XSLT/Saxon) and pass them to xslt:transform() in some way. 
“Some way” isn’t easy, either, since xslt:transform() still relies on JAXP, and 
you can’t pass arbitrary XDM items such as whole documents or maps as 
stylesheet parameters (or can you? $params as map(*)? doesn’t rule this out, 
but I doubt that a parameter may have another map as value and arrive safely at 
the stylesheet). So you might need to wrap all inputs in a single top-level 
element, which of course prevents you from letting the XSLT stylesheet decide 
which resource to load dynamically, and you might need to change matching 
patterns.
But switching to XDM and implementing XPath 3.1’s fn:transform() function that 
would allow to was too much of a stretch for Christian at the time we paid 
BaseX GmbH to implement xslt:transform-report(). I think this will need another 
significant investment, and Christian needs to find time to implement it.

Gerrit

On 02.06.2022 16:24, Imsieke, Gerrit, le-tex wrote:
> Hi Daniel,
> 
> I think the catalog in xslt:transform() is only used for XSLT 
> imports/includes and maybe for reading documents with doc(), and only 
> for Saxon. The catalog is probably *not* used for mapping system 
> identifiers in the documents accessed this way. We should document 
> this better once we find out what is/isn’t supported.
> 
> The background is that we desperately needed to use catalogs for 
> mapping import/include URIs, and we paid Liam to implement this. He 
> succeeded with a little help from Christian, but it was not an easy 
> feat because include/import URI resolution is different from doc() URI 
> resolution in Saxon which in turn is different from system identifier 
> resolution (that is probably done by the XML parser, not by Saxon).
> 
> So I think we need to pay Liam and Christian again so that they work 
> out how to pass the catalog to the XML parser that is invoked by 
> Saxon. This definitely isn’t a fun task.
> 
> Gerrit
> 
> On 02.06.2022 14:44, Zimmel, Daniel wrote:
>> Hi,
>>
>> after reading https://docs.basex.org/wiki/Catalog_Resolver and 
>> digging in the list archives 
>> (https://mailman.uni-konstanz.de/pipermail/basex-talk/2019-March/0141
>> 99.html
>> ) I still have trouble understanding catalog files.
>>
>> Is this supposed to work with xslt:transform() and BaseX GUI 9.7.2?
>> The default option (DTD = false) is ignored by xslt:transform() 
>> because the function is definitely requesting the external DTD.
>> This prevents transforming XML with DTD declarations that are not 
>> available (if I understand correctly, a problem that the DTD option 
>> is trying to solve in general).
>>
>> When I try to solve this via catalog files (actually I do not need 
>> the DTD), I do not have success.
>> Here are my mini examples:
>>
>> Saxon HE 10.3 resides in the lib folder
>>
>> .basex setting:
>> # Local Options
>> SERIALIZER = indent=no
>> DTD = true
>>
>> XML in local folder "C:/temp/catalog":
>> >    SYSTEM "http://www.blahblahblah.info/dtd/dokument.dtd;>
>> 
>>    01
>> 
>>
>> catalog.xml in local folder "C:/temp/catalog":
>> > xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">
>>    > systemIdStartString="http://www.blahblahblah.info/dtd/; 
>> rewritePrefix="file:///C:/temp/catalog/dtd/"/>
>> 
>>
>> dokument.dtd in local folder "C:/temp/catalog/dtd":
>> 
>> 
>>
>> XQuery query.xq in local folder "C:/temp/catalog":
>> (# db:catfile catalog.xml #) {
>>    xslt:transform('dokument.xml', 'transform.xsl') }
>>
>>
>> With or without pragma, this always results in a 
>> java.net.UnknownHostException (because the system ID is not available, 
>> that's true), but I would be expecting this would resolve to 
>> "file:///C:/temp/catalog/dtd/dokument.dtd"
>>
>> No

[basex-talk] XML Catalog and xslt:transform()

2022-06-02 Thread Zimmel, Daniel

Hi,

after reading https://docs.basex.org/wiki/Catalog_Resolver and digging in the 
list archives 
(https://mailman.uni-konstanz.de/pipermail/basex-talk/2019-March/014199.html ) 
I still have trouble understanding catalog files.

Is this supposed to work with xslt:transform() and BaseX GUI 9.7.2?
The default option (DTD = false) is ignored by xslt:transform() because the 
function is definitely requesting the external DTD.
This prevents transforming XML with DTD declarations that are not available (if 
I understand correctly, a problem that the DTD option is trying to solve in 
general).

When I try to solve this via catalog files (actually I do not need the DTD), I 
do not have success.
Here are my mini examples:

Saxon HE 10.3 resides in the lib folder

.basex setting:
# Local Options
SERIALIZER = indent=no
DTD = true

XML in local folder "C:/temp/catalog":
http://www.blahblahblah.info/dtd/dokument.dtd;>

  01


catalog.xml in local folder "C:/temp/catalog":

  http://www.blahblahblah.info/dtd/; 
rewritePrefix="file:///C:/temp/catalog/dtd/"/>


dokument.dtd in local folder "C:/temp/catalog/dtd":



XQuery query.xq in local folder "C:/temp/catalog":
(# db:catfile catalog.xml #) {
  xslt:transform('dokument.xml', 'transform.xsl')
}


With or without pragma, this always results in a java.net.UnknownHostException 
(because the system ID is not available, that's true), but I would be expecting 
this would resolve to "file:///C:/temp/catalog/dtd/dokument.dtd"

Not working in GUI nor via CCL.

What am I getting wrong?

Thanks, Daniel

Re: [basex-talk] Date picture and xslt:transform()

2022-04-29 Thread Zimmel, Daniel

I get "Saxon HE", but that's what I already know ;-)

Yes it works with Saxon EE, I only wonder why it does not with Saxon HE.
I will ask over at Saxonica, already found a bug report that might be 
related... there is nothing in the documentation that says it should not work 
with HE as well.

Thanks!

-Ursprüngliche Nachricht-
Von: Christian Grün  
Gesendet: Freitag, 29. April 2022 10:29
An: Zimmel, Daniel 
Cc: basex-talk@mailman.uni-konstanz.de
Betreff: Re: [basex-talk] Date picture and xslt:transform()

Hi Daniel,

What do you get if you invoke xslt:processor() ?

If it’s "Saxon EE", you should get "29. März 2022" as result of 
your query (at least that’s what I get). If it’s something else, it indicates 
that Saxon EE has not correctly been embedded in your Java classpath (see [1] 
for further information).

If it’s only about formatting date, you can also run your function call within 
BaseX …

format-date(xs:date('2022-03-29'), '[D]. [MNn] [Y]', 'de', (), ())

… but I guess that’s what you already know.

Best,
Christian

[1] https://docs.basex.org/wiki/XSLT_Module

On Fri, Apr 29, 2022 at 10:05 AM Zimmel, Daniel  wrote:
>
> Hi,
>
> why do I get different results with the following two queries?
> xslt:transform() does not respect my date picture.
>
> Expected result:
>
> 29. März 2022
>
> Query 1:
>
> {format-date(xs:date('2022-03-29'), '[D]. [MNn] [Y]', 
> 'de', (), ())}
>
> Result:
> 29. März 2022
>
> Query 2:
>
> declare namespace xsl = 'http://www.w3.org/1999/XSL/Transform';
> let $xslt :=xmlns:xs="http://www.w3.org/2001/XMLSchema;
>   exclude-result-prefixes="xs">
>   
> 
>   
> 
>   
> 
> let $xml := 
>
> return
>   for $xml in $xml
>   return
> $xml => xslt:transform($xslt)
>
> Result:
> [Language: en]29. March 2022
>
>
> Running the XSLT with Saxon EE (not in BaseX via xslt:transform) returns 
> (correctly):
>
> 29. März 2022
>
> Using BaseX 9.5
>
> ?
>
> Daniel
>

Re: [basex-talk] Date picture and xslt:transform()

2022-04-29 Thread Zimmel, Daniel

... for clarification, I can reproduce the behavior with stand-alone Saxon HE 
10.6 (but not in PE and EE).
Perhaps this needs to be addressed to Saxon then, since the documentation says 
"available in all editions"... still happy if anybody does have some helpful 
insights here.

Daniel

-Ursprüngliche Nachricht-
Von: Zimmel, Daniel <> 
Gesendet: Freitag, 29. April 2022 10:05
An: basex-talk@mailman.uni-konstanz.de
Betreff: Date picture and xslt:transform()

Hi,

why do I get different results with the following two queries?
xslt:transform() does not respect my date picture.

Expected result: 

29. März 2022

Query 1:

{format-date(xs:date('2022-03-29'), '[D]. [MNn] [Y]', 'de', (), 
())}

Result: 
29. März 2022

Query 2:

declare namespace xsl = 'http://www.w3.org/1999/XSL/Transform';
let $xslt := http://www.w3.org/2001/XMLSchema; 
  exclude-result-prefixes="xs">
  

  



let $xml := 

return
  for $xml in $xml
  return
$xml => xslt:transform($xslt)

Result: 
[Language: en]29. March 2022


Running the XSLT with Saxon EE (not in BaseX via xslt:transform) returns 
(correctly): 

29. März 2022

Using BaseX 9.5

?

Daniel

[basex-talk] Date picture and xslt:transform()

2022-04-29 Thread Zimmel, Daniel

Hi,

why do I get different results with the following two queries?
xslt:transform() does not respect my date picture.

Expected result: 

29. März 2022

Query 1:

{format-date(xs:date('2022-03-29'), '[D]. [MNn] [Y]', 'de', (), 
())}

Result: 
29. März 2022

Query 2:

declare namespace xsl = 'http://www.w3.org/1999/XSL/Transform';
let $xslt := http://www.w3.org/2001/XMLSchema; 
  exclude-result-prefixes="xs">
  

  



let $xml := 

return
  for $xml in $xml
  return
$xml => xslt:transform($xslt)

Result: 
[Language: en]29. March 2022


Running the XSLT with Saxon EE (not in BaseX via xslt:transform) returns 
(correctly): 

29. März 2022

Using BaseX 9.5

?

Daniel

Re: [basex-talk] Interrupted queries - partial results?

2022-04-19 Thread Zimmel, Daniel

How about some custom logging?

for $i in (1 to 10)
return 
  if ($i = 4) then error()
  else file:append-text-lines('C:\tmp\file',string($i))

There might be more sophisticated ways.

Daniel

-Ursprüngliche Nachricht-
Von: BaseX-Talk  Im Auftrag von 
Patrick Durusau
Gesendet: Dienstag, 19. April 2022 16:45
An: basex-talk@mailman.uni-konstanz.de
Betreff: [basex-talk] Interrupted queries - partial results?

Greetings!

Is it possible to capture partial results from an interrupted query?

When I realize a query is taking too long for the data involved, it might be 
helpful in correcting the mistake.

Thanks!

Patrick

--
Patrick Durusau
patr...@durusau.net
Technical Advisory Board, OASIS (TAB)
Editor, OpenDocument Format TC (OASIS), Project Editor ISO/IEC 26300 Co-Editor, 
ISO/IEC 13250-1, 13250-5 (Topic Maps)

Another Word For It (blog): http://tm.durusau.net
Homepage: http://www.durusau.net
Twitter: patrickDurusau

Re: [basex-talk] Enclosing strings by double quotes for CSV data output

2022-04-06 Thread Zimmel, Daniel

If any CSV parser complains about the perfectly valid data in this example, I 
would definitely say it is poorly implemented, because it does not comply to 
the RFC. 
I took a glance at the mail thread and I could not find a hint where exactly 
you are having trouble with the quotes missing apart from saying that they are 
missing - is this a piece of software that cannot handle CSV properly? If so, I 
would definitely switch my parser.

Daniel

-Ursprüngliche Nachricht-
Von: BaseX-Talk  Im Auftrag von 
Markus Elfring
Gesendet: Mittwoch, 6. April 2022 17:17
An: Bridger Dyson-Smith 
Cc: BaseX 
Betreff: Re: [basex-talk] Enclosing strings by double quotes for CSV data output

> My opinion of these results is that they are correct - they meet my 
> expectations.

I find this view interesting.


> Do you feel like these results are missing something?

Yes.


> If so, what specifically?

I am missing text quoting for the fields “ID”, “T1”, “T2” and “T4”.


> Is there something in the documentation that could change to make things more 
> explicit or provide better clarity?

I got such an impression.

Regards,
Markus

Re: [basex-talk] How to return/use the value of a nested counter?

2022-03-09 Thread Zimmel, Daniel

Are you simply counting the wrong items?

It seems to me you wanted to count:
 
for $Beurt at $CountInner in $Debat//spreekbeurt

Daniel

-Ursprüngliche Nachricht-
Von: BaseX-Talk  Im Auftrag von Ben 
Engbers
Gesendet: Mittwoch, 9. März 2022 13:11
An: Basex Mail-lijst 
Betreff: [basex-talk] How to return/use the value of a nested counter?

Hi,

I have a collection of 740 documents with the following structure:


http://www.w3.org/2001/XMLSchema-instance; 
xsi:noNamespaceSchemaLocation="http://technische-documentatie.oep.overheid.nl/schema/op-xsd-2012-1;>
   
 https://zoek.officielebekendmakingen.nl/h-tk-20202021-102-2/metadata.xml;
 
/>
   
   
 
   
 
   
 Allereerst hebben we het traditionele mondelinge vragenuur. 

   
 
   
   
 
   
 Voorzitter. Het was altijd al een eer om hier te staan.
   
   
 De vragen die ik ga stellen, gaan over stikstof.
   
   
 We zijn allemaal 100 kilometer per uur gaan rijden, maar er is 
nog geen gram ammoniak uit de veehouderij minder uitgestoten.
   
 
   
   
   
 
   
 U heeft helaas maar één vraag, meneer Ephraim, als Groep Van 
Haga.
   
   
 Ik wil de minister bedanken voor haar beantwoording.
   
 
   
 
   


I am trying to concatenate all the // childs from  elements. 
Together with an ID that I construct from //meta/@content and a counter for 
, I want this output:

20202021-102-2, 1, Allereerst ...
20202021-102-2, 2, Voorzitter... + De vragen ... + We zijn ...
20202021-102-2, 3, U heeft ... + Ik wil..

I expected that the following XQuery-statemnt would do it.

import module namespace  functx = "http://www.functx.com;;

for $Debat at $CountOuter in collection("Parliament")
 (: where $CountOuter <= 3:)
   let $debate-id := fn:analyze-string(
 $Debat/officiele-publicatie/metadata/meta/@content,
"(\d{8}-\d*-\d*)")//fn:match/*:group[@nr="1"]/text()
 order by $debate-id
 for $Beurt at $CountInner in $Debat
let $tekst := $Beurt//spreekbeurt//al/text()
 return($debate-id, $CountInner, $tekst)

Instead it returns:
20202021-102-2, 1, Allereerst ...+ Voorzitter... + De vragen ... + We zijn ... 
+ U heeft ... + Ik wil..

How can I use the value of $CountInner?

Ben

Re: [basex-talk] validate:xsd-info agains more than one schema

2022-03-04 Thread Zimmel, Daniel

Why not pass both schemas as a string? You could then tokenize it and get two 
validation reports.
Or do you want one validation report for two schemas at once? I am not sure if 
that’s possible...

let $schema:= 
"http://inspire.ec.europa.eu/schemas/bu-base/4.0/BuildingsBase.xsd 
http://schemas.opengis.net/gml/3.2.1/gml.xsd;
let $schemas := tokenize($schema,'\s')
for $sch in $schemas
return validate:xsd-info($xml,$sch)

Daniel

Von: BaseX-Talk  Im Auftrag von 
Arantza Etxebarria Pablo
Gesendet: Freitag, 4. März 2022 10:29
An: basex-talk@mailman.uni-konstanz.de
Betreff: [basex-talk] validate:xsd-info agains more than one schema


Dear Community,


We have been trying to validate a file using the function  
validate:xsd-info($input as item(), $schema as item()?) as xs:string*   against 
several different schemas and, we don't know how to send the parameter “schema” 
with more than one schema for this function.


We have tried to declare the variable schema like this :


let $schema:= 
"http://inspire.ec.europa.eu/schemas/bu-base/4.0/BuildingsBase.xsd 
http://schemas.opengis.net/gml/3.2.1/gml.xsd;


But it returns this error message:

XML document 'example.gml': -1:-1: schema_reference.4: Failed to read schema 
document 'http://inspire.ec.europa.eu/schemas/bu-base/4.0/BuildingsBase.xsd 
http://schemas.opengis.net/gml/3.2.1/gml.xsd' , because 1) could not find the 
document; 2) the document could not be read; 3) the root element of the 
document is not .



I have tried this function when no schema is given and it takes the contain of 
schemaLocation attribute, and in this case, is able to validate more than one 
schema. But I need to pass the schemas through a variable


Could any of you point us in the right direction?


Thanks

--
Arantza Etxebarria

Re: [basex-talk] string-join with a newline separator?

2022-02-24 Thread Zimmel, Daniel

> Ok, at least in the GUI using  as separator works.
> Is this a HTML-specific separator?

no, this is the decimal notation of a line feed in UTF-8.
Not sure what you environment expects your encoding, you might want to try 
carriage return instead ()

Line endings can be fun.

Best Daniel

Re: [basex-talk] string-join with a newline separator?

2022-02-24 Thread Zimmel, Daniel

or, if you prefer, you could even use a function for this, see 
https://docs.basex.org/wiki/Output_Module

out:nl()

Von: BaseX-Talk  Im Auftrag von 
Bridger Dyson-Smith
Gesendet: Donnerstag, 24. Februar 2022 16:57
An: ben.engb...@be-logical.nl
Cc: Basex Mail-lijst 
Betreff: Re: [basex-talk] string-join with a newline separator?

Hi Ben,

On Thu, Feb 24, 2022 at 10:44 AM Ben Engbers 
mailto:ben.engb...@be-logical.nl>> wrote:
Hi,

My xml has the structure

   
 
   bla
 
   
   
 
   bla
 
 
   bla
 
   


The  element contains 1 to many  elements.

let $tekst := fn:string-join(fn:data($par//al/text()), ".") concatenates
this to:
bla.bla.bla
But I want it to return:
bla
bla
bla

Is it possible to add a newline item-separator to fn:string-join?
how about string-join($input//al/text(), '')?

Ben Engbers
Best,
Bridger

Re: [basex-talk] Faster in the cloud?

2022-02-22 Thread Zimmel, Daniel

Wait – my tea kettle is not yet prepared for 3 seconds! When shall I do my 
breaks?

Can’t wait to try this in my updating pipelines.
It is always intriguing to learn from this list about performance 
optimizations, be it in the implementation or giving example designs, thanks!

Daniel

Von: BaseX-Talk  Im Auftrag von 
Christian Grün
Gesendet: Dienstag, 22. Februar 2022 14:57
An: ETANCHAUD Fabrice 
Cc: BaseX 
Betreff: Re: [basex-talk] Faster in the cloud?

A little announcement: With BaseX 10 [1], main memory updates will get much 
faster:

{
  (1 to 100) ! 
} update {
  y ! (insert node  into .)
}

BaseX 9: ages (6-7 minutes)
BaseX 10: 3 seconds

The reason: The disk-based block storage layout is now also used for the main 
memory representation of XML nodes.

[1] https://files.basex.org/releases/latest-10/

On Tue, Feb 22, 2022 at 9:49 AM ETANCHAUD Fabrice 
mailto:fabrice.etanch...@maif.fr>> wrote:
Hi Jonathan !

Apologizes for my late contribution...

Do you really have to use XQuery Update ? Do you have to stick to a specific 
format ?
If not, maybe you could use a schema on read approach ?
I mean, you could add new data as new documents,
and recombine these documents into the attribute based format when requesting 
the data.

Would that be a viable solution for you ?

I once had success with this solution, as BaseX is very quick at adding 
documents.

Best regards,
Fabrice

De : BaseX-Talk 
mailto:basex-talk-boun...@mailman.uni-konstanz.de>>
 de la part de Eliot Kimber 
mailto:eliot.kim...@servicenow.com>>
Envoyé : lundi 21 février 2022 18:06
À : BaseX 
mailto:basex-talk@mailman.uni-konstanz.de>>
Objet : Re: [basex-talk] Faster in the cloud?

You can use prof:track() to time your insertion operation for enough iterations 
to get a reasonable time and then multiply by 2.5 million to get an approximate 
time to completion.

On my machine I’m finding times around 0.05 seconds for my operations, which 
are more than just attribute insertions, where I need to do 40K iterations. I 
would expect attribute insertion to be faster, especially if you can batch up 
the insertions into a small number of transactions.

But five hours to do the update doesn’t seem entirely out of spec if your 
machine is significantly slower. Doing the math, I get 7ms per insertion:

Hours

Seconds/ Hour

Seconds

# operations

Time/operation

5

3600

18000

250

0.0072

That seems pretty fast on a per-operation standpoint.

If you can break your content into multiple databases you could parallelize the 
updates across multiple BaseX instances and then combine the result back at the 
end.

So spin up one server for each core, have a master server that provides a REST 
API to kick off the processing and then use the REST method to farm jobs out to 
each of the servers (using REST to make it easy to target each of the servers 
via a port. Could also do it from a shell script through the baseclient 
command-line.).

With that should be able to reduce the processing to the time it takes one 
server to process its share, which will be total objects/number of cores (its 
share, that is).

Cheers,

E.

_

Eliot Kimber

Sr Staff Content Engineer

O: 512 554 9368

M: 512 554 9368

servicenow.com

LinkedIn | 
Twitter | 
YouTube | 
Facebook

From: BaseX-Talk 
mailto:basex-talk-boun...@mailman.uni-konstanz.de>>
 on behalf of Jonathan Robie 
mailto:jonathan.ro...@gmail.com>>
Date: Monday, February 21, 2022 at 8:44 AM
To: Liam R. E. Quin mailto:l...@fromoldbooks.org>>
Cc: BaseX 
mailto:basex-talk@mailman.uni-konstanz.de>>
Subject: Re: [basex-talk] Faster in the cloud?

[External Email]

I have a 2013 Macbook Pro with 16 Gig RAM and a 1 Terabyte SSD.  So not 
entirely wimpy, but nowhere near as fast as the current Macbooks, I have no 
idea how that compares to a typical laptop these days.  Most things run fairly 
quickly, but inserting 2.5 million attributes into a document takes perhaps 5 
hours, I didn't time it.  I can run that overnight, and do test runs on smaller 
subsets, but I want to think through my options.

Jonathan

On Sat, Feb 19, 2022 at 6:11 PM Liam R. E. Quin 
mailto:l...@fromoldbooks.org>> wrote:

On Sat, 2022-02-19 at 16:05 -0500, Jonathan Robie wrote:
> If I am running my queries and updates on a typical laptop, would
> they run much faster if I ran them on a suitably configured instance
> in the cloud?

"suitably configured" is very subjective.  Potentially your queries
could run a lot faster.

A lot depends on the speed of the disk (or SSD) in the laptop, and the
amount of memory it has, as well as the CPU - a recent Macbook Pro will
be faster than a ten-year-old chromebook.  However, server blades (the
machines used

Re: [basex-talk] Questions about the performance of bulk string matching

2022-01-18 Thread Zimmel, Daniel

Catching up late, but thank you for your thoughts.
I have grown so fond of using maps in XQuery, I use them even with tiny data 
sets now all the time.
So I have learned that In combination with a large amount of large search 
strings they can speed up things a lot where a regular expression can only go 
so far.

Thanks!

-Ursprüngliche Nachricht-
Von: Christian Grün  
Gesendet: Freitag, 7. Januar 2022 18:49
An: Zimmel, Daniel 
Cc: BaseX 
Betreff: Re: [basex-talk] Questions about the performance of bulk string 
matching

Hi Daniel,

Thanks for your code.

My reply is a fairly general one and not specific to XQuery: Regular 
expressions are very powerful, but they can get fairly expensive, so it’s 
always a good idea to simplify or optimize them before running them on large 
datasets. Here is an example for a regex that looks fairly simple, but takes 
many seconds to execute:

matches(
  string-join((1 to 10) ! 'x'),
  '.*y'
)

This expression can e.g. be sped a lot by using '.*y' as replace string.

BaseX passes on most regex strings to Java without further changes. If an 
expression takes a long time to execute, chances are high that it’s due to the 
JDK.

Hope this helps,
Christian



On Tue, Jan 4, 2022 at 3:50 PM Zimmel, Daniel  wrote:
>
> Hi Christian,
>
>
>
> thanks for your interest, please find attached a complete example. It is all 
> about the length of strings in my dictionary, really.
>
>
>
> test-reduce.xq: 170.82ms
>
> test.xq: 6572.49ms
>
>
>
> I noticed that if the data is larger, performance-wise it is better to build 
> a comparison map for each textnode (not as in the example).
>
> On the other hand, If the strings are short, performance might even be worse.
>
>
>
> But with my use case, I have to search for complete strings like “Erste 
> Durchführungsverordnung zur Verordnung über Luftfahrtpersonal 
> (Anwendungsbestimmungen zur Lizenzierung von Piloten Flugzeuge, von Piloten 
> Hubschrauber, von Flugingenieuren, Freiballonführern, Flugdienstberatern und 
> Flugtechnikern auf Hubschraubern bei den Polizeien des Bundes und der 
> Länder)“ – this does not work too well with bulk matches/replace.
>
>
>
> So while I am quite happy with the results, I do not understand why 
> this works so fast – or if there is any feasible alternative to this 
> approach (I got the feeling that there is still much to learn for me 
> in the mysteries of XQuery ;-)
>
> Does this work because BaseX simply handles this well? Could match/replace in 
> some other way be optimized to handle bulk search including long strings?
>
>
>
> Daniel
>
>
>
> Von: Christian Grün 
> Gesendet: Donnerstag, 30. Dezember 2021 20:15
> An: Zimmel, Daniel 
> Cc: BaseX 
> Betreff: Re: [basex-talk] Questions about the performance of bulk 
> string matching
>
>
>
> Hi Daniel,
>
>
>
> Thanks for presenting your interesting use case.
>
>
>
> Your solution looks clever. Would you mind sharing some sample data with us 
> (an XML document and a string dictionary)?
>
>
>
> Cheers,
>
> Christian
>
>
>
>
>
> Zimmel, Daniel  schrieb am Mi., 29. Dez. 2021, 12:13:
>
> Hi,
>
> recently I ran into serious (as in SERIOUS) performance trouble regarding 
> expensive regexes, and no wonder why.
>
> Here is my scenario:
>
> * XML text documents with a total of 1m text nodes
> * A regex search string, including a huge string dictionary list of 
> 50.000 strings (some of them containing 50 words each)
> * a match must be wrapped in an element (= create a link)
>
> I could drink many cups of tea while waiting for the regex to complete... 
> hours... I ran out of tea!
>
> Then I found the 2021 Balisage paper from Mary Holstege titled "Fast 
> Bulk String Matching" [1] in which she explores the Aho-Corasick 
> algorithm, implementing it with XQuery - marvellous! Following this, 
> while I can build a data structure which gives me fast results, 
> building the same structure is still very slow due to the amount of 
> text to build from. So this was not fast enough for my use case - or I 
> may simply not be smart enough to apply it correctly :-|
>
> So, I tried tinkering with maps which turned out to give me extraordinary 
> performance gains:
>
> * build a map from the string dictionary
> * walk through all text nodes one by one
> * for each text node, put any combination of words in the text node in 
> word order (I need to find strings, not words) into another map
> * strip punctuation (for word boundaries) and do some normalization of 
> whitespaces in both maps
> * compare the keys of both maps
> * give the new reduced string dictionary to the regular regex search
>
> While comparing the maps, I

[basex-talk] Questions about the performance of bulk string matching

2021-12-29 Thread Zimmel, Daniel

Hi,

recently I ran into serious (as in SERIOUS) performance trouble regarding 
expensive regexes, and no wonder why.

Here is my scenario:

* XML text documents with a total of 1m text nodes
* A regex search string, including a huge string dictionary list of 50.000 
strings (some of them containing 50 words each)
* a match must be wrapped in an element (= create a link)

I could drink many cups of tea while waiting for the regex to complete... 
hours... I ran out of tea!

Then I found the 2021 Balisage paper from Mary Holstege titled "Fast Bulk 
String Matching" [1] in which she explores the Aho-Corasick algorithm, 
implementing it with XQuery - marvellous! Following this, while I can build a 
data structure which gives me fast results, building the same structure is 
still very slow due to the amount of text to build from. So this was not fast 
enough for my use case - or I may simply not be smart enough to apply it 
correctly :-|

So, I tried tinkering with maps which turned out to give me extraordinary 
performance gains:

* build a map from the string dictionary
* walk through all text nodes one by one
* for each text node, put any combination of words in the text node in word 
order (I need to find strings, not words) into another map
* strip punctuation (for word boundaries) and do some normalization of 
whitespaces in both maps
* compare the keys of both maps
* give the new reduced string dictionary to the regular regex search

While comparing the maps, I do not know where in the text my strings are, but 
at least I know if they are in there - to find out where exactly (and how do 
they fit my other regex context) I can then use a massively reduced regular 
regex search. Fast!

I am quite happy BUT I still cannot understand why this is so much faster for 
my sample data:

* plain method : 51323.72ms
* maps method: 597.94ms(!)

Is this due to any optimizations done by BaseX or have I simply discovered the 
power of maps?
How would you do it? Is there still room for improvement? 
Why does Aho-Corasick not help much with this scenario? Is it because the list 
of strings is simply too massive?
Why is this so much faster with text-splitting-to-map?

See below for the query examples to better understand what I am trying to do 
(bulk data not included) [2],[3]
There is no normalization of punctuation in the examples but that is only 
necessary for completeness.

Best, Daniel

[1] 
http://www.balisage.net/Proceedings/vol26/print/Holstege01/BalisageVol26-Holstege01.html

[2] plain method

let $textnodes := 
fetch:xml(file:base-dir()||file:dir-separator()||'example.xml')//text()
let $strings := 
file:read-text(file:base-dir()||file:dir-separator()||'strings.txt')
let $regex := $strings
for $textnode in $textnodes
where  matches($textnode,$regex) = true()
return $textnode

[3] maps method

(:~
 : Create map from string
 : Tokenization
 :
 : @param $strings
 : @return Map
 :)
declare function local:createMapFromString (
  $string as xs:string
) as map(*) {
  let $map_words :=  
  map:merge(
  for $string in tokenize($string,'\|')
  let $key := $string
  let $val := $string
  return map:entry($key,$val),
map { 'duplicates': 'use-first' })
return
  $map_words
};

(:~
 : Create map from text nodes
 : Write any combination of words in document order to the map
 :
 : @param $textnodes
 : @return Map
 :)
declare function local:createMapFromTextnodes (
  $textnodes as xs:string+
) as map(*) {
  map:merge(
  for $node in $textnodes
  let $text := normalize-space($node)
  let $tokens := tokenize($text,' ')
  let $map_nodes :=
map:merge(
 for $start in 1 to fn:count($tokens)   
  for $length in 1 to fn:count($tokens) - $start + 1
 return
   map:entry(fn:string-join(fn:subsequence($tokens, $start, 
$length), ' '),'x')
 )

  return
$map_nodes)
};

(:~
 : Compare two maps
 :
 : @param $map1
 : @param $map2
 : @return xs:string*
 :)
declare function local:reduceMaps (
  $map1 as map(*),
  $map2 as map(*)
) as xs:string* {
  for $key in map:keys($map1)
  where map:contains($map2,$key)
  let $value := map:get($map1,$key)
  return $value
};

let $textnodes := 
fetch:xml(file:base-dir()||file:dir-separator()||'example.xml')//text()
let $strings := 
file:read-text(file:base-dir()||file:dir-separator()||'strings.txt')

let $map_words := local:createMapFromString($strings)
let $map_textnodes := local:createMapFromTextnodes($textnodes)
let $matches :=  local:reduceMaps($map_words,$map_textnodes)

let $regex := string-join(for $match in $matches 
  group by $match 
  return $match,'|')

for $textnode in $textnodes
where  matches($textnode,$regex) = true()
return $textnode

Re: [basex-talk] Oxygen integration limitations?

2021-12-01 Thread Zimmel, Daniel

Thanks Christian for the confirmation of this behaviour and the useful hints.
I have not tried the IntelliJ plugin but I will! Also interesting feature on 
Linux desktops, I have to try this.

Best Daniel

Von: Christian Grün 
Gesendet: Mittwoch, 1. Dezember 2021 13:14
An: Zimmel, Daniel 
Cc: basex-talk@mailman.uni-konstanz.de
Betreff: Re: [basex-talk] Oxygen integration limitations?

Hi Daniel,

According to the wiki, there is no way to get 3.1 support in Oxygen because of 
the limitation of the XQJ interface?

Unfortunately, the XQJ API license is restricted to XQuery 1.0, so we’ll need 
to live with it (until someone feels a vocation to write a custom BaseX/oXygen 
plugin?).

However, Reece’s IntelliJ plugin is very up-to-date and supports all 
version-specific varieties and gimmicks of BaseX [1]. Did you give it a try?

> for its features or even for a silly reason like "hey why is there no dark 
> mode in BaseX-GUI I need a dark mode now because I have eye strain!"

Perfectly understandable ;) On Windows, I like to use NegativeScreen [2], which 
enables you to define custom conversion matrixes. BaseX additionally provides 
support for extra Look and Feel interfaces, including dark ones [3]. On some 
Linux systems, BaseX will switch to dark mode if a global dark environment is 
chosen.

Hope this helps
Christian

[1] https://docs.basex.org/wiki/Integrating_IntelliJ_IDEA
[2] https://zerowidthjoiner.net/negativescreen
[3] https://docs.basex.org/wiki/Graphical_User_Interface#Look_and_Feel

[basex-talk] Oxygen integration limitations?

2021-12-01 Thread Zimmel, Daniel

 Hi,

while the BaseX GUI is really great, there are times I want to use Oxygen (for 
its features or even for a silly reason like "hey why is there no dark mode in 
BaseX-GUI I need a dark mode now because I have eye strain!").

According to the wiki, there is no way to get 3.1 support in Oxygen because of 
the limitation of the XQJ interface?
So when I add a scenario with BaseX as transformer I can get Oxygen to respect 
any BaseX modules (and care for adding else instructions even if they are 
empty), but in the end it will always give me "invalid syntax, does not pass 
static validation".

Or is there some button I have been missing?

Thanks, Daniel

Re: [basex-talk] Running XQuery Update via apply Ant task

2021-05-26 Thread Zimmel, Daniel

hmm to answer myself, seems I can also change the query to prepare for multiple 
document-nodes input, something like this:

for $node in /*
return
for $f in $node
return insert node B as first into $f

With writeback enabled, all the files get updated from the commandline. Is this 
the way to go then?


-Ursprüngliche Nachricht-
Von: BaseX-Talk  Im Auftrag von 
Zimmel, Daniel
Gesendet: Mittwoch, 26. Mai 2021 17:31
An: basex-talk@mailman.uni-konstanz.de
Betreff: [basex-talk] Running XQuery Update via apply Ant task

Hi,

is there anyone using BaseX in an Apache Ant scenario?

I have the following target for XQuery Updating some files (no database 
access), but I am too dumb to make an apply task that reads every file in a 
fileset.
Usually you place the  where you need it to be in your external 
command, but unfortunately the BaseX command-line syntax won't allow this to be 
easily placed: the filename is supposed to follow directly suffixed to "-i" 
where I cannot use an element, of course.

  

 
   
   
  
  
  
  
  
  
   
  

  

As a workaround, I set a dummy.xml for input and bind the current folder as a 
parameter and in my Query I can use file:list() which then is updating all 
files in the directory, but this makes the Query dependent on the parameter, 
and updates are obviously not written back unless all files have been processed 
in memory. I would prefer to have a standalone query operating only on the 
current document-uri (so I can better reuse it elsewhere).

Can someone give me a hint if it is possible in plain Ant to apply a CCL-BaseX 
to any file in a given fileset?
Other than that, BaseX performs well as a file processor (if you ignore the 
time related to file handling).

Thanks, Daniel

[basex-talk] Running XQuery Update via apply Ant task

2021-05-26 Thread Zimmel, Daniel

Hi,

is there anyone using BaseX in an Apache Ant scenario?

I have the following target for XQuery Updating some files (no database 
access), but I am too dumb to make an apply task that reads every file in a 
fileset.
Usually you place the  where you need it to be in your external 
command, but unfortunately the BaseX command-line syntax won't allow this to be 
easily placed: the filename is supposed to follow directly suffixed to "-i" 
where I cannot use an element, of course.

  

 
   
   
  
  
  
  
  
  
   
  

  

As a workaround, I set a dummy.xml for input and bind the current folder as a 
parameter and in my Query I can use file:list() which then is updating all 
files in the directory, but this makes the Query dependent on the parameter, 
and updates are obviously not written back unless all files have been processed 
in memory. I would prefer to have a standalone query operating only on the 
current document-uri (so I can better reuse it elsewhere).

Can someone give me a hint if it is possible in plain Ant to apply a CCL-BaseX 
to any file in a given fileset?
Other than that, BaseX performs well as a file processor (if you ignore the 
time related to file handling).

Thanks, Daniel

Re: [basex-talk] Clarification on registering a job service

2021-05-06 Thread Zimmel, Daniel

Thank you for the insight on how this works: 
so you cannot update a scheduled job, but you can update the 
jobs-to-be-scheduled.

Helps a lot, 
Daniel

-Ursprüngliche Nachricht-
Von: Christian Grün  
Gesendet: Donnerstag, 6. Mai 2021 13:52
An: Zimmel, Daniel 
Cc: basex-talk@mailman.uni-konstanz.de
Betreff: Re: [basex-talk] Clarification on registering a job service

Hi Daniel,

I have decided to remove the misleading information in the documentation as it 
would probably be confusing to explain the full
background:

• If a job is currently registered for scheduled execution, a new job with the 
same name will be rejected.
• If the job has been registered as service, but if it’s not currently 
scheduled (e.g. because the job will only be run once at startup time), it can 
be replaced with a new job.

You can stop scheduled jobs as follows (and register new ones after that):

for $job in jobs:list-details()
where $job/@state = 'scheduled'
return jobs:stop($job/@id)

Hope this helps
Christian


On Thu, May 6, 2021 at 11:12 AM Zimmel, Daniel  wrote:
>
> Hi,
>
> can someone clarify on the following behaviour?
>
> The docs say:
>
> "Job services can be updated: If a new job is registered, and if 
> there is already a job with the same id, the old entry will be replaced."
> https://docs.basex.org/wiki/Jobs_Module#Services
>
> So when I try to update an existing job service with the following query, I 
> get an unexpected error "Job id already exists: myquery.":
>
> jobs:eval(
>   xs:anyURI('myquery.xq'),
>   (),
>   map { 'id':'myquery', 'start':'19:00:00', 'interval':'P1D', 
> 'service': true() }
>   )
>
> Am I reading the documentation wrong?
>
> Thanks Daniel
>

[basex-talk] Clarification on registering a job service

2021-05-06 Thread Zimmel, Daniel

Hi,

can someone clarify on the following behaviour?

The docs say:

"Job services can be updated: If a new job is registered, and if there 
is already a job with the same id, the old entry will be replaced."
https://docs.basex.org/wiki/Jobs_Module#Services

So when I try to update an existing job service with the following query, I get 
an unexpected error "Job id already exists: myquery.":

jobs:eval(
  xs:anyURI('myquery.xq'),
  (),
  map { 'id':'myquery', 'start':'19:00:00', 'interval':'P1D', 
'service': true() }
  )

Am I reading the documentation wrong?

Thanks Daniel

Re: [basex-talk] BaseX and Oxygen scenario

2021-04-22 Thread Zimmel, Daniel

Thanks Christian, this led me to reconfigure my connection - this helps, and 
works!
Revisiting the documentation, I think the Wiki description is wrong:

Under "Driver class", choose the preferred driver class:

Embedded (standalone): net.xqj.basex.BaseXXQDataSource
Client/server: net.xqj.basex.local.BaseXXQDataSource

On my end, it definitely works the other way round: standalone = 
net.xqj.basex.local.BaseXXQDataSource

I am well aware of the limitations (so long XQuery 3.0), but being able to 
access the module functions in an Oxygen setting is quite useful indeed.
Now I can make a comfortable button for an XML editor to produce a PDF report 
with Levenshtein distance markings. BaseX modules are fun.

Thanks, Daniel

-Ursprüngliche Nachricht-
Von: Christian Grün  
Gesendet: Donnerstag, 22. April 2021 12:16
An: Zimmel, Daniel 
Cc: basex-talk@mailman.uni-konstanz.de
Betreff: Re: [basex-talk] BaseX and Oxygen scenario

Hi Daniel,

The “embedded” solution, which is mentioned in the documentation, refers to 
BaseX as standalone instance. I might change the wording to make this clearer.

Hope this helps,
Christian

On Wed, Apr 21, 2021 at 11:29 AM Zimmel, Daniel  wrote:
>
> Hi,
>
> am I right in assuming that in order to use the XQJ interface as described 
> here (https://docs.basex.org/wiki/Integrating_oXygen) , you  will always need 
> a running local database instance of BaseX?
> I am not able to use the BaseX.jar standalone in a transformation scenario as 
> I would for example do in a custom ant scenario where I can use BaseX as a 
> standalone transformer (using it as a commandline process).
>
> Thanks, Daniel
>

[basex-talk] BaseX and Oxygen scenario

2021-04-21 Thread Zimmel, Daniel

Hi,

am I right in assuming that in order to use the XQJ interface as described here 
(https://docs.basex.org/wiki/Integrating_oXygen) , you  will always need a 
running local database instance of BaseX?
I am not able to use the BaseX.jar standalone in a transformation scenario as I 
would for example do in a custom ant scenario where I can use BaseX as a 
standalone transformer (using it as a commandline process).

Thanks, Daniel

Re: [basex-talk] A question about RESTXQ rest:form-param()

2020-04-23 Thread Zimmel, Daniel

hmm, what is wrong with this stripped example?

 form.xqm:

module namespace page = 'http://basex.org/examples/web-page';

declare
  %rest:path("/esv/fun/form")
  %rest:single

function page:form() {

  let $body :=
  

  
  
  

 
  return (web:response-header(map {'method':'xhtml'}),
  $body)
};

 test-form.xqm:

module namespace esv = 'esv/queries';

declare
  %rest:POST("{$body}")
  %rest:form-param("key", "{$key}")
  %rest:path("/esv/test-form")
  %rest:consumes("application/x-www-form-urlencoded")
  %rest:single
  %output:method("text")
function esv:test-form(
  $body as xs:string,
  $key as xs:string
)  {
  "parameter-names(): " || request:parameter-names(),
  "$body: " || $body,
  "$key: " || $key
};

 output:
parameter-names():  $body: foo=bar=key $key: key


Von: Marco Lettere 
Gesendet: Donnerstag, 23. April 2020 10:40
An: Zimmel, Daniel ; basex-talk@mailman.uni-konstanz.de
Betreff: Re: AW: AW: [basex-talk] A question about RESTXQ rest:form-param()

I set up this restxq [1].
And call it with something like [2].
Got an output of [3] which looks like what I expect.
M.

[1]
module namespace t = "urn:test";

declare
  %rest:path("test")
  %rest:POST
  %rest:consumes("application/x-www-form-urlencoded")
  %output:method("html")
function t:test() {
 
   
 {request:parameter-names() ! (. || "=" || request:parameter(.))}
   
 
};

[2]
curl --location --request POST 'http://localhost:8984/test' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'param1=Hello' \
--data-urlencode 'param2=World'

[3]

  
param1=Hello param2=World
  


On 22/04/20 16:03, Zimmel, Daniel wrote:

the mime type seems correct, all params are being sent. I do not have problems 
getting the body, only request:parameter-names() does not give any values. 
(request:header works, for example)



-Ursprüngliche Nachricht-

Von: Marco Lettere <mailto:m.lett...@gmail.com>

Gesendet: Mittwoch, 22. April 2020 15:31

An: Zimmel, Daniel <mailto:d.zim...@esvmedien.de>; 
basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de>

Betreff: Re: AW: [basex-talk] A question about RESTXQ rest:form-param()



Are you setting the mime type to "application/x-www-form-urlencoded"?

This is the prerequisite in order to get the body of a post parsed to form 
parameters.

M.



On 22/04/20 15:22, Zimmel, Daniel wrote:

Thanks for the hint Marco,



I just tried request:parameter-names(): that one works with GET but with POST I 
get an empty-sequence, I am not sure what the problem is here.

I can access the full (unparsed) body, though (I can use that).



Daniel



-Ursprüngliche Nachricht-

Von: Marco Lettere <mailto:m.lett...@gmail.com>

Gesendet: Mittwoch, 22. April 2020 14:24

An: 
basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de>

Betreff: Re: [basex-talk] A question about RESTXQ rest:form-param()



Hi Daniel,

another way is to use request:* functions [1] available for programmatic 
inspection of the request.

Cheers,

M.



[1] https://docs.basex.org/wiki/Request_Module





On 22/04/20 12:18, Zimmel, Daniel wrote:

Hi Daniel,



RTFM. Simply pass the body in %rest:POST("{$body}").



Sorry for disturbing!



Daniel



-Ursprüngliche Nachricht-

Von: Zimmel, Daniel <mailto:d.zim...@esvmedien.de>

Gesendet: Mittwoch, 22. April 2020 11:27

An: BaseX 
<mailto:basex-talk@mailman.uni-konstanz.de>

Betreff: [basex-talk] A question about RESTXQ rest:form-param()



Hi,



in an HTML form with a RESTXQ POST action, do I really have to declare 
%rest:form-param() for every param, or is there a way to pass the full query 
component in one parameter?



Best, Daniel

Re: [basex-talk] A question about RESTXQ rest:form-param()

2020-04-22 Thread Zimmel, Daniel

the mime type seems correct, all params are being sent. I do not have problems 
getting the body, only request:parameter-names() does not give any values. 
(request:header works, for example)

-Ursprüngliche Nachricht-
Von: Marco Lettere  
Gesendet: Mittwoch, 22. April 2020 15:31
An: Zimmel, Daniel ; basex-talk@mailman.uni-konstanz.de
Betreff: Re: AW: [basex-talk] A question about RESTXQ rest:form-param()

Are you setting the mime type to "application/x-www-form-urlencoded"?
This is the prerequisite in order to get the body of a post parsed to form 
parameters.
M.

On 22/04/20 15:22, Zimmel, Daniel wrote:
> Thanks for the hint Marco,
>
> I just tried request:parameter-names(): that one works with GET but with POST 
> I get an empty-sequence, I am not sure what the problem is here.
> I can access the full (unparsed) body, though (I can use that).
>
> Daniel
>
> -Ursprüngliche Nachricht-
> Von: Marco Lettere 
> Gesendet: Mittwoch, 22. April 2020 14:24
> An: basex-talk@mailman.uni-konstanz.de
> Betreff: Re: [basex-talk] A question about RESTXQ rest:form-param()
>
> Hi Daniel,
> another way is to use request:* functions [1] available for programmatic 
> inspection of the request.
> Cheers,
> M.
>
> [1] https://docs.basex.org/wiki/Request_Module
>
>
> On 22/04/20 12:18, Zimmel, Daniel wrote:
>> Hi Daniel,
>>
>> RTFM. Simply pass the body in %rest:POST("{$body}").
>>
>> Sorry for disturbing!
>>
>> Daniel
>>
>> -Ursprüngliche Nachricht-
>> Von: Zimmel, Daniel 
>> Gesendet: Mittwoch, 22. April 2020 11:27
>> An: BaseX 
>> Betreff: [basex-talk] A question about RESTXQ rest:form-param()
>>
>> Hi,
>>
>> in an HTML form with a RESTXQ POST action, do I really have to declare 
>> %rest:form-param() for every param, or is there a way to pass the full query 
>> component in one parameter?
>>
>> Best, Daniel
>>

Re: [basex-talk] A question about RESTXQ rest:form-param()

2020-04-22 Thread Zimmel, Daniel

Thanks for the hint Marco,

I just tried request:parameter-names(): that one works with GET but with POST I 
get an empty-sequence, I am not sure what the problem is here.
I can access the full (unparsed) body, though (I can use that).

Daniel

-Ursprüngliche Nachricht-
Von: Marco Lettere  
Gesendet: Mittwoch, 22. April 2020 14:24
An: basex-talk@mailman.uni-konstanz.de
Betreff: Re: [basex-talk] A question about RESTXQ rest:form-param()

Hi Daniel,
another way is to use request:* functions [1] available for programmatic 
inspection of the request.
Cheers,
M.

[1] https://docs.basex.org/wiki/Request_Module

On 22/04/20 12:18, Zimmel, Daniel wrote:
> Hi Daniel,
>
> RTFM. Simply pass the body in %rest:POST("{$body}").
>
> Sorry for disturbing!
>
> Daniel
>
> -Ursprüngliche Nachricht-
> Von: Zimmel, Daniel 
> Gesendet: Mittwoch, 22. April 2020 11:27
> An: BaseX 
> Betreff: [basex-talk] A question about RESTXQ rest:form-param()
>
> Hi,
>
> in an HTML form with a RESTXQ POST action, do I really have to declare 
> %rest:form-param() for every param, or is there a way to pass the full query 
> component in one parameter?
>
> Best, Daniel
>

Re: [basex-talk] A question about RESTXQ rest:form-param()

2020-04-22 Thread Zimmel, Daniel

Hi Daniel,

RTFM. Simply pass the body in %rest:POST("{$body}").

Sorry for disturbing!

Daniel

-Ursprüngliche Nachricht-
Von: Zimmel, Daniel  
Gesendet: Mittwoch, 22. April 2020 11:27
An: BaseX 
Betreff: [basex-talk] A question about RESTXQ rest:form-param()

Hi,

in an HTML form with a RESTXQ POST action, do I really have to declare 
%rest:form-param() for every param, or is there a way to pass the full query 
component in one parameter?

Best, Daniel

[basex-talk] A question about RESTXQ rest:form-param()

2020-04-22 Thread Zimmel, Daniel

Hi,

in an HTML form with a RESTXQ POST action, do I really have to declare 
%rest:form-param() for every param, or is there a way to pass the full query 
component in one parameter?

Best, Daniel

Re: [basex-talk] file:move() fails when moving to another file system

2020-02-12 Thread Zimmel, Daniel

ava:97)
at org.basex.query.QueryContext.iter(QueryContext.java:332)
at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90)
at org.basex.core.cmd.AQuery.query(AQuery.java:107)
at org.basex.core.cmd.XQuery.run(XQuery.java:22)
at org.basex.core.Command.run(Command.java:257)
at org.basex.core.Command.execute(Command.java:93)
... 5 more
Caused by: java.nio.file.FileSystemException: [...]: Die Operation ist nicht 
erlaubt
at 
java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
at 
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at 
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixCopyFile.copyFile(UnixCopyFile.java:288)
at java.base/sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:476)
at 
java.base/sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:263)
at java.base/java.nio.file.Files.move(Files.java:1413)
at org.basex.query.func.file.FileCopy.relocate(FileCopy.java:75)
at org.basex.query.func.file.FileCopy.relocate(FileCopy.java:53)
at org.basex.query.func.file.FileMove.item(FileMove.java:18)
   at org.basex.query.func.file.FileFn.item(FileFn.java:25)
... 15 more



Von: Christian Grün 
Gesendet: Dienstag, 11. Februar 2020 17:54
An: Zimmel, Daniel 
Cc: BaseX 
Betreff: Re: [basex-talk] file:move() fails when moving to another file system

>: Die Operation ist nicht erlaubt

A last try: What do you get if you run it with basex (the standalone, not the 
client)?

> I think the if/then can be left out, because the query will always fail 
> sequentially before the delete if something goes wrong with the copy (?)

Exactly.

Re: [basex-talk] file:move() fails when moving to another file system

2020-02-11 Thread Zimmel, Daniel

Christian,

basexclient -d gives only

[file:io-error] ... : Die Operation ist nicht erlaubtat 
org.basex.api.client.ClientSession.receive(ClientSession.java:192)
at org.basex.api.client.ClientSession.execute(ClientSession.java:161)
at org.basex.api.client.ClientSession.execute(ClientSession.java:166)
at org.basex.api.client.Session.execute(Session.java:36)
at org.basex.core.CLI.execute(CLI.java:92)
at org.basex.BaseX.(BaseX.java:100)
at org.basex.BaseXClient.(BaseXClient.java:35)
at org.basex.BaseXClient.main(BaseXClient.java:22)
Stopped at ., 1/10:

My alternative works:

  file:copy($sourcefilepath,$archivedir||file:dir-separator()||$file),
  if (file:exists($archivedir||file:dir-separator()||$file))
  then (file:delete($sourcefilepath))

but I think the if/then can be left out, because the query will always fail 
sequentially before the delete if something goes wrong with the copy (?)

Thanks, Daniel

Von: Christian Grün 
Gesendet: Dienstag, 11. Februar 2020 16:47
An: Zimmel, Daniel 
Cc: BaseX 
Betreff: Re: [basex-talk] file:move() fails when moving to another file system

Hi Daniel,

BaseX uses standard Java NIO to move files [1]; see [2] for a more 
comprehensive documentation.

You could enable debugging, and we could have a look at the full command line 
output; maybe that gives us some hints what goes wrong here.

Did you try the alternative to copy your files and delete the source afterwards?

Best,
Christian

[1] 
https://github.com/BaseXdb/basex/blob/master/basex-core/src/main/java/org/basex/query/func/file/FileCopy.java#L79
[2] 
https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#move(java.nio.file.Path,%20java.nio.file.Path,%20java.nio.file.CopyOption...)



Zimmel, Daniel mailto:d.zim...@esvmedien.de>> schrieb am 
Di., 11. Feb. 2020, 16:39:
Hi,

I am able to file:copy() or file:write() a file from a Linux system to a 
Windows mount (via cifs).
I am not able to file:move() to the same target.

The error says "file:io-error: Die Operation ist nicht erlaubt"

BaseX version is 9.2.1

Is this a bug or a known restriction with moving things around in Java?

Thanks, Daniel

[basex-talk] file:move() fails when moving to another file system

2020-02-11 Thread Zimmel, Daniel

Hi,

I am able to file:copy() or file:write() a file from a Linux system to a 
Windows mount (via cifs).
I am not able to file:move() to the same target.

The error says "file:io-error: Die Operation ist nicht erlaubt"

BaseX version is 9.2.1

Is this a bug or a known restriction with moving things around in Java?

Thanks, Daniel

Re: [basex-talk] Help with a Query/Performance

2020-01-20 Thread Zimmel, Daniel

Hi Tom,

I think that trying to copy/modify a huge tree is definitely the bottleneck 
here.
Why don’t you copy only your third Message element and then reconstruct the 
wrapping Container with ContainerMetaData?

Since the wanted result is a transformation, perhaps a typeswitch expression 
might be an alternative, if there is something that stops you from 
reconstructing.

Daniel

Von: Tom Rauchenwald (UNIFITS) 
Gesendet: Montag, 20. Januar 2020 10:01
An: basex-talk@mailman.uni-konstanz.de
Betreff: [basex-talk] Help with a Query/Performance

Hi list,

I'm struggling with a query.

We have XML documents with a structure similar to this:


  FOO
  FOO
  

  FOO
  FOO


  FOO
  FOO


  FOO
  FOO

  
  

  FOO
  FOO


  FOO
  FOO

  
  

  FOO
  FOO


  FOO
  FOO

  


Messages are bundled in a container (up to n times for each message), and each 
message has details (also up to n times). Container, Message contain data that 
is the same for all details (it's basically a grouping).
I'd like to retrieve a Detail with all corresponding data associated with it, 
so basically a MessageADetail, MessageA (without all the other 
MessageADetails), Container (without all the other Messages).
I know the position of the message (i.e., I know that I want the second 
MessageA for example), and I know the position of the Detail (i.e., I know that 
I want the 3rd Detail).
The use case is to show the detail in context in a UI.

The query to do this I came up with is (here I want to get the 2nd detail from 
the third MessageA):

  let $fh := (copy $x := /*:Container
   modify ( delete node $x/*:MessageA[position() != 3]
  , delete node $x/*:MessageA[3]/*:MessageADetail[position() != 2]
  , delete node $x/*:MessageB
  , delete node $x/*:MessageC
  )
  return $x)
  return $fh

This works well for small documents. For large documents it can take a couple 
of seconds to evaluate the query (our real-life documents do have more 
data/elements in Details and Message).
I'm wondering if there's a better/more efficient way to do this. I tried 
formulating a query that doesn't do deletes, but I couldn't come up with a 
solution that performs better and is correct.

Any pointers would be very much appreciated.

Here's a function to generate sufficiently large test data:

declare function local:sample($numberOfMessages, $numberOfDetails) {

  FOO
  FOO
  {for $i in 1 to $numberOfMessages
return
  

  FOO {$i}
  FOO {$i}

{for $j in 1 to $numberOfDetails
 return
 
   FOO {$j}
   FOO {$j}
 
}
  
  }
  

  FOO
  FOO


  FOO
  FOO

  
  

  FOO
  FOO


  FOO
  FOO

  

};

db:create('tr-test', local:sample(20, 10), 'test.xml')

Thanks,
Tom Rauchenwald

Re: [basex-talk] BaseX and validating the entire database

2019-12-13 Thread Zimmel, Daniel

I would second that using Schematron here seems more complicated than actually 
writing the code in XQuery; it is even shorter.

We do this kind of checks in XQuery all the time, similar to the examples below.

Schema validation can also be quite slow when compared to optimized queries in 
XQuery/XPath.



Having said that, Schematron validation does work seamlessly with BaseX, but as 
far as I know it is not possible to pass external parameters to a schematron 
file.

So you would have to write your Schematron code in an XQuery variable anyway or 
try to insert dynamically (which is possible but does not sound very robust).

For document (not consistency) checks we use the SchXslt implementation which 
does an excellent job (https://github.com/schxslt/schxslt) because the module 
implementation linked on the BaseX wiki is still XSLT 1.0-only (and 1.0 support 
was temporarily dropped in Saxon 9.8). There is also a BaseX module ready to 
use in SchXslt.


Daniel

Von: Hans-Juergen Rennau 
Gesendet: Freitag, 13. Dezember 2019 10:11
An: Christian Grün ; ERRINGTON Luke 

Cc: basex-talk@mailman.uni-konstanz.de
Betreff: Re: [basex-talk] BaseX and validating the entire database

Hi Luke, I would like to emphasize (or simply remind you) of two key features 
of XPath (and XML technology in general). The FIRST one is that treating the 
information in a single document or in a collection of documents or a 
collection of document fragments is identical. So, for example, $data//foo 
works regardless of whether $data is one document, or a collection of 
documents, or a single element extracted from some document, or a collection of 
elements extracted from multiple documents or even from a mixture of documents 
exposed by a database, the file system and REST service responses etc. 
Therefore collecting documents into a single document prior to processing is 
(according to my opinion) somewhat against the grain of what XML technology 
excels in accomplishing.

The SECOND point is that XPath has been specified with mathematical precision, 
so I cannot imagine being more precise and concise when it comes to defining 
*rules*. (That XPath expressions cannot easily replace a grammar is a different 
matter, of course.)

And finally - I would not overemphasize the importance of using schematron, as 
equivalent validation functionality is fairly easy to implement just using 
XQuery/XPath: it is the XPath language what is the engine and heartbeat of it 
all, it is a secondary question whether one uses the schematron framework, 
ingenious and handy though it is for typical single document checks.

Cheers,
Hans

Am Freitag, 13. Dezember 2019, 07:53:48 MEZ hat ERRINGTON Luke 
mailto:luke.erring...@sydac.com>> Folgendes 
geschrieben:


Hi Christian,

Thank you for your time in preparing your response and examples. You describe 
the approach that I thought would be necessary if we couldn't get some sort of 
schema validation to work. Unfortunately the specification of the validation 
requirements in XQuery code is not as clean, clear or minimal as might be 
desired.

It would be nice to have some sort of pre-commit hook for validating 
modifications to the database so that we are not restricted to only allowing 
modifications through XQuery. It looks as though this is the point of 
https://github.com/BaseXdb/basex/issues/1082, 
 but it looks as though that 
is on hold, after some significant discussion.

Presumably I could achieve schema validation by having the entire data set 
inside one document, but that would lose the benefits of collections, and 
having the data arranged similar to a file system, so ... I was hoping that I 
could define a Schematron rule something like this (untested, because I'm 
struggling to get Schematron working at the moment - content is not allowed in 
prolog):




Trying to map invalid 
object id
Trying to map invalid 
object id




This is relatively minimal and expressive. It seems to work just by XPath, so 
all I need is //object/@id to find the object IDs present in all documents, not 
just this one. But, when I use //object/@id as a path in BaseX it does just 
that! It returns all of the object IDs, in all of the documents - so maybe this 
schema can be used across all documents at once! That would be fantastic!

Of course, in practice I am not sure if this can be done, and I am pretty new 
to all of this. I see that currently schematron::validate requires a node as an 
input. I presume that db:open() will give me a sequence of document-nodes. What 
I presume would work is if I could turn this sequence into a single 
document-node, somehow. I am not sure if this can be done easily, or 
efficiently, in XQuery, or whether it would be easier to implement it within 
BaseX's implementation of db:open, or whether this is not really feasible at 
all ...

(With that working a similar line of thought would

Re: [basex-talk] Is there an API that provides XQuery compilation results?

2019-11-13 Thread Zimmel, Daniel

With xquery:parse(), I wondered about this myself, so I second this question 
without having an answer :-(

You will get a query plan if it is a non-static error:

xquery:parse("let $a :=  return $a + 1", map 
{'compile':true(),'pass':true()})

… but what the pass option does is not clear to me. Definitely not passing the 
type error here.

If you only need to catch the error message without the query plan, there’s 
always try/catch of course.

Best, Daniel

Von: Peter Villadsen 
Gesendet: Mittwoch, 13. November 2019 00:36
An: basex-talk@mailman.uni-konstanz.de
Betreff: [basex-talk] Is there an API that provides XQuery compilation results?

When I use the GUI application I can see some error description (even if it a 
little terse) when my queries are incorrect. For instance I might get:

Stopped at tableFields.xq, 27/50:
[XQST0118] Different start and end tag: 

If I submit a query like 

However, I cannot seem to find an API that brings me that information? I tried:

xquery:parse("", map {'pass':true()})

but that did not get the result I expected.  Also, while we’re at at it, it 
would be nice to also get the optimized query.


Best Regards

Peter Villadsen
Principal Architect
Microsoft Business Applications Group

[basex-talk] Built-in collations

2019-08-21 Thread Zimmel, Daniel

Hi,

I found out that after adding the third-party jar to the classpath as described 
here (http://docs.basex.org/wiki/XQuery_3.1#Collations), I can finally 
alphanumeric-sort my map:keys:

map:keys($map_ev) => 
sort('http://www.w3.org/2013/collation/UCA?lang=de;numeric=yes')

Which I like a lot.
Are there any plans on having this already shipped with BaseX in future 
releases?

Thanks, Daniel

Re: [basex-talk] Update operation is spawning new databases with underscore

2019-07-18 Thread Zimmel, Daniel

> In your first query, ADDCACHE was set to false. Does the problem exist
> in both cases?

Definitely.

> Just guessing:
> • Did you get any errors reported while performing updates?

No.

> • Do you have any symbolic links (and potential circular dependencies)
> in the directory you are parsing?

No.

BTW there is a db:lock message when I try to remove them via RESTXQ (DBA 
module).
I will try to get back if any other hint shows up that might help.

Thanks, Daniel

Re: [basex-talk] Update operation is spawning new databases with underscore

2019-07-18 Thread Zimmel, Daniel

> It looks as if databases are not cleaned up in your environment that
> are temporarily created during the update step.
> 
> I guess it will be difficult for you to get this reproduced for us?

Indeed.

It does happen on Windows and Linux, though. I pretty much run the default 
setup.

This is a complete example of a simple import query:

declare variable $system external;
declare variable $config := fetch:xml(file:base-dir() || '../../config/config.' 
|| $system ||'.xml')/basexconfig;
declare variable $dbname := string($config/esvxml/database/esvschema);
declare variable $sourcefile := string($config/esvxml/import/paths/sourcefile)

for $source in fetch:xml($sourcefile)/sources/*/path
where file:is-dir($source)
  let $path := $source
  let $produkt := $source/@produkt
  let $produkttyp := $source/parent::*/name()
  let $xmlprozess := $source/@xml
  for $file in file:list($path, true(), '*.xml')
return db:add($dbname, $path || $file ,  
$produkttyp||'/'||$xmlprozess||'/'||$produkt||'/'||$file, 
  map { 'intparse': true(),'chop': false(), 'addcache': true(), 
'skipcorrupt': true() })

Nothing special in here.

It seems safe to delete those temporary databases regularly, then?
Too bad I cannot reproduce.

Thanks, Daniel

[basex-talk] Update operation is spawning new databases with underscore

2019-07-17 Thread Zimmel, Daniel

Hi,

sometimes in my update operations BaseX deliberately is spawning several 
databases like the following, with a name of the database plus underscore and 
number:

DBNAME_2618234251

caused by

db:replace($dbname, $dbpath|| '/' || $fname, $file-to-import, map { 
'intparse': true(),
'chop': false(),
'addcache': false() })

I cannot find anything about this behavior in the docs.
It seems to happen randomly, and the main database does seem ok.

How can I avoid these extra databases? They will vanish only by force.

BaseX 9.2.1

Thanks, Daniel

Re: [basex-talk] Hyphenation with XQuery

2019-06-20 Thread Zimmel, Daniel

Yes, I think this could be fun. 
Or perhaps not. 
Either way, for my use case, for hyphenation analysis I can do a query to an 
API, but it just feels wrong not doing this in XQuery ;-)

> -Ursprüngliche Nachricht-
> Von: Christian Grün [mailto:christian.gr...@gmail.com]
> Gesendet: Mittwoch, 19. Juni 2019 09:56
> An: Zimmel, Daniel
> Cc: BaseX
> Betreff: Re: [basex-talk] Hyphenation with XQuery
> 
> Hi Daniel,
> 
> I’m not aware of any XQuery implementation. Would you like to give it a try? 
> ;)
> 
> Best,
> Christian
> 
> 
> 
> On Tue, Jun 4, 2019 at 10:26 AM Zimmel, Daniel 
> wrote:
> >
> > Hi,
> >
> > somewhat off-topic to BaseX, but I wonder if anybody already tried to port 
> > the
> TeX hyphenation algorithm to XQuery?
> >
> > See for example this Python port:
> >
> > https://nedbatchelder.com/code/modules/hyphenate.py
> >
> > >>> hyphenate_word("supercalifragilisticexpialidocious")
> > ['su', 'per', 'cal', 'ifrag', 'ilis', 'tic', 'ex', 'pi', 'ali', 'do', 
> > 'cious']
> >
> > Best, Daniel
> >

[basex-talk] Hyphenation with XQuery

2019-06-04 Thread Zimmel, Daniel

Hi,

somewhat off-topic to BaseX, but I wonder if anybody already tried to port the 
TeX hyphenation algorithm to XQuery?

See for example this Python port:

https://nedbatchelder.com/code/modules/hyphenate.py

>>> hyphenate_word("supercalifragilisticexpialidocious")
['su', 'per', 'cal', 'ifrag', 'ilis', 'tic', 'ex', 'pi', 'ali', 'do', 
'cious']

Best, Daniel

Re: [basex-talk] db:replace() in DBA resulting in duplicates

2019-05-20 Thread Zimmel, Daniel

> > * Optimize DB
> > * The first db:replace() via RESTXQ creates a second file with the same 
> > base-uri()
> > * The second db:replace() via RESTXQ replaces the earlier created second 
> > file (duplicate)

> Do you perform the next steps in the database instance that has been copied 
> to another server, or does the problem also happen on the original instance? 
> In the first case, do both servers use the same operating system (either 
> Linux or Windows)?

* both; does not happen in the original instance, does only happen in the 
copied instance
  * cannot reproduce with small fresh database created with 9.2.1
* not the same OS; original was created in Win 7, copied to linux server

OK, with a removal of the target database folder, a database optimization on 
the source and target and a fresh copy I cannot reproduce.
I will definitely write a note to myself to always optimize the database 
before/after moving them and be sure to remove any artifacts.

Daniel
.

Re: [basex-talk] db:replace() in DBA resulting in duplicates

2019-05-17 Thread Zimmel, Daniel

Follow-up, some details:

* Create large (5GB) DB in BaseX GUI
* Copy DB folder manually to another server
* Optimize DB
* The first db:replace() via RESTXQ creates a second file with the same 
base-uri()
* The second db:replace() via RESTXQ replaces the earlier created second file 
(duplicate)

I cannot reproduce with a small newly created db.
The initial database was created with BaseX 9.0.2, update operation runs with 
BaseX 9.2.1

Should I try to optimize the database with 9.1.2 before copying and try again? 
Perhaps this is a database-structure related issue?

> -Ursprüngliche Nachricht-
> Von: Christian Grün [mailto:christian.gr...@gmail.com]
> Gesendet: Montag, 29. April 2019 16:00
> An: Zimmel, Daniel
> Cc: BaseX
> Betreff: Re: [basex-talk] db:replace() in DBA resulting in duplicates
> 
> > Still, when I browse the database, the /DB/path only exists once (as far as 
> > I can
> see from the sorting), but querying is giving me two documents, according to
> db:path() living in the same path.
> > This should never happen, should it?
> 
> This is surprising indeed.
> 
> Right now, db:add is still faster than db:replace, because the check
> for existing database paths can be completely skipped. This allows
> high-performance database insertions.In a future version of BaseX, I
> would love to get rid of db:add and db:replace and replace these
> functions with a db:update function, which will enforce that each
> document path exists only once in the database. This would surely
> reduce patterns like the one you encountered.
> 
> If you still manage to get this reproduced, just come back to us.

Re: [basex-talk] 9.2: http://localhost:8984/dba/file-upload 400 bad request

2019-05-02 Thread Zimmel, Daniel

Yes, the snapshot seems to fix this.

Thanks, Daniel

> -Ursprüngliche Nachricht-
> Von: Christian Grün [mailto:christian.gr...@gmail.com]
> Gesendet: Donnerstag, 2. Mai 2019 11:16
> An: Zimmel, Daniel
> Cc: BaseX
> Betreff: Re: [basex-talk] 9.2: http://localhost:8984/dba/file-upload 400 bad 
> request
> 
> Hi Daniel,
> 
> This reminds me of a problem that has been discussed on the list some
> days ago. Could you please check out the latest snapshot and see if
> the problem persists [1]?
> 
> BaseX 9.2.1 will be released this or next week.
> 
> Best,
> Christian
> 
> [1] http://files.basex.org/releases/latest
> 
> 
> 
> 
> 
> On Thu, May 2, 2019 at 11:02 AM Zimmel, Daniel 
> wrote:
> >
> > Hi,
> >
> > in 9.2 web admin I get an HTTP 400 with any file upload:
> >
> > Stopped at D:/BaseX92/webapp/dba/files/file-upload.xqm, 23/10:
> > [XPTY0004] Cannot convert empty-sequence() to map(xs:string,
> xs:base64Binary): ().
> >
> > The file seems to be sent, but $files is empty.
> >
> > Is this a bug?
> >
> > This was working with the default settings in 9.0.2
> >
> > Daniel
> >
> > Host: localhost:8984
> > User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101
> Firefox/60.0
> > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> > Accept-Language: de,en-US;q=0.7,en;q=0.3
> > Accept-Encoding: gzip, deflate
> > Referer: http://localhost:8984/dba/files
> > Content-Type: multipart/form-data; boundary=---
> 54472172614771
> > Content-Length: 153
> > Cookie: JSESSIONID=node0npu7s85e3q5amqmmqir6aak0.node0
> > DNT: 1
> > Connection: keep-alive
> > Upgrade-Insecure-Requests: 1
> > Cache-Control: max-age=0

[basex-talk] 9.2: http://localhost:8984/dba/file-upload 400 bad request

2019-05-02 Thread Zimmel, Daniel

Hi,

in 9.2 web admin I get an HTTP 400 with any file upload:

Stopped at D:/BaseX92/webapp/dba/files/file-upload.xqm, 23/10:
[XPTY0004] Cannot convert empty-sequence() to map(xs:string, xs:base64Binary): 
().

The file seems to be sent, but $files is empty.

Is this a bug?

This was working with the default settings in 9.0.2

Daniel

Host: localhost:8984
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 
Firefox/60.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: http://localhost:8984/dba/files
Content-Type: multipart/form-data; 
boundary=---54472172614771
Content-Length: 153
Cookie: JSESSIONID=node0npu7s85e3q5amqmmqir6aak0.node0
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0

Re: [basex-talk] db:replace() in DBA resulting in duplicates

2019-04-29 Thread Zimmel, Daniel

Oddly enough, I cannot reproduce by myself!

Still, when I browse the database, the /DB/path only exists once (as far as I 
can see from the sorting), but querying is giving me two documents, according 
to db:path() living in the same path.
This should never happen, should it?

I will simply drop the database, but I wish I could understand what went wrong.
I just did the same updating query and it is politely replacing the second file 
without adding another one (still, the initial file remains unreplaced). 

Thanks,
Daniel

> -Ursprüngliche Nachricht-
> Von: Christian Grün [mailto:christian.gr...@gmail.com]
> Gesendet: Montag, 29. April 2019 11:08
> An: Zimmel, Daniel
> Cc: BaseX
> Betreff: Re: [basex-talk] db:replace() in DBA resulting in duplicates
> 
> Hi Daniel,
> 
> seems hard to reproduce. Could you tell me what you did step by step?
> This is what I tried:
> 
> 1. I created a DB database.
> 2. I added the document  to the database path /path/file.xml.
> 3. I replaced this document with the same document via the DBA.
> 4. Your query db:open('DB')/root[@id='id']/base-uri() returned a single 
> string.
> 
> Thanks in advance,
> Christian
> 
> PS: The form parameter handling in 9.2 contains a little bug (as a
> result, I couldn’t invoke the replace functionality with the current
> version). I just uploaded a new snapshot. Both the current and the
> snapshot version should behave identically in terms of adding and
> replacing documents.
> 
> 
> 
> On Fri, Apr 26, 2019 at 4:45 PM Zimmel, Daniel 
> wrote:
> >
> > Hello,
> >
> > something has gone awry in my database after a db:replace():
> >
> > query:
> > db:open('DB','/path/file.xml')/base-uri()
> >
> > result:
> > /DB/path/file.xml
> >
> > This is as expected. The result is the replaced file. But when I
> >
> > query:
> > db:open('DB')/root[@id='id']/base-uri()
> >
> > result is a sequence:
> > /DB/path/file.xml
> > /DB/path/file.xml
> >
> > The first result is the file before the db:replace(), the second after.
> >
> > Can somebody explain why db:open() with path parameter is returning one
> result, but without parameter two results?
> > Database statistics say there are indeed two documents. But when I export 
> > the
> database, only one gets exported.
> >
> > This happens with db:replace() on 9.2 from the DBA browser interface.
> > When I use the same db:replace() XQuery on 9.2 with my local database, all 
> > is
> well (document gets replaced, no duplicates). Settings should be the default
> ones.
> >
> > Thanks, Daniel
> >
> >

[basex-talk] db:replace() in DBA resulting in duplicates

2019-04-26 Thread Zimmel, Daniel

Hello,

something has gone awry in my database after a db:replace():

query:
db:open('DB','/path/file.xml')/base-uri()

result:
/DB/path/file.xml

This is as expected. The result is the replaced file. But when I

query:
db:open('DB')/root[@id='id']/base-uri()

result is a sequence:
/DB/path/file.xml
/DB/path/file.xml

The first result is the file before the db:replace(), the second after.

Can somebody explain why db:open() with path parameter is returning one result, 
but without parameter two results?
Database statistics say there are indeed two documents. But when I export the 
database, only one gets exported.

This happens with db:replace() on 9.2 from the DBA browser interface.
When I use the same db:replace() XQuery on 9.2 with my local database, all is 
well (document gets replaced, no duplicates). Settings should be the default 
ones.

Thanks, Daniel

Re: [basex-talk] jobs:eval as a service, "no variables allowed"

2019-04-03 Thread Zimmel, Daniel

Thanks for the clarification. This sound like a wise thing to do.

> -Ursprüngliche Nachricht-
> Von: Christian Grün [mailto:christian.gr...@gmail.com]
> Gesendet: Mittwoch, 3. April 2019 13:18
> An: Zimmel, Daniel
> Cc: BaseX
> Betreff: Re: [basex-talk] jobs:eval as a service, "no variables allowed"
> 
> Hi Daniel,
> 
> no variables can be bound if services are registered, that’s true
> (values that are bound to variables might potentially be huge, and
> blow up the initial startup time of BaseX). I have added a comment to
> our documentation.
> 
> Best,
> Christian
> 
> 
> On Wed, Apr 3, 2019 at 11:57 AM Zimmel, Daniel 
> wrote:
> >
> > Hi,
> >
> > when I try to register jobs:eval as a service, BaseX 9.2 says: "no variables
> allowed".
> >
> > Is it not possible to register a service with a variable?
> >
> > Not working:
> >
> > jobs:eval("declare variable $xml external; $xml",
> > map { 'xml': 'XML' },
> > map { 'id':'TEST', 'start':'11:31:50', 'interval':'P1D', 'service': true() 
> > })
> >
> > working:
> >
> > jobs:eval("declare variable $xml external; $xml",
> > map { 'xml': 'XML' },
> > map { 'id':'TEST', 'start':'11:31:50', 'interval':'P1D', 'service': false() 
> > })
> >
> > http://docs.basex.org/wiki/Jobs_Module#jobs:eval
> >
> > Thanks, Daniel
> >

[basex-talk] jobs:eval as a service, "no variables allowed"

2019-04-03 Thread Zimmel, Daniel

Hi,

when I try to register jobs:eval as a service, BaseX 9.2 says: "no variables 
allowed".

Is it not possible to register a service with a variable?

Not working:

jobs:eval("declare variable $xml external; $xml", 
map { 'xml': 'XML' }, 
map { 'id':'TEST', 'start':'11:31:50', 'interval':'P1D', 'service': true() })

working:

jobs:eval("declare variable $xml external; $xml", 
map { 'xml': 'XML' }, 
map { 'id':'TEST', 'start':'11:31:50', 'interval':'P1D', 'service': false() })

http://docs.basex.org/wiki/Jobs_Module#jobs:eval

Thanks, Daniel

Re: [basex-talk] following-sibling axis -- real data example

2019-03-04 Thread Zimmel, Daniel

Nothing wrong here with BaseX behavior.

If your xml import chops the whitespace, following-sibling::text()[1] will of 
course always match “.-“ for the first two word nodes.

Cheers, Daniel


Von: Mark Bordelon [mailto:markcborde...@yahoo.com]
Gesendet: Freitag, 1. März 2019 22:51
An: Michael Seiferle
Cc: BaseX
Betreff: Re: [basex-talk] following-sibling axis -- real data example

Gentlemen (especially Michael)!
My follow-up to my original question from a few days ago:


 XML:


  A
  toto. -A
  substantia
.

XPATH:
//clause[word and not(word[not(@lexmorph) or @lexmorph='' or 
contains(@lexmorph,' ')])]/string-join(word[not(@implicit)]/concat(
text(),'|',tokenize(@lexmorph, '\|')[2],'|'
,normalize-space(./following-sibling::text()[1])),'~~~’)


executing this in https://www.freeformatter.com/xpath-tester.html#ad-output 
returns the (correct) result:
A|P|~~~toto|D|. -~~~A|P|~~~substantia|N|

executing using XQUERY in basex returns the (incorrect) result:
A|P|. -~~~toto|D|. -~~~A|P|~~~substantia|N|

executing using XQUERY in basex adding Kristian's [. instance of text()] to the 
axis returns the (incorrect) result:
A|P|. -~~~toto|D|. -~~~A|P|~~~substantia|N|

I have tried using the -w option’s true and false values, but my results are 
always as above.

Any ideas?





On Feb 27, 2019, at 01:59, Michael Seiferle 
mailto:m...@basex.org>> wrote:

Hi Mark,

as Martin already stated, the '-w‘-Option has to be active at import time, 
otherwise the whitespace will be chopped.

If I were to do it, I’d reindex all data and explicitly mark all elements that 
should preserve whitespace, if this is not an option I’d reindex all data with 
whitespace chopping set to off.

Looking forward to your example, I am sure we can figure this out :-)

Best
Michael





Am 26.02.2019 um 18:52 schrieb Mark Bordelon 
mailto:markcborde...@yahoo.com>>:

A follow-up:  starting basex -w does NOT seem to solve completely my issue 
after all. Real data (more complicated than the simplified example) still does 
not query correctly: text nodes from after later elements are displayed in the 
place of null text nodes.
I’ll try to get a better example, still simplified, that shows this.

Re: [basex-talk] Schema 1.1 validation and Xerces cta-full-xpath-checking?

2019-02-14 Thread Zimmel, Daniel

This is FUN indeed. Thanks Christian!
Since I consider it a central feature when using Xerces 1.1 validation (it is 
one out of four controlled Xerces validation features in Oxygen options, too), 
you may consider adding it explicitly to the Wiki example:

validate:xsd($doc,$schema,map { 
'http://apache.org/xml/features/validation/cta-full-xpath-checking': true() })

Without the feature, you are limited to XPath 1.0 in your assertions.

Daniel

> -Ursprüngliche Nachricht-
> Von: Christian Grün [mailto:christian.gr...@gmail.com]
> Gesendet: Donnerstag, 14. Februar 2019 14:38
> An: Zimmel, Daniel; George Sofianos
> Cc: basex-talk@mailman.uni-konstanz.de
> Betreff: Re: [basex-talk] Schema 1.1 validation and Xerces cta-full-xpath-
> checking?
> 
> Hi Daniel, hi George,
> 
> With the latest snapshot of BaseX, XSD validation features can now be
> specified [1,2]. In addition to that,
> 
> • I have removed the version argument. The most recent supported
> version XML Schema will now be used by default.
> • Two additional functions can be used to retrieve the applied
> processor and the supported version.
> 
> Have fun,
> Christian
> 
> [1] http://docs.basex.org/wiki/Validation_Module#XML_Schema_Validation
> [2] http://files.basex.org/releases/latest/
> 
> 
> 
> On Mon, Nov 19, 2018 at 5:07 PM Zimmel, Daniel 
> wrote:
> >
> > Hi,
> >
> > is there any chance BaseX will support the cta-full-xpath-checking option 
> > (see
> https://xerces.apache.org/xerces2-j/features.html)? Without it, schema 1.1
> validation is limited.
> >
> > I could not find any switches to configure Xerces features, but did find 
> > someone
> mentioning it from two years ago:
> > https://www.mail-archive.com/basex-talk@mailman.uni-
> konstanz.de/msg08446.html
> >
> > Or have I missed something in the documentation?
> >
> > Regards, Daniel
> >
> >
> >

Re: [basex-talk] XSD 1.1 - Element not found

2019-01-22 Thread Zimmel, Daniel

With 1.1, I found that validate:xsd() requires you to explicitly give ‘1.1’ as 
third parameter.
Having vs:minVersion is not being honored by BaseX.

Daniel

Von: Hans-Juergen Rennau [mailto:hren...@yahoo.de]
Gesendet: Dienstag, 22. Januar 2019 15:15
An: BaseX Support
Betreff: [basex-talk] XSD 1.1 - Element not found

Dear BaseX team,

trying to use XSD 1.1, I encounter a problem.

As indicated in the documentation, I added to the lib folder the jar files from 
the newest Xerxes distribution (Xerces-J-bin.2.12.0-xml-schema-1.1.zip, 
download from http://xerces.apache.org/mirrors.cgi#binary ).

The problem: when using the schema attribute vc:minVersion="1.1", validation 
fails with the message:

1:41: cvc-elt.1.a: Cannot find the declaration of element 'Foo'.

But the element is there, and the namespace matches the target namespace. When 
changing the attribute vc:minVersion="1.0", validation works, but, of course, 
cannot cope with 1.1 features. (The same applies to validation after removing 
the attribute.)

I tried both - copying only the four jar files named in the documention,

cupv10k-runtime.jar
org.eclipse.wst.xml.xpath2.processor_1.2.0.jar
xercesImpl.jar
xml-apis.jar

and then I tried copying all jars:

cupv10k-runtime.jar
icu4j.jar
org.eclipse.wst.xml.xpath2.processor_1.2.0.jar
resolver.jar
serializer.jar
xercesImpl.jar
xercesSamples.jar
xml-apis.jar

Any suggestions would be welcome!

With kind regards,
Hans-Jürgen

PS: I use Java version 1.8.0_181-b13

Re: [basex-talk] Command line options not working with update operation on input file

2018-12-13 Thread Zimmel, Daniel

Looks like I was not getting the toggle part correctly.
Yes it is very confusing since it says:

"-w  Preserve whitespaces from input files"

Thanks! I will toggle this now in my internal documentation :-)

> -Ursprüngliche Nachricht-
> Von: Christian Grün [mailto:christian.gr...@gmail.com]
> Gesendet: Donnerstag, 13. Dezember 2018 14:26
> An: Zimmel, Daniel
> Cc: BaseX
> Betreff: Re: [basex-talk] Command line options not working with update 
> operation
> on input file
> 
> > With the same query, when CHOP=false in the .basex file and -w switch on the
> command line, whitespace gets chopped (not expected!)
> 
> The option is toggled (see one of my previous replies). We should
> revise the help text on the command-line in a future version.

Re: [basex-talk] Command line options not working with update operation on input file

2018-12-13 Thread Zimmel, Daniel

Thanks for the hint. Yes, in my .basex there is a CHOP=false; when I delete it, 
it works as expected.

But as I pointed out, this looks like a bug to me:

With the same query, when CHOP=false in the .basex file and -w switch on the 
command line, whitespace gets chopped (not expected!)

Input


TTT
eins und zwei

  


.basex:

CHOP=false

Result


TTTeinsundzwei

> -Ursprüngliche Nachricht-
> Von: Christian Grün [mailto:christian.gr...@gmail.com]
> Gesendet: Donnerstag, 13. Dezember 2018 13:49
> An: Zimmel, Daniel
> Cc: BaseX
> Betreff: Re: [basex-talk] Command line options not working with update 
> operation
> on input file
> 
> This is what I get:
> 
> 
> 
> REPLACED
>  eins
> 
>   
> 
> 
> Do you possibly have some settings in your .basex configuration file
> that I should consider? Or do you have a 'test' database with chopped
> whitespaces that is opened instead of your test.xml document?
> 
> 
> 
> On Thu, Dec 13, 2018 at 1:43 PM Zimmel, Daniel 
> wrote:
> >
> > I was referring to XQuery update writeback, chop option is set to false:
> >
> > $ basex.bat -u -w -c"set exporter omit-xml-declaration=no,indent=no" 
> > -itest.xml
> update.xquery
> >
> > update.xquery:
> >
> > for $c in doc(document-uri())//kennung
> > return
> > (replace value of node $c with "REPLACED")
> >
> > test.xml:
> >
> > 
> > 
> > test
> >  eins
> > 
> >   
> > 
> >
> >
> > Result
> >
> > 
> >
> REPLACEDeins >TestfettkursivText z.B. am 01.02.2018
> Text.
> >
> >
> > Saxon does not change indentation (with declare option saxon:output
> "indent=no")
> >
> > > -Ursprüngliche Nachricht-
> > > Von: Christian Grün [mailto:christian.gr...@gmail.com]
> > > Gesendet: Donnerstag, 13. Dezember 2018 12:41
> > > An: Zimmel, Daniel
> > > Cc: BaseX
> > > Betreff: Re: [basex-talk] Command line options not working with update
> operation
> > > on input file
> > >
> > > > The only thing is I prefer the Saxon default about not changing 
> > > > anything on
> > > already existing indentation (BaseX is deleting any indentation with
> indent=no),
> > > but I am aware this is implementation-defined.
> > >
> > > Existing identation is actually preserved by both BaseX and Saxon:
> > >
> > > declare namespace output = 'http://www.w3.org/2010/xslt-xquery-
> serialization';
> > > declare option output:indent 'no';
> > > parse-xml('
> > > 
> > > ')
> > >
> > > Were you referring to documents with chopped whitespaces?
> > >
> > > Cheers
> > > Christian
> > >
> > >
> > >
> > > > > -Ursprüngliche Nachricht-
> > > > > Von: Christian Grün [mailto:christian.gr...@gmail.com]
> > > > > Gesendet: Mittwoch, 12. Dezember 2018 18:47
> > > > > An: Zimmel, Daniel
> > > > > Cc: BaseX
> > > > > Betreff: Re: [basex-talk] Command line options not working with update
> > > operation
> > > > > on input file
> > > > >
> > > > > …I think I got what you were trying to achieve! There is a separate
> > > > > option for exporting databases and writing updates. This may be more
> > > > > obvious now after another updates of our documentation [1].
> > > > >
> > > > > Here is a little self-contained BaseX call that (as I hope)
> > > > > demonstrates the behavior:
> > > > >
> > > > > basex -u -c"set exporter omit-xml-declaration=no"
> > > > > "file:write('doc.xml', )" "insert node  into doc('doc.xml')/*"
> > > > > "fetch:xml('doc.xml')"
> > > > >
> > > > > 
> > > > > 
> > > > >   
> > > > > 
> > > > >
> > > > > You can also run the following command script ("basex script.bxs"):
> > > > >
> > > > > set exporter omit-xml-declaration=no
> > > > > set writeback on
> > > > > xquery file:write('doc.xml', )
> > > > > xquery insert node  into doc('doc.xml')/*
> > > > > xquery fetch:xml('doc.xml')
> > > > >
> > > > > Maybe we’ll merge these two options in a future release of BaseX.
> > > > > Until then, just add an EXPORTER option to your .basex file with the
> > > > > identical serialization parameters.
> > > > >
> > > > > Best
> > > > > Christian
> > > > >
> > > > > [1] http://docs.basex.org/wiki/Options#WRITEBACK
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Dec 12, 2018 at 6:35 PM Christian Grün
> > > > >  wrote:
> > > > > >
> > > > > > > Any serialization parameter gets ignored from the command line 
> > > > > > > with
> > > XQuery
> > > > > Update.
> > > > > > >
> > > > > > > Has this been solved? Is this a bug?
> > > > > >
> > > > > > XML declarations are never stored in the database. Instead, they 
> > > > > > will
> > > > > > be added during serialization. So I’m not exactly on what you just
> > > > > > tried to do. Could you please give some step-by-step instructions?

Re: [basex-talk] Command line options not working with update operation on input file

2018-12-13 Thread Zimmel, Daniel

I was referring to XQuery update writeback, chop option is set to false:

$ basex.bat -u -w -c"set exporter omit-xml-declaration=no,indent=no" -itest.xml 
update.xquery

update.xquery:

for $c in doc(document-uri())//kennung
return 
(replace value of node $c with "REPLACED")

test.xml:



test
 eins

  



Result


REPLACEDeinsTestfettkursivText
 z.B. am 01.02.2018 Text.


Saxon does not change indentation (with declare option saxon:output "indent=no")

> -Ursprüngliche Nachricht-
> Von: Christian Grün [mailto:christian.gr...@gmail.com]
> Gesendet: Donnerstag, 13. Dezember 2018 12:41
> An: Zimmel, Daniel
> Cc: BaseX
> Betreff: Re: [basex-talk] Command line options not working with update 
> operation
> on input file
> 
> > The only thing is I prefer the Saxon default about not changing anything on
> already existing indentation (BaseX is deleting any indentation with 
> indent=no),
> but I am aware this is implementation-defined.
> 
> Existing identation is actually preserved by both BaseX and Saxon:
> 
> declare namespace output = 'http://www.w3.org/2010/xslt-xquery-serialization';
> declare option output:indent 'no';
> parse-xml('
> 
> ')
> 
> Were you referring to documents with chopped whitespaces?
> 
> Cheers
> Christian
> 
> 
> 
> > > -Ursprüngliche Nachricht-
> > > Von: Christian Grün [mailto:christian.gr...@gmail.com]
> > > Gesendet: Mittwoch, 12. Dezember 2018 18:47
> > > An: Zimmel, Daniel
> > > Cc: BaseX
> > > Betreff: Re: [basex-talk] Command line options not working with update
> operation
> > > on input file
> > >
> > > …I think I got what you were trying to achieve! There is a separate
> > > option for exporting databases and writing updates. This may be more
> > > obvious now after another updates of our documentation [1].
> > >
> > > Here is a little self-contained BaseX call that (as I hope)
> > > demonstrates the behavior:
> > >
> > > basex -u -c"set exporter omit-xml-declaration=no"
> > > "file:write('doc.xml', )" "insert node  into doc('doc.xml')/*"
> > > "fetch:xml('doc.xml')"
> > >
> > > 
> > > 
> > >   
> > > 
> > >
> > > You can also run the following command script ("basex script.bxs"):
> > >
> > > set exporter omit-xml-declaration=no
> > > set writeback on
> > > xquery file:write('doc.xml', )
> > > xquery insert node  into doc('doc.xml')/*
> > > xquery fetch:xml('doc.xml')
> > >
> > > Maybe we’ll merge these two options in a future release of BaseX.
> > > Until then, just add an EXPORTER option to your .basex file with the
> > > identical serialization parameters.
> > >
> > > Best
> > > Christian
> > >
> > > [1] http://docs.basex.org/wiki/Options#WRITEBACK
> > >
> > >
> > >
> > >
> > > On Wed, Dec 12, 2018 at 6:35 PM Christian Grün
> > >  wrote:
> > > >
> > > > > Any serialization parameter gets ignored from the command line with
> XQuery
> > > Update.
> > > > >
> > > > > Has this been solved? Is this a bug?
> > > >
> > > > XML declarations are never stored in the database. Instead, they will
> > > > be added during serialization. So I’m not exactly on what you just
> > > > tried to do. Could you please give some step-by-step instructions?

Re: [basex-talk] Command line options not working with update operation on input file

2018-12-13 Thread Zimmel, Daniel

Ah, now this makes it much more clear in the documentation, I just checked the 
wiki page history.

The only thing is I prefer the Saxon default about not changing anything on 
already existing indentation (BaseX is deleting any indentation with 
indent=no), but I am aware this is implementation-defined.

Thanks for the update,
Daniel

> -Ursprüngliche Nachricht-
> Von: Christian Grün [mailto:christian.gr...@gmail.com]
> Gesendet: Mittwoch, 12. Dezember 2018 18:47
> An: Zimmel, Daniel
> Cc: BaseX
> Betreff: Re: [basex-talk] Command line options not working with update 
> operation
> on input file
> 
> …I think I got what you were trying to achieve! There is a separate
> option for exporting databases and writing updates. This may be more
> obvious now after another updates of our documentation [1].
> 
> Here is a little self-contained BaseX call that (as I hope)
> demonstrates the behavior:
> 
> basex -u -c"set exporter omit-xml-declaration=no"
> "file:write('doc.xml', )" "insert node  into doc('doc.xml')/*"
> "fetch:xml('doc.xml')"
> 
> 
> 
>   
> 
> 
> You can also run the following command script ("basex script.bxs"):
> 
> set exporter omit-xml-declaration=no
> set writeback on
> xquery file:write('doc.xml', )
> xquery insert node  into doc('doc.xml')/*
> xquery fetch:xml('doc.xml')
> 
> Maybe we’ll merge these two options in a future release of BaseX.
> Until then, just add an EXPORTER option to your .basex file with the
> identical serialization parameters.
> 
> Best
> Christian
> 
> [1] http://docs.basex.org/wiki/Options#WRITEBACK
> 
> 
> 
> 
> On Wed, Dec 12, 2018 at 6:35 PM Christian Grün
>  wrote:
> >
> > > Any serialization parameter gets ignored from the command line with XQuery
> Update.
> > >
> > > Has this been solved? Is this a bug?
> >
> > XML declarations are never stored in the database. Instead, they will
> > be added during serialization. So I’m not exactly on what you just
> > tried to do. Could you please give some step-by-step instructions?

Re: [basex-talk] Command line options not working with update operation on input file

2018-12-12 Thread Zimmel, Daniel

I already tried setting serialization params in .basex, per command line switch 
and as declare variable. 
Any serialization parameter gets ignored from the command line with XQuery 
Update. 

I found an ancient report about this behaviour in the archives, but no solution:
https://mailman.uni-konstanz.de/pipermail/basex-talk/2011-May/001593.html

Has this been solved? Is this a bug?

What does work is using fn:put() with writing to document-uri() and supplying a 
serialization map as third parameter. 
Unfortunately, for unknown reasons Saxon fails with a third parameter for 
fn:put(), even with XQuery 3.1 enabled (the BaseX implementation seems more 
correct after reading the specification).
Right now, I would need a variant when I want to run my script in both BaseX 
commandline and Oxygen with Saxon EE.

Daniel

> -Ursprüngliche Nachricht-
> Von: Christian Grün [mailto:christian.gr...@gmail.com]
> Gesendet: Mittwoch, 12. Dezember 2018 17:04
> An: Zimmel, Daniel
> Cc: BaseX
> Betreff: Re: [basex-talk] Command line options not working with update 
> operation
> on input file
> 
> Hi Daniel,
> 
> > Unfortunately, I still have not found a way to force BaseX NOT to delete my 
> > xml
> declaration with any update operation on a given input file.
> 
> The XML declaration is a part of the serialization process. You can
> enable it by adding…
> 
> SERIALIZER=omit-xml-declaration=no,indent=no
> 
> …to your .basex configuration file. I have added indent=no, as this is
> usually the best option when preserving whitespaces.
> 
> > - when CHOP=false in the .basex file and -w switch on the command line,
> whitespace gets chopped (not expected!)
> 
> The command-line help should probably be revised. With -w, the value
> of the option is actually toggled. I have updated our documentation to
> make this more obvious [1]. You can use -v to see which value will be
> assigned.
> 
> Best,
> Christian
> 
> [1] http://docs.basex.org/wiki/Command-Line_Options
> 
> 
> > > -Ursprüngliche Nachricht-----
> > > Von: Christian Grün [mailto:christian.gr...@gmail.com]
> > > Gesendet: Freitag, 7. Dezember 2018 17:41
> > > An: Zimmel, Daniel
> > > Cc: BaseX
> > > Betreff: Re: [basex-talk] Command line options not working with update
> operation
> > > on input file
> > >
> > > Hi Daniel.
> > >
> > > The argument order is important in BaseX; you could try the following 
> > > variant:
> > >
> > >   basex.bat -u -w -ifile.xml -sindent=no update.xquery
> > >
> > > If this doesn’t help, you could append the CHOP=true in your .basex
> > > configuration file.
> > >
> > > Hope this helps
> > > Christian
> > >
> > >
> > >
> > > On Fri, Dec 7, 2018 at 3:40 PM Zimmel, Daniel 
> > > wrote:
> > > >
> > > > Hello,
> > > >
> > > > can anyone give me a hint on the behaviour of the command line options 
> > > > in
> > > BaseX 9.0.2?
> > > >
> > > > When I try an update operation on a file given with "-i", neither the 
> > > > option "-
> w"
> > > nor any serialisation option gets evaluated; while the update operation 
> > > with "-
> u" is
> > > working correctly, my whitespace gets happily chopped.
> > > >
> > > > CCL:
> > > > $ basex.bat -ifile.xml -sindent=no -u -w update.xquery
> > > >
> > > > XQ:
> > > > for $c in doc(document-uri())//element
> > > > return replace value of node $c with "TEST"
> > > >
> > > > A workaround is using fetch:xml() with document-uri(), which does leave
> any
> > > whitespace intact and does no indentation (no parameters necessary).
> > > >
> > > > declare variable $doc := document-uri();
> > > > for $c in fetch:xml($doc)//element
> > > > return replace value of node $c with "TEST"
> > > >
> > > > Unfortunately my requirement is I cannot use specific BaseX-functions in
> this
> > > specific update script.
> > > >
> > > > My XML:
> > > >
> > > > 
> > > >   
> > > > Kennung
> > > >  eins
> > > > 
> > > >  Test
> fett
> > > kursiv Text Text.
> > > >   
> > > > 
> > > >
> > > > Thanks, Daniel
> > > >
> > > >

Re: [basex-talk] Command line options not working with update operation on input file

2018-12-10 Thread Zimmel, Daniel

Thank you Christian, changing the argument order does help indeed with the 
whitespace! The documentation is slightly vague about this.

Unfortunately, I still have not found a way to force BaseX NOT to delete my xml 
declaration with any update operation on a given input file.
For example if I use the same operation in Oxygen, not omitting the declaration 
is the default behaviour in Saxon EE.

But with BaseX, while I could use fn:put() with a map of serialisation 
parameters in the return statement, this does not help much because I can not 
use BaseX functions in my script :-(
Is changing this (default) outside the scope of an update script not possible?

BTW I noticed some strange behaviour (this is possibly a bug?)
- when CHOP=false in the .basex file and -w switch not set, whitespace does not 
get chopped (expected) 
- when CHOP=false in the .basex file and -w switch on the command line, 
whitespace gets chopped (not expected!)

Thanks, Daniel

> -Ursprüngliche Nachricht-
> Von: Christian Grün [mailto:christian.gr...@gmail.com]
> Gesendet: Freitag, 7. Dezember 2018 17:41
> An: Zimmel, Daniel
> Cc: BaseX
> Betreff: Re: [basex-talk] Command line options not working with update 
> operation
> on input file
> 
> Hi Daniel.
> 
> The argument order is important in BaseX; you could try the following variant:
> 
>   basex.bat -u -w -ifile.xml -sindent=no update.xquery
> 
> If this doesn’t help, you could append the CHOP=true in your .basex
> configuration file.
> 
> Hope this helps
> Christian
> 
> 
> 
> On Fri, Dec 7, 2018 at 3:40 PM Zimmel, Daniel 
> wrote:
> >
> > Hello,
> >
> > can anyone give me a hint on the behaviour of the command line options in
> BaseX 9.0.2?
> >
> > When I try an update operation on a file given with "-i", neither the 
> > option "-w"
> nor any serialisation option gets evaluated; while the update operation with 
> "-u" is
> working correctly, my whitespace gets happily chopped.
> >
> > CCL:
> > $ basex.bat -ifile.xml -sindent=no -u -w update.xquery
> >
> > XQ:
> > for $c in doc(document-uri())//element
> > return replace value of node $c with "TEST"
> >
> > A workaround is using fetch:xml() with document-uri(), which does leave any
> whitespace intact and does no indentation (no parameters necessary).
> >
> > declare variable $doc := document-uri();
> > for $c in fetch:xml($doc)//element
> > return replace value of node $c with "TEST"
> >
> > Unfortunately my requirement is I cannot use specific BaseX-functions in 
> > this
> specific update script.
> >
> > My XML:
> >
> > 
> >   
> > Kennung
> >  eins
> > 
> >  Test fett
> kursiv Text Text.
> >   
> > 
> >
> > Thanks, Daniel
> >
> >

[basex-talk] Command line options not working with update operation on input file

2018-12-07 Thread Zimmel, Daniel

Hello,

can anyone give me a hint on the behaviour of the command line options in BaseX 
9.0.2?

When I try an update operation on a file given with "-i", neither the option 
"-w" nor any serialisation option gets evaluated; while the update operation 
with "-u" is working correctly, my whitespace gets happily chopped.

CCL:
$ basex.bat -ifile.xml -sindent=no -u -w update.xquery

XQ:
for $c in doc(document-uri())//element
return replace value of node $c with "TEST"

A workaround is using fetch:xml() with document-uri(), which does leave any 
whitespace intact and does no indentation (no parameters necessary).

declare variable $doc := document-uri();
for $c in fetch:xml($doc)//element
return replace value of node $c with "TEST"

Unfortunately my requirement is I cannot use specific BaseX-functions in this 
specific update script.

My XML:


  
Kennung
 eins

 Test fett 
kursiv Text Text.
  


Thanks, Daniel

[basex-talk] Schema 1.1 validation and Xerces cta-full-xpath-checking?

2018-11-19 Thread Zimmel, Daniel

Hi,

is there any chance BaseX will support the cta-full-xpath-checking option (see 
https://xerces.apache.org/xerces2-j/features.html)? Without it, schema 1.1 
validation is limited.

I could not find any switches to configure Xerces features, but did find 
someone mentioning it from two years ago:
https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg08446.html

Or have I missed something in the documentation?

Regards, Daniel

76 matches

Mail list logo