Re: [basex-talk] BaseX GraalVM native-image

[Majewski, Steven Dennis (sdm7g)]

Maybe leave the server running and submit scripts from one of the non java 
clients ? 
( I haven’t used any of the other language clients myself, so no experience 
here. ) 

— Steve. 


> On Dec 1, 2022, at 3:08 AM, dli...@gmail.com wrote:
> 
> Hello Everyone,
> Didn't anybydy try and is that possible to convert the BaseX JAR to a
> "native-image" (using GraalVM's convertor or any other tool maybe)?
> The reason: I have to use BaseX in scripts and Makefiles as an XQuery
> engine and have to call it often. The problem is the JVM startup time
> affects the performance significantly when it has to start multiple
> times.
> I tried just naïvely running the GraalVM's native-image convertor
> against the BaseX103.jar file and got errors with long backtraces (that
> I didn't quite understand to be honest).
> Did anybody try that, maybe?
> Best regards
> 
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] GUI doesn't quit on Mac

[Majewski, Steven Dennis (sdm7g)]

It looks like it does exit, after a short delay, when closed by clicking the 
red button on the window, instead of quoting from the menu. 

— Steve. 


> On Nov 8, 2022, at 3:04 PM, Christian Grün  wrote:
> 
> Thanks for the observation. Mac support was improved in general with Java 11, 
> but it still requires some OS-specific tweaks in the code. Our current 
> solution may not be complete enough to achieve the expected default behavior 
> [1].
> 
> I’m no Mac user, but does it make a difference if the GUI is quit a) via the 
> menu or b) the red icon?
> 
> [1] 
> https://github.com/BaseXdb/basex/blob/main/basex-core/src/main/java/org/basex/gui/layout/BaseXLayout.java#L584
>  
> <https://github.com/BaseXdb/basex/blob/main/basex-core/src/main/java/org/basex/gui/layout/BaseXLayout.java#L584>
> 
> 
> 
> Joe Wicentowski mailto:joe...@gmail.com>> schrieb am Di., 
> 8. Nov. 2022, 20:54:
> I've been experiencing this problem too on macOS (both 12 Monterey and now 13 
> Ventura), installed via Homebrew, and started via the `basexgui` command line 
> command.
> 
> On Tue, Nov 8, 2022 at 1:02 PM Majewski, Steven Dennis (sdm7g) 
> mailto:sd...@virginia.edu>> wrote:
> I’ve been having a problem for a while where after attempting to quit the Mac 
> GUI, it doesn’t actually exit and has to be force quit. I’m sorry I wasn’t 
> able to track how this tracks with MacOS / JDK / BaseX updates and when it 
> actually started happening. Wondering if anyone else has had this issue, and 
> whether it’s Mac specific or JDK version specific. 
> 
> 
> MacOS 12.6.1 but problem goes back at least to earlier 12.6 versions
> Openjdk from home-brew install — currently openjdk/19.0.1, but was happening 
> on earlier versions. 
> And the current and last several versions of BaseX 
> 
> — Steve M. 
> 
> 



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] GUI doesn't quit on Mac

[Majewski, Steven Dennis (sdm7g)]

I’ve been having a problem for a while where after attempting to quit the Mac 
GUI, it doesn’t actually exit and has to be force quit. I’m sorry I wasn’t able 
to track how this tracks with MacOS / JDK / BaseX updates and when it actually 
started happening. Wondering if anyone else has had this issue, and whether 
it’s Mac specific or JDK version specific. 


MacOS 12.6.1 but problem goes back at least to earlier 12.6 versions
Openjdk from home-brew install — currently openjdk/19.0.1, but was happening on 
earlier versions. 
And the current and last several versions of BaseX 

— Steve M. 




smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Java 8 and XML Catalog

[Majewski, Steven Dennis (sdm7g)]

BTW: Just to note that Saxon 11.3 has been working fine for me under Basex 9.x,
But it requires including the included xmlresolver files in the class path:

export 
CLASSPATH=/usr/local/Cellar/saxon/11.3/libexec/saxon-he-11.3.jar:/usr/local/Cellar/saxon/11.3/libexec/lib/*.jar

But I haven’t done any special testing using the resolver from either Saxon or 
BaseX — 
Just running my current development code without noticing any problems. 

( If you only include saxon-he-11.3.jar, xslt:processor() and xslt:version() 
will make it appear that it’s working under BaseX, but xslt:transform() will 
fail on not finding resolver classes. ) 


I’ve had my share of resolver issues in the past, and happily discovered that 
by upgrading JDK, I could get resolver to work in some apps without any code 
change, just by adding 
'-Djavax.xml.catalog.files=…’ to the command line. But I spent a lot of effort 
trying to get resolver to work in cases where it apparently wasn’t intended to 
work ( like for XInclude’s ). 
I think Norm’s replacement does try to provide a lot more functionality — I’ll 
have to try intentionally using it!


 
— Steve M.


> On Jul 7, 2022, at 5:26 AM, Christian Grün  wrote:
> 
> Hi Vincent,
> 
> XML catalogs and Java 8 are an ungrateful combination indeed. As you
> know, support for catalogs will improve a lot with BaseX 10, as JDK 11
> comes with its own catalog resolver. With BaseX 10, we’ll also support
> Norm’s XML resolver, as you’ve already discovered (we’ll update our
> documentation soon).
> 
> Due to limited resources, we decided to focus on the new version
> exclusively. It’s interesting to hear, though, that your code is
> working with BaseX 9 and Java 17. Maybe it’s the inbuilt XML resolver
> that’s used by Java 17, no matter if an external resolver is added to
> the classpath? All I can do is guess …
> 
> Maybe JDK developers would be able to answer that question if we
> managed to create some self-contained code in Java?
> 
> Best,
> Christian



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Faceted searching and browsing ?

[Majewski, Steven Dennis (sdm7g)]

Yes - I mentioned index:facets() in the original post. 
I was not clear to me if or how that info could be used for this problem. 
Do you have any tips or examples ? 

I’m thinking that creating some sort of auxiliary index may be required, and 
wondering if others have done this and how they chose to do it, or the pros and 
cons of different approaches: generating an XML mapping to query directly in 
BaseX vs building an SQL table to query using the BaseX SQL module vs building 
Solr docs from basex and querying Solr for document lists. 

— Steve M. 

> On Jun 6, 2022, at 2:59 AM, Kristian Kankainen  wrote:
> 
> Hi,
> 
> Did you take a look at the Index module, especially the facet function [1]?
> 
> [1]: https://docs.basex.org/wiki/Index_Module#index:facets
> 
> Best regards,
> Kristian Kankainen
> 
>> On 6. Jun 2022, at 02:12, Liam R. E. Quin  wrote:
>> 
>> On Sun, 2022-06-05 at 21:45 +, Majewski, Steven Dennis (sdm7g)
>> wrote:
>>> 
>>> Is it possible to do faceted browsing with BaseX ? 
>> why wouldn't it be?
>> 
>> If you are having performance problems, it may help to maintain a
>> surrogate document in BaseX that just has the facet information, so you
>> don't have to search for it and collate it each time.
>> 
>> liam
>> 
>> -- 
>> Liam Quin, https://www.delightfulcomputing.com/
>> Available for XML/Document/Information Architecture/XSLT/
>> XSL/XQuery/Web/Text Processing/A11Y training, work & consulting.
>> Barefoot Web-slave, antique illustrations:  http://www.fromoldbooks.org
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] XML Catalog and xslt:transform()

[Majewski, Steven Dennis (sdm7g)]

I’ve also been struggling with catalogs in baseX and some other programs.

To amplify what Vincent already wrote, I did discover I was able to enable 
catalog support for DTDs and character entity files in XTF ( which uses a 
rather old version of Saxon ) without any source code modifications by:
[1] running it using Java 11 or 12
[2] passing the catalog file in my startup script by passing it in the startup 
command with:
-Djavax.xml.catalog.files=\"file://$home/WEB-INF/uvateip4-catalog.xml\” 


I was hoping this would also fix another issue I had with the same program: 
I have a number of XInclude files which are all specified with http: URLs. 
The servers those pointed to were all reconfigured to redirect those links to 
https: 
Opening those http: links using the browser follows the redirect, but when the 
java 
Parser is set to resolve XInclude links, it ignores the HTTP redirect and 
inserts the 
HTML redirect message instead of the XML fragment at the redirect location. 

I had hoped that could be fixed by using catalog to redirect using the same 
method above,
But no luck — it seems that there are different resolution pathways for DTDs & 
Charents,
xsl:import & xsl:includes and XInclude links and probably other sorts of URLs 
in documents.

Trying to follow the Saxon documentation on this to see if there is a different 
fix for this specific issue. 

In the mean time, I’m fixing up the result by replacing the HTML redirect 
messages with the correct XML fragment by modifying the URL added in the @base 
attribute by the XInclude processor: 




 





I’ve been trying to move some of the indexing I’m doing in XTF to BaseX, and 
I’m not clear on exactly what is working both with using the XInclude 
processing and with resolving entities defined in DTDs - with or without 
catalogs. (More on this perhaps later - some issues I’ve only just noticed 
today, so I’m not sure it’s not user error! ) 


— Steve M.



> On Jun 2, 2022, at 4:10 PM, Lizzi, Vincent 
>  wrote:
> 
> Hi Daniel and Gerrit,
>  
> If you are able to use Java version 11 or higher, it might be of use to try 
> the XML Catalog support that comes built in with Java. This ticket comment 
> has some details and an example for configuring Java and BaseX to use the 
> same XML Catalog:
>  
> https://github.com/BaseXdb/basex/issues/1903#issuecomment-1108822028 
> 
>  
> I’m not sure if this is relevant for your situation, but I’ve read somewhere 
> (although I can’t put my hands on the source right now) that Saxon uses the 
> XML Catalog for resolving URIs only in certain contexts. For example, a DTD 
> DOCTYPE can be resolved using an XML Catalog, but the function fn:json-doc() 
> does not use an XML Catalog.
>  
> Cheers,
> Vincent
>  
> _
> Vincent M. Lizzi
> Head of Information Standards | Taylor & Francis Group
> vincent.li...@taylorandfrancis.com 
>  
> 
> Information Classification: General
> From: BaseX-Talk  On Behalf Of 
> Zimmel, Daniel
> Sent: Thursday, June 2, 2022 10:57 AM
> To: 'Imsieke, Gerrit, le-tex' ; 
> basex-talk@mailman.uni-konstanz.de
> Subject: Re: [basex-talk] XML Catalog and xslt:transform()
>  
> I see, thanks Gerrit and Christian for the insight. This *does* sound 
> wickedly unfunny.
> 
> OK if I actually do not need to be able to parse the DTD wouldn't the simple 
> workaround be:
> 
> fetch:xml('file:///C:/temp/catalog/dokument.xml')
> => xslt:transform('transform.xsl')
> 
> At least this is what works here, resulting in a new document node and 
> trashing the DTD declaration.
> 
> Daniel
> 
> -Ursprüngliche Nachricht-
> Von: BaseX-Talk  > Im Auftrag von Imsieke, 
> Gerrit, le-tex
> Gesendet: Donnerstag, 2. Juni 2022 16:40
> An: basex-talk@mailman.uni-konstanz.de 
> 
> Betreff: Re: [basex-talk] XML Catalog and xslt:transform()
> 
> As a workaround, you might be able to read the documents using doc() in 
> XQuery (this might work with the help of the catalog, in contrast to
> doc() from within XSLT/Saxon) and pass them to xslt:transform() in some way. 
> “Some way” isn’t easy, either, since xslt:transform() still relies on JAXP, 
> and you can’t pass arbitrary XDM items such as whole documents or maps as 
> stylesheet parameters (or can you? $params as map(*)? doesn’t rule this out, 
> but I doubt that a parameter may have another map as value and arrive safely 
> at the stylesheet). So you might need to wrap all inputs in a single 
> top-level element, which of course prevents you from letting the XSLT 
> stylesheet decide which resource to load dynamically, and you might need to 
> change matching patterns.
> But switching to XDM and implementing XPath 3.1’s fn:transform() function 
> that would allow to was 

Re: [basex-talk] Constructing and querying map structures in a dynamic way

[Majewski, Steven Dennis (sdm7g)]

Note that the optional options argument in map:merge can change the way it 
handles duplicate keys.
I’ve used this to create maps of normalized document ids from two different 
collections to de-dup,
Where the ids SHOULD be uniq within a collection, but actually aren’t, or are 
missing. 

https://www.w3.org/TR/xpath-functions-31/#func-map-merge 


( Next step was to split the uniq and non-uniq ones to handle separately. ) 

declare variable $OAI := map:merge( for $d in $ORG_OAI 
  return map:entry( local:normalize( local:unitid($d )), $d ), 
map{'duplicates':'combine'}); 
declare variable $VH := map:merge( for $d in $ORG_VH
  return map:entry( local:normalize( local:unitid($d )), $d ), 
map{'duplicates':'combine'});


However, after discussion on this list about SQL join equivalents, I discovered 
I could do something almost equivalent with a for/join, and that full text 
search tokenization gives me a similar way of normalizing IDs: 


for $VH in collection( $R($ORG)('vh'))/ead/archdesc/did/unitid, 
$OAI in collection( $R($ORG)('oai'))/ead/archdesc/did/unitid[ . 
contains text  {string($VH)}]
order by $VH


I think the map version may be a bit faster, but I haven’t actually measured 
And they are doing things a bit differently, as I’m not removing the internal 
duplicates,
But using the order by to group multiple matches together to make it visible in 
the report. 


I would also note that dealing with messy data, often with hidden encoding 
errors in documents,
Is the reason I was asking in a previous thread about using the result of one 
script as context 
for other queries. Sometimes this ends up being a rather exploratory process, 
so that has come
in very handy. 

— Steve M.


> On May 25, 2022, at 12:13 PM, Hans-Juergen Rennau  wrote:
> 
> Hi Markus,
> 
> although I do not quite understand your description (why should I care for 
> the representation of the map I construct?), it may be that one piece of 
> information is useful to you, as it is not obvious from the spec: it's about 
> how to construct a map dynamically, that is, 
> 
> (1) when the keys are not known beforehand, 
> (2) and/or when the entry values are assigned in "iterations" of a FLWOR 
> expression
> 
> (Especially (2) might be what puzzles you.)
> 
> (1) and (2) can't be accomplished when using the map constructor expression, 
> like this: basex "map{'x':1, 'y': false()}" 
> 
> The trick consists of using a combination of the functions map:merge() [1] 
> and map:entry() [2].
> 
> So summarize, the code constructing the map entries is the argument of 
> function map:merge().This code may for example be (most often is) a FLWOR 
> expression. In this code, each entry is constructed by a call of function 
> map:entry(). For a tiny example see below.
> 
> Kind regards,
> Hans-Jürgen
> 
> PS: Tiny example demonstrating the construction of maps via map:merge() & 
> map:entry().
> 
> (: Creates a map with one entry for each element name
>encountered in the input document. The entry value
>is the number of elements with that name. As the
>map keys are not known in advance, map:merge() is
>used.
>  :)
> map:merge(
> for $elem in //*
> group by $name := $elem/local-name(.)
> return map:entry($name, count($elem))
> )
> 
> [1] https://www.w3.org/TR/xpath-functions-31/#func-map-merge 
> 
> [2] https://www.w3.org/TR/xpath-functions-31/#func-map-entry 
> 
> 
> 
> 
> 
> 
> 
> Am Mittwoch, 25. Mai 2022, 16:59:48 MESZ hat Markus Elfring 
>  Folgendes geschrieben:
> 
> 
> Hello,
> 
> I would like to construct a map structure based on a dynamic selection by
> the means of a FLWOR expression.
> https://www.w3.org/TR/2017/REC-xquery-31-20170321/#id-maps 
> 
> 
> The keys and associated values should be presented as a CSV-like file by
> a subsequent data processing step.
> 
> Can it become easier to find existing XQuery script examples for such an use 
> case?
> 
> Regards,
> Markus



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] [basex-announce] BaseX 9.7.1: Tweaks, Fixes, Features

[Majewski, Steven Dennis (sdm7g)]

I still had BaseX971-20220411.133348.zip ( 9.7.1 beta) sitting around to try
And I don’t get that error using it.   


> On May 3, 2022, at 11:12 AM, sdm7g  wrote:
> 
> 
> I’m also seeing a new error crop up with 9.7.1 
> 
> Could this be connected ? 
> 
> ( I’ll try to revert to previous version to make sure that there hasn’t been 
> a change at the source URL that would cause a change in result. )   
> 
> WTJU$ basex 'html:doc( "https://www.wtju.net/recent-shows/ 
> " )' 
> Stopped at /Users/sdm7g/Desktop/WTJU/, 1/9:
> [FODC0002] URI 'https://www.wtju.net/recent-shows/ 
> ' points to directory.
> WTJU$ 
> WTJU$ curl  -I https://www.wtju.net/recent-shows/ 
> 
> HTTP/2 200 
> server: nginx/1.10.3
> date: Tue, 03 May 2022 15:10:52 GMT
> content-type: text/html; charset=UTF-8
> x-pingback: https://www.wtju.net/xmlrpc.php 
> link: >; 
> rel="https://api.w.org/ "
> link: >; 
> rel=shortlink
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Basexgui Feature suggestion

[Majewski, Steven Dennis (sdm7g)]

Reading the docs closer 
https://docs.basex.org/wiki/Graphical_User_Interface#Editor 
<https://docs.basex.org/wiki/Graphical_User_Interface#Editor> 
It looks like the best shortcut would be saving the results to an .xml file, 
and using the feature of right clicking on the file to bind to ‘.’ , however 
that doesn’t appear to work as described. 

I get a shorter menu of options that what’s shown in that doc: 

Open
Open Externally
Run Tests  (greyed out and inactive) 
Refresh 
Copy Path

No “set as context” option.

I’m running BaseX 9.7 on a Mac. 
Is this difference from the docs due to a Mac / PC difference, or has this 
capability changed and the docs not updated? 


I’ve also realized, after thinking about some more use cases, that due to the 
fact that results could be things other than nodes, like maps or arrays, or 
could be serialized differently, that there may be more issues than I had 
initially thought with simply binding results to context. 

I would say that for my use cases, a few extra clicks is less trouble than 
managing temporary edits to files and reverting them back again. 

— Steve M.


> On Apr 7, 2022, at 1:25 PM, Tamara Marnell  wrote:
> 
> Hi Steven,
> 
> To save a few clicks, you can create a new database directly from your 
> results in XQuery, without saving them to a file first. Instead of returning 
> the results outright, assign them to a variable to pass as the input to 
> db:create() in the return, with a made-up file name for the path.
> 
> let $results := {
>   [Your original return in here]
> }
> return db:create('my_results_db', $results, 'my_results.xml') 
> 
> Then you can run further queries using db:open('my_results_db')/results, and 
> DROP my_results_db in the command input bar when you don't need it anymore.
> 
> This isn't to say a new feature isn't a good idea, just that there's a way 
> you can streamline your workflow before that feature exists!
> 
> -Tamara
> 
> 
> 
> On Thu, Apr 7, 2022 at 9:49 AM Majewski, Steven Dennis (sdm7g) 
> mailto:sd...@virginia.edu>> wrote:
> 
> It would be handy if there was a way to bind results to current context for 
> additional inspection/investigation/query of results. Currently, it seems, 
> you would have to save results to a file and then create database from that 
> file to make an additional query on results. 
> That binding would be available from editor or input bar. 
> 
> Or perhaps alternatively, it could include an option to base visualizations 
> on results instead of open database — although I’m guessing the former would 
> be easier to implement than the latter. 
> 
> — Steve Majewski
> 
> 
> 
> -- 
> 
> Tamara Marnell
> Program Manager, Systems
> Orbis Cascade Alliance (orbiscascade.org <https://www.orbiscascade.org/>)
> Pronouns: she/her/hers



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] Basexgui Feature suggestion

[Majewski, Steven Dennis (sdm7g)]

It would be handy if there was a way to bind results to current context for 
additional inspection/investigation/query of results. Currently, it seems, you 
would have to save results to a file and then create database from that file to 
make an additional query on results. 
That binding would be available from editor or input bar. 

Or perhaps alternatively, it could include an option to base visualizations on 
results instead of open database — although I’m guessing the former would be 
easier to implement than the latter. 

— Steve Majewski



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Add a comment to a backup?

[Majewski, Steven Dennis (sdm7g)]

It’s in ZipFile.getComment()

https://docs.oracle.com/javase/8/docs/api/java/util/zip/ZipFile.html#getComment--
 


— Steve M. 

> On Mar 30, 2022, at 12:17 PM, Christian Grün  
> wrote:
> 
>> I'm wondering if zip's `setComment` might be useful here.
> 
> Interesting hint.
> 
> I have discovered the ZipOutputStream.setComment function. There
> doesn’t seem to be an equivalent ZipInputStream.getComment function,
> though. Maybe I overlooked something?
> 
> 
> 
>> On Wed, Mar 30, 2022 at 1:44 PM Christian Grün
>>  wrote:
>>> 
>>> Hi Jonathan,
>>> 
>>> Currently, it’s not possible to attach comments to backups. I’ll have
>>> some more thoughts on your feature request. Maybe we could add an info
>>> file in the zipped archive and partially unzip it when listing
>>> available backups.
>>> 
>>> All the best,
>>> Christian
>>> 
>>> 
>>> On Wed, Mar 30, 2022 at 12:26 AM Jonathan Robie
>>>  wrote:
 
 I have been making backups before doing particularly complex things to  my 
 treebanks, and I find myself writing down information about what stage of 
 processing a given backup corresponds to.
 
 "after replacing subtrees for missing compounds"
 
 I wish I could associate these strings with backups in BaseX so I can more 
 easily know which one I would restore if something went wrong.
 
 Jonathan



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Feature request - file listing

[Majewski, Steven Dennis (sdm7g)]

Unnecessary and non-standard ( file: functions are not XQuery standard, but are 
EXPath functions). 

Use:  file:children( file:resolve-path( $folder ) )

If you give file:children an absolute path, that will be included in results, 
so just resolve the relative folder path first. 

— Steve M. 


> On Feb 11, 2022, at 1:40 PM, Hans-Juergen Rennau  wrote:
> 
> Dear BaseX people,
> 
> I have a feature request:
> 
> a function which does the same as file:list, but returns the full paths. (In 
> the explanation, I call it "file:list2")
> 
> Rationale: 
> (a) It is the full paths what we need in order to parse the files (doc(), 
> json:doc(), csv:doc(), html:doc()). 
> (b) The combination of file listing and parsing in a single expression is of 
> matchless elegance and expressiveness
> Just think about how we can extend this into a single 
> list+parse+navigate+report expression, like:
> 
> file:list2($folder, true(), $fname) ! doc(.) ! //returnCode[. ne 
> 0]/concat(base-uri(.), ': ', .)
> 
> (Wow! Is there any language or tool competing with this compactness and 
> readability?)
> 
> Currently, if I don't overlook something, I have to do this:
>file:list($dir, false(), $fname) ! concat($dir, '/', .) ! doc(.) => count()
> 
> I emphasize the psychological aspect. Every time I have the pleasure to give 
> a workshop about XQuery, I show off how elegant it is to parse a whole file 
> system tree into a forest of nodes in a single expression. And I am always 
> embarrassed about this clumsy fumbling montage of a path. It's as if Mozart 
> had fallen asleep while writing his xquery sonata, knocked over the inkwell 
> and now there is this splash smeared out on the score. You would be my heroes 
> if you can spare us that.
> 
> Kind regards,
> Hans-Jürgen



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] BaseX 9.6: The Summer Edition

[Majewski, Steven Dennis (sdm7g)]

Thank you. 
But a head’s up for any Mac/Homebrew users: 

The BaseX Homebrew formula was not yet update, so I impatiently edited my local 
copy of the formula to 9.6
and I repeatedly got:

basex 9.5.2 is installed but outdated
==> Upgrading basex
  9.5.2 -> 9.6 

==> Downloading https://ghcr.io/v2/homebrew/core/basex/manifests/9.6
curl: (22) The requested URL returned error: 404

Error: Failed to download resource "basex_bottle_manifest"
Download failed: https://ghcr.io/v2/homebrew/core/basex/manifests/9.6


I googled for similar errors and tried various suggestions, none of which 
worked. 


Eventually, I tried just deleting the following function from the formula, and 
it successfully installed.

  bottle do
sha256 cellar: :any_skip_relocation, all: 
"29de8b648eba2d7d20a37ba168ae1757a4724eb43c5307d99291329518a06c46"
  end


I only hesitate to submit that patched formula to Homebrew because I don’t 
actually understand what I’ve done or why it works!  ;-) , but offering it here 
as a suggestion in case anyone else runs into that problem.


— Steve M. 




> On Aug 19, 2021, at 8:22 AM, Christian Grün  wrote:
> 
> Dear all,
> 
> We provide you with a new and fresh version of BaseX, our open source
> XML framework, database system and XQuery 3.1 processor:
> 
>  https://basex.org/
> 
> Apart from our main focus (query rewritings and optimizations), we
> have added the following enhancements:
> 
> XQUERY: MODULES, FEATURES
> - Archive Module, archive:write: stream large archives to file
> - SQL Module: support for more SQL types
> - Full-Text Module, ft:thesaurus: perform Thesaurus queries
> - Fulltext, fuzzy search: specify Levenshtein limit
> - UNROLLLIMIT option: control limit for unrolling loops
> 
> XQUERY: JAVA BINDINGS
> - Java objects of unknown type are wrapped into function items
> - results of constructor calls are returned as function items
> - the standard package "java.lang." has become optional
> - array arguments can be specified with the middle dot notation
> - conversion can be controlled with the WRAPJAVA option
> - better support for XQuery arrays and maps
> 
> WEB APPLICATIONS
> - RESTXQ: Server-Timing HTTP headers are attached to the response
> 
> For a more comprehensive list of added and updated features, look into
> our documentation (docs.basex.org) and check out the GitHub issues
> (github.com/BaseXdb/basex/issues).
> 
> Have fun,
> Your BaseX Team



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] [EXTERNAL] Re: Using other Java runtimes?

[Majewski, Steven Dennis (sdm7g)]

I’ve been running BaseX webapp under Tomcat using either OpenJDK 8 or 11 on 
Mac. 
But I just checked, and the BaseX GUI and command line programs, which I 
installed using HomeBrew 
use the OpenJDK installed by HomeBrew, so it appears I’ve been running that 
variously under OpenJDK 8,11,12 and now 15.

JAVA_HOME="/usr/local/opt/openjdk" exec 
"/usr/local/Cellar/basex/9.5.1/libexec/bin/basex"  "$@“

/usr/local/opt/openjdk@ -> ../Cellar/openjdk/15.0.2

— Steve M. 


> On Apr 17, 2021, at 2:15 PM, Peter Villadsen  
> wrote:
> 
> I have installed OpenJDK on one machine so far, and it seems to work well on 
> windows. Thanks to those who helped make these suggestions.
> 
> Best Regards
> 
> Peter Villadsen
> 
> -Original Message-
> From: Graydon  
> Sent: Friday, April 16, 2021 11:20 PM
> To: Peter Villadsen 
> Cc: BaseX 
> Subject: [EXTERNAL] Re: [basex-talk] Using other Java runtimes?
> 
> On Fri, Apr 16, 2021 at 09:39:46PM +, Peter Villadsen scripsit:
>> I was wondering what the options are for running BaseX on top of a runtime 
>> that is different from the Oracle one? I see Oracle is aggressively 
>> monetizing the Java runtime, but I know there are alternatives out there. 
>> Have any been tested with BaseX?
> 
> I can't talk about _tested_, but I've been using BaseX on Fedora for a decade 
> now, and it's been fine as Fedora shifted to OpenJDK pacakages to provide 
> Java.  Current happens to be openjdk 11.0.10 2021-01-19.
> 
> I'm also not running any server anything; this is pretty strictly command 
> line or gui with not that much data (at most single-digit GB) and 
> not-that-complex queries, so this isn't a comprehensive thing, but for at 
> least those purposes I can say BaseX works fine on the OpenJDK.
> 
> --
> Graydon Saunders  | graydon...@gmail.com Þæs oferéode, ðisses swá mæg.
> -- Deor  ("That passed, so may this.")



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] Bind context to stdin: -i-

[Majewski, Steven Dennis (sdm7g)]

I just thought I would share this, as it was non-obvious to me.
I wanted to use xpath expressions in a shell script pipeline, and was looking 
to see if there was a way to bind context to stdin. It wasn’t clear from the 
documentation, so I just tried the common convention of using “-“ and:

 Pipeline commands producing XML output “| basex -i-  “  followed by a path or 
query works! 


And it works better that the PERL xpath command. 

— Steve Majewski
 

smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] multi-line queries in the command line client

[Majewski, Steven Dennis (sdm7g)]

Use the GUI ( run “basexgui &” from the command line )!

You can run and edit commands from the top command line, and if they start to 
grow into longer expressions, copy and paste them into the file window, where 
you can click on the run button.
Plus it will give you some syntax guidance while you’re typing. 

Really handy when you are learning XQuery. 

— Steve. 


> On Aug 20, 2020, at 5:07 PM, Bill Osmond  wrote:
> 
> Hi all,
> 
> First, thanks for making BaseX! I'm in the process of learning it (and 
> XQuery) after an ill-fated attempt to coerce complicated XML data into a 
> relational database, and I'm optimistic about the prospects of success.
> 
> But there is one thorn in my side: it seems like xquery commands have to all 
> be on a single line when run from the command line client. Multiple 
> lines/indentation goes a long way towards helping me mentally structure the 
> queries I'm (trying to) run, but it seems I have to join them together on a 
> single line in order to actually run them. I've tried the standard \ 
> character with no luck, and I couldn't find anything in the documentation 
> about it.
> 
> Is there any solution, or is that just the way it is?
> 
> Thanks,
> Bill



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Eclipse editor support and authentication

[Majewski, Steven Dennis (sdm7g)]

I haven’t tried Eclipse integration, but I’m using basex from Oxygen configured 
as in:

https://docs.basex.org/wiki/Integrating_oXygen 
 

You can then use BaseX in a transformation scenario. 
But the XQuery debugger is still restricted to Saxon only. 


Re: security and authentication: 

Have you looked at the sample DBA app or RESTXQ permissions:

https://docs.basex.org/wiki/DBA 

https://docs.basex.org/wiki/Permissions 



If you want to add some external authentication, you can run the basex war file 
under tomcat.
( And/or proxy thru Apache/nginx. ) 

— Steve M. 



> On May 24, 2020, at 9:37 PM, Ben Pracht  wrote:
> 
> Hi Folks,
> 
> Perhaps I'm using too new of an Eclipse (Photon), but I'm not able to 
> integrate Eclipse with BaseX.  I installed the trial Oxygen plugin, though I 
> don't want to pay for it.  Even then, it would not offer a BaseX datasource, 
> only one for ExistDB.  I tried to install XQDT, but it could not be installed 
> without downgrading many components, which I did not want to do.  
> 
> As an Eclipse person, using IntelliJ would not be very productive as I've 
> gotten used to the Eclipse keyboard shortcuts.
> 
> I'd like to create a server program based in BaseX with RESTful xquery based 
> services.  Initially, it would be a CRUD type application.  It doesn't need 
> high performance, just not terrible, and I'd like to be able to authenticate 
> into it.  I'm confident BaseX can handle the task, it's just getting an 
> editor capable of handling it.  Using the GUI that comes with BaseX is good 
> for small tasks, but not so good for larger applications.
> 
> It doesn't appear that there's much in the way of XQuery support in Eclipse 
> anymore, and even the XQDT if it could be installed doesn't support the newer 
> 3.1 type xquery (I believe I read).  Can someone recommend a good editor?  
> I'm using GVim for many small projects, though it takes work to get it to 
> work with larger projects efficiently (CTags helps for C/C++ code, but I 
> dont' think it works with XQuery).  
> 
> Separately, would it be possible to develop a secure BaseX based application 
> that users could log in, and authenticate into,  store delicate personal 
> information and have it be sufficiently secure?  Could it be at least as 
> secure as, say a Tomcat application?  Sorry to ask this question, I do app 
> development, but usually my role comes in after the user has been 
> authenticated using something like OAuth or other.
> 
> 
> Best Regards,
> Ben Prachtcom 



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Storing XML messages

[Majewski, Steven Dennis (sdm7g)]

Not completely clear what you’re trying to do, but:

1) you can’t have reserved characters like “<“ , “>” in an XML attribute, so 
you should use ,  entities — something like:



2) If you bind to the variable message with that statement, then you don’t need 
the ‘let $message” , but you do need to declare the variable as external, as 
it’s being bound with that variable element: 

declare variable $message external; 
. . . 


3) and then, since that variable is a text string, you can use 
Fn:parse-xml-fragment() to convert it into a node that you can insert somewhere:


insert node  fn:parse-xml-fragment($message) into . . . 

I’m not sure that the destination node for your insert into clause is going to 
do what you want, but I can’t tell without knowing what the initial value of 
the doc node is. 

I *think* this may answer your question, but if not, try to post what the 
initial value of that doc node is and what you want the final result to look 
like. 

— Steve M. 



> On May 14, 2020, at 3:44 PM, Ramzi Hammouda  
> wrote:
> 
> Hello BaseX team,
>  
> Using Resful API, I'm trying to make POST method update BaseX database, I 
> want to insert an xml node to an existing document calastone.xml in calastone 
> database , but the node contains '<', you replied to a previous email that I 
> should use fn:parse-xml to make this happen without error.
>  
> I tried using the function fn:parse-xml, I had success with using it in BaseX 
> GUI like this:
>  
> let $message := fn:parse-xml('CTN55')
> return insert nodes $message as last into doc("calastone/calastone.xml")
>  
> But when I tried to do the same with POST method from my application with 
> using this xquery:
>  
> string xquery = "let $message:= fn:parse-xml('CTN58')" +
> "return insert nodes $message as last into 
> doc(\"calastone/calastone.xml\")" +
> " fn:parse-xml('CTN58'))/>";
>  
> I get the error: "" (Line 1): Element type "variable" must be followed by 
> either attribute specifications, ">" or "/>"
>  
> Can you help on how to do this insertion?
>  
> Thanks.
>  
> 
> P Respectons ensemble l'environnement. N'imprimez ce message que si 
> nécessaire. Let's respect the environment together. Only print this message 
> if necessary.



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] Fwd: [xsl] [OT] New Slack workspace for XML discussions

[Majewski, Steven Dennis (sdm7g)]

FYI: 

> Begin forwarded message:
> 
> From: "Adam Retter adam.ret...@googlemail.com" 
> 
> Subject: [xsl] [OT] New Slack workspace for XML discussions
> Date: April 16, 2020 at 11:59:25 AM EDT
> To: xsl-l...@lists.mulberrytech.com
> Reply-To: 
> 
> Hopefully the moderators won't mind me posting this? I think it has
> value for readers of this list...
> 
> Just a quick heads up in-case anyone is interested, there is now a new
> vendor neutral Slack Workspace for discussing all things XML related -
> https://www.xml.com/news/2020-04-slack-workspace-for-the-xml-community/
> 
> Cheers Adam.
> -- 
> Adam Retter
> 
> skype: adam.retter
> tweet: adamretter
> http://www.adamretter.org.uk
> --~
> XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
> EasyUnsubscribe: http://lists.mulberrytech.com/unsub/xsl-list/504751
> or by email: xsl-list-un...@lists.mulberrytech.com
> --~--



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Should it be possible to declare a function in the client?

[Majewski, Steven Dennis (sdm7g)]

Please see my earlier message to the list, as I’m betting it’s the zero-length 
string that isn’t getting counted in R!  — Steve M.

> On Feb 28, 2020, at 5:45 AM, Ben Engbers  wrote:
> 
> Op 27-02-2020 om 22:03 schreef Majewski, Steven Dennis (sdm7g):
>> Also: is ‘(nbsp;)’ what you want as part of you regex to also catch the 
>> ampersand ?  
>> I’m just guessing your intent here. 
>> You could also try  ‘(\W|nbsp;)+’ - i.e. non-word, but I’m kind of 
>> assuming that it handles non-normalized unicode accented characters 
>> correctly and reads them as word chars and not delimiters. That would be, of 
>> course, the right thing, but I’ld probably test it first. 
>> 
>> — Steve. 
> 
> I just copied the regex-expression from this page
> "https://en.wikibooks.org/wiki/XQuery/Tag_Cloud; (using regex always
> gives me headaches ;-( ). But even after removing the "|[n][b][s][p][;]"
> from the regex, basexgui still returns 5843.
> 
> Ben
> 
> 
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Database specific xquery modules

[Majewski, Steven Dennis (sdm7g)]

Also, if you DO go the route of storing procedures in the database, you might 
consider whether you could express those procedures as XSLT stylesheets that 
you could call from XQuery using xslt:transform() function. Since XSLT is in 
XML syntax, it works a bit smoother for that scenario. XQuery modules would 
need to be stored in the DB as RAW or text.  XSLT modules are XML  — accessing 
them via the DB would return an XML node that is already parsed. I saw a 
significant  speedup pulling XSLT stylesheets from the DB as compared to using 
a filesystem URI. 

XQuery and xslt:transform work very well together: you can mix both methods in 
your code, as xslt:transform() is just another XQuery function that takes an 
input node or document and returns a result, usually as another XML node. 

The only downside is that your developers need to know both XQuery and XSLT. 
However, there is a lot of overlap between the two. 
Both use XPath and many of the same functions and the same data model, but a 
different syntax. 

— Steve M.



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Call for assistance : BaseX as a preprocessor ?

[Majewski, Steven Dennis (sdm7g)]

If you really want to read in all of the data as a single stream, I would 
suggest writing a preprocessor using SAX library ( from Python, Java or 
whatever language you want to use ) to break the Wikimedia stream into separate 
XML files for each page element, or else use the same language to do the 
streaming CSV conversion . 

However, for a file that large, you may have issues if there is a network 
interruption. 

Depending on how reliable you connection is, you might be better off 
downloading the separate chunks. It gives you easily recognizable restart 
points. 


Otherwise: Saxon can do streaming XSLT, but only with one of the paid license 
Enterprise versions. No idea if Saxon XQuery can also handle streaming input. 
Also, no idea if any of the non-Java versions of Saxon handle streaming. 


If all that is needed is to convert the XML stream into CSV records to dump 
into Postgres, I would probably use Python/SAX, but I wonder if Postgres is 
really a requirement, or if you can do your final queries in BaseX ?  If 
dumping everything in a BaseX database is just an intermediary step, then it’s 
probably not the most efficient way to go. 

— Steve M.


> On Feb 23, 2020, at 6:54 PM, maxzor <
> max...@maxzor.eu> wrote:
> 
> 
>> Do you mean stream a single large XML file ? A series of XML files, or 
>> stream a file thru a series of XQuery|XSLT|XPath transforms.
>> 
> Possibly poor wording, I meant read a large XML file and produce i.e. a csv 
> file.
>> I don’t believe BaseX uses a streaming XML parser, so probably can’t handle 
>> streaming a single large XML file and produce output before it’s parsed the 
>> complete file.
> Do you know of a streaming xml lib? other than StAX (no Java here :<)?
>> But it looks like, from the link in your stackoverflow post that the data is 
>> already sharded into a collection of separate XML files that each contain 
>> multiple  elements.
> 
> This is the alternative, instead of processing the monolithic multistream 
> file, I could crawl over the ~150MB bz2-compressed chunks.
> 
> Regards, Maxime
> 
> 



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] Broken links to papers on http://basex.org/about/publications/

[Majewski, Steven Dennis (sdm7g)]

Links to papers on http://basex.org/about/publications/ 
 are broken. 

It looks like the URL string is broken off early at the first whitespace 
character.

I can access directory http://files.basex.org/publications/ 
 

And it looks like the entire text in parenthesis should be the filename in the 
URL. 

— Steve M.



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Should it be possible to declare a function in the client?

[Majewski, Steven Dennis (sdm7g)]

I also note, that when I try to mock up something similar with one of my texts, 
the tokenize 
Seems to give me a zero length string at the start. 

It’s there in the output window of basexgui, in the first line, but easy to 
miss the fact that it’s significant whitespace in this context:

(tokenize(string-join(collection('BOV')[ends-with( db:path(.), '.tei' 
)][1]/TEI/text/body/p//text() ), '\W+' ) => distinct-values())[not(. = ( "of", 
"the", "in", "and", "at","by","to" ))] 


August
16
17
2015
Members
Board
Visitors
University
Virginia
met
Retreat
Open
Executive
Session
Forum
… 

But visible if I apply string-length to the sequence:

(tokenize(string-join(collection('BOV')[ends-with( db:path(.), '.tei' 
)][1]/TEI/text/body/p//text() ), '\W+' ) => distinct-values())[not(. = ( "of", 
"the", "in", "and", "at","by","to" ))]  ! string-length(.)

0
6
2
2
4
7
5
8
10
8
3
7
4
9
7
…

I wonder if that’s the semantic difference here. 

— Steve M.


> On Feb 27, 2020, at 3:43 PM, Majewski, Steven Dennis (sdm7g) 
>  wrote:
> 
> So, if the counts are different depending on who is counting ( R or BaseX ),
> The first question is : who is correct ? 
> ( And the 2nd question is probably: what do you mean by correct ? as the 
> semantics of XQuery sequences and whatever destination R datatype is being 
> counted may be slightly different. I don’t know R that well, but semantics of 
> XQuery sequences and arrays are rather different, for example. )
> 
> — Steve M.
> 
> 
>> On Feb 27, 2020, at 2:48 PM, Ben Engbers  wrote:
>> 
>> Op 27-02-2020 om 19:19 schreef Christian Grün:
>>> It’s difficult to understand what’s going on here. Could you please
>>> provide us self-contained queries without the R wrapper code?
>> 
>> Version 1:
>> 
>> import module namespace functx = 'http://www.functx.com';
>> (: Extract the text :)
>> let $txt := collection('IncidentRemarks/Incidents')/csv/record/INC_RM/text()
>> (: Convert to lower-case and tokenize :)
>> let $INC_RM := tokenize(lower-case(string-join($txt)),
>> '(\\s|[,.!:;]|[n][b][s][p][;])+')
>> (: Read Stopwords :)
>> let $Stoppers := doc('TextMining/Stopwoorden.txt')/text/line/text()
>> (: Remove Stopwords :)
>> let $Stop :=  functx:value-except($INC_RM, $Stoppers)
>> return $Stop"
>> 
>> My R-code first executes this as XQUERY and then calculates the length
>> of the returned list (=5842).
>> 
>> Version 2:
>> 
>> import module namespace functx = 'http://www.functx.com';
>> let $txt := collection('IncidentRemarks/Incidents')/csv/record/INC_RM/text()
>> let $INC_RM := tokenize(lower-case(string-join($txt)),
>> '(\\s|[,.!:;]|[n][b][s][p][;])+')
>> let $Stoppers := doc('TextMining/Stopwoorden.txt')/text/line/text()
>> let $Stop :=  functx:value-except($INC_RM, $Stoppers)
>> return count($Stop)
>> 
>> Returns the length of the sequence (counts 5843 words).
>> 
>> The '\\' in the regular expression is intentional (R-specific). With a
>> single '\' the query can be executed in BaseXGUI.
>> 
>> Does this help?
>> 
>> Ben
>> 
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Should it be possible to declare a function in the client?

[Majewski, Steven Dennis (sdm7g)]

So, if the counts are different depending on who is counting ( R or BaseX ),
The first question is : who is correct ? 
( And the 2nd question is probably: what do you mean by correct ? as the 
semantics of XQuery sequences and whatever destination R datatype is being 
counted may be slightly different. I don’t know R that well, but semantics of 
XQuery sequences and arrays are rather different, for example. )

— Steve M.


> On Feb 27, 2020, at 2:48 PM, Ben Engbers  wrote:
> 
> Op 27-02-2020 om 19:19 schreef Christian Grün:
>> It’s difficult to understand what’s going on here. Could you please
>> provide us self-contained queries without the R wrapper code?
> 
> Version 1:
> 
> import module namespace functx = 'http://www.functx.com';
> (: Extract the text :)
> let $txt := collection('IncidentRemarks/Incidents')/csv/record/INC_RM/text()
> (: Convert to lower-case and tokenize :)
> let $INC_RM := tokenize(lower-case(string-join($txt)),
> '(\\s|[,.!:;]|[n][b][s][p][;])+')
> (: Read Stopwords :)
> let $Stoppers := doc('TextMining/Stopwoorden.txt')/text/line/text()
> (: Remove Stopwords :)
> let $Stop :=  functx:value-except($INC_RM, $Stoppers)
> return $Stop"
> 
> My R-code first executes this as XQUERY and then calculates the length
> of the returned list (=5842).
> 
> Version 2:
> 
> import module namespace functx = 'http://www.functx.com';
> let $txt := collection('IncidentRemarks/Incidents')/csv/record/INC_RM/text()
> let $INC_RM := tokenize(lower-case(string-join($txt)),
> '(\\s|[,.!:;]|[n][b][s][p][;])+')
> let $Stoppers := doc('TextMining/Stopwoorden.txt')/text/line/text()
> let $Stop :=  functx:value-except($INC_RM, $Stoppers)
> return count($Stop)
> 
> Returns the length of the sequence (counts 5843 words).
> 
> The '\\' in the regular expression is intentional (R-specific). With a
> single '\' the query can be executed in BaseXGUI.
> 
> Does this help?
> 
> Ben
> 



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] DB with MS Word .docx files breaks on base-uri()

[Majewski, Steven Dennis (sdm7g)]

With a database containing MS Word .docx zip (and other) files:

collection('BOV') ! base-uri(.)

Returns =>

Error:
Stopped at ., 1/29:
[XQST0046] URI 
'/BOV/DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/[Content_Types].xml'
 is invalid.


I’m guessing it doesn’t like the brackets as part of a URI, but since I’m 
adding the whole docx container, I can’t specify a different path when creating 
the DB. 

collection('BOV') ! db:path(.)
However, works as a workaround:


DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/[Content_Types].xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/_rels/.rels
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/_rels/document.xml.rels
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/document.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/footer3.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/header1.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/footer1.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/endnotes.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/footnotes.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/_rels/header1.xml.rels
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/header2.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/_rels/footer2.xml.rels
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/footer2.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/theme/theme1.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/settings.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/styles.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/customXml/itemProps1.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/numbering.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/customXml/_rels/item1.xml.rels
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/customXml/item1.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/docProps/core.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/fontTable.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/webSettings.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/word/stylesWithEffects.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.docx/docProps/app.xml
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.tei
DCP-66/2015_AUG_BOV_RETREAT_MINS/2015_AUG_BOV_RETREAT_MINS.dc.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/[Content_Types].xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/_rels/.rels
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/_rels/document.xml.rels
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/document.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/footer3.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/header1.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/footer2.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/footer4.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/footer1.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/footnotes.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/endnotes.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/theme/theme1.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/settings.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/styles.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/numbering.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/customXml/_rels/item1.xml.rels
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/customXml/itemProps1.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/customXml/item1.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/docProps/core.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/fontTable.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/word/webSettings.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.docx/docProps/app.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.dc.xml
DCP-66/2010_FEB_BOARD_MINUTES/2010_FEB_BOARD_MINUTES.tei
...



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Database specific xquery modules

[Majewski, Steven Dennis (sdm7g)]

So, would the convention of using the database name as the function namespace 
for queries to that database be sufficient ? Or is convention not a sufficient 
encapsulation ? ( i.e. do you need stronger enforcement, so that db1 users 
can’t possibly call db2:func() ? ) 

— Steve M. 


> On Feb 26, 2020, at 11:07 PM, DYER Chris  wrote:
> 
> I will try to describe our situation a little bit more clearly.
> This is focused on our development environment, but similar situations occur 
> in a production environment.
>  
> We have multiple developers, each working on multiple projects.
> Each project will require a database.
> Each database will need to be accessed by multiple applications.
> Applications may be developed and executed by multiple developers on a netwok.
> We want to have a central server so that all developers are connecting to a 
> single location.
>  
> The content in each of these databases may be similar but with some 
> structural (schema) differences.
> As such, any queries for interacting with the data may be implemented 
> slightly differently.
>  
> When a new project is started, we would have a developer create an associated 
> database on the central server.
> We would like to create associated queries/functions to access the data for 
> this new project.
> These would only be applicable to the database we just created, so we would 
> not want them executed on another database.
> They also may conflict with similar functions for other project databases, so 
> keeping them isolated is ideal.
>  
> From the suggestions, it sounds like storing the queries as data within the 
> database is the most likely solution in the current software.
> It does mean we lose some of the benefits of them being actual modules.
>  
> As per Christian’s response, the software would need modifications to allow 
> these isolated modules.
> I would expect some isolated directory structure per database would be 
> necessary (if utilised).
> I would also expect this to be used in conjunction to the current global 
> modules (probably less concerned about how the implementation would end up).
>  
> Cheers,
> Chris
>  
> From: Majewski, Steven Dennis (sdm7g)  
> Sent: Wednesday, 26 February 2020 4:21 AM
> To: DYER Chris 
> Cc: basex-talk@mailman.uni-konstanz.de
> Subject: Re: [basex-talk] Database specific xquery modules
>  
> Also, question for the OP (Chris): 
>  
>  
> It’s not completely clear what exactly you want to hide from whom.
>  
> That is, if you only want to hide the implementation details from API 
> clients, then RESTXQ and some conditional code is probably sufficient, 
> however your comparison with local stored procedures in SQL make me think you 
> mean that database clients my provide their own implementations of lower 
> level functions, and you want higher level procedures to be able to call 
> those functions without being concerned with the implementations. ( Which is 
> what made me think of comparison with Object Oriented programming. ) 
>  
> — Steve M. 
>  
> 
> 
> On Feb 25, 2020, at 11:17 AM, Majewski, Steven Dennis (sdm7g) 
> mailto:sd...@virginia.edu>> wrote:
>  
> I was thinking that there would be a simpler way to bind function namespaces 
> and databases, but that runs into the asymmetry that string in  import and 
> namespace declarations in the prolog must be literal strings and can’t be 
> variables. ( I still tend to think from XSLT mental models instead of XQuery 
> sometimes. ) And it seems that even the extended Q{} syntax requires a 
> literal, so it’s not simple to make the function namespace variable. 
>  
> I can do something like this ( with modules for English, French, Italian 
> added to my repo ):
>  
> declare variable $it := 'http://example.org/modules/Italian 
> <http://example.org/modules/Italian>' ;
> declare variable $fr := 'http://example.org/modules/French 
> <http://example.org/modules/French>' ;
> declare variable $head := 'import module namespace lang = "' ;
> declare variable $tail :=  '";  ' ;
>  
> for $lang in ( $it, $fr )
> let $prolog := $head || $lang || $tail 
> return xquery:eval( $prolog || 'lang:hello("Everybody!")' ) 
>  
>  
> Returns =>
>  
> Ciao Everybody!
> Bonjour Everybody!
>  
> In the XSLT 1 days, we used to get around the lack of dynamic eval by the 
> functional programming technique of having one stylesheet produce another 
> with the variables resolved. 
> So another way of packaging this might be to create a RESTXQ endpoint that 
> generates an XQuery module from a template and use that endpoint URL in an 
> “import module … at” expression. But that location must also be a literal, so 
>

Re: [basex-talk] Database specific xquery modules

[Majewski, Steven Dennis (sdm7g)]

Also, question for the OP (Chris): 


It’s not completely clear what exactly you want to hide from whom.
 
That is, if you only want to hide the implementation details from API clients, 
then RESTXQ and some conditional code is probably sufficient, however your 
comparison with local stored procedures in SQL make me think you mean that 
database clients my provide their own implementations of lower level functions, 
and you want higher level procedures to be able to call those functions without 
being concerned with the implementations. ( Which is what made me think of 
comparison with Object Oriented programming. ) 

— Steve M. 


> On Feb 25, 2020, at 11:17 AM, Majewski, Steven Dennis (sdm7g) 
>  wrote:
> 
> I was thinking that there would be a simpler way to bind function namespaces 
> and databases, but that runs into the asymmetry that string in  import and 
> namespace declarations in the prolog must be literal strings and can’t be 
> variables. ( I still tend to think from XSLT mental models instead of XQuery 
> sometimes. ) And it seems that even the extended Q{} syntax requires a 
> literal, so it’s not simple to make the function namespace variable. 
> 
> I can do something like this ( with modules for English, French, Italian 
> added to my repo ):
> 
> declare variable $it := 'http://example.org/modules/Italian 
> <http://example.org/modules/Italian>' ;
> declare variable $fr := 'http://example.org/modules/French 
> <http://example.org/modules/French>' ;
> declare variable $head := 'import module namespace lang = "' ;
> declare variable $tail :=  '";  ' ;
> 
> for $lang in ( $it, $fr )
> let $prolog := $head || $lang || $tail 
> return xquery:eval( $prolog || 'lang:hello("Everybody!")' ) 
> 
> 
> Returns =>
>  
> Ciao Everybody!
> Bonjour Everybody!
> 
> In the XSLT 1 days, we used to get around the lack of dynamic eval by the 
> functional programming technique of having one stylesheet produce another 
> with the variables resolved. 
> So another way of packaging this might be to create a RESTXQ endpoint that 
> generates an XQuery module from a template and use that endpoint URL in an 
> “import module … at” expression. But that location must also be a literal, so 
> it’s just another variation of the above. 
> 
> I can also think of some tricks using higher level functions, but so far, 
> nothing that makes it all simple. 
> 
> 
> — Steve M. 
> 
> 
>> On Feb 25, 2020, at 5:50 AM, Christian Grün > <mailto:christian.gr...@gmail.com>> wrote:
>> 
>> Hi Luke,
>> 
>>> It isn't clear to me how RESTXQ helps with this.
>> 
>> Here is a (very minimal) example how an XQuery endpoint could look like:
>> 
>>  declare
>>%rest:path('{$db}/{$query}')
>>  function rest:execute(
>>$db as xs:string,
>>$query as xs:string
>>  ) {
>>xquery:eval(
>>  xs:anyURI($db || '/' || $query),
>>  map { '': db:open($db) }
>>)
>>  };
>> 
>> One endpoint for all databases would suffice indeed.
>> 
>> Extending the semantics of our REST API would be another option, but
>> we may need to clarify various questions (will there be an extra
>> server directory for each database? What happens if people want to
>> access both global and database-specific queries? etc).
>> 
>> Hope this helps,
>> Christian
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Database specific xquery modules

[Majewski, Steven Dennis (sdm7g)]

I was thinking that there would be a simpler way to bind function namespaces 
and databases, but that runs into the asymmetry that string in  import and 
namespace declarations in the prolog must be literal strings and can’t be 
variables. ( I still tend to think from XSLT mental models instead of XQuery 
sometimes. ) And it seems that even the extended Q{} syntax requires a literal, 
so it’s not simple to make the function namespace variable. 

I can do something like this ( with modules for English, French, Italian added 
to my repo ):

declare variable $it := 'http://example.org/modules/Italian' ;
declare variable $fr := 'http://example.org/modules/French' ;
declare variable $head := 'import module namespace lang = "' ;
declare variable $tail :=  '";  ' ;

for $lang in ( $it, $fr )
let $prolog := $head || $lang || $tail 
return xquery:eval( $prolog || 'lang:hello("Everybody!")' ) 


Returns =>
 
Ciao Everybody!
Bonjour Everybody!

In the XSLT 1 days, we used to get around the lack of dynamic eval by the 
functional programming technique of having one stylesheet produce another with 
the variables resolved. 
So another way of packaging this might be to create a RESTXQ endpoint that 
generates an XQuery module from a template and use that endpoint URL in an 
“import module … at” expression. But that location must also be a literal, so 
it’s just another variation of the above. 

I can also think of some tricks using higher level functions, but so far, 
nothing that makes it all simple. 


— Steve M. 


> On Feb 25, 2020, at 5:50 AM, Christian Grün  wrote:
> 
> Hi Luke,
> 
>> It isn't clear to me how RESTXQ helps with this.
> 
> Here is a (very minimal) example how an XQuery endpoint could look like:
> 
>  declare
>%rest:path('{$db}/{$query}')
>  function rest:execute(
>$db as xs:string,
>$query as xs:string
>  ) {
>xquery:eval(
>  xs:anyURI($db || '/' || $query),
>  map { '': db:open($db) }
>)
>  };
> 
> One endpoint for all databases would suffice indeed.
> 
> Extending the semantics of our REST API would be another option, but
> we may need to clarify various questions (will there be an extra
> server directory for each database? What happens if people want to
> access both global and database-specific queries? etc).
> 
> Hope this helps,
> Christian



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Call for assistance : BaseX as a preprocessor ?

[Majewski, Steven Dennis (sdm7g)]

What do you mean by “stream xml transforms? ? 

Do you mean stream a single large XML file ? A series of XML files, or stream a 
file thru a series of XQuery|XSLT|XPath transforms. 

Depends on what you mean by “stream” . 

I don’t believe BaseX uses a streaming XML parser, so probably can’t handle 
streaming a single large XML file and produce output before it’s parsed the 
complete file. 

But it looks like, from the link in your stackoverflow post that the data is 
already sharded into a collection of separate XML files that each contain 
multiple  elements. 

— Steve M. 


> On Feb 23, 2020, at 10:31 AM, maxzor  wrote:
> 
> Hello,
> 
> Thank you for your software which GUI has been my savior every time I needed 
> to deal with XML.
> 
> I would like to know if I can stream xml transforms, to pipe wikimedia XML 
> dumps into a format acceptable by postgres copy ?
> I know very well SQL, but nothing about XPath or XQuery
> 
> I managed to mock a XPath (or is it XQuery ? :/) snippet from postgres 
> itself, but obviously this would need rewriting for basex CLI :
> https://stackoverflow.com/questions/60361030/how-to-transform-and-stream-large-xml-files-to-postgres-mediawiki-dump-basex
> 
> Best regards, Maxime Chambonnet
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Database specific xquery modules

[Majewski, Steven Dennis (sdm7g)]

I’m not sure this is the best approach, but you seem determined to try to 
figure out how to turn XQuery into an Object-Oriented language, so, if you want 
to bind query functions to a particular database, you can put the functions in 
the database, and use xquery:eval() 
http://docs.basex.org/wiki/XQuery_Module 
 to call the function as a node() 
argument to eval. 
You would be using the context database as the “self” variable and 
xquery:eval() as dispatcher. 

If you don’t literally want to bind functions to DB, but instead dispatch by 
the schema of the data, you could use a typeswitch if the schema have different 
namespaces or root elements,. 
( I’ve tried this to switch between processing TEIP4 and TEIP5, for example. )

— Steve M.



> On Feb 21, 2020, at 1:41 AM, Markus Wittenberg  wrote:
> 
> If you want to use RESTXQ you would have to put the modules, or least the 
> functions defining the endpoints, in the webapp directory and not in the repo 
> directory. And far as I know, BaseX doesn't support installation for modules 
> in webapp as for in repo. So you have to explicitly change/add files in 
> webapp directory.
> 
> You could write a module for each database, defining its own endpoints, and 
> maybe still reuse some code by importing a module with common functions in 
> every specific module.
> 
> Am 21.02.2020 um 07:11 schrieb DYER Chris:
>> Hi Markus,
>> Not quite what I am aiming for.
>>  
>> We do want a single BaseX Instance to provide access to multiple databases.
>> 
>> Further to this though, a set of client applications will only connect to 
>> one database.
>> We have multiple sets of client applications, with each set each connecting 
>> to one database.
>>  
>> We do want to modularise the functions, however as an example we want to be 
>> able to have 2 versions of the same function, where 1 version works with the 
>> schema of the data in database 1, and the second version with a different 
>> schema in database 2.
>>  
>> If I could explicitly link the function modules with a database, I could 
>> call the same function from both application sets, and use the context of 
>> the open database to call the correct version of the function.
>>  
>> An example situation may help
>> Say we have Database for “Black Books”, which has book data in xml 
>> structures which conform to some version (1.1) of a schema.
>> We want a query stored on the server that returns books on art
>> Get-Books-About-Art
>> We have a client application specifically for this database that calls this 
>> function.
>> It will only need to query the data in this database.
>>  
>> Now we add a second Database for "Goliath Books"
>> The book data in this database uses a modified xml structure (version 2 of a 
>> schema)
>> We have a client application for this database that also wants to get books 
>> about art.
>> The query for this database is slightly different because the xml schema has 
>> changed.
>>  
>> I would like to store both versions of Get-Books-About-Art in the database.
>> Then for each client to connect to their associated database, and be able to 
>> execute the correct version of the xquery function.
>>  
>> The idea is that each time I add a new database that has data in some 
>> format, I can install associated query modules that are restricted to the 
>> context of that database to avoid conflicts with existing databases.
>>  
>> Maybe what I am looking for is effectively a database specific repo to 
>> install modules to, rather than a basex install repo.
>>  
>> From: Markus Wittenberg  
>>  
>> Sent: Friday, 21 February 2020 2:15 PM
>> To: DYER Chris  ; 
>> basex-talk@mailman.uni-konstanz.de 
>> 
>> Subject: Re: [basex-talk] Database specific xquery modules
>>  
>> A RESTXQ definition could look something like this:
>> 
>> declare
>> %rest:path("/a/{$client}/{$path}")
>> function yourmodule:test($client as xs:string, $path as xs:string) {
>> let $db := switch ($client)
>> case "clientA" return db:open("dbA")
>> case "clientA" return db:open("dbA")
>> default return db:open("dbX")
>> return yourmodule:do-something-with($db, $path)
>> };
>> And the called function do-something-with could also depend on the client
>> 
>>  
>> 
>> Am 21.02.2020 um 03:48 schrieb DYER Chris:
>> Hi Markus,
>> Our intent is to use REST and RESTXQ interfaces to interact with data on a 
>> single database using some server side XQueries.
>>  
>> However, we may have 2 (or more) databases provided by some server, and 
>> associated xqueries are specific to the contents of each database (as the 
>> contents of each of these databases may be structured differently).
>>  
>> We would like for a client to be able to open a single database and be able 
>> to run associated XQueries, without requiring client side knowledge of which 
>> 

Re: [basex-talk] was: increment a variable only when a conditional is true?

[Majewski, Steven Dennis (sdm7g)]

The reason I asked was: although XQuery looks more procedural than XSLT, it’s 
still basically a functional and declarative language, and thinking about 
incrementing or decrementing variables is probably the wrong way to think about 
solving your problem in XQuery. 
Variables bind names to values over a particular scope, and they aren’t 
updatable variables that you can increment or decrement. 

If what you want in English is roughly:
  For person element where numerical = true , group by the preceding-sibling 
where numerical = false 

That maps fairly simply into XQuery syntax: 

declare variable $XML := 
 people
 joe
 phone1
 phone2
 phone3
 sue
 cell4
 home5
 alice
 atrib6
 x7
 y9
 z10
 ;

for $P in $XML/person
where $P[@numerical="true"]
let $PREV := $P/preceding-sibling::person[@numerical="false"][1]
group by $PREV
return   { $P } 

Yields:



  phone1
  phone2
  phone3


  cell4
  home5


  atrib6
  x7
  y9
  z10


Once the grouping is working, you can work on tweaking the output format to 
exactly what you want. I tried to work towards something like what I THOUGHT 
you might want. 
I had a bit of trouble understanding that $PREV in the return expression is an 
atomic string value and not a node, and that the other variables in return 
expression are multiple valued ( thus the ‘[1]’s ), but I ended up with this:


 {
for $P in $XML/person
where $P[@numerical="true"]
let $PREV := $P/preceding-sibling::person[@numerical="false"][1]
let $X := count($P[1]/preceding-sibling::person[@numerical="false"])
group by $PREV
return   { for $I in $P return {$I/@id, attribute x {$X[1]-1}, 
> string($I)} }  
} 

Which yields:


  
phone1
phone2
phone3
  
  
cell4
home5
  
  
atrib6
x7
y9
z10
  



Also more functional to think of use count or position expressions instead of 
incrementing or decrementing variables. 

I’m hitting the Walmsley book myself now to try to understand those issues with 
group by / return that were puzzling to me. 


— Steve M. 


> On Feb 19, 2020, at 8:28 PM, thufir  wrote:
> 
> Hi Steve,
> 
> yes, it certainly does.  At this point, I'd settle for adding an attribute 
> like "recordID", but that's exactly it.  I'll take a closer look at "group 
> by" a bit later, thanks for the pointer.  See also:
> 
> https://stackoverflow.com/q/60237739/262852
> 
> 
> thanks,
> 
> Thufir
> 
> On 2020-02-19 1:05 p.m., Majewski, Steven Dennis (sdm7g) wrote:
>>> On Feb 19, 2020, at 10:01 AM, thufir >> <mailto:hawat.thu...@gmail.com>> wrote:
>>> 
>>> where I'm trying to use attributes because I'm not sure how to 
>>> conditionally nest tags.  But, this is interesting.  Not quite sure on 
>>> syntax to set and then conditionally increment $x, however.
>> Does “conditionally nest tags” mean that you want to make the 
>> person[@numerical=“true”] elements children of the immediately previous 
>> person[@numerical=“false”] elements ?
>> If that is the case you can use “group by”
>> — Steve M.



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] was: increment a variable only when a conditional is true?

[Majewski, Steven Dennis (sdm7g)]

> On Feb 19, 2020, at 10:01 AM, thufir  wrote:
> 
> where I'm trying to use attributes because I'm not sure how to conditionally 
> nest tags.  But, this is interesting.  Not quite sure on syntax to set and 
> then conditionally increment $x, however.

Does “conditionally nest tags” mean that you want to make the 
person[@numerical=“true”] elements children of the immediately previous 
person[@numerical=“false”] elements ? 

If that is the case you can use “group by” 


— Steve M. 




smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] Xslt:processor() and xslt:version() incorrect with Saxon 9.1B

[Majewski, Steven Dennis (sdm7g)]

I have noticed that BaseX xslt:processor() and xslt:version() returns incorrect 
info when used with Saxon 9.1B 
It appears to identify it as a Saxon processor, and assumes it’s Saxon HE and 
version 3.0 . 
This is different from what the XSLT system-property function returns. 

( I’m using Saxon9B because I have stylesheets based on that version that use 
external java functions. 
  I’m going to be updating these to use BaseX XQuery functions to pass params 
into the stylesheet instead of those external functions internal to the 
stylesheets, and I was hoping I could use those xslt:processor() function to 
keep track of which Saxon library was active as I was migrating the 
stylesheets. Setting CLASSPATH on the command line works OK for that use case, 
but I’m working on RESTXQ web apps, so I need to keep track of config when web 
app was started. ) 

Xq/processor.xq:
let $XS := http://www.w3.org/1999/XSL/Transform; 
version="1.0" >


, 

XSLT Version=


return (xslt:transform-text( , $XS ), concat('(', xslt:processor(), ', 
Version:', xslt:version(), ')') )

$ basex Xq/processor.xq 
Apache Software Foundation (Xalan XSLTC), 
http://xml.apache.org/xalan-j
XSLT Version=1.0
(Java, Version:1.0)

$ CLASSPATH=$SAXON9HE  basex Xq/processor.xq 
Saxonica, 
http://www.saxonica.com/
XSLT Version=3.0
(Saxon HE, Version:3.0)

$ CLASSPATH=$SAXON91B  basex Xq/processor.xq 
SAXON 9.1.0.8 from Saxonica, 
http://www.saxonica.com/
XSLT Version=2.0
(Saxon HE, Version:3.0)

I think it would be better to make XQuery xslt functions return the same values 
as XSLT system-property function. 
However, I would understand why you might not want to make an incompatible and 
slower change to current behavior, especially for a rather rare use case. 
If you keep the current behavior, I suggest a documentation note about how the 
function determines that value and that it may not be accurate in all 
circumstances. 

I can obviously write my own function as above to get correct values. 

( I know I mostly only write to this list when I have a problem or find an 
issue, however this *really* is a very solid implementation. Thank You! ) 


— Steve Majewski





smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] ePub zips via REST and functional XQuery

[Majewski, Steven Dennis (sdm7g)]

However, I neglected to notice that all of the html files in the EPUB container 
were all added as raw files. 
So it’s really just unpacking the EPUB zip into raw/ . 
Serving them up still works, but I’m guessing that will limit some of the 
search features on the unparsed files. 

— Steve. 


> On Aug 21, 2019, at 5:01 PM, Majewski, Steven Dennis (sdm7g) 
>  wrote:
> 
> BTW: I was pleasantly surprised to discover that when I made a BaseX database 
> with an ePub zip file, and 
> selected the appropriate options ( Parse files in archives, add other files 
> as raw files ) 
> I was able to serve and view the ePub files thru the rest interface, and 
> links from the TOC to other document parts,
> as well  as images in the ePUB all worked. 



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] ePub zips via REST and functional XQuery

[Majewski, Steven Dennis (sdm7g)]

BTW: I was pleasantly surprised to discover that when I made a BaseX database 
with an ePub zip file, and 
selected the appropriate options ( Parse files in archives, add other files as 
raw files ) 
I was able to serve and view the ePub files thru the rest interface, and links 
from the TOC to other document parts,
as well  as images in the ePUB all worked. 

I’m also enjoying learning to use some of the higher order function 
capabilities of XQuery 3.1 . 

The arrow syntax and the partial function application seem to be a great way to 
build a pipelines of transforms
that can mix  xslt:transform with XQuery functions . 


The results here aren’t very useful — I was just learning and testing the 
syntax, but this uses the  partial application ‘?’ 
Syntax to convert functions of N-args to functions of 1 arg, and build a list 
of 1-arg functions to feed to fold-left. 
The use case would be to create a map of lists of pipeline functions to be 
applied for different targets. 

Not sure if maintaining pipeline lists is a better way to manage this than just 
building pipelines using arrow => operator,
but I was just trying to explore some of the possible methods.

let $docid := 'SIB/sibv060.xml'
let $t1 := xslt:transform( ?,  "/projects/SIB/add_id_bov.xsl" )
let $t2 := xslt:transform( ?, 
"/projects/XTF/xtf.lib/style/dynaXML/docFormatter/tei/teiDocFormatter.xsl", 
map{ 'docId' : $docid  })
let $t3 := function($in) { $in//*  }
let $pipe := ( $t1, $t2, $t3, function($x) { $x ! name(.) }, distinct-values#1  
)
let $chain := function( $a, $b ) { $b($a) }
return fold-left( $pipe, doc($docid) , $chain )



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Db:list-details()/@modified-date

[Majewski, Steven Dennis (sdm7g)]

Thank you. 
I see that all of the WebDAV PUTS to the web app are logged, so I suppose I 
could search logs for updates via WebDAV. 
( And, presumably, I’ll be the only one doing direct DB updates via basexgui or 
command line basex, so I’ll just have to discipline myself! ;-)


I was checking this by mounting basex WebDAV volume as a remote filesystem on 
my Mac, which lead me to think that another solution
might be to use Linux DavFS2 to do the same on server  and just use the WebDAV 
volume as the filesystem store — thus there is only 
one canonical copy and keeping them in sync is not an issue. However, while 
testing that, I’m seeing some issues that suggest that won’t
work — I was getting errors from rsync trying to create temp files on the 
destination volume ( WebDAV ). And also, one of the reasons for
wanting file system access is to track changes in git. I looks like I can keep 
the .git files external to that filesystem by using —git-dir= and 
work-tree= options, but this may need more testing. It would simplify thinks if 
I don’t have to worry about keeping filesystem and database in sync. 


— Steve M. 



> On Aug 19, 2019, at 10:36 AM, Christian Grün  
> wrote:
> 
> Hi Steven,
> 
> Finally a quick response.
> 
> In a nutshell: “You are right” ;) I think you already summarized the
> status quo pretty well: Timestamps are only available for the whole
> database, and for binary files (for which simply the OS timestamp will
> be returned).
> 
> In our applications, we mostly use RESTXQ to organize additional meta
> data properties for documents. They will usually be stored in an
> additional meta data database. I agree this won’t be helpful if you
> use WebDAV and other low-level APIs. Apart from the discussed trigger
> feature, there is another issue that is somewhat related with your
> feature request for organizing document metadata [1]: Various default
> properties could be defined that will always be refreshed
> automatically if a document is created or updated. For now, due to
> lack of time and resources, I’m sorry to say that both issues are
> still on hold.
> 
> Best,
> Christian
> 
> [1] https://github.com/BaseXdb/basex/issues/988
> 
> 
> 
> On Wed, Aug 14, 2019 at 5:12 AM Majewski, Steven Dennis (sdm7g)
>  wrote:
>> 
>> 
>> I thought I could use db:list-details($db)/@modified-date to select the 
>> resources that have most recently been updated ( to generate an OAI or RSS 
>> feed ) however, on doing a db:replace of one of the resources, I’m either 
>> not seeing the date change at all, or I’m seeing the modified-date of all of 
>> the resources update together.
>> 
>> 
>> I find an explanation in the archives:
>> https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg09668.html
>> 
>>> Exactly: Nothing will be replaced in BaseX if your source and target
>>> XML node is identical (this holds true for both documents as well as
>>> any other node in a document).
>>> 
>>> Please note that XML documents have no individual timestamps in BaseX.
>>> Instead the timestamp of the database will be changed if nodes are
>>> changed.
>>> 
>> 
>> 
>> 
>> Do I parse that correctly in saying that when I try to replace a document 
>> with an identical document, it doesn’t register as an update, but when I do 
>> change and update the document, it changes the timestamp for the whole 
>> database, and there is no timestamp for individual resources ?
>> 
>> So I assume that if I want to track updated resources, I would have to 
>> either add timestamps to the documents themselves, or else track this info 
>> somewhere else, maybe in SQL and use sql: functions from within BaseX.
>> 
>> Maybe best method is to add timestamp as a processing instruction or comment 
>> to avoid any changes to the body of the document:
>> insert node processing-instruction modifiedDateTime { current-dateTime()} as 
>> first into $D
>> 
>> That would work if doing the updates from an XQuery script and calling 
>> db:replace and then adding the timestamp,
>> However, not very reliable if the resource can be updated by another method 
>> ( WEBDAV for example ) unless there is a way to trigger xquery function on 
>> update. I find from searching the mailing list archives a link to an issue 
>> discussing adding database triggers, but it appears to be on hold:
>> 
>> https://github.com/BaseXdb/basex/issues/1082
>> 
>> Any news on this or suggestions of alternatives ?
>> 
>> 
>> 
>> — Steve Majewski
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] Db:list-details()/@modified-date

[Majewski, Steven Dennis (sdm7g)]

I thought I could use db:list-details($db)/@modified-date to select the 
resources that have most recently been updated ( to generate an OAI or RSS feed 
) however, on doing a db:replace of one of the resources, I’m either not seeing 
the date change at all, or I’m seeing the modified-date of all of the resources 
update together. 


I find an explanation in the archives:
https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg09668.html

> Exactly: Nothing will be replaced in BaseX if your source and target
> XML node is identical (this holds true for both documents as well as
> any other node in a document).
> 
> Please note that XML documents have no individual timestamps in BaseX.
> Instead the timestamp of the database will be changed if nodes are
> changed.
> 



Do I parse that correctly in saying that when I try to replace a document with 
an identical document, it doesn’t register as an update, but when I do change 
and update the document, it changes the timestamp for the whole database, and 
there is no timestamp for individual resources ? 

So I assume that if I want to track updated resources, I would have to either 
add timestamps to the documents themselves, or else track this info somewhere 
else, maybe in SQL and use sql: functions from within BaseX. 

Maybe best method is to add timestamp as a processing instruction or comment to 
avoid any changes to the body of the document:
 insert node processing-instruction modifiedDateTime { current-dateTime()} as 
first into $D

That would work if doing the updates from an XQuery script and calling 
db:replace and then adding the timestamp,
However, not very reliable if the resource can be updated by another method ( 
WEBDAV for example ) unless there is a way to trigger xquery function on 
update. I find from searching the mailing list archives a link to an issue 
discussing adding database triggers, but it appears to be on hold: 

https://github.com/BaseXdb/basex/issues/1082

Any news on this or suggestions of alternatives ? 



— Steve Majewski










smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] console.log ?

[Majewski, Steven Dennis (sdm7g)]

Thanks!  That was what I was looking for: admin:write-log(). 

I’m building a dispatcher in RESTXQ to fetch a document and run one or more 
XSLT:transforms depending on the document type ( first by namespace, and if no 
namespace, falling thru to root element ), and for debugging and testing, I’ld 
like to log a message from the dispatcher before it calls the transform 
functions. 

Also: I may need to change the output type depending on the dispatch. 
If I declare the output with %output:method, it is fixed for that function. 
Is there a function to change output method dynamically ? 
Or is the solution to use a redirect from the dispatch function to another 
function with a specific output declared? 


— Steve.



> On Aug 10, 2019, at 10:48 AM, Christian Grün  
> wrote:
> 
> Hi Steve,
> 
> You can use fn:trace to output the argument, which will itself be passed on 
> unchanged, to standard error (or to the info view, if you use the GUI). If 
> you want to swallow the output value, you can use the BaseX-specific function 
> prof:dump. In web applications, you can output values to the database logs 
> via admin:write-log.
> 
> Hope this helps 
> Christian
> 
> 
> 
> Majewski, Steven Dennis (sdm7g)  <mailto:sd...@virginia.edu>> schrieb am Sa., 10. Aug. 2019, 16:42:
> Is there an equivalent to console.log or xsl:message in XQuery or BaseX ? 
> 
> I know I could use the basex file: module to write to a file, but is there 
> anything that works with the logging system to direct output according to the 
> context ? 
> 
> — Steve Majewski 
> 
> 



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] console.log ?

[Majewski, Steven Dennis (sdm7g)]

Is there an equivalent to console.log or xsl:message in XQuery or BaseX ? 

I know I could use the basex file: module to write to a file, but is there 
anything that works with the logging system to direct output according to the 
context ? 

— Steve Majewski 




smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] Serialization mismatch using Saxon for xslt:transform

[Majewski, Steven Dennis (sdm7g)]

Continuing with my test.xqm experiments… 

I have copied Saxon jar into WEB-INF/lib/  so xslt:transform is using Saxon. 

I have the following functions declared:

  declare
%rest:path("test/xqdoc.xml")
%output:method('xml')
function test:doc() {
  inspect:xqdoc( $test:module_name )
};

declare
   %rest:path('test/xdoc')
   %output:method('html’)   (: I’ve tried various values 
for method and html-version here ! :) 
   %output:html-version('5.0')
   function test:htmldoc() {
 (:  WTF? :)
 xslt:transform( test:doc(),'static/xsl/html-module.xsl', map { 'source' : 
'test.xqm' })
   };


Where html-module.xsl initially was:  
https://github.com/xquery/xquerydoc/blob/master/src/lib/html-module.xsl 
 


Accessing /basex/test/xdoc  returns error message: 

Stopped at /usr/local/tomcat/webapps/basex/test.xqm, 41/20:
[FODC0002] "" (Line 69): The entity "nbsp" was referenced, but not declared.

Which was initially very puzzling, as I could not find nbsp in either my 
test.xqm, html-module.xsl, or inspect:xsdoc() output. 
Turns out, even though it was encoded as  "” in html-module.xsl, Saxon 
was encoding it as “” on output of the transform. 

Initially, I wasn’t sure if Saxon or BaseX was doing the serialization. I guess 
it seems to be  both. 

Original output method for that stylesheet is:


And changing output:method  on the function to xhtml or html doesn’t have any 
effect on results.
Changing output method on the stylesheet to xhtml causes Saxon to serialized it 
as “”  instead of “” , and the serialization method on the basex 
RESTXQ function doesn’t seem to give an errors with any output methods.

I take from these results that the output from Saxon xslt:transform is 
serialized according to the stylesheet, and then parsed again by BaseX as xml 
on the way to being serialized again on output from the function, and the error 
is coming from that implicit parse. 

I don’t suppose there is a way to short circuit the parsing of the 
xslt:transform function output and just output the results directly (?)
I tried xslt:transform-text() but it escapes all of the element tags. 
Or a way to get it to parse the results of that transform function as html ? 

Clearly changing the stylesheet is the easy solution for this test file, but 
for some of my other real cases, I may be trying to repurpose stylesheets that 
will be used in different contexts: within BaseX and outside of that context, 
so keeping different versions of the stylesheets is going to be annoying. 
I suppose I could read in the stylesheets and modify the xml:output line in 
XQuery or XSLT before using it within BaseX.  


— Steve M. 






smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] comment()

[Majewski, Steven Dennis (sdm7g)]

Also literal  work just like other literal XML, as there is 
a different syntax for (: XQuery comments  :) 

XQuery: { collection()[1]//comment() }  
{ comment { " last comment " } }  (: XQuery Comment :) 

Produced output:


  
  
  
  
  
  



"comment()"   is the XPath syntax to match a comment node, not a function that 
creates comments. 

— Steve M. 



> On Aug 7, 2019, at 1:03 PM, Martin Honnen  wrote:
> 
> On 07.08.2019 18:52, Marc wrote:
> 
>> I want to write a comment in the xml I produce with my XQuery.
>> I try to use the comment() function, but each time I want to put
>> something inside the () i have a syntax error.
> 
> If you want to create a comment node computed by an expression use
>  comment { expression }
> 
> see https://www.w3.org/TR/xquery-31/#doc-xquery31-CompCommentConstructor
> 
> So
>  comment { $r }
> seems to be what you want to do.



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] Error from function-name(request:query#0): only from RESTXQ, not REST context.

[Majewski, Steven Dennis (sdm7g)]

I’ve been learning how to use the BaseX webapp, and below is one of the first 
test scripts for the REST interface I wrote.
No problems with this script. 


import module namespace request = "http://exquery.org/ns/request;;

declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization;;
declare option output:omit-xml-declaration "no";
declare option output:method "xhtml";
(: http://docs.basex.org/wiki/XQuery_3.0#Serialization  :)

(: 
declare variable $contains := 
try { request:parameter( 'contains' ) }
catch basex:http { '' }; 

 Simpler to use implicit assignment: 
http://docs.basex.org/wiki/REST#Assigning_Variables
"All query parameters that have not been processed before will be treated as 
variable assignments" 
:)

declare variable $contains as xs:string external := "";
declare function local:f0line( $f ) as node() {
  (: build a list item with function name and function values :)I
  {fn:function-name($f)}:{$f()}
};

declare variable $CTX := try { . }  catch err:XPDY0002 { () }; 
declare variable $DB := try { db:name( $CTX[1] ) } catch err:XPTY0004 { "" };
(: Could be, if ():  [XPTY0004] node() expected, empty sequence found. 
   but not sure if that's the only possible case.  :)

test

Database: { $DB }
Context documents (.)

{ for $doc in $CTX return {base-uri($doc)} }





HTTP Request

  { for $f in ( request:method#0, request:scheme#0, request:hostname#0, 
request:port#0,
   request:path#0, request:query#0, request:uri#0, 
request:context-path#0, request:address#0, 
   request:remote-hostname#0, request:remote-address#0, 
request:remote-port#0 ) 
  return local:f0line($f) } 



Request Parameters:

{ for $param in request:parameter-names() return 
{$param}:{request:parameter($param)} }



Headers

{ for $header in request:header-names() return 
  {$header}:{request:header($header)} }



Cookies

 { for $cookie in request:cookie-names() return 
  {$cookie}:{request:cookie($cookie)} }



XSLT

xslt:processor:  { xslt:processor()},  xslt:version: { 
xslt:version()}





I’m now trying to learn  RESTXQ programming, so I was trying to do the same 
sort of thing from a RESTXQ module. 

I was getting an error from the line where  (: request:query#0, :)   is now 
commented out, and added the line:

{ request:query( ), function-name(request:query#0)}

And I see that what happens is that it’s  the latter 
function-name(request:query#0) that is generating an error, but only when there 
is no query string in my request.  An empty query string, i.e. 
“http://localhost/basex/test? ”  , for example, 
works. But “http://localhost/basex/test   returns:

Stopped at /usr/local/tomcat/webapps/basex/test.xqm, 33/66:
[XPTY0004] Cannot convert empty-sequence() to xs:string: ().

I assume this has something to do with how RESTXQ processes form and request 
parameters, and I probably don’t actually need to do it this way, but I was 
wondering what exactly is the source of the problem with doing something like 
this or if there is an obvious work around. Maybe writing another local 
function to wrap around request:query and to use in that loop. 

( One reason I chose that test to convert to RESTXQ was that it wasn’t obvious 
to be from the docs if I could mix those different access methods, and so what 
was surprising was that it mostly worked — only that one function failed (so 
far)). 


module namespace test = 'http://localhost/test';

import module namespace request = "http://exquery.org/ns/request;;


declare function test:f0line( $f ) as node() {
  (: build a list item with function name and function values :)
  {fn:function-name($f)}:{$f()}
};

(:~
 : Generates a welcome page.
 : @return HTML page
 :)
declare
  %rest:path("test")
  %output:method("xhtml")
  %output:omit-xml-declaration("no")
  %output:doctype-public("-//W3C//DTD XHTML 1.0 Transitional//EN")
  
%output:doctype-system("http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;)
function test:test(
) as element(Q{http://www.w3.org/1999/xhtml}html) {
  http://www.w3.org/1999/xhtml;>

  BaseX HTTP Services
  


  
  TEST

  
{ request:query( ), function-name(request:query#0)}
  


HTTP Request

  { for $f in ( request:method#0, request:scheme#0, 
request:hostname#0, request:port#0,
   request:path#0, (: request:query#0, :) request:uri#0, 
request:context-path#0, request:address#0,
   request:remote-hostname#0, request:remote-address#0, 
request:remote-port#0 )
  return test:f0line($f) }



  
};






smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Coding help

[Majewski, Steven Dennis (sdm7g)]

Creating a bunch of temporary databases that you’re going to delete doesn’t 
sound like the most efficient way to process this data. 
But it hard to tell what alternative to recommend without more info about what 
your desired end result is. 

Is this something that you’re going to do once, or do repeatedly for different 
strings ? 

Taking your description literally, I would use ‘grep -l ‘ to generate a list of 
files with the specific string, and either feed that list into ‘cat’ or else 
use it to build a database of a subset of files for further investigation. 

There are also other ways to filter the data through a stream to select a 
subset if that is what you want to do. 

But if you’re going to do this repeatedly for different subsets, then it might 
make more sense to try to get everything parsed and indexed into the database 
once. If it really is too large for a single database after adjusting the java 
memory parameters in the basex scripts, you could try sharding the data into 
several databases, repeat the search on each collection, and concatenate the 
results. 




> On Aug 5, 2019, at 2:41 AM, Greg Kawchuk  wrote:
> 
> Hi everyone,
> I'm wondering if someone could provide what I think is a brief script for a 
> scientific project to do the following. 
> The goal is to identify XML documents from a very large collection that would 
> be too big to load into a database all at once.
> 
> Here is how I see the functions provided by the code. 
> 1. In the script, the user could enter the path of the target folder (with 
> millions of XML documents).
> 2. In the script, the user would enter the number of documents to load into a 
> database at a given time (i =. 1,000) depending on memory limitations.
> 3. The code would then create a temporary database from the first (i) xml 
> files in the target folder.
> 4. The code would then search the 1000 xml documents in the database for a 
> pre-defined text string.
> 5. If hits exist for the text string, the code would write those documents to 
> a unique XML file.
> 6. Clear the database.
> 7. Read in the next 1000 files (or remaining files in the folder).
> 8. Return to #4.
> 
> There would be no need to append XML files in step 5. The resulting XML files 
> could be concatenated afterwards. 
> Thank you in advance. If you have any questions, please feel free to email me 
> here. 
> Greg
> 
> ***
> Greg Kawchuk BSC, DC, MSc, PhD.
> Professor, Faculty of Rehabilitation Medicine
> University of Alberta
> greg.kawc...@ualberta.ca 
> 780-492-6891



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Managing external entities

[Majewski, Steven Dennis (sdm7g)]

OK: one answer to my own question is that instead of trying to tackle it by 
resolving  entities when creating the database,
I can use base-url() of the (in database) document to find the original, and 
then parse it as text using file:read-text-lines() .


for $LINE in file:read-text-lines( 'legacy/uvaBook/tei/PoeScap.xml' )
where starts-with( $LINE, "uva-lib:488578
uva-lib:488579
uva-lib:488580

That I could feed to a lookup function. 

Or maybe a MAP would be more direct:

map:merge( for $LINE in file:read-text-lines( 'legacy/uvaBook/tei/PoeScap.xml' )
where starts-with( $LINE, " On Jul 24, 2019, at 3:19 PM, Majewski, Steven Dennis (sdm7g) 
>  wrote:
> 
> I have a corpus of TEI files with figures and page images encoded as external 
> entities. 
> It appears that even when choosing “Parse DTDs and entities” this info is 
> lost when parsing files into database,
> And in any case, unparsed-entity-uri() is an XSLT only function. 
> 
> It would appear that I first need to transform the files first and replace 
> @entity attributes with @url attributes while these unparsed entity values 
> are available, before creating the database, or else generate another 
> database to map entity names to values later. 
> 
> Are there any better ways to handle this case ? 
> 
> Is there any way to do these transforms on the fly before parsing the files 
> into the database ? 
> 
> The only thing that comes to mind is to set up a local SaxonServlet to do the 
> transforms, and load from URLs instead of file paths. 
> ( I’ve been doing something similar for a different case, and running into 
> memory errors that I don’t see when loading from a directory when creating a 
> database.  Increasing memory didn’t help much, but inserting ‘flush’ command 
> between even ‘add’ commands seemed to work. ) 
> 
> 
> — Steve Majewski
> 
> 
> 
> 
> 
> 
> 



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] Managing external entities

[Majewski, Steven Dennis (sdm7g)]

I have a corpus of TEI files with figures and page images encoded as external 
entities. 
It appears that even when choosing “Parse DTDs and entities” this info is lost 
when parsing files into database,
And in any case, unparsed-entity-uri() is an XSLT only function. 

It would appear that I first need to transform the files first and replace 
@entity attributes with @url attributes while these unparsed entity values are 
available, before creating the database, or else generate another database to 
map entity names to values later. 

Are there any better ways to handle this case ? 

Is there any way to do these transforms on the fly before parsing the files 
into the database ? 

The only thing that comes to mind is to set up a local SaxonServlet to do the 
transforms, and load from URLs instead of file paths. 
( I’ve been doing something similar for a different case, and running into 
memory errors that I don’t see when loading from a directory when creating a 
database.  Increasing memory didn’t help much, but inserting ‘flush’ command 
between even ‘add’ commands seemed to work. ) 


— Steve Majewski









smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Change data directory for basex & BaseXGUI

[Majewski, Steven Dennis (sdm7g)]

Although, a bit confusing:

I set the database path in basexgui to match the value in web.xml 
( The setting is commented out, but I assumed that was because it was the 
default value. )

  
org.basex.dbpath
WEB-INF/data
  

But web app couldn’t see the database I created there. 

I moved created DB directory from webapp/basex/WEB-INF/data/  into 
webapp/basex/data/  and restarted, 
And the webapp finds it there. 



> On May 15, 2019, at 2:51 PM, Majewski, Steven Dennis (sdm7g) 
>  wrote:
> 
> Nevermind. I couldn’t find it on my first hour of searching the docs. 
> Then I found it 5 minutes after asking for help! 
> 
> http://docs.basex.org/wiki/Configuration 
> <http://docs.basex.org/wiki/Configuration> 
> 
> 
> — Steve M. 
> 
> 
>> On May 15, 2019, at 2:33 PM, Majewski, Steven Dennis (sdm7g) 
>> mailto:sd...@virginia.edu>> wrote:
>> 
>> I’ve been learning and testing basex with the default MacOS home-brew 
>> installation. 
>> 
>> For production, I’m planning on using the war file served using tomcat. 
>> 
>> I would like to use the basex command line client and BaseXGUI to do some 
>> initial setup and testing on the tomcat web app databases.  
>> 
>> Is there a java property setting or other way to set the data directory 
>> location for these programs ? 
>> Or do I have to set these paths in  ~/basex/.basex file . 
>> 
>> The latter is not ideal, as I may need to run update scripts from different 
>> user cron jobs, so I’ld rather not depend on home directory. 
>> 
>> 
>> — Steve Majewski
>> 
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] Change data directory for basex & BaseXGUI

[Majewski, Steven Dennis (sdm7g)]

Nevermind. I couldn’t find it on my first hour of searching the docs. 
Then I found it 5 minutes after asking for help! 

http://docs.basex.org/wiki/Configuration 
<http://docs.basex.org/wiki/Configuration> 


— Steve M. 


> On May 15, 2019, at 2:33 PM, Majewski, Steven Dennis (sdm7g) 
>  wrote:
> 
> I’ve been learning and testing basex with the default MacOS home-brew 
> installation. 
> 
> For production, I’m planning on using the war file served using tomcat. 
> 
> I would like to use the basex command line client and BaseXGUI to do some 
> initial setup and testing on the tomcat web app databases.  
> 
> Is there a java property setting or other way to set the data directory 
> location for these programs ? 
> Or do I have to set these paths in  ~/basex/.basex file . 
> 
> The latter is not ideal, as I may need to run update scripts from different 
> user cron jobs, so I’ld rather not depend on home directory. 
> 
> 
> — Steve Majewski
> 



smime.p7s
Description: S/MIME cryptographic signature

[basex-talk] Change data directory for basex & BaseXGUI

[Majewski, Steven Dennis (sdm7g)]

I’ve been learning and testing basex with the default MacOS home-brew 
installation. 

For production, I’m planning on using the war file served using tomcat. 

I would like to use the basex command line client and BaseXGUI to do some 
initial setup and testing on the tomcat web app databases.  

Is there a java property setting or other way to set the data directory 
location for these programs ? 
Or do I have to set these paths in  ~/basex/.basex file . 

The latter is not ideal, as I may need to run update scripts from different 
user cron jobs, so I’ld rather not depend on home directory. 


— Steve Majewski



smime.p7s
Description: S/MIME cryptographic signature

Re: [basex-talk] db:replace() in DBA resulting in duplicates

[Majewski, Steven Dennis (sdm7g)]

I’m glad I caught this thread. Replace was working for me in basexgui but not 
from command line. 
( And I needed to generate replacement list from the shell using ‘find … -newer 
basex/data/collection’ ) 

I pulled the basex/lib/basex-api-9.2.1-SNAPSHOT.jar file out of the snapshot 
zip file and copied it to home-brew
/usr/local/Cellar/basex/9.2/libexec/lib/basex-api-9.2.jar  and replacement from 
the basex shell command appears to work now. 

Previously, I was getting this error, trying several variations of 
absolute/relative filenames or file: URIs. 

published$ basex -i published -c "replace jmu/vihart00338.xml  
/projects/vh-migrate/published/jmu/vihart00338.xml" 
Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 9.2
Java: Oracle Corporation, 1.8.0_201
OS: Mac OS X, x86_64
Stack Trace: 
java.lang.RuntimeException: published: lock file does not exist.
at org.basex.util.Util.notExpected(Util.java:61)
at org.basex.data.DiskData.finishUpdate(DiskData.java:246)
at org.basex.core.cmd.ACreate.update(ACreate.java:97)
at org.basex.core.cmd.Replace.run(Replace.java:55)
at org.basex.core.Command.run(Command.java:257)
at org.basex.core.Command.execute(Command.java:93)
at org.basex.api.client.LocalSession.execute(LocalSession.java:132)
at org.basex.api.client.Session.execute(Session.java:36)
at org.basex.core.CLI.execute(CLI.java:92)
at org.basex.core.CLI.execute(CLI.java:76)
at org.basex.core.CLI.execute(CLI.java:63)
at org.basex.BaseX.(BaseX.java:80)
at org.basex.BaseX.main(BaseX.java:42)



— Steve Majewski



> On Apr 29, 2019, at 5:08 AM, Christian Grün  wrote:
> 
> Hi Daniel,
> 
> seems hard to reproduce. Could you tell me what you did step by step?
> This is what I tried:
> 
> 1. I created a DB database.
> 2. I added the document  to the database path /path/file.xml.
> 3. I replaced this document with the same document via the DBA.
> 4. Your query db:open('DB')/root[@id='id']/base-uri() returned a single 
> string.
> 
> Thanks in advance,
> Christian
> 
> PS: The form parameter handling in 9.2 contains a little bug (as a
> result, I couldn’t invoke the replace functionality with the current
> version). I just uploaded a new snapshot. Both the current and the
> snapshot version should behave identically in terms of adding and
> replacing documents.
> 
> 
> 
> On Fri, Apr 26, 2019 at 4:45 PM Zimmel, Daniel  wrote:
>> 
>> Hello,
>> 
>> something has gone awry in my database after a db:replace():
>> 
>> query:
>>db:open('DB','/path/file.xml')/base-uri()
>> 
>> result:
>>/DB/path/file.xml
>> 
>> This is as expected. The result is the replaced file. But when I
>> 
>> query:
>>db:open('DB')/root[@id='id']/base-uri()
>> 
>> result is a sequence:
>>/DB/path/file.xml
>>/DB/path/file.xml
>> 
>> The first result is the file before the db:replace(), the second after.
>> 
>> Can somebody explain why db:open() with path parameter is returning one 
>> result, but without parameter two results?
>> Database statistics say there are indeed two documents. But when I export 
>> the database, only one gets exported.
>> 
>> This happens with db:replace() on 9.2 from the DBA browser interface.
>> When I use the same db:replace() XQuery on 9.2 with my local database, all 
>> is well (document gets replaced, no duplicates). Settings should be the 
>> default ones.
>> 
>> Thanks, Daniel
>> 
>> 



smime.p7s
Description: S/MIME cryptographic signature