Re: [basex-talk] Crash when starting basexgui

[Joe Wicentowski]

Thanks, Michael, Christian, and Johan, for your help and ideas!

I finally had the chance to dive in. I couldn't just install temurin (Java
21), since I already had used Homebrew to install temurin@8 (which I need
for eXist). Whenever I've previously needed to switch between two versions
of Java I would use homebrew to uninstall one and install the other. This
takes time, so it wasn't ideal. I would like to run eXist and BaseX
whenever I need them, without this hassle. So this gave me a chance to try
out jenv. I followed a few tutorials. The best one I found was
https://www.baeldung.com/jenv-multiple-jdk.

Hopefully the underlying bug causing the weird error on macOS will be
solved, but in the meantime, this will do the trick.

Below are some of the commands I used, in the hope that this might help
others who need to run multiple versions of Java.

Thanks again!
Joe


# install jenv - and follow the installer's hints to add lines to your
.zshrc or .profile
brew install jenv

# install the versions of java you need
brew install temurin temurin@8

# register these versions
jenv add /Library/Java/JavaVirtualMachines/temurin-8.jdk/Contents/Home
jenv add /Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home

# set the global java to 8 - a good default for me
jenv global 1.8

# set the current shell to 21 - so I can start basexgui in this terminal
without affecting other places
jenv shell 21.0

# check current shell version
jenv versions
  system
  1.8
  1.8.0.392
* 21.0 (set by JENV_VERSION environment variable)
  21.0.1
  temurin64-1.8.0.392
  temurin64-21.0.1

On Wed, Oct 25, 2023 at 9:19 AM Johan Mörén  wrote:
>
> Thanks for the workaround Michael!
>
> It worked for me as well but I installed temurin via homebrew.
> brew install temurin
>
> Regards,
> Johan
>
> On Wed, Oct 25, 2023 at 3:03 PM Johan Mörén  wrote:
>>
>> Hi Michael, my setting for "Allow Applications Downloaded from…" is “App
Store and identified developers”.
>>
>> It's controlled by a central company profile so I can't change it.
>>
>> Thanks for pointing out that the warning is not the cause. I managed to
see the same error report as you posted when i choose "report to apple"
>>
>> Regards,
>> Johan
>>
>> On Wed, Oct 25, 2023 at 2:21 PM Michael Seiferle  wrote:
>>>
>>> Hi Johan,
>>>
>>>
>>>
>>> it works (well worked… see below) on my machine with the very same
setup (basex@10.7 & openjdk@21 via brew, see my previous email), so I am
sure it is an issue for certain setups.
>>>
>>>
>>>
>>> what’s your security settings? Apple Menu -> Privacy & Security ->
Allow Applications Downloaded from… mine is set to “App Store and
identified developers”.
>>> Now if I set it to “App Store” BaseX will not start with the following
error:
>>>
>>>
>>>
>>> BUT: now it won’t start again even if I set it back :-{
>>>
>>> So my motivation to find a fix or workaround is considerably higher now
I will report once I found something.
>>>
>>>
>>>
>>> The warning that is dumped to your console — something something
NSApplicationDelegate.applicationSupportsSecureRestorableState — doesn’t
cause BaseX to crash, as I could see this warning as well.
>>>
>>> Background:
>>>
>>>
https://stackoverflow.com/questions/77283578/sonoma-and-nsapplicationdelegate-applicationsupportssecurerestorablestate
>>>
>>>
>>>
>>> Best
>>>
>>> Michael
>>>
>>>
>>>
>>>
>>>
>>> Von: Christian Grün 
>>> Datum: Mittwoch, 25. Oktober 2023 um 13:09
>>> An: Johan Mörén 
>>> Cc: Michael Seiferle , BaseX <
basex-talk@mailman.uni-konstanz.de>
>>> Betreff: Re: [basex-talk] Crash when starting basexgui
>>>
>>> Thanks. It seems we have to wait for Apple to fix this. Unfortunately,
their bug tracker doesn’t seem to be public [2].
>>>
>>>
>>>
>>> [1]
https://github.com/Homebrew/homebrew-core/issues/150824#issuecomment-1776139189
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Oct 25, 2023 at 1:01 PM Johan Mörén 
wrote:
>>>
>>> Running MacOS Sonoma Apple Silicon M2 processor and ran into the same
problem when homebrew updated my JDK to 21.0.1 today.
>>>
>>>
>>>
>>> According to this article it seems like applications like jmeter and
IntelliJ needs to fix this by implementing a specific interface. I guess
this applies to Basexgui as well?
>>>
>>>
>>>
>>>
https://stackoverflow.com/questions/77283578/sonoma-and-nsapplicationdelegate-applicationsupportssecurerestorablestate
>>>
>>>
>>>
>>> Regards,
>>>
>>> Johan
>>>
>>>

>

[basex-talk] Crash when starting basexgui

[Joe Wicentowski]

Hi all,

I am experiencing a crash when starting basexgui. The splash screen appears
briefly, and then the message pasted in below appears, and I've also
attached the referenced error report file and the crash report that appears
in the Mac crash dialog. I'm using BaseX 10.7, installed via Homebrew
(which installed BaseX's openjdk dependency and thus is using openjdk 21),
on macOS Sonoma (14.0).

I'd be happy to provide any other info that would help in identifying the
cause of the crash.

Joe


% basexgui
2023-10-24 07:34:22.308 java[69449:3862142] WARNING: Secure coding is
automatically enabled for restorable state! However, not on all supported
macOS versions of this application. Opt-in to secure coding explicitly by
implementing
NSApplicationDelegate.applicationSupportsSecureRestorableState:.
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0x7ff80d2b1b73, pid=69449, tid=259
#
# JRE version: OpenJDK Runtime Environment Homebrew (21.0) (build 21)
# Java VM: OpenJDK 64-Bit Server VM Homebrew (21, mixed mode, sharing,
tiered, compressed oops, compressed class ptrs, g1 gc, bsd-amd64)
# Problematic frame:
# C  [AppKit+0xc8fb73]  _NSCarbonMenuCrashIfNeeded+0x258
#
# No core dump will be written. Core dumps have been disabled. To enable
core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/joe/workspace/hsg-project/repos/hsg-shell/hs_err_pid69449.log
#
# If you would like to submit a bug report, please visit:
#   https://github.com/Homebrew/homebrew-core/issues
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
zsh: abort  basexgui
<>
<>

Re: [basex-talk] Automated testing for XML databases

[Joe Wicentowski]

Dear Shuxin,

I just wanted to take the opportunity to thank you for the bug reports
you've submitted - both here and to the eXist-db project, as Andy noted. I
really like Christian's suggestion to contribute your test cases to the
official W3C test suite. But whichever means you choose, we welcome your
reports and thank you for your team's work on improving these great open
source tools.

Joe

On Thu, Mar 30, 2023 at 6:27 AM Andy Bunce  wrote:

> Hi Shuxin Li,
>
> It certainly is very impressive work and I am also keen to know more.
>
> I see you have also been working with eXist [1]
> I assume it builds on "Query Plan Guidance (QPG), a test case generation
> method with the guidance of query plans."?  [2]
>
> Best wishes
> /Andy
>
> [1] https://github.com/eXist-db/exist/issues/created_by/Twilight-Shuxin
> [2] https://zenodo.org/record/7623847
>
>
>
> On Thu, 30 Mar 2023 at 10:06, Christian Grün 
> wrote:
>
>> Dear Shuxin Li,
>>
>> I’m impressed by how you repeatedly manage to discover new bugs in
>> BaseX. How do you proceed to uncover the errors? Is it a
>> trial-and-error approach, or are the tests generated automatically? Do
>> you/does your group plan to write an article that describes your
>> approach?
>>
>> All the best from Germany,
>> Christian
>>
>>
>>
>> On Fri, Mar 17, 2023 at 10:22 AM Christian Grün
>>  wrote:
>> >
>> > Dear Shuxin Li,
>> >
>> > Thanks for sharing the bug reports with us. I appreciate that, they’re
>> definitely helpful.
>> >
>> > You could as well think about creating and sending pull requests for
>> your test cases to the official W3 test suite [1]. This way, all
>> implementors of the standard who run the tests may benefit from your
>> observations.
>> >
>> > I'm looking forward to the upcoming reports.
>> >
>> > Thanks again,
>> > Christian
>> >
>> > [1] https://github.com/w3c/qt3tests
>> >
>> >
>> >
>> >
>> >
>> > Shuxin Li  schrieb am Fr., 17. März 2023,
>> 10:01:
>> >>
>> >> Greetings!
>> >>
>> >> I'm Shuxin, an undergraduate CS student, who is currently an intern at
>> NUS Test lab. I’ve been working on an automated testing tool for XML
>> processors and databases. My goal is to find bugs in important systems, in
>> order to improve these systems’ reliability. In the past few days, I
>> submitted a few bug reports on Github (Github id: Twilight-Shuxin). Thanks
>> for having already fixed these!
>> >>
>> >> I would like to ask whether you find these reports useful and would
>> also appreciate future reports if I find more bugs? Any feedback for us is
>> highly appreciated.  We also plan to open-source this tool for public usage
>> after it's fully realized, and truly hope that it might be of help to the
>> community. Thanks!
>> >>
>> >> Best Regards,
>> >> Shuxin Li
>>
>

Re: [basex-talk] GUI doesn't quit on Mac

[Joe Wicentowski]

This works perfectly, Christian. Thank you!

On Wed, Nov 9, 2022 at 4:51 AM Christian Grün 
wrote:

> Hi Joe, hi Steve,
>
> Thanks to Michael’s initiative, a new snapshot is available that may
> solve the problems you observed [1].
>
> Your feedback is welcome,
> Christian
>
> PS: BaseX 10.4 will be released before end of this month.
>
> [1] https://files.basex.org/releases/latest/
>
>
>
>
> On Wed, Nov 9, 2022 at 9:47 AM Michael Seiferle  wrote:
> >
> > Hi Joe,
> >
> > Hi list,
> >
> >
> >
> > I remember having a shared screen-screen-session with Christian not too
> long ago, where we tried to pin down the issue and iirc “it worked back
> then” — but obviously not for too long. It’s a rather odd behavior
> >
> >
> >
> > I fully agree with Joe, whether we click that little X or select Quit
> from the menu bar, the app should fully close.
> >
> >
> >
> > We are looking into this again :-)
> >
> >
> > You may follow my progress here:
> https://github.com/BaseXdb/basex/pull/2149
> >
> >
> > Best
> >
> >
> > Michael
> >
> >
> >
> >
> > Von: BaseX-Talk  im Auftrag
> von Joe Wicentowski 
> > Datum: Mittwoch, 9. November 2022 um 01:21
> > An: Majewski, Steven Dennis (sdm7g) 
> > Cc: BaseX 
> > Betreff: Re: [basex-talk] GUI doesn't quit on Mac
> >
> > As a Mac user, I do not generally expect closing a window via the red
> window button to quit an app. With only a few exceptions (e.g., single
> window only apps like the Calculator), closing any/all of an app's windows
> will keep the app open; apps that remain open after their windows are
> closed generally take a click on the app's Dock icon as a signal to open a
> new window.
> >
> >
> >
> > But I do expect that selecting Quit via the menu bar will fully quit the
> app. Respecting that Quit command is far more important than treating the
> red window button as a request to Quit.
> >
> >
> >
> > A few times I've needed to restart my Mac to install a software update
> or finish installing a piece of software. Typically, these processes
> present the user with a dialog to trigger the restart, which, when
> accepted, causes the OS to quit all open apps. In this scenario, BaseX
> causes the restart to fail, because it will not quit when told, and the
> restart will time out unless you find out how to quit it. (I use control-C
> in the Terminal window where I ran the `basexgui` command.)
> >
> >
> >
> > I hope this sheds some light on the matter. Thanks!
> >
> >
> >
> > On Tue, Nov 8, 2022 at 3:11 PM Majewski, Steven Dennis (sdm7g) <
> sd...@virginia.edu> wrote:
> >
> > It looks like it does exit, after a short delay, when closed by clicking
> the red button on the window, instead of quoting from the menu.
> >
> >
> >
> > — Steve.
> >
> >
> >
> >
> >
> > On Nov 8, 2022, at 3:04 PM, Christian Grün 
> wrote:
> >
> >
> >
> > Thanks for the observation. Mac support was improved in general with
> Java 11, but it still requires some OS-specific tweaks in the code. Our
> current solution may not be complete enough to achieve the expected default
> behavior [1].
> >
> >
> >
> > I’m no Mac user, but does it make a difference if the GUI is quit a) via
> the menu or b) the red icon?
> >
> >
> >
> > [1]
> https://github.com/BaseXdb/basex/blob/main/basex-core/src/main/java/org/basex/gui/layout/BaseXLayout.java#L584
> >
> >
> >
> >
> >
> > Joe Wicentowski  schrieb am Di., 8. Nov. 2022, 20:54:
> >
> > I've been experiencing this problem too on macOS (both 12 Monterey and
> now 13 Ventura), installed via Homebrew, and started via the `basexgui`
> command line command.
> >
> >
> >
> > On Tue, Nov 8, 2022 at 1:02 PM Majewski, Steven Dennis (sdm7g) <
> sd...@virginia.edu> wrote:
> >
> > I’ve been having a problem for a while where after attempting to quit
> the Mac GUI, it doesn’t actually exit and has to be force quit. I’m sorry I
> wasn’t able to track how this tracks with MacOS / JDK / BaseX updates and
> when it actually started happening. Wondering if anyone else has had this
> issue, and whether it’s Mac specific or JDK version specific.
> >
> >
> >
> >
> >
> > MacOS 12.6.1 but problem goes back at least to earlier 12.6 versions
> >
> > Openjdk from home-brew install — currently openjdk/19.0.1, but was
> happening on earlier versions.
> >
> > And the current and last several versions of BaseX
> >
> >
> >
> > — Steve M.
> >
> >
> >
> >
> >
> >
>

Re: [basex-talk] GUI doesn't quit on Mac

[Joe Wicentowski]

As a Mac user, I do not generally expect closing a window via the red
window button to quit an app. With only a few exceptions (e.g., single
window only apps like the Calculator), closing any/all of an app's windows
will keep the app open; apps that remain open after their windows are
closed generally take a click on the app's Dock icon as a signal to open a
new window.

But I do expect that selecting Quit via the menu bar will fully quit the
app. Respecting that Quit command is far more important than treating the
red window button as a request to Quit.

A few times I've needed to restart my Mac to install a software update or
finish installing a piece of software. Typically, these processes present
the user with a dialog to trigger the restart, which, when accepted, causes
the OS to quit all open apps. In this scenario, BaseX causes the restart to
fail, because it will not quit when told, and the restart will time out
unless you find out how to quit it. (I use control-C in the Terminal window
where I ran the `basexgui` command.)

I hope this sheds some light on the matter. Thanks!

On Tue, Nov 8, 2022 at 3:11 PM Majewski, Steven Dennis (sdm7g) <
sd...@virginia.edu> wrote:

> It looks like it does exit, after a short delay, when closed by clicking
> the red button on the window, instead of quoting from the menu.
>
> — Steve.
>
>
> On Nov 8, 2022, at 3:04 PM, Christian Grün 
> wrote:
>
> Thanks for the observation. Mac support was improved in general with Java
> 11, but it still requires some OS-specific tweaks in the code. Our current
> solution may not be complete enough to achieve the expected default
> behavior [1].
>
> I’m no Mac user, but does it make a difference if the GUI is quit a) via
> the menu or b) the red icon?
>
> [1]
> https://github.com/BaseXdb/basex/blob/main/basex-core/src/main/java/org/basex/gui/layout/BaseXLayout.java#L584
>
>
>
> Joe Wicentowski  schrieb am Di., 8. Nov. 2022, 20:54:
>
>> I've been experiencing this problem too on macOS (both 12 Monterey and
>> now 13 Ventura), installed via Homebrew, and started via the `basexgui`
>> command line command.
>>
>> On Tue, Nov 8, 2022 at 1:02 PM Majewski, Steven Dennis (sdm7g) <
>> sd...@virginia.edu> wrote:
>>
>>> I’ve been having a problem for a while where after attempting to quit
>>> the Mac GUI, it doesn’t actually exit and has to be force quit. I’m sorry I
>>> wasn’t able to track how this tracks with MacOS / JDK / BaseX updates and
>>> when it actually started happening. Wondering if anyone else has had this
>>> issue, and whether it’s Mac specific or JDK version specific.
>>>
>>>
>>> MacOS 12.6.1 but problem goes back at least to earlier 12.6 versions
>>> Openjdk from home-brew install — currently openjdk/19.0.1, but was
>>> happening on earlier versions.
>>> And the current and last several versions of BaseX
>>>
>>> — Steve M.
>>>
>>>
>>>
>

Re: [basex-talk] GUI doesn't quit on Mac

[Joe Wicentowski]

I've been experiencing this problem too on macOS (both 12 Monterey and now
13 Ventura), installed via Homebrew, and started via the `basexgui` command
line command.

On Tue, Nov 8, 2022 at 1:02 PM Majewski, Steven Dennis (sdm7g) <
sd...@virginia.edu> wrote:

> I’ve been having a problem for a while where after attempting to quit the
> Mac GUI, it doesn’t actually exit and has to be force quit. I’m sorry I
> wasn’t able to track how this tracks with MacOS / JDK / BaseX updates and
> when it actually started happening. Wondering if anyone else has had this
> issue, and whether it’s Mac specific or JDK version specific.
>
>
> MacOS 12.6.1 but problem goes back at least to earlier 12.6 versions
> Openjdk from home-brew install — currently openjdk/19.0.1, but was
> happening on earlier versions.
> And the current and last several versions of BaseX
>
> — Steve M.
>
>
>

[basex-talk] Future of the BaseX crypto library

[Joe Wicentowski]

Dear fellow BaseX users,

For anyone interested in the future of the EXPath Crypto spec and lib, I
posted an important update to the EXPath community group with questions for
users of the spec and implementations thereof. Please feel free to chime
in: https://github.com/expath/expath-cg/issues/132. Thanks very much to
Christian, who already has. I'd welcome any other users of the crypto
library to register their ideas.

Thank you,
Joe

Re: [basex-talk] Websites that use BaseX

[Joe Wicentowski]

Hi all,

I'd welcome any and all contributions to my list of XQuery-powered
resources:

  https://github.com/joewiz/xquery-power

It would be great to have some entries whose descriptions read, "Runs on
BaseX."

Joe

On Tue, Jun 23, 2020 at 4:04 PM Chris Yocum  wrote:

> Hi Johnathan,
>
> > I have recommended BaseX to some people who are writing an application.
> >
> > The developers, who are more comfortable with SQL, have asked for a list
> of
> > websites that use BaseX.  I imagine they would be happy with other happy
> > customers too, they want to have basic confidence in the tool.
> >
> > So which users should give people confidence that BaseX is the tool for
> > them?  They know about my own happy experience with BaseX, they want to
> > know more about other projects.
>
> The Dictionary of Irish Language from Old and Middle Irish Sources
> (http://www.dil.ie/) has used it for six or more years.  As you can
> imagine, it does not change very often so it is mostly used for
> searching than updating and the project is very happy with BaseX.
>
> All the best,
> Chris
>

Re: [basex-talk] Stack overflow with random-number-generator

[Joe Wicentowski]

I found a version that works.  In the original version I must not have
been using tail recursion correctly.  Here is the version that works.  Can
anyone explain why the first version doesn't properly trigger tail
recursion?

```
xquery version "3.1";

declare function local:get-random-numbers($how-many as xs:integer,
$generator as map(*), $accumulator) {
if ($how-many ge 1) then
local:get-random-numbers($how-many - 1, $generator?next(),
($accumulator, $generator?number))
else
$accumulator
};

declare function local:get-random-numbers($how-many as xs:integer) {
local:get-random-numbers($how-many, random-number-generator(), ())
};

let $n := 2000
return
local:get-random-numbers($n) => count()
```

On Mon, May 11, 2020 at 5:20 PM Joe Wicentowski  wrote:

> Hi all,
>
> I encountered a stack overflow error with the following code, which is
> designed to return as many random numbers as the user requests.  BaseX's
> error suggested, "Try tail recursion?"  I think I am already using tail
> recursion.  Changing the $n variable from 1407 to 1406 or lower avoids the
> error.  Any number 1407 or higher triggers the error.
>
> Thank you for any hints!
> Joe
>
> ```
> xquery version "3.1";
>
> declare function local:get-random-numbers($how-many as xs:integer,
> $generator as map(*)) {
> if ($how-many ge 1) then
> (
> $generator?number,
> local:get-random-numbers($how-many - 1, $generator?next())
> )
> else
> ()
> };
>
> declare function local:get-random-numbers($how-many as xs:integer) {
> local:get-random-numbers($how-many, random-number-generator())
> };
>
> let $n := 1407
> return
> local:get-random-numbers($n) => count()
> ```
>

Re: [basex-talk] BaseX-specific serialization parameters

[Joe Wicentowski]

Thank you, Christian!  Your edit to the documentation completely clears
this up.

Best,
Joe

On Thu, Jan 17, 2019 at 4:22 AM Christian Grün 
wrote:

> Hi Joe,
>
> You are completely right; we ignored this specific rule of the
> specification in the past. One of the reasons is that there are many
> ways in BaseX how parameters can be supplied, and in some cases, no
> namespace prefixes can be declared. I have just updated our
> serialization [1]; I hope that the new wording is helpful.
>
> As you indicated, a change would require various changes in existing
> XQuery libraries. I’ll have some more thoughts on this; maybe we could
> tackle this with version 10 of BaseX.
>
> Thanks for your observation!
> Christian
>
> [1] http://docs.basex.org/wiki/Serialization
>
>
>
> > As I read the serialization spec (
> https://www.w3.org/TR/xslt-xquery-serialization-31/#serparam), additional
> values for the "method" parameter and additional serialization parameters
> should use a non-null namespace URI (see quotes below).
> >
> > I believe BaseX's "csv" method or "tabulator" parameter (
> http://docs.basex.org/wiki/Serialization) would fall into this category.
> Without such a non-null namespace URI, users may mistake BaseX-specific
> serialization features for those supported in the spec.  The solution that
> the spec seems to suggest would be to use "basex:csv" or
> "basex:tabulator".  I bet such a change would be unwelcome to those with
> code that uses the existing supported values, but aside from a disruptive
> cycle of deprecating existing values, etc., an alternative would be for the
> BaseX documentation to note this non-compliance with the spec.
> >
> > I realize this is nit-picky, but I thought it might be worth raising
> given the potential for confusion mentioned above, and should this
> discussion be of interest to anyone.
> >
> > Best,
> > Joe
> >
> > Regarding values for the "method" parameter:
> >
> > > The value of the method parameter is an expanded QName. If the value
> has a null namespace URI, then the local name identifies a method specified
> in this document and MUST be one of xml, html, xhtml, text, json , or
> adaptive; in this case, the output method specified MUST be used for
> serializing. If the namespace URI is non-null, then it identifies an
> implementation-defined output method; the behavior in this case is not
> specified by this document.
> >
> > Regarding additional serialization parameters:
> >
> > > Implementations MAY define additional serialization parameters, and
> MAY allow users to do so. For this purpose, the name of a serialization
> parameter is considered to be a QName; the parameters listed above are
> QNames whose expanded-QName has a null namespace URI, while any additional
> serialization parameters that are either implementation-defined or defined
> by the host language MUST have names that are namespace-qualified.
>

[basex-talk] BaseX-specific serialization parameters

[Joe Wicentowski]

Hi all,

As I read the serialization spec (
https://www.w3.org/TR/xslt-xquery-serialization-31/#serparam), additional
values for the "method" parameter and additional serialization parameters
should use a non-null namespace URI (see quotes below).

I believe BaseX's "csv" method or "tabulator" parameter (
http://docs.basex.org/wiki/Serialization) would fall into this category.
Without such a non-null namespace URI, users may mistake BaseX-specific
serialization features for those supported in the spec.  The solution that
the spec seems to suggest would be to use "basex:csv" or
"basex:tabulator".  I bet such a change would be unwelcome to those with
code that uses the existing supported values, but aside from a disruptive
cycle of deprecating existing values, etc., an alternative would be for the
BaseX documentation to note this non-compliance with the spec.

I realize this is nit-picky, but I thought it might be worth raising given
the potential for confusion mentioned above, and should this discussion be
of interest to anyone.

Best,
Joe

Regarding values for the "method" parameter:

> The value of the method parameter is an expanded QName. If the value has
a null namespace URI, then the local name identifies a method specified in
this document and MUST be one of xml, html, xhtml, text, json , or
adaptive; in this case, the output method specified MUST be used for
serializing. If the namespace URI is non-null, then it identifies an
implementation-defined output method; the behavior in this case is not
specified by this document.

Regarding additional serialization parameters:

> Implementations MAY define additional serialization parameters, and MAY
allow users to do so. For this purpose, the name of a serialization
parameter is considered to be a QName; the parameters listed above are
QNames whose expanded-QName has a null namespace URI, while any additional
serialization parameters that are either implementation-defined or defined
by the host language MUST have names that are namespace-qualified.

Re: [basex-talk] Converting a Date

[Joe Wicentowski]

Hi Ron,

You might find Ryan Grimm's date-parser library module useful if you have a
larger range of date formats to handle:


https://github.com/marklogic-community/commons/blob/master/dates/date-parser.xqy

While it was written with some MarkLogic-specific code, I adapted it for
use with eXist (but haven't tested it with BaseX):


https://github.com/HistoryAtState/twitter/blob/master/modules/date-parser.xqm

Best,
Joe

On Fri, Nov 2, 2018 at 6:48 PM Ron Katriel  wrote:

> Hi Christian,
>
> Much appreciated! I hardened the code (see below) since the dates (from
> CT.gov) occasionally also have the day of the month (e.g., “March 21,
> 2014”). Currently the function is dropping the day in such cases but I will
> look into capturing it in a future iteration.
>
> Best,
> Ron
>
>
> declare function local:to-date($string) {
>   if (fn:matches($string, '[A-Za-z]+ [0-9]+') or fn:matches($string,
> '[A-Za-z]+ [0-9]+, [0-9]+'))
>   then
>   let $m := index-of($MONTHS, substring-before($string, ' '))
>   let $y := xs:integer(functx:substring-after-last($string, ' '))
>   return xs:date(string-join(
> (
>   format-number($y, ''),
>   format-number($m, '00'),
>   '01'
> ),
> '-')
>   )
>   else
>   ()
> };
>
> On November 2, 2018 at 4:20:41 PM, Christian Grün (
> christian.gr...@gmail.com) wrote:
>
> Hi Ron,
>
> If your timestamp is available in IETF format, you can use
> fn:parse-ietf-date [1]. Otherwise, you’ll need to write a simple
> function by yourself:
>
> declare variable $MONTHS := (
> 'January', 'February', 'March', 'April', 'May', 'June',
> 'July', 'August', 'September', 'October', 'November', 'December'
> );
>
> declare function local:to-date($string) {
> let $m := index-of($MONTHS, substring-before($string, ' '))
> let $y := xs:integer(substring-after($string, ' '))
> return xs:date(string-join((
> format-number($y, ''),
> format-number($m, '00'),
> '01'
> ), '-'))
> };
> local:to-date('March 2017')
>
> Best,
> Christian
>
> [1]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.w3.org_TR_xpath-2Dfunctions-2D31_-23func-2Dparse-2Dietf-2Ddate=DwIFaQ=fi2D4-9xMzmjyjREwHYlAw=44jDQvzmnB_-ovfO6Iusj0ItciJrcWMOQQwd2peEBBE=Yy9sC1xS_Ix-pGSVDp-Lbmz8BOft0S1WdVpEM-qzRw4=uQ3NqCv8FpHulP4q1arjItJX3-gCHwi_06WN4znRz48=
>
>
>
> On Fri, Nov 2, 2018 at 9:09 PM Ron Katriel  wrote:
> >
> > Hi,
> >
> > Is there a BaseX function for converting a string date in the form of
> “March 2017” to xs:date or xs:dateTime?
> >
> > Thanks,
> > Ron
>
>

Re: [basex-talk] http:send-request - problem with form data

[Joe Wicentowski]

The Zorba HTTP spec is very interesting. I think the original EXPath HTTP
spec (and others) may have taken a more map-and-array form had those data
structures been available in XQuery at the time. It would be interesting to
reconsider some of these specs in a future major revision to them.

On Thu, Jul 12, 2018 at 4:33 PM Christian Grün 
wrote:

> Hi Bogdan,
>
> > I have one doubt why when we don't define body in http request third
> parameter is ignored ?
>
> It was defined like this in the EXPath spec [1]:
>
> “When a body element has an empty content (i.e. it has no child node
> at all) its content is given by the parameter $bodies. In a single
> part request, this param must have at most one item. If the body is
> empty, the param cannot be the empty sequence. In a multipart request,
> $bodies must have as many items as there are empty body elements. If
> there are three empty body elements, the content of the first of them
> is $bodies[1], and so on. The number of empty body elements must be
> equal to the number of items in $bodies.”
>
> I guess it would have been more user-friendly if an error was raised
> if these requirements are not met by the function call.
>
> Anyway, as indicated, I’d be happy if we had functions with
> unambiguous function arguments in the future. The Zorba HTTP Client
> Module could serve as inspiration [2]. I have never tried it in
> practice, but it looks pretty well-designed (an additional options
> argument should probably added to the simple functions).
>
> Best,
> Christian
>
> [1] http://expath.org/spec/http-client#d2e362
> [2] http://www.zorba.io/documentation/3.0/modules/zorba/io/http-client/
>
>
>
> > Only when body element is provided parseBody with bodies is invoked.
> > I have to check source code to figure out how to pass body payload via
> third argument.
> >
> > if(body != null) {
> > final QNm pl = body.qname();
> > // single part request
> > if(pl.eq(Q_BODY)) {
> >   parseBody(body, bodies, hr.payloadAtts, hr.payload);
> >   hr.isMultipart = false;
> >   // multipart request
> > } else if(pl.eq(Q_MULTIPART)) {
> >   parseMultipart(body, bodies.iter(), hr.payloadAtts, hr.parts);
> >   hr.isMultipart = true;
> > } else {
> >   throw HC_REQ_X.get(info, "Unknown payload element: " +
> body.qname());
> > }
> > }
> >
> > Best Regards
> > Bogdan Bogucki
> >
> >
> > -Wiadomość oryginalna-
> > Od: Christian Grün [mailto:christian.gr...@gmail.com]
> > Wysłano: 12 lipca 2018 15:36
> > Do: Bogdan Bogucki
> > DW: BaseX
> > Temat: Re: [basex-talk] http:send-request - problem with form data
> >
> > Hi Bogdan,
> >
> > I finally had some time to get to the bottom of the conversion
> inconsistencies:
> >
> > If the value is supplied as child of the http:body node, it will be
> serialized as text node. If it’s supplied as extra parameter, it will be
> serialized as string. If text nodes are serialized, entities will be
> encoded. Strings will be returned without encoded entities:
> >
> >  →   
> >   ``[&]``   →   &
> >
> > I agree this is not very intuitive, and I have now rewritten this as
> > follows: If the supplied output media-type does not result in one of the
> default serialization methods of the spec (xml, xhtml, html, json, text,
> adaptive), the text node will be atomized (i.e., converted to an item of
> type xs:untypedAtomic), which will then be serialized as plain string. With
> the new snapshot [1], it should make no difference if you supply your value
> as child of http:body or as extra argument.
> >
> > As a general guide, I would recommended everyone to use the third
> argument for body items. It will be both safer and faster (in particular if
> binary data can be streamed). As the rich syntax of http:send-request is
> regularly confusing users on the list, I have reopened an old Github issue,
> which is about providing additional and simpler functions for the most
> widely used HTTP methods [2].
> >
> > Best,
> > Christian
> >
> > [1] http://files.basex.org/releases/latest/
> > [2] https://github.com/BaseXdb/basex/issues/914
> >
> >
> >
> > On Wed, Jul 4, 2018 at 10:05 PM Christian Grün <
> christian.gr...@gmail.com> wrote:
> > >
> > > And thanks for the confirmation. I will try to find out why the two
> alternatives make a difference.
> > >
> > >
> > >
> > > Bogdan Bogucki  schrieb am Mi., 4. Juli 2018,
> 21:56:
> > >>
> > >> Hello,
> > >>
> > >> It works with body as a third parameter. Body has to be defined in
> http request because without that third parameter is ignored.
> > >>
> > >> Working example:
> > >>
> > >> for $x in http:send-request(
> > >>   
> > >> 
> > >> 
> > >>   
> > >> ,'http:localhost:', 'start=0length=100
> > >> ')[2]/data/results/content return $x
> > >>
> > >> Thank you Christian for help.
> > >>
> > >> Best Regards
> > >> Bogdan Bogucki
> > >>
> > >>
> > >> -Wiadomość oryginalna-
> > >> Od: Christian Grün 

Re: [basex-talk] Differences in serialization of arrays with JSON vs. adaptive methods

[Joe Wicentowski]

Hi Christian


> Our custom renderer was optimized for
> performance. It’s possible to open and highlight pretty large XML
> documents, but the highlighting is very basic.


Ah, I see.  Very interesting.

rendered queries will even be “bluer” than before, but the overall
> appearance will hopefully be more consistent.
>

Having tried out the snapshot, I can confirm the queries are bluer!

Best,
Joe

Re: [basex-talk] Differences in serialization of arrays with JSON vs. adaptive methods

[Joe Wicentowski]

Hi Christian,

I just gave it a try, and the new serialization preferences work like a
charm.

It's pretty amazing to see a request go from idea to implementation so
quickly - thank you!  I think it's a nice touch that serialization options
specified in a query are respected, so the preferences only affect the
defaults that apply to a query - they don't override anything in a query.

Your listing of ambiguities in the adaptive method are certainly valid.
The quote escaping does bother me, so it does come at a cost, I guess.  I
do find myself quickly switching away to JSON or text serialization to
avoid some of these quirks when I don't like the look of adaptive.

And, to close the loop, I agree that the spec remains ambiguous in the area
of indentation of maps and arrays, and while BaseX in no way violates the
spec here, I personally would advocate for consistency in the way maps and
arrays are indented and serialized across the various map- and array-aware
methods.  But this is a terribly minor point, and folks who want uniformity
can always use another tool to pretty-print their data.

Finally, I just remembered observation that came up during the class - the
BaseX GUI editor's syntax highlighting sometimes breaks down in odd places,
such as the middle of a function name, e.g., in `fn:format-date()`, the
`fn` and `-date` are colored black, but `format` is colored blue.  I've
attached a screenshot showing the phenomenon.  Again, this isn't a major
issue, and syntax highlighting probably can't always be perfect, but I
thought I'd mention it since it came up.

Thanks again for all of your kind assistance!
Joe

On Thu, Aug 10, 2017 at 11:29 PM, Christian Grün 
wrote:

> Hi there,
>
> To begin with: I forgot to mention that you can change the
> serialization method by launching the command "set serializer
> method=adaptive" in the command input text field on top of the BaseX
> window.
>
> I agree this is not very comfortable, so I have now added a new
> interaction component for changing all serialization parameters in the
> GUI. I have decided to move the input components to the Preferences
> dialog (Ctrl-Shift-P, Visualization panel), as I’d like to keep the
> main interface clean. If more people ask for it, I might add a
> dropdown menu for the serialization method on top of the Result View,
> let’s see.
>
> The new component for adjusting the serialization parameters is also
> available now in the Export Database dialog. A new stable snapshot is
> available [1], and the next minor release will be available around end
> of August.
>
> Looking forward to your feedback,
> Christian
>
> [1] http://files.basex.org/releases/latest/
>
>
>
> On Fri, Aug 11, 2017 at 12:40 AM, Christian Grün
>  wrote:
> > Hi Joe,
> >
> >> Have you considered adding a preference or toggle for selecting the
> default
> >> serialization method used in the GUI's results?
> >
> > Sounds like an enticing idea! Something similar is embedded in our
> > Database Export dialog (see menü items 'Database', 'Export…'). I
> > haven’t touched it for years, and it could surely be revised as well.
> > I will definitely think about adding something like this in to our
> > Result View [1].
> >
> >> From my perspective in teaching XQuery, showing an
> >> xs:string item in quotes (and integers sans quotes) helps reinforce the
> >> concept of data types.
> >
> > This is a good thought indeed.
> >
> >> Besides string handling, though, are
> >> there other aspects of "adaptive" that you dislike compared to the
> default
> >> "basex" method?
> >
> > I would say that both methods (now) serve different purposes:
> >
> > • Our 'basex' method was included because BaseX is used in many
> > different contexts, and we were looking for a single serialization
> > method that can be used for as many use cases as possible at the same
> > time. If BaseX is used on command-line, it can be convenient if the
> > textual output (usually XML, strings, numbers) can directly be passed
> > on to other commands, or saved in text files. If the GUI is used, text
> > from the result view can be copied and pasted to other tools (such as
> > CSV output, which can be pasted in Excel, etc.).
> >
> > • The 'adaptive' simplifies the recycling of results in other XQuery
> > expressions. I agree it also helps users to understand the differences
> > between data types. I find it a bit confusing, however, that some
> > items will be output with a constructor function, whereas other will
> > simply be output as strings. Some examples:
> >
> >   1,
> >   xs:double(1),
> >   'a"b',
> >   xs:anyURI('a"b'),
> >   xs:QName('xml:x'),
> >   /@a
> >
> > …will be serialized as…
> >
> >   1
> >   1
> >   "a""b"
> >   xs:anyURI("a""b")
> >   Q{http://www.w3.org/XML/1998/namespace}x
> >   a="b"
> >
> > It would probably have been more consistent to create output that can
> > always be reused, and that always contains the datatype:
> >
> >   

Re: [basex-talk] Differences in serialization of arrays with JSON vs. adaptive methods

[Joe Wicentowski]

Hi Christian,

I actually quite like the adaptive serialization method and have made it
the default in eXide.  From my perspective in teaching XQuery, showing an
xs:string item in quotes (and integers sans quotes) helps reinforce the
concept of data types.  It feels to me like a datatype-sensitive view of
results, rather than a debug method.  Besides string handling, though, are
there other aspects of "adaptive" that you dislike compared to the default
"basex" method?

Have you considered adding a preference or toggle for selecting the default
serialization method used in the GUI's results?  For
comparison/inspiration, you might see the screenshots of the serialization
method dropdown and indent checkbox I added to eXide:
https://github.com/wolfgangmm/eXide/pull/168#issuecomment-307592370 - which
in my mind makes eXide quite a powerful for quickly experimenting with
different serializations of results - a particularly time-saving feature
given how verbose the boilerplate is for specifying serialization methods.

Joe

On Thu, Aug 10, 2017 at 5:04 PM, Christian Grün <christian.gr...@gmail.com>
wrote:

> Hi Joe,
>
> Thanks for the link. So I noticed that you were quoting exactly the same
> phrase of the spec as I did. ;)
>
> I just checked what Saxon does: It seems to ignore the value of the indent
> parameter when serializing arrays with the adaptive method.
>
> So I guess that every implementation of XQuery 3.1 serializes arrays
> slightly differently, and the spec is probably too fuzzy to give a more
> precise answer.
>
> In general, I would have been happy if the adaptive method had been
> renamed to 'debug', and if another method had been added to the spec that
> is similar to our custom 'basex' method (which allows users to serialize
> all items – including maps, arrays and attributes – in a flavor that does
> not look like debugging output). In fact the initial version of the
> 'adaptive' method was more similar to ours (for example, strings were
> output without quotes). It changed a lot over the time, and we eventually
> decided to include a custom method.
>
> Well, it’s easy to ask for new features, and much more demanding to write
> specifications that satisfy everyone.
>
> Christian
>
>
>
>
> Am 10.08.2017 9:52 nachm. schrieb "Joe Wicentowski" <joe...@gmail.com>:
>
> Hi Christian,
>
> Thanks for your reply.  I agree that the spec is not entirely clear here,
> but my understanding of the spec was based on the interpretations advanced
> by Michael Kay and Liam Quin on this xquery-talk thread about the question
> of indentation under the adaptive method:
>
>   http://markmail.org/message/dixi7e7qq2ttde74
>
> Joe
>
> On Thu, Aug 10, 2017 at 1:37 PM, Christian Grün <christian.gr...@gmail.com
> > wrote:
>
>> Dear Joe,
>>
>> Thanks for the kind feedback. I am glad to hear BaseX was useful in
>> your DH 2017 workshops.
>>
>> > the serialization spec notes that the adaptive method delegates the
>> handling
>> > of the "indent" parameter to JSON.
>>
>> Could you possibly point me to this rule in the spec? I remember there
>> was a lot discussion about the adaptive serialization method in the W3
>> Working Group. As it was difficult to define rules that cover
>> requirements of all members, the initial version differs quite a lot
>> from the final proposal, and various details were left to the
>> implementation (because it was assumed that the method will mostly be
>> used for debugging). I looked up the final version serialization spec
>> [1], which states in 10.1.4 that:
>>
>>   “The indent and suppress-indentation parameters are
>>   not directly applicable to the Adaptive output method.”
>>
>> In BaseX, the parameter is considered indeed when serializing maps and
>> arrays (and other data types as well), but there are various
>> differences between the two serialization methods. Consider the
>> following example (which should also work with other XQuery
>> processors):
>>
>>   xquery version "3.1";
>>   for $method in ('adaptive', 'json')
>>   return (
>> "METHOD: " || $method,
>> "OUTPUT: " || (
>>   try {
>> serialize(
>>   map { 'functions': [ false#0, true#0 ]},
>>   map { 'method': $method }
>> )
>>   } catch * {
>> $err:description
>>   }
>> )
>>   )
>>
>> The adaptive can be used to serialize items of any type, whereas the
>> json method is restricted to types that can be represented in JSON.
>>
>> Does this help?
>> C

[basex-talk] Differences in serialization of arrays with JSON vs. adaptive methods

[Joe Wicentowski]

Hi all,

First, I'm just back from DH2017, where Clifford Anderson and I taught two
workshops on XQuery using BaseX, along with eXist and Saxon.  BaseX
performed like a champ.  We were able to configure the GUI window to show
just the query and results windows—perfect when you're projecting the
screen in a large room and want everyone to see.  Many thanks for such a
great teaching tool!  (Our materials are at
https://github.com/CliffordAnderson/XQuery4Humanists.)

Back to the topic of this post, though, I noticed a slight difference
between BaseX's serialization of arrays when using JSON vs. adaptive
methods: with JSON, the array's items are separated by newlines, whereas
with adaptive, the items are separated by spaces.  This is interesting
since the serialization spec notes that the adaptive method delegates the
handling of the "indent" parameter to JSON.  Some code to reproduce this is
below.

I'm curious to know - is there a particular reason for this difference?

Thanks,
Joe


serialization-test.xq
```xquery
xquery version "3.1";

declare namespace output="http://www.w3.org/2010/xslt-xquery-serialization;;
let $array := ["Cheapside","London","Dean Prior","Devon"]
for $method in ("json", "adaptive")
let $serialization-parameters :=
  
{$method}
yes
  
return
  fn:serialize($array, $serialization-parameters)
```

serialization-test_results.txt
```txt
[
  "Cheapside",
  "London",
  "Dean Prior",
  "Devon"
]
["Cheapside", "London", "Dean Prior", "Devon"]
```

Re: [basex-talk] suddenly...java function x is unknown

[Joe Wicentowski]

A guess out of left field: do you have two versions of the same library on
your class path?

On Thu, Mar 30, 2017 at 7:10 PM Erik Peterson  wrote:

> I've been integrating basex with 3rd party java jar files by dropping them
> in the basex/lib directory. This has worked fine until today when I now git
> "java function x is unknown". I've...
> - re-pulled everything from our repo to make sure all binaries are as they
> were when working
> - restarted osx
> - updated java
>
> - using BaseX VERSION 8.6.1 (February 24, 2017)
>
> It's working on other dev machines but not mine. Why would BaseX stop
> loading the classes from the lib jar files? Is there a better way I should
> be deploying jars? Thanks.
>
-- 
Sent from my iPhone

Re: [basex-talk] csv:parse in the age of XQuery 3.1

[Joe Wicentowski]

Hi all,

Forgive me.  Rather than post more code in this thread, I've created a
gist with revised code that resolves some inconsistencies in what I
posted here earlier.

  https://gist.github.com/joewiz/7581205ab5be46eaa25fe223acda42c3

Again, this isn't a full-featured CSV parser by any means; it assumes
fairly uniform CSV.  Its contribution is that it is a fairly concise
XQuery implementation that works around the absence of
lookahead/lookbehind regex support in XPath.

Joe

Re: [basex-talk] csv:parse in the age of XQuery 3.1

[Joe Wicentowski]

And corrected query body:

let $csv := 'Author,Title,ISBN,Binding,Year Published
Jeannette Walls,The Glass Castle,074324754X,Paperback,2006
James Surowiecki,The Wisdom of Crowds,9780385503860,Paperback,2005
Lawrence Lessig,The Future of Ideas,9780375505782,Paperback,2002
"Larry Bossidy, Ram Charan, Charles
Burck",Execution,9780609610572,Hardcover,2002
Kurt Vonnegut,Slaughterhouse-Five,9780791059258,Paperback,1999'
let $lines := tokenize($csv, '\n')
let $header-row := fn:head($lines)
let $body-rows := fn:tail($lines)
let $headers := local:get-tokens($header-row) ! replace(., '\s+', '_')
for $row in $body-rows
let $cells := local:get-tokens($row)
return
element row {
  for $cell at $count in $cells
  return element {$headers[$count]} {$cell}
}

Re: [basex-talk] csv:parse in the age of XQuery 3.1

[Joe Wicentowski]

Sorry, a typo crept in.  Here's the corrected function:

declare function local:get-cells($row as xs:string) as xs:string {
(: workaround lack of lookahead support in XPath: end row with comma :)
let $string-to-analyze := $row || ","
let $analyze := fn:analyze-string($string-to-analyze, '(("[^"]*")+|[^,]*),')
for $group in $analyze//fn:group[@nr="1"]
return
if (matches($group, '^".+"$')) then
replace($group, '^"([^"]+)"$', '$1')
else
$group/string()
};

Re: [basex-talk] csv:parse in the age of XQuery 3.1

[Joe Wicentowski]

Hi Christian,

Yes, that sounds like the culprit.  Searching back through my files,
Adam Retter responded on exist-open (at
http://markmail.org/message/3bxz55du3hl6arpr) to a call for help with
the lack of lookahead support in XPath, by pointing to an XSLT he
adapted for CSV parsing,
https://github.com/digital-preservation/csv-tools/blob/master/csv-to-xml_v3.xsl.
I adapted this technique to XQuery, and it works on the sample case in
my earlier email.

Joe

```xquery
xquery version "3.1";

declare function local:get-cells($row as xs:string) as xs:string {
(: workaround lack of lookahead support in XPath: end row with comma :)
let $string-to-analyze := $row || ","
let $analyze := fn:analyze-string($row, '(("[^"]*")+|[^,]*),')
for $group in $analyze//fn:group[@nr="1"]
return
if (matches($group, '^".+"$')) then
replace($group, '^"([^"]+)"$', '$1')
else
$group/string()
};

let $csv := 'Author,Title,ISBN,Binding,Year Published
Jeannette Walls,The Glass Castle,074324754X,Paperback,2006
James Surowiecki,The Wisdom of Crowds,9780385503860,Paperback,2005
Lawrence Lessig,The Future of Ideas,9780375505782,Paperback,2002
"Larry Bossidy, Ram Charan, Charles
Burck",Execution,9780609610572,Hardcover,2002
Kurt Vonnegut,Slaughterhouse-Five,9780791059258,Paperback,1999'
let $lines := tokenize($csv, '\n')
let $header-row := fn:head($lines)
let $body-rows := fn:tail($lines)
let $headers := local:get-cells($header-row)
for $row in $body-rows
let $cells := local:get-cells($row)
return
element row {
  for $cell at $count in $cells
  return element {$headers[$count]} {$cell}
}
```

On Mon, Sep 12, 2016 at 10:11 AM, Christian Grün
 wrote:
>> Christian: I tried removing the quote escaping but still get an error.
>> Here's a small test to reproduce:
>>
>> fn:analyze-string($row, '(?:\s*(?:"([^"]*)"|([^,]+))\s*,?|(?<=,)(),?)+?')
>
> I assume it’s the lookbehind assertion that is not allowed in XQuery
> (but I should definitely spend more time on it to give you a better
> answer..).

Re: [basex-talk] csv:parse in the age of XQuery 3.1

[Joe Wicentowski]

Hi all,

Christian: I completely agree, CSV is a nightmare.  One way to reduce
the headaches (in, say, developing an EXPath CSV library) might be to
require that CSV pass validation by a tool such as
http://digital-preservation.github.io/csv-validator/.  Adam Retter
presented his work on CSV Schema and CSV Validator at
http://slides.com/adamretter/csv-validation.  This might require the
user to fix issues in the CSV first, but would reduce the scope of
variation considerably.  I notice that the Jackson CSV parser
leverages the notion of a schema in its imports:
https://github.com/FasterXML/jackson-dataformat-csv.

Hans-Jürgen: Thanks for the pointer to your library - it looks
fantastic.  I look forward to trying it out.

Liam: Thanks for the info about XQuery's additional regex handling beyond XSD.

And, lastly, to keep this post still basex related...

Christian: I tried removing the quote escaping but still get an error.
Here's a small test to reproduce:

xquery version "3.1";

let $row := '"Larry Bossidy, Ram Charan, Charles
Burck",Execution,9780609610572,Hardcover,2002'
return
fn:analyze-string($row, '(?:\s*(?:"([^"]*)"|([^,]+))\s*,?|(?<=,)(),?)+?')

Joe

On Mon, Sep 12, 2016 at 7:29 AM, Christian Grün
 wrote:
> I didn’t check the regex in general, but one reason I think why it
> fails is the escaped quote. For example, the following query is
> illegal in XQuery 3.1…
>
>   matches('a"b', 'a\"b')
>
> …where as the following one is ok:
>
>   matches('a"b', 'a"b')
>
>
>
> On Mon, Sep 12, 2016 at 1:15 PM, Hans-Juergen Rennau  wrote:
>> Cordial thanks, Liam - I was not aware of that!
>>
>> @Joe: Rule of life: when one is especially sure to be right, one is surely
>> wrong, and so was I, and right were you(r first two characters).
>>
>>
>> Liam R. E. Quin  schrieb am 5:54 Montag, 12.September 2016:
>>
>>
>> Hans-Jürgen, wrote:
>>
>> ! Already the first
>>> two characters
>>> (?render the expression invalid:(1) An unescaped ? is an
>>> occurrence indicator, making the preceding entity optional(2) An
>>> unescaped ( is used for grouping, it does not repesent anything
>>> => there is no entity preceding the ? which the ? could make optional
>>> => error
>>
>>
>> Actually (?:  ) is a non-capturing group, defined in XPath 3.0 and
>> XQuery 3.0, based on the same syntax in other languages.
>>
>> This extension, like a number of others, is useful because the
>> expression syntax defined by XSD doesn't make use of capturing groups
>> (there's no \1 or $1 or whatever), and so it doesn't need non-capturing
>> groups, but in XPath and XQuery they are used.
>>
>> See e.g. https://www.w3.org/TR/xpath-functions-30/#regex-syntax
>>
>> Liam
>>
>>
>> --
>> Liam R. E. Quin 
>> The World Wide Web Consortium (W3C)
>>
>>
>>

Re: [basex-talk] csv:parse in the age of XQuery 3.1

[Joe Wicentowski]

Hans-Jürgen,
I figured as much. I wonder if we can come up with an xsd-compliant regex for 
this purpose? It may not give us a full-featured CSV parser, but would handle 
reasonably uniform cases.
Joe

Sent from my iPhone




On Sun, Sep 11, 2016 at 3:39 PM -0400, "Hans-Juergen Rennau" <hren...@yahoo.de> 
wrote:










Joe, concerning your regex, I would complain, too! Already the first two 
characters 
    (?render the expression invalid:(1) An unescaped ? is an occurrence 
indicator, making the preceding entity optional(2) An unescaped ( is used for 
grouping, it does not repesent anything
=> there is no entity preceding the ? which the ? could make optional => error

Please keep in mind that the regex flavor supported by XPath is the regex 
flavor defined by the XSD spec. There are a few constructs used in Perl & Co 
which are not defined in XPath regex.

What concerns the CSV implementation, I came to realize my error: the BaseX 
implementation *is* Java code, not XQuery code - the xqm module just contains 
the function signature, marked "external".
Cheers,Hans
 

Joe Wicentowski <joe...@gmail.com> schrieb am 21:27 Sonntag, 11.September 
2016:
  

 Thanks for your replies and interest, Hans-Jürgen, Marc, Vincent, and 
Christian.

The other day, short of a comprehensive solution, I went in search of
a regex that would handle quoted values that contain commas that
shouldn't serve as delimiters.  I found one that worked in eXist but
not in BaseX.

Source for the regex: http://stackoverflow.com/a/13259681/659732

The query:

```
xquery version "3.1";

let $csv := 'Author,Title,ISBN,Binding,Year Published
Jeannette Walls,The Glass Castle,074324754X,Paperback,2006
James Surowiecki,The Wisdom of Crowds,9780385503860,Paperback,2005
Lawrence Lessig,The Future of Ideas,9780375505782,Paperback,2002
"Larry Bossidy, Ram Charan, Charles
Burck",Execution,9780609610572,Hardcover,2002
Kurt Vonnegut,Slaughterhouse-Five,9780791059258,Paperback,1999'
let $lines := tokenize($csv, '
')
let $header-row := fn:head($lines)
let $body-rows := fn:tail($lines)
let $headers := fn:tokenize($header-row, ",") ! fn:replace(., " ", "")
for $row in $body-rows
let $cells := fn:analyze-string($row,
'(?:\s*(?:\"([^\"]*)\"|([^,]+))\s*,?|(?<=,)(),?)+?')//fn:group
return
    element Book {
      for $cell at $count in $cells
      return element {$headers[$count]} {$cell/string()}
    }
It produces the desired results:


    Jeannette Walls
    The Glass Castle
    074324754X
    Paperback
    2006


    James Surowiecki
    The Wisdom of Crowds
    9780385503860
    Paperback
    2005


    Lawrence Lessig
    The Future of Ideas
    9780375505782
    Paperback
    2002


    Larry Bossidy, Ram Charan, Charles Burck
    Execution
    9780609610572
    Hardcover
    2002


    Kurt Vonnegut
    Slaughterhouse-Five
    9780791059258
    Paperback
    1999


Unfortunately BaseX complains about the regex, with the following error:

Stopped at /Users/joe/file, 9/32: [FORX0002] Invalid regular
expression: (?:\s(?:\"([^\"])\"|([^,]+))\s*,?|(?<=,)(),?)+?.

Without a column location, I'm unable to tell where the problem is.
Is there something used in this expression that BaseX doesn't support?

On the topic of the potential memory pitfalls of a pure XQuery
solution for our hypothetical EXPath library, I think the primary
problem is that the entire CSV has to be loaded into memory.  I wonder
if implementations could use the new `fn:unparsed-text-lines()`
function from XQuery 3.0 to stream the CSV through XQuery without
requiring the entire thing to be in memory?  Or are we basically
setting ourselves up for the EXPath solution being a wrapper around an
external library written in a lower level language?

Joe

On Sun, Sep 11, 2016 at 4:53 AM, Christian Grün
<christian.gr...@gmail.com> wrote:
> Hi Joe,
>
> Thanks for your mail. You are completely right, using an array would
> be the natural choice with csv:parse. It’s mostly due to backward
> compatibility that we didn’t update the function.
>
> @All: I’m pretty sure that all of us would like having an EXPath spec
> for parsing CSV data. We still need one volunteer to make it happen ;)
> Anyone out there?
>
> Cheers
> Christian
>
>
> On Thu, Sep 8, 2016 at 6:13 AM, Joe Wicentowski <joe...@gmail.com> wrote:
>> Dear BaseX developers,
>>
>> I noticed in example 3 under
>> http://docs.basex.org/wiki/CSV_Module#Examples that csv:parse() with
>> option { 'format': 'map' } returns a map of maps, with hardcoded row
>> numbers:
>>
>> map {
>>     1: map {
>>         "City": "Newton",
>>         "Name": "John"
>>     },
>>     2: map {
>>         "City": "Oldtown",
>>    

Re: [basex-talk] csv:parse in the age of XQuery 3.1

[Joe Wicentowski]

Thanks for your replies and interest, Hans-Jürgen, Marc, Vincent, and Christian.

The other day, short of a comprehensive solution, I went in search of
a regex that would handle quoted values that contain commas that
shouldn't serve as delimiters.  I found one that worked in eXist but
not in BaseX.

Source for the regex: http://stackoverflow.com/a/13259681/659732

The query:

```
xquery version "3.1";

let $csv := 'Author,Title,ISBN,Binding,Year Published
Jeannette Walls,The Glass Castle,074324754X,Paperback,2006
James Surowiecki,The Wisdom of Crowds,9780385503860,Paperback,2005
Lawrence Lessig,The Future of Ideas,9780375505782,Paperback,2002
"Larry Bossidy, Ram Charan, Charles
Burck",Execution,9780609610572,Hardcover,2002
Kurt Vonnegut,Slaughterhouse-Five,9780791059258,Paperback,1999'
let $lines := tokenize($csv, '\n')
let $header-row := fn:head($lines)
let $body-rows := fn:tail($lines)
let $headers := fn:tokenize($header-row, ",") ! fn:replace(., " ", "")
for $row in $body-rows
let $cells := fn:analyze-string($row,
'(?:\s*(?:\"([^\"]*)\"|([^,]+))\s*,?|(?<=,)(),?)+?')//fn:group
return
element Book {
  for $cell at $count in $cells
  return element {$headers[$count]} {$cell/string()}
}
It produces the desired results:


Jeannette Walls
The Glass Castle
074324754X
Paperback
2006


James Surowiecki
The Wisdom of Crowds
9780385503860
Paperback
2005


Lawrence Lessig
The Future of Ideas
9780375505782
Paperback
2002


Larry Bossidy, Ram Charan, Charles Burck
Execution
9780609610572
Hardcover
2002


Kurt Vonnegut
Slaughterhouse-Five
9780791059258
Paperback
1999


Unfortunately BaseX complains about the regex, with the following error:

Stopped at /Users/joe/file, 9/32: [FORX0002] Invalid regular
expression: (?:\s(?:\"([^\"])\"|([^,]+))\s*,?|(?<=,)(),?)+?.

Without a column location, I'm unable to tell where the problem is.
Is there something used in this expression that BaseX doesn't support?

On the topic of the potential memory pitfalls of a pure XQuery
solution for our hypothetical EXPath library, I think the primary
problem is that the entire CSV has to be loaded into memory.  I wonder
if implementations could use the new `fn:unparsed-text-lines()`
function from XQuery 3.0 to stream the CSV through XQuery without
requiring the entire thing to be in memory?  Or are we basically
setting ourselves up for the EXPath solution being a wrapper around an
external library written in a lower level language?

Joe

On Sun, Sep 11, 2016 at 4:53 AM, Christian Grün
<christian.gr...@gmail.com> wrote:
> Hi Joe,
>
> Thanks for your mail. You are completely right, using an array would
> be the natural choice with csv:parse. It’s mostly due to backward
> compatibility that we didn’t update the function.
>
> @All: I’m pretty sure that all of us would like having an EXPath spec
> for parsing CSV data. We still need one volunteer to make it happen ;)
> Anyone out there?
>
> Cheers
> Christian
>
>
> On Thu, Sep 8, 2016 at 6:13 AM, Joe Wicentowski <joe...@gmail.com> wrote:
>> Dear BaseX developers,
>>
>> I noticed in example 3 under
>> http://docs.basex.org/wiki/CSV_Module#Examples that csv:parse() with
>> option { 'format': 'map' } returns a map of maps, with hardcoded row
>> numbers:
>>
>> map {
>> 1: map {
>> "City": "Newton",
>> "Name": "John"
>> },
>> 2: map {
>> "City": "Oldtown",
>> "Name": "Jack"
>> }
>> }
>>
>> Using maps, which are unordered, to represent something ordered like
>> rows in a CSV, hardcoded row numbers are necessary for reassembling
>> the map in document order.  I assume this was a necessary approach
>> when the module was developed in the map-only world of XQuery 3.0.
>> Now that 3.1 supports arrays, might an array of maps be a closer fit
>> for CSV parsing?
>>
>> array {
>> map {
>> "City": "Newton",
>> "Name": "John"
>> },
>> map {
>> "City": "Oldtown",
>> "Name": "Jack"
>> }
>> }
>>
>> I'm also curious, do you know of any efforts to create an EXPath spec
>> for CSV?  Putting spec and CSV in the same sentence is dangerous,
>> since CSV is a notoriously under-specified format: "The CSV file
>> format is not standardized" (see
>> https://en.wikipedia.org/wiki/Comma-separated_values).  But perhaps
>> there is a common enough need for CSV par

[basex-talk] csv:parse in the age of XQuery 3.1

[Joe Wicentowski]

Dear BaseX developers,

I noticed in example 3 under
http://docs.basex.org/wiki/CSV_Module#Examples that csv:parse() with
option { 'format': 'map' } returns a map of maps, with hardcoded row
numbers:

map {
1: map {
"City": "Newton",
"Name": "John"
},
2: map {
"City": "Oldtown",
"Name": "Jack"
}
}

Using maps, which are unordered, to represent something ordered like
rows in a CSV, hardcoded row numbers are necessary for reassembling
the map in document order.  I assume this was a necessary approach
when the module was developed in the map-only world of XQuery 3.0.
Now that 3.1 supports arrays, might an array of maps be a closer fit
for CSV parsing?

array {
map {
"City": "Newton",
"Name": "John"
},
map {
"City": "Oldtown",
"Name": "Jack"
}
}

I'm also curious, do you know of any efforts to create an EXPath spec
for CSV?  Putting spec and CSV in the same sentence is dangerous,
since CSV is a notoriously under-specified format: "The CSV file
format is not standardized" (see
https://en.wikipedia.org/wiki/Comma-separated_values).  But perhaps
there is a common enough need for CSV parsing that such a spec would
benefit the community?  I thought I'd start by asking here, since
BaseX's seems to be the most developed (or only?) CSV module in
XQuery.

Then there's the question of how to approach implementations of such a
spec.  While XQuery is probably capable of parsing and serializing
small enough CSV, CSVs do get large and naive processing with XQuery
would tend to run into memory issues (as I found with xqjson).  This
means implementations would tend to write in a lower-level language.
eXist, for example, uses Jackson for fn:parse-json().  I see Jackson
has a CSV extension too:
https://github.com/FasterXML/jackson-dataformat-csv.  Any thoughts on
the suitability of XQuery for the task?

Joe

[basex-talk] Adding basex-talk to markmail

[Joe Wicentowski]

Hi all,

I wasn't able to find basex-talk in markmail.org - a fantastic site for
searching mailing list archives.  Would the list owners/administrators
consider submitting this list to markmail?  Information about this can be
found at http://markmail.org/docs/feedback.xqy.

Best,
Joe

Re: [basex-talk] Lookups and arrows

[Joe Wicentowski]

Thanks very much for this info, Christian!  I'll pass your
observations on to the eXist team.

Joe

On Sun, Jan 10, 2016 at 3:50 AM, Christian Grün
<christian.gr...@gmail.com> wrote:
> Hi Tim, hi Joe,
>
> The query works if you use parentheses around the arrow operands:
>
>   (json-doc("http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context")
>   => json-doc())("@context")
>
> This is in compliance with the XQuery 3.1 spec [1].
>
> Out of interest, I spent some time and looked up the W3 bug tracker
> history, and I spotted a change in the precedence of the arrow
> operator [2]. Maybe this change is not reflected in eXist?
>
> Hope this helps,
> Christian
>
> [1] http://www.w3.org/TR/xquery-31/#doc-xquery31-ArrowExpr
> [2] https://www.w3.org/Bugs/Public/show_bug.cgi?id=27537
>
>
>
>
>> I tested your code with the closest thing at hand - a local build of
>> branch of Wolfgang Meier's eXist repo with support for the arrow
>> operator (https://github.com/wolfgangmm/exist/tree/feature/arrowop) -
>> and your code worked fine there.  So my hunch is that there's a bug in
>> BaseX or some syntax error I'm not seeing in your code.  For the
>> record, here's the code I'm running:
>>
>> --
>> xquery version "3.1";
>>
>> json-doc("http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context")
>> => json-doc()("@context")
>> --
>>
>> To check the results I actually use `=>
>> serialize(> xmlns="http://www.w3.org/2010/xslt-xquery-serialization;>jsonyes)`
>> on the results since eXist doesn't support adaptive serialization.
>>
>> I hope this is helpful.
>>
>> Joe
>>
>> On Sat, Jan 9, 2016 at 6:20 PM, Tim Thompson <timat...@gmail.com> wrote:
>>> Thanks, Joe! I guess I'd glossed over the "funky" example ;-) Regarding the
>>> arrow operator, I was wondering whether something like this was possible:
>>>
>>> json-doc(" http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context") =>
>>> json-doc()("@context")
>>>
>>> which throws an error: [XPST0003] Unexpected end of query: '("@context")'.
>>>
>>> Tim
>>>
>>> On Sat, Jan 9, 2016 at 3:53 PM, Joe Wicentowski <joe...@gmail.com> wrote:
>>>>
>>>> Hi Tim,
>>>>
>>>> For your first question, I think your example falls into what the spec
>>>> calls "funky looking" keys.  See the 3rd bullet point example under
>>>> http://www.w3.org/TR/xquery-31/#id-lookup:
>>>>
>>>> > funky / >>> > @string"), an appropriate lookup for a map with rather odd conventions 
>>>> > for
>>>> > keys.
>>>>
>>>> In other words, I think you're stuck with the ("@context") approach.
>>>>
>>>> As to your second question, it looks like what you're proposing should
>>>> work - but am I reading you as saying you get an error with your proposed
>>>> approach?  If so, what's the error?
>>>>
>>>> Joe
>>>>
>>>> Sent from my iPad
>>>>
>>>> _
>>>> From: Tim Thompson <timat...@gmail.com>
>>>> Sent: Saturday, January 9, 2016 3:11 PM
>>>> Subject: [basex-talk] Lookups and arrows
>>>> To: BaseX <basex-talk@mailman.uni-konstanz.de>
>>>>
>>>>
>>>>
>>>> Hello,
>>>>
>>>> I'm testing some XQuery 3.1 features against a JSON-LD[1] document and had
>>>> a few questions. In the JSON-LD format, the "@" symbol has special 
>>>> semantics
>>>> in key names, but seems to cause problems with the 3.1 lookup operator.
>>>>
>>>> For example:
>>>>
>>>> json-doc(" http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context")
>>>>
>>>> works as expected, but
>>>>
>>>> json-doc(" http://lae.princeton.edu/catalog/0bp35.jsonld;)?@context
>>>>
>>>> throws an error: [XPST0003] No specifier after lookup operator: '@'.
>>>>
>>>> Also, when using the "=>" operator, should it be possible to perform a
>>>> lookup on the last expression in a chain, if that expression returns a map?
>>>>
>>>> For example:
>>>>
>>>> json-doc(" http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context")
>>>> evaluates to " http://iiif.io/api/presentation/2/context.json;
>>>>
>>>> and
>>>>
>>>> json-doc(" http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context") =>
>>>> json-doc()
>>>>
>>>> returns another map object. So, how would one achieve this:
>>>>
>>>> json-doc(json-doc("
>>>> http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context"))("@context")
>>>>
>>>> using the arrow operator?
>>>>
>>>> Thanks,
>>>> Tim
>>>>
>>>>
>>>> [1] http://www.w3.org/TR/json-ld/
>>>>
>>>>
>>>> --
>>>> Tim A. Thompson
>>>> Metadata Librarian (Spanish/Portuguese Specialty)
>>>> Princeton University Library
>>>>
>>>>
>>>>
>>>

Re: [basex-talk] Lookups and arrows

[Joe Wicentowski]

Hi Tim,
For your first question, I think your example falls into what the spec calls 
"funky looking" keys.  See the 3rd bullet point example under 
http://www.w3.org/TR/xquery-31/#id-lookup:
> funky /  an appropriate lookup for a map with rather odd conventions for keys.
In other words, I think you're stuck with the ("@context") approach.
As to your second question, it looks like what you're proposing should work - 
but am I reading you as saying you get an error with your proposed approach?  
If so, what's the error?
Joe

Sent from my iPad

_
From: Tim Thompson 
Sent: Saturday, January 9, 2016 3:11 PM
Subject: [basex-talk] Lookups and arrows
To: BaseX 


   
Hello,

   I'm testing some XQuery 3.1 features against a JSON-LD[1] document 
and had a few questions. In the JSON-LD format, the "@" symbol has special 
semantics in key names, but seems to cause problems with the 3.1 lookup 
operator.   
   
  For example:  
  
json-doc("  http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context") 
 
  
 works as expected, but 
 
json-doc(" http://lae.princeton.edu/catalog/0bp35.jsonld;)?@context 

 
throws an error: [XPST0003] No specifier after lookup operator: '@'.


   Also, when using the "=>" operator, should it be possible to perform a 
lookup on the last expression in a chain, if that expression returns a map? 
  
   
  For example:  
  
json-doc("  http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context") 
evaluates to "  http://iiif.io/api/presentation/2/context.json;  
  
and  
  
json-doc("  http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context") => 
json-doc()  
  
returns another map object. So, how would one achieve this: 
 
  
json-doc(json-doc("  
http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context"))("@context")  
  
using the arrow operator?  
  
Thanks, 
 Tim  
 
  
[1]http://www.w3.org/TR/json-ld/   
   
   
  --  
Tim A. Thompson  
Metadata Librarian (Spanish/Portuguese Specialty)  
Princeton University Library  
  
   


  

Re: [basex-talk] Lookups and arrows

[Joe Wicentowski]

Hi Tim,

"Funky" made me chuckle too!

I tested your code with the closest thing at hand - a local build of
branch of Wolfgang Meier's eXist repo with support for the arrow
operator (https://github.com/wolfgangmm/exist/tree/feature/arrowop) -
and your code worked fine there.  So my hunch is that there's a bug in
BaseX or some syntax error I'm not seeing in your code.  For the
record, here's the code I'm running:

--
xquery version "3.1";

json-doc("http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context")
=> json-doc()("@context")
--

To check the results I actually use `=>
serialize(http://www.w3.org/2010/xslt-xquery-serialization;>jsonyes)`
on the results since eXist doesn't support adaptive serialization.

I hope this is helpful.

Joe

On Sat, Jan 9, 2016 at 6:20 PM, Tim Thompson <timat...@gmail.com> wrote:
> Thanks, Joe! I guess I'd glossed over the "funky" example ;-) Regarding the
> arrow operator, I was wondering whether something like this was possible:
>
> json-doc(" http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context") =>
> json-doc()("@context")
>
> which throws an error: [XPST0003] Unexpected end of query: '("@context")'.
>
> Tim
>
> On Sat, Jan 9, 2016 at 3:53 PM, Joe Wicentowski <joe...@gmail.com> wrote:
>>
>> Hi Tim,
>>
>> For your first question, I think your example falls into what the spec
>> calls "funky looking" keys.  See the 3rd bullet point example under
>> http://www.w3.org/TR/xquery-31/#id-lookup:
>>
>> > funky / > > @string"), an appropriate lookup for a map with rather odd conventions for
>> > keys.
>>
>> In other words, I think you're stuck with the ("@context") approach.
>>
>> As to your second question, it looks like what you're proposing should
>> work - but am I reading you as saying you get an error with your proposed
>> approach?  If so, what's the error?
>>
>> Joe
>>
>> Sent from my iPad
>>
>> _
>> From: Tim Thompson <timat...@gmail.com>
>> Sent: Saturday, January 9, 2016 3:11 PM
>> Subject: [basex-talk] Lookups and arrows
>> To: BaseX <basex-talk@mailman.uni-konstanz.de>
>>
>>
>>
>> Hello,
>>
>> I'm testing some XQuery 3.1 features against a JSON-LD[1] document and had
>> a few questions. In the JSON-LD format, the "@" symbol has special semantics
>> in key names, but seems to cause problems with the 3.1 lookup operator.
>>
>> For example:
>>
>> json-doc(" http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context")
>>
>> works as expected, but
>>
>> json-doc(" http://lae.princeton.edu/catalog/0bp35.jsonld;)?@context
>>
>> throws an error: [XPST0003] No specifier after lookup operator: '@'.
>>
>> Also, when using the "=>" operator, should it be possible to perform a
>> lookup on the last expression in a chain, if that expression returns a map?
>>
>> For example:
>>
>> json-doc(" http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context")
>> evaluates to " http://iiif.io/api/presentation/2/context.json;
>>
>> and
>>
>> json-doc(" http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context") =>
>> json-doc()
>>
>> returns another map object. So, how would one achieve this:
>>
>> json-doc(json-doc("
>> http://lae.princeton.edu/catalog/0bp35.jsonld;)("@context"))("@context")
>>
>> using the arrow operator?
>>
>> Thanks,
>> Tim
>>
>>
>> [1] http://www.w3.org/TR/json-ld/
>>
>>
>> --
>> Tim A. Thompson
>> Metadata Librarian (Spanish/Portuguese Specialty)
>> Princeton University Library
>>
>>
>>
>

Re: [basex-talk] [bxerr:BXDB0002] Too many open files

[Joe Wicentowski]

Hi Mansi,

The results of ulimit can be misleading.  See this article - which really
helped me when I encountered this issue (though not with BaseX):


https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors.html

Joe

On Mon, Dec 7, 2015 at 1:22 PM, Mansi Sheth  wrote:

> Thanks Christian,
>
> I had already set open files limit on the OS:
>
> ubuntu@ip-10-0-0-83:~$ ulimit -Hn
>
> 
>
> However, I still face exact same problem. Process breaks at the same db
> count
>
> [bxerr:BXDB0002] Resource
> "/veracode/msheth/BaseXDB/bi_output_715/inf.basex (Too many open files)"
> not found.
>
> On Sat, Dec 5, 2015 at 7:53 AM, Christian Grün 
> wrote:
>
>> Hi Mansi,
>>
>> If you are working with Linux, you may need to increase the maximum
>> file limit with "ulimit -n" [1].
>>
>> Hope this helps,
>> Christian
>>
>> [1] http://www.linuxhowtos.org/Tips%20and%20Tricks/ulimit.htm
>>
>>
>>
>> On Fri, Dec 4, 2015 at 7:52 PM, Mansi Sheth 
>> wrote:
>> > Hello,
>> >
>> > I am importing BaseX, with tons of XML files. Currently I have roughly
>> 1600
>> > databases, I am starting basexhttp service, to access it over a web
>> service
>> > endpoint, thru a xquery file. Using BaseX 8.2.3.
>> >
>> > I am receiving below error:
>> >
>> > [bxerr:BXDB0002] Resource
>> "/veracode/msheth/BaseXDB/bi_output_713/inf.basex
>> > (Too many open files)" not found.
>> >
>> > basexhttp, is running with 10240M virtual memory.
>> >
>> > I can share the xquery file, if thats needed.
>> >
>> > Has anyone experienced this before ? Is there a limit on no of databases
>> > supported by BaseX ? Is there some configuration option, which I can
>> use to
>> > close already queried database ?
>> >
>> > Thanks,
>> > - Mansi
>>
>
>
>
> --
> - Mansi
>