[basex-talk] Handling exceptions in the Python Client API
I am now happily using BaseX from Jupyter Notebooks for Greek data - see these links: http://jonathanrobie.biblicalhumanities.org/blog/2017/12/08/jupyter-tutorial/ http://jonathanrobie.biblicalhumanities.org/assets/greeksyntax-tutorial-proiel.html But I need to know how to handle exceptions, e.g. if the notebook user writes a query with an error in it. And I would also like to return some result if the query returns an empty sequence. Here is the current code: https://github.com/biblicalhumanities/greek-new-testament/blob/master/labnotes/greeksyntax/lowfat.py How can I improve what I am doing? Jonathan
[basex-talk] Setting a default collation for a database
I would like to set a default collation for a database, a collation that will be used for all queries unless the user asks for a different collation. My use case: a database in Greek will never be sensibly sorted using an English collation, and I don't want users to need to specify collations over and over. Is this possible? Jonathan
Re: [basex-talk] Handling exceptions in the Python Client API
Thanks! The try/except part is the part I understand. The part I don't understand is how to find the exception thrown by BaseX, I want to show syntax errors in a user friendly way like what you see in the GUI so people don't always have to go back to the GUI to figure out what's wrong with their queries. Jonathan On Tue, Dec 19, 2017 at 11:02 AM, Christian Grün wrote: > > http://jonathanrobie.biblicalhumanities.org/blog/ > 2017/12/08/jupyter-tutorial/ > > http://jonathanrobie.biblicalhumanities.org/assets/ > greeksyntax-tutorial-proiel.html > > Looks good! > > > But I need to know how to handle exceptions, e.g. if the notebook user > > writes a query with an error in it. > > Have you tried to play around with try/except [1]? Well, there are > surely other BaseX users out there that have more experience with > Python. > > [1] https://docs.python.org/3/tutorial/errors.html >
Re: [basex-talk] Handling exceptions in the Python Client API
OK, I did the obvious and examined the exception, which is apparently an
OSError. So this code works well:
def xquery(self, query):
try:
result = self.session.query(query).execute()
except OSError as err:
print("Error:", err)
else:
return result
e.g. if I do this call:
q.xquery("7 + 3 = 10'.
Posting here in case someone else finds it helpful.
Thanks!
Jonathan
On Tue, Dec 19, 2017 at 11:41 AM, Jonathan Robie
wrote:
> Thanks! The try/except part is the part I understand. The part I don't
> understand is how to find the exception thrown by BaseX, I want to show
> syntax errors in a user friendly way like what you see in the GUI so people
> don't always have to go back to the GUI to figure out what's wrong with
> their queries.
>
> Jonathan
>
> On Tue, Dec 19, 2017 at 11:02 AM, Christian Grün <
> christian.gr...@gmail.com> wrote:
>
>> > http://jonathanrobie.biblicalhumanities.org/blog/2017/12/08/
>> jupyter-tutorial/
>> > http://jonathanrobie.biblicalhumanities.org/assets/greeksynt
>> ax-tutorial-proiel.html
>>
>> Looks good!
>>
>> > But I need to know how to handle exceptions, e.g. if the notebook user
>> > writes a query with an error in it.
>>
>> Have you tried to play around with try/except [1]? Well, there are
>> surely other BaseX users out there that have more experience with
>> Python.
>>
>> [1] https://docs.python.org/3/tutorial/errors.html
>>
>
>
[basex-talk] One document per database or multiple?
If I have a set of related documents (a text, a lexicon, frequency counts, a discourse analysis, etc), how should I decide when to put more than one document in a single database at different paths, as opposed to putting one document in each database? When I create a database from the GUI, it seems to prefer one document per database. Should I take that as a hint? Jonathan
Re: [basex-talk] One document per database or multiple?
Thanks to both of you, Fabrice and Christian. I should have asked this question a long time ago - it will help a lot with organizing my work better. Jonathan On Tue, Feb 6, 2018 at 5:41 PM, Fabrice ETANCHAUD < fetanch...@pch.cerfrance.fr> wrote: > Jonathan, in my humble opinion, here are the main reasons you may need > several collections : > > > > - FullText indexing in several languages (because language is > collection wide) : a per language partition of your data > > - Size (usually in number of nodes) limitation > > - Huge updates : a read only backlog collection + a read/write > front collection of fresh data + queries tailored to read both collections. > > > > Best regards, > > And maybe good night ? > > > > Fabrice Etanchaud > > > > *De :* basex-talk-boun...@mailman.uni-konstanz.de [mailto: > basex-talk-boun...@mailman.uni-konstanz.de] *De la part de* Jonathan Robie > *Envoyé :* mardi 6 février 2018 23:31 > *À :* BaseX > *Objet :* [basex-talk] One document per database or multiple? > > > > If I have a set of related documents (a text, a lexicon, frequency counts, > a discourse analysis, etc), how should I decide when to put more than one > document in a single database at different paths, as opposed to putting one > document in each database? > > > > When I create a database from the GUI, it seems to prefer one document per > database. Should I take that as a hint? > > > > Jonathan >
[basex-talk] JSON input files
I would like to add a JSON file to a collection that is otherwise composed
of XML files. I'm hitting two issues:
1. Is there any way to import both JSON and XML files at the same time when
creating a database?
2. I would like to import the JSON as an XQuery/XPath map, without mapping
it to XML. Any way to do that?
Thanks!
Jonathan
P.S. the input JSON looks like this:
{
"ααρων": {
"gk": 2,
"strongs": [
2
],
"lemma": "Ἀαρών",
"transliteration": "Aarōn",
"frequencyCount": 5,
"definition": "Aaron, pr. name, indecl., the brother of Moses (Exod.
4:14), Lk. 1:15; Acts 7:40; Heb. 5:4; 7:11; 9:4*"
},
When I import it, I get this:
<ααρων type="object">
2
<_ type="number">2
Ἀαρών
Aarōn
5
Aaron, pr. name, indecl., the brother of Moses (Exod.
4:14), Lk. 1:15; Acts 7:40; Heb. 5:4; 7:11; 9:4*
<αβαδδων type="object">
3
etc.
[basex-talk] Deploying BaseX + data + client
I have written a Python package for querying Greek syntax from Jupyter Notebook. It relies on BaseX and one or more open data resources found in GitHub repositories. I want to make it available to the public in a way that is easy to install. One possibility is to write a Python script that uses Docker to install BaseX, retrieves the data and installs it in the desired directory structure - perhaps in the container, then loads it into the database. I assume that would work cleanly cross-platform? Are there better approaches that I should consider? Are there security considerations that I should consider? Jonathan
Re: [basex-talk] Scaling and Jsoniq
As an author of both JSONiq and XQuery 3.1, I generally think of XQuery 3.1 as the standard that implemented what JSONiq did. Some authors of JSONiq see this very differently, preferring the JSONiq way of doing things. Is there anything JSONiq can do that XQuery 3.1 can't? Jonathan On Wed, Jan 2, 2019 at 10:47 AM Christian Grün wrote: > Hi Ben, > > > First, are there any support plans for scaling outwards? I know > MarkLogic has splits clusters into multiple enodes and dnodes. But the idea > is that there isn't one central bottleneck for requests. > > BaseX as open source project provides no built-in features for > horizontal scaling. We have built individual solutions for our > commercial customers; feel free to write to i...@basexgmbh.de if you > are interested. > > > Second, I read about Jsoniq and was intrigued by it. It seemed like it > supported JSON without requiring an XML intermediary. Having good JSON > support would really help with frameworks like Angular or React. > > Yes, Jsoniq is cool. We had some thoughts on implementing some years > ago. In the end, we decided not to, because it would have introduced > incompatibilities with features of the official XQuery 3.1 > recommendation. > > Nowadays, you have two choices in BaseX: > > • If you are processing JSON documents in main-memory, you can use the > standard XQuery functions to convert JSON strings to XQuery data > structures [1]. Maps and arrays in XQuery are very efficient, and > often a better choice than XML if you have to repeatedly process the > same data. > • If you want to store your data in a database, and benefit from > indexing, updates, etc., you can convert JSON to an XML structure that > suits you best [2]. You will notice that the time required for > converting JSON to XML and backwards is hardly noticeable. In > practice, it is frequently used to store and query millions of JSON > documents in a database. > > Hope this helps, > Christian > > [1] http://docs.basex.org/wiki/XQuery_3.1#JSON_Functions > [2] http://docs.basex.org/wiki/Json_Module >
Re: [basex-talk] Stack Overflow: Try tail recursion
To make it tail-recursive, make the recursive call the last operation in
the function.
https://en.wikipedia.org/wiki/Tail_call
The else() clause is what keeps it from being tail recursive. Something
like this should work:
*declare variable* *$bookstore* :=
story
50.00
smith
history
150.00
kelly
epic
300.00
jones
;
*declare function* *local:sum*(*$books*, *$sum*)
{
*let* *$sum* := *$sum* + *$books*[1]/*price*
*return* (
{ *$sum* },
*$books*[2] ! *local:sum*(*tail*(*$books*), *$sum*)
)
};
{
*local:sum*(*$bookstore*/*book*, 0)
}
Jonathan
On Mon, Feb 18, 2019 at 10:24 AM Giuseppe G. A. Celano <
cel...@informatik.uni-leipzig.de> wrote:
> I am writing a recursive function which is similar to the one here:
>
>
> https://stackoverflow.com/questions/27702718/to-add-values-in-cumulative-format
>
> Interestingly, local:sum() works if there are not many . However
> with 38000 book element I get the error "Stack Overflow: Try tail
> recursion".
>
> Any idea?
>
> Ciao,
> Giuseppe
>
>
>
[basex-talk] Websites that use BaseX
I have recommended BaseX to some people who are writing an application. The developers, who are more comfortable with SQL, have asked for a list of websites that use BaseX. I imagine they would be happy with other happy customers too, they want to have basic confidence in the tool. So which users should give people confidence that BaseX is the tool for them? They know about my own happy experience with BaseX, they want to know more about other projects. Jonathan
[basex-talk] Command Files - string syntax
I decided to try creating some command files to simplify administration:
https://docs.basex.org/wiki/Commands#String_Syntax
I was surprised that this did not work:
% cat foo.bsx
CREATE DB test
ADD TO embedded.xml embedded
# run query
XQUERY { count(//text()) }
CLOSE
% basex foo.bsx
Stopped at /.../foo.bsx, 1/9:
[XPST0003] Unexpected end of query: 'DB test...'.
This works fine:
% basex CREATE DB test
Database 'test' created in 196.04 ms.
> ADD TO embedded.xml embedded
Resource(s) added in 101.36 ms.
> # run query
> XQUERY { count(//text()) }
1
Query executed in 393.09 ms.
> CLOSE
Database 'test' was closed.
What am I missing?
Jonathan
Re: [basex-talk] Command Files - string syntax
DOH! Thanks for the help, I seem to be a little dsylexic today ;->
Now it works, but SET WRITEBACK true doesn't seem to be working:
DROP DB oshb-morphology
CREATE DB oshb-morphology
ADD ./morphhb/wlc
SET WRITEBACK true
XQUERY ./xquery/merge-proper-nouns-oshb.xq
EXPORT ./out
This query does an update. If I run the same query in the GUI, it works,
but not in this ... um ... bxs file. What do I do to enable updates?
Jonathan
On Mon, Jun 28, 2021 at 3:57 PM Martin Honnen wrote:
> On 28.06.2021 20:54, Jonathan Robie wrote:
> > I decided to try creating some command files to simplify administration:
> >
> > https://docs.basex.org/wiki/Commands#String_Syntax
> > <https://docs.basex.org/wiki/Commands#String_Syntax>
> >
> > I was surprised that this did not work:
> >
> > % cat foo.bsx
> >
> > CREATE DB test
> >
> > ADD TO embedded.xml embedded
> >
> > # run query
> >
> > XQUERY { count(//text()) }
> >
> > CLOSE
> >
> >
> > % basex foo.bsx
> >
> > Stopped at /.../foo.bsx, 1/9:
> >
> > [XPST0003] Unexpected end of query: 'DB test...'.
>
>
> I think the suffix of the file needs to be ".bxs" to be treated as a
> command sequence, otherwise it is treated as XQuery code. So you have
> the wrong file suffix.
>
Re: [basex-talk] Command Files - string syntax
Thanks - this fixed my update problem.
One more question ... how do I run a query that is in a file and write the
results back to a different file? The last line in this script does not
produce output:
DROP DB oshb-morphology
CREATE DB oshb-morphology
ADD ./morphhb/wlc
SET WRITEBACK true
RUN ./xquery/merge-proper-nouns-oshb.xq
EXPORT ./out
XQUERY ./xquery/mappings.xq
It's not a problem with databases that are not open, I open databases
explicitly in mappings.xq, e.g. db:open("oshb-morphology") and the same
query works fine from the GUI with no databases explicitly opened.
Jonathan
On Tue, Jun 29, 2021 at 4:48 AM Christian Grün
wrote:
> Hi Jonathan,
>
> > ...
> > XQUERY ./xquery/merge-proper-nouns-oshb.xq
>
> I don’t have access to the query file you want to evaluate, but I
> assume your command script should work if you replace XQUERY with the
> RUN command [1].
>
> You can skip the WRITEBACK option if you perform updates on database.
> It is only required to update local files [2]:
>
> XQUERY file:write(file:base-dir() || 'example.xml', )
> SET WRITEBACK true
> XQUERY insert node into doc('example.xml')/*
> XQUERY doc('example.xml')
>
> Hope this helps,
> Christian
>
> [1] https://docs.basex.org/wiki/Commands#RUN
> [2] https://docs.basex.org/wiki/Options#WRITEBACK
>
Re: [basex-talk] Command Files - string syntax
Great, thanks! Jonathan On Tue, Jun 29, 2021 at 10:22 AM Christian Grün wrote: > Hi Jonathan, > > > SET WRITEBACK true > > RUN ./xquery/merge-proper-nouns-oshb.xq > > I see you are still using WRITEBACK… Does your query file > 'merge-proper-nouns-oshb.xq' access any local files that you are > updating? > > > XQUERY ./xquery/mappings.xq > > Once again, you may need to replace XQUERY with RUN. If you use > XQUERY, you must supply the query itself after the command. > > > how do I run a query that is in a file and write the results back to a > different file? > > You could use the File Module for that, or fn:put. If you want to > write the result of a query evaluation to a file, you’ll probably need > to do this on command line for all results returned by a script> > > basex -o result.xml script.bxs > > Hope this helps, > Christian >
[basex-talk] Preserving whitespace for XQuery Updates
I am doing some transformations of datasets, then submitting pull requests to upstream sources on GitHub. For instance, today I am inserting some attributes, but I may be restructuring in various ways or enhancing data in various ways. To make upstreams happy, I need to be disciplined about not changing whitespace. What do I have to do? Is it sufficient to preserve whitespace when importing, do an XQuery update, and export, or can that change whitespace beyond what the update operations explicitly say? Thanks! Jonathan
Re: [basex-talk] Preserving whitespace for XQuery Updates
Thanks, Christian, that's very helpful.
The query I am working on now simply adds a @type marker to indicate a
ketiv reading.
declare updating function local:mark-ketiv($variant)
{
if (fn:empty($variant/catchWord))
then ()
else
for $ketiv in get-ketiv($variant, $variant/catchWord)
return
if ($ketiv/@type)
then replace value of node $ketiv/@type with
fn:string-join(($ketiv/@type, "x-ketiv"), " ")
else insert node attribute type { "x-ketiv" } into $ketiv
};
Here's the output of the query. This function is called for each note of
type "variant". I am working with the Open Scriptures Hebrew Bible, which
marks the Qere reading but does not explicitly mark the Ketiv reading to
which it corresponds. I am inserting these attributes because my system
ignores the Ketiv and builds a syntax tree from the Qere.
וַֽ/יְהִי
־
אִ֣ישׁ
מ/בן
־
ימין
מ/בן־ימין
מִ/בִּנְיָמִ֗ין
If I can do this without messing up the whitespace, the Open Scriptures
Hebrew Bible people might accept it in the upstream, which is why the
whitespace is important.
Jonathan
On Fri, Jul 16, 2021 at 4:24 AM Christian Grün
wrote:
> Hi Jonathan,
>
> If you work with whitespace-sensitive documents, it’s recommendable to
> add the following two options at the end of your .basex configuration
> file:
>
> ...
> # Local Options
> CHOP = false
> SERIALIZER = indent=no
>
> The first option will ensure that no whitespaces will be chopped when
> parsing documents. The second one will disable automatic indentation.
>
> Apart from that, you’ll still need to be aware that whitespaces will
> often be dropped if you use node constructors (that’s the default
> behavior of the spec):
>
>
>
> You can avoid that by adding explicit spaces:
>
> { ' ' }
>
> Feel free to share your queries with us.
>
> Best,
> Christian
>
>
> On Fri, Jul 16, 2021 at 12:52 AM Jonathan Robie
> wrote:
> >
> > I am doing some transformations of datasets, then submitting pull
> requests to upstream sources on GitHub. For instance, today I am inserting
> some attributes, but I may be restructuring in various ways or enhancing
> data in various ways.
> >
> > To make upstreams happy, I need to be disciplined about not changing
> whitespace.
> >
> > What do I have to do? Is it sufficient to preserve whitespace when
> importing, do an XQuery update, and export, or can that change whitespace
> beyond what the update operations explicitly say?
> >
> > Thanks!
> >
> > Jonathan
>
Re: [basex-talk] Preserving whitespace for XQuery Updates
Yes, that is shorter and more readable. Thanks!
And if I don't have to worry about setting options, that's nicely
convenient. Again, thanks!
Jonathan
On Fri, Jul 16, 2021 at 8:53 AM Christian Grün
wrote:
> Thanks, Jonathan, for the code snippet.
>
> > replace value of node $ketiv/@type with fn:string-join(($ketiv/@type,
> "x-ketiv"), " ")
>
> This statement should be completely safe, no matter which options you
> have set. If you want to avoid if/then/else, you can also do the
> following (but it’s not much shorter):
>
> delete node $ketiv/@type,
> insert node attribute type { string-join(($ketiv/@type, "x-ketiv"),
> " ") } into $ketiv
>
Re: [basex-talk] Preserving whitespace for XQuery Updates
Hmmm, the original repo puts elements smack dab together on the same line
to avoid whitespace issues, perhaps using CSS. When I do the update, it
puts the updated elements on separate lines:
< בֶּן־מֵאָ֥ה
---
> בֶּן
> ־
> מֵאָ֥ה
Jonathan
On Fri, Jul 16, 2021 at 11:25 AM Jonathan Robie
wrote:
> Yes, that is shorter and more readable. Thanks!
>
> And if I don't have to worry about setting options, that's nicely
> convenient. Again, thanks!
>
> Jonathan
>
> On Fri, Jul 16, 2021 at 8:53 AM Christian Grün
> wrote:
>
>> Thanks, Jonathan, for the code snippet.
>>
>> > replace value of node $ketiv/@type with fn:string-join(($ketiv/@type,
>> "x-ketiv"), " ")
>>
>> This statement should be completely safe, no matter which options you
>> have set. If you want to avoid if/then/else, you can also do the
>> following (but it’s not much shorter):
>>
>> delete node $ketiv/@type,
>> insert node attribute type { string-join(($ketiv/@type, "x-ketiv"),
>> " ") } into $ketiv
>>
>
Re: [basex-talk] Preserving whitespace for XQuery Updates
I tried adding these options to .basex:
# Local Options
CHOP = false
SERIALIZER = indent=no
It still seems to be putting elements on individual lines, as above, and
not just for elements that have been modified. Is there a way to prevent
this?
Jonathan
On Fri, Jul 16, 2021 at 1:33 PM Jonathan Robie
wrote:
> Hmmm, the original repo puts elements smack dab together on the same line
> to avoid whitespace issues, perhaps using CSS. When I do the update, it
> puts the updated elements on separate lines:
>
> < בֶּן type="x-maqqef">־מֵאָ֥ה
> ---
> > בֶּן
> > ־
> > מֵאָ֥ה
>
> Jonathan
>
> On Fri, Jul 16, 2021 at 11:25 AM Jonathan Robie
> wrote:
>
>> Yes, that is shorter and more readable. Thanks!
>>
>> And if I don't have to worry about setting options, that's nicely
>> convenient. Again, thanks!
>>
>> Jonathan
>>
>> On Fri, Jul 16, 2021 at 8:53 AM Christian Grün
>> wrote:
>>
>>> Thanks, Jonathan, for the code snippet.
>>>
>>> > replace value of node $ketiv/@type with fn:string-join(($ketiv/@type,
>>> "x-ketiv"), " ")
>>>
>>> This statement should be completely safe, no matter which options you
>>> have set. If you want to avoid if/then/else, you can also do the
>>> following (but it’s not much shorter):
>>>
>>> delete node $ketiv/@type,
>>> insert node attribute type { string-join(($ketiv/@type, "x-ketiv"),
>>> " ") } into $ketiv
>>>
>>
Re: [basex-talk] Preserving whitespace for XQuery Updates
Sure. As I said, I am using these options in .basex:
CHOP = false
SERIALIZER = indent=no
I am using data from the wlc subdirectory of this repo:
https://github.com/openscriptures/morphhb
Here is my .bxs file:
DROP DB oshb-morphology
CREATE DB oshb-morphology
ADD ./morphhb/wlc
RUN ./xquery/oshb-use-qere.xq
EXPORT ./out/oshb
This is the query (oshb-use-quere.xq):
declare default element namespace "
http://www.bibletechnologies.net/2003/OSIS/namespace";;
declare default function namespace "
http://www.w3.org/2005/xquery-local-functions";;
declare function local:get-ketiv($base, $catchword)
{
let $prev := $base/preceding-sibling::*[1]
let $prevstring := fn:string($prev)
where $prev and fn:ends-with($catchword, $prevstring)
return (
$prev
,
if ($prevstring != $catchword)
then get-ketiv($prev, fn:substring($catchword, 1,
fn:string-length($catchword) - fn:string-length($prevstring)))
else ()
)
};
declare updating function local:mark-ketiv($variant)
{
for $ketiv in get-ketiv($variant, $variant/catchWord)
return (
delete node $ketiv/@type,
insert node attribute type { fn:string-join(($ketiv/@type, "x-ketiv"),"
") } into $ketiv
)
};
let $oshb := db:open("oshb-morphology")
for $verse in $oshb//verse[note[@type='variant']]
for $variant in $verse/note[@type='variant']
return mark-ketiv($variant)
I really appreciate all your help with this!
Jonathan
On Fri, Jul 16, 2021 at 1:55 PM Christian Grün
wrote:
> Hi Jonathan,
>
> Could you provide us with a little step-by-step description that allows us
> to reproduce your use case?
>
> Thanks in advance,
> Christian
>
>
>
> Jonathan Robie schrieb am Fr., 16. Juli 2021,
> 19:42:
>
>> I tried adding these options to .basex:
>>
>> # Local Options
>> CHOP = false
>> SERIALIZER = indent=no
>>
>> It still seems to be putting elements on individual lines, as above, and
>> not just for elements that have been modified. Is there a way to prevent
>> this?
>>
>> Jonathan
>>
>> On Fri, Jul 16, 2021 at 1:33 PM Jonathan Robie
>> wrote:
>>
>>> Hmmm, the original repo puts elements smack dab together on the same
>>> line to avoid whitespace issues, perhaps using CSS. When I do the update,
>>> it puts the updated elements on separate lines:
>>>
>>> < בֶּן>> type="x-maqqef">־מֵאָ֥ה
>>> ---
>>> > בֶּן
>>> > ־
>>> > מֵאָ֥ה
>>>
>>> Jonathan
>>>
>>> On Fri, Jul 16, 2021 at 11:25 AM Jonathan Robie <
>>> jonathan.ro...@gmail.com> wrote:
>>>
>>>> Yes, that is shorter and more readable. Thanks!
>>>>
>>>> And if I don't have to worry about setting options, that's nicely
>>>> convenient. Again, thanks!
>>>>
>>>> Jonathan
>>>>
>>>> On Fri, Jul 16, 2021 at 8:53 AM Christian Grün <
>>>> christian.gr...@gmail.com> wrote:
>>>>
>>>>> Thanks, Jonathan, for the code snippet.
>>>>>
>>>>> > replace value of node $ketiv/@type with
>>>>> fn:string-join(($ketiv/@type, "x-ketiv"), " ")
>>>>>
>>>>> This statement should be completely safe, no matter which options you
>>>>> have set. If you want to avoid if/then/else, you can also do the
>>>>> following (but it’s not much shorter):
>>>>>
>>>>> delete node $ketiv/@type,
>>>>> insert node attribute type { string-join(($ketiv/@type, "x-ketiv"),
>>>>> " ") } into $ketiv
>>>>>
>>>>
Re: [basex-talk] Preserving whitespace for XQuery Updates
Works like a charm - thanks!
I do think merging SERIALIZER and EXPORTER would be helpful. And don't be
afraid of over-teaching me. You already helped me simplify an XQuery, I'm
not too proud to learn ;->
Jonathan
On Fri, Jul 16, 2021 at 3:12 PM Christian Grün
wrote:
> …great, that was perfectly easy to reproduce.
>
> If the EXPORT command is used, an alternative option needs to be used
> … And it’s called EXPORTER [1]. The following script should do the
> job:
>
> SET CHOP false
> SET EXPORTER indent=no,omit-xml-declaration=no
> CREATE DB oshb-morphology morphhb/wlc
> RUN xquery/oshb-use-qere.xq
> EXPORT out/oshb
>
> Some notes (just ignore those that you are already aware of, or that
> may not matter to you):
>
> • If the options are specified in the script, the .basex file can be
> kept untouched
> • As the input files have an XML declaration, I have added the
> omit-xml-declaration parameter
> • DROP can be… skipped, as CREATE will remove an existing database
> • If the initial input is specified in the CREATE command, things will
> be slightly faster
>
> I think we should merge the options SERIALIZER and EXPORTER in a
> future version of BaseX, as they have already been a source of
> confusion in the past.
>
> > I really appreciate all your help with this!
>
> You are welcome!
> Christian
>
> [1] https://docs.basex.org/wiki/Options#EXPORTER
>
>
> On Fri, Jul 16, 2021 at 8:29 PM Jonathan Robie
> wrote:
> >
> > Sure. As I said, I am using these options in .basex:
> >
> > CHOP = false
> > SERIALIZER = indent=no
> >
> > I am using data from the wlc subdirectory of this repo:
> >
> > https://github.com/openscriptures/morphhb
> >
> > Here is my .bxs file:
> >
> > DROP DB oshb-morphology
> > CREATE DB oshb-morphology
> > ADD ./morphhb/wlc
> > RUN ./xquery/oshb-use-qere.xq
> > EXPORT ./out/oshb
> >
> > This is the query (oshb-use-quere.xq):
> >
> > declare default element namespace "
> http://www.bibletechnologies.net/2003/OSIS/namespace";;
> > declare default function namespace "
> http://www.w3.org/2005/xquery-local-functions";;
> >
> > declare function local:get-ketiv($base, $catchword)
> > {
> > let $prev := $base/preceding-sibling::*[1]
> > let $prevstring := fn:string($prev)
> > where $prev and fn:ends-with($catchword, $prevstring)
> > return (
> > $prev
> > ,
> > if ($prevstring != $catchword)
> > then get-ketiv($prev, fn:substring($catchword, 1,
> fn:string-length($catchword) - fn:string-length($prevstring)))
> > else ()
> > )
> > };
> >
> > declare updating function local:mark-ketiv($variant)
> > {
> > for $ketiv in get-ketiv($variant, $variant/catchWord)
> > return (
> > delete node $ketiv/@type,
> > insert node attribute type { fn:string-join(($ketiv/@type,
> "x-ketiv")," ") } into $ketiv
> > )
> > };
> >
> > let $oshb := db:open("oshb-morphology")
> > for $verse in $oshb//verse[note[@type='variant']]
> > for $variant in $verse/note[@type='variant']
> > return mark-ketiv($variant)
> >
> > I really appreciate all your help with this!
> >
> > Jonathan
> >
> > On Fri, Jul 16, 2021 at 1:55 PM Christian Grün <
> christian.gr...@gmail.com> wrote:
> >>
> >> Hi Jonathan,
> >>
> >> Could you provide us with a little step-by-step description that allows
> us to reproduce your use case?
> >>
> >> Thanks in advance,
> >> Christian
> >>
> >>
> >>
> >> Jonathan Robie schrieb am Fr., 16. Juli
> 2021, 19:42:
> >>>
> >>> I tried adding these options to .basex:
> >>>
> >>> # Local Options
> >>> CHOP = false
> >>> SERIALIZER = indent=no
> >>>
> >>> It still seems to be putting elements on individual lines, as above,
> and not just for elements that have been modified. Is there a way to
> prevent this?
> >>>
> >>> Jonathan
> >>>
> >>> On Fri, Jul 16, 2021 at 1:33 PM Jonathan Robie <
> jonathan.ro...@gmail.com> wrote:
> >>>>
> >>>> Hmmm, the original repo puts elements smack dab together on the same
> line to avoid whitespace issues, perhaps using CSS. When I do the update,
> it puts the updated elements on separate lines:
> >>>>
> >>>> < בֶּן type="x-maqqef&
[basex-talk] Dropping namespaces on import, serialization
I am playing with datasets in which namespaces are unhelpful because they make output harder to read. For production use, though, I want the namespaces to be there. Ideally, I would like to tell BaseX to drop namespaces while serializing. Sometimes. Ideally, I would like to be able to set this as an option in a .bxs file. Is that possible? Thanks! Jonathan
[basex-talk] Create a database and populate it with the result of a query
I would like to be able to create a database and populate it with the result of a query done on multiple databases. How do I do that? Jonathan
[basex-talk] java.lang.RuntimeException: Data Access out of bounds
Any ideas what this might mean? What should I do? Jonathan % basex compare-wlc-oshb2.xq Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.1 Java: Oracle Corporation, 1.8.0_271 OS: Mac OS X, x86_64 Stack Trace: java.lang.RuntimeException: Data Access out of bounds: - pre value: 63128148 - table size: 63128148 - first/next pre value: 63128064/63128148 - #total/used pages: 246595/246595 - accessed page: 246594 (246595 > 246594] at org.basex.util.Util.notExpected(Util.java:61) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:456) at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:151) at org.basex.data.Data.kind(Data.java:304) at org.basex.query.value.node.DBNode$4.next(DBNode.java:319) at org.basex.query.value.node.DBNode$4.next(DBNode.java:1) at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) at org.basex.query.expr.path.CachedPath.iter(CachedPath.java:72) at org.basex.query.expr.path.CachedPath.nodeIter(CachedPath.java:47) at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:66) at org.basex.query.func.fn.FnEmpty.item(FnEmpty.java:20) at org.basex.query.expr.ParseExpr.ebv(ParseExpr.java:92) at org.basex.query.expr.ParseExpr.test(ParseExpr.java:110) at org.basex.query.expr.Preds.preds(Preds.java:180) at org.basex.query.expr.path.IterStep.access$3(IterStep.java:1) at org.basex.query.expr.path.IterStep$1.next(IterStep.java:40) at org.basex.query.expr.path.IterStep$1.next(IterStep.java:1) at org.basex.query.QueryContext.next(QueryContext.java:399) at org.basex.query.expr.path.IterPath$1.next(IterPath.java:50) at org.basex.query.expr.path.IterPath$1.next(IterPath.java:1) at org.basex.query.QueryContext.next(QueryContext.java:399) at org.basex.query.expr.gflwor.For$1.next(For.java:75) at org.basex.query.expr.gflwor.OrderBy$1.sort(OrderBy.java:73) at org.basex.query.expr.gflwor.OrderBy$1.next(OrderBy.java:54) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:73) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) at org.basex.query.expr.gflwor.Window$WindowEval.prepareNext(Window.java:396) at org.basex.query.expr.gflwor.Window$TumblingEval.prepareNext(Window.java:457) at org.basex.query.expr.gflwor.Window$2.next(Window.java:176) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:88) at org.basex.query.QueryContext.next(QueryContext.java:399) at org.basex.query.expr.constr.Constr.add(Constr.java:71) at org.basex.query.expr.constr.CElem.item(CElem.java:93) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:48) at org.basex.query.scope.MainModule.iter(MainModule.java:118) at org.basex.query.QueryContext.iter(QueryContext.java:334) at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) at org.basex.core.cmd.AQuery.query(AQuery.java:109) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.api.client.LocalSession.execute(LocalSession.java:132) at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:92) at org.basex.BaseX.(BaseX.java:116) at org.basex.BaseX.main(BaseX.java:42)
Re: [basex-talk] java.lang.RuntimeException: Data Access out of bounds
The easiest way might be to make someone from BaseX a member of a private GitHub repo that has queries, .bxs files, and datasets. The behavior seems to vary depending on the indexes that have been defined. Jonathan On Thu, Jul 29, 2021 at 2:12 PM Christian Grün wrote: > How can we reproduce this? > > On Thu, Jul 29, 2021 at 7:59 PM Jonathan Robie > wrote: > > > > Any ideas what this might mean? What should I do? > > > > Jonathan >
Re: [basex-talk] java.lang.RuntimeException: Data Access out of bounds
Let me try to just program around it ... I think there are better ways to do what this query does. And I don't know that I can reproduce this without a rather large dataset that I am using. Jonathan On Thu, Jul 29, 2021 at 3:53 PM Christian Grün wrote: > Maybe you could try to find out how the behavior can be triggered from > scratch; I’ll then be happy to look at it. > > > On Thu, Jul 29, 2021 at 9:27 PM Jonathan Robie > wrote: > > > > The easiest way might be to make someone from BaseX a member of a > private GitHub repo that has queries, .bxs files, and datasets. > > > > The behavior seems to vary depending on the indexes that have been > defined. > > > > Jonathan > > > > On Thu, Jul 29, 2021 at 2:12 PM Christian Grün < > christian.gr...@gmail.com> wrote: > >> > >> How can we reproduce this? > >> > >> On Thu, Jul 29, 2021 at 7:59 PM Jonathan Robie < > jonathan.ro...@gmail.com> wrote: > >> > > >> > Any ideas what this might mean? What should I do? > >> > > >> > Jonathan >
Re: [basex-talk] Create a database and populate it with the result of a query
Thanks Kristian and Christian,
Very helpful answers. As usual.
Jonathan
On Fri, Jul 23, 2021 at 4:30 AM Christian Grün
wrote:
> …one more note: It’s also possible to pass on initial input to db:create:
>
> let $cities := element cities {
> for $i in 0 to 9
> return db:open('countries' || $i)//city
> }
> return db:create('db', $cities, 'cities.xml')
>
> Best,
> Christian
>
>
>
> On Fri, Jul 23, 2021 at 7:53 AM Kristian Kankainen
> wrote:
> >
> > Hello Jonathan,
> >
> > There are very many answers to your question and you should be more
> clear in what you want (how is the old database organized? how do you want
> to organize the new database? what items do you want to carry over from the
> old to the new db? etc). Depending on this, it might be good to first
> create a copy of the old database and remove unnecessary things from it and
> augment it with populating from the other databases.
> >
> > For creating and updating databases, the core thing to know (in my
> humble opinion) is the concept of the Pending Update List [1]. This means
> that a query does not make any updates on a db while the query is running,
> it instead creates a list of updates and these updates are carried out
> *after* the query ran *successfully*. This gives a lot of opportunities for
> optimisations behind the scenes.
> >
> > Anyway, this implies that the database needs to be existing before you
> run the query for populating it. This means you need to do it in two runs,
> first create the database:
> >
> > db:create($name-of-the-new-db)
> >
> > after that you need to run the query or queries that populates it, e.g
> something like:
> >
> > let $new-item :=
> >
> > {
> > for $old-item in db:open($old-db)/with/some/items
> > (: populate new-item with only 'this' and 'that' attributes from the
> old items :)
> > return ($old-item/@this, $old-item/@that)
> > }
> >
> > return db:add($name-of-the-new-db, $new-item, $path-to-new-item)
> >
> > As your question is very open, I would say it is the organisation of the
> $path-to-new-item that gives most room for answers. Do you want to have
> everything in one file or many files?
> >
> > Best rergards,
> > Kristian Kankainen
> >
> > [1]: https://docs.basex.org/wiki/XQuery_Update#Pending_Update_List
> >
> >
> > On 22. Jul 2021, at 16:33, Jonathan Robie
> wrote:
> >
> > I would like to be able to create a database and populate it with the
> result of a query done on multiple databases.
> >
> > How do I do that?
> >
> > Jonathan
> >
> >
>
[basex-talk] Sequence of operations in a .bxs file?
What order dependencies are there for operations in a bxs file? I am setting up an environment that populates two databases from upstream sources. It runs queries to modify the structure somewhat, they adds an index, then calls optimize. After that it exports to a directory for comparison. Does order matter? What is a good order? CREATE DB wlc-trees ./trees RUN ./xquery/add-wlc-verses-words.xq CREATE INDEX ATTRIBUTE OPTIMIZE EXPORT ./out/wlc SET CHOP false SET EXPORTER indent=no,omit-xml-declaration=no CREATE DB oshb-morphology ./morphhb/wlc RUN ./xquery/oshb-pure-qere-reading.xq RUN ./xquery/merge-proper-nouns-oshb.xq CREATE INDEX ATTRIBUTE OPTIMIZE EXPORT ./out/oshb Thanks! Jonathan
[basex-talk] XProc?
What hooks does BaseX provide for XProc? Where can I find examples of BaseX use in an XProc pipeline? Jonathan
[basex-talk] Pretty-print the -V optimized query? See what indexes are used?
Is there a way to pretty-print the optimized query from the -V option? What is the best way to see which indexes are being used? Jonathan
[basex-talk] Problems installing EXPath
I tried installing EXPath, but without success. > REPO INSTALL http://files.basex.org/modules/expath/functx-1.0.xar Stopped at , 1/66: [repo:parse] functx-1.0.xar: Resource "expath-pkg.xml" not found.. If I download the .xar and unzip it, it looks like this: [image: image.png] Here is the content of expath-pkg.xml: http://expath.org/ns/pkg"; name="http://www.functx.com"; abbrev="functx" version="1.0" spec="1.0"> FunctX library http://www.functx.com/functx.xsl functx.xsl http://www.functx.com functx.xq Any thoughts? Jonathan
Re: [basex-talk] Problems installing EXPath
This is BaseX 9.1. When I do a repo list, it does not list EXPath.
Using https fixed the problem - thanks!
Jonathan
On Fri, Sep 3, 2021 at 1:58 PM Bridger Dyson-Smith
wrote:
> Hi Jonathan -
>
> I'm curious: which distribution of BaseX are you using? The main BaseX.zip
> has the functx library already in ${basex}/repo.
>
> I did though do a quick test (below) and keeping https in the URL made a
> difference. I'm not quite sure why.
> Hope that's helpful.
> Best,
> Bridger
> ```
> > REPO INSTALL https://files.basex.org/modules/expath/functx-1.0.xar
> Package 'https://files.basex.org/modules/expath/functx-1.0.xar' replaced
> in 1158.39 ms.
> > REPO
> Stopped at , 1/5:
> Syntax: REPO [DELETE|INSTALL|LIST]
> Install, delete or list packages.
>
> Installs, deletes or lists packages from the repository.
> - DELETE [name|dir]:
> deletes the package with name [name] or directory [dir]
> - INSTALL [path]:
> installs package with path [path]
> - LIST:
> lists all installed packages
> > REPO LIST
> Name Version TypePath
> ---
> http://www.functx.com 1.0 EXPath http-www.functx.com-1.0
>
> 1 package(s).
> > REPO DELETE http://www.functx.com
> Package 'http://www.functx.com' deleted.
> > REPO LIST
> Name Version Type Path
> -
>
> 0 package(s).
> > REPO INSTALL http://files.basex.org/modules/expath/functx-1.0.xar
> Stopped at , 1/66:
> [repo:parse] functx-1.0.xar: Resource "expath-pkg.xml" not found..
> >
> ```
>
> On Thu, Sep 2, 2021 at 8:59 PM Jonathan Robie
> wrote:
>
>> I tried installing EXPath, but without success.
>>
>>
>> > REPO INSTALL http://files.basex.org/modules/expath/functx-1.0.xar
>> Stopped at , 1/66:
>> [repo:parse] functx-1.0.xar: Resource "expath-pkg.xml" not found..
>>
>>
>> If I download the .xar and unzip it, it looks like this:
>>
>>
>> [image: image.png]
>>
>> Here is the content of expath-pkg.xml:
>>
>>
>> http://expath.org/ns/pkg";
>> name="http://www.functx.com";
>> abbrev="functx"
>> version="1.0"
>> spec="1.0">
>>
>>FunctX library
>>
>>
>> http://www.functx.com/functx.xsl
>> functx.xsl
>>
>>
>> http://www.functx.com
>> functx.xq
>>
>>
>>
>>
>> Any thoughts?
>>
>>
>> Jonathan
>>
>>
[basex-talk] Importing TSV files with a command file
I have a command file that configures a database, throwing a bunch of sources into various paths. I have a set of TSV files. I want to do the equivalent of ADD TO, pointing to a directory of TSV files, so that they are parsed and converted to XML and placed in the appropriate path. How do I do that? Jonathan
Re: [basex-talk] Importing TSV files with a command file
On Tue, Oct 12, 2021 at 10:33 AM Bridger Dyson-Smith
wrote:
> Hi Jonathan,
>
> for $T in file:list($path-to-your-files)
> let $TSV := csv:parse($T, map { 'separator': 'tab', 'header': true(),
> 'format': 'direct' })
> return db:add($your-db, $TSV)
>
> maybe? I'm probably getting something wrong, semantics-wise, w/
> `db:add()`, but maybe it's close to get started?
> Best,
> Bridger
>
> On Tue, Oct 12, 2021 at 9:31 AM Jonathan Robie
> wrote:
>
>> I have a command file that configures a database, throwing a bunch of
>> sources into various paths.
>>
>> I have a set of TSV files. I want to do the equivalent of ADD TO,
>> pointing to a directory of TSV files, so that they are parsed and converted
>> to XML and placed in the appropriate path.
>>
>> How do I do that?
>>
>> Jonathan
>>
>
Re: [basex-talk] Importing TSV files with a command file
Thanks! I was getting hung up trying to do this in a command.
Jonathan
On Tue, Oct 12, 2021 at 11:44 AM Jonathan Robie
wrote:
>
>
> On Tue, Oct 12, 2021 at 10:33 AM Bridger Dyson-Smith <
> bdysonsm...@gmail.com> wrote:
>
>> Hi Jonathan,
>>
>> for $T in file:list($path-to-your-files)
>> let $TSV := csv:parse($T, map { 'separator': 'tab', 'header': true(),
>> 'format': 'direct' })
>> return db:add($your-db, $TSV)
>>
>> maybe? I'm probably getting something wrong, semantics-wise, w/
>> `db:add()`, but maybe it's close to get started?
>> Best,
>> Bridger
>>
>> On Tue, Oct 12, 2021 at 9:31 AM Jonathan Robie
>> wrote:
>>
>>> I have a command file that configures a database, throwing a bunch of
>>> sources into various paths.
>>>
>>> I have a set of TSV files. I want to do the equivalent of ADD TO,
>>> pointing to a directory of TSV files, so that they are parsed and converted
>>> to XML and placed in the appropriate path.
>>>
>>> How do I do that?
>>>
>>> Jonathan
>>>
>>
Re: [basex-talk] Importing TSV files with a command file
Excellent, thanks!
Is there a similar incantation for JSON?
Jonathan
On Tue, Oct 12, 2021 at 6:22 PM Christian Grün
wrote:
> Hi Bridger, hi Jonathan,
>
> When using the graphical user interface, you can learn more about commands
> by
>
> a) enabling the Info View panel
> b) performing a GUI operation, such as adding a directory with
> specific options, and
> c) checking the contents of the Info View.
>
> Here’s a .bxs solution for adding TSV files:
>
> SET CREATEFILTER *.tsv
> SET PARSER tsv
> SET CSVPARSER header=true,separator=tab
> CREATE DB test
> ADD csv-dir
>
> As usual, CREATE and ADD can be combined whenever appropriate:
>
> CREATE DB test tsv-dir
>
> Hope this helps,
> Christian
>
>
> On Tue, Oct 12, 2021 at 7:54 PM Bridger Dyson-Smith
> wrote:
> >
> > Jonathan -
> > apologies for misreading your email! I'm not well-versed in the .bxs
> COMMAND syntax, but maybe this is closer?
> > I couldn't quite get a handle on using the EXECUTE syntax for this, but
> I'm under-caffeinated :)
> >
> > Another piece of uncertainty: can you pass variables from the .bxs to a
> script you are RUNning? Maybe Christian can help us both understand some
> things!
> >
> > Best,
> > Bridger
> >
> > test-tsv.bxs
> > ```
> > CREATE DB test-tsv
> > RUN /home/bridger/test-tsv.xq
> > LIST test-tsv
> > ```
> > test-tsv.xq
> > ```
> > for $T in file:children("/home/bridger/tsvs/")
> > let $TSV := csv:doc($T, map { "format": "direct", "separator": "tab",
> "header": true() })
> > let $TSV-NAME := file:name($T)
> > return db:add("test-tsv", $TSV, "/" || $TSV-NAME)
> > ```
> > note: you could specify a db path here; e.g. db:add("test-tsv", $TSV,
> "/tsv-files/" || $TSV-NAME)
> >
> > I've attached my two sample TSV files if that helps.
> >
> >
> > On Tue, Oct 12, 2021 at 11:45 AM Jonathan Robie <
> jonathan.ro...@gmail.com> wrote:
> >>
> >> Thanks! I was getting hung up trying to do this in a command.
> >>
> >> Jonathan
> >>
> >> On Tue, Oct 12, 2021 at 11:44 AM Jonathan Robie <
> jonathan.ro...@gmail.com> wrote:
> >>>
> >>>
> >>>
> >>> On Tue, Oct 12, 2021 at 10:33 AM Bridger Dyson-Smith <
> bdysonsm...@gmail.com> wrote:
> >>>>
> >>>> Hi Jonathan,
> >>>>
> >>>> for $T in file:list($path-to-your-files)
> >>>> let $TSV := csv:parse($T, map { 'separator': 'tab', 'header': true(),
> 'format': 'direct' })
> >>>> return db:add($your-db, $TSV)
> >>>>
> >>>> maybe? I'm probably getting something wrong, semantics-wise, w/
> `db:add()`, but maybe it's close to get started?
> >>>> Best,
> >>>> Bridger
> >>>>
> >>>> On Tue, Oct 12, 2021 at 9:31 AM Jonathan Robie <
> jonathan.ro...@gmail.com> wrote:
> >>>>>
> >>>>> I have a command file that configures a database, throwing a bunch
> of sources into various paths.
> >>>>>
> >>>>> I have a set of TSV files. I want to do the equivalent of ADD TO,
> pointing to a directory of TSV files, so that they are parsed and converted
> to XML and placed in the appropriate path.
> >>>>>
> >>>>> How do I do that?
> >>>>>
> >>>>> Jonathan
>
Re: [basex-talk] Importing TSV files with a command file
Thanks.
And I guess this is the documentation I was looking for:
https://docs.basex.org/wiki/Parsers
Jonathan
On Wed, Oct 13, 2021 at 9:35 AM Christian Grün
wrote:
> > Is there a similar incantation for JSON?
>
> Absolutely; it’s *.json and JSONPARSER ;)
>
>
> > On Tue, Oct 12, 2021 at 6:22 PM Christian Grün <
> christian.gr...@gmail.com> wrote:
> >>
> >> Hi Bridger, hi Jonathan,
> >>
> >> When using the graphical user interface, you can learn more about
> commands by
> >>
> >> a) enabling the Info View panel
> >> b) performing a GUI operation, such as adding a directory with
> >> specific options, and
> >> c) checking the contents of the Info View.
> >>
> >> Here’s a .bxs solution for adding TSV files:
> >>
> >> SET CREATEFILTER *.tsv
> >> SET PARSER tsv
> >> SET CSVPARSER header=true,separator=tab
> >> CREATE DB test
> >> ADD csv-dir
> >>
> >> As usual, CREATE and ADD can be combined whenever appropriate:
> >>
> >> CREATE DB test tsv-dir
> >>
> >> Hope this helps,
> >> Christian
> >>
> >>
> >> On Tue, Oct 12, 2021 at 7:54 PM Bridger Dyson-Smith
> >> wrote:
> >> >
> >> > Jonathan -
> >> > apologies for misreading your email! I'm not well-versed in the .bxs
> COMMAND syntax, but maybe this is closer?
> >> > I couldn't quite get a handle on using the EXECUTE syntax for this,
> but I'm under-caffeinated :)
> >> >
> >> > Another piece of uncertainty: can you pass variables from the .bxs to
> a script you are RUNning? Maybe Christian can help us both understand some
> things!
> >> >
> >> > Best,
> >> > Bridger
> >> >
> >> > test-tsv.bxs
> >> > ```
> >> > CREATE DB test-tsv
> >> > RUN /home/bridger/test-tsv.xq
> >> > LIST test-tsv
> >> > ```
> >> > test-tsv.xq
> >> > ```
> >> > for $T in file:children("/home/bridger/tsvs/")
> >> > let $TSV := csv:doc($T, map { "format": "direct", "separator": "tab",
> "header": true() })
> >> > let $TSV-NAME := file:name($T)
> >> > return db:add("test-tsv", $TSV, "/" || $TSV-NAME)
> >> > ```
> >> > note: you could specify a db path here; e.g. db:add("test-tsv", $TSV,
> "/tsv-files/" || $TSV-NAME)
> >> >
> >> > I've attached my two sample TSV files if that helps.
> >> >
> >> >
> >> > On Tue, Oct 12, 2021 at 11:45 AM Jonathan Robie <
> jonathan.ro...@gmail.com> wrote:
> >> >>
> >> >> Thanks! I was getting hung up trying to do this in a command.
> >> >>
> >> >> Jonathan
> >> >>
> >> >> On Tue, Oct 12, 2021 at 11:44 AM Jonathan Robie <
> jonathan.ro...@gmail.com> wrote:
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Tue, Oct 12, 2021 at 10:33 AM Bridger Dyson-Smith <
> bdysonsm...@gmail.com> wrote:
> >> >>>>
> >> >>>> Hi Jonathan,
> >> >>>>
> >> >>>> for $T in file:list($path-to-your-files)
> >> >>>> let $TSV := csv:parse($T, map { 'separator': 'tab', 'header':
> true(), 'format': 'direct' })
> >> >>>> return db:add($your-db, $TSV)
> >> >>>>
> >> >>>> maybe? I'm probably getting something wrong, semantics-wise, w/
> `db:add()`, but maybe it's close to get started?
> >> >>>> Best,
> >> >>>> Bridger
> >> >>>>
> >> >>>> On Tue, Oct 12, 2021 at 9:31 AM Jonathan Robie <
> jonathan.ro...@gmail.com> wrote:
> >> >>>>>
> >> >>>>> I have a command file that configures a database, throwing a
> bunch of sources into various paths.
> >> >>>>>
> >> >>>>> I have a set of TSV files. I want to do the equivalent of ADD TO,
> pointing to a directory of TSV files, so that they are parsed and converted
> to XML and placed in the appropriate path.
> >> >>>>>
> >> >>>>> How do I do that?
> >> >>>>>
> >> >>>>> Jonathan
>
[basex-talk] Can't run siserle/blog-example ...
$ git clone https://github.com/siserle/blog-example $ basexhttp /Users/.../blog-example/.basex: Unknown option 'GLOBALLOCK'. /Users/.../blog-example/.basex: Unknown option 'EVENTPORT'. /Users/.../blog-example/.basex: writing new configuration file. [main] INFO org.eclipse.jetty.util.log - Logging initialized @992ms to org.eclipse.jetty.util.log.Slf4jLog [main] WARN org.eclipse.jetty.xml.XmlConfiguration - Config error at 6false2 org.eclipse.jetty.server.nio.SelectChannelConnector That resulted in rewriting the .basex config file: $ git diff .basex *diff --git a/.basex b/.basex* *index f5d8f6b..872cefa 100644* *--- a/.basex* *+++ b/.basex* @@ -1,24 +1,25 @@ -# BaseX Property File. - # General Options +DEBUG = false DBPATH = data +LOGPATH = .logs REPOPATH = repo -DEBUG = false LANG = English LANGKEYS = false -GLOBALLOCK = false +FAIRLOCK = false +CACHETIMEOUT = 3600 # Client/Server Architecture HOST = localhost PORT = 1984 SERVERPORT = 1984 -EVENTPORT = 1985 USER = PASSWORD = SERVERHOST = PROXYHOST = PROXYPORT = 80 NONPROXYHOSTS = +IGNORECERT = false +IGNOREHOSTNAME = false TIMEOUT = 30 KEEPALIVE = 600 PARALLEL = 8 @@ -27,9 +28,12 @@ LOGMSGMAXLEN = 1000 # HTTP Services WEBPATH = . +RESTPATH = RESTXQPATH = restxq +PARSERESTXQ = 3 HTTPLOCAL = false STOPPORT = 8985 +AUTHMETHOD = Basic If I try again, there are no rewrites, but it still does not start: $ basexhttp [main] INFO org.eclipse.jetty.util.log - Logging initialized @303ms to org.eclipse.jetty.util.log.Slf4jLog [main] WARN org.eclipse.jetty.xml.XmlConfiguration - Config error at 6false2 org.eclipse.jetty.server.nio.SelectChannelConnector Any clues? Thanks! Jonathan
[basex-talk] basex test test.xqm => XPDY002c
I get this error with either the test.xqm found in the BaseX GitHub repo or a copy of the file taken from the web page: % basex test test.xqm Stopped at /Users/jonathan/github/basex/basex-core/src/test/resources/, 1/5: [XPDY0002] test: Context is undeclared. What am I doing wrong? Jonathan
Re: [basex-talk] basex test test.xqm => XPDY002c
Excellent, thanks! Jonathan On Fri, Oct 22, 2021 at 4:02 PM Bridger Dyson-Smith wrote: > Jonathan - apologies for the truncated email. > > This also works for me: > bridger@ditrestle|~/src/basex-src on master! > ± basex -t basex-core/src/test/resources/test.xqm > >name="file:///usr/home/bridger/src/basex-src/basex-core/src/test/resources/test.xqm" > time="PT0.892S" tests="8" failures="4" errors="1" skipped="1"> > > > > Empty sequence. > > > ...etc... > > e.g. using the `-t` flag on the basex command. > > I hope that's helpful. > Best, > Bridger > > On Fri, Oct 22, 2021 at 4:00 PM Bridger Dyson-Smith > wrote: > >> Hi Jonathan - >> >> I needed to launch the command line basex; e.g. >> bridger@ditrestle|~/src/basex-src on master! >> ± basex >> BaseX 9.6.2 [Standalone] >> Try 'help' to get more information. >> > TEST basex-core/src/test/resources/test.xqm >> >> > name="file:///usr/home/bridger/src/basex-src/basex-core/src/test/resources/test.xqm" >> time="PT0.914S" tests="8" failures="4" errors="1" skipped="1"> >> >> >> >> Empty sequence. >> >> >> >> >> 9 >> 6 >> Item 1: 6 expected, 9 returned. >> >> >> >> >> err:FORG0001 >> >> >> >> >> >> Failure! >> >> >> >> >> Cannot convert to xs:double: . >> >> >> >> >> >> Result: 8 tests, 4 failures, 1 error, 1 skipped. >> >> >> >> On Fri, Oct 22, 2021 at 3:37 PM Jonathan Robie >> wrote: >> >>> I get this error with either the test.xqm found in the BaseX GitHub repo >>> or a copy of the file taken from the web page: >>> >>> % basex test test.xqm >>> Stopped at /Users/jonathan/github/basex/basex-core/src/test/resources/, >>> 1/5: >>> [XPDY0002] test: Context is undeclared. >>> >>> What am I doing wrong? >>> >>> Jonathan >>> >>
Re: [basex-talk] Can't run siserle/blog-example ...
Is there a sample project that works with the current BaseX? I am basically looking for an example to use to help me get started with my own project, and I want to use whatever the current approach is. Jonathan On Fri, Oct 22, 2021 at 4:10 PM Christian Grün wrote: > Hi Jonathan, > > The blog example seems to be based on BaseX 7 [1] (which was still > based on Jetty 8), so you may need to run it with an older version of > BaseX. > > Best, > Christian > > [1] https://github.com/siserle/blog-example/blob/master/pom.xml#L89 > > > On Fri, Oct 22, 2021 at 7:54 PM Jonathan Robie > wrote: > > > > $ git clone https://github.com/siserle/blog-example > > $ basexhttp > > > > /Users/.../blog-example/.basex: Unknown option 'GLOBALLOCK'. > > /Users/.../blog-example/.basex: Unknown option 'EVENTPORT'. > > /Users/.../blog-example/.basex: writing new configuration file. > > [main] INFO org.eclipse.jetty.util.log - Logging initialized @992ms to > org.eclipse.jetty.util.log.Slf4jLog > > [main] WARN org.eclipse.jetty.xml.XmlConfiguration - Config error at > > >class="org.eclipse.jetty.server.nio.SelectChannelConnector"> name="host"> default="0.0.0.0"/> name="org.basex.serverport" default="8984"/> name="maxIdleTime">6false name="Acceptors">2 > > > > org.eclipse.jetty.server.nio.SelectChannelConnector > > > > That resulted in rewriting the .basex config file: > > > > $ git diff .basex > > > > diff --git a/.basex b/.basex > > > > index f5d8f6b..872cefa 100644 > > > > --- a/.basex > > > > +++ b/.basex > > > > @@ -1,24 +1,25 @@ > > > > -# BaseX Property File. > > > > - > > > > # General Options > > > > +DEBUG = false > > > > DBPATH = data > > > > +LOGPATH = .logs > > > > REPOPATH = repo > > > > -DEBUG = false > > > > LANG = English > > > > LANGKEYS = false > > > > -GLOBALLOCK = false > > > > +FAIRLOCK = false > > > > +CACHETIMEOUT = 3600 > > > > > > > > # Client/Server Architecture > > > > HOST = localhost > > > > PORT = 1984 > > > > SERVERPORT = 1984 > > > > -EVENTPORT = 1985 > > > > USER = > > > > PASSWORD = > > > > SERVERHOST = > > > > PROXYHOST = > > > > PROXYPORT = 80 > > > > NONPROXYHOSTS = > > > > +IGNORECERT = false > > > > +IGNOREHOSTNAME = false > > > > TIMEOUT = 30 > > > > KEEPALIVE = 600 > > > > PARALLEL = 8 > > > > @@ -27,9 +28,12 @@ LOGMSGMAXLEN = 1000 > > > > > > > > # HTTP Services > > > > WEBPATH = . > > > > +RESTPATH = > > > > RESTXQPATH = restxq > > > > +PARSERESTXQ = 3 > > > > HTTPLOCAL = false > > > > STOPPORT = 8985 > > > > +AUTHMETHOD = Basic > > > > > > If I try again, there are no rewrites, but it still does not start: > > > > $ basexhttp > > > > [main] INFO org.eclipse.jetty.util.log - Logging initialized @303ms to > org.eclipse.jetty.util.log.Slf4jLog > > > > [main] WARN org.eclipse.jetty.xml.XmlConfiguration - Config error at > > > > >class="org.eclipse.jetty.server.nio.SelectChannelConnector"> name="host"> default="0.0.0.0"/> name="org.basex.serverport" default="8984"/> name="maxIdleTime">6false name="Acceptors">2 > > > > > > > > org.eclipse.jetty.server.nio.SelectChannelConnector > > > > > > Any clues? > > > > Thanks! > > > > Jonathan >
Re: [basex-talk] Can't run siserle/blog-example ...
Hi Christian, Thanks - I'll have a look. The database is a services layer that can serve many clients. The POC client is written in Vue, we will also have at least one C# client, there may be others, so I want to avoid any dependencies on specific web frameworks. The services deliver XML or JSON, depending on the nature of the request - document content is in XML, directory level stuff and various other things are in JSON. I definitely plan to use RESTXQ and your test framework. I very much appreciate your helpfulness. Jonathan On Mon, Oct 25, 2021 at 3:21 AM Christian Grün wrote: > Hi Jonathan, > > You could have a look at the DBA code (but it might require some > preknowledge on how RESTXQ works) [1]. We decided to keep the DBA code > pretty lean (it uses basic HTML and a few lines of JavaScript code). > If you are planning to create a full web application, you may want to > choose a client framework that suits you best (XForms; or Angular, > Vue, React, …). Do you have any preferences? > > Best, > Christian > > [1] https://docs.basex.org/wiki/DBA > > > > Is there a sample project that works with the current BaseX? I am > basically looking for an example to use to help me get started with my own > project, and I want to use whatever the current approach is. > > > > Jonathan > > > > On Fri, Oct 22, 2021 at 4:10 PM Christian Grün < > christian.gr...@gmail.com> wrote: > >> > >> Hi Jonathan, > >> > >> The blog example seems to be based on BaseX 7 [1] (which was still > >> based on Jetty 8), so you may need to run it with an older version of > >> BaseX. > >> > >> Best, > >> Christian > >> > >> [1] https://github.com/siserle/blog-example/blob/master/pom.xml#L89 > >> > >> > >> On Fri, Oct 22, 2021 at 7:54 PM Jonathan Robie < > jonathan.ro...@gmail.com> wrote: > >> > > >> > $ git clone https://github.com/siserle/blog-example > >> > $ basexhttp > >> > > >> > /Users/.../blog-example/.basex: Unknown option 'GLOBALLOCK'. > >> > /Users/.../blog-example/.basex: Unknown option 'EVENTPORT'. > >> > /Users/.../blog-example/.basex: writing new configuration file. > >> > [main] INFO org.eclipse.jetty.util.log - Logging initialized @992ms > to org.eclipse.jetty.util.log.Slf4jLog > >> > [main] WARN org.eclipse.jetty.xml.XmlConfiguration - Config error at > > >> >class="org.eclipse.jetty.server.nio.SelectChannelConnector"> name="host"> default="0.0.0.0"/> name="org.basex.serverport" default="8984"/> name="maxIdleTime">6false name="Acceptors">2 > >> > > >> > org.eclipse.jetty.server.nio.SelectChannelConnector > >> > > >> > That resulted in rewriting the .basex config file: > >> > > >> > $ git diff .basex > >> > > >> > diff --git a/.basex b/.basex > >> > > >> > index f5d8f6b..872cefa 100644 > >> > > >> > --- a/.basex > >> > > >> > +++ b/.basex > >> > > >> > @@ -1,24 +1,25 @@ > >> > > >> > -# BaseX Property File. > >> > > >> > - > >> > > >> > # General Options > >> > > >> > +DEBUG = false > >> > > >> > DBPATH = data > >> > > >> > +LOGPATH = .logs > >> > > >> > REPOPATH = repo > >> > > >> > -DEBUG = false > >> > > >> > LANG = English > >> > > >> > LANGKEYS = false > >> > > >> > -GLOBALLOCK = false > >> > > >> > +FAIRLOCK = false > >> > > >> > +CACHETIMEOUT = 3600 > >> > > >> > > >> > > >> > # Client/Server Architecture > >> > > >> > HOST = localhost > >> > > >> > PORT = 1984 > >> > > >> > SERVERPORT = 1984 > >> > > >> > -EVENTPORT = 1985 > >> > > >> > USER = > >> > > >> > PASSWORD = > >> > > >> > SERVERHOST = > >> > > >> > PROXYHOST = > >> > > >> > PROXYPORT = 80 > >> > > >> > NONPROXYHOSTS = > >> > > >> > +IGNORECERT = false > >> > > >> > +IGNOREHOSTNAME = false > >> > > >> > TIMEOUT = 30 > >> > > >> > KEEPALIVE = 600 > >> > > >> > PARALLEL = 8 > >> > > >> > @@ -27,9 +28,12 @@ LOGMSGMAXLEN = 1000 > >> > > >> > > >> > > >> > # HTTP Services > >> > > >> > WEBPATH = . > >> > > >> > +RESTPATH = > >> > > >> > RESTXQPATH = restxq > >> > > >> > +PARSERESTXQ = 3 > >> > > >> > HTTPLOCAL = false > >> > > >> > STOPPORT = 8985 > >> > > >> > +AUTHMETHOD = Basic > >> > > >> > > >> > If I try again, there are no rewrites, but it still does not start: > >> > > >> > $ basexhttp > >> > > >> > [main] INFO org.eclipse.jetty.util.log - Logging initialized @303ms > to org.eclipse.jetty.util.log.Slf4jLog > >> > > >> > [main] WARN org.eclipse.jetty.xml.XmlConfiguration - Config error at > > >> > > >> >class="org.eclipse.jetty.server.nio.SelectChannelConnector"> name="host"> default="0.0.0.0"/> name="org.basex.serverport" default="8984"/> name="maxIdleTime">6false name="Acceptors">2 > >> > > >> > > >> > > >> > org.eclipse.jetty.server.nio.SelectChannelConnector > >> > > >> > > >> > Any clues? > >> > > >> > Thanks! > >> > > >> > Jonathan >
Re: [basex-talk] Content-Encoding: gzip on incoming RESTXQ POST
Is it possible to encrypt the zip file for HTTP requests? I have some documents that are not freely licensed, and I need some way to protect the IP of the copyright holder if I want to enable their texts. Jonathan On Fri, Nov 5, 2021 at 8:32 AM Christian Grün wrote: > Hi James, > > Sounds like a good hint! I’ll be happy to have a look at your pull request. > > Thanks in advance, > Christian > > > > On Fri, Nov 5, 2021 at 1:20 PM James Ball > wrote: > > > > Hi Christian, > > > > My Google-fu was definitely letting me down. I completely missed that > POST and PUT had to be explicitly included. > > > > Thank you for the snapshot - which I’ve downloaded and tried. Sadly - > same error. > > > > I *think* I’ve found yet another default configuration for GZIPHandler > getting in the way [1]: > > > > "Both Request uncompress and Response compress are gated by a > configurable DispatcherType check on the GzipHandler … (Default: > DispatcherType.REQUEST).” > > > > I understand this to mean that requests will NOT be decompressed by > default. Would you read that the same way? > > > > Do you think adding a line like the below would be the answer? My Java > is getting out of date but if you think I’m on the right track I will try > it and prepare a pull request for you. > > > > > > Many thanks, James > > > > > > BaseXHTTP.java [2] > > > > import javax.servlet.* > > > > gzip.setDispatcherTypes(DispatcherType.REQUEST, DispatcherType.RESPONSE) > > > > > > > > [1] > https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/server/handler/gzip/GzipHandler.html#setDispatcherTypes(javax.servlet.DispatcherType.. > .) > > [2] > https://github.com/BaseXdb/basex/blob/5f4f4316c77f781275a3e8afd58c151792d20f40/basex-api/src/main/java/org/basex/BaseXHTTP.java > > > > > On 4 Nov 2021, at 08:22, Christian Grün > wrote: > > > > > > Hi James, > > > > > > This is due to the default gzip settings of Jetty, which exclude POST > > > requests. There had been a similar question in the past, and I have > > > now decided to add the POST and PUT method to the Jetty defaults > > > [1,2]. > > > > > > A new snapshot is available! > > > > > > Cheers, > > > Christian > > > > > > [1] > https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg12281.html > > > [2] https://github.com/BaseXdb/basex/issues/1748 > > > > > > > > > > > > On Tue, Nov 2, 2021 at 6:29 PM James Ball > wrote: > > >> > > >> Hello, > > >> > > >> I have a RESTXQ function that receives an XML file via HTTP POST. > This works perfectly. > > >> > > >> The client would now like to compress the data with gzip and set the > Content-Encoding: gzip header. > > >> > > >> I set GZIP = true in my .basex file based on the documentation [1] > > >> > > >> But these gzipped post requests fail - my function is called but > content seems to arrive at my function still compressed. > > >> > > >> (Responses from the server become compressed dynamically as expected > - and that works great). > > >> > > >> Is there something else I need to do to make it work for requests? > > >> > > >> Many thanks, James > > >> > > >> > > >> [1] https://docs.basex.org/wiki/Options#GZIP > > >
Re: [basex-talk] Content-Encoding: gzip on incoming RESTXQ POST
Yes, I do. And yeah, that's really the right solution ... Thanks! Jonathan On Mon, Nov 8, 2021 at 11:52 AM Christian Grün wrote: > Hi Jonathan, do you have a chance to use HTTPS? > > On Mon, Nov 8, 2021 at 3:40 PM Jonathan Robie > wrote: > > > > Is it possible to encrypt the zip file for HTTP requests? I have some > documents that are not freely licensed, and I need some way to protect the > IP of the copyright holder if I want to enable their texts. > > > > Jonathan > > > > On Fri, Nov 5, 2021 at 8:32 AM Christian Grün > wrote: > >> > >> Hi James, > >> > >> Sounds like a good hint! I’ll be happy to have a look at your pull > request. > >> > >> Thanks in advance, > >> Christian > >> > >> > >> > >> On Fri, Nov 5, 2021 at 1:20 PM James Ball > wrote: > >> > > >> > Hi Christian, > >> > > >> > My Google-fu was definitely letting me down. I completely missed that > POST and PUT had to be explicitly included. > >> > > >> > Thank you for the snapshot - which I’ve downloaded and tried. Sadly - > same error. > >> > > >> > I *think* I’ve found yet another default configuration for > GZIPHandler getting in the way [1]: > >> > > >> > "Both Request uncompress and Response compress are gated by a > configurable DispatcherType check on the GzipHandler … (Default: > DispatcherType.REQUEST).” > >> > > >> > I understand this to mean that requests will NOT be decompressed by > default. Would you read that the same way? > >> > > >> > Do you think adding a line like the below would be the answer? My > Java is getting out of date but if you think I’m on the right track I will > try it and prepare a pull request for you. > >> > > >> > > >> > Many thanks, James > >> > > >> > > >> > BaseXHTTP.java [2] > >> > > >> > import javax.servlet.* > >> > > >> > gzip.setDispatcherTypes(DispatcherType.REQUEST, > DispatcherType.RESPONSE) > >> > > >> > > >> > > >> > [1] > https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/server/handler/gzip/GzipHandler.html#setDispatcherTypes(javax.servlet.DispatcherType.. > .) > >> > [2] > https://github.com/BaseXdb/basex/blob/5f4f4316c77f781275a3e8afd58c151792d20f40/basex-api/src/main/java/org/basex/BaseXHTTP.java > >> > > >> > > On 4 Nov 2021, at 08:22, Christian Grün > wrote: > >> > > > >> > > Hi James, > >> > > > >> > > This is due to the default gzip settings of Jetty, which exclude > POST > >> > > requests. There had been a similar question in the past, and I have > >> > > now decided to add the POST and PUT method to the Jetty defaults > >> > > [1,2]. > >> > > > >> > > A new snapshot is available! > >> > > > >> > > Cheers, > >> > > Christian > >> > > > >> > > [1] > https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg12281.html > >> > > [2] https://github.com/BaseXdb/basex/issues/1748 > >> > > > >> > > > >> > > > >> > > On Tue, Nov 2, 2021 at 6:29 PM James Ball < > basex-t...@jamesball.co.uk> wrote: > >> > >> > >> > >> Hello, > >> > >> > >> > >> I have a RESTXQ function that receives an XML file via HTTP POST. > This works perfectly. > >> > >> > >> > >> The client would now like to compress the data with gzip and set > the Content-Encoding: gzip header. > >> > >> > >> > >> I set GZIP = true in my .basex file based on the documentation [1] > >> > >> > >> > >> But these gzipped post requests fail - my function is called but > content seems to arrive at my function still compressed. > >> > >> > >> > >> (Responses from the server become compressed dynamically as > expected - and that works great). > >> > >> > >> > >> Is there something else I need to do to make it work for requests? > >> > >> > >> > >> Many thanks, James > >> > >> > >> > >> > >> > >> [1] https://docs.basex.org/wiki/Options#GZIP > >> > >
[basex-talk] Running dba as a web app .... what am I doing wrong?
I am confused about what is required for running a web app. Here's what I have tried so far: 1. Running basexhttp from ./Downloads/basex If I download the .zip file from https://basex.org/download/ and run basexhttp from the /Downloads/basex subdirectory, I can see the "BaseX HTTP Services" page at localhost:8984, but when I look at localhost:8984/rest, I see no databases: 2. Running basexhttp from a random directory If I run basexhttp from a random or empty directory, localhost:8984 returns this message: Service not found. But localhost:8984/rest lists all of my databases. 3. Running basexhttp using a copy of ./Downloads/basex/webapp I created a ./basex directory in a temporary directory and did this: cp -r /Users/xxx/Downloads/basex/webapp . I then tried running basexhttp from the new ./basex directory I created in the temporary directory. The results were the same as running from a random directory. Running from within the webapp directory does not work in either case. What am I doing wrong? Thanks! Jonathan
[basex-talk] items of type map(*) cannot be serialized
I want to serialize JSON. Consider the following function:
declare function local:index()
{
array{ $levinsohn/levinsohn/feature/header ! map {./name : ./description !
string() } }
};
When I run that, I get output that looks like this:
[map {
"Ambiguous": "Marked but ambiguous constituent order."
}, map {
"annotations": "Inline annotations."
}
!!! SNIP !!!]
That won't work as output in a REST API, so I want to serialize that
without the map keyword. I tried to use fn:serialize():
declare function local:index()
{
array{ $levinsohn/levinsohn/feature/header ! map {./name : ./description !
string() } } ! serialize(.)
};
That gave me this error:
items of type map(*) cannot be serialized
So ... what's the right way to do this?
Jonathan
Re: [basex-talk] items of type map(*) cannot be serialized
Perfect - thanks!
Jonathan
On Wed, Nov 10, 2021 at 1:41 PM Martin Honnen wrote:
> On 10.11.2021 19:34, Jonathan Robie wrote:
> > I want to serialize JSON. Consider the following function:
> >
> > declare function local:index()
> > {
> > array{ $levinsohn/levinsohn/feature/header ! map {./name :
> > ./description ! string() } }
> > };
> >
> > When I run that, I get output that looks like this:
> >
> > [map {
> >"Ambiguous": "Marked but ambiguous constituent order."
> > }, map {
> >"annotations": "Inline annotations."
> > }
> > !!! SNIP !!!]
> >
> > That won't work as output in a REST API, so I want to serialize that
> > without the map keyword. I tried to use fn:serialize():
> >
> > declare function local:index()
> > {
> > array{ $levinsohn/levinsohn/feature/header ! map {./name :
> > ./description ! string() } } ! serialize(.)
> > };
> >
> > That gave me this error:
> >
> > items of type map(*) cannot be serialized
> >
> >
> > So ... what's the right way to do this?
>
> I would use
>
>serialize(local:index(), map { 'method' : 'json' })
>
> to serialize the whole array as JSON
>
>
[basex-talk] Wrong MIME type ...
Using RESTXQ in an application for the first time. I expected this
function to have the MIME type application/json:
declare
%rest:path("/api/abbott-smith/index")
%rest:GET
%rest:produces("application/json")
function as:index()
{
array { $as:abbott-smith/*:entry ! @lemma ! string() }
! serialize(., map {'method' : 'json'})
};
But both POSTMAN and WADL give me application/xml as the mime type:
What am I doing wrong?
Jonathan
Re: [basex-talk] Wrong MIME type ...
I want the query on the server side to specify the Content-Type header for
the response. I didn't see how to do that from the link you provided.
This seems to work - is there a better way?
declare function as:json-response-headers()
{
};
declare
%rest:path("/api/abbott-smith/index")
%rest:GET
%rest:produces("application/json")
function as:index()
{
as:json-response-headers(),
array { $as:abbott-smith/*:entry ! @lemma ! string() }
! serialize(., map {'method' : 'json'})
};
Jonathan
[1] https://docs.basex.org/wiki/RESTXQ#Custom_Response
On Wed, Nov 10, 2021 at 11:29 PM Kristian Kankainen
wrote:
> Hi Jonathan,
>
> Have you tried setting the media-type or method parameters to
> application/json instead of only the produces parameter, as explained here
> [1].
>
>
> [1] https://docs.basex.org/wiki/REST#Content_Type
>
> Best regards,
> Kristian Kankainen
>
>
>
> On 11. Nov 2021, at 02:32, Jonathan Robie
> wrote:
>
> Using RESTXQ in an application for the first time. I expected this
> function to have the MIME type application/json:
>
> declare
> %rest:path("/api/abbott-smith/index")
> %rest:GET
> %rest:produces("application/json")
> function as:index()
> {
> array { $as:abbott-smith/*:entry ! @lemma ! string() }
> ! serialize(., map {'method' : 'json'})
> };
>
> But both POSTMAN and WADL give me application/xml as the mime type:
>
>
>
>
>
>
>
>
>
>
>
> What am I doing wrong?
>
> Jonathan
>
>
>
Re: [basex-talk] Wrong MIME type ...
Works like a charm! Thanks, Kristian and Christian!
Jonathan
On Thu, Nov 11, 2021 at 1:19 PM Christian Grün
wrote:
> Hi Jonathan,
>
> As Kristian indicated, it suffices to specify the media-type or method
> parameter. The content type will be generated by BaseX (see the
> attached example).
>
> If you want to return a non-standard or custom Content-Type header,
> you’ll indeed need to construct your own response, either manually or
> via the web:response-header helper function.
>
> Hope this helps,
> Christian
>
> declare
> %rest:path('json')
> %output:method('json')
> function page:test() {
> map { 1: 2 }
> };
>
> > curl -i "http://localhost:8984/json";
> HTTP/1.1 200 OK
> Date: Thu, 11 Nov 2021 18:14:42 GMT
> Content-Type: application/json;charset=utf-8
> Server-Timing: parse;dur=0.05 ms,compile;dur=0.12 ms,evaluate;dur=0.0
> ms,serialize;dur=0.0 ms
> Content-Length: 14
> Server: Jetty(9.4.44.v20210927)
>
> {
> "1": 2
> }
>
>
>
> On Thu, Nov 11, 2021 at 2:11 PM Jonathan Robie
> wrote:
> >
> > I want the query on the server side to specify the Content-Type header
> for the response. I didn't see how to do that from the link you provided.
> >
> > This seems to work - is there a better way?
> >
> >
> > declare function as:json-response-headers()
> >
> > {
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > };
> >
> >
> > declare
> >
> > %rest:path("/api/abbott-smith/index")
> >
> > %rest:GET
> >
> > %rest:produces("application/json")
> >
> > function as:index()
> >
> > {
> >
> > as:json-response-headers(),
> >
> >
> > array { $as:abbott-smith/*:entry ! @lemma ! string() }
> >
> > ! serialize(., map {'method' : 'json'})
> >
> > };
> >
> >
> > Jonathan
> >
> > [1] https://docs.basex.org/wiki/RESTXQ#Custom_Response
> >
> > On Wed, Nov 10, 2021 at 11:29 PM Kristian Kankainen <
> krist...@keeleleek.ee> wrote:
> >>
> >> Hi Jonathan,
> >>
> >> Have you tried setting the media-type or method parameters to
> application/json instead of only the produces parameter, as explained here
> [1].
> >>
> >>
> >> [1] https://docs.basex.org/wiki/REST#Content_Type
> >>
> >> Best regards,
> >> Kristian Kankainen
> >>
> >>
> >>
> >> On 11. Nov 2021, at 02:32, Jonathan Robie
> wrote:
> >>
> >> Using RESTXQ in an application for the first time. I expected this
> function to have the MIME type application/json:
> >>
> >> declare
> >> %rest:path("/api/abbott-smith/index")
> >> %rest:GET
> >> %rest:produces("application/json")
> >> function as:index()
> >> {
> >> array { $as:abbott-smith/*:entry ! @lemma ! string() }
> >> ! serialize(., map {'method' : 'json'})
> >> };
> >>
> >> But both POSTMAN and WADL give me application/xml as the mime type:
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> What am I doing wrong?
> >>
> >> Jonathan
> >>
> >>
>
[basex-talk] Returning multi-part MIME in RESTXQ? Query parameters?
I'm having fun with RESTXQ, this is an easy and efficient way to create
REST interfaces once you learn how.
Two questions:
1. How do I return a forest of elements? Multi-part MIME types? If so,
how do I specify that? What do I need to add to this particular
declaration?
(:~
Look up all entries in the Abbott-Smith lexicon that contain an English
word in the gloss.
:)
declare
%rest:path("/api/abbott-smith/gloss/{$g}")
%rest:GET
%output:method('xml')
function as:gloss($g)
{
$as:abbott-smith/*:entry[.//*:gloss[contains-token(lower-case(.),
lower-case($g))] ]
};
2. Can I use query strings to specify parameters in RESTXQ? What if I
want URLs like this:
/api/dictionary/?lemma=cheese
Thanks!
Jonathan
Re: [basex-talk] RumbleDB
I don't think RumbleDB does XML, does it? Ghislain is incredible, and I expect it would be a great solution for people working in JSON. This Jupyter Notebook gives a bit of the feel: https://colab.research.google.com/github/RumbleDB/rumble/blob/master/RumbleSandbox.ipynb#scrollTo=tBA43mYG_4Ng Jonathan On Wed, Dec 1, 2021 at 10:21 PM Ben Pracht wrote: > I don't mean any ill will with this question but value the XQuery opinions > of this group. > > Has any BaseX users tried RumbleDB? I've been intrigued with the > scalability OOTB with the Spark. However, I used Spark in a previous job > and found pure Spark to be a pain to set up, complex and worst (to me > anyway), written in Scala. Also, at a glance it looks like the XQuery > library is a bit spartan by comparison to BaseX. > > Regards, > Ben > > >
Re: [basex-talk] Log4j vulnerability CVE-2021-44228
On Mon, Dec 13, 2021 at 10:18 AM Christian Grün wrote: > I was waiting for that question ;) > Waiting smugly, I gather ;-> Jonathan
[basex-talk] Controlling order of attributes with updates
Yes, I know that the order of attributes is insignificant. Except to some human beings who complain to me about the order of elements. Is there any way for me to control the order of attributes in BaseX when I insert new attributes? Jonathan
Re: [basex-talk] Controlling order of attributes with updates
> How do you currently proceed?
I have two queries that add attributes to a morphology I am processing.
Ideally, I would like each attribute to be added at the end of the
attribute list, after any existing attributes.
The first adds an @after attribute if there is a space or punctuation after
the token:
declare function local:segs($nodes)
{
if (head($nodes)[self::*:seg])
then concat(string(head($nodes)), local:segs(tail($nodes)))
else ()
};
declare variable $oshb := db:open("oshb-morphology");
for $w in $oshb//*:w
let $next := $w/following-sibling::*[1]
where $next
let $after :=
if ($next[self::*:w]) then " "
else if ($next[self::*:seg]) then local:segs($w/following-sibling::*)
else ""
return
insert node attribute after {$after} into $w
The second adds Book-Chapter-Verse-Word-Part numbering:
let $oshb := db:open("oshb-morphology")
for $verse in $oshb//*:verse
order by local:bcv($verse/@osisID)
for $w at $i in $verse/*[self::*:w or self::*:c]
for $p at $j in local:entities($w)
let $id := local:bcvwp($verse/@osisID, $i, $j)
return (
delete node $p/@n,
insert node attribute n {$id} into $p
)
On Thu, Feb 10, 2022 at 4:34 AM Christian Grün
wrote:
> Hi Jonathan,
>
> > Is there any way for me to control the order of attributes in BaseX when
> I insert new attributes?
>
> If you evaluate the following XQuery expression in BaseX …
>
>
>
> … you can be sure that the attributes will be placed in the original
> order. Please note that we cannot give a guarantee that this will stay
> that way forever, even though it’s unlikely to change.
>
> If you have an existing element, you can reinsert all attributes in
> the order you want:
>
> let $a :=
> return element { node-name($a) } {
> $a/@*, attribute d { 'd' }, $a/node()
> }
>
> How do you currently proceed?
>
> Best,
> Christian
>
Re: [basex-talk] combining queries from mutiple sources
Also, you can have the best of both worlds, saving results in a pipeline if
it's helpful for debugging, just running the updates in place once you know
it all works well. This is particularly helpful if your data is complex
and your datasets are largish.
I am currently merging various sources that were not originally meant to
work together, creating a common reference system for fairly complex data.
I find it helpful to write a bunch of simple updates that can be executed
sequentially, running them with a .bxs file. If I want to see the
output after any of these updates, I can do so with an OUTPUT statement.
Most of these stages are only useful for debugging purposes, so once a
pipeline is working well, I get rid of the OUTPUT statement.
If your data is large, it can also be helpful to have two different
statements for importing the data, one that imports everything, another
that imports only a small test subset that runs quickly. Comment one of
them out and use the other. For rapid development, use the small, quick
subset, then run it on the whole dataset once it works.
Jonathan
On Sun, Feb 13, 2022 at 6:19 PM Christian Grün
wrote:
> Hi Eric,
>
> That’s perfectly feasible. You can bind the dynamic query string to an
> external variable and invoke it on extracted data via xquery:eval.
> Here’s one way to do it:
>
> (: db.xml :)
>
>
> (: query.xq :)
> declare variable $query external;
>
> for $data in db:open('db')/path/to/source/data
> let $updated := xquery:eval($query, map { '': $data })
> return replace node $data with $updated
>
> The query could e.g. be invoked via
>
> basex -b query="{ . }" query.xq
>
> Hope this helps,
> Christian
>
>
>
> On Sun, Feb 13, 2022 at 11:26 PM Eric Levy wrote:
> >
> > I am completely new to BaseX, and trying to conceive of a way to use it
> > to process a compound query that combines an external query with a
> > static one.
> >
> > The core idea is a very simple application, expressed as an XQuery
> > file, that results in the following sequence of actions:
> >
> >1. Extraction of data from a database, by some query.
> >2. Modification of the results of (1), by some query.
> >3. Persistence of the results from (2), back into the database from
> > which the data was originally extracted.
> >
> > The modifications given in (2) would be expressed external to the
> > application, supplied as input per invocation, in a separate file or in
> > a command-line parameter.
> >
> > Thus, the overall operation would entail static components, the fixed
> > parts of the application, represented by (1) and (3), and dynamic
> > components, passed by the invocation, represented by (2).
> >
> > I realize such a processing pipeline may be achieved through queries
> > supplied in a full application accessing the database by one of the
> > language-specific APIs, but I am trying to consider an approach that
> > requires use only of XQuery statements, without involving a separate
> > programming language or formal build process.
> >
> > Does the design of BaseX accommodate an application having the
> > structure as described?
> >
> >
>
Re: [basex-talk] combining queries from mutiple sources
I've thought of using a build system or XProc for this. So far, I just
create a different .bxs for each of various flows. I am adding one or two
people to help me soon, I may get more organized before that happens ...
Jonathan
On Mon, Feb 14, 2022 at 9:49 AM Eliot Kimber
wrote:
> I’m doing more less the same thing. I create various reports and indexes
> over content stored in a primary database and use small XQuery scripts to
> perform actions that must be individual transactions (db:create, db:drop,
> db:alter, etc.), do the heavy data processing and large document loading in
> temporary databases and then swap the temporary databases with the
> production databases when appropriate.
>
>
>
> I’m doing the orchestration with bash scripts that call the basex
> client—maybe I’ve made it too complicated? But I didn’t see a way to pass
> parameters to BaseX scripts, thus shell scripts that just call the basex
> client to run small .xq files, specifying any needed parameters.
>
>
>
> Cheers,
>
>
>
> E.
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com <https://www.servicenow.com>
>
> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
> <https://twitter.com/servicenow> | YouTube
> <https://www.youtube.com/user/servicenowinc> | Facebook
> <https://www.facebook.com/servicenow>
>
>
>
> *From: *BaseX-Talk on behalf
> of Jonathan Robie
> *Date: *Monday, February 14, 2022 at 8:17 AM
> *To: *Christian Grün
> *Cc: *BaseX
> *Subject: *Re: [basex-talk] combining queries from mutiple sources
>
> *[External Email]*
>
>
>
> Also, you can have the best of both worlds, saving results in a pipeline
> if it's helpful for debugging, just running the updates in place once you
> know it all works well. This is particularly helpful if your data is
> complex and your datasets are largish.
>
>
>
> I am currently merging various sources that were not originally meant to
> work together, creating a common reference system for fairly complex data.
> I find it helpful to write a bunch of simple updates that can be executed
> sequentially, running them with a .bxs file. If I want to see the
> output after any of these updates, I can do so with an OUTPUT statement.
> Most of these stages are only useful for debugging purposes, so once a
> pipeline is working well, I get rid of the OUTPUT statement.
>
>
>
> If your data is large, it can also be helpful to have two different
> statements for importing the data, one that imports everything, another
> that imports only a small test subset that runs quickly. Comment one of
> them out and use the other. For rapid development, use the small, quick
> subset, then run it on the whole dataset once it works.
>
>
>
> Jonathan
>
>
>
> On Sun, Feb 13, 2022 at 6:19 PM Christian Grün
> wrote:
>
> Hi Eric,
>
> That’s perfectly feasible. You can bind the dynamic query string to an
> external variable and invoke it on extracted data via xquery:eval.
> Here’s one way to do it:
>
> (: db.xml :)
>
>
> (: query.xq :)
> declare variable $query external;
>
> for $data in db:open('db')/path/to/source/data
> let $updated := xquery:eval($query, map { '': $data })
> return replace node $data with $updated
>
> The query could e.g. be invoked via
>
> basex -b query="{ . }" query.xq
>
> Hope this helps,
> Christian
>
>
>
> On Sun, Feb 13, 2022 at 11:26 PM Eric Levy wrote:
> >
> > I am completely new to BaseX, and trying to conceive of a way to use it
> > to process a compound query that combines an external query with a
> > static one.
> >
> > The core idea is a very simple application, expressed as an XQuery
> > file, that results in the following sequence of actions:
> >
> >1. Extraction of data from a database, by some query.
> >2. Modification of the results of (1), by some query.
> >3. Persistence of the results from (2), back into the database from
> > which the data was originally extracted.
> >
> > The modifications given in (2) would be expressed external to the
> > application, supplied as input per invocation, in a separate file or in
> > a command-line parameter.
> >
> > Thus, the overall operation would entail static components, the fixed
> > parts of the application, represented by (1) and (3), and dynamic
> > components, passed by the invocation, represented by (2).
> >
> > I realize such a processing pipeline may be achieved through queries
> > supplied in a full application accessing the database by one of the
> > language-specific APIs, but I am trying to consider an approach that
> > requires use only of XQuery statements, without involving a separate
> > programming language or formal build process.
> >
> > Does the design of BaseX accommodate an application having the
> > structure as described?
> >
> >
>
>
[basex-talk] COPY DB vs. CREATE DB
What's the easiest way to copy a database, whether or not there is an existing copy with a given name? If I load a database with CREATE DB, it replaces any existing database with the same name. I would like to do the same with COPY DB, but it raises an error if there is a database with the same name. And DELETE DB raises an error if there isn't a database with that name. So what's the right way to do this? Jonathan
Re: [basex-talk] COPY DB vs. CREATE DB
Perfect.
In this particular case, doing it from within the query is better
regardless, since the query makes no sense if it doesn't start by doing
this.
Thanks!
Jonathan
On Wed, Feb 16, 2022 at 5:25 PM Christian Grün
wrote:
> Hi Jonathan,
>
> If you use the XQuery function db:copy, an existing database will
> simply be overwritten. The much older COPY command should by all means
> behave identically, so I’ve just updated to BaseX to make this happen
> [1,2]. You can use the command XQUERY db:copy('old', 'new') if you
> don’t want to wait for BaseX 9.7 (but it should only be a few days
> from now).
>
> Best,
> Christian
>
> [1] https://files.basex.org/releases/latest/
> [2] https://docs.basex.org/wiki/Commands#COPY
>
>
> On Wed, Feb 16, 2022 at 9:38 PM Jonathan Robie
> wrote:
> >
> > What's the easiest way to copy a database, whether or not there is an
> existing copy with a given name?
> >
> > If I load a database with CREATE DB, it replaces any existing database
> with the same name.
> >
> > I would like to do the same with COPY DB, but it raises an error if
> there is a database with the same name. And DELETE DB raises an error if
> there isn't a database with that name. So what's the right way to do this?
> >
> > Jonathan
>
[basex-talk] Long-running queries: command line or BaseX Gui?
Do long-running queries run faster from the command line, the GUI, or about the same from either? Jonathan
[basex-talk] Namespaces, replace, and performance
I wrote a query that replaced roughly 470153 text nodes with an element that had two namespaces declared on it. It ran for several hours. I didn't need or want these namespaces, they were in the original source. I rewrote the query to remove the namespaces from these nodes before replacing the text node. I don't know how quickly it ran, but I think not more than 15 minutes, the first time I checked on it, it had already finished. I made no other changes to the query. Is this surprising? Or is this a general lesson about performance and updates that add namespaces low down in the hierarchy? Or ... Jonathan
[basex-talk] Database 'name' is being updated, or update was not completed.
I have some processes that update databases, taking a long time to do so. I would like other processes to be able to read the last complete snapshot while this happens. Is that possible? Jonathan
[basex-talk] Faster in the cloud?
If I am running my queries and updates on a typical laptop, would they run much faster if I ran them on a suitably configured instance in the cloud? I know this is a very general question, but I'm wondering what experiences y'all have had with this. Jonathan
Re: [basex-talk] Faster in the cloud?
I have a 2013 Macbook Pro with 16 Gig RAM and a 1 Terabyte SSD. So not entirely wimpy, but nowhere near as fast as the current Macbooks, I have no idea how that compares to a typical laptop these days. Most things run fairly quickly, but inserting 2.5 million attributes into a document takes perhaps 5 hours, I didn't time it. I can run that overnight, and do test runs on smaller subsets, but I want to think through my options. Jonathan On Sat, Feb 19, 2022 at 6:11 PM Liam R. E. Quin wrote: > On Sat, 2022-02-19 at 16:05 -0500, Jonathan Robie wrote: > > If I am running my queries and updates on a typical laptop, would > > they run much faster if I ran them on a suitably configured instance > > in the cloud? > > "suitably configured" is very subjective. Potentially your queries > could run a lot faster. > > A lot depends on the speed of the disk (or SSD) in the laptop, and the > amount of memory it has, as well as the CPU - a recent Macbook Pro will > be faster than a ten-year-old chromebook. However, server blades (the > machines used in data centres) typically have much higher bandwidth > between memory and devices including both the CPU and the long-term > storage, and likely have more physical RAM than your laptop. > > On the other hand, connecting over the network to the cloud can be > slow > > Liam > > -- > Liam Quin, https://www.delightfulcomputing.com/ > Transformers team, Paligo.net > > Pictures from old books - www.fromoldbooks.org > >
[basex-talk] Adding lots of files in a defined order
I have 929 files with names like these: 01-Gen-001.xml 01-Gen-002.xml 01-Gen-003.xml 01-Gen-004.xml 01-Gen-005.xml 01-Gen-006.xml 01-Gen-007.xml 01-Gen-008.xml 01-Gen-009.xml 01-Gen-010.xml I would like to add them to a new database in the same order that I see if I do $ ls in that directory, without using XInclude to put them inside another node. Is there an easy way to do that? Jonathan
Re: [basex-talk] Adding lots of files in a defined order
Perfect! Thank you! Jonathan On Mon, Feb 21, 2022 at 5:52 PM Liam R. E. Quin wrote: > On Mon, 2022-02-21 at 16:28 -0500, Jonathan Robie wrote: > > I have 929 files with names like these: > > > > 01-Gen-001.xml > > > > I would like to add them to a new database in the same order that I > > see if > > I do $ ls in that directory, > > Are you doing this from within XQuery? > sort( > file:list($dir)[matches(., '\.xml$')] > ) ! db:add($db, doc(.), .) > > maybe? > > > -- > Liam Quin, https://www.delightfulcomputing.com/ > Available for XML/Document/Information Architecture/XSLT/ > XSL/XQuery/Web/Text Processing/A11Y training, work & consulting. > Barefoot Web-slave, antique illustrations: http://www.fromoldbooks.org > >
Re: [basex-talk] Faster in the cloud?
Cool!
That phrase "main memory updates" also implies there's something I
should learn. I am simply doing updates using a repository. Should I be
doing something to trigger main memory updates?
FWIW, here is a small sample of my data (one sentence) and the query I use
to expand a morphology code in each element into meaningful, readable
attributes. So a element in the output looks like this:
בְּ
רֵאשִׁית
בָּרָ֣א
אֱלֹהִ֑ים
אֵ֥ת
הַ
שּׁמַ֖יִם
וְ
אֵ֥ת
הָ
אָֽרֶץ
These are leaf nodes in a syntax tree - for simplicity, I am not showing
the syntax tree here, look to the input file for that.
Jonathan
On Tue, Feb 22, 2022 at 8:57 AM Christian Grün
wrote:
> A little announcement: With BaseX 10 [1], main memory updates will get
> much faster:
>
> {
> (1 to 100) !
> } update {
> y ! (insert node into .)
> }
>
> BaseX 9: ages (6-7 minutes)
> BaseX 10: 3 seconds
>
> The reason: The disk-based block storage layout is now also used for the
> main memory representation of XML nodes.
>
> [1] https://files.basex.org/releases/latest-10/
>
>
> On Tue, Feb 22, 2022 at 9:49 AM ETANCHAUD Fabrice <
> fabrice.etanch...@maif.fr> wrote:
>
>> Hi Jonathan !
>>
>> Apologizes for my late contribution...
>>
>> Do you really have to use XQuery Update ? Do you have to stick to a
>> specific format ?
>> If not, maybe you could use a schema on read approach ?
>> I mean, you could add new data as new documents,
>> and recombine these documents into the attribute based format when
>> requesting the data.
>>
>> Would that be a viable solution for you ?
>>
>> I once had success with this solution, as BaseX is very quick at adding
>> documents.
>>
>> Best regards,
>> Fabrice
>>
>>
>> --
>> *De :* BaseX-Talk de la
>> part de Eliot Kimber
>> *Envoyé :* lundi 21 février 2022 18:06
>> *À :* BaseX
>> *Objet :* Re: [basex-talk] Faster in the cloud?
>>
>>
>> You can use prof:track() to time your insertion operation for enough
>> iterations to get a reasonable time and then multiply by 2.5 million to get
>> an approximate time to completion.
>>
>>
>>
>> On my machine I’m finding times around 0.05 seconds for my operations,
>> which are more than just attribute insertions, where I need to do 40K
>> iterations. I would expect attribute insertion to be faster, especially if
>> you can batch up the insertions into a small number of transactions.
>>
>>
>>
>> But five hours to do the update doesn’t seem entirely out of spec if your
>> machine is significantly slower. Doing the math, I get 7ms per insertion:
>>
>>
>>
>> Hours
>>
>> Seconds/ Hour
>>
>> Seconds
>>
>> # operations
>>
>> Time/operation
>>
>> 5
>>
>> 3600
>>
>> 18000
>>
>> 250
>>
>> 0.0072
>>
>>
>>
>> That seems pretty fast on a per-operation standpoint.
>>
>>
>>
>> If you can break your content into multiple databases you could
>> parallelize the updates across multiple BaseX instances and then combine
>> the result back at the end.
>>
>>
>>
>> So spin up one server for each core, have a master server that provides a
>> REST API to kick off the processing and then use the REST method to farm
>> jobs out to each of the servers (using REST to make it easy to target each
>> of the servers via a port. Could also do it from a shell script through the
>> baseclient command-line.).
>>
>>
>>
>> With that should be able to reduce the processing to the time it takes
>> one server to process its share, which will be total objects/number of
>> cores (its share, that is).
>>
>>
>>
>> Cheers,
>>
>>
>>
>> E.
>>
>>
>>
>> _
>>
>> *Eliot Kimber*
>>
>> Sr Staff Content Engineer
>>
>> O: 512 554 9368
>>
>> M: 512 554 9368
>>
>> servicenow.com <https://www.servicenow.com>
>>
>> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
>> <https://twitter.com/servicenow> | YouTube
>> <https://www.youtube.com/user/servicenowinc> | Facebook
>> <https://www.facebook.com/servicenow>
>>
>>
>>
>> *From: *BaseX-Talk on
>> behalf of Jonathan Robie
>> *Date: *Monday, February 21, 2022 at 8:44 AM
>> *To: *Liam R. E. Quin
>> *Cc: *BaseX
>> *Subject: *Re: [basex-tal
Re: [basex-talk] feature request: opening database at arbitrary file path
Eric, I wonder if you are doing something similar to what I am doing.
I have a bunch of databases found in directories, and I currently use paths
that look a lot like my directories.
declare variable $gnt:Nestle1904.tei :=
doc("macula/GNT/Nestle1904/tei/nestle1904.xml");
declare variable $gnt:Nestle1904.morph :=
doc("macula/GNT/Nestle1904/morph/Nestle1904.xml");
declare variable $gnt:Nestle1904.lowfat :=
doc("macula/GNT/Nestle1904/lowfat/nestle1904lowfat.xml");
If I were writing a function that opened databases in a particular
directory, I might pass the "current directory" to that function. I
suspect that's an application-domain concept, not something that BaseX
should model for your application. After all, someone else might have a
rather different notion of a working directory for their application. And
"working directory" at the database level would add state in a way that
would be actively unhelpful for stateless APIs.
Jonathan
On Thu, Mar 3, 2022 at 10:39 AM Christian Grün
wrote:
> Hi Eric,
>
> There had been the suggestion that our documentation will provide you
> more information on how BaseX works. After you have invested more time
> on the status quo, we might be able to get some progress on this
> discussion.
>
> Best,
> Christian
>
>
> On Thu, Mar 3, 2022 at 4:31 PM Eric Levy wrote:
> >
> > Hello. I never had a chance to gain a full understanding of the
> > direction of this discussion. Had there been any suggestion that some
> > feature enhancement might be considered? I felt the suggestion may have
> > been given, but it was never resolved.
> >
> >
> > On Fri, 2022-02-25 at 02:37 -0500, Eric Levy wrote:
> > > > > The base case for an embedded database has no user model, and
> > > > > supports
> > > > > creation of new databases only through the target path. However,
> > > > > such
> > > > > differences might press the limits of a benign request.
> > > > >
> > > > > If a target path is needed beyond the database itself, then it
> > > > > might be
> > > > > passed on the command line. Alternatively, a special directory
> > > > > layout
> > > > > might be considered […]
> > > >
> > > > In both cases, we’d rebuild the existing solution step by step. And
> > > > if
> > > > no such path is specified by the user, we’d need to ensure for all
> > > > operations in the system that nothing unexpected happens.
> > >
> > > Sorry, I don't understand the meaning of the comment.
> > >
> > >
> > > > > Yes, but it doesn't mean I understand quite yet how all the parts
> > > > > fit
> > > > > together. Does it seem I missed or misunderstood something?
> > > >
> > > > I guess so. What other parts of our documentation have you read so
> > > > far
> > > > to get a more comprehensive picture?
> > >
> > > If you want to suggest a reference that is relevant, I would review
> > > it,
> > > if I have not done so yet. I don't wish to compile a list of
> > > documents
> > > or sections I have already reviewed.
> > >
> >
>
[basex-talk] OpenJDK vs. Oracle JDK versus ...
I am setting up a Windows 11 box. Which Java is best for BaseX? Jonathan
[basex-talk] Windows, WSL, and BaseX
I installed Windows System for Linux and installed BaseX on it, and also installed BaseX on Windows. Unfortunately, the two do not share databases. Is there a way to make them do so? Or a way to run basexgui from WSL? Or a better question that I should ask if I want to work productively on BaseX under Windows and am more accustomed to developing under Linux? Jonathan
[basex-talk] Local deployment of a database
We are creating server instances, but we also have users who want to work locally. Ideally, I would like to be able to create an installer that would add BaseX and a set of databases to a user's system. How hard would it be to do that? Jonathan
Re: [basex-talk] Local deployment of a database
Thanks, Christian, That looks great. I probably won't cross this bridge for a month or more, but this is what I was looking for. And for Windows and Mac, just tell them to install the databases from .zip files or whatever? Jonathan On Thu, Mar 24, 2022 at 6:45 AM Christian Grün wrote: > Hi Jonathan, hi Hans-Jürgen, > > If you are using our Windows installer, the NSIS installer > configuration file could be enhanced to additionally copy database > files to the target installation directory [1,2]. > > Does this help? > Christian > > [1] https://nsis.sourceforge.io/ > [2] https://github.com/BaseXdb/basex-dist/blob/master/win/BaseX.nsi > > > > On Wed, Mar 23, 2022 at 8:34 AM Hans-Juergen Rennau > wrote: > > > > Excellent - I think this is a common interest of those developing > solutions based on BaseX. > > > > And "as-a-service-optionally-local" might pass as a BaseX Solution > design pattern. > > > > Hans-Jürgen > > > > (PS Sometimes the most straightforward ideas come to mind last, or late.) > > > > Am Dienstag, 22. März 2022, 23:31:37 MEZ hat Jonathan Robie < > jonathan.ro...@gmail.com> Folgendes geschrieben: > > > > > > We are creating server instances, but we also have users who want to > work locally. > > > > Ideally, I would like to be able to create an installer that would add > BaseX and a set of databases to a user's system. How hard would it be to > do that? > > > > Jonathan >
Re: [basex-talk] BaseX 9.7 • Ukraine Edition
Thanks, Christian, I agree. I have done volunteer work with refugee families arriving in the United States. I have worked with Syrian refugees and Congolese refugees through World Relief here in Durham. I expect to work with Ukranian refugees. At local World Relief events, I see refugees from all of these countries, together with people who support them, all interacting with each other and sharing their stories. The victims of war have a lot in common, no matter where they are from. Jonathan On Thu, Mar 24, 2022 at 6:09 AM Christian Grün wrote: > Dear Mohammed Reda, > > Thanks for your reply. > > I can only agree on what you are saying: We are all human. > > By focusing on the Ukrainian war, we didn’t want to trivialise any > single other conflict in the world now and in the past. That’s one of > the reasons why donating money to large NGOs like Médecins Sans > Frontières is a good idea: They are operating in many countries > worldwide, including Syria, Irak, Afghanistan and Palestine, and they > have helped thousands of refugees to safely arrive in Europe. > > Next to that, Syrian refugees in Germany are now helping out > Ukrainians, and they share similar experiences. It’s wonderful to see > people supporting each other, no matter where they come from or what’s > their religion. > > It’s questionable for me to judge this as hypocritical, though. The > easiest way is always to be quiet. I wonder if that would be better? > Can we be fair at all as soon as we get specific? Maybe we should > neither focus on Ukraine nor Syria nor Afghanistan, but start with > Central Africa and think about the 5 million people who have died in > the Second Congo War and its aftermath. Does anyone remember that war > today, which had just ended 19 years ago? > > Thanks again, best regards, > Christian > > > > On Thu, Mar 24, 2022 at 9:41 AM reda chbihi wrote: > > > > Hi everybody, > > > > > > I hope you're doing well, > > > > I think that creating an Ukrainian edition will not stoping putin > > > > > > I don't know why you never thought about creating a palastinian, > irakian, syrian or afghan editions??? These people suffer since years or > decades... Creating Ukrainian edition is just hypocrisy!!! > > > > > > What i'm saying doesn't mean that i am agree with putin's war against > Ukrainian people. But we're all human and nobody deserves what's going on > Ukraine, palastine or Afghanistan. > > > > > > Regards > > Mohammed Reda CHBIHI >
[basex-talk] Storing maps in BaseX
Is there a way to store a map in a database and use it in other queries without recomputing it each time? Jonathan
Re: [basex-talk] Storing maps in BaseX
Hi Christian,
I think the database you describe looks like this:
one
two
three
four
five
six
seven
eight
nine
ten
So I always use XML for this? There's no persistent representation of the
map data structure?
Jonathan
On Thu, Mar 24, 2022 at 2:02 PM Christian Grün
wrote:
> Hi Jonathan,
>
> You can create databases that contain key/value pairs:
>
> let $words := {
> for $i in 1 to 10
> return { format-integer($i, 'w') }
> }
> return db:create('words', $words, 'words.xml')
>
> If you look up values in that database …
>
> for $n in (1 to 10) ! string()
> return db:open('words')//word[@n = $n] ! data()
>
> … the text index will be utilized, and your query will be rewritten as
> follows:
>
> (1 to 10) ! data(db:attribute("words", string(.))/
> self::attribute(n)/parent::word)
>
> If you don’t want to rely on the rewritings of the query optimizer,
> you can directly use db:attribute.
>
> Best,
> Christian
>
Re: [basex-talk] Storing maps in BaseX
Got it.
Thanks!
Jonathan
On Thu, Mar 24, 2022 at 3:06 PM Christian Grün
wrote:
> Right, that’s the approach I would recommend.
>
> But you can use the JSON representation to store a map as binary resource:
>
> (: store map :)
> map { 'a': 'b' }
> ! db:store('map', 'map.txt', json:serialize(.))
> (: retrieve map :)
> db:retrieve('map', 'map.txt')
> => bin:decode-string()
> => parse-json()
>
> It can also be stored as XML resource by converting it to one of our JSON
> representations [1]:
>
> (: store map :)
> let $map := map { 'a': 'b' }
> let $xml := json:parse(json:serialize($map))
> return db:add('map', $xml, 'map.xml')
>
> (: retrieve map :)
> db:open('map', 'map.xml')
> => json:serialize()
> => json:parse(map { 'format': 'xquery' })
>
> In both cases, the full map data will be processed and converted, so it
> will take some additional time. However, if you have numerous lookups in
> your query, it might still be faster than the initial XML proposal.
>
> [1] https://docs.basex.org/wiki/JSON_Module#Conversion_Formats
>
>
> Jonathan Robie schrieb am Do., 24. März 2022,
> 19:10:
>
>> Hi Christian,
>>
>> I think the database you describe looks like this:
>>
>>
>> one
>> two
>> three
>> four
>> five
>> six
>> seven
>> eight
>> nine
>> ten
>>
>>
>> So I always use XML for this? There's no persistent representation of
>> the map data structure?
>>
>> Jonathan
>>
>> On Thu, Mar 24, 2022 at 2:02 PM Christian Grün
>> wrote:
>>
>>> Hi Jonathan,
>>>
>>> You can create databases that contain key/value pairs:
>>>
>>> let $words := {
>>> for $i in 1 to 10
>>> return { format-integer($i, 'w') }
>>> }
>>> return db:create('words', $words, 'words.xml')
>>>
>>> If you look up values in that database …
>>>
>>> for $n in (1 to 10) ! string()
>>> return db:open('words')//word[@n = $n] ! data()
>>>
>>> … the text index will be utilized, and your query will be rewritten as
>>> follows:
>>>
>>> (1 to 10) ! data(db:attribute("words", string(.))/
>>> self::attribute(n)/parent::word)
>>>
>>> If you don’t want to rely on the rewritings of the query optimizer,
>>> you can directly use db:attribute.
>>>
>>> Best,
>>> Christian
>>>
>>
[basex-talk] Serialization ... one attribute per line ...
I have a set of syntax trees that I would like to serialize with one attribute per line, like this: https://github.com/biblicalhumanities/greek-new-testament/blob/master/syntax-trees/nestle1904-lowfat/xml/18-philemon.xml The current serialization looks more like this: https://github.com/biblicalhumanities/greek-new-testament/blob/master/syntax-trees/nestle1904/xml/18-philemon.xml Is there a way to get there using just serialization parameters, or do I need to do something fancier? Jonathan
[basex-talk] Add a comment to a backup?
I have been making backups before doing particularly complex things to my treebanks, and I find myself writing down information about what stage of processing a given backup corresponds to. "after replacing subtrees for missing compounds" I wish I could associate these strings with backups in BaseX so I can more easily know which one I would restore if something went wrong. Jonathan
[basex-talk] Simple delete bug
Expressions like this fail silently without raising an error in BaseXGUI: delete //m/@n Of course, it should be: delete nodes //m/@n But there should be an error to remind me ;-> Jonathan
Re: [basex-talk] Simple delete bug
DOH. Yeah, you're right. Jonathan On Wed, Mar 30, 2022 at 9:25 AM Christian Grün wrote: > Hi Jonathan, > > > delete //m/@n > > This may be surprising, but your expression is a valid query: It’s a > path expression starting with an initial child::delete step. > > Best, > Christian >
Re: [basex-talk] Add a comment to a backup?
Excellent! Thanks! Jonathan On Thu, Mar 31, 2022 at 7:27 AM Christian Grün wrote: > Thanks for all the pointers! > > Good news: It’s now possible to add comments to database backups [1,2]. > > Feedback is welcome. > Christian > > [1] https://github.com/BaseXdb/basex/issues/2084 > [2] https://files.basex.org/releases/latest/ > > > > > On Wed, Mar 30, 2022 at 6:40 PM Majewski, Steven Dennis (sdm7g) > wrote: > > > > It’s in ZipFile.getComment() > > > > > https://docs.oracle.com/javase/8/docs/api/java/util/zip/ZipFile.html#getComment-- > > > > — Steve M. > > > > On Mar 30, 2022, at 12:17 PM, Christian Grün > wrote: > > > > I'm wondering if zip's `setComment` might be useful here. > > > > > > Interesting hint. > > > > I have discovered the ZipOutputStream.setComment function. There > > doesn’t seem to be an equivalent ZipInputStream.getComment function, > > though. Maybe I overlooked something? > > > > > > > > On Wed, Mar 30, 2022 at 1:44 PM Christian Grün > > wrote: > > > > > > Hi Jonathan, > > > > Currently, it’s not possible to attach comments to backups. I’ll have > > some more thoughts on your feature request. Maybe we could add an info > > file in the zipped archive and partially unzip it when listing > > available backups. > > > > All the best, > > Christian > > > > > > On Wed, Mar 30, 2022 at 12:26 AM Jonathan Robie > > wrote: > > > > > > I have been making backups before doing particularly complex things to > my treebanks, and I find myself writing down information about what stage > of processing a given backup corresponds to. > > > > "after replacing subtrees for missing compounds" > > > > I wish I could associate these strings with backups in BaseX so I can > more easily know which one I would restore if something went wrong. > > > > Jonathan > > > > >
Re: [basex-talk] Joining a varying number of information sources (with XQuery)?
> > Customised data structures can be created together with XQuery maps and > arrays. > But I find that a join operation would be needed before based on available > identification data. > Could you please write up a few use cases with data to show where you are running into difficulty? I am working with fairly complex linguistic datasets and joining them every which way with XQuery. The designers of XQuery included Don Chamberlin. one of the inventors of SQL, and Jim Melton, the chair of the SQL committee. We were looking at joining and combining diverse datasets as a key use case for XQuery. It's possible that we missed something, but it would be extremely helpful to have a concrete use case to consider, with real data. Jonathan On Mon, Apr 18, 2022 at 4:04 AM Markus Elfring wrote: > >> See also: > >> https://www.w3.org/TR/xquery-31/#id-joins > >> > > You can do join _operations_, > > > I would appreciate further clarification for the distinction > which you present here. > > > > > but you aren't doing them on tables > > (unless you did extra work to represent the tables hierarchically) > > > Some “tables” can be transformed into XQuery sequences, can't they? > > > > and there's absolutely no need for the keywords because the existing > > more general mechanisms work fine. > > > I see further development challenges in this area for the safe and > convenient application > of join conditions (or constraints). > > > I guess that you prefer to refer to them as “predicates within steps” so > far > (according to path expressions). > https://www.w3.org/TR/xquery-31/#id-predicate > > > > Use your functions to create maps where the keys come from that id > > element's string value. > > > Customised data structures can be created together with XQuery maps and > arrays. > But I find that a join operation would be needed before based on available > identification data. > > > > (: bind to the sequence of id values :) > > for $id in $interesting_stuff1 > > return > > (: run the function per-id :) > > > my_fun:do_something($id,$interesting_stuff2($id),$interesting_stuff3($id)) > > > > You could decide to skip the for clause and use > > > > return $interesting_stuff1 ! > > my_fun:do_something(.,$interesting_stuff2(.),$interesting_stuff3(.)) > > > > instead. > > > How do you think about to work without an extra identification sequence > variable? > > > > >> Will any further comparisons evolve for the provided functionality? > > Don't think so. I find the trick with XQuery is to not fight with it > > about being some other language. > > > > Internalizing the sequence concept takes work; … > > > Would you like to extend programming interfaces for the management of > relationships > with various entities? > > > Regards, > Markus >
Re: [basex-talk] Joining a varying number of information sources (with XQuery)?
Responses inline ... On Mon, Apr 18, 2022 at 3:54 PM Markus Elfring wrote: > > > We were looking at joining and combining diverse datasets as a key use > case for XQuery. > > This technical requirement is generally fine. > > > It's possible that we missed something, > > Do preferences matter if combinations of information sources are performed > with a key word > like “JOIN” (or not)? > That's just syntax. And people clearly have different preferences for syntax. It's easy for that to turn into bike-shedding ( https://en.wikipedia.org/wiki/Law_of_triviality). > but it would be extremely helpful to have a concrete use case to > consider, with real data. > > Would you like to check the influence of numbers according to joinable > items > once more on data processing efforts? > Are you asking about how well XQuery implementations perform compared to SQL databases? For a particular kind of query? If not, what are you asking? Jonathan
Re: [basex-talk] Joining a varying number of information sources (with XQuery)?
Hi Markus,
Have you read the XQuery specifications? This section on joins, written by
one of the inventors of SQL, may be a helpful starting point:
https://www.w3.org/TR/xquery-31/#id-joins
The use case documents may also be helpful:
https://www.w3.org/TR/xquery-use-cases/
https://www.w3.org/TR/xquery-30-use-cases/
https://www.w3.org/TR/xquery-31-requirements/
Please read up on joins in these places, I think it will help establish
common ground for this conversation.
Jonathan
On Tue, Apr 19, 2022, 04:32 Markus Elfring wrote:
> > To join something, you minimally need at least two expressions which find
> > the things to be joined,
>
> I am still trying to clarify corresponding development possibilities
> according to
> bigger numbers of entities together with for clauses.
> (I hope that other meanings can be better distinguished from related
> applications
> of a function like “string-join”.)
>
>
> > some kind of rule for how to perform the join,
> > and a destination for the result of the join.
>
> This is usual.
>
>
> > [predicates aren't joins]
>
> How does this feedback fit to other documentations which describe the role
> of
> predicates for join operations?
>
>
> > A join requires the results of two expressions to be combined.
> >
> >
> (db:open('thing1'),db:open('thing2'))/descendant::patienti-identifier[local:check-range($interesting,.)]
> >
> > can be interpreted as a performing a join, you can re-write it as
> >
> >
> (db:open('thing1')/descendant::patient-identifier[local:check-range($interesting,.)],db:open('thing2')/descendant::patient-identifier[local:check-range($interesting,.)])
> >
> > but the predicate isn't doing the joining, the predicate is narrowing
> > the selection of the XPath expressions.
>
> This can be a desirable effect.
>
>
> > In this case, the comma operator is doing the joining.
>
> This example refers to another variant of a join operation for the
> construction
> of a sequence.
> (Such a XQuery code fragment does not use for clauses.)
>
>
> > > Do you care for the number of involved items here?
> >
> > No.
>
> Would you like to adjust this view according to the usage of for clauses?
> https://www.w3.org/TR/2017/REC-xquery-31-20170321/#id-joins
>
> Regards,
> Markus
>
Re: [basex-talk] Joining a varying number of information sources (with XQuery)?
Here's one thing you may be asking - do you want to know how to specify a
join for n sources? If so, maybe this example will help:
declare variable $eng := (one, two, three, four);
declare variable $deu := (eins, zwei, drei, vier);
declare variable $ukr := (один, два, три, чотири);
declare variable $heb := (אחד, שתיים, שלוש, ארבע);
for $e in $eng
for $d in $deu
for $u in $ukr
for $h in $heb
where $e/@n = $d/@n
and $e/@n = $u/@n
and $e/@n = $h/@n
return
{ $e, $d, $u, $h }
This returns:
one
eins
один
אחד
two
zwei
два
שתיים
three
drei
три
שלוש
four
vier
чотири
ארבע
Does that answer your question? I routinely do queries that join multiple
sources.
Jonathan
On Tue, Apr 19, 2022 at 6:04 AM Markus Elfring
wrote:
> > Have you read the XQuery specifications?
>
> Yes.
>
>
> > This section on joins, written by one of the inventors of SQL, may be a
> helpful starting point:
> >
> > https://www.w3.org/TR/xquery-31/#id-joins
>
> It seems that I stumble on communication difficulties for this application
> area.
>
> Regards,
> Markus
>
Re: [basex-talk] Add a comment to a backup?
Awesome, thanks! And yes, adding comments to a backup is very helpful for me. Jonathan On Fri, May 13, 2022, 05:56 Christian Grün wrote: > Hi Jonathan, > > I hope the facility to add comments to backups has turned out to be > helpful. > > We have further enhanced db:create-backup with a boolean compress flag > [1]: If you use a recent version, you’ll need to update your code with > the upcoming version 9.7.2 of BaseX. Nothing will change if you use > the CREATE BACKUP command or the GUI. > > Best, > Christian > > [1] https://github.com/BaseXdb/basex/issues/2103 > > > > On Wed, Mar 30, 2022 at 12:26 AM Jonathan Robie > wrote: > > > > I have been making backups before doing particularly complex things to > my treebanks, and I find myself writing down information about what stage > of processing a given backup corresponds to. > > > > "after replacing subtrees for missing compounds" > > > > I wish I could associate these strings with backups in BaseX so I can > more easily know which one I would restore if something went wrong. > > > > Jonathan >
Re: [basex-talk] Fonts problem with database list and system dialogs
This is a new problem. I don't know what changed, I sent my email when I first encountered it. Here's one possible explanation with a fix: https://stackoverflow.com/questions/71788510/java-font-size-changes-from-win-10-to-win-11 Jonathan On Tue, Jun 28, 2022, 07:00 Christian Grün wrote: > Thanks, Jonathan, for your additional tests! – So I guess you always had > that issue on your environments (i.e., with all older versions of BaseX as > well)? Or has it been introduced with a more recent version? > > If you like, you can check some of our older releases here: > > https://files.basex.org/releases/ > > > > On Mon, Jun 27, 2022 at 5:59 PM Jonathan Robie > wrote: > >> Hi Christian, >> >> > java --version >> openjdk 17.0.2 2022-01-18 >> OpenJDK Runtime Environment Temurin-17.0.2+8 (build 17.0.2+8) >> OpenJDK 64-Bit Server VM Temurin-17.0.2+8 (build 17.0.2+8, mixed mode, >> sharing) >> >> I have the same problem with Basex 10, or with reinstalling the latest >> BaseX from the downloads page. I also have the same problem with a later >> JDK: >> >> PS C:\Users\jonat> java --version >> openjdk 18.0.1 2022-04-19 >> OpenJDK Runtime Environment Temurin-18.0.1+10 (build 18.0.1+10) >> OpenJDK 64-Bit Server VM Temurin-18.0.1+10 (build 18.0.1+10, mixed mode, >> sharing) >> >> Jonathan >> >> On Mon, Jun 20, 2022 at 4:45 PM Christian Grün >> wrote: >> >>> Hi Jonathan, >>> >>> Interesting; we’ve switched to Windows 11 as well, and the UI of BaseX >>> didn’t change. Let me think of some more questions … >>> >>> 1. Which distribution of Java are you using (Adoptium, Corretto, …), and >>> which version? >>> 2. Does it make a difference if you switch to another version (8, 11, >>> 17, …)? >>> 3. Does the problem also occur with BaseX 10 [1]? >>> >>> And … Did anyone else encounter a similar issue? >>> >>> Thanks in advance, >>> Christian >>> >>> [1] >>> https://files.basex.org/maven/org/basex/basex/10.0-SNAPSHOT/basex-10.0-20220614.103321-6.jar >>> >>> >>> >>> >>> On Mon, Jun 20, 2022 at 10:18 PM Jonathan Robie < >>> jonathan.ro...@gmail.com> wrote: >>> >>>> Hi Christian, >>>> >>>> Windows 11. I think it's bog standard, it's a new system. >>>> >>>> Jonathan >>>> >>>> On Mon, Jun 20, 2022 at 3:55 PM Christian Grün < >>>> christian.gr...@gmail.com> wrote: >>>> >>>>> Hi Jonathan, >>>>> >>>>> Can you possibly detail which version of Windows you are using and how >>>>> it possibly differs from a standard Windows installation? >>>>> >>>>> Thanks, >>>>> Christian >>>>> >>>>> >>>>> >>>>> On Mon, Jun 20, 2022 at 7:40 PM Jonathan Robie < >>>>> jonathan.ro...@gmail.com> wrote: >>>>> >>>>>> I can't read the names of databases in my UI now - the font is tiny. >>>>>> And the systems dialogs I might otherwise be able to use to find and fix >>>>>> the problem also use tiny fonts. >>>>>> >>>>>> Other windows are fine. >>>>>> >>>>>> [image: image.png] >>>>>> >>>>>> Jonathan >>>>>> >>>>>
Re: [basex-talk] an Earley parser in XQuery?
Are you talking HPSG or something like that? What is your input? Jonathan On Mon, Aug 22, 2022 at 1:08 PM Graydon Saunders wrote: > Hello -- > > I'm trying to test if some extracted sentences validate as productions of > a particular context-free grammar expressed in a BNF dialect. > > Is there an available implementation of a parser in XQuery that can do > this? > > Thanks! > Graydon >
Re: [basex-talk] Pretty print
On Thu, Nov 17, 2022 at 12:01 PM Christian Grün wrote: > But the indentation is quite different from what I see in Saxon or oXygen >> output when I indent. You see this with more complex examples. >> > > That’s true, every query processor uses custom indentation algorithms; the > specification gives much freedom here [1]. If indentation is important, > it’s always recommendable to either preserve the original formatting or use > xml:space='preserve' for mixed-context sections. > DOH! I should be using xml:space="preserve". But is there no way to declare that when I import a file to the database? Sometimes I don't want to change the original file, but I do want to preserve whitespace. > I’ll never be happy with the decision in XML to lump together indentation > of structure and content. > > [1] https://www.w3.org/TR/xslt-xquery-serialization-31/#xml-indent > On standards groups, we always spent a LOT more time discussing whitespace than character content, it took up enormous amounts of time. And part of it is that there's not really a good way in XML to distinguish indentation from whitespace content. What would you have done differently? If there's an obvious, simple way this could have been improved, I'd be curious what it is. Jonathan
Re: [basex-talk] Pretty print
Specifying this in the schema is actually a rather good solution, I think - at least for many cases. Jonathan On Thu, Nov 17, 2022 at 1:44 PM Eliot Kimber wrote: > Note that the XML mixed content and whitespace design was inherited from > SGML, where DTDs were required, and so a parser always knew for sure > whether a given context was or was not mixed content. > > It’s been a couple decades, but my memory is that anything we did in XML > to address this in the face of not requiring any kind of grammar would have > been even more disruptive, such as not allowing mixed content at all and > having some special syntax just for identifying text nodes. > > > > So it wasn’t really a decision so much as there not really being a better > solution in the context of SGML as our starting point. > > > > Cheers, > > > > E. > > > > _ > > *Eliot Kimber* > > Sr Staff Content Engineer > > O: 512 554 9368 > > M: 512 554 9368 > > servicenow.com <https://www.servicenow.com> > > LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter > <https://twitter.com/servicenow> | YouTube > <https://www.youtube.com/user/servicenowinc> | Facebook > <https://www.facebook.com/servicenow> > > > > *From: *BaseX-Talk on behalf > of Christian Grün > *Date: *Thursday, November 17, 2022 at 11:01 AM > *To: *Jonathan Robie > *Cc: *basex-talk@mailman.uni-konstanz.de < > basex-talk@mailman.uni-konstanz.de> > *Subject: *Re: [basex-talk] Pretty print > > *[External Email]* > > > -- > > But the indentation is quite different from what I see in Saxon or oXygen > output when I indent. You see this with more complex examples. > > > > That’s true, every query processor uses custom indentation algorithms; the > specification gives much freedom here [1]. If indentation is important, > it’s always recommendable to either preserve the original formatting or use > xml:space='preserve' for mixed-context sections. > > > > I’ll never be happy with the decision in XML to lump together indentation > of structure and content. > > > > [1] https://www.w3.org/TR/xslt-xquery-serialization-31/#xml-indent > > >
Re: [basex-talk] Formatting attributes as an indented list?
Hi Christian, Currently, I am using HTML tidy to reformat the XML output. It gives me the formatting I need, which is Git-diff friendly. Jonathan $ nodes % tidy --version HTML Tidy for Apple macOS version 5.6.0 $ nodes % tidy -config tidy.config 03-luke.xml Sample Output: wrote: > Hi Patrick > > I noticed that the attributes for the wg element had not been aligned, so > I was wondering if you were thinking of a more advanced rule. > > Or would you possibly like to supply the names of the elements for which > the alignment should take place? > > Best, > Christian > > > > Patrick Durusau schrieb am Mi., 15. Feb. 2023, > 03:51: > >> Christian, >> >> Ah, no, it isn't a length of element name + attribute but the ability to >> align attributes for an element as you see in my post for the > element. Each key/value is followed by a line return. >> >> In the mean time, the current version of tidy has been added to the >> workflow to produce the desired results. >> >> But it would be great to have it native to BaseX! >> >> Thanks! >> >> Patrick >> >> On 2/14/23 01:30, Christian Grün wrote: >> > Hi Patrick, >> > >> > There’s currently no serialization parameter to control the custom >> > indentation of attributes. >> > >> > If I get you correctly, you’d like to get attributes indented if the >> > string length of the element name and the attributes exceed a specific >> > maximum length? >> > >> > Best, >> > Christian >> > >> > >> > On Mon, Feb 13, 2023 at 9:10 PM Patrick Durusau >> wrote: >> >> Greetings! >> >> >> >> I've been tasked with using BaseX to produce: >> >> >> >> * >> >> >> >> >> >> >> >> > rule="Np-Appos"> >> >> > >>after=" " >> >>class="noun" >> >>gbiType="proper" >> >>xml:id="n57001001001" >> >>lemma="Παῦλος" >> >>normalized="Παῦλος" >> >>strong="3972" >> >>number="singular" >> >>gender="masculine" >> >>case="nominative" >> >>gloss="Paul" >> >>domain="093001" >> >>ln="93.294a" >> >>morph="N-NSM" >> >>unicode="Παῦλος">Παῦλος >> >> >> >> * >> >> >> >> The indenting is easy enough and I can even make it deeper if required >> >> but is there a command for serialization that will properly format the >> >> attributes? >> >> >> >> My personal suspicion is that inserting \n when each attribute is >> >> serialized (and not on the last one) is the easier route but I promised >> >> to investigate the command line. >> >> >> >> Have I overlooked something in the very fine manual? >> >> >> >> Hope everyone is having a great week! >> >> >> >> Patrick >> >> >> >> -- >> >> Patrick Durusau >> >> patr...@durusau.net >> >> Technical Advisory Board, OASIS (TAB) >> >> Editor, OpenDocument Format TC (OASIS), Project Editor ISO/IEC 26300 >> >> Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps) >> >> >> >> Another Word For It (blog): http://tm.durusau.net >> >> Homepage: http://www.durusau.net >> >> Twitter: patrickDurusau >> >> >> -- >> Patrick Durusau >> patr...@durusau.net >> Technical Advisory Board, OASIS (TAB) >> Editor, OpenDocument Format TC (OASIS), Project Editor ISO/IEC 26300 >> Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps) >> >> Another Word For It (blog): http://tm.durusau.net >> Homepage: http://www.durusau.net >> Twitter: patrickDurusau >> >>
Re: [basex-talk] Formatting attributes as an indented list?
Hi Christian, I prefer to be able to require one attribute per line. This is important for Git diffs, which are the main reason we care. Jonathan On Wed, Feb 15, 2023 at 11:31 AM Christian Grün wrote: > Hi Jonathan, > > Thanks for sharing your tidy settings. > > With the given configuration, all attributes except for the first are > returned in a separate line… > > rule="S-IO" > cltype="VerbElided"> > > In Patrick’s example, some attributes were returned in a single line > (possibly depending on the expected string length). Maybe it was > generated via Saxon (just a guess): > > > > Do you have a preference which representation would be required, or do > you think the details are not that relevant? > > We could possibly add a custom serialization parameter similar to > tidy’s 'indent-attributes' option, and it would probably be easier to > ignore the expected string length. > > All the best, > Christian >
[basex-talk] basexgui is crashing for me ...
Basexgui has been crashing for me. I thought my environment might be messed up, so I uninstalled brew, reinstalled it, and checked my environment. Then I ran basexgui, and got another crash. % basexgui > 2024-01-09 15:09:58.791 java[18502:5023580] WARNING: Secure coding is > automatically enabled for restorable state! However, not on all supported > macOS versions of this application. Opt-in to secure coding explicitly by > implementing > NSApplicationDelegate.applicationSupportsSecureRestorableState:. > # > # A fatal error has been detected by the Java Runtime Environment: > # > # SIGILL (0x4) at pc=0x7ff80e07cb73, pid=18502, tid=259 > # > # JRE version: OpenJDK Runtime Environment Homebrew (21.0.1) (build 21.0.1) > # Java VM: OpenJDK 64-Bit Server VM Homebrew (21.0.1, mixed mode, sharing, > tiered, compressed oops, compressed class ptrs, g1 gc, bsd-amd64) > # Problematic frame: > # C [AppKit+0xc8fb73] _NSCarbonMenuCrashIfNeeded+0x258 > # > # Core dump will be written. Default location: /cores/core.18502 > # > # An error report file with more information is saved as: > # /Users/jonathan/github/Clear/hs_err_pid18502.log > # > # If you would like to submit a bug report, please visit: > # https://github.com/Homebrew/homebrew-core/issues > # The crash happened outside the Java Virtual Machine in native code. > # See problematic frame for where to report the bug. > # > zsh: abort basexgui > What should I do? Jonathan
Re: [basex-talk] basexgui is crashing for me ...
Thanks for the prompt, helpful response! Jonathan On Wed, Jan 10, 2024 at 5:54 AM Christian Grün wrote: > Hi Jonathan, > > This problem is related to Homebrew and Apple [1], and does not seem to be > solved yet. > > There are alternatives, though. One solution that has been suggested by my > colleague Michael is to: > > 1. Uninstall the homebrew openjdk: >brew uninstall openjdk --ignore-dependencies > 2. Manual install Adoptium Open JDK (11 or later): > https://adoptium.net: > > See [2] for further information. > > Hope this helps, > Christian > > [1] https://github.com/Homebrew/homebrew-core/issues/150824 > [2] > https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg15457.html > > > > On Tue, Jan 9, 2024 at 9:17 PM Jonathan Robie > wrote: > >> Basexgui has been crashing for me. >> >> I thought my environment might be messed up, so I uninstalled brew, >> reinstalled it, and checked my environment. >> >> Then I ran basexgui, and got another crash. >> >> % basexgui >>> 2024-01-09 15:09:58.791 java[18502:5023580] WARNING: Secure coding is >>> automatically enabled for restorable state! However, not on all supported >>> macOS versions of this application. Opt-in to secure coding explicitly by >>> implementing >>> NSApplicationDelegate.applicationSupportsSecureRestorableState:. >>> # >>> # A fatal error has been detected by the Java Runtime Environment: >>> # >>> # SIGILL (0x4) at pc=0x7ff80e07cb73, pid=18502, tid=259 >>> # >>> # JRE version: OpenJDK Runtime Environment Homebrew (21.0.1) (build >>> 21.0.1) >>> # Java VM: OpenJDK 64-Bit Server VM Homebrew (21.0.1, mixed mode, >>> sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-amd64) >>> # Problematic frame: >>> # C [AppKit+0xc8fb73] _NSCarbonMenuCrashIfNeeded+0x258 >>> # >>> # Core dump will be written. Default location: /cores/core.18502 >>> # >>> # An error report file with more information is saved as: >>> # /Users/jonathan/github/Clear/hs_err_pid18502.log >>> # >>> # If you would like to submit a bug report, please visit: >>> # https://github.com/Homebrew/homebrew-core/issues >>> # The crash happened outside the Java Virtual Machine in native code. >>> # See problematic frame for where to report the bug. >>> # >>> zsh: abort basexgui >>> >> >> What should I do? >> >> Jonathan >> >
Re: [basex-talk] basexgui is crashing for me ...
Sigh - I just did that, it still crashes ... Jonathan On Wed, Jan 10, 2024 at 1:22 PM Jonathan Robie wrote: > Thanks for the prompt, helpful response! > > Jonathan > > On Wed, Jan 10, 2024 at 5:54 AM Christian Grün > wrote: > >> Hi Jonathan, >> >> This problem is related to Homebrew and Apple [1], and does not seem to >> be solved yet. >> >> There are alternatives, though. One solution that has been suggested by >> my colleague Michael is to: >> >> 1. Uninstall the homebrew openjdk: >>brew uninstall openjdk --ignore-dependencies >> 2. Manual install Adoptium Open JDK (11 or later): >> https://adoptium.net: >> >> See [2] for further information. >> >> Hope this helps, >> Christian >> >> [1] https://github.com/Homebrew/homebrew-core/issues/150824 >> [2] >> https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg15457.html >> >> >> >> On Tue, Jan 9, 2024 at 9:17 PM Jonathan Robie >> wrote: >> >>> Basexgui has been crashing for me. >>> >>> I thought my environment might be messed up, so I uninstalled brew, >>> reinstalled it, and checked my environment. >>> >>> Then I ran basexgui, and got another crash. >>> >>> % basexgui >>>> 2024-01-09 15:09:58.791 java[18502:5023580] WARNING: Secure coding is >>>> automatically enabled for restorable state! However, not on all supported >>>> macOS versions of this application. Opt-in to secure coding explicitly by >>>> implementing >>>> NSApplicationDelegate.applicationSupportsSecureRestorableState:. >>>> # >>>> # A fatal error has been detected by the Java Runtime Environment: >>>> # >>>> # SIGILL (0x4) at pc=0x7ff80e07cb73, pid=18502, tid=259 >>>> # >>>> # JRE version: OpenJDK Runtime Environment Homebrew (21.0.1) (build >>>> 21.0.1) >>>> # Java VM: OpenJDK 64-Bit Server VM Homebrew (21.0.1, mixed mode, >>>> sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-amd64) >>>> # Problematic frame: >>>> # C [AppKit+0xc8fb73] _NSCarbonMenuCrashIfNeeded+0x258 >>>> # >>>> # Core dump will be written. Default location: /cores/core.18502 >>>> # >>>> # An error report file with more information is saved as: >>>> # /Users/jonathan/github/Clear/hs_err_pid18502.log >>>> # >>>> # If you would like to submit a bug report, please visit: >>>> # https://github.com/Homebrew/homebrew-core/issues >>>> # The crash happened outside the Java Virtual Machine in native code. >>>> # See problematic frame for where to report the bug. >>>> # >>>> zsh: abort basexgui >>>> >>> >>> What should I do? >>> >>> Jonathan >>> >>
Re: [basex-talk] basexgui is crashing for me ...
I should have included this: % java --version openjdk 21.0.1 2023-10-17 LTS OpenJDK Runtime Environment Temurin-21.0.1+12 (build 21.0.1+12-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.1+12 (build 21.0.1+12-LTS, mixed mode) % basexgui 2024-01-10 13:40:21.624 java[56086:5869820] WARNING: Secure coding is automatically enabled for restorable state! However, not on all supported macOS versions of this application. Opt-in to secure coding explicitly by implementing NSApplicationDelegate.applicationSupportsSecureRestorableState:. # # A fatal error has been detected by the Java Runtime Environment: # # SIGILL (0x4) at pc=0x7ff80e07cb73, pid=56086, tid=259 # # JRE version: OpenJDK Runtime Environment Homebrew (21.0.1) (build 21.0.1) # Java VM: OpenJDK 64-Bit Server VM Homebrew (21.0.1, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-amd64) # Problematic frame: # C [AppKit+0xc8fb73] _NSCarbonMenuCrashIfNeeded+0x258 # # Core dump will be written. Default location: /cores/core.56086 # # An error report file with more information is saved as: # /Users/jonathan/github/Clear/hs_err_pid56086.log # # If you would like to submit a bug report, please visit: # https://github.com/Homebrew/homebrew-core/issues # The crash happened outside the Java Virtual Machine in native code. # See problematic frame for where to report the bug. # zsh: abort basexgui On Wed, Jan 10, 2024 at 1:30 PM Jonathan Robie wrote: > Sigh - I just did that, it still crashes ... > > Jonathan > > On Wed, Jan 10, 2024 at 1:22 PM Jonathan Robie > wrote: > >> Thanks for the prompt, helpful response! >> >> Jonathan >> >> On Wed, Jan 10, 2024 at 5:54 AM Christian Grün >> wrote: >> >>> Hi Jonathan, >>> >>> This problem is related to Homebrew and Apple [1], and does not seem to >>> be solved yet. >>> >>> There are alternatives, though. One solution that has been suggested by >>> my colleague Michael is to: >>> >>> 1. Uninstall the homebrew openjdk: >>>brew uninstall openjdk --ignore-dependencies >>> 2. Manual install Adoptium Open JDK (11 or later): >>> https://adoptium.net: >>> >>> See [2] for further information. >>> >>> Hope this helps, >>> Christian >>> >>> [1] https://github.com/Homebrew/homebrew-core/issues/150824 >>> [2] >>> https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg15457.html >>> >>> >>> >>> On Tue, Jan 9, 2024 at 9:17 PM Jonathan Robie >>> wrote: >>> >>>> Basexgui has been crashing for me. >>>> >>>> I thought my environment might be messed up, so I uninstalled brew, >>>> reinstalled it, and checked my environment. >>>> >>>> Then I ran basexgui, and got another crash. >>>> >>>> % basexgui >>>>> 2024-01-09 15:09:58.791 java[18502:5023580] WARNING: Secure coding is >>>>> automatically enabled for restorable state! However, not on all supported >>>>> macOS versions of this application. Opt-in to secure coding explicitly by >>>>> implementing >>>>> NSApplicationDelegate.applicationSupportsSecureRestorableState:. >>>>> # >>>>> # A fatal error has been detected by the Java Runtime Environment: >>>>> # >>>>> # SIGILL (0x4) at pc=0x7ff80e07cb73, pid=18502, tid=259 >>>>> # >>>>> # JRE version: OpenJDK Runtime Environment Homebrew (21.0.1) (build >>>>> 21.0.1) >>>>> # Java VM: OpenJDK 64-Bit Server VM Homebrew (21.0.1, mixed mode, >>>>> sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-amd64) >>>>> # Problematic frame: >>>>> # C [AppKit+0xc8fb73] _NSCarbonMenuCrashIfNeeded+0x258 >>>>> # >>>>> # Core dump will be written. Default location: /cores/core.18502 >>>>> # >>>>> # An error report file with more information is saved as: >>>>> # /Users/jonathan/github/Clear/hs_err_pid18502.log >>>>> # >>>>> # If you would like to submit a bug report, please visit: >>>>> # https://github.com/Homebrew/homebrew-core/issues >>>>> # The crash happened outside the Java Virtual Machine in native code. >>>>> # See problematic frame for where to report the bug. >>>>> # >>>>> zsh: abort basexgui >>>>> >>>> >>>> What should I do? >>>> >>>> Jonathan >>>> >>>
