[basex-talk] Want event, not polling....
Hello,
I am using a client written in C#. For the sake of this discussion, imagine the
app being discussed as an app just like the Java GUI app that ships with BaseX:
You can basically author Xquery snippets and send them for execution by the
server.
We regularly have queries that tend to run for a long time, and it would be an
advantage for the user to be able to cancel these long running jobs, just like
he can in the GUI app. However, I am not sure how to do this; I can imagine two
solutions:
1. I see that jobs that are submitted normally to the server end up as jobs,
just like the ones that are submitted with jobs:eval(query): I can stop that
job with jobs:stop(jobId) on another session if the jobId is known. You would
need a separate session that fetches the running jobs and gets the current one
(recognizing it by creating a cookie in a comment in the query) with
jobs:list-details(), then getting the jobId and then calling jobs:stop(jobId).
It is possible I guess, but very heavy handed.
2. The other option is just as bad: submit the query through
jobs:eval(query), getting the jobId and then poll for it to finish; more
polling which will not scale with many users and will not provide the quick
feedback we want.
I see some traces of an eventing mechanism that was deprecated (or at least no
longer included in the client). That is what I would ideally have (ignoring
bindings etc):
Query q = new Query("1 to 10" , new Guid());
q.Ended += MyTerminationHandler(); // C# syntax for adding an event handler for
termination
q.Interrupted += MyInterruptedHandler();
await q.execute();
// Called when the query has terminated with a result
void MyTerminationHandler(Guid g, string result) { ... update UI ... }
// Called when the query has terminated with a result
void MyInterruptedHandler (Guid g) { ... update UI ... }
How do I do this without the polling, being able to pass massive results and
queries back and forth?
Best Regards
Peter Villadsen
Principal Architect
Microsoft Business Applications Group
Re: [basex-talk] how to specify a database in a.xq FLOWR query?
> They were never affiliated with W3C, […] Thanks for the info, Liam! I have added a little comment. Links to helpful resources are always welcome. On Mon, Oct 7, 2019 at 5:39 PM Liam R. E. Quin wrote: > > On Mon, 2019-10-07 at 03:38 -0700, thufir wrote: > > From SO (and the fine manual), the solution is to use: basex -i > > w3school_data titles.xq > > Seeing the filename propts me to note - w3schools (when i've looked at > it) isn't a good place to learn from. They were never affiliated with > W3C, and their tutorials often seem written by people without a deep > understanding. > > Mind you, i don't know many free online resources that are better > alternatives... > > -- > Liam Quin, https://www.delightfulcomputing.com/ > Available for XML/Document/Information Architecture/XSLT/ > XSL/XQuery/Web/Text Processing/A11Y training, work & consulting. > Barefoot Web-slave, antique illustrations: http://www.fromoldbooks.org >
Re: [basex-talk] how to specify a database in a.xq FLOWR query?
On Mon, 2019-10-07 at 03:38 -0700, thufir wrote: > From SO (and the fine manual), the solution is to use: basex -i > w3school_data titles.xq Seeing the filename propts me to note - w3schools (when i've looked at it) isn't a good place to learn from. They were never affiliated with W3C, and their tutorials often seem written by people without a deep understanding. Mind you, i don't know many free online resources that are better alternatives... -- Liam Quin, https://www.delightfulcomputing.com/ Available for XML/Document/Information Architecture/XSLT/ XSL/XQuery/Web/Text Processing/A11Y training, work & consulting. Barefoot Web-slave, antique illustrations: http://www.fromoldbooks.org
Re: [basex-talk] HTTPServer + gzip compression
Hi Christian,thx for your reply. Got it :)Solution 2 would be cool - maybe feature request? But I can live with a workaround.kind regards,Stefan -Ursprüngliche Nachricht-Von: Christian Grün [mailto:christian.gr...@gmail.com]Gesendet: Donnerstag, 3. Oktober 2019 11:51An: Stefan Koch Cc: BaseX Betreff: Re: [basex-talk] HTTPServer + gzip compressionHi Stefan,There’s a StackOverflow entry that has previously been referenced onthis list (by Michael Seiferle, see [2]). Based on this thread, inwhich Joakim Erdfelt (the magnificent Jetty core developer) explainswhy the existing approaches for enabling GZIP compression don’t workanymore, I see three choices:1. to wrap all HTTP responses in a GZIP output stream;2. to initialize GZipHandler in our basexhttp code; or3. enable GZIP compression outside BaseX,Alternative 1 would give us most control, but it would raise newquestions that would need to be solved. Alternative 2 may be thebetter approach: It only works if basexhttp is used, but we couldbenefit from existing optimizations and tweaks from the Jettyimplementation [2]. Alternative 3 is already available: You can useanother light-weight web server as proxy (caddy, nginx), or you canstart Jetty as described by Joakim.> Good idea to ask the Jetty devs - I'll try that.Thanks; feel free to keep us updated,Christian[1] https://stackoverflow.com/questions/38635262/jetty-9-and-gziphandler[2] https://www.eclipse.org/jetty/documentation/current/gzip-filter.html
Re: [basex-talk] how to specify a database in a.xq FLOWR query?
> Not to just return a
> result but write that result back to basex.
Sure! See here [1] (more specifically, [2]).
> thanks,
Welcome.
[1] http://docs.basex.org/
[2] http://docs.basex.org/wiki/Command-Line_Options#Standalone
> On 2019-10-06 11:53 p.m., thufir wrote:
> > how do I specify a database for a .xq file to use?
> >
> > a simple FLOWR:
> >
> > thufir@dur:~/basex/w3$
> > thufir@dur:~/basex/w3$ basex titles.xq
> > [warning] /usr/bin/basex: Unable to locate /usr/share/java/jing.jar in
> > /usr/share/java
> > Learning XML
> > XQuery Kick Startthufir@dur:~/basex/w3$
> > thufir@dur:~/basex/w3$
> > thufir@dur:~/basex/w3$ cat titles.xq
> > for $x in doc("books.xml")/bookstore/book
> > where $x/price>30
> > order by $x/title
> > return $x/title
> > thufir@dur:~/basex/w3$
> >
> > (from w3schools)
> >
> > thanks,
> >
> > Thufir
Re: [basex-talk] how to specify a database in a.xq FLOWR query?
From SO (and the fine manual), the solution is to use: basex -i
w3school_data titles.xq
Very interesting. I'm sure this can be piped or simply written to a
file, but is there a particularly "basex" method? Not to just return a
result but write that result back to basex.
thanks,
Thufir
On 2019-10-06 11:53 p.m., thufir wrote:
how do I specify a database for a .xq file to use?
a simple FLOWR:
thufir@dur:~/basex/w3$
thufir@dur:~/basex/w3$ basex titles.xq
[warning] /usr/bin/basex: Unable to locate /usr/share/java/jing.jar in
/usr/share/java
Learning XML
XQuery Kick Startthufir@dur:~/basex/w3$
thufir@dur:~/basex/w3$
thufir@dur:~/basex/w3$ cat titles.xq
for $x in doc("books.xml")/bookstore/book
where $x/price>30
order by $x/title
return $x/title
thufir@dur:~/basex/w3$
(from w3schools)
thanks,
Thufir
Re: [basex-talk] // versus /*/
Thanks, Liam. No, not homework, just me futzing about. I'll experiment a bit more -- and thanks for the suggestion. -Thufir On 2019-10-07 12:25 a.m., Liam R. E. Quin wrote: On Sun, 2019-10-06 at 21:28 -0700, thufir wrote: Do these have the same meaning? Might there be a subtle distinction, or might they be read differently but functionally identical? Are we doing your homework? :-) :-) //* is the same as /descendant-or-self::* //book means, search the whole database to find "book" elements. /*/book meeans make a list of all children of the top-level node, and find book elements that are children of items in that list. So, given //book will find one node, and /*/book won't find any. They're equally efficient, at least as used above? They are doing different things. To measure efficiency, use a much larger database than the XQuery use cases example :) You may find Priscilla Walmsley's XQuery book helpful in learning XPath version 3. Best, Liam
Re: [basex-talk] Migrating ~ 3M record db from BaseX to PostgreSQL results in OOM
I'm currently at work and my setup is at home. In about 7 hours I'll get home and I will send the stack trace. Meanwhile, is there any way to write a FLWOR, a loop, in a batched style? Like for example in my case, this approach I described to migrate data from BaseX to PostgreSQL makes use of BaseX as an XQuery processor and transfers the full-text indexing to PostgreSQL, this is what I'm trying to do. However, in order to avoid OOM, I am thinking of batching the transfer into chunks, and potentially restart the BaseX server in between the migration of each chunk. That's why I am asking how I could do that in BaseX. My hope is that the OOM could be avoided in this way, because not all the data would pass through main memory and there would be less chances of the JVM GC having to deal with this data. Restarting the BaseX server between each chunk transfer would help making sure that whatever memory was used is released. So I wonder if something like ( to ] would work here. Of course, some count would have to be done beforehand to know how many batches there will be. Or maybe even without knowing how many batches there will be, a while-type loop could be written in Bash with the stop conditon being to check if the current chunk is empty. Would an approach like this work to mitigate the OOM? Are there alternatives or work-arounds to this kind of OOM? Thanks On Mon, Oct 7, 2019, 1:13 AM Christian Grün wrote: > Some complementary notes (others may be able to tell you more about their > experiences with large data sets): > > a GiST index would have to be built there, to allow full-text searches; >> PostgreSQL is picked >> > > You could as well have a look at Elasticsearch or its predecessors. > > there might be a leak in the BaseX implementation of XQuery. >> > > I assume you are referring to the SQL Module? Feel free to attach the OOM > stack trace, it might give us more insight. > > I would recommend you to write SQL commands or an SQL dump to disk (see > the BaseX File Module for now information) and run/import this file in a > second step; this is probably faster than sending hundreds of thousands of > single SQL commands via JDBC, no matter if you are using XQuery or Java. > > > > >
[basex-talk] Unidentified java processes keeping locks on DBs (BaseX 9.2.3)
Hi, We have been getting process issues on Windows installations for a while now. Has anyone had fantom locks (non-existent processes locking indices) and if so, can you point to clues that could help resolve the issue? [image: image.png] This leads to batch processes failure and DB corruption so we are eager even for the tiniest clues. Thank you -- France Baril Architecte documentaire / Documentation architect france.ba...@architextus.com
Re: [basex-talk] // versus /*/
On Sun, 2019-10-06 at 21:28 -0700, thufir wrote: > Do these have the same meaning? Might there be a subtle distinction, > or > might they be read differently but functionally identical? Are we doing your homework? :-) :-) //* is the same as /descendant-or-self::* //book means, search the whole database to find "book" elements. /*/book meeans make a list of all children of the top-level node, and find book elements that are children of items in that list. So, given //book will find one node, and /*/book won't find any. > They're equally efficient, at least as used above? They are doing different things. To measure efficiency, use a much larger database than the XQuery use cases example :) You may find Priscilla Walmsley's XQuery book helpful in learning XPath version 3. Best, Liam -- Liam Quin, https://www.delightfulcomputing.com/ Available for XML/Document/Information Architecture/XSLT/ XSL/XQuery/Web/Text Processing/A11Y training, work & consulting. Barefoot Web-slave, antique illustrations: http://www.fromoldbooks.org
[basex-talk] how to specify a database in a.xq FLOWR query?
how do I specify a database for a .xq file to use?
a simple FLOWR:
thufir@dur:~/basex/w3$
thufir@dur:~/basex/w3$ basex titles.xq
[warning] /usr/bin/basex: Unable to locate /usr/share/java/jing.jar in
/usr/share/java
Learning XML
XQuery Kick Startthufir@dur:~/basex/w3$
thufir@dur:~/basex/w3$
thufir@dur:~/basex/w3$ cat titles.xq
for $x in doc("books.xml")/bookstore/book
where $x/price>30
order by $x/title
return $x/title
thufir@dur:~/basex/w3$
(from w3schools)
thanks,
Thufir
