Re: [basex-talk] Interrupted queries - partial results?
Thanks Christian!
Patrick
On 4/20/22 03:58, Christian Grün wrote:
Is it possible to capture partial results from an interrupted query?
Thanks for the suggestion, Patrick; it’s not the first time we got this request.
If you use BaseX on command line, results will be returned as soon as
they are available. In the GUI, all results are cached before they are
presented to the user. Maybe we can pass on the incomplete cache to
the result view if a query is interrupted; I’ll have more thoughts on
that.
As another alternative to Daniel’s solution, prof:dump and fn:trace
can be used to output results at runtime to the Info View:
for $i in 1 to 10
where $i mod 1000 = 0
return trace($i)
On Wed, Apr 20, 2022 at 1:45 AM Patrick Durusau wrote:
Thanks!
That should do it!
Patrick
On 4/19/22 11:40, Zimmel, Daniel wrote:
How about some custom logging?
for $i in (1 to 10)
return
if ($i = 4) then error()
else file:append-text-lines('C:\tmp\file',string($i))
There might be more sophisticated ways.
Daniel
-Ursprüngliche Nachricht-
Von: BaseX-Talk Im Auftrag von
Patrick Durusau
Gesendet: Dienstag, 19. April 2022 16:45
An: basex-talk@mailman.uni-konstanz.de
Betreff: [basex-talk] Interrupted queries - partial results?
Greetings!
Is it possible to capture partial results from an interrupted query?
When I realize a query is taking too long for the data involved, it might be
helpful in correcting the mistake.
Thanks!
Patrick
--
Patrick Durusau
patr...@durusau.net
Technical Advisory Board, OASIS (TAB)
Editor, OpenDocument Format TC (OASIS), Project Editor ISO/IEC 26300 Co-Editor,
ISO/IEC 13250-1, 13250-5 (Topic Maps)
Another Word For It (blog): http://tm.durusau.net
Homepage: http://www.durusau.net
Twitter: patrickDurusau
--
Patrick Durusau
patr...@durusau.net
Technical Advisory Board, OASIS (TAB)
Editor, OpenDocument Format TC (OASIS), Project Editor ISO/IEC 26300
Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps)
Another Word For It (blog): http://tm.durusau.net
Homepage: http://www.durusau.net
Twitter: patrickDurusau
--
Patrick Durusau
patr...@durusau.net
Technical Advisory Board, OASIS (TAB)
Editor, OpenDocument Format TC (OASIS), Project Editor ISO/IEC 26300
Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps)
Another Word For It (blog): http://tm.durusau.net
Homepage: http://www.durusau.net
Twitter: patrickDurusau
OpenPGP_signature
Description: OpenPGP digital signature
Re: [basex-talk] Using error and catch for error paths in REST API endpoint code
Hi Omar,
Thanks for this.
I also enjoyed your earlier message re:
>I tried to get creative at finding a way to optimize querying the data
without having many long lasting global lock situations
I have been in a similar situation and discovered how alternative code
approaches using different builtin functions can have very different lock
profiles.
I am hopeful that #[2063] will greatly reduce locking issues.
>Is there any sane way to get a QName with an unknown prefix of an error as
a string like in the catch all handler and resolve it against
all prefix-URI mappings known in some XQuery program?
Maybe using inspect:xqdoc can help to get the prefix to namespace map. e.g.
inspect:xqdoc("
https://raw.githubusercontent.com/acdh-oeaw/vicav-app/master/http.xqm
")/*:namespaces
http://www.xqdoc.org/1.0;>
https://tools.ietf.org/html/rfc7807"/>
http://exquery.org/ns/request"/>
https://www.oeaw.ac.at/acdh/tools/vle/cors"/>
http://basex.org/modules/admin"/>
https://tools.ietf.org/html/rfc7231#section-6"/>
http://exquery.org/ns/restxq"/>
http://expath.org/ns/http-client"/>
http://www.w3.org/2012/xquery"/>
Although this can run into issues like #[1194]. For example trying to do
all modules...
(: hack to scrape module names from github :)
let $mods:=fetch:xml("
https://github.com/acdh-oeaw/vleserver_basex/tree/main/vleserver
",map{"parser":"html"})
//a[@data-pjax="#repo-content-pjax-container"][ends-with(.,".xqm")]
/concat("https://raw.githubusercontent.com",replace(@href,"/blob/","/"))
(: get namespace prefixes in use :)
return $mods!( try{ inspect:xqdoc(.)/*:namespaces } catch *
{$err:description})
I feel the solution to this must be related to the scoping of names, and in
some way depend on #[2048]
/Andy
[2063] https://github.com/BaseXdb/basex/issues/2063
[1194] https://github.com/BaseXdb/basex/issues/1194
[2048] https://github.com/BaseXdb/basex/issues/2048
On Tue, 12 Apr 2022 at 18:58, Omar Siam wrote:
> Way too often I saw myself and my colleagues write huge, very hard to
> understand deeply nested if () then else code to handle any control path
> but the complete success one in RestXQ code. [1] is an example of such
> code even though it uses the module I want to introduce here. It seems I
> didn’t have time to refactor it yet.
> To add insult to injury the code producing a 500, 404, 403, 401, 302 or
> maybe even 201 response code was
> * neither short nor very uniform and
> * would respond to the browser with XML/XHTML or JSON or text but in
> most situations the format the browser side had the hardest time to handle
>
> For example [2] delivers an XHTML message no matter what the Accept
> header says and although it is an accurate message to a user, very
> similar XML snippets are found throughout that unit and maybe need to be
> adapted each separately if the HTML needs to updated.
>
> My XML snippets CRUD API started as a port of an apigility (now laminas
> api-tools [3]) API working with a relational backend to store XML
> snippets to something that does the same but just uses BaseX for any
> data storage and is much better at querying using sophisticated XPaths.
> So from that previous API I learned about RFC 7807. [4]
>
> RFC 7807 “Problem Details for HTTP APIs” [5] is one of several
> specification now available for reporting errors from REST APIs. This
> specification explicitly states how the errors should look like in XML
> as well as in JSON.
>
> Maybe this is a bit bold, but I used the URL of that very RFC 7807 as
> the resolved URI for my module. [6]
> Please note: I have to admit my modules have something unusual in
> common: I use the namespace prefix “_” within the module and only use a
> more talking prefix when I use a module elsewhere. So, the “_” prefix
> can map to numerous URIs in my code. I am not 100% sure if there are
> down sides to this style but I use it for a while now and no problems
> come to mind.
>
> I really dislike to get an error, especially during development of some
> service, without any indication of where that actually occurred. That is
> to say: I like stack traces in my errors.
> It also would be great if any runtime error in my code would be reported
> as XML or JSON, depending on the format the browser asked for, just as
> errors I explicitly raise.
>
> A few parts provided by BaseX greatly help in getting all of this
> packaged in some xqm-file.
> * A stack trace is always available in “$err:additional” when catching
> errors [7]
> * One catch-all error handler can be installed (“declare
> %rest:error('*') function”, although there are minor downsides to this
> catch-all handler) [8]
> * The XML based direct format BaseX uses to store JSON by default makes
> it very easy to transform RFC 7807 XML to JSON [9]
> * It is easy to query the request header anywhere in RestXQ XQuery code
> running on BaseX [10]
>
> I tried to have easy to remember function names that make the code
> readable as if it was a sentence. Therefore for example,
Re: [basex-talk] Interrupted queries - partial results?
> > Is it possible to capture partial results from an interrupted query?
Thanks for the suggestion, Patrick; it’s not the first time we got this request.
If you use BaseX on command line, results will be returned as soon as
they are available. In the GUI, all results are cached before they are
presented to the user. Maybe we can pass on the incomplete cache to
the result view if a query is interrupted; I’ll have more thoughts on
that.
As another alternative to Daniel’s solution, prof:dump and fn:trace
can be used to output results at runtime to the Info View:
for $i in 1 to 10
where $i mod 1000 = 0
return trace($i)
On Wed, Apr 20, 2022 at 1:45 AM Patrick Durusau wrote:
>
> Thanks!
>
> That should do it!
>
> Patrick
>
> On 4/19/22 11:40, Zimmel, Daniel wrote:
> > How about some custom logging?
> >
> > for $i in (1 to 10)
> > return
> >if ($i = 4) then error()
> >else file:append-text-lines('C:\tmp\file',string($i))
> >
> > There might be more sophisticated ways.
> >
> > Daniel
> >
> > -Ursprüngliche Nachricht-
> > Von: BaseX-Talk Im Auftrag von
> > Patrick Durusau
> > Gesendet: Dienstag, 19. April 2022 16:45
> > An: basex-talk@mailman.uni-konstanz.de
> > Betreff: [basex-talk] Interrupted queries - partial results?
> >
> > Greetings!
> >
> > Is it possible to capture partial results from an interrupted query?
> >
> > When I realize a query is taking too long for the data involved, it might
> > be helpful in correcting the mistake.
> >
> > Thanks!
> >
> > Patrick
> >
> > --
> > Patrick Durusau
> > patr...@durusau.net
> > Technical Advisory Board, OASIS (TAB)
> > Editor, OpenDocument Format TC (OASIS), Project Editor ISO/IEC 26300
> > Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps)
> >
> > Another Word For It (blog): http://tm.durusau.net
> > Homepage: http://www.durusau.net
> > Twitter: patrickDurusau
> >
> --
> Patrick Durusau
> patr...@durusau.net
> Technical Advisory Board, OASIS (TAB)
> Editor, OpenDocument Format TC (OASIS), Project Editor ISO/IEC 26300
> Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps)
>
> Another Word For It (blog): http://tm.durusau.net
> Homepage: http://www.durusau.net
> Twitter: patrickDurusau
>
