Re: [basex-talk] Hitting the Tail Recursion wall
Marco Lettere wrote onm 01.04.2019 at 18:01:
> declare function local:topath($path){ let $pathseg := tokenize($path,
> "/") return fold-left($pathseg, (), function($out, $segment){
> if($segment = "." or $segment = "") then $out else if($segment =
> "..") then $out[position() lt count($out)] else ($out,
> $segment) })}; local:topath("/a/b/c/../../../g")
Beautiful and very close, except for a minor caveat: the "/" are needed,
to reconstruct the absolute path-part from the relative.
I played around and tried to place some "/" to your function, but all
variants placed some "/" wrong or twice.
Last but not least: You asked for the use case, which is described on
https://tools.ietf.org/html/rfc3986#section-5.2.4 (note also the
sequences shown after the description of the steps)
5.2.4 .
Remove Dot Segments
The pseudocode also refers to a "remove_dot_segments" routine for
interpreting and removing the special "." and ".." complete path
segments from a referenced path. This is done after the path is
extracted from a reference, whether or not the path was relative, in
order to remove any invalid or extraneous dot-segments prior to
forming the target URI. Although there are many ways to accomplish
this removal process, we describe a simple method using two string
buffers.
1. The input buffer is initialized with the now-appended path
components and the output buffer is initialized to the empty
string.
2. While the input buffer is not empty, loop as follows:
A. If the input buffer begins with a prefix of "../" or "./",
then remove that prefix from the input buffer; otherwise,
B. if the input buffer begins with a prefix of "/./" or "/.",
where "." is a complete path segment, then replace that
prefix with "/" in the input buffer; otherwise,
C. if the input buffer begins with a prefix of "/../" or "/..",
where ".." is a complete path segment, then replace that
prefix with "/" in the input buffer and remove the last
segment and its preceding "/" (if any) from the output
buffer; otherwise,
D. if the input buffer consists only of "." or "..", then remove
that from the input buffer; otherwise,
E. move the first path segment in the input buffer to the end of
the output buffer, including the initial "/" character (if
any) and any subsequent characters up to, but not including,
the next "/" character or the end of the input buffer.
3. Finally, the output buffer is returned as the result of
remove_dot_segments.
Note that dot-segments are intended for use in URI references to
express an identifier relative to the hierarchy of names in the base
URI. The remove_dot_segments algorithm respects that hierarchy by
removing extra dot-segments rather than treat them as an error or
leaving them to be misinterpreted by dereference implementations.
The following illustrates how the above steps are applied for two
examples of merged paths, showing the state of the two buffers after
each step.
STEP OUTPUT BUFFER INPUT BUFFER
1 : /a/b/c/./../../g
2E: /a/b/c/./../../g
2E: /a/b /c/./../../g
2E: /a/b/c/./../../g
2B: /a/b/c/../../g
2C: /a/b /../g
2C: /a/g
2E: /a/g
STEP OUTPUT BUFFER INPUT BUFFER
1 :
mid/content=5/../6
2E: mid /content=5/../6
2E: mid/content=5 /../6
2C: mid /6
2E: mid/6
--
Goody Bye, Minden jót, Mit freundlichen Grüßen,
Andreas Mixich
Re: [basex-talk] Hitting the Tail Recursion wall
Hi Andreas,
I don't know whether I correctly understood you use-case but what about
going with hof functions [1]?
Maybe your code could turn to something as simple as
declare function local:topath($path){
let $pathseg := tokenize($path, "/")
return
fold-left($pathseg, (), function($out, $segment){
if($segment = "." or $segment = "") then $out
else if($segment = "..") then $out[position() lt count($out)]
else ($out, $segment)
})
};
local:topath("/a/b/c/../../../g")
Regards,
Marco.
[1] http://docs.basex.org/wiki/Higher-Order_Functions
On 01/04/19 15:49, Andreas Mixich wrote:
I think, I am finding it...
I replaced the first expression with
`prof:dump(concat("in: ", $path, " out: ",
string-join(array:flatten($out,`
and this gives me more info. So I may be able to solve this alone.
Re: [basex-talk] Using local xqj
Thanks for your reply, Christian. I purged my system of all things BaseX, and reloaded, based on 9.1.2 in fact. This time I was able to build and run with it. There were error messages about code in our existing xquery files, but that was what I was trying to find out: could I just swap one solution (DataDirect) out and another one in? The answer is no, our xquery files are not clean enough. Arthur Arthur Gardner | Infosys Ltd MetLife | Clarks Summit, PA Office 570-587-6898 | Mobile 262-442-7472 -Original Message- From: Christian Grün Sent: Monday, April 01, 2019 7:09 AM To: Gardner, Arthur Cc: basex-talk@mailman.uni-konstanz.de Subject: [EXT] Re: [basex-talk] Using local xqj Hi Arthur, The error message could indicate that you are using an outdated version of XQJ. Are you working with the XQJ libraries that are contained in the lib directory of BaseX? > Hi! I’ve downloaded BaseX 9.2 (Windows distro). As the latest official release is 9.1.1… Is it the snapshot that you have downloaded? > java.lang.NoClassDefFoundError: org/basex/core/Proc Could you please pass us on your Java code? Best, Christian The information contained in this message may be CONFIDENTIAL and is for the intended addressee only. Any unauthorized use, dissemination of the information, or copying of this message is prohibited. If you are not the intended addressee, please notify the sender immediately and delete this message.
Re: [basex-talk] Quick Windows issue: Invalid reply on port 1984, works fine on port 8984
Hey Jim, Thanks for sharing your startup experiences with BaseX. As reported back in GitHub, the issues with the Windows installers should be solved now. They have been triggered by the new variety of open source distributions for Open JDK (which are the result of Oracle’s stricter licensing policies). In the latest version, we got rid of our launch4j exe wrapper, and we are now directly linking the BaseX .bat startup scripts. The stack trace you observed was probably caused by the intent to connect to 1984 with HTTP requests. I have revised our code; the corresponding client will only be closed now if authentication was successful. > BTW, it is the case that when you do the basexhttp.bat script on the > command-line without the '-S' option, the response messages streamed to the > command line include a message that the server on 1984 started along with the > http server on 8984. When you invoke this script with the '-S' option, only a > short message informs you that the server on 8984 has started. Perhaps it > would be possible to return the message that the server on port 1984 started, > too? You’re right; resolved as well in the very latest snapshot. Cheers Christian > Oops... In my prior message I meant that the basexserver.bat script only > starts the service on port 1984 not 8984. > > Sorry for the unseen error. BTW, it is the case that when you do the > basexhttp.bat script on the command-line without the '-S' option, the > response messages streamed to the command line include a message that the > server on 1984 started along with the http server on 8984. When you invoke > this script with the '-S' option, only a short message informs you that the > server on 8984 has started. Perhaps it would be possible to return the > message that the server on port 1984 started, too? > > -: Jim :- > > -Original Message- > From: BaseX-Talk On Behalf Of > Jim Salmons > Sent: Friday, March 29, 2019 1:16 PM > To: 'Marco Lettere' ; basex-talk@mailman.uni-konstanz.de > Subject: Re: [basex-talk] Quick Windows issue: Invalid reply on port 1984, > works fine on port 8984 > > Hi Marco! > > Your tip put me on the right track... I have a handle on how things work now > and, so, I'll share my understanding in hope that it helps another nooby dev, > especially if under Windows... > > CONTEXT: I am pretty sure that my confusion came from working through the > Startup page of the Getting Started section of the wiki, > http://docs.basex.org/wiki/Startup. There are a number of different modes for > running BaseX and each page section gives a tip bullet list of how to invoke > these modes. Working through the page, both by double-clicking Windows > desktop icons and issuing batch commands in command prompt and Powershell > windows, I was inadvertently stopping and starting various combinations of > server, http server, standalone, etc. And since there were no directly > labeled processes or services that said anything about BaseX, I was in the > dark about what was starting, stopping, and running when inspecting Windows' > Task Manager. > > Given Marco's tip and rereading the bullet points of the wiki's Startup page, > I now understand "who's on first" in terms of what's running and serving > which ports. My confusion on Windows was that when you issue the > basexserver.bat command at a command prompt, you only start the server:8984 > while the "BaseX Sever (Start)" icon on Windows runs the basexhttp.bat script > with the '-S' option thus starting BOTH the basic and the http servers > (although the command-line messages only mention the HTTP server starting on > 8984). > > I had so many things starting and stopping hither and yon that I was my own > worst enemy in terms of entry learning. > > I now have my "sea legs" and am plowing into the Python API so I can start to > integrate BaseX with my research toolkit. > > Thanks, Marco, for the clarifying tip. :-) > > Happy-Healthy Vibes from Colorado, > -: Jim :- > > -Original Message- > From: BaseX-Talk On Behalf Of > Marco Lettere > Sent: Friday, March 29, 2019 1:56 AM > To: basex-talk@mailman.uni-konstanz.de > Subject: Re: [basex-talk] Quick Windows issue: Invalid reply on port 1984, > works fine on port 8984 > > Hi Jim, > > I think port 1984 is not meant to be connected to over HTTP. It's a pure > socket connection that ships BaseX internal client-server protocol. > When you connect to that port you probably have to use the Python client API > in order to get your BaseX COMMANDS (not HTTP requests) serialized over the > wire. > > Hope this helps [cit.]. > > Regards, > M. > > On 28/03/19 23:28, Jim Salmons wrote: > > Hello BaseX folk, > > > > I have just installed version 9.1.2 and am starting to learn BaseX in > > support of my post-cancer #PayItForward Bonus Round #CitizenScientist > > research in #DigitalHumanities and #MachineLearning. My goal is to > > "cut out the middleman" of writi
Re: [basex-talk] Hitting the Tail Recursion wall
I think, I am finding it...
I replaced the first expression with
`prof:dump(concat("in: ", $path, " out: ",
string-join(array:flatten($out,`
and this gives me more info. So I may be able to solve this alone.
--
Goody Bye, Minden jót, Mit freundlichen Grüßen,
Andreas Mixich
[basex-talk] Hitting the Tail Recursion wall
Hi,
I am trying to implement the algorithm outlined at
https://tools.ietf.org/html/rfc3986#section-5.2.4
(Two other implementations are at [1] and [2])
The following function is my work in progress. It seems, all goes fine,
till the last `else`,
which is commented with `(: E :)`. I hit the tail recursion barrier and
I do not see why.
The function may not (yet) make sense, since I am still experimenting.
`local:remove-dot-segments("/a/b/c/./../../g",[])`
```
(:~
Remove dot segments from a URI path component according to RFC 3986
Section 5.2.4
@param $path the path component of an URL
@param $out an empty (!) array
@return the path component with the relative part (dot segments)
removed and
resolved to the absolute path.
@see https://tools.ietf.org/html/rfc3986#section-5.2.4
:)
declare function local:remove-dot-segments(
$path as xs:string,
$out as array(*))
as item()*
{
concat("in: ", $path, " out: ", string-join(array:flatten($out))),
if (not(contains($path, ".")))
then $path
else
if (string-length($path) > 0)
then (: A :)
if (substring($path,1,3) = "../")
then local:remove-dot-segments(substring($path, 4), $out)
else
if (substring($path, 1, 2) = "./")
then local:remove-dot-segments(substring($path, 3), $out)
else (: B :)
if (substring($path,1,3) = "/./")
then local:remove-dot-segments(concat("/", substring($path,
4)), $out)
else
if (substring($path, 1, 2) = "/.")
then local:remove-dot-segments(concat("/", substring($path,
3)), $out)
else (: C :)
if (substring($path, 1, 4) = "/../")
then
let $ret := array:remove($out, array:size($out))
return local:remove-dot-segments(concat("/",
substring($path, 5)), $ret)
else
if (substring($path, 1, 3) = "/..")
then
let $ret := array:remove($out, array:size($out))
return local:remove-dot-segments(concat("/",
substring($path, 4)), $ret)
else (: D :)
if ($path = ".." or $path = ".")
then local:remove-dot-segments(concat(substring((),
4), ""), $out)
else (: E :)
let $ret :=
if (starts-with($path, "/"))
then array:append($out, concat("/",
substring-before(substring($path,2), "/")))
else array:append($out,
substring-before(substring($path,1), "/"))
return local:remove-dot-segments($path, $ret)
else $path
};
```
Sorry for throwing such a big function at you, but I do not find the
place, where the stack overflow gets triggered. It does not happen, if I
return an empty sequence at `else (: E :)`. If I replace that part
(after `else (: E :)`) with `local:remove-dot-segments($path, $out)` I
get the stack overflow as well, which is, what confuses me the most.
[1]https://github.com/ariutta/remove-dot-segments/blob/master/index.js
[2] https://gist.github.com/rdlowrey/5f56cc540099de9d5006
--
Goody Bye, Minden jót, Mit freundlichen Grüßen,
Andreas Mixich
Re: [basex-talk] Using local xqj
Hi Arthur, The error message could indicate that you are using an outdated version of XQJ. Are you working with the XQJ libraries that are contained in the lib directory of BaseX? > Hi! I’ve downloaded BaseX 9.2 (Windows distro). As the latest official release is 9.1.1… Is it the snapshot that you have downloaded? > java.lang.NoClassDefFoundError: org/basex/core/Proc Could you please pass us on your Java code? Best, Christian
