Hello Rony,
I sent a previous reply, but it was intercepted by the SourceForge filters because it contained a
zip file attachment. SourceForge informed me about this fact only some few hours ago (sigh). I
will take the opportunity to compose a new, more thoughtful, reply.
Missatge de Rony G. Flatscher <rony.flatsc...@wu.ac.at> del dia dg., 5 de febr.
2023 a les 19:40:
(snip)
Not being the expert in this corner a question to you as you have
developed a deep
insight in the meantime: while developing on json.cls there are two
json.cls present at
the moment (Windows layout):
C:\Program Files\ooRexx\json.cls
C:\Program Files\ooRexx\rexxtry.rex
F:\work\svn\oorexx\sandbox\rony\json\json.cls
current directory is: F:\work\svn\oorexx\sandbox\rony\json
Now while doing interactive tests via rexxtry.rex (i.e. "C:\Program
Files\ooRexx\rexxtry.rex") doing a "call json.cls" will find and call
"C:\Program
Files\ooRexx\json.cls" and *not* "json.cls" in the current directory!
Sure :) Rexxtry.rex is in the "C:\Program Files\ooRexx" directory. This is the
"same"
directory of the docs, or the "parent" directory of the source (I'd prefer
to call it the
"caller's" directory, I think it's a more accurate description; but I disgress).
"Call
json.cls" is one of the cases where Rexx works as documented. And the
search order begins...
by the "same" directory, as per the docs. That's why the version in "C:" is
called.
However, in the same rexxtry.rex session in the current directory doing a
"call
.\json.cls" *will* resolve and call
"F:\work\svn\oorexx\sandbox\rony\json\json.cls": so
"." will force using the current directory for finding external Rexx
programs.
That's exactly our bug.
Hmm, maybe it is not a bug after all, but working as intended as this
allows for overriding
the default search order that starts out in the source directory (the
directory the Rexx
script got loaded and run from as in the case of rexxtry.rex, doing a
"parse source . .
path2source" would denote the Rexx program's source directory.)
I would object to that. If it were working, as you say, "as intended", then it should have been
documented. But it is not. My impression is that /this is just a trick that works and that we are
used to this trick/, to a point that we find it natural.
The search order algorithm, by definition, imposes a certain /opacity/ when there are duplicates.
A super-path list is formed by concatenating the caller's directory, the present directory, the
application-defined extra path, and the contents of REXX_PATH and PATH. Let p_1, ..., p_n be the
component paths of this super-path list. Assume that in the search for name.ext we have several i
in [1..n] such that p_i || sep || name.ext exists (sometimes it's a little more complicated than a
simple concatenation, but allow me, for the sake of simplicity). Let i_1, ..., i_k be these
indexes, where i_1 < i_2 <...<i_k. Then the fact that we find name.ext in p_{i_1} /turns all the
other occurrences of /name.ext (namely, p_j || sep || name.ext, for j in [2..n]) /opaque/ /to the
search algorithm/, and consequently also to ourselves as users of RexxTry or of a trace instruction.
We cannot expect that there is a special "trick" to refer to any of the p_{i_2}, ..., p_{i_j}. The
fact that "." works /is a consequence of the malfunctioning of the search algorithm, not an escape
mechanism that works as intended/.
/Unless/ we first /define/ it to be so, which we have not done.
If I understand you properly, the only difference between this last test
and the previous one
is the addition of ".\". This is, precisely, one of the cases where what
Rexx does and what's
in the docs /differ/. ".\" should mean "in /this/ directory". But what does
"this" mean,
i.e., "this" relative to what?
Probably "the current directory".
If you were using the command line, you'd be right. For a programming language, it's not so
obvious. In any case, it's something that has to be (1) defined and (2) documented.
Let's see what other languages do.
The gcc compiler, for example, takes "." to mean "each and every directory specified with the -I"
compiler option. Then, for example, include "./this.h" will refer to whatever directories have
been specified. Each and every of them. "." is relative. I can provide testfiles if you want them
(but not in a zipfile, as I've had to learn).
#include <stdio.h>
#include "./inc.h"
int main() {
printf("%s\n", THIS);
return 0;
}
All that main does is to print THIS, which obviously has to be defined somewhere else, i.e., in
./inc.h. Now, what does ./inc.h refer to? Well, it depends on what do we specify in the
-I compiler option. Let's assume that we have two subdirectories called one and two. Now after
compiling with
gcc -Ione -o main main.c,
if we type "./main", we will get
one,
assuming of course that inc.h in one #defines THIS as "one"; and after
compiling with
gcc -Itwo -o main main.c,
we will get
two
assuming, once more, the necessary and symmetrical #defines. That's what (a compiler for) a
compiled language does.
Let's see, on the other hand, what are the path calculations performed by Python, a dynamic,
interpreted language:
>>> os.getcwd()
'D:\\Dropbox'
>>> (Path("C:") / "./int.rex").resolve()
WindowsPath('C:/Users/jmblasco/int.rex')
That is, "./", relative to "C:", is "C:/Users/jmblasco/int.rex", something relative to /the
current directory of the C: drive/ (which happened to be C:\users\jmblasco), not something
relative to the current directory (which resides in the D: drive).
Well, according to the docs, we have to respect the search order. And, in
the first place...
"this" should be relative to the "same" directory, that is, the very same
"same" directory
(sorry). That's the caller's directory, that is, where rexxtry resides.
BUT... here's this
bug we are talking about: since the callee's filespec begins with ".\", the
search path is
bypassed (it should not be!). This means that the filespec is resolved
(more or less) like in
the command line (to be true, the resolution mechanism provided by the
SearchPath Windows API
--what Rexx for Windows uses-- differs from the command line one, with the
clear intention to
get the programmers and users altogether completely braindamaged, I'm
obliged to presume).
But when you're in the command line (and, in this case, when you call SearchPath),
"." means
the /current/ directory, not the /same/ or caller's directory. Hence, the
F: version of
json.cls gets called.
This was actually the intention when doing a 'call ".\json.cls"', to override
"json.cls" in
rexxtry.rex' source directory. What would you suggest to use instead, a
fully qualified path
to ".\json.cls" (which one could hardly hard code in advance)? Maybe I have
myself not fully
understood all the ramifications of what you suggest, or why you would
regard 'call
".\json.cls"' resolving the file in the current directory would be an error
(for Rexx that is).
Well I would have called (temporarily) the new version "jsonnew", or "json2", or some such. Or I
would have moved it somewhere else, for example to the parent directory relative to the old
version, and then used call ..\json.cls. Or to a subdirectory of the same, and then used call
subdir\json.cls, and so on. As I've said, the expectation that "." refers to the current directory
is caused by the current behaviour of the interpreter, and is not according to the documentation.
Both gcc and python work in the following way: let pathlist = p_1,...,p_n be a path list, and let
fn = "path\name.ext" be a relative or absolute path specification. Then
Search(pathlist,fn) = fn, when fn is absolute, and
Search(pathlist,fn) = the first (p_i * fn) such that exists,or
Search(pathlist,fn) = .nil otherwise.
Now the "*" operation is, in general, a simple concatenation (adding a path separator if needed),
i.e., "." * "name.ext" = ".\name.ext", where "." refers to the current directory, "C:\some\dir" *
"..\name.ext" is "C:\some\dir\..\name.ext", that is "C:\some\name.ext", regardless of whether "C:"
is the current drive or not, and, similarly, "C:\some\dir" * ".\name.ext" =
"C:\some\dir\.\name.ext", that is, "C:\some\dir\name.ext", where the "." fragment operates as a no-op.
In some cases, "*" is not a direct concatenation, but a concatenation of the more composable parts
of p_i and fn. For example, in Python, (Path("C:/one") / "/two/a.file").resolve() =
WindowsPath('C:/two/a.file').
There are, of course, other algorithms; /they are all more complicated to describe than this one/,
more full of exceptions, more difficult to understand and memorize, and in this respect, more
error-prone. They also tend to exhibit subtle differences between them. For example, the
SearchPath Windows API is more stringent than the Windows CLI. My claim is that the algorithm I am
referring to is (1) the simplest non-banal one, and, in this sense, the easiest to remember for a
programmer, and (2) the algorithm used in very well-known languages, which would help to lower the
astonishment factor (see Mike's least astonishment principle).
Please note that the bug surfaced in the ".." case, not in the "." one, and applies also to the
slash-relative and drive-relative paths. If the bug had existed only in the "." case, I'd agree
that, being "." in the middle of a path a no-op, one could well /decide/ that it would mean "the
current directory" when used as a prefix (but then one would also have to /document that
decision/). But, in the ".." case, it seems obvious that the fact that the caller's directory is
bypassed (and that all the other directories are also bypassed, except the current one) is a bug.
And it would be very difficult to justify handling the ".." case differently than the
"." one.
If you had instead tried "Call C:json.cls", you'd have had the C: version
called. This is
maybe the easiest way not to get lost: "Call C:json.cls", and "Call
F:json.cls". Of course,
the current directory of C: /and/ of F: have to point to the right places.
So it seems that removing "." may have side effects that are not (yet)
covered by the
test cases?
To the contrary: if you had tried the Ubuntu version of the interpreter
after having applied
my patch (https://sourceforge.net/p/oorexx/bugs/1865/?limit=25#687f), then "Call
json.cls"
and "Call ./json.cls" would have had the same effect. As one would expect,
both would have
called the version in the /same /directory. To call the version in the
/current/ directory
without using a complete, absolute path, you'd have had to resort to
F:json.cls or the like :)
If possible the lookup should work the same on Unix and Windows such that a
Rexx programmer
would not have to worry on which platform the program gets run.
Yes, of course, I agree completely with you. As I said above, I don't think one can maintain the
expectation that there is a shortcut to overcome the opacity caused by the search algorithm.
One could devise a "full" search algorithm that returned, in a collection, all the possible
"hits". But that's not what we have, at present.
You can, of course, always resort to Call (Directory()"\json.cls"). Or even create a variable V
for this value and then Call (V).
Personally I never use relative file paths if not absolutely necessary (and
very rarely so,
hence being more than rusty by now!). Rather than using relative paths in
the Rexx programs I
would adjust the PATH environment variable such that the desired search
order is reflected via
it and then start the Rexx program in that environment.
Well, I think there are cases when the use of upwards-relative paths (what initiated our
conversation) is very reasonable. For example, when you have a master.cls somewhere, and you want
to store all their subclasses in a subdirectory in a system- and path- independent way, it's very
convenient to be able to write :.requires '../master.cls'.
I was using such constructions under Apache under Ubuntu 20.04 and Rexx 4.2.0. When I migrated to
Ubuntu 22.04 and Rexx 5.0.0 (which also upgraded Apache), they stopped working. I don't know what
happened; clearly, the current directory had changed, but I've not had the resources or the time
to investigate why. It's not so clear to me what the current directory is supposed when you have a
CGI called from an Apache action handler.
If the search order had worked as advertised, I would not have had any problem. That's what led me
to investigate this aspect of search.
If you look up the test framework and how to get to its Rexx
packages/programs that is exactly
what "setTestEnv.bat" (Windows) and "setTestEnv.sh" (Unix) do in
"test/trunk". That is the
reason why only unqualified file names are needed in all requires
directives in the entire
test package.
But again, I am no expert her, so hoping that Rick or Erich can shed some
more light on this.
Best regards
---rony
Once more: I don't think there's a clear, evident way to settle this conversation. A
/decision/ has to be taken. And it has to be /explained/ (i.e., documented) and, if possible,
/justified/. The last part is optional, of course: one can define a language as one sees fit.
The weight, if any, of my contribution, is only to emphasize two things:
* Other languages tackle this problem in a particular, coincident way.
* And that way is the most economic in terms of describing the search
procedure.
This does not mean that what I am proposing should be accepted. It's only my
point of view.
Josep Maria
