Still considering the much needed rewrite of the Platform Quirks section.
On 4/14/2017 9:15 PM, Warren Young wrote:
29. The Platform Quirks section needs a total rewrite. Sorry, but it left me
confused, and I know what’s going on.
How about this:
————————————————————
# Platform Quirks
Fossil glob patterns are based on the glob pattern feature of POSIX shells.
Fossil glob patterns also have a quoting mechanism, discussed above. Because
other parts of your operating system may interpret glob patterns and quotes
separately from Fossil, it is often difficult to give glob patterns correctly
to Fossil on the command line. Quotes and special characters in glob patterns
are likely to interpreted when given as part of a `fossil` command, causing
unexpected behavior.
These problems do not affect [versioned settings
files](/doc/trunk/www/settings.wiki) or Admin → Settings in Fossil UI.
Consequently, it is better to set long-term `*-glob` settings via these methods
than to use `fossil settings` commands.
That advice doesn’t help you when you are giving one-off glob patterns in
`fossil` commands. The remainder of this section gives remedies and workarounds
for these problems.
## POSIX Systems
If you are using Fossil on a system with a POSIX-compatible shell — Linux,
macOS, the BSDs, Unix, Cygwin, WSL etc. — the shell may expand the glob
patterns before passing the result to the `fossil` executable.
Sometimes this is exactly what you want. Consider this command for example:
$ fossil add RE*
If you give that command in a directory containing `README.txt` and
`RELEASE-NOTES.txt`, the shell will expand the command to:
$ fossil add README.txt RELEASE-NOTES.txt
…which is compatible with the `fossil add` command’s argument list, which
allows multiple files. Fossil doesn’t see the glob pattern at all, but since
the command does what you almost certainly wanted anyway, it’s fine.
If it did see the glob pattern in "fossil add RE*" it would actually try
to add a file named exactly "RE*" which is likely not at all what you
want! The file name arguments to fossil add (and the command like it)
are not globs, they are simply names relative to the current directory.
Now consider what happens instead if you say:
$ fossil add --ignore RE* src/*.c
This*doesn’t* do what you want because the shell will expand both `RE*` and
`src/*.c`, causing one of the two files matching the `RE*` glob pattern to be
ignored and the other to be added to the repository. You need to say this in
that case:
$ fossil add --ignore 'RE*' src/*.c
The single quotes force a POSIX shell to pass the `RE*` glob pattern through to
Fossil untouched, which will do its own glob pattern matching. There are other
methods of quoting a glob pattern or escaping its special characters; see your
shell’s manual.
POSIX shells also interpret the same quotation marks Fossil uses to handle
things like spaces in file names, as discussed above. For example, if you
needed to add all files matching `RE*` to the repository except for a file
called `REALLY SECRET STUFF.txt`, you could use nested quotes:
$ fossil add --ignore "'REALLY SECRET STUFF.txt'" RE*
The --ignore will not override a file name on the command line, but it
will cause a prompt asking what to do with the file. The very dangerous
-f or --force option will answer YES to that prompt for you, in this
case committing your secrets.
Bottom line there is don't put any secrets in a folder where fossil (or
any VCS) can touch it because an accident will happen and if that secret
is your Amazon AWS password, it can cost you an immense amount of money
in a surprisingly short amount of time!
You could instead escape a second set of double quotation marks:
$ fossil add --ignore "\"REALLY SECRET STUFF.txt\"" RE*
It bears repeating that the two glob patterns here are not interpreted the same
way when running this command from a*subdirectory* of the top checkout
directory as when running it at the top of the checkout tree. If these files
were in a subdirectory of the checkout tree called `doc` and that was your
current working directory, the command would have to be:
$ fossil add --ignore "'doc/REALLY SECRET STUFF.txt'" RE*
instead. The Fossil glob pattern still needs the `doc/` prefix because Fossil
always interprets glob patterns from the base of the checkout directory, not
from the current working directory as POSIX shells do.
## Windows
Neither standard Windows command shell — `cmd.exe` or PowerShell —
expands glob patterns the way POSIX shells do. Windows command shells rely on the
command itself to do the glob pattern expansion. The way this works depends on several
factors:
* the version of Windows you’re using
* which OS upgrades have been applied to it
* the compiler that built your Fossil executable
* whether you’re running the command interactively
* whether the command is built against a runtime system that does this at all
* whether the Fossil command is being run from a file named `*.BAT` vs being
named `*.CMD`
* the phase of the moon and whether this is an odd-numbered Thursday. (No,
not really, but the other caveats are all true. Yay, Windows!)
These factors also affect how a program like `fossil.exe` interprets quotation
marks on its command line.
The fifth item above doesn’t apply to `fossil.exe` when built with typical tool
chains, but we’ll see an example below where the exception applies in a way
that affects how Fossil interprets the glob pattern.
The most common problem is figuring out how to get a glob pattern passed on the
command line into `fossil.exe` without it being expanded by the C runtime
library that your particular Fossil executable is linked to, which tries to act
like the POSIX systems described above. Windows is not strongly governed by
POSIX, so it has not historically hewed closely to its strictures.
(This section does not cover the [Microsoft POSIX
subsystem](https://en.wikipedia.org/wiki/Microsoft_POSIX_subsystem), Windows’
obsolete [Services for Unix
3.*x*](https://en.wikipedia.org/wiki/Windows_Services_for_UNIX) feature, or the
[Windows Subsystem for
Linux](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux). (The latter
is sometimes incorrectly called “Bash on Windows” or “Ubuntu on Windows.”) See
the POSIX Systems section above for those cases.)
For example, consider how you would set `crlf-glob` to `*`. The naïve
approach will not work:
c:\...> fossil setting crlf-glob *
The C runtime library will expand that to the list of all files in the current
directory, which will probably cause a Fossil error because Fossil expects
either “`global`” or nothing after command line parameter giving the setting’s
new value. If you happened to run this in a directory with two files, one of
which was called `global`, it might appear to work but do the wrong thing,
depending on whether the `global` file name was expanded first or second.
Actually, the magic name would be --global which is much less likely to
be a file name due to the difficulty with getting most *nix commands to
create such a file. I was going to observe that the leading dash would
likely sort that name to the front of the command line, except despite
the help text telling you it must be after the value, fossil actually
doesn't care.
The sorting of wildcards does vary depending on the underlying file
system. If run in a FAT32 folder, I believe * will expand in disk order
which varies depending on the exact sequence of file creation and
deletion to this point in time. If run in an NTFS folder, * will expand
in alphabetical order (really in Unicode codepoint order) since the
underlying file system is more like a database than you might expect.
Let’s try again:
c:\...> fossil setting crlf-glob '*'
That may or may not work, depending on the factors listed above. On one system
where this was tested, it failed because the command shell sees that no file in
the current directory matches the glob pattern `'*'`, so the command shell
passed those three characters unchanged to `fossil.exe`, which stored them
as-is. Then when Fossil went to apply that glob pattern to file names, it saw
that the glob pattern is quoted, so it didn’t interpret `*` as meaning “any
series of characters;” the quotes made Fossil skip the “looks like a text file”
rules only for a file called exactly `'*'` rather than what we wanted, which
was to skip those rule checks for all files at the top of the checkout
directory.
I think that should actually work, at least using CMD not Powershell.
Yes, the stored value of crlf-glob would be '*' (with the single quotes)
but the glob list parser should strip the quotes off when it reads the
setting to use it.
C:...> fossil test-glob '*' abc
SQL expression: (x GLOB '*')
pattern[0] = [*]
1 abc
I wish I had a good link to point at discussing Windows shell quoting in
depth.
Powershell seems to have completely different quoting rules. I have
heard rumors the next revision of Windows will be making it the default
shell. Brace for the avalanche of confused users.
PS C:...> fossil test-glob '*' abc
SQL expression: (x GLOB '.all-help-9c65b5432e.html.espell')
pattern[0] = [.all-help-9c65b5432e.html.espell]
0 .dockerignore
0 .fossil-settings
.... (lots of lines omitted here)
0 abc
PS C:...> fossil test-glob "'* ?'" a "a bc" "ab c"
SQL expression: (x GLOB '* ?')
pattern[0] = [* ?]
0 a
0 a bc
1 ab c
PS C:...>
I think the file .all-help-9c65b5432e.html.espell is a result of my text
editor having been used once upon a time to spell check the output of
the fossil test-all-help command. (Which, IIRC, I added so that I could
do that spell check.)
An approach that*will* work reliably is:
c:\...> echo * | fossil setting crlf-glob --args -
This works because the built-in command `echo` does not expand its arguments,
and the global Fossil option `--args` makes it read further command arguments
from `-`, meaning Fossil’s standard input, which is connected to the output of
`echo` by the pipe.
Another correct approach is:
c:\...> fossil setting crlf-glob *,
This works because the trailing comma prevents the command shell from matching
any files, unless you happen to have files named with a trailing comma in the
current directory. If the pattern matches no files, it is passed into Fossil’s
`main()` function as-is by the C runtime system. Since Fossil uses commas to
separate multiple glob patterns, this means “all files at the root of the
Fossil checkout directory and nothing else.”
--
Ross Berteig r...@cheshireeng.com
Cheshire Engineering Corp. http://www.CheshireEng.com/
+1 626 303 1602
_______________________________________________
fossil-dev mailing list
fossil-dev@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/fossil-dev
