bug#38263: Bug in srfi-11

[Mark H Weaver]

Hi Tim,

> Mark H Weaver writes:
>> I agree that this example indicates a bug in Guile's 'let-values'
>> implementation (which was written by Andy Wingo in August 2009), but I
>> disagree that it should evaluate to '(9 2 (3) (4)).  I think that your
>> example should raise an error, because at the point where (set! a 9) is
>> found, neither of the 'a' variables are in scope.
>
> Oops, that `let` should have been a `let*` (Moving the first a into
> scope). But if you could verify that what I described is a bug I would
> like to propose a patch.

I agree that it's a bug, and that if you change 'let' to 'let*' in your
previous example, the result should be '(9 2 (3) (4)).

I took a quick look, and I believe the fix is simply to swap 'new-var'
and 'new-tmp' on line 95 of srfi-11.scm.  See the attached patch.  Does
it fix the problems you're seeing?

 Thanks,
   Mark


>From 4657b95713facffcde685b578ed19dbeb45624d0 Mon Sep 17 00:00:00 2001
From: Mark H Weaver 
Date: Sun, 24 Nov 2019 14:46:45 -0500
Subject: [PATCH] Fix 'let-values' where  is an improper list.

Fixes <https://bugs.gnu.org/38263>.
Reported by Tim Gesthuizen .

* module/srfi/srfi-11.scm (let-values): Swap 'new-tmp' and 'new-var' in
the pair pattern, to match the code that creates those pairs.
---
 module/srfi/srfi-11.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/module/srfi/srfi-11.scm b/module/srfi/srfi-11.scm
index 22bda21a2..42b8527ba 100644
--- a/module/srfi/srfi-11.scm
+++ b/module/srfi/srfi-11.scm
@@ -1,6 +1,7 @@
 ;;; srfi-11.scm --- let-values and let*-values
 
-;; Copyright (C) 2000, 2001, 2002, 2004, 2006, 2009 Free Software Foundation, Inc.
+;; Copyright (C) 2000, 2001, 2002, 2004, 2006, 2009,
+;;   2019 Free Software Foundation, Inc.
 ;;
 ;; This library is free software; you can redistribute it and/or
 ;; modify it under the terms of the GNU Lesser General Public
@@ -91,7 +92,7 @@
 (syntax (call-with-values (lambda () exp)
   (lambda (new-tmp ...) inner))
((vars exp)
-(with-syntax new-tmp . new-var) ...)
+(with-syntax new-var . new-tmp) ...)
(let lp ((vars (syntax vars)))
  (syntax-case vars ()
((id . rest)
-- 
2.24.0

bug#38263: Bug in srfi-11

[Mark H Weaver]

Hi Tim,

Tim Gesthuizen  wrote:

> Hi,
> I had a look today into the srfi-11 specificiation. It requires that the
> variables are bound to fresh locations so let me rephrase the bug:
>
>> (let ((a 1)
>>   (b (let-values (((a . b) (values 2 3))
>>   (c (begin (set! a 9) 4)))
>>(list a b c
>>   (cons a b))
>
> Evaluates to `(1 9 (3) (4))` while it should evaluate to
> `(9 2 (3) (4))`.

I agree that this example indicates a bug in Guile's 'let-values'
implementation (which was written by Andy Wingo in August 2009), but I
disagree that it should evaluate to '(9 2 (3) (4)).  I think that your
example should raise an error, because at the point where (set! a 9) is
found, neither of the 'a' variables are in scope.

 Regards,
   Mark

bug#38236: R7RS reader does not support datum labels

[Mark H Weaver]

Hi Andy,

Andy Wingo  writes:

> R7RS defines a lexical feature called "datum labels"; see section 2.4 in
> the report.  An example would be:
>
>#0=(a b c . #0#)
>
> Guile's reader doesn't support this feature and it's not clear if we
> should, in general.

FYI, I already implemented R7RS datum labels on the 'r7rs-wip' branch
(not to be confused with your new 'wip-r7rs' branch):

  
https://git.savannah.gnu.org/cgit/guile.git/commit/?h=r7rs-wip=92408ac20e921583b8e4ee26463dc5805ef01153

It depends on the preceding commit on the same branch:

  
https://git.savannah.gnu.org/cgit/guile.git/commit/?h=r7rs-wip=f687871eceb94bded109569880e696d8862d84fd

There was also a later commit on that branch that enabled compilation of
cyclic literals, but it's no longer applicable to the 'master' branch.

> Note, datum literals appear to be incompatible with array literals.

Can you elaborate on why you believe they're incompatible?  I haven't
looked closely in a while, but I didn't see any incompatibility when I
implemented this before.  Datum labels have '#' or '=' after the
numeral, and I'm not aware of any Guile array syntax that does.

I'm also a bit puzzled why you're apparently planning to rewrite
everything I already did on the 'r7rs-wip' branch.  The main issue on
that branch is that the implementation of *writing* cyclic data turned
out to be a mess, but it could be removed or replaced without affecting
much else.

   Mark

bug#37037: Guile 2.0.13+1-5.1 FTBFS on GNU/Hurd due to one failing test.

[Mark H Weaver]

Svante Signell  writes:

> On Thu, 2019-08-22 at 17:01 -0400, Mark H Weaver wrote:
>> It looks like this issue was already fixed in Guile 2.0.14, by commit
>> f2764cb1031379c47a17c02fef3f8164a6ce9cda on the 'stable-2.0' branch:
>>   
>> https://git.savannah.gnu.org/cgit/guile.git/commit/?h=stable-2.0=f2764cb1031379c47a17c02fef3f8164a6ce9cda
>> 
>> Would you like to try it and report back?
>
> Thank you, that commit fixed the problem. Reported to Debian version
> 2.0.13+1-5.1 as bug #935505, see 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=935505

Great, thanks for letting us know.  I'm closing this bug now.

 Regards,
   Mark

bug#37037: Guile 2.0.13+1-5.1 FTBFS on GNU/Hurd due to one failing test.

[Mark H Weaver]

Hi Svante,

Svante Signell  writes:

> When building Debian version 2.0.13+1-5.1 one test fails making the
> whole build to fail. The problem is in the file test-
> suite/tests/statprof.test. According to that file it is known that
> ITIMER_PROF is not implemented:
>
> ;; Throw `unresolved' upon ENOSYS.  This is used to skip tests on
> ;; platforms such as GNU/Hurd where `ITIMER_PROF' is is currently
> ;; unimplemented.
>   
> (define-syntax-rule (when-implemented body ...)
>   (catch 'system-error
> (lambda ()
>   body ...)
> (lambda args
>   (let ((errno (system-error-errno args)))
> (false-if-exception (statprof-stop))
> (if (= errno ENOSYS)
> (throw 'unresolved)
> (apply throw args))
>
> (pass-if-equal "return values"
> '(42 77)
>   (call-with-values
>   (lambda ()
> (with-output-to-port (%make-void-port "w")
>   (lambda ()
> (with-statprof
> (let loop ((i 1))
>   (if (zero? i)
>   (values 42 77)
>   (loop (1- i
> list))

It looks like this issue was already fixed in Guile 2.0.14, by commit
f2764cb1031379c47a17c02fef3f8164a6ce9cda on the 'stable-2.0' branch:

  
https://git.savannah.gnu.org/cgit/guile.git/commit/?h=stable-2.0=f2764cb1031379c47a17c02fef3f8164a6ce9cda

Would you like to try it and report back?

 Thanks,
   Mark

bug#37086: Guile Ice-9 Readline with-readline-completion-function

[Mark H Weaver]

tags 37086 + notabug
close 37086
thanks

Hi Matthew,

Matthew Henry  writes:

> Seen in: guile (GNU Guile) 2.2.4
>
> When using the with-readline-completion-function, the passed readline
> uses the default (apropos) completion function instead of the one
> provided to with-readline-completion-function.
>
> I believe that this is because root/guile-readline/ice-9/readline.scm
> has defined with-readline-completion-function as a function instead of
> as a macro.  The readline provided in thunk is executed before the
> body of with-readline-completion-function executes and overrides
> *readline-completion-function*.
>
> As an aside, I think the API would be better if the completion
> function could be provided to readline directly.
>
> Attached is a sample program.
>
> Below is sample output of a run of the attached program.  You can see
> that it's autocompleting Guile functions and variables (the default
> apropos completion) rather than the provided one which should have had
> only 3 options with just one starting in "th".
>
> ;;;
> Prompt:
> Display all 1859 possibilities? (y or n)
> Prompt: th
> the-eof-object   the-scm-module   thread-exited?   thunk?
> the-root-module  thread?  throw
> Prompt: th
>
> (use-modules (ice-9 readline))
>
> (with-readline-completion-function
>  (make-completion-function '("one" "two" "three"))
>  (readline "Prompt: "))

The problem is that 'with-readline-completion-function' expects a
"thunk" as the second argument.  A "thunk" is a procedure that takes no
arguments.  Typically this means that you wrap the relevant code within
(lambda () ...), like this:

  (use-modules (ice-9 readline))
  
  (with-readline-completion-function
   (make-completion-function '("one" "two" "three"))
   (lambda () (readline "Prompt: ")))

> I'm early in my Scheme journey, but here's a suggested fix:
> 
> (define-syntax-rule (with-readline-completion-function completer expr ...)
>   "With @var{completer} as readline completion function, call @var{expr ...}."
>   (let ((old-completer *readline-completion-function*))
> (dynamic-wind
>   (lambda ()
> (set! *readline-completion-function* completer))
>   (lambda () expr ...)
>   (lambda ()
> (set! *readline-completion-function* old-completer)
> 
> (export with-readline-completion-function)

This is fine, but it's a different API.  It's true that you need to use
a macro if you want to avoid wrapping the body within (lambda () ...),
but just like 'dynamic-wind' itself, 'with-readline-completion-function'
was designed to be an ordinary procedure that accepts the body
expression(s) as a THUNK.

   Happy hacking!
Mark

bug#36677: [PATCH] Don't truncate backtraces

[Mark H Weaver]

Hi David,

> I am very pleased to read that you think it is important to enable
> truncated printing as a default for backtrace, I think so to. But
> maybe Guile could provide an easy mechanism to overwrite these
> defaults, using procedures, or parameters? (not depending on an
> 'external' variable I mean (*)

You can see now that there are pressures coming from both directions on
this issue.  There are complaints that we truncate too much information,
and other complaints that we don't truncate often enough.  There are
proposals to never truncate anything by default, and proposals to
truncate everything by default.

> I wrote "these defaults", "procedures or parameters", using plural,
> because I think that the default should also enable truncated printing
> for the repl and the raised exception system, what do you think?

I'm reluctant to truncate REPL output by default for a few reasons:

(1) Historically, the Guile has never truncated REPL output, and I'm
concerned that changing the default behavior may violate longstanding user 
expectations.  I, for one,
often ask for a moderately large data structure to be computed and
printed at the REPL, and I normally want to see the whole thing.

(2) Some software may act as a front-end for the Guile REPL, sending
commands to it and interpreting the output.  For example, I believe that
Geiser does this.  I'm concerned that changing the default truncation
behavior may cause problems here.

(3) I'm not confident that 'truncated-print' is fully robust for
arbitrary data types.  I would need to make a careful audit of the code.

(4) The Guile REPL already provides a way to specify a custom printer,
so there's nothing stopping you from installing 'truncated-print' as the
REPL printer.

Regarding truncating exception printing by default, I'm inclined to
think that it's better to err on the side of printing too much than too
little.  If an uncaught exception occurs, that clearly indicates an
error, and if we truncate the error report, that might make the
difference between being able to figure out what went wrong and being
unable to do so, especially if the error is not easily reproducible.

Your position on this is pretty much the opposite of what Robert
Vollmert is advocating here, although to be precise he's talking about
backtraces whereas you're now talking about exception conditions.
Still, they both seem to be in the same area.

To be honest, the main reason I find lack of truncation painful
sometimes is because Emacs does not cope well with extremely long lines,
to put it mildly.  Honestly, that's a flaw in Emacs, and it seems like
it would be better to fix Emacs than to work around it in Guile by
omitting potentially useful information from error reports by default.

I need to think more on this.  In the meantime, I welcome opinions.

 Regards.
   Mark

bug#36709: Likely bug in scm_port_poll

[Mark H Weaver]

I sent the following message about a month ago, and now am forwarding to
bug-guile so that we do not forget about it.

   Mark


From: Mark H Weaver 
Subject: Likely bug in scm_port_poll
Date: Tue, 18 Jun 2019 00:40:27 -0400
Message-ID: <87a7ef1p61@netris.org>
Cc: guile-de...@gnu.org
To: Andy Wingo 

Hi Andy,

If you'd be willing to take a quick look at this, I'd be grateful.
Probably a bug, but I feel uneasy that I might be overlooking something.

scm_port_poll contains the following code:

  if (scm_i_string_contains_char (events, 'r'))
c_events |= POLLIN;
  if (scm_i_string_contains_char (events, '!'))
c_events |= POLLPRI;
  if (scm_i_string_contains_char (events, 'w'))
c_events |= POLLIN;

That second POLLIN should be POLLOUT, right?

  Thanks,
Mark

https://git.savannah.gnu.org/cgit/guile.git/tree/libguile/ports.c?h=stable-2.2=420c2632bb1f48e492a035c1d216f209734f45e6#n1423

bug#36677: [PATCH] Don't truncate backtraces

[Mark H Weaver]

Hi Robert,

Robert Vollmert  writes:

> * module/system/repl/debug.scm (print-frame): Print full object if
> width keyword is #f.
> * libguile/backtrace.c (display_backtrace_body): Call print-frames
> with #:width #f.
> ---
>
> This change was prompted by recent discussion on the Guix lists:
>   https://lists.gnu.org/archive/html/guix-devel/2019-07/msg00207.html
> In Guix, the truncation of stack traces frequently obscures
> important information due to the long filenames.

I'm sympathetic to this problem, but simply disabling the truncated
printing during backtraces is not workable.  It is quite often the case
that some of the structures printed in backtraces are *huge*, or even
cyclic.

Have you tried setting the COLUMNS environment variable to a larger
value?  I'd prefer a solution along those lines, where the user can set
an environment variable to ask for less truncation in backtraces.

 Thanks,
   Mark

bug#35920: strftime incorrectly assumes that nstrftime will produce UTF-8

[Mark H Weaver]

Hi John,

John Cowan  writes:

> That's a mug's game: I've been there and tried it (not in Scheme). I
> recommend writing a strftime in Scheme from scratch.  It's not that
> hard; the most annoying thing is getting into the locale files to
> handle the locale-sensitive directives (month name, weekday name,
> AM/PM, and the ordering of dates).

Is there a portable way to find the relevant locale files and interpret
them, on both POSIX and Windows systems?  If so, can you point out the
relevant documentation?

  Thanks,
Mark

bug#35920: strftime incorrectly assumes that nstrftime will produce UTF-8

[Mark H Weaver]

reopen 35920
thanks

Hi Ludovic,

> Mark H Weaver  skribis:
>
>> Here's a patch that might fix the problem, but I don't have time to test
>> it right now.
>
> It works! :-)  I wrote tests and pushed it as
> ab2fd70ef1e36c6532128b73082809ef3c056556.

On my system, I found that my proposed patch caused one of the existing
tests to fail.  The problem is that if the format string includes
characters that are not representable in the current locale encoding, it
will fail.  It seems to me that this could break existing code that
currently works.  User code that uses 'strftime' might never encode the
resulting string in the locale encoding.

I was planning to rewrite the code to scan for the '%' escapes
ourselves, to call 'strftime' for each escape sequence (without
including the surrounding text), and to concatenate the results.

> I forgot to change the commit author to you before pushing, apologies!

No worries.  Thanks for working on it.

  Mark

bug#36342: accept4 detection on illumos needs -lsocket -lnsl

[Mark H Weaver]

Hi Michal,

Michal Nowak  writes:

> GNU Guile 2.2.5 build fails because accept4 detection on OpenIndiana
> 2019.04 (illumos distribution) [1] fails due to "-lsocket -lnsl" not
> being part of the linking process and the build system then tries to
> build it's vendored accept4:

I believe this is an issue in gnulib, because the relevant autoconf
detection code, as well as the replacement implementation that fails to
compile on OpenIndiana, are imported from gnulib.

I've CC'd the bug-gnu...@gnu.org mailing list, for input from the gnulib
developers.

Thanks for this report,

  Mark


>CC   accept4.lo
> In file included from /usr/include/sys/time.h:462:0,
>   from ./sys/time.h:39,
>   from /usr/include/sys/select.h:53,
>   from ./sys/select.h:36,
>   from /usr/include/sys/types.h:640,
>   from ./sys/types.h:28,
>   from ./sys/socket.h:51,
>   from accept4.c:20:
> ./sys/socket.h:1034:1: error: conflicting types for 'accept4'
>   _GL_FUNCDECL_SYS (accept4, int,
>   ^
> In file included from ./sys/socket.h:58:0,
>   from accept4.c:20:
> /usr/include/sys/socket.h:540:12: note: previous declaration of
> 'accept4' was here
>   extern int accept4(int, struct sockaddr *_RESTRICT_KYWD, Psocklen_t,
> int);
>  ^~~
> accept4.c:32:1: error: conflicting types for 'accept4'
>   accept4 (int sockfd, struct sockaddr *addr, socklen_t *addrlen, int
> flags)
>   ^~~
> In file included from ./sys/socket.h:58:0,
>   from accept4.c:20:
> /usr/include/sys/socket.h:540:12: note: previous declaration of
> 'accept4' was here
>   extern int accept4(int, struct sockaddr *_RESTRICT_KYWD, Psocklen_t,
> int);
>  ^~~
>
> This is fixed by LIBS="-lsocket -lnsl" in the build environment.
>
> This was in the past reported both on the guile-user [2] and
> openindiana-discuss [3] mailing lists, where you also can see the
> build error in full.
>
> Thanks,
> Michal
>
> [1] https://illumos.org/man/3socket/accept4
> [2] https://lists.gnu.org/archive/html/guile-user/2018-03/msg6.html
> [3]
> https://openindiana.org/pipermail/openindiana-discuss/2018-March/021931.html

bug#36338: guile-2.2.5 fails to build with -ffast-math

[Mark H Weaver]

tags 36338 + notabug wontfix
close 36338
thanks

Hi Cyprien,

Cyprien Nicolas  writes:

> I apologize if the issue was already reported, but I didn't find a
> previous record.
>
> At Gentoo, users build packages themselves, and have the freedom to
> choose the CFLAGS they want.
>
> We had several reports [1,2] from users unable to build guile with
> -Ofast. Adding -fno-fast-math make the build pass.
>
> Compiling with -O0 -ffast-math also fails.

Guile generally assumes that the underlying C implementation will be
standards compliant.  -Ofast, -ffast-math and related options disregard
standards compliance.  Among other things, -ffast-math implies
-ffinite-math-only, which generates code that assumes that all floating
point values are finite.  However, there's code in Guile's compiler that
uses +inf.0 and -inf.0 and expects that they will behave as specified in
IEEE-754.

So, in summary, I would say that Guile does not support being compiled
with -Ofast, -ffast-math, or many of the options that -ffast-math
implies.

> I'm not sure if you can solve this. On our side, we will append
> -fno-fast-math to compiler's flags.

I think this is the right approach.

 Thanks,
   Mark

bug#36350: [2.2.5] ‘read-headers’ blocks, thereby breaking web servers

[Mark H Weaver]

Ludovic Courtès  writes:
> Perhaps we should consider releasing 2.0.6 soon and use that in Guix
> on ‘core-updates’.

Sure, sounds like a good idea.

   Mark

bug#36350: [2.2.5] ‘read-headers’ blocks, thereby breaking web servers

[Mark H Weaver]

Hi Ludovic,

Ludovic Courtès  writes:

>   ./meta/guile examples/web/hello.scm &
>   wget -O - http://localhost:8080
>
> You’ll notice that ‘wget’ hangs (never receives a response) because the
> server is actually stuck in a read(2) call that will never complete, in
> ‘read-headers’.
>
> Reverting 73cde5ed7218a090ecee70908af5445796f0 solves the problem.
>
> AIUI, before that commit, ‘read-header-line’ would read exactly one
> line.  After this change, it calls ‘lookahead-char’, which can block,
> and that’s exactly what’s happening here.

Gah, indeed!

Also, I see now that there was already some existing code to handle HTTP
continuation lines, before my commit referenced above, although it
neglects to fold the whitespace after the CRLF into a single SP octet.
I'm not sure how I failed to notice that.

The commit should simply be reverted, I think.  Pushed as commit
e1225d013ed8673382d6d8f9300dd6b175c8b820 on the stable-2.2 branch.
I tried leaving the new test in place, but it failed due to the lack of
whitespace folding in the previous code.

I really messed up here, sorry.

> I also noticed that there are no unit tests for (web server), which we
> should probably address while we’re at it.  :-)

Yes, that would be great.  I won't be able to do it anytime soon,
though.

   Mark

bug#36314: missing scmconfig.h, guile 2.2.4 and 2.2.5, build error

[Mark H Weaver]

Hi Carl,

Carl Hansen  writes:

> make[1]: Entering directory 
> '/td2/carlstuff/build/gsrc/pkg/gnu/guile/work/guile-2.2.5'
> make -C libguile scmconfig.h
> make[2]: Entering directory 
> '/td2/carlstuff/build/gsrc/pkg/gnu/guile/work/guile-2.2.5/libguile'
> \
> if [ "no" = "yes" ]; then \
> gcc -std=gnu11 -DHAVE_CONFIG_H  -I.. \
>-c -o gen-scmconfig.o gen-scmconfig.c; \
> else \
> gcc -std=gnu11 -DHAVE_CONFIG_H   -DBUILDING_LIBGUILE=1 -I.. -I.. 
> -I../lib -I../lib -I/opt/gsrc/lib/libffi-3.2.1/include 
> -I/td2/carlstuff/build/gsrc/pkg/gnu/guile/work/guile-2.2.5  -Wall 
> -Wmissing-prototypes -Wdeclaration-after-statement -Wpointer-arith 
> -Wswitch-enum -fno-strict-aliasing -fwrapv -fvisibility=hidden -g -O2 -c -o 
> gen-scmconfig.o gen-scmconfig.c; \
> fi
> In file included from ./strings.h:27,
>  from /usr/include/string.h:431,
>  from ../lib/string.h:41,
>  from gen-scmconfig.c:142:
> ../libguile/__scm.h:54:10: fatal error: libguile/scmconfig.h: No such file or 
> directory
>54 | #include "libguile/scmconfig.h"
>   |  ^~
> compilation terminated.
> make[2]: *** [Makefile:4132: gen-scmconfig.o] Error 1
>
>
>
> In other words, it is trying to create libguile/scmconfig.h, but it includes
> libguile/__scm.h  which includes  libguile/scmconfig.h, therefore build 
> error.

The problem seems to be that that the 'libguile' directory is somehow
being added to the C include search path, although it should not be.
The error message above implies that line 431 of /usr/include/string.h
includes  (it does on my system), which is obviously intended
to pick up a system header.  Instead, libguile/strings.h is being picked
up.

I don't see -I. or equivalent on the command line above.  Are you adding
"." to CPATH, C_INCLUDE_PATH, or some similar environment variable?
Note that an empty path component is typically interpreted as ".", so
two adjacent colons, or a colon at the beginning or end of the path
variable, could also explain this.

> Happens with 2.2.5 and 2.2.4

Note that Guile 2.2.4 was released almost a year ago, and you're the
first person to report this problem, so I suspect it's something unusual
in your build environment.

If the ideas above don't lead to a solution, could you send us more
details, e.g. the complete set of environment variables and arguments
passed to ./configure?

  Thanks,
Mark

bug#36251: Regex library doesn't recognize ']' in a character class

[Mark H Weaver]

Hi,

Abdulrahman Semrie  writes:

> I am using the pattern [\\[\\]a-zA-Z]+ to match a string with left or
> right bracket in it. However, the string-match function doesn’t match
> the ‘]’ character. To demonstrate with an example, try the following
> funciton:
>
> (string-match "[\\[\\]a-zA-Z]+" "Text[ab]”)
>
> The result for the above function should have been a match structure
> with Text[ab] matched. However, the string-match returns #f which is
> incorrect. To test if the pattern I am using was right, I tried on
> regex101.com and it works. Here (https://regex101.com/r/VAl6aI/1) is
> the link that demonstrates that it works.

It turns out that there are several flavors of regular expressions in
common use, with different features and syntax.  The link you provided
is using PCRE (PHP) regular expressions (see the "flavor" pane on the
left), and there are three other supported flavors on that web site.

Guile's (ice-9 regex) module provides a simpler flavor of regexps known
as "POSIX extended regular expressions", implemented as a thin wrapper
around your system's POSIX regular expression library ('regcomp' and
'regexec').  The web site you referenced does not appear to support
POSIX extended regular expressions, but here are some links about them:

  
https://en.wikibooks.org/wiki/Regular_Expressions/POSIX-Extended_Regular_Expressions
  
https://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap09.html#tag_09_04

One of the notable differences is that in POSIX extended regular
expressions, character classes do not support backslash escapes, but
instead use a more ad-hoc approach as  described.

 Regards,
   Mark

bug#36170: configure fails on FreeBSD

[Mark H Weaver]

Roger Mason  writes:

> I cloned the git repo yesterday (2019-06-10, most recent commit
> 8d469660525d74734f3184cb9ed01b6f2dcd0445).  I ran autogen.sh in the
> source directory.  I had to comment out the call to m4 --version, which
> failed (I have m4-1.4.18).

That's the latest stable m4 release, and the same version that I have.
It's surprising that "m4 --version" failed.  It makes me wonder if
autogen.sh is using Bash-specific syntax.

Does "bash -x autogen.sh" work, when using the autogen.sh that we
provide?

  Mark

bug#36170: configure fails on FreeBSD

[Mark H Weaver]

Hi Roger,

Roger Mason  writes:

> I cloned the git repo yesterday (2019-06-10, most recent commit
> 8d469660525d74734f3184cb9ed01b6f2dcd0445).  I ran autogen.sh in the
> source directory.  I had to comment out the call to m4 --version, which
> failed (I have m4-1.4.18).  I created a build directory.  When I
> configure like this:
>
> ../guile/configure --with-bdw-gc=bdw-gc-threaded
>
> it works:
>
> ---snip---
> config.status: executing depfiles commands
> config.status: executing libtool commands
> config.status: executing po-directories commands
>
> When I configure like this:
>
> ../guile/configure --prefix="/home/rmason/32-bit-install" 
> --with-bdw-gc=bdw-gc-threaded
>
> it fails to find various libraries & headers unless I list them
> explicitly in the invocation of configure like this:
>
> ../guile/configure --prefix=/home/rmason/32-bit-install \
>  --with-bdw-gc=bdw-gc-threaded \
>  --with-libltdl-prefix=/usr/local \
>  --with-libgmp-prefix=/usr/local \
>  --with-libunistring-prefix=/usr/local \
>  --with-libiconv-prefix=/usr/local
>
> Is that how it is supposed to work, or is there a bug in the build
> system?

I guess what's happening here is that ./configure automatically looks
for libraries in the prefix directory as specified by --prefix=DIR.
I don't have time right now to verify that guess.

> I note also that configure claims my readline library is too old (<
> 2.1), but I have:
>
> pkg info -x readline
> readline-7.0.5

Something went wrong here.  The relevant check involves compiling a
small test program that references 'rl_getc_function', which was
apparently added in readline 2.1.  Something must have gone wrong
compiling that test program.

After running ./configure, there should be a file 'config.log' which
includes the failed test program, the compile command used to compile
it, and the compiler error messages.  Can you search for 'readline' in
that file and see what went wrong?

  Thanks,
Mark

bug#36079: unhelpful error message

[Mark H Weaver]

Hi Ricardo and Robert,

Ricardo Wurmus  writes:

>> Working on Guix, I encountered the following:
>>
>> Within a guix checkout, I edited gnu/packages/haskell.scm, accidentally
>> making a Haskell comment:
>>
>> (arguments
>>  `(#:tests? #f)) -- sporadic failure: 
>> https://github.com/fpco/streaming-commons/issues/49
>>
>> Then I tried to keep working on my in-development package, and was able to 
>> trace
>> the `guix build` error back to the following:
>>
>> $ ./pre-inst-env guild compile ../modules/postgrest.scm
>> ;;; note: source file /home/rob/guix/gnu/packages/haskell.scm
>> ;;;   newer than compiled /home/rob/guix/gnu/packages/haskell.go
>> ;;; note: source file /home/rob/guix/gnu/packages/haskell.scm
>> ;;;   newer than compiled 
>> /run/current-system/profile/lib/guile/2.2/site-ccache/gnu/packages/haskell.go
>> ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
>> Syntax error:
>> unknown location: package: invalid field specifier in form —-
>
> This error is produced by the “package” macro, which is of this form:
>
>   (package
>(field value)
>(another-field its-value)
>…)
>
> The macro has a number of valid field names and it tries to do some
> simple validation.  That’s where the error comes from.  When “--” is
> encountered it is in the position of a field, so the macro tries to be
> helpful and reports that “--” is not a valid way to specify a field.
>
> This error message is not produced by Guile.

Well, yes and no.  It is true that code in Guix is calling
'syntax-violation' to report the error.  However, it is also true that
Guix passes to 'syntax-violation' the syntax object corresponding to
“--”, which ideally should have source location information attached to
it.  If the syntax object in question represented a list or some other
heap-allocated value, it *would* have had source location information
attached, and that would have been included in the error message.

This issue is a longstanding limitation in Guile's source location
tracking.  The problem is that Guile uses Scheme's standard 'read'
procedure to read the source code, which uses plain Scheme symbols to
represent identifiers.  Scheme symbols with the same name are
indistinguishable, i.e. every occurrence of 'x' or '--' is represented
by precisely the same heap object, and therefore there's no way to
attach source information to each occurrence of a symbol.

Fixing this would involve abandoning Scheme's standard 'read' procedure
for reading source code, and instead using a different reader (not yet
written) that uses a different object type to represent identifiers and
immediate values.  Our macro expander would need to be modified to
support this new representation.

I believe this should be done, and I hope to find the energy to do it
some day.

   Mark

bug#35920: strftime incorrectly assumes that nstrftime will produce UTF-8

[Mark H Weaver]

Here's a patch that might fix the problem, but I don't have time to test
it right now.

   Mark


--8<---cut here---start->8---
diff --git a/libguile/stime.c b/libguile/stime.c
index b681d7ee3..9a21b61fe 100644
--- a/libguile/stime.c
+++ b/libguile/stime.c
@@ -662,9 +662,9 @@ SCM_DEFINE (scm_strftime, "strftime", 2, 0, 0,
   SCM_VALIDATE_STRING (1, format);
   bdtime2c (stime, , SCM_ARG2, FUNC_NAME);
 
-  /* Convert string to UTF-8 so that non-ASCII characters in the
- format are passed through unchanged.  */
-  fmt = scm_to_utf8_stringn (format, );
+  /* Convert the format string to the locale encoding, as the underlying
+ 'strftime' C function expects.  */
+  fmt = scm_to_locale_stringn (format, );
 
   /* Ugly hack: strftime can return 0 if its buffer is too small,
  but some valid time strings (e.g. "%p") can sometimes produce
@@ -727,7 +727,7 @@ SCM_DEFINE (scm_strftime, "strftime", 2, 0, 0,
 #endif
 }
 
-  result = scm_from_utf8_string (tbuf + 1);
+  result = scm_from_locale_string (tbuf + 1);
   free (tbuf);
   free (myfmt);
 #if HAVE_STRUCT_TM_TM_ZONE
@@ -754,16 +754,16 @@ SCM_DEFINE (scm_strptime, "strptime", 2, 0, 0,
 {
   struct tm t;
   char *fmt, *str, *rest;
-  size_t used_len;
+  SCM used_len;
   long zoff;
 
   SCM_VALIDATE_STRING (1, format);
   SCM_VALIDATE_STRING (2, string);
 
-  /* Convert strings to UTF-8 so that non-ASCII characters are passed
- through unchanged.  */
-  fmt = scm_to_utf8_string (format);
-  str = scm_to_utf8_string (string);
+  /* Convert strings to the locale encoding, as the underlying
+ 'strptime' C function expects.  */
+  fmt = scm_to_locale_string (format);
+  str = scm_to_locale_string (string);
 
   /* initialize the struct tm */
 #define tm_init(field) t.field = 0
@@ -807,14 +807,14 @@ SCM_DEFINE (scm_strptime, "strptime", 2, 0, 0,
   zoff = 0;
 #endif
 
-  /* Compute the number of UTF-8 characters.  */
-  used_len = u8_strnlen ((scm_t_uint8*) str, rest-str);
+  /* Compute the number of characters parsed.  */
+  used_len = scm_string_length (scm_from_locale_stringn (str, rest-str));
   scm_remember_upto_here_2 (format, string);
   free (str);
   free (fmt);
 
   return scm_cons (filltime (, zoff, NULL),
-  scm_from_signed_integer (used_len));
+   used_len);
 }
 #undef FUNC_NAME
 #endif /* HAVE_STRPTIME */
--8<---cut here---end--->8---

bug#35920: strftime incorrectly assumes that nstrftime will produce UTF-8

[Mark H Weaver]

Hi Christopher,

Christopher Lam  writes:

> Addendum - wish to confirm if guile bug (guile-2.2 on Windows):
> - set locale to non-Anglo so that (setlocale LC_ALL) returns
> "French_France.1252"
> - call (strftime "%B" 400) - that's 4x10^6 -- this should return
> "février 1970"
>
> but the following error arises:
> Throw to key `decoding-error' with args `("scm_from_utf8_stringn" "input
> locale conversion error" 0 #vu8(102 233 118 114 105 101 114 32 49 57 55
> 48))'.
>
> Is this a bug?

Yes.  Guile's 'strftime' procedure currently assumes that the underlying
'nstrftime' C function (from Gnulib) will produce output in UTF-8,
although it almost certainly produces output in the locale encoding.
Indeed, the bytevector #vu8(102 233 118 114 105 101 114 32 49 57 55 48)
represents the characters "février 1970" in Windows-1252 encoding.

I'm CC'ing this reply to , so that a bug ticket will
be created.  In the future, that's the preferred address for sending bug
reports.

Anyway, thanks for letting us know about this.  I'll work on it soon.

  Mark

bug#31776: [PATCH] Fix gc.test "after-gc-hook gets called" failures

[Mark H Weaver]

Hi Andrea,

Andrea Azzarone  writes:

> "after-gc-hook gets called" test randomly fails as reported
> downstream, for example:
> - https://debbugs.gnu.org/cgi/bugreport.cgi?bug=31776
> - https://bugs.launchpad.net/ubuntu/+source/guile-2.2/+bug/1823459
>
> I'm attaching a patch that seems to fix the failures.
>
> From 2efba337d5b636cd975260f19ea74e27ecf0ca17 Mon Sep 17 00:00:00 2001
> From: Andrea Azzarone 
> Date: Thu, 11 Apr 2019 16:30:58 +0100
> Subject: Fix gc.test "after-gc-hook gets called" failures
>
> * libguile/scmsigs.c: Call scm_async_tick to give any pending asyncs a chance 
> to
>   run before we block indefinitely waiting for a signal to arrive.

Thanks for this.  I pushed your commit (with minor reformatting) to our
'stable-2.2' branch as commit 546b0e87294b837ec29164d87cf17102e9aeee0c.

I believe that this will prevent the problem from happening in the most
common cases, e.g. when there's only one user-visible thread, or when
there are no long-sleeping user-visible threads.

However, it occurs to me that in a multithreaded Guile program, a user
thread might trigger a GC and then sleep for a long time, without
calling 'scm_async_tick' in between.  If we're unlucky and the
'after_gc_async' gets queued in the wrong thread, it might be a long
time before the hook runs.

Fundamentally, the problem we face here is similar to the thorny
problems faced with finalizers and signal handlers: we must choose a
proper time and context for them to be run safely, when the data they
need to access is in a consistent state, etc.

To deal with the issues around finalizers, Guile recently gained a
finalizer thread.  It may be that we should arrange to run the
'after_gc_async' in the finalizer thread as well, instead of whatever
random thread we happen to be in when GC is triggered.

Thoughts?

  Regards,
Mark

bug#34860: Probing broken symlinks always involves errors

[Mark H Weaver]

Hi Tim,

Tim Gesthuizen  writes:

> I wrote a little application that tries to probe whether a symlink
> exists. The `stat` function has an undocumented optional parameter that
> lets stat return #f when the file does not exist. With this argument one
> can probe for any files excluding broken symlinks.
> lstat does not offer this argument. Therefore I needed to use
> guard-expressions around lstat to probe for broken symlinks.
>
> So probing broken symlinks relies on errors in the control flow and is
> not straight forward.

I would suggest using 'false-if-exception', which is documented in the
manual:

  (false-if-exception (lstat file-name))

That seems fairly straightforward and readable, no?

I'm not sure that errors in the control flow is necessarily something
that we need to avoid in cases like this.  It's true that there's some
minor expense involved in handling the exception, but I suspect it's
lost in the noise compared with the expense of the 'lstat' system call
itself.

Do you see a practical problem with this approach?

> Maybe we should add the optional argument that `stat` has and document
> the arguments for both commands.

I'm not strongly opposed to the idea, but it raises the question of what
to do with the corresponding C API function 'scm_lstat'.  We certainly
can't change the number of arguments to 'scm_lstat' in the 2.2.x series,
because that would break ABI compatibility.  We could change it in the
next release series, but that would require source-level changes in any
code that uses 'scm_lstat'.

Alternatively, we could keep 'scm_lstat' unchanged, and add a new
internal C function with the optional argument, bound to Scheme 'lstat'.

Anyway, I'm inclined to simply suggest using 'false-if-exception'.

What do you think?

Regardless, thanks for bringing my attention to the fact that 'scm_stat'
is incorrectly documented in the manual.  It asserts that there's only
one argument to that function, but in fact there are now two.  That's
definitely a bug.

 Regards,
   Mark

bug#32367: sigaction hangs

[Mark H Weaver]

Hi Matt,

Matt Wette  writes:

> On 12/15/18 8:46 AM, Matt Wette wrote:
>> -s seems to end up using load-in-vicinity: I can re-create this way:
>>
>> mwette$ guile -L `pwd` -c '(load-in-vicinity (getcwd) "foo.scm")'
>> ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
>> ;;;   or pass the --no-auto-compile argument to disable.
>> ;;; compiling /home/mwette/proj/guile/bugs-guile/32367/foo.scm
>> setting SIGCHILD to SIG_DFL...
>> ^C
>>
>
> And using `-l' to load scsh.scm fixes the problem:
>
> mwette$ guile -L `pwd` -l scsh.scm -c '(load-in-vicinity (getcwd)
> "foo.scm")'setting SIGCHILD to SIG_DFL...
> setting SIGCHILD to SIG_DFL...done
> ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
> ;;;   or pass the --no-auto-compile argument to disable.
> ;;; compiling /home/mwette/proj/guile/bugs-guile/32367/foo.scm
> ;;; compiled
> /home/mwette/.cache/guile/ccache/2.2-LE-8-3.A/home/mwette/proj/guile/bugs-guile/32367/foo.scm.go
> mwette $
>
> An update from 2.2.3 to 2.2.4 adds call-with-module-autoload-lock
> which places a
> mutex around autoload.  This looks like one place to start
> digging. See ice-9/threads.scm:
>
> (set! (@ (guile) call-with-module-autoload-lock)
>   (let ((mutex (make-mutex 'recursive)))
>     (lambda (thunk)
>   (with-mutex mutex
>     (thunk)

It looks like you might have missed some of the messages in this bug
report.  See 
and later comments.

 Regards,
   Mark

bug#33403: [Geiser-users] bug#33403: Data length limit in Guile/Geiser/Scheme evaluation

[Mark H Weaver]

A few more notes:

I wrote earlier:
> However, before doing this, some warnings are in order:
>
> When in noncanonical mode, the normal processing of ERASE (usually DEL
> or Ctrl-H) and KILL (usually Ctrl-U) characters are disabled,

Also the handling of Ctrl-D appears to be disabled in noncanonical mode
on my system, although this wasn't clear to me from the docs.

> At least in the case of the Guile REPL, one notable side effect of
> running in noncanonical mode is that when a list is entered at the REPL,
> the 'read' returns as soon as the final close parenthesis is entered.
> Nothing after that is read, not even the usual newline.

There's an additional wrinkle here: after 'read' returns, Guile tries to
read optional whitespace followed by a newline, but only if it's
immediately available.  See 'flush-to-newline' at the end of
module/system/repl/repl.scm in Guile.

So, unfortunately there's a race condition here, but typically if you
send the newline immediately after the final character of input, it is
likely that the newline will be consumed by the REPL reader and not by
the code that is subsequently run.

Finally, I should note that I consider this race condition suboptimal,
and will likely change how Guile behaves in the future, so please don't
rely on the behavior I have described above.  I will likely change
Guile's REPL reader to wait for the final newline in all cases.

  Mark

bug#33403: [Geiser-users] bug#33403: Data length limit in Guile/Geiser/Scheme evaluation

[Mark H Weaver]

Hello all,

"Jose A. Ortega Ruiz"  writes:

> On Fri, Nov 16 2018, Neil Jerram wrote:
>
>> Neil Jerram  writes:
>>
>>> Mark H Weaver  writes:
>>>
>>>> This is a documented limitation in Linux's terminal handling when in
>>>> canonical mode.  See the termios(3) man page, which includes this text:
>>>>
>>>>Canonical and noncanonical mode
>>>>
>>>>The setting of the ICANON canon flag in c_lflag determines
>>>>whether the terminal is operating in canonical mode (ICANON set)
>>>>or noncanonical mode (ICANON unset).  By default, ICANON is set.
>>> [...]
>>>>* The maximum line length is 4096 chars (including the
>>>>  terminating newline character); lines longer than 4096 chars
>>>>  are truncated.  After 4095 characters, input processing (e.g.,
>>>>  ISIG and ECHO* processing) continues, but any input data after
>>>>  4095 characters up to (but not including) any terminating
>>>>  newline is discarded.  This ensures that the terminal can
>>>>  always receive more input until at least one line can be read.
>>>>
>>>> Note that last item above.
>>>
>>> Awesome; thank you Mark.
>>>
>>> So possibly this limit can be removed, in my Org/Geiser context, by
>>> evaluating (system* "stty" "-icanon") when initializing the Geiser-Guile
>>> connection.  I'll try that.  Will the terminal that that 'stty' sees be
>>> the same as Guile's stdin?
>>>
>>> Jao, if that works, I wonder if it should be the default for Geiser?  It
>>> appears to me that Geiser shouldn't ever need the features of canonical
>>> mode.  Is that right?
>>>
>>> Anyway, I'll see first if the stty call is effective.
>>
>> Yes, with this in my ~/.guile-geiser -
>>
>> (system* "stty" "-icanon")
>>
>> - I can do evaluations past the 4K line length limit, and the Org-driven
>> problem that I first reported [1] has disappeared.
>
> Ah, system* is a scheme call! So yeah, maybe we could add that call to
> Geiser's guile initialization... i don't really see how that would cause
> any problem elsewhere.

I think something like this should be done, not only in the Guile
initialization, but ideally in the generic Geiser code that connects to
inferior processes via pseudo-tty.  After the pseudo-tty is allocated
but before launching the inferior Scheme process, something like "stty
--file=/dev/pts/N -icanon" should be run.

However, before doing this, some warnings are in order:

When in noncanonical mode, the normal processing of ERASE (usually DEL
or Ctrl-H) and KILL (usually Ctrl-U) characters are disabled, and input
characters are delivered to the subprocess immediately as they are
typed, rather than waiting for the newline as normally happens in
canonical mode.

At least in the case of the Guile REPL, one notable side effect of
running in noncanonical mode is that when a list is entered at the REPL,
the 'read' returns as soon as the final close parenthesis is entered.
Nothing after that is read, not even the usual newline.  The final
newline is only read if the reader is not yet sure that it has finished
reading the token, e.g. if a number or symbol is entered.  In those
cases, typically any delimiter may be typed to terminate the read,
e.g. space.

To see this, you can try running Guile from a traditional terminal
program (e.g. xterm or GNOME Terminal), and type:

  (system* "stty" "-icanon")

and then:

  (display "hello")

You will see that as soon as you type that close paren, "hello" is
immediately printed, followed by another REPL prompt, all on the same
line.

You might also try (use-modules (ice-9 rdelim)) and then:

  (read-line)

and you'll see that the newline you type at the end of that line is read
by 'read-line', which then immediately returns the empty string.  The
input that you wish for 'read-line' to see must be typed on the same
line, immediately after the close parenthesis.

So, it might be that Geiser needs to be adjusted somewhat to deal with
these differences.

Finally, you might consider the possibility that 'stty' might not be
available in PATH, or fails for some reason, and ideally this case
should be handled as well.

It might be simpler to always use REPL servers over a socket, than to
deal with these headaches, although I don't know if that will be an
option for the other Scheme implementations.

Regards,
  Mark

bug#33403: [Geiser-users] Data length limit in Guile/Geiser/Scheme evaluation

[Mark H Weaver]

Mark H Weaver  writes:

> If, after pasting this, you type another close quote, 5 close parens,
> and then repaste the last two lines, it will print the garbled input and
> return to a prompt.

Actually, instead of pasting the last two lines as-is, I replaced
"(length classification)" with "classification", so that instead of
printing the length, it prints the actual s-exp.  Then you can see what
happened to that final string literal.

> Anyway, to make a long story short, after some debugging, I found that
> precisely the same truncation of the first line happens when using 'cat'
> from GNU coreutils.  Simply type 'cat' and paste the same text, and
> you'll see that in the output, only the first 4095 bytes of the first
> line were retained.
>
> So, I'm not sure where the problem is, but it's not a problem in Guile.

This is a documented limitation in Linux's terminal handling when in
canonical mode.  See the termios(3) man page, which includes this text:

   Canonical and noncanonical mode
   
   The setting of the ICANON canon flag in c_lflag determines
   whether the terminal is operating in canonical mode (ICANON set)
   or noncanonical mode (ICANON unset).  By default, ICANON is set.

   In canonical mode:

   * Input is made available line by line.  An input line is
 available when one of the line delimiters is typed (NL, EOL,
 EOL2; or EOF at the start of line).  Except in the case of EOF,
 the line delimiter is included in the buffer returned by
 read(2).

   * Line editing is enabled (ERASE, KILL; and if the IEXTEN flag is
 set: WERASE, REPRINT, LNEXT).  A read(2) returns at most one
 line of input; if the read(2) requested fewer bytes than are
 available in the current line of input, then only as many bytes
 as requested are read, and the remaining characters will be
 available for a future read(2).

   * The maximum line length is 4096 chars (including the
 terminating newline character); lines longer than 4096 chars
 are truncated.  After 4095 characters, input processing (e.g.,
 ISIG and ECHO* processing) continues, but any input data after
 4095 characters up to (but not including) any terminating
 newline is discarded.  This ensures that the terminal can
 always receive more input until at least one line can be read.

Note that last item above.

   Mark

bug#33403: [Geiser-users] Data length limit in Guile/Geiser/Scheme evaluation

[Mark H Weaver]

Hi Neil,

Neil Jerram  writes:

> Hi, this is a report for Guile 2.2:
>
> neil@henry:~$ guile --version
> guile (GNU Guile) 2.2.3
> Packaged by Debian (2.2.3-deb+1-3ubuntu0.1)
>
> I'm seeing something that looks like a line or sexp length limit when
> reading from a terminal.  Sample inputs are in the attached file.
>
>
>
> If I run guile in a terminal (GNOME Terminal 3.28.2), select the first
> block from the file, and use my middle mouse button to paste it into the
> guile prompt, I get the expected answer:
>
> $4 = 139
> scheme@(guile-user)> 
>
> If I do the same with the second block, I get no response, and it
> appears that Guile has hung in some way.  I have to type C-c to get a
> new prompt:
>
> ^C^CWhile reading expression:
> User interrupt
> scheme@(guile-user)> 
>
> The max line length for the first block is 4087.  For the second it's
> 4113.  Could there be a 4K buffer or limit involved somewhere?

Indeed, I can reproduce the same issue when pasting into an Emacs shell
buffer.  I've verified that Guile only receives the first 4095 bytes of
the first line.  The following characters from the end of the first line
are lost:

A " "Aub")

So the second and third lines of the input become part of the string
literal whose closing quote was lost, and Guile's reader continues to
wait for a closing quote.

If, after pasting this, you type another close quote, 5 close parens,
and then repaste the last two lines, it will print the garbled input and
return to a prompt.

Anyway, to make a long story short, after some debugging, I found that
precisely the same truncation of the first line happens when using 'cat'
from GNU coreutils.  Simply type 'cat' and paste the same text, and
you'll see that in the output, only the first 4095 bytes of the first
line were retained.

So, I'm not sure where the problem is, but it's not a problem in Guile.

 Regards,
   Mark

bug#22925: ice-9/match named match-let is not working

[Mark H Weaver]

Stefan Israelsson Tampe  writes:
> This is fixed in the latest upstream match.scm released in the chibi
> repo.

Thanks, and sorry for the long delay.  This is now fixed in commit
8e86dd93a0640161fe0098a80ccc9b814280 on the stable-2.2 branch.  That
commit also includes several other fixes from the upstream match.scm in
Chibi-Scheme.

  Mark

bug#33340: named match-let doesn't work

[Mark H Weaver]

Alex Kost  writes:

> Ernesto Gabriel (2018-11-11 00:44 -0300) wrote:
>
>> match-let works as advertised, but a named match-let gives errors
>
> I think this is the same as:
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=22925

Indeed, thanks for pointing that out.  The bug is now fixed by commit
8e86dd93a0640161fe0098a80ccc9b814280 on the stable-2.2 branch.  That
commit also includes several other fixes from the upstream match.scm in
Chibi-Scheme.

  Mark

bug#26164: time-difference mishandles leap seconds

[Mark H Weaver]

Zefram  writes:

> Mark H Weaver wrote:
>>   every UTC day has
>>exactly 86400 UTC seconds,
>
> No, that's not how UTC works.  There are some time scales derived from UTC
> that have exactly 86400 seconds for each UTC day, such as Markus Kuhn's
> UTC-SLS, or that have exactly 86400 seconds per UTC day in the long run,
> such as Google's "leap smear".  But SRFI-19 doesn't refer to any of those,
> it refers to UTC.

BTW, I discussed this with John Cowan at length in bug 22034, starting
at message #19:

  https://bugs.gnu.org/22034#19

In particular, I would be curious to know how you would fill in the same
table that John attempted to fill in, here:

  https://bugs.gnu.org/22034#55

What numbers would you write in the second column of those tables?

Thanks,
  Mark

bug#26164: time-difference mishandles leap seconds

[Mark H Weaver]

Zefram  writes:

> Mark H Weaver wrote:
>>Having said all of this, I should admit that I'm not an expert on time
>>standards,
>
> I am.

Okay, you claim to be one, and maybe you're right, but I've also done a
great deal of research on this recently and in the past, and I'm not yet
convinced.

If you're right that there's still a problem, please post to the SRFI-19
mailing list and engage with the time experts there.  If you can
convince them to change the behavior of 'time-difference' in their
reference implementation, then I'll be glad to merge those changes into
Guile.

 Thanks,
   Mark

bug#22034: time-utc->date shows bogus zone-dependent leap second

[Mark H Weaver]

Hi John,

John Cowan  writes:

> On Mon, Oct 29, 2018 at 3:17 AM Mark H Weaver  wrote:
>
>  Can you please be more concrete and tell me what numbers you think
>  should be in the second column, to properly reflect the column heading?
>  I'm not asking for a prose description, but for the actual numbers.
>
> Here you go:
>
> +-+
> |  TAI seconds  UTC seconds Posix seconds |
> | since   sincesince  |
> | midnight TAI  midnight UTC midnight UTC |
> |  1 Jan 1970   1 Jan 1970   1 Jan 1970Result of 'time-tai->date' |
> |-|
> |$2 = ((126230410126230398126230398"1973-12-31T23:59:58Z")|
> |  (126230411126230399126230399"1973-12-31T23:59:59Z")|
> |  (126230412126230400126230400"1973-12-31T23:59:60Z")|
> |  (126230413126230401126230400"1974-01-01T00:00:00Z")|
> |  (126230414126230402126230401"1974-01-01T00:00:01Z"))   |
> +-+

Thank you, this is helpful.

> So as you see leap seconds are included in both the TAI and the UTC count,
> but not in the Posix count.

According to <http://maia.usno.navy.mil/ser7/tai-utc.dat>, the value of
TAI-UTC should change from 12 to 13 at JD 2442048.5, i.e. at midnight
UTC on 1 January 1974.  The same fact is shown on the graph at the
bottom of this page:

  http://jjy.nict.go.jp/mission/page1-e.html

In the values you proposed above, TAI-UTC is 12 uniformly throughout,
both before and after the leap second.

If you believe that TAI-UTC should not change at JD 2442048.5, then for
consistency, you should believe that it doesn't change at JD 2441683.5,
the previous leap second one year earlier:

--8<---cut here---start->8---
scheme@(guile-user)> ,pp (map (lambda (n)
(let* ((tai (make-time time-tai 0 n))
   (utc (time-tai->time-utc tai)))
  (list (time-second tai)
(time-second utc)
(date->string (time-tai->date tai 0)
  "~4"
  (iota 5 94694409))
$1 = ((94694409 94694398 "1972-12-31T23:59:58Z")
  (94694410 94694399 "1972-12-31T23:59:59Z")
  (94694411 94694400 "1972-12-31T23:59:60Z")
  (94694412 94694400 "1973-01-01T00:00:00Z")
  (94694413 94694401 "1973-01-01T00:00:01Z"))
--8<---cut here---end--->8---

If we apply the same changes here that you did above, with UTC=Posix at
the top and UTC=Posix+1 at the bottom, it would look like this:

> +-+
> |  TAI seconds  UTC seconds Posix seconds |
> | since   sincesince  |
> | midnight TAI  midnight UTC midnight UTC |
> |  1 Jan 1970   1 Jan 1970   1 Jan 1970Result of 'time-tai->date' |
> |-|
> |$2 = ((946944099469439894694398"1972-12-31T23:59:58Z")|
> |  (946944109469439994694399"1972-12-31T23:59:59Z")|
> |  (946944119469440094694400"1972-12-31T23:59:60Z")|
> |  (946944129469440194694400"1973-01-01T00:00:00Z")|
> |  (946944139469440294694401"1973-01-01T00:00:01Z"))   |
> +-+

Is that a reasonable extrapolation of what you did?

Note that in the table above, TAI-UTC is 11 uniformly throughout.  This
raises of the question of how to combine these two tables into a
coherent whole.  In order to produce a larger table that includes both
of these excerpts and everything in between, TAI-UTC would need to
change from 11 to 12 somewhere in the middle.

Do you see the problem?

Would you like to make another suggestion of what values should go in
the second column of these two tables?

 Thanks,
   Mark

bug#22034: time-utc->date shows bogus zone-dependent leap second

[Mark H Weaver]

Mark H Weaver  writes:

> John Cowan  writes:
>
>> On Sun, Oct 28, 2018 at 4:40 PM Mark H Weaver  wrote:
>>
>>  The difference between the two measuring tapes is that they assign
>>  different numbers to the markings, and moreover that the UTC analogue
>>  has a small handful of places where two adjacent markings have the same
>>  number assigned, and all subsequent numbers are shifted by 1.
>>
>> Now I think you are entirely right here, modulo a single term: what you are
>> calling "UTC", I call (and I think correctly), "Posix".  It is Posix time 
>> that
>> has two identical adjacent markings.
>>
>>126230400 |
>>126230400 |
>>
>> The numbers here are not UTC seconds since the Epoch, but Posix seconds
>> since the Epoch.
>
> Here's a more detailed display of the same leap second that I chose for
> my example, which you quoted above:
>
> +---+
> |  TAI seconds  UTC seconds |
> | since   since |
> | midnight UTC  midnight UTC|
> |  1 Jan 1970   1 Jan 1970Result of 'time-tai->date'|
> |---|
> |$2 = ((126230410126230398"1973-12-31T23:59:58Z")   |
> |  (126230411126230399"1973-12-31T23:59:59Z")   |
> |  (126230412126230400"1973-12-31T23:59:60Z")   |
> |  (126230413126230400"1974-01-01T00:00:00Z")   |
> |  (126230414126230401"1974-01-01T00:00:01Z"))  |
> +---+
>
> The table above illustrates my understanding of the relationship between
> "TAI seconds since midnight UTC on 1 Jan 1970", "UTC seconds since
> midnight UTC on 1 Jan 1970" and the date object expressed in UTC.  See
> my previous email for the code to produce the table above using SRFI-19.

Sorry, I made a mistake above.  Where I wrote:

  "TAI seconds since midnight UTC on 1 Jan 1970"

I should have written:

  "TAI seconds since midnight TAI on 1 Jan 1970"

I'm not aware of any other mistakes in my last message.

Given that correction to the heading over the first column, I'd still
like to know what numbers you believe should be in a corrected version
of the table above.

I admit that this is a surprisingly challenging subject, but I think
it's important to get to the bottom of this.  If there's a deep problem
in SRFI-19 regarding its interpretation of UTC seconds, we need to know
about it and fix it.  This issue affects the entire Scheme community.

 Thanks,
   Mark

bug#22034: time-utc->date shows bogus zone-dependent leap second

[Mark H Weaver]

Hi John,

John Cowan  writes:

> On Sun, Oct 28, 2018 at 4:40 PM Mark H Weaver  wrote:
>
>  The difference between the two measuring tapes is that they assign
>  different numbers to the markings, and moreover that the UTC analogue
>  has a small handful of places where two adjacent markings have the same
>  number assigned, and all subsequent numbers are shifted by 1.
>
> Now I think you are entirely right here, modulo a single term: what you are
> calling "UTC", I call (and I think correctly), "Posix".  It is Posix time that
> has two identical adjacent markings.
>
>126230400 |
>126230400 |
>
> The numbers here are not UTC seconds since the Epoch, but Posix seconds
> since the Epoch.

Here's a more detailed display of the same leap second that I chose for
my example, which you quoted above:

+---+
|  TAI seconds  UTC seconds |
| since   since |
| midnight UTC  midnight UTC|
|  1 Jan 1970   1 Jan 1970Result of 'time-tai->date'|
|---|
|$2 = ((126230410126230398"1973-12-31T23:59:58Z")   |
|  (126230411126230399"1973-12-31T23:59:59Z")   |
|  (126230412126230400"1973-12-31T23:59:60Z")   |
|  (126230413126230400"1974-01-01T00:00:00Z")   |
|  (126230414126230401"1974-01-01T00:00:01Z"))  |
+---+

The table above illustrates my understanding of the relationship between
"TAI seconds since midnight UTC on 1 Jan 1970", "UTC seconds since
midnight UTC on 1 Jan 1970" and the date object expressed in UTC.  See
my previous email for the code to produce the table above using SRFI-19.

If I understand correctly, based on your last sentence that I quoted
above, you believe that the numbers I've given in the second column are
not UTC seconds since the epoch, and that there is confusion here
between UTC seconds and Posix seconds.

Can you please be more concrete and tell me what numbers you think
should be in the second column, to properly reflect the column heading?
I'm not asking for a prose description, but for the actual numbers.

If the table above is incorrect, then it would be good to report the
problem to the SRFI-19 mailing list, because the current SRFI-19
reference implementation produces the same table.

 Thanks,
   Mark

bug#22034: time-utc->date shows bogus zone-dependent leap second

[Mark H Weaver]

Hi John,

John Cowan  writes:

> On Mon, Oct 22, 2018 at 2:12 AM Mark H Weaver  wrote:
>
>  Universal Time (UT) is not a measure of physical time, but rather is a
>  measure of the rotation angle of the Earth with respect to distant
>  quasars.  A UT second is identified with a fixed amount of rotation of
>  the Earth, which equals 1/86400 of a mean solar day.  That's why every
>  day has 86400 UT seconds.  
>
> Quite right.  Buit the whole point of UTC is that its seconds are not angles,
> but SI = TAI seconds.

I believe you're making a subtle error in your identification of UTC
seconds with TAI seconds.  It might be helpful to consider an analogy to
spatial measurement.

A TAI clock aims to measure the current TAI time, i.e. to measure the
time interval between the epoch and _you_ in units of TAI seconds,
i.e. SI seconds as observed on the geoid.

In spatial terms, a TAI clock is analogous to a very precise measuring
tape which aims to measure the distance between you and an fixed
conventional point in space, analogous to the TAI epoch.

In this analogy, a UTC clock is analogous to an equally precise
measuring tape that is _almost_ identical to the TAI analogue measuring
tape, but subtly different.  If we place the two measuring tapes side by
side and ignore pre-1972, we see that the markings are in _precisely_
the same positions along the entire length of the tape, without the
slightest deviation, even over long distances.

The difference between the two measuring tapes is that they assign
different numbers to the markings, and moreover that the UTC analogue
has a small handful of places where two adjacent markings have the same
number assigned, and all subsequent numbers are shifted by 1.  Such a
place might look something like this:

  126230388 |--
  126230389 |--
  126230390 |
  126230391 |--
  126230392 |--
  126230393 |--
  126230394 |--
  126230395 |--
  126230396 |--
  126230397 |--
  126230398 |--
  126230399 |--
  126230400 |
  126230400 |
  126230401 |--
  126230402 |--
  126230403 |--
  126230404 |--
  126230405 |--
  126230406 |--
  126230407 |--
  126230408 |--
  126230409 |--
  126230410 |
  126230411 |--
  126230412 |--

By asserting that UTC seconds are the same as TAI seconds, you're
emphasizing that the distance between any two adjacent markings are
precisely the same on the two measuring tapes.

What I'm trying to say is that when you use these two measuring tapes to
measure the interval between two arbitrary points, they will give
different answers, unless the entire interval happens to be between two
adjacent leap seconds (and post-1971).

I think it's highly questionable to claim that these two measuring tapes
measure the same units, given that when they are actually used to
measure intervals, they give different answers, and moreover that the
discrepancy grows without bound as larger intervals are measured, even
in an idealized thought experiment.

> There are a variable number of these in a day, and a UTC clock will
> indeed report 23:59:60 at the end of a day with a leap second in it

That's something that can be done when you convert TIME-TAI into a UTC
date, and indeed 'time-tai->date' from SRFI-19 will do this:

--8<---cut here---start->8---
scheme@(guile-user)> ,pp (map (lambda (n)
(let* ((tai (make-time time-tai 0 n)))
  (list (time-second tai)
(date->string (time-tai->date tai 0)
  "~4"
  (iota 5 126230410))
$1 = ((126230410 "1973-12-31T23:59:58Z")
  (126230411 "1973-12-31T23:59:59Z")
  (126230412 "1973-12-31T23:59:60Z")
  (126230413 "1974-01-01T00:00:00Z")
  (126230414 "1974-01-01T00:00:01Z"))
--8<---cut here---end--->8---


However, let's return to our disagreement which started this digression:

> On Sat, Oct 20, 2018 at 5:43 PM Mark H Weaver  wrote:
>
>  If I understand correctly, 'time-utc->date' should never return a date
>  object with 60 in the seconds field, because those extra seconds have no
>  representation in time-utc.  They only have representations in time-tai
>  and time-monotonic.
>
> As I understand it, this is incorrect.  UTC days can contain either
> 86400 or 86401 seconds (or in principle a different number), depending
> on whether the day has a leap second.

You seem to be asserting above that TIME-UTC _does_ have representations
of the extra leap seconds.  That is simply not true.  It's true that
*date objects* for UTC, broken down into separate fields, are able to
represent the leap seconds by putting 60 in the seconds field.  However,
TIME-UTC, represented 

bug#26632: TAI<->UTC conversion botches 1961 to 1971

[Mark H Weaver]

Hi Zefram,

Zefram  writes:
> The SRFI-19 library gets TAI<->UTC conversions badly wrong in the years
> 1961 to 1971 (inclusive).

Indeed.

> This has to be examined somewhat indirectly, because SRFI-19 doesn't offer
> any way to display a TAI time in its conventional form as a date-like
> structure, nor to input a TAI time from such a structure.

FWIW, here are two procedures I hacked up to support TAI dates:

--8<---cut here---start->8---
(define (time-tai->date-tai t . tz-offset)
  (apply time-tai->date
 (time-utc->time-tai! (make-time time-utc
 (time-nanosecond t)
 (time-second t)))
 tz-offset))

(define (date-tai->time-tai d)
  (let ((t (time-tai->time-utc (date->time-tai d
(set-time-type! t time-tai)
t))
--8<---cut here---end--->8---

> SRFI-19's date structure, as implemented, is always interpreted
> according to UTC.

Indeed, I discovered this as well, and found it surprising.  The text of
SRFI-19 fails to mention it, but the reference implementation makes it
quite clear.

> First I'll consider an ordinary day in 1967:
>
> scheme@(guile-user)> (use-modules (srfi srfi-19))
> scheme@(guile-user)> (time-difference
> ... (time-utc->time-tai (date->time-utc (make-date 0 0 0 0 15 3 1967 0)))
> ... (time-utc->time-tai (date->time-utc (make-date 0 0 0 0 14 3 1967 0
> $1 = #
>
> This takes the start and end of 1967-03-14, as judged by UTC, converts
> both of these times to TAI, and asks for the duration of that TAI
> interval.  It's asking how many TAI seconds long that UTC day was.
> As described in <http://maia.usno.navy.mil/ser7/tai-utc.dat>, there
> was no UTC leap on that day, but throughout 1967 UTC had a frequency
> offset from TAI such that each UTC second lasted exactly 1.0003 TAI
> seconds.  The correct answer to the above question is therefore exactly
> 86400.002592 s.  The answer shown above, of 86400.00 s, is incorrect.
>
> If time-tai->time-utc is applied to the times in the above example,
> it accurately inverts what time-utc->time-tai did.  It is good that the
> conversions are mutually consistent, but in this case it means they are
> both wrong.
>
> Second, I'll consider a less ordinary day:
>
> scheme@(guile-user)> (time-difference
> ... (time-utc->time-tai (date->time-utc (make-date 0 0 0 12 1 2 1968 0)))
> ... (time-utc->time-tai (date->time-utc (make-date 0 0 0 12 31 1 1968 0
> $2 = #
>
> This time the period considered is from noon 1968-01-31 to noon
> 1968-02-01.  The same frequency offset described above applies throughout
> this period.  The additional complication here is that at the end of
> 1968-01-31 there was a leap of -0.1 (TAI) seconds.  The true duration of
> this day is therefore exactly 86399.902592 s.  The answer shown above,
> of 86400.00 s, is incorrect in two ways, accounting for neither the
> frequency offset nor the leap.

I've attached two patches to fix this bug.  The first lays the
groundwork by adding support for non-integer TAI-UTC deltas.  The second
patch adds the TAI-UTC tables for 1961-1971 and uses them to implement
TAI<->UTC conversions over that time range with nanosecond accuracy.

Although the code is now written, I'm unsure whether we should add it to
Guile.  I'm vaguely concerned about violating widely-held assumptions,
e.g. that UTC runs at the same rate as TAI (except when leap seconds are
introduced), which might cause some code on top of Guile to misbehave if
the system clock is set pre-1972, although admittedly such a scenario
seems unlikely.

I'm curious to hear opinions on this.

Anyway, here are the patches.

  Mark


>From 3a67fbfd441b39630ff3c3201d2a731b51b1a8ee Mon Sep 17 00:00:00 2001
From: Mark H Weaver 
Date: Tue, 23 Oct 2018 00:39:30 -0400
Subject: [PATCH 1/2] SRFI-19: TAI<->UTC conversions support non-integer
 deltas.

Previously, 'utc->tai' and 'tai->utc' were unary procedures and returned
a second value, mapping seconds to seconds, based on the assumption that
the nanoseconds field would never be changed by these maps.  To support
non-integer values of TAI-UTC, here we change 'utc->tai' and 'tai->utc'
to accept two arguments and return two values: nanoseconds and seconds.

* module/srfi/srfi-19 (utc->tai, tai->utc): Add an additional 'ns'
argument, and return it as an additional value.
(current-time-tai, priv:time-tai->time-utc!, priv:time-utc->time-tai!)
(time-tai->julian-day, time-monotonic->julian-day): Adapt accordingly.
---
 module/srfi/srfi-19.scm | 56 +++--
 1 file changed, 31 insertions(+), 25 deletions(-)

diff --git a/module/srfi/srfi-1

bug#22034: time-utc->date shows bogus zone-dependent leap second

[Mark H Weaver]

John Cowan  writes:

> On Sat, Oct 20, 2018 at 5:43 PM Mark H Weaver  wrote:
>
>  If I understand correctly, 'time-utc->date' should never return a date
>  object with 60 in the seconds field, because those extra seconds have no
>  representation in time-utc.  They only have representations in time-tai
>  and time-monotonic.
>
> As I understand it, this is incorrect.  UTC days can contain either
> 86400 or 86401 seconds (or in principle a different number),

If you're talking about TAI seconds, then I agree.  However, the
'time-utc' representation in SRFI-19 is not a count of TAI seconds, but
rather a count _UTC_ seconds since the POSIX epoch.

Under the current "leap second" method of keeping UTC within 0.9 seconds
of UT1, UTC days contain either 86399, 86400, or 86401 _TAI_ seconds.
So far, all leap seconds have increased the TAI-UTC delta, but if the
(irregular) rotation rate of the Earth speeds up, we might some day need
a leap second that decreases the TAI-UTC delta.

Universal Time (UT) is not a measure of physical time, but rather is a
measure of the rotation angle of the Earth with respect to distant
quasars.  A UT second is identified with a fixed amount of rotation of
the Earth, which equals 1/86400 of a mean solar day.  That's why every
day has 86400 UT seconds.  UTC is kept within 0.9 seconds of UT1 (a
version of UT with certain corrections applied), so over long time
periods, with the leap seconds taken into account, UTC seconds are equal
to UT seconds.

   Mark

bug#21912: TAI<->UTC conversion botches the unknown

[Mark H Weaver]

Hi John,

John Cowan  writes:

> What is more, there are no TAI<->UTC conversion tables from before
> 1961 (when UTC began) and probably never will be.  There was heated
> debate in the R7RS-small working group, and we finally settled on a
> compromise:
>
> (current-second) [r]eturns an inexact number representing the current
> time on the International Atomic Time (TAI) scale. The value 0.0
> represents midnight on January 1, 1970 TAI (equivalent to ten seconds
> before midnight Universal Time) and the value 1.0 represents one TAI
> second later. Neither high accuracy nor high precision are required;
> in particular, returning Coordinated Universal Time plus a suitable
> constant might be the best an implementation can do.
>
> I now see that the "ten seconds" is incorrect, and I am filing an
> erratum: the correct figure is 4.2131700 seconds per
> .

Actually, the correct TAI-UTC delta on January 1, 1970 TAI is
approximately 8 seconds.  You are misinterpreting the relevant line from
that file:

 1968 FEB  1 =JD 2439887.5  TAI-UTC=   4.2131700 S + (MJD - 39126.) X 0.002592 S

This means that TAI-UTC = 4.2131700 + (MJD - 39126) * 0.002592, where
MJD is the modified julian day of the desired TAI-UTC delta.  In this
case, the MJD (modified julian day) of midnight UTC January 1, 1970 is
40587, and plugging that into the equation above yields TAI-UTC =
8.82 seconds.

If you are doubtful, see the "Atomic Time and Leap Seconds" graph, which
shows a graph of TAI-UTC over the years 1958 to 2017, on the following page:

  http://jjy.nict.go.jp/mission/page1-e.html

Note that between January 1961 and January 1972, the TAI-UTC delta was
not an integer, and instead of the discontinuous leap second jumps that
we've had since 1972, the two clocks ran at different rates in those
years.

   Mark

bug#26163: time-difference doesn't detect error of differing time types

[Mark H Weaver]

Zefram  writes:

> scheme@(guile-user)> (use-modules (srfi srfi-19))
> scheme@(guile-user)> (time-difference (make-time time-tai 0 1) (make-time 
> time-utc 0 1))
> $1 = #
>
> SRFI-19 is explicit that it "is an error" if the arguments to
> time-difference are of different time types, and correspondingly the
> Guile documentation says the arguments "must be" of the same type.
> It would be very easy for time-difference to detect and signal this error.
> It's not absolutely a bug that it currently doesn't, but it would be a
> useful improvement if it did.

Agreed.  Fixed in commit c9d903b6e4f8cc1d8382b20a2f0502c4ce8ffe0a on the
stable-2.2 branch.  I'm closing this bug, but feel free to reopen if
appropriate.

 Thanks!
   Mark

bug#26162: time-duration screws up negative durations

[Mark H Weaver]

Zefram  writes:

> Computing a difference between two SRFI-19 times, using time-difference,
> produces sensible results if the result is positive, but often nonsense
> if it's negative:
>
> scheme@(guile-user)> (use-modules (srfi srfi-19))
> scheme@(guile-user)> (time-difference (make-time time-tai 0 1) (make-time 
> time-tai 1000 0))
> $1 = #
> scheme@(guile-user)> (time-difference (make-time time-tai 1000 0) (make-time 
> time-tai 0 1))
> $2 = #
>
> The above is computing the same interval both ways round.  The first time
> is correct, but the second is obviously not the negative of the first.

This is fixed by commit 437e1aa03659b77a8eb4b5c6d2b104c03d038564 on the
stable-2.2 branch.  I'm closing this bug now, but feel free to reopen if
appropriate.

> The correct result for the second would be
>
> #
>
> or possibly, at a stretch,
>
> #

Most of the code in the SRFI-19 reference implementation assumes that
the nanoseconds and seconds fields will have matching signs (if both
non-zero), so I've taken care to use that convention exclusively.

 Thanks,
   Mark

bug#26164: time-difference mishandles leap seconds

[Mark H Weaver]

Zefram  writes:

> Computing the duration of the period between two UTC times, using
> SRFI-19 mechanisms:
>
> scheme@(guile-user)> (use-modules (srfi srfi-19))
> scheme@(guile-user)> (define t0 (date->time-utc (make-date 0 59 59 23 30 6 
> 2012 0))) 
> scheme@(guile-user)> (define t1 (date->time-utc (make-date 0 1 0 0 1 7 2012 
> 0)))
> scheme@(guile-user)> (time-difference t1 t0)
> $1 = #
>
> The two times are 2012-06-30T23:59:59 and 2012-07-01T00:00:01, so at
> first glance one would expect the duration to be 2 s as shown above,
> the two seconds being 23:59:59 and 00:00:00.  But in fact there was
> a leap second 2012-06-30T23:59:60, so the duration of this period is
> actually 3 s.

You seem to be assuming that SRFI-19 durations should _always_ represent
intervals of TAI time.  While I agree that TAI time is often the right
choice for durations, there are other cases where monotonic time is a
better choice.  Currently, monotonic time == TAI time, but SRFI-19 makes
it clear that this needn't be the case, and at some point we might want
to change monotonic time to be _truly_ monotonic, even in cases where
the system clock jumps backward.

Durations in UTC time have uses as well.  For example, every UTC day has
exactly 86400 UTC seconds, so (make-time time-duration 0 86400) means 1
UTC day, when added to a UTC time.  This is a duration that, when added
to any UTC time, always leaves the time-of-day unchanged.  There is no
fixed duration of TAI time that has this property, because not all UTC
days have the same number of TAI seconds.

> [...]  Since 1972, the seconds of UTC are of exactly
> the same duration as the seconds of TAI.  (They're also phase-locked to
> TAI seconds.)  Thus the period of three TAI seconds is also a period of
> three UTC seconds.  It is not somehow squeezed into two UTC seconds.

I believe you are mistaken here.  Not all UTC seconds have the same
duration as a TAI second.  Some TAI seconds correspond to 0 UTC seconds,
and maybe some day there will be a TAI second that corresponds to 2 UTC
seconds.

Having said all of this, I should admit that I'm not an expert on time
standards, so perhaps I've misunderstood something.

What do you think?

  Mark

bug#26151: date-year-day screws up leap days prior to AD 1

[Mark H Weaver]

This is fixed by commit a58c7abd72648f77e4ede5f62a2c4e7969bb7f95 on the
stable-2.2 branch.  I'm closing this bug now, but please reopen if
appropriate.

 Thanks!
   Mark

bug#26165: date-week-day screws up prior to AD 1

[Mark H Weaver]

Zefram  writes:

> Looking at day of the week, via SRFI-19's date-week-day:
>
> scheme@(guile-user)> (use-modules (srfi srfi-19))
> scheme@(guile-user)> (julian-day->date 1721426 0) 
> $1 = # 1 zone-offset: 0>
> scheme@(guile-user)> (date-week-day (julian-day->date 1721426 0))
> $2 = 1
> scheme@(guile-user)> (date-week-day (julian-day->date 1721425 0))
> $3 = 6
>
> The output for 0001-01-01, Monday, is correct.  The preceding day is
> actually a Sunday, but Saturday was shown.  Looking at the code, this
> bug arises for the same reason as the problem with date-year-day raised
> in bug#26151.

This is fixed by commit a58c7abd72648f77e4ede5f62a2c4e7969bb7f95 on the
stable-2.2 branch.  I'm closing this bug now, but please reopen if
appropriate.

 Thanks!
   Mark

bug#32367: 2.2.4 hangs when a script uses a module that calls sigaction

[Mark H Weaver]

I've attached a preliminary patch to fix this bug.

  Mark

>From 897a6f76280612e83f48d63430bf962520c0e7b3 Mon Sep 17 00:00:00 2001
From: Mark H Weaver 
Date: Sun, 21 Oct 2018 09:56:16 -0400
Subject: [PATCH] DRAFT: Fix thread-safe module loading.

* module/ice-9/boot-9.scm (%modules-being-loaded)
(%local-modules-being-loaded, %modules-waiting-for): New variables.
(%force-lazy-module-cell!, %module-waiting-for?)
(%module-waiting-for!): New procedures.
(resolve-module): If the requested module is not in the regular global
module table, look in '%local-modules-being-loaded' and
'%modules-being-loaded', and handle these cases appropriately.  Support
looping without recursively locking the autoload lock.  When
autoloading, unlock the mutex before calling 'try-load-module'.
(try-module-autoload): Add entries to '%modules-being-loaded' and
'%local-modules-being-loaded' before loading the module.  Also, load the
module with the autoload mutex unlocked.  When the load attempt
finishes (or fails), add the module to the regular global module table
if it was ever created, signal the threads waiting for this module, and
remove it from the '*-begin-loaded' and '%modules-waiting-for' tables.
(call-with-module-autoload-lock): Accept a unary procedure instead of a
thunk.
(module-name): Adapt to the new 'call-with-module-autoload-lock'.
(nested-define-module!): If we're asked to define a submodule of a
module that's currently being loaded, install the parent module being
loaded into the global module table.
* module/ice-9/threads.scm (call-with-module-autoload-lock):
Pass the mutex as an argument to the procedure.
* test-suite/tests/threads.test: Add tests.
* test-suite/tests/delayed-test.scm,
test-suite/tests/mutual-delayed-a.scm,
test-suite/tests/mutual-delayed-b.scm,
test-suite/tests/mutual-delayed-c.scm: New files.
* test-suite/Makefile.am (EXTRA_DIST): Add them.
---
 module/ice-9/boot-9.scm   | 292 ++
 module/ice-9/threads.scm  |   4 +-
 test-suite/Makefile.am|   7 +-
 test-suite/tests/delayed-test.scm |  28 +++
 test-suite/tests/mutual-delayed-a.scm |  29 +++
 test-suite/tests/mutual-delayed-b.scm |  29 +++
 test-suite/tests/mutual-delayed-c.scm |  29 +++
 test-suite/tests/threads.test |  66 +-
 8 files changed, 435 insertions(+), 49 deletions(-)
 create mode 100644 test-suite/tests/delayed-test.scm
 create mode 100644 test-suite/tests/mutual-delayed-a.scm
 create mode 100644 test-suite/tests/mutual-delayed-b.scm
 create mode 100644 test-suite/tests/mutual-delayed-c.scm

diff --git a/module/ice-9/boot-9.scm b/module/ice-9/boot-9.scm
index d8801dada..404a19d49 100644
--- a/module/ice-9/boot-9.scm
+++ b/module/ice-9/boot-9.scm
@@ -2502,13 +2502,32 @@ interfaces are added to the inports list."
  (tail (cdr names)))
 (if (null? tail)
 (module-define-submodule! cur head module)
-(let ((cur (or (module-ref-submodule cur head)
-   (let ((m (make-module 31)))
- (set-module-kind! m 'directory)
- (set-module-name! m (append (module-name cur)
- (list head)))
- (module-define-submodule! cur head m)
- m
+(let ((cur
+   (or (module-ref-submodule cur head)
+   (let ((dir-name (append (module-name cur)
+   (list head
+ (cond ((assoc dir-name %modules-being-loaded)
+=> (lambda (entry)
+ ;; The module we're being asked to define
+ ;; is a submodule of a module that's
+ ;; currently being loaded.  In this case,
+ ;; we must install the parent module
+ ;; being loaded into the global module
+ ;; table.  This is unfortunate, but it's
+ ;; not clear how to avoid this without
+ ;; changing the structure of the global
+ ;; module table.
+ (let ((m (%force-lazy-module-cell!
+   (cddr entry)
+   dir-name)))
+   (module-define-submodule! cur head m)
+   m)))
+   (else
+(let ((m (make-module 31)))
+  (set-module-kind! m 'directory)
+  (set-module-name! m dir-name)
+  (module-define-submodule!

bug#31878: Module autoloading is not thread safe

[Mark H Weaver]

I've written a preliminary patch to implement the improved thread-safe
module autoloading that I outlined in earlier messages in this bug
report.

Comments, suggestions, and testing welcome.

  Mark


>From 897a6f76280612e83f48d63430bf962520c0e7b3 Mon Sep 17 00:00:00 2001
From: Mark H Weaver 
Date: Sun, 21 Oct 2018 09:56:16 -0400
Subject: [PATCH] DRAFT: Fix thread-safe module loading.

* module/ice-9/boot-9.scm (%modules-being-loaded)
(%local-modules-being-loaded, %modules-waiting-for): New variables.
(%force-lazy-module-cell!, %module-waiting-for?)
(%module-waiting-for!): New procedures.
(resolve-module): If the requested module is not in the regular global
module table, look in '%local-modules-being-loaded' and
'%modules-being-loaded', and handle these cases appropriately.  Support
looping without recursively locking the autoload lock.  When
autoloading, unlock the mutex before calling 'try-load-module'.
(try-module-autoload): Add entries to '%modules-being-loaded' and
'%local-modules-being-loaded' before loading the module.  Also, load the
module with the autoload mutex unlocked.  When the load attempt
finishes (or fails), add the module to the regular global module table
if it was ever created, signal the threads waiting for this module, and
remove it from the '*-begin-loaded' and '%modules-waiting-for' tables.
(call-with-module-autoload-lock): Accept a unary procedure instead of a
thunk.
(module-name): Adapt to the new 'call-with-module-autoload-lock'.
(nested-define-module!): If we're asked to define a submodule of a
module that's currently being loaded, install the parent module being
loaded into the global module table.
* module/ice-9/threads.scm (call-with-module-autoload-lock):
Pass the mutex as an argument to the procedure.
* test-suite/tests/threads.test: Add tests.
* test-suite/tests/delayed-test.scm,
test-suite/tests/mutual-delayed-a.scm,
test-suite/tests/mutual-delayed-b.scm,
test-suite/tests/mutual-delayed-c.scm: New files.
* test-suite/Makefile.am (EXTRA_DIST): Add them.
---
 module/ice-9/boot-9.scm   | 292 ++
 module/ice-9/threads.scm  |   4 +-
 test-suite/Makefile.am|   7 +-
 test-suite/tests/delayed-test.scm |  28 +++
 test-suite/tests/mutual-delayed-a.scm |  29 +++
 test-suite/tests/mutual-delayed-b.scm |  29 +++
 test-suite/tests/mutual-delayed-c.scm |  29 +++
 test-suite/tests/threads.test |  66 +-
 8 files changed, 435 insertions(+), 49 deletions(-)
 create mode 100644 test-suite/tests/delayed-test.scm
 create mode 100644 test-suite/tests/mutual-delayed-a.scm
 create mode 100644 test-suite/tests/mutual-delayed-b.scm
 create mode 100644 test-suite/tests/mutual-delayed-c.scm

diff --git a/module/ice-9/boot-9.scm b/module/ice-9/boot-9.scm
index d8801dada..404a19d49 100644
--- a/module/ice-9/boot-9.scm
+++ b/module/ice-9/boot-9.scm
@@ -2502,13 +2502,32 @@ interfaces are added to the inports list."
  (tail (cdr names)))
 (if (null? tail)
 (module-define-submodule! cur head module)
-(let ((cur (or (module-ref-submodule cur head)
-   (let ((m (make-module 31)))
- (set-module-kind! m 'directory)
- (set-module-name! m (append (module-name cur)
- (list head)))
- (module-define-submodule! cur head m)
- m
+(let ((cur
+   (or (module-ref-submodule cur head)
+   (let ((dir-name (append (module-name cur)
+   (list head
+ (cond ((assoc dir-name %modules-being-loaded)
+=> (lambda (entry)
+ ;; The module we're being asked to define
+ ;; is a submodule of a module that's
+ ;; currently being loaded.  In this case,
+ ;; we must install the parent module
+ ;; being loaded into the global module
+ ;; table.  This is unfortunate, but it's
+ ;; not clear how to avoid this without
+ ;; changing the structure of the global
+ ;; module table.
+ (let ((m (%force-lazy-module-cell!
+   (cddr entry)
+   dir-name)))
+   (module-define-submodule! cur head m)
+   m)))
+   (else
+(let ((m (make-module 31)))
+  (set-module-kind! m

bug#21904: date->string duff ISO 8601 format for non-4-digit years

[Mark H Weaver]

Mark H Weaver  writes:
> Another question is whether or not we should raise an exception when
> attempting to print a year that cannot be represented in the requested
> year format.

I thought about it some more, and I'm now inclined to think that the
approach in your patches is reasonable, or at least it's the least bad
thing we can do when asked to print a year that doesn't fit within the
standard format, given the existing SRFI-19 API.

I also just noticed that the SRFI-19's reference implementation's
formatting of negative years is very badly broken (e.g. it prints "00-2"
when the year field is -2) and Guile had the same behavior after I
applied the fix from upstream to pad the year to 4 digits.

So, for now, I went ahead and implemented the behavior that you
recommended, with one difference: where you hardcode the padding
character to #\0 when formatting years, I use the padding character
specified by the user, following the SRFI-19 reference implementation.

What do you think?

  Mark

bug#21906: julian-day->date negative input breakage

[Mark H Weaver]

Zefram  writes:

> scheme@(guile-user)> (use-modules (srfi srfi-19))
> scheme@(guile-user)> (julian-day->date 0 0)
> $1 = # year: -4714 zone-offset: 0>
> scheme@(guile-user)> (julian-day->date -1 0)
> $2 = # year: -4714 zone-offset: 0>
> scheme@(guile-user)> (julian-day->date -10 0)
> $3 = # year: -4987 zone-offset: 0>
> scheme@(guile-user)> (julian-day->date -1000 0)
> $4 = # year: -32092 zone-offset: 0>

This is fixed by commit a58c7abd72648f77e4ede5f62a2c4e7969bb7f95 on the
stable-2.2 branch.  I'm closing this bug, but please reopen if
appropriate.

 Thanks!
   Mark

bug#21903: date->string duff ISO 8601 negative years

[Mark H Weaver]

Mark H Weaver  writes:

> Zefram  writes:
>
>> The date->string function from (srfi srfi-19), used on ISO 8601 formats
>> "~1", "~4" and "~5", for years preceding AD 1, has an off-by-one error:
>>
>> scheme@(guile-user)> (use-modules (srfi srfi-19))
>> scheme@(guile-user)> (date->string (julian-day->date 0 0) "~4")
>> $1 = "-4714-11-24T12:00:00Z"
>>
>> The date in question, the JD epoch, is 24 November 4714 BC (in the
>> proleptic Gregorian calendar).  In ISO 8601 format, that year is properly
>> represented as "-4713", not "-4714", because ISO 8601 uses the AD era
>> exclusively.  4714 BC = AD -4713.
>
> I agree that this is definitely a bug, but I'm nervous about deviating
> from the SRFI-19 reference implementation, and therefore probably from
> most other implementations of SRFI-19, in this way.

Also see my comments here:

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21904#17

which mention that ISO 8601 apparently requires that the sender and
receiver agree ahead of time whether an extended format will be used, in
which case a sign is *always* required, even when printing years in the
range 0-.

   Mark

bug#21904: date->string duff ISO 8601 format for non-4-digit years

[Mark H Weaver]

Zefram  writes:

> For year numbers 1 and above, it is necessary to use more than four
> digits for the year, and that's permitted, but ISO 8601 requires that
> more than four digits are preceded by a sign.  For positive year numbers
> the sign must be "+".  So one should write "+22666" rather than "22666",
> for example.

I skimmed a draft of ISO 8601 that I was able to find gratis online:

  
https://web.archive.org/web/20171019211402/https://www.loc.gov/standards/datetime/ISO_DIS%208601-1.pdf
  
https://web.archive.org/web/2017102043/https://www.loc.gov/standards/datetime/ISO_DIS%208601-2.pdf

and also the ISO 8601 Wikipedia page:

  https://en.wikipedia.org/wiki/ISO_8601#Years

and I'm left with a different interpretation about what the standard
permits.  As the Wikipedia page says:

  To represent years before  or after , the standard also
  permits the expansion of the year representation but only by prior
  agreement between the sender and the receiver.[19] An expanded year
  representation [±Y] must have an agreed-upon number of extra year
  digits beyond the four-digit minimum, and it must be prefixed with a +
  or − sign[20] [...]

Note the words "but only by prior agreement between the sender and the
receiver", and "must have an agreed-upon number of extra year digits".

You seem to have reached the conclusion that the sender can choose the
number of digits dynamically, leaving the receiver to auto-detect the
number of digits, but that seems to contradict to requirements given
above.

My interpretation is that although ISO 8601 permits the use of expanded
year formats, it seems to require that in a given format, the year must
have a fixed number of digits, and it must _always_ include a sign.  In
other words, the receiver should know ahead of time, by prior agreement,
how many digits to expect, and there should _always_ be a sign, even if
the year happens to be in the range 0-.

In order to support years outside the range 0- and in accordance
with ISO 8601, I think that 'date->string' and 'string->date' would need
to be extended to allow the caller to specify how many digits to use in
the expanded 'year' format, presumably by adding a new format escape.
If the specified number of digits is greater than 4, then a sign would
*always* be printed.  'string->date' would know how many digits to
expect, and whether to expect a sign.

Ideally, such an extension of 'date->string' and 'string->date' would be
adopted by upstream SRFI-19.  However, if that's unsuccessful, I'd be
open to unilaterally adding such an extension.  There's precedent for
this in Guile, e.g. see our (srfi srfi-9 gnu) extensions to SRFI-9.

Another question is whether or not we should raise an exception when
attempting to print a year that cannot be represented in the requested
year format.

What do you think?

  Mark

bug#21904: date->string duff ISO 8601 format for non-4-digit years

[Mark H Weaver]

Zefram  writes:

> scheme@(guile-user)> (date->string (julian-day->date 1722000 0) "~1")
> $4 = "2-07-29"
> scheme@(guile-user)> (date->string (julian-day->date 173 0) "~1")
> $5 = "24-06-23"
> scheme@(guile-user)> (date->string (julian-day->date 200 0) "~1")
> $6 = "763-09-18"

This particular subset of bugs, for years 0-, was fixed in the
upstream SRFI-19 reference implementation, and so I included the same
fix in commit 5106377a3460e1e35daf14ea6edbe80426347155.  That fix pads
the year to have at least 4 characters with the requested padding
character (0 by default).  However, it does not handle adding the sign
where mandated by ISO 8601.

As with your related bug , I think this bug
should be reported to upstream SRFI-19, and hopefully they will take it
seriously.  I'm reluctant to have Guile deviate from most (all?) other
SRFI-19 implementations in this respect.

There's also the issue that 'string->date' would need to be fixed to
successfully parse the years as printed by 'date->string'.

Would you like to report these issues to upstream SRFI-19?

 Regards,
   Mark

bug#21903: date->string duff ISO 8601 negative years

[Mark H Weaver]

Zefram  writes:

> The date->string function from (srfi srfi-19), used on ISO 8601 formats
> "~1", "~4" and "~5", for years preceding AD 1, has an off-by-one error:
>
> scheme@(guile-user)> (use-modules (srfi srfi-19))
> scheme@(guile-user)> (date->string (julian-day->date 0 0) "~4")
> $1 = "-4714-11-24T12:00:00Z"
>
> The date in question, the JD epoch, is 24 November 4714 BC (in the
> proleptic Gregorian calendar).  In ISO 8601 format, that year is properly
> represented as "-4713", not "-4714", because ISO 8601 uses the AD era
> exclusively.  4714 BC = AD -4713.

I agree that this is definitely a bug, but I'm nervous about deviating
from the SRFI-19 reference implementation, and therefore probably from
most other implementations of SRFI-19, in this way.

I think that this bug should be reported to the SRFI-19 mailing list.

   https://srfi.schemers.org/srfi-19/

There have been several other bugs reported and fixed in upstream
SRFI-19 over the years, including some as recently as June 2017, so I'm
hopeful that they will take this bug seriously and issue a fix.

Would you like to report it to them?

   Mark

bug#21901: bit shift wrong on maximal right shift

[Mark H Weaver]

Zefram  writes:

> With Guile 2.0.11:
>
> scheme@(guile-user)> (ash 123 (ash -1 63))
> $1 = 123

Fixed in commit 1990aa916382d0afcebd5315a6d6f555949ff654 on the
stable-2.2 branch.  Closing this bug now.

Thanks very much for finding this subtle bug.

  Mark

bug#21912: TAI<->UTC conversion botches the unknown

[Mark H Weaver]

Zefram  writes:

> Guile ought to be aware of how far its leap table extends, and signal
> an error when asked to perform a TAI<->UTC conversion that falls outside
> its scope.

I sympathize with your preference to raise an error rather than return
invalid results, but unfortunately I don't think this proposal is really
practical, given the SRFI-19 API.  The main problem is that, although
it's not specified in the document, the SRFI-19 reference implementation
uses UTC for all julian days, modified julian days, and dates.  For
example, 'time-tai->date' and 'time-tai->julian-day' include an implicit
conversion to UTC.

As a result, if we were to adopt your proposal, it would never be
possible to print a date more than 6 months in the future, and for users
of stable distributions such as Debian, it is not be possible to print
the current date.

We need a way to print future dates.  In theory, we could print future
dates in TAI without the leap second table, using the convention of
printing TAI dates as in , but
that's not what SRFI-19 does, and it has no API to support that way of
printing TAI dates.

Also, it's debatable whether it's desirable to print future dates in TAI
this way, since in the far future the times will drift far away from
mean solar time, and the calendar will eventually drift from the usual
alignment of the solar year to the months on the calendar.

So, I think this is a valid issue, but I don't see how it can be
addressed within SRFI-19.  I would need to be addressed in a new SRFI,
and even then it's not clear to me what's the right way forward.

What do you think?

Regards,
  Mark

bug#22033: time-utc format is lossy

[Mark H Weaver]

tags 22033 + notabug
close 22033
thanks

Hi Zefram,

Zefram  writes:

> I wrote:
>>   These two seconds are perfectly
>>distinct parts of the UTC time scale, and the time-utc format ought to
>>preserve their distinction.
>
> This is a problematic goal.  At the time I wrote the bug report I didn't
> have a satisfactory idea of how to achieve it, but I think I've come up
> with one now.
>
> The essential problem is that the SRFI-19 time structure expects to
> encapsulate a scalar value -- as it says, a count of seconds since
> some epoch -- but there is no natural scalar representation of a UTC
> time.  Because of the irregularity imposed by its leaps, the natural
> representation of a UTC time is a two-part structure, consisting of an
> integer identifying the day and a fractional count of seconds elapsed
> within the day.  Because UTC days contain differing numbers of seconds,
> this is a variable-radix system.

More precisely, UTC days contain differing numbers of TAI seconds.
However, they contain equal numbers of UTC seconds.

I don't see how we can fix this given the definition of UTC.  UTC, when
represented as a number of seconds since some epoch, simply cannot
represent leap seconds that cause UTC to jump backwards, as all leap
seconds so far have done.  This is an inherent problem with UTC, and is
one of the reasons that TAI is more appropriate than UTC for many
applications.

Your objections here are valid, and cut to the heart of the
long-standing debate over whether leap seconds are a good idea, a debate
which continues today.  If you're curious to read more on this,
 is a good starting point.

You might also be interested to know that your idea to encode leap
seconds within the 'nanoseconds' field was also proposed by Markus Kuhn
and mentioned by Olin Shivers on the SRFI-19 mailing list during the
early discussion of SRFI-19:

  https://srfi-email.schemers.org/srfi-19/msg/2772123

It's an interesting idea, but I don't think it's something that we can
unilaterally change in an existing, long-finalized SRFI.  It would need
to be part of a new SRFI, I think.

So, I'm closing this as not-a-bug, although I acknowledge that the issue
you raised is valid.  Feel free to reopen and continue the discussion if
you disagree.

In any case, thanks very much for your many interesting and detailed bug
reports, and I apologize for the long delay in addressing them.

Regards,
  Mark

bug#21902: doc incorrectly describes Julian Date

[Mark H Weaver]

Zefram  writes:

> The manual says, in the section "SRFI-19 Introduction",
>
> #Also, for those not familiar with the terminology, a "Julian Day" is
> # a real number which is a count of days and fraction of a day, in UTC,
> # starting from -4713-01-01T12:00:00Z, ie. midday Monday 1 Jan 4713 B.C.
>
> There are two errors in the first statement of the epoch for Julian Date,
> in ISO 8601 format.  The JD epoch is noon on 1 January 4713 BC *in the
> proleptic Julian calendar*.  The ISO 8601 format is properly never used on
> the Julian calendar: ISO 8601 specifies the use of the Gregorian calendar,
> including proleptically where necessary (as it most certainly is here).
> On the proleptic Gregorian calendar, the JD epoch is noon on 24 November
> 4714 BC, and so the ISO 8601 expression should have some "-11-24".
>
> The second error is in how the year is expressed in ISO 8601.  The initial
> "-" does not mean the BC era, it means that the year number is negative.
> ISO 8601 specifies that the AD era is always used, with year numbers
> going negative where necessary; this arrangement is commonly known as
> "astronomical year numbering".  So "" means 1 BC, "-0001" means 2
> BC, and "-4713" means 4714 BC.  So the "-4713" is not correct for the
> attempted expression of the Julian calendar date, but happens to be
> correct for the Gregorian calendar date.
>
> Putting it together, a correct ISO 8601 expression for the Julian Date
> epoch is "-4713-11-24T12:00:00Z".
>
> The word-based statement of the JD epoch is correct as far as it goes,
> but would benefit considerably by the addition of a clause stating that
> it is in the proleptic Julian calendar.  (Generally, a clarification
> of which calendar is being used is helpful with the statement of any
> date prior to the UK's switch of calendar in 1752.)  The description of
> Modified Julian Date is essentially correct.
>
> However, there's a third problem: misuse of the term "UTC" for historical
> times.  The description of Julian Date says it's counted "in UTC",
> and the statement of the MJD epoch describes its 1858 time as being
> specified in UTC.  UTC is defined entirely by its relationship to TAI,
> which is defined by the operation of atomic clocks.  TAI is therefore
> only defined for the period since the operation of the first caesium
> atomic clock in the middle of 1955.  The UTC<->TAI relationship isn't
> actually defined even that far back: UTC begins at the beginning of
> 1961 (and that was not in the modern form with leap seconds).  It is
> therefore incorrect to apply the term "UTC" to any time prior to 1961.
> These two references to UTC should instead be to "UT", the wider class
> of closely-matching time scales of which UTC is one representative.
> Also, in the first sentence of this doc section, the phrase "universal
> time (UTC)" should be either "universal time (UT)" or (more likely)
> "coordinated universal time (UTC)".

I changed the text, based partly on your proposed patch and partly based
on similar recent fixes in the upstream SRFI-19 document, in commit
5106377a3460e1e35daf14ea6edbe80426347155 on the stable-2.2 branch.

I'm closing this bug now, but feel free to reopen if there are still
problems.

 Thanks!
   Mark

bug#22034: time-utc->date shows bogus zone-dependent leap second

[Mark H Weaver]

Zefram  writes:

> time-utc->date seems to think that a leap second occurs at a different
> time in each time zone:
>
> scheme@(guile-user)> (use-modules (srfi srfi-19))
> scheme@(guile-user)> (define (tdate d) (write (list (date->string d "~4") 
> (date->string (time-utc->date (date->time-utc d) 3600) "~4"))) (newline))
> scheme@(guile-user)> (tdate (make-date 0 59 59 22 30 6 2012 0))
> ("2012-06-30T22:59:59Z" "2012-06-30T23:59:59+0100")
> scheme@(guile-user)> (tdate (make-date 0 0 0 23 30 6 2012 0))
> ("2012-06-30T23:00:00Z" "2012-06-30T23:59:60+0100")
> scheme@(guile-user)> (tdate (make-date 0 1 0 23 30 6 2012 0))
> ("2012-06-30T23:00:01Z" "2012-07-01T00:00:01+0100")
>
> These are three consecutive seconds that occur an hour before a genuine
> leap second (at 23:59:60Z).  Observe that time-utc->date, applied to the
> middle second, describes it as a leap second happening at 23:59:60+01:00,
> which is bogus.  [...]

This is fixed by commit 5106377a3460e1e35daf14ea6edbe80426347155 on the
stable-2.2 branch.

> Matching up with this, the actual leap second is never correctly described
> with a non-zero zone offset.  It should be, for example, 00:59:60+01:00.

If I understand correctly, 'time-utc->date' should never return a date
object with 60 in the seconds field, because those extra seconds have no
representation in time-utc.  They only have representations in time-tai
and time-monotonic.

Anyway, thanks very much for these reports!  I'm closing this bug now,
but feel free to reopen if you think there are still issues to resolve.

  Mark

bug#21911: TAI-to-UTC conversion leaps at wrong time

[Mark H Weaver]

Zefram  writes:

> Probing the TAI-to-UTC conversion offered by srfi-19's time-tai->date,
> in the minutes around the leap second in 2012:
>
> scheme@(guile-user)> (use-modules (srfi srfi-19))
> scheme@(guile-user)> (for-each (lambda (d) (write (list d
> (date->string (time-tai->date (add-duration (julian-day->time-tai
> 2456109) (make-time time-duration 0 d)) 0) "~4"))) (newline)) (list
> 43000 43160 43164 43165 43166 43167 43199 43200 43201 43202))
> (43000 "2012-06-30T23:56:40Z")
> (43160 "2012-06-30T23:59:20Z")
> (43164 "2012-06-30T23:59:24Z")
> (43165 "2012-06-30T23:59:25Z")
> (43166 "2012-06-30T23:59:25Z")
> (43167 "2012-06-30T23:59:26Z")
> (43199 "2012-06-30T23:59:58Z")
> (43200 "2012-06-30T23:59:59Z")
> (43201 "2012-06-30T23:59:60Z")
> (43202 "2012-07-01T00:00:01Z")

This is fixed by commit 5106377a3460e1e35daf14ea6edbe80426347155 on the
stable-2.2 branch.  I'm closing this bug now, but feel free to reopen if
appropriate.

Thanks!
  Mark

bug#33044: Guile misbehaves in the "ja_JP.sjis" locale

[Mark H Weaver]

Mark H Weaver  writes:
> I'll leave this bug open at least until 'seed->random-state your-seed'
> is fixed to support wide strings.

This part is now fixed in commit
fbdcf6358519c415bd2041ca09bee9b16e9d528a on the stable-2.2 branch.

  Mark

bug#33036: Bug with the procedure nil? inside a specific code

[Mark H Weaver]

Hi Andy,

Andy Wingo  writes:
> Thank you, Mark!  That looks great.

Great, thanks for looking it over.  I pushed it to the stable-2.2 branch
as commit c3e14b74e81d0fd3266b97e6bd629cd4e2f98803.

I'm closing this bug now, but feel free to reopen if appropriate.

 Thanks!
   Mark

bug#33044: Guile misbehaves in the "ja_JP.sjis" locale

[Mark H Weaver]

Hi Tom,

Tom de Vries  writes:

> On 10/16/18 3:57 AM, Mark H Weaver wrote:
>> Thanks for the report, analysis and patch.  I agree with your analysis,
>> and the patch looks good.
>> 
>
> If so, can the patch be committed?

I just pushed commit c2a654b7d29f5e2f32fd1313cc80162fd0c8f992 to the
stable-2.2 branch, which includes the fix from your patch (although I
used 'scm_from_utf8_string' instead of 'scm_from_latin1_string'), and
many other instances of the same problem.  These fixes will be in the
upcoming guile-2.2.5 release.

Does that address the problem for you?

I'll leave this bug open at least until 'seed->random-state your-seed'
is fixed to support wide strings.

Thanks again,

 Mark

bug#33044: Guile misbehaves in the "ja_JP.sjis" locale

[Mark H Weaver]

Mark H Weaver  writes:

> Shift_JIS is _mostly_ ASCII-compatible, except that code points 0x5C and
> 0x7E, which represent backslash (\) and tilde (~) in ASCII, are mapped
> to the Yen sign (¥) and overline (‾) in Shift_JIS.  Backslash (\) and
> tilde (~) are multibyte characters in Shift_JIS.

Although I wrote above that "Backslash (\) and tilde (~) are multibyte
characters in Shift_JIS", that was admittedly my assumption, based on
the absence of those characters in the "First byte" map shown here:

  https://en.wikipedia.org/wiki/Shift_JIS#As_defined_in_JIS_X_0208:1997

However, now I'm unsure.  I've spent some time attempting to find the
Shift_JIS encodings for backslash and tilde, but I've not yet found an
answer.

I've asked Emacs 26 to write a file containing backslashes and Yen signs
using the "shift_jis" encoding, and both characters seem to be mapped to
the same code: 0x5C.

I've also used the 'iconv' utility from GNU libc to convert backslashes
and Yen signs to Shift_JIS, and it also maps these two characters to the
same codes:

--8<---cut here---start->8---
mhw@jojen ~$ echo '\\¥¥' | iconv -f UTF-8 -t SHIFT-JIS > Shift_JIS_test.txt
mhw@jojen ~$ hexdump -C Shift_JIS_test.txt
  5c 5c 5c 5c 0a|.|
0005
--8<---cut here---end--->8---

While investigating, I found this bug for GNU libc asking to add an SJIS
locale, and the developers were strongly opposed:

  https://bugzilla.redhat.com/show_bug.cgi?id=136290

At this point, I'm inclined to believe that Shift_JIS is not suitable as
a locale encoding on POSIX systems, and that we should not try to
support it in Guile.

What do you think?

Can you tell me how backslash and tilde are represented in Shift JIS?

 Regards,
   Mark

bug#33044: Guile misbehaves in the "ja_JP.sjis" locale

[Mark H Weaver]

retitle 33044 Guile misbehaves in the "ja_JP.sjis" locale
thanks

Hi Tom,

Thanks for the report, analysis and patch.  I agree with your analysis,
and the patch looks good.

However, there's also a much deeper problem here.  You found and fixed
one occurrence of Guile assuming that the locale encoding is ASCII-
compatible.  In fact, this assumption is widespread in Guile, and I
would guess that it's widespread throughout the POSIX world.

I admit that before I saw your message, I believed that it was
legitimate to assume that the locale encoding was ASCII-compatible.  Now
I'm unsure, although I'll note that according to the 'localedef' utility
from GNU libc, this locale is "not ISO C compliant".  It printed the
following message when I asked it to generate the "ja_JP.sjis" locale:

  [warning] character map `SHIFT_JIS' is not ASCII compatible, locale not ISO C 
compliant [--no-warnings=ascii]

Shift_JIS is _mostly_ ASCII-compatible, except that code points 0x5C and
0x7E, which represent backslash (\) and tilde (~) in ASCII, are mapped
to the Yen sign (¥) and overline (‾) in Shift_JIS.  Backslash (\) and
tilde (~) are multibyte characters in Shift_JIS.

One common problem is that Guile often uses 'scm_from_locale_string' to
create Scheme strings from ASCII-only C string literals.  These should
all be changed to use either 'scm_from_latin1_string' or
'scm_from_utf8_string'.  I prefer the latter because modern C compilers
typically use UTF-8 as the default execution character set, i.e. the
character set used to encode string and character constants, regardless
of the locale settings.  GCC uses UTF-8 by default unless
-fexec-charset=CHARSET is given at compile time.  I'd prefer to promote
writing code that works for arbitrary string literals, so that code
needn't be adjusted if non-ASCII characters are later added.

A related set of problems is that Guile often applies
'scm_from_locale_string' to char* arguments passed in from the user, or
produced by third-party libraries.  These issues are more difficult to
address.  We provide several C APIs that accept C strings without
specifying what encoding is expected.  If the string ultimately derives
from a C string constant, we probably want UTF-8, whereas if the string
came from I/O, or program arguments, then we probably want the locale
encoding.

For example, consider 'scm_c_eval_string'.  This has been a public API
function since 2002, but we did not specify the encoding of its C string
argument until 2011.  We chose the locale encoding in this case, which I
think is reasonable, but I also expect that code exists in the wild that
passes a C string literal to 'scm_c_eval_string'.

Until now, problems like this have been mostly harmless, since the C
string literals are typically ASCII-only.  However, if we wish to
support non-ASCII-compatible encodings such as Shift_JIS, we can no
longer consider these problems harmless.  For example, programs which
pass C string literals to 'scm_c_eval_string' will fail when using the
"ja_JP.sjis" locale, if any tildes or backslashes are present.
Backslashes are fairly common in Scheme code.

There's various other code scattered in Guile that assumes ASCII
characters can searched for, and sometimes replaced with other ASCII
characters.  For example, several functions in load.c, including
'search_path', 'load_thunk_from_path' scan through file names in the
locale encoding, scanning the bytes looking for particular ASCII codes
such as '.', '/', and '\'.

On MingW, 'scm_i_mirror_backslashes' in load.c converts backslashes into
forward slashes byte-wise, assuming ASCII-compatibility, and this
transformation is applied to file names in several places.

While looking into this, I also discovered that Guile's S-expression
reader, i.e. the 'read' procedure, assumes an ASCII-compatible port
encoding, despite the fact that it is meant to support arbitrary
encodings such as UTF-16 and UTF-32.  I just filed a related bug
 to track this probem.

These are some of the problems that I'm currently aware of.  I expect
that this bug report will remain open for a while.

To begin, I've started working on a patch to change many occurrences of
'scm_from_locale_string' to 'scm_from_utf8_string', in cases where the C
string clearly originates from a C string literal.

Thanks again for the detailed bug report and analysis.

Regards,
  Mark

bug#33057: Guile's reader assumes the port encoding is ASCII-compatible

[Mark H Weaver]

In several places, Guile's reader assumes that the port encoding is
ASCII-compatible.  For example:

* 'scm_token' reads raw bytes and passes them to the CHAR_IS_DELIMITER
  macro to check for delimiters.

* 'scm_read_mixed_case_symbol' checks for the (optional) postfix keyword
  syntax by comparing the final _byte_ read by 'read_token' with ':'.

* 'scm_read_character' uses 'read_token' to read raw bytes up to the
  next delimiter, and if it returns a single byte with value 0-127, it
  is interpreted as the associated ASCII character regardless of the
  encoding.

* 'scm_read_semicolon_comment' reads bytes until it finds one equal to
  the ASCII code for '\n'.

There may be other problems as well.

   Mark

bug#33053: scm_i_mirror_backslashes assumes ASCII-compatible locale encoding

[Mark H Weaver]

Mark H Weaver  writes:

> The 'scm_i_mirror_backslashes' in load.c operates on C strings in the
> locale encoding, and assumes that the locale encoding is ASCII
> compatible.  In the Shift_JIS encoding, used in the "JP_jp.sjis" locale,
> backslash '\' is mapped to a multibyte character, and the Yen sign '¥'
> is represented using code 0x5C, the same code as backslash '\' in ASCII.
>
> As a result, users of the "JP_jp.sjis" locale will have Yen signs '¥' in
> their file names converted into slashes by this function.

I miswrote the locale name above.  The locale name is "ja_JP.sjis".

  Mark

bug#33053: scm_i_mirror_backslashes assumes ASCII-compatible locale encoding

[Mark H Weaver]

The 'scm_i_mirror_backslashes' in load.c operates on C strings in the
locale encoding, and assumes that the locale encoding is ASCII
compatible.  In the Shift_JIS encoding, used in the "JP_jp.sjis" locale,
backslash '\' is mapped to a multibyte character, and the Yen sign '¥'
is represented using code 0x5C, the same code as backslash '\' in ASCII.

As a result, users of the "JP_jp.sjis" locale will have Yen signs '¥' in
their file names converted into slashes by this function.

 Mark

bug#26058: utf16->string and utf32->string don't conform to R6RS

[Mark H Weaver]

Hi Taylan,

taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes:

> Andy Wingo  writes:
>
>> Adopting the behavior is more or less fine.  If it can be done while
>> relying on the existing behavior, that is better than something ad-hoc
>> in a module.

In general, I agree with Andy's sentiment that it would be better to
avoid redundant BOM handling code, and moreover, I appreciate his
reluctance to apply a fix without careful consideration of our existing
BOM semantics.

However, as Taylan discovered, Guile does not provide a mechanism to
specify a default endianness of a UTF-16 or UTF-32 port in case a BOM is
not found.  I see no straightforward way to implement these R6RS
interfaces using ports.

We could certainly add such a mechanism if needed, but I see another
problem with this approach: the expense of creating and later collecting
a bytevector port object would be a very heavy burden to place on these
otherwise fairly lightweight operations.  Therefore, I would prefer to
avoid that implementation strategy for these operations.

Although BOM handling for ports is quite complex with many subtle points
to consider, detecting a BOM at the beginning of a bytevector is so
trivial that I personally have no objection to this tiny duplication of
logic.

Therefore, my preference would be to adopt code similar to that proposed
by Taylan, although I believe it can, and should, be further simplified:

> diff --git a/module/rnrs/bytevectors.scm b/module/rnrs/bytevectors.scm
> index 9744359f0..997a8c9cb 100644
> --- a/module/rnrs/bytevectors.scm
> +++ b/module/rnrs/bytevectors.scm
> @@ -69,7 +69,9 @@
> bytevector-ieee-double-native-set!
>  
> string->utf8 string->utf16 string->utf32
> -   utf8->string utf16->string utf32->string))
> +   utf8->string
> +   (r6rs-utf16->string . utf16->string)
> +   (r6rs-utf32->string . utf32->string)))
>  
>  
>  (load-extension (string-append "libguile-" (effective-version))
> @@ -80,4 +82,52 @@
>`(quote ,sym)
>(error "unsupported endianness" sym)))
>  
> +(define (read-bom16 bv)
> +  (let ((c0 (bytevector-u8-ref bv 0))
> +(c1 (bytevector-u8-ref bv 1)))
> +(cond
> + ((and (= c0 #xFE) (= c1 #xFF))
> +  'big)
> + ((and (= c0 #xFF) (= c1 #xFE))
> +  'little)
> + (else
> +  #f

We should gracefully handle the case of an empty bytevector, returning
an empty string without error in that case.

Also, we should use a single 'bytevector-u16-ref' operation to check for
the BOM.  Pick an arbitrary endianness for the operation (big-endian?),
and compare the resulting integer with both #xFEFF and #xFFFE.  That
way, the code will be simpler and more efficient.  Note that our VM has
dedicated instructions for these multi-byte bytevector accessors, and
there will be fewer comparison operations as well.  Similarly for the
utf32 case.

What do you think?

> +(define r6rs-utf16->string
> +  (case-lambda
> +((bv default-endianness)
> + (let ((bom-endianness (read-bom16 bv)))
> +   (if (not bom-endianness)
> +   (utf16->string bv default-endianness)
> +   (substring/shared (utf16->string bv bom-endianness) 1

Better to use plain 'substring' here, I think.  The machinery of shared
substrings is more expensive, and unnecessary in this case.

Otherwise, it looks good to me.  Would you like to propose a revised
patch?

Andy, what do you think?

   Mark

bug#21883: unnecessary bit shifting range limits

[Mark H Weaver]

Stefan Israelsson Tampe  writes:
> how would this slow down the code. just add the correction where you
> throw the exception which should be in a branch outside the hot path.

If you have a suggestion that's simpler than what I did in commits
011aec7e, 9448a078, and 1990aa91, and just as fast in the common cases,
feel free to propose a patch.  The words above are insufficient.

  Mark

bug#21901: bit shift wrong on maximal right shift

[Mark H Weaver]

Zefram  writes:

> With Guile 2.0.11:
>
> scheme@(guile-user)> (ash 123 (ash -1 63))
> $1 = 123
>
> Correct result would of course be zero.  Problem only occurs for
> exactly this shift distance: one bit less produces the right answer.

Nice catch!

It's finally fixed in commit 1990aa916382d0afcebd5315a6d6f555949ff654 on
the stable-2.2 branch.  The fix will be in Guile 2.2.5.

Thanks for the report.

  Mark

bug#21883: unnecessary bit shifting range limits

[Mark H Weaver]

Zefram  writes:

> Not really outright bugs, but these responses are less than awesome:
>
> $ guile -c '(write (logbit? (ash 1 100) 123))'
> ERROR: Value out of range 0 to 18446744073709551615: 
> 1267650600228229401496703205376
> $ guile -c '(write (ash 0 (ash 1 100)))'
> ERROR: Value out of range -9223372036854775808 to 9223372036854775807: 
> 1267650600228229401496703205376
> $ guile -c '(write (ash 123 (ash -1 100)))'
> ERROR: Value out of range -9223372036854775808 to 9223372036854775807: 
> -1267650600228229401496703205376
>
> In all three cases, the theoretically-correct result of the expression
> is not only representable but easily computed.

Commit 011aec7e240ef987931548d90c53e6692c85d01c on the stable-2.2 branch
extends 'ash' and 'round-ash' to handle the easily computed cases of
huge shifts.

> The functions could be improved to avoid failing in these cases, by
> adding logic amounting to:
>
> (define (better-logbit? b v)
>   (if (>= b (integer-length v)) (< v 0) (logbit? b v)))
>
> (define (better-ash v s)
>   (cond
> ((= v 0) 0)
> ((<= s (- (integer-length v))) (if (< v 0) -1 0))
> (else (ash v s

Unfortunately, simple implementations like the ones above slow down the
common case with expensive checks that are rarely needed.  The
aforementioned commit takes pains to avoid slowing down the common case,
but at the cost of extra code complexity.

In theory we could do something similar with many other procedures that
implement operations on bits and bit fields, but I wonder if it's worth
the extra complexity.

   Mark

bug#32644: bytevector bug

[Mark H Weaver]

Stefan Israelsson Tampe  writes:
> The code velow does not compile when the define-inlinable of id is active. If 
> in stead
> id defined by define is used it all compiles just fine.

The problem was that, in some cases, the type inferrer would call 'ash'
with (- 1 (expt 2 64)) as the second argument during compilation.  The
implementation of 'ash' would raise an exception unless its second
argument (the shift count) fits in a C 'long'.

This is fixed in commit 011aec7e240ef987931548d90c53e6692c85d01c on the
stable-2.2 branch.  That commit extends 'ash' and 'round-ash' to
gracefully handle several cases where the shift count is too large to
fit in a 'long'.

Thanks for the report.

  Mark

bug#32938: guile 2.2.4 crashes (u8-list->bytevector (make-bytevector 32 0))

[Mark H Weaver]

Josh Datko  writes:
> If you try to convert a bytevector, to a bytevector, using
> u8-list->bytevector, guile crashes.

Fixed in commit fe73fedab40cf716cc39139a61c078e2c9a2f37f on the
stable-2.2 branch.  Thanks for the report!

  Mark

bug#33036: Bug with the procedure nil? inside a specific code

[Mark H Weaver]

Mark H Weaver  writes:

> calcium  writes:
>> There is a page that speak about nil :
>> https://www.gnu.org/software/guile/manual/html_node/Nil.html
>> However it doesn’t mention the nil? Procedure
>>
>> The only documentation that I found about nil? is this one :
>>
>> (procedure-documentation nil?)
>> => "- Scheme Procedure: nil? x\n Return `#t' iff X is nil, else
>> return `#f'."
>
> I agree that the documentation for 'nil?' is woefully inadequate.  We
> should add a manual entry for 'nil?' and improve its docstring.

I did this in commit b44f505f1571fc9c42e58982f161a9cfc81fb7f4 on the
'stable-2.2' branch.

https://git.savannah.gnu.org/cgit/guile.git/commit/?h=stable-2.2=b44f505f1571fc9c42e58982f161a9cfc81fb7f4

Thanks,
  Mark

bug#33036: Bug with the procedure nil? inside a specific code

[Mark H Weaver]

Mark H Weaver  writes:

> calcium  writes:
>> Here is the bug that I found :
>>
>> ;;; -START- code with the bug -START- ;;;
>>
>> (define (strange lst)
>>   (let loop ((lst lst)
>>  (is-empty '()))
>> (cond ((nil? lst)
>>(if (nil? is-empty) 'works
>>  (list 'should-not-occur is-empty)))
>>   (else
>>(loop (cdr lst)
>>  is-empty)
>>
>> (strange '())
>> => (should-not-occur ())
>>
>> (strange #nil)
>> => (should-not-occur ())
>
> Indeed, this certainly indicates a bug.
>
> I believe the bug is in 'local-type-fold' in (language cps type-fold).
> It contains a local procedure 'scalar-value' which, if I understand
> correctly, seems to incorrectly assume that (nil? x) returns #t if and
> only if X is 'eq?' to #nil.

The bug actually wasn't in 'local-type-fold', but it's true that the
type inferencer assumed in a few places that if (nil? X) returned #t
that X must be #nil, and similarly for 'null?' and ().

I've attached a proposed fix.

Andy, does this fix look right to you?

 Thanks,
   Mark


>From d904d5233582e51a4be06d2c08ccdd15a66b8d77 Mon Sep 17 00:00:00 2001
From: Mark H Weaver 
Date: Sat, 13 Oct 2018 23:02:05 -0400
Subject: [PATCH] Fix type inferencing for 'nil?' and 'null?' predicates.

Fixes <https://bugs.gnu.org/33036>.
Reported by .

* module/language/cps/types.scm (define-simple-type-inferrer):
Apply (logand ( val) <>) uniformly.  Previously, this was done only
in the false branch.  Rename local variable to 'type*', to allow the
macro operand 'type' to be an arbitrary expression.
(*type-inferrers*): Add  to the set of possible types.
(*type-inferrers*): Add  and  to the set the possible
types.
* module/language/cps/type-fold.scm (*branch-folders*): Add 
to the set of possible types.
(*branch-folders*): Add  and  to the set the possible
types.
* test-suite/tests/compiler.test: Add tests.
---
 module/language/cps/type-fold.scm |  6 ++--
 module/language/cps/types.scm | 13 
 test-suite/tests/compiler.test| 51 ++-
 3 files changed, 60 insertions(+), 10 deletions(-)

diff --git a/module/language/cps/type-fold.scm b/module/language/cps/type-fold.scm
index fc37fac50..163ef659d 100644
--- a/module/language/cps/type-fold.scm
+++ b/module/language/cps/type-fold.scm
@@ -1,5 +1,5 @@
 ;;; Abstract constant folding on CPS
-;;; Copyright (C) 2014, 2015 Free Software Foundation, Inc.
+;;; Copyright (C) 2014, 2015, 2018 Free Software Foundation, Inc.
 ;;;
 ;;; This library is free software: you can redistribute it and/or modify
 ;;; it under the terms of the GNU Lesser General Public License as
@@ -69,8 +69,8 @@
 
 ;; All the cases that are in compile-bytecode.
 (define-unary-type-predicate-folder pair? )
-(define-unary-type-predicate-folder null? )
-(define-unary-type-predicate-folder nil? )
+(define-unary-type-predicate-folder null? (logior  ))
+(define-unary-type-predicate-folder nil? (logior   ))
 (define-unary-type-predicate-folder symbol? )
 (define-unary-type-predicate-folder variable? )
 (define-unary-type-predicate-folder vector? )
diff --git a/module/language/cps/types.scm b/module/language/cps/types.scm
index 5c1d71299..61de971fe 100644
--- a/module/language/cps/types.scm
+++ b/module/language/cps/types.scm
@@ -529,13 +529,14 @@ minimum, and maximum."
 
 (define-syntax-rule (define-simple-predicate-inferrer predicate type)
   (define-predicate-inferrer (predicate val true?)
-(let ((type (if true?
-type
-(logand ( val) (lognot type)
-  (restrict! val type -inf.0 +inf.0
+(let ((type* (logand ( val)
+ (if true?
+ type
+ (lognot type)
+  (restrict! val type* -inf.0 +inf.0
 (define-simple-predicate-inferrer pair? )
-(define-simple-predicate-inferrer null? )
-(define-simple-predicate-inferrer nil? )
+(define-simple-predicate-inferrer null? (logior  ))
+(define-simple-predicate-inferrer nil? (logior   ))
 (define-simple-predicate-inferrer symbol? )
 (define-simple-predicate-inferrer variable? )
 (define-simple-predicate-inferrer vector? )
diff --git a/test-suite/tests/compiler.test b/test-suite/tests/compiler.test
index 4f644f339..64bb976fa 100644
--- a/test-suite/tests/compiler.test
+++ b/test-suite/tests/compiler.test
@@ -1,5 +1,5 @@
  compiler.test --- tests for the compiler  -*- scheme -*-
- Copyright (C) 2008, 2009, 2010, 2011, 2012, 2013, 2014 Free Software Foundation, Inc.
+ Copyright (C) 2008-2014, 2018 Free Software Foundation, Inc.
  
  This library is free software; you can redistribute it and/or
  modify it under the terms of the GNU Lesser General Public
@@ -251,3 +251,52 @@
 
   (pass-if-equal "test flonum" 0.0 (test-p

bug#33036: Bug with the procedure nil? inside a specific code

[Mark H Weaver]

Hi,

calcium  writes:
> The procedure nil? is not listed in the procedure index :
> https://www.gnu.org/software/guile/docs/docs-2.0/guile-ref/Procedure-Index.html

The link above is the manual for Guile 2.0, which didn't have 'nil?'.
It was added in Guile 2.2.

> There is a page that speak about nil :
> https://www.gnu.org/software/guile/manual/html_node/Nil.html
> However it doesn’t mention the nil? Procedure
>
> The only documentation that I found about nil? is this one :
>
> (procedure-documentation nil?)
> => "- Scheme Procedure: nil? x\n Return `#t' iff X is nil, else
> return `#f'."

I agree that the documentation for 'nil?' is woefully inadequate.  We
should add a manual entry for 'nil?' and improve its docstring.

'nil?' tests whether Emacs Lisp code would consider the value to be nil,
i.e. whether it would be considered false by Elisp 'if' or 'cond'.

Scheme has two distinct values for the empty list () and false #f.  In
Elisp, both of these concepts are represented by the same value: nil.
As a result, the empty list is considered "true" in Scheme, and "false"
in Elisp.

In other words, 'nil?' returns #t if its argument is #f, (), or #nil,
otherwise it returns #f.

> (procedure-documentation null?)
> => "- Scheme Procedure: null? x\n Return `#t' iff X is the empty
> list, else `#f'."
>
> The procedure documentation is not the same for nil? and null?
> They seem nonetheless to be the same.

No, they differ in their handling of #f:

  (null? #f) => #f
  (nil?  #f) => #t

> Here is the bug that I found :
>
> ;;; -START- code with the bug -START- ;;;
>
> (define (strange lst)
>   (let loop ((lst lst)
>  (is-empty '()))
> (cond ((nil? lst)
>(if (nil? is-empty) 'works
>  (list 'should-not-occur is-empty)))
>   (else
>(loop (cdr lst)
>  is-empty)
>
> (strange '())
> => (should-not-occur ())
>
> (strange #nil)
> => (should-not-occur ())

Indeed, this certainly indicates a bug.

I believe the bug is in 'local-type-fold' in (language cps type-fold).
It contains a local procedure 'scalar-value' which, if I understand
correctly, seems to incorrectly assume that (nil? x) returns #t if and
only if X is 'eq?' to #nil.

To be continued...

Thanks very much for this detailed report!

Regards,
  Mark

bug#32786: (system* ...) call somehow interferes with atomic-box on armv7l

[Mark H Weaver]

Hi,

Михаил Бахтерев  writes:

> I've tested the patch (i've applied it to the v2.2.4 source code). The
> test sample works now as expected with both test «payloads»: (sleep
> N-seconds) and (system* "any" "command" "here") calls.

Thanks for letting us know the results of your testing, this is very
helpful.  I just pushed a similar fix (same code, improved comments) to
the stable-2.2 branch, commit 2d09e0513fc11e2305077ba3653f6e4c2f266ddb,
which will be in the upcoming guile-2.2.5 release.

Thanks,
  Mark


>
> Thanks for fixing it.
>
> - MB. Respectfully
>
> On Thu, Sep 27, 2018 at 01:49:23AM -0400, Mark H Weaver wrote:
>> Hi,
>> 
>> Thanks for the additional details.  I was able to reproduce the bug, and
>> I believe I now see the problem.
>> 
>> 'atomic-box-compare-and-swap!' is implemented using
>> 'atomic_compare_exchange_weak' (if available), but neglects to handle
>> the case where 'atomic_compare_exchange_weak' may spuriously fail.  In
>> that case, the box is left unchanged, although the observed value was
>> equal to the expected value.
>> 
>> What's happening here is that the 'atomic-box-compare-and-swap!' in
>> 'sleep-loop' fails spuriously, leaving the box in state #:accepted
>> although it returns #:accepted as its result.  When the main loop
>> discovers this, it changes the state to #:need-to-sleep, although the
>> thread has already ended.
>> 
>> To confirm this hypothesis, I added a print statement to the main loop
>> showing the state of the box that it observed during the protocol
>> exchange, and indeed it sees #:accepted the first time it checks, and
>> #:need-to-sleep in all later iterations.
>> 
>> I've attached a proposed patch that I hope will fix this problem.  If
>> you'd be willing to test it, I'd be grateful, but otherwise I'll try to
>> test it in the next week or so.  My access to armv7l boxes is somewhat
>> limited.
>> 
>> Thanks for this report.
>> 
>>   Mark
>> 
>> 
>
>> From 958d37686a9ac65f48cb2ca32a341cf182c24b5a Mon Sep 17 00:00:00 2001
>> From: Mark H Weaver 
>> Date: Thu, 27 Sep 2018 01:00:11 -0400
>> Subject: [PATCH] UNTESTED: Fix 'atomic-box-compare-and-swap!'.
>> 
>> Fixes <https://bugs.gnu.org/32786>.
>> 
>> 'scm_atomic_compare_and_swap_scm' uses 'atomic_compare_exchange_weak' if
>> it's available on the host platform.  'atomic_compare_exchange_weak' may
>> fail spuriously, leaving the atomic object unchanged even when it
>> contained the expected value.  'atomic-box-compare-and-swap!' uses
>> 'scm_atomic_compare_and_swap_scm' without checking its return value, and
>> therefore may return the expected value while leaving the box unchanged.
>> This contradicts the documentation, which explicitly states that "you
>> can know if the swap worked by checking if the return value is 'eq?' to
>> EXPECTED.".
>> 
>> * libguile/atomic.c (scm_atomic_box_compare_and_swap_x): If
>> 'scm_atomic_compare_and_swap_scm' returns false and the observed value
>> is equal to 'expected', then try again.
>> * libguile/vm-engine.c (atomic-box-compare-and-swap!): Ditto.
>> ---
>>  libguile/atomic.c| 13 +
>>  libguile/vm-engine.c | 13 -
>>  2 files changed, 17 insertions(+), 9 deletions(-)
>> 
>> diff --git a/libguile/atomic.c b/libguile/atomic.c
>> index 950874030..504643912 100644
>> --- a/libguile/atomic.c
>> +++ b/libguile/atomic.c
>> @@ -1,4 +1,4 @@
>> -/* Copyright (C) 2016 Free Software Foundation, Inc.
>> +/* Copyright (C) 2016, 2018 Free Software Foundation, Inc.
>>   *
>>   * This library is free software; you can redistribute it and/or
>>   * modify it under the terms of the GNU Lesser General Public License
>> @@ -95,10 +95,15 @@ SCM_DEFINE (scm_atomic_box_compare_and_swap_x,
>>  "if the return value is @code{eq?} to @var{expected}.")
>>  #define FUNC_NAME s_scm_atomic_box_compare_and_swap_x
>>  {
>> +  SCM result = expected;
>> +
>>SCM_VALIDATE_ATOMIC_BOX (1, box);
>> -  scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (box),
>> -   , desired);
>> -  return expected;
>> +  while (!scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (box),
>> +   , desired)
>> + && scm_is_eq (result, expected))
>> +{
>> +}
>> +  return result;
>>  }
>>  #undef FUNC_NAME
>>  
>> diff --git a/libguile/vm-engine.c b/libguile/vm

bug#32938: guile 2.2.4 crashes (u8-list->bytevector (make-bytevector 32 0))

[Mark H Weaver]

Josh Datko  writes:

> If you try to convert a bytevector, to a bytevector, using
> u8-list->bytevector, guile crashes.
>
> $ guile -q
> GNU Guile 2.2.4
> Copyright (C) 1995-2017 Free Software Foundation, Inc.
>
> Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
> This program is free software, and you are welcome to redistribute it
> under certain conditions; type `,show c' for details.
>
> Enter `,help' for help.
> scheme@(guile-user)> (use-modules (rnrs bytevectors))
> scheme@(guile-user)> (u8-list->bytevector (make-bytevector 32 0))
> [1]126190 abort (core dumped)  guile -q

Indeed, the code in 'u8-list->bytevector' that's supposed to validate
that its argument is a list, is broken.

'u8-list->bytevector' uses the SCM_VALIDATE_LIST_COPYLEN macro to
validate the list and simultaneously compute its length.  That macro
implicitly assumes that its third operand will be a variable of type
'long', because the result of 'scm_ilength' is assigned to it, and
'scm_ilength' returns a 'long'.

After storing the result to the variable, it checks to see if the result
is negative, which would indicate that the operand wasn't a proper list.

The bytevector operations that convert integer lists to bytevectors pass
a variable of type 'size_t' to SCM_VALIDATE_LIST_COPYLEN.  Since
'size_t' is unsigned, the -1 result from 'scm_ilength' was interpreted
as ULONG_MAX instead.

Thanks for the report.

Mark

bug#32786: (system* ...) call somehow interferes with atomic-box on armv7l

[Mark H Weaver]

Hi,

Thanks for the additional details.  I was able to reproduce the bug, and
I believe I now see the problem.

'atomic-box-compare-and-swap!' is implemented using
'atomic_compare_exchange_weak' (if available), but neglects to handle
the case where 'atomic_compare_exchange_weak' may spuriously fail.  In
that case, the box is left unchanged, although the observed value was
equal to the expected value.

What's happening here is that the 'atomic-box-compare-and-swap!' in
'sleep-loop' fails spuriously, leaving the box in state #:accepted
although it returns #:accepted as its result.  When the main loop
discovers this, it changes the state to #:need-to-sleep, although the
thread has already ended.

To confirm this hypothesis, I added a print statement to the main loop
showing the state of the box that it observed during the protocol
exchange, and indeed it sees #:accepted the first time it checks, and
#:need-to-sleep in all later iterations.

I've attached a proposed patch that I hope will fix this problem.  If
you'd be willing to test it, I'd be grateful, but otherwise I'll try to
test it in the next week or so.  My access to armv7l boxes is somewhat
limited.

Thanks for this report.

  Mark


>From 958d37686a9ac65f48cb2ca32a341cf182c24b5a Mon Sep 17 00:00:00 2001
From: Mark H Weaver 
Date: Thu, 27 Sep 2018 01:00:11 -0400
Subject: [PATCH] UNTESTED: Fix 'atomic-box-compare-and-swap!'.

Fixes <https://bugs.gnu.org/32786>.

'scm_atomic_compare_and_swap_scm' uses 'atomic_compare_exchange_weak' if
it's available on the host platform.  'atomic_compare_exchange_weak' may
fail spuriously, leaving the atomic object unchanged even when it
contained the expected value.  'atomic-box-compare-and-swap!' uses
'scm_atomic_compare_and_swap_scm' without checking its return value, and
therefore may return the expected value while leaving the box unchanged.
This contradicts the documentation, which explicitly states that "you
can know if the swap worked by checking if the return value is 'eq?' to
EXPECTED.".

* libguile/atomic.c (scm_atomic_box_compare_and_swap_x): If
'scm_atomic_compare_and_swap_scm' returns false and the observed value
is equal to 'expected', then try again.
* libguile/vm-engine.c (atomic-box-compare-and-swap!): Ditto.
---
 libguile/atomic.c| 13 +
 libguile/vm-engine.c | 13 -
 2 files changed, 17 insertions(+), 9 deletions(-)

diff --git a/libguile/atomic.c b/libguile/atomic.c
index 950874030..504643912 100644
--- a/libguile/atomic.c
+++ b/libguile/atomic.c
@@ -1,4 +1,4 @@
-/* Copyright (C) 2016 Free Software Foundation, Inc.
+/* Copyright (C) 2016, 2018 Free Software Foundation, Inc.
  *
  * This library is free software; you can redistribute it and/or
  * modify it under the terms of the GNU Lesser General Public License
@@ -95,10 +95,15 @@ SCM_DEFINE (scm_atomic_box_compare_and_swap_x,
 "if the return value is @code{eq?} to @var{expected}.")
 #define FUNC_NAME s_scm_atomic_box_compare_and_swap_x
 {
+  SCM result = expected;
+
   SCM_VALIDATE_ATOMIC_BOX (1, box);
-  scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (box),
-   , desired);
-  return expected;
+  while (!scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (box),
+   , desired)
+ && scm_is_eq (result, expected))
+{
+}
+  return result;
 }
 #undef FUNC_NAME
 
diff --git a/libguile/vm-engine.c b/libguile/vm-engine.c
index 19ff3e498..9650e33eb 100644
--- a/libguile/vm-engine.c
+++ b/libguile/vm-engine.c
@@ -3868,16 +3868,19 @@ VM_NAME (scm_i_thread *thread, struct scm_vm *vp,
 {
   scm_t_uint16 dst, box;
   scm_t_uint32 expected, desired;
-  SCM scm_box, scm_expected;
+  SCM scm_box, scm_expected, scm_result;
   UNPACK_12_12 (op, dst, box);
   UNPACK_24 (ip[1], expected);
   UNPACK_24 (ip[2], desired);
   scm_box = SP_REF (box);
   VM_VALIDATE_ATOMIC_BOX (scm_box, "atomic-box-compare-and-swap!");
-  scm_expected = SP_REF (expected);
-  scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (scm_box),
-   _expected, SP_REF (desired));
-  SP_SET (dst, scm_expected);
+  scm_result = scm_expected = SP_REF (expected);
+  while (!scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (scm_box),
+   _result, SP_REF (desired))
+ && scm_is_eq (scm_result, scm_expected))
+{
+}
+  SP_SET (dst, scm_result);
   NEXT (3);
 }
 
-- 
2.19.0

bug#32841: assoc-set fails with dot notation association list

[Mark H Weaver]

Hi Christopher,

"Hood, Christopher L."  writes:
> This fails with code pulled straight out of the Guile manual example
> (section 6.6.20.6).

Indeed, the example code in the manual is bad.  Thanks for bringing this
to our attention.

> (define capitals '(("New York" . "Albany")
>("Oregon"   . "Salem")
>("Florida"  . "Miami")))

As John correctly pointed out, it's an error to mutate a program
literal.  It's analogous to trying to modify a string literal in C.
'assoc-set!' mutates the existing list structure in some cases, and so
it cannot be used on literals such as the one above.

To initialize an alist that will be mutated, you must instead do
something like this:

  (define capitals (list (cons "New York" "Albany")
 (cons "Oregon"   "Salem")
 (cons "Florida"  "Miami")))

> I’ll note that if you define the alist so its initial contents are
> defined using a quasiquote and the cons form instead of dot notation,
> this error is not reached.

Yes, that accomplishes essentially the same thing, although note that
quasiquote makes an effort to use program literals for parts of the
resulting list structure, e.g.:

  `(,a ,b c d)

expands to:

  (cons a (cons b '(c d)))

which means that the first two cons cells can be mutated, but the last
two are part of an immutable program literal.

> I’m not sure if the error is valid or not, but in any case, the code
> that produces is listed as an valid example in the manual, so that
> doesn’t seem right.

Indeed, the manual needs to be fixed.  Sorry for the confusion, and
thanks again for the bug report.

 Mark

bug#32847: Wrong macro expansion in eval

[Mark H Weaver]

Hi Stefan,

Stefan Israelsson Tampe  writes:

> This for guile 2.4 and master,
>
>> (eval `(let-syntax ((f (lambda (x) ,#'(+ (pk 'a 1) 2 f) 
>> (current-module)) 
>
> ;;; (# 1)
>
> But without eval:
>> (let-syntax ((f (lambda (x) #'(+ (pk 'a 1) 2 f)  
>
> ;;; (a 1)

I think the mistake is in your code above.  In the first case, what you
want is this:

  (eval `(let-syntax ((f (lambda (x) ,'#'(+ (pk 'a 1) 2 f)
(current-module))

Note the addition of a quote (') between the unquote (,) and syntax (#')
above.

The expression that follows unquote (,) should evaluate to an
s-expression.  In this case, you want it to evaluate to the s-expression
#'(+ (pk 'a 1) 2), i.e. (syntax (+ (pk 'a 1) 2)), i.e. a list with two
elements, the first being the symbol 'syntax'.  But that's not what
you're doing above.  Instead, you are returning the syntax object
itself, which is being spliced directly into the code.

Does that make sense?

   Mark

bug#32786: (system* ...) call somehow interferes with atomic-box on armv7l

[Mark H Weaver]

Hi,

Михаил Бахтерев  writes:

> I attach the code sample, which expected behaviour is:
>
>   1. to periodically restart thread, which executes «sleep 1s» with
>   system* call;
>
>   2. and then to check if it should repeat the execution.
>   
> The flag, which controls that re-execution is managed by another thread
> through atomic-box primitive.
>
> The code does not run as expected: the sleep executing sleep starts
> once, and change seems to remain invisible to main thread.

Can you describe in more detail the behavior you are seeing?  What
messages do you see on stderr from your 'dump' calls when it gets stuck?
What state is the atomic box left in?

I see a couple of problems with the code below:

* Two threads write concurrently to the same port, namely
  (current-error-port).  You should use thread synchronization to ensure
  that operations to a given port are serialized.

* There's a race condition when the atomic box is in the #:accepted
  state.  If the main thread finds the box in that state, it changes the
  state to #:need-to-sleep, which will apparently cause the other thread
  to sleep again.

I'm not sure if these problems could explain what you're seeing.

 Regards,
   Mark


> The code works as expected, when:
>
>   1. (sleep 1) is used instead (system* "sleep" "1s");
>   2. running the code on x86_64 cpus.
>
> The use of affinity mask
>
>   taskset 0x01 guile test.scm
>
> does not help.
>
> - MB. Respectfully
>
> P.S. Additional information:
>
> $ guile --version  
> guile (GNU Guile) 2.2.4
> Copyright (C) 2018 Free Software Foundation, Inc.
>
> License LGPLv3+: GNU LGPL 3 or later .
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
>
> $ sh config.guess
> armv7l-unknown-linux-gnueabihf
>
> $ pacman -Qi guile 
> Name: guile
> Version : 2.2.4-1
> Description : Portable, embeddable Scheme implementation written in C
> Architecture: armv7h
> URL : https://www.gnu.org/software/guile/
> Licenses: GPL
> Groups  : None
> Provides: None
> Depends On  : gmp  libltdl  ncurses  texinfo  libunistring  gc  libffi
> Optional Deps   : None
> Required By : make
> Optional For: gnutls
> Conflicts With  : None
> Replaces: None
> Installed Size  : 40.83 MiB
> Packager: Arch Linux ARM Build System 
> Build Date  : Tue 03 Jul 2018 04:47:53 PM +05
> Install Date: Mon 09 Jul 2018 01:02:48 PM +05
> Install Reason  : Installed as a dependency for another package
> Install Script  : No
> Validated By: Signature
>
> (use-modules (ice-9 atomic)
>  (ice-9 threads))
>
> (define dump
>   (let ((p (current-error-port)))
> (lambda (fmt . args) (apply format p fmt args
>
> (define state (make-atomic-box #:nothing-to-do))
>
> (define (expect e v)
>   (when (not (eq? e v))
> (error "Protocol violation; (expecting . got):" (cons e v 
>
> (define (sleep-loop)
>   (expect #:need-to-sleep (atomic-box-ref state))
>
>   (dump "R: Going to (sleep 1)~%")
>   (atomic-box-set! state #:accepted)
>   
>   ; SOMETHING WRONG IS HERE
>   (system* "sleep" "1s")
>   ; (sleep 1)
>
>   (let ((v (atomic-box-compare-and-swap! state #:accepted #:nothing-to-do)))
> (when (not (eq? #:accepted v))
>   (dump "S: Repeating sleep~%")
>   (sleep-loop)))
>
>   (dump "R: sleep-loop finished~%"))
>
> (define (main-loop)
>   (define (run-thread)
> (dump "M: new sleep thread~%")
> (atomic-box-set! state #:need-to-sleep)
> (call-with-new-thread sleep-loop)) 
>
>   (dump "M: (sleep 3)~%")
>   (sleep 3)
>
>   (dump "M: protocol exchange~%")
>   (let ((st (atomic-box-ref state)))
> (case st 
>   ((#:nothing-to-do) (run-thread))
>   ((#:accepted) (let ((v (atomic-box-compare-and-swap! state #:accepted 
> #:need-to-sleep)))
>   (when (not (eq? #:accepted v))
> (expect #:accepted v)
> (run-thread
>   (else (expect #:need-to-sleep st
>
>   (main-loop))
>
> (main-loop)

bug#15221: reiterating the request

[Mark H Weaver]

Hi Tom,

Tom Tromey  writes:
> Hi.  I tried guile today and was surprised to find that it doesn't use
> readline by default.  I think this would be a big improvement to the
> REPL.
>
> Second best would be a command line option, or editing some config file.

You can enable it by adding the following two lines to your ~/.guile:

  (use-modules (ice-9 readline))
  (activate-readline)

See .

I'm sympathetic to the desire to enable it by default, but I'm not sure
how to navigate the license issue, namely that GNU Readline is
GPL-licensed, whereas Guile is LGPL-licensed.

 Regards,
   Mark

bug#32644: bytevector bug

[Mark H Weaver]

Stefan Israelsson Tampe  writes:

> The code velow does not compile when the define-inlinable of id is active. If 
> in stead
> id defined by define is used it all compiles just fine.
>
> Tested on latest tar ball for guile 2.4

Did you mean to write guile 2.2.4?

  Mark


> ---
> (use-modules (rnrs bytevectors))
>
> (define-inlinable (id x) x)
> ;(define (id x) x)
> (define-syntax-rule (mkcrc crc_hqx high xor mask)
>   (define (crc_hqx data value)
>(let ((n (bytevector-length data))
>  (d data))
> (let lp ((i 0) (v value))
>(if (< i n)
>(let ((b (id (bytevector-u8-ref d i
> (let lp2 ((j 0) (x 1) (v v))
>(if (> j -8)
>(let ((bit   (ash (logand x b) j))
>  (hbit  (logand v high)))
> (if (= hbit 0)
>  (lp2 (- j 1) (ash x 1) (logior bit (ash v 1)))
>  (lp2 (- j 1) (ash x 1) (logxor
>  xor
>  (logand mask
>  (logior
>   bit
>   (ash v 1)))
>(lp (+ i 1) v
>v)
>
> (mkcrc crc_hqx #x8000 #x1021 #x)
> (mkcrc crc32   #x8000 #x04c11db7 #x)

bug#32528: http-post breaks with XML response payload containing boundary

[Mark H Weaver]

Mark H Weaver  writes:

> Ricardo Wurmus  writes:
>
>> I’m having a problem with http-post and I think it might be a bug.  I’m
>> talking to a Debbugs SOAP service over HTTP by sending (via POST) an XML
>> request.  The Debbugs SOAP service responds with a string of XML.
[...]
> The problem is simply that our Content-Type header parser is broken.
> It's very simplistic and merely splits the string wherever ';' is found,
> and then checks to make sure there's only one '=' in each parameter,
> without taking into account that quoted strings in the parameters might
> include those characters.
>
> I'll work on a proper parser for Content-Type headers.

I've attached preliminary patches to fix the Content-Type header parser,
and also to fix the parsing of response header lines to support
continuation lines.

With these patches applied, I'm able to fetch and decode the SOAP
response that you fetched with your 'wget' example, as follows:

--8<---cut here---start->8---
mhw@jojen ~/guile-stable-2.2 [env]$ meta/guile
GNU Guile 2.2.4.10-4c91d
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (use-modules (web http) (web uri) (web client) (sxml 
simple) (ice-9 receive))
scheme@(guile-user)> ,pp (let ((req-xml "http://schemas.xmlsoap.org/soap/envelope/\; 
xmlns:xsi=\"http://www.w3.org/1999/XMLSchema-instance\; 
xmlns:xsd=\"http://www.w3.org/1999/XMLSchema\; 
xmlns:soapenc=\"http://schemas.xmlsoap.org/soap/encoding/\; 
soapenc:encodingStyle=\"http://schemas.xmlsoap.org/soap/encoding/\;>http://schemas.xmlsoap.org/soap/encoding/\;>32514"))
   (receive (response body-port)
   (http-post "https://debbugs.gnu.org/cgi/soap.cgi;
  #:streaming? #t
  #:body req-xml
  #:headers
  `((content-type . (text/xml))
(content-length . ,(string-length 
req-xml
 (set-port-encoding! body-port "UTF-8")
 (xml->sxml body-port #:trim-whitespace? #t)))
$1 = (*TOP* (*PI* xml "version=\"1.0\" encoding=\"UTF-8\"")
   (http://schemas.xmlsoap.org/soap/envelope/:Envelope
 (@ (http://schemas.xmlsoap.org/soap/envelope/:encodingStyle
  "http://schemas.xmlsoap.org/soap/encoding/;))
 (http://schemas.xmlsoap.org/soap/envelope/:Body
   (urn:Debbugs/SOAP:get_bug_logResponse
 (http://schemas.xmlsoap.org/soap/encoding/:Array
   (@ (http://www.w3.org/1999/XMLSchema-instance:type
"soapenc:Array")
  (http://schemas.xmlsoap.org/soap/encoding/:arrayType
"xsd:ur-type[4]"))
   (urn:Debbugs/SOAP:item
 (urn:Debbugs/SOAP:header
   (@ (http://www.w3.org/1999/XMLSchema-instance:type
"xsd:string"))
   "Received: (at submit) by debbugs.gnu.org; 23 Aug 2018 
20:17:46 +\nFrom debbugs-submit-boun...@debbugs.gnu.org [...]
[...]
--8<---cut here---end--->8---

Note that I needed to make two other changes to your preliminary code,
namely:

* I passed "#:streaming? #t" to 'http-post', to ask for a port to read
  the response body instead of reading it eagerly.

* I explicitly set the port encoding to "UTF-8" on that port before
  using 'xml->sxml' to read it.

Otherwise, the entire 'body' response will be returned as a bytevector,
because the response Content-Type is not recognized as a textual type.
The HTTP Content-Type is "multipart/related", with a parameter:
type="text/xml".  I'm not sure if we should be automatically
interpreting that as a textual type or not.

There's no 'charset' parameter in the Content-Type header, but the XML
internally specifies: encoding="UTF-8".

Anyway, here are the preliminary patches.

   Mark


>From 41764d60dba80126b3c97f883d0225510b55f3fa Mon Sep 17 00:00:00 2001
From: Mark H Weaver 
Date: Tue, 28 Aug 2018 18:39:34 -0400
Subject: [PATCH 1/2] web: Add support for HTTP header continuation lines.

* module/web/http.scm (spaces-and-tabs, space-or-tab?): New variables.
(read-header-line): After reading a header, if a space or tab follows,
then read the continuation lines and append them all together.
---
 module/web/http.scm | 31 ---
 1 file changed, 24

bug#32528: http-post breaks with XML response payload containing boundary

[Mark H Weaver]

Ricardo Wurmus  writes:

> I’m having a problem with http-post and I think it might be a bug.  I’m
> talking to a Debbugs SOAP service over HTTP by sending (via POST) an XML
> request.  The Debbugs SOAP service responds with a string of XML.
>
> Here’s a simplified version of what I do:
>
>   (use-module (web http))
>   (let ((req-xml ""))
> (receive (response body)
> (http-post uri
>#:body req-xml
>#:headers
>`((content-type . (text/xml))
>  (content-length . ,(string-length req-xml
>  ;; Do something with the response body
>  (xml->sxml body #:trim-whitespace? #t)))
>
> This fails for some requests with an error like this:
>
> web/http.scm:1609:23: Bad Content-Type header: multipart/related; 
> type="text/xml"; start=""; boundary="=-=-="

[...]

> The reason why it fails is that Guile processes the response and treats
> the *payload* contained in the XML response as HTTP.

No, this was a good guess, but it's not actually the problem.

If you add --save-headers to the wget command line, you'll see the full
response, and the HTTP headers are what's being parsed, as it should be.
It looks like this (except that I removed the carriage returns below):

  HTTP/1.1 200 OK
  Date: Tue, 28 Aug 2018 21:40:30 GMT
  Server: Apache
  SOAPServer: SOAP::Lite/Perl/1.11
  Strict-Transport-Security: max-age=63072000
  Content-Length: 32650
  X-Content-Type-Options: nosniff
  X-Frame-Options: sameorigin
  X-XSS-Protection: 1; mode=block
  Keep-Alive: timeout=5, max=100
  Connection: Keep-Alive
  Content-Type: multipart/related; type="text/xml"; start=""; 
boundary="=-=-="
  
  

bug#32501: Bouncing parentheses broken in REPL with vi-mode enabled

[Mark H Weaver]

Hi Daniel,

Daniel Tam  writes:
> I'm excited to have a look but I have 0 knowledge of the code base (and
> am just learning Scheme although that's probably not an issue in this
> case whatsoever).

It's okay, please don't feel any obligation to work on this.  Sometimes
people are impatient to fix a problem ASAP, and that's why I wrote these
preliminary emails.

It's probably better for you to focus on learning Scheme for now, rather
than mucking around in the underbelly of an implementation :)

I appreciate you taking the time to file this bug report.  That's quite
useful by itself.

  Thanks!
Mark

bug#32501: Bouncing parentheses broken in REPL with vi-mode enabled

[Mark H Weaver]

I wrote:

>> The right fix might be to simply remove the 'if' check above.
>
> Actually, it probably won't be that simple.  Whereas in emacs mode, we
> can probably rely on these bindings being added to the correct keymap,
> namely 'emacs_standard_keymap', when in vi mode it's likely that these
> bindings would end up in the wrong keymap, namely 'vi_movement_keymap'.
>
> When in vi mode, these mappings should be added to
> 'vi_insertion_keymap', and that probably involves using
> 'rl_bind_key_in_map' instead of 'rl_bind_key'.

And I guess 'rl_get_keymap_by_name' is the function we should use to get
the keymap.  From keymaps.h in readline:

  /* Return the keymap corresponding to a given name.  Names look like
 `emacs' or `emacs-meta' or `vi-insert'.  */
  extern Keymap rl_get_keymap_by_name PARAMS((const char *));

  Mark

bug#32501: Bouncing parentheses broken in REPL with vi-mode enabled

[Mark H Weaver]

Hello again,

I wrote:

> The relevant function is 'init_bouncing_parens' in
> guile-readline/readline.c:
>
>   static void
>   init_bouncing_parens ()
>   {
> if (strncmp (rl_get_keymap_name (rl_get_keymap ()), "vi", 2))
>   {
> rl_bind_key (')', match_paren);
> rl_bind_key (']', match_paren);
> rl_bind_key ('}', match_paren);
>   }
>   }

[...]

> The right fix might be to simply remove the 'if' check above.

Actually, it probably won't be that simple.  Whereas in emacs mode, we
can probably rely on these bindings being added to the correct keymap,
namely 'emacs_standard_keymap', when in vi mode it's likely that these
bindings would end up in the wrong keymap, namely 'vi_movement_keymap'.

When in vi mode, these mappings should be added to
'vi_insertion_keymap', and that probably involves using
'rl_bind_key_in_map' instead of 'rl_bind_key'.

   Mark

bug#31878: Module autoloading is not thread safe

[Mark H Weaver]

Hi Ludovic,

l...@gnu.org (Ludovic Courtès) writes:

> Mark H Weaver  skribis:
>
>> Since Guile (unfortunately) allows cyclic module dependencies, we would
>> need a mechanism to avoid deadlocks in case modules A and B both import
>> each other, and two threads concurrently attempt to load those modules.
>>
>> The first idea that comes to mind is to also have a global structure
>> storing a partial order on the modules currently being loaded.  If,
>> while module A is being loaded, there's an attempt to auto-load module
>> B, then an entry (A < B) would added to the partial order.  The partial
>> order would not allow cycles to be introduced, reporting an error in
>> that case.  In case a cycle would be introduced when adding (A < B),
>> then the thread would simply be given access to the partially-loaded
>> module B, by adding B to its local list of modules-being-loaded.
>
> Would it enough to (1) use recursive mutexes, and (2) have
> ‘resolve-module’ lookup modules first in the global name space, and
> second in the local list of modules being loaded?

Item (2) above is something that I had already envisioned in my
proposal, although I neglected to mention it.

However, I don't see how recursive mutexes would help here, or how they
could obviate the need for the other mechanisms I described above.

Suppose module A and module B are mutually dependent on each other.  If
thread 1 is loading module A concurrently with thread 2 loading module
B, then thread 1 will be the only thread with access to module A (via
thread 1's local list) and will hold the lock on it, and similarly for
thread 2 and module B.

Now, when thread 1 tries to load module B (while it's in the process of
loading module A), it should normally be blocked until module B is
finished loading.  If those modules were _not_ mutually dependent on
each other, we should insist on thread 1 waiting for module B to finish
loading before gaining access to it.  Only if there is a cyclic
dependency should it be granted access to the partially-loaded module.

If we simply use recursive mutexes, I think deadlock would occur in this
case.  Thread 1 would try to grab the lock on module B, which is already
held by thread 2, and vice versa.  Since it's not self-held, I fail to
see the relevance of the recursive mutex.

What do you think?

  Mark

bug#32501: Bouncing parentheses broken in REPL with vi-mode enabled

[Mark H Weaver]

Hi again,

> Daniel Tam  writes:
>
>> I've activated readline support for the Guile repl, but I've found that
>> if my inputrc enables vi-mode, then the bouncing parentheses feature
>> doesn't work. Disabling vi-mode does the trick.
>
> Indeed.  For some reason that I cannot determine, the bouncing
> parentheses feature is specifically disabled when the vi keymap is in
> use.

I think I now see the reason for it.  I noticed that readline's default
vi keymap includes a binding for '%', which jumps to the paren matching
the one under the cursor.  That reminded me, from many years ago when I
used vi more often, that this is the way that old vi traditionally
allows matching parens to be found.

So, I guess the decision long ago to disable bouncing parens when in vi
mode was to match the way that emacs and vi behaved at that time.

However, I just tried modern vim, and I see that it now highlights
matching parens by default.  So, we should probably remove the 'if' to
match this newer behavior.

   Mark

bug#32501: Bouncing parentheses broken in REPL with vi-mode enabled

[Mark H Weaver]

Hi Daniel,

Daniel Tam  writes:

> I've activated readline support for the Guile repl, but I've found that
> if my inputrc enables vi-mode, then the bouncing parentheses feature
> doesn't work. Disabling vi-mode does the trick.

Indeed.  For some reason that I cannot determine, the bouncing
parentheses feature is specifically disabled when the vi keymap is in
use.  The relevant function is 'init_bouncing_parens' in
guile-readline/readline.c:

  static void
  init_bouncing_parens ()
  {
if (strncmp (rl_get_keymap_name (rl_get_keymap ()), "vi", 2))
  {
rl_bind_key (')', match_paren);
rl_bind_key (']', match_paren);
rl_bind_key ('}', match_paren);
  }
  }

This is ancient code, predating version control, present in the original
import into CVS in 1999.

I looked at the source code of readline-7.0, and IIUC none of those keys
have mappings in the default vi keymap.

The right fix might be to simply remove the 'if' check above.  Would you
like to try it and report back?

  Thanks,
Mark

bug#31878: Module autoloading is not thread safe

[Mark H Weaver]

I wrote:

> I thought about how to fix this thread-safety problem a long time ago,
> and came up with a rough outline of a solution.  The idea is that the
> module should not be added to the global module table until the module
> has finished loading.  While the module is being loaded, it would be
> made visible only to the loading thread, and to any other threads
> spawned during the loading process, by adding the module to a local list
> of modules-being-loaded referenced by a fluid variable.  If any other
> threads attempt to access the module, it would not be found in the
> global module table, and thus trigger an auto-load, which would wait for
> the lock to be released before proceeding.

I forgot to mention an important aspect of the proposed auto-load
locking here.  It would not be a global lock, but rather a lock specific
to the module being loaded.  So, I guess we would need a global table of
locks for modules-being-loaded.

Since Guile (unfortunately) allows cyclic module dependencies, we would
need a mechanism to avoid deadlocks in case modules A and B both import
each other, and two threads concurrently attempt to load those modules.

The first idea that comes to mind is to also have a global structure
storing a partial order on the modules currently being loaded.  If,
while module A is being loaded, there's an attempt to auto-load module
B, then an entry (A < B) would added to the partial order.  The partial
order would not allow cycles to be introduced, reporting an error in
that case.  In case a cycle would be introduced when adding (A < B),
then the thread would simply be given access to the partially-loaded
module B, by adding B to its local list of modules-being-loaded.

Comments and suggestions welcome,

Mark

bug#32367: 2.2.4 hangs when a script uses a module that calls sigaction

[Mark H Weaver]

Hi Derek,

Derek Upham  writes:

> Thanks.  I built from 4c91de3e4 and was able to reproduce the problem.
> Then I reverted 761cf0fb8c364e885e4c6fced34563f8157c3b84 and I was not
> able to reproduce it.

Thanks for checking.  I've reopened  which
led to the faulty commit, and sent some messages there proposing a rough
outline for a more proper fix.

  Mark

bug#31878: Module autoloading is not thread safe

[Mark H Weaver]

Hi Ludovic,

l...@gnu.org (Ludovic Courtès) writes:

> l...@gnu.org (Ludovic Courtès) skribis:
>
>> l...@gnu.org (Ludovic Courtès) skribis:
>>
>>> I believe this comes from the fact that ‘autoloads-done’ and related
>>> alists in (ice-9 boot-9) are manipulated in a non-thread-safe fashion.
>>
>> Here’s a proposed fix for ‘stable-2.2’ as discussed on #guile, Andy:
>
> After further discussion on IRC, I pushed a variant of this patch as
> commit 761cf0fb8c364e885e4c6fced34563f8157c3b84.

There are problems with this fix, e.g. .

More generally, nearly arbitrary code can be run in the top-level
expressions of a module.  It could launch other threads which try to
load modules, or even send messages to other existing threads asking
them to do work.  In some cases, the body of the module might never
terminate.  The entire main program might be run from there.  I suspect
that's not unusual.

I can see another problem as well: while the module is in the process of
loading, the partially-loaded module is globally visible and accessible
to other threads.  If I'm not mistaken, with this patch, there's nothing
preventing other threads from attempting to use the partially-loaded
module.

I thought about how to fix this thread-safety problem a long time ago,
and came up with a rough outline of a solution.  The idea is that the
module should not be added to the global module table until the module
has finished loading.  While the module is being loaded, it would be
made visible only to the loading thread, and to any other threads
spawned during the loading process, by adding the module to a local list
of modules-being-loaded referenced by a fluid variable.  If any other
threads attempt to access the module, it would not be found in the
global module table, and thus trigger an auto-load, which would wait for
the lock to be released before proceeding.

What do you think?

  Mark

bug#32367: 2.2.4 hangs when a script uses a module that calls sigaction

[Mark H Weaver]

I don't have time now for a proper investigation, but this might be
related to commit 761cf0fb8c364e885e4c6fced34563f8157c3b84 (Make module
autoloading thread-safe).

  Mark

bug#32369: git guile missing files from repository when cloning

[Mark H Weaver]

tags 32369 + notabug
close 32369
thanks

Hi,

Java House  writes:

> I am trying to install guile by following instructions.
> I cloned the source code in my unix FreeBSD 11.2-RELEASE
> git clone git://git.sv.gnu.org/guile.git
>
> And tried to follow instructions in the README file.
>
> unfortunately the INSTALL file as well as the ./configure file are
> missing from the guile directory.

When building from a git checkout, you must begin by running
./autogen.sh in the top-level source directory, to generate the
./configure script and others.  Note that more tools are required to
perform this step than to build from a tarball release.  See the
'HACKING' file in the top-level source directory for details.

  Mark

bug#32160: auto compile fails to recompile when included source files change

[Mark H Weaver]

Hi Arun,

Arun Isaac  writes:

>>> If you'd like to investigate further, I'd be glad to give you
>>> pointers and advice, but it's not a project for the faint-hearted :-/
>
> I am faint-hearted and do not wish to pursue this further. :-P I just
> thought it might be useful to report. Should I close this bug report, or
> leave it open for someone to fix in the future?

I'd prefer to leave it open, because it is indeed an important issue
that would be good to fix eventually.

  Thanks!
Mark

bug#32160: auto compile fails to recompile when included source files change

[Mark H Weaver]

Mark H Weaver  writes:

> My response to Arun Isaac  was rejected
> by his mail server:
[...]
> If someone could let him know about this issue, I would be grateful.

to...@tuxteam.de kindly forwarded my messages to Arun, so there's no
need for anyone else to do it now.  Thanks, Tomas!

  Mark

bug#32160: auto compile fails to recompile when included source files change

[Mark H Weaver]

My response to Arun Isaac  was rejected
by his mail server:

  SMTP error from remote mail server after RCPT TO::
  550 Sender Policy Framework (SPF) verification failed

I do not have DNS records supporting SPF for my domain netris.org, and
apparently his mail server is configured to reject all mail that is not
authenticated by SPF.  So, I'm unable to send him email.  Oh well.  This
is the first time I've run into this problem.

If someone could let him know about this issue, I would be grateful.

  Mark