Re: Maintenance and future of Guile

2021-12-21 Thread Ludovic Courtès
Hi,

Olivier Dion  skribis:

> On Fri, 17 Dec 2021, Ludovic Courtès  wrote:
>> Hi,
>>
>> Olivier Dion  skribis:
>>
>>> I would also like to contribute in some meaningful way.  In what way
>>> someone with none wizard knowledge of Scheme can contribute the most to the
>>> project?
>>
>> Triage of bugs and patches is always welcome I guess, and communicating
>> what needs to be applied/addressed first to whoever can actually commit
>> it.  That’s one possible way to help.
>
> Where can this be done?  I know that Guix is using debbugs, but do Guile
> does the same or is it all tracked on Savannah?

It’s happening on debbugs.gnu.org as well.  Debbugs this is easier to
work with via Emacs debbugs.el, which is nice if you already use Emacs
but otherwise unfortunate.

> Also, any way to help on the C side?

There isn’t much happening on the C side.  Maxime Devos submitted
patches adding bindings for openat(2) and friends that are unfortunately
still pending review.

Patches that add POSIX bindings and similar to libguile should in
general be relatively easy to review (though the openat(2) one may be
trickier because it’s a central functionality and we’d rather get the
interface right.)

Thanks,
Ludo’.



Re: Maintenance and future of Guile

2021-12-17 Thread Ludovic Courtès
Hi,

Olivier Dion  skribis:

> I would also like to contribute in some meaningful way.  In what way
> someone with none wizard knowledge of Scheme can contribute the most to the
> project?

Triage of bugs and patches is always welcome I guess, and communicating
what needs to be applied/addressed first to whoever can actually commit
it.  That’s one possible way to help.

Thanks,
Ludo’.



Re: Maintenance and future of Guile

2021-12-15 Thread Ludovic Courtès
Hi,

Blake Shaw  skribis:

> Ludovic Courtès  writes:
>
>> If someone is interested, please get in touch with us!
>>
>> While Andy focuses on major improvements to the compiler and VM with a
>> long-term vision, I think it would be great to also have people on the
>> maintainer team focusing on more day-to-day operations: incremental
>> improvements, bug fixes, etc.  I think we’re lucky that there’ve been a
>> number of talented contributors chiming in over the last couple of
>> years; let’s take advantage of this, at last!
>>
>
> Just want to chime in as a new Guile to say that while I'm still
> learning, I'm happy to work on any "chores" if there is guidance on how
> to do so. exploring guile has been half the fun so far.
>
> I know CHICKEN has a team of "janitors"; perhaps Guile could benefit
> from adopting a similar approach? 

Surely.  For my part, while I won’t pretend to do actual work myself, I
can dedicate time to mentor someone willing to step up.

Ludo’.



Re: Maintenance and future of Guile

2021-12-15 Thread Ludovic Courtès
Hello Guilers,

Jean Abou Samra  skribis:

> I see a number of similar cases on the mailing list.
>
> I understand the cost of reviewing and I know that
> nobody is entitled to anything in the free software
> world. However, I would like to voice the concern that
> Guile's maintenance is not scaling, and the project
> cannot attract new contributors if patches do not
> make it to the main branch. If the current maintainers
> need to drop their activity, it would be nice if
> they could share maintainership so that at least
> bug fixes can be applied.

I agree and I apologize for the unpleasant situation: it’s clear that I
haven’t been up to the task for several years already.  I’ll discuss
with Andy how I can formally leave the seat to someone with more energy,
whom I’ll be happy to help learn practices and conventions.

If someone is interested, please get in touch with us!

While Andy focuses on major improvements to the compiler and VM with a
long-term vision, I think it would be great to also have people on the
maintainer team focusing on more day-to-day operations: incremental
improvements, bug fixes, etc.  I think we’re lucky that there’ve been a
number of talented contributors chiming in over the last couple of
years; let’s take advantage of this, at last!

Thanks,
Ludo’.



Re: Guile 3 and wip-elisp/Emacs

2021-10-19 Thread Ludovic Courtès
Hello!

Christine Lemmer-Webber  skribis:

> I've pushed this as origin/wip-elisp-rebased.  I actually rebased it
> again, making some naming adjustments for myself and a couple of
> adjustments having talked to Robin.
>
> If nobody objects, I'd like to merge this into main.  Maintainers, if
> you have any objections, speak now or forever hold these commits!

I haven’t looked at the branch, but I think it’s great to see it live
and it’s great if it can be merged!

Some things to pay attention to before merging to ‘main’, since it
corresponds to the current 3.0 stable branch:

  • Make sure no backward incompatibilities are introduced in
preexisting modules;

  • Make sure the ABI of libguile-3.0.so and that of public modules
is unchanged, or is changed compatibly;

  • Make sure there are reasonable tests and doc so it can be maintained
(and used!) going forward;

  • Robin has a copyright assignment on file, so we should be fine
(whether we’ll keep doing copyright assignment for Guile is still
unclear, but we can discuss that separately).

I think we should also wait for a green light from Andy.

Thanks Gregg, Christine, Robin, and everyone involved!

Ludo’.



Re: new maintainers for guile libs

2021-08-04 Thread Ludovic Courtès
Hi Mike,

(Catching up on email…)

Mike Gran  skribis:

> Well 2021 has been a strange time for me as it has been for many.  At
> the beginning of 2021, I had the free time and the good health that
> allowed me to imagine being a real contributer again, but, stuff has
> changed.

I’m sad to read this; I hope you’ll recover soon and feel energized,
including to hack the good hack.

> So there are a few libraries in the Guile ecosystem I've worked on,
> and I'm hoping someone else might be interested in owning them
> henceforth.
>
> Roughly in order of maintainability, from simple to complex, these are
>
> - guile-aspell: a very simple binding to the Aspell spellcheck
>   library. This would be an easy task, since it is a Guile-only
>   binding with no C, and aspell rarely changes
> - guile-curl: a C-based but straight-forward binding the the cURL API.
> - guile-ncurses: a C-based binding for NCurses, libpanel, and libmenu.  It
>   is a bit convoluted bcause the memory model for ncurses it iself
>   quite strange.
>
> Then there a couple of very obscure and little used libraries that I could
> punt on, but, I don't really expect any interest
>
> - guile-plotutils: draw plots in Guile using the plotutils backend
> - zile-on-guile: a version of the zile editor, replacing its tiny LISP
>   interpreter with Guile.
>
> Guile-GI is a special case because at this point, I'm probably not the
> prinmary contributor. But it could use more contributers and could use
> a discussion about how to move it forward. I still hope to contribute
> as I can.

I won’t commit to anything (I’d do a poor job), but these are all useful
pieces of software and I hope someone picks them up!

Thanks,
Ludo’.




GNU Guile 3.0.4 released

2020-06-24 Thread Ludovic Courtès
We are pleased but also embarrassed to announce GNU Guile release 3.0.4.
This release fixes the SONAME of libguile-3.0.so, which was wrongfully
bumped in 3.0.3.  Distributions should use 3.0.4.  Apologies!

 *  *  *

The Guile web page is located at http://gnu.org/software/guile/, and
among other things, it contains a copy of the Guile manual and pointers
to more resources.

Guile is an implementation of the Scheme programming language, packaged
for use in a wide variety of environments.  In addition to implementing
the R5RS, R6RS, and R7RS Scheme standards, Guile includes full access to
POSIX system calls, networking support, multiple threads, dynamic
linking, a foreign function call interface, powerful string processing,
and HTTP client and server implementations.

Guile can run interactively, as a script interpreter, and as a Scheme
compiler to VM bytecode.  It is also packaged as a library so that
applications can easily incorporate a complete Scheme interpreter/VM.
An application can use Guile as an extension language, a clean and
powerful configuration language, or as multi-purpose "glue" to connect
primitives provided by the application.  It is easy to call Scheme code
from C code and vice versa.  Applications can add new functions, data
types, control structures, and even syntax to Guile, to create a
domain-specific language tailored to the task at hand.

Guile 3.0 can be installed in parallel with Guile 2.2.x; see
http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html.

 *  *  *

Changes in 3.0.4 (since 3.0.3)

This release fixes the SONAME of libguile-3.0.so, which was erroneously
bumped in 3.0.3 compared to 3.0.2.  Distributions are strongly
encouraged to use 3.0.4 instead of 3.0.3.

Thanks to Chris Vine for reporting the issue.


 *  *  *

Here are the compressed sources:
  https://ftp.gnu.org/gnu/guile/guile-3.0.4.tar.gz   (21MB)
  https://ftp.gnu.org/gnu/guile/guile-3.0.4.tar.lz   (11MB)
  https://ftp.gnu.org/gnu/guile/guile-3.0.4.tar.xz   (13MB)

Here are the GPG detached signatures[*]:
  https://ftp.gnu.org/gnu/guile/guile-3.0.4.tar.gz.sig
  https://ftp.gnu.org/gnu/guile/guile-3.0.4.tar.lz.sig
  https://ftp.gnu.org/gnu/guile/guile-3.0.4.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA256 checksums:

  13f2f61a665469d330e651c8e9f7faecbb87fe474767f9532f2e821701730957  
guile-3.0.4.tar.gz
  f2b0b66fd72bc24df76856e1549bf327c025b8a5d5fa3cb1696327c5941c89c4  
guile-3.0.4.tar.lz
  6b7947dc2e3d115983846a268b8f5753c12fd5547e42fbf2b97d75a3b79f0d31  
guile-3.0.4.tar.xz

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify guile-3.0.4.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver pool.sks-keyservers.net \
  --recv-keys 3CE464558A84FDC69DB40CFB090B11993D9AEBB5

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.69
  Automake 1.16.2
  Libtool 2.4.6
  Makeinfo 6.7
  Gnulib v0.1-1157-gb03f418

Happy hacking with Guile!

Ludovic Courtès and Andy Wingo.


signature.asc
Description: PGP signature


Re: GNU Guile 3.0.3 released

2020-06-24 Thread Ludovic Courtès
Hello,

Andy Wingo  skribis:

> On Tue 23 Jun 2020 11:36, Chris Vine  writes:

[...]

>>> I was hesitant about the SONAME: the ABI jump was unnecessary unless in
>>> ‘--disable-deprecated’ builds.  I erred on the side of cautiousness:
>>> 
>>>   
>>> https://git.savannah.gnu.org/cgit/guile.git/commit/?id=5d052c87bd8f0fd894e67f0bebd4fa6f6160d83c
>>
>> Hi,
>>
>> Ah right.  There must have been two SO breaks between guile-3.0.2 and
>> guile-3.0.3.
>>
>> It's a nuisance having SO bumps on micro releases and I wonder if that
>> could be included in the announcement so that you don't first notice it
>> when stuff fails to run?
>
> I think I agree with Chris.  The intention is certainly to have a stable
> ABI within a stable series, so 3.0.3 should have the same CURRENT.
>
> It's certainly correct that a --disable-deprecated 3.0.3 build has a
> different ABI than 3.0.2, and if that were what we were looking at, we
> would indeed need the CURRENT version bump; but I think the premise is
> wrong: we do *not* have a stable ABI in --disable-deprecated builds, and
> we never have.  Otherwise we wouldn't ever be able to deprecate anything
> within a stable series.

Following our discussion on IRC, I agree with restoring CURRENT and will
push a 3.0.4 in that direction.

Apologies for the annoyance!

Ludo’.




GNU Guile 3.0.3 released

2020-06-21 Thread Ludovic Courtès
ve been fixed
   (<https://bugs.gnu.org/40737>,
<https://gitlab.com/wingo/lightening/-/issues/12>)

** 'http-get', 'http-post', etc. now honor #:verify-certificates?
   (<https://bugs.gnu.org/40486>)

** web: Accept URI host names consisting only of hex digits
   (<https://bugs.gnu.org/40582>)

** (web http) parser recognizes the CONNECT and PATCH methods

** Initial revealed count of file ports is now zero
   (<https://bugs.gnu.org/41204>)

* New deprecations

** Old bitvector interfaces deprecated

See "Bit Vectors" in the manual, for details on all of these
replacements.

*** bit-count, bit-position

Use bitvector-count or bitvector-position instead.  

*** bitvector-ref

Use 'bitvector-bit-set?' or 'bitvector-bit-clear?' instead.

*** bitvector-set!

Use 'bitvector-set-bit!' or 'bitvector-clear-bit!' instead.

*** bitvector-fill!

Use 'bitvector-set-all-bits!' or 'bitvector-clear-all-bits!' instead.

*** bit-invert!

Use 'bitvector-flip-all-bits! instead.

*** bit-set*!

Use 'bitvector-set-bits!' or 'bitvector-clear-bits!' instead.

*** bit-count*

Use 'bitvector-count-bits' instead, subtracting from 'bitvector-count'
on the mask bitvector if you are counting unset bits.

*** Accessing generic arrays using the bitvector procedures

For the same efficiency reasons that use of 'vector-ref' on generic
arrays was deprecated in Guile 2.0.10, using 'bitvector->list' and
similar procedures on 1-dimensional boolean-typed arrays is now
deprecated.  Use 'array-ref' and similar procedures on arrays.

*** scm_istr2bve

This C-only procedure to parse a bitvector from a string should be
replaced by calling `read' on a string port instead, if needed.

 *  *  *

Here are the compressed sources:
  https://ftp.gnu.org/gnu/guile/guile-3.0.3.tar.gz   (21MB)
  https://ftp.gnu.org/gnu/guile/guile-3.0.3.tar.lz   (11MB)
  https://ftp.gnu.org/gnu/guile/guile-3.0.3.tar.xz   (13MB)

Here are the GPG detached signatures[*]:
  https://ftp.gnu.org/gnu/guile/guile-3.0.3.tar.gz.sig
  https://ftp.gnu.org/gnu/guile/guile-3.0.3.tar.lz.sig
  https://ftp.gnu.org/gnu/guile/guile-3.0.3.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA256 checksums:

  fb75d95d8f4aa2efd1d3a09643dc125698e8cf7497ecc9fdca27ad85e937  
guile-3.0.3.tar.gz
  eabb399928ef465d4dc2cf1107a2d410f32fcde3b6263edd0fbb0d521b778159  
guile-3.0.3.tar.lz
  b3242d61c5583560cbc9adadf1c8b4333139d9ad129d4481b5c6aa7cfa73e63b  
guile-3.0.3.tar.xz

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify guile-3.0.3.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver pool.sks-keyservers.net \
  --recv-keys 3CE464558A84FDC69DB40CFB090B11993D9AEBB5

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.69
  Automake 1.16.2
  Libtool 2.4.6
  Makeinfo 6.7
  Gnulib v0.1-1157-gb03f418

Thanks to everyone who helped make this release:

 1  Alex Sassmannshausen
 1  Andrew Gierth
   112  Andy Wingo
 4  Arun Isaac
 1  Christopher Baines
 1  Dale P. Smith
 2  Daniel Llorens
 1  Eric Bavier
11  Icecream95
 1  Jan (janneke) Nieuwenhuizen
 1  Jan Synacek
 1  Linus Björnstam
28  Ludovic Courtès
 1  Mathieu Lirzin
 2  Matt Wette
 1  Ricardo G. Herdt
 1  Rutger van Beusekom

Happy hacking with Guile!

Ludovic Courtès and Andy Wingo.


signature.asc
Description: PGP signature


The size of ‘.go’ files

2020-06-05 Thread Ludovic Courtès
Hello Guix!

On IRC there was a discussion about the size of ‘.go’ files.  The
discussion came from this observation:

--8<---cut here---start->8---
$ guix size $(readlink -f /run/current-system) | head -5
store item   totalself
/gnu/store/4d0p06xgaw8lqa9db0d6728kkba8bizj-qemu-5.0.01651.6   
745.2  18.8%
/gnu/store/abiva5ivq99x30r2s9pa3jj0pv9g16sv-guix-1.1.0-4.bdc801e   468.0   
268.8   6.8%
/gnu/store/111zp1qyind7hsnvrm5830jhankmx4ls-linux-libre-5.4.43 243.6   
243.6   6.2%
/gnu/store/skxkrhgn9z0fg9hmnbcyfdgzs5w4ryrr-llvm-9.0.1 199.9   
128.5   3.2%
--8<---cut here---end--->8---

On disk, those .go files take quite a bit of space (I hear you Btrfs
people, don’t say it! :-)).

The code snippet below sorts the ELF sections of a .go file by size; for
‘python-xyz.go’, I get this:

--8<---cut here---start->8---
$13 = ((".rtl-text" . 3417108)
 (".guile.arities" . 1358536)
 (".data" . 586912)
 (".rodata" . 361599)
 (".symtab" . 117000)
 (".debug_line" . 97342)
 (".debug_info" . 54519)
 (".guile.frame-maps" . 47114)
 ("" . 1344)
 (".guile.arities.strtab" . 681)
 ("" . 232)
 (".shstrtab" . 229)
 (".dynamic" . 112)
 (".debug_str" . 87)
 (".strtab" . 75)
 (".debug_abbrev" . 65)
 (".guile.docstrs.strtab" . 1)
 ("" . 0)
 (".guile.procprops" . 0)
 (".guile.docstrs" . 0)
 (".debug_loc" . 0))
scheme@(guile-user)> (stat:size (stat go))
$14 = 6083445
--8<---cut here---end--->8---

More than half of those 6 MiB is code, and more than 1 MiB is
“.guile.arities” (info "(guile) Object File Format"), which is
surprisingly large; presumably the file only contains thunks (the
‘thunked’ fields of ).

Stripping the .debug_* sections (if that works) clearly wouldn’t help.

So I guess we could generate less code (reduce ‘.rtl-text’), perhaps by
tweaking ‘define-record-type*’, but I have little hope there.

We could also investigate where “.guile.arities” could be made denser,
or use fewer thunked fields in .  Currently arity info takes
7x4 = 28 bytes per procedure as documented in (system vm assembler).
With an extra flag we could perhaps save 8 bytes for the simple case
where nopt = 0, nreq is small, and other flags are zero.

But anyway, currently there are (1358536 - 4) / 28 = 48K arity headers
in this file.  However, the file contains 970 packages, so we’re talking
about ~50 procedures per package, even though there are only 5 thunked
fields.  Weird!  Maybe I’m missing something.

But wait, that was with 3.0.2 and -O1.

With 3.0.3-to-be and -O1, python-xyz.go weighs in at 3.4 MiB instead of
5.9 MiB!  Here’s the section size distribution:

--8<---cut here---start->8---
$4 = ((".rtl-text" . 2101168)
 (".data" . 586392)
 (".rodata" . 360703)
 (".guile.arities" . 193106)
 (".symtab" . 117000)
 (".debug_line" . 76685)
 (".debug_info" . 53513)
 ("" . 1280)
 (".guile.arities.strtab" . 517)
 ("" . 232)
 (".shstrtab" . 211)
 (".dynamic" . 96)
 (".debug_str" . 87)
 (".strtab" . 75)
 (".debug_abbrev" . 56)
 (".guile.docstrs.strtab" . 1)
 ("" . 0)
 (".guile.procprops" . 0)
 (".guile.docstrs" . 0)
 (".debug_loc" . 0))
scheme@(guile-user)> (stat:size (stat go))
$5 = 3519323
--8<---cut here---end--->8---

“.rtl-text” is 38% smaller and “.guile.arities” is almost a tenth of
what it was.

Something’s going on here!  Thoughts?

Ludo’.

(use-modules (system vm elf)
 (rnrs io ports)
 (ice-9 match))

(define go
  (search-path %load-compiled-path "gnu/packages/python-xyz.go"))

(define elf
  (parse-elf (call-with-input-file go get-bytevector-all)))

(define (elf-section-name-as-string elf section)
  (let ((off (elf-section-offset
  (list-ref (elf-sections elf)
(elf-shstrndx elf)
(string-table-ref (elf-bytes elf)
  (+ off (elf-section-name section)

(sort (map (lambda (section)
 (cons (elf-section-name-as-string elf section)
   (elf-section-size section)))
   (elf-sections elf))
  (match-lambda*
(((name1 . size1) (name2 . size2))
 (> size1 size2


Re: [PATCH] Avoid malloc/free/dynwind for POSIX subrs that take strings

2020-05-22 Thread Ludovic Courtès
Hi!

Andy Wingo  skribis:

> On Sun 17 May 2020 23:46, Ludovic Courtès  writes:
>
>> The libunistring functions can take a pre-allocated buffer, but they
>> always malloc a fresh one if needed.  So the best we could do is have a
>> ‘scm_to_stringn’ variant that takes a buffer, but it’s not guaranteed
>> that it’ll actually be used.  All in all, it seems the added complexity
>> is not warranted.  The worst case of ‘scm_locale_string_data’ is also
>> rare enough.
>>
>> Thoughts?
>
> Interesting.  Probably we want to make a public
> scm_to_{,locale_,utf8_}gc_string{,n} API and use that as a fallback.
> GC-managed character buffers are less error-prone and probably just as
> fast.

Yeah.

> We can mostly avoid the double-copy by inline conversions, as we do with
> UTF-8.  For narrow strings scm_to_gc_stringn can always run iconv in a
> mode that just calculates output byte size; surely equivalent
> functionality is available from unistring, also.

Like I wrote, libunistring functions always malloc if the provided
buffer is not large enough to hold the converted string.  So we can’t
really ensure there won’t be any malloc.

Anyway, I’m putting this on hold for now!

Thanks,
Ludo’.



Re: [PATCH] http: Support CONNECT and PATCH HTTP methods.

2020-05-17 Thread Ludovic Courtès
Hello,

Christopher Baines  skribis:

> PATCH is described by RFC 5789 and CONNECT is described by RFC 7231.
>
> * module/web/http.scm (parse-http-method): Support CONNECT and PATCH.

Applied, thanks!

Ludo’.




Re: [PATCH] Avoid malloc/free/dynwind for POSIX subrs that take strings

2020-05-17 Thread Ludovic Courtès
Hi,

Andy Wingo  skribis:

> On Wed 29 Apr 2020 22:22, Ludovic Courtès  writes:
>
>> As discussed on IRC, the patch below arranges so that subrs that take
>> strings and pass them to syscall wrappers can avoid the
>> malloc/free/dynwind overhead.  This gives a 10% speedup on a tight loop
>> that calls these subrs:
>
> Neat optimization.  Since it's internal, no problem from me.  My concern
> is only about calcifying aspects of our C / GC interface in API.
>
>> On IRC, Andy mentioned concerns that the SCM could disappear and thus,
>> our the internal pointer returned by ‘scm_locale_string_data’ wouldn’t
>> be enough to prevent the stringbuf from being GC’d.
>
> I would suggest doing the whole optimization and being sure to do
> scm_remember_upto_here_1 on the SCM value at use sites.

Here’s an updated patch that does that.

The fallback case in ‘scm_locale_string_data’ is unfortunately not as
nice as I wrote before, because it has to call ‘scm_to_stringn’, which
mallocs the string, which must then be copied to GC-managed storage,
argh.  (The patch I sent earlier didn’t take care of the encoding
conversion in the fallback case.)

The libunistring functions can take a pre-allocated buffer, but they
always malloc a fresh one if needed.  So the best we could do is have a
‘scm_to_stringn’ variant that takes a buffer, but it’s not guaranteed
that it’ll actually be used.  All in all, it seems the added complexity
is not warranted.  The worst case of ‘scm_locale_string_data’ is also
rare enough.

Thoughts?

Ludo’.

diff --git a/libguile/dynl.c b/libguile/dynl.c
index e9c03e95b..784b04b5e 100644
--- a/libguile/dynl.c
+++ b/libguile/dynl.c
@@ -1,6 +1,6 @@
 /* dynl.c - dynamic linking
 
-   Copyright 1990-2003,2008-2011,2017-2018
+   Copyright 1990-2003,2008-2011,2017-2018,2020
  Free Software Foundation, Inc.
 
This file is part of Guile.
@@ -233,7 +233,7 @@ SCM_DEFINE (scm_dynamic_link, "dynamic-link", 0, 1, 0,
 #define FUNC_NAME s_scm_dynamic_link
 {
   void *handle;
-  char *file;
+  const char *file;
 
   scm_dynwind_begin (0);
   scm_i_dynwind_pthread_mutex_lock (_lock);
@@ -241,10 +241,7 @@ SCM_DEFINE (scm_dynamic_link, "dynamic-link", 0, 1, 0,
   if (SCM_UNBNDP (filename))
 file = NULL;
   else
-{
-  file = scm_to_locale_string (filename);
-  scm_dynwind_free (file);
-}
+file = scm_locale_string_data (filename);
 
   handle = sysdep_dynl_link (file, FUNC_NAME);
   scm_dynwind_end ();
@@ -315,13 +312,13 @@ SCM_DEFINE (scm_dynamic_pointer, "dynamic-pointer", 2, 0, 0,
 SCM_MISC_ERROR ("Already unlinked: ~S", dobj);
   else
 {
-  char *chars;
+  const char *chars;
 
   scm_dynwind_begin (0);
   scm_i_dynwind_pthread_mutex_lock (_lock);
-  chars = scm_to_locale_string (name);
-  scm_dynwind_free (chars);
+  chars = scm_locale_string_data (name);
   val = sysdep_dynl_value (chars, DYNL_HANDLE (dobj), FUNC_NAME);
+  scm_remember_upto_here_1 (name);
   scm_dynwind_end ();
 
   return scm_from_pointer (val, NULL);
diff --git a/libguile/filesys.c b/libguile/filesys.c
index 4f7115397..5d2965b1b 100644
--- a/libguile/filesys.c
+++ b/libguile/filesys.c
@@ -1,4 +1,4 @@
-/* Copyright 1996-2002,2004,2006,2009-2019
+/* Copyright 1996-2002,2004,2006,2009-2020
  Free Software Foundation, Inc.
 
This file is part of Guile.
@@ -118,25 +118,24 @@
 
 /* Two helper macros for an often used pattern */
 
-#define STRING_SYSCALL(str,cstr,code)\
-  do {   \
-int eno; \
-char *cstr = scm_to_locale_string (str); \
-SCM_SYSCALL (code);  \
-eno = errno; free (cstr); errno = eno;   \
+#define STRING_SYSCALL(str,cstr,code)   \
+  do {  \
+int eno;\
+const char *cstr = scm_locale_string_data (str);\
+SCM_SYSCALL (code); \
+scm_remember_upto_here_1 (str); \
+eno = errno; errno = eno;   \
   } while (0)
 
-#define STRING2_SYSCALL(str1,cstr1,str2,cstr2,code)  \
-  do {   \
-int eno; \
-char *cstr1, *cstr2; \
-scm_dynwind_begin (0); \
-cstr1 = scm_to_locale_string (str1); \
-scm_dynwind_free (cstr1);  \
-cstr2 = scm_to_locale_string (str2); \
-scm_dynwind_free (cstr2);  \
-SCM_SYSCALL (code);  \
-eno = errno; scm_dynwind_end (); errno = eno;  \
+#define STRING2_SYSCALL(str1,cstr1,str2,cstr2,code) \
+  do {

Re: compile-file execution time driven by `reap-pipes'

2020-05-17 Thread Ludovic Courtès
Hi Matt,

Matt Wette  skribis:

> I'm wanted to trace down why some of my "compile-file" w/ no optimization
> was tasking time.  I was able to turn compile-file under statprof for one
> of the moderate sized .scm files, which has about 45,000 lines of code.
> It was interesting that the major execution time consumer was reap-pipes.
> I'm curious where that fits in.  Does anyone know how compile-file could
> be banging on reap-pipes so much?  Below is the top output from statprof.
> The results are for a run using guile 3.0.2 on Ubunu 18.04/x86_64.
>
>
> % cumulative   self calls
> time   secondsseconds   procedure
>  12.67190.96154.99  ice-9/popen.scm:145:0:reap-pipes
>  12.30363.41150.49  language/cps/intset.scm:270:2:adjoin
>  11.23270.27137.34  anon #x55befcb4c430
>  10.89186.95133.18  language/cps/slot-allocation.scm:843:19
>  10.58129.40129.40  language/cps/intmap.scm:396:0:intmap-ref
>   5.31   1279.35 64.90  language/cps/intmap.scm:247:2:adjoin
>   3.81228.54 46.66  language/cps/intset.scm:269:0:intset-add
>   3.48   4695.90 42.60  language/cps/intset.scm:470:5:visit-branch
>   3.28 40.14 40.14  language/cps/intset.scm:349:0:intset-ref
>   2.98567.14 36.46  language/cps/intmap.scm:246:0:intmap-add

I don’t remember noticing it before but I see the same now.  The
compiler is GC-intensive, so after-GC hooks such as ‘reap-pipes’ are
called quite frequently.  On this 19K line file, it doesn’t come first
though:

--8<---cut here---start->8---
scheme@(guile-user)> ,pr (compile-file "gnu/packages/python-xyz.scm")
GC Warning: Repeated allocation of very large block (appr. size 13246464):
May lead to memory leak and poor performance
% cumulative   self 
time   seconds seconds  procedure
 16.51 54.14 38.72  language/cps/slot-allocation.scm:843:19
 14.84 34.79 34.79  language/cps/intmap.scm:396:0:intmap-ref
  6.95   1142.96 16.29  language/cps/intset.scm:470:5:visit-branch
  6.36 19.34 14.90  anon #x7138d0
  5.11 50.85 11.97  language/cps/intset.scm:729:2:subtract-nodes
  4.23 20.79  9.91  language/cps/intset.scm:270:2:adjoin
  4.21  9.88  9.88  language/cps/intset.scm:349:0:intset-ref
  3.45 28.07  8.08  language/cps/intset.scm:551:2:union
  3.42  8.14  8.01  ice-9/popen.scm:145:0:reap-pipes
[…]
---
Sample count: 7285
Total time: 234.475384815 seconds (128.869116983 seconds in GC)
--8<---cut here---end--->8---

Delaying the ‘add-hook!’ call as shown below removes ‘reap-pipes’ from
the profile but otherwise doesn’t have any significant effect.

So I’m not sure if there’s much to conclude from that, other than
the fact that it’s GC’ing a lot.

Ludo’.

diff --git a/module/ice-9/popen.scm b/module/ice-9/popen.scm
index 5ab93f275..55b6f09fc 100644
--- a/module/ice-9/popen.scm
+++ b/module/ice-9/popen.scm
@@ -90,6 +90,10 @@
(cons (port->fdes (car p))
  (port->fdes (cdr p)
 
+(define (initialize-popen!)
+  (add-hook! after-gc-hook reap-pipes)
+  (set! initialize-popen! (const #t)))
+
 (define (open-process mode command . args)
   "Backwards compatible implementation of the former procedure in
 libguile/posix.c (scm_open_process) replaced by
@@ -99,6 +103,7 @@ process (based on pipes) is created and returned.  @var{mode} specifies
 whether an input, an output or an input-output port to the process is
 created: it should be the value of @code{OPEN_READ}, @code{OPEN_WRITE}
 or @code{OPEN_BOTH}."
+  (initialize-popen!)
   (let* ((from (and (or (string=? mode OPEN_READ)
 (string=? mode OPEN_BOTH)) (pipe->fdes)))
  (to (and (or (string=? mode OPEN_WRITE)
@@ -185,8 +190,6 @@ information on how to interpret this value."
   (lambda args #f
 (loop)
 
-(add-hook! after-gc-hook reap-pipes)
-
 (define (open-input-pipe command)
   "Equivalent to @code{open-pipe} with mode @code{OPEN_READ}"
   (open-pipe command OPEN_READ))
@@ -204,6 +207,7 @@ information on how to interpret this value."
 program and its arguments as strings -- returning an input port to the
 end of the pipeline, an output port to the beginning of the pipeline and
 a list of PIDs of the processes executing the @var(commands)."
+  (initialize-popen!)
   (let* ((to (pipe->fdes))
  (pipes (map (lambda _ (pipe->fdes)) commands))
 	 (pipeline (fold (lambda (from proc prev)


Re: [PATCH]: Fix a documentation for SRFI-171

2020-05-17 Thread Ludovic Courtès
Hi Linus,

Linus Björnstam  skribis:

> From 5124d2a429a408f5b6fcd3d6780b047aacaaae48 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= 
> Date: Fri, 1 May 2020 15:11:25 +0200
> Subject: [PATCH] Fix documentation for srfi-171  * doc/ref/srfi-modules.texi
>  (SRFI-171): Fix broken documentation.
>
> ---
>  doc/ref/srfi-modules.texi | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

Applied, thanks!

Ludo’.




Re: guile pipeline do-over

2020-05-16 Thread Ludovic Courtès
Hi Rutger,

It’s been a looong process (apologies!), but I’m happy to say that this
patch is now in master!  Hopefully your future contributions will be
quicker to get in, otherwise there won’t be anyone to blame but the
maintainers.  ;-)

Rutger van Beusekom  skribis:

> From 9fa48fa3917eb1fab61b703de936471c3c24f4f4 Mon Sep 17 00:00:00 2001
> From: Rutger van Beusekom 
> Date: Mon, 2 Mar 2020 10:38:57 +0100
> Subject: [PATCH] Add pipeline procedure.
>
> * libguile/posix.c (scm_open_process): Remove.
> (scm_piped_process): Add to replace open_process.
> * module/ice-9/popen.scm (pipe->fdes): Add to convert pipe pair to fdes pair.
> (open-process): Add open-process for backwards compatibility.
> (pipeline): Add to implement a pipeline using piped-process.

I added bits that were missing from the commit log and followed up with
cosmetic tweaks to the tests and doc.

It’s all good, thank you for sharing this!

Ludo’.



Re: Guile's time execution issues

2020-05-04 Thread Ludovic Courtès
Hi,

Linus Björnstam  skribis:

> On Mon, 4 May 2020, at 11:36, Ludovic Courtès wrote:
>  
>> > One thing I found is that `match` is slow. The code looked nicer but had 
>> > to change it back to lets and conds as the performance
>> > increase was ~2 seconds.
>> 
>> Oh, in which case exactly?  And are you sure your hand-written code is
>> equivalent to the ‘match’ code (it’s common for hand-written code to be
>> more lax than ‘match’)?
>> 
>> One thing to pay attention to is the use of ‘list?’, which is O(N), and
>> is implied by ellipses in ‘match’.  If you want to use ‘match’ in a way
>> that avoids ‘list?’, write patterns such as (a . b) instead of (a b ...).
>> It doesn’t have the same meaning, but often the end result is the same,
>> for instance because you’ll later match on ‘b’ anyway.
>> 
>> (I wish we can one day have a proper list type disjoint from pairs…)
>
> The change is here: he is only matching against chars and predicates: 
> https://github.com/aconchillo/guile-json/commit/ad4b06d86e4822466983d00f55474c8f664b538d

It would be nice if you could pinpoint which one of these changes causes
a difference, because:

--8<---cut here---start->8---
scheme@(guile-user)> ,optimize (match (peek-char port) ((? eof-object?) x) ((? 
whitespace?) w) (_ e))
$84 = (let ((v (peek-char port)))
  (cond ((eof-object? v) x)
((whitespace? v) w)
(else e)))
--8<---cut here---end--->8---

What might make a difference is the code bloat when using ‘or’:

--8<---cut here---start->8---
scheme@(guile-user)> ,optimize (match (peek-char port) ((or #\a #\b #\c #\d) x))
$86 = (let ((v (peek-char port)))
  (cond ((equal? v #\a) x)
((equal? v #\b) x)
((equal? v #\c) x)
((equal? v #\d) x)
(else
 ((@@ (ice-9 match) error)
  'match
  "no matching pattern"
  v)
 #f)))
--8<---cut here---end--->8---

but even that sounds unlikely.

You’re compiling with -O2, right?

Thanks,
Ludo’.



Re: Guile's time execution issues

2020-05-04 Thread Ludovic Courtès
Hey!

Aleix Conchillo Flaqué  skribis:

> So weird I'm getting different numbers on 2.2.7. Not sure how I'm getting 
> those initial ~20s and you are getting consistent ~ 45s. It
> shouldn't have nothing to do with it, but could it be I'm running it on macOS?

Did you add this ‘->bool’ call to ensure the resulting alist is not kept
in memory?

>  Now, it would be good to profile ‘json->scm’ to see if there’s anything
>  that could be improved on the Guile side, or if it’s just a normal
>  profile for GC-intensive code.
>
> Good news is that I have been working on performance improvements and 
> json->scm is going down from my ~19 seconds to ~3
> seconds on the same sample file. Linus Björnstam was the one to bring up 
> performance issues so we've been back and forth trying to
> make it fast.

Nice!

> One thing I found is that `match` is slow. The code looked nicer but had to 
> change it back to lets and conds as the performance
> increase was ~2 seconds.

Oh, in which case exactly?  And are you sure your hand-written code is
equivalent to the ‘match’ code (it’s common for hand-written code to be
more lax than ‘match’)?

One thing to pay attention to is the use of ‘list?’, which is O(N), and
is implied by ellipses in ‘match’.  If you want to use ‘match’ in a way
that avoids ‘list?’, write patterns such as (a . b) instead of (a b ...).
It doesn’t have the same meaning, but often the end result is the same,
for instance because you’ll later match on ‘b’ anyway.

(I wish we can one day have a proper list type disjoint from pairs…)

Thanks,
Ludo’.



Re: Guile's time execution issues

2020-05-02 Thread Ludovic Courtès
Hola!

Aleix Conchillo Flaqué  skribis:

> On guile-json 3.5.0 (still using (string-append)) the first execution time
> goes from 19 seconds to 42 seconds. Then, the times keep increasing as in
> version 2.2.7 but numbers are much bigger:

With Guile 3.0.2 and Guile-JSON 3.5.0, I get:

--8<---cut here---start->8---
$ guile
GNU Guile 3.0.2
Copyright (C) 1995-2020 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> ,use(json)
scheme@(guile-user)> ,t (->bool (call-with-input-file "/tmp/large-file.json" 
json->scm))
$1 = #t
;; 55.142128s real time, 79.806656s run time.  65.418070s spent in GC.
scheme@(guile-user)> ,t (->bool (call-with-input-file "/tmp/large-file.json" 
json->scm))
$2 = #t
;; 47.416645s real time, 75.274219s run time.  62.421108s spent in GC.
scheme@(guile-user)> ,t (->bool (call-with-input-file "/tmp/large-file.json" 
json->scm))
$3 = #t
;; 41.292368s real time, 79.053120s run time.  67.266710s spent in GC.
--8<---cut here---end--->8---

So I think the time increase was just due to the fact that previous
parse results were kept around, somehow.

2.2.7 performs comparably for me:

--8<---cut here---start->8---
$ guix environment -C --ad-hoc guile-json guile@2.2 --share=/tmp -- guile 
guix environment: warning: plursenca pak-specifigo 'guile@2.2'
guix environment: warning: choosing guile@2.2.7 from 
gnu/packages/guile.scm:256:2
GNU Guile 2.2.7
Copyright (C) 1995-2019 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> ,use(json)
scheme@(guile-user)> ,t (->bool (call-with-input-file "/tmp/large-file.json" 
json->scm))
$1 = #t
;; 44.963180s real time, 90.606198s run time.  71.529811s spent in GC.
scheme@(guile-user)> ,t (->bool (call-with-input-file "/tmp/large-file.json" 
json->scm))
$2 = #t
;; 44.147740s real time, 87.796937s run time.  69.818018s spent in GC.
scheme@(guile-user)> ,t (->bool (call-with-input-file "/tmp/large-file.json" 
json->scm))
$3 = #t
;; 45.057761s real time, 89.689930s run time.  71.370764s spent in GC.
--8<---cut here---end--->8---

So to me, Guile is behaving correctly here.

Now, it would be good to profile ‘json->scm’ to see if there’s anything
that could be improved on the Guile side, or if it’s just a normal
profile for GC-intensive code.

Thanks,
Ludo’.



[PATCH] Avoid malloc/free/dynwind for POSIX subrs that take strings

2020-04-29 Thread Ludovic Courtès
Hello Guilers!

As discussed on IRC, the patch below arranges so that subrs that take
strings and pass them to syscall wrappers can avoid the
malloc/free/dynwind overhead.  This gives a 10% speedup on a tight loop
that calls these subrs:

--8<---cut here---start->8---
$ cat /tmp/tt.scm
(let loop ((i 200))
  (unless (zero? i)
(lstat "/var/run/")
(readlink "/bin/sh")
(loop (1- i
$ time ./meta/guile /tmp/tt.scm

real0m7.186s
user0m5.446s
sys 0m2.499s
$ time guile /tmp/tt.scm

real0m7.857s
user0m6.123s
sys 0m2.460s
--8<---cut here---end--->8---

(It’s crazy that two thirds of the wall-clock time is user!)

On IRC, Andy mentioned concerns that the SCM could disappear and thus,
our the internal pointer returned by ‘scm_locale_string_data’ wouldn’t
be enough to prevent the stringbuf from being GC’d.

If we comment out the pure-ASCII optimization in
‘scm_locale_string_data’, we still get a 5% speedup or so.

Anyhow, we get the other benefit, which is that it simplifies code:

 dynl.c|   14 +++---
 filesys.c |   84 +-
 foreign.c |   10 ++-
 fports.c  |   21 +--
 gettext.c |   73 ++---
 load.c|   24 +
 net_db.c  |   38 +++-
 strings.c |   46 +
 strings.h |3 +-
 9 files changed, 135 insertions(+), 178 deletions(-)

Thoughts?

Ludo’.

diff --git a/libguile/dynl.c b/libguile/dynl.c
index e9c03e95b..6b7575b4a 100644
--- a/libguile/dynl.c
+++ b/libguile/dynl.c
@@ -1,6 +1,6 @@
 /* dynl.c - dynamic linking
 
-   Copyright 1990-2003,2008-2011,2017-2018
+   Copyright 1990-2003,2008-2011,2017-2018,2020
  Free Software Foundation, Inc.
 
This file is part of Guile.
@@ -233,7 +233,7 @@ SCM_DEFINE (scm_dynamic_link, "dynamic-link", 0, 1, 0,
 #define FUNC_NAME s_scm_dynamic_link
 {
   void *handle;
-  char *file;
+  const char *file;
 
   scm_dynwind_begin (0);
   scm_i_dynwind_pthread_mutex_lock (_lock);
@@ -241,10 +241,7 @@ SCM_DEFINE (scm_dynamic_link, "dynamic-link", 0, 1, 0,
   if (SCM_UNBNDP (filename))
 file = NULL;
   else
-{
-  file = scm_to_locale_string (filename);
-  scm_dynwind_free (file);
-}
+file = scm_locale_string_data (filename);
 
   handle = sysdep_dynl_link (file, FUNC_NAME);
   scm_dynwind_end ();
@@ -315,12 +312,11 @@ SCM_DEFINE (scm_dynamic_pointer, "dynamic-pointer", 2, 0, 0,
 SCM_MISC_ERROR ("Already unlinked: ~S", dobj);
   else
 {
-  char *chars;
+  const char *chars;
 
   scm_dynwind_begin (0);
   scm_i_dynwind_pthread_mutex_lock (_lock);
-  chars = scm_to_locale_string (name);
-  scm_dynwind_free (chars);
+  chars = scm_locale_string_data (name);
   val = sysdep_dynl_value (chars, DYNL_HANDLE (dobj), FUNC_NAME);
   scm_dynwind_end ();
 
diff --git a/libguile/filesys.c b/libguile/filesys.c
index 4f7115397..8406bf831 100644
--- a/libguile/filesys.c
+++ b/libguile/filesys.c
@@ -1,4 +1,4 @@
-/* Copyright 1996-2002,2004,2006,2009-2019
+/* Copyright 1996-2002,2004,2006,2009-2020
  Free Software Foundation, Inc.
 
This file is part of Guile.
@@ -118,25 +118,22 @@
 
 /* Two helper macros for an often used pattern */
 
-#define STRING_SYSCALL(str,cstr,code)\
-  do {   \
-int eno; \
-char *cstr = scm_to_locale_string (str); \
-SCM_SYSCALL (code);  \
-eno = errno; free (cstr); errno = eno;   \
+#define STRING_SYSCALL(str,cstr,code)   \
+  do {  \
+int eno;\
+const char *cstr = scm_locale_string_data (str);\
+SCM_SYSCALL (code); \
+eno = errno; errno = eno;   \
   } while (0)
 
-#define STRING2_SYSCALL(str1,cstr1,str2,cstr2,code)  \
-  do {   \
-int eno; \
-char *cstr1, *cstr2; \
-scm_dynwind_begin (0); \
-cstr1 = scm_to_locale_string (str1); \
-scm_dynwind_free (cstr1);  \
-cstr2 = scm_to_locale_string (str2); \
-scm_dynwind_free (cstr2);  \
-SCM_SYSCALL (code);  \
-eno = errno; scm_dynwind_end (); errno = eno;  \
+#define STRING2_SYSCALL(str1,cstr1,str2,cstr2,code) \
+  do {  \
+int eno;\
+const char *cstr1, *cstr2;  \
+cstr1 = scm_locale_string_data (str1);  \
+cstr2 = scm_locale_string_data 

Re: Guile's time execution issues

2020-04-26 Thread Ludovic Courtès
Bon dia!

Aleix Conchillo Flaqué  skribis:

> I was trying to get some guile-json performance times loading large JSON
> file. However, I'm getting increasing numbers at each run, so I'm wondering
> if I'm doing something wrong. Below you can see how the first run took
> 19.95s and then running the same command kept increasing.
>
> I'm running Guile 2.2.7 on macOS Catalina 10.15.3.
>
> scheme@(guile-user)> (use-modules (json))
> scheme@(guile-user)> ,t (define a (call-with-input-file
> "/Users/aleix/Downloads/large-file.json" (lambda (port) (json->scm port
> ;; 19.956429s real time, 87.100982s run time.  75.270202s spent in GC.
> ;; 26.173179s real time, 143.645265s run time.  131.022631s spent in GC.
> ;; 28.193926s real time, 154.758375s run time.  141.697236s spent in GC.
> ;; 29.044218s real time, 160.745984s run time.  147.449073s spent in GC.
> ;; 30.480873s real time, 170.855527s run time.  157.332793s spent in GC.
> ;; 30.555700s real time, 172.938278s run time.  159.468737s spent in GC.
> ;; 32.190478s real time, 172.807551s run time.  158.905645s spent in GC.

Could this have to do with ?

Could you check if that happens with 3.0.2?  (Or suggest a
‘large-file.json’ to use.  :-))

Thanks in advance!

Ludo’.



GNU Guile 3.0.2 released

2020-03-27 Thread Ludovic Courtès
have the required public key,
then run this command to import it:

  gpg --keyserver pool.sks-keyservers.net \
  --recv-keys 3CE464558A84FDC69DB40CFB090B11993D9AEBB5

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.69
  Automake 1.16.2
  Libtool 2.4.6
  Makeinfo 6.7
  Gnulib v0.1-1157-gb03f418


Happy hacking with Guile!

Ludovic Courtès and Andy Wingo.


signature.asc
Description: PGP signature


Re: Replace ltdl with GLib's GModule

2020-03-27 Thread Ludovic Courtès
Hello!

Mike Gran  skribis:

> In a fit of pique, I pushed a new branch of Guile to the repo
> called wip-replace-ltdl-with-gmodule.  It replaces the dynamic linking
> library libltdl from libtool with the analagous library GModule
> from GLib.  It was remarkably easy, and, after a cursory test
> I was surprised to find that it seems to be working.
>
> GModule itself is a standalone library, but, I think it depends
> on GLib.  It might be possible to remove that dependency without
> much difficulty, but unfortunately, then you would be left with
> another library to maintain.

Fun!

I’m very much tired of “file not found” :-), but I don’t think Guile
should depend on GLib, which is quite big.

I think we should either fix ltdl (probably less work than porting Guile
to GModule, but also less exciting) or have a very basic wrapper around
dlopen instead of using ltdl (I think Andy had made experiments in that
direction.)

Thanks,
Ludo’.




Re: guile pipeline do-over

2020-03-26 Thread Ludovic Courtès
Hi Rutger,

(+Cc: Andy.)

Rutger van Beusekom  skribis:

> From d351c0a5ecde62e63368bec0e1f15108495a1a71 Mon Sep 17 00:00:00 2001
> From: Rutger van Beusekom 
> Date: Mon, 2 Mar 2020 10:38:57 +0100
> Subject: [PATCH] Add pipeline procedure.
>
> * libguile/posix.c (scm_open_process): Remove.
> (scm_piped_process): Add to replace open_process.
> * module/ice-9/popen.scm (pipe->fdes): Add to convert pipe pair to fdes pair.
> (open-process): Add open-process for backwards compatibility.
> (pipeline): Add to implement a pipeline using piped-process.

There are a couple super minor issues that I comment on below, but
otherwise LGTM!  If Andy agrees, we can apply it once the copyright
assignment is on file, so maybe it won’t be in 3.0.2, we’ll see!

> +@deffn (Scheme Procedure) pipeline commands
  ^^
Should be braces.

> +Execute a @code{pipeline} of @var{commands} -- where each command is a
> +list of a program and its arguments as strings -- returning an input

s/--/---/ so we get an em dash and not an en dash (I’m a typography
nitpicker :-)).

> +port to the end of the pipeline, an output port to the beginning of the
> +pipeline and a list of PIDs of the processes executing the @var{commands}.
> +
> +@example
> +(let ((commands '(("git" "ls-files")
> +   ("tar" "-cf-" "-T-")
> +   ("sha1sum" "-")))
 ^
There’s an extra space on these lines

> +   (pipe-fail? (compose not
> +zero?
> +status:exit-val
> +cdr
> +waitpid)))

I don’t think we should encourage this style, which could also look
obscure to newcomers.  I’d just make it a regular lambda.

That’s all for me.

Thanks again, Rutger!

Ludo’.



Re: [PATCH] Add srfi-171 to guile

2020-03-25 Thread Ludovic Courtès
Hi Linus,

Linus Björnstam  skribis:

> From 1450661c2432522ee41b43dffd05e46384a3ff1b Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= 
> Date: Mon, 23 Mar 2020 14:59:39 +0100
> Subject: [PATCH] Add SRFI-171 to guile
>
> This adds SRFI-171 (transducers) to guile. 
>
> The two guile-specific additions are powerful transducers which can be
> used to generalize transducers like tsegment. They are hard to get
> right, but powerful and useful enough to warrant inclusion.
>
>  * doc/ref/srfi-modules.texi: added srfi-171 section
>  * module/Makefile.am (SOURCES):
>  * module/srfi/srfi-171.scm:
>  * module/srfi/srfi-171/meta.scm: Add SRFI-171
>  * module/srfi/srfi-171/gnu.scm: Add 2 guile-specific extensions.
>  * test-suite/Makefile.am (SCM_TESTS):
>  * test-suite/tests/srfi-171.test: Add tests.

Perfect, applied!  It’s great to have this part of Guile proper.

Thank you!

Ludo’.



Re: Should guile-3.0 cond-expand guile-2 and guile-2.2?

2020-03-11 Thread Ludovic Courtès
Hi Rob,

(+ Cc: Andy.)

Rob Browning  skribis:

> Ludovic Courtès  writes:
>
>> Rob Browning  skribis:
>>
>>>   $ guile-3.0 -c '(display (cond-expand (guile-2.2 "?\n")))
>>>   ?
>>>
>>> Is that intentional?
>>
>> I think so, though I don’t think this was discussed here.
>>
>> The way I see it, it means that guile-3 is a superset of 2.2.
>
> OK, though that wasn't true for guile-2.2 with respect to 2.0?

Oh, but there was not ‘guile-2.0’ symbol, right?

> In any case, it'd be nice to have the policy documented, perhaps on
> the srfi-0 info page.

Agreed.

> At the moment, I just needed a way to write code that behaved
> differently with 3.0+ as compared to 2.2, because 2.2 doesn't support
> define-module #:re-export-and-replace, and there's no functional
> equivalent yet.
>
> For now I did this (I don't currently care about older than 2.2):
>
>   (define (re-export-and-replace! . names)
> (cond-expand
>   (guile-3.0
>(module-re-export! (current-module) names #:replace? #t))
>   (guile-2.2
>(module-re-export! (current-module) names))
>   (else
>(module-re-export! (current-module) names #:replace? #t
>
> And migrated all the relevant symbols out of the define-module form.
>
> Do we think that the norm will be for releases to cond-expand the
> symbols for all their ancestors (up to some point)?  i.e. guile 4 will
> likely cond expand guile-3, guile-3.0, guile-3.1, ... and guile-2,
> guile-2.2, and so on?

My interpretation is that ‘guile-2.2’ is to be interpreted as “2.2 or
any later backwards-compatible version [at a language level]”.

Thus, what ‘guile-4’ will mean will depend on the compatibility story of
4.0 wrt. to 3.x.

Ideally I guess we’d want to express things in terms of minor/major
version (in)equalities rather than plain symbol matches.  One can
actually do that with a ‘syntax-case’ macro looking at ‘minor-version’
etc., but that’s inconvenient.

Ludo’.



GNU Guile 3.0.1 released

2020-03-08 Thread Ludovic Courtès
u/guile/guile-3.0.1.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA256 checksums:

  c83659e515c97270c0647b34470092a109a3609124c9e86dbef53db7f7d1d7ec  
guile-3.0.1.tar.gz
  756279560e1d634a1ff1cb368379f2aa14206b65841a0323514b71d1269aaa94  
guile-3.0.1.tar.lz
  d696ead0fd138cc7ef883b50cc6b4be2898d18ff64bd1e9379081e1186be53c9  
guile-3.0.1.tar.xz

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify guile-3.0.1.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver pool.sks-keyservers.net \
  --recv-keys 3CE464558A84FDC69DB40CFB090B11993D9AEBB5

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.69
  Automake 1.16.1
  Libtool 2.4.6
  Makeinfo 6.7
  Gnulib v0.1-1157-gb03f418


Happy hacking with Guile!

Ludovic Courtès and Andy Wingo.


signature.asc
Description: PGP signature


Re: [PATCH] Add srfi-171 to guile

2020-03-08 Thread Ludovic Courtès
Hi Linus,

Linus Björnstam  skribis:

> From c382d7808a8d41cd4e9ef8a17b7ba9553835499b Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= 
> Date: Thu, 16 Jan 2020 20:31:45 +0100
> Subject: [PATCH] Add SRFI-171 (transducers) to guile.
>
> The two guile-specific additions are powerful transducers which can be
> used to generalize transducers like tsegment. They are hard to get
> right, but powerful and useful enough to warrant inclusion.
>
>  * doc/ref/srfi-modules.texi: added srfi-171 section
>  * module/Makefile.am (SOURCES):
>  * module/srfi/srfi-171.scm:
>  * module/srfi/srfi-171/meta.scm: Add SRFI-171
>  * module/srfi/srfi-171/gnu.scm: Add 2 guile-specific extensions.
>  * test-suite/Makefile.am (SCM_TESTS):
>  * test-suite/tests/srfi-171.test: Add tests.

I think the patch is almost ready for inclusion, thanks for taking the
time to address Andy’s comments!

I have additional stylistic comments, and then I think we’re ready to go:

> +Transducers are oblivious to what kind of process they are used in, and
> +are composable without building intermediate collections. This means we
> +can create a transducer that squares all even numbers: @code{(compose
> +(tfilter odd?) (tmap (lambda (x) (* x x} and reuse it with lists,

For readability, it’s probably best to use @example rather than @code
for the example above (for an example larger than a couple of words in
general.)

> +The central part of transducers are 3-arity reducing functions.

In general, RnRS and Guile use the term “procedure” rather than
“functions”.  Not a big deal, but bonus points if you can adjust the
documentation accordingly.  :-)

> +@itemize
> +@item
> +(): Produce an identity

s/an/the/ ?

> +@subheading The concept of transducers
> +
> +A transducer is a one-arity function that takes a reducer and produces a
> +reducing function that behaves as follows:
> +
> +@itemize
> +@item
> +(): calls reducer with no arguments (producing its identity)

It’s not clear from this where you’d write () (there’s a missing @code
here, right?), which is not a valid Scheme expression in itself.
Perhaps an extra bit of introduction is needed above for clarity?

Also, this is under the “Concept” heading, but it looks more like the
API, no?

> +@item
> +(result-so-far): Maybe transform the result-so-far and call reducer with it.
> +
> +@item
> +(result-so-far input) Maybe do something to input and maybe call the
> +reducer with result-so-far and the maybe-transformed input.

Missing @code ornaments here.

> +a simple example is as following: @code{ (list-transduce (tfilter odd?)
  ^
Capital.

@example for the example.

> ++ '(1 2 3 4 5))}. This first returns a transducer filtering all odd
> +elements, then it runs @code{+} without arguments to retrieve its
> +identity. It then starts the transduction by passing @code{+} to the

Please make sure to add two spaces after end-of-sentence periods
throughout the document.

> + Even though transducers appear to be somewhat of a generalisation of
> + map and friends, this is not really true. Since transducers don't know

s/map/@code{map}/

> +@deffn {Scheme Procedure} list-transduce xform f lst
> +@deffnx {Scheme Procedure} list-transduce xform f identity lst
> + Initializes the transducer @code{xform} by passing the reducer @code{f}
> + to it. If no identity is provided, @code{f} is run without arguments to
> + return the reducer identity. It then reduces over @code{lst} using the
> + identity as the seed.

Please remove the extra leading space.  Also, use @var, and use
imperative tense, like so:

  @deffn {Scheme Procedure} list-transduce @var{xform} @var{f} @var{lst}
  Initialize the transducer @var{xform} by passing the reduce @var{f} to
  it. …
  @end deffn

> +If one of the transducers finishes early (such as @code{ttake} or
> +@code{tdrop}), it communicates this by returning a reduced value, which
> +in the sample implementation is just a value wrapped in a SRFI 9 record
> +type named "reduced". If such a value is returned by the transducer,

For proper typesetting, write ``reduced'' instead of "reduced".

s/SRFI /SRFI-/

> +Same as @code{list-transduce}, but for vectors, strings, u8-bytevectors
> +and srfi-158-styled generators respectively.
> +
> +@end deffn

Please remove the newline before @end deffn.

> +@node SRFI-171 Helpers
> +@subsubsection Helper functions for writing transducers
> +@cindex transducers helpers
> +
> +These functions are in the @code{(srfi 171 meta)} module and are only
^
(srfi srfi-171 meta)

> +@deffn {Scheme Procedure} reduced value
> +
> +Wraps a value in a @code{} container, signalling that the
> +reduction should stop.
> +@end deffn

Please remove extra line before @deffn (throughout the document).

> +(define-module (srfi srfi-171)
> +  #:declarative? #t

Is it necessary?  If so, could you add a comment so our future selves
know whether this is still necessary?  :-)

> +(define reverse-rcons
> +  (case-lambda
> +

GNU Guile 2.2.7 released

2020-03-07 Thread Ludovic Courtès
We are delighted to announce GNU Guile release 2.2.7, the seventh
bug-fix release of the “legacy” 2.2 series (the current stable series is
3.0).  See the NEWS excerpt that follows for full details.

 *  *  *

Guile is an implementation of the Scheme programming language.

The Guile web page is located at https://gnu.org/software/guile/, and
among other things, it contains a copy of the Guile manual and pointers
to more resources.

Guile can run interactively, as a script interpreter, and as a Scheme
compiler to VM bytecode.  It is also packaged as a library so that
applications can easily incorporate a complete Scheme interpreter/VM.
An application can use Guile as an extension language, a clean and
powerful configuration language, or as multi-purpose "glue" to connect
primitives provided by the application.  It is easy to call Scheme code
from C code and vice versa.  Applications can add new functions, data
types, control structures, and even syntax to Guile, to create a
domain-specific language tailored to the task at hand.

Guile implements many common Scheme standards, including R5RS, R6RS, and
a number of SRFIs.  In addition, Guile includes its own module system,
full access to POSIX system calls, networking support, multiple threads,
dynamic linking, a foreign function call interface, and powerful string
processing.

Guile 2.2.7 can be installed in parallel with Guile 3.0.x; see
https://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html.

 *  *  *

Changes in 2.2.7 (since 2.2.6)

* New interfaces and functionality

** (texinfo plain-text) now exports '*line-width*' fluid

The new '*line-width*' fluid allows users to specify the width of a line
for the purposes of line wrapping.  See "texinfo plain-text" in the
manual.

* Bug fixes

** Reduce GC pressure when using bignums

Guile no longer installs a finalizer on each bignum (large integer) it
creates.  This significantly improves speed and memory usage on
applications that make heavy use of bignums, such as the compiler
itself.

** Fix peval bug that ignored excess arguments

In an expression like:

  ((lambda ()
 (define (add1 n)(+ 1 n))
 (add1 1 2)))

the compiler (specifically 'peval') would silently ignore the excess
argument to 'add1'.

** Respect thread local fluid defaults

Previously (fluid-ref (make-thread-local-fluid #t)) would return #f.
This is now fixed.

** Fix non-deterministic crash in 'finalization_thread_proc'
   (<https://bugs.gnu.org/37757>)

** texinfo properly renders @acronym in plain text
   (<https://bugs.gnu.org/37846>)

** 'scm_port_poll' honors "w" flags
   (<https://bugs.gnu.org/36709>)

** Do not record LDFLAGS in .pc file
   (<https://bugs.gnu.org/36339>)

** Fix Readline configure check for the sake of libedit

This fixes builds on macOS against the system-provided libedit.

** Fix build on platforms where the stack grows upwards

 *  *  *

Here are the compressed sources:
  https://ftp.gnu.org/gnu/guile/guile-2.2.7.tar.gz   (18MB)
  https://ftp.gnu.org/gnu/guile/guile-2.2.7.tar.lz   (9MB)
  https://ftp.gnu.org/gnu/guile/guile-2.2.7.tar.xz   (11MB)

Here are the GPG detached signatures[*]:
  https://ftp.gnu.org/gnu/guile/guile-2.2.7.tar.gz.sig
  https://ftp.gnu.org/gnu/guile/guile-2.2.7.tar.lz.sig
  https://ftp.gnu.org/gnu/guile/guile-2.2.7.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA256 checksums:

44b4c5fbbe257ccdebea18420212c9b3e90c3c86a54920d8554039fc6769a007  
guile-2.2.7.tar.gz
5de7c4d28fa25c232512a4d1e76e6152a721dde568a6b1310971e1ea49e18c46  
guile-2.2.7.tar.lz
cdf776ea5f29430b1258209630555beea6d2be5481f9da4d64986b077ff37504  
guile-2.2.7.tar.xz

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify guile-2.2.7.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver pool.sks-keyservers.net \
  --recv-keys 3CE464558A84FDC69DB40CFB090B11993D9AEBB5

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.69
  Automake 1.16.1
  Libtool 2.4.6
  Makeinfo 6.7
  Gnulib v0.1-1157-gb03f418


Happy hacking with Guile!

Ludovic Courtès and Andy Wingo.


signature.asc
Description: PGP signature


Re: README issues

2020-03-07 Thread Ludovic Courtès
Hello,

Jean-Christophe Helary 
skribis:

> Good evening everybody, thank you for helping on the IRC channel.
>
> Here is a README patch to minimally document fixes for the issues I had.
>
> Don't hesitate to edit to follow the guile documentation standards of any.
>
> If there is a need to have a FSF disclaimer, I'm OK with that. Let me know 
> how to do that (I already have one for emacs/texinfo/translation in general).

I pushed something similar to what you proposed (slightly less
detailed).  Thank you for the patch!

Ludo’.



Re: guile pipeline do-over

2020-03-07 Thread Ludovic Courtès
Hi Rutger!

Rutger van Beusekom  skribis:

> This patch replaces open-process with piped-process in posix.c and
> reimplement open-process with piped-process in popen.scm. This allows
> setting up a pipeline in guile scheme using the new pipeline procedure
> in popen.scm and enables its use on operating systems which happen to
> lack the capability to fork, but do offer the capability captured by
> start_child (see posix.c).

Nice!  That’s definitely very useful to have.  We’ll need to check what
Andy thinks, but I think it can be added in the 3.0 series.

> From 3c8f5d534419418234cfe7d3eda8227951bc208a Mon Sep 17 00:00:00 2001
> From: Rutger van Beusekom 
> Date: Mon, 2 Mar 2020 10:38:57 +0100
> Subject: [PATCH] Allow client code to create pipe pairs when opening a
>  process.
>
> * libguile/posix.c (scm_piped_process): Replace open_process by piped_process.

Could you mention functions renamed/removed here?  The ChangeLog format
is about boringly listing all the language-entity-level changes.  :-)

> * module/ice-9/popen.scm (pipe->fdes): Convert pipe pair to fdes pair.
> (open-process): Implement open-process with piped-process.
> (pipeline): Implement a pipeline with piped-process.
>  static SCM
> -scm_open_process (SCM mode, SCM prog, SCM args)
> -#define FUNC_NAME "open-process"
> +scm_piped_process (SCM prog, SCM args, SCM from, SCM to)
> +#define FUNC_NAME "piped-process"
> +/* SCM_DEFINE (scm_piped_process, "piped-process", 2, 2, 0, */
> +/*(SCM prog, SCM args, SCM from, SCM to), */
> +/* "Execute the command indicated by @var{prog} with arguments 
> @var(args),\n" */
> +/* "optionally connected by an input and an output pipe.\n" */
> +/* "@var(from) and @var(to) are either #f or a valid file descriptor\n" */
> +/* "of an input and an output pipe, respectively.\n" */
> +/* "\n" */
> +/* "This function returns the PID of the process executing @var(prog)." */
> +/* "\n" */
> +/* "Example:\n" */
> +/* "(let ((p (pipe)))\n" */
> +/* "  (piped-process \"echo\" '(\"foo\" \"bar\")\n" */
> +/* " (cons (port->fdes (car p))\n" */
> +/* "   (port->fdes (cdr p\n" */
> +/* "  (display (read-string (car p\n" */
> +/* "(let ((p (pipe)))\n" */
> +/* "  (read-string (piped-process \"echo\" '(\"foo\" \"bar\")\n" */
> +/* "  (port->fdes (car p)\n") */
> +/* #define FUNC_NAME scm_piped_process */

I guess you can remove the commented-out bits…

> -  int c2p[2]; /* Child to parent.  */
> -  int p2c[2]; /* Parent to child.  */
> +  int c2p[2] = {}; /* Child to parent.  */
> +  int p2c[2] = {}; /* Parent to child.  */

… and this hunk, to minimize change.

> +++ b/module/ice-9/popen.scm
> @@ -22,9 +22,10 @@
>#:use-module (rnrs bytevectors)
>#:use-module (ice-9 binary-ports)
>#:use-module (ice-9 threads)
> +  #:use-module (srfi srfi-1)
>#:use-module (srfi srfi-9)
>#:export (port/pid-table open-pipe* open-pipe close-pipe open-input-pipe
> -open-output-pipe open-input-output-pipe))
> +open-output-pipe open-input-output-pipe pipe->fdes piped-process 
> pipeline))

I would not export ‘pipe->fdes’.  I’m not sure about exporting
‘piped-process’: it’s a bit low-level and we might want to reserve
ourselves the possibility to change it, like this patch does actually.

WDYT?

> +(define (open-process mode command . args)
> +  (let* ((from (and (or (equal? mode OPEN_READ) (equal? mode OPEN_BOTH)) 
> (pipe->fdes)))
> + (to (and (or (equal? mode OPEN_WRITE) (equal? mode OPEN_BOTH)) 
> (pipe->fdes)))
> + (pid (piped-process command args from to)))
> +(values (and from (fdes->inport (car from))) (and to (fdes->outport (cdr 
> to))) pid)))

Please wrap lines to 80 chars.

I suggest using ‘string=?’ above instead of ‘equal?’.  Also, could you
add a docstring?

> +(define (pipeline procs)
> +  "Execute a pipeline of @code(procs) -- where a proc is a list of a
> +command and its arguments as strings -- returning an input port to the
> +end of the pipeline, an output port to the beginning of the pipeline and
> +a list of PIDs of the @code(procs)"

Perhaps s/procs/commands/ would be clearer?  Also, @var{commands}
instead of @code.

Could you also add an entry in doc/ref/*.texi, in the “Pipes” node,
perhaps with one of the examples you gave?

> +++ b/test-suite/tests/popen.test
> @@ -211,3 +211,37 @@ exec 2>~a; read REPLY"
>   (let ((st (close-pipe (open-output-pipe "exit 1"
> (and (status:exit-val st)
>  (= 1 (status:exit-val st)))
> +
> +
> +;;
> +;; pipeline related tests
> +;;
> +
> +(use-modules (ice-9 receive))
> +(use-modules (ice-9 rdelim))

Please move these to the top-level ‘define-module’ form.

One last thing: we’d need you to assign copyright to the FSF for this.
We can discuss it off-line if you want.

Thank you for this great and long overdue addition!

Ludo’.



Re: Should guile-3.0 cond-expand guile-2 and guile-2.2?

2020-03-07 Thread Ludovic Courtès
Hi,

Rob Browning  skribis:

>   $ guile-3.0 -c '(display (cond-expand (guile-2.2 "?\n")))
>   ?
>
> Is that intentional?

I think so, though I don’t think this was discussed here.

The way I see it, it means that guile-3 is a superset of 2.2.

Ludo’.



Re: [PATCH] Accept .sld as scheme extensions in r7rs

2020-03-07 Thread Ludovic Courtès
Hi,

Nguyễn Thái Ngọc Duy  skribis:

> This is similar to 0bb980f12 (New function: install-r6rs!, 2019-09-25)
> which accepts .sls extension for r6rs. In r7rs, most portable libraries
> use .sld.
>
> * module/ice-9/boot-9.scm (install-r7rs!): Update %load-extensions.

Applied.  Thank you, and apologies for the delay!

Ludo’.



Re: [PATCH, v2] Fix build on ia64

2020-03-05 Thread Ludovic Courtès
Hi,

John Paul Adrian Glaubitz  skribis:

> On 2/9/20 2:32 PM, Ludovic Courtès wrote:
>>> I have actually signed the copyright assignment for the FSF already, but
>>> only for gdb/binutils. I asked back then whether it would be possible to
>>> sign the copyright assignment for all GNU projects but that was rejected.
>> 
>> Yeah, that’s not possible.
>> 
>>> I'll sign the CA next week for Guile.
>> 
>> Awesome, thank you!
>
> I have signed the copyright assignment and sent it to the appropriate
> mail address at the FSF.

I see we just received the confirmation from the copyright clerk.  Thank you!

Ludo’.



Re: MIssing scm_memory_error

2020-02-09 Thread Ludovic Courtès
Hi Dale,

dsm...@roadrunner.com skribis:

> In Guile 3.0 (and probably earlier) scm_memory_error has gone away,
> with no mention of a replacement in NEWS.

It would seem that ‘scm_memory_error’ was deprecated in 2014 in commit
c2247b782a9234bb9aedee5204c30daf1d01a510 and removed in 2017 in commit
c248ea10beb2afa4c113dbc6dc707bed5dbfc92e.

However, ‘SCM_MEMORY_ERROR’ in ‘error.h’ still refers to it, and so does
the manual.

Commit 1a3e316c32562aec2665a0233d46d5635f9c1245 fixes that by removing
all traces of ‘scm_memory_error’.

Thank you,
Ludo’.



Re: [PATCH, v2] Fix build on ia64

2020-02-09 Thread Ludovic Courtès
Hi,

John Paul Adrian Glaubitz  skribis:

> I have actually signed the copyright assignment for the FSF already, but
> only for gdb/binutils. I asked back then whether it would be possible to
> sign the copyright assignment for all GNU projects but that was rejected.

Yeah, that’s not possible.

> I'll sign the CA next week for Guile.

Awesome, thank you!

Ludo’.



Re: [PATCH, v2] Fix build on ia64

2020-02-08 Thread Ludovic Courtès
John Paul Adrian Glaubitz  skribis:

>   * libguile/continuations.c (capture_auxiliary_stack): Fix
> logic in preprocessor code when checking for ia64 host;
> fix dereferencing of ctx variable.
>   * libguile/threads.h (struct scm_thread): Add missing member
> SCM_STACKITEM *auxiliary_stack_base.

Applied, thanks!  (I don’t have IA64 hardware to test it but it looks
good to me and I trust it works for you on actual hardware.)

BTW, these were simple changes and thus not requiring copyright
assignment, but I would encourage you to assign copyright to the FSF for
future changes if you plan to keep contributing (which would be great!).
We can discuss the implications and paperwork off-list if you want.

Thank you!

Ludo’.



Re: [PATCH, v2] Fix build on platforms where the stack grows upwards

2020-02-08 Thread Ludovic Courtès
Hi,

John Paul Adrian Glaubitz  skribis:

>  * libguile/continuations.c (scm_dynthrow): Fix missing mra
>parameter to grow_stack for SCM_STACK_GROWS_UP.

Applied, thanks!

Ludo'.



Re: CPU and GC cost of bignums

2020-02-08 Thread Ludovic Courtès
Hi,

Ludovic Courtès  skribis:

> To my surprise, on a pure bignum microbenchmark, this is
> counterproductive:

I found out that I was comparing my own Guile build, which was against
the ‘libgc-back-pointers’ package, with that ‘guile-next’ build against
‘libgc’; no wonder mine was slower…

--8<---cut here---start->8---
$ cat ~/src/guile-debugging/bignum-finalizers.scm
(use-modules (ice-9 time))

(time
 (let loop ((n (expt 2 18))
(i 1))
   (unless (zero? n)
 ;; (display ".")
 (loop (- n 1)
   (logior 0 (ash i 1))

(format #t "heap size: ~a MiB~%"
(round
 (/ (assoc-ref (gc-stats) 'heap-size) (expt 2. 20
--8<---cut here---end--->8---

There’s no noticeable difference on the microbenchmark when using the
same libgc both times, and there’s still ~2.5x difference when compiling
‘gnu/services/mail.scm’.

I went ahead and pushed the change as 00fbdfa7345765168e14438eed0b0b8c64c27ab9.

I did a full bootstrap build, which went fine and felt faster, though I
don’t have hard figures to compare to (110m user according to Bash’
‘time’ on my laptop).

Give it a spin and let me know!

Thanks,
Ludo’.



Re: garbage collection slowdown

2020-02-06 Thread Ludovic Courtès
Hi Mikael!

Mikael Djurfeldt  skribis:

> Could the frequency of GC be adapted similarly such that the balance
> between GC and allocation is shifted towards allocation in phases with a
> lot of heap growth?

I guess we’ve been mostly “end users” of libgc, assuming the right
design decisions were made and not touching it much.  Overall it’s
worked quite well, which is why I’m surprised by what Han-Wen reports,
and I don’t feel like diving in libgc.  :-)

Thanks,
Ludo’.



Re: CPU and GC cost of bignums

2020-02-06 Thread Ludovic Courtès
Hi!

Andy Wingo  skribis:

> Nice investigation!  Perhaps slot-allocation should track live variables
> using something that's not bigints, but who knows.

Yeah I wondered; it’s not clear whether bitvectors would be more
efficient, for instance, although we could make it perhaps locally
imperative.

> On Wed 05 Feb 2020 17:29, Ludovic Courtès  writes:
>
>>  /* The next three functions (custom_libgmp_*) are passed to
>> mp_set_memory_functions (in GMP) so that memory used by the digits
>> themselves is known to the garbage collector.  This is needed so
>> @@ -237,19 +227,20 @@ finalize_bignum (void *ptr, void *data)
>>  static void *
>>  custom_gmp_malloc (size_t alloc_size)
>>  {
>> -  return scm_malloc (alloc_size);
>> +  return scm_gc_malloc (alloc_size, "GMP");
>>  }
>>  
>>  static void *
>>  custom_gmp_realloc (void *old_ptr, size_t old_size, size_t new_size)
>>  {
>> -  return scm_realloc (old_ptr, new_size);
>> +  return scm_gc_realloc (old_ptr, old_size, new_size, "GMP");
>>  }
>>  
>>  static void
>>  custom_gmp_free (void *ptr, size_t size)
>>  {
>> -  free (ptr);
>> +  /* Do nothing: all memory allocated by GMP is under GC control and
>> + will be freed when needed.  */
>>  }
>
> I think this makes sense to me as a short-term fix.  The down-side is
> that limbs can alias Scheme objects.

Yes.

To my surprise, on a pure bignum microbenchmark, this is
counterproductive:

--8<---cut here---start->8---
$ guile ~/src/guile-debugging/bignum-finalizers.scm  # 3.0.0
clock utime stime cutime cstime gctime
 2.42  6.20  0.17   0.00   0.00   5.62
heap size: 2.0 MiB
$ /data/src/guile-3.0/meta/guile  ~/src/guile-debugging/bignum-finalizers.scm
clock utime stime cutime cstime gctime
 3.97 10.91  0.15   0.00   0.00  10.60
heap size: 3.0 MiB
$ cat ~/src/guile-debugging/bignum-finalizers.scm
(use-modules (ice-9 time))

(time
 (let loop ((n (expt 2 18))
(i 1))
   (unless (zero? n)
 ;; (display ".")
 (loop (- n 1)
   (logior 0 (ash i 1))

(format #t "heap size: ~a MiB~%"
(round
 (/ (assoc-ref (gc-stats) 'heap-size) (expt 2. 20
--8<---cut here---end--->8---

(Here we’re creating ~24 bignums, no more.)
I wonder if there’s another part of the story that I’m missing here.

Perf report for 3.0.0:

--8<---cut here---start->8---
  46.93%  guilelibgc.so.1.3.6 [.] GC_mark_from
  17.61%  guilelibgc.so.1.3.6 [.] GC_header_cache_miss
   9.96%  guilelibgc.so.1.3.6 [.] GC_add_to_black_list_normal
   5.20%  guilelibgmp.so.10.3.2   [.] __gmpn_lshift_coreisbr
   4.13%  guilelibgc.so.1.3.6 [.] GC_find_header
   2.28%  guilelibgc.so.1.3.6 [.] GC_finalize
   2.09%  guilelibgc.so.1.3.6 [.] GC_base
--8<---cut here---end--->8---

With the patch:

--8<---cut here---start->8---
  48.40%  guilelibgc.so.1.3.6 [.] GC_mark_from
  17.74%  guilelibgc.so.1.3.6 [.] GC_header_cache_miss
  11.90%  guilelibgc.so.1.3.6 [.] 
GC_add_to_black_list_normal
   4.45%  guilelibgc.so.1.3.6 [.] GC_find_header
   2.31%  guilelibgmp.so.10.3.2   [.] __gmpn_lshift_coreisbr
   2.30%  guilelibgc.so.1.3.6 [.] GC_base
   1.73%  guilelibgc.so.1.3.6 [.] GC_finalize
--8<---cut here---end--->8---

IOW, the relative part of computations drops from 5% to 2%.

Thoughts?

> In the long-term I think we should be representing bignums as
> pointerless objects whose first word is the tag and a word count,
> followed by inline "limbs" (in the sense of
> https://gmplib.org/manual/Nomenclature-and-Types.html#Nomenclature-and-Types).
> Generally we can use the low-level API to work on these
> (https://gmplib.org/manual/Low_002dlevel-Functions.html#Low_002dlevel-Functions),
> and if we need to use mpz_t, we can easily create an mpz_t that points
> to these values.

Yes, that sounds like the right approach longer-term.  Note that ‘mpz_t’
is exposed through “numbers.h”, which I guess means we cannot change
that in 3.0.x.

Thanks,
Ludo’.



Re: CPU and GC cost of bignums

2020-02-05 Thread Ludovic Courtès
Hey ho!

Ludovic Courtès  skribis:

> … but has the disadvantage that it doesn’t work: ‘numbers.test’ fails
> badly on bignums.

I think with the excitement I no longer knew what I was saying.  So,
here’s a revised patch that actually preserves memory management (as in:
‘mpz_t’ are eventually freed), while getting rid of finalizers.  Result:

--8<---cut here---start->8---
scheme@(guile-user)> ,use(system base optimize)
scheme@(guile-user)> ,use(statprof)
scheme@(guile-user)> (gcprof (lambda () (compile-file "gnu/services/mail.scm" 
#:opts (optimizations-for-level 1
% cumulative   self 
time   seconds seconds  procedure
 25.00  3.25  3.25  anon #x66e8e0
  9.38  2.44  1.22  language/cps/intset.scm:551:2:union
  6.25  0.81  0.81  ice-9/boot-9.scm:2201:0:%load-announce
  6.25  0.81  0.81  anon #x66e468
  3.13139.83  0.41  ice-9/threads.scm:388:4
  3.13 73.57  0.41  language/tree-il/peval.scm:710:2:loop
  3.13  4.47  0.41  system/vm/assembler.scm:1201:0:intern-constant
  3.13  1.63  0.41  ice-9/psyntax.scm:2964:6:match*
  3.13  1.22  0.41  language/cps/intset.scm:270:2:adjoin
  3.13  0.81  0.41  language/cps/intmap.scm:184:0:intmap-add!
  3.13  0.41  0.41  language/cps/slot-allocation.scm:843:19

[...]

Sample count: 32
Total time: 13.007326523 seconds (4.420918875 seconds in GC)
--8<---cut here---end--->8---

I’d like to go ahead with this patch if there are no objections!

Ludo’.

diff --git a/libguile/numbers.c b/libguile/numbers.c
index d1b463358..8606780a8 100644
--- a/libguile/numbers.c
+++ b/libguile/numbers.c
@@ -1,4 +1,4 @@
-/* Copyright 1995-2016,2018-2019
+/* Copyright 1995-2016,2018-2020
  Free Software Foundation, Inc.
 
Portions Copyright 1990-1993 by AT Bell Laboratories and Bellcore.
@@ -218,16 +218,6 @@ static mpz_t z_negative_one;
 
 
 
-/* Clear the `mpz_t' embedded in bignum PTR.  */
-static void
-finalize_bignum (void *ptr, void *data)
-{
-  SCM bignum;
-
-  bignum = SCM_PACK_POINTER (ptr);
-  mpz_clear (SCM_I_BIG_MPZ (bignum));
-}
-
 /* The next three functions (custom_libgmp_*) are passed to
mp_set_memory_functions (in GMP) so that memory used by the digits
themselves is known to the garbage collector.  This is needed so
@@ -237,19 +227,20 @@ finalize_bignum (void *ptr, void *data)
 static void *
 custom_gmp_malloc (size_t alloc_size)
 {
-  return scm_malloc (alloc_size);
+  return scm_gc_malloc (alloc_size, "GMP");
 }
 
 static void *
 custom_gmp_realloc (void *old_ptr, size_t old_size, size_t new_size)
 {
-  return scm_realloc (old_ptr, new_size);
+  return scm_gc_realloc (old_ptr, old_size, new_size, "GMP");
 }
 
 static void
 custom_gmp_free (void *ptr, size_t size)
 {
-  free (ptr);
+  /* Do nothing: all memory allocated by GMP is under GC control and
+ will be freed when needed.  */
 }
 
 
@@ -260,12 +251,10 @@ make_bignum (void)
   scm_t_bits *p;
 
   /* Allocate one word for the type tag and enough room for an `mpz_t'.  */
-  p = scm_gc_malloc_pointerless (sizeof (scm_t_bits) + sizeof (mpz_t),
- "bignum");
+  p = scm_gc_malloc (sizeof (scm_t_bits) + sizeof (mpz_t),
+ "bignum");
   p[0] = scm_tc16_big;
 
-  scm_i_set_finalizer (p, finalize_bignum, NULL);
-
   return SCM_PACK (p);
 }
 


Re: garbage collection slowdown

2020-02-05 Thread Ludovic Courtès
Hi Han-Wen,

Great to see you back here!

Han-Wen Nienhuys  skribis:

> On Tue, Jan 28, 2020 at 11:41 PM Han-Wen Nienhuys  wrote:
>> Unfortunately, it looks like the adoption of the BDW GC library caused
>> a ~6x slowdown, causing an overall end-to-end slowdown of 50%.
>>
>> I was wondering if you folks would have tips to further tune GC for
>> wall-time speed, and if there additional diagnostics to see if we're
>> doing something extraordinarily silly.
>
> For the record, I managed to solve this, by scaling up the heap more
> aggressively. See
> https://codereview.appspot.com/561390043/diff/557260051/lily/score-engraver.cc

Weird.  It would be interesting to see where the slowdown comes from.
Overall, my recollection of the 1.8 to 2.0 transition (where we
introduced libgc) is that GC was a bit faster, definitely not slower.

That said, does LilyPond happen to use lots of bignums and/or lots of
finalizers?  Finalizers, including those on bignums, end up being very
GC-intensive, as discussed in my recent message.  Perhaps that’s what’s
happening here, for instance if you create lots of SMOBs with a free
function.

Thanks,
Ludo’.



CPU and GC cost of bignums

2020-02-04 Thread Ludovic Courtès
Hello!

(If you’re in a hurry, there are good news at the bottom.)

I noticed that 3.0 (and also 2.2 actually) takes a long time to compile
Guix’ gnu/services/mail.scm, which is macro-heavy, producing lots of
top-level defines.

At -O2 (the default), we have:

--8<---cut here---start->8---
scheme@(guile-user)> ,pr (compile-file "gnu/services/mail.scm")
% cumulative   self 
time   seconds seconds  procedure
 13.79 19.16 14.46  language/cps/slot-allocation.scm:846:19
 11.05 11.58 11.58  language/cps/intmap.scm:396:0:intmap-ref
  7.56 12.63  7.92  anon #x10768e0
  6.61  7.70  6.92  ice-9/popen.scm:145:0:reap-pipes
  5.50182.23  5.76  language/cps/intset.scm:470:5:visit-branch
  4.65  4.87  4.87  system/vm/linker.scm:179:0:string-table-intern!
  4.07  5.04  4.26  ice-9/vlist.scm:534:0:vhash-assoc
  3.54  3.93  3.71  language/cps/intmap.scm:184:0:intmap-add!
  3.28  6.65  3.43  language/cps/intset.scm:270:2:adjoin
  2.70  2.82  2.82  language/cps/intset.scm:349:0:intset-ref
  1.80 34.84  1.88  language/cps/intmap.scm:247:2:adjoin
  1.80  5.93  1.88  language/cps/intset.scm:269:0:intset-add
  1.74 18.22  1.83  language/cps/intmap.scm:246:0:intmap-add
  1.22  3.27  1.27  language/cps/intset.scm:382:2:visit-node
  1.16  2.94  1.22  language/cps/intset.scm:551:2:union
  1.11  1.38  1.16  language/cps/intset.scm:204:0:intset-add!
  0.74   1281.59  0.78  language/cps/intset.scm:472:5:visit-branch

[...]

Sample count: 1892
Total time: 104.795540582 seconds (85.091574653 seconds in GC)
--8<---cut here---end--->8---

At -O1:

--8<---cut here---start->8---
scheme@(guile-user)> ,use(system base optimize)
scheme@(guile-user)> ,pr (compile-file "gnu/services/mail.scm" #:opts 
(optimizations-for-level 1))
% cumulative   self 
time   seconds seconds  procedure
 11.76129.78  7.60  language/cps/intset.scm:470:5:visit-branch
 10.77  6.96  6.96  language/cps/intmap.scm:396:0:intmap-ref
 10.43 11.69  6.74  language/cps/slot-allocation.scm:846:19
  8.99  7.39  5.81  ice-9/vlist.scm:534:0:vhash-assoc
  7.55  4.88  4.88  system/vm/linker.scm:179:0:string-table-intern!
  6.44  4.16  4.16  ice-9/popen.scm:145:0:reap-pipes
  4.22  2.80  2.72  language/cps/intmap.scm:184:0:intmap-add!
  1.89  1.86  1.22  language/cps/slot-allocation.scm:681:17
  1.89  1.43  1.22  ice-9/vlist.scm:539:0:vhash-assq
  1.78  1.51  1.15  language/cps/slot-allocation.scm:505:17
  1.22  1.36  0.79  language/cps/slot-allocation.scm:846:19
  1.22  1.08  0.79  language/cps/slot-allocation.scm:505:17

[...]

Sample count: 901
Total time: 64.602907835 seconds (55.87541493 seconds in GC)
--8<---cut here---end--->8---

language/cps/slot-allocation.scm:846:19 corresponds to:

(define (compute-live-slots* slots label live-vars)
  (intset-fold (lambda (var live)
 (match (get-slot slots var)
   (#f live)
   (slot (add-live-slot slot live;L846
   (intmap-ref live-vars label)
   0))

The GC times remain extremely high though, and it’s also coming from
‘compute-live-slots*’:

--8<---cut here---start->8---
scheme@(guile-user)> (gcprof (lambda () (compile-file "gnu/services/mail.scm" 
#:opts (optimizations-for-level 1
% cumulative   self 
time   seconds seconds  procedure
 58.14 34.56 34.56  language/cps/slot-allocation.scm:846:19
  8.01  4.76  4.76  language/cps/slot-allocation.scm:681:17
  8.01  4.76  4.76  language/cps/slot-allocation.scm:505:17
  6.98  4.15  4.15  language/cps/slot-allocation.scm:505:17
  6.46  3.84  3.84  language/cps/slot-allocation.scm:846:19
  1.29  0.77  0.77  anon #x23e88e0

[...]

Sample count: 387
Total time: 59.442422179 seconds (50.331193744 seconds in GC)
--8<---cut here---end--->8---

(I believe Guile commit 5675e46410c9a24b05ddf58cbe3b998a4c9cad7c and its
parent were made to optimize the -O1 case back in 2017¹.)

‘compute-live-slots*’ returns an integer and the allocation comes from
line 846, where we allocate a bignum, in this case a verybignum even.
And for each bignum, we register a finalizer, which itself takes space.

(Time passes…)

The patch below (also for 2.2) gives us better timing:

--8<---cut here---start->8---
scheme@(guile-user)> (gcprof (lambda () (compile-file "gnu/services/mail.scm" 
#:opts (optimizations-for-level 1
% cumulative   self 
time   seconds seconds  procedure
 18.75  2.49  2.49  anon #x6f58e0


Re: Segfault while building on 64-bit Cygwin

2020-01-25 Thread Ludovic Courtès
John Cowan  skribis:

> Both Cygwin and MacOS crash in pretty much the same way.  By disabling the
> JIT, I was able to get the Cygwin build to run to completion.

That I understand.  However, I was asking for the backtrace of the crash
on Cygwin when JIT is enabled.  Could you grab it?

Thanks in advance,
Ludo’.



Re: Segfault while building on 64-bit Cygwin

2020-01-23 Thread Ludovic Courtès
Hi,

John Cowan  skribis:

> Thanks.  Unfortunately, the standard recipe for making core dumps on Mac

This bug report is about Cygwin, not macOS, right?  :-)

Ludo’.



Re: Segfault while building on 64-bit Cygwin

2020-01-21 Thread Ludovic Courtès
Hello,

John Cowan  skribis:

> Yes, gladly, but I don't know how to get one in this context.

You would unpack, configure, and build like you did before (with JIT
enabled, so as to reproduce the crash), but before that you’d run
“ulimit -c unlimited” in that shell to make sure there’s a core dumped
when it crashes.

Once it has crashed, locate the ‘core’ file (or ‘core.*’), and run, say:

  gdb libguile/.libs/guile bootstrap/core

Then from the GDB prompt:

  thread apply all bt

TIA,
Ludo’.



Segfault while building on 64-bit Cygwin

2020-01-20 Thread Ludovic Courtès
Hi John,

John Cowan  skribis:

> Guile 2.9.9, like .8 and .7, does not build on Cygwin (64 bit).  Configure
> runs without error, but make crashes with this (truncated to just the tail):
>
> Making all in bootstrap
> make[2]: Entering directory
> '/cygdrive/c/Users/rr828893/Downloads/guile-2.9.9/bootstrap'
>   BOOTSTRAP GUILEC ice-9/eval.go
>   BOOTSTRAP GUILEC ice-9/psyntax-pp.go
>   BOOTSTRAP GUILEC language/cps/intmap.go
>   BOOTSTRAP GUILEC language/cps/intset.go
>   BOOTSTRAP GUILEC language/cps/graphs.go
>   BOOTSTRAP GUILEC ice-9/vlist.go
>   BOOTSTRAP GUILEC srfi/srfi-1.go
> /bin/sh: line 6:  4294 Segmentation fault  (core dumped)
> GUILE_AUTO_COMPILE=0 ../meta/build-env guild compile
> --target="x86_64-unknown-cygwin" -O1 -Oresolve-primitives -L
> "/home/rr828893/Downloads/guile-2.9.9/module" -L
> "/home/rr828893/Downloads/guile-2.9.9/guile-readline" -o "srfi/srfi-1.go"
> "../module/srfi/srfi-1.scm"
> make[2]: *** [Makefile:1930: srfi/srfi-1.go] Error 139
> make[2]: Leaving directory
> '/cygdrive/c/Users/rr828893/Downloads/guile-2.9.9/bootstrap'
> make[1]: *** [Makefile:1849: all-recursive] Error 1
> make[1]: Leaving directory
> '/cygdrive/c/Users/rr828893/Downloads/guile-2.9.9'
> make: *** [Makefile:1735: all] Error 2

Could you try building 3.0.0 with JIT enabled and grab a backtrace?

Thanks in advance!

Ludo’.



Re: Logo baseline

2020-01-20 Thread Ludovic Courtès
Hello,

Arne Babenhauserheide  skribis:

> Ludovic Courtès  writes:
>> Like I wrote, Guile remains an extension language, no argument here.
>>
>> However, describing it as “just” an extension language seems odd to me.
>> It doesn’t take into account what many have been doing with Guile, and
>> it doesn’t match the efforts that have gone into Guile since 2.0.
>
> I don’t think that it is described as "just" an extension language. The
> website makes it very clear that Guile is also a viable application
> language.
>
> Where do you get the impression that Guile is described as "just" an
> extension language?

This thread is about the logo baseline, which is: “GNU extension
language”.  I agree that the web site is clearer; I’m just talking about
the logo here.

>>> Actually I’d love to see Guile become better at this: to make it easier
>>> to deploy an application that uses Guile in a statically compiled
>>> binary.
>>>
>>> Basically to generalize what LilyPond is doing.
>>
>> In what ways could it become “better” for this use case?
>
> Currently the default way to embed assumes that Guile is provided as a
> shared library. But that can be problematic when you want to ship a
> binary people can test easily.

I think this is no different for Guile than for any other piece of
software: some would ship a Docker image, and I’d argue that you can use
‘guix pack’ to generate a minimal standalone tarball or Docker image.

(Though I think it’s also perfectly doable to provide a binary that’s
statically-linked against libguile.)

Thanks,
Ludo’.



Re: Logo baseline

2020-01-19 Thread Ludovic Courtès
Hello!

Arne Babenhauserheide  skribis:

> Thomas Morley  writes:
>
>>> Clearly, Guile is still an extension language, with many great
>>> applications (Gnucash, Lepton-EDA, OpenCog, GDB, etc.)
>>
>> Well, you forgot LilyPond
>
> The one tool that uses Guile while dominating its domain.

Yup, I don’t forget LilyPond!

>> Well, for me, Guile's _the_ extension language for my LilyPond.
>>
>> It may be more, it may have become more.
>> And yes, I used Guile for some other stuff as well.
>> Nevertheless, it remains the language to extend LilyPond.
>>
>> It feels very strange dropping said phrase.
>
> Same for me.

Like I wrote, Guile remains an extension language, no argument here.

However, describing it as “just” an extension language seems odd to me.
It doesn’t take into account what many have been doing with Guile, and
it doesn’t match the efforts that have gone into Guile since 2.0.

> That Guile takes effort to make extending easy distinguishes Guile
> from all the just-scripting languages out there, and also from many
> other Schemes.

I don’t find embeddability to be that much of a salient feature.  After
all, CPython, Script-Fu, Lua, ECL, GJS, etc. all fit into that category.

Other adjectives I proposed (fast, functional) don’t quite apply to
these, though.

> Actually I’d love to see Guile become better at this: to make it easier
> to deploy an application that uses Guile in a statically compiled
> binary.
>
> Basically to generalize what LilyPond is doing.

In what ways could it become “better” for this use case?

Personally, I encourage people to write more Scheme and less C, there
are lots of good reasons for that.  As a corollary, I encourage moving
from “embedding” Guile (linking your C/C++ program against libguile) to
“extending” Guile (having your main program written in Guile Scheme and
calling out to C/C++ bits as needed).  That’s been my position since the
2.0 days.

Thanks,
Ludo’.



Logo baseline

2020-01-18 Thread Ludovic Courtès
Hello Guilers!

The Guile logo has this “GNU extension language” baseline.  As Guile 3
came out, this baseline felt odd to me, not quite corresponding to the
way I see Guile.

Clearly, Guile is still an extension language, with many great
applications (Gnucash, Lepton-EDA, OpenCog, GDB, etc.), and I’m sure
libguile is here to stay.  Yet, to me, “extension language” does not
accurately capture what Guile today allows for and what people have been
doing with it; since 2.0, it’s more than an extension language, even
more so with the performance afforded by Guile 3.

Thus, I’d propose changing the baseline.  Something that would describe
what Guile is to me is:

  GNU, fast, fun, functional

What’s about you?  What’s Guile to you?  :-)

Ludo’.


signature.asc
Description: PGP signature


Re: GNU Guile 3.0.0 released

2020-01-18 Thread Ludovic Courtès
Hi!

Andy Wingo  skribis:

> We are delighted to announce GNU Guile release 3.0.0, the first in the
> new 3.0 stable release series.
>
> Compared to the previous stable series (2.2.x), Guile 3.0 adds support
> for just-in-time native code generation, speeding up all Guile programs.
> See the NEWS extract at the end of the mail for full details.

Woohoo!  As discussed privately, thumbs up to you Andy for the
incredible work on the compiler and JIT: you had expressed your plan a
few years ago, at a time where there was still a lng way to go, and
it’s impressive to see that it has worked according to the plan.

Thanks also to everyone who contributed to all the other features that
made it into 3.0: they’re also part of what makes Guile 3 so great!

I’m really happy we’ll be able to take advantage of all this in Guix!

Happy Guile 3 hacking!  :-D

Ludo’.



Re: Better HTTPS support in (web client)

2020-01-13 Thread Ludovic Courtès
Hi Andy,

Andy Wingo  skribis:

> On Fri 10 Jan 2020 15:49, Ludovic Courtès  writes:
>
>> Hello Guilers!
>>
>> I’ve pushed a ‘wip-https-client’ branch that contains improvements for
>> HTTPS support in (web client) that I’d like to be part of Guile 3:
>>
>>   https://git.savannah.gnu.org/cgit/guile.git/log/?h=wip-https-client
>
> Looks nice, sounds like a great thing to merge in!

Pushed with a ‘NEWS’ entry!

Apologies for missing 2.9.9.

Thanks,
Ludo’.



Re: Better HTTPS support in (web client)

2020-01-13 Thread Ludovic Courtès
Hello!

Chris Vine  skribis:

> Is the new implementation usable with suspendable ports?  When I last
> looked the read-response-body procedure was not, which meant that
> http-get and http-put were not, which meant that you could not really
> use them with fibers.

It’s not a “new implementation”, rather additional (and IMO important)
features that are added.

So it works as before, meaning that data is passed through a GnuTLS
“session record port”.  And that, in turn, that means this is not
suspendable, unfortunately.

To address that, it should be possible to avoid the session record port
and instead use the lower-level GnuTLS ‘record-receive!’ and
‘record-send’ procedures.  This is left as an excercise to the reader.
:-)

Thanks,
Ludo’.



Better HTTPS support in (web client)

2020-01-10 Thread Ludovic Courtès
Hello Guilers!

I’ve pushed a ‘wip-https-client’ branch that contains improvements for
HTTPS support in (web client) that I’d like to be part of Guile 3:

  https://git.savannah.gnu.org/cgit/guile.git/log/?h=wip-https-client

In a nutshell:

  • $https_proxy support and a ‘current-https-proxy’ parameter;

  • better TLS alert handling;

  • verification of server certificates (!).

You can test it with a program as simple as:

  (use-modules (web client))

  (call-with-values
  (lambda ()
(http-get "https://guix.gnu.org;))
pk)

You can test how expired certificates are handled with:

  guix environment --ad-hoc libfaketime -- \
 faketime 2022-01-01 ./meta/guile /tmp/https.scm

To check whether $https_proxy is honored, try:

  https_proxy=http://localhost:8118 strace -e connect \
./meta/guile /tmp/https.scm

(I have Privoxy running as a proxy on that port.)

Feedback welcome!

Ludo’.



Re: SHA256 performance with Guile 2.2 vs. Guile 3.0

2020-01-07 Thread Ludovic Courtès
Andy Wingo  skribis:

> On Tue 07 Jan 2020 12:08, Ludovic Courtès  writes:
>
>> Andy Wingo  skribis:
>>
>>> Concretely I would add a little part of the compiler to the Tree-IL
>>> phase to serialize a bytecode for the "small" definitions in the module,
>>> for declarative modules, both public and private (because public
>>> definitions may alias private definitions).  This would be stored as a
>>> bytevector in an additional field of the module, and the program being
>>> compiled would be transformed to initialize the "lto" field (placeholder
>>> name) of the module, so that once the compiled module is loaded, we have
>>> the inlinable bindings.  I think this can be done compatibly.
>>
>> OK, sounds great.  What are your thoughts about versioning that wire
>> Tree-IL representation?
>
> It would be a little bytecode language, with its own versioning
> considerations.  It would need to have a translation to and from
> Tree-IL, though not necessarily lossless.  It would change only in
> ABI-compatible ways, using the bytecode version of the Guile doing the
> compilation as a proxy for what is OK to support.

I see, that makes a lot of sense to me.  Thanks for explaining!

Ludo’.



Re: SHA256 performance with Guile 2.2 vs. Guile 3.0

2020-01-07 Thread Ludovic Courtès
Hi!

Andy Wingo  skribis:

> On Mon 06 Jan 2020 10:47, Ludovic Courtès  writes:
>
>> Andy Wingo  skribis:
>>
>>> With cross-module inlining of "small" definitions, I think we would
>>> solve a lot of this kind of problem.  I think we could add this during
>>> 3.0 and for this reason I would hesitate to apply this patch for 3.0
>>> because it changes "fx+" exports to be macros rather than "normal"
>>> values in the ABI.  WDYT?
>>
>> I agree that cross-module inlining is the better fix whereas this patch
>> is the immediate workaround.
>>
>> Are you confident that cross-module inlining can happen be added without
>> introducing incompatibilities over in the 3.0 series?  (At first sight
>> it seems tricky to me, notably because we’d have to store Tree-IL in
>> object files, which introduces compatibility and thus external
>> representation versioning considerations.)
>
> Concretely I would add a little part of the compiler to the Tree-IL
> phase to serialize a bytecode for the "small" definitions in the module,
> for declarative modules, both public and private (because public
> definitions may alias private definitions).  This would be stored as a
> bytevector in an additional field of the module, and the program being
> compiled would be transformed to initialize the "lto" field (placeholder
> name) of the module, so that once the compiled module is loaded, we have
> the inlinable bindings.  I think this can be done compatibly.

OK, sounds great.  What are your thoughts about versioning that wire
Tree-IL representation?

>> If you do, then it’s fine to drop this patch.  If conversely
>> cross-module inlining might take longer, then we can have this patch in
>> and drop it in 3.2.  Your call!  (I guess I’m not being that helpful
>> here.  :-))
>
> :)
>
> I hesitate to land this patch because we haven't shown that it
> significantly helps things, it would need to be undone, and it makes the
> ABI more fragile.  So if that's OK let's do nothing :)

Alright, fine with me!

Thanks,
Ludo’.



Re: Re-exporting a replaced binding

2020-01-06 Thread Ludovic Courtès
Hi!

Andy Wingo  skribis:

> On Fri 03 Jan 2020 19:30, Ludovic Courtès  writes:

[...]

>> Should the #:re-export clause propagate the replace bit, or should
>> it not?  :-)
>
> It is a good question :)  Before, if you re-exported a #:replace
> binding, it wasn't possible to have it be exported without the "replace"
> bit set.  After the change it is possible to do either, and the default
> changes to not replacing.  From NEWS:
>
>   Note to make this change, we had to change the way replacement flags
>   are stored, to being associated with modules instead of individual
>   variable objects.  This means that users who #:re-export an imported
>   binding that was already marked as #:replace by another module will
>   now see warnings, as they need to use #:re-export-and-replace instead.
>
> The 3.0 behavior differs from 2.2 in this regard, although it's just
> warnings and not run-time behavior.  I am sympathetic to the concern
> that it can be difficult to make a system that warns/doesn't warn in the
> same way on 2.2 vs 3.0 but I think the change is the right thing, as the
> new behavior is more expressive.  Because it's a user-visible change it
> is in NEWS.  LMK if you think we need a change here!

I agree that the new mechanism is more expressive, I guess I just hadn’t
realized how it affects re-exports of already-exported bindings.  It’s
easily addressed anyway.

Thank you!

Ludo’.



Re: SHA256 performance with Guile 2.2 vs. Guile 3.0

2020-01-06 Thread Ludovic Courtès
Hello,

Andy Wingo  skribis:

> On Sat 04 Jan 2020 01:40, Ludovic Courtès  writes:
>
>> Ludovic Courtès  skribis:
>>
>>> ludo@ribbon ~/src/guix$ ./pre-inst-env guix environment --pure --ad-hoc 
>>> guile-next guile3.0-hashing -- guile ~/tmp/sha256.scm
>>>
>>> ;;; (hash 
>>> "b33576331465a60b003573541bf3b1c205936a16c407bc69f8419a527bf5c988")
>>> clock utime stime cutime cstime gctime
>>> 65.17 89.75  0.45   0.00   0.00  35.63
>>
>>   (define fx32xor fxxor)
>>   …
>
> From a speed perspective I think there is one major issue and one minor
> issue.
>
> The major issue is that we don't do cross-module inlining.  But now that
> we have declarative modules, this is a possibility:
>
>   https://lists.gnu.org/archive/html/guile-devel/2016-03/msg00026.html
>   https://lists.gnu.org/archive/html/guile-devel/2016-03/msg00027.html

Neat.  Storing Tree-IL in object files reminds me of LTO in GCC & co.

> With cross-module inlining of "small" definitions, I think we would
> solve a lot of this kind of problem.  I think we could add this during
> 3.0 and for this reason I would hesitate to apply this patch for 3.0
> because it changes "fx+" exports to be macros rather than "normal"
> values in the ABI.  WDYT?

I agree that cross-module inlining is the better fix whereas this patch
is the immediate workaround.

Are you confident that cross-module inlining can happen be added without
introducing incompatibilities over in the 3.0 series?  (At first sight
it seems tricky to me, notably because we’d have to store Tree-IL in
object files, which introduces compatibility and thus external
representation versioning considerations.)

If you do, then it’s fine to drop this patch.  If conversely
cross-module inlining might take longer, then we can have this patch in
and drop it in 3.2.  Your call!  (I guess I’m not being that helpful
here.  :-))

> The minor issue, at least relatively speaking, is that IMO the (rnrs
> arithmetic fixnums) API is not appropriate for bitwise operations.  When
> you do bitwise operations and you want to ensure that you're within some
> fixed domain, it's best to do e.g. "(logand x #x)" on operands
> and results.  Guile will optimize this well.  The good optimization
> isn't fixnum vs other kinds of numbers, it's unboxing to raw unsigned
> integers; and you usually want to exclude negative numbers.  fx+ doesn't
> help with that.

I agree, of course.  The tragedy is that people would use this (clumsy)
API in the hope of getting better performance, and the result ends up
being worse performance, at least on today’s Guile (perhaps also on
other implementations?).

Thanks,
Ludo’.



Re: SHA256 performance with Guile 2.2 vs. Guile 3.0

2020-01-06 Thread Ludovic Courtès
Hi Göran,

Göran Weinholt  skribis:

> I've pushed a Guile-specific version of (hashing fixnums) that inlines
> the generic arithmetic procedures. This and some other small changes
> improved the runtime:
>
>clock utime stime cutime cstime gctime
>  before:
> 2.2.6  31.06 32.61  0.03   0.00   0.00   1.38
> 2.9.8  15.55 16.23  0.01   0.00   0.00   1.19
>  after:
> 2.2.6   2.95  3.01  0.00   0.00   0.00   0.10
> 2.9.8   1.98  1.99  0.00   0.00   0.00   0.08

Excellent!

> That's about 100 times slower than sha256sum from coreutils. You might
> get some more performance out of it by unrolling the loop in
> sha-256-transform!.

I suppose that’s a challenge for Andy.  ;-)

Thanks,
Ludo’.



Re: SHA256 performance with Guile 2.2 vs. Guile 3.0

2020-01-03 Thread Ludovic Courtès
Ludovic Courtès  skribis:

> ludo@ribbon ~/src/guix$ ./pre-inst-env guix environment --pure --ad-hoc 
> guile-next guile3.0-hashing -- guile ~/tmp/sha256.scm
>
> ;;; (hash "b33576331465a60b003573541bf3b1c205936a16c407bc69f8419a527bf5c988")
> clock utime stime cutime cstime gctime
> 65.17 89.75  0.45   0.00   0.00  35.63

The patch below gives us:

--8<---cut here---start->8---
ludo@ribbon /tmp/hashing$ guile --r6rs -L .. ~/tmp/sha256.scm

;;; (hash "b33576331465a60b003573541bf3b1c205936a16c407bc69f8419a527bf5c988")
clock utime stime cutime cstime gctime
59.31 80.65  0.39   0.00   0.00  30.73
--8<---cut here---end--->8---

It’s a disappointingly small improvement.  The reason is that (hashing
fixnums) adds another layer of opacity, where it ends up doing
essentially:

  (define fx32xor fxxor)
  …

Thus, no inlining, and no easy trick to solve that.  :-/

Anyhow, I think the patch is probably a good idea.  WDYT?

Thanks,
Ludo’.

diff --git a/module/rnrs/arithmetic/fixnums.scm b/module/rnrs/arithmetic/fixnums.scm
index 4ec1cae0c..c30807eb5 100644
--- a/module/rnrs/arithmetic/fixnums.scm
+++ b/module/rnrs/arithmetic/fixnums.scm
@@ -1,6 +1,6 @@
 ;;; fixnums.scm --- The R6RS fixnums arithmetic library
 
-;;  Copyright (C) 2010, 2011, 2013 Free Software Foundation, Inc.
+;;  Copyright (C) 2010, 2011, 2013, 2020 Free Software Foundation, Inc.
 ;;
 ;; This library is free software; you can redistribute it and/or
 ;; modify it under the terms of the GNU Lesser General Public
@@ -75,25 +75,26 @@
 	  fxrotate-bit-field
 	  fxreverse-bit-field)
   (import (only (guile) ash
-		cons*
-			define-inlinable
-			inexact->exact
-			logand
-			logbit?
-			logcount
-			logior
-			lognot
-			logxor
-			most-positive-fixnum 
-			most-negative-fixnum
-			object-address)
+		cons*
+		define-inlinable
+		inexact->exact
+		logand
+		logbit?
+		logcount
+		logior
+		lognot
+		logxor
+		most-positive-fixnum 
+		most-negative-fixnum
+		object-address)
 	  (ice-9 optargs)
 	  (rnrs base (6))
 	  (rnrs control (6))
 	  (rnrs arithmetic bitwise (6))
 	  (rnrs conditions (6))
 	  (rnrs exceptions (6))
-	  (rnrs lists (6)))
+  (rnrs lists (6))
+  (rnrs syntax-case (6)))
 
   (define fixnum-width
 (let ((w (do ((i 0 (+ 1 i))
@@ -121,70 +122,105 @@
 (or (for-all inline-fixnum? args) (raise (make-assertion-violation
 
   (define-syntax define-fxop*
+(lambda (s)
+  (syntax-case s ()
+((_ name op)
+ (with-syntax ((proc (datum->syntax
+  #'name
+  (string->symbol
+   (string-append "%"
+  (symbol->string
+   (syntax->datum #'name))
+  "-proc")
+   #'(begin
+   ;; Define a procedure for when the inline case doesn't
+   ;; apply.
+   (define proc
+ (case-lambda
+   ((x y)
+(assert-fixnum x y)
+(op x y))
+   (args
+(assert-fixnums args)
+(apply op args
+
+   (define-syntax name
+ (lambda (s)
+   (syntax-case s ()
+ ((_ args (... ...))
+  #'(begin
+  (assert-fixnum args (... ...))
+  (op args (... ...
+ (x
+  (identifier? #'x)
+  #'proc))
+
+  (define-syntax define-alias
 (syntax-rules ()
-  ((_ name op)
-   (define name
-	(case-lambda
-	  ((x y)
-	   (assert-fixnum x y)
-	   (op x y))
-	  (args
-	   (assert-fixnums args)
-   (apply op args)))
+  ((_ new old)
+   (define-syntax new (identifier-syntax old)
 
   ;; All these predicates don't check their arguments for fixnum-ness,
   ;; as this doesn't seem to be strictly required by R6RS.
 
-  (define fx=? =)
-  (define fx>? >)
-  (define fx=? >=)
-  (define fx<=? <=)
+  (define-alias fx=? =)
+  (define-alias fx>? >)
+  (define-alias fx=? >=)
+  (define-alias fx<=? <=)
 
-  (define fxzero? zero?)
-  (define fxpositive? positive?)
-  (define fxnegative? negative?)
-  (define fxodd? odd?)
-  (define fxeven? even?)
+  (define-alias fxzero? zero?)
+  (define-alias fxpositive? positive?)
+  (define-alias fxnegative? negative?)
+  (define-alias fxodd? odd?)
+  (define-alias fxeven? even?)
 
   (define-fxop* fxmax max)
   (define-fxop* fxmin min)
 
-  (define (fx+ fx1 fx2)
+  (define-inlinable (fx+ fx1 fx2)
 (assert-fixnum fx1 fx2) 
 (let ((r (+ fx1 fx2))) 
   (or (inline-fixnum? r)
   (raise (make-implementation

SHA256 performance with Guile 2.2 vs. Guile 3.0

2020-01-03 Thread Ludovic Courtès
Hello Guilers!

Göran Weinholt wrote a pure-Scheme (R6RS) implementation of common
cryptographic hash functions:

  https://github.com/weinholt/hashing

I thought this would make a useful benchmark (and probably favorable to
JIT because it’s one hot loop), so here goes (computing the SHA256 hash
of ‘guile-2.2.6.tar.xz’ with the code below, compiled with -O2, Guile
2.2.6 vs. 2.9.8):

--8<---cut here---start->8---
ludo@ribbon ~/src/guix$ ./pre-inst-env guix environment --pure --ad-hoc guile 
guile-hashing -- guile ~/tmp/sha256.scm

;;; (hash "b33576331465a60b003573541bf3b1c205936a16c407bc69f8419a527bf5c988")
clock utime stime cutime cstime gctime
135.07 164.13  0.52   0.00   0.00  42.14
ludo@ribbon ~/src/guix$ ./pre-inst-env guix environment --pure --ad-hoc 
guile-next guile3.0-hashing -- guile ~/tmp/sha256.scm

;;; (hash "b33576331465a60b003573541bf3b1c205936a16c407bc69f8419a527bf5c988")
clock utime stime cutime cstime gctime
65.17 89.75  0.45   0.00   0.00  35.63
--8<---cut here---end--->8---

Guile 3 is twice as fast!  No difference if we force ‘-O3’.

Still far from the libgcrypt implementation in C + asm, but hey!

--8<---cut here---start->8---
ludo@ribbon ~/src/guix$ time guix hash -f hex 
"/gnu/store/zfp7d4wr5hbl5lrnzs8c15bsc3ygq74g-guile-2.2.6.tar.xz"
b33576331465a60b003573541bf3b1c205936a16c407bc69f8419a527bf5c988

real0m0.097s
user0m0.093s
sys 0m0.024s
--8<---cut here---end--->8---

To be continued…

Ludo’.

(use-modules (hashing sha-2)
 (rnrs bytevectors)
 (ice-9 binary-ports)
 (ice-9 match)
 (ice-9 time))

(define (port-sha256 port)
  ;; Return the SHA256 of the data read from PORT.
  (define bv (make-bytevector 65536))
  (define hash (make-sha-256))

  (let loop ()
(match (get-bytevector-n! port bv 0
  (bytevector-length bv))
  ((? eof-object?)
   (sha-256-finish! hash)
   hash)
  (n
   (sha-256-update! hash bv 0 n)
   (loop)

(define (file-sha256 file)
  ;; Return the SHA256 of FILE.
  (call-with-input-file file port-sha256))

(time (pk 'hash (sha-256->string (file-sha256 
"/gnu/store/zfp7d4wr5hbl5lrnzs8c15bsc3ygq74g-guile-2.2.6.tar.xz"


Re-exporting a replaced binding

2020-01-03 Thread Ludovic Courtès
Hi again!

I’m not sure if this is an intended consequence of
cf08dbdc189f0005cab6f2ec7b23ed9d150ec43d, so I thought I’d share this
example of a practical effect:

--8<---cut here---start->8---
ludo@ribbon /tmp [env]$ cat x.scm
(define-module (x)
  #:use-module (srfi srfi-1)
  #:re-export (delete))
ludo@ribbon /tmp [env]$ cat y.scm
(define-module (y)
  #:use-module (x))

(pk 'delete delete)
ludo@ribbon /tmp [env]$ guile -L . -c '(use-modules (y))'
WARNING: (y): imported module (x) overrides core binding `delete'

;;; (delete #)
ludo@ribbon /tmp [env]$ guile --version
guile (GNU Guile) 2.9.8
--8<---cut here---end--->8---

Here ‘delete’ is replaced by srfi-1, but the replaced bit is not
propagated to module (x), even though (x) simply re-exports it.

Should the #:re-export clause propagate the replace bit, or should
it not?  :-)

(In Guile < 3.x, #:re-export would propagate the replace bit since that
bit was associated with the variable itself.)

Ludo’.



Re: Removing the locale warning?

2019-12-16 Thread Ludovic Courtès
Hi,

Andy Wingo  skribis:

> On Sat 14 Dec 2019 16:44, Ludovic Courtès  writes:
>
>> What about getting rid of the locale warning?
>>
>> More generally, I'm in favor of reducing run-time warnings to a bare
>> minimum, because application users often don’t care about them (plus
>> they’re not i18n’d), and because application developers cannot silence
>> them or handle them in a way that is more suitable for the application.
>>
>> Thoughts?
>
> Funny, I find them really useful as they let me know when my environment
> isn't correctly configured, so I can then take action.

Oh, I see.

> How can we resolve these two use cases?

Not sure.  Guix does an extra ‘setlocale’ call and prints a
user-friendly and Guix-specific message when the locale is unavailable:

  https://git.savannah.gnu.org/cgit/guix.git/tree/guix/ui.scm#n457

As an application developer, I feel that it’s the application’s job to
pay attention to that (when it matters) and it bothers me that libguile
prints its own warning (not i18n’d, not controllable, etc.), just like
it would bother me if libc were emitting such a warning.

Perhaps the REPL and the ‘guild’ script could emit such a warning, while
libguile itself would remain silent?

(I know that Perl and Bash emit a warning when ‘setlocale’ fails, while
Python doesn’t.  As for applications, I’ve seen all sorts of behaviors
ranging from silently ignoring the issue (Coreutils, sed, grep, Git,
etc.) to showing a popup window (Gramps) to emitting a warning (most
GTK+ programs).)

Thoughts?

Ludo’.



Removing the locale warning?

2019-12-14 Thread Ludovic Courtès
Hello!

What about getting rid of the locale warning, as shown below?

More generally, I'm in favor of reducing run-time warnings to a bare
minimum, because application users often don’t care about them (plus
they’re not i18n’d), and because application developers cannot silence
them or handle them in a way that is more suitable for the application.

Thoughts?

Ludo’.

diff --git a/libguile/guile.c b/libguile/guile.c
index fa5fef928..e8879caad 100644
--- a/libguile/guile.c
+++ b/libguile/guile.c
@@ -1,4 +1,4 @@
-/* Copyright 1996-1997,2000-2001,2006,2008,2011,2013,2018
+/* Copyright 1996-1997,2000-2001,2006,2008,2011,2013,2018,2019
  Free Software Foundation, Inc.
 
This file is part of Guile.
@@ -88,8 +88,10 @@ main (int argc, char **argv)
  error messages, use the right locale.  See
  
  for the rationale.  */
-  if (should_install_locale () && setlocale (LC_ALL, "") == NULL)
-fprintf (stderr, "guile: warning: failed to install locale\n");
+  if (should_install_locale ())
+/* Silently ignore 'setlocale' failures.  It's up to the application
+   to handle it.  */
+setlocale (LC_ALL, "");
 
   scm_install_gmp_memory_functions = 1;
   scm_boot_guile (argc, argv, inner_main, 0);


Re: [PATCH 1/2] srfi-34: Replace the 'raise' core binding.

2019-11-26 Thread Ludovic Courtès
Andy Wingo  skribis:

> On Mon 25 Nov 2019 17:45, Ludovic Courtès  writes:
>
>> In Guile 2.x, (srfi srfi-34) would already replace 'raise'.  Replacing
>> avoids a run-time warning about the core binding being overridden.
>>
>> * module/srfi/srfi-34.scm (raise): New variable.
>> Mark it as #:replace instead of #:re-export.
>> ---
>>  module/srfi/srfi-34.scm | 10 +++---
>>  1 file changed, 7 insertions(+), 3 deletions(-)
>>
>> diff --git a/module/srfi/srfi-34.scm b/module/srfi/srfi-34.scm
>> index 0e7ad995d..255bfecb9 100644
>> --- a/module/srfi/srfi-34.scm
>> +++ b/module/srfi/srfi-34.scm
>> @@ -1,6 +1,6 @@
>>  ;;; srfi-34.scm --- Exception handling for programs
>>  
>> -;; Copyright (C) 2003, 2006, 2008, 2010 Free Software Foundation, Inc.
>> +;; Copyright (C) 2003, 2006, 2008, 2010, 2019 Free Software Foundation, Inc.
>>  ;;
>>  ;; This library is free software; you can redistribute it and/or
>>  ;; modify it under the terms of the GNU Lesser General Public
>> @@ -27,12 +27,16 @@
>>  ;;; Code:
>>  
>>  (define-module (srfi srfi-34)
>> -  #:re-export (with-exception-handler
>> -   (raise-exception . raise))
>> +  #:re-export (with-exception-handler)
>> +  #:replace (raise)
>>#:export-syntax (guard))
>>  
>>  (cond-expand-provide (current-module) '(srfi-34))
>>  
>> +(define (raise exn)
>> +  "Raise the given exception, invoking the current exception handler on 
>> EXN."
>> +  (raise-exception exn))
>
> LGTM but it is better to re-export if possible.  The reason is that
> right now the compiler recognizes "throw" and "error" as not falling
> through, and this is good for a number of reasons; it would be nice to
> extend this to raise-exception.  We should make it possible to re-export
> and replace at the same time, IMO.

AFAICS there are two blockers:

  1. We cannot replace & re-export at the same time.

  2. ‘raise’ takes exactly one argument, whereas ‘raise-exception’ takes
 an additional keyword argument.

We could ignore #2, though it’s not great, but I’m not sure how to fix #1.

Perhaps also we should provide a mechanism similar to GCC attributes to
mark a procedure as throwing, so that the compiler can DTRT?

Anyway, in the meantime, should we go ahead and apply this patch?  I
think it’s important from a usability viewpoint.

Thanks!

Ludo’.



Re: For a cheaper ‘bytevector->pointer’

2019-11-25 Thread Ludovic Courtès
Hello!

Andy Wingo  skribis:

> Honestly I would prefer not to do this.  If I understand correctly, the
> problem is in FFI calls -- you have a bytevector and you want to pass it
> as a pointer.  In that case the "right" optimization is to avoid the
> scm_tc7_pointer altogether and instead having an unboxed raw pointer.
> The idioms used in FFI are local enough that a compiler can do this.

I agree!  I have a patch from the 2.0 era (attached), but it doesn’t
work because all the tc3s are already taken.  I don’t think this has
changed but I could well be missing something about the tag space.
WDYT?

> More broadly -- the current FFI is an interpreter but it should be a
> compiler.  When a call happens, the code interprets the description of
> the ABI.  Instead, pointer->function should ideally *compile* a
> trampoline.  In an ideal world this compilation can happen
> ahead-of-time, when the .go file is compiled.

Yes, agreed.

> In the short term, what about allowing bytevectors as arguments
> whereever a pointer is allowed?  Perhaps it's bad to expand the domain
> of these functions but it may be the right trade-off.

So in practice, every time there’s '* in the FFI, it’d accept a
bytevector, right?

I would prefer immediate pointers if that’s possible, and then one of
the two other solutions.

Thanks!

Ludo’.

commit c705f743031b305051549928cd91e5cfdfef7ec7
Author: Ludovic Courtès 
Date:   Sun Jan 30 23:28:13 2011 +0100

Attempt to support "immediate pointers".

Problem is, 3 is not a valid "immediate tag", because that would prevent
using an immediate number as the car of a pair.

diff --git a/libguile/evalext.c b/libguile/evalext.c
index ff2ff0ec0..c9dcf8b96 100644
--- a/libguile/evalext.c
+++ b/libguile/evalext.c
@@ -1,5 +1,6 @@
-/* Copyright (C) 1998,1999,2000,2001,2002,2003, 2006, 2008, 2009, 2010 Free Software Foundation, Inc.
- * 
+/* Copyright (C) 1998, 1999, 2000, 2001, 2002, 2003, 2006, 2008,
+ *   2009, 2010, 2011 Free Software Foundation, Inc.
+ *
  * This library is free software; you can redistribute it and/or
  * modify it under the terms of the GNU Lesser General Public License
  * as published by the Free Software Foundation; either version 3 of
@@ -72,6 +73,8 @@ SCM_DEFINE (scm_self_evaluating_p, "self-evaluating?", 1, 0, 0,
 case scm_tc3_imm24:
 	/* characters, booleans, other immediates */
   return scm_from_bool (!scm_is_null_and_not_nil (obj));
+case scm_tc3_aligned_pointer:
+  return SCM_BOOL_T;
 case scm_tc3_cons:
   switch (SCM_TYP7 (obj))
 	{
diff --git a/libguile/foreign.c b/libguile/foreign.c
index 52da23f6e..d00d4a975 100644
--- a/libguile/foreign.c
+++ b/libguile/foreign.c
@@ -60,7 +60,7 @@ SCM_SYMBOL (sym_null, "%null-pointer");
 SCM_SYMBOL (sym_null_pointer_error, "null-pointer-error");
 
 /* The cell representing the null pointer.  */
-static SCM null_pointer;
+static SCM null_pointer =  SCM_PACK (scm_tc3_aligned_pointer);
 
 #if SIZEOF_VOID_P == 4
 # define scm_to_uintptr   scm_to_uint32
@@ -139,8 +139,9 @@ scm_from_pointer (void *ptr, scm_t_pointer_finalizer finalizer)
 {
   SCM ret;
 
-  if (ptr == NULL && finalizer == NULL)
-ret = null_pointer;
+  if (SCM_LIKELY (((scm_t_uintptr) ptr & 3) == 0 && finalizer == NULL))
+/* Return an immediate pointer.  */
+ret = SCM_PACK ((scm_t_bits) ptr | scm_tc3_aligned_pointer);
   else
 {
   ret = scm_cell (scm_tc7_pointer, (scm_t_bits) ptr);
@@ -1125,7 +1126,6 @@ scm_init_foreign (void)
 #endif
 	  );
 
-  null_pointer = scm_cell (scm_tc7_pointer, 0);
   scm_define (sym_null, null_pointer);
 }
 
diff --git a/libguile/foreign.h b/libguile/foreign.h
index b29001962..bf16126e5 100644
--- a/libguile/foreign.h
+++ b/libguile/foreign.h
@@ -49,12 +49,18 @@ typedef enum scm_t_foreign_type scm_t_foreign_type;
 
 typedef void (*scm_t_pointer_finalizer) (void *);
 
-#define SCM_POINTER_P(x)\
-  (!SCM_IMP (x) && SCM_TYP7(x) == scm_tc7_pointer)
+#define SCM_POINTER_P(x)			\
+  (SCM_IMP (x)	\
+   ? SCM_ITAG3 (x) == scm_tc3_aligned_pointer	\
+   : SCM_TYP7 (x) == scm_tc7_pointer)
+
 #define SCM_VALIDATE_POINTER(pos, x)		\
   SCM_MAKE_VALIDATE (pos, x, POINTER_P)
+
 #define SCM_POINTER_VALUE(x)			\
-  ((void *) SCM_CELL_WORD_1 (x))
+  (SCM_IMP (x)	\
+   ? (void *) ((scm_t_uintptr) (x) & ~3UL)	\
+   : (void *) SCM_CELL_WORD_1 (x))
 
 SCM_API SCM scm_from_pointer (void *, scm_t_pointer_finalizer);
 
diff --git a/libguile/gc.c b/libguile/gc.c
index 91250ba57..1754f6b1d 100644
--- a/libguile/gc.c
+++ b/libguile/gc.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1995,1996,1997,1998,1999,2000,2001, 2002, 2003, 2006, 2008, 2009, 2010 Free Software Foundation, Inc.
+/* Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
+ *   2006, 2008, 2009, 2010, 2011 Free Software Foundation, Inc.
  *
  * This lib

Re: Mutating public bindings of a declarative module

2019-11-25 Thread Ludovic Courtès
Hello!

Andy Wingo  skribis:

> On Sun 24 Nov 2019 18:54, Ludovic Courtès  writes:
>
>> It seems that if you ‘set!’ a public variable of a declarative module,
>> the change is visible to all the module users, but it’s not necessarily
>> visible to procedures within that module, presumably because they use an
>> inlined or specialized variant of that thing.
>>
>> I would have imagined that public bindings are considered mutable and
>> thus not subject to inlining; OTOH, that would obviously be a loss, so
>> the current approach makes sense.
>
> Right, I understand the frustration.  For what it is worth, I think we
> have the right default for what it means to be a declarative module, but
> I'm definitely open to having that conversation.

In C on GNU/Linux, if we have this in the same compilation unit:

  int foo () { … }
  int bar () { foo (); }

then ‘bar’ will not necessarily call the ‘foo’ that’s just above.  It’s
possible to replace ‘foo’ via loader tricks such as ‘LD_PRELOAD’.

That means that (1) ‘foo’ is not subject to inlining, and (2) which
‘foo’ ‘bar’ refers to is determined at load time.

So the semantics we have go somewhat further than this.  Not saying we
should follow C, but that’s what came to mind when thinking about the
pros and cons.

>> Anyway, it complicates a use case for me.  In Guix, we “mock” bindings
>> like so:
>>
>>   (define-syntax-rule (mock (module proc replacement) body ...)
>> "Within BODY, replace the definition of PROC from MODULE with the 
>> definition
>>   given by REPLACEMENT."
>> (let* ((m (resolve-interface 'module))
>>(original (module-ref m 'proc)))
>>   (dynamic-wind
>> (lambda () (module-set! m 'proc replacement))
>> (lambda () body ...)
>> (lambda () (module-set! m 'proc original)
>>
>> and that allows us to write tests that temporarily modify public (or
>> private!) bindings.
>>
>> It seems like this could be addressed by compiling selected modules with
>> ‘user-modules-declarative?’ set to #false, or by avoiding the above hack
>> altogether when possible, but I thought I’d share my impressions and
>> listen to what people think.  :-)
>
> This works.  (Actually the way I would do it is to pass #:declarative?
> #f in the define-module for the modules in question.)

The problem of #:declarative? is that you can’t wrap it in ‘cond-expand’.
I suppose most users will have a transition period during which both 2.2
and 3.0 are supported, and they cannot rely on 3.0-only forms during
that period.  :-/

> Marking some
> bindings as not declarative also works (e.g. (set! foo foo)).

Yes, I thought about that.  Perhaps we could provide a
‘preserve-binding!’ macro that does exactly that but would look less
weird.  :-)

> For me the most robust solution would be to have `mock' verify that the
> module it's funging isn't declarative.  We don't currently have a way to
> know if an individual module binding is declarative or not (though we
> could add this).

‘mock’ would have to error out if finds out that it cannot fiddle with a
binding, so it wouldn’t make much of a difference, just changing the
error type.

Thanks for your feedback!

Ludo’.



[PATCH 2/2] srfi-35: Replace ''.

2019-11-25 Thread Ludovic Courtès
This ensures core binding '' is silently replaced by the SRFI-35
variant.

* module/srfi/srfi-35.scm (srfi:): New variable.  Use it to
 #:replace ''.
---
 module/srfi/srfi-35.scm | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/module/srfi/srfi-35.scm b/module/srfi/srfi-35.scm
index d1549f9d4..efa1566bf 100644
--- a/module/srfi/srfi-35.scm
+++ b/module/srfi/srfi-35.scm
@@ -1,6 +1,6 @@
 ;;; srfi-35.scm --- Conditions -*- coding: utf-8 -*-
 
-;; Copyright (C) 2007-2011, 2017 Free Software Foundation, Inc.
+;; Copyright (C) 2007-2011, 2017, 2019 Free Software Foundation, Inc.
 ;;
 ;; This library is free software; you can redistribute it and/or
 ;; modify it under the terms of the GNU Lesser General Public
@@ -39,8 +39,8 @@
(exception-message . condition-message)
( . )
(error? . serious-condition?)
-   ( . )
(external-error? . error?))
+  #:replace ((srfi: . ))
   #:export (make-condition
 define-condition-type
 condition-has-type?
@@ -139,3 +139,5 @@ by C."
 ((_ (type field ...) ...)
  (make-compound-condition (condition-instantiation type () field ...)
   ...
+
+(define srfi: )
-- 
2.24.0




[PATCH 1/2] srfi-34: Replace the 'raise' core binding.

2019-11-25 Thread Ludovic Courtès
In Guile 2.x, (srfi srfi-34) would already replace 'raise'.  Replacing
avoids a run-time warning about the core binding being overridden.

* module/srfi/srfi-34.scm (raise): New variable.
Mark it as #:replace instead of #:re-export.
---
 module/srfi/srfi-34.scm | 10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/module/srfi/srfi-34.scm b/module/srfi/srfi-34.scm
index 0e7ad995d..255bfecb9 100644
--- a/module/srfi/srfi-34.scm
+++ b/module/srfi/srfi-34.scm
@@ -1,6 +1,6 @@
 ;;; srfi-34.scm --- Exception handling for programs
 
-;; Copyright (C) 2003, 2006, 2008, 2010 Free Software Foundation, Inc.
+;; Copyright (C) 2003, 2006, 2008, 2010, 2019 Free Software Foundation, Inc.
 ;;
 ;; This library is free software; you can redistribute it and/or
 ;; modify it under the terms of the GNU Lesser General Public
@@ -27,12 +27,16 @@
 ;;; Code:
 
 (define-module (srfi srfi-34)
-  #:re-export (with-exception-handler
-   (raise-exception . raise))
+  #:re-export (with-exception-handler)
+  #:replace (raise)
   #:export-syntax (guard))
 
 (cond-expand-provide (current-module) '(srfi-34))
 
+(define (raise exn)
+  "Raise the given exception, invoking the current exception handler on EXN."
+  (raise-exception exn))
+
 (define-syntax guard
   (syntax-rules (else)
 "Syntax: (guard (   ...) )
-- 
2.24.0




[PATCH 0/2] Silence run-time warnings for SRFI-3[45]

2019-11-25 Thread Ludovic Courtès
Hi!

As it stands, Guix on Guile 2.9.5 produces lots of warnings:

--8<---cut here---start->8---
$ ./pre-inst-env guix build libreoffice -nd
WARNING: (guix build utils): imported module (srfi srfi-35) overrides core 
binding `'
WARNING: (guix utils): imported module (srfi srfi-35) overrides core binding 
`'
WARNING: (guix serialization): imported module (srfi srfi-35) overrides core 
binding `'
WARNING: (guix base32): imported module (srfi srfi-35) overrides core binding 
`'
WARNING: (guix store): imported module (srfi srfi-35) overrides core binding 
`'
WARNING: (guix derivations): imported module (srfi srfi-35) overrides core 
binding `'
WARNING: (guix gexp): imported module (srfi srfi-35) overrides core binding 
`'
WARNING: (guix packages): imported module (srfi srfi-35) overrides core binding 
`'
WARNING: (guix modules): imported module (srfi srfi-35) overrides core binding 
`'
WARNING: (guix profiles): imported module (srfi srfi-35) overrides core binding 
`'
WARNING: (guix build gremlin): imported module (srfi srfi-35) overrides core 
binding `'
/gnu/store/lyyllcqa1hq7k3zrv03m2h3xz4a6qawx-libreoffice-6.3.3.2.drv
--8<---cut here---end--->8---

These patches address that.  The first one restores #:replace for
srfi-34 (as was the case in Guile 2.x), and the second one adds
#:replace for srfi-35.

Let me know what you think!

Overall, I think we should be cautious with run-time warnings
because they’re hard to turn off and there’s nothing the user
(as in: the Guix user) can do about it.

Ludo’.

Ludovic Courtès (2):
  srfi-34: Replace the 'raise' core binding.
  srfi-35: Replace ''.

 module/srfi/srfi-34.scm | 10 +++---
 module/srfi/srfi-35.scm |  6 --
 2 files changed, 11 insertions(+), 5 deletions(-)

-- 
2.24.0




Mutating public bindings of a declarative module

2019-11-24 Thread Ludovic Courtès
Hello!

It seems that if you ‘set!’ a public variable of a declarative module,
the change is visible to all the module users, but it’s not necessarily
visible to procedures within that module, presumably because they use an
inlined or specialized variant of that thing.

I would have imagined that public bindings are considered mutable and
thus not subject to inlining; OTOH, that would obviously be a loss, so
the current approach makes sense.

Anyway, it complicates a use case for me.  In Guix, we “mock” bindings
like so:

  (define-syntax-rule (mock (module proc replacement) body ...)
"Within BODY, replace the definition of PROC from MODULE with the definition
  given by REPLACEMENT."
(let* ((m (resolve-interface 'module))
   (original (module-ref m 'proc)))
  (dynamic-wind
(lambda () (module-set! m 'proc replacement))
(lambda () body ...)
(lambda () (module-set! m 'proc original)

and that allows us to write tests that temporarily modify public (or
private!) bindings.

It seems like this could be addressed by compiling selected modules with
‘user-modules-declarative?’ set to #false, or by avoiding the above hack
altogether when possible, but I thought I’d share my impressions and
listen to what people think.  :-)

Thanks,
Ludo’.



For a cheaper ‘bytevector->pointer’

2019-11-24 Thread Ludovic Courtès
Hello!

A few days ago David was explaining on #guile how ‘bytevector->pointer’
was generating too much garbage for his use case.  An idea we came up
with was to embed the pointer object in the bytevector.

The patch below does that but it leads to segfaults because I’m guessing
there’s generated bytecode somewhere that still uses the wrong offset; I
adjusted code that emits ‘pointer-ref/immediate’, what else did I miss?

Also, since we disable internal pointers, we’d need to register an
additional displacement, and I’m not sure if this is a good idea.

Thoughts?

Thanks,
Ludo’.

diff --git a/libguile/bytevectors.c b/libguile/bytevectors.c
index 7dfdab499..00aab6911 100644
--- a/libguile/bytevectors.c
+++ b/libguile/bytevectors.c
@@ -196,10 +196,15 @@
   SCM_SET_BYTEVECTOR_FLAGS ((bv), SCM_BYTEVECTOR_FLAGS (bv) | flag)
 #define SCM_BYTEVECTOR_SET_LENGTH(_bv, _len)\
   SCM_SET_CELL_WORD_1 ((_bv), (scm_t_bits) (_len))
-#define SCM_BYTEVECTOR_SET_CONTENTS(_bv, _contents)	\
-  SCM_SET_CELL_WORD_2 ((_bv), (scm_t_bits) (_contents))
+#define SCM_BYTEVECTOR_SET_CONTENTS(_bv, _contents) \
+  do\
+{   \
+  SCM_SET_CELL_WORD_2 ((_bv), scm_tc7_pointer); \
+  SCM_SET_CELL_WORD_3 ((_bv), (scm_t_bits) (_contents));\
+}   \
+  while (0)
 #define SCM_BYTEVECTOR_SET_PARENT(_bv, _parent)	\
-  SCM_SET_CELL_OBJECT_3 ((_bv), (_parent))
+  SCM_SET_CELL_OBJECT ((_bv), 4, (_parent))
 
 #define SCM_VALIDATE_MUTABLE_BYTEVECTOR(pos, v) \
   SCM_MAKE_VALIDATE_MSG (pos, v, MUTABLE_BYTEVECTOR_P, "mutable bytevector")
diff --git a/libguile/bytevectors.h b/libguile/bytevectors.h
index 980d6e267..77a0ef2f3 100644
--- a/libguile/bytevectors.h
+++ b/libguile/bytevectors.h
@@ -1,7 +1,7 @@
 #ifndef SCM_BYTEVECTORS_H
 #define SCM_BYTEVECTORS_H
 
-/* Copyright 2009,2011,2018
+/* Copyright 2009,2011,2018,2019
  Free Software Foundation, Inc.
 
This file is part of Guile.
@@ -26,20 +26,23 @@
 #include "libguile/gc.h"
 
 #include "libguile/uniform.h"
+#include "libguile/foreign.h"
 
 
 /* R6RS bytevectors.  */
 
 /* The size in words of the bytevector header (type tag and flags, length,
and pointer to the underlying buffer).  */
-#define SCM_BYTEVECTOR_HEADER_SIZE   4U
+#define SCM_BYTEVECTOR_HEADER_SIZE   5U
 
 #define SCM_BYTEVECTOR_LENGTH(_bv)		\
   ((size_t) SCM_CELL_WORD_1 (_bv))
+#define SCM_BYTEVECTOR_POINTER(_bv) \
+  (SCM_PACK_POINTER (SCM_CELL_OBJECT_LOC ((_bv), 2)))
 #define SCM_BYTEVECTOR_CONTENTS(_bv)		\
-  ((signed char *) SCM_CELL_WORD_2 (_bv))
+  ((signed char *) SCM_POINTER_VALUE (SCM_BYTEVECTOR_POINTER (_bv)))
 #define SCM_BYTEVECTOR_PARENT(_bv)		\
-  (SCM_CELL_OBJECT_3 (_bv))
+  (SCM_CELL_OBJECT_4 (_bv))
 
 
 SCM_API SCM scm_endianness_big;
diff --git a/libguile/foreign.c b/libguile/foreign.c
index 1368cc9da..1879c23bc 100644
--- a/libguile/foreign.c
+++ b/libguile/foreign.c
@@ -1,4 +1,4 @@
-/* Copyright 2010-2016,2018
+/* Copyright 2010-2016,2018,2019
  Free Software Foundation, Inc.
 
This file is part of Guile.
@@ -313,8 +313,15 @@ SCM_DEFINE (scm_bytevector_to_pointer, "bytevector->pointer", 1, 1, 0,
 boffset = scm_to_unsigned_integer (offset, 0,
SCM_BYTEVECTOR_LENGTH (bv) - 1);
 
-  ret = scm_from_pointer (ptr + boffset, NULL);
-  register_weak_reference (ret, bv);
+  if (boffset == 0)
+/* The fast path: return the pre-allocated pointer.  */
+ret = SCM_BYTEVECTOR_POINTER (bv);
+  else
+{
+  ret = scm_from_pointer (ptr + boffset, NULL);
+  register_weak_reference (ret, bv);
+}
+
   return ret;
 }
 #undef FUNC_NAME
diff --git a/module/language/tree-il/compile-cps.scm b/module/language/tree-il/compile-cps.scm
index 8f048a504..c164f606b 100644
--- a/module/language/tree-il/compile-cps.scm
+++ b/module/language/tree-il/compile-cps.scm
@@ -883,7 +883,7 @@
(letk k ($kargs ('ptr) (ptr) ,body))
(build-term
  ($continue k src
-   ($primcall 'pointer-ref/immediate '(bytevector . 2)
+   ($primcall 'pointer-ref/immediate '(bytevector . 3)
   (bv
 (letk k ($kargs ('rlen) (rlen) ,access))
 (letk kassume
diff --git a/module/system/base/types.scm b/module/system/base/types.scm
index 418c9fed4..438aee9df 100644
--- a/module/system/base/types.scm
+++ b/module/system/base/types.scm
@@ -1,5 +1,5 @@
 ;;; 'SCM' type tag decoding.
-;;; Copyright (C) 2014, 2015, 2017, 2018 Free Software Foundation, Inc.
+;;; Copyright (C) 2014, 2015, 2017, 2018, 2019 Free Software Foundation, Inc.
 ;;;
 ;;; This library is free software; you can redistribute it and/or modify it
 ;;; under the terms of the GNU Lesser General Public License as published by
@@ -415,7 +415,7 @@ using BACKEND."
 

Re: guile 3 update, halloween edition

2019-11-16 Thread Ludovic Courtès
Hi Andy!

Andy Wingo  skribis:

> On Fri 15 Nov 2019 10:03, Ludovic Courtès  writes:
>
>> 0. Do I get it right that ‘throw’ and ‘catch’ are not “deprecated” in
>> the sense of (ice-9 deprecated) and are instead simply not the
>> “preferred” exception mechanism?
>
> Correct.  I think we could envision deprecating them in some future but
> not within the next couple years at least.

Alright, sounds good.

>> I guess we could add a specific ‘’ exception or similar,
>> which would allow us to improve error reporting (that can come later, of
>> course.)

What I meant is that type errors are “special” enough to deserve their
own type more specific than the catch-all ‘’ (just
like there’s already a separate ‘’, for instance.)

> Yes.  So right now Guile is in a bit of a transitional state -- it still
> signals 99.9% of errors via `throw'.  Probably we want to change to have
> structured exceptions for almost all of these.  To preserve
> compatibility we would probably need to mix in an
>  to all of these exceptions, or otherwise
> augment `exception-kind' and `exception-args' to synthesize these values
> when appropriate.

Yes, I agree.

Speaking of which, it seems that ‘set-guile-exception-converter!’ is
currently private, but I wonder if the goal was to make it public (it
seems to be unused)?

If it were public, I imagine that users could take advantage of it to
support both exception styles, just like Guile does.  For instance, C
bindings that currently call ‘throw’ could provide additional “exception
converters” for the benefit of Scheme users who’d rather use structured
exceptions.  (That would also give less of an incentive to provide a C
API for all of this.)

WDYT?

>> Guix has ‘’ error conditions, which I’ve found useful when
>> combined with other error conditions in cases where location info from
>> the stack isn’t useful:
>>
>>   https://git.savannah.gnu.org/cgit/guix.git/tree/guix/utils.scm#n832
>>
>> I wonder if (ice-9 exceptions) should provide something like that.
>
> Neat :)  Yes sure.  I think, any exception type can be added to (ice-9
> exceptions) -- there's little "name cost" like there is in boot-9.
> Which reminds me, I want to make boot-9 do a (resolve-module '(ice-9
> exceptions)) so that the more capable make-exception-from-throw always
> gets installed.
>
>> 2. What are you thoughts regarding exposing structured exceptions to C?
>> I’ve always been frustrated by ‘system-error’ :-).  Guix has a hack to
>> augment ‘system-error’ with information about the offending file name:
>>
>>   https://git.savannah.gnu.org/cgit/guix.git/tree/guix/ui.scm#n520
>>
>> If the POSIX bindings would emit a structured ‘’ record,
>> that’d be pretty cool.
>
> I don't know :)  Right now raise-exception is marked SCM_INTERNAL.
> Probably it should be public.  There is no public C API for any of this
> new functionality, as it stands; a TODO.
>
> Regarding exception objects, there are two questions: one, how to create
> exceptions of specific kinds; I suspect scm_make_system_error () would
> be fine, and probably you want scm_make_exception to be able to mix
> various exceptions.  Second question, do we want to expose accessors
> too?  It can be a lot of API surface and I am a bit wary of it.  But,
> perhaps it is the right thing.  I do not know.

Yeah, it might be that having C stick to ‘throw’ + allowing for
user-defined exception converters would be enough.

>> 4. Is ‘’ actually used?  Is the goal to make it continuable?
>> That sounds great.
>
> Any exception can be raised in a continuable way.  Whether a raise is
> continuable or not depends on the value of the #:continuable? keyword to
> raise-exception.  I think that's the intention of  but I don't
> really have instincts about how it might be used.  Guile defines it
> because it's in R6RS, but how it will be used is an open question :)

I suppose the intent is to effectively allow users to implement the UI
stuff as a sort of co-routine to support separation of concerns: you
just raise a ‘’ that some other code displays in its preferred
way (console message, popup window, whatever) and eventually calls your
continuation.

That’s something I’ve been wanting for some time, because right now
we’re able to separate out the UI concern for exception display, but not
for warnings.

However, it seems that the handler passed to ‘with-exception-handler’
does not receive the continuation, so is it the case that currently
handlers cannot resume exceptions?  (Again not a showstopper IMO but
rather another wishlist item :-)).

Thank you!

Ludo’.



Re: guile 3 update, halloween edition

2019-11-15 Thread Ludovic Courtès
Hello Andy & all!

Thanks for the great summary.

I’ve taken a look at ‘wip-exceptions’, which is also remarkably easy to
follow because all the changes are incremental and follow the path you
explained in your message; thanks a lot for making it this clear!

I’ve very much support this change, I always found the key+args
convention to be poor compared to structured error condition objects.

The changes in ‘wip-exceptions’ all make sense to me; some random
comments below.


0. Do I get it right that ‘throw’ and ‘catch’ are not “deprecated” in
the sense of (ice-9 deprecated) and are instead simply not the
“preferred” exception mechanism?

At least, that’s how I would view it :-), because ‘throw’ cannot (yet)
disappear from C code, and because it’s a migration that could take more
than two stable series to really complete.


1. I see things like:

+(define (make-condition type . field+value)
+  "Return a new condition of type TYPE with fields initialized as specified
+by FIELD+VALUE, a sequence of field names (symbols) and values."
+  (unless (exception-type? type)
+(scm-error 'wrong-type-arg "make-condition" "Not a condition type: ~S"
+   (list type) #f))

and:

+  (unless (symbol? key)
+(throw 'wrong-type-arg "throw" "Wrong type argument in position ~a: ~a"
+   (list 1 key) (list key)))

I guess we could add a specific ‘’ exception or similar,
which would allow us to improve error reporting (that can come later, of
course.)

Guix has ‘’ error conditions, which I’ve found useful when
combined with other error conditions in cases where location info from
the stack isn’t useful:

  https://git.savannah.gnu.org/cgit/guix.git/tree/guix/utils.scm#n832

I wonder if (ice-9 exceptions) should provide something like that.


2. What are you thoughts regarding exposing structured exceptions to C?
I’ve always been frustrated by ‘system-error’ :-).  Guix has a hack to
augment ‘system-error’ with information about the offending file name:

  https://git.savannah.gnu.org/cgit/guix.git/tree/guix/ui.scm#n520

If the POSIX bindings would emit a structured ‘’ record,
that’d be pretty cool.


3. I wonder if we could take advantage of the new ‘’ exception
to start i18n of error messages.  It might be as simple as telling
xgettext to recognize ‘make-exception-with-message’ as a keyword, though
currently there are few calls to ‘make-exception-with-message’ followed
by a literal.


4. Is ‘’ actually used?  Is the goal to make it continuable?
That sounds great.


Bah, you give us a present and I reply with an additional wishlist.  ;-)

Anyway, ‘wip-exceptions’ looks great to me as it is, so I’m all for
merging it to ‘master’.

Thanks a lot!

Ludo’.



Re: conflicts in the gnu project now affect guile

2019-10-18 Thread Ludovic Courtès
Hi Mark,

Mark H Weaver  skribis:

> RMS has not yet appointed me as a co-maintainer.  To my knowledge, the
> only thing he has done so far is to *ask* me if I wanted to be appointed
> co-maintainer.  I answered "yes", but I've not yet received any further
> messages from him on this topic.  I also note that I'm not listed as a
> maintainer in the official list of maintainers.

Thanks for clarifying, Mark.  In the current context, I, like others
here, find this action of RMS very problematic.  Far from helping
deescalate tensions, it has evidently poured oil onto the fire and moved
the conflict to Guile, which until now was a safe harbor far away from
the private GNU mailing lists.

It has also complicated our relation, though I’m happy the discussion
we’re having is helping avoid misunderstandings and misconceptions.

Ludo’.



Re: [FOSDEM] FOSDEM 2020 Minimalistic, Experimental and Emerging Languages Devroom CfP

2019-10-12 Thread Ludovic Courtès
Hello!

Manolis Ragkousis  skribis:

> We are excited to announce a devroom on minimalistic, experimental
> and/or emerging languages (with big ideas) at FOSDEM on Sunday
> February 2nd 2020!

Woohoo!

Thank you Manolis, Pjotr, and everyone who makes it possible once again!

Ludo’.



GNU Guile 2.2.6 released

2019-06-30 Thread Ludovic Courtès
We are delighted to announce GNU Guile release 2.2.6, the sixth bug-fix
release in the 2.2 stable release series.  See the NEWS excerpt that
follows for full details.

 *  *  *

Guile is an implementation of the Scheme programming language.

The Guile web page is located at https://gnu.org/software/guile/, and
among other things, it contains a copy of the Guile manual and pointers
to more resources.

Guile can run interactively, as a script interpreter, and as a Scheme
compiler to VM bytecode.  It is also packaged as a library so that
applications can easily incorporate a complete Scheme interpreter/VM.
An application can use Guile as an extension language, a clean and
powerful configuration language, or as multi-purpose "glue" to connect
primitives provided by the application.  It is easy to call Scheme code
from C code and vice versa.  Applications can add new functions, data
types, control structures, and even syntax to Guile, to create a
domain-specific language tailored to the task at hand.

Guile implements many common Scheme standards, including R5RS, R6RS, and
a number of SRFIs.  In addition, Guile includes its own module system,
full access to POSIX system calls, networking support, multiple threads,
dynamic linking, a foreign function call interface, and powerful string
processing.

Guile 2.2.6 can be installed in parallel with Guile 2.0.x; see
https://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html.

 *  *  *

Changes in 2.2.6 (since 2.2.5)

* Bug fixes

** Fix regression introduced in 2.2.5 that would break HTTP servers

Guile 2.2.5 introduced a bug that would break the built-in HTTP server
provided by the (web server) module.  Specifically, HTTP servers would
hang while reading requests.  See <https://bugs.gnu.org/36350>.

** 'strftime' and 'strptime' honor the current locale encoding

Until now these procedures would wrongfully assume that the locale
encoding is always UTF-8.  See <https://bugs.gnu.org/35920>.

** Re-export 'current-load-port'

This procedure was erroneously removed in the 2.2 series but was still
documented.

** Minor documentation mistakes were fixed


 *  *  *

Here are the compressed sources:
  https://ftp.gnu.org/gnu/guile/guile-2.2.6.tar.gz   (18MB)
  https://ftp.gnu.org/gnu/guile/guile-2.2.6.tar.lz   (9MB)
  https://ftp.gnu.org/gnu/guile/guile-2.2.6.tar.xz   (11MB)

Here are the GPG detached signatures[*]:
  https://ftp.gnu.org/gnu/guile/guile-2.2.6.tar.gz.sig
  https://ftp.gnu.org/gnu/guile/guile-2.2.6.tar.lz.sig
  https://ftp.gnu.org/gnu/guile/guile-2.2.6.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA256 checksums:

08c0e748740b61cdd97949b69e8a5e2997d8c2fe6c7e175819eb18444506  
guile-2.2.6.tar.gz
1a71fd3d37f97423a402b2e38b1be9d80387dafa5c66fc3e5967307d85624aa5  
guile-2.2.6.tar.lz
b33576331465a60b003573541bf3b1c205936a16c407bc69f8419a527bf5c988  
guile-2.2.6.tar.xz

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify guile-2.2.6.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys 
3CE464558A84FDC69DB40CFB090B11993D9AEBB5

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.69
  Automake 1.16.1
  Libtool 2.4.6
  Makeinfo 6.5
  Gnulib v0.1-1157-gb03f418


Happy hacking with Guile!

Ludovic Courtès, Mark H Weaver, and Andy Wingo.


signature.asc
Description: PGP signature


Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

2019-06-20 Thread Ludovic Courtès
Hi Mark,

Mark H Weaver  skribis:

> I've finished my updates to the NEWS file in preparation for the 2.2.5
> release.  Feel free to reorganize, edit, or expand on the NEWS entries
> as you think best.  I believe that all of the relevant changes are
> listed, anyway.

Thanks a *lot* for all the preparation work, that made things really
easy for me.

> Note that GUILE-VERSION still needs to be updated.  See below for the
> changes that I believe should be made to it.
>
> If it all looks okay to you, feel free to run distcheck and upload the
> release.

Done!  :-)

Thanks,
Ludo’.

PS: This is the first release that I build and upload while on the
train, crazy stuff.  ;-)



GNU Guile 2.2.5 released

2019-06-20 Thread Ludovic Courtès
he user when 'madvise' returns ENOSYS.
   Fixed in commit 45e4ace6603e00b297e6542362273041aebe7305.
** Add 'texinfo' as a dependency in the README.
   Fixed in commit 1bbce71501198c3c7abdf07941f5cdc1434858c2.
** Don't mutate read-only string in ports test.
   Fixed in commit 552f007e91a97f136aad1b22918688b61d03a4a3.
** Remove redefinition of when & unless in snarf-check-and-output-texi.
   Fixed in commit 1ba5d6f47a54dceee4452a1e7726d2635e5b3449.
** Fix strftime when Guile is built without threading support.
   Fixed in commit 139c702fc8b61fdeb813c3428fef3701ea8677f9.
** Avoid leaking a file descriptor in test-unwind.
   Fixed in commit 1437b76777e576b3d000e2f80c5ecdb33a74ac33.
** Fix binary output on files created by mkstemp!.
   Fixed in commit 78468baa118d316050a27e43250966e52ffd3d54.
** Fix crypt-on-glibc test error.
   Fixed in commit 27ffbfb0235de466016ea5a6421508f6548971b6.
** Fix race when expanding syntax-parameterize and define-syntax-parameter.
   <https://bugs.gnu.org/27476#102>
** Add a fallback value for the locale-monetary-decimal-point.
   Fixed in commit 9ba449643d4c2ac1d2174befca7d765af222bcc0.
** Handle newlib C library's langinfo constant names.
   Fixed in commit 92105d13ad1363b511214589b7d62d95304beb17.
** Make locale monetary conversion tests be less strict on terminal whitespace.
   Fixed in commit 2a3ccfb66714efc1c081ea6e921336f80b756d3c.
** Disable test for current value of setitimer on Cygwin.
   Fixed in commit 3a64c504caaf83e9faf2ec9b7d0e031e1a6a09b9.
** Fix gc.test "after-gc-hook gets called" failures.
   <https://bugs.gnu.org/31776#17>
** Update iconv.m4 from gnulib, to fix an iconv leak during configure.
   <https://lists.gnu.org/archive/html/guile-devel/2019-05/msg00011.html>
** guild compile: Add missing newline in "unrecognized option" error message.
   Fixed in commit 85c5bae4fd94f8686d26fd792b7c0f588c23bd94.
** 'basename' now correctly handles "/" and "//".
   Fixed in commit 36ad1d24b3d2c174a64c445502a36f19605dbd65.
** Make srfi-71 visible through 'cond-expand'.
   Fixed in commit 59a06d8392234fbec8b3605cec266a7a0a7b7a56.


 *  *  *

Here are the compressed sources:
  https://ftp.gnu.org/gnu/guile/guile-2.2.5.tar.gz   (18MB)
  https://ftp.gnu.org/gnu/guile/guile-2.2.5.tar.lz   (9MB)
  https://ftp.gnu.org/gnu/guile/guile-2.2.5.tar.xz   (11MB)

Here are the GPG detached signatures[*]:
  https://ftp.gnu.org/gnu/guile/guile-2.2.5.tar.gz.sig
  https://ftp.gnu.org/gnu/guile/guile-2.2.5.tar.lz.sig
  https://ftp.gnu.org/gnu/guile/guile-2.2.5.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA256 checksums:

c3c7a2f6ae0d8321a240c7ebc532a1d47af8c63214157a73789e2b2305b4c927  
guile-2.2.5.tar.gz
340a2ec7a6beb080f1b6f34e642dc2fb2235176468dcad2d01fdc695ed5a0128  
guile-2.2.5.tar.lz
fc3073f12d43650e643dc0111730b73a9fa11f131b9b72f66d01e82d83e685a7  
guile-2.2.5.tar.xz

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify guile-2.2.5.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys 
3CE464558A84FDC69DB40CFB090B11993D9AEBB5

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.69
  Automake 1.16.1
  Libtool 2.4.6
  Makeinfo 6.5
  Gnulib v0.1-1157-gb03f418

Thanks to Arun Isaac, Christopher Lemmer Webber, Jan Smydke, and
Shea Levy who all contributed to this release.

Happy hacking with Guile!

Ludovic Courtès, Mark H Weaver, and Andy Wingo.


signature.asc
Description: PGP signature


Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

2019-06-17 Thread Ludovic Courtès
Hi Mark,

Mark H Weaver  skribis:

> Sure, I can take care of updating NEWS in the next day or two.

Awesome, thank you!

>>> Regrettably, Guile 2.2 has become too heavy to build on the only machine
>>> in my possession that I have any trust in.  I don't have a machine that
>>> I consider sufficiently trustworthy to produce build outputs for wider
>>> distribution.  I'm not sure that any of us do.
>>
>> Note that “make dist” is rather inexpensive;
>
> I assume it builds the prebuilt .go files, no?  That involves running
> Guile's compiler, which is too heavy to run on my Yeeloong.

I think it’s not that bad if you already have build tree with Guile’s
compiler.  If you start from scratch, it’s definitely expensive.

> If you'd like to produce the 2.2.5 release in the traditional way,
> that's fine with me.  I'm not comfortable doing it myself, though.

OK, I can do this.

>> One issue is that “make dist” is non-deterministic because the archive
>> contains timestamps; I’m sure there of other sources of non-determinism
>> though, because “make dist” was not designed with that in mind.
>
> Right.  I suppose the right approach is to start a conversation with the
> autotools developers.  In the meantime, I wonder if we could implement
> our own deterministic version of "make dist" using Guix, and use that
> instead.  Or perhaps it would be easier to use "make dist" and then
> canonicalize the timestamps in the resulting tarball in a later step?
>
> Thoughts?

I think you raise valid concerns, but they are to some extent beyond the
scope of Guile.

Regarding “make dist”, there’s the issue of tar timestamps, of
version.texi, and probably others of that sort in the
autotools-generated machinery.

So yes, I think this should be discussed with the Automake/Autotools
developers.  Namely: how can we achieve reproducible “dist” builds?
Which tools should honor SOURCE_DATE_EPOCH and how? etc.

As a PoC and/or interim solution, we could also try to hack a
reproducible “make dist” in Guix.

For Guix, from a bootstrapping viewpoint, the alternative is to do away
with tarballs produced by “make dist” and instead always run
“autoreconf” ourselves, like Debian does.  That would solve a large part
of the problem.

>> The non-source byproducts in release tarballs are: the pre-built .go
>> files (which are optional), psyntax-pp.scm, and then Info files and all
>> the autotools machinery.  Are these those you had in mind?
>
> Yes, all of the above are potential security risks, except possibly for
> the info files.

I think psyntax-pp.scm is the main issue since all the others can be
trivially rebuilt (the package in Guix deletes the pre-built .go files
for instance.)

With regards to bootstrapping in the context of Guile, I believe
psyntax-pp.scm should be our primary concern.

Thanks,
Ludo’.



Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

2019-06-16 Thread Ludovic Courtès
Hi Mark,

Mark H Weaver  skribis:

> Ludovic Courtès  writes:
>> What would you think of releasing ‘stable-2.2’ as 2.2.5?
>
> I think it's a fine idea.

Awesome.  We’ll have to update NEWS; I can give it a go, but if you
could add bullet items for the things you’ve worked on, that’d be great.

>> It’s great if you can do it, Mark, but otherwise I can do it.
>
> Regrettably, Guile 2.2 has become too heavy to build on the only machine
> in my possession that I have any trust in.  I don't have a machine that
> I consider sufficiently trustworthy to produce build outputs for wider
> distribution.  I'm not sure that any of us do.

Note that “make dist” is rather inexpensive; “distcheck” is much more
expensive though, but maybe avoidable for a minor release tarball.

> To mitigate the risk that a compromised development machine could be
> used to attack others, I propose that we adopt a practice of distributed
> verification of release tarballs.  We would publish code that uses Guix
> to produce the release tarball deterministically, and put out a call for
> volunteers to generate the tarball and post signed declarations
> containing the hash of the resulting tarball.  After we have received
> several such declarations, we can sign and publish the official tarball.

I don’t think this should block 2.2.5, but I think it’s an idea we
should explore.

One issue is that “make dist” is non-deterministic because the archive
contains timestamps; I’m sure there of other sources of non-determinism
though, because “make dist” was not designed with that in mind.

The non-source byproducts in release tarballs are: the pre-built .go
files (which are optional), psyntax-pp.scm, and then Info files and all
the autotools machinery.  Are these those you had in mind?

Thoughts?

Ludo’.



Re: Immediate doubles (up to 2^256) and rationals coming to Guile 3

2019-06-11 Thread Ludovic Courtès
Hi,

Mark H Weaver  skribis:

> Ludovic Courtès  writes:
>> Though an immediate, like a fixnum or an iflo, is still something
>> different from a tagged heap object like a pair, right?  So I would
>> expect SCM_THOB_P to be a different test, not a drop-in replacement for
>> SCM_NIMP, is that correct?
>
> That's right.  It's not possible to create a drop-in replacement for
> SCM_NIMP, because it is being used to answer two different questions
> which used to be effectively equivalent, but no longer are:
>
> (1) Is X a pointer to a heap object with a heap tag in the first word?
> (2) Is X a reference to a heap object?
>
> Test (1) needs to be done before checking the heap tag, to implement
> type predicates for heap objects.  Test (2) is needed in relatively few
> places, e.g. to decide whether to register disappearing links when
> adding an entry to a weak hash table.
>
> Actually, in my current branch I've removed the SCM_IMP and SCM_NIMP
> macros outright, because it seems to me they are likely to be misused.
>
> SCM_THOB_P implements test (1) and SCM_HEAP_OBJECT_P implements test (2).

I see.

> There's no masking involved.  Rather, it is subtracted from the pointer,
> which allows the tag to be fused with the field offset.  For example, on
> x86-64, whereas CAR and CDR were previously:
>
>  1c0:   48 8b 07mov(%rdi),%rax  ;old car
> and:
>  1d0:   48 8b 47 08 mov0x8(%rdi),%rax   ;old cdr
>
> Now they become:
>
>  1e0:   48 8b 47 fa mov-0x6(%rdi),%rax  ;new car
> and:
>  1f0:   48 8b 47 02 mov0x2(%rdi),%rax   ;new cdr

Looks reasonable.  :-)

> Fortunately, BDW-GC provides GC_REGISTER_DISPLACEMENT, which allows us
> to register a small offset K, such that BDW-GC should recognize pointers
> that point K bytes into a heap block.  We've been using this in both 2.0
> and 2.2 from scm_init_struct (), and in 'master' it's done in
> scm_storage_prehistory ().
>
> This new approach entails registering one additional displacement.

Cool; if it’s just one displacement, that’s OK.

> What do you think?

It all looks perfectly reasonable to me!

Thanks for explaining,
Ludo’.



Re: Immediate doubles (up to 2^256) and rationals coming to Guile 3

2019-06-11 Thread Ludovic Courtès
Hello,

Mark H Weaver  skribis:

> Ludovic Courtès  writes:

[...]

>> IIUC, your plan is to have a different tagging on 32-bit platforms,
>> without fixflos, right?  I’m curious to see how much complexity would
>> entail from that.
>
> Yes, although I'm avoiding the term "fixflos" because IEEE doubles are
> also fixed width, and thus the term "fixflos" wouldn't adequately
> distinguish them from IEEE doubles.

Right!

> Anyway, I agree that it's inconvenient to have different tags on
> different targets, and I've been working to minimize the differences.
>
> At present, I'm currently implementing an alternative strategy where
> pairs are tagged in their pointers instead of in their CARs, which
> enables us to separate the heap tags and immediate tags into two
> independent spaces.

At first this sounds rather radical :-), but maybe it’s preferable this
way.

> In this new approach, the heap tags are left unchanged, and the only
> tags that vary with target word size are the fixints, fixrats, iflos,
> and pair pointers.  All other tags will be uniform across targets,
> including the non-number immediates.  Here's the new version:
>
> ;; /* with iflos:   xxx:  iflo (000 < xxx < 110)
> ;;(64-bit) 0111:  fixrat
> ;; :  fixnum
> ;; 0110:  pair
> ;;  000:  tagged heap object (thob)
> ;; 1110:  other immediate
> ;;
> ;; without iflos: 1:  fixnum
> ;;(32-bit)  010:  fixrat
> ;;  100:  pair
> ;;  000:  tagged heap object (thob)
> ;; 1110:  other immediate
>
> This new approach brings its own complications, mainly two:
>
> (1) It breaks the long-standing assumptions in Guile that all
> non-immediates have a tag in their first word and that pointers are
> always untagged.  In my preliminary patch, I introduce a new concept
> called a "tagged heap object" or "thob", and most existing checks
> for SCM_NIMP or !SCM_IMP must be changed to use SCM_THOB_P.

Though an immediate, like a fixnum or an iflo, is still something
different from a tagged heap object like a pair, right?  So I would
expect SCM_THOB_P to be a different test, not a drop-in replacement for
SCM_NIMP, is that correct?

> (2) Our existing VM instructions almost invariably specify offsets with
> a granularity of whole words.  To support tagged pair pointers with
> good performance, I think we need a few new instructions that
> specify byte offsets, to avoid the expensive extra step of removing
> the tag before accessing the CAR or CDR of a pair.

So instead of a pointer dereference, SCM_CAR becomes mask + dereference,
right?

I think we disable GC “interior pointer” scanning.  With this scheme, an
SCM for a pair would actually point in the middle of a pair; could this
be an issue for GC?

Thank you!

Ludo’.



Re: Immediate doubles (up to 2^256) and rationals coming to Guile 3

2019-06-07 Thread Ludovic Courtès
Hi Mark,

Mark H Weaver  skribis:

> I've found a way to efficiently support both immediate IEEE binary-64
> doubles up to ~1.158e77 (with larger ones transparently allocated on the
> heap), and also immediate exact rationals with up to 54 binary digits
> (~16 decimal digits), without restricting the 64-bit pointer space at
> all, and without any loss of arithmetic precision.

Woow, impressive, and really clever!

As Dave Thompson wrote on IRC yesterday, this can make a significant
difference for applications such as graphics and game engines.  I hadn’t
read the message yet and thought “hey, why not make instructions for
things like trigonometric functions so you get unboxed floats” but
obviously, as Dave pointed out, that wouldn’t scale well.  :-)

> Here's the format of fixrats on 64-bit systems:
>
>   Srrx0111
>   |\/\___/\__/
>   |  |  |  |
>   | rank (6 bits) data (53 bits)  tag
>  sign

[...]

> I chose this representation because it allows us to leverage existing
> floating-point hardware to efficiently pack fixrats.  Simply convert the
> denominator to an IEEE double, and now the rank will be in the low bits
> of the IEEE exponent field, immediately adjacent to the denominator with
> its high bit removed.  This simplifies the packing operation.

Fun.  :-)

Fixrats can make rationals more practical in applications where one
would have avoided them for performance.

IIUC, your plan is to have a different tagging on 32-bit platforms,
without fixflos, right?  I’m curious to see how much complexity would
entail from that.

I don’t know what Andy thinks, but if there’s a good way forward for
both 32-bit and 64-bit, it’d be a nice bonus for 3.0!

Thanks,
Ludo’.



Releasing 2.2.5?

2019-06-06 Thread Ludovic Courtès
Hello comrades!

What would you think of releasing ‘stable-2.2’ as 2.2.5?

It’s great if you can do it, Mark, but otherwise I can do it.

Thanks,
Ludo’.



Re: Cross-compiling guile libraries

2019-03-23 Thread Ludovic Courtès
Hi Mathieu,

Mathieu Othacehe  skribis:

> Even if guild has a "target" option allowing to cross-compile sources,
> autotools does not seem to have support for guile
> cross-compilation. For instance, passing "--host=aarch64-linux-gnu" to
> configure script doesn't set the "target" argument of guild to the
> specified triplet.

Libraries like Guix itself pass --target to ‘guild’ just fine, so I
guess we’d have to fix Autotools-based Guile packages that don’t do
that.

Thanks,
Ludo’.



Re: FOSDEM 2019

2019-02-06 Thread Ludovic Courtès
Nala Ginrut  skribis:

> Does it mean someone is working on writing Racket specific dialect
> front-end on Guile? I appreciate!

Nope!  Actually Chris Webber raised the issue of collaboration between
Racket and Guile in their talk, which is why we’re discussing it.

But Guile is still Guile and there’s also lots of Guile hacking waiting
for us.  :-)

Ludo’.



Re: FOSDEM 2019

2019-02-05 Thread Ludovic Courtès
Hi,

Amirouche Boubekki  skribis:

> What does it mean in practice? What are the tasks that must dealt with?

Like I wrote, an immediate task is to write a Racket importer for Guix¹
and to actually package things.

Next, we could ensure the subset of these packages that use #r6rs can
also be used as Guile packages.  However Chris noted that there are few
of them; most use #lang racket.

Thus, the next idea is to have #lang racket in Guile, which is quite a
bit of work but probably doable.  And the converse: #lang guile in
Racket.

There are probably other things that could be done, or variants on this
theme.

Is that clearer?

Thanks,
Ludo’.

¹ https://gnu.org/software/guix/manual/en/html_node/Invoking-guix-import.html



Re: FOSDEM 2019

2019-02-04 Thread Ludovic Courtès
Hello Mikael!

Mikael Djurfeldt  skribis:

> It was a great experience and joy for me to meet some of you at FOSDEM
> 2019. Thank you all!

Seconded, that was awesome!  Meeting Guilers, both newcomers and
old-timers :-), was just great!

> Everyone who works with Guile knows that it's crap and look with envy at
> projects like Chez and Racket, right?

Actually, while there are many things I like about Racket (Typed Racket,
DrRacket, and the wealth of available libraries come to mind), I’m also
very happy with Guile and everything that’s been going on around it
lately!

Things like Gash/Geesh, nyacc, MesCC (we have a C compiler in Guile!),
libfive, Haunt, Mumi (which runs at ), the
picture language, the many bindings that came into existence over the
last couple of years (parted, newt, GIR, Git, etc.), and of course Guix…
I think we can all be very happy with how our community and software has
been growing!  There are many Scheme implementations out there, but few
can pride themselves on having seen this much activity lately.

I also consider it a strength that Guile is very much connected to the
practical GNU/Linux environment and not insulated like so many
programming language implementations.

The amazing work Andy has been putting into Guile to keep improving it
is another reason to be excited about Guile, this year maybe more than
ever.  I do wish more of us were joining Andy on that journey, but maybe
2019 is the year to make it happen!

So… I’m just super excited about Guile, and I bet I’m not the only one.
Chris Webber, Andy, Janneke, myself, and a couple of other people talked
about this yesterday at FOSDEM, and we agree that we must work on
increasing the bandwidth with Racketeers.  As it turns out, Chris in the
right position to do that: we almost tricked him into writing a Racket
package importer for Guix and perhaps, who knows, toying with the idea
of #lang guile in Racket.  :-)

Happy hacking, Guilers!

Ludo’.




Re: GNU Guix Days before FOSDEM

2019-01-08 Thread Ludovic Courtès
Hello!

zimoun  skribis:

> At Paris' Guix' meeting in December, I really enjoyed the after lunch
> session: quick demo about tools.
> Janneke showed how they uses Emacs as Window Manager.
> Ricardo showed some tricks with git-worktree.
> Ludo showed guile-guix stuff---I do not remember well.
>
> Reproducible idea? ;-)

Definitely.  That’s the so-called “skill share” session inspired by the
R-B Summit as you know Pierre ;-).  We could have one of these and Next
would be a good fit for that.

Ludo’.



Re: GNU Guix Days before FOSDEM

2019-01-05 Thread Ludovic Courtès
Hello,

Pjotr Prins  skribis:

> We have an exciting program for the minimalism devroom at FOSDEM, see 
>
>   https://fosdem.org/2019/schedule/track/minimalistic_languages/
>
> We also have two GNU Guix days before FOSDEM. The idea is to break out
> into working groups during those two days. I have added a few ideas
> here:
>
>   https://libreplanet.org/wiki/Group:Guix/FOSDEM2019#Programme
>
> Feel free to add your favourite topics! If you want to attend be sure
> to add your name. We can't allow more than 40 people.

Awesome, looking forward to this year’s FOSDEM!

Ludo’.



Re: guile 3 update, september edition

2018-09-18 Thread Ludovic Courtès
Hello,

Andy Wingo  skribis:

> On Mon 17 Sep 2018 11:35, l...@gnu.org (Ludovic Courtès) writes:
>
>>> The threshold at which Guile will automatically JIT-compile is set from
>>> the GUILE_JIT_THRESHOLD environment variable.  By default it is 5.
>>> If you set it to -1, you disable the JIT.  If you set it to 0, *all*
>>> code will be JIT-compiled.  The test suite passes at
>>> GUILE_JIT_THRESHOLD=0, indicating that all features in Guile are
>>> supported by the JIT.  Set the GUILE_JIT_LOG environment variable to 1
>>> or 2 to see JIT progress.
>>
>> Just to be clear, does GUILE_JIT_THRESHOLD represents the number of
>> times a given instruction pointer is hit?
>
> No.  It is an abstract "hotness" counter associated with a function's
> code.  (I say "function's code" because many closures can share the same
> code and thus the same counter.  It's not in the scm_tc7_program object
> because some procedures don't have these.)
>
> All counters start at 0 when Guile starts.  A function's counters
> increment by 30 when a function is called, currently, and 2 on every
> loop back-edge.  I have not attempted to tweak these values yet.

OK, I see.

Exciting times!

Ludo’.



Re: guile 3 update, september edition

2018-09-17 Thread Ludovic Courtès
Hello!

Andy Wingo  skribis:

> This is an update on progress towards Guile 3.  In our last update, we
> saw the first bits of generated code:
>
>   https://lists.gnu.org/archive/html/guile-devel/2018-08/msg5.html
>
> Since then, the JIT is now feature-complete.  It can JIT-compile *all*
> code in Guile, including delimited continuations, dynamic-wind, all
> that.  It runs automatically, in response to a function being called a
> lot.  It can also tier up from within hot loops.

Woohoo!  It’s awesome that JIT can already handle all Guile code and run
all the test suite.  To me that means it can be merged into ‘master’.

> The threshold at which Guile will automatically JIT-compile is set from
> the GUILE_JIT_THRESHOLD environment variable.  By default it is 5.
> If you set it to -1, you disable the JIT.  If you set it to 0, *all*
> code will be JIT-compiled.  The test suite passes at
> GUILE_JIT_THRESHOLD=0, indicating that all features in Guile are
> supported by the JIT.  Set the GUILE_JIT_LOG environment variable to 1
> or 2 to see JIT progress.

Just to be clear, does GUILE_JIT_THRESHOLD represents the number of
times a given instruction pointer is hit?

> For debugging (single-stepping, tracing, breakpoints), Guile will fall
> back to the bytecode interpreter (the VM), for the thread that has
> debugging enabled.  Once debugging is no longer enabled (no more hooks
> active), that thread can return to JIT-compiled code.

Cool.

> Using GNU Lightning has been useful but in the long term I don't think
> it's the library that we need, for a few reasons:

[...]

It might be that the lightning 1.x branch would be a better fit (it was
exactly as you describe.)  I think that’s what Racket was (is?) using.

> Meaning that "eval" in Guile 3 is somewhere around 80% faster than in
> Guile 2.2 -- because "eval" is now JIT-compiled.

Very cool.

> Incidentally, as a comparison, Guile 2.0 (whose "eval" is slower for
> various reasons) takes 70s real time for the same benchmark.  Guile 1.8,
> whose eval was written in C, takes 4.536 seconds real time.  It's still
> a little faster than Guile 3's eval-in-Scheme, but it's close and we're
> catching up :)

It’s an insightful comparison; soon we can say it’s “as fast as
hand-optimized C” (and more readable, too :-)).

> I have also tested with ecraven's r7rs-benchmarks and we make a nice
> jump past the 2.2 results; but not yet at Racket or Chez levels yet.  I
> think we need to tighten up our emitted code.  There's another 2x of
> performance that we should be able to get with incremental improvements.
> For the last bit we will need global register allocation though, I
> think.

Looking forward to reading ecraven’s updated benchmark results.

Thank you for the awesomeness!

Ludo’.



Re: FOSDEM 2019

2018-08-29 Thread Ludovic Courtès
Heya,

Christopher Lemmer Webber  skribis:

> I added a talk topic: "A Guiler's Year of Racket".  I think there's a
> lot we could learn from our fellow friends in Racket-land, and I'd like
> to share some of that!

I’d be very interested in listening to it!

Ludo’.



GNU Guix Days before FOSDEM

2018-08-24 Thread Ludovic Courtès
Hello!

Pjotr Prins  skribis:

> If you intend to come and/or want to speak please add your name and
> proposed title to the new page at
>
>   https://libreplanet.org/wiki/Group:Guix/FOSDEM2018
>
> Alternatively, reply here or mail me privately so we can keep a tally.

The real URL is:

  https://libreplanet.org/wiki/Group:Guix/FOSDEM2019#GNU_Guix_Days

I copied and adjusted the text from last year’s page.

Anyway, feel free to add your name if you would come, and ideas of talks
or hacking sessions we may have, so we can start planning!

Thanks,
Ludo’.



Re: bug#22608: Module system thread unsafety and .go compilation

2018-07-03 Thread Ludovic Courtès
Hello,

taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis:

> To speed up the compilation of the many Scheme files in Guix, we use a
> script that first loads all modules to be compiled into the Guile
> process (by calling 'resolve-interface' on the module names), and then
> the corresponding Scheme files are compiled in a par-for-each.
>
> While Guile's module system is known to be thread unsafe, the idea was
> that all mutation should happen in the serial loading phase, and the
> parallel compile-file calls should then be thread safe.
>
> Sadly that assumption isn't met when autoloads are involved.

For the record, these issues should be fixed in Guile 2.2.4:

533e3ff17 * Serialize accesses to submodule hash tables.
46bcbfa56 * Module import obarrays are accessed in a critical section.
761cf0fb8 * Make module autoloading thread-safe.

‘guix pull’ now defaults to 2.2.4, so we’ll see if indeed those crashes
disappear.

Ludo’.



Re: crashes with Fibers

2018-07-02 Thread Ludovic Courtès
Hello Clément,

Clément Lassieur  skribis:

> ;; bad
> (define (test4)
>   (run-fibers
>(lambda ()
>  (spawn-fiber
>   (lambda ()
> (let ((channel (make-channel)))
>   (call-with-new-thread
>(lambda ()
>  (put-message channel "hello world")))
>#:drain? #t))
> ⊣ scheme@(guile-user)> In 
> /home/clement/.guix-profile/share/guile/site/2.2/fibers/internal.scm:
>   402:6  1 (suspend-current-fiber _)
>   In unknown file:
>  0 (scm-error misc-error #f "~A" ("Attempt to suspend fiber 
> within continuation barrier") #f)
>   ERROR: In procedure scm-error:
>   Attempt to suspend fiber within continuation barrier

I think the problem here is that the new thread inherit the dynamic
environment of the spawning thread.  Thus, ‘put-message’, called in that
new thread, thinks it’s running within a Fiber, but it’s not.

Because of that, ‘put-message’ tries to suspend itself, but it cannot:
‘call-with-new-thread’ is written in C, so it’s a “continuation barrier”
(meaning that it’s a continuation that cannot be captured and resumed
later.)

So I think if you really want that, you can perhaps do something like
(untested):

  (call-with-new-thread
(lambda ()
  (parameterize ((current-fiber #f))
(put-message channel "hello world"

HTH!
Ludo’.




Re: guile 3 update, june 2018 edition

2018-07-02 Thread Ludovic Courtès
Hello!

Andy Wingo  skribis:

> The news is that the VM has been completely converted over to call out
> to the Guile runtime through an "intrinsics" vtable.  For some
> intrinsics, the compiler will emit specialized call-intrinsic opcodes.
> (There's one of these opcodes for each intrinsic function type.)  For
> others that are a bit more specialized, like the intrinsic used in
> call-with-prompt, the VM calls out directly to the intrinsic.
>
> The upshot is that we're now ready to do JIT compilation.  JIT-compiled
> code will use the intrinsics vtable to embed references to runtime
> routines.  In some future, AOT-compiled code can keep the intrinsics
> vtable in a register, and call indirectly through that register.

Exciting!  It sounds like a really good strategy because it means that
the complex instructions don’t have to be implemented in lightning
assembly by hand, which would be a pain.

> My current plan is that the frame overhead will still be two slots: the
> saved previous FP, and the saved return address.  Right now the return
> address is always a bytecode address.  In the future it will be bytecode
> or native code.  Guile will keep a runtime routine marking regions of
> native code so it can know if it needs to if an RA is bytecode or native
> code, for debugging reasons; but in most operation, Guile won't need to
> know.  The interpreter will tier up to JIT code through an adapter frame
> that will do impedance matching over virtual<->physical addresses.  To
> tier down to the interpreter (e.g. when JIT code calls interpreted
> code), the JIT will simply return to the interpreter, which will pick up
> state from the virtual IP, SP, and FP saved in the VM state.

What will the “adapter frame” look like?

> We do walk the stack from Scheme sometimes, notably when making a
> backtrace.  So, we'll make the runtime translate the JIT return
> addresses to virtual return addresses in the frame API.  To Scheme, it
> will be as if all things were interpreted.

Currently you can inspect the locals of a stack frame.  Will that be
possible with frames corresponding to native code? (I suppose that’d be
difficult.)

> My current problem is knowing when a callee has JIT code.  Say you're in
> JITted function F which calls G.  Can you directly jump to G's native
> code, or is G not compiled yet and you need to use the interpreter?  I
> haven't solved this yet.  "Known calls" that use call-label and similar
> can of course eagerly ensure their callees are JIT-compiled, at
> compilation time.  Unknown calls are the problem.  I don't know whether
> to consider reserving another word in scm_tc7_program objects for JIT
> code.  I have avoided JIT overhead elsewhere and would like to do so
> here as well!

In the absence of a native code pointer in scm_tc7_program objects, how
will libguile find the native code for a given program?

Thanks for sharing this plan!  Good times ahead!

Ludo’.




GNU Guile 2.2.4 released

2018-07-02 Thread Ludovic Courtès
ristopher Lemmer Webber, Jan Smydke, and
Shea Levy who all contributed to this release.

Happy hacking with Guile,

Andy Wingo, Ludovic Courtès, and Mark H Weaver.


signature.asc
Description: PGP signature


  1   2   3   4   5   6   7   8   9   10   >