Re: GNU Guile 2.1.7 released (beta)

[David Pirotte]

Hi Andy,

> So!  Release blockers.
> ...

Not a blocker, at all, but I was thinking to this, wrt manipulating (very) large
vectors, arrays, lists ...

-]  repl - truncated-print

Right now I edit the installed (system repl common), and wrote a tip in 
Guile-CV's
manual so users can do that as well:  less then optimal :).  It would be nice to
provide an option, so users could set it 'just like that', in the repl, or as a
global config in their .guile (I did see lloda does that in his for arrays, but 
it's
not an obvious option to set, it is a 'sophisticated' little piece of code (for 
an
end-user at least)).

-]  error(s) while manipulating (very) large vectors or arrays

Unlike the above, it appears there is currently no way to have error (the 
procedure)
and raised exceptions in general, to use truncated-print, it would be cool to 
'link'
the above option so error reports use it as well.


WDYT?
David


pgpOn48Jc_LK4.pgp
Description: OpenPGP digital signature

Re: Guile benchmark

[Chris Vine]

On Mon, 27 Feb 2017 21:00:54 +0100
Andy Wingo  wrote:
> Hi,
> 
> On Thu 26 Jan 2017 09:39, Rchar  writes:
> 
> > I wanted to compare Guile scheme to other scheme implementations
> > and I found
> > this:https://ecraven.github.io/r7rs-benchmarks/benchmark.html
> >
> > Is Guile slow or fast, comparing to others?  
> 
> Besides what Mike said, there are two kinds of Schemes in that
> benchmark: the ones that compile to native machine code (ahead of
> time, either with their own compilers or by emitting C, or
> just-in-time), and the ones that run some kind of bytecode
> interpreter.
> 
> Schemes that compile to native code go faster.  Guile compiles to
> bytecode right now, so it's generally (though not always!) slower than
> the native-compiling schemes.  But compared to the
> bytecode-interpreter schemes it's pretty fast.
> 
> One day we'll have a native compiler and we can start playing
> benchmark games with the big kids :)

On reading this, out of interest I wrote a very simple program solving
primes, using the basic 'seive odd divisors to square root' algorithm.
I tested guile-2.0, guile-2.2, chicken and chez scheme on it.

guile-2.0 was a little slower than chicken with chicken compiled to C,
but guile-2.2 on that test took about 75% of the time of chicken, and
about 50% of the time of guile-2.0.  chez scheme was fastest of all,
taking about 50% of the time of chicken.  OK, chicken may not be the
fastest of "compile to C" schemes.

Having said that, all of them were considerably slower than the same
code written in native C++, probably because the compiler optimizes
better (it was about 5 times faster than chez scheme).  The performance
of clang and gcc was pretty much the same.

Solving primes is probably not particularly representative and I didn't
spend any more time with other benchmarks, so take it as you will. The
performance of guile-2.2 at run time was encouraging however.

Chris

Re: Using '-1' in a method named '*'

[Alejandro Sanchez]

> It could be that this is totally the wrong thing.  Is (* x 2) -> (+ x x)
> a valid transformation if you don't know the type of x?  I don't know.
Only if both addition and multiplication are defined for the type. In a 
group-like structure there is only one operation, while ring-like structures 
have two (usually called addition and multiplication). All the numeric types 
included in Guile are ring-like, so that works fine. Outside of that all bets 
are off. What would transforming (* -1 v) to (- 0 v) mean? You cannot add or 
subtract scalars and vectors, unless you were to embed both into a field where 
they are both defined, like the quaternion field. Or you would have to use the 
zero-vector as the value of 0.

> WDYT?  We could remove this transformation, or only apply it when type
> inference has run.
I don’t know. I guess there must be a good reason why Guile does these 
transformations, all I know is that they are not guaranteed to make sense 
outside Guile’s numeric types. Maybe it was my mistake for trying to make 
methods called * or + instead of making up my own names. The reason why I used 
those names (aside from simplicity) is that in those case it does make sense to 
mix multiplication of different types. I can write (* 2 3 q 4 5) where q is a 
quaternion and it will apply the correct operation to all operands, just as if 
I had written it on paper. It also allows me to use any type that implements * 
as the component of a dual number and have it work.

If you can stop Guile from re-writing code when it is not certain that the 
rewritten code makes sense it would be great. Pretty much every test that I 
have written over the past few days was broken even though my math was correct.

Re: guile can't find a chinese named file

[Jan Wedekind]

Hi :)

On Mon 27 Feb 2017 17:07, Eli Zaretskii  writes:


From: Andy Wingo 
Date: Sun, 26 Feb 2017 22:20:31 +0100

In Scheme, strings are sequences of characters.  Encoding and decoding
is only needed when going to and from bytes.  Guile supports a finite
number of encodings, so in general some encoding/decoding will always be
needed.  The specific encoding may change over time.


The lesson of Emacs development is that there's a need for
"characters" that represent raw bytes which cannot be decoded into the
internal representation, for whatever reasons.  These special
"characters" need to be representable in strings, among "normal"
recognizable characters (and thus distinguishable from the latter
kind), and they need to be converted back to their single-byte form
when the string is output to the external world.  An implementation of
text that doesn't include these features will always fail to support
some important use cases.


Thanks for this note (and upthread).  I didn't know Emacs settled on
this strategy.  It could fit in as a new "conversion strategy" (see
Encoding in the manual).

I think this feature will probably slip for 2.2.0 for lack of time,
though.  When someone does go to look at it, this thread is a useful
resource, or parts of it anyway :) I especially appreciated the
tradeoffs between surrogates and strange UTF-8 hacks.

Andy





The encoding support of the Ruby programming language [1] is IMHO pretty 
good. It can handle different encodings for source code, input/output, 
string variables, and regular expressions. UTF-8 is the preferred encoding 
but other encodings are required. E.g. Ruby is used a lot in Japan and 
there are many "Kanji" which are currently not covered by UTF-8.


[1] http://nuclearsquid.com/writings/ruby-1-9-encodings/

Re: guile can't find a chinese named file

[Eli Zaretskii]

> Date: Mon, 27 Feb 2017 20:24:19 + (GMT)
> From: Jan Wedekind 
> cc: Eli Zaretskii , guile-user@gnu.org
> 
> The encoding support of the Ruby programming language [1] is IMHO pretty 
> good. It can handle different encodings for source code, input/output, 
> string variables, and regular expressions. UTF-8 is the preferred encoding 
> but other encodings are required. E.g. Ruby is used a lot in Japan and 
> there are many "Kanji" which are currently not covered by UTF-8.

Emacs solves the latter problem as well, by using codepoints beyond
the end of the Unicode range.  (Don't forget that the Emacs m17n
features were designed and implemented by people who came from Japan.)
The advantage of the Emacs solution is that the internal
representation is still (a superset of) UTF-8, even though the byte
sequences for these codepoints could be longer than the maximum of the
standard UTF-8.

Re: Stack traces

[Amirouche]

Le 27/02/2017 à 21:23, Andy Wingo a écrit :

On Sat 18 Feb 2017 20:59, Amirouche  writes:


How do you access variables in the REPL?

,locals

Andy

It doesn't display something that I can use.

Re: GNU Guile 2.1.7 released (beta)

[Andy Wingo]

On Mon 27 Feb 2017 20:32, Mike Gran  writes:

> A C++ STL container holds a set of STL-allocated (non-GC allocated)
> structs.
>
> Those STL-allocated structs are also used as the payloads of
> SCM foreign objects.
>
> STL destruction can free those objects.  That free
> should not allow SCM foreign objects that continue to exist but
> contain junk payloads.
>
> Also should SCM GC free the SCM foreign objects, this GC
> should not cause the non-GC-allocated payload to be freed.

I believe this issue was fixed in
8dff3af087c6eaa83ae0d72aa8b22aef5c65d65d in Guile 2.0 and in a related
commit in 2.2.

Andy

Re: Guile benchmark

[Andy Wingo]

Hi,

On Thu 26 Jan 2017 09:39, Rchar  writes:

> I wanted to compare Guile scheme to other scheme implementations and I found 
> this:https://ecraven.github.io/r7rs-benchmarks/benchmark.html
>
> Is Guile slow or fast, comparing to others?

Besides what Mike said, there are two kinds of Schemes in that
benchmark: the ones that compile to native machine code (ahead of time,
either with their own compilers or by emitting C, or just-in-time), and
the ones that run some kind of bytecode interpreter.

Schemes that compile to native code go faster.  Guile compiles to
bytecode right now, so it's generally (though not always!) slower than
the native-compiling schemes.  But compared to the bytecode-interpreter
schemes it's pretty fast.

One day we'll have a native compiler and we can start playing benchmark
games with the big kids :)

Andy

Re: debugging help: how to read/use a backtrace?

[Andy Wingo]

On Sat 21 Jan 2017 11:21, Jan Nieuwenhuizen  writes:

> I often find myself struggling to pinpoint an error location from
> Guile's backtrace (see below) and I am starting to wonder if there is
> something that I'm missing.

I believe this is comprehensively cleaned up and improved in 2.1.x.
(Guile 2.0 tries to identify the procedure by looking at slot 0; Guile
2.2 instead uses the instruction pointer of the frame.)  Can you
confirm?

Andy

Re: C-c in guile

[Andy Wingo]

On Sat 21 Jan 2017 12:55, Alex Vong  writes:

> However, when running the external program "yes" in guile,
>
>   $ guile -c '(system* "yes")'
>
> We cannot terminate the process by pressing C-c,

Indeed I can reproduce this!

> but we can suspend it by pressing C-z.

I believe the C-z is handled by the shell, not Guile; AFAIU.

I don't know how this should work (mechanics) but I agree with you that
probably it should!

Andy

Re: guile can't find a chinese named file

[Andy Wingo]

Hi :)

On Mon 27 Feb 2017 17:07, Eli Zaretskii  writes:

>> From: Andy Wingo 
>> Date: Sun, 26 Feb 2017 22:20:31 +0100
>> 
>> In Scheme, strings are sequences of characters.  Encoding and decoding
>> is only needed when going to and from bytes.  Guile supports a finite
>> number of encodings, so in general some encoding/decoding will always be
>> needed.  The specific encoding may change over time.
>
> The lesson of Emacs development is that there's a need for
> "characters" that represent raw bytes which cannot be decoded into the
> internal representation, for whatever reasons.  These special
> "characters" need to be representable in strings, among "normal"
> recognizable characters (and thus distinguishable from the latter
> kind), and they need to be converted back to their single-byte form
> when the string is output to the external world.  An implementation of
> text that doesn't include these features will always fail to support
> some important use cases.

Thanks for this note (and upthread).  I didn't know Emacs settled on
this strategy.  It could fit in as a new "conversion strategy" (see
Encoding in the manual).

I think this feature will probably slip for 2.2.0 for lack of time,
though.  When someone does go to look at it, this thread is a useful
resource, or parts of it anyway :) I especially appreciated the
tradeoffs between surrogates and strange UTF-8 hacks.

Andy

Re: Using '-1' in a method named '*'

[Andy Wingo]

Hi,

On Mon 27 Feb 2017 11:06, Alejandro Sanchez  writes:

> (define v (make  #:x 1))
>
> (* -1 v)  ; Does not work
> (* -2 v)  ; Works fine

I believe that Guile is doing strength reduction, transforming (* -1 v)
to (- 0 v).

It could be that this is totally the wrong thing.  Is (* x 2) -> (+ x x)
a valid transformation if you don't know the type of x?  I don't know.

I think there's currently an assumption that if you extend *, that you
will do so in a mathy way, and that you implement - + and similar.  But
in this case it's not the clear right thing to do.

WDYT?  We could remove this transformation, or only apply it when type
inference has run.

Andy

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Andy Wingo 
> Cc: Chris Vine ,  guile-user@gnu.org
> Date: Sun, 26 Feb 2017 21:58:00 +0100
> 
> On Wed 15 Feb 2017 18:07, Eli Zaretskii  writes:
> 
> > the [Emacs] MS-Windows port pretends towards Emacs internals that file
> > names are encoded in UTF-8, and shadows relevant system APIs that
> > accept or return file names, like fopen, opendir/readdir, stat,
> > etc. with its own versions that convert UTF-8 to and from UTF-16
> > before calling the real OS APIs.
> >
> > Once again, just use that experience, and maybe even some
> > infrastructure code.
> 
> FWIW we are up for good suggestions.  It's clear that file names (and
> command line arguments and environment variables) aren't handled ideally
> in Guile as they aren't fundamentally strings of characters in any
> particular encoding, and hence this class of bug.

Let me know what kind of suggestions would help.  E.g., if you need a
more detailed descriptions of how Emacs goes about these issues, I can
do that (guile-devel is probably a better place for that).

Using '-1' in a method named '*'

[Alejandro Sanchez]

Hello,

I have been trying to write a Guile library for 3D motion and I wrote a 
‘vector3’ class and added the method ‘*’ for scaling a vector. This works fine 
for any scalar, except for ‘-1’. Here is a minimal code example, I am using 
Guile 2.0.14 on OS X 10.11.6:

(use-modules (oop goops))

(define-class  ()
  ;; Only one slot, for simlicity
  (x #:init-value 0 #:getter get-x #:init-keyword #:x))

(define-method (*  (n ) (v ))
  ;; Return anything, doesn't matter
  3)


(define v (make  #:x 1))

(* -1 v)  ; Does not work
(* -2 v)  ; Works fine


The error message is:

:14:0: In procedure #:14:0 
()>:
:14:0: In procedure -: Wrong type argument in position 1: 
#< 104ccdce0>

The backtrace is:

 16:0  0 (#:16:0 ()>)

This issue only happens when the argument is -1, it doesn’t happen for other 
negative integers and it doesn’t happen for ‘-1.0’ either. And it’s not just 
the literal ‘-1’, if instead I use an expression that evaluates to ‘-1’ like 
‘(- 2 3)’ the same happens. However, changing the order of arguments in the 
definition of the method does work.

The reason for this seems that the expression ‘(* -1 x)’ for any x is 
re-written by Guile to ‘(- x)’, because by adding the following method it gets 
working:

(define-method (- (v ))
  2)

However, since '(* -1 v)’ gets re-written to ‘(- v)’ the result is ‘2' instead 
of ‘3’. Granted, this is a contrived example and in any scenario I can think of 
'(* -1 v)’ and ‘(- v)’ would yield the same result, but is this intended 
behaviour? It was something that got me hung up over the last few days quite a 
lot (I was already in the process of writing this email when it occurred to me 
to define the ‘-‘ method).

Re: guile can't find a chinese named file

[Andy Wingo]

On Mon 27 Feb 2017 13:09, David Kastrup  writes:

> Andy Wingo  writes:
>> I seriously invite you to read the fine manual, specifically the first
>> four subsections of this node:
>>
>>   
>> https://www.gnu.org/software/guile/docs/master/guile.html/Input-and-Output.html
>
> ...somewhat unlikely that my thoughts are merely the outcome of
> incompetency.

Me suggesting that you read the manual is not suggesting you are
incompetent.  I would appreciate you not reading more into my words than
what I wrote.  I merely invite you to read the manual, especially since
has been updated in 2.2.  It explains clearly what Guile ports are and
are not.  This part of the manual was less clear in the past.

You seem to be specifically under the misconception that binary I/O on
string ports is impossible or not useful.  This is a misunderstanding on
your part.

Andy

Re: guile can't find a chinese named file

[David Kastrup]

Andy Wingo  writes:

> On Mon 27 Feb 2017 10:10, David Kastrup  writes:
>
>>> String ports have nothing to do with the discussion AFAIU.  (Ports in
>>> Guile are sequences of bytes also.  They may be accessed using
>>> textual interfaces as well.
>>
>> They can _only_ be accessed using textual interfaces.  They are
>> character-in/character-out.
>
> You misunderstand what Guile ports are.

The topic was "string ports".  String ports and soft ports operate on
characters and strings in Guile's encoding which makes a separate
reencoding pass both error-prone as well as inefficient.  In particular
since one use of string ports is reading sexps which involves frequent
peek/unget operations for which Guile reencodes characters only to
decode them right again on the next read.

> I seriously invite you to read the fine manual, specifically the first
> four subsections of this node:
>
>   
> https://www.gnu.org/software/guile/docs/master/guile.html/Input-and-Output.html

The number of errors I reported with regard to Guile's string and port
handling (and which ultimately got fixed) as well as the fact that I did
all of the low-level work for migrating probably the largest existing
Guile-based application from Guile-1.8 to Guile-2.0 makes it somewhat
unlikely that my thoughts are merely the outcome of incompetency.

Where in

(with-output-to-string
  (format #t "~s\n" (make-list 42)))

do you see an encoding inherent?  From the Scheme side of things, it is
characters and strings which are involved here exclusively.  Reencoding
into an external coding system does not make sense here.  The situation
is similar for soft ports.

-- 
David Kastrup

Re: guile can't find a chinese named file

[Andy Wingo]

Hello,

On Mon 27 Feb 2017 10:10, David Kastrup  writes:

> Andy Wingo  writes:
>
>> Legacy programs don't use codepoints >255.
>
> Sort of a moot point when Guile makes the decision to interpret external
> files with codepoints >255.  Not every data processed by a "legacy
> program" originates from inside the program.

Not a moot point at all.  If you want to decode/encode characters
to/from ports, you have to call Guile's setlocale function; that's a
choice you can make.  In Guile 1.8 and earlier regardless you would just
get ISO-8859-1 one-character-per-byte, so no significant change here.
If you would prefer to continue to use this encoding with every port in
your program, you can do that.

>> In Scheme, strings are sequences of characters.  Encoding and decoding
>> is only needed when going to and from bytes.
>
> A string port is strictly passing characters to characters completely
> inside of Guile

This is an implementation concern.  May I remind you and the list that
we have kindly asked you to not post to guile-devel because
implementation discussions with you are not productive.  I'm not
interested in having similar discussions, only on another list.  Thanks.

>>> PostScript files are usually encoded in Latin-1 with occasional UCS-16
>>> passages.  Reading and writing and copying such files byte-correctly
>>> while trying to actually parse their contents is not feasible with
>>> Guile.
>>
>> Works perfectly well.  The web server for example reads the request as
>> Latin-1 and the body as something else.  Just re-set the port encoding
>> and there you go.
>
> Reading and writing and copying cannot always afford to _parse_ and
> switch encodings based on the content.  It needs to work even when you
> don't do that.

If you would like to read just the bytes and parse yourself, you can do
that too.  Re-setting the encoding while parsing from a port can often
be more efficient though, as you don't have to read all of the data and
then parse it all; you can parse incrementally.

>> String ports have nothing to do with the discussion AFAIU.  (Ports in
>> Guile are sequences of bytes also.  They may be accessed using
>> textual interfaces as well.
>
> They can _only_ be accessed using textual interfaces.  They are
> character-in/character-out.

You misunderstand what Guile ports are.  I seriously invite you to read
the fine manual, specifically the first four subsections of this node:

  
https://www.gnu.org/software/guile/docs/master/guile.html/Input-and-Output.html

Thanks,

Andy

Re: guile can't find a chinese named file

[David Kastrup]

Andy Wingo  writes:

> Hello,
>
> I feel the need to correct points in this mail for the benefit of
> guile-user.  No reply is needed.
>
> On Wed 15 Feb 2017 00:58, David Kastrup  writes:
>
>> Mike Gran  writes:
>>
>>> But, for what it is worth, the Latin-1/UCS-32 design decision came
>>> from a couple of conflicting requirements.  The switch happened in the
>>> 1.9.x series.
>>>
>>> There was several examples of legacy C code using Guile for an
>>> extension language that accessed the bytes of a string directly, using
>>>
>>> SCM_STRING_CHARS or scm_i_string_chars.  To keep from breaking legacy
>>> code, we needed to retain the capability to use this (then already
>>> deprecated) capability to have C programs access 8-bit-locale string
>>> internals directly.
>>
>> But if you don't know whether the strings are Latin-1 or UCS-32, that's
>> sort of academical.
>
> Not at all.  Legacy programs don't use codepoints >255.

Sort of a moot point when Guile makes the decision to interpret external
files with codepoints >255.  Not every data processed by a "legacy
program" originates from inside the program.

>> The problem is that Guile is _constantly_ required to recode strings
>> it is processing.  And to add insult to injury, it cannot do this
>> without data loss when its string encoding assumptions are wrong.
>
> In Scheme, strings are sequences of characters.  Encoding and decoding
> is only needed when going to and from bytes.

A string port is strictly passing characters to characters completely
inside of Guile and its data structures and yet it needs to encode and
decode from Latin-1/UCS-32 to UTF-8.  A string port is _explicitly_ not
a binary stream (there are special binary ports for that) but a
character sequence and yet Guile is encoding and decoding for working
with its own internal data.

And the string API contains only scm_from_utf8_string (which always
requires reencoding) for accessing the whole character set.  It isn't
named scm_decode_utf8_bytestream: its target conceptually is a _string_,
yet it is expensive to pass into Guile and back out and there is no
cheaper or more transparent mechanism available.

>> PostScript files are usually encoded in Latin-1 with occasional UCS-16
>> passages.  Reading and writing and copying such files byte-correctly
>> while trying to actually parse their contents is not feasible with
>> Guile.
>
> Works perfectly well.  The web server for example reads the request as
> Latin-1 and the body as something else.  Just re-set the port encoding
> and there you go.

Reading and writing and copying cannot always afford to _parse_ and
switch encodings based on the content.  It needs to work even when you
don't do that.

>> As I said: the problem is not the chosen internal representation.
>> The problem is that there is no API to access it, and it does not
>> even map to string ports.
>
> String ports have nothing to do with the discussion AFAIU.  (Ports in
> Guile are sequences of bytes also.

Which is exactly the problem.

> They may be accessed using textual interfaces as well.

They can _only_ be accessed using textual interfaces.  They are
character-in/character-out.

> Therefore a string port must have an associated encoding, to
> read/write the bytes.

Why does a pure character-in/character-out structure need an associated
encoding?  The semi-equivalent in Emacs are buffers (which have a
manipulation point where you can write/read but are also random-access,
so it's sort of a superset).  Buffers have an _internal_ encoding but it
isn't exposed and it is identical to strings' internal encodings.

In contrast, the internal encoding of Guile string ports _is_ exposed
since its positioning uses byte offsets rather than character offsets
and thus is not compatible with string addressing.

Emacs got rid of this catastrophic user interface mistake (responsible
for the last major wave of migration to its competitor XEmacs) in Emacs
20.3 or 20.4.  Buffers are only ever addressed using character positions
from Emacs Lisp.

It's just painful to see Guile go through all of the expensive mistakes
Emacs made 15 or 20 years ago, just at a tenth of the speed since
getting encodings wrong was seen as more of a deal-breaker with Emacs.

> But no error is possible for textual I/O with the default UTF-8
> encoding as all characters are representable.

But all bytes aren't.

> Encoding to UTF-8 is fast and space-efficient.)

There is a reason that LilyPond on Guile-2.0 runs slower by a factor
of 5 than on Guile-1.8, and the large costs associated with constant
string reencoding are definitely contributing.

-- 
David Kastrup