Re: MinGW open-process, take N

[Andy Wingo]

Greets :)

On Thu 14 Jul 2016 20:41, Eli Zaretskii  writes:

>> Just that spawn_child isn't the only way to get a PID.  getpid would be
>> the most obvious one, but reading a PID value over a socket or whatever
>> is also possible.  But then your code is now introducing not-quite-PIDs
>> as well.  What if an interface expects one but gets the other?
>
> I see your point, and I think I can fix this, together with the pid_t
> width issue in 64-bit build, if I introduce an array private to
> posix-w32.c that will hold the PIDs of all processes known to Guile
> for which waitpid did not yet return an exit code, and their
> corresponding process handles.  Then we can return an int to Scheme,
> and the w32 functions will get hold of the handle by looking up the
> PID in that array.
>
> How's that sound?  If you agree, I can work on this tomorrow.

That actually sounds really nice!  That way the PID-using functions will
all see proper PIDS and we also get the waitpid() behavior.

I would be happy to apply such a patch.  Thank you :-)

Andy

Re: MinGW open-process, take N

[Eli Zaretskii]

> From: Andy Wingo 
> Cc: m...@netris.org,  l...@gnu.org,  guile-devel@gnu.org
> Date: Thu, 14 Jul 2016 20:11:28 +0200
> 
> On Thu 14 Jul 2016 17:34, Eli Zaretskii  writes:
> 
> >> > The process ID is indeed an int, but my code hides a process handle
> >> > inside it.
> >> 
> >> If you don't mind my asking: why? :)
> >
> > Because that's the only way on Windows to make sure the process object
> > is kept around by the kernel.  Integer PIDs are reused very quickly on
> > Windows, so by the time you get to scm_waitpid or some other function
> > that wants to query the process, that PID might not exist, or even
> > name a different process.  Having a handle open on the process
> > prevents the process object from being recycled.  Also, _cwait, used
> > by the Windows emulation of waitpid, needs a handle.
> 
> Are you saying that Windows doesn't wait until you waitpid() on a PID to
> reap it?

If there's no handle open on a process, then when it exits, its object
is recycled and its PID can easily be reused (on a busy system), yes.

> >> The caller effectively just returns the PID to Scheme, at which
> >> point it has a not-quite-PID floating around.  Sounds like trouble
> >> to me.
> >
> > Not sure why you think it's trouble.  Can you explain?
> 
> Just that spawn_child isn't the only way to get a PID.  getpid would be
> the most obvious one, but reading a PID value over a socket or whatever
> is also possible.  But then your code is now introducing not-quite-PIDs
> as well.  What if an interface expects one but gets the other?

I see your point, and I think I can fix this, together with the pid_t
width issue in 64-bit build, if I introduce an array private to
posix-w32.c that will hold the PIDs of all processes known to Guile
for which waitpid did not yet return an exit code, and their
corresponding process handles.  Then we can return an int to Scheme,
and the w32 functions will get hold of the handle by looking up the
PID in that array.

How's that sound?  If you agree, I can work on this tomorrow.

GNU Guile 2.0.12 released

[Andy Wingo]

We are pleased to announce GNU Guile release 2.0.12, the next maintenance
release for the 2.0.x stable series.

Guile is an implementation of the Scheme programming language.  It is
designed to help programmers create flexible applications that can be
extended by users or other programmers with plug-ins, modules, or
scripts.  With Guile you can create applications and games for the
desktop, the Web, the command-line, and more.

See the lovely Guile web page at http://gnu.org/software/guile/, for
more information on Guile.

This release contains 245 commits by 23 people over more than two years,
and is essentially a maintenance release, maintaining full compatibility
with the 2.0 release series.  See the end of this mail for a full
description of user-visible changes.

In parallel the Guile development team has been hard at work on the next
stable series, which we hope will see a stable release within the next
couple months.  Inquisitive users should see the recent 2.1.3 release
notes at
https://lists.gnu.org/archive/html/guile-user/2016-06/msg00058.html for
a preview of our future stable series.



Here are the compressed sources:
  ftp://ftp.gnu.org/gnu/guile/guile-2.0.12.tar.gz   (7.1MB)
  ftp://ftp.gnu.org/gnu/guile/guile-2.0.12.tar.xz   (4.3MB)

Here are the GPG detached signatures[*]:
  ftp://ftp.gnu.org/gnu/guile/guile-2.0.12.tar.gz.sig
  ftp://ftp.gnu.org/gnu/guile/guile-2.0.12.tar.xz.sig

Use a mirror for higher download bandwidth:
  http://www.gnu.org/order/ftp.html

Here are the MD5 and SHA1 checksums:

666000e0842a81c81fbbb48c7c5c27ef  guile-2.0.12.tar.gz
081fdf80cd3a76f260a2a0d87f773d6b  guile-2.0.12.tar.xz
51eeedadb1e12be2b728afca0a8685f69cf55f04  guile-2.0.12.tar.gz
d3a4c11b9faa01e3da6d58cd0d64fcf26d937e63  guile-2.0.12.tar.xz

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify guile-2.0.12.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys 8812F8F2

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.69
  Automake 1.15
  Libtool 2.4.6
  Gnulib v0.1-800-g68b6ade
  Makeinfo 6.1



Changes in 2.0.12 (since 2.0.11):

* Notable changes

** FFI: Add support for functions that set 'errno'

When accessing POSIX functions from a system's libc via Guile's dynamic
FFI, you commonly want to access the 'errno' variable to be able to
produce useful diagnostic messages.

This is now possible using 'pointer->procedure' or
'scm_pointer_to_procedure_with_errno'.  See "Dynamic FFI" in the manual.

** The #!r6rs directive now influences read syntax

The #!r6rs directive now changes the per-port reader options to make
Guile's reader conform more closely to the R6RS syntax.  In particular:

  - It makes the reader case sensitive.
  - It disables the recognition of keyword syntax in conflict with the
R6RS (and R5RS).
  - It enables the `square-brackets', `hungry-eol-escapes' and
`r6rs-hex-escapes' reader options.

** 'read' now accepts "\(" as equivalent to "("

This is indented for use at the beginning of lines in multi-line strings
to avoid confusing Emacs' lisp modes.  Previously "\(" was an error.

** SRFI-14 character data set upgraded to Unicode 8.0.0

** SRFI-19 table of leap seconds updated

** 'string-hash', 'read-string', and 'write' have been optimized

** GOOPS bug fix for inherited accessor methods

In the port of GOOPS to Guile 2.0, we introduced a bug related to
accessor methods.  The bug resulted in GOOPS assuming that a slot S in
an object whose class is C would always be present in instances of all
subclasses C, and allocated to the same struct index.  This is not the
case for multiple inheritance.  This behavior has been fixed to be as it
was in 1.8.

One aspect of this change may cause confusion among users.  Previously
if you defined a class C:

  (use-modules (oop goops))
  (define-class C ()
(a #:getter get-a))

And now you define a subclass, intending to provide an #:init-value for
the slot A:

  (define-class D ()
(a #:init-value 42))

Really what you have done is define in D a new slot with the same name,
overriding the existing slot.  The problem comes in that before fixing
this bug (but not in 1.8), the getter 'get-a' would succeed for
instances of D, even though 'get-a' should only work for the slot 'a'
that is defined on class C, not any other slot that happens to have the
same name and be in a class with C as a superclass.

It would be possible to "merge" the slot definitions on C and D, but
that part of the meta-object protocol (`compute-slots' et al) is not
fully implemented.

Somewhat 

Re: [PATCH] Re: Patchset related to array functions

[Andy Wingo]

On Thu 14 Jul 2016 17:41, Daniel Llorens  writes:

> On 14 Jul 2016, at 11:46, Andy Wingo  wrote:
>
>>  (1) Can we support C99 on all targets we care about?
>
> Emacs

http://git.savannah.gnu.org/cgit/emacs.git/tree/configure.ac#n764

"Emacs needs C99".  Sweet!  We check this point off.

>>  (2) Can we use C99 in our public interface, or just internally?  If we
>>  use it publically, what should we change?  No more scm_t_uint8 I
>>  hope, besides for back-compat?  This patch set does not have to
>>  include these changes, but we should have a plan.
>
> I think we'd want C89/C90 users to still be able to #include . 
> Dunno.

Really?  I would *love* to be able to say "just use c99, or at least
something with stdint.h".  Apparently gcc's default has been gnu11 for a
while...

>>  (3) Most importantly, what is the impact on inlining?  See the comment
>>  in __scm.h around line 165.
>
> Apparently the standard practice in C99 is to put inline definition in
> the header and extern declaration in the .c, while with ‘Guile inline’
> both SCM_INLINE and SCM_INLINE_IMPLEMENTATION are in the header.

I believe that Guile tries to do this as well.  By default the headers
define inline definitions and there is the extern inline declaration,
and then inline.c re-includes those headers with
SCM_INLINE_C_IMPLEMENTING_INLINES defined which reifies the definitions
so that the symbols end up in the .so.  But this landscape is quite
gnarly.  The specific implementation actually relies on the gnu_inline
attribute, so I guess we are using GNU extensions either way, at least
when compiled with GCC...

> I can try to fix Guile to follow the C99 practice and remove most of
> the #define guards. Would a patch doing this be accepted? I'd welcome
> advice on how to test such a patch. E.g. with both -O2 and -O0 or
> so. I'm mostly a C++ programmer and I don't want to mess anything up.

I think the concerns are:

 (1) Do inlined definitions get inlined?
 (2) Are external definitions reified as well?
 (3) Do we avoid reifying definitions in each compilation unit?
 (4) Can you dlsym() an inline function?

All these answers should be yes.  No benchmarking needed, just
inspection of the build artifacts under different configurations.

>> If you want your patch set to depend on C99 that's fine, but you have to
>> answer these questions first for the project as a whole and get some
>> consensus.
>
> That is a very reasonable viewpoint. Since C99 was just a minor
> convenience to me, I withdraw that particular patch. I have rebased
> everything to avoid the C99 requirement. Revised patchset attached.

Tx, will review separately.  In the future would you mind please
spamming the list with these patches as a thread of multiple mails, as
git-send-email would do?  That makes it easy for me to review just one
patch, say LGTM or whatever on that patch, then work on other patches on
other days.  But I will make an initial pass on this mail, later though
:)

Andy

Re: MinGW open-process, take N

[Andy Wingo]

On Thu 14 Jul 2016 17:34, Eli Zaretskii  writes:

>> > The process ID is indeed an int, but my code hides a process handle
>> > inside it.
>> 
>> If you don't mind my asking: why? :)
>
> Because that's the only way on Windows to make sure the process object
> is kept around by the kernel.  Integer PIDs are reused very quickly on
> Windows, so by the time you get to scm_waitpid or some other function
> that wants to query the process, that PID might not exist, or even
> name a different process.  Having a handle open on the process
> prevents the process object from being recycled.  Also, _cwait, used
> by the Windows emulation of waitpid, needs a handle.

Are you saying that Windows doesn't wait until you waitpid() on a PID to
reap it?

>> The caller effectively just returns the PID to Scheme, at which
>> point it has a not-quite-PID floating around.  Sounds like trouble
>> to me.
>
> Not sure why you think it's trouble.  Can you explain?

Just that spawn_child isn't the only way to get a PID.  getpid would be
the most obvious one, but reading a PID value over a socket or whatever
is also possible.  But then your code is now introducing not-quite-PIDs
as well.  What if an interface expects one but gets the other?

Andy

Re: MinGW open-process, take N

[Eli Zaretskii]

> From: Andy Wingo 
> Cc: m...@netris.org,  l...@gnu.org,  guile-devel@gnu.org
> Date: Thu, 14 Jul 2016 12:20:09 +0200
> 
> On Tue 12 Jul 2016 16:46, Eli Zaretskii  writes:
> 
> >> But in reality the getuid is of this form:
> >> 
> >>   (define (load-user-init)
> >> (let* ((home (or (getenv "HOME")
> >>  (false-if-exception (passwd:dir (getpwuid (getuid
> >>  file-name-separator-string))  ;; fallback for cygwin 
> >> etc.
> >>(init-file (in-vicinity home ".guile")))
> >>   (if (file-exists? init-file)
> >>   (primitive-load init-file
> >> 
> >> So, no problem.
> >
> > Won't this fail to compile during boot, or at least produce a warning?
> 
> It will produce a warning at compile-time, yes, but no problem at
> run-time because of the false-if-exception block.

Sigh.

> > The process ID is indeed an int, but my code hides a process handle
> > inside it.
> 
> If you don't mind my asking: why? :)

Because that's the only way on Windows to make sure the process object
is kept around by the kernel.  Integer PIDs are reused very quickly on
Windows, so by the time you get to scm_waitpid or some other function
that wants to query the process, that PID might not exist, or even
name a different process.  Having a handle open on the process
prevents the process object from being recycled.  Also, _cwait, used
by the Windows emulation of waitpid, needs a handle.

> The caller effectively just returns the PID to Scheme, at which
> point it has a not-quite-PID floating around.  Sounds like trouble
> to me.

Not sure why you think it's trouble.  Can you explain?

Re: MinGW open-process, take N

[Andy Wingo]

On Tue 12 Jul 2016 16:46, Eli Zaretskii  writes:

>> But in reality the getuid is of this form:
>> 
>>   (define (load-user-init)
>> (let* ((home (or (getenv "HOME")
>>  (false-if-exception (passwd:dir (getpwuid (getuid
>>  file-name-separator-string))  ;; fallback for cygwin 
>> etc.
>>(init-file (in-vicinity home ".guile")))
>>   (if (file-exists? init-file)
>>   (primitive-load init-file
>> 
>> So, no problem.
>
> Won't this fail to compile during boot, or at least produce a warning?

It will produce a warning at compile-time, yes, but no problem at
run-time because of the false-if-exception block.

>> Hu, I don't know, there are a number of other things in that file
>> which treat PIDs as ints (scm_waitpid, scm_getppid, etc etc etc).  I
>> think that would need to be another patch that adds scm_from_pid_t and
>> scm_to_pid_t and so on.  And then what is this?
>> 
>>   http://permalink.gmane.org/gmane.comp.gnu.mingw.w64.general/12235
>
> The process ID is indeed an int, but my code hides a process handle
> inside it.

If you don't mind my asking: why? :)  The caller effectively just
returns the PID to Scheme, at which point it has a not-quite-PID
floating around.  Sounds like trouble to me.

Andy

Re: add command line option to quiet compiler messages

[Andy Wingo]

Hi :)

Thanks for the patch!

On Wed 13 Jul 2016 06:08, Tobin Harding  writes:

> While working on this I discovered that compile messages are output from two
> separate places (load.c and boot-9.scm). Each file contains identical strings
> for the messages. This goes against the rule of SPOT.

Sadly it has to be like this for now.  I do not have suggestions to fix
it; it's gnarly.

> -  scm_puts (";;; note: source file ", scm_current_warning_port ());
> -  scm_display (full_filename, scm_current_warning_port ());
> -  scm_puts ("\n;;;   newer than compiled ", scm_current_warning_port 
> ());
> -  scm_display (compiled_filename, scm_current_warning_port ());
> -  scm_puts ("\n", scm_current_warning_port ());
> +  if (scm_is_false (*scm_loc_quiet)) {
> + scm_puts (";;; note: source file ", scm_current_warning_port ());
> + scm_display (full_filename, scm_current_warning_port ());
> + scm_puts ("\n;;;   newer than compiled ", scm_current_warning_port 
> ());
> + scm_display (compiled_filename, scm_current_warning_port ());
> + scm_puts ("\n", scm_current_warning_port ());
> +  }
>  }

Please resend without tabs, please.  Thanks :)

>  
>  
> +;;; Don't know where to put these:
> +(define (cond_warn_compiling file)
> +  (unless %quiet
> +(format (current-warning-port) ";;; compiling ~a\n" file)))
> +
> +(define (cond_warn_compiled file)
> +  (unless %quiet
> +(format (current-warning-port) ";;; compiled ~a\n" file)))
> +
> +(define (cond_warn_newer new old)
> +  (unless %quiet
> +(format (current-warning-port)
> +";;; note: source file ~a\n;;;   newer than compiled ~a\n"
> +name go-file-name)))

These definitions are inappropriate for being at the top-level of
boot-9.scm -- they would become automatically a part of all
environments.  I would leave out this change.

Note also that the Guile convention is hyphen-case not snake_case.

> diff --git a/module/ice-9/command-line.scm b/module/ice-9/command-line.scm
> index 98d3855..9a3f7e1 100644
> --- a/module/ice-9/command-line.scm
> +++ b/module/ice-9/command-line.scm
> @@ -136,6 +136,7 @@ If FILE begins with `-' the -s switch is mandatory.
>--listen[=P]   listen on a local port or a path for REPL clients;
>   if P is not given, the default is local port 37146
>-q inhibit loading of user init file
> +  --quietinhibit compile and load messages
>--use-srfi=LS  load SRFI modules for the SRFIs in LS,
>   which is a list of numbers like \"2,13,14\"
>-h, --help display this help and exit

What do people think about the name of this command line option?

To me "inhibit compile and load messages" does not quite capture what it
does.  But maybe it is good enough.

Incidentally this change will need a documentation update as well.  I
often find that adding documentation makes it more clear how things
should be named -- that the process of explaining things makes it
apparent what things should be changed :)

> +   ((string=? arg "--quiet")
> +(set! %quiet #t)
> +(parse args out))
> +

Probably %quiet should be a parameter, and unless we plan on using it
for other purposes it should have a more specific name
(%verbose-auto-compilation-messages or so; no idea).

Or, you could always just (current-warning-port (%make-void-port "w"));
then the patch is super simple, no conditions needed.  That is what I
was originally thinking, but it has the disadvantage that you lose other
uses of the current warning port, but maybe that's an advantage too.

WDYT?

Andy

Re: Patchset related to array functions

[Andy Wingo]

Thanks for the update.

On Tue 12 Jul 2016 19:16, Daniel Llorens  writes:

> Subject: [PATCH 01/12] Compile in C99 mode

This could be a good change but it is not the fastest path to patch
review :)  There are three considerations here:

  (1) Can we support C99 on all targets we care about?

  (2) Can we use C99 in our public interface, or just internally?  If we
  use it publically, what should we change?  No more scm_t_uint8 I
  hope, besides for back-compat?  This patch set does not have to
  include these changes, but we should have a plan.

  (3) Most importantly, what is the impact on inlining?  See the comment
  in __scm.h around line 165.

If you want your patch set to depend on C99 that's fine, but you have to
answer these questions first for the project as a whole and get some
consensus.

Andy