Re: [PATCH] Re: Eggs not installing on msys-mingw32

2023-11-05 Thread Matt Welland
Well, I have no idea what is going on but I'm suspecting a hardware
problem. In my last reply I was confident the problem was reproducible but
as I continued to try things that proved to not be the case.

I switched to working with Chicken-5.3.0rc2 and the build there failed on
creation of modules.db, again it was very reliably reproducible. Thinking
this was maybe a race condition between the creation of the modules.db file
and the following file copy I added a little code to read the file back and
compare it. The problem went away. Unfortunately when I reverted the code
the problem did not return. That does not make any sense whatsoever.

Currently I'm able to get to where when I install the sqlite3 egg it fails
with:

Error: unterminated list

If I run the install a couple times it results in a working egg.

I'll keep chipping away at this but I don't think I have anything tangible
to offer in terms of a problem statement other than ... "Windows sucks".

On Tue, Oct 31, 2023 at 3:25 AM Peter Bex  wrote:

> On Mon, Oct 30, 2023 at 05:45:28PM -0400, Matt Welland wrote:
> > Well, maybe there is a silver lining here, I can currently reliably
> > reproduce the problem!
>
> That's excellent!  Do you have to do any special steps, or does it
> simply always happen?
>
> > One hypothesis I'm looking at is replacing calls using
> with-input-from-file
> > and with-output-to-file with open-*-file ... close loops for the reading
> > and creation of the egg-info files.
>
> You're probably on the wrong track there.  AFAICT, egg-info files are
> only *created* in the foo.install.sh / foo.install.bat file, at the very
> end where (on *nix) it uses cat(1) to write into it.
>
> If the file is already empty, changing how it's read won't make a
> difference.
>
> The only way it would fail that I can see is if one presses ^C to abort
> installation at a very unlucky time.  This sounds like a race condition
> but depending on the shell, it might be that it will process that signal
> after evaluating the line that redirects stdout to the egg-info file,
> but before actually invoking cat.  Dunno, grasping at straws here :)
>
> Since you're using mingw, it's a UNIX shell at least.  Could you try
> inserting a sync(1) call at the end of the .install.sh file to see if
> that fixes things?
>
> Cheers,
> Peter
>


-- 
--
Complexity is your enemy. Any fool can make something complicated.
It is hard to keep things simple. - Richard Branson.


Re: [PATCH] Re: Eggs not installing on msys-mingw32

2023-10-31 Thread Peter Bex
On Mon, Oct 30, 2023 at 05:45:28PM -0400, Matt Welland wrote:
> Well, maybe there is a silver lining here, I can currently reliably
> reproduce the problem!

That's excellent!  Do you have to do any special steps, or does it
simply always happen?

> One hypothesis I'm looking at is replacing calls using with-input-from-file
> and with-output-to-file with open-*-file ... close loops for the reading
> and creation of the egg-info files.

You're probably on the wrong track there.  AFAICT, egg-info files are
only *created* in the foo.install.sh / foo.install.bat file, at the very
end where (on *nix) it uses cat(1) to write into it.

If the file is already empty, changing how it's read won't make a
difference.

The only way it would fail that I can see is if one presses ^C to abort
installation at a very unlucky time.  This sounds like a race condition
but depending on the shell, it might be that it will process that signal
after evaluating the line that redirects stdout to the egg-info file,
but before actually invoking cat.  Dunno, grasping at straws here :)

Since you're using mingw, it's a UNIX shell at least.  Could you try
inserting a sync(1) call at the end of the .install.sh file to see if
that fixes things?

Cheers,
Peter


signature.asc
Description: PGP signature


Re: [PATCH] Re: Eggs not installing on msys-mingw32

2023-10-30 Thread Matt Welland
On Mon, Oct 30, 2023, 4:04 PM Mario Domenech Goulart 
wrote:

> Hi Matt,
>
> On Mon, 30 Oct 2023 09:43:00 -0400 Matt Welland 
> wrote:
>
> > On Mon, Oct 30, 2023 at 9:37 AM  wrote:
> >
> >  > How about
> >  > "empty egg-info file, possibly due to an aborted egg-install - please
> remove the file and reinstall the corresponding egg"?
> >  >
> >  > Rationale: Matt's case.  check-errors.egg-info was empty, but the
> >  > installation of base64 was failing.  If Matt had removed
> >  > check-errors.egg-info and retried installing base64, the installation
> of
> >  > base64 would work, but check-errors would still be broken
> >  > (chicken-status and chicken-uninstall would ignore it, for example).
> >
> >  Sure, whatever - feel free to change the message, if you think that
> makes it clearer.
> >
> > My reporting on this was incorrect. Sorry for creating the churn. I
> believe there is something going genuinely wrong with check-errors. One
> hypothesis
> > I have is that Windows is seeing the file name containing "errors" and
> doing something stupid and annoying. I am not experienced with Windows and
> have
> > no clue what that thing might be but I'll keep digging.
>
> Nothing to be sorry about.  I'm afraid this issue might not related to
> Windows, exclusively.  I took a look at IRC logs and and saw that we had
> at least four cases like that, two on Windows and two on Linux.  We have
> traces of three of those issues in paste.call-cc.org and in pastebin.com:
>
> *
> http://paste.call-cc.org/paste?id=476adbc8ef7b47d8a7ce9ccd7dc201d4737de7b7
> *
> http://paste.call-cc.org/paste?id=a7c9d3bd2dbc8837b02b418a33f6d7c5ca9d9455
> * https://pastebin.com/e2BdgGGC
>
> I think we have a bug somewhere.
>
> The eggs that triggered the problem (i.e., the ones that had their
> .egg-info file empty), according to the records we have, are:
>
> * apropos
> * check-errors
> * math
> * srfi-45
>

Well, maybe there is a silver lining here, I can currently reliably
reproduce the problem!

One hypothesis I'm looking at is replacing calls using with-input-from-file
and with-output-to-file with open-*-file ... close loops for the reading
and creation of the egg-info files. The reason is that I've seen problems
on creation/reading of files on NFS that *felt* like they were improved by
replacing the calls as described. I found where the egg-info was read but
not where it was created. Can you give me an idea as to where the egg-info
is being written or copied? It is a long shot but maybe worth a try.


> I've have pushed the patch submitted by Felix, as I think it improves
> the situation for users.  With it, users get a better error message and
> have at least a chance to work around the issue.
>
> I've created https://bugs.call-cc.org/ticket/1827 for that issue.
>
> All the best.
> Mario
> --
> http://parenteses.org/mario
>


Re: [PATCH] Re: Eggs not installing on msys-mingw32

2023-10-30 Thread Mario Domenech Goulart
Hi Matt,

On Mon, 30 Oct 2023 09:43:00 -0400 Matt Welland  wrote:

> On Mon, Oct 30, 2023 at 9:37 AM  wrote:
>
>  > How about
>  > "empty egg-info file, possibly due to an aborted egg-install - please 
> remove the file and reinstall the corresponding egg"?
>  > 
>  > Rationale: Matt's case.  check-errors.egg-info was empty, but the
>  > installation of base64 was failing.  If Matt had removed
>  > check-errors.egg-info and retried installing base64, the installation of
>  > base64 would work, but check-errors would still be broken
>  > (chicken-status and chicken-uninstall would ignore it, for example).
>
>  Sure, whatever - feel free to change the message, if you think that makes it 
> clearer.
>
> My reporting on this was incorrect. Sorry for creating the churn. I believe 
> there is something going genuinely wrong with check-errors. One hypothesis
> I have is that Windows is seeing the file name containing "errors" and doing 
> something stupid and annoying. I am not experienced with Windows and have
> no clue what that thing might be but I'll keep digging.

Nothing to be sorry about.  I'm afraid this issue might not related to
Windows, exclusively.  I took a look at IRC logs and and saw that we had
at least four cases like that, two on Windows and two on Linux.  We have
traces of three of those issues in paste.call-cc.org and in pastebin.com:

* http://paste.call-cc.org/paste?id=476adbc8ef7b47d8a7ce9ccd7dc201d4737de7b7
* http://paste.call-cc.org/paste?id=a7c9d3bd2dbc8837b02b418a33f6d7c5ca9d9455
* https://pastebin.com/e2BdgGGC

I think we have a bug somewhere.

The eggs that triggered the problem (i.e., the ones that had their
.egg-info file empty), according to the records we have, are:

* apropos
* check-errors
* math
* srfi-45

I've have pushed the patch submitted by Felix, as I think it improves
the situation for users.  With it, users get a better error message and
have at least a chance to work around the issue.

I've created https://bugs.call-cc.org/ticket/1827 for that issue.

All the best.
Mario
-- 
http://parenteses.org/mario



Re: [PATCH] Re: Eggs not installing on msys-mingw32

2023-10-30 Thread Matt Welland
On Mon, Oct 30, 2023 at 9:37 AM  wrote:

> > How about
> > "empty egg-info file, possibly due to an aborted egg-install - please
> remove the file and reinstall the corresponding egg"?
> >
> > Rationale: Matt's case.  check-errors.egg-info was empty, but the
> > installation of base64 was failing.  If Matt had removed
> > check-errors.egg-info and retried installing base64, the installation of
> > base64 would work, but check-errors would still be broken
> > (chicken-status and chicken-uninstall would ignore it, for example).
>
> Sure, whatever - feel free to change the message, if you think that makes
> it clearer.
>

My reporting on this was incorrect. Sorry for creating the churn. I believe
there is something going genuinely wrong with check-errors. One hypothesis
I have is that Windows is seeing the file name containing "errors" and
doing something stupid and annoying. I am not experienced with Windows and
have no clue what that thing might be but I'll keep digging.


>
> felix
>
>

-- 
--
Complexity is your enemy. Any fool can make something complicated.
It is hard to keep things simple. - Richard Branson.


Re: [PATCH] Re: Eggs not installing on msys-mingw32

2023-10-30 Thread felix . winkelmann
> How about
> "empty egg-info file, possibly due to an aborted egg-install - please remove 
> the file and reinstall the corresponding egg"?
> 
> Rationale: Matt's case.  check-errors.egg-info was empty, but the
> installation of base64 was failing.  If Matt had removed
> check-errors.egg-info and retried installing base64, the installation of
> base64 would work, but check-errors would still be broken
> (chicken-status and chicken-uninstall would ignore it, for example).

Sure, whatever - feel free to change the message, if you think that makes it 
clearer.


felix




Re: [PATCH] Re: Eggs not installing on msys-mingw32

2023-10-30 Thread Matt Welland
Copying the check-errors.egg-info file from another build into the
lib/chicken/11 directory seems to have genuinely let me move forward. Now
I'm getting a different error on the sqlite3 egg "The filename, directory
name, or volume label syntax is incorrect." The path it is complaining
about has a mix of forward and backward slashes. Installing sql-de-lite
worked fine so the sqlite3 problem is unrelated I think.

On Mon, Oct 30, 2023 at 9:21 AM Matt Welland  wrote:

> I think I was premature in concluding that removing the empty files was
> sufficient to fix this. I can remove the empty file but the moment I hit an
> egg that depends on check-errors it seems to get stuck.
>
> I turned off the continuous virus feature and I tried using
> CHICKEN_EGG_CACHE to put the cache in /tmp and each time it seems to go
> further but fails on check-errors.egg-info. Note that check-errors appears
> to be properly installed as it imports fine. I think something is
> attempting to remove or rewrite the check-errors.egg-info file and that
> causes the problem, at least that is currently my best guess.
>
> Unfortunately I don't think that stopping and reporting the empty file
> will be sufficient but I will try the patch. Any suggestions as to what to
> try next?
>
> On Mon, Oct 30, 2023 at 9:14 AM Mario Domenech Goulart <
> ma...@parenteses.org> wrote:
>
>> On Mon, 30 Oct 2023 14:01:44 +0100 felix.winkelm...@bevuta.com wrote:
>>
>> >> That was it. Thanks. I do have to keep running:
>> >>
>> >> find /software/ -name \*.egg-info -empty -delete -print
>> >>
>> >> as check-errors.egg-info seems to be a pervasive problem. Maybe it
>> would be
>> >> worth it to add a defence against this?
>> >
>> > Sorry - wrong patch, this is the right one.
>> >
>> >
>> > felix
>> >
>> >
>> > From 8a2a82a8171744546badacaa297bcf754bff3eba Mon Sep 17 00:00:00 2001
>> > From: felix 
>> > Date: Mon, 30 Oct 2023 14:00:41 +0100
>> > Subject: [PATCH] handle case when egg-info file is empty
>> >
>> > ---
>> >  egg-information.scm | 7 +--
>> >  1 file changed, 5 insertions(+), 2 deletions(-)
>> >
>> > diff --git a/egg-information.scm b/egg-information.scm
>> > index 6f5f7cb9..1e7f8aec 100644
>> > --- a/egg-information.scm
>> > +++ b/egg-information.scm
>> > @@ -34,8 +34,11 @@
>> >  (if (file-exists? fname*) fname* fname)))
>> >
>> >  (define (load-egg-info fname)
>> > -  (let ((fname (locate-egg-file fname)))
>> > -(with-input-from-file fname read)))
>> > +  (let* ((fname (locate-egg-file fname))
>> > + (info (with-input-from-file fname read)))
>> > +(if (eof-object? info)
>> > +(error "empty egg-info file, possibly due to an aborted
>> egg-install - please remove the file and retry" fname)
>> > +info)))
>>
>> How about
>> "empty egg-info file, possibly due to an aborted egg-install - please
>> remove the file and reinstall the corresponding egg"?
>>
>> Rationale: Matt's case.  check-errors.egg-info was empty, but the
>> installation of base64 was failing.  If Matt had removed
>> check-errors.egg-info and retried installing base64, the installation of
>> base64 would work, but check-errors would still be broken
>> (chicken-status and chicken-uninstall would ignore it, for example).
>>
>> All the best.
>> Mario
>> --
>> http://parenteses.org/mario
>>
>
>
> --
> --
> Complexity is your enemy. Any fool can make something complicated.
> It is hard to keep things simple. - Richard Branson.
>


-- 
--
Complexity is your enemy. Any fool can make something complicated.
It is hard to keep things simple. - Richard Branson.


Re: [PATCH] Re: Eggs not installing on msys-mingw32

2023-10-30 Thread Matt Welland
I think I was premature in concluding that removing the empty files was
sufficient to fix this. I can remove the empty file but the moment I hit an
egg that depends on check-errors it seems to get stuck.

I turned off the continuous virus feature and I tried using
CHICKEN_EGG_CACHE to put the cache in /tmp and each time it seems to go
further but fails on check-errors.egg-info. Note that check-errors appears
to be properly installed as it imports fine. I think something is
attempting to remove or rewrite the check-errors.egg-info file and that
causes the problem, at least that is currently my best guess.

Unfortunately I don't think that stopping and reporting the empty file will
be sufficient but I will try the patch. Any suggestions as to what to try
next?

On Mon, Oct 30, 2023 at 9:14 AM Mario Domenech Goulart 
wrote:

> On Mon, 30 Oct 2023 14:01:44 +0100 felix.winkelm...@bevuta.com wrote:
>
> >> That was it. Thanks. I do have to keep running:
> >>
> >> find /software/ -name \*.egg-info -empty -delete -print
> >>
> >> as check-errors.egg-info seems to be a pervasive problem. Maybe it
> would be
> >> worth it to add a defence against this?
> >
> > Sorry - wrong patch, this is the right one.
> >
> >
> > felix
> >
> >
> > From 8a2a82a8171744546badacaa297bcf754bff3eba Mon Sep 17 00:00:00 2001
> > From: felix 
> > Date: Mon, 30 Oct 2023 14:00:41 +0100
> > Subject: [PATCH] handle case when egg-info file is empty
> >
> > ---
> >  egg-information.scm | 7 +--
> >  1 file changed, 5 insertions(+), 2 deletions(-)
> >
> > diff --git a/egg-information.scm b/egg-information.scm
> > index 6f5f7cb9..1e7f8aec 100644
> > --- a/egg-information.scm
> > +++ b/egg-information.scm
> > @@ -34,8 +34,11 @@
> >  (if (file-exists? fname*) fname* fname)))
> >
> >  (define (load-egg-info fname)
> > -  (let ((fname (locate-egg-file fname)))
> > -(with-input-from-file fname read)))
> > +  (let* ((fname (locate-egg-file fname))
> > + (info (with-input-from-file fname read)))
> > +(if (eof-object? info)
> > +(error "empty egg-info file, possibly due to an aborted
> egg-install - please remove the file and retry" fname)
> > +info)))
>
> How about
> "empty egg-info file, possibly due to an aborted egg-install - please
> remove the file and reinstall the corresponding egg"?
>
> Rationale: Matt's case.  check-errors.egg-info was empty, but the
> installation of base64 was failing.  If Matt had removed
> check-errors.egg-info and retried installing base64, the installation of
> base64 would work, but check-errors would still be broken
> (chicken-status and chicken-uninstall would ignore it, for example).
>
> All the best.
> Mario
> --
> http://parenteses.org/mario
>


-- 
--
Complexity is your enemy. Any fool can make something complicated.
It is hard to keep things simple. - Richard Branson.


Re: [PATCH] Re: Eggs not installing on msys-mingw32

2023-10-30 Thread Mario Domenech Goulart
On Mon, 30 Oct 2023 14:01:44 +0100 felix.winkelm...@bevuta.com wrote:

>> That was it. Thanks. I do have to keep running:
>> 
>> find /software/ -name \*.egg-info -empty -delete -print
>> 
>> as check-errors.egg-info seems to be a pervasive problem. Maybe it would be
>> worth it to add a defence against this?
>
> Sorry - wrong patch, this is the right one.
>
>
> felix
>
>
> From 8a2a82a8171744546badacaa297bcf754bff3eba Mon Sep 17 00:00:00 2001
> From: felix 
> Date: Mon, 30 Oct 2023 14:00:41 +0100
> Subject: [PATCH] handle case when egg-info file is empty
>
> ---
>  egg-information.scm | 7 +--
>  1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/egg-information.scm b/egg-information.scm
> index 6f5f7cb9..1e7f8aec 100644
> --- a/egg-information.scm
> +++ b/egg-information.scm
> @@ -34,8 +34,11 @@
>  (if (file-exists? fname*) fname* fname)))
>  
>  (define (load-egg-info fname)
> -  (let ((fname (locate-egg-file fname)))
> -(with-input-from-file fname read)))
> +  (let* ((fname (locate-egg-file fname))
> + (info (with-input-from-file fname read)))
> +(if (eof-object? info)
> +(error "empty egg-info file, possibly due to an aborted egg-install 
> - please remove the file and retry" fname)
> +info)))

How about
"empty egg-info file, possibly due to an aborted egg-install - please remove 
the file and reinstall the corresponding egg"?

Rationale: Matt's case.  check-errors.egg-info was empty, but the
installation of base64 was failing.  If Matt had removed
check-errors.egg-info and retried installing base64, the installation of
base64 would work, but check-errors would still be broken
(chicken-status and chicken-uninstall would ignore it, for example).

All the best.
Mario
-- 
http://parenteses.org/mario