Re: bug#30680: [racket-users] Using Racket's raco on on Guix(SD)

[Timothy Sample]

Christopher Lemmer Webber  writes:

> Timothy Sample writes:
>
>> Christopher Lemmer Webber  writes:
>>
>>> Likewise, Gregor and Raart do not install:
>>>
>>> [...]
>>
>> This is a timezone issue.  The “tzinfo” package cannot find the
>> “zoneinfo” directory in GuixSD.  If you install the “tzdata” Racket
>> package, things seem to settle down.  (It would be better to tell
>> “tzinfo” to use the system database, but that’s harder to do.)
>
> Oh that's true.  I guess this was multiple issues.  Anyway, horray, that
> one seems ok now!

I’m glad to hear it!

>>> ... install raart, lots of "cannot open output file" error messages ...
>>> [...]
>>
>> I got better results with “raart” when “gcc-toolchain” was available
>> (i.e., “guix environment --ad-hoc gcc-toolchain”).  I guess it has to
>> compile a bit of native code, so it needs a compiler.  It still brakes
>> due to a syntax error, but I get the same error on Debian, so I guess
>> that’s something.  :)
>
> Yep... that seems to have fixed the install of that issue.

Cool!

>> Also, I checked all of this from Racket without grafts, and it never
>> complained about compiling OpenSSL stuff.  Running “raco setup” gives
>> some other errors, though.
>
> You're right... without grafts it doesn't have the openssl error.  The
> other writing to the store issues still seem to persist, but it doesn't
> block running "raco setup" (after a "raco pkg install", a step I had
> omitted earlier).

Okay.  I was confused about the “raco setup” example, but the other step
makes more sense now.

I will say that even on Debian, with an regular user, I have seen
“permission denied” errors because Racket tries to update files in
“/usr/share”.

> ISTM that this is a separate bug.  In fact I'm afraid I've polluted this
> bug with what I thought were all the same bug but turned out to be
> several different bugs, of which a couple are fixed now thanks to your
> help.
>
> PS: About the bounty, my thoughts are that some of these smaller issues
> being resolved are already worth a smaller amount of compensation (and
> thanks!), but there are *two different* larger issues of which probably
> either is worth the full amount (though I can only afford to pay for
> one)... one of them is the issue of the grafts breaking eg openssl
> (which maybe we should file as a separate bug?), and the other is this
> original bug (30680) about the attempts to compile to the store (which
> does not seem as big of a blocker as it did previously, but is still
> very annoying).  Does that seem fair?  (Feel free to contact me
> off-list.)

Actually I think there is only one bug, which is the grafts thing.  This
bug was originally about compiling OpenSSL files to the store.  Grafting
doesn’t break OpenSSL it just makes Racket try to recompile its OpenSSL
FFI wrappers.

I have a patch, too.  I sent it to guix-patches, but I must have made a
mistake because it ended up in bug-guix attached to this bug report.
The patch can be found at .  Also, the
attachment didn’t get sent to the list, but did make it to the bug page.
Hm  Sorry for the goof!

Re: bug#30680: [racket-users] Using Racket's raco on on Guix(SD)

[Christopher Lemmer Webber]

Nils Gillmann writes:

> Christopher Lemmer Webber transcribed 3.3K bytes:

>> BTW, some examples of packages where I've had trouble, in case it helps
>> with testing:
>> 
>>  - Raart
>>  - Gregor
>>  - crypto (seemed to work last time, not sure why it wasn't working before)
>> 
>> Though at this point I also can't do just "raco setup" on a local
>> package either, but maybe resolving this issue will fix that.
>> 
> From what I've learned in the last couple of hours, the last problem could
> be due to leftovers in your raco / dotRacket folder from an older 
> installation.

Would be nice if it were so simple, but:

Having removed ~/.racket, try:

$ git clone https://github.com/cwebber/racket-linkeddata.git
$ cd racket-linkeddata/linkeddata

cwebber@jasmine:~/devel/racket-linkeddata/linkeddata$ raco setup
open-output-file: cannot open output file
  path: 
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/compiled/tmp15340298121534029812614
  system error: Read-only file system; errno=30
  context...:
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/racket/private/more-scheme.rkt:261:28
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/racket/file.rkt:199:0:
 call-with-atomic-output-file20
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/compiler/cm.rkt:363:0:
 compile-zo*
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/compiler/cm.rkt:572:26
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/compiler/cm.rkt:564:42
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/compiler/cm.rkt:635:0:
 compile-root
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/compiler/cm.rkt:688:15
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/compiler/cm.rkt:635:0:
 compile-root
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/compiler/cm.rkt:688:15
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/compiler/cm.rkt:635:0:
 compile-root
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/compiler/cm.rkt:688:15
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/compiler/cm.rkt:635:0:
 compile-root
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/compiler/cm.rkt:782:4:
 compilation-manager-load-handler
   standard-module-name-resolver
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/setup/main.rkt:
 [running body]
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/raco/main.rkt:
 [running body]
cwebber@jasmine:~/devel/racket-linkeddata/linkeddata$ 


Likewise, Gregor and Raart do not install:

$ mv ~/.racket ~/.racket-borked
$ raco pkg install gregor   # lots of errors during install
$ racket
racket@> (require gregor)
explode-path: contract violation
  expected: (or/c path-for-some-system? path-string?)
  given: #f
  context...:
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/racket/path.rkt:116:0:
 do-explode-path
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/racket/path.rkt:126:0:
 find-relative-path7
   /home/cwebber/.racket/6.12/pkgs/tzinfo/tzinfo/private/zoneinfo.rkt:117:2: 
for-loop
   /home/cwebber/.racket/6.12/pkgs/tzinfo/tzinfo/private/zoneinfo.rkt:107:0: 
read-tzids
   /home/cwebber/.racket/6.12/pkgs/tzinfo/tzinfo/private/zoneinfo.rkt:70:0: 
make-zoneinfo-source
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/racket/contract/private/arrow-val-first.rkt:388:18
   /home/cwebber/.racket/6.12/pkgs/tzinfo/tzinfo/main.rkt:63:0: system-tzid
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/racket/contract/private/arrow-val-first.rkt:388:18
   /home/cwebber/.racket/6.12/pkgs/gregor-lib/gregor/private/moment.rkt: 
[running body]
   /home/cwebber/.racket/6.12/pkgs/gregor-lib/gregor/private/generics.rkt: 
[traversing imports]
   /home/cwebber/.racket/6.12/pkgs/gregor-lib/gregor/private/clock.rkt: 
[traversing imports]
   /home/cwebber/.racket/6.12/pkgs/gregor-lib/gregor/main.rkt: [traversing 
imports]
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/racket/private/misc.rkt:88:7

... install raart, lots of "cannot open output file" error messages ...
racket@> (require raart)
get-module-code: no such file: 
#
  context...:
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/syntax/modcode.rkt:120:0:
 get-module-path54
   
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/syntax/modcode.rkt:225:0:
 get-module-code82
   

Re: bug#30680: [racket-users] Using Racket's raco on on Guix(SD)

[Nils Gillmann]

Christopher Lemmer Webber transcribed 3.3K bytes:
> Timothy Sample writes:
> 
> > Christopher Lemmer Webber  writes:
> >
> >> Konrad Hinsen writes:
> >>
> >>> In my tests, all packages ended up working, but performance is indeed
> >>> worse than with a Racket installation outside of Guix.
> >>>
> >>> It would be nice if someone with more knowledge of Racket internals
> >>> could give a hint or two for debugging this issue!
> >>>
> >>> Konrad.
> >>
> >> I'm posting a bug bounty on this issue: if someone can fix this I will
> >> pay them $250 USD.  I don't have the time or knowledge enough of Racket
> >> internals to do so myself.
> >
> > I have discovered a few things, but I’m not sure how to fix the
> > underlying problem(s).
> >
> > The reason Racket is trying to recompile the OpenSSL files is because of
> > a hash mismatch.  This can be seen by enabling debugging output:
> >
> > $ PLTSTDERR=debug raco setup openssl
> >
> > Which says a lot of things, but most interestingly it says:
> >
> > 
> > ...
> > compiler/cm: checking: 
> > /gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/libcrypto.rkt
> > compiler/cm: different src hash... 
> > (5d9ca57f3e267d956c7b5e62578467beb8ccc1d2 
> > 4d21ac412723fbf33f97669c2f73f0e9367f4510)
> > compiler/cm: maybe-compile-zo starting 
> > /gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/libcrypto.rkt
> > compiler/cm:   start-compile: 
> > /gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/libcrypto.rkt
> > compiler/cm:   compiling 
> > /gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/libcrypto.rkt
> > open-output-file: cannot open output file
> >   path: 
> > /gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/compiled/tmp15340167971534016797570
> >   system error: Read-only file system; errno=30
> >   context...:
> > ...
> > 
> >
> > This hash mismatch is caused by grafting.  When the package is built,
> > the path to OpenSSL gets hard-coded in a source file.  The SHA-1 hash
> > for this file is stored in its “.dep” file.  When the output is
> > grafted, the source file gets updated with a new OpenSSL path, but the
> > hash does not get updated.  This makes Racket think that the cached
> > bytecode file is incorrect (even though it was likely grafted too),
> > and it tries to recompile it.  It fails because it tries to write this
> > new bytecode file to the store.
> 
> Interesting... I hadn't even considered grafting.  (I still wonder why
> it's even trying to open *any* file in the store for output though...)
> 
> > I double checked this by trying with an ungrafted Racket, and got better
> > results.  (There was still a warning about writing to the store, but it
> > seemed less significant.)
> 
> Cool!
> 
> > The only thing I can think of for a fix would be to patch Racket to be
> > more lenient with bytecode files in the store.  That is, ignore hash
> > mismatches in store-files.  I might give this a try later tonight if
> > nobody has any better ideas.
> >
> > -- Tim
> 
> BTW, some examples of packages where I've had trouble, in case it helps
> with testing:
> 
>  - Raart
>  - Gregor
>  - crypto (seemed to work last time, not sure why it wasn't working before)
> 
> Though at this point I also can't do just "raco setup" on a local
> package either, but maybe resolving this issue will fix that.
> 
From what I've learned in the last couple of hours, the last problem could
be due to leftovers in your raco / dotRacket folder from an older installation.

Re: bug#30680: [racket-users] Using Racket's raco on on Guix(SD)

[Christopher Lemmer Webber]

Timothy Sample writes:

> Christopher Lemmer Webber  writes:
>
>> Konrad Hinsen writes:
>>
>>> In my tests, all packages ended up working, but performance is indeed
>>> worse than with a Racket installation outside of Guix.
>>>
>>> It would be nice if someone with more knowledge of Racket internals
>>> could give a hint or two for debugging this issue!
>>>
>>> Konrad.
>>
>> I'm posting a bug bounty on this issue: if someone can fix this I will
>> pay them $250 USD.  I don't have the time or knowledge enough of Racket
>> internals to do so myself.
>
> I have discovered a few things, but I’m not sure how to fix the
> underlying problem(s).
>
> The reason Racket is trying to recompile the OpenSSL files is because of
> a hash mismatch.  This can be seen by enabling debugging output:
>
> $ PLTSTDERR=debug raco setup openssl
>
> Which says a lot of things, but most interestingly it says:
>
> 
> ...
> compiler/cm: checking: 
> /gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/libcrypto.rkt
> compiler/cm: different src hash... (5d9ca57f3e267d956c7b5e62578467beb8ccc1d2 
> 4d21ac412723fbf33f97669c2f73f0e9367f4510)
> compiler/cm: maybe-compile-zo starting 
> /gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/libcrypto.rkt
> compiler/cm:   start-compile: 
> /gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/libcrypto.rkt
> compiler/cm:   compiling 
> /gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/libcrypto.rkt
> open-output-file: cannot open output file
>   path: 
> /gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/compiled/tmp15340167971534016797570
>   system error: Read-only file system; errno=30
>   context...:
> ...
> 
>
> This hash mismatch is caused by grafting.  When the package is built,
> the path to OpenSSL gets hard-coded in a source file.  The SHA-1 hash
> for this file is stored in its “.dep” file.  When the output is
> grafted, the source file gets updated with a new OpenSSL path, but the
> hash does not get updated.  This makes Racket think that the cached
> bytecode file is incorrect (even though it was likely grafted too),
> and it tries to recompile it.  It fails because it tries to write this
> new bytecode file to the store.

Interesting... I hadn't even considered grafting.  (I still wonder why
it's even trying to open *any* file in the store for output though...)

> I double checked this by trying with an ungrafted Racket, and got better
> results.  (There was still a warning about writing to the store, but it
> seemed less significant.)

Cool!

> The only thing I can think of for a fix would be to patch Racket to be
> more lenient with bytecode files in the store.  That is, ignore hash
> mismatches in store-files.  I might give this a try later tonight if
> nobody has any better ideas.
>
> -- Tim

BTW, some examples of packages where I've had trouble, in case it helps
with testing:

 - Raart
 - Gregor
 - crypto (seemed to work last time, not sure why it wasn't working before)

Though at this point I also can't do just "raco setup" on a local
package either, but maybe resolving this issue will fix that.

Re: bug#30680: [racket-users] Using Racket's raco on on Guix(SD)

[Timothy Sample]

Christopher Lemmer Webber  writes:

> Konrad Hinsen writes:
>
>> In my tests, all packages ended up working, but performance is indeed
>> worse than with a Racket installation outside of Guix.
>>
>> It would be nice if someone with more knowledge of Racket internals
>> could give a hint or two for debugging this issue!
>>
>> Konrad.
>
> I'm posting a bug bounty on this issue: if someone can fix this I will
> pay them $250 USD.  I don't have the time or knowledge enough of Racket
> internals to do so myself.

I have discovered a few things, but I’m not sure how to fix the
underlying problem(s).

The reason Racket is trying to recompile the OpenSSL files is because of
a hash mismatch.  This can be seen by enabling debugging output:

$ PLTSTDERR=debug raco setup openssl

Which says a lot of things, but most interestingly it says:


...
compiler/cm: checking: 
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/libcrypto.rkt
compiler/cm: different src hash... (5d9ca57f3e267d956c7b5e62578467beb8ccc1d2 
4d21ac412723fbf33f97669c2f73f0e9367f4510)
compiler/cm: maybe-compile-zo starting 
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/libcrypto.rkt
compiler/cm:   start-compile: 
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/libcrypto.rkt
compiler/cm:   compiling 
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/libcrypto.rkt
open-output-file: cannot open output file
  path: 
/gnu/store/jx0bkmaafb8fq0mqs5ywgnxq8rbpn8j1-racket-6.12/share/racket/collects/openssl/compiled/tmp15340167971534016797570
  system error: Read-only file system; errno=30
  context...:
...


This hash mismatch is caused by grafting.  When the package is built,
the path to OpenSSL gets hard-coded in a source file.  The SHA-1 hash
for this file is stored in its “.dep” file.  When the output is grafted,
the source file gets updated with a new OpenSSL path, but the hash does
not get updated.  This makes Racket think that the cached bytecode file
is incorrect (even though it was likely grafted too), and it tries to
recompile it.  It fails because it tries to write this new bytecode file
to the store.

I double checked this by trying with an ungrafted Racket, and got better
results.  (There was still a warning about writing to the store, but it
seemed less significant.)

The only thing I can think of for a fix would be to patch Racket to be
more lenient with bytecode files in the store.  That is, ignore hash
mismatches in store-files.  I might give this a try later tonight if
nobody has any better ideas.


-- Tim