[racket-users] Re: racket2nix: Failure to run raco setup

[Claes Wallin]

On Tuesday, February 27, 2018 at 6:13:43 PM UTC+8, Claes Wallin wrote:
 

> If I try to `racket -l- some/file` or try to (require) it, it won't be 
> found, and racket will again complain "collection not found", even though 
> there is:
>
> 1. An entry for the collection in 
> /nix/store/...-mypackage/share/racket/links.rktd put there by racket install
> 2. An entry for that particular share/racket in 
> ...-mypackage/etc/racket/config.rktd ('links-search-files key) and I'm 
> running racket -G ...-mypackage/etc/racket
>
> I've tried with and without -A ...-mypackage/share/racket, no difference.
>
> Is there something I'm missing, or have these options just never been made 
> use of before, and can't be guaranteed to work?
>

Solved minutes later. As the name indicates, 'links-search-files are 
supposed to point at the files, not the directories in which they can be 
found. Somewhere along the way I convinced myself otherwise. Apologize for 
the rubber-ducking.

-- 
   /c

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Re: racket2nix: Failure to run raco setup

[Claes Wallin]

On Sunday, February 25, 2018 at 1:03:08 PM UTC+8, Claes Wallin wrote:
 

> I've worked around it by simply looking at ((get-info/full 
> "$package_path") 'collection), creating the appropriate .../collects 
> directories based on the value, and then after raco setup I prune all 
> empty directories from .../collects. The full racket install doesn't do 
> this when setting up its bundled packages, but I guess there are two 
> important things that differ:
>
> 1. The layout -- I have multiple collects directories.
> 2. I'm running raco setup --only --pkgs. I guess the (collection-path) 
> call happens in the part of the code that tries to be specific about what 
> it's compiling, and the unrestricted raco setup in the racket install 
> simply doesn't take that code path.
>

The workaround helps for installing packages and packages that depend on 
them, but not for running them.

If I try to `racket -l- some/file` or try to (require) it, it won't be 
found, and racket will again complain "collection not found", even though 
there is:

1. An entry for the collection in 
/nix/store/...-mypackage/share/racket/links.rktd put there by racket install
2. An entry for that particular share/racket in 
...-mypackage/etc/racket/config.rktd ('links-search-files key) and I'm 
running racket -G ...-mypackage/etc/racket

I've tried with and without -A ...-mypackage/share/racket, no difference.

Is there something I'm missing, or have these options just never been made 
use of before, and can't be guaranteed to work?

-- 
   /c

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Re: racket2nix: Failure to run raco setup

[Claes Wallin]

On Thursday, February 22, 2018 at 2:14:53 PM UTC+8, Claes Wallin wrote:
>
> Here's where we are now with racket2nix:
>
> I'm able to install but not actually build drracket and all its 
> dependencies. I pull down the zip files, run raco install --no-setup, and 
> the files are all there in their respective share/racket/pkgs, and 
> share/racket/links.rktd is updated. Awesome! It's been a rewarding week.
>
> But I can't run raco setup, even on a simple package like at-exp-lib:
>
> ++ 
> /nix/store/20jh9mdzw4ngyg6h2205lfpwhcjwj0ca-racket-minimal-6.12/bin/racket 
> -G /nix/store/kgrvx3swjp7a1vbzk6384jx27hli6pjc-at-exp-lib/etc/racket -U -X 
> /nix/store/kgrvx3swjp7a1vbzk6384jx27hli6pjc-at-exp-lib/share/racket/collects 
> -N raco -l- raco setup --no-user --no-pkg-deps --fail-fast --only --pkgs 
> at-exp-lib
> raco setup: bootstrapping from source...
>  triggered by use of non-".zo" file
>   path: 
> /nix/store/kgrvx3swjp7a1vbzk6384jx27hli6pjc-at-exp-lib/share/racket/collects/compiler/compilation-path.rkt
> collection-path: collection not found
>   collection: "at-exp"
>   in collection directories:
>
> /nix/store/kgrvx3swjp7a1vbzk6384jx27hli6pjc-at-exp-lib/share/racket/collects
>
> /nix/store/20jh9mdzw4ngyg6h2205lfpwhcjwj0ca-racket-minimal-6.12/share/racket/collects
>/nix/store/bxhgb75zr9yrzsa496s5830i9wq7dw2d-base/share/racket/collects
>
> I've been digging around, but as there is no stack trace I cannot figure 
> out where this happens. Could anyone clue me in?
>

I've worked around it by simply looking at ((get-info/full "$package_path") 
'collection), creating the appropriate .../collects directories based on 
the value, and then after raco setup I prune all empty directories from 
.../collects. The full racket install doesn't do this when setting up its 
bundled packages, but I guess there are two important things that differ:

1. The layout -- I have multiple collects directories.
2. I'm running raco setup --only --pkgs. I guess the (collection-path) call 
happens in the part of the code that tries to be specific about what it's 
compiling, and the unrestricted raco setup in the racket install simply 
doesn't take that code path.

-- 
   /c

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Re: racket2nix

[Claes Wallin]

On Tuesday, February 20, 2018 at 4:40:38 AM UTC+8, cwebber wrote:
>
> This is great Claes, and as I said on IRC, I hope we can eventually 
> collaborate... as some others know I'm hoping to get more Racket 
> packages into Guix... which is like Nix, but written in Scheme! 
> But that work will probably build on experiences from this Nix work if I 
> get to it first at all (and it sounds like you might beat me to it 
> anyway) ;) 
>
> Thanks for working on this! 
>

It sounded on IRC that you're pretty busy, so yeah, I'll probably get there 
first. :-)

Status update: racket2nix is now in a limited useful state! I'm able to 
build the drracket package without any error conditions occurring along the 
way, but:
 - The resulting drracket package is empty. :-D
 - I'm not actually building the packages, just installing them (raco pkg 
install --no-setup).
 - There is a lot of cleaning up to do in the nix expressions, the 
low-hanging fruit being e.g. redundant dependency listings -- drracket has 
over 1500 dependencies, but a simple sort -u on them reveals that they are 
actually only 135.
 - It probably doesn't put any stuff in .../bin .

My next step is dogfooding racket2nix: Put the logic in a library, generate 
a thin wrapper that calls it, add rackunit tests, make a man page, etc. 
Hopefully this will teach me enough about packaging that I can go back and 
work toward a working drracket package.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Re: racket2nix

[johnbclements]

Test posting, please ignore, sorry

On Monday, February 19, 2018 at 12:40:38 PM UTC-8, cwebber wrote:
>
> Claes Wallin writes: 
>
> > On Sunday, February 11, 2018 at 9:43:34 PM UTC+8, stewart mackenzie 
> wrote: 
> >> 
> >> Hello, 
> >> 
> >> We're partially through the development of a nix{os} utility which 
> >> transforms an info.rkt into a nix expression. 
> >> 
> >> https://github.com/clacke/racket2nix 
> >> 
> >> It'll be helpful if other nixers/racketers could contribute or test the 
> >> project! 
> >> 
> > 
> > Hi, Racketeers! I'm the one mainly working on this, and I could use some 
> > help right now. :-) 
> > 
> > I have a concrete problem, and I've been banging my head against this 
> for a 
> > bit, so I hope someone has some insights to share. 
>
> This is great Claes, and as I said on IRC, I hope we can eventually 
> collaborate... as some others know I'm hoping to get more Racket 
> packages into Guix... which is like Nix, but written in Scheme! 
> But that work will probably build on experiences from this Nix work if I 
> get to it first at all (and it sounds like you might beat me to it 
> anyway) ;) 
>
> Thanks for working on this! 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Re: racket2nix

[Christopher Lemmer Webber]

Claes Wallin writes:

> On Sunday, February 11, 2018 at 9:43:34 PM UTC+8, stewart mackenzie wrote:
>>
>> Hello,
>>
>> We're partially through the development of a nix{os} utility which 
>> transforms an info.rkt into a nix expression.
>>
>> https://github.com/clacke/racket2nix
>>
>> It'll be helpful if other nixers/racketers could contribute or test the 
>> project!
>>
>
> Hi, Racketeers! I'm the one mainly working on this, and I could use some 
> help right now. :-)
>
> I have a concrete problem, and I've been banging my head against this for a 
> bit, so I hope someone has some insights to share.

This is great Claes, and as I said on IRC, I hope we can eventually
collaborate... as some others know I'm hoping to get more Racket
packages into Guix... which is like Nix, but written in Scheme!
But that work will probably build on experiences from this Nix work if I
get to it first at all (and it sounds like you might beat me to it
anyway) ;)

Thanks for working on this!

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Re: racket2nix

[reilithion]

I'm excited for this project! As a user of both Racket and NixOS, I've been 
hoping for something like this to come along for a while. I wish I had more 
time to devote to helping it succeed. I'll probably poke at it a little 
with a small side project and try to give feedback on using it.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Re: racket2nix

[stewart mackenzie]

Re opengl: I recall running into a similar problem, I eventually scrapped
the opengl dependency but it might be possible to wrap the whole thing up
in a makeWrapper then pass in the opengl executable path for non-nixos
systems into it.

The reason, I believe, is that opengl is packaged as a static library and
not a dynamic lib.

On 14 Feb 2018 21:01, "Anthony Carrico"  wrote:

Any opengl program was going to have similar trouble since the libs are a
function of the drivers which can't really be provided by Nix on non NixOS
platforms. This situation was more of a reflection on the state of
OpenGL than Nix in my opinion, but we ended up developing in a NixOs vm
for this reason only. I don't remember what other changes I made, this
was the biggest pain point. I did integrate with other foreign functions.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Re: racket2nix

[Claes Wallin]

On Thursday, February 15, 2018 at 3:01:08 AM UTC+8, Anthony Carrico wrote:
>
> The Racket derivation needed to be 
> parameterized to find the opengl libs/drivers. I also remember that I 
> couldn't get this to work reliably on any OS except NixOS. Any opengl 
> program was going to have similar trouble since the libs are a function 
> of the drivers which can't really be provided by Nix on non NixOS 
> platforms.


Wow, ok.
 

> This situation was more of a reflection on the state of 
> OpenGL than Nix in my opinion, but we ended up developing in a NixOs vm 
> for this reason only.
>

Yeah, that sounds like an oddity.

One nice thing you can do once you have racket2nix is that you can have 
your racket package generated as a nix package, and then override the nix 
package to have native dependencies, which I hope will be very helpful to 
people in your situation in the future.

-- 
   /c 

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Re: racket2nix

[Claes Wallin]

On Thursday, February 15, 2018 at 3:16:24 AM UTC+8, Matthew Flatt wrote:
>
> At Wed, 14 Feb 2018 03:44:44 -0800 (PST), Claes Wallin wrote: 
>
 

> > I've finally found that the way to customize where racket finds things 
> is 
> > to mess about with `racket -G` and the config.rktd file. I was hoping 
> that 
> > I could just add the share/racket directories of my dependencies to 
> > 'links-search-files, but it doesn't seem that simple. 
>
> I think this is also the right idea, but I think you need to set 
> 'pkg-search-dirs in parallel to 'links-search-files within 
> "config.rktd". 
>
> The two separate entries reflect how the package system is built on top 
> of the collection-link layer, but it has to keep additional information 
> that is specific to the package layer. 
>

Thanks! I'll try that.

I thought packages were always looked up through the collections, but maybe 
that's a difference between package dependencies when installing and the 
way you look them up when you refer to them in the code?

-- 
   /c

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Re: racket2nix

[Matthew Flatt]

At Wed, 14 Feb 2018 03:44:44 -0800 (PST), Claes Wallin wrote:
> 1. The package I'm currently building probably needs to be the installation 
> path for racket, so that I can use installation scope and have links.rktd 
> and other things end up in the expected places.

That sounds right.

> 2. As my dependencies and their dependencies have already been installed 
> once, and I want to use them from their already-prepared destination, it 
> should be possible to just merge together the links.rktd of my dependencies 
> and add my own package to the list, and that should be my new links.rktd.
>
> [...]
>
> I've finally found that the way to customize where racket finds things is 
> to mess about with `racket -G` and the config.rktd file. I was hoping that 
> I could just add the share/racket directories of my dependencies to 
> 'links-search-files, but it doesn't seem that simple.

I think this is also the right idea, but I think you need to set
'pkg-search-dirs in parallel to 'links-search-files within
"config.rktd".

The two separate entries reflect how the package system is built on top
of the collection-link layer, but it has to keep additional information
that is specific to the package layer.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Re: racket2nix

[Anthony Carrico]

I use Nix, but mostly for Haskell currently, rather than Racket. I did
prototype a nix project with Racket. I don't have the sources at my
fingertips. I'll report if I look back and see anything helpful for you.

It looks like you are trying to tie into the Racket package system. We
didn't try to do so, we were just trying to go from a git clone to a
common development environment in one step.

I do remember modifying the racket derivation in the nixpkgs repo. I
don't think that got pushed upstream. The Racket derivation needed to be
parameterized to find the opengl libs/drivers. I also remember that I
couldn't get this to work reliably on any OS except NixOS. Any opengl
program was going to have similar trouble since the libs are a function
of the drivers which can't really be provided by Nix on non NixOS
platforms. This situation was more of a reflection on the state of
OpenGL than Nix in my opinion, but we ended up developing in a NixOs vm
for this reason only. I don't remember what other changes I made, this
was the biggest pain point. I did integrate with other foreign functions.

-- 
Anthony Carrico

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Re: racket2nix

[Claes Wallin]

On Sunday, February 11, 2018 at 9:43:34 PM UTC+8, stewart mackenzie wrote:
>
> Hello,
>
> We're partially through the development of a nix{os} utility which 
> transforms an info.rkt into a nix expression.
>
> https://github.com/clacke/racket2nix
>
> It'll be helpful if other nixers/racketers could contribute or test the 
> project!
>

Hi, Racketeers! I'm the one mainly working on this, and I could use some 
help right now. :-)

I have a concrete problem, and I've been banging my head against this for a 
bit, so I hope someone has some insights to share.

First some background:

The way Nix works is, each package has its own directory under /nix/store 
with an FHS-like structure. The directory is called /nix/store/-, so for example we would have a directory 
/nix/store/vawj9y05mlvaflflswd7f7ibw3kj4x92-base/share/racket/pkgs/base , 
where the info.rkt for base would reside.

Now, when I want to build another package I would need that package to find 
racket-lib (in the racket package itself) and base (in the package base). 
What I have been doing, just to get things working, is to `raco pkg install 
--scope-dir /nix/...mypkg/share/racket/pkgs 
/nix/...base/share/racket/pkgs/base`, and so on for each transitive 
dependency of mypkg, and then finally `raco pkg install --copy --scope-dir 
/nix/.../pkgs ./mypkg`.

If you're familiar with the bowels of racket packaging, you are probably 
already seeing a number of issues with this. It "works", as in, I manage to 
build packages that depend on other packages, but it seems that doing 
install on each transitive dependency creates quite a lot of unnecessary 
work, and using --scope-dir the way I am places links.rktd in an unexpected 
place. I'm also having trouble with packages that want to write to 
collects, or copy man pages.

To improve this I'd like to do a number of things:
1. The package I'm currently building probably needs to be the installation 
path for racket, so that I can use installation scope and have links.rktd 
and other things end up in the expected places.
2. As my dependencies and their dependencies have already been installed 
once, and I want to use them from their already-prepared destination, it 
should be possible to just merge together the links.rktd of my dependencies 
and add my own package to the list, and that should be my new links.rktd.

Are you following me so far, or am I already misunderstanding something?

I've finally found that the way to customize where racket finds things is 
to mess about with `racket -G` and the config.rktd file. I was hoping that 
I could just add the share/racket directories of my dependencies to 
'links-search-files, but it doesn't seem that simple.

Here's what I'm currently getting when installing base:

++ /nix/store/hmdy6zq34vq0761l4lym9cl3piw9b6gc-racket-6.12/bin/racket -G 
/nix/store/m5msdbznjzjn0jkrknbk6cswybjqpi4m-base/etc/racket -U -X 
/nix/store/m5msdbznjzjn0jkrknbk6cswybjqpi4m-base/share/racket/collects -S 
/nix/store/hmdy6zq34vq0761l4lym9cl3piw9b6gc-racket-6.12/share/racket/collects 
-e '
(require setup/dirs)
(display (format "(current-library-collection-links): ~a~n" 
(current-library-collection-links)))
(display (format "(find-library-collection-links): ~a~n" 
(find-library-collection-links)))
(display (format "(find-links-file): ~a~n" (find-links-file)))
'
(current-library-collection-links): (#f 
/nix/store/hmdy6zq34vq0761l4lym9cl3piw9b6gc-racket-6.12/share/racket)
(find-library-collection-links): (#f 
/nix/store/hmdy6zq34vq0761l4lym9cl3piw9b6gc-racket-6.12/share/racket)
(find-links-file): 
/nix/store/m5msdbznjzjn0jkrknbk6cswybjqpi4m-base/share/racket/links.rktd
++ /nix/store/hmdy6zq34vq0761l4lym9cl3piw9b6gc-racket-6.12/bin/racket -G 
/nix/store/m5msdbznjzjn0jkrknbk6cswybjqpi4m-base/etc/racket -U -X 
/nix/store/m5msdbznjzjn0jkrknbk6cswybjqpi4m-base/share/racket/collects -S 
/nix/store/hmdy6zq34vq0761l4lym9cl3piw9b6gc-racket-6.12/share/racket/collects 
/nix/store/hmdy6zq34vq0761l4lym9cl3piw9b6gc-racket-6.12/share/racket/collects/raco/main.rkt
 
pkg install --copy --deps fail --fail-fast --scope installation ./base
main.rkt pkg install: missing dependencies;
 specify `--deps search-auto' to install them, or
 specify `--deps search-ask' to be asked about installing them
  for package: ./base
  missing packages:
   racket-lib

/nix/store/m5msdbznjzjn0jkrknbk6cswybjqpi4m-base/etc/racket/config.rktd is:
#hash(
  (share-dir . 
"/nix/store/m5msdbznjzjn0jkrknbk6cswybjqpi4m-base/share/racket")
  (links-search-files . ( 
"/nix/store/hmdy6zq34vq0761l4lym9cl3piw9b6gc-racket-6.12/share/racket" ))
)


I would be very grateful for just a hint or nudge in the right direction. 
I'm happy to already have received a PR on my code style from Felleisen 
himself, and am open to suggestions on everything. I want this to be a tool 
that will benefit the community, not something you will have to fight to 
have any use for it, or something that is too obscure or idiosyncratic for 
anybody to be able to contribute.