I find this discussion interesting, it caused me to search around for links, blogposts, papers and examples about package management. The things I found so far are accessible here (a online collaborative text editor), but feel free to move it to some other tool, wiki etc.: https://cryptpad.fr/pad/#/2/pad/edit/YBPAp3HwnjdxbaDIVWaEcfrV/
Maybe we can collect links to papers that could be relevant. Probably should be converted to a github repo / wiki at some point? For packaging standalone applications this issue might be partially related to this discussion: "Easy to use and scriptable cross platform distribution support" https://github.com/racket/rhombus-brainstorming/issues/113 I am interested in helping with any potential efforts towards new or extended tools related to this. Simon Am Sa., 2. Mai 2020 um 00:03 Uhr schrieb Robby Findler < ro...@cs.northwestern.edu>: > Thanks! Although I cannot really see the full implications, the basic idea > sounds really great. > > DrRacket does share some things, but tries to minimize the sharing; if it > were possible to share less, that'd be great, but maybe that'll end up > falling under the same category as reasons you'd install something > globally. But certainly it would be important for the system to support the > ability to change which "project" you're in without having to start a new > command-line racket binary somehow. > > Robby > > > On Fri, May 1, 2020 at 11:04 AM Sam Tobin-Hochstadt <sa...@cs.indiana.edu> > wrote: > >> I'll try to write out in more detail what the kinds of package >> managers Sage is referring to do, to make this clearer for everyone. >> >> The high-level idea is that you have a bunch of directories on your >> system(Alex's analogy to git is a good one), and in each of them, your >> run appropriate commands and you see things set up as if the other >> directories you've worked with didn't exist. This includes both >> executing code (like running `racket foo.rkt`) and managing packages >> (like `raco pkg install`). >> >> There are basically two approaches in wide use here, depending on >> whether the underlying system was developed after this kind of package >> management approach became popular (npm, cargo, stack) or before >> (virtualenv for python and others). >> >> Approach 1 involves mostly working using the package manager tool >> (such as npm), but also setting up the semantics of the module system >> to look in the local directory. >> >> Approach 2 involves setting environment variables and command line >> arguments, and wrapping the underlying command (such as the python >> binary) to trick it into looking for files in the right place. >> >> In both approaches, installing libraries "globally" is not >> recommended, and likely reserved for command line tools that aren't >> depended upon as libraries, or for core system libraries that can't >> change independently of the compiler. >> >> One additional detail -- often performing package operations will >> automatically record the exact versions of everything that was used in >> that directory (with data similar to a pkg catalog), so that a the >> same code can be installed with the same dependent packages in a >> different situation. >> >> I think supporting this mode of working would be very useful for >> Racket, but there are definitely some challenges. The biggest one >> relates to what Matthew mentioned, with globally-installed packages >> shadowed by local-scope packages. This would be particularly hard for >> Racket, where lots of tools we want to install once globally (like >> DrRacket) would pull in packages that would also be needed locally. >> Additionally, how DrRacket would work in this mode would be tricky, >> since DrRacket's architecture relies on sharing of lower-level modules >> between the program and DrRacket itself. >> >> With that said, I think the approach Matthew outlined would take a big >> step, and would let us find out if we needed more steps, and what they >> might be. >> >> Sam >> >> On Thu, Apr 30, 2020 at 11:59 PM Robby Findler >> <ro...@cs.northwestern.edu> wrote: >> > >> > I'm not familiar enough with NPM or Pipenv to fully appreciate what >> you're saying, but I'm glad it sounds like I have the right basic idea. >> raco link is, roughly speaking, a layer below raco pkg that is used to >> implement raco pkg. >> > >> > Well, at any rate, I'll be curious to see how things progress! >> > >> > Robby >> > >> > >> > On Thu, Apr 30, 2020 at 8:18 PM Sage Gerard <s...@sagegerard.com> >> wrote: >> >> >> >> I'm looking for an experience similar to how NPM, Pipenv, and other >> package managers work in their respective ecosystems. You described the >> desired conditions correctly. >> >> >> >> Racket's separation of packages and collections complicates that a >> bit. Sam mentioned on the Slack that this problem could be approached in >> terms of 'raco link', but it would lead to a large scope of work. I wonder >> how much that overlaps here. >> >> >> >> >> >> >> >> >> >> -------- Original Message -------- >> >> On Apr 30, 2020, 9:01 PM, Robby Findler < ro...@cs.northwestern.edu> >> wrote: >> >> >> >> >> >> I'm trying to understand what you guys are talking about here and am >> not sure I'm getting it. Is the idea that a "project" (new term definition) >> would be a place one can install a package and its dependencies in a way >> that avoids any conflicts with other pkgs that are already installed (even >> if they are different versions of those same packages)? So I would have >> some command to configure my installation so that I would tell it what >> project I want to work with and then uses of `racket` after that would find >> requires inside the pkgs that belong to that project? >> >> >> >> (Sorry if this is a dumb question; I'm very curious about this.) >> >> >> >> Robby >> >> >> >> >> >> On Thu, Apr 30, 2020 at 4:33 PM Matthew Flatt <mfl...@cs.utah.edu> >> wrote: >> >>> >> >>> That's right: there's no one-step operation right now, and a new >> >>> one-step tool that sets things up the way you need would be a great >> >>> contribution. >> >>> >> >>> To avoid anything currently in user scope, maybe your new tool would >> >>> configure the project space with a fresh 'installation-name. After >> >>> setting 'pkgs-dir to be the project-specific package directory while >> >>> setting 'pkg-search-dir to chain to the original installation's >> package >> >>> directory, it could install some initial packages in the new >> >>> "installation" scope --- without affecting "user" scope or the >> original >> >>> "installation" scope. By setting 'lib-dir and 'shared-dir and similar >> >>> directories to a project-specific location, any files installed by >> >>> packages in the new "installation" scope would not pollute the old >> >>> installation. And so on. >> >>> >> >>> I'm not sure of the details, and it's possible that something new is >> >>> would still be needed in the collection or package or `setup/dirs` >> >>> layers to make the one-step tool work right. It's still possible that >> >>> the existing support doesn't do what you want at all. >> >>> >> >>> I'm unclear on whether you want to install packages in project scope >> >>> that would collide with things in the original "installation" scope, >> >>> but that's tricky at best. Conceptually, it works to install a package >> >>> in a narrower scope and shadow one in a wider scope, but only if >> >>> everything in the wider scope that depends on the shadowed package is >> >>> also shadowed in the narrower scope. If you want to be able to do >> that, >> >>> then probably this isn't the right direction. >> >>> >> >>> At Thu, 30 Apr 2020 20:54:33 +0000, Sage Gerard wrote: >> >>> > Some of the pieces are falling into place, but I'm still chewing on >> something. >> >>> > >> >>> > Let me rephrase my goal with added details: As a library author, I >> need my >> >>> > users to have ONE command that safely installs a package with >> collection paths >> >>> > that would collide in a user or (system) installation scope. >> >>> > >> >>> > The docs on package scopes and your email make it sound like I have >> to ask my >> >>> > users to create and configure an installation, and make sure no >> colliding >> >>> > packages exist at user scope. All before installing the package. >> Can your >> >>> > recipe be done in one step? If not, I think that's where I'd like to >> >>> > contribute. >> >>> > >> >>> > ~slg >> >>> > >> >>> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >> >>> > On Thursday, April 30, 2020 3:24 PM, Matthew Flatt < >> mfl...@cs.utah.edu> wrote: >> >>> > >> >>> > > Consider viewing "project" scope as "installation" scope, where an >> >>> > > existing installation becomes a wider scope after "installation". >> Then, >> >>> > > the exiting configuration and scope machinery will match what you >> need. >> >>> > > >> >>> > > If you take that view, then a project gets its own "etc" where >> you put >> >>> > > a "config.rktd" to configure the project-as-installation. That >> >>> > > "config.rktd" can point to the original installation as the next >> layer >> >>> > > for various searches: "collects", "lib", and so on. >> >>> > > >> >>> > > You can even set `config-tethered-console-bin-dir` and >> >>> > > `config-tethered-gui-bin-dir` to get project-local `racket`, >> `raco`, >> >>> > > etc., executables that have the right configuration path built in, >> >>> > > instead of having to use something like environment variables to >> select >> >>> > > a project's configuration. >> >>> > > >> >>> > > At least, that's one of the intentions of the existing layers. I >> didn't >> >>> > > push it further (e.g., to a `raco` command that would set up a >> project >> >>> > > in this sense) because there was not demand at the time. >> >>> > > >> >>> > > At Thu, 30 Apr 2020 19:10:11 +0000, Sage Gerard wrote: >> >>> > > >> >>> > > > Alright, so after thinking about how to improve Racket package >> management I >> >>> > > > decided that I want to add project scoping logic to raco pkg >> install. I'm >> >>> > not >> >>> > > > sure --scope-dir is enough because the docs say that the >> installation is >> >>> > > > apparently checked before the scoped directories, and there are >> no lock >> >>> > files. >> >>> > > > Correct me if I'm wrong. >> >>> > > > I've previously made commitments re: a CI dashboard and a syntax >> >>> > highlighter. >> >>> > > > Those remain, but don't seem to need me right this minute. >> Meanwhile, this >> >>> > is >> >>> > > > one of those features that hurts to not have, especially with >> current >> >>> > > > limitations in the default package catalog. >> >>> > > > https://akkuscm.org says it supports plt-r6rs, so I >> >>> > > > see two paths: Either implement the diff between that and >> Racket today in >> >>> > > > Akku, or add just enough of Akku to a raco pkg install --scope >> project >> >>> > > > command. I expect both efforts would be painful, but the latter >> seems >> >>> > better. >> >>> > > > Are there any landmines are in the raco pkg install code that I >> need to >> >>> > know >> >>> > > > about? I'd rather not make a bunch of changes with subtle >> consequences that >> >>> > > > I'm not seeing. I'd also like to know who would be the best >> person to >> >>> > review >> >>> > > > my proposed changes in this area. >> >>> > > > ~slg >> >>> > > > -- >> >>> > > > You received this message because you are subscribed to the >> Google Groups >> >>> > > > "Racket Developers" group. >> >>> > > > To unsubscribe from this group and stop receiving emails from >> it, send an >> >>> > > > email to racket-dev+unsubscr...@googlegroups.com. >> >>> > > > To view this discussion on the web visit >> >>> > > > >> >>> > >> https://groups.google.com/d/msgid/racket-dev/04pE-yKBt1FzXg9NpgHlRoMEPSp3rIKJMJ >> >>> > > > >> >>> > >> fzPRd7HBHxyF1gEwxK70y4uFONwvTXUADMdH5_Yz-Xy7fHA7XrbWe6NYYHXvE65gvPZ4H_uNU%3D%40 >> >>> > > > sagegerard.com. >> >>> > > >> >>> > > -- >> >>> > > >> >>> > > You received this message because you are subscribed to the >> Google Groups >> >>> > "Racket Developers" group. >> >>> > > To unsubscribe from this group and stop receiving emails from it, >> send an >> >>> > email to racket-dev+unsubscr...@googlegroups.com. >> >>> > > To view this discussion on the web visit >> >>> > >> https://groups.google.com/d/msgid/racket-dev/5eab25df.1c69fb81.8f98.25f6SMTPIN_ >> >>> > ADDED_MISSING%40gmr-mx.google.com. >> >>> > >> >>> > >> >>> > -- >> >>> > You received this message because you are subscribed to the Google >> Groups >> >>> > "Racket Developers" group. >> >>> > To unsubscribe from this group and stop receiving emails from it, >> send an >> >>> > email to racket-dev+unsubscr...@googlegroups.com. >> >>> > To view this discussion on the web visit >> >>> > >> https://groups.google.com/d/msgid/racket-dev/il5M1m5pESWE_CWHdsPHg9WvQ5BhXin9sT >> >>> > >> UJ9KMwaoKzuZldA1874eMaM5py5OFrsd23Az6Kr9mH69YblDfqrGS77flNW3BnwVzQGvygG84%3D%40 >> >>> > sagegerard.com. >> >>> >> >>> -- >> >>> You received this message because you are subscribed to the Google >> Groups "Racket Developers" group. >> >>> To unsubscribe from this group and stop receiving emails from it, >> send an email to racket-dev+unsubscr...@googlegroups.com. >> >>> To view this discussion on the web visit >> https://groups.google.com/d/msgid/racket-dev/5eab440d.1c69fb81.87557.2d50SMTPIN_ADDED_MISSING%40gmr-mx.google.com >> . >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> Groups "Racket Developers" group. >> >> To unsubscribe from this group and stop receiving emails from it, send >> an email to racket-dev+unsubscr...@googlegroups.com. >> >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/racket-dev/CAL3TdOOgXhcPV-ppNy%3D%2Be%2B_LU6B-1jd-L1jmUiZT%2B6O9WYCa4g%40mail.gmail.com >> . >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "Racket Developers" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email to racket-dev+unsubscr...@googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/racket-dev/CAL3TdOMwspLvuha7FoOm_RApfY-YGq%2BsB2625%3DLxBBJ0vhiYng%40mail.gmail.com >> . >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Racket Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to racket-dev+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/racket-dev/CAK%3DHD%2BYR_x4nuGV78-TB0dDzbAF%3D6hZDWQ%3DkbY9Pn4QqBJawcw%40mail.gmail.com >> . >> > -- > You received this message because you are subscribed to the Google Groups > "Racket Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-dev+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/racket-dev/CAL3TdOP0T693W2Avw3R-rgmAUPbdovJcVn2zJRqx0Cb-gwD66w%40mail.gmail.com > <https://groups.google.com/d/msgid/racket-dev/CAL3TdOP0T693W2Avw3R-rgmAUPbdovJcVn2zJRqx0Cb-gwD66w%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Racket Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-dev/CAHs8QBqfxDHvDbEA73cO7ak_sny4J0G03UMBJUXJeLrN0iVJsg%40mail.gmail.com.
