Re: The problem of packaging Minetest mods/games
On Wed, May 20, 2020 at 00:36:44 +0200, Leo Prikler wrote: >> Could we for example place the mods in the ~/.minetest/games folder? > That's not very functional of you. In theory, you can put stuff there, > but not using Guix. As an example, Stepmania reads all its > configuration, data and cache from ~/.stepmania-, so that's of > course where I put the data – data, that is already unfit to be managed > by Guix due to licensing issues, mind you. In the case of Minetest > mods/games, that folder quickly gets stale however, so I'd not suggest > doing so unless you really need to. I still think it ought to search ~/.minetest/games, though, for those things that may not be installed using guix. For example, minetest has a built-in means of downloading games/mods. While it's best to use a package manager, it also breaks functionality if it doesn't work both ways. -- Mike Gerwitz signature.asc Description: PGP signature
Re: Slurm with containers (i.e., orchestration)
P.S Just to keep things interesting - GWL: A workflow management language extension for GNU Guix The Guix Workflow Language (GWL) provides a scientific computing extension to GNU Guix's declarative language for package management for the declaration of scientific workflows. https://www.guixwl.org/tutorial On Tue, May 19, 2020 at 5:33 PM Begley Brothers Inc wrote: > > On Mon, May 18, 2020 at 7:50 AM Pjotr Prins wrote: > > > > I am looking into some light-weight style orchestration. One > > We think there is such a niche, the 80/20 rule. > We think containers are too limiting and a bad idea to target - but we > use them and they have mind share. > We also have some other ideas in mind, related to this context, but > we'll keep this on-topic. > > Compromise: > Cast the issue in terms of a VM and let the 'hello world' MVP/example > be a VM that grabs a container from a registry and runs it to > completion and shutsdown. > Then demostrate the use of Guix building the VM and show the you can > not only discard the the container overhead, cruft and headaches, but > you also get a more powerful Dockerfile, and all the other Guix > features. > Then show that you can easily repurposethat VM workflow to a Metal Machine. > > Of course in real world cases there are many scenarios where people > are looking for the reverse incrementalist pathway: > 1.) Legacy-App + MM > 2.) Legacy MM + (Guix + Legacy-App) > 3.) Legacy MM + Guix + workflow > 4.) Guix + workflow + (VM or MM or both) > > > possibility is to use Slurm with Guix containers - on a cluster with > > Guix that is almost trivial (we use Guix containers a lot! They are > > great) and would also allow non-container jobs. > > Hmm, doesn't slurm break the opening objective 'light-weight'? > Maybe better to write a VM abstraction/adapter for something like > Tinkerbell/tink[1], its Apache-2.0, and some project context is > here[5]. > > Define the use case as: VM's that run a task lauched by init and shut > themselves down when done - many of course have open-ended run times. > For multiple VM use cases: > There are a multitude of distributed computing tools that Guix leaves > the user free to chose amoung to build into their VM - Guix could take > no position on whether Condor, Nomad, etc, etc., etc. are better > suited to someone's problem. > > With those constraints in mind, and lightweight being primary, then it > is simple to imagine Guix generating a VM version of such a > workflow[2]and delgating the workflow heavy lifting to > Tinkerbell/tink: > > ```bash > guix light ~/src/project/hello-world.tmpl > ``` > > > Once we have containers and Slurm it should also be possible to deploy > > slurm: -1 > containers: -1 > > > in some cloud infrastructure, provided there are no dependencies on > > I think you could get there and beyond with some relatively minor > (compared to Slurm) contributions to Tinkerbell (Apache-2.0). > This setup[3] targets an AWS instance, so you could likely leverage > `guix deploy` too. > > > the cluster itself. I think it would make a terrific BLOG story if we > > put something like that together. > > > > Bcbio describes an architecture that uses the common workflow language > > (CWL) to run pipelines with containers > > > > > > https://bcbio-nextgen.readthedocs.io/en/latest/contents/cwl.html#running-with-cromwell-local-hpc > > > > I am not promoting the use of this, but it shows that infrastructure > > exists that can deploy workflows on containers in different setups > > Again we believe if you think in terms of VM's (rather than > containers) there is a wider set of possible use cases. > If you build on Tinkerbell/tink or re-implement its logic - not clear > what you have in mind - you could also expand the Guix use cases to > workflows that include metal machine (MM) users/managers. > > > (Bcbio supports Slurm). I know the Guix infrastructure uses Guix > > deploy to achieve similar roll-outs. What that lacks is the > > orchestration mechanism itself which should handle dependencies > > between jobs (i.e. a workflow). > > We're not familiar with GNU workflow language - with that caveat: > We think that for a lightweight implementation you might be better off > defining the scope more narrowly but still expanding the universe of > Guix use cases as we outlined. > Once that is in place, experience might lead the project to be > opinionated on how 'jobs' are handled between machines (VM and MM). > > Maybe getting to the point of a Guix entry in such Awesome-Metal lists[4] > > > The GNU Workflow Language goes some > > way, but it does not handle orchestration itself. > > > > In other words, we almost have the pieces, but one thing is missing > > :). > > Agreed. This does seem to be a gap. > The challenge is keeping the solution elegant, focussed, yet general. > Tinkerbell targets the “17 Unix Rules” so they may be interested in > accomodating a VM use case? > If not it would still be possible to 'define' a VM that looks to >
Re: The problem of packaging Minetest mods/games
Jan writes: > The second problem I encountered during examining the package was every > time I changed the path of the minetest_game in the minetest-data > package minetest was also recompiled, which took long time. (I thought > the path is improper) > My question is, what is the Guix way of dealing with such packages? > Imagine we have like 100 packaged minetest mods/games. Say I wanted to > add one, would I have to recompile the whole minetest package (C++), > despite the fact all mods are just Lua scripts put into folder and > interpreted by the engine? Is there a search path? > Could we for example place the mods in the ~/.minetest/games folder? If minetest looks in ~/.minetest/games then it probably can also be made to look in a list of directories specified via an environment variable. That would be the Guixy way. Let minetest specify a search path and install the data files and plugins in a well-known directory suffix. -- Ricardo
The problem of packaging Minetest mods/games
Hello Jan, > And this package is in the propagated-inputs fiend of the minetest > package, but it doesn't work. > > I would like to understand why it doesn't work, fix it and learn > something new about Guix by the way :) My guess is, that minetest searches in /gnu/store/-minetest/share/... when the actual path is /gnu/store/-minetest-data/share/... If you just want to get the base game to work, it should probably be okay to add a symlink instead of propagating the input, but this obviously won't work for third party extensions. > My question is, what is the Guix way of dealing with such packages? > Imagine we have like 100 packaged minetest mods/games. Say I wanted > to > add one, would I have to recompile the whole minetest package (C++), > despite the fact all mods are just Lua scripts put into folder and > interpreted by the engine? The canonical Guix way is to look for environment variables to set (usually called STUFF_WERE_LOOKING_FOR_PATH) and add a search path specification for that in the original package. Take gstreamer as an example, for which we define GST_PLUGIN_SYSTEM_PATH. If the game does not yet support such variables (it appears, Minetest does indeed have MINETEST_SUBGAME_PATH, but no MINETEST_MOD_PATH in case you need the latter), consider patching the game itself so that it does. If you're lucky, you can even convince people to accept your patch upstream. If not or if it's really Guix-specific (as in the case of GUIX_GTK3_PATH for example), we'll have to carry around that patch in the Guix tree and keep it updated. > Could we for example place the mods in the ~/.minetest/games folder? That's not very functional of you. In theory, you can put stuff there, but not using Guix. As an example, Stepmania reads all its configuration, data and cache from ~/.stepmania-, so that's of course where I put the data – data, that is already unfit to be managed by Guix due to licensing issues, mind you. In the case of Minetest mods/games, that folder quickly gets stale however, so I'd not suggest doing so unless you really need to. Regards, Leo PS: minetest-data sounds like a good candidate for copy-build-system. PPS: Ignore the older (non-public) mail I sent you mentioning MINETEST_GAME_PATH instead of MINETEST_SUBGAME_PATH. At that point, I had not yet encountered that variable in my research.
Re: The problem of packaging Minetest mods/games
Le 19 mai 2020 17:05:49 GMT-04:00, Jan a écrit : >Hello, > >Recently I decided to update the Minetest package, which isn't a >problem >itself, but I discovered the package fails to provide the default >minetest game. For those who don't know, Minetest is extensible by >desing - all you do is you put a modification/game into a folder - >usually /share/minetest/games or ~/.minetest/games and it is run by the >game engine. >I checked the source code and there's a non-public minetest-data >package, which provides the minetest_game and it contains this: > >(arguments > `(#:modules ((guix build utils)) > #:builder (begin > (use-modules (guix build utils)) > (let ((install-dir (string-append > %output > "/share/minetest/games/minetest_game"))) > (mkdir-p install-dir) > (copy-recursively > (assoc-ref %build-inputs "source") > install-dir) > #t > >And this package is in the propagated-inputs fiend of the minetest >package, but it doesn't work. > >I would like to understand why it doesn't work, fix it and learn >something new about Guix by the way :) > >The second problem I encountered during examining the package was every >time I changed the path of the minetest_game in the minetest-data >package minetest was also recompiled, which took long time. (I thought >the path is improper) >My question is, what is the Guix way of dealing with such packages? >Imagine we have like 100 packaged minetest mods/games. Say I wanted to >add one, would I have to recompile the whole minetest package (C++), >despite the fact all mods are just Lua scripts put into folder and >interpreted by the engine? > >Could we for example place the mods in the ~/.minetest/games folder? > > >Jan Wielkiewicz The minetest package declares MINETEST_SUBGAME_PATH. This means that minetest reads (or used to read) that environment variable to find its games. Can you check it is set in your system? Guix should have told you about it (and relogin should set it automatically). There might be a similar variable for mods we need to declare. Minetest gets rebuilt because one of its inputs is different. That's the way guix works, because we cannot be sure the package will be the same with a different input. If we didn't, we would not guarantee reproducibility. I think simply installing mods/games in share/minetest/games should work. Installing the additional package in your profile will make them available to the game. Setting MINETEST_SUBGAME_PATH to a local directory in your home might work as well. HTH!
Re: Slurm with containers (i.e., orchestration)
On Mon, May 18, 2020 at 7:50 AM Pjotr Prins wrote: > > I am looking into some light-weight style orchestration. One We think there is such a niche, the 80/20 rule. We think containers are too limiting and a bad idea to target - but we use them and they have mind share. We also have some other ideas in mind, related to this context, but we'll keep this on-topic. Compromise: Cast the issue in terms of a VM and let the 'hello world' MVP/example be a VM that grabs a container from a registry and runs it to completion and shutsdown. Then demostrate the use of Guix building the VM and show the you can not only discard the the container overhead, cruft and headaches, but you also get a more powerful Dockerfile, and all the other Guix features. Then show that you can easily repurposethat VM workflow to a Metal Machine. Of course in real world cases there are many scenarios where people are looking for the reverse incrementalist pathway: 1.) Legacy-App + MM 2.) Legacy MM + (Guix + Legacy-App) 3.) Legacy MM + Guix + workflow 4.) Guix + workflow + (VM or MM or both) > possibility is to use Slurm with Guix containers - on a cluster with > Guix that is almost trivial (we use Guix containers a lot! They are > great) and would also allow non-container jobs. Hmm, doesn't slurm break the opening objective 'light-weight'? Maybe better to write a VM abstraction/adapter for something like Tinkerbell/tink[1], its Apache-2.0, and some project context is here[5]. Define the use case as: VM's that run a task lauched by init and shut themselves down when done - many of course have open-ended run times. For multiple VM use cases: There are a multitude of distributed computing tools that Guix leaves the user free to chose amoung to build into their VM - Guix could take no position on whether Condor, Nomad, etc, etc., etc. are better suited to someone's problem. With those constraints in mind, and lightweight being primary, then it is simple to imagine Guix generating a VM version of such a workflow[2]and delgating the workflow heavy lifting to Tinkerbell/tink: ```bash guix light ~/src/project/hello-world.tmpl ``` > Once we have containers and Slurm it should also be possible to deploy slurm: -1 containers: -1 > in some cloud infrastructure, provided there are no dependencies on I think you could get there and beyond with some relatively minor (compared to Slurm) contributions to Tinkerbell (Apache-2.0). This setup[3] targets an AWS instance, so you could likely leverage `guix deploy` too. > the cluster itself. I think it would make a terrific BLOG story if we > put something like that together. > > Bcbio describes an architecture that uses the common workflow language > (CWL) to run pipelines with containers > > > https://bcbio-nextgen.readthedocs.io/en/latest/contents/cwl.html#running-with-cromwell-local-hpc > > I am not promoting the use of this, but it shows that infrastructure > exists that can deploy workflows on containers in different setups Again we believe if you think in terms of VM's (rather than containers) there is a wider set of possible use cases. If you build on Tinkerbell/tink or re-implement its logic - not clear what you have in mind - you could also expand the Guix use cases to workflows that include metal machine (MM) users/managers. > (Bcbio supports Slurm). I know the Guix infrastructure uses Guix > deploy to achieve similar roll-outs. What that lacks is the > orchestration mechanism itself which should handle dependencies > between jobs (i.e. a workflow). We're not familiar with GNU workflow language - with that caveat: We think that for a lightweight implementation you might be better off defining the scope more narrowly but still expanding the universe of Guix use cases as we outlined. Once that is in place, experience might lead the project to be opinionated on how 'jobs' are handled between machines (VM and MM). Maybe getting to the point of a Guix entry in such Awesome-Metal lists[4] > The GNU Workflow Language goes some > way, but it does not handle orchestration itself. > > In other words, we almost have the pieces, but one thing is missing > :). Agreed. This does seem to be a gap. The challenge is keeping the solution elegant, focussed, yet general. Tinkerbell targets the “17 Unix Rules” so they may be interested in accomodating a VM use case? If not it would still be possible to 'define' a VM that looks to tinkerbell like a MM. > Thoughts? I know I have brought this up before in different > guises, but we start to really need something here. > > What makes orchestration? I guess it concerns a dynamic database of > machines that can execute jobs and some type of software registry > (Guix). That seems a resonable inital scope definition, especially if you recognize VM and MM as two distinct categories of machines to apply Guix to. > Next it should be able to schedule and execute jobs using > some constraint specifiers (like network/CPU/RAM). It could be a > 'dynamic' Slurm
The problem of packaging Minetest mods/games
Hello, Recently I decided to update the Minetest package, which isn't a problem itself, but I discovered the package fails to provide the default minetest game. For those who don't know, Minetest is extensible by desing - all you do is you put a modification/game into a folder - usually /share/minetest/games or ~/.minetest/games and it is run by the game engine. I checked the source code and there's a non-public minetest-data package, which provides the minetest_game and it contains this: (arguments `(#:modules ((guix build utils)) #:builder (begin (use-modules (guix build utils)) (let ((install-dir (string-append %output "/share/minetest/games/minetest_game"))) (mkdir-p install-dir) (copy-recursively (assoc-ref %build-inputs "source") install-dir) #t And this package is in the propagated-inputs fiend of the minetest package, but it doesn't work. I would like to understand why it doesn't work, fix it and learn something new about Guix by the way :) The second problem I encountered during examining the package was every time I changed the path of the minetest_game in the minetest-data package minetest was also recompiled, which took long time. (I thought the path is improper) My question is, what is the Guix way of dealing with such packages? Imagine we have like 100 packaged minetest mods/games. Say I wanted to add one, would I have to recompile the whole minetest package (C++), despite the fact all mods are just Lua scripts put into folder and interpreted by the engine? Could we for example place the mods in the ~/.minetest/games folder? Jan Wielkiewicz
New Swedish PO file for 'shepherd' (version 0.6.2-pre1)
Hello, gentle maintainer. This is a message from the Translation Project robot. A revised PO file for textual domain 'shepherd' has been submitted by the Swedish team of translators. The file is available at: https://translationproject.org/latest/shepherd/sv.po (We can arrange things so that in the future such files are automatically e-mailed to you when they arrive. Ask at the address below if you want this.) All other PO files for your package are available in: https://translationproject.org/latest/shepherd/ Please consider including all of these in your next release, whether official or a pretest. Whenever you have a new distribution with a new version number ready, containing a newer POT file, please send the URL of that distribution tarball to the address below. The tarball may be just a pretest or a snapshot, it does not even have to compile. It is just used by the translators when they need some extra translation context. The following HTML page has been updated: https://translationproject.org/domain/shepherd.html If any question arises, please contact the translation coordinator. Thank you for all your work, The Translation Project robot, in the name of your translation coordinator.
Re: [bug#41382] [PATCH 0/6] Allow for a cryptographic hash function migration
Ludovic, (+ guix-devel) Ludovic Courtès writes: > Hello, > > Ludovic Courtès skribis: > >> Another option would be to create a data type that specifies >> its algorithm and its value. We’d replace the ‘sha256’ field with >> a ‘hash’ field of that type (in a backward-compatible way). Thinking >> about it, this is perhaps the better option. > > Here’s a v2 that does that: instead of adding a ‘sha512’ field to > , it replaces the ‘sha256’ field with ‘hash’ and introduces a > data type (similar to the data type we have). > > One can now write things like: > > (origin > ;; … > (hash (content-hash (base64 "…") sha512))) > > Since it’s a bit verbose, one can also pass a literal string directly, > in which case it’s base32-decoded: > > (origin > ;; … > (hash (content-hash "…"))) > > ‘content-hash’ uses macrology to validate as much as possible at > macro-expansion time. > > There’s a compatibility ‘origin’ macro intended to allow people to keep > writing: > > (origin > (url …) > (method …) > (sha256 …)) > > and to automatically “convert” the ‘sha256’ field specification to a > ‘content-hash’. Due to the way identifiers are matched, there are cases > where we can’t preserve the illusion of compatibility, as can be seen > with the patch below. Perhaps that’s acceptable, though. > > Thoughts? This is a great initiative, and the patches LGTM. I think that if we are to move away from SHA256, we should go with something that is immune to length extension attacks[0] such as BLAKE2/3 or SHA-3 (Keccak). Although I don't know any Guile implementations of those as of yet. SHA512 does not improve much security-wise IMO, but maybe it's worthwhile as s stop-gap. 0: https://en.wikipedia.org/wiki/Length_extension_attack signature.asc Description: PGP signature
Re: [PATCH] gnu: fontconfig: Add replacement with font-dejavu instead of gs-fonts.
On Sun, May 17, 2020 at 04:50:12PM +0200, Marius Bakke wrote: > This is a hack to make (some) fonts working when users don't have fonts > specified in their system configuration, and (crucially) places where > the fontconfig cache may be unavailable such as 'guix pack's. > > I'm not sure whether font-dejavu is a good replacement here. Another > approach could be to convert gs-fonts to TrueType or OpenType format. > > Thoughts? I don't know much about fonts and would appreciate feedback. I think you should push right away, assuming that it helps and doesn't rebuild the world. The gs-fonts are important for printing so we might need a real fix later but for now a quick fix is the right thing.
Re: [GNU-linux-libre] Replacing Yocto with Guix kernel image builds: best practices
Hi, Thanks for all the thoughtful input. We think we have worked out a preferred path - ironically from a closer reading of the linux.scm - once we had some familiarity with Guile. We think (with doubts we'll elaborate on): 1. All config files and any such files should go in a folder `gnu/packages/aux-files/`. 2. All patches should go in `gnu/packages/patches` where they all seem to live at the toplevel. The doubts are: Is this is the 'correct' way going forward - it appears only three packages adopt this location. Are we able to store patches in `gnu/packages/patches/` - it appears other packages just drop everythin in `patches` Appreciate any thoughts. On Tue, May 19, 2020 at 2:09 AM Begley Brothers Inc wrote: > > On Mon, May 18, 2020 at 5:24 AM Ricardo Wurmus wrote: >> >> >> Begley Brothers Inc writes: >> >> > > You can either put your config files in a separate git repository and add >> > > that to >> > > the native inputs, or you can include the config files in your channel >> > > repository (or later in Guix itself). >> > > >> > >> > Thanks for the suggestion. That gives some assurance. >> > Could you point to an existing guix (upstream) package that is a best >> > practice >> > example of each of those two approaches? >> > - accessing files from a separate repo >> > - a guix (upstream) package using other files >> >> There are many examples in the Guix repository. One example is >> java-cisd-args4j in gnu/packages/java.scm, which has “build-resources” >> as a native-input, which is an SVN origin. > > > Thanks Ricardo, That was great - I expected external linkages like that to be > rejected for inclusion in upstream, so it is nice to see - I think that > approach > gives us a fallback in case our preferred approach does not work out. > >> >> > Can "add it to their ~/.config/guix/channels.scm file" be scripted as part >> > of the >> > package? >> > Is there an example of a guix (upstream) package that does this? >> >> No, channel configuration is a user action. The channel would be the >> thing that provides your package in the first place. >> >> But since you want to add your package variants to Guix itself a >> discussion of channels isn’t really interesting. > > > Agreed. > > Thanks again. > > -- > Kind Regards > > Begley Brothers Inc. > > The content of this email is confidential and intended for the recipient > specified in message only. It is strictly forbidden to share any part of this > message with any third party, without a written consent of the sender. If you > received this message by mistake, please reply to this message and follow > with its deletion, so that we can ensure such a mistake does not occur in the > future. > This message has been sent as a part of discussion between Begley Brothers > Inc. and the addressee whose name is specified above. Should you receive this > message by mistake, we would be most grateful if you informed us that the > message has been sent to you. In this case, we also ask that you delete this > message from your mailbox, and do not forward it or any part of it to anyone > else. Thank you for your cooperation and understanding. > Begley Brothers Inc. puts the security of the client at a high priority. > Therefore, we have put efforts into ensuring that the message is error and > virus-free. Unfortunately, full security of the email cannot be ensured as, > despite our efforts, the data included in emails could be infected, > intercepted, or corrupted. Therefore, the recipient should check the email > for threats with proper software, as the sender does not accept liability for > any damage inflicted by viewing the content of this email. > The views and opinions included in this email belong to their author and do > not necessarily mirror the views and opinions of the company. Our employees > are obliged not to make any defamatory clauses, infringe, or authorize > infringement of any legal right. Therefore, the company will not take any > liability for such statements included in emails. In case of any damages or > other liabilities arising, employees are fully responsible for the content of > their emails. -- Kind Regards Begley Brothers Inc. The content of this email is confidential and intended for the recipient specified in message only. It is strictly forbidden to share any part of this message with any third party, without a written consent of the sender. If you received this message by mistake, please reply to this message and follow with its deletion, so that we can ensure such a mistake does not occur in the future. This message has been sent as a part of discussion between Begley Brothers Inc. and the addressee whose name is specified above. Should you receive this message by mistake, we would be most grateful if you informed us that the message has been sent to you. In this case, we also ask that you delete this message from your mailbox, and do not forward it or any part of it to anyone else. Thank you for
Guix mirrors
Hi, Over the last 24 hours I've experienced `guix pull` etc being unavailable (HTTP 504's then 502's) more than available. Is there a reason why a post receive hook can't be added to the guix repo to push to github, gitlab, etc. and in that way at least give users some protection against these outages? There is a mirror[1] possibly (unofficial?) but it looks like it is driven by some chron task. The required post receive hook is well documented[2], and not un-common amoung reputable OS projects: - Android - The Apache Software Foundation - The Chromium Project - The Eclipse Foundation - The FreeBSD Project - The Glasgow Haskell Compiler - GNOME - The Linux kernel source tree - Qt [1]: https://github.com/guix-mirror/guix [2]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks -- Kind Regards Begley Brothers Inc. The content of this email is confidential and intended for the recipient specified in message only. It is strictly forbidden to share any part of this message with any third party, without a written consent of the sender. If you received this message by mistake, please reply to this message and follow with its deletion, so that we can ensure such a mistake does not occur in the future. This message has been sent as a part of discussion between Begley Brothers Inc. and the addressee whose name is specified above. Should you receive this message by mistake, we would be most grateful if you informed us that the message has been sent to you. In this case, we also ask that you delete this message from your mailbox, and do not forward it or any part of it to anyone else. Thank you for your cooperation and understanding. Begley Brothers Inc. puts the security of the client at a high priority. Therefore, we have put efforts into ensuring that the message is error and virus-free. Unfortunately, full security of the email cannot be ensured as, despite our efforts, the data included in emails could be infected, intercepted, or corrupted. Therefore, the recipient should check the email for threats with proper software, as the sender does not accept liability for any damage inflicted by viewing the content of this email. The views and opinions included in this email belong to their author and do not necessarily mirror the views and opinions of the company. Our employees are obliged not to make any defamatory clauses, infringe, or authorize infringement of any legal right. Therefore, the company will not take any liability for such statements included in emails. In case of any damages or other liabilities arising, employees are fully responsible for the content of their emails.
Re: [GNU-linux-libre] Replacing Yocto with Guix kernel image builds: best practices
On Mon, May 18, 2020 at 5:24 AM Ricardo Wurmus wrote: > > Begley Brothers Inc writes: > > > > You can either put your config files in a separate git repository and > add > > > that to > > > the native inputs, or you can include the config files in your channel > > > repository (or later in Guix itself). > > > > > > > Thanks for the suggestion. That gives some assurance. > > Could you point to an existing guix (upstream) package that is a best > > practice > > example of each of those two approaches? > > - accessing files from a separate repo > > - a guix (upstream) package using other files > > There are many examples in the Guix repository. One example is > java-cisd-args4j in gnu/packages/java.scm, which has “build-resources” > as a native-input, which is an SVN origin. > Thanks Ricardo, That was great - I expected external linkages like that to be rejected for inclusion in upstream, so it is nice to see - I think that approach gives us a fallback in case our preferred approach does not work out. > > Can "add it to their ~/.config/guix/channels.scm file" be scripted as > part > > of the > > package? > > Is there an example of a guix (upstream) package that does this? > > No, channel configuration is a user action. The channel would be the > thing that provides your package in the first place. > > But since you want to add your package variants to Guix itself a > discussion of channels isn’t really interesting. > Agreed. Thanks again. -- Kind Regards Begley Brothers Inc. 1. *The content of this email is confidential and intended for the recipient specified in message only. It is strictly forbidden to share any part of this message with any third party, without a written consent of the sender. If you received this message by mistake, please reply to this message and follow with its deletion, so that we can ensure such a mistake does not occur in the future.* 2. *This message has been sent as a part of discussion between Begley Brothers Inc. and the addressee whose name is specified above. Should you receive this message by mistake, we would be most grateful if you informed us that the message has been sent to you. In this case, we also ask that you delete this message from your mailbox, and do not forward it or any part of it to anyone else. Thank you for your cooperation and understanding.* 3. *Begley Brothers Inc. puts the security of the client at a high priority. Therefore, we have put efforts into ensuring that the message is error and virus-free. Unfortunately, full security of the email cannot be ensured as, despite our efforts, the data included in emails could be infected, intercepted, or corrupted. Therefore, the recipient should check the email for threats with proper software, as the sender does not accept liability for any damage inflicted by viewing the content of this email.* 4. *The views and opinions included in this email belong to their author and do not necessarily mirror the views and opinions of the company. Our employees are obliged not to make any defamatory clauses, infringe, or authorize infringement of any legal right. Therefore, the company will not take any liability for such statements included in emails. In case of any damages or other liabilities arising, employees are fully responsible for the content of their emails.*
Re: [GNU-linux-libre] Replacing Yocto with Guix kernel image builds: best practices
On Mon, May 18, 2020 at 2:17 AM Efraim Flashner wrote: > On Mon, May 18, 2020 at 07:40:53AM +1000, Begley Brothers Inc wrote: > > On Sun, May 17, 2020 at 10:55 PM Ricardo Wurmus > wrote: > > > > > > > > Hi, > > > > > > [+guix-devel, -gnu-linux-libre] > > > > > > > We are now looking to build Linux kernels using Guix instead of > Yocto. > > > We > > > > can't see any reason why the builds wouldn't be linux-libre. Ideally > we'd > > > > like our effort to be accepted by upstream guix. > > > […] > > > > We'd appreciate any pointers to package definition(s) that > demonstrate > > > best > > > > practices to do what we'd like: > > > > > > > > - We'd like to build custom configured kernels for each patch series > in > > > the > > > > LTS 4.14.72+, 4.19+ and 5.4+. > > > > - Currently we have two `base` kernel configs that each 'variant' > > > > configuration is applied to for each of a machine 'type' (3 machine > > > types) > > > > and one of two 'arch'. > > > > - Currently we can generate a full kernel `.config` for a > > > > kernel+base+variant+arch (we are working on the best way to handle > > > > different machines if we are not using Yocto.) > > > > - We'd ideally like to generate `vmlinux`, `initrd` and `rootfs` > images > > > for > > > > each. > > > > > > > > Based on Efraim's post we think the first example is the least > friction - > > > > "including an actual .config file as a native input to our custom > > > kernel". > > > > Assume we resolve the machine definition issue. However we're > puzzled > > > > about how to best distribute the configuration file such that a > build of > > > > kernel x.y.z can be updated with fixes. > > > > > > You can either put your config files in a separate git repository and > add > > > that to > > > the native inputs, or you can include the config files in your channel > > > repository (or later in Guix itself). > > > > > > > Thanks for the suggestion. That gives some assurance. > > Could you point to an existing guix (upstream) package that is a best > > practice > > example of each of those two approaches? > > - accessing files from a separate repo > > - a guix (upstream) package using other files > > > > Do you mean something like this? It's a container instead of a full disk > image but it seems close enough. The 'gn' namespace is local to that > repo and 'gnu' and 'guix' are from the upstream guix repo. > > http://git.genenetwork.org/guix-bioinformatics/guix-bioinformatics/src/branch/master/gn/services/bnw-container.scm Thanks for sharing that definition - very interesting. It seems gexp are something to be mastered. We're trying to disentangle ourselves from containers. Guix gets us a big step in that direction, and that definitions shows some of its power. Much appreciated. -- Kind Regards Begley Brothers Inc. 1. *The content of this email is confidential and intended for the recipient specified in message only. It is strictly forbidden to share any part of this message with any third party, without a written consent of the sender. If you received this message by mistake, please reply to this message and follow with its deletion, so that we can ensure such a mistake does not occur in the future.* 2. *This message has been sent as a part of discussion between Begley Brothers Inc. and the addressee whose name is specified above. Should you receive this message by mistake, we would be most grateful if you informed us that the message has been sent to you. In this case, we also ask that you delete this message from your mailbox, and do not forward it or any part of it to anyone else. Thank you for your cooperation and understanding.* 3. *Begley Brothers Inc. puts the security of the client at a high priority. Therefore, we have put efforts into ensuring that the message is error and virus-free. Unfortunately, full security of the email cannot be ensured as, despite our efforts, the data included in emails could be infected, intercepted, or corrupted. Therefore, the recipient should check the email for threats with proper software, as the sender does not accept liability for any damage inflicted by viewing the content of this email.* 4. *The views and opinions included in this email belong to their author and do not necessarily mirror the views and opinions of the company. Our employees are obliged not to make any defamatory clauses, infringe, or authorize infringement of any legal right. Therefore, the company will not take any liability for such statements included in emails. In case of any damages or other liabilities arising, employees are fully responsible for the content of their emails.*
Re: Replacing Yocto with Guix kernel image builds: best practices
On Mon, May 18, 2020 at 1:35 AM Efraim Flashner wrote: > On Mon, May 18, 2020 at 07:23:12AM +1000, Begley Brothers Inc wrote: > > On Sun, May 17, 2020 at 11:21 PM Efraim Flashner > > wrote: > > > > > On Sun, May 17, 2020 at 09:41:00PM +1000, Begley Brothers Inc wrote: > > > > Hi, > > > > We are now looking to build Linux kernels using Guix instead of > Yocto. > > > We > > > > can't see any reason why the builds wouldn't be linux-libre. Ideally > we'd > > > > like our effort to be accepted by upstream guix. > > > > > > > > However, being new to Guix we are still coming grips with the best > > > > practice(s) for what we would like to do. > > > > We've looked at Guix's linux.scm, and > > > > Efraim Flashner's post[1] is our primary reference - many thanks > Efraim. > > > > > > I'd like to embarrassingly mention that I never actually booted any of > > > my custom kernels. My machine was a bit too slow to regularly compile > > > for some guess-and-check. > > > > > > > No worries. Something is better than nothing and that was very useful > > outline of possible alternatives. > > > > > > > > > 1) We did wonder if channels[2] were the way to go with each kernel > x.y.z > > > > in its own branch and config files therein. Could anyone point us to > > > > packages that setup and use package specific channels? > > > > 2) Should we be aiming to provide a single package with multiple > > > parameters > > > > or is it better to provide a package for each kernel x.y.z, or some > other > > > > partitioning. We'd likely want to script the package definition then > - > > > > correct? > > > > > > I would probably start with one package each and then see how they > could > > > be made to inherit from one another or grouped together. The way the > > > current make-linux-libre procedures work has grown quite a bit but they > > > should still mostly work as a starting point to add/remove bits for > > > customizing for your needs. > > > > > > I don't think I'd go with different branches. If you keep everything in > > > one branch then it's easier to deduplicate work between the different > > > variants. > > > > > > > That was our thought too. > > We're just wondering how best to get that branch content down to the > users > > machine without them having to do anything. > > > > Thanks again. > > > > One option would be to include a channels file in the repo that's setup > so that you can add to the README "this is where our customizations > live. In order to build an image, run 'git pull; guix pull > --channels=our-channel-file.scm --profile > ~/.config/begley/current; ~/.config/begley/current/bin/guix > system build image.scm'". Then you can distribute the channels file you > Yes, that is somthing like what I had in mind, but would prefer to make it frictionless. Channels have the disadvantage of requiring the user do somthing that adds no value - by that I mean we've asked them to do somthing generic, and haven't saved or gotten any benefit. For example if in creating the channel they gave data that allowed us to automagically build x,y and z when they `guix pull`, and they don't have to remeber to rebuild after `guix pull` then there has been some value add. Right now I just don't see the value we could add - but we're still coming to grips with Guix. Immediately I can see that our objection is eliminated if we build a package that adds a channel in the background. Not sure if that is possible or in the spirit of Guix - we'd be altering one of their config files. How would we get their permission? etc. etc. Thanks for the feedback. -- Kind Regards Begley Brothers Inc. 1. *The content of this email is confidential and intended for the recipient specified in message only. It is strictly forbidden to share any part of this message with any third party, without a written consent of the sender. If you received this message by mistake, please reply to this message and follow with its deletion, so that we can ensure such a mistake does not occur in the future.* 2. *This message has been sent as a part of discussion between Begley Brothers Inc. and the addressee whose name is specified above. Should you receive this message by mistake, we would be most grateful if you informed us that the message has been sent to you. In this case, we also ask that you delete this message from your mailbox, and do not forward it or any part of it to anyone else. Thank you for your cooperation and understanding.* 3. *Begley Brothers Inc. puts the security of the client at a high priority. Therefore, we have put efforts into ensuring that the message is error and virus-free. Unfortunately, full security of the email cannot be ensured as, despite our efforts, the data included in emails could be infected, intercepted, or corrupted. Therefore, the recipient should check the email for threats with proper software, as the sender does not accept liability for any damage inflicted by
Re: bug#35728: Tor & IceCat's TorButton shows it's connected but doesn't route the traffic
I can confirm this issue and I've been experiencing it for almost a year. It used to work. This is very embarrassing since our IceCat is misleading the users that they are browsing anonymously while they are not. Mark, any idea about this? -- Pierre Neidhardt https://ambrevar.xyz/
