Re: SSH key management for Guix cloud machines
Hi Fabio, On Fri, Jan 19 2024, Fabio Natali wrote: > How does the publishing happen exactly You can query SSH server keys remotely [1] but I would deploy keys I know already. Kind regards Felix [1] https://serverfault.com/a/1033095
Re: December/January update on QA and related things
Hi Chris, On Sat, Jan 20 2024, Christopher Baines wrote: > I hopefully mitigated the port encoding issue [3] by switching from > using the display procedure to log to using string->utf8 and > put-bytevector. > > [...] > > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=62590 Could the error be due to the 7-bit encoding in HTTP headers? [1] The "84" is EILSEQ from errno.h. [2] Wingo added the functionality here. [3] I left that comment in your bug, and in Bug#59510. [4] I also submitted a patch to resolve the errno in the future. [5] Then I wrote here in case anyone sees the error again. Thank you for your hard work on the Guix infrastructure! Your generous contributions serve as inspirations to many. Kind regards Felix [1] https://lists.gnu.org/archive/html/guile-user/2020-09/msg00015.html [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/asm-generic/errno.h#n67 [3] https://git.savannah.gnu.org/cgit/guile.git/commit/?id=43b6feeb1adafe54170304e7cc3c29a15c1d3808 [4] https://issues.guix.gnu.org/59510 [5] https://debbugs.gnu.org/68624
Re: Guix CLI, thoughts and suggestions
Hi Carlo, Thank you for the thoughtful reply. Carlo Zancanaro writes: Hi Ian, Much of what you've written is fair, and I'm sure that Guix's commands could be better organised. I'm not really involved in Guix development, but I think there are two "inconsistencies" that you've mentioned which can be explained. On Mon, Jan 15 2024, Ian Eure wrote: Some examples of where I think Guix could do better. This is an illustrative list, not an exhaustive one. Inconsistent organization = Most package-related commands are under `guix package', but many are sibling commands. Examples are `guix size', `guix lint', `guix hash', etc. I think the real inconsistency here is that `guix package' is poorly named. This command really operates on profiles, and performs operations (install, remove, list, etc.) on those profiles. Packages are given as arguments to this command. The other commands operate on, and show the properties of, packages. Similarly with `guix build'. Yes, I agree the behavior makes a bit more sense from that viewpoint. However, it does have non-profile-related things in it, such as `--show' and `--search'. This is getitng into another thing I’ve seen a bit of, which is overloaded commands -- ones that do multiple things that are unrelated or tangentally related. But, I didn’t have a good example, and my message was long enough already. Inconsistency between verbs and options === ... For example, installing a package is `guix package -i foo' rather than `guix package install foo', removing is `guix package -r foo' rather than `guix package remove foo', and listing installed packages is `guix package -I' rather than `guix package installed' (or similar). The specific example of `guix package' might be explained by considering it as a single transaction to update the profile. The command `guix package' really says "perform a transaction on the profile", and the options are the commands in the transaction. Since there can be multiple commands, and the command names look like package names, they are provided as options. This doesn't fully explain the behaviour. In particular the example you give: This means that users can express commands which *seem* like they should work, but do not. For example `guix package -i emacs -r emacs-pgtk -I' represents a command to 1) install emacs 2) remove emacs-pgtk 3) list installed packages (which would verify the previous two operations occurred). ... seems reasonable to have working within the view of `guix package' as a transactional operation. I agree that this would make sense, but my understanding is that `guix package' doesn’t work like that -- it only performs the final operation in the list. IMO, it should either do *everything* the commands specify, or print an error and take no action. It's also worth noting that there are convenience shortcuts in `guix install' and `guix remove'. It seems like a lot of work to change, and backwards compatibility also is an issue. I see backwards compatibility as the main issue here. There was a lot of discussion preceding the inclusion of `guix shell', because of the prospect of breaking existing tutorials/documentation floating around on the internet. This is an even bigger concern for a more drastic reorganisation of the CLI. I agree, I don’t think the situation can be improved without finding a solution to preserve BC. But, I didn’t think it was worth making detailed plans for any of this before gauging whether the problem was one broadly considered to be worth solving. — Ian
Re: Golang mudules to follow common grouping
Hi Oleg, > I've added comments in commentary section in the top of the file asking to > keep packages alphabetically sorted seen in julia-xyz.scm as well. > python-*.scm ordered semi random grouped closer to package purpose which > require more thinking where to put a new one :-) Ah, I suppose it isn't that important given you can grep for the right package. > Good point her, I did manual split, with Emacs keyboard macros, magit history > scan for copyright lines and manual check where package was used to include > new module name. > > The split into golang-crypto is in review now and there would be 2 more common > grouping: golang-compression and golang-build (or golang-extension). Rest > packages which are hard to determine a group wound go to generic golang-xyz > sorted alphabetically. Hmm, there seems to be a limit in the degree of parallelizability in this process unfortunately. But if there's anything you can think of that could help (manually) in this effort, then I'd be happy to help! > Let me know your tooling which you familiar with I might think > about some sort of automation. I've had a grep around the web and I can't see any tools or libraries that have been created already to perform functions like this. I know everyone has their own scripts (I've got one written in shell but it won't be of much use here. I would suggest writing it in Scheme so it could be easier to add more complex features and it's familiar to Guix developers. 1. Put a magic comment above each package that you would like to move. 2. Run a simple script that makes a note of all of these into a to-move-list. 3. Then stash the change with the comments you made (in case you need to change things) 4. Run another script that takes the package list and performs the move in one's repository. 5. Sort out the use-package declarations manually and run tests. 6. When satisfied, stash the change and keep just the use-package changes. 7. Run a final script that loops through all the packages and commits each one in turn. 8. Rebase to suit. That's 3 scripts: - xyz-source-transform-read-packages scm-filename package-list (for step 2) - xyz-source-transform-move-packages package-list from-filename to-filename (for step 4) - xyz-source-transform-stage-package-commit package-list from-filename to-filename (for step 7) (Where 'xyz' is either 'guix' or something else depending on where the code ends up part of Guix or not. That's not important for now.) Some caveats: - I'm not a scheme programmer, but I did use Haskell at university so I'm familiar with thinking in a functional style. - For this to work, you'd have to update most package files that reference (gnu package golang) to also reference the new package, so there will be some use-module redundancy (which could be resolved in a final clean-up commit). I'm also imagining some the possibility of having a script that can remove redundant #:use-module's in the future, though I don't know if we care about a few unneeded modules being included. I'd love to hear what you think before I fire up another emacs instance. Does it sound like a realistic workflow? All comments welcome! Kind regards, - Christina
Re: Golang mudules to follow common grouping
Hi Christina, Christina O'Donnell writes: > Hi Oleg, > > On 13/01/2024 21:05, Sharlatan Hellseher wrote: >> Hi Guix, >> >> I'm about to prepare split and aggregation of all golag packages >> related to cryptography. The process would be the same as for >> golang-check and golang-web. >> >> >> In progress: >> golang-cryptography >> >> Planned: >> golang-compression >> golang-build > > I just wanted to offer my labor to this task. I'm very new to guix > development > (I've only packaged one package so far), but I've got a fair amount of > free time > and I've been looking for ways to contribute to Guix. (Plus I've > already got my > head in golang.scm trying to package gitleaks and all its dependencies.) Thanks for offering to help! The most important part in doing so is to synchronize with the other members of the Go team to avoid duplicating the work, I think. > I know that there's a lot to do already, but I was also wondering whether > they're going in any particular order once they're moved. There's a > comment at > the end of golang.scm that says, > >> ;;; Avoid adding new packages to the end of this file. To reduce the > chances >> ;;; of a merge conflict, place them above by existing packages with > similar >> ;;; functionality or similar names. > > Would it be more organized if they was just one order: either in > alphabetical > order or grouped by function? My suggestion would be to use the file > split to > group by function and then sort each file alphabetically. Do you know > how it is > arranged for other languages? That'd be neat, but in practice is hard to maintain as the module becomes quite large and people easily (and understandably) miss the intended grouping when adding new packages. > Another question I have: Is there any tooling that can help big package > migrations like this go faster? Eg. a script to split one big diff into > individual package moves with appropriate change-log entries. Not that I'm aware of. -- Thanks, Maxim
Re: [Guix-europe-sac] Shutting down qa.guix?
Andreas Enge writes: > Am Sat, Dec 09, 2023 at 11:54:59AM +0100 schrieb Ludovic Courtès: >> I think this underlines a collective failure to get our act together. > > indeed, and besides what Simon mentioned about the bank situation I think > there was a certain lack of consistency between deciding on the technical > and on the financial questions. And of course the lack of urgency, since > anyway things continued thanks to Chris... So thank you for all you have > done, Chris, and thank you for taking action now to force us to get our act > together! Indeed QA has become a central part of our project infrastructure. > > I suggest the following, which resolves the lockstep between technology and > finance: > - Decide that the funds at the FSF pay for the hosting in its current form. > Ideally move the billing to Guix Foundation, and then, as we use to do > for bayfront, periodically ask the FSF to reimburse the hosting cost. > I think we have an informal consensus for this, and just require a formal > vote at the Guix spending committee and at the Guix Foundation SAC. > - Reimburse Chris for the costs incurred for hosting before this move. > As Simon has said, we have a vote for this already, but could also > start over with the exact amount once the first point is handled, so > that Chris does not pay for future hosting any more. > > Then in a separate step, other people can discuss about potential > technical changes to the hosting situation. It would probably be good > to have a small group of people, including Chris at least for a > transitory period, who take care of the sysadmin part. > > Any thoughts on this proposal? Sounds good to me. signature.asc Description: PGP signature
Re: Golang mudules to follow common grouping
Hi Christina, > Would it be more organized if they was just one order: either > in alphabetical order or grouped by function? My suggestion > would be to use the file split to group by function and then sort > each file alphabetically. Do you know how it is arranged for other > languages? I've added comments in commentary section in the top of the file asking to keep packages alphabetically sorted seen in julia-xyz.scm as well. python-*.scm ordered semi random grouped closer to package purpose which require more thinking where to put a new one : -) > Another question I have: Is there any tooling that can help big package > migrations like this go faster? Eg. a script to split one big diff into > individual package moves with appropriate change-log entries. Good point her, I did manual split, with Emacs keyboard macros, magit history scan for copyright lines and manual check where package was used to include new module name. The split into golang-crypto is in review now and there would be 2 more common grouping: golang-compression and golang-build (or golang-extension). Rest packages which are hard to determine a group wound go to generic golang-xyz sorted alphabetically. Let me know your tooling which you familiar with I might think about some sort of automation. Thanks, Oleg
December/January update on QA and related things
Hey, I sent out the last update [1] back near the start of December. 1: https://lists.gnu.org/archive/html/guix-devel/2023-12/msg00021.html In summary, QA has been not really working since mid December as data.qa.guix.gnu.org wasn't keeping up with processing revisions for patches and branches. I was also looking to shutdown data.qa.guix.gnu.org but this didn't happen, there still seems to be some interest in managing the infrastructure though the project (and shutting it down also takes more time and effort than leaving it running). Things are still running though, and I'm hopeful that QA will be back processing patches in the next few weeks. Specifically for the QA frontpage, little has changed in the last month. I have got around to making a diagram to put QA in context and it's now included in the README [2]. Let me know if any more diagrams would be useful as I've now got a process. 2: https://qa.guix.gnu.org/README#orgd169767 For the Guix Data Service, I've put some time in to speeding up the processing of revisions. I replaced all uses of delete-duplicates with a sort and pair-fold alternative and parallelised computing the guix derivations, computing the package derivations and a few other things that happen through inferiors. This still needs a bit more testing, but the changes are deployed on data.qa.guix.gnu.org and I think it's sped up processing individual revisions at least. Since data.qa.guix.gnu.org had less revisions in the database, I took advantage of this to do some maintenance and managed to reduce the size of the database considerably. Hopefully the frequent cleanup tasks will prevent it from getting this large again. Finally, I made various small fixes and speedups in the Guix Build Coordinator. I hopefully mitigated the port encoding issue [3] by switching from using the display procedure to log to using string->utf8 and put-bytevector. I opened a bug for a Guile segfault I hadn't seen before [4]. I also hopefully reduced the impact of the build coordinator stopping listening for connections by checking this internally and exiting if there's an issue. Unfortunately I've been quite slow in tracking down and trying to fix or at least mitigate these frequent but hard to reproduce bugs, but I think I've made some progress recently. 3: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=62590 4: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68221 Looking forward, there's still the issue of me hosting various parts of QA. Andreas has sent a proposal around on this in the last couple of days though. There's been some discussion in the past about using the data service to monitor performance related things, but maybe this is important for just keeping QA running as well. Failing to spot these issues before they're introduced can cause significant disruption, so maybe we need the data service to start monitoring and reporting on how particular performance characteristics change between revisions so that this can be reported by QA. While the bordeaux build farm is still doing well I think, I still haven't got around to implementing a way of pruning the nars that aren't for the master branch from being stored indefinitely. I've got some design ideas, but they need implementing and testing. There's also the ongoing issue of build hardware for current and up and coming architectures. Let me know if you have any comments or questions! I'm also planning to be at the Guix Days event and FOSDEM in a couple of weeks. Thanks, Chris signature.asc Description: PGP signature
Re: Golang mudules to follow common grouping
Hi Oleg, On 13/01/2024 21:05, Sharlatan Hellseher wrote: > Hi Guix, > > I'm about to prepare split and aggregation of all golag packages > related to cryptography. The process would be the same as for > golang-check and golang-web. > > > In progress: > golang-cryptography > > Planned: > golang-compression > golang-build I just wanted to offer my labor to this task. I'm very new to guix development (I've only packaged one package so far), but I've got a fair amount of free time and I've been looking for ways to contribute to Guix. (Plus I've already got my head in golang.scm trying to package gitleaks and all its dependencies.) I know that there's a lot to do already, but I was also wondering whether they're going in any particular order once they're moved. There's a comment at the end of golang.scm that says, > ;;; Avoid adding new packages to the end of this file. To reduce the chances > ;;; of a merge conflict, place them above by existing packages with similar > ;;; functionality or similar names. Would it be more organized if they was just one order: either in alphabetical order or grouped by function? My suggestion would be to use the file split to group by function and then sort each file alphabetically. Do you know how it is arranged for other languages? Another question I have: Is there any tooling that can help big package migrations like this go faster? Eg. a script to split one big diff into individual package moves with appropriate change-log entries. If you could share any scripts that you're using then that could help speed up similar tasks in future. Kind regards, - Christina
Re: Golang mudules to follow common grouping
Hi, Part III of the split is sent for review, https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68605 Thank, Oleg -- VCS: https://github.incerto.xyz/; https://git.sr.ht/~hellseher/ GPG: 9847 81DE 689C 21C2 6418 0867 76D7 27BF F62C D2B5 … наш разум - превосходная объяснительная машина которая способна найти смысл почти в чем угодно, истолковать любой феномен, но совершенно не в состоянии принять мысль о непредсказуемости.
Re: An update on ‘core-updates’
Janneke Nieuwenhuizen writes: Hi, > Janneke Nieuwenhuizen writes: > (building the info fails in the Hurd cross build), but lateron there's > another segfault. This is now also fixed, by including libc-utf8-locales-for-target as native input. Greetings, Janneke -- Janneke Nieuwenhuizen | GNU LilyPond https://LilyPond.org Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com
