Re: Linux-libre 5.8 and beyond
Bengt, Bengt Richter 写道： BTW, how did nix get such a weird alphabet for 0-31 ? Watermarking themselves? :) This question probably deserves a Nix FAQ entry by now, if there isn't one already :-) “This is to reduce the possibility that hash representations contain character sequences that are potentially offensive to someusers (a known possibility with alphanumeric representations of numbers).” -- https://edolstra.github.io/pubs/phd-thesis.pdf Excercises for the puerile reader are obvious. Kind regards, T G-R signature.asc Description: PGP signature
Re: wip-postfix
Hello Jan, Jan Nieuwenhuizen ezt írta (időpont: 2020. aug. 10., Hét 8:50): > Gábor Boskovits writes: > > Hello! > > >> Jan Nieuwenhuizen ezt írta (időpont: 2020. márc. > 17., Ke 9:02): > > > > Gábor Boskovits writes: > > I took the liberty of rebasing wip-postfix on latest master and > found it does not compile > > --8<---cut here---start->8--- > gcc -fPIC -I. -I../../include -DNO_EAI -DDEF_SMTPUTF8_ENABLE=\"no\" > -DHAS_DEV_URANDOM > -DDEF_SHLIB_DIR=\"/gnu/store/hbdrbb84krvjvw58vmr1pvzb6l3gbmyv-postfix-minimal-3.4.8\" > -DUSE_DYNAMIC_LIBS -DUSE_DYNAMIC_MAPS -Wmissing-prototypes -Wformat > -Wno-comment -fPIC -g -O -I. -I../../include -DLINUX5 -c dns_str_resflags.c > dns_str_resflags.c:55:13: warning: RES_AAONLY is deprecated > "RES_AAONLY", RES_AAONLY, > ^ > dns_str_resflags.c:57:13: warning: RES_PRIMARY is deprecated > "RES_PRIMARY", RES_PRIMARY, > ^~~ > dns_str_resflags.c:63:22: error: ‘RES_INSECURE1’ undeclared here (not in a > function); did you mean ‘RES_RECURSE’? > "RES_INSECURE1", RES_INSECURE1, > ^ > RES_RECURSE > --8<---cut here---end--->8--- > > Luckily, that was easily fixed by updating postfix to 3.5.0. > Thanks for having a look. > > >> When I hack around and create /etc/ailases.db, it works. > > I would like to add a service config for this. > > I found we already have mail-aliases-service-type, so I used that, > together with running postalias. Now, queuing mail works ootb...but > delivery seems not to work: it remains queued. > > I rebased wip-postfix and added a couple of patches for this. Please > feel free to revert them if you don't like it :-) > > When starting postfix like so > > --8<---cut here---start->8--- > ./pre-inst-env guix system vm gnu/system/examples/postfix.tmpl`\ >--nographic -m 1G\ >--nic > user,model=virtio-net-pci,hostfwd=tcp::12025-:25,hostfwd=tcp:127.0.0.1:12022 > -: > --8<---cut here---end--->8--- > > I'm seeing > > --8<---cut here---start->8--- > 07:39:18 janneke@dundal:~/src/guix/wip-postfix [env] > $ telnet localhost 12025 > Trying 127.0.0.1... > Connected to localhost. > Escape character is '^]'. > 220 komputilo.localdomain ESMTP Postfix > mail from: root > mail from: root > 250 2.1.0 Ok > rcpt to: alice > rcpt to: alice > 250 2.1.5 Ok > data > data > 354 End data with . > hello Alice! > hello Alice! > . > . > 250 2.0.0 Ok: queued as E26BA3116 > quit > quit > 221 2.0.0 Bye > Connection closed by foreign host. > 08:03:53 janneke@dundal:~/src/guix/wip-postfix [env] > $ ssh -p 12022 root@localhost > /gnu/store/mydn0wr0bs7mz3rx9fwihpma26r0dpqq-postfix-minimal-3.5.0/mailq -C > /gnu/store/nj5pa9l9zy6vx5484pbdsqnilva8bivc-postfix-config-dir > -Queue ID- --Size-- Arrival Time -Sender/Recipient--- > E26BA3116* 175 Mon Aug 10 08:00:50 root@komputilo.localdomain > alice@komputilo.localdomain > > -- 0 Kbytes in 1 Request. > --8<---cut here---end--->8--- > > Ideas? > I will have a look early next week. Most probably the setuid stuff is missing, and access is denied to something. > > >> It looks like most everything is installed in a single, flat directory > >> > >> /gnu/store/pyv0rpd6zs0m2i482cb8qxd6mhf5b47z-postfix-minimal-3.4.8 > >> > >> executables, copies of readmes, (unused?) config files (main.cf, > >> aliases)? > > > > Yes, but can be easily separated. The config files are installer > > generated, and not used. > > Ok => TODO :-) > > >> Anyhow, this is a great start; next Mailman? > > > > One thing that blocks me from finishing this is that the setuid > > programs in the os declatation should be extended, so that we can use > > the privilege separation of postfix. I would like to propose a patch > > later this week. > > Any insight here, something blocking maybe? > Nothing in particular. I had little time recently. I just finished a bigger project, and I was on holiday. I will try to propose an interface for this next week. > Greetings, > Janneke > Regards, g_bor > > Jan (janneke) Nieuwenhuizen (5): > gnu: postfix-minimal: Updato to 3.5.0. > system: examples: Add postfix.tmpl. > gnu: postfix-minimal: Fix startup warnings. > service: postfix: Use mail-aliases-service-type. > system: postfix.tmpl: Add mail-aliases-service. > > gnu/packages/mail.scm| 32 --- > gnu/services/mail.scm| 12 ++- > gnu/system/examples/postfix.tmpl | 54 > 3 files changed, 86 insertions(+), 12 deletions(-) > create mode 100644 gnu/system/examples/postfix.tmpl > > -- > Jan Nieuwenhuizen | GNU LilyPond http://lilypond.org > Freelance IT http://JoyofSource.com | Avatar®
Re: Linux-libre 5.8 and beyond
On +2020-08-09 18:17:48 -0400, Mark H Weaver wrote: > > Note that although base32 encodes 5 bits per character, the first > character of a base32-encoded sha256 hash can only be 0 or 1, since > there's only 1 bit remaining to encode after the other 255 bits have > been encoded in the last 51 characters. > UIAM, that's only true for the nix flavor (which is default for guix hash, I think) of base32. Again UIAM, the nix view of a 256-bit sha256sum hash is little-endian, and shifts 5 bits out the bottom, as if with euclidean/ 32, and so winds up with the 1 or 0 last, at the top. I think all the others base32's shift 5 bits at a time from the big end, and could have the full range 0-31 for the top digit, however translated to glyphs. Which also means the last value on the right is a 1 or 0 in the top bit, valued 16 or 0. Of course, different length digests may produce other remainder end values. BTW, how did nix get such a weird alphabet for 0-31 ? Watermarking themselves? :) -- Regards, Bengt Richter
Re: Guix containers 'advertised' in git repos
Hi! We do the same thing, kind of, but due to our CLA restrictions we have everything in separate repository. https://github.com/ipdb/bigchaindb-guix I've named the file "activate" which is python's idiom taken from virtualenv. And it creates persistent dev environment in ".genv" directory (maybe .guix-profile is a better name). It would be cool to have an naming conventions in place, the only problem I see here is that guix is capable of so much stuff it is hard to predict ways people are going to be using it. -- David aka zzappie
File search progress: database review and question on triggers
Hi! After much delay I finally got down to work on file search support for Guix. By "file search", I mean the ability to find which package contains files matching the queried pattern. If we want to be able to know which package to install, we need file search to be able to work for packages that have not yet been installed nor built on the system. As we previously discussed, a good approach, mostly for performance reasons, would be to store all files in a database that's populated on demand from the substitute servers. What I've done so far: 1. An SQLite database with the following schema: --8<---cut here---start->8--- create table if not exists Packages ( nametext not null, output text default "out", system text not null, pathtext primary key not null, -- store path, e.g. /gnu/store/abcd...-foo version text not null, guixtext not null -- The Guix version in which the package can be found. ); create table if not exists Files ( subpath text not null, package text not null, primary key (subpath, package), -- Same subpath can occur in multiple packages. foreign key (package) references Packages(path) on delete cascade ); --8<---cut here---end--->8--- I'm not very good with SQL, so thanks in advance for reviewing this carefully; let me know if we can do better. 2. A procedure that persists the filepaths of a given package in the database. 3. Size of the database: I've persisted all locally-present store items for my current Guix version and it produced a database of 72 MiB. It compresses down to 8 MiB in zstd. But since we can have multiple Guix versions, this means that the packages have one entry per store path, so we might end up with more entries than that as the number of Guix generations grows. The worse case is around (number of guix generations) x ~100 MiB. If we compress, it would be at least 10x less, maybe way less. To be sustainable, I suggest that when we remove a Guix generation we "garbage-collect" the corresponding database entries. Thoughts? 4. Indexing speed: The above items took some 20 minutes to complete (on my rather powerful machine). A single store path takes a fraction of a second to index (on an SSD). The storage device is the bottleneck here. Not sure we can do better than the following procedure: --8<---cut here---start->8--- (define (directory-files path) "Return a list of all files within PATH, recursively. Each file is returned as the path relative to PATH, starting with a '/'. It's important that the first character be the directory separator because it gives more expressive power for search. For instance, searching \"/bin\" matches both \"/bin/foo\" and \"/usr/bin/foo\" but not \"barbin\"." ;; TODO: This does not include empty directories. Should we? ;; REVIEW: Use vlist for performance? Big packages take a fraction of a ;; second on a hot cache, so it's probably not worth it. (let ((file-list '())) (ftw path (lambda (filename statinfo flag) (when (eq? flag 'regular) (set! file-list (cons (string-drop filename (string-length path)) file-list))) #t)) file-list)) --8<---cut here---end--->8--- Most of the indexing will be done by the substitute servers however, so this is of little concern for the end user. Question: Should we include empty directories in the database? I'm tempted to answer no. 5. Search speed: It completes in a fraction of a second and supports SQLite patterns. Example: --8<---cut here---start->8--- > (format-search (search-file-package "%libio%")) samba:out@4.12.3/lib/libiov-buf-samba4.so guix:out@1.1.0-18.218a67d /share/guile/site/3.0/gnu/packages/patches/m4-gnulib-libio.patch --8<---cut here---end--->8--- Question: This bounds us to the SQLite syntax for pattern matching. Is it a problem? It seems powerful enough in practice. But maybe we can use regular expression in SQLite as well? Next points I'd like to address: 6. Automatically persist the database entry when building a package. Any idea where I should plug that in? 7. Have substitute servers distribute database content. When the user performs a file search, Guix asks the substitute server for a database update. Only the diff should be sent over the network, not the whole thing since it might be very large. Question 1: If the substitute server does not have data corresponding to the Guix server of the user, shall we send data of the version that's the closest to that of the user? Locally, if there are not many entries for the current Guix version, but many for an
Guix containers 'advertised' in git repos
We are using Guix containers for deployment and software development. I was thinking it would be nice to advertise them in the git repos. Maybe we can standardise on that. What I have now is have a file .git-deploy for deployment. E.g., https://github.com/encryption4genetics/HEGP-website/blob/master/.guix-deploy and .git-dev for development. E.g., https://github.com/genetics-statistics/GEMMA/blob/master/.guix-dev These are just examples and can be generalised with a manifest. And I should probable use plain guix instead of ~/opt/guix/bin/guix ;). But maybe we can document a 'standard' for people to follow that allow for quickly finding the build using guix and deploying with a single command sh .guix-deploy
Re: wip-postfix
Gábor Boskovits writes: Hello! >> Jan Nieuwenhuizen ezt írta (időpont: 2020. márc. 17., Ke >> 9:02): > > Gábor Boskovits writes: I took the liberty of rebasing wip-postfix on latest master and found it does not compile --8<---cut here---start->8--- gcc -fPIC -I. -I../../include -DNO_EAI -DDEF_SMTPUTF8_ENABLE=\"no\" -DHAS_DEV_URANDOM -DDEF_SHLIB_DIR=\"/gnu/store/hbdrbb84krvjvw58vmr1pvzb6l3gbmyv-postfix-minimal-3.4.8\" -DUSE_DYNAMIC_LIBS -DUSE_DYNAMIC_MAPS -Wmissing-prototypes -Wformat -Wno-comment -fPIC -g -O -I. -I../../include -DLINUX5 -c dns_str_resflags.c dns_str_resflags.c:55:13: warning: RES_AAONLY is deprecated "RES_AAONLY", RES_AAONLY, ^ dns_str_resflags.c:57:13: warning: RES_PRIMARY is deprecated "RES_PRIMARY", RES_PRIMARY, ^~~ dns_str_resflags.c:63:22: error: ‘RES_INSECURE1’ undeclared here (not in a function); did you mean ‘RES_RECURSE’? "RES_INSECURE1", RES_INSECURE1, ^ RES_RECURSE --8<---cut here---end--->8--- Luckily, that was easily fixed by updating postfix to 3.5.0. >> When I hack around and create /etc/ailases.db, it works. > I would like to add a service config for this. I found we already have mail-aliases-service-type, so I used that, together with running postalias. Now, queuing mail works ootb...but delivery seems not to work: it remains queued. I rebased wip-postfix and added a couple of patches for this. Please feel free to revert them if you don't like it :-) When starting postfix like so --8<---cut here---start->8--- ./pre-inst-env guix system vm gnu/system/examples/postfix.tmpl`\ --nographic -m 1G\ --nic user,model=virtio-net-pci,hostfwd=tcp::12025-:25,hostfwd=tcp:127.0.0.1:12022-: --8<---cut here---end--->8--- I'm seeing --8<---cut here---start->8--- 07:39:18 janneke@dundal:~/src/guix/wip-postfix [env] $ telnet localhost 12025 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 komputilo.localdomain ESMTP Postfix mail from: root mail from: root 250 2.1.0 Ok rcpt to: alice rcpt to: alice 250 2.1.5 Ok data data 354 End data with . hello Alice! hello Alice! . . 250 2.0.0 Ok: queued as E26BA3116 quit quit 221 2.0.0 Bye Connection closed by foreign host. 08:03:53 janneke@dundal:~/src/guix/wip-postfix [env] $ ssh -p 12022 root@localhost /gnu/store/mydn0wr0bs7mz3rx9fwihpma26r0dpqq-postfix-minimal-3.5.0/mailq -C /gnu/store/nj5pa9l9zy6vx5484pbdsqnilva8bivc-postfix-config-dir -Queue ID- --Size-- Arrival Time -Sender/Recipient--- E26BA3116* 175 Mon Aug 10 08:00:50 root@komputilo.localdomain alice@komputilo.localdomain -- 0 Kbytes in 1 Request. --8<---cut here---end--->8--- Ideas? >> It looks like most everything is installed in a single, flat directory >> >> /gnu/store/pyv0rpd6zs0m2i482cb8qxd6mhf5b47z-postfix-minimal-3.4.8 >> >> executables, copies of readmes, (unused?) config files (main.cf, >> aliases)? > > Yes, but can be easily separated. The config files are installer > generated, and not used. Ok => TODO :-) >> Anyhow, this is a great start; next Mailman? > > One thing that blocks me from finishing this is that the setuid > programs in the os declatation should be extended, so that we can use > the privilege separation of postfix. I would like to propose a patch > later this week. Any insight here, something blocking maybe? Greetings, Janneke Jan (janneke) Nieuwenhuizen (5): gnu: postfix-minimal: Updato to 3.5.0. system: examples: Add postfix.tmpl. gnu: postfix-minimal: Fix startup warnings. service: postfix: Use mail-aliases-service-type. system: postfix.tmpl: Add mail-aliases-service. gnu/packages/mail.scm| 32 --- gnu/services/mail.scm| 12 ++- gnu/system/examples/postfix.tmpl | 54 3 files changed, 86 insertions(+), 12 deletions(-) create mode 100644 gnu/system/examples/postfix.tmpl -- Jan Nieuwenhuizen | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
