When will we announce transition to Weblate
Hi Julien, Julien Lepiller writes: > Yes, translations should now go through weblate, at > https://translate.fedoraproject.org/projects/guix/website/ > > We also manage the cookbook, manual, guix and packages there, though I still > need to announce it :) When will this be officially announced? I should tell this news to my friends at zh_CN team. I am trying my best to recruit new volunteers to join zh_CN translation team. There are already enthusiasts who agree to join. Personally, I prefer modifying PO files in Emacs with po-mode, but I guess Weblate will be better for teamwork. The translation progress of zh_CN lags behind my previous schedule. This is not a problem at the moment since there are only a few active Guix user in China and most of them are OK with en_US locale. But the sooner we finish zh_CN translation the better. -- Peng Mei Yu
Re: Make guix-publish's URL identical to cache file name
Hi, Ludovic Courtès writes: > Peng Mei Yu skribis: > >> Ludovic Courtès writes: >> >>> Ricardo Wurmus skribis: >>> >>>> We need to make sure that *all* the files produced by “guix publish” >>>> have correct permissions; IIRC some of the files are not readable at all >>>> by users other than the owner of the files. >>> >>> Oops, I had forgotten, my bad. I’ll push the attached patch later >>> today. >>> >>> Next we’ll need to update the ‘guix’ package, restart ‘guix publish’ on >>> berlin, and chmod a+r -R /var/cache/guix/publish. > > These two issues are now fixed. > >> Ping. What's the status of rsync? > > It’s not running yet. Ricardo, do you have a diff against berlin.scm to > run it? Ping again. -- Peng Mei Yu
Apply for commit access
Hi, Can I apply for commit access to guix-artwork.git? I may make changes to the Chinese translation and fix small bugs like this one: https://issues.guix.gnu.org/45416. Additional access to guix.git will also be great. I won't make frequent commits. My login name on Savannah is pmy. A PGP key has been uploaded there. The PGP key ID is: "C3DE DC9B DD65 6937 D320 2818 B7F7 78DB 66D7 6565". I usually sign messages with the subkey: "9445 CC98 52DE 6A0D E5DA C02E 0125 A7C5 B59C EF92". This email is signed with the PGP subkey to prove that I do hold the key. -- Peng Mei Yu signature.asc Description: PGP signature
Re: SJTUG's Guix mirror is online now
Hi, Ludovic Courtès writes: > Peng Mei Yu skribis: > >> After extensive cooperation with SJTUG's maintainers, we are happy to >> announce that SJTUG's Guix mirror is online now. >> >> The substitute URL is: https://mirrors.sjtug.sjtu.edu.cn/guix > > This is great news! > >> Asia residents, especially China residents, will greatly benefit from >> this mirror. Average download speed is about 3MB/s within China, which >> is a 100 times increase compared to ci.guix.gnu.org. >> >> SJTUG's Guix mirror is maintained by students of Shanghai Jiao Tong >> University Linux User Group (SJTUG). We will continue to cooperate with >> them in the near future to ensure this mirror server is fast and stable. > > So you are rsync’ing from ci, right? Is the mirror running ‘guix > publish’ to cope with the “weird” file layout in > /var/cache/guix/publish? No. To deal with software repositories that are not easy to have a full sync, maintainers of SJTUG wrote a cache program. When an HTTP request comes, the program tries to find the requested resource from an S3-like storage server. If the resource is found on the storage server, return an HTTP 302 redirection to storage server. Otherwise, return an HTTP 302 redirection to upstream server and fetch the resource from upstream and store it into the storage server. As I said earlier, this does not work well for Guix because Guix client does not handle HTTP 302 redirection correctly. So we tweaked the program a little bit to offer transparent proxy to Guix client. The program is here: https://github.com/sjtug/mirror-intel By the way, I am still waiting for the rsync account of our CI server. If rsync is offered, we may pull guix-publish' cache and write another program to feed the mirror with a full sync. > If you’re interested, you could write an article for the blog post (in > English, but it’d be a good opportunity to start translating the web > site into Chinese :-)) about your setup and the choices you made. I don't think there are enough things to write in a blog post but I will consider translating the web site. -- Peng Mei Yu
SJTUG's Guix mirror is online now
Hello, After extensive cooperation with SJTUG's maintainers, we are happy to announce that SJTUG's Guix mirror is online now. The substitute URL is: https://mirrors.sjtug.sjtu.edu.cn/guix Asia residents, especially China residents, will greatly benefit from this mirror. Average download speed is about 3MB/s within China, which is a 100 times increase compared to ci.guix.gnu.org. SJTUG's Guix mirror is maintained by students of Shanghai Jiao Tong University Linux User Group (SJTUG). We will continue to cooperate with them in the near future to ensure this mirror server is fast and stable. Find the announcement and usage instructions here: https://guix.org.cn/blog/sjtug-guix-mirror-is-online-now/ https://sjtug.org/post/mirror-news/2020-12-21-release-guix/ https://mirrors.sjtug.sjtu.edu.cn/docs/guix -- Peng Mei Yu
Let guix client accept HTTP redirection
Hello, I have a good news and a bad news. Good news. The maintainers of mirrors.sjtug.sjtu.edu.cn (an academic free software mirror site) agreed to accept Guix into their support list. Bad news. Their first implementation of a Guix mirror failed. Their server architecture is like this: A front-end program accepts HTTP requests. It looks up the requested resource from a storage server similar to Amazon S3. If the resource is found on storage server, return an HTTP 302 redirection to the storage server. If the resource is not found, return an HTTP 302 redirection to ci.guix.gnu.org and fetch the resource from ci.guix.gnu.org then store it in the storage server. So the next time the same resource is requested, it can be served from the storage server. During the test, we found that guix client does not accept HTTP 302 redirection. So we are working on a different implementation. In the meantime, I think the problem should be reported here and let's find out if something can be done from the Guix side. -- Peng Mei Yu
Re: Make guix-publish's URL identical to cache file name
Hello, Ludovic Courtès writes: > Ricardo Wurmus skribis: > >> We need to make sure that *all* the files produced by “guix publish” >> have correct permissions; IIRC some of the files are not readable at all >> by users other than the owner of the files. > > Oops, I had forgotten, my bad. I’ll push the attached patch later > today. > > Next we’ll need to update the ‘guix’ package, restart ‘guix publish’ on > berlin, and chmod a+r -R /var/cache/guix/publish. > >> Once that’s done we just need to start the rsync daemon again, >> preferably as a shepherd service. > > Yes. > > Sounds like we have a plan! > > Peng Mei Yu: make sure to ping us in the coming weeks if you don’t hear > from us by then! Ping. What's the status of rsync? -- Peng Mei Yu
Re: Make guix-publish's URL identical to cache file name
Hello, Ludovic Courtès writes: >> Then a mirror site can simply pull the directory >> /var/cache/guix/publish/nar from the Berlin server and serve this >> directory through a static HTTP server. There will be cache misses. >> But guix-daemon will safely fallback to the next server in >> substitute-urls. > > The simplest solution for now (I think that’s what Ricardo & co. had in > mind) would be for you to retrieve /var/cache/guix/publish on your > server, as is, and then run ‘guix publish’ on your sever: it will know > where to find files. As I wrote to Jonathan, you can/should also run > nginx on top of that as a proxy to your local ‘guix publish’. I am afraid having to install Guix and run 'guix publish' will be a major obstacle for most mirror sites, especially when their maintainers have no previous experience with Guix and have no interest in learning a new software and do not want to pollute their server with an unknown software and do not want to increase maintenance burden. This is the simplest solution on our side, but a complex solution on mirror site maintainers' side. I guess a complex Nginx URL rewrite rule might also convert the URL into real filename in the cache. Can anyone familiar with Nginx explain? -- Peng Mei Yu
Re: Make guix-publish's URL identical to cache file name
Hi Jonathan, Jonathan Brielmaier writes: > I'm running myself a smaller cuirass+publish server and I have > configured the caching on the nginx side and not with `guix publish`. > > So my cache resides in: > /var/cache/nginx/nar/ > > For me it has a dramatic speed up on my German server reaching from > Germany. So 2-3MB/s on a cold cache hit versus a hot cache with 15MB/s > and even more. > > Maybe you can give that a try. I'm not sure if it does help as you said > you are a bit limited by the bandwidth... I am perfectly fine with any method to setup a mirror site. But the academic mirror sites I am talking about only want a simple approach. They want to pull files with rsync, rclone, or FTP and serve files with static HTTP server. If we make Guix's mirror as simple as that, I am confident I can persuade more mirror sites to support Guix. > On 04.11.20 09:46, Peng Mei Yu wrote: >> I have to decide next year's server specs and budget for >> mirror.guix.org.cn before the Chinese shopping festival ends on November >> 11. If the proposal above is doable, I will keep mirror.guix.org.cn >> running for half a year and help academic mirror sites add support for >> Guix in the meantime. Otherwise I prefer to buy a prepaid three years >> VPS with a 90% discount during the shopping festival. The discount is >> huge. I don't want to miss it. > > I think we should discuss if Guix can support your effort money wise. > AFAIK we have decent fundings available. That would be great. Although I still think letting academic mirror sites do the job and saving the money would be better. As I said, the ISP cost in China is expensive and academic mirror sites have plenty of bandwidth and storage resources available. -- Peng Mei Yu
Make guix-publish's URL identical to cache file name
---cut here---end--->8--- When a client tries to download /gnu/store/87kif0bpf0anwbsaw0jvg8fyciw4sz67-bash-5.0.16, it sends a request to http://example.com/87kif0bpf0anwbsaw0jvg8fyciw4sz67.narinfo and gets the content of /var/cache/guix/publish/gzip/87kif0bpf0anwbsaw0jvg8fyciw4sz67-bash-5.0.16.narinfo: --8<---cut here---start->8--- StorePath: /gnu/store/87kif0bpf0anwbsaw0jvg8fyciw4sz67-bash-5.0.16 URL: nar/gzip/87kif0bpf0anwbsaw0jvg8fyciw4sz67-bash-5.0.16 Compression: gzip FileSize: 2284657 URL: nar/lzip/87kif0bpf0anwbsaw0jvg8fyciw4sz67-bash-5.0.16 Compression: lzip FileSize: 1256260 NarHash: sha256:1ap2s3xz3bbp5n78v826gxagy7pic1wpgzz3ka72jdyk6qpmw3qr NarSize: 6597040 References: ... System: x86_64-linux Deriver: cccyyn4xq59aimybmhlrfl2bi8kslhlm-bash-5.0.16.drv Signature: ... --8<---cut here---end--->8--- Client then sends a request to the URL as written in the URL field: http://example.com/nar/lzip/87kif0bpf0anwbsaw0jvg8fyciw4sz67-bash-5.0.16. This URL returns the file /var/cache/guix/publish/lzip/87kif0bpf0anwbsaw0jvg8fyciw4sz67-bash-5.0.16.nar. I propose we make the URL field in narinfo the same as nar file name on disk. We can change the directory structure to: --8<---cut here---start->8--- /var/cache/guix/publish/nar ├── 87kif0bpf0anwbsaw0jvg8fyciw4sz67.narinfo ├── 87kif0bpf0anwbsaw0jvg8fyciw4sz67-bash-5.0.16.nar.gz └── 87kif0bpf0anwbsaw0jvg8fyciw4sz67-bash-5.0.16.nar.lz --8<---cut here---end--->8--- And change the URL field in narinfo to --8<---cut here---start->8--- URL: 87kif0bpf0anwbsaw0jvg8fyciw4sz67-bash-5.0.16.nar.lz --8<---cut here---end--->8--- Then a mirror site can simply pull the directory /var/cache/guix/publish/nar from the Berlin server and serve this directory through a static HTTP server. There will be cache misses. But guix-daemon will safely fallback to the next server in substitute-urls. What's your opinion? I have to decide next year's server specs and budget for mirror.guix.org.cn before the Chinese shopping festival ends on November 11. If the proposal above is doable, I will keep mirror.guix.org.cn running for half a year and help academic mirror sites add support for Guix in the meantime. Otherwise I prefer to buy a prepaid three years VPS with a 90% discount during the shopping festival. The discount is huge. I don't want to miss it. -- Peng Mei Yu
