Re: how to customize mirror list used in custom channels?
Hi, zimoun skribis: > Does something like that would make sense? Yes, that’s the idea. > diff --git a/guix/download.scm b/guix/download.scm > index 2e9ecb43fc..2497ea0f75 100644 > --- a/guix/download.scm > +++ b/guix/download.scm > @@ -499,6 +499,7 @@ (define %download-fallback-test > (define* (url-fetch* url hash-algo hash > #:optional name > #:key (system (%current-system)) > + (mirrors %mirrors) > (guile (default-guile)) > executable?) >"Return a fixed-output derivation that fetches data from URL (a string, or > a > @@ -519,7 +520,8 @@ (define file-name >(_ > (basename url > > - (let ((uri (and (string? url) (string->uri url > + (let ((uri (and (string? url) (string->uri url))) > +(mirror-file (plain-file "mirrors" (object->string mirrors There are subtleties though: we don’t want to create a new at each call as this would lead to poor performance (lack of memoization). > This change would a world rebuild, right? No, not at all. Thanks, Ludo’.
Re: how to customize mirror list used in custom channels?
Hi, On Tue, 29 Nov 2022 at 09:41, Ludovic Courtès wrote: > The more I write, the closer I get to an actual implementation. :-) > If ‘url-fetch’ takes a #:mirrors, that should allow us to extend the set > of mirrors quite conveniently, along the lines of: > > (define (my-url-fetch . args) > (apply url-fetch (append args '(#:mirrors … Well, the ’url-fetch’ defined in (guix build download) already takes #:mirrors, no? --8<---cut here---start->8--- (define* (url-fetch url file #:key (timeout 10) (verify-certificate? #t) (mirrors '()) (content-addressed-mirrors '()) (disarchive-mirrors '()) (hashes '()) print-build-trace?) --8<---cut here---end--->8--- So, I guess you are talking about ’url-fetch*’ from (guix download) and exported as ’url-fetch’. Right? The one which put the bits in the store. :-) Does something like that would make sense? --8<---cut here---start->8--- diff --git a/guix/download.scm b/guix/download.scm index 2e9ecb43fc..2497ea0f75 100644 --- a/guix/download.scm +++ b/guix/download.scm @@ -499,6 +499,7 @@ (define %download-fallback-test (define* (url-fetch* url hash-algo hash #:optional name #:key (system (%current-system)) + (mirrors %mirrors) (guile (default-guile)) executable?) "Return a fixed-output derivation that fetches data from URL (a string, or a @@ -519,7 +520,8 @@ (define file-name (_ (basename url - (let ((uri (and (string? url) (string->uri url + (let ((uri (and (string? url) (string->uri url))) +(mirror-file (plain-file "mirrors" (object->string mirrors (if (or (and (string? url) (not uri)) (and uri (memq (uri-scheme uri) '(#f file (interned-file (if uri (uri-path uri) url) @@ -540,7 +542,7 @@ (define file-name #:hash-algo hash-algo #:hash hash #:executable? executable? - #:mirrors %mirror-file + #:mirrors mirror-file #:content-addressed-mirrors (match (%download-fallback-test) ((or #f 'content-addressed-mirrors) --8<---cut here---end--->8--- This change would a world rebuild, right? Last, note that ’download-to-store’ uses %mirrors so it seems straightforward here to add ’#:key (mirrors %mirrors)’, I guess. Cheers, simon
Re: how to customize mirror list used in custom channels?
zimoun skribis: > Hi, > > On Mon, 28 Nov 2022 at 11:49, Ludovic Courtès wrote: >> Hi, >> >> zimoun skribis: >> >>> On Sat, 26 Nov 2022 at 12:11, Ludovic Courtès wrote: >> >> [...] >> Now, if you really want to extend the set of things recognized, you could write variant of ‘url-fetch’ that passes a different #:mirrors argument to ‘built-in-download’. Maybe ‘url-fetch’ could have a #:mirrors parameter to simplify it. >>> >>> Why is it not possible to extend ’%mirrors? >> >> What I described above is one way to extend it, just not via ‘set!’. > > Just to be sure, one way is this extension mechanism, I was referring to #:mirrors etc. above. [...] > The other one way is to implement a variant of url-fetch method. What > would be the interface? Where the extended list of mirrors would be > provided? The more I write, the closer I get to an actual implementation. :-) If ‘url-fetch’ takes a #:mirrors, that should allow us to extend the set of mirrors quite conveniently, along the lines of: (define (my-url-fetch . args) (apply url-fetch (append args '(#:mirrors … HTH! Ludo’.
Re: how to customize mirror list used in custom channels?
Hi, On Mon, 28 Nov 2022 at 11:49, Ludovic Courtès wrote: > Hi, > > zimoun skribis: > >> On Sat, 26 Nov 2022 at 12:11, Ludovic Courtès wrote: > > [...] > >>> Now, if you really want to extend the set of things recognized, you >>> could write variant of ‘url-fetch’ that passes a different #:mirrors >>> argument to ‘built-in-download’. Maybe ‘url-fetch’ could have a >>> #:mirrors parameter to simplify it. >> >> Why is it not possible to extend ’%mirrors? > > What I described above is one way to extend it, just not via ‘set!’. Just to be sure, one way is this extension mechanism, --8<---cut here---start->8--- (define (qt-urls component version) "Return a list of URLs for VERSION of the Qt5 COMPONENT." ;; We can't use a mirror:// scheme because these URLs are not exact copies: ;; the layout differs between them. (list (string-append "https://download.qt.io/official_releases/qt/; (version-major+minor version) "/" version "/submodules/" component "-everywhere-opensource-src-" version ".tar.xz") (string-append "https://download.qt.io/official_releases/qt/; (version-major+minor version) "/" version "/submodules/" component "-everywhere-src-" version ".tar.xz") (string-append "https://download.qt.io/archive/qt/; (version-major+minor version) "/" version "/submodules/" component "-everywhere-opensource-src-" version ".tar.xz") (let ((directory (string-append "qt5" (string-drop component 2 (string-append "http://sources.buildroot.net/; directory "/" component "-everywhere-opensource-src-" version ".tar.xz")) (string-append "https://distfiles.macports.org/qt5/; component "-everywhere-opensource-src-" version ".tar.xz"))) --8<---cut here---end--->8--- then, (uri (qt-urls name version)) Right? Well, for what it is worth, it does not appear to me consistent with the rest. We could do the same for everything, and remove "mirror://" after all. :-) The other one way is to implement a variant of url-fetch method. What would be the interface? Where the extended list of mirrors would be provided? Cheers, simon
Re: how to customize mirror list used in custom channels?
Hi, zimoun skribis: > On Sat, 26 Nov 2022 at 12:11, Ludovic Courtès wrote: [...] >> Now, if you really want to extend the set of things recognized, you >> could write variant of ‘url-fetch’ that passes a different #:mirrors >> argument to ‘built-in-download’. Maybe ‘url-fetch’ could have a >> #:mirrors parameter to simplify it. > > Why is it not possible to extend ’%mirrors? What I described above is one way to extend it, just not via ‘set!’. Ludo’.
Re: how to customize mirror list used in custom channels?
Hi, On Sat, 26 Nov 2022 at 12:11, Ludovic Courtès wrote: > (origin > (method url-fetch) > (uri (list URL1 URL2 URL3)) > (base32 …)) Neat! This is nice for one or two packages. > See for example how ‘qtbase’ does it. Well, it looks like it is a manual implementation of the Guix mirror mechanism. > Now, if you really want to extend the set of things recognized, you > could write variant of ‘url-fetch’ that passes a different #:mirrors > argument to ‘built-in-download’. Maybe ‘url-fetch’ could have a > #:mirrors parameter to simplify it. Why is it not possible to extend ’%mirrors? For instance, it is possible to extend ’%load-path’. Why not %mirrors? Cheers, simon
Re: how to customize mirror list used in custom channels?
Hi! :-) zimoun skribis: > In ’(guix download)’ module, the list of mirrors are provided by the > list ’%mirrors’. So I would like to add mine, e.g., > > (cons > '(mine >"https://fast-server.org/sources/; >"hhtps://kikoo.io/path/to/something/") > %mirrors) > > and then in my package definition, > > (uri "mirror://mine/stuff.tar.gz") > > Somehow, I would like to replace the list %mirrors by an extended one. The easiest solution is to avoid mirror:// and pass a list of regular URLs: (origin (method url-fetch) (uri (list URL1 URL2 URL3)) (base32 …)) See for example how ‘qtbase’ does it. Now, if you really want to extend the set of things recognized, you could write variant of ‘url-fetch’ that passes a different #:mirrors argument to ‘built-in-download’. Maybe ‘url-fetch’ could have a #:mirrors parameter to simplify it. But really, the URL list should do the job most of the time. Ludo’.
how to customize mirror list used in custom channels?
Hi, I have a custom channel defining a package. The source of this package is mirrored. In ’(guix download)’ module, the list of mirrors are provided by the list ’%mirrors’. So I would like to add mine, e.g., (cons '(mine "https://fast-server.org/sources/; "hhtps://kikoo.io/path/to/something/") %mirrors) and then in my package definition, (uri "mirror://mine/stuff.tar.gz") Somehow, I would like to replace the list %mirrors by an extended one. Well, I am stuck because I am puzzled by the Guile module system. :-) How to customize the mirror list? Cheers, simon
