Re: [gentoo-dev] Experimental binary package hosting
* Matt Turner schrieb am 23.09.21 um 09:28 Uhr: > On Thu, Sep 23, 2021 at 7:12 AM Andreas K. Huettel > wrote: > > > > Hi Vadim, > > > > > Finally it happened! > > > I already planned to try to ask infra/council about sponsoring few > > > servers for build farm for "official gentoo binhosts" when I had > > > enough time, but fortunately, you've already did that. > > > It's very good news. > > > > Thanks! Nice to see that this is appreciated :) > > > > So far I'm only using "spare time" on the machine that builds the > > releng stages (amd64, x86, m68k, riscv). So no need for a big server > > farm. > > > > > Btw, do you need any help with that? > > > I'd be very happy to help with that project. > > > > Sure! Feel free to add yourself to the Project:Binhost wiki page. I'll > > ask for an alias and a channel soon. > > > > The most useful steps now are only half related to actual building. I > > barely know any python and am not very familiar with portage > > internals... this is what in my opinion we'd need next: > > > > 1) a tool to manage and manipulate a binpkg/ directory tree > > The main functions that I see needed are > > * delete packages/versions that are not in the gentoo repository > > anymore (xpak and in index file), maybe with some grace time > > * merge xpak files built elsewhere into the directory (also in the > > index file) > > eclean packages from gentoolkit does this exactly. plus 'emaint binhost' for rebuilding the index after adding packages manually :) -Marc -- 0xCA3E7BF67F979BE5 - F7FB 78F7 7CC3 79F6 DF07 6E9E CA3E 7BF6 7F97 9BE5
Re: [gentoo-dev] Experimental binary package hosting
On Thursday, September 23rd, 2021 at 10:11, Andreas K. Huettel wrote: > 3. an easy way to figure out if a binary package repo is suitable for > a profile / arch / ... or not, and a standard for path names > This is not so important right now, and partially also already present > There is a PROFILE data in Packages index, but it is not useful if we are going to mix multiple profiles together, like KDE/GNOME. Also currently package manager does not care about the CFLAGS / microarchitecture of the binary package, like from x86-64 to x86-64-v4. Additional metadata and matches need to be added. Thanks, Sheng Yu
Re: [gentoo-dev] Experimental binary package hosting
On Thu, Sep 23, 2021 at 7:12 AM Andreas K. Huettel wrote: > > Hi Vadim, > > > Finally it happened! > > I already planned to try to ask infra/council about sponsoring few > > servers for build farm for "official gentoo binhosts" when I had > > enough time, but fortunately, you've already did that. > > It's very good news. > > Thanks! Nice to see that this is appreciated :) > > So far I'm only using "spare time" on the machine that builds the > releng stages (amd64, x86, m68k, riscv). So no need for a big server > farm. > > > Btw, do you need any help with that? > > I'd be very happy to help with that project. > > Sure! Feel free to add yourself to the Project:Binhost wiki page. I'll > ask for an alias and a channel soon. > > The most useful steps now are only half related to actual building. I > barely know any python and am not very familiar with portage > internals... this is what in my opinion we'd need next: > > 1) a tool to manage and manipulate a binpkg/ directory tree > The main functions that I see needed are > * delete packages/versions that are not in the gentoo repository > anymore (xpak and in index file), maybe with some grace time > * merge xpak files built elsewhere into the directory (also in the > index file) eclean packages from gentoolkit does this exactly.
Re: [gentoo-dev] Experimental binary package hosting
Hi Vadim, > Finally it happened! > I already planned to try to ask infra/council about sponsoring few > servers for build farm for "official gentoo binhosts" when I had > enough time, but fortunately, you've already did that. > It's very good news. Thanks! Nice to see that this is appreciated :) So far I'm only using "spare time" on the machine that builds the releng stages (amd64, x86, m68k, riscv). So no need for a big server farm. > Btw, do you need any help with that? > I'd be very happy to help with that project. Sure! Feel free to add yourself to the Project:Binhost wiki page. I'll ask for an alias and a channel soon. The most useful steps now are only half related to actual building. I barely know any python and am not very familiar with portage internals... this is what in my opinion we'd need next: 1) a tool to manage and manipulate a binpkg/ directory tree The main functions that I see needed are * delete packages/versions that are not in the gentoo repository anymore (xpak and in index file), maybe with some grace time * merge xpak files built elsewhere into the directory (also in the index file) (imagine you have a second container that builds with same CFLAGS, but with use settings for gnome, not plasma... or with updated dependencies because of changes in gentoo.git... you want to merge the trees for distribution without having duplicate builds) 2) binary package cryptographic signing and verification Essentially we need to finish support for GLEP78; this is being worked on in RinCat's pull request https://github.com/gentoo/portage/pull/562 See also https://www.gentoo.org/glep/glep-0078.html 3) an easy way to figure out if a binary package repo is suitable for a profile / arch / ... or not, and a standard for path names This is not so important right now, and partially also already present I guess. The actual builder right now is very simple and wired up with a single daily cron job; the mirrors are only updated manually by me until bug 813528 is handled. Cheers Andreas -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, toolchain, base-system, perl, libreoffice) signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Experimental binary package hosting
Finally it happened! I already planned to try to ask infra/council about sponsoring few servers for build farm for "official gentoo binhosts" when I had enough time, but fortunately, you've already did that. It's very good news. Btw, do you need any help with that? I'd be very happy to help with that project. signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Experimental binary package hosting
Thanks! I never used an external binhost repo so mislead myself. Now I see that it's like a remote analogue of "/usr/portage/packages" and tarballs from remote binhost are fetched on demand into "local binhost" ( /usr/portage/packages/ ) before installation with replacing early presented tarballs. > binhost.conf should be in binrepos.conf instead of repos.conf. See also: > https://wiki.gentoo.org/wiki/Binary_package_guide#Pulling_packages_from_a_binary_package_host > A binhost is different from an ebuild repository, it is just a > collection of .xpak files so there is no git sync involved.
Re:[gentoo-dev] Experimental binary package hosting
Thanks. These settings are works. I just forgot to add USE="bindist" for tested dev-qt/qtwebengine package as mentioned early in mailing list. Great! Now no need to compile qtwebengine for hours next time! :)
Re: [gentoo-dev] Experimental binary package hosting
Am Mittwoch, 22. September 2021, 10:20:10 CEST schrieb Torokhov Sergey: > Sorry for previous html message. I tried to recend it as plaintext. > > I have repos configs placed into /etc/portage/repos.conf with > "rsync-type = git" fo all repos so I created binhost.cond file here > instead of /etc/portage/ as mentioned in blog post. Nope. As the blog post says, you need to put that text into a *file* with name -->>> /etc/portage/binrepos.conf <<<--- ... -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, toolchain, base-system, perl, libreoffice) signature.asc Description: This is a digitally signed message part.
Re:[gentoo-dev] Experimental binary package hosting
Sorry for previous html message. I tried to recend it as plaintext. I have repos configs placed into /etc/portage/repos.conf with "rsync-type = git" fo all repos so I created binhost.cond file here instead of /etc/portage/ as mentioned in blog post. Then I tried to sync and got warnings: ... !!! Section 'binhost' in repos.conf is missing location attribute !!! Repository 'binhost' has sync-uri attribute, but is missing sync-type attribute ... So it seems it's dosn't sync with binhost index without location attribute. After setup location attribute to specific directory I got warning on sync trying: !!! Repository 'binhost' has sync-uri attribute, but is missing sync-type attribute !!! Repository 'binhost' is missing masters attribute in '/usr/local/portage/binhost/metadata/layout.conf' !!! Set 'masters = gentoo' in this file for future compatibility I tried to set up "rsync-type = rsync" or "webrsync" with "amsters = gentoo" but then I got error: !!! Repository 'binhost' is missing masters attribute in '/usr/local/portage/binhost/metadata/layout.conf' !!! Set 'masters = gentoo' in this file for future compatibility So I created /usr/local/portage/binhost/metadata/layout.conf : masters = gentoo auto-sync = false thin-manifests = true So then on sync I got: The current sync-type attribute of repository 'gentoo' is not set to 'rsync' or 'webrsync': sync-type=git Is it currently git sync available for binhost?
Re: [gentoo-dev] Experimental binary package hosting
On 22/09/2021 09:58, Torokhov Sergey wrote: I have repos configs placed into /etc/portage/repos.conf with "rsync-type = git" fo all repos so I created binhost.cond file here instead of /etc/portage/ as mentioned in blog post. binhost.conf should be in binrepos.conf instead of repos.conf. See also: https://wiki.gentoo.org/wiki/Binary_package_guide#Pulling_packages_from_a_binary_package_host Then I tried to sync and got warnings: ... !!! Section 'binhost' in repos.conf is missing location attribute !!! Repository 'binhost' has sync-uri attribute, but is missing sync-type attribute ... So it seems it's dosn't sync with binhost index without location attribute. After setup location attribute to specific directory I got warning on sync trying: !!! Repository 'binhost' has sync-uri attribute, but is missing sync-type attribute !!! Repository 'binhost' is missing masters attribute in '/usr/local/portage/binhost/metadata/layout.conf' !!! Set 'masters = gentoo' in this file for future compatibility I tried to set up "rsync-type = rsync" or "webrsync" with "amsters = gentoo" but then I got error: !!! Repository 'binhost' is missing masters attribute in '/usr/local/portage/binhost/metadata/layout.conf' !!! Set 'masters = gentoo' in this file for future compatibility So I created /usr/local/portage/binhost/metadata/layout.conf : masters = gentoo auto-sync = false thin-manifests = true So then on sync I got: The current sync-type attribute of repository 'gentoo' is not set to 'rsync' or 'webrsync': sync-type=git Is it currently git sync available for binhost? A binhost is different from an ebuild repository, it is just a collection of .xpak files so there is no git sync involved.
Re: [gentoo-dev] Experimental binary package hosting
I have repos configs placed into /etc/portage/repos.conf with "rsync-type = git" fo all reposso I created binhost.cond file here instead of /etc/portage/ as mentioned in blog post. Then I tried to sync and got warnings:...!!! Section 'binhost' in repos.conf is missing location attribute !!! Repository 'binhost' has sync-uri attribute, but is missing sync-type attribute... So it seems it's dosn't sync with binhost index without location attribute. After setup location attribute to specific directory I got warning on sync trying:!!! Repository 'binhost' has sync-uri attribute, but is missing sync-type attribute !!! Repository 'binhost' is missing masters attribute in '/usr/local/portage/binhost/metadata/layout.conf'!!! Set 'masters = gentoo' in this file for future compatibility I tried to set up "rsync-type = rsync" or "webrsync" with "amsters = gentoo" but then I got error:!!! Repository 'binhost' is missing masters attribute in '/usr/local/portage/binhost/metadata/layout.conf' !!! Set 'masters = gentoo' in this file for future compatibility So I created /usr/local/portage/binhost/metadata/layout.conf :masters = gentoo auto-sync = falsethin-manifests = trueSo then on sync I got: The current sync-type attribute of repository 'gentoo' is not set to 'rsync' or 'webrsync': sync-type=git Is it currently git sync available for binhost?21.09.2021, 20:22, "Andreas K. Huettel" :So let's experiment with this... :) announcing:https://gentoo.osuosl.org/experimental/amd64/binpkg/default/linux/17.1/x86-64/More information can be found in a blog post, which will also be on planet.g.osoon:https://dilfridge.blogspot.com/2021/09/experimental-binary-gentoo-package.htmlCheers -A --Andreas K. Hütteldilfri...@gentoo.orgGentoo Linux developer(council, qa, toolchain, base-system, perl, libreoffice)
Re: [gentoo-dev] Experimental binary package hosting
Nice job! On Tue, Sep 21, 2021 at 2:25 PM Andreas K. Huettel wrote: > > Andreas, > > > > How is USE=bindist treated? > > Its on for stage building and off in profiles. > > USE=bindist is switched on, and in addition we have > ACCEPT_RESTRICT="* -bindist" > > -- > Andreas K. Hüttel > dilfri...@gentoo.org > Gentoo Linux developer > (council, qa, toolchain, base-system, perl, libreoffice) >
Re: [gentoo-dev] Experimental binary package hosting
> Andreas, > > How is USE=bindist treated? > Its on for stage building and off in profiles. USE=bindist is switched on, and in addition we have ACCEPT_RESTRICT="* -bindist" -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, qa, toolchain, base-system, perl, libreoffice) signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Experimental binary package hosting
On 2021.09.21 18:22, Andreas K. Huettel wrote: > So let's experiment with this... :) announcing: > https://gentoo.osuosl.org/experimental/amd64/binpkg/default/linux/17.1/x86-64/ > > More information can be found in a blog post, which will also be on > planet.g.o > soon: > https://dilfridge.blogspot.com/2021/09/experimental-binary-gentoo-package.html > > Cheers -A > > -- > Andreas K. Hüttel > dilfri...@gentoo.org > Gentoo Linux developer > (council, qa, toolchain, base-system, perl, libreoffice) > Andreas, How is USE=bindist treated? Its on for stage building and off in profiles. -- Regards, Roy Bamford (Neddyseagoon) a member of elections gentoo-ops forum-mods arm64 pgp5qFocEumHc.pgp Description: PGP signature
[gentoo-dev] Experimental binary package hosting
So let's experiment with this... :) announcing: https://gentoo.osuosl.org/experimental/amd64/binpkg/default/linux/17.1/x86-64/ More information can be found in a blog post, which will also be on planet.g.o soon: https://dilfridge.blogspot.com/2021/09/experimental-binary-gentoo-package.html Cheers -A -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, qa, toolchain, base-system, perl, libreoffice) signature.asc Description: This is a digitally signed message part.
