Re: [Emc-developers] Buildbot
Please correct me if I am wrong. The current buildbot hosted and mentioned by Seb: The buildbot currently consists of ~20 VMs, each with 2 CPUs, 4 GB RAM, and ~20 GB disk. The buildmaster has 200 GB disk, mostly to host the package archive some of those VMs can be retired when we EOL 2.8 The buildmaster needs 2 CPUs, 2+ GB RAM, 200 GB disk, and a fast network connection. it needs to accept TCP connections on an arbitrary port from all the build workers. The buildbot uses the standard debian toolchain for building/testing/packaging, the only custom stuff is a few pages of shell scripts to manage the debian package archive at blo/dists and the buildmaster config itself (which it a bit like the github actions file, it says how to do each step) Some VMs will need to be added to support bullseye, bookworm, and sid terminology: "one buildbot" consists of "one buildmaster" and "one or more build workers". We only need one buildbot, and it only needs to contain one buildmaster, but it needs one build worker for each platform we want to support if we're testing kernel-independent userspace stuff then we can get away with running the build workers in containers or chroots or whatever, so we don't need one VM per build worker But for kernel-mode RTAI we absolutely need a VM for each build worker and if we want to test the realtime-specific parts of rtapi_app, then we need a VM for each build worker Alec (ntulinux, RTAI dev) said that he does not need Sebs RTAI scripts for a buildbot anymore due to the changes in RTAI and LCNC. Rod W. was offered a free vps for hosting a buildbot.for lcnc turboss has offered the use of his server that he currently uses for hosting qtpyvcp as a buildbot for lcnc jt is working on a buildbot at his location On 11/15/22 12:30, John Thornton wrote: Seb, I've been working with buildbot 3.6.1 and have it building a deb for me. Next task for me is to figure out how to setup a remote builder. On 11/15/2022 12:02 PM, Sebastian Kuzminsky wrote: On 11/14/22 18:43, m...@mattshaver.com wrote: On 2022-11-14 15:54, Sebastian Kuzminsky wrote: In order to support testing LinuxCNC on RTAI the buildbot uses custom VMs which i created a long time ago and maintain poorly. It all runs on hardware in my house, on my local network, so I am reluctant to open it up to a more collaborative mode of development/maintenance. Is there some way of replicating your home based setup (or obtaining a compatible hardware/software environment) by contracting with a commercial hosting company, or co-location facility, or something like that? If this could be done for a reasonable amount of money, I'd be happy to pay for it, or at least chip in. I can't do much to help, but I can spend money! Hosting the hardware is relatively little effort. Most of the effort is in the initial creation of new builder VMs (i.e. to build on new platforms), and ongoing updates to the buildmaster. These tasks would still fall on us of the LinuxCNC project, rather than on whatever hosting provider we use. The lack of builders newer than Buster in the buildbot is due to a version-skew issue within the buildbot project: to support newer builders I need to upgrade the buildmaster from the current (ancient) 0.8 version to the new 2.x version. Again, this is work that a hosting provider can't help with. I have been working on it on and off, and I've made some progress, but it's not ready to deploy yet. ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
[Emc-developers] linuxcnc is marked for autoremoval from testing
linuxcnc 2.9.0~pre0+git20221105.ffb6bda926-1.2 is marked for autoremoval from testing on 2022-12-20 It (build-)depends on packages with these RC bugs: 1023548: pyopengl: migrate to libglut3.12 https://bugs.debian.org/1023548 This mail is generated by: https://salsa.debian.org/release-team/release-tools/-/blob/master/mailer/mail_autoremovals.pl Autoremoval data is generated by: https://salsa.debian.org/qa/udd/-/blob/master/udd/testing_autoremovals_gatherer.pl ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] 2.9 Release Manager Required.
Le mar. 15 nov. 2022 à 19:14, Sebastian Kuzminsky a écrit : > > Hosting the hardware is relatively little effort. But it has cost, real estate, and environmental impact potentially freely offloadable to dedicated, optimized, on-demand alternative. > Most of the effort is in the initial creation of new builder VMs (i.e. > to build on new platforms), and ongoing updates to the buildmaster. > > These tasks would still fall on us of the LinuxCNC project, rather than > on whatever hosting provider we use. I can't see any more tasks on us than creating the containers/VMs definitions and setting up the build pipeline. This is one of the selling points of moving to a build service ! ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
[Emc-developers] Buildbot
Seb, I've been working with buildbot 3.6.1 and have it building a deb for me. Next task for me is to figure out how to setup a remote builder. On 11/15/2022 12:02 PM, Sebastian Kuzminsky wrote: On 11/14/22 18:43, m...@mattshaver.com wrote: On 2022-11-14 15:54, Sebastian Kuzminsky wrote: In order to support testing LinuxCNC on RTAI the buildbot uses custom VMs which i created a long time ago and maintain poorly. It all runs on hardware in my house, on my local network, so I am reluctant to open it up to a more collaborative mode of development/maintenance. Is there some way of replicating your home based setup (or obtaining a compatible hardware/software environment) by contracting with a commercial hosting company, or co-location facility, or something like that? If this could be done for a reasonable amount of money, I'd be happy to pay for it, or at least chip in. I can't do much to help, but I can spend money! Hosting the hardware is relatively little effort. Most of the effort is in the initial creation of new builder VMs (i.e. to build on new platforms), and ongoing updates to the buildmaster. These tasks would still fall on us of the LinuxCNC project, rather than on whatever hosting provider we use. The lack of builders newer than Buster in the buildbot is due to a version-skew issue within the buildbot project: to support newer builders I need to upgrade the buildmaster from the current (ancient) 0.8 version to the new 2.x version. Again, this is work that a hosting provider can't help with. I have been working on it on and off, and I've made some progress, but it's not ready to deploy yet. ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] 2.9 Release Manager Required.
On 11/14/22 18:43, m...@mattshaver.com wrote: On 2022-11-14 15:54, Sebastian Kuzminsky wrote: In order to support testing LinuxCNC on RTAI the buildbot uses custom VMs which i created a long time ago and maintain poorly. It all runs on hardware in my house, on my local network, so I am reluctant to open it up to a more collaborative mode of development/maintenance. Is there some way of replicating your home based setup (or obtaining a compatible hardware/software environment) by contracting with a commercial hosting company, or co-location facility, or something like that? If this could be done for a reasonable amount of money, I'd be happy to pay for it, or at least chip in. I can't do much to help, but I can spend money! Hosting the hardware is relatively little effort. Most of the effort is in the initial creation of new builder VMs (i.e. to build on new platforms), and ongoing updates to the buildmaster. These tasks would still fall on us of the LinuxCNC project, rather than on whatever hosting provider we use. The lack of builders newer than Buster in the buildbot is due to a version-skew issue within the buildbot project: to support newer builders I need to upgrade the buildmaster from the current (ancient) 0.8 version to the new 2.x version. Again, this is work that a hosting provider can't help with. I have been working on it on and off, and I've made some progress, but it's not ready to deploy yet. -- Sebastian Kuzminsky ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] 2.9 Release Manager Required.
On 11/14/22 16:58, Rod Webster wrote: Perhaps the Github actions could be extended to build the 2.9 branch? Our current Github Actions already builds & tests all branches (but does not publish any packages). -- Sebastian Kuzminsky ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] linuxcnc.org/dists
Andy has tried to help but I still don't have sftp to http://linuxcnc.org/dists/ , can anyone help? JT On 11/14/2022 8:49 AM, John Thornton wrote: If someone can supply me with the credentials for http://linuxcnc.org/dists/ I can build debs and upload them via sftp for the ones that the current buildbot does not create. JT ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] 2.9 Release Manager Required.
>It may well be able to build the live ISO too, but I'm still not sure about that... I can't see why not, the code is there to make debs (and your docs!) so we can surely just script that into the workflow. Rod Webster *1300 896 832* +61 435 765 611 Vehicle Modifications Network www.vehiclemods.net.au On Tue, 15 Nov 2022 at 21:45, Rod Webster wrote: > Jérémie > Thats great! I also found this shortcut! > > https://linuxkamarada.com/en/2019/03/19/integrating-the-open-build-service-with-github/#.Y3N6yj1By3A > The repo for osc is in Debian too, Unfortunately it uses some python that > was deprecated in V 3.9 so you'd have to run it on Buster I think. I tried > in Bookworm... > Rod Webster > *1300 896 832* > +61 435 765 611 > Vehicle Modifications Network > www.vehiclemods.net.au > > > On Tue, 15 Nov 2022 at 20:59, Jérémie Tarot wrote: > >> Le mar. 15 nov. 2022 à 01:03, Rod Webster a >> écrit : >> > >> > I'll just wait until you guys decide on a way forward and iff the offer >> we >> > have is required. >> > Perhaps the Github actions could be extended to build the 2.9 branch? >> >> I don't see what would prevent us to build whatever branches we'd want to >> !? >> >> > If that were possible, maybe the VM here does the RTAI build? >> >> That remains possible with the GitHub actions thanks to the >> self-hosted runners feature: >> https://github.com/actions/runner >> https://docs.github.com/en/actions/hosting-your-own-runners >> https://github.com/actions/runner/blob/main/docs/start/envlinux.md >> >> > I'm not sure git will host the resulting debs >> >> Sadly, the GitHub Packages feature only supports programming languages >> modules packages. >> This is why I looked for an hosting solution and found PackageCloud.io >> which does just that. >> >> > but it might be worth mentioning the Open build Service I linked to >> earlier can be incorporated >> > right into sources.list with the appropriate security keys. >> >> It is indeed ! Can't believe I didn't remember it and even that it >> didn't pop up in the many searches I did :/ >> This could actually be the best option to keep all builds (RTAI >> included), as it looks like OBS supports custom-built containers and >> VMs, and package hosting all in the same place. It may well be able to >> build the live ISO too, but I'm still not sure about that... >> It also supports GitHub integration via webhooks and status reporting, >> as well as it is Open Source and self-hostable, and even available in >> Debian... what else ?! :) >> >> https://openbuildservice.org/help/manuals/obs-user-guide/cha.obs.scm_ci_workflow_integration.html >> >> OBS repositories could at least be mirrored/synced to >> http://linuxcnc.org/dists/ if no more elegant solution is available >> >> WDYT ? >> >> PS: Brother projects like MesaCT, QtPyVCP, etc. packages could also be >> built under LinuxCNC umbrella there too >> >> >> ___ >> Emc-developers mailing list >> Emc-developers@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/emc-developers >> > ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] 2.9 Release Manager Required.
Jérémie Thats great! I also found this shortcut! https://linuxkamarada.com/en/2019/03/19/integrating-the-open-build-service-with-github/#.Y3N6yj1By3A The repo for osc is in Debian too, Unfortunately it uses some python that was deprecated in V 3.9 so you'd have to run it on Buster I think. I tried in Bookworm... Rod Webster *1300 896 832* +61 435 765 611 Vehicle Modifications Network www.vehiclemods.net.au On Tue, 15 Nov 2022 at 20:59, Jérémie Tarot wrote: > Le mar. 15 nov. 2022 à 01:03, Rod Webster a > écrit : > > > > I'll just wait until you guys decide on a way forward and iff the offer > we > > have is required. > > Perhaps the Github actions could be extended to build the 2.9 branch? > > I don't see what would prevent us to build whatever branches we'd want to > !? > > > If that were possible, maybe the VM here does the RTAI build? > > That remains possible with the GitHub actions thanks to the > self-hosted runners feature: > https://github.com/actions/runner > https://docs.github.com/en/actions/hosting-your-own-runners > https://github.com/actions/runner/blob/main/docs/start/envlinux.md > > > I'm not sure git will host the resulting debs > > Sadly, the GitHub Packages feature only supports programming languages > modules packages. > This is why I looked for an hosting solution and found PackageCloud.io > which does just that. > > > but it might be worth mentioning the Open build Service I linked to > earlier can be incorporated > > right into sources.list with the appropriate security keys. > > It is indeed ! Can't believe I didn't remember it and even that it > didn't pop up in the many searches I did :/ > This could actually be the best option to keep all builds (RTAI > included), as it looks like OBS supports custom-built containers and > VMs, and package hosting all in the same place. It may well be able to > build the live ISO too, but I'm still not sure about that... > It also supports GitHub integration via webhooks and status reporting, > as well as it is Open Source and self-hostable, and even available in > Debian... what else ?! :) > > https://openbuildservice.org/help/manuals/obs-user-guide/cha.obs.scm_ci_workflow_integration.html > > OBS repositories could at least be mirrored/synced to > http://linuxcnc.org/dists/ if no more elegant solution is available > > WDYT ? > > PS: Brother projects like MesaCT, QtPyVCP, etc. packages could also be > built under LinuxCNC umbrella there too > > > ___ > Emc-developers mailing list > Emc-developers@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/emc-developers > ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] 2.9 Release Manager Required.
Le mar. 15 nov. 2022 à 01:03, Rod Webster a écrit : > > I'll just wait until you guys decide on a way forward and iff the offer we > have is required. > Perhaps the Github actions could be extended to build the 2.9 branch? I don't see what would prevent us to build whatever branches we'd want to !? > If that were possible, maybe the VM here does the RTAI build? That remains possible with the GitHub actions thanks to the self-hosted runners feature: https://github.com/actions/runner https://docs.github.com/en/actions/hosting-your-own-runners https://github.com/actions/runner/blob/main/docs/start/envlinux.md > I'm not sure git will host the resulting debs Sadly, the GitHub Packages feature only supports programming languages modules packages. This is why I looked for an hosting solution and found PackageCloud.io which does just that. > but it might be worth mentioning the Open build Service I linked to earlier > can be incorporated > right into sources.list with the appropriate security keys. It is indeed ! Can't believe I didn't remember it and even that it didn't pop up in the many searches I did :/ This could actually be the best option to keep all builds (RTAI included), as it looks like OBS supports custom-built containers and VMs, and package hosting all in the same place. It may well be able to build the live ISO too, but I'm still not sure about that... It also supports GitHub integration via webhooks and status reporting, as well as it is Open Source and self-hostable, and even available in Debian... what else ?! :) https://openbuildservice.org/help/manuals/obs-user-guide/cha.obs.scm_ci_workflow_integration.html OBS repositories could at least be mirrored/synced to http://linuxcnc.org/dists/ if no more elegant solution is available WDYT ? PS: Brother projects like MesaCT, QtPyVCP, etc. packages could also be built under LinuxCNC umbrella there too ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] 2.9 Release Manager Required.
Le lun. 14 nov. 2022 à 22:10, John Thornton a écrit : > > This sounds like a good long term solution if it's free. > I'm only looking at free (beer) options, at least as long as LinuxCNC doesn't have a registered organization to receive money and subscribe to services in its own name. ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
Re: [Emc-developers] 2.9 Release Manager Required.
Le lun. 14 nov. 2022 à 21:49, andy pugh a écrit : > > On Mon, 14 Nov 2022 at 20:42, Jérémie Tarot wrote: > > > Could someone explain me how relocating the buildbot on someone's > > machine can be better for the project than migrating it to GitHub > > Actions, build farm and package hoster ? > > What would the user install instructions look like in that scenario? > I'm looking for options that preferably don't change anything to current admin/integrator procedures. Meaning, at least, built packages should end in proper repositories that can be added to package sources. Debian is a priority, but I think people are working on Ubuntu, Fedora, and Gentoo. Ideally, http://linuxcnc.org/dists/ should remain operational and migration should be seamless for users, but may need some DNS and web redirection trickery... ___ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
