Hello my fellow GNUnetians, As some of you know, I have been pushing for and working on getting us to CI/CD system based on Gitlab CI. This is pretty much ready to start using and building out the pipelines for now.
In a previous thread[0] there was a decision to give Gitlab a try, as more than just our CI system, and to migrate our repos from Gitolite to our own Gitlab instance. There was less agreement regarding the idea to move from MantisBT to Gitlab's issue tracker, but in the end we decided to try importing all of the >2k Mantis tickets into Gitlab.[1] This email is a status update on the overall progress of moving to Gitlab. Current Gitlab status: - Gitlab is running and accessible at `https://gitlab.gnunet.org` - Registration is open. There are no guarantees on uptime, or even data retention (though I don't expect data to disappear). - wldhx has kindly offered two "Gitlab Runners" for running CI jobs. These will be added as shared runners, to be used by any projects on the instance. This may be changed later to only be shared by projects under the GNUnet namespace. **TODO** - Setup email. Used for registration, password resets, notifcations, and interaction (eg. issue threads). - Currently run using containers with docker-compose. Will switch to using systemd services to with the containers, removing docker-compose. - Daily remote backups. Perhaps 'firefly.gnunet.org' could be the backup site, hmm? - Change configured hostname (in Gitlab) to 'gitlab.gnunet.org'. - Setup redirect from 'git.gnunet.org' -> 'gitlab.gnunet.org' ---- Current [MantisBT -> Gitlab Issues] status: Exporting: - Mantis can export to CSV and "Excell" XML - These do not contain comments (bugnotes). It looks like there might be a possibility to enable them via a configuration option[2]. Not sure who all has admin access, whom I could coordinate with. Maybe easier if I can get admin rights? Grothoff, what do you think? Importing: - I have found only a couple of scripts[3,4] for this. They are both out of date, for old versions of both softwares. I have tried both to no avail. [4] is the most promising; It's not so old. I would really appreciate any help working on this. I suppose this means that we will continue to use Mantis, and disable issues in Gitlab for now. Any protests or ideas? ---- Migrating from Gitolite: For those whom are not aware, we currently use gitolite for all of the lovely repos in our collection. We will need to copy all of the repos to Gitlab, as well as duplicate permissions, and setup githooks. 1. Create namespaces/groups on our Gitlab 2. Clone repos. This can be done via the web interface "Import" option when creating a new repo, or the new remote can be added, and then pushed. The little script found here can help with getting all the repos from Gitolite[5] 3. Setup redirects. eg. https://gnunet.org/git -> https://gitlab.gnunet.org 4. Manually replicate permissions. Will need a Gitolite admin's help on this. 5. Setup githooks. We have quite a few githooks setup, so we will need to recreate those. After all of that is done, I think we should be ready to switch over to Gitlab for at least the git management and CI/CD. ---- That brings us to the final update: The CI System... - We have a couple of small runners (thanks wldhx). - We have some very basic '.gitlab-ci.yml'[6] files, defining jobs. - I will begin expanding these in the coming weeks. **TODO** - As we build out a matrix of pipelines, we will need more resources. 'firefly.gnunet.org' is a logical option. In the past I've seen it utilized heavily by experiments. As long as we are okay with dedicating some CPU and RAM to runners, then I will begin setting them up. - Setup Gitlab Container Registry [7] for storing our CI artifacts. - Expand our '.gitlab-ci.yml' files to include e2e tests, builds for multiple architecures, and continuous delivery of packages for various package managers. ---- Wow, so that's a lot of text. A lot of people have been asking me about the status of Gitlab, and if and how they can help with CI. I hope this gives people a thorough update, and answers. I really believe Gitlab can be a useful software suite, despite its shortcomings. My hopes are that it will help increase the feedback loop between development and testing, as well as make it easier and more welcoming for new contributors. Be well, and Happy Hacking! - Devan [0] https://lists.gnu.org/archive/html/gnunet-developers/2019-01/msg00071.html [1] https://lists.gnu.org/archive/html/gnunet-developers/2019-01/msg00082.html [2] http://www.mantisbt.org/docs/master/en-US/Admin_Guide/html-desktop/#admin.config.display [3] https://gitlab.kitware.com/utils/mantis-to-gitlab [4] https://github.com/timwiel/mantis2gitlab [5] https://tutorials.technology/tutorials/89-show-all-repositories-of-a-git-server.html [6] https://docs.gitlab.com/ce/ci/quick_start/ [7] https://docs.gitlab.com/ce/user/project/container_registry.html
signature.asc
Description: PGP signature
_______________________________________________ GNUnet-developers mailing list [email protected] https://lists.gnu.org/mailman/listinfo/gnunet-developers
