Re: [eigen] Bitbucket migration
> This also remind me that we'll have to do something about this git-mirror. We > can either: > (1) - make it empty with a link to the new git repo > (2) - keep it as is (i.e., no sync) for a short period and then proceed as (1) > (3) - delete it and recreate it as a synced mirror of the new repo, and then > after a short period proceed as (1) > If I'm not mistaken option (3) will probably break all users of this mirror, > so that's probably not the best option! I am one of the users of the current github mirror, and yes, as I understand it option 3 isn’t really an option because of the re-written history. I see no point in it. I think option (1) is best. I look forward to swapping over to the gitlab mirror. Toby
Re: [eigen] Bitbucket migration
On Wed, Sep 18, 2019 at 9:10 AM Christoph Hertzberg < c...@informatik.uni-bremen.de> wrote: > I guess the main decision left is to decide on a schedule when to > officially switch to gitlab and if/how long we will keep the bitbucket > repository as a mirror (or essentially as a snapshot) > What about trying to sync it with new releases, e.g.: - Releasing a 3.3.x within a few days and take this opportunity to announce the migration to Gitlab.com on the "...". - Do the migration. - Release 3.4-beta right after to celebrate the migration. Maybe we could target the 8 of October? This should give us enough time to fix the very few true 3.4 blockers. We also need to create a group for Eigen on gitlab.com, since "eigen" is already taken (and I got no answer from the owner despite one reminder), we have to chose something else. On github, I created "eigenteam" to hold the current git-mirror, but we can think of: - eigenteam - eigen-team - eigenlib - libeigen - eigen-official I think that keeping the current hg repository sync with the new git repo is going to be a nightmare (because we rewrote the history), so I would probably keep it as a snapshot (unless someone come up we a working solution). This also remind me that we'll have to do something about this git-mirror. We can either: (1) - make it empty with a link to the new git repo (2) - keep it as is (i.e., no sync) for a short period and then proceed as (1) (3) - delete it and recreate it as a synced mirror of the new repo, and then after a short period proceed as (1) If I'm not mistaken option (3) will probably break all users of this mirror, so that's probably not the best option! Once bitbucket closes hg-support (or sooner), we can replace our current > main-repo by an otherwise empty git repository which just links to the > new repository (or make a git-mirror, but I don't see any benefits here). > good idea for the empty git repo. > Also, we should probably host the "last official" state of the mercurial > repository somewhere (as read-only). > sure. gael > > Christoph > > > On 18/09/2019 00.30, Gael Guennebaud wrote: > > Thanks to Joseph's work, and after fighting with Gitlab.com's super > > aggressive spam filter, I finally managed to get the following Gitlab.com > > project as a demo of what could be the outcome of a full migration: > > > > https://gitlab.com/ggael/eigen-migration6 > > > > This project includes: > > - a git repo with bug ids, commit hashes, and pull-request links > updated > > in commit message. > > - a migration of all our bugzilla entries and comments with similar > > updates as the commit messages. > > > > Some examples: > > - attachments, links to commits and bugs: > > https://gitlab.com/ggael/eigen-migration6/issues/1305 > > - link to PR: > > https://gitlab.com/ggael/eigen-migration6/issues/1306#note_218167101 > > - migration of our "3.x" bug entries as milestones: > > https://gitlab.com/ggael/eigen-migration6/-/milestones/6 > > - link to PR from commits: > > > https://gitlab.com/ggael/eigen-migration6/commit/97f1e1d89ff759186f4a9c866f7ea3afa7b4c325 > > - link to bug from commits: > > > https://gitlab.com/ggael/eigen-migration6/commit/c06e6fd115d747c42a2b2ea029c53bbdf41276d6 > > > > The bugzilla to gitlab migration script is there: > > https://gitlab.com/ggael/bztogl (adapted from > > https://gitlab.gnome.org/World/bztogl) > > > > PS1: If you notice many extra newlines in issue descriptions/comments, > > that's normal, I already fixed this shortcoming. > > > > What do you think ? > > > > gael > > > > > > > > On Wed, Sep 11, 2019 at 6:03 PM Gael Guennebaud < > gael.guenneb...@gmail.com> > > wrote: > > > >> To prepare the migration from bitbucket, I started to play a bit with > its > >> API to see what could be done. So far I've quickly draft two (ugly) > python > >> scripts to archive the forks and pull-requests. Since this is a one shot > >> for us, I did not cared about robustness, safety, generality, beauty, > etc. > >> > >> You can see them there : > >> https://gitlab.com/ggael/bitbucket-migration-tools and contribute! > >> > >> ** Forks ** > >> > >> You can see the summary of the fork script there: > >> http://manao.inria.fr/eigen_tmp/archive_forks_log.html > >> > >> The hg clones (history+checkout) represents 20GB, maybe 12GB if we > remove > >> the checkouts. Among the 460 forks, 214 seems to have no change at all > >> (according to "hg out") and could be dropped. I don't know yet where to > >> host them though. > >> > >> This script can be ran incrementally. > >> > >> > >> ** Pull-Requests ** > >> > >> You can find the output of the pull-requests script there: > >> http://manao.inria.fr/eigen_tmp/pullrequests/ > >> > >> There is a short summary, and then for each PR a static .html file plus > >> diff/patch files, and other details. For instance, see: > >> http://manao.inria.fr/eigen_tmp/pullrequests/OPEN/686/pr686.html > >> > >> Currently this script cannot be ran incrementally. You have to run it >
Re: [eigen] Bitbucket migration
Very nice indeed! Thanks Gael and Joseph! I guess the main decision left is to decide on a schedule when to officially switch to gitlab and if/how long we will keep the bitbucket repository as a mirror (or essentially as a snapshot). Once bitbucket closes hg-support (or sooner), we can replace our current main-repo by an otherwise empty git repository which just links to the new repository (or make a git-mirror, but I don't see any benefits here). Also, we should probably host the "last official" state of the mercurial repository somewhere (as read-only). Christoph On 18/09/2019 00.30, Gael Guennebaud wrote: Thanks to Joseph's work, and after fighting with Gitlab.com's super aggressive spam filter, I finally managed to get the following Gitlab.com project as a demo of what could be the outcome of a full migration: https://gitlab.com/ggael/eigen-migration6 This project includes: - a git repo with bug ids, commit hashes, and pull-request links updated in commit message. - a migration of all our bugzilla entries and comments with similar updates as the commit messages. Some examples: - attachments, links to commits and bugs: https://gitlab.com/ggael/eigen-migration6/issues/1305 - link to PR: https://gitlab.com/ggael/eigen-migration6/issues/1306#note_218167101 - migration of our "3.x" bug entries as milestones: https://gitlab.com/ggael/eigen-migration6/-/milestones/6 - link to PR from commits: https://gitlab.com/ggael/eigen-migration6/commit/97f1e1d89ff759186f4a9c866f7ea3afa7b4c325 - link to bug from commits: https://gitlab.com/ggael/eigen-migration6/commit/c06e6fd115d747c42a2b2ea029c53bbdf41276d6 The bugzilla to gitlab migration script is there: https://gitlab.com/ggael/bztogl (adapted from https://gitlab.gnome.org/World/bztogl) PS1: If you notice many extra newlines in issue descriptions/comments, that's normal, I already fixed this shortcoming. What do you think ? gael On Wed, Sep 11, 2019 at 6:03 PM Gael Guennebaud wrote: To prepare the migration from bitbucket, I started to play a bit with its API to see what could be done. So far I've quickly draft two (ugly) python scripts to archive the forks and pull-requests. Since this is a one shot for us, I did not cared about robustness, safety, generality, beauty, etc. You can see them there : https://gitlab.com/ggael/bitbucket-migration-tools and contribute! ** Forks ** You can see the summary of the fork script there: http://manao.inria.fr/eigen_tmp/archive_forks_log.html The hg clones (history+checkout) represents 20GB, maybe 12GB if we remove the checkouts. Among the 460 forks, 214 seems to have no change at all (according to "hg out") and could be dropped. I don't know yet where to host them though. This script can be ran incrementally. ** Pull-Requests ** You can find the output of the pull-requests script there: http://manao.inria.fr/eigen_tmp/pullrequests/ There is a short summary, and then for each PR a static .html file plus diff/patch files, and other details. For instance, see: http://manao.inria.fr/eigen_tmp/pullrequests/OPEN/686/pr686.html Currently this script cannot be ran incrementally. You have to run it just before closing the respective repository! Also, this script does not grab inline comments. Only the main discussions is archived. Those can be obtained by iterating over the "activity" pages, but I don't think that's worth the effort because they would be difficult to exploit anyway. ** hg to git ** As discussed in the other thread, if we switch from hg to git, then all hashes will have to be updated. Generating a map file is easy, and thus updating the links/hashes in bug comments and PR comments should not be too difficult (we only have to figure out the right regex to catch all variants). However, updating the hashes within the commit messages will require to rewrite the whole history in a careful order. Does anyone here feels brave enough to write such a script? If not, I guess we could live with an online php script doing the hash conversion on demand. I don't think we'll have to follow such hashes so frequently. cheers, gael -- Dr.-Ing. Christoph Hertzberg Besuchsadresse der Nebengeschäftsstelle: DFKI GmbH Robotics Innovation Center Robert-Hooke-Straße 5 28359 Bremen, Germany Postadresse der Hauptgeschäftsstelle Standort Bremen: DFKI GmbH Robotics Innovation Center Robert-Hooke-Straße 1 28359 Bremen, Germany Tel.: +49 421 178 45-4021 Zentrale: +49 421 178 45-0 E-Mail: christoph.hertzb...@dfki.de Weitere Informationen: http://www.dfki.de/robotik - Deutsches Forschungszentrum für Künstliche Intelligenz GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany Geschäftsführung: Prof. Dr. Jana Koehler (Vorsitzende) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Amtsgericht Kaiserslautern, HRB 2313
