On 2013-09-03 9:39 PM, Aki Sasaki wrote:
On 9/3/13 8:25 AM, Ehsan Akhgari wrote:
Thanks for the update on this, John! I have a number of questions about
this:
1. On the issue of the hg tags, can you please be more specific about the
problem that you're encountering? In my experience, git deals with updated
tags the same way as it does with any non-fast-forward merge, and I've
never experienced any problems with that.
I'm explicitly not converting most tags, and only whitelisting certain
ones to avoid the issue caused by moving tags.
To quote
https://www.kernel.org/pub/software/scm/git/docs/git-tag.html#_on_re_tagging
on moving git tags:
``Does this seem a bit complicated? It should be. There is no way that
it would be correct to just "fix" it automatically. People need to know
that their tags might have been changed.''
We move tags regularly on hg repos; this is standard operating procedure
for a release build 2, or if a build 1 has an automation hiccup. While
we *could* convert the tags automatically, then either never move them
or move them behind users' backs, users would then never get the updated
tags unless they explicitly delete and re-fetch the tag by name...
something people wouldn't typically do without prompting. In my
opinion, tags pointing at the wrong revision are worse than no tags.
Huh, interesting! I was actually under the impression that tags are
updated in a non-fast-forward manner similar to branches if you want,
but it seems to not be the case given the documentation.
Also, I need to limit the tags pushed to gecko.git, as there is a hard
fastforward-only rule there, and notifying partners to delete and
recreate tags seems like a non-starter. So I built in tag-limiting
whitelists for safety.
However, there appears to be an issue with the way I'm limiting tags.
Rather than delay things further, I decided to publish as-is and see if
anyone really cares about the tags, or if they would be fine using the
tip of each relbranch instead.
Since you bring up the point of tags, can you give examples of how you
use tags, or how you have seen others use tags?
I don't use tags myself at all, I remember someone pointing out that my
repo did not contain the hg tags a long time ago (2+ years) so I
unfortunately don't remember who that person was. That's when I started
pushing the tags as well. I just pointed this out as something that
caught my attention.
2. How frequently are these branches updated?
The current job is running on a 5 minute cron job. We can, of course,
change that if needed. When I add a new repo or a slew of new branches
to convert, the job can take longer to complete, but it typically
finishes in about 6 minutes.
Sounds good, that's exactly the same setup and frequency that I have as
well. It seems to be working fine for most people.
3. What are the plans for adding the rest of the branches that we currently
have in https://github.com/mozilla/mozilla-central, an what is the process
for developers to request their project branches to be added to the
conversion jobs? Right now the process is pretty light-weight (they
usually just ping me on IRC or send me an email). It would be nice if we
kept that property.
There are two concerns here: Project branches are often reset, and also
individual developers care about different subsets of branches.
Providing a core set of branches that everyone uses, and which we can
safely support at scale, seems a good set to include by default. For
users of other branches, one approach we're looking at is to provide
supported documentation that shows developers how to add
any-branch-of-their-choice to their local repo. We're still figuring out
what this default set should be, and as you have clearly expressed
opinions on this in the past, we'd of course be interested in your
current opinions here.
I strongly disagree that providing the current subset of branches is
enough. Firstly, to address your point about project branches being
reset, it's true and it works perfectly fine for the people who are
using those branches, since they will get a non-fast-forward merge which
signals them about this change, which they can choose to avoid if they
want. Also, if somebody gives up their project branch (such as is the
case for twigs) we can always delete the branch from the "main" remote
and doing that will not affect people who have local branches based on
that. Git does the right thing in every case.
About the fact that individual developers care about different subsets
of branches, that's precisely correct, and it's fine, since when using
git, you *always* ignore branches that you're not interested in, and are
never affected by what changes happen in those branches. We have an
extensive amount of experience with the current github mirror about both
of these points and we _know_ that neither of these two are issues here.
Part of the value of having a git mirror for developers is that we don't
require individual people to have their own mirroring infrastructure, so
just providing documentation that tells people how to set up their own
branches by setting up hg-git locally, etc. is kind of defeating the
purpose of having a mirror that helps everybody.
Here's my proposal: the RelEng built git mirror should contains every
one of the branches that the current github mirror contains, and the
process of people requesting new branches to be mirrored in the future
should be as lightweight as possible (i.e., we should never say "no" to
a developer who asks for a new branch to be added to the mirror -- we
should trust them to know which branches are useful to them if they have
a git workflow.) This has served us quite well since the beginning of
the the github mirror.
4. About the interaction between gecko.git and beagle, when you say that
they cannot be identical, are you talking about the location where they're
hosted or the SHA1s for the same commits? I think we need to guarantee the
latter being identical going forward, but the former doesn't matter as much.
It's a requirement to keep the SHA1s the same, and we are confident we
can do that.
The location and naming for legacy b2g branches will be different, as
well as a hard fastforward-only rule on gecko.git.
That sounds fine.
One point to note here is that because of reasons which I have not
explored, some of our branches such as aurora and beta are regularly
updated in a non-fast-forward manner. You can see this if you run git
fetch in a local clone (there will be a "(forced update)" next to the
name of the branch in the output.) You might want to investigate why
that happens in case B2G starts to follow regular release trains in the
future (but please also note that it might just be a bug in my existing
setup.)
5. Is the git-mapfile generated from the conversion scrips available
somewhere? (Mine is currently available here:
http://git.mozilla.org/?p=users/eakhg...@mozilla.com/mozilla-history-tools.git;a=history;f=updates/git-mapfile;h=5c8ecf4bf1d7339ad1dff1ea4ab62a712da2fb6c;hb=HEAD
)
I'm uploading this per run, but it's currently not going anywhere
publicly accessible.
We'll change the location of the upload to a public location before
signing off on this repo.
Sounds good! Just to confirm, do you end up with a single git-mapfile
in your setup? i.e., do you run hg-git only on a single local clone,
and just pull different hg branches into it, differentiated by bookmarks?
6. Do you have any plans for how we're going to approach the problem of
migrating the current git mirror with beagle? Specifically, I think we
need to:
a) Figure out how github deals with a repository being deleted/recreated,
would that break people's forks, stars, etc.?
b) Are there any alternative solutions in replacing all of the
branches/tags under mozilla/mozilla-central with the new repo?
c) What is the developer work flow going to look like in rebasing the local
branches once the move has been completed?
This is a one time hit, but obviously, we want the least churn and
disruption possible. We're still putting that plan together, so all good
questions to add to the planning criteria.
Sounds good!
First, lets get feedback that the test-beagle repos looks good and can
be signed off.
What kind of feedback are you interested in here? Just comparing the
contents of the repo on the tip of the branches can be performed using
md5sum.
Cheers,
Ehsan
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
