Re: Git messes up 'ø' character

2015-01-20 Thread Nico Williams
On Tue, Jan 20, 2015 at 10:38:40PM +0100, Noralf Trønnes wrote:
 Yes:
 $ echo Noralf Trønnes | xxd
 000: 4e6f 7261 6c66 2054 72f8 6e6e 6573 0aNoralf Tr.nnes.
 
 Is there a command I can run that shows that I'm using ISO-8859-1 ?
 I need something to google with, my previous search only gave locale
 stuff, which seems fine.

The locale(1) command tells you what your locale is set to, but it
doesn't say anything about your input method -- it only tells you what
your shell and commands started from it expect for input and what they
should produce for output.

The input method will generally be part of your windowing environment,
for which you'll have to search how to check/configure your OS
(sometimes it can be set on a per-window basis, sometimes it's a global
setting).

Even if the windowing environment is set to UTF-8, your terminal
emulator might be set to ISO-8859-something, so check the terminal
emulator (e.g., rxvt, Terminator, GNOME Terminal, PuTTY, ...).

Finally, check what stty(1) says (e.g., on Linux it should show that
iutf8 is enabled) (this is mostly so that when you backspace in cooked
mode the line discipline knows how many bytes to drop from the buffer).

Nico
-- 
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

2014-11-24 Thread Nico Williams
Is there a plan for upgrading to a better hash function in the future?
 (E.g., should it become an urgent need.)

What are the roadblocks to adoption of a replacement hash function?
Just documenting this would go a long way towards making it possible
to upgrade some day.

Thanks,

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 00/18] Signed push

2014-08-20 Thread Nico Williams
No code == no substance might be a stretch, but definitely fair enough.

I thought the idea was clear enough, but I can flesh it out if
desired.  The particular advantage I saw in it is that it would reuse
the existing object infrastructure, and extend to branches the
first-class treatment that [signed] tags already get.  I.e.,
generality.  Other benefits include the ability to fetch and view a
remote branch's object and its history (which would represent branch
history in detail and with metadata that would otherwise not be
available).

I'm not interested in pushing something different when you already
have code that would achieve much of what I'd wanted.  At this point
my interest is in seeing if the architecture would be purer (in some
sense) by reusing existing infrastructure.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 00/18] Signed push

2014-08-19 Thread Nico Williams
On Tue, Aug 19, 2014 at 03:06:09PM -0700, Junio C Hamano wrote:
 While signed tags and commits assert that the objects thusly signed
 came from you, who signed these objects, there is not a good way to
 assert that you wanted to have a particular object at the tip of a
 particular branch.  My signing v2.0.1 tag only means I want to call
 the version v2.0.1, and it does not mean I want to push it out to my
 'master' branch---it is likely that I only want it in 'maint', so
 the signature on the object alone is insufficient.
 
 [...]
 
 This series introduces a cryptographic assurance for ref updates
 done by git push by introducing a mechanism that allows you to
 sign a push certificate (for the lack of better name) every time
 you push.  Think of it as working on an axis orthogonal to the
 traditional signed tags.

Sounds a lot like the branch object concept I suggest earlier, where
each push would also push a commit to a branch object describing the
updates to the branch, including signing of the updates to the branch
(hey, it's just a signed commit), groups of commits pushed together / to
be backed out together, rebase history, ...  (What about pushing
orphaned commits?)

Code-wise, would that be more or less generic?

Nico
-- 
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Sharing merge conflict resolution between multiple developers

2014-08-11 Thread Nico Williams
IIUC, this might help,

http://www.mail-archive.com/git@vger.kernel.org/msg56418.html
http://www.mail-archive.com/git@vger.kernel.org/msg56468.html
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Rebase safely (Re: cherry picking and merge)

2014-08-08 Thread Nico Williams
On Fri, Aug 08, 2014 at 10:34:43AM -0700, Mike Stump wrote:
 On Aug 6, 2014, at 10:11 PM, Nico Williams n...@cryptonector.com wrote:
  Nah.  Sun managed this for decades without a hitch, and for products
  much larger than GCC.  See above.
 
 Ok.  Ah, ok, perfect.  I see how that method of working would cure the
 cherry-pick and merge don’t work problem mentioned at the top of the
 thread.
 
  Do some experiments based on the above hardcopy.  If that doesn't
  convince you that it works, oh well, I'll have given it a good try.
 
 Thank you for taking the time to diagram that as it appears to violate
 everyones how to use git guide.   I see the workflow does an onto,
 which was the ‘fix’ people talked about on stack overflow, and I see
 just how things would work.

There's nothing scary about --onto.  You're saying figure out which are
my local commits (the ones on top of the previous upstream) and pick
them onto the new upstream.

We only need to do it manually (though it can be scripted[*]) because
git doesn't track rebase history so that it can be done automatically.

[*] And then there's Tony Finch's
https://git.csx.cam.ac.uk/x/ucs/git/git-repub.git , which is kinda
awesome!

 If the old master branches are deleted and gc is run, then all the old
 references go away, and then the refs from email and bugzilla then
 don’t work.  Did you guys ever remove them and then prune (or gc)?

Product gates' repos and snapshots stuck around forever, though it was
Teamware, and finding really old ones wasn't necessarily easy,
particularly since their names didn't always reflect product names.

Prominent project gate repos and their snapshots also stuck around
forever.

Lesser project gate repos tended to be as ephemeral as the project.

 Now, the biggest issue, if that is recognized as `fixing’ the
 cherry-pick problem, then certainly the problem is understood to be a
 problem.  If one recognized it as a problem, then one can envision

Not really.  This isn't about git.  We followed a rebase-only workflow
with VCSes that nominally didn't support rebase.  We did it because it
was easier on everyone and kept history in the upstream clean.  We
didn't do it because git has issues when combining merge and
cherry-pick.

 cherry-pick and merge working together so that the problem doesn’t
 need fixing in the first place.  And, if it doesn’t need fixing, then

If you buy into the Sun model then this is all a non-issue.  If you
don't then I think you have other problems (because I have bought into
the Sun model) :)

 the cost of the solution isn’t needed either.  The biggest problem
 with git, is that two features don’t work nicely together when they
 [...]

The Sun model has no additional cost.  It moves costs around so that the
people dealing with conflicts are the downstreams, not the upstreams,
and that's exactly as it should be.

(Keep in mind that Solaris gates tended to have large numbers of commits
on any given day, so it was quite common that one would have to rebase
multiple times before successfully pushing.  For large projects with
long test cycles the gates would close to avoid the need to rebase and
re-test.)

 I still favor fixing the underlying problem with cherry-pick and merge
 not working.  :-)  That said, I see how to work around the bug with
 rebase, if I need to.

IMO it could be done, but I can't help that.

 I wish the top google hit were your guide and I wish I never saw all
 the other pages…  I see now your position, and I see why all the

Me too!  I should blog it.

 guides are wrong, if you know just how to use rebase.  I wish the git
 documentation were improved to say as the first sentence under

The Sun model is not the only way to use git though.

 cherry-pick, this feature sucks and doesn’t really work well, it can
 cause excess merge conflicts.  rebase can be used to work around the
 bugs in cherry-pick for now.  And under rebase, instead of saying what
 it said now, that how one can can trivially and effortlessly use git,
 instead of saying, Do not rebase commits that you have pushed to a
 public repository which I now see is wrong.

I'm glad you understand the Sun model now.  You should evaluate its
applicability to your use case on its own merits.  Don't use it just to
workaround a problem in git; use it because it's good, or don't use it
because it doesn't fit your team's needs.

Nico
-- 
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Branch objects (was: Re: cherry picking and merge)

2014-08-07 Thread Nico Williams
On Thu, Aug 07, 2014 at 12:38:48PM +0100, Tony Finch wrote:
 I have been fiddling around in this area.
 
 What I want to be able to do is develop fixes for open source code
 that I run, and get those fixes upstream. This means I need a rebasing
 workflow, to keep the patches up-to-date and to deal with code review
 feedback.

Right.

 But this is inconvenient for deploying the patched version to
 production (which is the point of developing the fixes) - I want a

I'm not sure I follow this.  You deploy what you build, and you build
the HEAD of the production branch, whatever that is.  If it gets
rebased, so it it does.

 fast-forwarding branch for that. And it would be nice to be able to
 share the history of the patch series, so others can see what changed
 between revisions more easily.

But yes, it's nice to have a history of all the rebases.  For example:
so you can show the work you've done (rebasing to please an upstream is
work).

The reflog does this, of course, but you can't push it.  Of course, my
conception of branch object wouldn't push rebase history to an upstream
that doesn't want it, but you could push it to repos that do.

 So I have a small tool which maintains a publication branch which
 tracks the head of a rebasing branch. It's reasonably satisfactory so
 far...
 
 https://git.csx.cam.ac.uk/x/ucs/git/git-repub.git

Yeah, that's useful.

Nico
-- 
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Branch objects (was: Re: cherry picking and merge)

2014-08-07 Thread Nico Williams
On Thu, Aug 07, 2014 at 05:42:34PM +0100, Tony Finch wrote:
 Nico Williams n...@cryptonector.com wrote:
  On Thu, Aug 07, 2014 at 12:38:48PM +0100, Tony Finch wrote:
   But [a rebasing workflow] is inconvenient for deploying the patched
   version to production (which is the point of developing the fixes) - I
   want a fast-forwarding branch for that.
 
  I'm not sure I follow this.  You deploy what you build, and you build
  the HEAD of the production branch, whatever that is.  If it gets
  rebased, so it it does.
 
 The problem is that the production branch gets copied around: pushed to
 the repo server, pulled by other team members, etc. Forced pushes
 are accident-prone, as is resetting a rebased branch after a pull.

When I rebase and I need the old HEAD around I do something like this:

$ git checkout $branch_to_rebase
$ ver=${branch_to_rebase##*-}
$ git checkout -b ${branch_to_rebase%-${ver}}-$((ver+1))
$ git rebase ...

or like this:

$ git checkout $branch_to_rebase
$ git branch ${branch_to_rebase}-$(date +%Y-%m-%d)
$ git rebase ...

Either way I retain the old HEAD with some name.  This requires
discipline, so scripting it is useful.  But if you want discipline then
you want git to know that for this branch, don't prune/gc old HEADs
orphaned after rebases and push the rebase history for this branch.

   https://git.csx.cam.ac.uk/x/ucs/git/git-repub.git
 
  Yeah, that's useful.
 
 Glad you think so :-)

Thank you.

Nico
-- 
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Branch objects (was: Re: cherry picking and merge)

2014-08-07 Thread Nico Williams
On Thu, Aug 7, 2014 at 12:47 PM, Tony Finch d...@dotat.at wrote:
 Nico Williams n...@cryptonector.com wrote:
 Either way I retain the old HEAD with some name.

 Hmm, yes, I can see that would work. However my previous workflow was
 rather branch-heavy and I found the accumulation of names annoying. I have
 not yet had enough usage out of git-repub to see if it goes too far in the
 direction of lack-of-names. A big omission is no opportunity to edit its
 commit messages.

Oh, I just read your script more carefully and looked at your example
history again.  You're using parent metadata in the commits to keep
the history alive without the extra names, correct?  *That* is
_clever_.  Hats off.  I may have to steal this script :)

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Branch objects (was: Re: cherry picking and merge)

2014-08-07 Thread Nico Williams
On Thu, Aug 7, 2014 at 6:38 AM, Tony Finch d...@dotat.at wrote:
 So I have a small tool which maintains a publication branch which tracks
 the head of a rebasing branch. It's reasonably satisfactory so far...

 https://git.csx.cam.ac.uk/x/ucs/git/git-repub.git

 ... though the structure of the publication branch is weird and not very
 easy to navigate. You can see it in action in my git.git repo:

You know, maybe you could even use this to automatically figure out
the merge base for downstreams that follow your rebased branch:
auto-generate the git rebase --onto head head as it was prior to
all the upstream rebases.

That would be awesome, particularly if integrated into git.  It would
then be fine to rebase published branches in most cases, for example.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Rebase safely (Re: cherry picking and merge)

2014-08-06 Thread Nico Williams
On Wed, Aug 06, 2014 at 05:38:43PM -0700, Mike Stump wrote:
 Oh, wait, maybe I have misunderstood the prohibition.  I have:
 
upstream  —— fsf
|
 \
  |
  v
 Me  —   master  — coworker.

This looks a lot like what I meant about project repos.

 Me is a git clone of master, coworker is a git clone of master.
 Master is a bare repo on a shared server where we put all of our work.
 upstream is a bare git repo of the fsf git tree for gcc.  fsf is a box

Yes, exactly.  We did used exactly this at Sun, with a rebase-only
workflow.  You won't believe it till you see it [below].

 owned by other that hosts the gcc git repository.  I do a git pull fsf
 in upstream from time to time, and a corresponding git merge fsf in Me
 from time to time.  When I like my work I do a git push (to master
 exclusively).  To go to upstream, we submit patches by hand, git is
 not really involved.  I never pull into master from upstream (don’t
 even think that’s possible since they are both bare).

I see.  Hey, if that works for you...  You could, of course, merge or
cherry-pick, or rebase your team's commits onto another copy of the FSF
(upstream) master and then send those commits: sending commits is better
than sending diffs, IMO, mostly because you get to have some metadata
and integrity protection, and because git can ensure lineage and so on.

But you could live purely with diff/patch, no question, and anywhere
between that and making full use of a VCS' powers.

Here now is what we did at Sun, mapped onto git, written as something of
a hardcopy to be more exact.

Remember, this was what we did for _all_ of Solaris.  You can probably
still find docs from the OpenSolaris days describing how to do it with
Mercurial, so you can see I'm not lying.  Thousands of engineers,
working on many discrete projects, with a large OS broken up into a few
consolidations (each with its own repo).

(Here the project gate is the team repo, that I think you call
master above.)

$ # on a bi-weekly (or whatever's best) basis:
$
$ git clone $foo_project_gate foo
$ cd foo
$ git remote add upstream ...
$ git fetch upstream
$ git checkout $current_master
$ new_snapshot=master-$(date +%Y-%m-%d)
$ git checkout -b $new_snapshot
$ git rebase upstream/master
$ git push origin $new_snapshot
$
$ mutt -s PROJECT FOO: Rebase onto new master branch master-$(date +%Y-%m-%d) 
foo-engineers  /dev/null

Then the engineers on this project do this (at their leisure):

$ old_snapshot=-mm-dd from current master branch
$ new_snapshot=-mm-dd from new master branch
$ cd $my_foo_project_clone
$ git fetch origin
$ for topic_branch in ...; do
git checkout -b ${topic_branch%-${old_snapshot}}-$new_snapshot
git rebase --onto master-$new_snapshot master-$old_snapshot
  done
$
$ # Ready to pick up where I left off!
...

Eventually engineers integrate commits into the project gate:

$ # I'm ready to push to the project gate!
$
$ git checkout some_topic_branch
$
$ # Note: no -f!
$ git push origin HEAD:master-$current_snapshot
...
$ # yay

Eventually the project is ready to push its commits upstream:

$ git clone $project_gate foo
$ cd foo
$ git remote add upstream ...
$ git checkout master-$current_snapshot
$ git push upstream HEAD:master

If you're not going to be sending all local commits upstream yet then
you can do an interactive rebase, put the commits you do want to send
immediately after the upstream's HEAD commit, all the others after, and
send just those.  If you do this you should create a new snapshot and
tell your team members to git rebase --onto it.

Note that we're always rebasing _new_ branches.  Never old ones.  The
project gate does plain rebases of those new branches.  Downstreams have
to rebase --onto to recover (it works fine).

This is a very rebase-happy workflow.  It keeps as-yet-not-contributed
commits on top relative to the immediate upstream of any repo.  This
makes them easy to identify, and it keeps the author/date/subject
metadata.  Because you rebase often, you don't lag the upstream by much.
Because they are on top it's always fast-forward merge to push --
you're always merged, with some lag, yes, but merged.  And the person
doing the merging is the owner of the repo (team members, project
gatekeeper).

It's a bit more work each time you rebase than a merge-heavy workflow.
But it's also easier to contribute, and it's easier on each successive
upstream's maintainers.

(The upstream also kept snapshot branches.  Doing this has many good
side effects, not the least of which is that git prune (and gc, which I
knew about) doesn't go deleting the past of each rebase.)

  The only use-case I've seen where a rebase-based workflow doesn't work
 
 Well, and now mine, which I claim is a the canonical open source use
 [...]

Nah.  Sun managed this for decades without a hitch, and for products
much larger than GCC.  See above.

(It's true that it's difficult to sell some people on this workflow,

Re: cherry picking and merge

2014-08-06 Thread Nico Williams
On Wed, Aug 6, 2014 at 10:58 AM, Jakub Narębski jna...@gmail.com wrote:
 W dniu 2014-08-01 22:55, Nico Williams pisze:
 It would help if cherry-pick history where recorded somewhere (beyond
 the reflog)...

 Cherry-picks should record two parents, like merges.

 (Of course, it does no good to know about an unreachable parent, when
 a commit with two parents is pushed to a repo that doesn't have one of
 those parents, which can happen when topic branches aren't pushed
 upstream.)

 There was (long time ago) a long thread about idea of adding some
 kind of 'weak' references (links), 'weakparent' that can be automatically
 used by Git but do not pollute the commit message,
 and do not affect reachability calculations.  Ultimately it went
 nowhere (as you can see) - there were many problems.

My proposal was to put this sort of ancillary history info in a
branch object (and that branches should be objects).  This would
have a number of benefits, not the least of which is that at push time
you can drop such ancillary history without having to alter the
commits being pushed.

 For example: how it would work for reverts and rebases?

Reverts upstream?  The revert should record the commit hash of the
commit it reverts (but file-level reverts lose), so that this could be
noticed.

Rebases upstream?  Well, that shouldn't happen, but if it does then
you must rebase --onto and any cherry-picks of upstream rebased
commits lose their ties to those (but this can be detected).

In general recording more metadata (assuming there's not privacy
issues to consider) can't hurt.  Using it might, but having the option
to can also help.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Rebase safely (Re: cherry picking and merge)

2014-08-06 Thread Nico Williams
On Wed, Aug 06, 2014 at 12:11:16PM -0700, Mike Stump wrote:
 On Aug 1, 2014, at 4:40 PM, Nico Williams n...@cryptonector.com wrote:
  As for rebase, I still don't understand why it doesn't work for you.
 
 http://git-scm.com/docs/git-rebase says:
 
   Rebasing (or any other form of rewriting) a branch that others have
   based work on is a bad idea
 
 If you read stack-overflow, you will discover a ton of people that
 like doing this, and they get hammered because of it.  My use case
 fits exactly (as near as I can tell) the hammer case.

It's not a good idea to rebase a branch in a repo that others pull from.

There's nothing wrong with rebasing your patches on that same branch in
your clone as long as the end result is a fast forward merge when you
push.

$ git clone https:///blah
$ cd blah
$ do some work
$ git commit ...
$ git fetch origin
---$ git rebase origin/master
$ git push origin master

There's NOTHING wrong with that rebase.  It's perfectly safe because
it's only in your *private* clone.

(If later you publish that clone and expect people to track your master
branch, then rebase becomes problematic, but that's not something most
people ever do with their clones.)

The only use-case I've seen where a rebase-based workflow doesn't work
is where you have multiple upstreams that you're following.  I.e., the
upstream forked and you want to take some commits from one, some from
the other, or otherwise keep a merge of both.

(Also, if an upstream is ever rebased you can usually recover on the
downstream side by rebasing with the --onto option, so it's not the end
of the world.)

 Now, I found the stack-overflow commentary first, and all the horrors
 of it, and all the nuances.  I carefully read what people were doing,
 how what I wanted to related to what they were doing, and it sure felt
 like I was in the, don’t go there camp.

A lot of people rant about rebase.  They're wrong.  They've distorted
your perspective.

 So, I like to know if I’m driving off a cliff, before I do.  I’m the
 [...]

There's just two simple rules to follow and you'll be safe:

1) NEVER git push -f (--force) to a published repo/branch.

   The upstream should enforce this with a receive hook.

2) NEVER work directly in a published repo.  Instead work in a private
   clone.  To help make sure of this, never publish a non-bare repo
   (bare == has no workspace; non-bare == has a workspace).

If you ever do a rebase that produces results you're unhappy with you
can undo that rebase like so:

 - use git reflog to find the branch's previous HEAD commit
 - reset the branch to point to that commit

It really helps to think of git as a pile of commits arranged in a
Merkle has tree.  Branches and tags are just symbolic names for specific
commits.  Rebase builds a new line of commits in the tree then it
changes the symbolic branch name's HEAD to point to the head of that new
line of commits, BUT NOTHING IS LOST in the pile of commits that is the
repo, not until you git-prune(1) to remove commits not reachable from
symbolic names (branches and tags).

  The only case where I can imagine not using a
  rebase-heavy workflow is where I have to track multiple forked
  upstreams and so I want to merge each into my branch.
 
 So, sounds like I fit that use case and rebase could be my friend.

Excellent.

 How do I square what you said and:
 
   Rebasing (or any other form of rewriting) a branch that others have
   based work on is a bad idea
 
 ?

See above.

 I want all old refs in old emails to work.  I want all refs in

They will if you stick to the two rules I mention above.

 bugzilla to work.  I want to see the original dates of all the work.

Ditto.

 I want git blame to report those artifacts in email and bugzilla.  I
 have coworkers that I push to, pull from (through a single sharing
 point, we call the master tree).  We work on gcc, we pull git gcc down
 to a local copy, then merge it into our tree.  I want to cherry pick
 changes from upstream.  I do work and push to our master, I pull work
 of coworkers from the master, my coworkers do the same.  Isn’t this
 the canonical open source use case?

That means that you have/maintain an intermediate upstream, yes?

This is a bit trickier since once in a while that intermediate upstream
and everyone downstream of it has to catch up with the real upstream.

Here you have two options:

 - the intermediate diverges from the real upstream, and then you
   merge/cherry-pick from the upstream as needed
   
   The intermediate's maintainer must still merge/rebase/cherry-pick
   from the intermediate branch and onto a branch of the upstream in
   order to push to the upstream.

or

 - the intermediate occasionally rebases onto the upstream, and then the
   repos downstream of the intermediate must also rebase with --onto.

   In this case the intermediate's maintainer must tell the downstreams
   what rebase command to execute.

   This makes it easier to push from

Re: Branch objects (was: Re: cherry picking and merge)

2014-08-06 Thread Nico Williams
On Wed, Aug 06, 2014 at 08:31:18PM +0200, Jakub Narębski wrote:
 On Wed, Aug 6, 2014 at 6:26 PM, Nico Williams n...@cryptonector.com wrote:
  My proposal was to put this sort of ancillary history info in a
  branch object (and that branches should be objects).  This would
  have a number of benefits, not the least of which is that at push time
  you can drop such ancillary history without having to alter the
  commits being pushed.
 
 Is it something like object-ified reflog, similar to how replacement
 objects (git-replace) can be thought to be object-ified grafts (I know
 they are more)? Do I understand it correctly?

Yes, per-branch.  At push time a commit would be pushed to the upstream
branch listing the commits pushed now (and who by).  Locally every
rebase/cherry-pick/merge/commit onto the branch would appear in the
branch object's history, kinda just like the reflog.  The main
difference is that the upstream branch's history could be viewed.

 Did you plan to (ab)use known object types: tree and commit (a bit
 similar to git-replace and git-note object, though there is no need for
 fanout trees - the top level tree can reproduce refs hierarchy)? I see
 that you planned to (ab)use existing transfer mechanism of refs and
 objects...

Just like signed tags, basically.

  Reverts upstream?  The revert should record the commit hash of the
  commit it reverts (but file-level reverts lose), so that this could be
  noticed.
 
 If it is object-ified reflog then reverts are not a problem...

Right.

  Rebases upstream?  Well, that shouldn't happen, but if it does then
  you must rebase --onto and any cherry-picks of upstream rebased
  commits lose their ties to those (but this can be detected).
 
 With rebases the problem is that it would be nice to have (at least
 for a short time) the history of series of patches (the metahistory,
 or history of a branch), but usually one doesn't need old pre-rebase
 version after cleaning up the history for publishing.

Right.

  In general recording more metadata (assuming there's not privacy
  issues to consider) can't hurt.  Using it might, but having the option
  to can also help.
 
 True...

The principle should be to record as much metadata as possible, pruning
ancillary metadata (reflog-like metadata that isn't on the commits) only
at push time.  

Nico
-- 
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Rebase safely (Re: cherry picking and merge)

2014-08-06 Thread Nico Williams
On Wed, Aug 06, 2014 at 02:44:59PM -0500, Nico Williams wrote:
 That means that you have/maintain an intermediate upstream, yes?
 
 This is a bit trickier since once in a while that intermediate upstream
 and everyone downstream of it has to catch up with the real upstream.
 
 Here you have two options:
 
  - the intermediate diverges from the real upstream, and then you
merge/cherry-pick from the upstream as needed

The intermediate's maintainer must still merge/rebase/cherry-pick
from the intermediate branch and onto a branch of the upstream in
order to push to the upstream.

I should add something important here.

Rebasing makes life easier for the intermediate maintainer, and for any
upstream maintainer who has to merge pull requests or patches sent in
email.  Rebasing puts the onus for merging on the contributor, exactly
where it belongs!

(Granted, for an e-mail based workflow one's patches might have made for
a fast-forward merge when sent but not when the upstream gets to them.
With long enough latency this gets painful.  Which is why I don't
recommend an e-mail based commit integration workflow.)

Nico
-- 
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Pluggable backends for refs,wip

2014-08-05 Thread Nico Williams
Personally (a user of, not a maintainer of, git) I really want some
alternative backends.  In particular I'm after something like Fossil's
use of SQLite3; I want a SQLite3 backend for several reasons, not the
least of which is the power of SQL for looking at history.

I'm not sure that I necessarily want a daemon/background process.  I
get the appeal (add inotify and bingo, very fast git status, always),
but it seems likely to add obnoxious failure modes.

As to a SQLite3-type backend, I am of two minds: either add it as a
bolt-on to the builtin backend, or add it as a first-class backend
that replaces the builtin one.  The former is nice because the SQLite3
DB becomes more of a cache/index and query engine than a store, and
can be used without migrating any repos, but the latter is also nice
because SQLite3 provides strong ACID transactional semantics on local
filesystems.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Pluggable backends for refs,wip

2014-08-05 Thread Nico Williams
Excellent.  Thanks!
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Is there a way to mark a branch as published?

2014-08-01 Thread Nico Williams
On Friday, August 1, 2014, Dennis Kaarsemaker den...@kaarsemaker.net wrote:

 On di, 2014-07-29 at 17:40 -0500, Nico Williams wrote:
  (or all of a repo's branches)
 
  Teamware recorded whether it had any children and warned about
  rebasing published contents.  Perhaps git could do the same.

 Git doesn't record this directly, but you can see which known remote
 branches contain the tip of a branch:

Yes, but that says nothing about whether they accept non-ff merge
pushes, or should reject them, and it says nothing about whether they
are intended to be seen as published in the sense of having stable
history.  (Teamware would mark repos as published when cloned, but
that often meant needing to reset that flag [it wasn't a flag, but i'm
eliding irrelevant details].)

Yes, non-fast-forward pushes can be rejected with a hook, but a flag
that could be set with a git command would be more convenient.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


cherry picking and merge

2014-08-01 Thread Nico Williams
On Thursday, July 31, 2014, Mike Stump mikest...@comcast.net wrote:

 Cherry picking doesn’t work as well as it should.  I was testing on git 
 version 1.7.9.5.

 Put in a line in a file, call it:

 first version

 then cherry pick this into your branch. Then update on master and transform 
 that into:

 second version

 then, merge that branch back to master.  Death in the form of conflicts.

The problem is that cherry-picked commits lack the metadata that git
merge could use to avoid this spurious conflict report.  The reflog
has the metadata in question, but there's no guarantee that that
reflog will be available where you do the merge.  (IMO this is another
reason to want branches as objects, so such ancillary information can
be recorded somewhere, but in a way that can get dropped if desired
and without changing commit hashes, but I digress.)

If you always rebase your commits on top of the upstream, then this
problem goes away.  You can't always rebase your commits on top of the
upstream though, but wherever possible it's the best course of action
for this and other reasons.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: cherry picking and merge

2014-08-01 Thread Nico Williams
On Fri, Aug 1, 2014 at 3:50 PM, Jonathan Nieder jrnie...@gmail.com wrote:
 Jonathan Nieder wrote:

 Do you mean that git merge should be aware of what changes you have
 already cherry-picked?

 It isn't, and that's deliberate

 That said, when today's git merge fails to resolve conflicts, it's
 easily possible that we could do better at resolving the merge by
 walking through both sides and understanding what happened.

It would help if cherry-pick history where recorded somewhere (beyond
the reflog)...

Cherry-picks should record two parents, like merges.

(Of course, it does no good to know about an unreachable parent, when
a commit with two parents is pushed to a repo that doesn't have one of
those parents, which can happen when topic branches aren't pushed
upstream.)

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: cherry picking and merge

2014-08-01 Thread Nico Williams
On Fri, Aug 1, 2014 at 4:44 PM, Junio C Hamano gits...@pobox.com wrote:
 Nico Williams n...@cryptonector.com writes:

 Cherry-picks should record two parents, like merges.

 No.

 It is OK to record where it came from, and we let you do so with the
 -x option.

 But the where it came from commit is very different from being
 parent, which implies all the history behind it.  The whole point
 of a cherry-pick is that you do not want to grab the changes behind
 the commit you are cherry-picking and you want the _change_ the
 cherry-picked commit (and that commit alone) brings in.  It should
 never record two parents, like merges.

I didn't mean to imply all that.  s/parent/where it came from/, but -x
edits the commit message, not the metadata...

The point remains: to do what the OP wants git merge would have to be
able to notice that a given commit was cherry-picked from the other
branch, and what commit it was on that other branch, and right now the
only place where that information is available is in the reflog.
Recording that metadata somewhere in the commit resulting from the
cherry-pick would be better.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: cherry picking and merge

2014-08-01 Thread Nico Williams
On Fri, Aug 1, 2014 at 5:13 PM, Mike Stump mikest...@comcast.net wrote:
 On Aug 1, 2014, at 12:22 PM, Nico Williams n...@cryptonector.com wrote:
 If you always rebase

 I can’t use rebase unless you make rebase work with multiple users and 
 pushing pulling.

That works now, and I do it all the time.  Have a single repo (the
truth), always rebase local commits on top of the latest upstream,
always do fast-forward pushes.  Done.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: cherry picking and merge

2014-08-01 Thread Nico Williams
On Fri, Aug 1, 2014 at 6:06 PM, Mike Stump mikest...@comcast.net wrote:
 On Aug 1, 2014, at 1:12 PM, Sam Vilain s...@vilain.net wrote:
 Git merge has a notion of discrete merge strategies”.

 There's no particular reason that you couldn't implement a merge
 strategy which works more like SVN's approach, which essentially does an
 internal rebase and then commits the result.

 Well, being a simple user, wanting to do a simple thing, I want the default 
 strategy to just work.  [...]

Different users want different defaults.  You can't always have the
one you want.

As for rebase, I still don't understand why it doesn't work for you.
You didn't really explain.  Suppose we're right and it's the right
solution for you, then you might be ecstatic, but you gotta try it
first.

My workflow is rebase-heavy.  It's long been so, and it was so before
git happened.  The only case where I can imagine not using a
rebase-heavy workflow is where I have to track multiple forked
upstreams and so I want to merge each into my branch.

If tracking multiple forked upstreams is not your case and yet rebase
can't work for you then I'd like to understand why.  Please help me
understand your use case.  OTOH, if your use case is amenable to
rebase, then I highly recommend that you try it.

(I find that many users are allergic to rebasing.  Many people have
told me that rebase is lying, that history must be immutable, and so
on, all ignoring that: git users don't rebase published branches, and
that other VCSes tend to squash (therefore lose) history anyways when
pushing merges upstream.  But this all seems theological rather than
rational.  It's true that I dislike merge commits, but that's a
different story; I'm not allergic to merging after all.)

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Amending merge commits?

2014-07-30 Thread Nico Williams
On Wed, Jul 30, 2014 at 3:42 AM, Sergei Organov o...@javad.com wrote:
 Nico Williams n...@cryptonector.com writes:
 Local merge commits mean that you either didn't rebase to keep all
 your local commits on top of the upstream, or that you have multiple
 upstreams (the example exception I gave).

 I rather have multiple (release) branches on single upstream, say, v2.3
 and v2.4. When something needs to be fixed in 2.3, it's fixed there and
 pushed upstream, then, on 2.4, the 2.3 is merged to it, and result is
 pushed upstream. When I do this merge, I need to push the merge

Hmm, why not cherry-pick the fix?  That's how every project I know
that ports fixes across release branches does it.

 upstream, and this won't work reliably when --rebase=true is acitve
 (through pull.merge=rebase). If nothing changes upstream, I can simply
 push this, and the merge is correctly preserved. However, if somebody
 makes any changes upstream while I perform the merge, I'll need to pull
 before pushing, and this immediately flattens-out my merge, that is
 absolutely not what is needed here. Or I can simply pull before push,
 just in case, and this flattens history even when there are no any
 changes upstream!

Does this change if you give your merge commits an different commit message?

 Conversely, if you always rebase your local commits on top of the
 upstream then you won't have merge commits to worry about.

 Wrong. I do alwys rebase my local commits on top of upstream, but I
 still do have my own merge commits to worry about, as explained above.

If you cherry-pick the cross-release-branch commits you'll not have a
merge commit to worry about.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Amending merge commits?

2014-07-29 Thread Nico Williams
On Tue, Jul 29, 2014 at 4:58 AM, Sergei Organov o...@javad.com wrote:
 Nico Williams n...@cryptonector.com writes:
 That exception aside, keeping all local commits on top by always
 rebasing them onto the upstream is extremely useful: a) in simplifying
 conflict resolution, b) making it easy to identify as-yet-unintegrated
 local commits, c) making it easy to contribute local commits.

 But 'pull --rebase=preserve' does rebase local commits onto the
 upstream, and result is exactly the same as 'pull --rebase=true', unless
 you have some of your own merges to be rebased. That's where the
 difference between these two options appears. It's --rebase=false that
 performs merges rather than rebase.

Local merge commits mean that you either didn't rebase to keep all
your local commits on top of the upstream, or that you have multiple
upstreams (the example exception I gave).

Conversely, if you always rebase your local commits on top of the
upstream then you won't have merge commits to worry about.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Amending merge commits?

2014-07-29 Thread Nico Williams
On Tue, Jul 29, 2014 at 2:29 PM, Philip Oakley philipoak...@iee.org wrote:
 From: Nico Williams n...@cryptonector.com
 Local merge commits mean that you either didn't rebase to keep all
 your local commits on top of the upstream, or that you have multiple
 upstreams (the example exception I gave).

 Conversely, if you always rebase your local commits on top of the
 upstream then you won't have merge commits to worry about.

 Whilst it may not be the Git Way, I'd expect that in many less well
 informed companies, the need to keep merge commits fom other lines of
 development would be quite a common (political ) technique where some
 preparatory branch needs to be merged in before one's feature can be
 completed (similar to all those cases on the list when folk say 'builds on
 top of xy's commit deadbeaf)

The way we did this at Sun, first with Teamware, then later with
Mercurial, was as follows:

 - projects kept their own clone repos of the upstream
 - engineers working on a project cloned the project repo (project gate)
 - engineers pushed/pulled to/from the project gate
 - each project gate had a gatekeeper whose job it was to periodically
rebase onto the latest upstream
 - then engineers would rebase onto the new project gate

No merge turds (Sun speak) were ever allowed in any upstream,
whether a project gate or the ultimate upstream.  All commits had to
be organized according to specific rules, and squashed.  These rules
applied at the project gate and in the upstream.

 - when the project was ready for integration the gatekeeper would
rebase and squash as necessary, then push to the upstream

(I'm eliding some details.  In particular when an intermediate
upstream rebased the previous head was left available as a snapshot
to make the equivalent of git rebase --onto possible.)

The upshot was: all local commits were always on top of whatever the
next upstream in the chain was.  Always.  No merge commits ever.

That workflow works just fine with git.  It worked really well at Sun
(with thousands of engineers working on Solaris alone).  And it should
work well for anyone who doesn't have two or more forked upstreams to
follow.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Amending merge commits?

2014-07-29 Thread Nico Williams
On Tue, Jul 29, 2014 at 4:38 PM, Philip Oakley philipoak...@iee.org wrote:
 From: Nico Williams n...@cryptonector.com
 That workflow works just fine with git.

 I'm not saying that it isn't a good technique and can work well. Rather I'm
 saying we should be tolerant of the rules and techniques of others who do
 [...]

Sure.  I was just giving advice as to how to avoid any problems at
pull time w.r.t. local merge commits.

Better merge commit handling at pull time might be great (I'd not
know; I avoid local merge commits!), but I would still strongly
recommend keeping all local commits on top because otherwise you lose
local history.  Even if you use -m to set a better commit message, you
might prefer to have kept the original N local -now rebased- commits
around so you can tell what each discrete change was, even if you'll
eventually squash them (you might not squash them all into one).

  It worked really well at Sun
 (with thousands of engineers working on Solaris alone).  And it should
 work well for anyone who doesn't have two or more forked upstreams to
 follow.

 I'm just cautious of an accidental one size fits all approach, so the
 ability to rebase lines of development which contain merge commits should be
 possible (with an appropriate and documented option) without hidden traps.

Thus far the only case I've seen where this approach doesn't work _at
all_ is the case where you have multiple forked upstreams.  The only
other case where it doesn't work is a social problem: rebase allergy.
For the repos I maintain I insist on contributed commits applying as
fast forward merges, or I'll rebase them myself if need be.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Is there a way to mark a branch as published?

2014-07-29 Thread Nico Williams
(or all of a repo's branches)

Teamware recorded whether it had any children and warned about
rebasing published contents.  Perhaps git could do the same.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Amending merge commits?

2014-07-28 Thread Nico Williams
On Mon, Jul 28, 2014 at 3:00 PM, Jonathan Nieder jrnie...@gmail.com wrote:
 Sergei Organov wrote:

 Is there any scenario at all where pull --rebase=true wins over
 preserve?

 Basically always in my book. ;-)

 When people turn on 'pull --rebase', they are asking for a clean,
 simplified history where their changes are small discrete patches in a
 clump on top of upstream.

+1.  Words to develop by.

There are exceptions.  E.g., when you pull commits from multiple
[forked] upstreams, then you can't keep your local commits on top.

That exception aside, keeping all local commits on top by always
rebasing them onto the upstream is extremely useful: a) in simplifying
conflict resolution, b) making it easy to identify as-yet-unintegrated
local commits, c) making it easy to contribute local commits.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: No fchmod() under msygit - Was: Re: [PATCH 00/14] Add submodule test harness

2014-07-14 Thread Nico Williams
On Mon, Jul 14, 2014 at 6:31 AM, Erik Faye-Lund kusmab...@gmail.com wrote:
 On Wed, Jul 9, 2014 at 10:00 PM, Eric Wong normalper...@yhbt.net wrote:
 Torsten Bögershausen tbo...@web.de wrote:


 You're saying this as if Windows is a single-user system. It's not,
 but it uses ACLs rather than POSIX permissions to manage file-system
 permissions. So far we've opted to ignore ACLs in Git for Windows,
 though.

The clever thing to do (that some versions of ZFS do nowadays) is to
use the new mask to edit the ACL as follows:

 - leave any DENY ACEs as-is; for all others continue

 - remove (reset) from any ACEs for Everyone (and/or Authenticated
Users) any bits corresponding to zero'ed bits in the new mask's other
bits

 - remove from any other ACEs that are not for the owner (@OWNER in
NFSv4 speak) any bits corresponding to zero'ed bits in the new mask's
group bits

 - remove from the owner's ACEs any bits corresponding to zero'ed bits
in the new mask's owner bits, but with some exceptions, in particular
the owner must retain the right to edit the ACL

 - add (set) to the Everyone (and/or Authenticated Users) ACEs a set
of bits corresponding to the set bits in the new mask's other bits

 - add (set) either only to the ACE for the file's group
(alternatively, to all non-owner, non-Everyone/Authenticated Users
ACEs) a set of bits corresponding to the set bits in the new mask's
group bits

...

I.e., use the chmod mask to decrease/increase access without changing
the shape of the ACL.

Determining a file's mode_t from an ACL is similar, though it must
take DENY entries into account: make a set of users/groups referred to
by any ACEs in the ACL, divide them into owner, other (Everyone and/or
Authenticated Users), and group (all others), find the maximal access
granted.

Still, git might like to know what ACLs to apply to files at checkout
time.  That would be a vast new feature, I think, and probably not
worth it, particularly since that would require dealing with the
different types of ACLs: NTFS/NFSv4/ZFS on the one hand, POSIX Draft
on the other, plus AFS and who knows what else -- ETOOMUCH IMO.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: No fchmod() under msygit - Was: Re: [PATCH 00/14] Add submodule test harness

2014-07-14 Thread Nico Williams
 Still, git might like to know what ACLs to apply to files at checkout
 time.  That would be a vast new feature, I think, and probably not
 worth it, particularly since that would require dealing with the
 different types of ACLs: NTFS/NFSv4/ZFS on the one hand, POSIX Draft
 on the other, plus AFS and who knows what else -- ETOOMUCH IMO.

To complete that thought...

Inheritable ACLs + chmod should suffice.  The user should setup
inheritable ACLs for the directory where a repo is to be cloned, then
on checkout git should just apply mode changes in such a way as to
leave ACL shapes unchanged, only adding or removing bits from
non-DENY ACEs according to the saved file modes.

Inheritable ACEs should be left as-is, since git doesn't track
directory permissions (right?), or if it did, then those should be
edited just like normal ACEs.

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Use case (was Re: Should branches be objects?)

2014-06-24 Thread Nico Williams
On Tue, Jun 24, 2014 at 6:09 AM, Theodore Ts'o ty...@mit.edu wrote:

 On Mon, Jun 23, 2014 at 10:20:14PM -0500, Nico Williams wrote:
 
  Now, suppose that branches were objects.  Then at push time one might
  push with a message about the set of commits being pushed, and this
  message (and time of push, and pusher ID) would get recorded in the
  branch object.  At fetch time the branch objects's histories would be
  pulled (but usually never pushed), and would be available for browsing
  with git log at remotes/remote/branch.  Each commit of the branch
  object (as it were) would record each logical set of commits.

 This seems pretty close to what we have with signed tags.  When I send
 a pull request to Linus, I create a signed tag which createscontains a
 message about a set of commits, and this message is automatically
 included in the pull request message generated with git
 request-pull, and when Linus merges my pull request, the
 cryptographically signed tag, along with the message, date of the
 signature, etc., is preserved for all posterity.

Thanks for pointing this out.  Signed tags are objects -- that's a
clear and strong precedent..  That's another thing that branches as
objects could have: signatures of pushed commits (separately from the
commits themselves).

 It seems the major difference is that it's a pull model, where some
 projects seem much happier with a push model.  But that sounds like
 what is needed is that someone replaces Linus Torvalds with a shell
 script --- namely, an e-mail bot that receives pull requests, checks
 the signed tag against an access control list, and if it is an
 authorized committer, accepts the pull request automatically (or
 rejects it if there are merge conflicts).

Shell script, protocol..  The git push protocol is convenient.  The
fact that git supports a patches-via-email, push, and pull models,
that's a great aspect of git.  Why disadvantage the push case, when
it's so popular (e.g., via github and such)?

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Use case (was Re: Should branches be objects?)

2014-06-23 Thread Nico Williams
(thinking more about this, digesting Jonathan's response...)

The Illumos repo, like OpenSolaris before it, and Solaris itself at
Sun (and now at Oracle) requires that fixes be broken down into small
commits, with related fixes, tests, and docs changes all typically in
separate commits, but all pushed together, so that a single push of N
commits is a logical set of changes (e.g., to be backed out together
if, say, any one of them breaks a build).  With git the only way to
record this grouping at push time is with a post-receive hook that
does the recording (which is what the Illumos repo does, sending email
to a list about all the commits pushed in one go).

Now, suppose that branches were objects.  Then at push time one might
push with a message about the set of commits being pushed, and this
message (and time of push, and pusher ID) would get recorded in the
branch object.  At fetch time the branch objects's histories would be
pulled (but usually never pushed), and would be available for browsing
with git log at remotes/remote/branch.  Each commit of the branch
object (as it were) would record each logical set of commits.

Side effects besides addressing the contiguous and related commit grouping need:

 - no more need to sign-off on cherry-picks: the branch will record
the ousher's ID, which can then be taken as the person signing off;

 - branch objects substantially replace/augment reflogs;

 - no need to ammend commits: just push an empty set of commits just
to update the branch object with a note!

The UI would mostly consist of an option to git push to include a push
message, and a way to review branch history (much like git log -g, but
with access to the push-time metadata).  Also along for the ride: a
way to get the new metadata in post-receive hooks.

Problem: if pushing via an intermediary the push metadat would get
lost.  This would argue for either a stronger still notion of related
commits, or none stronger than what exists now (because ETOOMUCH).
But this branch object concept could also be just right: if pushing
through a an intermediary (what at Sun was called a project gate) then
it becomes that intermedirary's (gatekeeper's) job to squash, rebase,
regroup, edit, drop, reword, ... commits.

Just a thought,

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Our merge bases sometimes suck

2014-06-20 Thread Nico Williams
On Fri, Jun 20, 2014 at 1:53 AM, Junio C Hamano gits...@pobox.com wrote:
 Michael Haggerty mhag...@alum.mit.edu writes:
 [...]

 Hmph, but that obviously will become very expensive to compute as
 project grows.

That's the main reason to like Fossil's approach (namely, the use of
SQL, specifically SQLite3): you can write declarative queries and let
the query planner optimize.

(That's also about the only thing I like about Fossil: the use of
SQL/SQLite3, and all that that implies.  Fossil is otherwise an
anti-git, and that makes it useless for me.)

Pardon the intrusion,

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Should branches be objects?

2014-06-19 Thread Nico Williams
[I'm a list newbie here, but a git power user.]

If branches were objects...

 - one could see the history of branches, including

 - how commits were grouped when pushed/pulled (push 5 commits, and
the branch object will record that its head moved by those five
commits at once)

 - rebase history (git log branch-object - better than git reflog!)

 - object transactional APIs would be used to update branches

Branch objects might be purely local, recording what was done in a
local repo to a branch, but they might be pullable, to make branch
history viewable in clones.

Just a thought,

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Should branches be objects?

2014-06-19 Thread Nico Williams
On Thu, Jun 19, 2014 at 6:46 PM, Jonathan Nieder jrnie...@gmail.com wrote:
 Nico Williams wrote:

  - one could see the history of branches, including

 Interesting.  'git log -g' is good for getting that information
 locally, but the protocol doesn't have a way to get it from a remote
 server so you have to ssh in.  Ronnie (cc-ed) and I were talking
 recently about whether it would make sense to update git protocol to
 have a way to get at the remote reflogs more easily --- would that be
 useful to you?

Yes and no.  I've thought about that some concept, but:

a) reflogs include information about what's done to the workspace
(checkout...) that's not relevant to any branch,

b) reflogs aren't objects, which ISTM has caused transactional issued
(even if they are fixed or soon to be),

c) the fewer kinds of things, the more elegant the design, so maybe
reflogs ought to be objects themselves, which is one thought that led
me to branches should be objects.

Another thought that led me there is that I often do:

$ git checkout -b ${branch}-rebase1
$ git rebase -i master
...
$ git checkout -b ${branch}-rebase2
$ git rebase -i master
...

I iterate through this until a set of commits is the way the upstream wants it.

No one really needs that history, except me: possibly to show my
boss/customer, possibly to put together a list of changes I've done to
show the upstream maintainer, ...   Yes, this is in the reflog, but...
it's mixed up with unrelated stuff.

Also, I'd like to be able to git diff
branch-version..same-branch-diff-branch-version.  Again, for my
own purposes in collating changes I've done to previously submitted
PRs.

Now, I can do that as I always have, but it litters my branch namespace.

Lastly, there are people who just don't get rebasing.  They think it's
horrible because it changes the truth.  You've met them, I'm certain.
Branches as objects might help mollify them.

  - how commits were grouped when pushed/pulled (push 5 commits, and
 the branch object will record that its head moved by those five
 commits at once)

 The reflog on the server (if enabled) records this.

Yeah, though as you point out I can't see it.

  - rebase history (git log branch-object - better than git reflog!)

 The local reflog ('git log -g branch') records this.

See above.

  - object transactional APIs would be used to update branches

 Ronnie's recent ref-transaction code does this.

Speaking of which: are there any power failure corruption cases left
in git?  How is this tested?

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Should branches be objects?

2014-06-19 Thread Nico Williams
Another thing is that branches as objects could store a lot more
information, like:

 - the merge-base and HEAD for a rebase (and the --onto)

 - the interactive rebase plan!  (and diffs to what would have been
the non-interactive plan)

 - the would-be no-op non-interactive rebase plan post rebase (again,
so elucidate what commit splitting and such things occurred during a
rebase)

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html