Re: [sqlite] Please help me fix the SQLite Git mirror

2019-04-23 Thread Jens Alfke 


> On Apr 23, 2019, at 9:17 AM, Wout Mertens  wrote:
> 
> Note that many git clients (https://git-scm.com/downloads/guis/ 
> ) do allow
> you to see the entire DAG, just not GitHub. If you use a Mac, a
> particularly great one is https://gitup.co 

My preference is Fork > (Mac/Win, 
free), which has a slightly more traditional DAG view, i.e. a table view with 
the graph displayed in the left column. (GitUp has some great features, but 
most of the time I want to look at more than one commit message at a time.)

—Jens
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Please help me fix the SQLite Git mirror

2019-04-23 Thread Wout Mertens 
On Tue, Apr 23, 2019 at 1:22 PM Richard Hipp  wrote:

> The inability to see the entire DAG on a single screen in GitHub is a
> persistent source of annoyance
> to users like me who are accustomed to Fossil.
>

Note that many git clients (https://git-scm.com/downloads/guis/) do allow
you to see the entire DAG, just not GitHub. If you use a Mac, a
particularly great one is https://gitup.co

Wout.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Please help me fix the SQLite Git mirror

2019-04-23 Thread Richard Hipp 
On 4/23/19, Rowan Worth  wrote:
>
> How can you justify the claim that history was unchanged on trunk between
> time (1) and time (2)?

Short answer:  I look at the entire DAG, not individual branches.

Long answer:

From prior discussion and feedback on this thread, I've come to
realize that there is a philosophical difference between Fossil and
Git.  Both systems use a Directed Acyclic Graph of check-ins (the DAG)
as their underlying data structure.  But in Git, the fundamental unit
of management is a branch, which Git defines as a linear sequence of
check-ins from a leaf back to the root, whereas in Fossil, the
fundamental unit of management is the entire DAG.

Git-ers think in terms of branches.  And so if you move a node from
one branch to another, that changes history.  But Fossil-ers think in
terms of the entire DAG.  Relabeling a node to be on a different
branch is just a notational convenience and does not change the DAG in
any way, and is hence not a history change.

Additional observations that support of this dichotomy:

(1) The default synchronization unit in Git is a single branch.  You
can sync the entire DAG using the --mirror option, but that is rarely
done in practice it seems.  In Fossil, the default and only
synchronization option is the entire DAG.

(2) The primary history display in Git/GitHub shows only a single
branch (ex: https://github.com/sqlite/sqlite/commits/master) whereas
the primary history display in Fossil shows the entire DAG (ex:
https://www.sqlite.org/src/timeline). The inability to see the entire
DAG on a single screen in GitHub is a persistent source of annoyance
to users like me who are accustomed to Fossil.

(3) Git puts special emphasis on rebase, which is used to transfer or
replicate a span of check-ins from one branch into another.  Having
all the important check-ins on a single branch is important if your
focus is on that one branch.  Fossil, in contrast, keeps the entire
DAG in view all at once, and so is no issue with keeping span of
check-ins remain on a separate arm of the DAG and merely merge in the
changes.

The Git approach makes sense in a highly distributed project like the
Linux Kernel, where there are many thousands of developers each
working on their own branches and where it is desirable to prune the
DAG down to a more manageable size by omitting extraneous arms.  The
Fossil approach works better for cathedral-style development where
there is a small team of contributors, all of which know each other
and work together daily, and where developers want to keep track of
everything that is going on, for improved situational awareness and
project coherence.

SQLite Git Mirror Update:

I made enhancements last night so that any future branch relabelings
that occur in SQLite should automatically be mirrored into the Git
repository on GitHub.  (Aside: See the interesting 5-way join used to
determine which "git update-ref" commands are needed on each
incremental export here:
https://www.fossil-scm.org/fossil/artifact/b0ace47d4c6a?ln=1492-1505)
There are definite management advantages to relabeling branches, and
so I will not preclude such actions in SQLite moving forward, though I
will try to keep relabelings to a minimum. Nevertheless, users of the
Git mirror should keep in mind that I think in terms of the entire
DAG, not individual branches, and so if you are tracking SQLite
development in a Git clone, you should to take steps to ensure that
you do not find yourself stalled on a side-tracked branch.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Please help me fix the SQLite Git mirror

2019-04-22 Thread Rowan Worth 
Richard Hipp wrote (quoting from several emails):

> The problem is that Git now thinks that 9b888fcc is the HEAD of master
> and that the true continuation of master (check-in 4f35b3b7 and
> beyond) are disconnected check-ins
>

Because from the git perspective it _is_ still the HEAD -- there's been no
further changes made on top of that commit. The "true" changes are in a
separate branch hanging off some historic fork point.

I don't understand this part.  From the Fossil perspective, moving a
> check-in from one branch to another is just adding a new tag to that
> check-in.  No history is changed.  The DAG of check-ins (the block-chain)
> is unmodified.


Hm. Initially, the commits on the primary branch looked like this:

1. HISTORY - FORK - MISTAKE

Then you changed it to this:

2. HISTORY - FORK - FIXED - BEYOND

How can you justify the claim that history was unchanged on trunk between
time (1) and time (2)? You haven't just added a new check-in to the branch
in this situation (which git is more than happy to do via cherry-pick),
you've also erased the MISTAKE check-in.

What happens to fossil users who updated trunk while MISTAKE was the head?
Does the next update somehow pathfind to the new BEYOND head, backtracking
via FORK?

-Rowan
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Please help me fix the SQLite Git mirror

2019-04-22 Thread Carl Edquist 


I would like to better understand how rewiring the refs this way 
constitutes "changing history".  The refs/heads entries are all 
ephemeral - they are constantly changing on their own, and no historical 
record of their past values is retained.


The key bit here is that in git, every commit references its parent commit 
hash, and thus implies the entire past history for that commit.  And local 
git repos contain that entire history.  So when a branch head is updated 
upstream, git can tell if history has moved forward (ie, the new head 
commit contains the previous head commit in its history), and if not, it's 
considered a "forced update", which is to say, history was re-written -- 
since the last previously know state of history is nolonger part of 
history.


Carl

On Mon, 22 Apr 2019, Carl Edquist wrote:


Hi Richard,

As Jonathan mentioned, in git land, if you have already published a 
"mistake" commit publicly, the proper way to revert it is to make another 
commit to reverse/undo the change.


By removing a commit from the public history of the published 'master' 
branch, it forces everyone downstream to manually fix their history.


If they do a normal "git pull", git will attempt to merge their master 
(the mistake commit) with the latest upstream master, which is not 
actually your intention.


But if you make an "revert" commit to undo the change, history will 
continue forward for the master branch from the downstream perspective.




I fixed the recent breakage of the SQLite Git mirror as follows:

(1) cd into the refs/heads directory
(2) run "cat master >mistake"
(3) run "echo a9a5465eb44d0d8f1c3c9d288b7f23f628ddb50b >master"
(4) run "git push --mirror https://github.com/sqlite/sqlite.git";


Not that you want to do it this way again if you can avoid it, but the 
safe git way to do (2),(3) is:


2) git update-ref refs/heads/mistake refs/heads/master
3) git update-ref refs/heads/master a9a5465eb44d0d8f1c3c9d288b7f23f628ddb50b


Carl

On Mon, 22 Apr 2019, Richard Hipp wrote:


Thanks for the help.  See additional questions and remarks below

On 4/22/19, Jonathan Brandmeyer  wrote:

```
# Construct the matching branch name
git branch mistake 9b888fc
# Push the name alone to the remote
git push -u origin mistake
# Move the name of master
git checkout master && git reset --hard 
# Push the new name of master
git push --force
```

Git reset --hard will move the name of the current working branch to
another branch SHA, which is why you need to first check out the
branch being moved: Its context sensitive.  You are re-writing
history, though.


I don't understand this part.  From the Fossil perspective, moving a
check-in from one branch to another is just adding a new tag to that
check-in.  No history is changed.  The DAG of check-ins (the
block-chain) is unmodified.

Subsequent to your message, I fixed the recent breakage of the SQLite
Git mirror as follows:

(1) cd into the refs/heads directory
(2) run "cat master >mistake"
(3) run "echo a9a5465eb44d0d8f1c3c9d288b7f23f628ddb50b >master"
(4) run "git push --mirror https://github.com/sqlite/sqlite.git";

This was a one-time fix.  I have not yet enhanced the mirroring
mechanism to make this happen automatically, but probably I will soon.

But before I proceed, I would like to better understand how rewiring
the refs this way constitutes "changing history".  The refs/heads
entries are all ephemeral - they are constantly changing on their own,
and no historical record of their past values is retained.  So if I
modify the refs to synchronize with the canonical Fossil repository,
how is that changing history, exactly?

Any further explanation is appreciated.

--
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Please help me fix the SQLite Git mirror

2019-04-22 Thread Carl Edquist 

Hi Richard,

As Jonathan mentioned, in git land, if you have already published a 
"mistake" commit publicly, the proper way to revert it is to make another 
commit to reverse/undo the change.


By removing a commit from the public history of the published 'master' 
branch, it forces everyone downstream to manually fix their history.


If they do a normal "git pull", git will attempt to merge their master 
(the mistake commit) with the latest upstream master, which is not 
actually your intention.


But if you make an "revert" commit to undo the change, history will 
continue forward for the master branch from the downstream perspective.




I fixed the recent breakage of the SQLite Git mirror as follows:

(1) cd into the refs/heads directory
(2) run "cat master >mistake"
(3) run "echo a9a5465eb44d0d8f1c3c9d288b7f23f628ddb50b >master"
(4) run "git push --mirror https://github.com/sqlite/sqlite.git";


Not that you want to do it this way again if you can avoid it, but the 
safe git way to do (2),(3) is:


2) git update-ref refs/heads/mistake refs/heads/master
3) git update-ref refs/heads/master a9a5465eb44d0d8f1c3c9d288b7f23f628ddb50b


Carl

On Mon, 22 Apr 2019, Richard Hipp wrote:


Thanks for the help.  See additional questions and remarks below

On 4/22/19, Jonathan Brandmeyer  wrote:

```
# Construct the matching branch name
git branch mistake 9b888fc
# Push the name alone to the remote
git push -u origin mistake
# Move the name of master
git checkout master && git reset --hard 
# Push the new name of master
git push --force
```

Git reset --hard will move the name of the current working branch to
another branch SHA, which is why you need to first check out the
branch being moved: Its context sensitive.  You are re-writing
history, though.


I don't understand this part.  From the Fossil perspective, moving a
check-in from one branch to another is just adding a new tag to that
check-in.  No history is changed.  The DAG of check-ins (the
block-chain) is unmodified.

Subsequent to your message, I fixed the recent breakage of the SQLite
Git mirror as follows:

(1) cd into the refs/heads directory
(2) run "cat master >mistake"
(3) run "echo a9a5465eb44d0d8f1c3c9d288b7f23f628ddb50b >master"
(4) run "git push --mirror https://github.com/sqlite/sqlite.git";

This was a one-time fix.  I have not yet enhanced the mirroring
mechanism to make this happen automatically, but probably I will soon.

But before I proceed, I would like to better understand how rewiring
the refs this way constitutes "changing history".  The refs/heads
entries are all ephemeral - they are constantly changing on their own,
and no historical record of their past values is retained.  So if I
modify the refs to synchronize with the canonical Fossil repository,
how is that changing history, exactly?

Any further explanation is appreciated.

--
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Please help me fix the SQLite Git mirror

2019-04-22 Thread Jonathan Brandmeyer 
On Mon, Apr 22, 2019 at 12:22 PM Richard Hipp  wrote:

> But before I proceed, I would like to better understand how rewiring
> the refs this way constitutes "changing history".  The refs/heads
> entries are all ephemeral - they are constantly changing on their own,
> and no historical record of their past values is retained.  So if I
> modify the refs to synchronize with the canonical Fossil repository,
> how is that changing history, exactly?
>

Certainly no new SHA's were created, so this is much less obvious of a
re-write than if you had performed a rebase of some kind.
Nonetheless, I claim that this constitutes rewriting history because
it has a similar impact to downstream users.  Some user-visible
symptoms, after a user had already synchronized to the master which
was later abandoned:

- From a context of master, `git pull` alone would construct a merge
commit between the abandoned branch and the new master.  `git pull
--ff-only` would fail.

- From a context of a custom patch series, `git rebase master` has
unexpected effects, in that it also rebases the mistake you tried to
orphan.

- `git fetch` shows a forced update to origin/master.

- A user who was using a merge-based workflow and had merged to your
mistake branch would have a rough time following the change in branch
name.  One method would be to construct a new merge to the newly
corrected master and then rebase any of their subsequent changes onto
the new merge commit.  Their workflow is no longer strictly
merge-based and they still have to deal with the impacts of re-writing
their history.  Alternatively, they could construct the inverse of the
mistake via `git revert` onto their own working branch and then merge
again against the new master.

These user-visible impacts and the recovery actions are almost the
same as what a Git user would see if you had initially constructed (A,
B, C, D) and re-written it to be (A, C', D') instead via a rebase.

IMO, the proper corrective action after pushing the commit with a
mistake in it would have been to commit the inverse of the mistake and
then merge it to the alternate path.  Yes, it would have constructed a
merge commit in the history, which is unfortunate when you are trying
to maintain a clean and linear history.  But the impact to downstream
users would have been negligible.  `git pull --ff-only` would have
Just Worked, `git rebase master` from a patch series would have Just
Worked, and a merge-based workflow would have Just Worked, too.

--
Jonathan Brandmeyer
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Please help me fix the SQLite Git mirror

2019-04-22 Thread Richard Hipp 
Thanks for the help.  See additional questions and remarks below

On 4/22/19, Jonathan Brandmeyer  wrote:
> ```
> # Construct the matching branch name
> git branch mistake 9b888fc
> # Push the name alone to the remote
> git push -u origin mistake
> # Move the name of master
> git checkout master && git reset --hard 
> # Push the new name of master
> git push --force
> ```
>
> Git reset --hard will move the name of the current working branch to
> another branch SHA, which is why you need to first check out the
> branch being moved: Its context sensitive.  You are re-writing
> history, though.

I don't understand this part.  From the Fossil perspective, moving a
check-in from one branch to another is just adding a new tag to that
check-in.  No history is changed.  The DAG of check-ins (the
block-chain) is unmodified.

Subsequent to your message, I fixed the recent breakage of the SQLite
Git mirror as follows:

(1) cd into the refs/heads directory
(2) run "cat master >mistake"
(3) run "echo a9a5465eb44d0d8f1c3c9d288b7f23f628ddb50b >master"
(4) run "git push --mirror https://github.com/sqlite/sqlite.git";

This was a one-time fix.  I have not yet enhanced the mirroring
mechanism to make this happen automatically, but probably I will soon.

But before I proceed, I would like to better understand how rewiring
the refs this way constitutes "changing history".  The refs/heads
entries are all ephemeral - they are constantly changing on their own,
and no historical record of their past values is retained.  So if I
modify the refs to synchronize with the canonical Fossil repository,
how is that changing history, exactly?

Any further explanation is appreciated.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Please help me fix the SQLite Git mirror

2019-04-22 Thread Jonathan Brandmeyer 
```
# Construct the matching branch name
git branch mistake 9b888fc
# Push the name alone to the remote
git push -u origin mistake
# Move the name of master
git checkout master && git reset --hard 
# Push the new name of master
git push --force
```

Git reset --hard will move the name of the current working branch to
another branch SHA, which is why you need to first check out the
branch being moved: Its context sensitive.  You are re-writing
history, though.  It shouldn't construct any new SHA's, but the impact
on a downstream user's workflow is rough.  Once it got published to
public git the least impactful way forward would be to construct the
inverse of the mistake and push that as its own commit instead of
orphaning it.  `git revert` does this in git-land.

If I'm maintaining some patches against your master, then my normal
workflow might be to rebase them against the current master every once
in a while, with just `git rebase master`.  If I did that once to
rebase against the SHA which was is currently named `master`, and then
invoke `git rebase master` again after your change to history, then
the second rebase will also attempt to rebase your mistake onto the
corrected master.  User's would need to perform a one-time `git rebase
--onto master mistake ` instead.


-- 
Jonathan Brandmeyer
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Please help me fix the SQLite Git mirror

2019-04-22 Thread Richard Hipp 
On 4/22/19, Jeffrey Schiller  wrote:
> So if I understand you correctly, you just want to make "master" point to a
> particular known commit. To do this, you can issue the commands (in a local
> copy):
>
> git branch -m master oldmaster # Move it out of the way
> git branch master 4f35b3b7
>

This is to be done via automation.  I don't want to have to write,
debug, and test the code to detect whether or not there is an existing
"master" that needs to be moved out of the way.  I'd rather do the
equivalent of REPLACE INTO - overwriting the existing ref it exists
and create a new one if it does not.  How might that be done in Git?


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Please help me fix the SQLite Git mirror

2019-04-22 Thread Jeffrey Schiller 
So if I understand you correctly, you just want to make "master" point to a
particular known commit. To do this, you can issue the commands (in a local
copy):

git branch -m master oldmaster # Move it out of the way
git branch master 4f35b3b7

Then do a "git push -f origin master" (assuming that the github repo is
defined as "origin", replace that with whatever name you use for the remote
if it isn't origin).

-Jeff

On Mon, Apr 22, 2019 at 12:05 PM Richard Hipp  wrote:

> The Git mirror of SQLite found at https://github.com/sqlite/sqlite is
> busted.  I don't know how to fix it and would appreciate advice from
> people who have more experience with Git internals.
>
> To describe the problem, consider this excerpt from the check-in
> sequence for SQLite:
>
> https://www.sqlite.org/src/timeline?d=6de980a09c3a7adf&n=5
>
> Notes to Git-ers:  (1) Graph arrows in Fossil point forwards in time,
> not backwards as Git does.  In other words, the arrows point from
> parent to child, not from child to parent.  (2) The main branch is
> called "trunk" in Fossil instead of "master".  The name is changed
> automatically during the mirroring process.
>
> What happened here is that the 9b888fcc check-in was originally on
> trunk/master.  But after it was checked in, I discovered a problem
> with it.  So I diverted that check-in off into the "mistake" branch
> (which you can do in Fossil by adding a special tag.)  Then the
> check-in sequence for trunk/master continued with 6cf16703 and
> 4f35b3b7 and so forth.
>
> The problem is that Git now thinks that 9b888fcc is the HEAD of master
> and that the true continuation of master (check-in 4f35b3b7 and
> beyond) are disconnected check-ins, awaiting garbage collection.
> There is no "ref" pointing to the HEAD of the true continuation.
>
> I think what I need to do is change refs/heads/master to point to
> 4f35b3b7 (or whatever check-ins come afterwards - the snippet shown is
> not the complete graph).  Then create a new entry refs/heads/mistake
> that points to 9b888fcc.
>
> Question 1:  Does my analysis seem correct.  Or have I misinterpreted
> the malfunction?
>
> Question 2:  Assuming that my analysis is correct, what is the
> preferred way of rewiring the refs in Git?
>
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Please help me fix the SQLite Git mirror

2019-04-22 Thread Richard Hipp 
The Git mirror of SQLite found at https://github.com/sqlite/sqlite is
busted.  I don't know how to fix it and would appreciate advice from
people who have more experience with Git internals.

To describe the problem, consider this excerpt from the check-in
sequence for SQLite:

https://www.sqlite.org/src/timeline?d=6de980a09c3a7adf&n=5

Notes to Git-ers:  (1) Graph arrows in Fossil point forwards in time,
not backwards as Git does.  In other words, the arrows point from
parent to child, not from child to parent.  (2) The main branch is
called "trunk" in Fossil instead of "master".  The name is changed
automatically during the mirroring process.

What happened here is that the 9b888fcc check-in was originally on
trunk/master.  But after it was checked in, I discovered a problem
with it.  So I diverted that check-in off into the "mistake" branch
(which you can do in Fossil by adding a special tag.)  Then the
check-in sequence for trunk/master continued with 6cf16703 and
4f35b3b7 and so forth.

The problem is that Git now thinks that 9b888fcc is the HEAD of master
and that the true continuation of master (check-in 4f35b3b7 and
beyond) are disconnected check-ins, awaiting garbage collection.
There is no "ref" pointing to the HEAD of the true continuation.

I think what I need to do is change refs/heads/master to point to
4f35b3b7 (or whatever check-ins come afterwards - the snippet shown is
not the complete graph).  Then create a new entry refs/heads/mistake
that points to 9b888fcc.

Question 1:  Does my analysis seem correct.  Or have I misinterpreted
the malfunction?

Question 2:  Assuming that my analysis is correct, what is the
preferred way of rewiring the refs in Git?

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users