Re: [openstack-dev] [git-upstream] [Duplicate Changes] How to view only commits applied since last import
On 18/11/16 15:27, Darragh Bailey - mailing lists wrote: > > At this point, now you'll see X, X' & X'', and Y, Y' & Y''. Obviously > this cause a bit of confusion when listing the changes using: > > git log --oneline --graph E~1..local/mitaka > > you see something like the following (see [1] for how I created this): > > * 899cb6e [O] Merging Y2 into N > |\ > | * 3c08f48 [Y] Adding tmp7wtzvo69 <--- really Y'' > | * db8d2c3 [X] Adding tmpnxot0u9s > | * 97cc90c [I] Adding tmps2xhxp2f > * | 9ea35c3 [N] Merging Y1 into Y > |\ \ > | * | f361e9f [Y] Adding tmp7wtzvo69 <--- really Y' > | * | 90d58eb [X] Adding tmpnxot0u9s > | |/ > | * ed973e6 [G] Adding tmpb443aabz > | * 74cd9b8 [E] Adding tmpwcrm4bxi > * 3cc85cf [Y] Adding tmp7wtzvo69 <--- original Y > * e93f6cb [X] Adding tmpnxot0u9s I forgot to add the reference for how I generated that tree: git clone git://git.openstack.org/openstack/git-upstream cd git-upstream cat < test.yaml - tree: - [A, []] - [E, [A]] - [G, [E]] - [I, [G]] - [X, [A]] - [Y, [X]] - [X1, [G]] - [Y1, [X1]] - [N, [Y, =Y1]] - [X2, [I]] - [Y2, [X2]] - [O, [N, =Y2]] branches: head: [local/mitaka, O] upstream: [stable/mitaka, I] EOF tox -e build-tree -- test.yaml cd .git-test-trees/test -- Regards, Darragh Bailey IRC: electrofelix "Nothing is foolproof to a sufficiently talented fool" - Unknown __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [git-upstream] [Duplicate Changes] How to view only commits applied since last import
On 17/11/16 12:47, Paul Bourke wrote: > Hi Darragh / git-upstream community, > > I've been looking at a way to easily view a log of what commits made > since the last upstream import when managing a branch with git-upstream. > Right now this can be hard to do - something like 'git log > upstream/master..HEAD' shows a lot of duplicate commits reasons I don't > understand well enough to explain. As mentioned I thought it might be worth addressing this piece separately, and then hopefully re-factor what's here to be added to the git-upstream docs. To start with, lets step through what things look like, and then I'll try explain why git-upstream merges history in this way. Assuming there are two local changes being carried locally that have not yet been accepted upstream. These were initally added to a local/mitaka branch and two imports were subsequently performed and the changes have not yet landed on stable/mitaka: X---Y---NO local/mitaka / // / X'--Y' / (rebase of X & Y onto G) / // / / X''--Y'' / / / -E---G---Iupstreams stable/mitaka branch When syncing latest from upstream stable/mitaka was initially at G, and this resulted git-upstream replaying X' being onto G and then creating a merge commit N which has exactly the same contents as the tree at X'. When you look at the history of N you'll see: X---Y---N / / / X'--Y' / / -E---G And then looking at O shows X---Y---NO local/mitaka / // / X'--Y' / / // / / X''--Y'' / / / -E---G---I At this point, now you'll see X, X' & X'', and Y, Y' & Y''. Obviously this cause a bit of confusion when listing the changes using: git log --oneline --graph E~1..local/mitaka you see something like the following (see [1] for how I created this): * 899cb6e [O] Merging Y2 into N |\ | * 3c08f48 [Y] Adding tmp7wtzvo69 <--- really Y'' | * db8d2c3 [X] Adding tmpnxot0u9s | * 97cc90c [I] Adding tmps2xhxp2f * | 9ea35c3 [N] Merging Y1 into Y |\ \ | * | f361e9f [Y] Adding tmp7wtzvo69 <--- really Y' | * | 90d58eb [X] Adding tmpnxot0u9s | |/ | * ed973e6 [G] Adding tmpb443aabz | * 74cd9b8 [E] Adding tmpwcrm4bxi * 3cc85cf [Y] Adding tmp7wtzvo69 <--- original Y * e93f6cb [X] Adding tmpnxot0u9s If this is limited further to: git log --oneline --graph stable/mitaka..local/mitaka * 899cb6e [O] Merging Y2 into N |\ | * 3c08f48 [Y] Adding tmp7wtzvo69 <--- really Y'' | * db8d2c3 [X] Adding tmpnxot0u9s * 9ea35c3 [N] Merging Y1 into Y |\ | * f361e9f [Y] Adding tmp7wtzvo69 <--- really Y' | * 90d58eb [X] Adding tmpnxot0u9s * 3cc85cf [Y] Adding tmp7wtzvo69 * e93f6cb [X] Adding tmpnxot0u9s <--- original Y This looks like the same 2 changes have been applied three times, and in a way, they have. This obviously can be confusing. Hence you're proposed change to provide a way to display only the interesting commits so instead would see: * 899cb6e [O] Merging Y2 into N * 3c08f48 [Y] Adding tmp7wtzvo69 <--- really Y'' * db8d2c3 [X] Adding tmpnxot0u9s > Thanks in advance for anything that might help cut through some of the > confusion. > > Cheers, > -Paul Back to the question as to why git-upstream does it this way: Looked at possibility of merges, just land patches against the tree and then on a regular basis attempt to merge in the latest from upstream. * It's possible (if a little awkward at the moment) to extract information on how many local patches are being carried. Merging would meant that you wouldn't see the same patches from the series duplicated, but you would have multiple commits in your local history that are likely not identifiable as cherry-picks of the final accepted change upstream (how many patches are accepted without any changes?) * Re-applying changes allows for conflicts with changes to be resolved in the relevant patch. Otherwise the conflicts are resolved inside a merge commit can be quite hard to review. Subsequently when the changes are accepted upstream, and then merged in on the next sync the conflicts generated will frequently be different since the change accepted upstream will be slightly different. Using a patch series approach allows to automatically drop the duplicate changes, and re-apply an update series, using a manual merge commit with conflict resolution means likely someone is going to have to spend more time resolving conflicts. * Want to avoid rebasing published branches. As far as developers inside a company are concerned, the local branches are published history, and for them to co-operate when issues need to be fixed for internal testing/releases to continue, rewriting history is likely going to make life more difficult. Rebasing a published branch can work, and I know some linux kernel developers have
Re: [openstack-dev] [git-upstream] [Additional Branches] How to view only commits applied since last import
On 17/11/16 12:47, Paul Bourke wrote: > Hi Darragh / git-upstream community, My current understanding is we have a branch, that consists of a mixture > of upstream commits from previous imports, and custom commits. We want > to show a list of commits added since the last import. However, if those > commits also contain commits from another non upstream branch, we want > to exclude those? This makes sense with the example of say a packaging > branch, but what if commits came from say a feature branch? Does it also > make sense to ignore those? Probably worth explaining additional branches a bit more in the docs (where did I say that before?) For any feature branch that is contributing to the tree and touching the same files as the upstream project, definitely they should be included, and git will do so by default. The packaging branch item stems from when we were carrying packaging related files that would never go upstream, and were actually from a completely separate history with it's own root, and own upstream as we did have some previously tracking ubuntu packaging repos to pull in the packaging files from there. However I think can regard the additional branches as just being an edge case that if they get excluded from the listed output, it's not the worst case. If it makes sense down the road to include them, or just ensure something is included about them, hopefully I'll have had a chat with some more git experts and might know how to do so. A quick off the top of my head guess. Given the graph mentioned in the other email: A / \ -M---X---N---O local/mitaka / / / X'---(rebase/cherry-pick of X onto G) / / -E---F---G upstream stable/mitaka branch Adding '--not M..N^1' might do the right thing in only excluding a range rather than everything behind it, though would need to check with a scenario that if another commit existed between X and N, that this wouldn't accidentally exclude A as well. > Thanks in advance for anything that might help cut through some of the > confusion. > > Cheers, > -Paul In any case, I think don't worry about the additional branches for now, just let them be excluded, as they way they are merged in only works if they are unrelated history. Since any feature branches wouldn't be merged in this way, they would get included the same way any other commit that is merged to the tree currently does, just that there will be more of them listed. -- Regards, Darragh Bailey IRC: electrofelix "Nothing is foolproof to a sufficiently talented fool" - Unknown signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [git-upstream] How to view only commits applied since last import
Hi Paul This might be a bit long, noticed a few more items to be added to the documentation for git-upstream. Good thing documentation is the major item for the next release ;-) After an initial attempt at writing one response, I've decided to break it up a bit into a couple of emails, because there a few topics in here and one of them is pretty lengthy and the others follow somewhat independent issues. For this one I'll stick to just the 2nd and 4th paragraphs since they are related. And apologies in advance for anyone that didn't need to see such a dive into git log behaviour here. :-) On 17/11/16 12:47, Paul Bourke wrote: > Hi Darragh / git-upstream community, > > I've been looking at a way to easily view a log of what commits made > since the last upstream import when managing a branch with git-upstream. > Right now this can be hard to do - something like 'git log > upstream/master..HEAD' shows a lot of duplicate commits reasons I don't > understand well enough to explain. I'll go into that in a little more detail with a separate response, to help anyone else reading or in the future searching. Definitely should have been documented before now... O_o > Darragh had suggested using the --dense option will help here, so 'git > log --dense upstream/master..HEAD -- .' seems to generally give the > right result. I've put up a patch to add this as a git-upstream command > in the form of "carrying" at https://review.openstack.org/#/c/381669/. > However, it seems there are in fact cases where the above command will > show incorrect commits, and I'm struggling a bit to fully grok this. Unfortunately explanation is going to be a bit lengthy, and there are pieces of git here I'm not fully sure I understand what it's doing, and plan to ask the git-users google group for a bit more info. Possibly may require going to the git mailing list to get a complete picture (g...@vger.kernel.org). Until recently, I didn't fully grok the git-log(1) manpage, https://www.kernel.org/pub/software/scm/git/docs/git-log.html, specifically the section about "History Simplification", naturally I didn't realise that until I understood what it was saying better. Mainly to do with how TREESAME and !TREESAME apply to simplifying the history walked and displayed. TREESAME means that the resulting git tree is identical and you can have multiple commits referencing the same TREE SHA1 in git. It's one of the ways it avoids duplicating the same information, identical directories result in the same SHA1, so each commit to a git tree only adds blobs (for files) and tree (for directory) objects for what is different. !TREESAME basically means there was something different between the two directories or trees in the repo, and as git allows for path limiting, this can be applied to any subdirectory in a git repository as well. In the review I was referencing a particular scenario that we've come across locally (see the url above): 1) Dev uploads a patch 'A' for review against branch tracking latest from stable/mitaka (presumably also sent it upstream) 2) Nightly sync jobs run and bring down all the patches that landed in the stable branch upstream over the last 24 hours 2.1) Git upstream is able to replay the current local changes without conflict, dropping those that landed upstream automatically 2.2) Git-upstream merges the result into the tracking branch, and it now becomes the new head (assuming it passes a gate check locally) 3) Following day or so, reviewer approves patch 'A' to land and it merges without conflict. See following for the actual test scenario, it's a bit more complex than what I've discribed and the letters used differ: https://github.com/openstack/git-upstream/blob/a52f01f89401e79db95ad9fee9195de90e5a71f2/git_upstream/tests/searchers/scenarios/changes_upload_prior_to_import.yaml If git-upstream didn't do a special type of merge, and just produced a new branch each time, we'd have to re-target patch 'A' to the new branch, alternatively we could also have triggered a rebase of any outstanding changes to avoid the particular git graph this produces. Both of these seemed like unnecessary confusion and make-work for developers/reviewers just trying to focus on landing changes. The result is a git graph like the following (hope this displays correctly): A / \ -M---X---N---O local/mitaka / / / X'---(rebase/cherry-pick of X onto G) / / -E---F---G upstream stable/mitaka branch M = the merge commit created by git-upstream for the previous sync X = local change not yet upsteam X' = replay of the local change X onto the latest stable/mitaka N = most recent merge commit created by git-upstream A is the commit created locally by a developer to fix a issue affecting local deployments (and may need to be reworked for upstream) O = merge commit created in Gerrit by approval of 'A' When git-upstream created N, it made t
[openstack-dev] [release] git-upstream 0.12.1 release
Pleased to announce the 0.12.1 release of git-upstream. Main reason to ensure the man page is installed by default. With source available at: http://git.openstack.org/cgit/openstack/git-upstream Please report any issues through launchpad: https://bugs.launchpad.net/git-upstream git-upstream is an open source Python application that can be used to keep in sync with upstream open source projects, mainly OpenStack. For more info on what git-upstream is for: https://pypi.python.org/pypi/git-upstream Full changelog available from: https://git.openstack.org/cgit/openstack/git-upstream/tree/ChangeLog For more details see below. Changes in git-upstream 0.12.0..0.12.1 -- bbf2511 Update ChangeLog for 0.12.1 98b283e Include files for sphinx based documentation b95b257 Auto build and install default man page ac76015 Move build_manpage utility to doc area bc9fc64 Ensure correct prog name available to manpage d2c0306 Update authors 2455b7e Remove requirement sphinxcontrib-programoutput Diffstat (except docs and test files) - AUTHORS| 5 + ChangeLog | 12 +++ build_manpage.py | 136 - doc/source/conf.py | 246 + doc/source/index.rst | 21 doc/utils/build_manpage.py | 151 docs-requirements.txt | 2 + git_upstream/main.py | 3 +- setup.cfg | 10 ++ setup.py | 5 + test-requirements.txt | 1 - tox.ini| 4 +- 12 files changed, 457 insertions(+), 139 deletions(-) Requirements updates diff --git a/test-requirements.txt b/test-requirements.txt index 63c4a34..3f35ec1 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -9,5 +9,4 @@ python-subunit testrepository>=0.0.17 testscenarios>=0.4 testtools>=0.9.32 -sphinxcontrib-programoutput PyYAML>=3.1.0 -- Regards, Darragh Bailey IRC: electrofelix "Nothing is foolproof to a sufficiently talented fool" - Unknown (possibly Douglas Adams) 0xCDD2EB9C.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [release] git-upstream 0.12.0 release
Pleased to announce the 0.12.0 release of git-upstream. With source available at: http://git.openstack.org/cgit/openstack/git-upstream Please report any issues through launchpad: https://bugs.launchpad.net/git-upstream git-upstream is an open source Python application that can be used to keep in sync with upstream open source projects, mainly OpenStack. For more info on what git-upstream is for: https://pypi.python.org/pypi/git-upstream Full changelog available from: _https://git.openstack.org/cgit/openstack/git-upstream/tree/ChangeLog_ For more details see below. Changes in git-upstream 0.11.0..0.12.0 -- d50f0a3 Changelog for 0.12.0 Release 7dd634a Fix help subcommand 5c0a4a7 Remove compatibility hacks causing issues with newer GitPython 691ae78 Check valid import branch argument value 0e30d38 Report import branch name used by default f243bd6 Use tag for import branch naming when given be781cc Support tags for input args 5f4038e Allow SHA1 as reference for inputs 7897e35 Use python for pre/post scripts 47092a9 Fix finish merge to ensure correct contents 54d4876 Always have rebase perform finish a24debc Ensure correct mode of git-rebase executed d8bb479 Ensure parent options retained for exec'ed finish 9e077c5 Support old GitPython transform_kwargs signature 79b31d9 Refactor test to allow additional 3b5e241 Fix access for pre-script dict member b7dc443 Update .gitignore with .eggs path for setuptools 7a135a1 Enable default GIT_PYTHON_TRACE for tests 26c4274 Consolidate to a single separator character 10e8f92 Example in USAGE.md should reference .gitreview c195b69 Fix broken interactive mode to be usable 307637d Record upstream branch in import merge commit e231483 Handle all local changes landed upstream 8ab6ccc Detect cherry-picked changes without Change-Id a05425f Use a recent pbr f37890b Add tool to recreate git repo from test scenarios 90bf42d Detect when nothing to import 343c3e7 Merge multiple unrelated additional branches 62def85 Support for python 3 0037786 Show rebase help during interactive import 45a617a Simplify USAGE.md example by using a local remote eaaeae8 Fix / simplify the example patch in USAGE.md 54c40a7 Minor change to the USAGE.md 69ed25e Handle missing merge during initial import 1810d1e Include parent commits in git graph 5687a7b Match name and result of is_detached() method 81ad238 Install coverage and fix command line to call 246439b Refactor import command tests to use scenarios af6f690 Fix a tiny typo in USAGE 30f6bf2 Update package author/maintainer details 754db13 Consolidate note reference and header constants 4a55a31 Consolidate BadName exception compatibility 9640363 Keep modifications to GitPython objects in single module fb05151 Support test specific scripts to modify test areas 677c17a Ensure end of arguments marked clearly Diffstat (except docs and test files) - .gitignore | 2 + ChangeLog | 50 +++ USAGE.md | 49 +++ git_upstream/__init__.py | 3 + git_upstream/commands/__init__.py | 2 +- git_upstream/commands/help.py | 12 +- git_upstream/commands/import.py| 98 +++-- git_upstream/lib/__init__.py | 18 +++ git_upstream/lib/drop.py | 20 +-- git_upstream/lib/importupstream.py | 173 -- git_upstream/lib/note.py | 5 - git_upstream/lib/pygitcompat.py| 100 ++--- git_upstream/lib/rebaseeditor.py | 194 + git_upstream/lib/searchers.py | 142 ++ git_upstream/lib/strategies.py | 26 +++- git_upstream/lib/supersede.py | 22 +-- git_upstream/lib/utils.py | 5 +- git_upstream/log.py| 9 +- git_upstream/main.py | 22 ++- git_upstream/rebase_editor.py | 44 +++--- git_upstream/resources/todo_epilogue_1_7_5.txt | 18 +++ git_upstream/resources/todo_epilogue_2_6_0.txt | 19 +++ requirements.txt | 11 +- setup.cfg | 9 +- test-requirements.txt | 3 +- tox.ini| 9 +- 26 files changed, 660 insertions(+), 405 deletions(-) Requirements updates diff --git a/requirements.txt b/requirements.txt index 9401cff..e93d264 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,4 +1,9 @@ -pbr>=0.5.21,<1.0 - argcomplete -GitPython>=0.3.2.RC1,!=0.3.2 +# GitPython 1.0.1 required by openstack global-requirements +# python>=3 should work with >=0.3.4 +# python<3 should work with >=0.3.2.RC1,!=0.3.2 +GitPyth
