Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Thu, 2017-03-09 at 21:13 +1100, Brian May wrote: > Brian May writes: > > > git read-tree --reset -u upstream > > git reset -- debian > > git checkout debian > > git rm debian/.git-dpm > > I have tried these steps on python-mkdocs in the debian/experimental > branch, and then upgraded to the latest upstream (using instructions on > wiki). Works perfectly[1]. > > The only unexpected problem I had is that "gbp import-orig --uscan", by > default, switches to the master branch and attempts to merge the new > upstream there. Which wasn't going to work, because master still is the > patches-applied git-dpm version. I had assumed that it would work on the > current branch; it doesn't. You can override the target debian / upstream branches with `gbp import-orig --debian-branch=debian/experimental --upstream- branch=upstream/latest`. Long-term you'd want to write your DEP-14 compliant configuration in debian/gbp.conf indeed. Ghis
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
❦ 9 mars 2017 21:13 +1100, Brian May : > The above also assumes that upstream==origin/upstream==latest > upstream. Which means you need to have done a git pull recently on the > upstream branch. Depending on the circumstances, using origin/upstream > might be a better choice rather then upstream. Only if everyone names the origin branch origin. The alternative is to use "gbp pull" instead of "git pull". -- Make sure every module hides something. - The Elements of Programming Style (Kernighan & Plauger) signature.asc Description: PGP signature
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
Brian May writes: > git read-tree --reset -u upstream > git reset -- debian > git checkout debian > git rm debian/.git-dpm I have tried these steps on python-mkdocs in the debian/experimental branch, and then upgraded to the latest upstream (using instructions on wiki). Works perfectly[1]. The only unexpected problem I had is that "gbp import-orig --uscan", by default, switches to the master branch and attempts to merge the new upstream there. Which wasn't going to work, because master still is the patches-applied git-dpm version. I had assumed that it would work on the current branch; it doesn't. The wiki should be ammended with instructions on how to edit debian/gbp.conf at the appropriate steps. Writing a new default to debian/gbp.conf does fix this. The above also assumes that upstream==origin/upstream==latest upstream. Which means you need to have done a git pull recently on the upstream branch. Depending on the circumstances, using origin/upstream might be a better choice rather then upstream. Notes: [1] Well apart from a new "source-is-missing" lintian warning, probably from the new upstream. So probably not ready for upload just yet. -- Brian May
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Feb 14 2017, Scott Kitterman wrote: > How does dgit avoid maintainer forgot to push problems without being > limited to the granularity of one commit per upload? It does not. Until 'dgit push' is called the next time, it will revert to one commit per upload for the "dark" period. I am not sure if the next dgit push will retroactively fix history, or only affect changes that have not yet been uploaded. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Tuesday, February 14, 2017 06:23:43 PM Brian May wrote: > Scott Kitterman writes: > > We know in the DPMT context what debcheckout will produce, so for our > > purposes they don't matter. > > > > How does dgit avoid maintainer forgot to push problems without being > > limited to the granularity of one commit per upload? > When you upload a package, you upload the *.debs and push to git a the > same time. With the one dgit command that checks everything is > consistant, and tags things appropriately. I not sure of the exact > details of how the push is done yet. > > I think dgit would really help with the problem I occasionally get: Does > this git source really correspond with the package that was > uploaded. Mistakes can happen in git that can result in you looking at > one git version that is very different to what was uploaded. Yes, this > does happen. > > Mostly however, I think the prime benefit of using dgit would be that it > helps other non-team members maintain the package - as does happen from > time to time in the form on NMUs. We can help these people by sticking > to a standard that others can use. > > It would not directly help DPMT workflow, as that mostly remains as > is. Hence my first priority would be to change to GBP PQ for work flow, > and then worry about dgit after I have had a chance to play with dgit a > bit more. Thanks. I agree we should wait on this. Scott K
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
Barry Warsaw writes: > On Feb 14, 2017, at 06:30 PM, Arto Jantunen wrote: >>The patch-queue branch is based on the Debian branch, not upstream. Try >>merging the new upstream version to your Debian branch, and then running >>gbp pq rebase. > > This confuses me, or I'm doing something wrong. With git-dpm the way to drop > patches was to rebase interactively against upstream. That doesn't seem to > work with gbp pq rebase, or with gbp pq import & git rebase -i master (or > upstream). > > So how do I drop a patch with gbp-pq? The later works for me. Since there is no gbp pq rebase -i (perhaps there should be one?), this is what I do: gbp pq import git rebase -i master gbp pq export And git status shows the patch as deleted, and removed from the series file. -- Arto Jantunen
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Tue, 14 Feb 2017 at 11:44:33 -0500, Barry Warsaw wrote: > So how do I drop a patch with gbp-pq? rm debian/patches/this-got-fixed-upstream.patch, vi debian/patches/series, commit? :-) Or more generally, to do it the git way, if the rest of the patch series might need non-trivial adjustment: git checkout debian/master # old version, patches-unapplied gbp pq import # moves to patch-queue/debian/master git checkout debian/master # or gbp pq switch gbp import-orig ../whatever.tar.gz dch git commit -m "New upstream version" git checkout patch-queue/debian/master # or gbp pq switch git rebase -i debian/master gbp pq export # back to debian/master git add debian/patches git commit -m "Refresh patches or whatever" (Substitute master for debian/master if DPMT doesn't use DEP-14, but moving to gbp pq might be a good flag day to do that too. Then you'll never have to get your local version of Debian's master branch mixed up with your local version of upstream's master branch.) S
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Feb 14, 2017, at 06:30 PM, Arto Jantunen wrote: >The patch-queue branch is based on the Debian branch, not upstream. Try >merging the new upstream version to your Debian branch, and then running >gbp pq rebase. This confuses me, or I'm doing something wrong. With git-dpm the way to drop patches was to rebase interactively against upstream. That doesn't seem to work with gbp pq rebase, or with gbp pq import & git rebase -i master (or upstream). So how do I drop a patch with gbp-pq? Cheers, -Barry
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
Barry Warsaw writes: > On Feb 14, 2017, at 05:54 PM, Brian May wrote: >>Maybe something like the following? >> >>git read-tree --reset -u upstream >>git reset -- debian >>git checkout debian >>git rm debian/.git-dpm >>git commit > > This gets closer, but there still seems to be problems. > > gbp pq import > gbp pq export --drop > > That seems to round trip okay, and it just removes a few crufty lines from the > patches. The problem comes when you try to rebase your patches on top of > upstream. > > gbp pq import > git rebase -i upstream > (way way more commits to pick from than expected) The patch-queue branch is based on the Debian branch, not upstream. Try merging the new upstream version to your Debian branch, and then running gbp pq rebase. -- Arto Jantunen
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Feb 14, 2017, at 05:54 PM, Brian May wrote: >Not sure how to unapply all patches. "quilt pop -a" won't work, quilt >doesn't realize the patches are applied. Yep, that does seem to be the problem. >Maybe something like the following? > >git read-tree --reset -u upstream >git reset -- debian >git checkout debian >git rm debian/.git-dpm >git commit This gets closer, but there still seems to be problems. gbp pq import gbp pq export --drop That seems to round trip okay, and it just removes a few crufty lines from the patches. The problem comes when you try to rebase your patches on top of upstream. gbp pq import git rebase -i upstream (way way more commits to pick from than expected) Hmm. -Barry
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
Scott Kitterman writes: > We know in the DPMT context what debcheckout will produce, so for our > purposes they don't matter. > > How does dgit avoid maintainer forgot to push problems without being limited > to the granularity of one commit per upload? When you upload a package, you upload the *.debs and push to git a the same time. With the one dgit command that checks everything is consistant, and tags things appropriately. I not sure of the exact details of how the push is done yet. I think dgit would really help with the problem I occasionally get: Does this git source really correspond with the package that was uploaded. Mistakes can happen in git that can result in you looking at one git version that is very different to what was uploaded. Yes, this does happen. Mostly however, I think the prime benefit of using dgit would be that it helps other non-team members maintain the package - as does happen from time to time in the form on NMUs. We can help these people by sticking to a standard that others can use. It would not directly help DPMT workflow, as that mostly remains as is. Hence my first priority would be to change to GBP PQ for work flow, and then worry about dgit after I have had a chance to play with dgit a bit more. -- Brian May
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
Brian May writes: > git read-tree --reset -u upstream > git reset -- debian > git checkout debian > git rm debian/.git-dpm git commit Of course... -- Brian May
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
Barry Warsaw writes: > One section I think we should add at some point is instructions on how to > manually convert to gbp-pq, at least until we do a mass conversion. Yes agreed. Not sure how to unapply all patches. "quilt pop -a" won't work, quilt doesn't realize the patches are applied. "git checkout upstream -- ." would work, but won't delete any files that were created by the patch. Maybe something like the following? git read-tree --reset -u upstream git reset -- debian git checkout debian git rm debian/.git-dpm The first line sets the tree as per the latest upstream (assuming this is uptodate) and then restores the debian directory that got deleted. Then we just have to delete the debian/.git-dpm file and are finished. -- Brian May
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On February 11, 2017 4:05:46 PM EST, Nikolaus Rath wrote: >On Feb 10 2017, Scott Kitterman wrote: >> On February 9, 2017 8:29:32 PM PST, Nikolaus Rath >wrote: >>>On Feb 10 2017, Scott Kitterman wrote: >No. You are confusing dgit with one particular way to use it. You >can >use dgit with the maint-merge workflow mentioned above, you can use >dgit >with git-dpm, and you can use dgit with gbp. OK. So then I gather it's effectively a layer on top of 'normal' things like gbp-pq or git-dpm? What added value would it provide? >>> >>>Among other things, it enables users to run 'dgit clone', and get an >>>up-to-date, patches-applied copy of the most recent source package. >> >> How is that different/better than debcheckout? > >It works all the time. debcheckout does not guarantee you the newest >version (VCS may lag behind Debian archive), nor does it guarantee a >patches applied, complete package (you might end up with just debian/, >patches-unapplied, or even weirder things). We know in the DPMT context what debcheckout will produce, so for our purposes they don't matter. How does dgit avoid maintainer forgot to push problems without being limited to the granularity of one commit per upload? Scott K
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Feb 13, 2017, at 04:56 PM, Brian May wrote: >There might be errors, as I was going from memory for some of this >stuff. Thanks Brian. I did a quick review (without testing) and it looks pretty good. One section I think we should add at some point is instructions on how to manually convert to gbp-pq, at least until we do a mass conversion. I don't personally expect to do much package maintenance until after Stretch, but when I do, I'll pick a package to test this workflow on. Cheers, -Barry
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
Brian May writes: > https://wiki.debian.org/Python/GitPackagingPQ > > So far it is just a clone, I haven't tried making any changes. I have gone through and made numerous updates. There might be errors, as I was going from memory for some of this stuff. I didn't specify anything for dgit. The only sections that would require changing would be the building and uploading sections, everything else documented here would be unchanged. -- Brian May
Re: git-dpm breakage src:faker
On 01/29/2017 08:17 AM, Arto Jantunen wrote: > Scott Kitterman writes: >> Does that then result in one big undifferentiated mass of diff in the source >> package? > > No, it results in separate patches with their headers intact in the > source package. I assume git-dpm (which I've never used) produces the > same end result. You wish! git-dpm just scrap any patch header lines you may manually edit. This alone is a good reason to get out of it quickly.
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Feb 10 2017, Scott Kitterman wrote: > On February 9, 2017 8:29:32 PM PST, Nikolaus Rath wrote: >>On Feb 10 2017, Scott Kitterman wrote: No. You are confusing dgit with one particular way to use it. You can use dgit with the maint-merge workflow mentioned above, you can use dgit with git-dpm, and you can use dgit with gbp. >>> >>> OK. So then I gather it's effectively a layer on top of 'normal' >>> things like gbp-pq or git-dpm? What added value would it provide? >> >>Among other things, it enables users to run 'dgit clone', and get an >>up-to-date, patches-applied copy of the most recent source package. > > How is that different/better than debcheckout? It works all the time. debcheckout does not guarantee you the newest version (VCS may lag behind Debian archive), nor does it guarantee a patches applied, complete package (you might end up with just debian/, patches-unapplied, or even weirder things). Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
kuLa writes: > dgit indeed is using it's own repos: > https://browse.dgit.debian.org/ > git.dgit.debian.org The way I understand it is that dgit is simply a replacement for the upload operation. To implement this with the required checks, it is recommended (but not required) that you use its build operation. Instead of using "dput" or similar, you use "dgit push". This checks the most recent build and makes sure it was built correctly from git HEAD, and then uploads the build to debian *and* the source to git.dgit.debian.org Apart from build and upload there are no other changes to standard git-dpm or gbp pq procedures. This provides one key benefit for us, we can be reasonably confident that the upload corresponds with a published git source. As an example of an existing dgit package with multiple patches, have a look at Samba: https://browse.dgit.debian.org/samba.git/tree/debian/patches Having said that, it looks like the server might be having problems at the moment, I can't clone any packages. [brian:/tmp] % dgit clone samba canonical suite name for unstable is sid fetching existing git history remote: fatal: Out of memory, calloc failed remote: aborting due to possible repository corruption on the remote side. fatal: protocol error: bad pack header dgit: failed command: git fetch -p -n -q https://git.dgit.debian.org/samba '+refs/tags/archive/debian/*:refs/dgit-fetch/debian/tags/archive/debian/*' '+refs/tags/debian/*:refs/dgit-fetch/debian/tags/debian/*' '+refs/dgit/sid:refs/dgit-fetch/debian/dgit/sid' dgit: subprocess failed with error exit status 128 -- Brian May
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
Brian May writes: > [brian:~/tree … modules/factory-boy] master 255 ± dgit --gbp --clean=git > sbuild --verbose > Format `3.0 (quilt)', need to check/update patch stack > examining quilt state (multiple patches, gbp mode) > dgit: split brain (separate dgit view) may be needed (--quilt=gbp). > dgit: base trees orig=3b9eaf2cb2a9245a1c10 o+d/p=6f1d04c157a8cd0a7eea > dgit: quilt differences: src: ## orig ## gitignores: == orig == > dgit: quilt differences: HEAD ## o+d/p HEAD == o+d/p > dgit: --quilt=gbp specified, implying patches-unapplied git tree > dgit: but git tree differs from orig in upstream files. > > This error puzzles me, to me it seems to be saying there is a mismatch > between git with patches unapplied and the *.orig.tar.gz file - however > I don't think this is the case. Ok, so there really was a difference: the git archive was missing the "docs/_static/.keep_dir" file that is included in the upstream file. Probably good that dgit detects discrepancies such as these, however not so good that it took me sometime to work out the problem based on the given message. If it told me what was different, it would be much be helpful. (also leads to the question why this file was missing from my import last month - but I can't remember what I did now) When I did build the package I can confirm that the generated source package had multiple patch files (as expected). -- Brian May
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On 2017-02-10 21:11:42, Brian May wrote: > Scott Kitterman writes: > > > How is that different/better than debcheckout? > > I am still learning dgit, however I think that dgit uses its own git > repositories. dgit-repos. I am not sure where these are located or how > access is controlled. dgit indeed is using it's own repos: https://browse.dgit.debian.org/ git.dgit.debian.org All DDs are having RW access, but I'm not sure how exactly it's done, service is managed by DSA from what I know. -- |_|0|_| | |_|_|0| "Panta rei" | |0|0|0| kuLa | gpg --keyserver pgp.mit.edu --recv-keys 0x686930DD58C338B3 3DF1 A4DF C732 4688 38BC F121 6869 30DD 58C3 38B3 signature.asc Description: PGP signature
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
Scott Kitterman writes: > How is that different/better than debcheckout? I am still learning dgit, however I think that dgit uses its own git repositories. dgit-repos. I am not sure where these are located or how access is controlled. >From the man page for push "This will push your git history to the dgit-repos, but you probably want to follow it up with a push to alioth." -- Brian May
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
Nikolaus Rath writes: > But it seems to me that you are contemplating whether or not the team > should be using dgit. This is also not a decision that needs to be made > prior to any switch away from dgit-dpm, you can start using dgit at any > point. My vague understanding is that dgit complements the workflow, whatever workflow you may want - including git-dpm and pq. Hence I was surprised and confused when people said it only supported the single patch model - I don't think this is correct. My attempt to build a package (in this case pq based) was less then successful. Possibly because I am not awake. [brian:~/tree … modules/factory-boy] master 255 ± dgit --gbp --clean=git sbuild --verbose Format `3.0 (quilt)', need to check/update patch stack examining quilt state (multiple patches, gbp mode) dgit: split brain (separate dgit view) may be needed (--quilt=gbp). dgit: base trees orig=3b9eaf2cb2a9245a1c10 o+d/p=6f1d04c157a8cd0a7eea dgit: quilt differences: src: ## orig ## gitignores: == orig == dgit: quilt differences: HEAD ## o+d/p HEAD == o+d/p dgit: --quilt=gbp specified, implying patches-unapplied git tree dgit: but git tree differs from orig in upstream files. This error puzzles me, to me it seems to be saying there is a mismatch between git with patches unapplied and the *.orig.tar.gz file - however I don't think this is the case. -- Brian May
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On February 9, 2017 8:29:32 PM PST, Nikolaus Rath wrote: >On Feb 10 2017, Scott Kitterman wrote: >>>No. You are confusing dgit with one particular way to use it. You can >>>use dgit with the maint-merge workflow mentioned above, you can use >>>dgit >>>with git-dpm, and you can use dgit with gbp. >> >> OK. So then I gather it's effectively a layer on top of 'normal' >> things like gbp-pq or git-dpm? What added value would it provide? > >Among other things, it enables users to run 'dgit clone', and get an >up-to-date, patches-applied copy of the most recent source package. How is that different/better than debcheckout? >But it seems to me that you are contemplating whether or not the team >should be using dgit. This is also not a decision that needs to be made >prior to any switch away from dgit-dpm, you can start using dgit at any >point. OK. Scott K
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Feb 10 2017, Scott Kitterman wrote: >>No. You are confusing dgit with one particular way to use it. You can >>use dgit with the maint-merge workflow mentioned above, you can use >>dgit >>with git-dpm, and you can use dgit with gbp. > > OK. So then I gather it's effectively a layer on top of 'normal' > things like gbp-pq or git-dpm? What added value would it provide? Among other things, it enables users to run 'dgit clone', and get an up-to-date, patches-applied copy of the most recent source package. But it seems to me that you are contemplating whether or not the team should be using dgit. This is also not a decision that needs to be made prior to any switch away from dgit-dpm, you can start using dgit at any point. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On February 9, 2017 10:52:04 AM PST, Nikolaus Rath wrote: >On Feb 07 2017, Barry Warsaw wrote: >> On Feb 07, 2017, at 10:47 AM, Ghislain Vaillant wrote: >> >>>I know the discussion is leaning towards replacing usage of git-dpm >>>with gbp-pq. I have nothing against it but, since we are talking >about >>>solutions for a git-centric workflow, has anyone considered the dgit- >>>maint-merge workflow [1]? >> >> As I understand it, there are a few design choices that make dgit >less >> desirable for this team. > >No. You are confusing dgit with one particular way to use it. You can >use dgit with the maint-merge workflow mentioned above, you can use >dgit >with git-dpm, and you can use dgit with gbp. OK. So then I gather it's effectively a layer on top of 'normal' things like gbp-pq or git-dpm? What added value would it provide? Scott K
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Feb 07 2017, Barry Warsaw wrote: > On Feb 07, 2017, at 10:47 AM, Ghislain Vaillant wrote: > >>I know the discussion is leaning towards replacing usage of git-dpm >>with gbp-pq. I have nothing against it but, since we are talking about >>solutions for a git-centric workflow, has anyone considered the dgit- >>maint-merge workflow [1]? > > As I understand it, there are a few design choices that make dgit less > desirable for this team. No. You are confusing dgit with one particular way to use it. You can use dgit with the maint-merge workflow mentioned above, you can use dgit with git-dpm, and you can use dgit with gbp. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
Barry Warsaw writes: > On Feb 05, 2017, at 04:01 PM, Brian May wrote: > >>What should we call the clone page? >> >>PqGitPackaging??? > > GitPackagingPq ? https://wiki.debian.org/Python/GitPackagingPQ So far it is just a clone, I haven't tried making any changes. -- Brian May
Re: Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Tue, Feb 07, 2017 at 10:47:00AM +, Ghislain Vaillant wrote: > I know the discussion is leaning towards replacing usage of git-dpm > with gbp-pq. I have nothing against it but, since we are talking about > solutions for a git-centric workflow, has anyone considered the dgit- > maint-merge workflow [1]? > > [1] https://www.mankier.com/7/dgit-maint-merge Quoting from that manpage: | Debian modifications to the upstream source are squashed into a single diff, | rather than a series of quilt patches. No, please let’s not use this. (I am with ScottK here.) The gbp-pq approach looks fine to me. -- Dmitry Shachnev signature.asc Description: PGP signature
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Feb 07, 2017, at 10:47 AM, Ghislain Vaillant wrote: >I know the discussion is leaning towards replacing usage of git-dpm >with gbp-pq. I have nothing against it but, since we are talking about >solutions for a git-centric workflow, has anyone considered the dgit- >maint-merge workflow [1]? As I understand it, there are a few design choices that make dgit less desirable for this team. The first is that dgit uses single-debian-patch rather than a series of patches as with quilt. The individual patches can be viewed in git but that implies more work for interacting with upstreams and requires the use of the git repo to examine the individual patches, making it harder for non-developers or others outside of Debian to see what we've done to their packages. The second is that dgit prefers to use the upstream git repo but our work is heavily orig-tar based since our main input is PyPI and there orig-tars (or zips) are the predominant distribution format. This may not be a showstopper since dgit does say it will work with tarball workflows, but I don't know how natural that is. Cheers, -Barry
Re: Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Tue, 07 Feb 2017 at 10:47:00 +, Ghislain Vaillant wrote: > I know the discussion is leaning towards replacing usage of git-dpm > with gbp-pq. I have nothing against it but, since we are talking about > solutions for a git-centric workflow, has anyone considered the dgit- > maint-merge workflow? The dgit-maint-merge man page starts with some axioms: The workflow makes the following opinionated assumptions: · Git histories should be the non-linear histories produced by git-merge(1), preserving all information about divergent development that was later brought together. I don't think that is actually a useful model of distro development. I'm sure the rest of dgit-maint-merge(7) is a perfectly reasonable workflow when you start from its assumptions, but I think they're the wrong assumptions. I think the downstream maintainer job in vendors like Debian (and Red Hat, etc.) is essentially a rebasing patch series, whether we represent it like that in git or not: * we track an upstream version, which we treat as somehow canonical[1] * we track what the downstream does as divergence from that upstream version, and only secondarily as a product in its own right (this is a core assumption in the design of all the non-native dpkg-source formats, and also SRPMs, so this is clearly something that has been considered important to downstreams) * when we import a new upstream version, we adjust our divergence to apply on top of that new version * some of the divergence is vendor-specific and we will never upstream it (for example adjusting paths/defaults to meet Debian Policy) * some of the divergence is intended to go upstream, although our upstreams don't always apply in-principle-upstreamable changes as fast as we'd like them to; when it does get applied, it is often applied to their current development branch (like a git-cherry-pick) rather than being merged, and even if we send Github pull requests or similar, the upstream will want them to be based on some upstream branch and not on Debian's * in Debian's case, even if they want to apply all the patches we propose to their upstream source, our upstreams will never want to `git merge` the contents of our VCS, because they usually don't want to merge debian/ (and in fact we actively recommend that they don't) That's why, although I find dgit an interesting idea, I think dgit-maint-merge(7) is a trap. If I use dgit at all, it'll be dgit-maint-gbp(7) or similar. [Conflict-of-interest disclaimer: I am a happy user of gbp-pq for every patched package I maintain, except for tap.py and pkg-gnome svn. I would love to see gbp-pq adopted by DPMT so I can use it for tap.py, and when pkg-gnome finally moves from svn to git, given the overlap among active maintainers between pkg-{systemd,utopia,gnome}, I suspect they are going to use gbp-pq like pkg-systemd and pkg-utopia do. I also seriously considered maintaining tap.py outside DPMT as a way to avoid git-dpm.] Regards, S [1] but in most cases not Canonical :-)
Re: Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
I know the discussion is leaning towards replacing usage of git-dpm with gbp-pq. I have nothing against it but, since we are talking about solutions for a git-centric workflow, has anyone considered the dgit- maint-merge workflow [1]? It is very well documented and would simplify team-packaging policies a good deal. Assuming dgit-maint-merge were widely adopted, packaging policies would only need to cover team-specific details, such as infrastructure or communication channels for sponsorship, and then reference the dgit-maint-merge manpages for the packaging workflow. [1] https://www.mankier.com/7/dgit-maint-merge Best regards, Ghis
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Feb 05, 2017, at 02:07 AM, Scott Kitterman wrote: >Experimentation with a few packages to prepare for a migration and make sure >the documentation is good, is fine. We really ought to switch for real all >at once like we did for svn -> git. It's not much of a team repository >without a consistent approach for VCS use. +1 -Barry
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Feb 05, 2017, at 04:01 PM, Brian May wrote: >What should we call the clone page? > >PqGitPackaging??? GitPackagingPq ? Cheers, -Barry
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Sunday, February 05, 2017 03:59:37 PM Brian May wrote: > Scott Kitterman writes: > > We should probably be thinking in terms of post-release for this change. > > During the pre-release freeze, the release team doesn't typically allow > > changes that extraneous to fixing the specific issue they are letting a > > package into Testing to fix. The .git-dpm file is shipped in the package, > > so if we drop git-dpm, we're going to have to deal with getting .git-dpm > > removals through the release team for any package that needs update > > during the freeze. > > The .git-dpm file is only shipped with the Debian source package, and > AFAIK has no meaning outside git. So it is a useless file for the Debian > source package. There should be no impact whatsoever in removing it, and > you could even argue that it was a bug to distribute it in the Debian > source in the fist place. The opinion of the release team matters here more than yours or mine. Before doing anything pre-release, we ought to get a reading from them. > > That will also give us time to make sure we have a proper migration > > strategy and sufficient documentation. > > That may be a better reason. > > Hence the reason I suggested not doing a mass migration of all packages > at once (at least for now) but to update the package when it otherwise > needs updating. Experimentation with a few packages to prepare for a migration and make sure the documentation is good, is fine. We really ought to switch for real all at once like we did for svn -> git. It's not much of a team repository without a consistent approach for VCS use. Scott K
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
Barry Warsaw writes: > What I'd really like to see is a page like > https://wiki.debian.org/Python/GitPackaging for a non-git-dpm workflow. (The > page itself could probably use a bit of gardening anyway.) Specifically, I'd > like to see guidance on any tasks which are different for git-pq (or > non-git-dpm as the case may be). > > I'd suggest cloning that page instead of modify that page to cover both > cases. Edit the clone as if it were the opinionated view of just using gbp > tools and gbp-pq. The page should also have instructions on opportunistically > switching away from git-dpm. What should we call the clone page? PqGitPackaging??? -- Brian May
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
Scott Kitterman writes: > We should probably be thinking in terms of post-release for this change. > During the pre-release freeze, the release team doesn't typically allow > changes that extraneous to fixing the specific issue they are letting a > package into Testing to fix. The .git-dpm file is shipped in the package, so > if we drop git-dpm, we're going to have to deal with getting .git-dpm > removals > through the release team for any package that needs update during the > freeze. The .git-dpm file is only shipped with the Debian source package, and AFAIK has no meaning outside git. So it is a useless file for the Debian source package. There should be no impact whatsoever in removing it, and you could even argue that it was a bug to distribute it in the Debian source in the fist place. > That will also give us time to make sure we have a proper migration strategy > and sufficient documentation. That may be a better reason. Hence the reason I suggested not doing a mass migration of all packages at once (at least for now) but to update the package when it otherwise needs updating. -- Brian May
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Jan 31, 2017, at 02:51 PM, Scott Kitterman wrote: >We should probably be thinking in terms of post-release for this change. Oh, definitely +1 -Barry
Re: Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Tuesday, January 31, 2017 02:23:29 PM Barry Warsaw wrote: > On Jan 29, 2017, at 09:39 AM, Brian May wrote: > >I would think "gbp pq" is the most popular. > > I've used it on some of my non-team packages and while it takes a little > getting used to for the standard git-dpm workflow, it's been mostly fine. > > What I'd really like to see is a page like > https://wiki.debian.org/Python/GitPackaging for a non-git-dpm workflow. > (The page itself could probably use a bit of gardening anyway.) > Specifically, I'd like to see guidance on any tasks which are different for > git-pq (or non-git-dpm as the case may be). > > I'd suggest cloning that page instead of modify that page to cover both > cases. Edit the clone as if it were the opinionated view of just using gbp > tools and gbp-pq. The page should also have instructions on > opportunistically switching away from git-dpm. > > Then we can start to use those instructions in anger and add any other > recommendations for corner cases. Once we have enough experience with > gpb-pq throughout the team, we can consider making an official switch. We should probably be thinking in terms of post-release for this change. During the pre-release freeze, the release team doesn't typically allow changes that extraneous to fixing the specific issue they are letting a package into Testing to fix. The .git-dpm file is shipped in the package, so if we drop git-dpm, we're going to have to deal with getting .git-dpm removals through the release team for any package that needs update during the freeze. That will also give us time to make sure we have a proper migration strategy and sufficient documentation. Scott K
Moving off of git-dpm (Re: git-dpm breakage src:faker)
On Jan 29, 2017, at 09:39 AM, Brian May wrote: >I would think "gbp pq" is the most popular. I've used it on some of my non-team packages and while it takes a little getting used to for the standard git-dpm workflow, it's been mostly fine. What I'd really like to see is a page like https://wiki.debian.org/Python/GitPackaging for a non-git-dpm workflow. (The page itself could probably use a bit of gardening anyway.) Specifically, I'd like to see guidance on any tasks which are different for git-pq (or non-git-dpm as the case may be). I'd suggest cloning that page instead of modify that page to cover both cases. Edit the clone as if it were the opinionated view of just using gbp tools and gbp-pq. The page should also have instructions on opportunistically switching away from git-dpm. Then we can start to use those instructions in anger and add any other recommendations for corner cases. Once we have enough experience with gpb-pq throughout the team, we can consider making an official switch. Cheers, -Barry
Re: git-dpm breakage src:faker
Scott Kitterman writes: > OK. Where do I find documentation to give this a try? I used the following documentation: http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.patches.html -- Brian May
Re: git-dpm breakage src:faker
Raphael Hertzog writes: > You don't need that. dpkg-buildpackage unapplies them automatically after > the build if it has applied them. If they were already applied before the > build, then it leaves them applied. Guess it depends how you build the package. I have been using: gbp buildpackage "--git-builder=debuild -i -I -S" In order to get a *.dsc file that sbuild can use. This command appears to leave the patches applied. Guess I should investigate changing that line to somethine else... -- Brian May
Re: git-dpm breakage src:faker
On Sun, 29 Jan 2017 at 20:50:48 +0100, Raphael Hertzog wrote: > On Sun, 29 Jan 2017, Brian May wrote: > > 3. Update debian/source/options with "unapply-patches" (anything else?). > > You don't need that. dpkg-buildpackage unapplies them automatically after > the build if it has applied them. If they were already applied before the > build, then it leaves them applied. Not only do you not need that, it isn't even allowed: --no-unapply-patches, --unapply-patches [...] Thoseoptions are only allowed in debian/source/local-options so that all generated source packages have the same behavior by default. See flatpak, dbus, ioquake3, or basically anything else I maintain if you need typical examples of gbp-pq packages that sometimes or always have patches. (I would be delighted to switch src:tap.py to gbp-pq if you want an example of migration.) S
Re: git-dpm breakage src:faker
On January 29, 2017 2:17:16 AM EST, Arto Jantunen wrote: >Scott Kitterman writes: > >> On Sunday, January 29, 2017 09:39:10 AM Brian May wrote: >>> Scott Kitterman writes: >>> > On Sunday, January 29, 2017 08:54:57 AM Brian May wrote: >>> >> Can we switch away from git-dpm yet? Sure this is most likely >user >>> >> error, however I want to try to solve an RC bug, not fix broken >git-dpm >>> >> first. >>> > >>> > Much like the switch from svn to git, I think we need an agreed >new >>> > workflow and tools and a migration plan. >>> > >>> > What do you propose? >>> >>> I would think "gbp pq" is the most popular. >>> >>> I think something like: >>> >>> * Don't touch existing packages just for the sake of doing so. >>> * Next time you do need to update a package with a debian/.git-dpm: >>> 1. Delete debian/.git-dpm. >>> 2. Unapply all patches and commit (not sure what the easiest way >is) >>> 3. Update debian/source/options with "unapply-patches" (anything >else?). >>> * If you encounter a package without debian/.git-dpm, don't re-add >it. >>> * Don't push the gbp pq patches queue branch. >> >> I've never used it. >> >> Does that then result in one big undifferentiated mass of diff in the >source >> package? > >No, it results in separate patches with their headers intact in the >source package. I assume git-dpm (which I've never used) produces the >same end result. > >The git repository is of course different, with gbp pq carrying the >patches as patches in the packaging branch, and git-dpm having separate >magical patch branches. OK. Where do I find documentation to give this a try? Scott K
Re: git-dpm breakage src:faker
On Sun, 29 Jan 2017, Brian May wrote: > 3. Update debian/source/options with "unapply-patches" (anything else?). You don't need that. dpkg-buildpackage unapplies them automatically after the build if it has applied them. If they were already applied before the build, then it leaves them applied. Cheers, -- Raphaël Hertzog ◈ Debian Developer Support Debian LTS: https://www.freexian.com/services/debian-lts.html Learn to master Debian: https://debian-handbook.info/get/
Re: git-dpm breakage src:faker
Scott Kitterman writes: > On Sunday, January 29, 2017 09:39:10 AM Brian May wrote: >> Scott Kitterman writes: >> > On Sunday, January 29, 2017 08:54:57 AM Brian May wrote: >> >> Can we switch away from git-dpm yet? Sure this is most likely user >> >> error, however I want to try to solve an RC bug, not fix broken git-dpm >> >> first. >> > >> > Much like the switch from svn to git, I think we need an agreed new >> > workflow and tools and a migration plan. >> > >> > What do you propose? >> >> I would think "gbp pq" is the most popular. >> >> I think something like: >> >> * Don't touch existing packages just for the sake of doing so. >> * Next time you do need to update a package with a debian/.git-dpm: >> 1. Delete debian/.git-dpm. >> 2. Unapply all patches and commit (not sure what the easiest way is) >> 3. Update debian/source/options with "unapply-patches" (anything else?). >> * If you encounter a package without debian/.git-dpm, don't re-add it. >> * Don't push the gbp pq patches queue branch. > > I've never used it. > > Does that then result in one big undifferentiated mass of diff in the source > package? No, it results in separate patches with their headers intact in the source package. I assume git-dpm (which I've never used) produces the same end result. The git repository is of course different, with gbp pq carrying the patches as patches in the packaging branch, and git-dpm having separate magical patch branches. -- Arto Jantunen
Re: git-dpm breakage src:faker
On Sunday, January 29, 2017 09:39:10 AM Brian May wrote: > Scott Kitterman writes: > > On Sunday, January 29, 2017 08:54:57 AM Brian May wrote: > >> Can we switch away from git-dpm yet? Sure this is most likely user > >> error, however I want to try to solve an RC bug, not fix broken git-dpm > >> first. > > > > Much like the switch from svn to git, I think we need an agreed new > > workflow and tools and a migration plan. > > > > What do you propose? > > I would think "gbp pq" is the most popular. > > I think something like: > > * Don't touch existing packages just for the sake of doing so. > * Next time you do need to update a package with a debian/.git-dpm: > 1. Delete debian/.git-dpm. > 2. Unapply all patches and commit (not sure what the easiest way is) > 3. Update debian/source/options with "unapply-patches" (anything else?). > * If you encounter a package without debian/.git-dpm, don't re-add it. > * Don't push the gbp pq patches queue branch. I've never used it. Does that then result in one big undifferentiated mass of diff in the source package? Scott K
Re: git-dpm breakage src:faker
Scott Kitterman writes: > On Sunday, January 29, 2017 08:54:57 AM Brian May wrote: >> Can we switch away from git-dpm yet? Sure this is most likely user >> error, however I want to try to solve an RC bug, not fix broken git-dpm >> first. > > Much like the switch from svn to git, I think we need an agreed new workflow > and tools and a migration plan. > > What do you propose? I would think "gbp pq" is the most popular. I think something like: * Don't touch existing packages just for the sake of doing so. * Next time you do need to update a package with a debian/.git-dpm: 1. Delete debian/.git-dpm. 2. Unapply all patches and commit (not sure what the easiest way is) 3. Update debian/source/options with "unapply-patches" (anything else?). * If you encounter a package without debian/.git-dpm, don't re-add it. * Don't push the gbp pq patches queue branch. -- Brian May
Re: git-dpm breakage src:faker
On Sunday, January 29, 2017 08:54:57 AM Brian May wrote: > Can we switch away from git-dpm yet? Sure this is most likely user > error, however I want to try to solve an RC bug, not fix broken git-dpm > first. Much like the switch from svn to git, I think we need an agreed new workflow and tools and a migration plan. What do you propose? Scott K
Re: git-dpm breakage src:faker
Brian May writes: > Hello, > > Looks like somebody applied a change to this git-dpm repo that manually > deleted a patch under debian/patches. > > Instead of using "git-dpm checkout-patched" and then updating. > > This looks like it has broken git-dpm. > > How do I fix? > > Guessing I will have to revert the dodgy commit, and then delete the > patch correctly. Looks like the damage is more extensive then just one patch: [brian:~/tree/debian/python-modules/faker] master ± git-dpm status grep: warning: GREP_OPTIONS is deprecated; please use an alias or script grep: warning: GREP_OPTIONS is deprecated; please use an alias or script NOT UP TO DATE: 'upstream' is newer than listed in debian/.git-dpm Could not find '../fake-factory_0.5.7.orig.tar.gz! Without that file dpkg-source will not work! (Have you forgotten to run git-dpm prepare?) git-dpm: WARNING: guessing from your debian/changelog, your upstream file should be named 'faker_0.7.7.orig.tar.*' but it is named 'fake-factory_0.5.7.orig.tar.gz' grep: warning: GREP_OPTIONS is deprecated; please use an alias or script grep: warning: GREP_OPTIONS is deprecated; please use an alias or script grep: warning: GREP_OPTIONS is deprecated; please use an alias or script grep: warning: GREP_OPTIONS is deprecated; please use an alias or script grep: warning: GREP_OPTIONS is deprecated; please use an alias or script git-dpm: ERROR: Debian branch contains non-debian changes: .bumpversion.cfg .travis.yml CHANGELOG.rst CONTRIBUTING.rst Makefile README.rst docs/coding_style.rst docs/conf.py docs/index.rst faker/__init__.py faker/providers/__init__.py faker/providers/address/en_AU/__init__.py faker/providers/address/en_CA/__init__.py faker/providers/address/en_GB/__init__.py faker/providers/address/en_US/__init__.py faker/providers/address/es/__init__.py faker/providers/address/es_ES/__init__.py faker/providers/address/es_MX/__init__.py faker/providers/address/fa_IR/__init__.py faker/providers/address/fi_FI/__init__.py faker/providers/address/fr_CH/__init__.py faker/providers/address/fr_FR/__init__.py faker/providers/address/hi_IN/__init__.py faker/providers/address/it_IT/__init__.py faker/providers/address/ja_JP/__init__.py faker/providers/address/ko_KR/__init__.py faker/providers/address/ne_NP/__init__.py faker/providers/address/nl_BE/__init__.py faker/providers/address/no_NO/__init__.py faker/providers/address/pl_PL/__init__.py faker/providers/address/pt_BR/__init__.py faker/providers/address/ru_RU/__init__.py faker/providers/address/sl_SI/__init__.py faker/providers/address/uk_UA/__init__.py faker/providers/address/zh_CN/__init__.py faker/providers/address/zh_TW/__init__.py faker/providers/barcode/__init__.py faker/providers/color/__init__.py faker/providers/color/uk_UA/__init__.py faker/providers/company/fr_CH/__init__.py faker/providers/company/pt_BR/__init__.py faker/providers/company/ru_RU/__init__.py faker/providers/credit_card/__init__.py faker/providers/currency/__init__.py faker/providers/date_time/__init__.py faker/providers/file/__init__.py faker/providers/internet/__init__.py faker/providers/internet/fr_CH/__init__.py faker/providers/internet/fr_FR/__init__.py faker/providers/internet/ru_RU/__init__.py faker/providers/internet/uk_UA/__init__.py faker/providers/internet/zh_CN/__init__.py faker/providers/job/__init__.py faker/providers/job/fr_CH/__init__.py faker/providers/job/hr_HR/__init__.py faker/providers/job/zh_TW/__init__.py faker/providers/misc/__init__.py faker/providers/person/__init__.py faker/providers/person/de_AT/__init__.py faker/providers/person/de_DE/__init__.py faker/providers/person/dk_DK/__init__.py faker/providers/person/en/__init__.py faker/providers/person/en_GB/__init__.py faker/providers/person/en_US/__init__.py faker/providers/person/fa_IR/__init__.py faker/providers/person/fr_CH/__init__.py faker/providers/person/ko_KR/__init__.py faker/providers/person/pt_BR/__init__.py faker/providers/person/ru_RU/__init__.py faker/providers/person/uk_UA/__init__.py faker/providers/person/zh_CN/__init__.py faker/providers/phone_number/en_US/__init__.py faker/providers/phone_number/fa_IR/__init__.py faker/providers/phone_number/fr_CH/__init__.py faker/providers/phone_number/nl_BE/__init__.py faker/providers/phone_number/zh_TW/__init__.py faker/providers/profile/__init__.py faker/providers/python/__init__.py faker/providers/ssn/__init__.py faker/providers/ssn/en_CA/__init__.py faker/providers/ssn/fr_CH/__init__.py faker/providers/ssn/hr_HR/__init__.py faker/providers/ssn/ko_KR/__init__.py faker/providers/ssn/nl_BE/__init__.py faker/providers/ssn/ru_RU/__init__.py faker/providers/ssn/sv_SE/__init__.py faker/providers/ssn/uk_UA/__init__.py faker/tests/__init__.py faker/tests/hr_HR/__init__.py faker/tests/no_NO/__init__.py faker/tests/pt_BR/__init__.py faker/utils/datasets.py readthedocs.yml setup.py Can we switch away from git-dpm yet? Sure this is most li
git-dpm breakage src:faker
Hello, Looks like somebody applied a change to this git-dpm repo that manually deleted a patch under debian/patches. Instead of using "git-dpm checkout-patched" and then updating. This looks like it has broken git-dpm. How do I fix? Guessing I will have to revert the dodgy commit, and then delete the patch correctly. Regards -- Brian May