Re: Using git for patch management in Fedora
On 11/19/2013 10:55 AM, Mathieu Bridon wrote: On Tue, 2013-11-19 at 10:32 +, Richard W.M. Jones wrote: On Tue, Nov 19, 2013 at 06:28:47PM +0800, Mathieu Bridon wrote: Or maybe we could start using %autosetup ? http://www.rpm.org/wiki/PackagerDocs/Autosetup '%autosetup -S git' for sure, not plain %autosetup. Git correctly handles file modes and binary patches (not that binary patches should be much of a concern here, but file modes definitely are). BTW patch 2.7 does support file mode changes: http://savannah.gnu.org/forum/forum.php?forum_id=7361 Binary patches are not yet supported but they are quite useful and I've hit that with patches adding new png files for example. On the topic of new files, `%autosetup -S git` currently doesn't handle those: https://bugzilla.redhat.com/1059285 cheers, Pádraig. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On 19 Nov 2013, at 9:51 pm, Daniel P. Berrange berra...@redhat.com wrote:
On Tue, Nov 19, 2013 at 10:22:42AM +, Richard W.M. Jones wrote:
Several packages are using git for patch management. eg:
http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n46
http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/libguestfs.spec?h=f20#n22
http://pkgs.fedoraproject.org/cgit/qemu.git/tree/
http://pkgs.fedoraproject.org/cgit/ocaml.git/tree/ocaml.spec#n16
Some of these packages have invented home-brewed methods to generate
the Patch lines in the spec file, eg:
http://pkgs.fedoraproject.org/cgit/erlang.git/tree/otp-get-patches.sh
http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/copy-patches.sh?h=f20
More importantly, all are using random git repositories to store the
exploded tree. This makes it difficult for co-maintainers and proven
packagers to fit in with the patch management chosen by the
maintainer. Usually they won't have access to the git repository for
these patches, making it difficult to add patches and near impossible
to upgrade to a new version.
Yep, I've done the same thing for all the packages I maintain in Fedora
and RHEL too. It makes maintaining RPMs soo much easier, particularly
when you have lots of patches to manage. The script I use is
http://pkgs.fedoraproject.org/cgit/libvirt-sandbox.git/tree/update-patches.pl
For what its worth, here is the one I use for pacemaker in rhel:
https://gist.github.com/beekhof/8677396
It both helps maintain a git repo of patches and to reconcile that repo with
patches in the spec file (both built patches git doesn't know about yet and
patches in git that aren't being built yet).
As well as asking for bugzilla numbers when updating the spec file ;-)
I think that git is an excellent way to manage patches, but we ought
to think about formalizing this process. I think the goals should be:
(1) A git repository is used that co-maintainers and proven packagers
automatically have access to.
(2) A single method script is used to update the patches in the spec file.
Although there is already a git repository satisfying (1) above,
namely dist-git, it isn't suitable for storing the exploded tree since
commits to the spec file would conflict with commits (patches) to the
tree. So either a separate side repository which packages could
opt-in to, or perhaps a separate branch of the same git repo could be
used. I think using a branch would require no additional
infrastructure.
For (2) I would suggest a lightweight technique where git-managed
patches are marked in the spec file using:
### GIT-MANAGED-PATCHES ###
### END-GIT-MANGED-PATCHES ###
and a simple script that replaces everything between those marks with
Patch lines. The script could be adapted from copy-patches.sh
(see above).
To apply the patches, a standard RPM macro could be created:
%prep
%setup -q
%{git_apply_patches}
which would expand to something like:
git init
git config user.email %{name}-ow...@fedoraproject.org
git config user.name %{name}
git add .
git commit -a -q -m %{version} baseline
git am %{patches}
Thoughts on this?
It would certainly be nice to formalize things in this area given that
so many people are re-inventing the wheel here.
I agree that I wouldn't want to have the upstream GIT repo merged into
the Fedora dist-git repos, since that would increase their size by
many orders of magnitude.
It seems like we'd want to have a Fedora hosted parallel GIT repos for
tracking upstream GIT repos that are relevant to the Fedora RPMs.
Preferrably something that could be automated so it auto-synchronizes
the upstream branches, so Fedora maintainers just need to create a new
branch and then cherry-pick -x the patches they need.
Daniel
--
|: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
signature.asc
Description: Message signed with OpenPGP using GPGMail
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Fri, Nov 22, 2013 at 04:25:00PM +0100, Kevin Kofler wrote: Richard W.M. Jones wrote: Several packages are using git for patch management. eg: http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n46 http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/libguestfs.spec?h=f20#n22 http://pkgs.fedoraproject.org/cgit/qemu.git/tree/ http://pkgs.fedoraproject.org/cgit/ocaml.git/tree/ocaml.spec#n16 Ewww, we need packaging guidelines banning this bizarre practice. I can see using git-am if you're backporting upstream patches from upstream git (though patch and thus %patchN is usually good enough for that, too), but for Fedora-specific patches, it's really the wrong tool for the job. Some of these packages have invented home-brewed methods to generate the Patch lines in the spec file, eg: http://pkgs.fedoraproject.org/cgit/erlang.git/tree/otp-get-patches.sh http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/copy-patches.sh?h=f20 Ewww! Yuck! Could you explain why you don't like this? If you had actually used it, I'm sure you would see it is far more sensible than manually managing and rebasing patches. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
Richard W.M. Jones wrote: This recipe has worked for me for years. The patches are probably implicitly ordered by their names (ie. 0001-...patch etc) I think they're actually ordered by their patch number in the specfile. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
Richard W.M. Jones wrote: Several packages are using git for patch management. eg: http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n46 http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/libguestfs.spec?h=f20#n22 http://pkgs.fedoraproject.org/cgit/qemu.git/tree/ http://pkgs.fedoraproject.org/cgit/ocaml.git/tree/ocaml.spec#n16 Ewww, we need packaging guidelines banning this bizarre practice. I can see using git-am if you're backporting upstream patches from upstream git (though patch and thus %patchN is usually good enough for that, too), but for Fedora-specific patches, it's really the wrong tool for the job. Some of these packages have invented home-brewed methods to generate the Patch lines in the spec file, eg: http://pkgs.fedoraproject.org/cgit/erlang.git/tree/otp-get-patches.sh http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/copy-patches.sh?h=f20 Ewww! Yuck! Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On 11/19/2013 11:22 AM, Richard W.M. Jones wrote: Some of these packages have invented home-brewed methods to generate the Patch lines in the spec file, eg: FYI: Several projects use Tito [1], which allows you to operate on (unpacked) git tree (usually clonned). Do changes directly in git. And if you call simply `tito build --rpm` with builder configured as `tito.distributionbuilder.DistributionBuilder`. It will create pristine tar ball from last upstream tag (and that tar have same timestamps as upstream and therefore md5 will match). And automatically modify spec (in temporary directory) with: Patch0: foo-1.2-1-to-foo-1.2-2.patch Patch1: foo-1.2-1-to-foo-1.2-3.patch Where each patch is concated commits between tagged releases. You can find more information in: yum install tito man tito.props man tito And BTW Tito have interface to Fedora dist-git (man releasers.conf). [1] https://github.com/dgoodwin/tito -- Miroslav Suchy, RHCE, RHCDS Red Hat, Software Engineer, #brno, #devexp, #fedora-buildsys -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Thu, Nov 21, 2013 at 09:29:35AM +0100, Miroslav Suchý wrote: And if you call simply `tito build --rpm` with builder configured as `tito.distributionbuilder.DistributionBuilder`. It will create pristine tar ball from last upstream tag (and that tar have same timestamps as upstream and therefore md5 will match). It's pretty common that people don't maintain autotools generated stuff in git. It means that upstream release (tarball) is not only source code from git but it also contains unique files generated on maintainer's machine. It means that for spec file Source: we still need the original upstream tarball. And automatically modify spec (in temporary directory) with: Patch0: foo-1.2-1-to-foo-1.2-2.patch Patch1: foo-1.2-1-to-foo-1.2-3.patch Where each patch is concated commits between tagged releases. You can find more information in: yum install tito man tito.props man tito And BTW Tito have interface to Fedora dist-git (man releasers.conf). [1] https://github.com/dgoodwin/tito Nice, thanks. Karel -- Karel Zak k...@redhat.com http://karelzak.blogspot.com -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Thu, Nov 21, 2013 at 11:41:34AM +0100, Karel Zak wrote: On Thu, Nov 21, 2013 at 09:29:35AM +0100, Miroslav Suchý wrote: And if you call simply `tito build --rpm` with builder configured as `tito.distributionbuilder.DistributionBuilder`. It will create pristine tar ball from last upstream tag (and that tar have same timestamps as upstream and therefore md5 will match). It's pretty common that people don't maintain autotools generated stuff in git. It means that upstream release (tarball) is not only source code from git but it also contains unique files generated on maintainer's machine. It means that for spec file Source: we still need the original upstream tarball. Correct. Some other random points slightly related to this: - Patches wouldn't normally patch the generated configure script, so the git repo wouldn't need to store the generated code in order to be useful as a store of patches. I have used this git system to manage patches which are then applied on top of the pristine upstream tarball and it has always worked fine. - If you did patch the configure script you could add the generated files to git. Or use the exploded tree git to store the tarball. - In any case I don't want to force anyone to use this system. It's just a way to consolidate the duplicated work that at least 5 teams are currently using to manage patches. You can quite happily keep using your own way of managing patches if you don't like it. - configure.ac that depends on specific old versions of autoconf is, as a rule of thumb, broken. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 02:20:30PM +, Daniel P. Berrange wrote: On Tue, Nov 19, 2013 at 01:29:06PM +, Richard W.M. Jones wrote: On Tue, Nov 19, 2013 at 01:39:50PM +0100, Karel Zak wrote: We have to learn fedpkg to do all the magic ;-) Something like add remote git tree with exploded tree: fedpkg exploded-tree add ssh://git.fedorahosted.org/git/foo.git This is all great, but the problem is that co-maintainers and provenpackagers need to be (automatically if possible) added to the fedorahosted tree. Otherwise there's a big extra step for them if they want to follow the package owner's preferred patching system. BTW, why we cannot use pkgs.fedoraproject.org for exploded tree too? Just another set of repositories, maybe with a different HTTP alias, e.g. extree.fedoraproject.org, or so... Ideally the GIT SCM request added to bugzilla when reviews are approved would have a Upstream GIT URL option and would setup a clone of this, and create branches for the fedora releases, and apply the same permissioning model from dist-git branches of the same names. All such information belong to spec file :-) Fedora-SCM: Upstream-SCM: Exploded-tree-SCM: for example upstream SCM URL is already missing in our spec files, it's more important information than URL to tarball. Karel -- Karel Zak k...@redhat.com http://karelzak.blogspot.com -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
2013/11/20 Karel Zak k...@redhat.com: All such information belong to spec file :-) Fedora-SCM: Upstream-SCM: Exploded-tree-SCM: for example upstream SCM URL is already missing in our spec files, it's more important information than URL to tarball. Love this idea but there is a problem - RPM (at least older versions) doesn't like unknown fields. It throws error and dies. -- With best regards, Peter Lemenkov. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 03:36:54PM -0800, Toshio Kuratomi wrote: One note though, I think that in the past one of the discussion points we've foundered on is whether we want to be mirroring upstream's git repo or (approximately) upstream's releases. I think that's probably where we'd need to start discussion. From the point of view of patch management, I have a strong view here: We should be mirroring upstream's git, if they have one. The reason is that it makes it trivial to cherry pick upstream patches on top of the Fedora branch. So I'd arrange it by having a straight mirror of upstream, then have our own 'fNN-branch' branches which contain the upstream releases (ideally from upstream tags if they are using git sensibly) + our cherry picked patches. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 03:36:54PM -0800, Toshio Kuratomi wrote: On Tue, Nov 19, 2013 at 05:32:01PM +, Daniel P. Berrange wrote: On Tue, Nov 19, 2013 at 07:27:20PM +0200, Alek Paunov wrote: On 19.11.2013 16:20, Daniel P. Berrange wrote: On Tue, Nov 19, 2013 at 01:29:06PM +, Richard W.M. Jones wrote: On Tue, Nov 19, 2013 at 01:39:50PM +0100, Karel Zak wrote: We have to learn fedpkg to do all the magic ;-) Something like add remote git tree with exploded tree: fedpkg exploded-tree add ssh://git.fedorahosted.org/git/foo.git This is all great, but the problem is that co-maintainers and provenpackagers need to be (automatically if possible) added to the fedorahosted tree. Otherwise there's a big extra step for them if they want to follow the package owner's preferred patching system. Ideally the GIT SCM request added to bugzilla when reviews are approved would have a Upstream GIT URL option and would setup a clone of this, and create branches for the fedora releases, and apply the same permissioning model from dist-git branches of the same names. What about intermediate step: optional fNN-upstream branch in addition to fNN, containing relevant upstream revision as git submodule (preferably referencing fedorahosted mirror, but initially also allowing external clones)? I think it would be better to keep the upstream repo separate for sake of size. People who just want to checkout the latest RPM branches don't want to have to pull down 100's of MB, potentially GB, worth of GIT repo, when current Fedora GIT repos are a mere few MB. Only maintainers actively updating patches need that burden. It probably makes sense to have a lookaside git that's similar to the lookaside cache. One note though, I think that in the past one of the discussion points we've foundered on is whether we want to be mirroring upstream's git repo or (approximately) upstream's releases. I think that's probably where we'd need to start discussion. For it to be useful to me we'd need to be mirroring the upstream git repo continuously, not just at releases. I will frequently cherry-pick patches directly from GIT master into the Fedora maint branches that I maintain. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Wed, Nov 20, 2013 at 09:58:00AM +0100, Karel Zak wrote: On Tue, Nov 19, 2013 at 02:20:30PM +, Daniel P. Berrange wrote: On Tue, Nov 19, 2013 at 01:29:06PM +, Richard W.M. Jones wrote: On Tue, Nov 19, 2013 at 01:39:50PM +0100, Karel Zak wrote: We have to learn fedpkg to do all the magic ;-) Something like add remote git tree with exploded tree: fedpkg exploded-tree add ssh://git.fedorahosted.org/git/foo.git This is all great, but the problem is that co-maintainers and provenpackagers need to be (automatically if possible) added to the fedorahosted tree. Otherwise there's a big extra step for them if they want to follow the package owner's preferred patching system. BTW, why we cannot use pkgs.fedoraproject.org for exploded tree too? Just another set of repositories, maybe with a different HTTP alias, e.g. extree.fedoraproject.org, or so... That's entirely possibile - the key points to me are just that the repos are separate, and access control is applied uniformly across both. Beyond that I don't care where they are actually hosted. Ideally the GIT SCM request added to bugzilla when reviews are approved would have a Upstream GIT URL option and would setup a clone of this, and create branches for the fedora releases, and apply the same permissioning model from dist-git branches of the same names. All such information belong to spec file :-) Fedora-SCM: Upstream-SCM: Exploded-tree-SCM: for example upstream SCM URL is already missing in our spec files, it's more important information than URL to tarball. I suggested the GIT SCM request, because that's where the tools already look for this kind of info, so it'd be a simpler addition to the GIT creation process. Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
Hi, For github-hosted upstreams, you can suffix some URLs with '.diff' or '.patch'. I have used it for vcsh [1] (which btw is a tool I'd recommend to anyone who keeps dotfiles in sync on several machines). I did that on a pull request, but you can also do that on a commit (for cherry-picking) [2] and github even allows you to use a shortened sha-1 [3]. Of course this doesn't prevent you from putting the patch files in the package's repository, which seems to be the main issue. But one could write a script to fetch such patches at %prep time. Dridi [1] http://pkgs.fedoraproject.org/cgit/vcsh.git/tree/vcsh.spec [2] https://github.com/RichiH/vcsh/commit/c0d0f80513f26e7b6a485bbd6239191a3fe4be59.diff [3] https://github.com/RichiH/vcsh/commit/c0d0f80.diff On Wed, Nov 20, 2013 at 10:19 AM, Richard W.M. Jones rjo...@redhat.com wrote: On Tue, Nov 19, 2013 at 03:36:54PM -0800, Toshio Kuratomi wrote: One note though, I think that in the past one of the discussion points we've foundered on is whether we want to be mirroring upstream's git repo or (approximately) upstream's releases. I think that's probably where we'd need to start discussion. From the point of view of patch management, I have a strong view here: We should be mirroring upstream's git, if they have one. The reason is that it makes it trivial to cherry pick upstream patches on top of the Fedora branch. So I'd arrange it by having a straight mirror of upstream, then have our own 'fNN-branch' branches which contain the upstream releases (ideally from upstream tags if they are using git sensibly) + our cherry picked patches. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, 2013-11-19 at 10:22 +, Richard W.M. Jones wrote:
which would expand to something like:
git init
git config user.email %{name}-ow...@fedoraproject.org
git config user.name %{name}
git add .
git commit -a -q -m %{version} baseline
git am %{patches}
Trivial point of correctness:
git am %{patches} /dev/null
Otherwise, in the delightful case of having no patches to apply,
non-mock-contained builds will never complete because git-am will be
waiting for a patch on stdin.
- ajax
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On 20.11.2013 11:19, Richard W.M. Jones wrote: On Tue, Nov 19, 2013 at 03:36:54PM -0800, Toshio Kuratomi wrote: One note though, I think that in the past one of the discussion points we've foundered on is whether we want to be mirroring upstream's git repo or (approximately) upstream's releases. I think that's probably where we'd need to start discussion. From the point of view of patch management, I have a strong view here: We should be mirroring upstream's git, if they have one. The reason is that it makes it trivial to cherry pick upstream patches on top of the Fedora branch. So I'd arrange it by having a straight mirror of upstream, then have our own 'fNN-branch' branches which contain the upstream releases (ideally from upstream tags if they are using git sensibly) + our cherry picked patches. What you (and Daniel yesterday, and Karel, and ... likely everybody) clarifying is the natural scheme: fsource: XYZ: XYZ[ref1]: fNN-overlay XYZ[ref2]: fNM-overlay fsource[XYZ][fNN].acls - dist-git[XYZ][fNN].acls dist-git[XYZ][fNN].patches - XYZ[fNN-overlay] - XYZ[ref1] dist-git[XYZ][fNN].source - XYZ[ref1].tarball Optionally: dist-git[XYZ][fNN].changelog - ~.filter(~.patches) I have been thinking for intermediate steps with less resources allocation, but obviously there are consensus across the maintainers according the workable approach. Kind Regards, Alek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Using git for patch management in Fedora
Several packages are using git for patch management. eg:
http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n46
http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/libguestfs.spec?h=f20#n22
http://pkgs.fedoraproject.org/cgit/qemu.git/tree/
http://pkgs.fedoraproject.org/cgit/ocaml.git/tree/ocaml.spec#n16
Some of these packages have invented home-brewed methods to generate
the Patch lines in the spec file, eg:
http://pkgs.fedoraproject.org/cgit/erlang.git/tree/otp-get-patches.sh
http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/copy-patches.sh?h=f20
More importantly, all are using random git repositories to store the
exploded tree. This makes it difficult for co-maintainers and proven
packagers to fit in with the patch management chosen by the
maintainer. Usually they won't have access to the git repository for
these patches, making it difficult to add patches and near impossible
to upgrade to a new version.
I think that git is an excellent way to manage patches, but we ought
to think about formalizing this process. I think the goals should be:
(1) A git repository is used that co-maintainers and proven packagers
automatically have access to.
(2) A single method script is used to update the patches in the spec file.
Although there is already a git repository satisfying (1) above,
namely dist-git, it isn't suitable for storing the exploded tree since
commits to the spec file would conflict with commits (patches) to the
tree. So either a separate side repository which packages could
opt-in to, or perhaps a separate branch of the same git repo could be
used. I think using a branch would require no additional
infrastructure.
For (2) I would suggest a lightweight technique where git-managed
patches are marked in the spec file using:
### GIT-MANAGED-PATCHES ###
### END-GIT-MANGED-PATCHES ###
and a simple script that replaces everything between those marks with
Patch lines. The script could be adapted from copy-patches.sh
(see above).
To apply the patches, a standard RPM macro could be created:
%prep
%setup -q
%{git_apply_patches}
which would expand to something like:
git init
git config user.email %{name}-ow...@fedoraproject.org
git config user.name %{name}
git add .
git commit -a -q -m %{version} baseline
git am %{patches}
Thoughts on this?
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, 2013-11-19 at 10:22 +, Richard W.M. Jones wrote:
For (2) I would suggest a lightweight technique where git-managed
patches are marked in the spec file using:
### GIT-MANAGED-PATCHES ###
### END-GIT-MANGED-PATCHES ###
and a simple script that replaces everything between those marks with
Patch lines. The script could be adapted from copy-patches.sh
(see above).
To apply the patches, a standard RPM macro could be created:
%prep
%setup -q
%{git_apply_patches}
which would expand to something like:
git init
git config user.email %{name}-ow...@fedoraproject.org
git config user.name %{name}
git add .
git commit -a -q -m %{version} baseline
git am %{patches}
Or maybe we could start using %autosetup ?
http://www.rpm.org/wiki/PackagerDocs/Autosetup
--
Mathieu
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
2013/11/19 Richard W.M. Jones rjo...@redhat.com: Several packages are using git for patch management. eg: http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n46 http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/libguestfs.spec?h=f20#n22 http://pkgs.fedoraproject.org/cgit/qemu.git/tree/ http://pkgs.fedoraproject.org/cgit/ocaml.git/tree/ocaml.spec#n16 Some of these packages have invented home-brewed methods to generate the Patch lines in the spec file, eg: I hope we'll see some progress in RPM in regards to VCS integrations soon. Because that's the main issue with RPM and related infrastructure nowadays. More importantly, all are using random git repositories to store the exploded tree. This makes it difficult for co-maintainers and proven packagers to fit in with the patch management chosen by the maintainer. Usually they won't have access to the git repository for these patches, making it difficult to add patches and near impossible to upgrade to a new version. I'm using https://git.fedorahosted.org/git/ for that. For example erlang is stored here: https://git.fedorahosted.org/git/erlang.git It contains a mirror of main upstream repo and few branches with Fedora-related patches. I think we should expand this practise (mirroring of a git-reposiories of upstream projects with a Fedora-specific patches) further and add more git mirrors here, at Fedorahosted. I personally love to see fedora kernel as a Git-managed tree instead of few dozens of random patches and a spec-file. -- With best regards, Peter Lemenkov. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 06:28:47PM +0800, Mathieu Bridon wrote:
On Tue, 2013-11-19 at 10:22 +, Richard W.M. Jones wrote:
For (2) I would suggest a lightweight technique where git-managed
patches are marked in the spec file using:
### GIT-MANAGED-PATCHES ###
### END-GIT-MANGED-PATCHES ###
and a simple script that replaces everything between those marks with
Patch lines. The script could be adapted from copy-patches.sh
(see above).
To apply the patches, a standard RPM macro could be created:
%prep
%setup -q
%{git_apply_patches}
which would expand to something like:
git init
git config user.email %{name}-ow...@fedoraproject.org
git config user.name %{name}
git add .
git commit -a -q -m %{version} baseline
git am %{patches}
Or maybe we could start using %autosetup ?
http://www.rpm.org/wiki/PackagerDocs/Autosetup
'%autosetup -S git' for sure, not plain %autosetup.
Git correctly handles file modes and binary patches (not that binary
patches should be much of a concern here, but file modes definitely
are).
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 02:32:58PM +0400, Peter Lemenkov wrote: 2013/11/19 Richard W.M. Jones rjo...@redhat.com: Several packages are using git for patch management. eg: http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n46 http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/libguestfs.spec?h=f20#n22 http://pkgs.fedoraproject.org/cgit/qemu.git/tree/ http://pkgs.fedoraproject.org/cgit/ocaml.git/tree/ocaml.spec#n16 Some of these packages have invented home-brewed methods to generate the Patch lines in the spec file, eg: I hope we'll see some progress in RPM in regards to VCS integrations soon. Because that's the main issue with RPM and related infrastructure nowadays. More importantly, all are using random git repositories to store the exploded tree. This makes it difficult for co-maintainers and proven packagers to fit in with the patch management chosen by the maintainer. Usually they won't have access to the git repository for these patches, making it difficult to add patches and near impossible to upgrade to a new version. I'm using https://git.fedorahosted.org/git/ for that. For example erlang is stored here: https://git.fedorahosted.org/git/erlang.git It contains a mirror of main upstream repo and few branches with Fedora-related patches. Peter, is the comment in the spec file wrong? It refers to two github repos: http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n46 In any case, fedorahosted would be an improvement, but AIUI it doesn't automatically give access to co-maintainers and proven packagers. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
2013/11/19 Richard W.M. Jones rjo...@redhat.com: Peter, is the comment in the spec file wrong? It refers to two github repos: http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n46 Oops, thanks for pointing me on this. Yes, that's just leftover and should be removed. And the entire patch generation process should be greatly simplified. Patches are welcome btw :) In any case, fedorahosted would be an improvement, but AIUI it doesn't automatically give access to co-maintainers and proven packagers. Yes indeed. Our infrastructure lacks proper integration. What should I do to add you into giterlang group? -- With best regards, Peter Lemenkov. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
2013/11/19 Peter Lemenkov lemen...@gmail.com: 2013/11/19 Richard W.M. Jones rjo...@redhat.com: Peter, is the comment in the spec file wrong? It refers to two github repos: http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n46 Oops, thanks for pointing me on this. Yes, that's just leftover and should be removed. And the entire patch generation process should be greatly simplified. Patches are welcome btw :) In any case, fedorahosted would be an improvement, but AIUI it doesn't automatically give access to co-maintainers and proven packagers. Yes indeed. Our infrastructure lacks proper integration. What should I do to add you into giterlang group? Ok, just figured it out by myself. -- With best regards, Peter Lemenkov. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
Richard W.M. Jones rjo...@redhat.com writes:
Several packages are using git for patch management. eg:
http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n46
http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/libguestfs.spec?h=f20#n22
http://pkgs.fedoraproject.org/cgit/qemu.git/tree/
http://pkgs.fedoraproject.org/cgit/ocaml.git/tree/ocaml.spec#n16
Some of these packages have invented home-brewed methods to generate
the Patch lines in the spec file, eg:
http://pkgs.fedoraproject.org/cgit/erlang.git/tree/otp-get-patches.sh
http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/copy-patches.sh?h=f20
More importantly, all are using random git repositories to store the
exploded tree. This makes it difficult for co-maintainers and proven
packagers to fit in with the patch management chosen by the
maintainer. Usually they won't have access to the git repository for
these patches, making it difficult to add patches and near impossible
to upgrade to a new version.
I think that git is an excellent way to manage patches, but we ought
to think about formalizing this process. I think the goals should be:
(1) A git repository is used that co-maintainers and proven packagers
automatically have access to.
(2) A single method script is used to update the patches in the spec file.
Although there is already a git repository satisfying (1) above,
namely dist-git, it isn't suitable for storing the exploded tree since
commits to the spec file would conflict with commits (patches) to the
tree. So either a separate side repository which packages could
opt-in to, or perhaps a separate branch of the same git repo could be
used. I think using a branch would require no additional
infrastructure.
For (2) I would suggest a lightweight technique where git-managed
patches are marked in the spec file using:
### GIT-MANAGED-PATCHES ###
### END-GIT-MANGED-PATCHES ###
and a simple script that replaces everything between those marks with
Patch lines. The script could be adapted from copy-patches.sh
(see above).
To apply the patches, a standard RPM macro could be created:
%prep
%setup -q
%{git_apply_patches}
which would expand to something like:
git init
git config user.email %{name}-ow...@fedoraproject.org
git config user.name %{name}
git add .
git commit -a -q -m %{version} baseline
git am %{patches}
Thoughts on this?
ordering, you cannot run $ git am without ordering. Nevertheless, I
would rather use quilt for managing patches in dist-git.
--
Nikola
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 10:22:42AM +, Richard W.M. Jones wrote:
Several packages are using git for patch management. eg:
http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n46
http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/libguestfs.spec?h=f20#n22
http://pkgs.fedoraproject.org/cgit/qemu.git/tree/
http://pkgs.fedoraproject.org/cgit/ocaml.git/tree/ocaml.spec#n16
Some of these packages have invented home-brewed methods to generate
the Patch lines in the spec file, eg:
http://pkgs.fedoraproject.org/cgit/erlang.git/tree/otp-get-patches.sh
http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/copy-patches.sh?h=f20
More importantly, all are using random git repositories to store the
exploded tree. This makes it difficult for co-maintainers and proven
packagers to fit in with the patch management chosen by the
maintainer. Usually they won't have access to the git repository for
these patches, making it difficult to add patches and near impossible
to upgrade to a new version.
Yep, I've done the same thing for all the packages I maintain in Fedora
and RHEL too. It makes maintaining RPMs soo much easier, particularly
when you have lots of patches to manage. The script I use is
http://pkgs.fedoraproject.org/cgit/libvirt-sandbox.git/tree/update-patches.pl
I think that git is an excellent way to manage patches, but we ought
to think about formalizing this process. I think the goals should be:
(1) A git repository is used that co-maintainers and proven packagers
automatically have access to.
(2) A single method script is used to update the patches in the spec file.
Although there is already a git repository satisfying (1) above,
namely dist-git, it isn't suitable for storing the exploded tree since
commits to the spec file would conflict with commits (patches) to the
tree. So either a separate side repository which packages could
opt-in to, or perhaps a separate branch of the same git repo could be
used. I think using a branch would require no additional
infrastructure.
For (2) I would suggest a lightweight technique where git-managed
patches are marked in the spec file using:
### GIT-MANAGED-PATCHES ###
### END-GIT-MANGED-PATCHES ###
and a simple script that replaces everything between those marks with
Patch lines. The script could be adapted from copy-patches.sh
(see above).
To apply the patches, a standard RPM macro could be created:
%prep
%setup -q
%{git_apply_patches}
which would expand to something like:
git init
git config user.email %{name}-ow...@fedoraproject.org
git config user.name %{name}
git add .
git commit -a -q -m %{version} baseline
git am %{patches}
Thoughts on this?
It would certainly be nice to formalize things in this area given that
so many people are re-inventing the wheel here.
I agree that I wouldn't want to have the upstream GIT repo merged into
the Fedora dist-git repos, since that would increase their size by
many orders of magnitude.
It seems like we'd want to have a Fedora hosted parallel GIT repos for
tracking upstream GIT repos that are relevant to the Fedora RPMs.
Preferrably something that could be automated so it auto-synchronizes
the upstream branches, so Fedora maintainers just need to create a new
branch and then cherry-pick -x the patches they need.
Daniel
--
|: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, 2013-11-19 at 10:32 +, Richard W.M. Jones wrote: On Tue, Nov 19, 2013 at 06:28:47PM +0800, Mathieu Bridon wrote: Or maybe we could start using %autosetup ? http://www.rpm.org/wiki/PackagerDocs/Autosetup '%autosetup -S git' for sure, not plain %autosetup. Git correctly handles file modes and binary patches (not that binary patches should be much of a concern here, but file modes definitely are). Fair enough. I wasn't really suggesting any option (or the absence of it), but point taken. To be honest, I'd go even further: just like we got rid of %clean sections, maybe we could get rid of %prep sections in most cases? rpmbuild could just assume that if no %prep section is present, then it should just use %autosetup, with a certain set of options agreed upon. -- Mathieu -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 02:42:00PM +0400, Peter Lemenkov wrote: 2013/11/19 Peter Lemenkov lemen...@gmail.com: 2013/11/19 Richard W.M. Jones rjo...@redhat.com: Peter, is the comment in the spec file wrong? It refers to two github repos: http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n46 Oops, thanks for pointing me on this. Yes, that's just leftover and should be removed. And the entire patch generation process should be greatly simplified. Patches are welcome btw :) In any case, fedorahosted would be an improvement, but AIUI it doesn't automatically give access to co-maintainers and proven packagers. Yes indeed. Our infrastructure lacks proper integration. What should I do to add you into giterlang group? Ok, just figured it out by myself. Thanks. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 11:46:46AM +0100, Nikola Pajkovsky wrote:
Richard W.M. Jones rjo...@redhat.com writes:
git init
git config user.email %{name}-ow...@fedoraproject.org
git config user.name %{name}
git add .
git commit -a -q -m %{version} baseline
git am %{patches}
Thoughts on this?
ordering, you cannot run $ git am without ordering.
This recipe has worked for me for years. The patches are probably
implicitly ordered by their names (ie. 0001-...patch etc)
Nevertheless, I
would rather use quilt for managing patches in dist-git.
This doesn't stop you from using another method to apply the patches.
It doesn't even stop you from using your own home-brew method,
although I think standardizing would help for the majority of packages
since it lets co-maintainers and PPs work on them more easily.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 10:51:01AM +, Daniel P. Berrange wrote: Yep, I've done the same thing for all the packages I maintain in Fedora and RHEL too. It makes maintaining RPMs soo much easier, particularly when you have lots of patches to manage. The script I use is http://pkgs.fedoraproject.org/cgit/libvirt-sandbox.git/tree/update-patches.pl Cool. It would be nice to improve the script to read References: or Addresses: lines from the git commit messages and use the lines as comments for Patch%d: in the SPEC file. Something like: # https://bugzilla.redhat.com/show_bug.cgi?id=850355 Patch123: 0001-foo-bar.patch Maybe it would be also possible to use the URL to ask bugzilla for bug Id and Subject and then generate SPEC %changelog :-) Karel -- Karel Zak k...@redhat.com http://karelzak.blogspot.com -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 12:51:29PM +0100, Karel Zak wrote: On Tue, Nov 19, 2013 at 10:51:01AM +, Daniel P. Berrange wrote: Yep, I've done the same thing for all the packages I maintain in Fedora and RHEL too. It makes maintaining RPMs soo much easier, particularly when you have lots of patches to manage. The script I use is http://pkgs.fedoraproject.org/cgit/libvirt-sandbox.git/tree/update-patches.pl Cool. It would be nice to improve the script to read References: or Addresses: lines from the git commit messages and use the lines as comments for Patch%d: in the SPEC file. Something like: # https://bugzilla.redhat.com/show_bug.cgi?id=850355 Patch123: 0001-foo-bar.patch Maybe it would be also possible to use the URL to ask bugzilla for bug Id and Subject and then generate SPEC %changelog :-) The erlang patches script also reads various information from the commit comments too. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 12:14:42PM +, Richard W.M. Jones wrote: On Tue, Nov 19, 2013 at 12:51:29PM +0100, Karel Zak wrote: On Tue, Nov 19, 2013 at 10:51:01AM +, Daniel P. Berrange wrote: Yep, I've done the same thing for all the packages I maintain in Fedora and RHEL too. It makes maintaining RPMs soo much easier, particularly when you have lots of patches to manage. The script I use is http://pkgs.fedoraproject.org/cgit/libvirt-sandbox.git/tree/update-patches.pl Cool. It would be nice to improve the script to read References: or Addresses: lines from the git commit messages and use the lines as comments for Patch%d: in the SPEC file. Something like: # https://bugzilla.redhat.com/show_bug.cgi?id=850355 Patch123: 0001-foo-bar.patch Maybe it would be also possible to use the URL to ask bugzilla for bug Id and Subject and then generate SPEC %changelog :-) The erlang patches script also reads various information from the commit comments too. We have to learn fedpkg to do all the magic ;-) Something like add remote git tree with exploded tree: fedpkg exploded-tree add ssh://git.fedorahosted.org/git/foo.git list log for exploded/upstream..exploded/f20 (or only --new commits which not found in .spec): fedpkg exploded-tree log [--new] generate PatchN: .spec file lines, call git format-patch and git-add from exploded/upstream..exploded/f20 stuff: fedpkg exploded-tree update-patches generate %changelog according to References: from commit messages: fedpkg exploded-tree update-changelog ... dreaming is so simple, volunteers? :-) Karel -- Karel Zak k...@redhat.com http://karelzak.blogspot.com -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 01:39:50PM +0100, Karel Zak wrote: We have to learn fedpkg to do all the magic ;-) Something like add remote git tree with exploded tree: fedpkg exploded-tree add ssh://git.fedorahosted.org/git/foo.git This is all great, but the problem is that co-maintainers and provenpackagers need to be (automatically if possible) added to the fedorahosted tree. Otherwise there's a big extra step for them if they want to follow the package owner's preferred patching system. I agree in general that this could be something added to / aided by fedpkg. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
This is definitely worth formalizing.
On 11/19/2013 10:22 AM, Richard W.M. Jones wrote:
Several packages are using git for patch management. eg:
http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n46
http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/libguestfs.spec?h=f20#n22
http://pkgs.fedoraproject.org/cgit/qemu.git/tree/
http://pkgs.fedoraproject.org/cgit/ocaml.git/tree/ocaml.spec#n16
All the OpenStack packages also use this technique too:
Specifically you use a comment of the form to mark the auto generated patches:
# patches_base=$tag
For example see:
http://pkgs.fedoraproject.org/cgit/openstack-nova.git/tree/openstack-nova.spec#n37
One can also add an optional +number to skip that many patches after a tag,
which we found useful in various cases.
Some of these packages have invented home-brewed methods to generate
the Patch lines in the spec file, eg:
http://pkgs.fedoraproject.org/cgit/erlang.git/tree/otp-get-patches.sh
http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/copy-patches.sh?h=f20
https://github.com/redhat-openstack/redhat-openstack.github.com/blob/scripts/update-patches.sh
More importantly, all are using random git repositories to store the
exploded tree. This makes it difficult for co-maintainers and proven
packagers to fit in with the patch management chosen by the
maintainer. Usually they won't have access to the git repository for
these patches, making it difficult to add patches and near impossible
to upgrade to a new version.
I think that git is an excellent way to manage patches, but we ought
to think about formalizing this process. I think the goals should be:
(1) A git repository is used that co-maintainers and proven packagers
automatically have access to.
This would be the crux of formalizing this.
For the OpenStack packages we use a separate github organisation
to manage this (as well as other things):
http://github.com/redhat-openstack/
(2) A single method script is used to update the patches in the spec file.
Although there is already a git repository satisfying (1) above,
namely dist-git, it isn't suitable for storing the exploded tree since
commits to the spec file would conflict with commits (patches) to the
tree. So either a separate side repository which packages could
opt-in to, or perhaps a separate branch of the same git repo could be
used. I think using a branch would require no additional
infrastructure.
For (2) I would suggest a lightweight technique where git-managed
patches are marked in the spec file using:
### GIT-MANAGED-PATCHES ###
### END-GIT-MANGED-PATCHES ###
and a simple script that replaces everything between those marks with
Patch lines. The script could be adapted from copy-patches.sh
(see above).
To apply the patches, a standard RPM macro could be created:
%prep
%setup -q
%{git_apply_patches}
which would expand to something like:
git init
git config user.email %{name}-ow...@fedoraproject.org
git config user.name %{name}
git add .
git commit -a -q -m %{version} baseline
git am %{patches}
Thoughts on this?
Using patch(1) to apply patches mostly works with a few caveats
(binary patches come to mind). Using git to apply is a new dependency
but also the most general method for applying.
thanks,
Pádraig.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 01:29:06PM +, Richard W.M. Jones wrote: On Tue, Nov 19, 2013 at 01:39:50PM +0100, Karel Zak wrote: We have to learn fedpkg to do all the magic ;-) Something like add remote git tree with exploded tree: fedpkg exploded-tree add ssh://git.fedorahosted.org/git/foo.git This is all great, but the problem is that co-maintainers and provenpackagers need to be (automatically if possible) added to the fedorahosted tree. Otherwise there's a big extra step for them if they want to follow the package owner's preferred patching system. Ideally the GIT SCM request added to bugzilla when reviews are approved would have a Upstream GIT URL option and would setup a clone of this, and create branches for the fedora releases, and apply the same permissioning model from dist-git branches of the same names. Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On 19.11.2013 16:20, Daniel P. Berrange wrote: On Tue, Nov 19, 2013 at 01:29:06PM +, Richard W.M. Jones wrote: On Tue, Nov 19, 2013 at 01:39:50PM +0100, Karel Zak wrote: We have to learn fedpkg to do all the magic ;-) Something like add remote git tree with exploded tree: fedpkg exploded-tree add ssh://git.fedorahosted.org/git/foo.git This is all great, but the problem is that co-maintainers and provenpackagers need to be (automatically if possible) added to the fedorahosted tree. Otherwise there's a big extra step for them if they want to follow the package owner's preferred patching system. Ideally the GIT SCM request added to bugzilla when reviews are approved would have a Upstream GIT URL option and would setup a clone of this, and create branches for the fedora releases, and apply the same permissioning model from dist-git branches of the same names. What about intermediate step: optional fNN-upstream branch in addition to fNN, containing relevant upstream revision as git submodule (preferably referencing fedorahosted mirror, but initially also allowing external clones)? Kind Regards, Alek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 07:27:20PM +0200, Alek Paunov wrote: On 19.11.2013 16:20, Daniel P. Berrange wrote: On Tue, Nov 19, 2013 at 01:29:06PM +, Richard W.M. Jones wrote: On Tue, Nov 19, 2013 at 01:39:50PM +0100, Karel Zak wrote: We have to learn fedpkg to do all the magic ;-) Something like add remote git tree with exploded tree: fedpkg exploded-tree add ssh://git.fedorahosted.org/git/foo.git This is all great, but the problem is that co-maintainers and provenpackagers need to be (automatically if possible) added to the fedorahosted tree. Otherwise there's a big extra step for them if they want to follow the package owner's preferred patching system. Ideally the GIT SCM request added to bugzilla when reviews are approved would have a Upstream GIT URL option and would setup a clone of this, and create branches for the fedora releases, and apply the same permissioning model from dist-git branches of the same names. What about intermediate step: optional fNN-upstream branch in addition to fNN, containing relevant upstream revision as git submodule (preferably referencing fedorahosted mirror, but initially also allowing external clones)? I think it would be better to keep the upstream repo separate for sake of size. People who just want to checkout the latest RPM branches don't want to have to pull down 100's of MB, potentially GB, worth of GIT repo, when current Fedora GIT repos are a mere few MB. Only maintainers actively updating patches need that burden. Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On 19.11.2013 19:32, Daniel P. Berrange wrote: On Tue, Nov 19, 2013 at 07:27:20PM +0200, Alek Paunov wrote: What about intermediate step: optional fNN-upstream branch in addition to fNN, containing relevant upstream revision as git submodule (preferably referencing fedorahosted mirror, but initially also allowing external clones)? I think it would be better to keep the upstream repo separate for sake of size. People who just want to checkout the latest RPM branches don't want to have to pull down 100's of MB, potentially GB, worth of GIT repo, when current Fedora GIT repos are a mere few MB. Only maintainers actively updating patches need that burden. Git submodules are just references to external revision, not materialized inclusion of the referenced repo, aren't they? Someone should explicitly add --recursive or git submodule command to pull them. Kind Regards, Alek I just checked once again: git clone https://github.com/JulesAU/node-msgpack Working directory contains file .gitmodules with 3 lines: [submodule deps/msgpack-full] path = deps/msgpack-full url = https://github.com/msgpack/msgpack.git Site web interface shows: https://github.com/JulesAU/node-msgpack/tree/master/deps git submodule status: -d642487f9916dfe571531d0c6d0727b3ee449d36 deps/msgpack-full ls -al deps/msgpack-full: total 8 drwxrwsr-x 2 alek src 4096 Nov 19 20:06 . drwxrwsr-x 3 alek src 4096 Nov 19 20:06 .. However if: git clone --recursive https://github.com/JulesAU/node-msgpack node-msgpack-with-deps ls -al deps/msgpack-full: total 72 drwxrwsr-x 15 alek src 4096 Nov 19 20:14 . drwxrwsr-x 3 alek src 4096 Nov 19 20:13 .. drwxrwsr-x 4 alek src 4096 Nov 19 20:14 cpp drwxrwsr-x 2 alek src 4096 Nov 19 20:14 csharp ... -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 07:27:20PM +0200, Alek Paunov wrote: On 19.11.2013 16:20, Daniel P. Berrange wrote: On Tue, Nov 19, 2013 at 01:29:06PM +, Richard W.M. Jones wrote: On Tue, Nov 19, 2013 at 01:39:50PM +0100, Karel Zak wrote: We have to learn fedpkg to do all the magic ;-) Something like add remote git tree with exploded tree: fedpkg exploded-tree add ssh://git.fedorahosted.org/git/foo.git This is all great, but the problem is that co-maintainers and provenpackagers need to be (automatically if possible) added to the fedorahosted tree. Otherwise there's a big extra step for them if they want to follow the package owner's preferred patching system. Ideally the GIT SCM request added to bugzilla when reviews are approved would have a Upstream GIT URL option and would setup a clone of this, and create branches for the fedora releases, and apply the same permissioning model from dist-git branches of the same names. What about intermediate step: optional fNN-upstream branch in addition to fNN, containing relevant upstream revision as git submodule (preferably referencing fedorahosted mirror, but initially also allowing external clones)? The real issue is still access control. The upstream repo should be accessible read/write by the same people who are permitted to commit to the dist-git repo. It's my understanding that git submodules don't necessarily help with that. It's a shame that git can't reference an external repo (for history). That would massively reduce the amount of storage needed. [AFAIK ...] Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Using git for patch management in Fedora
On Tue, Nov 19, 2013 at 05:32:01PM +, Daniel P. Berrange wrote: On Tue, Nov 19, 2013 at 07:27:20PM +0200, Alek Paunov wrote: On 19.11.2013 16:20, Daniel P. Berrange wrote: On Tue, Nov 19, 2013 at 01:29:06PM +, Richard W.M. Jones wrote: On Tue, Nov 19, 2013 at 01:39:50PM +0100, Karel Zak wrote: We have to learn fedpkg to do all the magic ;-) Something like add remote git tree with exploded tree: fedpkg exploded-tree add ssh://git.fedorahosted.org/git/foo.git This is all great, but the problem is that co-maintainers and provenpackagers need to be (automatically if possible) added to the fedorahosted tree. Otherwise there's a big extra step for them if they want to follow the package owner's preferred patching system. Ideally the GIT SCM request added to bugzilla when reviews are approved would have a Upstream GIT URL option and would setup a clone of this, and create branches for the fedora releases, and apply the same permissioning model from dist-git branches of the same names. What about intermediate step: optional fNN-upstream branch in addition to fNN, containing relevant upstream revision as git submodule (preferably referencing fedorahosted mirror, but initially also allowing external clones)? I think it would be better to keep the upstream repo separate for sake of size. People who just want to checkout the latest RPM branches don't want to have to pull down 100's of MB, potentially GB, worth of GIT repo, when current Fedora GIT repos are a mere few MB. Only maintainers actively updating patches need that burden. It probably makes sense to have a lookaside git that's similar to the lookaside cache. One note though, I think that in the past one of the discussion points we've foundered on is whether we want to be mirroring upstream's git repo or (approximately) upstream's releases. I think that's probably where we'd need to start discussion. -Toshio pgp2ZlEZbZSlo.pgp Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
