Re: How to cope with patches sanely
On Sun, 03 Feb 2008, Raphael Hertzog wrote: On Sat, 02 Feb 2008, Charles Plessy wrote: Is there sombody working on WigPen? Is the format consensual enough that it would be accepted in Debian? I plan to work on it (but have not done anything yet except thinking about it and following discussions), the format might need some tweaks (explicit list of patches instead of implicit for example) but the basic principles are sane and I believe it will be accepted in Debian. The wigpen format has been implemented and much more: http://www.ouaza.com/wp/2008/03/16/new-source-package-formats-call-for-tests/ http://lists.debian.org/debian-dpkg/2008/03/msg00155.html http://lists.debian.org/debian-dpkg/2008/03/msg00166.html Testers are needed so that we can merge that in time for lenny. Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
also sprach Manoj Srivastava [EMAIL PROTECTED] [2008.02.29.2153 +0100]:
3) I propose ./debian/branches/{TopicA,TopicB,TopicC}.diff.gz
files. Each diff, applied to the orig.tar.gz , shall recreate for
the interested user the corresponding branch in my development.
Bingo. With this addition, every user that want to see where the
integration branch comes from can examine each topic patch or topic
branch. Each of these topic branches can then be compiled and
experimented with. Upstream can incorporate each of these topic
patches, if they wish.
... until I want to play with two branches at the same time, or
upstream wants to pull two branches. Now you are forcing users to
deal with potential conflicts.
The downside, of course, is shipping the same patch twice, once
in the diff.gz, and once in ./debian/branches/*.diff.gz.
I don't see the added value in your approach. I don't see the use
case. I know your workflow and note how this is a continuation
thereof, but I can't identify the benefit to others and the project
in doing this. Do you really think there are many people or
upstreams who want pristine feature branches without being able to
use the net? Why wouldn't these people be helped with a quilt
series? They just want to work on feature B, do you think they
actually care that quilt first pops A before it applies B,
especially with tools like interdiff around?
Can you try to quantify?
--
.''`. martin f. krafft [EMAIL PROTECTED]
: :' : proud Debian developer, author, administrator, and user
`. `'` http://people.debian.org/~madduck - http://debiansystem.info
`- Debian - when you have better things to do than fixing systems
good advice is something a man gives
when he is too old to set a bad example.
-- la rouchefoucauld
digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
Re: How to cope with patches sanely
* martin f. krafft:
also sprach Manoj Srivastava [EMAIL PROTECTED] [2008.02.29.2153 +0100]:
3) I propose ./debian/branches/{TopicA,TopicB,TopicC}.diff.gz
files. Each diff, applied to the orig.tar.gz , shall recreate for
the interested user the corresponding branch in my development.
Bingo. With this addition, every user that want to see where the
integration branch comes from can examine each topic patch or topic
branch. Each of these topic branches can then be compiled and
experimented with. Upstream can incorporate each of these topic
patches, if they wish.
... until I want to play with two branches at the same time, or
upstream wants to pull two branches. Now you are forcing users to
deal with potential conflicts.
Yes, but that's way it is. You can't publish all possible combinations
of branches upstream might need. If you lump some of them together
(either by combining them altogether, or branching one from the other),
upstream won't be able to take them because there are some changes they
don't want. If you keep them entirely separate, upstream needs to do
some integreation work. Either way, there is some work left to do.
I don't see the added value in your approach. I don't see the use
case. I know your workflow and note how this is a continuation
thereof, but I can't identify the benefit to others and the project
in doing this.
The nice thing about Manoj's proposal that we (as in the security
team, for instance) need not care if the Debian maintainer thinks that
upstream needs pristine topic branches, an integration branch, a weave,
or whatever. We just patch the source and be done with it. This isn't
a problem as long as we tell upstream to pick patches from unstable
(which they will likely do anyway because that version is much closer to
theirs most of the time).
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
also sprach Florian Weimer [EMAIL PROTECTED] [2008.03.01.1334 +0100]: The nice thing about Manoj's proposal that we (as in the security team, for instance) need not care if the Debian maintainer thinks that upstream needs pristine topic branches, an integration branch, a weave, or whatever. We just patch the source and be done with it. This isn't a problem as long as we tell upstream to pick patches from unstable (which they will likely do anyway because that version is much closer to theirs most of the time). A quilt series should satisfy those needs as well. If not, please explain where it falls short. -- .''`. martin f. krafft [EMAIL PROTECTED] : :' : proud Debian developer, author, administrator, and user `. `'` http://people.debian.org/~madduck - http://debiansystem.info `- Debian - when you have better things to do than fixing systems wahnsinn bei individuen ist selten, aber in gruppen, nationen und epochen die regel. - friedrich nietzsche digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/)
Re: How to cope with patches sanely
* martin f. krafft: also sprach Florian Weimer [EMAIL PROTECTED] [2008.03.01.1334 +0100]: The nice thing about Manoj's proposal that we (as in the security team, for instance) need not care if the Debian maintainer thinks that upstream needs pristine topic branches, an integration branch, a weave, or whatever. We just patch the source and be done with it. This isn't a problem as long as we tell upstream to pick patches from unstable (which they will likely do anyway because that version is much closer to theirs most of the time). A quilt series should satisfy those needs as well. If not, please explain where it falls short. It does, if you ship the sources with the series applied. AFAICT, this is not what's usually done. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
also sprach Florian Weimer [EMAIL PROTECTED] [2008.03.01.1650 +0100]: It does, if you ship the sources with the series applied. AFAICT, this is not what's usually done. ... or if the patches were automatically applied when the source is unpacked, which is where I think we're heading. -- .''`. martin f. krafft [EMAIL PROTECTED] : :' : proud Debian developer, author, administrator, and user `. `'` http://people.debian.org/~madduck - http://debiansystem.info `- Debian - when you have better things to do than fixing systems you can't assign IP address 127.0.0.1 to the loopback adapter, because it is a reserved address for loopback devices. -- micro$oft windoze xp professional digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/)
Re: How to cope with patches sanely
On Sat, 1 Mar 2008 14:16:20 +0100, martin f krafft [EMAIL PROTECTED] said: also sprach Florian Weimer [EMAIL PROTECTED] [2008.03.01.1334 +0100]: The nice thing about Manoj's proposal that we (as in the security team, for instance) need not care if the Debian maintainer thinks that upstream needs pristine topic branches, an integration branch, a weave, or whatever. We just patch the source and be done with it. This isn't a problem as long as we tell upstream to pick patches from unstable (which they will likely do anyway because that version is much closer to theirs most of the time). A quilt series should satisfy those needs as well. If not, please explain where it falls short. A quilt series is hard to generate from my setup; branch diffs are not. A quilt series only becomes viable as an exclusive source package format if it can be created in all cases; forcing people to abandon all their work flows and migrate to quilt is a non-starter. If we are not talking about exclusively using quilt, then I do not understand your question -- sure, some people use quilt. Some of us do not. Unless there is a way to generate a quilt series for the rest of us easily, we are not going to have quilt in all source packages. Why is this so hard to understand? manoj -- Time is an illusion perpetrated by the manufacturers of space. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sat, 1 Mar 2008 12:21:03 +0100, martin f krafft [EMAIL PROTECTED] said:
also sprach Manoj Srivastava [EMAIL PROTECTED] [2008.02.29.2153
+0100]:
3) I propose ./debian/branches/{TopicA,TopicB,TopicC}.diff.gz
files. Each diff, applied to the orig.tar.gz , shall recreate for the
interested user the corresponding branch in my development.
Bingo. With this addition, every user that want to see where the
integration branch comes from can examine each topic patch or topic
branch. Each of these topic branches can then be compiled and
experimented with. Upstream can incorporate each of these topic
patches, if they wish.
... until I want to play with two branches at the same time, or
upstream wants to pull two branches. Now you are forcing users to deal
with potential conflicts.
I am not asking users to do any such thing. I am asking
developers who want to play with tweo unrelated and conflcting features
to do some integration work, with my integration branch to use as an
example to see how such conflicts can be resolved.
If you are a developer, and want to play with conflicting or
overlapping features, and can't do integration work given a working
example of how such integration can be done, stay away from the
kitchen. I am not going to jump through hoops to cater to your
incompetence.
The downside, of course, is shipping the same patch twice, once in
the diff.gz, and once in ./debian/branches/*.diff.gz.
I don't see the added value in your approach. I don't see the use
case. I know your workflow and note how this is a continuation
thereof, but I can't identify the benefit to others and the project in
doing this. Do you really think there are many people or upstreams who
want pristine feature branches without being able to use the net?
I am providing people with the same development infrastructure I
use, in the debian source package, without impacting
repeatability. There was a criticism that the source package that
contains only a diff.gz is fairly opaque to developers -- and that an
explanation should be provided to how the diff.gz came about, and what
are its constituent parts. I am providing people a better understanding
of each feature by putting in stand alone, and not making it dependent
on a bunch of unrelated features.
Let me reiterate the goals I put forth for me:
A) All the branches that I use (the pure feature branches, the
upstream branch, and the integration branch) should be made
available to the users. This will give them the same environment I
have, and thus they have the preferred place to make modifications.
B) When people do dpkg-source -x, they should have a fully unpacked
tree that is compiled, with no further action that needs to be
taken
C) In order to reconstitue all the other branches, no network
connectivity should be required. There a lot of people in the
developing world that do not have a readily available network
connection -- my solution must work with source DVD's
D) No knowledge of my SCM should be required. People should be able
to construct the topic branches without knowing arch or git or bzr
or Hg or what have you.
I am also providing developers the pure features, uncluttered by
integration junk, so people can see what the intent of the changes was,
cleanly; and apply any such feature to upstream, and have it work.
There is a combinatorics problem here. If you have N features,
you might want to see them one at a time, two at a time, three at a
time or N! ways. I provide the common cases: One at a time, or all
at the same time.
People who want the rest of the N! ways may have to do some
integration work -- but I do provide a working, fully integrated
example for them to program by example from.
Why wouldn't these people be helped with a quilt series? They just
want to work on feature B, do you think they actually care that quilt
first pops A before it applies B, especially with tools like interdiff
around?
Show me the code. Get me a quilt series from my feature
branches. Until you can get me an automated way to get a quilt series
from my work-flow, the quilt series option is off the table. I find a
quilt series to be inferior to topic branches. I understand others do
not feel that way. I am not forcing them not to use quilt -- though
that would, in my opinion, improve the quality of the distribution.
Stop trying to make me put in work to use quilt.
I dislike the extra work and indirection of using patches (I
am much better at reading code than reading code + patch). I dislike
the extra effort it takes to read code in different branches without
doing extra work, poping and applying stuff -- and dealing with the
integration work every time you pop and reapply and there is a
conflict). I am never going to use quilt -- and if you try to force me,
t.
Now,
Re: How to cope with patches sanely
On Sat, 1 Mar 2008 17:24:49 +0100, martin f krafft [EMAIL PROTECTED] said: also sprach Florian Weimer [EMAIL PROTECTED] [2008.03.01.1650 +0100]: It does, if you ship the sources with the series applied. AFAICT, this is not what's usually done. ... or if the patches were automatically applied when the source is unpacked, which is where I think we're heading. Why do we have to settle on a quilt based source package, when my proposal meets all the requirements anyway? Why does it have to be one or the other? Why is the requirement not just: a) on dpkg-source -x; you get what you need to compile and build the package b) The monolithic diff.gz has additional information provided that shows the user the different lines of development that have been integrated into the Debian package c) This additional information should not need knowledge of an SCM or network access And let people figure out on their own how to make this happen? (Like, perhaps, teaching dpkg about the top few stacked patch mechanisms). Why standardize on tools instead of specifying results? Methinks that useless conformity comes close to foolish consistency, and I am opposed to hobgoblins. manoj -- Operator, please trace this call and tell me where I am. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sat, 01 Mar 2008, Manoj Srivastava wrote: Why do we have to settle on a quilt based source package, when my proposal meets all the requirements anyway? Why does it have to be one or the other? It's not going to be one or the other. Note that your changes on upstream code can be a single diff in a quilt serie anyway. The way to store patches has no relation with the way you construct your patches. :) Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
Hi, For people who are trying to figure out what my merging and branching workflow looks like, I have uploaded a recent picture for fvwm at: http://people.debian.org/~srivasta/fvwm.png I should warn you that this is a large image; and is known to crash iceweasel. I suggest downloading it and using gqview, and go to the 1:1 zoom, and pan around. Having done that, ask yourself if you really want to write a script that trolls through such a branch and merge history to automatically generate a stacked patch series. manoj -- Four be the things I'd have been better without: love, curiosity, freckles and doubt. -- Dorothy Parker Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Fri, Feb 29, 2008 at 09:30:16PM +0100, Florian Weimer wrote: * Ben Finney: It's no security risk to unpack a tarball, apply a patch to it via GNU 'patch', and examine the result. History should tell you that this is not true. 8-) I can even understand people who state that GNU tar should never be used to uncompress tarballs from untrusted sources, and we therefore do not need to provide security support for it, but this is going a bit too far for my taste. But my point really is: Please do do not use potential security issues as arguments. The overall situation is sufficiently bad that this can be used to prove *anything*. I think the difference between the occasional vulnerability in GNU tar and a system that is designed to operate by executing arbitrary marginally-trusted code is, erm, rather significant. -- Colin Watson [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Let's kill this part of the discussion right away [was: Re: How to cope with patches sanely]
Manoj Srivastava wrote: And no, I can do this using plain old arch, and I don't really have to change my SCM. But not all Debian maintainers are using git; Version control systems that have content-addressable filesystems (essentially, git and Monotone) are inherently efficient to distribute; [...] Which is great, but I fear it will not fly as a the one and only Ok, I apologise for leaving my note to this effect to the end of my e-mail. But let me repeat. I'm not asking that you change the SCM that you use. I'll say it again, this time for the other subscribers. I'm NOT asking that you change the SCM that you use or the way that you work. All I'm talking about is using git as a replacement for the *source archive* format. How the files are archived and distributed. There are compelling reasons to want to do this; not least getting around the fact that shipping patch series in .diff.gz doesn't handle cases like integration branches without adding a new patch format (which is also something I think is probably useful, and a complementary approach). Thanks for the rest of your e-mail, which contains constructive feedback which I will now digest and respond to! Cheers, Sam. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
* Ben Finney: It's no security risk to unpack a tarball, apply a patch to it via GNU 'patch', and examine the result. History should tell you that this is not true. 8-) I can even understand people who state that GNU tar should never be used to uncompress tarballs from untrusted sources, and we therefore do not need to provide security support for it, but this is going a bit too far for my taste. But my point really is: Please do do not use potential security issues as arguments. The overall situation is sufficiently bad that this can be used to prove *anything*. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
* Manoj Srivastava: But there is no such linearization, not in the way that quilt et al do it. The state of such integration is not maintained in the feature branches; it is in the history of the integration branch. As each feature branch was created or developed, if there were any conflicts, these conflicts were resolved in the integration branch. And the integration branch does not keep track of what changes come from which branch -- that is not its job. And it does not keep the integration patches up to date either -- that again is not its job. Details below. There are some systems that automatically push changes on a branch to all branches that branched from it (Clearcase is in that category, IIRC, and Aegis does something similar). My experience with Aegis is that this is usually *not* what you want, and the prevalent DVCS model uniformly rejects this idea. However, this would help to get rid of the integration branch. Changes to lower branches would bubble up, if necessary with manual help, and a separate integration branch would not be necessary. I'm not sure if this is actually workable (probably not with the branching model currently in fashion), but it might be. Anyway, this gets me back to my original question: Is there tools support for dealing with patch series (quilt or dpatch) which lets me bubble up patches (including new upstream versions, at least conceptually) and sink them down the queue? With three-way merging, so that it's comparable to typical in-VCS operation? Has anybody written the converse to git-quiltimport (which should go to great lengths not to create any gratuitous changes), and a port of that script to dpatch? -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sat, Jan 26, 2008 at 11:39:32PM +0100, Pierre Habouzit wrote:
Another thing is that people have the old habit to see the source
package be the preferred form of modification for a Debian package.
Hmm. This started off a train of thought. In one sense, one
could see the source code for the binary package as the sources
delivered by applying the diff.gz to the upstream tarball, and the rest
merely tools to deliver the sources, somewhat like editors. I don't
want to go down that debate, since the binary blob people would jump in
and claim that the hardware editors creating the binary blobs are,
ummm, editors.
David Nusinow has criticized my methodology as delivering a
mostly opaque diff.gz; and I have been thinking about that too. The
criticism has merit; and I allowed myself to get distracted by the
whole quilt subthread, which was a red herring. quilt introduces
linearlization of patches, etc, and ordering, which were merely
distractions.
But the comment above by Pierre got me thinking: when someone
reports a bug in my package, and I fix it, or when I add a whole new
feature; I do not do it on the integration branch. I do it on a new
branch, dedicated to that series of fixes or the new feature, and the
goal of the branch is to maintain a clean topic patch to feed
upstream. I then post the same fix into the integration branch, easily
done. So the modifications are being made in a topic branch -- but
then, I ship only the integration branch (which is what the diff.gz
represents) in the source package.
So, in a very real sense, I do not ship the branch where I
prefer to make the changes.
Now, David had a point that people who need to make changes need
to understand how to take the diff.gz apart, and understand what parts
of the diff.gz correspond to which logical line of development. This
annotation of the diff lies in the topic branches; the integration
branch has elided the annotation, in one sense. So, by only shipping
the integration branch (as a diff.gz), I am eliding information that is
present in my SCM, and not presenting that to the end user. This is not
a good thing.
Then it struck me: why not present the end users with the same
environment and history that I have when I make changes? So, here are
the goals of my proposal:
A) All the branches that I use (the pure feature branches, the
upstream branch, and the integration branch) should be made
available to the users. This will give them the same environment I
have, and thus they have the preferred place to make modifications.
B) When people do dpkg-source -x, they should have a fully unpacked
tree that is compiled, with no further action that needs to be
taken
C) In order to reconstitue all the other branches, no network
connectivity should be required. There a lot of people in the
developing world that do not have a readily available network
connection -- my solution must work with source DVD's
D) No knowledge of my SCM should be required. People should be able
to construct the topic branches without knowing arch or git or bzr
or Hg or what have you.
Now, a lot of what I need is already present.
1) the orig.tar.gz represents the upstream branch, exactly.
2) the diff.gz + orig.tar.gz represents the integration branch,
exactly.
So the missing thing is the topic branches.
3) I propose ./debian/branches/{TopicA,TopicB,TopicC}.diff.gz
files. Each diff, applied to the orig.tar.gz , shall recreate for
the interested user the corresponding branch in my development.
Bingo. With this addition, every user that want to see where the
integration branch comes from can examine each topic patch or topic
branch. Each of these topic branches can then be compiled and
experimented with. Upstream can incorporate each of these topic
patches, if they wish.
Any end user who wishes to modify Topic branch C can, then,
modify the topic branch, and apply the same delta to the integration
branch, and they have done the same thing that I would do with the
patch. In other words, they have the same work flow, and preferred
means of modification, even if they do not know my SCM.
People who do not care about independent lines of development
can just ignore ./debian/branches, since that directory is never used
in building, and is for human consumption.
The downside, of course, is shipping the same patch twice, once
in the diff.gz, and once in ./debian/branches/*.diff.gz.
Comments welcome.
manoj
--
Laugh while you can, monkey-boy. Dr. Emilio Lizardo
Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
* Manoj Srivastava:
Now, a lot of what I need is already present.
1) the orig.tar.gz represents the upstream branch, exactly.
2) the diff.gz + orig.tar.gz represents the integration branch,
exactly.
So the missing thing is the topic branches.
3) I propose ./debian/branches/{TopicA,TopicB,TopicC}.diff.gz
files. Each diff, applied to the orig.tar.gz , shall recreate for
the interested user the corresponding branch in my development.
I like it a lot. It's somewhat debatable how to deal with divergence
between the sume of topic branches and the integration branch/real
source package contents, but that's something the package maintainers
can be tasked with. What's particularly charming about this scheme is
that such discrepancies do not impact repeatable builds.
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
Manoj Srivastava wrote: Feature branches don't magically allow you to avoid merge conflicts either, so this is a red herring. Once you've resolved the conflict, then it becomes just another change. This change can become a diff in a stack of diffs. This whole message is a red herring, since hte feature branches do not attempt to handle merge conflicts -- that is not their purpose. They capture one single feature, independently from every other feature, and thumb their collective noses at merge conflicts. Yes. Feature branches are effectively forking a particular version of a project - this is not a problem, and is essential for efficient development. People jumbling together changes in trunk branches is perhaps one of the worst upshots of the 2002-2006 or so obsession with poorly designed centralised systems and in my opinion sank many projects. The history of the integration branch captures the integration effort; and the integration branch makes no effort to keep the integration work up to date with current upstream and feature branches. Initially perhaps. However, once a feature is considered ready for inclusion, it is important that it contains merges FROM the branch they are targetting. They mean that a later merge back the other way, to merge the feature branch into the target branch, can happen painlessly. ASSUMING that you're using a system which has commutative merge characteristics, such as git or mercurial. If you think you can extract an up to date integration patch from the entrails of the integration branch -- feel free o smack me down. But please provide some substance to the assertion that it is doable. Perhaps I missed the context to this discussion - certainly expressing a history containing merge nodes in patches is non-trivial and can't be done with standard patch format - but I believe that this is certainly possible. Can you express this problem with reference to a particular history of an integration branch? I will provide some short git commands to extract the information in the form you are after. Sam -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Fri, 29 Feb 2008 12:35:30 +1300, Sam Vilain [EMAIL PROTECTED] said: Manoj Srivastava wrote: Feature branches don't magically allow you to avoid merge conflicts either, so this is a red herring. Once you've resolved the conflict, then it becomes just another change. This change can become a diff in a stack of diffs. This whole message is a red herring, since hte feature branches do not attempt to handle merge conflicts -- that is not their purpose. They capture one single feature, independently from every other feature, and thumb their collective noses at merge conflicts. Yes. Feature branches are effectively forking a particular version of a project - this is not a problem, and is essential for efficient development. People jumbling together changes in trunk branches is perhaps one of the worst upshots of the 2002-2006 or so obsession with poorly designed centralised systems and in my opinion sank many projects. Err. If you go back and read this thread in the archive, You'll note that I have stated that my feature branches are always kept up to date with the latest upstream branch I am basing my Debian package on. When I have been creating patches for inclusion with upstream, I essentially feed them the source patch and a changelog entry -- essentially, creating a single patch series; squashing the underlying history. Most upstream do not care about the messy history of my development; and most do not grok arch well enough to pull directly. I am not sure what the relevance of trunk changes you mention has to the current thread. The history of the integration branch captures the integration effort; and the integration branch makes no effort to keep the integration work up to date with current upstream and feature branches. Initially perhaps. However, once a feature is considered ready for inclusion, it is important that it contains merges FROM the branch they are targetting. Please do read the thread history. The feature branches being kept updated with the upstream branch means that my feature branches _always_ apply to the current upstream. They mean that a later merge back the other way, to merge the feature branch into the target branch, can happen painlessly. ASSUMING that you're using a system which has commutative merge characteristics, such as git or mercurial. I use Arch. If you think you can extract an up to date integration patch from the entrails of the integration branch -- feel free o smack me down. But please provide some substance to the assertion that it is doable. Perhaps I missed the context to this discussion - certainly expressing I think you have. a history containing merge nodes in patches is non-trivial and can't be done with standard patch format - but I believe that this is certainly possible. Great. Show me the code. My arch repo is publicly available on arch.debian.org. As they say in Missouri, Show me. Can you express this problem with reference to a particular history of an integration branch? I will provide some short git commands to extract the information in the form you are after. http://arch.debian.org/cgi-bin/archzoom.cgi/[EMAIL PROTECTED] Take any package. Say, flex. Or flex-old. You have all my feature branches there. The --devo branch is the integration branch. Please show me an automated way you can grab the feature branches and generate a quilt series that gives you the devo branch. The diff.gz is how we get from upstream to the devo branch (modulo ./debian); if you can break that down nicely for the folks who want each feature separate, that would work as well. If you code works well enough every single time a new upstream comes around and I release a new version of flex or whatever, I'll throw in the generated quilt patches. Until then, could people stop telling me how easy it is to automatically generate quilt series for my packages, and that I should jut shut up and code it and not stand in the way of other people trying to make such quilt series generation the standard way of doing source packages? BTW, I have heard no one comment on my offer to generate a pure patch for each feature branch, with no warranty that the patches can be applied linearly, along with the diff.gz that defines the integration branch, so that a human can probably tell what most of the changes mean (which is what people seemed to be after with neatly separated out patches). manoj who does not think that one can go easily from a set of independent pure feature patches that separately apply to upstream to a quilt series programmatically -- Did I do an INCORRECT THING?? Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe.
Re: How to cope with patches sanely
Manoj Srivastava wrote: Yes. Feature branches are effectively forking a particular version of a project - this is not a problem, and is essential for efficient development. People jumbling together changes in trunk branches is perhaps one of the worst upshots of the 2002-2006 or so obsession with poorly designed centralised systems and in my opinion sank many projects. Err. If you go back and read this thread in the archive, You'll note that I have stated that my feature branches are always kept up to date with the latest upstream branch I am basing my Debian package on. This technique is also called rebasing the patch set; it's fine, but it's just one approach. When I have been creating patches for inclusion with upstream, I essentially feed them the source patch and a changelog entry -- essentially, creating a single patch series; squashing the underlying history. Most upstream do not care about the messy history of my development; and most do not grok arch well enough to pull directly. This is sometimes worthwhile and sometimes a bad idea. The driving motive, if you want to aim for patches to be easily reviewed, is that each patch should introduce a single change, which is well explained. I agree that the upstream will not want a messy history; which is why you reshape the individual changes using a tool such as Quilt, Stacked Git, Guilt, Mercurial Queues, etc, so that they are more easily reviewed. They mean that a later merge back the other way, to merge the feature branch into the target branch, can happen painlessly. ASSUMING that you're using a system which has commutative merge characteristics, such as git or mercurial. I use Arch. Arch is critically deficient in this respect; it doesn't really have a concept of tracking branches, and merging is not commutative; if you merge a branch that just merged from your branch, an unnecessary new changeset is made. But if you are rebasing then you don't need to worry about that. As I said, it's just more work. Can you express this problem with reference to a particular history of an integration branch? I will provide some short git commands to extract the information in the form you are after. http://arch.debian.org/cgi-bin/archzoom.cgi/[EMAIL PROTECTED] Take any package. Say, flex. Or flex-old. You have all my feature branches there. The --devo branch is the integration branch. Please show me an automated way you can grab the feature branches and generate a quilt series that gives you the devo branch. The diff.gz is how we get from upstream to the devo branch (modulo ./debian); if you can break that down nicely for the folks who want each feature separate, that would work as well. Thanks for restating the problem clearly. While the underlying problem is easily approached and I would still call it trivial, the details of what you are asking for make it impossible - because quilt series cannot contain merges (someone correct me here if it can and I can go forward). Shipping changes for upstream inclusion as a *single* set of quilt patches is not possible if you are including merges, but if you allow the patches to be grouped, and introduce a new type of patch which encapsulates a merge (gitk has one example of this; it uses different identifiers to represent which file's lines are included), then it can be done. The apply-patches script would need extending to support this, but I don't think that's particularly show-stopping. However, ignoring the merges, so far we're not that far away from the script being 'git-log -p' or 'git format-patch upstreamrev' Also having never really used arch, if you can provide me with the commands to get a copy of those branches (the man page is sadly not very forthcoming), and I'll give the git-archimport script a whorl and see if I can get it imported and show how this can work in practice. If someone with git-archimport experience can perform this and publish the repositories somewhere, I'd be very grateful. If you code works well enough every single time a new upstream comes around and I release a new version of flex or whatever, I'll throw in the generated quilt patches. I think what is required is a rethink of the problem. What is being tried to be achieved, and are there any other ways to achieve it which will solve the problem in a vastly more effective way. Version control systems that have content-addressable filesystems (essentially, git and Monotone) are inherently efficient to distribute; as only the changes between versions need be distributed. The notion of stream compressing tarballs is archaic compared with being able to search for deltas anywhere in the source tree. You can see this in effect with git, which is capable of very quickly identifying which objects are new, and sending them all in impressively small packs on the network. It's amazing how many tarballs will then fit into the
Re: How to cope with patches sanely
On Fri, 29 Feb 2008 16:11:48 +1300, Sam Vilain [EMAIL PROTECTED] said: Manoj Srivastava wrote: Yes. Feature branches are effectively forking a particular version of a project - this is not a problem, and is essential for efficient development. People jumbling together changes in trunk branches is perhaps one of the worst upshots of the 2002-2006 or so obsession with poorly designed centralised systems and in my opinion sank many projects. Err. If you go back and read this thread in the archive, You'll note that I have stated that my feature branches are always kept up to date with the latest upstream branch I am basing my Debian package on. This technique is also called rebasing the patch set; it's fine, but it's just one approach. Actually, that is not it. I am not rebasing -- I am doing repeated merges. Arch does not rebase -- it just applies the upstream delta, with full history. This allows me to replay into the integration branch at will. When I have been creating patches for inclusion with upstream, I essentially feed them the source patch and a changelog entry -- essentially, creating a single patch series; squashing the underlying history. Most upstream do not care about the messy history of my development; and most do not grok arch well enough to pull directly. This is sometimes worthwhile and sometimes a bad idea. The driving motive, if you want to aim for patches to be easily reviewed, is that each patch should introduce a single change, which is well explained. I agree that the upstream will not want a messy history; which is why you reshape the individual changes using a tool such as Quilt, Stacked Git, Guilt, Mercurial Queues, etc, so that they are more easily reviewed. I can do this by cherry picking the chnages from my topic branch, and feeding it separately. Emacs and diff mode makes it easy to split off chunks if I want to do it after the fact from the squashed diff; or I can regenerate changesets and cherry pick the series. And no, I can do this using plain old arch, and I don't really have to change my SCM. They mean that a later merge back the other way, to merge the feature branch into the target branch, can happen painlessly. ASSUMING that you're using a system which has commutative merge characteristics, such as git or mercurial. I use Arch. Arch is critically deficient in this respect; it doesn't really have a concept of tracking branches, and merging is not commutative; if you merge a branch that just merged from your branch, an unnecessary new changeset is made. But if you are rebasing then you don't need to worry about that. As I said, it's just more work. Which is why we have sync-tree. Yes, I have to keep track myself of which delta I am currently merging; and only apply a merge once into each branch; and immedately syn-tree with the other branches. And since it is all in one fully automated script, called arch_upgrade, that takes any new upstream, updates all my topic branches and my integration branch automatically, I don't see this as much more work. Indeed, I have yet to see any porcelain that makes merging a new upstream commit into all my topic branches and the integration branch as a single operation, I suspect that it is more work in git; at least until I can replicate my arch scaffolding for the git porcelain. Can you express this problem with reference to a particular history of an integration branch? I will provide some short git commands to extract the information in the form you are after. http://arch.debian.org/cgi-bin/archzoom.cgi/[EMAIL PROTECTED] Take any package. Say, flex. Or flex-old. You have all my feature branches there. The --devo branch is the integration branch. Please show me an automated way you can grab the feature branches and generate a quilt series that gives you the devo branch. The diff.gz is how we get from upstream to the devo branch (modulo ./debian); if you can break that down nicely for the folks who want each feature separate, that would work as well. Thanks for restating the problem clearly. While the underlying problem is easily approached and I would still call it trivial, the details of what you are asking for make it impossible - because quilt series cannot contain merges (someone correct me here if it can and I can go forward). I don't use quilt, so I am not the one to answer this. Shipping changes for upstream inclusion as a *single* set of quilt patches is not possible if you are including merges, but if you allow the patches to be grouped, and introduce a new type of patch which encapsulates a merge (gitk has one example of this; it uses different identifiers to represent which file's lines are included), then it can be done. The apply-patches script would need extending to support this, but I don't think that's particularly show-stopping. Great. This
Re: How to cope with patches sanely
On Mon, 25 Feb 2008 19:11:16 -0500, David Nusinow [EMAIL PROTECTED] said: On Sun, Feb 24, 2008 at 09:31:10PM -0600, Manoj Srivastava wrote: On Mon, 25 Feb 2008 10:34:55 +1100, Ben Finney [EMAIL PROTECTED] said: Manoj Srivastava [EMAIL PROTECTED] writes: David Nusinow [EMAIL PROTECTED] said: No matter what you want to say about your feature branches, you *must* apply them in a linear fashion to your final source tree that you ship in the package. This is no way around it. But there is no such linearization, not in the way that quilt et al do it. The state of such integration is not maintained in the feature branches; it is in the history of the integration branch. Is this (the integration branch and its history of changes) not the linear sequence of changes that David Nusinow is asking for? No, it is not. I Apply a update to feature A. The comes an upstream update. Then updates on feature B, a patch that needed conflict resoution, then patches on branches C, D, and A again. Another upstream change. At this point, none of the original patches to A, B, and C apply any more -- and then come another upstream update, and all the patches get even more bent out of shape. At this point, before you're ready to release, you regenerate the patches. Then they apply just fine. Nothing gets bent out of shape and you don't include old code in your patch that's now incorporated upstream, you just make an appropriate diff that applies cleanly to your source package. I don't see what the problem is here and why you believe this can't be done. Sure, I can re-generate the patches: but then I have to do all the integration work that I did for the integration branch over the years, and I have to do this over and over and over again every single darn package upload. And why am I doing all this busy work? A compromise would be to provide a patch for each pure fearure branch, along with the giant diff -- this can be automated. But these individual patches will not apply in sequence unless the manual integration work is done again -- which is not something I am willing to do for every package upload. manoj -- #else /* !STDSTDIO */ /* The big, slow, and stupid way */ --Larry Wall #in str.c from the perl source code Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Mon, 25 Feb 2008 19:04:10 -0500, David Nusinow [EMAIL PROTECTED] said: On Mon, Feb 25, 2008 at 03:56:49AM -0600, Manoj Srivastava wrote: No, it does not. If branch A has pi = 2.34567; and branch B has pi = 3.14159; No matter how much quilting you do you cannot reconcile the fundamental conflict in the final. Either pi is 3.14159; or it is not; and if branch A requires pi not to be that value, and branch B requires pi to be that value, quilt can't make C be quantum like and have the value be both. Feature branches don't magically allow you to avoid merge conflicts either, so this is a red herring. Once you've resolved the conflict, then it becomes just another change. This change can become a diff in a stack of diffs. This whole message is a red herring, since hte feature branches do not attempt to handle merge conflicts -- that is not their purpose. They capture one single feature, independently from every other feature, and thumb their collective noses at merge conflicts. The history of the integration branch captures the integration effort; and the integration branch makes no effort to keep the integration work up to date with current upstream and feature branches. If you think you can extract an up to date integration patch from the entrails of the integration branch -- feel free o smack me down. But please provide some substance to the assertion that it is doable. manoj -- One friend in a lifetime is much; two are many; three are hardly possible. Henry Adams Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
also sprach Manoj Srivastava [EMAIL PROTECTED] [2008.02.25.0828 +0100]: I am not opposed to it. If you can somehow magically create a tool that can linearize the feature branches, more power to you. I personally find the prospect highly unlikely; and I would like to see some code, please, before I grant that such a thing is possible. The tool I envision would simply surf through the history of the integration branch and identify merge commits. Each merge would become a patch in the quilt series. Sure. You can't integrate two features that fundamentally conflict with each other. No amount of smoke and mirrors can obscure that fundamental obstacle. This is independent of the tool set you use. Except that quilt provides the necessary glue to handle it, while feature branches don't. -- .''`. martin f. krafft [EMAIL PROTECTED] : :' : proud Debian developer, author, administrator, and user `. `'` http://people.debian.org/~madduck - http://debiansystem.info `- Debian - when you have better things to do than fixing systems echo '[dO%O+38%O+PO/d00]Fi22os0CC4BA64E418CE7l0xAP' | dc digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/)
Re: How to cope with patches sanely
On Mon, Feb 25, 2008 at 03:37:07AM +, Manoj Srivastava wrote: On Sun, 24 Feb 2008 21:17:10 -0500, David Nusinow [EMAIL PROTECTED] said: On Sun, Feb 24, 2008 at 06:08:17PM -0800, Russ Allbery wrote: David Nusinow [EMAIL PROTECTED] writes: The problem is that you and Manoj assume that this is the only way to do things. I don't believe this. Pierre Habouzit has been experimenting with an alternative method of feature branches that exports to a linear stack of diffs just fine. Just because Manoj is doing something one way right now doesn't mean it's the only or even the correct way to do it. I would be interested in details of this, and whether this approach works with pure feature branches where the features are being developed contemporaneously with each other an upstream development; and thus the branches overlap both temporally and in code space. I'm planning to write a textual version of what I demonstrated at FOSDEM, with some more ideas that I had talking with Julien Cristau on the grass after. You developped them contemporaneously, okay, but in the end you merge them one after the other. If you're doing criss-cross merges, well, I can nothing for you, and you're creating really messy histories, and yes, you need an SCM to represent that in a satisfying way. But if you really merge one feature branch on top of the other, and it's in my experience *way enough* for what we need in Debian packaging, then multiple branches are just multiple series to be applied in a specific relative order. The problem is, I believe that there are two kind of things: patches that you backport from upstream. If you're lucky enough to have an upstream using git, it's just a matter of merging the stable branch into yours, or cherry-picking some patches, which will not create conflicts when you merge back. This goes in the .diff.gz, and it's okay (at least I think so) because it's patches that _everyone_ can take from upstream as well. You don't need to make them special, and it's always possible to generate some kind of flat file to say, okay, I cherry-picked this patch this patch and this patch from upstream, or merged up to this point of this upstream branch. This information is more useful than the patch series for derived distros, or co-maints. When it comes to specific patches of yours, I really believe that topic branches like you advertise them are the best answer. Git makes merging easy (s/Git/reasonnable $DSCM/ for this matter btw) in the sense that merging is fast enough, and easy enough when the branches you merge have not diverged too much. I mean, no matter which SCM you use, merging from a branch that is _very_ old, and still not merged upstream is jut a pain. And it's again not an SCM issue. A patch queue _is_ a branch in itself. Really. There are two ways to look at that. Either you say, I always want to remember I started from this point, and then you merge and merge and merge, and your history looks like that: R are uptream releases, M your repeated merges to keep the feature branch current. -o---o---o--[...]--R--[...]--R--[...]--o-- \\ \ p--p--p--p---M-M...[feature branch] Well with this approach, upstream will have to take a messy history with a _lot_ of merge points they don't care about, and won't be able to try your feature branch on top of their current work and maybe eventually adopt it. And worse, if you have to add new patches along the way, you get an history with a mixed suite of patches and merges, which is unreadable to upstream. The other way is to forget about giving depth *in* the SCM to the patches history. Because it's what it's about. What you really want IMSHO is: I have this patch queue [pq] and at upstream R0 it was in state pq0, in upstream R1 it was in state pq1 and so on. Without any useless merge points in it. This way your feature branch is a free (as in only attached to history by its base) branch that you rewrite for each new upstream, serialize under debian/patches/featurebranch. In git, there is this awesome git-quiltimport command that allow you to rebuild a branch from a quilt series in a trivial way. If you want to work on the patch queue, you just need to make it a branch again, do your stuff, serialize it again, and you're done. While doing that, your workflow allow people to do meaningful changes to your package (by adding patches to a given queue), that you'll transparently *painlessly* import into your workflow. Whereas with your current one, you'll have to extract whatever the NMUer did that is a flat debdiff, and split it. It's horrible for you, don't please pretend otherwise, I won't believe you. The other gain, is that upstream can look at a current, unencumbered patch queue about the feature you added, and can take a decent decision about the fact that it's good to take upstream or not, and it's trivial to export such a branch to upstream:
Re: How to cope with patches sanely
On Mon, 25 Feb 2008 09:35:13 +0100, martin f krafft [EMAIL PROTECTED] said: also sprach Manoj Srivastava [EMAIL PROTECTED] [2008.02.25.0828 +0100]: I am not opposed to it. If you can somehow magically create a tool that can linearize the feature branches, more power to you. I personally find the prospect highly unlikely; and I would like to see some code, please, before I grant that such a thing is possible. The tool I envision would simply surf through the history of the integration branch and identify merge commits. Each merge would become a patch in the quilt series. How are you planning on doing this identification? Looking at the arch logs, it is not trivial to identify merge commits and the upgrade patches (which are just merges from the upstream branch), unless you start with an ancient version (like, from my 2003 repo) and then apply every single commit to the integration branch over the last five years (with really really huge numbers of patches). You'll have to track repo changes, figure out how to overcome sealed branch boundaries, etc. I am not sure I believe this to be feasible until I see some code. Sure. You can't integrate two features that fundamentally conflict with each other. No amount of smoke and mirrors can obscure that fundamental obstacle. This is independent of the tool set you use. Except that quilt provides the necessary glue to handle it, while feature branches don't. No, it does not. If branch A has pi = 2.34567; and branch B has pi = 3.14159; No matter how much quilting you do you cannot reconcile the fundamental conflict in the final. Either pi is 3.14159; or it is not; and if branch A requires pi not to be that value, and branch B requires pi to be that value, quilt can't make C be quantum like and have the value be both. manoj -- Griffin's Thought: When you starve with a tiger, the tiger starves last. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Mon, 25 Feb 2008 10:33:48 +0100, Pierre Habouzit [EMAIL PROTECTED] said: On Mon, Feb 25, 2008 at 03:37:07AM +, Manoj Srivastava wrote: On Sun, 24 Feb 2008 21:17:10 -0500, David Nusinow [EMAIL PROTECTED] said: On Sun, Feb 24, 2008 at 06:08:17PM -0800, Russ Allbery wrote: David Nusinow [EMAIL PROTECTED] writes: The problem is that you and Manoj assume that this is the only way to do things. I don't believe this. Pierre Habouzit has been experimenting with an alternative method of feature branches that exports to a linear stack of diffs just fine. Just because Manoj is doing something one way right now doesn't mean it's the only or even the correct way to do it. I would be interested in details of this, and whether this approach works with pure feature branches where the features are being developed contemporaneously with each other an upstream development; and thus the branches overlap both temporally and in code space. I'm planning to write a textual version of what I demonstrated at FOSDEM, with some more ideas that I had talking with Julien Cristau on the grass after. You developped them contemporaneously, okay, but in the end you merge them one after the other. No, I do not. I developed feature A a bit, merged that. Then I developed feature B a bit, and merged that. Then I developed feature A some more. The there came an upstream version. Then feature B ... If you're doing criss-cross merges, well, I can nothing for you, and you're creating really messy histories, and yes, you need an SCM to represent that in a satisfying way. Thanks. So most of my packages will not get any help from the tool you are talking about -- and thus, it can't be made into a policy requirement. But if you really merge one feature branch on top of the other, and it's in my experience *way enough* for what we need in Debian packaging, then multiple branches are just multiple series to be applied in a specific relative order. But that is not how development happens in long running sets of features, which have been under development, incrementally, over a large number of upstream versions. When it comes to specific patches of yours, I really believe that topic branches like you advertise them are the best answer. Git makes merging easy (s/Git/reasonnable $DSCM/ for this matter btw) in the sense that merging is fast enough, and easy enough when the branches you merge have not diverged too much. I mean, no matter which SCM you use, merging from a branch that is _very_ old, and still not merged upstream is jut a pain. Depends. I keep the topic branches updated with each upstream release, and I have carried fvwm/make/flex patches around for years and several upstream upgrades, and not had much problems. Indeed, most upstream upgrades have taken _no_ manual effort. And it's again not an SCM issue. A patch queue _is_ a branch in itself. Really. There are two ways to look at that. Either you say, I always want to remember I started from this point, and then you merge and merge and merge, and your history looks like that: R are uptream releases, M your repeated merges to keep the feature branch current. -o---o---o--[...]--R--[...]--R--[...]--o-- \ \ \ p--p--p--p---M-M...[feature branch] Right. Well with this approach, upstream will have to take a messy history with a _lot_ of merge points they don't care about, and won't be able to try your feature branch on top of their current work and maybe eventually adopt it. And worse, if you have to add new patches along the way, you get an history with a mixed suite of patches and merges, which is unreadable to upstream. Heh. Most of my upstream are fed just a patch, since lots of them are using CVS. In Git terms, I always rebase my patch on each upstream version, and can then feed a nice, coherent, minimal patch with no real complex history. The other way is to forget about giving depth *in* the SCM to the patches history. Because it's what it's about. What you really want IMSHO is: I have this patch queue [pq] and at upstream R0 it was in state pq0, in upstream R1 it was in state pq1 and so on. Without any useless merge points in it. This way your feature branch is a free (as in only attached to history by its base) branch that you rewrite for each new upstream, serialize under debian/patches/featurebranch. In git, there is this awesome git-quiltimport command that allow you to rebuild a branch from a quilt series in a trivial way. If you want to work on the patch queue, you just need to make it a branch again, do your stuff, serialize it again, and you're done. Err, I see little benefit in doing that; and I think that I prefer my current feature branch mechanism as being less hassle. I periodically build and test each feature branch (this is why having ./debian as a submodule is
Re: How to cope with patches sanely
On 25/02/2008, Pierre Habouzit wrote: I'm planning to write a textual version of what I demonstrated at FOSDEM, with some more ideas that I had talking with Julien Cristau on the grass after. Please, pretty please, include graphics. Be it ASCII art-like drawings, or gitk screenshots, with some additional arrows and other comments. Not that I need it, but that would (have been|be) nice. Cheers, -- Cyril Brulebois pgpITYEH0ADJu.pgp Description: PGP signature
Re: How to cope with patches sanely
On Mon, Feb 25, 2008 at 04:06:31PM +, Cyril Brulebois wrote: On 25/02/2008, Pierre Habouzit wrote: I'm planning to write a textual version of what I demonstrated at FOSDEM, with some more ideas that I had talking with Julien Cristau on the grass after. Please, pretty please, include graphics. Be it ASCII art-like drawings, or gitk screenshots, with some additional arrows and other comments. Not that I need it, but that would (have been|be) nice. That's exactly the plan. -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpvfCOA8DotC.pgp Description: PGP signature
Re: How to cope with patches sanely
On Mon, Feb 25, 2008 at 03:56:49AM -0600, Manoj Srivastava wrote: No, it does not. If branch A has pi = 2.34567; and branch B has pi = 3.14159; No matter how much quilting you do you cannot reconcile the fundamental conflict in the final. Either pi is 3.14159; or it is not; and if branch A requires pi not to be that value, and branch B requires pi to be that value, quilt can't make C be quantum like and have the value be both. Feature branches don't magically allow you to avoid merge conflicts either, so this is a red herring. Once you've resolved the conflict, then it becomes just another change. This change can become a diff in a stack of diffs. - David Nusinow -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sun, 24 Feb 2008, Charles Plessy wrote: Therefore we did not make progress since the beginning of the discussion: You're trying to make progress somewhere where it's not expected. - The most efficient way to deal with changes to the sources for the packager is to use his preferred tools. That should stay as it is. - We do not know if the whole concept of breaking up the monolithic diff into logical units is something that the persons who often do NMUs in Debian are intersted in. Seperating logical changes has always been useful to understand the changes and is thus required for any independant reviewer who hasn't access to the VCS where the changes are reviewable separately. - When modifying a package that uses dpatch, quilt or simple-patchsys, developpers have to find out by themselves if the target for patching the sources is patch, apply-patches or apply-dpatches. Once the new dpkg-source format is in standard use, those rules disappear completely... because they are no more needed during build. Frank Lichtenheld and myself have started hacking on dpkg-source with the goal to support an extend wigpen and possibly even more. I'd encourage interested parties to subscribe to debian-dpkg to follow discussions there. http://lists.debian.org/debian-dpkg/2008/02/msg00012.html http://lists.debian.org/debian-dpkg/2008/02/msg00036.html We're working in the sourcev3 branch of dpkg's git repo. We've only doing some ground work for now, but we've been doing good progress and I expect the work on new features to start pretty soon. http://git.debian.org/?p=dpkg/dpkg.git;a=shortlog;h=sourcev3 (Subscribe to the PTS with cvs keyword and you'll get git commit notice in live :-)) Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely -- Debian New Maintainers'
On Sat, Feb 09, 2008 at 10:53:55AM +0100, Patrick Schoenfeld wrote: (BTW. there is no need to CC me with your answers, I did not ask for that as I am subscribed to the list :-) On Sat, Feb 09, 2008 at 12:50:46AM +0100, Pierre Habouzit wrote: quilt is way more powerful to refresh patches when a new upstream occurs. It does what it can do best without having mergeing features, that only an advanced SCM can provide anyways. That does mean quilt is able to refresh patches on upstream changes, so that with luck the maintainer does not have to refresh the patches for changes sources himself? That would be quiet nice, however I still fail to see why this is a reason to prefer quilts *format* as an *exchange format* if quilt itself is not to be used, which is what you say. Who said quilt itself is not to be used? The point of the discussion is that we agree on a patch-exchange format, without forcing an implementation on people. People who like the quilt implementation will still be able and encouraged to use quilt with it. Others might just use vi to edit quilt patches, or git, or something else. Michael -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
Le Sun, Feb 24, 2008 at 10:47:05AM +0100, Raphael Hertzog a écrit : - When modifying a package that uses dpatch, quilt or simple-patchsys, developpers have to find out by themselves if the target for patching the sources is patch, apply-patches or apply-dpatches. Once the new dpkg-source format is in standard use, those rules disappear completely... because they are no more needed during build. I must have missed something. I thought that the new format was to keep the debian directory in a tar.gz format. With this format, people who want to modify upstream sources will have to use a patch system. What is the plan to make the patch targets in debian/rules unneeded? Have a nice day (and thank you for your work on a new format for source pacakages.) -- Charles Plessy http://charles.plessy.org Wakō, Saitama, Japan -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sat, 2008-02-23 at 09:08:23 -0600, Manoj Srivastava wrote: On Sat, 23 Feb 2008 08:46:03 -0500, David Nusinow [EMAIL PROTECTED] said: This argument assumes that dpkg-source -x will apply that patch stack automatically as well, which has been discussed elsewhere. Currently vapourware, no? I want to clarify this, as I've seen it mentioned several times on this thread. dpkg-source supports extracting WigPen since etch. The missing bit is source package generation (I posted a proof of concept script to convert quilted source packages to debian-dpkg [0]). regards, guillem [0] http://lists.debian.org/debian-dpkg/2008/02/msg00079.html -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sat, 23 Feb 2008 11:20:55 -0500, David Nusinow [EMAIL PROTECTED] said: On Sat, Feb 23, 2008 at 09:08:23AM -0600, Manoj Srivastava wrote: Now, you are trying to make me go towards a mechanism I think is inferior (a liner, dependent, and in my opinion, opaque, and somewhat shaky linear patch series) as opposed to pure, independent feature branches, for benefits I still think are dubious, so expect serious push back. Especially if for every upload I'll have to redo all the integration work of the past; ain't gonna happen. I am not trying to convince you of the error of your ways. Please desist trying to convince me of mine. No one, not me, nor anyone else in this thread that I've seen, is saying that you should abandon your sytem. What we want is a consistent method of dealing with differences from upstream. Currently, we have one giant .diff.gz, which people hack around either with patch systems or vcs's. If we switch to something other than a monolithic .diff.gz on top of a .orig.tar.gz, then we win because currently we just have a lot of chaos. So far, this is OK. The devil, though, lies in the details. No matter what you want to say about your feature branches, you *must* apply them in a linear fashion to your final source tree that you ship in the package. This is no way around it. But there is no such linearization, not in the way that quilt et al do it. The state of such integration is not maintained in the feature branches; it is in the history of the integration branch. As each feature branch was created or developed, if there were any conflicts, these conflicts were resolved in the integration branch. And the integration branch does not keep track of what changes come from which branch -- that is not its job. And it does not keep the integration patches up to date either -- that again is not its job. Details below. There are only Two use cases I care about. A) Feed each pure feature to upstream as an up to date patch against current upstream -- this is what the feature branch does. B) Have an integrated source of all features ready to compile for Debian -- and easy to keep updated with lates revisions. This is the task of the integration branch. This idea, a linear stack of changes one on top of the other, is the commonality between every one of the different means of handling changes to upstream, be they maintained by a patch system or a vcs. What we want is to stanardize this concept so that our general tools like dpkg are aware of it. This is where we start to differ. More on this below. Given what I know about your system, the only change you would have to make is that instead of integrating directly in a final branch, you generate a diff and throw that along with the diff's name in to debian/patches. I do not think you have thought it through. Let us try seeing how this will not work: Say I have have feature branches A-F. When development happens on branch A that requires some conflict resolution on the integration branch, I do the resolution, and fix up the integration branch. The fix applied depends on what order the features were merged in -- and this is not something I need to keep track of, so I do not. As more development happens, and non-overlapping changes in the same area occur, the patch would no longer apply. Other feature branches may make changes in the area; and again, make the old patch obsolete. The a new upstream version comes along, and more changes happen. By this time, the old patch will not apply even to upstream. Now rinse and repeat -- the patch you blithely threw into ./debian/patches is just dead bit by now. That's it. It could be totally automated from your current setup without much work, if that's what you want. I don't think this can be automated. However, if you think you can solve the problem, I'll stand down my objections. Love to see it demonstrated first. Indeed, the only way to redo the information in the integration branch is to start with the original upstream, from several years ago, determine which feature branch was merged in what order, what order any improvements were applied, redo the conflict resolution patches, redo the upstream version updates in the correct order, and get to the current integration branch. Heh. Good luck. This is not making you give up anything, it's about improving the base standard so everyone can get more information without learning a zillion different patch systems and vcs's. As a result, we can focus on improving the code itself instead of the process of managing the code. So, in summary, stop trying to act like you're being forced to do something that's going to hurt you. No one is taking away your toys. Words. Lets us see some code, eh? manoj -- When a man knows no this shore, other shore, or both - such a one, free from anxiety,
Re: How to cope with patches sanely
Manoj Srivastava [EMAIL PROTECTED] writes: David Nusinow [EMAIL PROTECTED] said: No matter what you want to say about your feature branches, you *must* apply them in a linear fashion to your final source tree that you ship in the package. This is no way around it. But there is no such linearization, not in the way that quilt et al do it. The state of such integration is not maintained in the feature branches; it is in the history of the integration branch. Is this (the integration branch and its history of changes) not the linear sequence of changes that David Nusinow is asking for? And the integration branch does not keep track of what changes come from which branch -- that is not its job. Doesn't each commit message in the integration branch's history state what merge you were performing at each revision? You've previously described your workflow as one where you carefully integrate each feature branch separately into the integration branch. Do your commit messages in the integration branch not state what individual feature branch you're merging in? It seems to me that the analogue to a linear sequence of patches is the revision history of your integration branch. Granted, this doesn't give patches against a pristine upstream except from some initial state. -- \ Some mornings, it's just not worth chewing through the leather | `\ straps. -- Emo Philips | _o__) | Ben Finney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
Ben Finney [EMAIL PROTECTED] writes: Manoj Srivastava [EMAIL PROTECTED] writes: But there is no such linearization, not in the way that quilt et al do it. The state of such integration is not maintained in the feature branches; it is in the history of the integration branch. Is this (the integration branch and its history of changes) not the linear sequence of changes that David Nusinow is asking for? Yes, but you don't, in the general case, completely develop some feature on a branch and then merge it only once. You do some work on one branch, merge it, do some work on another branch, merge it, do more work on the first branch and merge it again, import a new upstream and merge it into all of your branches, do some more work on the feature and merge it again I can see Manoj's point. It's not at all clear to me that there's a useful linearization of the feature branches after that sort of workflow has continued for a while. (I'm now maintaining two of my packages using only Git and feature branches without any patch system so that I can get some practical experience with this and understand the workflow better.) -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sun, Feb 24, 2008 at 05:10:16PM -0800, Russ Allbery wrote: Ben Finney [EMAIL PROTECTED] writes: Manoj Srivastava [EMAIL PROTECTED] writes: But there is no such linearization, not in the way that quilt et al do it. The state of such integration is not maintained in the feature branches; it is in the history of the integration branch. Is this (the integration branch and its history of changes) not the linear sequence of changes that David Nusinow is asking for? Yes, but you don't, in the general case, completely develop some feature on a branch and then merge it only once. You do some work on one branch, merge it, do some work on another branch, merge it, do more work on the first branch and merge it again, import a new upstream and merge it into all of your branches, do some more work on the feature and merge it again I can see Manoj's point. It's not at all clear to me that there's a useful linearization of the feature branches after that sort of workflow has continued for a while. (I'm now maintaining two of my packages using only Git and feature branches without any patch system so that I can get some practical experience with this and understand the workflow better.) The problem is that you and Manoj assume that this is the only way to do things. I don't believe this. Pierre Habouzit has been experimenting with an alternative method of feature branches that exports to a linear stack of diffs just fine. Just because Manoj is doing something one way right now doesn't mean it's the only or even the correct way to do it. - David Nusinow -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
David Nusinow [EMAIL PROTECTED] writes: The problem is that you and Manoj assume that this is the only way to do things. I don't believe this. Pierre Habouzit has been experimenting with an alternative method of feature branches that exports to a linear stack of diffs just fine. Just because Manoj is doing something one way right now doesn't mean it's the only or even the correct way to do it. Well, I definitely don't think it's the only way to do things, and I've been one of the people arguing in favor of quilt and exporting to a set of patches. :) But the native Git workflow that people have previously written up for Debian packages doesn't seem to me to linearize very easily, and IMO one of the points here was to let maintainers keep using their native workflows and use the package format for interchange. Changing the workflow to allow easier export to a particular package format seems to be going the wrong direction to me. In other words, I still think a patch-based package format is a good idea and would be very valuable for a lot of what's in Debian, but I have to agree with Manoj's point, based on what I've seen so far, that converting an arbitrary Git or Arch repository for Debian package maintenance to such a package format isn't necessarily easy. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sun, Feb 24, 2008 at 06:08:17PM -0800, Russ Allbery wrote: David Nusinow [EMAIL PROTECTED] writes: The problem is that you and Manoj assume that this is the only way to do things. I don't believe this. Pierre Habouzit has been experimenting with an alternative method of feature branches that exports to a linear stack of diffs just fine. Just because Manoj is doing something one way right now doesn't mean it's the only or even the correct way to do it. Well, I definitely don't think it's the only way to do things, and I've been one of the people arguing in favor of quilt and exporting to a set of patches. :) But the native Git workflow that people have previously written up for Debian packages doesn't seem to me to linearize very easily, and IMO one of the points here was to let maintainers keep using their native workflows and use the package format for interchange. Changing the workflow to allow easier export to a particular package format seems to be going the wrong direction to me. In other words, I still think a patch-based package format is a good idea and would be very valuable for a lot of what's in Debian, but I have to agree with Manoj's point, based on what I've seen so far, that converting an arbitrary Git or Arch repository for Debian package maintenance to such a package format isn't necessarily easy. Ok, that's fair. In the worst case then people who want to use this sort of workflow could stick everything in a giant diff like we do now, so nothing would be lost. - David Nusinow -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Mon, 25 Feb 2008 10:34:55 +1100, Ben Finney [EMAIL PROTECTED] said: Manoj Srivastava [EMAIL PROTECTED] writes: David Nusinow [EMAIL PROTECTED] said: No matter what you want to say about your feature branches, you *must* apply them in a linear fashion to your final source tree that you ship in the package. This is no way around it. But there is no such linearization, not in the way that quilt et al do it. The state of such integration is not maintained in the feature branches; it is in the history of the integration branch. Is this (the integration branch and its history of changes) not the linear sequence of changes that David Nusinow is asking for? No, it is not. I Apply a update to feature A. The comes an upstream update. Then updates on feature B, a patch that needed conflict resoution, then patches on branches C, D, and A again. Another upstream change. At this point, none of the original patches to A, B, and C apply any more -- and then come another upstream update, and all the patches get even more bent out of shape. And the integration branch does not keep track of what changes come from which branch -- that is not its job. Doesn't each commit message in the integration branch's history state what merge you were performing at each revision? It usually states what changes were made, not necessarily what feature branch I imported from. You've previously described your workflow as one where you carefully integrate each feature branch separately into the integration branch. But not in order, since not all features are developed on the same time scale, or even in sequence. And so no, all feature branches do not get integrated nicely in separate chunks and for the same upstream version either. Do your commit messages in the integration branch not state what individual feature branch you're merging in? Not usually. I describe the changes as it affects the integration branch, and sometimes I mention which branch it came from. It seems to me that the analogue to a linear sequence of patches is the revision history of your integration branch. Granted, this doesn't give patches against a pristine upstream except from some initial state. It does not even apply to the current version of upstream. If a feature branch was developed over the course of a dozen or so upstream versions, and intertwined with development on other feature branches, the integration branch might give the sequence of merges, but will not give a patch set that applies to any given upstream version, and you'll have to retrace the exact sequence of merges and upstream updates -- in other words, playing back the whole history of the package. For make, for instance, the history stretches back over a decade. manoj -- I never vote for anyone. I always vote against. W.C. Fields Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sun, 24 Feb 2008 21:17:10 -0500, David Nusinow [EMAIL PROTECTED] said: On Sun, Feb 24, 2008 at 06:08:17PM -0800, Russ Allbery wrote: David Nusinow [EMAIL PROTECTED] writes: The problem is that you and Manoj assume that this is the only way to do things. I don't believe this. Pierre Habouzit has been experimenting with an alternative method of feature branches that exports to a linear stack of diffs just fine. Just because Manoj is doing something one way right now doesn't mean it's the only or even the correct way to do it. I would be interested in details of this, and whether this approach works with pure feature branches where the features are being developed contemporaneously with each other an upstream development; and thus the branches overlap both temporally and in code space. Well, I definitely don't think it's the only way to do things, and I've been one of the people arguing in favor of quilt and exporting to a set of patches. :) But the native Git workflow that people have previously written up for Debian packages doesn't seem to me to linearize very easily, and IMO one of the points here was to let maintainers keep using their native workflows and use the package format for interchange. Changing the workflow to allow easier export to a particular package format seems to be going the wrong direction to me. In other words, I still think a patch-based package format is a good idea and would be very valuable for a lot of what's in Debian, but I have to agree with Manoj's point, based on what I've seen so far, that converting an arbitrary Git or Arch repository for Debian package maintenance to such a package format isn't necessarily easy. Ok, that's fair. In the worst case then people who want to use this sort of workflow could stick everything in a giant diff like we do now, so nothing would be lost. Or have dpkg understand not just quilt, but git. I mean, if we are making dpkg understand quilt-as-a-version-control system, why not also have it grok a modern SCM like git? (I know that trying to get it to understand arch is a lost cause). Would joeyh's efforts to get dpkg v3 format be a got repo make a difference here? manoj -- Freedom begins when you tell Mrs. Grundy to go fly a kite. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
Hi, I think one of the differences that patch series mechanism has wrt to new development, either in a feature or upstream, is that it requires updates to the integration work doe every single upload. It also requires a strict ordering between each feature, making it harder to compile and test each feature independently. It is nice that quilt helps keep the integration patch up to date; but this is extra work I am glad not to have to do. An integration branch does the integration once, and does not bother to explicitly keep the integration patch updated for every upload -- but it does keep each feature branch up to date, and the integration branch has a current set of features. This is reduced work load for each change; at the expense of stashing integration knowledge deep in the entrails of the integration branch. I like the reduced work for each upload, and since it satisfies the use cases of being able to present upstream with a pure feature changeset; to be able to rapidly compare any feature branch against each other or upstream, and being able to maintain an integrated source tree for Debian packages, I am happy not to keep integration patches up to date. Now, if there is indeed a mechanism that can automatically keep up the integration patches and convert feature branches to an integrated source tree via a patch series, I'll gladly eat crow and try to convert over. But I still want to keep pure feature branches, and I do not want to have to worry about integration all over again at every upload. Unless new information comes up on this thread, I am done. manoj -- Reinhart was never his mother's favorite -- and he was an only child. Thomas Berger Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
also sprach Manoj Srivastava [EMAIL PROTECTED] [2008.02.22.1627 +0100]: I am not sure you have understood feature branches. They are independent, no matter what the overlap. Each feature branch tracks one feature against upstream, no matter how the other features work. The overlap management is done in the integration branch. This is significantly different from a quilt series, as others have already mentioned in this thread,which is a dependent series of linearized patches; and a change in an earlier feature impacts all subsequent patches (and quilt is good at automating the handling of that impact). [[duplicated so this can be archived on the vcs-package mailing list as well]] well, I know what feature branches are but I suppose I jumped the gun. They are independent until you integrate them. But you will integrate them, thus I tend to think of them as interdependent from the start. Anyway, we're not talking about developing with quilt. We are talking about converting feature branches to quilt patches for the sake of representation in the source package. I don't see why you would oppose to that at all, to be honest. Or the patch manager does integration. *Shrug*. Someone has to do integration sometime. That is the nature of the beast. The trick is to do it once, and never have to think about it again. ... except when one feature needs to change and then conflicts with another feature on re-integration. B) Development happens on the feature branch. [...] 2) Development on one of the branches conflicts with one or more other features. Here, the human has to figure out the difference on the integration branch -- once. No, every time you do development on the branch. And every time, you have to remember which branch dependended/conflicted on the feature branch you're currently working on, or figure it out by trial and error. I don't consider this efficient. This is work that a computer should be doing. -- .''`. martin f. krafft [EMAIL PROTECTED] : :' : proud Debian developer, author, administrator, and user `. `'` http://people.debian.org/~madduck - http://debiansystem.info `- Debian - when you have better things to do than fixing systems windoze 98: n. useless extension to a minor patch release for 32-bit extensions and a graphical shell for a 16-bit patch to an 8-bit operating system originally coded for a 4-bit microprocessor, written by a 2-bit company that can't stand for 1 bit of competition. digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/)
Re: How to cope with patches sanely
On Sun, 24 Feb 2008 11:04:21 +0100, martin f krafft [EMAIL PROTECTED] said: also sprach Manoj Srivastava [EMAIL PROTECTED] [2008.02.22.1627 +0100]: I am not sure you have understood feature branches. They are independent, no matter what the overlap. Each feature branch tracks one feature against upstream, no matter how the other features work. The overlap management is done in the integration branch. This is significantly different from a quilt series, as others have already mentioned in this thread,which is a dependent series of linearized patches; and a change in an earlier feature impacts all subsequent patches (and quilt is good at automating the handling of that impact). [[duplicated so this can be archived on the vcs-package mailing list as well]] well, I know what feature branches are but I suppose I jumped the gun. They are independent until you integrate them. But you will integrate them, thus I tend to think of them as interdependent from the start. You have a strange definition of interdependent. The majority of my feature branches are really orthogonal; seldom on integration there is no conflict; so it is hard for me to think of them as somehow inter dependent. Sure, there are overlapping changes, sometimes, but these are mostly one time resolution issues. Anyway, we're not talking about developing with quilt. We are talking about converting feature branches to quilt patches for the sake of representation in the source package. I don't see why you would oppose to that at all, to be honest. I am not opposed to it. If you can somehow magically create a tool that can linearize the feature branches, more power to you. I personally find the prospect highly unlikely; and I would like to see some code, please, before I grant that such a thing is possible. Or the patch manager does integration. *Shrug*. Someone has to do integration sometime. That is the nature of the beast. The trick is to do it once, and never have to think about it again. ... except when one feature needs to change and then conflicts with another feature on re-integration. Sure. You can't integrate two features that fundamentally conflict with each other. No amount of smoke and mirrors can obscure that fundamental obstacle. This is independent of the tool set you use. B) Development happens on the feature branch. [...] 2) Development on one of the branches conflicts with one or more other features. Here, the human has to figure out the difference on the integration branch -- once. No, every time you do development on the branch. And every time, you have to remember which branch dependended/conflicted on the feature branch you're currently working on, or figure it out by trial and error. I don't consider this efficient. This is work that a computer should be doing. Strange. In all my years of using feature branches, I have never had to consider which branch depended on what. So no, I don't think I _have_ to remember any such thing; trust me, I would have noticed. I have 30+ packages, and have been using feature branches since early this decade. If you have a whole slew of features that depend on other features, then your feature set is very different from ones I have encountered. Dependent features do require more work; but not as much, in my opinion, as using quilt or dpatch; but your mileage may vary. For the most part, I just develop on each branch independently. I compile, test, hack, compile, on each individual featre branch. And never ever worry about what the other feature branches are like, while doing so. Once in a blue moon (more or less) I have to deconflict the integration branch; but mostly those are swiftly resolved issues. And once resoved, I tend to forget about them too. All this worrying about branch conflicts seem to be more the realm of quilt and other patch series mechanisms; which is mostly my reason to dislike them. manoj -- (It is an old Debian tradition to leave at least twice a year ...) Sven Rudolph Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sun, 24 Feb 2008, Charles Plessy wrote: Le Sun, Feb 24, 2008 at 10:47:05AM +0100, Raphael Hertzog a écrit : - When modifying a package that uses dpatch, quilt or simple-patchsys, developpers have to find out by themselves if the target for patching the sources is patch, apply-patches or apply-dpatches. Once the new dpkg-source format is in standard use, those rules disappear completely... because they are no more needed during build. I must have missed something. I thought that the new format was to keep the debian directory in a tar.gz format. With this format, people who want to modify upstream sources will have to use a patch system. What is the plan to make the patch targets in debian/rules unneeded? If the patch are applied by default, there's no need to apply them again at build time. Then quilt/dpatch tools will only be used by the packager to modify/updated its patch serie during maintenance but the tool won't be needed during recompilation (and thus doesn't need to be in Build-Depends). Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
Hi, On Sun, 24 Feb 2008, Manoj Srivastava wrote: I like the reduced work for each upload, and since it satisfies the use cases of being able to present upstream with a pure feature changeset; It doesn't satisfy it completely. You can always generate a patch for a pure feature changeset but you can't guarantee that all those feature patches apply at the same time. Let's say you have 5 feature patches, the upstream maintainer wants to integrate 3 of them. How can you submit 3 patches that would apply one after the other? You have to redo some of your integration work that you already did. That said, even with a pure quilt set of patches, you can't guarantee it either. If one of the wanted patch has some overlap with one of the non-wanted ones... All in all, I think we do all exagerate the problems. In my experience, most of the patches applied to a Debian package are relatively independant of each other and inter-relationships is the exception, not the rule. But if we can come up with a solution that handles perfectly inter-dependant patches, that would still be great. :-) Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Fri, Feb 22, 2008 at 09:37:24AM -0600, Manoj Srivastava wrote: On Thu, 21 Feb 2008 16:20:49 +0100, martin f krafft [EMAIL PROTECTED] said: That does not help me during an NMU from the source package. For an NMU of one of my source packages, if you can't deal with the distributed SCM, then you need not worry about differentiating the feature branches, fix the source conflicts, upload. I'll deal with fallout. Comes with the territory. If you're applying 10 to 20 different feature branches to your upstream, then that all comes to the NMU'er as one giant diff. This obviously sucks and it's what we've been complaining about Ubuntu doing to us for years. We can do better. If you want to use your special custom system then that's fine, but I think that ultimately shipping things as a linear patch stack in the package makes a lot of sense because you can provide the NMU'er with the ability to conceptually see the differences in your branches without having to learn your DVCS. That way you can still use whatever custom system you want to manage your packages, but at the same time you present your packages to others in a way that makes it easier for them to work on. This argument assumes that dpkg-source -x will apply that patch stack automatically as well, which has been discussed elsewhere. - David Nusinow -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sat, 23 Feb 2008 08:46:03 -0500, David Nusinow [EMAIL PROTECTED] said: On Fri, Feb 22, 2008 at 09:37:24AM -0600, Manoj Srivastava wrote: On Thu, 21 Feb 2008 16:20:49 +0100, martin f krafft [EMAIL PROTECTED] said: That does not help me during an NMU from the source package. For an NMU of one of my source packages, if you can't deal with the distributed SCM, then you need not worry about differentiating the feature branches, fix the source conflicts, upload. I'll deal with fallout. Comes with the territory. If you're applying 10 to 20 different feature branches to your upstream, then that all comes to the NMU'er as one giant diff. This obviously sucks and it's what we've been complaining about Ubuntu doing to us for years. We can do better. I have hear you say this before, but I am not convinced that the situations are really similar. You see, with Ubuntu, I do not see any place they provide the information in a nice separate area, even using their preferred SCM, bzr. That is not the case when using featrure branches, the NMUer can get the information they need. Now, we should also consider the type of NMU, and the need for being able to tell the distinction between different threads of development. As a maintainer, where you are juggling user needs for features, and trying to add features, and so on, you do need to deal with large changes, perhaps touching a large number of files. Here, it do need to tell the lines of development apart, and ensure that all the lines of development are indivdually undamaged. For this kind of NMU, you need to be familiar with the package, and yes, you need to be able to keep features apart. Quilt is suboptimal here, since it can't keep features independent; since all feaures are impacted by all other features prior to it in the linear chain, but I guess it performs well enough. Nothing beats a pure featrure ranch in keeping the features independent. But if you are a security NMUer, you have a short time frame, and a very targeted fix to install, and you do not have to keep track of independent features, and I believe not having a patched source to deal with immediately inder such a use case. The feature branch wins again, since dpkg-source -x gives you the sources that are used to build from, no special shennanigans needed. If you want to use your special custom system then that's fine, but I think that ultimately shipping things as a linear patch stack in the package makes a lot of sense because you can provide the NMU'er with the ability to conceptually see the differences in your branches without having to learn your DVCS. That way you can still use whatever custom system you want to manage your packages, but at the same time you present your packages to others in a way that makes it easier for them to work on. In this day and age a DSCM is not a mere custom system. If you want to hack on modern packages, you need to be aware of modern tools. Being a Luddite is not something I would encourage. Now, you are trying to make me go towards a mechanism I think is inferior (a liner, dependent, and in my opinion, opaque, and somewhat shaky linear patch series) as opposed to pure, independent feature branches, for benefits I still think are dubious, so expect serious push back. Especially if for every upload I'll have to redo all the integration work of the past; ain't gonna happen. I am not trying to convince you of the error of your ways. Please desist trying to convince me of mine. This argument assumes that dpkg-source -x will apply that patch stack automatically as well, which has been discussed elsewhere. Currently vapourware, no? manoj -- You will be singled out for promotion in your work. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sat, Feb 23, 2008 at 03:08:23PM +, Manoj Srivastava wrote: On Sat, 23 Feb 2008 08:46:03 -0500, David Nusinow [EMAIL PROTECTED] said: On Fri, Feb 22, 2008 at 09:37:24AM -0600, Manoj Srivastava wrote: On Thu, 21 Feb 2008 16:20:49 +0100, martin f krafft [EMAIL PROTECTED] said: That does not help me during an NMU from the source package. For an NMU of one of my source packages, if you can't deal with the distributed SCM, then you need not worry about differentiating the feature branches, fix the source conflicts, upload. I'll deal with fallout. Comes with the territory. If you're applying 10 to 20 different feature branches to your upstream, then that all comes to the NMU'er as one giant diff. This obviously sucks and it's what we've been complaining about Ubuntu doing to us for years. We can do better. I have hear you say this before, but I am not convinced that the situations are really similar. You see, with Ubuntu, I do not see any place they provide the information in a nice separate area, even using their preferred SCM, bzr. That is not the case when using featrure branches, the NMUer can get the information they need. Not really, he needs to grok your $SCM, and well, really, nobody but you (and his author maybe, and even that is unsure) grok baz. And not even everybody groks git that I'm so fond of. Having a clean exchange format where _anyone_ groking diff, ls, $EDITOR, their shell and some as basic tools is much much better. Nobody asks you to _work_ with those, it would be just plain great if you just could generate those instead of a big bloat of unreadable diff (maybe your packages don't have megabytes of diffs, but as soon as you relibtoolize a package, they do). IOW, I'm totally seconding David here. -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpq4FLHOBX2L.pgp Description: PGP signature
Re: How to cope with patches sanely
On Sat, Feb 23, 2008 at 09:08:23AM -0600, Manoj Srivastava wrote: Now, you are trying to make me go towards a mechanism I think is inferior (a liner, dependent, and in my opinion, opaque, and somewhat shaky linear patch series) as opposed to pure, independent feature branches, for benefits I still think are dubious, so expect serious push back. Especially if for every upload I'll have to redo all the integration work of the past; ain't gonna happen. I am not trying to convince you of the error of your ways. Please desist trying to convince me of mine. No one, not me, nor anyone else in this thread that I've seen, is saying that you should abandon your sytem. What we want is a consistent method of dealing with differences from upstream. Currently, we have one giant .diff.gz, which people hack around either with patch systems or vcs's. If we switch to something other than a monolithic .diff.gz on top of a .orig.tar.gz, then we win because currently we just have a lot of chaos. No matter what you want to say about your feature branches, you *must* apply them in a linear fashion to your final source tree that you ship in the package. This is no way around it. This idea, a linear stack of changes one on top of the other, is the commonality between every one of the different means of handling changes to upstream, be they maintained by a patch system or a vcs. What we want is to stanardize this concept so that our general tools like dpkg are aware of it. Given what I know about your system, the only change you would have to make is that instead of integrating directly in a final branch, you generate a diff and throw that along with the diff's name in to debian/patches. That's it. It could be totally automated from your current setup without much work, if that's what you want. This is not making you give up anything, it's about improving the base standard so everyone can get more information without learning a zillion different patch systems and vcs's. As a result, we can focus on improving the code itself instead of the process of managing the code. So, in summary, stop trying to act like you're being forced to do something that's going to hurt you. No one is taking away your toys. This argument assumes that dpkg-source -x will apply that patch stack automatically as well, which has been discussed elsewhere. Currently vapourware, no? This thread is partially about how to implement this very feature, so calling it vapor while it's still being planned is unfair. - David Nusinow -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
Le Sat, Feb 23, 2008 at 09:08:23AM -0600, Manoj Srivastava a écrit : That is not the case when using featrure branches, the NMUer can get the information they need. But if you are a security NMUer, you have a short time frame, Hi all, actually, we as packagers have wrote a lot of statements a lot in the name of NMUers and the security team, but in fact independant opinions of NMUers have not been read often in this thread, nor from the security team about what they expect from source packages. Therefore we can all be plain wrong, as we have no evidence that NMUers want what we are arguing for: stack of patches and/or link to a fully version-controlled source tree. The only thing that seems to make a consensus is that when people have to modify a package that they have not created, and that it uses a patch management system, they waste time trying to figure out what is the debian/rules target to patch the source tree. Changing this would need to modify two or three packages out of dpatch, quilt, and cdbs, and unfortunately only the maintainers of quilt package have expressed interest in a standardisation. Therefore we did not make progress since the beginning of the discussion: - The most efficient way to deal with changes to the sources for the packager is to use his preferred tools. - We do not know if the whole concept of breaking up the monolithic diff into logical units is something that the persons who often do NMUs in Debian are intersted in. - When modifying a package that uses dpatch, quilt or simple-patchsys, developpers have to find out by themselves if the target for patching the sources is patch, apply-patches or apply-dpatches. Have a nice day, -- Charles Plessy http://charles.plessy.org Wakō, Saitama, Japan -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Wed, 20 Feb 2008 23:09:03 +0100, sean finney [EMAIL PROTECTED] said: i think this discussion is in part to propose a system that can serve as a layer of abstraction between $developer's $scm implementation and some common format/methodology, so that when $otherdeveloper comes along and needs to muck with the package (say, for a security NMU), they don't need to have such knowledge, apart from fakeroot debian/rules patch, or possibly even just dpkg-source --patch -x $dscfile if a common source format could be established for build-time patching tools. A security team already gets that for feature branches in a DSCM bit -- they just need to dpkg-source -x; apply their patch, build, and go. They don't really need to understand which of the lines in the file belongs to feature A, and which belongs to feature B. So creating a quilt series from feature branches is not something that is being done to address the NMUer or security team uploader needs, as far as I can see. It is not like quilt provides easy means to compare different feature branches -- let alone how the current feature branch C compares to the second from last revisions of feature branch F. i think that would be: $vcs get 2nd from last rev of quilt_patch_F tmpfile interdiff quilt_patch_C tmpfile Not really, since quilt patch F depends on quilt patch D and E being applied, so you are comparing feature C with featrures D, E, And F. Not what was asked for. See, the feature branches are really different from quilt series, since they are pure features, each independently tracked against upstream, and are independent of each other. The integration branch is the only one where you need to be worried about featuer overlap. I can still compare each feature branch against each other, without being worried about _other_ features, because my system tracks each feature independently. A quilt series is, on the other hand, a dependent series of linearized patches; and a change in an earlier feature impacts all subsequent patches (and quilt is good at automating the handling of that impact). manoj -- He who is good for making excuses is seldom good for anything else. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Thu, 21 Feb 2008 16:20:49 +0100, martin f krafft [EMAIL PROTECTED] said: also sprach Manoj Srivastava [EMAIL PROTECTED] [2008.02.20.1722 +0100]: I have to take care of it manually once. That is the first time I setup the integration branch that merges in changes from the overlapping feature branches. This is not a big deal, because the human has to spend some time disambiguating the overlap. It is critical to me that I never have to spend that time again -- and in my case, I never do, since future non overlapping changes (like, say, a new upstream release) produce deltas that apply to the feature and integration branches seamlessly. I never have to rethink or redo any patches. What if feature B depends on feature A Interesting. I have never used feature branches where one feature was dependent on another feature; feature branches are usually Pure features. I'll probably end up as creating feature branches A and A+B. and a new upstream causes a merge conflict in feature A. Then you have to fix feature A, which causes a conflict with feature B. Now you have to redo the entire integration, taking care to merge in the right order... The latter part is not true; you just pull into the delta the change from feature branch A+B; you'll still have to manually intervene twice, once in the A branch, once in the A+B branch, and the latter delta goes into the integration branch. Yes, just like I want to have feature branches instead of one gigantic debian branch. I use my CSM to provide me the changeset: baz diff branch A upstream That does not help me during an NMU from the source package. For an NMU of one of my source packages, if you can't deal with the distributed SCM, then you need not worry about differentiating the feature branches, fix the source conflicts, upload. I'll deal with fallout. Comes with the territory. manoj -- Sic Transit Gloria Thursdi. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Thu, 21 Feb 2008 16:23:10 +0100, martin f krafft [EMAIL PROTECTED] said: also sprach James Vega [EMAIL PROTECTED] [2008.02.21.0020 +0100]: The difference here being that feature branches are, in my experience, changes against the pristine upstream source. The merging of different feature branches is done in some integration branch. Quilt patches are a dependent series where the merging of changes is inherent in the patch ordering. Thus it's easier to get an upstream ready patch from $vcs than from a series of interdependent patches. ... unless feature branches interdepend and you have to store dependency information somewhere. I am not sure you have understood feature branches. They are independent, no matter what the overlap. Each feature branch tracks one feature against upstream, no matter how the other features work. The overlap management is done in the integration branch. This is significantly different from a quilt series, as others have already mentioned in this thread,which is a dependent series of linearized patches; and a change in an earlier feature impacts all subsequent patches (and quilt is good at automating the handling of that impact). [[duplicated so this can be archived on the vcs-package mailing list as well]] Congratulations, you have just reinvented $patch_manager. Or the patch manager does integration. *Shrug*. Someone has to do integration sometime. That is the nature of the beast. The trick is to do it once, and never have to think about it again. What are the most common cases? A) There is a new upstream delta. In this case, 1) either there is no conflict in any feature branch, in which case the upstream delta applies to each feature branch and integration branch with no manual thought required. Indeed, I have a script that does this case for me with one command line invocation: arch_upgrade. 2) The upstream change conflicts with a feature. A human can then disambiguate that feature branch, and the same change then is applied to the integration branch. Only once did I need to think about it. 3) If more than one feature was affected, then I'll need to intervene once for each feature affected, and possibly once more for the integration branch. Again, I think this is the minimal disambiguation that is needed. B) Development happens on the feature branch. 1) There is no conflict with any other feature. The delta on the feature branch is then applied mindlessly to the integration branch. 2) Development on one of the branches conflicts with one or more other features. Here, the human has to figure out the difference on the integration branch -- once. So, in my experience, A1 is the most common case, followed by B1, A2, B2, and A3. The only time I have to even thing about doing something manual is A2, A3 and B2 -- and in those cases, human intervention can not be eliminated, no matter what tools you use. My point is that I never have to even think about different features or patch series except in cases where human intervention is indispensable anyway. I like that. I am lazy. manoj -- Reliable source, n.: The guy you just met. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
also sprach Manoj Srivastava [EMAIL PROTECTED] [2008.02.20.1722 +0100]: I have to take care of it manually once. That is the first time I setup the integration branch that merges in changes from the overlapping feature branches. This is not a big deal, because the human has to spend some time disambiguating the overlap. It is critical to me that I never have to spend that time again -- and in my case, I never do, since future non overlapping changes (like, say, a new upstream release) produce deltas that apply to the feature and integration branches seamlessly. I never have to rethink or redo any patches. What if feature B depends on feature A and a new upstream causes a merge conflict in feature A. Then you have to fix feature A, which causes a conflict with feature B. Now you have to redo the entire integration, taking care to merge in the right order... Yes, just like I want to have feature branches instead of one gigantic debian branch. I use my CSM to provide me the changeset: baz diff branch A upstream That does not help me during an NMU from the source package. -- .''`. martin f. krafft [EMAIL PROTECTED] : :' : proud Debian developer, author, administrator, and user `. `'` http://people.debian.org/~madduck - http://debiansystem.info `- Debian - when you have better things to do than fixing systems work consists of whatever a body is obliged to do. play consists of whatever a body is not obliged to do. -- mark twain digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/)
Re: How to cope with patches sanely
On Thu, Feb 21, 2008 at 04:23:10PM +0100, martin f krafft wrote: also sprach James Vega [EMAIL PROTECTED] [2008.02.21.0020 +0100]: The difference here being that feature branches are, in my experience, changes against the pristine upstream source. The merging of different feature branches is done in some integration branch. Quilt patches are a dependent series where the merging of changes is inherent in the patch ordering. Thus it's easier to get an upstream ready patch from $vcs than from a series of interdependent patches. ... unless feature branches interdepend and you have to store dependency information somewhere. Each feature is still a separate candidate for inclusion upstream. If you have features A and B, which touch similar files and are therefore interdependent *in your tree*, the patches sent upstream still need to be a diff against their vanilla upstream source. Either you maintain the patches purely against vanilla upstream initially and perform your own merging when you prepare the Debian package or you maintain dependent patches and rediff against upstream's vanilla source before sending the patch their way. Whether using $vcs or $patch_manager, there is going to be some manual work to a) get a patch that is purely against vanilla upstream and/or b) rediff B when A is accepted upstream. You're just changing when you do the work. James -- GPG Key: 1024D/61326D40 2003-09-02 James Vega [EMAIL PROTECTED] signature.asc Description: Digital signature
Re: How to cope with patches sanely
also sprach James Vega [EMAIL PROTECTED] [2008.02.21.0020 +0100]: The difference here being that feature branches are, in my experience, changes against the pristine upstream source. The merging of different feature branches is done in some integration branch. Quilt patches are a dependent series where the merging of changes is inherent in the patch ordering. Thus it's easier to get an upstream ready patch from $vcs than from a series of interdependent patches. ... unless feature branches interdepend and you have to store dependency information somewhere. Congratulations, you have just reinvented $patch_manager. http://lists.debian.org/debian-devel/2008/02/msg00093.html -- .''`. martin f. krafft [EMAIL PROTECTED] : :' : proud Debian developer, author, administrator, and user `. `'` http://people.debian.org/~madduck - http://debiansystem.info `- Debian - when you have better things to do than fixing systems violence is the last refuge of the incompetent -- isaac asimov digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/)
Re: How to cope with patches sanely
also sprach Manoj Srivastava [EMAIL PROTECTED] [2008.02.17.0029 +0100]: But sometimes you will have to touch an integration branch and then things get messy, especially if there are dependencies between feature branches. I think David is making a very strong point here... No. You never, ever touch an integration branch. If you do, you are correct in assuming the mouth of hell shall open and spew forth the demons. The integration branch is only for pulling in changes from the feature branches; this is why we have a debian feature branch. I meant to say feature branch. You hack in the sloppy branch. You merge change sets from the sloppy branch into the feature branches. You merge the delta from the feature branch into the integration branch. In my experience, there is usually no new mess -- since you are only merging the delta in the integration branch; the step you took earlier to deal with the mess usually still apply. So what about dependencies between feature branches? Aren't you just taking extra care to do it right manually, when in fact the system should take care of it for you? But I may want to see the differences from upstream nicely separated into patches, rather than whole chunks? You mean you do not want to see the differences between a feature branch and upstream as one chunk, but nicely segregated in smaller pieces? Yes, just like I want to have feature branches instead of one gigantic debian branch. Not really, unless your feature branches are poorly chosen. Each feature branch, in my packages, represent onlogical feature that belongs in the same patch (series). Right, but given five feature branches, once you create the diff.gz, there are no more patch*es* or series anymore, there's just one huge file with everything. Why should we lose the logical separation between features you have in the VCS when creating the source package. -- .''`. martin f. krafft [EMAIL PROTECTED] : :' : proud Debian developer, author, administrator, and user `. `'` http://people.debian.org/~madduck - http://debiansystem.info `- Debian - when you have better things to do than fixing systems i doubt larry wall ever uses strict. -- frederick heckel digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/)
Re: How to cope with patches sanely
On Wed, 20 Feb 2008 11:17:29 +0100, martin f krafft [EMAIL PROTECTED] said: You hack in the sloppy branch. You merge change sets from the sloppy branch into the feature branches. You merge the delta from the feature branch into the integration branch. In my experience, there is usually no new mess -- since you are only merging the delta in the integration branch; the step you took earlier to deal with the mess usually still apply. So what about dependencies between feature branches? Aren't you just taking extra care to do it right manually, when in fact the system should take care of it for you? I have to take care of it manually once. That is the first time I setup the integration branch that merges in changes from the overlapping feature branches. This is not a big deal, because the human has to spend some time disambiguating the overlap. It is critical to me that I never have to spend that time again -- and in my case, I never do, since future non overlapping changes (like, say, a new upstream release) produce deltas that apply to the feature and integration branches seamlessly. I never have to rethink or redo any patches. cd upstream-working-dir tla_load_dir ../new-upstream-branch cd .. arch_upgrade and then update the changelog, build, test, piu-parts tests, other tests, upload, etc. But I may want to see the differences from upstream nicely separated into patches, rather than whole chunks? You mean you do not want to see the differences between a feature branch and upstream as one chunk, but nicely segregated in smaller pieces? Yes, just like I want to have feature branches instead of one gigantic debian branch. I use my CSM to provide me the changeset: baz diff branch A upstream Indeed, I can get diffs between branch A and branch B -- how do you do that using quilt? Get diffs between arbitrary branches? Trivial using my scheme. Not really, unless your feature branches are poorly chosen. Each feature branch, in my packages, represent onlogical feature that belongs in the same patch (series). Right, but given five feature branches, once you create the diff.gz, there are no more patch*es* or series anymore, there's just one huge file with everything. Why should we lose the logical separation between features you have in the VCS when creating the source package. Not every need needs be satisfied in the source package. If I ship the metadata for arch in the source package, and there is the arch grab file that is in the control file, someone who wants to see the history, or how feature branch A stacks up to feature branch C, can do so using the SCM stuff. It is not like quilt provides easy means to compare different feature branches -- let alone how the current feature branch C compares to the second from last revisions of feature branch F. I find comparing feature branches to each other as useful as comparing them to upstream, personally. manoj -- Welcome to the Zoo! Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Wednesday 20 February 2008 05:22:08 pm Manoj Srivastava wrote: Yes, just like I want to have feature branches instead of one gigantic debian branch. I use my CSM to provide me the changeset: baz diff branch A upstream Indeed, I can get diffs between branch A and branch B -- how do you do that using quilt? Get diffs between arbitrary branches? Trivial using my scheme. we discussed this on irc, but for posterity i'll say it here too: the you could think of each individual quilt patch as a feature diff, thus the quilt equivalent of a feature branch would be the (ideally) pristine source plus the diff in question. so you have explicitly the comparison between the feature branch and the source by opening the quilt patch in a pager. to compare the feature branches with each other, you could just do something like interdiff patch1 patch2. Not every need needs be satisfied in the source package. If I ship the metadata for arch in the source package, and there is the arch grab file that is in the control file, someone who wants to see the history, or how feature branch A stacks up to feature branch C, can do so using the SCM stuff. i think this discussion is in part to propose a system that can serve as a layer of abstraction between $developer's $scm implementation and some common format/methodology, so that when $otherdeveloper comes along and needs to muck with the package (say, for a security NMU), they don't need to have such knowledge, apart from fakeroot debian/rules patch, or possibly even just dpkg-source --patch -x $dscfile if a common source format could be established for build-time patching tools. It is not like quilt provides easy means to compare different feature branches -- let alone how the current feature branch C compares to the second from last revisions of feature branch F. i think that would be: $vcs get 2nd from last rev of quilt_patch_F tmpfile interdiff quilt_patch_C tmpfile sean signature.asc Description: This is a digitally signed message part.
Re: How to cope with patches sanely
On Wed, Feb 20, 2008 at 11:09:03PM +0100, sean finney wrote: On Wednesday 20 February 2008 05:22:08 pm Manoj Srivastava wrote: Yes, just like I want to have feature branches instead of one gigantic debian branch. I use my CSM to provide me the changeset: baz diff branch A upstream Indeed, I can get diffs between branch A and branch B -- how do you do that using quilt? Get diffs between arbitrary branches? Trivial using my scheme. we discussed this on irc, but for posterity i'll say it here too: the you could think of each individual quilt patch as a feature diff, thus the quilt equivalent of a feature branch would be the (ideally) pristine source plus the diff in question. so you have explicitly the comparison between the feature branch and the source by opening the quilt patch in a pager. to compare the feature branches with each other, you could just do something like interdiff patch1 patch2. The difference here being that feature branches are, in my experience, changes against the pristine upstream source. The merging of different feature branches is done in some integration branch. Quilt patches are a dependent series where the merging of changes is inherent in the patch ordering. Thus it's easier to get an upstream ready patch from $vcs than from a series of interdependent patches. James -- GPG Key: 1024D/61326D40 2003-09-02 James Vega [EMAIL PROTECTED] signature.asc Description: Digital signature
Re: How to cope with patches sanely
Sorry for the late reply. I'll send only a single reply this time since my flood reply seemed to annoy some people. I still think short single replies are better than long, unified ones... also sprach Pierre Habouzit [EMAIL PROTECTED] [2008.02.03.2038 +1100]: I'm less and less sure that a git-based format is a brilliant idea. I like git more than a lot, but it's a poor idea to base source packages on them. Why? Because it's a way to high level tool for the task. Git (and I suppose that the following is true for other DVCS, and if not, they _REALLY_ suck, and I mean it, really) has been designed so that you can only exchange patches. Many people actually tend to think of Git as a filesystem... IOW, all that you should need to grok what is in a source package should basically be tar/ar/cpio/... and vi. Are you sure that this has to be the case in 10 years? Are you sure git will be there and backward compatible in 10 years ? Okay, it's likely given the current upstream, that focuses a _lot_ on backward compatibility. But still. Some repository formats have been deprecated (the object store is of version3 and IIRC git doesn't grok version1 anymore, but I may be mistaken). That's the kind of gamble I wouldn't take. Whereas I guess tar will always grok tarballs it generates today in 10 years, and gzip/bzip2/lzma/$compressor won't change either, and text is text for enough time to assume it'll remain as editeable in 10 years. Fair enough, but why would e.g. version3 ever be removed? It's not like versioned storage formats need a whole lot more support than the infrastructure to identify them, which is already in place. Nice, I didn't see this before. This *is* in fact nice and puts the quilt *format* very high on my list. Yes, quilt preserves *everything* you put in the header, so if you use git (or $scm) to generate the patches with authoring information, commit message and so on, it wont be lost. Excellent. What I ask you is just to be consistent. Either we _will_ modify source packages, either we won't. If we will, adding features to it is a good idea, if we won't, let's just focus on how to let it be expressive enough to encode in it all what we use as new features upstream from it. And as a matter of a fact, quilt is enough for that. Good arguments. Thanks! also sprach David Nusinow [EMAIL PROTECTED] [2008.02.03.1154 +1100]: - patch/feature-branch-specific history. Say feature branch 'foo' has a bug, so you check it out and work on it again... now you're suddenly in the context of all the work you've previously done and you can trivially browse the history of changes applicable only to what you're currently working on. How do you decide the order to apply your feature branches? Do you encode it in the branch name? Congratulations, you've just reinvented dbs. Do you have a separate file in the debian directory that tells you the merge order? Congratulations, you've just reimplemented quilt. Do you write a script to update your branches automatically, only failing when you have to manually update yourself? Congratulations, you've just reinvented every other patch system in existence. Good points, thanks David. also sprach Manoj Srivastava [EMAIL PROTECTED] [2008.02.05.1751 +1100]: On Sat, 26 Jan 2008 11:34:27 -0500, David Nusinow [EMAIL PROTECTED] said: [...] An alternate idea I keep seeing is feature branches. I have absolutely no idea why these are considered preferable to most people. In the case of the X server we regularly carry around 30 patches on our stack. If we were to put each of these on its own unique feature branch, merging them becomes an exercise in pain. Sure, we could implement our own custom scripts and have very strict naming policies, but then you've re-invented the wheel and congratulations, you have yet another custom piece of crap software that you have to maintain on top of the already complicated one you actually care about. Additionally, you have a lot more branches getting in the way when you're trying to figure out which one you're looking for. In my experience, once I have created the integration branch, most upstream versions require little to none re-merging; I just apply the delta to each of the feature branches, _and_ the integration branch. Very rarely do I have to fix up a feature branch, and then apply a second delta with that fix up to the integration branch; but it has not been, for me, painful at all, nor do I have to jump through hoops every single time to accommodate new upstream. But sometimes you will have to touch an integration branch and then things get messy, especially if there are dependencies between feature branches. I think David is making a very strong point here... also sprach Ben Finney [EMAIL PROTECTED] [2008.02.07.2242 +1100]: On Thu, Feb 07, 2008 at 05:12:00AM
Re: How to cope with patches sanely
On Sat, Feb 16, 2008 at 09:24:12AM +, martin f krafft wrote: Because it's a way to high level tool for the task. Git (and I suppose that the following is true for other DVCS, and if not, they _REALLY_ suck, and I mean it, really) has been designed so that you can only exchange patches. Many people actually tend to think of Git as a filesystem... I do too, it's just that if you know where you come from, all you need to rebuild the same state are the patches. Most of the kernel and git developments are done this way. For the kernel not even every major contributor uses git (akpm didn't last time I checked) and their exchange format is patch series, completely bijective with quilt format. And given their current tremendous patch integration rate, I _really_ think this format has proven its robustness, while being good enough for git. Are you sure git will be there and backward compatible in 10 years ? Okay, it's likely given the current upstream, that focuses a _lot_ on backward compatibility. But still. Some repository formats have been deprecated (the object store is of version3 and IIRC git doesn't grok version1 anymore, but I may be mistaken). That's the kind of gamble I wouldn't take. Whereas I guess tar will always grok tarballs it generates today in 10 years, and gzip/bzip2/lzma/$compressor won't change either, and text is text for enough time to assume it'll remain as editeable in 10 years. Fair enough, but why would e.g. version3 ever be removed? It's not like versioned storage formats need a whole lot more support than the infrastructure to identify them, which is already in place. Well, you still don't know if upstream will become mad in the next 10 years, can you ? :) And git code isn't trivial, so I don't think you'll be able to hack version3 back into it if it's stripped from upstream at some point. Nice, I didn't see this before. This *is* in fact nice and puts the quilt *format* very high on my list. Yes, quilt preserves *everything* you put in the header, so if you use git (or $scm) to generate the patches with authoring information, commit message and so on, it wont be lost. Excellent. In fact afaict, git-quiltimport is able to get authoring informations and so on, and if it's not able to use embedded blob sha's yet, it's probably not hard to build from there. As a matter of a fact, I'm experimenting right now with the llvm2 packaging a way to share our patch branch (with a co-maint) through debian/patches, and so far, it doesn't work so badly. IOW, what the xorg people do for months actually, except that I use git all over the place. [ and to be fair we export patches through git-format-patch so there isn't a series file, but the series file would basically built this way: (ls -1 * 2/dev/null || :) series ] What I ask you is just to be consistent. Either we _will_ modify source packages, either we won't. If we will, adding features to it is a good idea, if we won't, let's just focus on how to let it be expressive enough to encode in it all what we use as new features upstream from it. And as a matter of a fact, quilt is enough for that. Good arguments. Thanks! :) With Extremadura coming up, who would be interested in joining a workgroup to work on this stuff? I would volunteer to chair such a session... I would be very much interested, but I'm not really sure that I'd have the time to join a session. Note that my talk at FOSDEM could be also a place to discuss some points too on the subject (If I get the time to write the damn slides *cough*) :) -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpWT7YscntFp.pgp Description: PGP signature
Re: How to cope with patches sanely
On Sat, 16 Feb 2008 20:24:12 +1100, martin f krafft [EMAIL PROTECTED] said: also sprach Manoj Srivastava [EMAIL PROTECTED] [2008.02.05.1751 +1100]: On Sat, 26 Jan 2008 11:34:27 -0500, David Nusinow [EMAIL PROTECTED] said: [...] An alternate idea I keep seeing is feature branches. I have absolutely no idea why these are considered preferable to most people. In the case of the X server we regularly carry around 30 patches on our stack. If we were to put each of these on its own unique feature branch, merging them becomes an exercise in pain. Sure, we could implement our own custom scripts and have very strict naming policies, but then you've re-invented the wheel and congratulations, you have yet another custom piece of crap software that you have to maintain on top of the already complicated one you actually care about. Additionally, you have a lot more branches getting in the way when you're trying to figure out which one you're looking for. In my experience, once I have created the integration branch, most upstream versions require little to none re-merging; I just apply the delta to each of the feature branches, _and_ the integration branch. Very rarely do I have to fix up a feature branch, and then apply a second delta with that fix up to the integration branch; but it has not been, for me, painful at all, nor do I have to jump through hoops every single time to accommodate new upstream. But sometimes you will have to touch an integration branch and then things get messy, especially if there are dependencies between feature branches. I think David is making a very strong point here... No. You never, ever touch an integration branch. If you do, you are correct in assuming the mouth of hell shall open and spew forth the demons. The integration branch is only for pulling in changes from the feature branches; this is why we have a debian feature branch. You hack in the sloppy branch. You merge change sets from the sloppy branch into the feature branches. You merge the delta from the feature branch into the integration branch. In my experience, there is usually no new mess -- since you are only merging the delta in the integration branch; the step you took earlier to deal with the mess usually still apply. If you are speculating there will be unforeseen and unmentionable problems, all I can say is that I have yet to experience them, and not all my packages are trivial packaging of upstream versions. also sprach Ben Finney [EMAIL PROTECTED] [2008.02.07.2242 +1100]: On Thu, Feb 07, 2008 at 05:12:00AM +, Manoj Srivastava wrote: That's not really about bring your feature branches into a patch system, but rather export them into something that looks like one so that the security team or the NMUer can have a good outlook of the modifications you apply to upstream. In the scenario Manoj presents above, the modifications applied to upstream are easily available all in one place: the foo.diff.gz. But I may want to see the differences from upstream nicely separated into patches, rather than whole chunks? You mean you do not want to see the differences between a feature branch and upstream as one chunk, but nicely segregated in smaller pieces? This is the same argument why 32k diffs provided on patches.ubuntu.com are useless to Debian maintainers. Not really, unless your feature branches are poorly chosen. Each feature branch, in my packages, represent onlogical feature that belongs in the same patch (series). manoj -- Mountain Dew and doughnuts... because breakfast is the most important meal of the day. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
Manoj Srivastava [EMAIL PROTECTED] writes: On Thu, 7 Feb 2008 11:48:47 +, Matthew Johnson [EMAIL PROTECTED] said: On Thu Feb 07 22:42, Ben Finney wrote: In the scenario Manoj presents above, the modifications applied to upstream are easily available all in one place: the foo.diff.gz. But all as one patch, not as your nice separated list of commits and/or branches. True. We now have to evaluate the benefits of providing sources that the binary packages are built from with no fuss (dpkg -x); which can then be inspected and patched. Well, this whole thread was really about how to have our cake and eat it too, so I don't think we should assume that we can't provide the sources with no muss and not *also* use a patch system. That's what many of us were trying to work towards with the discussion of how quilt would fit into wigpen. What is the use case this effort is designed to address? I have not actually heard NMU/porters express a need for converting monolithic patches to patch series. Have I missed the need statements? It's to let those of us who think of a patch as a useful unit of development instead of a branch continue to develop Debian packages using the methodology that we feel makes us the most productive without causing extra problems for the security team and NMUs. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/
Re: How to cope with patches sanely
On Thu, 7 Feb 2008 11:48:47 +, Matthew Johnson [EMAIL PROTECTED] said: On Thu Feb 07 22:42, Ben Finney wrote: In the scenario Manoj presents above, the modifications applied to upstream are easily available all in one place: the foo.diff.gz. But all as one patch, not as your nice separated list of commits and/or branches. True. We now have to evaluate the benefits of providing sources that the binary packages are built from with no fuss (dpkg -x); which can then be inspected and patched. How often do people performing NMU's have to figure out the various threads of development which are merged into the integration branch (which is what is uploaded to Debian)? It is not as if the threads of development are not available; and people actively helping to develop the Debian package could just learn the DCVS tool -- but that is not the use case I thought we were considering. This patch format can then be created from DVCSen if you like that, quilt if you like that or diff/patch if you like that. It still has separate, commented patches, but doesn't require the security team to know '*every* patch bundle system' or to know how to deal with a DVCS. What is the use case this effort is designed to address? I have not actually heard NMU/porters express a need for converting monolithic patches to patch series. Have I missed the need statements? manoj -- All Bibles are man-made. Thomas Edison Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Fri, 08 Feb 2008 09:45:06 +0100, Pierre Habouzit [EMAIL PROTECTED] said: On Thu, Feb 07, 2008 at 11:42:16AM +, Ben Finney wrote: Pierre Habouzit [EMAIL PROTECTED] writes: On Thu, Feb 07, 2008 at 05:12:00AM +, Manoj Srivastava wrote: Why should I bring my feature branches into a patch system, when there is no need to? As far as the end user or NMUer is ocnerned, they do apt-get source foo, and they get the sources they may hack on. Adding to the chaos by converting my nice, clean source format to the blecherousness of a patch system does seem like regression to me.. That's not really about bring your feature branches into a patch system, but rather export them into something that looks like one so that the security team or the NMUer can have a good outlook of the modifications you apply to upstream. In the scenario Manoj presents above, the modifications applied to upstream are easily available all in one place: the foo.diff.gz. Yeah, in the very same way ubuntu gives patches to debian[0], in a big fat unreadable blurb without comments or explanations. However, with my proposal, you do have alternate methods of getting to the threads of development, indeoendenntly of each other, in a manner that allows you to individually build org + thread1 separately from orig + thread2 -- not something that quilt allows, if I understand it. I am also not convinced that the threads of development need to be all in the source package, as opposed to the DCVS. However, a number of people do think that dpkg-source -x providing the sources that are used to build the binary package are important. You didn't read carefully it seems, I'm advocating using _THE_ (not a, the sole only one format) quilt format to split the .diff.gz in something readable and understandable by a mere mortal. For all I care, I think wig pen was supposed to support multiple .diff.gz, well, maybe it should just support the .diff.gz being a .quilt.tar.gz. _that_'s *all* I say. I am not convinced of the practicality of this proposed solution (actually, I am skeptical of all the models proposed so far.) manoj -- Kime's Law for the Reward of Meekness: Turning the other cheek merely ensures two bruised cheeks. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely -- Debian New Maintainers'
Hi Pierre, (BTW. there is no need to CC me with your answers, I did not ask for that as I am subscribed to the list :-) On Sat, Feb 09, 2008 at 12:50:46AM +0100, Pierre Habouzit wrote: quilt is way more powerful to refresh patches when a new upstream occurs. It does what it can do best without having mergeing features, that only an advanced SCM can provide anyways. That does mean quilt is able to refresh patches on upstream changes, so that with luck the maintainer does not have to refresh the patches for changes sources himself? That would be quiet nice, however I still fail to see why this is a reason to prefer quilts *format* as an *exchange format* if quilt itself is not to be used, which is what you say. And while being powerful, it remains simple, which dpatch is not. dpatch is not simple? When refreshing patches or overall? As a dpatch user I cannot really confirm this. Different to that I tried to work into quilt these days and it seems to be way more complicated (and needs some setup before working for debian packaging at all, which dpatch does not need). Best Regards, Patrick -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Thu, Feb 07, 2008 at 11:42:16AM +, Ben Finney wrote: Pierre Habouzit [EMAIL PROTECTED] writes: On Thu, Feb 07, 2008 at 05:12:00AM +, Manoj Srivastava wrote: Why should I bring my feature branches into a patch system, when there is no need to? As far as the end user or NMUer is ocnerned, they do apt-get source foo, and they get the sources they may hack on. Adding to the chaos by converting my nice, clean source format to the blecherousness of a patch system does seem like regression to me.. That's not really about bring your feature branches into a patch system, but rather export them into something that looks like one so that the security team or the NMUer can have a good outlook of the modifications you apply to upstream. In the scenario Manoj presents above, the modifications applied to upstream are easily available all in one place: the foo.diff.gz. Yeah, in the very same way ubuntu gives patches to debian[0], in a big fat unreadable blurb without comments or explanations. It sounds like a fair thing to me. You can't decently ask the security team to know every single $SCM out ther, and guess how $random_developer use it (especially, sorry but it's true, when it's a SCM as complicated as tla is). In the scenario Manoj presents above, no-one in the security team needs to know how to do anything but get the source package, since Neiter in mine. Whereas with a patch bundle system, the security team needs to know *every* patch bundle system in use and how to use it, just to have a chance of operating on arbitrary Debian source packages. You didn't read carefully it seems, I'm advocating using _THE_ (not a, the sole only one format) quilt format to split the .diff.gz in something readable and understandable by a mere mortal. For all I care, I think wig pen was supposed to support multiple .diff.gz, well, maybe it should just support the .diff.gz being a .quilt.tar.gz. _that_'s *all* I say. [0] a troll ? where ? :P -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpy1W63Gb3tA.pgp Description: PGP signature
Re: How to cope with patches sanely -- Debian New Maintainers'
Hi Matt, On Wed, Feb 06, 2008 at 11:25:19AM +, Matthew Johnson wrote: I'd have said that it would be more sensible to define a reasonable subset of quilt features. A set of patches with comments at the top and a yes, this sounds reasonable. But I'm not a quilt user and therefore don't know which features it support that may be useful for others too. But editting patches in a prepared environment seems to be common to all patch systems, so at least I have anything I need ;-) Can't speak for others, though. series file can be manipulated by many tools, isn't really quilt-specific. Indeed. After all the series file is just a text file. Allowing obscure quilt-only features wasn't what I'd understood from the discussion. Neither did I. I just thought that it could have some features that make it so much better then dpatch, which could actually make up the reason why everybody is crying damn dpatch, glorify quilt. So now everybody says that they are *not* even talking about the tool, but instead about the quilt *format* as an interchange format. That doesn't appear to be logical, because there seems not to be a big difference between the two exchange formats but indeed between the tools. I was just wondering and I am still. Regards, Patrick -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely -- Debian New Maintainers'
On Fri, Feb 08, 2008 at 09:50:20AM +, Patrick Schoenfeld wrote: Neither did I. I just thought that it could have some features that make it so much better then dpatch, which could actually make up the reason why everybody is crying damn dpatch, glorify quilt. So now everybody says that they are *not* even talking about the tool, but instead about the quilt *format* as an interchange format. That doesn't appear to be logical, because there seems not to be a big difference between the two exchange formats but indeed between the tools. I was just wondering and I am still. quilt is way more powerful to refresh patches when a new upstream occurs. It does what it can do best without having mergeing features, that only an advanced SCM can provide anyways. And while being powerful, it remains simple, which dpatch is not. -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpirwZrhlppK.pgp Description: PGP signature
Re: How to cope with patches sanely
On Thu, Feb 07, 2008 at 05:12:00AM +, Manoj Srivastava wrote: On Tue, 5 Feb 2008 09:00:52 +, Matthew Johnson [EMAIL PROTECTED] said: On Tue Feb 05 00:51, Manoj Srivastava wrote: If we can't figure out a good and clean way to keep a large stack of long-lived patches in the vcs then I firmly believe we should standardize on quilt. I think I have indeed solved the issue of long standing feature sets using feature branches, integration branches, and sloppy branches while upgrading, and would not want to be forced to regress to a patch system. I don't think anyone is talking about forcing DVCS users to regress to a patch system, merely to change the interchange format; which all DVCS-based maintenance methods can easily export to/import from. The only reason which you would have to interact with it would be a more standard interface for NMUs, which can only be a good thing. Why should I bring my feature branches into a patch system, when there is no need to? As far as the end user or NMUer is ocnerned, they do apt-get source foo, and they get the sources they may hack on. Adding to the chaos by converting my nice, clean source format to the blecherousness of a patch system does seem like regression to me. That's not really about bring your feature branches into a patch system, but rather export them into something that looks like one so that the security team or the NMUer can have a good outlook of the modifications you apply to upstream. It sounds like a fair thing to me. You can't decently ask the security team to know every single $SCM out ther, and guess how $random_developer use it (especially, sorry but it's true, when it's a SCM as complicated as tla is). -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgp4RScm9eFKg.pgp Description: PGP signature
Re: How to cope with patches sanely
On Thu Feb 07 22:42, Ben Finney wrote: In the scenario Manoj presents above, the modifications applied to upstream are easily available all in one place: the foo.diff.gz. But all as one patch, not as your nice separated list of commits and/or branches. Whereas with a patch bundle system, the security team needs to know *every* patch bundle system in use and how to use it, just to have a chance of operating on arbitrary Debian source packages. That's not what is being suggested (although it is the current state). If there is one standard patch _format_ then they need only learn one set of commands (which may be as simple as diff and patch, or may be quilt). This patch format can then be created from DVCSen if you like that, quilt if you like that or diff/patch if you like that. It still has separate, commented patches, but doesn't require the security team to know '*every* patch bundle system' or to know how to deal with a DVCS. Matt -- Matthew Johnson signature.asc Description: Digital signature
Re: How to cope with patches sanely
Pierre Habouzit [EMAIL PROTECTED] writes: On Thu, Feb 07, 2008 at 05:12:00AM +, Manoj Srivastava wrote: Why should I bring my feature branches into a patch system, when there is no need to? As far as the end user or NMUer is ocnerned, they do apt-get source foo, and they get the sources they may hack on. Adding to the chaos by converting my nice, clean source format to the blecherousness of a patch system does seem like regression to me. That's not really about bring your feature branches into a patch system, but rather export them into something that looks like one so that the security team or the NMUer can have a good outlook of the modifications you apply to upstream. In the scenario Manoj presents above, the modifications applied to upstream are easily available all in one place: the foo.diff.gz. It sounds like a fair thing to me. You can't decently ask the security team to know every single $SCM out ther, and guess how $random_developer use it (especially, sorry but it's true, when it's a SCM as complicated as tla is). In the scenario Manoj presents above, no-one in the security team needs to know how to do anything but get the source package, since that gets the complete original source and a foo.diff.gz showing exactly what changes are applied. With 'debcheckout' even knowing what VCS is used will be unnecessary. Whereas with a patch bundle system, the security team needs to know *every* patch bundle system in use and how to use it, just to have a chance of operating on arbitrary Debian source packages. -- \ “It is far better to grasp the universe as it really is than to | `\ persist in delusion, however satisfying and reassuring.” | _o__) —Carl Sagan | Ben Finney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sat, Feb 02, 2008 at 10:26:52PM +0100, Pierre Habouzit wrote: Bah that's the worst reason I ever seen to not use a DVCS in Debian. Please look at the debian packages with patches, and please tell me how many come with a comment about what the patch do. Please start with the glibc if you don't know where to start. Then please resume the discussion about how having patches series make them more commented. Well, we sort of managed to get our debian/changelog entries more readable due to social pressure, if we standardize on a patch exchange format which mandates a description in the header, we can probably continously improve the description of patches as well. Michael -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
Michael Banck [EMAIL PROTECTED] writes: Well, we sort of managed to get our debian/changelog entries more readable due to social pressure, if we standardize on a patch exchange format which mandates a description in the header, we can probably continously improve the description of patches as well. A lintian check can help, too. lintian already complains about dpatch patches without descriptions, and only doesn't complain about quilt ones because I haven't gotten around to writing the code. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Tue, 5 Feb 2008 09:00:52 +, Matthew Johnson [EMAIL PROTECTED] said: On Tue Feb 05 00:51, Manoj Srivastava wrote: If we can't figure out a good and clean way to keep a large stack of long-lived patches in the vcs then I firmly believe we should standardize on quilt. I think I have indeed solved the issue of long standing feature sets using feature branches, integration branches, and sloppy branches while upgrading, and would not want to be forced to regress to a patch system. I don't think anyone is talking about forcing DVCS users to regress to a patch system, merely to change the interchange format; which all DVCS-based maintenance methods can easily export to/import from. The only reason which you would have to interact with it would be a more standard interface for NMUs, which can only be a good thing. Why should I bring my feature branches into a patch system, when there is no need to? As far as the end user or NMUer is ocnerned, they do apt-get source foo, and they get the sources they may hack on. Adding to the chaos by converting my nice, clean source format to the blecherousness of a patch system does seem like regression to me. manoj -- IBM Advanced Systems Group -- a bunch of mindless jerks, who'll be first against the wall when the revolution comes... -- with regrets to D. Adams Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Tue, 05 Feb 2008 18:30:40 +0100, Sven Mueller [EMAIL PROTECTED] said: On another note, I have a slight problem with the (other) proposal of using a (git/$DVCS) repository as the form of source package distribution. Mainly because I think this will usually add bloat. While developing/packaging, I often see intermediate commits which need to be heavily modified or even reverted to finally build the patch I wanted to get working. Though these commits are fine in the Repository, I wouldn't want to see all the intermediate steps in the source package. I take it that you do not use sloppy branches to do development. I can see why you generate cruft in the repo. I generally do my development in a sloppy branch, which is in a private repo. When things work as I like, I then refactor patches from the sloppy branch into a feature branch (I have a catch-all features branch for stuff that is small and unlikely to make it into upstream). Each commit into a feature branch can be clean -- and then merged forward into the integration branch immediately. This way, all th cruft lies in the sloppy branch, which I then dump. manoj -- Real Programmers don't write in PL/I. PL/I is for programmers who can't decide whether to write in COBOL or FORTRAN. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Tue, Feb 05, 2008 at 12:34:08AM -0600, Manoj Srivastava wrote: On Thu, 2008-01-31 at 10:54:11 +0100, Pierre Habouzit wrote: Of course, that would mean that the few debian/rules (hi Manoj) in Debian not being makefiles [...] What do you mean, still? I have never used anything but make in my rules files; indeed, I was the one who has (over the objections of a few people) maintained that the policy of requiring Make files make sense. I do have Perl maintainer scripts, though. Hmm... I think that maybe Joey Hess uses such scripts, too. -- 1KB // Microsoft corollary to Hanlon's razor: // Never attribute to stupidity what can be // adequately explained by malice. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Tue Feb 05 00:51, Manoj Srivastava wrote: If we can't figure out a good and clean way to keep a large stack of long-lived patches in the vcs then I firmly believe we should standardize on quilt. I think I have indeed solved the issue of long standing feature sets using feature branches, integration branches, and sloppy branches while upgrading, and would not want to be forced to regress to a patch system. I don't think anyone is talking about forcing DVCS users to regress to a patch system, merely to change the interchange format; which all DVCS-based maintenance methods can easily export to/import from. The only reason which you would have to interact with it would be a more standard interface for NMUs, which can only be a good thing. I am against patch system users being forced to changed to a DVCS system, however, which _has_ been suggested. (It may be technically superior, but let me change when I'm ready by demonstrating the virtues and providing tutorials, don't force me to use it because it's the new source format) Matt -- Matthew Johnson signature.asc Description: Digital signature
Re: How to cope with patches sanely
Manoj Srivastava [EMAIL PROTECTED] writes:
Umm. Why would any distributed version control system always
need history truncation? I am not even sure that arch has such a
thing; and I have never felt the need for such a beast.
A distributed VCS that bundles in the whole archive in every
checkout might well need that, but not all distributed VCS systems do
that.
I wouldn't be surprised if arch took as much room to store log files
({arch}/...) as e.g. git takes to store the actual content of historical
revisions. Deleting those log files is arch's version of history
truncation.
[Not that I've actually felt the need to do so, but they do take up a
lot of room...]
-Miles
--
.Numeric stability is probably not all that important when you're guessing.
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
Matthew Johnson [EMAIL PROTECTED] writes: I am against patch system users being forced to changed to a DVCS system, however, which _has_ been suggested. I've not seen that suggested in this thread. Can you give a reference, please? What I've seen, that might be confused with the above, is: * arguments against patch systems on the basis of forcing others to use those specific tools; followed by requests; * arguments for DVCS workflows on the basis that they provide the advantages of tracking changes without forcing others to know the specific tool used; * requests from, and responses to, people wanting to know how a DVCS workflow can do this. -- \ Everything you read in newspapers is absolutely true, except | `\for that rare story of which you happen to have first-hand | _o__) knowledge. —Erwin Knoll | Ben Finney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On ti, 2008-02-05 at 12:07 +, Matthew Johnson wrote: Also: * The package format should be standardised such that the same workflow works for everyone. If that's a reference to my first post to this discussion, it's not accurate. I don't care about standardizing package formats, but I do want to have one, simple sequence of operations to work on all packages. Specifically I want dpkg-source -x to unpack a source package so that it is ready for modification, and dpkg-source -b to build a new source package after it's been edited. Patch systems can and should conform to that, since it's important for those doing archive-wide QA. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Tue Feb 05 22:43, Ben Finney wrote: Matthew Johnson [EMAIL PROTECTED] writes: I am against patch system users being forced to changed to a DVCS system, however, which _has_ been suggested. I've not seen that suggested in this thread. Can you give a reference, please? What I've seen, that might be confused with the above, is: * arguments against patch systems on the basis of forcing others to use those specific tools; followed by requests; * arguments for DVCS workflows on the basis that they provide the advantages of tracking changes without forcing others to know the specific tool used; Also: * The package format should be standardised such that the same workflow works for everyone. This implies to me that those who dislike patch systems and like DVCS workflows wish to standardise on the latter. I don't see how someone who wants to deal with patches can do that, whereas I can see how DVCS users can continue their workflow, but still have a patch system in the source format. (I would go through and get references, but it's a long thread, and I should be working. Sorry) Matt -- Matthew Johnson signature.asc Description: Digital signature
Re: How to cope with patches sanely
On Tue Feb 05 14:16, Lars Wirzenius wrote: On ti, 2008-02-05 at 12:07 +, Matthew Johnson wrote: Also: * The package format should be standardised such that the same workflow works for everyone. If that's a reference to my first post to this discussion, it's not accurate. I don't care about standardizing package formats, but I do want to have one, simple sequence of operations to work on all packages. Specifically I want dpkg-source -x to unpack a source package so that it is ready for modification, and dpkg-source -b to build a new source package after it's been edited. Patch systems can and should conform to that, since it's important for those doing archive-wide QA. I agree with that, however the discussion following that had lots of people (at least to me) saying that this set of operations should be a DVCS workflow. If I've misinterpreted that then I'm sorry to whoever I've inadvertently maligned (-: Standardising NMU/QA workflow is good. I just don't think it should involve invoking any sort of VCS and knowing how the merging works. Matt -- Matthew Johnson signature.asc Description: Digital signature
Re: How to cope with patches sanely
Simon Huggins wrote on 29/01/2008 02:51: [wigpen] The meta data can easily be in these version control systems that everyone on these threads seems to love so much. If you want to keep more patches than you expose through wigpen then just don't publish them in the dsc. That won't work well for one of the packages I (co-)maintain (and I assume for some other packages, too): I want a patch to be included in the Debian source, since it implements an often requested feature, but it can break other things. In other words: It conflicts with another patch (or maybe even two), but I want somebody who does apt-get source on the package to be able to do a (trivial) modification to the unpacked source and rebuild the package to have that feature (though knowing that this breaks something else). I think all Debian really needs is tools to generate a wigpen source package and the appropriate tools to then deal with it e.g. dak and sbuild updates. But still, wigpen looks nice to me. I could include the necessary patch files (and short docs on how to get alternative build working) in the debian.tar.gz instead of the .patches.tar.gz IIUIC and reach the same goal I mentioned above. If wigpen supported a series file (quilt) or 00list file (dpatch) in the .patches.tar.gz archive, I wouldn't need to work around anything. I don't mind getting the patches applied during unpack of the source package as long as the tool that is able to generate the source package from that unpacked source has a way to find out that someone changed the source after all patches were applied and handling that in a sane way (e.g. creating and enlisting an addition patch in .patches.tar.gz). On another note, I have a slight problem with the (other) proposal of using a (git/$DVCS) repository as the form of source package distribution. Mainly because I think this will usually add bloat. While developing/packaging, I often see intermediate commits which need to be heavily modified or even reverted to finally build the patch I wanted to get working. Though these commits are fine in the Repository, I wouldn't want to see all the intermediate steps in the source package. cu, Sven signature.asc Description: OpenPGP digital signature
Re: How to cope with patches sanely -- Debian New Maintainers' Guide
Hi, On Sat, Jan 26, 2008 at 12:40:14AM +0900, Osamu Aoki wrote: This seems to be much cleaner than dpatch or quilt. Also with the help of gitk, history is much more visible. I look forward to see it matured and accepted. personally I am a fan of the diversity in the Debian project. Its really fine, that developers are not forced to work with certain software (besides the minimal common standardisation that is needed), while I don't see a reason why they shouldn't be able to. However I also agree that in some levels diversity is way too much. The way patches to the upstream source is handled is one of these points where the diversity is way to much, as I and obviously a lot of other people think as well. So, yes, there should be a common agreement of how we handle patches and I am a fan of version control systems (for a number of reasons), but I would not agree that forcing all developers to use git is a good idea. Personally I would think that it makes more sense to keep version control and patch management seperated. There could be a common agreement on a patch system and that would be really fine. I'm using dpatch so far, but it has its weakness, so I don't see a reason why I shouldn't go with another patch system. So, I would agree with those recommending quilt, if it has significant pros besides dpatch. That would be forcing to a specific tool and so giving up some diversity, but it would keep giving up freedom of choice on a low level, while forcing everybody to a specific VCS wouldn't keep it low. Best Regards, Patrick -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely -- Debian New Maintainers' Guide
On Tue, Feb 05, 2008 at 06:24:38PM +, Patrick Schoenfeld wrote: So, I would agree with those recommending quilt, if it has significant pros besides dpatch. That would be forcing to a specific tool and so giving up some diversity, but it would keep giving up freedom of choice on a low level, while forcing everybody to a specific VCS wouldn't keep it low. Again, the discussion isn't (for me) about a tool, but an exchange format. We are discussing having patches served in a quilt series, and let people adapt their tools to be able to export their work as a quilt series so that other can find it and import it in their own tool of choice if needed. Enforcing maintenance tools is A Bad Idea™. The difference is huge: having common interfaces means that there is One True Tool that everyone can use if he doesn't know the Real Tools the Maintainer uses (apt-get source versus $SCM clone/checkout/whatever). -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpTVJqKeUgIB.pgp Description: PGP signature
Re: How to cope with patches sanely
On Sun February 3 2008 10:47:59 am Raphael Hertzog wrote: On Sat, 02 Feb 2008, Kumar Appaiah wrote: I was explaining how I would handle patches on top of upstream code. And where you have to update the patches when the upstream code changes. Rebase is exactly the process of updating patches while merge is really keep the old patch and add fixups patch to reconcile after conflicts. From what I remember of looking at rebase when I was evaluating git, isn't a side effect of it that it causes inconvenience for everyone that tracks your tree? That is, if you are going to use rebase on a tree, it is generally a private tree used on only one of your own machines, rather than something that others pull from as well? I have yet to see a convenient way to manage this, with any VCS whatsoever, that: 1. Lets you easily check out the state of the patched tree as of any given point in time 2. Lets you easily maintain a patchset where the patches on any given version are clearly separated into logical patches that can easily be sent to upstream 3. Facilitates teamwork with others using your repo The closest that I've seen yet is Mercurial's mq extension, which is quilt-like, in the mode where both the main repo and the mq area are versioned. This fails #1 because you have to get the main repo and the mq area to the same point manually, but succeeds on the others. I also am meaning to toy around with Mercurial's transplant extension, which can be similar to git's rebase (I gather), but with some other tree-manipulating features. But that fails the easily part because it's fairly esoteric VCS stuff. Frankly I have been able to keep my deltas to upstream small enough that just using a basic Mercurial repo for this has been fine. Although I maintain a fairly large package (Bacula), it is still nowhere near the size of KDE and X, and I also have the benefit of a very good working relationship with upstream. -- John
Re: How to cope with patches sanely
On Mon, 04 Feb 2008, John Goerzen wrote: On Sun February 3 2008 10:47:59 am Raphael Hertzog wrote: On Sat, 02 Feb 2008, Kumar Appaiah wrote: I was explaining how I would handle patches on top of upstream code. And where you have to update the patches when the upstream code changes. Rebase is exactly the process of updating patches while merge is really keep the old patch and add fixups patch to reconcile after conflicts. From what I remember of looking at rebase when I was evaluating git, isn't a side effect of it that it causes inconvenience for everyone that tracks your tree? That is, if you are going to use rebase on a tree, it is generally a private tree used on only one of your own machines, rather than something that others pull from as well? Yes it is. One needs to make it clear that a given branch is going to be rebased and can't be used for merging. Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Thu, 2008-01-31 at 10:54:11 +0100, Pierre Habouzit wrote: Of course, that would mean that the few debian/rules (hi Manoj) in Debian not being makefiles [...] You were probably thinking about Josip (and now the VDR team): http://lintian.debian.org/reports/Tdebian-rules-not-a-makefile.html regards, guillem -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Mon, Feb 04, 2008 at 08:02:42PM +, Guillem Jover wrote: On Thu, 2008-01-31 at 10:54:11 +0100, Pierre Habouzit wrote: Of course, that would mean that the few debian/rules (hi Manoj) in Debian not being makefiles [...] You were probably thinking about Josip (and now the VDR team): http://lintian.debian.org/reports/Tdebian-rules-not-a-makefile.html Damn, I was pretty sure Manoj still had some custom perl things, but I stand corrected :) -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgp8Ph3x3dalv.pgp Description: PGP signature
Re: How to cope with patches sanely
Pierre Habouzit [EMAIL PROTECTED] writes: On Mon, Feb 04, 2008 at 08:02:42PM +, Guillem Jover wrote: On Thu, 2008-01-31 at 10:54:11 +0100, Pierre Habouzit wrote: Of course, that would mean that the few debian/rules (hi Manoj) in Debian not being makefiles [...] You were probably thinking about Josip (and now the VDR team): http://lintian.debian.org/reports/Tdebian-rules-not-a-makefile.html Damn, I was pretty sure Manoj still had some custom perl things, but I stand corrected :) The VDR stuff actually is makefiles, too; it's a shell script wrapper around make that, so far as I could tell with a quick look, doesn't do anything that one couldn't do with a regular make include. But I may be missing some subtlety. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Mon, 04 Feb 2008 22:24:09 +0100, Pierre Habouzit [EMAIL PROTECTED] said: On Mon, Feb 04, 2008 at 08:02:42PM +, Guillem Jover wrote: On Thu, 2008-01-31 at 10:54:11 +0100, Pierre Habouzit wrote: Of course, that would mean that the few debian/rules (hi Manoj) in Debian not being makefiles [...] You were probably thinking about Josip (and now the VDR team): http://lintian.debian.org/reports/Tdebian-rules-not-a-makefile.html Damn, I was pretty sure Manoj still had some custom perl things, but I stand corrected :) What do you mean, still? I have never used anything but make in my rules files; indeed, I was the one who has (over the objections of a few people) maintained that the policy of requiring Make files make sense. I do have Perl maintainer scripts, though. manoj -- A clever prophet makes sure of the event first. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sat, 26 Jan 2008 00:05:32 +, Darren Salt [EMAIL PROTECTED] said: Whatever DSCM is used, it needs history truncation. Umm. Why would any distributed version control system always need history truncation? I am not even sure that arch has such a thing; and I have never felt the need for such a beast. A distributed VCS that bundles in the whole archive in every checkout might well need that, but not all distributed VCS systems do that. manoj -- The wise and intelligent are coming belatedly to realize that alcohol, and not the dog, is man's best friend. Rover is taking a beating -- and he should. -- W.C. Fields Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely
On Sat, 26 Jan 2008 11:34:27 -0500, David Nusinow [EMAIL PROTECTED] said: What bothers me about Joey's blog entry, and other similar proposals, is that it ignores a major reason for having external patch systems in the first place. Many of us have to carry long-lived patches to very complicated software. I'm personally thinking of the X server, but there are bound to be other examples. Additionally, we often have to carry several of these throughout the lifetime of the package. We also have to carry patches that aren't yet suitable for upstream, but may eventually become so. Finally, we have to carry patches backported from upstream HEAD. I have beeing doing this for some of my packages using arch; and while none of my packages are as large as X, it has worked out quite well for me. The idea proposed by many people, to keep the patches merged in your debian-specific branch, only really improves lives in the last case, when those patches are going to be merged in for the future anyhow. In any case where you have a patch that isn't going to go upstream for a long while, if ever, keeping these patches merged becomes painful. This has not been my experience with feature branches + integration branch. If the patches are large, then you'll forget most all of the details which makes merging painful. Additionally, it becomes quite difficult to see a large patch that touches many files in the whole. A great deal of archaelology becomes necessary to cope with that. If the patch becomes suitable to go upstream after some time, it also becomes hard to extract that patch to be sent later. An alternate idea I keep seeing is feature branches. I have absolutely no idea why these are considered preferable to most people. In the case of the X server we regularly carry around 30 patches on our stack. If we were to put each of these on its own unique feature branch, merging them becomes an exercise in pain. Sure, we could implement our own custom scripts and have very strict naming policies, but then you've re-invented the wheel and congratulations, you have yet another custom piece of crap software that you have to maintain on top of the already complicated one you actually care about. Additionally, you have a lot more branches getting in the way when you're trying to figure out which one you're looking for. In my experience, once I have created the integration branch, most upstream versions require little to none re-merging; I just apply the delta to each of the feature branches, _and_ the integration branch. Very rarely do I have to fix up a feature branch, and then apply a second delta with that fix up to the integration branch; but it has not been, for me, painful at all, nor do I have to jump through hoops every single time to accommodate new upstream. If we can't figure out a good and clean way to keep a large stack of long-lived patches in the vcs then I firmly believe we should standardize on quilt. I think I have indeed solved the issue of long standing feature sets using feature branches, integration branches, and sloppy branches while upgrading, and would not want to be forced to regress to a patch system. manoj -- Usage: fortune -P [] -a [xsz] [Q: [file]] [rKe9] -v6[+] dataspec ... inputdir Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely (Was: State of the project - input needed)
On Sat, 2 Feb 2008, martin f krafft wrote: PS: this figure is 80% correct but will surely be questioned by 70% of the people. The likelihood that someone replies to this message is 50%. I just take the mere chance to do this. ;-) The likelihood that someone flames is currently 12.6%. Is this number related to the 50% answers or absolute? The likelihood that I err is 0. I like this. ;-) I also like the feeling when sun might have risen behind the mountains in high North Andreas. -- http://fam-tille.de -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How to cope with patches sanely (Was: State of the project - input needed)
On Sat, Feb 02, 2008 at 08:21:44AM +, martin f krafft wrote: also sprach Pierre Habouzit [EMAIL PROTECTED] [2008.01.27.0505 +1100]: Seconded. I'd add, that in fact we should standardize on quilt as an exchange format for patches, because it's simple, and that there are powerful tools to handle them. Except those patches contain no VCS-specific data, like the author. In relation to licencing issues, I think it's especially interesting to have the ability to cherry-pick and keep the original committer info unchanged. I'd say what we really want is a way to cherry-pick between different VCS, and a higher-level wrapper on top of all VCS implementing a generic, cross-distro workflow. FWIW I export my patches in an almost quilt compatible way (ls -1 series and that would be true), using git format-patch, hence retaining VCS informations. Using quilt as an exchange format doesn't mean that you should strip off VCS metadatas (for the clever ones that supports it). Git works the same with a series of patches with authoring information. That's carved into its design. -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpYMOVI7eNCh.pgp Description: PGP signature
Re: How to cope with patches sanely (Was: State of the project - input needed)
On Sat, Feb 02, 2008 at 08:38:14AM +, martin f krafft wrote: also sprach Pierre Habouzit [EMAIL PROTECTED] [2008.01.27.0939 +1100]: On Fri, Jan 25, 2008 at 06:00:02PM +, Raphael Hertzog wrote: On Fri, 25 Jan 2008, Michael Banck wrote: On Fri, Jan 25, 2008 at 10:51:05AM +0100, Lucas Nussbaum wrote: On 25/01/08 at 08:01 +, Steve Langasek wrote: As a second runner up, quilt is ok by me. :) ... I'm less and less sure that a git-based format is a brilliant idea. I like git more than a lot, but it's a poor idea to base source packages on them. Why? Because it's a way to high level tool for the task. Git (and I suppose that the following is true for other DVCS, and if not, they _REALLY_ suck, and I mean it, really) has been designed so that you can only exchange patches. That means that you can work on your git repository, extract your patches, send them upstream, see upstream merge them into his git repository and when you fetch back, git figures things out. *DANG* that's how the kernel and git itself are developed. There is no *need* to force people using git to grok the format since git *is* able to grok it (not the current one, but some form of wigpen should probably work). IOW, all that you should need to grok what is in a source package should basically be tar/ar/cpio/... and vi. Are you sure that this has to be the case in 10 years? Are you sure git will be there and backward compatible in 10 years ? Okay, it's likely given the current upstream, that focuses a _lot_ on backward compatibility. But still. Some repository formats have been deprecated (the object store is of version3 and IIRC git doesn't grok version1 anymore, but I may be mistaken). That's the kind of gamble I wouldn't take. Whereas I guess tar will always grok tarballs it generates today in 10 years, and gzip/bzip2/lzma/$compressor won't change either, and text is text for enough time to assume it'll remain as editeable in 10 years. At the same time, I don't think it's wise to expect people to know the different ways to handle git.tar.gz and bzr.tar.gz and so on. I think what we want is the often-mentioned wrapper which hides the actual storage backend in use. No IMSHO we want the tools we use to be able to import informations from these packages easily, and in fact, we rather just want them to generate those packages. I see little point in .git.tar.gz because it only holds _recent_ history, and you can't base new devels on it (you cannot commit in a shallow clone in git). AFAUI in bzr it's even worse: shallow clones have a reference to the original repository, that could have disappeared in 10 years. So it makes this extra history usefulness quite useless in the long term [I'm not saying it's a waste of space or whatever, I just say it's IMHO a not so useful feature, which adds a clear complexity and non-editability to the package]. On Sat, Feb 02, 2008 at 08:43:12AM +, martin f krafft wrote: also sprach Theodore Tso [EMAIL PROTECTED] [2008.01.28.1613 +1100]: From: Author O' The Patch [EMAIL PROTECTED] Detailed patch description Signed-off-by: Theodore Ts'o [EMAIL PROTECTED] This is at the beginning of every single quilt patch, and because of this, we can easily important the patch into either a mercurial or git repository, while preserving the authorship information of the patch. Nice, I didn't see this before. This *is* in fact nice and puts the quilt *format* very high on my list. Yes, quilt preserves *everything* you put in the header, so if you use git (or $scm) to generate the patches with authoring information, commit message and so on, it wont be lost. You agree on the fact that a Debian source package isn't (or shouldn't for big enough packages) be the preferred form of modification. Then it's that it's an exchange format. As an exchange format quilt is brilliant, because it holds *EVERYTHING* a SCM need to figure out how to rebuild meregeable and rebaseable patches from it. So why bother with anything else ? An exchange format *Must* be simple. That's the very reason why Debian uses rfc822-style flat files everywhere, and that's one of our best strength: (1) this format is ubiquitous in Debian ; (2) it's really easy to parse (in the Debian flavour that doesn't need the stupid quoted-printeable escapings and so on at least) ; (3) it's human readable ; (4) implementing parsers and generators take usually less than a hundred lines in a high level enough language. .git.tar.gz fails in those 4 points. What I ask you is just to be consistent. Either we _will_ modify source packages, either we won't. If we will, adding features to it is a good idea, if we won't, let's just focus on how to let it be expressive enough to encode in it all what we use as new features upstream from it. And as a matter of a fact, quilt is enough for that. -- ·O· Pierre Habouzit ··O
