Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
On Friday 30 May 2008 08:42:27 am Stefano Zacchiroli wrote: On Fri, May 30, 2008 at 12:33:01AM +0200, sean finney wrote: one open question i have now is with regards to the eventual hosting of the system: does it seem reasonable that the final version could either coexist on an existing archive, or alternatively have enough space for the .dsc and .diff.gz files from the source packages being tracked to be stored alongside it? Can't you just make it dependent, as a configuration variable or something, on an on disk mirror containing at least source packages? If yes, that is what i was implying by coexist on an existing archive :) signature.asc Description: This is a digitally signed message part.
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
On Friday 30 May 2008 08:32:18 am Lucas Nussbaum wrote: How much static content will you have? I mean: it's not like this service will receive millions of requests per day. It could be better to have only a few static pages, and do everything else on-demand. well, i'm taking a fairly conservative approach, at least for starting off, at trying to pre-generate everything. the less dynamic stuff going on the less chance for bad things happening with shoddy code, and the less one can be distracted by various other unrelated problems and instead put the focus on what should be presented and how. i'm sure that we'll inevitably need some level of interactiveness, like having a way to query the patches for a package, add/remove patch metadata, interacting with external sources, etc, but starting simple with something that can hopefully have smaller portions gutted later to do this. again, had less time than hoped for on this, but i've been making enough progress that i can at least share the humble beginnings of the code. there's not a whole lot to see, just a breakdown of the patch info from the diff.gz in various ways (./debian , non-./debian, series patch listing and info) and downloadable/marked-up versions of all the patches: http://git.debian.org/?p=users/seanius/patch-tracker.git;a=summary there's a README with some brief info. and i've rsync'd a copy of my partial source archive tracking data (okay, so it's three semi-randomly chosen source packages, one native, one diff.gz. only, one with quilt-like series) here: http://people.debian.org/~seanius/pts/patches/ the design is fairly spartan at this point, but it's all xhtml compliant and using a centralized stylesheet, so that can be fixed later :) anyway, the lines are open for comments/suggestions/patches/etc :) sean signature.asc Description: This is a digitally signed message part.
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
On Fri, May 30, 2008 at 12:33:01AM +0200, sean finney wrote: one open question i have now is with regards to the eventual hosting of the system: does it seem reasonable that the final version could either coexist on an existing archive, or alternatively have enough space for the .dsc and .diff.gz files from the source packages being tracked to be stored alongside it? Can't you just make it dependent, as a configuration variable or something, on an on disk mirror containing at least source packages? If so we can then either install it on a machine already having a local mirror, or on a machine set up for that purpose which will also be running a debmirror (or whatever) to mirror sources. Cheers. -- Stefano Zacchiroli -*- PhD in Computer Science ... now what? [EMAIL PROTECTED],cs.unibo.it,debian.org} -%- http://upsilon.cc/zack/ (15:56:48) Zack: e la demo dema ?/\All one has to do is hit the (15:57:15) Bac: no, la demo scema\/right keys at the right time signature.asc Description: Digital signature
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
On 30/05/08 at 00:33 +0200, sean finney wrote: hi folks, just a quick update, the PoC that i was doing was quickly getting complicated enough that i've abandoned it in favor of an initial implementation code that will hopefully be useful in the final version as well. thus it has introduced some more delay, as i was evaluating a few different tools/bindings/etc. Cool :-) - partial source archive mirror with some sample packages, via reprepro - python script for generating static content - python-cheetah for template output generation - python-debian for parsing various archive/control files - python-pygments for diff markup How much static content will you have? I mean: it's not like this service will receive millions of requests per day. It could be better to have only a few static pages, and do everything else on-demand. one open question i have now is with regards to the eventual hosting of the system: does it seem reasonable that the final version could either coexist on an existing archive, or alternatively have enough space for the .dsc and .diff.gz files from the source packages being tracked to be stored alongside it? Yes. I'm sure that you will receive offers to host this service on a suitable host, when necessary. -- | Lucas Nussbaum | [EMAIL PROTECTED] http://www.lucas-nussbaum.net/ | | jabber: [EMAIL PROTECTED] GPG: 1024D/023B3F4F | signature.asc Description: Digital signature
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
Lucas Nussbaum wrote: patches.debian.org == Objective: Provide simple place to learn about Debian-specific patches, for upstream, users and other DDs. Features: browsing patches, downloading them, tracking status of patches. Split .diff.gz to extract patches for: - each quilt/dpatch/simple-patchsys patch Hi Lucas, Sean, I just wanted to mention a potential gotcha to consider with dpatch -- note that it allows patch files to be arbitrary scripts. So if a dpatch file begins with a shebang line that doesn't include the string dpatch, it should probably just be shown by the patch browsing system as a normal text file without patch-style formatting. (Or even with special formatting for the most common scripting languages -- sh, perl, python, etc. -- if not too much trouble.) Of course it's possible that I am the only person that uses this obscure dpatch feature (I have some scripts to move source code files around during the patch step -- yes, they honor both the -patch and -unpatch flags). So maybe it's easiest not to worry about it too much. best regards, -- Kevin B. McCarty [EMAIL PROTECTED] WWW: http://www.starplot.org/ WWW: http://people.debian.org/~kmccarty/ GPG: public key ID 4F83C751 signature.asc Description: OpenPGP digital signature
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
hi folks, just a quick update, the PoC that i was doing was quickly getting complicated enough that i've abandoned it in favor of an initial implementation code that will hopefully be useful in the final version as well. thus it has introduced some more delay, as i was evaluating a few different tools/bindings/etc. what i'm working with now: - partial source archive mirror with some sample packages, via reprepro - python script for generating static content - python-cheetah for template output generation - python-debian for parsing various archive/control files - python-pygments for diff markup what i have so far: - front page with quick shortcut links to a-z(and liba-libz) summary pages - summary pages for a-z (and liba-libz) listings of packages with available versions listed - per-package-per-version page with listing of diffs for series format packages i'll be putting some extra polish on this in the next day or so, including adding a few more page templates such as per-package summaries comparing versions between releases and also adding support for at least non-debian-dir diff.gz information and v3 (quilt) format packages. after that i'll rsync what i have generated to somewhere under my p.d.o page, and throw the code into a git repo for perusal/patching/etc. this will likely take place towards the end of the weekend. one open question i have now is with regards to the eventual hosting of the system: does it seem reasonable that the final version could either coexist on an existing archive, or alternatively have enough space for the .dsc and .diff.gz files from the source packages being tracked to be stored alongside it? sean signature.asc Description: This is a digitally signed message part.
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
On 20/05/08 at 21:01 +0200, sean finney wrote: hi, this is both a request and an notice of intent that i may try to implement something :) Hi Sean, Have you had time to work on something? -- | Lucas Nussbaum | [EMAIL PROTECTED] http://www.lucas-nussbaum.net/ | | jabber: [EMAIL PROTECTED] GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
hiya, Hi Sean, Have you had time to work on something? progress was slower than i would have liked, but yes there's progress. i should have something to show in a day or two. sean signature.asc Description: This is a digitally signed message part.
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
On 20/05/08 at 21:01 +0200, sean finney wrote: Package: qa.debian.org Severity: wishlist -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 hi, this is both a request and an notice of intent that i may try to implement something :) my proposal is that the current patches section on a source package's qa package is modified so that the following information is somehow provided, where appropriate (depending on structure/format of source package naturally). What you propose sounds like adding a lot of information to the PTS page. Why did you choose that, instead of a separate website (http://patches.d.o)? - - entire debian patch - - individual quilt/dpatch patches - - individual patches from v3 (quilt) format packages - - feature-branch patches for v3 (git) format packages, if possible (would require more thinking through and i do not intend to do this initially) It's not clear yet whether this format will be accepted in the archive. additionally, for each patch, when possible provide the following information: - - which version of the package introduced it how are you planning to determine that? why is that important? - - any notes at the top of the patch - - any relevant vcs metadata - - any relevant bts metadata (could be found using heuristics in the previous items) example? -- | Lucas Nussbaum | [EMAIL PROTECTED] http://www.lucas-nussbaum.net/ | | jabber: [EMAIL PROTECTED] GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
On Tue, 20 May 2008, sean finney wrote: my proposal is that the current patches section on a source package's qa package is modified so that the following information is somehow provided, where appropriate (depending on structure/format of source package naturally). - - entire debian patch - - individual quilt/dpatch patches - - individual patches from v3 (quilt) format packages - - feature-branch patches for v3 (git) format packages, if possible (would require more thinking through and i do not intend to do this initially) additionally, for each patch, when possible provide the following information: - - which version of the package introduced it - - any notes at the top of the patch - - any relevant vcs metadata - - any relevant bts metadata (could be found using heuristics in the previous items) - - html markup and standard format versions of each patch for download/review It really looks like the patches.debian.org discussed on -devel. ;-) I would suggest you to design it somewhat independently from the PTS and only plan the PTS to point to it. I don't think that the PTS should be a source of information, but rather a portal. It could possibly be mole-based so that it can automatically process any new source package that gets uploaded to Debian. http://qa.debian.org/cgi-bin/mole/doc (The table devpackages-src would be your TodoSource) Various other considerations that I'd like to add: - make it cross-distro from the beginning so that it can contain patches from multiple distributions (ie each patch is associated with one or more distros) - have a generic way to upload patches to this system so that each distribution can extract their patches in the way that they want (rsync over ssh + ssh keys is what mole uses, IMO it would fit the bill here too, you just have to define the format of what gets uploaded) - the stuff that extracts Debian patches uploads the required data to the system above In the long term, I would also like a more interactive web portal around this so that people can comment on patches, etc. Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
hiya, On Wednesday 21 May 2008 07:02:30 am Lucas Nussbaum wrote: What you propose sounds like adding a lot of information to the PTS page. Why did you choose that, instead of a separate website (http://patches.d.o)? after looking over the sources for the pts, i agree that the best place to do this is no in the pts itself, but to have a portal/summary from the pts to a page with the full information. - - feature-branch patches for v3 (git) format packages, if possible (would require more thinking through and i do not intend to do this initially) It's not clear yet whether this format will be accepted in the archive. right, that is simply me acknowledging that this is a potential case that could be covered. additionally, for each patch, when possible provide the following information: - - which version of the package introduced it how are you planning to determine that? why is that important? how is perhaps left for future exploration, but i assume via tracking each source package upload. as for why, i thought it was pretty self-evident. - - any notes at the top of the patch - - any relevant vcs metadata - - any relevant bts metadata (could be found using heuristics in the previous items) example? i'll try and generate (or create a mock-up page at least) over the next few days for an example package or two. sean signature.asc Description: This is a digitally signed message part.
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
oh, missed some comments at the top: On Wednesday 21 May 2008 08:56:20 am Raphael Hertzog wrote: It really looks like the patches.debian.org discussed on -devel. ;-) if it's in that debian patches discussion, i killfiled the whole thing because the S:N ratio was getting a little low. but i assumed someone must have proposed something similar. I would suggest you to design it somewhat independently from the PTS and only plan the PTS to point to it. I don't think that the PTS should be a source of information, but rather a portal. agreed. some kind of machine-parsable summary information could be generated, so that the pts page had more than just a link but something perhaps more at a glance useful, but the bulk of the information would be on a seperate page. sean signature.asc Description: This is a digitally signed message part.
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
hiya, On Wednesday 21 May 2008 08:56:20 am Raphael Hertzog wrote: It could possibly be mole-based so that it can automatically process any new source package that gets uploaded to Debian. http://qa.debian.org/cgi-bin/mole/doc (The table devpackages-src would be your TodoSource) thanks for the pointer, i'll look at that. Various other considerations that I'd like to add: - make it cross-distro from the beginning so that it can contain patches from multiple distributions (ie each patch is associated with one or more distros) interesting. i don't see any reason why it couldn't be cross distro, but think i will start small and just try to leave room for expansion later. - have a generic way to upload patches to this system so that each distribution can extract their patches in the way that they want (rsync over ssh + ssh keys is what mole uses, IMO it would fit the bill here too, you just have to define the format of what gets uploaded) - the stuff that extracts Debian patches uploads the required data to the system above i'm not sure what you mean by uploading. In the long term, I would also like a more interactive web portal around this so that people can comment on patches, etc. another good idea. sean signature.asc Description: This is a digitally signed message part.
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
On Wed, 21 May 2008, sean finney wrote: Various other considerations that I'd like to add: - make it cross-distro from the beginning so that it can contain patches from multiple distributions (ie each patch is associated with one or more distros) interesting. i don't see any reason why it couldn't be cross distro, but think i will start small and just try to leave room for expansion later. Sure, just add the required field now but always put Debian in it. :) - have a generic way to upload patches to this system so that each distribution can extract their patches in the way that they want (rsync over ssh + ssh keys is what mole uses, IMO it would fit the bill here too, you just have to define the format of what gets uploaded) - the stuff that extracts Debian patches uploads the required data to the system above i'm not sure what you mean by uploading. Well, you'll have a web page to display the patches and the information, and you'll have some script that extract the patches and the meta-information. Uploading is the process used by the scripts to make patches available on the web interface. It makes sense to decouple both parts as the extraction scripts might run on several machines (one for each distro for instance). Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
On 21/05/08 at 09:03 +0200, sean finney wrote: additionally, for each patch, when possible provide the following information: - - which version of the package introduced it how are you planning to determine that? why is that important? how is perhaps left for future exploration, but i assume via tracking each source package upload. Mmh, sounds complicated. as for why, i thought it was pretty self-evident. Erm, is it? If we know which bug the patch is trying to fix, it doesn't really sound interesting to know which upload introduced the patch. (OK, it might be useful in some cases, but is it worth the hassle?) - - any notes at the top of the patch - - any relevant vcs metadata - - any relevant bts metadata (could be found using heuristics in the previous items) example? i'll try and generate (or create a mock-up page at least) over the next few days for an example package or two. Cool! I also have written down some stuff about this. Will try to copy it to this bug report later. -- | Lucas Nussbaum | [EMAIL PROTECTED] http://www.lucas-nussbaum.net/ | | jabber: [EMAIL PROTECTED] GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
Here is what I wrote down about this patches.d.o idea (sorry, it's not very well organized, and doesn't include some ideas mentioned in the -devel@ thread). patches.debian.org == Objective: Provide simple place to learn about Debian-specific patches, for upstream, users and other DDs. Features: browsing patches, downloading them, tracking status of patches. Split .diff.gz to extract patches for: - each quilt/dpatch/simple-patchsys patch - debian packaging stuff (remaining stuff in debian/ after patches have been removed) - other patches done directly in .diff.gz If a file was added (case for debian packaging) download the file, not a diff. Provide a way to download each patch separately. Web interface available at http://patches.d.o/package or http://patches.d.o/package/version where version can be: - a version number in stable/testing/unstable/experimental - stable,testing,unstable,experimental (so we need to store the version of the package in each suite) (goal: provide stable urls) - all, means patches for all suites. Merge identical (= same patch, not just same name) patches. Patches extracted to http://patches.d.o/p/package/version Tags to process in the patches: Type: debian: the patch addresses a debian-specific problem upstream: the candidate is a candidate for upstream integration backport: the patch is a backport from an upstream fix Fixes: URL of the bug that is fixed by the patch (possibly more than one bug) Forwarded: URL of the bug where the patch was forwarded upstream (possibly more than one) others, see -devel@ thread. Use bts-link to track the status of Forwarded: bugs. (talk with Pierre Habouzit) -- | Lucas Nussbaum | [EMAIL PROTECTED] http://www.lucas-nussbaum.net/ | | jabber: [EMAIL PROTECTED] GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
On 21/05/08 at 08:56 +0200, Raphael Hertzog wrote: Various other considerations that I'd like to add: - make it cross-distro from the beginning so that it can contain patches from multiple distributions (ie each patch is associated with one or more distros) I really don't think that it is a good idea. I agree that a cross-distro effort to share patches would be a good thing, but let's not over-engineer this now: let's write a debian-specific solution, and then discuss this with other distros. In the long term, I would also like a more interactive web portal around this so that people can comment on patches, etc. I don't think it's a good idea either. The discussion about patches should happen at a single place (and the best place for that is probably a bug tracking system, like upstream's if the bug was forwarded) -- | Lucas Nussbaum | [EMAIL PROTECTED] http://www.lucas-nussbaum.net/ | | jabber: [EMAIL PROTECTED] GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#482102: qa.debian.org: intelligent listing of debian/patches, git/quilt v3 format packages, etc
Package: qa.debian.org Severity: wishlist -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 hi, this is both a request and an notice of intent that i may try to implement something :) my proposal is that the current patches section on a source package's qa package is modified so that the following information is somehow provided, where appropriate (depending on structure/format of source package naturally). - - entire debian patch - - individual quilt/dpatch patches - - individual patches from v3 (quilt) format packages - - feature-branch patches for v3 (git) format packages, if possible (would require more thinking through and i do not intend to do this initially) additionally, for each patch, when possible provide the following information: - - which version of the package introduced it - - any notes at the top of the patch - - any relevant vcs metadata - - any relevant bts metadata (could be found using heuristics in the previous items) - - html markup and standard format versions of each patch for download/review obviously not all of this could be gleaned automatically, but i imagine much of it could, and with a few heuristics the rest could be figured out. future versions could also provide a way for packagers to override such heuristics in the case of errors/false-positives. questions/comments/etc welcome sean - -- System Information: Debian Release: lenny/sid APT prefers unstable APT policy: (500, 'unstable'), (500, 'testing'), (1, 'experimental') Architecture: amd64 (x86_64) Kernel: Linux 2.6.24-1-amd64 (SMP w/2 CPU cores) Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/bash -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFIMyAMynjLPm522B0RAn/sAJ0TgRckJm/4yDdcxcrOaEC4vcLl4QCfVKvQ lUtRYrVFQZPR6PqxxBOH+qM= =Sggx -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
