Re: feedback for NEW packages: switch to using the BTS?
Hello, On Sun 01 May 2022 at 07:52am +08, Paul Wise wrote: > On Sat, 2022-04-30 at 14:20 -0700, Sean Whitton wrote: > >> This would really slow things down; >> I don't think we could work that way. > > Using separate bug reports or not would of course be up to the person > filing them, but I expect the process could be automated well enough > that it wouldn't be much different to the current process (which I am > not familiar with). An automated multi-bug process might be something > like this; press the button for filing a bug, get a template with the > right version etc, type out the problem details, press send, repeat the > process. Is the current process any different other than the repeating? It's quicker to to edit a single text file, review the whole thing and then send it. -- Sean Whitton
Re: feedback for NEW packages: switch to using the BTS?
On Sat, 30 Apr 2022, Paul Wise wrote: > I suppose debbugs could allow the package state to go out of sync with > NEW and instead retain the addresses for a certain period of time > after packages are removed from NEW and while there are open bugs on > the packages that were removed from NEW. This might be a reasonable process to figure out. We currently don't disambiguate between "this package never existed" and "this package used to exist". Doing that would allow the current FTP master process to work while tracking things in the BTS. REJECT process would create a bug against the source package which can be optionally cloned by the maintainer if they want to fix issues separately. Packages uploaded to NEW would create entries in the to-be-created "historical Maintainers" file. We wouldn't bother to track detailed versioning information; that will get fixed up when the package actually transits is ACCEPTED. Still unresolved is how this intersects with the search that people use to reassign bugs against outdated packages to the current package (or close bugs in removed packages). -- Don Armstrong https://www.donarmstrong.com Grimble left his mother in the food store and went to the launderette and watched the clothes go round. It was a bit like color television only with less plot. -- Clement Freud _Grimble_
Re: feedback for NEW packages: switch to using the BTS?
What does it help to have it sitting there instead of rejected? I gave it more thought, and I don't think it has to be sitting in the NEW queue at all. As far as I understand it, the main advantage to gain is additional context; a bug report provides documentation and a place to discuss solutions for packaging issues, especially if the NEW package is maintained by a team. None of this requires the packages to keep waiting in NEW, it just requires the BTS to learn about them, so the bug reports can be assigned properly. There are probably edge cases to consider (such as invalid package names or hijacking uploads) where it is not possible to assign the bugs. I still like the idea to expand the BTS scope to include pre-archive packaging bugs, provided that the assumption that most rejected package uploads will be fixed and accepted eventually, holds. Cheers Timo -- ⢀⣴⠾⠻⢶⣦⠀ ╭╮ ⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │ ⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │ ⠈⠳⣄ ╰╯ signature.asc Description: PGP signature
Re: feedback for NEW packages: switch to using the BTS?
On May 1, 2022 4:44:21 PM UTC, "Timo Röhling" wrote: >* Scott Kitterman [2022-04-29 23:32]: >> I don't understand why this is any better than just rejecting the >> package? Once it's been determined that the upload won't be >> accepted, I don't see a benefit to having it remain in New. >I think this is a matter of perspective: for the FTP team, the >upload is basically a stateless process; a package is either fit for >inclusion in the archive or it is not. For the uploading DD, it is >merely a particular (undesirable) state in the packaging process. >The NEW queue is a mandatory review of their packaging effort, and >the REJECT is feedback which they will (hopefully) address, and then >try again. Maybe, in some circumstances, they will abandon the >package, but I'd say that this not typical. > >Converting the NEW review process into a bug-centric approach >instead of a mail-centric approach reinforces the DD's point of >view, focusing more on the package upload as a part of package >maintainership. > >The question is: how will this affect the FTP team's point of view? >If it will clutter the NEW queue with half-processed packages, I'd >say it is a bad tradeoff, because it adds mental load to the FTP >team. At a minimum, there needs to be appropriate tooling which >allows to distinguish easily between new packages requiring FTP team >review and packages waiting for fixes to be applied. I don't think that answers my question at all. Yes, someone could invest a bunch of time in updating DAK so that known unacceptable packages wouldn't clutter the review que, but how would that be better than what we have now? What does it help to have it sitting there instead of rejected? Scott K
Re: feedback for NEW packages: switch to using the BTS?
* Scott Kitterman [2022-04-29 23:32]: I don't understand why this is any better than just rejecting the package? Once it's been determined that the upload won't be accepted, I don't see a benefit to having it remain in New. I think this is a matter of perspective: for the FTP team, the upload is basically a stateless process; a package is either fit for inclusion in the archive or it is not. For the uploading DD, it is merely a particular (undesirable) state in the packaging process. The NEW queue is a mandatory review of their packaging effort, and the REJECT is feedback which they will (hopefully) address, and then try again. Maybe, in some circumstances, they will abandon the package, but I'd say that this not typical. Converting the NEW review process into a bug-centric approach instead of a mail-centric approach reinforces the DD's point of view, focusing more on the package upload as a part of package maintainership. The question is: how will this affect the FTP team's point of view? If it will clutter the NEW queue with half-processed packages, I'd say it is a bad tradeoff, because it adds mental load to the FTP team. At a minimum, there needs to be appropriate tooling which allows to distinguish easily between new packages requiring FTP team review and packages waiting for fixes to be applied. Cheers Timo -- ⢀⣴⠾⠻⢶⣦⠀ ╭╮ ⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │ ⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │ ⠈⠳⣄ ╰╯ signature.asc Description: PGP signature
Re: feedback for NEW packages: switch to using the BTS?
Paul Wise left as an exercise for the reader: > I think the same problem and suggestions also apply with the current > system of prods and rejects, so please do add or propose adding it in > the appropriate documentation and templates. I will of course seek to > preserve these statements if I end up working on the BTS proposal. absolutely, there's just enough opacity in a current REJECT (in my fairly minimal experience) that i think it less susceptible to such invalid inferences. but yes, this is something that could go in the Mentors notes now. (to be sure, at no time did i intend to impugn your suggestions in this thread, which seem a definite improvement from the user's perspective.) -- nick black -=- https://www.nick-black.com to make an apple pie from scratch, you need first invent a universe. signature.asc Description: PGP signature
Re: feedback for NEW packages: switch to using the BTS?
On Sat, 2022-04-30 at 22:47 -0400, nick black wrote: > i understand. i suppose that what i'm saying is it will probably > be worthwhile to convey in Mentors etc. documentation that > "resolving the bugs filed thus far [regarding the package in > NEW] does not imply that no further bugs will be filed." > > i'm just worried that people will get a bug filed that is not a > complete, exhaustive analysis, and think that it's the only > thing standing between them and archive glory. perhaps whatever > files the bug ought indicate this in the text of the bug itself. I think the same problem and suggestions also apply with the current system of prods and rejects, so please do add or propose adding it in the appropriate documentation and templates. I will of course seek to preserve these statements if I end up working on the BTS proposal. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
Paul Wise left as an exercise for the reader: > > if resolving the resulting bug report is the bar needing clearing to > > enter the archive, that does seem to require FTPmasters to create a > > complete statement of REJECT-worthy problems in each uploaded > > package, which seems like more work than they must currently do. > > The bar for acceptance would be the same as it is now, the proposal > just changes how the issues blocking acceptance are communicated. > Now you get a REJECT mail, under the proposal you get a serious bug. i understand. i suppose that what i'm saying is it will probably be worthwhile to convey in Mentors etc. documentation that "resolving the bugs filed thus far [regarding the package in NEW] does not imply that no further bugs will be filed." i'm just worried that people will get a bug filed that is not a complete, exhaustive analysis, and think that it's the only thing standing between them and archive glory. perhaps whatever files the bug ought indicate this in the text of the bug itself. -- nick black -=- https://www.nick-black.com to make an apple pie from scratch, you need first invent a universe. signature.asc Description: PGP signature
Re: feedback for NEW packages: switch to using the BTS?
On Sat, 2022-04-30 at 14:20 -0700, Sean Whitton wrote: > This would really slow things down; > I don't think we could work that way. Using separate bug reports or not would of course be up to the person filing them, but I expect the process could be automated well enough that it wouldn't be much different to the current process (which I am not familiar with). An automated multi-bug process might be something like this; press the button for filing a bug, get a template with the right version etc, type out the problem details, press send, repeat the process. Is the current process any different other than the repeating? -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
Hello, On Sat 30 Apr 2022 at 07:44AM +08, Paul Wise wrote: > It also means the ftpmasters can file separate bugs for each problem, > rather than combining them all into one mail. This would really slow things down; I don't think we could work that way. -- Sean Whitton signature.asc Description: PGP signature
Re: feedback for NEW packages: switch to using the BTS?
On Sat, 2022-04-30 at 03:09 -0400, nick black wrote: > currently, as far as i understand things, a REJECT can be issued > for the first REJECT-worthy problem. The same would occur under the proposal, except that there would presumably be separate bug reports for separate issues. > if resolving the resulting bug report is the bar needing clearing to > enter the archive, that does seem to require FTPmasters to create a > complete statement of REJECT-worthy problems in each uploaded > package, which seems like more work than they must currently do. The bar for acceptance would be the same as it is now, the proposal just changes how the issues blocking acceptance are communicated. Now you get a REJECT mail, under the proposal you get a serious bug. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
Paul Wise left as an exercise for the reader: > Packages with incomplete or incorrect debian/copyright information > currently would recieve a REJECT rather than acceptance. The proposal > would not change that, just turn that REJECT into a bug report, which > you could fix in a second upload to NEW and then the package would be > reprocessed and get an ACCEPT or another bug. currently, as far as i understand things, a REJECT can be issued for the first REJECT-worthy problem. if resolving the resulting bug report is the bar needing clearing to enter the archive, that does seem to require FTPmasters to create a complete statement of REJECT-worthy problems in each uploaded package, which seems like more work than they must currently do. if the resulting bug is *not* a complete statement of problems, and that is understood, this isn't an issue. just a thought. -- nick black -=- https://www.nick-black.com to make an apple pie from scratch, you need first invent a universe. signature.asc Description: PGP signature
Re: feedback for NEW packages: switch to using the BTS?
On Sat, 2022-04-30 at 00:13 +, Scott Kitterman wrote: > I'm still not understanding how any of that needs to have a package > we've decided not to accept sitting in New? My thinking is that debbugs would require a package (imported from new.822) to exist for maintainer addresses. If you file a bug, then remove the package from NEW, then it disappears from the BTS too and subsequent mails to the bug wouldn't have a place to be forwarded to and would go to the address for unknown packages. I suppose debbugs could allow the package state to go out of sync with NEW and instead retain the addresses for a certain period of time after packages are removed from NEW and while there are open bugs on the packages that were removed from NEW. Then some process could regularly close and archive the bugs filed by ftpmasters that were not solved. I think it would be simpler to handle this on the dak side though, by keeping the packages around, perhaps in OLD instead of NEW :) and then rejecting them after a certain time and closing the bugs. > In my mind if a package is in New it's in one of two states: > > 1. Not reviewed yet. > > 2. Partly/completely reviewed, pending resolution of a question > (e.g. prod sent to maintainer, waiting for reply). > > What's the advantage of added a third: > > 3. Not going to be accepted, waiting for a new upload or for enough > time to pass before rejecting. States 2 and 3 are very similar; there is something to be resolved with the package and it is waiting on maintainer/uploader action for that. The only difference is 2 is waiting on a reply, 3 waits on an upload. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
On April 29, 2022 11:44:54 PM UTC, Paul Wise wrote: >On Fri, 2022-04-29 at 23:32 +, Scott Kitterman wrote: > >> I don't understand why this is any better than just rejecting the >> package? Once it's been determined that the upload won't be >> accepted, I don't see a benefit to having it remain in New. > >The initial upload might not get an ACCEPT, but later revisions of the >package might be suitable for inclusion in Debian. > >The main aim of the proposal is transparency for all problems getting >the package into NEW. Often it isn't clear what happened to a package >that was rejected, the proposal makes it easy to find out; just check >the bug reports for the source package. > >It also means the ftpmasters can file separate bugs for each problem, >rather than combining them all into one mail. > >It also means the ftpmasters can easily point out other issues they >noticed that don't block the package from being accepted and ensure >that those issues won't be forgotten. > Most of that seems mostly reasonable, but I'm still not understanding how any of that needs to have a package we've decided not to accept sitting in New? In my mind if a package is in New it's in one of two states: 1. Not reviewed yet. 2. Partly/completely reviewed, pending resolution of a question (e.g. prod sent to maintainer, waiting for reply). What's the advantage of added a third: 3. Not going to be accepted, waiting for a new upload or for enough time to pass before rejecting. Scott K
Re: feedback for NEW packages: switch to using the BTS?
On Fri, 2022-04-29 at 23:32 +, Scott Kitterman wrote: > I don't understand why this is any better than just rejecting the > package? Once it's been determined that the upload won't be > accepted, I don't see a benefit to having it remain in New. The initial upload might not get an ACCEPT, but later revisions of the package might be suitable for inclusion in Debian. The main aim of the proposal is transparency for all problems getting the package into NEW. Often it isn't clear what happened to a package that was rejected, the proposal makes it easy to find out; just check the bug reports for the source package. It also means the ftpmasters can file separate bugs for each problem, rather than combining them all into one mail. It also means the ftpmasters can easily point out other issues they noticed that don't block the package from being accepted and ensure that those issues won't be forgotten. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
On Fri, 2022-04-29 at 18:08 +0200, Andreas Tille wrote: > To give some actual examples that IMHO are candidates for accept + bug > report: Packages with incomplete or incorrect debian/copyright information currently would recieve a REJECT rather than acceptance. The proposal would not change that, just turn that REJECT into a bug report, which you could fix in a second upload to NEW and then the package would be reprocessed and get an ACCEPT or another bug. > I think Paul was not talking about non-distributable software but rather > code that is considered DFSG free but missing proper paragraphs inside > d/copyright which can be easily fixed in BTS. I think completely non-distributable software should get a bug against the package in NEW tagged as wontfix and then the NEW package should get removed with a REJECT mail mentioning the reason. There may be cases where it might be possible to fix some issue with a unredistributable file in an otherwise redistributable package, in those cases a serious bug against the package in NEW should be filed and the maintainer be given a chance to fix the issue. The proposal doesn't change anything with regard to packages with incomplete or incorrect debian/copyright information, that is a separate issue to what this proposal aims to solve. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
On April 29, 2022 11:04:57 PM UTC, Paul Wise wrote: >On Fri, 2022-04-29 at 13:36 +0100, Steve McIntyre wrote: > >> Just to clarify: is this suggesting that packages from NEW would end >> up in the archive even with serious bugs? If not, what's the point of >> the "eventual removal" above? I'm not following you here... > >No, the bugs I propose to be filed would be filed *while* the package >is in NEW and then *after* the serious ones are all fixed, that is a >positive indication that the package is probably suitable for an ACCEPT >action by the ftpmasters, although of course a final review is needed. I don't understand why this is any better than just rejecting the package? Once it's been determined that the upload won't be accepted, I don't see a benefit to having it remain in New. Scott K
Re: feedback for NEW packages: switch to using the BTS?
On Fri, 2022-04-29 at 11:26 -0400, Scott Kitterman wrote: > 1. Making reject/prod emails more public: > > There are actually two different types of messages we can send while > reviewing > packages in DAK: > > a. Reject: As indicated by the name, this is an email that accompanies the > package being rejected from the New queue. > > b. Prod: These leave the package in New and are used to ask the maintainer a > question. The proposal essentially turns most Reject mails into Prod mails and turns all Prod mails to bugs. The only Reject mails that will be sent are if the bugs aren't being solved after a certain amount of time, then the packages will be removed from NEW and get a REJECT. > My recollection is that these go to the maintainer and changed by. In many > cases the maintainer is a team, so these are already not considered private. That makes it seem more reasonable to make everything public, also I note the REJECT mails are available to all Debian members. ssh://mirror.ftp-master.debian.org/srv/ftp-master.debian.org/queue/reject/ I have been reading these files and note that many of them are automated, probably the most common one is people mistakenly doing source-only uploads to NEW. I have another idea to add a pre-reject API to dakweb for use by dupload/dput/dput-ng but haven't a proposal yet. > Whatever is done in this area should definitely be integrated into DAK as > part > of process-new. I don't think it would be helpful to try to add steps to the > work the FTP Team has to do. New is slow enough that we should definitely > not > make is slower. That seems reasonable. > Also, if any of this will depend on the existence of a WNPP bug, then policy > should be changed to require them. They are currently not required and not > everyone files them. If something is going to be required to get a package > accepted, it should be in policy. The proposal was created because I think filing WNPP bugs for every NEW upload is very much suboptimal, so I came up with the proposal as an alternative to that, so I'd avoid filing additional WNPP bugs for NEW above the existing ITP/etc bugs. > On a related note: It is not unusual for me to notice packaging issues which > are not serious enough to reject the package, but should be documented. One advantage of this proposal is that it allows filing bugs against any package already in NEW, so you could do that even if you notice a minor issue while processing a package, then have to move on to something else, then come back to the package the next day. > This is particularly true when an existing package goes through New due to a > new > binary package. It would be useful to have the option to accept with bug > report so that we don't need to go through the steps of filing a bug by hand. > > When I do this I also send a prod to the uploader/maintainer and it's > effectively doing work twice. Making New processing more efficient will help > make it faster. That seems like a useful feature, but I think I would separate this into two steps, first file the bug, maybe from dak and then accept. > 2. Not rejecting packages with serious defects: > > I'm not sure I understand what it proposed: > > > The ftpmasters could simply file severity serious bug reports against > > NEW packages that have issues blocking their entry into Debian. When > > there are minor issues noticed at the same time, then file bugs of a > > lower severity. Only when a NEW package has not had its serious bugs > > fixed in a long time would an eventual removal and REJECT mail happen, > > perhaps after a few months of zero action on the bug reports. > > I think this proposes to accept all packages regardless of how defective they > are and then remove them later if the bugs aren't fixed. If that's what is > proposed, my thought is absolutely not. That is definitely *not* what was proposed, and I think it is a bad idea to accept packages that are not suitable for the archive. The proposal is simply to change the NEW packages feedback mechanism and change the procedure for packages that don't yet meet Debian standards. > If a package is not suitable for the archive then it should be rejected with > appropriate feedback and re-uploaded. The proposal is to instead of rejecting the package, keep it in NEW, file a serious bug against the package in NEW, wait for it to be fixed and then reprocess the package. > Note: Although I am a member of the FTP Team, I am only speaking for myself > here, not the team as a whole. Thanks for your response. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
On Fri, 2022-04-29 at 13:36 +0100, Steve McIntyre wrote: > Just to clarify: is this suggesting that packages from NEW would end > up in the archive even with serious bugs? If not, what's the point of > the "eventual removal" above? I'm not following you here... No, the bugs I propose to be filed would be filed *while* the package is in NEW and then *after* the serious ones are all fixed, that is a positive indication that the package is probably suitable for an ACCEPT action by the ftpmasters, although of course a final review is needed. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
On Friday, April 29, 2022 12:08:21 PM EDT Andreas Tille wrote: > Hi Scott, > > thanks a lot for becoming involved into this discussion. > > Am Fri, Apr 29, 2022 at 11:26:33AM -0400 schrieb Scott Kitterman: > > 2. Not rejecting packages with serious defects: > > > > I'm not sure I understand what it proposed: > > > The ftpmasters could simply file severity serious bug reports against > > > NEW packages that have issues blocking their entry into Debian. When > > > there are minor issues noticed at the same time, then file bugs of a > > > lower severity. Only when a NEW package has not had its serious bugs > > > fixed in a long time would an eventual removal and REJECT mail happen, > > > perhaps after a few months of zero action on the bug reports. > > > > I think this proposes to accept all packages regardless of how defective > > they are and then remove them later if the bugs aren't fixed. If that's > > what is proposed, my thought is absolutely not. > > > > If a package is not suitable for the archive then it should be rejected > > with appropriate feedback and re-uploaded. > > To give some actual examples that IMHO are candidates for accept + bug > report: > >1. In case versioneer.py (Creative Commons "Public Domain Dedication" > license (CC0-1.0) is missing in d/copyright like in propka[1] > >2. Packages which are DFSG free but might miss some single copyright > statement. > My favourite example would be r-bioc-basilisk[2]. In this specific > example I even disagree with ftpmaster[3] but I see no real chance > as a maintainer to discuss my point and can only re-re-re-upload > what I consider correct. So I gave up and this package is not yet > inside Debian. This could be discussed more sensibly in a bug > report IMHO. > > I think Paul was not talking about non-distributable software but rather > code that is considered DFSG free but missing proper paragraphs inside > d/copyright which can be easily fixed in BTS. The in that case, it's just a variant of accept plus file a bug. I don't think it's productive to get into a discussion on the distinction between "buggy" and "too buggy to go in the archive". It's very dependent on the specifics of the situation. Also, accept with severe bugs and remove later if not fixed requires someone to track these issues and follow-up on getting them removed. That seems like more work that isn't easily automated (we don't do automatic removals from Unstable now - they are at most semi-automatic where an FTP Team member still needs to review and do the actual removal). To summarize: The reject/prod messages are already not considered private. Any additional addressing of them to whatever location in the BTS is a matter of have the BTS be ready for it and adding it to process-new to send it. If the answer is that ITP bugs are now required, then policy should be updated first. It would be useful to have a mechanism within DAK process-new to file a bug with the prod note of the appropriate severity. I think if this existed you would be inclined to see more accept with bug resolutions. I think accepting something to buggy to remain in unstable and removing it later if not fixed is not something that should be pursued. Still not speaking for the FTP Team. Scott K signature.asc Description: This is a digitally signed message part.
Re: feedback for NEW packages: switch to using the BTS?
Hi Scott, thanks a lot for becoming involved into this discussion. Am Fri, Apr 29, 2022 at 11:26:33AM -0400 schrieb Scott Kitterman: > 2. Not rejecting packages with serious defects: > > I'm not sure I understand what it proposed: > > > The ftpmasters could simply file severity serious bug reports against > > NEW packages that have issues blocking their entry into Debian. When > > there are minor issues noticed at the same time, then file bugs of a > > lower severity. Only when a NEW package has not had its serious bugs > > fixed in a long time would an eventual removal and REJECT mail happen, > > perhaps after a few months of zero action on the bug reports. > > I think this proposes to accept all packages regardless of how defective they > are and then remove them later if the bugs aren't fixed. If that's what is > proposed, my thought is absolutely not. > > If a package is not suitable for the archive then it should be rejected with > appropriate feedback and re-uploaded. To give some actual examples that IMHO are candidates for accept + bug report: 1. In case versioneer.py (Creative Commons "Public Domain Dedication" license (CC0-1.0) is missing in d/copyright like in propka[1] 2. Packages which are DFSG free but might miss some single copyright statement. My favourite example would be r-bioc-basilisk[2]. In this specific example I even disagree with ftpmaster[3] but I see no real chance as a maintainer to discuss my point and can only re-re-re-upload what I consider correct. So I gave up and this package is not yet inside Debian. This could be discussed more sensibly in a bug report IMHO. I think Paul was not talking about non-distributable software but rather code that is considered DFSG free but missing proper paragraphs inside d/copyright which can be easily fixed in BTS. > Note: Although I am a member of the FTP Team, I am only speaking for myself > here, not the team as a whole. Thanks a lot for speaking at a "competent yourself" ;-) Kind regards Andreas. [1] https://alioth-lists.debian.net/pipermail/debian-med-packaging/2022-February/098605.html [2] https://alioth-lists.debian.net/pipermail/r-pkg-team/2022-February/024165.html [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=991859#42 -- http://fam-tille.de
Re: feedback for NEW packages: switch to using the BTS?
On Wednesday, April 27, 2022 8:54:05 PM EDT Paul Wise wrote: > Hi all, > > During the discussions about NEW on debian-devel in recent times, I had > the idea that instead of the current mechanism of sending REJECT mails, > Debian could switch to using the BTS for most feedback on NEW packages. > > This means that most discussion about NEW packages would become public, > but of course the ftpmasters could opt to send private mail instead if > in some cases if there were sensitive issues to be discussed. > > The ftpmasters could simply file severity serious bug reports against > NEW packages that have issues blocking their entry into Debian. When > there are minor issues noticed at the same time, then file bugs of a > lower severity. Only when a NEW package has not had its serious bugs > fixed in a long time would an eventual removal and REJECT mail happen, > perhaps after a few months of zero action on the bug reports. > > The changes that are needed to make this happen include: > > The community needs to decide that this change is a good idea and be > willing to recieve most feedback on their work in public. > > The BTS and ftpmaster teams need to agree with this change. > One BTS admin said this is feasible in principle and one of the > ftpmasters seemed to like the idea when I mentioned it on IRC. > > Where the bug mail should go to needs to be decided on. Personally I'm > thinking the person who did the upload plus the Maintainer field. > > The people doing processing of NEW packages need to be willing to file > bug reports against them where necessary. > > dak needs to export debian/changelog version lists for NEW packages. > > debbugs needs to import packages/versions from the new.822 file: > > https://ftp-master.debian.org/new.822 > > dak needs to link to the BTS from new.822 and the NEW packages info. > > https://ftp-master.debian.org/new.html > > The technical work needed here is minimal, so if there are no other > volunteers to do that work, then I would be willing to do it. I have > experience with Perl/Python but only a small amount of experience with > working on the debbugs and dak codebases. > > Thoughts welcome, especially from those who don't like this idea. This proposal has a number of parts and I'm not entirely confident I understand what it being proposed. A few thoughts/clarifications. 1. Making reject/prod emails more public: There are actually two different types of messages we can send while reviewing packages in DAK: a. Reject: As indicated by the name, this is an email that accompanies the package being rejected from the New queue. b. Prod: These leave the package in New and are used to ask the maintainer a question. My recollection is that these go to the maintainer and changed by. In many cases the maintainer is a team, so these are already not considered private. Whatever is done in this area should definitely be integrated into DAK as part of process-new. I don't think it would be helpful to try to add steps to the work the FTP Team has to do. New is slow enough that we should definitely not make is slower. Also, if any of this will depend on the existence of a WNPP bug, then policy should be changed to require them. They are currently not required and not everyone files them. If something is going to be required to get a package accepted, it should be in policy. On a related note: It is not unusual for me to notice packaging issues which are not serious enough to reject the package, but should be documented. This is particularly true when an existing package goes through New due to a new binary package. It would be useful to have the option to accept with bug report so that we don't need to go through the steps of filing a bug by hand. When I do this I also send a prod to the uploader/maintainer and it's effectively doing work twice. Making New processing more efficient will help make it faster. 2. Not rejecting packages with serious defects: I'm not sure I understand what it proposed: > The ftpmasters could simply file severity serious bug reports against > NEW packages that have issues blocking their entry into Debian. When > there are minor issues noticed at the same time, then file bugs of a > lower severity. Only when a NEW package has not had its serious bugs > fixed in a long time would an eventual removal and REJECT mail happen, > perhaps after a few months of zero action on the bug reports. I think this proposes to accept all packages regardless of how defective they are and then remove them later if the bugs aren't fixed. If that's what is proposed, my thought is absolutely not. If a package is not suitable for the archive then it should be rejected with appropriate feedback and re-uploaded. Note: Although I am a member of the FTP Team, I am only speaking for myself here, not the team as a whole. Scott K signature.asc Description: This is a digitally signed message part.
Re: feedback for NEW packages: switch to using the BTS?
On 2022-04-29 at 08:36, Steve McIntyre wrote: > Paul Wise wrote: > >> During the discussions about NEW on debian-devel in recent times, I >> had the idea that instead of the current mechanism of sending >> REJECT mails, Debian could switch to using the BTS for most >> feedback on NEW packages. >> >> This means that most discussion about NEW packages would become >> public, but of course the ftpmasters could opt to send private mail >> instead if in some cases if there were sensitive issues to be >> discussed. >> >> The ftpmasters could simply file severity serious bug reports >> against NEW packages that have issues blocking their entry into >> Debian. When there are minor issues noticed at the same time, then >> file bugs of a lower severity. Only when a NEW package has not had >> its serious bugs fixed in a long time would an eventual removal and >> REJECT mail happen, perhaps after a few months of zero action on >> the bug reports. > > Just to clarify: is this suggesting that packages from NEW would end > up in the archive even with serious bugs? If not, what's the point > of the "eventual removal" above? I'm not following you here... I parsed that not as "removal from the archive" but as "removal from the NEW queue", much as now happens (in some order and via some mechanism, perhaps a manual one) when a REJECT mail is sent. -- The Wanderer The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. -- George Bernard Shaw signature.asc Description: OpenPGP digital signature
Re: feedback for NEW packages: switch to using the BTS?
Paul Wise wrote: > >During the discussions about NEW on debian-devel in recent times, I had >the idea that instead of the current mechanism of sending REJECT mails, >Debian could switch to using the BTS for most feedback on NEW packages. > >This means that most discussion about NEW packages would become public, >but of course the ftpmasters could opt to send private mail instead if >in some cases if there were sensitive issues to be discussed. > >The ftpmasters could simply file severity serious bug reports against >NEW packages that have issues blocking their entry into Debian. When >there are minor issues noticed at the same time, then file bugs of a >lower severity. Only when a NEW package has not had its serious bugs >fixed in a long time would an eventual removal and REJECT mail happen, >perhaps after a few months of zero action on the bug reports. Just to clarify: is this suggesting that packages from NEW would end up in the archive even with serious bugs? If not, what's the point of the "eventual removal" above? I'm not following you here... -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: feedback for NEW packages: switch to using the BTS?
Hi Paul, Am Fri, Apr 29, 2022 at 07:13:42AM +0800 schrieb Paul Wise: > > The packaging work is supposed to start *after* the ITP is filed, Sure. > so > this is a suboptimal order to do things in and doesn't provide much > value, The "packaging work" to create the debian/ dir is a 1min process since for R this is automated. Do you expect a racing condition in this minute? > except as a way to prevent people packaging the same R library > during the wait in NEW, but the presence of the R library in the R-pkg > team VCS already provides that, so the ITP bug isn't really needed. The automated process also checks existing WNPP bugs so it makes sense to file such a bug. Moreover it is used to document rejects. I don't want to say that WNPP bugs are a better solution than your proposal. My point was simply that there is some existing relation between BTS and NEW which might be usable or not. > Even when filing ITPs before packaging, for language ecosystem teams > where all library packages for the language go through the same team, > team co-ordination mechanisms are probably enough of a way to prevent > duplication of work, and that work is mostly automated anyway for > several such teams, so the ITP bug mostly isn't really needed. I confirm that it is not strictly needed (that's another reason why we feel pretty safe to file the ITP after the packaging) but as I explained it has some use anyway and since it is so brain dead simple to create we do it. > Some teams make exceptions for packages that aren't strictly just > libraries; for eg Rust folks to ITPs for things that ship executables, > so that people can hear about things they might want to use on Debian. > > > That's true. I just wanted to mention that some of your ideas > > are in a way used even now. > > Yeah, the proposal was derived from the suggestion to file ITPs for all > NEW packages, mostly aimed at avoiding the deficiencies with that idea. Nice. Kind regards Andreas. -- http://fam-tille.de
Re: feedback for NEW packages: switch to using the BTS?
On Thu, 2022-04-28 at 20:24 +0200, Andreas Tille wrote: > But filing an ITP bug is cheap. The R-pkg team has a script > itp_from_debian_dir[1] which creates this bug automatically once the > packaging work is done. The packaging work is supposed to start *after* the ITP is filed, so this is a suboptimal order to do things in and doesn't provide much value, except as a way to prevent people packaging the same R library during the wait in NEW, but the presence of the R library in the R-pkg team VCS already provides that, so the ITP bug isn't really needed. Even when filing ITPs before packaging, for language ecosystem teams where all library packages for the language go through the same team, team co-ordination mechanisms are probably enough of a way to prevent duplication of work, and that work is mostly automated anyway for several such teams, so the ITP bug mostly isn't really needed. Some teams make exceptions for packages that aren't strictly just libraries; for eg Rust folks to ITPs for things that ship executables, so that people can hear about things they might want to use on Debian. > That's true. I just wanted to mention that some of your ideas > are in a way used even now. Yeah, the proposal was derived from the suggestion to file ITPs for all NEW packages, mostly aimed at avoiding the deficiencies with that idea. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
On Thu, 2022-04-28 at 13:48 +0200, Marco d'Itri wrote: > This looks like a good idea to me, but I think that the big problem > which needs to be solved is not discussing REJECTs but the packages > which stay in NEW for many months with no feedback. Thanks. The idea is narrowly aimed at improving the transparency and communication issues with NEW. It would probably mean packages packages stay in NEW for longer, rather than getting a REJECT, then reuploaded, more issues found, another REJECT etc (not sure if that happens tho). As I understand it, there is currently no specified criteria for prioritisation of processing of NEW packages, but requesting a review on #debian-ftp for particular reasons (RC bug fixes, new binaries, waiting time etc) often means that someone will review it much sooner. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
Hi Paul, Am Thu, Apr 28, 2022 at 06:46:43PM +0800 schrieb Paul Wise: > There are two main categories of NEW packages; source and binary. > Packages adding an new source should have an ITP bug, but don't > always, for eg the Rust/Golang teams don't file them for every > library. Packages adding a new binary package often have a transition > bug, but those are usually filed after the upload is accepted. > > Filing a new WNPP bug for every NEW package upload seems a bit tedious > and I think lots of people aren't going to bother/remember to do it. But filing an ITP bug is cheap. The R-pkg team has a script itp_from_debian_dir[1] which creates this bug automatically once the packaging work is done. > The proposal I make would mean that the manual bug filing for every > upload wouldn't be needed, instead the BTS would know about packages > in NEW and ftpmasters and others could file bugs against them, which > would mainly only be needed when there are REJECTable issues present. > > In addition there could be separate bugs for each individual issue, > rather than one long discussion about multiple different issues. > This is probably especially important for low severity issues. I'm fine with this suggestion as well. > > (BTW, I usually bounce any reject to the according WNPP bug to have > > the issue documented in a publicly visible place where any interested > > person should look.) > > That seems reasonable if the ftpmasters are fine with that. Ftpmaster did not agreed with my suggestion so far. > If the > BTS for NEW proposal gets implemented it would no longer be needed. That's true. I just wanted to mention that some of your ideas are in a way used even now. > Am Thu, Apr 28, 2022 at 08:54:05AM +0800 schrieb Paul Wise: > > > > The changes that are needed to make this happen include: > > I guess tracker.d.o would also need to show NEW packages. This would be extremely helpful. Kind regards Andreas. [1] https://salsa.debian.org/r-pkg-team/dh-r/-/blob/master/scripts/itp_from_debian_dir -- http://fam-tille.de
Re: feedback for NEW packages: switch to using the BTS?
On Apr 28, Paul Wise wrote: > During the discussions about NEW on debian-devel in recent times, I had > the idea that instead of the current mechanism of sending REJECT mails, > Debian could switch to using the BTS for most feedback on NEW packages. This looks like a good idea to me, but I think that the big problem which needs to be solved is not discussing REJECTs but the packages which stay in NEW for many months with no feedback. -- ciao, Marco signature.asc Description: PGP signature
Re: feedback for NEW packages: switch to using the BTS?
On Thu, 2022-04-28 at 09:36 +0200, Andreas Tille wrote: > What about WNPP bug? When I asked ftpmaster to kindly CC their > rejects to the WNPP bug I was told that not all packages in new have WNPP > bugs. If we want to formalise this it could probably be enforced that new > packages really need to have such a bug and we only need to decide > for existing packages that need to pass NEW. There are two main categories of NEW packages; source and binary. Packages adding an new source should have an ITP bug, but don't always, for eg the Rust/Golang teams don't file them for every library. Packages adding a new binary package often have a transition bug, but those are usually filed after the upload is accepted. Filing a new WNPP bug for every NEW package upload seems a bit tedious and I think lots of people aren't going to bother/remember to do it. The proposal I make would mean that the manual bug filing for every upload wouldn't be needed, instead the BTS would know about packages in NEW and ftpmasters and others could file bugs against them, which would mainly only be needed when there are REJECTable issues present. In addition there could be separate bugs for each individual issue, rather than one long discussion about multiple different issues. This is probably especially important for low severity issues. > (BTW, I usually bounce any reject to the according WNPP bug to have > the issue documented in a publicly visible place where any interested > person should look.) That seems reasonable if the ftpmasters are fine with that. If the BTS for NEW proposal gets implemented it would no longer be needed. Am Thu, Apr 28, 2022 at 08:54:05AM +0800 schrieb Paul Wise: > > The changes that are needed to make this happen include: I guess tracker.d.o would also need to show NEW packages. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
Hi Paul, Am Thu, Apr 28, 2022 at 08:54:05AM +0800 schrieb Paul Wise: > During the discussions about NEW on debian-devel in recent times, I had > the idea that instead of the current mechanism of sending REJECT mails, > Debian could switch to using the BTS for most feedback on NEW packages. > > This means that most discussion about NEW packages would become public, > but of course the ftpmasters could opt to send private mail instead if > in some cases if there were sensitive issues to be discussed. > > The ftpmasters could simply file severity serious bug reports against > NEW packages that have issues blocking their entry into Debian. When > there are minor issues noticed at the same time, then file bugs of a > lower severity. Only when a NEW package has not had its serious bugs > fixed in a long time would an eventual removal and REJECT mail happen, > perhaps after a few months of zero action on the bug reports. I consider this a really nice solution for the majority of rejects and I think a lot of good arguments were given inside the discussion you was refering to. > The changes that are needed to make this happen include: > > The community needs to decide that this change is a good idea and be > willing to recieve most feedback on their work in public. > > The BTS and ftpmaster teams need to agree with this change. > One BTS admin said this is feasible in principle and one of the > ftpmasters seemed to like the idea when I mentioned it on IRC. I admit I would be happy if this "seemed to like the idea" would be confirmed here in a public place. ;-) > Where the bug mail should go to needs to be decided on. Personally I'm > thinking the person who did the upload plus the Maintainer field. What about WNPP bug? When I asked ftpmaster to kindly CC their rejects to the WNPP bug I was told that not all packages in new have WNPP bugs. If we want to formalise this it could probably be enforced that new packages really need to have such a bug and we only need to decide for existing packages that need to pass NEW. (BTW, I usually bounce any reject to the according WNPP bug to have the issue documented in a publicly visible place where any interested person should look.) > The people doing processing of NEW packages need to be willing to file > bug reports against them where necessary. > > dak needs to export debian/changelog version lists for NEW packages. > > debbugs needs to import packages/versions from the new.822 file: > > https://ftp-master.debian.org/new.822 > > dak needs to link to the BTS from new.822 and the NEW packages info. > > https://ftp-master.debian.org/new.html > > The technical work needed here is minimal, so if there are no other > volunteers to do that work, then I would be willing to do it. I have > experience with Perl/Python but only a small amount of experience with > working on the debbugs and dak codebases. > > Thoughts welcome, especially from those who don't like this idea. Sorry, I'm amongst those who absolutely like this idea. ;-) Thanks a lot for the follow up on the past discussion. Kind regards Andreas. -- http://fam-tille.de
feedback for NEW packages: switch to using the BTS?
Hi all, During the discussions about NEW on debian-devel in recent times, I had the idea that instead of the current mechanism of sending REJECT mails, Debian could switch to using the BTS for most feedback on NEW packages. This means that most discussion about NEW packages would become public, but of course the ftpmasters could opt to send private mail instead if in some cases if there were sensitive issues to be discussed. The ftpmasters could simply file severity serious bug reports against NEW packages that have issues blocking their entry into Debian. When there are minor issues noticed at the same time, then file bugs of a lower severity. Only when a NEW package has not had its serious bugs fixed in a long time would an eventual removal and REJECT mail happen, perhaps after a few months of zero action on the bug reports. The changes that are needed to make this happen include: The community needs to decide that this change is a good idea and be willing to recieve most feedback on their work in public. The BTS and ftpmaster teams need to agree with this change. One BTS admin said this is feasible in principle and one of the ftpmasters seemed to like the idea when I mentioned it on IRC. Where the bug mail should go to needs to be decided on. Personally I'm thinking the person who did the upload plus the Maintainer field. The people doing processing of NEW packages need to be willing to file bug reports against them where necessary. dak needs to export debian/changelog version lists for NEW packages. debbugs needs to import packages/versions from the new.822 file: https://ftp-master.debian.org/new.822 dak needs to link to the BTS from new.822 and the NEW packages info. https://ftp-master.debian.org/new.html The technical work needed here is minimal, so if there are no other volunteers to do that work, then I would be willing to do it. I have experience with Perl/Python but only a small amount of experience with working on the debbugs and dak codebases. Thoughts welcome, especially from those who don't like this idea. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
