Re: [PATCH] improving github experience, kindly ask people to reproduce bugs on latest haproxy
Hi Lukas, On Tue, Sep 24, 2019 at 04:56:14PM +0200, Lukas Tribus wrote: > I'd suggest to release 2.0.7, seeing as how many people are affected > by this (and 2 maintainers - FreeBSD and Vincent's private uptodate > build for Debian/Ubuntu - already ship builds with ab160a47ac > reverted). Definitely, but Christopher and I are still checkinga few nasty things on H2, I'd hate to have to emit 2.0.8 next week you see ;-) But in the worst case 2.0.7 should be issued in a matter a no more than a few days, as I agree that the issue with the checks is really painful. > As for the template change, let's maintain the situation as is then. Agreed. Thanks! Willy
Re: [PATCH] improving github experience, kindly ask people to reproduce bugs on latest haproxy
Hello Tim, Willy, On Tue, Sep 24, 2019 at 2:48 PM Tim Düsterhus wrote: > > Unless you feel that it's starting to cause too much work on your side, > > from my developer's perspective at least it's still manageable as it is > > right now, with a positive balance, so I'd also be all for keeping it > > as it is, possibly just rewording a little bit or so if needed. > > > > I did not do much in the last weeks in the issue tracker, mainly because > someone else was faster than me. The turnaround time from report to fix > sometimes is very impressive and the issue is already closed fixed > before I see it. > > However I read all the notifications and currently I don't see a need > for issue template adjustments regarding version upgrades. > > To me it feels that the recent duplicates are mostly only fixed in git > and not yet released as a proper version. So users would need to upgrade > to some intermediate version or apply a patch. This is hardly something > that we should expect of them. > So the "problem" is not "users don't upgrade", but rather "users don't > have something to upgrade to". Speaking of which; we have two high profile fixes in the 2.0 tree currently, the health check fix and the header mangling fix: 0f0393fc0d2ba #278 6884aa3eb00d #116 #290 #292 I'd suggest to release 2.0.7, seeing as how many people are affected by this (and 2 maintainers - FreeBSD and Vincent's private uptodate build for Debian/Ubuntu - already ship builds with ab160a47ac reverted). As for the template change, let's maintain the situation as is then. Lukas
Re: [PATCH] improving github experience, kindly ask people to reproduce bugs on latest haproxy
Willy, Lukas, Am 24.09.19 um 04:49 schrieb Willy Tarreau: >> I'm actually also fine with maintaining the current template as-is and >> I agree the issue tracker needs to be used to collect informations on >> unkown bugs, which is especially useful if there are multiple >> reporters. > > This definitely is one of the most important value it has brought so far. > >> I just think the bar can be a little bit higher without >> negative impact. > > Unless you feel that it's starting to cause too much work on your side, > from my developer's perspective at least it's still manageable as it is > right now, with a positive balance, so I'd also be all for keeping it > as it is, possibly just rewording a little bit or so if needed. > I did not do much in the last weeks in the issue tracker, mainly because someone else was faster than me. The turnaround time from report to fix sometimes is very impressive and the issue is already closed fixed before I see it. However I read all the notifications and currently I don't see a need for issue template adjustments regarding version upgrades. To me it feels that the recent duplicates are mostly only fixed in git and not yet released as a proper version. So users would need to upgrade to some intermediate version or apply a patch. This is hardly something that we should expect of them. So the "problem" is not "users don't upgrade", but rather "users don't have something to upgrade to". Best regards Tim Düsterhus
Re: [PATCH] improving github experience, kindly ask people to reproduce bugs on latest haproxy
Hi Lukas, On Tue, Sep 24, 2019 at 01:22:38AM +0200, Lukas Tribus wrote: > I realize I did not explain my point very well. I have to emphasize > that there is an important distinction between what we suggest in the > bug reporting template and what we actually *enforce*. Enforcements > happen considering the context, and that context is often more > important than actual rules. I believe we have done a good job > understanding and considering that context in the nearly 300 issue's > so far. I think so as well. If you remember, initially I was the one who wanted that only a few of people could create issues, in order to avoid the issue tracker becoming a horrible mess. You and Tim suggested that it would work fine the way it works now and I accepted to test without being much convinced. As you can see you were right and I was wrong, because I think we mainly have a good quality in the bug reports (and the template does help a lot, just like the periodic cleanup you do there by chasing duplicates or closing those already fixed). My focus remains on limiting the overhead for people who can be a bottleneck, like the core developers and those sorting all these bugs like you do. Thus I always balance the savings on one side and the extra cost that could happen on the other side. And here I think that *right now* with users being autonomous enough to post their reports we don't have to do it ourselves, and keeping such a good balance while watching for risks of overloading the central people seems optimal. > On the other hand, important reports come in with > good informations in older release and are *clearly* relevant. But if > we have a report in 2.0.4 it is obvious that the OP did not install > haproxy from a CentOs repository, so asking for an upgrade is not some > far fetched fantasy. Sure but the primary purpose for which I initially wanted an issue tracker was for the hard reproducers that we were used to forget and that I was keeping enumerated in my own "TODO" e-mail. And where this issue tracker really helps is by helping to collect such information. When these rare reproducers happen, they rarely are on the latest version and this equation is hard to solve for users waiting for the next core, watchdog, deadlock, or strange issue to happen again. Despite this, developers seeing these reports can often judge whether this bug sounds totally new or not. I've see some excellent quality reports from William Dauchy and from Patrik there, I don't even remember if they were always on the last one but definitely they were insane bugs that I wanted to understand. > Context is key to understanding whether or not to > apply a rule (or whether to ask to reproduce in the latest release). > > I did not mean we should reject all issues that are not reproduced in > the latest release. However I do not feel like raising barriers to > file new issues is detrimental because in the absolute worst case, > people will ask about their problem on the mailing list or on > discourse as opposed to directly filing the issue; This was my initial expectation and based on how it works now I've figured I was wrong. Do you think we have that many bugs reported on too old versions that it is causing an issue (and possibly too much work on your side, because I know that I'm only seeing what remains from your cleanups) ? > > [ ] I confirm that I did my best to upgrade to the latest version > > before reporting this bug and I am also conscious that developers > > tend to be much less reactive on older versions. > > Frankly I'd not use this particular statement, because this wording > sounds vague to me and I feel like people can read into it whatever > they want to hear, including support for the 1.2 branch given enough > patience on their end. Also people fill out annoying forms every day, > a checkbox that begins with the words "I can confirm that I did my > best ..." is just a psychological NOP, personally I'm not sure I'd > actually read further than those words, immediately hitting that > checkbox. That's true. Thus maybe we can simply add this information (not a checkbox) : Bugs are expected to be reported on the latest version only. Beware that issues not respecting this expectation may simply be ignored or closed without explanation if developers think they match an already fixed issue. If you're facing constraints making it difficult for you to report on the latest version, you're highly encouraged to discuss your issue on the mailing list or discourse. This way we instruct the user while keeping the door open. > > Just thinking (not sure it's a good idea as it could attract too many > > reports), maybe we could have a check box that automatically sets an > > "incomplete" status for a bug report. This allows reporters to stick to > > facts and share them with others, this is convenient for rare bugs. > > We will have to check whether it's possible to map checkboxes to > labels, but yes, I'd be
Re: [PATCH] improving github experience, kindly ask people to reproduce bugs on latest haproxy
Hello Willy, On Mon, Sep 23, 2019 at 7:02 PM Willy Tarreau wrote: > > However it needs to be clear that the issue tracker in Github is not a > > support forum, and that filing a report will needs some ground work. > > It is also *always* a good idea to discuss an issue on the ML first, > > before filing an issue. > > I'm a bit torn between you two on this actually. There are several > reasons. The first one is that nobody is running the latest version > since a latest exists after a commit is pushed, and that at this game > it can circle forever. Some would say "but not released doesn't count" > and I'd argue that *new* bugs affecting -dev are even more important > to me than older ones since they are regressions, which is also why we > now have the CI running. > > However I also agree that it's important to encourage users to upgrade > *when they can*. > [...] > My hope is that we can encourage good faith among the reporters. I realize I did not explain my point very well. I have to emphasize that there is an important distinction between what we suggest in the bug reporting template and what we actually *enforce*. Enforcements happen considering the context, and that context is often more important than actual rules. I believe we have done a good job understanding and considering that context in the nearly 300 issue's so far. People can spend 3 minutes thinking about an issue, not reading the documentation or doing some basic due diligence and still fill out the template properly (including flagging whatever checkboxes we throw at them obviously). On the other hand, important reports come in with good informations in older release and are *clearly* relevant. But if we have a report in 2.0.4 it is obvious that the OP did not install haproxy from a CentOs repository, so asking for an upgrade is not some far fetched fantasy. Context is key to understanding whether or not to apply a rule (or whether to ask to reproduce in the latest release). I did not mean we should reject all issues that are not reproduced in the latest release. However I do not feel like raising barriers to file new issues is detrimental because in the absolute worst case, people will ask about their problem on the mailing list or on discourse as opposed to directly filing the issue; which is one additional round-trip as a consequence for a "false-positive" and at the same time, we may reduce the noise a lot. Templates and the rules implied in it don't apply to responses in issues (from other people), so nobody willing to provide additional infos to existing issues is impacted anyway. > [ ] I confirm that I did my best to upgrade to the latest version > before reporting this bug and I am also conscious that developers > tend to be much less reactive on older versions. Frankly I'd not use this particular statement, because this wording sounds vague to me and I feel like people can read into it whatever they want to hear, including support for the 1.2 branch given enough patience on their end. Also people fill out annoying forms every day, a checkbox that begins with the words "I can confirm that I did my best ..." is just a psychological NOP, personally I'm not sure I'd actually read further than those words, immediately hitting that checkbox. > Just thinking (not sure it's a good idea as it could attract too many > reports), maybe we could have a check box that automatically sets an > "incomplete" status for a bug report. This allows reporters to stick to > facts and share them with others, this is convenient for rare bugs. We will have to check whether it's possible to map checkboxes to labels, but yes, I'd be fine with that. I'm actually also fine with maintaining the current template as-is and I agree the issue tracker needs to be used to collect informations on unkown bugs, which is especially useful if there are multiple reporters. I just think the bar can be a little bit higher without negative impact. Lukas
Re: [PATCH] improving github experience, kindly ask people to reproduce bugs on latest haproxy
пн, 23 сент. 2019 г. в 22:02, Willy Tarreau : > Hi guys, > > On Fri, Sep 20, 2019 at 03:47:17PM +0200, Lukas Tribus wrote: > > Hello Patrick, > > > > > > > I dunno, I've personally never been fond of it when bug reporters are > blindly asked to upgrade to the latest version. > > > > Everything you are saying is relevant for a support environment, like > > the mailing list or discourse. However the bug tracker is not a > > support forum, it's a bug tracker only. We need factual data, > > everything else belongs to the support channels. We needed this to > > file known bugs and features requests so we stop forgetting about them > > in the stream of emails and to coordinate fixes of known, confirmed > > bugs. > > > > Nobody is saying that problems need to be reported on Github. People > > seeking help ought to report their issues to the mailing list. If on > > the other hand those people are willing to file a bug report - and > > this implies some groundwork for them - then that's great, because it > > helps us. > > > > However it needs to be clear that the issue tracker in Github is not a > > support forum, and that filing a report will needs some ground work. > > It is also *always* a good idea to discuss an issue on the ML first, > > before filing an issue. > > I'm a bit torn between you two on this actually. There are several > reasons. The first one is that nobody is running the latest version > since a latest exists after a commit is pushed, and that at this game > it can circle forever. Some would say "but not released doesn't count" > and I'd argue that *new* bugs affecting -dev are even more important > to me than older ones since they are regressions, which is also why we > now have the CI running. > > However I also agree that it's important to encourage users to upgrade > *when they can*. Patrick is right regarding the difficulty to upgrade > in some environments, I've worked in such environments and sometimes > you get a very strange behavior, you collect traces as fast as you can > and you want to archive your report in a safe place, shared by others, > knowing that it's incomplete but that it may help in the future when > it's confronted to another apparently similar one, coming from someone > with much less traces. In this regard the issue tracker is convenient > to keep some warm issues available and not forgotten even if we consider > them incomplete but credible. > > Similarly some issues which trigger very late can by definition never > happen on the latest version. For example, just remove 3 lines in the > scheduler to allow to loop at the end of the tree back to the beginning > and suddenly all haproxy code deployed will fail after 49.7 days. And > we do want to get such reports that are extremely rare and valuable by > nature. > > My hope is that we can encourage good faith among the reporters. I'd > suggest changing the question to something like this instead : > > [ ] I confirm that I did my best to upgrade to the latest version > before reporting this bug and I am also conscious that developers > tend to be much less reactive on older versions. > I'm ok with that. > > It doesn't necessarily *have* to be a check box, it just needs to be > prominent so that it's well understood. > > With all this said, we're starting to see more bugs with multiple > participants and this is encouraging because that's exactly what is > needed to help narrow down an issue. Thus I'm not for really filtering > at the door, but rather making sure that the reporter thinks twice and > decides. > > Typically those who install their fresh new PC with a default distro, > start the default haproxy (without updating the distro) and report an > issue should be reminded about trying again and upgrading first. But > those who have been chasing a bug for a month (like Veiko in the other > thread) an finally been hit by it with subtantial information (not > necessarily traces/conf, sometimes context explanations) should be > able to post if they think it does provide some value. > > > > In corporate environments, it can be difficult to perform such > upgrades. > > > Sometimes these issues are only reproducible in production > environments. > > > > I know that. But that's not a bug report, it's a support request > > Not necessarily. Regarding the example about the time looping at 49.7 days, > I actually experienced it with other products long ago. It definitely is a > bug. Just like I've seen some equipements get their VRRP desynchronized > progressively over time, the amount of desync grew by a few seconds in sine > waves of something like 24 days. The initially observable issue was that > sometimes there would be a quick failover and a switch back, and that over > a long period that would happen more often then less often. It's quite > difficult to qualify such a think but it definitely is a bug an not someone > asking for help. Sharing elements as they come can be useful, provided the > person accep
Re: [PATCH] improving github experience, kindly ask people to reproduce bugs on latest haproxy
Hi guys, On Fri, Sep 20, 2019 at 03:47:17PM +0200, Lukas Tribus wrote: > Hello Patrick, > > > > I dunno, I've personally never been fond of it when bug reporters are > > blindly asked to upgrade to the latest version. > > Everything you are saying is relevant for a support environment, like > the mailing list or discourse. However the bug tracker is not a > support forum, it's a bug tracker only. We need factual data, > everything else belongs to the support channels. We needed this to > file known bugs and features requests so we stop forgetting about them > in the stream of emails and to coordinate fixes of known, confirmed > bugs. > > Nobody is saying that problems need to be reported on Github. People > seeking help ought to report their issues to the mailing list. If on > the other hand those people are willing to file a bug report - and > this implies some groundwork for them - then that's great, because it > helps us. > > However it needs to be clear that the issue tracker in Github is not a > support forum, and that filing a report will needs some ground work. > It is also *always* a good idea to discuss an issue on the ML first, > before filing an issue. I'm a bit torn between you two on this actually. There are several reasons. The first one is that nobody is running the latest version since a latest exists after a commit is pushed, and that at this game it can circle forever. Some would say "but not released doesn't count" and I'd argue that *new* bugs affecting -dev are even more important to me than older ones since they are regressions, which is also why we now have the CI running. However I also agree that it's important to encourage users to upgrade *when they can*. Patrick is right regarding the difficulty to upgrade in some environments, I've worked in such environments and sometimes you get a very strange behavior, you collect traces as fast as you can and you want to archive your report in a safe place, shared by others, knowing that it's incomplete but that it may help in the future when it's confronted to another apparently similar one, coming from someone with much less traces. In this regard the issue tracker is convenient to keep some warm issues available and not forgotten even if we consider them incomplete but credible. Similarly some issues which trigger very late can by definition never happen on the latest version. For example, just remove 3 lines in the scheduler to allow to loop at the end of the tree back to the beginning and suddenly all haproxy code deployed will fail after 49.7 days. And we do want to get such reports that are extremely rare and valuable by nature. My hope is that we can encourage good faith among the reporters. I'd suggest changing the question to something like this instead : [ ] I confirm that I did my best to upgrade to the latest version before reporting this bug and I am also conscious that developers tend to be much less reactive on older versions. It doesn't necessarily *have* to be a check box, it just needs to be prominent so that it's well understood. With all this said, we're starting to see more bugs with multiple participants and this is encouraging because that's exactly what is needed to help narrow down an issue. Thus I'm not for really filtering at the door, but rather making sure that the reporter thinks twice and decides. Typically those who install their fresh new PC with a default distro, start the default haproxy (without updating the distro) and report an issue should be reminded about trying again and upgrading first. But those who have been chasing a bug for a month (like Veiko in the other thread) an finally been hit by it with subtantial information (not necessarily traces/conf, sometimes context explanations) should be able to post if they think it does provide some value. > > In corporate environments, it can be difficult to perform such upgrades. > > Sometimes these issues are only reproducible in production environments. > > I know that. But that's not a bug report, it's a support request Not necessarily. Regarding the example about the time looping at 49.7 days, I actually experienced it with other products long ago. It definitely is a bug. Just like I've seen some equipements get their VRRP desynchronized progressively over time, the amount of desync grew by a few seconds in sine waves of something like 24 days. The initially observable issue was that sometimes there would be a quick failover and a switch back, and that over a long period that would happen more often then less often. It's quite difficult to qualify such a think but it definitely is a bug an not someone asking for help. Sharing elements as they come can be useful, provided the person accepts that this issue will not be handled for a while. Just thinking (not sure it's a good idea as it could attract too many reports), maybe we could have a check box that automatically sets an "incomplete" status for a bug report. This all
Re: [PATCH] improving github experience, kindly ask people to reproduce bugs on latest haproxy
Hello Patrick, > I dunno, I've personally never been fond of it when bug reporters are blindly > asked to upgrade to the latest version. Everything you are saying is relevant for a support environment, like the mailing list or discourse. However the bug tracker is not a support forum, it's a bug tracker only. We need factual data, everything else belongs to the support channels. We needed this to file known bugs and features requests so we stop forgetting about them in the stream of emails and to coordinate fixes of known, confirmed bugs. Nobody is saying that problems need to be reported on Github. People seeking help ought to report their issues to the mailing list. If on the other hand those people are willing to file a bug report - and this implies some groundwork for them - then that's great, because it helps us. However it needs to be clear that the issue tracker in Github is not a support forum, and that filing a report will needs some ground work. It is also *always* a good idea to discuss an issue on the ML first, before filing an issue. But if you are unable to reproduce the problem on the latest release, your report definitely belongs to the mailing list. > In corporate environments, it can be difficult to perform such upgrades. > Sometimes these issues are only reproducible in production environments. I know that. But that's not a bug report, it's a support request and it belongs to the mailing list. Acked-by: Lukas Tribus Lukas
Re: [PATCH] improving github experience, kindly ask people to reproduce bugs on latest haproxy
*From:* Илья Шипицин [mailto:chipits...@gmail.com] *Sent:* Thursday, September 19, 2019, 15:10 EDT *To:* HAProxy *Subject:* [PATCH] improving github experience, kindly ask people to reproduce bugs on latest haproxy hello, please find attached patch Ilya Shipitsin I dunno, I've personally never been fond of it when bug reporters are blindly asked to upgrade to the latest version. Sometimes the request is justified, such as when the project maintainers have reason to believe the bug is fixed, or if the version is years old. But otherwise it can causes difficulties for the reporter. In corporate environments, it can be difficult to perform such upgrades. Sometimes these issues are only reproducible in production environments. So by asking them to upgrade, you're making them go through the difficulty, and potentially cause impact to their clients. And because that process can take a while, it's possible that by the time they do complete the upgrade, another version has been released. I personally also find the use of heavy bug templates, with nuanced little checkboxes to be annoying. In this case more so because we already ask them to provide the version information, which answers the question the checkbox is for. And the checkbox "yes i'm on the latest" might be accurate at the time of submission, but can become out of date. Now all that said, I'm not against encouraging people to try a later version if available. Just that I don't think it should be the expectation. -Patrick
