Re: [PATCH] REG-TEST: mailers: add new test for 'mailers' section
Hi Cyril, On Fri, Jan 11, 2019 at 12:35:12AM +0100, Cyril Bonté wrote: > Well, from what I've seen with a small test, I've a different conclusion > about the commit which introduced the issue. It looks to have been > introduced earlier with commit 0108bb3e4 "MEDIUM: mailers: Init alerts > during conf parsing and refactor their processing" > http://git.haproxy.org/?p=haproxy.git;a=commit;h=0108bb3e4 > > Hope this helps. Ah cool, sure it helps, I like it when we end up on one of Christopher's commits, usually he figures quickly where the problem is :-) Thanks! Willy
error while extracting 1.8.17 tar file
Hi we are getting error while extracting tar file 1.8.17 on windows error screen shot attched in mail [image: image.png] -- Thanks & Regards, Girish --
Re: [PATCH] REG-TEST: mailers: add new test for 'mailers' section
Hi all, Le 08/01/2019 à 10:06, Willy Tarreau a écrit : On Tue, Jan 08, 2019 at 09:31:22AM +0100, Frederic Lecaille wrote: Indeed this script could worked with a short mailer timeout before af4021e6 commit. Another git bisect shows that 53216e7d introduced the email bombing issue. Note that 33a09a5f refers to 53216e7d commit. I am not sure this can help. Sure it helps, at least to know whom to ping :-) Well, from what I've seen with a small test, I've a different conclusion about the commit which introduced the issue. It looks to have been introduced earlier with commit 0108bb3e4 "MEDIUM: mailers: Init alerts during conf parsing and refactor their processing" http://git.haproxy.org/?p=haproxy.git;a=commit;h=0108bb3e4 Hope this helps. -- Cyril Bonté
Re: Lots of mail from email alert on 1.9.x
Hi Pieter,
On Thu, Jan 10, 2019 at 08:05:47PM +0100, PiBa-NL wrote:
> Hi Johan, Olivier, Willy,
>
> Op 10-1-2019 om 17:00 schreef Johan Hendriks:
> > I just updated to 1.9.1 on my test system.
> >
> > We noticed that when a server fails we now get tons of mail, and with
> > tons we mean a lot.
> >
> > After a client backend server fails we usually get 1 mail on 1.8.x now
> > with 1.9.1 within 1 minute we have the following.
> >
> > mailq | grep -B2 l...@testdomain.nl | grep '^[A-F0-9]' | awk '{print
> > $1}' | sed 's/*//' | postsuper -d -
> > postsuper: Deleted: 19929 messages
> >
> > My setting from the backend part is as follows.
> >
> > email-alert mailers alert-mailers
> > email-alert from l...@testdomain.nl
> > email-alert to not...@testdomain.nl
> > server webserver09 11.22.33.44:80 check
> >
> > Has something changed in 1.9.x (it was on 1.9.0 also)
> >
> > regards
> > Johan Hendriks
> >
> >
> Its a 'known issue' see:
> https://www.mail-archive.com/haproxy@formilux.org/msg32290.html
> a 'regtest' is added in that mail thread also to aid developers in
> reproducing the issue and validating a possible fix.
>
> @Olivier, Willy, may i assume this mailbomb feature is 'planned' to get
> fixed in 1.9.2 ? (perhaps a bugtracker with a 'target version' would be nice
> ;) ?)
Planned to be worked on, sure, but we need to figure how all this stuff
is supposed to work at all first. And I will definitely not want to keep
1.9.2 hostage of this. So the sooner it's figured the better, but that's
all I can say at this point :-(
Cheers,
Willy
Re: Lots of mail from email alert on 1.9.x
Hi Johan, Olivier, Willy,
Op 10-1-2019 om 17:00 schreef Johan Hendriks:
I just updated to 1.9.1 on my test system.
We noticed that when a server fails we now get tons of mail, and with
tons we mean a lot.
After a client backend server fails we usually get 1 mail on 1.8.x now
with 1.9.1 within 1 minute we have the following.
mailq | grep -B2 l...@testdomain.nl | grep '^[A-F0-9]' | awk '{print
$1}' | sed 's/*//' | postsuper -d -
postsuper: Deleted: 19929 messages
My setting from the backend part is as follows.
email-alert mailers alert-mailers
email-alert from l...@testdomain.nl
email-alert to not...@testdomain.nl
server webserver09 11.22.33.44:80 check
Has something changed in 1.9.x (it was on 1.9.0 also)
regards
Johan Hendriks
Its a 'known issue' see:
https://www.mail-archive.com/haproxy@formilux.org/msg32290.html
a 'regtest' is added in that mail thread also to aid developers in
reproducing the issue and validating a possible fix.
@Olivier, Willy, may i assume this mailbomb feature is 'planned' to get
fixed in 1.9.2 ? (perhaps a bugtracker with a 'target version' would be
nice ;) ?)
Regards,
PiBa-NL (Pieter)
Re: haproxy issue tracker discussion
Willy, Am 10.01.19 um 19:40 schrieb Willy Tarreau: > Conclusion : the affected status is only temporary and enough to go > once the backport is done. This simply means we don't need a "fixed-1.9" > or whatever, we just have to remove the "affected" label exactly as it > would have been if the issue had been reported the day after. > > So in the end we can live with simply "affects-1.8" etc and remove these > flags once the fix(es) is/are backported. Makes sense. > We could decide that bugs for which no "affects-X.Y" label exist anymore > can be closed. That doesn't mean the issue doesn't affect older branch, > it means it's not known to affect them yet, which is similar to before > the bug report (since an issue tracker only tracks known defects and not > hypothetical ones). Ideally the backports would happen more timely to keep the list of "Open" issues clean. Otherwise some of the "Open" issues are completely unfixed, while for others "merely" the backport is missing. > With this model, there is less work for everyone involved, all the info > is concentrated together, users can see that their version remains bogus > because we don't know how to backport the fix but the next one is fixed > so it might be time to upgrade, and there's much less info duplication > leading to the inevitable consistency that comes from it. I guess that will work then. Best regards Tim Düsterhus
Re: haproxy issue tracker discussion
Hi Tim, On Thu, Jan 10, 2019 at 04:12:54PM +0100, Tim Düsterhus wrote: > > I tend to think that if labels already mark the relevance to a branch, > > then they override the status and probably we don't really care about > > the status. The "moby" project above does that by the way, with > > "status/foo" labels. We could possibly have had "v1.8-open" and > > "v1.8x-done". This way instead of searching by status users can > > search by clicking on the labels. I could just be missing something. > > When should the binary "issue open" / "issue closed" property be > toggled? When the issue is fixed in Dev? When the issue is fixed in the > lowest affected version? In fact, I've given some thinking to this and came to the conclusion that a status is only valid at the moment it is consulted. Example : - bug is reported as affecting 1.8 - it is diagnosed - it is figured that 1.9 is affected as well, but for an unkonwn reason, dev is not - it gets marked for 1.9, 1.8, all of which have to be fixed. - in the mean time another backport from dev to 1.9 happens to fix it, so the "affects 1.9" status is not true anymore. - later it is figured that dev was safe and 1.9 became safe thanks to this later commit => in this case both 1.8 and 1.9 were initially marked as affected But now imagine the exact same bug was reported only one day later : - a seemingly unrelated backport from dev to 1.9 protects 1.9 against the bug - bug is reported as affecting 1.8 - it is diagnosed - it is figured that 1.8 is the only one affected, but for an unkonwn reason, 1.9 and dev are not - later it is figured that dev and 1.9 were safe thanks to the former commit => in this case only 1.8 was marked as affected Conclusion : the affected status is only temporary and enough to go once the backport is done. This simply means we don't need a "fixed-1.9" or whatever, we just have to remove the "affected" label exactly as it would have been if the issue had been reported the day after. So in the end we can live with simply "affects-1.8" etc and remove these flags once the fix(es) is/are backported. > Both is not ideal, I guess. Maybe we need to automatically create follow > up issues for the various branches once the fix lands in dev: I really don't like this for the reason I explained : they cut the discussion in the middle most often to restart it in another one for the exact branch the issue was reported for. User reports issue on 1.9, we create it, then create another issue for 1.9, which makes no sense for users' point of view and is extremely confusing from the outside. You look at the bug fixed in dev, it was reported as 1.9 and marked fixed, while 1.9 is still affected. Given that the primary purpose of the public issue tracker is to expose statuses that users can search into, I'd really want to have them all together. There's hardly version-specific discussions, and the rare times it will happen, it doesn't cost much to have them together but it is a huge pain to have them separate. Also, if an issue is met during the backport (while working on the automatically created branch-specific issue), there's no easy way to share the info with the other branches which are covered by different issues. Since the other branches statuses will have to be checked in the primary one anyway, better have all statuses there. > 1. User creates issue #123. > > > Subject: If I foo then haproxy replies with bar instead of baz > > > > If I foo then haproxy replies with bar and this is very bad! > > 2. Developer fixes the issue in dev, noting the branches to backport in > the commit message. Ideal in a machine-parseable format (Backport-To: 1.8) > > > BUG/MINOR: foo: Make haproxy reply with baz > > > > This fixes issue #123. > > > > Backport-To: 1.8 > > Backport-To: 1.7 There is a race here : the diag of affected branches arrives much earlier than the fix, hence the commit message. And very often the whole extent is not known at fix time, and has to be extended afterwards when doing the backports to the other branches. We already have the cherry-picked tracking in the commit messages, so I'd suggest that anyone aware of another affected branch adds the "affects-X.Y" label, and that (once we have it) a bot simply checks for the cherry-picked commits to figure the issue and remove the label. For the rare cases where it doesn't work correctly (because as I said cherry-picking is not rocket science), it's easy to manually remove the label to mark the branch as unaffected at the time of reading. We could decide that bugs for which no "affects-X.Y" label exist anymore can be closed. That doesn't mean the issue doesn't affect older branch, it means it's not known to affect them yet, which is similar to before the bug report (since an issue tracker only tracks known defects and not hypothetical ones). With this model, there is less work for everyone involved, all the info is concentrated together, users can see that their
Lots of mail from email alert on 1.9.x
I just updated to 1.9.1 on my test system.
We noticed that when a server fails we now get tons of mail, and with
tons we mean a lot.
After a client backend server fails we usually get 1 mail on 1.8.x now
with 1.9.1 within 1 minute we have the following.
mailq | grep -B2 l...@testdomain.nl | grep '^[A-F0-9]' | awk '{print
$1}' | sed 's/*//' | postsuper -d -
postsuper: Deleted: 19929 messages
My setting from the backend part is as follows.
email-alert mailers alert-mailers
email-alert from l...@testdomain.nl
email-alert to not...@testdomain.nl
server webserver09 11.22.33.44:80 check
Has something changed in 1.9.x (it was on 1.9.0 also)
regards
Johan Hendriks
Re: haproxy issue tracker discussion
Aleks, Am 10.01.19 um 15:30 schrieb Aleksandar Lazic: > In general I also see a huge benefit to add a issue tracker I also know > that's a > workflow change for the developers. > > As I also follow the discussion let suggest me the following. > > * add some templates for the different cases e. g.: ISSUE_TEMPLATE.md > https://blog.github.com/2016-02-17-issue-and-pull-request-templates/ Yes. We definitely need one template for "Bug" and another for "Wishlist". > * use some labels, as labels are very flexible and easy to use as selector. > for example: bug, 1.7, backport-1.7, 2.0-dev, fixed_in_1.9.1, ... I disagree here. We should not create too many labels, because one should not need to search for the proper labels. Also once an issue is marked closed it disappears from the view. Labels that make sense: - Affected version: affects/1.7, affects/1.8, ... - Type: Bug, Wishlist - Subsystem: dns, h2, ... - Resolution: fixed, invalid, cannot-reproduce, ... To track whether an issue is fixed in a specific branch Milestones + automated follow-up issues are superior. > * in the commit message(s) can the issue number be added to create a > corelation > to the issue with `#`. > https://help.github.com/articles/basic-writing-and-formatting-syntax/#referencing-issues-and-pull-requests Yes, this should be done. Best regards Tim Düsterhus
Re: haproxy issue tracker discussion
Willy, Am 09.01.19 um 15:22 schrieb Willy Tarreau: >> Here's some more recent projects that probably grew up with GitHub. I >> can't comment how they do the backports, though: >> >> https://github.com/nodejs/node/issues (has LTS / Edge) >> https://github.com/zfsonlinux/zfs/issues (has stable / dev) >> https://github.com/antirez/redis/issues >> https://github.com/moby/moby/issues (tons of automation based on an >> issue template) > > I only knew 3 of them by name and never used any ;-) > > Node is interesting here. the have tags per affected version. E.g. > https://github.com/nodejs/node/issues/25221 Yes, that definitely makes sense. > I tend to think that if labels already mark the relevance to a branch, > then they override the status and probably we don't really care about > the status. The "moby" project above does that by the way, with > "status/foo" labels. We could possibly have had "v1.8-open" and > "v1.8x-done". This way instead of searching by status users can > search by clicking on the labels. I could just be missing something. When should the binary "issue open" / "issue closed" property be toggled? When the issue is fixed in Dev? When the issue is fixed in the lowest affected version? Both is not ideal, I guess. Maybe we need to automatically create follow up issues for the various branches once the fix lands in dev: 1. User creates issue #123. > Subject: If I foo then haproxy replies with bar instead of baz > > If I foo then haproxy replies with bar and this is very bad! 2. Developer fixes the issue in dev, noting the branches to backport in the commit message. Ideal in a machine-parseable format (Backport-To: 1.8) > BUG/MINOR: foo: Make haproxy reply with baz > > This fixes issue #123. > > Backport-To: 1.8 > Backport-To: 1.7 3. A bot automatically creates follow-up issues for each noted branch referring to the initial issue, stating that a backport is needed: > Subject: Backport 1.7: If I foo then haproxy replies with bar instead of baz > > This is a follow-up issue, because #123 is not yet fixed for haproxy 1.7. 4. Developer backports the commit, closing the follow-up issues whenever he did so (this probably can be automated as well. If a Backport-To: 1.7 line appears in the 1.7 branch the matching issue will be closed). Best regards Tim Düsterhus
Re: haproxy issue tracker discussion
Am 09.01.2019 um 15:22 schrieb Willy Tarreau: > Hi Tim, > > On Wed, Jan 09, 2019 at 12:58:30PM +0100, Tim Düsterhus wrote: >> Am 09.01.19 um 05:31 schrieb Willy Tarreau: >>> Except that the "naturally" part here is manually performed by someone, >>> and an issue tracker is nothing more than an organized todo list, which >>> *is* useful to remind that you missed some backports. It regularly happens >>> to us, like when the safety of some fixes is not certain and we prefer to >>> let them run for a while in the most recent versions before backporting >>> them to older branches. This is exactly where an issue tracker is needed, >>> to remind us that these fixes are still needed in older branches. >> >> So the commits are not being cherry-picked in the original order? I >> imagined that the process went like this: >> >> 1. List all the commits since the last cherry-picks >> 2. Look in the commit message to see whether the commit should be >> backported. >> 3. Cherry-pick the commit. > > It's what we *try* to do, but cherry-picking never is rocket science, for > various reasons, some ranging from uncertainty regarding some fixes that > need to cool down later, other because an add-on was made, requiring an > extra patch that are much more convenient to deal with together (think > about bisect for example). That's why I created the git-show-backport > script which gives us significant help in comparing lists of commits from > various branches. > >>> If the issue tracker only tracks issues related to the most recent branch, >> >> I believe you misunderstood me. What I attempted to say is: >> >> The issue tracker tracks which branches the bug affects. But IMO it does >> not need to track whether the backport already happened, because the >> information that the backport needs to happen is in the commit itself >> (see above). > > For me it is important to have the info that the branch is still unfixed > because as I explained, the presence of a given commit is not equivalent > to the issue being fixed. A commit is for a branch. It will often beckport > as a 1-to-1 to the closest branches, but 10% of the time you need to > backport extra stuff as well that is not part of the fix but which the > fix uses, and sometimes you figure that the issue is still not completely > fixed despite the backport being there because it's more subtle. > >>> it will only solve the problem for this branch. For example, Veiko Kukk >>> reported in November that compression in 1.7.11 was broken again. How do >>> I know this ? Just because I've added an entry for this in my TODO file. >>> This bug is apparently a failed backport, so it requires that the original >>> bug is reopened and that any backport attempt to an older version is paused. >> >> Is the failed backport a new bug or is it not? I'd say it's a new bug, >> because the situation changed. It's a new bug (someone messed up the >> backport) that affects haproxy-1.7, but does not affect haproxy-dev. You >> describe it as an old bug that needs to be re-opened. > > For me it's not a new bug at all, it's the same description. Worse, often > it will even be the one the reporter used! For example someone might report > an issue with 1.7, that we diagnose covers 1.7 to 2.0-dev. We finally find > the bug and if it in 2.0-dev then backport it. The backport stops working > when reaching 1.7. It's hard to claim it's a new bug while it exactly is the > bug the person reported! Doing otherwise would make issue lookups very > cumbersome, even more than the mailing list archives where at least you > can sort by threads. Thus for me it's only the status in the old branch > which is not resolved. It's also more convenient for users looking for a > solution to figure that the same bug is already fixed in 1.8 and that > possibly an upgrade would be the path to least pain. > >>> You'll note that for many of them the repository is only a mirror by >>> the way, so that's another hint. >> >> I suspect the reason is simple: The project already had a working issue >> tracker that predated GitHub. Many of these projects are way older than >> GitHub. > > It's possible. > >> Here's some more recent projects that probably grew up with GitHub. I >> can't comment how they do the backports, though: >> >> https://github.com/nodejs/node/issues (has LTS / Edge) >> https://github.com/zfsonlinux/zfs/issues (has stable / dev) >> https://github.com/antirez/redis/issues >> https://github.com/moby/moby/issues (tons of automation based on an >> issue template) > > I only knew 3 of them by name and never used any ;-) > > Node is interesting here. the have tags per affected version. E.g. > https://github.com/nodejs/node/issues/25221 I like this as then you can see all effected Versions for a issue and PR. > I tend to think that if labels already mark the relevance to a branch, > then they override the status and probably we don't really care about > the status. The "moby" project above does that by the way,
