Re: [BUG] haproxy retries dispatch to wrong server

[Christopher Faulet]

Le 07/07/2020 à 23:02, Christopher Faulet a écrit :

Le 07/07/2020 à 15:16, Michael Wimmesberger a écrit :

Hi,

I might have found a potentially critical bug in haproxy. It occurs when
haproxy is retrying to dispatch a request to a server. If haproxy fails
to dispatch a request to a server that is either up or has no health
checks enabled it dispatches the request to a random server on any
backend in any mode (tcp or http) as long as they are in the up state
(via tcp-connect or httpchk health checks). In addition haproxy logs the
correct server although it dispatches the request to a wrong server.



Hi Michael,

Thanks for the reproducer and the detailed description. I'm able to reproduce
the bug thanks to it. I attached a patch to address it. I will see with Willy
tomorrow morning if it is the good way to fix it. But it should do the trick.



Hi,

I finally pushed this fix in the 2.0. Note the same bug affected the HTTP proxy 
mode (using http_proxy option). In this case, the connection retries is now 
disabled (on the 2.0 only) because the destination address is definitely lost. 
It was the easiest way to work around the bug without backporting a bunch of 
sensitive patches from the 2.1.


Thanks again Michael. You're bug report was really helpful.

--
Christopher Faulet

Re: [ANNOUNCE] haproxy-2.2.0

[Tim Düsterhus]

Ryan,

Am 09.07.20 um 20:34 schrieb Ryan O'Hara:
> I'm currently packaging this for Fedora. It seems to build just fine on
> Fedora 32 and rawhide. Is there any new build options or dependencies to be
> aware of? I'm looking at the Makefile now and nothing jumps out at me. That
> said, I am totally capable of missing something.
> 

I've just run `git diff v2.2-dev0..v2.3-dev0 -- Makefile`. The only
thing I'm seeing that might of of interest to you is HLUA_PREPEND_PATH /
HLUA_PREPEND_CPATH if you plan to ship any HAProxy specific Lua
libraries that don't make sense in the global Lua library path.

See this mailing list thread for details:
https://www.mail-archive.com/haproxy@formilux.org/msg35839.html

Specifically this email for the patch:
https://www.mail-archive.com/haproxy@formilux.org/msg35896.html

Best regards
Tim Düsterhus

Re: [ANNOUNCE] haproxy-2.2.0

[Willy Tarreau]

Hi Ryan,

On Thu, Jul 09, 2020 at 01:34:40PM -0500, Ryan O'Hara wrote:
> On Tue, Jul 7, 2020 at 12:41 PM Willy Tarreau  wrote:
> 
> > Hi,
> >
> > HAProxy 2.2.0 was released on 2020/07/07. It added 24 new commits
> > after version 2.2-dev12.
> >
> 
> This is great. Thank you to all who contributed to this release.
> 
> I'm currently packaging this for Fedora. It seems to build just fine on
> Fedora 32 and rawhide. Is there any new build options or dependencies to be
> aware of? I'm looking at the Makefile now and nothing jumps out at me. That
> said, I am totally capable of missing something.

For Linux there's nothing new that isn't automatic. However we've removed
some obsolete options that you were very unlikely to be using anyway:
USE_MY_EPOLL, USE_MY_SPLICE, USE_REGPARM, USE_VSYSCALL, USE_MY_ACCEPT4.

You may want to set HLUA_PREPEND_PATH/HLUA_PREPEND_CPATH to propose a
default search path for Lua modules, though I'm not sure about the
possible impacts in a distro (can't possibly be as bad as searching
at the wrong place :-)).

If you consider Fedora the right place to report early bugs, you could
also enable DEBUG_STRICT (which enables the few BUG_ON statements),
DEBUG_MEM_STATS (which allows to report in "debug dev memstats" the places
in the code where allocations were made), and DEBUG_FD, which adds a calls
counter to each FD and helps figure if an FD is stuck when someone reports
a 100% CPU condition (very low overhead as well). All of them are used on
my systems and even on haproxy.org so they are safe.

Cheers,
Willy

Re: [ANNOUNCE] haproxy-2.2.0

[Ryan O'Hara]

On Tue, Jul 7, 2020 at 12:41 PM Willy Tarreau  wrote:

> Hi,
>
> HAProxy 2.2.0 was released on 2020/07/07. It added 24 new commits
> after version 2.2-dev12.
>

This is great. Thank you to all who contributed to this release.

I'm currently packaging this for Fedora. It seems to build just fine on
Fedora 32 and rawhide. Is there any new build options or dependencies to be
aware of? I'm looking at the Makefile now and nothing jumps out at me. That
said, I am totally capable of missing something.

Ryan

HAProxy Contract Role with JPMorgan Chase, Dallas, TX (remote)

[Gregory Inguagiato]

Good day, I work on behalf of JPMorgan Chase Bank and we are looking for 
someone with strong experience in HAProxy for a launch set for Feb. 2021.  It 
is for the area of the bank that provides authentication services globally and 
is a high availability system.  We want to add HAProxy technology to the stack.

Since it's new, we need someone who has done large scale implementations and 
can advise us. The resource won't be doing the design work, we will do that, 
but they will provide expertise and say this looks good or not good and help 
the JPMC team test and implement, take design out to production environment. 
Use cases are already documented, so that's not needed.

We need a resource who can bring the team up to speed, do the training, and 
know all the ins and out of the product.  Knows how to implement in very large 
scale. Our team will do most of hands-on, but we need an advisor. How to 
deploy, scale, configure it.

Contract Term: 6 months through Feb/March 2021
Terms: W2, EAD/OPT, Permanent Resident , U.S. Citizen (no corp to corp)
Location: Dallas, TX (but role is remote because of Covid-19)

Contact: Greg Inguagiato
Email: 
greg.inguagi...@resourcesolutions.com
Phone: 1-805-587-9117



Gregory Inguagiato
Direct Recruiter
Resource Solutions
8381 Dix Ellis Trail, Prominence 500, Suite 100, Jacksonville, 32256
Tel: +1 904 831 5682
E-mail: greg.inguagi...@resourcesolutions.com
Web: www.resourcesolutions.com
LinkedIn | 
Twitter | 
YouTube | 
Instagram


Registered Office: Resource Solutions Limited, 11 Slingsby Place, St Martin's 
Courtyard, London WC2E 9AB. Reg No. 02041065

Resource Solutions is a provider of Recruitment Process Outsourcing (RPO) and 
Managed Service Provider (MSP) solutions. We have delivered these solutions to 
leading organisations since 1997 and manage a recruitment budget of over ?2 
billion on behalf of our clients, placing candidates into over 50 countries 
globally. As part of the Robert Walters Group our business has considerable 
resources at its disposal, operating from 31 countries, which enables us to 
work in partnership with organisations, managing everything from global 
accounts with demanding resourcing strategies to single sites with low 
recruitment volumes.

Resource Solutions takes its obligations in respect of the privacy of your 
personal information seriously. For further information in relation to how we 
use, handle, share and store your personal data, please see our privacy policy 
available at: https://www.resourcesolutions.com/privacy-policy.html


[http://online.robertwalters.com/autosig/emeaa2020/RSRAPIDSOURCE2020.jpg]

P Please don't print this e-mail unless you really need to. Think before you 
print...


**
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
Mimecast for the presence of computer viruses.

www.mimecast.com
**


HAProxy Engineer Plano.docx
Description: HAProxy Engineer Plano.docx

Stick to the same upstream socks5 server for the same tcp session.

[Hongyi Zhao]

Hi,

I config haproxy as a layer 4 socks5 load balancing gateway which use
several upstream socks5 servers and listening on 127.0.0.1:1. When
I use this aggregated local socks5 server for registering a user
account on wine forum here:  https://forum.winehq.org/, I always
failed in this job. Finally I find that it requires a fixed client
side IP address during the whole registration process.

So, it seems that if haproxy can stick to the same upstream socks5
server for the same tcp session, the above problem will be solved. How
should I config my haproxy for this purpose?

-- 
Hongyi Zhao 

Re: OSX builds in Travis

[Willy Tarreau]

On Thu, Jul 09, 2020 at 01:35:57PM +0500,  ??? wrote:
> let it settle down a bit/
> 
> 
> I'll have a look in few days

Great, thanks!
Willy

Re: OSX builds in Travis

[Илья Шипицин]

let it settle down a bit/


I'll have a look in few days

чт, 9 июл. 2020 г. в 13:29, Willy Tarreau :

> On Thu, Jul 09, 2020 at 12:19:32PM +0500,  ??? wrote:
> > We install socat, because it is (or was?) needed for some tests. OSX
> > requires to update whole brew for that. Otherwise it works unstable
>
> Wow, so the we'd rather build and install socat ourselves from sources,
> because quite frankly here it looks like a full OS upgrade!
>
> Willy
>

Re: OSX builds in Travis

[Willy Tarreau]

On Thu, Jul 09, 2020 at 12:19:32PM +0500,  ??? wrote:
> We install socat, because it is (or was?) needed for some tests. OSX
> requires to update whole brew for that. Otherwise it works unstable

Wow, so the we'd rather build and install socat ourselves from sources,
because quite frankly here it looks like a full OS upgrade!

Willy

Re: OSX builds in Travis

[Dinko Korunic]

Hi Илья,

I think that Travis’ Homebrew plugin is just fine, but I would definitely avoid 
updating/upgrading Homebrew as that’s certainly going to make builds much 
slower.

Do you have any sample logs of the situation where socat failed to install with 
non-updated Homebrew? It doesn’t make sense that socat has been failing to 
install, as there are binary downloads available for almost every OSX and even 
more so, Formula/socat.rb is being changed really infrequently (once per year 
at most, according to git history for socat.rb).


> On 9 Jul 2020, at 10:07, Илья Шипицин  wrote:
> 
> we have homebrew --> update --> true
> 
> https://github.com/haproxy/haproxy/blob/master/.travis.yml#L26 
> 
> 
> 
> if we remove it, brew will not get updated.
> 
> 
> most of the time it is not an issue and we can install socat. but under some 
> circumstances socat refuses to install without brew update
> 
> чт, 9 июл. 2020 г. в 12:42, Dinko Korunic  >:
> I would suggest using HOMEBREW_NO_AUTO_UPDATE environment variable to avoid 
> Brew auto-updating where it’s not really needed, for instance as in:
> 
> HOMEBREW_NO_AUTO_UPDATE=1 brew install socat
> 
> If that doesn’t work (but I think it should), pinning will cause none of 
> dependancies to be installed automatically and only through manual install:
> 
> brew list | xargs brew pin
> brew install socat
> 
> 
>> On 9 Jul 2020, at 09:19, Илья Шипицин > > wrote:
>> 
>> We install socat, because it is (or was?) needed for some tests. OSX 
>> requires to update whole brew for that. Otherwise it works unstable
>> 
>> On Thu, Jul 9, 2020, 9:16 AM Willy Tarreau > > wrote:
>> Hi Ilya,
>> 
>> is it normal that the OSX build procedure in travis pulls gigabytes of
>> ruby and python crap, including fonts, libgpg, gnutls, qt, postgresql
>> and whatnot for many minutes ?
>> 
>>  https://travis-ci.com/github/haproxy/haproxy/jobs/359124175 
>> 
>> 
>> It's been doing this for more than 12 minutes now without even starting
>> to build haproxy. I'm suspecting that it's reinstalling a full-featured
>> desktop operating system, which seems awkward and counter productive at
>> best.
>> 
>> I don't know if that's automatically done based on broken dependencies
>> or if it is caused by a preliminary upgrade of the whole system, but I
>> think we need to address this as it's quite not efficient in this form.
>> 
>> Thanks!
>> Willy
> 
> -- 
> Dinko Korunic   ** Standard disclaimer applies **
> Sent from OSF1 osf1v4b V4.0 564 alpha
> 

-- 
Dinko Korunic   ** Standard disclaimer applies **
Sent from OSF1 osf1v4b V4.0 564 alpha

Re: OSX builds in Travis

[Илья Шипицин]

Dinko,

do you think does it make sense to use scripted brew instead of travis
plugin ?

if so, we can try to "brew instal blah-blah-blah || ok, we failed, lets'
update and install one more time"

чт, 9 июл. 2020 г. в 13:07, Илья Шипицин :

> we have homebrew --> update --> true
>
> https://github.com/haproxy/haproxy/blob/master/.travis.yml#L26
>
>
> if we remove it, brew will not get updated.
>
>
> most of the time it is not an issue and we can install socat. but under
> some circumstances socat refuses to install without brew update
>
> чт, 9 июл. 2020 г. в 12:42, Dinko Korunic :
>
>> I would suggest using HOMEBREW_NO_AUTO_UPDATE environment variable to
>> avoid Brew auto-updating where it’s not really needed, for instance as in:
>>
>> HOMEBREW_NO_AUTO_UPDATE=1 brew install socat
>>
>> If that doesn’t work (but I think it should), pinning will cause none of
>> dependancies to be installed automatically and only through manual install:
>>
>> brew list | xargs brew pin
>> brew install socat
>>
>>
>> On 9 Jul 2020, at 09:19, Илья Шипицин  wrote:
>>
>> We install socat, because it is (or was?) needed for some tests. OSX
>> requires to update whole brew for that. Otherwise it works unstable
>>
>> On Thu, Jul 9, 2020, 9:16 AM Willy Tarreau  wrote:
>>
>>> Hi Ilya,
>>>
>>> is it normal that the OSX build procedure in travis pulls gigabytes of
>>> ruby and python crap, including fonts, libgpg, gnutls, qt, postgresql
>>> and whatnot for many minutes ?
>>>
>>>  https://travis-ci.com/github/haproxy/haproxy/jobs/359124175
>>>
>>> It's been doing this for more than 12 minutes now without even starting
>>> to build haproxy. I'm suspecting that it's reinstalling a full-featured
>>> desktop operating system, which seems awkward and counter productive at
>>> best.
>>>
>>> I don't know if that's automatically done based on broken dependencies
>>> or if it is caused by a preliminary upgrade of the whole system, but I
>>> think we need to address this as it's quite not efficient in this form.
>>>
>>> Thanks!
>>> Willy
>>>
>>
>> --
>> Dinko Korunic   ** Standard disclaimer applies **
>> Sent from OSF1 osf1v4b V4.0 564 alpha
>>
>>

Re: OSX builds in Travis

[Илья Шипицин]

we have homebrew --> update --> true

https://github.com/haproxy/haproxy/blob/master/.travis.yml#L26


if we remove it, brew will not get updated.


most of the time it is not an issue and we can install socat. but under
some circumstances socat refuses to install without brew update

чт, 9 июл. 2020 г. в 12:42, Dinko Korunic :

> I would suggest using HOMEBREW_NO_AUTO_UPDATE environment variable to
> avoid Brew auto-updating where it’s not really needed, for instance as in:
>
> HOMEBREW_NO_AUTO_UPDATE=1 brew install socat
>
> If that doesn’t work (but I think it should), pinning will cause none of
> dependancies to be installed automatically and only through manual install:
>
> brew list | xargs brew pin
> brew install socat
>
>
> On 9 Jul 2020, at 09:19, Илья Шипицин  wrote:
>
> We install socat, because it is (or was?) needed for some tests. OSX
> requires to update whole brew for that. Otherwise it works unstable
>
> On Thu, Jul 9, 2020, 9:16 AM Willy Tarreau  wrote:
>
>> Hi Ilya,
>>
>> is it normal that the OSX build procedure in travis pulls gigabytes of
>> ruby and python crap, including fonts, libgpg, gnutls, qt, postgresql
>> and whatnot for many minutes ?
>>
>>  https://travis-ci.com/github/haproxy/haproxy/jobs/359124175
>>
>> It's been doing this for more than 12 minutes now without even starting
>> to build haproxy. I'm suspecting that it's reinstalling a full-featured
>> desktop operating system, which seems awkward and counter productive at
>> best.
>>
>> I don't know if that's automatically done based on broken dependencies
>> or if it is caused by a preliminary upgrade of the whole system, but I
>> think we need to address this as it's quite not efficient in this form.
>>
>> Thanks!
>> Willy
>>
>
> --
> Dinko Korunic   ** Standard disclaimer applies **
> Sent from OSF1 osf1v4b V4.0 564 alpha
>
>

Re: OSX builds in Travis

[Dinko Korunic]

I would suggest using HOMEBREW_NO_AUTO_UPDATE environment variable to avoid 
Brew auto-updating where it’s not really needed, for instance as in:

HOMEBREW_NO_AUTO_UPDATE=1 brew install socat

If that doesn’t work (but I think it should), pinning will cause none of 
dependancies to be installed automatically and only through manual install:

brew list | xargs brew pin
brew install socat


> On 9 Jul 2020, at 09:19, Илья Шипицин  wrote:
> 
> We install socat, because it is (or was?) needed for some tests. OSX requires 
> to update whole brew for that. Otherwise it works unstable
> 
> On Thu, Jul 9, 2020, 9:16 AM Willy Tarreau mailto:w...@1wt.eu>> 
> wrote:
> Hi Ilya,
> 
> is it normal that the OSX build procedure in travis pulls gigabytes of
> ruby and python crap, including fonts, libgpg, gnutls, qt, postgresql
> and whatnot for many minutes ?
> 
>  https://travis-ci.com/github/haproxy/haproxy/jobs/359124175 
> 
> 
> It's been doing this for more than 12 minutes now without even starting
> to build haproxy. I'm suspecting that it's reinstalling a full-featured
> desktop operating system, which seems awkward and counter productive at
> best.
> 
> I don't know if that's automatically done based on broken dependencies
> or if it is caused by a preliminary upgrade of the whole system, but I
> think we need to address this as it's quite not efficient in this form.
> 
> Thanks!
> Willy

-- 
Dinko Korunic   ** Standard disclaimer applies **
Sent from OSF1 osf1v4b V4.0 564 alpha

Re: OSX builds in Travis

[Илья Шипицин]

We install socat, because it is (or was?) needed for some tests. OSX
requires to update whole brew for that. Otherwise it works unstable

On Thu, Jul 9, 2020, 9:16 AM Willy Tarreau  wrote:

> Hi Ilya,
>
> is it normal that the OSX build procedure in travis pulls gigabytes of
> ruby and python crap, including fonts, libgpg, gnutls, qt, postgresql
> and whatnot for many minutes ?
>
>  https://travis-ci.com/github/haproxy/haproxy/jobs/359124175
>
> It's been doing this for more than 12 minutes now without even starting
> to build haproxy. I'm suspecting that it's reinstalling a full-featured
> desktop operating system, which seems awkward and counter productive at
> best.
>
> I don't know if that's automatically done based on broken dependencies
> or if it is caused by a preliminary upgrade of the whole system, but I
> think we need to address this as it's quite not efficient in this form.
>
> Thanks!
> Willy
>