tfo on default-server settings

[William Dauchy]

Hello,

On haproxy v2.0.x, while using tfo option in default-server:
 default-server inter 5s fastinter 1s fall 3 slowstart 20s observe layer7 
error-limit 5 on-error fail-check pool-purge-delay 10s tfo
we are getting:
 'default-server inter' : 'tfo' option is not accepted in default-server 
sections.

from https://cbonte.github.io/haproxy-dconv/2.0/configuration.html#5.2-tfo

Is tfo excluded from default-server options?
-- 
William

Help with 1.8.1/4 and spoa_server/spoa_example

[Aleksandar Lazic]

Hi.

I know this is a old haproxy version but I have not the option to update as it's
part of a vendor product.

I need to bring the `[haproxy-2.0.git]/ contrib/spoa_server/` up and runnig or
`[haproxy-1.8.git]/contrib/spoa_example/` from 1.8.4 to be able to run a python
script.

Any help is welcome and I can offer some money for the help. It's urgent so I
will need the help asap.

You can contact me also off the list.

Best regards
Aleks

Re: global maxconn behaviour in haproxy2.0

[William Dauchy]

On Wed, Jun 26, 2019 at 11:29:47AM +1000, Igor Cicimov wrote:
> Those maxconn values are per frontend so if your backend is referenced by
> two frontends you might end up with a limit of 2 x maxconn on the backend.
> Hence it is recommended to set maxconn per server too to protect from
> situation like this. So read about maxconn and even fullconn in the server
> config and tuning guide for more details.

thanks for the precision. I however later discovered in the code:

ha_warning("Proxy %s stopped (FE: %lld conns, BE: %lld conns).\n",
p->id, p->fe_counters.cum_conn, p->be_counters.cum_conn);

which means this is simply a cumulative counter displayed in the log.

Best,
-- 
William

Re: haproxy=2.0.1: socket leak

[Christopher Faulet]

Le 28/06/2019 à 15:09, Максим Куприянов a écrit :

Hi!

I found out that in some situations under high rate of incoming connections 
haproxy=2.0.1 starts leaking sockets. It looks like haproxy doesn't close 
connections to its backends after request is finished (FIN received from client) 
thus leaving its server-sockets in close-wait state.


As an example this simple config starts leaking right from the start for me. And 
everything ends with messages: "local0.emerg Proxy server.local:18400 reached 
process FD limit (maxsock=4130). Please check 'ulimit-n' and restart."




Hi Maksim,

The bug should now be fixed. See commit 6c7e96a3e for details. The commit was 
backported to 2.0 and 1.9.


Thanks !
--
Christopher Faulet

RE: issue with small object caching

[Senthil Naidu]

Hi Tim,

Thanks , I am seeing the same headers on chrome too any idea how to disable 
this.

Regards
Senthil

-Original Message-
From: Tim Düsterhus [mailto:t...@bastelstu.be] 
Sent: 03 July 2019 16:57
To: Senthil Naidu; Christopher Faulet; haproxy@formilux.org
Subject: Re: issue with small object caching

Senthil,

Am 03.07.19 um 13:20 schrieb Senthil Naidu:
> From Firefox
> *snip*
> Cache-Control: no-cache

This is the issue. Firefox requests that an intermediate proxy does not cache 
(or rather: that it re-validates its cached copy).

Best regards
Tim Düsterhus

Debugging protobuf and HTX

[GARDAIS Ionel]

Hi list, 

We ran into an issue with haproxy 2.0.1 in front of SonarQube. 
With HTX enabled, despite no errors being displayed either in haproxy nor 
SonarQube, some but not all analysis failed. 

>From SonarQube, debug shows protobuf requests 
>(/sonarqube/api/rules/list.protobuf) failing and fallbacking to default, or 
>complete failure if no default available. 

When switching back to 'no option http-use-htx', protobuf requests' answers did 
not had the same size and no more issues with SonarQube. 

How could I specifically debug protobuf handling with HTX enabled ? 

Thanks, 
Ionel 

--
232 avenue Napoleon BONAPARTE 92500 RUEIL MALMAISON
Capital EUR 219 300,00 - RCS Nanterre B 408 832 301 - TVA FR 09 408 832 301

Re: issue with small object caching

[Tim Düsterhus]

Senthil,

Am 03.07.19 um 13:20 schrieb Senthil Naidu:
> From Firefox
> *snip*
> Cache-Control: no-cache

This is the issue. Firefox requests that an intermediate proxy does not
cache (or rather: that it re-validates its cached copy).

Best regards
Tim Düsterhus

RE: issue with small object caching

[Senthil Naidu]

Hi,

Have tried setting "no option http-use-htx" but still the cache is not working 
from Firefox its working only from IE.

Below are the request headers taken from developer options of both IE and 
Firefox

From Firefox

Host: testingsite.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:67.0) Gecko/20100101 
Firefox/67.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Pragma: no-cache
Cache-Control: no-cache

From IE

Key Value
Request GET / HTTP/1.1
Accept  text/html, application/xhtml+xml, */*
Accept-Language en-IN
User-Agent  Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; 
Trident/5.0)
UA-CPU  AMD64
Accept-Encoding gzip, deflate
Hosttestingsite.com
If-Modified-Since   Wed, 19 Jun 2019 07:56:46 GMT
If-None-Match   "119-58ba89226aecd"
Connection  Keep-Alive

Regards
Senthil

-Original Message-
From: Christopher Faulet [mailto:cfau...@haproxy.com] 
Sent: 21 June 2019 23:10
To: Senthil Naidu; haproxy@formilux.org
Subject: Re: issue with small object caching

Le 21/06/2019 à 17:00, Senthil Naidu a écrit :
> Hi,
> 
> I am using haproxy 2.0.0 , when I am using IE I can see in the logs the first 
> request is reaching the real server and when I do the refresh all subsequent 
> request is hitting the cache of haproxy, when I use firefox/crome all the 
> request are served by real server only its not hitting the cache on haproxy.
> 
> Configuration
> 
> 
> global
> log /dev/loglocal0 info
> stats socket /var/run/haproxy.stat
> 
> defaults
> option httplog
> cache test
> total-max-size 100
> max-object-size 100
> max-age 240
> 
> #TESTGRP STARTS# frontend  TESTGRP bind 
> 0.0.0.0:80 mode http http-request cache-use test http-response 
> cache-store test log global option httplog option forwardfor maxconn 
> 2000 timeout client 180s bind-process 1-2 default_backend  TESTGRPBACK
> 
> #TESTGRPBACK STARTS# backend TESTGRPBACK 
> balance roundrobin mode http log global option httpchk HEAD / fullconn  
> 2000 timeout server 180s default-server inter 3s rise 2 fall 3 
> slowstart 0 server vm-ayw87o 172.30.1.250:80 weight 12 maxconn 2000 
> #TESTGRP ENDS#
> 
> ==
> 
> haproxy -vv
> HA-Proxy version 2.0.0 2019/06/16 - https://haproxy.org/ Build options 
> :
>TARGET  = linux-glibc
>CPU = x86_64
>CC  = gcc
>CFLAGS  = -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv 
> -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter 
> -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered 
> -Wno-missing-field-initializers -Wtype-limits
>OPTIONS = USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 USE_SYSTEMD=1
> 
> Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER +PCRE 
> -PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD 
> -PTHREAD_PSHARED -REGPARM -STATIC_PCRE -STATIC_PCRE2 +TPROXY 
> +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -VSYSCALL +GETADDRINFO 
> +OPENSSL -LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY 
> +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL +SYSTEMD 
> -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS
> 
> Default settings :
>bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
> 
> Built with multi-threading support (MAX_THREADS=64, default=2).
> Built with OpenSSL version : OpenSSL 1.1.1c  28 May 2019 Running on 
> OpenSSL version : OpenSSL 1.1.1c  28 May 2019 OpenSSL library supports 
> TLS extensions : yes OpenSSL library supports SNI : yes OpenSSL 
> library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3 Built with network 
> namespace support.
> Built with transparent proxy support using: IP_TRANSPARENT 
> IPV6_TRANSPARENT IP_FREEBIND Built with zlib version : 1.2.7 Running 
> on zlib version : 1.2.7 Compression algorithms supported : 
> identity("identity"), deflate("deflate"), raw-deflate("deflate"), 
> gzip("gzip") Built with PCRE version : 8.32 2012-11-30 Running on PCRE 
> version : 8.32 2012-11-30 PCRE library supports JIT : no (USE_PCRE_JIT 
> not set) Encrypted password support via crypt(3): yes
> 
> Available polling systems :
>epoll : pref=300,  test result OK
> poll : pref=200,  test result OK
>   select : pref=150,  test result OK
> Total: 3 (3 usable), will use epoll.
> 
> Available multiplexer protocols :
> (protocols marked as  cannot be specified using 'proto' keyword)
>h2 : mode=HTTP   side=FEmux=H2
>h2 : mode=HTXside=FE|BE mux=H2
>  : mode=HTXside=FE|BE mux=H1
>  : mode=TCP|HTTP   side=FE|BE mux=PASS
> 
> Available services : none
> 
> Available filters :
>  [SPOE] spoe
>  [COMP] compression
>  [CACHE] cache
>  [TRACE] 

question about spoe doc

[Aleksandar Lazic]

Hi.

in the spoe doc in section **2.5. Example** are the following lines.

```
 620 spoe-message get-ip-reputation
 621 args ip=src
 622 event on-client-session if ! { src -f /etc/haproxy/whitelist.lst }
```

As far as I understood the doc in line `496 args [name=] ...` right
could be the check also against `ip` not only `src`?

git links:

http://git.haproxy.org/?p=haproxy-1.8.git;a=blob;f=doc/SPOE.txt;h=9602df95dacc3aa5f6fa9154c0cf3f2a3f4ed092;hb=HEAD

http://git.haproxy.org/?p=haproxy.git;a=blob;f=doc/SPOE.txt;h=19f00ad98d51f155494b47e680f14d6973dd91bb;hb=HEAD


Regards
Aleks

Re: The case for changing the documentation syntax

[Aleksandar Lazic]

Hi.

Am 02.07.2019 um 20:29 schrieb Hugues Alary:
> And for comparison's sake, here's Asciidoc renders on github:
> https://github.com/asciidoctor/asciidoctor/blob/master/README.adoc
> 
> Other features of the asciidoc/asciidoctor ecosystem are:
> - Asciidoc is also standardized
> - https://antora.org/ allows you to build 1 documentation website, from 
> multiple
> documentation repositories.
> - asciidoctor is extendable, either by writing an extension in Javascript
> (https://asciidoctor-docs.netlify.com/asciidoctor.js/extend/extensions/register/)
>  or
> in Ruby (https://asciidoctor.org/docs/user-manual/#example-extensions) and it
> supports custom backends
> 
> Though I have no skin in the game. ReStructuredText is great, I'm merely
> presenting other options.

I have used asciidoctor for some projects, my reason to switch to pandoc
markdown was hat the pdf output of asciidoctor isn't very good especially when
you come to footmarks and bibliography.

Even though I don't like reStructuredText I vote for them as I think it fits
more into the haproxy workflows, AFAIK.

Jm2c

Regards
Aleks

> On Tue, Jul 2, 2019 at 9:05 AM Nick Ramirez  > wrote:
> 
> I found this page on Github. It uses reStructuredText and demonstrates how
> Github would render various elements out of the box. Of course, it can be
> made more visually appealing with other tools, but it's a free benefit 
> that
> it renders on Github.
> 
> https://gist.github.com/ionelmc/e876b73e2001acd2140f
> 
> 
> -- Original Message --
> From: "Lukas Tribus" mailto:li...@ltri.eu>>
> To: "Nick Ramirez" mailto:nrami...@haproxy.com>>
> Cc: "haproxy@formilux.org "
> mailto:haproxy@formilux.org>>; "Cyril"
> mailto:cyril.bo...@free.fr>>
> Sent: 7/1/2019 6:49:50 PM
> Subject: Re: The case for changing the documentation syntax
> 
>> Hello Nick,
>>  
>>  
>> On Mon, 1 Jul 2019 at 17:02, Nick Ramirez > > wrote:
>>>  
>>> Hello all,
>>>  
>>> I'd like to propose something radical, but that will greatly help us in
>>> terms of documentation. (And documentation is very important when it
>>> comes to people choosing whether to use a piece of software, as I am 
>>> sure
>>> you agree!)
>>>  
>>> First, the problem: Our documentation at
>>> https://github.com/haproxy/haproxy/blob/master/doc/configuration.txt is
>>> written using a sort of home-grown syntax that uses various conventions
>>> for indicating sections, keywords, etc.
>>>  
>>> However, parsing this home-grown documentation is difficult. For 
>>> example,
>>> I contribute to the HAProxy Syntax Support for Atom project
>>> (https://github.com/abulimov/atom-language-haproxy). This is a python
>>> program that must parse the HAProxy configuration.txt file and find the
>>> keywords by first finding specific section titles, then looking for 
>>> lines
>>> that don't have spaces in front of them. That's how we find the keywords
>>> in each section. It must be updated when new versions of HAProxy are
>>> released because new sections are added and the section numbers may
>>> change, and some sections are not reliably using the home-grown syntax.
>>> In short, parsing configuration.txt is difficult, error-prone and
>>> requires regular maintenance because its syntax is:
>>>  
>>> * Not a standard
>>> * Not used consistently throughout the document
>>> * Not easily parsed by existing tools (home-grown tools must be created
>>> and maintained)
>>>  
>>> You may wonder, why do we need to parse configuration.txt? The reasons 
>>> are:
>>>  
>>> * A text file without any styling is difficult to read, so we want to 
>>> add
>>> styling (e.g. convert it to HTML with CSS or offer a PDF download)
>>> * We want search functionality of the document and an auto-generated
>>> table of contents
>>> * We want to write haproxy.cfg files and have them displayed in
>>> syntax-highlighted color when using Github Gist or any modern text 
>>> editor
>>> (Atom, VS Code, Sublime Text, etc.). For that, we must currently parse
>>> configuration.txt to learn the keywords (as in the atom-language-haproxy
>>> project mentioned). For example, we use Github Gist, with the
>>> atom-language-haproxy project, to display HAProxy configuration snippets
>>> in color on the haproxy.com/blog . It would be
>>> easier to maintain this if we could parse configuration.text more 
>>> easily.
>>  
>>  
>> Actually since 7 years we do 2 of the 3 things you mention here;
>> documentation.txt and others are parsed automatically (in python) and
>> generate a verify nice HTML site, searchable and indexed with table of
>> contents, etc:
>>  
>> 

[PATCH] DOC: contrib: spoa_server Add some hints for building, spoa_server

[Aleksandar Lazic]

Hi.

I have added some hints into the spoa_server.
Thanks Thierry for the nice server ;-)

Regards
Aleks
From a627fd7d0e190e657d7a14b6f120c12f4df0e033 Mon Sep 17 00:00:00 2001
From: Aleksandar Lazic 
Date: Wed, 3 Jul 2019 08:16:17 +
Subject: [PATCH] DOC: contrib: spoa_server Add some hints for building
 spoa_server

---
 contrib/spoa_server/README | 13 +
 1 file changed, 13 insertions(+)

diff --git a/contrib/spoa_server/README b/contrib/spoa_server/README
index d820807..fa039c1 100644
--- a/contrib/spoa_server/README
+++ b/contrib/spoa_server/README
@@ -8,12 +8,25 @@ callback write variables which are sent as response when the 
processing
 is done.
 
 
+  Prerequirement
+
+
+You have to install the development packages, either from the
+distribution repositories or from the source.
+
+CentOS/RHEL: yum install python-devel
+
+The current python version in use is 2.7.
+
+
   Compilation
 ---
 
 Actually, the server support Lua and Python. Type "make" with the options:
 USE_LUA=1 and/or USE_PYTHON=1.
 
+You can add LUA_INC=.. LUA_LIB=.. to the make command to set the paths to
+the lua header files and lua libraries.
 
   Start the service
 -
-- 
1.8.3.1