[PATCH] BUG/MEDIUM: contrib/spoa_server: Set FIN flag on agent frames
Hello, When communicating over SPOP the AGENT-HELLO, AGENT-DISCONNECT, and ACK frames must have the FIN flag set. This patch also upgrades the SPOP_VERSION to 2.0 and adds the "ip_score" transaction variable to the python & lua scripts. Thanks to Christopher for giving me the answer on how to solve this issue. -- Daniel 0001-BUG-MEDIUM-contrib-spoa_server-Set-FIN-flag-on-agent.patch Description: Binary data
Re: MEDIUM: Adding upstream socks4 proxy support
Hi Willy,
Thank you once again for all the detail reply.
On Wed, May 29, 2019 at 1:49 PM Willy Tarreau wrote:
>
> Hi Alec,
>
> On Thu, May 23, 2019 at 12:43:54AM +0800, Alec Liu wrote:
> > Finally got sometime to have the patch revised.
>
> And finally got some time to review it :-)
>
> > I am reusing the "send_proxy_ofs" right now.
>
> OK.
>
> > Due to it was used as some kind of flag for send Proxy Protocol in some
> > places.
> > I have to replace those usage to check the srv->pp_opts.
>
> Hmmm this will break health checks then, since you don't know whether a
> connection is made for data traffic or for a check. That makes me think
> that we've been quite annoyed for a while with these problems of finding
> all relevant connection parameters, and that maybe it could be useful to
> think about defining a "connection settings" structure with a lot of
> stuff that we want to use in various situations. A server would then
> have 2 or 3 of these (one for regular traffic or dataplane, one for
> health checks, one for agent checks). We could then use this for the
> source bindings, TLS/PP options etc. When setting up a connection, we'd
> just have to pass a pointer to the desired structure to provide all the
> settings. This definitely is something to think about for 2.1.
>
That should be able to make things more clear and easier to understand
for someone like
me new to the project. But that, I will try to read the code more
carefully to understand
how it works, and will have those changes reversed according to your comments.
> > From 6767c73a40a9d242cce061cc7feaf480ad416314 Mon Sep 17 00:00:00 2001
> > From: Alexander Liu
> > Date: Wed, 22 May 2019 19:44:48 +0800
> > Subject: [PATCH] MEDIUM: connection: Upstream SOCKS4 proxy support
> >
> > Have "socks4" and "check-via-socks4" server keyword added.
> > Implement handshake with SOCKS4 proxy server for tcp stream connection.
> > See issue #82.
> >
> > I have the "SOCKS: A protocol for TCP proxy across firewalls" doc found
> > at openssh's website under OpenSSH Sprcifications along with other RFCs,
> > at "https://www.openssh.com/specs.html;,
> > and "https://www.openssh.com/txt/socks4.protocol;.
> >
> > OpenSSH is using GPL, so I believe it should be fine to have the doc here.
> ^
>
> It's written exactly the opposite on their site and in the license file :-)
>
> LICENSE:
> "OpenSSH contains no GPL code."
>
> Well, I think you'll have to find the license covering the doc (probably
> BSD), include it in the doc/ directory and mention in the LICENSE file that
> this doc is provided under that license. Or we consider that it's too much
> hassle for just a protocol specification and you simply put a link to a
> stable URL in the commit message, which will be much simpler (and will also
> support fixes to the spec, that a copy doesn't provide).
>
Sorry for missing the license.
Due to it is a very old specification, most link just not working
well, that's why I try
to have the full text included at first place, and I have been try to
send email to the
author, but those email address just not working anymore, any way I
will leave the url
"https://www.openssh.com/txt/socks4.protocol; first but the full text.
> > diff --git a/include/proto/connection.h b/include/proto/connection.h
> > index ede372638..915be87bf 100644
> > --- a/include/proto/connection.h
> > +++ b/include/proto/connection.h
> > @@ -60,6 +60,10 @@ int conn_sock_send(struct connection *conn, const void
> > *buf, int len, int flags)
> > /* drains any pending bytes from the socket */
> > int conn_sock_drain(struct connection *conn);
> >
> > +/* scoks4 proxy handshake */
> > +int conn_send_socks4_proxy_request(struct connection *conn);
> > +int conn_recv_socks4_proxy_response(struct connection *conn);
> > +
> > /* returns true is the transport layer is ready */
> > static inline int conn_xprt_ready(const struct connection *conn)
> > {
> > @@ -889,6 +893,11 @@ static inline const char *conn_err_code_str(struct
> > connection *c)
> > case CO_ER_SSL_HANDSHAKE_HB: return "SSL handshake failure after
> > heartbeat";
> > case CO_ER_SSL_KILLED_HB: return "Stopped a TLSv1 heartbeat attack
> > (CVE-2014-0160)";
> > case CO_ER_SSL_NO_TARGET: return "Attempt to use SSL on an unknown
> > target (internal error)";
> > +
> > + case CO_ER_SOCKS4_SEND:return "SOCKS4 Proxy write error during
> > handshake";
> > + case CO_ER_SOCKS4_RECV:return "SOCKS4 Proxy read error during
> > handshake";
> > + case CO_ER_SOCKS4_DENY:return "SOCKS4 Proxy deny the request";
> denied
>
> > + case CO_ER_SOCKS4_ABORT: return "SOCKS4 Proxy handshake aborted by
> > server";
> > }
> > return NULL;
> > }
> > diff --git a/include/types/connection.h b/include/types/connection.h
> > index be590f098..9a362f05d 100644
> > --- a/include/types/connection.h
> > +++
Re: ACL criterion src [!] contained in ipset?
On 2019/05/29 10:35, Frank Myhr wrote: On 2019/05/28 22:34, Willy Tarreau wrote: In fact ACLs and maps are *exactly* the same, the only difference is that ACLs have a single column (the pattern) while maps associate a value (2nd column) to a pattern. Both are loaded from files, both can be updated at runtime from the CLI or from http-request rules. Note that what we're missing compared to ipset is a simple tool to append/remove entries to a file and to apply it to the CLI at the same time. It's quite trivial to do, it's just that nobody has apparently contributed such a thing (but probably many sysadmins out there have their own version). Thank you very much for the clarification. I now plan to use src ACL files with hitless reload. I might keep my whitelists & blacklists in files with slightly more elaborate syntax and use a script to translate them into ipset or haproxy ACL files as appropriate. That's indeed the cleanest way to do it. OK, this is now definitely the way I'll do it! Thanks for your confirmation. :-) For example, the ipset "nomatch" exception can be very handy for "punching out" ip subnets. A script could translate a single list file that included "nomatch" ranges into a haproxy whitelist and blacklist file pair to be combined in compound acl test. Indeed. I used to have an example utility somewhere based on ebtree to punch holes in networks and to merge other networks, I just don't remember where I've put it. This would be something useful. In this case you can use a single list which is exhaustive. That sounds like a very useful utility. Interesting that you haven't ended up using it much. Thanks, Frank
Re: ACL criterion src [!] contained in ipset?
On 2019/05/28 22:25, Willy Tarreau wrote: Frank, I've wanted to support ipset 10 years ago or so, before we had the ability to update ACLs at run time. I also wanted to be able to update them at run time. But later I figured that 1) it wouldn't bring any extra functionality over ACLs, 2) would not be portable, 3) would require haproxy to run as root in order to support being updated at runtime. So overall it would add extra dependencies and deployment constraints with little to no real benefit. Hoping this helps, Willy Willy, Those are excellent reasons for not supporting ipset from haproxy. Thank you for detailing them, it's very helpful indeed. Off-topic here, but the need to remain portable across platforms is likely the reason apache also doesn't support ipset. Good to be reminded to remove my Linux blinders occasionally. Frank
Re: upcoming 2.0 release: freebsd-11 seem to be broken ?
> On 29 May 2019, at 13:31, Илья Шипицин wrote: > > > > ср, 29 мая 2019 г. в 15:25, Dmitry Sivachenko : > > > > On 26 May 2019, at 23:40, Илья Шипицин wrote: > > > > Hello, > > > > I added freebsd-11 to cirrus-ci > > > > https://cirrus-ci.com/task/5162023978008576 > > > > should we fix it before 2.0 release ? > > > > > BTW, latest -dev release does not build at all on FreeBSD (I tried > FreeBSD-12): > > cc -Iinclude -Iebtree -Wall -Wextra -O2 -pipe -fstack-protector > -fno-strict-aliasing -fno-strict-aliasing -Wdeclaration-after-statement > -fwrapv -Wno-address-of-packed-member -Wno-unused-label -Wno-sign-compare > -Wno-unused-parameter -Wno-ignored-qualifiers > -Wno-missing-field-initializers -Wno-implicit-fallthrough -Wtype-limits > -Wshift-negative-value -Wnull-dereference -DFREEBSD_PORTS > -DUSE_KQUEUE-DUSE_PCRE -DUSE_PCRE_JIT -DUSE_POLL -DUSE_THREAD > -DUSE_REGPARM -DUSE_STATIC_PCRE -DUSE_TPROXY -DUSE_LIBCRYPT > -DUSE_GETADDRINFO -DUSE_OPENSSL -DUSE_ACCEPT4 -DUSE_ZLIB > -DUSE_CPU_AFFINITY -DCONFIG_REGPARM=3 -I/usr/include -DUSE_PCRE > -I/usr/local/include -DCONFIG_HAPROXY_VERSION=\"2.0-dev4\" > -DCONFIG_HAPROXY_DATE=\"2019/05/22\" -c -o src/wdt.o src/wdt.c > src/wdt.c:63:13: error: no member named 'si_int' in 'struct __siginfo' > thr = si->si_int; > ~~ ^ > src/wdt.c:105:7: error: use of undeclared identifier 'SI_TKILL' > case SI_TKILL: > ^ > 2 errors generated. > gmake[2]: *** [Makefile:830: src/wdt.o] Error 1 > > it was fixed right after 2.0-dev4. > please try current master > > Ah, okay. Thanks!
Re: upcoming 2.0 release: freebsd-11 seem to be broken ?
ср, 29 мая 2019 г. в 15:25, Dmitry Sivachenko : > > > > On 26 May 2019, at 23:40, Илья Шипицин wrote: > > > > Hello, > > > > I added freebsd-11 to cirrus-ci > > > > https://cirrus-ci.com/task/5162023978008576 > > > > should we fix it before 2.0 release ? > > > > > BTW, latest -dev release does not build at all on FreeBSD (I tried > FreeBSD-12): > > cc -Iinclude -Iebtree -Wall -Wextra -O2 -pipe -fstack-protector > -fno-strict-aliasing -fno-strict-aliasing -Wdeclaration-after-statement > -fwrapv -Wno-address-of-packed-member -Wno-unused-label -Wno-sign-compare > -Wno-unused-parameter -Wno-ignored-qualifiers > -Wno-missing-field-initializers -Wno-implicit-fallthrough -Wtype-limits > -Wshift-negative-value -Wnull-dereference -DFREEBSD_PORTS > -DUSE_KQUEUE-DUSE_PCRE -DUSE_PCRE_JIT -DUSE_POLL -DUSE_THREAD > -DUSE_REGPARM -DUSE_STATIC_PCRE -DUSE_TPROXY -DUSE_LIBCRYPT > -DUSE_GETADDRINFO -DUSE_OPENSSL -DUSE_ACCEPT4 -DUSE_ZLIB > -DUSE_CPU_AFFINITY -DCONFIG_REGPARM=3 -I/usr/include > -DUSE_PCRE -I/usr/local/include -DCONFIG_HAPROXY_VERSION=\"2.0-dev4\" > -DCONFIG_HAPROXY_DATE=\"2019/05/22\" -c -o src/wdt.o src/wdt.c > src/wdt.c:63:13: error: no member named 'si_int' in 'struct __siginfo' > thr = si->si_int; > ~~ ^ > src/wdt.c:105:7: error: use of undeclared identifier 'SI_TKILL' > case SI_TKILL: > ^ > 2 errors generated. > gmake[2]: *** [Makefile:830: src/wdt.o] Error 1 > it was fixed right after 2.0-dev4. please try current master
Re: upcoming 2.0 release: freebsd-11 seem to be broken ?
> On 26 May 2019, at 23:40, Илья Шипицин wrote: > > Hello, > > I added freebsd-11 to cirrus-ci > > https://cirrus-ci.com/task/5162023978008576 > > should we fix it before 2.0 release ? > BTW, latest -dev release does not build at all on FreeBSD (I tried FreeBSD-12): cc -Iinclude -Iebtree -Wall -Wextra -O2 -pipe -fstack-protector -fno-strict-aliasing -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -Wno-address-of-packed-member -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-ignored-qualifiers -Wno-missing-field-initializers -Wno-implicit-fallthrough -Wtype-limits -Wshift-negative-value -Wnull-dereference -DFREEBSD_PORTS -DUSE_KQUEUE-DUSE_PCRE -DUSE_PCRE_JIT -DUSE_POLL -DUSE_THREAD -DUSE_REGPARM -DUSE_STATIC_PCRE -DUSE_TPROXY -DUSE_LIBCRYPT -DUSE_GETADDRINFO -DUSE_OPENSSL -DUSE_ACCEPT4 -DUSE_ZLIB -DUSE_CPU_AFFINITY -DCONFIG_REGPARM=3 -I/usr/include -DUSE_PCRE -I/usr/local/include -DCONFIG_HAPROXY_VERSION=\"2.0-dev4\" -DCONFIG_HAPROXY_DATE=\"2019/05/22\" -c -o src/wdt.o src/wdt.c src/wdt.c:63:13: error: no member named 'si_int' in 'struct __siginfo' thr = si->si_int; ~~ ^ src/wdt.c:105:7: error: use of undeclared identifier 'SI_TKILL' case SI_TKILL: ^ 2 errors generated. gmake[2]: *** [Makefile:830: src/wdt.o] Error 1
Re: upcoming 2.0 release: freebsd-11 seem to be broken ?
ср, 29 мая 2019 г. в 07:36, Willy Tarreau : > Hi Ilya, > > On Mon, May 27, 2019 at 01:40:42AM +0500, ??? wrote: > > Hello, > > > > I added freebsd-11 to cirrus-ci > > > > https://cirrus-ci.com/task/5162023978008576 > > > > should we fix it before 2.0 release ? > > As I mentioned on the github issue, if you look closer to the logs, you > see that it's not haproxy, it's vtest which fails to create a socket. I > suspect that Cirrus runs in a jail where network syscalls are not allowed. > Thus we should simply disable regtests on Cirrus, and probably just run > haproxy -vv to be sure it starts, and be done with it. > I'll have a look at it. we will address it after 2.0 release > > Thanks > Willy >
