Hello,
On 2014-07-02, 18:39:09, Willy Tarreau wrote:
> > BTW, if abns@ bind() fails during a "-sf" reload, then /tmp/test1.sock of
> > the
> > old HAProxy instance is unlinked/unreachable and thus cannot accept new
> > connections. The new unix@ bind() succeeded, the ".tmp" got renamed, the
> > ".bak"
> > was unlinked. So far so good. But when the abns@ bind() fails, then the
> > whole
> > new HAProxy instance fails and leaves the old instance "degraded".
> >
> > I do not know how to simply fix this.
>
> That's a good point. We have no link()/unlink() on abstract sockets, so
> I don't see how we'll be able to temporarily rename a socket. Since this
> is linux-only, we can consider any os-specific trick available and see if
> one is satisfying (eg: shutdown(RD) followed by listen(), etc).
I was unable to make it work that way. I used the attached code in one
console, and in the other:
strace -e trace=bind socat abstract-listen:test,unix-tightsocklen=0 stdio
I tried a few combinations of shutdown() / listen(), the ordering, ...,
but no luck.
> So what socat does prevents us from building binary addresses (eg: random
> addresses for internal communications). But, in practice, we're making names
> out of human-readable strings, we're not building addresses out of binary
> data, so I'd be tempted to think that it would make more sense to adopt
> socat's naming method instead of padding the remaining bytes with zeroes.
>
> Still I feel concerned about future evolutions. I have a deep feeling that
> this is something we could regret sooner or later. Indeed, if we use strlen()
> over possibly binary contents to retrieve a socket address length, something
> tells me that we're sacrifying the ability to later extend this scheme to
> other usages.
Indeed. I wasn't happy with using strlen() either :).
>
> > The second attached patch (haproxy_correct_abns_length.diff) seems to fix
> > it.
> > No idea whether this is the right way to do it.
>
> I've just found in socat's man that it supports both modes :
>
> unix-tightsocklen=[0|1]
> On socket operations, pass a socket address length that does not
> include the whole struct sockaddr_un record but (besides
> other components) only the relevant part of the filename or
> abstract string.
> Default is 1.
>
> and indeed it works here :
>
> $ strace -s 200 -e trace=connect socat readline
> abstract-connect:foo,unix-tightsocklen=0
> connect(3, {sa_family=AF_FILE, path=@"foo"}, 110) = 0
>
> So given that, I'd prefer to stay on the current scheme and document this
> way to interface with socat instead. It seems safer and more durable to me.
> What do you think ?
Oh, I didn't know socat had such an option until you mentioned it :).
Neat. Keeping the current scheme makes sense now.
>
> > I can work around all of those problems by using plain old unix sockets
> > instead, but I like the idea of abstract namespace sockets :).
>
> I like it as well, but you see, another concern I have is the fact that
> any other process -not just haproxy- could be bound to an abstract socket.
> While there are well-known ports for TCP/UDP services, there's no official
> registry of well-known abstract socket addresses and this is a problematic
> point. I suspect that we'll quickly come up with a complement which is a
> random address : we would generate a random address and try to bind. Then
> we would map each abstract name to that random address. It would solve all
> the communications issues within a group of processes belonging to the same
> config space.
Also with abstract sockets, we lack any access control (apart from the
apparent possibility of collisions). Any process can bind() / connect()
to any name.
> My gut feeling is that we should use file-system for anything system-wide
> (ie: socat to haproxy), as it's the only way to safely connect two ends. But
> we could use abstract sockets inside the same process or group of processes.
>
> An intermediate mechanism could be to have haproxy automatically prefix
> abstract sockets with the starting process' pid or with a short random.
But that would kill the possibility of two processes inside different
chroots to communicate efficiently (without some mount --bind tricks).
(I don't have any practical example of such a setup, though :))
> So I have applied patch #1 and not #2. Instead I think we'd rather continue
> to discuss the design choices and options here.
Thanks.
> Best regards,
> Willy
Best regards,
--
hodor
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <sys/un.h>
#include <assert.h>
#include <unistd.h>
int main(int argc, char ** argv)
{
int s, err, one=1;
struct sockaddr_un sun;
s = socket(AF_UNIX, SOCK_STREAM, 0);
assert(s != -1);
memset(&sun, 0, sizeof sun);
sun.sun_family = AF_UNIX;
strcpy(&sun.sun_path[1], "test");
err = setsockopt(s, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one));
assert(err != -1);
err = bind(s, &sun, sizeof sun);
assert(err != -1);
err = listen(s, 5);
assert(err != -1);
err = shutdown(s, SHUT_WR);
// err = listen(s, 5);
err = shutdown(s, SHUT_RD);
// sun.sun_path[1] = 'r';
// err = bind(s, &sun, sizeof sun);
// assert(err != -1);
pause();
return 0;
}
