After some further troubleshooting, tonight I took some time to sit down
and
read again the man pages as everything on my config files was looking
fine and
no errors were showing up in any log. With Brian's help we were leading
to the
direction that something was wrong with the pf translation itself and so
I
tested a static rdr-to configuration with pf only in the same
environment, and
neither this test worked as expected. So I went back to read the pf.conf
man
page and here comes the rdr-to relevant section:
"Redirections cannot reflect packets back through the interface they
arrive on, they can only be redirected to hosts connected to different
interfaces or to the firewall itself."
Focusing on relayd, my oversight was to not going back and read again
the
pf.conf man page in order to make sure that my box's network
configuration was
ok, since apparently I got it to work with relays without problems.
The next challenge now is to find if there is another way to make this
setup
working with just 1 network interface and implement relayd redirects for
SSL
passthrough, or give up. There seems to be few options here that I can
think of:
- Keep my current configuration with HAproxy
- Add another network interface to the box and configure an additional
network to
it (it might be tricky when deploying a droplet with a direct public IP
address)
- Migrate to relayd relays and give up with SSL passthrough (with the
benefit of
SSL offloading if want to implement it)
Thank you to the community and the devs for the great work on this OS!
Especially
on the man pages :)
On 2020-07-11 12:58, Gabri Tofano wrote:
It isn’t. rdr-to, and by extension redirects, are not natting the
source address.
Clients connecting through relayd and to the backend will have source
addresses
not that of the relayd machine but of the original client.
Thank you for correcting me on this as it was a bad statement told
before
getting coffee in the morning :)
I’m going to play around on my boxes and try and come up with some
options for you.
I’ll get back to you later.
Thank you for dedicating time in looking to this issue!
On 2020-07-11 12:08, Brian Brombacher wrote:
On Jul 11, 2020, at 11:20 AM, Gabri Tofano <ga...@tofanos.com>
wrote:
On 2020-07-11 06:33, Brian Brombacher wrote:
On Jul 10, 2020, at 11:42 PM, Gabri Tofano <ga...@tofanos.com>
wrote:
Does http work with redirects? It wasn’t clear if it did or not
in
your first post.
It doesn't work with http and that is the redirect that I was
testing.
Indications from your pf anchor rules and the down
status above, and the check http attribute on the https forward
to
directives tell me relayd isn’t liking your check http
configuration
for port 443.
Start by switching to check icmp or check tcp or something else,
see
if it works, unless you can fix the check http based on logs or
otherwise.
I changed it to tcp and now the servers are showing as "up":
LAB1-LB1# relayctl sh sum
Id Type Name Avlblty
Status
1 redirect http
active
1 table web_servers:80
active (1 hosts)
1 host 172.16.101.31 100.00%
up
2 table nc_servers:80
active (1 hosts)
2 host 172.16.101.32 100.00%
up
2 redirect https
active
3 table web_servers:443
active (1 hosts)
3 host 172.16.101.31 100.00%
up
4 table nc_servers:443
active (1 hosts)
4 host 172.16.101.32 100.00%
up
However I was hoping to fix the http redirect first and then move
to https, but it
looks like more of a "general issue" with redirects in my current
configuration.
Thanks
If http redirection isn’t working, I’d be curious from where you’re
trying to connect or what router you have configured on the backend
hosts. I see you’re relayd box and back ends are on the same
network.
If you’re trying to connect from another address in 172.16.101.x to
your relayd setup, it won’t work reliably. It might also not work
reliably or at all, if you are not routing responses through the
relayd host.
If they are replying direct, any PF scrub normalization, tcp
sequence
handling, etc., all get lost, among other issues.
I hope this is the cause of your issues, otherwise you’re going to
need to include more information for your setup, or at a minimum
some
relayd logs.
-Brian
I have a layer3 switch doing routing between 2 vlans, relayd and the
2
backend web servers are on the same vlan and the client is on another
vlan 172.16.100.x. The relayd VM is configured with only 1 network
interface. When the client try to reach the web servers directly
everything work fine. When the client is passing through relayd I see
the following:
- Only SYN packets coming into relayd box which they become
retransmissions
- The relayd anchor rules do not have the log parameter set so I
cannot
see passing traffic from the client to the backend servers, but at
least
no traffic is being blocked. I haven't found a way to manipulate an
anchor
via pfctl in order to add the log parameter
- The web server does not see any traffic reaching out on port 80
beside
the http checks from relayd IP address
- I have set "log connection" in relayd.conf and then relayctl log
verbose
but /var/log/daemon unfortunately is not showing much:
relayd[84883]: startup
relayd[84883]: unused protocol: http
relayd[84883]: unused protocol: https
relayd[33541]: host 172.16.101.32, check tcp (1ms,tcp connect ok),
state unknown -> up, availability 100.00%
relayd[33541]: host 172.16.101.31, check tcp (2ms,tcp connect ok),
state unknown -> up, availability 100.00%
relayd[33541]: host 172.16.101.31, check http code (3ms,http code
ok), state unknown -> up, availability 100.00%
relayd[33541]: host 172.16.101.32, check http code (3ms,http code
ok), state unknown -> up, availability 100.00%
relayd[17311]: table http: 1 added, 0 deleted, 0 changed, 0 killed
relayd[17311]: table https: 1 added, 0 deleted, 0 changed, 0 killed
Since with relayd redirects traffic is being natted,
It isn’t. rdr-to, and by extension redirects, are not natting the
source address. Clients connecting through relayd and to the backend
will have source addresses not that of the relayd machine but of the
original client.
I'm not sure if the
issue could be with the fact that relayd is natting on the same
interface
for the same network subnet, as with relayd relays everything works
fine.
Correct, my assumption is this is your problem as well. Relays work
like a tcp proxy, so the source address changes and your backends
happily respond direct to the relayd machine.
I'm currently stuck in trying to find a way to log verbose what is
happening
on relayd as at least the client sessions are seen on relayd itself:
LAB1-LB1# relayctl sh redirects
Id Type Name Avlblty Status
1 redirect http active
total: 18 sessions
last: 0/60s 2/h 18/d sessions
average: 0/60s 0/h 0/d sessions
2 redirect https active
Thank you,
I’m going to play around on my boxes and try and come up with some
options for you. I’ll get back to you later.
