> On Jul 13, 2020, at 8:30 PM, Gabri Tofano <ga...@tofanos.com> wrote:
>
> I have tried to implement the workaround as per man page
> but it still doesn't work, here the pf.conf config:
> ----
> eth0 = "xnf0"
> web1 = "172.16.101.31"
>
> anchor "relayd/*"
>
> set skip on lo
>
> block return log
> pass log
>
> pass out quick on $eth0 proto tcp to $web1 port 80 \
> received-on $eth0 nat-to $eth0
Try putting this before the anchor. The quick entry in the anchor that relayd
creates takes precedence.
>
> block return in on ! lo0 proto tcp to port 6000:6010
> block return out log proto {tcp udp} user _pbuild
> ----
>
> I'm trying to gather some useful log on relayd and see if
> there's any error but even with "relayctl log verbose"
> nothing is showing beside the startup entries
>
> Thank you!
>
>> There's a "workaround" also mentioned in pf.conf(5) which also works
>> with relayd inserted rdr-rules, e.g.
>> pass out quick on vlan99 proto tcp to 192.168.89.13 received-on vlan99
>> nat-to 192.168.89.1
>> vlan99 has 'inet 192.168.89.1/24' and 192.168.89.13 is the relayd rdr
>> "target".
>> HTH,
>> --
>> pb
>>> On 2020-07-13 01:08, Gabri Tofano wrote:
>> After some further troubleshooting, tonight I took some time to sit down and
>> read again the man pages as everything on my config files was looking fine
>> and
>> no errors were showing up in any log. With Brian's help we were leading to
>> the
>> direction that something was wrong with the pf translation itself and so I
>> tested a static rdr-to configuration with pf only in the same environment,
>> and
>> neither this test worked as expected. So I went back to read the pf.conf man
>> page and here comes the rdr-to relevant section:
>> "Redirections cannot reflect packets back through the interface they
>> arrive on, they can only be redirected to hosts connected to different
>> interfaces or to the firewall itself."
>> Focusing on relayd, my oversight was to not going back and read again the
>> pf.conf man page in order to make sure that my box's network configuration
>> was
>> ok, since apparently I got it to work with relays without problems.
>> The next challenge now is to find if there is another way to make this setup
>> working with just 1 network interface and implement relayd redirects for SSL
>> passthrough, or give up. There seems to be few options here that I can think
>> of:
>> - Keep my current configuration with HAproxy
>> - Add another network interface to the box and configure an additional
>> network to
>> it (it might be tricky when deploying a droplet with a direct public IP
>> address)
>> - Migrate to relayd relays and give up with SSL passthrough (with the
>> benefit of
>> SSL offloading if want to implement it)
>> Thank you to the community and the devs for the great work on this OS!
>> Especially
>> on the man pages :)
>> On 2020-07-11 12:58, Gabri Tofano wrote:
>>>> It isn’t. rdr-to, and by extension redirects, are not natting the source
>>>> address.
>>>> Clients connecting through relayd and to the backend will have source
>>>> addresses
>>>> not that of the relayd machine but of the original client.
>>> Thank you for correcting me on this as it was a bad statement told before
>>> getting coffee in the morning :)
>>>> I’m going to play around on my boxes and try and come up with some options
>>>> for you.
>>>> I’ll get back to you later.
>>> Thank you for dedicating time in looking to this issue!
>>> On 2020-07-11 12:08, Brian Brombacher wrote:
>>>>>> On Jul 11, 2020, at 11:20 AM, Gabri Tofano <ga...@tofanos.com> wrote:
>>>>> On 2020-07-11 06:33, Brian Brombacher wrote:
>>>>>>>>>>> On Jul 10, 2020, at 11:42 PM, Gabri Tofano <ga...@tofanos.com>
>>>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Does http work with redirects? It wasn’t clear if it did or not in
>>>>>>>>>> your first post.
>>>>>>>>> It doesn't work with http and that is the redirect that I was testing.
>>>>>>>>>> Indications from your pf anchor rules and the down
>>>>>>>>>> status above, and the check http attribute on the https forward to
>>>>>>>>>> directives tell me relayd isn’t liking your check http configuration
>>>>>>>>>> for port 443.
>>>>>>>>>> Start by switching to check icmp or check tcp or something else, see
>>>>>>>>>> if it works, unless you can fix the check http based on logs or
>>>>>>>>>> otherwise.
>>>>>>>>> I changed it to tcp and now the servers are showing as "up":
>>>>>>>>> LAB1-LB1# relayctl sh sum
>>>>>>>>> Id Type Name Avlblty Status
>>>>>>>>> 1 redirect http active
>>>>>>>>> 1 table web_servers:80
>>>>>>>>> active (1 hosts)
>>>>>>>>> 1 host 172.16.101.31 100.00% up
>>>>>>>>> 2 table nc_servers:80
>>>>>>>>> active (1 hosts)
>>>>>>>>> 2 host 172.16.101.32 100.00% up
>>>>>>>>> 2 redirect https active
>>>>>>>>> 3 table web_servers:443
>>>>>>>>> active (1 hosts)
>>>>>>>>> 3 host 172.16.101.31 100.00% up
>>>>>>>>> 4 table nc_servers:443
>>>>>>>>> active (1 hosts)
>>>>>>>>> 4 host 172.16.101.32 100.00% up
>>>>>>>>> However I was hoping to fix the http redirect first and then move to
>>>>>>>>> https, but it
>>>>>>>>> looks like more of a "general issue" with redirects in my current
>>>>>>>>> configuration.
>>>>>>>>> Thanks
>>>>>> If http redirection isn’t working, I’d be curious from where you’re
>>>>>> trying to connect or what router you have configured on the backend
>>>>>> hosts. I see you’re relayd box and back ends are on the same network.
>>>>>> If you’re trying to connect from another address in 172.16.101.x to
>>>>>> your relayd setup, it won’t work reliably. It might also not work
>>>>>> reliably or at all, if you are not routing responses through the
>>>>>> relayd host.
>>>>>> If they are replying direct, any PF scrub normalization, tcp sequence
>>>>>> handling, etc., all get lost, among other issues.
>>>>>> I hope this is the cause of your issues, otherwise you’re going to
>>>>>> need to include more information for your setup, or at a minimum some
>>>>>> relayd logs.
>>>>>> -Brian
>>>>> I have a layer3 switch doing routing between 2 vlans, relayd and the 2
>>>>> backend web servers are on the same vlan and the client is on another
>>>>> vlan 172.16.100.x. The relayd VM is configured with only 1 network
>>>>> interface. When the client try to reach the web servers directly
>>>>> everything work fine. When the client is passing through relayd I see
>>>>> the following:
>>>>> - Only SYN packets coming into relayd box which they become
>>>>> retransmissions
>>>>> - The relayd anchor rules do not have the log parameter set so I cannot
>>>>> see passing traffic from the client to the backend servers, but at least
>>>>> no traffic is being blocked. I haven't found a way to manipulate an anchor
>>>>> via pfctl in order to add the log parameter
>>>>> - The web server does not see any traffic reaching out on port 80 beside
>>>>> the http checks from relayd IP address
>>>>> - I have set "log connection" in relayd.conf and then relayctl log verbose
>>>>> but /var/log/daemon unfortunately is not showing much:
>>>>> relayd[84883]: startup
>>>>> relayd[84883]: unused protocol: http
>>>>> relayd[84883]: unused protocol: https
>>>>> relayd[33541]: host 172.16.101.32, check tcp (1ms,tcp connect ok), state
>>>>> unknown -> up, availability 100.00%
>>>>> relayd[33541]: host 172.16.101.31, check tcp (2ms,tcp connect ok), state
>>>>> unknown -> up, availability 100.00%
>>>>> relayd[33541]: host 172.16.101.31, check http code (3ms,http code ok),
>>>>> state unknown -> up, availability 100.00%
>>>>> relayd[33541]: host 172.16.101.32, check http code (3ms,http code ok),
>>>>> state unknown -> up, availability 100.00%
>>>>> relayd[17311]: table http: 1 added, 0 deleted, 0 changed, 0 killed
>>>>> relayd[17311]: table https: 1 added, 0 deleted, 0 changed, 0 killed
>>>>> Since with relayd redirects traffic is being natted,
>>>> It isn’t. rdr-to, and by extension redirects, are not natting the
>>>> source address. Clients connecting through relayd and to the backend
>>>> will have source addresses not that of the relayd machine but of the
>>>> original client.
>>>>> I'm not sure if the
>>>>> issue could be with the fact that relayd is natting on the same interface
>>>>> for the same network subnet, as with relayd relays everything works fine.
>>>> Correct, my assumption is this is your problem as well. Relays work
>>>> like a tcp proxy, so the source address changes and your backends
>>>> happily respond direct to the relayd machine.
>>>>> I'm currently stuck in trying to find a way to log verbose what is
>>>>> happening
>>>>> on relayd as at least the client sessions are seen on relayd itself:
>>>>> LAB1-LB1# relayctl sh redirects
>>>>> Id Type Name Avlblty Status
>>>>> 1 redirect http active
>>>>> total: 18 sessions
>>>>> last: 0/60s 2/h 18/d sessions
>>>>> average: 0/60s 0/h 0/d sessions
>>>>> 2 redirect https active
>>>>> Thank you,
>>>> I’m going to play around on my boxes and try and come up with some
>>>> options for you. I’ll get back to you later.
