Re: relayd + pfsync
On 2/1/21 8:20 PM, Kapetanakis Giannis wrote:
> On 02/02/2021 05:18, Jordan Geoghegan wrote:
>> Hello,
>>
>> I had a question about using relayd with pfsync.
>>
>> I have a small gateway/load-balancer set up with relayd, carp and pfsync
>> plus BGPd for IP failover, and everything is working great. I was pleasantly
>> surprised at how easy it was to get pfsync tunnelled over wireguard. Things
>> failover perfectly, and I'm happy as a clam.
>>
>> I however do have a question about some pfsync/relayd details that I'm not
>> fully clear on:
>>
>> With all the plumbing being done with relayd and all the associated
>> TCP/TLS/HTTP(s) checks it's doing, it ends up setting up and tearing down a
>> decent number of connections on a recurring basis. I know in PF you can use
>> the "no-sync" keyword to prevent states created by certain rules from being
>> synced across the wire, but I haven't found a way to do this with
>> rules/states generated by relayd.
>>
>> It's probably largely irrelevant in the grand scheme of things, but I found
>> it slightly irritating having hundreds or thousands of state table entries
>> experiencing constant churn while being synced over the wire. Having the
>> noise from the relayd connectivity checks syncing back and forth makes using
>> tcpdump on a pfsync interface much less convenient. All these state table
>> entries will never be used should the machine fail-over, as all the
>> connectivity checks are initiated from the local IP address, rather that the
>> CARP address.
>>
>> So I guess what I'm trying to ask is: Is there a way to have relayd not sync
>> it's TCP/TLS/etc connectivity checks via pfsync?
>>
>> I was hoping to get a sanity check here so I can confirm weather or not I'm
>> totally off base here.
>>
>> I currently have "keep state (no-sync)" peppered throughout my config for
>> rules I want excluded from pfsync, as the pf config is quite simple. Maybe
>> I'm missing something obvious, but is there a "sync" option? ie the ability
>> to manually specify exactly which rules/states you want synced?
>>
>> Would some sort of rule like "pass out on $int_if proto tcp to any user
>> _relayd keep state (no-sync)" do what I want, or would that also catch the
>> traffic I'm trying to load balance as well?
>>
>> Any insight or advice would be much appreciated.
>>
>> P.S Sorry for the wall of text
>>
>> Regards,
>>
>> Jordan
>
> Hi,
>
> As you said, you can use the no-sync.
>
> Relayd checks don't create any pf rules. Only the listen creates rules
> pfctl -sr -a'relayd/ldap'
>
> pass in quick on rdomain 0 inet proto tcp from any to x.x.x.x port = 636
> flags S/SA keep state (tcp.established 4200) tag RELAYD_ldap rdr-to
> port 1636 least-states sticky-address
>
> local checks from LB to hosts can have the no-sync.
> I have these in my config
>
> # checks from LB
> pass out quick on $ldap_if proto tcp from ($ldap_if) to ($ldap_if:network)
> port {1389, 1636} keep state (no-sync)
>
> Maybe you have another rule (out on $ldap_if) before, that allows the traffic?
>
> I handle incoming traffic (to LB) with pftag on relayd.conf and I
> specifically allow them on out direction.
>
> # client rules
> pass out quick on $ldap_if tagged RELAYD_ldap keep state (tcp.established
> 4200)
>
> G
>
>
Hello,
Thanks for the sanity check, I've got everything working as expected now!
I need to learn to step away from the keyboard when I'm getting tired and
frustrated. The answer seems obvious in retrospect, but at least its figured
out now.
Regards,
Jordan
Re: relayd + pfsync
On 02/02/2021 05:18, Jordan Geoghegan wrote:
Hello,
I had a question about using relayd with pfsync.
I have a small gateway/load-balancer set up with relayd, carp and pfsync plus
BGPd for IP failover, and everything is working great. I was pleasantly
surprised at how easy it was to get pfsync tunnelled over wireguard. Things
failover perfectly, and I'm happy as a clam.
I however do have a question about some pfsync/relayd details that I'm not
fully clear on:
With all the plumbing being done with relayd and all the associated TCP/TLS/HTTP(s)
checks it's doing, it ends up setting up and tearing down a decent number of connections
on a recurring basis. I know in PF you can use the "no-sync" keyword to prevent
states created by certain rules from being synced across the wire, but I haven't found a
way to do this with rules/states generated by relayd.
It's probably largely irrelevant in the grand scheme of things, but I found it
slightly irritating having hundreds or thousands of state table entries
experiencing constant churn while being synced over the wire. Having the noise
from the relayd connectivity checks syncing back and forth makes using tcpdump
on a pfsync interface much less convenient. All these state table entries will
never be used should the machine fail-over, as all the connectivity checks are
initiated from the local IP address, rather that the CARP address.
So I guess what I'm trying to ask is: Is there a way to have relayd not sync
it's TCP/TLS/etc connectivity checks via pfsync?
I was hoping to get a sanity check here so I can confirm weather or not I'm
totally off base here.
I currently have "keep state (no-sync)" peppered throughout my config for rules I want
excluded from pfsync, as the pf config is quite simple. Maybe I'm missing something obvious, but is
there a "sync" option? ie the ability to manually specify exactly which rules/states you
want synced?
Would some sort of rule like "pass out on $int_if proto tcp to any user _relayd keep
state (no-sync)" do what I want, or would that also catch the traffic I'm trying to
load balance as well?
Any insight or advice would be much appreciated.
P.S Sorry for the wall of text
Regards,
Jordan
Hi,
As you said, you can use the no-sync.
Relayd checks don't create any pf rules. Only the listen creates rules
pfctl -sr -a'relayd/ldap'
pass in quick on rdomain 0 inet proto tcp from any to x.x.x.x port = 636
flags S/SA keep state (tcp.established 4200) tag RELAYD_ldap rdr-to
port 1636 least-states sticky-address
local checks from LB to hosts can have the no-sync.
I have these in my config
# checks from LB
pass out quick on $ldap_if proto tcp from ($ldap_if) to
($ldap_if:network) port {1389, 1636} keep state (no-sync)
Maybe you have another rule (out on $ldap_if) before, that allows the
traffic?
I handle incoming traffic (to LB) with pftag on relayd.conf and I
specifically allow them on out direction.
# client rules
pass out quick on $ldap_if tagged RELAYD_ldap keep state
(tcp.established 4200)
G
relayd + pfsync
Hello, I had a question about using relayd with pfsync. I have a small gateway/load-balancer set up with relayd, carp and pfsync plus BGPd for IP failover, and everything is working great. I was pleasantly surprised at how easy it was to get pfsync tunnelled over wireguard. Things failover perfectly, and I'm happy as a clam. I however do have a question about some pfsync/relayd details that I'm not fully clear on: With all the plumbing being done with relayd and all the associated TCP/TLS/HTTP(s) checks it's doing, it ends up setting up and tearing down a decent number of connections on a recurring basis. I know in PF you can use the "no-sync" keyword to prevent states created by certain rules from being synced across the wire, but I haven't found a way to do this with rules/states generated by relayd. It's probably largely irrelevant in the grand scheme of things, but I found it slightly irritating having hundreds or thousands of state table entries experiencing constant churn while being synced over the wire. Having the noise from the relayd connectivity checks syncing back and forth makes using tcpdump on a pfsync interface much less convenient. All these state table entries will never be used should the machine fail-over, as all the connectivity checks are initiated from the local IP address, rather that the CARP address. So I guess what I'm trying to ask is: Is there a way to have relayd not sync it's TCP/TLS/etc connectivity checks via pfsync? I was hoping to get a sanity check here so I can confirm weather or not I'm totally off base here. I currently have "keep state (no-sync)" peppered throughout my config for rules I want excluded from pfsync, as the pf config is quite simple. Maybe I'm missing something obvious, but is there a "sync" option? ie the ability to manually specify exactly which rules/states you want synced? Would some sort of rule like "pass out on $int_if proto tcp to any user _relayd keep state (no-sync)" do what I want, or would that also catch the traffic I'm trying to load balance as well? Any insight or advice would be much appreciated. P.S Sorry for the wall of text Regards, Jordan
