Hello @misc,

Just some further information on this.

When I stop relayd and enter the pf rules like relayd does with
its anchor, then it's - more or less - working as expected.

When I start an upload within an SFTP session and failover, then
the session is "stalled" nearly forever. When I set the
tcp.established to 60 (instead of 600), then the "state" times
out but the SFTP client starts reconnecting after a while
(about 2-3 minutes) and the sessions keeps running.

So it looks like relayd is "terminating" the session when
carp fails over. With relayd and doing a carp failover, I get
an

Broken pipe. Connection reset by peer

immediately.

I just want to know, if this is a normal behaviour with this setup.

Thanks.

Robert


> Gesendet: Mittwoch, 12. Oktober 2016 um 14:21 Uhr
> Von: "Robert Paschedag" <robert.pasche...@web.de>
> An: "Robert Paschedag" <robert.pasche...@web.de>
> Cc: misc@openbsd.org
> Betreff: Aw: Re: pf on carp backup resets connection after failover
>
> This time it should be better. Again sorry..
>
>
> Hi all,
>
> basically, if have exactly this problem already described here
>
(https://groups.google.com/forum/#!topic/bit.listserv.openbsd-pf/yZn4EUjxwfY)
.
> But because there is no answer since 2009, I'll give it a try.
>
> The setup of the 2 servers is also the same as in the other thread
> only exception is, that my boxes are behind a "master" firewall
> which I do not manage.
>
> I have 2 OpenBSD 6.0 servers that should just act as a load balancer
> for SFTP connections. We use DSR mode because huge files get
> downloaded from the SFTP servers and don't want the "load" to
> pass completly through the OpenBSD load balancers.
>
> Everything is working as long as I don't do a failover to the backup
system.
> In this situation, I see, that the "new" carp master "resets" the
connection
> of the client. Immediatly opening a new SFTP sessions then works as
> expected through the "new" carp master.
>
> This is my /etc/pf.conf (identical on both). Still testing..
>
> # cat /etc/pf.conf
> carp_if = "vmx0"
> sync_if = "vmx1"
> # already allow pfsync and carp protocols
> pass quick on $sync_if proto pfsync keep state (no-sync)
> pass on $carp_if proto carp keep state (no-sync)
> # allow relayd to communicate with pf and set rules
> anchor "relayd/*"
>
> And this is the relayd.conf
>
> log updates
> prefork 5
> fx_vip = "VIP"
> table <fxhosts> {
>     "host1"
>     "host2"
> }
> redirect FX-SFTP {
>     listen on $fx_vip port 22 interface vmx0
>     route to <fxhosts> check tcp interface vmx0
>     sticky-address
> }
>
> This is the "ruleset" (identical on both) after reloading pf
>
> # pfctl -a '*' -s rules
> pass quick on vmx1 proto pfsync all keep state (no-sync)
> pass on vmx0 proto carp all keep state (no-sync)
> anchor "relayd/*" all {
>   anchor "FX-SFTP" all {
>     pass in quick on vmx0 on rdomain 0 inet proto tcp from any to VIP port =
22
>     flags any keep state (sloppy, tcp.established 600)
>     route-to <FX-SFTP>@vmx0 round-robin sticky-address
>   }
> }
>
> When the first connection is made, I see the state on the
> backup carp machine. But with slightly different content.
>
> This is on "master"
>
> all tcp VIP:22 <- CLIENT:43334       ESTABLISHED:ESTABLISHED
>    [0 + 1]  [946261580 + 2]
>    age 00:00:35, expires in 00:09:37, 16:0 pkts, 913:0 bytes, anchor 2, rule
2, sloppy
>    id: 57fbd5520000a2b4 creatorid: d4cdd00a
>
> "expires" is 10 minutes (tcp.established 600) and I see the anchor and rule
> which generated state
>
> This in on "backup"
>
> all tcp VIP:22 <- CLIENT:43334       ESTABLISHED:ESTABLISHED
>    [0 + 1]  [946261580 + 2]
>    age 00:00:32, expires in 23:59:41, 0:0 pkts, 0:0 bytes, sloppy
>    id: 57fbd5520000a2b4 creatorid: d4cdd00a
>
> expires is 1 day (?) and "backup" did not yet see any packes.
>
> Now, how can I get this to work, so the sessions won't be terminated
> in case of a failover.
>
> Every help will be appreciated.
>
> Kind regards,
> Robert
>
>
> > Gesendet: Mittwoch, 12. Oktober 2016 um 14:18 Uhr
> > Von: "Robert Paschedag" <robert.pasche...@web.de>
> > An: misc@openbsd.org
> > Betreff: Re: pf on carp backup resets connection after failover
> >
> > Sorry for this bad web mailer formatting. I didn't want that.Am 12.10.2016
2:08 nachm. schrieb Robert Paschedag <robert.pasche...@web.de>:
> > >
> > > Hi all, basically, if have exactly this problem already described
here(https://groups.google.com/forum/#!topic/bit.listserv.openbsd-pf/yZn4EUjx
wfY).But
> > > because there is no answer since 2009, I'll give it a try. The setup of
> > > the 2 servers is also the same as in the other threadonly exception is,
> > > that my boxes are behind a "master" firewallwhich I do not manage. I
have
> > > 2 OpenBSD 6.0 servers that should just act as a load balancerfor SFTP
> > > connections. We use DSR mode because huge files getdownloaded from the
> > > SFTP servers and don't want the "load" topass completly through the
> > > OpenBSD load balancers. Everything is working as long as I don't do a
> > > failover to the backup system.In this situation, I see, that the "new"
> > > carp master "resets" the connectionof the client. Immediatly opening a
> > > new SFTP sessions then works asexpected through the "new" carp master.
> > > This is my /etc/pf.conf (identical on both). Still testing.. # cat
> > > /etc/pf.conf
> > > carp_if = "vmx0"
> > > sync_if = "vmx1"# already allow pfsync and carp protocols
> > > pass quick on $sync_if proto pfsync keep state (no-sync)
> > > pass on $carp_if proto carp keep state (no-sync)# allow relayd to
> > > communicate with pf and set rules
> > > anchor "relayd/*" And this is the relayd.conf log updates
> > > prefork 5fx_vip = "VIP"table <fxhosts> {
> > > "host1"
> > > "host2"
> > > }redirect FX-SFTP {
> > > listen on $fx_vip port 22 interface vmx0
> > > route to <fxhosts> check tcp interface vmx0
> > > sticky-address
> > > }
> > > This is the "ruleset" (identical on both) after reloading pf # pfctl -a
> > > '*' -s rules
> > > pass quick on vmx1 proto pfsync all keep state (no-sync)
> > > pass on vmx0 proto carp all keep state (no-sync)
> > > anchor "relayd/*" all {
> > > anchor "FX-SFTP" all {
> > > pass in quick on vmx0 on rdomain 0 inet proto tcp from any to VIP port =
> > > 22 flags any keep state (sloppy, tcp.established 600) route-to
> > > <FX-SFTP>@vmx0 round-robin sticky-address
> > > }
> > > } When the first connection is made, I see the state on thebackup carp
> > > machine. But with slightly different content. This is on "master" all
tcp
> > > VIP:22 <- CLIENT:43334 ESTABLISHED:ESTABLISHED
> > > [0 + 1]  [946261580 + 2]
> > > age 00:00:35, expires in 00:09:37, 16:0 pkts, 913:0 bytes, anchor 2,
> > > rule 2, sloppy  id: 57fbd5520000a2b4 creatorid: d4cdd00a "expires" is
10
> > > minutes (tcp.established 600) and I see the anchor and rulewhich
> > > generated state This in on "backup" all tcp VIP:22 <- CLIENT:43334
> > > ESTABLISHED:ESTABLISHED
> > > [0 + 1]  [946261580 + 2]
> > > age 00:00:32, expires in 23:59:41, 0:0 pkts, 0:0 bytes, sloppy
> > > id: 57fbd5520000a2b4 creatorid: d4cdd00a expires is 1 day (?) and
> > > "backup" did not yet see any packes. Now, how can I get this to work, so
> > > the sessions won't be terminatedin case of a failover. Every help will
be
> > > appreciated. Kind regards,Robert

Reply via email to