On Mon, Jan 09, 2017 at 10:01:21PM -0500, Ian FREISLICH wrote: > On 01/09/17 17:17, Marek Zarychta wrote: > > On Mon, Jan 09, 2017 at 09:58:38PM +0100, Kristof Provost wrote: > >> On 9 Jan 2017, at 18:25, Marek Zarychta wrote: > >>> On Sun, Jan 08, 2017 at 07:08:10PM +0100, Kristof Provost wrote: > >>>> On 8 Jan 2017, at 15:55, Marek Zarychta wrote: > >>>> The problem description doesn’t ring any bells with me, but I’m > >>>> also > >>>> not sure > >> I’ve fully understood it. Can you document a minimal > >>>> reproduction > >>>> scenario, > >>>> with a pf.conf and perhaps network captures documenting the problem? > >>>> > >>>> There’s certainly not been a conscious decision to break UDP > >>>> reply-to. > >>>> > >>> Let me apologize, the problem wasn't previously properly identified. > >>> It > >>> seems to be more problem of UDP protocol implementation than PF issue. > >>> UDP sockets are opened and bound to address of the outgoing interface > >>> (interface which has a route to the client). Because the socket is not > >>> bound to the incoming interface, the PF reply-to rules couldn't be > >>> evaluated. By the way, TCP sockets are bound to the interface where > >>> the > >>> traffic arrives and everything works fine. > >>> This machine is i386 running 11.0-STABLE r311772 > >>> > >>> The problem remains unresolved. Are there any corresponding sysctls > >>> correcting this behavior and enabling the opportunity to use PF > >>> assisted > >>> symmetric routing scenario again? > >>> > >> How are your UDP listen sockets set up? > >> Are they bound to a specific interface, or are they listening on > >> 0.0.0.0? > > Yes, socket is listening on 0.0.0.0, the client from outside network is > > initiating connection and initiating packet arrives on interface B, > > which is supposed only to communicate with devices on its own network > > (no additional routes go via this interface), so normally the reply > > would be passed via interface A having default gateway in scope and > > communication would fail. > > With the assistance of PF reply-to rule, TCP services are able to pass > > reply from interface B via other, second gateway: reply-to (B GW2). > > Are you saying that your network looks approximately like this and there > is no route to the client network where X resides on your server: > > iface-A--<default route>--GW1 > iface-B--_local network_--GW2--_client X_ > > Client X originates a UDP "connection" to B and that return traffic to X > leaves interface A despite your reply-to rule. > > I would be very interested to know: > > 1. whether the reply-to rule actually matches on the inbound traffic. > You can make the rule log and tcpdump on the pflog0 interface. I > believe the -e option to tcpdump will show the rule that matched. > > 2. the output of "pfctl -s sta |grep IP_of_X" > > 3. what software you're using for your UDP server. I can try to > reproduce your issue. > > Ian Both interfaces have public IP addresses, server has a route to client via GW1, client originates connection to B via GW2. TCP replies originate from socket associated with (bound to) interface B so reply-to works as expected and client is able to connect to both interfaces simultaneously. Corresponding PF states are shown below all tcp ADDR_A:22 <- CLIENT_ADDR:57598 ESTABLISHED:ESTABLISHED all tcp ADDR_B:22 <- CLIENT_ADDRR:58856 ESTABLISHED:ESTABLISHED
UDP datagrams never originate from socket B ignoring client effort to
use socket associated with interface B, what was investigated using
sockstat. Server always tries to respond via interface A and simply
create new PF state as shown below.
all udp ADDR_B:1194 <- CLIENT_ADDRR:35607 NO_TRAFFIC:SINGLE
all udp ADDR_A:1194 -> CLIENT_ADDRR:35607 SINGLE:NO_TRAFFIC
This is also seen by pflog:
06:38:52.195266 rule 487/0(match): pass in on B: CLIENT_ADDRR.35607
> ADDR_B.1194: UDP, length 42
06:38:52.196970 rule 488/0(match): pass out on A: ADDR_A.1194 >
CLIENT_ADDRR.35607: UDP, length 54
Rules 487 and 488 are shown below:
pass in log quick on $B \
reply-to ( $B $GW2 ) \
inet proto udp \
from any \
to ($ext_if_2:0) port $udp_services \
keep state
pass out log quick on $A proto
udp \
from any to any \
keep state
Successful setup has been previously used to allow client X use address
of interface B as OpenVPN remote address. The scenario could be
reproduced simply using netcat as a server and client: nc -u -l 1194,
nc -u ADDR_B 1194.
>
> > This functionality is currently broken for any UDP service, because UDP
> > sockets are always opened on supposed_to_be_outgoing interface A and
> > bound to the address of this interface, which is considered good from
> > routing table perspective, but silently breaks PF reply-to for UDP.
> >
> > When the machine was running 9-STABLE reply-to had successfully been
> > used to assist both TCP and UDP driven services.
> >
> > Is anyone reading this list still using reply-to rule for routing UDP
> > traffic back via incoming interface?
> >
> > Maybe currently, the better place to discuss this questions would be
> > freebsd-net, but the thread was initiated here.
> >
> >> I’m afraid I’m still struggling to understand what your setup is,
> >> what you’re
> >> expecting to see and what you’re seeing instead.
> >>
> >> Regards,
> >> Kristof
> >
> >
>
>
> --
>
>
> Cape Augusta Digital Properties, LLC a Cape Augusta Company
>
> *Breach of confidentiality & accidental breach of confidentiality *
>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you have received this email in error please notify the system manager.
> This message contains confidential information and is intended only for the
> individual named. If you are not the named addressee you should not
> disseminate, distribute or copy this e-mail. Please notify the sender
> immediately by e-mail if you have received this e-mail by mistake and
> delete this e-mail from your system. If you are not the intended recipient
> you are notified that disclosing, copying, distributing or taking any
> action in reliance on the contents of this information is strictly
> prohibited.
> _______________________________________________
> [email protected] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-pf
> To unsubscribe, send any mail to "[email protected]"
>
--
Marek Zarychta
signature.asc
Description: PGP signature
