Fwd: Understanding PF behavior

2023-04-01 Thread Kaya Saman

Well... somehow I managed to get inter rdomain forwarding.


I have no idea how...?


I think things started to work when I changed this statement in PF: 
block log on rdomain 0 from "block log"



Right now I can only communicate between rdomain 2 and rdomain 0.


I moved my ISP-B interface onto rdomain 3 and now can't ping the public 
IP address either from domain 2 or domain 0



I did take a snapshot of the routing tables for each domain and of 
course pf.conf is unaltered but I should back it up and transfer it locally.



Like I wrote previously about using 'tcpdump enipflog'... the rule 
numbers don't make any sense at all to me. I don't understand why I keep 
seeing "rule 1" for just about all traffic. - It's definitely strange?? 
Perhaps my pf.conf file is totally messed up as far as rules go? I am 
not sure. It would be really nice to see the matched evaluation numbers 
from: pfctl -vv -sr


so something like 'rule 1183' or so


Currently I am seriously thinking about just spending $$$ and buying a 
Cisco router with 3x interfaces to use as a multi WAN gateway, though 
it's probably more out of frustration then anything else. I'm sure it is 
possible to get working in OpenBSD as Stu has said already but not 
making any headway or little headway after so long is well gr lol



I guess right now my goal is just to be able to ping the ISP-B interface 
from rdomain 2. If I can managed that I should have a better path 
forward. Really what I do need is a test box... something with 2x or 3x 
physical interfaces that won't cause my whole system to stop working by 
starting with a clean pf.conf file. That said I the SuperMicro uATX box 
I have doesn't work either :-( as it's started clicking so no idea where 
the fault is? M/B or PSU?? More headache :-(



Too much crap on my shoulders right now also with 5x HDD failures and a 
15 year old Cisco WLC system which is flaky to connect to meaning that 
more often then not 802.11 devices are not connected. I really wish I 
could just upgrade to a nice Gen6 system sigh.



What a frustrating way to spend a Saturday evening but I guess it won't 
get any better so bla :-(



Anyway will keep trying to solve this darn riddle



Understanding PF behavior

2023-03-31 Thread Kaya Saman

Hi guys,


So far I have spent a week on this and I feel like I'm not progressing, 
now I just feel like I'm banging against a brick wall.



To start with, I managed to send icmp echos over my WAN link through 
ISP-B within the same routing domain rdomain 2.



I then started looking at inter-Rdomain communication. The only 
reference material I have got outside of the man pages are a few 
outdated websites and other material which at least have gotten me in 
the right direction.



From here on I have two issues.


The first issue is that my WAN communication for rdomain 2 broke. I 
haven't adjusted any rules at all so the original working ruleset is 
still in place.



Currently what I can see is tcpdump telling me that I have icmp 
echo-reply packets on my test vlan for rdomain 2. I have checked pflog0 
and the external IF for ISP-B using tcpdump and all seem normal too with 
lots of "pass" statements and of course NAT is being hit.



Unfortunately the machine sending the icmp requests is claiming that 
there is 100% packet loss.



I don't see any relevant "Block" information in tcpdump when initiating 
"tcpdump -enipflog0 host x.x.x.2" - where x.x.x.2 is my test machine 
address.



What could beĀ  going on here? I have considered the fact that it maybe a 
rule blocking on the outbound side of the interface (local test vlan), 
but I tried adding a generic: pass out quick on $test_vlan rule which 
didn't seem to do anything at all :-(




The second issue I have is to do with the routing domains. At the moment 
I have a ruleset which allows me to get from my test vlan to one of the 
internal vlans on rdomain0. I can verify with tcpdump that the pinged 
machine can see the icmp packets and respond to them. What I am seeing 
however in pflog0 is the lo0 is blocking traffic outbound.



As this is the reference guide I'm following: 
https://www.packetmischief.ca/2011/09/20/virtualizing-the-openbsd-routing-table/



I'm using y.y.y.y/24 subnet for the icmp destination in rd0


I created routes to the loopback addresses:


route -T 0 add y.y.y/24 127.0.0.1

route -T 2 add y.y.y/24 127.0.0.1


I found that if I didn't add this rule, information to the source of the 
icmp echos would be routed to my egress interface on ISP-A (existing ISP):


route -T 0 add x.x.x/24 127.0.0.1


Now, the unfortunate part is that I am seeing:

rule 2/(match) block out on lo0: y.y.y.N > x.x.x.2 icmp: echo reply


The thing I don't understand is that adding: pass out log quick on lo0

unfortunately doesn't seem to do anything??


Just to confirm that "set skip on lo" is disabled additionally.


If anyone could help in suggesting ideas of what to look at because I 
have tried many things up till now and none have worked so I'm probably 
approaching the problem in the wrong way.




Another quick question regarding the output of tcpdump. When it says 
"rule 2" which rule is it referring to and how to tell this information? 
In the past I recall the rule numbers given by tcpdump where the actual 
rule line numbers as were written in pf.conf. Now I am completely 
confused of what this means...


If I use "pfctl -sr" to show the rules, the second rule inline is 
actually: "match in all scrub (no-df)" ???



Many thanks.


Kaya