Re: rdomains finally working!!
On 4/3/23 11:25, Claudio Jeker wrote:
On Mon, Apr 03, 2023 at 10:53:26AM +0100, Kaya Saman wrote:
Hey guys,
...
Taking an excerpt from the website I was following:
https://www.packetmischief.ca/2011/09/20/virtualizing-the-openbsd-routing-table/
Citing:
Creating a loopback interface in rdomain 2 so that Host 1 can talk to Host 2
would look like:
ifconfig lo2 rdomain 2 127.0.0.1
route -T 2 add 192.168.1/24 127.0.0.1
Since lo2 is created inside rdomain 2, the IP address assigned to it doesn't
conflict with lo0 in rdomain 0.
Sure I can see traffic inside one of the loopbacks and tcpdump does claim
"pass out" but then nothing else happens. The other loopback interfaces have
no traffic in them and the destination network has no traffic either.
This is very much expected since you probably did not carefully read the
cited website.
You need a special pf.conf setup to make that work. As one caveat
mentioned in the article is that the default pf.conf rulesets skips lo(4)
interfaces and so the traffic will just be dropped (since there is no
state lookup and so no way to bounce the reverse traffic back into the
other rdomain).
In general I would suggest use pair(4) to route traffic between rdomains.
Doing it in pf(4) gives you more control but it requires careful handling
of the rulesets (as you noticed).
Hi Claudio,
thanks for the response and advice on pair (4), I will definitely read it.
Maybe you are correct in that I didn't carefully read or perhaps I
confused things badly I have a mixture of ASD and most likely ADHD
and when the panic kicks in things become difficult including
communication and understanding.
I wish I had a spare system to test things properly and understand
better, unfortunately I am having to adapt a live production system and
with a large number of lines in the PF rule set is not easy.
The caveat you mention about skipping lo (4) I disabled (I think?) as
per changing the 'set skip' to this:
#set skip on { lo, enc0 }
set skip on { enc0 }
so the loopback should be active in PF, further down in my config I have
a clause like this too:
#Allow Internal Communications
pass in on lo0
pass out on lo0
pass in on lo2
pass out on lo2
pass in on lo3
pass out on lo3
Maybe due to my existing PF things might not work properly in anycase, I
just wish I knew more about what I was doing to really have a handle on
the situation.
I'm still eager to contribute with a write up if you are interested but
due to my "circumstances" I maybe the only one with these issues :-(
so maybe writing things up in my case maybe useless and will probably
not be understood by anyone in general.
With PF the biggest handling issue is when mixing and matching using the
'quick' keyword as things get handled differently. Both the websites I
cited do *not* contain the 'quick' keyword at all. My local pf ruleset
contains many "pass in quick" or "block in quick" statements.
I would definitely be extremely happy to hear any more suggestions if
there are any but for now I will look at studying "pair (4)".
Thank you so much for chiming in! I really appreciate it :-D
Best regards,
Kaya
Re: rdomains finally working!!
On Mon, Apr 03, 2023 at 10:53:26AM +0100, Kaya Saman wrote: > Hey guys, > ... > Taking an excerpt from the website I was following: > > https://www.packetmischief.ca/2011/09/20/virtualizing-the-openbsd-routing-table/ > > Citing: > > Creating a loopback interface in rdomain 2 so that Host 1 can talk to Host 2 > would look like: > > ifconfig lo2 rdomain 2 127.0.0.1 > route -T 2 add 192.168.1/24 127.0.0.1 > Since lo2 is created inside rdomain 2, the IP address assigned to it doesn't > conflict with lo0 in rdomain 0. > > > Sure I can see traffic inside one of the loopbacks and tcpdump does claim > "pass out" but then nothing else happens. The other loopback interfaces have > no traffic in them and the destination network has no traffic either. This is very much expected since you probably did not carefully read the cited website. You need a special pf.conf setup to make that work. As one caveat mentioned in the article is that the default pf.conf rulesets skips lo(4) interfaces and so the traffic will just be dropped (since there is no state lookup and so no way to bounce the reverse traffic back into the other rdomain). In general I would suggest use pair(4) to route traffic between rdomains. Doing it in pf(4) gives you more control but it requires careful handling of the rulesets (as you noticed). -- :wq Claudio
rdomains finally working!!
Hey guys, I can't spend too much time right now in writing up what I have done right now as I'm just about to head out of town to a local nature reserve in order to clear my head, but basically things are working!! At this stage with such limited documentation on the topic I wonder if I should do some kind of a writeup of it to share my own experiences? Hopefully later in the week and I'll pop it up on my GDrive maybe it can be turned into part of the doc or faq or an example page on the OBSD website? What I can gloss over right now is that the setup is very sensitive with minute changes having drastic effects. To be honest I don't even know how or when things started working but I checked my laptop test machine and icmp packet responses were being received properly. Running tcpdump on pflog does lie sometimes. It will often suggest "block" when in fact the problem is routing. Another issue that I came across is that the loopback addresses don't inter-route. I don't know how to make that one work at all? Taking an excerpt from the website I was following: https://www.packetmischief.ca/2011/09/20/virtualizing-the-openbsd-routing-table/ Citing: Creating a loopback interface in rdomain 2 so that Host 1 can talk to Host 2 would look like: ifconfig lo2 rdomain 2 127.0.0.1 route -T 2 add 192.168.1/24 127.0.0.1 Since lo2 is created inside rdomain 2, the IP address assigned to it doesn't conflict with lo0 in rdomain 0. Sure I can see traffic inside one of the loopbacks and tcpdump does claim "pass out" but then nothing else happens. The other loopback interfaces have no traffic in them and the destination network has no traffic either. Over the weekend having gotten completely fedup I decided to try to use one of the rdomains as a transit system and connected my Cisco 2801 BGP/IPSec/GRE test bed to the vlan in question and attempted to use it as a simple WAN Edge gateway. At first I managed to ping the interface of the Cisco, as I set OSPF up so that I didn't need to bother with static routes. As soon as I moved the rdomain of the vlan from 0 over to 3, everything stopped working and I couldn't find a way to use OSPF within two separate routing tables. The man page does have a setting for this in the config but there is no mention if you can use 2x instances between rdomains. After that I fought with static routes and of course the strange behavior kicked in where sending icmp packets between the Cisco and OpenBSD machines just wasn't working. I really didn't understand this at all as they are Next-Hop connected devices. No specific PF rule and a default route was set to be the interface of the Cisco routers internal IP address. I ended up moving the setup back to what I had originally with 3x rdomains and again I could not communicate between rdomain 2 and rdomain 3, though I did have communication to rdomain 0 from rdomain 2. Having done some more thinking about it and looking at the Cisco setup I decided to try creating an extra vlan in rdomain 3 with the intention of using it as a transit vlan. I already had success in pinging the internal IP address of the OpenBSD machine from domain 2 to domain 3, so I figured what the heck I'll give it a try. It did work but I still couldn't ping the public address. I then moved the test machine from rdomain 2 to rdomain 3 to see if that would help with internal communication between the machine and the public IP address. Oddly enough it did work and local traffic sent to rdomain 0 was still working too.. though I was still unable to get any response from public address. This is a bit of a black hole for me because I left the system for a while and came back to find that it actually started to work?? It isn't logical of course but I can't recall if I made changes in the meantime and simply wasn't aware of anything happening. After testing a bit and confirming that in fact things were working and I was *not* sending traffic over my existing PlusNET WAN link, I proceeded to migrate my OpenVPN tunnel across. This proved to be tricky as I kept running into dreaded "tcpdump -enipflog" "block in on tun0" issues in fact that turned out to be a routing problem instead of a PF issue. Currently this is working too but with some strange adjustments to PF. The current 'pass' per port rulesets don't seem to work even with the 'rtable (x)' clause added. What does seem to work however is a generic 'pass to rtable (x)' ruleset I'll probably need to study my rules further in order to try to understand the behavior here Anyway, am gona shoot over to the Warburg Nature Reserve now and continue with my photography to hopefully share more images with the BBOWT later. Oh and of course to try to get rid of my headache too ;-) Thanks to everyone for trying to help on a subject that contains very limited documentation and examples. Like I said hopefully if I can do
