Re: Incoming connection via VLAN
On Fri, Aug 30, 2019 at 11:32:02AM +1000, Felix Hanley wrote:
> Hello all,
>
> My home internet connection (Internode Australia) has recently been
> "upgraded" and is now delivered via vlan ID 2. Previously had the
> following configuration which worked without issue:
>
> # cat /etc/hostname.em0
> up
>
> # cat /etc/hostname.pppoe0
> inet 0.0.0.0 255.255.255.255 NONE \
> pppoedev em0 authproto pap \
> authname 'x...@internode.on.net' \
> authkey '' up
> dest 0.0.0.1
> inet6 eui64
> !/sbin/route add default -ifp pppoe0 0.0.0.1
> !/sbin/route add -inet6 default -ifp pppoe0 fe80::%pppoe0
> !/etc/rc.d/dhcp6c restart
> !/sbin/pfctl -ef /etc/pf.conf
>
> After working out the vlan stuff I now have the following:
>
> # cat /etc/hostname.em0
> up
>
> # cat /etc/hostname.vlan2
> vnetid 2 parent em0 txprio 1
> up
>
> # cat /etc/hostname.pppoe0
> inet 0.0.0.0 255.255.255.255 NONE \
> llprio 1 mtu 1440 \
> pppoedev vlan2 authproto pap \
> authname 'x...@internode.on.net' \
> authkey '' up
> dest 0.0.0.1
> inet6 eui64
> !/sbin/route add default -ifp pppoe0 0.0.0.1
> !/sbin/route add -inet6 default -ifp pppoe0 fe80::%pppoe0
> !/etc/rc.d/dhcp6c restart
> !/sbin/pfctl -ef /etc/pf.conf
>
> I am able to access the internet fine. My problem is incoming
> connections are unable to access the OBSD router but are able to be
> redirected to internal hosts just fine. There was no problems with this
> prior to the vlan stuff. My stripped down pf.conf is:
>
> # cat /etc/pf.conf
> egress = "pppoe0"
> zappa = "10.0.1.2"
>
> set skip on lo
> set skip on vlan2
> set block-policy drop
> set loginterface $egress
>
> queue outq on $egress bandwidth 13M max 13M flows 1024 qlimit 1024 default
>
> match in inet all scrub (no-df random-id)
> match on $egress inet scrub (max-mss 1440)
> # NAT all outbound IPv4 traffic from the rest of our network
> match out on $egress inet from !($egress:network) to any nat-to ($egress:0)
>
> antispoof quick for lo
>
> pass in on $egress proto { tcp udp } from any to ($egress) port { ssh
> http https }
> pass in on $egress proto tcp from any to ($egress) port 51022 rdr-to
> $zappa port ssh
>
> Running tcpdump on pppoe0 show ICMP packets but never any SSH (or other
> TCP) packets coming in on egress. I am confused that rdr-to works but
> not connections to the router do not.
>
> Any help would be greatly appreciated.
>
> -felix
So it turns out that my configuration was fine.
Internode reinstated their default firewall when they set up my new
service. This was blocking incoming ports 80, 443 etc.
Thanks for your time.
-felix
Re: Incoming connection via VLAN
On Tue, Sep 03, 2019 at 09:54:24PM -, Stuart Henderson wrote: > Please show ifconfig -A output. Not sure but maybe it will give us a clue. > Looking through it now, I am not sure about the 'llprio' and 'txprio' on vlan2 and pppoe0, but I can't play with them right now as I am connected remotely (I have to forward to another internal host and then back to the router!) and don't want it to break. $ doas ifconfig -A lo0: flags=8049 mtu 32768 index 6 priority 0 llprio 3 groups: lo inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6 inet 127.0.0.1 netmask 0xff00 em0: flags=8843 mtu 1500 lladdr 00:0d:b9:4c:03:74 index 2 priority 0 llprio 3 media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause) status: active em1: flags=8b43 mtu 1500 lladdr 00:0d:b9:4c:03:75 index 3 priority 0 llprio 3 media: Ethernet autoselect (100baseTX full-duplex,rxpause,txpause) status: active em2: flags=8b43 mtu 1500 lladdr 00:0d:b9:4c:03:76 index 4 priority 0 llprio 3 media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause) status: active enc0: flags=0<> index 5 priority 0 llprio 3 groups: enc status: active bridge0: flags=41 index 7 llprio 3 groups: bridge priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp em2 flags=3 port 4 ifpriority 0 ifcost 0 em1 flags=3 port 3 ifpriority 0 ifcost 0 vether0 flags=3 port 10 ifpriority 0 ifcost 0 Addresses (max cache: 100, timeout: 240): 00:17:c8:3e:08:22 em2 0 flags=0<> 1c:c3:eb:68:05:29 em1 0 flags=0<> b8:bc:1b:1e:9d:9f em1 0 flags=0<> 38:f9:d3:47:db:54 em1 1 flags=0<> 48:bf:6b:e6:27:c2 em1 0 flags=0<> 74:d4:35:80:51:91 em2 1 flags=0<> 74:44:01:81:9b:7e em1 0 flags=0<> pflow0: flags=1 mtu 1492 index 8 priority 0 llprio 3 pflow: sender: 10.0.1.1 receiver: 10.0.1.2:INVALID version: 5 groups: pflow vether0: flags=8943 mtu 1500 lladdr fe:e1:ba:d0:39:34 index 10 priority 0 llprio 3 groups: vether media: Ethernet autoselect status: active inet 10.0.1.1 netmask 0xff00 broadcast 10.0.1.255 inet6 fe80::71c1:1036:9b53:e5ff%vether0 prefixlen 64 scopeid 0xa inet6 2001:44b8:41ae:b001::baff:: prefixlen 64 pflog0: flags=141 mtu 33136 index 12 priority 0 llprio 3 groups: pflog vlan2: flags=8843 mtu 1500 lladdr 00:0d:b9:4c:03:74 index 14 priority 0 llprio 3 encap: vnetid 2 parent em0 txprio 1 groups: vlan media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause) status: active inet 10.0.3.1 netmask 0xff00 broadcast 10.0.3.255 pppoe0: flags=8851 mtu 1440 index 15 priority 0 llprio 1 dev: vlan2 state: session sid: 0xf7a1 PADI retries: 0 PADR retries: 0 time: 3d 11:50:50 sppp: phase network authproto pap authname "xxx...@internode.on.net" groups: pppoe egress status: active inet6 fe80::516a:6d3c:8072:6303%pppoe0 -> prefixlen 64 scopeid 0xf inet 194.193.XXX.XXX --> 10.20.25.29 netmask 0x
Re: Incoming connection via VLAN
Please show ifconfig -A output. Not sure but maybe it will give us a clue.
Re: Incoming connection via VLAN
On Mon, Sep 02, 2019 at 05:51:23PM -, Stuart Henderson wrote: > On 2019-09-01, Felix Hanley wrote: > > I had assumed I would be able to use the existing pf.conf (which has > > worked for years) even after the introduction of the vlan2 interface > > as the pppoe0 parent. To get anything to work I had to remove all > > queueing references. > > Note that queues should be done on the *physical* interface, i.e. the > ethernet interface that is the parent of the vlan that is the parent of > the pppoe. I did not know that, thank you. I have no queueing at the moment. It is as if the daemons do not listen on the new em0 -> vlan2 -> pppoe0 chain of interfaces. I cannot even rdr-to localhost to connect to them. I have tried all the following variations: - IP address on vlan2 - Explicitly listening on various IP addresses (on vlan2 and pppoe0) - Disabling IPv6 completely The only incoming connections that work are those that I rdr-to hosts on the internal network. I am suspicious of my vlan2 config, particularly the txprio setting. It does not work without it but I know little about DSCP so I am not sure if I need to add something to pf.conf as well. Would that even stop packets to local daemons??: # cat /etc/hostname.vlan2 vnetid 2 parent em0 txprio 1 up Thanks again for your help. -felix
Re: Incoming connection via VLAN
On Mon, Sep 02, 2019 at 10:55:18AM -0400, Daniel Ouellet wrote: > It's hard trying to help you as. > > Vlan syntax changed from the upgrade or 6.1 to 6.2 and the pf queuing > changed from 6.3 to 6.4. > > So looks like you skip a few version and no where did you provide any > details on your configuration. I have not skipped any versions. My configs were in the original post. > So I would suggest to go and read either the man page or look at the > upgrade from 61. to 6.2 for your vlan part. > > https://www.openbsd.org/faq/upgrade62.html Yes, I am using the new syntax. > and then 6.3 to 6.4 for your pf part. > > https://www.openbsd.org/faq/upgrade64.html > > If you do upgrade a system it's always a good idea to go read the > excellent upgrade page before doing it. I have read the precious man pages and have not resolved the issue, hence posting to misc. > Assuming things never changed is not a good idea. Agreed. > OpenBSD will changed everything if that make sense to do at time, but > they also document it as well. > > For what I can read anyway and guess from your info is that look to me > to upgrade or skip a few version, or run an old configuration on a much > newer system without looking changes that happens. > > Worst case get your system working again and then read the vlan part if > you still have issue and experiment with that and get it back where you > want it. Read it, numerous times. > In any case with what you provided it's not possible to help or tell you > more, everything I wrote here is simply a guess based on your info. > > Hope this help you some. > > Daniel Thanks.
Re: Incoming connection via VLAN
On 2019-09-01, Felix Hanley wrote: > I had assumed I would be able to use the existing pf.conf (which has worked > for years) even after the introduction of the > vlan2 interface as the pppoe0 parent. To get anything to work I had to remove > all queueing references. > > BTW, I am running 6.5: > > # uname -a > OpenBSD malkmus.xx.xx 6.5 GENERIC.MP#3 amd64 > > Thank you for any suggestions to try. > > -felix > > Note that queues should be done on the *physical* interface, i.e. the ethernet interface that is the parent of the vlan that is the parent of the pppoe.
Re: Incoming connection via VLAN
It's hard trying to help you as. Vlan syntax changed from the upgrade or 6.1 to 6.2 and the pf queuing changed from 6.3 to 6.4. So looks like you skip a few version and no where did you provide any details on your configuration. So I would suggest to go and read either the man page or look at the upgrade from 61. to 6.2 for your vlan part. https://www.openbsd.org/faq/upgrade62.html and then 6.3 to 6.4 for your pf part. https://www.openbsd.org/faq/upgrade64.html If you do upgrade a system it's always a good idea to go read the excellent upgrade page before doing it. Assuming things never changed is not a good idea. OpenBSD will changed everything if that make sense to do at time, but they also document it as well. For what I can read anyway and guess from your info is that look to me to upgrade or skip a few version, or run an old configuration on a much newer system without looking changes that happens. Worst case get your system working again and then read the vlan part if you still have issue and experiment with that and get it back where you want it. In any case with what you provided it's not possible to help or tell you more, everything I wrote here is simply a guess based on your info. Hope this help you some. Daniel On 9/1/19 9:04 AM, Felix Hanley wrote: > I had assumed I would be able to use the existing pf.conf (which has worked > for years) even after the introduction of the > vlan2 interface as the pppoe0 parent. To get anything to work I had to remove > all queueing references. > > BTW, I am running 6.5: > > # uname -a > OpenBSD malkmus.xx.xx 6.5 GENERIC.MP#3 amd64 > > Thank you for any suggestions to try. > > -felix >
Re: Incoming connection via VLAN
I had assumed I would be able to use the existing pf.conf (which has worked for years) even after the introduction of the vlan2 interface as the pppoe0 parent. To get anything to work I had to remove all queueing references. BTW, I am running 6.5: # uname -a OpenBSD malkmus.xx.xx 6.5 GENERIC.MP#3 amd64 Thank you for any suggestions to try. -felix
Re: Incoming connection via VLAN
> On xxx.xxx Sep 2019, at 12:33 am, Stuart Henderson
> wrote:
>
> On 2019-08-30, Felix Hanley wrote:
>> Hello all,
>>
>> My home internet connection (Internode Australia) has recently been
>> "upgraded" and is now delivered via vlan ID 2. Previously had the
>> following configuration which worked without issue:
>>
>> # cat /etc/hostname.em0
>> up
>>
>> # cat /etc/hostname.pppoe0
>> inet 0.0.0.0 255.255.255.255 NONE \
>>pppoedev em0 authproto pap \
>>authname 'x...@internode.on.net' \
>>authkey '' up
>> dest 0.0.0.1
>> inet6 eui64
>> !/sbin/route add default -ifp pppoe0 0.0.0.1
>> !/sbin/route add -inet6 default -ifp pppoe0 fe80::%pppoe0
>> !/etc/rc.d/dhcp6c restart
>> !/sbin/pfctl -ef /etc/pf.conf
>>
>> After working out the vlan stuff I now have the following:
>>
>> # cat /etc/hostname.em0
>> up
>>
>> # cat /etc/hostname.vlan2
>> vnetid 2 parent em0 txprio 1
>> up
>>
>> # cat /etc/hostname.pppoe0
>> inet 0.0.0.0 255.255.255.255 NONE \
>>llprio 1 mtu 1440 \
>>pppoedev vlan2 authproto pap \
>>authname 'x...@internode.on.net' \
>>authkey '' up
>> dest 0.0.0.1
>> inet6 eui64
>> !/sbin/route add default -ifp pppoe0 0.0.0.1
>> !/sbin/route add -inet6 default -ifp pppoe0 fe80::%pppoe0
>> !/etc/rc.d/dhcp6c restart
>> !/sbin/pfctl -ef /etc/pf.conf
>>
>> I am able to access the internet fine. My problem is incoming
>> connections are unable to access the OBSD router but are able to be
>> redirected to internal hosts just fine. There was no problems with this
>> prior to the vlan stuff. My stripped down pf.conf is:
>>
>> # cat /etc/pf.conf
>> egress = "pppoe0"
>> zappa = "10.0.1.2"
>>
>> set skip on lo
>> set skip on vlan2
>> set block-policy drop
>> set loginterface $egress
>>
>> queue outq on $egress bandwidth 13M max 13M flows 1024 qlimit 1024 default
>>
>> match in inet all scrub (no-df random-id)
>> match on $egress inet scrub (max-mss 1440)
>> # NAT all outbound IPv4 traffic from the rest of our network
>> match out on $egress inet from !($egress:network) to any nat-to ($egress:0)
>>
>> antispoof quick for lo
>>
>
> I'd suggest adding "block all" or "block log all" here. That way you can be
> sure that any traffic making it through the ruleset has been permitted by one
> of the following "pass" rules (which are stateful rules). Otherwise things
> might only be making it through due to the implicit default-permit rule
> which is not stateful.
Thank you, yes, I actually have that currently. With all the experimentation I
failed to paste it into the email properly.
>
>> pass in on $egress proto { tcp udp } from any to ($egress) port { ssh
>> http https }
>> pass in on $egress proto tcp from any to ($egress) port 51022 rdr-to
>> $zappa port ssh
>>
>> Running tcpdump on pppoe0 show ICMP packets but never any SSH (or other
>> TCP) packets coming in on egress. I am confused that rdr-to works but
>> not connections to the router do not.
>>
>> Any help would be greatly appreciated.
>>
>> -felix
>>
>>
>
> It's odd that you don't see any TCP packets coming in on pppoe0 with
> tcpdump; does that even include port 51022 if you're connected via
> the rdr-to?
I have the simplest pf.conf now with NAT, default block all and a single pass
for port 22. Outgoing connections work fine.
I can see incoming packets destined for port 22 but the ssh client just times
out.
This is the tcpdump output on the server (pppoe0 parent is still the vlan2):
# tcpdump -nvvi pppoe0 port ssh
tcpdump: listening on pppoe0, link-type PPP_ETHER
22:48:07.975217 194.193.xxx.xxx.60497 > 103.236.xxx.xxx.22: P [tcp sum ok]
3286752776:3286752812(36) ack 802539959 win 2048 [tos 0x48] (ttl 63, id 64547, len 88)
22:48:07.984046 103.236.xxx.xxx.22 > 194.193.xxx.xxx.60497: P [tcp sum ok]
1:37(36) ack 36 win 1035 (DF) [tos
0x8] (ttl 56, id 0, len 88)
22:48:07.985298 103.236.xxx.xxx.22 > 194.193.xxx.xxx.60497: P [tcp sum ok]
37:81(44) ack 36 win 1035 (DF) [tos
0x8] (ttl 56, id 0, len 96)
22:48:07.985333 103.236.xxx.xxx.22 > 194.193.xxx.xxx.60497: P [tcp sum ok]
81:133(52) ack 36 win 1035 (DF) [tos
0x8] (ttl 56, id 0, len 104)
22:48:07.985684 194.193.xxx.xxx.60497 > 103.236.xxx.xxx.22: . [tcp sum ok]
36:36(0) ack 37 win 2047 [tos 0x48]
(ttl 63, id 51836, len 52)
22:48:07.986678 194.193.xxx.xxx.60497 > 103.236.xxx.xxx.22: . [tcp sum ok]
36:36(0) ack 81 win 2047 [tos 0x48]
(ttl 63, id 27571, len 52)
22:48:07.987287 194.193.xxx.xxx.60497 > 103.236.xxx.xxx.22: . [tcp sum ok]
36:36(0) ack 133 win 2047 [tos 0x48]
(ttl 63, id 47998, len 52)
22:48:08.306791 194.193.xxx.xxx.60497 > 103.236.xxx.xxx.22: P [tcp sum ok]
36:80(44) ack 133 win 2048 [tos 0x48]
(ttl 63, id 64091, len 96)
22:48:08.315519 103.236.xxx.xxx.22 > 194.193.xxx.xxx.60497: P 133:201(68) ack
80 win 1035 (DF) [tos 0x8] (ttl 56,
id 0, len 120)
22:48:08.319338 194.193.xxx.xxx.60497 > 103.236.xxx.xxx.22: . [tcp sum ok]
80:80(0) ack 201 win 2046 [tos 0x48]
(ttl 63, id 28558, len 52)
Re: Incoming connection via VLAN
On 2019-08-30, Felix Hanley wrote:
> Hello all,
>
> My home internet connection (Internode Australia) has recently been
> "upgraded" and is now delivered via vlan ID 2. Previously had the
> following configuration which worked without issue:
>
> # cat /etc/hostname.em0
> up
>
> # cat /etc/hostname.pppoe0
> inet 0.0.0.0 255.255.255.255 NONE \
> pppoedev em0 authproto pap \
> authname 'x...@internode.on.net' \
> authkey '' up
> dest 0.0.0.1
> inet6 eui64
> !/sbin/route add default -ifp pppoe0 0.0.0.1
> !/sbin/route add -inet6 default -ifp pppoe0 fe80::%pppoe0
> !/etc/rc.d/dhcp6c restart
> !/sbin/pfctl -ef /etc/pf.conf
>
> After working out the vlan stuff I now have the following:
>
> # cat /etc/hostname.em0
> up
>
> # cat /etc/hostname.vlan2
> vnetid 2 parent em0 txprio 1
> up
>
> # cat /etc/hostname.pppoe0
> inet 0.0.0.0 255.255.255.255 NONE \
> llprio 1 mtu 1440 \
> pppoedev vlan2 authproto pap \
> authname 'x...@internode.on.net' \
> authkey '' up
> dest 0.0.0.1
> inet6 eui64
> !/sbin/route add default -ifp pppoe0 0.0.0.1
> !/sbin/route add -inet6 default -ifp pppoe0 fe80::%pppoe0
> !/etc/rc.d/dhcp6c restart
> !/sbin/pfctl -ef /etc/pf.conf
>
> I am able to access the internet fine. My problem is incoming
> connections are unable to access the OBSD router but are able to be
> redirected to internal hosts just fine. There was no problems with this
> prior to the vlan stuff. My stripped down pf.conf is:
>
> # cat /etc/pf.conf
> egress = "pppoe0"
> zappa = "10.0.1.2"
>
> set skip on lo
> set skip on vlan2
> set block-policy drop
> set loginterface $egress
>
> queue outq on $egress bandwidth 13M max 13M flows 1024 qlimit 1024 default
>
> match in inet all scrub (no-df random-id)
> match on $egress inet scrub (max-mss 1440)
> # NAT all outbound IPv4 traffic from the rest of our network
> match out on $egress inet from !($egress:network) to any nat-to ($egress:0)
>
> antispoof quick for lo
>
I'd suggest adding "block all" or "block log all" here. That way you can be
sure that any traffic making it through the ruleset has been permitted by one
of the following "pass" rules (which are stateful rules). Otherwise things
might only be making it through due to the implicit default-permit rule
which is not stateful.
> pass in on $egress proto { tcp udp } from any to ($egress) port { ssh
> http https }
> pass in on $egress proto tcp from any to ($egress) port 51022 rdr-to
> $zappa port ssh
>
> Running tcpdump on pppoe0 show ICMP packets but never any SSH (or other
> TCP) packets coming in on egress. I am confused that rdr-to works but
> not connections to the router do not.
>
> Any help would be greatly appreciated.
>
> -felix
>
>
It's odd that you don't see any TCP packets coming in on pppoe0 with
tcpdump; does that even include port 51022 if you're connected via
the rdr-to?
