Re: When IPSec destination 0.0.0.0/0, I cannot ping directly connected Interfaces

[Hrvoje Popovski]

On 12.3.2024. 17:11, Samuel Jayden wrote:
> Dear Misc,
> 
> I have an OpenBSD device with two interfaces: vport10 with an IP address of
> 192.168.83.1/24 and vport20 with an IP address of 192.168.85.1/24. I have
> configured IPSec to route all traffic from these two vport interfaces to
> another point through an IPSec tunnel using the destination network
> 0.0.0.0/0.
> 
> Due to IPSec operating before kernel routing, I cannot even ping the
> directly connected interfaces' IP addresses.
> 
> I've attempted to implement route-based PF rules to solve the issue, but
> unfortunately, the problem persists.
> I'm looking for a solution that allows for the local traffic between these
> two interfaces to bypass the IPSec tunnel, ensuring they can communicate
> with each other while keeping the IPSec destination network as 0.0.0.0/0.
> 
> Here's my IPSec configuration:
> 
> ike active esp tunnel from { 192.168.83.0/24 192.168.85.0/24 } to {
> 0.0.0.0/0 } \
> peer A.B.C.D \
> main auth hmac-md5 enc 3des group modp1024 lifetime 86400 \
> quick auth hmac-md5 enc 3des group none lifetime 43200 \
> psk "verysecret"
> 
> Thanks in advance.
> 

Hi,

put in ipsec.conf

flow from 192.168.83.0/24 to 192.168.83.0/24 type bypass
flow from 192.168.83.0/24 to 192.168.85.0/24 type bypass
flow from 192.168.85.0/24 to 192.168.85.0/24 type bypass
flow from 192.168.85.0/24 to 192.168.83.0/24 type bypass

and if you have carp than put this also

flow from 192.168.83.0/24 to 224.0.0.18/32 type bypass
flow from 192.168.85.0/24 to 224.0.0.18/32 type bypass

or something like that . .

When IPSec destination 0.0.0.0/0, I cannot ping directly connected Interfaces

[Samuel Jayden]

Dear Misc,

I have an OpenBSD device with two interfaces: vport10 with an IP address of
192.168.83.1/24 and vport20 with an IP address of 192.168.85.1/24. I have
configured IPSec to route all traffic from these two vport interfaces to
another point through an IPSec tunnel using the destination network
0.0.0.0/0.

Due to IPSec operating before kernel routing, I cannot even ping the
directly connected interfaces' IP addresses.

I've attempted to implement route-based PF rules to solve the issue, but
unfortunately, the problem persists.
I'm looking for a solution that allows for the local traffic between these
two interfaces to bypass the IPSec tunnel, ensuring they can communicate
with each other while keeping the IPSec destination network as 0.0.0.0/0.

Here's my IPSec configuration:

ike active esp tunnel from { 192.168.83.0/24 192.168.85.0/24 } to {
0.0.0.0/0 } \
peer A.B.C.D \
main auth hmac-md5 enc 3des group modp1024 lifetime 86400 \
quick auth hmac-md5 enc 3des group none lifetime 43200 \
psk "verysecret"

Thanks in advance.

Re: ipsec hardware recommendation

[Marko Cupać]

Hi,

thank you for suggestions, took me some time to think about them and
reply here.

On Fri, 11 Aug 2023 14:19:44 - (UTC)
Stuart Henderson  wrote:

> If you post your IPsec configuration, perhaps someone can suggest
> whether the choice of ciphers etc could be improved. It can make
> quite a difference.

I have just recently bumped quick enc from aes-128-gcm to aes-256-gcm,
as well as group from modp3072 to ecp256:

ike passive esp transport proto gre from $me to $peer \
  main auth hmac-sha2-256 enc aes-256 group ecp256 lifetime 24h \
  quick enc aes-256-gcm group ecp256 lifetime 8h

I have also increased lifetime from default values because I was
getting quite a lot of INVALID COOKIE messages from isakmpd:

isakmpd[51306]: message_recv: invalid cookie(s) cookiea cookieb
isakmpd[51306]: dropped message from $peer port 500 due to notification
type INVALID_COOKIE


On Sat, 12 Aug 2023 12:17:36 +1000
David Gwynne  wrote:

> The things you can do Right Now(tm) are:
> 
> - upgrade to -current
> 
> the pf purge code has been taken out from under the big kernel lock.
> if you have a lot of pf states, this will give more time to crypto.

I have ~50,000 states during peak time. I can't go -current, but I will
look forward to 7.4. I also read the following articles on undeadly.org:

https://undeadly.org/cgi?action=article;sid=20230807094305
https://undeadly.org/cgi?action=article;sid=20230706115843

Once 7.4 hits, is it expected that changing gre/ipsec to sec(4) could
make positive difference in throughput on same hardware?

> - pick faster crypto algorithms

I posted mine above, I would be thankful to get latest recommendation.

> - try wireguard?

I am testing replacing a few of gre/ipsec with wg interfaces on 7.3 at
the moment. Main problem I am encountering so far is the fact that
`ospfctl reload` does not seem to pick newly added (to ospfd.conf) wg
interfaces. `ospfctl sh int` shows them in DOWN state after reload, and
no OSPFv2-hello packets are being sent until `rcctl restart ospfd`.

It is quite unmaintainable to have to restart ospfd every time
wg interfaces are added or removed from ospfd.conf. Any way around it?
Perhaps on some later releases this will improve? Or am I doing it
wrong?

I have more questions about wireguard but I guess I should better ask
them in another topic.

Thank you in advance,

-- 
Before enlightenment - chop wood, draw water.
After  enlightenment - chop wood, draw water.

Marko Cupać
https://www.mimar.rs/

Re: IPsec over PPPoE

[Jiri Navratil]

Hello Tobias,

Thank you for reply.

On Thu, Aug 24, 2023 at 12:36:07AM +0200, Tobias Heider wrote:
> On Wed, Aug 23, 2023 at 08:03:34AM +0200, Jiri Navratil wrote:
> > Hello,
> > 
> > Thank you for quick and helpful replies.
> > 
> > Adding line
> > 
> > set skip on enc0  
> > 
> > to pf.conf enabled traffic between my sites.
> > 
> > I see in https://www.openbsd.org/faq/faq17.html
> > 
> > "Traffic between them should appear after decapsulation on the enc0
> > interface, and can be filtered as such." and next line works with VPN
> > tag, but there are no lines "pass in ... tag VPN" in pf.conf before this
> > part. Shall that be added to FAQ? I expect, that switch from "set skip on
> > enc0" to "pass in ... tag VPN" will be better in my case.
> > 
> > If someone with IPsec experiences will propose changes to FAQ17, then I
> > also noted:
> > 
> > In "road warrior" part, there is "We'll assume the public IP for the
> > client is 203.0.113.2.", but the example uses "any".
> 
> I think any is the better choice here. This would allow other clients
> to connect to the same server (if they have a valid key) which is probably
> what most people want.

Right, understand, then the text above example for consistence shall
switch from "We'll assume the public IP for the client is 203.0.113.2."
to something "any IPv4 is allowed (if they have a valid key)"

> 
> > 
> > I think, that word "daemon" is better then "server" here: 
> > 
> > The ikectl(8) utility is used to control the server,
> 
> Agree
> 
> > 
> > I want to extend my IKEv2 Site-to-site VPN with road warrior
> > configuration. If the road warrior part will include few lines about,
> > how to extend responder to handle both site-to-site and road warrior, it
> > will be very helpful.
> 
> Are you thinking of an example with multiple "ikev2 ..." blocks or a comment
> mentioning that you can have multiple of those in the same config file?
> Because that is technically all you need.

I have some assumptions, so the comment about "you can combine
Site-to-site and road warrior via combining ..." will be helpful.

Also next comment for me, which still have to do the road warrior part,
I would welcome approach / best practice / configuration / comment for two 
situations

1) I'm outside my two locations (IPsec connected) in road warrior position
2) I'm in one of my two locations with internal IPv4

Thank you,
Jiří

> 
> > 
> > Thank you OpenBSD for IPsec and thank you for your support to let me
> > configure it.
> > 
> > BR,
> > Jiří
> > 
> > -- 
> > Jiri Navratil, https://nocloud.cz


smime.p7s
Description: S/MIME cryptographic signature

Re: IPsec over PPPoE

[Tobias Heider]

On Wed, Aug 23, 2023 at 08:03:34AM +0200, Jiri Navratil wrote:
> Hello,
> 
> Thank you for quick and helpful replies.
> 
> Adding line
> 
> set skip on enc0  
> 
> to pf.conf enabled traffic between my sites.
> 
> I see in https://www.openbsd.org/faq/faq17.html
> 
> "Traffic between them should appear after decapsulation on the enc0
> interface, and can be filtered as such." and next line works with VPN
> tag, but there are no lines "pass in ... tag VPN" in pf.conf before this
> part. Shall that be added to FAQ? I expect, that switch from "set skip on
> enc0" to "pass in ... tag VPN" will be better in my case.
> 
> If someone with IPsec experiences will propose changes to FAQ17, then I
> also noted:
> 
> In "road warrior" part, there is "We'll assume the public IP for the
> client is 203.0.113.2.", but the example uses "any".

I think any is the better choice here. This would allow other clients
to connect to the same server (if they have a valid key) which is probably
what most people want.

> 
> I think, that word "daemon" is better then "server" here: 
> 
> The ikectl(8) utility is used to control the server,

Agree

> 
> I want to extend my IKEv2 Site-to-site VPN with road warrior
> configuration. If the road warrior part will include few lines about,
> how to extend responder to handle both site-to-site and road warrior, it
> will be very helpful.

Are you thinking of an example with multiple "ikev2 ..." blocks or a comment
mentioning that you can have multiple of those in the same config file?
Because that is technically all you need.

> 
> Thank you OpenBSD for IPsec and thank you for your support to let me
> configure it.
> 
> BR,
> Jiří
> 
> -- 
> Jiri Navratil, https://nocloud.cz
> 

Re: IPsec over PPPoE

[Jiri Navratil]

Hello,

Thank you for quick and helpful replies.

Adding line

set skip on enc0  

to pf.conf enabled traffic between my sites.

I see in https://www.openbsd.org/faq/faq17.html

"Traffic between them should appear after decapsulation on the enc0
interface, and can be filtered as such." and next line works with VPN
tag, but there are no lines "pass in ... tag VPN" in pf.conf before this
part. Shall that be added to FAQ? I expect, that switch from "set skip on
enc0" to "pass in ... tag VPN" will be better in my case.

If someone with IPsec experiences will propose changes to FAQ17, then I
also noted:

In "road warrior" part, there is "We'll assume the public IP for the
client is 203.0.113.2.", but the example uses "any".

I think, that word "daemon" is better then "server" here: 

The ikectl(8) utility is used to control the server,

I want to extend my IKEv2 Site-to-site VPN with road warrior
configuration. If the road warrior part will include few lines about,
how to extend responder to handle both site-to-site and road warrior, it
will be very helpful.

Thank you OpenBSD for IPsec and thank you for your support to let me
configure it.

BR,
Jiří

-- 
Jiri Navratil, https://nocloud.cz

Re: ipsec hardware recommendation

[David Gwynne]

> On 11 Aug 2023, at 21:08, Marko Cupać  wrote:
> 
> Hi,
> 
> I have star topology network where dozens of spokes communicate with
> other spokes through central hub over GRE tunnels protected with
> transport-mode ipsec.
> 
> This worked great for years, but lately all the locations got bandwidth
> upgrade (spokes: 10Mbit -> 50Mbit, hub: 2x200Mbit -> 2x500Mbit), and I'm
> starting to experience problems.
> 
> Spokes have APU4D4s, and my tests show they can push up to 30Mbit/s of
> ipsec bidirectionally. Hub has HPE DL360g9 with Xeon CPU E5-2623 v4 @
> 2.60GHz and bge NICs, and it seems it can push no more than 200Mbit/s
> of ipsec bidirectionally (I have no chance to test this thoroughly in a
> lab, but what I see in production indicate this strongly).
> 
> Are there any commands I can run which would indicate ipsec traffic is
> being throttled due to hardware being underspecced? top shows CPU is
> more than 50% idle. netstat shows ~1 Ierrs / Ifail (no Oerrs /
> Ifail) on interfaces that deal with ipsec for two months worth of
> uptime.
> 
> Would replacing Xeon box with AMD EPYC 7262 likely result in an
> improvement? Should I go for some NICs other than bge? What hardware do
> I need at Hub location to accomodate ~400Mbit/s of ipsec
> bidirectionally?

>From recent experience it looks like IPsec, and the crypto processing in 
>particular, still runs under the giant kernel lock. This means you're only 
>going to go as fast as a single core can go, and you'll be particularly 
>sensitive to contention on that lock. The things you can do Right Now(tm) are:

- upgrade to a system with the fastest single core performance you can afford

- upgrade to -current

the pf purge code has been taken out from under the big kernel lock. if you 
have a lot of pf states, this will give more time to crypto.

- pick faster crypto algorithms

you might already be using the fastest, so maybe this wont help.

- terminate ipsec on multiple hosts

two kernels will be faster than one. however, this adds complexity to the 
network, so not an obvious benefit.

- try wireguard?

if it's a single tunnel IP tunnel (ie, one gre(4), and not egre(4)) between the 
hubs and spokes then wg might be simpler and faster. simpler because wg is less 
layers than gre over ipsec, and faster cos it should be able to do crypto in 
parallel.


in the future i'm sure the ipsec stack will improve, but it's hard work that 
takes time.

dlg

> 
> Thank you in advance,
> 
> 
> -- 
> Before enlightenment - chop wood, draw water.
> After  enlightenment - chop wood, draw water.
> 
> Marko Cupać
> https://www.mimar.rs/
> 

Re: ipsec hardware recommendation

[Stuart Henderson]

On 2023-08-11, Marko Cupać  wrote:
> Hi,
>
> I have star topology network where dozens of spokes communicate with
> other spokes through central hub over GRE tunnels protected with
> transport-mode ipsec.
>
> This worked great for years, but lately all the locations got bandwidth
> upgrade (spokes: 10Mbit -> 50Mbit, hub: 2x200Mbit -> 2x500Mbit), and I'm
> starting to experience problems.
>
> Spokes have APU4D4s, and my tests show they can push up to 30Mbit/s of
> ipsec bidirectionally. Hub has HPE DL360g9 with Xeon CPU E5-2623 v4 @
> 2.60GHz and bge NICs, and it seems it can push no more than 200Mbit/s
> of ipsec bidirectionally (I have no chance to test this thoroughly in a
> lab, but what I see in production indicate this strongly).

If possible, I suggest putting a fast client machine (laptop or
server) on local network near the server and doing some tests that way.

If you post your IPsec configuration, perhaps someone can suggest
whether the choice of ciphers etc could be improved. It can make quite a
difference.

> Are there any commands I can run which would indicate ipsec traffic is
> being throttled due to hardware being underspecced? top shows CPU is
> more than 50% idle. netstat shows ~1 Ierrs / Ifail (no Oerrs /
> Ifail) on interfaces that deal with ipsec for two months worth of
> uptime.
>
> Would replacing Xeon box with AMD EPYC 7262 likely result in an
> improvement? Should I go for some NICs other than bge? What hardware do
> I need at Hub location to accomodate ~400Mbit/s of ipsec
> bidirectionally?

I doubt the NIC choice will be hugely important, in terms of overall
network traffic pretty much anything should be able to cope with the
available bandwidth.

EPYC is certainly a bunch faster than the 2016 Xeon, but the 'Jaguar'
AMDs in the APUs are going to be the slowest point overall.

You also don't mention which OpenBSD versions you're using. That could
make quite a difference.

-- 
Please keep replies on the mailing list.

Re: ipsec hardware recommendation

[Matthew Ernisse]

On Fri, Aug 11, 2023 at 01:08:07PM +0200, Marko Cupać said:

Are there any commands I can run which would indicate ipsec traffic is
being throttled due to hardware being underspecced? top shows CPU is
more than 50% idle. netstat shows ~1 Ierrs / Ifail (no Oerrs /
Ifail) on interfaces that deal with ipsec for two months worth of
uptime.


I believe the crypto work will show up as system% in systat(1) and 
top(1).  I'm not sure if it is still the case but at one point it was 
single-threaded.



Would replacing Xeon box with AMD EPYC 7262 likely result in an
improvement? Should I go for some NICs other than bge? What hardware do
I need at Hub location to accomodate ~400Mbit/s of ipsec
bidirectionally?


I would start by testing your throughput without ipsec to a system on 
your local ethernet segment.  Maybe using something like iperf.  If you 
can exceed your ipsec throughput you know it probably isn't the NIC

driver.  Try to set it up so you are testing forwarding performance.
I have a Xeon D-1521 with ix(4) NICs and I can forward enough 
(unencrypted traffic) to saturate the 1Gbe ports on the switch.


--
Please direct replies to the list.

ipsec hardware recommendation

[Marko Cupać]

Hi,

I have star topology network where dozens of spokes communicate with
other spokes through central hub over GRE tunnels protected with
transport-mode ipsec.

This worked great for years, but lately all the locations got bandwidth
upgrade (spokes: 10Mbit -> 50Mbit, hub: 2x200Mbit -> 2x500Mbit), and I'm
starting to experience problems.

Spokes have APU4D4s, and my tests show they can push up to 30Mbit/s of
ipsec bidirectionally. Hub has HPE DL360g9 with Xeon CPU E5-2623 v4 @
2.60GHz and bge NICs, and it seems it can push no more than 200Mbit/s
of ipsec bidirectionally (I have no chance to test this thoroughly in a
lab, but what I see in production indicate this strongly).

Are there any commands I can run which would indicate ipsec traffic is
being throttled due to hardware being underspecced? top shows CPU is
more than 50% idle. netstat shows ~1 Ierrs / Ifail (no Oerrs /
Ifail) on interfaces that deal with ipsec for two months worth of
uptime.

Would replacing Xeon box with AMD EPYC 7262 likely result in an
improvement? Should I go for some NICs other than bge? What hardware do
I need at Hub location to accomodate ~400Mbit/s of ipsec
bidirectionally?

Thank you in advance,


-- 
Before enlightenment - chop wood, draw water.
After  enlightenment - chop wood, draw water.

Marko Cupać
https://www.mimar.rs/

Re: ip6-only ipsec tunnel over ip4

[Stuart Henderson]

On 2023-07-26, Lyndon Nerenberg (VE7TFX/VE6BBM)  wrote:
> I need to set up an ipsec tunnel between a couple of ip6 networks,
> but I only have an ip4 path between the two gateways.  I don't want
> any ip4 traffic inside the ipsec tunnel, so I'm a bit puzzled about
> how to set this up.  Once I have the end-points up, can I just point
> the ip6 traffic and routes at enc0?  All the example I can find
> assume you're tunneling ip4 traffic through an ip4 tunnel. (Sorry,
> but after three decades of trying, I still can't make heads nor
> tails of ipsec :-P)

IPsec normally uses flows rather than the route table. Just configure
the tunnel between v6 addresses e.g. "from  to
 peer ".

Re: ip6-only ipsec tunnel over ip4

[deich...@placebonol.com]

I have an L2 tunnel ( eoip ) going across IPsec tunnel, I'm routing ip4 across 
it.

You could try the same with ipv6.

diana
KI5PGJ 

On July 25, 2023 8:07:16 PM MDT, "Lyndon Nerenberg (VE7TFX/VE6BBM)" 
 wrote:
>I need to set up an ipsec tunnel between a couple of ip6 networks,
>but I only have an ip4 path between the two gateways.  I don't want
>any ip4 traffic inside the ipsec tunnel, so I'm a bit puzzled about
>how to set this up.  Once I have the end-points up, can I just point
>the ip6 traffic and routes at enc0?  All the example I can find
>assume you're tunneling ip4 traffic through an ip4 tunnel. (Sorry,
>but after three decades of trying, I still can't make heads nor
>tails of ipsec :-P)
>
>--lyndon
>

ip6-only ipsec tunnel over ip4

[Lyndon Nerenberg (VE7TFX/VE6BBM)]

I need to set up an ipsec tunnel between a couple of ip6 networks,
but I only have an ip4 path between the two gateways.  I don't want
any ip4 traffic inside the ipsec tunnel, so I'm a bit puzzled about
how to set this up.  Once I have the end-points up, can I just point
the ip6 traffic and routes at enc0?  All the example I can find
assume you're tunneling ip4 traffic through an ip4 tunnel. (Sorry,
but after three decades of trying, I still can't make heads nor
tails of ipsec :-P)

--lyndon

Re: IPsec "road warrior" VPN not getting set up properly.

[Tobias Heider]

) provides: NDP-proxying an entire subnet
> instead of just the one host in question. Under these circumstances,
> an attacker can fill the router's NDP cache with a bunch of
> incorrectly-proxied addresses that don't actually point to anything,
> just by trying to hit a bunch of random addresses in the subnet and
> tricking the NDP proxy into responding to all of them.
> 
> The use case I have in mind is to only proxy an address when it is
> allocated to a client that has connected to the VPN, and to stop
> proxying that address when the client disconnects. That's why I thought
> it was something iked ought to be able to do: when the client connects
> to iked, iked knows what address it assigned the client so it can
> automatically add this address to the router table with the RTF_ANNOUNCE
> flag set (which is all that "ndp -s" does when I add it to the cache
> manually). Since iked also knows when the client disconnects, it would
> be an excellent candidate for removing the router table entry as well.
> Thus at any moment, the router table on the IKEv2 responder is set to
> NDP proxy only those addresses which correspond to connected clients
> for which the "ndp-proxy" configuration flag was set in iked.conf. This
> would not increase the risk of NDP cache DoS attacks any more than
> physically plugging an additional host into the link-local network
> would, because neighbor advertisements are only sent for neighbors that
> actually exist (albeit across a tunnel).
> 
> 
> === Part II. My new approach ===
> 
> Since my proposal to have iked enable NDP proxying itself failed to
> gain traction I looked into other options (that don't involve
> requesting a larger subnet from my ISP and VPS providers). The
> fundamental technical obstacle I have to overcome is allowing a
> separate process to detect when clients connect and disconnect so that
> it can add or remove the RTF_ANNOUNCE cache entry as appropriate.
> 
> Instead of tailing the iked log in /var/log/daemon, which seems awfully
> brittle, my new idea is to open a routing socket and watch to see when
> iked opens and closes the tunnel. (I'm learning about a lot of this as
> I go so ideas that are obvious to you might not be obvious to me.) This
> is still architecturally simple in the sense that my server process can
> do it all with a single routing socket, which it uses both to detect
> the incoming client and to add the NDP cache entry. So *now* my problem
> narrows to figuring out which routing messages correspond to incoming
> VPN clients that need to be proxied. I could hack it to look for
> certain telltale signs of the changes I want (i.e. an individual
> address from a particular subnet being added with a local destination
> and particular flags set) but that still looks brittle so I have a
> better plan involving routing labels.
> 
> My new idea still involves a change to iked, but I think this one will
> be less controversial. I now propose the addition of the "rtlabel"
> keyword to the iked.conf grammar:
> 
> ikev2 'server_config' passive esp \
> from any to dynamic \
> local 2001:db8:2::1 \
> srcid server.example.org \
> config address 2001:db8:2::/64 \
> tag "ROADW" \
>   rtlabel "vpn_tunnel"
> 
> All this keyword does is cause ikev2 to add an RTA_LABEL with the value
> "vpn_tunnel" to the RTM_ADD and RTM_DELETE updates that it performs
> when setting up and tearing down this particular tunnel. This is no
> more intrusive than the "tag" keyword that appears on the preceding
> line---it just adds internal labels to messages that flow through the
> kernel so that other processes can hook in and do whatever custom
> behaviors they want. I've already verified that RTM_DELETE messages
> can carry routing labels even though the label doesn't get added to the
> table.
> 
> === Part III. Some questions ===
> 
> I have a couple of questions about the routing labels and iked:
> 
> 1. Right now slaacd automatically labels all the routes it creates with
> "slaacd". As a result, I'm tempted to tag *all* of iked's routing
> messages with "iked" by default (even routing messages that aren't
> related to the tunnels) since the user already has to be aware that
> some labels are taken. As an added bonus, a casual "route -v show" will
> make it easier to figure out where all the routes are coming from.
> Would this change have support if I did the legwork?

Yes, setting a fixed iked route label seems like a good idea.
I am not convinced we need a per policy config setting.

> 
> 2. What is the purpose of iked's IMSG_VROUTE_CLONE? Apparently this
> looks up the current route to t

Re: IPsec "road warrior" VPN not getting set up properly.

[Anthony Coulter]

Stuart Henderson wrote:
> Currently iked (and isakmpd) use flows, not routes. These use messages
> on the PF_KEY socket not the route socket. (If I watch route -nv monitor
> while iked starts and brings up tunnels, I don't see any messages).
>
> IIUC the parts you found which currently exist are for the "iface"
> option which tells an iked initiator (a machine, typically but not
> necessarily mobile) to use mode-config to fetch an address, and
> configure it on an interface.
>
> There is a diff for route-based IPsec which does use the route table
> (https://marc.info/?l=openbsd-tech=168844868110327=2), but I think
> if you were using that, you could just look for messages relating to
> sec(4) interfaces anyway and not have to worry about labels?

Oops, you are correct. I was doing my initial testing on the client
(since it's my primary desktop) and didn't look closely at at the
routes that were getting set up.

The good news, though, is that now I can get away without route labels
even before the sec(4) code gets merged. Since the flows that are
visible in the PF_KEY socket are tunneled routes only, I can apply the
following reasoning: for this use case, any flow (not route) for which
one side is a single address and the other side is "::" (for "any") is
one that needs to be proxied. So now my monitoring program needs to
monitor two sockets instead of one, but I don't have to make any
changes to iked to support it.


> If you're on a low end VPS that is not already doing filtering like
> this, you may find that later on they change to implement that, when
> they figure out why the L3 switch they're using as a router is
> running out of steam when handling junk traffic...

As Zack may have figured out earlier, I am in fact using Vultr. Their
IPv6 guide for OpenBSD specifically says they use routeradv,
neighbrsol, and neighbradv ICMPv6 messages to identify the active IP
addresses, and they also specifically permit the addition of additional
IPv6 addresses to the same interface. Source:

https://www.vultr.com/docs/configuring-ipv6-on-your-vps/#IPv6_on_OpenBSD

The consensus in this thread seems to be that Vultr is doing it wrong
and their bad decisions will cause problems both for them (because
their gateway router is susceptible to NDP cache exhaustion from
external scans even if I don't do anything creative with my host
configuration) and for me (because I've spent the last week in a
fifteen-email exchange about figuring out how to make down with a /64
instead of using that time to figure out how to configure dhcpcd and
rad to make use of the /56 that is my IETF birthright). I won't try to
defend Vultr's IPv6 choices here; I can only say that since they're
explicitly telling me to use NDP to request IPv6 addresses then they
won't change that behavior behind my back without telling me.


>> Since my proposal to have iked enable NDP proxying itself failed to
>> gain traction I looked into other options (that don't involve
>> requesting a larger subnet from my ISP and VPS providers).
>
> I think you're thinking of having a subnet routed across as something
> that is going to cause more pain for the upstream provider.
>
> The reality is the opposite.
>
> If the provider doesn't understand this, there are probably a few
> other things they don't understand too, it would be a bit of a red
> flag for me.
>
> Yes there are a huge number of addresses in a /64, but really a /64
> is what providers are expected to assign where they would assign an
> individual address for IPv4.
>
> For a situation where you'd have a couple of addresses with v4,
> with v6 it's really normal to have a /56 or /48.

In Vultr's defense, they did only assign me a single IPv4 address. You
would probably counter, "Yes, but if you want your VPN connection to
support IPv4 when the server has a single address then you have to use
NAT, which you can also do with IPv6. If you want to avoid using NAT,
then for IPv4 you should request another address and for IPv6 you
should request a larger subnet." And to *that* I will admit that Vultr
has an option to buy additional IPv4 addresses but doesn't have an
option to increase the size of your IPv6 subnet. Presumably this is
where you and Zack would both say, "Exactly, that's the problem."

So OK, the final conclusions here seem to be:

- Vultr is bad for not offering a way to allocate and statically route
  a larger subnet. (Note that it would also work for me if they
  statically routed the entire /64 to my VPS instead of filtering it in
  their gateway router, but they don't offer a way to do that either.)

- The NDP proxy trick is too hacky to justify making changes to iked to
  make it easier to implement.

- I can still automate NDP proxying by creating a second process that
  monitors the PF_KEY socket for changes to the "flow 

Re: IPsec "road warrior" VPN not getting set up properly.

[Zack Newman]

I'm sure this is obvious to people, but just in case it is not:


I pay $25/month for my VPS, and I think I could bring that down to $10
or $15 if I wanted. My VPS routes me a /48 IPv6 network...


I clearly meant "My VPS _provider_ routes me...".

Re: IPsec "road warrior" VPN not getting set up properly.

[Zack Newman]

Before I essentially echo back what Stuart said, let me clarify
something. I don't really recommend NAT over NDP proxying more than the
other way around. I was merely stating that a hack is a hack is a hack.
If you are forced to use a hack, then insisting on one over the other
is bizarre unless one legitimately solves something the other doesn't.
Personally, I would use NAT since it is more familiar to me.

What I am still befuddled by is your insistence on sticking with your
incompetent VPS provider. I sympathize with your plight vis-a-vis a
residential ISP. It is not uncommon to have monopolies/duopolies
dominate a particular residential ISP market; so if they are
incompetent, then you're SOL. VPS providers, however, are a dime a
dozen.

Why would you want to stick with a VPS provider that (un)knowingly
does not adhere to RFCs? If they don't follow one RFC, then I wouldn't
be surprised if they don't follow others. To me I don't view it much
differently than insisting on renting a laptop with malfunctioning
hardware or BIOS/UEFI and waving $200 at kernel developers to hack
around the bugs. They might; but even if they do, is it not better to
solve the problem at the root (i.e., rent a functioning laptop)?

I pay $25/month for my VPS, and I think I could bring that down to $10
or $15 if I wanted. My VPS routes me a /48 IPv6 network via a link-local
address and a /29 IPv4 network via a private address. I have two
WireGuard interfaces: one for "normal" peers where they connect to this
VPN server via WireGuard software (e.g., the Android app) and another
for my server/router at home which subsequently gets routed a /56 IPv6
block and the whole /29 IPv4 block. Bam. Finito. No BGP, no problem. I
get to stay within the cozy confines of good ole routing. No hacks or
other shenanigans. It just works.

Just in case people misinterpret above: I am _not_ recommending this
over BGP if you have the money/resources for BGP. Obviously you want
the shortest routes possible and avoid any overhead associated with a
VPN.

Re: IPsec "road warrior" VPN not getting set up properly.

[Stuart Henderson]

On 2023-07-10, Anthony Coulter  wrote:
> 2. I abandon my quest to get NDP proxying added to iked and instead ask
>if we can add a "rtlabel" keyword to iked.conf to make it easier for
>me to write a separate process that monitors the routing table to
>detect when the tunnel gets set up.

Currently iked (and isakmpd) use flows, not routes. These use messages
on the PF_KEY socket not the route socket. (If I watch route -nv monitor
while iked starts and brings up tunnels, I don't see any messages).

IIUC the parts you found which currently exist are for the "iface"
option which tells an iked initiator (a machine, typically but not
necessarily mobile) to use mode-config to fetch an address, and
configure it on an interface.

There is a diff for route-based IPsec which does use the route table
(https://marc.info/?l=openbsd-tech=168844868110327=2), but I think
if you were using that, you could just look for messages relating to
sec(4) interfaces anyway and not have to worry about labels?


>> NDP proxying is also vulnerable to NDP cache DoS.

It's more than just NDP proxying that results in this. Put a big chunk
of address space on an interface that requires IP-to-physical address
resolution and you end up having to do that resolution for a bunch of
junk traffic coming from the internet. Say someone does some kind of
sweep looking for hosts and sending unsolicited traffic to a bunch of
addresses within a prefix, each address attempt will result in having
to try to resolve that address. Whether it's successful or not, some
work must be done for that (e.g. slow-path lookups on the gateway
router, caching, etc).

FWIW if I was a VPS/colo provider wanting to keep router resources low,
I would probably assign a single address from a /64 to the customer
machine/VM, and filter traffic from the internet so that packets to
other addresses in the /64 are blocked - so junk traffic would not
trigger junk NDP attempts..

(alternatively it's possible to use link-locals for that one "route
via" address, though I would prefer to have that on a routable address
for ease of debugging - traceroutes etc).

Then for any additional addresses I'd route either multiple /64's,
or a /56 or /48, via that one address - that way NDP is only ever done
for the one address and it's likely to stay in cache.

If you're on a low end VPS that is not already doing filtering like
this, you may find that later on they change to implement that, when
they figure out why the L3 switch they're using as a router is
running out of steam when handling junk traffic...


> Since my proposal to have iked enable NDP proxying itself failed to
> gain traction I looked into other options (that don't involve
> requesting a larger subnet from my ISP and VPS providers).

I think you're thinking of having a subnet routed across as something
that is going to cause more pain for the upstream provider.

The reality is the opposite.

If the provider doesn't understand this, there are probably a few
other things they don't understand too, it would be a bit of a red
flag for me.

Yes there are a huge number of addresses in a /64, but really a /64
is what providers are expected to assign where they would assign an
individual address for IPv4.

For a situation where you'd have a couple of addresses with v4,
with v6 it's really normal to have a /56 or /48.

Re: IPsec "road warrior" VPN not getting set up properly.

[Anthony Coulter]

ought to be able to do: when the client connects
to iked, iked knows what address it assigned the client so it can
automatically add this address to the router table with the RTF_ANNOUNCE
flag set (which is all that "ndp -s" does when I add it to the cache
manually). Since iked also knows when the client disconnects, it would
be an excellent candidate for removing the router table entry as well.
Thus at any moment, the router table on the IKEv2 responder is set to
NDP proxy only those addresses which correspond to connected clients
for which the "ndp-proxy" configuration flag was set in iked.conf. This
would not increase the risk of NDP cache DoS attacks any more than
physically plugging an additional host into the link-local network
would, because neighbor advertisements are only sent for neighbors that
actually exist (albeit across a tunnel).


=== Part II. My new approach ===

Since my proposal to have iked enable NDP proxying itself failed to
gain traction I looked into other options (that don't involve
requesting a larger subnet from my ISP and VPS providers). The
fundamental technical obstacle I have to overcome is allowing a
separate process to detect when clients connect and disconnect so that
it can add or remove the RTF_ANNOUNCE cache entry as appropriate.

Instead of tailing the iked log in /var/log/daemon, which seems awfully
brittle, my new idea is to open a routing socket and watch to see when
iked opens and closes the tunnel. (I'm learning about a lot of this as
I go so ideas that are obvious to you might not be obvious to me.) This
is still architecturally simple in the sense that my server process can
do it all with a single routing socket, which it uses both to detect
the incoming client and to add the NDP cache entry. So *now* my problem
narrows to figuring out which routing messages correspond to incoming
VPN clients that need to be proxied. I could hack it to look for
certain telltale signs of the changes I want (i.e. an individual
address from a particular subnet being added with a local destination
and particular flags set) but that still looks brittle so I have a
better plan involving routing labels.

My new idea still involves a change to iked, but I think this one will
be less controversial. I now propose the addition of the "rtlabel"
keyword to the iked.conf grammar:

ikev2 'server_config' passive esp \
from any to dynamic \
local 2001:db8:2::1 \
srcid server.example.org \
config address 2001:db8:2::/64 \
tag "ROADW" \
rtlabel "vpn_tunnel"

All this keyword does is cause ikev2 to add an RTA_LABEL with the value
"vpn_tunnel" to the RTM_ADD and RTM_DELETE updates that it performs
when setting up and tearing down this particular tunnel. This is no
more intrusive than the "tag" keyword that appears on the preceding
line---it just adds internal labels to messages that flow through the
kernel so that other processes can hook in and do whatever custom
behaviors they want. I've already verified that RTM_DELETE messages
can carry routing labels even though the label doesn't get added to the
table.

=== Part III. Some questions ===

I have a couple of questions about the routing labels and iked:

1. Right now slaacd automatically labels all the routes it creates with
"slaacd". As a result, I'm tempted to tag *all* of iked's routing
messages with "iked" by default (even routing messages that aren't
related to the tunnels) since the user already has to be aware that
some labels are taken. As an added bonus, a casual "route -v show" will
make it easier to figure out where all the routes are coming from.
Would this change have support if I did the legwork?

2. What is the purpose of iked's IMSG_VROUTE_CLONE? Apparently this
looks up the current route to the peer, replaces the destination field
(which I assume must be either the peer's address or a subnet
containing the peer's address) with the peer's address, and re-adds it
to the routing table. What benefit does this provide? I think I
understand "cloning" in the context of ARP and NDP caches with their
RTF_CLONING and RTF_CLONED flags, and I recognize the pattern here of
"take an existing route and re-add a more specific version of it"
(which isn't what "clone" means anywhere else in the world, but
whatever). But I don't understand why we would want to do it in this
context. Maybe it's a trick to make sure that encapsulated IPsec
packets don't get routed into the same tunnel a second time, in the
instance that the peer's address is contained in the subnet that is
supposed to be tunneled. But I don't see how that trick would work
for transport-mode IPsec, where the destination address is the same
before and after encapsulation---the cloned route seems to get the
same priority (IKED_VROUTE_PRIO) as the tunnel. So what does this
cloned route accomplish?

3. It's a commo

Re: IPsec "road warrior" VPN not getting set up properly.

[Andy Bradford]

Thus said Anthony Coulter on Thu, 06 Jul 2023 21:52:54 -0400:

> I would also suggest comparing the  "hackiness" of NDP proxying to the
> hackiness of NAT, which is how we solve this same problem in IPv4.

I realize  I'm coming in late  to this discussion, and  may not actually
have anything of value to add, but...

I'm not sure how NDP proxying and NAT  are related at all. I seems to me
that NDP proxying is more akin to proxy ARP than NAT:

http://man.openbsd.org/arp#s

Andy

Re: IPsec "road warrior" VPN not getting set up properly.

[Anthony Coulter]

> veering slightly from the topic (typical setup for a server host would
> not be to use DHCPv6 but just statically route another block - usually a
> /56 or /48), but...

I don't doubt this is typical for serious network operators. But I
would counter that for every user who is in a position to request a
statically-routed /48 there are dozens if not hundreds of us who have
one /64 at home and another /64 on each of our VPSes.


> (to request a prefix be routed to you, you need a DHCPv6-PD client,
> not a server)

Oops, correct. I was thinking "DHCPv6 daemon" when I typed it. The
point is that configuring all those subnets and static routes is an
awful lot of extra work when all I want to do is set up a VPN proxy.


> why when people are looking for dhcpv6 software do they always find that
> unmaintained-for-years run-the-whole-lot-as-root wide-dhcp6 thing?
>
> use dhcpcd if you want a client that can do DHCPv6. the most recent release
> was a couple of months ago, it's sensibly written, uses pledge where possible,
> and has decent privilege separation for the parts which can't pledge).

Noted. I'm betting the reason people keep finding wide-dhcpv6 is that
it is the only result for "pkg_info -Q dhcpv6". I used to search
package descriptions at https://openports.se but that site is now
defunct. It would be great if pkg_info could search descriptions as
well as names, but so far none of the ideas that I think would be great
hold much appeal to anyone else.


> picking one other bit..
>
>> I would also suggest comparing the "hackiness" of NDP proxying to the
>> hackiness of NAT, which is how we solve this same problem in IPv4.
>
> it might be how some people solve it for v4. others solve it in a non-hacky
> way which is exactly the same as the non-hacky way for v6; put the vpn clients
> on a different subnet that's routed to the vpn gateway.

I would note here that "some people" includes the OpenBSD FAQ. See:
https://www.openbsd.org/faq/faq17.html#clientikev2

In any event, most of us can't readily get extra publically-routable
IPv4 subnets. According to Google a /24 costs about $9000 right now. I
could make do with a /28 since I only really want to connect three
servers at a time, but how do I get the /28 routed to my server? Either
I track down an ISP that provides these directly or else I have to set
up BGP. All this to connect my laptop to a VPN proxy server! Most
people use NAT for IPv4 because that's all most people can afford.


Maybe I'm missing something here. Why is NDP proxying considered so
distasteful? Can it cause technical problems down the road? I can see
how cycling the client through temporary addresses gets complicated and
has a lot of moving parts, but NDP proxying involves making a change to
the routing table (which iked already has to do whenever VPN clients
connect and disconnect) and its only effect (as I understand it) is
announcing to nodes on the local link that the VPN tunnel host is a
suitable destination for the address in question.

Consider the laptop-connecting-to-home-VPN-from-coffee-shop use case.
Suppose my home network is a /56 and I allocate a /64 for address
assignments to VPN clients like that laptop. Now: how do I make sure
that traffic on my local network which is destined for the laptop
actually gets sent to the laptop? Am I supposed to configure static
routes on all of the nodes on my home network that the laptop might
want to communicate with? That doesn't sound right. I'm guessing the
right way to advertise an entire subnet is with router advertisement
messages, so I would want to configure rad to support this. But that's
still a whole other thing to configure. So instead of using neighbor
advertisements to direct two or three addresses at a time to the VPN
proxy host, I'm using router advertisements to direct a whole subnet to
it---which seems reasonable if I could justify the hassle of getting
that whole subnet configured and advertised properly, but again, that's
a *lot* of trouble.

I recognize that there are other ways to get my individual use case to
work without requesting changes to iked. If I don't expect to have more
than 16 simultaneous connections, I could constrain iked to allocate
addresses from a /124 subnet and add those 16 addresses to the NDP
proxy list at server startup to get the equivalent of static routing
while still not having to reconfigure the ISP's (or hypervisor's)
router. But I would consider *that* to be hackier, because now I really
am using NDP to advertise a subnet instead of just advertising the
addresses that are actually in use. The other option I have on the
table is a shell script that runs as a daemon with root privileges,
monitors /var/log/daemon for IKEv2 connection and disconnection
events, and runs ndp -s and ndp -d appropriately. But if you thought
wide-dhcpv6 was bad for not being privilege separated, you'd hate my
shell script. And I would too! That's the point. This is something iked
could do automatically.

I'm 

Re: IPsec "road warrior" VPN not getting set up properly.

[Zack Newman]

Yeah, I don't have the interest to get into it about this; but I find
it (informally) inconsistent to take an ideological stance against NAT
and not have a similar stance against NDP proxying. Networking is a lot
cleaner when it can be reasoned about with a rudimentary grasp of graph
theory where a network can roughly be seen as a complete graph and a
node belonging to multiple complete graphs represents a destination in
a routing table.

It is not challenging at all to use route(8) to subnet from a routed
/48 or /56. How many subnets you want is up to you. If you want to
only carve out a single /64 for all your hosts, then do so. You can
then avoid DHCPv6 altogether and use rad(8) to send routing
advertisements allowing clients to use SLAAC which is the much more
popular way for clients to automatically configure IPv6 addresses.

Also not sure where you heard that ICMP does not work with NAT. Surely
you don't believe that. Go ahead and use ping(8) on any device that
relies on NAT to talk to the outside world and witness how it
"magically" works. ICMP uses the Query ID in lieu of a port number.

Will NDP proxying work? Depending on what you want, sure just like NAT
will likely work. Relying on a simple routing table is far more ideal.
NDP proxying is also vulnerable to NDP cache DoS. You can use your
favorite search engine to learn why NDP proxying is not as good as
simple routes.

If you want to use NAT or NDP proxying, then be my guest. It is one
thing to not be willing to leave your ISP because you likely don't have
many to choose from, but that is not the case for VPS providers.

Challenge: reach out to the maintainers of popular NDP proxying daemons
and inquire if they think NDP proxying is "clean" when compared to a
simple routing table. In NDP proxying a host is responsible for
responding to Neighbor Solicitation messages for IPs that don't belong
to it. Hm, sounds a lot like NAT where a host uses its IP to masquerade
the IPs of other hosts as opposed to traffic being handled by the
actual host it was intended for.

Re: IPsec "road warrior" VPN not getting set up properly.

[Stuart Henderson]

veering slightly from the topic (typical setup for a server host would
not be to use DHCPv6 but just statically route another block - usually a
/56 or /48), but...

On 2023-07-07, Anthony Coulter  wrote:
> The trouble with subnets is that they have to be configured. I would
> have to install a DHCPv6 server to request that subnet. OpenBSD doesn't
> have one in base so I have to install the wide-dhcp6 package.

(to request a prefix be routed to you, you need a DHCPv6-PD client,
not a server)

why when people are looking for dhcpv6 software do they always find that
unmaintained-for-years run-the-whole-lot-as-root wide-dhcp6 thing?

use dhcpcd if you want a client that can do DHCPv6. the most recent release
was a couple of months ago, it's sensibly written, uses pledge where possible,
and has decent privilege separation for the parts which can't pledge).


picking one other bit..

> I would also suggest comparing the "hackiness" of NDP proxying to the
> hackiness of NAT, which is how we solve this same problem in IPv4.

it might be how some people solve it for v4. others solve it in a non-hacky
way which is exactly the same as the non-hacky way for v6; put the vpn clients
on a different subnet that's routed to the vpn gateway.

Re: IPsec "road warrior" VPN not getting set up properly.

[Anthony Coulter]

Summary of this email: I repeat my argument that automatic NDP
proxying is the right way to handle the "road warrior" use case for
IPv6. The reasons I'm pushing this so hard are that (1) including this
functionality in iked would be much more robust than any hacky script I
could write that tries to monitor routing state changes, and (2) both
of the responses to my routing question claim that the correct way to
connect a laptop to my VPN is to negotiate with my ISP to get a larger
subnet which just sounds bonkers when "ndp -s" solves the technical
problem so perfectly. Then I repeat my offer to fund this solution.

> While I suppose the /64 your VPS provider gives you is "enormous"
> compared to IPv4, I don't find such a comparison relevant since IPv6
> and IPv4 are entirely different protocols. In fact I actually think it
> is small. Why? RFC 6177 (https://datatracker.ietf.org/doc/html/rfc6177)
> recommends that /48 or at least /56 subnets be given to end sites, so
> your _small_ /64 violates that recommendation. Hell, even my lowly
> residential ISP, Xfinity/Comcast, gives me a /60. Unfortunately a great
> many ISPs and VPS providers violate this. Not sure if it is due to
> incompetence where they incorrectly think such allocations are
> "wasteful" or what. IPv6 not only restores end-to-end communication the
> way IPv4 initially started, but it is designed so that sites have many
> _subnets_. This brings me to the next point.

The trouble with subnets is that they have to be configured. I would
have to install a DHCPv6 server to request that subnet. OpenBSD doesn't
have one in base so I have to install the wide-dhcp6 package. Then I
have to configure it. Can a host on my home network request a /60 from
my ISP's router? Or do I have to replace the ISP's router with my own
device in order to make that request? If I replace the ISP's router, I
have to provide all the other router functionality, like the firewall
and IPv4-style DHCP. Then there is the host on which iked is running:
it needs to advertise ownership of this subnet so I have to configure
rad properly. This is insane! I'm not trying to sell VPN services to
customers; I just want a simple proxy for my desktop, a couple of
laptops, and maybe a friend or two. It's the simple sort of low-volume,
hobbyist thing that I *could* accomplish with a layer 2 VPN that
tunnels entire Ethernet frames to the remote server---but that would
also be a huge pain to configure because now I have to set up and
configure a bunch of extra tunnel interfaces.

The "road warrior" VPN (I hate that name) described in the OpenBSD FAQ
is actually really simple to set up. You just have to copy the
/etc/iked/local.pub keys into the appropriate places, writing a pair
of very short /etc/iked.conf files, and then rcctl enable iked and
away we go! Almost. The one thing iked doesn't do is "last-step
routing" between the gateway router and the IKEv2 responder. So for
IPv4 we also have to enable nat in /etc/pf.conf, and for IPv6 we have
to *either* switch from Vultr to a fancier VPS and do a crapton of
configuration to set up the subnets, *or* use NDP proxying to advertise
that final hop.

NDP proxying is simple. It's also elegant. What does it take for the
proxied IP address to be globally routable? Three things have to
happen: the Internet has to deliver the packet to the IKEv2 responder's
gateway router. The gateway router has to send the packet to the IKEv2
responder. And the IKEv2 responder has to send the packet into the
IPsec tunnel. The first step (internet to gateway router) is the
fundamental job of the Internet and requires no extra work on our part.
The last step (pushing the packet into the tunnel) is handled by iked.
All that's left is the middle step: getting the packet from a router to
a host on the same link. How do we do that? This is the problem NDP was
designed to solve. I don't think it's hacky or unclean at all. The
responder wants to tell the hosts on the local link that it is the
link-layer endpoint for a single destination address that is on the
local link's subnet. That's all NDP does: it instructs hosts on the
lcoal link to update their local routing tables. It's perfect for this
purpose. And it's even *more* perfect because OpenBSD implements this
functionality by updating its own internal routing tables, which iked
already does. So it would be even *more* simple if iked set up the
NDP proxying automatically instead of forcing me to run a script every
time I bounce the connection.

I would also suggest comparing the "hackiness" of NDP proxying to the
hackiness of NAT, which is how we solve this same problem in IPv4. If
we put aside all the political issues around NAT, it's still a really
janky process. It only works for TCP and UDP, so you can't send ICMP
messages or use any other fancy experimental protocols. You can't
listen on ports unless you create 

Re: IPsec "road warrior" VPN not getting set up properly.

[Anthony Coulter]

First, thank you! The "ndp -s" trick does exactly what I need. (I did
not need to consider ndp-reflector.) The rest of this email could be
summarized as "That works so perfectly I would pay for someone to make
it automatic; meanwhile the other things I asked about were in fact
bad ideas and I will stop wishing for them."

> What you're actually trying to do here is respond to NDP requests as a
> proxy. It's not a clean way to do it (the clean way is "upstream routes
> you a subnet"), but some people are trying to do this on cheap services
> that are only intended for a single host connection and that's about the
> only way.
>
> In this case you might be able to do it with ndp -s, as the "proxied"
> address is down a tunnel and not going to respond to NDP itself anyway,
> so it doesn't hit the main problem for arp/ndp proxy in OpenBSD which
> is that it answers on all interfaces, you can't tie it to one in
> particular.
>
> (For some other use cases, where you're on a /64 behind an ISP router,
> and have other hosts on a different ethernet interface, it looks like
> https://github.com/toru-mano/nd-reflector might do the trick).

What makes it unclean? (The "ndp -s" trick is simple and works
perfectly; I'm trying to understand why it's architecturally bad.) The
use case is:
- I have a whole /64 subnet.
- I cannot easily modify the gateway router on this subnet.
- I want to connect a single client to this network over a tunnel that
  is administered by a host (not the gateway router!) on that network.
- I want the client to be assigned a single [dynamic] IPv6 address on
  the /64 subnet. This address must be globally routable!
- IPv4 is explicitly out-of-scope because addresses are hard to come
  by; I'm fine just using NAT for IPv4.

These items describe both my VPS (where the hypervisor has allocated me
an entire /64 subnet but only delivers packets to the vio0 interface
when their destination addresses can be found among my NDP neighbor
advertisements) and my physical network at home (where again the whole
/64 is mine, but replacing the ISP-provided router is a nontrivial
affair).

Since iked already supports dynamic address assignments, that part is
noncontroversial. What is needed for this address to be globally
routable? The Internet at large already knows how to look at the top
64 bits of an address to figure out what gateway router to deliver the
packet to, and if the packet somehow reaches the IKEv2 responder, iked
has already set up the routes necessary to push the packet down the
tunnel to its final destination. All we're missing is the step between
the gateway router and the IKEv2 responder. They're on the same link.
What is the proper way to for iked to announce the dynamic IPv6 address
to other hosts on the local link? NDP. And it turns out that OpenBSD
already has a utility in base that does NDP proxying. It seems like
this should be the recommended solution for IPv6. Taking responsibility
for routing the whole /64 subnet just to support a couple of IPsec
tunnels seems wrong. I'm not trying to bridge networks; I just want to
use an IPsec tunnel to proxy traffic from my laptop. The "ndp -s"
trick seems simple and elegant, and it does exactly what we need it to
and no more.

I would go a step further and ask if *this* functionality could be done
automatically by iked. I looked at the code in the ndp utility and it
appears to work just by writing messages to a kernel routing socket.
Seeing as iked *already* writes messages to a routing socket whenever a
flow starts or stops, it seems like it would be easy (and
architecturally sound) to add an option to tell iked to do the
equivalent of calling "ndp -s $client_addr $server_mac proxy" when a
client connects and "ndp -d $client_addr" when it disconnects---just
treat it like any other routing step. It could be enabled with a
configuration option like "ndp-proxy vio0". (The point of the interface
name is to free me from the burden of entering the MAC address by
hand.) The option would only be valid for connections with a "config
address" option where the address is an IPv6 address, because of course
that is the address which would be added to the NDP proxy list.

I would gladly pay $200 to a developer who can add this functionality
to iked. That's one reason I care so much about whether this is really
a "clean" solution. If it isn't clean then maybe adding it isn't such a
good idea. But to *me* it looks like iked is already responsible for
updating routing information to match the currently-active flows, and
the code in the ndp utility suggests that from the kernel's point of
view, neighbor advertisements are a routing issue controlled through a
routing socket. I can automate this process with a script, but I really
do think this is the right way to do "road warrior" IPv6 connections.

Re: IPsec "road warrior" VPN not getting set up properly.

[Zack Newman]

While I suppose the /64 your VPS provider gives you is "enormous"
compared to IPv4, I don't find such a comparison relevant since IPv6
and IPv4 are entirely different protocols. In fact I actually think it
is small. Why? RFC 6177 (https://datatracker.ietf.org/doc/html/rfc6177)
recommends that /48 or at least /56 subnets be given to end sites, so
your _small_ /64 violates that recommendation. Hell, even my lowly
residential ISP, Xfinity/Comcast, gives me a /60. Unfortunately a great
many ISPs and VPS providers violate this. Not sure if it is due to
incompetence where they incorrectly think such allocations are
"wasteful" or what. IPv6 not only restores end-to-end communication the
way IPv4 initially started, but it is designed so that sites have many
_subnets_. This brings me to the next point.

You would like to rely on SLAAC for your VPN peers, but SLAAC will
likely not work on anything smaller than /64. Why? Because the first
64 bits of an IPv6 address is designated as the network identifier.
You already carved out some IPs from the /64 though which means you
have less than /64 to use for SLAAC inside the tunnel.

I used to use Vultr; but when they were unwilling to provide something
bigger than a /64 in addition to actually routing the entire block, I
left them. If you insist on using IPv6 without relying on NAT or NDP
proxying, then I recommend you find another provider. What you are
trying to do is trivial when IPv6 is done properly. I have a similar
setup myself except I use WireGuard, but I'm confident IKEv2/IPSec
would be easy to set up as well.

Re: IPsec "road warrior" VPN not getting set up properly.

[Stuart Henderson]

On 2023-07-05, Anthony Coulter  wrote:
> OK, I've sorted out my network issues server but it turns out that I
> was misinterpreting the tcpdump output on my VPS. When an external
> computer tries to ping my client's virtual IP address, the VPS's
> gateway router is *not* forwarding the pings to my server where they
> can be shoved into the IPsec tunnel to the client. So it looks like
> the iked responder needs to send IPv6 neighbor advertisements for the
> allocated address after all. Is there a way to do this?

Your simplest options are either to ask upstream to route traffic to you
for the whole subnet (this will probably involve adding a new subnet and
routing it via the existing address), or NAT traffic so it comes from
the address the upstream router is expecting.

> I have tried two things so far:
>
> 1. Manually assign the [randomly allocated] IP address to the
>responder's physical network interface so it knows it has ownership
>of that address:
>
> # ifconfig vio0 inet6 alias 2001:db8:2::1234:5678
>
> Of course, this breaks the tunnel. When the responder sees packets with
> destination 2001:db8:2::1234:5678 it consumes them locally instead of
> routing them down the IPsec tunnel.

What you're actually trying to do here is respond to NDP requests as a
proxy. It's not a clean way to do it (the clean way is "upstream routes
you a subnet"), but some people are trying to do this on cheap services
that are only intended for a single host connection and that's about the
only way.

In this case you might be able to do it with ndp -s, as the "proxied"
address is down a tunnel and not going to respond to NDP itself anyway,
so it doesn't hit the main problem for arp/ndp proxy in OpenBSD which
is that it answers on all interfaces, you can't tie it to one in
particular.

(For some other use cases, where you're on a /64 behind an ISP router,
and have other hosts on a different ethernet interface, it looks like
https://github.com/toru-mano/nd-reflector might do the trick).

> Here is my vision of a perfect world. It's common for IPv6 nodes to
> request a new random IPv6 address every couple of hours; OpenBSD's
> inet6 autoconf does this by default. There is no shortage of IPv6
> addresses on a LAN, and the process of requesting new ones is pretty
> well standardized .

On a LAN, yes there's a common way to do it. Not so much over a tunnel
though.

> So what would be fantastic is if instead of
> putting "config address 2001:db8:2::/64" in the responder's iked.conf,
> I could put "config inet6 autoconf vio0" instead. Then iked would
> make sure that vio0 is using autoconf addresses and, if so, it would
> request a new address via SLAAC (preferably randomizing all 64 bits
> instead of just the bottom 32 bits like it does right now), assign
> that address to the client, configure all the routing, and then would
> somehow configure vio0 to claim ownership of that address for
> NDP neighbor solicitation purposes, but have the kernel *not* claim
> the address for routing purposes. The client config wouldn't change
> at all; it would still say "request address any".

autoconf privacy addresses require that old addresses stay active so
that long-running connections don't die. You're asking hell of a lot
from a number of different subsystems, some of which do not have a way
to handle this, to make it all work.


-- 
Please keep replies on the mailing list.

Re: IPsec "road warrior" VPN not getting set up properly.

[Anthony Coulter]

OK, I've sorted out my network issues server but it turns out that I
was misinterpreting the tcpdump output on my VPS. When an external
computer tries to ping my client's virtual IP address, the VPS's
gateway router is *not* forwarding the pings to my server where they
can be shoved into the IPsec tunnel to the client. So it looks like
the iked responder needs to send IPv6 neighbor advertisements for the
allocated address after all. Is there a way to do this?

I have tried two things so far:

1. Manually assign the [randomly allocated] IP address to the
   responder's physical network interface so it knows it has ownership
   of that address:

# ifconfig vio0 inet6 alias 2001:db8:2::1234:5678

Of course, this breaks the tunnel. When the responder sees packets with
destination 2001:db8:2::1234:5678 it consumes them locally instead of
routing them down the IPsec tunnel.

2. Advertise ownership of the address with rad:

$ cat /etc/rad.conf
interface vio0 {
prefix 2001:db8:2::1234:5678/128 {
autonomous address-configuration no
}
}

This doesn't work either because a router advertisement is not the same
as a neighbor advertisement. But I tried!

A third option would be to use a layer 2 tunnel so that the client
would be responsible for sending its own neighbor advertisements. But
the cost is high: I need to configure a gif(4) interface on the client
and now I'm pushing big fat Ethernet frames across the tunnel instead
of modestly portly IPv6 packets. Worse, if I want non-OpenBSD clients
to connect to the server, I have to figure out how to set up the client
side of a layer 2 tunnel on whatever OS they're using. So much work
just to get a globally-routable IP address!


Here is my vision of a perfect world. It's common for IPv6 nodes to
request a new random IPv6 address every couple of hours; OpenBSD's
inet6 autoconf does this by default. There is no shortage of IPv6
addresses on a LAN, and the process of requesting new ones is pretty
well standardized . So what would be fantastic is if instead of
putting "config address 2001:db8:2::/64" in the responder's iked.conf,
I could put "config inet6 autoconf vio0" instead. Then iked would
make sure that vio0 is using autoconf addresses and, if so, it would
request a new address via SLAAC (preferably randomizing all 64 bits
instead of just the bottom 32 bits like it does right now), assign
that address to the client, configure all the routing, and then would
somehow configure vio0 to claim ownership of that address for
NDP neighbor solicitation purposes, but have the kernel *not* claim
the address for routing purposes. The client config wouldn't change
at all; it would still say "request address any". But since we're
visualizing a perfect world and not just a great world, there would
also be an optional "temporary" flag which causes the client to
request a new address every couple of hours (just like how inet6
autconf does on regular interfaces)---since one of the intended use
cases for this would be concealing your region of origin, it seems
like privacy features should be easy to request.

But since I'm not [currently] offering to figure out how to implement
this myself I'll make do with what I have. Is there a way to make the
the responder/server send neighbor advertisements for the address it
allocated for the client, so that this address will be usable on the
global Internet?

Thanks,
Anthony

Re: IPsec "road warrior" VPN not getting set up properly.

[Tobias Heider]

On July 5, 2023 4:35:30 AM GMT+03:00, Anthony Coulter 
 wrote:
>Short version:
>
>I'm trying to set up a "road warrior"-style VPN like the one described
>at https://www.openbsd.org/faq/faq17.html but I'm trying to use IPv6 so
>I can have globally-routable addresses (so I'm not using NAT). So far
>I've gotten the initiator and the responder to set up a security
>association (to be precise, according to "ipsecctl -sa" the
>initiator/client sees one SA and zero flows, and the responder/server
>sees two SA's and two flows). When I try to ping the client's virtual
>IP address from the server, tcpdump running on the egress interfaces of
>both the client and the server see appropriately-sized "esp" packets
>going from the server to the client as expected, but tcpdump doesn't
>see any traffic emerging on the "lo1" loopback interface the FAQ told
>me to create, nor does it see any evidence of ping-responses being
>sent. When I try to ping from the client to the server, I get
>"ping6: sendmsg: Permission denied".
>
>
>Longer version:
>
>The client is running -current and the server is running 7.3.
>
>The server is a VPS hosted in a datacenter somewhere, and my VPS
>provider gives me a whole /64 subnet to work with. My VPS provider
>doesn't appear to look at neighbor discovery traffic; it routes all
>traffic in that /64 to my VirtIO interface, regardless of which IP
>address it is, which seems like a handy feature for what I'm trying to
>do.
>
>Here is what I want to do: the client should open an IPsec tunnel to
>the server. The client should request an IPv6 address from the server's
>enormous /64 subnet. Then, applications running on the client should
>have the option to use that "virtual" IPv6 address as though it was
>available on a local interface on the client machine. The client would
>effectively be multihosted: in addition to its regular physical
>Internet connection it would also have this IPsec tunnel that also
>connects to the global Internet. My plan right now is to use rdomains
>to control which applications use the tunnel and which ones use the
>regular egress interface, but I'm not attached to that specific
>approach.
>
>The application should be obvious: I'm trying to set up a region-faking
>VPN, so that I can have applications appear to connect to the Internet
>from different continents.
>
>Both client and server are using the same pf.conf that is installed by
>default. I tried adding "pass on enc0" to the client's pf.conf and
>"pass on enc0 tagged ROADW" to the server's, but it hasn't made a
>difference because (I assume) my existing pf.conf rules weren't
>blocking much to begin with.
>
>Here are my config files, with IP addresses and server names changed.
>But basically you should see the client at 2001:db8:1::1 connecting to
>the server at 2001:db8:2::1 and requesting a random IP address in the
>server's subnet, e.g. 2001:db8:2::485b:4ac7. I have also put the
>appropriate keys in /etc/iked/pubkeys/fqdn/.
>
>=== Client-side configuration ===
>$ cat /etc/hostname.lo1
>rdomain 1
>
>$ cat /etc/iked.conf
>ikev2 'client_config' active esp \
>rdomain 1 \
>from dynamic to any \
>   local 2001:db8:1::1 \
>peer  2001:db8:2::1 \
>srcid client.example.org \
>dstid server.example.org \
>request address any \
>iface lo1
>
>=== Server-side configuration ===
>
>$ cat /etc/sysctl.conf
>net.inet6.ip6.forwarding=1
>
>$ cat /etc/iked.conf
>ikev2 'server_config' passive esp \
>from any to dynamic \
>local 2001:db8:2::1 \
>   peer  2001:db8:1::1 \
>srcid server.example.org \
>config address 2001:db8:2::/64 \
>tag "ROADW"
>
>=== End of configurations ===
>
>Does it work? Sort of. Here's what I see on the server:
>
>=== Checking status on the server ===
>
># ipsecctl -sa
>FLOWS:
>flow esp in from 2001:db8:2::485b:4ac7 to ::/0 peer 2001:db8:1::1 srcid 
>FQDN/server.example.org dstid FQDN/client.example.org type require
>flow esp out from ::/0 to 2001:db8:2::485b:4ac7 peer 2001:db8:1::1 srcid 
>FQDN/server.example.org dstid FQDN/client.example.org type require
>
>SAD:
>esp tunnel from 2001:db8:2::1 to 2001:db8:1::1 spi 0x001a3754 enc aes-128-gcm
>esp tunnel from 2001:db8:1::1 to 2001:cb8:2::1 spi 0x7634a3b6 enc aes-128-gcm
>
>$ route show
>Internet6:
>DestinationGateway Flags   Refs  Use   Mtu  Prio Iface
>defaultfe80::1234%vio0 UGS8  578 - 
>8 vio0 
>::/96  localhost   UGRS   00 32768 8 lo0  
>

IPsec "road warrior" VPN not getting set up properly.

[Anthony Coulter]

Short version:

I'm trying to set up a "road warrior"-style VPN like the one described
at https://www.openbsd.org/faq/faq17.html but I'm trying to use IPv6 so
I can have globally-routable addresses (so I'm not using NAT). So far
I've gotten the initiator and the responder to set up a security
association (to be precise, according to "ipsecctl -sa" the
initiator/client sees one SA and zero flows, and the responder/server
sees two SA's and two flows). When I try to ping the client's virtual
IP address from the server, tcpdump running on the egress interfaces of
both the client and the server see appropriately-sized "esp" packets
going from the server to the client as expected, but tcpdump doesn't
see any traffic emerging on the "lo1" loopback interface the FAQ told
me to create, nor does it see any evidence of ping-responses being
sent. When I try to ping from the client to the server, I get
"ping6: sendmsg: Permission denied".


Longer version:

The client is running -current and the server is running 7.3.

The server is a VPS hosted in a datacenter somewhere, and my VPS
provider gives me a whole /64 subnet to work with. My VPS provider
doesn't appear to look at neighbor discovery traffic; it routes all
traffic in that /64 to my VirtIO interface, regardless of which IP
address it is, which seems like a handy feature for what I'm trying to
do.

Here is what I want to do: the client should open an IPsec tunnel to
the server. The client should request an IPv6 address from the server's
enormous /64 subnet. Then, applications running on the client should
have the option to use that "virtual" IPv6 address as though it was
available on a local interface on the client machine. The client would
effectively be multihosted: in addition to its regular physical
Internet connection it would also have this IPsec tunnel that also
connects to the global Internet. My plan right now is to use rdomains
to control which applications use the tunnel and which ones use the
regular egress interface, but I'm not attached to that specific
approach.

The application should be obvious: I'm trying to set up a region-faking
VPN, so that I can have applications appear to connect to the Internet
from different continents.

Both client and server are using the same pf.conf that is installed by
default. I tried adding "pass on enc0" to the client's pf.conf and
"pass on enc0 tagged ROADW" to the server's, but it hasn't made a
difference because (I assume) my existing pf.conf rules weren't
blocking much to begin with.

Here are my config files, with IP addresses and server names changed.
But basically you should see the client at 2001:db8:1::1 connecting to
the server at 2001:db8:2::1 and requesting a random IP address in the
server's subnet, e.g. 2001:db8:2::485b:4ac7. I have also put the
appropriate keys in /etc/iked/pubkeys/fqdn/.

=== Client-side configuration ===
$ cat /etc/hostname.lo1
rdomain 1

$ cat /etc/iked.conf
ikev2 'client_config' active esp \
rdomain 1 \
from dynamic to any \
local 2001:db8:1::1 \
peer  2001:db8:2::1 \
srcid client.example.org \
dstid server.example.org \
request address any \
iface lo1

=== Server-side configuration ===

$ cat /etc/sysctl.conf
net.inet6.ip6.forwarding=1

$ cat /etc/iked.conf
ikev2 'server_config' passive esp \
from any to dynamic \
local 2001:db8:2::1 \
peer  2001:db8:1::1 \
srcid server.example.org \
config address 2001:db8:2::/64 \
tag "ROADW"

=== End of configurations ===

Does it work? Sort of. Here's what I see on the server:

=== Checking status on the server ===

# ipsecctl -sa
FLOWS:
flow esp in from 2001:db8:2::485b:4ac7 to ::/0 peer 2001:db8:1::1 srcid 
FQDN/server.example.org dstid FQDN/client.example.org type require
flow esp out from ::/0 to 2001:db8:2::485b:4ac7 peer 2001:db8:1::1 srcid 
FQDN/server.example.org dstid FQDN/client.example.org type require

SAD:
esp tunnel from 2001:db8:2::1 to 2001:db8:1::1 spi 0x001a3754 enc aes-128-gcm
esp tunnel from 2001:db8:1::1 to 2001:cb8:2::1 spi 0x7634a3b6 enc aes-128-gcm

$ route show
Internet6:
Destination Gateway Flags   Refs  Use   Mtu  Prio Iface
default fe80::1234%vio0 UGS8  578 - 8 vio0 
::/96   localhost   UGRS   00 32768 8 lo0  
localhost   localhost   UHhl  10   90 32768 1 lo0  
:::0.0.0.0/96   localhost   UGRS   00 32768 8 lo0  
2001:db8:2::/64 server  UCn3 3630 - 4 vio0 
2001:db8:2::485b:4ac7   link#1  UHLc   0 3633 - 3 vio0 
...and a whole lot more

=== Checking status on the client ===

# ipsecctl -sa
FLOWS:
No flows

SAD:
esp tunnel from 2001:db8:2::1 to 2600:db8:1::1 spi 0x001a3754 enc aes-128-gcm

$ ifconfig lo1

Re: IPsec over PPPoE

[Stuart Henderson]

On 2023-06-28, Stefan Sperling  wrote:
> Flow source/destination IPs must exactly match packets leaving the box.
>
> So you will either need to put the private IP as the from "src" of the
> flow (which can be annoying if it changes at run-time, you need to adjust
> and reload flows somehow when the private IP changes) or use "any" as
> source with the other end's static public IP as the destination (and be
> careful with your flow traffic selectors if you still need to send some
> traffic towards the other side without IPsec).

or you can do things with the route table to make sure that packets
destined for the lan-to-lan tunnel hit a route table entry where the
"interface address" is part of the tunnel's "from" address,

e.g. if both sides of the lan-to-lan tunnel are covered by say
172.28.128.0/19, you can configure a dummy interface with an address
that's inside the "from" bit of the configured flow

$ route -n get 172.28.129.1
   route to: 172.28.129.1
destination: 172.28.128.0
   mask: 255.255.224.0
  interface: vether25
 if address: 172.28.128.1
   priority: 4 (connected)
  flags: 
 use   mtuexpire
  108413 0 0 

so now when you try to connect to a host via the tunnel, it'll match
that route lookup, use the "if address" as the source address, and
that will match the flow

no need for an "any" that you might not want, and no need for a separate
flow covering the private address.

Re: IPsec over PPPoE

[Stuart Henderson]

On 2023-06-28, Jiri Navratil  wrote:
> Hello,
>
> I'm trying to build Site-to-site VPN based on "Configuring an IKEv2 Server"
> in https://www.openbsd.org/faq/faq17.html
>
> I see in iked -dv output to terminal (I replaced some parts with dots)
>
> spi=0x4905:
> established peer ...:4500[FQDN/.]
> local .:4500[FQDN/.]
> policy '._rsa' as responder
> (enc aes-128-gcm group curve25519 prf hmac- sha2-256
>
>
> and
>
> spi=0x16f.:
> established peer ..:4500[FQDN/.]
> local ...:4500[FQDN/.]
> policy '._rsa' as initiator
> (enc aes-128-gcm group curve25519 prf hmac-sha2-256)
>
> I can't ping from one location to other one. I see from ipsecctl -sa on
> responder side FLOWS and SAD expected lines, but there is nothing from

Show your config (mask external IPs if you must but leave the private
ones), and show how you test (what commands you typed, on what machines,
and what happens - show the actual outout).

> nc -u -l 500
> tcpdump -nei pflog0 rnr 35
>
> Could you please help me with some answers?
>
> 1) The FAQ have pf rules for responder only. No rules are needed for 
> initiator?

It depends on your other rules and what traffic you want toallow. In
many cases an initiator will have "pass out" or similar already and then
often nothing more is needed, but it depends.

> 2) The FAQ describe ipsec.conf changes and enabling only in "Connecting
> to an IKEv1/L2TP VPN". Nothing is needed in "IKEv2 Site-to-site VPNs"?

With a couple of notable exceptions that are probably irrelevant to you
(bypass and deny flows), ipsec.conf is just for isakmpd (IKEv1).

> 3) The sites I'm configuring are both using PPPoE. One have VLAN and I
> see external statical IPv4 on PPPoE, but other site uses NAT 1:1, so I
> see private IPv4 on PPPoE, but I have to access it over allocated
> external IPv4. I'm not sure, which IP comes where. I switched responder
> and initiator, to have responder on site with VLAN, but anyway I'm not
> sure, where in pf.conf and /etc/iked.conf use external and where NAT IP.

Is the pppoe run on the openbsd machine or a separate router?

You will want the 'peer' addresses to be "whatever address you need to
connect to, to send packets to that machine". And 'local' addresses, if
used, to be the address on the interface on the local machine.

> 4) Using enc0 in pf.conf not worked. I had to switch to pppoe. Is that
> correct? No rules for enc0 and vlan?

Not worked how?

Show the config, show what happened.

You will need to either pass or skip pf processing of traffic on enc0
one way or another.

> 4) I don't see any output from nc and tcdump commands. How I can see,
> which pf rule stops ping from other site?

*IF* it is being blocked by a PF rule, make sure you have "log" on any
block rules, and watch tcpdump -neipflog0.

But you might just be trying to ping from an address which does not
match a flow sent over the tunnel, in which case it will just be sent
over the standard internet connection. (Might be easier to test from
another machine on the tunneled subnet rather than the iked box itself).

If you're only used to packet forwarding via the route table (which only
considers the *destination* address), you won't be familiar with flow-
based traffic selectors, which also consider the *source* address (and
maybe also port numbers etc, if configured to do so).

When using tcpdump, don't just look at interfaces where you expect to
see the traffic. Look at others where it might conceivably end up, too.

> 5) There is note in FAQ, that Native WireGuard support is also
> available. As both IPsec and WireGuard are new to me, may wg(4) be an
> option?

Yes it's an option, as are openvpn, ssh tun-forwarding, dsvpn, isakmpd, [...]

> 6) Any good IPsec reading next to FAQ and man pages?

It is a wide subject, you'll need something more targetted than just
"good reading on IPsec" to give recommendations. But for iked-to-iked,
you can broadly expect things to work, and there are sufficient tools
in OpenBSD base to help diagnose what's going wring.

-- 
Please keep replies on the mailing list.

Re: IPsec over PPPoE

[Stefan Sperling]

On Wed, Jun 28, 2023 at 09:53:38AM +0200, Jiri Navratil wrote:
> 3) The sites I'm configuring are both using PPPoE. One have VLAN and I
> see external statical IPv4 on PPPoE, but other site uses NAT 1:1, so I
> see private IPv4 on PPPoE, but I have to access it over allocated
> external IPv4. I'm not sure, which IP comes where. I switched responder
> and initiator, to have responder on site with VLAN, but anyway I'm not
> sure, where in pf.conf and /etc/iked.conf use external and where NAT IP.

Flow source/destination IPs must exactly match packets leaving the box.

So you will either need to put the private IP as the from "src" of the
flow (which can be annoying if it changes at run-time, you need to adjust
and reload flows somehow when the private IP changes) or use "any" as
source with the other end's static public IP as the destination (and be
careful with your flow traffic selectors if you still need to send some
traffic towards the other side without IPsec).

Flows towards a box behind NAT need to use the NAT's public IP to match
outgoing packets sent towards the internet.

You can ignore 'srcnat' in this context. 'srcnat' could be used later
to NAT traffic which crosses the tunnel, once the tunnel is working.

> 4) Using enc0 in pf.conf not worked. I had to switch to pppoe. Is that
> correct? No rules for enc0 and vlan?

You can just set skip on enc0 until it is working. Filtering on enc0 is
only needed if you want to prevent some traffic matched by flows from
crossing the tunnel (depending on the situation, adjusting flows to
avoid matching the unwanted traffic in the first place might be better).

> 4) I don't see any output from nc and tcdump commands. How I can see,
> which pf rule stops ping from other site?

Because one side is behind NAT the ESP packets should automatically be
encapsulated in UDP (ESP by itself cannot cross NAT).
You should see UDP packets on port 4500 (ipsec-nat-t) being exchanged,
both during handshakes and during regular operation of the tunnel.
These UDP packets need to pass all the way through for things to work.

> 5) There is note in FAQ, that Native WireGuard support is also
> available. As both IPsec and WireGuard are new to me, may wg(4) be an
> option?

Yes, you can try it as an alternative. It can be easier to get going across
NAT since external IP addresses don't matter to Wireguard. Wireguard only
cares about whether incoming packets have been signed with the correct key.

Re: IPsec over PPPoE

[Janne Johansson]

>
> 5) There is note in FAQ, that Native WireGuard support is also
> available. As both IPsec and WireGuard are new to me, may wg(4) be an
> option?
>

Yes, it should be a good option for site2site tunnels.

-- 
May the most significant bit of your life be positive.

IPsec over PPPoE

[Jiri Navratil]

Hello,

I'm trying to build Site-to-site VPN based on "Configuring an IKEv2 Server"
in https://www.openbsd.org/faq/faq17.html

I see in iked -dv output to terminal (I replaced some parts with dots)

spi=0x4905:
established peer ...:4500[FQDN/.]
local .:4500[FQDN/.]
policy '._rsa' as responder
(enc aes-128-gcm group curve25519 prf hmac- sha2-256


and

spi=0x16f.:
established peer ..:4500[FQDN/.]
local ...:4500[FQDN/.]
policy '._rsa' as initiator
(enc aes-128-gcm group curve25519 prf hmac-sha2-256)

I can't ping from one location to other one. I see from ipsecctl -sa on
responder side FLOWS and SAD expected lines, but there is nothing from

nc -u -l 500
tcpdump -nei pflog0 rnr 35

Could you please help me with some answers?

1) The FAQ have pf rules for responder only. No rules are needed for initiator?

2) The FAQ describe ipsec.conf changes and enabling only in "Connecting
to an IKEv1/L2TP VPN". Nothing is needed in "IKEv2 Site-to-site VPNs"?

3) The sites I'm configuring are both using PPPoE. One have VLAN and I
see external statical IPv4 on PPPoE, but other site uses NAT 1:1, so I
see private IPv4 on PPPoE, but I have to access it over allocated
external IPv4. I'm not sure, which IP comes where. I switched responder
and initiator, to have responder on site with VLAN, but anyway I'm not
sure, where in pf.conf and /etc/iked.conf use external and where NAT IP.

4) Using enc0 in pf.conf not worked. I had to switch to pppoe. Is that
correct? No rules for enc0 and vlan?

4) I don't see any output from nc and tcdump commands. How I can see,
which pf rule stops ping from other site?

5) There is note in FAQ, that Native WireGuard support is also
available. As both IPsec and WireGuard are new to me, may wg(4) be an
option?

6) Any good IPsec reading next to FAQ and man pages?

Thank you,
Jiří

Re: Route based IPsec

[B. Atticus Grobe]

On 5/31/23 05:03, Valdrin MUJA wrote:
> Hi Claudio & David,
>
> Wireguard can work behind NAT. In that case maybe the solution is 
wireguard + BGP.



I've been using OSPF over wireguard for several years now. It works 
quite well. You just have to add `wgaip 224.0.0.0/8' to allow multicast 
over the link.

Re: Route based IPsec

[Valdrin MUJA]

Hi Claudio & David,

Wireguard can work behind NAT. In that case maybe the solution is wireguard + 
BGP.

Infact, I already tried this and wanted to use BGP multipath but failed and 
sent it to the misc list in a separate mail.

(I wrote gre + bgp in the related mail, my aim was not to prolong my work with 
the wireguard config.)

From: owner-m...@openbsd.org  on behalf of Claudio 
Jeker 
Sent: Wednesday, May 31, 2023 12:09
To: David Gwynne 
Cc: Misc 
Subject: Re: Route based IPsec

On Wed, May 31, 2023 at 06:39:27PM +1000, David Gwynne wrote:
>
>
> > On 31 May 2023, at 18:33, Claudio Jeker  wrote:
> >
> > On Wed, May 31, 2023 at 08:35:45AM +1000, David Gwynne wrote:
> >>
> >>
> >>> On 27 May 2023, at 21:40, Stuart Henderson  
> >>> wrote:
> >>>
> >>> On 2023-05-27, Valdrin MUJA  wrote:
> >>>>   Does OpenBSD have routed based IPsec support?
> >>>
> >>> Not yet.
> >>
> >> while you wait, it might be possible to configure a gif tunnel protected
> >> by ipsec transport mode.
> >>
> >
> > The annoying bit with gif tunnels in transport mode is the need for static
> > IPs on both sides of the tunnel. I ended up tunneling gif in tunnel mode
> > because of that.
>
> that's an annoying thing about gif, even without ipsec in the mix.

Indeed. Both gif and gre share this issue.

> should i make it possible to specify an interface as the source of local
> addresses on tunnels?

Not sure if it is worth the effort since the other end of the tunnel needs
to adjust the tunnel remote address as well. Neither gif nor gre support
authentication. Using wg(4) for that is an option but because of dynamic
routing I ended up packing a gif tunnel into wg(4) (so I'm back to square
one).

--
:wq Claudio

Re: Route based IPsec

[Claudio Jeker]

On Wed, May 31, 2023 at 06:39:27PM +1000, David Gwynne wrote:
> 
> 
> > On 31 May 2023, at 18:33, Claudio Jeker  wrote:
> > 
> > On Wed, May 31, 2023 at 08:35:45AM +1000, David Gwynne wrote:
> >> 
> >> 
> >>> On 27 May 2023, at 21:40, Stuart Henderson  
> >>> wrote:
> >>> 
> >>> On 2023-05-27, Valdrin MUJA  wrote:
> >>>>   Does OpenBSD have routed based IPsec support?
> >>> 
> >>> Not yet.
> >> 
> >> while you wait, it might be possible to configure a gif tunnel protected
> >> by ipsec transport mode.
> >> 
> > 
> > The annoying bit with gif tunnels in transport mode is the need for static
> > IPs on both sides of the tunnel. I ended up tunneling gif in tunnel mode
> > because of that.
> 
> that's an annoying thing about gif, even without ipsec in the mix.

Indeed. Both gif and gre share this issue.
 
> should i make it possible to specify an interface as the source of local
> addresses on tunnels?
 
Not sure if it is worth the effort since the other end of the tunnel needs
to adjust the tunnel remote address as well. Neither gif nor gre support
authentication. Using wg(4) for that is an option but because of dynamic
routing I ended up packing a gif tunnel into wg(4) (so I'm back to square
one).

-- 
:wq Claudio

Re: Route based IPsec

[David Gwynne]

> On 31 May 2023, at 18:33, Claudio Jeker  wrote:
> 
> On Wed, May 31, 2023 at 08:35:45AM +1000, David Gwynne wrote:
>> 
>> 
>>> On 27 May 2023, at 21:40, Stuart Henderson  
>>> wrote:
>>> 
>>> On 2023-05-27, Valdrin MUJA  wrote:
>>>>   Does OpenBSD have routed based IPsec support?
>>> 
>>> Not yet.
>> 
>> while you wait, it might be possible to configure a gif tunnel protected
>> by ipsec transport mode.
>> 
> 
> The annoying bit with gif tunnels in transport mode is the need for static
> IPs on both sides of the tunnel. I ended up tunneling gif in tunnel mode
> because of that.

that's an annoying thing about gif, even without ipsec in the mix.

should i make it possible to specify an interface as the source of local 
addresses on tunnels?

Re: Route based IPsec

[Claudio Jeker]

On Wed, May 31, 2023 at 08:35:45AM +1000, David Gwynne wrote:
> 
> 
> > On 27 May 2023, at 21:40, Stuart Henderson  
> > wrote:
> > 
> > On 2023-05-27, Valdrin MUJA  wrote:
> >>    Does OpenBSD have routed based IPsec support?
> > 
> > Not yet.
> 
> while you wait, it might be possible to configure a gif tunnel protected
> by ipsec transport mode.
> 

The annoying bit with gif tunnels in transport mode is the need for static
IPs on both sides of the tunnel. I ended up tunneling gif in tunnel mode
because of that.

-- 
:wq Claudio

Re: Route based IPsec

[Valdrin MUJA]

Thanks David, I'll try it soon.

From: owner-m...@openbsd.org  on behalf of David Gwynne 

Sent: Wednesday, May 31, 2023 01:35
To: Stuart Henderson 
Cc: misc@openbsd.org 
Subject: Re: Route based IPsec



> On 27 May 2023, at 21:40, Stuart Henderson  wrote:
>
> On 2023-05-27, Valdrin MUJA  wrote:
>>Does OpenBSD have routed based IPsec support?
>
> Not yet.

while you wait, it might be possible to configure a gif tunnel protected by 
ipsec transport mode.

dlg

Re: Route based IPsec

[David Gwynne]

> On 27 May 2023, at 21:40, Stuart Henderson  wrote:
> 
> On 2023-05-27, Valdrin MUJA  wrote:
>>Does OpenBSD have routed based IPsec support?
> 
> Not yet.

while you wait, it might be possible to configure a gif tunnel protected by 
ipsec transport mode.

dlg

Re: Route based IPsec

[Hrvoje Popovski]

On 27.5.2023. 9:24, Valdrin MUJA wrote:
> Hello,
> 
> I need Route based IPsec solution to set up between a firewall device and 
> my OpenBSD firewall.
> However, I am a little confused about this:
> I created more than one enc device, I did policy based routing with PF but no 
> results. I guess this is not the intended use of interfaces like enc[0,1]. 
> But since I am not sure, I would to ask:
> Does OpenBSD have routed based IPsec support? Thanks in advance.
> 

little off topic ...if other side is aws ipsec gateway or vmware nsx,
then policy based ipsec is working quite nice, but yeah, route based
ipsec would be awesome

Re: Route based IPsec

[Stuart Henderson]

On 2023-05-27, Valdrin MUJA  wrote:
> Does OpenBSD have routed based IPsec support?

Not yet.

Route based IPsec

[Valdrin MUJA]

Hello,

I need Route based IPsec solution to set up between a firewall device and 
my OpenBSD firewall.
However, I am a little confused about this:
I created more than one enc device, I did policy based routing with PF but no 
results. I guess this is not the intended use of interfaces like enc[0,1]. But 
since I am not sure, I would to ask:
Does OpenBSD have routed based IPsec support? Thanks in advance.

ipsec via strongswan (traffic present but no response)

[Gregory Edigarov]

Hello,

lbld12# uname -a
OpenBSD lbld12.duckdns.org 7.3 GENERIC.MP#1130 amd64

Our current vpn uses user/password authentication, mschapv2. so I am
trying to use strongswan to connect to my workplace.

# ipsec statusall 

Security Associations (1 up, 0 connecting):
   qarea[1]: ESTABLISHED 62 minutes ago, 
178.151.162.44[edigarov]...185.78.xxx.1[vpn.xxx.org]
   qarea[1]: IKEv2 SPIs: 62417f797a2ca675_i* 6db16adc7d9f5355_r, EAP 
reauthentication in 101 minutes
   qarea[1]: IKE proposal: 
AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
   qarea{2}:  INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: f07d99fb_i 
0ef2e82a_o
   qarea{2}:  AES_CBC_256/HMAC_SHA2_512_256, 0 bytes_i, 67604 bytes_o (806 
pkts, 18s ago), rekeying in 32 minutes
   qarea{2}:   192.168.112.215/32 === 192.168.12.0/22

# pfctl -s st  |grep 185.78 
all udp 178.151.162.44:4500 -> 185.78.235.1:4500   MULTIPLE:MULTIPLE

tcpdump on external physical interface:

12:06:56.040573 185.78.xxx.1.4500 > 178.151.162.44.4500: udpencap: esp spi 
0xf07d99fb seq 812 len 152 [tos 0x8]
12:06:57.037764 178.151.162.44.4500 > 185.78.235.1.4500: udpencap: esp spi 
0x0ef2e82a seq 812 len 152
12:06:57.044270 185.78.235.1.4500 > 178.151.162.44.4500: udpencap: esp spi 
0xf07d99fb seq 813 len 152 [tos 0x8]
12:06:58.037795 178.151.162.44.4500 > 185.78.235.1.4500: udpencap: esp spi 
0x0ef2e82a seq 813 len 152
12:06:58.044250 185.78.235.1.4500 > 178.151.162.44.4500: udpencap: esp spi 
0xf07d99fb seq 814 len 152 [tos 0x8]
12:06:58.239755 185.78.235.1.4500 > 178.151.162.44.4500: udpencap: isakmp v2.0 
exchange INFORMATIONAL
cookie: 62417f797a2ca675->6db16adc7d9f5355 msgid: 0020 len: 160 
(DF) [tos 0x8]
12:06:58.240035 178.151.162.44.4500 > 185.78.235.1.4500: udpencap: isakmp v2.0 
exchange INFORMATIONAL
cookie: 62417f797a2ca675->6db16adc7d9f5355 msgid: 0020 len: 80
12:06:59.037758 178.151.162.44.4500 > 185.78.235.1.4500: udpencap: esp spi 
0x0ef2e82a seq 814 len 152
12:06:59.044223 185.78.235.1.4500 > 178.151.162.44.4500: udpencap: esp spi 
0xf07d99fb seq 815 len 152 [tos 0x8]
12:07:00.037804 178.151.162.44.4500 > 185.78.235.1.4500: udpencap: esp spi 
0x0ef2e82a seq 815 len 152
12:07:00.044319 185.78.235.1.4500 > 178.151.162.44.4500: udpencap: esp spi 
0xf07d99fb seq 816 len 152 [tos 0x8]
12:07:01.037803 178.151.162.44.4500 > 185.78.235.1.4500: udpencap: esp spi 
0x0ef2e82a seq 816 len 152
12:07:01.044248 185.78.235.1.4500 > 178.151.162.44.4500: udpencap: esp spi 
0xf07d99fb seq 817 len 152 [tos 0x8]

however, on tunnel interface, that is tun1 there are no responses:

tcpdump: listening on tun1, link-type LOOP
12:08:53.037668 192.168.112.215 > 192.168.12.49: icmp: echo request
12:08:54.037698 192.168.112.215 > 192.168.12.49: icmp: echo request
12:08:55.037682 192.168.112.215 > 192.168.12.49: icmp: echo request
12:08:56.037679 192.168.112.215 > 192.168.12.49: icmp: echo request
12:08:57.037671 192.168.112.215 > 192.168.12.49: icmp: echo request
12:08:58.037683 192.168.112.215 > 192.168.12.49: icmp: echo request
12:08:59.037677 192.168.112.215 > 192.168.12.49: icmp: echo request
12:09:00.037671 192.168.112.215 > 192.168.12.49: icmp: echo request
12:09:01.037690 192.168.112.215 > 192.168.12.49: icmp: echo request
12:09:02.037678 192.168.112.215 > 192.168.12.49: icmp: echo request
12:09:03.037680 192.168.112.215 > 192.168.12.49: icmp: echo request

if I disable pf the picture stays the same. in pf.conf i have:
pass out on tun1 from self to any #nat-to (tun1)
pass out from self to any
pass in on egress proto udp from 185.78.235.1 to (egress) port 4500

# netstat -rn | grep tun1
192.168.12/22  192.168.112.215US 0   18 - 8 tun1 
192.168.112.215192.168.112.215UHl01 - 1 tun1 

What gives?

Re: Ipsec + bridge + egre issue with multiple bridges an non-static ip

[Markus Wipp]

Hi all,

Sorry for the noise. I found out that it was pf.
When I tested with pf disabled I always only did this with pf disabled on one 
side. Once I disabled on both sides it worked.
So I need to figure out now, what exactly is the issue.

Thanks
Markus

> On 26. Nov 2022, at 11:19, Markus Wipp  wrote:
> 
> Hi all,
> 
> I hope that someone here on the list could give me some hints on how I can 
> make my setup working.
> 
> I have the following setup:
> 
> "Virtual server 1" is connected to "Virtual server 2" via egre over ipsec on 
> both sides I’m using a bridge and a vether interface.
> Both virtual servers are located at different hosters and have public ip 
> addresses.
> Between them the mentioned private connection is always coming up and working 
> (I can ping 192.168.79.1 / 192.168.79.2 from each other)
> 
> In addition I have my router at home which connects via separate egre over 
> ipsec with a bridge and a vether interface connections
> to each of the virtual servers. This router unfortunately has only a dynamic 
> ipv4 address.
> The connection between the router and the virtual servers is for some reason 
> not coming up completely.
> To my analysis so far it seems that the router bridge learns the Mac 
> addresses of the remote virtual servers vether interfaces, but for
> some reason the bridges on the virtual servers do not learn the address of 
> the routers vether interface.
> tcpdump does show traffic coming into enc0, but it never reaches the bridge, 
> even with pf disabled.
> 
> 
> As I can ping the interface with ip 192.168.66.1 from each of the virtual 
> servers on the router, I’m leaving out the iced configuration.
> If this is needed I could also provide it.
> 
> Find here the corresponding configurations of each of the machines:
> 
> Virtual server 1:
> (Working between virtual server 1 and 2)
> /etc/hostname.bridge0
> add vether0
> add egre0
> up
> 
> /etc/hostname.vether0
> mtu 1500
> inet 192.168.79.1/24
> up
> 
> /etc/hostname.egre0
> mtu 1500 -tunneldf
> tunnel a.b.c.d w.x.y.z
> vnetid 12
> up
> 
> (Not working between virtual server 1 and router)
> /etc/hostname.bridge2
> add vether1
> add egre1
> up
> 
> /etc/hostname.vether1
> mtu 1500
> inet 192.168.80.1/24
> up
> 
> /etc/hostname.egre1
> mtu 1500 -tunneldf
> tunnel a.b.c.d 192.168.66.1
> vnetid 31
> up
> 
> Virtual server 2:
> (Working between virtual server 1 and 2)
> /etc/hostname.bridge0
> add vether0
> add egre0
> up
> 
> /etc/hostname.vether0
> mtu 1500
> inet 192.168.79.2/24
> up
> 
> /etc/hostname.egre0
> mtu 1500 -tunneldf
> tunnel w.x.y.z a.b.c.d
> vnetid 12
> up
> 
> (Not working between virtual server 1 and router)
> /etc/hostname.bridge2
> add vether2
> add egre2
> up
> 
> /etc/hostname.vether2
> mtu 1500
> inet 192.168.81.1/24
> up
> 
> /etc/hostname.egre2
> mtu 1500 -tunneldf
> tunnel w.x.y.z 192.168.66.1
> vnetid 32
> up
> 
> 
> Router:
> /etc/hostname.bridge0
> add vether1
> add egre1
> up
> 
> /etc/hostname.vether1
> mtu 1500
> inet 192.168.80.2/24
> up
> 
> /etc/hostname.egre1
> mtu 1500 -tunneldf
> tunnel 192.168.66.1 a.b.c.d
> vnetid 31
> up
> 
> /etc/hostname.bridge2
> add vether2
> add egre2
> up
> 
> /etc/hostname.vether2
> mtu 1500
> inet 192.168.81.2/24
> up
> 
> /etc/hostname.egre2
> mtu 1500 -tunneldf
> tunnel 192.168.66.1 w.x.y.z
> vnetid 32
> up
> 
> As an example I provide here the output of ifconfig for the relevant 
> interfaces on virtual server 1 (ipv6 stuff removed):
> 
> 
> vio0: 
> flags=e08843
>  mtu 1500
> lladdr 56:00:03:8c:96:8c
> index 1 priority 0 llprio 3
> groups: egress
> media: Ethernet autoselect
> status: active
> inet a.b.c.d netmask 0xfe00 broadcast 199.247.3.255
> 
> enc0: flags=41
> index 2 priority 0 llprio 3
> groups: enc
> status: active
> 
> bridge0: flags=41 mtu 1500
> index 4 llprio 3
> groups: bridge
> priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp
> egre0 flags=3
> port 6 ifpriority 0 ifcost 0
> vether0 flags=3
> port 8 ifpriority 0 ifcost 0
> 
> bridge2: flags=41 mtu 1500
> index 5 llprio 3
> groups: bridge
> priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp
> egre1 flags=3
> port 12 ifpriority 0 ifcost 0
> vether1 flags=3
> port 9 ifpriority 0 ifcost 0
> 
> egre0: flags=8943 mtu 1500
> lladdr fe:e1:ba:d0:b9:3c
> index 6 priority 0 llprio 3
> encap: vnetid 12 txprio 0 rxprio packet
> groups: egre
> tunnel: inet a.b.c.d --> w.x.y.z ttl 64 nodf
>

Ipsec + bridge + egre issue with multiple bridges an non-static ip

[Markus Wipp]

Hi all,

I hope that someone here on the list could give me some hints on how I can make 
my setup working.

I have the following setup:

"Virtual server 1" is connected to "Virtual server 2" via egre over ipsec on 
both sides I’m using a bridge and a vether interface.
Both virtual servers are located at different hosters and have public ip 
addresses.
Between them the mentioned private connection is always coming up and working 
(I can ping 192.168.79.1 / 192.168.79.2 from each other)

In addition I have my router at home which connects via separate egre over 
ipsec with a bridge and a vether interface connections
to each of the virtual servers. This router unfortunately has only a dynamic 
ipv4 address.
The connection between the router and the virtual servers is for some reason 
not coming up completely.
To my analysis so far it seems that the router bridge learns the Mac addresses 
of the remote virtual servers vether interfaces, but for
some reason the bridges on the virtual servers do not learn the address of the 
routers vether interface.
tcpdump does show traffic coming into enc0, but it never reaches the bridge, 
even with pf disabled.


As I can ping the interface with ip 192.168.66.1 from each of the virtual 
servers on the router, I’m leaving out the iced configuration.
If this is needed I could also provide it.

Find here the corresponding configurations of each of the machines:

Virtual server 1:
(Working between virtual server 1 and 2)
/etc/hostname.bridge0
add vether0
add egre0
up

/etc/hostname.vether0
mtu 1500
inet 192.168.79.1/24
up

/etc/hostname.egre0
mtu 1500 -tunneldf
tunnel a.b.c.d w.x.y.z
vnetid 12
up

(Not working between virtual server 1 and router)
/etc/hostname.bridge2
add vether1
add egre1
up

/etc/hostname.vether1
mtu 1500
inet 192.168.80.1/24
up

/etc/hostname.egre1
mtu 1500 -tunneldf
tunnel a.b.c.d 192.168.66.1
vnetid 31
up

Virtual server 2:
(Working between virtual server 1 and 2)
/etc/hostname.bridge0
add vether0
add egre0
up

/etc/hostname.vether0
mtu 1500
inet 192.168.79.2/24
up

/etc/hostname.egre0
mtu 1500 -tunneldf
tunnel w.x.y.z a.b.c.d
vnetid 12
up

(Not working between virtual server 1 and router)
/etc/hostname.bridge2
add vether2
add egre2
up

/etc/hostname.vether2
mtu 1500
inet 192.168.81.1/24
up

/etc/hostname.egre2
mtu 1500 -tunneldf
tunnel w.x.y.z 192.168.66.1
vnetid 32
up


Router:
/etc/hostname.bridge0
add vether1
add egre1
up

/etc/hostname.vether1
mtu 1500
inet 192.168.80.2/24
up

/etc/hostname.egre1
mtu 1500 -tunneldf
tunnel 192.168.66.1 a.b.c.d
vnetid 31
up

/etc/hostname.bridge2
add vether2
add egre2
up

/etc/hostname.vether2
mtu 1500
inet 192.168.81.2/24
up

/etc/hostname.egre2
mtu 1500 -tunneldf
tunnel 192.168.66.1 w.x.y.z
vnetid 32
up

As an example I provide here the output of ifconfig for the relevant interfaces 
on virtual server 1 (ipv6 stuff removed):


vio0: 
flags=e08843
 mtu 1500
lladdr 56:00:03:8c:96:8c
index 1 priority 0 llprio 3
groups: egress
media: Ethernet autoselect
status: active
inet a.b.c.d netmask 0xfe00 broadcast 199.247.3.255

enc0: flags=41
index 2 priority 0 llprio 3
groups: enc
status: active

bridge0: flags=41 mtu 1500
index 4 llprio 3
groups: bridge
priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp
egre0 flags=3
port 6 ifpriority 0 ifcost 0
vether0 flags=3
port 8 ifpriority 0 ifcost 0

bridge2: flags=41 mtu 1500
index 5 llprio 3
groups: bridge
priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp
egre1 flags=3
port 12 ifpriority 0 ifcost 0
vether1 flags=3
port 9 ifpriority 0 ifcost 0

egre0: flags=8943 mtu 1500
lladdr fe:e1:ba:d0:b9:3c
index 6 priority 0 llprio 3
encap: vnetid 12 txprio 0 rxprio packet
groups: egre
tunnel: inet a.b.c.d --> w.x.y.z ttl 64 nodf

vether0: flags=8943 mtu 1500
lladdr fe:e1:ba:d2:eb:05
index 8 priority 0 llprio 3
groups: vether
media: Ethernet autoselect
status: active
inet 192.168.79.1 netmask 0xff00 broadcast 192.168.79.255

vether1: flags=8943 mtu 1500
lladdr fe:e1:ba:d3:94:e9
index 9 priority 0 llprio 3
groups: vether
media: Ethernet autoselect
status: active
inet 192.168.80.1 netmask 0xff00 broadcast 192.168.80.255

egre1: flags=8943 mtu 1500
lladdr fe:e1:ba:d4:c5:8f
index 12 priority 0 llprio 3
encap: vnetid 31 txprio 0 rxprio packet
groups: egre
tunnel: inet a.b.c.d --> 192.168.66.1 ttl 64 nodf


And here the router side (ipv6 stuff removed):

em0: 
flags=808b43 
mtu 1500
lladdr 00:0d:b9:44:ec:dc
description: External Connection 1 Cable
index 1 priority 0 llprio 3
groups: egress
media: Ethe

Re: ipsec traffic is dropped between two machines

[readme]

On Wed, Mar 23, 2022 at 02:10:03PM +0100, Tobias Heider wrote:
>On Mon, Mar 21, 2022 at 01:04:28PM -0500, rea...@catastrophe.net wrote:
>> I have two openbsd machines configured to connect their respective
>> downstream networks over ipsec. When I try to generate traffic (ping)
>> from server-west's enc0 interface (10.255.255.1) to server-east's enc0
>> interface (10.254.255.1), traffic is sent out the corresponding
>> SA but is never seen on server-east's enc0 interface. Only when I
>> simultaneously generate traffic (ping, again) on server-east back to 
>> server-west do I see the echo replies from server-east on server-west.
>> 
>I don't fully understand your setup but having both 10.255.255.0/24 to
>10.254.255.0/24 and 10.254.255.0/24 to 10.255.255.0/24 configured on both
>sides does not make sense to me.

Good point, I've cleaned the configs up and just created statements
necessary following your configs here, with one addition on each side (so
the servers can ping each other over the tunnel without using their
respective enc0 interfaces as a source.

>Assuming 10.255.255.0/24 is reachable via server-west and 10.254.255.0/24 via
>server-east the configs should probably be:
>
>server-west:/etc/iked.conf
>-
>ikev2 'server-east.example.com' passive esp \
>from 10.255.255.0/24 to 10.254.255.0/24 \
>from 203.0.113.50/32 to 10.254.255.0/24 \
+from 203.0.113.50/32 to 100.64.1.92/32 \
>local 203.0.113.50 peer server-east.example.com \
>srcid server-west.example.com \
>dstid server-east.example.com \
>psk "12345" \
>tag "VPN.EAST"
>
>server-east:/etc/iked.conf
>-
>ikev2 'server-west.example.com' active esp \
>from 10.254.255.0/24 to 10.255.255.0/24 \
>from 100.64.1.92/32 to 10.255.255.0/24 \
+from 100.64.1.92/32 to 203.0.113.50/32 \
>local 100.64.1.92 peer server-west.example.com \
>srcid server-east.example.com \
>dstid server-west.example.com \
>psk "12345" \
>tag "VPN.WEST"
>

The general diagram of what this looks like is:

em0:203.0.113.50 -~-~- ipsec tunnel -~-~-~- vio0:100.64.1.92
 | SERVER-WEST | | SERVER-EAST |
enc0:10.255.255.1/24enc0:10.254.255.1/24

Trying to generate traffic from the physical interfaces on either server
(em0 or vio) fails towards either the remote physical interface or the
remote enc0 interface. I've included flows and `iked -dvvv` at the bottom.


Ping from enc0 on server-west to enc0 on server-east. Works as expected.

server-west# ping -I 10.255.255.1 10.254.255.1
PING 10.254.255.1 (10.254.255.1): 56 data bytes
64 bytes from 10.254.255.1: icmp_seq=0 ttl=255 time=46.493 ms
64 bytes from 10.254.255.1: icmp_seq=1 ttl=255 time=46.439 ms
64 bytes from 10.254.255.1: icmp_seq=2 ttl=255 time=46.222 ms
^C
--- 10.254.255.1 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 46.222/46.385/46.493/0.117 ms


Now try pinging from em0 on server-west to the remote side with no success.

server-west# ifconfig em0 |grep 174.136.105
inet 203.0.113.50 netmask 0xfffc broadcast 174.136.105.51

server-west# ping -I 203.0.113.50 10.254.255.1
PING 10.254.255.1 (10.254.255.1): 56 data bytes
^C
--- 10.254.255.1 ping statistics ---
7 packets transmitted, 0 packets received, 100.0% packet loss


Shouldn't this be covered by the following flow?
spi=0x3a561aeb30f190ce: ikev2_childsa_enable: loaded flows:
ESP-10.254.255.0/24=10.255.255.0/24(0),ESP-100.64.1.92/32=10.255.255.0/24(0), 
ESP-100.64.1.92/32=203.0.113.50/32(0) spi=0x3a561aeb30f190ce: sa_state: VALID 
-> ESTABLISHED from 203.0.113.50:500 to 100.64.1.92:500 policy 
'server-west.example.com'


Same problem on server-east ping server-west enc0 with traffic sourcing 
from server-east enc0

server-east# ping -I 10.254.255.1 10.255.255.1
PING 10.255.255.1 (10.255.255.1): 56 data bytes
64 bytes from 10.255.255.1: icmp_seq=0 ttl=255 time=46.407 ms
64 bytes from 10.255.255.1: icmp_seq=1 ttl=255 time=46.360 ms
64 bytes from 10.255.255.1: icmp_seq=2 ttl=255 time=46.361 ms
^C
--- 10.255.255.1 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 46.360/46.376/46.407/0.022 ms


Now try and ping from server-east vio0 to the remote side. This fails.

server-east# ifconfig vio0 |grep 45.76.227
inet 100.64.1.92 netmask 0xfe00 broadcast 45.76.227.255

server-east# ping -I 100.64.1.92 10.255.255.1
PING 10.255.255.1 (10.255.255.1): 56 data bytes
^C
--- 10.255.255.1 ping statistics ---
7 packets transmitted, 0 packets received, 100.0% packet loss


FLOWS
=

server-west# ipsecctl -sa
FLOWS:
flow esp in from 10.254.255.

Re: ipsec traffic is dropped between two machines

[Tobias Heider]

On Mon, Mar 21, 2022 at 01:04:28PM -0500, rea...@catastrophe.net wrote:
> I have two openbsd machines configured to connect their respective
> downstream networks over ipsec. When I try to generate traffic (ping)
> from server-west's enc0 interface (10.255.255.1) to server-east's enc0
> interface (10.254.255.1), traffic is sent out the corresponding
> SA but is never seen on server-east's enc0 interface. Only when I
> simultaneously generate traffic (ping, again) on server-east back to 
> server-west do I see the echo replies from server-east on server-west.
> 
> The flows look correct in the SA table on server-west and traffic leaves on
> enc0, hits vio0 on server-east as ESP traffic, but then is dropped. Again,
> only when I also start a ping on server-east (10.254.255.1) to server-west
> (10.255.255.1) does the original ping session see replies.
> 
> Any help is appreciated. Here are the relevant configs and outputs.

I don't fully understand your setup but having both 10.255.255.0/24 to
10.254.255.0/24 and 10.254.255.0/24 to 10.255.255.0/24 configured on both
sides does not make sense to me.

Assuming 10.255.255.0/24 is reachable via server-west and 10.254.255.0/24 via
server-east the configs should probably be:

server-west:/etc/iked.conf
-
ikev2 'server-east.example.com' passive esp \
from 10.255.255.0/24 to 10.254.255.0/24 \
from 203.0.113.50/32 to 10.254.255.0/24 \
local 203.0.113.50 peer server-east.example.com \
srcid server-west.example.com \
dstid server-east.example.com \
psk "12345" \
tag "VPN.EAST"

server-east:/etc/iked.conf
-
ikev2 'server-west.example.com' active esp \
from 10.254.255.0/24 to 10.255.255.0/24 \
from 100.64.1.92/32 to 10.255.255.0/24 \
local 100.64.1.92 peer server-west.example.com \
srcid server-east.example.com \
dstid server-west.example.com \
psk "12345" \
tag "VPN.WEST"

Re: ipsec traffic is dropped between two machines

[readme]

On Tue, Mar 22, 2022 at 09:56:49AM -0500, rea...@catastrophe.net wrote:
>Rules on both sides are:
>
># server-east 
>--
>pass in  proto udp from any to self port { isakmp, ipsec-nat-t } keep state 
>pass out proto udp from any to any port { isakmp, ipsec-nat-t } keep state
>
>pass in  proto { esp, ah } from any to vio0 keep state 
>pass out proto { esp, ah } from vio0 to any keep state
>
>pass on log enc0 keep state (if-bound) tagged VPN.LAX
>pass on log enc0 keep state (if-bound)

[..]

It definitely looks like my issue is the ipsec flows and not pf.

I start a ping on server-east after the ipsec flows are established and
traffic sourcing from server-east vio0 (100.64.1.92) take the default route
and do not get encapsulated when destined for enc0 on server-west
(10.255.255.1):

Flows:

server-east# ipsecctl -sa | grep 10.255.255
flow esp in from 10.255.255.0/24 to 10.254.255.0/24 peer 203.0.113.50 srcid 
FQDN/server-east.example.com dstid FQDN/server-west.example.com type require
flow esp in from 10.255.255.0/24 to 100.64.1.92 peer 203.0.113.50 srcid 
FQDN/server-east.example.com dstid FQDN/server-west.example.com type require
flow esp out from 10.254.255.0/24 to 10.255.255.0/24 peer 203.0.113.50 srcid 
FQDN/server-east.example.com dstid FQDN/server-west.example.com type require
flow esp out from 100.64.1.92 to 10.255.255.0/24 peer 203.0.113.50 srcid 
FQDN/server-east.example.com dstid FQDN/server-west.example.com type require


Start generating traffic:

server-east# ping 10.255.255.1 &
[1] 55341
server-east# PING 10.255.255.1 (10.255.255.1): 56 data bytes


Capture packets:

server-east# tcpdump -ni vio0 host 10.255.255.1
tcpdump: listening on vio0, link-type EN10MB
14:58:42.514298 100.64.1.92 > 10.255.255.1: icmp: echo request
14:58:43.514396 100.64.1.92 > 10.255.255.1: icmp: echo request
14:58:44.514326 100.64.1.92 > 10.255.255.1: icmp: echo request
14:58:45.514310 100.64.1.92 > 10.255.255.1: icmp: echo request
14:58:46.514340 100.64.1.92 > 10.255.255.1: icmp: echo request
14:58:47.514263 100.64.1.92 > 10.255.255.1: icmp: echo request
^C
29 packets received by filter
0 packets dropped by kernel
server-east# fg
ping 10.255.255.1
^C
--- 10.255.255.1 ping statistics ---
19 packets transmitted, 0 packets received, 100.0% packet loss


If I source from server-east enc0 (10.254.255.1), then the traffic goes over
the ipsec tunnel:

server-east# ping -c 5 -I 10.254.255.1 10.255.255.1  
PING 10.255.255.1 (10.255.255.1): 56 data bytes
64 bytes from 10.255.255.1: icmp_seq=0 ttl=255 time=26.369 ms
64 bytes from 10.255.255.1: icmp_seq=1 ttl=255 time=26.412 ms
64 bytes from 10.255.255.1: icmp_seq=2 ttl=255 time=26.519 ms
64 bytes from 10.255.255.1: icmp_seq=3 ttl=255 time=26.426 ms
64 bytes from 10.255.255.1: icmp_seq=4 ttl=255 time=26.575 ms

--- 10.255.255.1 ping statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 26.369/26.460/26.575/0.075 ms


Flows from em0 on server-west (203.0.113.50) have no problem when sourcing
from either it's enc0 (10.255.255.1) or em0 (203.0.113.50) interfaces toward
10.254.255.1. Those flows look like this, and they're basically a reverse of
that which is on server-east.

server-west# ipsecctl -sa | grep 10.254.255
flow esp in from 10.254.255.0/24 to 10.255.255.0/24 peer 100.64.1.92 srcid 
FQDN/server-west.example.com dstid FQDN/server-east.example.com type require
flow esp in from 10.254.255.0/24 to 203.0.113.50 peer 100.64.1.92 srcid 
FQDN/server-west.example.com dstid FQDN/server-east.example.com type require
flow esp out from 10.255.255.0/24 to 10.254.255.0/24 peer 100.64.1.92 srcid 
FQDN/server-west.example.com dstid FQDN/server-east.example.com type require
flow esp out from 203.0.113.50 to 10.254.255.0/24 peer 100.64.1.92 srcid 
FQDN/server-west.example.com dstid FQDN/server-east.example.com type require

Re: ipsec traffic is dropped between two machines

[readme]

On Tue, Mar 22, 2022 at 02:38:15AM +, Philipp Buehler wrote:
>Am 21.03.2022 19:04 schrieb rea...@catastrophe.net:
>> The flows look correct in the SA table on server-west and traffic leaves
>> on
>> enc0, hits vio0 on server-east as ESP traffic, but then is dropped.
>> Again,
>> only when I also start a ping on server-east (10.254.255.1) to
>> server-west
>> (10.255.255.1) does the original ping session see replies.
>
>Out of balance / asymmetric rule set not generating needed state.
>
[..]
>Check back your actual interfaces (vio0..) for ESP traffic allowance.
>The '@73' and '@58' already indicates a major difference so check for 'pass
>... proto esp'.

Thanks. There are only differences as one side has other rules for 
local access (some web server, etc.).

Rules on both sides are:

# server-east 
------
pass in  proto udp from any to self port { isakmp, ipsec-nat-t } keep state 
pass out proto udp from any to any port { isakmp, ipsec-nat-t } keep state

pass in  proto { esp, ah } from any to vio0 keep state 
pass out proto { esp, ah } from vio0 to any keep state

pass on log enc0 keep state (if-bound) tagged VPN.LAX
pass on log enc0 keep state (if-bound)

# server-west
------
pass in  proto udp from any to self port { isakmp, ipsec-nat-t } keep state 
pass out proto udp from any to any port { isakmp, ipsec-nat-t } keep state

pass in  proto { esp, ah } from any to em0 keep state 
pass out proto { esp, ah } from em0 to any keep state

pass on log enc0 keep state (if-bound) tagged VPN.ORD
pass on log enc0 keep state (if-bound)

Re: ipsec traffic is dropped between two machines

[Stuart Henderson]

On 2022-03-22, Philipp Buehler  
wrote:
>> server-east PF rule:
>> -
>> @58 pass log quick on enc0 all flags S/SA tagged VPN.WEST
>
> enc(4) is an observer interface and not meant to take pf rules besides 
> "set skip on enc0" :-)

I disagree, that's where I hang my "scrub max-mss" rules for tuneLled traffic..

Re: ipsec traffic is dropped between two machines

[Pawel Kraszewski]

Problem with service working after cross-pinging the other sides seems like
some stateful firewall that needs a nudge from inside.


-- 
Paweł Kraszewski

Re: ipsec traffic is dropped between two machines

[Philipp Buehler]

Am 21.03.2022 19:04 schrieb rea...@catastrophe.net:
The flows look correct in the SA table on server-west and traffic 
leaves on
enc0, hits vio0 on server-east as ESP traffic, but then is dropped. 
Again,
only when I also start a ping on server-east (10.254.255.1) to 
server-west

(10.255.255.1) does the original ping session see replies.


Out of balance / asymmetric rule set not generating needed state.


server-west PF rule:
-
@73 pass log quick on enc0 all flags S/SA tagged VPN.EAST


server-east PF rule:
-
@58 pass log quick on enc0 all flags S/SA tagged VPN.WEST


enc(4) is an observer interface and not meant to take pf rules besides 
"set skip on enc0" :-)


Check back your actual interfaces (vio0..) for ESP traffic allowance.
The '@73' and '@58' already indicates a major difference so check for 
'pass ... proto esp'.


HTH,

--
pb

Re: ipsec traffic is dropped between two machines

[readme]

On Mon, Mar 21, 2022 at 01:04:28PM -0500, rea...@catastrophe.net wrote:
[..]
>SAD:
>esp tunnel from 203.0.113.50 to 100.64.1 spi 0x54e00602 enc aes-128-gcm
>esp tunnel from 100.64.1 to 203.0.113.50 spi 0xcb8f2ddb enc aes-128-gcm

This flow should be:

esp tunnel from 203.0.113.50 to 100.64.1.92 spi 0x54e00602 enc aes-128-gcm
esp tunnel from 100.64.1.92 to 203.0.113.50 spi 0xcb8f2ddb enc aes-128-gcm


Some bits got lost in the email, but the flow was correct.

ipsec traffic is dropped between two machines

[readme]

I have two openbsd machines configured to connect their respective
downstream networks over ipsec. When I try to generate traffic (ping)
from server-west's enc0 interface (10.255.255.1) to server-east's enc0
interface (10.254.255.1), traffic is sent out the corresponding
SA but is never seen on server-east's enc0 interface. Only when I
simultaneously generate traffic (ping, again) on server-east back to 
server-west do I see the echo replies from server-east on server-west.

The flows look correct in the SA table on server-west and traffic leaves on
enc0, hits vio0 on server-east as ESP traffic, but then is dropped. Again,
only when I also start a ping on server-east (10.254.255.1) to server-west
(10.255.255.1) does the original ping session see replies.

Any help is appreciated. Here are the relevant configs and outputs.

server-west:/etc/iked.conf
-
ikev2 'server-east.example.com' passive esp \
from 10.255.255.0/24 to 10.254.255.0/24 \
from 10.254.255.0/24 to 10.255.255.0/24 \
from 203.0.113.50/32 to 10.254.255.0/24 \
local 203.0.113.50 peer server-east.example.com \
srcid server-west.example.com \
dstid server-east.example.com \
psk "12345" \
tag "VPN.EAST"

server-east:/etc/iked.conf
-
ikev2 'server-west.example.com' active esp \
from 10.254.255.0/24 to 10.255.255.0/24 \
from 10.255.255.0/24 to 10.254.255.0/24 \
from 100.64.1.92/32 to 10.255.255.0/24 \
local 100.64.1.92 peer server-west.example.com \
srcid server-east.example.com \
dstid server-west.example.com \
psk "12345" \
tag "VPN.WEST"


server-west SA table:
-
FLOWS:
flow esp in from 10.254.255.0/24 to 10.255.255.0/24 peer 100.64.1 srcid 
FQDN/server-west.example.com dstid FQDN/server-east.example.com type require
flow esp in from 10.254.255.0/24 to 203.0.113.50 peer 100.64.1 srcid 
FQDN/server-west.example.com dstid FQDN/server-east.example.com type require
flow esp in from 10.255.255.0/24 to 10.254.255.0/24 peer 100.64.1 srcid 
FQDN/server-west.example.com dstid FQDN/server-east.example.com type require
flow esp out from 10.254.255.0/24 to 10.255.255.0/24 peer 100.64.1 srcid 
FQDN/server-west.example.com dstid FQDN/server-east.example.com type require
flow esp out from 10.255.255.0/24 to 10.254.255.0/24 peer 100.64.1 srcid 
FQDN/server-west.example.com dstid FQDN/server-east.example.com type require
flow esp out from 203.0.113.50 to 10.254.255.0/24 peer 100.64.1 srcid 
FQDN/server-west.example.com dstid FQDN/server-east.example.com type require

SAD:
esp tunnel from 203.0.113.50 to 100.64.1 spi 0x54e00602 enc aes-128-gcm
esp tunnel from 100.64.1 to 203.0.113.50 spi 0xcb8f2ddb enc aes-128-gcm


server-east SA table:
-
FLOWS:
flow esp in from 10.254.255.0/24 to 10.255.255.0/24 peer 203.0.113.50 srcid 
FQDN/server-east.example.com dstid FQDN/server-west.example.com type require
flow esp in from 10.255.255.0/24 to 10.254.255.0/24 peer 203.0.113.50 srcid 
FQDN/server-east.example.com dstid FQDN/server-west.example.com type require
flow esp in from 10.255.255.0/24 to 100.64.1.92 peer 203.0.113.50 srcid 
FQDN/server-east.example.com dstid FQDN/server-west.example.com type require
flow esp out from 10.254.255.0/24 to 10.255.255.0/24 peer 203.0.113.50 srcid 
FQDN/server-east.example.com dstid FQDN/server-west.example.com type require
flow esp out from 10.255.255.0/24 to 10.254.255.0/24 peer 203.0.113.50 srcid 
FQDN/server-east.example.com dstid FQDN/server-west.example.com type require
flow esp out from 100.64.1.92 to 10.255.255.0/24 peer 203.0.113.50 srcid 
FQDN/server-east.example.com dstid FQDN/server-west.example.com type require

SAD:
esp tunnel from 203.0.113.50 to 100.64.1.92 spi 0x54e00602 enc aes-128-gcm
esp tunnel from 100.64.1.92 to 203.0.113.50 spi 0xcb8f2ddb enc aes-128-gcm


server-west PF rule:
-
@73 pass log quick on enc0 all flags S/SA tagged VPN.EAST


server-east PF rule:
-
@58 pass log quick on enc0 all flags S/SA tagged VPN.WEST


server-west `tcpdump -ni enc0'
-
11:03:11.000828 (authentic,confidential): SPI 0x54e00602: 10.255.255.1 > 
10.254.255.1: icmp: echo request (encap)
11:03:12.009003 (authentic,confidential): SPI 0x54e00602: 10.255.255.1 > 
10.254.255.1: icmp: echo request (encap)
11:03:13.008926 (authentic,confidential): SPI 0x54e00602: 10.255.255.1 > 
10.254.255.1: icmp: echo request (encap)
11:03:14.008978 (authentic,confidential): SPI 0x54e00602: 10.255.255.1 > 
10.254.255.1: icmp: echo request (encap)
11:03:15.009014 (authentic,confidential): SPI 0x54e00602: 10.255.255.1 > 
10.254.255.1: icmp: echo request (encap)
11:03:16.009057 (authentic,confidential): SPI 0x54e00602: 10.255.255.1 > 
10.254.255.1: icmp: echo request (encap)
11:03:17.008943 (authentic,confidential)

Re: IPSec fails with NO_PROPOSAL_CHOSEN when connecting from recent MacOS/iOS clients

[Dmitry Petrakoff]

Hi fix...@gmail.com.

I use this set of parameters for l2tp+IPSec. It works fine both with 
Windows and Apple ( includng iOS15 and OSX 12 )


Hope it'll help you.

ike passive esp transport proto udp from 100.88.99.100 to any port 1701 \
    main auth hmac-sha1 enc aes-256 group modp2048 \
    quick auth hmac-sha1 enc aes \
    psk ""

Regards.

On 19.02.22 00:55, fix...@gmail.com wrote:

On Fri, Feb 18, 2022 at 15:06 Stuart Henderson wrote...

On Fri, Feb 18, 2022 at 11:43 AM I wrote:

ike passive esp transport proto udp from $public_ip to any \
   main auth "hmac-sha2-256" enc "aes-256" group "modp2048" \
   quick auth "hmac-sha2-256" enc "aes-256" group "modp2048" \
   psk "THIS_IS_MY_IPSEC_PASSPHRASE"

ike passive esp transport proto udp from $public_ip to any \
  main auth "hmac-md5" enc "3des" group "modp1024" \
  quick auth "hmac-md5" enc "3des" group "modp1024" \
   psk "THIS_IS_MY_IPSEC_PASSPHRASE"

With isakmpd and ipsec.conf you can't have two proposals for the default
("to any") peer with different PFS groups, you will have to choose one
or the other. As-is the second will overwrite the first config block.
(Use ipsecctl -v to see the commands sent by ipsecctl to isakmpd;
it generates what are basically isakmpd.conf style config blocks and
sends them over the control socket).

It still fails even with one block, but this is good to know. Thanks.


You will save yourself a lot of trouble if you can move the newer machines
to IKEv2 .. (It would not be possible to run both isakmpd and iked on a
single OpenBSD machine though). Or alternatively wireguard or openvpn
(which _can_ coexist with IKEv1) though IKEv2 generally has a simpler
client config.

I'm not opposed to this and I've tried, but even now it still gives me
proposal errors from both iOS and MacOS

Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #4 ENCR=AES_CBC-128
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #4 PRF=HMAC_SHA1
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #4 INTEGR=HMAC_SHA1_96
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #4 DH=MODP_1024
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #5 ENCR=3DES
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #5 PRF=HMAC_SHA1
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #5 INTEGR=HMAC_SHA1_96
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #5 DH=MODP_1024
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_add_error: NO_PROPOSAL_CHOSEN
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65: send
IKE_SA_INIT res 0 peer 100.64.10.10:57904 local 203.0.113.1:500, 36
bytes


--
Mit Freundliche Gruesse
Dimon
tel: +4158 1000428
mobile: +4178 7299592

Re: IPSec fails with NO_PROPOSAL_CHOSEN when connecting from recent MacOS/iOS clients

[fixied]

On Fri, Feb 18, 2022 at 15:06 Stuart Henderson wrote...
> On Fri, Feb 18, 2022 at 11:43 AM I wrote:
>> ike passive esp transport proto udp from $public_ip to any \
>>   main auth "hmac-sha2-256" enc "aes-256" group "modp2048" \
>>   quick auth "hmac-sha2-256" enc "aes-256" group "modp2048" \
>>   psk "THIS_IS_MY_IPSEC_PASSPHRASE"
>>
>> ike passive esp transport proto udp from $public_ip to any \
>>  main auth "hmac-md5" enc "3des" group "modp1024" \
>>  quick auth "hmac-md5" enc "3des" group "modp1024" \
>>   psk "THIS_IS_MY_IPSEC_PASSPHRASE"

> With isakmpd and ipsec.conf you can't have two proposals for the default
> ("to any") peer with different PFS groups, you will have to choose one
> or the other. As-is the second will overwrite the first config block.
> (Use ipsecctl -v to see the commands sent by ipsecctl to isakmpd;
> it generates what are basically isakmpd.conf style config blocks and
> sends them over the control socket).

It still fails even with one block, but this is good to know. Thanks.

> You will save yourself a lot of trouble if you can move the newer machines
> to IKEv2 .. (It would not be possible to run both isakmpd and iked on a
> single OpenBSD machine though). Or alternatively wireguard or openvpn
> (which _can_ coexist with IKEv1) though IKEv2 generally has a simpler
> client config.

I'm not opposed to this and I've tried, but even now it still gives me
proposal errors from both iOS and MacOS

Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #4 ENCR=AES_CBC-128
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #4 PRF=HMAC_SHA1
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #4 INTEGR=HMAC_SHA1_96
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #4 DH=MODP_1024
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #5 ENCR=3DES
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #5 PRF=HMAC_SHA1
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #5 INTEGR=HMAC_SHA1_96
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_log_proposal: IKE #5 DH=MODP_1024
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65:
ikev2_add_error: NO_PROPOSAL_CHOSEN
Feb 18 15:51:04 server iked[86219]: spi=0xdc6e75a2891b8e65: send
IKE_SA_INIT res 0 peer 100.64.10.10:57904 local 203.0.113.1:500, 36
bytes

Re: IPSec fails with NO_PROPOSAL_CHOSEN when connecting from recent MacOS/iOS clients

[Stuart Henderson]

On 2022-02-18, fix...@gmail.com  wrote:
> On Fri, Feb 18, 2022 at 11:43 AM I wrote:
>> I recently started seeing some ipsec clients fail on newer versions of
>> MacOS and iOS. After MacOS 12.1, connecting to my head end now fails
>> with NO_PROPOSAL_CHOSEN using mod1024 in my ipsec.conf. I've also
>> tried, with no success:
>>
>> main auth "hmac-sha2" enc "aes" group modp1024
>> quick auth "hmac-sha2" enc "aes" group modp1024
>
> Doing further research shows things are just not working out.
>
> I added two different configs that should match in /etc/ipsec.conf
>
> ike passive esp transport proto udp from $public_ip to any \
>   main auth "hmac-sha2-256" enc "aes-256" group "modp2048" \
>   quick auth "hmac-sha2-256" enc "aes-256" group "modp2048" \
>   psk "THIS_IS_MY_IPSEC_PASSPHRASE"
>
> ike passive esp transport proto udp from $public_ip to any \
>   main auth "hmac-md5" enc "3des" group "modp1024" \
>   quick auth "hmac-md5" enc "3des" group "modp1024" \
>   psk "THIS_IS_MY_IPSEC_PASSPHRASE"

With isakmpd and ipsec.conf you can't have two proposals for the default
("to any") peer with different PFS groups, you will have to choose one
or the other. As-is the second will overwrite the first config block.
(Use ipsecctl -v to see the commands sent by ipsecctl to isakmpd;
it generates what are basically isakmpd.conf style config blocks and
sends them over the control socket).

There is a chance that by writing your own isakmpd.conf you might be
able to have two main mode (phase 1) proposals with differing groups,
but for quick mode it's a protocol limitation and it's only possible to
send one - and I don't think isakmpd can let you choose what to send in
phase 2 depending on what was used for phase 1. It will still be painful
though.

You will save yourself a lot of trouble if you can move the newer machines
to IKEv2 .. (It would not be possible to run both isakmpd and iked on a
single OpenBSD machine though). Or alternatively wireguard or openvpn
(which _can_ coexist with IKEv1) though IKEv2 generally has a simpler
client config.

> Running a packet capture on the server shows that both of these
> proposals are being sent by the client and should match in isakmpd.
> Alas, they do not.
>
> Here's the pcap -- again, thanks in advance for any assistance.
>
> server# tcpdump -nvvs 16000 -i em0 src host 100.64.10.10 and port 500
> 13:14:50.662540 64:9e:f3:XX:XX:XX 52:54:00:XX:XX:XX 0800 830:
> 100.64.10.10.63823 > 203.0.113.1.500: [udp
>  sum ok] isakmp v1.0 exchange ID_PROT
> cookie: 197d16ba79870fd2-> msgid:  len: 788
> payload: SA len: 516 DOI: 1(IPSEC) situation: IDENTITY_ONLY
> payload: PROPOSAL len: 504 proposal: 1 proto: ISAKMP
> spisz: 0 xforms: 14
> payload: TRANSFORM len: 36
> transform: 1 ID: ISAKMP
> attribute LIFE_TYPE = SECONDS
> attribute LIFE_DURATION = 3600
> attribute ENCRYPTION_ALGORITHM = AES_CBC
> attribute KEY_LENGTH = 256
> attribute AUTHENTICATION_METHOD = PRE_SHARED
> attribute HASH_ALGORITHM = SHA2_256
> attribute GROUP_DESCRIPTION = MODP_2048
> [..]
>payload: TRANSFORM len: 32
> transform: 14 ID: ISAKMP
> attribute LIFE_TYPE = SECONDS
>attribute LIFE_DURATION = 3600
> attribute ENCRYPTION_ALGORITHM = 3DES_CBC
> attribute AUTHENTICATION_METHOD = PRE_SHARED
> attribute HASH_ALGORITHM = MD5
> attribute GROUP_DESCRIPTION = MODP_1024
> [..]
> 13:14:50.667404 52:54:00:XX:XX:XX 64:9e:f3:XX:XX:XX 0800 82:
> 203.0.113.1.500 > 100.64.10.10.63823: [bad udp cksum 1608! -> 2224]
> isakmp v1.0 exchange INFO
> cookie: 3cbfd92c4ea7626d-> msgid:  len: 40
> payload: NOTIFICATION len: 12
> notification: NO PROPOSAL CHOSEN (ttl 64, id 48652, len 68)
> [..]
>
>


-- 
Please keep replies on the mailing list.

Re: IPSec fails with NO_PROPOSAL_CHOSEN when connecting from recent MacOS/iOS clients

[fixied]

Matthew Ernisse writes...

> How are you setting the proposals on the MacOS end?  Your first instance I
> think you figured out that you had not specified PSK and so you had a mismatch
> there.  In the second case you didn't supply the iked(8) debugging information
> so I'm not sure what is happening.  I am also not sure why you have two 
> stanzas
> in ipsec.conf(5) (much less why you are allowing md5/3des).  You should
> probably run iked(8) with debugging cranked up and see what it says, I've 
> found
> it to always tell me why it is unhappy.

I didn't have a mismatch in my PSKs (the full config was further down
in my first message), nor am I running iked. This is just simple l2tp
and everything is handled by isakmpd.

The two stanzas are to test a match on different proposals, but
nothing on the server seems to match proposals chosen by the client.

> I have tunnels between OpenBSD 7.0, iOS/iPadOS 15.3.1, and MacOS 10.15.7.

I'm envious at this point!

Thanks.

Re: IPSec fails with NO_PROPOSAL_CHOSEN when connecting from recent MacOS/iOS clients

[Matthew Ernisse]

On Fri, Feb 18, 2022 at 01:30:27PM -0600, fix...@gmail.com said:
> Date: Fri, 18 Feb 2022 13:30:27 -0600
> From: fix...@gmail.com
> To: misc@openbsd.org
> Subject: Re: IPSec fails with NO_PROPOSAL_CHOSEN when connecting from
>  recent MacOS/iOS clients
> 
> On Fri, Feb 18, 2022 at 11:43 AM I wrote:
> > I recently started seeing some ipsec clients fail on newer versions of
> > MacOS and iOS. After MacOS 12.1, connecting to my head end now fails
> > with NO_PROPOSAL_CHOSEN using mod1024 in my ipsec.conf. I've also

How are you setting the proposals on the MacOS end?  Your first instance I
think you figured out that you had not specified PSK and so you had a mismatch
there.  In the second case you didn't supply the iked(8) debugging information
so I'm not sure what is happening.  I am also not sure why you have two stanzas
in ipsec.conf(5) (much less why you are allowing md5/3des).  You should
probably run iked(8) with debugging cranked up and see what it says, I've found
it to always tell me why it is unhappy.

I have tunnels between OpenBSD 7.0, iOS/iPadOS 15.3.1, and MacOS 10.15.7.

--Matt

-- 
Matthew Ernisse
m...@going-flying.com
https://www.going-flying.com/

Re: IPSec fails with NO_PROPOSAL_CHOSEN when connecting from recent MacOS/iOS clients

[fixied]

On Fri, Feb 18, 2022 at 11:43 AM I wrote:
> I recently started seeing some ipsec clients fail on newer versions of
> MacOS and iOS. After MacOS 12.1, connecting to my head end now fails
> with NO_PROPOSAL_CHOSEN using mod1024 in my ipsec.conf. I've also
> tried, with no success:
>
> main auth "hmac-sha2" enc "aes" group modp1024
> quick auth "hmac-sha2" enc "aes" group modp1024

Doing further research shows things are just not working out.

I added two different configs that should match in /etc/ipsec.conf

ike passive esp transport proto udp from $public_ip to any \
  main auth "hmac-sha2-256" enc "aes-256" group "modp2048" \
  quick auth "hmac-sha2-256" enc "aes-256" group "modp2048" \
  psk "THIS_IS_MY_IPSEC_PASSPHRASE"

ike passive esp transport proto udp from $public_ip to any \
  main auth "hmac-md5" enc "3des" group "modp1024" \
  quick auth "hmac-md5" enc "3des" group "modp1024" \
  psk "THIS_IS_MY_IPSEC_PASSPHRASE"

Running a packet capture on the server shows that both of these
proposals are being sent by the client and should match in isakmpd.
Alas, they do not.

Here's the pcap -- again, thanks in advance for any assistance.

server# tcpdump -nvvs 16000 -i em0 src host 100.64.10.10 and port 500
13:14:50.662540 64:9e:f3:XX:XX:XX 52:54:00:XX:XX:XX 0800 830:
100.64.10.10.63823 > 203.0.113.1.500: [udp
 sum ok] isakmp v1.0 exchange ID_PROT
cookie: 197d16ba79870fd2-> msgid:  len: 788
payload: SA len: 516 DOI: 1(IPSEC) situation: IDENTITY_ONLY
payload: PROPOSAL len: 504 proposal: 1 proto: ISAKMP
spisz: 0 xforms: 14
payload: TRANSFORM len: 36
transform: 1 ID: ISAKMP
attribute LIFE_TYPE = SECONDS
attribute LIFE_DURATION = 3600
attribute ENCRYPTION_ALGORITHM = AES_CBC
attribute KEY_LENGTH = 256
attribute AUTHENTICATION_METHOD = PRE_SHARED
attribute HASH_ALGORITHM = SHA2_256
attribute GROUP_DESCRIPTION = MODP_2048
[..]
   payload: TRANSFORM len: 32
transform: 14 ID: ISAKMP
attribute LIFE_TYPE = SECONDS
   attribute LIFE_DURATION = 3600
attribute ENCRYPTION_ALGORITHM = 3DES_CBC
attribute AUTHENTICATION_METHOD = PRE_SHARED
attribute HASH_ALGORITHM = MD5
attribute GROUP_DESCRIPTION = MODP_1024
[..]
13:14:50.667404 52:54:00:XX:XX:XX 64:9e:f3:XX:XX:XX 0800 82:
203.0.113.1.500 > 100.64.10.10.63823: [bad udp cksum 1608! -> 2224]
isakmp v1.0 exchange INFO
cookie: 3cbfd92c4ea7626d-> msgid:  len: 40
payload: NOTIFICATION len: 12
notification: NO PROPOSAL CHOSEN (ttl 64, id 48652, len 68)
[..]

IPSec fails with NO_PROPOSAL_CHOSEN when connecting from recent MacOS/iOS clients

[fixied]

I recently started seeing some ipsec clients fail on newer versions of
MacOS and iOS. After MacOS 12.1, connecting to my head end now fails
with NO_PROPOSAL_CHOSEN using mod1024 in my ipsec.conf. I've also
tried, with no success:

main auth "hmac-sha2" enc "aes" group modp1024
quick auth "hmac-sha2" enc "aes" group modp1024

Has anyone gotten ipsec working between recent versions of Apple
devices? If so, could you share your proposals and/or configs?

Below are my configs -- thanks in advance for any help.

macbook-client:~$ uname -a
Darwin neptune.example.org 21.2.0 Darwin Kernel Version 21.2.0: Sun
Nov 28 20:28:54 PST 2021; root:xnu-8019.61.5~1/RELEASE_X86_64 x86_64

server# uname -a
OpenBSD lax 7.0 GENERIC#5 amd64

# tail -f /var/log/messages
Feb 18 11:11:31 server isakmpd[10385]: attribute_unacceptable:
ENCRYPTION_ALGORITHM: got AES_CBC, expected 3DES_CBC
Feb 18 11:11:31 server last message repeated 11 times
Feb 18 11:11:31 server isakmpd[10385]:
attribute_unacceptable:AUTHENTICATION_METHOD: got PRE_SHARED, expected
RSA_SIG
Feb 18 11:11:31 server isakmpd[10385]: attribute_unacceptable:
AUTHENTICATION_METHOD: got PRE_SHARED, expected RSA_SIG
Feb 18 11:11:31 server isakmpd[10385]: message_negotiate_sa: no
compatible proposal found
Feb 18 11:11:31 server isakmpd[10385]: dropped message from
100.64.10.10 port 63434 due to notification type NO_PROPOSAL_CHOSEN

# cat /etc/ipsec.conf
public_ip = "203.0.113.1"
ike passive esp tunnel proto udp from $public_ip to any \
  main group "modp1024" quick group "modp1024" \
  psk "THIS_IS_MY_IPSEC_PASSPHRASE"

# ipssecctl -vnf /etc/ipsec.conf
public_ip = "203.0.113.1"
C set [Phase 1]:Default=peer-default force
C set [peer-default]:Phase=1 force
C set [peer-default]:Authentication=THIS_IS_MY_IPSEC_PASSPHRASE force
C set [peer-default]:Configuration=phase1-peer-default force
C set [phase1-peer-default]:EXCHANGE_TYPE=ID_PROT force
C add 
[phase1-peer-default]:Transforms=phase1-transform-peer-default-PRE_SHARED-SHA-AES128-MODP_1024
force
C set 
[phase1-transform-peer-default-PRE_SHARED-SHA-AES128-MODP_1024]:AUTHENTICATION_METHOD=PRE_SHARED
force
C set 
[phase1-transform-peer-default-PRE_SHARED-SHA-AES128-MODP_1024]:HASH_ALGORITHM=SHA
force
C set 
[phase1-transform-peer-default-PRE_SHARED-SHA-AES128-MODP_1024]:ENCRYPTION_ALGORITHM=AES_CBC
force
C set 
[phase1-transform-peer-default-PRE_SHARED-SHA-AES128-MODP_1024]:KEY_LENGTH=128,128:256
force
C set 
[phase1-transform-peer-default-PRE_SHARED-SHA-AES128-MODP_1024]:GROUP_DESCRIPTION=MODP_1024
force
C set 
[phase1-transform-peer-default-PRE_SHARED-SHA-AES128-MODP_1024]:Life=LIFE_MAIN_MODE
force
C set [from-203.0.113.1=17-to-0.0.0.0/0=17]:Phase=2 force
C set [from-203.0.113.1=17-to-0.0.0.0/0=17]:ISAKMP-peer=peer-default force
C set 
[from-203.0.113.1=17-to-0.0.0.0/0=17]:Configuration=phase2-from-203.0.113.1=17-to-0.0.0.0/0=17
force
C set [from-203.0.113.1=17-to-0.0.0.0/0=17]:Local-ID=from-203.0.113.1=17 force
C set [from-203.0.113.1=17-to-0.0.0.0/0=17]:Remote-ID=to-0.0.0.0/0=17 force
C set [phase2-from-203.0.113.1=17-to-0.0.0.0/0=17]:EXCHANGE_TYPE=QUICK_MODE
force
C set 
[phase2-from-203.0.113.1=17-to-0.0.0.0/0=17]:Suites=phase2-suite-from-203.0.113.1=17-to-0.0.0.0/0=17
force
C set 
[phase2-suite-from-203.0.113.1=17-to-0.0.0.0/0=17]:Protocols=phase2-protocol-from-203.0.113.1=17-to-0.0.0.0/0=17
force
C set 
[phase2-protocol-from-203.0.113.1=17-to-0.0.0.0/0=17]:PROTOCOL_ID=IPSEC_ESP
force
C set 
[phase2-protocol-from-203.0.113.1=17-to-0.0.0.0/0=17]:Transforms=phase2-transform-from-203.0.113.1=17-to-0.0.0.0/0=17-AES128-SHA
C set 
[phase2-transform-from-203.0.113.1=17-to-0.0.0.0/0=17-AES128-SHA2_256-MODP_1024-TUNNEL]:TRANSFORM_ID=AES
force
C set 
[phase2-transform-from-203.0.113.1=17-to-0.0.0.0/0=17-AES128-SHA2_256-MODP_1024-TUNNEL]:KEY_LENGTH=128,128:256
force
C set 
[phase2-transform-from-203.0.113.1=17-to-0.0.0.0/0=17-AES128-SHA2_256-MODP_1024-TUNNEL]:ENCAPSULATION_MODE=TUNNEL
force
C set 
[phase2-transform-from-203.0.113.1=17-to-0.0.0.0/0=17-AES128-SHA2_256-MODP_1024-TUNNEL]:AUTHENTICATION_ALGORITHM=HMAC_SHA2_256
f
C set 
[phase2-transform-from-203.0.113.1=17-to-0.0.0.0/0=17-AES128-SHA2_256-MODP_1024-TUNNEL]:GROUP_DESCRIPTION=MODP_1024
force
C set 
[phase2-transform-from-203.0.113.1=17-to-0.0.0.0/0=17-AES128-SHA2_256-MODP_1024-TUNNEL]:Life=LIFE_QUICK_MODE
force
C set [from-203.0.113.1=17]:ID-type=IPV4_ADDR force
C set [from-203.0.113.1=17]:Address=203.0.113.1 force
C set [to-0.0.0.0/0=17]:ID-type=IPV4_ADDR_SUBNET force
C set [to-0.0.0.0/0=17]:Network=0.0.0.0 force
C set [to-0.0.0.0/0=17]:Netmask=0.0.0.0 force
C set [from-203.0.113.1=17]:Protocol=17 force
C set [to-0.0.0.0/0=17]:Protocol=17 force
C add [Phase 2]:Passive-Connections=from-203.0.113.1=17-to-0.0.0.0/0=17

Re: ipsec with default route and routing of internal networks

[Hrvoje Popovski]

On 14.9.2021. 13:12, Hrvoje Popovski wrote:
> On 13.9.2021. 15:52, Stuart Henderson wrote:
>> On 2021-09-13, Hrvoje Popovski  wrote:
>>> On 13.9.2021. 14:08, Tom Smyth wrote:
 Can you do  an exception for the ranges ...  so internet - private ips
 you dont want over the tunnel)

 ike esp from 10.90.0.0/24  to any encrypt  
 and 

  10.90.0.0/24  to   NOT  [networks you dont want
 over the tunnel)  ? 

>>>
>>> :) this was the first thought that i've had ... but i couldn't find how
>>> to do it ... at least in man ipsec.conf or isakmpd.conf
>>>
>>>
>>
>> You do this with a "bypass flow" in /etc/ipsec.conf:
>>
>> flow from $network/$prefix to $network/$prefix type bypass
>>
>> and loading it with ipsecctl. Note if you use iked, you cannot configure
>> this directly in iked.conf, but you can still use ipsecctl and ipsec.conf
>> for this purpose in conjunction with iked for tunnel setup.
>>
>>
> 
> Thank you guys ... with "type bypass" everything is working as expected
> 
> c/p from config
> ike esp from 10.90.0.0/24 to any \
> local $localip peer $peerip \
> main auth hmac-sha1 enc aes group modp1024 \
> quick enc aes-128-gcm group modp1024 \
> psk 123
> flow from 10.90.0.0/24 to 10.90.0.0/24 type bypass
> flow from 10.90.0.0/24 to 10.91.0.0/24 type bypass
> flow from 10.90.0.0/24 to 10.92.0.0/24 type bypass
> 

and if you have carp (multicast) than you need
flow from 10.90.0.0/24 to 224.0.0.18/32 type bypass

Re: ipsec with default route and routing of internal networks

[Hrvoje Popovski]

On 13.9.2021. 15:52, Stuart Henderson wrote:
> On 2021-09-13, Hrvoje Popovski  wrote:
>> On 13.9.2021. 14:08, Tom Smyth wrote:
>>> Can you do  an exception for the ranges ...  so internet - private ips
>>> you dont want over the tunnel)
>>>
>>> ike esp from 10.90.0.0/24  to any encrypt  
>>> and 
>>>
>>>  10.90.0.0/24  to   NOT  [networks you dont want
>>> over the tunnel)  ? 
>>>
>>
>> :) this was the first thought that i've had ... but i couldn't find how
>> to do it ... at least in man ipsec.conf or isakmpd.conf
>>
>>
> 
> You do this with a "bypass flow" in /etc/ipsec.conf:
> 
> flow from $network/$prefix to $network/$prefix type bypass
> 
> and loading it with ipsecctl. Note if you use iked, you cannot configure
> this directly in iked.conf, but you can still use ipsecctl and ipsec.conf
> for this purpose in conjunction with iked for tunnel setup.
> 
> 

Thank you guys ... with "type bypass" everything is working as expected

c/p from config
ike esp from 10.90.0.0/24 to any \
local $localip peer $peerip \
main auth hmac-sha1 enc aes group modp1024 \
quick enc aes-128-gcm group modp1024 \
psk 123
flow from 10.90.0.0/24 to 10.90.0.0/24 type bypass
flow from 10.90.0.0/24 to 10.91.0.0/24 type bypass
flow from 10.90.0.0/24 to 10.92.0.0/24 type bypass




ipsecctl -sa | grep 10.9
flow esp in from 0.0.0.0/0 to 10.90.0.0/24 peer $peerip srcid $localip
dstid $peerip type require
flow esp in from 10.90.0.0/24 to 10.90.0.0/24 type bypass
flow esp in from 10.91.0.0/24 to 10.90.0.0/24 type bypass
flow esp in from 10.92.0.0/24 to 10.90.0.0/24 type bypass

flow esp out from 10.90.0.0/24 to 0.0.0.0/0 peer $peerip srcid $localip
dstid $peerip type require
flow esp out from 10.90.0.0/24 to 10.90.0.0/24 type bypass
flow esp out from 10.90.0.0/24 to 10.91.0.0/24 type bypass
flow esp out from 10.90.0.0/24 to 10.92.0.0/24 type bypass

Re: ipsec with default route and routing of internal networks

[Stuart Henderson]

On 2021-09-13, Hrvoje Popovski  wrote:
> On 13.9.2021. 14:08, Tom Smyth wrote:
>> Can you do  an exception for the ranges ...  so internet - private ips
>> you dont want over the tunnel)
>> 
>> ike esp from 10.90.0.0/24  to any encrypt  
>> and 
>> 
>>  10.90.0.0/24  to   NOT  [networks you dont want
>> over the tunnel)  ? 
>> 
>
>:) this was the first thought that i've had ... but i couldn't find how
> to do it ... at least in man ipsec.conf or isakmpd.conf
>
>

You do this with a "bypass flow" in /etc/ipsec.conf:

flow from $network/$prefix to $network/$prefix type bypass

and loading it with ipsecctl. Note if you use iked, you cannot configure
this directly in iked.conf, but you can still use ipsecctl and ipsec.conf
for this purpose in conjunction with iked for tunnel setup.

Re: ipsec with default route and routing of internal networks

[Hrvoje Popovski]

On 13.9.2021. 14:08, Tom Smyth wrote:
> Can you do  an exception for the ranges ...  so internet - private ips
> you dont want over the tunnel)
> 
> ike esp from 10.90.0.0/24  to any encrypt  
> and 
> 
>  10.90.0.0/24  to   NOT  [networks you dont want
> over the tunnel)  ? 
> 

:) this was the first thought that i've had ... but i couldn't find how
to do it ... at least in man ipsec.conf or isakmpd.conf

Re: ipsec with default route and routing of internal networks

[Tom Smyth]

Can you do  an exception for the ranges ...  so internet - private ips you
dont want over the tunnel)

ike esp from 10.90.0.0/24 to any encrypt
and

 10.90.0.0/24 to   NOT  [networks you dont want over the tunnel)  ?

On Mon, 13 Sept 2021 at 13:02, Hrvoje Popovski  wrote:

> Hi,
>
> On 13.9.2021. 12:58, Tom Smyth wrote:
> > Hi Hrvoje,
> >
> > is 10.90.0.0/24 <http://10.90.0.0/24> local to your firewall, and if I
> > understand your rule,
> > ike esp from 10.90.0.0/24 <http://10.90.0.0/24> to anyyou are
> saying
> > encrypt all traffic comming from 10.90.0.0/24 <http://10.90.0.0/24>
> >
> > should the tunnel be more specific ? like
> >
> > from 10.90.0.0/24 <http://10.90.0.0/24>  to another network across the
> > tunnel
> >
>
> 10.90/24 is my local internal network, as other networks (10.91/24,
> 10.92/24).
> i need "ike esp from 10.90.0.0/24 to any"... because hosts on that
> network need to go out to internet over ipsec tunnel ... but at the same
> time hosts in that 10.90/24 network needs to communicate to other
> internal networks...
>


-- 
Kindest regards,
Tom Smyth.

Re: ipsec with default route and routing of internal networks

[Hrvoje Popovski]

Hi,

On 13.9.2021. 12:58, Tom Smyth wrote:
> Hi Hrvoje, 
> 
> is 10.90.0.0/24 <http://10.90.0.0/24> local to your firewall, and if I
> understand your rule,
> ike esp from 10.90.0.0/24 <http://10.90.0.0/24> to any    you are saying  
> encrypt all traffic comming from 10.90.0.0/24 <http://10.90.0.0/24> 
> 
> should the tunnel be more specific ? like 
> 
> from 10.90.0.0/24 <http://10.90.0.0/24>  to another network across the
> tunnel  
> 

10.90/24 is my local internal network, as other networks (10.91/24,
10.92/24).
i need "ike esp from 10.90.0.0/24 to any"... because hosts on that
network need to go out to internet over ipsec tunnel ... but at the same
time hosts in that 10.90/24 network needs to communicate to other
internal networks...

Re: ipsec with default route and routing of internal networks

[Tom Smyth]

Hi Hrvoje,

is 10.90.0.0/24 local to your firewall, and if I understand your rule,
ike esp from 10.90.0.0/24 to anyyou are saying
encrypt all traffic comming from 10.90.0.0/24

should the tunnel be more specific ? like

from 10.90.0.0/24  to another network across the tunnel

ike esp from 10.90.0.0/24 to  {list of private network ranges that are
across the tunnel}

(remove any and replace with specific subnets to be routed across the Ipsec
tunnel)

without a diagram I cant help much more...


On Mon, 13 Sept 2021 at 11:36, Hrvoje Popovski  wrote:

> Hi all,
>
> I have a firewall that routes few internal networks, 10.90/24, 10.91/24,
> 10.92/24. And i have some static routes to other firewalls, but i don't
> think that is relevant to this problem.
>
> For network 10.90/24 i have ipsec tunnel, and i need to push any traffic
> from that network to the internet, but not to local networks,
> over that ipsec tunnel.
>
> something like this:
> ike esp from 10.90.0.0/24 to any
>
> I thought that the routing table will take care of that, but i seems
> that when ipsec tunnel is up, i can't connect from local networks
> (10.91/24, 10.92/24) to 10.90/24 and I can't even ping hosts on the
> 10.90/24 network ...
> something like this ping -I 10.90.0.1 10.90.0.8 ...
> traffic from 10.90/24 to the internet is working just fine ..
>
> I need to make network 10.90/24 reachable to all local networks.
> Could someone please point me in the right direction on what to look and
> configure?
>
> Thank you ..
>
>

-- 
Kindest regards,
Tom Smyth.

ipsec with default route and routing of internal networks

[Hrvoje Popovski]

Hi all,

I have a firewall that routes few internal networks, 10.90/24, 10.91/24,
10.92/24. And i have some static routes to other firewalls, but i don't
think that is relevant to this problem.

For network 10.90/24 i have ipsec tunnel, and i need to push any traffic
from that network to the internet, but not to local networks,
over that ipsec tunnel.

something like this:
ike esp from 10.90.0.0/24 to any

I thought that the routing table will take care of that, but i seems
that when ipsec tunnel is up, i can't connect from local networks
(10.91/24, 10.92/24) to 10.90/24 and I can't even ping hosts on the
10.90/24 network ...
something like this ping -I 10.90.0.1 10.90.0.8 ...
traffic from 10.90/24 to the internet is working just fine ..

I need to make network 10.90/24 reachable to all local networks.
Could someone please point me in the right direction on what to look and
configure?

Thank you ..

OpenSMTPD priority and IPSEC relay distribution

[Riccardo Giuntoli]

Hello there. I've got some domains that I want to point to a bounce of MX
all with the same priorities. Once one of them receive the message I want
that it resend it relaying to all the others in a IPSEC GRE network.

What is the correct configuration of smtpd.conf?

Nice regards,

-- 
Name: Riccardo Giuntoli
Email: tag...@gmail.com
Location: sant Pere de Ribes, BCN, Spain
PGP Key: 0x67123739
PGP Fingerprint: CE75 16B5 D855 842FAB54 FB5C DDC6 4640 6712 3739
Key server: hkp://wwwkeys.eu.pgp.net

Re: Can't connect to IKE1 VPN Server via OpenBsd 6.8 with IPSEC/L2TP

[Marko Bauhardt]

> Marko Bauhardt  hat am 31.12.2020 00:05 
> geschrieben:
> 
>  
> Hi,
> I have a dell xps laptop with OpenBsd 6.8 running. I want to connect to an 
> IKEv1 L2TP VPN Server.
> 
> I followed the steps on https://www.openbsd.org/faq/faq17.html#clientikev1
> and /usr/local/share/doc/pkg-readmes/xl2tpd
> 
> I created the following config files

I'm now able to connect. I believe I did some misconfiguration.
Just for closing this thread, here are the configs

/etc/ipsec.conf
ike dynamic esp transport proto udp from egress to vpn_server port l2tp  \
  main auth "hmac-sha1" enc "aes-128" group modp2048  \  
  quick auth "hmac-sha1" enc "aes-128" \ 
  psk pre-shared-secret

/etc/xl2tpd/xl2tpd.conf
[global]
debug avp = yes
debug network = yes
debug state = yes
debug tunnel = yes
port = 1701

[lac office]
lns = vpn_server
ppp debug = yes
pppoptfile = /etc/ppp/options.office

/etc/ppp/options.office
ipcp-accept-local
ipcp-accept-remote
noccp
noauth
mtu 1400
mru 1400
debug
lock
user my_username
netmask 255.255.255.255

/etc/ppp/pap-secrets
my_username * my_pwd


Marko

Can't connect to IKE1 VPN Server via OpenBsd 6.8 with IPSEC/L2TP

[Marko Bauhardt]

Hi,
I have a dell xps laptop with OpenBsd 6.8 running. I want to connect to an 
IKEv1 L2TP VPN Server.

I followed the steps on https://www.openbsd.org/faq/faq17.html#clientikev1
and /usr/local/share/doc/pkg-readmes/xl2tpd

I created the following config files

/etc/ipsec.conf

ike esp from $IP1 to $IP2 peer $VPNSERVER \
  main auth hmac-sha1 enc aes-128 group modp2048 \
  quick auth hmac-sha1 enc aes-128 \
  psk my-pre-shared-secret


/etc/xl2tpd/x2ltpd.conf
==
[global]
debug avp = yes
debug network = yes
debug state = yes
debug tunnel = yes
auth file = /etc/ppp/pap-secrets
port = 1701

[lac l2tp]
lns = vpn_server_ip
ppp debug = yes
pppoptfile = /etc/ppp/options.l2tp
require authentication = yes
require pap = yes
require chap = no
length bit = yes


/etc/ppp/options.l2tp

ipcp-accept-local
ipcp-accept-remote
refuse-eap
refuse-mschap-v2
noccp
noauth
idle 1800
mtu 1410
mru 1410
connect-delay 5000
usepeerdns
defaultroute
debug
lock
netmask 255.255.255.0
user myuser
password mypwd


/etc/ppp/pap-secrets
myuser * mypwd *


I added an interface ppp0. and started isakmpd, xl2tpd
ipsecctl -sa show flows and SAD's
But, when i try to connect via 
'echo c l2tp | doas tee /var/run/xl2tpd/l2tp-control'
the /var/log/daemon show only

Dec 30 23:47:20 2147NFS xl2tpd[1160]: Connecting to host $VPNSERVER, port 1701
Dec 30 23:47:51 2147NFS xl2tpd[1160]: Maximum retries exceeded for tunnel 113.  
Closing.
Dec 30 23:47:51 2147NFS xl2tpd[1160]: Connection 0 closed to VPNSERVERIP, port 
1701 (Timeout)

I would expect to see more logging, but there is no pppd logging. Looks like 
the process won't start. Is this maybe the issue here?
Any hint how I can enable more logging? Or do you see any mistake in my config 
pasted above.

Thanks
Marko

iked vs IPsec failover (carp & sasyncd)

[Harald Dunkel]

Hi folks,

wrt IPsec failover via sasyncd and carp: sasyncd(8) and iked(8) don't
seem to tell, but I would guess that all hosts on the carp interface
have to share the private key to support renegotiation.

How can I tell iked which private key to use, instead of local.key?
Is there a similar naming scheme as for the foreign public keys?

Every insightful comment is highly appreciated
Harri

Re: IPsec and MTU / fragmentation

[Brian Brombacher]

> On Oct 30, 2020, at 11:44 AM, Brian Brombacher  wrote:
> 
> 
> 
>>> On Oct 29, 2020, at 11:56 PM, David Diggles  wrote:
>>> 
>>> On Mon, Feb 10, 2020 at 05:15:00PM +, Peter M??ller wrote:
>>> Hello Lucas,
>>> 
>>> as far as I understood, setting MTU on encN interfaces is not supported
>>> since it is not mentioned by enc(4) and setting it manually fails:
>>> 
>>>> machine# ifconfig enc0 mtu 1500
>>>> ifconfig: SIOCSIFMTU: Inappropriate ioctl for device
>>> 
>>> If you do not want to use GRE tunnels or gif interfaces, I suppose 
>>> truncating
>>> MSS via pf might be an acceptable but not elegant solution:
>> 
>> I have max-mss and reassemble tcp:
>> 
>> match in on gre0 scrub (max-mss 1456, reassemble tcp)
>> 
> 
> How did you calculate the max-mss?  It seems too high for a double tunnel 
> setup.

Also, sorry for double post, you need the match rule on enc0 to impact TCP 
streams going over IPSec to change their mss.  I don’t have the old emails for 
this thread, so not sure if IPSec is your outer tunnel or inner here.

> 
>> However still experienced about 5% packet loss when i run speedtest.net 
>> through
>> the tunnel.
>> 
>> In my instance, the solution for eliminating packet loss over the long 
>> distance
>> ipsec/gre tunnel was putting in a queue:
>> 
>> queue hfsq-gre0 on gre0 flows 1024 bandwidth $BW_LIMIT max $BW_LIMIT quantum 
>> 400 qlimit 1000 default
>> 
>> .d.d.
>> 

Re: IPsec and MTU / fragmentation

[Brian Brombacher]

> On Oct 29, 2020, at 11:56 PM, David Diggles  wrote:
> 
> On Mon, Feb 10, 2020 at 05:15:00PM +, Peter M??ller wrote:
>> Hello Lucas,
>> 
>> as far as I understood, setting MTU on encN interfaces is not supported
>> since it is not mentioned by enc(4) and setting it manually fails:
>> 
>>> machine# ifconfig enc0 mtu 1500
>>> ifconfig: SIOCSIFMTU: Inappropriate ioctl for device
>> 
>> If you do not want to use GRE tunnels or gif interfaces, I suppose truncating
>> MSS via pf might be an acceptable but not elegant solution:
> 
> I have max-mss and reassemble tcp:
> 
> match in on gre0 scrub (max-mss 1456, reassemble tcp)
> 

How did you calculate the max-mss?  It seems too high for a double tunnel setup.

> However still experienced about 5% packet loss when i run speedtest.net 
> through
> the tunnel.
> 
> In my instance, the solution for eliminating packet loss over the long 
> distance
> ipsec/gre tunnel was putting in a queue:
> 
> queue hfsq-gre0 on gre0 flows 1024 bandwidth $BW_LIMIT max $BW_LIMIT quantum 
> 400 qlimit 1000 default
> 
> .d.d.
> 

Re: IPsec and MTU / fragmentation

[David Diggles]

On Mon, Feb 10, 2020 at 05:15:00PM +, Peter M??ller wrote:
> Hello Lucas,
> 
> as far as I understood, setting MTU on encN interfaces is not supported
> since it is not mentioned by enc(4) and setting it manually fails:
> 
> > machine# ifconfig enc0 mtu 1500
> > ifconfig: SIOCSIFMTU: Inappropriate ioctl for device
> 
> If you do not want to use GRE tunnels or gif interfaces, I suppose truncating
> MSS via pf might be an acceptable but not elegant solution:

I have max-mss and reassemble tcp:

match in on gre0 scrub (max-mss 1456, reassemble tcp)

However still experienced about 5% packet loss when i run speedtest.net through
the tunnel.

In my instance, the solution for eliminating packet loss over the long distance
ipsec/gre tunnel was putting in a queue:

queue hfsq-gre0 on gre0 flows 1024 bandwidth $BW_LIMIT max $BW_LIMIT quantum 
400 qlimit 1000 default

.d.d.

Re: question about IPsec

[Stuart Henderson]

On 2020-08-15, Riccardo Giuntoli  wrote:
> Hello there nice people.
>
> It's possible have in the same machine IKEv2 and IKEv1 running?

Not with iked/isakmpd, they conflict on the kernel interface for adding
ipsec information.  Possibly with strongswan.

> How can I open IKEv2 socket only on an IP or an interface?
>
> Perhaps with different routing tables?

Maybe with different routing tables but I'm not sure. Otherwise there is
no "bind to X ip" option.

question about IPsec

[Riccardo Giuntoli]

Hello there nice people.

It's possible have in the same machine IKEv2 and IKEv1 running?

How can I open IKEv2 socket only on an IP or an interface?

Perhaps with different routing tables?

Nice regards

-- 
Name: Riccardo Giuntoli
Email: tag...@gmail.com
Location: sant Pere de Ribes, BCN, Spain
PGP Key: 0x67123739
PGP Fingerprint: CE75 16B5 D855 842FAB54 FB5C DDC6 4640 6712 3739
Key server: hkp://wwwkeys.eu.pgp.net

Re: IPSec heavy traffic slows down all network traffic

[jean-yves boisiaud]

Hello, i replaced the MP kernel with the SP one and made some tests.

Perfomances are better, all cpu goes to the kernel and user processes. But
it is slow. I will ask to change the hardware, as it is old.

jy boisiaud

Le mer. 22 juil. 2020 à 08:36, jean-yves boisiaud <
jean-yves.boisi...@alcor-consulting.fr> a écrit :

> ok, i'll try with the bsd.sp kernel.
>
> thank you for your help.
>
> :-(
>
>
> Le dim. 19 juil. 2020 à 07:41, Chris Cappuccio  a
> écrit :
>
>> jean-yves boisiaud [jean-yves.boisi...@alcor-consulting.fr] wrote:
>> > Last week, I upgraded a couple of firewalls using carp/pfsync and
>> sasyncd
>> > from 6.0 to 6.7 (yes, big jump !).
>> >
>> > I also applied all the 6.7 published patches.
>> >
>> > When some heavy traffic takes one of the IPSec tunnel, I noticed that :
>> > - all network connections are slowed down
>> > - unused network bandwidth increase instead of decrease
>> > - idle CPU move towards 0, and spinning increase to take about 50% of
>> the
>> > CPU
>> >
>> > When I stop the IPSec traffic :
>> > - network connections increase immediatly
>> > - unused network bandwidth cecreases immediately
>> > - spinning CPU is low.
>> >
>>
>> This is basically a performance regression that could be due to the MP
>> work. You are seemingly running into contention that wasn't possible
>> before.
>> The question is, where is this happening? I don't know if the dynamic
>> tracer
>> can help here.
>>
>
>
> --
> Jean-Yves Boisiaud - Alcor Consulting
> 49, rue du Chemin Vert
> 49300 Cholet
> mobile : +33 6 63 71 73 46
>

Re: IPSec heavy traffic slows down all network traffic

[jean-yves boisiaud]

ok, i'll try with the bsd.sp kernel.

thank you for your help.

:-(


Le dim. 19 juil. 2020 à 07:41, Chris Cappuccio  a écrit :

> jean-yves boisiaud [jean-yves.boisi...@alcor-consulting.fr] wrote:
> > Last week, I upgraded a couple of firewalls using carp/pfsync and sasyncd
> > from 6.0 to 6.7 (yes, big jump !).
> >
> > I also applied all the 6.7 published patches.
> >
> > When some heavy traffic takes one of the IPSec tunnel, I noticed that :
> > - all network connections are slowed down
> > - unused network bandwidth increase instead of decrease
> > - idle CPU move towards 0, and spinning increase to take about 50% of the
> > CPU
> >
> > When I stop the IPSec traffic :
> > - network connections increase immediatly
> > - unused network bandwidth cecreases immediately
> > - spinning CPU is low.
> >
>
> This is basically a performance regression that could be due to the MP
> work. You are seemingly running into contention that wasn't possible
> before.
> The question is, where is this happening? I don't know if the dynamic
> tracer
> can help here.
>


-- 
Jean-Yves Boisiaud - Alcor Consulting
49, rue du Chemin Vert
49300 Cholet
mobile : +33 6 63 71 73 46

Re: l2ip + ipsec question

[kasak]

21.07.2020 11:43, Stuart Henderson пишет:


most endpoints cope wigh slightly less terrible crypto, you can try
something like

ike passive esp transport \
 proto udp from my.external.ip to any port 1701 \
 main auth "hmac-sha1" enc "aes-256" group modp2048 \
 quick auth "hmac-sha2-256" enc "aes-256" \
 psk "0s5jTDcMziOVw3DXZqaGOVlEZyoe8I9c"

(psk generated randomly from "openssl rand -base64 (length)", use
something complex if you can copy-and-paste to the other devices)


Yep, mod2048 works, thanks!

2) ipsec.conf man, says that "esp" is default. But if I omit this
option, it stops working with error like: PAYLOAD_MALFORMED.

3) and the most difficult for me to understand: Why does all howto's use
this fragment:

proto udp from my.ga.te.ip to any port 1701 ??

the ipsec.conf man says: from src [port sport] [(srcnat)] to dst [port
dport]

so, this line declare a tunnel, where our gate use any port, and our
expected remote client use port 1701?? why does this even work?

Thank you in advance for help!




It relies on the fact that l2tp uses a fixed source port, iirc you can
use "from my.gate.ip port 1701 to any port 1701" if you want.

btw I strongly recommend avoiding l2tp+ipsec if you have another choice.
Plain ipsec (ikev1 or ikev2) or other protocols like wireguard/openvpn
cope better if you end up on a natted network.


i'm sorry but i still do not understand. I have fired up tcpdump on enc0

and what's that I see there:

12:20:01.791795 (authentic,confidential): SPI 0x0e3e51b6: 
212.233.112.12.l2tp > mx.kasakoff.net.59516: 
l2tp:[LS](14/9936)Ns=13,Nr=65535[hdlc|][|l2tp]
12:20:01.894911 (authentic,confidential): SPI 0x0e3e51b6: 
212.233.112.12.l2tp > mx.kasakoff.net.59516: 
l2tp:[LS](14/9936)Ns=14,Nr=65535[hdlc|][|l2tp]
12:20:05.066256 (authentic,confidential): SPI 0xd5815d86: 
mx.kasakoff.net.59516 > 212.233.112.12.l2tp: l2tp:[L](83/7415)[hdlc|][|l2tp]
12:20:06.073233 (authentic,confidential): SPI 0xd5815d86: 
mx.kasakoff.net.59516 > 212.233.112.12.l2tp: l2tp:[L](83/7415)[hdlc|][|l2tp]


Here, 212.233.112.12 is my gateway ip, and mx.kasakoff.net is the client.

As I can see, the client side does not use 1701 port.

But either

"from 212.233.112.12 port l2tp to any"

or

"from 212.233.112.12 to any port l2tp" works!

I can't fully understand why.

Re: l2ip + ipsec question

[Stuart Henderson]

On 2020-07-20, kasak  wrote:
> Hello misc.
> Recently, i needed to setup l2tp-ipsec for some ip phones to reach my 
> network.
>
> so, the l2tp part is not trouble at all with npppd, but, the ipsec part 
> is harder to understand.
>
> after reading ipsec and ipsec.conf man,
>
> i tryed to add just one line:
>
> ike passive from my.ga.te.ip to any psk "mykey"
>
> but this didn't work.
>
> after some googling, i have found this line:
>
> ike passive esp transport \
>  proto udp from 1.2.3.4 to any port 1701 \
>  main auth "hmac-sha1" enc "3des" group modp1024 \
>  quick auth "hmac-sha1" enc "aes" \
>  psk "password"
>
> it was found on undeadly.org

most endpoints cope wigh slightly less terrible crypto, you can try
something like

ike passive esp transport \
proto udp from my.external.ip to any port 1701 \
main auth "hmac-sha1" enc "aes-256" group modp2048 \
quick auth "hmac-sha2-256" enc "aes-256" \
psk "0s5jTDcMziOVw3DXZqaGOVlEZyoe8I9c"

(psk generated randomly from "openssl rand -base64 (length)", use
something complex if you can copy-and-paste to the other devices)

> I need help to understand how it even works.
>
> 1) why does somebody use "transport" here and somebody use "tunnel"? I 
> myself tryed "transport" and it works. than, what is the difference for 
> l2tp?

"tunnel" adds an extra header to the packet carrying src/dest addresses
so ipsec can directly protect packets from other machines.

"transport" doesn't have the extra header so ipsec can only carry
packets from endpoint to endpoint - but this reduces overheads and
increases max usable packet size. the "endpoint-endpoint" traffic can
itself be a tunnel as is the case with l2tp.

the usual setup for l2tp+ipsec has transport mode to reduce overheads.
(both ends need to be set the same way).

> 2) ipsec.conf man, says that "esp" is default. But if I omit this 
> option, it stops working with error like: PAYLOAD_MALFORMED.
>
> 3) and the most difficult for me to understand: Why does all howto's use 
> this fragment:
>
> proto udp from my.ga.te.ip to any port 1701 ??
>
> the ipsec.conf man says: from src [port sport] [(srcnat)] to dst [port 
> dport]
> 
> so, this line declare a tunnel, where our gate use any port, and our 
> expected remote client use port 1701?? why does this even work?
>
> Thank you in advance for help!
>
>
>

It relies on the fact that l2tp uses a fixed source port, iirc you can
use "from my.gate.ip port 1701 to any port 1701" if you want.

btw I strongly recommend avoiding l2tp+ipsec if you have another choice.
Plain ipsec (ikev1 or ikev2) or other protocols like wireguard/openvpn
cope better if you end up on a natted network.

l2ip + ipsec question

[kasak]

Hello misc.
Recently, i needed to setup l2tp-ipsec for some ip phones to reach my 
network.


so, the l2tp part is not trouble at all with npppd, but, the ipsec part 
is harder to understand.


after reading ipsec and ipsec.conf man,

i tryed to add just one line:

ike passive from my.ga.te.ip to any psk "mykey"

but this didn't work.

after some googling, i have found this line:

ike passive esp transport \
proto udp from 1.2.3.4 to any port 1701 \
main auth "hmac-sha1" enc "3des" group modp1024 \
quick auth "hmac-sha1" enc "aes" \
psk "password"

it was found on undeadly.org

I need help to understand how it even works.

1) why does somebody use "transport" here and somebody use "tunnel"? I 
myself tryed "transport" and it works. than, what is the difference for 
l2tp?


2) ipsec.conf man, says that "esp" is default. But if I omit this 
option, it stops working with error like: PAYLOAD_MALFORMED.


3) and the most difficult for me to understand: Why does all howto's use 
this fragment:


proto udp from my.ga.te.ip to any port 1701 ??

the ipsec.conf man says: from src [port sport] [(srcnat)] to dst [port 
dport]


so, this line declare a tunnel, where our gate use any port, and our 
expected remote client use port 1701?? why does this even work?


Thank you in advance for help!

Re: IPSec heavy traffic slows down all network traffic

[Chris Cappuccio]

jean-yves boisiaud [jean-yves.boisi...@alcor-consulting.fr] wrote:
> Last week, I upgraded a couple of firewalls using carp/pfsync and sasyncd
> from 6.0 to 6.7 (yes, big jump !).
> 
> I also applied all the 6.7 published patches.
> 
> When some heavy traffic takes one of the IPSec tunnel, I noticed that :
> - all network connections are slowed down
> - unused network bandwidth increase instead of decrease
> - idle CPU move towards 0, and spinning increase to take about 50% of the
> CPU
> 
> When I stop the IPSec traffic :
> - network connections increase immediatly
> - unused network bandwidth cecreases immediately
> - spinning CPU is low.
> 

This is basically a performance regression that could be due to the MP
work. You are seemingly running into contention that wasn't possible before.
The question is, where is this happening? I don't know if the dynamic tracer 
can help here. 

Re: IPSec heavy traffic slows down all network traffic

[Hrvoje Popovski]

On 17.7.2020. 20:17, jean-yves boisiaud wrote:
> hello,
> 
> Last week, I upgraded a couple of firewalls using carp/pfsync and sasyncd
> from 6.0 to 6.7 (yes, big jump !).
> 
> I also applied all the 6.7 published patches.
> 
> When some heavy traffic takes one of the IPSec tunnel, I noticed that :
> - all network connections are slowed down
> - unused network bandwidth increase instead of decrease
> - idle CPU move towards 0, and spinning increase to take about 50% of the
> CPU
> 
> When I stop the IPSec traffic :
> - network connections increase immediatly
> - unused network bandwidth cecreases immediately
> - spinning CPU is low.
> 
> Yes I know, my hardware is a bit old. I understand that CPU raises due to
> IPSec crypto, but I do not understand why network performance decrease.


maybe intel mitigation stuff decreased your performance. it in from
openbsd 6.3 ...
don't know if you are using aes for ipsec, but you cpu doesn't have
aes-ni... maybe to try wireguard ? :)

IPSec heavy traffic slows down all network traffic

[jean-yves boisiaud]

hello,

Last week, I upgraded a couple of firewalls using carp/pfsync and sasyncd
from 6.0 to 6.7 (yes, big jump !).

I also applied all the 6.7 published patches.

When some heavy traffic takes one of the IPSec tunnel, I noticed that :
- all network connections are slowed down
- unused network bandwidth increase instead of decrease
- idle CPU move towards 0, and spinning increase to take about 50% of the
CPU

When I stop the IPSec traffic :
- network connections increase immediatly
- unused network bandwidth cecreases immediately
- spinning CPU is low.

Yes I know, my hardware is a bit old. I understand that CPU raises due to
IPSec crypto, but I do not understand why network performance decrease.

1) Situation before doing anything:

# pktstat -ntT -m 1  -i em1
interface: em1total: 122.6Mb (7m18s)
cur: 260.1k (0%) min: 0.0 max: 100.0M avg: 279.3k bps

   bps%  b desc

 69.6k   0% 348.6k tcp 109.7.96.229:54880 <-> 52.113.194.132:443
 60.0k   0%  36.1M ip proto 50 109.7.96.226 <-> 92.174.146.73
 36.5k   0% 182.8k tcp 109.7.96.229:59950 <-> 52.113.194.132:443
 12.3k   0%  61.5k tcp 109.7.96.229:51009 <-> 216.58.214.78:443
 11.8k   0%  58.9k tcp 109.7.96.229:61287 <-> 216.58.206.229:443

# top
load averages:  0.14,  0.12,  0.14 ..fr
20:00:05
81 processes: 2 running, 77 idle, 2 on processor   up
10:53
CPU0: 31.9% user,  0.0% nice, 21.4% sys,  5.8% spin,  0.4% intr, 40.5% idle
CPU1: 30.9% user,  0.0% nice, 17.2% sys,  5.2% spin,  0.0% intr, 46.7% idle
Memory: Real: 166M/403M act/tot Free: 561M Cache: 128M Swap: 0K/0K

  PID USERNAME PRI NICE  SIZE   RES STATE WAIT  TIMECPU COMMAND
35828 osadmin   520 1676K 3504K run/0 - 0:03  8.35% sshd
68723 _openvpn   20 4016K 6404K sleep/1   poll 11:41  1.12% openvpn
16143 root   20 1372K 4056K sleep/0   poll  0:00  0.49% sshd
95804 root  280 5440K 6892K run/0 - 0:05  0.34% pktstat

2) Making heavy traffic NOT using IPSec :
Notice bandwidth usage.

heavy traffic NOT using the IPSec tunnel
# ssh ardee dd if=/dev/urandom bs=1M | dd of=/dev/null bs=1M
0+12031 records in
0+12031 records out
198180864 bytes (198 MB, 189 MiB) copied, 23.3799 s, 8.5 MB/s
0+19257 records in
0+19257 records out
316571648 bytes (317 MB, 302 MiB) copied, 37.167 s, 8.5 MB/s

# pktstat -ntT -m 1  -i em1
interface: em1total: 8.2Gb (11m49s)
cur: 72.6M (72%) min: 0.0 max: 100.0M avg: 11.5M bps

   bps%  b desc

 72.4M  72%   8.0G tcp 109.7.96.226:63663 <-> 212.83.131.76:2
 66.4k   0%  60.2M ip proto 50 109.7.96.226 <-> 92.174.146.73
 33.5k   0% 167.7k tcp 109.7.96.229:52670 <-> 52.97.168.210:443
 10.3k   0%   7.5M ip proto 112 109.7.96.227 <-> 224.0.0.18
  9.2k   0%  46.3k tcp 109.7.96.229:56973 <-> 40.101.92.178:443

# top
load averages:  1.11,  0.61,  0.34 billy.basystemes.fr
20:04:41
76 processes: 75 idle, 1 on processor  up
10:58
CPU0: 13.8% user,  0.0% nice, 18.6% sys,  1.2% spin, 11.2% intr, 55.3% idle
CPU1: 10.2% user,  0.0% nice, 29.3% sys,  0.6% spin,  0.0% intr, 59.9% idle
Memory: Real: 166M/390M act/tot Free: 574M Cache: 115M Swap: 0K/0K

  PID USERNAME PRI NICE  SIZE   RES STATE WAIT  TIMECPU COMMAND
95804 root   20 9760K 8696K sleep/1   poll  0:36 15.77% pktstat
68723 _openvpn   20 4012K 6332K sleep/1   poll 11:46  1.17% openvpn
33560 _isakmpd   20   11M   15M sleep/0   select7:28  0.59% isakmpd
83650 _openvpn   20 3928K 6388K sleep/0   poll 20:10  0.00% openvpn

3) Making heavy traffic using the IPSec tunnel in addition to the previous
heavy traffic :
Notice bandwidth usage, which has decreased, and spinning value in top.
Also notice the weak rate tranfer in the IPSec tunnel.

heavy traffic NOT using the IPSec tunnel
# ssh ardee dd if=/dev/urandom bs=1M | dd of=/dev/null bs=1M
0+11902 records in
0+11902 records out
231751680 bytes (232 MB, 221 MiB) copied, 109.809 s, 2.1 MB/s
0+12372 records in
0+12372 records out
247152640 bytes (247 MB, 236 MiB) copied, 131.151 s, 1.9 MB/s

heavy traffic using the IPSec tunnel
# ssh doon dd if=/dev/urandom bs=1M | dd of=/tmp/null bs=1M
0+2496 records in
0+2496 records out
81723392 bytes (82 MB, 78 MiB) copied, 91.6991 s, 891 kB/s
0+3078 records in
0+3078 records out
100794368 bytes (101 MB, 96 MiB) copied, 113.042 s, 892 kB/s

# pktstat -ntT -m 1  -i em1
interface: em1total: 15.3Gb (13m44s)
cur: 11.1M (11%) min: 0.0 max: 100.0M avg: 18.5M bps

   bps%  b desc

  6.2M   6% 163.3M ip proto 50 109.7.96.226 <-> 92.174.146.73
  4.7M   4%   1.2G tcp 109.7.96.226:52734 <-> 212.83.131.76:2
 33.7k   0% 474.5k ip fragments
 25.8k   0%   2.5M udp 109.7.96.228:1195 <-> 92.135.30.8:52978
 18.2k   0%   9.8M udp 109.7.96.228:1195 <-> 91.166.166.68:17587
 17.6k   0%  88.3k tcp 109.7.96.229:443 <-> 213.32.72.115:47

Re: strongSwan cannot install IPsec policies on OpenBSD

[Stuart Henderson]

On 2020-02-20, Peter Müller  wrote:
> Hello openbsd-misc,
>
> is anybody out there running strongSwan as an IPsec client for a net-to-net 
> connection
> on an OpenBSD machine?
>
> If so, I would be very grateful to know which steps are necessary in order to 
> successfully
> route traffic through this n2n connection and what your ipsec.conf file (and 
> other ones,
> if necessary) looks like.
>
> Sorry for bringing this up again, but I am out of ideas now and packaging 
> strongSwan
> for OpenBSD would not make sense if it could not be used properly. :-)
>
> Thanks again for any advice on this.
>
> Best regards,
> Peter Müller
>
>

strongSwan is packaged because it covers for some deficiencies in the
tools in base and works in some use cases (say, single machine connecting
to a VPN which needs EAP for authentication), that is a good enough use
case that it makes sense to package it.

I don't know how I could make it clearer than I already did in the
package description and pkg-readme file about the state of support - you
really want something else for lan-to-lan on OpenBSD.

Re: strongSwan cannot install IPsec policies on OpenBSD

[Hrvoje Popovski]

On 20.2.2020. 18:47, Peter Müller wrote:
> Hello openbsd-misc,
> 
> is anybody out there running strongSwan as an IPsec client for a net-to-net 
> connection
> on an OpenBSD machine?
> 
> If so, I would be very grateful to know which steps are necessary in order to 
> successfully
> route traffic through this n2n connection and what your ipsec.conf file (and 
> other ones,
> if necessary) looks like.
> 
> Sorry for bringing this up again, but I am out of ideas now and packaging 
> strongSwan
> for OpenBSD would not make sense if it could not be used properly. :-)
> 
> Thanks again for any advice on this.
> 
> Best regards,
> Peter Müller
> 

Maybe stupid question... can you use isakmpd on openbsd box and
strongswan on that other box ? i have working configuration for
site-to-site setup and it's working quite well ..

Re: strongSwan cannot install IPsec policies on OpenBSD

[Peter Müller]

Hello openbsd-misc,

is anybody out there running strongSwan as an IPsec client for a net-to-net 
connection
on an OpenBSD machine?

If so, I would be very grateful to know which steps are necessary in order to 
successfully
route traffic through this n2n connection and what your ipsec.conf file (and 
other ones,
if necessary) looks like.

Sorry for bringing this up again, but I am out of ideas now and packaging 
strongSwan
for OpenBSD would not make sense if it could not be used properly. :-)

Thanks again for any advice on this.

Best regards,
Peter Müller

Re: strongSwan cannot install IPsec policies on OpenBSD

[Peter Müller]

Hello Stuart,

>>>
>>> strongSwan's module to install policies to the kernel (kernel-pfkey) does
>>> not support OpenBSD without making code changes. Not impossible but hasn't
>>> been done. Only their userland setup that works with tun(4) devices
>>> (slightly confusingly called kernel-ipsec) is available.
>>
>> Hm, after fiddling around for a while, I am a bit helpless on this. Do you 
>> happen to have
>> some example configuration? If yes, I would be very grateful to see it. :-)
> 
> I put a sanitized version of my config in the pkg-readme file in the
> strongswan package - but I only used it for a very basic EAP-MSCHAP
> client (and I don't know strongswan very well; I normally only use it
> on Android with the gui configuration tool) so there is nothing fancy
> in there.
> 

Thank you - unfortunately, it does not seem to work here. An IKE_SA is 
successfully
established, CHILD_SA fails with the same error message. If "installpolicy=no" 
is
appended to the appropriate connection in /etc/strongswan/ipsec.conf, both 
IKE_SA
and CHILD_SA can be established but no traffic will be routed through the 
tunnel:

> Status of IKE charon daemon (strongSwan 5.8.1, OpenBSD 6.6, amd64):
>   uptime: 2 minutes, since Feb 17 15:44:04 2020
>   worker threads: 6 of 16 idle, 6/0/4/0 working, job queue: 0/0/0/0, 
> scheduled: 6
>   loaded plugins: charon pkcs11 aes des rc2 sha2 sha1 md4 md5 mgf1 random 
> nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey 
> sshkey pem botan fips-prf gmp curve25519 chapoly xcbc cmac hmac gcm attr 
> kernel-libipsec kernel-pfroute resolve socket-default stroke vici updown 
> eap-identity eap-gtc eap-mschapv2 eap-radius eap-tls eap-ttls eap-peap 
> xauth-generic xauth-eap counters
> Listening IP addresses:
>   94.xxx.xxx.xxx
>   2a03:::::::
> Connections:
>N2NTESTCONN:  xxx...yyy  IKEv2, dpddelay=10s
>N2NTESTCONN:   local:  [xxx] uses public key authentication
>N2NTESTCONN:cert:  "C=EU, O=xxx, CN=xxx"
>N2NTESTCONN:   remote: [yyy] uses public key authentication
>N2NTESTCONN:cert:  "C=EU, O=yyy, CN=yyy"
>N2NTESTCONN:   child:  10.xxx.xxx.2/32 === 10.yyy.yyy.0/24 TUNNEL, 
> dpdaction=restart
> Security Associations (1 up, 0 connecting):
>N2NTESTCONN[1]: ESTABLISHED 2 minutes ago, 
> 94.xxx.xxx.xxx[xxx]...87.yyy.yyy.yyy[yyy]
>N2NTESTCONN[1]: IKEv2 SPIs: a14ff33decbcc124_i* 2a6d95dc56127468_r, 
> public key reauthentication in 2 hours
>N2NTESTCONN[1]: IKE proposal: 
> AES_GCM_16_256/PRF_HMAC_SHA2_512/CURVE_25519
>N2NTESTCONN{1}:  INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: 
> f44fa42e_i cf5467e8_o
>N2NTESTCONN{1}:  AES_GCM_16_256, 5040 bytes_i (60 pkts, 0s ago), 0 
> bytes_o, rekeying in 42 minutes
>N2NTESTCONN{1}:   10.xxx.xxx.2/32 === 10.yyy.yyy.0/24

Traffic from the remote IPsec peer (which is a Linux machine) successfully 
reaches the
OpenBSD system ("5040 bytes_i"), but responses do not make it back ("0 
bytes_o"). Actually,
this is where I need help - manually installing SAs does not make sense to me.

Thank you in advance for any hints.

Best regards,
Peter Müller

P.S.: Sorry, I thought I had sent this to  already, but put 
in some
crappy To-Header. Sleep is no adequate substitution for caffeine... :-/

Re: strongSwan cannot install IPsec policies on OpenBSD

[Stuart Henderson]

On 2020/02/16 18:25, Peter Müller wrote:
> Hello Stuart,
> 
> thanks for your quick reply.
> 
> 
> > On 2020-02-14, Peter Müller  wrote:
> >> Hello openbsd-misc,
> >>
> >> during some flaws in OpenIKED, I am forced to use strongSwan as an IPsec 
> >> client on an
> >> OpenBSD 6.6 machine. While establishing an IKE_SA works fine, installing 
> >> policies for CHILD_SA
> >> fails (as expected):
> >>
> >>> unable to install IPsec policies (SPD) in kernel
> >>> failed to establish CHILD_SA, keeping IKE_SA
> >>
> >> To those who are running strongSwan as an IPsec client on OpenBSD: Which 
> >> is the best
> >> procedure in this case? Are there other methods of installing IPsec 
> >> policies into the
> >> kernel available?
> > 
> > strongSwan's module to install policies to the kernel (kernel-pfkey) does
> > not support OpenBSD without making code changes. Not impossible but hasn't
> > been done. Only their userland setup that works with tun(4) devices
> > (slightly confusingly called kernel-ipsec) is available.
> 
> Hm, after fiddling around for a while, I am a bit helpless on this. Do you 
> happen to have
> some example configuration? If yes, I would be very grateful to see it. :-)

I put a sanitized version of my config in the pkg-readme file in the
strongswan package - but I only used it for a very basic EAP-MSCHAP
client (and I don't know strongswan very well; I normally only use it
on Android with the gui configuration tool) so there is nothing fancy
in there.

Re: strongSwan cannot install IPsec policies on OpenBSD

[Peter Müller]

Hello Stuart,

thanks for your quick reply.


> On 2020-02-14, Peter Müller  wrote:
>> Hello openbsd-misc,
>>
>> during some flaws in OpenIKED, I am forced to use strongSwan as an IPsec 
>> client on an
>> OpenBSD 6.6 machine. While establishing an IKE_SA works fine, installing 
>> policies for CHILD_SA
>> fails (as expected):
>>
>>> unable to install IPsec policies (SPD) in kernel
>>> failed to establish CHILD_SA, keeping IKE_SA
>>
>> To those who are running strongSwan as an IPsec client on OpenBSD: Which is 
>> the best
>> procedure in this case? Are there other methods of installing IPsec policies 
>> into the
>> kernel available?
> 
> strongSwan's module to install policies to the kernel (kernel-pfkey) does
> not support OpenBSD without making code changes. Not impossible but hasn't
> been done. Only their userland setup that works with tun(4) devices
> (slightly confusingly called kernel-ipsec) is available.

Hm, after fiddling around for a while, I am a bit helpless on this. Do you 
happen to have
some example configuration? If yes, I would be very grateful to see it. :-)

Thanks, and best regards,
Peter Müller

> 
> 
>> P.S.: In case anybody wonders about the "OpenIKED flaws", these are as 
>> follows:
>> (a) Restarting single connections is not possible
>> (b) Dead Peer Detection is missing (I am aware of ifstated as a 
>> "replacement", but since
>> there seems to be no way of restarting a single IPsec connection, 
>> restarting the whole
>> iked daemon causes operational tunnels to crash)
>> (c) IKE is missing AES-GCM support (while ESP does - not sure why this is)
>> (d) Does not seem to support more than one private key
> 
> (e) no client side address-config
> (f) doesn't work with intermediate certs

Glad you mention it. I was bumping into something similar already and wondered 
why thinks
won't work...

> (plus some other missing things that would make life a lot easier, especially
> punting EAP off to a radius server ;)
> 
>> Apart from that, I really appreciate OpenIKED especially for its 
>> configuration file
>> syntax, but unfortunately cannot use it primarily due to (a) and (d).
> 

Re: strongSwan cannot install IPsec policies on OpenBSD

[Stuart Henderson]

On 2020-02-14, Peter Müller  wrote:
> Hello openbsd-misc,
>
> during some flaws in OpenIKED, I am forced to use strongSwan as an IPsec 
> client on an
> OpenBSD 6.6 machine. While establishing an IKE_SA works fine, installing 
> policies for CHILD_SA
> fails (as expected):
>
>> unable to install IPsec policies (SPD) in kernel
>> failed to establish CHILD_SA, keeping IKE_SA
>
> To those who are running strongSwan as an IPsec client on OpenBSD: Which is 
> the best
> procedure in this case? Are there other methods of installing IPsec policies 
> into the
> kernel available?

strongSwan's module to install policies to the kernel (kernel-pfkey) does
not support OpenBSD without making code changes. Not impossible but hasn't
been done. Only their userland setup that works with tun(4) devices
(slightly confusingly called kernel-ipsec) is available.


> P.S.: In case anybody wonders about the "OpenIKED flaws", these are as 
> follows:
> (a) Restarting single connections is not possible
> (b) Dead Peer Detection is missing (I am aware of ifstated as a 
> "replacement", but since
> there seems to be no way of restarting a single IPsec connection, 
> restarting the whole
> iked daemon causes operational tunnels to crash)
> (c) IKE is missing AES-GCM support (while ESP does - not sure why this is)
> (d) Does not seem to support more than one private key

(e) no client side address-config
(f) doesn't work with intermediate certs
(plus some other missing things that would make life a lot easier, especially
punting EAP off to a radius server ;)

> Apart from that, I really appreciate OpenIKED especially for its 
> configuration file
> syntax, but unfortunately cannot use it primarily due to (a) and (d).

strongSwan cannot install IPsec policies on OpenBSD

[Peter Müller]

Hello openbsd-misc,

during some flaws in OpenIKED, I am forced to use strongSwan as an IPsec client 
on an
OpenBSD 6.6 machine. While establishing an IKE_SA works fine, installing 
policies for CHILD_SA
fails (as expected):

> unable to install IPsec policies (SPD) in kernel
> failed to establish CHILD_SA, keeping IKE_SA

To those who are running strongSwan as an IPsec client on OpenBSD: Which is the 
best
procedure in this case? Are there other methods of installing IPsec policies 
into the
kernel available?

Thanks for any help in advance.

Best regards,
Peter Müller

P.S.: In case anybody wonders about the "OpenIKED flaws", these are as follows:
(a) Restarting single connections is not possible
(b) Dead Peer Detection is missing (I am aware of ifstated as a "replacement", 
but since
there seems to be no way of restarting a single IPsec connection, 
restarting the whole
iked daemon causes operational tunnels to crash)
(c) IKE is missing AES-GCM support (while ESP does - not sure why this is)
(d) Does not seem to support more than one private key

Apart from that, I really appreciate OpenIKED especially for its configuration 
file
syntax, but unfortunately cannot use it primarily due to (a) and (d).

Re: IPsec and MTU / fragmentation

[Stuart Henderson]

On 2020-02-11, Simen Stavdal  wrote:
> tunnel will be able to fragment all incoming ip before sending it into the
> ipsec, which will not fragment for you.
> The clients will not have to change, nor any other protocol that sends ip
> via the double-tunnel.>
>
> If a client and a server set up a new conversation over tcp.
> They both have an MTU of 1500 and DF=1
> How will you fragment this, even being a L3 tunnel?

If you encapsulate the packets you can run it like this:

The "outer" packets get fragmented. The "inner" packets stay full-size

<1500 byte inner>
<--encap1--> <--encap2-->

The other end reassembles the outer packets before decapsulating the
(full size) inner packet.

I've done this personally with full-size ethernet frames through an
ipsec+etherip bridge, I think it also works for L3 encap.

Re: IPsec and MTU / fragmentation

[Janne Johansson]

Den tis 11 feb. 2020 kl 10:25 skrev Simen Stavdal :

>  tunnel will be able to fragment all incoming ip before sending it into the
> ipsec, which will not fragment for you.
> The clients will not have to change, nor any other protocol that sends ip
> via the double-tunnel.>
>
> If a client and a server set up a new conversation over tcp.
> They both have an MTU of 1500 and DF=1
> How will you fragment this, even being a L3 tunnel?
>

You don't fragment DF=1 packets, you send "Fragmentation Needed and Don't
Fragment was Set" back if they don't fit, like any L3 box would do
regardless and they adapt or fail.
That is what you should get for setting DF=1

-- 
May the most significant bit of your life be positive.

Re: IPsec and MTU / fragmentation

[Simen Stavdal]

If a client and a server set up a new conversation over tcp.
They both have an MTU of 1500 and DF=1
How will you fragment this, even being a L3 tunnel?

/S

On Tue, 11 Feb 2020 at 08:22, Janne Johansson  wrote:

> Den mån 10 feb. 2020 kl 20:53 skrev Simen Stavdal :
>
>> I think the more complete solution is to run some gif/gre inside ipsec
>>> and set low-enough MTU on that one, so it can correctly fragment incoming
>>> packets, and optionally rebuild the packets at the remote end, while also
>>> giving you an idea of "state" on the link so you optionally can run things
>>> like routing daemons or something that cares about and acts on tunnel
>>> state. This would cause even lower MTU, but still allow all kinds of
>>> traffic and not just the "popular" one.
>>>
>>
>> So, how will your client/server know about this lower mtu? And df bit is
>> set more often than not, so fragmentation is now allowed in a lot of cases.
>> This is exactly the problem that started this thread...
>>
>>>
>>>
> If the inner gif/gre tunnel has a lower mtu, then it being a layer-3
> tunnel will be able to fragment all incoming ip before sending it into the
> ipsec, which will not fragment for you.
> The clients will not have to change, nor any other protocol that sends ip
> via the double-tunnel.
>
> --
> May the most significant bit of your life be positive.
>