Trying to establish an ipsec tunnel to a debian linux box with openswan,
using this entry in ipsec.conf:
ike active esp from 192.168.1.0/24 to 192.168.2.0/24 peer a.b.c.d srcid
"[EMAIL PROTECTED]" dstid "[EMAIL PROTECTED]" psk xxxxxxxxxxx
I get 'PAYLOAD MALFORMED' in the middle of the phase 1 negotiation:
After the transforms are agreed upon and the nonces are exchanged, the
message containing the ID payload is rejected by openBSD, either with a
notification 'PAYLOAD MALFORMED' or with notification 'INVALID PAYLOAD
TYPE'
Here is a snippet of isakmpd.pcap:
21:38:55.438591 a.b.c.d.500 > u.v.w.x.500: [udp sum ok] isakmp v1.0
exchange ID_PROT
cookie: 251068307c823c51->5086ce0f33dfbb37 msgid: 00000000 len:
92
payload: ID len: 9336 [|isakmp] [ttl 0] (id 1, len 120)
21:38:55.439228 u.v.w.x.500 > a.b.c.d.500: [udp sum ok] isakmp v1.0
exchange INFO
cookie: 88fba1fcd13186bd->0000000000000000 msgid: 00000000 len:
40
payload: NOTIFICATION len: 12
notification: INVALID PAYLOAD TYPE [ttl 0] (id 1, len 68)
where a.b.c.d is openswan, and u.v.w.x is openbsd.
The IDs are of type USER_FQDN ( or should be, at least ).
The len field in the received packet seems queer. Maybe this causes the
problem.
This error only occurs, when the phase-1 exchange is initiated by
openswan. If openbsd starts the phase-1 exchange, all seems ok.
I would think this is an openswan problem, but how can I prove this? I
have no access to the openswan box. Can I get more information about the
offending packet, like a decrypted hexdump or else?
Any hints are welcome.
Regards
Christoph
