weizhouapache commented on PR #7719:
URL: https://github.com/apache/cloudstack/pull/7719#issuecomment-1624938671
> > @benj-n have you tested it ? if so , can you share the related
iptables/ebtables rules and output of `ipset list` command ?
>
> In the test, a VM has a first NIC with shared network (and security
groups) on vlan100/vnet16 and a second NIC with an L2 network (vlan2150/vnet20):
>
> ```
> root@node01:~# virsh domiflist i-2-596-VM
> Interface Type Source Model MAC
> -----------------------------------------------------------------
> vnet16 bridge brbond0-100 virtio 1e:00:06:00:20:29
> vnet20 bridge brbond9-2150 virtio 02:00:1d:52:00:01
> ```
>
> The vnet20 and the L2 network are completely absent from iptables :
>
> ```
> Chain BF-brbond0-100-IN (1 references)
> pkts bytes target prot opt in out source
destination
> (...)
> 240 24611 i-2-596-def all -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
>
> Chain BF-brbond0-100-OUT (1 references)
> pkts bytes target prot opt in out source
destination
> (...)
> 876 43779 i-2-596-def all -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-out vnet16 --physdev-is-bridged
>
>
> Chain i-2-596-VM (1 references)
> pkts bytes target prot opt in out source
destination
> 14 973 ACCEPT icmp -- * * 0.0.0.0/0
0.0.0.0/0 icmptype 255
> 417 21629 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 state NEW
> 150 6004 DROP all -- * * 0.0.0.0/0
0.0.0.0/0
>
> Chain i-2-596-VM-eg (1 references)
> pkts bytes target prot opt in out source
destination
> 136 9456 RETURN all -- * * 0.0.0.0/0
0.0.0.0/0 state NEW
> 0 0 DROP all -- * * 0.0.0.0/0
0.0.0.0/0
>
> Chain i-2-596-def (2 references)
> pkts bytes target prot opt in out source
destination
> 0 0 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 state RELATED,ESTABLISHED
> 32 10556 ACCEPT udp -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet16 --physdev-is-bridged udp
spt:68 dpt:67
> 2 667 ACCEPT udp -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-out vnet16 --physdev-is-bridged
udp spt:67 dpt:68
> 0 0 DROP udp -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet16 --physdev-is-bridged udp
spt:67
> 0 0 DROP all -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet16 --physdev-is-bridged !
match-set i-2-596-VM src
> 293 14506 DROP all -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-out vnet16 --physdev-is-bridged !
match-set i-2-596-VM dst
> 72 4599 RETURN udp -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
match-set i-2-596-VM src udp dpt:53
> 0 0 RETURN tcp -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
match-set i-2-596-VM src tcp dpt:53
> 136 9456 i-2-596-VM-eg all -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
match-set i-2-596-VM src
> 581 28606 i-2-596-VM all -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-out vnet16 --physdev-is-bridged
> ```
>
> The L2 is also completely absent from ip6tables:
>
> ```
> Chain BF-brbond0-100-IN (1 references)
> pkts bytes target prot opt in out source
destination
> (...)
> 90 6768 i-2-596-def all * * ::/0 ::/0
PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
>
> Chain BF-brbond0-100-OUT (1 references)
> pkts bytes target prot opt in out source
destination
> (...)
> 21 2184 i-2-596-def all * * ::/0 ::/0
PHYSDEV match --physdev-out vnet16 --physdev-is-bridged
>
> Chain i-2-596-VM (1 references)
> pkts bytes target prot opt in out source
destination
> 0 0 ACCEPT icmpv6 * * ::/0 ::/0
> 0 0 ACCEPT all * * ::/0 ::/0
state NEW
> 0 0 DROP all * * ::/0 ::/0
>
> Chain i-2-596-VM-eg (1 references)
> pkts bytes target prot opt in out source
destination
> 0 0 RETURN all * * ::/0 ::/0
state NEW
> 0 0 DROP all * * ::/0 ::/0
>
> Chain i-2-596-def (2 references)
> pkts bytes target prot opt in out source
destination
> 0 0 ACCEPT all * * ::/0 ::/0
state RELATED,ESTABLISHED
> 21 2184 ACCEPT icmpv6 * * fe80::/64
ff02::1 PHYSDEV match --physdev-out vnet16 --physdev-is-bridged
ipv6-icmptype 134 HL match HL == 255
> 0 0 RETURN icmpv6 * * ::/0
ff02::2 PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
ipv6-icmptype 133 HL match HL == 255
> 0 0 DROP icmpv6 * * ::/0 ::/0
PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
ipv6-icmptype 134
> 18 1296 RETURN icmpv6 * * ::/0 ::/0
PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
ipv6-icmptype 135 HL match HL == 255
> 0 0 ACCEPT icmpv6 * * ::/0 ::/0
PHYSDEV match --physdev-out vnet16 --physdev-is-bridged
ipv6-icmptype 135 HL match HL == 255
> 0 0 RETURN icmpv6 * * ::/0 ::/0
PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
ipv6-icmptype 136 match-set i-2-596-VM-6 src HL match HL == 255
> 0 0 ACCEPT icmpv6 * * ::/0 ::/0
PHYSDEV match --physdev-out vnet16 --physdev-is-bridged
ipv6-icmptype 136 HL match HL == 255
> 0 0 RETURN icmpv6 * * ::/0 ::/0
PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
ipv6-icmptype 2 match-set i-2-596-VM-6 src
> 0 0 ACCEPT icmpv6 * * ::/0 ::/0
PHYSDEV match --physdev-out vnet16 --physdev-is-bridged
ipv6-icmptype 2
> 0 0 RETURN icmpv6 * * ::/0 ::/0
PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
ipv6-icmptype 1 match-set i-2-596-VM-6 src
> 0 0 ACCEPT icmpv6 * * ::/0 ::/0
PHYSDEV match --physdev-out vnet16 --physdev-is-bridged
ipv6-icmptype 1
> 0 0 RETURN icmpv6 * * ::/0 ::/0
PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
ipv6-icmptype 3 match-set i-2-596-VM-6 src
> 0 0 ACCEPT icmpv6 * * ::/0 ::/0
PHYSDEV match --physdev-out vnet16 --physdev-is-bridged
ipv6-icmptype 3
> 0 0 RETURN icmpv6 * * ::/0 ::/0
PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
ipv6-icmptype 4 match-set i-2-596-VM-6 src
> 0 0 ACCEPT icmpv6 * * ::/0 ::/0
PHYSDEV match --physdev-out vnet16 --physdev-is-bridged
ipv6-icmptype 4
> 72 5472 RETURN icmpv6 * * ::/0
ff02::16 PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
> 0 0 RETURN udp * * fe80::1c00:6ff:fe00:2029
ff02::1:2 PHYSDEV match --physdev-in vnet16 --physdev-is-bridged udp
spt:546
> 0 0 ACCEPT udp * * fe80::/64
fe80::1c00:6ff:fe00:2029 PHYSDEV match --physdev-out vnet16
--physdev-is-bridged udp dpt:546
> 0 0 DROP udp * * ::/0
!fe80::/64 PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
udp spt:547
> 0 0 RETURN udp * * ::/0 ::/0
PHYSDEV match --physdev-in vnet16 --physdev-is-bridged udp dpt:53
match-set i-2-596-VM-6 src
> 0 0 RETURN tcp * * ::/0 ::/0
PHYSDEV match --physdev-in vnet16 --physdev-is-bridged tcp dpt:53
match-set i-2-596-VM-6 src
> 0 0 DROP all * * ::/0 ::/0
PHYSDEV match --physdev-in vnet16 --physdev-is-bridged !
match-set i-2-596-VM-6 src
> 0 0 i-2-596-VM-eg all * * ::/0
::/0 PHYSDEV match --physdev-in vnet16 --physdev-is-bridged
match-set i-2-596-VM-6 src
> 0 0 i-2-596-VM all * * ::/0 ::/0
PHYSDEV match --physdev-out vnet16 --physdev-is-bridged
> ```
>
> And the ipset only references IPs from the shared network:
>
> ```
>
> Name: i-2-596-VM
> Type: hash:ip
> Revision: 4
> Header: family inet hashsize 1024 maxelem 65536
> Size in memory: 248
> References: 5
> Number of entries: 1
> Members:
> 85.(redacted-public-ipv4).6
>
> Name: i-2-596-VM-6
> Type: hash:net
> Revision: 6
> Header: family inet6 hashsize 1024 maxelem 65536
> Size in memory: 1456
> References: 9
> Number of entries: 2
> Members:
> fe80::1c00:6ff:fe00:2029
> 2001:(redacted-ipv6):2029
> ```
>
> It's the same for ebtables, as expected, the L2 network is totally ignored
there too.
>
> ```
> Bridge chain: PREROUTING, entries: 6, policy: ACCEPT
> (...)
> -i vnet16 -j i-2-596-VM-in
>
> Bridge chain: POSTROUTING, entries: 6, policy: ACCEPT
> (...)
> -o vnet16 -j i-2-596-VM-out
>
> Bridge chain: i-2-596-VM-in, entries: 5, policy: ACCEPT
> -j i-2-596-VM-in-src
> -p ARP -j i-2-596-VM-in-ips
> -p ARP --arp-op Request -j ACCEPT
> -p ARP --arp-op Reply -j ACCEPT
> -p ARP -j DROP
>
> Bridge chain: i-2-596-VM-out, entries: 5, policy: ACCEPT
> -p ARP --arp-op Reply -j i-2-596-VM-out-dst
> -p ARP -j i-2-596-VM-out-ips
> -p ARP --arp-op Request -j ACCEPT
> -p ARP --arp-op Reply -j ACCEPT
> -p ARP -j DROP
>
> Bridge chain: i-2-596-VM-in-ips, entries: 2, policy: ACCEPT
> -p ARP -s 1e:00:06:00:20:29 --arp-ip-src 85.(redacted-public-ipv4).6
--arp-mac-src 1e:00:06:00:20:29 -j RETURN
> -j DROP
>
> Bridge chain: i-2-596-VM-out-ips, entries: 2, policy: ACCEPT
> -p ARP --arp-ip-dst 85.(redacted-public-ipv4).6 -j RETURN
> -j DROP
>
> Bridge chain: i-2-596-VM-in-src, entries: 2, policy: ACCEPT
> -s 1e:00:06:00:20:29 -j RETURN
> -j DROP
>
> Bridge chain: i-2-596-VM-out-dst, entries: 2, policy: ACCEPT
> -p ARP --arp-op Reply --arp-mac-dst 1e:00:06:00:20:29 -j RETURN
> -p ARP --arp-op Reply -j DROP
> ```
@benj-n
looks ok.
can you create a vm with only L2 networks ?
btw: did you create the L2 network with specified vlan id ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
