On Thu, Apr 17, 2025 at 03:25:11AM -0000, icefrog1...@gmail.com wrote:
Hi,

I hit a libvirt networking problem that guest cannot access host HTTP service. 
I debug this issue and tried some efforts.  Thanks for your suggestions!

Environment
-------------
guest IP: 192.168.122.46   (Linux, default NAT, installed using virt-manager)
host1 IP: 192.168.3.16     (Centos 8.5 running libvirt and qemu, default 
libvirt iptable rules)
HTTP service: 192.168.3.16:70    (firewall rules have allowed this port)

host2: 192.168.3.65:70  (for test only)

guest network xml
<interface type="network">
 <mac address="52:54:00:f5:a8:9f"/>
 <source network="default" portid="6e8ce7e7-6517-43fa-b113-aaddb6c1bc08" 
bridge="virbr0"/>
 <target dev="vnet2"/>
 <model type="e1000"/>
 <alias name="net0"/>
 <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0"/>
</interface>


1. guest and host1/host2 can ping each other
2. *guest can visit host2 HTTP
3. *guest cannot visit host1 HTTP

When I capture traffic in guest, Wireshark shows:
192.168.122.46 -> 192.168.3.16    // SYN ok
192.168.122.46 <- 192.168.3.16    // Destination unreachable (Port unreachable)

guest route table:
------------------
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.122.1   0.0.0.0         UG    100    0        0 eth0
192.168.122.0   0.0.0.0         255.255.255.0   U     100    0        0 eth0

host1 route table:
------------------
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.3.1     0.0.0.0         UG    100    0        0 enp3s0
192.168.3.0     0.0.0.0         255.255.255.0   U     100    0        0 enp3s0
192.168.122.0   0.0.0.0         255.255.255.0   UG    0      0        0 virbr0


I delete the last rule and add a rule to make sure host1 visits guests will go 
through 192.168.122.1
------------------
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
(other rules omitted)
192.168.122.0   192.168.122.1   255.255.255.0   UG    0      0        0 virbr0


However, traceroute shows this new rule does not work (which should go to 
192.168.122.1 first), and guest cannot visit host1 HTTP request.


[!] guest visit http://192.168.3.16:70 does not go through 192.168.122.1
traceroute 192.168.122.46
traceroute to 192.168.122.46 (192.168.122.46), 30 hops max, 60 byte packets
1  192.168.122.46 (192.168.122.46)  0.200 ms  0.194 ms  0.194 ms

# guest visit http://192.168.3.65:70 goes through 192.168.122.1
traceroute 192.168.3.65
traceroute to 192.168.3.65 (192.168.3.65), 30 hops max, 60 byte packets
1  192.168.122.1 (192.168.122.1)  0.244 ms  0.050 ms  0.120 ms
2  192.168.3.65 (192.168.3.65)  0.823 ms !X  0.802 ms !X  0.789 ms !X

In sum, is there a way to force the guest go through 192.168.122.1 when 
visiting the hosting machine?


The information is a bit confusing to me and I might not have the right
answer, but AFAIU the guest is going through the gateway when accessing
the machine, the routing table shows it has nowhere else to go when
going outside the 192.168.122.0/24 network.  So either do not use the
outside IP, but 192.168.122.1 _or_ check the rp_filter sysctl for your
interfaces as that seems like it could do something related to that.

-------------------
libvirt iptable rules:
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N LIBVIRT_INP
-N LIBVIRT_OUT
-N LIBVIRT_FWO
-N LIBVIRT_FWI
-N LIBVIRT_FWX
-A INPUT -j LIBVIRT_INP
-A FORWARD -j LIBVIRT_FWX
-A FORWARD -j LIBVIRT_FWI
-A FORWARD -j LIBVIRT_FWO
-A OUTPUT -j LIBVIRT_OUT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 68 -j ACCEPT
-A LIBVIRT_FWO -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable    # 
delete this rule does not work
-A LIBVIRT_FWI -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate 
RELATED,ESTABLISHED -j ACCEPT
-A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable    # 
delete this rule does not work
-A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT

Attachment: signature.asc
Description: PGP signature

Reply via email to