[libvirt] [PATCH] fixed xt_physdev warning when defining ip(6)tables rules
--- src/nwfilter/nwfilter_ebiptables_driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/nwfilter/nwfilter_ebiptables_driver.c b/src/nwfilter/nwfilter_ebiptables_driver.c index 4fec52d..db2276c 100644 --- a/src/nwfilter/nwfilter_ebiptables_driver.c +++ b/src/nwfilter/nwfilter_ebiptables_driver.c @@ -166,7 +166,7 @@ static const char ebiptables_script_set_ifs[] = snprintf(buf, sizeof(buf), %c%c-%s, prefix[0], prefix[1], ifname) #define PHYSDEV_IN --physdev-in -#define PHYSDEV_OUT --physdev-out +#define PHYSDEV_OUT --physdev-is-bridged --physdev-out static const char *m_state_out_str = -m state --state NEW,ESTABLISHED; static const char *m_state_in_str= -m state --state ESTABLISHED; -- 1.7.11.5 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH] fixed xt_physdev warning when defining ip(6)tables rules
Op 17-1-2013 19:33, Eric Blake schreef: On 01/17/2013 04:24 AM, Reinier Schoof wrote: --- src/nwfilter/nwfilter_ebiptables_driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/nwfilter/nwfilter_ebiptables_driver.c b/src/nwfilter/nwfilter_ebiptables_driver.c index 4fec52d..db2276c 100644 --- a/src/nwfilter/nwfilter_ebiptables_driver.c +++ b/src/nwfilter/nwfilter_ebiptables_driver.c @@ -166,7 +166,7 @@ static const char ebiptables_script_set_ifs[] = snprintf(buf, sizeof(buf), %c%c-%s, prefix[0], prefix[1], ifname) #define PHYSDEV_IN --physdev-in -#define PHYSDEV_OUT --physdev-out +#define PHYSDEV_OUT --physdev-is-bridged --physdev-out Stefan ACKd in the other thread, so I've gone ahead and pushed this, after amending the commit message to include details from the other thread. -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list Thanks for committing the patch! On a side note, please be aware that when upgrading to a libvirt version with this patch included, libvirt will not be able to remove the earlier ip(6)tables rules without the '--physdev-is-bridged' addition. When restarting libvirt, it will look for rules that match with '--physdev-is-bridged' and since that wasn't there before, you'll end up with a duplicate/malfunctioning ruleset. You'll have to remove these rules/chains manually. -- TransIP BV | https://www.transip.nl/ -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] iptables --physdev-out warnings
Hi, we've experienced some issues with starting lots of KVM based VM's with libvirt. Since I couldn't find any clues on the libvirt mailing list, I'm posting the way I fixed the issues. When starting a VM, /var/log/messages was spammed with the following message: xt_physdev: using --physdev-out in the OUTPUT, FORWARD and POSTROUTING chains for non-bridged traffic is not supported anymore. With each extra VM I start, the messages get amplified exponentially. This results in longer starting times every new VM, relative the the previously started VM. When I ran a test with starting 100 equal VM's, the first VM started in about 2 seconds, the 100th VM took 48 seconds to start. I'm running a vanilla 3.7.1 kernel, but I have the same issue on VM hosts with kernel 3.2.28 or 3.2.0, running libvirt 0.9.12 and 0.9.8 respectively. Looking into the warning, it seemed that iptables need an extra argument, --physdev-is-bridged, in commands like: iptables -A libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet99 -g FP-vnet99 I patched the libvirt source (version 1.0.0) to test whether this works or not: --- src/nwfilter/nwfilter_ebiptables_driver.c.orig 2013-01-16 10:51:43.0 +0100 +++ src/nwfilter/nwfilter_ebiptables_driver.c 2013-01-16 10:52:07.0 +0100 @@ -166,7 +166,7 @@ snprintf(buf, sizeof(buf), %c%c-%s, prefix[0], prefix[1], ifname) #define PHYSDEV_IN --physdev-in -#define PHYSDEV_OUT --physdev-out +#define PHYSDEV_OUT --physdev-is-bridged --physdev-out static const char *m_state_out_str = -m state --state NEW,ESTABLISHED; static const char *m_state_in_str= -m state --state ESTABLISHED; The warnings in /var/log/messages are gone and running the test again proved the 100th VM started in 3.8 seconds. It suprises me I'm the first to mention this problem on the libvirt mailing list and I wondering if I'm doing something wrong. Until then, this fix helps me a lot! Reinier Schoof -- TransIP BV | https://www.transip.nl/ -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] iptables --physdev-out warnings
Hi, we've experienced some issues with starting lots of KVM based VM's with libvirt. Since I couldn't find any clues on the libvirt mailing list, I'm posting the way I fixed the issues. When starting a VM, /var/log/messages was spammed with the following message: xt_physdev: using --physdev-out in the OUTPUT, FORWARD and POSTROUTING chains for non-bridged traffic is not supported anymore. With each extra VM I start, the messages get amplified exponentially. This results in longer starting times every new VM, relative the the previously started VM. When I ran a test with starting 100 equal VM's, the first VM started in about 2 seconds, the 100th VM took 48 seconds to start. I'm running a vanilla 3.7.1 kernel, but I have the same issue on VM hosts with kernel 3.2.28 or 3.2.0, running libvirt 0.9.12 and 0.9.8 respectively. Looking into the warning, it seemed that iptables need an extra argument, --physdev-is-bridged, in commands like: iptables -A libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet99 -g FP-vnet99 I patched the libvirt source (version 1.0.0) to test whether this works or not: --- src/nwfilter/nwfilter_ebiptables_driver.c.orig 2013-01-16 10:51:43.0 +0100 +++ src/nwfilter/nwfilter_ebiptables_driver.c 2013-01-16 10:52:07.0 +0100 @@ -166,7 +166,7 @@ snprintf(buf, sizeof(buf), %c%c-%s, prefix[0], prefix[1], ifname) #define PHYSDEV_IN --physdev-in -#define PHYSDEV_OUT --physdev-out +#define PHYSDEV_OUT --physdev-is-bridged --physdev-out static const char *m_state_out_str = -m state --state NEW,ESTABLISHED; static const char *m_state_in_str= -m state --state ESTABLISHED; The warnings in /var/log/messages are gone and running the test again proved the 100th VM started in 3.8 seconds. It suprises me I'm the first to mention this problem on the libvirt mailing list and I wondering if I'm doing something wrong. Until then, this fix helps me a lot! Reinier Schoof -- TransIP BV | https://www.transip.nl/ -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] libvirt starts KVM domain without -enable-kvm
Hey, we're running a VM-pool based on libvirt and QEMU/KVM. The host machines run debian 6 (squeeze). All VM's have a similar libvirt XML-definition. Since we've moved to the combination of Linux kernel 3.2.0-1 and libvirt 0.9.9 (installed from testing/wheezy repo's), we periodically notice poor performance of new(ly started) VM's and after inspection they turn out to be started without the qemu option -enable-kvm. In /var/log/libvirt/qemu/$VM-name.log a line like this is shown: 20529: error : qemuBuildCommandLine:3681 : unsupported configuration: the QEMU binary /usr/bin/kvm does not support kvm After a virsh destroy/start, using the same XML-definition, the VM is started WITH -enable-kvm. Looking at the libvirt source of 0.9.9, I can't imagine this being the result of a race condition or other strange circumstance: it simply parse 'qemu -help', which always give exactly the same output. I've tried reproducing the problem by continuously destroy, undefine, define and start the same VM and then grepping the kvm-commandline for the absence of -enable-kvm, without any result. The XML-definition includes domain type='kvm' and emulator/usr/bin/kvm/emulator Does anyone recognize this behaviour of libvirt? Thanks, Reinier Schoof # virsh version Compiled against library: libvir 0.9.9 Using library: libvir 0.9.9 Using API: QEMU 0.9.9 Running hypervisor: QEMU 0.15.0 Linux 3.2.0-1-amd64 #1 SMP Fri Feb 17 05:17:36 UTC 2012 x86_64 GNU/Linux -- TransIP BV | https://www.transip.nl/ -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] issues with migrating using copy-storage-all
Hi, today I was trying to use the --copy-storage-all feature of virsh migrate, in an attempt to migrate KVM-instances to another storage backend. Doing so, I ran into some trouble: First of all, it turned out the disk image-file has to be present on the receiving end of the migration. When, just to check, this disk image is smaller than the original image, migration suddenly stops (after filling the maximum size of the too small disk image). This is expected behaviour ofcourse. But when I fix the disk image to the right size and start the migration again, the migration fails immediately, the domain is -undefined- and crashes. Qemu log showed: kvm: block.c:2889: bdrv_set_in_use: Assertion `bs-in_use != in_use' failed. 2011-12-09 10:58:16.211: shutting down Is this behaviour known to you guys? Also, when I migrate a domain (10G qcow2 disk image, which is only used for 1GB), the qcow2 image on the receiving end shows 10G for both 'virtual disk' and 'disk size', while this was 10G and 1G respectively on the sending end. Why is the image expanded? Or is this a limitation of the copy-storage-all? How does the copy-storage-all function works for raw disk images? Does it send incremental copies of blocks which are written too since the migration is started? Thanks in advance for your help! Reinier Schoof -- TransIP BV | https://www.transip.nl/ -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] problem with nwfilter and ip6tables
That actually works, thanks a lot! Op 21-11-2011 17:07, Stefan Berger schreef: On 11/21/2011 08:46 AM, Reinier Schoof wrote: Going back to the original email: Hi, I'm investigating using the nwfilter-functionality of libvirt to give my clients the possibility to block ports of their VPSes. The same mechanism allows me to restrict the outgoing traffic a VPS is generating. In the end, I want to restrict MAC, IPv4 and IPv6 traffic, while the client can also restrict traffic to UDP and TCP. All goes well, until I want to restrict the UDP/TCP traffic to certain IPv6 addresses. Where iptables shows the IPv4-restriction I've put up, ip6tables doesn't show anything. In the logs, I only see some ip6tables -D, -X and -F commands failing, which is expected when libvirt tries to delete/flush rules that were never there. I've built my nwfilter containing the following IPv6-rules, which I for instance reference once for all the TCP-ports which should be open. !-- Allow established traffic -- filter name='ipv6-allow-statefull' chain='ipv6' rule action='accept' direction='in' priority='500' all state='ESTABLISHED'/ /rule rule action='accept' direction='out' priority='500' all state='ESTABLISHED,RELATED'/ /rule /filter Replace with filter name='ipv6-allow-statefull' chain='root' uuidd7ca42fe-a2f5-6491-cdee-10d8a0956772/uuid rule action='accept' direction='in' priority='100' all-ipv6 state='ESTABLISHED'/ /rule rule action='accept' direction='out' priority='100' all-ipv6 state='ESTABLISHED,RELATED'/ /rule /filter !-- Allow TCP in $PORT -- filter name='ipv6-allow-create-state-by-port' chain='ipv6' rule action='accept' direction='in' priority='500' tcp state='NEW' dstportstart='$PORT'/ /rule rule action='accept' direction='in' priority='500' udp state='NEW' dstportstart='$PORT'/ /rule /filter Replace with filter name='ipv6-allow-create-state-by-port' chain='root' uuidff97e825-712d-6b1a-c5d1-46fe635f9dd6/uuid rule action='accept' direction='in' priority='500' tcp-ipv6 state='NEW' dstportstart='$PORT'/ /rule rule action='accept' direction='in' priority='500' udp-ipv6 state='NEW' dstportstart='$PORT'/ /rule /filter !-- Allow IPv6 traffic from $RANGE -- filter name='ipv6-allow-create-state-by-range' chain='ipv6' rule action='accept' direction='out' priority='500' ipv6 srcipaddr='$RANGE' srcipmask='64'/ /rule /filter This probably should either be direction='in' or you may want to replace srcipaddr and srcipmask with dstipaddr and dstipmask. Replace with filter name='ipv6-allow-create-state-by-range' chain='root' uuid6e738070-9505-730d-14e6-ee01a6eb5885/uuid rule action='accept' direction='in' priority='500' all-ipv6 srcipaddr='$RANGE' srcipmask='62'/ /rule /filter You may want to add state='NEW' to the rule as well. !-- Drop all other IPv6 traffic -- filter name='ipv6-drop-stateless' chain='ipv6' rule action='drop' direction='inout' priority='999' all/ /rule /filter Replace with filter name='ipv6-drop-stateless' chain='root' uuid4377aca7-18fb-b373-4462-4ee2ba3db7cd/uuid rule action='drop' direction='inout' priority='999' all-ipv6/ /rule /filter You have to change the chain to 'root' and the protocol in the rules has to be tcp-ipv6, all-ipv6 etc. for ipv6 traffic. The reason is that most of these rules could be applied to either iptables or ip6tables and the network filtering system needs some more 'hints' whether it is indeed an ipv6 rule so it create ip6tables commands versus iptables commands. I hope this helps. 'ip6tables -L -n' here now shows: Chain FI-vnet0 (1 references) target prot opt source destination RETURN all ::/0 ::/0 state RELATED,ESTABLISHED RETURN all ::/0 ::/64 state ESTABLISHED ctdir ORIGINAL DROP all ::/0 ::/0 Chain FO-vnet0 (1 references) target prot opt source destination ACCEPT all ::/0 ::/0 state ESTABLISHED ACCEPT tcp ::/0 ::/0 tcp dpt:90 state NEW ACCEPT udp ::/0 ::/0 udp dpt:90 state NEW ACCEPT all ::/64 ::/0 state NEW,ESTABLISHED ctdir REPLY DROP all ::/0 ::/0 Chain HI-vnet0 (1 references) target prot opt source destination RETURN all ::/0 ::/0 state RELATED,ESTABLISHED RETURN all ::/0 ::/64 state ESTABLISHED ctdir ORIGINAL DROP all ::/0 ::/0 Stefan I use a similar approach for my IPv4 firewall, and it works perfectly. When I use these IPv6 rules, all IPv6 traffic is apparently dropped, but it's hard to debug when the result of this config is abscent in ip6tables. I'm using these version of software on debian 6.0 squeeze: virsh # version Compiled against library: libvir 0.9.2 Using library: libvir 0.9.2 Using API: QEMU 0.9.2 Running hypervisor: QEMU 0.15.0 Does anyone have any clues? Thanks in advance! Regards, Reinier Schoof -- TransIP BV | https://www.transip.nl/ -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] problem with nwfilter and ip6tables
Hi, I'm investigating using the nwfilter-functionality of libvirt to give my clients the possibility to block ports of their VPSes. The same mechanism allows me to restrict the outgoing traffic a VPS is generating. In the end, I want to restrict MAC, IPv4 and IPv6 traffic, while the client can also restrict traffic to UDP and TCP. All goes well, until I want to restrict the UDP/TCP traffic to certain IPv6 addresses. Where iptables shows the IPv4-restriction I've put up, ip6tables doesn't show anything. In the logs, I only see some ip6tables -D, -X and -F commands failing, which is expected when libvirt tries to delete/flush rules that were never there. I've built my nwfilter containing the following IPv6-rules, which I for instance reference once for all the TCP-ports which should be open. !-- Allow established traffic -- filter name='ipv6-allow-statefull' chain='ipv6' rule action='accept' direction='in' priority='500' all state='ESTABLISHED'/ /rule rule action='accept' direction='out' priority='500' all state='ESTABLISHED,RELATED'/ /rule /filter !-- Allow TCP in $PORT -- filter name='ipv6-allow-create-state-by-port' chain='ipv6' rule action='accept' direction='in' priority='500' tcp state='NEW' dstportstart='$PORT'/ /rule rule action='accept' direction='in' priority='500' udp state='NEW' dstportstart='$PORT'/ /rule /filter !-- Allow IPv6 traffic from $RANGE -- filter name='ipv6-allow-create-state-by-range' chain='ipv6' rule action='accept' direction='out' priority='500' ipv6 srcipaddr='$RANGE' srcipmask='64'/ /rule /filter !-- Drop all other IPv6 traffic -- filter name='ipv6-drop-stateless' chain='ipv6' rule action='drop' direction='inout' priority='999' all/ /rule /filter I use a similar approach for my IPv4 firewall, and it works perfectly. When I use these IPv6 rules, all IPv6 traffic is apparently dropped, but it's hard to debug when the result of this config is abscent in ip6tables. I'm using these version of software on debian 6.0 squeeze: virsh # version Compiled against library: libvir 0.9.2 Using library: libvir 0.9.2 Using API: QEMU 0.9.2 Running hypervisor: QEMU 0.15.0 Does anyone have any clues? Thanks in advance! Regards, Reinier Schoof -- TransIP BV | https://www.transip.nl/ -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] problem with nwfilter and ip6tables
Hi Stefan, thanks for your quick response Op 21-11-2011 15:21, Stefan Berger schreef: On 11/21/2011 08:46 AM, Reinier Schoof wrote: Hi, I'm investigating using the nwfilter-functionality of libvirt to give my clients the possibility to block ports of their VPSes. The same mechanism allows me to restrict the outgoing traffic a VPS is generating. In the end, I want to restrict MAC, IPv4 and IPv6 traffic, while the client can also restrict traffic to UDP and TCP. All goes well, until I want to restrict the UDP/TCP traffic to certain IPv6 addresses. Where iptables shows the IPv4-restriction I've put up, ip6tables doesn't show anything. In the logs, I only see some ip6tables -D, -X and -F commands failing, which is expected when libvirt tries to delete/flush rules that were never there. I've built my nwfilter containing the following IPv6-rules, which I for instance reference once for all the TCP-ports which should be open. !-- Allow established traffic -- filter name='ipv6-allow-statefull' chain='ipv6' rule action='accept' direction='in' priority='500' all state='ESTABLISHED'/ /rule rule action='accept' direction='out' priority='500' all state='ESTABLISHED,RELATED'/ /rule /filter With the configuration as you have it, can you post the dump of ip6tables -L -n ? Nothing to see there. All I see in my libvirt.log is some ip6tables -D, -X, -F and -E commands failing: /sbin/ip6tables -D libvirt-out -m physdev --physdev-out vpsif3 -g FO-vpsif3 /sbin/ip6tables -D libvirt-in -m physdev --physdev-in vpsif3 -g FI-vpsif3 /sbin/ip6tables -D libvirt-host-in -m physdev --physdev-in vpsif3 -g HI-vpsif3 [...] Command stderr: ip6tables v1.4.12: goto 'FO-vpsif3' is not a chain Try `ip6tables -h' or 'ip6tables --help' for more information. ip6tables v1.4.12: goto 'FI-vpsif3' is not a chain Try `ip6tables -h' or 'ip6tables --help' for more information. ip6tables v1.4.12: goto 'HI-vpsif3' is not a chain Try `ip6tables -h' or 'ip6tables --help' for more information. [...] Can you put the two rules above on priority '100'. !-- Allow TCP in $PORT -- filter name='ipv6-allow-create-state-by-port' chain='ipv6' rule action='accept' direction='in' priority='500' tcp state='NEW' dstportstart='$PORT'/ /rule rule action='accept' direction='in' priority='500' udp state='NEW' dstportstart='$PORT'/ /rule /filter !-- Allow IPv6 traffic from $RANGE -- filter name='ipv6-allow-create-state-by-range' chain='ipv6' rule action='accept' direction='out' priority='500' ipv6 srcipaddr='$RANGE' srcipmask='64'/ /rule /filter Can you put this one in priority '600'? !-- Drop all other IPv6 traffic -- filter name='ipv6-drop-stateless' chain='ipv6' rule action='drop' direction='inout' priority='999' all/ /rule /filter With the above comments 'implemented', does it 'work'? Doesn't change a thing. If not: - I'd be curious about the content of $RANGE that you're passing in. parameter name='RANGE' value='2a01:7c8::5::'/ - how does the filter look like that ties all the above filters together. The order in which the above filters are referenced matters as long as you keep the rules at priority '500'. The filters are referenced in the exact order as I pasted them. This is the same as with the IPv4-version of the ruleset. It should work with ipv6 as well as it does for ipv4, but I must say that I don't really test with IPv6 much... Stefan I use a similar approach for my IPv4 firewall, and it works perfectly. When I use these IPv6 rules, all IPv6 traffic is apparently dropped, but it's hard to debug when the result of this config is abscent in ip6tables. I'm using these version of software on debian 6.0 squeeze: virsh # version Compiled against library: libvir 0.9.2 Using library: libvir 0.9.2 Using API: QEMU 0.9.2 Running hypervisor: QEMU 0.15.0 Does anyone have any clues? Thanks in advance! Regards, Reinier Schoof -- TransIP BV | https://www.transip.nl/ -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
