Re: ezjails, systat -ifstat, and multiple network cards
On Thu, Feb 13, 2014 at 2:22 PM, Allan Jude free...@allanjude.com wrote: On 2014-02-13 13:59, Preston Hagar wrote: I have a server setup with FreeBSD-10.0-RELEASE. It has 3 Intel gigabit network cards in it, em0, em1, and em2. I have multiple ezjails setup that run various things. One jail, called db, runs a postgresql database. It was my intention to give it em0 all to itself. The other jails and host machine should be going through em2. em1 currently isn't being used. If I do an ifconfig, I see that em0 has the alias IP for my db jail and em2 has the alias IP for all other jails. All the jails respond to network traffic as expected and seemingly work fine. The weird thing is when I do a systat -ifstat from the host, it should essentially all traffic going through em0. Some of the jails that run off of em2 (as defined in their jail config files and seen in ifconfig) have large data transfers and/or are web servers with lots of photos. I have even tried to manually scp a large file out of a jail setup through em2 and the numbers don't seem to budge. If I do netstat -i -b -n -I and check em0 and em2, it seems to support the numbers shown by systat -ifstat. However, if I use trafshow or iftop (both of which require choosing one interface at a time), they both seem to indicate the traffic flowing through the interfaces as I would expect. So I was curious if anyone had seen something like this before or had any ideas of what is going on. I have net.fibs=2 set in /boot/loader.conf, but in all the jails I current have jail_name_fib= as I haven't got around to fullying setting up fibs. Is that perhaps the issue? Is there any way to determine with certainty which jail is using which interface short of physically pulling a network cable and seeing what stops working? Here are the relevant lines from my db (the one that should be on em0) config: export jail_db_hostname=db export jail_db_ip=em0|10.1.10.2 From another jail on em2 called www: export jail_www_hostname=www export jail_www_ip=em2|10.1.10.7 from ifconfig em0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=4219bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO ether 08:60:6e:13:94:06 inet 10.1.1.4 netmask 0x broadcast 10.1.255.255 inet6 fe80::a60:6eff:fe13:9406%em0 prefixlen 64 scopeid 0x1 inet 10.1.10.2 netmask 0x broadcast 10.1.10.2 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL media: Ethernet autoselect (1000baseT full-duplex) status: active em2: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=4219bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO ether 68:05:ca:13:74:2a inet 10.1.1.2 netmask 0x broadcast 10.1.255.255 inet6 fe80::6a05:caff:fe13:742a%em2 prefixlen 64 scopeid 0x3 inet 10.1.10.3 netmask 0x broadcast 10.1.10.3 inet 10.1.10.1 netmask 0x broadcast 10.1.10.1 inet 10.1.10.8 netmask 0x broadcast 10.1.10.8 inet 10.1.10.10 netmask 0x broadcast 10.1.10.10 inet 10.1.10.4 netmask 0x broadcast 10.1.10.4 inet 10.1.10.9 netmask 0x broadcast 10.1.10.9 inet 10.1.10.7 netmask 0x broadcast 10.1.10.7 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL media: Ethernet autoselect (1000baseT full-duplex) status: active Let me know if any more detail would be helpful or if you have any ideas of things to check. Thanks, Preston ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org All traffic going out from the jails will using the routing table from the host system. The routing table will use the network card that is in the same subnet as your default gateway to route the traffic to the internet. In your case, I would imagine this is 10.1.1.4/16 (and 10.1.1.2/16). 'netstat -rn' will tell the tale, but I imagine it is whichever was added first. If you want to have separate routing tables per jail, you'd have to either use FIBs, and set the jails to use the different FIBs, or use VNET jails and have a routing table in each jail. -- Allan Jude Makes sense, thank you. I'll setup the FIBs. Preston ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
ezjails, systat -ifstat, and multiple network cards
I have a server setup with FreeBSD-10.0-RELEASE. It has 3 Intel gigabit network cards in it, em0, em1, and em2. I have multiple ezjails setup that run various things. One jail, called db, runs a postgresql database. It was my intention to give it em0 all to itself. The other jails and host machine should be going through em2. em1 currently isn't being used. If I do an ifconfig, I see that em0 has the alias IP for my db jail and em2 has the alias IP for all other jails. All the jails respond to network traffic as expected and seemingly work fine. The weird thing is when I do a systat -ifstat from the host, it should essentially all traffic going through em0. Some of the jails that run off of em2 (as defined in their jail config files and seen in ifconfig) have large data transfers and/or are web servers with lots of photos. I have even tried to manually scp a large file out of a jail setup through em2 and the numbers don't seem to budge. If I do netstat -i -b -n -I and check em0 and em2, it seems to support the numbers shown by systat -ifstat. However, if I use trafshow or iftop (both of which require choosing one interface at a time), they both seem to indicate the traffic flowing through the interfaces as I would expect. So I was curious if anyone had seen something like this before or had any ideas of what is going on. I have net.fibs=2 set in /boot/loader.conf, but in all the jails I current have jail_name_fib= as I haven't got around to fullying setting up fibs. Is that perhaps the issue? Is there any way to determine with certainty which jail is using which interface short of physically pulling a network cable and seeing what stops working? Here are the relevant lines from my db (the one that should be on em0) config: export jail_db_hostname=db export jail_db_ip=em0|10.1.10.2 From another jail on em2 called www: export jail_www_hostname=www export jail_www_ip=em2|10.1.10.7 from ifconfig em0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=4219bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO ether 08:60:6e:13:94:06 inet 10.1.1.4 netmask 0x broadcast 10.1.255.255 inet6 fe80::a60:6eff:fe13:9406%em0 prefixlen 64 scopeid 0x1 inet 10.1.10.2 netmask 0x broadcast 10.1.10.2 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL media: Ethernet autoselect (1000baseT full-duplex) status: active em2: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=4219bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO ether 68:05:ca:13:74:2a inet 10.1.1.2 netmask 0x broadcast 10.1.255.255 inet6 fe80::6a05:caff:fe13:742a%em2 prefixlen 64 scopeid 0x3 inet 10.1.10.3 netmask 0x broadcast 10.1.10.3 inet 10.1.10.1 netmask 0x broadcast 10.1.10.1 inet 10.1.10.8 netmask 0x broadcast 10.1.10.8 inet 10.1.10.10 netmask 0x broadcast 10.1.10.10 inet 10.1.10.4 netmask 0x broadcast 10.1.10.4 inet 10.1.10.9 netmask 0x broadcast 10.1.10.9 inet 10.1.10.7 netmask 0x broadcast 10.1.10.7 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL media: Ethernet autoselect (1000baseT full-duplex) status: active Let me know if any more detail would be helpful or if you have any ideas of things to check. Thanks, Preston ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: ezjails, systat -ifstat, and multiple network cards
On 2014-02-13 13:59, Preston Hagar wrote: I have a server setup with FreeBSD-10.0-RELEASE. It has 3 Intel gigabit network cards in it, em0, em1, and em2. I have multiple ezjails setup that run various things. One jail, called db, runs a postgresql database. It was my intention to give it em0 all to itself. The other jails and host machine should be going through em2. em1 currently isn't being used. If I do an ifconfig, I see that em0 has the alias IP for my db jail and em2 has the alias IP for all other jails. All the jails respond to network traffic as expected and seemingly work fine. The weird thing is when I do a systat -ifstat from the host, it should essentially all traffic going through em0. Some of the jails that run off of em2 (as defined in their jail config files and seen in ifconfig) have large data transfers and/or are web servers with lots of photos. I have even tried to manually scp a large file out of a jail setup through em2 and the numbers don't seem to budge. If I do netstat -i -b -n -I and check em0 and em2, it seems to support the numbers shown by systat -ifstat. However, if I use trafshow or iftop (both of which require choosing one interface at a time), they both seem to indicate the traffic flowing through the interfaces as I would expect. So I was curious if anyone had seen something like this before or had any ideas of what is going on. I have net.fibs=2 set in /boot/loader.conf, but in all the jails I current have jail_name_fib= as I haven't got around to fullying setting up fibs. Is that perhaps the issue? Is there any way to determine with certainty which jail is using which interface short of physically pulling a network cable and seeing what stops working? Here are the relevant lines from my db (the one that should be on em0) config: export jail_db_hostname=db export jail_db_ip=em0|10.1.10.2 From another jail on em2 called www: export jail_www_hostname=www export jail_www_ip=em2|10.1.10.7 from ifconfig em0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=4219bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO ether 08:60:6e:13:94:06 inet 10.1.1.4 netmask 0x broadcast 10.1.255.255 inet6 fe80::a60:6eff:fe13:9406%em0 prefixlen 64 scopeid 0x1 inet 10.1.10.2 netmask 0x broadcast 10.1.10.2 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL media: Ethernet autoselect (1000baseT full-duplex) status: active em2: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=4219bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO ether 68:05:ca:13:74:2a inet 10.1.1.2 netmask 0x broadcast 10.1.255.255 inet6 fe80::6a05:caff:fe13:742a%em2 prefixlen 64 scopeid 0x3 inet 10.1.10.3 netmask 0x broadcast 10.1.10.3 inet 10.1.10.1 netmask 0x broadcast 10.1.10.1 inet 10.1.10.8 netmask 0x broadcast 10.1.10.8 inet 10.1.10.10 netmask 0x broadcast 10.1.10.10 inet 10.1.10.4 netmask 0x broadcast 10.1.10.4 inet 10.1.10.9 netmask 0x broadcast 10.1.10.9 inet 10.1.10.7 netmask 0x broadcast 10.1.10.7 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL media: Ethernet autoselect (1000baseT full-duplex) status: active Let me know if any more detail would be helpful or if you have any ideas of things to check. Thanks, Preston ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org All traffic going out from the jails will using the routing table from the host system. The routing table will use the network card that is in the same subnet as your default gateway to route the traffic to the internet. In your case, I would imagine this is 10.1.1.4/16 (and 10.1.1.2/16). 'netstat -rn' will tell the tale, but I imagine it is whichever was added first. If you want to have separate routing tables per jail, you'd have to either use FIBs, and set the jails to use the different FIBs, or use VNET jails and have a routing table in each jail. -- Allan Jude signature.asc Description: OpenPGP digital signature
