On Jun 3, 2013, at 6:33 AM, Pietro Paolini wrote: > > On Jun 1, 2013, at 5:26 AM, "Teske, Devin" <devin.te...@fisglobal.com> wrote: > >> >> On May 31, 2013, at 3:05 PM, Teske, Devin wrote: >> >>> >>> On May 31, 2013, at 1:48 AM, Pietro Paolini wrote: >>> >>>> >>>> On May 30, 2013, at 6:25 PM, "Teske, Devin" <devin.te...@fisglobal.com> >>>> wrote: >>>> >>>>> >>>>> On May 30, 2013, at 3:35 AM, Pietro Paolini wrote: >>>>> >>>>>> Hello all, >>>>>> >>>>>> I am a new bye on the FreeBSD and I am looking at the VIMAGE features >>>>>> experiencing some problems. >>>>>> I added the options : >>>>>> VIMAGE >>>>>> if_bridge >>>>>> >>>>>> and I removed >>>>>> STCP >>>>>> >>>>>> then I recompiled my kernel and install it. >>>>>> >>>>>> After that, following this tutorial >>>>>> https://urldefense.proofpoint.com/v1/url?u=http://imunes.tel.fer.hr/virtnet/eurobsdcon07_tutorial.pdf&k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D%0A&r=Mrjs6vR4%2Faj2Ns9%2FssHJjg%3D%3D%0A&m=xe0XNgnKBiT9v8HzxwWwnNMOVN3YdEHmTsIZfFoQA9Y%3D%0A&s=b271820faf310ac274ded8c3135b4931f2a5278e78ec1a66ce6a5ab971ff64f5 >>>>>> I tried the "Exercise 2" which consist on >>>>>> the following commands: >>>>>> >>>>>> vimage -c n1 >>>>>> vimage -c n2 >>>>>> ngctl mkpeer efface ether ether >>>>>> ngctl mkpeer efface ether ether >>>>> >>>>> Don't you just love autocorrect? (does the same thing to me… turns >>>>> "eiface" into "efface") >>>>> >>>>> >>>>>> ngctl mkpeer em0: bridge lower link0 >>>>> >>>>> Looks good. >>>>> >>>>> >>>>>> ngctl name em0:lower bridge0 >>>>> >>>>> I usually do my "connect" before the "name"… but shouldn't matter. Should >>>>> work all the same. >>>>> >>>>> >>>>>> ngctl connect em0: bridge0: upper link1 >>>>> >>>>> This looks wrong to me. >>>>> >>>>> I'd expect: >>>>> >>>>> ngctl connect em0: bridge0:lower upper link1 >>>>> >>>> >>>> >>>> Many thanks for the answer Devin, >>>> when I try to use that last command I receive: >>>> >>>> ngctl connect em0: bridge0:lower upper link1 >>>> ngctl: send msg: Invalid argument >>>> >>>> What's wrong ? >>>> >>> >>> Let's start from scratch on a freshly booted box… >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l >>> [sudo] Password: >>> There are 4 total nodes: >>> Name: em0 Type: ether ID: 00000002 Num hooks: 0 >>> Name: em1 Type: ether ID: 00000003 Num hooks: 0 >>> Name: ngctl1719 Type: socket ID: 00000004 Num hooks: 0 >>> Name: msk0 Type: ether ID: 00000001 Num hooks: 0 >>> >>> Ok… we have an "ether" type node for each of our physical adapters (these >>> are provided by ng_ether(4); you didn't have to do anything to get these >>> nodes). >>> >>> We also have a single "socket" type node. This is the "ngctl" connection to >>> the netgraph subsystem (you can learn more by reading ng_socket(4)). >>> >>> Here's the corresponding hardware behind em0, em1, and msk0: >>> >>> === >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ grep '\(em\|e1000phy\|mskc\?\)[[:digit:]]' >>> /var/run/dmesg.boot >>> mskc0: <Marvell Yukon 88E8050 Gigabit Ethernet> port 0xdc00-0xdcff mem >>> 0xfcffc000-0xfcffffff irq 16 at device 0.0 on pci5 >>> msk0: <Marvell Technology Group Ltd. Yukon EC Id 0xb6 Rev 0x02> on mskc0 >>> msk0: Ethernet address: xx:xx:xx:xx:xx:xx >>> miibus0: <MII bus> on msk0 >>> e1000phy0: <Marvell 88E1111 Gigabit PHY> PHY 0 on miibus0 >>> e1000phy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, >>> 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto >>> mskc0: [ITHREAD] >>> em0: <Intel(R) PRO/1000 Legacy Network Connection 1.0.3> port 0xec80-0xecbf >>> mem 0xfebe0000-0xfebfffff irq 16 at device 4.0 on pci7 >>> em0: [FILTER] >>> em0: Ethernet address: xx:xx:xx:xx:xx:xx >>> em1: <Intel(R) PRO/1000 Legacy Network Connection 1.0.3> port 0xec00-0xec3f >>> mem 0xfeba0000-0xfebbffff,0xfeb80000-0xfeb9ffff irq 18 at device 6.0 on pci7 >>> em1: [FILTER] >>> em1: Ethernet address: xx:xx:xx:xx:xx:xx >>> em0: link state changed to UP >>> >>> === >>> >>> Next, let's make a bridge (think of it as a big software switch that we're >>> going to hook a bunch of interfaces; created, physical, or otherwise). >>> >>> Since I'm doing this over an SSH connection (a mistake I made earlier >>> today), I'm not going to touch em0 (the adapter my SSH connection is >>> using). Creating the bridge on an actively configured PHY will knock it off >>> the net. This is not to say you can't have an active configuration on a >>> bridged interface… just that the creation of the bridge (something you >>> should only do once each time you boot) will disrupt an active connection. >>> >>> So… >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl mkpeer em1: bridge lower link0 >>> >>> NOTE: No output == Success. >>> >>> === >>> >>> Now let's look at our handiwork… >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl info em1:lower >>> Name: <unnamed> Type: bridge ID: 00000007 Num hooks: 1 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> link0 em1 ether 00000003 lower >>> >>> >>> Ok, we see that the lower peer hook of the em1 ether-node goes off to >>> something named "link0". >>> >>> To see where link0 is off-to… we need a full listing (back to "ngctl ls >>> -l"). >>> >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l >>> There are 5 total nodes: >>> Name: <unnamed> Type: bridge ID: 00000007 Num hooks: 1 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> link0 em1 ether 00000003 lower >>> Name: em0 Type: ether ID: 00000002 Num hooks: 0 >>> Name: em1 Type: ether ID: 00000003 Num hooks: 1 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> lower <unnamed> bridge 00000007 link0 >>> Name: ngctl1762 Type: socket ID: 0000000b Num hooks: 0 >>> Name: msk0 Type: ether ID: 00000001 Num hooks: 0 >>> >>> >>> Matching "link0" in the first column to "link0" in the last-column, we can >>> see that this lower-link0 is to a bridge (with no name). >>> >>> NOTE: When you're digesting the above output… it helps to imagine >>> whitespace in between the nodes with their respective hooks and other >>> nodes. Future pastes below will introduce such whitespace to make it easier >>> to read. >>> >>> === >>> >>> Right now, the only way to refer to the bridge is by way of "em1:lower" >>> (because we created the bridge right on the lower hook of the em1 >>> ether-node). >>> >>> At this point, let's talk about naming. Giving our bridge a name is >>> entirely optional, but greatly clarifies the output of both "ngctl ls -l" >>> and "ngctl dot". >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl name em1:lower em1bridge >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l >>> There are 5 total nodes: >>> Name: em0 Type: ether ID: 00000002 Num hooks: 0 >>> >>> Name: em1 Type: ether ID: 00000003 Num hooks: 1 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> lower em1bridge bridge 00000007 link0 >>> >>> Name: ngctl1831 Type: socket ID: 0000001a Num hooks: 0 >>> >>> Name: em1bridge Type: bridge ID: 00000007 Num hooks: 1 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> link0 em1 ether 00000003 lower >>> >>> Name: msk0 Type: ether ID: 00000001 Num hooks: 0 >>> >>> The new "em1bridge" name acts as an alias to "em1:lower" in future ngctl >>> commands. For example, "ngctl info em1:lower" and "ngctl info em1bridge" >>> can now be used interchangeably and produce the same results. >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl info em1bridge: >>> Name: em1bridge Type: bridge ID: 00000007 Num hooks: 1 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> link0 em1 ether 00000003 lower >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl info em1:lower >>> Name: em1bridge Type: bridge ID: 00000007 Num hooks: 1 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> link0 em1 ether 00000003 lower >>> >>> === >>> >>> We're not done with the bridge yet. Because we foresee the possibility that >>> it might be nice to be able to communicate with the jail that we're going >>> to later hook into this bridge… we should hook the physical adapter's >>> "upper" hook into the bridge. >>> >>> If you don't do this, you won't be able to (for example) ping a jail from >>> the host where the host has only the PHY and the jail has only a (yet >>> uncreated) eiface. Regardless of the fact that the bridge uses the PHY and >>> the jail uses the bridge, to communicate with an IP that is configured on >>> the base host, you must hook the upper. >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl connect em1: em1:lower upper >>> link1 >>> >>> If you want to use the alias I set up earlier (of "em1bridge") that works >>> too (just don't forget the colon at the end of the alias): >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl connect em1: em1bridge: upper >>> link1 >>> >>> Here's the results: >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l >>> There are 5 total nodes: >>> Name: em0 Type: ether ID: 00000002 Num hooks: 0 >>> >>> Name: em1 Type: ether ID: 00000003 Num hooks: 2 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> upper em1bridge bridge 0000002a link1 >>> lower em1bridge bridge 0000002a link0 >>> >>> Name: ngctl1874 Type: socket ID: 00000030 Num hooks: 0 >>> >>> Name: em1bridge Type: bridge ID: 0000002a Num hooks: 2 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> link1 em1 ether 00000003 upper >>> link0 em1 ether 00000003 lower >>> >>> Name: msk0 Type: ether ID: 00000001 Num hooks: 0 >>> >>> >>> NOTE: Some of the Peer ID's have changed, because I wanted to test that the >>> alias could be used; I used "sudo ngctl shutdown em1bridge:" and >>> re-executed up to the point where I connect the em1:upper into the bridge… >>> except this time using the alias of "em1bridge" instead of "em1:lower" >>> (indeed, you can use them interchangeably). >>> >>> === >>> >>> Ok… We've now done the hard part… which was to create and configure a >>> bridge that is usable by any new nodes we connect to it and also (if you >>> hooked the upper portion of em1 back into its own lower which is acting as >>> the bridge) the base machine can communicate with any of the forth-coming >>> jails (if on the same subnet at least). >>> >>> There's an easy step that shouldn't be skipped though… >>> >>> Before you can truly use this bridge with any other interfaces… >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ifconfig em1 up >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1: setpromisc 1 >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1: setautosrc 0 >>> >>> A bridge cannot send packets out if the interface is down. >>> A bridge cannot work properly without promiscuous mode. >>> A bridge cannot send out packets for different addresses unless you turn >>> off "setautosrc" >>> >>> === >>> >>> Let's create our first virtual NIC and connect it to the bridge. >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl mkpeer em1bridge: eiface link2 >>> ether >>> >>> This command did two things. It created a new "eiface" node (see >>> ng_eiface(4)), and connected it to the bridge. >>> >>> Let's have a look: >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l >>> There are 6 total nodes: >>> Name: em0 Type: ether ID: 00000002 Num hooks: 0 >>> >>> Name: em1 Type: ether ID: 00000003 Num hooks: 2 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> upper em1bridge bridge 0000002a link1 >>> lower em1bridge bridge 0000002a link0 >>> >>> Name: ngeth0 Type: eiface ID: 00000035 Num hooks: 1 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> ether em1bridge bridge 0000002a link2 >>> >>> Name: ngctl2800 Type: socket ID: 00000036 Num hooks: 0 >>> >>> Name: em1bridge Type: bridge ID: 0000002a Num hooks: 3 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> link2 ngeth0 eiface 00000035 ether >>> link1 em1 ether 00000003 upper >>> link0 em1 ether 00000003 lower >>> >>> Name: msk0 Type: ether ID: 00000001 Num hooks: 0 >>> >>> The list of hooks for our bridge (em1bridge) is growing, and now we see a >>> new node (ngeth0) with one hook into that bridge. >>> >>> === >>> >>> ASIDE: If you wanted to script this… here's how you can test for an unused >>> link: >>> >>> Right now, we have link0, link1, and link2 for the bridge. If a link exists >>> for a bridge, the following command will return some info about the link >>> and return success (whereas if the link does not exist, the command will >>> return an error and exit with error-status): >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1bridge: getstats 0 >>> Rec'd response "getstats" (4) from "[2a]:": >>> Args: >>> {} >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1bridge: getstats 1 >>> Rec'd response "getstats" (4) from "[2a]:": >>> Args: >>> {} >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1bridge: getstats 2 >>> Rec'd response "getstats" (4) from "[2a]:": >>> Args: >>> {} >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1bridge: getstats 3 >>> ngctl: send msg: Socket is not connected >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1bridge: getstats 4 >>> ngctl: send msg: Socket is not connected >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1bridge: getstats 5 >>> ngctl: send msg: Socket is not connected >>> >>> As you can see from the above output… we get errors for link3, link4, and >>> link5, because they don't exist. Naturally, testing $? exit status after >>> each of these commands would show how this can be scripted (HINT: throw >>> stdout/stderr to /dev/null and test $?). >>> >>> === >>> >>> At this point… you say "ifconfig": >>> >>> dte...@oos0a.lbxrich.vicor.com ~ $ ifconfig >>> msk0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 >>> >>> options=c011a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,VLAN_HWTSO,LINKSTATE> >>> ether xx:xx:xx:xx:xx:xx >>> media: Ethernet autoselect >>> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 >>> >>> options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC> >>> ether xx:xx:xx:xx:xx:xx >>> inet xx.xx.xx.xx netmask 0xffffff80 broadcast xx.xx.xx.xx >>> media: Ethernet autoselect (1000baseT <full-duplex>) >>> status: active >>> em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 >>> mtu 1500 >>> >>> options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC> >>> ether xx:xx:xx:xx:xx:xx >>> media: Ethernet autoselect >>> status: no carrier >>> ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536 >>> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 >>> options=3<RXCSUM,TXCSUM> >>> inet 127.0.0.1 netmask 0xff000000 >>> ngeth0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 >>> ether 00:00:00:00:00:00 >>> >>> === >>> >>> Ok, there are two problems with the network interface. >>> >>> 1. It has a NULL MAC address (00:00:00:00:00:00). Good luck communicating >>> on the Internet (remember, we disabled setautosrc -- we intend to make up a >>> MAC address that is unique). >>> >>> 2. The name leaves something to be desired (if we're going to use this with >>> a vimage jail, it would be nice if the interface had the jail name in it, >>> so that when you do an "ngctl ls -l" or an "ngctl dot" … you're going to >>> see the jail name so it becomes clear which jails are hooked to which PHY's >>> through which bridges). >>> >>> === >>> >>> Let's tackle the easier one first… let's rename this new interface. >>> >>> You and I already know that this interface that we want to rename is >>> "ngeth0"… but you can actually extract the name from the link in the bridge. >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl show -n em1bridge:link2 >>> Name: ngeth0 Type: eiface ID: 00000035 Num hooks: 1 >>> >>> >>> First, we rename it in netgraph (this does not affect the output of >>> ifconfig -- and again, we do this to make "ngctl ls -l" and "ngctl dot" >>> more palatable): >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl name em1bridge:link2 ng0_myjail >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l >>> There are 6 total nodes: >>> Name: em0 Type: ether ID: 00000002 Num hooks: 0 >>> >>> Name: em1 Type: ether ID: 00000003 Num hooks: 2 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> upper em1bridge bridge 0000002a link1 >>> lower em1bridge bridge 0000002a link0 >>> >>> Name: ngctl2843 Type: socket ID: 00000046 Num hooks: 0 >>> >>> Name: ng0_myjail Type: eiface ID: 00000035 Num hooks: 1 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> ether em1bridge bridge 0000002a link2 >>> >>> Name: em1bridge Type: bridge ID: 0000002a Num hooks: 3 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> link2 ng0_myjail eiface 00000035 ether >>> link1 em1 ether 00000003 upper >>> link0 em1 ether 00000003 lower >>> >>> Name: msk0 Type: ether ID: 00000001 Num hooks: 0 >>> >>> >>> Looking good. However, ifconfig hasn't changed… >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ ifconfig >>> ... >>> ngeth0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 >>> ether 00:00:00:00:00:00 >>> >>> We want to rename the interface with ifconfig for a different reason. >>> >>> We renamed the interface with netgraph earlier so that netgraph outputs >>> would be nice and easy to digest. >>> >>> This time, we rename with ifconfig so that we can layer jails onto the same >>> rootdir. >>> >>> The naming convention (which is the same naming convention I use for >>> renaming on the netgraph side) is: >>> >>> ng#_name >>> >>> The # always starts at zero for each jail where "name" is the name of the >>> jail. >>> >>> Again… I use this scheme so that I can layer jails onto the same root-dir; >>> /etc/rc.conf is then populated with things like: >>> >>> ifconfig_ng0_myjail=... >>> ifconfig_ng0_myrouter=... >>> ifconfig_ng1_myrouter=... >>> ifconfig_ng0_anotherjail=... >>> >>> So that when you say "service netif start" inside the vnet jail… it applies >>> the right settings. >>> >>> So… we rename with ifconfig: >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ifconfig ngeth0 name ng0_myjail >>> dte...@scu0a.jbsd.vicor.com ~ $ ifconfig >>> ... >>> ng0_myjail: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 >>> ether 00:00:00:00:00:00 >>> >>> === >>> >>> We're almost ready to shove this interface into a jail (which we haven't >>> created yet). >>> >>> But… we come back to that NULL MAC address. >>> >>> NOTE: Forming your own MAC address, or even coming up with your own formula >>> should not be taken lightly. >>> >>> Here's a formula I use (which is based on several RFC's for MAC address >>> formation): >>> >>> NOTE: In this context, ${_bridge} is em1 and $LINKNUM is 2 >>> >>> # Set the MAC address of the new interface >>> # using a sensible algorithm to prevent >>> # conflicts on the network. >>> # >>> # MAC LP:LL:LB:BB:BB:BB >>> # P 2, 6, A, or E but usually 2 >>> # NOTE: Indicates "privately administered" MAC >>> # L ng_bridge(4) link number (1-65535) >>> # B Same as bridged interface >>> # >>> _bridge_ether=$( ifconfig ${_bridge} ether | >>> awk '/ether/{print $2}' ) >>> _ether_devid="${_bridge_ether#??:??:?}" >>> n=$LINKNUM >>> _quad=$(($n & 15)) >>> case "${_quad}" in >>> 10) _quad=a;; 11) _quad=b;; 12) _quad=c;; >>> 13) _quad=d;; 14) _quad=e;; 15) _quad=f;; >>> esac >>> _ether_devid=":${_quad}${_ether_devid}" >>> n=$(($n >> 4)) >>> _quad=$(($n & 15)) >>> case "${_quad}" in >>> 10) _quad=a;; 11) _quad=b;; 12) _quad=c;; >>> 13) _quad=d;; 14) _quad=e;; 15) _quad=f;; >>> esac >>> _ether_devid="${_quad}${_ether_devid}" >>> n=$(($n >> 4)) >>> _quad=$(($n & 15)) >>> case "${_quad}" in >>> 10) _quad=a;; 11) _quad=b;; 12) _quad=c;; >>> 13) _quad=d;; 14) _quad=e;; 15) _quad=f;; >>> esac >>> _ether_devid="2:${_quad}${_ether_devid}" >>> n=$(($n >> 4)) >>> _quad=$(($n & 15)) >>> case "${_quad}" in >>> 10) _quad=a;; 11) _quad=b;; 12) _quad=c;; >>> 13) _quad=d;; 14) _quad=e;; 15) _quad=f;; >>> esac >>> _ether_devid="${_quad}${_ether_devid}" >>> n=$(($n >> 4)) >>> >>> After which… ${_ether_devid} holds a properly formed MAC address that can >>> (in every case I've tested) "get out". >>> >>> Here's what I do to set it: >>> >>> ifconfig ng0_myjail ether "${_ether_devid}" >>> >>> Here's an example of how the MAC address was translated from the physical >>> adapter to the ng_eiface(4) interface: >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ ifconfig em1; ifconfig ng0_myjail >>> em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 >>> mtu 1500 >>> options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC> >>> ether 00:0e:0c:ab:1b:76 >>> media: Ethernet autoselect >>> status: no carrier >>> ng0_myjail: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 >>> ether 02:00:2c:ab:1b:76 >>> >>> === >>> >>> OK… we're now ready to shove that interface into a vimage jail. >>> >>> But… >>> >>> First we need a vimage jail. (this is not a tutorial on how to create, >>> manage, build, or do anything else with jails, vimage-jails, or vps-jails >>> *other* than give it a netgraph based interface) >>> >>> I'm going to use my existing base machine as a fake jail (by pointing my >>> jail's rootdir at "/"). >>> >>> NOTE: Certain sysctl's have to be set appropriately before you fire up the >>> jail to make this vimage jail able to do "more" on the net. >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo sysctl >>> security.jail.set_hostname_allowed=1 security.jail.sysvipc_allowed=1 >>> security.jail.socket_unixiproute_only=1 >>> security.jail.set_hostname_allowed: 1 -> 1 >>> security.jail.sysvipc_allowed: 1 -> 1 >>> security.jail.socket_unixiproute_only: 0 -> 1 >>> >>> NOTE: Unless you intend to reboot to restore the defaults later… you might >>> want to take down those previous values for restoration *after* we fire up >>> the "vimage" jail. >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo jail -i -c vnet name=myjail >>> host.hostname=myjail path=/ persist >>> 1 >>> dte...@scu0a.jbsd.vicor.com ~ $ jls >>> JID IP Address Hostname Path >>> 1 - myjail / >>> >>> OK… we have a running jail (with the vnet property, making it a "vimage" >>> jail -- which can accept network interfaces). >>> >>> === >>> >>> Right now our jail has no network interfaces (well, it has an unconfigured >>> lo0). >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo jexec myjail ifconfig >>> lo0: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384 >>> options=3<RXCSUM,TXCSUM> >>> >>> So let's pass the netgraph created interface into the jail… >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo ifconfig ng0_myjail vnet 1 >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo jexec myjail ifconfig >>> lo0: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384 >>> options=3<RXCSUM,TXCSUM> >>> ng0_myjail: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 >>> ether 02:00:2c:ab:1b:76 >>> >>> Sweet! >>> >>> === >>> >>> Almost there… >>> >>> Let's go into /etc/rc.conf, give it an IP, and start the network… >>> >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo sysrc ifconfig_ng0_myjail="inet >>> 192.168.1.1 netmask 255.255.255.0" >>> /etc/rc.conf: ifconfig_ng0_myjail: -> inet 192.168.1.1 netmask >>> 255.255.255.0 >>> dte...@scu0a.jbsd.vicor.com ~ $ grep ng0 /etc/rc.conf >>> ifconfig_ng0_myjail="inet 192.168.1.1 netmask 255.255.255.0" >>> dte...@scu0a.jbsd.vicor.com ~ $ sudo jexec myjail service netif start >>> Starting Network: lo0 ng0_myjail. >>> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 >>> options=3<RXCSUM,TXCSUM> >>> inet 127.0.0.1 netmask 0xff000000 >>> ng0_myjail: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu >>> 1500 >>> ether 02:00:2c:ab:1b:76 >>> inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 >>> >>> Now we're cookin' with gasoline! >>> >>> === >>> >>> Optionally go configure your base machine with an IP and have fun. >> >> A quick conclusion… >> >> Because we've built this all on top of netgraph… we can … graph it. >> >> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl dot | dot -Tsvg -o >> netgraph-scu0a.svg >> >> I then uploaded the file to the web and here it is: >> >> https://urldefense.proofpoint.com/v1/url?u=http://druidbsd.sourceforge.net/download/netgraph-scu0a.svg&k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D%0A&r=Mrjs6vR4%2Faj2Ns9%2FssHJjg%3D%3D%0A&m=xe0XNgnKBiT9v8HzxwWwnNMOVN3YdEHmTsIZfFoQA9Y%3D%0A&s=0a6ee990c4871ecc2a5b69879d11f65798fe70d8948fb9d4592bd915b1fd0199 >> >> You should compare this directly to the output of "ngctl ls -l": >> >> dte...@scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l >> There are 6 total nodes: >> Name: em0 Type: ether ID: 00000002 Num hooks: 0 >> >> Name: em1 Type: ether ID: 00000003 Num hooks: 2 >> Local hook Peer name Peer type Peer ID Peer hook >> ---------- --------- --------- ------- --------- >> upper em1bridge bridge 0000002a link1 >> lower em1bridge bridge 0000002a link0 >> >> Name: ng0_myjail Type: eiface ID: 00000035 Num hooks: 1 >> Local hook Peer name Peer type Peer ID Peer hook >> ---------- --------- --------- ------- --------- >> ether em1bridge bridge 0000002a link2 >> >> Name: em1bridge Type: bridge ID: 0000002a Num hooks: 3 >> Local hook Peer name Peer type Peer ID Peer hook >> ---------- --------- --------- ------- --------- >> link2 ng0_myjail eiface 00000035 ether >> link1 em1 ether 00000003 upper >> link0 em1 ether 00000003 lower >> >> Name: ngctl8676 Type: socket ID: 00000049 Num hooks: 0 >> >> Name: msk0 Type: ether ID: 00000001 Num hooks: 0 >> >> You'll notice that when you graph the layout with "ngctl dot", the nodes are >> rendered as boxes displaying their "Peer Name" up top, their "Peer Type" in >> the lower-left, and their "Peer ID" in the bottom-right. >> >> The edges from one node to another contains two octagons. These are the >> "Local hook" and "Peer hook". >> -- >> Devin >> >> _____________ >> The information contained in this message is proprietary and/or >> confidential. If you are not the intended recipient, please: (i) delete the >> message and all copies; (ii) do not disclose, distribute or use the message >> in any manner; and (iii) notify the sender immediately. In addition, please >> be aware that any message addressed to our domain is subject to archiving >> and review by persons other than the intended recipient. Thank you. > > Hello Devin, > > If you live in the same city I will invite you for a couple of beer (I have > to pay of course!) - I live in the Netherlands then let me know - >
If I ever make it out there, I'll certainly shoot you an e-mail. > I followed your tutorial with the expected result: I can ping the em1 > interface but I still have the same problems of before regarding external to > internal networks communication. > > Please note that on my original host (no jail) the default gateway is > 192.168.1.254, that's important for what I am going to do. > > I added a default route like : > > route add default -interface ng0_myjail > Why not: jexec myjail route add -net default 192.168.1.254 ?? -- Devin > and then I try to : > > jexec myjail ping 8.8.8.8 > > I analyzed the wireshark capture and I can see that an ARP request for the > 192.168.254 (with the MAC address of our virtual NIC as source, as expected) > go out my freebsd env - which is on a virtualbox - and it gets the answer but > when I read the ARP table of the jail I can see it as incomplete. It seems > that packet going out but the answer is not received and it is confirmed when > I try to sniff with tcpdump on my FREEBSD, I can't see any ARP request going > in both from my physical and virtual NIC, the same if I try : > > ping 192.168.1.254 > > Then I can see ping reply coming from wireshark but not from tcpdump on > FreeBDS. > > Wireshark is attached on the host machine on the physical interface where VB > is attached in Bridged Mode, my original physical interface on FreeBSD took > the IP address from DHCP without problem then the problem is related to the > jail. > > do I try do accomplish a task which is not possible with JAIL or something is > wrong in my configuration or worst, in my brain :P ? > > Thanks in advance, > Pietro. > > > > > > _____________ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. _______________________________________________ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
