Hello list

I cant get to the internet using this netgraph setup script.
I sure would appreciate giving this console log a look over for
errors. My netgraph knowledge level is not sufficient to see what is
wrong. The goal is to run this script to setup and break down a netgraph
network for a single vnet jail at a time. rl0 is the real nic interface
device name of the nic facing the internet. This box is on my lan and
the gateway box does NAT for all lan boxes. The host running this script can ping the internet ok.

Thank you very much for your help.





The host's kernel has modules with vimage & ipfw compiled in.

From the host
# /root >ifconfig
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
        options=2008<VLAN_MTU,WOL_MAGIC>
        ether 00:0c:6e:09:8b:74
        inet 10.0.10.5 netmask 0xfffffff8 broadcast 10.0.10.7
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
plip0: flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> metric 0 mtu 1500
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x8
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

The jails config file
# /root >cat /usr/local/etc/vnet/vdir4
vdir4 {
host.hostname       =  "vdir4";
path                =  "/usr/jails/vdir4";
mount.fstab         =  "/usr/local/etc/fstab/vdir4";
vnet;
persist;
}

The netgraph script
# /root >cat /usr/local/bin/vnet.ng.test
#!/bin/sh
# snip comments for displaying here
# This script is based on this /usr/share/examples/netgraph/virtual.lan

# Give the name of ethernet interface.
ETHER_INTF="rl0"

# List the names of virtual nodes and their IP addresses. Use ':'
# character to separate node name from node IP address and netmask.

#TARGET_TOPOLOGY="c1|10.0.2.20/24 c2|10.0.2.21/24 c3|10.0.2.22/24"
TARGET_TOPOLOGY="vdir4|10.0.2.20/24"

# MAC manufacturer prefix. This can be modified according to needs.
MAC_PREFIX="00:1d:92"

# Temporary file is important for proper execution of script.
TEMP_FILE="/var/tmp/virtual.lan.tmp"

virtual_lan_start() {

# Load netgraph KLD's as necessary.

for KLD in ng_ether ng_bridge ng_eiface; do
        if ! kldstat -v | grep -qw ${KLD}; then
                echo -n "Loading ${KLD}.ko... "
                kldload ${KLD} || exit 1
                echo "done"
        fi
done

# Reset all interfaces and jails. If temporary file can not be found
# script assumes that there is no previous configuration.

if [ ! -e ${TEMP_FILE} ]; then
  echo "No previous configuration(${TEMP_FILE}) found to clean-up."
else
  echo -n "Cleaning previous configuration..."
  virtual_lan_stop
  echo "done"
fi

# Create temporary file for usage. This file includes generated
# interface names and jail names. All bridges, interfaces and jails
# are written to file while created. In clean-up process written
# objects are cleaned (i.e. removed) from system.

if [ -e ${TEMP_FILE} ]; then
        touch ${TEMP_FILE}
fi

echo -n "Verifying ethernet interface existence..."
# Verify ethernet interface exist.
if ! ngctl info ${ETHER_INTF}: >/dev/null 2>&1; then
        echo "Error: interface ${ETHER_INTF} does not exist"
        exit 1
fi

ifconfig ${ETHER_INTF} up || exit 1
echo "done"

# Get current number of bridge interfaces in the system. This number
# is used to create a name for new bridge.
BRIDGE_COUNT=`ngctl l | grep bridge | wc -l | sed -e "s/ //g"`
BRIDGE_NAME="bridge${BRIDGE_COUNT}"

# Create new ng_bridge(4) node and attach it to the ethernet interface.
# Connect ng_ether:lower hook to bridge:link0 when creating bridge and
# connect ng_ether:upper hook to bridge:link1 after bridge name is set.

echo "Creating bridge interface: ${BRIDGE_NAME}..."
ngctl mkpeer ${ETHER_INTF}: bridge lower link0 || exit 1
ngctl name ${ETHER_INTF}:lower ${BRIDGE_NAME} || exit 1
ngctl connect ${ETHER_INTF}: ${BRIDGE_NAME}: upper link1 || exit 1
echo "Bridge ${BRIDGE_NAME} is created and ${ETHER_INTF} is connected."

# In the above code block two hooks are connected to bridge interface,
# therefore LINKNUM is set to 2 indicating total number of connected
# hooks on the bridge interface.
LINKNUM=2

# Write name of the bridge to temp file. Clean-up procedure will use
# this name to shutdown bridge interface.
echo "bridge ${BRIDGE_NAME}" > ${TEMP_FILE}


# Attach vnet jail.
for NODE in ${TARGET_TOPOLOGY}; do

  # Virtual nodes are defined in TARGET_TOPOLOGY variable. They
  # have the form of 'nodeName|IPaddr'. Below two lines split
  # node definition to get node name and node IP.

  NODE_NAME=`echo ${NODE} | awk -F"|" '{print $1}'`
  NODE_IP=`echo ${NODE} | awk -F"|" '{print $2}'`

  # Create virtual node (jail) with given name
  echo -n "Creating virtual node (jail) ${NODE_NAME}..."

  jail -f "/usr/local/etc/vnet/${NODE_NAME}" -c ${NODE_NAME}
  if [ $? -ne 0 ]; then
    echo "Error: /usr/sbin/jail failed to start jail ${NODE_NAME}."
    virtual_lan_stop
    exit 2
  fi
  echo "done"

  # Write name of the jail to temp file. Clean-up procedure will
  # use this name to remove jail.

  echo "node ${NODE_NAME}" >> ${TEMP_FILE}

  # Create a ng_eiface object for virtual node. ng_eiface
  # object has a hook that can be connected to one of bridge
  # links. After creating interface get its automatically
  # generated name for further usage.

  echo "Creating eiface interface for virtual node ${NODE_NAME}."
  ngctl mkpeer eiface ether ether
  EIFACE=`ngctl l | grep ngeth | tail -n 1| awk '{print $2}'`
  echo "Interface ${EIFACE} is created."

  # Write name of the interface to temp file. Clean-up procedure
  # will use this name to shutdown interface.

  echo "interface ${EIFACE}" >> ${TEMP_FILE}

  # Move virtual interface to virtual node. Note that Interface
  # name will not be changed at the end of this movement. Moved
  # interface can be seen at the output of ifconfig command in
  # jail: 'jexec jailname ifconfig'

  echo "Moving ${EIFACE} to ${NODE_NAME}"
  ifconfig ${EIFACE} vnet ${NODE_NAME}

  # Make lo0 interface localhost.
  jexec ${NODE_NAME} ifconfig lo0 localhost

  # Generate a random mac address for virtual interface. First
  # three octets can be changed by user. Last three octets are
  # generated randomly.
  M4=`od -An -N2 -i /dev/random | sed -e 's/ //g' | \
                awk '{ print $1 % 256 }'`
  M5=`od -An -N2 -i /dev/random | sed -e 's/ //g' | \
                awk '{ print $1 % 256 }'`
  M6=`od -An -N2 -i /dev/random | sed -e 's/ //g' | \
        awk '{ print $1 % 256 }'`

  MAC=`printf ${MAC_PREFIX}:%02x:%02x:%02x ${M4} ${M5} ${M6}`

  # Set the link address (mac address) of virtual interface in
  # virtual node to randomly generated MAC.
  echo "Setting MAC address of ${EIFACE} to '${MAC}'"
  jexec ${NODE_NAME} ifconfig ${EIFACE} link $MAC

  # Either IPv4 or IPv6 can be used in this script. Ifconfig
  # IP setting syntax differs slightly for two IP versions.
  # For version 4 'inet' keyword is used whereas for version 6
  # 'inet6' is used. Below line tries to decide which IP version
  # is given and sets IPVER to 'inet' or 'inet6'.

  IPVER=`echo ${NODE_IP} | awk -F"." '{ split($4,last,"/"); \
        if( NF==4 && $1>0 && $1<256 && $2<256 && $3<256 && \
        last[1]<256) print "inet"; else print "inet6"}'`

  # Set IP address of virtual interface in virtual node.
  echo "Setting IP address of ${EIFACE} to '${NODE_IP}'"
  jexec ${NODE_NAME} ifconfig ${EIFACE} ${IPVER} ${NODE_IP}

  # Connect virtual interface to bridge interface. Syntax is :
  # ngctl connect INTERFACE: BRIDGE: INTERFACE_HOOK EMPTY_LINK.
  # Interface has one hook named 'ether' and below line connects
  # ether hook to bridge's first unconnected link.

  echo -n "Connecting ${EIFACE}:ether to ${BRIDGE_NAME}:link${LINKNUM}..."

  ngctl connect ${EIFACE}: ${BRIDGE_NAME}: ether link${LINKNUM} \
        || exit 1
  echo "done"

  # Now, bridge has one more connected link thus link count is
  # incremented.
  LINKNUM=`expr ${LINKNUM} + 1`
done
echo "Virtual LAN established successfully!"

}


# Stop routine.
virtual_lan_stop() {

if [ ! -e ${TEMP_FILE} ]; then
  echo "Nothing to stop! ${TEMP_FILE}: temp file not found"
else
  echo -n "Shutdown bridge interface.."
  OBJECTS=`cat ${TEMP_FILE} | grep bridge | awk '{print $2}'`
  for BRIDGE in ${OBJECTS}; do
    ngctl shutdown ${BRIDGE}: >/dev/null 2>&1
  done
  echo "done"

  echo -n "Shutdown all eiface interfaces..."
  OBJECTS=`cat ${TEMP_FILE} | grep interface | awk '{print $2}'`
  for INTERFACE in ${OBJECTS}; do
    ngctl shutdown ${INTERFACE}: >/dev/null 2>&1
  done
  echo "done"

  echo -n "Removing all jails..."
  OBJECTS=`cat ${TEMP_FILE} | grep node | awk '{print $2}'`
  for NODE in ${OBJECTS}; do
     jail -f "/usr/local/etc/vnet/${NODE}" -r ${NODE}
  done
  echo "done"

  echo "Removing tempfile ${TEMP_FILE}"
  rm ${TEMP_FILE}
fi
echo "Virtual LAN objects removed successfully!"

}


# Main entry point.

case $# in
        1)
                case $1 in
                        start)
                                echo -n "Creating default target topology:"
                                echo " ${TARGET_TOPOLOGY}"
                                virtual_lan_start
                                ;;
                        stop)

                                if [ ! -e ${TEMP_FILE} ]; then
                                        echo -n "Noting to stop! ${TEMP_FILE}:"
                                        echo " temp file not found"
                                else
                                        virtual_lan_stop
                                fi
                                ;;
                        help)
                                virtual_lan_usage
                                exit 1
                                ;;
                        *)
                                virtual_lan_usage
                                exit 1

                esac
                ;;
        2)
                case $1 in
                        start)
                                TARGET_TOPOLOGY=$2
                                echo -n "Creating target topology:"
                                echo "${TARGET_TOPOLOGY}"
                                virtual_lan_start
                                ;;
                        *)
                                virtual_lan_usage
                                exit 1
                esac
                ;;

        *)
                virtual_lan_usage
                exit 1
esac


# /root >vnet.ng.test start
Creating default target topology: vdir4|10.0.2.20/24
Loading ng_ether.ko... done
Loading ng_bridge.ko... done
Loading ng_eiface.ko... done
No previous configuration(/var/tmp/virtual.lan.tmp) found to clean-up.
Verifying ethernet interface existence...done
Creating bridge interface: bridge0...
Bridge bridge0 is created and rl0 is connected.
Creating virtual node (jail) vdir4...vdir4: created
done
Creating eiface interface for virtual node vdir4.
Interface ngeth0 is created.
Moving ngeth0 to vdir4
Setting MAC address of ngeth0 to '00:1d:92:df:92:8e'
Setting IP address of ngeth0 to '10.0.2.20/24'
Connecting ngeth0:ether to bridge0:link2...done
Virtual LAN established successfully!


# /root >ngctl ls -l
There are 5 total nodes:
  Name: rl0             Type: ether           ID: 00000001   Num hooks: 2
  Local hook      Peer name       Peer type    Peer ID         Peer
hook
  ----------      ---------       ---------    -------
---------
  upper           bridge0         bridge       00000006        link1

  lower           bridge0         bridge       00000006        link0

  Name: ipfw0           Type: ether           ID: 00000002   Num hooks: 0
  Name: bridge0         Type: bridge          ID: 00000006   Num hooks: 3
  Local hook      Peer name       Peer type    Peer ID         Peer
hook
  ----------      ---------       ---------    -------
---------
  link2           ngeth0          eiface       0000000a        ether

  link1           rl0             ether        00000001        upper

  link0           rl0             ether        00000001        lower

  Name: ngeth0          Type: eiface          ID: 0000000a   Num hooks: 1
  Local hook      Peer name       Peer type    Peer ID         Peer
hook
  ----------      ---------       ---------    -------
---------
  ether           bridge0         bridge       00000006        link2

  Name: ngctl1513       Type: socket          ID: 0000000d   Num hooks: 0


# /root >jexec vdir4 tcsh
vdir4 / >ping -c 1 8.8.178.135
PING 8.8.178.135 (8.8.178.135): 56 data bytes
ping: sendto: No route to host

--- 8.8.178.135 ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss
vdir4 / >exit
exit


# /root >vnet.ng.test stop
Shutdown bridge interface..done
Shutdown all eiface interfaces...done
Removing all jails...vdir4: removed
done
Removing tempfile /var/tmp/virtual.lan.tmp
Virtual LAN objects removed successfully!

# /root >jls
   JID  IP Address      Hostname                      Path

# /root >ngctl ls -l
There are 3 total nodes:
  Name: ngctl1540       Type: socket          ID: 00000010   Num hooks: 0
  Name: rl0             Type: ether           ID: 00000001   Num hooks: 0
  Name: ipfw0           Type: ether           ID: 00000002   Num hooks: 0




_______________________________________________
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Reply via email to