Re: bhyve networking

[Rodney W. Grimes]

[ Charset ISO-8859-1 unsupported, converting... ]
> Hi Folks,
> 
> I've just realised that the igb1 interface is not up in any of the output I 
> shared.  So I took the switch out of the equation and created tap and bridge 
> interfaces manually, then added igb1 and tap0 to bridge0 and brought the 
> bridge up.  Finally, I brought igb1 and tap0 up.  Once all the interfaces 
> were up I amended the guest configuration to replace network0_switch="public" 
> with network0_device="tap0".  Now when I start my guest I have network 
> connectivity on the guest VLAN.
> 
> I'd really like to try and use the switch approach if possible and had 
> thought that creating the switch and adding the igb1 interface would have 
> brought igb1 up automatically.  Is that the expected behaviour?

No, the expected behavior is to not alter the state of igb1, that would be 
doing automagic stuff behind your back, you should add
ifconfig_igb1="up"
to the hosts /etc/rc.conf file.  And I think all your issues well resolve and 
things shall work as you wanted.

> Regards,
> 
> Paul Esson??|??Redstor Limited
> t??+44 (0)118 951 5235??|???m??+44 (0)776 690 6514
> e??paul.es...@redstor.com
> www.redstor.com
> 
> 
> 
> 
> 
> -Original Message-
> From: Paul Esson 
> Sent: 26 April 2018 13:41
> To: Harry Schmalzbauer 
> Cc: freebsd-virtualization@freebsd.org
> Subject: RE: bhyve networking
> 
> Hi Folks,
> 
> Apologies for the lack of detail on my first post.  To recap, I am attempting 
> to set-up a guest using vm-bhyve.  I have a Dell PER730xd server with 
> quad-port INTEL 350 NIC.  The first two ports have been configured on a) a 
> management LAN for the host and b) an application LAN for the guests.
> 
> FreeBSD 11.1-RELEASE-p9
> Dell PowerEdge R730xd - INTEL i350 NICs
> 
> NIC-1 igb0 24:6E:96:B4:61:CC VLAN92  ge-6/0/11 (Host)
> NIC-2 igb1 24:6E:96:B4:61:CD VLAN101 ge-6/0/18 (Guests) - not a trunk
> 
> Both interfaces are active as viewed from the host, but I have only assigned 
> an ipv4 address to igb0 for management of the host
> 
> igb0: flags=8843 metric 0 mtu 1500 
> options=6403bb
> ether 24:6e:96:b4:61:cc
> hwaddr 24:6e:96:b4:61:cc
> inet 172.16.92.20 netmask 0xff00 broadcast 172.16.92.255
> nd6 options=29
> media: Ethernet autoselect (1000baseT )
> status: active
> 
> igb1: flags=8c02 metric 0 mtu 1500   
>  
> options=6403bb
> ether 24:6e:96:b4:61:cd
> hwaddr 24:6e:96:b4:61:cd
> nd6 options=29
> media: Ethernet autoselect (1000baseT )
> status: active
>  
> If I assign a temporary address to igb1 I can then ping other computers on 
> the guests subnet - I've had to hide the address as the network is restricted.
> 
> # ifconfig igb1 inet xx.xxx.xxx.xx/25 up # ping xx.xxx.xxx.xx PING 
> xx.xxx.xxx.xx (xx.xxx.xxx.xx): 56 data bytes
> 64 bytes from xx.xxx.xxx.xx: icmp_seq=0 ttl=64 time=0.145 ms
> 64 bytes from xx.xxx.xxx.xx: icmp_seq=1 ttl=64 time=0.080 ms
> 64 bytes from xx.xxx.xxx.xx: icmp_seq=2 ttl=64 time=0.078 ms
> 64 bytes from xx.xxx.xxx.xx: icmp_seq=3 ttl=64 time=0.077 ms
> 64 bytes from xx.xxx.xxx.xx: icmp_seq=4 ttl=64 time=0.076 ms
> 
> I then used the "vm" command to create a virtual switch and add interface 
> igb1 to it.  This automatically created the bridge interface.
> 
> root@dc1-olbp-hn-01:~ # vm switch create public root@dc1-olbp-hn-01:~ # vm 
> switch add public igb1 root@dc1-olbp-hn-01:~ # vm switch info public
> 
> Virtual Switch: public
> 
>   type: auto
>   ident: bridge0
>   vlan: -
>   nat: -
>   physical-ports: igb1
>   bytes-in: 0 (0.000B)
>   bytes-out: 0 (0.000B)
> 
> Finally, I created a guest VM and gave its NIC the same ipv4 address details 
> I used previously to test igb1 from the host.  This automatically created the 
> tap interface.
> 
> igb0: flags=8843 metric 0 mtu 1500
>   
> options=6403bb
> ether 24:6e:96:b4:61:cc
> hwaddr 24:6e:96:b4:61:cc
> inet 172.16.92.20 netmask 0xff00 broadcast 172.16.92.255
> nd6 options=29
> media: Ethernet autoselect (1000baseT )
> status: active
> 
> igb1: flags=8d02 metric 0 mtu 
> 1500   
> options=6403bb
> ether 24:6e:96:b4:61:cd
>  

Re: bhyve networking

[Rodney W. Grimes]

[ Charset ISO-8859-1 unsupported, converting... ]
> Hi Folks,
> 
> Apologies for the lack of detail on my first post.  To recap, I am attempting 
> to set-up a guest using vm-bhyve.  I have a Dell PER730xd server with 
> quad-port INTEL 350 NIC.  The first two ports have been configured on a) a 
> management LAN for the host and b) an application LAN for the guests.
> 
> FreeBSD 11.1-RELEASE-p9
> Dell PowerEdge R730xd - INTEL i350 NICs
> 
> NIC-1 igb0 24:6E:96:B4:61:CC VLAN92  ge-6/0/11 (Host)
> NIC-2 igb1 24:6E:96:B4:61:CD VLAN101 ge-6/0/18 (Guests) - not a trunk
> 
> Both interfaces are active as viewed from the host, but I have only assigned 
> an ipv4 address to igb0 for management of the host
> 
> igb0: flags=8843 metric 0 mtu 1500 
> options=6403bb
> ether 24:6e:96:b4:61:cc
> hwaddr 24:6e:96:b4:61:cc
> inet 172.16.92.20 netmask 0xff00 broadcast 172.16.92.255
> nd6 options=29
> media: Ethernet autoselect (1000baseT )
> status: active
> 
> igb1: flags=8c02 metric 0 mtu 1500   
>  
> options=6403bb
  ^^ MIssing UP, interface is down
> ether 24:6e:96:b4:61:cd
> hwaddr 24:6e:96:b4:61:cd
> nd6 options=29
> media: Ethernet autoselect (1000baseT )
> status: active
>  
> If I assign a temporary address to igb1 I can then ping other computers on 
> the guests subnet - I've had to hide the address as the network is restricted.
> 
> # ifconfig igb1 inet xx.xxx.xxx.xx/25 up
> # ping xx.xxx.xxx.xx
> PING xx.xxx.xxx.xx (xx.xxx.xxx.xx): 56 data bytes
> 64 bytes from xx.xxx.xxx.xx: icmp_seq=0 ttl=64 time=0.145 ms
> 64 bytes from xx.xxx.xxx.xx: icmp_seq=1 ttl=64 time=0.080 ms
> 64 bytes from xx.xxx.xxx.xx: icmp_seq=2 ttl=64 time=0.078 ms
> 64 bytes from xx.xxx.xxx.xx: icmp_seq=3 ttl=64 time=0.077 ms
> 64 bytes from xx.xxx.xxx.xx: icmp_seq=4 ttl=64 time=0.076 ms
> 
> I then used the "vm" command to create a virtual switch and add interface 
> igb1 to it.  This automatically created the bridge interface.
> 
> root@dc1-olbp-hn-01:~ # vm switch create public
> root@dc1-olbp-hn-01:~ # vm switch add public igb1
> root@dc1-olbp-hn-01:~ # vm switch info public
> 
> Virtual Switch: public
> 
>   type: auto
>   ident: bridge0
>   vlan: -
>   nat: -
>   physical-ports: igb1
>   bytes-in: 0 (0.000B)
>   bytes-out: 0 (0.000B)
> 
> Finally, I created a guest VM and gave its NIC the same ipv4 address details 
> I used previously to test igb1 from the host.  This automatically created the 
> tap interface.
> 
> igb0: flags=8843 metric 0 mtu 1500
>   
> options=6403bb
> ether 24:6e:96:b4:61:cc
> hwaddr 24:6e:96:b4:61:cc
> inet 172.16.92.20 netmask 0xff00 broadcast 172.16.92.255
> nd6 options=29
> media: Ethernet autoselect (1000baseT )
> status: active
> 
> igb1: flags=8d02 metric 0 mtu 
> 1500   
> options=6403bb
  ^^ mising up, interface is down
> ether 24:6e:96:b4:61:cd
> hwaddr 24:6e:96:b4:61:cd
> nd6 options=29
> media: Ethernet autoselect (1000baseT )
> status: active
> 
> lo0: flags=8049 metric 0 mtu 16384
> options=63
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
> inet 127.0.0.1 netmask 0xff00
> nd6 options=21
> groups: lo
> 
> bridge0: flags=8843 metric 0 mtu 1500
> description: vm-public
> ether 02:ee:ce:b0:6a:00
> nd6 options=1
> groups: bridge
> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
> member: tap0 flags=143
> ifmaxaddr 0 port 7 priority 128 path cost 200
> member: igb1 flags=143
> ifmaxaddr 0 port 2 priority 128 path cost 2
> 
> tap0: flags=8943 metric 

Re: bhyve networking

[Patrick M. Hausen]

Hi!

> Am 26.04.2018 um 15:32 schrieb Paul Esson :
> I'd really like to try and use the switch approach if possible and had 
> thought that creating the switch and adding the igb1 interface would have 
> brought igb1 up automatically.  Is that the expected behaviour?

You have to "ifconfig igb1 up" manually for any of the bridging technologies in 
FreeBSD
as far as I know. Definitely with if_bridge. It is not sufficient to "ifconfig 
addm" the physical
interface.

But of course one just puts

ifconfig_igb1="up"

into rc.conf and forgets about it on a production system ...

HTH,
Patrick
-- 
punkt.de GmbH   Internet - Dienstleistungen - Beratung
Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100
76133 Karlsruhe i...@punkt.de   http://punkt.de
AG Mannheim 108285  Gf: Juergen Egeling

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve networking

[Luciano Mannucci]

On Thu, 26 Apr 2018 13:32:20 +
Paul Esson  wrote:

> I'd really like to try and use the switch approach if possible and had
> thought that creating the switch and adding the igb1 interface would have
> brought igb1 up automatically.
I had to put

ifconfig_igb0="UP"

in order to have vm and bhyve working. I think this is not documented.

Cheers,

Luciano.
-- 
 /"\ /Via A. Salaino, 7 - 20144 Milano (Italy)
 \ /  ASCII RIBBON CAMPAIGN / PHONE : +39 2 485781 FAX: +39 2 48578250
  X   AGAINST HTML MAIL/  E-MAIL: posthams...@sublink.sublink.org
 / \  AND POSTINGS/   WWW: http://www.lesassaie.IT/
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

RE: bhyve networking

[Paul Esson]

Hi Folks,

I've just realised that the igb1 interface is not up in any of the output I 
shared.  So I took the switch out of the equation and created tap and bridge 
interfaces manually, then added igb1 and tap0 to bridge0 and brought the bridge 
up.  Finally, I brought igb1 and tap0 up.  Once all the interfaces were up I 
amended the guest configuration to replace network0_switch="public" with 
network0_device="tap0".  Now when I start my guest I have network connectivity 
on the guest VLAN.

I'd really like to try and use the switch approach if possible and had thought 
that creating the switch and adding the igb1 interface would have brought igb1 
up automatically.  Is that the expected behaviour?

Regards,

Paul Esson  |  Redstor Limited
t  +44 (0)118 951 5235  |   m  +44 (0)776 690 6514
e  paul.es...@redstor.com
www.redstor.com





-Original Message-
From: Paul Esson 
Sent: 26 April 2018 13:41
To: Harry Schmalzbauer 
Cc: freebsd-virtualization@freebsd.org
Subject: RE: bhyve networking

Hi Folks,

Apologies for the lack of detail on my first post.  To recap, I am attempting 
to set-up a guest using vm-bhyve.  I have a Dell PER730xd server with quad-port 
INTEL 350 NIC.  The first two ports have been configured on a) a management LAN 
for the host and b) an application LAN for the guests.

FreeBSD 11.1-RELEASE-p9
Dell PowerEdge R730xd - INTEL i350 NICs

NIC-1 igb0 24:6E:96:B4:61:CC VLAN92  ge-6/0/11 (Host)
NIC-2 igb1 24:6E:96:B4:61:CD VLAN101 ge-6/0/18 (Guests) - not a trunk

Both interfaces are active as viewed from the host, but I have only assigned an 
ipv4 address to igb0 for management of the host

igb0: flags=8843 metric 0 mtu 1500 
options=6403bb
ether 24:6e:96:b4:61:cc
hwaddr 24:6e:96:b4:61:cc
inet 172.16.92.20 netmask 0xff00 broadcast 172.16.92.255
nd6 options=29
media: Ethernet autoselect (1000baseT )
status: active

igb1: flags=8c02 metric 0 mtu 1500
options=6403bb
ether 24:6e:96:b4:61:cd
hwaddr 24:6e:96:b4:61:cd
nd6 options=29
media: Ethernet autoselect (1000baseT )
status: active
 
If I assign a temporary address to igb1 I can then ping other computers on the 
guests subnet - I've had to hide the address as the network is restricted.

# ifconfig igb1 inet xx.xxx.xxx.xx/25 up # ping xx.xxx.xxx.xx PING 
xx.xxx.xxx.xx (xx.xxx.xxx.xx): 56 data bytes
64 bytes from xx.xxx.xxx.xx: icmp_seq=0 ttl=64 time=0.145 ms
64 bytes from xx.xxx.xxx.xx: icmp_seq=1 ttl=64 time=0.080 ms
64 bytes from xx.xxx.xxx.xx: icmp_seq=2 ttl=64 time=0.078 ms
64 bytes from xx.xxx.xxx.xx: icmp_seq=3 ttl=64 time=0.077 ms
64 bytes from xx.xxx.xxx.xx: icmp_seq=4 ttl=64 time=0.076 ms

I then used the "vm" command to create a virtual switch and add interface igb1 
to it.  This automatically created the bridge interface.

root@dc1-olbp-hn-01:~ # vm switch create public root@dc1-olbp-hn-01:~ # vm 
switch add public igb1 root@dc1-olbp-hn-01:~ # vm switch info public

Virtual Switch: public

  type: auto
  ident: bridge0
  vlan: -
  nat: -
  physical-ports: igb1
  bytes-in: 0 (0.000B)
  bytes-out: 0 (0.000B)

Finally, I created a guest VM and gave its NIC the same ipv4 address details I 
used previously to test igb1 from the host.  This automatically created the tap 
interface.

igb0: flags=8843 metric 0 mtu 1500  
options=6403bb
ether 24:6e:96:b4:61:cc
hwaddr 24:6e:96:b4:61:cc
inet 172.16.92.20 netmask 0xff00 broadcast 172.16.92.255
nd6 options=29
media: Ethernet autoselect (1000baseT )
status: active

igb1: flags=8d02 metric 0 mtu 1500 
  
options=6403bb
ether 24:6e:96:b4:61:cd
hwaddr 24:6e:96:b4:61:cd
nd6 options=29
media: Ethernet autoselect (1000baseT )
status: active

lo0: flags=8049 metric 0 mtu 16384
options=63
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
inet 127.0.0.1 netmask 0xff00
nd6 options=21
groups: lo

bridge0: 

RE: bhyve networking

[Paul Esson]

Hi Folks,

Apologies for the lack of detail on my first post.  To recap, I am attempting 
to set-up a guest using vm-bhyve.  I have a Dell PER730xd server with quad-port 
INTEL 350 NIC.  The first two ports have been configured on a) a management LAN 
for the host and b) an application LAN for the guests.

FreeBSD 11.1-RELEASE-p9
Dell PowerEdge R730xd - INTEL i350 NICs

NIC-1 igb0 24:6E:96:B4:61:CC VLAN92  ge-6/0/11 (Host)
NIC-2 igb1 24:6E:96:B4:61:CD VLAN101 ge-6/0/18 (Guests) - not a trunk

Both interfaces are active as viewed from the host, but I have only assigned an 
ipv4 address to igb0 for management of the host

igb0: flags=8843 metric 0 mtu 1500 
options=6403bb
ether 24:6e:96:b4:61:cc
hwaddr 24:6e:96:b4:61:cc
inet 172.16.92.20 netmask 0xff00 broadcast 172.16.92.255
nd6 options=29
media: Ethernet autoselect (1000baseT )
status: active

igb1: flags=8c02 metric 0 mtu 1500
options=6403bb
ether 24:6e:96:b4:61:cd
hwaddr 24:6e:96:b4:61:cd
nd6 options=29
media: Ethernet autoselect (1000baseT )
status: active
 
If I assign a temporary address to igb1 I can then ping other computers on the 
guests subnet - I've had to hide the address as the network is restricted.

# ifconfig igb1 inet xx.xxx.xxx.xx/25 up
# ping xx.xxx.xxx.xx
PING xx.xxx.xxx.xx (xx.xxx.xxx.xx): 56 data bytes
64 bytes from xx.xxx.xxx.xx: icmp_seq=0 ttl=64 time=0.145 ms
64 bytes from xx.xxx.xxx.xx: icmp_seq=1 ttl=64 time=0.080 ms
64 bytes from xx.xxx.xxx.xx: icmp_seq=2 ttl=64 time=0.078 ms
64 bytes from xx.xxx.xxx.xx: icmp_seq=3 ttl=64 time=0.077 ms
64 bytes from xx.xxx.xxx.xx: icmp_seq=4 ttl=64 time=0.076 ms

I then used the "vm" command to create a virtual switch and add interface igb1 
to it.  This automatically created the bridge interface.

root@dc1-olbp-hn-01:~ # vm switch create public
root@dc1-olbp-hn-01:~ # vm switch add public igb1
root@dc1-olbp-hn-01:~ # vm switch info public

Virtual Switch: public

  type: auto
  ident: bridge0
  vlan: -
  nat: -
  physical-ports: igb1
  bytes-in: 0 (0.000B)
  bytes-out: 0 (0.000B)

Finally, I created a guest VM and gave its NIC the same ipv4 address details I 
used previously to test igb1 from the host.  This automatically created the tap 
interface.

igb0: flags=8843 metric 0 mtu 1500  
options=6403bb
ether 24:6e:96:b4:61:cc
hwaddr 24:6e:96:b4:61:cc
inet 172.16.92.20 netmask 0xff00 broadcast 172.16.92.255
nd6 options=29
media: Ethernet autoselect (1000baseT )
status: active

igb1: flags=8d02 metric 0 mtu 1500 
  
options=6403bb
ether 24:6e:96:b4:61:cd
hwaddr 24:6e:96:b4:61:cd
nd6 options=29
media: Ethernet autoselect (1000baseT )
status: active

lo0: flags=8049 metric 0 mtu 16384
options=63
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
inet 127.0.0.1 netmask 0xff00
nd6 options=21
groups: lo

bridge0: flags=8843 metric 0 mtu 1500
description: vm-public
ether 02:ee:ce:b0:6a:00
nd6 options=1
groups: bridge
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap0 flags=143
ifmaxaddr 0 port 7 priority 128 path cost 200
member: igb1 flags=143
ifmaxaddr 0 port 2 priority 128 path cost 2

tap0: flags=8943 metric 0 mtu 
1500
description: vmnet-testvm-0-public
options=8
ether 00:bd:dd:51:0a:00
hwaddr 00:bd:dd:51:0a:00
nd6 options=29
media: Ethernet autoselect
status: active
groups: tap
Opened by PID 1791

>From the guest VM I can see that the interface vtnet0 is 

Re: bhyve networking

[Harry Schmalzbauer]

Bezüglich Paul Esson's Nachricht vom 25.04.2018 23:15 (localtime):
> Hi Rod,
> Can you share a command line for that?  I also tried presenting an
> access port from my switch on a specific VLAN - not trimmed.  Would I
> still have to tag the interface on the guest in that scenario?

Hmm, I lost the overview – I'm not familar with 'vm'.
To filter a specific id (tag/untag frames) inside the guest:
'ifconfig vlan[N] create vlandev vtnet0 vlan '
'ifconfig vlan[N] create vlandev vtnet0 vlan nnnm'

At boot time by rc(8):
vlans_vtnet0="vtnet_dmz vtnet_dmz2"
create_args_vtnet_dmz="vlan "
create_args_vtnet_dmz2="vlan nnnm"

[To optionally also rename the vlan interfaces after manually creating
cloned vlan interfaces, which is what the rc.conf(5) example does:
ifconfig rename vlan0 vtnet_dmz; ifconfig rename vlan0 vtnet_dmz2; ]

Hth,

-harry


___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"