Hi Greg,
I have 4 NICs on the box. I am using the supermicro x10sdvf-tln4f motherboard
(2 x 1Gb and 2 X 10Gb network ports). I have a Ubiquiti Edgeswitch 24port 250W
POE switch. The port I am using for KVM traffic on the Ubiquiti edgeswitch has
been setup as a trunk port (I use a separate NIC on my box for admin traffic
for smartos). I have previously got the setup working using VMware ESXi where
I create different port groups with different VLANs. In ESXi I could set a
port group with VLAN ID 4095 (which I believe passed all tagged packets) and
assign that port group to my VYOS machine so that I could setup different VIF
IP addresses in VYOS for routing. I am trying to do the same on SmartOS.
However when I try and ping out from my VYOS machine I dont get any network
connectivity.
Could you post your dladm show-vnic output so that I could see what is
different? When I do mine, I have the following for the nic I assigned to my
VYOS KVM. I have other nics using the same igb0 interface with different VID
values. Abbreviated output shown below. So the other KVMs using VLANs on 90
and 77 work fine. I am trying to get the entire "trunk" port passed to the
VYOS KVM and tag the traffic within VYOS but it is not working.
LINK OVER SPEED MACADDRESS MACADDRTYPE VID ZONE
eth1 igb0 0 82:66:81:58:9f:5e fixed
90 9c20ce0c-64d3-6
eth0 igb0 0 12:8c:db:7e:3:32 fixed
77 d1ce2a61-1529-4
eth0 igb0 0 c2:9a:aa:f8:41:8c fixed
0 cc549d5c-9dd6-e
The output of my JSON file is as follows: -
{
"zonename": "cc549d5c-9dd6-e896-d917-b8732fb61797",
"autoboot": true,
"brand": "kvm",
"limit_priv":
"default,-file_link_any,-net_access,-proc_fork,-proc_info,-proc_session",
"v": 1,
"create_timestamp": "2018-01-27T08:14:49.425Z",
"cpu_shares": 100,
"max_lwps": 2000,
"max_msg_ids": 4096,
"max_sem_ids": 4096,
"max_shm_ids": 4096,
"max_shm_memory": 3072,
"zfs_io_priority": 100,
"max_physical_memory": 3072,
"max_locked_memory": 3072,
"max_swap": 3072,
"billing_id": "00000000-0000-0000-0000-000000000000",
"owner_uuid": "00000000-0000-0000-0000-000000000000",
"resolvers": [
],
"ram": 2048,
"vcpus": 1,
"cpu_type": "host",
"disks": [
{
.....
}
],
"vnc_port": 35285,
"nics": [
{
"interface": "eth0",
"mac": "c2:9a:aa:f8:41:8c",
"nic_tag": "vpg4095vlan",
"ip": "dhcp",
"ips": [
"dhcp"
],
"model": "virtio",
"allow_ip_spoofing": true,
"allow_mac_spoofing": true,
"allow_restricted_traffic": true,
"allow_unfiltered_promisc": true,
"primary": true
}
],
"uuid": "cc549d5c-9dd6-e896-d917-b8732fb61797",
"zone_state": "running",
"zonepath": "/zones/cc549d5c-9dd6-e896-d917-b8732fb61797",
"zoneid": 69,
"last_modified": "2018-02-20T10:01:32.000Z",
"firewall_enabled": false,
"server_uuid": "00000000-0000-0000-0000-0cc47ac59400",
"platform_buildstamp": "20171221T020409Z",
"state": "running",
"boot_timestamp": "2018-02-20T10:01:31.000Z",
"pid": 24615,
"customer_metadata": {},
"internal_metadata": {},
"routes": {},
"tags": {},
"quota": 10,
"zfs_root_compression": "on",
"zfs_root_recsize": 131072,
"zfs_filesystem": "zones/cc549d5c-9dd6-e896-d917-b8732fb61797",
"zpool": "zones",
"snapshots": []
}
________________________________
From: Greg Treantos <[email protected]>
Sent: Wednesday, 21 February 2018 2:21 a.m.
To: [email protected]
Subject: Re: [smartos-discuss] Assign trunk port to KVM host in Smartos
Eugene,
I set up one physical NIC for the router. To accomplish proper
packet VLANing I found it more logical to set the VLANs at the VM not the
global zone. I wanted to take away the chance of messing with the GZ on any
network changes. Spoofing is needed to get router services working on the LAN
side of things. Its not needed for the WAN side.
How many physical NICs do you have on your box? Are you dedicating a NIC for
your router or are you passing all your traffic through the SmartOS management
interface?
On Tue, Feb 20, 2018 at 12:06 AM, Eugene Lee
<[email protected]<mailto:[email protected]>> wrote:
Thanks Greg for your json config files. I notice that you still have a vlan_id
tag set for the nics. I gather the key to having this work is the following
options?
"allow_dhcp_spoofing": true,
"allow_ip_spoofing": true,
"allow_mac_spoofing": true,
"allow_restricted_traffic": true,
"allow_unfiltered_promisc": true
________________________________
From: Greg Treantos <[email protected]<mailto:[email protected]>>
Sent: Tuesday, 20 February 2018 12:44 a.m.
To: [email protected]<mailto:[email protected]>
Subject: Re: [smartos-discuss] Assign trunk port to KVM host in Smartos
Eugene,
I am running two routers on my Smartos box for my home network. I
have Verizon Fios and to get remote DVR functions I have to use a mid-router to
fake the Verizon router in thinking it is connected to the Verizon ONT. I port
forward all Verzion packets from the main Opnsense router to Vyos, then off to
the Verizon router. This setup is well documented. anyway I use the setup you
describe. I have one interface or Trunk as you call it then vlan the routers
through that interface. Here are the JSONs for Vyos and Opnsense. One note do
not spoof the WAN interface, only the lan interfaces. The key is your nic_tag,
that points to your trunk interface. All nics get set up in /usbkey/config.
Hope this helps
/usb/config
#firewall nic definition
firewall_nic=a0:36:9f:3:51:6d
Vyos JSONs
vmadm get 89168e3f-dec5-676e-879d-c90b69a7d988 |json
{
"zonename": "89168e3f-dec5-676e-879d-c90b69a7d988",
"autoboot": true,
"brand": "kvm",
"limit_priv":
"default,-file_link_any,-net_access,-proc_fork,-proc_info,-proc_session",
"v": 1,
"create_timestamp": "2017-01-10T00:25:40.170Z",
"cpu_shares": 100,
"max_lwps": 2000,
"max_msg_ids": 4096,
"max_sem_ids": 4096,
"max_shm_ids": 4096,
"max_shm_memory": 1536,
"zfs_io_priority": 100,
"max_physical_memory": 1536,
"max_locked_memory": 1536,
"max_swap": 1536,
"billing_id": "00000000-0000-0000-0000-000000000000",
"owner_uuid": "00000000-0000-0000-0000-000000000000",
"alias": "vyos vz bridge router",
"ram": 512,
"vcpus": 1,
"vnc_port": 23456,
"disks": [
{
"path": "/dev/zvol/rdsk/zones/89168e3f-dec5-676e-879d-c90b69a7d988-disk0",
"boot": true,
"model": "virtio",
"media": "disk",
"zfs_filesystem": "zones/89168e3f-dec5-676e-879d-c90b69a7d988-disk0",
"zpool": "zones",
"size": 1024,
"compression": "off",
"refreservation": 1024,
"block_size": 8192
}
],
"nics": [
{
"interface": "net0",
"mac": "52:54:00:b6:c8:21",
"vlan_id": 100,
"nic_tag": "firewall",
"ip": "dhcp",
"ips": [
"dhcp"
],
"model": "virtio",
"allow_dhcp_spoofing": false,
"allow_ip_spoofing": false,
"allow_mac_spoofing": false,
"allow_restricted_traffic": false,
"allow_unfiltered_promisc": false,
"primary": true
},
{
"interface": "net1",
"mac": "72:7c:9d:dd:6e:a3",
"vlan_id": 173,
"nic_tag": "firewall",
"netmask": "255.255.255.0",
"ip": "173.48.255.1",
"ips": [
"173.48.255.1/24<http://173.48.255.1/24>"
],
"model": "virtio",
"allow_dhcp_spoofing": true,
"allow_ip_spoofing": true,
"allow_mac_spoofing": true,
"allow_restricted_traffic": true,
"allow_unfiltered_promisc": true
}
],
"uuid": "89168e3f-dec5-676e-879d-c90b69a7d988",
"zone_state": "running",
"zonepath": "/zones/89168e3f-dec5-676e-879d-c90b69a7d988",
"zoneid": 6,
"last_modified": "2018-02-18T12:59:40.000Z",
"resolvers": [],
"firewall_enabled": false,
"server_uuid": "20c32547-dad7-dd11-abd6-d017c2956b6b",
"platform_buildstamp": "20180203T031130Z",
"state": "running",
"boot_timestamp": "2018-02-18T12:59:40.000Z",
"pid": 5356,
"customer_metadata": {},
"internal_metadata": {},
"routes": {},
"tags": {},
"quota": 10,
"zfs_root_recsize": 131072,
"zfs_filesystem": "zones/89168e3f-dec5-676e-879d-c90b69a7d988",
"zpool": "zones",
"snapshots": []
}
[root@smugglers ~]# vmadm get c4cc5cd2-9d63-4546-c405-9e9bf724268a | json
{
"zonename": "c4cc5cd2-9d63-4546-c405-9e9bf724268a",
"autoboot": true,
"brand": "kvm",
"limit_priv":
"default,-file_link_any,-net_access,-proc_fork,-proc_info,-proc_session",
"v": 1,
"create_timestamp": "2017-10-27T16:13:45.003Z",
"cpu_shares": 100,
"max_lwps": 2000,
"max_msg_ids": 4096,
"max_sem_ids": 4096,
"max_shm_ids": 4096,
"max_shm_memory": 2048,
"zfs_io_priority": 100,
"max_physical_memory": 2048,
"max_locked_memory": 2048,
"max_swap": 2048,
"billing_id": "00000000-0000-0000-0000-000000000000",
"owner_uuid": "00000000-0000-0000-0000-000000000000",
"alias": "Backup Opnsense Firewall",
"ram": 1024,
"vcpus": 2,
"vnc_port": 99999,
"disks": [
{
"path": "/dev/zvol/rdsk/zones/c4cc5cd2-9d63-4546-c405-9e9bf724268a-disk0",
"boot": true,
"model": "virtio",
"media": "disk",
"zfs_filesystem": "zones/c4cc5cd2-9d63-4546-c405-9e9bf724268a-disk0",
"zpool": "zones",
"size": 5120,
"compression": "off",
"refreservation": 5120,
"block_size": 8192
}
],
"nics": [
{
"interface": "net0",
"mac": "ab:cd:ef:gh:ij:kl",
"vlan_id": 10,
"nic_tag": "firewall",
"ip": "dhcp",
"ips": [
"dhcp"
],
"model": "virtio",
"allow_dhcp_spoofing": false,
"allow_ip_spoofing": false,
"allow_mac_spoofing": false,
"allow_restricted_traffic": false,
"allow_unfiltered_promisc": false,
"primary": true
},
{
"interface": "net1",
"mac": "42:50:0e:e8:c7:28",
"vlan_id": 100,
"nic_tag": "firewall",
"netmask": "255.255.255.0",
"ip": "192.168.1.254",
"ips": [
"192.168.1.254/24<http://192.168.1.254/24>"
],
"model": "virtio",
"allow_dhcp_spoofing": true,
"allow_ip_spoofing": true,
"allow_mac_spoofing": true,
"allow_restricted_traffic": true,
"allow_unfiltered_promisc": true
},
{
"interface": "net2",
"mac": "f2:8d:d3:20:1c:20",
"vlan_id": 200,
"nic_tag": "firewall",
"netmask": "255.255.255.224",
"ip": "192.168.200.30",
"ips": [
"192.168.200.30/27<http://192.168.200.30/27>"
],
"model": "virtio",
"allow_dhcp_spoofing": true,
"allow_ip_spoofing": true,
"allow_mac_spoofing": true,
"allow_restricted_traffic": true,
"allow_unfiltered_promisc": true
}
],
"uuid": "c4cc5cd2-9d63-4546-c405-9e9bf724268a",
"zone_state": "running",
"zonepath": "/zones/c4cc5cd2-9d63-4546-c405-9e9bf724268a",
"zoneid": 7,
"last_modified": "2018-02-18T12:59:41.000Z",
"resolvers": [],
"firewall_enabled": false,
"server_uuid": "20c32547-dad7-dd11-abd6-d017c2956b6b",
"platform_buildstamp": "20180203T031130Z",
"state": "running",
"boot_timestamp": "2018-02-18T12:59:41.000Z",
"pid": 5431,
"customer_metadata": {},
"internal_metadata": {},
"routes": {},
"tags": {},
"quota": 10,
"zfs_root_recsize": 131072,
"zfs_filesystem": "zones/c4cc5cd2-9d63-4546-c405-9e9bf724268a",
"zpool": "zones",
"snapshots": []
}
On Mon, Feb 19, 2018 at 4:08 AM, Eugene Lee
<[email protected]<mailto:[email protected]>> wrote:
Hi,
I am fairly new still to Smartos and learning my way around it. I have been in
the process of migrating my environment from VMware ESXi to a Smartos host. I
have a Vyos virtual router that I would like to migrate from VMware to Smartos.
In VMware I know how to assign a trunk port to the Vyos virtual machine and
then assign various IPs in different VLANs for routing in the Vyos router. I
am not sure what the equivalent way of doing this is in SmartOs.
I know that you can add the vlan_id tag to the JSON file for the KVM machine
but given I want a trunk port assigned to the KVM machine (that will be running
Vyos), how do I go about doing this? Hopefully what I want to achieve makes
sense.
Thanks,
Eugene
--
Greg
http://www.linkedin.com/in/gregtreantos
--
Greg
http://www.linkedin.com/in/gregtreantos
smartos-discuss | Archives<https://www.listbox.com/member/archive/184463/=now>
[https://www.listbox.com/images/feed-icon-10x10.jpg071777d.jpg?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2ZlZWQtaWNvbi0xMHgxMC5qcGc]
<https://www.listbox.com/member/archive/rss/184463/29581085-44f2ca1e> |
Modify<https://www.listbox.com/member/?&;> Your Subscription
[https://www.listbox.com/images/listbox-logo-small.png071777d.png?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2xpc3Rib3gtbG9nby1zbWFsbC5wbmc]
<http://www.listbox.com>
-------------------------------------------
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription:
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com
