Hi Greg,
Just wondering, with this comment "You can use the same wire, just tag each VIF with its ultimate Vlan ID and trunk at the switch." Are you saying that if I have 5 VLANs that I create 5 nics in my KVM JSON file each tagged appropriately? Of course they will all use the same "trunk" physical NIC port, but the KVM will see a separate ethernet interface for each VLAN it needs? ________________________________ From: Greg Treantos <[email protected]> Sent: Thursday, 22 February 2018 2:19 a.m. To: [email protected] Subject: Re: [smartos-discuss] Assign trunk port to KVM host in Smartos Eugene, I can't grab my dladm output right now. Work blocks outbound VPN access, I can get it tonight when I am home. I don't know if SmartOS following the VMware vlan 4095 paradigm. I would do a normal Vlan trunk setup with SmartOS. You can use the same wire, just tag each VIF with its ultimate Vlan ID and trunk at the switch. This is how I have it setup and all works peachy. I find the only gotcha with this is getting the Vlans on the switch right. I have 3 brands of switches. Going from trunk to port based Vlans is slightly different. They all work well together once setup properly. Look at my /usb/config and JSON again. You have to match the MAC on the physical NIC with each VIF. Then configure each VIF with its VLAN tag. Trunk at the switch and you will be good to go. This is probably equivalent to VLAN 4095 in VMWare world. Hope this makes sense. On Wed, Feb 21, 2018 at 5:32 AM, Eugene Lee <[email protected]<mailto:[email protected]>> wrote: Hi Greg, I have 4 NICs on the box. I am using the supermicro x10sdvf-tln4f motherboard (2 x 1Gb and 2 X 10Gb network ports). I have a Ubiquiti Edgeswitch 24port 250W POE switch. The port I am using for KVM traffic on the Ubiquiti edgeswitch has been setup as a trunk port (I use a separate NIC on my box for admin traffic for smartos). I have previously got the setup working using VMware ESXi where I create different port groups with different VLANs. In ESXi I could set a port group with VLAN ID 4095 (which I believe passed all tagged packets) and assign that port group to my VYOS machine so that I could setup different VIF IP addresses in VYOS for routing. I am trying to do the same on SmartOS. However when I try and ping out from my VYOS machine I dont get any network connectivity. Could you post your dladm show-vnic output so that I could see what is different? When I do mine, I have the following for the nic I assigned to my VYOS KVM. I have other nics using the same igb0 interface with different VID values. Abbreviated output shown below. So the other KVMs using VLANs on 90 and 77 work fine. I am trying to get the entire "trunk" port passed to the VYOS KVM and tag the traffic within VYOS but it is not working. LINK OVER SPEED MACADDRESS MACADDRTYPE VID ZONE eth1 igb0 0 82:66:81:58:9f:5e fixed 90 9c20ce0c-64d3-6 eth0 igb0 0 12:8c:db:7e:3:32 fixed 77 d1ce2a61-1529-4 eth0 igb0 0 c2:9a:aa:f8:41:8c fixed 0 cc549d5c-9dd6-e The output of my JSON file is as follows: - { "zonename": "cc549d5c-9dd6-e896-d917-b8732fb61797", "autoboot": true, "brand": "kvm", "limit_priv": "default,-file_link_any,-net_access,-proc_fork,-proc_info,-proc_session", "v": 1, "create_timestamp": "2018-01-27T08:14:49.425Z", "cpu_shares": 100, "max_lwps": 2000, "max_msg_ids": 4096, "max_sem_ids": 4096, "max_shm_ids": 4096, "max_shm_memory": 3072, "zfs_io_priority": 100, "max_physical_memory": 3072, "max_locked_memory": 3072, "max_swap": 3072, "billing_id": "00000000-0000-0000-0000-000000000000", "owner_uuid": "00000000-0000-0000-0000-000000000000", "resolvers": [ ], "ram": 2048, "vcpus": 1, "cpu_type": "host", "disks": [ { ..... } ], "vnc_port": 35285, "nics": [ { "interface": "eth0", "mac": "c2:9a:aa:f8:41:8c", "nic_tag": "vpg4095vlan", "ip": "dhcp", "ips": [ "dhcp" ], "model": "virtio", "allow_ip_spoofing": true, "allow_mac_spoofing": true, "allow_restricted_traffic": true, "allow_unfiltered_promisc": true, "primary": true } ], "uuid": "cc549d5c-9dd6-e896-d917-b8732fb61797", "zone_state": "running", "zonepath": "/zones/cc549d5c-9dd6-e896-d917-b8732fb61797", "zoneid": 69, "last_modified": "2018-02-20T10:01:32.000Z", "firewall_enabled": false, "server_uuid": "00000000-0000-0000-0000-0cc47ac59400", "platform_buildstamp": "20171221T020409Z", "state": "running", "boot_timestamp": "2018-02-20T10:01:31.000Z", "pid": 24615, "customer_metadata": {}, "internal_metadata": {}, "routes": {}, "tags": {}, "quota": 10, "zfs_root_compression": "on", "zfs_root_recsize": 131072, "zfs_filesystem": "zones/cc549d5c-9dd6-e896-d917-b8732fb61797", "zpool": "zones", "snapshots": [] } ________________________________ From: Greg Treantos <[email protected]<mailto:[email protected]>> Sent: Wednesday, 21 February 2018 2:21 a.m. To: [email protected]<mailto:[email protected]> Subject: Re: [smartos-discuss] Assign trunk port to KVM host in Smartos Eugene, I set up one physical NIC for the router. To accomplish proper packet VLANing I found it more logical to set the VLANs at the VM not the global zone. I wanted to take away the chance of messing with the GZ on any network changes. Spoofing is needed to get router services working on the LAN side of things. Its not needed for the WAN side. How many physical NICs do you have on your box? Are you dedicating a NIC for your router or are you passing all your traffic through the SmartOS management interface? On Tue, Feb 20, 2018 at 12:06 AM, Eugene Lee <[email protected]<mailto:[email protected]>> wrote: Thanks Greg for your json config files. I notice that you still have a vlan_id tag set for the nics. I gather the key to having this work is the following options? "allow_dhcp_spoofing": true, "allow_ip_spoofing": true, "allow_mac_spoofing": true, "allow_restricted_traffic": true, "allow_unfiltered_promisc": true ________________________________ From: Greg Treantos <[email protected]<mailto:[email protected]>> Sent: Tuesday, 20 February 2018 12:44 a.m. To: [email protected]<mailto:[email protected]> Subject: Re: [smartos-discuss] Assign trunk port to KVM host in Smartos Eugene, I am running two routers on my Smartos box for my home network. I have Verizon Fios and to get remote DVR functions I have to use a mid-router to fake the Verizon router in thinking it is connected to the Verizon ONT. I port forward all Verzion packets from the main Opnsense router to Vyos, then off to the Verizon router. This setup is well documented. anyway I use the setup you describe. I have one interface or Trunk as you call it then vlan the routers through that interface. Here are the JSONs for Vyos and Opnsense. One note do not spoof the WAN interface, only the lan interfaces. The key is your nic_tag, that points to your trunk interface. All nics get set up in /usbkey/config. Hope this helps /usb/config #firewall nic definition firewall_nic=a0:36:9f:3:51:6d Vyos JSONs vmadm get 89168e3f-dec5-676e-879d-c90b69a7d988 |json { "zonename": "89168e3f-dec5-676e-879d-c90b69a7d988", "autoboot": true, "brand": "kvm", "limit_priv": "default,-file_link_any,-net_access,-proc_fork,-proc_info,-proc_session", "v": 1, "create_timestamp": "2017-01-10T00:25:40.170Z", "cpu_shares": 100, "max_lwps": 2000, "max_msg_ids": 4096, "max_sem_ids": 4096, "max_shm_ids": 4096, "max_shm_memory": 1536, "zfs_io_priority": 100, "max_physical_memory": 1536, "max_locked_memory": 1536, "max_swap": 1536, "billing_id": "00000000-0000-0000-0000-000000000000", "owner_uuid": "00000000-0000-0000-0000-000000000000", "alias": "vyos vz bridge router", "ram": 512, "vcpus": 1, "vnc_port": 23456, "disks": [ { "path": "/dev/zvol/rdsk/zones/89168e3f-dec5-676e-879d-c90b69a7d988-disk0", "boot": true, "model": "virtio", "media": "disk", "zfs_filesystem": "zones/89168e3f-dec5-676e-879d-c90b69a7d988-disk0", "zpool": "zones", "size": 1024, "compression": "off", "refreservation": 1024, "block_size": 8192 } ], "nics": [ { "interface": "net0", "mac": "52:54:00:b6:c8:21", "vlan_id": 100, "nic_tag": "firewall", "ip": "dhcp", "ips": [ "dhcp" ], "model": "virtio", "allow_dhcp_spoofing": false, "allow_ip_spoofing": false, "allow_mac_spoofing": false, "allow_restricted_traffic": false, "allow_unfiltered_promisc": false, "primary": true }, { "interface": "net1", "mac": "72:7c:9d:dd:6e:a3", "vlan_id": 173, "nic_tag": "firewall", "netmask": "255.255.255.0", "ip": "173.48.255.1", "ips": [ "173.48.255.1/24<http://173.48.255.1/24>" ], "model": "virtio", "allow_dhcp_spoofing": true, "allow_ip_spoofing": true, "allow_mac_spoofing": true, "allow_restricted_traffic": true, "allow_unfiltered_promisc": true } ], "uuid": "89168e3f-dec5-676e-879d-c90b69a7d988", "zone_state": "running", "zonepath": "/zones/89168e3f-dec5-676e-879d-c90b69a7d988", "zoneid": 6, "last_modified": "2018-02-18T12:59:40.000Z", "resolvers": [], "firewall_enabled": false, "server_uuid": "20c32547-dad7-dd11-abd6-d017c2956b6b", "platform_buildstamp": "20180203T031130Z", "state": "running", "boot_timestamp": "2018-02-18T12:59:40.000Z", "pid": 5356, "customer_metadata": {}, "internal_metadata": {}, "routes": {}, "tags": {}, "quota": 10, "zfs_root_recsize": 131072, "zfs_filesystem": "zones/89168e3f-dec5-676e-879d-c90b69a7d988", "zpool": "zones", "snapshots": [] } [root@smugglers ~]# vmadm get c4cc5cd2-9d63-4546-c405-9e9bf724268a | json { "zonename": "c4cc5cd2-9d63-4546-c405-9e9bf724268a", "autoboot": true, "brand": "kvm", "limit_priv": "default,-file_link_any,-net_access,-proc_fork,-proc_info,-proc_session", "v": 1, "create_timestamp": "2017-10-27T16:13:45.003Z", "cpu_shares": 100, "max_lwps": 2000, "max_msg_ids": 4096, "max_sem_ids": 4096, "max_shm_ids": 4096, "max_shm_memory": 2048, "zfs_io_priority": 100, "max_physical_memory": 2048, "max_locked_memory": 2048, "max_swap": 2048, "billing_id": "00000000-0000-0000-0000-000000000000", "owner_uuid": "00000000-0000-0000-0000-000000000000", "alias": "Backup Opnsense Firewall", "ram": 1024, "vcpus": 2, "vnc_port": 99999, "disks": [ { "path": "/dev/zvol/rdsk/zones/c4cc5cd2-9d63-4546-c405-9e9bf724268a-disk0", "boot": true, "model": "virtio", "media": "disk", "zfs_filesystem": "zones/c4cc5cd2-9d63-4546-c405-9e9bf724268a-disk0", "zpool": "zones", "size": 5120, "compression": "off", "refreservation": 5120, "block_size": 8192 } ], "nics": [ { "interface": "net0", "mac": "ab:cd:ef:gh:ij:kl", "vlan_id": 10, "nic_tag": "firewall", "ip": "dhcp", "ips": [ "dhcp" ], "model": "virtio", "allow_dhcp_spoofing": false, "allow_ip_spoofing": false, "allow_mac_spoofing": false, "allow_restricted_traffic": false, "allow_unfiltered_promisc": false, "primary": true }, { "interface": "net1", "mac": "42:50:0e:e8:c7:28", "vlan_id": 100, "nic_tag": "firewall", "netmask": "255.255.255.0", "ip": "192.168.1.254", "ips": [ "192.168.1.254/24<http://192.168.1.254/24>" ], "model": "virtio", "allow_dhcp_spoofing": true, "allow_ip_spoofing": true, "allow_mac_spoofing": true, "allow_restricted_traffic": true, "allow_unfiltered_promisc": true }, { "interface": "net2", "mac": "f2:8d:d3:20:1c:20", "vlan_id": 200, "nic_tag": "firewall", "netmask": "255.255.255.224", "ip": "192.168.200.30", "ips": [ "192.168.200.30/27<http://192.168.200.30/27>" ], "model": "virtio", "allow_dhcp_spoofing": true, "allow_ip_spoofing": true, "allow_mac_spoofing": true, "allow_restricted_traffic": true, "allow_unfiltered_promisc": true } ], "uuid": "c4cc5cd2-9d63-4546-c405-9e9bf724268a", "zone_state": "running", "zonepath": "/zones/c4cc5cd2-9d63-4546-c405-9e9bf724268a", "zoneid": 7, "last_modified": "2018-02-18T12:59:41.000Z", "resolvers": [], "firewall_enabled": false, "server_uuid": "20c32547-dad7-dd11-abd6-d017c2956b6b", "platform_buildstamp": "20180203T031130Z", "state": "running", "boot_timestamp": "2018-02-18T12:59:41.000Z", "pid": 5431, "customer_metadata": {}, "internal_metadata": {}, "routes": {}, "tags": {}, "quota": 10, "zfs_root_recsize": 131072, "zfs_filesystem": "zones/c4cc5cd2-9d63-4546-c405-9e9bf724268a", "zpool": "zones", "snapshots": [] } On Mon, Feb 19, 2018 at 4:08 AM, Eugene Lee <[email protected]<mailto:[email protected]>> wrote: Hi, I am fairly new still to Smartos and learning my way around it. I have been in the process of migrating my environment from VMware ESXi to a Smartos host. I have a Vyos virtual router that I would like to migrate from VMware to Smartos. In VMware I know how to assign a trunk port to the Vyos virtual machine and then assign various IPs in different VLANs for routing in the Vyos router. I am not sure what the equivalent way of doing this is in SmartOs. I know that you can add the vlan_id tag to the JSON file for the KVM machine but given I want a trunk port assigned to the KVM machine (that will be running Vyos), how do I go about doing this? Hopefully what I want to achieve makes sense. Thanks, Eugene -- Greg http://www.linkedin.com/in/gregtreantos -- Greg http://www.linkedin.com/in/gregtreantos -- Greg http://www.linkedin.com/in/gregtreantos smartos-discuss | Archives<https://www.listbox.com/member/archive/184463/=now> [https://www.listbox.com/images/feed-icon-10x10.jpg071777d.jpg?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2ZlZWQtaWNvbi0xMHgxMC5qcGc] <https://www.listbox.com/member/archive/rss/184463/29581085-44f2ca1e> | Modify<https://www.listbox.com/member/?&;> Your Subscription [https://www.listbox.com/images/listbox-logo-small.png071777d.png?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2xpc3Rib3gtbG9nby1zbWFsbC5wbmc] <http://www.listbox.com> ------------------------------------------- smartos-discuss Archives: https://www.listbox.com/member/archive/184463/=now RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00 Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb Powered by Listbox: http://www.listbox.com
