I see. You are using the admin nictag as external interface and then you add additional vnic with provider-assigned failover subnet.
On 12 September 2017 14:01:23 CEST, Daniel Plominski <[email protected]> wrote: >Hi Jan, > >but exactly this Setup works at Hetzner on our Rrootservers. > >Hetzner sends the complete traffic to the MAIN IP: >admin_ip=138.XXX.XX.XXA >The vnic0 gets the first IP from the SUBNET: 88.XXX.XXX.XXA > >For IPv6 use vnic1 and plumb 2a01:AAAA:AAAA:AAAA::EEEA/64 >Hetzner specifically you push the entire traffic over fe80::1 > >[root@root1 /zones/ass.de/template]# cat vm01-root1-fw1-opnsense.json > >{ > "brand": "kvm", > "alias": "root1-fw1-opnsense", > "resolvers": [ > "8.8.8.8", > "8.8.4.4" > ], > "ram": "4096", > "vcpus": "2", > "nics": [ > { > "__comment" : "hetzner: 88.XXX.XXX.XXB", > "nic_tag": "admin", > "allowed_ips": [ > "2a01:AAAA:AAAA:AAAA::B:CCCC" > ], > "ip": "88.XXX.XXX.XXB", > "ips": ["88.XXX.XXX.XXB/29", "addrconf"], > "netmask": "255.255.255.248", > "gateway": "88.XXX.XXX.XXA", > "model": "virtio", > "primary": true > }, > { > "__comment" : "internal: 10.XXX.XXX.XXD", > "nic_tag": "vswitch0", > "ip": "10.XXX.XXX.XXD", > "ips": ["10.XXX.XXX.XXD/22", "addrconf"], > "netmask": "255.255.252.0", > "gateway": "10.XXX.XXX.XXE", > "model": "virtio" > } > ], > "disks": [ > { > "boot": true, > "model": "virtio", > "compression": "lz4", > "size": 16384, > "block_size": 8192 > } > ] >} > >[root@root1 /zones/ass.de/template]# >For security reasons create firewall rules at Hetzner Robot for the >Root Servers. (DROP all traffic to the MAIN IP, but allow all other for >the Subnet IPs) >vmadm update UUID vnc_port=ZZZA (this only activates the vnc port on >the MAIN IP) / to disable the vnc access -> use: vmadm update UUID >vnc_port=-1 >And with ssh -p XXXX -i /home/fuu/.ssh/id_bar -L 9999: >138.XXX.XX.XXA:ZZZA [email protected]<mailto:[email protected]> you >can tunnel the plain vnc access locally >Works like a charm. > >From linux I recognize that you can rewrite mac addresses on the bridge >(proxyarp) but I did not try this under smartos. > >I have used a lot of network stuff in my LXC-to-GO Project: >https://github.com/plitc/lxc-to-go/blob/master/content/README.DIAGRAM.md > >Or crazy stuff on FreeBSD with up to 256 Bridges: >https://blog.plitc.eu/2014/freebsd-10-komplexe-bridge-zones-mit-lacp-uplink/ > >But my impression is, the more one uses complicated techniques, the >more cumbersome it becomes to the conclusion to debug (like proxyarp, >multiple source & destination nat between vms on the same host) > >😉 > >Mit freundlichen Grüßen > > >DANIEL PLOMINSKI >Leiter – IT / Head of IT > >Telefon 09265 808-151 | Mobil 0151 58026316 | >[email protected]<mailto:[email protected]> >PGP Key: http://pgp.ass.de/2B4EB20A.key > > >[cid:C17DB6FB-5F79-4BCC-AAB4-CAB59266BC29@localdomain] > >ASS-Einrichtungssysteme GmbH >ASS-Adam-Stegner-Straße 19 | D-96342 Stockheim > >Geschäftsführer: Matthias Stegner, Michael Stegner, Stefan Weiß >Amtsgericht Coburg HRB 3395 | Ust-ID: DE218715721 > >[cid:E40AEC87-91EE-472A-901A-ECAD3F5801FB@localdomain] > >Von: Ján Poctavek [mailto:[email protected]] >Gesendet: Dienstag, 12. September 2017 13:08 >An: [email protected] >Betreff: Re: AW: [smartos-discuss] smartos in dedicated hosting > > >Thank you Daniel for sharing your setup. I use your scenario in some >installations, also with etherstubs and GZ routing. >But: > >1. this is exactly I'd like to avoid - need for creating an own custom >script for networking > >2. you are creating a vnic0 interface over e1000g0. It will not work >with e.g. Hetzner or OVH because you are changing the external MAC. > >Jan >On 12. 9. 2017 11:17, Daniel Plominski wrote: > >Hi Poctavek, > > > >Example: DATACENTER <=> DC Switch <=> Rootserver (SmartOS + VMs) > > > >SmartOS has 1 ADMIN interfac e with an additional /29 Subnet > > >[root@root1 /usbkey]# cat config ># ># This file was auto-generated and must be source-able by bash. ># >### ### ### ASS // ### ### ### > >admin_nic=AA:BB:CC:DD:EE:00 >admin_ip=dhcp >headnode_default_gateway=138.XXX.XX.XXF > >dns_resolvers=8.8.8.8,8.8.4.4 >dns_domain=ass.de > >ntp_hosts=0.smartos.pool.ntp.org >compute_node_ntp_hosts=dhcp > >... … … > >### ### ### // ASS ### ### ### ># EOF >[root@root1 /usbkey]# > >[root@root1 /opt/custom/smf]# cat subnet-routing-setup.xml ><!DOCTYPE service_bundle SYSTEM >'/usr/share/lib/xml/dtd/service_bundle.dtd.1'> ><service_bundle type='manifest' name='export'> > <service name='site/subnet-routing-setup' type='service' version='0'> > <create_default_instance enabled='true'/> > <single_instance/> ><dependency name='network' grouping='require_all' restart_on='error' >type='service'> > <service_fmri value='svc:/milestone/network:default'/> > </dependency> ><dependency name='filesystem' grouping='require_all' restart_on='error' >type='service'> > <service_fmri value='svc:/system/filesystem/local'/> > </dependency> ><exec_method name='start' type='method' >exec='/opt/custom/scripts/subnet-routing-setup' timeout_seconds='60'> > <method_context> > <method_credential user='root' group='staff'/> > <method_environment> > <envvar name='PATH' value='/usr/bin:/usr/sbin:/bin'/> > </method_environment> > </method_context> > </exec_method> ><exec_method name='restart' type='method' exec=':kill' >timeout_seconds='60'> > <method_context> > <method_credential user='root' group='staff'/> > </method_context> > </exec_method> ><exec_method name='stop' type='method' exec=':kill' >timeout_seconds='60'> > <method_context> > <method_credential user='root' group='staff'/> > </method_context> > </exec_method> > <property_group name='startd' type='framework'> > <propval name='duration' type='astring' value='transient'/> > <propval name='ignore_error' type='astring' value='core,signal'/> > </property_group> > <property_group name='application' type='application'/> > <stability value='Evolving'/> > <template> > <common_name> > <loctext xml:lang='C'>subnet-routing-setup</loctext> > </common_name> > </template> > </service> ></service_bundle> >[root@root1 /opt/custom/smf]# > >[root@root1 /opt/custom/scripts]# cat subnet-routing-setup >#!/bin/sh > >. /lib/svc/share/smf_include.sh > >#// disable services >svcadm disable svc:/network/rpc/bind:default > >#// HOST: ipv6 >#/dladm create-vnic -l e1000g0 vnic1 >ifconfig e1000g0 inet6 plumb >ifconfig e1000g0 inet6 addif 2a01:AAAA:AAAA:AAAA::EEEA/64 up >route add -inet6 fe80::1 2a01:AAAA:AAAA:AAAA::EEEA -interface >route add -inet6 default fe80::1 >svcadm enable ipv6-forwarding >routeadm -e ipv6-forwarding >routeadm -e ipv6-routing >routeadm -u > >#// VM: ipv4 >dladm create-vnic -l e1000g0 vnic0 >ifconfig vnic0 plumb 88.XXX.XXX.XXA netmask 255.255.255.248 up >svcadm enable route >routeadm -e ipv4-forwarding >routeadm -e ipv4-routing >routeadm -u > >#// VM: internal vswitch (intern / ass vpn) ># create a etherstub >dladm create-etherstub vswitch0 >dladm set-linkprop -p mtu=1500 vswitch0 > >#// VM: internal vswitch (intern / coorp vpn) ># create a etherstub > dladm create-etherstub vswitch1 > dladm set-linkprop -p mtu=1500 vswitch1 > > exit $SMF_EXIT_OK > > [root@root1 /opt/custom/scripts]# > > Now use the SmartOS IP: 88.XXX.XXX.XXA as default gateway for ALL your > Zone / KVM Machines > > Another method would be: NAT > > Mit freundlichen Grüßen > > DANIEL PLOMINSKI > Leiter – IT / Head of IT > > Telefon 09265 808-151 | Mobil 0151 58026316 | > [email protected]<mailto:[email protected]> > PGP Key: http://pgp.ass.de/2B4EB20A.key > > [cid: C17DB6FB-5F79-4BCC-AAB4-CAB59266BC29@localdomain] > > ASS-Einrichtungssysteme GmbH > ASS-Adam-Stegner-Straße 19 | D-96342 Stockheim > > Geschäftsführer: Matthias Stegner, Michael Stegner, Stefan Weiß > Amtsgericht Coburg HRB 3395 | Ust-ID: DE218715721 > > [cid: E40AEC87-91EE-472A-901A-ECAD3F5801FB@localdomain] > > -----Ursprüngliche Nachricht----- > Von: Ján Poctavek [mailto:[email protected]] > Gesendet: Dienstag, 12. September 2017 10:45 > An: > [email protected]<mailto:[email protected]> > Betreff: [smartos-discuss] smartos in dedicated hosting > > Hi guys, > > I have a bit of complications installing SmartOS in the dedicated > hosting. > > Many hosting providers have an additional security (network filter) > that allows a dedicated server to communicate to the internet only by > using the assigned IP address *together* with the default MAC address. > But when I configure the external interface with IP address in the > config file, the IP address is created over external0 vnic. And this > new vnic has a new MAC address that is different from default HW NIC > address. As a result, all communication gets dropped. > > Is there a way to solve this using a config file? > > The workarounds I can come with: > > 1. add a new SMF service that manually adds the IP address over the > physical NIC > > 2. modify the network/physical script > > 3. add <nictag>_preserve_mac config property to add IP address > directly to physical NIC > > The thing is that the first two options do not scale and I don't want > to implement the third if it already exists. > > Thanks for hints. > > Jan > > smartos-discuss | > Archives<https://www.listbox.com/member/archive/184463/=now> > [https://www.listbox.com/images/feed-icon-10x10.jpge18b463.jpg?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2ZlZWQtaWNvbi0xMHgxMC5qcGc] > <https://www.listbox.com/member/archive/rss/184463/29198361-7a6753c0> | > Modify<https://www.listbox.com/member/?&;> Your Subscription > > [https://www.listbox.com/images/listbox-logo-small.pnge18b463.png?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2xpc3Rib3gtbG9nby1zbWFsbC5wbmc]<http://www.listbox.com> > ------------------------------------------- smartos-discuss Archives: https://www.listbox.com/member/archive/184463/=now RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00 Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb Powered by Listbox: http://www.listbox.com
