Hi Jan,
but exactly this Setup works at Hetzner on our Rrootservers.
Hetzner sends the complete traffic to the MAIN IP: admin_ip=138.XXX.XX.XXA
The vnic0 gets the first IP from the SUBNET: 88.XXX.XXX.XXA
For IPv6 use vnic1 and plumb 2a01:AAAA:AAAA:AAAA::EEEA/64
Hetzner specifically you push the entire traffic over fe80::1
[root@root1 /zones/ass.de/template]# cat vm01-root1-fw1-opnsense.json
{
"brand": "kvm",
"alias": "root1-fw1-opnsense",
"resolvers": [
"8.8.8.8",
"8.8.4.4"
],
"ram": "4096",
"vcpus": "2",
"nics": [
{
"__comment" : "hetzner: 88.XXX.XXX.XXB",
"nic_tag": "admin",
"allowed_ips": [
"2a01:AAAA:AAAA:AAAA::B:CCCC"
],
"ip": "88.XXX.XXX.XXB",
"ips": ["88.XXX.XXX.XXB/29", "addrconf"],
"netmask": "255.255.255.248",
"gateway": "88.XXX.XXX.XXA",
"model": "virtio",
"primary": true
},
{
"__comment" : "internal: 10.XXX.XXX.XXD",
"nic_tag": "vswitch0",
"ip": "10.XXX.XXX.XXD",
"ips": ["10.XXX.XXX.XXD/22", "addrconf"],
"netmask": "255.255.252.0",
"gateway": "10.XXX.XXX.XXE",
"model": "virtio"
}
],
"disks": [
{
"boot": true,
"model": "virtio",
"compression": "lz4",
"size": 16384,
"block_size": 8192
}
]
}
[root@root1 /zones/ass.de/template]#
For security reasons create firewall rules at Hetzner Robot for the Root
Servers. (DROP all traffic to the MAIN IP, but allow all other for the Subnet
IPs)
vmadm update UUID vnc_port=ZZZA (this only activates the vnc port on the MAIN
IP) / to disable the vnc access -> use: vmadm update UUID vnc_port=-1
And with ssh -p XXXX -i /home/fuu/.ssh/id_bar -L 9999: 138.XXX.XX.XXA:ZZZA
[email protected]<mailto:[email protected]> you can tunnel the plain vnc
access locally
Works like a charm.
From linux I recognize that you can rewrite mac addresses on the bridge
(proxyarp) but I did not try this under smartos.
I have used a lot of network stuff in my LXC-to-GO Project:
https://github.com/plitc/lxc-to-go/blob/master/content/README.DIAGRAM.md
Or crazy stuff on FreeBSD with up to 256 Bridges:
https://blog.plitc.eu/2014/freebsd-10-komplexe-bridge-zones-mit-lacp-uplink/
But my impression is, the more one uses complicated techniques, the more
cumbersome it becomes to the conclusion to debug (like proxyarp, multiple
source & destination nat between vms on the same host)
😉
Mit freundlichen Grüßen
DANIEL PLOMINSKI
Leiter – IT / Head of IT
Telefon 09265 808-151 | Mobil 0151 58026316 |
[email protected]<mailto:[email protected]>
PGP Key: http://pgp.ass.de/2B4EB20A.key
[cid:C17DB6FB-5F79-4BCC-AAB4-CAB59266BC29@localdomain]
ASS-Einrichtungssysteme GmbH
ASS-Adam-Stegner-Straße 19 | D-96342 Stockheim
Geschäftsführer: Matthias Stegner, Michael Stegner, Stefan Weiß
Amtsgericht Coburg HRB 3395 | Ust-ID: DE218715721
[cid:E40AEC87-91EE-472A-901A-ECAD3F5801FB@localdomain]
Von: Ján Poctavek [mailto:[email protected]]
Gesendet: Dienstag, 12. September 2017 13:08
An: [email protected]
Betreff: Re: AW: [smartos-discuss] smartos in dedicated hosting
Thank you Daniel for sharing your setup. I use your scenario in some
installations, also with etherstubs and GZ routing.
But:
1. this is exactly I'd like to avoid - need for creating an own custom script
for networking
2. you are creating a vnic0 interface over e1000g0. It will not work with e.g.
Hetzner or OVH because you are changing the external MAC.
Jan
On 12. 9. 2017 11:17, Daniel Plominski wrote:
Hi Poctavek,
Example: DATACENTER <=> DC Switch <=> Rootserver (SmartOS + VMs)
SmartOS has 1 ADMIN interfac e with an additional /29 Subnet
[root@root1 /usbkey]# cat config
#
# This file was auto-generated and must be source-able by bash.
#
### ### ### ASS // ### ### ###
admin_nic=AA:BB:CC:DD:EE:00
admin_ip=dhcp
headnode_default_gateway=138.XXX.XX.XXF
dns_resolvers=8.8.8.8,8.8.4.4
dns_domain=ass.de
ntp_hosts=0.smartos.pool.ntp.org
compute_node_ntp_hosts=dhcp
... … …
### ### ### // ASS ### ### ###
# EOF
[root@root1 /usbkey]#
[root@root1 /opt/custom/smf]# cat subnet-routing-setup.xml
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='manifest' name='export'>
<service name='site/subnet-routing-setup' type='service' version='0'>
<create_default_instance enabled='true'/>
<single_instance/>
<dependency name='network' grouping='require_all' restart_on='error'
type='service'>
<service_fmri value='svc:/milestone/network:default'/>
</dependency>
<dependency name='filesystem' grouping='require_all' restart_on='error'
type='service'>
<service_fmri value='svc:/system/filesystem/local'/>
</dependency>
<exec_method name='start' type='method'
exec='/opt/custom/scripts/subnet-routing-setup' timeout_seconds='60'>
<method_context>
<method_credential user='root' group='staff'/>
<method_environment>
<envvar name='PATH' value='/usr/bin:/usr/sbin:/bin'/>
</method_environment>
</method_context>
</exec_method>
<exec_method name='restart' type='method' exec=':kill' timeout_seconds='60'>
<method_context>
<method_credential user='root' group='staff'/>
</method_context>
</exec_method>
<exec_method name='stop' type='method' exec=':kill' timeout_seconds='60'>
<method_context>
<method_credential user='root' group='staff'/>
</method_context>
</exec_method>
<property_group name='startd' type='framework'>
<propval name='duration' type='astring' value='transient'/>
<propval name='ignore_error' type='astring' value='core,signal'/>
</property_group>
<property_group name='application' type='application'/>
<stability value='Evolving'/>
<template>
<common_name>
<loctext xml:lang='C'>subnet-routing-setup</loctext>
</common_name>
</template>
</service>
</service_bundle>
[root@root1 /opt/custom/smf]#
[root@root1 /opt/custom/scripts]# cat subnet-routing-setup
#!/bin/sh
. /lib/svc/share/smf_include.sh
#// disable services
svcadm disable svc:/network/rpc/bind:default
#// HOST: ipv6
#/dladm create-vnic -l e1000g0 vnic1
ifconfig e1000g0 inet6 plumb
ifconfig e1000g0 inet6 addif 2a01:AAAA:AAAA:AAAA::EEEA/64 up
route add -inet6 fe80::1 2a01:AAAA:AAAA:AAAA::EEEA -interface
route add -inet6 default fe80::1
svcadm enable ipv6-forwarding
routeadm -e ipv6-forwarding
routeadm -e ipv6-routing
routeadm -u
#// VM: ipv4
dladm create-vnic -l e1000g0 vnic0
ifconfig vnic0 plumb 88.XXX.XXX.XXA netmask 255.255.255.248 up
svcadm enable route
routeadm -e ipv4-forwarding
routeadm -e ipv4-routing
routeadm -u
#// VM: internal vswitch (intern / ass vpn)
# create a etherstub
dladm create-etherstub vswitch0
dladm set-linkprop -p mtu=1500 vswitch0
#// VM: internal vswitch (intern / coorp vpn)
# create a etherstub
dladm create-etherstub vswitch1
dladm set-linkprop -p mtu=1500 vswitch1
exit $SMF_EXIT_OK
[root@root1 /opt/custom/scripts]#
Now use the SmartOS IP: 88.XXX.XXX.XXA as default gateway for ALL your Zone /
KVM Machines
Another method would be: NAT
Mit freundlichen Grüßen
DANIEL PLOMINSKI
Leiter – IT / Head of IT
Telefon 09265 808-151 | Mobil 0151 58026316 |
[email protected]<mailto:[email protected]>
PGP Key: http://pgp.ass.de/2B4EB20A.key
[cid: C17DB6FB-5F79-4BCC-AAB4-CAB59266BC29@localdomain]
ASS-Einrichtungssysteme GmbH
ASS-Adam-Stegner-Straße 19 | D-96342 Stockheim
Geschäftsführer: Matthias Stegner, Michael Stegner, Stefan Weiß
Amtsgericht Coburg HRB 3395 | Ust-ID: DE218715721
[cid: E40AEC87-91EE-472A-901A-ECAD3F5801FB@localdomain]
-----Ursprüngliche Nachricht-----
Von: Ján Poctavek [mailto:[email protected]]
Gesendet: Dienstag, 12. September 2017 10:45
An: [email protected]<mailto:[email protected]>
Betreff: [smartos-discuss] smartos in dedicated hosting
Hi guys,
I have a bit of complications installing SmartOS in the dedicated hosting.
Many hosting providers have an additional security (network filter) that
allows a dedicated server to communicate to the internet only by using the
assigned IP address *together* with the default MAC address. But when I
configure the external interface with IP address in the config file, the IP
address is created over external0 vnic. And this new vnic has a new MAC address
that is different from default HW NIC address. As a result, all communication
gets dropped.
Is there a way to solve this using a config file?
The workarounds I can come with:
1. add a new SMF service that manually adds the IP address over the physical
NIC
2. modify the network/physical script
3. add <nictag>_preserve_mac config property to add IP address directly to
physical NIC
The thing is that the first two options do not scale and I don't want to
implement the third if it already exists.
Thanks for hints.
Jan
smartos-discuss | Archives<https://www.listbox.com/member/archive/184463/=now>
[https://www.listbox.com/images/feed-icon-10x10.jpge18b463.jpg?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2ZlZWQtaWNvbi0xMHgxMC5qcGc]
<https://www.listbox.com/member/archive/rss/184463/29198361-7a6753c0> |
Modify<https://www.listbox.com/member/?&;> Your Subscription
[https://www.listbox.com/images/listbox-logo-small.pnge18b463.png?uri=aHR0cHM6Ly93d3cubGlzdGJveC5jb20vaW1hZ2VzL2xpc3Rib3gtbG9nby1zbWFsbC5wbmc]<http://www.listbox.com>
-------------------------------------------
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription:
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com
