here are updates to the documentation that include the RTF_ZONE related
changes, as well as those based on Erik's comments about allowing
persistent interface configuration management in the non-global zone.
The full text of the updated document is attached. The diffs are:
--- zonecfg.txt 2010/05/19 19:28:37 1.2
+++ zonecfg.txt 2010/05/25 15:08:12
@@ -79,10 +79,20 @@
other zonecfg(1m) resources such as the "physical" datalink. At the same
time, the zone boot process will also ensure 'ip-nospoof' protection
for the datalink with the specified addresses used as input to the
-'allowed-ips' property. The stored nvlist information for 'address'
-and 'defrouter' will be retrieved and re-applied in the non-global zone
-by the daemon associated with the ip-interface-management service
-before any other IP configuration is applied.
+'allowed-ips' property. On the first boot of an installed non-global
+zone, the stored nvlist information for 'address' will be retrieved by
+the ipmgmtd daemon which will create the interface persistently, and
+apply the IP addresses non-persistently before any other IP
+configuration is applied. On subsequent boots, any IP interface with
+persistent configuration in the ipadm data-store will be recreated using
+IP address information from the kernel's nvlists set up by zoneadmd.
+
+On every boot of a non-global zone, ipmgmtd will retrieve and attempt to
+re-apply any 'defrouter' information set up by the global zone before
+any other IP configuration is applied. Routes added by ipmgmtd will be
+identifiable in the 'netstat -r' output by the 'Z' flag which will
+indicate that the associated route was based on information configured
+in the global zone.
The GLDv3 layer communicates the set of "allowed-ips" to the IP layer
through DLPI notifications. The notification sent is a DL_NOTIFY_IND
@@ -93,13 +103,44 @@
the IP layer to track the current set of permitted IP addresses per
interface, so that the IP layer can return informative error diagnostics
if an attempt is made to enable forwarding or set addresses
-on the interface that are outside the permitted set.
+on the interface that are outside the permitted set. IP interfaces
+with 'allowed-ips' constraints will be identifiable by the "L3PROTECT"
+flag in the output of ifconfig, and by a 'Z' flag in the output of
+'ipadm show-if'.
All configuration state, including datalink properties like protection,
allowed-ips and zone, and associated nvlist information, that is
created during zone boot will be reset to the default values when the
zone is halted.
+Interfaces added by this case:
+------------------------------
+This case adds the following:
+
+- a new RTF_ZONE flag which will be passed down in the routing socket
+ message sent by the ipmgmtd daemon in the non-global zone to add
+ default route information set up via zonecfg(1m). The output of
+ 'netstat -r' will display 'Z' when the route was added by ipmgmtd in
+ this manner. The output of the 'route get' command will also display
+ the "ZONE" string as part of the 'flags' field for the route.
+
+- a read-only interface flag IFF_L3PROTECT which will be set by the
+ kernel on the IP interface when the 'allowed-ips' link property
+ has been set for the interface. The IFF_L3PROTECT flag will be indicated
+ by a 'Z' in the CURRENT column of the output from 'ipadm show-if'
+
+
+ Interface Classification Comments
+ _______________________________________________________________________
+ netstat(1m) -r output Uncommitted (unchanged)
+ RTF_ZONE Committed <net/route.h>
+ route(1m) Uncommitted (unchanged)
+ IFF_L3PROTECT Committed <net/if.h>, read-only
+
+The output of 'ipadm show-addr' will be modified to display a TYPE of
+'from-gz' when the address has been configured based on IP address
+information allocated to the non-global zone from the global zone.
+
Relationship to other configuration mechanisms
----------------------------------------------
@@ -191,16 +232,24 @@
r...@gz# zlogin tz1 ipadm show-addr
ADDROBJ TYPE STATE ADDR
- vnic0/_a static ok 11.1.1.1/24
- vnic0/_b static ok 12.2.3.4/16
+ vnic0/_a from-gz ok 11.1.1.1/24
+ vnic0/_b from-gz ok 12.2.3.4/16
lo0/v4 static ok 127.0.0.1/8
lo0/v6 static ok ::1/128
+ r...@gz# zlogin tz1 ipadm show-if
+ IFNAME STATE CURRENT PERSISTENT
+ vnic0 ok bm-------Z4- -4-
+ lo0 ok -m-v------46 ---
+
+
+
+
r...@gz# zlogin tz1 netstat -rn
Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ---------- ---------
- default 11.1.1.2 UG 2 2 vnic0
+ default 11.1.1.2 UGZ 2 2 vnic0
11.1.1.0 11.1.1.1 U 3 2 vnic0
12.2.0.0 12.2.3.4 U 2 0 vnic0
127.0.0.1 127.0.0.1 UH 2 0 lo0
@@ -211,6 +260,16 @@
::1 ::1 UH 2 0 lo0
+ ro...@gz# zlogin tz1 route get default
+ route to: default
+ destination: default
+ mask: default
+ gateway: 11.1.1.2
+ interface: vnic0
+ flags: <UP,GATEWAY,DONE,ZONE>
+ recvpipe sendpipe ssthresh rtt,ms rttvar,ms hopcount mtu
expire
+
+
Proposed man page changes:
--------------------------
--- zonecfg.1m.orig Thu Apr 22 14:41:33 2010
@@ -264,6 +323,78 @@
DHCP client ID (DUID for DHCPv6) and hardware
+--- netstat.orig Wed May 19 17:02:56 2010
++++ netstat.new Wed May 19 17:05:17 2010
+@@ -740,6 +740,9 @@
+ I Indirect routes (gateway not directly reachable) esta-
+ blished with the -indirect option.
+
++ Z (non-global zone only) The route was statically added
++ on boot based on routing information configured using
++ zonecfg(1m) in the global zone.
+
+
+ If the -a option is specified, there will be routing entries
+@@ -881,7 +884,7 @@
+ iostat(1M), kstat(1M), mibiisa(1M), savecore(1M),
+ vmstat(1M), hosts(4), inet_type(4), networks(4), proto-
+ cols(4), services(4), attributes(5), dhcp(5), kstat(7D),
+- inet(7P), inet6(7P)
++ inet(7P), inet6(7P), zonecfg(1m)
+
+
+ Droms, R., RFC 2131, Dynamic Host Configuration Protocol,
+
+--- ipadm.1m.orig Tue May 25 09:12:35 2010
++++ ipadm.1m.new Tue May 25 10:34:55 2010
+@@ -321,10 +321,11 @@
+ a VRRP interface is in accept mode
+ (~IFF_NOACCEPT)
+
++ Z Layer-3 protection of IP addresses for the
++ interface has been administratively enforced.
+
+
+
+-
+ SunOS 5.11 Last change: 29 Mar 2010 5
+
+
+@@ -891,9 +892,13 @@
+ TYPE
+
+ Type of the address object. It will be one of:
+- static, dhcp, or addrconf. It corresponds to the
+- type of the address object specified by the -T
+- option of create-addr.
++ from-gz, static, dhcp, or addrconf. The static,
++ dhcp and addrconf types correspond to the type of the
++ address object specified by the -T option of
++ create-addr. The "from-gz" type will only be displayed
++ in non-global zones, and indicates that the address
++ was configured based on IP address information
++ allocated to the non-global zone from the global zone.
+
+
+ STATE
+
+--- ifconfig.1m.orig Tue May 25 11:05:48 2010
++++ ifconfig.1m.new Tue May 25 11:07:31 2010
+@@ -1599,7 +1599,13 @@
+ used on multiple interfaces. (See also the usesrc
+ option.)
+
++ L3PROTECT
+
++ Indicates that Layer-3 protection has been enforced on
++ the physical interface using the 'allowed-ips' link
++ property in dladm(1m)
++
++
+ LOGICAL INTERFACES
+ Solaris TCP/IP allows multiple logical interfaces to be
+ associated with a physical network interface. This allows a
+
References:
----------
zonecfg(1m) enhancements for exclusive-IP zones
Requested release binding: Minor
Summary:
-------
This case proposes a solution for
6944327 need to support address and defrouter resources for exclusive-IP zones
Problem Description
--------------------
Typical zone deployments that exist today use shared-IP zones to run
applications and services like Apache or Weblogic in the contained
environment provided by the shared-IP zone. In these use-cases, the
Administrator in the global zone has full control over the networking
resources used by the non-global zone. In the common case, networking is
simply configured by specifying the IP interface, IP addresses and,
optionally, the default routers from zonecfg(1m). The configuration
resources thus supplied are then applied for the non-global
zone when it is booted, and the non-global zone itself may not modify
any of these configuration parameters.
However, there is no such simple configuration mechanism for the
simple networking use-case in place for exclusive IP zones, which must be
configured through sysidcfg, ifconfig, and an assortment of other
methods, all of which are not controllable from the global zone, and
may not be managed through zonecfg(1m).
The addition of many new virtualization and resource management
features to Solaris (PSARC 2006/357, PSARC 2005/132 and associated cases)
makes Exclusive-IP zones a cleaner and more powerful Zone model than
shared-IP zones. Thus, bridging any gaps in the configuration methods
for the simple use cases between shared- and exlusive-IP zones is
important to ease the transition of shared-IP customer configurations
to exlusive-IP.
In addition, there are many ongoing projects in the "Zones Networking"
effort [ZONES-NET] to facilitate the consolidation of existing Solaris 10
host installations as Solaris 10 Containers. These efforts would leverage
from the ability to specify networking resource values for address and
default router uniformly for zones using zonecfg(1m).
This case proposes support in zonecfg(1m) for 'address' and 'defrouter'
properties in the 'net' resource for exclusive-IP zones. The semantics
of the values will be the same as with the shared-IP zone definitions
that exist today ([PSARC/2002/174], [PSARC/2003/621], [PSARC/2008/057])
When the global-zone specifies the 'address' for an interface via
zonecfg(1m), the non-global zone may not use any other IP addresses for
that interface. The address information provided via zonecfg(1m) will
be used to set up Layer-3 protection [PSARC/2009/436] for the
non-global zone during zone-boot to filter out all other addresses for
the selected interface. For instance, when zonecfg(1m) has been used
in the global-zone to set one or more IPv4 addresses on an interface,
an attempt to set an IPv4 or IPv6 address on the interface that is
outside the globally defined set will encounter the EPERM failure.
Thus ifconfig(1m), ipadm(1m), and associated ioctls will receive this
error if they are used within the non-global zone to set addresses that
are not in the set that is permitted from the global-zone, and attempts
by the non-global zone to turn on forwarding on the interface will also
encounter EPERM.
Attempts to boot a zone that has already been configured for IP, or has
previously customized values for Layer-3 protection [PSARC/2009/436]
will fail.
Implementation Overview:
------------------------
A brief overview of the implementation is provided here. Note that all
interfaces between zoneadmd(1m) and the kernel/zonecfg(1m),
are Private interfaces, subject to change in the future.
When the non-global zone is booted, zoneadmd(1m) will store the information
specified by the 'address' and 'defrouter' properties as nvlists
in the kernel following a mechanism similar to that in use today for
other zonecfg(1m) resources such as the "physical" datalink. At the same
time, the zone boot process will also ensure 'ip-nospoof' protection
for the datalink with the specified addresses used as input to the
'allowed-ips' property. On the first boot of an installed non-global
zone, the stored nvlist information for 'address' will be retrieved by
the ipmgmtd daemon which will create the interface persistently, and
apply the IP addresses non-persistently before any other IP
configuration is applied. On subsequent boots, any IP interface with
persistent configuration in the ipadm data-store will be recreated using
IP address information from the kernel's nvlists set up by zoneadmd.
On every boot of a non-global zone, ipmgmtd will retrieve and attempt to
re-apply any 'defrouter' information set up by the global zone before
any other IP configuration is applied. Routes added by ipmgmtd will be
identifiable in the 'netstat -r' output by the 'Z' flag which will
indicate that the associated route was based on information configured
in the global zone.
The GLDv3 layer communicates the set of "allowed-ips" to the IP layer
through DLPI notifications. The notification sent is a DL_NOTIFY_IND
message of (Project Private) type DL_NOTE_ALLOWED_IPS, sent to the IP
clients that have registered for this notification. The payload of the
DL_NOTE_ALLOWED_IPS message is the mac_protect_t data-structure
introduced by PSARC 2009/436. Receipt of this notification enables
the IP layer to track the current set of permitted IP addresses per
interface, so that the IP layer can return informative error diagnostics
if an attempt is made to enable forwarding or set addresses
on the interface that are outside the permitted set. IP interfaces
with 'allowed-ips' constraints will be identifiable by the "L3PROTECT"
flag in the output of ifconfig, and by a 'Z' flag in the output of
'ipadm show-if'.
All configuration state, including datalink properties like protection,
allowed-ips and zone, and associated nvlist information, that is
created during zone boot will be reset to the default values when the
zone is halted.
Interfaces added by this case:
------------------------------
This case adds the following:
- a new RTF_ZONE flag which will be passed down in the routing socket
message sent by the ipmgmtd daemon in the non-global zone to add
default route information set up via zonecfg(1m). The output of
'netstat -r' will display 'Z' when the route was added by ipmgmtd in
this manner. The output of the 'route get' command will also display
the "ZONE" string as part of the 'flags' field for the route.
- a read-only interface flag IFF_L3PROTECT which will be set by the
kernel on the IP interface when the 'allowed-ips' link property
has been set for the interface. The IFF_L3PROTECT flag will be indicated
by a 'Z' in the CURRENT column of the output from 'ipadm show-if'
Interface Classification Comments
_______________________________________________________________________
netstat(1m) -r output Uncommitted (unchanged)
RTF_ZONE Committed <net/route.h>
route(1m) Uncommitted (unchanged)
IFF_L3PROTECT Committed <net/if.h>, read-only
The output of 'ipadm show-addr' will be modified to display a TYPE of
'from-gz' when the address has been configured based on IP address
information allocated to the non-global zone from the global zone.
Relationship to other configuration mechanisms
----------------------------------------------
sysidcfg:
The Solaris Install team is currently working in conjunction
with the Network Configuration team [INSTALL] on the implementation of
a framework to replace sysidcfg(4) that will provide configuration
profiles to be processed by startup scripts when a system boots. However
this configuration information is only applied on the first boot of the
zone, and unlike zonecfg, may not be easily modified in the global zone
across reboot. Moreover, there is no mechanism in place to prevent
the non-global zone from modifying the configuration provided via profiles.
As mentioned earlier, the global-zone is the authoritative source of
IP configuration information for the "secure container" use-case.
This implies that attempts to use boot profiles in the non-global zone
to add addresses other than those mandated in the global-zone will
fail with EINVAL erors.
Note that the profiles defined in [INSTALL] may still be applied to
interfaces for which 'net' resource properties have not been defined
in the global-zone, i.e., this proposal is fully backward-compatible.
svc:/network/physical:{default, nwam}
Configuration specified via zonecfg(1m) in the global-zone overrides
any information specified via /etc/hostname.intf, ipadm(1m) or nwam.
Thus if the IP address information specified by these other methods
is not a subset of the globally managed information, attempts to apply
it within the non-global zone will encounter an error.
DHCP:
It is assumed that when IP resources for an IP interface have been set up
in the global-zone using zonecfg(1m), then that interface has been marked
for static IP configuration only, and attempts to send out outgoing
DHCP DISCOVER packets will be dropped in GLDv3. Thus attempts to start
DHCP on these interfaces in the non-global zone will time out.
Examples
--------
r...@gz# zonecfg -z tz1
zonecfg:tz1> info
zonename: tz1
zonepath: /rpool/zones/tz1
brand: ipkg
autoboot: false
bootargs:
pool:
limitpriv:
scheduling-class:
ip-type: exclusive
hostid:
net:
address: 11.1.1.1/24
physical: vnic0
defrouter: 11.1.1.2
net:
address: taddr2
physical: vnic0
defrouter not specified
net:
address not specified
physical: vnic1
defrouter not specified
In this example, the zone 'tz1' has been assigned 2 interfaces: vnic0 and vnic1.
The vnic1 interface does not have any IP address or default router assigned
to it, so that it may be configured in any desired way from within the zone
using ifconfig(1m), ipadm(1m), NWAM etc.
The vnic0 interface has 2 addresses assigned: 11.1.1.1/24 (with default
router 11.1.1.2 on the 11.1.1.0/24 subnet) and the address 'taddr2' whose
numeric value and netmask will be resolved when the zone boots. Thus,
if, at the time of zone boot, the global-zone is also configured as:
r...@gz# egrep 'hosts|netmask' /etc/nsswitch.conf
hosts: files nis dns
netmasks: files nis
r...@gz# grep taddr2 /etc/hosts
12.2.3.4 taddr2
r...@gz# grep 12 /etc/netmasks
12.2.0.0 255.255.0.0
tz1 will be booted as
r...@gz# zoneadm -z tz1 boot
r...@gz# zlogin tz1 ipadm show-addr
ADDROBJ TYPE STATE ADDR
vnic0/_a from-gz ok 11.1.1.1/24
vnic0/_b from-gz ok 12.2.3.4/16
lo0/v4 static ok 127.0.0.1/8
lo0/v6 static ok ::1/128
r...@gz# zlogin tz1 ipadm show-if
IFNAME STATE CURRENT PERSISTENT
vnic0 ok bm-------Z4- -4-
lo0 ok -m-v------46 ---
r...@gz# zlogin tz1 netstat -rn
Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ---------- ---------
default 11.1.1.2 UGZ 2 2 vnic0
11.1.1.0 11.1.1.1 U 3 2 vnic0
12.2.0.0 12.2.3.4 U 2 0 vnic0
127.0.0.1 127.0.0.1 UH 2 0 lo0
Routing Table: IPv6
Destination/Mask Gateway Flags Ref Use If
--------------------------- ---------------------- ----- --- ------- -----
::1 ::1 UH 2 0 lo0
ro...@gz# zlogin tz1 route get default
route to: default
destination: default
mask: default
gateway: 11.1.1.2
interface: vnic0
flags: <UP,GATEWAY,DONE,ZONE>
recvpipe sendpipe ssthresh rtt,ms rttvar,ms hopcount mtu
expire
Proposed man page changes:
--------------------------
--- zonecfg.1m.orig Thu Apr 22 14:41:33 2010
+++ zonecfg.1m.new Mon May 17 17:09:46 2010
@@ -676,10 +676,17 @@
for the interface should be specified here.
For an exclusive-IP zone, the physical property must be
- set and the address and default router properties cannot
- be set.
+ set and the address and default router properties may
+ be set. The interface specifed for the physical property
+ must not be in use in the global zone. If an address and
+ default router are specified via zonecfg(1m), these will be
+ applied to the interface when the non-global zone is booted,
+ and the non-global zone will not be able to apply any other
+ addresses to that interface nor transmit packets with a different
+ source address for that IP Version. A default router set up via
+ zonecfg(1m) may not be persistently deleted from within the
+ non-global zone using the -p flag with route(1m).
-
device: match
Device name to match.
@@ -1817,7 +1824,7 @@
svcadm(1M), sysidtool(1M), zfs(1M), zoneadm(1M),
priv_str_to_set(3C), kstat(3KSTAT), vfstab(4), attri-
butes(5), brands(5), fnmatch(5), lx(5), privileges(5),
- resource_controls(5), zones(5)
+ resource_controls(5), zones(5), route(1m)
System Administration Guide: Solaris Containers-Resource
--- dladm.1m.txt Mon May 10 15:24:49 2010
+++ dladm.1m.txt.new Fri May 14 10:24:02 2010
@@ -4176,7 +4176,14 @@
option's link layer address must match the
link's MAC address.
+ When a datalink has been protected by setting allowed-ips
+ to a set of one or more IP addresses, IP interfaces over
+ that datalink may not configure IP addresses outside the
+ allowed-ips set, or enable forwarding for the IP interface
+ and attempts to do so will fail with an EPERM error being
+ returned to the user.
+
dhcp-nospoof
DHCP client ID (DUID for DHCPv6) and hardware
--- netstat.orig Wed May 19 17:02:56 2010
+++ netstat.new Wed May 19 17:05:17 2010
@@ -740,6 +740,9 @@
I Indirect routes (gateway not directly reachable) esta-
blished with the -indirect option.
+ Z (non-global zone only) The route was statically added
+ on boot based on routing information configured using
+ zonecfg(1m) in the global zone.
If the -a option is specified, there will be routing entries
@@ -881,7 +884,7 @@
iostat(1M), kstat(1M), mibiisa(1M), savecore(1M),
vmstat(1M), hosts(4), inet_type(4), networks(4), proto-
cols(4), services(4), attributes(5), dhcp(5), kstat(7D),
- inet(7P), inet6(7P)
+ inet(7P), inet6(7P), zonecfg(1m)
Droms, R., RFC 2131, Dynamic Host Configuration Protocol,
--- ipadm.1m.orig Tue May 25 09:12:35 2010
+++ ipadm.1m.new Tue May 25 10:34:55 2010
@@ -321,10 +321,11 @@
a VRRP interface is in accept mode
(~IFF_NOACCEPT)
+ Z Layer-3 protection of IP addresses for the
+ interface has been administratively enforced.
-
SunOS 5.11 Last change: 29 Mar 2010 5
@@ -891,9 +892,13 @@
TYPE
Type of the address object. It will be one of:
- static, dhcp, or addrconf. It corresponds to the
- type of the address object specified by the -T
- option of create-addr.
+ from-gz, static, dhcp, or addrconf. The static,
+ dhcp and addrconf types correspond to the type of the
+ address object specified by the -T option of
+ create-addr. The "from-gz" type will only be displayed
+ in non-global zones, and indicates that the address
+ was configured based on IP address information
+ allocated to the non-global zone from the global zone.
STATE
--- ifconfig.1m.orig Tue May 25 11:05:48 2010
+++ ifconfig.1m.new Tue May 25 11:07:31 2010
@@ -1599,7 +1599,13 @@
used on multiple interfaces. (See also the usesrc
option.)
+ L3PROTECT
+ Indicates that Layer-3 protection has been enforced on
+ the physical interface using the 'allowed-ips' link
+ property in dladm(1m)
+
+
LOGICAL INTERFACES
Solaris TCP/IP allows multiple logical interfaces to be
associated with a physical network interface. This allows a
References:
----------
[PSARC/2002/174] Virtualization and Namespace Isolation in Solaris, aka "Zones"
[PSARC/2003/621] Zone Administration Updates
[PSARC/2008/057] Default Route For Zones
[PSARC/2009/436] Anti-spoofing Link Protection
[INSTALL]
http://solaris-networking.sfbay/index.php/Network_Configuration:install
[ZONES-NET] http://solaris-networking.sfbay/index.php/Zones_Networking
_______________________________________________
opensolaris-arc mailing list
[email protected]
