On (06/02/10 12:01), Edward Pilatowicz wrote:
> > > so i had some concerns about one small bit below...
following up on this...
Ed, Erik and I have had several discussions about when the proposed
IP address configuration of zonecfg IP address and default router
parameters should be applied (first boot of zone vs every boot of
zone). To summarize this in Ed's words:
" we still have different opinions on how this functionality
should work but we were able to come up with a compromise solution.
that compromise would be to rename the proposed zonecfg net resource
property from "address=" to "allowed-address=". this compromise would
prevent user confusion resulting from the same zonecfg property having
different semantics in shared vs exclusive ip stack zones."
Updated material is enclosed. These updates reflect the rename
from 'address' to 'allowed-address'. In addition, man pages have been
modified to explicitly document how an exclusive IP NGZ may recover the
current set of allowed-address values for a from-gz interfaces at any time.
The primary differences between 'address' and 'allowed-address' are
1. "address" is always enforced for a shared-IP zone when it boots, and is
always configured for the NGZ
2. "allowed-address" for an NGZ is configured via ipadm in the
NGZ, and all interfaces in the NGZ are subsequently managed by ipadm.
The NGZ may use 'ipadm delete-if' to persistently delete the assigned
interface if desired, but it may not violate the resource controls
applied from the GZ.
For both "address" and "allowed-address", the NGZ may not use any IP
addresses other than those specified by the [allowed-]address set.
--Sowmini
zonecfg(1m) enhancements for exclusive-IP zones
Requested release binding: Minor
Summary:
-------
This case proposes a solution for
6944327 need to support address and defrouter resources for exclusive-IP zones
Problem Description
--------------------
Typical zone deployments that exist today use shared-IP zones to run
applications and services like Apache or Weblogic in the contained
environment provided by the shared-IP zone. In these use-cases, the
Administrator in the global zone has full control over the networking
resources used by the non-global zone. In the common case, networking is
simply configured by specifying the IP interface, IP addresses and,
optionally, the default routers from zonecfg(1m). The configuration
resources thus supplied are then applied for the non-global
zone when it is booted, and the non-global zone itself may not modify
any of these configuration parameters.
However, there is no such simple configuration mechanism for the
simple networking use-case in place for exclusive IP zones, which must be
configured through sysidcfg, ifconfig, and an assortment of other
methods, all of which are not controllable from the global zone, and
may not be managed through zonecfg(1m).
The addition of many new virtualization and resource management
features to Solaris (PSARC 2006/357, PSARC 2005/132 and associated cases)
makes Exclusive-IP zones a cleaner and more powerful Zone model than
shared-IP zones. Thus, bridging any gaps in the configuration methods
for the simple use cases between shared- and exlusive-IP zones is
important to ease the transition of shared-IP customer configurations
to exlusive-IP.
In addition, there are many ongoing projects in the "Zones Networking"
effort [ZONES-NET] to facilitate the consolidation of existing Solaris 10
host installations as Solaris 10 Containers. These efforts would leverage
from the ability to specify networking resource values for address and
default router uniformly for zones using zonecfg(1m).
This case proposes support in zonecfg(1m) for IP configuration
through the support of 'allowed-address' and 'defrouter'
properties in the 'net' resource for exclusive-IP zones.
When the global-zone specifies the 'allowed-address' for an interface via
zonecfg(1m), the non-global zone may not use any other IP addresses for
that interface. The address information provided via zonecfg(1m) will
be used to set up Layer-3 protection [PSARC/2009/436] for the
non-global zone during zone-boot to filter out all other addresses for
the selected interface. For instance, when zonecfg(1m) has been used
in the global-zone to set one or more IPv4 addresses on an interface,
an attempt to set an IPv4 or IPv6 address on the interface that is
outside the globally defined set will encounter the EPERM failure.
Thus ifconfig(1m), ipadm(1m), and associated ioctls will receive this
error if they are used within the non-global zone to set addresses that
are not in the set that is permitted from the global-zone, and attempts
by the non-global zone to turn on forwarding on the interface will also
encounter EPERM.
Attempts to boot a zone that has already been configured for IP, or has
previously customized values for Layer-3 protection [PSARC/2009/436]
will fail.
Implementation Overview:
------------------------
A brief overview of the implementation is provided here. Note that all
interfaces between zoneadmd(1m) and the kernel/zonecfg(1m),
are Private interfaces, subject to change in the future.
When the non-global zone is booted, zoneadmd(1m) will store the information
specified by the 'allowed-address' and 'defrouter' properties as nvlists
in the kernel following a mechanism similar to that in use today for
other zonecfg(1m) resources such as the "physical" datalink. At the same
time, the zone boot process will also ensure 'ip-nospoof' protection
for the datalink with the specified addresses used as input to the
'allowed-ips' property. On the first boot of an installed non-global
native (ipkg) zone, and on every boot of a solaris10 branded
zone [PSARC 2010/111], the stored nvlist information for
'allowed-address' will be retrieved by the ipmgmtd daemon which will
create the interface (persistently, for native ipkg zones), and apply
the IP addresses non-persistently before any other IP configuration is
applied. On subsequent boots of native (ipkg) zones, any IP interface
with persistent configuration in the ipadm data-store will be recreated
using IP address information from the kernel's nvlists set up by zoneadmd.
The 'defrouter' information set up by the global zone will be applied
after the 'allowed-address' information has been configured, so that
the behavior of the IP configuration scheme matches that of other
address acquisition protocols like DHCP. Routes added by ipmgmtd will
be identifiable in the 'netstat -r' output by the 'Z' flag which will
indicate that the associated route was based on information configured
in the global zone.
The GLDv3 layer communicates the set of "allowed-ips" to the IP layer
through DLPI notifications. The notification sent is a DL_NOTIFY_IND
message of (Project Private) type DL_NOTE_ALLOWED_IPS, sent to the IP
clients that have registered for this notification. The payload of the
DL_NOTE_ALLOWED_IPS message is the mac_protect_t data-structure
introduced by PSARC 2009/436. Receipt of this notification enables
the IP layer to track the current set of permitted IP addresses per
interface, so that the IP layer can return informative error diagnostics
if an attempt is made to enable forwarding or set addresses
on the interface that are outside the permitted set. IP interfaces
with 'allowed-ips' constraints will be identifiable by the "L3PROTECT"
flag in the output of ifconfig, and by a 'Z' flag in the output of
'ipadm show-if'.
All configuration state, including datalink properties like protection,
allowed-ips and zone, and associated nvlist information, that is
created during zone boot will be reset to the default values when the
zone is halted.
Interfaces added by this case:
------------------------------
This case adds the following:
- a new RTF_ZONE flag which will be passed down in the routing socket
message sent by the ipmgmtd daemon in the non-global zone to add
default route information set up via zonecfg(1m). The output of
'netstat -r' will display 'Z' when the route was added by ipmgmtd in
this manner. The output of the 'route get' command will also display
the "ZONE" string as part of the 'flags' field for the route.
- a read-only interface flag IFF_L3PROTECT which will be set by the
kernel on the IP interface when the 'allowed-ips' link property
has been set for the interface. The IFF_L3PROTECT flag will be indicated
by a 'Z' in the CURRENT column of the output from 'ipadm show-if'
Interface Classification Comments
_______________________________________________________________________
netstat(1m) -r output Uncommitted (unchanged)
RTF_ZONE Committed <net/route.h>
route(1m) Uncommitted (unchanged)
IFF_L3PROTECT Committed <net/if.h>, read-only
The output of 'ipadm show-addr' will be modified to display a TYPE of
'from-gz' when the address has been configured based on IP address
information allocated to the non-global zone from the global zone.
Relationship to other configuration mechanisms
----------------------------------------------
sysidcfg:
The Solaris Install team is currently working in conjunction
with the Network Configuration team [INSTALL] on the implementation of
a framework to replace sysidcfg(4) that will provide configuration
profiles to be processed by startup scripts when a system boots. However
this configuration information is only applied on the first boot of the
zone, and unlike zonecfg, may not be easily modified in the global zone
across reboot. Moreover, there is no mechanism in place to prevent
the non-global zone from modifying the configuration provided via profiles.
As mentioned earlier, the global-zone is the authoritative source of
IP configuration information for the "secure container" use-case.
This implies that attempts to use boot profiles in the non-global zone
to add addresses other than those mandated in the global-zone will
fail with EINVAL erors.
Note that the profiles defined in [INSTALL] may still be applied to
interfaces for which 'net' resource properties have not been defined
in the global-zone, i.e., this proposal is fully backward-compatible.
svc:/network/physical:{default, nwam}
Configuration specified via zonecfg(1m) in the global-zone overrides
any information specified via /etc/hostname.intf, ipadm(1m) or nwam.
Thus if the IP address information specified by these other methods
is not a subset of the globally managed information, attempts to apply
it within the non-global zone will encounter an error.
DHCP:
It is assumed that when IP resources for an IP interface have been set up
in the global-zone using zonecfg(1m), then that interface has been marked
for static IP configuration only, and attempts to send out outgoing
DHCP DISCOVER packets will be dropped in GLDv3. Thus attempts to start
DHCP on these interfaces in the non-global zone will time out.
Examples
--------
r...@gz# zonecfg -z tz1
zonecfg:tz1> info
zonename: tz1
zonepath: /rpool/zones/tz1
brand: ipkg
autoboot: false
bootargs:
pool:
limitpriv:
scheduling-class:
ip-type: exclusive
hostid:
net:
allowed-address: 11.1.1.1/24
physical: vnic0
defrouter: 11.1.1.2
net:
allowed-address: taddr2
physical: vnic0
defrouter not specified
net:
address not specified
physical: vnic1
defrouter not specified
In this example, the zone 'tz1' has been assigned 2 interfaces: vnic0 and vnic1.
The vnic1 interface does not have any IP address or default router assigned
to it, so that it may be configured in any desired way from within the zone
using ifconfig(1m), ipadm(1m), NWAM etc.
The vnic0 interface has 2 addresses assigned: 11.1.1.1/24 (with default
router 11.1.1.2 on the 11.1.1.0/24 subnet) and the address 'taddr2' whose
numeric value and netmask will be resolved when the zone boots. Thus,
if, at the time of zone boot, the global-zone is also configured as:
r...@gz# egrep 'hosts|netmask' /etc/nsswitch.conf
hosts: files nis dns
netmasks: files nis
r...@gz# grep taddr2 /etc/hosts
12.2.3.4 taddr2
r...@gz# grep 12 /etc/netmasks
12.2.0.0 255.255.0.0
tz1 will be booted as
r...@gz# zoneadm -z tz1 boot
r...@gz# zlogin tz1 ipadm show-addr
ADDROBJ TYPE STATE ADDR
vnic0/_a from-gz ok 11.1.1.1/24
vnic0/_b from-gz ok 12.2.3.4/16
lo0/v4 static ok 127.0.0.1/8
lo0/v6 static ok ::1/128
r...@gz# zlogin tz1 ipadm show-if
IFNAME STATE CURRENT PERSISTENT
vnic0 ok bm-------Z4- -4-
lo0 ok -m-v------46 ---
r...@gz# zlogin tz1 netstat -rn
Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ---------- ---------
default 11.1.1.2 UGZ 2 2 vnic0
11.1.1.0 11.1.1.1 U 3 2 vnic0
12.2.0.0 12.2.3.4 U 2 0 vnic0
127.0.0.1 127.0.0.1 UH 2 0 lo0
Routing Table: IPv6
Destination/Mask Gateway Flags Ref Use If
--------------------------- ---------------------- ----- --- ------- -----
::1 ::1 UH 2 0 lo0
ro...@gz# zlogin tz1 route get default
route to: default
destination: default
mask: default
gateway: 11.1.1.2
interface: vnic0
flags: <UP,GATEWAY,DONE,ZONE>
recvpipe sendpipe ssthresh rtt,ms rttvar,ms hopcount mtu
expire
Proposed man page changes:
--------------------------
--- zonecfg.1m.orig Thu Apr 22 14:41:33 2010
+++ zonecfg.1m.new Fri Jun 11 18:54:58 2010
@@ -625,7 +625,7 @@
The directory path.
- net: address, physical, defrouter
+ net: address, allowed-address, physical, defrouter
The network address and physical interface name of the
network interface. The network address is one of:
@@ -673,13 +673,33 @@
to booting the non-global zone. However, if the inter-
face is not used by the global zone, it should be con-
figured down in the global zone, and the default router
- for the interface should be specified here.
+ for the interface should be specified here. The
+ 'allowed-address' property cannot be set for a shared-IP
+ zone
For an exclusive-IP zone, the physical property must be
- set and the address and default router properties cannot
- be set.
+ set and the 'address' property may not be set. Optionally,
+ the set of IP addresses that the exclusive-IP zone can use
+ may be constrained by specifying the 'allowed-address'
+ property. If 'allowed-address' has not been specified, then the
+ exclusive-IP zone may use any IP address on the associated 'physical'
+ interface for the net resource. Otherwise, when 'allowed-address'
+ is specified, the exclusive-IP zone cannot use IP addresses that
+ are not in any of the 'allowed-address' list for the 'physical'.
+ In addition, when the 'allowed-address' has been specified,
+ the 'defrouter' property may also be optionally specified. The
+ interface specifed for the physical property must not be in
+ use in the global zone. If an allowed-address and default
+ router are specified via zonecfg(1m), these will be applied to
+ the interface when it is enabled via ipadm(1m) in the
+ non-global exclusive-IP zone, typically during zone boot.
+ The non-global exclusive-IP zone will not be able to apply any other
+ addresses to that interface nor transmit packets with a different
+ source address for the specified IP Version. A default router
+ set up via zonecfg(1m) may not be persistently deleted from
+ within the non-global exclusive-IP zone using the -p flag with
+ route(1m).
-
device: match
Device name to match.
@@ -1817,7 +1837,7 @@
svcadm(1M), sysidtool(1M), zfs(1M), zoneadm(1M),
priv_str_to_set(3C), kstat(3KSTAT), vfstab(4), attri-
butes(5), brands(5), fnmatch(5), lx(5), privileges(5),
- resource_controls(5), zones(5)
+ resource_controls(5), zones(5), route(1m)
System Administration Guide: Solaris Containers-Resource
--- dladm.1m.orig Fri May 7 10:10:28 2010
+++ dladm.1m.new Fri Jun 11 08:20:35 2010
@@ -3746,17 +3746,25 @@
IP address anti-spoof. This protection type works in
conjunction with the link property allowed-ips.
- allowed-ips is a list containing IPv4 addresses.
+ allowed-ips is a list containing IP (IPv4 or IPv6) addresses.
This list is empty by default. The unspecified
address (all-zeros) is implicitly in this list.
- An outbound IPv4 packet may pass if its source
- address is in allowed-ips.
+ An outbound IP packet may pass if its source
+ address is in allowed-ips
An outbound ARP packet may pass if its sender proto-
col address is in allowed-ips.
+ When a datalink has been protected by setting allowed-ips
+ to a set of one or more IP addresses, any attempts to
+ configure IP addresses that are not in this set will
+ fail with an EPERM error being returned to the user.
+ Moreover, the interface may not be used for forwarding
+ IP packets, and attempts to set the ipadm(1m) forwarding
+ property on the interface will encounter EPERM error.
+
restricted
This protection restricts outgoing packet types to
@@ -4572,7 +4580,7 @@
SEE ALSO
acctadm(1M), autopush(1M), ifconfig(1M), ipsecconf(1M),
ndd(1M), vrrpadm(1M), psrset(1M), wpad(1M), zonecfg(1M),
- attributes(5), ieee802.3(5), dlpi(7P)
+ attributes(5), ieee802.3(5), dlpi(7P), ipadm(1m)
--- netstat.orig Wed May 19 17:02:56 2010
+++ netstat.new Fri Jun 11 06:21:08 2010
@@ -740,6 +740,9 @@
I Indirect routes (gateway not directly reachable) esta-
blished with the -indirect option.
+ Z (non-global exclusive-IP zone only) The route was statically added
+ on boot based on routing information configured using
+ zonecfg(1m) in the global zone.
If the -a option is specified, there will be routing entries
@@ -881,7 +884,7 @@
iostat(1M), kstat(1M), mibiisa(1M), savecore(1M),
vmstat(1M), hosts(4), inet_type(4), networks(4), proto-
cols(4), services(4), attributes(5), dhcp(5), kstat(7D),
- inet(7P), inet6(7P)
+ inet(7P), inet6(7P), zonecfg(1m)
Droms, R., RFC 2131, Dynamic Host Configuration Protocol,
--- ipadm.1m.orig Tue May 25 09:12:35 2010
+++ ipadm.1m.new Fri Jun 11 08:30:45 2010
@@ -321,10 +321,11 @@
a VRRP interface is in accept mode
(~IFF_NOACCEPT)
+ Z Layer-3 protection of IP addresses for the
+ interface has been administratively enforced.
-
SunOS 5.11 Last change: 29 Mar 2010 5
@@ -891,9 +892,14 @@
TYPE
Type of the address object. It will be one of:
- static, dhcp, or addrconf. It corresponds to the
- type of the address object specified by the -T
- option of create-addr.
+ from-gz, static, dhcp, or addrconf. The static,
+ dhcp and addrconf types correspond to the type of the
+ address object specified by the -T option of
+ create-addr. The "from-gz" type will only be displayed
+ in non-global zones, and indicates that the address
+ was configured based on the 'allowed-address' property
+ configured for the non-global exclusive-IP zone from
+ the global zone.
STATE
@@ -2315,7 +2321,21 @@
address objects configured on that interface are enabled
also.
+ The following command creates persistent configuration for the
+ net0 interface in a non-global exclusive-IP zone
+ so that the net0 interface will be configured with the set of
+ addresses made available through the 'allowed-address' resource
+ from the global zone on the next reboot.
+ # ipadm create-if net0
+
+ The net0 interface may also be configured with the available set
+ of 'allowed-address' values in the non-global exclusive-IP zone
+ without a reboot by executing the following commands:
+
+ # ipadm disable-if -t net0
+ # ipadm enable-if -t net0
+
ATTRIBUTES
See attributes(5) for descriptions of the following attri-
butes:
--- ifconfig.1m.orig Tue May 25 11:05:48 2010
+++ ifconfig.1m.new Tue May 25 11:07:31 2010
@@ -1599,7 +1599,13 @@
used on multiple interfaces. (See also the usesrc
option.)
+ L3PROTECT
+ Indicates that Layer-3 protection has been enforced on
+ the physical interface using the 'allowed-ips' link
+ property in dladm(1m)
+
+
LOGICAL INTERFACES
Solaris TCP/IP allows multiple logical interfaces to be
associated with a physical network interface. This allows a
References:
----------
[PSARC/2002/174] Virtualization and Namespace Isolation in Solaris, aka "Zones"
[PSARC/2003/621] Zone Administration Updates
[PSARC/2008/057] Default Route For Zones
[PSARC/2009/436] Anti-spoofing Link Protection
[PSARC/2010/111] exclusive IP for s10c
[INSTALL]
http://solaris-networking.sfbay/index.php/Network_Configuration:install
[ZONES-NET] http://solaris-networking.sfbay/index.php/Zones_Networking
_______________________________________________
opensolaris-arc mailing list
[email protected]
