Guys,
As promised earlier, the two manpages for Crossbow are attached.
dladm (1M) allows creation/modification/deletion of Virtual NICs
and assigning bandwidth resources to it including binding
them to processors.
netrcm (1M) allows assigning bandwidth and CPU resources to
flows without creating the VNICs as administrative entity (primarily
used for service consolidation and tradition QOS based usage).
This is the first draft of the man pages that we are targetting for
next release of Crossbow bits. Comments/feedback welcome.
Cheers,
Sunay
--
Sunay Tripathi
Sr. Staff Engineer
Solaris Core Networking Technologies
Sun MicroSystems Inc.
Solaris Networking: http://www.opensolaris.org/os/community/networking
Project Crossbow: http://www.opensolaris.org/os/project/crossbow
# ident "@(#)dladm.1m.txt 1.3 06/10/23 SMI"
System Administration Commands dladm(1M)
NAME
dladm - configure data-link interfaces
SYNOPSIS
dladm show-link [-s [-i interval]] [-p] [name]
! dladm show-dev [-s [-i interval]] [-m] [-p] [dev]
dladm create-aggr [-t] [-R root-dir] [-P policy] [-l mode]
[-T time] [-u address] -d dev [-d dev] ... key
dladm delete-aggr [-t] [-R root-dir] key
dladm add-aggr [-t] [-R root-dir] -d dev [-d dev] ... key
dladm remove-aggr [-t] [-R root-dir] -d dev [-d dev] ... key
dladm modify-aggr [-t] [-R root-dir] [-P policy] [-l mode]
[-T time] [-u address] key
dladm show-aggr [-L] [-s [-i interval]] [-p] [key]
+ dladm create-vnic [-t] [-R root-dir] -d dev
+ [-m {[factory|shared -n slot-identifier] | [random|value]}]
+ [[-b bandwidth] {-L | -G}] [-H] [-p pri]
+ [-c processor_id=<value>] vnic-id
+
+ dladm delete-vnic [-t] [-R root-dir] vnic-id
+
+ dladm show-vnic [-s [-i interval]] [-p] [vnic-id]
+
+ dladm modify-dev [-t] [-R root-dir] [[-b bandwidth] {-L | -G}]
+ [-H] [-p pri] [-r] [-c processor_id=<value>] [-d] [dev]
dladm -?
DESCRIPTION
The dladm command is used to configure data-links. A config-
ured data-link is represented in the system as a STREAMS
DLPI (v2) interface which may be plumbed under protocol
stacks such as TCP/IP. Each data-link relies on either a
single network device or an aggregation of devices to send
packets to or receive packets from a network.
SunOS 5.11 Last change: 22 May 2006 1
System Administration Commands dladm(1M)
The dladm command operates on the following kinds of object:
link Data-links, identified by a name
aggr Aggregations of network devices, identified
by a key
+ vnic Virtual interfaces to a network, identified
+ by a vnic-id. The VNIC is a virtual-link created
+ on a NIC by programming the NIC hardware
+ classifier. Its a pseudo device node in the
+ Solaris device tree which can be treated like a
+ real physical device. Only the packets meant
+ for the VNIC selected by the VNICs unique MAC
+ address or IP address are delivered to it. All
+ the kernel resources including layer 2, 3, & 4
+ queues, processing threads, and data structures
+ are unique per VNIC such that traffic for two
+ different VNIC (on same physical NIC or separate
+ physical NIC) do not interfere with each other.
+ The bandwidth limit or guarantees and packet
+ processing priority for VNICs can be specified
+ independent of each other and are typically
+ enforced with near zero performance impact.
dev Network devices, identified by concatenation
of a driver name and an instance number
Some devices do not support configurable data-links or
aggregations. The fixed data-links provided by such devices
can be viewed using dladm, but can not be configured.
SUBCOMMANDS
The following subcommands are supported:
show-link Show configuration information for all
data-links or the specified data-link. By
default, the system is configured to have
one data-link for each known network device.
show-dev Shows information for all devices or the
! specified device. The m option shows
+ the factory configured mac addresses and their
+ slot identifier.
create-aggr Creates an aggregation using the given key
value from as many dev objects as are speci-
fied. A data-link is created by default, and
is given a name which is the concatenation
of "aggr" and the key value of the aggrega-
tion.
SunOS 5.11 Last change: 22 May 2006 2
System Administration Commands dladm(1M)
delete-aggr Deletes the specified aggregation.
add-aggr Adds as many dev objects as are specified to
the given aggregation.
remove-aggr Removes as many dev objects as are specified
from the given aggregation.
modify-aggr Modifies the parameters of the given aggre-
gation.
show-aggr Shows configuration information for all
aggregations or the specified aggregation.
+ create-vnic Creates a VNIC with the given vnic-id over the
+ specified device, dev. A data-link is also
+ created and given a name which is the con-
+ catenation of "vnic" and the given vnic-id.
+ delete-vnic Deletes the specified VNIC and its
+ corresponding data-link.
SunOS 5.11 Last change: 22 May 2006 3
System Administration Commands dladm(1M)
+ show-vnic Shows configuration information for all
+ VNICs or the specified VNIC. the s option
+ can show the stats and when coupled with i
+ option, it can show the stats delta for the
+ time interval specified.
+ modify-dev modifies the bandwidth and priority related
+ parameters of a given device which can be
+ a VNIC as well.
OPTIONS
The following options are supported:
SunOS 5.11 Last change: 22 May 2006 4
System Administration Commands dladm(1M)
key
The key of an aggregation or VNIC. This must be an
integer value between 1 and 999.
-d dev
--dev=dev
A device specifier. This must be a concatenation of the
name and instance of the driver bound to the device.
-P policy
--policy=policy
Specifies the port selection policy to use for load
spreading of outbound traffic. The policy specifies
which dev object is used to send packets. A policy con-
sists of a list of one or more layers specifiers
separated by commas. A layer specifier is one of the
following:
L2 Select outbound device according to source and
destination MAC addresses of the packet.
L3 Select outbound device according to source and
destination IP addresses of the packet.
L4 Select outbound device according to the upper
layer protocol information contained in the
packet. For TCP and UDP, this includes source
and destination ports. For IPsec, this includes
the SPI (Security Parameters Index.)
For example, to use upper layer protocol information,
the following policy can be used:
-P L4
To use the source and destination MAC addresses as well
SunOS 5.11 Last change: 22 May 2006 5
System Administration Commands dladm(1M)
as the source and destination IP addresses, the follow-
ing policy can be used:
-P L2,L3
-l mode
--lacp-mode=mode
Specifies whether LACP should be used and, if used, the
mode in which it should operate. Legal values are off,
active or passive.
-T time
--lacp-timer=time
Specifies the LACP timer value. The legal values are
short or long.
-u address
--unicast=address
Specifies a fixed unicast address to be used for the
aggregation. If this option is not specified then an
address is automatically chosen from the set of
addresses of the component devices.
-L
--lacp
Specifies whether detailed LACP information should be
displayed.
-s
--statistics
Used with the show-link, show-aggr, show-vnic, or show-
dev subcommands to show the statistics of data-links,
aggregations, VNICs, or devices, respectively.
SunOS 5.11 Last change: 22 May 2006 6
System Administration Commands dladm(1M)
-i interval
--interval=interval
Used with the -s option to specify an interval, in
seconds, at which statistics should be displayed. If
this option is not specified, statistics will only be
displayed once.
-t
--temporary
Specifies that the change is temporary. Temporary
changes last until the next reboot.
-R root-dir
--root-dir=root-dir
Specifies an alternate root directory where dladm
applies changes. This can be useful in JumpStart
scripts, where the root directory of the system being
modified is mounted elsewhere.
-p
--parseable
Specifies that configuration information should be
displayed in parseable format.
-?
--help
Displays help information. (Stops interpretation of sub-
sequent arguments).
+ VNIC PARAMETERS
+ The following options are available with the add-vnic,
+ modify-dev, commands to configure a VNIC and add/modify bandwidth
+ resource control to both VNICs and regular devices.
+ -b bandwidth
+ --bandwidth
+ Sets the full duplex bandwidth for the VNIC. The bandwidth
+ is specified as an integer with one of the scale suffixes
+ k, kbps, M, Mbps, G, Gbps, %, or percent. In addition, the
+ "L" or "G" option below specifies whether the bandwidth
+ specified is a limit or a guarantee. Different VNICs
+ on same device can have a limit or a guarantee but the sum
+ total of all limits and guarantee can not exceed the total
+ configured and available capacity of the physical device.
+ -L
+ --limit
+ Sets the full duplex bandwidth limit for the VNIC. The
+ VNIC bandwidth usage will not be allowed to exceed the
+ bandwidth specified.
+ -G
+ --guarantee
+ Sets the bandwidth guarantee in the VNIC. If there is
+ available bandwidth, the VNIC is allowed to exceed
+ its guaranteed entitlement.
+ -p value
+ --priority value
+ Sets the relative priority for the VNIC. The value may be
+ given as an integer from 0 to 100 or as one
+ of the tokens high, normal, or low, optionally followed
+ by a positive or negative offset, as in priority nor-
+ mal+2.
+ -H
+ --hw
+ Sets the hardware resource preference in the VNIC.
+ VNICs created with this option are preferentially assigned
+ to a hardware classifier if one is available.
+ -r
+ --remove
+ Removes any previously assigned bandwidth limits or guarantees
+ -c
+ --cpu_binding
+ processor_id=<value>
+ Bind the processing of packets for a given device to a processor
+ or a set of processors. The value can be a comma separated list
+ of one or more processor id. If the list consists of more than one
+ processor, the processing will spread out to all the processors
+ although connection to processor affinity and packet ordering for
+ packets of any individual connection will be maintained.
+
+ The processor or set of processors are not exclusively reserved
+ for the network device. Only the kernel threads and interrupt
+ associated with processing of the device are bound to the processor
+ or the set of processor specified. In case, it is desired that
+ processors be dedicated to the device, psrset(1M) can be used
+ to create a processor_set and then specifiying the processors from
+ the processor_set to bind the device to.
+
+ If the device was already bound to processor or set of processors
+ due to a previous operation, the binding will be removed and the
+ new set of processors will be used instead.
+ -d
+ --delete_binding
+ Unbind the device from any previous binding.
SunOS 5.11 Last change: 22 May 2006 8
System Administration Commands dladm(1M)
+ -m keyword value
+ --mac keyword value
+ Sets the VNICs MAC address based to specified keyword and/or
+ value. For create-vnic, the following special keywords may
+ be used:
+ factory
+ Use an available factory-assigned global MAC
+ address. The factory keyword may be followed
+ slot-ident which is the slot-identifier for the
+ factory address that should be used. If no
+ slot-identifier is specified, system will
+ choose the next available identifier. show-dev
+ subcommand with option m can be used to list
+ all the factory configured MAC addresses and
+ their identifiers
+
+ shared
+ Share an existing factory assigned MAC address
+ to build VNICs. A shared MAC address can
+ only have VNICs on top of it. If a data link
+ STREAM has the physical NIC open directly and
+ is using a particular MAC address, that MAC
+ address can not be shared. The data link STREAM
+ needs to be closed, and all consumers should create
+ VNICs using the "shared" option for the MAC
+ address that will be shared by the VNCIs. The
+ show-dev subcommand with m option can be used to
+ list all the factory assigned MAC addresses and
+ their slot-identifier.
+
+ random
+ Use a randomly generated local MAC address.
+
+ value
+ A user specified mac address can be programmed
+ to the NIC if the NIC is capable of filtering on
+ more than one MAC addresses. If the NICs
+ hardware filter table for mac address is
+ exhausted or the NIC is not capable of supporting
+ more than one MAC address in its hardware filter
+ table, then the hardware is put in promiscuous mode
+ and software filtering is used.
+ If no "m" flag is specified for create-vnic subcommand, the
+ default will be to try and use a factory assigned address
+ first and if none is available, randomly assign a MAC
+ address.
EXAMPLES
Example 1: Configuring an Aggregation
To configure an aggregate data-link with vinc-id 1 over the dev-
ices bge0 and bge1, enter the following command:
# dladm create-aggr -d bge0 -d bge1 1
+ Example 2: Configuring VNICs
+ To create two VNICs interfaces with vinc-ids 1 and 2
+ over a single physical device bge0, enter the following com-
+ mands:
+ # dladm create-vnic -d bge0 1
+ # dladm create-vnic -d bge0 2
+ The new links will be called vnic1 and vnic2.
+ Example 3: Configuring VNICs and allocating bandwidth & priority
+ To create two VNIC interfaces with vinc-ids 1 and 2
+ over a single physical device bge0 and make vnic1 a higher
+ priority VNIC using factory assigned MAC address with guarantee
+ to use upto 90% of the bandwidth and vnic2 having a lower priority
+ with a random MAC address and a hard limit of 100Mbps:
+ # dladm create-vnic -d bge0 -m factory -b 90% -G -p high 1
+ # dladm create-vnic -d bge0 -m random -b 100M -L -p low 2
+ Example 4: Configure a VNIC by choosing a factory MAC address
+ To create a VNIC interface with vinc-id 1 by first
+ listing the factory available MAC address and then using one
+ of them:
+ # dladm show-dev -d bge0 -m
+ bge0
+ link: up speed: 1000 Mbps duplex: full
+ MAC addresses:
+ slot-ident Address In Use
+ 1 0:e0:81:27:d4:47 Yes
+ 2 8:0:20:fe:4e:a5 No
+ # dladm create-vnic -d bge0 -m factory -n 2 1
+ # dladm show-dev -d bge0
+ bge0
+ link: up speed: 1000 Mbps duplex: full
+ MAC addresses:
+ slot-ident Address In Use
+ 1 0:e0:81:27:d4:47 Yes
+ 2 8:0:20:fe:4e:a5 Yes
+ Example 5: Configuring VNICs sharing a MAC address
+ To create two VNICs with vnic-id 1 and 2 by first listing the
+ available factory assigned MAC addresses and then picking one
+ that will be shared by the newly created VNICs
+ # dladm show-dev -d bge0 -m
+ bge0
+ link: up speed: 1000 Mbps duplex: full
+ MAC addresses:
+ slot-ident Address In Use
+ 1 0:e0:81:27:d4:47 Yes
+ 2 8:0:20:fe:4e:a5 No
+ # dladm create-vnic -d bge0 -m shared -n 2 1
+ # dladm create-vnic -d bge0 -m shared -n 2 2
+ Example 6: Creating a VNIC with user specified MAC address and
binding it to a set of processors
+ To create a VNIC with vnic-id 1 by providing a user specified
+ mac address and binding the processing to processor 0,1,2 and 3.
+ # dladm create-vnic -d bge0 -m 8:0:20:fe:4e:b8 -c processor_id=0,1,2,3 1
ATTRIBUTES
See attributes(5) for descriptions of the following attri-
butes:
/usr/sbin
____________________________________________________________
|_____________________________|_____________________________|
| Availability | SUNWcsu |
|_____________________________|_____________________________|
|_____________________________|_____________________________|
/sbin
____________________________________________________________
|_____________________________|_____________________________|
| Availability | SUNWcsr |
|_____________________________|_____________________________|
|_____________________________|_____________________________|
SEE ALSO
ifconfig(1M), attributes(5), dlpi(7P)
NOTES
The configuration of all objects will persist across reboot.
SunOS 5.11 Last change: 22 May 2006 10
#pragma ident "@(#)netrcm.1m.txt 1.5 06/10/23 SMI"
System Administration Commands netrcm(1M)
NAME
netrcm - administer bandwidth resource control and priority
for protocols, services, Containers and Virtual machines
SYNOPSIS
netrcm add-flow [-t] [-R root-dir] -d dev [[-b bandwidth] {-L | -G}]
[-H] [-p pri] -q {flow_attributes} [-c processor_id=<value>] flow-id
netrcm remove-flow [-t] [-R root-dir] -d dev flow-id
netrcm modify-flow [-t] [-R root-dir] -d dev
[[-b bandwidth] {-L | -G}] [-H] [-p pri] [-r]
[-c processor_id=<value>] [-d] flow-id
netrcm show-flow [-s [-i interval]] [-d dev] [flow-id]
netrcm -?
DESCRIPTION
The netrcm command is used to create, modify, remove and show
networking bandwidth and associated resources for a type of traffic on
a particular NIC.
CONCEPTS
The netrcm command allows users to manage networking bandwidth
resource for a transport, service, or a virtual machine. The
service is specified as a combination of transport and port
while a virtual machine is specified by its mac address or an
IP address. The command can also be used on Virtual NICs
which are created for a virtual machine or Solaris Containers
to allow the Virtual machine or Container administrator to
further sub divide the bandwidth allocated to it when the VNIC
was created.
A flow is defined as a set of attributes based on Layer 2, 3 and
4 headers which can be used to identify a protocol, service, or a
virtual machine. When a flow is identified based on flow
attributes, separate kernel resources including layer 2, 3, and 4
queues, their processing threads, etc are uniquely created for it
such that other traffic has minimal or zero impact on it. The
processing of the packets for a given flow can be restricted to
a given CPU or processor set such that all the resources
necessary for processing the packets of the flow are dedicated to
it. The in bound packets for the identified transport, service or
virtual machine are separated by the NIC hardware (when possible)
which utilizes separate FIFOs and DMA channels so that bandwidth
limits or guarantees can be enforced with near zero performance
impact.
The netrcm command can be used to identify a flow without
imposing any bandwidth resource control. This would result in
the traffic type getting its own hardware and software resources
and queues so that its isolated from rest of the networking
traffic for more deterministic behavior.
SunOS 5.11 Last change: 22 May 2006 1
System Administration Commands netrcm(1M)
FLOW ATTRIBUTES
The flow attribute that identify a flow is a comma separated list
of one or more keyword,value pair from below
local_mac_addr=<value>
Identifies a virtual machine by the destination mac address
<value>, which is specified in the x:x:x:x:x:x
representation. The user needs to ensure that the mac address
exclusively belongs to virtual machine for which the policy
is intended.
sap=<value>
Identifies a vnic by the SAP <value>. The <value>
may be specified in decimal, hexadecimal (preceded by
the 0x or 0X prefix) or octal (preceded by the prefix 0).
local_ip_addr=<value>
Identifies a virtual machine by the local IP address.
<value> must be a IPv4 address in dotted-decimal notation.
As in case of a mac address, the user must ensure that
IP address exclusively belongs to the virtual machine for
which the policy is being created.
transport={TCP|UDP|SCTP}
Identifies a layer 4 protocol to be used. Is typically used
in combination with lport to identify the service which
needs special attention.
lport=<value>
Identifies a service specified by the local port.
rport=<value>
Identifies a service identified by the remote port
SUBCOMMANDS
add-flow
Adds a flow to the system identified by its flow attributes.
As part of identifying a particular flow, its bandwidth
resource can be guaranteed or limited and its relative
priority to other traffic can be specified. If no
bandwidth limit, guarantee or priority is specified,
the traffic still gets its unique layer 2, 3, and 4
queues and processing threads including NIC hardware
resources (when supported) so that the selected traffic
can be separated from others and can flow with minimal
impact from other traffic.
delete-flow
netrcm delete-flow destroys an existing flow identified by
its flow-id.
modify-flow
netrcm modify-flow modifies an existing flow identified by
its flow-id.
show-flow
netrcm show-flow shows the existing policies configured
system wide or on a particular data-link. The -s option
shows the statistics including inbound, outbound packets for
the traffic impacted by the policy. Coupled with -i option,
the stats delta can be show per specified time interval.
OPTIONS
-t
The changes are temporary and will not persist across reboots.
POLICY PARAMETERS
The following parameters are supported:
-b bandwidth
--bandwidth
Sets the full duplex bandwidth available. The bandwidth
is specified as an integer with one of the scale suffixes
k, kbps, M, Mbps, G, Gbps, %, or percent. In addition, the
"L" or "G" option below specifies whether the bandwidth
specified is a limit or a guarantee. Different policies
on same device can have a limit or a guarantee but the sum
total of all limits and guarantee can not exceed the total
configured and available capacity of the data-link.
-L
--limit
Sets the full duplex bandwidth limit for. The bandwidth
usage will not be allowed to exceed the bandwidth specified.
-G
--guarantee
Sets the bandwidth guarantee. If there is available bandwidth,
the traffic related to this policy is allowed to exceed
its guaranteed entitlement.
-p value
--priority value
Sets the relative priority for the traffic. The value may be
given as an integer from 0 to 100 or as one
of the tokens high (90), normal (50), or low (10),
optionally followed by a positive or negative offset,
as in priority 'normal+2'.
-H
--hw
Sets the hardware resource preference in the underlying NIC.
Policies created with this option are preferentially assigned
to a hardware classifier if one is available.
-r
--remove
Removes any previously assigned Bandwidth limits or guarantees
to the flow.
-c
--cpu_binding
processor_id=<value>
Bind the processing of packets for a given flow to a processor
or a set of processors. The value can be a comma separated list
of one or more processor ids. If the list consists of more than one
processor, the processing will spread out to all the processors
although connection to processor affinity and packet ordering for
packets of any individual connection will be maintained.
The processor or set of processors are not exclusively reserved
for the flow. Only the kernel threads and interrupt
associated with processing of the flow are bound to the processor
or the set of processor specified. In case, it is desired that
processors be dedicated to the device, psrset(1M) can be used
to create a processor_set and then specifiying the processors from
the processor_set to bind the flow to.
If the flow was already bound to processor or set of processors
due to a previous operation, the binding will be removed and the
new set of processors will be used instead.
-d
--delete_binding
Unbind the device from any previous binding.
SunOS 5.11 Last change: 22 May 2006 2
System Administration Commands netrcm(1M)
EXAMPLES
Example 1: Create a policy around mission critical port 443 traffic
which is https service.
To create a policy around inbound https traffic on a https server
so that https gets it dedicated NIC hardware and kernel TCP/IP
resources. The flow-id specified is https-1 which is used to
later modify of delete the policy.
# netrcm add-flow -d bge0 -H -q transport=TCP,lport=443 https-1
Example 2: Modify an existing policy to add bandwidth resource control
To modify https-1 policy to add bandwidth control, give it a
high priority and bind the processing of the flow to CPU# 9
# netrcm modify-flow -d bge0 -b 90% -G -p high -c processor_id=9 https-1
Example 3: Limit the bandwidth usage of UDP protocol
To create a policy for UDP protocol so that it can not consume more
than 10% of available bandwidth. The flow-id is called limit-udp-1.
# netrcm add-flow -d bge0 -b 10% -L -q transport=UDP -p low limit-udp-1
EXIT STATUS
The following exit values are returned:
0 All actions were performed successfully.
>0 An error occurred.
SunOS 5.11 Last change: 22 May 2006 4
System Administration Commands netrcm(1M)
ATTRIBUTES
See attributes(5) for descriptions of the following attri-
butes:
____________________________________________________________
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
|_____________________________|_____________________________|
|_____________________________|_____________________________|
| Interface Stability | Evolving |
|_____________________________|_____________________________|
SEE ALSO
dladm(1M), attributes(5), dlpi(7P)
NOTES
The configuration of all policies will persist across reboot.
SunOS 5.11 Last change: 22 May 2006 5
_______________________________________________
networking-discuss mailing list
[email protected]
