On Mon, 26 Nov 2007, Rajagopal Kunhappan wrote:
>
>
> venugopal iyer wrote:
>> Following is a short description on assigning priorities to a link (via
>> dladm). Flows will have a similar mechanism (via flowadm).
>>
>> thanks,
>>
>> -venu
>>
>>
>> ----------------------------------------------------------------------------
>>
>> The components associated with a link or flow, such as Soft Ring Sets,
>> Soft Rings and squeues, have kernel threads processing inbound/outbound
>> packets. The scheduler processes threads based on a thread's priority
>> level.
>>
>> The priority level of a thread determine how it is scheduled w.r.t other
>> kernel threads. Thus by assigning a priority to a link/flow, we can control
>> how the threads associated with that link/flow is processed w.r.t other
>> link/flows. When the system is busy, threads with higher priority will have
>> a greater slice of CPU time, than the ones with lower priority.
>>
>> The kernel provides priority range between MINCLSYSPRI (60) and MAXCLSYSPRI
>> (100) for threads in the system class.
>>
>> We define three levels of priority - low, medium and high within the range
>> MINCLSYSPRI and MAXCLSYSPRI. The priority level can be assigned when
>> creating
>> a virtual link (say VNIC) or can be modified for an existing link using
>> dladm(1m). The priority level is a property of the link.
>>
>> Setting a priority essentially means setting priorities for the following
>> elements associated with a MAC client[1], specifically:
>>
>> Soft Ring Set(s), SRS, (Rx and Tx):
>> (srs_pri)
>> o Worker thread
>> o Poll Thread (Rx)
>> o Associated Soft Ring(s)
>> -> Worker Thread
>> -> Poll Thread (Rx)
>> -> squeue
>> (sq_priority)
>> worker thread
>> poll thread (Rx)
>>
>>
>> Setting a Default Priority
>> --------------------------
>> There are two ways a MAC client can be opened, either implicitly when
>> plumbing
>> a physical link, e.g. bge0 (by DLS) or explicitly when creating a virtual
>> link, e.g VNIC. In the case of the latter, it is quite possible that the
>> properties of the link, including priority, is present when the MAC client
>> is created.
>>
>> When a MAC client is created, we always set a default priority. The default
>> priority is set to be medium. Subsequently, the priority can be modified,
>> if specified (e.g. in the case of a VNIC), as part of creating the virtual
>> link itself.
>>
>> We add the following members to mac_client_impl_t
>>
>> pri_t mci_pri;
>> pri_t mci_max_pri;
>> pri_t mci_min_pri;
>>
>> where mci_pri is the priority assigned with the MAC client. mci_min_pri and
>> mci_max_pri is the available range corresponding to the priority for
>> the MAC client. The range will be used for any flows that are created on
>> top of the MAC client.
>>
>> Given a range min and max, we assign low, medium and high value as follows:
>>
>> priority = ((min) + ((((max) - (min)) / MAC_PRI_LEVELS) * ((pri)
>> + 1)))
>>
>> where MAC_PRI_LEVELS is 3 and pri is 0 (low), 1 (medium) or 2 (high).
>> Additionally, we obtain the range for the given priority as:
>>
>> min_pri = ((min) + ((((max) - (min)) / MAC_PRI_LEVELS) * (pri)))
>>
>> max_pri = ((min_pri) + (((max) - (min)) / MAC_PRI_LEVELS))
>>
>> for e.g.: when we create a MAC client given the range of MINCLSYSPRI and
>> MAXCLSYSPRI, we set a default priority of medium, which translates to the
>> following for the MAC client:
>>
>> mci_pri = 86
>> mci_min_pri = 73
>> mci_max_pri = 86
>>
>> All the threads associated with the MAC client elements (listed above) will
>> have a thread priority of 86. Any flow created on the MAC client will now
>> derive its priority value using the specified priority (low, medium and
>> high)
>> for that flow within the range 73 and 86.
>>
> Since there are only three priority levels, can you list the thread
> priorities (min/max) for the low and high cases also (instead of me
> calculating them ;-)).
Within the range MINCLSYSPRI and MAXCLSYSPRI:
low would be:
mci_pri = 73
mci_min_pri = 60
mci_max_pri = 73
high would be:
mci_pri = 99
mci_min_pri = 86
mci_max_pri = 99
-venu
>
> Thanks,
> -krgopi
>> If we create a virtual link, say a VNIC, with a specific priority, say
>> high,
>> mac_client_open() will initialize the values for the VNIC's MAC client with
>> the default priority. Subsequently, mac_resource_ctl_set() will result in
>> modifying the priority values for the MAC client to the value corresponding
>> to high.
>>
>> Modifying Priority
>> ------------------
>>
>> A set-linkprop after the MAC client has been opened can be used to modify
>> the priority of a link. The result is a new priority value and range for
>> the MAC client based on the new priority. Additionally, any flows on top
>> of the MAC client will also need to be reprioritized as the range for the
>> MAC client will change.
>>
>> Setting priority before opening a MAC client
>> ---------------------------------------------
>>
>> A set-linkprop before a MAC client has been created will require the
>> priority
>> values (in fact, any property) to be cached in the mac_impl_t. When the
>> primary MAC client is opened, the associated properties can be set on the
>> newly created MAC client post mac_client_open().
>>
>> _______________________________________________________________________________
>> References:
>>
>> [1]http://opensolaris.org/os/project/crossbow/Docs/crossbow-virt.pdf
>>
>> _______________________________________________
>> crossbow-discuss mailing list
>> crossbow-discuss at opensolaris.org
>> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss
>>
