On 10/21/08 19:12, Li, Aubrey wrote:
> Bill Holler wrote:
>
>
>> Another group is working on system policy. They are thinking
>> of settings such as max-power-savings, max-performance,
>> elastic, etc. One way to implement these system policies may
>> be to ignore different processor groups similar to this.
>>
>> Do we want policy to be at the thread level or at the system level?
>>
>>
>
> AFAIK, the policy only takes effect in cmt_balance().
> There is an event driver in Eric's policy patch. My thoughts is, this event
> driver can exactly be used to employ both coalescence and load balance
> policy across the processor group. When performance is needed, we dispatch
> the next thread by max-performance policy, and when power is needed, we
> dispatch the next thread by max-power-savings policy. IMHO, this needs kthead
> to be policy sensitive.
>
> Are you suggesting we use one policy at the same time, and throw the setting
> out
> to the userland?
>
That sounds like the current direction, but I am not positive.
Eric has more contact with the group working on system-level
policy, but I did not see him respond to your email today.
Eric also mentioned the possibility of changing the per-cpu
load balancing lineages to reflect different system policies.
Regards,
Bill
> Regards,
> -Aubrey
>
>
>> On 10/20/08 23:43, Li, Aubrey wrote:
>>
>>> Eric.Saxe wrote:
>>>
>>>
>>>
>>>> Because it works top down, in this case, the coalescence policy will
>>>> be implemented until there's nothing more to do at that level,
>>>> before moving lower. This is the first time we've allowed different
>>>> policies to be implemented across the hierarchy, so i'm really
>>>> happy you are taking a look at this. I haven't looked at it closely
>>>> yet since I've still been focused on getting the nuts and bolts of
>>>> the dispatcher driven p-state changes to happen... :)
>>>>
>>>>
>>> Currently we have two kinds of cmt policy across the hierarchical
>>> processor group. So it's possible a thread want to be dispatched by
>>> the performance policy but there is only power related processor
>>> group avaiable.
>>>
>>> The following patch adds a member into kthread_t structure, and use
>>> the same event driver which is used to notify cpupm to change
>>> p-state, to make the next kernel thread sensitive to the cmt policy,
>>> so that in later cmt balance, we can choose cpu by searching the
>>> related target processor group, unrelated groups will be ignored.
>>>
>>> What do you think?
>>>
>>> Thanks,
>>> -Aubrey
>>>
>>> diff -r 7bc9bb1331de usr/src/uts/common/disp/cmt.c
>>> --- a/usr/src/uts/common/disp/cmt.c Tue Oct 21 10:17:20 2008
>>> +0800 +++ b/usr/src/uts/common/disp/cmt.c Tue Oct 21 14:24:01
>>> 2008 +0800 @@ -951,6 +951,7 @@ {
>>> pg_cmt_t *cmt = (pg_cmt_t *)pg;
>>> cpupm_domain_t *dom;
>>> + pghw_type_t pghw_hw = ((pghw_t *)pg)->pghw_hw;
>>> uint32_t u;
>>>
>>> if (old == cp->cpu_idle_thread) {
>>> @@ -960,7 +961,6 @@
>>> * significant increase in utiliation unless the
>>> thread makes
>>> * it through one time slice expiration.
>>> */ - ASSERT((cp->cpu_disp_flags & CPU_DISP_CMT_TRANS)
>>> == 0); cp->cpu_disp_flags |= CPU_DISP_CMT_TRANS;
>>>
>>> } else if (new == cp->cpu_idle_thread) {
>>> @@ -984,8 +984,16 @@
>>> * zero, so we should notify the CPUPM
>>> subsystem. */ if (u
>>> == 1) {
>>> - dom = (cpupm_domain_t
>>> *)cmt->cmt_pg.pghw_handle;
>>> - cpupm_utilization_change(cp, dom, 0);
>>> + if (pghw_hw == PGHW_POW_ACTIVE) {
>>> + dom = (cpupm_domain_t *)
>>> + cmt->cmt_pg.pghw_handle;
>>> +
>>>
>> cpupm_utilization_change(cp, dom, 0);
>>
>>> + }
>>> + /*
>>> + * next thread will be dispatched by
>>> power + * policy
>>> + */
>>> + new->t_cmt_policy = CMT_COALESCE;
>>> } }
>>> } else {
>>> @@ -1004,6 +1012,7 @@
>>> {
>>> pg_cmt_t *cmt = (pg_cmt_t *)pg;
>>> cpupm_domain_t *dom;
>>> + pghw_type_t pghw_hw = ((pghw_t *)pg)->pghw_hw;
>>> uint32_t u;
>>>
>>> /*
>>> @@ -1029,8 +1038,15 @@
>>> * the CPUPM subsystem.
>>> */
>>> if (u == 0) {
>>> - dom = (cpupm_domain_t
>>> *)cmt->cmt_pg.pghw_handle;
>>> - cpupm_utilization_change(cp, dom, 1);
>>> + if (pghw_hw == PGHW_POW_ACTIVE) {
>>> + dom = (cpupm_domain_t
>>> *)cmt->cmt_pg.pghw_handle; +
>>> cpupm_utilization_change(cp, dom, 1); + } +
>>> /* + * next thread will be dispatched by the
>>> performance + * policy
>>> + */
>>> + t->t_cmt_policy = CMT_BALANCE;
>>> } }
>>> }
>>> diff -r 7bc9bb1331de usr/src/uts/common/disp/cmt_policy.c
>>> --- a/usr/src/uts/common/disp/cmt_policy.c Tue Oct 21 10:17:20
>>> 2008 +0800 +++ b/usr/src/uts/common/disp/cmt_policy.c Tue Oct
>>> 21 14:24:01 2008 +0800 @@ -140,6 +140,26 @@ do {
>>> pg = GROUP_ACCESS(cmt_pgs, level);
>>>
>>> + /*
>>> + * if next thread is dispatched by the performance
>>> + * policy, power related group should be ignored +
>>> */ + if (tp->t_cmt_policy == CMT_BALANCE) {
>>> + if (pg->cmt_policy & CMT_COALESCE) {
>>> + continue;
>>> + }
>>> + /*
>>> + * if next thread is dispatched by the power
>>> + * policy, loading balance related group should +
>>> * be ignored + */
>>> +
>>> + } else if (tp->t_cmt_policy == CMT_COALESCE) {
>>> + if (pg->cmt_policy & CMT_BALANCE) {
>>> + continue;
>>> + }
>>> + }
>>> +
>>> siblings = pg->cmt_siblings;
>>> nsiblings = GROUP_SIZE(siblings); /* self
>>> inclusive */ if (nsiblings == 1)
>>> diff -r 7bc9bb1331de usr/src/uts/common/sys/thread.h
>>> --- a/usr/src/uts/common/sys/thread.h Tue Oct 21 10:17:20 2008
>>> +0800 +++ b/usr/src/uts/common/sys/thread.h Tue Oct 21 14:24:01
>>> 2008 +0800 @@ -342,6 +342,7 @@ hrtime_t t_hrtime;
>>> /* high-res last time on cpu */ kmutex_t t_ctx_lock;
>>> /* protects t_ctx in removectx() */ struct waitq
>>> *t_waitq; /* wait queue */ + uint_t
>>> t_cmt_policy; /* cmt policy */ } kthread_t;
>>>
>>> /*
>>> _______________________________________________
>>> tesla-dev mailing list
>>> tesla-dev at opensolaris.org
>>> http://mail.opensolaris.org/mailman/listinfo/tesla-dev
>>>
>>>
>> _______________________________________________
>> tesla-dev mailing list
>> tesla-dev at opensolaris.org
>> http://mail.opensolaris.org/mailman/listinfo/tesla-dev
>>
>
>
