Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 09/10/17 17:36, Todor Tomov wrote:
> Hi,
> 
> On 30.05.2017 06:41, Rajendra Nayak wrote:
>> []..
>>
> I was proposing to have such a lower-layer by splitting the existing
> genpd framework so the drivers would have the option of calling the
> lower-level power control functions to look-up pm-domains and control
> them directly from their rpm callbacks (if they need to). Same as we do
> for clocks. This way you would not need to mess with the genpd ->start()
> callback and leave it to the driver to handle itself as it knows what
> needs to be done. This assumes that the device is never bound to the
> pm-domain by the genpd core.

 Yes, agree! To me this is the only solution what would really work.
>>>
>>> I agree! :-)
>>>
 Perhaps Rafael can confirm that he is fine with a solution like this?
>>>
>>> Yes and Rafael, please can you also elaborate on what you meant by
>>> "allow genpd to use either a list of power resources or the on/off
>>> callbacks provided by itself to cover different use cases"?
>>>
>>> I would like to understand exactly what you meant by allowing genpd to
>>> use a list of power resources (ie. how you envisioned we could achieve
>>> this).
>>
>> While thinking through the problem of devices associated with multiple Power
>> domains (or power resources) and controlling them individually (or together)
>> I was wondering if something like a PM domain governor (with PM resource 
>> level constraints) could help.
>>
>> So with just one set of PM domain callbacks, its quite easy to control 
>> multiple power
>> resources, if they need to be *all* turned on/off together, using something 
>> similar to
>> what Jon proposed in his RFC [1]
>>
>> However, there could be instances where in we might need to control them 
>> individually
>> and in such cases we could hook up a PM domain governor which decides if an 
>> individual
>> PM resource can be turned on or off while the device is runtime 
>> suspended/resumed.
>> We can expose some PM resource level QoS APIs which the drivers can use to 
>> express their
>> needs, which the PM domain governor then takes into account during the 
>> decision making.
>>
>> if this seems worth pursuing further, I can post some RFCs on these lines and
>> get the discussion going.
>>
>> thanks,
>> Rajendra
>>
>> [1] https://lkml.org/lkml/2016/9/20/173
>>
> 
> I have come to a similar case with multiple power domains on Qualcomm APQ8096 
> - the
> camera subsystem has two VFE modules (Video Front End - these are image 
> processing modules)
> and each of them has a separate power domain but we might want to control 
> these from
> a single driver.
> 
> So I wanted to ask if there have been any news on this topic lately?

There has not but there is still clearly a need for this and we need it
for Tegra. I still plan to get back to this but recently a few items
have cropped up and I have not had chance. So sorry about that.

My plan was to split the current GPD framework so there is a lower-level
set of APIs for managing the power-domains as I discussed with Rafael [0].

Cheers
Jon

[0] https://lkml.org/lkml/2017/5/2/203

-- 
nvpublic

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 09/10/17 17:36, Todor Tomov wrote:
> Hi,
> 
> On 30.05.2017 06:41, Rajendra Nayak wrote:
>> []..
>>
> I was proposing to have such a lower-layer by splitting the existing
> genpd framework so the drivers would have the option of calling the
> lower-level power control functions to look-up pm-domains and control
> them directly from their rpm callbacks (if they need to). Same as we do
> for clocks. This way you would not need to mess with the genpd ->start()
> callback and leave it to the driver to handle itself as it knows what
> needs to be done. This assumes that the device is never bound to the
> pm-domain by the genpd core.

 Yes, agree! To me this is the only solution what would really work.
>>>
>>> I agree! :-)
>>>
 Perhaps Rafael can confirm that he is fine with a solution like this?
>>>
>>> Yes and Rafael, please can you also elaborate on what you meant by
>>> "allow genpd to use either a list of power resources or the on/off
>>> callbacks provided by itself to cover different use cases"?
>>>
>>> I would like to understand exactly what you meant by allowing genpd to
>>> use a list of power resources (ie. how you envisioned we could achieve
>>> this).
>>
>> While thinking through the problem of devices associated with multiple Power
>> domains (or power resources) and controlling them individually (or together)
>> I was wondering if something like a PM domain governor (with PM resource 
>> level constraints) could help.
>>
>> So with just one set of PM domain callbacks, its quite easy to control 
>> multiple power
>> resources, if they need to be *all* turned on/off together, using something 
>> similar to
>> what Jon proposed in his RFC [1]
>>
>> However, there could be instances where in we might need to control them 
>> individually
>> and in such cases we could hook up a PM domain governor which decides if an 
>> individual
>> PM resource can be turned on or off while the device is runtime 
>> suspended/resumed.
>> We can expose some PM resource level QoS APIs which the drivers can use to 
>> express their
>> needs, which the PM domain governor then takes into account during the 
>> decision making.
>>
>> if this seems worth pursuing further, I can post some RFCs on these lines and
>> get the discussion going.
>>
>> thanks,
>> Rajendra
>>
>> [1] https://lkml.org/lkml/2016/9/20/173
>>
> 
> I have come to a similar case with multiple power domains on Qualcomm APQ8096 
> - the
> camera subsystem has two VFE modules (Video Front End - these are image 
> processing modules)
> and each of them has a separate power domain but we might want to control 
> these from
> a single driver.
> 
> So I wanted to ask if there have been any news on this topic lately?

There has not but there is still clearly a need for this and we need it
for Tegra. I still plan to get back to this but recently a few items
have cropped up and I have not had chance. So sorry about that.

My plan was to split the current GPD framework so there is a lower-level
set of APIs for managing the power-domains as I discussed with Rafael [0].

Cheers
Jon

[0] https://lkml.org/lkml/2017/5/2/203

-- 
nvpublic

Re: Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Todor Tomov]

Hi,

On 30.05.2017 06:41, Rajendra Nayak wrote:
> []..
> 
 I was proposing to have such a lower-layer by splitting the existing
 genpd framework so the drivers would have the option of calling the
 lower-level power control functions to look-up pm-domains and control
 them directly from their rpm callbacks (if they need to). Same as we do
 for clocks. This way you would not need to mess with the genpd ->start()
 callback and leave it to the driver to handle itself as it knows what
 needs to be done. This assumes that the device is never bound to the
 pm-domain by the genpd core.
>>>
>>> Yes, agree! To me this is the only solution what would really work.
>>
>> I agree! :-)
>>
>>> Perhaps Rafael can confirm that he is fine with a solution like this?
>>
>> Yes and Rafael, please can you also elaborate on what you meant by
>> "allow genpd to use either a list of power resources or the on/off
>> callbacks provided by itself to cover different use cases"?
>>
>> I would like to understand exactly what you meant by allowing genpd to
>> use a list of power resources (ie. how you envisioned we could achieve
>> this).
> 
> While thinking through the problem of devices associated with multiple Power
> domains (or power resources) and controlling them individually (or together)
> I was wondering if something like a PM domain governor (with PM resource 
> level constraints) could help.
> 
> So with just one set of PM domain callbacks, its quite easy to control 
> multiple power
> resources, if they need to be *all* turned on/off together, using something 
> similar to
> what Jon proposed in his RFC [1]
> 
> However, there could be instances where in we might need to control them 
> individually
> and in such cases we could hook up a PM domain governor which decides if an 
> individual
> PM resource can be turned on or off while the device is runtime 
> suspended/resumed.
> We can expose some PM resource level QoS APIs which the drivers can use to 
> express their
> needs, which the PM domain governor then takes into account during the 
> decision making.
> 
> if this seems worth pursuing further, I can post some RFCs on these lines and
> get the discussion going.
> 
> thanks,
> Rajendra
> 
> [1] https://lkml.org/lkml/2016/9/20/173
> 

I have come to a similar case with multiple power domains on Qualcomm APQ8096 - 
the
camera subsystem has two VFE modules (Video Front End - these are image 
processing modules)
and each of them has a separate power domain but we might want to control these 
from
a single driver.

So I wanted to ask if there have been any news on this topic lately?

Thank you.
Best regards,
Todor

Re: Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Todor Tomov]

Hi,

On 30.05.2017 06:41, Rajendra Nayak wrote:
> []..
> 
 I was proposing to have such a lower-layer by splitting the existing
 genpd framework so the drivers would have the option of calling the
 lower-level power control functions to look-up pm-domains and control
 them directly from their rpm callbacks (if they need to). Same as we do
 for clocks. This way you would not need to mess with the genpd ->start()
 callback and leave it to the driver to handle itself as it knows what
 needs to be done. This assumes that the device is never bound to the
 pm-domain by the genpd core.
>>>
>>> Yes, agree! To me this is the only solution what would really work.
>>
>> I agree! :-)
>>
>>> Perhaps Rafael can confirm that he is fine with a solution like this?
>>
>> Yes and Rafael, please can you also elaborate on what you meant by
>> "allow genpd to use either a list of power resources or the on/off
>> callbacks provided by itself to cover different use cases"?
>>
>> I would like to understand exactly what you meant by allowing genpd to
>> use a list of power resources (ie. how you envisioned we could achieve
>> this).
> 
> While thinking through the problem of devices associated with multiple Power
> domains (or power resources) and controlling them individually (or together)
> I was wondering if something like a PM domain governor (with PM resource 
> level constraints) could help.
> 
> So with just one set of PM domain callbacks, its quite easy to control 
> multiple power
> resources, if they need to be *all* turned on/off together, using something 
> similar to
> what Jon proposed in his RFC [1]
> 
> However, there could be instances where in we might need to control them 
> individually
> and in such cases we could hook up a PM domain governor which decides if an 
> individual
> PM resource can be turned on or off while the device is runtime 
> suspended/resumed.
> We can expose some PM resource level QoS APIs which the drivers can use to 
> express their
> needs, which the PM domain governor then takes into account during the 
> decision making.
> 
> if this seems worth pursuing further, I can post some RFCs on these lines and
> get the discussion going.
> 
> thanks,
> Rajendra
> 
> [1] https://lkml.org/lkml/2016/9/20/173
> 

I have come to a similar case with multiple power domains on Qualcomm APQ8096 - 
the
camera subsystem has two VFE modules (Video Front End - these are image 
processing modules)
and each of them has a separate power domain but we might want to control these 
from
a single driver.

So I wanted to ask if there have been any news on this topic lately?

Thank you.
Best regards,
Todor

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Rajendra Nayak]

[]..

>>> I was proposing to have such a lower-layer by splitting the existing
>>> genpd framework so the drivers would have the option of calling the
>>> lower-level power control functions to look-up pm-domains and control
>>> them directly from their rpm callbacks (if they need to). Same as we do
>>> for clocks. This way you would not need to mess with the genpd ->start()
>>> callback and leave it to the driver to handle itself as it knows what
>>> needs to be done. This assumes that the device is never bound to the
>>> pm-domain by the genpd core.
>>
>> Yes, agree! To me this is the only solution what would really work.
> 
> I agree! :-)
> 
>> Perhaps Rafael can confirm that he is fine with a solution like this?
> 
> Yes and Rafael, please can you also elaborate on what you meant by
> "allow genpd to use either a list of power resources or the on/off
> callbacks provided by itself to cover different use cases"?
> 
> I would like to understand exactly what you meant by allowing genpd to
> use a list of power resources (ie. how you envisioned we could achieve
> this).

While thinking through the problem of devices associated with multiple Power
domains (or power resources) and controlling them individually (or together)
I was wondering if something like a PM domain governor (with PM resource 
level constraints) could help.

So with just one set of PM domain callbacks, its quite easy to control multiple 
power
resources, if they need to be *all* turned on/off together, using something 
similar to
what Jon proposed in his RFC [1]

However, there could be instances where in we might need to control them 
individually
and in such cases we could hook up a PM domain governor which decides if an 
individual
PM resource can be turned on or off while the device is runtime 
suspended/resumed.
We can expose some PM resource level QoS APIs which the drivers can use to 
express their
needs, which the PM domain governor then takes into account during the decision 
making.

if this seems worth pursuing further, I can post some RFCs on these lines and
get the discussion going.

thanks,
Rajendra

[1] https://lkml.org/lkml/2016/9/20/173

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Rajendra Nayak]

[]..

>>> I was proposing to have such a lower-layer by splitting the existing
>>> genpd framework so the drivers would have the option of calling the
>>> lower-level power control functions to look-up pm-domains and control
>>> them directly from their rpm callbacks (if they need to). Same as we do
>>> for clocks. This way you would not need to mess with the genpd ->start()
>>> callback and leave it to the driver to handle itself as it knows what
>>> needs to be done. This assumes that the device is never bound to the
>>> pm-domain by the genpd core.
>>
>> Yes, agree! To me this is the only solution what would really work.
> 
> I agree! :-)
> 
>> Perhaps Rafael can confirm that he is fine with a solution like this?
> 
> Yes and Rafael, please can you also elaborate on what you meant by
> "allow genpd to use either a list of power resources or the on/off
> callbacks provided by itself to cover different use cases"?
> 
> I would like to understand exactly what you meant by allowing genpd to
> use a list of power resources (ie. how you envisioned we could achieve
> this).

While thinking through the problem of devices associated with multiple Power
domains (or power resources) and controlling them individually (or together)
I was wondering if something like a PM domain governor (with PM resource 
level constraints) could help.

So with just one set of PM domain callbacks, its quite easy to control multiple 
power
resources, if they need to be *all* turned on/off together, using something 
similar to
what Jon proposed in his RFC [1]

However, there could be instances where in we might need to control them 
individually
and in such cases we could hook up a PM domain governor which decides if an 
individual
PM resource can be turned on or off while the device is runtime 
suspended/resumed.
We can expose some PM resource level QoS APIs which the drivers can use to 
express their
needs, which the PM domain governor then takes into account during the decision 
making.

if this seems worth pursuing further, I can post some RFCs on these lines and
get the discussion going.

thanks,
Rajendra

[1] https://lkml.org/lkml/2016/9/20/173

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 03/05/17 18:12, Ulf Hansson wrote:
> [...]
> 

> What is missing, is how a call to pm_runtime_get_sync() by a driver,
> can inform the ->start() callback about what exact power resource(s)
> it shall turn on, because it may not always be all of them. Similar
> problem exists for pm_runtime_put().

 Yes that is missing too, but I still don't see how you bind a device to
 more than one power-domain :-(
>>>
>>> I think this is what Rafael wanted us avoid (because of the complexity
>>> involved). Instead the suggestion is to deal with this on top of the
>>> existing PM domain structure, as "power resources" instead. Unless I
>>> missed his point. :-)
>>>
>>> Then *my* point is: To be able to implement that, still only allowing
>>> one PM domain per device, we would anyway need a new layer in-between
>>> the PM domain (genpd) and the driver controlling the device. Simply
>>> because these two entities, needs to be able to exchange information
>>> about which "power resources" that shall be turned off/on, when the
>>> driver do pm_runtime_get|put().
>>
>> Right, but isn't this similar to what I was suggesting above in my
>> previous email?
> 
> Perhaps it was, apologize for my ignorance in that case.

No problem, sometimes email is not the clearest medium (or maybe it is
just the author/me who struggles ;-)

> Then, isn't that quite what you already started hacking on in $subject
> series? What's the new thing in your previous reply?

Well this series just adds new APIs, but does not partition the
framework into the two layers we discussed. So it seems we should
partition the code into layers first, then add these new APIs.

> However, I was under the impression that Rafael preferred another way
> - but again I might have missed his point.
> 
>>
>> I was proposing to have such a lower-layer by splitting the existing
>> genpd framework so the drivers would have the option of calling the
>> lower-level power control functions to look-up pm-domains and control
>> them directly from their rpm callbacks (if they need to). Same as we do
>> for clocks. This way you would not need to mess with the genpd ->start()
>> callback and leave it to the driver to handle itself as it knows what
>> needs to be done. This assumes that the device is never bound to the
>> pm-domain by the genpd core.
> 
> Yes, agree! To me this is the only solution what would really work.

I agree! :-)

> Perhaps Rafael can confirm that he is fine with a solution like this?

Yes and Rafael, please can you also elaborate on what you meant by
"allow genpd to use either a list of power resources or the on/off
callbacks provided by itself to cover different use cases"?

I would like to understand exactly what you meant by allowing genpd to
use a list of power resources (ie. how you envisioned we could achieve
this).

Cheers
Jon

-- 
nvpublic

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 03/05/17 18:12, Ulf Hansson wrote:
> [...]
> 

> What is missing, is how a call to pm_runtime_get_sync() by a driver,
> can inform the ->start() callback about what exact power resource(s)
> it shall turn on, because it may not always be all of them. Similar
> problem exists for pm_runtime_put().

 Yes that is missing too, but I still don't see how you bind a device to
 more than one power-domain :-(
>>>
>>> I think this is what Rafael wanted us avoid (because of the complexity
>>> involved). Instead the suggestion is to deal with this on top of the
>>> existing PM domain structure, as "power resources" instead. Unless I
>>> missed his point. :-)
>>>
>>> Then *my* point is: To be able to implement that, still only allowing
>>> one PM domain per device, we would anyway need a new layer in-between
>>> the PM domain (genpd) and the driver controlling the device. Simply
>>> because these two entities, needs to be able to exchange information
>>> about which "power resources" that shall be turned off/on, when the
>>> driver do pm_runtime_get|put().
>>
>> Right, but isn't this similar to what I was suggesting above in my
>> previous email?
> 
> Perhaps it was, apologize for my ignorance in that case.

No problem, sometimes email is not the clearest medium (or maybe it is
just the author/me who struggles ;-)

> Then, isn't that quite what you already started hacking on in $subject
> series? What's the new thing in your previous reply?

Well this series just adds new APIs, but does not partition the
framework into the two layers we discussed. So it seems we should
partition the code into layers first, then add these new APIs.

> However, I was under the impression that Rafael preferred another way
> - but again I might have missed his point.
> 
>>
>> I was proposing to have such a lower-layer by splitting the existing
>> genpd framework so the drivers would have the option of calling the
>> lower-level power control functions to look-up pm-domains and control
>> them directly from their rpm callbacks (if they need to). Same as we do
>> for clocks. This way you would not need to mess with the genpd ->start()
>> callback and leave it to the driver to handle itself as it knows what
>> needs to be done. This assumes that the device is never bound to the
>> pm-domain by the genpd core.
> 
> Yes, agree! To me this is the only solution what would really work.

I agree! :-)

> Perhaps Rafael can confirm that he is fine with a solution like this?

Yes and Rafael, please can you also elaborate on what you meant by
"allow genpd to use either a list of power resources or the on/off
callbacks provided by itself to cover different use cases"?

I would like to understand exactly what you meant by allowing genpd to
use a list of power resources (ie. how you envisioned we could achieve
this).

Cheers
Jon

-- 
nvpublic

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Ulf Hansson]

[...]

>>>
 What is missing, is how a call to pm_runtime_get_sync() by a driver,
 can inform the ->start() callback about what exact power resource(s)
 it shall turn on, because it may not always be all of them. Similar
 problem exists for pm_runtime_put().
>>>
>>> Yes that is missing too, but I still don't see how you bind a device to
>>> more than one power-domain :-(
>>
>> I think this is what Rafael wanted us avoid (because of the complexity
>> involved). Instead the suggestion is to deal with this on top of the
>> existing PM domain structure, as "power resources" instead. Unless I
>> missed his point. :-)
>>
>> Then *my* point is: To be able to implement that, still only allowing
>> one PM domain per device, we would anyway need a new layer in-between
>> the PM domain (genpd) and the driver controlling the device. Simply
>> because these two entities, needs to be able to exchange information
>> about which "power resources" that shall be turned off/on, when the
>> driver do pm_runtime_get|put().
>
> Right, but isn't this similar to what I was suggesting above in my
> previous email?

Perhaps it was, apologize for my ignorance in that case.

Then, isn't that quite what you already started hacking on in $subject
series? What's the new thing in your previous reply?

However, I was under the impression that Rafael preferred another way
- but again I might have missed his point.

>
> I was proposing to have such a lower-layer by splitting the existing
> genpd framework so the drivers would have the option of calling the
> lower-level power control functions to look-up pm-domains and control
> them directly from their rpm callbacks (if they need to). Same as we do
> for clocks. This way you would not need to mess with the genpd ->start()
> callback and leave it to the driver to handle itself as it knows what
> needs to be done. This assumes that the device is never bound to the
> pm-domain by the genpd core.

Yes, agree! To me this is the only solution what would really work.

Perhaps Rafael can confirm that he is fine with a solution like this?

Kind regards
Uffe

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Ulf Hansson]

[...]

>>>
 What is missing, is how a call to pm_runtime_get_sync() by a driver,
 can inform the ->start() callback about what exact power resource(s)
 it shall turn on, because it may not always be all of them. Similar
 problem exists for pm_runtime_put().
>>>
>>> Yes that is missing too, but I still don't see how you bind a device to
>>> more than one power-domain :-(
>>
>> I think this is what Rafael wanted us avoid (because of the complexity
>> involved). Instead the suggestion is to deal with this on top of the
>> existing PM domain structure, as "power resources" instead. Unless I
>> missed his point. :-)
>>
>> Then *my* point is: To be able to implement that, still only allowing
>> one PM domain per device, we would anyway need a new layer in-between
>> the PM domain (genpd) and the driver controlling the device. Simply
>> because these two entities, needs to be able to exchange information
>> about which "power resources" that shall be turned off/on, when the
>> driver do pm_runtime_get|put().
>
> Right, but isn't this similar to what I was suggesting above in my
> previous email?

Perhaps it was, apologize for my ignorance in that case.

Then, isn't that quite what you already started hacking on in $subject
series? What's the new thing in your previous reply?

However, I was under the impression that Rafael preferred another way
- but again I might have missed his point.

>
> I was proposing to have such a lower-layer by splitting the existing
> genpd framework so the drivers would have the option of calling the
> lower-level power control functions to look-up pm-domains and control
> them directly from their rpm callbacks (if they need to). Same as we do
> for clocks. This way you would not need to mess with the genpd ->start()
> callback and leave it to the driver to handle itself as it knows what
> needs to be done. This assumes that the device is never bound to the
> pm-domain by the genpd core.

Yes, agree! To me this is the only solution what would really work.

Perhaps Rafael can confirm that he is fine with a solution like this?

Kind regards
Uffe

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 03/05/17 14:43, Ulf Hansson wrote:
> On 3 May 2017 at 10:32, Jon Hunter  wrote:
>> On 03/05/17 09:12, Ulf Hansson wrote:
>>> On 2 May 2017 at 23:51, Rafael J. Wysocki  wrote:
 On Tuesday, May 02, 2017 11:10:29 AM Jon Hunter wrote:

...

> So unless I am misunderstanding you here, it seems that what we need to do
> is split the current genpd framework into a couple layers:
>
> 1. Low-level power resource control which has:
> - Power resource registration (ie. pm_genpd_init/remove())
> - Power resource provider registration (ie. of_genpd_add_xxx())
> - Power resource control (on/off etc)

 And reference counting.

> - Power resource lookup (what this series is adding)
>
> 2. Generic power-domain infrastructure which is a client of the
> low-level power resource control that can automatically bind a device to
> a singular power resource entity (ie. power-domain).

 Something like that, but I would not require an additional complex 
 framework
 to be present below genpd.  I would make it *possible* for genpd to use 
 that
 framework if available, but if something simpler is sufficient, it should 
 be
 fine to use that instead.

 That is, I would allow genpd to use either a list of power resources or 
 the on/off
 callbacks provided by itself to cover different use cases.  That should be
 flexible enough.

> Is this along the right lines?

 It is, at least for the very narrow definition of "right" as being done 
 along
 the lines I would do that. :-)
>>>
>>> Let me first give some more background to how it looks like today.
>>>
>>> We have the following device callbacks being used by genpd:
>>>
>>> struct gpd_dev_ops {
>>> int (*start)(struct device *dev);
>>> int (*stop)(struct device *dev);
>>> bool (*active_wakeup)(struct device *dev);
>>> };
>>>
>>> The ->stop|start() callback is invoked from genpd's
>>> ->runtime_suspend|resume() callbacks and may be assigned by a genpd
>>> client before it registers a genpd though pm_genpd_init(). Typically
>>> these callbacks can control any "power resources" that the genpd
>>> client finds suitable.
>>>
>>> To allow clients to instantiate "power resources" per device, we have
>>> the following callbacks, part of the struct generic_pm_domain.
>>>
>>> int (*attach_dev)(struct generic_pm_domain *domain, struct device *dev);
>>> void (*detach_dev)(struct generic_pm_domain *domain, struct device *dev);
>>>
>>> These callbacks are invoked when the device gets attached/detached
>>> from its PM domain (genpd).
>>>
>>> Moreover, the struct dev_pm_get_subsys_data is being used from within
>>> genpd (via struct pm_domain_data *domain_data), as it allows genpd and
>>> its client, to allocate and associate device specific data, which may
>>> be needed to store information about the "power resources".
>>>
>>> Currently Renesas SoCs are a good example of how to deploy this, as it
>>> implements its clock PM domain on top of this. In this regards I
>>> assume that we could consider the pm_clk_*() APIs as in control of the
>>> "power resources".
>>>
>>> So my conclusion is; unless I am totally misunderstanding the ideas
>>> here; I think we already have the infrastructure in place and we also
>>> have some good references of how to use it.
>>
>> So how do you represent more than one power-domain in device-tree for a
>> device which requires more than one? That is the fundamental problem IMO.
> 
> Describing this in DT is a separate, but related question. However can
> we first focus on how to manage this in software. :-)
> 
>>
>> From what Geert has described (and what I understand), it appears that
>> for the Renesas SoCs, the current infrastructure works because a device
>> has either a clock-domain or a power-domain and clock-domain. For
>> devices with both a power-domain and clock-domain, the clock-domain can
>> be handled via the clock callbacks (pm_clk_suspend/resume I assume).
>> However, this does not scale or work for devices that truly need more
>> than one power-domain which XUSB on Tegra does.
> 
> Exactly.
> 
>>
>>> What is missing, is how a call to pm_runtime_get_sync() by a driver,
>>> can inform the ->start() callback about what exact power resource(s)
>>> it shall turn on, because it may not always be all of them. Similar
>>> problem exists for pm_runtime_put().
>>
>> Yes that is missing too, but I still don't see how you bind a device to
>> more than one power-domain :-(
> 
> I think this is what Rafael wanted us avoid (because of the complexity
> involved). Instead the suggestion is to deal with this on top of the
> existing PM domain structure, as "power resources" instead. Unless I
> missed his point. :-)
> 
> Then *my* point is: To be able to implement that, still only allowing
> one PM domain per device, we would anyway need a new layer in-between
> the PM 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 03/05/17 14:43, Ulf Hansson wrote:
> On 3 May 2017 at 10:32, Jon Hunter  wrote:
>> On 03/05/17 09:12, Ulf Hansson wrote:
>>> On 2 May 2017 at 23:51, Rafael J. Wysocki  wrote:
 On Tuesday, May 02, 2017 11:10:29 AM Jon Hunter wrote:

...

> So unless I am misunderstanding you here, it seems that what we need to do
> is split the current genpd framework into a couple layers:
>
> 1. Low-level power resource control which has:
> - Power resource registration (ie. pm_genpd_init/remove())
> - Power resource provider registration (ie. of_genpd_add_xxx())
> - Power resource control (on/off etc)

 And reference counting.

> - Power resource lookup (what this series is adding)
>
> 2. Generic power-domain infrastructure which is a client of the
> low-level power resource control that can automatically bind a device to
> a singular power resource entity (ie. power-domain).

 Something like that, but I would not require an additional complex 
 framework
 to be present below genpd.  I would make it *possible* for genpd to use 
 that
 framework if available, but if something simpler is sufficient, it should 
 be
 fine to use that instead.

 That is, I would allow genpd to use either a list of power resources or 
 the on/off
 callbacks provided by itself to cover different use cases.  That should be
 flexible enough.

> Is this along the right lines?

 It is, at least for the very narrow definition of "right" as being done 
 along
 the lines I would do that. :-)
>>>
>>> Let me first give some more background to how it looks like today.
>>>
>>> We have the following device callbacks being used by genpd:
>>>
>>> struct gpd_dev_ops {
>>> int (*start)(struct device *dev);
>>> int (*stop)(struct device *dev);
>>> bool (*active_wakeup)(struct device *dev);
>>> };
>>>
>>> The ->stop|start() callback is invoked from genpd's
>>> ->runtime_suspend|resume() callbacks and may be assigned by a genpd
>>> client before it registers a genpd though pm_genpd_init(). Typically
>>> these callbacks can control any "power resources" that the genpd
>>> client finds suitable.
>>>
>>> To allow clients to instantiate "power resources" per device, we have
>>> the following callbacks, part of the struct generic_pm_domain.
>>>
>>> int (*attach_dev)(struct generic_pm_domain *domain, struct device *dev);
>>> void (*detach_dev)(struct generic_pm_domain *domain, struct device *dev);
>>>
>>> These callbacks are invoked when the device gets attached/detached
>>> from its PM domain (genpd).
>>>
>>> Moreover, the struct dev_pm_get_subsys_data is being used from within
>>> genpd (via struct pm_domain_data *domain_data), as it allows genpd and
>>> its client, to allocate and associate device specific data, which may
>>> be needed to store information about the "power resources".
>>>
>>> Currently Renesas SoCs are a good example of how to deploy this, as it
>>> implements its clock PM domain on top of this. In this regards I
>>> assume that we could consider the pm_clk_*() APIs as in control of the
>>> "power resources".
>>>
>>> So my conclusion is; unless I am totally misunderstanding the ideas
>>> here; I think we already have the infrastructure in place and we also
>>> have some good references of how to use it.
>>
>> So how do you represent more than one power-domain in device-tree for a
>> device which requires more than one? That is the fundamental problem IMO.
> 
> Describing this in DT is a separate, but related question. However can
> we first focus on how to manage this in software. :-)
> 
>>
>> From what Geert has described (and what I understand), it appears that
>> for the Renesas SoCs, the current infrastructure works because a device
>> has either a clock-domain or a power-domain and clock-domain. For
>> devices with both a power-domain and clock-domain, the clock-domain can
>> be handled via the clock callbacks (pm_clk_suspend/resume I assume).
>> However, this does not scale or work for devices that truly need more
>> than one power-domain which XUSB on Tegra does.
> 
> Exactly.
> 
>>
>>> What is missing, is how a call to pm_runtime_get_sync() by a driver,
>>> can inform the ->start() callback about what exact power resource(s)
>>> it shall turn on, because it may not always be all of them. Similar
>>> problem exists for pm_runtime_put().
>>
>> Yes that is missing too, but I still don't see how you bind a device to
>> more than one power-domain :-(
> 
> I think this is what Rafael wanted us avoid (because of the complexity
> involved). Instead the suggestion is to deal with this on top of the
> existing PM domain structure, as "power resources" instead. Unless I
> missed his point. :-)
> 
> Then *my* point is: To be able to implement that, still only allowing
> one PM domain per device, we would anyway need a new layer in-between
> the PM domain (genpd) and the driver controlling 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Ulf Hansson]

On 3 May 2017 at 10:32, Jon Hunter  wrote:
>
> On 03/05/17 09:12, Ulf Hansson wrote:
>> Rafael, Jon,
>>
>> On 2 May 2017 at 23:51, Rafael J. Wysocki  wrote:
>>> On Tuesday, May 02, 2017 11:10:29 AM Jon Hunter wrote:

 On 25/04/17 22:17, Rafael J. Wysocki wrote:
> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  
> wrote:
>> On 25 April 2017 at 13:13, Jon Hunter  wrote:
>>>
>>> On 28/03/17 15:13, Jon Hunter wrote:
 The current generic PM domain framework (GenDP) only allows a single
 PM domain to be associated with a given device. There are several
 use-cases for various system-on-chip devices where it is necessary for
 a PM domain consumer to control more than one PM domain where the PM
 domains:
 i).  Do not conform to a parent-child relationship so are not nested
 ii). May not be powered on and off at the same time so need independent
  control.

 The solution proposed in this RFC is to allow consumers to explictly
 control PM domains, by getting a handle to a PM domain and explicitly
 making calls to power on and off the PM domain. Note that referencing
 counting is used to ensure that a PM domain shared between consumers
 is not powered off incorrectly.

 The Tegra124/210 XUSB subsystem (that consists of both host and device
 controllers) is an example of a consumer that needs to control more 
 than
 one PM domain because the logic is partitioned across 3 PM domains 
 which
 are:
 - XUSBA: Superspeed logic (for USB 3.0)
 - XUSBB: Device controller
 - XUSBC: Host controller

 These power domains are not nested and can be powered-up and down
 independently of one another. In practice different scenarios require
 different combinations of the power domains, for example:
 - Superspeed host: XUSBA and XUSBC
 - Superspeed device: XUSBA and XUSBB

 Although it could be possible to logically nest both the XUSBB and 
 XUSBC
 domains under the XUSBA, superspeed may not always be used/required and
 so this would keep it on unnecessarily.

 Given that Tegra uses device-tree for describing the hardware, it would
 be ideal that the device-tree 'power-domains' property for generic PM
 domains could be extended to allow more than one PM domain to be
 specified. For example, define the following the Tegra210 xHCI device 
 ...

   usb@7009 {
   compatible = "nvidia,tegra210-xusb";
   ...
   power-domains = <_xusbhost>, <_xusbss>;
   power-domain-names = "host", "superspeed";
   };

 This RFC extends the generic PM domain framework to allow a device to
 define more than one PM domain in the device-tree 'power-domains'
 property. If there is more than one then the assumption is that these
 PM domains will be controlled explicitly by the consumer and the device
 will not be automatically bound to any PM domain.
>>>
>>> Any more comments/inputs on this? I can address Rajendra's feedback, but
>>> before I did I wanted to see if this is along the right lines or not?
>>
>> I discussed this with Rafael at the OSPM summit in Pisa a couple of
>> weeks ago. Apologize for the delay in providing additional feedback.
>>
>> First, whether the problem is really rare, perhaps adding a new
>> API/framework can't be justified - then it may be better to add some
>> kind of aggregation layer on top of the current PM domain
>> infrastructure (something along the first attempt you made for genpd).
>> That was kind of Rafael's thoughts (Rafael, please correct me if I am
>> wrong).
>
> We were talking about the original idea behind the pm_domain pointer
> concept, which was about adding a set of PM operations above the bus
> type/class layer, which could be used for intercepting bus-type PM
> operations and providing some common handling above them.  This is
> still relevant IMO.
>
> The basic observation here is that the PM core takes only one set of
> PM operation per device into account and therefore, in every stage of
> system suspend, for example, the callback invoked by it has to take
> care of all actions that need to be carried out for the given device,
> possibly by invoking callbacks from other code layers.  That
> limitation cannot be removed easily, because it is built into the PM
> core design quite fundamentally.
>
> However, this series seems to be about controlling power resources
> represented by power domain 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Ulf Hansson]

On 3 May 2017 at 10:32, Jon Hunter  wrote:
>
> On 03/05/17 09:12, Ulf Hansson wrote:
>> Rafael, Jon,
>>
>> On 2 May 2017 at 23:51, Rafael J. Wysocki  wrote:
>>> On Tuesday, May 02, 2017 11:10:29 AM Jon Hunter wrote:

 On 25/04/17 22:17, Rafael J. Wysocki wrote:
> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  
> wrote:
>> On 25 April 2017 at 13:13, Jon Hunter  wrote:
>>>
>>> On 28/03/17 15:13, Jon Hunter wrote:
 The current generic PM domain framework (GenDP) only allows a single
 PM domain to be associated with a given device. There are several
 use-cases for various system-on-chip devices where it is necessary for
 a PM domain consumer to control more than one PM domain where the PM
 domains:
 i).  Do not conform to a parent-child relationship so are not nested
 ii). May not be powered on and off at the same time so need independent
  control.

 The solution proposed in this RFC is to allow consumers to explictly
 control PM domains, by getting a handle to a PM domain and explicitly
 making calls to power on and off the PM domain. Note that referencing
 counting is used to ensure that a PM domain shared between consumers
 is not powered off incorrectly.

 The Tegra124/210 XUSB subsystem (that consists of both host and device
 controllers) is an example of a consumer that needs to control more 
 than
 one PM domain because the logic is partitioned across 3 PM domains 
 which
 are:
 - XUSBA: Superspeed logic (for USB 3.0)
 - XUSBB: Device controller
 - XUSBC: Host controller

 These power domains are not nested and can be powered-up and down
 independently of one another. In practice different scenarios require
 different combinations of the power domains, for example:
 - Superspeed host: XUSBA and XUSBC
 - Superspeed device: XUSBA and XUSBB

 Although it could be possible to logically nest both the XUSBB and 
 XUSBC
 domains under the XUSBA, superspeed may not always be used/required and
 so this would keep it on unnecessarily.

 Given that Tegra uses device-tree for describing the hardware, it would
 be ideal that the device-tree 'power-domains' property for generic PM
 domains could be extended to allow more than one PM domain to be
 specified. For example, define the following the Tegra210 xHCI device 
 ...

   usb@7009 {
   compatible = "nvidia,tegra210-xusb";
   ...
   power-domains = <_xusbhost>, <_xusbss>;
   power-domain-names = "host", "superspeed";
   };

 This RFC extends the generic PM domain framework to allow a device to
 define more than one PM domain in the device-tree 'power-domains'
 property. If there is more than one then the assumption is that these
 PM domains will be controlled explicitly by the consumer and the device
 will not be automatically bound to any PM domain.
>>>
>>> Any more comments/inputs on this? I can address Rajendra's feedback, but
>>> before I did I wanted to see if this is along the right lines or not?
>>
>> I discussed this with Rafael at the OSPM summit in Pisa a couple of
>> weeks ago. Apologize for the delay in providing additional feedback.
>>
>> First, whether the problem is really rare, perhaps adding a new
>> API/framework can't be justified - then it may be better to add some
>> kind of aggregation layer on top of the current PM domain
>> infrastructure (something along the first attempt you made for genpd).
>> That was kind of Rafael's thoughts (Rafael, please correct me if I am
>> wrong).
>
> We were talking about the original idea behind the pm_domain pointer
> concept, which was about adding a set of PM operations above the bus
> type/class layer, which could be used for intercepting bus-type PM
> operations and providing some common handling above them.  This is
> still relevant IMO.
>
> The basic observation here is that the PM core takes only one set of
> PM operation per device into account and therefore, in every stage of
> system suspend, for example, the callback invoked by it has to take
> care of all actions that need to be carried out for the given device,
> possibly by invoking callbacks from other code layers.  That
> limitation cannot be removed easily, because it is built into the PM
> core design quite fundamentally.
>
> However, this series seems to be about controlling power resources
> represented by power domain objects rather than about PM operations.
> In ACPI there is a power resource concept 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Geert Uytterhoeven]

Hi Ulf,

On Wed, May 3, 2017 at 8:43 AM, Geert Uytterhoeven  wrote:
> On Wed, Apr 26, 2017 at 11:55 AM, Ulf Hansson  wrote:
>> On 26 April 2017 at 11:17, Geert Uytterhoeven  wrote:
>>> On Wed, Apr 26, 2017 at 11:04 AM, Ulf Hansson  
>>> wrote:
 On 26 April 2017 at 10:06, Geert Uytterhoeven  wrote:
> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  
> wrote:
>> However, we currently know about at least two different SoCs that need
>> this. Perhaps we can extend the below list to justify adding a new
>> framework/APIs. Something along the lines what you propose in $subject
>> patchset.
>>
>> 1) Nvidia; to solve the USB super-speed host/device problem.
>> 2) QCOM, which has pointed to several cases where the PM topology is
>> laid out like devices having two PM domains..
>> 3?) I don't fully remember - but I think Geert also pointed to some
>> examples where a device could reside in a clock domain but also in
>> power domain for a Renesas SoC!?
>> 4) ?
>
> Most Renesas SoCs have module clocks, which we model as a clock domain.
> Some Renesas SoCs have power domains for CPUs, others have them for
> devices as well.
> As we always provide a virtual "always-on" power domain in the power 
> domain
> controller, all devices can refer to it using "power-domains" properties,
> and the driver for the power domain controller can just forward the clock
> domain operations to the clock driver.

 Okay, thanks for clarifying this.

 Thinking about this as bit more, when I realized that *if* we would
 add a new PM domain framework for explicit control of PM domains, that
 would mean you need to deploy support for that in the drivers.
>>>
>>> Correct.  And we have to update DT bindings and DTS.
>>>
 On the other hand, as you anyway would need to change the drivers, you
 could instead deploy clock support in the drivers, which would avoid
 using the clock domain. In that way, you could still stay with one PM
 domain pointer per device, used to control the power domains instead.
 Right? Or would that have other implications?
>>>
>>> That's exactly what we're doing already.
>>
>> No really, but perhaps I was not clear enough.
>>
>> Currently you deploy only runtime PM support in the driver and don't
>> do any clk_get() etc. Then you have a PM domain (genpd) attached to

Note that drivers that care about clock properties do use clk_get().
E.g. an SPI controller driver needs to know the clock rate to program
its internal divider to drive external SPI slaves.

>> the device and makes use of genpd's device specific callbacks, in
>> struct gpd_dev_ops ->start|stop(), which allows you to control clocks
>> for each device. Of course this is perfectly okay.
>
> OK.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Geert Uytterhoeven]

Hi Ulf,

On Wed, May 3, 2017 at 8:43 AM, Geert Uytterhoeven  wrote:
> On Wed, Apr 26, 2017 at 11:55 AM, Ulf Hansson  wrote:
>> On 26 April 2017 at 11:17, Geert Uytterhoeven  wrote:
>>> On Wed, Apr 26, 2017 at 11:04 AM, Ulf Hansson  
>>> wrote:
 On 26 April 2017 at 10:06, Geert Uytterhoeven  wrote:
> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  
> wrote:
>> However, we currently know about at least two different SoCs that need
>> this. Perhaps we can extend the below list to justify adding a new
>> framework/APIs. Something along the lines what you propose in $subject
>> patchset.
>>
>> 1) Nvidia; to solve the USB super-speed host/device problem.
>> 2) QCOM, which has pointed to several cases where the PM topology is
>> laid out like devices having two PM domains..
>> 3?) I don't fully remember - but I think Geert also pointed to some
>> examples where a device could reside in a clock domain but also in
>> power domain for a Renesas SoC!?
>> 4) ?
>
> Most Renesas SoCs have module clocks, which we model as a clock domain.
> Some Renesas SoCs have power domains for CPUs, others have them for
> devices as well.
> As we always provide a virtual "always-on" power domain in the power 
> domain
> controller, all devices can refer to it using "power-domains" properties,
> and the driver for the power domain controller can just forward the clock
> domain operations to the clock driver.

 Okay, thanks for clarifying this.

 Thinking about this as bit more, when I realized that *if* we would
 add a new PM domain framework for explicit control of PM domains, that
 would mean you need to deploy support for that in the drivers.
>>>
>>> Correct.  And we have to update DT bindings and DTS.
>>>
 On the other hand, as you anyway would need to change the drivers, you
 could instead deploy clock support in the drivers, which would avoid
 using the clock domain. In that way, you could still stay with one PM
 domain pointer per device, used to control the power domains instead.
 Right? Or would that have other implications?
>>>
>>> That's exactly what we're doing already.
>>
>> No really, but perhaps I was not clear enough.
>>
>> Currently you deploy only runtime PM support in the driver and don't
>> do any clk_get() etc. Then you have a PM domain (genpd) attached to

Note that drivers that care about clock properties do use clk_get().
E.g. an SPI controller driver needs to know the clock rate to program
its internal divider to drive external SPI slaves.

>> the device and makes use of genpd's device specific callbacks, in
>> struct gpd_dev_ops ->start|stop(), which allows you to control clocks
>> for each device. Of course this is perfectly okay.
>
> OK.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 03/05/17 09:12, Ulf Hansson wrote:
> Rafael, Jon,
> 
> On 2 May 2017 at 23:51, Rafael J. Wysocki  wrote:
>> On Tuesday, May 02, 2017 11:10:29 AM Jon Hunter wrote:
>>>
>>> On 25/04/17 22:17, Rafael J. Wysocki wrote:
 On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  
 wrote:
> On 25 April 2017 at 13:13, Jon Hunter  wrote:
>>
>> On 28/03/17 15:13, Jon Hunter wrote:
>>> The current generic PM domain framework (GenDP) only allows a single
>>> PM domain to be associated with a given device. There are several
>>> use-cases for various system-on-chip devices where it is necessary for
>>> a PM domain consumer to control more than one PM domain where the PM
>>> domains:
>>> i).  Do not conform to a parent-child relationship so are not nested
>>> ii). May not be powered on and off at the same time so need independent
>>>  control.
>>>
>>> The solution proposed in this RFC is to allow consumers to explictly
>>> control PM domains, by getting a handle to a PM domain and explicitly
>>> making calls to power on and off the PM domain. Note that referencing
>>> counting is used to ensure that a PM domain shared between consumers
>>> is not powered off incorrectly.
>>>
>>> The Tegra124/210 XUSB subsystem (that consists of both host and device
>>> controllers) is an example of a consumer that needs to control more than
>>> one PM domain because the logic is partitioned across 3 PM domains which
>>> are:
>>> - XUSBA: Superspeed logic (for USB 3.0)
>>> - XUSBB: Device controller
>>> - XUSBC: Host controller
>>>
>>> These power domains are not nested and can be powered-up and down
>>> independently of one another. In practice different scenarios require
>>> different combinations of the power domains, for example:
>>> - Superspeed host: XUSBA and XUSBC
>>> - Superspeed device: XUSBA and XUSBB
>>>
>>> Although it could be possible to logically nest both the XUSBB and XUSBC
>>> domains under the XUSBA, superspeed may not always be used/required and
>>> so this would keep it on unnecessarily.
>>>
>>> Given that Tegra uses device-tree for describing the hardware, it would
>>> be ideal that the device-tree 'power-domains' property for generic PM
>>> domains could be extended to allow more than one PM domain to be
>>> specified. For example, define the following the Tegra210 xHCI device 
>>> ...
>>>
>>>   usb@7009 {
>>>   compatible = "nvidia,tegra210-xusb";
>>>   ...
>>>   power-domains = <_xusbhost>, <_xusbss>;
>>>   power-domain-names = "host", "superspeed";
>>>   };
>>>
>>> This RFC extends the generic PM domain framework to allow a device to
>>> define more than one PM domain in the device-tree 'power-domains'
>>> property. If there is more than one then the assumption is that these
>>> PM domains will be controlled explicitly by the consumer and the device
>>> will not be automatically bound to any PM domain.
>>
>> Any more comments/inputs on this? I can address Rajendra's feedback, but
>> before I did I wanted to see if this is along the right lines or not?
>
> I discussed this with Rafael at the OSPM summit in Pisa a couple of
> weeks ago. Apologize for the delay in providing additional feedback.
>
> First, whether the problem is really rare, perhaps adding a new
> API/framework can't be justified - then it may be better to add some
> kind of aggregation layer on top of the current PM domain
> infrastructure (something along the first attempt you made for genpd).
> That was kind of Rafael's thoughts (Rafael, please correct me if I am
> wrong).

 We were talking about the original idea behind the pm_domain pointer
 concept, which was about adding a set of PM operations above the bus
 type/class layer, which could be used for intercepting bus-type PM
 operations and providing some common handling above them.  This is
 still relevant IMO.

 The basic observation here is that the PM core takes only one set of
 PM operation per device into account and therefore, in every stage of
 system suspend, for example, the callback invoked by it has to take
 care of all actions that need to be carried out for the given device,
 possibly by invoking callbacks from other code layers.  That
 limitation cannot be removed easily, because it is built into the PM
 core design quite fundamentally.

 However, this series seems to be about controlling power resources
 represented by power domain objects rather than about PM operations.
 In ACPI there is a power resource concept which seems to be quite
 similar to this, so it is not entirely new. :-)

 Of course, 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 03/05/17 09:12, Ulf Hansson wrote:
> Rafael, Jon,
> 
> On 2 May 2017 at 23:51, Rafael J. Wysocki  wrote:
>> On Tuesday, May 02, 2017 11:10:29 AM Jon Hunter wrote:
>>>
>>> On 25/04/17 22:17, Rafael J. Wysocki wrote:
 On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  
 wrote:
> On 25 April 2017 at 13:13, Jon Hunter  wrote:
>>
>> On 28/03/17 15:13, Jon Hunter wrote:
>>> The current generic PM domain framework (GenDP) only allows a single
>>> PM domain to be associated with a given device. There are several
>>> use-cases for various system-on-chip devices where it is necessary for
>>> a PM domain consumer to control more than one PM domain where the PM
>>> domains:
>>> i).  Do not conform to a parent-child relationship so are not nested
>>> ii). May not be powered on and off at the same time so need independent
>>>  control.
>>>
>>> The solution proposed in this RFC is to allow consumers to explictly
>>> control PM domains, by getting a handle to a PM domain and explicitly
>>> making calls to power on and off the PM domain. Note that referencing
>>> counting is used to ensure that a PM domain shared between consumers
>>> is not powered off incorrectly.
>>>
>>> The Tegra124/210 XUSB subsystem (that consists of both host and device
>>> controllers) is an example of a consumer that needs to control more than
>>> one PM domain because the logic is partitioned across 3 PM domains which
>>> are:
>>> - XUSBA: Superspeed logic (for USB 3.0)
>>> - XUSBB: Device controller
>>> - XUSBC: Host controller
>>>
>>> These power domains are not nested and can be powered-up and down
>>> independently of one another. In practice different scenarios require
>>> different combinations of the power domains, for example:
>>> - Superspeed host: XUSBA and XUSBC
>>> - Superspeed device: XUSBA and XUSBB
>>>
>>> Although it could be possible to logically nest both the XUSBB and XUSBC
>>> domains under the XUSBA, superspeed may not always be used/required and
>>> so this would keep it on unnecessarily.
>>>
>>> Given that Tegra uses device-tree for describing the hardware, it would
>>> be ideal that the device-tree 'power-domains' property for generic PM
>>> domains could be extended to allow more than one PM domain to be
>>> specified. For example, define the following the Tegra210 xHCI device 
>>> ...
>>>
>>>   usb@7009 {
>>>   compatible = "nvidia,tegra210-xusb";
>>>   ...
>>>   power-domains = <_xusbhost>, <_xusbss>;
>>>   power-domain-names = "host", "superspeed";
>>>   };
>>>
>>> This RFC extends the generic PM domain framework to allow a device to
>>> define more than one PM domain in the device-tree 'power-domains'
>>> property. If there is more than one then the assumption is that these
>>> PM domains will be controlled explicitly by the consumer and the device
>>> will not be automatically bound to any PM domain.
>>
>> Any more comments/inputs on this? I can address Rajendra's feedback, but
>> before I did I wanted to see if this is along the right lines or not?
>
> I discussed this with Rafael at the OSPM summit in Pisa a couple of
> weeks ago. Apologize for the delay in providing additional feedback.
>
> First, whether the problem is really rare, perhaps adding a new
> API/framework can't be justified - then it may be better to add some
> kind of aggregation layer on top of the current PM domain
> infrastructure (something along the first attempt you made for genpd).
> That was kind of Rafael's thoughts (Rafael, please correct me if I am
> wrong).

 We were talking about the original idea behind the pm_domain pointer
 concept, which was about adding a set of PM operations above the bus
 type/class layer, which could be used for intercepting bus-type PM
 operations and providing some common handling above them.  This is
 still relevant IMO.

 The basic observation here is that the PM core takes only one set of
 PM operation per device into account and therefore, in every stage of
 system suspend, for example, the callback invoked by it has to take
 care of all actions that need to be carried out for the given device,
 possibly by invoking callbacks from other code layers.  That
 limitation cannot be removed easily, because it is built into the PM
 core design quite fundamentally.

 However, this series seems to be about controlling power resources
 represented by power domain objects rather than about PM operations.
 In ACPI there is a power resource concept which seems to be quite
 similar to this, so it is not entirely new. :-)

 Of course, question is whether or not to extend genpd this way and I'm
 not 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Ulf Hansson]

Rafael, Jon,

On 2 May 2017 at 23:51, Rafael J. Wysocki  wrote:
> On Tuesday, May 02, 2017 11:10:29 AM Jon Hunter wrote:
>>
>> On 25/04/17 22:17, Rafael J. Wysocki wrote:
>> > On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  
>> > wrote:
>> >> On 25 April 2017 at 13:13, Jon Hunter  wrote:
>> >>>
>> >>> On 28/03/17 15:13, Jon Hunter wrote:
>>  The current generic PM domain framework (GenDP) only allows a single
>>  PM domain to be associated with a given device. There are several
>>  use-cases for various system-on-chip devices where it is necessary for
>>  a PM domain consumer to control more than one PM domain where the PM
>>  domains:
>>  i).  Do not conform to a parent-child relationship so are not nested
>>  ii). May not be powered on and off at the same time so need independent
>>   control.
>> 
>>  The solution proposed in this RFC is to allow consumers to explictly
>>  control PM domains, by getting a handle to a PM domain and explicitly
>>  making calls to power on and off the PM domain. Note that referencing
>>  counting is used to ensure that a PM domain shared between consumers
>>  is not powered off incorrectly.
>> 
>>  The Tegra124/210 XUSB subsystem (that consists of both host and device
>>  controllers) is an example of a consumer that needs to control more than
>>  one PM domain because the logic is partitioned across 3 PM domains which
>>  are:
>>  - XUSBA: Superspeed logic (for USB 3.0)
>>  - XUSBB: Device controller
>>  - XUSBC: Host controller
>> 
>>  These power domains are not nested and can be powered-up and down
>>  independently of one another. In practice different scenarios require
>>  different combinations of the power domains, for example:
>>  - Superspeed host: XUSBA and XUSBC
>>  - Superspeed device: XUSBA and XUSBB
>> 
>>  Although it could be possible to logically nest both the XUSBB and XUSBC
>>  domains under the XUSBA, superspeed may not always be used/required and
>>  so this would keep it on unnecessarily.
>> 
>>  Given that Tegra uses device-tree for describing the hardware, it would
>>  be ideal that the device-tree 'power-domains' property for generic PM
>>  domains could be extended to allow more than one PM domain to be
>>  specified. For example, define the following the Tegra210 xHCI device 
>>  ...
>> 
>>    usb@7009 {
>>    compatible = "nvidia,tegra210-xusb";
>>    ...
>>    power-domains = <_xusbhost>, <_xusbss>;
>>    power-domain-names = "host", "superspeed";
>>    };
>> 
>>  This RFC extends the generic PM domain framework to allow a device to
>>  define more than one PM domain in the device-tree 'power-domains'
>>  property. If there is more than one then the assumption is that these
>>  PM domains will be controlled explicitly by the consumer and the device
>>  will not be automatically bound to any PM domain.
>> >>>
>> >>> Any more comments/inputs on this? I can address Rajendra's feedback, but
>> >>> before I did I wanted to see if this is along the right lines or not?
>> >>
>> >> I discussed this with Rafael at the OSPM summit in Pisa a couple of
>> >> weeks ago. Apologize for the delay in providing additional feedback.
>> >>
>> >> First, whether the problem is really rare, perhaps adding a new
>> >> API/framework can't be justified - then it may be better to add some
>> >> kind of aggregation layer on top of the current PM domain
>> >> infrastructure (something along the first attempt you made for genpd).
>> >> That was kind of Rafael's thoughts (Rafael, please correct me if I am
>> >> wrong).
>> >
>> > We were talking about the original idea behind the pm_domain pointer
>> > concept, which was about adding a set of PM operations above the bus
>> > type/class layer, which could be used for intercepting bus-type PM
>> > operations and providing some common handling above them.  This is
>> > still relevant IMO.
>> >
>> > The basic observation here is that the PM core takes only one set of
>> > PM operation per device into account and therefore, in every stage of
>> > system suspend, for example, the callback invoked by it has to take
>> > care of all actions that need to be carried out for the given device,
>> > possibly by invoking callbacks from other code layers.  That
>> > limitation cannot be removed easily, because it is built into the PM
>> > core design quite fundamentally.
>> >
>> > However, this series seems to be about controlling power resources
>> > represented by power domain objects rather than about PM operations.
>> > In ACPI there is a power resource concept which seems to be quite
>> > similar to this, so it is not entirely new. :-)
>> >
>> > Of course, question is whether or not to extend genpd this way 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 02/05/17 22:51, Rafael J. Wysocki wrote:
> On Tuesday, May 02, 2017 11:10:29 AM Jon Hunter wrote:
>>
>> On 25/04/17 22:17, Rafael J. Wysocki wrote:
>>> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
 On 25 April 2017 at 13:13, Jon Hunter  wrote:
>
> On 28/03/17 15:13, Jon Hunter wrote:
>> The current generic PM domain framework (GenDP) only allows a single
>> PM domain to be associated with a given device. There are several
>> use-cases for various system-on-chip devices where it is necessary for
>> a PM domain consumer to control more than one PM domain where the PM
>> domains:
>> i).  Do not conform to a parent-child relationship so are not nested
>> ii). May not be powered on and off at the same time so need independent
>>  control.
>>
>> The solution proposed in this RFC is to allow consumers to explictly
>> control PM domains, by getting a handle to a PM domain and explicitly
>> making calls to power on and off the PM domain. Note that referencing
>> counting is used to ensure that a PM domain shared between consumers
>> is not powered off incorrectly.
>>
>> The Tegra124/210 XUSB subsystem (that consists of both host and device
>> controllers) is an example of a consumer that needs to control more than
>> one PM domain because the logic is partitioned across 3 PM domains which
>> are:
>> - XUSBA: Superspeed logic (for USB 3.0)
>> - XUSBB: Device controller
>> - XUSBC: Host controller
>>
>> These power domains are not nested and can be powered-up and down
>> independently of one another. In practice different scenarios require
>> different combinations of the power domains, for example:
>> - Superspeed host: XUSBA and XUSBC
>> - Superspeed device: XUSBA and XUSBB
>>
>> Although it could be possible to logically nest both the XUSBB and XUSBC
>> domains under the XUSBA, superspeed may not always be used/required and
>> so this would keep it on unnecessarily.
>>
>> Given that Tegra uses device-tree for describing the hardware, it would
>> be ideal that the device-tree 'power-domains' property for generic PM
>> domains could be extended to allow more than one PM domain to be
>> specified. For example, define the following the Tegra210 xHCI device ...
>>
>>   usb@7009 {
>>   compatible = "nvidia,tegra210-xusb";
>>   ...
>>   power-domains = <_xusbhost>, <_xusbss>;
>>   power-domain-names = "host", "superspeed";
>>   };
>>
>> This RFC extends the generic PM domain framework to allow a device to
>> define more than one PM domain in the device-tree 'power-domains'
>> property. If there is more than one then the assumption is that these
>> PM domains will be controlled explicitly by the consumer and the device
>> will not be automatically bound to any PM domain.
>
> Any more comments/inputs on this? I can address Rajendra's feedback, but
> before I did I wanted to see if this is along the right lines or not?

 I discussed this with Rafael at the OSPM summit in Pisa a couple of
 weeks ago. Apologize for the delay in providing additional feedback.

 First, whether the problem is really rare, perhaps adding a new
 API/framework can't be justified - then it may be better to add some
 kind of aggregation layer on top of the current PM domain
 infrastructure (something along the first attempt you made for genpd).
 That was kind of Rafael's thoughts (Rafael, please correct me if I am
 wrong).
>>>
>>> We were talking about the original idea behind the pm_domain pointer
>>> concept, which was about adding a set of PM operations above the bus
>>> type/class layer, which could be used for intercepting bus-type PM
>>> operations and providing some common handling above them.  This is
>>> still relevant IMO.
>>>
>>> The basic observation here is that the PM core takes only one set of
>>> PM operation per device into account and therefore, in every stage of
>>> system suspend, for example, the callback invoked by it has to take
>>> care of all actions that need to be carried out for the given device,
>>> possibly by invoking callbacks from other code layers.  That
>>> limitation cannot be removed easily, because it is built into the PM
>>> core design quite fundamentally.
>>>
>>> However, this series seems to be about controlling power resources
>>> represented by power domain objects rather than about PM operations.
>>> In ACPI there is a power resource concept which seems to be quite
>>> similar to this, so it is not entirely new. :-)
>>>
>>> Of course, question is whether or not to extend genpd this way and I'm
>>> not really sure.  I actually probably wouldn't do that, because
>>> poweron/poweroff operations used by genpd can be implemeted in 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Ulf Hansson]

Rafael, Jon,

On 2 May 2017 at 23:51, Rafael J. Wysocki  wrote:
> On Tuesday, May 02, 2017 11:10:29 AM Jon Hunter wrote:
>>
>> On 25/04/17 22:17, Rafael J. Wysocki wrote:
>> > On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  
>> > wrote:
>> >> On 25 April 2017 at 13:13, Jon Hunter  wrote:
>> >>>
>> >>> On 28/03/17 15:13, Jon Hunter wrote:
>>  The current generic PM domain framework (GenDP) only allows a single
>>  PM domain to be associated with a given device. There are several
>>  use-cases for various system-on-chip devices where it is necessary for
>>  a PM domain consumer to control more than one PM domain where the PM
>>  domains:
>>  i).  Do not conform to a parent-child relationship so are not nested
>>  ii). May not be powered on and off at the same time so need independent
>>   control.
>> 
>>  The solution proposed in this RFC is to allow consumers to explictly
>>  control PM domains, by getting a handle to a PM domain and explicitly
>>  making calls to power on and off the PM domain. Note that referencing
>>  counting is used to ensure that a PM domain shared between consumers
>>  is not powered off incorrectly.
>> 
>>  The Tegra124/210 XUSB subsystem (that consists of both host and device
>>  controllers) is an example of a consumer that needs to control more than
>>  one PM domain because the logic is partitioned across 3 PM domains which
>>  are:
>>  - XUSBA: Superspeed logic (for USB 3.0)
>>  - XUSBB: Device controller
>>  - XUSBC: Host controller
>> 
>>  These power domains are not nested and can be powered-up and down
>>  independently of one another. In practice different scenarios require
>>  different combinations of the power domains, for example:
>>  - Superspeed host: XUSBA and XUSBC
>>  - Superspeed device: XUSBA and XUSBB
>> 
>>  Although it could be possible to logically nest both the XUSBB and XUSBC
>>  domains under the XUSBA, superspeed may not always be used/required and
>>  so this would keep it on unnecessarily.
>> 
>>  Given that Tegra uses device-tree for describing the hardware, it would
>>  be ideal that the device-tree 'power-domains' property for generic PM
>>  domains could be extended to allow more than one PM domain to be
>>  specified. For example, define the following the Tegra210 xHCI device 
>>  ...
>> 
>>    usb@7009 {
>>    compatible = "nvidia,tegra210-xusb";
>>    ...
>>    power-domains = <_xusbhost>, <_xusbss>;
>>    power-domain-names = "host", "superspeed";
>>    };
>> 
>>  This RFC extends the generic PM domain framework to allow a device to
>>  define more than one PM domain in the device-tree 'power-domains'
>>  property. If there is more than one then the assumption is that these
>>  PM domains will be controlled explicitly by the consumer and the device
>>  will not be automatically bound to any PM domain.
>> >>>
>> >>> Any more comments/inputs on this? I can address Rajendra's feedback, but
>> >>> before I did I wanted to see if this is along the right lines or not?
>> >>
>> >> I discussed this with Rafael at the OSPM summit in Pisa a couple of
>> >> weeks ago. Apologize for the delay in providing additional feedback.
>> >>
>> >> First, whether the problem is really rare, perhaps adding a new
>> >> API/framework can't be justified - then it may be better to add some
>> >> kind of aggregation layer on top of the current PM domain
>> >> infrastructure (something along the first attempt you made for genpd).
>> >> That was kind of Rafael's thoughts (Rafael, please correct me if I am
>> >> wrong).
>> >
>> > We were talking about the original idea behind the pm_domain pointer
>> > concept, which was about adding a set of PM operations above the bus
>> > type/class layer, which could be used for intercepting bus-type PM
>> > operations and providing some common handling above them.  This is
>> > still relevant IMO.
>> >
>> > The basic observation here is that the PM core takes only one set of
>> > PM operation per device into account and therefore, in every stage of
>> > system suspend, for example, the callback invoked by it has to take
>> > care of all actions that need to be carried out for the given device,
>> > possibly by invoking callbacks from other code layers.  That
>> > limitation cannot be removed easily, because it is built into the PM
>> > core design quite fundamentally.
>> >
>> > However, this series seems to be about controlling power resources
>> > represented by power domain objects rather than about PM operations.
>> > In ACPI there is a power resource concept which seems to be quite
>> > similar to this, so it is not entirely new. :-)
>> >
>> > Of course, question is whether or not to extend genpd this way and I'm
>> > not really sure.  I actually probably wouldn't do 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 02/05/17 22:51, Rafael J. Wysocki wrote:
> On Tuesday, May 02, 2017 11:10:29 AM Jon Hunter wrote:
>>
>> On 25/04/17 22:17, Rafael J. Wysocki wrote:
>>> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
 On 25 April 2017 at 13:13, Jon Hunter  wrote:
>
> On 28/03/17 15:13, Jon Hunter wrote:
>> The current generic PM domain framework (GenDP) only allows a single
>> PM domain to be associated with a given device. There are several
>> use-cases for various system-on-chip devices where it is necessary for
>> a PM domain consumer to control more than one PM domain where the PM
>> domains:
>> i).  Do not conform to a parent-child relationship so are not nested
>> ii). May not be powered on and off at the same time so need independent
>>  control.
>>
>> The solution proposed in this RFC is to allow consumers to explictly
>> control PM domains, by getting a handle to a PM domain and explicitly
>> making calls to power on and off the PM domain. Note that referencing
>> counting is used to ensure that a PM domain shared between consumers
>> is not powered off incorrectly.
>>
>> The Tegra124/210 XUSB subsystem (that consists of both host and device
>> controllers) is an example of a consumer that needs to control more than
>> one PM domain because the logic is partitioned across 3 PM domains which
>> are:
>> - XUSBA: Superspeed logic (for USB 3.0)
>> - XUSBB: Device controller
>> - XUSBC: Host controller
>>
>> These power domains are not nested and can be powered-up and down
>> independently of one another. In practice different scenarios require
>> different combinations of the power domains, for example:
>> - Superspeed host: XUSBA and XUSBC
>> - Superspeed device: XUSBA and XUSBB
>>
>> Although it could be possible to logically nest both the XUSBB and XUSBC
>> domains under the XUSBA, superspeed may not always be used/required and
>> so this would keep it on unnecessarily.
>>
>> Given that Tegra uses device-tree for describing the hardware, it would
>> be ideal that the device-tree 'power-domains' property for generic PM
>> domains could be extended to allow more than one PM domain to be
>> specified. For example, define the following the Tegra210 xHCI device ...
>>
>>   usb@7009 {
>>   compatible = "nvidia,tegra210-xusb";
>>   ...
>>   power-domains = <_xusbhost>, <_xusbss>;
>>   power-domain-names = "host", "superspeed";
>>   };
>>
>> This RFC extends the generic PM domain framework to allow a device to
>> define more than one PM domain in the device-tree 'power-domains'
>> property. If there is more than one then the assumption is that these
>> PM domains will be controlled explicitly by the consumer and the device
>> will not be automatically bound to any PM domain.
>
> Any more comments/inputs on this? I can address Rajendra's feedback, but
> before I did I wanted to see if this is along the right lines or not?

 I discussed this with Rafael at the OSPM summit in Pisa a couple of
 weeks ago. Apologize for the delay in providing additional feedback.

 First, whether the problem is really rare, perhaps adding a new
 API/framework can't be justified - then it may be better to add some
 kind of aggregation layer on top of the current PM domain
 infrastructure (something along the first attempt you made for genpd).
 That was kind of Rafael's thoughts (Rafael, please correct me if I am
 wrong).
>>>
>>> We were talking about the original idea behind the pm_domain pointer
>>> concept, which was about adding a set of PM operations above the bus
>>> type/class layer, which could be used for intercepting bus-type PM
>>> operations and providing some common handling above them.  This is
>>> still relevant IMO.
>>>
>>> The basic observation here is that the PM core takes only one set of
>>> PM operation per device into account and therefore, in every stage of
>>> system suspend, for example, the callback invoked by it has to take
>>> care of all actions that need to be carried out for the given device,
>>> possibly by invoking callbacks from other code layers.  That
>>> limitation cannot be removed easily, because it is built into the PM
>>> core design quite fundamentally.
>>>
>>> However, this series seems to be about controlling power resources
>>> represented by power domain objects rather than about PM operations.
>>> In ACPI there is a power resource concept which seems to be quite
>>> similar to this, so it is not entirely new. :-)
>>>
>>> Of course, question is whether or not to extend genpd this way and I'm
>>> not really sure.  I actually probably wouldn't do that, because
>>> poweron/poweroff operations used by genpd can be implemeted in terms
>>> of lower-level power resource 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Geert Uytterhoeven]

Hi Ulf,

On Wed, Apr 26, 2017 at 11:55 AM, Ulf Hansson  wrote:
> On 26 April 2017 at 11:17, Geert Uytterhoeven  wrote:
>> On Wed, Apr 26, 2017 at 11:04 AM, Ulf Hansson  wrote:
>>> On 26 April 2017 at 10:06, Geert Uytterhoeven  wrote:
 On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  
 wrote:
> However, we currently know about at least two different SoCs that need
> this. Perhaps we can extend the below list to justify adding a new
> framework/APIs. Something along the lines what you propose in $subject
> patchset.
>
> 1) Nvidia; to solve the USB super-speed host/device problem.
> 2) QCOM, which has pointed to several cases where the PM topology is
> laid out like devices having two PM domains..
> 3?) I don't fully remember - but I think Geert also pointed to some
> examples where a device could reside in a clock domain but also in
> power domain for a Renesas SoC!?
> 4) ?

 Most Renesas SoCs have module clocks, which we model as a clock domain.
 Some Renesas SoCs have power domains for CPUs, others have them for
 devices as well.
 As we always provide a virtual "always-on" power domain in the power domain
 controller, all devices can refer to it using "power-domains" properties,
 and the driver for the power domain controller can just forward the clock
 domain operations to the clock driver.
>>>
>>> Okay, thanks for clarifying this.
>>>
>>> Thinking about this as bit more, when I realized that *if* we would
>>> add a new PM domain framework for explicit control of PM domains, that
>>> would mean you need to deploy support for that in the drivers.
>>
>> Correct.  And we have to update DT bindings and DTS.
>>
>>> On the other hand, as you anyway would need to change the drivers, you
>>> could instead deploy clock support in the drivers, which would avoid
>>> using the clock domain. In that way, you could still stay with one PM
>>> domain pointer per device, used to control the power domains instead.
>>> Right? Or would that have other implications?
>>
>> That's exactly what we're doing already.
>
> No really, but perhaps I was not clear enough.
>
> Currently you deploy only runtime PM support in the driver and don't
> do any clk_get() etc. Then you have a PM domain (genpd) attached to
> the device and makes use of genpd's device specific callbacks, in
> struct gpd_dev_ops ->start|stop(), which allows you to control clocks
> for each device. Of course this is perfectly okay.

OK.

> So then my question is/was; does there exist cases when these devices
> (already attached to a PM domain) would needed to be attach to yet
> another separate PM domain? From the nicely detailed description
> below, I find the answer to be *no*!?

Apart from the SYSC power areas and the CPG/MSSR clock domain
we don't have a use case for multiple power domains.

>> Which means that if you allow multiple entries in power-domains, we
>> have to change drivers, DT bindings, and DTS again (which we may
>> decide not to do ;-)
>>
>> On SH/R-Mobile, we always did it that way, as the user manual had an
>> explicit "always-on" power domain.
>>
>> On R-Car Gen2, the power domains contain CPU and L2 and GPU only,
>> so devices had their power-domains pointing to the clock controller.
>>
>> On R-Car Gen3, some devices were moved into power domains, so we
>> generalized this by creating a virtual "always-on" power domain, and
>> letting all devices point their power-domains properties to the power
>> domain controller, which forwards clock handling to the clock controller.
>> For consistency, this was applied to R-Car Gen2 as well.
>>
>> Cfr. some late relics fixed in e.g. commit 24b2d930a50662c1
>> ('ARM: dts: r8a7794: Use SYSC "always-on" PM Domain for sound'),
>> but technically the fix was not needed, as it worked fine without.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Geert Uytterhoeven]

Hi Ulf,

On Wed, Apr 26, 2017 at 11:55 AM, Ulf Hansson  wrote:
> On 26 April 2017 at 11:17, Geert Uytterhoeven  wrote:
>> On Wed, Apr 26, 2017 at 11:04 AM, Ulf Hansson  wrote:
>>> On 26 April 2017 at 10:06, Geert Uytterhoeven  wrote:
 On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  
 wrote:
> However, we currently know about at least two different SoCs that need
> this. Perhaps we can extend the below list to justify adding a new
> framework/APIs. Something along the lines what you propose in $subject
> patchset.
>
> 1) Nvidia; to solve the USB super-speed host/device problem.
> 2) QCOM, which has pointed to several cases where the PM topology is
> laid out like devices having two PM domains..
> 3?) I don't fully remember - but I think Geert also pointed to some
> examples where a device could reside in a clock domain but also in
> power domain for a Renesas SoC!?
> 4) ?

 Most Renesas SoCs have module clocks, which we model as a clock domain.
 Some Renesas SoCs have power domains for CPUs, others have them for
 devices as well.
 As we always provide a virtual "always-on" power domain in the power domain
 controller, all devices can refer to it using "power-domains" properties,
 and the driver for the power domain controller can just forward the clock
 domain operations to the clock driver.
>>>
>>> Okay, thanks for clarifying this.
>>>
>>> Thinking about this as bit more, when I realized that *if* we would
>>> add a new PM domain framework for explicit control of PM domains, that
>>> would mean you need to deploy support for that in the drivers.
>>
>> Correct.  And we have to update DT bindings and DTS.
>>
>>> On the other hand, as you anyway would need to change the drivers, you
>>> could instead deploy clock support in the drivers, which would avoid
>>> using the clock domain. In that way, you could still stay with one PM
>>> domain pointer per device, used to control the power domains instead.
>>> Right? Or would that have other implications?
>>
>> That's exactly what we're doing already.
>
> No really, but perhaps I was not clear enough.
>
> Currently you deploy only runtime PM support in the driver and don't
> do any clk_get() etc. Then you have a PM domain (genpd) attached to
> the device and makes use of genpd's device specific callbacks, in
> struct gpd_dev_ops ->start|stop(), which allows you to control clocks
> for each device. Of course this is perfectly okay.

OK.

> So then my question is/was; does there exist cases when these devices
> (already attached to a PM domain) would needed to be attach to yet
> another separate PM domain? From the nicely detailed description
> below, I find the answer to be *no*!?

Apart from the SYSC power areas and the CPG/MSSR clock domain
we don't have a use case for multiple power domains.

>> Which means that if you allow multiple entries in power-domains, we
>> have to change drivers, DT bindings, and DTS again (which we may
>> decide not to do ;-)
>>
>> On SH/R-Mobile, we always did it that way, as the user manual had an
>> explicit "always-on" power domain.
>>
>> On R-Car Gen2, the power domains contain CPU and L2 and GPU only,
>> so devices had their power-domains pointing to the clock controller.
>>
>> On R-Car Gen3, some devices were moved into power domains, so we
>> generalized this by creating a virtual "always-on" power domain, and
>> letting all devices point their power-domains properties to the power
>> domain controller, which forwards clock handling to the clock controller.
>> For consistency, this was applied to R-Car Gen2 as well.
>>
>> Cfr. some late relics fixed in e.g. commit 24b2d930a50662c1
>> ('ARM: dts: r8a7794: Use SYSC "always-on" PM Domain for sound'),
>> but technically the fix was not needed, as it worked fine without.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Rafael J. Wysocki]

On Tuesday, May 02, 2017 11:10:29 AM Jon Hunter wrote:
> 
> On 25/04/17 22:17, Rafael J. Wysocki wrote:
> > On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
> >> On 25 April 2017 at 13:13, Jon Hunter  wrote:
> >>>
> >>> On 28/03/17 15:13, Jon Hunter wrote:
>  The current generic PM domain framework (GenDP) only allows a single
>  PM domain to be associated with a given device. There are several
>  use-cases for various system-on-chip devices where it is necessary for
>  a PM domain consumer to control more than one PM domain where the PM
>  domains:
>  i).  Do not conform to a parent-child relationship so are not nested
>  ii). May not be powered on and off at the same time so need independent
>   control.
> 
>  The solution proposed in this RFC is to allow consumers to explictly
>  control PM domains, by getting a handle to a PM domain and explicitly
>  making calls to power on and off the PM domain. Note that referencing
>  counting is used to ensure that a PM domain shared between consumers
>  is not powered off incorrectly.
> 
>  The Tegra124/210 XUSB subsystem (that consists of both host and device
>  controllers) is an example of a consumer that needs to control more than
>  one PM domain because the logic is partitioned across 3 PM domains which
>  are:
>  - XUSBA: Superspeed logic (for USB 3.0)
>  - XUSBB: Device controller
>  - XUSBC: Host controller
> 
>  These power domains are not nested and can be powered-up and down
>  independently of one another. In practice different scenarios require
>  different combinations of the power domains, for example:
>  - Superspeed host: XUSBA and XUSBC
>  - Superspeed device: XUSBA and XUSBB
> 
>  Although it could be possible to logically nest both the XUSBB and XUSBC
>  domains under the XUSBA, superspeed may not always be used/required and
>  so this would keep it on unnecessarily.
> 
>  Given that Tegra uses device-tree for describing the hardware, it would
>  be ideal that the device-tree 'power-domains' property for generic PM
>  domains could be extended to allow more than one PM domain to be
>  specified. For example, define the following the Tegra210 xHCI device ...
> 
>    usb@7009 {
>    compatible = "nvidia,tegra210-xusb";
>    ...
>    power-domains = <_xusbhost>, <_xusbss>;
>    power-domain-names = "host", "superspeed";
>    };
> 
>  This RFC extends the generic PM domain framework to allow a device to
>  define more than one PM domain in the device-tree 'power-domains'
>  property. If there is more than one then the assumption is that these
>  PM domains will be controlled explicitly by the consumer and the device
>  will not be automatically bound to any PM domain.
> >>>
> >>> Any more comments/inputs on this? I can address Rajendra's feedback, but
> >>> before I did I wanted to see if this is along the right lines or not?
> >>
> >> I discussed this with Rafael at the OSPM summit in Pisa a couple of
> >> weeks ago. Apologize for the delay in providing additional feedback.
> >>
> >> First, whether the problem is really rare, perhaps adding a new
> >> API/framework can't be justified - then it may be better to add some
> >> kind of aggregation layer on top of the current PM domain
> >> infrastructure (something along the first attempt you made for genpd).
> >> That was kind of Rafael's thoughts (Rafael, please correct me if I am
> >> wrong).
> > 
> > We were talking about the original idea behind the pm_domain pointer
> > concept, which was about adding a set of PM operations above the bus
> > type/class layer, which could be used for intercepting bus-type PM
> > operations and providing some common handling above them.  This is
> > still relevant IMO.
> > 
> > The basic observation here is that the PM core takes only one set of
> > PM operation per device into account and therefore, in every stage of
> > system suspend, for example, the callback invoked by it has to take
> > care of all actions that need to be carried out for the given device,
> > possibly by invoking callbacks from other code layers.  That
> > limitation cannot be removed easily, because it is built into the PM
> > core design quite fundamentally.
> > 
> > However, this series seems to be about controlling power resources
> > represented by power domain objects rather than about PM operations.
> > In ACPI there is a power resource concept which seems to be quite
> > similar to this, so it is not entirely new. :-)
> > 
> > Of course, question is whether or not to extend genpd this way and I'm
> > not really sure.  I actually probably wouldn't do that, because
> > poweron/poweroff operations used by genpd can be implemeted in terms
> > of lower-level power resource 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Rafael J. Wysocki]

On Tuesday, May 02, 2017 11:10:29 AM Jon Hunter wrote:
> 
> On 25/04/17 22:17, Rafael J. Wysocki wrote:
> > On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
> >> On 25 April 2017 at 13:13, Jon Hunter  wrote:
> >>>
> >>> On 28/03/17 15:13, Jon Hunter wrote:
>  The current generic PM domain framework (GenDP) only allows a single
>  PM domain to be associated with a given device. There are several
>  use-cases for various system-on-chip devices where it is necessary for
>  a PM domain consumer to control more than one PM domain where the PM
>  domains:
>  i).  Do not conform to a parent-child relationship so are not nested
>  ii). May not be powered on and off at the same time so need independent
>   control.
> 
>  The solution proposed in this RFC is to allow consumers to explictly
>  control PM domains, by getting a handle to a PM domain and explicitly
>  making calls to power on and off the PM domain. Note that referencing
>  counting is used to ensure that a PM domain shared between consumers
>  is not powered off incorrectly.
> 
>  The Tegra124/210 XUSB subsystem (that consists of both host and device
>  controllers) is an example of a consumer that needs to control more than
>  one PM domain because the logic is partitioned across 3 PM domains which
>  are:
>  - XUSBA: Superspeed logic (for USB 3.0)
>  - XUSBB: Device controller
>  - XUSBC: Host controller
> 
>  These power domains are not nested and can be powered-up and down
>  independently of one another. In practice different scenarios require
>  different combinations of the power domains, for example:
>  - Superspeed host: XUSBA and XUSBC
>  - Superspeed device: XUSBA and XUSBB
> 
>  Although it could be possible to logically nest both the XUSBB and XUSBC
>  domains under the XUSBA, superspeed may not always be used/required and
>  so this would keep it on unnecessarily.
> 
>  Given that Tegra uses device-tree for describing the hardware, it would
>  be ideal that the device-tree 'power-domains' property for generic PM
>  domains could be extended to allow more than one PM domain to be
>  specified. For example, define the following the Tegra210 xHCI device ...
> 
>    usb@7009 {
>    compatible = "nvidia,tegra210-xusb";
>    ...
>    power-domains = <_xusbhost>, <_xusbss>;
>    power-domain-names = "host", "superspeed";
>    };
> 
>  This RFC extends the generic PM domain framework to allow a device to
>  define more than one PM domain in the device-tree 'power-domains'
>  property. If there is more than one then the assumption is that these
>  PM domains will be controlled explicitly by the consumer and the device
>  will not be automatically bound to any PM domain.
> >>>
> >>> Any more comments/inputs on this? I can address Rajendra's feedback, but
> >>> before I did I wanted to see if this is along the right lines or not?
> >>
> >> I discussed this with Rafael at the OSPM summit in Pisa a couple of
> >> weeks ago. Apologize for the delay in providing additional feedback.
> >>
> >> First, whether the problem is really rare, perhaps adding a new
> >> API/framework can't be justified - then it may be better to add some
> >> kind of aggregation layer on top of the current PM domain
> >> infrastructure (something along the first attempt you made for genpd).
> >> That was kind of Rafael's thoughts (Rafael, please correct me if I am
> >> wrong).
> > 
> > We were talking about the original idea behind the pm_domain pointer
> > concept, which was about adding a set of PM operations above the bus
> > type/class layer, which could be used for intercepting bus-type PM
> > operations and providing some common handling above them.  This is
> > still relevant IMO.
> > 
> > The basic observation here is that the PM core takes only one set of
> > PM operation per device into account and therefore, in every stage of
> > system suspend, for example, the callback invoked by it has to take
> > care of all actions that need to be carried out for the given device,
> > possibly by invoking callbacks from other code layers.  That
> > limitation cannot be removed easily, because it is built into the PM
> > core design quite fundamentally.
> > 
> > However, this series seems to be about controlling power resources
> > represented by power domain objects rather than about PM operations.
> > In ACPI there is a power resource concept which seems to be quite
> > similar to this, so it is not entirely new. :-)
> > 
> > Of course, question is whether or not to extend genpd this way and I'm
> > not really sure.  I actually probably wouldn't do that, because
> > poweron/poweroff operations used by genpd can be implemeted in terms
> > of lower-level power resource control and I don't see the reason for
> > mixing 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 25/04/17 22:17, Rafael J. Wysocki wrote:
> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
>> On 25 April 2017 at 13:13, Jon Hunter  wrote:
>>>
>>> On 28/03/17 15:13, Jon Hunter wrote:
 The current generic PM domain framework (GenDP) only allows a single
 PM domain to be associated with a given device. There are several
 use-cases for various system-on-chip devices where it is necessary for
 a PM domain consumer to control more than one PM domain where the PM
 domains:
 i).  Do not conform to a parent-child relationship so are not nested
 ii). May not be powered on and off at the same time so need independent
  control.

 The solution proposed in this RFC is to allow consumers to explictly
 control PM domains, by getting a handle to a PM domain and explicitly
 making calls to power on and off the PM domain. Note that referencing
 counting is used to ensure that a PM domain shared between consumers
 is not powered off incorrectly.

 The Tegra124/210 XUSB subsystem (that consists of both host and device
 controllers) is an example of a consumer that needs to control more than
 one PM domain because the logic is partitioned across 3 PM domains which
 are:
 - XUSBA: Superspeed logic (for USB 3.0)
 - XUSBB: Device controller
 - XUSBC: Host controller

 These power domains are not nested and can be powered-up and down
 independently of one another. In practice different scenarios require
 different combinations of the power domains, for example:
 - Superspeed host: XUSBA and XUSBC
 - Superspeed device: XUSBA and XUSBB

 Although it could be possible to logically nest both the XUSBB and XUSBC
 domains under the XUSBA, superspeed may not always be used/required and
 so this would keep it on unnecessarily.

 Given that Tegra uses device-tree for describing the hardware, it would
 be ideal that the device-tree 'power-domains' property for generic PM
 domains could be extended to allow more than one PM domain to be
 specified. For example, define the following the Tegra210 xHCI device ...

   usb@7009 {
   compatible = "nvidia,tegra210-xusb";
   ...
   power-domains = <_xusbhost>, <_xusbss>;
   power-domain-names = "host", "superspeed";
   };

 This RFC extends the generic PM domain framework to allow a device to
 define more than one PM domain in the device-tree 'power-domains'
 property. If there is more than one then the assumption is that these
 PM domains will be controlled explicitly by the consumer and the device
 will not be automatically bound to any PM domain.
>>>
>>> Any more comments/inputs on this? I can address Rajendra's feedback, but
>>> before I did I wanted to see if this is along the right lines or not?
>>
>> I discussed this with Rafael at the OSPM summit in Pisa a couple of
>> weeks ago. Apologize for the delay in providing additional feedback.
>>
>> First, whether the problem is really rare, perhaps adding a new
>> API/framework can't be justified - then it may be better to add some
>> kind of aggregation layer on top of the current PM domain
>> infrastructure (something along the first attempt you made for genpd).
>> That was kind of Rafael's thoughts (Rafael, please correct me if I am
>> wrong).
> 
> We were talking about the original idea behind the pm_domain pointer
> concept, which was about adding a set of PM operations above the bus
> type/class layer, which could be used for intercepting bus-type PM
> operations and providing some common handling above them.  This is
> still relevant IMO.
> 
> The basic observation here is that the PM core takes only one set of
> PM operation per device into account and therefore, in every stage of
> system suspend, for example, the callback invoked by it has to take
> care of all actions that need to be carried out for the given device,
> possibly by invoking callbacks from other code layers.  That
> limitation cannot be removed easily, because it is built into the PM
> core design quite fundamentally.
> 
> However, this series seems to be about controlling power resources
> represented by power domain objects rather than about PM operations.
> In ACPI there is a power resource concept which seems to be quite
> similar to this, so it is not entirely new. :-)
> 
> Of course, question is whether or not to extend genpd this way and I'm
> not really sure.  I actually probably wouldn't do that, because
> poweron/poweroff operations used by genpd can be implemeted in terms
> of lower-level power resource control and I don't see the reason for
> mixing the two in one framework.

That seems fine to me. However, it seems that genpd itself should also
be a client of this 'low-level power resource control' so that
power-domains are registered once 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 25/04/17 22:17, Rafael J. Wysocki wrote:
> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
>> On 25 April 2017 at 13:13, Jon Hunter  wrote:
>>>
>>> On 28/03/17 15:13, Jon Hunter wrote:
 The current generic PM domain framework (GenDP) only allows a single
 PM domain to be associated with a given device. There are several
 use-cases for various system-on-chip devices where it is necessary for
 a PM domain consumer to control more than one PM domain where the PM
 domains:
 i).  Do not conform to a parent-child relationship so are not nested
 ii). May not be powered on and off at the same time so need independent
  control.

 The solution proposed in this RFC is to allow consumers to explictly
 control PM domains, by getting a handle to a PM domain and explicitly
 making calls to power on and off the PM domain. Note that referencing
 counting is used to ensure that a PM domain shared between consumers
 is not powered off incorrectly.

 The Tegra124/210 XUSB subsystem (that consists of both host and device
 controllers) is an example of a consumer that needs to control more than
 one PM domain because the logic is partitioned across 3 PM domains which
 are:
 - XUSBA: Superspeed logic (for USB 3.0)
 - XUSBB: Device controller
 - XUSBC: Host controller

 These power domains are not nested and can be powered-up and down
 independently of one another. In practice different scenarios require
 different combinations of the power domains, for example:
 - Superspeed host: XUSBA and XUSBC
 - Superspeed device: XUSBA and XUSBB

 Although it could be possible to logically nest both the XUSBB and XUSBC
 domains under the XUSBA, superspeed may not always be used/required and
 so this would keep it on unnecessarily.

 Given that Tegra uses device-tree for describing the hardware, it would
 be ideal that the device-tree 'power-domains' property for generic PM
 domains could be extended to allow more than one PM domain to be
 specified. For example, define the following the Tegra210 xHCI device ...

   usb@7009 {
   compatible = "nvidia,tegra210-xusb";
   ...
   power-domains = <_xusbhost>, <_xusbss>;
   power-domain-names = "host", "superspeed";
   };

 This RFC extends the generic PM domain framework to allow a device to
 define more than one PM domain in the device-tree 'power-domains'
 property. If there is more than one then the assumption is that these
 PM domains will be controlled explicitly by the consumer and the device
 will not be automatically bound to any PM domain.
>>>
>>> Any more comments/inputs on this? I can address Rajendra's feedback, but
>>> before I did I wanted to see if this is along the right lines or not?
>>
>> I discussed this with Rafael at the OSPM summit in Pisa a couple of
>> weeks ago. Apologize for the delay in providing additional feedback.
>>
>> First, whether the problem is really rare, perhaps adding a new
>> API/framework can't be justified - then it may be better to add some
>> kind of aggregation layer on top of the current PM domain
>> infrastructure (something along the first attempt you made for genpd).
>> That was kind of Rafael's thoughts (Rafael, please correct me if I am
>> wrong).
> 
> We were talking about the original idea behind the pm_domain pointer
> concept, which was about adding a set of PM operations above the bus
> type/class layer, which could be used for intercepting bus-type PM
> operations and providing some common handling above them.  This is
> still relevant IMO.
> 
> The basic observation here is that the PM core takes only one set of
> PM operation per device into account and therefore, in every stage of
> system suspend, for example, the callback invoked by it has to take
> care of all actions that need to be carried out for the given device,
> possibly by invoking callbacks from other code layers.  That
> limitation cannot be removed easily, because it is built into the PM
> core design quite fundamentally.
> 
> However, this series seems to be about controlling power resources
> represented by power domain objects rather than about PM operations.
> In ACPI there is a power resource concept which seems to be quite
> similar to this, so it is not entirely new. :-)
> 
> Of course, question is whether or not to extend genpd this way and I'm
> not really sure.  I actually probably wouldn't do that, because
> poweron/poweroff operations used by genpd can be implemeted in terms
> of lower-level power resource control and I don't see the reason for
> mixing the two in one framework.

That seems fine to me. However, it seems that genpd itself should also
be a client of this 'low-level power resource control' so that
power-domains are registered once and can be used by either method. So
unless I am 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Ulf Hansson]

On 26 April 2017 at 11:17, Geert Uytterhoeven  wrote:
> Hi Ulf,
>
> On Wed, Apr 26, 2017 at 11:04 AM, Ulf Hansson  wrote:
>> On 26 April 2017 at 10:06, Geert Uytterhoeven  wrote:
>>> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
 However, we currently know about at least two different SoCs that need
 this. Perhaps we can extend the below list to justify adding a new
 framework/APIs. Something along the lines what you propose in $subject
 patchset.

 1) Nvidia; to solve the USB super-speed host/device problem.
 2) QCOM, which has pointed to several cases where the PM topology is
 laid out like devices having two PM domains..
 3?) I don't fully remember - but I think Geert also pointed to some
 examples where a device could reside in a clock domain but also in
 power domain for a Renesas SoC!?
 4) ?
>>>
>>> Most Renesas SoCs have module clocks, which we model as a clock domain.
>>> Some Renesas SoCs have power domains for CPUs, others have them for
>>> devices as well.
>>> As we always provide a virtual "always-on" power domain in the power domain
>>> controller, all devices can refer to it using "power-domains" properties,
>>> and the driver for the power domain controller can just forward the clock
>>> domain operations to the clock driver.
>>
>> Okay, thanks for clarifying this.
>>
>> Thinking about this as bit more, when I realized that *if* we would
>> add a new PM domain framework for explicit control of PM domains, that
>> would mean you need to deploy support for that in the drivers.
>
> Correct.  And we have to update DT bindings and DTS.
>
>> On the other hand, as you anyway would need to change the drivers, you
>> could instead deploy clock support in the drivers, which would avoid
>> using the clock domain. In that way, you could still stay with one PM
>> domain pointer per device, used to control the power domains instead.
>> Right? Or would that have other implications?
>
> That's exactly what we're doing already.

No really, but perhaps I was not clear enough.

Currently you deploy only runtime PM support in the driver and don't
do any clk_get() etc. Then you have a PM domain (genpd) attached to
the device and makes use of genpd's device specific callbacks, in
struct gpd_dev_ops ->start|stop(), which allows you to control clocks
for each device. Of course this is perfectly okay.

So then my question is/was; does there exist cases when these devices
(already attached to a PM domain) would needed to be attach to yet
another separate PM domain? From the nicely detailed description
below, I find the answer to be *no*!?

> Which means that if you allow multiple entries in power-domains, we
> have to change drivers, DT bindings, and DTS again (which we may
> decide not to do ;-)
>
> On SH/R-Mobile, we always did it that way, as the user manual had an
> explicit "always-on" power domain.
>
> On R-Car Gen2, the power domains contain CPU and L2 and GPU only,
> so devices had their power-domains pointing to the clock controller.
>
> On R-Car Gen3, some devices were moved into power domains, so we
> generalized this by creating a virtual "always-on" power domain, and
> letting all devices point their power-domains properties to the power
> domain controller, which forwards clock handling to the clock controller.
> For consistency, this was applied to R-Car Gen2 as well.
>
> Cfr. some late relics fixed in e.g. commit 24b2d930a50662c1
> ('ARM: dts: r8a7794: Use SYSC "always-on" PM Domain for sound'),
> but technically the fix was not needed, as it worked fine without.

Thanks for the detailed summary!

Kind regards
Uffe

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Ulf Hansson]

On 26 April 2017 at 11:17, Geert Uytterhoeven  wrote:
> Hi Ulf,
>
> On Wed, Apr 26, 2017 at 11:04 AM, Ulf Hansson  wrote:
>> On 26 April 2017 at 10:06, Geert Uytterhoeven  wrote:
>>> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
 However, we currently know about at least two different SoCs that need
 this. Perhaps we can extend the below list to justify adding a new
 framework/APIs. Something along the lines what you propose in $subject
 patchset.

 1) Nvidia; to solve the USB super-speed host/device problem.
 2) QCOM, which has pointed to several cases where the PM topology is
 laid out like devices having two PM domains..
 3?) I don't fully remember - but I think Geert also pointed to some
 examples where a device could reside in a clock domain but also in
 power domain for a Renesas SoC!?
 4) ?
>>>
>>> Most Renesas SoCs have module clocks, which we model as a clock domain.
>>> Some Renesas SoCs have power domains for CPUs, others have them for
>>> devices as well.
>>> As we always provide a virtual "always-on" power domain in the power domain
>>> controller, all devices can refer to it using "power-domains" properties,
>>> and the driver for the power domain controller can just forward the clock
>>> domain operations to the clock driver.
>>
>> Okay, thanks for clarifying this.
>>
>> Thinking about this as bit more, when I realized that *if* we would
>> add a new PM domain framework for explicit control of PM domains, that
>> would mean you need to deploy support for that in the drivers.
>
> Correct.  And we have to update DT bindings and DTS.
>
>> On the other hand, as you anyway would need to change the drivers, you
>> could instead deploy clock support in the drivers, which would avoid
>> using the clock domain. In that way, you could still stay with one PM
>> domain pointer per device, used to control the power domains instead.
>> Right? Or would that have other implications?
>
> That's exactly what we're doing already.

No really, but perhaps I was not clear enough.

Currently you deploy only runtime PM support in the driver and don't
do any clk_get() etc. Then you have a PM domain (genpd) attached to
the device and makes use of genpd's device specific callbacks, in
struct gpd_dev_ops ->start|stop(), which allows you to control clocks
for each device. Of course this is perfectly okay.

So then my question is/was; does there exist cases when these devices
(already attached to a PM domain) would needed to be attach to yet
another separate PM domain? From the nicely detailed description
below, I find the answer to be *no*!?

> Which means that if you allow multiple entries in power-domains, we
> have to change drivers, DT bindings, and DTS again (which we may
> decide not to do ;-)
>
> On SH/R-Mobile, we always did it that way, as the user manual had an
> explicit "always-on" power domain.
>
> On R-Car Gen2, the power domains contain CPU and L2 and GPU only,
> so devices had their power-domains pointing to the clock controller.
>
> On R-Car Gen3, some devices were moved into power domains, so we
> generalized this by creating a virtual "always-on" power domain, and
> letting all devices point their power-domains properties to the power
> domain controller, which forwards clock handling to the clock controller.
> For consistency, this was applied to R-Car Gen2 as well.
>
> Cfr. some late relics fixed in e.g. commit 24b2d930a50662c1
> ('ARM: dts: r8a7794: Use SYSC "always-on" PM Domain for sound'),
> but technically the fix was not needed, as it worked fine without.

Thanks for the detailed summary!

Kind regards
Uffe

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Geert Uytterhoeven]

Hi Ulf,

On Wed, Apr 26, 2017 at 11:04 AM, Ulf Hansson  wrote:
> On 26 April 2017 at 10:06, Geert Uytterhoeven  wrote:
>> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
>>> However, we currently know about at least two different SoCs that need
>>> this. Perhaps we can extend the below list to justify adding a new
>>> framework/APIs. Something along the lines what you propose in $subject
>>> patchset.
>>>
>>> 1) Nvidia; to solve the USB super-speed host/device problem.
>>> 2) QCOM, which has pointed to several cases where the PM topology is
>>> laid out like devices having two PM domains..
>>> 3?) I don't fully remember - but I think Geert also pointed to some
>>> examples where a device could reside in a clock domain but also in
>>> power domain for a Renesas SoC!?
>>> 4) ?
>>
>> Most Renesas SoCs have module clocks, which we model as a clock domain.
>> Some Renesas SoCs have power domains for CPUs, others have them for
>> devices as well.
>> As we always provide a virtual "always-on" power domain in the power domain
>> controller, all devices can refer to it using "power-domains" properties,
>> and the driver for the power domain controller can just forward the clock
>> domain operations to the clock driver.
>
> Okay, thanks for clarifying this.
>
> Thinking about this as bit more, when I realized that *if* we would
> add a new PM domain framework for explicit control of PM domains, that
> would mean you need to deploy support for that in the drivers.

Correct.  And we have to update DT bindings and DTS.

> On the other hand, as you anyway would need to change the drivers, you
> could instead deploy clock support in the drivers, which would avoid
> using the clock domain. In that way, you could still stay with one PM
> domain pointer per device, used to control the power domains instead.
> Right? Or would that have other implications?

That's exactly what we're doing already.
Which means that if you allow multiple entries in power-domains, we
have to change drivers, DT bindings, and DTS again (which we may
decide not to do ;-)

On SH/R-Mobile, we always did it that way, as the user manual had an
explicit "always-on" power domain.

On R-Car Gen2, the power domains contain CPU and L2 and GPU only,
so devices had their power-domains pointing to the clock controller.

On R-Car Gen3, some devices were moved into power domains, so we
generalized this by creating a virtual "always-on" power domain, and
letting all devices point their power-domains properties to the power
domain controller, which forwards clock handling to the clock controller.
For consistency, this was applied to R-Car Gen2 as well.

Cfr. some late relics fixed in e.g. commit 24b2d930a50662c1
('ARM: dts: r8a7794: Use SYSC "always-on" PM Domain for sound'),
but technically the fix was not needed, as it worked fine without.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Geert Uytterhoeven]

Hi Ulf,

On Wed, Apr 26, 2017 at 11:04 AM, Ulf Hansson  wrote:
> On 26 April 2017 at 10:06, Geert Uytterhoeven  wrote:
>> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
>>> However, we currently know about at least two different SoCs that need
>>> this. Perhaps we can extend the below list to justify adding a new
>>> framework/APIs. Something along the lines what you propose in $subject
>>> patchset.
>>>
>>> 1) Nvidia; to solve the USB super-speed host/device problem.
>>> 2) QCOM, which has pointed to several cases where the PM topology is
>>> laid out like devices having two PM domains..
>>> 3?) I don't fully remember - but I think Geert also pointed to some
>>> examples where a device could reside in a clock domain but also in
>>> power domain for a Renesas SoC!?
>>> 4) ?
>>
>> Most Renesas SoCs have module clocks, which we model as a clock domain.
>> Some Renesas SoCs have power domains for CPUs, others have them for
>> devices as well.
>> As we always provide a virtual "always-on" power domain in the power domain
>> controller, all devices can refer to it using "power-domains" properties,
>> and the driver for the power domain controller can just forward the clock
>> domain operations to the clock driver.
>
> Okay, thanks for clarifying this.
>
> Thinking about this as bit more, when I realized that *if* we would
> add a new PM domain framework for explicit control of PM domains, that
> would mean you need to deploy support for that in the drivers.

Correct.  And we have to update DT bindings and DTS.

> On the other hand, as you anyway would need to change the drivers, you
> could instead deploy clock support in the drivers, which would avoid
> using the clock domain. In that way, you could still stay with one PM
> domain pointer per device, used to control the power domains instead.
> Right? Or would that have other implications?

That's exactly what we're doing already.
Which means that if you allow multiple entries in power-domains, we
have to change drivers, DT bindings, and DTS again (which we may
decide not to do ;-)

On SH/R-Mobile, we always did it that way, as the user manual had an
explicit "always-on" power domain.

On R-Car Gen2, the power domains contain CPU and L2 and GPU only,
so devices had their power-domains pointing to the clock controller.

On R-Car Gen3, some devices were moved into power domains, so we
generalized this by creating a virtual "always-on" power domain, and
letting all devices point their power-domains properties to the power
domain controller, which forwards clock handling to the clock controller.
For consistency, this was applied to R-Car Gen2 as well.

Cfr. some late relics fixed in e.g. commit 24b2d930a50662c1
('ARM: dts: r8a7794: Use SYSC "always-on" PM Domain for sound'),
but technically the fix was not needed, as it worked fine without.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Ulf Hansson]

On 26 April 2017 at 10:06, Geert Uytterhoeven  wrote:
> Hi Ulf,
>
> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
>> However, we currently know about at least two different SoCs that need
>> this. Perhaps we can extend the below list to justify adding a new
>> framework/APIs. Something along the lines what you propose in $subject
>> patchset.
>>
>> 1) Nvidia; to solve the USB super-speed host/device problem.
>> 2) QCOM, which has pointed to several cases where the PM topology is
>> laid out like devices having two PM domains..
>> 3?) I don't fully remember - but I think Geert also pointed to some
>> examples where a device could reside in a clock domain but also in
>> power domain for a Renesas SoC!?
>> 4) ?
>
> Most Renesas SoCs have module clocks, which we model as a clock domain.
> Some Renesas SoCs have power domains for CPUs, others have them for
> devices as well.
> As we always provide a virtual "always-on" power domain in the power domain
> controller, all devices can refer to it using "power-domains" properties,
> and the driver for the power domain controller can just forward the clock
> domain operations to the clock driver.

Okay, thanks for clarifying this.

Thinking about this as bit more, when I realized that *if* we would
add a new PM domain framework for explicit control of PM domains, that
would mean you need to deploy support for that in the drivers.

On the other hand, as you anyway would need to change the drivers, you
could instead deploy clock support in the drivers, which would avoid
using the clock domain. In that way, you could still stay with one PM
domain pointer per device, used to control the power domains instead.
Right? Or would that have other implications?

Kind regards
Uffe

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Ulf Hansson]

On 26 April 2017 at 10:06, Geert Uytterhoeven  wrote:
> Hi Ulf,
>
> On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
>> However, we currently know about at least two different SoCs that need
>> this. Perhaps we can extend the below list to justify adding a new
>> framework/APIs. Something along the lines what you propose in $subject
>> patchset.
>>
>> 1) Nvidia; to solve the USB super-speed host/device problem.
>> 2) QCOM, which has pointed to several cases where the PM topology is
>> laid out like devices having two PM domains..
>> 3?) I don't fully remember - but I think Geert also pointed to some
>> examples where a device could reside in a clock domain but also in
>> power domain for a Renesas SoC!?
>> 4) ?
>
> Most Renesas SoCs have module clocks, which we model as a clock domain.
> Some Renesas SoCs have power domains for CPUs, others have them for
> devices as well.
> As we always provide a virtual "always-on" power domain in the power domain
> controller, all devices can refer to it using "power-domains" properties,
> and the driver for the power domain controller can just forward the clock
> domain operations to the clock driver.

Okay, thanks for clarifying this.

Thinking about this as bit more, when I realized that *if* we would
add a new PM domain framework for explicit control of PM domains, that
would mean you need to deploy support for that in the drivers.

On the other hand, as you anyway would need to change the drivers, you
could instead deploy clock support in the drivers, which would avoid
using the clock domain. In that way, you could still stay with one PM
domain pointer per device, used to control the power domains instead.
Right? Or would that have other implications?

Kind regards
Uffe

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Geert Uytterhoeven]

Hi Ulf,

On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
> However, we currently know about at least two different SoCs that need
> this. Perhaps we can extend the below list to justify adding a new
> framework/APIs. Something along the lines what you propose in $subject
> patchset.
>
> 1) Nvidia; to solve the USB super-speed host/device problem.
> 2) QCOM, which has pointed to several cases where the PM topology is
> laid out like devices having two PM domains..
> 3?) I don't fully remember - but I think Geert also pointed to some
> examples where a device could reside in a clock domain but also in
> power domain for a Renesas SoC!?
> 4) ?

Most Renesas SoCs have module clocks, which we model as a clock domain.
Some Renesas SoCs have power domains for CPUs, others have them for
devices as well.
As we always provide a virtual "always-on" power domain in the power domain
controller, all devices can refer to it using "power-domains" properties,
and the driver for the power domain controller can just forward the clock
domain operations to the clock driver.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Geert Uytterhoeven]

Hi Ulf,

On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
> However, we currently know about at least two different SoCs that need
> this. Perhaps we can extend the below list to justify adding a new
> framework/APIs. Something along the lines what you propose in $subject
> patchset.
>
> 1) Nvidia; to solve the USB super-speed host/device problem.
> 2) QCOM, which has pointed to several cases where the PM topology is
> laid out like devices having two PM domains..
> 3?) I don't fully remember - but I think Geert also pointed to some
> examples where a device could reside in a clock domain but also in
> power domain for a Renesas SoC!?
> 4) ?

Most Renesas SoCs have module clocks, which we model as a clock domain.
Some Renesas SoCs have power domains for CPUs, others have them for
devices as well.
As we always provide a virtual "always-on" power domain in the power domain
controller, all devices can refer to it using "power-domains" properties,
and the driver for the power domain controller can just forward the clock
domain operations to the clock driver.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Rafael J. Wysocki]

On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
> On 25 April 2017 at 13:13, Jon Hunter  wrote:
>>
>> On 28/03/17 15:13, Jon Hunter wrote:
>>> The current generic PM domain framework (GenDP) only allows a single
>>> PM domain to be associated with a given device. There are several
>>> use-cases for various system-on-chip devices where it is necessary for
>>> a PM domain consumer to control more than one PM domain where the PM
>>> domains:
>>> i).  Do not conform to a parent-child relationship so are not nested
>>> ii). May not be powered on and off at the same time so need independent
>>>  control.
>>>
>>> The solution proposed in this RFC is to allow consumers to explictly
>>> control PM domains, by getting a handle to a PM domain and explicitly
>>> making calls to power on and off the PM domain. Note that referencing
>>> counting is used to ensure that a PM domain shared between consumers
>>> is not powered off incorrectly.
>>>
>>> The Tegra124/210 XUSB subsystem (that consists of both host and device
>>> controllers) is an example of a consumer that needs to control more than
>>> one PM domain because the logic is partitioned across 3 PM domains which
>>> are:
>>> - XUSBA: Superspeed logic (for USB 3.0)
>>> - XUSBB: Device controller
>>> - XUSBC: Host controller
>>>
>>> These power domains are not nested and can be powered-up and down
>>> independently of one another. In practice different scenarios require
>>> different combinations of the power domains, for example:
>>> - Superspeed host: XUSBA and XUSBC
>>> - Superspeed device: XUSBA and XUSBB
>>>
>>> Although it could be possible to logically nest both the XUSBB and XUSBC
>>> domains under the XUSBA, superspeed may not always be used/required and
>>> so this would keep it on unnecessarily.
>>>
>>> Given that Tegra uses device-tree for describing the hardware, it would
>>> be ideal that the device-tree 'power-domains' property for generic PM
>>> domains could be extended to allow more than one PM domain to be
>>> specified. For example, define the following the Tegra210 xHCI device ...
>>>
>>>   usb@7009 {
>>>   compatible = "nvidia,tegra210-xusb";
>>>   ...
>>>   power-domains = <_xusbhost>, <_xusbss>;
>>>   power-domain-names = "host", "superspeed";
>>>   };
>>>
>>> This RFC extends the generic PM domain framework to allow a device to
>>> define more than one PM domain in the device-tree 'power-domains'
>>> property. If there is more than one then the assumption is that these
>>> PM domains will be controlled explicitly by the consumer and the device
>>> will not be automatically bound to any PM domain.
>>
>> Any more comments/inputs on this? I can address Rajendra's feedback, but
>> before I did I wanted to see if this is along the right lines or not?
>
> I discussed this with Rafael at the OSPM summit in Pisa a couple of
> weeks ago. Apologize for the delay in providing additional feedback.
>
> First, whether the problem is really rare, perhaps adding a new
> API/framework can't be justified - then it may be better to add some
> kind of aggregation layer on top of the current PM domain
> infrastructure (something along the first attempt you made for genpd).
> That was kind of Rafael's thoughts (Rafael, please correct me if I am
> wrong).

We were talking about the original idea behind the pm_domain pointer
concept, which was about adding a set of PM operations above the bus
type/class layer, which could be used for intercepting bus-type PM
operations and providing some common handling above them.  This is
still relevant IMO.

The basic observation here is that the PM core takes only one set of
PM operation per device into account and therefore, in every stage of
system suspend, for example, the callback invoked by it has to take
care of all actions that need to be carried out for the given device,
possibly by invoking callbacks from other code layers.  That
limitation cannot be removed easily, because it is built into the PM
core design quite fundamentally.

However, this series seems to be about controlling power resources
represented by power domain objects rather than about PM operations.
In ACPI there is a power resource concept which seems to be quite
similar to this, so it is not entirely new. :-)

Of course, question is whether or not to extend genpd this way and I'm
not really sure.  I actually probably wouldn't do that, because
poweron/poweroff operations used by genpd can be implemeted in terms
of lower-level power resource control and I don't see the reason for
mixing the two in one framework.

> However, we currently know about at least two different SoCs that need
> this. Perhaps we can extend the below list to justify adding a new
> framework/APIs. Something along the lines what you propose in $subject
> patchset.
>
> 1) Nvidia; to solve the USB super-speed host/device problem.
> 2) QCOM, which has pointed to several 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Rafael J. Wysocki]

On Tue, Apr 25, 2017 at 9:34 PM, Ulf Hansson  wrote:
> On 25 April 2017 at 13:13, Jon Hunter  wrote:
>>
>> On 28/03/17 15:13, Jon Hunter wrote:
>>> The current generic PM domain framework (GenDP) only allows a single
>>> PM domain to be associated with a given device. There are several
>>> use-cases for various system-on-chip devices where it is necessary for
>>> a PM domain consumer to control more than one PM domain where the PM
>>> domains:
>>> i).  Do not conform to a parent-child relationship so are not nested
>>> ii). May not be powered on and off at the same time so need independent
>>>  control.
>>>
>>> The solution proposed in this RFC is to allow consumers to explictly
>>> control PM domains, by getting a handle to a PM domain and explicitly
>>> making calls to power on and off the PM domain. Note that referencing
>>> counting is used to ensure that a PM domain shared between consumers
>>> is not powered off incorrectly.
>>>
>>> The Tegra124/210 XUSB subsystem (that consists of both host and device
>>> controllers) is an example of a consumer that needs to control more than
>>> one PM domain because the logic is partitioned across 3 PM domains which
>>> are:
>>> - XUSBA: Superspeed logic (for USB 3.0)
>>> - XUSBB: Device controller
>>> - XUSBC: Host controller
>>>
>>> These power domains are not nested and can be powered-up and down
>>> independently of one another. In practice different scenarios require
>>> different combinations of the power domains, for example:
>>> - Superspeed host: XUSBA and XUSBC
>>> - Superspeed device: XUSBA and XUSBB
>>>
>>> Although it could be possible to logically nest both the XUSBB and XUSBC
>>> domains under the XUSBA, superspeed may not always be used/required and
>>> so this would keep it on unnecessarily.
>>>
>>> Given that Tegra uses device-tree for describing the hardware, it would
>>> be ideal that the device-tree 'power-domains' property for generic PM
>>> domains could be extended to allow more than one PM domain to be
>>> specified. For example, define the following the Tegra210 xHCI device ...
>>>
>>>   usb@7009 {
>>>   compatible = "nvidia,tegra210-xusb";
>>>   ...
>>>   power-domains = <_xusbhost>, <_xusbss>;
>>>   power-domain-names = "host", "superspeed";
>>>   };
>>>
>>> This RFC extends the generic PM domain framework to allow a device to
>>> define more than one PM domain in the device-tree 'power-domains'
>>> property. If there is more than one then the assumption is that these
>>> PM domains will be controlled explicitly by the consumer and the device
>>> will not be automatically bound to any PM domain.
>>
>> Any more comments/inputs on this? I can address Rajendra's feedback, but
>> before I did I wanted to see if this is along the right lines or not?
>
> I discussed this with Rafael at the OSPM summit in Pisa a couple of
> weeks ago. Apologize for the delay in providing additional feedback.
>
> First, whether the problem is really rare, perhaps adding a new
> API/framework can't be justified - then it may be better to add some
> kind of aggregation layer on top of the current PM domain
> infrastructure (something along the first attempt you made for genpd).
> That was kind of Rafael's thoughts (Rafael, please correct me if I am
> wrong).

We were talking about the original idea behind the pm_domain pointer
concept, which was about adding a set of PM operations above the bus
type/class layer, which could be used for intercepting bus-type PM
operations and providing some common handling above them.  This is
still relevant IMO.

The basic observation here is that the PM core takes only one set of
PM operation per device into account and therefore, in every stage of
system suspend, for example, the callback invoked by it has to take
care of all actions that need to be carried out for the given device,
possibly by invoking callbacks from other code layers.  That
limitation cannot be removed easily, because it is built into the PM
core design quite fundamentally.

However, this series seems to be about controlling power resources
represented by power domain objects rather than about PM operations.
In ACPI there is a power resource concept which seems to be quite
similar to this, so it is not entirely new. :-)

Of course, question is whether or not to extend genpd this way and I'm
not really sure.  I actually probably wouldn't do that, because
poweron/poweroff operations used by genpd can be implemeted in terms
of lower-level power resource control and I don't see the reason for
mixing the two in one framework.

> However, we currently know about at least two different SoCs that need
> this. Perhaps we can extend the below list to justify adding a new
> framework/APIs. Something along the lines what you propose in $subject
> patchset.
>
> 1) Nvidia; to solve the USB super-speed host/device problem.
> 2) QCOM, which has pointed to several cases where the PM topology is
> laid out like 

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Ulf Hansson]

On 25 April 2017 at 13:13, Jon Hunter  wrote:
>
> On 28/03/17 15:13, Jon Hunter wrote:
>> The current generic PM domain framework (GenDP) only allows a single
>> PM domain to be associated with a given device. There are several
>> use-cases for various system-on-chip devices where it is necessary for
>> a PM domain consumer to control more than one PM domain where the PM
>> domains:
>> i).  Do not conform to a parent-child relationship so are not nested
>> ii). May not be powered on and off at the same time so need independent
>>  control.
>>
>> The solution proposed in this RFC is to allow consumers to explictly
>> control PM domains, by getting a handle to a PM domain and explicitly
>> making calls to power on and off the PM domain. Note that referencing
>> counting is used to ensure that a PM domain shared between consumers
>> is not powered off incorrectly.
>>
>> The Tegra124/210 XUSB subsystem (that consists of both host and device
>> controllers) is an example of a consumer that needs to control more than
>> one PM domain because the logic is partitioned across 3 PM domains which
>> are:
>> - XUSBA: Superspeed logic (for USB 3.0)
>> - XUSBB: Device controller
>> - XUSBC: Host controller
>>
>> These power domains are not nested and can be powered-up and down
>> independently of one another. In practice different scenarios require
>> different combinations of the power domains, for example:
>> - Superspeed host: XUSBA and XUSBC
>> - Superspeed device: XUSBA and XUSBB
>>
>> Although it could be possible to logically nest both the XUSBB and XUSBC
>> domains under the XUSBA, superspeed may not always be used/required and
>> so this would keep it on unnecessarily.
>>
>> Given that Tegra uses device-tree for describing the hardware, it would
>> be ideal that the device-tree 'power-domains' property for generic PM
>> domains could be extended to allow more than one PM domain to be
>> specified. For example, define the following the Tegra210 xHCI device ...
>>
>>   usb@7009 {
>>   compatible = "nvidia,tegra210-xusb";
>>   ...
>>   power-domains = <_xusbhost>, <_xusbss>;
>>   power-domain-names = "host", "superspeed";
>>   };
>>
>> This RFC extends the generic PM domain framework to allow a device to
>> define more than one PM domain in the device-tree 'power-domains'
>> property. If there is more than one then the assumption is that these
>> PM domains will be controlled explicitly by the consumer and the device
>> will not be automatically bound to any PM domain.
>
> Any more comments/inputs on this? I can address Rajendra's feedback, but
> before I did I wanted to see if this is along the right lines or not?

I discussed this with Rafael at the OSPM summit in Pisa a couple of
weeks ago. Apologize for the delay in providing additional feedback.

First, whether the problem is really rare, perhaps adding a new
API/framework can't be justified - then it may be better to add some
kind of aggregation layer on top of the current PM domain
infrastructure (something along the first attempt you made for genpd).
That was kind of Rafael's thoughts (Rafael, please correct me if I am
wrong).

However, we currently know about at least two different SoCs that need
this. Perhaps we can extend the below list to justify adding a new
framework/APIs. Something along the lines what you propose in $subject
patchset.

1) Nvidia; to solve the USB super-speed host/device problem.
2) QCOM, which has pointed to several cases where the PM topology is
laid out like devices having two PM domains..
3?) I don't fully remember - but I think Geert also pointed to some
examples where a device could reside in a clock domain but also in
power domain for a Renesas SoC!?
4) ?

Moreover, perhaps this could also be useful for SoCs using ACPI? If
so, we shouldn't tie this immediately to genpd, but in the layer above
genpd/ACPI PM domain. Something like dev_pm_domain_get(), which would
be implemented similar as the dev_pm_domain_attach() (which calls both
the ACPI PM domain and genpd to find a match).

Kind regards
Uffe

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Ulf Hansson]

On 25 April 2017 at 13:13, Jon Hunter  wrote:
>
> On 28/03/17 15:13, Jon Hunter wrote:
>> The current generic PM domain framework (GenDP) only allows a single
>> PM domain to be associated with a given device. There are several
>> use-cases for various system-on-chip devices where it is necessary for
>> a PM domain consumer to control more than one PM domain where the PM
>> domains:
>> i).  Do not conform to a parent-child relationship so are not nested
>> ii). May not be powered on and off at the same time so need independent
>>  control.
>>
>> The solution proposed in this RFC is to allow consumers to explictly
>> control PM domains, by getting a handle to a PM domain and explicitly
>> making calls to power on and off the PM domain. Note that referencing
>> counting is used to ensure that a PM domain shared between consumers
>> is not powered off incorrectly.
>>
>> The Tegra124/210 XUSB subsystem (that consists of both host and device
>> controllers) is an example of a consumer that needs to control more than
>> one PM domain because the logic is partitioned across 3 PM domains which
>> are:
>> - XUSBA: Superspeed logic (for USB 3.0)
>> - XUSBB: Device controller
>> - XUSBC: Host controller
>>
>> These power domains are not nested and can be powered-up and down
>> independently of one another. In practice different scenarios require
>> different combinations of the power domains, for example:
>> - Superspeed host: XUSBA and XUSBC
>> - Superspeed device: XUSBA and XUSBB
>>
>> Although it could be possible to logically nest both the XUSBB and XUSBC
>> domains under the XUSBA, superspeed may not always be used/required and
>> so this would keep it on unnecessarily.
>>
>> Given that Tegra uses device-tree for describing the hardware, it would
>> be ideal that the device-tree 'power-domains' property for generic PM
>> domains could be extended to allow more than one PM domain to be
>> specified. For example, define the following the Tegra210 xHCI device ...
>>
>>   usb@7009 {
>>   compatible = "nvidia,tegra210-xusb";
>>   ...
>>   power-domains = <_xusbhost>, <_xusbss>;
>>   power-domain-names = "host", "superspeed";
>>   };
>>
>> This RFC extends the generic PM domain framework to allow a device to
>> define more than one PM domain in the device-tree 'power-domains'
>> property. If there is more than one then the assumption is that these
>> PM domains will be controlled explicitly by the consumer and the device
>> will not be automatically bound to any PM domain.
>
> Any more comments/inputs on this? I can address Rajendra's feedback, but
> before I did I wanted to see if this is along the right lines or not?

I discussed this with Rafael at the OSPM summit in Pisa a couple of
weeks ago. Apologize for the delay in providing additional feedback.

First, whether the problem is really rare, perhaps adding a new
API/framework can't be justified - then it may be better to add some
kind of aggregation layer on top of the current PM domain
infrastructure (something along the first attempt you made for genpd).
That was kind of Rafael's thoughts (Rafael, please correct me if I am
wrong).

However, we currently know about at least two different SoCs that need
this. Perhaps we can extend the below list to justify adding a new
framework/APIs. Something along the lines what you propose in $subject
patchset.

1) Nvidia; to solve the USB super-speed host/device problem.
2) QCOM, which has pointed to several cases where the PM topology is
laid out like devices having two PM domains..
3?) I don't fully remember - but I think Geert also pointed to some
examples where a device could reside in a clock domain but also in
power domain for a Renesas SoC!?
4) ?

Moreover, perhaps this could also be useful for SoCs using ACPI? If
so, we shouldn't tie this immediately to genpd, but in the layer above
genpd/ACPI PM domain. Something like dev_pm_domain_get(), which would
be implemented similar as the dev_pm_domain_attach() (which calls both
the ACPI PM domain and genpd to find a match).

Kind regards
Uffe

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 28/03/17 15:13, Jon Hunter wrote:
> The current generic PM domain framework (GenDP) only allows a single
> PM domain to be associated with a given device. There are several
> use-cases for various system-on-chip devices where it is necessary for
> a PM domain consumer to control more than one PM domain where the PM
> domains:
> i).  Do not conform to a parent-child relationship so are not nested
> ii). May not be powered on and off at the same time so need independent
>  control.
> 
> The solution proposed in this RFC is to allow consumers to explictly
> control PM domains, by getting a handle to a PM domain and explicitly
> making calls to power on and off the PM domain. Note that referencing
> counting is used to ensure that a PM domain shared between consumers
> is not powered off incorrectly.
> 
> The Tegra124/210 XUSB subsystem (that consists of both host and device
> controllers) is an example of a consumer that needs to control more than
> one PM domain because the logic is partitioned across 3 PM domains which
> are:
> - XUSBA: Superspeed logic (for USB 3.0)
> - XUSBB: Device controller
> - XUSBC: Host controller
> 
> These power domains are not nested and can be powered-up and down
> independently of one another. In practice different scenarios require
> different combinations of the power domains, for example:
> - Superspeed host: XUSBA and XUSBC
> - Superspeed device: XUSBA and XUSBB
> 
> Although it could be possible to logically nest both the XUSBB and XUSBC
> domains under the XUSBA, superspeed may not always be used/required and
> so this would keep it on unnecessarily.
> 
> Given that Tegra uses device-tree for describing the hardware, it would
> be ideal that the device-tree 'power-domains' property for generic PM
> domains could be extended to allow more than one PM domain to be
> specified. For example, define the following the Tegra210 xHCI device ...
> 
>   usb@7009 {
>   compatible = "nvidia,tegra210-xusb";
>   ...
>   power-domains = <_xusbhost>, <_xusbss>;
>   power-domain-names = "host", "superspeed";
>   };
> 
> This RFC extends the generic PM domain framework to allow a device to
> define more than one PM domain in the device-tree 'power-domains'
> property. If there is more than one then the assumption is that these
> PM domains will be controlled explicitly by the consumer and the device
> will not be automatically bound to any PM domain.

Any more comments/inputs on this? I can address Rajendra's feedback, but
before I did I wanted to see if this is along the right lines or not?

Cheers
Jon

-- 
nvpublic

Re: [RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

On 28/03/17 15:13, Jon Hunter wrote:
> The current generic PM domain framework (GenDP) only allows a single
> PM domain to be associated with a given device. There are several
> use-cases for various system-on-chip devices where it is necessary for
> a PM domain consumer to control more than one PM domain where the PM
> domains:
> i).  Do not conform to a parent-child relationship so are not nested
> ii). May not be powered on and off at the same time so need independent
>  control.
> 
> The solution proposed in this RFC is to allow consumers to explictly
> control PM domains, by getting a handle to a PM domain and explicitly
> making calls to power on and off the PM domain. Note that referencing
> counting is used to ensure that a PM domain shared between consumers
> is not powered off incorrectly.
> 
> The Tegra124/210 XUSB subsystem (that consists of both host and device
> controllers) is an example of a consumer that needs to control more than
> one PM domain because the logic is partitioned across 3 PM domains which
> are:
> - XUSBA: Superspeed logic (for USB 3.0)
> - XUSBB: Device controller
> - XUSBC: Host controller
> 
> These power domains are not nested and can be powered-up and down
> independently of one another. In practice different scenarios require
> different combinations of the power domains, for example:
> - Superspeed host: XUSBA and XUSBC
> - Superspeed device: XUSBA and XUSBB
> 
> Although it could be possible to logically nest both the XUSBB and XUSBC
> domains under the XUSBA, superspeed may not always be used/required and
> so this would keep it on unnecessarily.
> 
> Given that Tegra uses device-tree for describing the hardware, it would
> be ideal that the device-tree 'power-domains' property for generic PM
> domains could be extended to allow more than one PM domain to be
> specified. For example, define the following the Tegra210 xHCI device ...
> 
>   usb@7009 {
>   compatible = "nvidia,tegra210-xusb";
>   ...
>   power-domains = <_xusbhost>, <_xusbss>;
>   power-domain-names = "host", "superspeed";
>   };
> 
> This RFC extends the generic PM domain framework to allow a device to
> define more than one PM domain in the device-tree 'power-domains'
> property. If there is more than one then the assumption is that these
> PM domains will be controlled explicitly by the consumer and the device
> will not be automatically bound to any PM domain.

Any more comments/inputs on this? I can address Rajendra's feedback, but
before I did I wanted to see if this is along the right lines or not?

Cheers
Jon

-- 
nvpublic

[RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

The current generic PM domain framework (GenDP) only allows a single
PM domain to be associated with a given device. There are several
use-cases for various system-on-chip devices where it is necessary for
a PM domain consumer to control more than one PM domain where the PM
domains:
i).  Do not conform to a parent-child relationship so are not nested
ii). May not be powered on and off at the same time so need independent
 control.

The solution proposed in this RFC is to allow consumers to explictly
control PM domains, by getting a handle to a PM domain and explicitly
making calls to power on and off the PM domain. Note that referencing
counting is used to ensure that a PM domain shared between consumers
is not powered off incorrectly.

The Tegra124/210 XUSB subsystem (that consists of both host and device
controllers) is an example of a consumer that needs to control more than
one PM domain because the logic is partitioned across 3 PM domains which
are:
- XUSBA: Superspeed logic (for USB 3.0)
- XUSBB: Device controller
- XUSBC: Host controller

These power domains are not nested and can be powered-up and down
independently of one another. In practice different scenarios require
different combinations of the power domains, for example:
- Superspeed host: XUSBA and XUSBC
- Superspeed device: XUSBA and XUSBB

Although it could be possible to logically nest both the XUSBB and XUSBC
domains under the XUSBA, superspeed may not always be used/required and
so this would keep it on unnecessarily.

Given that Tegra uses device-tree for describing the hardware, it would
be ideal that the device-tree 'power-domains' property for generic PM
domains could be extended to allow more than one PM domain to be
specified. For example, define the following the Tegra210 xHCI device ...

usb@7009 {
compatible = "nvidia,tegra210-xusb";
...
power-domains = <_xusbhost>, <_xusbss>;
power-domain-names = "host", "superspeed";
};

This RFC extends the generic PM domain framework to allow a device to
define more than one PM domain in the device-tree 'power-domains'
property. If there is more than one then the assumption is that these
PM domains will be controlled explicitly by the consumer and the device
will not be automatically bound to any PM domain.

This RFC is a follow-up to the following RFC but because it is a
completely different approach has not been titled V2.

https://lkml.org/lkml/2016/9/20/173

Jon Hunter (4):
  PM / Domains: Prepare for supporting explicit PM domain control
  PM / Domains: Add support for explicit control of PM domains
  PM / Domains: Add OF helpers for getting PM domains
  dt-bindings: Add support for devices with multiple PM domains

 .../devicetree/bindings/power/power_domain.txt |  11 +-
 drivers/base/power/domain.c| 203 -
 include/linux/pm_domain.h  |  35 
 3 files changed, 246 insertions(+), 3 deletions(-)

-- 
2.7.4

[RFC PATCH 0/4] PM / Domains: Add support for explicit control of PM domains

[Jon Hunter]

The current generic PM domain framework (GenDP) only allows a single
PM domain to be associated with a given device. There are several
use-cases for various system-on-chip devices where it is necessary for
a PM domain consumer to control more than one PM domain where the PM
domains:
i).  Do not conform to a parent-child relationship so are not nested
ii). May not be powered on and off at the same time so need independent
 control.

The solution proposed in this RFC is to allow consumers to explictly
control PM domains, by getting a handle to a PM domain and explicitly
making calls to power on and off the PM domain. Note that referencing
counting is used to ensure that a PM domain shared between consumers
is not powered off incorrectly.

The Tegra124/210 XUSB subsystem (that consists of both host and device
controllers) is an example of a consumer that needs to control more than
one PM domain because the logic is partitioned across 3 PM domains which
are:
- XUSBA: Superspeed logic (for USB 3.0)
- XUSBB: Device controller
- XUSBC: Host controller

These power domains are not nested and can be powered-up and down
independently of one another. In practice different scenarios require
different combinations of the power domains, for example:
- Superspeed host: XUSBA and XUSBC
- Superspeed device: XUSBA and XUSBB

Although it could be possible to logically nest both the XUSBB and XUSBC
domains under the XUSBA, superspeed may not always be used/required and
so this would keep it on unnecessarily.

Given that Tegra uses device-tree for describing the hardware, it would
be ideal that the device-tree 'power-domains' property for generic PM
domains could be extended to allow more than one PM domain to be
specified. For example, define the following the Tegra210 xHCI device ...

usb@7009 {
compatible = "nvidia,tegra210-xusb";
...
power-domains = <_xusbhost>, <_xusbss>;
power-domain-names = "host", "superspeed";
};

This RFC extends the generic PM domain framework to allow a device to
define more than one PM domain in the device-tree 'power-domains'
property. If there is more than one then the assumption is that these
PM domains will be controlled explicitly by the consumer and the device
will not be automatically bound to any PM domain.

This RFC is a follow-up to the following RFC but because it is a
completely different approach has not been titled V2.

https://lkml.org/lkml/2016/9/20/173

Jon Hunter (4):
  PM / Domains: Prepare for supporting explicit PM domain control
  PM / Domains: Add support for explicit control of PM domains
  PM / Domains: Add OF helpers for getting PM domains
  dt-bindings: Add support for devices with multiple PM domains

 .../devicetree/bindings/power/power_domain.txt |  11 +-
 drivers/base/power/domain.c| 203 -
 include/linux/pm_domain.h  |  35 
 3 files changed, 246 insertions(+), 3 deletions(-)

-- 
2.7.4