[PATCH 00/21] On-demand device registration
On Mon, Jun 22, 2015 at 10:23 AM, Tomeu Vizoso wrote: > On 28 May 2015 at 06:33, Rob Herring wrote: >> On Mon, May 25, 2015 at 9:53 AM, Tomeu Vizoso >> wrote: >>> Hello, >>> >>> I have a problem with the panel on my Tegra Chromebook taking longer than >>> expected to be ready during boot (Stéphane Marchesin reported what is >>> basically the same issue in [0]), and have looked into ordered probing as a >>> better way of solving this than moving nodes around in the DT or playing >>> with >>> initcall levels. >>> >>> While reading the thread [1] that Alexander Holler started with his series >>> to >>> make probing order deterministic, it occurred to me that it should be >>> possible >>> to achieve the same by registering devices as they are referenced by other >>> devices. >> >> I like the concept and novel approach. >> >>> This basically reuses the information that is already implicit in the >>> probe() >>> implementations, saving us from refactoring existing drivers or adding >>> information to DTBs. >>> >>> Something I'm not completely happy with is that I have had to move the call >>> to >>> of_platform_populate after all platform drivers have been registered. >>> Otherwise I don't see how I could register drivers on demand as we don't >>> have >>> yet each driver's compatible strings. >> >> Yeah, this is the opposite of what we'd really like. > > Can you elaborate on the reasons why we would like to have devices > registered before built-in drivers finish registering, even if we > don't probe them yet? My main thought was for modules we will almost always have devices appearing first. More generally, we can have devices and drivers coming or going at any point in time and should not put restrictions on ordering. Also, I think all the probe ordering and dependency tracking should be done within the driver core (i.e. dependencies are a list of struct devices). At some level it has to become firmware specific, but we want to minimize that part. I could be convinced otherwise and you have put more thought into this problem than I have. >> Ideally, we would >> have a solution that works for modules too. However, we're no worse >> off. We pretty much build-in dependencies to avoid module ordering >> problems. > > Nod, I haven't looked yet at requesting modules on-demand, but I guess > it should be doable. Modules that have dependencies described in the > firmware should get them probed automatically already though. > >> Perhaps we need to make the probing on-demand rather than simply on >> device<->driver match occurring. > > I'm afraid that too much old code depends on that. For example, Rafael > pointed out to the PNP subsystem, which registers a driver that will > probe devices with the EISA ID PNP0c02 to reserve memory regions for > devices that will be probed later. > > http://lxr.free-electrons.com/source/drivers/pnp/system.c > > My understanding is that probing of PNP0c02 devices must happen before > the actual devices that depend on those regions are probed, so if we > decoupled the probing from the driver/device registration, we would be > breaking that assumption. That shouldn't matter as PNP matching is PNP specific. We already have different ways of matching with device/driver name or of_match_table for example. Changing how and when OF matching occurs would not affect PNP matching. We do matching on device and driver add currently. For the "when" part, we would need to add what I'll call async matching or deferred matching which in addition to matching on the of_match_table also matches on the dependency list having probed. Your last series essentially does this, but the difference is yours is not OF specific and I think it needs to be. I mean it is OF specific only in the aspect that matching already is. From a driver and subsystem standpoint, it should not be OF specific much like deferred probe is not OF specific, but in reality only occurs (currently) on OF probed drivers. >>> For machs that don't move of_platform_populate() to a later point, these >>> patches shouldn't cause any problems but it's not guaranteed that we'll >>> avoid >>> all the deferred probes as some drivers may not be registered yet. >> >> Ideally, of_platform_populate is not explicitly called by each >> platform. So I think we need to make this work for the default case. > > The problem is that some platforms will need fixing because some > initcalls assume that some devices will have been registered already, > or even probed. I think removing those assumptions shouldn't be > problematic because I haven't had much trouble with this on the four > platforms I have tested with, but I cannot test every board that is > supported upstream. > > I can ask though the KernelCI folks to boot my branch in all their > boards and make sure that those work when of_platform_populate is > called in late_initcall. I'd imagine Kevin would be happy to. That is still a subset of h/w, so we'd need a way to disable any
[PATCH 00/21] On-demand device registration
On 28 May 2015 at 06:33, Rob Herring wrote: > On Mon, May 25, 2015 at 9:53 AM, Tomeu Vizoso > wrote: >> Hello, >> >> I have a problem with the panel on my Tegra Chromebook taking longer than >> expected to be ready during boot (Stéphane Marchesin reported what is >> basically the same issue in [0]), and have looked into ordered probing as a >> better way of solving this than moving nodes around in the DT or playing with >> initcall levels. >> >> While reading the thread [1] that Alexander Holler started with his series to >> make probing order deterministic, it occurred to me that it should be >> possible >> to achieve the same by registering devices as they are referenced by other >> devices. > > I like the concept and novel approach. > >> This basically reuses the information that is already implicit in the probe() >> implementations, saving us from refactoring existing drivers or adding >> information to DTBs. >> >> Something I'm not completely happy with is that I have had to move the call >> to >> of_platform_populate after all platform drivers have been registered. >> Otherwise I don't see how I could register drivers on demand as we don't have >> yet each driver's compatible strings. > > Yeah, this is the opposite of what we'd really like. Can you elaborate on the reasons why we would like to have devices registered before built-in drivers finish registering, even if we don't probe them yet? > Ideally, we would > have a solution that works for modules too. However, we're no worse > off. We pretty much build-in dependencies to avoid module ordering > problems. Nod, I haven't looked yet at requesting modules on-demand, but I guess it should be doable. Modules that have dependencies described in the firmware should get them probed automatically already though. > Perhaps we need to make the probing on-demand rather than simply on > device<->driver match occurring. I'm afraid that too much old code depends on that. For example, Rafael pointed out to the PNP subsystem, which registers a driver that will probe devices with the EISA ID PNP0c02 to reserve memory regions for devices that will be probed later. http://lxr.free-electrons.com/source/drivers/pnp/system.c My understanding is that probing of PNP0c02 devices must happen before the actual devices that depend on those regions are probed, so if we decoupled the probing from the driver/device registration, we would be breaking that assumption. >> For machs that don't move of_platform_populate() to a later point, these >> patches shouldn't cause any problems but it's not guaranteed that we'll avoid >> all the deferred probes as some drivers may not be registered yet. > > Ideally, of_platform_populate is not explicitly called by each > platform. So I think we need to make this work for the default case. The problem is that some platforms will need fixing because some initcalls assume that some devices will have been registered already, or even probed. I think removing those assumptions shouldn't be problematic because I haven't had much trouble with this on the four platforms I have tested with, but I cannot test every board that is supported upstream. I can ask though the KernelCI folks to boot my branch in all their boards and make sure that those work when of_platform_populate is called in late_initcall. http://kernelci.org/boot/all/job/next/kernel/next-20150619/ >> I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these >> patches were enough to eliminate all the deferred probes. >> >> With this series I get the kernel to output to the panel in 0.5s, instead of >> 2.8s. > > That's certainly compelling. Have to say that those numbers are with the serial console enabled (without, it's 0.5s vs 1.5s), but on machines that take longer to boot we should see bigger gains because we won't be sending devices to the end of the queue when their probe is deferred. Regards, Tomeu > Rob > >> >> Regards, >> >> Tomeu >> >> [0] http://lists.freedesktop.org/archives/dri-devel/2014-August/066527.html >> >> [1] https://lkml.org/lkml/2014/5/12/452 >> >> Tomeu Vizoso (21): >> regulator: core: Reduce critical area in _regulator_get >> ARM: tegra: Add gpio-ranges property >> ARM: tegra: Register drivers before devices >> ARM: EXYNOS: Register drivers before devices >> ARM i.MX6q: Register drivers before devices >> of/platform: Add of_platform_device_ensure() >> of/platform: Ensure device registration on lookup >> gpio: Probe GPIO drivers on demand >> gpio: Probe pinctrl devices on demand >> regulator: core: Probe regulators on demand >> drm: Probe panels on demand >> drm/tegra: Probe dpaux devices on demand >> i2c: core: Probe i2c master devices on demand >> pwm: Probe PWM chip devices on demand >> backlight: Probe backlight devices on demand >> usb: phy: Probe phy devices on demand >> clk: Probe clk providers on demand >> pinctrl: Probe pinctrl devices on demand >> phy: core: Probe phy providers
[PATCH 00/21] On-demand device registration
Am 15.06.2015 um 10:58 schrieb Linus Walleij: > On Sat, Jun 13, 2015 at 8:27 PM, Alexander Holler > wrote: > >> And because you've said that "problem space is a bit convoluted" and I >> disagree, here's a summary from my point of view: >> >> 1. All the necessary information (dependencies between drivers) already >> exists at compile time. The set of dependencies between drivers might become >> smaller by configuration, but will not become larger. So there should be NO >> need to collect them at runtime, e.g. by instrumenting function calls. > > I think you arrived at the core of the crux here. I've hoped so, that's why I've written it. > I guess your suggested approach then need to introduce a special > build tool to order the initcalls accordingly. > > Again this will fall short if you don't know at compile time exactly > *which* board file will be executed. I've just tried to describe the facts in order to make the problem space more clear, because, as said, I don't think it's convoluted. Besides that, I didn't want to suggest anything else other than what I've already posted working patches for. What I've mentioned as possible other solutions above is stuff which might be possible too in order to give some starting points for people which are searching another solution. But I wouldn't have written my patches as they are, if I would think there is another more easier solution. And of course, there is still a bit to resolve at runtime, even in the DT case (look at the "compatible" attribute). But there is already a runtime solution to find the right driver (in case of DT) and I haven't mentioned it in order to no confuse people again. Mentioning every little detail doesn't make sense if you want to describe something understandable (which is what I've tried). > So the only practical way to solve this at compile time is to predict > an initcall ordering sequence for all possible boot paths, compile in > all of them, and choose the right one at boot. But the number of boot > paths is equal to the number of device trees / ACPI tables or > board files supported, and that space is uncontrolled and ordered > infinite. You just need one working ordered sequence which includes all options. This one will work for all others too. > Basically I think the root problem with your approach is that you > assume we know what hardware we will boot on at compile time. We Totally wrong. If you assume that I assume this, than either I was totally unable to describe something clearly, or you were unable or unwilling to understand what I've written. And as the result is the same, we don't need to find out which was reason. Anyway, have fun. I'm quitting the discussion here as I don't have any business with the kernel and already decided some time again to not post patches anymore as it seems to be a waste of my (and maybe others) time. Regards, Alexander Holler
[PATCH 00/21] On-demand device registration
On Sat, Jun 13, 2015 at 8:27 PM, Alexander Holler wrote: > And because you've said that "problem space is a bit convoluted" and I > disagree, here's a summary from my point of view: > > 1. All the necessary information (dependencies between drivers) already > exists at compile time. The set of dependencies between drivers might become > smaller by configuration, but will not become larger. So there should be NO > need to collect them at runtime, e.g. by instrumenting function calls. I think you arrived at the core of the crux here. When we look up a resource provided from another driver, e.g. from regulator_get(), clk_get(), pinctrl_get(), gpiod_get() etc - the dependency is resolved by looking in a cross-reference table for either a struct device* pointer or a string, an index, or both or all three. Examples: struct regulator *regulator_get(struct device *dev, const char *id); struct clk *clk_get(struct device *dev, const char *id); struct gpio_desc *__must_check __gpiod_get(struct device *dev, const char *con_id, enum gpiod_flags flags); (...) (*_index() variants exist on some of the resource retrieveal functions.) struct device * is the device requesting the resource, con_id is the string name of the resource on the provider side. This is all solved by looking in cross reference tables. ONE way of resolving that cross reference is to look into the device tree or the ACPI table. But for the board file case, this is resolved at runtime by the cross reference table, registered with calls such as gpiod_add_lookup_table(). It is true that in the theoretical sense, all of this exist at compile time especially if you can parse something like a device tree and figure out what struct device * nodes will correspond to the struct device_node:s in it. For ACPI I guess a similar procedure is viable. Problem: this requires the kernel compile to know exactly *which* device tree or ACPI table it is going to boot on. The expressed goal of device tree and ACPI is to have *ONE* kernel booting several device trees. Here your approach stops short: you are suggesting instrumenting the kernel at compile time to one single device tree or ACPI table. But we never know really what device tree or ACPI table will be used. This just cannot be done at compile time for that reason alone. Example: in boot case (A) the regulator may be provided by regulator "foo" driver on an i2c bus. But in boot case (B) the very same regulator may be provided by regulator "bar" on an SPI bus. These are very real usecases, for example for drivers/net/ethernet/smsc/smsc911x.c, will get regulators from the most diverse places depending on what device tree is used. For board files, it is neither possible in theory: you need to compile the code to figure out the struct device * provider, and/or the string name of the providing device (.name field in struct device for the provider) to resolve dependencies at compile time. For the board file case, resolving dependencies at compile time will require a quite complex two-stage rocket: compile the code to get resources out, then recompile with known resources. I guess your suggested approach then need to introduce a special build tool to order the initcalls accordingly. Again this will fall short if you don't know at compile time exactly *which* board file will be executed. So the only practical way to solve this at compile time is to predict an initcall ordering sequence for all possible boot paths, compile in all of them, and choose the right one at boot. But the number of boot paths is equal to the number of device trees / ACPI tables or board files supported, and that space is uncontrolled and ordered infinite. Basically I think the root problem with your approach is that you assume we know what hardware we will boot on at compile time. We discarded that development path years ago. We have no clue, this is resolved at runtime. Alas, people still create super-optimized systems using exactly this knowledge, but it is not our main target here, it is a special optimization case. Yours, Linus Walleij
[PATCH 00/21] On-demand device registration
Am 12.06.2015 um 13:36 schrieb Alexander Holler: > Am 12.06.2015 um 13:19 schrieb Alexander Holler: >> Am 12.06.2015 um 09:25 schrieb Linus Walleij: >>> On Thu, Jun 11, 2015 at 6:40 PM, Alexander Holler >>> wrote: Am 11.06.2015 um 14:30 schrieb Linus Walleij: >>> > Certainly it is possible to create deadlocks in this scenario, but the > scope is not to create an ubreakable system. IAnd what happens if you run into a deadlock? Do you print "you've lost, try changing your kernel config" in some output hidden by a splash-screen? ;) >>> >>> Sorry it sounds like a blanket argument, the fact that there are >>> mutexes in the kernel makes it possible to deadlock, it doesn't >>> mean we don't use mutexes. Some programming problems are >>> just like such. >> >> I'm not talking about specific deadlocks through mutexes. I'm talking >> about what happens when driver A needs driver B which needs driver A. >> How do you recognise and handle that with your instrumented on-demand >> device initialization? Such a circular dependency might happen by just >> adding a new fucntion call or by changing the kernel configuration. And >> with the on-demand stuff, the possibility that the developer introducing >> this new (maybe optional) call will never hit such a circular dependency >> is high. So you will end up with a never ending stream of problem >> reports whenever someone introduced such a circular dependecy without >> having noticed it. >> >> And to come back to specific deadlocks, if you are extending function >> calls from something former simple to something which might initialize a >> whole bunch of drivers, needing maybe seconds, I wouldn't say this is a >> blanket argument, but a real thread. > > Keep in mind, that the possibility that a function call ends up with > initializing a whole bunch of other drivers, is not determined > statically, but depends on the configuration and runtime behaviour of > the actual system the on-demand stuff actually happens. > > E.g. if driver A is faster one system that driver B, the whole bunch of > drivers might become initialized by a call in driver A. But if driver B > was faster on the developers system (or the system is configured to > first init driver B), than the whole bunch of drivers might have become > initialized by driver B on the developers system. Thus he never might > have hit a possible problem when the whole bunch of drivers got > initialized in driver A. > > That means it isn't always a good idea to create dynamic systems (like > on-demand device initialization), because it's very hard to foresee and > correctly handle their runtime behaviour. And because you've said that "problem space is a bit convoluted" and I disagree, here's a summary from my point of view: 1. All the necessary information (dependencies between drivers) already exists at compile time. The set of dependencies between drivers might become smaller by configuration, but will not become larger. So there should be NO need to collect them at runtime, e.g. by instrumenting function calls. I've described the problems I see with that above. I've choosen DT as source of dependencies because it offers an easy accessible and almost complete set of dependencies. I just had to add some type information to the dtb in order to identify the dependencies (phandles). But other ways to collect the dependencies would work too. Even the most simple way to add a static list of dependencies to each driver (which later on might be automated by some more clever stuff than adding them manually) would do the trick. 2. The problem to sort a set of nodes (drivers) with dependencies is solved since a long time and almost any developers uses it regularly in form of make. And everyone who used make -jN knows that the possible parallel initialization of drivers I've talked about, is already solved too. 3. In order to initialize the drivers in some specific order, their initcalls must be identified. I've offered a possible solution to that without much changes, but many other, even better ways, are possible too. It just depends on how much you want to change and on how much of these changes you will be able to feed into mainline kernel (which depends on your connections/relations inside the core kernel crew). E.g. instead of still just relying on one-dimensional arrays with (anonymous) pointers to initcalls, a multidimensional array of initcalls and drivername (and maybe more information) might be thinkable. 4. x86/amd64/ACPI-people, so most longtime and core kernel maintainers obviously don't have much interest until you've solved 1. in a way they can use too. So the necessary changes for 2. or 3. will have a big hurdle to take if 1. isn't solved usable for them too. >> Alexander Holler
[PATCH 00/21] On-demand device registration
Am 12.06.2015 um 13:19 schrieb Alexander Holler: > Am 12.06.2015 um 09:25 schrieb Linus Walleij: >> On Thu, Jun 11, 2015 at 6:40 PM, Alexander Holler >> wrote: >>> Am 11.06.2015 um 14:30 schrieb Linus Walleij: >> Certainly it is possible to create deadlocks in this scenario, but the scope is not to create an ubreakable system. >>> >>> IAnd what happens if you run into a deadlock? Do you print "you've >>> lost, try >>> changing your kernel config" in some output hidden by a >>> splash-screen? ;) >> >> Sorry it sounds like a blanket argument, the fact that there are >> mutexes in the kernel makes it possible to deadlock, it doesn't >> mean we don't use mutexes. Some programming problems are >> just like such. > > I'm not talking about specific deadlocks through mutexes. I'm talking > about what happens when driver A needs driver B which needs driver A. > How do you recognise and handle that with your instrumented on-demand > device initialization? Such a circular dependency might happen by just > adding a new fucntion call or by changing the kernel configuration. And > with the on-demand stuff, the possibility that the developer introducing > this new (maybe optional) call will never hit such a circular dependency > is high. So you will end up with a never ending stream of problem > reports whenever someone introduced such a circular dependecy without > having noticed it. > > And to come back to specific deadlocks, if you are extending function > calls from something former simple to something which might initialize a > whole bunch of drivers, needing maybe seconds, I wouldn't say this is a > blanket argument, but a real thread. Keep in mind, that the possibility that a function call ends up with initializing a whole bunch of other drivers, is not determined statically, but depends on the configuration and runtime behaviour of the actual system the on-demand stuff actually happens. E.g. if driver A is faster one system that driver B, the whole bunch of drivers might become initialized by a call in driver A. But if driver B was faster on the developers system (or the system is configured to first init driver B), than the whole bunch of drivers might have become initialized by driver B on the developers system. Thus he never might have hit a possible problem when the whole bunch of drivers got initialized in driver A. That means it isn't always a good idea to create dynamic systems (like on-demand device initialization), because it's very hard to foresee and correctly handle their runtime behaviour. > Alexander Holler
[PATCH 00/21] On-demand device registration
Am 12.06.2015 um 09:25 schrieb Linus Walleij: > On Thu, Jun 11, 2015 at 6:40 PM, Alexander Holler > wrote: >> Am 11.06.2015 um 14:30 schrieb Linus Walleij: > >>> Certainly it is possible to create deadlocks in this scenario, but the >>> scope is not to create an ubreakable system. >> >> IAnd what happens if you run into a deadlock? Do you print "you've lost, try >> changing your kernel config" in some output hidden by a splash-screen? ;) > > Sorry it sounds like a blanket argument, the fact that there are > mutexes in the kernel makes it possible to deadlock, it doesn't > mean we don't use mutexes. Some programming problems are > just like such. I'm not talking about specific deadlocks through mutexes. I'm talking about what happens when driver A needs driver B which needs driver A. How do you recognise and handle that with your instrumented on-demand device initialization? Such a circular dependency might happen by just adding a new fucntion call or by changing the kernel configuration. And with the on-demand stuff, the possibility that the developer introducing this new (maybe optional) call will never hit such a circular dependency is high. So you will end up with a never ending stream of problem reports whenever someone introduced such a circular dependecy without having noticed it. And to come back to specific deadlocks, if you are extending function calls from something former simple to something which might initialize a whole bunch of drivers, needing maybe seconds, I wouldn't say this is a blanket argument, but a real thread. Alexander Holler
[PATCH 00/21] On-demand device registration
On Thu, Jun 11, 2015 at 6:40 PM, Alexander Holler wrote: > Am 11.06.2015 um 14:30 schrieb Linus Walleij: >> Certainly it is possible to create deadlocks in this scenario, but the >> scope is not to create an ubreakable system. > > IAnd what happens if you run into a deadlock? Do you print "you've lost, try > changing your kernel config" in some output hidden by a splash-screen? ;) Sorry it sounds like a blanket argument, the fact that there are mutexes in the kernel makes it possible to deadlock, it doesn't mean we don't use mutexes. Some programming problems are just like such. Yours, Linus Walleij
[PATCH 00/21] On-demand device registration
Am 11.06.2015 um 14:30 schrieb Linus Walleij: > On Thu, Jun 11, 2015 at 12:17 PM, Alexander Holler > wrote: >> Am 11.06.2015 um 10:12 schrieb Linus Walleij: >>> On Wed, Jun 10, 2015 at 10:28 AM, Alexander Holler >>> wrote: > You would end up with the same problem of deadlocks as currently, and you would still need something ugly like the defered probe brutforce to avoid them. >>> >>> >>> Sorry I don't get that. Care to elaborate on why? >> >> >> Because loading/initializing on demand doesn't give you any solved order of >> drivers to initialize. And it can't because it has no idea about the >> requirements of other drivers. The reason why it might work better in the >> case of the tegra is that it might give you another initialization order >> than the one which is currently choosen, which, by luck, might be a better >> one. >> >> But maybe I missed something, I haven't looked at the patches at all. But >> just loading on demand, can't magically give you a working order of drivers >> to initialize. E.g. how do you choose the first driver to initialize? > > So the current patch set introduces dependencies (just for device tree) > and Tomeu is working on a more generic dependency approach for > any HW description. > > The first driver to initialize will be as usual the first one in the list for > that initlevel, then walking up the initilevels. > > However if any driver runs into a resource roadblock it will postpone > and wait for dependencies to probe first. > > Certainly it is possible to create deadlocks in this scenario, but the > scope is not to create an ubreakable system. IAnd what happens if you run into a deadlock? Do you print "you've lost, try changing your kernel config" in some output hidden by a splash-screen? ;) That sounds like the fun with duck typed languages where you have to test any and every possible screnario (something which is almost impossible) in order to not run into something unexpected. Anyway, have fun, good luck. Alexander Holler
[PATCH 00/21] On-demand device registration
On 06/11/2015 12:17 PM, Alexander Holler wrote: > Am 11.06.2015 um 10:12 schrieb Linus Walleij: >> On Wed, Jun 10, 2015 at 10:28 AM, Alexander Holler >> wrote: >>> Am 10.06.2015 um 09:30 schrieb Linus Walleij: >> i2c host comes out, probes the regulator driver, regulator driver probes and then the regulator_get() call returns. This requires instrumentation on anything providing a resource to another driver like those I mentioned and a lot of overhead infrastructure, but I think it's the right approach. However I don't know if I would ever be able to pull that off myself, I know talk is cheap and I should show the code instead. >>> >>> You would end up with the same problem of deadlocks as currently, and you >>> would still need something ugly like the defered probe brutforce to avoid >>> them. >> >> Sorry I don't get that. Care to elaborate on why? > > Because loading/initializing on demand doesn't give you any solved order > of drivers to initialize. And it can't because it has no idea about the > requirements of other drivers. So, this is only about ordering device probing. All built-in drivers have already registered themselves by when we start probing. > The reason why it might work better in > the case of the tegra is that it might give you another initialization > order than the one which is currently choosen, which, by luck, might be > a better one. Note that this series was also tested on iMX.6, Exynos and OMAP4. > But maybe I missed something, I haven't looked at the patches at all. It's a really small patchset :) 19 files changed, 130 insertions(+), 45 deletions(-) Thanks, Tomeu > But just loading on demand, can't magically give you a working order of > drivers to initialize. E.g. how do you choose the first driver to > initialize? > > Regards, > > Alexander Holler >
[PATCH 00/21] On-demand device registration
On Thu, Jun 11, 2015 at 12:17 PM, Alexander Holler wrote: > Am 11.06.2015 um 10:12 schrieb Linus Walleij: >> On Wed, Jun 10, 2015 at 10:28 AM, Alexander Holler >> wrote: >>> You would end up with the same problem of deadlocks as currently, and you >>> would still need something ugly like the defered probe brutforce to avoid >>> them. >> >> >> Sorry I don't get that. Care to elaborate on why? > > > Because loading/initializing on demand doesn't give you any solved order of > drivers to initialize. And it can't because it has no idea about the > requirements of other drivers. The reason why it might work better in the > case of the tegra is that it might give you another initialization order > than the one which is currently choosen, which, by luck, might be a better > one. > > But maybe I missed something, I haven't looked at the patches at all. But > just loading on demand, can't magically give you a working order of drivers > to initialize. E.g. how do you choose the first driver to initialize? So the current patch set introduces dependencies (just for device tree) and Tomeu is working on a more generic dependency approach for any HW description. The first driver to initialize will be as usual the first one in the list for that initlevel, then walking up the initilevels. However if any driver runs into a resource roadblock it will postpone and wait for dependencies to probe first. Certainly it is possible to create deadlocks in this scenario, but the scope is not to create an ubreakable system. Yours, Linus Walleij
[PATCH 00/21] On-demand device registration
Am 11.06.2015 um 13:24 schrieb Alexander Holler: > Am 11.06.2015 um 12:17 schrieb Alexander Holler: >> Am 11.06.2015 um 10:12 schrieb Linus Walleij: >>> On Wed, Jun 10, 2015 at 10:28 AM, Alexander Holler >>> wrote: Am 10.06.2015 um 09:30 schrieb Linus Walleij: >>> > i2c host comes out, probes the regulator driver, regulator driver > probes and then the regulator_get() call returns. > > This requires instrumentation on anything providing a resource > to another driver like those I mentioned and a lot of overhead > infrastructure, but I think it's the right approach. However I don't > know if I would ever be able to pull that off myself, I know talk > is cheap and I should show the code instead. You would end up with the same problem of deadlocks as currently, and you would still need something ugly like the defered probe brutforce to avoid them. >>> >>> Sorry I don't get that. Care to elaborate on why? >> >> Because loading/initializing on demand doesn't give you any solved order >> of drivers to initialize. And it can't because it has no idea about the >> requirements of other drivers. The reason why it might work better in >> the case of the tegra is that it might give you another initialization >> order than the one which is currently choosen, which, by luck, might be >> a better one. >> >> But maybe I missed something, I haven't looked at the patches at all. >> But just loading on demand, can't magically give you a working order of >> drivers to initialize. E.g. how do you choose the first driver to >> initialize? > > Other problems you will run into are time constraints and multithreaded > drivers. > > E.g. we all should know how tricky it sometimes is to avoid deadlocks. > And with loading on demand, you are extending this problem over the > initialization of maybe a whole bunch of other drivers which might be > started by calling one function of another driver. And a function call > might need a very long time to finish during which an unpredictable > amount of things may happen. > > It would make me wonder if that will end up with a good, usable and as > simple as possible solution. Besides that instrumenting every call to another driver in order to fix a onetime operation (the initialization) sounds like an enormous overhead. Initialization is done pnly once, regardless how long a system runs, but the instrumentation to fix this onetime operation would slow down the operation during the whole runtime of a system. I don't think this is what should be done. > > Regards, > > Alexander Holler
[PATCH 00/21] On-demand device registration
Am 11.06.2015 um 12:17 schrieb Alexander Holler: > Am 11.06.2015 um 10:12 schrieb Linus Walleij: >> On Wed, Jun 10, 2015 at 10:28 AM, Alexander Holler >> wrote: >>> Am 10.06.2015 um 09:30 schrieb Linus Walleij: >> i2c host comes out, probes the regulator driver, regulator driver probes and then the regulator_get() call returns. This requires instrumentation on anything providing a resource to another driver like those I mentioned and a lot of overhead infrastructure, but I think it's the right approach. However I don't know if I would ever be able to pull that off myself, I know talk is cheap and I should show the code instead. >>> >>> You would end up with the same problem of deadlocks as currently, and >>> you >>> would still need something ugly like the defered probe brutforce to >>> avoid >>> them. >> >> Sorry I don't get that. Care to elaborate on why? > > Because loading/initializing on demand doesn't give you any solved order > of drivers to initialize. And it can't because it has no idea about the > requirements of other drivers. The reason why it might work better in > the case of the tegra is that it might give you another initialization > order than the one which is currently choosen, which, by luck, might be > a better one. > > But maybe I missed something, I haven't looked at the patches at all. > But just loading on demand, can't magically give you a working order of > drivers to initialize. E.g. how do you choose the first driver to > initialize? Other problems you will run into are time constraints and multithreaded drivers. E.g. we all should know how tricky it sometimes is to avoid deadlocks. And with loading on demand, you are extending this problem over the initialization of maybe a whole bunch of other drivers which might be started by calling one function of another driver. And a function call might need a very long time to finish during which an unpredictable amount of things may happen. It would make me wonder if that will end up with a good, usable and as simple as possible solution. Regards, Alexander Holler
[PATCH 00/21] On-demand device registration
Am 11.06.2015 um 10:12 schrieb Linus Walleij: > On Wed, Jun 10, 2015 at 10:28 AM, Alexander Holler > wrote: >> Am 10.06.2015 um 09:30 schrieb Linus Walleij: > >>> i2c host comes out, probes the regulator driver, regulator driver >>> probes and then the regulator_get() call returns. >>> >>> This requires instrumentation on anything providing a resource >>> to another driver like those I mentioned and a lot of overhead >>> infrastructure, but I think it's the right approach. However I don't >>> know if I would ever be able to pull that off myself, I know talk >>> is cheap and I should show the code instead. >> >> You would end up with the same problem of deadlocks as currently, and you >> would still need something ugly like the defered probe brutforce to avoid >> them. > > Sorry I don't get that. Care to elaborate on why? Because loading/initializing on demand doesn't give you any solved order of drivers to initialize. And it can't because it has no idea about the requirements of other drivers. The reason why it might work better in the case of the tegra is that it might give you another initialization order than the one which is currently choosen, which, by luck, might be a better one. But maybe I missed something, I haven't looked at the patches at all. But just loading on demand, can't magically give you a working order of drivers to initialize. E.g. how do you choose the first driver to initialize? Regards, Alexander Holler
[PATCH 00/21] On-demand device registration
On 06/11/2015 10:15 AM, Linus Walleij wrote: > On Wed, Jun 10, 2015 at 12:19 PM, Tomeu Vizoso > wrote: >> On 10 June 2015 at 09:30, Linus Walleij wrote: > >>> regulator_get(...) -> not available, so: >>> - identify target regulator provider - this will need instrumentation >>> - probe it >>> >>> It then turns out the regulator driver is on the i2c bus, so we >>> need to probe the i2c driver: >>> - identify target i2c host for the regulator driver - this will need >>> instrumentation >>> - probe the i2c host driver >>> >>> i2c host comes out, probes the regulator driver, regulator driver >>> probes and then the regulator_get() call returns. >> >> Hmm, if I understand correctly what you say, this is exactly what this >> particular series does: >> >> regulator_get -> of_platform_device_ensure -> probe() on the platform >> device that encloses the requested device node (i2c host) -> i2c slave >> gets probed and the regulator registered -> regulator_get returns the >> requested resource > > Yes. But only for device tree. > >> The downside I'm currently looking at is that an explicit dependency >> graph would be useful to have for other purposes. For example to print >> a neat warning when a dependency cannot be fulfilled. Or to refuse to >> unbind a device which other devices depend on, or to automatically >> unbind the devices that depend on it, or to print a warning if a >> device is hotplugged off and other devices depend on it. > > Unbind/remove() calls are the inverse usually yes. > > But also the [runtime] power up/down sequences for the > devices tend to depend on a similar ordering or mostly > the same. (Mentioned this before I think.) > >>> This requires instrumentation on anything providing a resource >>> to another driver like those I mentioned and a lot of overhead >>> infrastructure, but I think it's the right approach. However I don't >>> know if I would ever be able to pull that off myself, I know talk >>> is cheap and I should show the code instead. >> >> Yeah, if you can give it a second look and say if it matches what you >> wrote above, it would be very much appreciated. > > Yes you are right. But what about ACPI, board files, > Simple Firmware and future hardware description languages... Ah ok, got it now. With fwnode and by moving a bit of code around that shouldn't be a problem. I'm actually now implementing the alternative approach in which dependencies are discovered before the device is probed, then probed in turn until all are available. So functionally is very similar but I expect to find big differences in how the codebase is impacted. Regards, Tomeu > Yours, > Linus Walleij >
[PATCH 00/21] On-demand device registration
On Wed, Jun 10, 2015 at 12:19 PM, Tomeu Vizoso wrote: > On 10 June 2015 at 09:30, Linus Walleij wrote: >> regulator_get(...) -> not available, so: >> - identify target regulator provider - this will need instrumentation >> - probe it >> >> It then turns out the regulator driver is on the i2c bus, so we >> need to probe the i2c driver: >> - identify target i2c host for the regulator driver - this will need >> instrumentation >> - probe the i2c host driver >> >> i2c host comes out, probes the regulator driver, regulator driver >> probes and then the regulator_get() call returns. > > Hmm, if I understand correctly what you say, this is exactly what this > particular series does: > > regulator_get -> of_platform_device_ensure -> probe() on the platform > device that encloses the requested device node (i2c host) -> i2c slave > gets probed and the regulator registered -> regulator_get returns the > requested resource Yes. But only for device tree. > The downside I'm currently looking at is that an explicit dependency > graph would be useful to have for other purposes. For example to print > a neat warning when a dependency cannot be fulfilled. Or to refuse to > unbind a device which other devices depend on, or to automatically > unbind the devices that depend on it, or to print a warning if a > device is hotplugged off and other devices depend on it. Unbind/remove() calls are the inverse usually yes. But also the [runtime] power up/down sequences for the devices tend to depend on a similar ordering or mostly the same. (Mentioned this before I think.) >> This requires instrumentation on anything providing a resource >> to another driver like those I mentioned and a lot of overhead >> infrastructure, but I think it's the right approach. However I don't >> know if I would ever be able to pull that off myself, I know talk >> is cheap and I should show the code instead. > > Yeah, if you can give it a second look and say if it matches what you > wrote above, it would be very much appreciated. Yes you are right. But what about ACPI, board files, Simple Firmware and future hardware description languages... Yours, Linus Walleij
[PATCH 00/21] On-demand device registration
Am 10.06.2015 um 14:23 schrieb Andrzej Hajda: > On 06/10/2015 12:19 PM, Tomeu Vizoso wrote: >> On 10 June 2015 at 09:30, Linus Walleij wrote: >>> On Tue, Jun 2, 2015 at 12:14 PM, Tomeu Vizoso >>> wrote: On 2 June 2015 at 10:48, Linus Walleij wrote: >>> > This is what systemd is doing in userspace for starting services: > ask for your dependencies and wait for them if they are not > there. So drivers ask for resources and wait for them. It also > needs to be abstract, so for example we need to be able to > hang on regulator_get() until the driver is up and providing that > regulator, and as long as everything is in slowpath it should > be OK. (And vice versa mutatis mutandis for clk, gpio, pin > control, interrupts (!) and DMA channels for example.) I understood above that you propose probing devices in order, but now you mention that resource getters would block until the dependency is fulfilled which confuses me because if we are probing in order then all dependencies would be fulfilled before the device in question gets probed. >>> >>> Sorry, the problem space is a bit convoluted so the answers >>> get a bit convoluted. Maybe I'm thinking aloud and altering the course >>> of my thoughts as I type... >>> >>> I guess there can be explicit dependencies for resources like this >>> patch does, but another way would be for all resource fetch functions >>> to be instrumented, so that you do not block until you try to take >>> a resource that is not yet there, e.g.: >>> >>> regulator_get(...) -> not available, so: >>> - identify target regulator provider - this will need instrumentation >>> - probe it >>> >>> It then turns out the regulator driver is on the i2c bus, so we >>> need to probe the i2c driver: >>> - identify target i2c host for the regulator driver - this will need >>> instrumentation >>> - probe the i2c host driver >>> >>> i2c host comes out, probes the regulator driver, regulator driver >>> probes and then the regulator_get() call returns. >> >> Hmm, if I understand correctly what you say, this is exactly what this >> particular series does: >> >> regulator_get -> of_platform_device_ensure -> probe() on the platform >> device that encloses the requested device node (i2c host) -> i2c slave >> gets probed and the regulator registered -> regulator_get returns the >> requested resource > > The downside of this solution is that it will not work without device > tree or even without device dependencies not explicitly specified in > device tree. Solution for what? The goal isn't to search another fancy registration/initialization algorithm. So I wonder which problem that would solve at all. It doesn't give you some deterministic initialization order nor does it (re)solve dependencies (besides directly from one driver to another, but that isn't enough), nor does it solve the problem of identifying drivers (the other end of such an instrumented on-demand-initialization-call). So all it would be is some fancy on-demand initialization without having solved any problem. Sorry if that sounds hard. Maybe I miss something. But I don't see any currently existing problem the above described solution would solve, besides beeing something different (which shouldn't be the goal). Alexander Holler
[PATCH 00/21] On-demand device registration
On 06/10/2015 12:19 PM, Tomeu Vizoso wrote: > On 10 June 2015 at 09:30, Linus Walleij wrote: >> On Tue, Jun 2, 2015 at 12:14 PM, Tomeu Vizoso >> wrote: >>> On 2 June 2015 at 10:48, Linus Walleij wrote: >> This is what systemd is doing in userspace for starting services: ask for your dependencies and wait for them if they are not there. So drivers ask for resources and wait for them. It also needs to be abstract, so for example we need to be able to hang on regulator_get() until the driver is up and providing that regulator, and as long as everything is in slowpath it should be OK. (And vice versa mutatis mutandis for clk, gpio, pin control, interrupts (!) and DMA channels for example.) >>> >>> I understood above that you propose probing devices in order, but now >>> you mention that resource getters would block until the dependency is >>> fulfilled which confuses me because if we are probing in order then >>> all dependencies would be fulfilled before the device in question gets >>> probed. >> >> Sorry, the problem space is a bit convoluted so the answers >> get a bit convoluted. Maybe I'm thinking aloud and altering the course >> of my thoughts as I type... >> >> I guess there can be explicit dependencies for resources like this >> patch does, but another way would be for all resource fetch functions >> to be instrumented, so that you do not block until you try to take >> a resource that is not yet there, e.g.: >> >> regulator_get(...) -> not available, so: >> - identify target regulator provider - this will need instrumentation >> - probe it >> >> It then turns out the regulator driver is on the i2c bus, so we >> need to probe the i2c driver: >> - identify target i2c host for the regulator driver - this will need >> instrumentation >> - probe the i2c host driver >> >> i2c host comes out, probes the regulator driver, regulator driver >> probes and then the regulator_get() call returns. > > Hmm, if I understand correctly what you say, this is exactly what this > particular series does: > > regulator_get -> of_platform_device_ensure -> probe() on the platform > device that encloses the requested device node (i2c host) -> i2c slave > gets probed and the regulator registered -> regulator_get returns the > requested resource The downside of this solution is that it will not work without device tree or even without device dependencies not explicitly specified in device tree. > > The downside I'm currently looking at is that an explicit dependency > graph would be useful to have for other purposes. For example to print > a neat warning when a dependency cannot be fulfilled. Or to refuse to > unbind a device which other devices depend on, As I understand Greg you cannot prevent unbinding by design, see [1]. [1]: http://thread.gmane.org/gmane.linux.kernel/1154308/focus=1154648 > or to automatically > unbind the devices that depend on it, What about devices that have weak dependency? They should not be unbound but they should be somehow noticed about unbinding. In general many kernel frameworks are broken in handling hot-unbinding of drivers, consumers are not noticed about unbinding of their resource providers and usually they stay with broken handles or handles to dummy resources. I suspect the only proper solution for handling resources that can dynamically appear/disappear is to provide notification to their consumers about appearance change of the resource. I have proposed some times ago solution for above problems based on the statement above, cover letter explains it in more detail [2]. In short it solves following issues: - consumer receives resource as soon as it becomes available, - consumer is notified just before resource removal, - it can properly handle provider unbind/re-bind, - it avoids late init due to deferred probing, - it allows to track optional resources. [2]: http://thread.gmane.org/gmane.linux.kernel.gpio/5201 Regards Andrzej > or to print a warning if a > device is hotplugged off and other devices depend on it. > >> This requires instrumentation on anything providing a resource >> to another driver like those I mentioned and a lot of overhead >> infrastructure, but I think it's the right approach. However I don't >> know if I would ever be able to pull that off myself, I know talk >> is cheap and I should show the code instead. > > Yeah, if you can give it a second look and say if it matches what you > wrote above, it would be very much appreciated. > >> Deepest respect for your efforts! > > Thanks! > > Tomeu > >> Yours, >> Linus Walleij >> ___ >> dri-devel mailing list >> dri-devel at lists.freedesktop.org >> http://lists.freedesktop.org/mailman/listinfo/dri-devel > -- > To unsubscribe from this list: send the line "unsubscribe linux-gpio" in > the body of a message to majordomo at vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >
[PATCH 00/21] On-demand device registration
On 10 June 2015 at 09:30, Linus Walleij wrote: > On Tue, Jun 2, 2015 at 12:14 PM, Tomeu Vizoso > wrote: >> On 2 June 2015 at 10:48, Linus Walleij wrote: > >>> This is what systemd is doing in userspace for starting services: >>> ask for your dependencies and wait for them if they are not >>> there. So drivers ask for resources and wait for them. It also >>> needs to be abstract, so for example we need to be able to >>> hang on regulator_get() until the driver is up and providing that >>> regulator, and as long as everything is in slowpath it should >>> be OK. (And vice versa mutatis mutandis for clk, gpio, pin >>> control, interrupts (!) and DMA channels for example.) >> >> I understood above that you propose probing devices in order, but now >> you mention that resource getters would block until the dependency is >> fulfilled which confuses me because if we are probing in order then >> all dependencies would be fulfilled before the device in question gets >> probed. > > Sorry, the problem space is a bit convoluted so the answers > get a bit convoluted. Maybe I'm thinking aloud and altering the course > of my thoughts as I type... > > I guess there can be explicit dependencies for resources like this > patch does, but another way would be for all resource fetch functions > to be instrumented, so that you do not block until you try to take > a resource that is not yet there, e.g.: > > regulator_get(...) -> not available, so: > - identify target regulator provider - this will need instrumentation > - probe it > > It then turns out the regulator driver is on the i2c bus, so we > need to probe the i2c driver: > - identify target i2c host for the regulator driver - this will need > instrumentation > - probe the i2c host driver > > i2c host comes out, probes the regulator driver, regulator driver > probes and then the regulator_get() call returns. Hmm, if I understand correctly what you say, this is exactly what this particular series does: regulator_get -> of_platform_device_ensure -> probe() on the platform device that encloses the requested device node (i2c host) -> i2c slave gets probed and the regulator registered -> regulator_get returns the requested resource The downside I'm currently looking at is that an explicit dependency graph would be useful to have for other purposes. For example to print a neat warning when a dependency cannot be fulfilled. Or to refuse to unbind a device which other devices depend on, or to automatically unbind the devices that depend on it, or to print a warning if a device is hotplugged off and other devices depend on it. > This requires instrumentation on anything providing a resource > to another driver like those I mentioned and a lot of overhead > infrastructure, but I think it's the right approach. However I don't > know if I would ever be able to pull that off myself, I know talk > is cheap and I should show the code instead. Yeah, if you can give it a second look and say if it matches what you wrote above, it would be very much appreciated. > Deepest respect for your efforts! Thanks! Tomeu > Yours, > Linus Walleij > ___ > dri-devel mailing list > dri-devel at lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/dri-devel
[PATCH 00/21] On-demand device registration
Am 10.06.2015 um 09:30 schrieb Linus Walleij: > On Tue, Jun 2, 2015 at 12:14 PM, Tomeu Vizoso > wrote: >> On 2 June 2015 at 10:48, Linus Walleij wrote: > >>> This is what systemd is doing in userspace for starting services: >>> ask for your dependencies and wait for them if they are not >>> there. So drivers ask for resources and wait for them. It also >>> needs to be abstract, so for example we need to be able to >>> hang on regulator_get() until the driver is up and providing that >>> regulator, and as long as everything is in slowpath it should >>> be OK. (And vice versa mutatis mutandis for clk, gpio, pin >>> control, interrupts (!) and DMA channels for example.) >> >> I understood above that you propose probing devices in order, but now >> you mention that resource getters would block until the dependency is >> fulfilled which confuses me because if we are probing in order then >> all dependencies would be fulfilled before the device in question gets >> probed. > > Sorry, the problem space is a bit convoluted so the answers > get a bit convoluted. Maybe I'm thinking aloud and altering the course > of my thoughts as I type... > > I guess there can be explicit dependencies for resources like this > patch does, but another way would be for all resource fetch functions > to be instrumented, so that you do not block until you try to take > a resource that is not yet there, e.g.: > > regulator_get(...) -> not available, so: > - identify target regulator provider - this will need instrumentation > - probe it > > It then turns out the regulator driver is on the i2c bus, so we > need to probe the i2c driver: > - identify target i2c host for the regulator driver - this will need >instrumentation > - probe the i2c host driver > > i2c host comes out, probes the regulator driver, regulator driver > probes and then the regulator_get() call returns. > > This requires instrumentation on anything providing a resource > to another driver like those I mentioned and a lot of overhead > infrastructure, but I think it's the right approach. However I don't > know if I would ever be able to pull that off myself, I know talk > is cheap and I should show the code instead. You would end up with the same problem of deadlocks as currently, and you would still need something ugly like the defered probe brutforce to avoid them. So what would you win with that instrumentation? Alexander Holler
[PATCH 00/21] On-demand device registration
On Tue, Jun 2, 2015 at 12:14 PM, Tomeu Vizoso wrote: > On 2 June 2015 at 10:48, Linus Walleij wrote: >> This is what systemd is doing in userspace for starting services: >> ask for your dependencies and wait for them if they are not >> there. So drivers ask for resources and wait for them. It also >> needs to be abstract, so for example we need to be able to >> hang on regulator_get() until the driver is up and providing that >> regulator, and as long as everything is in slowpath it should >> be OK. (And vice versa mutatis mutandis for clk, gpio, pin >> control, interrupts (!) and DMA channels for example.) > > I understood above that you propose probing devices in order, but now > you mention that resource getters would block until the dependency is > fulfilled which confuses me because if we are probing in order then > all dependencies would be fulfilled before the device in question gets > probed. Sorry, the problem space is a bit convoluted so the answers get a bit convoluted. Maybe I'm thinking aloud and altering the course of my thoughts as I type... I guess there can be explicit dependencies for resources like this patch does, but another way would be for all resource fetch functions to be instrumented, so that you do not block until you try to take a resource that is not yet there, e.g.: regulator_get(...) -> not available, so: - identify target regulator provider - this will need instrumentation - probe it It then turns out the regulator driver is on the i2c bus, so we need to probe the i2c driver: - identify target i2c host for the regulator driver - this will need instrumentation - probe the i2c host driver i2c host comes out, probes the regulator driver, regulator driver probes and then the regulator_get() call returns. This requires instrumentation on anything providing a resource to another driver like those I mentioned and a lot of overhead infrastructure, but I think it's the right approach. However I don't know if I would ever be able to pull that off myself, I know talk is cheap and I should show the code instead. Deepest respect for your efforts! Yours, Linus Walleij
[PATCH 00/21] On-demand device registration
Am 08.06.2015 um 20:14 schrieb Alexander Holler: > Am 08.06.2015 um 14:26 schrieb Enrico Weigelt, metux IT consult: >> Am 04.06.2015 um 22:39 schrieb Alexander Holler: >> >> > As it seems to have been forgotten or overread, I've mentioned in my >>> series of patches last year that, with a few changes, it's possible to >>> let the algorithm I've used (dfs) to spit out all drivers which can be >>> initialized in parallel. >> >> Unfortunately, I've missed that ... could you please resend you patches? >> Boot time reduction is one of the topics on my 2do in several weeks. > > https://lkml.org/lkml/2014/5/12/452 > And don't forget patch 10/9 which fixed a bug in my previous patch series and which alos was the reason for the large difference in boot times with and without deps: https://lkml.org/lkml/2014/5/13/567
[PATCH 00/21] On-demand device registration
Am 08.06.2015 um 14:26 schrieb Enrico Weigelt, metux IT consult: > Am 04.06.2015 um 22:39 schrieb Alexander Holler: > > > As it seems to have been forgotten or overread, I've mentioned in my >> series of patches last year that, with a few changes, it's possible to >> let the algorithm I've used (dfs) to spit out all drivers which can be >> initialized in parallel. > > Unfortunately, I've missed that ... could you please resend you patches? > Boot time reduction is one of the topics on my 2do in several weeks. https://lkml.org/lkml/2014/5/12/452
[PATCH 00/21] On-demand device registration
Am 04.06.2015 um 22:39 schrieb Alexander Holler: > As it seems to have been forgotten or overread, I've mentioned in my > series of patches last year that, with a few changes, it's possible to > let the algorithm I've used (dfs) to spit out all drivers which can be > initialized in parallel. Unfortunately, I've missed that ... could you please resend you patches? Boot time reduction is one of the topics on my 2do in several weeks. cu -- Enrico Weigelt, metux IT consult +49-151-27565287 MELAG Medizintechnik oHG Sitz Berlin Registergericht AG Charlottenburg HRA 21333 B Wichtiger Hinweis: Diese Nachricht kann vertrauliche oder nur für einen begrenzten Personenkreis bestimmte Informationen enthalten. Sie ist ausschlieÃlich für denjenigen bestimmt, an den sie gerichtet worden ist. Wenn Sie nicht der Adressat dieser E-Mail sind, dürfen Sie diese nicht kopieren, weiterleiten, weitergeben oder sie ganz oder teilweise in irgendeiner Weise nutzen. Sollten Sie diese E-Mail irrtümlich erhalten haben, so benachrichtigen Sie bitte den Absender, indem Sie auf diese Nachricht antworten. Bitte löschen Sie in diesem Fall diese Nachricht und alle Anhänge, ohne eine Kopie zu behalten. Important Notice: This message may contain confidential or privileged information. It is intended only for the person it was addressed to. If you are not the intended recipient of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you received this email in error please notify the sender by replying and delete this message and any attachments without retaining a copy.
[PATCH 00/21] On-demand device registration
Am 03.06.2015 um 23:12 schrieb Rob Clark: > On Mon, May 25, 2015 at 10:53 AM, Tomeu Vizoso > wrote: >> Hello, >> >> I have a problem with the panel on my Tegra Chromebook taking longer than >> expected to be ready during boot (Stéphane Marchesin reported what is >> basically the same issue in [0]), and have looked into ordered probing as a >> better way of solving this than moving nodes around in the DT or playing with >> initcall levels. >> >> While reading the thread [1] that Alexander Holler started with his series to >> make probing order deterministic, it occurred to me that it should be >> possible >> to achieve the same by registering devices as they are referenced by other >> devices. >> >> This basically reuses the information that is already implicit in the probe() >> implementations, saving us from refactoring existing drivers or adding >> information to DTBs. >> >> Something I'm not completely happy with is that I have had to move the call >> to >> of_platform_populate after all platform drivers have been registered. >> Otherwise I don't see how I could register drivers on demand as we don't have >> yet each driver's compatible strings. >> >> For machs that don't move of_platform_populate() to a later point, these >> patches shouldn't cause any problems but it's not guaranteed that we'll avoid >> all the deferred probes as some drivers may not be registered yet. >> >> I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these >> patches were enough to eliminate all the deferred probes. >> >> With this series I get the kernel to output to the panel in 0.5s, instead of >> 2.8s. > > So, complete drive-by comment (and I won't claim to be a DT expert, > etc, etc, so take this with a few grains of salt), but why not push > the problem to the DT compiler (or a pre-process step that could be > run on existing DT blobs), which generates an optional DT node that is > the recommended probe order? That seems like it avoids adding > complexity into the early boot code (which seems like a good thing).. I've played with that approach too (as my patches for dtc do contain the same code I've put into the kernel, but decided that it doesn't make much sense. The sort algorithm is really small (some dozen lines), very fast (around 3-5ms on a omap) and might be later used to sort necessary module loading too. So there would be no advantage to put a sorted list into the DT. And having the sort algorithm in the kernel, would make it possible to use it for acpi or something else too, if they manage it to provide the necessary dependencies. Regards, Alexander Holler
[PATCH 00/21] On-demand device registration
Am 03.06.2015 um 21:57 schrieb Grygorii.Strashko at linaro.org: ... > So few comments from above: > - registering devices later during the System boot may improve boot time. > But resolving of all deferred probes may NOT improve boot time ;) > Have you seen smth like this? If someone is out for boot time reduction, I think one of the best ways would by making driver initialization parallel. Keep in mind that all linked in drivers currently are initialized in series. As it seems to have been forgotten or overread, I've mentioned in my series of patches last year that, with a few changes, it's possible to let the algorithm I've used (dfs) to spit out all drivers which can be initialized in parallel. But as I'm not paid for the work I've done and just did it out of curiosity, interest or how ever you want name it, I haven't spend any more time into that topic, especially as I'm missing the necessary connections to get patches into the kernel. ;) But, as said, it's easy (at least if aren't getting panic when it comes to a bit of algorithm theory) to get a list drivers you can start in parallel if you have such a complete list of dependencies as DT already offers. Just look at the pictures generate by dtc (using my patches), you will see, they already show which drivers can be initialized in parallel. So it would be easy to use e.g. all cores already very early at boot to initialize drivers, not just after init got started. Besides that the würgaround of defered init (which, btw. leads devs to supress error messages, which is especially bad if you are searching a problem) isn't needed anymore if you have a list of dependecies (however you get it, I've used DT because the dependencies already are all there). Regards, Alexander Holler
[PATCH 00/21] On-demand device registration
On 06/04/2015 11:39 AM, Tomeu Vizoso wrote:
> On 3 June 2015 at 21:57, Grygorii.Strashko at linaro.org
> wrote:
>> On 05/28/2015 07:33 AM, Rob Herring wrote:
>>> On Mon, May 25, 2015 at 9:53 AM, Tomeu Vizoso >> collabora.com> wrote:
I have a problem with the panel on my Tegra Chromebook taking longer than
expected to be ready during boot (Stéphane Marchesin reported what is
basically the same issue in [0]), and have looked into ordered probing as a
better way of solving this than moving nodes around in the DT or playing
with
initcall levels.
While reading the thread [1] that Alexander Holler started with his series
to
make probing order deterministic, it occurred to me that it should be
possible
to achieve the same by registering devices as they are referenced by other
devices.
>>>
>>> I like the concept and novel approach.
>>>
This basically reuses the information that is already implicit in the
probe()
implementations, saving us from refactoring existing drivers or adding
information to DTBs.
Something I'm not completely happy with is that I have had to move the
call to
of_platform_populate after all platform drivers have been registered.
Otherwise I don't see how I could register drivers on demand as we don't
have
yet each driver's compatible strings.
>>>
>>> Yeah, this is the opposite of what we'd really like. Ideally, we would
>>> have a solution that works for modules too. However, we're no worse
>>> off. We pretty much build-in dependencies to avoid module ordering
>>> problems.
>>>
>>> Perhaps we need to make the probing on-demand rather than simply on
>>> device<->driver match occurring.
>>>
For machs that don't move of_platform_populate() to a later point, these
patches shouldn't cause any problems but it's not guaranteed that we'll
avoid
all the deferred probes as some drivers may not be registered yet.
>>>
>>> Ideally, of_platform_populate is not explicitly called by each
>>> platform. So I think we need to make this work for the default case.
>>>
I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these
patches were enough to eliminate all the deferred probes.
With this series I get the kernel to output to the panel in 0.5s, instead
of 2.8s.
>>>
>>> That's certainly compelling.
>>
>> I've found your idea about moving device registration later during System
>> boot
>> very interesting so I've decided to try it on dra7-evem (TI) :).
>> It's good to know time during Kernel boot when we can assume that all
>> drivers are
>> ready for probing, so there are more ways to control probing order.
>
> Thanks, though right now I'm following Rob's suggestion and only delay
> probing, not registration. The patch is really simple (applies on
> linux-next, with async probing):
>
> diff --git a/drivers/base/dd.c b/drivers/base/dd.c
> index 8da8e07..7e6b1e1 100644
> --- a/drivers/base/dd.c
> +++ b/drivers/base/dd.c
> @@ -407,6 +407,11 @@ int driver_probe_device(struct device_driver
> *drv, struct device *dev)
> if (!device_is_registered(dev))
> return -ENODEV;
>
> + if (!driver_deferred_probe_enable) {
> + driver_deferred_probe_add(dev);
> + return 0;
> + }
> +
> pr_debug("bus: '%s': %s: matched device %s with driver %s\n",
> drv->bus->name, __func__, dev_name(dev), drv->name);
>
> @@ -585,7 +590,7 @@ EXPORT_SYMBOL_GPL(device_attach);
>
> void device_initial_probe(struct device *dev)
> {
> - __device_attach(dev, true);
> + __device_attach(dev, driver_deferred_probe_enable);
> }
>
> static int __driver_attach(struct device *dev, void *data)
Can't boot my 3.14 kernel with this change :( Most probably,
the problem is related to platform_driver_probe() usage :(
Have no time to play with it now :(, but recommend you to check also
earlyprintk, last log message I can see:
[1.435522] bootconsole [earlycon0] disabled
But, nice try ;) Seems -EPROBE_DEFER is reality of life which
has to be accepted as is.
>
>> Pls, Note here that TI OMAP2+ mach is not pure DT mach - it's combination of
>> DT and not DT devices/drivers.
>>
>> Ok. So What was done...
>>
>> LKML Linux 4.1-rc3 (a simple case)
>> 1) use your patches 3/4 as reference (only these two patches :)
>> 2) move of_platform_populate later at device_initcall_sync time
>> Boot time reduction ~0.4 sec
>
> I'm a bit surprised at such a big improvement. May I ask how you are
> measuring it?
Ah. My measurements are not precise. I've just tracking time of message
"[4.110756] Freeing unused kernel memory: 344K (c0994000 - c09ea000)"
>
>> TI Android Kernel 3.14 (NOT a simple case)
>> 1) use your patches 3/4 as reference (only these two patches :)
>> 2) move of_platform_populate later at device_initcall_sync time
>> 3) make it to boot (not sure I've fixed
[PATCH 00/21] On-demand device registration
On 3 June 2015 at 21:57, Grygorii.Strashko at linaro.org
wrote:
> Hi Tomeu,
>
> On 05/28/2015 07:33 AM, Rob Herring wrote:
>> On Mon, May 25, 2015 at 9:53 AM, Tomeu Vizoso > collabora.com> wrote:
>>> I have a problem with the panel on my Tegra Chromebook taking longer than
>>> expected to be ready during boot (Stéphane Marchesin reported what is
>>> basically the same issue in [0]), and have looked into ordered probing as a
>>> better way of solving this than moving nodes around in the DT or playing
>>> with
>>> initcall levels.
>>>
>>> While reading the thread [1] that Alexander Holler started with his series
>>> to
>>> make probing order deterministic, it occurred to me that it should be
>>> possible
>>> to achieve the same by registering devices as they are referenced by other
>>> devices.
>>
>> I like the concept and novel approach.
>>
>>> This basically reuses the information that is already implicit in the
>>> probe()
>>> implementations, saving us from refactoring existing drivers or adding
>>> information to DTBs.
>>>
>>> Something I'm not completely happy with is that I have had to move the call
>>> to
>>> of_platform_populate after all platform drivers have been registered.
>>> Otherwise I don't see how I could register drivers on demand as we don't
>>> have
>>> yet each driver's compatible strings.
>>
>> Yeah, this is the opposite of what we'd really like. Ideally, we would
>> have a solution that works for modules too. However, we're no worse
>> off. We pretty much build-in dependencies to avoid module ordering
>> problems.
>>
>> Perhaps we need to make the probing on-demand rather than simply on
>> device<->driver match occurring.
>>
>>> For machs that don't move of_platform_populate() to a later point, these
>>> patches shouldn't cause any problems but it's not guaranteed that we'll
>>> avoid
>>> all the deferred probes as some drivers may not be registered yet.
>>
>> Ideally, of_platform_populate is not explicitly called by each
>> platform. So I think we need to make this work for the default case.
>>
>>> I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these
>>> patches were enough to eliminate all the deferred probes.
>>>
>>> With this series I get the kernel to output to the panel in 0.5s, instead
>>> of 2.8s.
>>
>> That's certainly compelling.
>
> I've found your idea about moving device registration later during System boot
> very interesting so I've decided to try it on dra7-evem (TI) :).
> It's good to know time during Kernel boot when we can assume that all drivers
> are
> ready for probing, so there are more ways to control probing order.
Thanks, though right now I'm following Rob's suggestion and only delay
probing, not registration. The patch is really simple (applies on
linux-next, with async probing):
diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index 8da8e07..7e6b1e1 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -407,6 +407,11 @@ int driver_probe_device(struct device_driver
*drv, struct device *dev)
if (!device_is_registered(dev))
return -ENODEV;
+ if (!driver_deferred_probe_enable) {
+ driver_deferred_probe_add(dev);
+ return 0;
+ }
+
pr_debug("bus: '%s': %s: matched device %s with driver %s\n",
drv->bus->name, __func__, dev_name(dev), drv->name);
@@ -585,7 +590,7 @@ EXPORT_SYMBOL_GPL(device_attach);
void device_initial_probe(struct device *dev)
{
- __device_attach(dev, true);
+ __device_attach(dev, driver_deferred_probe_enable);
}
static int __driver_attach(struct device *dev, void *data)
> Pls, Note here that TI OMAP2+ mach is not pure DT mach - it's combination of
> DT and not DT devices/drivers.
>
> Ok. So What was done...
>
> LKML Linux 4.1-rc3 (a simple case)
> 1) use your patches 3/4 as reference (only these two patches :)
> 2) move of_platform_populate later at device_initcall_sync time
> Boot time reduction ~0.4 sec
I'm a bit surprised at such a big improvement. May I ask how you are
measuring it?
> TI Android Kernel 3.14 (NOT a simple case)
> 1) use your patches 3/4 as reference (only these two patches :)
> 2) move of_platform_populate later at device_initcall_sync time
> 3) make it to boot (not sure I've fixed all issues, but those which
>break the System boot):
> - split non-DT and DT devices registration in platform code;
> - keep non-DT devices registration from .init_machine() [arch_initcall]
> - move DT-devices registration at device_initcall_sync time
> - fix drivers which use platform_driver_probe().
>Note. Now there are at about ~190 occurrences of this macro in Kernel.
> - re-order few devices in DT (4 devices)
> - fix one driver which uses of_find_device_by_node() wrongly
>Note. This API is used some times with assumption that
>requested dev has been probed already.
> Boot time reduction ~0.3 sec. Probing of some devices are still deferred.
I got no deferred probes on a
[PATCH 00/21] On-demand device registration
Hi Tomeu, On 05/28/2015 07:33 AM, Rob Herring wrote: > On Mon, May 25, 2015 at 9:53 AM, Tomeu Vizoso > wrote: >> I have a problem with the panel on my Tegra Chromebook taking longer than >> expected to be ready during boot (Stéphane Marchesin reported what is >> basically the same issue in [0]), and have looked into ordered probing as a >> better way of solving this than moving nodes around in the DT or playing with >> initcall levels. >> >> While reading the thread [1] that Alexander Holler started with his series to >> make probing order deterministic, it occurred to me that it should be >> possible >> to achieve the same by registering devices as they are referenced by other >> devices. > > I like the concept and novel approach. > >> This basically reuses the information that is already implicit in the probe() >> implementations, saving us from refactoring existing drivers or adding >> information to DTBs. >> >> Something I'm not completely happy with is that I have had to move the call >> to >> of_platform_populate after all platform drivers have been registered. >> Otherwise I don't see how I could register drivers on demand as we don't have >> yet each driver's compatible strings. > > Yeah, this is the opposite of what we'd really like. Ideally, we would > have a solution that works for modules too. However, we're no worse > off. We pretty much build-in dependencies to avoid module ordering > problems. > > Perhaps we need to make the probing on-demand rather than simply on > device<->driver match occurring. > >> For machs that don't move of_platform_populate() to a later point, these >> patches shouldn't cause any problems but it's not guaranteed that we'll avoid >> all the deferred probes as some drivers may not be registered yet. > > Ideally, of_platform_populate is not explicitly called by each > platform. So I think we need to make this work for the default case. > >> I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these >> patches were enough to eliminate all the deferred probes. >> >> With this series I get the kernel to output to the panel in 0.5s, instead of >> 2.8s. > > That's certainly compelling. I've found your idea about moving device registration later during System boot very interesting so I've decided to try it on dra7-evem (TI) :). It's good to know time during Kernel boot when we can assume that all drivers are ready for probing, so there are more ways to control probing order. Pls, Note here that TI OMAP2+ mach is not pure DT mach - it's combination of DT and not DT devices/drivers. Ok. So What was done... LKML Linux 4.1-rc3 (a simple case) 1) use your patches 3/4 as reference (only these two patches :) 2) move of_platform_populate later at device_initcall_sync time Boot time reduction ~0.4 sec TI Android Kernel 3.14 (NOT a simple case) 1) use your patches 3/4 as reference (only these two patches :) 2) move of_platform_populate later at device_initcall_sync time 3) make it to boot (not sure I've fixed all issues, but those which break the System boot): - split non-DT and DT devices registration in platform code; - keep non-DT devices registration from .init_machine() [arch_initcall] - move DT-devices registration at device_initcall_sync time - fix drivers which use platform_driver_probe(). Note. Now there are at about ~190 occurrences of this macro in Kernel. - re-order few devices in DT (4 devices) - fix one driver which uses of_find_device_by_node() wrongly Note. This API is used some times with assumption that requested dev has been probed already. Boot time reduction ~0.3 sec. Probing of some devices are still deferred. TI Android Kernel 3.14 + of_platform_device_ensure 1) backport of_platform_device_ensure() (also need patches "of: Introduce device tree node flag helpers" and "of: Keep track of populated platform devices") 2) back-port all your patches which uses of_platform_device_ensure() 3) make it to boot: - drop patch dma: of: Probe DMA controllers on demand - fix deadlock in regulator core: regulator_dev_lookup() called from regulator_register() in K3.14 4) get rid of deferred probes - add of_platform_device_ensure() calls in: - drivers/video/fbdev/omap2/dss/output.c - drivers/extcon/extcon-class.c Boot time reduction: NONE !? So few comments from above: - registering devices later during the System boot may improve boot time. But resolving of all deferred probes may NOT improve boot time ;) Have you seen smth like this? - usage of of_platform_device_ensure() will require continuous fixing of Kernel :( - late_initcall is not (as for me not safe) a good time to register devices. A lot of platforms/subsystems/frameworks perform their final initialization or clean-up steps, with assumption that System mostly ready to work. For example, CPUIdle/CPUFreq are allowed and other PM staff. CPUIdle and driver's probing are not friends. What would be nice to have for now in my
[PATCH 00/21] On-demand device registration
On Mon, May 25, 2015 at 10:53 AM, Tomeu Vizoso wrote: > Hello, > > I have a problem with the panel on my Tegra Chromebook taking longer than > expected to be ready during boot (Stéphane Marchesin reported what is > basically the same issue in [0]), and have looked into ordered probing as a > better way of solving this than moving nodes around in the DT or playing with > initcall levels. > > While reading the thread [1] that Alexander Holler started with his series to > make probing order deterministic, it occurred to me that it should be possible > to achieve the same by registering devices as they are referenced by other > devices. > > This basically reuses the information that is already implicit in the probe() > implementations, saving us from refactoring existing drivers or adding > information to DTBs. > > Something I'm not completely happy with is that I have had to move the call to > of_platform_populate after all platform drivers have been registered. > Otherwise I don't see how I could register drivers on demand as we don't have > yet each driver's compatible strings. > > For machs that don't move of_platform_populate() to a later point, these > patches shouldn't cause any problems but it's not guaranteed that we'll avoid > all the deferred probes as some drivers may not be registered yet. > > I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these > patches were enough to eliminate all the deferred probes. > > With this series I get the kernel to output to the panel in 0.5s, instead of > 2.8s. So, complete drive-by comment (and I won't claim to be a DT expert, etc, etc, so take this with a few grains of salt), but why not push the problem to the DT compiler (or a pre-process step that could be run on existing DT blobs), which generates an optional DT node that is the recommended probe order? That seems like it avoids adding complexity into the early boot code (which seems like a good thing).. As a bonus, a bootarg (or something like that) which runs through the recommended probe order in reverse (to continue our current state of ensuring that EPROBE_DEFER error paths are well tested) At any rate, for a device like a drm driver that has multiple sub-components, and depends on various other clk/gpio/regulator/etc drivers, the current EPROBE_DEFER situation is pretty comical, so any solution that improves on things is very much welcome :-) BR, -R > Regards, > > Tomeu > > [0] http://lists.freedesktop.org/archives/dri-devel/2014-August/066527.html > > [1] https://lkml.org/lkml/2014/5/12/452 > > Tomeu Vizoso (21): > regulator: core: Reduce critical area in _regulator_get > ARM: tegra: Add gpio-ranges property > ARM: tegra: Register drivers before devices > ARM: EXYNOS: Register drivers before devices > ARM i.MX6q: Register drivers before devices > of/platform: Add of_platform_device_ensure() > of/platform: Ensure device registration on lookup > gpio: Probe GPIO drivers on demand > gpio: Probe pinctrl devices on demand > regulator: core: Probe regulators on demand > drm: Probe panels on demand > drm/tegra: Probe dpaux devices on demand > i2c: core: Probe i2c master devices on demand > pwm: Probe PWM chip devices on demand > backlight: Probe backlight devices on demand > usb: phy: Probe phy devices on demand > clk: Probe clk providers on demand > pinctrl: Probe pinctrl devices on demand > phy: core: Probe phy providers on demand > dma: of: Probe DMA controllers on demand > power-supply: Probe power supplies on demand > > arch/arm/boot/dts/tegra124.dtsi | 1 + > arch/arm/mach-exynos/exynos.c | 4 +-- > arch/arm/mach-imx/mach-imx6q.c | 12 - > arch/arm/mach-tegra/tegra.c | 21 ++- > drivers/clk/clk.c | 3 +++ > drivers/dma/of-dma.c| 3 +++ > drivers/gpio/gpiolib-of.c | 5 > drivers/gpu/drm/drm_panel.c | 3 +++ > drivers/gpu/drm/tegra/dpaux.c | 3 +++ > drivers/i2c/i2c-core.c | 3 +++ > drivers/of/platform.c | 53 > + > drivers/phy/phy-core.c | 3 +++ > drivers/pinctrl/devicetree.c| 2 ++ > drivers/power/power_supply_core.c | 3 +++ > drivers/pwm/core.c | 3 +++ > drivers/regulator/core.c| 45 +++ > drivers/usb/phy/phy.c | 3 +++ > drivers/video/backlight/backlight.c | 3 +++ > include/linux/of_platform.h | 2 ++ > 19 files changed, 130 insertions(+), 45 deletions(-) > > -- > 2.4.1 > > > ___ > linux-arm-kernel mailing list > linux-arm-kernel at lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
[PATCH 00/21] On-demand device registration
Am 02.06.2015 um 10:48 schrieb Linus Walleij: > On Mon, May 25, 2015 at 4:53 PM, Tomeu Vizoso > wrote: > >> have looked into ordered probing as a >> better way of solving this than moving nodes around in the DT or playing with >> initcall levels. >> >> While reading the thread [1] that Alexander Holler started with his series to >> make probing order deterministic, it occurred to me that it should be >> possible >> to achieve the same by registering devices as they are referenced by other >> devices. > > This is pretty cool, but a too local solution to a global problem. > > Deferred probe and initcall reordering, silly as they may seem, > does not require you to use device tree. > > The real solution, which I think I pointed out already when we > added deferred probe, is to put dependency graphs in the drivers > and have the kernel device driver core percolate dependecies by > walking the graph on probing driver, removing driver (usually the > inverse use case), [runtime] suspend and [runtime] resumeing > a driver. Possibly the dependencies will even be different > depending on use case. > > This is what systemd is doing in userspace for starting services: > ask for your dependencies and wait for them if they are not > there. So drivers ask for resources and wait for them. It also > needs to be abstract, so for example we need to be able to > hang on regulator_get() until the driver is up and providing that > regulator, and as long as everything is in slowpath it should > be OK. (And vice versa mutatis mutandis for clk, gpio, pin > control, interrupts (!) and DMA channels for example.) > > > So if this should be solved it should be solved in an abstract way > in the device driver core available for all, then have calls calling > out to DT, ACPI, possibly even PCI or USB (as these > enumerate devices themselves) to obtain a certain > dependency. I suggest to start with making it possible to identify (at least most) drivers. I've already posted a patch for that around a year ago and now Tomeu did almost the same. However one wants to make a deterministic order to load drivers, there will be always the need to know which drivers one has to sort. Regards, Alexander Holler
[PATCH 00/21] On-demand device registration
On 2 June 2015 at 10:48, Linus Walleij wrote: > On Mon, May 25, 2015 at 4:53 PM, Tomeu Vizoso > wrote: > >> have looked into ordered probing as a >> better way of solving this than moving nodes around in the DT or playing with >> initcall levels. >> >> While reading the thread [1] that Alexander Holler started with his series to >> make probing order deterministic, it occurred to me that it should be >> possible >> to achieve the same by registering devices as they are referenced by other >> devices. > > This is pretty cool, but a too local solution to a global problem. > > Deferred probe and initcall reordering, silly as they may seem, > does not require you to use device tree. > > The real solution, which I think I pointed out already when we > added deferred probe, is to put dependency graphs in the drivers By this you mean something like what Thierry suggested here? http://article.gmane.org/gmane.linux.kernel/1774623 > and have the kernel device driver core percolate dependecies by > walking the graph on probing driver, removing driver (usually the > inverse use case), [runtime] suspend and [runtime] resumeing > a driver. Possibly the dependencies will even be different > depending on use case. > > This is what systemd is doing in userspace for starting services: > ask for your dependencies and wait for them if they are not > there. So drivers ask for resources and wait for them. It also > needs to be abstract, so for example we need to be able to > hang on regulator_get() until the driver is up and providing that > regulator, and as long as everything is in slowpath it should > be OK. (And vice versa mutatis mutandis for clk, gpio, pin > control, interrupts (!) and DMA channels for example.) I understood above that you propose probing devices in order, but now you mention that resource getters would block until the dependency is fulfilled which confuses me because if we are probing in order then all dependencies would be fulfilled before the device in question gets probed. > So if this should be solved it should be solved in an abstract way > in the device driver core available for all, then have calls calling > out to DT, ACPI, possibly even PCI or USB (as these > enumerate devices themselves) to obtain a certain > dependency. Yeah, I was planning looking into this now that I got it working with async probing. Thanks, Tomeu > Yours, > Linus Walleij > ___ > dri-devel mailing list > dri-devel at lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/dri-devel
[PATCH 00/21] On-demand device registration
On Mon, May 25, 2015 at 4:53 PM, Tomeu Vizoso wrote: > have looked into ordered probing as a > better way of solving this than moving nodes around in the DT or playing with > initcall levels. > > While reading the thread [1] that Alexander Holler started with his series to > make probing order deterministic, it occurred to me that it should be possible > to achieve the same by registering devices as they are referenced by other > devices. This is pretty cool, but a too local solution to a global problem. Deferred probe and initcall reordering, silly as they may seem, does not require you to use device tree. The real solution, which I think I pointed out already when we added deferred probe, is to put dependency graphs in the drivers and have the kernel device driver core percolate dependecies by walking the graph on probing driver, removing driver (usually the inverse use case), [runtime] suspend and [runtime] resumeing a driver. Possibly the dependencies will even be different depending on use case. This is what systemd is doing in userspace for starting services: ask for your dependencies and wait for them if they are not there. So drivers ask for resources and wait for them. It also needs to be abstract, so for example we need to be able to hang on regulator_get() until the driver is up and providing that regulator, and as long as everything is in slowpath it should be OK. (And vice versa mutatis mutandis for clk, gpio, pin control, interrupts (!) and DMA channels for example.) So if this should be solved it should be solved in an abstract way in the device driver core available for all, then have calls calling out to DT, ACPI, possibly even PCI or USB (as these enumerate devices themselves) to obtain a certain dependency. Yours, Linus Walleij
[PATCH 00/21] On-demand device registration
On Mon, May 25, 2015 at 9:53 AM, Tomeu Vizoso wrote: > Hello, > > I have a problem with the panel on my Tegra Chromebook taking longer than > expected to be ready during boot (Stéphane Marchesin reported what is > basically the same issue in [0]), and have looked into ordered probing as a > better way of solving this than moving nodes around in the DT or playing with > initcall levels. > > While reading the thread [1] that Alexander Holler started with his series to > make probing order deterministic, it occurred to me that it should be possible > to achieve the same by registering devices as they are referenced by other > devices. I like the concept and novel approach. > This basically reuses the information that is already implicit in the probe() > implementations, saving us from refactoring existing drivers or adding > information to DTBs. > > Something I'm not completely happy with is that I have had to move the call to > of_platform_populate after all platform drivers have been registered. > Otherwise I don't see how I could register drivers on demand as we don't have > yet each driver's compatible strings. Yeah, this is the opposite of what we'd really like. Ideally, we would have a solution that works for modules too. However, we're no worse off. We pretty much build-in dependencies to avoid module ordering problems. Perhaps we need to make the probing on-demand rather than simply on device<->driver match occurring. > For machs that don't move of_platform_populate() to a later point, these > patches shouldn't cause any problems but it's not guaranteed that we'll avoid > all the deferred probes as some drivers may not be registered yet. Ideally, of_platform_populate is not explicitly called by each platform. So I think we need to make this work for the default case. > I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these > patches were enough to eliminate all the deferred probes. > > With this series I get the kernel to output to the panel in 0.5s, instead of > 2.8s. That's certainly compelling. Rob > > Regards, > > Tomeu > > [0] http://lists.freedesktop.org/archives/dri-devel/2014-August/066527.html > > [1] https://lkml.org/lkml/2014/5/12/452 > > Tomeu Vizoso (21): > regulator: core: Reduce critical area in _regulator_get > ARM: tegra: Add gpio-ranges property > ARM: tegra: Register drivers before devices > ARM: EXYNOS: Register drivers before devices > ARM i.MX6q: Register drivers before devices > of/platform: Add of_platform_device_ensure() > of/platform: Ensure device registration on lookup > gpio: Probe GPIO drivers on demand > gpio: Probe pinctrl devices on demand > regulator: core: Probe regulators on demand > drm: Probe panels on demand > drm/tegra: Probe dpaux devices on demand > i2c: core: Probe i2c master devices on demand > pwm: Probe PWM chip devices on demand > backlight: Probe backlight devices on demand > usb: phy: Probe phy devices on demand > clk: Probe clk providers on demand > pinctrl: Probe pinctrl devices on demand > phy: core: Probe phy providers on demand > dma: of: Probe DMA controllers on demand > power-supply: Probe power supplies on demand > > arch/arm/boot/dts/tegra124.dtsi | 1 + > arch/arm/mach-exynos/exynos.c | 4 +-- > arch/arm/mach-imx/mach-imx6q.c | 12 - > arch/arm/mach-tegra/tegra.c | 21 ++- > drivers/clk/clk.c | 3 +++ > drivers/dma/of-dma.c| 3 +++ > drivers/gpio/gpiolib-of.c | 5 > drivers/gpu/drm/drm_panel.c | 3 +++ > drivers/gpu/drm/tegra/dpaux.c | 3 +++ > drivers/i2c/i2c-core.c | 3 +++ > drivers/of/platform.c | 53 > + > drivers/phy/phy-core.c | 3 +++ > drivers/pinctrl/devicetree.c| 2 ++ > drivers/power/power_supply_core.c | 3 +++ > drivers/pwm/core.c | 3 +++ > drivers/regulator/core.c| 45 +++ > drivers/usb/phy/phy.c | 3 +++ > drivers/video/backlight/backlight.c | 3 +++ > include/linux/of_platform.h | 2 ++ > 19 files changed, 130 insertions(+), 45 deletions(-) > > -- > 2.4.1 >
[PATCH 00/21] On-demand device registration
Hello, I have a problem with the panel on my Tegra Chromebook taking longer than expected to be ready during boot (Stéphane Marchesin reported what is basically the same issue in [0]), and have looked into ordered probing as a better way of solving this than moving nodes around in the DT or playing with initcall levels. While reading the thread [1] that Alexander Holler started with his series to make probing order deterministic, it occurred to me that it should be possible to achieve the same by registering devices as they are referenced by other devices. This basically reuses the information that is already implicit in the probe() implementations, saving us from refactoring existing drivers or adding information to DTBs. Something I'm not completely happy with is that I have had to move the call to of_platform_populate after all platform drivers have been registered. Otherwise I don't see how I could register drivers on demand as we don't have yet each driver's compatible strings. For machs that don't move of_platform_populate() to a later point, these patches shouldn't cause any problems but it's not guaranteed that we'll avoid all the deferred probes as some drivers may not be registered yet. I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these patches were enough to eliminate all the deferred probes. With this series I get the kernel to output to the panel in 0.5s, instead of 2.8s. Regards, Tomeu [0] http://lists.freedesktop.org/archives/dri-devel/2014-August/066527.html [1] https://lkml.org/lkml/2014/5/12/452 Tomeu Vizoso (21): regulator: core: Reduce critical area in _regulator_get ARM: tegra: Add gpio-ranges property ARM: tegra: Register drivers before devices ARM: EXYNOS: Register drivers before devices ARM i.MX6q: Register drivers before devices of/platform: Add of_platform_device_ensure() of/platform: Ensure device registration on lookup gpio: Probe GPIO drivers on demand gpio: Probe pinctrl devices on demand regulator: core: Probe regulators on demand drm: Probe panels on demand drm/tegra: Probe dpaux devices on demand i2c: core: Probe i2c master devices on demand pwm: Probe PWM chip devices on demand backlight: Probe backlight devices on demand usb: phy: Probe phy devices on demand clk: Probe clk providers on demand pinctrl: Probe pinctrl devices on demand phy: core: Probe phy providers on demand dma: of: Probe DMA controllers on demand power-supply: Probe power supplies on demand arch/arm/boot/dts/tegra124.dtsi | 1 + arch/arm/mach-exynos/exynos.c | 4 +-- arch/arm/mach-imx/mach-imx6q.c | 12 - arch/arm/mach-tegra/tegra.c | 21 ++- drivers/clk/clk.c | 3 +++ drivers/dma/of-dma.c| 3 +++ drivers/gpio/gpiolib-of.c | 5 drivers/gpu/drm/drm_panel.c | 3 +++ drivers/gpu/drm/tegra/dpaux.c | 3 +++ drivers/i2c/i2c-core.c | 3 +++ drivers/of/platform.c | 53 + drivers/phy/phy-core.c | 3 +++ drivers/pinctrl/devicetree.c| 2 ++ drivers/power/power_supply_core.c | 3 +++ drivers/pwm/core.c | 3 +++ drivers/regulator/core.c| 45 +++ drivers/usb/phy/phy.c | 3 +++ drivers/video/backlight/backlight.c | 3 +++ include/linux/of_platform.h | 2 ++ 19 files changed, 130 insertions(+), 45 deletions(-) -- 2.4.1
