Re: Why is the deferred initcall patch not mainline?
On Thu, 30 Oct 2014, Geert Uytterhoeven wrote: On Thu, Oct 30, 2014 at 12:49 AM, Tim Bird tim.b...@sonymobile.com wrote: The way the feature is expressed in the current code is that a set of drivers are marked for deferred initialization (I'll refer to this as issue 0). Then, at boot: 1) most drivers are initialized normally, 2) user space is started, and then 3) user space indicates to the kernel that the deferred drivers should be initialized. One (IMHO important) point in the current implementation is that the call to free_initmem() is also delayed until after initialization of the deferred drivers. This is different from modular drivers, which are loaded after free_initmem(). This is because modules have their __initmem sections freed right after each module is initialized. The deferred initcalls could also have a separate initmem section which freeing is also deferred. But I don't think it makes such a big difference in the end. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why is the deferred initcall patch not mainline?
On Wed, 29 Oct 2014, Tim Bird wrote: I have been thinking about the points you made previously, and have given the problem space some more thought. I agree that as it stands this is a very niche solution, and it would be good to think about the broader picture and how things might be designed differently to make the feature usable more easily and to a broader group. Taking a step back, the overall goal is to allow user space to do stuff while the kernel is still initializing statically linked drivers, so the device's primary function can be ready as soon as possible (and not wait for secondarily-needed functionality to initialize). For things that are able to be made into a module (and for situations where the kernel module loading is turned on), this feature should not be needed in its current form. In that case, user space already has control over module load ordering and timing. The way the feature is expressed in the current code is that a set of drivers are marked for deferred initialization (I'll refer to this as issue 0). Then, at boot: 1) most drivers are initialized normally, 2) user space is started, and then 3) user space indicates to the kernel that the deferred drivers should be initialized. This is very coarse, allowing only two categories of drivers: (ignoring other boot phases for the moment) - regular drivers and deferred drivers. It also requires source code changes to mark the drivers to be deferred. Finally, it requires an explicit notification from user-space to complete the process. All of these attributes are undesirable. There may also be an opportunity here to work out more granular driver load ordering, which would benefit other systems (especially those that are hitting the EPROBE_DEFER issue). As it stands now, the ordering of the initcalls within a particular level is pretty much arbitrary (determined by link order, usually without oversight by the developer). Just FYI, here are some numbers culled from a recent kernel: initcall macronumber of instances in kernel source -- early_init446 core_init 614 postcore_init 150 arch_init 751 subsys_init 573 fs_init 1372 device_init 1211 late_init 440 Did you count module_init instances which are folded into the device_init leven when built-in? I'm going to rattle off a few ideas - I'm not sure which ones might stick, I just want to bounce these around and see what people think. Note that I didn't think of most of these, but I'm just repeating ones that have been stated, and adding a few thoughts of my own. First, if the ordering of initialization is not the default provided by the kernel, it needs to be recorded somewhere. A developer needs to express it (or a tool needs to figure it out), but if it is going to be reflected in the final kernel behaviour (or image), the kernel needs it at boot time (if not compile time). The current initcall system hardcodes a level for each driver initialization routine in the source code itself, by putting it in the macro name for each init routine. There can only be one such order expressed in the code itself. For developers who wish to express another order (or priority), a new mechanism will need to be used. If possible, I strongly prefer putting this into the KCONFIG system, as that is where other details about kernel configuration are stored, and there are pre-existing tools for dealing with the format. I am hesitant to create a special language or config format for this (unless it is much simpler than adding something to Kconfig). As Nicolas pointed out, Kconfig already has information about dependencies in terms of not allowing a driver to be a module if a dependent module is statically linked. Having the tool warn for violations of that ordering would be valuable. I think you're confusing two issues: ordering and dependency. The dependency affects some of the ordering, but only a small portion of it. Within an initcall level the ordering is a result of the link order and therefore rather arbitrary. IMHO the current initcall level system is simply too simple for the current kernel complexity. The number of levels, and especially their names, are also completely arbitrary. It probably made sense back when initcalls were introduced, but it is just too inflexible now. Initcalls should instead be turned into targets and prerequisites, just like dependencies in a makefile. This way, the ultimate target execute /sbin/init in userspace could indicate its prerequisite as mount root fs. Then mount root fs could have USB storage as a prerequisite depending on the boot args. From USB storage you could have two prerequisites: USB stack and USB device enumeration. And so down to the very first initcalls with no
Re: Why is the deferred initcall patch not mainline?
On Fri, 24 Oct 2014, Geert Uytterhoeven wrote: Several patches are linked from http://elinux.org/Deferred_Initcalls Latest version is http://elinux.org/images/5/51/0001-Port-deferred-initcalls-to-3.10.patch In the hope of providing some constructive and concrete feedback to this thread, here's what I have to say about the patch linked above ( I looked only at the latest version): - Commented out code is not acceptable for mainline. But everyone knows that already. - Returning a null byte through the /proc file is dubious. - The /proc interface is probably not the best. I'd go with an entry in /sys/kernel instead. - If the deferred_initcall section is empty, this could return 1 upfront and do the free_initmem() earlier as it used to. - It was mentioned somewhere that the config system could use a 4th state in addition to n, m and y. That would be required before this goes upstream simply to express all the dependencies between modules. Right now if a core module is configured with m, then all the submodules that depend on it inherit the modular-only restriction. The same would need to be enforced for deferred initcalls. - Currently all deferred initcalls are lumped together in a single section with no regards to the original initcall level. This is likely to cause trouble if two initcalls are called in a different order than intended. Nothing prevents that from happening right now. This patch is still not generic enough for mainline inclusion IMHO. It currently falls in the you better know what you're doing category and that is possibly good enough for its actual users. Trying to make this more generic is going to require some more work. And this would have to come with serious arguments explaining why simply using modules in the first place is not acceptable. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why is the deferred initcall patch not mainline?
On Fri, 24 Oct 2014, Rob Landley wrote: On 10/23/14 19:36, Nicolas Pitre wrote: As you know already, you can do anything you want on your own. That's granted by the GPL. I'm pretty sure I could have done anything I wanted on my own with System 6 unix in the 1970's (modulo being 7 years old), since the BSD guys _did_ and their stuff is still around (and is powering obscure things like the iPhone). Incidentally there is this thing called Linux powering similarly obscur curiosities such as Android, and outnumbering iPhones in terms of units shipped. So what's your point again? And I learned C in 1989 to apply mod files to the WWIV bulletin board system (an open source development community that didn't even have the patch program). You needed to pay a license to get the WWIV source code. At least that was the case when I was a sysop in 1993. But by all means, credit the GPL for the existence of open source. Oh my! Obviously that's exactly what I did, right? And now you want me to take what you say seriously? The impression I get from your diatribe is that you might be living in the past. I don't dispute the fact that You had issues with the Linux community before, but one has to admit that a _lot_ of people don't. And I'm lucky enough to be one of them, and in that context I was trying to help. Did you notice that there's no such thing as the GPL anymore? Linux and Samba implement two ends of the same protocol, each one is GPL, and they can't share code. Poor QEMU wants to suck GPL processor definitions out of binutils/gdb to emulate processors and GPL driver code out of Linux to emulate devices, and there _is_ no license that allows it to combine code from both sources. (Making qemu GPLv2 or later means it couldn't accept code from _either_ source.) And now we're far far away from $subject that started this thread. This is going nowhere. I'm going to recuse myself from the rest of this thread because I'm clearly getting annoyed with us talking past each other. Somebody's got an actual patch (which they still haven't linked to). I'll shut up and let them show you the code. On that I agree with you. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Why is the deferred initcall patch not mainline?
On Thu, 23 Oct 2014, Bird, Tim wrote: I'm not sure why this attention to reading the status. The salient feature here is that the initializations are deferred until user space tells the kernel to proceed. It's the initiation of the trigger from user-space that matters. The whole purpose of this feature is to defer some driver initializations until the product can get into a state where it is already ready to perform it's primary function. Only user space knows when that is. This is still a rather restrictive view of the problem IMHO. Let's step back a bit. Your concern is that some initcalls are taking too long and preventing user space from executing early, right? I'm suggesting that they no longer prevent user space from executing earlier. Why would you then still want an explicit trigger from user space? There seems to be a desire to have an automatic mechanism for triggering the deferred initializations. I'm OK with this, as long as there's some reasonable use case for it. There are lots of possible trigger mechanisms, including just a simple timer, but I think it's important that the primary use case of 'trigger-when-user-space-says-to' is still supported. Why a trigger? I'm suggesting no trigger at all is needed. Let all initcalls start initializing whenever they can. Simply that they shouldn't prevent user space from running early. Because initcalls are running in parallel, then they must be using separate kernel threads. It may be possible to adjust their priority so if one of them is actually using a lot of CPU cycles then it will run only when all the other threads (including user space) are no longer running. This code is really intended for a very specialized kernel configuration, where all the modules are statically linked, and indeed module loading itself is turned off. I think that's a minority of Linux deployments out there. This configuration implies some other attributes, like configuration for very small size and/or very fast boot, where KALLSYMS may not be present, and other kernel features may not be available as well. Indeed, in the smallest systems /proc or /sys may not be there, so an alternative (maybe a sysctl or even a new syscall) might be appropriate. Quite frankly, the hacky way this is often done is to make stuff like this a one-time side effect of a rarely called syscall (like sync). Please note I'm not recommending this for mainline, just pointing out there are interesting ways that embedded developers just make the existing code work for their weird cases. Agreed. However if you're looking for a solution that may go into mainline, it just can't be hackish like that. There might be generic solutions that meet your goal while still being useful to others. Focussing on the best way to implement a particular solution while there might be other solutions to explore is a bad approach. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why is the deferred initcall patch not mainline?
On Thu, 23 Oct 2014, Alexandre Belloni wrote: On 23/10/2014 at 13:56:44 -0400, Nicolas Pitre wrote : On Thu, 23 Oct 2014, Bird, Tim wrote: I'm not sure why this attention to reading the status. The salient feature here is that the initializations are deferred until user space tells the kernel to proceed. It's the initiation of the trigger from user-space that matters. The whole purpose of this feature is to defer some driver initializations until the product can get into a state where it is already ready to perform it's primary function. Only user space knows when that is. This is still a rather restrictive view of the problem IMHO. Let's step back a bit. Your concern is that some initcalls are taking too long and preventing user space from executing early, right? I'm suggesting that they no longer prevent user space from executing earlier. Why would you then still want an explicit trigger from user space? There seems to be a desire to have an automatic mechanism for triggering the deferred initializations. I'm OK with this, as long as there's some reasonable use case for it. There are lots of possible trigger mechanisms, including just a simple timer, but I think it's important that the primary use case of 'trigger-when-user-space-says-to' is still supported. Why a trigger? I'm suggesting no trigger at all is needed. Let all initcalls start initializing whenever they can. Simply that they shouldn't prevent user space from running early. Because initcalls are running in parallel, then they must be using separate kernel threads. It may be possible to adjust their priority so if one of them is actually using a lot of CPU cycles then it will run only when all the other threads (including user space) are no longer running. You probably can't do that without introducing race conditions. A number of userspace libraries and script are actually expecting init and probe to be synchronous. They already have to cope with the fact that most things can be available through not-yet-loaded modules, or may never be there at all. If not then they should be fixed. And if you do rely on such a feature for your small embedded system then you won't have that many libs and scripts to fix. I will refer to the async probe discussion and the following thread: http://thread.gmane.org/gmane.linux.kernel/1781529 I still don't think that is a good idea at all. This async probe concept requires a trigger from user space and that opens many cans of worms as user space now has to be aware of specific kernel driver modules, their ordering dependencies, etc. My point is simply not to defer any initialization at all. This way you don't have to select which module or initcall to send a trigger for later on. Once again, what is the actual problem you want to solve? If it is about making sure user space can execute ASAP then _that_ should be the topic, not figuring out how to implement a particular solution. Anyway, your userspace will have to have a way to know what has been initialized. Hotplug notifications via dbus. On my side, I was also using that mechanism to delay the network stack init but I still want to know when my dhcp client can start for example. Ditto. And not only do you want to know when the network stack is initialized, but you also need to wait for a link to be established before DHCP can work. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Why is the deferred initcall patch not mainline?
On Thu, 23 Oct 2014, Bird, Tim wrote: On Thursday, October 23, 2014 12:05 PM, Nicolas Pitre wrote: On Thu, 23 Oct 2014, Alexandre Belloni wrote: On 23/10/2014 at 13:56:44 -0400, Nicolas Pitre wrote : On Thu, 23 Oct 2014, Bird, Tim wrote: I'm not sure why this attention to reading the status. The salient feature here is that the initializations are deferred until user space tells the kernel to proceed. It's the initiation of the trigger from user-space that matters. The whole purpose of this feature is to defer some driver initializations until the product can get into a state where it is already ready to perform it's primary function. Only user space knows when that is. This is still a rather restrictive view of the problem IMHO. Let's step back a bit. Your concern is that some initcalls are taking too long and preventing user space from executing early, right? Well, not exactly. That is not the exact problem we're trying to solve, although it is close. The problem is not that users-space doesn't start early enough, per se, it's that there are a set of drivers statically linked to the kernel that are not needed until after (possibly well after) user space starts. Any cycles whatsoever being spent on those drivers (either in their initialization routines, or in processing them or scheduling them) impairs the primary function of the device. On a very old presentation I gave on this, the use case I gave was getting a picture of a baby's smile. USB drivers are NOT needed for this, but they *are* needed for full product operation. As I suggested earlier, those cycles spent on those drivers may be deferred to a moment when the CPU has nothing else to do anyway by giving a lower priority to the threads handling them. In some cases, the system may want to defer initialization of some drivers until explicit action through the user interface. So the trigger may not be called until well after boot is completed. In that case the trigger for initializing those drivers should be the first time they're accessed from user space. That could be the very first time libusb or similar tries to enumerate available USB devices for example. No special interface needed. I'm suggesting that they no longer prevent user space from executing earlier. Why would you then still want an explicit trigger from user space? Because only the user space knows when it is now OK to initialize those drivers, and begin using CPU cycles on them. So what? That is still not a good answer. User space shouldn't have to care as long as it has all the CPU cycles it wants in priority. But as soon as user space relinquishes the CPU then there is no reason why driver initialization couldn't take over until user space is made runnable again. [...] My point is simply not to defer any initialization at all. This way you don't have to select which module or initcall to send a trigger for later on. If you are going to avoid having a sub-set of modules consume CPU cycles in early boot, you're going to have to identify them somehow. How do you propose to enumerate the modules to defer (or de-prioritize, as the case may be)? Anything that is not involved with making the root fs available. Note that this solution should work on UP systems, were there is essentially a zero-sum game on using CPU cycles at boot. The scheduler knows how to prioritize things on UP as well. The top priority thread will always go to sleep at some point allowing other threads to run. But I'm sure you know all that. Once again, what is the actual problem you want to solve? If it is about making sure user space can execute ASAP then _that_ should be the topic, not figuring out how to implement a particular solution. See above. The actual problem is that we want some sub-set of statically linked drivers to not consume any cycles during a period of time defined by user space. Once again you're defining a solution (i.e. not consume any cycles ...) rather than the problem motivating this particular solution. That's not how you're going to have something merged upstream. And I'm not saying your solution is completely bad either if you're looking for the simplest way and willing to keep it to yourself. What I'm saying is that there are other possible solutions that could solve your initial problem _and_ be acceptable to mainline... but they're unlikely to look like what you have now. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why is the deferred initcall patch not mainline?
On Thu, 23 Oct 2014, Rob Landley wrote: On 10/23/14 14:05, Nicolas Pitre wrote: On Thu, 23 Oct 2014, Alexandre Belloni wrote: On 23/10/2014 at 13:56:44 -0400, Nicolas Pitre wrote : On Thu, 23 Oct 2014, Bird, Tim wrote: Why a trigger? I'm suggesting no trigger at all is needed. Let all initcalls start initializing whenever they can. Simply that they shouldn't prevent user space from running early. Because initcalls are running in parallel, then they must be using separate kernel threads. It may be possible to adjust their priority so if one of them is actually using a lot of CPU cycles then it will run only when all the other threads (including user space) are no longer running. You probably can't do that without introducing race conditions. A number of userspace libraries and script are actually expecting init and probe to be synchronous. They already have to cope with the fact that most things can be available through not-yet-loaded modules, or may never be there at all. If not then they should be fixed. And if you do rely on such a feature for your small embedded system then you won't have that many libs and scripts to fix. There are userspace libraries distinguishing between init and probe? I.E. treating them as two separate things already? Why not? So how were they accessing them as two separate things before this patch set? Before engaging a conversation with a device, you verify if it exists first? I will refer to the async probe discussion and the following thread: http://thread.gmane.org/gmane.linux.kernel/1781529 I still don't think that is a good idea at all. This async probe concept requires a trigger from user space and that opens many cans of worms as user space now has to be aware of specific kernel driver modules, their ordering dependencies, etc. My point is simply not to defer any initialization at all. This way you don't have to select which module or initcall to send a trigger for later on. Why would this be hard? for i in $(find /sys/module -name initstate) do [ $(cat $i) != live ] echo kick $i done You should have a look at /sys/bus/*/*probe then. Maybe it does what you need already. And I'm confused that you're concerned about init order so your solution is to do nothing, thereby preserving the existing init order which could not _possibly_ be exposed verbatim to userspace... The kernel already has the deferred probe mechanism to cope with the init ordering which, as experience has shown, may only be dealt with at run time. All attempts to create that ordering statically in the past have failed. So what do you want exposed verbatim to user space again? Once again, what is the actual problem you want to solve? If it is about making sure user space can execute ASAP then _that_ should be the topic, not figuring out how to implement a particular solution. Anyway, your userspace will have to have a way to know what has been initialized. Hotplug notifications via dbus. Wait, we need a _third_ mechanism for hotplug notifications now? (The /proc/sys/kernel/hotplug helper, netlink, and you want another one?) No, I actually meant hotplug and netlink. My bad. On my side, I was also using that mechanism to delay the network stack init but I still want to know when my dhcp client can start for example. Ditto. And not only do you want to know when the network stack is initialized, but you also need to wait for a link to be established before DHCP can work. Um, doesn't the existing hotplug mechanism _already_ give us notification that eth0 and similar showed up? (Pretty sure I hit that while poking at mdev, although it was a while ago...) Indeed it does. So no new user space notification mechanisms are needed which is my point. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why is the deferred initcall patch not mainline?
On Thu, 23 Oct 2014, Rob Landley wrote: Doing hardware probing at low priorities can cause really _fun_ latency spikes in the system as something grabs a lock and then sleeps. (And doing this at the realtime scheduling where it won't do that translates those latency spikes into the aforementioned hard lockup, so not actually a solution per se.) Trying to fix this in the general case is the priority inheritance problem, and last I heard was really hard. Maybe it's been fixed in the past few years and I hadn't noticed. (The rise of SMP made it a less pressing issue, but system bringup is its own little world.) I know you're a smart *ss. But: 1) All this is not about fixing the RT scheduler for the general case. 2) System bring-up being its own world may have special scheduling treatment that doesn't necessarily have to be RT. 3) You, too, conveniently avoided to define the initial problem so far. That makes for rather sterile conversations about alternative solutions that could score higher on the mainline acceptance scale. In some cases, the system may want to defer initialization of some drivers until explicit action through the user interface. So the trigger may not be called until well after boot is completed. In that case the trigger for initializing those drivers should be the first time they're accessed from user space. Which gets us back to one of the big reasons strikesystemd/strike devfsd failed years ago: you have to probe the hardware in order to know which /dev nodes to create, so you can't have accessing the /dev node probe the hardware. (There's no /dev node for a usb controller...) There is /sys/bus/usb/devices that could be accessed in order to trigger the initial setup and probe. It is most likely that libusb does that, but this could be made to work with a simple 'cat' or 'touch' invocation as well. That could be the very first time libusb or similar tries to enumerate available USB devices for example. No special interface needed. So now you're requiring libusb enumerating usb devices, when before this you could just reach out and open /dev/ttyUSB0 and it would be there. You can't just reach out with the deferred initcall scheme either, do you? This is an embedded solution? I'm suggesting that they no longer prevent user space from executing earlier. Why would you then still want an explicit trigger from user space? Because only the user space knows when it is now OK to initialize those drivers, and begin using CPU cycles on them. So what? That is still not a good answer. Why? I believe Tim's proposal was to take a category of existing device probing, one already done on a background thread, and wait to start it until userspace says go. That's about as nonintrusive a change as you get. You might still be able to do better. If you really want to be non intrusive, you could e.g. make those background threads into SIGSTOP and let user space SIGCONT them as it sees fit. No new special interfaces needed. You're talking about requiring weird arbitrary things to have side effects. Like if stalling arbitrary initcalls wouldn't have side effects? What I'm suggesting is to let the system do its thing the most efficient way while giving a strong bias to running user space first. How arbitrarily weird can that be? If you're running in initramfs we haven't necessarily done _any_ driver probing yet. That's what initramfs is for. You can put device firmware in there so static drivers can make hotplug firmware loading requests to userspce during their device programming. (It's one of those usermode helper callback things.) True if you need firmware, or if you want to actually load modules to get to the root fs device. Otherwise all built-in driver init functions have been called and waited for at that point. Note that this solution should work on UP systems, were there is essentially a zero-sum game on using CPU cycles at boot. The scheduler knows how to prioritize things on UP as well. The top priority thread will always go to sleep at some point allowing other threads to run. But I'm sure you know all that. The top priority threads will get preempted. (Did you follow any of the work Con Kolivas and company were doing a few years ago?) Yeah... and I also notice it is still maintained, still out of mainline. As you know already, you can do anything you want on your own. That's granted by the GPL. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why is the deferred initcall patch not mainline?
On Wed, 22 Oct 2014, Rob Landley wrote: On 10/21/14 14:58, Nicolas Pitre wrote: On Tue, 21 Oct 2014, Bird, Tim wrote: I'm going to respond to several comments in this one message (sorry for the likely confusion) On Tuesday, October 21, 2014 9:31 AM, Nicolas Pitre [n...@fluxnic.net] wrote: On Tue, 21 Oct 2014, Grant Likely wrote: On Sat, Oct 18, 2014 at 9:11 AM, Bird, Tim tim.b...@sonymobile.com wrote: The answer is pretty easy, I think. I tried to mainline it once but failed, and didn't really try again. If it is being found useful, we should try to mainline it again, this time with more persistence. The reason it got rejected before IIRC was that you can accomplish a similar thing with modules, with no changes to the kernel. But that doesn't cover the case where the loadable modules feature of the kernel is turned off, which is common in very small systems. It is a rather clumsy approach though since it requires changes to modules and it makes the configuration static per build. Could it instead be done by the kernel accepting a list of initcalls that should be deferred? It would depend I suppose on the cost of finding the initcalls to defer at boot time. Yeah, I'm not a big fan of having to change kernel code in order to use the feature. I am quite intrigued by Geert Uytterhoeven's idea to add a 'D' option to the config system, so that the record of which modules to defer could be stored there. This is much better than hand-altering code. I don't know how difficult this would be to add to the kbuild system, but the mechanism for altering the macro would be, IMHO, very straightforward. Straight forward but IMHO rather suboptimal. Sure it might be good enough if all you want is to ship products out the door, but for mainline something better should be done. This patch predated Arjan Van de Ven's fastboot work. I don't know if some of his parallelization (asynchronous module loading), and optimizations for USB loading made things substantially better than this. The USB spec makes in impossible to avoid a certain amount of delay in probing the USB busses USB was the main culprit, but we sometimes deferred other modules, if they were not in the fastpath for taking a picture. Sony cameras had a goal of booting in .5 seconds, but I think the best we ever achieved was about 1.1 seconds, using deferred initcalls and a variety of other techniques. Some initcalls can be executed in parallel, but they currently all have to complete before user space is started. It should be possible to still do the parallel initcall thing, and let user space run before they are done as well. Only waiting for the root fs to be available should be sufficient. That would be completely generic, and help embedded as well as desktop systems. What would actually be nice is if initramfs could read something out of /proc or /sys to check the status of initcalls. (Or maybe get notification through the hotplug netlink mechanism.) Since initramfs is _already_ up really early, before needing any particular drivers and way before the real root filesystem, we can trivially punt this sort of synchronization to userspace if userspace can just get the information about when kernel deferred processing is done. Possibly we already have this: /sys/module has directories for all the kernel modules including the ones built static, so possibly userspace can just wait for /sys/module/zlib_delfate/initstate to say live. It would just be nice to have a way to notice that's happened without spinning reading a file. Again, not generic enough. Instead, the reading of that file could be suspended by the kernel until all initcalls have completed and then return an appropriate error code if the corresponding resource is actually not there. Otherwise the standard hotplug notification mechanism is already available. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why is the deferred initcall patch not mainline?
On Tue, 21 Oct 2014, Grant Likely wrote: On Sat, Oct 18, 2014 at 9:11 AM, Bird, Tim tim.b...@sonymobile.com wrote: The answer is pretty easy, I think. I tried to mainline it once but failed, and didn't really try again. If it is being found useful, we should try to mainline it again, this time with more persistence. The reason it got rejected before IIRC was that you can accomplish a similar thing with modules, with no changes to the kernel. But that doesn't cover the case where the loadable modules feature of the kernel is turned off, which is common in very small systems. It is a rather clumsy approach though since it requires changes to modules and it makes the configuration static per build. Could it instead be done by the kernel accepting a list of initcalls that should be deferred? It would depend I suppose on the cost of finding the initcalls to defer at boot time. I missed the session unfortunately, are there some measurements available that I could look at? Which subsystems are typically the problem? I, too, would like to know more about the problem. Any pointers? Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Why is the deferred initcall patch not mainline?
On Tue, 21 Oct 2014, Bird, Tim wrote: I'm going to respond to several comments in this one message (sorry for the likely confusion) On Tuesday, October 21, 2014 9:31 AM, Nicolas Pitre [n...@fluxnic.net] wrote: On Tue, 21 Oct 2014, Grant Likely wrote: On Sat, Oct 18, 2014 at 9:11 AM, Bird, Tim tim.b...@sonymobile.com wrote: The answer is pretty easy, I think. I tried to mainline it once but failed, and didn't really try again. If it is being found useful, we should try to mainline it again, this time with more persistence. The reason it got rejected before IIRC was that you can accomplish a similar thing with modules, with no changes to the kernel. But that doesn't cover the case where the loadable modules feature of the kernel is turned off, which is common in very small systems. It is a rather clumsy approach though since it requires changes to modules and it makes the configuration static per build. Could it instead be done by the kernel accepting a list of initcalls that should be deferred? It would depend I suppose on the cost of finding the initcalls to defer at boot time. Yeah, I'm not a big fan of having to change kernel code in order to use the feature. I am quite intrigued by Geert Uytterhoeven's idea to add a 'D' option to the config system, so that the record of which modules to defer could be stored there. This is much better than hand-altering code. I don't know how difficult this would be to add to the kbuild system, but the mechanism for altering the macro would be, IMHO, very straightforward. Straight forward but IMHO rather suboptimal. Sure it might be good enough if all you want is to ship products out the door, but for mainline something better should be done. This patch predated Arjan Van de Ven's fastboot work. I don't know if some of his parallelization (asynchronous module loading), and optimizations for USB loading made things substantially better than this. The USB spec makes in impossible to avoid a certain amount of delay in probing the USB busses USB was the main culprit, but we sometimes deferred other modules, if they were not in the fastpath for taking a picture. Sony cameras had a goal of booting in .5 seconds, but I think the best we ever achieved was about 1.1 seconds, using deferred initcalls and a variety of other techniques. Some initcalls can be executed in parallel, but they currently all have to complete before user space is started. It should be possible to still do the parallel initcall thing, and let user space run before they are done as well. Only waiting for the root fs to be available should be sufficient. That would be completely generic, and help embedded as well as desktop systems. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Expose system Serial Number to userspace.
On Wed, 29 Aug 2012, Brad Arnold wrote: Hi, I'm working on an embedded board which uses u-boot + linux. At manufacturing time, the device serial number will be programmed into OTP memory on NAND (probably from within u-boot). We'd like the linux kernel to make this serial number available to be read from userspace. Is there an accepted method to do this sort of thing? A serial number field is available from /proc/cpuinfo already. you just have to initialize it through the system_serial_high and system_serial_low global variables. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Embedded Linux Flag Version
On Tue, 9 Nov 2010, Nicholas Mc Guire wrote: On Tue, 09 Nov 2010, Tim Bird wrote: On 11/09/2010 03:19 AM, Mike Frysinger wrote: On Thu, Nov 4, 2010 at 18:07, Tim Bird wrote: It was noted at the summit that several CE companies and embedded projects will be using (or are already using) 2.6.35 for upcoming products or releases. This includes Sony, Google, Meego, and Linaro. On behalf of the CE Linux Forum and a number of consumer electronics companies, projects and community developers, we therefore declare 2.6.35 as a flag version of the kernel for embedded use. Several companies will be investing in development, integration and testing of this version. Entities wanting to do business with those companies would therefore be well-advised to make sure their hardware, drivers and enhancements work well with this version of the kernel. wouldnt it make more sense to piggy back the extensive work going into the stable tree ? many of the points you raise after all are the entire founding point of it. plus, all the main distros form around that, are spending time working on that, is marked as supported for 2 or 3 years, and there is already infrastructure/framework/process in place (sta...@kernel.org). so instead of picking arbitrary versions (like 2.6.35) and needlessly replicating the huge work load, simply declare these stable trees as the flag versions. that means today it'd be 2.6.32.y. The fact that this tree is already a year old, and not likely to be brought forward for at least another 2 years is the reason we didn't choose it this time. Most of the high-profile, active embedded projects are already on 2.6.35. For companies looking to adopt a new base kernel in the next 12 months, I don't want to have them start with a year-old kernel. We did consider the utility of synchronizing with the enterprise stable tree, but the timing just didn't work too well this time around. I guess one of the key issues is that it would need to be defined beforehand what version will be used as the next flag version so vendors could make sure that there drivers are available. Further if such a selection were to be made then there would need to be consesus on maintaining the respective version for a sufficiently long time to allow for the next flag version to evolve. Finally for those working on products even if you would now define the current version as the flag version it would not help much as they would hardly be able to shift to a different kernel short term - so again the key is defining what version would be atleast planed for the next flag version - something like now defining it to be 2.6.38 - and then testing appropriately in the runup to 2.6.38 to deliver, most notably for the embedded architectures. FWIW, the imminent Linaro release is using 2.6.35. The next release scheduled for May 2011 will most probably use 2.6.38. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Avoiding platform-specific callbacks in drivers?
On Tue, 9 Nov 2010, Bill Gatliff wrote: Let's say that on a given platform, I need to twiddle with a GPIO pin when a chip enters and exits suspend. What driver? What platform? This may depend on those. One way to do that is to hack the driver itself; a slightly less-inelegant way is to add a function pointer in the platform data, and have the driver call back in its suspend() and resume() methods. I'm not real keen on either strategy, because they both involve touching driver code that should be platform-agnostic. They seem... hacky. :) Sure. but if it makes sense for your hardware to do this GPIO twiddling, maybe someone else is doing that too. In which case a driver solution might be justifiable. I would love to come up with a way that prevents touching the driver at all, since the activity is terribly platform-specific. Is there such a way? One possibility is to set up some sort of parent-child relationship between the device and a pseudo-device that deals with the GPIO pin. But I'm not sure that will actually work, and it seems a bit overly-complicated. That would still be your best bet. The pseudo-device could register the real device and set itself as the parent. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Recommendation for activating a deferred module init in the kernel
On Wed, 18 Jun 2008, Geert Uytterhoeven wrote: On Wed, 18 Jun 2008, Adrian Bunk wrote: On Wed, Jun 18, 2008 at 10:59:50AM +0100, David Woodhouse wrote: On Wed, 2008-06-18 at 10:57 +0200, Geert Uytterhoeven wrote: On Wed, 18 Jun 2008, Adrian Bunk wrote: You miss the size increase imposed by CONFIG_MODULES=y. E.g. setting CONFIG_MODULES=y in the arm collie_defconfig will increase the size of vmlinux by 14% (sic). I haven't investigated why it takes that much space, but stuff like kernel/module.o taking 23kB and each EXPORT_SYMBOL requiring a few bytes simply cannot be completely eliminated. Sounds like we need a tool that strips out the unneeded symbols, given a list of modules? And do the --gc-sections step again after that... :) But even after all optimizations CONFIG_MODULES=y will still cause a significant additional cost [1] when thinking in the dimensions of Tim's the 30 or so Linux-tiny patches that I use get me about 110k of reductions. For me, this is about 5% of my kernel size statement in another thread. [1] as I said, kernel/module.o alone takes 23kB Which could be freed after the last module has been loaded? Even if you free the module loading infrastructure afterwards, you still have to carry it in flash for both the kernel part and the user part. And the whole process (having to sort out module loading in user space, etc.) is still more complex than not having to load modules at all. Many embedded designs have fixed hardware configuration and purpose which makes dynamic module loading an unnecessary bloat. Having the kernel span a parallel thread with a lower priority to initialize drivers could be a much simpler solution in terms of size and speed. This way the init process could run early, and devices would become available eventually just as if corresponding modules were loaded. The only difficulty is to properly determine how to distinguish between drivers that have to be initialized early in order to mount the root fs and the others that can wait. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
