RE: [OT, JOB] Xilinx, San Jose: 2 full-time openings for embedded systems/embedded software group
On Wednesday, June 18, 2014 5:33 PM, Wojciech Koszek wrote: Hello, [This isn't strictly Linux-related, I know, I know. But I assumed some of you may like embedded systems. We're the same group as guys who did Linux for Xilinx FPGA SoC (Zynq). We also use a lot of Open-Source stuff and look for people with open-source code reading/writing skills] Xilinx (San Jose, California) has 2 full-time openings for people interested in software development, embedded systems, SoC chips, ARM architecture and FPGAs. Ideal candidate would love seeing hardware and software cooperate. As a member of our team you'll spend most of your time with Vim/Emacs, designing, writing, testing and breaking our proprietary OS. It's written in BSD-style ANSI C and assembler, has 101k lines of code and is used across the company in all sorts of unusual ways. If that sounds interesting to you, here are the links: IRC104504: http://tinyurl.com/XILINX-IRC104504 IRC104645: http://tinyurl.com/XILINX-IRC104645 If you think you qualify, send an e-mail with attached resume in the PDF file to: pssa_hiring (at) xilinx.com E-mail should have a format: To: pssa_hiring (at) xilinx.com From: you@mail.domain Subject: IRC104504 or IRC104645 Otherwise it'll go to spam. It is NOT appropriate to use a kernel community mailing list for job solicitations. Especially for non-Linux jobs. There are numerous other mechanisms and settings for finding Linux and embedded engineers. Please use those in the future, and avoid using this mailing list. -- Tim Tim Bird, Senior Software Engineer, Sony Mobile Communications, Inc. Architecture Group Chair, CE Workgroup of the Linux Foundation-- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Why is the deferred initcall patch not mainline?
The answer is pretty easy, I think. I tried to mainline it once but failed, and didn't really try again. If it is being found useful, we should try to mainline it again, this time with more persistence. The reason it got rejected before IIRC was that you can accomplish a similar thing with modules, with no changes to the kernel. But that doesn't cover the case where the loadable modules feature of the kernel is turned off, which is common in very small systems. -- Tim Sent from my Sony smartphone on T-Mobile’s 4G LTE Network Dirk Behme wrote Hi, During the ELCE 2014 in Duesseldorf in Chris Hallinan's talk [1] there has been the unanswered question why the deferred initcall patch [2] isn't mainline, yet. Anybody remembers? Best regards Dirk [1] http://sched.co/1yG5fmY [2] http://elinux.org/Deferred_Initcalls -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why is the deferred initcall patch not mainline?
Alexandre Belloni wrote
Hi,
On 18/10/2014 at 10:11:27 +0200, Bird, Tim wrote :
The answer is pretty easy, I think. I tried to mainline it once but
failed, and didn't really try again. If it is being found useful, we
should try to mainline it again, this time with more persistence. The
reason it got rejected before IIRC was that you can accomplish a similar
thing with modules, with no changes to the kernel. But that doesn't cover
the case where the loadable modules feature of the kernel is turned off,
which is common in very small systems.
There is also the case of subsystems that can't be compiled as modules.
I didn't even try to push that to the mainline because I believe we
prefer not having code without any users/calls in the kernel. You would
still have to patch your kernel to use deferred_module_init().
It is also quite easy to port, maybe you can try to push it to mainline
or if you want I can try to send an updated patch myself.
I won't have time to get to it any time soon. I'm traveling this week and will
be swamped the next few weeks. If you send something, I certainly won't object.
-- Tim
Sent from my Sony smartphone
Dirk Behme wrote
Hi,
During the ELCE 2014 in Duesseldorf in Chris Hallinan's talk [1] there
has been the unanswered question why the deferred initcall patch [2]
isn't mainline, yet.
Anybody remembers?
Best regards
Dirk
[1] http://sched.co/1yG5fmY ;
[2] http://elinux.org/Deferred_Initcalls ;
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to
majord...@vger.kernel.orgmailto:majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html ;
--
Alexandre Belloni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
N�r��yb�X��ǧv�^�){.n�+{�zf�uם�{ay� ʇڙ�,j��f���h���z� �w���
���j:+v���w�j�mzZ+�ݢj��!�i
RE: Why is the deferred initcall patch not mainline?
I'm going to respond to several comments in this one message (sorry for the likely confusion) On Tuesday, October 21, 2014 9:31 AM, Nicolas Pitre [n...@fluxnic.net] wrote: On Tue, 21 Oct 2014, Grant Likely wrote: On Sat, Oct 18, 2014 at 9:11 AM, Bird, Tim tim.b...@sonymobile.com wrote: The answer is pretty easy, I think. I tried to mainline it once but failed, and didn't really try again. If it is being found useful, we should try to mainline it again, this time with more persistence. The reason it got rejected before IIRC was that you can accomplish a similar thing with modules, with no changes to the kernel. But that doesn't cover the case where the loadable modules feature of the kernel is turned off, which is common in very small systems. It is a rather clumsy approach though since it requires changes to modules and it makes the configuration static per build. Could it instead be done by the kernel accepting a list of initcalls that should be deferred? It would depend I suppose on the cost of finding the initcalls to defer at boot time. Yeah, I'm not a big fan of having to change kernel code in order to use the feature. I am quite intrigued by Geert Uytterhoeven's idea to add a 'D' option to the config system, so that the record of which modules to defer could be stored there. This is much better than hand-altering code. I don't know how difficult this would be to add to the kbuild system, but the mechanism for altering the macro would be, IMHO, very straightforward. I should say that it's been quite some time since I worked on this, so some of my recollections may be fuzzy. With regards to doing it dynamically, I'd have to think about how to do that. Having text-based lists of things to do at runtime seems to fit with how we're using device tree these days, but I'm not sure how that would work. The code as it stands now is quite simple, just creating a new linker section to hold the list of deferred function pointers, re-using all existing routines for processing such lists, doing a few code changes to handle actually deferring the initialization and memory free-ing, and finally creating a /proc entry to trigger the whole thing. In a modern kernel, the /proc trigger should definitely be moved to /sys. Other than this, though, if you move to some other system of processing the list, you will have to create new infrastructure for working through the deferred module list, or make a change in the way the items are handled in the generic init function pointer processing. A simple solution would be to just compare each item from each ...initcall.init section with a list of deferred functions, and not process them, until doing the deferred init. Note that the current technique uses the compiler and linker do some of the work for list aggregation and processing, so that would have to be replaced with something else if you do it differently. I missed the session unfortunately, are there some measurements available that I could look at? Which subsystems are typically the problem? I, too, would like to know more about the problem. Any pointers? Here is the elinux wiki page with some historical measurements: http://elinux.org/Deferred_Initcalls The example on the wiki page defers 2 USB modules, and it saved 530 milliseconds on an x86 system. This is consistent with what we saw on cameras at Sony. This patch predated Arjan Van de Ven's fastboot work. I don't know if some of his parallelization (asynchronous module loading), and optimizations for USB loading made things substantially better than this. The USB spec makes in impossible to avoid a certain amount of delay in probing the USB busses USB was the main culprit, but we sometimes deferred other modules, if they were not in the fastpath for taking a picture. Sony cameras had a goal of booting in .5 seconds, but I think the best we ever achieved was about 1.1 seconds, using deferred initcalls and a variety of other techniques. -- Tim -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Why is the deferred initcall patch not mainline?
On Wednesday, October 22, 2014 8:49 AM, Nicolas Pitre [n...@fluxnic.net] wrote: On Wed, 22 Oct 2014, Rob Landley wrote: On 10/21/14 14:58, Nicolas Pitre wrote: On Tue, 21 Oct 2014, Bird, Tim wrote: I'm going to respond to several comments in this one message (sorry for the likely confusion) On Tuesday, October 21, 2014 9:31 AM, Nicolas Pitre [n...@fluxnic.net] wrote: On Tue, 21 Oct 2014, Grant Likely wrote: On Sat, Oct 18, 2014 at 9:11 AM, Bird, Tim tim.b...@sonymobile.com wrote: The answer is pretty easy, I think. I tried to mainline it once but failed, and didn't really try again. If it is being found useful, we should try to mainline it again, this time with more persistence. The reason it got rejected before IIRC was that you can accomplish a similar thing with modules, with no changes to the kernel. But that doesn't cover the case where the loadable modules feature of the kernel is turned off, which is common in very small systems. It is a rather clumsy approach though since it requires changes to modules and it makes the configuration static per build. Could it instead be done by the kernel accepting a list of initcalls that should be deferred? It would depend I suppose on the cost of finding the initcalls to defer at boot time. Yeah, I'm not a big fan of having to change kernel code in order to use the feature. I am quite intrigued by Geert Uytterhoeven's idea to add a 'D' option to the config system, so that the record of which modules to defer could be stored there. This is much better than hand-altering code. I don't know how difficult this would be to add to the kbuild system, but the mechanism for altering the macro would be, IMHO, very straightforward. Straight forward but IMHO rather suboptimal. Sure it might be good enough if all you want is to ship products out the door, but for mainline something better should be done. This patch predated Arjan Van de Ven's fastboot work. I don't know if some of his parallelization (asynchronous module loading), and optimizations for USB loading made things substantially better than this. The USB spec makes in impossible to avoid a certain amount of delay in probing the USB busses USB was the main culprit, but we sometimes deferred other modules, if they were not in the fastpath for taking a picture. Sony cameras had a goal of booting in .5 seconds, but I think the best we ever achieved was about 1.1 seconds, using deferred initcalls and a variety of other techniques. Some initcalls can be executed in parallel, but they currently all have to complete before user space is started. It should be possible to still do the parallel initcall thing, and let user space run before they are done as well. Only waiting for the root fs to be available should be sufficient. That would be completely generic, and help embedded as well as desktop systems. What would actually be nice is if initramfs could read something out of /proc or /sys to check the status of initcalls. (Or maybe get notification through the hotplug netlink mechanism.) Since initramfs is _already_ up really early, before needing any particular drivers and way before the real root filesystem, we can trivially punt this sort of synchronization to userspace if userspace can just get the information about when kernel deferred processing is done. Possibly we already have this: /sys/module has directories for all the kernel modules including the ones built static, so possibly userspace can just wait for /sys/module/zlib_delfate/initstate to say live. It would just be nice to have a way to notice that's happened without spinning reading a file. Again, not generic enough. Instead, the reading of that file could be suspended by the kernel until all initcalls have completed and then return an appropriate error code if the corresponding resource is actually not there. Otherwise the standard hotplug notification mechanism is already available. I'm not sure why this attention to reading the status. The salient feature here is that the initializations are deferred until user space tells the kernel to proceed. It's the initiation of the trigger from user-space that matters. The whole purpose of this feature is to defer some driver initializations until the product can get into a state where it is already ready to perform it's primary function. Only user space knows when that is. There seems to be a desire to have an automatic mechanism for triggering the deferred initializations. I'm OK with this, as long as there's some reasonable use case for it. There are lots of possible trigger mechanisms, including just a simple timer, but I think it's important that the primary use case of 'trigger-when-user-space-says-to' is still supported. This code is really intended
RE: Why is the deferred initcall patch not mainline?
On Thursday, October 23, 2014 12:05 PM, Nicolas Pitre wrote: On Thu, 23 Oct 2014, Alexandre Belloni wrote: On 23/10/2014 at 13:56:44 -0400, Nicolas Pitre wrote : On Thu, 23 Oct 2014, Bird, Tim wrote: I'm not sure why this attention to reading the status. The salient feature here is that the initializations are deferred until user space tells the kernel to proceed. It's the initiation of the trigger from user-space that matters. The whole purpose of this feature is to defer some driver initializations until the product can get into a state where it is already ready to perform it's primary function. Only user space knows when that is. This is still a rather restrictive view of the problem IMHO. Let's step back a bit. Your concern is that some initcalls are taking too long and preventing user space from executing early, right? Well, not exactly. That is not the exact problem we're trying to solve, although it is close. The problem is not that users-space doesn't start early enough, per se, it's that there are a set of drivers statically linked to the kernel that are not needed until after (possibly well after) user space starts. Any cycles whatsoever being spent on those drivers (either in their initialization routines, or in processing them or scheduling them) impairs the primary function of the device. On a very old presentation I gave on this, the use case I gave was getting a picture of a baby's smile. USB drivers are NOT needed for this, but they *are* needed for full product operation. In some cases, the system may want to defer initialization of some drivers until explicit action through the user interface. So the trigger may not be called until well after boot is completed. I'm suggesting that they no longer prevent user space from executing earlier. Why would you then still want an explicit trigger from user space? Because only the user space knows when it is now OK to initialize those drivers, and begin using CPU cycles on them. There seems to be a desire to have an automatic mechanism for triggering the deferred initializations. I'm OK with this, as long as there's some reasonable use case for it. There are lots of possible trigger mechanisms, including just a simple timer, but I think it's important that the primary use case of 'trigger-when-user-space-says-to' is still supported. Why a trigger? I'm suggesting no trigger at all is needed. Let all initcalls start initializing whenever they can. Simply that they shouldn't prevent user space from running early. Because initcalls are running in parallel, then they must be using separate kernel threads. It may be possible to adjust their priority so if one of them is actually using a lot of CPU cycles then it will run only when all the other threads (including user space) are no longer running. You probably can't do that without introducing race conditions. A number of userspace libraries and script are actually expecting init and probe to be synchronous. They already have to cope with the fact that most things can be available through not-yet-loaded modules, or may never be there at all. If not then they should be fixed. And if you do rely on such a feature for your small embedded system then you won't have that many libs and scripts to fix. I will refer to the async probe discussion and the following thread: http://thread.gmane.org/gmane.linux.kernel/1781529 I still don't think that is a good idea at all. This async probe concept requires a trigger from user space and that opens many cans of worms as user space now has to be aware of specific kernel driver modules, their ordering dependencies, etc. My point is simply not to defer any initialization at all. This way you don't have to select which module or initcall to send a trigger for later on. If you are going to avoid having a sub-set of modules consume CPU cycles in early boot, you're going to have to identify them somehow. How do you propose to enumerate the modules to defer (or de-prioritize, as the case may be)? Note that this solution should work on UP systems, were there is essentially a zero-sum game on using CPU cycles at boot. Once again, what is the actual problem you want to solve? If it is about making sure user space can execute ASAP then _that_ should be the topic, not figuring out how to implement a particular solution. See above. The actual problem is that we want some sub-set of statically linked drivers to not consume any cycles during a period of time defined by user space. This is rather trivial to accomplish with modules, and the proposed implementation tries to provide similar functionality for a statically linked kernel. I'm open to discussing solutions other than the particular implementation proposed, just not ones that don't actually solve that problem
