Re: [PATCH 1/6] Add infrastructure for conditional code and data sections
On Sun, 2 Jan 2011, Paul Walmsley wrote: Hello Thomas On Tue, 21 Dec 2010, Thomas Petazzoni wrote: WARNING: This is only a proof-of-concept, there are many known issues. The sole purpose of this patch is to get some feedback on whether the idea is useful or not, and whether it's worth cleaning up the remaining issues. As Aaron s/Aaron/Aaro/. Sorry, Aaro. - Paul -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/6] Add infrastructure for conditional code and data sections
Hello Thomas On Tue, 21 Dec 2010, Thomas Petazzoni wrote: WARNING: This is only a proof-of-concept, there are many known issues. The sole purpose of this patch is to get some feedback on whether the idea is useful or not, and whether it's worth cleaning up the remaining issues. As Aaron and Tony commented, I too think this is really good. I note in your examples that you use omap2 rather than splitting 2420 and 2430. It seems like it would be a good idea to deallocate 2420 data structures when booting on 2430, and vice versa. e.g., #define __omap2420_data cond_data_section(omap2420) #define __omap2430_data cond_data_section(omap2430) What do you think? - Paul -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/6] Add infrastructure for conditional code and data sections
Hi, On Wed, 22 Dec 2010, Tony Lindgren wrote: * Thomas Petazzoni thomas.petazz...@free-electrons.com [101221 14:00]: On Tue, 21 Dec 2010 11:27:35 -0800 Tony Lindgren t...@atomide.com wrote: Therefore, we introduce an infrastructure that allows to put code and data into specific sections, called conditional sections. All those sections are compiled into the final kernel image, but at runtime, by calling a function, we can get rid of the unused sections. Great, something is certainly needed to free the unused memory. Nice to see that the idea is welcome. Did you had a look at the implementation in patch 1/6 ? No not yet, will take a look after we're done with this upcoming merge window.. I also think the idea is good, and this should be maybe posted to wider audience than just linux-omap. However, in order to be able to free each section independently from another, I have to page align all those conditional sections. This means that having one section for only a tiny amount of data is going to waste space instead of saving space. So the conditional section should gather a sufficiently large amount of data ( 4 KB) to actually be valuable. Yeah I don't know how much non-init data we have for each board-*.c file. Maybe there is not much for each machine. I took a quick look at omap2plus_defconfig kernel, and non-init text/data for 29 boards takes 85K. If we would page align those memory consumption would increase during init to 29*8=232K, but after other boards are freed we would eventually save 85-8=77K. Under mach-omap2, another big consumer is clock data which takes also around 80 K, while e.g. on 2420 only 14 K is needed. A. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/6] Add infrastructure for conditional code and data sections
On Thu, 23 Dec 2010 14:31:46 +0200 (EET)
Aaro Koskinen aaro.koski...@nokia.com wrote:
No not yet, will take a look after we're done with this upcoming
merge window..
I also think the idea is good, and this should be maybe posted to wider
audience than just linux-omap.
Yes, definitely. As the proof of concept implementation was
demonstrating usage on OMAP, I wanted to have some early comments of
a few kernel developers before going to a wider audience.
Yeah I don't know how much non-init data we have for each board-*.c
file. Maybe there is not much for each machine.
I took a quick look at omap2plus_defconfig kernel, and non-init text/data
for 29 boards takes 85K. If we would page align those memory consumption
would increase during init to 29*8=232K, but after other boards are
freed we would eventually save 85-8=77K.
Thanks for those numbers. 77K is nice, but is not really a huge saving.
Under mach-omap2, another big consumer is clock data which takes also
around 80 K, while e.g. on 2420 only 14 K is needed.
My patch 3/6 (which didn't make it to the list, since it was probably
too large) already marked some OMAP2 clock data.
Unfortunately, I haven't found a nice way of putting strings into a
separate sections. For example:
static struct foobar __omap2 foo = {
.name = blabla,
.id = 12,
};
The struct foobar goes into the OMAP2-specific section (8 bytes), but
the blabla string (6 bytes) remains in the global .rodata section,
without anyway from removing it. The only solution I see so far would
be :
static char __omap2 foo_name[] = blabla;
static struct foobar __omap2 foo = {
.name = foo_name,
.id = 12,
};
but it's really unpleasant.
Other ideas ?
Regards,
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
--
To unsubscribe from this list: send the line unsubscribe linux-omap in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/6] Add infrastructure for conditional code and data sections
* Aaro Koskinen aaro.koski...@nokia.com [101223 04:31]: Hi, On Wed, 22 Dec 2010, Tony Lindgren wrote: * Thomas Petazzoni thomas.petazz...@free-electrons.com [101221 14:00]: On Tue, 21 Dec 2010 11:27:35 -0800 Tony Lindgren t...@atomide.com wrote: Therefore, we introduce an infrastructure that allows to put code and data into specific sections, called conditional sections. All those sections are compiled into the final kernel image, but at runtime, by calling a function, we can get rid of the unused sections. Great, something is certainly needed to free the unused memory. Nice to see that the idea is welcome. Did you had a look at the implementation in patch 1/6 ? No not yet, will take a look after we're done with this upcoming merge window.. I also think the idea is good, and this should be maybe posted to wider audience than just linux-omap. However, in order to be able to free each section independently from another, I have to page align all those conditional sections. This means that having one section for only a tiny amount of data is going to waste space instead of saving space. So the conditional section should gather a sufficiently large amount of data ( 4 KB) to actually be valuable. Yeah I don't know how much non-init data we have for each board-*.c file. Maybe there is not much for each machine. I took a quick look at omap2plus_defconfig kernel, and non-init text/data for 29 boards takes 85K. If we would page align those memory consumption would increase during init to 29*8=232K, but after other boards are freed we would eventually save 85-8=77K. That does not sounds like a huge savings as some bootloaders like nolo have a 2MB kernel size limitation.. Under mach-omap2, another big consumer is clock data which takes also around 80 K, while e.g. on 2420 only 14 K is needed. This eventually should be all __initdata and only the clock data for the booted omap should be allocated during the boot. Sounds like the best way to reduce the size of the image immediately is to make everything possible a module for the defconfigs, then use the standard minimal kernel + initramfs booting. That will make it easy for all the distros to use it too. Regards, Tony -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/6] Add infrastructure for conditional code and data sections
* Thomas Petazzoni thomas.petazz...@free-electrons.com [101221 14:00]: On Tue, 21 Dec 2010 11:27:35 -0800 Tony Lindgren t...@atomide.com wrote: Therefore, we introduce an infrastructure that allows to put code and data into specific sections, called conditional sections. All those sections are compiled into the final kernel image, but at runtime, by calling a function, we can get rid of the unused sections. Great, something is certainly needed to free the unused memory. Nice to see that the idea is welcome. Did you had a look at the implementation in patch 1/6 ? No not yet, will take a look after we're done with this upcoming merge window.. For example, on OMAP, you can declare data as being omap2 specific this way: static int __omap2_data foobar; Then, in the board code of an OMAP3 or OMAP4 platform, you can call: free_unused_cond_section(omap2); Sounds like this could be done after the cpu detection automatically? Yes, it definitely should. I don't know what the section limitations are, but it would be nice to have a separate section for each machine.. Then we could just free_unused_machines() during the init.. :) I don't think there are any specific limitations, so we can just create as many section as we want. However, in order to be able to free each section independently from another, I have to page align all those conditional sections. This means that having one section for only a tiny amount of data is going to waste space instead of saving space. So the conditional section should gather a sufficiently large amount of data ( 4 KB) to actually be valuable. Yeah I don't know how much non-init data we have for each board-*.c file. Maybe there is not much for each machine. Ideally the new sections would be arch/arm generic sections and not omap specific. I could see ARMv6 and ARMv7 sections being one way to group them, but that does not help to drop omap3 specific data on omap4. Regards, Tony -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/6] Add infrastructure for conditional code and data sections
On Wed, 22 Dec 2010 10:28:41 -0800 Tony Lindgren t...@atomide.com wrote: Nice to see that the idea is welcome. Did you had a look at the implementation in patch 1/6 ? No not yet, will take a look after we're done with this upcoming merge window.. Great, thanks. Ideally the new sections would be arch/arm generic sections and not omap specific. I could see ARMv6 and ARMv7 sections being one way to group them, but that does not help to drop omap3 specific data on omap4. The mechanism is architecture independent, not OMAP-specific or even ARM-specific (the only architecture-dependent part is the modification of the kernel linker script, but it's trivial to adapt to other arches). You can put as many sections as you want, you just need to declare some macros: #define __something_data cond_data_section(something) #define __something_text cond_text_section(something) and then mark whatever you want with __something_data or __something_text. It will then be part of separate sections, that are page-aligned so that they can independently be freed. For the moment, the API allows to tell which sections you want to *free*, but I think I should turn it into an API that allows to tell which sections you want to *keep*. This way, if you have sections for 100 machines and you boot on a given machine, you only need to say I'm using this section. At the end of the kernel boot process, all sections that have not been marked as useful would be removed. Regards, Thomas -- Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/6] Add infrastructure for conditional code and data sections
* Thomas Petazzoni thomas.petazz...@free-electrons.com [101222 10:55]: On Wed, 22 Dec 2010 10:28:41 -0800 Tony Lindgren t...@atomide.com wrote: Nice to see that the idea is welcome. Did you had a look at the implementation in patch 1/6 ? No not yet, will take a look after we're done with this upcoming merge window.. Great, thanks. Ideally the new sections would be arch/arm generic sections and not omap specific. I could see ARMv6 and ARMv7 sections being one way to group them, but that does not help to drop omap3 specific data on omap4. The mechanism is architecture independent, not OMAP-specific or even ARM-specific (the only architecture-dependent part is the modification of the kernel linker script, but it's trivial to adapt to other arches). You can put as many sections as you want, you just need to declare some macros: #define __something_data cond_data_section(something) #define __something_text cond_text_section(something) and then mark whatever you want with __something_data or __something_text. It will then be part of separate sections, that are page-aligned so that they can independently be freed. OK sounds like you've already made it generic :) For the moment, the API allows to tell which sections you want to *free*, but I think I should turn it into an API that allows to tell which sections you want to *keep*. This way, if you have sections for 100 machines and you boot on a given machine, you only need to say I'm using this section. At the end of the kernel boot process, all sections that have not been marked as useful would be removed. Yeah keeping only the code for the current machine might be easier. Regards, Tony -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/6] Add infrastructure for conditional code and data sections
* Thomas Petazzoni thomas.petazz...@free-electrons.com [101221 10:20]: From: Thomas Petazzoni tpetazz...@ti.com WARNING: This is only a proof-of-concept, there are many known issues. The sole purpose of this patch is to get some feedback on whether the idea is useful or not, and whether it's worth cleaning up the remaining issues. A trend in the kernel support for SoC is to build a single kernel that works accross a wide range of SoC inside a SoC family, or even in the future SoC of different families. While this is very interesting to reduce the number of kernel images needed to support a large number of hardware platforms, it allows means that the kernel image size is increasing. Portions of code and data are specific to a given SoC (clock structures, hwmod structures on OMAP, etc.) and only the portion relevant for the current SoC the kernel is running on is actually useful. The rest of the code and data remains in memory forever. While __init and __initdata can solve some of those cases, it is not necessarly easy to use, since the code/data that is actually useful needs to be copied so that it is kept after the init memory cleanup. Therefore, we introduce an infrastructure that allows to put code and data into specific sections, called conditional sections. All those sections are compiled into the final kernel image, but at runtime, by calling a function, we can get rid of the unused sections. Great, something is certainly needed to free the unused memory. For example, on OMAP, you can declare data as being omap2 specific this way: static int __omap2_data foobar; Then, in the board code of an OMAP3 or OMAP4 platform, you can call: free_unused_cond_section(omap2); Sounds like this could be done after the cpu detection automatically? I don't know what the section limitations are, but it would be nice to have a separate section for each machine.. Then we could just free_unused_machines() during the init.. :) Tony -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/6] Add infrastructure for conditional code and data sections
On Tue, 21 Dec 2010 11:27:35 -0800 Tony Lindgren t...@atomide.com wrote: Therefore, we introduce an infrastructure that allows to put code and data into specific sections, called conditional sections. All those sections are compiled into the final kernel image, but at runtime, by calling a function, we can get rid of the unused sections. Great, something is certainly needed to free the unused memory. Nice to see that the idea is welcome. Did you had a look at the implementation in patch 1/6 ? For example, on OMAP, you can declare data as being omap2 specific this way: static int __omap2_data foobar; Then, in the board code of an OMAP3 or OMAP4 platform, you can call: free_unused_cond_section(omap2); Sounds like this could be done after the cpu detection automatically? Yes, it definitely should. I don't know what the section limitations are, but it would be nice to have a separate section for each machine.. Then we could just free_unused_machines() during the init.. :) I don't think there are any specific limitations, so we can just create as many section as we want. However, in order to be able to free each section independently from another, I have to page align all those conditional sections. This means that having one section for only a tiny amount of data is going to waste space instead of saving space. So the conditional section should gather a sufficiently large amount of data ( 4 KB) to actually be valuable. Regards, Thomas -- Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
