Re: [PATCH 0/6] Shrinking DT memory usage
On 10/05/17 12:44, Rob Herring wrote: > On kernels with a minimal config and a RAM target in the 100s of KB, DT > is quite a hog of runtime memory usage. How much is dependent on how many > nodes and properties in the DT which have a corresponding struct device_node > and struct property in the kernel. Just skipping disabled nodes saves a > lot by not creating the device_nodes in the first place[1], but there's > more low hanging fruit by making some of the fields in struct property and > struct device_node optional. With the changes here, the memory usage goes > from 17KB to under 8KB on QEMU's ARM virt machine which is a relatively > small DT. > > The majority of the diffstat here is just moving all the kobject/sysfs > related code to its own file so we can avoid adding a bunch of ifdefs. > > There's more drastic approaches we could take such as doing the > unflattening at build time and storing the bulk of the unflattened tree > as const data. Grant also has some ideas on storing properties as ids > instead. He's explained it to me, but I still don't understand it. > > Rob > > [1] https://patchwork.kernel.org/patch/9983203/ > > Rob Herring (6): > of: fix missing kobject init for !SYSFS && OF_DYNAMIC config > of: remove struct property.unique_id for FDT > of: make struct property _flags field configurable > of: move kobj_to_device_node() into dynamic.c > of: wrap accesses to device_node kobject > of: make kobject and bin_attribute support configurable > > drivers/base/core.c | 2 +- > drivers/of/Kconfig | 4 ++ > drivers/of/Makefile | 1 + > drivers/of/base.c | 133 --- > drivers/of/dynamic.c| 27 ++-- > drivers/of/kobj.c | 164 > > drivers/of/of_private.h | 29 +++-- > include/linux/of.h | 28 + > 8 files changed, 215 insertions(+), 173 deletions(-) > create mode 100644 drivers/of/kobj.c > For the entire series: Reviewed-by: Frank Rowand I don't know if there will be any conflicts between my series "[PATCH 00/12] of: overlay: clean up device tree overlay code" and this series. If you choose to apply this series first, I will make any needed changes to my series. -Frank
Re: [PATCH 0/6] Shrinking DT memory usage
On Fri, 6 Oct 2017, Rob Herring wrote: > On Fri, Oct 6, 2017 at 4:30 PM, Nicolas Pitre > wrote: > > On Thu, 5 Oct 2017, Rob Herring wrote: > > > >> On kernels with a minimal config and a RAM target in the 100s of KB, DT > >> is quite a hog of runtime memory usage. How much is dependent on how many > >> nodes and properties in the DT which have a corresponding struct > >> device_node > >> and struct property in the kernel. Just skipping disabled nodes saves a > >> lot by not creating the device_nodes in the first place[1], but there's > >> more low hanging fruit by making some of the fields in struct property and > >> struct device_node optional. With the changes here, the memory usage goes > >> from 17KB to under 8KB on QEMU's ARM virt machine which is a relatively > >> small DT. > > > > My test case went from 118072 bytes ddown to 21548 bytes with this > > series. > > Did that include the changes for FDT skipping status=disabled and to > stop storing the full path of every node? With those 2 alone you said > it dropped to 25K. No -- that's with those 6 patches only. Adding the other 2 patches on top and it is down to 11732 bytes. Nicolas
Re: [PATCH 0/6] Shrinking DT memory usage
On Fri, Oct 6, 2017 at 4:30 PM, Nicolas Pitre wrote: > On Thu, 5 Oct 2017, Rob Herring wrote: > >> On kernels with a minimal config and a RAM target in the 100s of KB, DT >> is quite a hog of runtime memory usage. How much is dependent on how many >> nodes and properties in the DT which have a corresponding struct device_node >> and struct property in the kernel. Just skipping disabled nodes saves a >> lot by not creating the device_nodes in the first place[1], but there's >> more low hanging fruit by making some of the fields in struct property and >> struct device_node optional. With the changes here, the memory usage goes >> from 17KB to under 8KB on QEMU's ARM virt machine which is a relatively >> small DT. > > My test case went from 118072 bytes ddown to 21548 bytes with this > series. Did that include the changes for FDT skipping status=disabled and to stop storing the full path of every node? With those 2 alone you said it dropped to 25K. > Tested-by: Nicolas Pitre Thanks. Rob
Re: [PATCH 0/6] Shrinking DT memory usage
On Thu, 5 Oct 2017, Rob Herring wrote: > On kernels with a minimal config and a RAM target in the 100s of KB, DT > is quite a hog of runtime memory usage. How much is dependent on how many > nodes and properties in the DT which have a corresponding struct device_node > and struct property in the kernel. Just skipping disabled nodes saves a > lot by not creating the device_nodes in the first place[1], but there's > more low hanging fruit by making some of the fields in struct property and > struct device_node optional. With the changes here, the memory usage goes > from 17KB to under 8KB on QEMU's ARM virt machine which is a relatively > small DT. My test case went from 118072 bytes ddown to 21548 bytes with this series. Tested-by: Nicolas Pitre Nicolas
Re: [PATCH 0/6] Shrinking DT memory usage
On Thu, Oct 5, 2017 at 8:44 PM, Rob Herring wrote: > On kernels with a minimal config and a RAM target in the 100s of KB, DT > is quite a hog of runtime memory usage. How much is dependent on how many > nodes and properties in the DT which have a corresponding struct device_node > and struct property in the kernel. Just skipping disabled nodes saves a > lot by not creating the device_nodes in the first place[1], but there's > more low hanging fruit by making some of the fields in struct property and > struct device_node optional. With the changes here, the memory usage goes > from 17KB to under 8KB on QEMU's ARM virt machine which is a relatively > small DT. > > The majority of the diffstat here is just moving all the kobject/sysfs > related code to its own file so we can avoid adding a bunch of ifdefs. > > There's more drastic approaches we could take such as doing the > unflattening at build time and storing the bulk of the unflattened tree > as const data. Grant also has some ideas on storing properties as ids > instead. He's explained it to me, but I still don't understand it. It was a two part idea. The first part is to start using numerical IDs for property names instead of strings. The DTB format kind of already does this because all the property names are stored in the string table at the end of the dtb. The actual properties just reference an offset into the string table. If we predefined ids for some property names (reg, compatible, status, etc.), then we could have a variant of the of_property_* API to find properties by numerical ID instead of by strcmp(). The second idea is to eliminate struct property entirely and always store a node's properties as a single DTB tagged list blob. It would reduce the space required at the expense of processing time when parsing the blob. The upshot is the property list could be left in a flash-resident copy of the DTB. Details need to be investigated, and it would be much more invasive than what Rob has been able to do here. It may not be worth the effort. g.
[PATCH 0/6] Shrinking DT memory usage
On kernels with a minimal config and a RAM target in the 100s of KB, DT is quite a hog of runtime memory usage. How much is dependent on how many nodes and properties in the DT which have a corresponding struct device_node and struct property in the kernel. Just skipping disabled nodes saves a lot by not creating the device_nodes in the first place[1], but there's more low hanging fruit by making some of the fields in struct property and struct device_node optional. With the changes here, the memory usage goes from 17KB to under 8KB on QEMU's ARM virt machine which is a relatively small DT. The majority of the diffstat here is just moving all the kobject/sysfs related code to its own file so we can avoid adding a bunch of ifdefs. There's more drastic approaches we could take such as doing the unflattening at build time and storing the bulk of the unflattened tree as const data. Grant also has some ideas on storing properties as ids instead. He's explained it to me, but I still don't understand it. Rob [1] https://patchwork.kernel.org/patch/9983203/ Rob Herring (6): of: fix missing kobject init for !SYSFS && OF_DYNAMIC config of: remove struct property.unique_id for FDT of: make struct property _flags field configurable of: move kobj_to_device_node() into dynamic.c of: wrap accesses to device_node kobject of: make kobject and bin_attribute support configurable drivers/base/core.c | 2 +- drivers/of/Kconfig | 4 ++ drivers/of/Makefile | 1 + drivers/of/base.c | 133 --- drivers/of/dynamic.c| 27 ++-- drivers/of/kobj.c | 164 drivers/of/of_private.h | 29 +++-- include/linux/of.h | 28 + 8 files changed, 215 insertions(+), 173 deletions(-) create mode 100644 drivers/of/kobj.c -- 2.11.0
