Re: [U-Boot] [PATCH 00/19] fdt: Move to the new upstream pylibfdt library
Hi, On 19 April 2017 at 08:27, Tom Riniwrote: > On Tue, Apr 18, 2017 at 10:48:07AM -0600, Simon Glass wrote: >> Hi Tom, >> >> On 18 April 2017 at 10:33, Tom Rini wrote: >> > On Tue, Apr 18, 2017 at 10:27:37AM -0600, Simon Glass wrote: >> >> Hi Tom, >> >> >> >> On 18 April 2017 at 10:25, Tom Rini wrote: >> >> > >> >> > On Sun, Apr 16, 2017 at 08:22:14PM -0600, Simon Glass wrote: >> >> > >> >> > > Python libfdt bindings have recently been accepted upstream. While the >> >> > > internals have changed a fair bit most of the API remains the same. >> >> > > Still, >> >> > > a few functions are different from how they are used in U-Boot so >> >> > > changes >> >> > > are needed to make this work. >> >> > > >> >> > > At present in U-Boot there are two libraries for accessing a device >> >> > > tree >> >> > > file: >> >> > > >> >> > > - FdtNormal which uses U-Boot's own Python bindings >> >> > > - FdtFallback which uses the fdtget command-line utility >> >> > > >> >> > > The latter is not a great solution: it is fairly slow since the DT is >> >> > > re-read for every access and it cannot provide DT offsets or packing >> >> > > of >> >> > > the DT. >> >> > > >> >> > > In addition, U-Boot now builds the libfdt module if swig is available, >> >> > > meaning that the fallback module is not used in that case. >> >> > > >> >> > > Finally, at some point in the future distributions may start >> >> > > packaging the >> >> > > libfdt Python module and it will be available without U-Boot needing >> >> > > to >> >> > > build it itself. >> >> > > >> >> > > Therefore it seems like a good idea to take this opportunity to drop >> >> > > the >> >> > > fallback module and just require that the Python libfdt bindings be >> >> > > present (at least if need by the build). >> >> > > >> >> > > The bindings are needed in two situations: >> >> > > - When dtoc is used to convert a device tree into C code. This is >> >> > > enabled >> >> > > by CONFIG_SPL_OF_PLATDATA >> >> > > - When binman is used to produce a firmware image. This is used on >> >> > > all x86 >> >> > > and sunxi boards at present >> >> > > >> >> > > This series: >> >> > > - Plumbs in building the Python libfdt module to the U-Boot build >> >> > > system >> >> > > - Ensures that the module is always built if needed, print an error if >> >> > > swig is not available (and thus the module cannot be built) >> >> > > - Allows use of a libfdt.py module already installed on the machine >> >> > > - Drops the FdtFallback support >> >> > > - Moves fdt.h and libfdt.h into lib/libfdt to aid with syncing with >> >> > > upstream, building the Python bindings and to keep the code >> >> > > together >> >> > > - Merges Fdt and FdtNormal to simplify the code >> >> > > - Adjusts the Fdt library to work with the new libfdt module >> >> > > - Adds a few more tests to check access to properties in the DT >> >> > > - Adjusts binman and dtoc to work with the new approach >> >> > > >> >> > > It should be possible to easily sync libfdt's Python bindings with >> >> > > U-Boot >> >> > > in the future, as development there proceeds. >> >> > >> >> > While this came in late, my gut feeling is that it would be best to have >> >> > this change in the next release (so that various upstreams can get used >> >> > to the idea of basically always needing python installed to build). >> >> >> >> I'm a little worried that it might cause problems. I made it so that >> >> it only *requires* python if is it actually needed, though it always >> >> builds libfdt.py if it can. Or perhaps that is what you are saying, >> >> that the only way to flush out upstream issues is to apply it? >> > >> > What I'm saying is that distros (Fedora, openSUSE, OpenEmbedded) are >> > going to make a blanket change (if they haven't already) to say that >> > building U-Boot requires python to be installed (since they aren't >> > likely to try and optimize it on a board-by-board basis when it's "easy" >> > to just say it's an always dependency). If you think there's risk with >> > the change itself, we can wait a release, that's fine too. Thanks! >> >> I see. I'm not sure what is best. I checked for build errors and added >> some more tests so I'm fairly confident. But it is late (sorry, it was >> more work than I expected). I'll leave this up to you. > > Alright, lets just have this queued up for early next window. I've pushed this to u-boot-fdt/next. If anyone has time to test that this builds OK (particularly 'buildman sandbox') it would be appreciated. Regards, Simon ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH 00/19] fdt: Move to the new upstream pylibfdt library
On Tue, Apr 18, 2017 at 10:48:07AM -0600, Simon Glass wrote: > Hi Tom, > > On 18 April 2017 at 10:33, Tom Riniwrote: > > On Tue, Apr 18, 2017 at 10:27:37AM -0600, Simon Glass wrote: > >> Hi Tom, > >> > >> On 18 April 2017 at 10:25, Tom Rini wrote: > >> > > >> > On Sun, Apr 16, 2017 at 08:22:14PM -0600, Simon Glass wrote: > >> > > >> > > Python libfdt bindings have recently been accepted upstream. While the > >> > > internals have changed a fair bit most of the API remains the same. > >> > > Still, > >> > > a few functions are different from how they are used in U-Boot so > >> > > changes > >> > > are needed to make this work. > >> > > > >> > > At present in U-Boot there are two libraries for accessing a device > >> > > tree > >> > > file: > >> > > > >> > > - FdtNormal which uses U-Boot's own Python bindings > >> > > - FdtFallback which uses the fdtget command-line utility > >> > > > >> > > The latter is not a great solution: it is fairly slow since the DT is > >> > > re-read for every access and it cannot provide DT offsets or packing of > >> > > the DT. > >> > > > >> > > In addition, U-Boot now builds the libfdt module if swig is available, > >> > > meaning that the fallback module is not used in that case. > >> > > > >> > > Finally, at some point in the future distributions may start packaging > >> > > the > >> > > libfdt Python module and it will be available without U-Boot needing to > >> > > build it itself. > >> > > > >> > > Therefore it seems like a good idea to take this opportunity to drop > >> > > the > >> > > fallback module and just require that the Python libfdt bindings be > >> > > present (at least if need by the build). > >> > > > >> > > The bindings are needed in two situations: > >> > > - When dtoc is used to convert a device tree into C code. This is > >> > > enabled > >> > > by CONFIG_SPL_OF_PLATDATA > >> > > - When binman is used to produce a firmware image. This is used on all > >> > > x86 > >> > > and sunxi boards at present > >> > > > >> > > This series: > >> > > - Plumbs in building the Python libfdt module to the U-Boot build > >> > > system > >> > > - Ensures that the module is always built if needed, print an error if > >> > > swig is not available (and thus the module cannot be built) > >> > > - Allows use of a libfdt.py module already installed on the machine > >> > > - Drops the FdtFallback support > >> > > - Moves fdt.h and libfdt.h into lib/libfdt to aid with syncing with > >> > > upstream, building the Python bindings and to keep the code > >> > > together > >> > > - Merges Fdt and FdtNormal to simplify the code > >> > > - Adjusts the Fdt library to work with the new libfdt module > >> > > - Adds a few more tests to check access to properties in the DT > >> > > - Adjusts binman and dtoc to work with the new approach > >> > > > >> > > It should be possible to easily sync libfdt's Python bindings with > >> > > U-Boot > >> > > in the future, as development there proceeds. > >> > > >> > While this came in late, my gut feeling is that it would be best to have > >> > this change in the next release (so that various upstreams can get used > >> > to the idea of basically always needing python installed to build). > >> > >> I'm a little worried that it might cause problems. I made it so that > >> it only *requires* python if is it actually needed, though it always > >> builds libfdt.py if it can. Or perhaps that is what you are saying, > >> that the only way to flush out upstream issues is to apply it? > > > > What I'm saying is that distros (Fedora, openSUSE, OpenEmbedded) are > > going to make a blanket change (if they haven't already) to say that > > building U-Boot requires python to be installed (since they aren't > > likely to try and optimize it on a board-by-board basis when it's "easy" > > to just say it's an always dependency). If you think there's risk with > > the change itself, we can wait a release, that's fine too. Thanks! > > I see. I'm not sure what is best. I checked for build errors and added > some more tests so I'm fairly confident. But it is late (sorry, it was > more work than I expected). I'll leave this up to you. Alright, lets just have this queued up for early next window. -- Tom signature.asc Description: Digital signature ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH 00/19] fdt: Move to the new upstream pylibfdt library
Hi Tom, On 18 April 2017 at 10:33, Tom Riniwrote: > On Tue, Apr 18, 2017 at 10:27:37AM -0600, Simon Glass wrote: >> Hi Tom, >> >> On 18 April 2017 at 10:25, Tom Rini wrote: >> > >> > On Sun, Apr 16, 2017 at 08:22:14PM -0600, Simon Glass wrote: >> > >> > > Python libfdt bindings have recently been accepted upstream. While the >> > > internals have changed a fair bit most of the API remains the same. >> > > Still, >> > > a few functions are different from how they are used in U-Boot so changes >> > > are needed to make this work. >> > > >> > > At present in U-Boot there are two libraries for accessing a device tree >> > > file: >> > > >> > > - FdtNormal which uses U-Boot's own Python bindings >> > > - FdtFallback which uses the fdtget command-line utility >> > > >> > > The latter is not a great solution: it is fairly slow since the DT is >> > > re-read for every access and it cannot provide DT offsets or packing of >> > > the DT. >> > > >> > > In addition, U-Boot now builds the libfdt module if swig is available, >> > > meaning that the fallback module is not used in that case. >> > > >> > > Finally, at some point in the future distributions may start packaging >> > > the >> > > libfdt Python module and it will be available without U-Boot needing to >> > > build it itself. >> > > >> > > Therefore it seems like a good idea to take this opportunity to drop the >> > > fallback module and just require that the Python libfdt bindings be >> > > present (at least if need by the build). >> > > >> > > The bindings are needed in two situations: >> > > - When dtoc is used to convert a device tree into C code. This is enabled >> > > by CONFIG_SPL_OF_PLATDATA >> > > - When binman is used to produce a firmware image. This is used on all >> > > x86 >> > > and sunxi boards at present >> > > >> > > This series: >> > > - Plumbs in building the Python libfdt module to the U-Boot build system >> > > - Ensures that the module is always built if needed, print an error if >> > > swig is not available (and thus the module cannot be built) >> > > - Allows use of a libfdt.py module already installed on the machine >> > > - Drops the FdtFallback support >> > > - Moves fdt.h and libfdt.h into lib/libfdt to aid with syncing with >> > > upstream, building the Python bindings and to keep the code together >> > > - Merges Fdt and FdtNormal to simplify the code >> > > - Adjusts the Fdt library to work with the new libfdt module >> > > - Adds a few more tests to check access to properties in the DT >> > > - Adjusts binman and dtoc to work with the new approach >> > > >> > > It should be possible to easily sync libfdt's Python bindings with U-Boot >> > > in the future, as development there proceeds. >> > >> > While this came in late, my gut feeling is that it would be best to have >> > this change in the next release (so that various upstreams can get used >> > to the idea of basically always needing python installed to build). >> >> I'm a little worried that it might cause problems. I made it so that >> it only *requires* python if is it actually needed, though it always >> builds libfdt.py if it can. Or perhaps that is what you are saying, >> that the only way to flush out upstream issues is to apply it? > > What I'm saying is that distros (Fedora, openSUSE, OpenEmbedded) are > going to make a blanket change (if they haven't already) to say that > building U-Boot requires python to be installed (since they aren't > likely to try and optimize it on a board-by-board basis when it's "easy" > to just say it's an always dependency). If you think there's risk with > the change itself, we can wait a release, that's fine too. Thanks! I see. I'm not sure what is best. I checked for build errors and added some more tests so I'm fairly confident. But it is late (sorry, it was more work than I expected). I'll leave this up to you. > > -- > Tom Regards, Simon ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH 00/19] fdt: Move to the new upstream pylibfdt library
On Tue, Apr 18, 2017 at 10:27:37AM -0600, Simon Glass wrote: > Hi Tom, > > On 18 April 2017 at 10:25, Tom Riniwrote: > > > > On Sun, Apr 16, 2017 at 08:22:14PM -0600, Simon Glass wrote: > > > > > Python libfdt bindings have recently been accepted upstream. While the > > > internals have changed a fair bit most of the API remains the same. Still, > > > a few functions are different from how they are used in U-Boot so changes > > > are needed to make this work. > > > > > > At present in U-Boot there are two libraries for accessing a device tree > > > file: > > > > > > - FdtNormal which uses U-Boot's own Python bindings > > > - FdtFallback which uses the fdtget command-line utility > > > > > > The latter is not a great solution: it is fairly slow since the DT is > > > re-read for every access and it cannot provide DT offsets or packing of > > > the DT. > > > > > > In addition, U-Boot now builds the libfdt module if swig is available, > > > meaning that the fallback module is not used in that case. > > > > > > Finally, at some point in the future distributions may start packaging the > > > libfdt Python module and it will be available without U-Boot needing to > > > build it itself. > > > > > > Therefore it seems like a good idea to take this opportunity to drop the > > > fallback module and just require that the Python libfdt bindings be > > > present (at least if need by the build). > > > > > > The bindings are needed in two situations: > > > - When dtoc is used to convert a device tree into C code. This is enabled > > > by CONFIG_SPL_OF_PLATDATA > > > - When binman is used to produce a firmware image. This is used on all x86 > > > and sunxi boards at present > > > > > > This series: > > > - Plumbs in building the Python libfdt module to the U-Boot build system > > > - Ensures that the module is always built if needed, print an error if > > > swig is not available (and thus the module cannot be built) > > > - Allows use of a libfdt.py module already installed on the machine > > > - Drops the FdtFallback support > > > - Moves fdt.h and libfdt.h into lib/libfdt to aid with syncing with > > > upstream, building the Python bindings and to keep the code together > > > - Merges Fdt and FdtNormal to simplify the code > > > - Adjusts the Fdt library to work with the new libfdt module > > > - Adds a few more tests to check access to properties in the DT > > > - Adjusts binman and dtoc to work with the new approach > > > > > > It should be possible to easily sync libfdt's Python bindings with U-Boot > > > in the future, as development there proceeds. > > > > While this came in late, my gut feeling is that it would be best to have > > this change in the next release (so that various upstreams can get used > > to the idea of basically always needing python installed to build). > > I'm a little worried that it might cause problems. I made it so that > it only *requires* python if is it actually needed, though it always > builds libfdt.py if it can. Or perhaps that is what you are saying, > that the only way to flush out upstream issues is to apply it? What I'm saying is that distros (Fedora, openSUSE, OpenEmbedded) are going to make a blanket change (if they haven't already) to say that building U-Boot requires python to be installed (since they aren't likely to try and optimize it on a board-by-board basis when it's "easy" to just say it's an always dependency). If you think there's risk with the change itself, we can wait a release, that's fine too. Thanks! -- Tom signature.asc Description: Digital signature ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH 00/19] fdt: Move to the new upstream pylibfdt library
Hi Tom, On 18 April 2017 at 10:25, Tom Riniwrote: > > On Sun, Apr 16, 2017 at 08:22:14PM -0600, Simon Glass wrote: > > > Python libfdt bindings have recently been accepted upstream. While the > > internals have changed a fair bit most of the API remains the same. Still, > > a few functions are different from how they are used in U-Boot so changes > > are needed to make this work. > > > > At present in U-Boot there are two libraries for accessing a device tree > > file: > > > > - FdtNormal which uses U-Boot's own Python bindings > > - FdtFallback which uses the fdtget command-line utility > > > > The latter is not a great solution: it is fairly slow since the DT is > > re-read for every access and it cannot provide DT offsets or packing of > > the DT. > > > > In addition, U-Boot now builds the libfdt module if swig is available, > > meaning that the fallback module is not used in that case. > > > > Finally, at some point in the future distributions may start packaging the > > libfdt Python module and it will be available without U-Boot needing to > > build it itself. > > > > Therefore it seems like a good idea to take this opportunity to drop the > > fallback module and just require that the Python libfdt bindings be > > present (at least if need by the build). > > > > The bindings are needed in two situations: > > - When dtoc is used to convert a device tree into C code. This is enabled > > by CONFIG_SPL_OF_PLATDATA > > - When binman is used to produce a firmware image. This is used on all x86 > > and sunxi boards at present > > > > This series: > > - Plumbs in building the Python libfdt module to the U-Boot build system > > - Ensures that the module is always built if needed, print an error if > > swig is not available (and thus the module cannot be built) > > - Allows use of a libfdt.py module already installed on the machine > > - Drops the FdtFallback support > > - Moves fdt.h and libfdt.h into lib/libfdt to aid with syncing with > > upstream, building the Python bindings and to keep the code together > > - Merges Fdt and FdtNormal to simplify the code > > - Adjusts the Fdt library to work with the new libfdt module > > - Adds a few more tests to check access to properties in the DT > > - Adjusts binman and dtoc to work with the new approach > > > > It should be possible to easily sync libfdt's Python bindings with U-Boot > > in the future, as development there proceeds. > > While this came in late, my gut feeling is that it would be best to have > this change in the next release (so that various upstreams can get used > to the idea of basically always needing python installed to build). I'm a little worried that it might cause problems. I made it so that it only *requires* python if is it actually needed, though it always builds libfdt.py if it can. Or perhaps that is what you are saying, that the only way to flush out upstream issues is to apply it? Regards, Simon ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH 00/19] fdt: Move to the new upstream pylibfdt library
On Sun, Apr 16, 2017 at 08:22:14PM -0600, Simon Glass wrote: > Python libfdt bindings have recently been accepted upstream. While the > internals have changed a fair bit most of the API remains the same. Still, > a few functions are different from how they are used in U-Boot so changes > are needed to make this work. > > At present in U-Boot there are two libraries for accessing a device tree > file: > > - FdtNormal which uses U-Boot's own Python bindings > - FdtFallback which uses the fdtget command-line utility > > The latter is not a great solution: it is fairly slow since the DT is > re-read for every access and it cannot provide DT offsets or packing of > the DT. > > In addition, U-Boot now builds the libfdt module if swig is available, > meaning that the fallback module is not used in that case. > > Finally, at some point in the future distributions may start packaging the > libfdt Python module and it will be available without U-Boot needing to > build it itself. > > Therefore it seems like a good idea to take this opportunity to drop the > fallback module and just require that the Python libfdt bindings be > present (at least if need by the build). > > The bindings are needed in two situations: > - When dtoc is used to convert a device tree into C code. This is enabled > by CONFIG_SPL_OF_PLATDATA > - When binman is used to produce a firmware image. This is used on all x86 > and sunxi boards at present > > This series: > - Plumbs in building the Python libfdt module to the U-Boot build system > - Ensures that the module is always built if needed, print an error if > swig is not available (and thus the module cannot be built) > - Allows use of a libfdt.py module already installed on the machine > - Drops the FdtFallback support > - Moves fdt.h and libfdt.h into lib/libfdt to aid with syncing with > upstream, building the Python bindings and to keep the code together > - Merges Fdt and FdtNormal to simplify the code > - Adjusts the Fdt library to work with the new libfdt module > - Adds a few more tests to check access to properties in the DT > - Adjusts binman and dtoc to work with the new approach > > It should be possible to easily sync libfdt's Python bindings with U-Boot > in the future, as development there proceeds. While this came in late, my gut feeling is that it would be best to have this change in the next release (so that various upstreams can get used to the idea of basically always needing python installed to build). -- Tom signature.asc Description: Digital signature ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
[U-Boot] [PATCH 00/19] fdt: Move to the new upstream pylibfdt library
Python libfdt bindings have recently been accepted upstream. While the internals have changed a fair bit most of the API remains the same. Still, a few functions are different from how they are used in U-Boot so changes are needed to make this work. At present in U-Boot there are two libraries for accessing a device tree file: - FdtNormal which uses U-Boot's own Python bindings - FdtFallback which uses the fdtget command-line utility The latter is not a great solution: it is fairly slow since the DT is re-read for every access and it cannot provide DT offsets or packing of the DT. In addition, U-Boot now builds the libfdt module if swig is available, meaning that the fallback module is not used in that case. Finally, at some point in the future distributions may start packaging the libfdt Python module and it will be available without U-Boot needing to build it itself. Therefore it seems like a good idea to take this opportunity to drop the fallback module and just require that the Python libfdt bindings be present (at least if need by the build). The bindings are needed in two situations: - When dtoc is used to convert a device tree into C code. This is enabled by CONFIG_SPL_OF_PLATDATA - When binman is used to produce a firmware image. This is used on all x86 and sunxi boards at present This series: - Plumbs in building the Python libfdt module to the U-Boot build system - Ensures that the module is always built if needed, print an error if swig is not available (and thus the module cannot be built) - Allows use of a libfdt.py module already installed on the machine - Drops the FdtFallback support - Moves fdt.h and libfdt.h into lib/libfdt to aid with syncing with upstream, building the Python bindings and to keep the code together - Merges Fdt and FdtNormal to simplify the code - Adjusts the Fdt library to work with the new libfdt module - Adds a few more tests to check access to properties in the DT - Adjusts binman and dtoc to work with the new approach It should be possible to easily sync libfdt's Python bindings with U-Boot in the future, as development there proceeds. Simon Glass (19): pci: Correct cast for sandbox fdt: Correct cast for sandbox in fdtdec_setup_memory_size() fdt: Use SPDX format for licenses in the libfdt headers fdt: Move header files into lib/libfdt fdt: Allow swig options to be provided by Makefile fdt: Add all source files to the libfdt build fdt: Rename existing python libfdt module fdt: Build the new python libfdt module fdt: Update fdt_test to use 'dt' instead of 'fdt' fdt: dtoc: Add a full set of property tests fdt: Support use of the new python libfdt library fdt: Makefile: Build python libfdt library if needed fdt: Stop building the old python libfdt module fdt: Drop use of the legacy libfdt python module fdt: Drop fdt_fallback library binman: Drop a special case related to fdt_fallback fdt: Merge fdt_normal with its base class binman: Rename fdt variable to dtb fdt: Drop fdt_select.py Makefile| 16 +- cmd/pci.c |3 +- include/fdt.h | 112 +- include/libfdt.h| 2138 +- lib/fdtdec.c|3 +- lib/libfdt/fdt.h| 67 + lib/libfdt/libfdt.h | 2144 +++ lib/libfdt/libfdt.swig | 113 -- lib/libfdt/pylibfdt/setup.py|4 +- lib/libfdt/setup.py | 38 - scripts/Makefile.spl| 17 +- tools/Makefile | 43 +- tools/binman/binman.py |3 + tools/binman/control.py | 12 +- tools/binman/etype/u_boot_dtb_with_ucode.py | 24 +- tools/binman/fdt_test.py| 64 +- tools/binman/func_test.py | 48 +- tools/binman/test/45_prop_test.dts | 23 + tools/dtoc/dtoc.py |3 +- tools/dtoc/fdt.py | 183 ++- tools/dtoc/fdt_fallback.py | 181 --- tools/dtoc/fdt_normal.py| 225 --- tools/dtoc/fdt_select.py| 36 - 23 files changed, 2539 insertions(+), 2961 deletions(-) create mode 100644 lib/libfdt/fdt.h create mode 100644 lib/libfdt/libfdt.h delete mode 100644 lib/libfdt/libfdt.swig delete mode 100644 lib/libfdt/setup.py create mode 100644 tools/binman/test/45_prop_test.dts delete mode 100644 tools/dtoc/fdt_fallback.py delete mode 100644 tools/dtoc/fdt_normal.py delete mode 100644 tools/dtoc/fdt_select.py -- 2.12.2.762.g0e3151a226-goog ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot