Hello Karel,
Am 15.07.23 um 12:17 schrieb Karel Gardas:
On 7/15/23 09:47, o...@c-mauderer.de wrote:
Am 15.07.23 um 01:48 schrieb Karel Gardas:
Hello,
I've created setup where I've put updated STM32H7 HAL consisting of
two submodules:
- stm32h7xx_hal_driver
- cmsis_device_h7
and ARM's
- CMSIS_5
into hals/arm/stm32h7 to follow related BSP code location.
I think I would prefer "contrib/hals", "external/hals" or some other
more generic top directory name. It would allow to use that for other
parts like for example zlib in "contrib/libs/zlib" in the future (if
we agree on having libs like that there too).
Indeed, name is not the problem here. Both looks fine, somewhat prefer
external a bit.
At the moment, you have absolute URLs for your submodules:
url = https://github.com/ARM-software/CMSIS_5.git
url = https://github.com/karelfv/stm32h7xx_hal_driver.git
url = https://github.com/karelfv/cmsis_device_h7.git
If we use submodules, I would go for relative paths and clone the
repositories to our git.rtems.org and to all mirrors. It has the
advantage that even if some upstream URL changes, we won't be affected.
This is dirty PoC not a final solution. Due to this I've settled on
absolute urls and not relative. For final solution relative are needed,
but both url (relative/abolute) and name/location in RTEMS tree is very
easy to change based on decision.
I like submodules because they are well-supported by the usual tools.
Honestly. I like submodules idea, but hate its implementation. Some
reasons:
- submodules were added as a light weight feature and during the
development of git/submodule feature they accumulated some more.
- submodules started as way to get commit id of the external project to
the tree. But only master branch was supported. (not suitable for RTEMS)
- later submodules added capability to use tag (still not suitable for
RTEMS)
- later submodules added capability to use different branch from the
project, but only a head of it. (finally what RTEMS need).
I don't like using branches instead of fixed commit IDs. Otherwise, it's
not possible to reliable re-build a fixed version.
- in the field you can see gits supporting different sets of submodule
features described above.
Yes, it will force us to use a minimum git version that supports the
necessary features. From my point of view, it's mainly something we
should document.
- submodules are easy to be broken. Personal experience from Haskell
compiler (GHC) project which uses submodules a lot. I have git 2.25 and
I always have an issue to pull from GHC repo since alwyas for me one
submodule (when changed) will be broken for me. Conclusion: clone with
submodules may kind of work, but later pull may be tricky.
Agreed: It's easy to miss that a submodule is out of date.
- besides submodules are not supported in 'got' which is 3rd party
independent implementation of git like SCM with more sane UI
I would say that this is a problem of the third-party client. We should
make sure to support the official git client. Maybe additionally some
(python?) git library if it is necessary for our tools.
- due to submodules trickiness development policy for external
integration with project code modification (aka HAL patches) needs to be
very well written. I smell horrible increase in cognitive load here...
Nonetheless, we should take a look at some of the reasons why other
projects decided against them. For example Google has the "repo" tool
used in Android. Zephyr has the "west" tool. OpenWRT uses "feeds" for
something similar. I'm sure there are more.
https://gerrit.googlesource.com/git-repo/
https://docs.zephyrproject.org/latest/develop/west/why.html#
https://openwrt.org/docs/guide-developer/feeds
All solve a similar problem. Part of them even overlap already with
functionality from our waf based build system. We should consider
whether we need any of the functionality of these tools and whether we
want to re-implement it or whether we just want to use one of the
tools and integrate with it.
Oh, do I sense starting academic debate now? The point of my dirty PoC
was to ground that a bit and show real stuff to get a feel of
possibilities.
I think the Proof of Concept is great and important. Thanks for doing
it. I'm fully OK with submodules as a general direction. I just wanted
to point out that we should take at least a short look at other systems
that solve similar problems. If they work around serious limitations of
submodules we can learn from them and avoid that we hit the same
limitations.
I used submodules myself in a lot of cases. And there are mainly two
problems that I have:
1. I often miss pushing a submodule when I push the base repo. And I
only note it way later if I try to update the repo on another PC. That
happened to me multiple times when working with submodules. But another
tool will just have other problems so that's not really an argument.
2. They have a strong hierarchy and sometimes can be a bit unflexible
due to that. It can be difficult to use (for example) the Makefiles in
one repo to build stuff in the parent repo. Or build a different version
of a library that is a submodule of a submodule. But again: Other tools
will just have other limitations.
Anyway, based on my PoC experience I would probably prefer something
more KISS and also more flexible by tool(s) being also aware of RTEMS BSPs
Awareness of RTEMS BSPs or libraries is a good point. It's something
that an external tool most likely can't do.
- first lets start with just 'boot' and 'update' scripts hosted in the
repo and used to clone/update externals would be needed. And put
'external' into git ignore.
Why do you want to ignore the "external" directory? Wouldn't this
suppress that git shows that a submodule has been changed?
- later boot/update my be made BSP aware. I mean when doing development
on H7 I do not need HALs for Xilinx and other ARM/v8/Rx stuff. Neither I
need HAL for NXP/Microchip chips.
- may this boot/update functionality be made part of ./waf tooling?
Perhaps it may....
From my point of view, the integration into the waf tooling should
happen sooner rather than later. Simple scripts are fine for a start,
but I would avoid growing the scripts to a point where they know about
BSPs. The waf based build system already has that information. What it
needs to learn is that if it doesn't find a file for the current build
configuration, it should take a look at whether the file is in some
submodule path. If yes, it can print an error ("Didn't find <file>. Did
you miss updating the submodule <path>?") in a first step or execute the
necessary git command itself in a second step.
So based on this I can see how Zephyr gravitated to west...
Or google Android to repo.
I've also completely removed HAL code from the BSP directory and also
CMSIS v4 files from bsps/arm/include to verify compilation is really
using submodules.
I think the CMSIS headers are used by some other BSPs too. So that's
fine for a prototype. But for the final solution, we need something
different.
The PoC needs to prove some things hence removal of CMSIS from the tree.
Nothing more. It's a "gain experience" vehicle, not a final solution.
And btw, you will not be able to find final solution without
touching/testing stuff and getting your hands dirty with it.
Yes, of course. I'm fully OK with a PoC as long as it is clear that it
still needs work.
Everything has its own price: (i) adaptation/support of west for RTEMS
or (ii) writing homegrown script(s) or (iii) never ending fight with git
submodule incompatibilities and user confusion and most importantly (iv)
never ending discussion or discussion fading away without any decision :-)
The user confusion in (iii) is most likely the most difficult one
(regardless what tool or script is used).
I see the problem of (iv). I will try to be open to every working solution.
Best regards
Christian
Thanks for all your comments!
Karel
_______________________________________________
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel