Re: Discussion: How to handle HALs, SDKs and libraries
Hello Karel, Am 16.07.23 um 12:37 schrieb Karel Gardas: On 7/16/23 10:03, o...@c-mauderer.de wrote: Of course the revision from two months back has to be available in the repo. It doesn't really matter whether it's on master because the code just worked or whether it's a RTEMS specific branch because adaptions have been necessary. You "it doesn't really matter" worries me a little bit. I'm not sure this functionality is even supported hence my question above if you have tested it yourself. Yes, I use that regularly. For an (old) example, take a look at a branch in the repo that I use to test GSoC work: https://gitlab.com/c-mauderer/rtems-bbb/-/tree/6 I have a libs/rtems-libbsd submodule there that uses the revision c9474c0b which is on a cm/20200726_fix branch in my clone of rtems-libbsd. In the .gitmodules the branch is still set to "master". Despite that, git can find thre right commit ID. Cool! So it didn't work for me due to (i) my git bug or (ii) my git doesn't supporting whatever or simply due to (iii) me making silly mistake somewhere. Will need to recheck this. Thanks for the info! Karel Seems that we are not the only ones with a waf based build system that want to manipulate submodules: https://github.com/ArduPilot/ardupilot/pull/3592 The final solution seems to be a bit different. Their wscript in the latest version has a "--no-submodule-update" parameter. So they decided to just update the submodules every time. Can be interesting for users because they don't have to care for outdated submodules if they pull a new version. But I think the problem with adding it as a fixed step is, that we have a problem with the releases. A release tar ball is not a git repo anymore. So a "git submodule init --update" or similar will not work. We would need some mechanism to detect whether it's a git repo or not. Best regards Christian ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Discussion: How to handle HALs, SDKs and libraries
On 7/16/23 10:03, o...@c-mauderer.de wrote: Of course the revision from two months back has to be available in the repo. It doesn't really matter whether it's on master because the code just worked or whether it's a RTEMS specific branch because adaptions have been necessary. You "it doesn't really matter" worries me a little bit. I'm not sure this functionality is even supported hence my question above if you have tested it yourself. Yes, I use that regularly. For an (old) example, take a look at a branch in the repo that I use to test GSoC work: https://gitlab.com/c-mauderer/rtems-bbb/-/tree/6 I have a libs/rtems-libbsd submodule there that uses the revision c9474c0b which is on a cm/20200726_fix branch in my clone of rtems-libbsd. In the .gitmodules the branch is still set to "master". Despite that, git can find thre right commit ID. Cool! So it didn't work for me due to (i) my git bug or (ii) my git doesn't supporting whatever or simply due to (iii) me making silly mistake somewhere. Will need to recheck this. Thanks for the info! Karel ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Westernization of RTEMS/stm32H7 [was: Re: Discussion: How to handle HALs, SDKs and libraries]
Hello Karel, Am 16.07.23 um 00:07 schrieb Karel Gardas: Hello, Christian mentioned Zephyr west tool as one of good/possible candidates for RTEMS tree modularization especially with regarding to HALs and other 3rd party libraries. I mentioned it as one of three tools that I know that solve similar problems (Google `repo` from Android, `west` from Zephyr and the feeds from OpenWRT). And clearly I haven't made clear enough what my goal was: I don't want to suggest that we jump on a random tool and use that. What I wanted is that we take a look at the reasons why the tools exist and evaluate whether we - as RTEMS project - have similar problems that have been solved with the tool. You wanted to avoid the academic debate in an earlier mail, so I didn't start to go into more details of every reason. But now there is a second prototype with a different tool so most likely we have to compare some more. West has the best documentation why they created that tool: https://docs.zephyrproject.org/latest/develop/west/why.html R1. Keeping code out of the main repo. That has advantages and disadvantages. And I'm not sure what's more important. Main advantage is that it makes more clear what is RTEMS and what is some external source. Main disadvantage is most likely that it is a bit more difficult to integrate it into the release process. If it's only one repo, the release will be to check out the submodules and archive everything as one big file. With multiple separate repos, we have to create one file for every repo. Additionally, a user has to download all archives to build the release version. So most likely that adds complexity. R2. Tool, that users can use. Depends quite heavily on the workflow and use case of the user. RTEMS has a tendency to not adopt new tools easily, especially if they have to be used by the end user. So that could be a disadvantage. R3. Allow to override repos in custom distributions. Not sure whether that is useful for us as a project. R4. Continuous tracking of a HEAD or commit-based tracking. Git does the commit-based tracking and I already said that I wouldn't be happy about continuous tracking of a branch. So that wouldn't be a reason for me. From my point of view, what we can learn from that tool is two things: A) We should take a look whether putting the code in the repo is a disadvantage in some situations. I think size of releases and licenses could be two. Licenses can be most likely solved with a clear directory structure where external libs are in one subdirectory with a good description. B) Do we need to support use cases, where a user wants to replace one submodule version by another version without having to create a private fork of RTEMS? C) Do we need to support use cases where we don't want to track fixed commit IDs but the HEAD of a branch instead? For Google repo, I didn't find the original reasons. What I did find are some comparisons with submodules like https://www.edureka.co/blog/git-submodules-versus-googles-repo-tool Sounds like the main intention is to make working with multiple repos simpler and integrate well with Gerrit and Jenkins. Beneath that, they heavily support tracking HEADs of branches. Main point to learn: Again: C) D) We should be careful how we handle submodules so that the usage is as simple as possible. So that's part of how we integrate it in our build system. E) We maybe can get problems on a future CI/CD integration. The OpenWRT Feeds: https://openwrt.org/docs/guide-developer/feeds I think the main reason why they use that are: 1. Allow to track a HEAD of a branch instead of a certain commit ID. Not something that I would want for HALs. 2. Allow a user to add additional feeds. That's similar to the point 3 of `west`. I think the main point to learn is basically the same as B) and C). Best regards Christian For comparison with my previous git submodulization of RTEMS/stm32H7 I've also westernized the same in a similar way. Yes, it is intended to remove CMSIS v4 *just* to proof compilation is working with CMSIS 5 from outside module! This is just dirty PoC nothing more. A tree structure looks a bit different as (folowing zephyr example) I keep modules outside the rtems tree. Hence result for west init command below is tree: modules/hal/stm32h7/stm32h7xx_hal_driver modules/hal/stm32h7/cmsis_device_h7 modules/hal/stm32h7/CMSIS_5 rtems/... So if you like to see thing, then install west: $ pip3 install --user west $ export PATH=$PATH:$HOME/.local/bin go into some testing directory and: $ west init -m https://github.com/karelfv/rtems --mr rtems-west-stm32h7-hal rtems-workspace $ cd rtems-workspace/ $ west update Compilation again produces working samples. In comparison with git submodule approach I find west a bit more civilized (user friendly). Nothing is shooting in my own feet etc. What may resonate in RTEMS devs is: - yaml format for
Re: Discussion: How to handle HALs, SDKs and libraries
Hello Karel, Am 15.07.23 um 19:22 schrieb Karel Gardas: On 7/15/23 16:00, o...@c-mauderer.de wrote: We should avoid rebasing and overwriting the old branch because we have to preserve a version that can be used with older RTEMS versions. So for rebasing, we would have to use multiple branches. Instead of rtems-6-branch, it could be a rtems- or something similar. Thanks for confirmation. Have the same feeling about it. I think that depends heavily on how the upstream project works. If the upstream project uses a typical git workflow and is quite active, merging is most likely the better solution because we don't get big amounts of branches with that. But this means we still merge to some rtems specific branch since master is used to sync with upstream, right? Yes. That would be my suggested workflow: Track the upstream master or main branch and merge it into an RTEMS specific one. On the other hand I know that some semiconductor manufacturers tend to just dump code into a repo and they don't care whether files move around or change heavily. They happily mix changes with reformatting the complete code. In that case you have no chance of a useful merge-commit. I think in these cases it's most likely better to create a new branch on every update and rebase the patches. Anyway, still this is a branch based workflow. How would you do that workflow with just fixed commit IDs? I'm asking since so far commit IDs on submodules were always obtain from master branch which is what we basically fork and does not give us ability to patch with RTEMS changes... So I'm curious how would you do that? I think I have put that in the wrong words. I assumed you mean that you want to use some feature that always uses the latest commit of a branch (also I don't know whether git even has that feature). What I wanted to say is that if I check out a RTEMS version from two months back, it should check out the revisions of the submodules that have been used in RTEMS two months back too. It must not use the latest HEAD of the branch from today. It's just part of having a version of the code that can be reproduced later. Right! Now crucial question: is that even supported in git? I'm asking since so far I've seen only: - use fixed commit ids, but those are obtained only from master branch or - use branch and then this is always branch HEAD I've not seen anything like "use fixed commit ID from branch XXX". So question for you as an git expert: are you absolutely sure that this workflow is supported by git at all? And by this I mean have you tested that actually? :-) I know only of the method that git uses a fixed commit ID in a submodule. It expects that the commit ID is on some branch in the remote repository. In my experience, it's not relevant whether the commit is on the branch `main`, `master`, `foo-bar`, `my-cool-new-feature` or any other branch. And yes: I use that regularly. Additionally, git support adding an optional "branch" parameter to each submodule. But as far as I know, that is only used during git submodule update --remote which basically tells git to pull the latest version of the branch instead of a special commit ID. I usually don't use that. See https://stackoverflow.com/a/18797720 for a better explanation. Of course the revision from two months back has to be available in the repo. It doesn't really matter whether it's on master because the code just worked or whether it's a RTEMS specific branch because adaptions have been necessary. You "it doesn't really matter" worries me a little bit. I'm not sure this functionality is even supported hence my question above if you have tested it yourself. Yes, I use that regularly. For an (old) example, take a look at a branch in the repo that I use to test GSoC work: https://gitlab.com/c-mauderer/rtems-bbb/-/tree/6 I have a libs/rtems-libbsd submodule there that uses the revision c9474c0b which is on a cm/20200726_fix branch in my clone of rtems-libbsd. In the .gitmodules the branch is still set to "master". Despite that, git can find thre right commit ID. Best regards Christian Thanks! Karel ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
