Re: [PATCH rtems-libbsd] CONTRIBUTING: Sharpen priority development goals
On 06.02.23 05:16, Chris Johns wrote: On 3/2/2023 6:31 pm, Christian Mauderer wrote: This patch tries to make the most important goals of LibBSD development more clear based on the results of the discussion on the mailing list: https://lists.rtems.org/pipermail/devel/2023-January/074164.html --- CONTRIBUTING.rst | 39 ++- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 0b6fc7a0..31561f6a 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -21,15 +21,36 @@ RTEMS specific support files, general guidelines on what modifications to the FreeBSD source are permitted, and some other topics. For building the library, see the `README `_. -Goals of the LibBSD activity are - -* provide functionality from FreeBSD to RTEMS, -* ease updating to future FreeBSD versions, -* ease tracking changes in FreeBSD code, -* minimize manual changes in FreeBSD code. - -We will work to push our changes upstream to the FreeBSD Project and minimize -changes required at each update point. +Every change to LibBSD has to consider the following points in groups of +descending priority: + +#. Real-time Impacts + Maintainability Loss + * LibBSD itself doesn't make any real time claims because it is basically + FreeBSD and they don't make any real time claims either. But all + development in LibBSD must make sure that the real time ability of the + RTEMS core system or the application is not affected. + * It's utterly important that LibBSD is simple to maintain. One of the most + important points for that is that upgrades to newer FreeBSD versions have + to be easy. Correct and it is important we adopt and use what FreeBSD provides rather than adding extra complexity. I wrote about this in the FreeBSD journal years ago. We already have a file descriptor to file object mapping in RTEMS so adding an additional file descriptor to file object mapping on top of it makes little sense. The libbsd should add complementary functionality otherwise you could just use FreeBSD directly. By adding the FreeBSD file descriptors you added a lot of unnecessary complexity. The FreeBSD file descriptor handing has to meet a couple of requirements which are not present in the RTEMS context. The bespoke handing of fd and file objects is something I consider an unneeded complexity for specific system related reasons. We need NFSv4 and that uses VFS and that in turn uses the FreeBSD fd and file objects. This conclusion is simply wrong. You don't need the FreeBSD file descriptors for the VFS support. See: https://github.com/sebhub/rtems-libbsd/tree/filedesc For VFS you need a proper struct file object associated with a file descriptor. For sockets, you don't need a struct file object. What is your hard technical reason to keep the FreeBSD file descriptors? Source transparency is what matters and as a test of what is acceptable it must be preferred between competing implementations. Whatever source transparency is, it is just one aspect out of many. -- embedded brains GmbH Herr Sebastian HUBER Dornierstr. 4 82178 Puchheim Germany email: sebastian.hu...@embedded-brains.de phone: +49-89-18 94 741 - 16 fax: +49-89-18 94 741 - 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [PATCH] wscript: Deduplicate installed files
On 06.02.23 03:35, Chris Johns wrote: On 4/2/2023 6:11 am, Sebastian Huber wrote: On 03.02.23 19:45, Kinsey Moore wrote: This is my first stab at solving this duplicate install problem. I could manually solve the problem by deduplicating the object includes and moving it up to the BSP, but that is less intuitive since these drivers both depend on the same code and the BSP doesn't depend on it directly. Why don't you add the shared stuff to a objxilcommon.yml? The approach in the wscript is a bit complex from my point of view. I am OK with adding this code or something similar. It is no more complex than other places I have reviewed, eg `Item._init_link()`. The issue is currently not easy to see and may be present in other places without us knowing. I am also fine with a spec file check that highlights a clash to draw attention to a problem when the spec files are parsed. I feeling we need something. If you install with ./waf install -vv you see the duplicate install targets. See also https://gitlab.com/ita1024/waf/-/issues/2329#note_467849523 Before we add double for loops we should first analyze the underlying problem. In this case it is a diamond shaped build dependency graph. spec/build/bsps/objnandpsu.yml: uid: objxilinxsupport spec/build/bsps/objqspipsu.yml: uid: objxilinxsupport spec/build/bsps/aarch64/xilinx-zynqmp/objjffs2qspinor.yml: uid: ../../objqspipsu spec/build/bsps/aarch64/xilinx-zynqmp/grp_zu3eg.yml: uid: ../../objnandpsu In addition to the duplicate install targets you build also the objects of objxilinxsupport twice and add them to the library. I would simply move the links to grp_zu3eg: grp_zu3eg.yml: uid: ../../objxilinxspport grp_zu3eg.yml: uid: ../../objnandpsu grp_zu3eg.yml: uid: ../../objqspipsu -- embedded brains GmbH Herr Sebastian HUBER Dornierstr. 4 82178 Puchheim Germany email: sebastian.hu...@embedded-brains.de phone: +49-89-18 94 741 - 16 fax: +49-89-18 94 741 - 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [PATCH rtems-libbsd] CONTRIBUTING: Sharpen priority development goals
On 3/2/2023 6:31 pm, Christian Mauderer wrote: > This patch tries to make the most important goals of LibBSD development > more clear based on the results of the discussion on the mailing list: > > https://lists.rtems.org/pipermail/devel/2023-January/074164.html > --- > CONTRIBUTING.rst | 39 ++- > 1 file changed, 30 insertions(+), 9 deletions(-) > > diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst > index 0b6fc7a0..31561f6a 100644 > --- a/CONTRIBUTING.rst > +++ b/CONTRIBUTING.rst > @@ -21,15 +21,36 @@ RTEMS specific support files, general guidelines on what > modifications to the > FreeBSD source are permitted, and some other topics. For building the > library, > see the `README `_. > > -Goals of the LibBSD activity are > - > -* provide functionality from FreeBSD to RTEMS, > -* ease updating to future FreeBSD versions, > -* ease tracking changes in FreeBSD code, > -* minimize manual changes in FreeBSD code. > - > -We will work to push our changes upstream to the FreeBSD Project and minimize > -changes required at each update point. > +Every change to LibBSD has to consider the following points in groups of > +descending priority: > + > +#. Real-time Impacts + Maintainability Loss > + * LibBSD itself doesn't make any real time claims because it is basically > + FreeBSD and they don't make any real time claims either. But all > + development in LibBSD must make sure that the real time ability of the > + RTEMS core system or the application is not affected. > + * It's utterly important that LibBSD is simple to maintain. One of the > most > + important points for that is that upgrades to newer FreeBSD versions > have > + to be easy. Correct and it is important we adopt and use what FreeBSD provides rather than adding extra complexity. I wrote about this in the FreeBSD journal years ago. The bespoke handing of fd and file objects is something I consider an unneeded complexity for specific system related reasons. We need NFSv4 and that uses VFS and that in turn uses the FreeBSD fd and file objects. Source transparency is what matters and as a test of what is acceptable it must be preferred between competing implementations. > +#. Transparency Loss + Modularity Loss + Code/RAM Size Increase > + * LibBSD code should be easy to read and easy to debug. Changes should be > + integrated in a way that are easy to understand. Of course that's a > + subjective goal. As a general rule: If it adds lot's of extra code or > even > + more layers than already exist in FreeBSD, it's harder to understand. > + * There are a number of methods used in LibBSD to make it modular. If you > + add new functionality, use one of the existing methods to allow > enabling or > + disabling your module. For example make sure that the linker can remove > + unused modules. If that doesn't work for your feature, try to use the > + buildsets to allow to switch a module on or off. > + * A lot of different hardware uses LibBSD as network or USB stack or > maybe in > + the future even only for other subsystems. Some of the targets have > + hundreds of megabytes memory. Others can only have a few megabytes > (like > + the ATSAMV71). Make sure that changes don't increase the RAM / Flash > size > + of the default build so that it can't be used on the small targets. This is not realistic or achievable and I find confusing the reasons it is being pushed over and over. I would have not have agreed to this being added before now and nothing has changed. The central reason for rejecting this statement is a change in FreeBSD may add a few meg of memory to the footprint and this type of statement would conflict with that addition and that in turn would conflict with the need for transparency of source. And as stated before transparency must be preferred. System requirements are for the developers of those systems and not RTEMS. Derating designs is an important part of system design and not the domain of this project. Memory constrained systems can consider another networking stack option or bespoke changes internally maintained. That is a cost trade off no one here can help make. > +#. Performance Loss > + * There are applications, that require (for example) a high network > + throughput or a fast storage access. Make sure to take that into > account > + if adding changes. I prefer we do not add statements that have no definition or boundaries someone could use to test against. Selection of RTEMS in a systems is the choice of the system designer. There are systems RTEMS is not a good fit for. Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [PATCH] wscript: Deduplicate installed files
On 4/2/2023 6:11 am, Sebastian Huber wrote: > On 03.02.23 19:45, Kinsey Moore wrote: >> This is my first stab at solving this duplicate install problem. I could >> manually solve the problem by deduplicating the object includes and moving it >> up to the BSP, but that is less intuitive since these drivers both depend on >> the same code and the BSP doesn't depend on it directly. > > Why don't you add the shared stuff to a objxilcommon.yml? > > The approach in the wscript is a bit complex from my point of view. I am OK with adding this code or something similar. It is no more complex than other places I have reviewed, eg `Item._init_link()`. The issue is currently not easy to see and may be present in other places without us knowing. I am also fine with a spec file check that highlights a clash to draw attention to a problem when the spec files are parsed. I feeling we need something. Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: libbsd development policy clarification needed?
Am 04.02.23 um 01:25 schrieb Gedare Bloom: On Fri, Feb 3, 2023 at 3:29 PM Christian Mauderer wrote: Am 3. Februar 2023 22:52:48 MEZ schrieb Gedare Bloom : On Fri, Feb 3, 2023 at 2:39 PM Christian Mauderer wrote: Am 3. Februar 2023 22:12:06 MEZ schrieb Gedare Bloom : On Fri, Feb 3, 2023 at 12:52 PM wrote: Hello Gedare, Am 03.02.23 um 19:51 schrieb Gedare Bloom: On Thu, Feb 2, 2023 at 11:24 PM Christian MAUDERER wrote: Hello Karel, On 2023-02-02 12:43, Karel Gardas wrote: Guys, recently I needed to work with RTEMS/NFS. As this is provided by libbsd I took this and following two sentences below from master branch description provided in README I took as granted that master does have all the features which are currently available and provided by the project: "This branch must be used for libbsd development. Back ports to the 6-freebsd-12 are allowed." I was surprised to be proven wrong then by Fabrizio here: https://devel.rtems.org/ticket/4723 and by later investigation which shows that 6-freebsd-12 branch accumulated NFS work by Chris done in 2021 which is not presented on master. I've investigated just NFS as this was my focus here. So if 6-freebsd-12 became development branch of some sort, then it would be great to have that clarified in the project README file to prevent users confusion? Or if the policy is still the same, then perhaps some branch sync is needed here? That currently is an open issue. Basically there is a pending patch set that should fix that since several months. But there is a disagreement about some of the changes in that patch set (and about the patches checked in to 6-freebsd-12). Therefore, it still hasn't been merged. If you want to know some more about the problematic points, I recommend reading this (long) thread: https://lists.rtems.org/pipermail/devel/2023-January/074164.html The statement that development has to happen on the master branch is still true. The master is intended to track the FreeBSD upstream development. Only changes on that branch are guaranteed to live through an upgrade to a newer base version of FreeBSD. It's very unfortunate, that there are some patches on the 6-freebsd-12 branch only. On the long term, that issue has to be resolved. I have been investigating this problem in the background, and I have some findings and some questions. First, I have found that there is a most-common ancestor between master and 6-freebsd-12 at commit https://git.rtems.org/rtems-libbsd/commit/?h=6-freebsd-12=2ce13cf6dc73855f28bc7edbbc64dc4b482a4976 This is at least promising that the discrepancy between the branches can be resolved. The proposed pending patch set to "fix" the NFS issue does not fix the underlying problem. Instead, it introduces further divergence between the branches. I would instead suggest that we should resolve to fix the underlying problem. I can see two paths forward. 1. Abandon 6-freebsd-12 after releasing 6. This is probably not ideal since what I understand is some users have projects based on 6-freebsd-12 and would like an upgrade path. (I guess there is also the option to abandon master, which also makes little sense.) A variant for this would be to introduce a 6-freebsd-13 that is based on the master branch as soon as we have one. That would allow a longer maintenance because FreeBSD 12 reaches it's EoL December 2023. 2. Pull commits from 6-freebsd-12 into master to make sure master is the development head. in the future, reject patches that only go toward release branches. This has its own problems too. It can realistically only be done in three ways: Please note that Sebastian mentioned that the file descriptors broke the NTP support (at least I think it was NTP; possible that it was another submodule). So picking the current version of the patches into the master without adding fixes makes the master unusable for some cases. Fixing the problems after making the branches the same will be better for the long-term, if we can find a path to do it. 2a: Rebase master and cherry-pick commits from 6-freebsd-12 and master back into master. This rewrites the history of master, and unfortunately will cause the head of 5-freebsd-12 and the tags for rtems-5 to no longer exist on the master branch. They will still exist in the '5' branch. The advantage is in the end there will be a linear history of development on master that reflects the timeline of actual development that spanned both branches. Theoretically, this should make it easier to git-bisect. 2b: Cherry-pick commits from 6-freebsd-12 to master and fix conflicts. This puts all the missing commits from 6-freebsd-12 on to the current head of master. I don't know how messy this would be. It ends up making the history of master convoluted to understand, with fairly old commits from 2018 being placed on top of newer commits from 2020s. 2c: Merge 6-freebsd-12 into master and fixup conflicts in the merge commit. This is pretty similar to 2a but
