Re: [PATCH rtems-libbsd] CONTRIBUTING: Sharpen priority development goals
Hello Chris, On 2023-02-07 05:42, Chris Johns wrote: On 6/2/2023 7:07 pm, Christian MAUDERER wrote: Hello Chris, thanks for your feedback on the patch. No problem :) On 2023-02-06 05:16, Chris Johns wrote: On 3/2/2023 6:31 pm, Christian Mauderer wrote: This patch tries to make the most important goals of LibBSD development more clear based on the results of the discussion on the mailing list: https://lists.rtems.org/pipermail/devel/2023-January/074164.html --- CONTRIBUTING.rst | 39 ++- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 0b6fc7a0..31561f6a 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -21,15 +21,36 @@ RTEMS specific support files, general guidelines on what modifications to the FreeBSD source are permitted, and some other topics. For building the library, see the `README `_. -Goals of the LibBSD activity are - -* provide functionality from FreeBSD to RTEMS, -* ease updating to future FreeBSD versions, -* ease tracking changes in FreeBSD code, -* minimize manual changes in FreeBSD code. - -We will work to push our changes upstream to the FreeBSD Project and minimize -changes required at each update point. +Every change to LibBSD has to consider the following points in groups of +descending priority: + +#. Real-time Impacts + Maintainability Loss + * LibBSD itself doesn't make any real time claims because it is basically + FreeBSD and they don't make any real time claims either. But all + development in LibBSD must make sure that the real time ability of the + RTEMS core system or the application is not affected. + * It's utterly important that LibBSD is simple to maintain. One of the most + important points for that is that upgrades to newer FreeBSD versions have + to be easy. Correct and it is important we adopt and use what FreeBSD provides rather than adding extra complexity. I wrote about this in the FreeBSD journal years ago. The bespoke handing of fd and file objects is something I consider an unneeded complexity for specific system related reasons. We need NFSv4 and that uses VFS and that in turn uses the FreeBSD fd and file objects. I'm trying to find out what rules are agreed by most of us. I feel we need simplicity. I was surprised by the number of undefined rules my changes tripped over and to be honest I still have no idea what they are, why they exist and who made them? It's of course OK to bring examples. But I think we should try to evaluate the currently discussed patch set based on the results together with as many of the other maintainers as possible as soon as we agree on the rules that we want to use to evaluate them. The rules for the changes I made are simple. It is transparency of the FreeBSD code. The file descriptors are an example where I know that Sebastian and you will argument that their solution is the right one. He may but his view is only one view and fixed and I am sorry but the forceful and sometimes short nature of his responses has not helped his cause. Then let's hope that my tendency to write long mails can help to bring the discussion a bit forward ;-) I found the port of NFSv4 easy until I hit the fd/file parts. The complexity then grew and as I attempted to sort one piece out I hit another location and in the end I had little confidence about the changes I had made. Each change started to appear like a slight variation of the same thing for each descriptor type. In the end I decided the size of changes had grown to big so I decided to switch direction and bring across the FreeBSD code as close as possible and to move libbsd in the that direction because the closer we had VFS to FreeBSD the simpler the testing of complex pieces like the vnode cache and lookup would be. As far as I understood Sebastian, the fd/file parts add quite a bit of code which has to be maintained too and which duplicates part of RTEMS core infrastructure. Is it really simpler to understand more code and an extra layer? Maybe that also depends on the background of the person who looks at the code. If someone reads that code first time: Let's assume he or she has worked with FreeBSD sources before. In that case it might is simpler to understand the code if the FreeBSD file descriptors are in the sources. Let's assume the other case: He or she has worked only with small real time systems. In that case it might is quite odd that there are two code parts that more or less do the same and handle files. It is a trade-off whether it's simpler to cut above the fd/file parts or below them. And most likely it's one that is very subjective. It is important to note we need code the project can maintain and as a result of my deep dive I now question who can maintain the fd/file support as it was? You could ask the same question: Who (except for you) can maintain fd/file support as it is now on
Re: [PATCH rtems-libbsd] CONTRIBUTING: Sharpen priority development goals
On 6/2/2023 7:07 pm, Christian MAUDERER wrote: > Hello Chris, > > thanks for your feedback on the patch. No problem :) > On 2023-02-06 05:16, Chris Johns wrote: >> On 3/2/2023 6:31 pm, Christian Mauderer wrote: >>> This patch tries to make the most important goals of LibBSD development >>> more clear based on the results of the discussion on the mailing list: >>> >>> https://lists.rtems.org/pipermail/devel/2023-January/074164.html >>> --- >>> CONTRIBUTING.rst | 39 ++- >>> 1 file changed, 30 insertions(+), 9 deletions(-) >>> >>> diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst >>> index 0b6fc7a0..31561f6a 100644 >>> --- a/CONTRIBUTING.rst >>> +++ b/CONTRIBUTING.rst >>> @@ -21,15 +21,36 @@ RTEMS specific support files, general guidelines on what >>> modifications to the >>> FreeBSD source are permitted, and some other topics. For building the >>> library, >>> see the `README `_. >>> -Goals of the LibBSD activity are >>> - >>> -* provide functionality from FreeBSD to RTEMS, >>> -* ease updating to future FreeBSD versions, >>> -* ease tracking changes in FreeBSD code, >>> -* minimize manual changes in FreeBSD code. >>> - >>> -We will work to push our changes upstream to the FreeBSD Project and >>> minimize >>> -changes required at each update point. >>> +Every change to LibBSD has to consider the following points in groups of >>> +descending priority: >>> + >>> +#. Real-time Impacts + Maintainability Loss >>> + * LibBSD itself doesn't make any real time claims because it is >>> basically >>> + FreeBSD and they don't make any real time claims either. But all >>> + development in LibBSD must make sure that the real time ability of the >>> + RTEMS core system or the application is not affected. >>> + * It's utterly important that LibBSD is simple to maintain. One of the >>> most >>> + important points for that is that upgrades to newer FreeBSD versions >>> have >>> + to be easy. >> >> Correct and it is important we adopt and use what FreeBSD provides rather >> than >> adding extra complexity. I wrote about this in the FreeBSD journal years ago. >> The bespoke handing of fd and file objects is something I consider an >> unneeded >> complexity for specific system related reasons. We need NFSv4 and that uses >> VFS >> and that in turn uses the FreeBSD fd and file objects. >> > > I'm trying to find out what rules are agreed by most of us. I feel we need simplicity. I was surprised by the number of undefined rules my changes tripped over and to be honest I still have no idea what they are, why they exist and who made them? > It's of course OK to > bring examples. But I think we should try to evaluate the currently discussed > patch set based on the results together with as many of the other maintainers > as > possible as soon as we agree on the rules that we want to use to evaluate > them. The rules for the changes I made are simple. It is transparency of the FreeBSD code. > The file descriptors are an example where I know that Sebastian and you will > argument that their solution is the right one. He may but his view is only one view and fixed and I am sorry but the forceful and sometimes short nature of his responses has not helped his cause. I found the port of NFSv4 easy until I hit the fd/file parts. The complexity then grew and as I attempted to sort one piece out I hit another location and in the end I had little confidence about the changes I had made. Each change started to appear like a slight variation of the same thing for each descriptor type. In the end I decided the size of changes had grown to big so I decided to switch direction and bring across the FreeBSD code as close as possible and to move libbsd in the that direction because the closer we had VFS to FreeBSD the simpler the testing of complex pieces like the vnode cache and lookup would be. It is important to note we need code the project can maintain and as a result of my deep dive I now question who can maintain the fd/file support as it was? > I think that can only be decided with the help of some more people. I agree. This is about the long term and beyond any one of us. >> Source transparency is what matters and as a test of what is acceptable it >> must >> be preferred between competing implementations. >> >>> +#. Transparency Loss + Modularity Loss + Code/RAM Size Increase >>> + * LibBSD code should be easy to read and easy to debug. Changes should >>> be >>> + integrated in a way that are easy to understand. Of course that's a >>> + subjective goal. As a general rule: If it adds lot's of extra code or >>> even >>> + more layers than already exist in FreeBSD, it's harder to understand. >>> + * There are a number of methods used in LibBSD to make it modular. If >>> you >>> + add new functionality, use one of the existing methods to allow >>> enabling or >>> + disabling your module. For example make sure that the
Re: [PATCH rtems-libbsd] CONTRIBUTING: Sharpen priority development goals
On 2/6/23 05:16, Chris Johns wrote: + * A lot of different hardware uses LibBSD as network or USB stack or maybe in + the future even only for other subsystems. Some of the targets have + hundreds of megabytes memory. Others can only have a few megabytes (like + the ATSAMV71). Make sure that changes don't increase the RAM / Flash size + of the default build so that it can't be used on the small targets. This is not realistic or achievable and I find confusing the reasons it is being pushed over and over. I would have not have agreed to this being added before now and nothing has changed. The central reason for rejecting this statement is a change in FreeBSD may add a few meg of memory to the footprint and this type of statement would conflict with that addition and that in turn would conflict with the need for transparency of source. And as stated before transparency must be preferred. System requirements are for the developers of those systems and not RTEMS. Derating designs is an important part of system design and not the domain of this project. Memory constrained systems can consider another networking stack option or bespoke changes internally maintained. That is a cost trade off no one here can help make. From the discussion it looks like guys from embedded brains push that policy forward I mean policy that memory consumption is somehow important metric for patch acceptance. Seeing their commit track log I would not oppose that rule that much as at the end of the day somebody needs to do the job and from the log it looks like embedded brains stay behind their word. As an RTEMS user here, I'm curious if this discussion about few paragraphs in CONTRIBUTING file is not too much over-thinking of the issue. So far majority of patches were done by just three subjects: embedded brains, aorcorp and you. So I would assume that engineering discussion with respect for each other contribution and energy spent should result in a policy which may be vague and compromise but working well for what you all guys do for us RTEMS users here... Thanks! Karel rtems@silence:~/git/rtems/ssh-rtems-libbsd$ git shortlog -s -n --all --no-merges 1554 Sebastian Huber 242 Christian Mauderer 242 Joel Sherrill 175 Chris Johns 163 Jennifer Averett 31 Kevin Kirspel 31 Kinsey Moore 25 Vijay Kumar Banerjee 21 Jan Sommer 12 Moyano, Gabriel 12 Sichen Zhao ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [PATCH rtems-libbsd] CONTRIBUTING: Sharpen priority development goals
Hello Chris, thanks for your feedback on the patch. On 2023-02-06 05:16, Chris Johns wrote: On 3/2/2023 6:31 pm, Christian Mauderer wrote: This patch tries to make the most important goals of LibBSD development more clear based on the results of the discussion on the mailing list: https://lists.rtems.org/pipermail/devel/2023-January/074164.html --- CONTRIBUTING.rst | 39 ++- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 0b6fc7a0..31561f6a 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -21,15 +21,36 @@ RTEMS specific support files, general guidelines on what modifications to the FreeBSD source are permitted, and some other topics. For building the library, see the `README `_. -Goals of the LibBSD activity are - -* provide functionality from FreeBSD to RTEMS, -* ease updating to future FreeBSD versions, -* ease tracking changes in FreeBSD code, -* minimize manual changes in FreeBSD code. - -We will work to push our changes upstream to the FreeBSD Project and minimize -changes required at each update point. +Every change to LibBSD has to consider the following points in groups of +descending priority: + +#. Real-time Impacts + Maintainability Loss + * LibBSD itself doesn't make any real time claims because it is basically + FreeBSD and they don't make any real time claims either. But all + development in LibBSD must make sure that the real time ability of the + RTEMS core system or the application is not affected. + * It's utterly important that LibBSD is simple to maintain. One of the most + important points for that is that upgrades to newer FreeBSD versions have + to be easy. Correct and it is important we adopt and use what FreeBSD provides rather than adding extra complexity. I wrote about this in the FreeBSD journal years ago. The bespoke handing of fd and file objects is something I consider an unneeded complexity for specific system related reasons. We need NFSv4 and that uses VFS and that in turn uses the FreeBSD fd and file objects. I'm trying to find out what rules are agreed by most of us. It's of course OK to bring examples. But I think we should try to evaluate the currently discussed patch set based on the results together with as many of the other maintainers as possible as soon as we agree on the rules that we want to use to evaluate them. The file descriptors are an example where I know that Sebastian and you will argument that their solution is the right one. I think that can only be decided with the help of some more people. Source transparency is what matters and as a test of what is acceptable it must be preferred between competing implementations. +#. Transparency Loss + Modularity Loss + Code/RAM Size Increase + * LibBSD code should be easy to read and easy to debug. Changes should be + integrated in a way that are easy to understand. Of course that's a + subjective goal. As a general rule: If it adds lot's of extra code or even + more layers than already exist in FreeBSD, it's harder to understand. + * There are a number of methods used in LibBSD to make it modular. If you + add new functionality, use one of the existing methods to allow enabling or + disabling your module. For example make sure that the linker can remove + unused modules. If that doesn't work for your feature, try to use the + buildsets to allow to switch a module on or off. + * A lot of different hardware uses LibBSD as network or USB stack or maybe in + the future even only for other subsystems. Some of the targets have + hundreds of megabytes memory. Others can only have a few megabytes (like + the ATSAMV71). Make sure that changes don't increase the RAM / Flash size + of the default build so that it can't be used on the small targets. This is not realistic or achievable and I find confusing the reasons it is being pushed over and over. I would have not have agreed to this being added before now and nothing has changed. The central reason for rejecting this statement is a change in FreeBSD may add a few meg of memory to the footprint and this type of statement would conflict with that addition and that in turn would conflict with the need for transparency of source. And as stated before transparency must be preferred. Maintainability is the point that has the highest priority in my suggestion. I grouped transparency, modularity and size like Gedare suggested because there might be cases where we want to prefer one over the other. Transparency is also a bit difficult and often subjective. I defined it here as 'easy to read' and 'easy to debug'. These two alone can conflict each other. For example more layers can sometimes make something simpler to read because it is nicely abstracted, but it can make it horrible hard to debug a problem because you have to step through all the
Re: [PATCH rtems-libbsd] CONTRIBUTING: Sharpen priority development goals
On 06.02.23 05:16, Chris Johns wrote: On 3/2/2023 6:31 pm, Christian Mauderer wrote: This patch tries to make the most important goals of LibBSD development more clear based on the results of the discussion on the mailing list: https://lists.rtems.org/pipermail/devel/2023-January/074164.html --- CONTRIBUTING.rst | 39 ++- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 0b6fc7a0..31561f6a 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -21,15 +21,36 @@ RTEMS specific support files, general guidelines on what modifications to the FreeBSD source are permitted, and some other topics. For building the library, see the `README `_. -Goals of the LibBSD activity are - -* provide functionality from FreeBSD to RTEMS, -* ease updating to future FreeBSD versions, -* ease tracking changes in FreeBSD code, -* minimize manual changes in FreeBSD code. - -We will work to push our changes upstream to the FreeBSD Project and minimize -changes required at each update point. +Every change to LibBSD has to consider the following points in groups of +descending priority: + +#. Real-time Impacts + Maintainability Loss + * LibBSD itself doesn't make any real time claims because it is basically + FreeBSD and they don't make any real time claims either. But all + development in LibBSD must make sure that the real time ability of the + RTEMS core system or the application is not affected. + * It's utterly important that LibBSD is simple to maintain. One of the most + important points for that is that upgrades to newer FreeBSD versions have + to be easy. Correct and it is important we adopt and use what FreeBSD provides rather than adding extra complexity. I wrote about this in the FreeBSD journal years ago. We already have a file descriptor to file object mapping in RTEMS so adding an additional file descriptor to file object mapping on top of it makes little sense. The libbsd should add complementary functionality otherwise you could just use FreeBSD directly. By adding the FreeBSD file descriptors you added a lot of unnecessary complexity. The FreeBSD file descriptor handing has to meet a couple of requirements which are not present in the RTEMS context. The bespoke handing of fd and file objects is something I consider an unneeded complexity for specific system related reasons. We need NFSv4 and that uses VFS and that in turn uses the FreeBSD fd and file objects. This conclusion is simply wrong. You don't need the FreeBSD file descriptors for the VFS support. See: https://github.com/sebhub/rtems-libbsd/tree/filedesc For VFS you need a proper struct file object associated with a file descriptor. For sockets, you don't need a struct file object. What is your hard technical reason to keep the FreeBSD file descriptors? Source transparency is what matters and as a test of what is acceptable it must be preferred between competing implementations. Whatever source transparency is, it is just one aspect out of many. -- embedded brains GmbH Herr Sebastian HUBER Dornierstr. 4 82178 Puchheim Germany email: sebastian.hu...@embedded-brains.de phone: +49-89-18 94 741 - 16 fax: +49-89-18 94 741 - 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [PATCH rtems-libbsd] CONTRIBUTING: Sharpen priority development goals
On 3/2/2023 6:31 pm, Christian Mauderer wrote: > This patch tries to make the most important goals of LibBSD development > more clear based on the results of the discussion on the mailing list: > > https://lists.rtems.org/pipermail/devel/2023-January/074164.html > --- > CONTRIBUTING.rst | 39 ++- > 1 file changed, 30 insertions(+), 9 deletions(-) > > diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst > index 0b6fc7a0..31561f6a 100644 > --- a/CONTRIBUTING.rst > +++ b/CONTRIBUTING.rst > @@ -21,15 +21,36 @@ RTEMS specific support files, general guidelines on what > modifications to the > FreeBSD source are permitted, and some other topics. For building the > library, > see the `README `_. > > -Goals of the LibBSD activity are > - > -* provide functionality from FreeBSD to RTEMS, > -* ease updating to future FreeBSD versions, > -* ease tracking changes in FreeBSD code, > -* minimize manual changes in FreeBSD code. > - > -We will work to push our changes upstream to the FreeBSD Project and minimize > -changes required at each update point. > +Every change to LibBSD has to consider the following points in groups of > +descending priority: > + > +#. Real-time Impacts + Maintainability Loss > + * LibBSD itself doesn't make any real time claims because it is basically > + FreeBSD and they don't make any real time claims either. But all > + development in LibBSD must make sure that the real time ability of the > + RTEMS core system or the application is not affected. > + * It's utterly important that LibBSD is simple to maintain. One of the > most > + important points for that is that upgrades to newer FreeBSD versions > have > + to be easy. Correct and it is important we adopt and use what FreeBSD provides rather than adding extra complexity. I wrote about this in the FreeBSD journal years ago. The bespoke handing of fd and file objects is something I consider an unneeded complexity for specific system related reasons. We need NFSv4 and that uses VFS and that in turn uses the FreeBSD fd and file objects. Source transparency is what matters and as a test of what is acceptable it must be preferred between competing implementations. > +#. Transparency Loss + Modularity Loss + Code/RAM Size Increase > + * LibBSD code should be easy to read and easy to debug. Changes should be > + integrated in a way that are easy to understand. Of course that's a > + subjective goal. As a general rule: If it adds lot's of extra code or > even > + more layers than already exist in FreeBSD, it's harder to understand. > + * There are a number of methods used in LibBSD to make it modular. If you > + add new functionality, use one of the existing methods to allow > enabling or > + disabling your module. For example make sure that the linker can remove > + unused modules. If that doesn't work for your feature, try to use the > + buildsets to allow to switch a module on or off. > + * A lot of different hardware uses LibBSD as network or USB stack or > maybe in > + the future even only for other subsystems. Some of the targets have > + hundreds of megabytes memory. Others can only have a few megabytes > (like > + the ATSAMV71). Make sure that changes don't increase the RAM / Flash > size > + of the default build so that it can't be used on the small targets. This is not realistic or achievable and I find confusing the reasons it is being pushed over and over. I would have not have agreed to this being added before now and nothing has changed. The central reason for rejecting this statement is a change in FreeBSD may add a few meg of memory to the footprint and this type of statement would conflict with that addition and that in turn would conflict with the need for transparency of source. And as stated before transparency must be preferred. System requirements are for the developers of those systems and not RTEMS. Derating designs is an important part of system design and not the domain of this project. Memory constrained systems can consider another networking stack option or bespoke changes internally maintained. That is a cost trade off no one here can help make. > +#. Performance Loss > + * There are applications, that require (for example) a high network > + throughput or a fast storage access. Make sure to take that into > account > + if adding changes. I prefer we do not add statements that have no definition or boundaries someone could use to test against. Selection of RTEMS in a systems is the choice of the system designer. There are systems RTEMS is not a good fit for. Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
[PATCH rtems-libbsd] CONTRIBUTING: Sharpen priority development goals
This patch tries to make the most important goals of LibBSD development more clear based on the results of the discussion on the mailing list: https://lists.rtems.org/pipermail/devel/2023-January/074164.html --- CONTRIBUTING.rst | 39 ++- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 0b6fc7a0..31561f6a 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -21,15 +21,36 @@ RTEMS specific support files, general guidelines on what modifications to the FreeBSD source are permitted, and some other topics. For building the library, see the `README `_. -Goals of the LibBSD activity are - -* provide functionality from FreeBSD to RTEMS, -* ease updating to future FreeBSD versions, -* ease tracking changes in FreeBSD code, -* minimize manual changes in FreeBSD code. - -We will work to push our changes upstream to the FreeBSD Project and minimize -changes required at each update point. +Every change to LibBSD has to consider the following points in groups of +descending priority: + +#. Real-time Impacts + Maintainability Loss + * LibBSD itself doesn't make any real time claims because it is basically + FreeBSD and they don't make any real time claims either. But all + development in LibBSD must make sure that the real time ability of the + RTEMS core system or the application is not affected. + * It's utterly important that LibBSD is simple to maintain. One of the most + important points for that is that upgrades to newer FreeBSD versions have + to be easy. +#. Transparency Loss + Modularity Loss + Code/RAM Size Increase + * LibBSD code should be easy to read and easy to debug. Changes should be + integrated in a way that are easy to understand. Of course that's a + subjective goal. As a general rule: If it adds lot's of extra code or even + more layers than already exist in FreeBSD, it's harder to understand. + * There are a number of methods used in LibBSD to make it modular. If you + add new functionality, use one of the existing methods to allow enabling or + disabling your module. For example make sure that the linker can remove + unused modules. If that doesn't work for your feature, try to use the + buildsets to allow to switch a module on or off. + * A lot of different hardware uses LibBSD as network or USB stack or maybe in + the future even only for other subsystems. Some of the targets have + hundreds of megabytes memory. Others can only have a few megabytes (like + the ATSAMV71). Make sure that changes don't increase the RAM / Flash size + of the default build so that it can't be used on the small targets. +#. Performance Loss + * There are applications, that require (for example) a high network + throughput or a fast storage access. Make sure to take that into account + if adding changes. What is in the Git Repository = -- 2.35.3 ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
