Re: Fwd: Identify 3rd party source in spec?
On Mon, Nov 7, 2022 at 9:47 AM Gedare Bloom wrote: > > On Mon, Nov 7, 2022 at 2:19 AM Sebastian Huber > wrote: > > > > On 04/11/2022 21:08, Gedare Bloom wrote: > > >> Given the complexity of this tagging, I'm going to start with just the > > >> true/false approach to categorize third-party sources. We can do > > >> something like the above in the future. > > > I have come across one problem with my approach. Headers that aren't > > > installed (e.g., #include "fastlz.h") do not appear within the build > > > system. These might need to be identified and handled as some kind of > > > special case. There are 79 such files in cpukit. > > > rtems/cpukit$ find . -name "*.h" | grep -v include | wc > > > 79 792072 > > > > We could use "third-party: null" for non-third-party files. Then start with: > > > > third-party: > >extra-files: > >- cpukit/libdl/fastlz.h > > > > third-party: > >extra-files: [] > > > > New third-party attributes can be added on demand. > > > This makes sense to me as a way to handle these files. Thanks for the > suggestion. I think I'll mock something up today and hopefully have a > full patch set ready for cpukit soon. > I put something together, now available at: https://git.rtems.org/gedare/rtems.git/log/?h=split-3rd-party I think this is correct/complete enough to use as a starting point for the cpukit, at least based on the list of third-party sources provided by Ryan/OAR in the other email thread. > > -- > > embedded brains GmbH > > Herr Sebastian HUBER > > Dornierstr. 4 > > 82178 Puchheim > > Germany > > email: sebastian.hu...@embedded-brains.de > > phone: +49-89-18 94 741 - 16 > > fax: +49-89-18 94 741 - 08 > > > > Registergericht: Amtsgericht München > > Registernummer: HRB 157899 > > Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler > > Unsere Datenschutzerklärung finden Sie hier: > > https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On Mon, Nov 7, 2022 at 2:19 AM Sebastian Huber wrote: > > On 04/11/2022 21:08, Gedare Bloom wrote: > >> Given the complexity of this tagging, I'm going to start with just the > >> true/false approach to categorize third-party sources. We can do > >> something like the above in the future. > > I have come across one problem with my approach. Headers that aren't > > installed (e.g., #include "fastlz.h") do not appear within the build > > system. These might need to be identified and handled as some kind of > > special case. There are 79 such files in cpukit. > > rtems/cpukit$ find . -name "*.h" | grep -v include | wc > > 79 792072 > > We could use "third-party: null" for non-third-party files. Then start with: > > third-party: >extra-files: >- cpukit/libdl/fastlz.h > > third-party: >extra-files: [] > > New third-party attributes can be added on demand. > This makes sense to me as a way to handle these files. Thanks for the suggestion. I think I'll mock something up today and hopefully have a full patch set ready for cpukit soon. > -- > embedded brains GmbH > Herr Sebastian HUBER > Dornierstr. 4 > 82178 Puchheim > Germany > email: sebastian.hu...@embedded-brains.de > phone: +49-89-18 94 741 - 16 > fax: +49-89-18 94 741 - 08 > > Registergericht: Amtsgericht München > Registernummer: HRB 157899 > Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler > Unsere Datenschutzerklärung finden Sie hier: > https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On 04/11/2022 21:08, Gedare Bloom wrote: Given the complexity of this tagging, I'm going to start with just the true/false approach to categorize third-party sources. We can do something like the above in the future. I have come across one problem with my approach. Headers that aren't installed (e.g., #include "fastlz.h") do not appear within the build system. These might need to be identified and handled as some kind of special case. There are 79 such files in cpukit. rtems/cpukit$ find . -name "*.h" | grep -v include | wc 79 792072 We could use "third-party: null" for non-third-party files. Then start with: third-party: extra-files: - cpukit/libdl/fastlz.h third-party: extra-files: [] New third-party attributes can be added on demand. -- embedded brains GmbH Herr Sebastian HUBER Dornierstr. 4 82178 Puchheim Germany email: sebastian.hu...@embedded-brains.de phone: +49-89-18 94 741 - 16 fax: +49-89-18 94 741 - 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On 5/11/2022 4:00 am, Gedare Bloom wrote: Given the complexity of this tagging, I'm going to start with just the true/false approach to categorize third-party sources. We can do something like the above in the future. Sounds good. We can consider a dict when someone maps out how to manage 3rd party source. I think it would premature to do it without further consideration. Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On Fri, Nov 4, 2022 at 11:00 AM Gedare Bloom wrote:
>
> On Fri, Nov 4, 2022 at 9:55 AM Gedare Bloom wrote:
> >
> > On Fri, Nov 4, 2022 at 8:58 AM Sebastian Huber
> > wrote:
> > >
> > > On 04/11/2022 15:38, Gedare Bloom wrote:
> > > > On Fri, Nov 4, 2022 at 2:59 AM Sebastian Huber
> > > > wrote:
> > > >> On 31/10/2022 20:01, Gedare Bloom wrote:
> > > >>> I would like to float the idea of managing 3rd party source tracking
> > > >>> through the build system spec files. I believe this would be the most
> > > >>> efficient way to maintain this information, and we can leverage the
> > > >>> existing build system code for tasks such as automatic format checks,
> > > >>> generating lists of third-party code, etc.
> > > >>>
> > > >>> This will require refactoring some spec files to pull 3rd party code
> > > >>> out to a separate .yml file that gets linked. Once that is done, then
> > > >>> we could add another attribute for this tracking purpose. I would like
> > > >>> to keep it simple as a boolean, maybe just "third-party: true/false"
> > > >>> Attached is an example patch showing how this might work for the
> > > >>> dtc/libfdt code as a build objects item type 'obj', and for zlib
> > > >>> library as a build library item type 'lib' with some proof-of-concept
> > > >>> code for generating a listing of third party source files.
> > > >> In the future we could expand this third-party attribute to a
> > > >> dictionary
> > > >> to provide more detailed information about the third-party sources, for
> > > >> example the upstream Git repository, the import commit, path
> > > >> translation
> > > >> mappings, post and pre import actions. This could be used to
> > > >> automatically generate updates from upstream changes.
> > > >>
> > > > Nice. I don't see an explicit "dictionary" type for attributes, but I
> > > > do see that dict-like behavior could be achieved using a 'list' of
> > > > attributes nested below an attribute. I have the time allocated that i
> > > > could try to learn how to do this with a simple example, for example
> > > > to add a "upstream" and an "import-commit" field. Then, the
> > > > third-party indicator will change from
> > > > third-party: false ->
> > > > third-party: []
> > > > third-party: true->
> > > > third-party:
> > > > - upstream: URL
> > > > - import-commit: git-sha
> > > > I would guess. Would this kind of structure "just work" in the current
> > > > support of spec/build or will it require some integration with
> > > > rtems-central? Or, am I following the wrong ideas,
> > >
> > > third-party: null
> > >
> > > would indicate that this object/library doesn't have an upstream project.
> > >
> > > If there is an upstream project, then we could describe this through
> > > additional attributes, for example:
> > >
> > > third-party:
> > >type: git
> > >commit: abc
> > >repository: def
> > >file-map:
> > >- replace regex 1
> > >- replace regex 2
> > >
> >
> > OK, that makes sense. I think I can make a first pass just to put in
> > the framework for extending this later. Taking DTC as an example,
> > maybe something like the following makes sense?
> >
> > third-party:
> > - type: git
> > - repository: git://git.kernel.org/pub/scm/utils/dtc/dtc.git
> > - commits:
> > - upstream: c0c2e115f82ed3bc5f9d3f9e5380f0f7e81a1c21
> > - import: 175263ec91f00f6668ae7f28f385850a1c77bf6b
> > - update: cherry-pick
> >
> >
> > Where we will have to define some update methods such as applying
> > patches from the upstream (cherry-pick).
> >
> > As POC I can process this with wscript and obtain
> > [{'type': 'git'}, {'repository':
> > 'git://git.kernel.org/pub/scm/utils/dtc/dtc.git'}, {'commits':
> > [{'upstream': 'c0c2e115f82ed3bc5f9d3f9e5380f0f7e81a1c21'}, {'import':
> > '175263ec91f00f6668ae7f28f385850a1c77bf6b'}]}, {'update':
> > 'cherry-pick'}, {'source': ['cpukit/dtc/libfdt/fdt.c',
> > 'cpukit/dtc/libfdt/fdt_addresses.c',
> > 'cpukit/dtc/libfdt/fdt_empty_tree.c', 'cpukit/dtc/libfdt/fdt_ro.c',
> > 'cpukit/dtc/libfdt/fdt_rw.c', 'cpukit/dtc/libfdt/fdt_strerror.c',
> > 'cpukit/dtc/libfdt/fdt_sw.c', 'cpukit/dtc/libfdt/fdt_wip.c']}]
> >
> > If this looks reasonable, I can try to add these fields as part of my
> > sweep through the cpukit tagging third-party sources.
> >
>
> Given the complexity of this tagging, I'm going to start with just the
> true/false approach to categorize third-party sources. We can do
> something like the above in the future.
I have come across one problem with my approach. Headers that aren't
installed (e.g., #include "fastlz.h") do not appear within the build
system. These might need to be identified and handled as some kind of
special case. There are 79 such files in cpukit.
rtems/cpukit$ find . -name "*.h" | grep -v include | wc
79 792072
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On Fri, Nov 4, 2022 at 9:55 AM Gedare Bloom wrote:
>
> On Fri, Nov 4, 2022 at 8:58 AM Sebastian Huber
> wrote:
> >
> > On 04/11/2022 15:38, Gedare Bloom wrote:
> > > On Fri, Nov 4, 2022 at 2:59 AM Sebastian Huber
> > > wrote:
> > >> On 31/10/2022 20:01, Gedare Bloom wrote:
> > >>> I would like to float the idea of managing 3rd party source tracking
> > >>> through the build system spec files. I believe this would be the most
> > >>> efficient way to maintain this information, and we can leverage the
> > >>> existing build system code for tasks such as automatic format checks,
> > >>> generating lists of third-party code, etc.
> > >>>
> > >>> This will require refactoring some spec files to pull 3rd party code
> > >>> out to a separate .yml file that gets linked. Once that is done, then
> > >>> we could add another attribute for this tracking purpose. I would like
> > >>> to keep it simple as a boolean, maybe just "third-party: true/false"
> > >>> Attached is an example patch showing how this might work for the
> > >>> dtc/libfdt code as a build objects item type 'obj', and for zlib
> > >>> library as a build library item type 'lib' with some proof-of-concept
> > >>> code for generating a listing of third party source files.
> > >> In the future we could expand this third-party attribute to a dictionary
> > >> to provide more detailed information about the third-party sources, for
> > >> example the upstream Git repository, the import commit, path translation
> > >> mappings, post and pre import actions. This could be used to
> > >> automatically generate updates from upstream changes.
> > >>
> > > Nice. I don't see an explicit "dictionary" type for attributes, but I
> > > do see that dict-like behavior could be achieved using a 'list' of
> > > attributes nested below an attribute. I have the time allocated that i
> > > could try to learn how to do this with a simple example, for example
> > > to add a "upstream" and an "import-commit" field. Then, the
> > > third-party indicator will change from
> > > third-party: false ->
> > > third-party: []
> > > third-party: true->
> > > third-party:
> > > - upstream: URL
> > > - import-commit: git-sha
> > > I would guess. Would this kind of structure "just work" in the current
> > > support of spec/build or will it require some integration with
> > > rtems-central? Or, am I following the wrong ideas,
> >
> > third-party: null
> >
> > would indicate that this object/library doesn't have an upstream project.
> >
> > If there is an upstream project, then we could describe this through
> > additional attributes, for example:
> >
> > third-party:
> >type: git
> >commit: abc
> >repository: def
> >file-map:
> >- replace regex 1
> >- replace regex 2
> >
>
> OK, that makes sense. I think I can make a first pass just to put in
> the framework for extending this later. Taking DTC as an example,
> maybe something like the following makes sense?
>
> third-party:
> - type: git
> - repository: git://git.kernel.org/pub/scm/utils/dtc/dtc.git
> - commits:
> - upstream: c0c2e115f82ed3bc5f9d3f9e5380f0f7e81a1c21
> - import: 175263ec91f00f6668ae7f28f385850a1c77bf6b
> - update: cherry-pick
>
>
> Where we will have to define some update methods such as applying
> patches from the upstream (cherry-pick).
>
> As POC I can process this with wscript and obtain
> [{'type': 'git'}, {'repository':
> 'git://git.kernel.org/pub/scm/utils/dtc/dtc.git'}, {'commits':
> [{'upstream': 'c0c2e115f82ed3bc5f9d3f9e5380f0f7e81a1c21'}, {'import':
> '175263ec91f00f6668ae7f28f385850a1c77bf6b'}]}, {'update':
> 'cherry-pick'}, {'source': ['cpukit/dtc/libfdt/fdt.c',
> 'cpukit/dtc/libfdt/fdt_addresses.c',
> 'cpukit/dtc/libfdt/fdt_empty_tree.c', 'cpukit/dtc/libfdt/fdt_ro.c',
> 'cpukit/dtc/libfdt/fdt_rw.c', 'cpukit/dtc/libfdt/fdt_strerror.c',
> 'cpukit/dtc/libfdt/fdt_sw.c', 'cpukit/dtc/libfdt/fdt_wip.c']}]
>
> If this looks reasonable, I can try to add these fields as part of my
> sweep through the cpukit tagging third-party sources.
>
Given the complexity of this tagging, I'm going to start with just the
true/false approach to categorize third-party sources. We can do
something like the above in the future.
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On Fri, Nov 4, 2022 at 8:58 AM Sebastian Huber
wrote:
>
> On 04/11/2022 15:38, Gedare Bloom wrote:
> > On Fri, Nov 4, 2022 at 2:59 AM Sebastian Huber
> > wrote:
> >> On 31/10/2022 20:01, Gedare Bloom wrote:
> >>> I would like to float the idea of managing 3rd party source tracking
> >>> through the build system spec files. I believe this would be the most
> >>> efficient way to maintain this information, and we can leverage the
> >>> existing build system code for tasks such as automatic format checks,
> >>> generating lists of third-party code, etc.
> >>>
> >>> This will require refactoring some spec files to pull 3rd party code
> >>> out to a separate .yml file that gets linked. Once that is done, then
> >>> we could add another attribute for this tracking purpose. I would like
> >>> to keep it simple as a boolean, maybe just "third-party: true/false"
> >>> Attached is an example patch showing how this might work for the
> >>> dtc/libfdt code as a build objects item type 'obj', and for zlib
> >>> library as a build library item type 'lib' with some proof-of-concept
> >>> code for generating a listing of third party source files.
> >> In the future we could expand this third-party attribute to a dictionary
> >> to provide more detailed information about the third-party sources, for
> >> example the upstream Git repository, the import commit, path translation
> >> mappings, post and pre import actions. This could be used to
> >> automatically generate updates from upstream changes.
> >>
> > Nice. I don't see an explicit "dictionary" type for attributes, but I
> > do see that dict-like behavior could be achieved using a 'list' of
> > attributes nested below an attribute. I have the time allocated that i
> > could try to learn how to do this with a simple example, for example
> > to add a "upstream" and an "import-commit" field. Then, the
> > third-party indicator will change from
> > third-party: false ->
> > third-party: []
> > third-party: true->
> > third-party:
> > - upstream: URL
> > - import-commit: git-sha
> > I would guess. Would this kind of structure "just work" in the current
> > support of spec/build or will it require some integration with
> > rtems-central? Or, am I following the wrong ideas,
>
> third-party: null
>
> would indicate that this object/library doesn't have an upstream project.
>
> If there is an upstream project, then we could describe this through
> additional attributes, for example:
>
> third-party:
>type: git
>commit: abc
>repository: def
>file-map:
>- replace regex 1
>- replace regex 2
>
OK, that makes sense. I think I can make a first pass just to put in
the framework for extending this later. Taking DTC as an example,
maybe something like the following makes sense?
third-party:
- type: git
- repository: git://git.kernel.org/pub/scm/utils/dtc/dtc.git
- commits:
- upstream: c0c2e115f82ed3bc5f9d3f9e5380f0f7e81a1c21
- import: 175263ec91f00f6668ae7f28f385850a1c77bf6b
- update: cherry-pick
Where we will have to define some update methods such as applying
patches from the upstream (cherry-pick).
As POC I can process this with wscript and obtain
[{'type': 'git'}, {'repository':
'git://git.kernel.org/pub/scm/utils/dtc/dtc.git'}, {'commits':
[{'upstream': 'c0c2e115f82ed3bc5f9d3f9e5380f0f7e81a1c21'}, {'import':
'175263ec91f00f6668ae7f28f385850a1c77bf6b'}]}, {'update':
'cherry-pick'}, {'source': ['cpukit/dtc/libfdt/fdt.c',
'cpukit/dtc/libfdt/fdt_addresses.c',
'cpukit/dtc/libfdt/fdt_empty_tree.c', 'cpukit/dtc/libfdt/fdt_ro.c',
'cpukit/dtc/libfdt/fdt_rw.c', 'cpukit/dtc/libfdt/fdt_strerror.c',
'cpukit/dtc/libfdt/fdt_sw.c', 'cpukit/dtc/libfdt/fdt_wip.c']}]
If this looks reasonable, I can try to add these fields as part of my
sweep through the cpukit tagging third-party sources.
> --
> embedded brains GmbH
> Herr Sebastian HUBER
> Dornierstr. 4
> 82178 Puchheim
> Germany
> email: sebastian.hu...@embedded-brains.de
> phone: +49-89-18 94 741 - 16
> fax: +49-89-18 94 741 - 08
>
> Registergericht: Amtsgericht München
> Registernummer: HRB 157899
> Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
> Unsere Datenschutzerklärung finden Sie hier:
> https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On 04/11/2022 15:38, Gedare Bloom wrote: On Fri, Nov 4, 2022 at 2:59 AM Sebastian Huber wrote: On 31/10/2022 20:01, Gedare Bloom wrote: I would like to float the idea of managing 3rd party source tracking through the build system spec files. I believe this would be the most efficient way to maintain this information, and we can leverage the existing build system code for tasks such as automatic format checks, generating lists of third-party code, etc. This will require refactoring some spec files to pull 3rd party code out to a separate .yml file that gets linked. Once that is done, then we could add another attribute for this tracking purpose. I would like to keep it simple as a boolean, maybe just "third-party: true/false" Attached is an example patch showing how this might work for the dtc/libfdt code as a build objects item type 'obj', and for zlib library as a build library item type 'lib' with some proof-of-concept code for generating a listing of third party source files. In the future we could expand this third-party attribute to a dictionary to provide more detailed information about the third-party sources, for example the upstream Git repository, the import commit, path translation mappings, post and pre import actions. This could be used to automatically generate updates from upstream changes. Nice. I don't see an explicit "dictionary" type for attributes, but I do see that dict-like behavior could be achieved using a 'list' of attributes nested below an attribute. I have the time allocated that i could try to learn how to do this with a simple example, for example to add a "upstream" and an "import-commit" field. Then, the third-party indicator will change from third-party: false -> third-party: [] third-party: true-> third-party: - upstream: URL - import-commit: git-sha I would guess. Would this kind of structure "just work" in the current support of spec/build or will it require some integration with rtems-central? Or, am I following the wrong ideas, third-party: null would indicate that this object/library doesn't have an upstream project. If there is an upstream project, then we could describe this through additional attributes, for example: third-party: type: git commit: abc repository: def file-map: - replace regex 1 - replace regex 2 -- embedded brains GmbH Herr Sebastian HUBER Dornierstr. 4 82178 Puchheim Germany email: sebastian.hu...@embedded-brains.de phone: +49-89-18 94 741 - 16 fax: +49-89-18 94 741 - 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On Fri, Nov 4, 2022 at 2:59 AM Sebastian Huber wrote: > > On 31/10/2022 20:01, Gedare Bloom wrote: > > I would like to float the idea of managing 3rd party source tracking > > through the build system spec files. I believe this would be the most > > efficient way to maintain this information, and we can leverage the > > existing build system code for tasks such as automatic format checks, > > generating lists of third-party code, etc. > > > > This will require refactoring some spec files to pull 3rd party code > > out to a separate .yml file that gets linked. Once that is done, then > > we could add another attribute for this tracking purpose. I would like > > to keep it simple as a boolean, maybe just "third-party: true/false" > > Attached is an example patch showing how this might work for the > > dtc/libfdt code as a build objects item type 'obj', and for zlib > > library as a build library item type 'lib' with some proof-of-concept > > code for generating a listing of third party source files. > > In the future we could expand this third-party attribute to a dictionary > to provide more detailed information about the third-party sources, for > example the upstream Git repository, the import commit, path translation > mappings, post and pre import actions. This could be used to > automatically generate updates from upstream changes. > Nice. I don't see an explicit "dictionary" type for attributes, but I do see that dict-like behavior could be achieved using a 'list' of attributes nested below an attribute. I have the time allocated that i could try to learn how to do this with a simple example, for example to add a "upstream" and an "import-commit" field. Then, the third-party indicator will change from third-party: false -> third-party: [] third-party: true-> third-party: - upstream: URL - import-commit: git-sha I would guess. Would this kind of structure "just work" in the current support of spec/build or will it require some integration with rtems-central? Or, am I following the wrong ideas, -Gedare > -- > embedded brains GmbH > Herr Sebastian HUBER > Dornierstr. 4 > 82178 Puchheim > Germany > email: sebastian.hu...@embedded-brains.de > phone: +49-89-18 94 741 - 16 > fax: +49-89-18 94 741 - 08 > > Registergericht: Amtsgericht München > Registernummer: HRB 157899 > Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler > Unsere Datenschutzerklärung finden Sie hier: > https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On 31/10/2022 20:01, Gedare Bloom wrote: I would like to float the idea of managing 3rd party source tracking through the build system spec files. I believe this would be the most efficient way to maintain this information, and we can leverage the existing build system code for tasks such as automatic format checks, generating lists of third-party code, etc. This will require refactoring some spec files to pull 3rd party code out to a separate .yml file that gets linked. Once that is done, then we could add another attribute for this tracking purpose. I would like to keep it simple as a boolean, maybe just "third-party: true/false" Attached is an example patch showing how this might work for the dtc/libfdt code as a build objects item type 'obj', and for zlib library as a build library item type 'lib' with some proof-of-concept code for generating a listing of third party source files. In the future we could expand this third-party attribute to a dictionary to provide more detailed information about the third-party sources, for example the upstream Git repository, the import commit, path translation mappings, post and pre import actions. This could be used to automatically generate updates from upstream changes. -- embedded brains GmbH Herr Sebastian HUBER Dornierstr. 4 82178 Puchheim Germany email: sebastian.hu...@embedded-brains.de phone: +49-89-18 94 741 - 16 fax: +49-89-18 94 741 - 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On 02/11/2022 08:10, Christian MAUDERER wrote: Am 01.11.22 um 22:08 schrieb Chris Johns: On 2/11/2022 3:25 am, o...@c-mauderer.de wrote:> Is it a good idea to make it a mandatory attribute? It makes the yaml files bigger. It will only mean that we have to look for copy and paste bugs instead of missing attributes if someone adds a new third party library. Can you avoid having to add the item to all? I am no spec build system expert (having invested time) and seem to remember needing to hit a lot of files when adding something ... https://git.rtems.org/rtems/commit/?id=6f2aa8ad36e3aaffc9fa2cb8c744b04da7339ee2 Chris The documentation mentions at least some optional attributes in the specification files. For example "format" in a build option item or the "do-configure" in a build script item: https://docs.rtems.org/branches/master/eng/req/items.html#build-option-item-type https://docs.rtems.org/branches/master/eng/req/items.html#build-script-item-type But I think the wscript has to take into account that the value might not exist. I'm not sure whether it does that for the existing "optional" attributes. For example my first guess would be that the "do-configure" could throw a KeyError: https://git.rtems.org/rtems/tree/wscript#n1127 def do_configure(self, conf, cic): script = self.data["do-configure"] if script: exec(script) Usually I would have expected the following code instead: def do_configure(self, conf, cic): try: script = self.data["do-configure"] except KeyError: script = None if script: exec(script) So I can't clearly answer your question. I would have to try it. But regardless whether there are currently such options or not: They shouldn't be hard to implement. I just hope that this doesn't break some use case. I'll try to remember to ask Sebastian about it next week. When I worked on the new build system there was a discussion whether all attributes should be explicit in the items or not. It was decided to make all attributes mandatory in the items. We could of course reconsider this decision, however, from my point of view it was a good decision. It keeps the wscript code simple and the items are self-contained. You don't have to know the source code to figure out what an item describes. Changing the items is very simple with a Python script which uses the stuff in rtems-central. Simple things can be done using grep and sed. -- embedded brains GmbH Herr Sebastian HUBER Dornierstr. 4 82178 Puchheim Germany email: sebastian.hu...@embedded-brains.de phone: +49-89-18 94 741 - 16 fax: +49-89-18 94 741 - 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On 3/11/2022 2:50 pm, Chris Johns wrote: > On 3/11/2022 10:30 am, Gedare Bloom wrote: >> It was pointed out in Discord that the "third_party_list" command is >> unusual with respect to other waf commands, which are given as oneword >> formats (other than bsp_list and bsp_defaults). I think it makes sense >> to go with such a oneword format for consistent commands. So I will >> instead start prototyping this as >> ./waf thirdparties >> >> instead of >> ./waf third_party_list >> > > Great and thanks. > >> Relatedly: I might look into changing >> ./waf bsp_list >> to >> ./waf bsps >> >> and >> ./waf bsp_defaults >> to >> ./waf bspdefaults >> >> the hard part will be chasing down the documentation. > > I like the change and doing it now would mean it would change across releases. Would not be changing across releases ... that is Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On 3/11/2022 10:30 am, Gedare Bloom wrote: > It was pointed out in Discord that the "third_party_list" command is > unusual with respect to other waf commands, which are given as oneword > formats (other than bsp_list and bsp_defaults). I think it makes sense > to go with such a oneword format for consistent commands. So I will > instead start prototyping this as > ./waf thirdparties > > instead of > ./waf third_party_list > Great and thanks. > Relatedly: I might look into changing > ./waf bsp_list > to > ./waf bsps > > and > ./waf bsp_defaults > to > ./waf bspdefaults > > the hard part will be chasing down the documentation. I like the change and doing it now would mean it would change across releases. Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
It was pointed out in Discord that the "third_party_list" command is unusual with respect to other waf commands, which are given as oneword formats (other than bsp_list and bsp_defaults). I think it makes sense to go with such a oneword format for consistent commands. So I will instead start prototyping this as ./waf thirdparties instead of ./waf third_party_list Relatedly: I might look into changing ./waf bsp_list to ./waf bsps and ./waf bsp_defaults to ./waf bspdefaults the hard part will be chasing down the documentation. On Wed, Nov 2, 2022 at 9:50 AM Gedare Bloom wrote: > > On Wed, Nov 2, 2022 at 9:10 AM Gedare Bloom wrote: > > > > On Wed, Nov 2, 2022 at 1:10 AM Christian MAUDERER > > wrote: > > > > > > Hello Chris, > > > > > > Am 01.11.22 um 22:08 schrieb Chris Johns: > > > > On 2/11/2022 3:25 am, o...@c-mauderer.de wrote:> Is it a good idea to > > > > make it a > > > > mandatory attribute? It makes the yaml files > > > >> bigger. It will only mean that we have to look for copy and paste bugs > > > >> instead > > > >> of missing attributes if someone adds a new third party library. > > > > > > > > Can you avoid having to add the item to all? I am no spec build system > > > > expert > > > > (having invested time) and seem to remember needing to hit a lot of > > > > files when > > > > adding something ... > > > > > > > > https://git.rtems.org/rtems/commit/?id=6f2aa8ad36e3aaffc9fa2cb8c744b04da7339ee2 > > > > > > > > Chris > > > > > > The documentation mentions at least some optional attributes in the > > > specification files. For example "format" in a build option item or the > > > "do-configure" in a build script item: > > > > > > https://docs.rtems.org/branches/master/eng/req/items.html#build-option-item-type > > > https://docs.rtems.org/branches/master/eng/req/items.html#build-script-item-type > > > > > > But I think the wscript has to take into account that the value might > > > not exist. I'm not sure whether it does that for the existing "optional" > > > attributes. For example my first guess would be that the "do-configure" > > > could throw a KeyError: > > > > > > https://git.rtems.org/rtems/tree/wscript#n1127 > > > > > > def do_configure(self, conf, cic): > > > script = self.data["do-configure"] > > > if script: > > > exec(script) > > > > > > Usually I would have expected the following code instead: > > > > > > def do_configure(self, conf, cic): > > > try: > > > script = self.data["do-configure"] > > > except KeyError: > > > script = None > > > if script: > > > exec(script) > > > > > Thanks for this recommendation and the discussion/example. Indeed, the > > plumbing right now in the wscript is not very robust to optional > > attributes. For this particular third-party attribute I will take a > > stab at making it optional. > > > > This works pretty well. See attached. I'll continue with the optional > attribute route for now. > > > I think the additional metadata we would like to track regarding > > third-party sources can be added incrementally or even separately. I > > would like to first solve the problem of how to separate third-party > > from project-owned sources. Once that is done, then we can discuss > > more ways to improve or use this capability. > > > > > So I can't clearly answer your question. I would have to try it. But > > > regardless whether there are currently such options or not: They > > > shouldn't be hard to implement. I just hope that this doesn't break some > > > use case. I'll try to remember to ask Sebastian about it next week. > > > > > Thanks, I will appreciate that feedback and any tricky things to watch out > > for. > > > > > Best regards > > > > > > Christian > > > -- > > > > > > embedded brains GmbH > > > Herr Christian MAUDERER > > > Dornierstr. 4 > > > 82178 Puchheim > > > Germany > > > email: christian.maude...@embedded-brains.de > > > phone: +49-89-18 94 741 - 18 > > > mobile: +49-176-152 206 08 > > > > > > Registergericht: Amtsgericht München > > > Registernummer: HRB 157899 > > > Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler > > > Unsere Datenschutzerklärung finden Sie hier: > > > https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On Wed, Nov 2, 2022 at 9:10 AM Gedare Bloom wrote: > > On Wed, Nov 2, 2022 at 1:10 AM Christian MAUDERER > wrote: > > > > Hello Chris, > > > > Am 01.11.22 um 22:08 schrieb Chris Johns: > > > On 2/11/2022 3:25 am, o...@c-mauderer.de wrote:> Is it a good idea to > > > make it a > > > mandatory attribute? It makes the yaml files > > >> bigger. It will only mean that we have to look for copy and paste bugs > > >> instead > > >> of missing attributes if someone adds a new third party library. > > > > > > Can you avoid having to add the item to all? I am no spec build system > > > expert > > > (having invested time) and seem to remember needing to hit a lot of files > > > when > > > adding something ... > > > > > > https://git.rtems.org/rtems/commit/?id=6f2aa8ad36e3aaffc9fa2cb8c744b04da7339ee2 > > > > > > Chris > > > > The documentation mentions at least some optional attributes in the > > specification files. For example "format" in a build option item or the > > "do-configure" in a build script item: > > > > https://docs.rtems.org/branches/master/eng/req/items.html#build-option-item-type > > https://docs.rtems.org/branches/master/eng/req/items.html#build-script-item-type > > > > But I think the wscript has to take into account that the value might > > not exist. I'm not sure whether it does that for the existing "optional" > > attributes. For example my first guess would be that the "do-configure" > > could throw a KeyError: > > > > https://git.rtems.org/rtems/tree/wscript#n1127 > > > > def do_configure(self, conf, cic): > > script = self.data["do-configure"] > > if script: > > exec(script) > > > > Usually I would have expected the following code instead: > > > > def do_configure(self, conf, cic): > > try: > > script = self.data["do-configure"] > > except KeyError: > > script = None > > if script: > > exec(script) > > > Thanks for this recommendation and the discussion/example. Indeed, the > plumbing right now in the wscript is not very robust to optional > attributes. For this particular third-party attribute I will take a > stab at making it optional. > This works pretty well. See attached. I'll continue with the optional attribute route for now. > I think the additional metadata we would like to track regarding > third-party sources can be added incrementally or even separately. I > would like to first solve the problem of how to separate third-party > from project-owned sources. Once that is done, then we can discuss > more ways to improve or use this capability. > > > So I can't clearly answer your question. I would have to try it. But > > regardless whether there are currently such options or not: They > > shouldn't be hard to implement. I just hope that this doesn't break some > > use case. I'll try to remember to ask Sebastian about it next week. > > > Thanks, I will appreciate that feedback and any tricky things to watch out > for. > > > Best regards > > > > Christian > > -- > > > > embedded brains GmbH > > Herr Christian MAUDERER > > Dornierstr. 4 > > 82178 Puchheim > > Germany > > email: christian.maude...@embedded-brains.de > > phone: +49-89-18 94 741 - 18 > > mobile: +49-176-152 206 08 > > > > Registergericht: Amtsgericht München > > Registernummer: HRB 157899 > > Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler > > Unsere Datenschutzerklärung finden Sie hier: > > https://embedded-brains.de/datenschutzerklaerung/ From b2e059a34e7e307c15f7542467400dae10f49bdd Mon Sep 17 00:00:00 2001 From: Gedare Bloom Date: Mon, 31 Oct 2022 12:52:09 -0600 Subject: [PATCH 2/3] spec/build: mark objdtc and libz as third-party --- spec/build/cpukit/libz.yml | 1 + spec/build/cpukit/objdtc.yml | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/spec/build/cpukit/libz.yml b/spec/build/cpukit/libz.yml index 5e81384e28..90f804f44b 100644 --- a/spec/build/cpukit/libz.yml +++ b/spec/build/cpukit/libz.yml @@ -5,6 +5,7 @@ copyrights: - Copyright (C) 2020 embedded brains GmbH (http://www.embedded-brains.de) cppflags: [] cxxflags: [] +third-party: true enabled-by: true includes: [] install: diff --git a/spec/build/cpukit/objdtc.yml b/spec/build/cpukit/objdtc.yml index 2492cbca74..43f16f2b02 100644 --- a/spec/build/cpukit/objdtc.yml +++ b/spec/build/cpukit/objdtc.yml @@ -5,7 +5,7 @@ copyrights: - Copyright (C) 2022 Gedare Bloom cppflags: [] cxxflags: [] -third-party: false +third-party: true enabled-by: true includes: [] install: -- 2.34.1 From 89da3dd8d2ab55f3dba4cff214de282c0072d73b Mon Sep 17 00:00:00 2001 From: Gedare Bloom Date: Mon, 31 Oct 2022 12:52:35 -0600 Subject: [PATCH 3/3] wscript: add command to print third-party sources --- wscript | 32 1 file changed, 32 insertions(+) diff --git a/wscript b/wscript index 4071cc9ef8..3bd88598a4 100755 --- a/wscript +++
Re: Fwd: Identify 3rd party source in spec?
On Wed, Nov 2, 2022 at 9:14 AM Joel Sherrill wrote: > > > > On Wed, Nov 2, 2022 at 10:10 AM Gedare Bloom wrote: >> >> On Wed, Nov 2, 2022 at 1:10 AM Christian MAUDERER >> wrote: >> > >> > Hello Chris, >> > >> > Am 01.11.22 um 22:08 schrieb Chris Johns: >> > > On 2/11/2022 3:25 am, o...@c-mauderer.de wrote:> Is it a good idea to >> > > make it a >> > > mandatory attribute? It makes the yaml files >> > >> bigger. It will only mean that we have to look for copy and paste bugs >> > >> instead >> > >> of missing attributes if someone adds a new third party library. >> > > >> > > Can you avoid having to add the item to all? I am no spec build system >> > > expert >> > > (having invested time) and seem to remember needing to hit a lot of >> > > files when >> > > adding something ... >> > > >> > > https://git.rtems.org/rtems/commit/?id=6f2aa8ad36e3aaffc9fa2cb8c744b04da7339ee2 >> > > >> > > Chris >> > >> > The documentation mentions at least some optional attributes in the >> > specification files. For example "format" in a build option item or the >> > "do-configure" in a build script item: >> > >> > https://docs.rtems.org/branches/master/eng/req/items.html#build-option-item-type >> > https://docs.rtems.org/branches/master/eng/req/items.html#build-script-item-type >> > >> > But I think the wscript has to take into account that the value might >> > not exist. I'm not sure whether it does that for the existing "optional" >> > attributes. For example my first guess would be that the "do-configure" >> > could throw a KeyError: >> > >> > https://git.rtems.org/rtems/tree/wscript#n1127 >> > >> > def do_configure(self, conf, cic): >> > script = self.data["do-configure"] >> > if script: >> > exec(script) >> > >> > Usually I would have expected the following code instead: >> > >> > def do_configure(self, conf, cic): >> > try: >> > script = self.data["do-configure"] >> > except KeyError: >> > script = None >> > if script: >> > exec(script) >> > >> Thanks for this recommendation and the discussion/example. Indeed, the >> plumbing right now in the wscript is not very robust to optional >> attributes. For this particular third-party attribute I will take a >> stab at making it optional. >> >> I think the additional metadata we would like to track regarding >> third-party sources can be added incrementally or even separately. I >> would like to first solve the problem of how to separate third-party >> from project-owned sources. Once that is done, then we can discuss >> more ways to improve or use this capability. > > > Are you thinking of a more distinct placement of third party sources > in the source tree? like a cpukit/thirdparty/... sub-tree? No. I would like to support 3rd party sources wherever they are best found, and to keep track of them separately through the build system since it already understands the various files/sources involved. I think this makes the most sense for maintenance in the longer-term. >> >> >> > So I can't clearly answer your question. I would have to try it. But >> > regardless whether there are currently such options or not: They >> > shouldn't be hard to implement. I just hope that this doesn't break some >> > use case. I'll try to remember to ask Sebastian about it next week. >> > >> Thanks, I will appreciate that feedback and any tricky things to watch out >> for. >> >> > Best regards >> > >> > Christian >> > -- >> > >> > embedded brains GmbH >> > Herr Christian MAUDERER >> > Dornierstr. 4 >> > 82178 Puchheim >> > Germany >> > email: christian.maude...@embedded-brains.de >> > phone: +49-89-18 94 741 - 18 >> > mobile: +49-176-152 206 08 >> > >> > Registergericht: Amtsgericht München >> > Registernummer: HRB 157899 >> > Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler >> > Unsere Datenschutzerklärung finden Sie hier: >> > https://embedded-brains.de/datenschutzerklaerung/ >> ___ >> devel mailing list >> devel@rtems.org >> http://lists.rtems.org/mailman/listinfo/devel ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On Wed, Nov 2, 2022 at 10:10 AM Gedare Bloom wrote: > On Wed, Nov 2, 2022 at 1:10 AM Christian MAUDERER > wrote: > > > > Hello Chris, > > > > Am 01.11.22 um 22:08 schrieb Chris Johns: > > > On 2/11/2022 3:25 am, o...@c-mauderer.de wrote:> Is it a good idea to > make it a > > > mandatory attribute? It makes the yaml files > > >> bigger. It will only mean that we have to look for copy and paste > bugs instead > > >> of missing attributes if someone adds a new third party library. > > > > > > Can you avoid having to add the item to all? I am no spec build system > expert > > > (having invested time) and seem to remember needing to hit a lot of > files when > > > adding something ... > > > > > > > https://git.rtems.org/rtems/commit/?id=6f2aa8ad36e3aaffc9fa2cb8c744b04da7339ee2 > > > > > > Chris > > > > The documentation mentions at least some optional attributes in the > > specification files. For example "format" in a build option item or the > > "do-configure" in a build script item: > > > > > https://docs.rtems.org/branches/master/eng/req/items.html#build-option-item-type > > > https://docs.rtems.org/branches/master/eng/req/items.html#build-script-item-type > > > > But I think the wscript has to take into account that the value might > > not exist. I'm not sure whether it does that for the existing "optional" > > attributes. For example my first guess would be that the "do-configure" > > could throw a KeyError: > > > > https://git.rtems.org/rtems/tree/wscript#n1127 > > > > def do_configure(self, conf, cic): > > script = self.data["do-configure"] > > if script: > > exec(script) > > > > Usually I would have expected the following code instead: > > > > def do_configure(self, conf, cic): > > try: > > script = self.data["do-configure"] > > except KeyError: > > script = None > > if script: > > exec(script) > > > Thanks for this recommendation and the discussion/example. Indeed, the > plumbing right now in the wscript is not very robust to optional > attributes. For this particular third-party attribute I will take a > stab at making it optional. > > I think the additional metadata we would like to track regarding > third-party sources can be added incrementally or even separately. I > would like to first solve the problem of how to separate third-party > from project-owned sources. Once that is done, then we can discuss > more ways to improve or use this capability. > Are you thinking of a more distinct placement of third party sources in the source tree? like a cpukit/thirdparty/... sub-tree? > > > So I can't clearly answer your question. I would have to try it. But > > regardless whether there are currently such options or not: They > > shouldn't be hard to implement. I just hope that this doesn't break some > > use case. I'll try to remember to ask Sebastian about it next week. > > > Thanks, I will appreciate that feedback and any tricky things to watch out > for. > > > Best regards > > > > Christian > > -- > > > > embedded brains GmbH > > Herr Christian MAUDERER > > Dornierstr. 4 > > 82178 Puchheim > > Germany > > email: christian.maude...@embedded-brains.de > > phone: +49-89-18 94 741 - 18 > > mobile: +49-176-152 206 08 > > > > Registergericht: Amtsgericht München > > Registernummer: HRB 157899 > > Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler > > Unsere Datenschutzerklärung finden Sie hier: > > https://embedded-brains.de/datenschutzerklaerung/ > ___ > devel mailing list > devel@rtems.org > http://lists.rtems.org/mailman/listinfo/devel ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On Wed, Nov 2, 2022 at 1:10 AM Christian MAUDERER wrote: > > Hello Chris, > > Am 01.11.22 um 22:08 schrieb Chris Johns: > > On 2/11/2022 3:25 am, o...@c-mauderer.de wrote:> Is it a good idea to make > > it a > > mandatory attribute? It makes the yaml files > >> bigger. It will only mean that we have to look for copy and paste bugs > >> instead > >> of missing attributes if someone adds a new third party library. > > > > Can you avoid having to add the item to all? I am no spec build system > > expert > > (having invested time) and seem to remember needing to hit a lot of files > > when > > adding something ... > > > > https://git.rtems.org/rtems/commit/?id=6f2aa8ad36e3aaffc9fa2cb8c744b04da7339ee2 > > > > Chris > > The documentation mentions at least some optional attributes in the > specification files. For example "format" in a build option item or the > "do-configure" in a build script item: > > https://docs.rtems.org/branches/master/eng/req/items.html#build-option-item-type > https://docs.rtems.org/branches/master/eng/req/items.html#build-script-item-type > > But I think the wscript has to take into account that the value might > not exist. I'm not sure whether it does that for the existing "optional" > attributes. For example my first guess would be that the "do-configure" > could throw a KeyError: > > https://git.rtems.org/rtems/tree/wscript#n1127 > > def do_configure(self, conf, cic): > script = self.data["do-configure"] > if script: > exec(script) > > Usually I would have expected the following code instead: > > def do_configure(self, conf, cic): > try: > script = self.data["do-configure"] > except KeyError: > script = None > if script: > exec(script) > Thanks for this recommendation and the discussion/example. Indeed, the plumbing right now in the wscript is not very robust to optional attributes. For this particular third-party attribute I will take a stab at making it optional. I think the additional metadata we would like to track regarding third-party sources can be added incrementally or even separately. I would like to first solve the problem of how to separate third-party from project-owned sources. Once that is done, then we can discuss more ways to improve or use this capability. > So I can't clearly answer your question. I would have to try it. But > regardless whether there are currently such options or not: They > shouldn't be hard to implement. I just hope that this doesn't break some > use case. I'll try to remember to ask Sebastian about it next week. > Thanks, I will appreciate that feedback and any tricky things to watch out for. > Best regards > > Christian > -- > > embedded brains GmbH > Herr Christian MAUDERER > Dornierstr. 4 > 82178 Puchheim > Germany > email: christian.maude...@embedded-brains.de > phone: +49-89-18 94 741 - 18 > mobile: +49-176-152 206 08 > > Registergericht: Amtsgericht München > Registernummer: HRB 157899 > Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler > Unsere Datenschutzerklärung finden Sie hier: > https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
Hello Chris, Am 01.11.22 um 22:08 schrieb Chris Johns: On 2/11/2022 3:25 am, o...@c-mauderer.de wrote:> Is it a good idea to make it a mandatory attribute? It makes the yaml files bigger. It will only mean that we have to look for copy and paste bugs instead of missing attributes if someone adds a new third party library. Can you avoid having to add the item to all? I am no spec build system expert (having invested time) and seem to remember needing to hit a lot of files when adding something ... https://git.rtems.org/rtems/commit/?id=6f2aa8ad36e3aaffc9fa2cb8c744b04da7339ee2 Chris The documentation mentions at least some optional attributes in the specification files. For example "format" in a build option item or the "do-configure" in a build script item: https://docs.rtems.org/branches/master/eng/req/items.html#build-option-item-type https://docs.rtems.org/branches/master/eng/req/items.html#build-script-item-type But I think the wscript has to take into account that the value might not exist. I'm not sure whether it does that for the existing "optional" attributes. For example my first guess would be that the "do-configure" could throw a KeyError: https://git.rtems.org/rtems/tree/wscript#n1127 def do_configure(self, conf, cic): script = self.data["do-configure"] if script: exec(script) Usually I would have expected the following code instead: def do_configure(self, conf, cic): try: script = self.data["do-configure"] except KeyError: script = None if script: exec(script) So I can't clearly answer your question. I would have to try it. But regardless whether there are currently such options or not: They shouldn't be hard to implement. I just hope that this doesn't break some use case. I'll try to remember to ask Sebastian about it next week. Best regards Christian -- embedded brains GmbH Herr Christian MAUDERER Dornierstr. 4 82178 Puchheim Germany email: christian.maude...@embedded-brains.de phone: +49-89-18 94 741 - 18 mobile: +49-176-152 206 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On 2/11/2022 3:25 am, o...@c-mauderer.de wrote:> Is it a good idea to make it a mandatory attribute? It makes the yaml files > bigger. It will only mean that we have to look for copy and paste bugs instead > of missing attributes if someone adds a new third party library. Can you avoid having to add the item to all? I am no spec build system expert (having invested time) and seem to remember needing to hit a lot of files when adding something ... https://git.rtems.org/rtems/commit/?id=6f2aa8ad36e3aaffc9fa2cb8c744b04da7339ee2 Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
On Tue, Nov 1, 2022 at 11:25 AM wrote: > Hello Gedare, > > Am 31.10.22 um 20:01 schrieb Gedare Bloom: > > Resending without the first patch since it may trigger size filters. > > > > -- Forwarded message - > > From: Gedare Bloom > > Date: Mon, Oct 31, 2022 at 12:55 PM > > Subject: Identify 3rd party source in spec? > > To: devel@rtems.org > > > > > > Hello all, > > > > I would like to float the idea of managing 3rd party source tracking > > through the build system spec files. I believe this would be the most > > efficient way to maintain this information, and we can leverage the > > existing build system code for tasks such as automatic format checks, > > generating lists of third-party code, etc. > > Sounds like a good idea to have it in the build specification items. > > > > > This will require refactoring some spec files to pull 3rd party code > > out to a separate .yml file that gets linked. Once that is done, then > > we could add another attribute for this tracking purpose. I would like > > to keep it simple as a boolean, maybe just "third-party: true/false" > > Attached is an example patch showing how this might work for the > > dtc/libfdt code as a build objects item type 'obj', and for zlib > > library as a build library item type 'lib' with some proof-of-concept > > code for generating a listing of third party source files. > > Is it a good idea to make it a mandatory attribute? It makes the yaml > files bigger. It will only mean that we have to look for copy and paste > bugs instead of missing attributes if someone adds a new third party > library. > I agree that an implicit "third-party: false" is preferable. > > Do we need some additional information / scripts / stuff for third party > code? Like an upstream source or upstream version? For example the JFFS2 > code has a VERSION file with lots of information: > > https://git.rtems.org/rtems/tree/cpukit/libfs/src/jffs2/VERSION I don't think this should be required for the initial implementation with existing 3rd party source, but it would be good to define how we do this for any new 3rd party source that is brought in. Adding this for existing 3rd party inclusions could easily be a janitor task. For rtems-lwip, I've been enforcing addition of an ORIGIN.* file for new source imports with an upstream commit hash in the commit message, but that was added basically from the beginning and almost all directories in the root of that project are 3rd party source imports that have pretty clear origin, upstream, and licensing. > > > For the imxrt I added something similar in the commit description: > > > https://git.rtems.org/rtems/commit/bsps/arm/imxrt/nxp?id=48f6a6c302a3e1a3f8915e2503d0fe618d1af285 > > Full text is a bit big for an attribute but maybe the location of a > VERSION file in the source tree? Or a minimal link to an upstream code > and a version info? > > Best regards > > Christian > > > > > As an initial step before making this refactoring, I have added an > > explicit default "third-party: false" attribute to every yml file > > preceding the enabled-by: attribute, using the following bit of shell: > > rtems.git/spec/build$ find . -name "*.yml" | xargs sed -i -e > > 's/\(enabled-by.*\)/third-party: false\n\1/' > > > > This touches 2333 files adding that one line, which is the contents of > > the first 1 MiB patch attached in the series. The remaining patches > > then layer on the top and are functional, outputting: > > > > rtems.git$ ./waf third_party_list > > cpukit/dtc/libfdt/fdt.c > > cpukit/dtc/libfdt/fdt_addresses.c > > cpukit/dtc/libfdt/fdt_empty_tree.c > > cpukit/dtc/libfdt/fdt_ro.c > > cpukit/dtc/libfdt/fdt_rw.c > > cpukit/dtc/libfdt/fdt_strerror.c > > cpukit/dtc/libfdt/fdt_sw.c > > cpukit/dtc/libfdt/fdt_wip.c > > cpukit/zlib/adler32.c > > cpukit/zlib/compress.c > > cpukit/zlib/crc32.c > > cpukit/zlib/deflate.c > > cpukit/zlib/gzclose.c > > cpukit/zlib/gzlib.c > > cpukit/zlib/gzread.c > > cpukit/zlib/gzwrite.c > > cpukit/zlib/infback.c > > cpukit/zlib/inffast.c > > cpukit/zlib/inflate.c > > cpukit/zlib/inftrees.c > > cpukit/zlib/trees.c > > cpukit/zlib/uncompr.c > > cpukit/zlib/zutil.c > > > > I'll continue to work on this, feedback is requested though if this is > > a good direction or how to improve. > > > > Gedare > > > > > > ___ > > devel mailing list > > devel@rtems.org > > http://lists.rtems.org/mailman/listinfo/devel > ___ > devel mailing list > devel@rtems.org > http://lists.rtems.org/mailman/listinfo/devel > ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Fwd: Identify 3rd party source in spec?
Hello Gedare, Am 31.10.22 um 20:01 schrieb Gedare Bloom: Resending without the first patch since it may trigger size filters. -- Forwarded message - From: Gedare Bloom Date: Mon, Oct 31, 2022 at 12:55 PM Subject: Identify 3rd party source in spec? To: devel@rtems.org Hello all, I would like to float the idea of managing 3rd party source tracking through the build system spec files. I believe this would be the most efficient way to maintain this information, and we can leverage the existing build system code for tasks such as automatic format checks, generating lists of third-party code, etc. Sounds like a good idea to have it in the build specification items. This will require refactoring some spec files to pull 3rd party code out to a separate .yml file that gets linked. Once that is done, then we could add another attribute for this tracking purpose. I would like to keep it simple as a boolean, maybe just "third-party: true/false" Attached is an example patch showing how this might work for the dtc/libfdt code as a build objects item type 'obj', and for zlib library as a build library item type 'lib' with some proof-of-concept code for generating a listing of third party source files. Is it a good idea to make it a mandatory attribute? It makes the yaml files bigger. It will only mean that we have to look for copy and paste bugs instead of missing attributes if someone adds a new third party library. Do we need some additional information / scripts / stuff for third party code? Like an upstream source or upstream version? For example the JFFS2 code has a VERSION file with lots of information: https://git.rtems.org/rtems/tree/cpukit/libfs/src/jffs2/VERSION For the imxrt I added something similar in the commit description: https://git.rtems.org/rtems/commit/bsps/arm/imxrt/nxp?id=48f6a6c302a3e1a3f8915e2503d0fe618d1af285 Full text is a bit big for an attribute but maybe the location of a VERSION file in the source tree? Or a minimal link to an upstream code and a version info? Best regards Christian As an initial step before making this refactoring, I have added an explicit default "third-party: false" attribute to every yml file preceding the enabled-by: attribute, using the following bit of shell: rtems.git/spec/build$ find . -name "*.yml" | xargs sed -i -e 's/\(enabled-by.*\)/third-party: false\n\1/' This touches 2333 files adding that one line, which is the contents of the first 1 MiB patch attached in the series. The remaining patches then layer on the top and are functional, outputting: rtems.git$ ./waf third_party_list cpukit/dtc/libfdt/fdt.c cpukit/dtc/libfdt/fdt_addresses.c cpukit/dtc/libfdt/fdt_empty_tree.c cpukit/dtc/libfdt/fdt_ro.c cpukit/dtc/libfdt/fdt_rw.c cpukit/dtc/libfdt/fdt_strerror.c cpukit/dtc/libfdt/fdt_sw.c cpukit/dtc/libfdt/fdt_wip.c cpukit/zlib/adler32.c cpukit/zlib/compress.c cpukit/zlib/crc32.c cpukit/zlib/deflate.c cpukit/zlib/gzclose.c cpukit/zlib/gzlib.c cpukit/zlib/gzread.c cpukit/zlib/gzwrite.c cpukit/zlib/infback.c cpukit/zlib/inffast.c cpukit/zlib/inflate.c cpukit/zlib/inftrees.c cpukit/zlib/trees.c cpukit/zlib/uncompr.c cpukit/zlib/zutil.c I'll continue to work on this, feedback is requested though if this is a good direction or how to improve. Gedare ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Fwd: Identify 3rd party source in spec?
Resending without the first patch since it may trigger size filters. -- Forwarded message - From: Gedare Bloom Date: Mon, Oct 31, 2022 at 12:55 PM Subject: Identify 3rd party source in spec? To: devel@rtems.org Hello all, I would like to float the idea of managing 3rd party source tracking through the build system spec files. I believe this would be the most efficient way to maintain this information, and we can leverage the existing build system code for tasks such as automatic format checks, generating lists of third-party code, etc. This will require refactoring some spec files to pull 3rd party code out to a separate .yml file that gets linked. Once that is done, then we could add another attribute for this tracking purpose. I would like to keep it simple as a boolean, maybe just "third-party: true/false" Attached is an example patch showing how this might work for the dtc/libfdt code as a build objects item type 'obj', and for zlib library as a build library item type 'lib' with some proof-of-concept code for generating a listing of third party source files. As an initial step before making this refactoring, I have added an explicit default "third-party: false" attribute to every yml file preceding the enabled-by: attribute, using the following bit of shell: rtems.git/spec/build$ find . -name "*.yml" | xargs sed -i -e 's/\(enabled-by.*\)/third-party: false\n\1/' This touches 2333 files adding that one line, which is the contents of the first 1 MiB patch attached in the series. The remaining patches then layer on the top and are functional, outputting: rtems.git$ ./waf third_party_list cpukit/dtc/libfdt/fdt.c cpukit/dtc/libfdt/fdt_addresses.c cpukit/dtc/libfdt/fdt_empty_tree.c cpukit/dtc/libfdt/fdt_ro.c cpukit/dtc/libfdt/fdt_rw.c cpukit/dtc/libfdt/fdt_strerror.c cpukit/dtc/libfdt/fdt_sw.c cpukit/dtc/libfdt/fdt_wip.c cpukit/zlib/adler32.c cpukit/zlib/compress.c cpukit/zlib/crc32.c cpukit/zlib/deflate.c cpukit/zlib/gzclose.c cpukit/zlib/gzlib.c cpukit/zlib/gzread.c cpukit/zlib/gzwrite.c cpukit/zlib/infback.c cpukit/zlib/inffast.c cpukit/zlib/inflate.c cpukit/zlib/inftrees.c cpukit/zlib/trees.c cpukit/zlib/uncompr.c cpukit/zlib/zutil.c I'll continue to work on this, feedback is requested though if this is a good direction or how to improve. Gedare From 44ae3a1ae62875047c57fbd25163fa78e21c4ed5 Mon Sep 17 00:00:00 2001 From: Gedare Bloom Date: Mon, 31 Oct 2022 12:52:35 -0600 Subject: [PATCH 4/4] wscript: add command to print third-party sources --- wscript | 28 1 file changed, 28 insertions(+) diff --git a/wscript b/wscript index 4071cc9ef8..051f35ca52 100755 --- a/wscript +++ b/wscript @@ -226,6 +226,17 @@ class Item(object): p.build(bld, bic) self.do_build(bld, bic) +def third_parties(self, ctx): +tpl = [] +for p in self.links(): +x = p.third_parties(ctx) +if x is not None: +tpl.extend(x) +x = self.do_third_parties(ctx) +if x is not None: +tpl.extend(x) +return tpl + def do_defaults(self, variant, family): return @@ -241,6 +252,11 @@ class Item(object): def do_build(self, bld, bic): return +def do_third_parties(self, ctx): +if self.data["third-party"]: +source = self.data["source"] +return source + def substitute(self, ctx, value): if isinstance(value, str): try: @@ -1711,3 +1727,15 @@ def bsp_list(ctx): print(variant) if first: no_matches_error(ctx, white_list) + +def third_party_list(ctx): +"""lists third-party sources""" +check_forbidden_options( +ctx, ["compiler", "config", "options", "tools", "top_group", "version"] +) +add_log_filter(ctx.cmd) +load_items_from_options(ctx) +top_group = get_top_group(ctx) +tp = items[top_group].third_parties(ctx) +for s in tp: +print(s) -- 2.34.1 From a0cf7dc9dc56c83c3949a67f9c5aa439aef94994 Mon Sep 17 00:00:00 2001 From: Gedare Bloom Date: Mon, 31 Oct 2022 12:52:09 -0600 Subject: [PATCH 3/4] spec/build: mark objdtc and libz as third-party --- spec/build/cpukit/libz.yml | 2 +- spec/build/cpukit/objdtc.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/build/cpukit/libz.yml b/spec/build/cpukit/libz.yml index d1ade6c18c..90f804f44b 100644 --- a/spec/build/cpukit/libz.yml +++ b/spec/build/cpukit/libz.yml @@ -5,7 +5,7 @@ copyrights: - Copyright (C) 2020 embedded brains GmbH (http://www.embedded-brains.de) cppflags: [] cxxflags: [] -third-party: false +third-party: true enabled-by: true includes: [] install: diff --git a/spec/build/cpukit/objdtc.yml b/spec/build/cpukit/objdtc.yml index 2492cbca74..43f16f2b02 100644 --- a/spec/build/cpukit/objdtc.yml +++ b/spec/build/cpukit/objdtc.yml @@ -5,7 +5,7 @@ copyrights: - Copyright (C) 2022 Gedare Bloom cppflags: [] cxxflags: []
