Re: Change build specification files from YAML to JSON?

[Sebastian Huber]

On 27.04.23 00:47, Joel Sherrill wrote:


On Wed, Apr 26, 2023 at 3:58 PM Gedare Bloom > wrote:


Is the result of this discussion to use CSafeLoader for now? Or do we
still debate the transition to JSON?

I see some quirks need to be dealt with as you mentioned with the
multi-line stuff. The yaml is pretty simple to understand and work
with. I've dealt with hand-editing JSON and it's a pain with the
quotes and brackets/braces.

Since the transition is (apparently) an implementation detail, I don't
actually see that it matters do it now or do it after releasing 6. If
it doesn't change the user experience other than build time, then it
should matter when we do it. In that case, better to be conservative
to avoid rushing to push something this major so close to a release.


I'm going to pile on and cynically say at this point in time, we should be
focusing on the 6 release and not imagining new sources of churn. I am
avoiding any opinion that would lead to change at this point in time.


I already abandoned the file format change.



I will say that with the current waf, I think I understand in principle 
at least

some of the high level goals for the user an I am not positive me meet these
as well as possible even if the implementation approach could be improved.
And nothing being discussed would improve what I see.

+ look at the bsp defaults to see what configuration options are available
    for a specific BSP family.

+ write a config.ini that they put under THEIR configuration control

+ possibly "create" a local BSP family variant with that INI file.
    Assuming the configuration points exist in the BSP family.

+ If their variant reflects a generally available board, then add it to
    the BSP family as a variant.

I tried to explain the first three this week in the class and it isn't easy.
A few observations that have nothing to do with YAML/JSON or even
carving the settings into stone tablets.


It depends on the BSP if this is possible. The more BSP options you 
have, the easier is it to make a custom variant through the INI file.




+ There was discussion that documentation of the BSP options could
be generated from waf/specs. The comments I have seen are not enough
to do anything useful. The BSP specific ones are usually short and cryptic.
This is especially true when the setting is a specific hex value for an CPU
or SoC specific register. One could guess RAM size, base address, or UART
N present type settings.


Yes, but this is not an issue of the build system per se. The BSP 
developers are just too lazy to properly document the options.




+ The philosophical goals behind the bsp defaults, a user writing an INI 
file,

the flexibility, etc.are not being communicated. Why do this? What's good
about it? Why do users care? >
+ If there is discussion of using the configuration INI file to do a custom
BSP variant, I didn't find it while looking for the class.

The last two could be me not finding something.



We have this:

https://docs.rtems.org/branches/master/user/bld/index.html#configuration

Customizing BSPs is not covered as far as I remember.



But ultimately we need to cut the 6 release branch. I think we have been
talking about it over a year now. Let's focus and kill it.


--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Chris Johns]

On 26/4/2023 6:22 pm, Sebastian Huber wrote:
> On 26.04.23 01:52, Chris Johns wrote:
>> On 25/4/2023 5:35 pm, Sebastian Huber wrote:
>>> Tooling makes sense if you have a high change rate.
>>
>> That is not the use case I am discussing and it is not a factor.
>>
>>> This is not the case for the RTEMS build specification items.
>>
>> I am not saying it is.
>>
>>> For which use case do we need tooling support?
>>
>> We need tooling to lets us all understand this build system. YAML is easy to
>> learn however the data-structures, rules and the large number of file 
>> fragments
>> we have are complex and not apparent to anyone looking at it even if you have
>> invested time doing so. It reflects the fact that RTEMS is complicated to 
>> build.
> 
> I don't think building an RTEMS BSP is complicated once you have the tools
> installed.

Sorry, my "user" is someone developing something for RTEMS and needing to update
the spec files. I was not clear about this.

>> I am advocate for what we have and will continue to be one so I am 
>> attempting to
>> address some things we could do better. History has shown the RTEMS core
>> developers are not great judges of the community's view of the project's
>> usability and accessibility.
>>
>> The analogy is to consider the git command then the roles github and gitlab
>> provide with their user interfaces and tooling.
>>
>>> For a new option or BSP, you just copy and past from existing files/BSPs.
>>> Changing a specific part of an existing file is just copy and paste some 
>>> lines
>>> and edit them in most cases.
>>
>> My experience tells me it is not as simple as this. There is an element of 
>> guess
>> work a change is right. The recent dependency cases highlighted this and the
>> need for checking of some form to be present in the rtems.git repo.
>>
>> We need to step back and consider the role of the build system before we 
>> discuss
>> implementation details.
>>
>> The first requirement by a large margin is ease of use for users and the
>> community to make contributions. Meeting this requirement can be done a 
>> number
>> of ways. For example a user focused format for the relationships rather than 
>> one
>> that aids machine integration. The original ground work Amar did for the 
>> move to
>> waf was to define the build in Python as documented by waf. It was simple and
>> transparent. Another example is a machine focused format however we need 
>> tooling
>> to help the users manage making changes and accessing what is happening 
>> without
>> needing to learn the details of how it is implemented.
>>
>> For tooling my order of importance is:
>>
>> 1. Visualise the structures and dependencies in a human readable manner. An
>> indication of which file is doing what would be helpful.
>>
>> 2. A check of changes users make that raises errors, dependency problems, 
>> etc.
>> This can be a command you run if you are making changes and does not need to 
>> run
>> every build.
>>
>> I see JSON and tooling as linked together. I also not expect complete 
>> tooling to
>> be present for the change to happen. All I am wanting is the need for 
>> tooling be
>> agreed on and it is located in rtems.git. The development can then happen and
>> evolve as the community sees a need.
> 
> From my experience with waf I would not say that the waf build system is 
> simple
> and transparent. Things which are very easy with make are complicated with waf
> at least for me. 

Yes this is valid and I agree. If you are outside the scope of something simple
it can get hard. It does some things well and easily and others are hard. Waf is
not alone here.

> I asked a couple of waf questions on the RTEMS mailing list
> which no core developer could answer. The support from the waf developer was a
> great help, but without this help I would not have been able to write the waf
> based build system for RTEMS.

Developing a framework does require a different level of integration with waf
and it can be involved. Waf is good as the base of a framework and I think what
we have is a good example.

I agree with Joel, we have not explained or sold the change as well as we need
to have.

What ever the format I would like the ability to create user focused tools but
currently the code to load the YAML (pickled for not) is not accessible as a
module in rtems.git.

> I think our biggest obstacle to improve things in the area you outlined above 
> is
> the scattered project infrastructure with several Git repositories and the 
> lack
> of CI pipelines. I will probably start a discussion about this topic because I
> think our current project setting has severe maintenance issues.

I am not so sure, there are problems with both approaches. We recently stripped
the rtems.git repo down to be what it is now and so moving back to a single
integrated repo would be returning to the past.

If you consider a single repo to be the right approach then I suggest you
present that first and explain how we release, 

Re: Change build specification files from YAML to JSON?

[Joel Sherrill]

On Wed, Apr 26, 2023 at 3:58 PM Gedare Bloom  wrote:

> Is the result of this discussion to use CSafeLoader for now? Or do we
> still debate the transition to JSON?
>
> I see some quirks need to be dealt with as you mentioned with the
> multi-line stuff. The yaml is pretty simple to understand and work
> with. I've dealt with hand-editing JSON and it's a pain with the
> quotes and brackets/braces.
>
> Since the transition is (apparently) an implementation detail, I don't
> actually see that it matters do it now or do it after releasing 6. If
> it doesn't change the user experience other than build time, then it
> should matter when we do it. In that case, better to be conservative
> to avoid rushing to push something this major so close to a release.
>

I'm going to pile on and cynically say at this point in time, we should be
focusing on the 6 release and not imagining new sources of churn. I am
avoiding any opinion that would lead to change at this point in time.

I will say that with the current waf, I think I understand in principle at
least
some of the high level goals for the user an I am not positive me meet these
as well as possible even if the implementation approach could be improved.
And nothing being discussed would improve what I see.

+ look at the bsp defaults to see what configuration options are available
   for a specific BSP family.

+ write a config.ini that they put under THEIR configuration control

+ possibly "create" a local BSP family variant with that INI file.
   Assuming the configuration points exist in the BSP family.

+ If their variant reflects a generally available board, then add it to
   the BSP family as a variant.

I tried to explain the first three this week in the class and it isn't easy.
A few observations that have nothing to do with YAML/JSON or even
carving the settings into stone tablets.

+ There was discussion that documentation of the BSP options could
be generated from waf/specs. The comments I have seen are not enough
to do anything useful. The BSP specific ones are usually short and cryptic.
This is especially true when the setting is a specific hex value for an CPU
or SoC specific register. One could guess RAM size, base address, or UART
N present type settings.

+ The philosophical goals behind the bsp defaults, a user writing an INI
file,
the flexibility, etc.are not being communicated. Why do this? What's good
about it? Why do users care?

+ If there is discussion of using the configuration INI file to do a custom
BSP variant, I didn't find it while looking for the class.

The last two could be me not finding something.

But ultimately we need to cut the 6 release branch. I think we have been
talking about it over a year now. Let's focus and kill it.


>
> On Wed, Apr 26, 2023 at 2:30 AM Sebastian Huber
>  wrote:
> > On 26.04.23 00:37, Chris Johns wrote:
> > > On 26/4/2023 4:01 am, Sebastian Huber wrote:
> > >> We have a fraction of a second if we use the CSafeLoader and no item
> cache.
> > >> Currently we use the SafeLoader which results in about 5 seconds. I
> would like
> > >> to use the build system for more stuff, so this could grow to 10, 15,
> or more
> > >> seconds which then start to get annoying if you work frequently with
> it.
> > > Then please outline what all you have in mind and what the long term
> plan is
> > > then we can consider the overall direction.
> >
> > We work on a prototype to build libbsd, lwip, abseil-cpp, googletest,
> > jsoncpp, libsodium, nats.c, protobuf, protobuf-c, utf8_range, cfs,
> > zephyr, chip vendor HALs, etc. with the RTEMS build system.
> >
>
> This is interesting although it appears to be quite a bit of scope
> creep. I guess the intent is to help with traceability of these
> projects too?
>
> -Gedare
> ___
> devel mailing list
> devel@rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Gedare Bloom]

Is the result of this discussion to use CSafeLoader for now? Or do we
still debate the transition to JSON?

I see some quirks need to be dealt with as you mentioned with the
multi-line stuff. The yaml is pretty simple to understand and work
with. I've dealt with hand-editing JSON and it's a pain with the
quotes and brackets/braces.

Since the transition is (apparently) an implementation detail, I don't
actually see that it matters do it now or do it after releasing 6. If
it doesn't change the user experience other than build time, then it
should matter when we do it. In that case, better to be conservative
to avoid rushing to push something this major so close to a release.

On Wed, Apr 26, 2023 at 2:30 AM Sebastian Huber
 wrote:
> On 26.04.23 00:37, Chris Johns wrote:
> > On 26/4/2023 4:01 am, Sebastian Huber wrote:
> >> We have a fraction of a second if we use the CSafeLoader and no item cache.
> >> Currently we use the SafeLoader which results in about 5 seconds. I would 
> >> like
> >> to use the build system for more stuff, so this could grow to 10, 15, or 
> >> more
> >> seconds which then start to get annoying if you work frequently with it.
> > Then please outline what all you have in mind and what the long term plan is
> > then we can consider the overall direction.
>
> We work on a prototype to build libbsd, lwip, abseil-cpp, googletest,
> jsoncpp, libsodium, nats.c, protobuf, protobuf-c, utf8_range, cfs,
> zephyr, chip vendor HALs, etc. with the RTEMS build system.
>

This is interesting although it appears to be quite a bit of scope
creep. I guess the intent is to help with traceability of these
projects too?

-Gedare
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Sebastian Huber]

On 26.04.23 00:37, Chris Johns wrote:

On 26/4/2023 4:01 am, Sebastian Huber wrote:

On 25.04.23 13:02, Karel Gardas wrote:

On 4/25/23 12:32, Sebastian Huber wrote:

On 25.04.23 12:10, Karel Gardas wrote:

On 4/25/23 11:03, Sebastian Huber wrote:


On 25.04.23 11:00, Karel Gardas wrote:

On 4/25/23 09:35, Sebastian Huber wrote:

The change from YAML to JSON for the build specification files is just an
implementation detail of the new build system. For the users of the new
build system nothing changes.

Let me ask for clarification. Does this yaml -> json transition include
(or not) files in RTEMS spec directory, e.g. files here:

https://git.rtems.org/rtems/tree/spec

It would affect all YAML files in this directory and no other files.

Oh. Let me ask, what is your future plan with the spec files? Considering
you would like to move them to JSON, would you also like to provide some DSL
+ compiler which would generates those? Or would you just like to keep JSON
and have developers editing those?

I would keep the JSON and have developer editing those.

An alternative to changing the format could be to use the CSafeLoader of the
system yaml module if it is available. It is not as fast as the JSON loader,
but acceptable (0.65s to 0.2s on my machine for loading the items).

Sorry for perhaps silly question, but why do you invest that much energy in
optimizing build speed -- by fraction of seconds here? I so far fail so to see
driving motivation force hence my question...

We have a fraction of a second if we use the CSafeLoader and no item cache.
Currently we use the SafeLoader which results in about 5 seconds. I would like
to use the build system for more stuff, so this could grow to 10, 15, or more
seconds which then start to get annoying if you work frequently with it.

Then please outline what all you have in mind and what the long term plan is
then we can consider the overall direction.


We work on a prototype to build libbsd, lwip, abseil-cpp, googletest, 
jsoncpp, libsodium, nats.c, protobuf, protobuf-c, utf8_range, cfs, 
zephyr, chip vendor HALs, etc. with the RTEMS build system.


--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Sebastian Huber]

On 26.04.23 01:52, Chris Johns wrote:

On 25/4/2023 5:35 pm, Sebastian Huber wrote:

Tooling makes sense if you have a high change rate.


That is not the use case I am discussing and it is not a factor.


This is not the case for the RTEMS build specification items.


I am not saying it is.


For which use case do we need tooling support?


We need tooling to lets us all understand this build system. YAML is easy to
learn however the data-structures, rules and the large number of file fragments
we have are complex and not apparent to anyone looking at it even if you have
invested time doing so. It reflects the fact that RTEMS is complicated to build.


I don't think building an RTEMS BSP is complicated once you have the 
tools installed.




I am advocate for what we have and will continue to be one so I am attempting to
address some things we could do better. History has shown the RTEMS core
developers are not great judges of the community's view of the project's
usability and accessibility.

The analogy is to consider the git command then the roles github and gitlab
provide with their user interfaces and tooling.


For a new option or BSP, you just copy and past from existing files/BSPs.
Changing a specific part of an existing file is just copy and paste some lines
and edit them in most cases.


My experience tells me it is not as simple as this. There is an element of guess
work a change is right. The recent dependency cases highlighted this and the
need for checking of some form to be present in the rtems.git repo.

We need to step back and consider the role of the build system before we discuss
implementation details.

The first requirement by a large margin is ease of use for users and the
community to make contributions. Meeting this requirement can be done a number
of ways. For example a user focused format for the relationships rather than one
that aids machine integration. The original ground work Amar did for the move to
waf was to define the build in Python as documented by waf. It was simple and
transparent. Another example is a machine focused format however we need tooling
to help the users manage making changes and accessing what is happening without
needing to learn the details of how it is implemented.

For tooling my order of importance is:

1. Visualise the structures and dependencies in a human readable manner. An
indication of which file is doing what would be helpful.

2. A check of changes users make that raises errors, dependency problems, etc.
This can be a command you run if you are making changes and does not need to run
every build.

I see JSON and tooling as linked together. I also not expect complete tooling to
be present for the change to happen. All I am wanting is the need for tooling be
agreed on and it is located in rtems.git. The development can then happen and
evolve as the community sees a need.


From my experience with waf I would not say that the waf build system 
is simple and transparent. Things which are very easy with make are 
complicated with waf at least for me. I asked a couple of waf questions 
on the RTEMS mailing list which no core developer could answer. The 
support from the waf developer was a great help, but without this help I 
would not have been able to write the waf based build system for RTEMS.


I think our biggest obstacle to improve things in the area you outlined 
above is the scattered project infrastructure with several Git 
repositories and the lack of CI pipelines. I will probably start a 
discussion about this topic because I think our current project setting 
has severe maintenance issues.


--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Chris Johns]

On 25/4/2023 5:35 pm, Sebastian Huber wrote:
> Tooling makes sense if you have a high change rate. 

That is not the use case I am discussing and it is not a factor.

> This is not the case for the RTEMS build specification items.

I am not saying it is.

> For which use case do we need tooling support?

We need tooling to lets us all understand this build system. YAML is easy to
learn however the data-structures, rules and the large number of file fragments
we have are complex and not apparent to anyone looking at it even if you have
invested time doing so. It reflects the fact that RTEMS is complicated to build.

I am advocate for what we have and will continue to be one so I am attempting to
address some things we could do better. History has shown the RTEMS core
developers are not great judges of the community's view of the project's
usability and accessibility.

The analogy is to consider the git command then the roles github and gitlab
provide with their user interfaces and tooling.

> For a new option or BSP, you just copy and past from existing files/BSPs.
> Changing a specific part of an existing file is just copy and paste some lines
> and edit them in most cases.

My experience tells me it is not as simple as this. There is an element of guess
work a change is right. The recent dependency cases highlighted this and the
need for checking of some form to be present in the rtems.git repo.

We need to step back and consider the role of the build system before we discuss
implementation details.

The first requirement by a large margin is ease of use for users and the
community to make contributions. Meeting this requirement can be done a number
of ways. For example a user focused format for the relationships rather than one
that aids machine integration. The original ground work Amar did for the move to
waf was to define the build in Python as documented by waf. It was simple and
transparent. Another example is a machine focused format however we need tooling
to help the users manage making changes and accessing what is happening without
needing to learn the details of how it is implemented.

For tooling my order of importance is:

1. Visualise the structures and dependencies in a human readable manner. An
indication of which file is doing what would be helpful.

2. A check of changes users make that raises errors, dependency problems, etc.
This can be a command you run if you are making changes and does not need to run
every build.

I see JSON and tooling as linked together. I also not expect complete tooling to
be present for the change to happen. All I am wanting is the need for tooling be
agreed on and it is located in rtems.git. The development can then happen and
evolve as the community sees a need.

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Chris Johns]

On 26/4/2023 4:01 am, Sebastian Huber wrote:
> On 25.04.23 13:02, Karel Gardas wrote:
>> On 4/25/23 12:32, Sebastian Huber wrote:
>>> On 25.04.23 12:10, Karel Gardas wrote:
 On 4/25/23 11:03, Sebastian Huber wrote:
>
>
> On 25.04.23 11:00, Karel Gardas wrote:
>> On 4/25/23 09:35, Sebastian Huber wrote:
>>> The change from YAML to JSON for the build specification files is just 
>>> an
>>> implementation detail of the new build system. For the users of the new
>>> build system nothing changes.
>>
>> Let me ask for clarification. Does this yaml -> json transition include
>> (or not) files in RTEMS spec directory, e.g. files here:
>>
>> https://git.rtems.org/rtems/tree/spec
>
> It would affect all YAML files in this directory and no other files.

 Oh. Let me ask, what is your future plan with the spec files? Considering
 you would like to move them to JSON, would you also like to provide some 
 DSL
 + compiler which would generates those? Or would you just like to keep JSON
 and have developers editing those?
>>>
>>> I would keep the JSON and have developer editing those.
>>>
>>> An alternative to changing the format could be to use the CSafeLoader of the
>>> system yaml module if it is available. It is not as fast as the JSON loader,
>>> but acceptable (0.65s to 0.2s on my machine for loading the items).
>>
>> Sorry for perhaps silly question, but why do you invest that much energy in
>> optimizing build speed -- by fraction of seconds here? I so far fail so to 
>> see
>> driving motivation force hence my question...
> 
> We have a fraction of a second if we use the CSafeLoader and no item cache.
> Currently we use the SafeLoader which results in about 5 seconds. I would like
> to use the build system for more stuff, so this could grow to 10, 15, or more
> seconds which then start to get annoying if you work frequently with it.

Then please outline what all you have in mind and what the long term plan is
then we can consider the overall direction.

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Sebastian Huber]

On 25.04.23 13:02, Karel Gardas wrote:

On 4/25/23 12:32, Sebastian Huber wrote:

On 25.04.23 12:10, Karel Gardas wrote:

On 4/25/23 11:03, Sebastian Huber wrote:



On 25.04.23 11:00, Karel Gardas wrote:

On 4/25/23 09:35, Sebastian Huber wrote:
The change from YAML to JSON for the build specification files is 
just an implementation detail of the new build system. For the 
users of the new build system nothing changes.


Let me ask for clarification. Does this yaml -> json transition 
include (or not) files in RTEMS spec directory, e.g. files here:


https://git.rtems.org/rtems/tree/spec


It would affect all YAML files in this directory and no other files.


Oh. Let me ask, what is your future plan with the spec files? 
Considering you would like to move them to JSON, would you also like 
to provide some DSL + compiler which would generates those? Or would 
you just like to keep JSON and have developers editing those?


I would keep the JSON and have developer editing those.

An alternative to changing the format could be to use the CSafeLoader 
of the system yaml module if it is available. It is not as fast as the 
JSON loader, but acceptable (0.65s to 0.2s on my machine for loading 
the items).


Sorry for perhaps silly question, but why do you invest that much energy 
in optimizing build speed -- by fraction of seconds here? I so far fail 
so to see driving motivation force hence my question...


We have a fraction of a second if we use the CSafeLoader and no item 
cache. Currently we use the SafeLoader which results in about 5 seconds. 
I would like to use the build system for more stuff, so this could grow 
to 10, 15, or more seconds which then start to get annoying if you work 
frequently with it.


Given the performance of the CSafeLoader I will probably use it and keep 
the YAML format.


--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Karel Gardas]

On 4/25/23 12:32, Sebastian Huber wrote:

On 25.04.23 12:10, Karel Gardas wrote:

On 4/25/23 11:03, Sebastian Huber wrote:



On 25.04.23 11:00, Karel Gardas wrote:

On 4/25/23 09:35, Sebastian Huber wrote:
The change from YAML to JSON for the build specification files is 
just an implementation detail of the new build system. For the 
users of the new build system nothing changes.


Let me ask for clarification. Does this yaml -> json transition 
include (or not) files in RTEMS spec directory, e.g. files here:


https://git.rtems.org/rtems/tree/spec


It would affect all YAML files in this directory and no other files.


Oh. Let me ask, what is your future plan with the spec files? 
Considering you would like to move them to JSON, would you also like 
to provide some DSL + compiler which would generates those? Or would 
you just like to keep JSON and have developers editing those?


I would keep the JSON and have developer editing those.

An alternative to changing the format could be to use the CSafeLoader of 
the system yaml module if it is available. It is not as fast as the JSON 
loader, but acceptable (0.65s to 0.2s on my machine for loading the items).


Sorry for perhaps silly question, but why do you invest that much energy 
in optimizing build speed -- by fraction of seconds here? I so far fail 
so to see driving motivation force hence my question...


Thanks,
Karel




___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Sebastian Huber]

On 25.04.23 12:10, Karel Gardas wrote:

On 4/25/23 11:03, Sebastian Huber wrote:



On 25.04.23 11:00, Karel Gardas wrote:

On 4/25/23 09:35, Sebastian Huber wrote:
The change from YAML to JSON for the build specification files is 
just an implementation detail of the new build system. For the users 
of the new build system nothing changes.


Let me ask for clarification. Does this yaml -> json transition 
include (or not) files in RTEMS spec directory, e.g. files here:


https://git.rtems.org/rtems/tree/spec


It would affect all YAML files in this directory and no other files.


Oh. Let me ask, what is your future plan with the spec files? 
Considering you would like to move them to JSON, would you also like to 
provide some DSL + compiler which would generates those? Or would you 
just like to keep JSON and have developers editing those?


I would keep the JSON and have developer editing those.

An alternative to changing the format could be to use the CSafeLoader of 
the system yaml module if it is available. It is not as fast as the JSON 
loader, but acceptable (0.65s to 0.2s on my machine for loading the items).


--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Karel Gardas]

On 4/25/23 11:03, Sebastian Huber wrote:



On 25.04.23 11:00, Karel Gardas wrote:

On 4/25/23 09:35, Sebastian Huber wrote:
The change from YAML to JSON for the build specification files is 
just an implementation detail of the new build system. For the users 
of the new build system nothing changes.


Let me ask for clarification. Does this yaml -> json transition 
include (or not) files in RTEMS spec directory, e.g. files here:


https://git.rtems.org/rtems/tree/spec


It would affect all YAML files in this directory and no other files.


Oh. Let me ask, what is your future plan with the spec files? 
Considering you would like to move them to JSON, would you also like to 
provide some DSL + compiler which would generates those? Or would you 
just like to keep JSON and have developers editing those?


Thanks,
Karel

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Sebastian Huber]

On 25.04.23 11:00, Karel Gardas wrote:

On 4/25/23 09:35, Sebastian Huber wrote:
The change from YAML to JSON for the build specification files is just 
an implementation detail of the new build system. For the users of the 
new build system nothing changes.


Let me ask for clarification. Does this yaml -> json transition include 
(or not) files in RTEMS spec directory, e.g. files here:


https://git.rtems.org/rtems/tree/spec


It would affect all YAML files in this directory and no other files.

--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Karel Gardas]

On 4/25/23 09:35, Sebastian Huber wrote:
The change from YAML to JSON for the build specification files is just 
an implementation detail of the new build system. For the users of the 
new build system nothing changes.


Let me ask for clarification. Does this yaml -> json transition include 
(or not) files in RTEMS spec directory, e.g. files here:


https://git.rtems.org/rtems/tree/spec

Thanks!
Karel

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Sebastian Huber]

On 24.04.23 23:45, Chris Johns wrote:

On 25/4/2023 12:20 am, Sebastian Huber wrote:

On 24.04.23 16:17, Kinsey Moore wrote:

I think we've been moving away from in-file comments in the YAML and toward
actual labels in the data, anyway. If it's important information, it should be
kept properly.


Yes, there should be no comments in the build specification files. They would be
lost in YAML load/save cycle.


I am fine with json as a format. Comments are useful even for copyright notices
which would be lost. Is that OK?


We don't have comments in the build specification files. There is an 
attribute for copyright notices, for example:


SPDX-License-Identifier: CC-BY-SA-4.0 OR BSD-2-Clause
build-type: library
cflags:
- ${COVERAGE_COMPILER_FLAGS}
copyrights:
- Copyright (C) 2020 embedded brains GmbH (http://www.embedded-brains.de)
cppflags: []



My concern is a major shift on top of a major change to waf so close together.
As a community we have only just started to get a number of people across what
we have and then things change? I see the counter point about making this happen
before 6 however I am still not sure what the cost/benefit is?


The change from YAML to JSON for the build specification files is just 
an implementation detail of the new build system. For the users of the 
new build system nothing changes. If we want to change to JSON, then 
this should definitely happen before the RTEMS 6 release or not at all. 
The cost of moving to JSON is that the file format is a bit more verbose 
(lots of " quotation). The benefit is a less complex wscript and faster 
build times. The change is trivial to do:


https://github.com/sebhub/rtems/commit/47c1e7218984ad0140d677d21c8526ec4fef653e

The JSON files produced by the Python json.dump(data2, out, 
sort_keys=True, indent=2) have a Git friendly format.


A bit problematic is multiline content:

https://github.com/sebhub/rtems/commit/47c1e7218984ad0140d677d21c8526ec4fef653e#diff-5a11531b07bc2a92e083a72414b45f96824a276a4dd925f9639cfc40081cba4b

We could work around this by using a list of lines.



The YAML lacks support tooling in rtems.git but it is usable as is. I am
concerned a move to json would compound the problem. On the other hand if we
considered json as the data could tooling mean we avoid hand editing?

If that was the approach how would we move to have tooling to help?


Tooling makes sense if you have a high change rate. This is not the case 
for the RTEMS build specification items. For which use case do we need 
tooling support? For a new option or BSP, you just copy and past from 
existing files/BSPs. Changing a specific part of an existing file is 
just copy and paste some lines and edit them in most cases.


--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Chris Johns]

On 25/4/2023 12:20 am, Sebastian Huber wrote:
> On 24.04.23 16:17, Kinsey Moore wrote:
>> I think we've been moving away from in-file comments in the YAML and toward
>> actual labels in the data, anyway. If it's important information, it should 
>> be
>> kept properly.
> 
> Yes, there should be no comments in the build specification files. They would 
> be
> lost in YAML load/save cycle.

I am fine with json as a format. Comments are useful even for copyright notices
which would be lost. Is that OK?

My concern is a major shift on top of a major change to waf so close together.
As a community we have only just started to get a number of people across what
we have and then things change? I see the counter point about making this happen
before 6 however I am still not sure what the cost/benefit is?

The YAML lacks support tooling in rtems.git but it is usable as is. I am
concerned a move to json would compound the problem. On the other hand if we
considered json as the data could tooling mean we avoid hand editing?

If that was the approach how would we move to have tooling to help?

Chrid
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Sebastian Huber]

On 24.04.23 16:17, Kinsey Moore wrote:
I think we've been moving away from in-file comments in the YAML and 
toward actual labels in the data, anyway. If it's important information, 
it should be kept properly.


Yes, there should be no comments in the build specification files. They 
would be lost in YAML load/save cycle.


--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Kinsey Moore]

I think we've been moving away from in-file comments in the YAML and toward
actual labels in the data, anyway. If it's important information, it should
be kept properly.

Kinsey

On Mon, Apr 24, 2023 at 8:55 AM Sam Price  wrote:

> Yaml files can have comments, json files can’t.  So you would lose some
> documentation…
> --
> Sincerely,
>
> Sam Price
>
> ___
> devel mailing list
> devel@rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Sam Price]

Yaml files can have comments, json files can’t.  So you would lose some
documentation…
-- 
Sincerely,

Sam Price
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Change build specification files from YAML to JSON?

[Sebastian Huber]

Hello Andrew,

On 24.04.23 11:56, Andrew Butterfield wrote:

  would this also affect the specification item YAML files in RTEMS Central?


no, I already added support to load specification items from JSON files.

JSON files are a bit harder to write manually than YAML files. For the 
build items it doesn't really matter. For the other parts which contain 
source code snippets and documentation the YAML files are more convenient.


--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel