Re: [riot-devel] Updates to the build system - modules definition

[Gaëtan Harter]

Hi all,


I want to give my opinion on the discussions that happened here in 
Berlin regarding the build system evolution.

And also, choose the direction for next development steps.



# Laze presentation #

Kaspar presented his new tool that replaces the current Make based build 
system based on a custom python software reading the configuration and 
generating `ninja` files to builds.

It uses YAML description for modules, application, and toolchain elements.


GOOD

* The YAML description is a good direction into what I need as a 
"descriptive language" for the modules.
* Using a meta build-system, removes the need to parse all the 
configuration from scratch each times an application is build which is a 
static time consumption for CI.

 It could be done in make but may be more cumbersome to write and maintain.
* Refined module build configuration to only show the needed flags and 
dependencies, it will help a lot for caching.

* I love python.

BAD

* Python is not a build system language, so not designed to do 'targets' 
generation but more executing commands after the other.
* Introducing our own custom build language framework instead of using 
an existing one, so a non standard tool.
* The build system implementation (how to parse yaml files, generate 
build rules), is not separated from the laze tool itself, so hard to 
have the implementation stored in the repository while having a generic 
tool installed globally.
* Many argument are based on speed, but the speed improvement are not 
all a result of using 'laze' only, but cleaning up concepts in general.



# Criticism of the current make build system #

However, I agree that our current build system has a lot of flaws that I 
would like to see solved.


* Its necessary to run `make clean` all the time because there are 
changes that do not correctly trigger rebuild, so you are never sure
* The module definition/configuration is not structured and descriptive 
so is problematic to provide finer granularity module building, all 
modules see all CFLAGS, INCLUDES, and included modules MODULE_ macros.
    * Its now impossible to cache results between applications as they 
are done with different config.
* Not all dependencies are explicitly described, so hard to understand 
the optional dependencies, the dependencies on interface, dependencies 
depending on configuration. This would also help refining module build 
configuration.
* Our current build system only takes care of building one application 
for one board. For the CI and maintainers, when building several 
targets, they must re-execute make from scratch for each configuration.
* And the initial  tool checking, is it linux, modules discovery, should 
be done every time. And this really adds up on many targets builds
    * It can only be solved correctly by first having structured module 
description as you can parse it once and apply it to different 
configurations
* There is no support for building in a common build directory so it 
takes a lot of space and also rebuilds multiple time the same thing.
    * It currently makes no sense, as module have different 
configurations, but after fixing the previous issues, one built module 
configuration could be shared between different apps, saving space and 
compile time.



# Next steps together #

To go forward and use what he developed as a possible solution, I 
proposed to start a documentation phase, on both his new system + YAML 
module format, and on what and how things are currently done in the 
build system.
Adding user and developer documentation for the current system would 
fill the lack of documentation and also help describing all the 
currently existing features, show the problems, find forgotten requirements.
And documenting his description format and his tool could show what are 
the benefits of his choices and also the tradeoffs.



One of my question is, is it interesting to integrate some changes in 
the current make system, like some early module description, even if 
replaced later by another build system ?


I will open issues on github to follow the next steps.

Cheers,
Gaëtan









On 01.12.2017 10:49, Kaspar Schleiser wrote:

Hi Matthias,

On 12/01/2017 10:29 AM, Matthias Waehlisch wrote:

When are you gonna take a look at my ninja-based build system? It
solves 1-3 quite nicely, and more. You could save a lot of time.


   Pointer?

https://github.com/kaspar030/RIOT/tree/add_laze_buildfiles

Don't expect a nice out-of-the-box experience (yet).

Kaspar
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel


___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Thomas C. Schmidt]

Hiho Gaetan,

wasn't it in particular that the current build system does not produce 
well-defined results in the sense that the outcome depends on the order 
and hidden interferences??


Wasn't it also that the current configuration system includes overloaded 
semantics and is thus rather intransparent??


These to me seemed to be the core issues??

Cheers,
 Thomas

On 30/11/2017 22:27, Kaspar Schleiser wrote:

Hi,

On 11/30/2017 04:32 PM, Gaëtan Harter wrote:

   1. Configuration is not documented
   2. Information is not readable
   3. Modules definition is scattered but in RIOT global files



With these issues in mind, I propose to add parseable module meta-data
definitions in a file in each module directory to first replace the
existing information and then extend it to add more precise ones.
How to do in practice it is still to be defined and discussed.


When are you gonna take a look at my ninja-based build system? It solves
1-3 quite nicely, and more. You could save a lot of time.

Kaspar
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel



--

Prof. Dr. Thomas C. Schmidt
° Hamburg University of Applied Sciences   Berliner Tor 7 °
° Dept. Informatik, Internet Technologies Group20099 Hamburg, Germany °
° http://www.haw-hamburg.de/inet   Fon: +49-40-42875-8452 °
° http://www.informatik.haw-hamburg.de/~schmidtFax: +49-40-42875-8409 °
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Kaspar Schleiser]

Hi Matthias,

On 12/01/2017 10:29 AM, Matthias Waehlisch wrote:
>> When are you gonna take a look at my ninja-based build system? It 
>> solves 1-3 quite nicely, and more. You could save a lot of time.
>>
>   Pointer?

https://github.com/kaspar030/RIOT/tree/add_laze_buildfiles

Don't expect a nice out-of-the-box experience (yet).

Kaspar
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Matthias Waehlisch]

On Thu, 30 Nov 2017, Kaspar Schleiser wrote:

> When are you gonna take a look at my ninja-based build system? It 
> solves 1-3 quite nicely, and more. You could save a lot of time.
> 
  Pointer?


Cheers
  matthias


-- 
Matthias Waehlisch
.  Freie Universitaet Berlin, Computer Science
.. http://www.cs.fu-berlin.de/~waehl
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Kaspar Schleiser]

Hi,

On 11/30/2017 04:32 PM, Gaëtan Harter wrote:
>   1. Configuration is not documented
>   2. Information is not readable
>   3. Modules definition is scattered but in RIOT global files

> With these issues in mind, I propose to add parseable module meta-data
> definitions in a file in each module directory to first replace the
> existing information and then extend it to add more precise ones.
> How to do in practice it is still to be defined and discussed.

When are you gonna take a look at my ninja-based build system? It solves
1-3 quite nicely, and more. You could save a lot of time.

Kaspar
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Gaëtan Harter]

Hi,

On 11/30/2017 11:52 AM, Kaspar Schleiser wrote:

Hi Dan,

On 11/29/2017 02:33 PM, Daniel Petry wrote:

1. The current build system isn't suitable to support the front end for
RAPstore that Hendrik developed for his bachelor thesis, which requires
that certain information can be displayed to the users.

I hear about this for the first time. Are there any pointers?
Its not specific to the current version of the interface, but to what I 
would like to achieve for the RAPstore project.
Configure a firmware using a graphical interface by selecting modules 
and configuring them.



2. The current build system doesn't allow developers to easily include
modules in their applications

USEMODULE += module_name?

In our context, it could be important to know the consequences of this.
On my computer, when I say 'aptitude install PACKAGE' I am informed of 
the additional dependencies.
Here with interdependencies, and modules affecting the global 
configuration it can be more important than this.


The cbor_ctime pseudomodule, when used with newlib, makes every source 
files compile with -DGNU_SOURCE=1.

I would not know that when simply adding it to USEMODULE.


built can be affected by files/code outside the module directory
2.1 API changes as a result of including other modules aren't
immediately visible in that module
2.2 API changes on other modules as a result of including that module
isn't immediately visible
2.3 The complete build information for a module isn't localised only in
the module directory

If modules are interdependent, they will affect each other. How can a
different module definition help here?
Regarding 'interdependency', I think about module that behave 
differently => are compiled differently if another module is included.

The optional dependencies that affect either your code or your API.

Adding this to a definition would help as documentation. It still allow 
specific requirements, still allows strange architecture for compilation 
optimization, but at least its said somewhere.


It took me some time to understand that `at86rf231` and `at86rf212b` are 
pseudomodules and are both implemented by `at86rf2xx` which is compiled 
differently depending on which one is included. 'git grep at86rf231' 
says nothing about it.


It would of course require tooling to help writing and maintaining these 
modules definition.
And even more for each information that the build system is not using 
and enforcing.



Gaëtan


Kaspar
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel


___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Gaëtan Harter]

Hi Thomas, Francisco, all,

I will try to summarize with three aspects formatted as problems.

Just for clarity, in the following text, use the words 'description', 
'informations', 'configuration', 'dependencies' to describe all the 
meta-information that can be attached to a module in addition to its 
source code.



I also am in demand of other issues that you have and that should be 
taken into account if the current system is refactored.
So I invite you to talk about your case too, I want to start discussions 
before working on it.


Also I welcome all remarks regarding my problems and the way I want to 
address them.



 1. Configuration is not documented

As Francisco is saying, about configuring an application and so its modules.
The problem, for me, is that its hard to know the possible 
configurations of a module and the impact of the configuration/dependencies.



   Details:

When building an application, I need to:

 * describe what I want to use in my application, which usually means
   saying which module/package I want
 * configure the application and the modules, by using either specific
   variables `DEVELHELP`, `OUTPUT` or manually setting CFLAGS
 * Specify requirements to work only on boards with this hardware support.

 * But it also means understanding the impact of adding a specific
   module / setting one configuration

Right now the impact is tedious to find. A dependency pulled by one 
module can have an impact on another module code, on its API, on all 
modules.
And modules configuration, is often only described into the doxygen or 
the source code: what CFLAGS should I put to configure the radio channel ?


I would like to clearly write down these public configurations and 
dependencies relations.



 2. Information is not readable

The module information cannot be presented and re-interpreted to show on 
a graphical interface.
It is not machine readable, and the same way of defining things has 
multiple meaning.



   Details:

I would need to have parseable files usable by an automated system, 
which is currently not the case.


For my needs, it would be necessary to refine the current information to 
make it more precise.
The 'dependencies' and configurations now used in the build system can 
be ambiguous to present because they have different meaning.


 * Some optional dependencies have different impact, they:
 o can change the source code behavior
 o can change the module API
 o can affect every modules
 * The same goes for configuration, is the configuration affecting the
   module API ?



 3. Modules definition is scattered but in RIOT global files

The meta-information on a module is not grouped together with the module 
but written in 3-4 files global to the RIOT repository.
This means reading different files to get a complete view on a module 
but also that it is not possible to use external modules trees without 
modifying the RIOT repository.



   Details:

I would like to make modules self contained. I think it would help 
developers as it improves readability and documentation by having only 
one place to understand a module.
Also, external modules would be defined in the same way as internal ones 
by just adding a tree of modules to load.



   Bonus. Unnecessary globally visible configuration

A bonus aspect that I would like to tackle in the long term when the 
updates would be in place.


Right now all CFLAGS/INCLUDES are global to all modules.
This means a different version of the `core` module is build if another 
high level module is included or has its configuration changed.


Having more informations on modules dependencies and configuration scope 
could help refine this.



 Details:

Having a determined context would makes tests more reliable, because we 
could know which configuration has been tested and which not.
It would require explicit build system configuration to make a 
configuration affect another module.
Solving this could allow doing build caching at a the build system level 
when build with common local context.
I am also interested in reproducible builds and so being able to 
describe what has been build with the minimal information required and 
verifying the produced output.



With these issues in mind, I propose to add parseable module meta-data 
definitions in a file in each module directory to first replace the 
existing information and then extend it to add more precise ones.

How to do in practice it is still to be defined and discussed.


Cheers,
Gaëtan - cladmi


On 11/27/2017 04:39 PM, Francisco Javier Acosta Padilla wrote:

Hi,

I think the problem to solve is that RIOT is very difficult to configure.

While working with other IoT operating systems, one can notice that 
RIOT is very hardly configurable, unless we are very familiar with the 
build system and specific RIOT CFLAGS and special key-words 
(USEMODULE, USEPKG, etc…).


Besides, I have no knowledge of a tool that can p

Re: [riot-devel] Updates to the build system - modules definition

[Matthias Waehlisch]

Hi Kaspar,

On Thu, 30 Nov 2017, Kaspar Schleiser wrote:

> > 1. The current build system isn't suitable to support the front end for
> > RAPstore that Hendrik developed for his bachelor thesis, which requires
> > that certain information can be displayed to the users.
> 
> I hear about this for the first time. Are there any pointers?
> 
  it was briefly presented during the RIOT Summit this year. As soon as 
we have a public demo ready, we will post on the list.


Cheers
  matthias

-- 
Matthias Waehlisch
.  Freie Universitaet Berlin, Computer Science
.. http://www.cs.fu-berlin.de/~waehl
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Kaspar Schleiser]

Hi Dan,

On 11/29/2017 02:33 PM, Daniel Petry wrote:
> 1. The current build system isn't suitable to support the front end for
> RAPstore that Hendrik developed for his bachelor thesis, which requires
> that certain information can be displayed to the users.

I hear about this for the first time. Are there any pointers?

> 2. The current build system doesn't allow developers to easily include
> modules in their applications

USEMODULE += module_name?

> built can be affected by files/code outside the module directory
>   2.1 API changes as a result of including other modules aren't
> immediately visible in that module
>   2.2 API changes on other modules as a result of including that module
> isn't immediately visible
>   2.3 The complete build information for a module isn't localised only in
> the module directory

If modules are interdependent, they will affect each other. How can a
different module definition help here?

Kaspar
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Daniel Petry]

Hi

> I wonder, what problems are we trying to solve?

The problems as I understand it are the following - Gaetan, please
correct me if I haven't got this quite right, or if there are additions!


1. The current build system isn't suitable to support the front end for
RAPstore that Hendrik developed for his bachelor thesis, which requires
that certain information can be displayed to the users. Information for
each module is not readily available in one place and in a format which
is parseable to human-readable text.
1.1 Information on other required modules is not readily available in a
common format
1.2 How the inclusion of modules changes functionality of other modules
is not readily available in a common format
1.3 Type of module is not readily available in a common format
1.4 Include path for applications that use the module is not readily
available in a common format


2. The current build system doesn't allow developers to easily include
modules in their applications, and they need to investigate
cross-dependencies etc. They have to look in a range of different places
rather than just one place to see what effects the inclusion of a module
has.
2.1 Any module API/functionality changes to modules as a result of
including other modules are difficult to find
2.2 Any other modules they need to include are difficult to find
2.3 The order in which they need to include the modules is difficult to
figure out
2.4 The CFLAGS that are added and whether they're global or local to a
module is difficult to figure out


3. The current build system doesn't allow developers to easily work on
modules. This includes creating new modules, or changing current
modules, either just to make them work or without introducing regression
bugs in related modules. This is because module code and the way they're
built can be affected by files/code outside the module directory
2.1 API changes as a result of including other modules aren't
immediately visible in that module
2.2 API changes on other modules as a result of including that module
isn't immediately visible
2.3 The complete build information for a module isn't localised only in
the module directory


@Gaetan, would you say this is the case or do you have any comments?

Maybe we could decide on the problems - i.e., refine/change this list -
before moving on to discuss solutions? A good target might be to decide
on the problems that need to be solved - i.e. the requirements - by a
week on Friday then go on to a design/development phase?


Dan.


___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Francisco Javier Acosta Padilla]

Hi,

I think the problem to solve is that RIOT is very difficult to configure.

While working with other IoT operating systems, one can notice that RIOT is 
very hardly configurable, unless we are very familiar with the build system and 
specific RIOT CFLAGS and special key-words (USEMODULE, USEPKG, etc…).

Besides, I have no knowledge of a tool that can parse RIOT makefiles and give 
an insight about module usage, dependencies, features, etc. 

Thus IMHO a way to make, first, RIOT more *friendly* configurable it’s a good 
and necessary feature. This would also allow to eventually build a dependency 
tree in a more human-readable fashion, as well as give some help to get 
different product lines according to the selected features.

However, I’d say that *deep* modifications to the current build system can lead 
to instability or buggy builds. I’m not against any improvement to the build 
system, but maybe as a first approach some additions like files in any more 
descriptive (parseable) language can help to describe RIOT in a higher level, 
without affecting at all the underlying system. Then, with the knowledge 
obtained thanks to such a description, we can imagine to generate makefiles or 
make “make” calls with the specific RIOT CFLAGS, USEMODULE, etc.

My 2c.

Cheers,

-- 
Francisco Javier Acosta Padilla
Research Engineer at INRIA Saclay
INFINE Team

On 24 November 2017 at 19:16:29, Thomas C. Schmidt (t.schm...@haw-hamburg.de) 
wrote:

Hi Dan, Gaetan,

I wonder, what problems are we trying to solve?

Maybe we can clearly enumerate them first so that we can check later,  
whether an improved solution really solves these problems.

Cheers,
Thomas

On 24/11/2017 16:47, Daniel Petry wrote:
> Hi Gaetan
>  
>  
>> I would like to introduce some packaging concepts around RIOT.
>> To consider modules like well defined distribution packages and not only
>> source files added to an application firmware.
>  
> From what you're saying, I understand that the aim is to make modules
> completely self-contained with respect to build definitions, and make
> those definitions much more human readable. In this way, an application
> developer can go to only one place to find out information regarding a
> module's build (dependency information, etc.), and a module developer
> only has to write/change the module directory Makefile and can do so
> with declarative language. Also, the information can be easily parseable
> for eg a user interface.
>  
> So the changes you're proposing are:
>  
> 1) Move build information concerning a particular module into that
> module's Makefile
> 2) Make the module makefiles able to be written with purely declarative
> language
> 3) Retain backwards compatibility with the current build system
>  
> Is this correct?
>  
> I think 2) would be great for user friendliness, and 3) is a no-brainer.
> 1) is a bigger topic as there are a number of different ways in which
> dependencies are manifested for example, so I think this can be a point
> for further discussion... do you have any particular examples?
>  
>  
> Dan.
>  
> ___
> devel mailing list
> devel@riot-os.org
> https://lists.riot-os.org/mailman/listinfo/devel
>  

--  

Prof. Dr. Thomas C. Schmidt
° Hamburg University of Applied Sciences Berliner Tor 7 °
° Dept. Informatik, Internet Technologies Group 20099 Hamburg, Germany °
° http://www.haw-hamburg.de/inet Fon: +49-40-42875-8452 °
° http://www.informatik.haw-hamburg.de/~schmidt Fax: +49-40-42875-8409 °
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Thomas C. Schmidt]

Hi Dan, Gaetan,

I wonder, what problems are we trying to solve?

Maybe we can clearly enumerate them first so that we can check later, 
whether an improved solution really solves these problems.


Cheers,
 Thomas

On 24/11/2017 16:47, Daniel Petry wrote:

Hi Gaetan



I would like to introduce some packaging concepts around RIOT.
To consider modules like well defined distribution packages and not only
source files added to an application firmware.


 From what you're saying, I understand that the aim is to make modules
completely self-contained with respect to build definitions, and make
those definitions much more human readable.  In this way, an application
developer can go to only one place to find out information regarding a
module's build (dependency information, etc.), and a module developer
only has to write/change the module directory Makefile and can do so
with declarative language. Also, the information can be easily parseable
for eg a user interface.

So the changes you're proposing are:

1) Move build information concerning a particular module into that
module's Makefile
2) Make the module makefiles able to be written with purely declarative
language
3) Retain backwards compatibility with the current build system

Is this correct?

I think 2) would be great for user friendliness, and 3) is a no-brainer.
1) is a bigger topic as there are a number of different ways in which
dependencies are manifested for example, so I think this can be a point
for further discussion... do you have any particular examples?


Dan.

___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel



--

Prof. Dr. Thomas C. Schmidt
° Hamburg University of Applied Sciences   Berliner Tor 7 °
° Dept. Informatik, Internet Technologies Group20099 Hamburg, Germany °
° http://www.haw-hamburg.de/inet   Fon: +49-40-42875-8452 °
° http://www.informatik.haw-hamburg.de/~schmidtFax: +49-40-42875-8409 °
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Gaëtan Harter]

On 11/24/2017 04:55 PM, Martine Lenders wrote:

Hi Daniel,

2017-11-24 16:47 GMT+01:00 Daniel Petry >:


[…]

1) Move build information concerning a particular module into that
module's Makefile
2) Make the module makefiles able to be written with purely
declarative
language
3) Retain backwards compatibility with the current build system


 I don't think it is possible this way and I don't think that's what 
Gaëtan meant. Makefiles demand to be in make's language, so you can't 
just make up your own declarative language in them. You could however 
use some declaritive language files like json or yaml and generate 
Makefiles from them (or let make parse them).



Hi,

I may have mixed 'Makefile' and file written in a 'GNUMake' language in 
my description.


They address different steps of the build:
* Makefile is when you actually compile the source files to an archive 
with a given configuration (CC, CFLAGS, INCLUDES)
* All the Makefile.dep/Makefile.include/Makefile.features files that 
prepare this configuration and say which Makefile to execute


And its the part of configuration I would like to address.
Regarding `Makefile` I just want to remove the `DIRS +=` part of it.


What I want is add another file `module.desc` grouping the existing 
information and metadata on the module:

* Submodule
* is it a pseudomodule
* includes path
* dependencies
* global CFLAGS
* … and add more.

Regarding the "purely declarative", I would like to express it in 
GNUMake syntax and restrict to setting specially named variables.
My rational about this, is that if its written in XML, and let make 
parse them, it is still required to translate the information to 
variables that the make build system can understand.
So still needs to have a syntax to describe, for example, that a module 
has an optional dependency on another, and so be definable in a gnu make 
syntax.


Cheers,
Gaëtan


Cheers,
Martine



___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel


___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Martine Lenders]

Hi Daniel,

2017-11-24 16:47 GMT+01:00 Daniel Petry :

> […]
>
> 1) Move build information concerning a particular module into that
> module's Makefile
> 2) Make the module makefiles able to be written with purely declarative
> language
> 3) Retain backwards compatibility with the current build system


 I don't think it is possible this way and I don't think that's what Gaëtan
meant. Makefiles demand to be in make's language, so you can't just make up
your own declarative language in them. You could however use some
declaritive language files like json or yaml and generate Makefiles from
them (or let make parse them).

Cheers,
Martine
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Daniel Petry]

Hi Gaetan


> I would like to introduce some packaging concepts around RIOT.
> To consider modules like well defined distribution packages and not only 
> source files added to an application firmware. 

From what you're saying, I understand that the aim is to make modules
completely self-contained with respect to build definitions, and make
those definitions much more human readable.  In this way, an application
developer can go to only one place to find out information regarding a
module's build (dependency information, etc.), and a module developer
only has to write/change the module directory Makefile and can do so
with declarative language. Also, the information can be easily parseable
for eg a user interface.

So the changes you're proposing are:

1) Move build information concerning a particular module into that
module's Makefile
2) Make the module makefiles able to be written with purely declarative
language
3) Retain backwards compatibility with the current build system

Is this correct?

I think 2) would be great for user friendliness, and 3) is a no-brainer.
1) is a bigger topic as there are a number of different ways in which
dependencies are manifested for example, so I think this can be a point
for further discussion... do you have any particular examples?


Dan.

___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

Re: [riot-devel] Updates to the build system - modules definition

[Martine Lenders]

Hi Gaëtan,

I think most of us are happy about any more structural approach to the
build system, and me personally would prefer if we can stick to GNU
standard tools. So in my opinion: go ahead. Maybe - as discussed offline -
also start drafting some GitHub issues, outlining the drawbacks you found
in more detail.

Kind Regards,
Martine

2017-11-23 15:24 GMT+01:00 Gaëtan Harter :

> Dear RIOT developers,
>
> I would like to introduce some packaging concepts around RIOT.
> To consider modules like well defined distribution packages and not only
> source files added to an application firmware. This would allow software
> architecture/configuration informations to be parseable by external tools
> without reading source code and without an application context.
>
> My need is allowing a graphical interface to describe, configure
> applications with possibly external modules and generate the Makefile.
>
> I want to start a task force on this in two weeks after some preliminary
> discussions on the concepts.
> Current state
>
> Some of these module information is already present but not in a easy to
> get way so it does not fit my need.
>
> To highlight some of the problems:
>
>- it is scattered in different centralized files
>   - Makes it hard to add external modules
>   - Needs different files to get one module metadata
>- some is defined in a Turing complete language
>   - Hard to parse
>   - only available in a compilation context
>- some information/configuration only in source files
>
> Implementation
>
> I am thinking about adding declaratively defined information/metadata on
> modules distributed to each module directory. This could be achieved by
> changing to a new build system with these features already included.
> However, integration into the current build system would be preferred in
> order to be able
> to incrementally integrate, test and document new changes.
> Roadmap
>
>- Initial feedback period of two weeks
>- Define and break down intermediate tasks
>- Iteration cycles on each sub task and integration
>
> Further steps
>
> I would like to gather feedback on the concept here, and find interested
> people to start working on it, for reviewing, advices, giving their
> constraints and implementing if you have time.
>
> I will give a two weeks time limit for the initial review until I start
> working onthis.
>
> Thank you in advance.
> Cheers,
> Gaëtan
>
> ___
> devel mailing list
> devel@riot-os.org
> https://lists.riot-os.org/mailman/listinfo/devel
>
>
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel

[riot-devel] Updates to the build system - modules definition

[Gaëtan Harter]

Dear RIOT developers,

I would like to introduce some packaging concepts around RIOT.
To consider modules like well defined distribution packages and not only 
source files added to an application firmware. This would allow software 
architecture/configuration informations to be parseable by external 
tools without reading source code and without an application context.


My need is allowing a graphical interface to describe, configure 
applications with possibly external modules and generate the Makefile.


I want to start a task force on this in two weeks after some preliminary 
discussions on the concepts.



 Current state

Some of these module information is already present but not in a easy to 
get way so it does not fit my need.


To highlight some of the problems:

 * it is scattered in different centralized files
 o Makes it hard to add external modules
 o Needs different files to get one module metadata
 * some is defined in a Turing complete language
 o Hard to parse
 o only available in a compilation context
 * some information/configuration only in source files


 Implementation

I am thinking about adding declaratively defined information/metadata on 
modules distributed to each module directory. This could be achieved by
changing to a new build system with these features already included. 
However, integration into the current build system would be preferred in 
order to be able

to incrementally integrate, test and document new changes.


   Roadmap

 * Initial feedback period of two weeks
 * Define and break down intermediate tasks
 * Iteration cycles on each sub task and integration


 Further steps

I would like to gather feedback on the concept here, and find interested 
people to start working on it, for reviewing, advices, giving their 
constraints and implementing if you have time.


I will give a two weeks time limit for the initial review until I start 
working onthis.


Thank you in advance.

Cheers,
Gaëtan
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel