Re: [xpcc-dev] Towards modm

2016-02-05 Thread Fabian Greif
Hi,

> Fabian is back!

\o/

>> On 04/02/16 14:49, Fabian Greif wrote:
>>> To solve these issues a possible solution is extract the system
builder
>>> part of the SCons system (basically everything in /architecture) into
a
>>> separate tool and make the remaining parts more modular. One approach
>>> for
>>> this would be the library builder [1].
>> 
>> […]
>> Maybe you could suggest a list of modules, to start a discussion about
>> the amount of modularity.
> 
> Let’s do this step by step and build a prototype first.
> The means doing xpcc/architecture or xpcc/architecture/platform first,
> then worrying about the rest later.
> At the very least, driver/core, driver/gpio and perhaps driver/uart.
> 
> The complicated stuff is the code generation especially with parameters.
> If that works well, we can apply the lessons learnt to other modules.

Agreed.

Nonetheless I think it is useful to have the code generation part as a
separate bundle of python functions/classes. Those can be then used by the
library builder to create the actual library. But it should make testing
easier if it is possible to invoke them separately.

>> I would like to add scons support to the library builder.
>> Could you point be to a small example project to prototype that?
>> Is there a plugin API for build systems?
>> And btw. what build systems do you want to support? CMake? Make?
> 
> If I understand the library builder source code correctly, it will only
> copy files and templates, it will not actually build them.
> So lbuild is completely build system agnostic, right?

Yes. The build process should later on be as simple as:

env = Environment(...)
env.Program('project', env.Glob('src/*.cpp'))

I already extracted the SCons part in xpcc and decoupled it from the
system building part for a different (work) project. This should become its
own library, independent from xpcc and modm.

>>> The device driver generation part of SCons would be extracted into a
>>> separate tool which generates the content of the architecture folder.
>>> This
>>> will be the biggest amount of work. This tool will then be used by the
>>> library builder to combine the architecture stuff with the other parts
>>> of
>>> the library.
>> 
>> Is every device driver going to be its own module? Or do you plan to
>> make a big `architecture` module?
>> We need to be careful not to have two different build systems: The
>> library builder for the modules and the device file code for the
>> `architecture` module. This would imho be redundant.
> 
> +1.
> The device files and driver files are essentially already modules.
> I don’t know enough of lbuild to say whether or not it can be
generalized.

Should be doable.
If it is not I'll talk to the guy that wrote lbuild ... oh, wait ;-)

>> In general I would recommend you go ahead and start migrating code for
a
>> single platform to the mbed repository so that we can start playing
>> around with your library builder solution and so that we have concrete
>> examples to discuss.
> 
> +1.
> May I suggest STM32F407? It is the most well tested, most understood
> platform we have.
> Also not a simple as an AVR, so we might spot design problems earlier.

Ok.

>> start migrating code […] to the mbed repository
> 
> Thewhatnow?!?
> I think I’m way ahead of you on that one ;-)

Lets just use the modm repository for now :-)

Cheers,
Fabian

___
xpcc-dev mailing list
xpcc-dev@lists.rwth-aachen.de
http://mailman.rwth-aachen.de/mailman/listinfo/xpcc-dev

Re: [xpcc-dev] Towards modm

2016-02-04 Thread Niklas Hauser
Fabian is back!

> On 04/02/16 14:49, Fabian Greif wrote:
>> To solve these issues a possible solution is extract the system builder
>> part of the SCons system (basically everything in /architecture) into a
>> separate tool and make the remaining parts more modular. One approach for
>> this would be the library builder [1].
> 
> […]
> Maybe you could suggest a list of modules, to start a discussion about the 
> amount of modularity.

Let’s do this step by step and build a prototype first.
The means doing xpcc/architecture or xpcc/architecture/platform first, then 
worrying about the rest later.
At the very least, driver/core, driver/gpio and perhaps driver/uart.

The complicated stuff is the code generation especially with parameters.
If that works well, we can apply the lessons learnt to other modules.


> I would like to add scons support to the library builder.
> Could you point be to a small example project to prototype that?
> Is there a plugin API for build systems?
> And btw. what build systems do you want to support? CMake? Make?

If I understand the library builder source code correctly, it will only copy 
files and templates, it will not actually build them.
So lbuild is completely build system agnostic, right?


>> I would start to
>> migrating code from the xpcc repository over, separating it into
>> independent modules.

+1.
Anything we can review and learn from.


>> The device driver generation part of SCons would be extracted into a
>> separate tool which generates the content of the architecture folder. This
>> will be the biggest amount of work. This tool will then be used by the
>> library builder to combine the architecture stuff with the other parts of
>> the library.
> 
> Is every device driver going to be its own module? Or do you plan to make a 
> big `architecture` module?
> We need to be careful not to have two different build systems: The library 
> builder for the modules and the device file code for the `architecture` 
> module. This would imho be redundant.

+1.
The device files and driver files are essentially already modules.
I don’t know enough of lbuild to say whether or not it can be generalized.


> In general I would recommend you go ahead and start migrating code for a 
> single platform to the mbed repository so that we can start playing around 
> with your library builder solution and so that we have concrete examples to 
> discuss.

+1.
May I suggest STM32F407? It is the most well tested, most understood platform 
we have.
Also not a simple as an AVR, so we might spot design problems earlier.


> start migrating code […] to the mbed repository


Thewhatnow?!?
I think I’m way ahead of you on that one ;-)


Cheers,
Niklas


___
xpcc-dev mailing list
xpcc-dev@lists.rwth-aachen.de
http://mailman.rwth-aachen.de/mailman/listinfo/xpcc-dev


Re: [xpcc-dev] Towards modm

2016-02-04 Thread Kevin Laeufer

Hi Fabian,

thank you for your efforts.

On 04/02/16 14:49, Fabian Greif wrote:

To solve these issues a possible solution is extract the system builder
part of the SCons system (basically everything in /architecture) into a
separate tool and make the remaining parts more modular. One approach for
this would be the library builder [1].


I have one question about this:
How fine grained do you want to make the individual modules?
If we make them to big, we loose flexibility, if we make them to small, 
we could end up with a dependency hell.


Generally, I think, we need to define a core system, that is available 
on all platforms and that all other modules can depend on. This could be 
a subset of the STL library and a module that contains the core xpcc 
functionality like the timer and the clock.
When we do that, I would like to move our Container API to a STL 
compatible version.


But how about the other modules? Do we add one math module? Or do we put 
e.g. the Quaternion stuff into its own module?


Maybe you could suggest a list of modules, to start a discussion about 
the amount of modularity.



The disadvantage mainly comes for the library developer, because the
generation of the library becomes a separate step. For most projects you do
this once at the beginning, after you selected your target architecture,
but for library development you generate the library all the time. It may
require a different workflow, by developing a new feature within a project,
and only after it is finished moving it into the library.


I would like to add scons support to the library builder.
Could you point be to a small example project to prototype that?
Is there a plugin API for build systems?
And btw. what build systems do you want to support? CMake? Make?




So how to start the work on this:
Niklas already reserved modm-io on GitHub (modm stands for "Modular
Object-Oriented Development for Microcontrollers") to resolve the naming
issues between xpcc (the protocol) and xpcc (the library). I would start to
migrating code from the xpcc repository over, separating it into
independent modules. This requires a lot of structural changes which makes
it difficult to automatically merge changes from xpcc to modm.
The device driver generation part of SCons would be extracted into a
separate tool which generates the content of the architecture folder. This
will be the biggest amount of work. This tool will then be used by the
library builder to combine the architecture stuff with the other parts of
the library.



Is every device driver going to be its own module? Or do you plan to 
make a big `architecture` module?
We need to be careful not to have two different build systems: The 
library builder for the modules and the device file code for the 
`architecture` module. This would imho be redundant.




In general I would recommend you go ahead and start migrating code for a 
single platform to the mbed repository so that we can start playing 
around with your library builder solution and so that we have concrete 
examples to discuss.


Best regards,

Kevin
___
xpcc-dev mailing list
xpcc-dev@lists.rwth-aachen.de
http://mailman.rwth-aachen.de/mailman/listinfo/xpcc-dev


[xpcc-dev] Towards modm

2016-02-04 Thread Fabian Greif
Hi,

during the last developer meeting we had some discussions about the next
steps for xpcc.

In my opinion one of the main problems in using xpcc for bigger projects
is its build system. Although I like SCons and and implemented its
beginning in xpcc, it has grown to much into a system builder. That is fine
as long as xpcc is your primary platform, but makes it really awkward in
cases where it is not. It also makes it difficult to full integrate it into
IDEs like Eclipse, because the indexer has some problems deciding which
file is used and which isn't.

Another problem off xpcc is its size in combination with the release
management. A fresh checkout is 77 MB at the moment, without the Git
history its still 43 MB. In general that is not that much, but becomes
annoying when you have a lot of small projects which you need to support in
with different versions. Without a useful version schema, each project
needs its own separate copy of the library. It is not possible/to much work
to keep all projects always up-to-date to the most recent version of xpcc.
Yet still new features are only added to newer versions, therefore it is
also not useful to freeze an older version for all projects.

To solve these issues a possible solution is extract the system builder
part of the SCons system (basically everything in /architecture) into a
separate tool and make the remaining parts more modular. One approach for
this would be the library builder [1]. With this, a project would no longer
be build against the complete library, but a subset of the library would be
bundled together, containing only the needed and applicable parts for the
project/architecture. E.g. a LPC11 application does not need all the
STM32F.. library code. This greatly reduces the complexity of the generated
source tree, which allows for an easy integration with IDEs and build
systems other than SCons.

The disadvantage mainly comes for the library developer, because the
generation of the library becomes a separate step. For most projects you do
this once at the beginning, after you selected your target architecture,
but for library development you generate the library all the time. It may
require a different workflow, by developing a new feature within a project,
and only after it is finished moving it into the library.

So how to start the work on this:
Niklas already reserved modm-io on GitHub (modm stands for "Modular
Object-Oriented Development for Microcontrollers") to resolve the naming
issues between xpcc (the protocol) and xpcc (the library). I would start to
migrating code from the xpcc repository over, separating it into
independent modules. This requires a lot of structural changes which makes
it difficult to automatically merge changes from xpcc to modm.
The device driver generation part of SCons would be extracted into a
separate tool which generates the content of the architecture folder. This
will be the biggest amount of work. This tool will then be used by the
library builder to combine the architecture stuff with the other parts of
the library.

So far the idea, any comments/other suggestions on that?

Cheers,
Fabian

[1] https://github.com/dergraaf/library-builder
___
xpcc-dev mailing list
xpcc-dev@lists.rwth-aachen.de
http://mailman.rwth-aachen.de/mailman/listinfo/xpcc-dev