Re: CI tests (was: Re: [OT] Learning Makefiles)
Why not start the test infrastructure from sim/qemu? It's more simple to set up and has unlimited resources. Once the sim/qemu test workflow is ready, it isn't very hard to duplicate to the real boards. On Tue, May 23, 2023 at 8:42 AM Alan C. Assis wrote: > On 5/22/23, Tomek CEDRO wrote: > > This is why I asked not that long ago about > > Software-Hardware-Support-Compatibility-Matrix.. this would be really > > big table with hardware boards in columns and features in rows with > > green marks (or +1) where full support is confirmed, yellow (or 0) > > meaning work-in-progress, red (or -1) meaning no support or known > > problems. > > > > According to that Compatibility Matrix it would be possible to create > > proof-based configurations to build, and builds would prove the > > configurations. > > > > To be honest I have no idea how that could be implemented in such a > > complex project as NuttX with all those possible configurations.. that > > would really require big CI automation and I am not really familiar > > with GH CI yet maybe this is possible.. does GH charge $ for this CI > > operations? :-) > > > > When working for ARM at mbed they had this big wall of boards and such > > tests were performed not only at build stage but also on a real > > hardware.. each board had DAPLink that allowed flashing and serial > > port shell that executed some test scripts :-) > > > > Yes, I and Sebastien tried to create a testing farm for NuttX using > Raspberry Pi: > > https://bitbucket.org/acassis/raspi-nuttx-farm/src/master/ > > but soon we realize it will not scale well, for each board we need a > Raspi, or a USB HUB with Power Control over on each port (to > physically turn ON/OFF the board). > > In the past using Raspberry Pi Zero was a good idea: > https://www.raspberrypi.com/news/raspberry-pi-zero/ > The price was U$ 5.00, so we could by 100 that it was not too expensive :-) > > Maybe a better alternative should be create some USB/HUB board using > ESP32-S2 that we could use as bridge to program from a central > computer/board over WiFi. > > BR, > > Alan >
Re: [OT] Learning Makefiles
On Tue, May 23, 2023 at 4:28 AM Xiang Xiao wrote: > CWIKI mayn't be a good place since it requires that the user has an Apache > account at least to make any change as far as I know. > It's better to be tracked by a github issue. That is more to keep RFC and administrative stuff documented.. anyone from PMC can update.. I have created a content from out mailing list discussion.. but anything that works best for everyone is fine.. can be GH Issues sure thing :-) -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
Re: [OT] Learning Makefiles
CWIKI mayn't be a good place since it requires that the user has an Apache account at least to make any change as far as I know. It's better to be tracked by a github issue. On Tue, May 23, 2023 at 9:27 AM Nathan Hartman wrote: > On Mon, May 22, 2023 at 8:44 PM Alan C. Assis wrote: > > > On 5/22/23, Tomek CEDRO wrote: > > > On Tue, May 23, 2023 at 1:55 AM Alan C. Assis wrote: > > >> I think it is better to keep the documentation in a single place: > > >> https://nuttx.apache.org/docs/latest/contributing/index.html > > >> We're moving those documentations from confluence to our internal > > >> repository. > > >> So, that could be nice if you could send patches to Documentation/ > > >> directory directly. > > > > > > Yup, cwiki at this moment is a kind of scratchpad, all documentation > > > is being transferred to git / documentation.. but we may still use > > > cwiki to keep project administrative information like notes, rfc, > > > etc..? > > > > > > > No, I don't think so. Please search in the mailing list about this > > moving docs discussion. > > > > Docs should move to repo, yes, BUT if I understand correctly Tomek is > talking about things like ongoing discussions, debates within the > community, such as arguments for and against adopting CMake. That is not > documentation for users of NuttX. That is a place for the community to > collect all the arguments in a debate so they will be easy to reference in > one place and easy to see if there are more pros than cons, or more cons > than pros. That should be able to go in the CWIKI and not docs in the repo, > IMHO. > > Cheers > Nathan >
Re: [OT] Learning Makefiles
On Mon, May 22, 2023 at 8:44 PM Alan C. Assis wrote: > On 5/22/23, Tomek CEDRO wrote: > > On Tue, May 23, 2023 at 1:55 AM Alan C. Assis wrote: > >> I think it is better to keep the documentation in a single place: > >> https://nuttx.apache.org/docs/latest/contributing/index.html > >> We're moving those documentations from confluence to our internal > >> repository. > >> So, that could be nice if you could send patches to Documentation/ > >> directory directly. > > > > Yup, cwiki at this moment is a kind of scratchpad, all documentation > > is being transferred to git / documentation.. but we may still use > > cwiki to keep project administrative information like notes, rfc, > > etc..? > > > > No, I don't think so. Please search in the mailing list about this > moving docs discussion. Docs should move to repo, yes, BUT if I understand correctly Tomek is talking about things like ongoing discussions, debates within the community, such as arguments for and against adopting CMake. That is not documentation for users of NuttX. That is a place for the community to collect all the arguments in a debate so they will be easy to reference in one place and easy to see if there are more pros than cons, or more cons than pros. That should be able to go in the CWIKI and not docs in the repo, IMHO. Cheers Nathan
Re: [OT] Learning Makefiles
On 5/22/23, Tomek CEDRO wrote: > On Tue, May 23, 2023 at 1:55 AM Alan C. Assis wrote: >> I think it is better to keep the documentation in a single place: >> https://nuttx.apache.org/docs/latest/contributing/index.html >> We're moving those documentations from confluence to our internal >> repository. >> So, that could be nice if you could send patches to Documentation/ >> directory directly. > > Yup, cwiki at this moment is a kind of scratchpad, all documentation > is being transferred to git / documentation.. but we may still use > cwiki to keep project administrative information like notes, rfc, > etc..? > No, I don't think so. Please search in the mailing list about this moving docs discussion. BR, Alan
Re: CI tests (was: Re: [OT] Learning Makefiles)
On 5/22/23, Tomek CEDRO wrote: > This is why I asked not that long ago about > Software-Hardware-Support-Compatibility-Matrix.. this would be really > big table with hardware boards in columns and features in rows with > green marks (or +1) where full support is confirmed, yellow (or 0) > meaning work-in-progress, red (or -1) meaning no support or known > problems. > > According to that Compatibility Matrix it would be possible to create > proof-based configurations to build, and builds would prove the > configurations. > > To be honest I have no idea how that could be implemented in such a > complex project as NuttX with all those possible configurations.. that > would really require big CI automation and I am not really familiar > with GH CI yet maybe this is possible.. does GH charge $ for this CI > operations? :-) > > When working for ARM at mbed they had this big wall of boards and such > tests were performed not only at build stage but also on a real > hardware.. each board had DAPLink that allowed flashing and serial > port shell that executed some test scripts :-) > Yes, I and Sebastien tried to create a testing farm for NuttX using Raspberry Pi: https://bitbucket.org/acassis/raspi-nuttx-farm/src/master/ but soon we realize it will not scale well, for each board we need a Raspi, or a USB HUB with Power Control over on each port (to physically turn ON/OFF the board). In the past using Raspberry Pi Zero was a good idea: https://www.raspberrypi.com/news/raspberry-pi-zero/ The price was U$ 5.00, so we could by 100 that it was not too expensive :-) Maybe a better alternative should be create some USB/HUB board using ESP32-S2 that we could use as bridge to program from a central computer/board over WiFi. BR, Alan
Re: [OT] Learning Makefiles
On Tue, May 23, 2023 at 1:55 AM Alan C. Assis wrote: > I think it is better to keep the documentation in a single place: > https://nuttx.apache.org/docs/latest/contributing/index.html > We're moving those documentations from confluence to our internal repository. > So, that could be nice if you could send patches to Documentation/ > directory directly. Yup, cwiki at this moment is a kind of scratchpad, all documentation is being transferred to git / documentation.. but we may still use cwiki to keep project administrative information like notes, rfc, etc..? -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
Re: [OT] Learning Makefiles
On 5/22/23, Tomek CEDRO wrote: > On Sat, May 20, 2023 at 2:12 AM Brennan Ashton wrote: >> On Fri, May 19, 2023, 3:23 PM Tomek CEDRO wrote: >> > I am thinking about this. "If it works don't fix it" comes to my mind. >> > Current build system is amazingly simple coherent and fast. Building >> > firmware takes 17 seconds. Why change it? >> > Such change will flip everything upside down. Adds lots of work and >> > even more possible problems. >> > What would be the real benefit? >> > How would it improve that 17 seconds? >> > I think the practical presentation and comparison of results (i.e. ide >> > integration, ci automation) along with numbers (i.e. build time) needs >> > to take place before making any decisions. >> >> I very much disagree. The current build system is much slower with a ton >> of >> boilerplate that is difficult to maintain and make cross platform. Are we >> building an RTOS or reinventing the modern build tools? >> >> Infact I have pull requests unmerged because the hacks we have in place >> break and I have very little interest in tacking on even more hacks when >> we >> could just not worry about it with cmake. >> >> I would encourage those that thing this complicates things to actually >> look >> at the pull requests for cmake and see how much cleaner it is. > > Thanks Brennan, this is a solid pro for cmake, we need to create > detailed list pros and cons, verify in practice, to get better proof > based RFC analysis. > > I can see that this decision will be quite controversial and it cannot > lead to any more split in the community. > > I have created a simple page at CWIKI with an initial list of impacts, > pros, cons. Please update folks. I have updated Proposals name to > NuttX RFC Proposals. Big changes like that can be processed that way > as RFC and documented on cwiki. > > https://cwiki.apache.org/confluence/display/NUTTX/NuttX+RFC+0004%3A+Add+CMake+build+system > I think it is better to keep the documentation in a single place: https://nuttx.apache.org/docs/latest/contributing/index.html We're moving those documentations from confluence to our internal repository. So, that could be nice if you could send patches to Documentation/ directory directly. BR, Alan
Re: [OT] Learning Makefiles
On Sat, May 20, 2023 at 2:12 AM Brennan Ashton wrote: > On Fri, May 19, 2023, 3:23 PM Tomek CEDRO wrote: > > I am thinking about this. "If it works don't fix it" comes to my mind. > > Current build system is amazingly simple coherent and fast. Building > > firmware takes 17 seconds. Why change it? > > Such change will flip everything upside down. Adds lots of work and > > even more possible problems. > > What would be the real benefit? > > How would it improve that 17 seconds? > > I think the practical presentation and comparison of results (i.e. ide > > integration, ci automation) along with numbers (i.e. build time) needs > > to take place before making any decisions. > > I very much disagree. The current build system is much slower with a ton of > boilerplate that is difficult to maintain and make cross platform. Are we > building an RTOS or reinventing the modern build tools? > > Infact I have pull requests unmerged because the hacks we have in place > break and I have very little interest in tacking on even more hacks when we > could just not worry about it with cmake. > > I would encourage those that thing this complicates things to actually look > at the pull requests for cmake and see how much cleaner it is. Thanks Brennan, this is a solid pro for cmake, we need to create detailed list pros and cons, verify in practice, to get better proof based RFC analysis. I can see that this decision will be quite controversial and it cannot lead to any more split in the community. I have created a simple page at CWIKI with an initial list of impacts, pros, cons. Please update folks. I have updated Proposals name to NuttX RFC Proposals. Big changes like that can be processed that way as RFC and documented on cwiki. https://cwiki.apache.org/confluence/display/NUTTX/NuttX+RFC+0004%3A+Add+CMake+build+system -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
Re: CI tests (was: Re: [OT] Learning Makefiles)
This is why I asked not that long ago about Software-Hardware-Support-Compatibility-Matrix.. this would be really big table with hardware boards in columns and features in rows with green marks (or +1) where full support is confirmed, yellow (or 0) meaning work-in-progress, red (or -1) meaning no support or known problems. According to that Compatibility Matrix it would be possible to create proof-based configurations to build, and builds would prove the configurations. To be honest I have no idea how that could be implemented in such a complex project as NuttX with all those possible configurations.. that would really require big CI automation and I am not really familiar with GH CI yet maybe this is possible.. does GH charge $ for this CI operations? :-) When working for ARM at mbed they had this big wall of boards and such tests were performed not only at build stage but also on a real hardware.. each board had DAPLink that allowed flashing and serial port shell that executed some test scripts :-) -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
Re: CI tests (was: Re: [OT] Learning Makefiles)
Checking different configurations is an academic problem, I think they call it configuration sampling and it is part of variability modelling. There were some papers about sampling of Linux configurations. The simplest approach is to enable all possible, disable all possible, but it is not trivial. Each selection multiplies the number of configurations by the number of available options. That has very bad complexity. They use SAT solvers to generate many configurations instead of brute force. The goal is to sample configuration space in a uniform way. Am Mo., 22. Mai 2023 um 21:14 Uhr schrieb Nathan Hartman < hartman.nat...@gmail.com>: > On Mon, May 22, 2023 at 9:29 AM Sebastien Lorquet > wrote: > > > > > If the untold reason is to speed up github tests, then run less tests. > > Do we really need to test build on 13 or 20 arm platforms when only one > > config of the other architectures is tested, and the actual value of > > these build test is dubious? > > > > This is an interesting point. It reminds me that (at least in the old days, > I don't know now) FreeBSD had a build config that basically enabled all > options, even if that's impossible for actually running, for build testing. > I don't know if we can do that but maybe we need one ARM config that > enables as many options as possible and then use other archs for other > tests. > > Just a thought > > Nathan >
CI tests (was: Re: [OT] Learning Makefiles)
On Mon, May 22, 2023 at 9:29 AM Sebastien Lorquet wrote: > > If the untold reason is to speed up github tests, then run less tests. > Do we really need to test build on 13 or 20 arm platforms when only one > config of the other architectures is tested, and the actual value of > these build test is dubious? This is an interesting point. It reminds me that (at least in the old days, I don't know now) FreeBSD had a build config that basically enabled all options, even if that's impossible for actually running, for build testing. I don't know if we can do that but maybe we need one ARM config that enables as many options as possible and then use other archs for other tests. Just a thought Nathan
Re: esp32 custom board gpio and ws2812 problem
On Mon, May 22, 2023 at 5:02 PM Alan C. Assis wrote: > Just a update from Espressif: probably ESP32-C5 will have USB OTG, but > since it was not released yet we cannot confirm it (things can change > before releasing the chip). Thank You Alan!! :-) -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
Re: esp32 custom board gpio and ws2812 problem
Hi Tomek, Just a update from Espressif: probably ESP32-C5 will have USB OTG, but since it was not released yet we cannot confirm it (things can change before releasing the chip). They suggested that companies interested on get this information should ask directly to the business support: sa...@espressif.com BR, Alan On 5/20/23, Tomek CEDRO wrote: > Well all ESP32-C3 boards that I have (DevKit, M5) have UART-to-USB > converter so no JTAG so far.. I will order Beetle ESP32-C3 this one > should have direct USB/UART/JTAG pins on the USB-C connector :-) > > I have ordered ESP32-S2 as it has USB OTG. I really miss RISC-V MCU > with USB Device/OTG.. ESP32-C6 would be perfect if it only had this > (it has WiFi + BLE + Mesh already) :-) > > Is there any chance that Espressif releases ESP32-C6 + USB OTG anytime soon? > :-) > > -- > CeDeROM, SQ7MHZ, http://www.tomek.cedro.info >
Re: [OT] Learning Makefiles
Hello,
Build performance is not an argument. it's already very fast.
Integration with various tools is a case of "I dont like it", it's not a
good reason for such a fundamental change, it can be resolved with local
configurations.
I cant find any good reason to change except, maybe, complexity, with
reserves. Again possibly in the "NIH"or "dont like it" category.
One more con: we add a dependency on yet another external tool.
As we say in french it's a hammer to crush a fly.
When a vote happen, my vote will be NO. I know it's not binding so it's
not counted, but I dont care. I'm against this change.
If the untold reason is to speed up github tests, then run less tests.
Do we really need to test build on 13 or 20 arm platforms when only one
config of the other architectures is tested, and the actual value of
these build test is dubious?
Sebastien
Le 22/05/2023 à 14:57, Alan C. Assis a écrit :
Hi Sebastien,
There are good cons and pros arguments for moving to CMake.
Just like many here I don' t have preference for one or another, but
we need to analyze what is better for NuttX evolution and make a good
decision.
The main pros of moving to CMake:
1) It is easier to integrate with new projects and IDEs
2) It will speed up the compilation of the building system
3) It will avoid cumbersome integration to make things work (see
Brennan comments)
The main cons of moving to CMake:
1) The current building system already works "fine" ("If it is not
broken, don't fix", comfort zone)
2) It could spend time and energy to maintain two building systems
during the transition period
3) People will need to learn a new too, although CMake seems easy at
first look, it is hard when you want to do something different.
I think I depicted the three more important point, case someone else
thing about some other important point of each side, please bring it
to the table.
BR,
Alan
On 5/22/23, Sebastien Lorquet wrote:
I very much agree with all of these arguments.
Thats a too large disruption for too little benefits.
I dont want to be forced to use cmake.
Everything we use here to integrate NuttX is based on makefiles.
Why do we have to bring in yet another dependency? No, cmake is not
installed in our build systems.
Sebastien
Le 20/05/2023 à 00:23, Tomek CEDRO a écrit :
I am thinking about this. "If it works don't fix it" comes to my mind.
Current build system is amazingly simple coherent and fast. Building
firmware takes 17 seconds. Why change it?
Such change will flip everything upside down. Adds lots of work and
even more possible problems.
What would be the real benefit?
How would it improve that 17 seconds?
I think the practical presentation and comparison of results (i.e. ide
integration, ci automation) along with numbers (i.e. build time) needs
to take place before making any decisions.
Re: [OT] Learning Makefiles
Hi Sebastien,
There are good cons and pros arguments for moving to CMake.
Just like many here I don' t have preference for one or another, but
we need to analyze what is better for NuttX evolution and make a good
decision.
The main pros of moving to CMake:
1) It is easier to integrate with new projects and IDEs
2) It will speed up the compilation of the building system
3) It will avoid cumbersome integration to make things work (see
Brennan comments)
The main cons of moving to CMake:
1) The current building system already works "fine" ("If it is not
broken, don't fix", comfort zone)
2) It could spend time and energy to maintain two building systems
during the transition period
3) People will need to learn a new too, although CMake seems easy at
first look, it is hard when you want to do something different.
I think I depicted the three more important point, case someone else
thing about some other important point of each side, please bring it
to the table.
BR,
Alan
On 5/22/23, Sebastien Lorquet wrote:
> I very much agree with all of these arguments.
>
> Thats a too large disruption for too little benefits.
>
> I dont want to be forced to use cmake.
>
> Everything we use here to integrate NuttX is based on makefiles.
>
> Why do we have to bring in yet another dependency? No, cmake is not
> installed in our build systems.
>
> Sebastien
>
>
> Le 20/05/2023 à 00:23, Tomek CEDRO a écrit :
>> I am thinking about this. "If it works don't fix it" comes to my mind.
>>
>> Current build system is amazingly simple coherent and fast. Building
>> firmware takes 17 seconds. Why change it?
>>
>> Such change will flip everything upside down. Adds lots of work and
>> even more possible problems.
>>
>> What would be the real benefit?
>>
>> How would it improve that 17 seconds?
>>
>> I think the practical presentation and comparison of results (i.e. ide
>> integration, ci automation) along with numbers (i.e. build time) needs
>> to take place before making any decisions.
>>
>
Re: STM32 SmartCard mode happening right now!
Hi, First of all I need reliable communication with a particular brand of card I cant disclose, and I dont think any certification is on the scope :) But EMV certification is probably possible with a bit more dedication. Usually the complexities are around timeouts, error behaviour, and a lot of idiosyncrasies. You could help by porting to your platform, and testing the code that will be common to all platforms. I believe implementing the pcsclite de-facto standard would benefit a wider community, since it would allow interoperability with linux programs that use smartcards? For the moment I am focused on uart smartcard support only. Thanks for your interest, Sebastien Le 17/05/2023 à 16:17, Michael Jung a écrit : Hi Sebastien, one of the projects I am working on requires an EMV 4.3d level 1 compliant smartcard reader on an STM32U5. So I would be very interested in the work you are planning to do. I have a lot of background knowledge on EVM compliant contactless card readers, but unfortunately not so for the contact cards. We do have people in the team that do have this knowledge, though. Do you think your approach would allow to get EMV 4.3d certification? Would it be possible to get early access to your code? Could I help you in developing this? Thanks! Michael On Tue, May 9, 2023 at 7:13 PM Sebastien Lorquet wrote: Hi, it is time! I am, right now, starting to implement the smart card mode for STM32 USART, starting with STM32H7, this will be portable to other STM32 chips I guess. The UART mode will just configure the uart for smartcard shenanigans, eg, put it in working order for this mode. Smart card contact protocols (T=0, T=1) will go into a user space app that manages readers, similar to pcsclite daemon in linux. So the smart card mode of the stm32 usarts is just an activation of the proper registers, that will : -wire the hardware as expected: enable the clock output, define one single full duplex IO line -reconfigure the uart character framing, including parity generation and detection, which is a fundamental mechanism, required for protocol compliance testing. The exchanges will still be made with the read() and write() primitives (protocol is a kind of IPC), so I am expecting to add just a few IOCTLs: -ENABLE smart card mode with a bool parameter to enable or disable, default disabled -START or STOP the clock signal (if possible, I did not dive in the RM yet) -Define the communication speed in terms of ETU (clock pulses per bit, which is different from the baud rate and the industry standard way to define smart card comm speeds). This need to be modifiable, the speed is negotiated dynamically at card insertion. This will return a ENOTTY error if a particular uart does not support Smart Card mode. However, to be future proof, another way is possible. USB CCID smart card readers are also a thing. In pcsclite (linux), these are managed via libusb from userspace. But we can do something special in NuttX, like defining a smart card "lower half" that can be implemented by both the stm32 uart in smart card mode, or by the future usb ccid driver. I do not plan to *port* the full pcsclite, but I will do something that tend to be compatible, while being much smaller. The great question is then how to manage the "upper half" of such drivers. If it was just me, it would be a character device, that would have IOCTLs to enumerate lower halves (eg card readers) so a user space app can dispatch commands to multiple readers. But I know NuttX despise character devices. So, I am going to need some inspiration here. Sockets do not seem appropriate to how things work in the smart card world, I can tell that from 13 years of professional experience :) If NuttX has a libusb-like protocol now or later, then we dont need to be concerned by this lower half and char dev, this can all happen in the management daemon. There are also modular card readers that use plain old serial protocols, these would also be handled from the user space daemon via specific drivers. So, this is the correct time to define this in a way that please everyone and will be useful for a long time. We need compromises between imperative needs of this mode and the NuttX way of doing things. The goal is to facilitate the upstreaming of this feature. I know smart cards are unusual and quite niche, you can ask me as many questions as needed to understand smart card context, behaviour, and habits. Thank you for your reading and comments Sebastien
Re: [OT] Learning Makefiles
I very much agree with all of these arguments. Thats a too large disruption for too little benefits. I dont want to be forced to use cmake. Everything we use here to integrate NuttX is based on makefiles. Why do we have to bring in yet another dependency? No, cmake is not installed in our build systems. Sebastien Le 20/05/2023 à 00:23, Tomek CEDRO a écrit : I am thinking about this. "If it works don't fix it" comes to my mind. Current build system is amazingly simple coherent and fast. Building firmware takes 17 seconds. Why change it? Such change will flip everything upside down. Adds lots of work and even more possible problems. What would be the real benefit? How would it improve that 17 seconds? I think the practical presentation and comparison of results (i.e. ide integration, ci automation) along with numbers (i.e. build time) needs to take place before making any decisions.
