Re: exFAT or other large options for file system storage.

[Xiang Xiao]

If you search exfat implementation: FatFS from
http://elm-chan.org/fsw/ff/00index_e.html is a good choice, since it's:

   1. has a better friend license than GPL/LGPL
   2. designed for small embedded device natively

BTW, if the community wants another fat implementation, we could upstream
NuttX FS wrapper, which has been used in the product for several years.

On Sat, Mar 23, 2024 at 3:51 AM Tomek CEDRO  wrote:

> On Fri, Mar 22, 2024 at 7:54 PM Alan C. Assis  wrote:
> > Hi Dmitri,
> > But I think even an open-source implementation will end-up paying
> > royalties, unless you are member of OIN:
> > https://www.paragon-software.com/exfat-license/
>
> Here is a good page on exFAT along with Legal Status and List of
> Implementations sections:
>
> https://en.wikipedia.org/wiki/ExFAT
>
> Looks like Paragon Software is a proprietary implementation too.
>
>
> OIN membership is something else and it seems free:
>
> https://openinventionnetwork.com/our-members/funding-members-and-licensees/
>
> According to wikipedia Apache is covered by OIN, not sure if this is
> about web server or all of the Apache Foundation projects? Maybe this
> is the way to get exFAT in NuttX?
>
> https://en.wikipedia.org/wiki/Open_Invention_Network
>
>
> > So, maybe you could think about some alternatives, like porting EXT3 to
> > NuttX (but that is not an easy task, since even FreeBSD doesn't support
> > EXT3).
>
> On FreeBSD we have lots of filesystems supported by FUSE and it works
> pretty well now. Its not a part of the system but user can install it
> from ports/packages. This is the implementation in use (GNU license):
>
> https://github.com/relan/exfat
>
> Here is a detailed filesystem comparison page:
>
> https://en.wikipedia.org/wiki/Comparison_of_file_systems
>
> Before ZFS was introduced to FreeBSD we used UFS(2) (Unix File System)
>
> https://en.wikipedia.org/wiki/Unix_File_System
>
>
> The most universal and open filesystem is UDF (POSIX compliant) but it
> varies in implementations and versions and no write support for all of
> the versions so its kinda messy and platform specific. It was (and
> probably still is) not really a good multiplatform replacement. I
> tried using it long time ago with FreeBSD, Linux and Windows as common
> filesystem on data partition for all of the OS on a single machine.
> But I sticked to EXT2. Now I simply use ZFS on FreeBSD. But ZFS even
> nowadays is still not a solution to work with a single data partition
> on various OS and definitely not an embedded solution. People simply
> used separate machine with NAS for storage over NFS. Idea for
> universal filesystem seems dead :-(
>
> UDF is widely used nowadays on top of ISO9660 to burn optical disks
> with large files (i.e. double layer DVD or BluRay). All systems can
> read it. Not all can write it on the fly (it is usually used to create
> static ISO+UDF image). But in theory it seems better than exFAT!
>
> https://en.wikipedia.org/wiki/Universal_Disk_Format
>
> I found some old GSoC 2012 page on UDF implementation in FreeBSD
> (based on NetBSD code):
>
> https://wiki.freebsd.org/SummerOfCode2012/UDFImplementation
>
>
> Having a modern universal filesystem support in NuttX seems important.
>
> Maybe UDF 2.60 implementation could happen?
>
> exFAT seems more popular. Maybe Apache could help here over the OIN?
> And most of the existing exFAT implementations are covered by GNU
> license so that may be no go for some companies / projects.
>
> And the standard question who is willing to invest time / cash for
> that implementation?
>
>
> UDF has some interesting features that may beneficial for instance
> when streaming video of backing up data to a memory for instance VAT /
> Virtual Allocation Table mode that is incremental write once, or
> Spared mode that is used to write to rewritable media. And there is a
> BSD licensed implementation already out there.
>
> Also when recording multimedia stream is it VERY important to choose
> proper container format. For instance MP4 must be properly closed in
> order to be opened - kind of nightmare of aviation recording when
> camera hangs/breaks in the middle of flight and then folks cannot even
> open a recording. MKV seems better choice. But (in)famous GoPro still
> uses MP4 ;-)
>
> --
> CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
>

Re: [EXT] Re: uart rpmsg driver compatibility

[Xiang Xiao]

We will upstream these patches in the next couple months, but you know the
linux review process is uncontrollable.

On Tue, Mar 12, 2024 at 9:28 PM Andre Heinemans 
wrote:

> Thanks! I got it to work on the imx8mp. Do you know if there are any plans
> to upstream this driver to mainline linux?
>
> Kind regards,
> Andre
>
> From: Bowen Wang 
> Sent: Tuesday, 12 March 2024 05:36
> To: dev@nuttx.apache.org
> Subject: [EXT] Re: uart rpmsg driver compatibility
>
> Hi,
>
> The attachments are the rpmsg-tty patches based on linux LTS 5.15.
> If you have any problems, feel free to contact me.
>
> Regards,
> Bowen Wang
>
> Xiang Xiao <mailto:xiaoxiang781...@gmail.com> 于2024年3月11日周一 23:43写道：
> On Mon, Mar 11, 2024 at 11:13 PM Andre Heinemans  andre.heinem...@nxp.com>
> wrote:
>
> > Hi,
> >
> > Does the NuttX uart_rpmsg.c driver have a Linux counterpart to interact
> > with?
>
>
> Yes, the old version is here:
>
> https://lore.kernel.org/lkml/CAH2Cfb87Wacgsh=xz9h9kgwygbkxnbdbcdj4w3ups2likbt...@mail.gmail.com/
>
>
> > I want to achieve a virtual uart connection through rpmsg on an imx8mp
> > between NuttX (m7) and Linux (a53).
> > The tty_rpmsg.c driver in mainline linux does not seem compatible as it
> > read and writes the raw data directly from the rpmsg buffers.
>
>
> The mainline version comes from ST developer, which lacks of the flow
> control and very easy to lose the data with the fast transaction.
>
>
> > Whereas the NuttX driver uses a struct ‘uart_rpmsg_write_s’ which
> contains
> > the raw data in one of its fields.
> >
> >
> We renewed the rpmsg tty driver on top of Linux 5.14 recently, which works
> perfectly with the NuttX mainline rpmsg_uart driver.
> Bowen could share the implementation tomorrow.
>
>
>
> > Kind regards,
> > Andre
> >
>

Re: uart rpmsg driver compatibility

[Xiang Xiao]

On Mon, Mar 11, 2024 at 11:13 PM Andre Heinemans 
wrote:

> Hi,
>
> Does the NuttX uart_rpmsg.c driver have a Linux counterpart to interact
> with?


Yes, the old version is here:
https://lore.kernel.org/lkml/CAH2Cfb87Wacgsh=xz9h9kgwygbkxnbdbcdj4w3ups2likbt...@mail.gmail.com/


> I want to achieve a virtual uart connection through rpmsg on an imx8mp
> between NuttX (m7) and Linux (a53).
> The tty_rpmsg.c driver in mainline linux does not seem compatible as it
> read and writes the raw data directly from the rpmsg buffers.


The mainline version comes from ST developer, which lacks of the flow
control and very easy to lose the data with the fast transaction.


> Whereas the NuttX driver uses a struct ‘uart_rpmsg_write_s’ which contains
> the raw data in one of its fields.
>
>
We renewed the rpmsg tty driver on top of Linux 5.14 recently, which works
perfectly with the NuttX mainline rpmsg_uart driver.
Bowen could share the implementation tomorrow.



> Kind regards,
> Andre
>

Re: littlefs BIOC_FLUSH in NAND flash

[Xiang Xiao]

BIOC_FLUSH is used to flush the cache which may exist in mtd driver to
guarantee the power-loss resilience. It's optional, so it's enough that
mtd_nand returns -ENOTTY.

On Sat, Feb 24, 2024 at 8:28 PM Saurav Pal  wrote:

> Hi all,
>
> I've been trying to develop a device simulator for the NAND flash (in the
> sim).
>
> A short note on what this does is create a virtual NAND flash (from RAM)
> and provide a driver, as well as a wrapper around the upper half (which for
> now, does nothing extra). This helps in creating records of
> reads/writes/erases (in a separate file) for testing and statistics
> purposes.
>
> Since littlefs is one of the supported filesystems on MTD devices, I tried
> to use it to test mount but it tries to use an ioctl with BIOC_FLUSH in the
> format process, which is not supported as far as code in
> drivers/mtd/mtd_nand.c go.
>
> *Should that be something that the NAND driver should support?*
>
> Here are the logs of that process, the first ioctl is for MTDIOC_GEOMETRY
> (which is supported), and second for BIOC_FLUSH (in *bold*):
>
> nuttx ➤ make && ./nuttx
> LD:  nuttx
> login: admin
> password:
> User Logged-in!
> nsh> ls /dev
> /dev:
>  console
>  gpio0
>  gpio1
>  gpio2
>  gpio3
>  loop
>  null
>  oneshot
>  ram0
>  ram1
>  ram2
>  zero
> nsh> mtddev
> Size:   2097152 B
>
> Pages:4096
> Blocks: 33
> Writing...Done
> Driver running
> nsh> ls /dev
> /dev:
>  console
>  gpio0
>  gpio1
>  gpio2
>  gpio3
>  loop
>  mtd
>  null
>  oneshot
>  ram0
>  ram1
>  ram2
>  zero
> nsh> mount -t littlefs -o forceformat /dev/mtd /hi
> Upper! ioctl
> Command: 1537 | BIOC_FLUSH: 1293
> Upper! bread
> Page 128
> Lower! rawread : Page 128
> Lower! rawread : Page 129
> Lower! rawread : Page 130
> Lower! rawread : Page 131
> Upper! erase
> Lower! eraseblock : Block 0
> Upper! bread
> Page 0
> Lower! rawread : Page 0
> Lower! rawread : Page 1
> Lower! rawread : Page 2
> Lower! rawread : Page 3
> Upper! bread
> Page 4
> Lower! rawread : Page 4
> Lower! rawread : Page 5
> Lower! rawread : Page 6
> Lower! rawread : Page 7
> Upper! bwrite
> Lower! rawwrite : Page 0
> Lower! rawwrite : Page 1
> Lower! rawwrite : Page 2
> Lower! rawwrite : Page 3
> Upper! ioctl
> *Command: 1293 | BIOC_FLUSH: 1293*
> Error Upper! ioctl | -25
> Upper! bread
> Page 0
> Lower! rawread : Page 0
> Lower! rawread : Page 1
> Lower! rawread : Page 2
> Lower! rawread : Page 3
> nx_mount: ERROR: Bind method failed: -28
> nsh: mount: mount failed: 28
> nsh>
>
> The logs with "Upper!" refer to the upper half of the driver being
> executed, and same for the lower, and the pages are numbered from the start
> of the device (ie. (block << LOG_N_BLOCKS_PER_PAGE) + page_in_block)).
>
> Regards,
> SP
>

Re: Device Discovery

[Xiang Xiao]

Even though PCs just have one udev daemon, I don't see why MCU needs
multiple daemon.

On Wed, Feb 21, 2024 at 2:07 PM Saurav Pal  wrote:

> Hi,
>
> Thanks for the clarification.
>
> Also, am I right in thinking it would increase performance if there was a
> single daemon working rather than multiple? (I am thinking in terms of
> context switches for tasks, even if, say, somehow the memory consumption
> remains the same).
>
> Regards,
> SP
>
> On Tue, Feb 20, 2024 at 10:13 PM Xiang Xiao 
> wrote:
>
> > Since all devices expose through vfs (/dev, /proc), udev could be
> > implemented through inotify API.
> >
> > On Tue, Feb 20, 2024 at 11:31 PM Alan C. Assis 
> wrote:
> >
> > > Hi Saurav,
> > >
> > > There is not something like a "udev" for NuttX, but there are
> > > daemons/services used to detect when a SDCard is plugged, when a USB
> > > device is attached, etc.
> > >
> > > We could add something like a lite "udev" for NuttX.
> > >
> > > Best Regards,
> > >
> > > Alan
> > >
> > > On 2/20/24, Saurav Pal  wrote:
> > > > Hi all,
> > > >
> > > > I am not able to understand how NuttX performs device discovery. I am
> > > > trying to look for something that is similar to "udev". Can someone
> > > please
> > > > help me understand?
> > > >
> > > > Thanks in advance.
> > > > Regards,
> > > > SP
> > > >
> > >
> >
>

Re: Assert macro causing problems when building Sqlite

[Xiang Xiao]

Here is the fix: https://github.com/apache/nuttx/pull/11734.
sqlite porting is also upstreamed:
https://github.com/apache/nuttx-apps/pull/2297

On Tue, Feb 20, 2024 at 3:17 PM Jernej Turnsek 
wrote:

> Hi,
>
> the same problem as new UNUSED macro was causing me problems when building
> external libraries, like LuaJIT, an assert macro is also causing problems
> when building Sqlite3 library. Sqlite code has a lot of assert statements
> which are usually excluded when NDEBUG is set. But when assert is defined
> as:
>
> #ifdef NDEBUG
> #  define assert(f) ((void)(1 || (f)))
> #  define VERIFY(f) assert(f)
> #else
>
> compiler throws undefined variable errors all the time. With simple (void)0
> definition there is no problem. Yet another reason to evaluate this change
> of UNUSED and assert macro?
>
> Jernej
>

Re: Device Discovery

[Xiang Xiao]

Since all devices expose through vfs (/dev, /proc), udev could be
implemented through inotify API.

On Tue, Feb 20, 2024 at 11:31 PM Alan C. Assis  wrote:

> Hi Saurav,
>
> There is not something like a "udev" for NuttX, but there are
> daemons/services used to detect when a SDCard is plugged, when a USB
> device is attached, etc.
>
> We could add something like a lite "udev" for NuttX.
>
> Best Regards,
>
> Alan
>
> On 2/20/24, Saurav Pal  wrote:
> > Hi all,
> >
> > I am not able to understand how NuttX performs device discovery. I am
> > trying to look for something that is similar to "udev". Can someone
> please
> > help me understand?
> >
> > Thanks in advance.
> > Regards,
> > SP
> >
>

Re: fcntl F_SETLK, F_SETLKW, F_GETLK

[Xiang Xiao]

here is the patch set: https://github.com/apache/nuttx/pull/11724

On Tue, Feb 20, 2024 at 3:29 PM Jernej Turnsek 
wrote:

> Cool. Is there a possibility to share your work via PM? This way I can test
> the solution on my side also.
> Jernej
>
> On Tue, Feb 20, 2024 at 8:23 AM Xiang Xiao 
> wrote:
>
> > We have implemented file lock to support the database engine like sqlite
> > and unqlite. The implementation will go upstream soon.
> >
> > On Tue, Feb 20, 2024 at 3:04 PM Jernej Turnsek  >
> > wrote:
> >
> > > Hi,
> > > I would like to use Sqlite on NuttX and I have managed to successfully
> > > build the Sqlite3 library. When running the library, I have noticed
> that
> > > Sqlite is using fcntl F_SETLK and F_SETLKW for locking and those are
> not
> > > implemented on NuttX. Is there a plan to implement this in the future?
> > >
> > > Regards,
> > > Jernej
> > >
> >
>

Re: fcntl F_SETLK, F_SETLKW, F_GETLK

[Xiang Xiao]

We have implemented file lock to support the database engine like sqlite
and unqlite. The implementation will go upstream soon.

On Tue, Feb 20, 2024 at 3:04 PM Jernej Turnsek 
wrote:

> Hi,
> I would like to use Sqlite on NuttX and I have managed to successfully
> build the Sqlite3 library. When running the library, I have noticed that
> Sqlite is using fcntl F_SETLK and F_SETLKW for locking and those are not
> implemented on NuttX. Is there a plan to implement this in the future?
>
> Regards,
> Jernej
>

Re: RPTUN questions

[Xiang Xiao]

On Tue, Feb 13, 2024 at 9:49 AM yfliu2008  wrote:

> Dear Xiang and Bowen,
>
>
>
>
> Hope you've had a pleasant spring festival.
>
>
>
>
> As for RPTUN, I have it basically work using share memory only,
> though not very efficient but it has less hardware dependencies. I even can
> mount master's file system and run programs from it on the remote
> node, as the remote node's capability is superset of that of the master
> node.
>
>
>
>
>
> Here I have some questions regarding RPTUN:
>
>
>
>
>
> How to judge if RPTUN state is ready? For example when can a remote node
> know it is time to mount a folder at master side? Should we refer to the
> "rproc.state" inside rptun device?
>

There are two concept about the readiness:
1.the virtio/rpmsg transport, which is defined by "device status":

https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html#x1-110001
2.The services on top of virtio/rpmsg, which is controlled by individual
service. For example, all rpmsgfs fs operation will block until the remote
side ready:

https://github.com/apache/nuttx/blob/master/fs/rpmsgfs/rpmsgfs_client.c#L278-L283

So, rproc.state just only means the transport is ready for communication.


>
>
> How does the lifecycle of RPTUN remote node look like?
>
>
it's defined by virtio spec:
https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html
https://www.kernel.org/doc/Documentation/remoteproc.txt


>
> What is the meaning for reset? is this for the remote side to gracefully
> restart itself? master side shall also reset the shared memory after
> getting ACK?
>
>
reset may happen in two situation:

   1. The remote reset by self
   2. The master request the remote reset

But it's very hard to restore the state the same as before resetting.


> What is the menaing for panic? is it to tell remote node that master side
> encountered panic situation and the remote node should live on their own
> from now on?
>
>
No, panic is used to notify the remote to dump the internal status and
shutdown self.


> How can master tell the remote to get prepared for shutdown? This gives
> chance for the remote node to flush any file system cache and notify the
> master.
>

using a new id of mailbox or IPI.


>
>
> How can master tell the remote to go standby or wake up?
>
>
the remote core should go to standby byself as before, the same interrupt
used for normal ipc can be reused to wake up the remote


> Is the lifecycle management mostly done by in-band communications?
> It feels natural to have in-band administrative communication channel for
> lifecycle management purposes, though for some rare cases additional
> mechanisms are needed at master side like kicking off or powering down the
> remote. However, I saw some samples uses inter-processor interrupts
> for reset or panic, which confused me a lot because some back-n-forth
> message exchanges are needed to handle lifecycle gracefully, this is
> improper and unnecessary for interrupts.
>
>
It depends on the hardware design. It's better to have the special hardware
design to notify the lifecycle(e.g. reset, low power) event, otherwise the
software on peer core need send the special IPI or mailbox IRQ manually.


>
>
>
>
> Regards,
> yf
>
>
>
>
>
>
>
>
>
>
>
>
> 

Re: Re: RPTUN usage

[Xiang Xiao]

So, please check the remote core call uart_rpmsg_init with isconsole=true,
and no other uart driver register as console.

On Wed, Feb 7, 2024 at 9:15 PM yfliu2008  wrote:

> Xiang.
>
>
>
> Thanks for that. Yes I can see hello echo from master to remote and vice
> versa as you suggested.
>
>
>
>
> Regards,
> yf
>
>
>
> Original
>
>
>
> From:"Xiang Xiao"< xiaoxiang781...@gmail.com ;
>
> Date:2024/2/7 20:09
>
> To:"dev"< dev@nuttx.apache.org ;
>
> Subject:Re: RPTUN usage
>
>
> You can try:
>
>1. echo hello  /dev/ttyRpmsg on one core
>2. and cat /dev/ttyRpmsg on another core
>
> to confirm rpmsg uart work as expected before testing with cu.
>
> On Wed, Feb 7, 2024 at 8:00 PM yfliu2008  wrote:
>
>  Xiang and Bowen,
> 
> 
> 
>  I noticed that I didn't initialize the call the "uart_rpmsg_init()"
>  properly thus last time "rpmsg-ttyRpmsg" end point didn't shown in the
>  "rpmsg dump" result. After fixing the call parameters, they shows out
> now:
> 
> 
>  master rpmsg dump /dev/rptun/remote
> 
>   
>Dump rpmsg info
> between cpu
>  (master: yes)master <== remote: 
>  
>  rpmsg vq RX:
>   
>   
>   
>  rpmsg vq TX:
>   
>   
>   
>   rpmsg ept list:  
> 
>   
>   
> 
>ept NS   
>   
>   
> 
>ept rpmsg-ttyRpmsg 
> 
>   
>   
>
>ept rpmsg-ping  
>   
>   
>  
>   rpmsg buffer list:  
> 
>   
>   
>
>RX buffer, total 8, pending 0   
> 
>   
>  
>TX buffer, total 8, pending 0
> 
>  The "echo hello  /dev/ttyRpmsg" command doesn't block any
> more,
>  however the "cu" command doesn't work yet. I am not sure what is
>  still missing. I think it is better to make uart-rpmsg work first
> before
>  try any other RPMsg device types. Please let me know if you have any
>  suggestions.
> 
> 
> 
>  Regards,
>  yf
> 
> 
> 
> 
>  Original
> 
> 
> 
>  From:"yfliu2008"< yfliu2...@qq.com ;
> 
>  Date:2024/2/7 19:20
> 
>  To:"dev"< dev@nuttx.apache.org ;
> 
>  Subject:RPTUN usage
> 
> 
> 
>  Xiang  Bowen,
> 
> 
> 
> 
>  Here I seemed have RPTUN running on the two cores of K230 device and
> I can
>  use "rpmsg dump" to show some basic information on both cores:
> 
> 
> 
>  master rpmsg dump /dev/rptun/remote
> 
>   
>Dump rpmsg info
> between cpu
>  (master: yes)master <== remote: 
>  
>  rpmsg vq RX:
>   
>   
>   
>  rpmsg vq TX:
>   
>   
>   
>   rpmsg ept list:  
> 
>   
>   
> 
>ept NS   
>   
>   
> 
>ept rpmsg-ping  
>   
>   
>  
>   rpmsg buffer list:  
> 
>   
>   
>
>RX buffer, total 8, pending 0   
> 
>   
>  
> 
>TX buffer, total 8, pending 0
> 
> 
>  remote rpmsg /dev/rptun/master 
>   
>   nsh:
> rpmsg:
>  missing required argument(s)  
>   
> 
>  remote rpmsg dump /dev/rptun/master
> 
>   
>
>  Dump rpmsg info between cpu (master: no)remote <== master: 
>  
>  rpmsg vq RX:
>   
>   
>   
>  rpmsg vq TX:
>   
>   
>   
>   rpmsg ept list:  
> 
>   
>   
> 
>ept NS   
>   
>   
> 
>   rpmsg buffer list:  
> 
>   
>   
>
>RX buffer, total 8, pending 0   
> 
>   
>  
>TX buffer, total 8, pending 0
> 
> 
> 
>  However, I don't know how to further check the RPTUN link yet. It
> seems
>  that "rpmsg ping /dev/rptun/remote 1 64 0 10" doesn't show anything.
> Also
>  "echo hello  /dev/ttyRpmsg" gets blocked.
> 
> 
> 
> 
> 
>  I tried "rpmsg ping" with "sim/rpserver" and "sim/rpproxy", it also
>  doesn't show any thing. However, the "echo hello  /dev/ttyproxy"
>  command can show the message on console.
> 
> 
> 
> 
>  Can you please suggest where should I start looking for RPTUN trouble
>  shooting information?
> 
> 
> 
> 
>  Regards,
> 
>  yf

Re: RPTUN usage

[Xiang Xiao]

You can try:

   1. echo hello > /dev/ttyRpmsg on one core
   2. and cat /dev/ttyRpmsg on another core

to confirm rpmsg uart work as expected before testing with cu.

On Wed, Feb 7, 2024 at 8:00 PM yfliu2008  wrote:

> Xiang and Bowen,
>
>
>
> I noticed that I didn't initialize the call the "uart_rpmsg_init()"
> properly thus last time "rpmsg-ttyRpmsg" end point didn't shown in the
> "rpmsg dump" result. After fixing the call parameters, they shows out now:
>
>
> master rpmsg dump /dev/rptun/remote 
>  
>   Dump rpmsg info between cpu
> (master: yes)master <== remote: 
> 
> rpmsg vq RX:
>  
>  
>  
> rpmsg vq TX:
>  
>  
>  
>  rpmsg ept list:   
>  
>  
>
>   ept NS   
>  
>  
>
>   ept rpmsg-ttyRpmsg  
>  
>  
>   
>   ept rpmsg-ping  
>  
>  
> 
>  rpmsg buffer list:   
>  
>  
>   
>   RX buffer, total 8, pending 0
>  
> 
>   TX buffer, total 8, pending 0
>
> The "echo hello  /dev/ttyRpmsg" command doesn't block any more,
> however the "cu" command doesn't work yet. I am not sure what is
> still missing. I think it is better to make uart-rpmsg work first before
> try any other RPMsg device types. Please let me know if you have any
> suggestions.
>
>
>
> Regards,
> yf
>
>
>
>
> Original
>
>
>
> From:"yfliu2008"< yfliu2...@qq.com ;
>
> Date:2024/2/7 19:20
>
> To:"dev"< dev@nuttx.apache.org ;
>
> Subject:RPTUN usage
>
>
>
> Xiang  Bowen,
>
>
>
>
> Here I seemed have RPTUN running on the two cores of K230 device and I can
> use "rpmsg dump" to show some basic information on both cores:
>
>
>
> master rpmsg dump /dev/rptun/remote 
>  
>   Dump rpmsg info between cpu
> (master: yes)master <== remote: 
> 
> rpmsg vq RX:
>  
>  
>  
> rpmsg vq TX:
>  
>  
>  
>  rpmsg ept list:   
>  
>  
>
>   ept NS   
>  
>  
>
>   ept rpmsg-ping  
>  
>  
> 
>  rpmsg buffer list:   
>  
>  
>   
>   RX buffer, total 8, pending 0
>  
> 
>
>   TX buffer, total 8, pending 0
>
>
> remote rpmsg /dev/rptun/master 
>  
>  nsh: rpmsg:
> missing required argument(s)  
>  
>
> remote rpmsg dump /dev/rptun/master 
>  
>   
> Dump rpmsg info between cpu (master: no)remote <== master: 
> 
> rpmsg vq RX:
>  
>  
>  
> rpmsg vq TX:
>  
>  
>  
>  rpmsg ept list:   
>  
>  
>
>   ept NS   
>  
>  
>
>  rpmsg buffer list:   
>  
>  
>   
>   RX buffer, total 8, pending 0
>  
> 
>   TX buffer, total 8, pending 0
>
>
>
> However, I don't know how to further check the RPTUN link yet. It seems
> that "rpmsg ping /dev/rptun/remote 1 64 0 10" doesn't show anything. Also
> "echo hello  /dev/ttyRpmsg" gets blocked.
>
>
>
>
>
> I tried "rpmsg ping" with "sim/rpserver" and "sim/rpproxy", it also
> doesn't show any thing. However, the "echo hello  /dev/ttyproxy"
> command can show the message on console.
>
>
>
>
> Can you please suggest where should I start looking for RPTUN trouble
> shooting information?
>
>
>
>
> Regards,
>
> yf

Re: RPTUN share memory requirements

[Xiang Xiao]

On Thu, Feb 1, 2024 at 10:03 AM yfliu2008  wrote:

> Xiang  Bowen,
>
>
> Can you please teach if there are any requirements from RPTUN/RPMSG
> drivers for the share memory? For example, the share memory should be
> cache-less etc.
>
>
It's better to use bufferable, but no cacheable memory. If you prefer
cacheable memory, Please enable CONFIG_OPENAMP_CACHE, so OpenAMP could
flash the shared memory(virtio vring, rpmsg buffer) for you. But if you
send the pointer in rpmsg buffer, which point to other shared memory, you
need flash cache by self.


>
> Currently I used "xd " command checked in both NuttX instances and found
> inconsist dump, so I am wondering what is the assumption of RPTUN/RPMSG
> drivers.
>
>
> Regards,
> yf

Re: Re:Re:Re: RPTUN and RPMSG

[Xiang Xiao]

On Mon, Jan 29, 2024 at 2:13 PM yfliu2008  wrote:

> Bowen  Xiang,
>
>
>
>
> I am considering to use RPTUN on K230 target, initally between the two
> RISC-V cores. NuttX can run on both RISC-V cores with the help of U-Boot. I
> browsed the "nrf53_rptun.c" and "sim_rptun.c" in NuttX repository, however
> I can only run "sim_rptun" on Linux.
>
>
> I still have a few questions as a RPTUN user:
>
>
>
>
> Does RPTUN only support nodes running NuttX? we can't use RPTUN on a DSP
> if it can't run NuttX, right?
>

Yes, RPTUN can't be used outside of NuttX. Please use OpenAMP or rpmsg-lite
directly if you run other OS.


>
> Can a RPTUN master manage a baremetal remote? is there any existing
> samples in the repo now for this case?
>
>
You need reference OpenAMP, there are many examples.


>
> Can we use RPTUN without inter-processor-interruptions? I am wondering if
> a minimum RPTUN implemtation can basically work without IPI, and can work
> better when IPI us added?
>

You need an interrupt to trigger the remote core either through IPI,
mailbox or doorbell, otherwise you have to poll the ring buffer
periodically like sim, but it is very inefficient.


>
> Does RPTUN support loading of remote NuttX firmware? I didn't find how
> "nrf5340-dk/rptun_cpuapp" manages the firmware of "rptun_cpunet" yet. But I
> saw "sim/rpserver" can spawn the remote node as a host process. If there is
> an existing RPTUN sample that shows loading remote firmware, please teach.
>
>
Yes, RPTUN can support loading ELF to the remote core, we use this feature
in some products, but no upstreaming project enables this feature yet.


> Does RPTUN support NuttX in Kernel/Protected modes? If so, we can manage a
> Flat remote from a Kernel master.


It could be. Eero Nurkkala may use this configuration on risc-v/mpfs, but
we haven't tried this configuration yet.


>
>
>
>
>
> Regards,
> yf
>
>
>
>
>
>
>
>
> Original
>
>
>
> From:"汪博文"< wangbow...@xiaomi.com.INVALID ;
>
> Date:2024/1/26 19:04
>
> To:"yfliu2008"< yfliu2...@qq.com.INVALID ;
>
> CC:"dev"< dev@nuttx.apache.org ;
>
> Subject:Re: [External Mail]Re:Re: RPTUN and RPMSG
>
>
> Hi, yf
>
> Yes, in some scenarios, we do not need the Remoteproc to manage the remote
> core lifecycle.
> (e.g, the remote core is startup automatically and we do not want to
> stop/restart remote core)
>
> So we do not need the remoteproc, and the remoteproc in rptun may appear
> redundant.
>
> Actually, we are also working on a standalone Rpmsg Virtio transport in
> NuttX (new file in nuttx/drivers/rpmsg/rpmsg_virtio.c),
> we want this new rpmsg virtio transport only call the rpmsg virtio related
> API in OpenAMP (not be coupled with OpenAMP.RemoteProc),
> then we can achieve smaller footprint and maybe fater intialization speed.
>
> OpenAMP community has similar discussions: rpmsg without remoteproc? ·
> OpenAMP/open-amp · Discussion #275 (github.com)
>
> And Rpmsg-Lite is a lightweight implementation of the Remote Processor
> Messaging (RPMsg) protocol (No remoteproc):
> nxp-mcuxpresso/rpmsg-lite: RPMsg implementation for small MCUs (github.com
> )
>
> 
> Bowen Wang, wangbow...@xiaomi.com
>
> From: yfliu2008
> Date: 2024-01-26 18:32
> To: dev
> Subject: [External Mail]Re:Re: RPTUN and RPMSG
> [外部邮件] 此邮件来源于小米公司外部，请谨慎处理。若对邮件安全性存疑，请将邮件转发给mi...@xiaomi.com进行反馈
>
> Xiang and Bowen,
>
>
>
> Thank you both the prompt explantions. Though still need time to get
> familiar with both rptun and rpmsg drivers in NuttX, I see that refactoring
> is ongoing currently to make rpmsg more independent. Meanwhile, one
> should still use rptun as the AMP bridge by preparing
> core-specific hooks and call rptun_initialize() during boot to enable
> AMP, right?
>
>
> Suppose the master and remote cores both can run NuttX, can we simply put
> the remote NuttX binary in ROMFS of the master NuttX then prepare
> rptun hooks and then leave the lifecycle details to rptun/openamp?
>
>
> Regards,
> yf
>
>
>
>
>
> Original
>
>
>
> From:"汪博文"< wangbow...@xiaomi.com.INVALID ;
>
> Date:2024/1/26 14:38
>
> To:"yfliu2008"< yfliu2...@qq.com.INVALID ;
>
> CC:"dev"< dev@nuttx.apache.org ;
>
> Subject:Re: [External Mail]RPTUN and RPMSG
>
>
> Hi, yf
>
> NuttX.RPTUN abstracts the OpenAMP.RemoteProc and the OpenAMP.RpmsgVirtio
> NuttX.Rpmsg wraps the OpenAMP.Rpmsg and add more rpmsg api.
>
> I guess you want to ask whether Rpmsg can support different transport
> layer such as SPI/UART/I2C.
> Acctually, Rpmsg was designed to be compatible with different transport
> layer, and VirtIO is a very common transport, we call it Rpmsg VirtIO.
>
> And we are implementing the Rpmsg SPI/Uart Transport Layer in NuttX, we
> divide it to serveral steps:
> 1. NuttX.Rpmsg wrapper the OpenAMP.Rpmsg but the coupling between
> NuttX.Rpmsg and NuttX.Rptun (the wrapper of OpenAMP Rpmsg VirtIO and
> Remoteproc) is severe,
> so we decoupled the NuttX.Rpmsg and Nuttx.Rptun, here are some PRs:
> decoupling rptun ping and rptun_virtio_device, 

Re: RPTUN and RPMSG

[Xiang Xiao]

On Fri, Jan 26, 2024 at 1:01 PM yfliu2008  wrote:

> Dear experts,
>
>
>
>
> I am wondering where we can find high level introduction about the RPTUN
> and RPMSG concepts in NuttX?
>
>
>
2022 online workshop has a introduction presentation:
https://www.youtube.com/watch?v=6uehz3WQ0vU=211s


>
>
> I can't find explanations of the two NuttX drivers and their relationship
> with the two OpenAMP components "RemoteProc" and "RPMsg". Can we simply
> take that the "NuttX.RPTUN" abstracts the "OpenAMP.RemoteProc" and the
> "NuttX.RPMSG" wraps the "OpenAMP.RPMsg"?
>
>
Yes, NuttX.RPTUN equals OpenAMP.RemoteProc, which implements many OpenAMP
required functions, and defines a bunch of callbacks for customization.


>
> Can we also take that RPMSG models the communication channel between
> master and a remote core, like other master/slave buses such as SPI/I2C,
> thus various proxy drivers (like block, uart, syslog etc) for remote
> to use can be built on top of the RPMSG bus?
>
>
Yes, we are working on this area to:

   1. Decouple rpmsg from virtio
   2. Implement new rpmsg transport(e.g. SPI, UART)

Part of this work upstream recently, more will come soon:
https://github.com/apache/nuttx/pulls?q=is%3Apr+author%3Awyr8899+
After we finish all work, all high level rpmsg driver/service(e.g. rpmsg
syslog/uart/usrsock/fs...) will work for shared memory and spi/uart.



>
>
> Regards,
> yf

Re: debugassert vs assert in apps

[Xiang Xiao]

Here is the issue to track debug.h created before:
https://github.com/apache/nuttx/issues/8986

On Wed, Jan 3, 2024 at 2:30 AM Gregory Nutt  wrote:

> > On Tue, Jan 2, 2024 at 11:16 AM Fotis Panagiotopoulos
> >   wrote:
> >> DEBUGASSERT shall only be used for the kernel.
> >> assert shall only be used for apps.
> >>
> >> Rationale:
> >>
> >> DEBUGASSERT is a custom macro that only exists in the NuttX world.
> >> As such it is best being used in NuttX-specific code.
> >>
> >> assert on the other hand is a standard function, it exists in all
> systems.
> >> It seems better suited for generic and portable code (i.e. apps).
>
> We have no way to enforce this.  What we do do is at least make it clear
> that the application is using a kernel-only macro or function
> inappropriately by putting the macro definition or function prototype in
> NuttX private file.  For example,
>
>   * Remove the DEBUGASSERT (and ASSERT?) macros from include/assert.h
>   * Add them to include/nuttx/assert.h
>   * All applications that use DEBUGASSERT (or ASSERT) should be changed
> to use assert()
>   * Kernel functions that use DEBUGASSERT (or ASSERT) would have to
> explicitly include 
>
> Not foolproof, but that is how we have dealt this this in the past.  It
> if is in include/assert.h, is is essentially being advertised as OK for
> use by applications.  This is logically equivalent to the Linux/LibC
> conventions of putting Linux-specific files in /usr/include/linux vs.
> normal, standard application files in /usr/include.
>
> There may be some issues in common application/kernel LibC files that
> use DEBUGASSERT (or ASSERT)
>
> This same discussion should also apply to include/debug.h which contains
> only nonstandard debug macros.  Shouldn't it also be moved to
> include/nuttx/debug.h?
>
>

Re: SPI EEPROM, losetup, LittleFS

[Xiang Xiao]

On Fri, Dec 15, 2023 at 5:17 PM Sebastien Lorquet 
wrote:

> Hi,
>
> I dont think you can use losetup to create a mtd device. loop devices
> are regular block devices.


You can use lomtd to setup a mtd device loop:
https://github.com/apache/nuttx-apps/blob/master/nshlib/nsh_fscmds.c#L1144-L1306




> This will only work if littlefs can mount a
> block device (no idea if thats possible)
>
>
Yes, littlefs can work with block devices directly, the difference between
mtd and block is hidden by littlefs wrapper.
Of course, littlefs isn't an efficient solution for block devices since
the wear leveling built inside littlefs is useless and
harms the performance a lot.


> what you need is an adaptation layer that will implement a mtd device on
> an eeprom.
>
> You can duplicate a spi flash driver, and change its read/erase/write
> routines (and initialization) by copying code from a spi eeprom driver.
>
> Sebastien.
>
> Le 14/12/2023 à 19:13, Tim Hardisty a écrit :
> > Question first, then explanation.
> >
> > How do I use losetup (which is what I think I need) with a character
> > memory device and mount an FS on it? SPI EEPROM and LittleFS
> > specifically.
> >
> > Yes, it's a bad idea, probably, but I just want to have a play and
> > lack of NuttX/POSIX/Linux knowledge has bitten my behind again. This
> > is perhaps the 3rd time over the years I've asked but even checking
> > the dev emails archive and I can't find what I need.
> >
> > SPI EEPROM is correctly registered as /dev/at25 and I can
> > read/write/etc to it as a character device. NB: it does not initialize
> > in the same way as an I2C EEPROM so you can't directly initialise it
> > as an mtd. It is for storage of settings, and some of the values may
> > change 10,000's time over the life of the product, hence EEPROM with
> > 1,000,000+ write cycles.
> >
> > But there is then no wear levelling or anything like that as just a
> > character device, so I thought I'd have a play, and LittleFS has a
> > small minimum sector size and *might* work. A quick Google turns up
> > the same Linux-related losetup examples that don't seem relevant; my
> > Googling skills are probably as bad as those of my software
> > engineering .
> >
> > Any pointers/suggestions please?
> >
>

Re: regarding latest boards/Board.mk

[Xiang Xiao]

The attachment is deleted by email server, could you create a pr on:
https://github.com/apache/nuttx?

On Fri, Dec 15, 2023 at 11:20 AM yfliu2008  wrote:

> Hi,
>
>
> It seems that the latest boards/Board.mk leaks "etctmp.c" and "etctmp/"
> folder.
>
> I am unsure if it is desired or not.
>
> If it is not desired, attached pacth can resolve it. hope someone can
> confirm if this is right or not.
>
>
>
> Regards,
>
> yf
>
>
>
>
>

Re: Job opportunity

[Xiang Xiao]

Sure.

On Wed, Nov 22, 2023 at 12:44 AM Tomek CEDRO  wrote:

> On Tue, Nov 21, 2023 at 5:38 PM Xiang Xiao wrote:
> > Is anyone interested in signing a contract with Xiaomi to improve the
> NuttX
> > components(e.g. X86_64, Xtensa, hypervisor etc)?
> > Please send me an email if you want to know more information.
> > Thanks
> > Xiang
>
> Hey there Xiang I will be searching for a contract soon maybe I could
> help? :-)
>
> --
> CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
>

Job opportunity

[Xiang Xiao]

Is anyone interested in signing a contract with Xiaomi to improve the NuttX
components(e.g. X86_64, Xtensa, hypervisor etc)?
Please send me an email if you want to know more information.

Thanks
Xiang

Re: [VOTE] Apache NuttX 12.3.0 RC1 release

[Xiang Xiao]

+1 for sim:
./tools/checkrelease.sh --release 12.3.0-RC1
Downloading release files from
https://dist.apache.org/repos/dist/dev/nuttx/12.3.0-RC1/
gpg: directory '/tmp/nuttx-checkrelease/.gnupg' created
gpg: keybox '/tmp/nuttx-checkrelease/.gnupg/pubring.kbx' created
gpg: /tmp/nuttx-checkrelease/.gnupg/trustdb.gpg: trustdb created
gpg: key E1B6E30DB05D6280: public key "Brennan Ashton "
imported
gpg: key 2B8C7F0EAB22000E: public key "Abdelatif Guettouche (CODE SIGNING
KEY) " imported
gpg: key 4137A71698C5E4DB: public key "Alin Jerpelea "
imported
gpg: key 9E711BAD3264C061: public key "Alin Jerpelea "
imported
gpg: key A57CE1279F1E7328: public key "Alin Jerpelea (CODE SIGNING KEY) <
jerpe...@apache.org>" imported
gpg: key 6E72660F995FBC42: public key "Brennan Ashton <
bash...@brennanashton.com>" imported
gpg: Total number processed: 6
gpg:   imported: 6
 OK: https://dist.apache.org/repos/dist/dev/nuttx/KEYS is imported.
Checking apache-nuttx-12.3.0.tar.gz sha512...
 OK: apache-nuttx-12.3.0.tar.gz sha512 hash matches.

Checking apache-nuttx-12.3.0.tar.gz GPG signature:
gpg: Signature made Wed Oct 18 15:14:21 2023 CST
gpg:using RSA key 9208D2E4B800D66F749AD4E94137A71698C5E4DB
gpg: Good signature from "Alin Jerpelea " [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:  There is no indication that the signature belongs to the
owner.
Primary key fingerprint: 9208 D2E4 B800 D66F 749A  D4E9 4137 A716 98C5 E4DB
 OK: apache-nuttx-12.3.0.tar.gz gpg signature matches.

Checking apache-nuttx-12.3.0.tar.gz for required files:
 OK: all required files exist in nuttx.

Checking apache-nuttx-apps-12.3.0.tar.gz sha512...
 OK: apache-nuttx-apps-12.3.0.tar.gz sha512 hash matches.

Checking apache-nuttx-apps-12.3.0.tar.gz GPG signature:
gpg: Signature made Wed Oct 18 15:14:22 2023 CST
gpg:using RSA key 9208D2E4B800D66F749AD4E94137A71698C5E4DB
gpg: Good signature from "Alin Jerpelea " [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:  There is no indication that the signature belongs to the
owner.
Primary key fingerprint: 9208 D2E4 B800 D66F 749A  D4E9 4137 A716 98C5 E4DB
 OK: apache-nuttx-apps-12.3.0.tar.gz gpg signature matches.

Checking apache-nuttx-apps-12.3.0.tar.gz for required files:
 OK: all required files exist in apps.

Trying to build nuttx sim:asan...
 OK: we were able to build sim:asan.

Trying to run nuttx sim:asan...
==6439==WARNING: ASan is ignoring requested __asan_handle_no_return: stack
type: default top: 0x7fffbcbc3000; bottom 0x63123000; size:
0x1cefbcba (31815989067776)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
 OK: ostest with ASAN pass.

Thanks
Xiang

On Wed, Oct 18, 2023 at 3:20 PM Alin Jerpelea  wrote:

> Hello all,
> Apache NuttX 12.3.0 RC1 has been staged under [1] and it's
> time to vote on accepting it for release. Voting will be open for 72hr.
>
> A minimum of 3 binding +1 votes and more binding +1 than binding -1 are
> required to pass.
>
> The Apache requirements for approving a release can be found here [3]
> "Before voting +1 PMC members are required to download the signed
> source code package, compile it as provided, and test the resulting
> executable on their own platform, along with also verifying that the
> package meets the requirements of the ASF policy on releases."
>
> A document to walk through some of this process has been published on
> our project wiki and can be found here [4].
>
> [ ] +1 accept (indicate what you validated - e.g. performed the non-RM
> items in [4])
> [ ] -1 reject (explanation required)
>
> Thank you all,
> Alin Jerpelea
>
> SCM Information:
>   Release tag: nuttx-12.3.0-RC1
>   Hash for the release nuttx tag: 8fdb56b5f2124de351a48125a252ba4264665e3d
>   Hash for the release nuttx-apps tag:
> 2d6d199c492196c4680ae7f5b97bb6e2a7b38649
>
> [1] https://dist.apache.org/repos/dist/dev/nuttx/12.3.0-RC1/
> [2]
> https://raw.githubusercontent.com/apache/nuttx/nuttx-12.3.0-RC1/ReleaseNotes
> [3] https://www.apache.org/dev/release.html#approving-a-release
> [4]
> https://cwiki.apache.org/confluence/display/NUTTX/Validating+a+staged+Release
>

Re: execv obsolet ?

[Xiang Xiao]

you need enable CONFIG_LIBC_EXECFUNCS:
https://github.com/apache/nuttx/blob/master/libs/libc/unistd/lib_execv.c#L29

On Fri, Oct 13, 2023 at 1:59 AM MIGUEL ALEXANDRE WISINTAINER <
tcpipc...@hotmail.com> wrote:

> #include 
> #include 
> #include 
> #include 
>
> #ifdef CONFIG_BUILD_KERNEL
> int main(int argc, FAR char *argv[])
> #else
> int hello_main(int argc, char *argv[])
> #endif
> {
>
> if (argc < 2)
> {
> static char * const args[2] =
> {
> "/bin/ls",
> NULL
> };
>
> execv(args[0], args);
> }
>
> else
> {
> int i;
>
> for (i = 1; i < argc; i++)
> {
> static char * const args[3] =
> {
> "/bin/ls",
> NULL,
> NULL
> };
> args[1] = argv[i];
> execv(args[0], args);
> }
> }
> return EXIT_SUCCESS;
> }
>
> hello_main.c:(.text.hello_main+0xe): undefined reference to `execv'
> xtensa-esp32-elf-ld: hello_main.c:(.text.hello_main+0x20): undefined
> reference to `execv'
> make[1]: *** [Makefile:173: nuttx] Error 1
> make: *** [tools/Unix.mk:537: nuttx] Error 2
>

Re: [VOTE] Apache NuttX 12.3.0 RC0 release

[Xiang Xiao]

A recent change limits c++20 requirement to libcxx:
https://github.com/apache/nuttx/pull/10860
The owner of libcxx decided not to support the build of libcxx by
an old compiler, It's very hard to sync with the last libcxx version, and
keep the support of the old compiler at the same time in the long term.


On Thu, Oct 12, 2023 at 7:31 AM Tomek CEDRO  wrote:

> On Thu, Oct 12, 2023 at 12:19 AM Nathan Hartman wrote:
> > On Wed, Oct 11, 2023 at 3:21 PM Tomek CEDRO wrote:
> > > Hello world,
> > > I have encountered a basic esp32:nsh build error in nuttx-12.3.0-RC0
> > > related to xtensa g++ 8.4.0 compiler:
> > > xtensa-esp32-elf-g++: error: unrecognized command line option
> > > '-std=gnu++20'; did you mean '-std=gnu++2a'?
> >
> > This was changed in d6453cfc3cd4771a5221528cab3056660be4b1e3 (PR # 8244).
> >
> > See mail thread:
> > https://lists.apache.org/thread/rkq99os8ql28bv1w8fy5mdqwl3h2vp6m
> >
> > The workaround is to set Kconfig CXX_STANDARD to gnu++2a instead of
> gnu++20.
>
> Thank you Nathan :-)
>
> I am following the mailing list and github PRs so I know this was
> recent change to make better compatibility with C++ and this is fine
> to have this support. But this should not happen at the cost of
> standard build setup and/or enforcing specific compilers / versions to
> everyone by default (see inviolables). This probably also impacts
> other platforms/architectures, especially the older ones. I am sure
> this can be fixed easily.
>
> I am sorry for giving my first -1 as I know there are many new
> features and improvements in the RC, but I am afraid if we don't fix
> that compiler problem now then there will be no incentive to do this
> in future.
>
>
> > It would be nice if the build system could automatically detect the
> > compiler version and set this value accordingly and automatically,
> > without a Kconfig. IIRC there are programs in tools that determine
> > compiler options based on the compiler and version. (Can't remember
> > details off the top of my head at the moment.)
>
> Yes that would be perfect!
>
> Quick search shows that people do this in many ways by calling
> compiler with a list of flags and checking return status for each flag
> to see if it is supported.
>
> Current C++ standards supported by GCC and CLANG are listed here:
> https://gcc.gnu.org/projects/cxx-status.html
> https://clang.llvm.org/cxx_status.html
>
> Quote from GCC:
> C++20 Support in GCC
> GCC has experimental support for the latest revision of the C++
> standard, which was published in 2020. The status of C++20 library
> features is described in the library documentation.
> C++20 features are available since GCC 8. To enable C++20 support, add
> the command-line parameter -std=c++20 (use -std=c++2a in GCC 9 and
> earlier) to your g++ command line. Or, to enable GNU extensions in
> addition to C++20 features, add -std=gnu++20.
>
> Maybe a simple check can be added to a Makefile?
>
> Maybe a C++ support could be added as option that is disabled by
> default (+compiler check)?
>
>
> > > REMARKS:
> > > 1. flock is a new system build dependency.
> >
> > I think documentation needs to be updated? I think not all systems
> > have flock. Is it a Linux-only thing? (Non-BSD?)
>
> Not a big deal I guess :-) As compared to 12.2.1 in order to build
> 12.3.0-RC0 I had to install additional "flock" application. It comes
> from Linux and it is available on FreeBSD. On macOS (BSD) it is not
> part of the base system too and it can be installed with brew. It
> seems to be fairly old commit that fixes parallel build (which I also
> use). Just a notice :-)
>
>
> https://github.com/apache/nuttx-apps/blob/91a682f11bba1a537bd095cef45f17db57c9e7b9/Make.defs#L157
>
> https://github.com/apache/nuttx-apps/commit/f774e8ee7202a61bcf64a93b2f3ddfd240288cf1
>
> Thank you :-)
> Tomek
>
> --
> CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
>

Re: Using static analysis to find (potential) bugs

[Xiang Xiao]

There are patch to integrate the CodeChecker into ci:
https://github.com/apache/nuttx/pull/7114
https://github.com/apache/nuttx/pull/7090
you can follow them for cppcheck too.
The hard problem is that tools normally report many false alarms which make
it impractical to enable the check in ci/cd.

On Wed, Oct 11, 2023 at 2:34 AM Daniel Appiagyei
 wrote:

> Hey,
> I was running the [cppcheck](https://cppcheck.sourceforge.io/) static
> analysis tool, found a few potential bugs, and wrote the following to share
> how tools like this can help us ensure the integrity of our code. The
> following are some bugs found. If anyone is interested in running cppcheck
> on their project, scroll to the bottom for a HOW-TO. Is using a static
> analysis tool like this something we'd be interested in adding to ci/cd?
>
> *The following were found in NuttX 12.2.1*
> ## Null pointer dereference
>  1: sched/signal/sig_dispatch.c:325
> ```
> src/deps/nuttx/sched/signal/sig_dispatch.c:325:26: warning: Either the
> condition 'info!=NULL' is redundant or there is possible null pointer
> dereference: info. [nullPointerRedundantCheck]
> stcb, stcb->pid, info->si_signo, info->si_code,
>
>
>  ^
>
>
> src/deps/nuttx/sched/signal/sig_dispatch.c:329:36: note: Assuming that
> condition 'info!=NULL' is not redundant
>
>   DEBUGASSERT(stcb != NULL && info != NULL);
>
>
>^
>
>
> src/deps/nuttx/sched/signal/sig_dispatch.c:325:26: note: Null pointer
> dereference
>
> stcb, stcb->pid, info->si_signo, info->si_code,
>
>
>  ^
>
>
> ```
>
>   2: sched/signal/sig_dispatch.c:326
> ```
> src/deps/nuttx/sched/signal/sig_dispatch.c:326:9: warning: Either the
> condition 'info!=NULL' is redundant or there is possible null pointer
> dereference: info. [nullPointerRedundantCheck]
> info->si_value.sival_int,
>
>
> ^
> src/deps/nuttx/sched/signal/sig_dispatch.c:329:36: note: Assuming that
> condition 'info!=NULL' is not redundant
>   DEBUGASSERT(stcb != NULL && info != NULL);
>^
> src/deps/nuttx/sched/signal/sig_dispatch.c:326:9: note: Null pointer
> dereference
> info->si_value.sival_int,
> ^
> ```
>
>  3: sched/signal/sig_dispatch.c:327
> ```
> src/deps/nuttx/sched/signal/sig_dispatch.c:327:41: warning: Either the
> condition 'info!=NULL' is redundant or there is possible null pointer
> dereference: info. [nullPointerRedundantCheck]
> sigismember(>sigprocmask, info->si_signo) == 1 ? "YES" :
> "NO");
> ^
> src/deps/nuttx/sched/signal/sig_dispatch.c:329:36: note: Assuming that
> condition 'info!=NULL' is not redundant
>   DEBUGASSERT(stcb != NULL && info != NULL);
>^
> src/deps/nuttx/sched/signal/sig_dispatch.c:327:41: note: Null pointer
> dereference
> sigismember(>sigprocmask, info->si_signo) == 1 ? "YES" :
> "NO");
> ^
> ```
>
>  4: src/deps/nuttx/libs/libc/stdlib/lib_mbstowcs.c:42
> ```
> src/deps/nuttx/libs/libc/stdlib/lib_mbstowcs.c:42:36: error: Null pointer
> dereference [nullPointer]
>
>   return mbsrtowcs(dst, , len, NULL);
>
>
>^
> ```
>
>  5: nuttx/libs/libc/stdlib/lib_wcstombs.c:38
> ```
> src/deps/nuttx/libs/libc/stdlib/lib_wcstombs.c:38:36: error: Null pointer
> dereference [nullPointer]
>   return wcsrtombs(dst, , len, NULL);
> ```
>
>  6: drivers/net/netdev_upperhalf.c:168
> ```
> src/deps/nuttx/drivers/net/netdev_upperhalf.c:168:42: warning: Either the
> condition 'dev' is redundant or there is possible null pointer dereference:
> dev. [nullPointerRedundantCheck]
>   FAR struct netdev_upperhalf_s *upper = dev->d_private;
>  ^
> src/deps/nuttx/drivers/net/netdev_upperhalf.c:170:15: note: Assuming that
> condition 'dev' is not redundant
>   DEBUGASSERT(dev && pkt);
>   ^
> src/deps/nuttx/drivers/net/netdev_upperhalf.c:168:42: note: Null pointer
> dereference
>   FAR struct netdev_upperhalf_s *upper = dev->d_private;
>  ^
> ```
>
>  7: sched/task/task_init.c:90
> ```
> src/deps/nuttx/sched/task/task_init.c:90:19: warning: Either the condition
> 'tcb' is redundant or there is possible null pointer dereference: tcb.
> [nullPointerRedundantCheck]
>   uint8_t ttype = tcb->cmn.flags & TCB_FLAG_TTYPE_MASK;
>   ^
> src/deps/nuttx/sched/task/task_init.c:96:15: note: Assuming that condition
> 'tcb' is not redundant
>   DEBUGASSERT(tcb && ttype != TCB_FLAG_TTYPE_PTHREAD);
>   ^
> ```
>
> ## Signed integer overflow
> The [C standard](
>
> https://www.gnu.org/software/autoconf/manual/autoconf-2.63/html_node/Integer-Overflow-Basics.html
> )
> treats _signed_ integer overflow as undefined behavior.
>  8: arch/arm/src/armv7-m/arm_fpuconfig.c:75
> ```
> 

Re: Enforcing C++ compatibility with C code in CI/CD

[Xiang Xiao]

Let' me find some time to write a test, please wait for a moment.

On Tue, Oct 10, 2023 at 2:53 AM Daniel Appiagyei
 wrote:

> Good solution.
> I'm not too knowledgeable about makefiles, but if you could set that part
> up then I can do the work to make the files compatible?
>
> On Sun, Oct 8, 2023 at 10:44 PM Xiang Xiao 
> wrote:
>
> > We can add a simple test like this:
> >
> >1. Create a context target in Makefile to generate a .cxx file which
> >list and include all header files in nuttx/include
> >2. Add the generated file to CXXSRCS
> >
> > If we just want to ensure the public nuttx header files are compatible
> with
> > c++ syntax.
> >
> > On Mon, Oct 9, 2023 at 9:47 AM Daniel Appiagyei
> >  wrote:
> >
> > > Late follow up, sorry
> > >
> > > Thanks guys, I do understand that only headers need to be compatible,
> > > however, the GCC check I mentioned above does not discriminate between
> > > headers and source files. If we don't have some sort of automated check
> > for
> > > C++ compatibility, then we will run into issues in the future like I
> > > mentioned above. The nuttx style guide mentions that all C code must be
> > C89
> > > compliant. As long as C++ compatibility doesn't interfere with that
> > > standard then we won't be losing much.
> > > Are there any other ideas on how to automatically enforce headers being
> > c++
> > > comaptible?
> > > Best,
> > > Daniel
> > >
> > > On Sun, Sep 17, 2023 at 12:56 PM Petro Karashchenko <
> > > petro.karashche...@gmail.com> wrote:
> > >
> > > > Since C++ most likely will never be used in the kernel I see that
> > > > compatibility border should stay in the "include" folder in nuttx
> repo.
> > > The
> > > > nuttx-apps headers should be C++ compatible as we can mix things at
> > > > application level.
> > > >
> > >
> > >
> > > --
> > >
> > >
> > > *Daniel Appiagyei | Embedded Software Engineer *Email:
> > > daniel.appiag...@braincorp.com
> > > *Brain*
> > > * Corp™ *10182 Telesis Ct, Suite 100
> > > San Diego, CA 92121
> > >
> > > (858)-689-7600
> > > www.braincorp.com
> > >
> >
>
>
> --
>
>
> *Daniel Appiagyei | Embedded Software Engineer *Email:
> daniel.appiag...@braincorp.com
> *Brain*
> * Corp™ *10182 Telesis Ct, Suite 100
> San Diego, CA 92121
>
> (858)-689-7600
> www.braincorp.com
>

Re: Enforcing C++ compatibility with C code in CI/CD

[Xiang Xiao]

We can add a simple test like this:

   1. Create a context target in Makefile to generate a .cxx file which
   list and include all header files in nuttx/include
   2. Add the generated file to CXXSRCS

If we just want to ensure the public nuttx header files are compatible with
c++ syntax.

On Mon, Oct 9, 2023 at 9:47 AM Daniel Appiagyei
 wrote:

> Late follow up, sorry
>
> Thanks guys, I do understand that only headers need to be compatible,
> however, the GCC check I mentioned above does not discriminate between
> headers and source files. If we don't have some sort of automated check for
> C++ compatibility, then we will run into issues in the future like I
> mentioned above. The nuttx style guide mentions that all C code must be C89
> compliant. As long as C++ compatibility doesn't interfere with that
> standard then we won't be losing much.
> Are there any other ideas on how to automatically enforce headers being c++
> comaptible?
> Best,
> Daniel
>
> On Sun, Sep 17, 2023 at 12:56 PM Petro Karashchenko <
> petro.karashche...@gmail.com> wrote:
>
> > Since C++ most likely will never be used in the kernel I see that
> > compatibility border should stay in the "include" folder in nuttx repo.
> The
> > nuttx-apps headers should be C++ compatible as we can mix things at
> > application level.
> >
>
>
> --
>
>
> *Daniel Appiagyei | Embedded Software Engineer *Email:
> daniel.appiag...@braincorp.com
> *Brain*
> * Corp™ *10182 Telesis Ct, Suite 100
> San Diego, CA 92121
>
> (858)-689-7600
> www.braincorp.com
>

Re: Error: Unrecognized gnu++20, did you mean gnu++2a?

[Xiang Xiao]

>From https://gcc.gnu.org/onlinedocs/gcc-11.2.0/gcc/C-Dialect-Options.html,
gnu++2a equals gnu++2a, so it's fine to change the default o gnu++2a.


On Mon, Oct 2, 2023 at 6:43 AM Nathan Hartman 
wrote:

> Hi all,
>
> Commit d6453cfc3cd4771a5221528cab3056660be4b1e3 (PR # 8244) changes
> the default CXX_STANDARD for libxx to gnu++20 to make llvm libcxx
> 15.0.7 happy.
>
> Unfortunately this makes GCC versions < 10.3 fail with something like:
>
> arm-none-eabi-g++: error: unrecognized command line option
> '-std=gnu++20'; did you mean '-std=gnu++2a'?
>
> And (as you can see in the PR discussions) on some host OSes
> (including mine) it isn't convenient to install a newer GCC.
>
> In commit 4e93f93f303a85e04bdd7e4da22df028c0f76de3, the documentation
> is updated to tell us that we need to update GCC to 10.3.
>
> The ---help--- for Kconfig CXX_STANDARD lists possible values:
> gnu++98/c++98, gnu++11/c++11, gnu++14/c++14, gnu++17/c++17 and
> gnu++20/c++20. Is it possible to add gnu++2a to that list, so that
> people with old(er) host OSes (like me) can continue?
>
> Thanks,
> Nathan
>

Re: riscv64 libm link question

[Xiang Xiao]

On Mon, Sep 18, 2023 at 7:01 AM Lee, Lup Yuen  wrote:

> Here's what Ken and I have tried for the riscv64 libm linking problem:
>
> (1) The recommended toolchain for NuttX on riscv64 is SiFive Freedom Tools.
> But it doesn't support Raspberry Pi OS (64-bit Debian).
>
> (2) So we used Debian gcc-riscv64-unknown-elf instead. Which shows the
> missing libm error that Ken described.
>
> (3) We're trying an alternative toolchain: xPack GNU RISC-V Embedded GCC.
> Which might fix the libm problem, but we're not sure if it's supported for
> NuttX.
>
>
Lup, could you update the toolchain used by ci after you get a working new
binary?
ci is blocked by this now: https://github.com/apache/nuttx/issues/10594.


> Wonder if anyone has tried building riscv64 NuttX on Raspberry Pi OS /
> Arm64 Linux? More details below. Thanks!
>
>
> https://lupyuen.github.io/articles/riscv#appendix-xpack-gnu-risc-v-embedded-gcc-toolchain-for-64-bit-risc-v
>
> https://lupyuen.github.io/articles/release#appendix-missing-mathh
>
> Lup
>
> On Mon, Sep 18, 2023 at 4:15 AM  wrote:
>
> > On 2023-09-17 12:47, Petro Karashchenko wrote:
> > > Hi,
> > >
> > > What kind of RISC-V toolchain are you using?
> >
> > Raspian:RasPi4:~/RISCV/NuttX/apps >>> apt search riscv64-unknown
> > Sorting... Done
> > Full Text Search... Done
> > binutils-riscv64-unknown-elf/oldstable,now 2.32.2020.04+dfsg-2 arm64
> > [installed]
> >GNU assembler, linker and binary utilities for RISC-V processors
> >
> > gcc-riscv64-unknown-elf/oldstable,now 8.3.0.2019.08+dfsg-1 arm64
> > [installed]
> >GCC compiler for embedded RISC-V chips
> >
> > picolibc-riscv64-unknown-elf/oldstable,oldstable,now 1.5.1-2 all
> > [installed]
> >Smaller embedded C library for RISC-V development
> >
> > So just `sudo apt-get install ...` of the above.
> >
> > > Please try to remove  "CFLAGS += -lm " and feedback if that helps.
> >
> > No change.
> >
> > The problem is not that I am trying to use some released libm library
> > but that the NuttX nuttx/libs/libm/libm.a is not being linked into the
> > scheme binary file.
> >
> > I have not used other than trivial makefiles in a couple of decades,
> > never used `make menuconfig`, which allows one to specify many
> > non-bootable systems, and so have little clue how to specify linking the
> > NuttX libm into the scheme executable or alternately getting nuttx to
> > dynamically load the libm library.
> >
> > Note that there is no file 'kconfig-language.txt' in the NuttX tools
> > repository.
> >
> > I am kinda stuck.  Thanks for any help!
> > -KenD
> >
> >
>

Re: riscv64 libm link question

[Xiang Xiao]

On Mon, Sep 18, 2023 at 4:16 AM  wrote:

> On 2023-09-17 12:47, Petro Karashchenko wrote:
> > Hi,
> >
> > What kind of RISC-V toolchain are you using?
>
> Raspian:RasPi4:~/RISCV/NuttX/apps >>> apt search riscv64-unknown
> Sorting... Done
> Full Text Search... Done
> binutils-riscv64-unknown-elf/oldstable,now 2.32.2020.04+dfsg-2 arm64
> [installed]
>GNU assembler, linker and binary utilities for RISC-V processors
>
> gcc-riscv64-unknown-elf/oldstable,now 8.3.0.2019.08+dfsg-1 arm64
> [installed]
>GCC compiler for embedded RISC-V chips
>
> picolibc-riscv64-unknown-elf/oldstable,oldstable,now 1.5.1-2 all
> [installed]
>Smaller embedded C library for RISC-V development
>
> So just `sudo apt-get install ...` of the above.
>
> > Please try to remove  "CFLAGS += -lm " and feedback if that helps.
>
> No change.
>
> The problem is not that I am trying to use some released libm library
> but that the NuttX nuttx/libs/libm/libm.a is not being linked into the
> scheme binary file.
>
>
Do you build scheme as a separate elf image?


> I have not used other than trivial makefiles in a couple of decades,
> never used `make menuconfig`, which allows one to specify many
> non-bootable systems, and so have little clue how to specify linking the
> NuttX libm into the scheme executable or alternately getting nuttx to
> dynamically load the libm library.
>
>
Could you first try the builtin scheme work as expect?


> Note that there is no file 'kconfig-language.txt' in the NuttX tools
> repository.
>
>
it host here:
https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-language.rst


I am kinda stuck.  Thanks for any help!
> -KenD
>
>

Re: Enforcing C++ compatibility with C code in CI/CD

[Xiang Xiao]

I think at least the c header file should be compatible with c++ compiler,
but the c source file could use sme incompatible language features.

On Sat, Sep 16, 2023 at 8:47 AM Daniel Appiagyei
 wrote:

> Hi,
> I'm following up on an issue
>  I
> opened about a C header file not being compatible with a C++ compiler
> because of it using reserved C++ keywords. I'm fixing this issue for header
> files but wanted to know if we want to endorse C++ compatibility for our C
> source files as well.
>
> GCC has the -Wc++-compat
> <
> https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wc_002b_002b-compat
> >
> flag
> which we could enable as an error instead of a warning. Here's an example
> of it catching us using the reserved keyword 'or':
> ```
> misc/lib_fnmatch.c:193:19: warning: identifier "or" is a special operator
> name in C++ [-Wc++-compat]
> ```
> Aside from reserved keywords being used, C++ doesn't support *implicit
> *conversions
> from void* to other types. The majority of nuttx code I've seen so far
> *explicitly* casts anyway, so I don't see requiring explicit casts being a
> major hurdle. Refactoring our codebase to be c++ compliant would therefore
> mainly be find/replace for reserved keywords and adding explicit casts
> (straightforward). I can do the work and it would only take a couple weeks.
>
> My questions are these:
> *Does NuttX want to officially agree to and enforce that*:
> 1. all C headers and source file be compatible with C++, or
> 2. all C headers be compatible *and* only C++ source files with 'extern
> "C"' in them be compatible
>
> Secondly, if we decide on one of the above rulings, do we want to add the
> ruling to our C coding style guidelines
> <
> https://nuttx.apache.org/docs/latest/contributing/coding_style.html#general-conventions
> >
> ?
>
> Thanks,
> Daniel
>
>
> 
>

Re: Serialrx

[Xiang Xiao]

On Wed, Aug 30, 2023 at 5:49 AM Bert Voldenuit 
wrote:

> Hi,
> Alan,
> I use ./tool/configure.sh arduino-m0/nsh
> Thanks to your blog, I am porting an arduino project to Nuttx
>
> Xiang Xiao,
> Indeed  DEFAULT_SMALL was activated. it works with CONFIG_FILE_STREAM
> enabled. How did you know?
>
>
This is a recent improvement to reduce the code size in the minimal
configuration:
https://github.com/apache/nuttx-apps/pull/1559
https://github.com/apache/nuttx/pull/4819


> Thank you for your support.
> Bert.
>
> Le mar. 29 août 2023 à 04:20, Xiang Xiao  a
> écrit :
>
> > Do you enable DEFAULT_SMALL? If so, you have to enable CONFIG_FILE_STREAM
> > in defconfig explicitly.
> >
> > On Tue, Aug 29, 2023 at 6:11 AM Bert Voldenuit 
> > wrote:
> >
> > > Hello,
> > >
> > > I am trying to run the serialrx app (out of the box) from the examples,
> > but
> > > i get the following errors:
> > >
> > > - Some warnings because of conversion; no big deal
> > > - undefined reference to standard librarie function like: fprintf,
> > > lib_get_stream, fflush; I think this is the main problem but why?
> > >
> > > bert@DESKTOP-UO6EU0I:/mnt/d/BertDocs/nuttxspace12/nuttx$ make
> > > Create version.h
> > > LN: platform/board to /mnt/d/BertDocs/nuttxspace12/apps/platform/dummy
> > > Register: custom_hello
> > > Register: serialrx
> > > Register: nsh
> > > Register: sh
> > > CC:  serialrx_main.c serialrx_main.c: In function 'serialrx_main':
> > > serialrx_main.c:83:15: warning: passing argument 1 of 'fprintf' makes
> > > pointer from integer without a cast [-Wint-conversion]
> > >83 |   fprintf(stderr, "Usage: %s [devpath]\n", argv[0]);
> > >   |   ^~
> > >   |   |
> > >   |   int
> > > In file included from serialrx_main.c:31:
> > > /mnt/d/BertDocs/nuttxspace12/nuttx/include/stdio.h:147:26: note:
> expected
> > > 'FILE *' {aka 'struct file_struct *'} but argument is of type 'int'
> > >   147 | intfprintf(FAR FILE *stream, FAR const IPTR char *format,
> > ...)
> > >   |~~^~
> > > serialrx_main.c:95:15: warning: passing argument 1 of 'fprintf' makes
> > > pointer from integer without a cast [-Wint-conversion]
> > >95 |   fprintf(stderr, "ERROR: malloc failed: %d\n", errno);
> > >   |   ^~
> > >   |   |
> > >   |   int
> > > In file included from serialrx_main.c:31:
> > > /mnt/d/BertDocs/nuttxspace12/nuttx/include/stdio.h:147:26: note:
> expected
> > > 'FILE *' {aka 'struct file_struct *'} but argument is of type 'int'
> > >   147 | intfprintf(FAR FILE *stream, FAR const IPTR char *format,
> > ...)
> > >   |~~^~
> > > serialrx_main.c:110:15: warning: passing argument 1 of 'fprintf' makes
> > > pointer from integer without a cast [-Wint-conversion]
> > >   110 |   fprintf(stderr, "ERROR: open failed: %d\n", errno);
> > >   |   ^~
> > >   |   |
> > >   |   int
> > > In file included from serialrx_main.c:31:
> > > /mnt/d/BertDocs/nuttxspace12/nuttx/include/stdio.h:147:26: note:
> expected
> > > 'FILE *' {aka 'struct file_struct *'} but argument is of type 'int'
> > >   147 | intfprintf(FAR FILE *stream, FAR const IPTR char *format,
> > ...)
> > >   |~~^~
> > > serialrx_main.c:116:10: warning: passing argument 1 of 'fflush' makes
> > > pointer from integer without a cast [-Wint-conversion]
> > >   116 |   fflush(stdout);
> > >   |  ^~
> > >   |  |
> > >   |  int
> > > In file included from serialrx_main.c:31:
> > > /mnt/d/BertDocs/nuttxspace12/nuttx/include/stdio.h:139:25: note:
> expected
> > > 'FILE *' {aka 'struct file_struct *'} but argument is of type 'int'
> > >   139 | intfflush(FAR FILE *stream);
> > >   |   ~~^~
> > > serialrx_main.c:143:18: warning: passing argument 1 of 'fflush' makes
> > > pointer from integer without a cast [-Wint-conversion]
> > >   143 |   fflush(stdout);
> > >   |  ^~
> > >   |  |
> > >   |  int
> > > In file included from serialr

Re: Serialrx

[Xiang Xiao]

Do you enable DEFAULT_SMALL? If so, you have to enable CONFIG_FILE_STREAM
in defconfig explicitly.

On Tue, Aug 29, 2023 at 6:11 AM Bert Voldenuit 
wrote:

> Hello,
>
> I am trying to run the serialrx app (out of the box) from the examples, but
> i get the following errors:
>
> - Some warnings because of conversion; no big deal
> - undefined reference to standard librarie function like: fprintf,
> lib_get_stream, fflush; I think this is the main problem but why?
>
> bert@DESKTOP-UO6EU0I:/mnt/d/BertDocs/nuttxspace12/nuttx$ make
> Create version.h
> LN: platform/board to /mnt/d/BertDocs/nuttxspace12/apps/platform/dummy
> Register: custom_hello
> Register: serialrx
> Register: nsh
> Register: sh
> CC:  serialrx_main.c serialrx_main.c: In function 'serialrx_main':
> serialrx_main.c:83:15: warning: passing argument 1 of 'fprintf' makes
> pointer from integer without a cast [-Wint-conversion]
>83 |   fprintf(stderr, "Usage: %s [devpath]\n", argv[0]);
>   |   ^~
>   |   |
>   |   int
> In file included from serialrx_main.c:31:
> /mnt/d/BertDocs/nuttxspace12/nuttx/include/stdio.h:147:26: note: expected
> 'FILE *' {aka 'struct file_struct *'} but argument is of type 'int'
>   147 | intfprintf(FAR FILE *stream, FAR const IPTR char *format, ...)
>   |~~^~
> serialrx_main.c:95:15: warning: passing argument 1 of 'fprintf' makes
> pointer from integer without a cast [-Wint-conversion]
>95 |   fprintf(stderr, "ERROR: malloc failed: %d\n", errno);
>   |   ^~
>   |   |
>   |   int
> In file included from serialrx_main.c:31:
> /mnt/d/BertDocs/nuttxspace12/nuttx/include/stdio.h:147:26: note: expected
> 'FILE *' {aka 'struct file_struct *'} but argument is of type 'int'
>   147 | intfprintf(FAR FILE *stream, FAR const IPTR char *format, ...)
>   |~~^~
> serialrx_main.c:110:15: warning: passing argument 1 of 'fprintf' makes
> pointer from integer without a cast [-Wint-conversion]
>   110 |   fprintf(stderr, "ERROR: open failed: %d\n", errno);
>   |   ^~
>   |   |
>   |   int
> In file included from serialrx_main.c:31:
> /mnt/d/BertDocs/nuttxspace12/nuttx/include/stdio.h:147:26: note: expected
> 'FILE *' {aka 'struct file_struct *'} but argument is of type 'int'
>   147 | intfprintf(FAR FILE *stream, FAR const IPTR char *format, ...)
>   |~~^~
> serialrx_main.c:116:10: warning: passing argument 1 of 'fflush' makes
> pointer from integer without a cast [-Wint-conversion]
>   116 |   fflush(stdout);
>   |  ^~
>   |  |
>   |  int
> In file included from serialrx_main.c:31:
> /mnt/d/BertDocs/nuttxspace12/nuttx/include/stdio.h:139:25: note: expected
> 'FILE *' {aka 'struct file_struct *'} but argument is of type 'int'
>   139 | intfflush(FAR FILE *stream);
>   |   ~~^~
> serialrx_main.c:143:18: warning: passing argument 1 of 'fflush' makes
> pointer from integer without a cast [-Wint-conversion]
>   143 |   fflush(stdout);
>   |  ^~
>   |  |
>   |  int
> In file included from serialrx_main.c:31:
> /mnt/d/BertDocs/nuttxspace12/nuttx/include/stdio.h:139:25: note: expected
> 'FILE *' {aka 'struct file_struct *'} but argument is of type 'int'
>   139 | intfflush(FAR FILE *stream);
>   |   ~~^~
> serialrx_main.c:153:22: warning: passing argument 1 of 'fflush' makes
> pointer from integer without a cast [-Wint-conversion]
>   153 |   fflush(stdout);
>   |  ^~
>   |  |
>   |  int
> In file included from serialrx_main.c:31:
> /mnt/d/BertDocs/nuttxspace12/nuttx/include/stdio.h:139:25: note: expected
> 'FILE *' {aka 'struct file_struct *'} but argument is of type 'int'
>   139 | intfflush(FAR FILE *stream);
>   |   ~~^~
> serialrx_main.c:182:10: warning: passing argument 1 of 'fflush' makes
> pointer from integer without a cast [-Wint-conversion]
>   182 |   fflush(stdout);
>   |  ^~
>   |  |
>   |  int
> In file included from serialrx_main.c:31:
> /mnt/d/BertDocs/nuttxspace12/nuttx/include/stdio.h:139:25: note: expected
> 'FILE *' {aka 'struct file_struct *'} but argument is of type 'int'
>   139 | intfflush(FAR FILE *stream);
>   |   ~~^~
> serialrx_main.c:198:10: warning: passing argument 1 of 'fflush' makes
> pointer from integer without a cast [-Wint-conversion]
>   198 |   fflush(stderr);
>   |  ^~
>   |  |
>   |  int
> In file included from serialrx_main.c:31:
> /mnt/d/BertDocs/nuttxspace12/nuttx/include/stdio.h:139:25: note: expected
> 'FILE *' {aka 

Re: [DISCUSS] Out-of-tree builds should be standard?

[Xiang Xiao]

NuttX supports two build system, when a new feature is added, three cases
can happen:

   1. Modify Makefile only
   2. Modify CMakelist.txt only
   3. Modify both Makefile and CMakelist.txt

Since different people use different workflows, tools and have different
priorities. What I suggest it that:

   1. Spend your time to review the code and prompt the contributor update
   both build system at the same time
   2. Improve CI to catch the mismatch automatically

Only CI can ensure that your features never stop work. For example, Windows
frequently fails the build since nobody spends time to set up Windows CI.
CI always uses make before and cmake was added recently, so make is in a
much better position than cmake from my statistics.

On Fri, Aug 25, 2023 at 7:13 PM raiden00pl  wrote:

> > Yes, but this project has rules that were set by Greg.
>
> People have limited resources and the NuttX community is not numerous.
> The question is what is more valuable for the project - adding
> functionality supported
> only by cmake, hoping someone will add support for make later, or not
> adding any
> functionality at all?
> Is it against INVIOLABLES? Probably yes.
> Is it better for the project? Probably yes.
> Do we have the resources to act differently? Probably not.
>
> > This situation would not have happened if people had not introduced
> > cmake in the first place
>
> Nobody is forcing anyone to use cmake. Nobody is forcing anyone to use
> cmake-dependent features. You want cmake functionality in make but the
> previous contributor didn't add it for some reason - feel free to add it,
> but
> don't force others to do your work for you.
> Without cmake, there would be no nrf53, nrf91, dual core h7 and many others
> that will soon appear due to easier integration with external projects and
> easy
> build for multiple images. ...Or they won't, if the integration with make
> takes so
> much time and irritation that contributors give up PRs to the upstream.
>
> pt., 25 sie 2023 o 12:35 Sebastien Lorquet 
> napisał(a):
>
> > Hi,
> >
> > Le 25/08/2023 à 11:48, raiden00pl a écrit :
> > > Forcing people using cmake to support make is the worst thing that can
> > > happen.
> > > Open source is voluntary, any contribution is voluntary, any form of
> > > forcing
> > > others to do anything is unacceptable.
> > > Modifying both systems will be good practice (and an act of kindness),
> > but
> > > only
> > > if the contributor wants it. In other cases, a community interested in
> a
> > > particular
> > > build system should take care of it.
> >
> > Yes, but this project has rules that were set by Greg.
> >
> > This situation would not have happened if people had not introduced
> > cmake in the first place
> >
> > Sebastien
> >
> >
>

Re: how to enable RPMSG...

[Xiang Xiao]

On Tue, Jul 25, 2023 at 10:48 PM Ed Sutter  wrote:

> Xiang,
> Thanks for your responses.
> I'm still trying to get my head around OpenAMP, RPMSG and RPTUN.
>
> I see RPMSG mentioned almost everywhere OpenAMP is discussed; however, no
> mention of RPTUN.  Is RPTUN something unique to Nuttx or is it actually
> part of OpenAMP?
>
>
Yes, RPTUN is specific to NuttX, which include:

   1. Implement remoteproc_ops and image_store_ops
   2. Add new API to bind rpmsg device and driver

 Basically, RRPTUN converts OpenAMP porting interface to match NuttX driver
model(e.g. file base) and provides the stock implementation.

I've already got M7-A53 communication hacked-in-but-working using NXP's
> RPMSG-lite.  I believe all I need beyond what I have is to make the
> interface look like a bi-directional serial port in Nuttx,


rpmsg serial driver is based on OpenAMP rpmsg API plus the extension from
RPTUN. Since it provides the standard serial driver interface, you can use
it directly to access nsh on remote core.


> so I'm not sure I need RPTUN.
>
>
Yes, you need implement  rptun_ops for OpenAMP integration.


> Or maybe I'm just not getting it and RPTUN is a layer below RPMSG???
>

RPTUN is a bridge between rpmsg/remoteproc and nuttx.


> Ed

Re: how to enable RPMSG...

[Xiang Xiao]

On Mon, Jul 24, 2023 at 7:47 AM Ed Sutter  wrote:

> Thanks Xiang,
> I'm working off of 11.0.0, and don't see any of the files you point to in
> my tree.
> Can I start with one of the files below an integrate it into my
> board-specific port
> of 11.0.0 or do I need to start with a newer version?
>

Since the framework isn't changed from last year, you can reference the
implementation from nrf53/mpfs.


> Will I be missing any of the RPTUN/RPMSG generic framework if I stick with
> 11.0.0?
>

11.0.0 was released at Oct-2022, which contains all full features of the
PRTUN/RPMSG framework.


> Ed
> > On Sat, Jul 22, 2023 at 1:56 AM Ed Sutter  wrote:
> >
> >> Hi,
> >> I'm in the process of building up Nuttx on the M7 core of an iMX8MP SOC.
> >> I started by creating a new chip and board directory duplicating the
> imxrt
> >> stuff but renaming them appropriately.  From there I've been slowly
> >> creeping up in functionality with each of my needed device interfaces,
> and
> >> I'm now at a point where I want to implement the RPMSG stuff in Nuttx to
> >> talk to the Linux running on the A53s.  I've got my own hack of RPMSG
> >> running using in a Nuttx task using the NXP-SDK code.  This just allows
> me
> >> to send and receive messages between the M7 and the A53.  That works
> fine;
> >> however, I wanna redo this using whatever is the best path to take in
> Nuttx
> >> for this.
> >>
> >>
> > You need implement rptun to enable OpenAMP, here are some implementation:
> >
> https://github.com/apache/nuttx/blob/master/arch/arm/src/nrf53/nrf53_rptun.c
> >
> https://github.com/apache/nuttx/blob/master/arch/risc-v/src/mpfs/mpfs_ihc.c
> > https://github.com/apache/nuttx/blob/master/arch/sim/src/sim/sim_rptun.c
> >
> >
> >> In my case the "remote" processor I want to use RPMSG to communicate
> with
> >> is in the same silicon, so there is no external interface (serial or
> >> network), its a shared memory interface.  I tried building up the
> >> RPMSGSOCKET example but have not had any luck thus far, and then I
> noticed
> >> RPMSG_UART.   Is it reasonable to just make this look like a serial
> >> connection on the Nuttx side, but under the covers use NXP's shared
> memory
> >> hook to connect to the A53?
> >>
> >   Yes, you can use either or both between M7 and A53, please watch the
> > introduction video here:https://www.youtube.com/watch?v=6uehz3WQ0vU
> > Since you want to communicate with Linux kernel, you have to write Linux
> > rpmsg driver.
> >
> >
> >> Thanks
> >> Ed
> >>
> >>
>

Re: how to enable RPMSG...

[Xiang Xiao]

On Sat, Jul 22, 2023 at 1:56 AM Ed Sutter  wrote:

> Hi,
> I'm in the process of building up Nuttx on the M7 core of an iMX8MP SOC.
> I started by creating a new chip and board directory duplicating the imxrt
> stuff but renaming them appropriately.  From there I've been slowly
> creeping up in functionality with each of my needed device interfaces, and
> I'm now at a point where I want to implement the RPMSG stuff in Nuttx to
> talk to the Linux running on the A53s.  I've got my own hack of RPMSG
> running using in a Nuttx task using the NXP-SDK code.  This just allows me
> to send and receive messages between the M7 and the A53.  That works fine;
> however, I wanna redo this using whatever is the best path to take in Nuttx
> for this.
>
>
You need implement rptun to enable OpenAMP, here are some implementation:
https://github.com/apache/nuttx/blob/master/arch/arm/src/nrf53/nrf53_rptun.c
https://github.com/apache/nuttx/blob/master/arch/risc-v/src/mpfs/mpfs_ihc.c
https://github.com/apache/nuttx/blob/master/arch/sim/src/sim/sim_rptun.c


> In my case the "remote" processor I want to use RPMSG to communicate with
> is in the same silicon, so there is no external interface (serial or
> network), its a shared memory interface.  I tried building up the
> RPMSGSOCKET example but have not had any luck thus far, and then I noticed
> RPMSG_UART.   Is it reasonable to just make this look like a serial
> connection on the Nuttx side, but under the covers use NXP's shared memory
> hook to connect to the A53?
>

 Yes, you can use either or both between M7 and A53, please watch the
introduction video here: https://www.youtube.com/watch?v=6uehz3WQ0vU
Since you want to communicate with Linux kernel, you have to write Linux
rpmsg driver.


> Thanks
> Ed
>
>

Re: OpenFLASHLoader

[Xiang Xiao]

If you are using JLINK, you may implement a general flash programer by
reusing NuttX flash driver:
https://wiki.segger.com/SEGGER_Flash_Loader

On Thu, Jul 13, 2023 at 10:51 PM Simon Filgis <
si...@ingenieurbuero-filgis.de> wrote:

> Dear all,
>
> I need to load an FPGA bitstream via JTAG bitbang.
>
> In my case the straight way would be to use the lattice embeddedvm (ispVM)
> but I'm wondering if there is something more generic out there. Like
> OpenFPGALoader.
>
> What do you think?
>
> Regards,
>
> Simon
>
> --
> Ingenieurbüro-Filgis
> USt-IdNr.: DE305343278
> --
> sent by mobile phone
>

Re: App crashes if entrypoint but not from nsh

[Xiang Xiao]

Do you try CONFIG_INIT_STACKSIZE instead?

On Thu, Jul 6, 2023 at 1:01 AM Tim Hardisty  wrote:

> Having got u-boot to actually do something, I now have a board that
> powers up autonomously and runs nsh with no intervention from me. Quite
> relieved!
>
> As a step towards it booting my own (barely started) app I thought I'd
> play around with booting other example apps, and went for lvgldemo.
>
> This works just fine called from the nsh, but crashes very early on if
> set as the startup program.
>
> - I have traced it a little way through so know it is calling the
> lvgldemo main function. If I don't specify which demo to run in
> INIT_ARGS it correctly prompts me that it needs this argument. With it
> set to "music" or "widgets" (which both usually work just fine) it fails.
>
> - I have looked at Alan's (?) video (#047) but nothing I've missed
> according to that.
>
> - I changed the lvgldemo stack size from the default 4096 to 65535 and
> same behaviour. And done a distclean etc.
>
> - I tried another (much simpler) program (uniqueid) and that runs fine.
>
> Anyone seen anything similar? This is a good time for me to learn any
> must-know stuff when it comes to running graphics apps at startup!
>
>

Re: Bootloaders other than u-boot?

[Xiang Xiao]

On Sun, Jul 2, 2023 at 12:42 AM Gregory Nutt  wrote:

> On 7/1/2023 10:27 AM, Tim Hardisty wrote:
> > Lots of work to do a full u-boot replacement, yes. But a basic one
> > using existing NuttX stuff such as RNDIS, dfu, mtd etc. - for MY board
> > at least - would not be much work. He said...naively!
>
> You often can't know how deep the water is until you get a little wet.
>
> One thing I would mention.  A bootloader is an app alright.  But it may
> need to use some of internal OS interfaces to control things like
> interrupts, MMU, caches, etc. it may need to reset the CPU or other
> actions that normal applications would not do.  That raises a few
> architectural issues (and explains why those primitive loaders are in
> the board src/ directory and not in apps/).
>
>
Some recent added board ioctl is designed to fix this problem:
https://github.com/apache/nuttx/blob/master/include/sys/boardctl.h#L209C9-L211
The real boot could delegate to the board specific code.


> Ideally it would run in supervisor mode with a FLAT memory model.
>
>

Re: Bootloaders other than u-boot?

[Xiang Xiao]

Why not develop a nuttx based bootloader?
We have produced more than ten million devices, all run nuttx with
different configurations in the different phases(bootloader, ota, gui app),
cores(sensor hub, audio dsp), even security mode(tee).
The benefit of this approach include:

   1. You just need write driver once and reuse in all possible scenario
   2. Don't need learn and switch the development environment


On Sat, Jul 1, 2023 at 5:56 PM Tim Hardisty  wrote:

> Not directly related to NuttX but I hope you don't mind me asking here.
>
> Bottom line: has anyone used any bootloader other than u-boot to allow usb
> dfu or RNDIS-type firmware updates, and that easily boot NuttX?
>
> My board uses the usual SAMA5 methodology of AT91
> bootstrap->u-boot->NuttX. It all works as expected, as far as running code
> is concerned, but I simply cannot get u-boot to enable the USB port for DFU
> or RNDIS functions and I need the product to have user-friendly in-field
> updates, ideally without me having to write any PC update software.
>
> U-boot has defeated me for over 2 weeks now, and simply refuses to even
> "activate" the USB (i.e. a PC never even sees the connect let alone any
> functionality: yes, the USB port does work for NuttX and sam-ba code
> flashing); device trees are the invention of the devil; and an unholy
> apparent mish mash of driver model and "legacy" modes; no SAM boards in the
> u-boot trees appearing to have relevant config options I can copy. Add to
> that: using latest u-boot rather than the 2020 version I currently have
> "working"means it won't even find a console serial port any more, the lack
> of response on the relevant AT91 u-boot github to questions, and I am just
> about ready to give up :(
>
> But I need a bootloader!
>
>

Re: C++ Support

[Xiang Xiao]

Mark, could you try this patch:
https://github.com/apache/nuttx/pull/9610

On Sun, Jun 25, 2023 at 3:50 PM Mark Stevens  wrote:

> So a little more progress on this one.
>
> Turning on C++ Exception support for the PicoW:nsh build configuration
> generates errors:
>
> CC:  pthread/pthread_testcancel.c
> machine/arm/gnu_unwind_find_exidx.c:32:8: error: unknown type name
> '__EIT_entry'
>32 | static __EIT_entry *__exidx_start_elf;
>   |^~~
>
> Configuration is:
>
> - C++ enabled
> - uCLib++ library
> - GNU low level support
> - Exceptions enabled.
>
> Turning exceptions off gives me missing symbols, guess that is going to be
> a missing library in the link phase.
>
> Regards,
> Mark
> _
> Blog: blog.thepcsite.co.uk
> Twitter: @nevynuk
>
>
>
>
>
> > On 24 Jun 2023, at 18:59, Mark Stevens  wrote:
> >
> > I’m currently working on a home project with NuttX and my preferred
> language is really C++ for what I am doing.  I can work in C if necessary
> but I’d like to use some C++ features.
> >
> > So far I have managed to get the framework of what is needed configured
> and the next step is to start some application implementation.  So I’m
> thinking core logic unit tested using the simulator and basic C++
> application on the laptop.
> >
> > So I have started a fairly basic test scenario and hit a problem with
> the C++ support in NuttX.
> >
> > My application is going to be using some of the STL, at the moment I’m
> hitting issues with string and vector, two fairly basic and simple
> classes.  I’m not getting any application to compile successfully.  I have
> tried:
> >
> > - Basic C++ support
> > - LLVM with both the LLVM and GNU low level libraries
> > - uCLib++ with both LLVM and GNU libraries
> >
> > I have tried this with both the sim:nsh and the raspberrypi-pico-w:nsh
> configurations.
> >
> > The classes I am using are fairly basic so I could implement them
> myself, I would prefer to use a library version though.
> >
> > Has anyone managed to use the C++ libraries with NuttX ?
> >
> > Regards,
> > Mark
> > _
> > Blog: blog.thepcsite.co.uk
> > Twitter: @nevynuk
> >
> >
> >
> >
> >
>
>

Re: sendfile() problems - Destination unreachable

[Xiang Xiao]

On Sun, Jun 4, 2023 at 10:40 PM Fotis Panagiotopoulos 
wrote:

> As said, this is sendfile(). I do not have control on the size of the
> chunks sent. sendfile is also using TCP.
>
> So, sendfile cannot take advantage of PMTUD?
>

>From thttps://
github.com/apache/nuttx/blob/master/net/tcp/tcp_sendfile.c#L259-L262:
sendfile never sends one package larger than MSS which should smaller than
MTU in most case.



> On Sun, Jun 4, 2023 at 5:33 PM Xiang Xiao 
> wrote:
>
> > On Sun, Jun 4, 2023 at 10:23 PM Fotis Panagiotopoulos <
> f.j.pa...@gmail.com
> > >
> > wrote:
> >
> > > I am trying this new PMTUD functionality, but it seems it doesn't work.
> > >
> > > I have configured:
> > > CONFIG_NET_ICMP_PMTU_ENTRIES = 10
> > > CONFIG_NET_ICMP_PMTU_TIMEOUT = 10
> > >
> > > Again I see lots of segments being sent (all with size 1400), and all
> of
> > > them are responded with the same ICMP reply.
> > > A couple of retransmissions are attempted, and then the connection is
> > > reset.
> > >
> >
> > The option can discover the minimal MTU from the source to the
> destination.
> > You need to use PMTU to split your data into small packets(<= PMTU) by
> > yourself to improve the efficiency.
> >
> >
> > >
> > > I try again the same procedure (now that the system had the chance to
> > > discover the maximum PMTU), but it still fails.
> > > Again all segments have a size of 1400, instead of less.
> > >
> >
> > If you don't split your package to fit MTU by yourself, you have
> > to enable NET_IPFRAG. But I would suggest that.you switch UDP to TCP
> > because the protocol you implement on top of UDP is likely very
> > inefficient.
> >
> >
> > >
> > > I am using sendfile() in case this matters.
> > >
> > > On Tue, May 30, 2023 at 12:28 AM Fotis Panagiotopoulos <
> > > f.j.pa...@gmail.com>
> > > wrote:
> > >
> > > > I can see that there are two different problems with MTU.
> > > > They are completely independent from each other, so let's start with
> > the
> > > > simple case first.
> > > >
> > > > I am testing on an STM32F427, using Ethernet.
> > > >
> > > > As previously noted, the following code will cause the running task
> to
> > > > hang.
> > > >
> > > > netlib_set_mtu(CONFIG_NETIF_DEV_NAME, 1500);
> > > >
> > > > int sd = socket(AF_INET, SOCK_DGRAM, 0);
> > > >
> > > > struct sockaddr_in server;
> > > > server.sin_family  = AF_INET;
> > > > server.sin_port= 1000;
> > > > server.sin_addr.s_addr = inet_addr("192.168.1.235");
> > > >
> > > > uint8_t * data = malloc(2048);
> > > > memset(data, 0xAA, 2048);
> > > >
> > > > sendto(sd, data, 2048, 0, (struct sockaddr*), sizeof(server));
> > > >
> > > > close(sd);
> > > >
> > > > As you can see, the MTU is set to 1500, and then I try to send a UDP
> > > > datagram with a larger size (2048).
> > > > Indeed `devif_send()` fails, and the aforementioned semaphore is
> never
> > > > posted.
> > > >
> > > > (This is without buffering in UTP, in case this is important).
> > > >
> > > > This draft PR, provides a solution to the issue.
> > > > https://github.com/apache/nuttx/pull/9423
> > > >
> > > > If this is correct, I will also check buffered UDP, and other uses of
> > > > devif_send().
> > > >
> > > > Alternatively, devif_send() may be changed to actually return an
> error
> > > > code (instead of returning void), so improved error handling can take
> > > place.
> > > >
> > > >
> > > > On Mon, May 29, 2023 at 1:41 PM Fotis Panagiotopoulos <
> > > f.j.pa...@gmail.com>
> > > > wrote:
> > > >
> > > >> The failure scenario is a bit more complicated...
> > > >>
> > > >> Give me some time and I will provide a correct and reproducible
> > example,
> > > >> with a clear explanation.
> > > >>
> > > >> On Mon, May 29, 2023, 13:27 Fotis Panagiotopoulos <
> > f.j.pa...@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> > sendfile should return an error in this case, but senfile should
> > o

Re: sendfile() problems - Destination unreachable

[Xiang Xiao]

On Sun, Jun 4, 2023 at 10:23 PM Fotis Panagiotopoulos 
wrote:

> I am trying this new PMTUD functionality, but it seems it doesn't work.
>
> I have configured:
> CONFIG_NET_ICMP_PMTU_ENTRIES = 10
> CONFIG_NET_ICMP_PMTU_TIMEOUT = 10
>
> Again I see lots of segments being sent (all with size 1400), and all of
> them are responded with the same ICMP reply.
> A couple of retransmissions are attempted, and then the connection is
> reset.
>

The option can discover the minimal MTU from the source to the destination.
You need to use PMTU to split your data into small packets(<= PMTU) by
yourself to improve the efficiency.


>
> I try again the same procedure (now that the system had the chance to
> discover the maximum PMTU), but it still fails.
> Again all segments have a size of 1400, instead of less.
>

If you don't split your package to fit MTU by yourself, you have
to enable NET_IPFRAG. But I would suggest that.you switch UDP to TCP
because the protocol you implement on top of UDP is likely very inefficient.


>
> I am using sendfile() in case this matters.
>
> On Tue, May 30, 2023 at 12:28 AM Fotis Panagiotopoulos <
> f.j.pa...@gmail.com>
> wrote:
>
> > I can see that there are two different problems with MTU.
> > They are completely independent from each other, so let's start with the
> > simple case first.
> >
> > I am testing on an STM32F427, using Ethernet.
> >
> > As previously noted, the following code will cause the running task to
> > hang.
> >
> > netlib_set_mtu(CONFIG_NETIF_DEV_NAME, 1500);
> >
> > int sd = socket(AF_INET, SOCK_DGRAM, 0);
> >
> > struct sockaddr_in server;
> > server.sin_family  = AF_INET;
> > server.sin_port= 1000;
> > server.sin_addr.s_addr = inet_addr("192.168.1.235");
> >
> > uint8_t * data = malloc(2048);
> > memset(data, 0xAA, 2048);
> >
> > sendto(sd, data, 2048, 0, (struct sockaddr*), sizeof(server));
> >
> > close(sd);
> >
> > As you can see, the MTU is set to 1500, and then I try to send a UDP
> > datagram with a larger size (2048).
> > Indeed `devif_send()` fails, and the aforementioned semaphore is never
> > posted.
> >
> > (This is without buffering in UTP, in case this is important).
> >
> > This draft PR, provides a solution to the issue.
> > https://github.com/apache/nuttx/pull/9423
> >
> > If this is correct, I will also check buffered UDP, and other uses of
> > devif_send().
> >
> > Alternatively, devif_send() may be changed to actually return an error
> > code (instead of returning void), so improved error handling can take
> place.
> >
> >
> > On Mon, May 29, 2023 at 1:41 PM Fotis Panagiotopoulos <
> f.j.pa...@gmail.com>
> > wrote:
> >
> >> The failure scenario is a bit more complicated...
> >>
> >> Give me some time and I will provide a correct and reproducible example,
> >> with a clear explanation.
> >>
> >> On Mon, May 29, 2023, 13:27 Fotis Panagiotopoulos 
> >> wrote:
> >>
> >>> > sendfile should return an error in this case, but senfile should only
> >>> be
> >>> > used with TCP, not UDP, since sendfile doesn't have any logic to ack
> or
> >>> > retry..
> >>>
> >>> Sorry if this wasn't clear. This last test was with plain old
> `send()`...
> >>>
> >>> I opened a UDP socket, and used `send()` to transmit a buffer larger
> >>> than the MTU.
> >>> Instead of getting an error, the application hangs indefinitely.
> >>> `devif_send()` is called periodically, but of course it always fails.
> >>>
> >>>
> >>>
> >>> On Mon, May 29, 2023 at 1:13 PM Xiang Xiao 
> >>> wrote:
> >>>
> >>>> On Mon, May 29, 2023 at 5:02 PM Fotis Panagiotopoulos <
> >>>> f.j.pa...@gmail.com>
> >>>> wrote:
> >>>>
> >>>> > > You need to enable IP fragmentation in this case, which is also
> >>>> added
> >>>> > > recently and disabled by default:
> >>>> > > https://github.com/apache/nuttx/pull/8059
> >>>> > <https://github.com/apache/nuttx/pull/8059>
> >>>> > > Otherwise, any packet bigger than MTU will be dropped silently.
> >>>> >
> >>>> > Yes, this is the expected behavior.
> >>>> > But, instead of dropping the packet, the system hangs because the
> >>>> semaphore

Re: sendfile() problems - Destination unreachable

[Xiang Xiao]

On Mon, May 29, 2023 at 5:02 PM Fotis Panagiotopoulos 
wrote:

> > You need to enable IP fragmentation in this case, which is also added
> > recently and disabled by default:
> > https://github.com/apache/nuttx/pull/8059
> <https://github.com/apache/nuttx/pull/8059>
> > Otherwise, any packet bigger than MTU will be dropped silently.
>
> Yes, this is the expected behavior.
> But, instead of dropping the packet, the system hangs because the semaphore
> is never posted.
> It just tries endlessly to call devif_send() which always fails.
>
>
sendfile should return an error in this case, but senfile should only be
used with TCP, not UDP, since sendfile doesn't have any logic to ack or
retry..


>
>
> On Mon, May 29, 2023 at 11:42 AM Xiang Xiao 
> wrote:
>
> > On Sun, May 28, 2023 at 11:55 PM Fotis Panagiotopoulos <
> > f.j.pa...@gmail.com>
> > wrote:
> >
> > > While experimenting with MTU, and checking the stability of my system,
> I
> > > noticed the following.
> > >
> > > I try to send a UDP datagram that is larger than the configured MTU.
> > > In this case, the offending thread seems to hang indefinitely (or at
> > least
> > > waiting for a very long timeout?)
> > >
> >
> > You need to enable IP fragmentation in this case, which is also added
> > recently and disabled by default:
> > https://github.com/apache/nuttx/pull/8059
> > Otherwise, any packet bigger than MTU will be dropped silently.
> >
> >
> > > The problem seems to be this line:
> > >
> > >
> >
> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L197
> > > `devif_send()` fails because the datagram is too large, but
> > > `pstate->st_sem` is never posted (the code returns immediately).
> > >
> > > This leaves the sending task to be blocked here:
> > >
> > >
> >
> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L469
> > >
> > > Shouldn't this failure also post the semaphore?
> > > And let the code proceed returning an error in `send()`?
> > >
> > >
> > > On Sun, May 28, 2023 at 5:26 PM Fotis Panagiotopoulos <
> > f.j.pa...@gmail.com
> > > >
> > > wrote:
> > >
> > > >
> > > > On Sat, May 27, 2023 at 5:35 PM Xiang Xiao <
> xiaoxiang781...@gmail.com>
> > > > wrote:
> > > >
> > > >> On Sat, May 27, 2023 at 8:19 PM Fotis Panagiotopoulos <
> > > >> f.j.pa...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hello,
> > > >> >
> > > >> > I encounter some problems using sendfile().
> > > >> >
> > > >> > I am using sendfile to... send a file to a remote server, with my
> > own
> > > >> > implementation of an FTP client.
> > > >> > sendfile() indeed starts to transmit chunks of the file, but as I
> > see
> > > in
> > > >> > Wireshark, I get an ICMP response "Destination unreachable
> > > >> (Fragmentation
> > > >> > needed)".
> > > >> > I have verified that the Ethrenet MTU is correctly set to 1500.
> > > >> >
> > > >> > I tried lowering the MTU a lot (1000 bytes), and the problem is
> > > solved.
> > > >> > Communication succeeds.
> > > >> >
> > > >> > This raises some questions, and indicates some potential bugs:
> > > >> >
> > > >> > 1. Why is there a problem with MTU in the first place? Shouldn't
> MTU
> > > be
> > > >> > negotiated? (Is this functionality available in NuttX?)
> > > >> >
> > > >>
> > > >> MTU isn't negotiated but a physical attribute of your
> > transport(netdev).
> > > >> On
> > > >> the other hand, PMTU could be discovered from ICMP.
> > > >>
> > > >
> > > > I am not very familiar with MTU negotiation, so it seems that it
> > doesn't
> > > > happen in the network layer that I thought...
> > > >
> > > >
> > > >>
> > > >>
> > > >> > 2. Why is the ICMP response not handled? It seems that sendfile()
> > just
> > > >> > ignores it and continues to send chunks, nevertheless.
> > > >> >
> > > >>
> > 

Re: sendfile() problems - Destination unreachable

[Xiang Xiao]

On Sun, May 28, 2023 at 11:55 PM Fotis Panagiotopoulos 
wrote:

> While experimenting with MTU, and checking the stability of my system, I
> noticed the following.
>
> I try to send a UDP datagram that is larger than the configured MTU.
> In this case, the offending thread seems to hang indefinitely (or at least
> waiting for a very long timeout?)
>

You need to enable IP fragmentation in this case, which is also added
recently and disabled by default:
https://github.com/apache/nuttx/pull/8059
Otherwise, any packet bigger than MTU will be dropped silently.


> The problem seems to be this line:
>
> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L197
> `devif_send()` fails because the datagram is too large, but
> `pstate->st_sem` is never posted (the code returns immediately).
>
> This leaves the sending task to be blocked here:
>
> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L469
>
> Shouldn't this failure also post the semaphore?
> And let the code proceed returning an error in `send()`?
>
>
> On Sun, May 28, 2023 at 5:26 PM Fotis Panagiotopoulos  >
> wrote:
>
> >
> > On Sat, May 27, 2023 at 5:35 PM Xiang Xiao 
> > wrote:
> >
> >> On Sat, May 27, 2023 at 8:19 PM Fotis Panagiotopoulos <
> >> f.j.pa...@gmail.com>
> >> wrote:
> >>
> >> > Hello,
> >> >
> >> > I encounter some problems using sendfile().
> >> >
> >> > I am using sendfile to... send a file to a remote server, with my own
> >> > implementation of an FTP client.
> >> > sendfile() indeed starts to transmit chunks of the file, but as I see
> in
> >> > Wireshark, I get an ICMP response "Destination unreachable
> >> (Fragmentation
> >> > needed)".
> >> > I have verified that the Ethrenet MTU is correctly set to 1500.
> >> >
> >> > I tried lowering the MTU a lot (1000 bytes), and the problem is
> solved.
> >> > Communication succeeds.
> >> >
> >> > This raises some questions, and indicates some potential bugs:
> >> >
> >> > 1. Why is there a problem with MTU in the first place? Shouldn't MTU
> be
> >> > negotiated? (Is this functionality available in NuttX?)
> >> >
> >>
> >> MTU isn't negotiated but a physical attribute of your transport(netdev).
> >> On
> >> the other hand, PMTU could be discovered from ICMP.
> >>
> >
> > I am not very familiar with MTU negotiation, so it seems that it doesn't
> > happen in the network layer that I thought...
> >
> >
> >>
> >>
> >> > 2. Why is the ICMP response not handled? It seems that sendfile() just
> >> > ignores it and continues to send chunks, nevertheless.
> >> >
> >>
> >> It is handled by the recent addition here:
> >> https://github.com/apachey/nuttx/pull/9254
> >> <https://github.com/apache/nuttx/pull/9254>
> >> but this feature is disabled by default, you have to enable it
> manually..
> >>
> >
> > I will definitely take a look at this. Thank you.
> >
> >
> >>
> >>
> >
> >> > 3. Why sendfile() sends TCP segments without receiving any ACKs back?
> >> > AFAIK, depending on the configuration, TCP allows at most two pending
> >> > segments on the wire. But I see dozens of them, till sendfile finally
> >> > fails.
> >> >
> >> >
> >> Why only two segments? TCP can send packages until the slide window is
> >> full.
> >>
> >> Disregard this. I was confused with delayed ACKs. Which is a receiver's
> > functionality, not a sender's...
> >
> >
> >>
> >> > This last point is also verified in my MQTT client.
> >> > I have seen NuttX TCP allowing sending lots of TCP segments without
> >> ACKing
> >> > the previous data.
> >> >
> >> > So, is there any insight on the above?
> >> > Is my configuration wrong, or is there anything wrong with TCP?
> >> >
> >> > Thank you.
> >> >
> >>
> >
>

Re: sendfile() problems - Destination unreachable

[Xiang Xiao]

On Sat, May 27, 2023 at 8:19 PM Fotis Panagiotopoulos 
wrote:

> Hello,
>
> I encounter some problems using sendfile().
>
> I am using sendfile to... send a file to a remote server, with my own
> implementation of an FTP client.
> sendfile() indeed starts to transmit chunks of the file, but as I see in
> Wireshark, I get an ICMP response "Destination unreachable (Fragmentation
> needed)".
> I have verified that the Ethrenet MTU is correctly set to 1500.
>
> I tried lowering the MTU a lot (1000 bytes), and the problem is solved.
> Communication succeeds.
>
> This raises some questions, and indicates some potential bugs:
>
> 1. Why is there a problem with MTU in the first place? Shouldn't MTU be
> negotiated? (Is this functionality available in NuttX?)
>

MTU isn't negotiated but a physical attribute of your transport(netdev). On
the other hand, PMTU could be discovered from ICMP.


> 2. Why is the ICMP response not handled? It seems that sendfile() just
> ignores it and continues to send chunks, nevertheless.
>

It is handled by the recent addition here:
https://github.com/apache/nuttx/pull/9254
but this feature is disabled by default, you have to enable it manually..


> 3. Why sendfile() sends TCP segments without receiving any ACKs back?
> AFAIK, depending on the configuration, TCP allows at most two pending
> segments on the wire. But I see dozens of them, till sendfile finally
> fails.
>
>
Why only two segments? TCP can send packages until the slide window is full.


> This last point is also verified in my MQTT client.
> I have seen NuttX TCP allowing sending lots of TCP segments without ACKing
> the previous data.
>
> So, is there any insight on the above?
> Is my configuration wrong, or is there anything wrong with TCP?
>
> Thank you.
>

Re: CI tests (was: Re: [OT] Learning Makefiles)

[Xiang Xiao]

Here is the script to run the basic test:
https://github.com/apache/nuttx/blob/master/tools/ci/cirun.sh
Add a link in your config:
https://github.com/apache/nuttx/blob/master/boards/risc-v/qemu-rv/rv-virt/configs/citest/run
Build script will run it and report the result after the build pass:
https://github.com/apache/nuttx/blob/master/tools/testbuild.sh#L369C1-L380

On Wed, May 24, 2023 at 3:35 AM Nathan Hartman 
wrote:

> On Tue, May 23, 2023 at 10:06 AM Brennan Ashton
>  wrote:
> >
> > On Tue, May 23, 2023, 5:05 AM Nathan Hartman 
> > wrote:
> >
> > > On Tue, May 23, 2023 at 6:12 AM Brennan Ashton
> > >  wrote:
> > > > I have also asked in the past about cutting down on the amount of
> configs
> > > > we have checked in to be something like
> > > >
> > > > board:nsh -- only nsh and somewhat small
> > > > board:jumbo -- nsh, plus as many features as can fit and are
> interesting
> > > in
> > > > the platform.
> > > >
> > > > For sim and some other targets it would make sense to have more
> targets,
> > > > but not for every board.
> > >
> > >
> > > The idea of "board:jumbo" is very similar to what I was saying
> > > earlier. Maybe it will allow us to test fewer boards in less time but
> > > still get better test coverage. I am in favor of *better* test
> > > coverage, not less test coverage!!
> > >
> > > In the past, we talked about having some tests in CI for each PR, and
> > > then a bigger nightly test that builds all boards/configs like Greg
> > > used to do before releases. I don't think that ever happened, but ASF
> > > has a build farm separate from GitHub that we might use, or we could
> > > request from INFRA a virtual machine to set up a complete environment.
> > > Maybe that's something to think about.
> > >
> >
> >
> > I'm not sure why we would need anything new? We can still run this in
> > GitHub actions, but generally I don't think we should be having PRs merge
> > that are not passing build tests.
> >
> >
> > As for more testing of system on boards, QEMU is great for some tests and
> > there is a thin framework that does some of that work that Xiang and
> others
> > have started.  A few years ago I also gave a talk to see if there was
> > interest in working with the folks a renode.io. Their open source
> simulator
> > is what Zypher is using and at the time had minimal support, but check
> out
> > this awesome dashboard.
> >
> > https://zephyr-dashboard.renode.io/
> >
> >
> > It would be really cool if we could join forces a bit and continue to
> build
> > off that effort and improve some of the emulation as needed (some work is
> > required).
>
>
> Wow, that is cool! I must have missed it when you mentioned it
> previously. Yes, this is something the NuttX project should look into.
>
> I agree with the QEMU idea as well, for those who want to build a
> test/development setup they can run locally.
>
> Both of these ideas are very good and we should pursue them. If we do
> QEMU, we should document it, or script it, or both, so that other
> community members can reproduce it and run a test system locally.
> Personally I would like such a setup, and I am interested in helping
> to document it.
>
> Cheers,
> Nathan
>

Re: CI tests (was: Re: [OT] Learning Makefiles)

[Xiang Xiao]

Why not start the test infrastructure from sim/qemu? It's more simple to
set up and has unlimited resources. Once the sim/qemu test workflow is
ready, it isn't very hard to duplicate to the real boards.

On Tue, May 23, 2023 at 8:42 AM Alan C. Assis  wrote:

> On 5/22/23, Tomek CEDRO  wrote:
> > This is why I asked not that long ago about
> > Software-Hardware-Support-Compatibility-Matrix.. this would be really
> > big table with hardware boards in columns and features in rows with
> > green marks (or +1) where full support is confirmed, yellow (or 0)
> > meaning work-in-progress, red (or -1) meaning no support or known
> > problems.
> >
> > According to that Compatibility Matrix it would be possible to create
> > proof-based configurations to build, and builds would prove the
> > configurations.
> >
> > To be honest I have no idea how that could be implemented in such a
> > complex project as NuttX with all those possible configurations.. that
> > would really require big CI automation and I am not really familiar
> > with GH CI yet maybe this is possible.. does GH charge $ for this CI
> > operations? :-)
> >
> > When working for ARM at mbed they had this big wall of boards and such
> > tests were performed not only at build stage but also on a real
> > hardware.. each board had DAPLink that allowed flashing and serial
> > port shell that executed some test scripts :-)
> >
>
> Yes, I and Sebastien tried to create a testing farm for NuttX using
> Raspberry Pi:
>
> https://bitbucket.org/acassis/raspi-nuttx-farm/src/master/
>
> but soon we realize it will not scale well, for each board we need a
> Raspi, or a USB HUB with Power Control over on each port (to
> physically turn ON/OFF the board).
>
> In the past using Raspberry Pi Zero was a good idea:
> https://www.raspberrypi.com/news/raspberry-pi-zero/
> The price was U$ 5.00, so we could by 100 that it was not too expensive :-)
>
> Maybe a better alternative should be create some USB/HUB board using
> ESP32-S2 that we could use as bridge to program from a central
> computer/board over WiFi.
>
> BR,
>
> Alan
>

Re: [OT] Learning Makefiles

[Xiang Xiao]

CWIKI mayn't be a good place since it requires that the user has an Apache
account at least to make any change as far as I know.
It's better to be tracked by a github issue.

On Tue, May 23, 2023 at 9:27 AM Nathan Hartman 
wrote:

> On Mon, May 22, 2023 at 8:44 PM Alan C. Assis  wrote:
>
> > On 5/22/23, Tomek CEDRO  wrote:
> > > On Tue, May 23, 2023 at 1:55 AM Alan C. Assis wrote:
> > >> I think it is better to keep the documentation in a single place:
> > >> https://nuttx.apache.org/docs/latest/contributing/index.html
> > >> We're moving those documentations from confluence to our internal
> > >> repository.
> > >> So, that could be nice if you could send patches to Documentation/
> > >> directory directly.
> > >
> > > Yup, cwiki at this moment is a kind of scratchpad, all documentation
> > > is being transferred to git / documentation.. but we may still use
> > > cwiki to keep project administrative information like notes, rfc,
> > > etc..?
> > >
> >
> > No, I don't think so. Please search in the mailing list about this
> > moving docs discussion.
>
>
>
> Docs should move to repo, yes, BUT if I understand correctly Tomek is
> talking about things like ongoing discussions, debates within the
> community, such as arguments for and against adopting CMake. That is not
> documentation for users of NuttX. That is a place for the community to
> collect all the arguments in a debate so they will be easy to reference in
> one place and easy to see if there are more pros than cons, or more cons
> than pros. That should be able to go in the CWIKI and not docs in the repo,
> IMHO.
>
> Cheers
> Nathan
>

Re: [OT] Learning Makefiles

[Xiang Xiao]

The change doesn't replace Makefile with CMake, both can work. So I would
suggest the vote is "Enable CMake support".

On Sat, May 20, 2023 at 12:53 AM Gregory Nutt  wrote:

> On 5/19/2023 10:25 AM, Lwazi Dube wrote:
> > Alan,
> >
> > Can you summarize? I have not been following this PR. Is make going away?
> >
> > Thanks,
> >
> > -Lwazi
> >
> > On Fri, 19 May 2023 at 11:47, Alan C. Assis  wrote:
> >> Hi Everyone,
> >>
> >> While PR #6718 is waiting to get merged, please take a look:
> >>
> >> https://makefiletutorial.com
> >>
> >> BR,
> >>
> >> Alan
>
> The decision to switch from GNU Make to CMake is highly controversial.
> Many people are wildly in favor and a large number are strong opposed.
> This is a change that must be brought before the PMC for a vote before
> any action is taken.
>
> Special rules apply to code change votes.  We don't often do votes on
> code changes.  But such votes are critical in order to serve the
> community with fairness.
>
>
>
>

Re: BL616/618 Support

[Xiang Xiao]

bouffalolab is porting BL616 to NuttX, you can ask FAE for the detailed
schedule.

On Fri, Apr 28, 2023 at 1:40 PM Yebin Lee  wrote:

> Is anyone working on adding BL616/618 to NuttX?
> It seems it's good timing to add support for this wireless SoC as
> their recent SDK now supports WiFi, BLE, and almost all peripherals
> this chip has.
> If someone is already doing it, I'm wondering whether I can join and
> collaborate on it.
>
> YB
>

Re: Usage of mutex_t inside libc

[Xiang Xiao]

Yes, this is what I said before. You don't need to validate the stack
allocated list since it comes from the kernel stack which can't be modified
or even read by the userspace code.

On Thu, Apr 27, 2023 at 3:43 PM Ville Juven  wrote:

> Hi,
>
> I think Xiang refers to how Linux does it. It simply creates a new
> "waitobj" variable into the kernel stack by declaring it inside the
> sem_wait()-equivalent function. The wait object is created into the
> kernel stack, and a hash is calculated for the user semaphore address.
> The hash is used to identify the semaphore and the wait object. The hash
> buckets exist in a massive global hash bucket that is allocated for the
> kernel during init.
>
>
> A very simplified example could be something like this:
>
> struct waitobj_s
>
> {
>
>dq_waitlist waitlist;
>
>struct semholder_s *holders;
>
> }
>
> int nxsem_wait(FAR sem_t *sem)
> {
>
>if (sem->semcount > 0)
>
>{
>
>  // No need to block, no need for a waitobj
>
>}
>
>else
>
>{
>
>  // Need to wait / block, create a waitobj and go to sleep
>
>  struct waitobj_s waitobj; // The "frame" I was talking about is
> created here
>
>  sem->waitobj =  // Just an example, this is a BAD IDEA!
> (At this point Linux would insert the object into the hash bucket)
>
>  ...
>
>} // And the "frame" gets destroyed here
>
>return ...
>
> }
>
> The semantics of waitobj are wrong in my example, but the basic idea is
> valid still. I don't know / understand (yet) how the validity of the
> stack allocated lists is guaranteed.
>
>
> Br,
>
> Ville
>
>
> On 27.4.2023 3.25, Gregory Nutt wrote:
> > I didn't do a careful analysis, but glancing at the ARMv7-A code, it
> > looks to me that none of the things needed to create stack frames are
> > available for the kernel stack.  It looks like the system call entry
> > logic just sets the SP to the top of the kernel stack and does very
> > little more.  The address stack base and size of the stack do not get
> > set up in the TCB on a system call so creating a frame with
> > arm_stackframe() would not work.
> >
> > Am I missing something?
> >
> > On 4/26/2023 2:14 PM, Ville Juven wrote:
> >> Hi,
> >>
> >>> But, why need involve kheap here? If the security is your concern, you
> >>> should enable per-thread kernel stack in the first place. It's safe to
> >>> allocate waitlist/holder from this stack directly, since userspace code
> >>> can't touch this area too.
> >> Not from a security perspective, and of course I have the per-thread
> >> kstack
> >> enabled (kernel mode won't work without it) but how do you keep the
> >> frame
> >> from being corrupted / freed until it is not needed any more? Do you
> >> suggest the frame is allocated for each waiting thread separately ? How
> >> does that work ?
> >>
> >> I was thinking the first one who waits in sem_wait() allocates the
> >> kernel
> >> structure, puts a reference to it into e.g. the user semaphore, so other
> >> waiters and the kernel itself can find it later (reference being ID, not
> >> pointer!).
> >>
> >>> But, which scenario needs to access kernel struct through user
> >>> semaphore?
> >> It is not necessary to make the access via user semaphore, but
> >> somehow the
> >> kernel side structure needs to be tied to the semaphore, so the
> >> kernel can
> >> find it. Linux does this by a hash function, if I remember correctly.
> >> I'm
> >> not sure duplicating this is a good approach, my concern is the
> >> real-time
> >> aspect.
> >>
> >> sem_waitirq() is the problem I have, and fixing it is nasty. The
> >> semaphore
> >> is accessed via tcb->waitobj and this access can happen from
> >> interrupt or
> >> signal dispatch. This means the context of sem_wait() / sem_post() is
> >> _not_
> >> the only place the access happens. sem_waitirq() currently also modifies
> >> the semaphore counter value, so it needs access to the userspace
> >> counter.
> >> This I will do by mapping the page where the counter exists to kernel
> >> (implement something like kmap() / vmap()).
> >>
> >> If you have an alternative suggestion on how to fix sem_waitirq() I will
> >> gladly implement it.
> >>
> >> Br,
> >> Ville Juven
> >>
> >> On 

Re: Usage of mutex_t inside libc

[Xiang Xiao]

On Thu, Apr 27, 2023 at 2:18 AM Ville Juven  wrote:

> Hi,
>
> Yes exactly, this is the basic idea. Like I said, the waitlist/holder
> structure is needed only when sem_wait needs to block / wait for the
> semaphore to become available.
>
> How to protect the integrity of the stack allocated structure is still a
> bit open but one option is to use kheap as well. Semantics to be figured
> out, the solution should be feasible.
>

But, why need involve kheap here? If the security is your concern, you
should enable per-thread kernel stack in the first place. It's safe to
allocate waitlist/holder from this stack directly, since userspace code
can't touch this area too.


> My idea was to put the handle to this data into the user semaphore, however
> a pointer must not be used, a handle / integer id is needed, which then
> holds the pointer (much like files etc). As the user can spoof / destroy
> the pointer it is unsafe to do that. Spoofing the id can cause the user
> process to crash, but the kernel integrity remains.
>
>
But, which scenario needs to access kernel struct through user semaphore?


> Br,
> Ville Juven
>
> On Wed, Apr 26, 2023 at 9:09 PM Xiang Xiao 
> wrote:
>
> > But why not allocate list entry and holder temporally inside the kernel
> > thread stack? Both data is just used when the calling thread can't hold,
> > but wait the semphare, this trick is widely used inside Linux kernel.
> >
> > On Thu, Apr 27, 2023 at 1:32 AM Ville Juven 
> wrote:
> >
> > > Hi,
> > >
> > > Thanks for the explanation, it makes sense. I had an idea to move
> struct
> > > sem_s entirely to kernel and make the user use it via a handle (in
> KERNEL
> > > build) but due to the ubiquitous init macros, that is not a feasible
> > > solution.
> > >
> > > I will explore allocating the kernel structures on-demand next.
> > >
> > > What does this mean ? The kernel side structures (dq_waitlist and
> > priority
> > > inheritance holders) are only needed when the semaphore is locked and
> > > someone is actually waiting for it to be released. This means that when
> > > sem_wait() results in the thread getting blocked, the kernel part can
> be
> > > allocated here and a reference placed into the user sem_t structure.
> When
> > > the semaphore unlocks, the kernel parts can be freed.
> > >
> > > This should be completely regression free for FLAT/PROTECTED, which is
> > very
> > > important for me because I don't want to be responsible for breaking
> > such a
> > > fundamental OS API for current users ;)
> > >
> > > Br,
> > > Ville
> > >
> > > On Wed, Apr 26, 2023 at 5:26 PM Gregory Nutt 
> > wrote:
> > >
> > > >
> > > > > I have a question about using mutex_t / struct mutex_s inside
> > > > > libs/libc. The mutex_t definition is kernel internal but some
> modules
> > > > > inside the libs folder use it directly. My question is, is this
> > > > > intentional ? I don't think such modules should be using mutex_t.
> > > > >
> > > > > ...
> > > > >
> > > > > My question ? Should these be sem_t or perhaps pthread_mutex_t ?
> Both
> > > > > of which are valid user APIs.
> > > >
> > > > It was sem_t before it was changed to the non-standard mutex_t. See
> > > > commit d1d46335df6:
> > > >
> > > > commit d1d46335df6cc1bc63f1cd7e7bffe3735b8c275d
> > > > Author: anjiahao 
> > > > Date:   Tue Sep 6 14:18:45 2022 +0800
> > > >
> > > >  Replace nxsem API when used as a lock with nxmutex API
> > > >
> > > >  Signed-off-by: anjiahao 
> > > >  Signed-off-by: Xiang Xiao 
> > > >
> > > > Parts of the above which introduce inappropriate use of mutex_t could
> > be
> > > > reverted back to sem_t.  Any use of any non-standard OS function or
> > > > structure should be prohibited in application code.
> > > >
> > > > However, nuttx/libs is a special creature.  It is not confined by the
> > > > rules of POSIX interface because user-space libraries are a
> > > > non-privileged extension of the operating system.  They have intimate
> > > > knowledge of OS internals and can do non-standard things to
> accomplish
> > > > what they need to do.  So although I think it is wrong
> architecturally
> > > > for applications to use OS internals, I don't think it is wrong for
> > > > nuttx/libs to do so.  It is a friend of the OS.
> > > >
> > > > But it is should not break the KERNEL build either.
> > > >
> > > > Perhaps we will need a user-space function to allocate (and
> initialize)
> > > > kernel memory.  Use of such functions should not performed outside of
> > > > nuttx/libs, although we have no way to prohibit that use:  Such an
> API
> > > > would be a security hole. user-space code cannot access kernel
> memory,
> > > > but it can use the kernel memory address like a "handle" to interact
> > > > with the OS.
> > > >
> > > >
> > > >
> > >
> >
>

Re: Usage of mutex_t inside libc

[Xiang Xiao]

But why not allocate list entry and holder temporally inside the kernel
thread stack? Both data is just used when the calling thread can't hold,
but wait the semphare, this trick is widely used inside Linux kernel.

On Thu, Apr 27, 2023 at 1:32 AM Ville Juven  wrote:

> Hi,
>
> Thanks for the explanation, it makes sense. I had an idea to move struct
> sem_s entirely to kernel and make the user use it via a handle (in KERNEL
> build) but due to the ubiquitous init macros, that is not a feasible
> solution.
>
> I will explore allocating the kernel structures on-demand next.
>
> What does this mean ? The kernel side structures (dq_waitlist and priority
> inheritance holders) are only needed when the semaphore is locked and
> someone is actually waiting for it to be released. This means that when
> sem_wait() results in the thread getting blocked, the kernel part can be
> allocated here and a reference placed into the user sem_t structure. When
> the semaphore unlocks, the kernel parts can be freed.
>
> This should be completely regression free for FLAT/PROTECTED, which is very
> important for me because I don't want to be responsible for breaking such a
> fundamental OS API for current users ;)
>
> Br,
> Ville
>
> On Wed, Apr 26, 2023 at 5:26 PM Gregory Nutt  wrote:
>
> >
> > > I have a question about using mutex_t / struct mutex_s inside
> > > libs/libc. The mutex_t definition is kernel internal but some modules
> > > inside the libs folder use it directly. My question is, is this
> > > intentional ? I don't think such modules should be using mutex_t.
> > >
> > > ...
> > >
> > > My question ? Should these be sem_t or perhaps pthread_mutex_t ? Both
> > > of which are valid user APIs.
> >
> > It was sem_t before it was changed to the non-standard mutex_t. See
> > commit d1d46335df6:
> >
> > commit d1d46335df6cc1bc63f1cd7e7bffe3735b8c275d
> > Author: anjiahao 
> > Date:   Tue Sep 6 14:18:45 2022 +0800
> >
> >  Replace nxsem API when used as a lock with nxmutex API
> >
> >  Signed-off-by: anjiahao 
> >  Signed-off-by: Xiang Xiao 
> >
> > Parts of the above which introduce inappropriate use of mutex_t could be
> > reverted back to sem_t.  Any use of any non-standard OS function or
> > structure should be prohibited in application code.
> >
> > However, nuttx/libs is a special creature.  It is not confined by the
> > rules of POSIX interface because user-space libraries are a
> > non-privileged extension of the operating system.  They have intimate
> > knowledge of OS internals and can do non-standard things to accomplish
> > what they need to do.  So although I think it is wrong architecturally
> > for applications to use OS internals, I don't think it is wrong for
> > nuttx/libs to do so.  It is a friend of the OS.
> >
> > But it is should not break the KERNEL build either.
> >
> > Perhaps we will need a user-space function to allocate (and initialize)
> > kernel memory.  Use of such functions should not performed outside of
> > nuttx/libs, although we have no way to prohibit that use:  Such an API
> > would be a security hole. user-space code cannot access kernel memory,
> > but it can use the kernel memory address like a "handle" to interact
> > with the OS.
> >
> >
> >
>

Re: The NuttX Handbook

[Xiang Xiao]

not yet, we are busy preparing the Chinese documentation.

On Tue, Apr 25, 2023 at 10:27 PM Alan C. Assis  wrote:

> On 4/25/23, Alan C. Assis  wrote:
> > On 4/25/23, Gregory Nutt  wrote:
> >>
> >>> Yes, using the Documentation/ as base to create a book is a good idea,
> >>> but it should focus on "OS Components" and "API Reference".
> >>
> >> I don't think that the documentation should be limited to simple HowTo
> >> and WhatIs documentation.  I think it should have some technical meat
> >> too!  Like explanations of how things work and why they work that way:
> >> Theory of Operation documents.
> >>
> >> I really like Zephyr's Documentation, for example:
> >> https://docs.zephyrproject.org/latest/index.html  .  There is quite a
> >> bit of meat there; something for everyone.
> >>
> >
> > No, I meant the opposite, the Documentation/ is mostly focused on
> > Installations, HowTos, etc, it is better to be more technical.
> > Currently only "OS Components" and "API Reference" fit this criteria.
> >
> > I think our online Documentation is evolving slowly but it is getting
> > a better shape, the Zephyr documentation you cited is better organized
> > and reach more features. We can walk on that direction.
> >
> > It seems that their "PDF Handbook" is also based on their online
> > Documentation: https://docs.zephyrproject.org/2.7.4/zephyr.pdf
> >
> > Question: How we can do better, even with smaller team / resources?
> > Ideas? ChatGPT is not an option here :-D
> >
>
> Replying myself after seen confluence pages (like
> https://cwiki.apache.org/confluence/display/NUTTX/Memory+Configurations)
>
> We need some task force to move Confluence documentations to Documentation/
>
> Xiang Xiao: about that guy that you said Xiaomi was planing to
> contract to help us with Documentation, do you have some news?
>
> BR,
>
> Alan
>

Re: Telnet broken?

[Xiang Xiao]

Nathan, could you try:
https://github.com/apache/nuttx/blob/master/boards/sim/sim/sim/configs/adb/defconfig
I just try it work as expect in last mainline:
telnet 127.0.0.1 2323
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

Welcome to NuttShell(NSH) Telnet Server...
login: admin
password:
User Logged-in!
Username: admin Password: Administrator
Maybe, some configuration mismatch.

On Tue, Apr 25, 2023 at 5:06 AM Nathan Hartman 
wrote:

> On Mon, Apr 24, 2023 at 4:54 PM Gregory Nutt  wrote:
>
> >
> > > Including the doubling of characters and momentary display of some
> > > characters which seem to change rapidly?
> >
> > That probably comes from NSH.  I think it still sends an escape sequence
> > to clear to the end of the line.  So any additional garbage at the end
> > after the first four characters would only be present momentarily.
> >
> > But things are changing rapidly, obviously more rapidly than can be
> > tested.  So I would have to studly the nshlib code again to see what it
> > is doing today.
> >
> > So do you think that that last release was too shaky?  Should there be a
> > bugfix release for that one?
>
>
>
> Unfortunately I wasn't available to test and vote on the most recent
> release and I also don't currently know in exactly which commit or PR this
> broke, but it is definitely broken now, so if it has been released, then
> yes, I think it makes sense to do another release once it's fixed.
>
> I don't know how many people are using the telnet/nsh combination but I
> have to do it with this particular board since the only communication port
> it has is a network port. When telnet doesn't work correctly it really
> messes everything up for me because I can't really interface to it.
>
> Cheers
> Nathan
>

Add g_ prefix to fs file_operations and mountpt_operations

[Xiang Xiao]

>From https://nuttx.apache.org/docs/latest/contributing/coding_style.html:


   -

   Global variable prefix. All global variables begin with the prefix g_ to
   indicate the scope of variable.

Most global variables follow this guide except a few subsystem, which is
fixed here:
https://github.com/apache/nuttx/pull/9063
https://github.com/apache/nuttx/pull/9033
David suggests asking for consensus before merging, please give your
opinion, thanks.

Re: sched_lock() should be completely removed

[Xiang Xiao]

On Sun, Apr 23, 2023 at 10:42 PM Gregory Nutt  wrote:

>
> > Let's focus on the first two item:
> >
> > 1. Add sched_lock()'s ability to the enter_critical_section()
>
> Some will need to explain what this means.  What is this?


If someone wakes up a high priority thread inside the critical section(not
process interrupt), the high priority thread will run immediately. The
suggestion is disabling the schedule preempt too until the caller waits
something proactively or leaves the critical section.


>   Why is it
> needed?  There is not sufficient information in this statement to
> understand it.
>

Here is an example which replace sched lock with the critical section to
fix a SMP bug:
https://github.com/apache/nuttx/pull/3597/files#diff-2319a97549d1e962ba4aa93ef2de6fafe077a454637afb3b5d1d5a6c9ba65ab2R212-R221
But, since the critical section isn't a superset of the schedule lock, it
fails to achieve the requirement highlighted in the comment:
 /* Make sure that pre-emption is disabled to that we signal all of the
   * members of the group before any of them actually run. (This does
   * nothing if were were called from an interrupt handler).
   */

Since many people don't understand this special behaviour, we found several
timing(very hard to reproduce) issues related to this behaviour in the
mainline before, that's why Jun made this proposal.

Re: sched_lock() should be completely removed

[Xiang Xiao]

Here is an example we found recently:
https://github.com/apache/nuttx/pull/9002/files
it's always wrong to use shced_lock as lock to protect the shared resource
like this.
Here is an issue which list all fixed done before:
https://github.com/apache/nuttx/issues/3600
Since it's very hard to reproduce and fix this type of timing issue, we
need to review all places which call sched lock/unlock and fix the wrong
usage.

The suggestion:

   1. disable the userspace program call sched_lock/unlock
   2. but kernel can still call sched_lock/unlock

Since sched_lock/unlock is hard to use correctly if the caller doesn't
understand how the scheduler works. I agree with Greg that it's better
to forbid the usage sched_lock/unlock in userspace.
But, it's fine to keep it as before if you prefer to keep the freedom.
Let's focus on the first two item:

   1. Add sched_lock()'s ability to the enter_critical_section()
   2. Correct the incorrect usage of sched_lock step by step

Thanks

On Sun, Apr 23, 2023 at 7:54 PM Fotis Panagiotopoulos 
wrote:

> Hello, just a recommendation.
>
> If something is prone to missuse by the users, don't remove it.
>
> Rather:
> * Document clearly it's quirks.
> * Provide checks and assertions catching the missuse.
>
>
> On Sun, Apr 23, 2023, 14:49 Sebastien Lorquet 
> wrote:
>
> > Hello,
> >
> > I have a similar feeling about this.
> >
> > Remember, if it isn't broken, dont fix it.
> >
> > Sebastien
> >
> >
> > On 4/23/23 13:32, Tomek CEDRO wrote:
> > > hey there Yun, i am also strongly against removing working code that
> had
> > > been here for a long time has been carefully designed gives granular
> > > control etc.
> > >
> > > understanding of the code and possible misuse is up to developer.
> > >
> > > maybe the problem needs better explanation and discussion which is
> always
> > > good.
> > >
> > > but removing basic building blocks that work and already have strong
> > > dependencies is not how things are done in unix world. in fact this is
> > the
> > > kind of approach that destroys modern world (and the ict technologies).
> > it
> > > is impossible to build anything with no solid fundaments that
> constantly
> > > change. progress is about systematic development, step by step, on what
> > is
> > > already here, not by enforcing changes and removing things that work,
> > > evolution not revolution.
> > >
> > > have a good day :-)
> > >
> > > --
> > > CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
> > >
> >
>

Re: ostest with CONFIG_PRIORITY_INHERITANCE=Y

[Xiang Xiao]

On Sat, Apr 15, 2023 at 5:07 AM Petro Karashchenko <
petro.karashche...@gmail.com> wrote:

> Hello team,
>
> I want to ask a question regarding running the ostest with priority
> inheritance on the sim target. The priority inheritance test uses a special
> routine for "hog_cpu" on the sim target by calling "up_idle()". I
> understand that it is due to the simulator not having any mechanism to do
> asynchronous pre-emption and we simulate a "fake" timer interrupt by
> calling "up_idle()" (that is usually called in the idle task). The question
> is that I can pass priority inheritance ostest only if
> "CONFIG_SIM_WALLTIME_SIGNAL=y"


Yes, but up_idle in hog_cpu would be removed safely since the host timer
signal is used to interrupt the busy loop.


> and ostest run hangs in case of
> "CONFIG_SIM_WALLTIME_SLEEP=y".


No, the special hack you just described above could keep the progress
continue.


> Maybe this is something known so please
> point me to a place where it is described.
>
> Best regards,
> Petro
>

Re: Unavoidable newline in debug printfs

[Xiang Xiao]

The spec require to append the newline if the caller doesn't add one:
https://pubs.opengroup.org/onlinepubs/9699919799/functions/syslog.html
https://www.man7.org/linux/man-pages/man3/syslog.3.html
The original patch is here:
https://github.com/apache/nuttx/pull/8004
In your cae, you have to buffer the output before calling syslog. You have
to do this even without this change since other thread may call syslog
between yours.

On Thu, Apr 6, 2023 at 7:03 AM Lwazi Dube  wrote:

> Hi,
>
> I don't have a newline in my format string but it gets added anyway.
> This is a bug. I need to use a loop to print several bytes on the same
> line.
>
> Is there a reason why this is forced on everyone ?
>
>   if (stream.last_ch != '\n')
> {
>   lib_stream_putc(, '\n');
>   ret++;
> }
>
> (drivers/syslog/vsyslog.c: around line 252)
>
> It was added by this change:
>
> commit 275b2e70a47c1d8e2bfd1797a7a283c1ba14d8c4
> Author: yinshengkai 
> Date:   Fri Dec 30 17:09:45 2022 +0800
>
>syslog: fix extra line breaks in syslog when SYSLOG_COLOR_OUTPUT is
> enabled
>

Re: [Breaking change] Move nxmutex to sched

[Xiang Xiao]

When the priority inheritance is enabled on a semaphore, sem_wait will add
the current thread to the semaphore's holder table and expect that the same
thread will call sem_post later to remove it from the holder table.
If we mess this fundamental assumption by waiting/posting from different
threads, many strange things will happen. For example, let's consider
what's happen when a program send a TCP packet:

   1. The send task call sem_wait to become a holder and get IOB
   2. Network subsystem copy the user buffer into IOB and add IOB to the
   queue
   3. The send task exit and then semphare contain a dangling pointer to
   the sending tcb
   4. After network subsystem send IOB to the wire and return it  the pool,
   sem_post is called and will touch the dangling pointer

Zeng Zhaoxiu provides a patch(https://github.com/apache/nuttx/pull/5171) to
workaround this issue.
But, the semaphore holder tracking can't work as we expect anymore.

On Sat, Apr 1, 2023 at 3:52 AM Petro Karashchenko <
petro.karashche...@gmail.com> wrote:

> Xiang Xiao, is that still true for the latest code in master branch?
> And by "system will
> crash if  the priority inheritance enabled semaphore is waited or posted
> from different threads" do you mean at the point of sem_post/sem_wait or
> some system instability in general?
>
> Best regards,
> Petro
>
> On Fri, Mar 31, 2023, 10:39 PM Xiang Xiao 
> wrote:
>
> > BTW, https://github.com/apache/nuttx/pull/5070 report that the system
> will
> > crash if  the priority inheritance enabled semaphore is waited or posted
> > from different threads.
> >
> > On Sat, Apr 1, 2023 at 3:20 AM Xiang Xiao 
> > wrote:
> >
> > >
> > >
> > > On Sat, Apr 1, 2023 at 12:34 AM Gregory Nutt 
> > wrote:
> > >
> > >> On 3/31/2023 8:56 AM, Gregory Nutt wrote:
> > >> >
> > >> >> Even more. In my previous example if semaphore is posted from the
> > >> >> interrupt
> > >> >> we do not know which of TaskA or TaskB is no longer a "holder l"
> of a
> > >> >> semaphore.
> > >> >>
> > >> > You are right.  In this usage case, the counting semaphore is not
> > >> > being used for locking; it is being used for signalling an event per
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/NUTTX/Signaling+Semaphores+and+Priority+Inheritance
> > >> >
> > >> > In that case, priority inheritance must be turned off.
> > >> >
> > >> You example is really confusing because you are mixing two different
> > >> concepts, just subtly enough to be really confusing.  If a counting
> > >> semaphore is used for locking a set of multiple resources, the posting
> > >> the semaphore does not release the resource.  That is not the way that
> > >> it is done.  Here is a more believable example of how this would work:
> > >>
> > >>  1. TaskA with priority 10 allocates DMA0 channel by calling a DMA
> > >> channel allocation function.  If a DMA channel is available, it is
> > >> allocated and the allocation function takes the semaphore. TaskA
> > >> then starts DMA activity.
> > >>  2. TaskA waits on another signaling semaphore for the DMA to
> complete.
> > >>  3. TaskB with priority 20 does the same.
> > >>  4. TaskC with priority 30 wants to allocate a DMA channel.  It calls
> > >> the channel allocation function which waits on the sempahore for a
> > >> count to be available.  This boost the priority of TaskA and TaskB
> > >> to 30.
> > >>  5. When the DMA started by TaskA completes, it signals TaskA which
> > >> calls the resource deallocation function which increments the
> > >> counting semaphore, giving the count to TaskC and storing the base
> > >> priorities.
> > >>
> > >>
> > >
> > > Normally, the resource(dma channel here) is allocated from one
> > > thread/task, but may be freed in another thread/task. Please consider
> how
> > > we malloc and free memory.
> > >
> > >
> > >> The confusion arises because you are mixing the signaling logic with
> the
> > >> resource deallocation logic.
> > >>
> > >> The mm/iob logic works basically this way.  The logic more complex
> then
> > >> you would think from above.  IOBs is an example of a critical system
> > >> resource that has multiple copies and utilizes a counting sem

Re: [Breaking change] Move nxmutex to sched

[Xiang Xiao]

BTW, https://github.com/apache/nuttx/pull/5070 report that the system will
crash if  the priority inheritance enabled semaphore is waited or posted
from different threads.

On Sat, Apr 1, 2023 at 3:20 AM Xiang Xiao  wrote:

>
>
> On Sat, Apr 1, 2023 at 12:34 AM Gregory Nutt  wrote:
>
>> On 3/31/2023 8:56 AM, Gregory Nutt wrote:
>> >
>> >> Even more. In my previous example if semaphore is posted from the
>> >> interrupt
>> >> we do not know which of TaskA or TaskB is no longer a "holder l" of a
>> >> semaphore.
>> >>
>> > You are right.  In this usage case, the counting semaphore is not
>> > being used for locking; it is being used for signalling an event per
>> >
>> https://cwiki.apache.org/confluence/display/NUTTX/Signaling+Semaphores+and+Priority+Inheritance
>> >
>> > In that case, priority inheritance must be turned off.
>> >
>> You example is really confusing because you are mixing two different
>> concepts, just subtly enough to be really confusing.  If a counting
>> semaphore is used for locking a set of multiple resources, the posting
>> the semaphore does not release the resource.  That is not the way that
>> it is done.  Here is a more believable example of how this would work:
>>
>>  1. TaskA with priority 10 allocates DMA0 channel by calling a DMA
>> channel allocation function.  If a DMA channel is available, it is
>> allocated and the allocation function takes the semaphore. TaskA
>> then starts DMA activity.
>>  2. TaskA waits on another signaling semaphore for the DMA to complete.
>>  3. TaskB with priority 20 does the same.
>>  4. TaskC with priority 30 wants to allocate a DMA channel.  It calls
>> the channel allocation function which waits on the sempahore for a
>> count to be available.  This boost the priority of TaskA and TaskB
>> to 30.
>>  5. When the DMA started by TaskA completes, it signals TaskA which
>> calls the resource deallocation function which increments the
>> counting semaphore, giving the count to TaskC and storing the base
>> priorities.
>>
>>
>
> Normally, the resource(dma channel here) is allocated from one
> thread/task, but may be freed in another thread/task. Please consider how
> we malloc and free memory.
>
>
>> The confusion arises because you are mixing the signaling logic with the
>> resource deallocation logic.
>>
>> The mm/iob logic works basically this way.  The logic more complex then
>> you would think from above.  IOBs is an example of a critical system
>> resource that has multiple copies and utilizes a counting semaphore with
>> priority inheritance to achieve good real time performance.   IOB
>> handling is key logic for the correct real time operation of the overall
>> system.  Nothing we do must risk this.
>>
>>
> IOB is a very good example to demonstrate why it's a bad and dangerous
> idea to enable priority inheritance for the counting semaphore. IOB is
> normally allocated in the send thread but free in the work thread. If we
> want the priority inheritance to work as expected instead of crashing the
> system, sem_wait/sem_post must come from the same thread, which is a kind
> of lock.
>
>
>> Other places where this logic is (probably) used:
>>
>> arch/arm/src/rp2040/rp2040_i2s.c: nxsem_init(>bufsem, 0,
>> CONFIG_RP2040_I2S_MAXINFLIGHT);
>> arch/arm/src/rtl8720c/amebaz_depend.c:  if (sem_init(_sema, 0,
>> init_val))
>> arch/arm/src/sama5/sam_ssc.c: nxsem_init(>bufsem, 0,
>> CONFIG_SAMA5_SSC_MAXINFLIGHT);
>> arch/arm/src/samv7/sam_ssc.c: nxsem_init(>bufsem, 0,
>> CONFIG_SAMV7_SSC_MAXINFLIGHT);
>> arch/arm/src/stm32/stm32_i2s.c: nxsem_init(>bufsem, 0,
>> CONFIG_STM32_I2S_MAXINFLIGHT);
>> drivers/can/mcp2515.c: nxsem_init(>txfsem, 0,
>> MCP2515_NUM_TX_BUFFERS);
>> drivers/video/vnc/vnc_server.c: nxsem_init(>freesem, 0,
>> CONFIG_VNCSERVER_NUPDATES);
>> sched/pthread/pthread_completejoin.c: nxsem_init(>data_sem,
>> 0, (ntasks_waiting + 1));
>> wireless/bluetooth/bt_hcicore.c: nxsem_init(_btdev.le_pkts_sem, 0,
>> g_btdev.le_pkts);
>> wireless/bluetooth/bt_hcicore.c: nxsem_init(_btdev.ncmd_sem, 0, 1);
>> wireless/ieee802154/mac802154.c: nxsem_init(>txdesc_sem, 0,
>> CONFIG_MAC802154_NTXDESC);
>> wireless/ieee802154/mac802154.c: nxsem_init(>opsem, 0, 1);
>>
>> Maybe:
>>
>> arch/risc-v/src/bl602/bl602_os_hal.c: ret = nxsem_init(sem, 0, init);
>> arch/risc-v/src/esp32c3/esp32c3_ble_adapter.c: ret =
>> sem_init(_sem->sem, 0, init);
>> arch/risc-v/src/esp32c3/esp32c3_wifi_adapter.c: ret =
>> nxsem_init(sem, 0, init);
>> arch/xtensa/src/esp32/esp32_ble_adapter.c: ret = sem_init(sem, 0,
>> init);
>> arch/xtensa/src/esp32/esp32_wifi_adapter.c: ret = nxsem_init(sem, 0,
>> init);
>> arch/xtensa/src/esp32s3/esp32s3_wifi_adapter.c: ret =
>> nxsem_init(sem, 0, init);
>>
>>
>
>
>

Re: [Breaking change] Move nxmutex to sched

[Xiang Xiao]

On Sat, Apr 1, 2023 at 12:34 AM Gregory Nutt  wrote:

> On 3/31/2023 8:56 AM, Gregory Nutt wrote:
> >
> >> Even more. In my previous example if semaphore is posted from the
> >> interrupt
> >> we do not know which of TaskA or TaskB is no longer a "holder l" of a
> >> semaphore.
> >>
> > You are right.  In this usage case, the counting semaphore is not
> > being used for locking; it is being used for signalling an event per
> >
> https://cwiki.apache.org/confluence/display/NUTTX/Signaling+Semaphores+and+Priority+Inheritance
> >
> > In that case, priority inheritance must be turned off.
> >
> You example is really confusing because you are mixing two different
> concepts, just subtly enough to be really confusing.  If a counting
> semaphore is used for locking a set of multiple resources, the posting
> the semaphore does not release the resource.  That is not the way that
> it is done.  Here is a more believable example of how this would work:
>
>  1. TaskA with priority 10 allocates DMA0 channel by calling a DMA
> channel allocation function.  If a DMA channel is available, it is
> allocated and the allocation function takes the semaphore. TaskA
> then starts DMA activity.
>  2. TaskA waits on another signaling semaphore for the DMA to complete.
>  3. TaskB with priority 20 does the same.
>  4. TaskC with priority 30 wants to allocate a DMA channel.  It calls
> the channel allocation function which waits on the sempahore for a
> count to be available.  This boost the priority of TaskA and TaskB
> to 30.
>  5. When the DMA started by TaskA completes, it signals TaskA which
> calls the resource deallocation function which increments the
> counting semaphore, giving the count to TaskC and storing the base
> priorities.
>
>

Normally, the resource(dma channel here) is allocated from one thread/task,
but may be freed in another thread/task. Please consider how we malloc and
free memory.


> The confusion arises because you are mixing the signaling logic with the
> resource deallocation logic.
>
> The mm/iob logic works basically this way.  The logic more complex then
> you would think from above.  IOBs is an example of a critical system
> resource that has multiple copies and utilizes a counting semaphore with
> priority inheritance to achieve good real time performance.   IOB
> handling is key logic for the correct real time operation of the overall
> system.  Nothing we do must risk this.
>
>
IOB is a very good example to demonstrate why it's a bad and dangerous idea
to enable priority inheritance for the counting semaphore. IOB is normally
allocated in the send thread but free in the work thread. If we want the
priority inheritance to work as expected instead of crashing the system,
sem_wait/sem_post must come from the same thread, which is a kind of lock.


> Other places where this logic is (probably) used:
>
> arch/arm/src/rp2040/rp2040_i2s.c: nxsem_init(>bufsem, 0,
> CONFIG_RP2040_I2S_MAXINFLIGHT);
> arch/arm/src/rtl8720c/amebaz_depend.c:  if (sem_init(_sema, 0,
> init_val))
> arch/arm/src/sama5/sam_ssc.c: nxsem_init(>bufsem, 0,
> CONFIG_SAMA5_SSC_MAXINFLIGHT);
> arch/arm/src/samv7/sam_ssc.c: nxsem_init(>bufsem, 0,
> CONFIG_SAMV7_SSC_MAXINFLIGHT);
> arch/arm/src/stm32/stm32_i2s.c: nxsem_init(>bufsem, 0,
> CONFIG_STM32_I2S_MAXINFLIGHT);
> drivers/can/mcp2515.c: nxsem_init(>txfsem, 0,
> MCP2515_NUM_TX_BUFFERS);
> drivers/video/vnc/vnc_server.c: nxsem_init(>freesem, 0,
> CONFIG_VNCSERVER_NUPDATES);
> sched/pthread/pthread_completejoin.c: nxsem_init(>data_sem,
> 0, (ntasks_waiting + 1));
> wireless/bluetooth/bt_hcicore.c: nxsem_init(_btdev.le_pkts_sem, 0,
> g_btdev.le_pkts);
> wireless/bluetooth/bt_hcicore.c: nxsem_init(_btdev.ncmd_sem, 0, 1);
> wireless/ieee802154/mac802154.c: nxsem_init(>txdesc_sem, 0,
> CONFIG_MAC802154_NTXDESC);
> wireless/ieee802154/mac802154.c: nxsem_init(>opsem, 0, 1);
>
> Maybe:
>
> arch/risc-v/src/bl602/bl602_os_hal.c: ret = nxsem_init(sem, 0, init);
> arch/risc-v/src/esp32c3/esp32c3_ble_adapter.c: ret =
> sem_init(_sem->sem, 0, init);
> arch/risc-v/src/esp32c3/esp32c3_wifi_adapter.c: ret =
> nxsem_init(sem, 0, init);
> arch/xtensa/src/esp32/esp32_ble_adapter.c: ret = sem_init(sem, 0,
> init);
> arch/xtensa/src/esp32/esp32_wifi_adapter.c: ret = nxsem_init(sem, 0,
> init);
> arch/xtensa/src/esp32s3/esp32s3_wifi_adapter.c: ret =
> nxsem_init(sem, 0, init);
>
>

Re: Sensor implementation

[Xiang Xiao]

On Sun, Mar 12, 2023 at 12:12 AM Tim Hardisty  wrote:

> I submitted a PR for a driver for the Broadcom APDS-9922 ambient light and
> proximity sensor, written with what one might call the "traditional"
> method of setting up the device via ioctl, then reading data when available
> according to the device setup, via poll notify.
>
> The reviewer (Hi Alan - I'm not complaining!!) has suggested it perhaps
> ought to use the "new" sensor methodology with a sensor_lower_half_s etc.
>
> Looking at sensors that use this I can't see any that have the range of
> set-up options of this device and are just "there", in the main apart from
> calibration, for example. Nor any NuttX documentation I can see?
>
> Is this following a Linux methodology, like the power devices now do? When
> I did a power supply device drive I was pointed in the direction of Linux
> documentation that (after much reading and cogitation) helped explain what
> NuttX was essentially emulating and, with the addition of some more members
> to the regulator_desc_s struct it was then fine and I wrote it that way.
>
> I am still not convinced that this ALS/Proximity sensor necessarily fits
> the "new" sensor methodology but if someone can point me in the direction
> of relevant documentation I will gladly take a look.
>
>
Here is an intro video: https://www.youtube.com/watch?v=ESpAE6wqy9o


> Thx,
>
> TimH
>
>
>

Re: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver

[Xiang Xiao]

On Sun, Mar 12, 2023 at 2:09 AM Tomek CEDRO  wrote:

> Please consult Unix manual and source code (for instance any of the BSD).
> Linux was never a reference in Unix world, it was about to mimic Unix, but
> was not even self-compatible. Not a good reference point. See how big mess
> it introduced in current drivers implementation (i.e. drm kms) and their
> propagation to other OS. Is this similar situation here?
>
>
This change isn't specific to Linux:
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/termios.h.html


> Device driver should not provide any additional processing / silent rewrite
> of data, just provide raw data, that is then processed by the application.
> Driver may be configured by ioctl/sysclt, to enable additional / specific
> behavior (i.e. debug, buffer size, latency, etc), so the user always knows
> exactly what the driver does.
>
> I am pro Greg approach, so driver is always simple and generic, everything
> else is up to application. This is the Unix way and you will know how most
> things work, because they all work the same generic standardized way, as
> simple as possible, nothing else happens "in the background unnoticed".
>
> If a specific device needs a specific handling by a generic driver, then
> "quirks" may be used. That explicitly mark "specific" behavior for a given
> device (and only this device).
>
>
All changes are controlled by ioctl and specified by standard, please read
the spec:
https://pubs.opengroup.org/onlinepubs/9699919799/functions/tcsetattr.html
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/termios.h.html


> --
> CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
>

Re: [Breaking change] Echo character by serial driver

[Xiang Xiao]

On Sat, Mar 11, 2023 at 9:51 AM Gregory Nutt  wrote:

>
> On 3/10/2023 7:44 PM, Huang Qi wrote:
> > >>  The ECHO behavior can be disabled ONLY if TERMIOS are enabled.
> > >>  TERIMIOS is now required by POSIX and, further, if TERMIOS is not
> > >>  system, many features are now broken.  Like hiding the password when
> > >>  logging into NSH.
> > > Yes maybe we should select TERMIOS if some features really need it ?
> > Most apps don't need it, only
> > > termcurse/nsh login rely on it now in my known.
> >
> > > The those Kconfig files should select TERMIOS_SERIAL, right?
> >
> > Yes, this is a chioce, or we can make TERMIOS enabled forcely, but
> > will cause a extra 1KB overhead for all target.
> >
> That 1Kb number seems large to me.  Most of that is in the lower-half,
> UART driver, right?  If so then the size would vary dramatically from
> chip-to-chip.
>
>
Terminal setting include two part:

   1. Hardware related setting(e.g. baud rate, parity check etc)
   2. Software related setting(e.g. echo, \r\n<->\n etc)

The major code size increase comes from the first item, but it's
unfortunate that TERMIOS_SERIAL controls both settings.
So, here is my suggestion:

   1. TERMIOS_SERIAL only control the hardware related setting
   2. The software setting is always enabled
   3. isconsole decide the initial software setting
  - isconsole equals false, disable all special process
  - isconsole equals true, enable \r\n<->\n, echo and crtl+c handling
   4. terminal aware function or application change the terminal to raw
   mode and restore to the original setting before exit
   5. other normal application could assume that the terminal do all
   special process

This could achieve POSIX compliance with the minimal cost.


>
> As a percentage growth, I suppose even 1Kb is not so large.  Probably
> less then 2%
>

Re: [Breaking change] Echo character by serial driver

[Xiang Xiao]

On Fri, Mar 10, 2023 at 10:39 PM Gregory Nutt  wrote:

>
> On 3/10/2023 8:12 AM, Xiang Xiao wrote:
> > The only problem is that terminal is optional:
> >
> https://github.com/apache/nuttx/blob/master/drivers/serial/Kconfig#L167-L174
> > So, if the user doesn't enable CONFIG_SERIAL_TERMIOS in defconfig,
> readline
> > will stop working due to failure to disable ECHO.
>
> Isn't the default setting of ECHO disabled?  My understanding is the
> ECHO is always /disabled /unless it is specifically /enabled /in the
> TERMIOS settings.
>

No, it depends on isconsole:
https://github.com/apache/nuttx/blob/master/drivers/serial/serial.c#L1303-L1308
https://github.com/apache/nuttx/blob/master/drivers/serial/serial.c#L1821-L1840
https://github.com/apache/nuttx/blob/master/drivers/serial/serial.c#L339-L344
All terminal special handling(\r\n<->\n, ECHO CRTL+C) is activated if
isconsole equals true.
And all serial ports designed for shell set isconsole=true statically.


> > Three option here:
> >
> > 1. Remove SERIAL_TERMIOS and enable terminal setting always
> I think so.  I previously argued that TERMIOS should not be optional.
> It is required by POSIX.  There was discussion about the size increase,
> but I don't think there is any significant size increase due to this.
> We could measure the increase with TERMIOS enabled and disabled to see.
> > 2. readline or special tool designed for raw mode select
> SERIAL_TERMIOS
> > in their config
> That would be insufficient.  Other logic needs TERMIOS too.  Other than
> a size increase, there is no harm in enabling TERMIOS.  But if TERMIOS
> is not enabled, then that can be a source of errors.
> > 3. Keep SERIAL_TERMIOS option, but simulate terminal setting by
> change
> > isconsole flag
> My recollection is that the isconsole flag was added ONLY to assure that
> lines sent from the console terminated in CR-LF as required by most
> terminal emulators.  It never had anything to do with ECHO or other
> TERMIOS settings recently.  That extension of the semantics is not good
> and introduces lots of problems.  Just because a serial device is the
> console device, that should not generally effect its behavior.
> > Both code size and change is small for option 3.
>
> Except I don't think it is correct.
>
>

Re: [Breaking change] Echo character by serial driver

[Xiang Xiao]

The only problem is that terminal is optional:
https://github.com/apache/nuttx/blob/master/drivers/serial/Kconfig#L167-L174
So, if the user doesn't enable CONFIG_SERIAL_TERMIOS in defconfig, readline
will stop working due to failure to disable ECHO.
Three option here:

   1. Remove SERIAL_TERMIOS and enable terminal setting always
   2. readline or special tool designed for raw mode select SERIAL_TERMIOS
   in their config
   3. Keep SERIAL_TERMIOS option, but simulate terminal setting by change
   isconsole flag

Both code size and change is small for option 3.

On Fri, Mar 10, 2023 at 9:52 PM Gregory Nutt  wrote:

>
> > Thank you for this good change for better posix compliance.
>
> The TERMIOS changes are basically good and does help with POSIX
> compliance, but the console ECHO behavior is not and has nothing to do
> with POSIX compliance.  POSIX does not deal with implementation.  We can
> look at standard implementations for guidance, however:
>
> Standard implementations of readline place the console in raw mode in
> which any echo from the driver is disabled (ECHO is not selected in the
> TERMIOS).  All echo is preformed selectively by the readline
> implementation without trashing the users terminal.
>
>
>

Re: DISCUSSION - Usage of mailing lists for apache projects

[Xiang Xiao]

If so, most likely xiaomi will fork nuttx like Samuang and make our own
change. We don't want to spend the time to prepare the patch but no
feedback guarantee, since it consumes a huge resource to prepare the patch.

On Thu, Mar 9, 2023 at 6:15 PM Sebastien Lorquet 
wrote:

> Hi,
>
> This sounds like a corporate rule, does it? Not being critical here,
> this is just to understand.
>
>
> There should be no clear limit for a shared project, we just need time
> to take the best decisions.
>
> I feel that you cant force progress if there are no resources to do it.
>
> Remember, most of us are volunteers, right?
>
> That means you should not try to overcome volunteer availability by
> enforcing decisions.
>
> That also means you could have a company managed "staging repository" to
> match the speed you need for your internal products and roadmap, while
> upstream contributions are discussed and validated.
>
>
> I believe progress should happen more naturally if the issue is
> presented to us with an explanation that you actually need decisions on
> a point.
>
>
> Sebastien
>
> Le 09/03/2023 à 10:52, Xiang Xiao a écrit :
> > Yes, this is how I do normally, but we need the rule to ensure the PR
> gets
> > progress. e.g. the reviewer needs to give the feedback in one week, one
> > month or one year.
> > The reviewer has the rights to approve the change and also has duties to
> > make progress.
> >
> >
> > On Thu, Mar 9, 2023 at 5:14 PM alin.jerpe...@sony.com <
> > alin.jerpe...@sony.com> wrote:
> >
> >> Hi Xiang,
> >>
> >> Simply add some reviewers on the right side and they will be notified
> that
> >> someone asks them to step up
> >>
> >> Best regards
> >> Alin
> >>
> >>
> >> -Original Message-
> >> From: Xiang Xiao 
> >> Sent: den 9 mars 2023 10:12
> >> To: dev@nuttx.apache.org
> >> Subject: Re: DISCUSSION - Usage of mailing lists for apache projects
> >>
> >> If some PR waits for a long time without any review, how to make
> progress?
> >> For example, this PR sent two weaks ago:
> >> https://github.com/apache/nuttx/pull/8610
> >>
> >> On Thu, Mar 9, 2023 at 4:40 PM alin.jerpe...@sony.com <
> >> alin.jerpe...@sony.com> wrote:
> >>
> >>> Hi all,
> >>>
> >>> I  feel that this thread is getting too long without a real outcome
> >>>
> >>> Some observations from my daily interactions with the project:
> >>> - I like doing reviews on github and I think that many people in this
> >>> thread would agree that this flow is good.
> >>> - I like to be able to see all bugs in one place and get statistics
> >>> for the ASF reports
> >>>
> >>> What I don’t feel right
> >>> - even if I spend time daily on reviewing patches there are still
> >>> changes that I miss and it is hard to get the flow on release date
> >>> - some breaking changes are not discussed enough with the community
> >>> since there are some people that do not have time to review code on
> >> gihub.
> >>> As a way going forward I propose that we improve in 2 aspects
> >>> - All breaking commits should be discusses on dev so that people get
> >>> enough time to digest the change and even better get involved int the
> >>> flow
> >>> - all breaking changes should be documented on the release confluence
> >>> page before merging so that we don’t miss mentioning them on release
> >> date.
> >>> - there should be at least 1 independent reviewer (not from the same
> >>> company) so that a patch is merged except board changes (ex an
> >>> employee from the same company merges a patch submitted by another
> >>> employee from the same company, for a board provided by the same
> >>> company)
> >>>
> >>> Thanks
> >>> Alin
> >>>
> >>> -Original Message-
> >>> From: Alan C. Assis 
> >>> Sent: den 8 mars 2023 19:15
> >>> To: dev@nuttx.apache.org
> >>> Cc: Sebastien Lorquet 
> >>> Subject: Re: DISCUSSION - Usage of mailing lists for apache projects
> >>>
> >>> Hi Lwazi,
> >>>
> >>> It is not sarcarm, I'm talking about facts.
> >>>
> >>> Also I didn't say Sebastien points aren't valid, but is diverting from
> >>> the real issue.
> >>>

Re: Shell alias support for NSH

[Xiang Xiao]

On Thu, Mar 9, 2023 at 6:14 PM Ville Juven  wrote:

> Hi all,
>
> I'm in the process of writing support for shell aliases into NSH, but since
> the shell is the front end for NuttX I decided to make a post here for
> opinions on this matter and to open discussion for requirements, mostly to
> prevent unexpected regression issues that might arise from this.
>
> 1. Are shell aliases seen as something useful in the community ? I need
> them for a project of mine which is the reason I started implementing
> support but I'd like to know if they are seen as useful whatsoever by
> others.
>
> 2. To properly support aliases, the shell parser needs to support single
> quotes and double quotes. This support will generate a bit of extra code
> into the parser, is this seen as OK ?
>
> alias foo='bar baz'
> alias foo="bar $baz"
>
> In both cases, the parser stops on the first delimiter (whitespace) even
> though it is quoted.
>
> 3. I will implement support for *single quotes* as this is a simple case
> but not for double quotes as the rules are more complex (need to perform
> variable expansions etc). Is this seen as a blocker / fatal issue?
>
>
This feature already exist
https://github.com/apache/nuttx-apps/blob/master/nshlib/Kconfig#L126-L146


> 4. I added a Kconfig option to turn alias support on/off, what should the
> default state of this option be ? Currently it is defined as follows:
>
> config NSH_ALIAS
> bool "Enable alias support"
> default !DEFAULT_SMALL
>
> I noticed that many of the shell features are flagged behind DEFAULT_SMALL.
>
> Also the alias amount is configurable.
>
> 5. Currently aliases are implemented globally i.e. every shell shares the
> same aliases, is this seen as a problem or should I make the aliases stick
> to a shell session only ?
>
>
 It's better to follow the POSIX spec.

This is my first time posting on this board, so apologies if I did
> something improper.
>
> Br, Ville Juven
>

Re: DISCUSSION - Usage of mailing lists for apache projects

[Xiang Xiao]

Yes, this is how I do normally, but we need the rule to ensure the PR gets
progress. e.g. the reviewer needs to give the feedback in one week, one
month or one year.
The reviewer has the rights to approve the change and also has duties to
make progress.


On Thu, Mar 9, 2023 at 5:14 PM alin.jerpe...@sony.com <
alin.jerpe...@sony.com> wrote:

> Hi Xiang,
>
> Simply add some reviewers on the right side and they will be notified that
> someone asks them to step up
>
> Best regards
> Alin
>
>
> -Original Message-
> From: Xiang Xiao 
> Sent: den 9 mars 2023 10:12
> To: dev@nuttx.apache.org
> Subject: Re: DISCUSSION - Usage of mailing lists for apache projects
>
> If some PR waits for a long time without any review, how to make progress?
> For example, this PR sent two weaks ago:
> https://github.com/apache/nuttx/pull/8610
>
> On Thu, Mar 9, 2023 at 4:40 PM alin.jerpe...@sony.com <
> alin.jerpe...@sony.com> wrote:
>
> > Hi all,
> >
> > I  feel that this thread is getting too long without a real outcome
> >
> > Some observations from my daily interactions with the project:
> > - I like doing reviews on github and I think that many people in this
> > thread would agree that this flow is good.
> > - I like to be able to see all bugs in one place and get statistics
> > for the ASF reports
> >
> > What I don’t feel right
> > - even if I spend time daily on reviewing patches there are still
> > changes that I miss and it is hard to get the flow on release date
> > - some breaking changes are not discussed enough with the community
> > since there are some people that do not have time to review code on
> gihub.
> >
> > As a way going forward I propose that we improve in 2 aspects
> > - All breaking commits should be discusses on dev so that people get
> > enough time to digest the change and even better get involved int the
> > flow
> > - all breaking changes should be documented on the release confluence
> > page before merging so that we don’t miss mentioning them on release
> date.
> > - there should be at least 1 independent reviewer (not from the same
> > company) so that a patch is merged except board changes (ex an
> > employee from the same company merges a patch submitted by another
> > employee from the same company, for a board provided by the same
> > company)
> >
> > Thanks
> > Alin
> >
> > -Original Message-
> > From: Alan C. Assis 
> > Sent: den 8 mars 2023 19:15
> > To: dev@nuttx.apache.org
> > Cc: Sebastien Lorquet 
> > Subject: Re: DISCUSSION - Usage of mailing lists for apache projects
> >
> > Hi Lwazi,
> >
> > It is not sarcarm, I'm talking about facts.
> >
> > Also I didn't say Sebastien points aren't valid, but is diverting from
> > the real issue.
> >
> > The issue is not if the discussion is happening here or there, the
> > Problem is that we don't have enough reviewers.
> >
> > So, first step is that NuttX needs to increase the user base, but have
> > few users really engaged with the project, reviewing patches every
> single day.
> > Currently today he have few: Petro and Xiang are exceptional on this
> point.
> > They are my inspiration to try do more!
> >
> > Welcome back go NuttX Lwazi (I'm not been sarcastic, I'm happy to hear
> > from you again! You have a great knowledge of BLE can we need! I was
> > expecting you to share that working example of BLE application using
> > our BLE stack).
> >
> > BR,
> >
> > Alan
> >
> > On 3/8/23, Lwazi Dube  wrote:
> > > On Wed, 8 Mar 2023 at 09:55, Alan C. Assis  wrote:
> > >>
> > >> Sebastien,
> > >>
> > >> If all the discussions that happens on github start to happen here,
> > >> this mailing list will be just like the nuttx-commits mailing list.
> > >
> > > I'll take this as sarcasm. Sebastien is making a lot of valid
> > > points, in good faith, and being dismissive does not help the
> community.
> > >
> >
>

Re: DISCUSSION - Usage of mailing lists for apache projects

[Xiang Xiao]

If some PR waits for a long time without any review, how to make progress?
For example, this PR sent two weaks ago:
https://github.com/apache/nuttx/pull/8610

On Thu, Mar 9, 2023 at 4:40 PM alin.jerpe...@sony.com <
alin.jerpe...@sony.com> wrote:

> Hi all,
>
> I  feel that this thread is getting too long without a real outcome
>
> Some observations from my daily interactions with the project:
> - I like doing reviews on github and I think that many people in this
> thread would agree that this flow is good.
> - I like to be able to see all bugs in one place and get statistics  for
> the ASF reports
>
> What I don’t feel right
> - even if I spend time daily on reviewing patches there are still changes
> that I miss and it is hard to get the flow on release date
> - some breaking changes are not discussed enough with the community since
> there are some people that do not have time to review code on gihub.
>
> As a way going forward I propose that we improve in 2 aspects
> - All breaking commits should be discusses on dev so that people get
> enough time to digest the change and even better get involved int the flow
> - all breaking changes should be documented on the release confluence page
> before merging so that we don’t miss mentioning them on release date.
> - there should be at least 1 independent reviewer (not from the same
> company) so that a patch is merged except board changes (ex an employee
> from the same company merges a patch submitted by another employee from the
> same company, for a board provided by the same company)
>
> Thanks
> Alin
>
> -Original Message-
> From: Alan C. Assis 
> Sent: den 8 mars 2023 19:15
> To: dev@nuttx.apache.org
> Cc: Sebastien Lorquet 
> Subject: Re: DISCUSSION - Usage of mailing lists for apache projects
>
> Hi Lwazi,
>
> It is not sarcarm, I'm talking about facts.
>
> Also I didn't say Sebastien points aren't valid, but is diverting from the
> real issue.
>
> The issue is not if the discussion is happening here or there, the Problem
> is that we don't have enough reviewers.
>
> So, first step is that NuttX needs to increase the user base, but have few
> users really engaged with the project, reviewing patches every single day.
> Currently today he have few: Petro and Xiang are exceptional on this point.
> They are my inspiration to try do more!
>
> Welcome back go NuttX Lwazi (I'm not been sarcastic, I'm happy to hear
> from you again! You have a great knowledge of BLE can we need! I was
> expecting you to share that working example of BLE application using our
> BLE stack).
>
> BR,
>
> Alan
>
> On 3/8/23, Lwazi Dube  wrote:
> > On Wed, 8 Mar 2023 at 09:55, Alan C. Assis  wrote:
> >>
> >> Sebastien,
> >>
> >> If all the discussions that happens on github start to happen here,
> >> this mailing list will be just like the nuttx-commits mailing list.
> >
> > I'll take this as sarcasm. Sebastien is making a lot of valid points,
> > in good faith, and being dismissive does not help the community.
> >
>

Re: [Question] About block device partition support

[Xiang Xiao]

On Thu, Mar 9, 2023 at 2:34 PM KIKUCHI Takeyoshi 
wrote:

> Thanks for your reply.
>
> I see, there are hard-coded devices...
>
> Then, would it be acceptable to make an entry in CONFIG that says, for
> example, "support automatic partition parsing" and if enabled,
> automatically parse the partition when a block device is registered?
>
>
Since both builtin block/flash device and external storage(usb mass
storage, sd/emmc) exist in the complex case, the compiler time selection
may be hard to handle.


> I don't think the partition parsing in the initialization process can
> handle the case of active wire insertion and removal, such as USB
> storage and SD cards.
>
> However, the problem is how to handle the case when it is pulled out
> during mount, and I think it is difficult to know how to handle it (the
> same for Linux ?).
>
> If the implementation concept is OK, I could automatically analyze the
> partition as shown in the example below, so I would like to organize it
> a bit more and submit a pull request.
> (Device names have been changed to match Linux)
>
>
fs already has the code handle the auto mount/unmount case:
https://github.com/apache/nuttx/blob/master/fs/mount/fs_automount.c
The improvement could be added parse_block_partition to the suitable place.


> nsh> ls -l /dev/
> /dev:
>   crw-rw-rw-   0 console
>   crw-rw-rw-   0 eeprom
>   cr--r--r--   0 eeprom.uuid
>   crw-rw-rw-   0 i2c1
>   crw-rw-rw-   0 kmsg
>   crw-rw-rw-   0 loop
>   brw-rw-rw-  1929379840 mmcblk0
>   brw-rw-rw-   629145600 mmcblk0p1
>   brw-rw-rw-  1299185664 mmcblk0p2
>   brw-rw-rw-  2016411648 mmcblk1
>   brw-rw-rw-   268435456 mmcblk1p1
>   brw-rw-rw-  1746927616 mmcblk1p2
>   crw-rw-rw-   0 null
>   crw-rw-rw-   0 rtc0
>   crw-rw-rw-   0 telnet
>   crw-rw-rw-   0 ttyS0
>   crw-rw-rw-   0 tun
>   crw-rw-rw-   0 zero
>
>
> KIKUCHI Takeyoshi
>
>
> On 2023/03/09 15:11, Xiang Xiao wrote:
> > On Thu, Mar 9, 2023 at 6:05 AM KIKUCHI Takeyoshi <
> kiku...@centurysys.co.jp>
> > wrote:
> >
> >> Hi, All.
> >>
> >> The code to parse block device partitions is included (as well as the
> >> code to register partitions as devices), but it seems that the process
> >> to parse partitions is not called from the location where the block
> >> device is registered.
> >> (There is a function called "register_blockpartition", but it is not
> >> called from anywhere.)
> >>
> >> Is there an intentional reason for not parsing partitions?
> >
> >
> > It's because some devices hardcode the partition layout in code. So, the
> > board file will call parse_block_partition after creating  the block
> device
> > and finish the initialization in the original design.
> >
> >
> >> Would it be a
> >> problem to add an automatic partition parsing process call?
> >>
> >>
> > It could be added a new function to simplify the common usage.
> >
> > KIKUCHI Takeyoshi
> >>
> >> ---
> >> kiku...@centurysys.co.jp
> >>
> >
>
> --
> kiku...@centurysys.co.jp
>

Re: [Question] About block device partition support

[Xiang Xiao]

On Thu, Mar 9, 2023 at 6:05 AM KIKUCHI Takeyoshi 
wrote:

> Hi, All.
>
> The code to parse block device partitions is included (as well as the
> code to register partitions as devices), but it seems that the process
> to parse partitions is not called from the location where the block
> device is registered.
> (There is a function called "register_blockpartition", but it is not
> called from anywhere.)
>
> Is there an intentional reason for not parsing partitions?


It's because some devices hardcode the partition layout in code. So, the
board file will call parse_block_partition after creating  the block device
and finish the initialization in the original design.


> Would it be a
> problem to add an automatic partition parsing process call?
>
>
It could be added a new function to simplify the common usage.

KIKUCHI Takeyoshi
>
> ---
> kiku...@centurysys.co.jp
>

Re: Serial console BROKEN

[Xiang Xiao]

On Thu, Mar 9, 2023 at 3:07 AM Gregory Nutt  wrote:

>
>  I imagine that this is occurring because readline also echos the
> input:
> 
> 
> 
> >>
> https://github.com/apache/nuttx-apps/blob/master/system/readline/readline_common.c#L644
>  The low-level serial driver should not echo just because /isconsole
> /is
>  true.  Console echo is always handled by the application.  I would say
>  that that is a bug.
> 
> >>>   From the spec:
> >>> https://pubs.opengroup.org/onlinepubs/7908799/xsh/termios.h.html
> >>> the serial driver should echo the char if ECHO is enabled. Since NuttX
> >>> doesn't follow the spec, it makes many POSIX compliant interactive
> shells
> >>> doesn't work very well.
> >>> We have to make the breaking change to fix the wrong initial
> >> implementation.
> >>
> >> I still seems like the isconsole does not belong in the conditional:
> >>
> >>
> >>
> https://github.com/apache/nuttx/commit/68384e9db42e254b2cf6720bc3380aebdd2b298c
> >>
> >> if (dev->isconsole
> >> #ifdef CONFIG_SERIAL_TERMIOS
> >> || (dev->tc_iflag & ECHO)
> >> #endif
> >>)
> >>   {
> >>
> >> Currently:
> >>
> >>893   if (
> >>894 #ifdef CONFIG_SERIAL_TERMIOS
> >>895   dev->tc_iflag & ECHO
> >>896 #else
> >>897   dev->isconsole
> >>898 #endif
> >>899  )
> >>900 {
> >>
> >> If CONFIG_SERIAL_TERMIOS is not set then shouldn't the entire 'if '
> >> condition should be removed?
> >>
> >  From my understanding, when CONFIG_SERIAL_TERMIOS isn't set whether
> serial
> > driver do some special terminal handling(e.g. \n<->\r\n and echo) is
> > controlled by isconsole flag:
> >
> > 1. isconsole equals false, all special handling is disable
> > 2. isconsole equals true, all special handling is enable
> >
> > So, the check needs to be kept to ensure that non-console uart doesn't
> add
> > the unxpected process.
> >
> If CONFIG_SERIAL_TERMIOS is not selected then non-consoles will do
> nothing.  In order to restore legacy behavior you would have to do this:
>
>+#ifdef CONFIG_SERIAL_TERMIOS
>if (
>-#ifdef CONFIG_SERIAL_TERMIOS
>   dev->tc_iflag & ECHO
> -#else - dev->isconsole -#endif
> )
>...
>   }
> +#endif
>
> That should eliminate the double character echo.
>
>
So the command line tool has to do the different thing(echo or non echo by
self) based on CONFIG_SERIAL_TERMIOS?


> I suppose that you could also eliminate the echo in readline() (and
> cle()???) but that would probably mess up the command line editing since
> the in the edit commands would also be echoed by the serial driver.
>
>
Since the code to handle the special process is very small, is it better to
always allow application change ECHO and OPOST through TCSETS? So, the
special function or program can disable ECHO/OPOST programmatically.


> By the way, the reason that this is implemented in a non-standard way like
> this with a flag is historical.  The serial driver is one of the older
> parts of the system.  There was no TERMIOS support in NuttX until much,
> much later in the development.  The serial driver was just a flat,
> read/write character device like any other.  readline() fakes all the the
> necessary operations of raw mode and other termios settings.
>
>
Yes, but it's good time to follow the POSIX spec that moves the special
process from readline to serial driver and make TCSETS for ECHO and
OPOST always exist.

Re: Serial console BROKEN

[Xiang Xiao]

On Thu, Mar 9, 2023 at 2:17 AM Gregory Nutt  wrote:

>
> >> I imagine that this is occurring because readline also echos the input:
> >>
> >>
> >>
> https://github.com/apache/nuttx-apps/blob/master/system/readline/readline_common.c#L644
> >>
> >> The low-level serial driver should not echo just because /isconsole /is
> >> true.  Console echo is always handled by the application.  I would say
> >> that that is a bug.
> >>
> >  From the spec:
> > https://pubs.opengroup.org/onlinepubs/7908799/xsh/termios.h.html
> > the serial driver should echo the char if ECHO is enabled. Since NuttX
> > doesn't follow the spec, it makes many POSIX compliant interactive shells
> > doesn't work very well.
> > We have to make the breaking change to fix the wrong initial
> implementation.
>
> I still seems like the isconsole does not belong in the conditional:
>
>
> https://github.com/apache/nuttx/commit/68384e9db42e254b2cf6720bc3380aebdd2b298c
>
>if (dev->isconsole
> #ifdef CONFIG_SERIAL_TERMIOS
>|| (dev->tc_iflag & ECHO)
> #endif
>   )
>  {
>
> Currently:
>
>   893   if (
>   894 #ifdef CONFIG_SERIAL_TERMIOS
>   895   dev->tc_iflag & ECHO
>   896 #else
>   897   dev->isconsole
>   898 #endif
>   899  )
>   900 {
>
> If CONFIG_SERIAL_TERMIOS is not set then shouldn't the entire 'if '
> condition should be removed?
>

>From my understanding, when CONFIG_SERIAL_TERMIOS isn't set whether serial
driver do some special terminal handling(e.g. \n<->\r\n and echo) is
controlled by isconsole flag:

   1. isconsole equals false, all special handling is disable
   2. isconsole equals true, all special handling is enable

So, the check needs to be kept to ensure that non-console uart doesn't add
the unxpected process.

Re: Serial console BROKEN

[Xiang Xiao]

On Wed, Mar 8, 2023 at 12:09 AM Gregory Nutt  wrote:

> I imagine that this is occurring because readline also echos the input:
>
>
> https://github.com/apache/nuttx-apps/blob/master/system/readline/readline_common.c#L644
>
> The low-level serial driver should not echo just because /isconsole /is
> true.  Console echo is always handled by the application.  I would say
> that that is a bug.
>

>From the spec:
https://pubs.opengroup.org/onlinepubs/7908799/xsh/termios.h.html
the serial driver should echo the char if ECHO is enabled. Since NuttX
doesn't follow the spec, it makes many POSIX compliant interactive shells
doesn't work very well.
We have to make the breaking change to fix the wrong initial implementation.


> On 3/7/2023 9:45 AM, Sebastien Lorquet wrote:
> > Hi,
> >
> > I am connecting to my board using
> >
> > python3 -m serial --raw --eol cr /dev/ttyUSB0 115200
> >
> > This has ALWAYS worked.
> >
> > Today it does not work anymore.
> >
> > Every character I send to NSH is echoed TWICE, see below just typing
> > help:
> >
> > --
> >
> > seb@lap:~$ python3 -m serial --raw /dev/ttyUSB0 115200
> > --- Miniterm on /dev/ttyUSB0  115200,8,N,1 ---
> > --- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
> > hheellpp
> >
> > help usage:  help [-v] []
> >
> > --
> >
> > Only the echo is double, normal output is OK and characters are only
> > inputed once
> >
> > Miniterm is not doing any echo. This worked fine before.
> >
> >
> > I have found that this commit is the problem
> >
> >
> https://github.com/apache/nuttx/commit/68384e9db42e254b2cf6720bc3380aebdd2b298c
> >
> >
> > Reverting this fixes the dual echo.
> >
> >
> > I dont know what dev->isconsole does but
> >
> > Just HOW can such a general breakage be missed by the super duper cool
> > testing that takes hours to complete?
> >
> >
> > Please STOP breaking NuttX, AGAIN
> >
> > This is a lot of issues for a single day work.
> >
> >
> > Thank you
> >
> > Sebatien
> >
>

Re: Manjaro Distro

[Xiang Xiao]

On Thu, Mar 9, 2023 at 12:57 AM Simon Filgis 
wrote:

> Dear all,
>
> I switched to Manjaro a few weeks ago. Now I'm trying to build NuttX - of
> course.
>
> This packages where necessary so far:
>
> sudo pacman -S ncurses base-devel gmp mpfr libisl elfutils expat
> lib32-gcc-libs uboot-tools unzip
>
>
> # Install genromfs as per PX4
>
> wget
>
> https://sourceforge.net/projects/romfs/files/genromfs/0.5.2/genromfs-0.5.2.tar.gz
> tar zxvf genromfs-0.5.2.tar.gz
> cd genromfs-0.5.2 && make && make install && cd ..
> rm genromfs-0.5.2.tar.gz genromfs-0.5.2 -r
>
>
> --
>
> I cannot resolve the following error:
> fatal error: math.h: No such file or directory
>
> NuttX documentation says:
>
> The math library header file, `math.h`, is a then special case.  If you do
> nothing, the standard math.h header file that is provided with your
> toolchain will be used.
>

> Can somebody please give me a hint for how to overcome this?
>
>
Which toolchain do you use? You have two choices now:

   - CONFIG_LIBM=y which enable libm provided by nuttx
   - CONFIG_LIBM_TOOLCHAIN=y which select libm provided by toolchain

New selection will come soon, e.g.:
https://github.com/JuliaMath/openlibm
https://gitlab.com/gtd-gmbh/libmcs

For more info, please reference this PR:
https://github.com/apache/nuttx/pull/8584

Best regards,
>
> Simon
>
> --
> Hard- and Softwaredevelopment Consultant
> Ingenieurbüro-Filgis
> USt-IdNr.: DE305343278
>

Re: DISCUSSION - Usage of mailing lists for apache projects

[Xiang Xiao]

On Wed, Mar 8, 2023 at 10:05 PM Sebastien Lorquet 
wrote:

> I dont think your point of view is very realistic. You seem to be
> turning the situation into something that pleases you but is not really
> compatible with what can be observed from outside.
>
> In the archive for 2023 there are 2035 topics, I just overlooked more
> than 600 of them and ALL of them are just copies of github messages. And
> this is normal, because this is unusable.
>
> Also, I cannot actively review ALL the hundreds of pull requests that
> are regularly sent by large companies.
>
> I am primarily a user, that sometimes sends small contributions.
>
> And I am unhappy when I see that other people were careless with nice
> code and do whatever they want for their own agenda without any
> consideration for other users.
>
> To be honest I dont fully understand why SO MANY changes are required to
> core OS features that should be stable.
>
>
Here is the commit summary for nuttx and zephyr:
https://github.com/apache/nuttx/graphs/contributors
https://github.com/zephyrproject-rtos/zephyr/graphs/contributors
You can see that nuttx is far behind than zephyr.


> There need to be some triage when a new pull request is sent, its
> severity evaluated by someone who is NOT in the company of the original
> contributor, and if severe, it has to be approved on the mailing list. I
> cannot do that alone, even if I try.
>
>
There is no real difference whether you review the change from email or
github. The important thing is that you have to take the time to review and
approve/reject the change, So, I would suggest that you spend half an hour
every day reviewing and commenting on the patch.


> At this point do you realize that more than !~90ish% of changes seem to
> come from xiaomi developers and most of these are approved by xiaoxiang
> alone?
>

Here is a summary Alan collected recently:
 https://www.mail-archive.com/dev@nuttx.apache.org/msg08967.html
Xiaomi makes around 50% contrution, not 90%.

Because this HAS to be said. They send overwhelming numbers
> contributions that cannot be possibly understood as a whole, there is no
> clear agenda, they add whatever they like with no roadmap and no review
> from other users that may have other uses.
>
> This has to be under control somehow. I believe even you Alan are not
> aware of the amount of code that enters nuttx. No one can be now.
>
>
First, most of my changes are reviewed and merged by non-xiaomi committers.
Other patches from xiaomi normally are reviewed and merged by non-xiaomi
committers.
In rare cases, I may review and merge the patch if it is a trivial fix or
not be reviewed for more than three days.
But I will review carefully regardless the patch come from xiaomi or
others, Here is one example:
https://github.com/apache/nuttx/pull/8619



> Sebastien
>
>
> Le 08/03/2023 à 14:39, Alan C. Assis a écrit :
> > Hi Sebastien,
> >
> > Yes, that commit list is mostly for people who don't want to use
> > github but still wanting to see what is going on.
> >
> > Sometimes when a very important PR raises some concern on GitHub PR,
> > people post their concern he, it already happened many times.
> >
> > But again: the best way to guarantee that current changes please you
> > is to keep actively reviewing the PRs and contributing to the project.
> >
> > BR,
> >
> > Alan
> >
> > On 3/8/23, Sebastien Lorquet  wrote:
> >> Hi,
> >>
> >> I had a look and this mailing list is not made for human consumption.
> >>
> >> No one has ever sent a message on it manually, right?
> >>
> >> In practice, important changes are still NOT discussed on the DEV
> >> mailing list.
> >>
> >> You said yourself that "all development has moved to github".
> >>
> >> As soon as automated tests are passed, every commit is considered with a
> >> similar level of importance, however this is clearly not the case.
> >>
> >> Sebastien
> >>
> >> Le 08/03/2023 à 13:21, Alan C. Assis a écrit :
> >>> HI Sebastien,
> >>>
> >>> It is already done, you just need to subscribe to
> >>> https://lists.apache.org/list.html?comm...@nuttx.apache.org to receive
> >>> all commits messages and discussions.
> >>>
> >>> Everything is archived on apache side!
> >>>
> >>> BR,
> >>>
> >>> Alan
> >>>
> >>> On 3/8/23, Sebastien Lorquet  wrote:
>  Apache projects are required to use mailing lists for long term
> archival
>  purposes.
> 
>  It seems to me that this project is avoiding that rule and moved all
>  development to github
> 
>  This is in contradiction with the Apache project rules.
> 
> 
>  I request clarification on this situation and requirement (instead of
>  just asserting it), including a review by the apache boards.
> 
> 
>  If it is proved that development happens primarily in github and not
>  here I will request the following:
> 
>  -every people committing patches and submitting pull request to be
>  registered on the mailing list so they are able to reply to 

Re: W25Nx NAND flash support

[Xiang Xiao]

On Wed, Mar 1, 2023 at 10:44 PM Sebastien Lorquet 
wrote:

> Also, sorry for the multiple messages, but why is that not new IOCTL
> calls (used to "Support other, less frequently used commands") so the
> structure is not modified ?
>
>
Because the design mimic Linux mtd interface:
https://github.com/torvalds/linux/blob/master/include/linux/mtd/mtd.h#L352-L353



> That would be a much more compatible way of doing things. Devices that
> dont implement this would just automatically return -ENOSYS or -ENXIO
> (cant remember).
>
>
Yes, it could be. It's better to comment in the PR. before merging. But,
anyway you can provide a patch.


> Sebastien
>
> Le 01/03/2023 à 15:41, Sebastien Lorquet a écrit :
> >
> > the callbacks are not activated by conditional compilation and are in
> > the middle of struct mtd_dev_s , so how are these optional?
> >
>

Yes, it's betteer to move to the end of struct.


> > Sebastien
> >
> > Le 01/03/2023 à 15:36, Xiang Xiao a écrit :
> >>
> >>
> >> On Wed, Mar 1, 2023 at 10:26 PM Sebastien Lorquet
> >>  wrote:
> >>
> >> Hi,
> >>
> >> Please dont break the mtd interface, make it compatible with the
> >> previous one.
> >>
> >>
> >> The new callbacks are optional:
> >> https://github.com/apache/nuttx/pull/8683.
> >>
> >>
> >> Sebastien
> >>
> >> Le 01/03/2023 à 14:53, Xiang Xiao a écrit :
> >> > On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis
> >>  wrote:
> >> >
> >> >> Hi Xiang Xiao,
> >> >>
> >> >> This is a great news, but I'm afraid yaffs is not a good
> >> option for
> >> >> everybody because it uses GPL license.
> >> >>
> >> >>
> >> > Yes, but you can pay some money to switch to a more friendly
> >> license.
> >> >
> >> >
> >> >> But yes, it could be an option for people and companies that can
> >> >> release all their software as open-source. PX4 project for
> example
> >> >> could use it.
> >> >>
> >> >>
> >> > The enhancement in mtd_s is decoupled from yaffs, actually the new
> >> > interface follows Linux MTD model.
> >> > BTW, We plan to improve LittleFS to utilize the new interface.
> >> >
> >> >
> >> >> BR,
> >> >>
> >> >> Alan
> >> >>
> >> >> On 2/28/23, Xiang Xiao  wrote:
> >> >>> We are working on yaffs porting, mtd_s need a little bit of
> >> extension to
> >> >>> support NAND flash.
> >> >>> Here is the patch: https://github.com/apache/nuttx/pull/8670
> >> >>> LittleFS needs some modification to get the benefit.
> >> >>>
> >> >>> On Sun, Feb 26, 2023 at 10:14 PM Nathan Hartman <
> >> >> hartman.nat...@gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>>> On Sat, Feb 25, 2023 at 8:53 AM Gregory Nutt
> >> 
> >> >> wrote:
> >> >>>>>> Maybe LittleFS or SmartFS could be extended to handle NAND.
> >> >>>>> I believe that LittleFS has been used with NAND with a NAND
> >> FLASH
> >> >>>>> translation layer.  I am not sure of the state of that
> >> work, but I
> >> >> know
> >> >>>>> that people have tried it.  Google "LittleFS on NAND flash"
> >> and see
> >> >> the
> >> >>>>> hits.
> >> >>>>>
> >> >>>>> If a NAND flash translation layer (NFTL) were ported to
> >> NuttX, then
> >> >> you
> >> >>>>> should be able to use almost any file system, although some
> >> would be
> >> >>>>> extremely inefficient with NAND. The NAND flash layer would
> >> need to
> >> >> do
> >> >>>>> sparing, wear-leveling, ECC calculations, etc. as needed by
> >> NAND.
> >> >>>>>
> >> >>>>> I have used NXFFS with NAND and it /almost /works.
> >> >

Re: W25Nx NAND flash support

[Xiang Xiao]

I will just talk about some possible solutions for small companies. We
already pay the license from Aleph One which is very very cheap from my
point of view.

On Wed, Mar 1, 2023 at 10:53 PM Sebastien Lorquet 
wrote:

> Oh come on you are just trying to circumvent the GPL here
>
> Sebastien
>
> Le 01/03/2023 à 15:50, Xiang Xiao a écrit :
> > On Wed, Mar 1, 2023 at 10:45 PM Alan C. Assis  wrote:
> >
> >> oh, I just found it:
> >>
> >> "The fee depends on the number of distinct Products in which Yaffs
> >> will be used, and there is a base price of GBP £10,000.  If more than
> >> one Product is included in the same Licence there is usually a
> >> reduction in the second fee."
> >>
> >> For a big company it is fine, but small companies could be a "don't
> go!".
> >>
> > If the cost isn't affordable and the company doesn't want to public all
> > source code, he can try this approach:
> >
> > 1. Enable userfs to move yaffs from kernel to userspace
> > 2. Build yaffs as wasm module and run it in:
> > https://github.com/apache/nuttx-apps/tree/master/interpreters/wamr
> >
> > This technology could satisfy all GPL requirements but without
> > exposing your private source code.
> >
> >
> >> BR,
> >>
> >> Alan
> >>
> >> On 3/1/23, Alan C. Assis  wrote:
> >>> Hi Xiang,
> >>>
> >>> On 3/1/23, Xiang Xiao  wrote:
> >>>> On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis 
> wrote:
> >>>>
> >>>>> Hi Xiang Xiao,
> >>>>>
> >>>>> This is a great news, but I'm afraid yaffs is not a good option for
> >>>>> everybody because it uses GPL license.
> >>>>>
> >>>>>
> >>>> Yes, but you can pay some money to switch to a more friendly license.
> >>>>
> >>> Do you have more info about it? Is it a reasonable price for small
> >>> companies (let's say: less than U$ 5k to use as you wish or less than
> >>> U$ 1 per device) ?
> >>>
> >>> BR,
> >>>
> >>> Alan
> >>>
>

Re: W25Nx NAND flash support

[Xiang Xiao]

On Wed, Mar 1, 2023 at 10:45 PM Alan C. Assis  wrote:

> oh, I just found it:
>
> "The fee depends on the number of distinct Products in which Yaffs
> will be used, and there is a base price of GBP £10,000.  If more than
> one Product is included in the same Licence there is usually a
> reduction in the second fee."
>
> For a big company it is fine, but small companies could be a "don't go!".
>

If the cost isn't affordable and the company doesn't want to public all
source code, he can try this approach:

   1. Enable userfs to move yaffs from kernel to userspace
   2. Build yaffs as wasm module and run it in:
   https://github.com/apache/nuttx-apps/tree/master/interpreters/wamr

This technology could satisfy all GPL requirements but without
exposing your private source code.


> BR,
>
> Alan
>
> On 3/1/23, Alan C. Assis  wrote:
> > Hi Xiang,
> >
> > On 3/1/23, Xiang Xiao  wrote:
> >> On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis  wrote:
> >>
> >>> Hi Xiang Xiao,
> >>>
> >>> This is a great news, but I'm afraid yaffs is not a good option for
> >>> everybody because it uses GPL license.
> >>>
> >>>
> >> Yes, but you can pay some money to switch to a more friendly license.
> >>
> >
> > Do you have more info about it? Is it a reasonable price for small
> > companies (let's say: less than U$ 5k to use as you wish or less than
> > U$ 1 per device) ?
> >
> > BR,
> >
> > Alan
> >
>

Re: W25Nx NAND flash support

[Xiang Xiao]

On Wed, Mar 1, 2023 at 10:43 PM Alan C. Assis  wrote:

> Hi Xiang,
>
> On 3/1/23, Xiang Xiao  wrote:
> > On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis  wrote:
> >
> >> Hi Xiang Xiao,
> >>
> >> This is a great news, but I'm afraid yaffs is not a good option for
> >> everybody because it uses GPL license.
> >>
> >>
> > Yes, but you can pay some money to switch to a more friendly license.
> >
>
> Do you have more info about it? Is it a reasonable price for small
> companies (let's say: less than U$ 5k to use as you wish or less than
> U$ 1 per device) ?
>

One product costs 1 Euro without the quantity limitation.

>
> BR,
>
> Alan
>

Re: W25Nx NAND flash support

[Xiang Xiao]

On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis  wrote:

> Hi Xiang Xiao,
>
> This is a great news, but I'm afraid yaffs is not a good option for
> everybody because it uses GPL license.
>
>
Yes, but you can pay some money to switch to a more friendly license.


> But yes, it could be an option for people and companies that can
> release all their software as open-source. PX4 project for example
> could use it.
>
>
The enhancement in mtd_s is decoupled from yaffs, actually the new
interface follows Linux MTD model.
BTW, We plan to improve LittleFS to utilize the new interface.


> BR,
>
> Alan
>
> On 2/28/23, Xiang Xiao  wrote:
> > We are working on yaffs porting, mtd_s need a little bit of extension to
> > support NAND flash.
> > Here is the patch: https://github.com/apache/nuttx/pull/8670
> > LittleFS needs some modification to get the benefit.
> >
> > On Sun, Feb 26, 2023 at 10:14 PM Nathan Hartman <
> hartman.nat...@gmail.com>
> > wrote:
> >
> >> On Sat, Feb 25, 2023 at 8:53 AM Gregory Nutt 
> wrote:
> >>
> >> >
> >> > > Maybe LittleFS or SmartFS could be extended to handle NAND.
> >> >
> >> > I believe that LittleFS has been used with NAND with a NAND FLASH
> >> > translation layer.  I am not sure of the state of that work, but I
> know
> >> > that people have tried it.  Google "LittleFS on NAND flash" and see
> the
> >> > hits.
> >> >
> >> > If a NAND flash translation layer (NFTL) were ported to NuttX, then
> you
> >> > should be able to use almost any file system, although some would be
> >> > extremely inefficient with NAND.  The NAND flash layer would need to
> do
> >> > sparing, wear-leveling, ECC calculations, etc. as needed by NAND.
> >> >
> >> > I have used NXFFS with NAND and it /almost /works.
> >> >
> >> > The basic filesystem issue is that all available filesystems need to
> do
> >> > read-modify-write operations on flash sections.  And they assume that
> >> > you can always write a '1' bit to a '0' bit.  SmartFS and NXFFS both
> >> > assume this behavior explicitly.  It is a good assumption with NOR
> >> > flash.  The problem is that NAND can only be written a whole sector at
> >> > a
> >> > time.  This is because the error correction coding (ECC):  If you
> >> > change
> >> > even one bit in the NAND sector, you have to re-write the whole sector
> >> > with new ECC (because you can't change the '0' bits in the ECC back
> >> > into
> >> > '1''s without erasing the sector).
> >> >
> >> > Since SmartFS and NXFFS both do bit twiddling in the sectors they
> would
> >> > be very inefficient with an NFTL:  Each bit twiddle would cause a
> whole
> >> > sector re-write.  The same is probably true for LittleFS.  Other file
> >> > systems like FAT could also be used if there were an NFTL, however,
> FAT
> >> > would also be inefficient (each directory update or FAT update and
> each
> >> > file data size change would cause another sector write).
> >> >
> >> > So the only real solution to support NAND efficiently is use a file
> >> > system that is designed specifically around the peculiarities of NAND.
> >> > That would require some research (Alan has suggested a couple of
> places
> >> > to start).
> >>
> >>
> >>
> >> I seem to recall reading that in NAND flash, one of the challenges is
> >> that
> >> modifying a bit causes a disturbance that can corrupt other bits, even
> in
> >> unrelated sectors, and that even reading can have this effect. The
> >> disturbance only affects bits a little at a time, but after some number
> >> of
> >> accesses, the affected sectors need to be written to new sectors,
> keeping
> >> track of bad sectors formed in the process, to avoid data loss. This may
> >> lead to a cascade effect in which reading a sector may cause numerous
> >> sectors to be re-written. This phenomenon is called write amplification.
> >> The takeaway from a usage perspective is that you want to over-provision
> >> the amount of NAND flash you install, to leave plenty of room for all
> >> this
> >> swapping, and plenty of extra sectors to increase the amount of accesses
> >> before failure. The greater the over-provisioning, the longer until
> >> failure.
> >>
> >> I haven't tried yet, but I would like at some point to support a
> micro-SD
> >> and/or micro-MMC card as the backing store and use one that has built in
> >> hardware wear leveling, with some appropriate file system of course.
> This
> >> may add $50 or more to the bill of materials depending on the card
> >> capacity, but makes it possible for users to replace the card if it
> >> fails,
> >> in contrast to soldered-on flash chips, which can be replaced too, but
> >> only
> >> with the appropriate tools and skills.
> >>
> >> Cheers
> >> Nathan
> >>
> >
>

Re: W25Nx NAND flash support

[Xiang Xiao]

We are working on yaffs porting, mtd_s need a little bit of extension to
support NAND flash.
Here is the patch: https://github.com/apache/nuttx/pull/8670
LittleFS needs some modification to get the benefit.

On Sun, Feb 26, 2023 at 10:14 PM Nathan Hartman 
wrote:

> On Sat, Feb 25, 2023 at 8:53 AM Gregory Nutt  wrote:
>
> >
> > > Maybe LittleFS or SmartFS could be extended to handle NAND.
> >
> > I believe that LittleFS has been used with NAND with a NAND FLASH
> > translation layer.  I am not sure of the state of that work, but I know
> > that people have tried it.  Google "LittleFS on NAND flash" and see the
> > hits.
> >
> > If a NAND flash translation layer (NFTL) were ported to NuttX, then you
> > should be able to use almost any file system, although some would be
> > extremely inefficient with NAND.  The NAND flash layer would need to do
> > sparing, wear-leveling, ECC calculations, etc. as needed by NAND.
> >
> > I have used NXFFS with NAND and it /almost /works.
> >
> > The basic filesystem issue is that all available filesystems need to do
> > read-modify-write operations on flash sections.  And they assume that
> > you can always write a '1' bit to a '0' bit.  SmartFS and NXFFS both
> > assume this behavior explicitly.  It is a good assumption with NOR
> > flash.  The problem is that NAND can only be written a whole sector at a
> > time.  This is because the error correction coding (ECC):  If you change
> > even one bit in the NAND sector, you have to re-write the whole sector
> > with new ECC (because you can't change the '0' bits in the ECC back into
> > '1''s without erasing the sector).
> >
> > Since SmartFS and NXFFS both do bit twiddling in the sectors they would
> > be very inefficient with an NFTL:  Each bit twiddle would cause a whole
> > sector re-write.  The same is probably true for LittleFS.  Other file
> > systems like FAT could also be used if there were an NFTL, however, FAT
> > would also be inefficient (each directory update or FAT update and each
> > file data size change would cause another sector write).
> >
> > So the only real solution to support NAND efficiently is use a file
> > system that is designed specifically around the peculiarities of NAND.
> > That would require some research (Alan has suggested a couple of places
> > to start).
>
>
>
> I seem to recall reading that in NAND flash, one of the challenges is that
> modifying a bit causes a disturbance that can corrupt other bits, even in
> unrelated sectors, and that even reading can have this effect. The
> disturbance only affects bits a little at a time, but after some number of
> accesses, the affected sectors need to be written to new sectors, keeping
> track of bad sectors formed in the process, to avoid data loss. This may
> lead to a cascade effect in which reading a sector may cause numerous
> sectors to be re-written. This phenomenon is called write amplification.
> The takeaway from a usage perspective is that you want to over-provision
> the amount of NAND flash you install, to leave plenty of room for all this
> swapping, and plenty of extra sectors to increase the amount of accesses
> before failure. The greater the over-provisioning, the longer until
> failure.
>
> I haven't tried yet, but I would like at some point to support a micro-SD
> and/or micro-MMC card as the backing store and use one that has built in
> hardware wear leveling, with some appropriate file system of course. This
> may add $50 or more to the bill of materials depending on the card
> capacity, but makes it possible for users to replace the card if it fails,
> in contrast to soldered-on flash chips, which can be replaced too, but only
> with the appropriate tools and skills.
>
> Cheers
> Nathan
>

Re: Maximum file descriptor number in select()?

[Xiang Xiao]

On Sun, Feb 19, 2023 at 10:21 AM Gregory Nutt  wrote:

>  > fd_set is a 32bit array, the max fd number is hardcode to 256:
> >https://github.com/apache/nuttx/blob/master/include/sys/select.h#L38-L91
> <https://github.com/apache/nuttx/blob/master/include/sys/select.h#L38-L91>
>
> I just found that and was sending an email.  I could not find where
> select() or poll() are implemented in the nuttx repository.  Is there some
> kind of magic going on with that?
>
>
They hare here:
https://github.com/apache/nuttx/blob/master/fs/vfs/fs_select.c
https://github.com/apache/nuttx/blob/master/fs/vfs/fs_poll.c



> > it's better to define FD_SETSIZE to OPEN_MAX.
>
> That was my suggestion too..
>
>
Here is the patch:
https://github.com/apache/nuttx/pull/8581


>
>
> On Sat, Feb 18, 2023 at 8:12 PM Xiang Xiao 
> wrote:
>
> > fd_set is a 32bit array, the max fd number is hardcode to 256:
> > https://github.com/apache/nuttx/blob/master/include/sys/select.h#L38-L91
> > it's better to define FD_SETSIZE to OPEN_MAX.
> >
> >
> > On Sun, Feb 19, 2023 at 9:53 AM Gregory Nutt 
> wrote:
> >
> > > That is okay.  My answer is possibly wrong anyway.  OPEN_MAX is the
> > maximum
> > > number of open files.  For the select, the file descriptor set was held
> > in
> > > a 32-bit bit set so the maximum was 31.  But that logic has been
> > redesigned
> > > and I don’t know how it works now.
> > >
> > >
> > >
> > > Hopefully, it supports up to OPEN_MAX – 1, otherwise that would be a
> bug.
> > >
> > >
> > >
> > > On Sat, Feb 18, 2023 at 7:37 PM KIKUCHI Takeyoshi <
> > > kiku...@centurysys.co.jp>
> > > wrote:
> > >
> > > > Hi Greg,
> > > >
> > > > Sorry, it seems that the same one was delivered late, probably
> because
> > I
> > > > sent it with the wrong email source account (not the one I subscribed
> > to
> > > > the list).
> > > >
> > > > best regards,
> > > > KIKUCHI Takeyoshi
> > > >
> > > >
> > > > On 2023/02/19 9:20, Gregory Nutt wrote:
> > > > > The maximum number of file descriptors is a constant value provided
> > by
> > > > > the preprocessor definition OPEN_MAX.
> > > > >
> > > > > On 2/16/2023 5:44 PM, KIKUCHI Takeyoshi wrote:
> > > > >> currently porting the Nim language to NuttX.
> > > > >> (It has been merged into the Nim devel branch)
> > > > >>
> > > > >> When using select/epoll, it is necessary to determine the maximum
> > > > >> value of file descriptor in order to assign a structure to store
> the
> > > > >> fd to be waited in advance.
> > > > >>
> > > > >> It is used in this way.
> > > > >>
> > > > >> -
> > > > >> proc newSelector*[T](): Selector[T] =
> > > > >>   # Retrieve the maximum fd count (for current OS) via getrlimit()
> > > > >>   var maxFD = maxDescriptors() <---
> > > > >> # Start with a reasonable size, checkFd() will grow this on demand
> > > > >>   const numFD = 1024
> > > > >> ...
> > > > >> for i in 0 ... < numFD:
> > > > >> result.fds[i].ident = InvalidIdent
> > > > >> -
> > > > >> (Now I realize that numFD also needs to be switched for NuttX...)
> > > > >>
> > > > >> Since the available memory capacity varies from device to device,
> > > > >> Is there any way to find the best value for each build config?
> > > > >>
> > > > >> Porting to FreeRTOS in the Nim language uses the maximum number of
> > > > >> sockets available, set by the user at build time.
> > > > >>
> > > > >> -
> > > > >> var FD_MAX* {.importc: "CONFIG_LWIP_MAX_SOCKETS", header:
> > > > >> "".} : cint
> > > > >> -
> > > > >>
> > > > >> best regards,
> > > > >> KIKUCHI Takeyoshi
> > > > >>
> > > > >
> > > >
> > > > --
> > > > kiku...@centurysys.co.jp
> > > >
> > >
> >
>

Re: Maximum file descriptor number in select()?

[Xiang Xiao]

fd_set is a 32bit array, the max fd number is hardcode to 256:
https://github.com/apache/nuttx/blob/master/include/sys/select.h#L38-L91
it's better to define FD_SETSIZE to OPEN_MAX.


On Sun, Feb 19, 2023 at 9:53 AM Gregory Nutt  wrote:

> That is okay.  My answer is possibly wrong anyway.  OPEN_MAX is the maximum
> number of open files.  For the select, the file descriptor set was held in
> a 32-bit bit set so the maximum was 31.  But that logic has been redesigned
> and I don’t know how it works now.
>
>
>
> Hopefully, it supports up to OPEN_MAX – 1, otherwise that would be a bug.
>
>
>
> On Sat, Feb 18, 2023 at 7:37 PM KIKUCHI Takeyoshi <
> kiku...@centurysys.co.jp>
> wrote:
>
> > Hi Greg,
> >
> > Sorry, it seems that the same one was delivered late, probably because I
> > sent it with the wrong email source account (not the one I subscribed to
> > the list).
> >
> > best regards,
> > KIKUCHI Takeyoshi
> >
> >
> > On 2023/02/19 9:20, Gregory Nutt wrote:
> > > The maximum number of file descriptors is a constant value provided by
> > > the preprocessor definition OPEN_MAX.
> > >
> > > On 2/16/2023 5:44 PM, KIKUCHI Takeyoshi wrote:
> > >> currently porting the Nim language to NuttX.
> > >> (It has been merged into the Nim devel branch)
> > >>
> > >> When using select/epoll, it is necessary to determine the maximum
> > >> value of file descriptor in order to assign a structure to store the
> > >> fd to be waited in advance.
> > >>
> > >> It is used in this way.
> > >>
> > >> -
> > >> proc newSelector*[T](): Selector[T] =
> > >>   # Retrieve the maximum fd count (for current OS) via getrlimit()
> > >>   var maxFD = maxDescriptors() <---
> > >> # Start with a reasonable size, checkFd() will grow this on demand
> > >>   const numFD = 1024
> > >> ...
> > >> for i in 0 ... < numFD:
> > >> result.fds[i].ident = InvalidIdent
> > >> -
> > >> (Now I realize that numFD also needs to be switched for NuttX...)
> > >>
> > >> Since the available memory capacity varies from device to device,
> > >> Is there any way to find the best value for each build config?
> > >>
> > >> Porting to FreeRTOS in the Nim language uses the maximum number of
> > >> sockets available, set by the user at build time.
> > >>
> > >> -
> > >> var FD_MAX* {.importc: "CONFIG_LWIP_MAX_SOCKETS", header:
> > >> "".} : cint
> > >> -
> > >>
> > >> best regards,
> > >> KIKUCHI Takeyoshi
> > >>
> > >
> >
> > --
> > kiku...@centurysys.co.jp
> >
>

Re: SIGEV_THREAD not sharing file descriptors

[Xiang Xiao]

On Wed, Feb 8, 2023 at 6:31 PM Carlos Sanchez
 wrote:

> Hi all,
>
> If I understand correctly the POSIX standard, events using
> SIGEV_THREAD notification should behave as threads created using
> pthread_create. Quoting the standard:
>
>  QUOTE BEGIN 
> Multi-threaded programs can use an alternate event notification
> mechanism. When a notification is processed, and the sigev_notify
> member of the sigevent structure has the value SIGEV_THREAD, the
> function sigev_notify_function is called with parameter sigev_value.
>
> The function shall be executed in an environment as if it were the
> start_routine for a newly created thread with thread attributes
> specified by sigev_notify_attributes. If sigev_notify_attributes is
> NULL, the behavior shall be as if the thread were created with the
> detachstate attribute set to PTHREAD_CREATE_DETACHED. Supplying an
> attributes structure with a detachstate attribute of
> PTHREAD_CREATE_JOINABLE results in undefined behavior. The signal mask
> of this thread is implementation-defined.
> --- QUOTE END 
>
> However, in Nuttx, the functions are sent to the work queues, which
> are tasks which do not belong to the same task group as the main task
> so the file descriptors are not shared. This makes it difficult to
> implement code that reads from a file (device node) when a timeout
> happens.
>
> I think this is a deviation from the POSIX standard, is there any kind
> of document listing known deviations?
>
>
Yes, it's a well known limitation documented here:
https://github.com/apache/nuttx/issues/1352.


> Possible solutions:
> - Create another work queue for timers which belongs to the same task
> group as the main task.
> - Use nx_pthread_create internally for notification actions instead of
> work queues.
>
>
Both are possible solutions. Another one is sent the signal as before and
creates or wakes up the user space thread in the signal handler. One
workaround we are using now is opening a file with file_open or a socket
with psock_socket.


> Thoughts?
>
> BR
>
> Carlos
>
> --
>
> Carlos Sanchez (he, him, his)
> Geotab
>
> Embedded Systems Developer Team Lead | Europe
>
> Visit
>
> www.geotab.com
>
> Twitter | Facebook | YouTube | LinkedIn
>

Re: ZeroMQ

[Xiang Xiao]

On Fri, Feb 3, 2023 at 4:31 PM Marco Casaroli
 wrote:

> Hello,
>
> I've been experimenting to build ZeroMQ work with NuttX and I wanted to
> share my results so far.
>
> TLDR: it works
>
> Please, understand that I am very new to NuttX so please tell me with any
> conceptual errors I might be facing!
>
> https://github.com/casaroli/incubator-nuttx/tree/zeromq
>
> I think we need to be able to use this library both from kernel code and
> from applications, so I guessed this would fit along libxx and libdsp, etc.
>
>
Kernel code needs to call some special API(e.g. kmm_malloc, psock_send and
file_open), I amn't sure whether is it easy to adjust ZeroMQ for this type
of change.


> So I basically copied many stuff from those directories into libs/libzmq
>
> Still needs a lot of work, so next steps are to find a way to run the
> tests, then to make platform.hpp defines use nuttx/config.h, add czmq
> (higher level interface).
>
> However, in the current state, I could get IPC through Unix Domain Sockets
> and TCP. inproc also works fine.
>
> I wrote some simple examples (not on the repo) and they work.
>
> I want to know if anyone else is interested in helping with this (make
> ZeroMQ/czmq, nng, etc. work).
>
>
Yes, it's definitely good to support more 3rd party general libraries.


> Thank you.
>

Re: Including Lua scripts on filesystem

[Xiang Xiao]

As I said before, most features I talk about is disabled by default. Nobody
forbids you to use an alternative approach.

On Fri, Feb 3, 2023 at 6:40 AM Alan C. Assis  wrote:

> Hi Sebastien,
>
> Please once again, don't start a frame war, we need people to
> contribute to NuttX, to make our loved project great, not to destroy
> it.
>
> That said, I want to say a famous phrase from Linus Torvalds:
>
> "Doers decide!"
>
> If you want to decide how ROMFS should work, please do it.
>
>
I agree with Alan. Let's talk about technology and code.


> It is an open-source project after all, so you are invited to
> contribute, instead of starting a frame war against does who really do
> and really contribute to make NuttX great.
>
> About the System-V init you can use it from some projects, I heard
> someone using it on NuttX:
>
> https://github.com/arachsys/init/tree/master
>
> BR,
>
> Alan
>
> On 2/2/23, Sebastien Lorquet  wrote:
> > Hi,
> >
> > Once again please remember this project is Apache NuttX and not Xiaomi
> OS.
> >
> > The fact that your company is a major contributor and that you have
> > established your own local usage patterns does not mean these are the
> > only valid ones.
> >
>

But, in principle NuttX prefer POSIX way, this is the rule set by Greg and
agree by many community member:
https://github.com/apache/nuttx/blob/master/INVIOLABLES.md#strict-posix-compliance
.
Almost features added by Xiaomi strictly follow these rule:

   1. Implement this feature If POSIX define it
   2. If not, Implement it like other POSIX OS(*BSD or Linux)


> Each user has its own vision of how such a flexible system should work,
> > and all users count.
> >
> > So please dont suggest "bandaid" solutions to me like /etc is the only
> > place to mount a romfs because the init script can "do things".
> >
> > It is perfectly valid to mount three romfs each at /foo /bar and /baz at
> > boot and NuttX should allow it without telling me what's better.
> >
>

You can mount to any place already:
https://github.com/apache/nuttx-apps/blob/master/nshlib/Kconfig#L782-L787
I just mentioned the default value.


> And I'm sorry but nsh is a demo shell and not an init system, on my own
> > product there is NO WAY I want to give full shell access to my customer.
> >
> > This wont happen and nsh is not even part of my production build.
> >
> > FSes can be initialized in the board's _bringup routine and I dont want
> > a systemd/init/shell to read scripts to mount /proc and bind a littlefs
> > to a directory of my choosing. And _bringup is already used to init spi
> > and mtd.


Nobody forbids you to do this. Actually we also use this approach for the
very small system(e.g. BLE only module) too.
Let's come back to Russell's question, he wants to try Lua script in the
sim environment.



> Because NuttX is a RTOS and is meant to be more lightweight and
> > flexible than a generic freebsd/macos/linux OS.
> >
>

But, we also need the generic solution to handle the complex SoC(e.g.
4XA55+HiFi DSP+2xM33) and application(e.g. Smartwatch and IPCamera).


> >
> > Patch will come, I think this is a good change for nsh, but I'm a bit
> > busy at work at this moment, stuck in a kind of dual-PIC18 cursed hell
> > with hard deadlines lol.
> >
> > I also have a kind of "raid0" to aggregate two MTD with compatible
> > geometries into one, because reasons. It needs more tests, but it could
> > be useful for other users.
> >
> > Sebastien
> >
> >
> > On 2/2/23 19:48, Xiang Xiao wrote:
> >> On Fri, Feb 3, 2023 at 2:06 AM Sebastien Lorquet 
> >> wrote:
> >>
> >>> No, and this was exactly my problem.
> >>>
> >>> The romfs should NOT be mounted by nsh.
> >>>
> >>>
> >> It's an option and disabled by default.
> >> BTW, which place do you think is better than the current one?
> >>
> >>
> >>> Mounting to /etc/ is also NOT flexible enough.
> >>>
> >>> Because in a normal board the boot app is not nsh but the customer app.
> >>>
> >>>
> >> You can boot other applications/services from scripts.
> >>
> >>
> >>> At the moment this romfs mount is also closely coupled to the boot
> >>> script mechanism, which is very annoying if your boot script is on a
> >>> flash device (eg littlefs) and not on a romfs.
> >>>
> >>> I have decoupled this, but still not provided a pull request, that was
> >>> in fact easy and the result 

Re: Including Lua scripts on filesystem

[Xiang Xiao]

On Fri, Feb 3, 2023 at 2:06 AM Sebastien Lorquet 
wrote:

> No, and this was exactly my problem.
>
> The romfs should NOT be mounted by nsh.
>
>
It's an option and disabled by default.
BTW, which place do you think is better than the current one?


> Mounting to /etc/ is also NOT flexible enough.
>
> Because in a normal board the boot app is not nsh but the customer app.
>
>
You can boot other applications/services from scripts.


>
> At the moment this romfs mount is also closely coupled to the boot
> script mechanism, which is very annoying if your boot script is on a
> flash device (eg littlefs) and not on a romfs.
>
> I have decoupled this, but still not provided a pull request, that was
> in fact easy and the result is cleaner than the current way it is done.
>
>
 Yes, the Kconfig option makes the romfs mount couple with boot script, but
it is easy to decouple them. Patch is welcome.

>
> You cant expect customers to follow a nsh-based design. The learning
> curve for nuttx is hard enough, and if you add more constraints in apps
> building it is even harder.
>
>
It depends on the developer's background. If the developer has
experience with other POSIX OS(Android, Ubuntu and macOS), all is natural.


> Remember that NSH and apps are just "examples" and nuttx should work
> with completely separated apps.
>
>
Of course, all we discuss so far is the user space feature, not related to
the kernel.

Thats why I think nsh should not be responsible for too much.
>
>
nsh equals init pus sh. One improvement is split init from nsh like other
OS.


>
> Sebastien
>
>
> Le 02/02/2023 à 17:47, Xiang Xiao a écrit :
> > The infrastructure is there, you can:
> >
> > 1. Enable CONFIG_NSH_ROMFSETC
> > 2. Add your files to RCSRCS/RCRAWS in board's Makefile
> >
> > Then nsh will auto mount it to /etc. sim already has a demo for this:
> > https://github.com/apache/nuttx/tree/master/boards/sim/sim/sim/src/etc
> >
> >
> > On Fri, Feb 3, 2023 at 12:33 AM Russell Haley 
> wrote:
> >
> >> On Thu, Feb 2, 2023 at 6:35 AM Alan C. Assis  wrote:
> >>
> >>> It should be nice to have a simple logic to let users to use ROMFS.
> >>>
> >>> Currently everyone needs to duplicate it in their own code/application.
> >>>
> >>> I'm thinking something like apps/romfiles/ where people just put the
> >>> files that they want to be in the ROM and it will be included
> >>> automatically, instead reinventing the wheel.
> >>>
> >> This is what I was hoping to find.  I thought that since nuttx was
> already
> >> building a root filesystem there would be a simple way to include files
> >> with the image.
> >>
> >> Cheers,
> >> Russ
> >>
> >>> BR,
> >>>
> >>> Alan
> >>>
> >>> On 2/2/23, Sebastien Lorquet  wrote:
> >>>> Hi,
> >>>>
> >>>> Dont use boardctl for the romfs, this is too intertwined in the
> >>>> mechanism used to initialize nsh, which is not suitable for
> >>> customization.
> >>>> This need to be decoupled, I've done this in my own apps. I'll send it
> >>>> later, maybe.
> >>>>
> >>>>
> >>>> Here is the proper way to mount a romfs from your board bringup()
> >>> routines:
> >>>> I am using stm32 as example, the ROM fs directory holding the file is
> a
> >>>> brother to board/
> >>>>
> >>>> boarddir:
> >>>> +-- romfs
> >>>>+-- your files...
> >>>> +-- board
> >>>>+-- include
> >>>>+-- src
> >>>>
> >>>> You can do something else if you change the makefile below
> >>>>
> >>>> First change your board/src makefile to add a dependency so that
> >>>> stm32_bringup.c depends on romfs.h
> >>>>
> >>>> .PHONY: myromfs.h #might not be required
> >>>> stm32_bringup.c: myromfs.h
> >>>> myromfs.h:
> >>>>   @echo "ROMFS"
> >>>>   @genromfs -v -f my.romfs -d $(BOARD_DIR)/../romfs
> >>>>   @xxd -i my.romfs > myromfs.h
> >>>>
> >>>> Then do this in stm32_bringup.c:
> >>>>
> >>>> #include "myromfs.h"
> >>>> #define SECTORSIZE  64
> >>>> #define NSECTORS(b) (((b)+SECTORSIZE-1)/SECTORSIZE)
> >&g

Re: Including Lua scripts on filesystem

[Xiang Xiao]

Then, they should run the mount or whatever they want from rcS like how
Unix did. Actually, don't invert the wheel again, just follow how Unix is
done.

On Fri, Feb 3, 2023 at 1:27 AM Alan C. Assis  wrote:

> I think if it was generic but with some CONFIG_ROMFS_MNTPOINT it could
> be usable for all customer environment and preferences.
>
> Well, there still some cases that could be covered: ie. a user app
> that needs files mounted at two or more places inside the root file
> system.
>
> On 2/2/23, Sebastien Lorquet  wrote:
> > IMHO this is too dependent on customer environment and preferences.
> >
> > But it would be nice to be an official out-of-tree board skeleton to
> > show people how to organize it. libopencm3 has a demo project that is
> > very helpful.
> >
> > Because it's always the first step when you use nuttx on real hardware
> > that is not an unmodified commercial devboard.
> >
> > Sebastien
> >
> > Le 02/02/2023 à 15:35, Alan C. Assis a écrit :
> >> It should be nice to have a simple logic to let users to use ROMFS.
> >>
> >> Currently everyone needs to duplicate it in their own code/application.
> >>
> >> I'm thinking something like apps/romfiles/ where people just put the
> >> files that they want to be in the ROM and it will be included
> >> automatically, instead reinventing the wheel.
> >>
> >> BR,
> >>
> >> Alan
> >>
> >> On 2/2/23, Sebastien Lorquet  wrote:
> >>> Hi,
> >>>
> >>> Dont use boardctl for the romfs, this is too intertwined in the
> >>> mechanism used to initialize nsh, which is not suitable for
> >>> customization.
> >>>
> >>> This need to be decoupled, I've done this in my own apps. I'll send it
> >>> later, maybe.
> >>>
> >>>
> >>> Here is the proper way to mount a romfs from your board bringup()
> >>> routines:
> >>>
> >>> I am using stm32 as example, the ROM fs directory holding the file is a
> >>> brother to board/
> >>>
> >>> boarddir:
> >>> +-- romfs
> >>>+-- your files...
> >>> +-- board
> >>>+-- include
> >>>+-- src
> >>>
> >>> You can do something else if you change the makefile below
> >>>
> >>> First change your board/src makefile to add a dependency so that
> >>> stm32_bringup.c depends on romfs.h
> >>>
> >>> .PHONY: myromfs.h #might not be required
> >>> stm32_bringup.c: myromfs.h
> >>> myromfs.h:
> >>>   @echo "ROMFS"
> >>>   @genromfs -v -f my.romfs -d $(BOARD_DIR)/../romfs
> >>>   @xxd -i my.romfs > myromfs.h
> >>>
> >>> Then do this in stm32_bringup.c:
> >>>
> >>> #include "myromfs.h"
> >>> #define SECTORSIZE  64
> >>> #define NSECTORS(b) (((b)+SECTORSIZE-1)/SECTORSIZE)
> >>> #include 
> >>> #include 
> >>> #include 
> >>>
> >>> and then in stm32_bringup(void):
> >>>
> >>>ret = romdisk_register(0, // /dev/ram0
> >>>myromfs, //var in the xxd header
> >>>NSECTORS(myromfs_len),
> >>>SECTORSIZE);
> >>>
> >>> ret = nx_mount("/dev/ram0", "/romfs", "romfs", MS_RDONLY, NULL);
> >>>
> >>> Then you will have your files mounted in /romfs at boot.
> >>>
> >>> You can do symlinks:
> >>>
> >>>ret = symlink("/romfs/etc" , "/etc");
> >>> _info("Linking /etc: %d\n", ret);
> >>>
> >>> Sebastien
> >>>
> >>> Le 02/02/2023 à 05:59, Russell Haley a écrit :
> >>>> I am mistaken. I understood that the rom image was part of the
> >>>> application
> >>>> on flash, but had mis-read the documentation of
> >>>> boardctl(BOARDIOC_ROMDISK,
> >>>> (uintptr_t)); and thought that the command loaded the image into
> >>>> *RAM*
> >>>> (the  BOARDIOC_MKRD command makes the RAM disk. oops!).
> >>>>
> >>>> Incidentally, neither the romfs example nor my attempt to re-create it
> >>>> works in my sim:
> >>>>
> >>>> osboxes@osboxes ~/n

Re: Article: NuttX for PinePhone: LVGL Terminal for NSH Shell

[Xiang Xiao]

Lup, the demo could upstream to lvgl repo once you change NuttX special
task_create to posix_spawn.

On Thu, Feb 2, 2023 at 10:57 PM Tomek CEDRO  wrote:

> On Thu, Feb 2, 2023 at 4:44 AM Lee, Lup Yuen wrote:
> > Now we can run NuttX Console Apps on PinePhone, without a Serial Cable!
> > This article explains how we call the LVGL GUI Library to create a
> > Touchscreen Terminal App that runs NSH Commands...
> > https://lupyuen.github.io/articles/terminal
>
> CONGRATZ!! :-)
>
> --
> CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
>

Re: Including Lua scripts on filesystem

[Xiang Xiao]

The infrastructure is there, you can:

   1. Enable CONFIG_NSH_ROMFSETC
   2. Add your files to RCSRCS/RCRAWS in board's Makefile

Then nsh will auto mount it to /etc. sim already has a demo for this:
https://github.com/apache/nuttx/tree/master/boards/sim/sim/sim/src/etc


On Fri, Feb 3, 2023 at 12:33 AM Russell Haley  wrote:

> On Thu, Feb 2, 2023 at 6:35 AM Alan C. Assis  wrote:
>
> > It should be nice to have a simple logic to let users to use ROMFS.
> >
> > Currently everyone needs to duplicate it in their own code/application.
> >
> > I'm thinking something like apps/romfiles/ where people just put the
> > files that they want to be in the ROM and it will be included
> > automatically, instead reinventing the wheel.
> >
>
> This is what I was hoping to find.  I thought that since nuttx was already
> building a root filesystem there would be a simple way to include files
> with the image.
>
> Cheers,
> Russ
>
> >
> > BR,
> >
> > Alan
> >
> > On 2/2/23, Sebastien Lorquet  wrote:
> > > Hi,
> > >
> > > Dont use boardctl for the romfs, this is too intertwined in the
> > > mechanism used to initialize nsh, which is not suitable for
> > customization.
> > >
> > > This need to be decoupled, I've done this in my own apps. I'll send it
> > > later, maybe.
> > >
> > >
> > > Here is the proper way to mount a romfs from your board bringup()
> > routines:
> > >
> > > I am using stm32 as example, the ROM fs directory holding the file is a
> > > brother to board/
> > >
> > > boarddir:
> > > +-- romfs
> > >   +-- your files...
> > > +-- board
> > >   +-- include
> > >   +-- src
> > >
> > > You can do something else if you change the makefile below
> > >
> > > First change your board/src makefile to add a dependency so that
> > > stm32_bringup.c depends on romfs.h
> > >
> > > .PHONY: myromfs.h #might not be required
> > > stm32_bringup.c: myromfs.h
> > > myromfs.h:
> > >  @echo "ROMFS"
> > >  @genromfs -v -f my.romfs -d $(BOARD_DIR)/../romfs
> > >  @xxd -i my.romfs > myromfs.h
> > >
> > > Then do this in stm32_bringup.c:
> > >
> > > #include "myromfs.h"
> > > #define SECTORSIZE  64
> > > #define NSECTORS(b) (((b)+SECTORSIZE-1)/SECTORSIZE)
> > > #include 
> > > #include 
> > > #include 
> > >
> > > and then in stm32_bringup(void):
> > >
> > >   ret = romdisk_register(0, // /dev/ram0
> > >   myromfs, //var in the xxd header
> > >   NSECTORS(myromfs_len),
> > >   SECTORSIZE);
> > >
> > >ret = nx_mount("/dev/ram0", "/romfs", "romfs", MS_RDONLY, NULL);
> > >
> > > Then you will have your files mounted in /romfs at boot.
> > >
> > > You can do symlinks:
> > >
> > >   ret = symlink("/romfs/etc" , "/etc");
> > >_info("Linking /etc: %d\n", ret);
> > >
> > > Sebastien
> > >
> > > Le 02/02/2023 à 05:59, Russell Haley a écrit :
> > >> I am mistaken. I understood that the rom image was part of the
> > >> application
> > >> on flash, but had mis-read the documentation of
> > >> boardctl(BOARDIOC_ROMDISK,
> > >> (uintptr_t)); and thought that the command loaded the image into
> > >> *RAM*
> > >> (the  BOARDIOC_MKRD command makes the RAM disk. oops!).
> > >>
> > >> Incidentally, neither the romfs example nor my attempt to re-create it
> > >> works in my sim:
> > >>
> > >> osboxes@osboxes ~/n/nuttx (master)> ./nuttx
> > >>
> > >> NuttShell (NSH) NuttX-12.0.0
> > >> nsh> romfs
> > >> ERROR: Failed to create RAM disk: Unknown error
> > >> nsh> rapp
> > >> Starting Russells App 1...
> > >> ERROR: Failed to create RAM disk: Unknown error
> > >> nsh>
> > >>
> > >> I don't know how to debug this "unknown error". I guess I will try to
> > >> find
> > >> where that error message comes from and hook in GDB? I'm terrible at
> > this
> > >> stuff, someone needs to revoke my compiler license (tee hee).
> > >>
> > >> Cheers,
> >

Re: Including Lua scripts on filesystem

[Xiang Xiao]

You need to prepare /dev/ram1 with the mkrd command.

On Thu, Feb 2, 2023 at 2:29 PM Russell Haley  wrote:

> On Wed, Feb 1, 2023 at 9:23 PM Xiang Xiao 
> wrote:
>
> > You can debug sim nuttx with your favorite PC debugger just like a normal
> > host program.
> > Of course, please ensure the debug system is
> > generated(CONFIG_DEBUG_SYMBOLS=y).
> >
>
> I was able to set up KDevelop to run nuttx in the debugger, but the console
> never gave me a nsh> prompt so I wasn't able to enter my app name and run
> it. Nonetheless I was able to run in GDB on the command line. For some
> reason inode_search is returning 0, which is converted to -EEXIST or -17.
>
> Breakpoint 3, inode_reserve (path=0x7fff  memory at address 0x7fff>, mode=48, inode=0x55671220
> ) at inode/fs_inodereserve.c:174
> 174 {
> (gdb) n
> 184  *inode = NULL;
> (gdb)
> 186  if (path[0] == '\0')
> (gdb) p path
> $47 = 0x73f2f370 "/dev/ram1"
> (gdb) n
> 193  SETUP_SEARCH(, path, false);
> (gdb)
> 195  ret = inode_search();
> (gdb)
> 196  if (ret >= 0)
> (gdb) p ret
> $49 = 0
> (gdb) n
> 202  ret = -EEXIST;
> (gdb)
>
> Not sure what to do from here? Any feedback would be great?
>
> Thanks,
> Russ
>
>
> >
> > On Thu, Feb 2, 2023 at 1:00 PM Russell Haley 
> wrote:
> >
> > > I am mistaken. I understood that the rom image was part of the
> > application
> > > on flash, but had mis-read the documentation of
> > boardctl(BOARDIOC_ROMDISK,
> > > (uintptr_t)); and thought that the command loaded the image into
> > *RAM*
> > > (the  BOARDIOC_MKRD command makes the RAM disk. oops!).
> > >
> > > Incidentally, neither the romfs example nor my attempt to re-create it
> > > works in my sim:
> > >
> > > osboxes@osboxes ~/n/nuttx (master)> ./nuttx
> > >
> > > NuttShell (NSH) NuttX-12.0.0
> > > nsh> romfs
> > > ERROR: Failed to create RAM disk: Unknown error
> > > nsh> rapp
> > > Starting Russells App 1...
> > > ERROR: Failed to create RAM disk: Unknown error
> > > nsh>
> > >
> > > I don't know how to debug this "unknown error". I guess I will try to
> > find
> > > where that error message comes from and hook in GDB? I'm terrible at
> this
> > > stuff, someone needs to revoke my compiler license (tee hee).
> > >
> > > Cheers,
> > > Russ
> > >
> > > On Wed, Feb 1, 2023 at 7:29 PM Xiang Xiao 
> > > wrote:
> > >
> > > > romfs is part of your image as the const string. There is no
> difference
> > > > from the below manual step.
> > > >
> > > > On Thu, Feb 2, 2023 at 10:00 AM Russell Haley 
> > > > wrote:
> > > >
> > > > > On Tue, Jan 31, 2023 at 6:16 AM Fotis Panagiotopoulos <
> > > > f.j.pa...@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > Indeed the "proper" way of including a script would be to store
> it
> > > in a
> > > > > > file system.
> > > > > >
> > > > > > However, when I needed to include a single and small script and I
> > > > didn't
> > > > > > want to introduce a complete FS just for this, I used xxd.
> > > > > > xxd can convert any file to a C header file.
> > > > > >
> > > > > > You can then include the header, and access the whole file as a
> > > > variable.
> > > > > > Here is an example:
> > > > > >
> > > > > > I added this in my app Makefile:
> > > > > >
> > > > > > # Setup any special pre-build context
> > > > > > context: header
> > > > > > $(Q) cd path/to/libs/json.lua/ && xxd -i json.lua >
> > > json_lua.h
> > > > &&
> > > > > > echo -n "const " | cat - json_lua.h > temp && mv temp json_lua.h
> > > > > >
> > > > > >
> > > > > >
> > > > > > And then I used the file like this:
> > > > > >
> > > > > > #include "lua.h"#include "lauxlib.h"#include 
> > > > > >  #include "json_lua.h"
> > > > > >  static int luaopen_json(lua_State * L);
> > &

Re: Including Lua scripts on filesystem

[Xiang Xiao]

You can debug sim nuttx with your favorite PC debugger just like a normal
host program.
Of course, please ensure the debug system is
generated(CONFIG_DEBUG_SYMBOLS=y).

On Thu, Feb 2, 2023 at 1:00 PM Russell Haley  wrote:

> I am mistaken. I understood that the rom image was part of the application
> on flash, but had mis-read the documentation of boardctl(BOARDIOC_ROMDISK,
> (uintptr_t)); and thought that the command loaded the image into *RAM*
> (the  BOARDIOC_MKRD command makes the RAM disk. oops!).
>
> Incidentally, neither the romfs example nor my attempt to re-create it
> works in my sim:
>
> osboxes@osboxes ~/n/nuttx (master)> ./nuttx
>
> NuttShell (NSH) NuttX-12.0.0
> nsh> romfs
> ERROR: Failed to create RAM disk: Unknown error
> nsh> rapp
> Starting Russells App 1...
> ERROR: Failed to create RAM disk: Unknown error
> nsh>
>
> I don't know how to debug this "unknown error". I guess I will try to find
> where that error message comes from and hook in GDB? I'm terrible at this
> stuff, someone needs to revoke my compiler license (tee hee).
>
> Cheers,
> Russ
>
> On Wed, Feb 1, 2023 at 7:29 PM Xiang Xiao 
> wrote:
>
> > romfs is part of your image as the const string. There is no difference
> > from the below manual step.
> >
> > On Thu, Feb 2, 2023 at 10:00 AM Russell Haley 
> > wrote:
> >
> > > On Tue, Jan 31, 2023 at 6:16 AM Fotis Panagiotopoulos <
> > f.j.pa...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > Indeed the "proper" way of including a script would be to store it
> in a
> > > > file system.
> > > >
> > > > However, when I needed to include a single and small script and I
> > didn't
> > > > want to introduce a complete FS just for this, I used xxd.
> > > > xxd can convert any file to a C header file.
> > > >
> > > > You can then include the header, and access the whole file as a
> > variable.
> > > > Here is an example:
> > > >
> > > > I added this in my app Makefile:
> > > >
> > > > # Setup any special pre-build context
> > > > context: header
> > > > $(Q) cd path/to/libs/json.lua/ && xxd -i json.lua >
> json_lua.h
> > &&
> > > > echo -n "const " | cat - json_lua.h > temp && mv temp json_lua.h
> > > >
> > > >
> > > >
> > > > And then I used the file like this:
> > > >
> > > > #include "lua.h"#include "lauxlib.h"#include 
> > > >  #include "json_lua.h"
> > > >  static int luaopen_json(lua_State * L);
> > > >
> > > >  void ExtLibs_load(lua_State * L){
> > > > // json.lua#ifdef CONFIG_EXT_LIB_JSON_LUA
> > > > luaL_requiref(L, "json", luaopen_json, 1);
> > > > lua_pop(L, 1);#endif}
> > > >
> > > >  int luaopen_json(lua_State * L){
> > > > const char * modname = lua_tostring(L, 1);
> > > >
> > > > if (strcmp(modname, "json") != 0)
> > > > return luaL_error(L, "cannot load json module");
> > > >
> > > > if (luaL_loadbufferx(L, (char*)json_lua, json_lua_len,
> "json",
> > > > "t") != LUA_OK)
> > > > return lua_error(L);
> > > >
> > > > lua_call(L, 0, 1);
> > > >
> > > > return 1;}
> > > >
> > > >
> > > > I hope this helps...
> > > >
> > > That is very helpful, and not just for nuttx development! Thanks for
> the
> > > tip.
> > >
> > > The romfs example actually uses xxd as well to convert the filesystem
> > into
> > > hex code that is also stored in a header file. If I am reading the code
> > > correctly, the example app loads the entire filesystem into memory,
> which
> > > isn't very efficient and not at all what I wanted. Can someone tell me
> if
> > > that's true?
> > >
> > > Thanks,
> > > Russ
> > >
> > > >
> > > >
> > > >
> > > > On Sun, Jan 29, 2023 at 7:34 AM Xiang Xiao <
> xiaoxiang781...@gmail.com>
> > > > wrote:
> > > >
> > > > > You can use the real file system on the device, there are many
> > choic

Re: Including Lua scripts on filesystem

[Xiang Xiao]

romfs is part of your image as the const string. There is no difference
from the below manual step.

On Thu, Feb 2, 2023 at 10:00 AM Russell Haley  wrote:

> On Tue, Jan 31, 2023 at 6:16 AM Fotis Panagiotopoulos  >
> wrote:
>
> > Hello,
> >
> > Indeed the "proper" way of including a script would be to store it in a
> > file system.
> >
> > However, when I needed to include a single and small script and I didn't
> > want to introduce a complete FS just for this, I used xxd.
> > xxd can convert any file to a C header file.
> >
> > You can then include the header, and access the whole file as a variable.
> > Here is an example:
> >
> > I added this in my app Makefile:
> >
> > # Setup any special pre-build context
> > context: header
> > $(Q) cd path/to/libs/json.lua/ && xxd -i json.lua > json_lua.h &&
> > echo -n "const " | cat - json_lua.h > temp && mv temp json_lua.h
> >
> >
> >
> > And then I used the file like this:
> >
> > #include "lua.h"#include "lauxlib.h"#include 
> >  #include "json_lua.h"
> >  static int luaopen_json(lua_State * L);
> >
> >  void ExtLibs_load(lua_State * L){
> > // json.lua#ifdef CONFIG_EXT_LIB_JSON_LUA
> > luaL_requiref(L, "json", luaopen_json, 1);
> > lua_pop(L, 1);#endif}
> >
> >  int luaopen_json(lua_State * L){
> > const char * modname = lua_tostring(L, 1);
> >
> > if (strcmp(modname, "json") != 0)
> > return luaL_error(L, "cannot load json module");
> >
> > if (luaL_loadbufferx(L, (char*)json_lua, json_lua_len, "json",
> > "t") != LUA_OK)
> > return lua_error(L);
> >
> > lua_call(L, 0, 1);
> >
> > return 1;}
> >
> >
> > I hope this helps...
> >
> That is very helpful, and not just for nuttx development! Thanks for the
> tip.
>
> The romfs example actually uses xxd as well to convert the filesystem into
> hex code that is also stored in a header file. If I am reading the code
> correctly, the example app loads the entire filesystem into memory, which
> isn't very efficient and not at all what I wanted. Can someone tell me if
> that's true?
>
> Thanks,
> Russ
>
> >
> >
> >
> > On Sun, Jan 29, 2023 at 7:34 AM Xiang Xiao 
> > wrote:
> >
> > > You can use the real file system on the device, there are many choices:
> > > romfs, littlefs, fatfs, starmtfs and spiffs.
> > >
> > > On Sun, Jan 29, 2023 at 12:59 PM Russell Haley 
> > > wrote:
> > >
> > > > On Sat, Jan 28, 2023 at 7:35 PM Xiang Xiao <
> xiaoxiang781...@gmail.com>
> > > > wrote:
> > > >
> > > > > You can enable CONFIG_FS_HOSTFS/CONFIG_SIM_HOSTFS, put your scripts
> > > into
> > > > > some PC folder and run mount this folder from nsh:
> > > > > mount -t hostfs -o fs=/path/to/your/pc/folder. /data
> > > > >
> > > > > While I appreciate the answer, I am using the sim as a testing
> > platform
> > > > and hoping to move to either an STM32F4/7 or a Sony Spresense. I am
> > > hoping
> > > > for a solution that is applicable to an embedded project. If I can't
> > just
> > > > add files to the initial image then I will look at the romfs example
> > and
> > > > maybe the next best thing?
> > > >
> > > >
> > > > >
> > > > > On Sun, Jan 29, 2023 at 2:24 AM Russell Haley <
> russ.ha...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Big thanks to Xiang Xiao for pointing me to the sim:lua
> > > configuration.
> > > > I
> > > > > > was unable to simply include the defconfig file that you linked
> to,
> > > > but I
> > > > > > was able to reconfigure for the sim:lua configuration.  I've now
> > got
> > > an
> > > > > app
> > > > > > in the examples folder that includes the Lua interpreter. Is
> there
> > a
> > > > > > tutorial on how to include folders and lua scripts or extra files
> > in
> > > > the
> > > > > > initial file system?
> > > > > >
> > > > > > Much appreciated,
> > > > > > Russ
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Including Lua scripts on filesystem

[Xiang Xiao]

You can use the real file system on the device, there are many choices:
romfs, littlefs, fatfs, starmtfs and spiffs.

On Sun, Jan 29, 2023 at 12:59 PM Russell Haley  wrote:

> On Sat, Jan 28, 2023 at 7:35 PM Xiang Xiao 
> wrote:
>
> > You can enable CONFIG_FS_HOSTFS/CONFIG_SIM_HOSTFS, put your scripts into
> > some PC folder and run mount this folder from nsh:
> > mount -t hostfs -o fs=/path/to/your/pc/folder. /data
> >
> > While I appreciate the answer, I am using the sim as a testing platform
> and hoping to move to either an STM32F4/7 or a Sony Spresense. I am hoping
> for a solution that is applicable to an embedded project. If I can't just
> add files to the initial image then I will look at the romfs example and
> maybe the next best thing?
>
>
> >
> > On Sun, Jan 29, 2023 at 2:24 AM Russell Haley 
> > wrote:
> >
> > > Hi,
> > >
> > > Big thanks to Xiang Xiao for pointing me to the sim:lua configuration.
> I
> > > was unable to simply include the defconfig file that you linked to,
> but I
> > > was able to reconfigure for the sim:lua configuration.  I've now got an
> > app
> > > in the examples folder that includes the Lua interpreter. Is there a
> > > tutorial on how to include folders and lua scripts or extra files in
> the
> > > initial file system?
> > >
> > > Much appreciated,
> > > Russ
> > >
> >
>

Re: Including Lua scripts on filesystem

[Xiang Xiao]

You can enable CONFIG_FS_HOSTFS/CONFIG_SIM_HOSTFS, put your scripts into
some PC folder and run mount this folder from nsh:
mount -t hostfs -o fs=/path/to/your/pc/folder. /data


On Sun, Jan 29, 2023 at 2:24 AM Russell Haley  wrote:

> Hi,
>
> Big thanks to Xiang Xiao for pointing me to the sim:lua configuration. I
> was unable to simply include the defconfig file that you linked to, but I
> was able to reconfigure for the sim:lua configuration.  I've now got an app
> in the examples folder that includes the Lua interpreter. Is there a
> tutorial on how to include folders and lua scripts or extra files in the
> initial file system?
>
> Much appreciated,
> Russ
>