Re: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
ACK! :-) -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info On Sat, Mar 11, 2023, 23:53 Xiang Xiao wrote: > On Sun, Mar 12, 2023 at 2:09 AM Tomek CEDRO wrote: > > > Please consult Unix manual and source code (for instance any of the BSD). > > Linux was never a reference in Unix world, it was about to mimic Unix, > but > > was not even self-compatible. Not a good reference point. See how big > mess > > it introduced in current drivers implementation (i.e. drm kms) and their > > propagation to other OS. Is this similar situation here? > > > > > This change isn't specific to Linux: > https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/termios.h.html > > > > Device driver should not provide any additional processing / silent > rewrite > > of data, just provide raw data, that is then processed by the > application. > > Driver may be configured by ioctl/sysclt, to enable additional / specific > > behavior (i.e. debug, buffer size, latency, etc), so the user always > knows > > exactly what the driver does. > > > > I am pro Greg approach, so driver is always simple and generic, > everything > > else is up to application. This is the Unix way and you will know how > most > > things work, because they all work the same generic standardized way, as > > simple as possible, nothing else happens "in the background unnoticed". > > > > If a specific device needs a specific handling by a generic driver, then > > "quirks" may be used. That explicitly mark "specific" behavior for a > given > > device (and only this device). > > > > > All changes are controlled by ioctl and specified by standard, please read > the spec: > https://pubs.opengroup.org/onlinepubs/9699919799/functions/tcsetattr.html > https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/termios.h.html > > > > -- > > CeDeROM, SQ7MHZ, http://www.tomek.cedro.info > > >
Re: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
On Sun, Mar 12, 2023 at 2:09 AM Tomek CEDRO wrote: > Please consult Unix manual and source code (for instance any of the BSD). > Linux was never a reference in Unix world, it was about to mimic Unix, but > was not even self-compatible. Not a good reference point. See how big mess > it introduced in current drivers implementation (i.e. drm kms) and their > propagation to other OS. Is this similar situation here? > > This change isn't specific to Linux: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/termios.h.html > Device driver should not provide any additional processing / silent rewrite > of data, just provide raw data, that is then processed by the application. > Driver may be configured by ioctl/sysclt, to enable additional / specific > behavior (i.e. debug, buffer size, latency, etc), so the user always knows > exactly what the driver does. > > I am pro Greg approach, so driver is always simple and generic, everything > else is up to application. This is the Unix way and you will know how most > things work, because they all work the same generic standardized way, as > simple as possible, nothing else happens "in the background unnoticed". > > If a specific device needs a specific handling by a generic driver, then > "quirks" may be used. That explicitly mark "specific" behavior for a given > device (and only this device). > > All changes are controlled by ioctl and specified by standard, please read the spec: https://pubs.opengroup.org/onlinepubs/9699919799/functions/tcsetattr.html https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/termios.h.html > -- > CeDeROM, SQ7MHZ, http://www.tomek.cedro.info >
Re: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
It is true that Linux is not fully POSIX compliant (e.g., https://linuxhint.com/is_linux_posix_compliant/). But Linux folk have gone to a lot of work to clean up a Linux specification with the LSB: https://refspecs.linuxfoundation.org/lsb.shtml and that generally adopts POSIX standards. We use the the term "Linux" loosely here. POSIX specifies the interface between applications and an operating system. However, the user application never interacts directly with Linux which resides in kernel space but rather with various user space libraries like libc that, in turn, interface with Linux via call gates (uncontrolled and not not prototype-able C interfaces). So more correctly, it is the GNU libraries that determine the user application interface. Calling this Gnu/Linux is more accurate but that phrase manages to piss off everyone. NuttX has evolved over the years in regard to what it uses as its standards base. 1. Originally, in the beginning, NuttX was a very tiny OS (think FreeRTOS or ChibiOS) with a very limited user interface but drawing primarily from POSIX interface definitions. Since NuttX provides both the OS and the libc, we can safely call this interface NuttX. 2. Later the project adopted OpenGroup.org as the standards base. OpenGroup.org is a good clean POSIX definition and no user interface could come in that was not specified by OpenGroup.org. OpenGroup.org is THE Unix spcification. Documentation (like the Inviolables.md) still claim that OpenGroup.org is the standards base. 3. Most NuttX users are also Linux users so there has always been pressure to follow Linux interface definitions (the LSB). After Xiaomi adopted NuttX, the pressure to adopt the LSB as the specification base (NSB?) was irresistible. From my point of view, I am just happy that we can point to specifications and say, "NuttX is compliant with that" -- whether that is OpenGroup.org or the LSB. I have led technical teams for several decades and one thing I have learned is that everyone does thing differently and as a technical leader, you have to stand back and simply ask does the propose solution work? Does the proposed solution meet the need? If yes, then you just have to accept the solution and let the rest go as personal preference. Unless you are the BDFL as I was for so many years. Also, the TERMIOS changes are POSIX and compliant with the LSB, OpenGroup.org and, I am sure, BSD. A bunch of caveats. - Use of the term POSIX compliance OS in regard to any OS that is not POSIX certified is a trademark violation, but people say this all of the time. - http://get.posixcertified.ieee.org/certification_guide.html - POSIX is a trademark of the IEEE - Unix is a trademark of OpenGroup.org - Linux is a trademark of Linux Torvalds On 3/11/2023 12:08 PM, Tomek CEDRO wrote: Please consult Unix manual and source code (for instance any of the BSD). Linux was never a reference in Unix world, it was about to mimic Unix, but was not even self-compatible. Not a good reference point. See how big mess it introduced in current drivers implementation (i.e. drm kms) and their propagation to other OS. Is this similar situation here? Device driver should not provide any additional processing / silent rewrite of data, just provide raw data, that is then processed by the application. Driver may be configured by ioctl/sysclt, to enable additional / specific behavior (i.e. debug, buffer size, latency, etc), so the user always knows exactly what the driver does. I am pro Greg approach, so driver is always simple and generic, everything else is up to application. This is the Unix way and you will know how most things work, because they all work the same generic standardized way, as simple as possible, nothing else happens "in the background unnoticed". If a specific device needs a specific handling by a generic driver, then "quirks" may be used. That explicitly mark "specific" behavior for a given device (and only this device). -- CeDeROM, SQ7MHZ,http://www.tomek.cedro.info
Re: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Please consult Unix manual and source code (for instance any of the BSD). Linux was never a reference in Unix world, it was about to mimic Unix, but was not even self-compatible. Not a good reference point. See how big mess it introduced in current drivers implementation (i.e. drm kms) and their propagation to other OS. Is this similar situation here? Device driver should not provide any additional processing / silent rewrite of data, just provide raw data, that is then processed by the application. Driver may be configured by ioctl/sysclt, to enable additional / specific behavior (i.e. debug, buffer size, latency, etc), so the user always knows exactly what the driver does. I am pro Greg approach, so driver is always simple and generic, everything else is up to application. This is the Unix way and you will know how most things work, because they all work the same generic standardized way, as simple as possible, nothing else happens "in the background unnoticed". If a specific device needs a specific handling by a generic driver, then "quirks" may be used. That explicitly mark "specific" behavior for a given device (and only this device). -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
Re: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
On 3/10/2023 5:30 PM, Qi3 Huang 黄齐 wrote: I have to agree that the code is looking better all of the time. Many of the obvious problems that I saw just a couple of days ago have been addressed. Many of the things I have complained about in this thread have been fixed. But there are probably more issues that will show up. I think my biggest concern with the solution is that it > effectively makes it impossible to received any out-of-band data: Any > encoded, context-dependent, escaped data will be echoed back to the > host unconditionally making it impossible to receive cleanly. But this behavior can be disabled by termios if needed, and it only affect console port by default. And cr/lf convertion also break this. That is not always true. * The ECHO behavior can be disabled ONLY if TERMIOS are enabled. TERIMIOS is now required by POSIX and, further, if TERMIOS is not system, many features are now broken. Like hiding the password when logging into NSH. If isconsole is set and TERMIOS is not, then it will always echo the input. The CR-LF behavior only depends on isconsole and can't be changed even if TERMIOS is enabled. * Also, the trick of disabling ECHO is not a general solution. You cannot disable TERMIOS for asynchronous, out of band input because you don't know when that input is going to arrive. We have already seen this in escape sequences sent from the > host appearing appearing on the user console. An awful solution > was adopted: Just remove the escape sequence. That is called > "sweeping the problem under the rug." This will be a serious > limitation that will effect our capability to implement many serial > protocols and bad thing to happen to the OS. The current approach could be improved, such as let readline/cle to erase the escape sequence on console, what's your suggestion ? In the case of readline, we know when the ESC sequence is coming. readline() asked for the position and the host responded. For all such synchronous input, we could control the ECHO with TERMIOS. But we cannot for other asynchronous, out-of-band input. In general, I think it's a step forward, why we submit such a patch? 1.Telnet before this patch always echo input twice since telnet echo character itself by default. I have never seen that behavior. That bug is probably introduced recently.
答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
> I think my biggest concern with the solution is that it > effectively makes > it impossible to received any out-of-band data: Any > encoded, > context-dependent, escaped data will be echoed back to the > host > unconditionally making it impossible to receive cleanly. But this behavior can be disabled by termios if needed, and it only affect console port by default. And cr/lf convertion also break this. > We have already seen this in escape sequences sent from the > host appearing > appearing on the user console. An awful solution > was adopted: Just remove > the escape sequence. That is called > "sweeping the problem under the rug." > This will be a serious > limitation that will effect our capability to > implement many serial > protocols and bad thing to happen to the OS. The current approach could be improved, such as let readline/cle to erase the escape sequence on console, what's your suggestion ? In general, I think it's a step forward, why we submit such a patch? 1.Telnet before this patch always echo input twice since telnet echo character itself by default. 2.Many application must do echo itself, such as readline/cle/bas. But the bigge problem is, for some software ported from other posix compatible system, they don't do echo by themself, for these applications like WAMR's repl (respacially with REPL) you can't see the input unless you modify the source to adapt it to NuttX. ________ 发件人: Qi3 Huang 黄齐 发送时间: 2023年3月11日 0:05:41 收件人: dev@nuttx.apache.org 主题: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver Thanks for point that, so my change will affect them by serial.c since it's in common logic (excatly, is in uart_read()). ____________ 发件人: Gregory Nutt 发送时间: 2023年3月10日 23:58:29 收件人: dev@nuttx.apache.org 主题: Re: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver [外部邮件] 此邮件来源于小米公司外部,请谨慎处理。若对邮件安全性存疑,请将邮件转发给mi...@xiaomi.com进行反馈 > Emm, I confirmed that one of my test platform (esp32c3), its cdcacm driver > implement the uart_ops, but rp2040 seems not. > > > CLE should works since it will override the content from driver ECHO if over > serial driver, but readline shouldn't echo input. > > > I'll test more to confirm it. cdcacm.c is the part that is similar to serial.c. It implements the serial driver interface and also interfaces with the lower level USB device driver. cdcacm.c resides at drivers/usbdev/cdcacm.c and it common for all MCUs. The MCU-specif logic has no idea that there is a cdcacm "gadget" attached to it. cdcacm.c implements the common uart_ops that are used by all MCUs to support CDC/ACM serial. pl2303.c is another USB serial device. It also implements uart_ops and has its own TERMIOS implementation. #/**本邮件及其附件含有小米公司的保密信息,仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本邮件! This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!**/#
Re: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
On 3/10/2023 10:05 AM, Qi3 Huang 黄齐 wrote: Thanks for point that, so my change will affect them by serial.c since it's in common logic (excatly, is in uart_read()). You are right. But that is not true of telnet, is it?
答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Thanks for point that, so my change will affect them by serial.c since it's in common logic (excatly, is in uart_read()). 发件人: Gregory Nutt 发送时间: 2023年3月10日 23:58:29 收件人: dev@nuttx.apache.org 主题: Re: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver [外部邮件] 此邮件来源于小米公司外部,请谨慎处理。若对邮件安全性存疑,请将邮件转发给mi...@xiaomi.com进行反馈 > Emm, I confirmed that one of my test platform (esp32c3), its cdcacm driver > implement the uart_ops, but rp2040 seems not. > > > CLE should works since it will override the content from driver ECHO if over > serial driver, but readline shouldn't echo input. > > > I'll test more to confirm it. cdcacm.c is the part that is similar to serial.c. It implements the serial driver interface and also interfaces with the lower level USB device driver. cdcacm.c resides at drivers/usbdev/cdcacm.c and it common for all MCUs. The MCU-specif logic has no idea that there is a cdcacm "gadget" attached to it. cdcacm.c implements the common uart_ops that are used by all MCUs to support CDC/ACM serial. pl2303.c is another USB serial device. It also implements uart_ops and has its own TERMIOS implementation. #/**本邮件及其附件含有小米公司的保密信息,仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本邮件! This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!**/#
Re: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Emm, I confirmed that one of my test platform (esp32c3), its cdcacm driver implement the uart_ops, but rp2040 seems not. CLE should works since it will override the content from driver ECHO if over serial driver, but readline shouldn't echo input. I'll test more to confirm it. cdcacm.c is the part that is similar to serial.c. It implements the serial driver interface and also interfaces with the lower level USB device driver. cdcacm.c resides at drivers/usbdev/cdcacm.c and it common for all MCUs. The MCU-specif logic has no idea that there is a cdcacm "gadget" attached to it. cdcacm.c implements the common uart_ops that are used by all MCUs to support CDC/ACM serial. pl2303.c is another USB serial device. It also implements uart_ops and has its own TERMIOS implementation.
答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
> Telnet is the same. It does not use any of the changes to serial.c. It > redirects stdin and stdout to the socket interface exchages raw data > with the host. It has now TERMIOS support. I'm sure telnet will echo input itself, so it works with some display issues before. 发件人: Gregory Nutt 发送时间: 2023年3月10日 23:47:52 收件人: dev@nuttx.apache.org 主题: Re: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver [外部邮件] 此邮件来源于小米公司外部,请谨慎处理。若对邮件安全性存疑,请将邮件转发给mi...@xiaomi.com进行反馈 > >> nsh over usb works with readline/cle now. >> >> >> In face, isconsole in current implementation is more like a very tiny >> `line driver` inside the serial driver. > > That is very interesting. The serial driver > (drivers/usbdev/cdcacm.c). Does not include any of your change. It > does not use serial.c . It has its own serial interface and its own > TERMIOS implementation that does not use your changes. > > So I would expect serial.c to be better behaved without the ECHO as well. > Telnet is the same. It does not use any of the changes to serial.c. It redirects stdin and stdout to the socket interface exchages raw data with the host. It has now TERMIOS support. #/**本邮件及其附件含有小米公司的保密信息,仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本邮件! This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!**/#
答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Emm, I confirmed that one of my test platform (esp32c3), its cdcacm driver implement the uart_ops, but rp2040 seems not. CLE should works since it will override the content from driver ECHO if over serial driver, but readline shouldn't echo input. I'll test more to confirm it. 发件人: Gregory Nutt 发送时间: 2023年3月10日 23:22:45 收件人: dev@nuttx.apache.org 主题: Re: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver [外部邮件] 此邮件来源于小米公司外部,请谨慎处理。若对邮件安全性存疑,请将邮件转发给mi...@xiaomi.com进行反馈 > nsh over usb works with readline/cle now. > > > In face, isconsole in current implementation is more like a very tiny `line > driver` inside the serial driver. That is very interesting. The serial driver (drivers/usbdev/cdcacm.c). Does not include any of your change. It does not use serial.c . It has its own serial interface and its own TERMIOS implementation that does not use your changes. So I would expect serial.c to be better behaved without the ECHO as well. #/**本邮件及其附件含有小米公司的保密信息,仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本邮件! This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!**/#