Re: [Interest] Parsing data from serialport

[Roland Hughes]

On Monday April 8 22:59:23 CEST 2019 Martin Marmsoler wrote:

>What is, when I ignore the ready read? Has the internal buffer a max 
size?

>Because at the moment I have a memory leak and maybe it comes from that?

Martin,

Sorry to take so long in responding but this was going to be a long post
and I might have to bust it up in to multiple messages. I don't remember
what the maximum digest message size is. Easter Sunday is the only day I've
had enough time to write something like this. I'm also sorry you got so
much bad advice.
There is no way anyone who is actually a Dev would have missed

>each line consists of multiple columns of data

So the sloppy non-solutions you got are a sad thing.

I'm going to start from the ground up so you understand the how and why of
things without any code getting in the way. I'm also going to do this in a
basic text editor (KWrite) because HTML gets stripped out of digest email
and the archive so any email with HTML formatting which actually had a good
answer is lost to the sands of time.

There are two primary architectures, some call them schools of design and
others call them design patterns when it comes to serial communications.

Transient Comm - It doesn't really matter because you only use a tiny
piece of the data. No, the "solution" you were provided cannot be
justified even for this.

Persistent Comm - Every packet matters. Full hardware flow control is
mandatory. Software flow control is forbidden. Senders are designed to
stop operation when one or more pins drop. (If the collector/controller
crashes they don't continue operation.)

You can find the current DB-9 pin definitions here:
http://www.usconverters.com/index.php?main_page=page=61

It's important to note the DB-25  pinout came from the industrial
revolution, long before we had computers.

Transient Comm is most often found in systems using truck scales. All of
the packets are discarded until an operator hits a key or clicks on
something which either starts a new ticket transaction or looks to
complete an existing ticket transaction by adding a tare weight.

Settling Read/Weight

You don't get just one weight from a truck scale. It will bounce around
for a long time. Fuel sloshing in tanks, breeze catching side of trailer,
driving rummaging around in cab for phone charger, the list goes on and
on. Windy days mean the scale will never get to a single value.

What your application has to do is have all of the settling rules
configured in an indexed file or database table, NOT in INI, JSON, XML or
some other text file. (The reason why comes much later.)

Every scale at every company (unless they all happen to be the exact same
model and brand) will have different settling rules. Even if they are all
the same model and brand they will have different settling rules when
weighing different products.

https://www.beverlymaterials.com/sand-and-gravel-prices/
ftp://www.ilga.gov/jcar/admincode/008/00800600ZZ9998bR.html

Pea gravel at $11/ton really doesn't matter if you allow the spread
between high and low weights over time be 500 pounds. As you can see bi
the ilga.gov link, where a bushel of spinach is 12 pounds, the settling
weight range has to be very tight.

Settling is ordinarily range over time. In the computer world the time is
usually a second or less. You must keep track of the highest read and the
lowest read for the length of settling time. If the difference is less
than the allowed spread your settled reading is exactly in the middle.

Some places and products are very very strict. Settling is N-reads in a
row which are "the same." Time is unlimited. This form of settling is
event driven. "The Same" can be exactly identical or it can be plus or
minus a small percentage from the initial read.

In a Transient Comm world most reads don't matter. The serial reading
still has to be done in its own thread. Packet processing also has to be
done in its own thread. What really differs between Transient and
Persistent is the hand off from the reading thread to the rest of the
application.

In a Persistent Comm application you aren't allowed to lose a single read.
Hardware flow control is required so the sender will shut down when the
receiver goes dead. Transient Comm basically throws the read away at this
point unless you are in that time window of a transaction. Persistent Comm
has to record it, usually to a local fake database on disk or battery
backed RAM. There is a timestamp, processed flag set to false and usually
the port number/ID as the primary key with an alternate key of processed
flag and timestamp.

Once successfully written a signal of some kind (Qt signal, DBUS,
whatever) to the process control thread which could be the main event loop
in a Qt application. Worker threads can have a long run-time depending on
how many data sources are required to process a packet. The process
control thread is tasked with spinning up however many worker threads have
been configured based on the depth of the 

Re: [Interest] Parsing data from serialport

[Martin Marmsoler]

Thank you Andre for the fast response.

Ok than I found the problem, why the program need so much RAM 

Am Di., 9. Apr. 2019 um 08:21 Uhr schrieb Andre Hartmann <
andre.hartm...@iseg-hv.de>:

> Hi Martin,
>
> > What is, when I ignore the ready read?
>
> The data will stay in the buffer, of course.
>
> > Has the internal buffer a max size?
>
> Your RAM or 2^32, whichever is lower.
>
> > Because at the moment I have a memory leak and maybe it comes from
> > that?
>
> I doubt that. A leak would mean, you cannot use the data anymore, which
> is wrong here - you just need to fetch the data from the buffer.
>
> Regards,
> André
>
> >
> > Martin
> >
> > Am Fr., 5. Apr. 2019 um 21:12 Uhr schrieb Konstantin Shegunov
> > mailto:kshegu...@gmail.com>>:
> >
> > On Thu, Apr 4, 2019 at 6:42 PM Jérôme Godbout  > > wrote:
> >
> > You jus did the same loop into your slot, the function I did
> > made can be called as slot to the connect you just told, I agree
> > the serial_port->waitForReadyRead(5); is not necessary, just
> > often the transmission have partial or bunch of data by small
> > burst (USB VCOM) and was handy for me, but it is not ncessary in
> > any way.
> >
> > A similar loop yes, however your loop goes until there's something
> > to read on the port, mine is run a few times only when the port
> > already has some information pending in its buffer. The QSerialPort
> > class already knows if data has arrived and buffers it*, so what
> > would be the point to loop over the port when there's no data? When
> > you talk with someone do you ask every 5ms if [s]he has something to
> > say to you? I really doubt it. You'd normally just wait for
> > something to be said to decide if and how to respond to it.
> >
> > Side note: ReadLines made it less general purpose, you will need
> > a protocol that use endline to signal end of message which is
> > not always the case.
> >
> > Involving realistic protocol handling is another kettle of fish. It
> > would require much more thought and effort and is beyond the scope
> > of what was asked initially.
> >
> > * Unless it's in unbuffered mode, as Thiago already noted.
> >
> > ___
> > Interest mailing list
> > Interest@qt-project.org 
> > https://lists.qt-project.org/listinfo/interest
> >
> >
> > ___
> > Interest mailing list
> > Interest@qt-project.org
> > https://lists.qt-project.org/listinfo/interest
> >
>
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Andre Hartmann]

Hi Martin,

What is, when I ignore the ready read? 


The data will stay in the buffer, of course.

Has the internal buffer a max size? 


Your RAM or 2^32, whichever is lower.

Because at the moment I have a memory leak and maybe it comes from 
that?


I doubt that. A leak would mean, you cannot use the data anymore, which 
is wrong here - you just need to fetch the data from the buffer.


Regards,
André



Martin

Am Fr., 5. Apr. 2019 um 21:12 Uhr schrieb Konstantin Shegunov 
mailto:kshegu...@gmail.com>>:


On Thu, Apr 4, 2019 at 6:42 PM Jérôme Godbout mailto:godbo...@amotus.ca>> wrote:

You jus did the same loop into your slot, the function I did
made can be called as slot to the connect you just told, I agree
the serial_port->waitForReadyRead(5); is not necessary, just
often the transmission have partial or bunch of data by small
burst (USB VCOM) and was handy for me, but it is not ncessary in
any way.

A similar loop yes, however your loop goes until there's something
to read on the port, mine is run a few times only when the port
already has some information pending in its buffer. The QSerialPort
class already knows if data has arrived and buffers it*, so what
would be the point to loop over the port when there's no data? When
you talk with someone do you ask every 5ms if [s]he has something to
say to you? I really doubt it. You'd normally just wait for
something to be said to decide if and how to respond to it.

Side note: ReadLines made it less general purpose, you will need
a protocol that use endline to signal end of message which is
not always the case.

Involving realistic protocol handling is another kettle of fish. It
would require much more thought and effort and is beyond the scope
of what was asked initially.

* Unless it's in unbuffered mode, as Thiago already noted.

___
Interest mailing list
Interest@qt-project.org 
https://lists.qt-project.org/listinfo/interest


___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest


___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Martin Marmsoler]

What is, when I ignore the ready read? Has the internal buffer a max size?
Because at the moment I have a memory leak and maybe it comes from that?

Martin

Am Fr., 5. Apr. 2019 um 21:12 Uhr schrieb Konstantin Shegunov <
kshegu...@gmail.com>:

> On Thu, Apr 4, 2019 at 6:42 PM Jérôme Godbout  wrote:
>
>> You jus did the same loop into your slot, the function I did made can be
>> called as slot to the connect you just told, I agree the
>> serial_port->waitForReadyRead(5); is not necessary, just often the
>> transmission have partial or bunch of data by small burst (USB VCOM) and
>> was handy for me, but it is not ncessary in any way.
>>
> A similar loop yes, however your loop goes until there's something to read
> on the port, mine is run a few times only when the port already has some
> information pending in its buffer. The QSerialPort class already knows if
> data has arrived and buffers it*, so what would be the point to loop over
> the port when there's no data? When you talk with someone do you ask every
> 5ms if [s]he has something to say to you? I really doubt it. You'd normally
> just wait for something to be said to decide if and how to respond to it.
>
>> Side note: ReadLines made it less general purpose, you will need a
>> protocol that use endline to signal end of message which is not always the
>> case.
>>
> Involving realistic protocol handling is another kettle of fish. It would
> require much more thought and effort and is beyond the scope of what was
> asked initially.
>
> * Unless it's in unbuffered mode, as Thiago already noted.
>
>> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
>
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Konstantin Shegunov]

On Thu, Apr 4, 2019 at 6:42 PM Jérôme Godbout  wrote:

> You jus did the same loop into your slot, the function I did made can be
> called as slot to the connect you just told, I agree the
> serial_port->waitForReadyRead(5); is not necessary, just often the
> transmission have partial or bunch of data by small burst (USB VCOM) and
> was handy for me, but it is not ncessary in any way.
>
A similar loop yes, however your loop goes until there's something to read
on the port, mine is run a few times only when the port already has some
information pending in its buffer. The QSerialPort class already knows if
data has arrived and buffers it*, so what would be the point to loop over
the port when there's no data? When you talk with someone do you ask every
5ms if [s]he has something to say to you? I really doubt it. You'd normally
just wait for something to be said to decide if and how to respond to it.

> Side note: ReadLines made it less general purpose, you will need a
> protocol that use endline to signal end of message which is not always the
> case.
>
Involving realistic protocol handling is another kettle of fish. It would
require much more thought and effort and is beyond the scope of what was
asked initially.

* Unless it's in unbuffered mode, as Thiago already noted.

>
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Konstantin Shegunov]

On Thu, Apr 4, 2019 at 9:37 PM  wrote:

> Try testing them with a million+ row table.
>

Been there, done that. Be a lamb and stop with the nonsense. I'm sure you
run every file operation in a dedicated thread, 'cause you'd *"never ever
ever do I/O in the main GUI thread of a Qt application" ...*
Writing like a politician, with the all the generalizations without any
context, and context is everything, ain't impressing nobody here. You're
talking to devs, not to some old lady in the mall.

On Fri, Apr 5, 2019 at 5:14 PM Jason H  wrote:

> Yea, I'm going to have to agree with Denis over Roland.
>

Makes three of us then (and I'm sure many more silent ones if they cared to
comment on the silliness).


> In general it's not good to do synchronous operations (expecially I/O) in
> the main thread, but Qt I/O is mostly* asyc.
>

Yes, depending on the case, the volume of information and latency you're
after. For a console application I wouldn't care much about blocking the
event loop most of the time ...

>

> A lot of people don't realize that as soon as you ask for I/O your thread
> gets suspended by the kernel anyway.
>

Indeed. It used to be, on low-level side of things, that waiting/sleeping
was implemented like an empty IO operation (not sure if it's still the
case).
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Jason H]

Yea, I'm going to have to agree with Denis over Roland. In general it's not good to do synchronous operations (expecially I/O) in the main thread, but Qt I/O is mostly* asyc.

 

*caveat: data reading/writng is async, but things like QDir::mkDir() are sync. So if you're on a slow 5400RPM disk with a swap file on it on a system that is actively swapping, in a deep direcyory structure, on old fielsystems, it might take a while (100s of ms).

 

I'm not sure if Qt wants fully async I/O as I've witnessed what it does to a code base - some functions are async others are sync and you don't want that mess in your code. Coroutines will be a C++ thing (C++20), but I migh prefer the clarity of just being syncronous in your own thread. A lot of people don't realize that as soon as you ask for I/O your thread gets suspended by the kernel anyway. There are corner cases where you can get slughtly better performance because you can avoid the context switch, but the penalty on codebase is large as readability suffers. If the async nature could be completely hidden, that would be fine. But cyrrently in JS and Python, it's an imperfect abstraction.

 

 

Sent: Friday, April 05, 2019 at 5:45 AM
From: "Denis Shienkov" 
To: rol...@logikalsolutions.com
Cc: h...@gmx.com, interest@qt-project.org
Subject: Re: [Interest] Parsing data from serialport












> Never ever ever do I/O in the main GUI thread of a Qt application.

 

Why not? It is an asynchronous. You can do I/O in a main thread, but handle data in a separate thread (e.g. to parse it). Of course, it depends on the used communication protocol and etc.

A main limitation is that on Windows all I/O stops when a user hold-on the application window (ot to drag it, or do resizing). It is a Windows 'feature'.

 

So, yes, in a common case it is makes sense to move the QSerialPort object to the different thread (a concept of workers).

 

> Real world serial comm is fraught with peril. I've been doing it since the days of DOS 3.1.

 

It's not true.

 

> Unless something has dramatically changed in QSerialPort, readyRead is only emitted for the first character in the buffer because there is no concept of block or packet. A UART handles one byte at a time.

 

It's not true too. The readyRead() will be triggered when the device's handle/descriptor becomes 'signalled/activated' (when something comes into FIFO of driver), in this case the QSerialPort reads all data which are in this time in the FIFO (there are may be some bytes). So, no any byte-per-byte handling as you say.

 

> When you get a readyRead and fail to completely empty the buffer, that's it. You never get another one.

 

I'm don't understand what do you mean...

 

> When QSerialPort was taken from Playground and had all its method names changed, there was an odd timing bug too. In a production system running multiple ports of embedded target the code loading the next byte into the class buffer (perhaps not completely Qt code as it could have been in the device driver). The adding of the next byte could begin after readLine() determined how many bytes were in the buffer but prior to actually pulling them. The new byte got added. One less than a full buffer got read and readyRead never fired again because the next byte wasn't being written to a pristine empty buffer.

 

I don't know about what you say... It is trash for my opinion.. For you has been provided a good code example how to read the lines:

 

>     QSerialPort * port;
>     QObject::connect(port, ::readyRead, port, [port] () -> void  {
>         while (port->canReadLine())  {
>             QByteArray data = "">
>             // emit with data as argument and do the parsing
>         }
>     });

 

BR,

Denis

 

 

 








 


пт, 5 апр. 2019 г. в 10:38, <rol...@logikalsolutions.com>:


Quoting interest-requ...@qt-project.org:

> I think the point is that there's little reason to poll the serial port if
> you can react to the event. Exactly what you'd do if you had a network
> socket. Qt already does the heavy lifting for you, so you only need to
> react to the signal and read as much as you want/need. Basically:
>
>     QSerialPort * port;
>     QObject::connect(port, ::readyRead, port, [port] () -> void  {
>         while (port->canReadLine())  {
>             QByteArray data = "">
>             // emit with data as argument and do the parsing
>         }
>     });
>
> Whether you have the port in another thread or not is irrelevant in this
> case, either can work fine (unlike your while-sleep loop).

Never ever ever do I/O in the main GUI thread of a Qt application. 
It's a recipe for disaster despite the countless examples you will 
find posted on-line and even in the Qt examples themselves. Those 
database examples are really nice. Try testing them with a million+ 
row table.

Real world serial comm is f

Re: [Interest] Parsing data from serialport

[Denis Shienkov]

> Never ever ever do I/O in the main GUI thread of a Qt application.

Why not? It is an asynchronous. You can do I/O in a main thread, but handle
data in a separate thread (e.g. to parse it). Of course, it depends on the
used communication protocol and etc.
A main limitation is that on Windows all I/O stops when a user hold-on the
application window (ot to drag it, or do resizing). It is a Windows
'feature'.

So, yes, in a common case it is makes sense to move the QSerialPort object
to the different thread (a concept of workers).

> Real world serial comm is fraught with peril. I've been doing it since
the days of DOS 3.1.

It's not true.

> Unless something has dramatically changed in QSerialPort, readyRead is
only emitted for the first character in the buffer because there is no
concept of block or packet. A UART handles one byte at a time.

It's not true too. The readyRead() will be triggered when the device's
handle/descriptor becomes 'signalled/activated' (when something comes into
FIFO of driver), in this case the QSerialPort reads all data which are in
this time in the FIFO (there are may be some bytes). So, no any
byte-per-byte handling as you say.

> When you get a readyRead and fail to completely empty the buffer, that's
it. You never get another one.

I'm don't understand what do you mean...

> When QSerialPort was taken from Playground and had all its method names
changed, there was an odd timing bug too. In a production system running
multiple ports of embedded target the code loading the next byte into the
class buffer (perhaps not completely Qt code as it could have been in the
device driver). The adding of the next byte could begin after readLine()
determined how many bytes were in the buffer but prior to actually pulling
them. The new byte got added. One less than a full buffer got read and
readyRead never fired again because the next byte wasn't being written to a
pristine empty buffer.

I don't know about what you say... It is trash for my opinion.. For you has
been provided a good code example how to read the lines:

> QSerialPort * port;
> QObject::connect(port, ::readyRead, port, [port] () ->
void  {
> while (port->canReadLine())  {
> QByteArray data = port->readLine();
> // emit with data as argument and do the parsing
> }
> });

BR,
Denis




пт, 5 апр. 2019 г. в 10:38, :

>
> Quoting interest-requ...@qt-project.org:
>
> > I think the point is that there's little reason to poll the serial port
> if
> > you can react to the event. Exactly what you'd do if you had a network
> > socket. Qt already does the heavy lifting for you, so you only need to
> > react to the signal and read as much as you want/need. Basically:
> >
> > QSerialPort * port;
> > QObject::connect(port, ::readyRead, port, [port] () ->
> void  {
> > while (port->canReadLine())  {
> > QByteArray data = port->readLine();
> > // emit with data as argument and do the parsing
> > }
> > });
> >
> > Whether you have the port in another thread or not is irrelevant in this
> > case, either can work fine (unlike your while-sleep loop).
>
> Never ever ever do I/O in the main GUI thread of a Qt application.
> It's a recipe for disaster despite the countless examples you will
> find posted on-line and even in the Qt examples themselves. Those
> database examples are really nice. Try testing them with a million+
> row table.
>
> Real world serial comm is fraught with peril. I've been doing it since
> the days of DOS 3.1.
>
> Before anyone goes running off and using the above lambda, they need
> to consider a few things.
>
> Unless something has dramatically changed in QSerialPort, readyRead is
> only emitted for the first character in the buffer because there is no
> concept of block or packet. A UART handles one byte at a time.
>
> When you get a readyRead and fail to completely empty the buffer,
> that's it. You never get another one.
>
> When QSerialPort was taken from Playground and had all its method
> names changed, there was an odd timing bug too. In a production system
> running multiple ports of embedded target the code loading the next
> byte into the class buffer (perhaps not completely Qt code as it could
> have been in the device driver). The adding of the next byte could
> begin after readLine() determined how many bytes were in the buffer
> but prior to actually pulling them. The new byte got added. One less
> than a full buffer got read and readyRead never fired again because
> the next byte wasn't being written to a pristine empty buffer.
>
>
> --
> Roland Hughes, President
> Logikal Solutions
> (630) 205-1593
>
> http://www.theminimumyouneedtoknow.com
> http://www.infiniteexposure.net
> http://www.johnsmith-book.com
> http://www.logikalblog.com
> http://www.interestingauthors.com/blog
> http://lesedi.us
>
> ___
> Interest mailing list
> Interest@qt-project.org
> 

Re: [Interest] Parsing data from serialport

[roland]

Quoting interest-requ...@qt-project.org:


I think the point is that there's little reason to poll the serial port if
you can react to the event. Exactly what you'd do if you had a network
socket. Qt already does the heavy lifting for you, so you only need to
react to the signal and read as much as you want/need. Basically:

QSerialPort * port;
QObject::connect(port, ::readyRead, port, [port] () -> void  {
while (port->canReadLine())  {
QByteArray data = port->readLine();
// emit with data as argument and do the parsing
}
});

Whether you have the port in another thread or not is irrelevant in this
case, either can work fine (unlike your while-sleep loop).


Never ever ever do I/O in the main GUI thread of a Qt application.  
It's a recipe for disaster despite the countless examples you will  
find posted on-line and even in the Qt examples themselves. Those  
database examples are really nice. Try testing them with a million+  
row table.


Real world serial comm is fraught with peril. I've been doing it since  
the days of DOS 3.1.


Before anyone goes running off and using the above lambda, they need  
to consider a few things.


Unless something has dramatically changed in QSerialPort, readyRead is  
only emitted for the first character in the buffer because there is no  
concept of block or packet. A UART handles one byte at a time.


When you get a readyRead and fail to completely empty the buffer,  
that's it. You never get another one.


When QSerialPort was taken from Playground and had all its method  
names changed, there was an odd timing bug too. In a production system  
running multiple ports of embedded target the code loading the next  
byte into the class buffer (perhaps not completely Qt code as it could  
have been in the device driver). The adding of the next byte could  
begin after readLine() determined how many bytes were in the buffer  
but prior to actually pulling them. The new byte got added. One less  
than a full buffer got read and readyRead never fired again because  
the next byte wasn't being written to a pristine empty buffer.



--
Roland Hughes, President
Logikal Solutions
(630) 205-1593

http://www.theminimumyouneedtoknow.com
http://www.infiniteexposure.net
http://www.johnsmith-book.com
http://www.logikalblog.com
http://www.interestingauthors.com/blog
http://lesedi.us

___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Jérôme Godbout]

You jus did the same loop into your slot, the function I did made can be called 
as slot to the connect you just told, I agree the 
serial_port->waitForReadyRead(5); is not necessary, just often the transmission 
have partial or bunch of data by small burst (USB VCOM) and was handy for me, 
but it is not ncessary in any way.



Perfoming heavy IO on main thread is often a bad idea and lead bad GUI 
reactivity, that’s all, but nothing prevent doing the call into it.



Side note: ReadLines made it less general purpose, you will need a protocol 
that use endline to signal end of message which is not always the case.

From: Konstantin Shegunov 
Sent: April 4, 2019 11:19 AM
To: Jérôme Godbout 
Cc: Paolo Angelelli ; interest@qt-project.org
Subject: Re: [Interest] Parsing data from serialport

On Thu, Apr 4, 2019 at 5:46 PM Jérôme Godbout 
mailto:godbo...@amotus.ca>> wrote:
It relief the main thread from read data all the time and keep a good 
reactivity. The dedicated thread try to read and can wait until something come 
along and once it found something that can be parsed, it emit his own signal 
that the main thread only have to handle into normal slot (will be queued, 
since it's not the same thread).

But yeah, you can have the main thread do it and process the readyRead(), you 
main thread will perform the read and parsing. if you have any CRC and other 
things, this might be bad for application reactivity, depending on the amount 
of data flowing.

-Original Message-
From: Interest 
mailto:interest-boun...@qt-project.org>> On 
Behalf Of Paolo Angelelli
Sent: April 4, 2019 10:19 AM
To: interest@qt-project.org<mailto:interest@qt-project.org>
Subject: Re: [Interest] Parsing data from serialport

What is the advantage of having such a continuous reading loop in a separate 
thread?

I mean it as opposed to reacting to the readyRead() signal, and having a 
while(canReadLine()) { ... } in the reacting slot.

I think the point is that there's little reason to poll the serial port if you 
can react to the event. Exactly what you'd do if you had a network socket. Qt 
already does the heavy lifting for you, so you only need to react to the signal 
and read as much as you want/need. Basically:

QSerialPort * port;
QObject::connect(port, ::readyRead, port, [port] () -> void  {
while (port->canReadLine())  {
QByteArray data = port->readLine();
// emit with data as argument and do the parsing
}
});
Whether you have the port in another thread or not is irrelevant in this case, 
either can work fine (unlike your while-sleep loop).
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Konstantin Shegunov]

On Thu, Apr 4, 2019 at 5:46 PM Jérôme Godbout  wrote:

> It relief the main thread from read data all the time and keep a good
> reactivity. The dedicated thread try to read and can wait until something
> come along and once it found something that can be parsed, it emit his own
> signal that the main thread only have to handle into normal slot (will be
> queued, since it's not the same thread).
>
> But yeah, you can have the main thread do it and process the readyRead(),
> you main thread will perform the read and parsing. if you have any CRC and
> other things, this might be bad for application reactivity, depending on
> the amount of data flowing.
>
> -Original Message-
> From: Interest  On Behalf Of Paolo
> Angelelli
> Sent: April 4, 2019 10:19 AM
> To: interest@qt-project.org
> Subject: Re: [Interest] Parsing data from serialport
>
> What is the advantage of having such a continuous reading loop in a
> separate thread?
>
> I mean it as opposed to reacting to the readyRead() signal, and having a
> while(canReadLine()) { ... } in the reacting slot.
>

I think the point is that there's little reason to poll the serial port if
you can react to the event. Exactly what you'd do if you had a network
socket. Qt already does the heavy lifting for you, so you only need to
react to the signal and read as much as you want/need. Basically:

QSerialPort * port;
QObject::connect(port, ::readyRead, port, [port] () -> void  {
while (port->canReadLine())  {
QByteArray data = port->readLine();
// emit with data as argument and do the parsing
}
});

Whether you have the port in another thread or not is irrelevant in this
case, either can work fine (unlike your while-sleep loop).
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Jason H]

Maybe you don't need another thread, just maybe an event loop (QEventLoop)?


> Sent: Thursday, April 04, 2019 at 10:43 AM
> From: "Jérôme Godbout" 
> To: "Paolo Angelelli" , "interest@qt-project.org" 
> 
> Subject: Re: [Interest] Parsing data from serialport
>
> It relief the main thread from read data all the time and keep a good 
> reactivity. The dedicated thread try to read and can wait until something 
> come along and once it found something that can be parsed, it emit his own 
> signal that the main thread only have to handle into normal slot (will be 
> queued, since it's not the same thread).
> 
> But yeah, you can have the main thread do it and process the readyRead(), you 
> main thread will perform the read and parsing. if you have any CRC and other 
> things, this might be bad for application reactivity, depending on the amount 
> of data flowing.
> 
> -Original Message-
> From: Interest  On Behalf Of Paolo Angelelli
> Sent: April 4, 2019 10:19 AM
> To: interest@qt-project.org
> Subject: Re: [Interest] Parsing data from serialport
> 
> What is the advantage of having such a continuous reading loop in a separate 
> thread?
> 
> I mean it as opposed to reacting to the readyRead() signal, and having a 
> while(canReadLine()) { ... } in the reacting slot.
> 
> 
> On Tue, 2 Apr 2019 22:01:49 +
> Jérôme Godbout  wrote:
> 
> > Make sure your reading loop and processing data are separated. Call you 
> > read device when needed or into a loop that can take some pause to avoid 
> > 100% CPU usage for nothing.
> > 
> > QByteArray buffer;
> > 
> > void ReadDeviceHaveData()
> > {
> > 
> >while(serial_port->bytesAvailable()) // This can be dangerous is 
> > data keep coming  and might be removed {
> > // You can read bytes per bytes or smaller chunk over here for 
> > better reactivity and less memory consumption
> > buffer.append(serial_port->readAll());
> > processData();
> > serial_port->waitForReadyRead(5);
> >  }
> > }
> > 
> > void ProcessData()
> > {
> >int pos = buffer.indexOf(‘\n’);
> > 
> >while(pos >= 0)
> >{
> >  QByteArray line = buffer.left(pos);
> >  // Strip trailing \r for windows here
> >  // Do whatever you need with your line, check data integrity
> > 
> >  // Remove the processed data but leave the unprocessed data alone
> >  buffer.remove(0, pos + 1); // Remove \n too
> >  pos = buffer.indexOf(‘\n’);
> >}
> > }
> > 
> > From: Interest  On Behalf Of Martin 
> > Marmsoler
> > Sent: April 2, 2019 3:58 PM
> > To: Thiago Macieira 
> > Cc: interest@qt-project.org
> > Subject: Re: [Interest] Parsing data from serialport
> > 
> >  > To be able to roll back, in case your reading from the device 
> > didn't result in what you wanted or you got an error. See QDataStream.
> > Ah ok I understand.
> > 
> > So this minimal example
> > QSerialPort sPort;
> > sPort.open(QIODevice::ReadOnly);
> > 
> > if(sPort.waitForReadyRead(2000)){
> > 
> >   while (!device.atEnd()) {
> > 
> > if (device.canReadLine()) {
> > 
> >newData.push_back(device.readLine());
> > 
> >linesToRead++;
> > 
> > } else {
> > 
> >return;
> > 
> > }
> > 
> >}
> > 
> >...
> > 
> > }
> > 
> > works fine, if I go trough it step by step (maybe, because enouth data come 
> > in). But if I'm to fast it does not work.
> > 
> > If I'm using the signal readyRead I will have the same problem, 
> > because new data come everytime. So I check that in the
> > 
> > readyRead function if a complete line come in, and if no complete line 
> > I return without doing something otherwise I do
> > 
> > something with the data? Is this the right way?
> > 
> > 
> > 
> > Martin
> > 
> > 
> > 
> > 
> > Thiago Macieira 
> > mailto:thiago.macie...@intel.com>> schrieb am 
> > Di., 2. Apr. 2019, 18:02:
> > On Tuesday, 2 April 2019 07:04:03 PDT Martin Marmsoler wrote:
> > > Thank you Thiago for your response. But what is transactionstart for?  
> > 
> > To be able to roll back, in case your reading from the device didn't 
> > result in what you wanted or you got an error. See QDataStream.
> > 
> > --
> > Thiago Macieira - thiago.macieira (AT) intel.com<http://intel.com>
> >   Software Architect - Intel System Software Products
> > 
> > 
> > 
> > ___
> > Interest mailing list
> > Interest@qt-project.org<mailto:Interest@qt-project.org>
> > https://lists.qt-project.org/listinfo/interest
> 
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
>
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Jérôme Godbout]

It relief the main thread from read data all the time and keep a good 
reactivity. The dedicated thread try to read and can wait until something come 
along and once it found something that can be parsed, it emit his own signal 
that the main thread only have to handle into normal slot (will be queued, 
since it's not the same thread).

But yeah, you can have the main thread do it and process the readyRead(), you 
main thread will perform the read and parsing. if you have any CRC and other 
things, this might be bad for application reactivity, depending on the amount 
of data flowing.

-Original Message-
From: Interest  On Behalf Of Paolo Angelelli
Sent: April 4, 2019 10:19 AM
To: interest@qt-project.org
Subject: Re: [Interest] Parsing data from serialport

What is the advantage of having such a continuous reading loop in a separate 
thread?

I mean it as opposed to reacting to the readyRead() signal, and having a 
while(canReadLine()) { ... } in the reacting slot.


On Tue, 2 Apr 2019 22:01:49 +
Jérôme Godbout  wrote:

> Make sure your reading loop and processing data are separated. Call you read 
> device when needed or into a loop that can take some pause to avoid 100% CPU 
> usage for nothing.
> 
> QByteArray buffer;
> 
> void ReadDeviceHaveData()
> {
> 
>while(serial_port->bytesAvailable()) // This can be dangerous is 
> data keep coming  and might be removed {
> // You can read bytes per bytes or smaller chunk over here for 
> better reactivity and less memory consumption
> buffer.append(serial_port->readAll());
> processData();
> serial_port->waitForReadyRead(5);
>  }
> }
> 
> void ProcessData()
> {
>int pos = buffer.indexOf(‘\n’);
> 
>while(pos >= 0)
>{
>  QByteArray line = buffer.left(pos);
>  // Strip trailing \r for windows here
>  // Do whatever you need with your line, check data integrity
> 
>  // Remove the processed data but leave the unprocessed data alone
>  buffer.remove(0, pos + 1); // Remove \n too
>  pos = buffer.indexOf(‘\n’);
>}
> }
> 
> From: Interest  On Behalf Of Martin 
> Marmsoler
> Sent: April 2, 2019 3:58 PM
> To: Thiago Macieira 
> Cc: interest@qt-project.org
> Subject: Re: [Interest] Parsing data from serialport
> 
>  > To be able to roll back, in case your reading from the device 
> didn't result in what you wanted or you got an error. See QDataStream.
> Ah ok I understand.
> 
> So this minimal example
> QSerialPort sPort;
> sPort.open(QIODevice::ReadOnly);
> 
> if(sPort.waitForReadyRead(2000)){
> 
>   while (!device.atEnd()) {
> 
> if (device.canReadLine()) {
> 
>newData.push_back(device.readLine());
> 
>linesToRead++;
> 
> } else {
> 
>return;
> 
> }
> 
>}
> 
>...
> 
> }
> 
> works fine, if I go trough it step by step (maybe, because enouth data come 
> in). But if I'm to fast it does not work.
> 
> If I'm using the signal readyRead I will have the same problem, 
> because new data come everytime. So I check that in the
> 
> readyRead function if a complete line come in, and if no complete line 
> I return without doing something otherwise I do
> 
> something with the data? Is this the right way?
> 
> 
> 
> Martin
> 
> 
> 
> 
> Thiago Macieira mailto:thiago.macie...@intel.com>> 
> schrieb am Di., 2. Apr. 2019, 18:02:
> On Tuesday, 2 April 2019 07:04:03 PDT Martin Marmsoler wrote:
> > Thank you Thiago for your response. But what is transactionstart for?  
> 
> To be able to roll back, in case your reading from the device didn't 
> result in what you wanted or you got an error. See QDataStream.
> 
> --
> Thiago Macieira - thiago.macieira (AT) intel.com<http://intel.com>
>   Software Architect - Intel System Software Products
> 
> 
> 
> ___
> Interest mailing list
> Interest@qt-project.org<mailto:Interest@qt-project.org>
> https://lists.qt-project.org/listinfo/interest

___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Paolo Angelelli]

What is the advantage of having such a continuous reading loop in a separate 
thread?

I mean it as opposed to reacting to the readyRead() signal, and having a 
while(canReadLine()) { ... } in the reacting
slot.


On Tue, 2 Apr 2019 22:01:49 +
Jérôme Godbout  wrote:

> Make sure your reading loop and processing data are separated. Call you read 
> device when needed or into a loop that can take some pause to avoid 100% CPU 
> usage for nothing.
> 
> QByteArray buffer;
> 
> void ReadDeviceHaveData()
> {
> 
>while(serial_port->bytesAvailable()) // This can be dangerous is data keep 
> coming  and might be removed
> {
> // You can read bytes per bytes or smaller chunk over here for 
> better reactivity and less memory consumption
> buffer.append(serial_port->readAll());
> processData();
> serial_port->waitForReadyRead(5);
>  }
> }
> 
> void ProcessData()
> {
>int pos = buffer.indexOf(‘\n’);
> 
>while(pos >= 0)
>{
>  QByteArray line = buffer.left(pos);
>  // Strip trailing \r for windows here
>  // Do whatever you need with your line, check data integrity
> 
>  // Remove the processed data but leave the unprocessed data alone
>  buffer.remove(0, pos + 1); // Remove \n too
>  pos = buffer.indexOf(‘\n’);
>}
> }
> 
> From: Interest  On Behalf Of Martin Marmsoler
> Sent: April 2, 2019 3:58 PM
> To: Thiago Macieira 
> Cc: interest@qt-project.org
> Subject: Re: [Interest] Parsing data from serialport
> 
>  > To be able to roll back, in case your reading from the device didn't 
> result in  
> what you wanted or you got an error. See QDataStream.
> Ah ok I understand.
> 
> So this minimal example
> QSerialPort sPort;
> sPort.open(QIODevice::ReadOnly);
> 
> if(sPort.waitForReadyRead(2000)){
> 
>   while (!device.atEnd()) {
> 
> if (device.canReadLine()) {
> 
>newData.push_back(device.readLine());
> 
>linesToRead++;
> 
> } else {
> 
>return;
> 
> }
> 
>}
> 
>...
> 
> }
> 
> works fine, if I go trough it step by step (maybe, because enouth data come 
> in). But if I'm to fast it does not work.
> 
> If I'm using the signal readyRead I will have the same problem, because new 
> data come everytime. So I check that in the
> 
> readyRead function if a complete line come in, and if no complete line I 
> return without doing something otherwise I do
> 
> something with the data? Is this the right way?
> 
> 
> 
> Martin
> 
> 
> 
> 
> Thiago Macieira mailto:thiago.macie...@intel.com>> 
> schrieb am Di., 2. Apr. 2019, 18:02:
> On Tuesday, 2 April 2019 07:04:03 PDT Martin Marmsoler wrote:
> > Thank you Thiago for your response. But what is transactionstart for?  
> 
> To be able to roll back, in case your reading from the device didn't result in
> what you wanted or you got an error. See QDataStream.
> 
> --
> Thiago Macieira - thiago.macieira (AT) intel.com<http://intel.com>
>   Software Architect - Intel System Software Products
> 
> 
> 
> ___
> Interest mailing list
> Interest@qt-project.org<mailto:Interest@qt-project.org>
> https://lists.qt-project.org/listinfo/interest

___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Martin Marmsoler]

Thank you Jérome for the explanation. I implement it like you said.

Martin

Jérôme Godbout  schrieb am Mi., 3. Apr. 2019, 15:12:

> Because you cannot unread data from the device and you have no guarantee
> that all the data will be present when you do read, you need a buffer since
> you need a whole line to process the data and data might be present but the
> line might not be fully arrived yet, maybe just half of it. The IODevice
> (Serial) buffer is buffering the reception of the data until you read it,
> your buffer is checking data is competed to be parsed, they have a
> different purpose.
>
>
>
>
> [image: 36E56279]
>
> une compagnie
>
> RAPPROCHEZ LA DISTANCE
>
> *Jérôme Godbout*
> Développeur Logiciel Sénior /
> Senior Software Developer
>
> *p:* +1 (418) 800-1073 ext.:109
>
> amotus.ca <http://www.amotus-solutions.com/>
> statum-iot.com
>
> <https://www.facebook.com/LesSolutionsAmotus/>
> <https://www.linkedin.com/company/amotus-solutions/>
> <https://twitter.com/AmotusSolutions>
> <https://www.youtube.com/channel/UCoYpQgsmj1iJZyDjTQ3x8Ig>
>
>
>
>
>
> *From:* Martin Marmsoler 
> *Sent:* April 3, 2019 3:49 AM
> *To:* Jérôme Godbout 
> *Cc:* Thiago Macieira ; interest@qt-project.org
> *Subject:* Re: [Interest] Parsing data from serialport
>
>
>
> But why I need a new buffer when there already one exist? (the QIODevice
> and my own buffer or?)
>
>
>
> My idea:
>
>
>
> Main() {
>
> Connect(readyread, myreadyread)
>
> }
>
>
>
> myreadyread() {
>
>
>
> If (!canreadline())
>
>Return;
>
>
>
> Parsedata() ;
>
>
>
> }
>
>
>
> With this idea I have again the problem with the while loop, but I dont
> need two buffers.
>
> Is it an alternative?
>
>
>
> Martin
>
>
>
> Jérôme Godbout  schrieb am Mi., 3. Apr. 2019, 00:01:
>
> Make sure your reading loop and processing data are separated. Call you
> read device when needed or into a loop that can take some pause to avoid
> 100% CPU usage for nothing.
>
>
>
> QByteArray buffer;
>
>
>
> void ReadDeviceHaveData()
>
> {
>
>while(serial_port->*bytesAvailable*()) // This can be dangerous is data 
> keep coming  and might be removed
>
> {
>
> // You can read bytes per bytes or smaller chunk over here for
> better reactivity and less memory consumption
>
> buffer.append(serial_port->readAll());
>
> processData();
>
> serial_port->*waitForReadyRead*(5);
>
>  }
>
> }
>
>
>
> void ProcessData()
>
> {
>
>int pos = buffer.indexOf(‘\n’);
>
>
>
>while(pos >= 0)
>
>{
>
>  QByteArray line = buffer.left(pos);
>
>      // Strip trailing \r for windows here
>
>  // Do whatever you need with your line, check data integrity
>
>
>
>  // Remove the processed data but leave the unprocessed data alone
>
>  buffer.remove(0, pos + 1); // Remove \n too
>
>  pos = buffer.indexOf(‘\n’);
>
>}
>
> }
>
>
>
> *From:* Interest  *On Behalf Of *Martin
> Marmsoler
> *Sent:* April 2, 2019 3:58 PM
> *To:* Thiago Macieira 
> *Cc:* interest@qt-project.org
> *Subject:* Re: [Interest] Parsing data from serialport
>
>
>
>  > To be able to roll back, in case your reading from the device didn't
> result in
> what you wanted or you got an error. See QDataStream.
>
> Ah ok I understand.
>
>
>
> So this minimal example
>
> QSerialPort sPort;
>
> sPort.open(QIODevice::ReadOnly);
>
> if(sPort.waitForReadyRead(2000)){
>
>   while (!device.*atEnd*()) {
>
> if (device.*canReadLine*()) {
>
>newData.push_back(device.readLine());
>
>linesToRead++;
>
> } else {
>
>return;
>
> }
>
>}
>
>...
>
> }
>
> works fine, if I go trough it step by step (maybe, because enouth data come 
> in). But if I'm to fast it does not work.
>
> If I'm using the signal readyRead I will have the same problem, because new 
> data come everytime. So I check that in the
>
> readyRead function if a complete line come in, and if no complete line I 
> return without doing something otherwise I do
>
> something with the data? Is this the right way?
>
>
>
> Martin
>
>
>
>
>
>
>
>
>
> Thiago Macieira  schrieb am Di., 2. Apr. 2019,
> 18:02:
>
> On Tuesday, 2 April 2019 07:04:03 PDT Martin Marmsoler wrote:
> > Thank you Thiago for your response. But what is transactionstart for?
>
> To be able to roll back, in case your reading from the device didn't
> result in
> what you wanted or you got an error. See QDataStream.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
>   Software Architect - Intel System Software Products
>
>
>
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
>
>
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Roland Hughes]

You probably want to read as far as I got with this series.

http://www.logikalsolutions.com/wordpress/information-technology/how-far-weve-come-pt-1/

Be sure to pull down the code for the various SerialKeypad implementations.

Here is a direct link to one of the zip files which was posted in case 
you are impatient.


http://www.logikalsolutions.com/wordpress/wp-content/uploads/2016/07/SerialKeypad.zip

While that series wasn't meant to be a tutorial on serial communications 
with Qt, quite a few nuggets of what you need to know are there.


I've got some other code floating around for doing serial communication 
with Qt but, as you can tell, I don't check this list regularly.


You can do a search for CommLib because whenever I'm talking about 
GreenLeaf CommLib from back in the days of DOS I'm generally talking 
about how to do serial communications right. There are many many many 
many gotchas today's platforms do not magically fix for you.


At the core of GreenLeaf's success/brilliance was the double ring 
buffer. Their little TSR (under DOS) had one ring buffer for writing to 
serial port and one the TSR stuff read characters into. It also kept 
track of whether it wrapped either because you wrote too much or waited 
too long to read. The ring buffer layer allowed the library to do 
_exactly_ what every kid wants to do when they write their first serial 
program. Read and write a packet. At the serial comm layer there is no 
concept of a "packet" beyond what you configured to be a byte. While 
many use 8-N-1, one cannot rely on that being the case.


Some devices won't even transmit a full 8 bits. Most newer ones will, 
but don't count on it. In particular, scale systems which only need to 
transmit the digits 0-9 and a "settled" indicator along with a checksum 
or CRC of some kind were notorious for nibbles. Some could even use 
their own form of packed decimal to keep the packet short for 300 baud 
transmission. Adding insult to injury were those scales (and other 
devices) which had no self control when it came to calculating the 
CRC/CheckSum byte. They would have a start-packet indicator as well as 
an end-packet value and the CRC/Checksum would end up being one of those 
when the weight was just right.


Unlike TCP/IP, _you_ have to do all error handling and planning. Do not 
assume a clear communications channel. Random electrical currents, 
magnets or even someone slapping something down on a desk can generate 
noise on the transmission line. (That last part applies more to modems 
than hard wire port to port but it does apply to hard wire, specially if 
the connection isn't screwed down tight. Just "feels tight" when pushed 
on isn't good enough. An imperceptible amount of wiggle can introduce 
noise on the line.)


If this is a production system, forget trying to do serial comm in your 
primary (GUI) thread. In a perfect world with only happy path testing 
you will get this to work and it will fail in the field. Something the 
user does or the GUI does or the database or some other thing stolen 
from an example program and left to run in the primary event loop will 
take __just__ long enough for your intricate well designed 
communications code to miss one or more critical control characters 
causing it to get completely lost.


Roland

On 4/1/19 5:00 AM, interest-requ...@qt-project.org wrote:

Hello,

I'm trying to implement a serial connection with QIODevice (SerialPort). I
would like to parse line by line (each line consists of multiple columns of
data) and for this I found canReadLine().
But as I understand I have to use startTransaction, because otherwise the
buffer will not be used. I'm I correct?

bool QIODevice::canReadLine() const
{
 Q_D(const QIODevice);
 return d->buffer.indexOf('\n', d->buffer.size(),
  d->isSequential() ? d->transactionPos :
Q_INT64_C(0)) >= 0;
}

So to read a full line, I have to open a QSerialPort and then start a
Transaction. Then I have to wait a little bit (till data is in the buffer)
and then I can use canReadLine. Am I correct? Is there a better way instead
to wait a little bit? Or is it the wrong way to go?


--
Roland Hughes, President
Logikal Solutions
(630)-205-1593  (cell)
http://www.theminimumyouneedtoknow.com
http://www.infiniteexposure.net
http://www.johnsmith-book.com

___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Jérôme Godbout]

Because you cannot unread data from the device and you have no guarantee that 
all the data will be present when you do read, you need a buffer since you need 
a whole line to process the data and data might be present but the line might 
not be fully arrived yet, maybe just half of it. The IODevice (Serial) buffer 
is buffering the reception of the data until you read it, your buffer is 
checking data is competed to be parsed, they have a different purpose.


[36E56279]
une compagnie  [cid:image002.jpg@01D4E9FD.50AA6910]
RAPPROCHEZ LA DISTANCE

Jérôme Godbout
Développeur Logiciel Sénior /
Senior Software Developer

p: +1 (418) 800-1073 ext.:109

amotus.ca<http://www.amotus-solutions.com/>
statum-iot.com<http://statum-iot.com/>
[cid:image003.png@01D4E9FD.50AA6910]<https://www.facebook.com/LesSolutionsAmotus/>
 [cid:image004.png@01D4E9FD.50AA6910] 
<https://www.linkedin.com/company/amotus-solutions/>  
[cid:image005.png@01D4E9FD.50AA6910] <https://twitter.com/AmotusSolutions>  
[cid:image006.jpg@01D4E9FD.50AA6910] 
<https://www.youtube.com/channel/UCoYpQgsmj1iJZyDjTQ3x8Ig>





From: Martin Marmsoler 
Sent: April 3, 2019 3:49 AM
To: Jérôme Godbout 
Cc: Thiago Macieira ; interest@qt-project.org
Subject: Re: [Interest] Parsing data from serialport

But why I need a new buffer when there already one exist? (the QIODevice and my 
own buffer or?)

My idea:

Main() {
Connect(readyread, myreadyread)
}

myreadyread() {

If (!canreadline())
   Return;

Parsedata() ;

}

With this idea I have again the problem with the while loop, but I dont need 
two buffers.
Is it an alternative?

Martin

Jérôme Godbout mailto:godbo...@amotus.ca>> schrieb am Mi., 
3. Apr. 2019, 00:01:
Make sure your reading loop and processing data are separated. Call you read 
device when needed or into a loop that can take some pause to avoid 100% CPU 
usage for nothing.

QByteArray buffer;

void ReadDeviceHaveData()
{

   while(serial_port->bytesAvailable()) // This can be dangerous is data keep 
coming  and might be removed
{
// You can read bytes per bytes or smaller chunk over here for 
better reactivity and less memory consumption
buffer.append(serial_port->readAll());
processData();
serial_port->waitForReadyRead(5);
 }
}

void ProcessData()
{
   int pos = buffer.indexOf(‘\n’);

   while(pos >= 0)
   {
 QByteArray line = buffer.left(pos);
 // Strip trailing \r for windows here
 // Do whatever you need with your line, check data integrity

 // Remove the processed data but leave the unprocessed data alone
 buffer.remove(0, pos + 1); // Remove \n too
 pos = buffer.indexOf(‘\n’);
   }
}

From: Interest 
mailto:interest-boun...@qt-project.org>> On 
Behalf Of Martin Marmsoler
Sent: April 2, 2019 3:58 PM
To: Thiago Macieira 
mailto:thiago.macie...@intel.com>>
Cc: interest@qt-project.org<mailto:interest@qt-project.org>
Subject: Re: [Interest] Parsing data from serialport

 > To be able to roll back, in case your reading from the device didn't result 
 > in
what you wanted or you got an error. See QDataStream.
Ah ok I understand.

So this minimal example
QSerialPort sPort;
sPort.open(QIODevice::ReadOnly);

if(sPort.waitForReadyRead(2000)){

  while (!device.atEnd()) {

if (device.canReadLine()) {

   newData.push_back(device.readLine());

   linesToRead++;

} else {

   return;

}

   }

   ...

}

works fine, if I go trough it step by step (maybe, because enouth data come 
in). But if I'm to fast it does not work.

If I'm using the signal readyRead I will have the same problem, because new 
data come everytime. So I check that in the

readyRead function if a complete line come in, and if no complete line I return 
without doing something otherwise I do

something with the data? Is this the right way?



Martin




Thiago Macieira mailto:thiago.macie...@intel.com>> 
schrieb am Di., 2. Apr. 2019, 18:02:
On Tuesday, 2 April 2019 07:04:03 PDT Martin Marmsoler wrote:
> Thank you Thiago for your response. But what is transactionstart for?

To be able to roll back, in case your reading from the device didn't result in
what you wanted or you got an error. See QDataStream.

--
Thiago Macieira - thiago.macieira (AT) intel.com<http://intel.com>
  Software Architect - Intel System Software Products



___
Interest mailing list
Interest@qt-project.org<mailto:Interest@qt-project.org>
https://lists.qt-project.org/listinfo/interest
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Martin Marmsoler]

But why I need a new buffer when there already one exist? (the QIODevice
and my own buffer or?)

My idea:

Main() {
Connect(readyread, myreadyread)
}

myreadyread() {

If (!canreadline())
   Return;

Parsedata() ;

}

With this idea I have again the problem with the while loop, but I dont
need two buffers.
Is it an alternative?

Martin

Jérôme Godbout  schrieb am Mi., 3. Apr. 2019, 00:01:

> Make sure your reading loop and processing data are separated. Call you
> read device when needed or into a loop that can take some pause to avoid
> 100% CPU usage for nothing.
>
>
>
> QByteArray buffer;
>
>
>
> void ReadDeviceHaveData()
>
> {
>
>while(serial_port->*bytesAvailable*()) // This can be dangerous is data 
> keep coming  and might be removed
>
> {
>
> // You can read bytes per bytes or smaller chunk over here for
> better reactivity and less memory consumption
>
> buffer.append(serial_port->readAll());
>
> processData();
>
> serial_port->*waitForReadyRead*(5);
>
>  }
>
> }
>
>
>
> void ProcessData()
>
> {
>
>int pos = buffer.indexOf(‘\n’);
>
>
>
>while(pos >= 0)
>
>{
>
>  QByteArray line = buffer.left(pos);
>
>  // Strip trailing \r for windows here
>
>  // Do whatever you need with your line, check data integrity
>
>
>
>  // Remove the processed data but leave the unprocessed data alone
>
>  buffer.remove(0, pos + 1); // Remove \n too
>
>  pos = buffer.indexOf(‘\n’);
>
>    }
>
> }
>
>
>
> *From:* Interest  *On Behalf Of *Martin
> Marmsoler
> *Sent:* April 2, 2019 3:58 PM
> *To:* Thiago Macieira 
> *Cc:* interest@qt-project.org
> *Subject:* Re: [Interest] Parsing data from serialport
>
>
>
>  > To be able to roll back, in case your reading from the device didn't
> result in
> what you wanted or you got an error. See QDataStream.
>
> Ah ok I understand.
>
>
>
> So this minimal example
>
> QSerialPort sPort;
>
> sPort.open(QIODevice::ReadOnly);
>
> if(sPort.waitForReadyRead(2000)){
>
>   while (!device.*atEnd*()) {
>
> if (device.*canReadLine*()) {
>
>newData.push_back(device.readLine());
>
>linesToRead++;
>
> } else {
>
>return;
>
> }
>
>}
>
>...
>
> }
>
> works fine, if I go trough it step by step (maybe, because enouth data come 
> in). But if I'm to fast it does not work.
>
> If I'm using the signal readyRead I will have the same problem, because new 
> data come everytime. So I check that in the
>
> readyRead function if a complete line come in, and if no complete line I 
> return without doing something otherwise I do
>
> something with the data? Is this the right way?
>
>
>
> Martin
>
>
>
>
>
>
>
>
>
> Thiago Macieira  schrieb am Di., 2. Apr. 2019,
> 18:02:
>
> On Tuesday, 2 April 2019 07:04:03 PDT Martin Marmsoler wrote:
> > Thank you Thiago for your response. But what is transactionstart for?
>
> To be able to roll back, in case your reading from the device didn't
> result in
> what you wanted or you got an error. See QDataStream.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
>   Software Architect - Intel System Software Products
>
>
>
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
>
>
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Jérôme Godbout]

Make sure your reading loop and processing data are separated. Call you read 
device when needed or into a loop that can take some pause to avoid 100% CPU 
usage for nothing.

QByteArray buffer;

void ReadDeviceHaveData()
{

   while(serial_port->bytesAvailable()) // This can be dangerous is data keep 
coming  and might be removed
{
// You can read bytes per bytes or smaller chunk over here for 
better reactivity and less memory consumption
buffer.append(serial_port->readAll());
processData();
serial_port->waitForReadyRead(5);
 }
}

void ProcessData()
{
   int pos = buffer.indexOf(‘\n’);

   while(pos >= 0)
   {
 QByteArray line = buffer.left(pos);
 // Strip trailing \r for windows here
 // Do whatever you need with your line, check data integrity

 // Remove the processed data but leave the unprocessed data alone
 buffer.remove(0, pos + 1); // Remove \n too
 pos = buffer.indexOf(‘\n’);
   }
}

From: Interest  On Behalf Of Martin Marmsoler
Sent: April 2, 2019 3:58 PM
To: Thiago Macieira 
Cc: interest@qt-project.org
Subject: Re: [Interest] Parsing data from serialport

 > To be able to roll back, in case your reading from the device didn't result 
 > in
what you wanted or you got an error. See QDataStream.
Ah ok I understand.

So this minimal example
QSerialPort sPort;
sPort.open(QIODevice::ReadOnly);

if(sPort.waitForReadyRead(2000)){

  while (!device.atEnd()) {

if (device.canReadLine()) {

   newData.push_back(device.readLine());

   linesToRead++;

} else {

   return;

}

   }

   ...

}

works fine, if I go trough it step by step (maybe, because enouth data come 
in). But if I'm to fast it does not work.

If I'm using the signal readyRead I will have the same problem, because new 
data come everytime. So I check that in the

readyRead function if a complete line come in, and if no complete line I return 
without doing something otherwise I do

something with the data? Is this the right way?



Martin




Thiago Macieira mailto:thiago.macie...@intel.com>> 
schrieb am Di., 2. Apr. 2019, 18:02:
On Tuesday, 2 April 2019 07:04:03 PDT Martin Marmsoler wrote:
> Thank you Thiago for your response. But what is transactionstart for?

To be able to roll back, in case your reading from the device didn't result in
what you wanted or you got an error. See QDataStream.

--
Thiago Macieira - thiago.macieira (AT) intel.com<http://intel.com>
  Software Architect - Intel System Software Products



___
Interest mailing list
Interest@qt-project.org<mailto:Interest@qt-project.org>
https://lists.qt-project.org/listinfo/interest
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Martin Marmsoler]

 > To be able to roll back, in case your reading from the device didn't
result in
what you wanted or you got an error. See QDataStream.
Ah ok I understand.

So this minimal example
QSerialPort sPort;
sPort.open(QIODevice::ReadOnly);

if(sPort.waitForReadyRead(2000)){

  while (!device.atEnd()) {

if (device.canReadLine()) {

newData.push_back(device.readLine());

linesToRead++;

} else {

return;

}

}

   ...

}

works fine, if I go trough it step by step (maybe, because enouth data
come in). But if I'm to fast it does not work.

If I'm using the signal readyRead I will have the same problem,
because new data come everytime. So I check that in the

readyRead function if a complete line come in, and if no complete line
I return without doing something otherwise I do

something with the data? Is this the right way?


Martin





Thiago Macieira  schrieb am Di., 2. Apr. 2019,
18:02:

> On Tuesday, 2 April 2019 07:04:03 PDT Martin Marmsoler wrote:
> > Thank you Thiago for your response. But what is transactionstart for?
>
> To be able to roll back, in case your reading from the device didn't
> result in
> what you wanted or you got an error. See QDataStream.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
>   Software Architect - Intel System Software Products
>
>
>
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
>
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Thiago Macieira]

On Tuesday, 2 April 2019 07:04:03 PDT Martin Marmsoler wrote:
> Thank you Thiago for your response. But what is transactionstart for?

To be able to roll back, in case your reading from the device didn't result in 
what you wanted or you got an error. See QDataStream.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel System Software Products



___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Martin Marmsoler]

Thank you Thiago for your response. But what is transactionstart for?

Best regards,

Martin



Thiago Macieira  schrieb am Mo., 1. Apr. 2019,
09:03:

> On Sunday, 31 March 2019 21:48:30 CEST Martin Marmsoler wrote:
> > But as I understand I have to use startTransaction, because otherwise the
> > buffer will not be used. I'm I correct?
>
> No, you're not.
>
> You're using the buffer if you opened the device in buffered mode, which
> you
> did if you did not pass QIODevice::Unbuffered during on open().
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
>   Software Architect - Intel System Software Products
>
>
>
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
>
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Marian Beermann]

Why not use QIODevice::readyRead signal and check if a complete line can
be read in the slot?

-Marian

Am 31.03.19 um 21:48 schrieb Martin Marmsoler:
> Hello,
> 
> I'm trying to implement a serial connection with QIODevice (SerialPort).
> I would like to parse line by line (each line consists of multiple
> columns of data) and for this I found canReadLine().
> But as I understand I have to use startTransaction, because otherwise
> the buffer will not be used. I'm I correct?
> 
> |bool QIODevice::canReadLine() const { Q_D(const QIODevice); return
> d->buffer.indexOf('\n', d->buffer.size(), d->isSequential() ?
> d->transactionPos : Q_INT64_C(0)) >= 0; } |
> 
> So to read a full line, I have to open a QSerialPort and then start a
> Transaction. Then I have to wait a little bit (till data is in the
> buffer) and then I can use canReadLine. Am I correct? Is there a better
> way instead to wait a little bit? Or is it the wrong way to go?
> 
> 
> Best regards,
> 
> Martin
> 
> 
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
> 

___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Zielinski, Mariusz UTC CCS]

Hi,

If I understand documentation correctly, it suggests to connect 
QIODevice::readyRead() signal to slot that will contain:

if(QIODevice::canReadLine)
{
QByteArray oneLine=QIODevice::ReadLine();
// process data here
}

--
Mariusz Zieliński different results.”

From: Martin Marmsoler [mailto:martin.marmso...@gmail.com]
Sent: 31 marca 2019 23:49
To: interest@qt-project.org
Subject: [Interest] Parsing data from serialport


Hello,

I'm trying to implement a serial connection with QIODevice (SerialPort). I 
would like to parse line by line (each line consists of multiple columns of 
data) and for this I found canReadLine().
But as I understand I have to use startTransaction, because otherwise the 
buffer will not be used. I'm I correct?

bool QIODevice::canReadLine() const

{

Q_D(const QIODevice);

return d->buffer.indexOf('\n', d->buffer.size(),

 d->isSequential() ? d->transactionPos : 
Q_INT64_C(0)) >= 0;

}

So to read a full line, I have to open a QSerialPort and then start a 
Transaction. Then I have to wait a little bit (till data is in the buffer) and 
then I can use canReadLine. Am I correct? Is there a better way instead to wait 
a little bit? Or is it the wrong way to go?



Best regards,

Martin
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Parsing data from serialport

[Thiago Macieira]

On Sunday, 31 March 2019 21:48:30 CEST Martin Marmsoler wrote:
> But as I understand I have to use startTransaction, because otherwise the
> buffer will not be used. I'm I correct?

No, you're not.

You're using the buffer if you opened the device in buffered mode, which you 
did if you did not pass QIODevice::Unbuffered during on open().

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel System Software Products



___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

[Interest] Parsing data from serialport

[Martin Marmsoler]

Hello,

I'm trying to implement a serial connection with QIODevice (SerialPort). I
would like to parse line by line (each line consists of multiple columns of
data) and for this I found canReadLine().
But as I understand I have to use startTransaction, because otherwise the
buffer will not be used. I'm I correct?

bool QIODevice::canReadLine() const
{
Q_D(const QIODevice);
return d->buffer.indexOf('\n', d->buffer.size(),
 d->isSequential() ? d->transactionPos :
Q_INT64_C(0)) >= 0;
}

So to read a full line, I have to open a QSerialPort and then start a
Transaction. Then I have to wait a little bit (till data is in the buffer)
and then I can use canReadLine. Am I correct? Is there a better way instead
to wait a little bit? Or is it the wrong way to go?


Best regards,

Martin
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest