subject:"\[beagleboard\] Re\: Repeatable bug where beaglebone black loses bytes from it's serial connection."

[beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-16 Thread David Howlett

One last thing I forgot to reply to:

 >> Something else to consider: does your data need to be (ASCII) strings?

Not really, they are just easy to read. If I hit further performance issues 
I will switch format.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/b1f9a04a-ca35-4649-9c65-cee9478968e4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-16 Thread David Howlett

 >>And I will finally learn a python ;)

Thanks for using my language. I read and thought about your code but in the 
end I chose to use SSH rather then telnet. SSH support is already present 
in the debian distribution I am using so I could get it working without 
writing any more code on the beagle bone side.

>>  If that's a sustained rate, and not burst, it is likely extreme for 
serial protocols.

After rechecking my numbers, I agree it was a bad idea for me to be using 
serial for this data rate. I have switched to using SSH. I attach a minimal 
example below that I wrote to prove that the communication is faster and 
error free. To run this the paramiko library is required.

import paramiko
import time

# setup logging
paramiko.util.log_to_file('BBB_paramiko.log')
port = 22
# hostname = '192.168.1.34'  # highest measured bytes per second over 
ethernet: 7,868,481
hostname = '192.168.7.2'  # highest measured bytes per second over usb: 
5,941,004
username = 'root'
password = ''

client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(hostname, port, username, password)
stdin, stdout, stderr = client.exec_command('yes 1234567890', bufsize=2**24)
print('streaming started')
startTime = time.perf_counter()
total_data = 0
for i in range(10_000):
line = stdout.readline()
if line == '1234567890\n':
total_data += len(line)
else:
print(line)
# x = stdout.read(10_000) # this is faster then readline
# total_data += len(x)
print(f'bytes per second: 
{int(total_data/(time.perf_counter()-startTime))}')
client.close()

The above program transfers data with no losses at an average rate of 
2,580,544 bytes per second. The speed changes according to how the data is 
processed on the PC side so that is probably the speed limitation now. Now 
that I have proven the communication works fine I will make my pressure 
data stream over SSH. At some point in the future I will probably remove 
the checksums on the pressure data once I can show there are no checksum 
failures in production.

I don't need any more help, thanks everyone.



-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/693dd66c-6764-479d-9c36-d573dad7c7d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-06 Thread arsi




>> Why you want to use RS232, i would choose TCP ,UDP or WebServer

Personal preexisting skill set. I have been using serial connections 
for 4 years on multiple projects with multiple pieces of hardware. I 
wrongly believed that it would be easy to just use serial again.




And I will finally learn a python ;)


Test this: (on linux *telnet ip_of_python_host 5000* on windows with 
putty telnet to port 5000 )


|import  select
import  socket
from  timeimport  sleep
#telnet escape sequences
CLEAR_SCREEN = chr(27) +"[2J"
MOVE_TO_LINE_1 = chr(27) +"[1;1H"
MOVE_TO_LINE_2 = chr(27) +"[2;1H"
MOVE_TO_LINE_3 = chr(27) +"[3;1H"
MOVE_TO_LINE_4 = chr(27) +"[4;1H"
MOVE_TO_LINE_5 = chr(27) +"[5;1H"
MOVE_TO_LINE_6_ST = chr(27) +"[6;1H"
MOVE_TO_LINE_6 = chr(27) +"[6;15H"
CLEAR_LINE = chr(27) +"[2K"
CLEAR_LINE_TO_END = chr(27) +"[K"
SAVE_CURSOR = chr(27) +"7"
RESTORE_CURSOR = chr(27) +"8"
TXT_CMD ="Enter command:"
#
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setblocking(0)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR,1)
server.bind(('0.0.0.0',5000))
server.listen(5)
inputs = [server]
counter1 =0
counter2 =0
counter3 =0
counter4 =0
counter5 =0
addrs = {}
terminalInitOk = []
data =None
while  inputs:
readable, writable, exceptional = select.select(inputs, inputs, inputs)
for  sin  readable:
if  sis  server:
connection, client_address = s.accept()
connection.setblocking(0)
inputs.append(connection)
print('Client connected...'  + str(client_address))
addrs[connection] = client_address
else:
try:
if  sin  inputs:
data = s.recv(1024)
if  data:
print('Client:'  + str(addrs[s]) +' Received command:'  
+ data +'\n')
if  sin  terminalInitOk:
   s.send(MOVE_TO_LINE_6_ST+ CLEAR_LINE+ TXT_CMD + 
MOVE_TO_LINE_6)
except:
inputs.remove(s)
terminalInitOk.remove(s)
c.close();
print('Client'  + str(addrs[s]) +' disconnected..')
if  len(inputs):
counter1 = counter1 +1  #At least one client is attached make the 
calculation
counter2 = counter2 +10
counter3 = counter3 +100
counter4 = counter4 +1000
counter5 = counter5 +1
for  sin  writable:
try:
if  sin  inputs:#send new value to client
if  sin  terminalInitOk:
s.send(SAVE_CURSOR + MOVE_TO_LINE_1 + CLEAR_LINE + 
str(counter1) \
+ MOVE_TO_LINE_2 + CLEAR_LINE + str(counter2) \
+ MOVE_TO_LINE_3 + CLEAR_LINE + str(counter3) \
+ MOVE_TO_LINE_4 + CLEAR_LINE + str(counter4)
+ MOVE_TO_LINE_5 + CLEAR_LINE + str(counter5) + 
RESTORE_CURSOR)
else:
 #init telnet terminal
s.send(CLEAR_SCREEN +MOVE_TO_LINE_1+ str(counter1) \
+ MOVE_TO_LINE_2 + str(counter2) \
+ MOVE_TO_LINE_3 + str(counter3) \
+ MOVE_TO_LINE_4 + str(counter4) \
+ MOVE_TO_LINE_5 + str(counter5) \
+ MOVE_TO_LINE_6_ST + TXT_CMD + MOVE_TO_LINE_6)
terminalInitOk.append(s)
except:
inputs.remove(s)
terminalInitOk.remove(s)
print('Client'  + str(addrs[s]) +' disconnected..')
s.close();
if  len(writable):
sleep(0.1)#Slow down writing
for  sin  exceptional:
if  sin  inputs:
inputs.remove(s)
terminalInitOk.remove(s)
print('Client'  + str(addrs[s]) +' disconnected..')
s.close();
|




arsi

--
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/590E1F46.20305%40chello.sk.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-06 Thread William Hermans

>
> $ python3.6 ~/reusable/BBB_pressure_streamer.py > some_log_file.txt
>
> When I view the log file I can't see any errors
>

So, actually, scripting languages in general can be far slower, that their
native code counterparts. Firstly, because there is an interpreter, and
secondly because of code bloat, through using libraries. Almost as a rule,
larger executables will lead to poorer performing code. Python in
particular is one of if not the slowest languages also when bench marked in
some tests. And I do not mean by a second or two. I mean by a huge margin
in seconds, for specific tests. Just as an example though, in a few tests I
remember reading, Compared to Nodejs, Python was behind by a minute or more.

My point here is not to say Nodejs is better or not though. Nodejs is
pretty terrible it's self. This is meant demonstrate in general that
scripting languages can be terrible when your app is I/O intensive. The
reason why I suggested piping the output of your script to a file was to
see if there was potentially any "normal" I/O bottlenecks involved. All
languages can suffer from this however, because printing to screen( stdout
) is, or can be very I/O intensive. In C, you're maybe going to max out
around a couple hundred "samples" to stdout a second, before your
application will noticeably slow down. With Nodejs, it's actually far less.
I do not know if Python is slower than Nodejs in this case. It's been a
while since I've read the benchmarks, but actually the benchmarks were not
testing this capability anyhow. Maybe they should have been ?

All in all, I'm still not convinced this is the problem in whole. But I do
feel that it could be a contributing factor. It still would not explain why
you seem to be getting your full amount of transmissions. Just that some of
those transmissions are "empty". Especially when those empty transmissions
are not consistent in timing / frequency. I would suggest that you do keep
this in mind, and perhaps at least consider testing output to a file
further. Instead of testing while output is to stdout. At minimum, this
will allow you to rule out, output to stdout as a contributing factor in
your code. With very little wasted time on your behalf.

On Sat, May 6, 2017 at 8:25 AM, David Howlett 
wrote:

> Sorry for double posting last time, this is my first time using google
> groups.
>
> >> I wonder if you run the application that failing for you there is you
> pipe that data to a file,
> >> instead of to stdout, if you would be experiencing the same problem ?
>
> The data loss problem also happens when the yes command is just repeating
> a string over and over to stdout so I don't think the problem is python
> related.
>
> >> Why don't you humor me, and give that a try ?
>
> I tried:
> $ python3.6 ~/reusable/BBB_pressure_streamer.py > some_log_file.txt
>
> When I view the log file I can't see any errors
>
> >> Have you tried reducing the receive FIFO level?
>
> I had not thought of that. Today I tried setting the Tx and Rx buffers to
> 1, 8 and 14 bytes. Data continued to be lost on all settings.
>
> >> Suspect I'd start with
> >>  stty ixon ixoff -ixany
> >> and ensure the other end is set for equivalent.
>
> I tried:
>
> root@beaglebone:~# stty --file=/dev/ttyGS0 ixon ixoff -ixany
> root@beaglebone:~# stty --all
> speed 9600 baud; rows 0; columns 0; line = 0;
> intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = ;
> eol2 = ; swtch = ; start = ^Q; stop = ^S; susp = ^Z;
> rprnt = ^R;
> werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
> -parenb -parodd cs8 hupcl -cstopb cread clocal -crtscts
> -ignbrk brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon
> ixoff
> -iuclc -ixany imaxbel -iutf8
> opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0
> bs0 vt0 ff0
> isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop
> -echoprt
> echoctl echoke
>
> and I also turned on flow control on the python side at the same time but
> I still got data loss.
>
> >> Recommend you specify /which/ serial port
>
> I was originally running stty while connected to the virtual serial port
> but I forgot to mention that detail.
>
> >> I can't duplicate your results as
> >> debian@beaglebone:~$ python3.6 -c "while True: print(sum(i for i in
> range(1)))"
>
> I previously installed python3.6 on all my boards. I should have given an
> example that only depended on base Debian.
>
> >> #flush input buffer (where did .read_all() come from?)
>
> It is from https://github.com/pyserial/pyserial/blob/master/serial/
> serialutil.py
> I was using it to clear the serial buffer of all data. Calling
> .reset_input_buffer() is better
>
> >> OKAY -- THIS PRODUCED a drop-out 2574..2659
>
> Thanks, it is nice to know that someone else can replicate the issue, it
> is not just me doing something stupid.
>
> >> For a stable system, I would recommend considering doing the data
>

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-06 Thread David Howlett

Sorry for double posting last time, this is my first time using google 
groups.

>> I wonder if you run the application that failing for you there is you 
pipe that data to a file,
>> instead of to stdout, if you would be experiencing the same problem ?

The data loss problem also happens when the yes command is just repeating a 
string over and over to stdout so I don't think the problem is python 
related.

>> Why don't you humor me, and give that a try ?

I tried:
$ python3.6 ~/reusable/BBB_pressure_streamer.py > some_log_file.txt

When I view the log file I can't see any errors

>> Have you tried reducing the receive FIFO level?

I had not thought of that. Today I tried setting the Tx and Rx buffers to 
1, 8 and 14 bytes. Data continued to be lost on all settings.

>> Suspect I'd start with
>>  stty ixon ixoff -ixany
>> and ensure the other end is set for equivalent.

I tried:

root@beaglebone:~# stty --file=/dev/ttyGS0 ixon ixoff -ixany
root@beaglebone:~# stty --all
speed 9600 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = ;
eol2 = ; swtch = ; start = ^Q; stop = ^S; susp = ^Z; 
rprnt = ^R;
werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
-parenb -parodd cs8 hupcl -cstopb cread clocal -crtscts
-ignbrk brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon 
ixoff
-iuclc -ixany imaxbel -iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 
vt0 ff0
isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop 
-echoprt
echoctl echoke

and I also turned on flow control on the python side at the same time but I 
still got data loss.

>> Recommend you specify /which/ serial port

I was originally running stty while connected to the virtual serial port 
but I forgot to mention that detail.

>> I can't duplicate your results as
>> debian@beaglebone:~$ python3.6 -c "while True: print(sum(i for i in 
range(1)))"

I previously installed python3.6 on all my boards. I should have given an 
example that only depended on base Debian.

>> #flush input buffer (where did .read_all() come from?)

It is from 
https://github.com/pyserial/pyserial/blob/master/serial/serialutil.py
I was using it to clear the serial buffer of all data. Calling 
.reset_input_buffer() is better

>> OKAY -- THIS PRODUCED a drop-out 2574..2659

Thanks, it is nice to know that someone else can replicate the issue, it is 
not just me doing something stupid.

>> For a stable system, I would recommend considering doing the data 
transfer over a real hardware serial port

Today I purchased two connectors for the hardware serial pins from:
https://www.dfrobot.com/product-147.html
I will see if that helps.

>> You never have been specific about how many bits per second of data that 
you have to move.

Today I measured the average data rate of my application to be 69,218 
bytes/second.

>> This seems to me that you are indeed overflowing your buffer.
>> How or why I'm not sure, but if sending 6 characters all is fine,
>> and 7 characters does not transmit properly then obviously something is 
wrong.

When I send:
x.write(b'yes "123456"\n')
I receive 8 bytes per line not 6 due to the addition of /r/n. I don't think 
this is terribly important though

>> I'm suspecting you're not showing us all your code

My working repository is here:
https://github.com/DavidHowlett/BBB_fault_demo
A permanent link to the relevant commit is here:

https://github.com/DavidHowlett/BBB_fault_demo/blob/389785ec4e50af35fd78673d338063458da6be04/on_PC.py

There is not much to see in the repository but you are welcome to have a 
look.
The only thing worth noting is I use python 3.6 on the PC side too.

>> Here you can see the source code of USB serial gadged driver

Thankyou

>> The easiest solution is to do a simple handshake

That is a good idea for avoiding data loss, the troublesome buffer would 
never fill so there would be no data loss.

>> Why you want to use RS232, i would choose TCP ,UDP or WebServer

Personal preexisting skill set. I have been using serial connections for 4 
years on multiple projects with multiple pieces of hardware. I wrongly 
believed that it would be easy to just use serial again.

In the future I plan to use the hardware serial connection.

>

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/d14843cc-8eb0-4fbb-bec1-3e9052c272ae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-05 Thread arsi


Hi,

I plugged the micro USB port on the BBB into my USB hub. The USB hub is
currently attached to a windows PC but the same fault happens when the USB
cable is plugged into my mac book running OSX.


Which, since there is no FTDI chip on the BBB, is effectively an
emulated serial port... The USB-serial driver responds to I/O operations as
if it were a serial port, but things like baud rate are mostly meaningless
-- the driver moves bytes from the client buffer to the USB buffer, and USB
transfers at (if USB2.0 full speed -- 480kbps; ignoring the breaks caused
by USB packet size AND that the host computer has to poll the USB device to
ask for each packet of data; effective throughput being closer to 250kbps).
I haven't noticed that you are using USB serial gadged driver.What 
Dennis wrote is true.


Here you can see the source code of USB serial gadged driver: 
http://elixir.free-electrons.com/linux/v4.3.5/source/drivers/usb/gadget/function/u_serial.c


Presented baudrate for tty is fixed to 9600:

|
/* 9600-8-N-1 ... matches defaults expected by "usbser.sys" on
* MS-Windows.  Otherwise, most of these flags shouldn't affect
* anything unless we were to actually hook up to a serial line.
*/  
gs_tty_driver->init_termios.c_cflag = B9600 | CS8 | CREAD | HUPCL | CLOCAL;

gs_tty_driver->init_termios.c_ispeed =9600;
gs_tty_driver->init_termios.c_ospeed =9600;
tty_set_operations(gs_tty_driver, _tty_ops);
|


Change of baud rate is not handled anywhere, data are transferred at 
full USB speed.


The easiest solution is to do a simple handshake:

On PC:

|import  serial.tools.list_ports

# connect to the BBB serial port on Windows
x = serial.Serial('COM9')
while  True:
x.write(b"\r\n")
x.flush()
received = x.readline()
print(received)
|


On BBB:

|import  serial.tools.list_ports

x = serial.Serial('/dev/ttyGS0')
while  True:
received = x.readline()
x.write(b"DATA\r\n")
x.flush()|


The BBB app must be run as first... I do not know much about python, 
maybe it's ok ;)


The Second solution with baudrate emulation without RS232/USB converter:
On BBB enable ttyO1 and ttyO2. Connect Tx  pins with Rx..
Install ser2net and configure it on port ttyO2.
Run your app on ttyO1

On windows install http://www.eterlogic.com/Products.VSPE.html and 
connect it to ser2net port.



And you have "simple" baud rate emulator ;)

BTW:

Why you want to use RS232, i would choose TCP ,UDP or WebServer..

if you want, can I send you, ready to use modular web server in java.. 
jetty+vaadin+netbeans lookup



ArSi


--
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/590C3512.2030202%40chello.sk.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-04 Thread William Hermans

On Thu, May 4, 2017 at 5:42 PM, Graham Haddock  wrote:

> So, the COM port on Windows is talking to a virtual serial port in the BBB
> USB widget.
> There is no hardware UART involved, and no hardware clocks anywhere to
> throttle or pace anything.
> Then, you push this for transfer rate, while running it under a non
> realtime OS.
> I guess I am not surprised that you are having problems.
>

This is kind of what I'm thinking, but just sayin' Graham, "gadget" not
"widgit" :P But yeah I do not know this hardware at all, but the conversion
from USB to RS485, there can not be any flow control there. So the gadget
side of the hardware has no flow control, where the RS485 side *may*. I'd
actually speed up the transmission rate, and perhaps put in a usleep()
between each send to fine tune things, and call it quits.

>
> The USB widget is nice for doing simple things, with fast, simple set up
> for beginners.
> I suspect you are trying to run it in an application where it has not been
> well tested.
>
> For a stable system, I would recommend considering doing the data transfer
> over a real hardware serial port, or if that is not fast enough, do the
> transfer using Ethernet. (And I don't mean Ethernet over USB using the
> Widget.)
>
> I find the true hardware serial ports, and the genuine FTDI serial to USB
> converters to be stable well above 100,000 bits per second. Some report
> success in the megabits per second range, I have just not had to personally
> go there, yet.
>
> You never have been specific about how many bits per second of data that
> you have to move.
> I think that is an important place to start.
>
>
So *maybe* if UART4 on the beaglebone was used in full RS485 mode, talking
to the USB to RS485 converter connected to the Windows machine through USB.
*Maybe* that could, or would help solve these issues ? Just a guess but hey
probably worth looking into.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/CALHSORoXByX_RLLcB-_SKTHYtuTfqiy05-EbsjcJaPTeivd29w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-04 Thread William Hermans

Dennis, keep your comments directed at helping the OP, and stop worrying
about what I'm saying.  We're obviously not talking about the same thing,
and your comments to me are not helping the OP solve his problem.

David,

As an example if I run the following program there is no data loss:
>
> import serial.tools.list_ports
>
> # connect to the BBB serial port on Windows
> x = serial.Serial('COM9')
> x.write(b'yes "123456"\n')
>
> while True:
>received = x.readline()
>if received != b'123456\r\n':
> print(received)
>
> But there is data loss if I run this:
>
> import serial.tools.list_ports
>
> # connect to the BBB serial port on Windows
> x = serial.Serial('COM9')
>
> x.write(b'yes "1234567"\n')
>
> while True:
> received = x.readline()
> if received != b'1234567\r\n':
> print(received)
>

This seems to me that you are indeed overflowing your buffer. How or why
I'm not sure, but if sending 6 characters all is fine, and 7 characters
does not transmit properly then obviously something is wrong. I'd look into
googling the driver name this device uses against any known reliability
issues in Linux. One odd thing I did notice about your output you've shown
us, is that everything in your received = x.readline() loop seems to be
working fine.But all the failures are related to this, or a similar line
you've not shown us yet: x.write(b'yes "1234567"\n')

I'm suspecting you're not showing us all your code however. Because the
output you've shown us does not necessarily "jibe" with the code I'm seeing
here.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/CALHSORqGCXZgqDeBwS09jc2gD9BUfC_wJZMYXs%3D37vSs60oWPQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-04 Thread Graham Haddock

So, the COM port on Windows is talking to a virtual serial port in the BBB
USB widget.
There is no hardware UART involved, and no hardware clocks anywhere to
throttle or pace anything.
Then, you push this for transfer rate, while running it under a non
realtime OS.
I guess I am not surprised that you are having problems.

The USB widget is nice for doing simple things, with fast, simple set up
for beginners.
I suspect you are trying to run it in an application where it has not been
well tested.

For a stable system, I would recommend considering doing the data transfer
over a real hardware serial port, or if that is not fast enough, do the
transfer using Ethernet. (And I don't mean Ethernet over USB using the
Widget.)

I find the true hardware serial ports, and the genuine FTDI serial to USB
converters to be stable well above 100,000 bits per second. Some report
success in the megabits per second range, I have just not had to personally
go there, yet.

You never have been specific about how many bits per second of data that
you have to move.
I think that is an important place to start.

--- Graham

==

On Thu, May 4, 2017 at 6:53 PM, Dennis Lee Bieber 
wrote:

> On Thu, 4 May 2017 15:39:43 -0700 (PDT), David Howlett
>  declaimed the
> following:
>
>
> >I plugged the micro USB port on the BBB into my USB hub. The USB hub is
> >currently attached to a windows PC but the same fault happens when the USB
> >cable is plugged into my mac book running OSX.
> >
>
> Which, since there is no FTDI chip on the BBB, is effectively an
> emulated serial port... The USB-serial driver responds to I/O operations as
> if it were a serial port, but things like baud rate are mostly meaningless
> -- the driver moves bytes from the client buffer to the USB buffer, and USB
> transfers at (if USB2.0 full speed -- 480kbps; ignoring the breaks caused
> by USB packet size AND that the host computer has to poll the USB device to
> ask for each packet of data; effective throughput being closer to 250kbps).
>
> And, the Windows (or Mac) side is also an emulated serial, but as
> the
> host side, the USB driver controls the polling of the other end asking for
> data or to send packets.
>
>
> >I have also come to suspect that there are no hardware UARTS involved. I
> >believe that the serial line is being emulated.
> >
>
> Unless you use wires on the UART pins of the BBB, likely true.
> Those
> may be hardware UARTs, but as soon as something is connected that converts
> to USB, the actual data transfer is USB and the other end emulated.
>
> >
> >On the PC side I can set the baudrate to any number. The data rate and the
> >error rate appear to be unaffected. For example:
> >
> >x = serial.Serial('COM10', baudrate=10)
> >x = serial.Serial('COM10', baudrate=9600)
> >x = serial.Serial('COM10', baudrate=100_000_000_000)
> >
>
> As I suspect -- the baud rate is irrelevant when you get into the
> USB
> protocol. At most, it may control a "filter" as to how rapidly the drivers
> relay stuff to the USB protocol. That is -- at slow rates there may only be
> a few bytes of data per USB packet, whereas at really high rates each USB
> packet will contain more data (and hence be more efficient when taking the
> polling overhead into account).
>
> --
> Wulfraed Dennis Lee Bieber AF6VN
> wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/
>
> --
> For more options, visit http://beagleboard.org/discuss
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "BeagleBoard" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/to
> pic/beagleboard/1n0IkcmhEq8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> beagleboard+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/beagleboard/n0fngc1j3g8kfrh49uqngndtsj8b0ohjqi%404ax.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/CANN_KV4Z%3DnnsDDJK-br7yxVLc3XRA10z%2BM1Ys4WU8doRyr8UhQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-04 Thread William Hermans

On Thu, May 4, 2017 at 3:32 PM, Dennis Lee Bieber 
wrote:

> On Thu, 4 May 2017 15:10:07 -0700, William Hermans
>  declaimed the following:
>
>
> >I think it's important to note that the OP is not sending data from the
> >beaglebone, but receiving it. At least looking at the last code he pasted.
> >Usually you wont see "COM9" on a Linux system ;)
> >
> The OP is running a program on Windows that is sending a command
> string
> TO the BBB, and that command at the shell generates output (on stdout,
> connected to the Windows COM port).
>

So, other than repeating exactly what I said, what's your point ? Also
newline eliminates the need for flush.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/CALHSORo3gF4UJNcEmyuF-azGW3-zgKA-fqJ3AGn-3QXuYvY%3DGA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-04 Thread David Howlett

>> I am confused as to what your actual hardware configuration is, with
respect to the "serial communications."
The actual hardware used in the final project with the i2c connectors
soldered on to a prototyping cape looks like this:


I have repeated the exact same fault on this:



>> Are you using a COM port in Windows to talk to a USB to serial cable,
which is talking to a hardware serial port in the BBB?
I plugged the micro USB port on the BBB into my USB hub. The USB hub is
currently attached to a windows PC but the same fault happens when the USB
cable is plugged into my mac book running OSX.

Within a minute of the BBB being plugged in an extra COM port is added in
windows. I connect to the com port with Termite (terminal program), ZOC7
(terminal program) or pyserial (the python serial library).
When I connect to the COM port I can login and get a shell. I ran some
relevant commands in the shell so you can see the results.

root@beaglebone:/dev# w
 20:57:21 up  6:52,  1 user,  load average: 0.02, 0.06, 0.05
USER TTY  FROM LOGIN@   IDLE   JCPU   PCPU WHAT
root ttyGS014Nov15  1.00s  0.97s  0.02s w
root@beaglebone:/dev# who
root ttyGS0   2015-11-14 00:15
root@beaglebone:/dev# tty
/dev/ttyGS0
root@beaglebone:/dev# whoami
root
root@beaglebone:/dev# ls
alarmlog_system  ram11tty16  tty44   ubi_ctrl
ashmem   loop0   ram12tty17  tty45   uinput
audioloop1   ram13tty18  tty46   urandom
autofs   loop2   ram14tty19  tty47   usbmon0
binder   loop3   ram15tty2   tty48   usbmon1
blockloop4   ram2 tty20  tty49   usbmon2
btrfs-controlloop5   ram3 tty21  tty5vcs
bus  loop6   ram4 tty22  tty50   vcs1
char loop7   ram5 tty23  tty51   vcs2
console  loop-controlram6 tty24  tty52   vcs3
core mapper  ram7 tty25  tty53   vcs4
cpu_dma_latency  mem ram8 tty26  tty54   vcs5
disk mixer   ram9 tty27  tty55   vcs6
dri  mmcblk0 random   tty28  tty56   vcs7
dsp  mmcblk0boot0root tty29  tty57   vcsa
fb0  mmcblk0boot1rtc0 tty3   tty58   vcsa1
fd   mmcblk0p1   shm  tty30  tty59   vcsa2
full mmcblk0p2   snd  tty31  tty6vcsa3
fuse mqueue  sndstat  tty32  tty60   vcsa4
hwrngnet stderr   tty33  tty61   vcsa5
i2c-0network_latency stdintty34  tty62   vcsa6
i2c-1network_throughput  stdout   tty35  tty63   vcsa7
initctl  nulltty  tty36  tty7watchdog
inputppp tty0 tty37  tty8watchdog0
kmem psaux   tty1 tty38  tty9xconsole
kmsg ptmxtty10tty39  ttyGS0  zero
log  ptp0tty11tty4   ttyO0
log_events   pts tty12tty40  ttyS0
logibone_mem ram0tty13tty41  ttyS1
log_main ram1tty14tty42  ttyS2
log_radioram10   tty15tty43  ttyS3


>> So, you've actually delved further into using the UARTs on this platform
than I ever have.

I have also come to suspect that there are no hardware UARTS involved. I
believe that the serial line is being emulated.

>> Anyway, my first instinct wants to say baud rate is involved somehow.

On the PC side I can set the baudrate to any number. The data rate and the
error rate appear to be unaffected. For example:

x = serial.Serial('COM10', baudrate=10)
x = serial.Serial('COM10', baudrate=9600)
x = serial.Serial('COM10', baudrate=100_000_000_000)

I have also set the baud rate with Termite 3.1 to demonstrate that it is
not a bug in the python serial library.

On the BBB end the speed is always reported as 9600. Some commands to
change the speed succeed and others fail but the reported speed is
unchanged.

root@beaglebone:/dev# stty ispeed 7200
root@beaglebone:/dev# stty
speed 9600 baud; line = 0;
root@beaglebone:/dev# stty ispeed 14400
root@beaglebone:/dev# stty
speed 9600 baud; line = 0;
root@beaglebone:/dev# stty ispeed 2400
stty: standard input: unable to perform all requested operations
root@beaglebone:/dev# stty
speed 9600 baud; line = 0;

I will finish replying to everyone tomorrow. It is getting late where I am.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-04 Thread William Hermans

On Thu, May 4, 2017 at 3:10 PM, William Hermans  wrote:

>
>
> On Thu, May 4, 2017 at 3:02 PM, arsi  wrote:
>
>> Hi,
>>
>> call *flush* after write command!!!
>>
>> *flush**(**)*
>>
>> *Flush of file like objects. In this case, wait until all data is
>> written.*
>> Arsi
>>
>
> I think it's important to note that the OP is not sending data from the
> beaglebone, but receiving it. At least looking at the last code he pasted.
> Usually you wont see "COM9" on a Linux system ;)
>
> At any rate if receiving, you don't need to use flush at all. If it's
> anything like fflush() in C.
>

Additionally, the output had newline "\n" every time output was sent.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/CALHSORorHBZ8m3uU-isbgbuXWKgiMSC4JF_U5VYmxwDMG9WUCQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-04 Thread William Hermans

On Thu, May 4, 2017 at 3:02 PM, arsi  wrote:

> Hi,
>
> call *flush* after write command!!!
>
> *flush**(**)*
>
> *Flush of file like objects. In this case, wait until all data is written.*
> Arsi
>

I think it's important to note that the OP is not sending data from the
beaglebone, but receiving it. At least looking at the last code he pasted.
Usually you wont see "COM9" on a Linux system ;)

At any rate if receiving, you don't need to use flush at all. If it's
anything like fflush() in C.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/CALHSORpN4jWLhAs-GSZ5K1zTyNQ-d5FWp06cA1yCFfF1RVKW-w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-04 Thread arsi


Hi,

call *flush* after write command!!!

*|flush|**(**)*

   *Flush of file like objects. In this case, wait until all data is
   written.*

Arsi

--
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/590BA513.2050307%40chello.sk.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-04 Thread William Hermans

On Thu, May 4, 2017 at 9:32 AM, David Howlett 
wrote:

> >> First, you're going to need to learn how to use systemd, and
> specifically, you're going to need to learn how to create, and use a
> startup service.
>
> I happen to have used systemd startup services in a previous project so I
> can see how it would be useful for setting settings on startup.
>

That good, something less to get in your way to fixing your problem.

> Conclusions
> ===
>
> - The virtual serial port provided is a poor imitation of a real serial
> port. Every command to change any property of either end of the connection
> either fails or is ignored.
>
> - There is undocumented flow control somewhere behind the scenes outside
> my control. It is probably in a driver.
>
> - The flow control mechanism loses data if the data is not a multiple of 4
> bytes long.
>

So, you've actually delved further into using the UARTs on this platform
than I ever have. Even with other( bare metal ) platforms, I only really
use UARTs for a printf() style serial debug output. With this platform, we
do not need it, unless you're troublshooting the boot process.

Anyway, my first instinct wants to say baud rate is involved somehow. It's
very unlikely a baud rate difference between systems, because usually when
this occurs, your transmission will be garbled. At least a few characters
here and there will be different between send, and receive. So this leads
me to speculate that you're somehow exceeding your maximum baud rate.

Another thing I'm noticing from what your code is outputting. is that your
"data loss" is not consistent. Which immediately makes me want to jump to
the conclusion that you code is somehow being preempted. Almost as if
you're data is on the stack, something preempts your code, and by the time
the part of your code is given control back, the data on the stack is no
longer there. This sort of situation is what I refer to as "stepping all
over the stack". e.g. some of those routines you're using could be stack
unfriendly. Granted, I know very little about Python, or the libraries
Python uses. But I've personally experienced this effect first hand, when
using a third party web server API, when dealing with a lot of data fast.
CANBUS at 1Mbit, while decoding PGNs in real time, and attempting to send
this data in real time using this third part web server API.

I wonder if you run the application that failing for you there is you pipe
that data to a file, instead of to stdout, if you would be experiencing the
same problem ? Why don't you humor me, and give that a try ? e.g. run your
application as such:

$ myapp > some_log_file.txt

Run it for a few hundred iterations and see if you get any blank
transmission, and if you do, if they are fewer. If this stops your blank
tramissions, or lessens them a fair bit. Then your problem could either be
python not being able to keep up( I'm kind of doubting that ), or Windows
for whatever reason is preempting your code mid transmission. Or even
something else not yet considered.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/CALHSORrMbrWmS22aaJbObbhbp-PXx3h04bfEvNkS_aj1zxwmaQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-04 Thread Graham

David:

I am confused as to what your actual hardware configuration is, with 
respect to the "serial communications."

Are you using a COM port in Windows to talk to a USB to serial cable, which 
is talking to a hardware serial port in the BBB?

Or, are you doing something else?

If so, please describe.

I have never seen these kinds of problems when dealing with hardware serial 
ports.

--- Graham

==

On Thursday, May 4, 2017 at 11:32:04 AM UTC-5, David Howlett wrote:
>
> >> First, you're going to need to learn how to use systemd, and 
> specifically, you're going to need to learn how to create, and use a 
> startup service.
>
> I happen to have used systemd startup services in a previous project so I 
> can see how it would be useful for setting settings on startup.
>
> >> Second, you need a tool that is capable of working with serial devices 
> at a "low level".  Such as: 
> https://manpages.debian.org/jessie/coreutils/stty.1.en.html and 
> specifically this setting:
>
> This is the tool I was looking for and could not find, thank you. The 
> default settings for the serial connection I am using are:
>
> root@beaglebone:~/reusable# stty --all
> speed 9600 baud; rows 0; columns 0; line = 0;
> intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = ;
> eol2 = ; swtch = ; start = ^Q; stop = ^S; susp = ^Z; rprnt = 
> ^R;
> werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
> -parenb -parodd cs8 hupcl -cstopb cread clocal -crtscts
> -ignbrk brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon 
> -ixoff
> -iuclc -ixany imaxbel -iutf8
> opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 
> vt0 ff0
> isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt
> echoctl echoke
>
> I can now use the stty command to set ixon on or off.
>
> root@beaglebone:~/reusable# stty -ixon
> root@beaglebone:~/reusable# stty --all
> speed 9600 baud; rows 0; columns 0; line = 0;
> intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = ;
> eol2 = ; swtch = ; start = ^Q; stop = ^S; susp = ^Z; rprnt = 
> ^R;
> werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
> -parenb -parodd cs8 hupcl -cstopb cread clocal -crtscts
> -ignbrk brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl -ixon 
> -ixoff
> -iuclc -ixany imaxbel -iutf8
> opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 
> vt0 ff0
> isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt
> echoctl echoke
>
> Unfortunately the data loss is the same regardless of whether ixon is 
> enabled or disabled. Changing ixoff does not affect the data loss either.
> I tried setting crtscts and cdtrdsr but this is not permitted.
>
> root@beaglebone:~/reusable# stty crtscts
> stty: standard input: unable to perform all requested operations
> root@beaglebone:~/reusable# stty cdtrdsr
> stty: invalid argument `cdtrdsr'
>
> There is flow control regardless of what I do
> =
>
> There does appear to be flow control of some sort. I realised I can get 
> debug information with the CPU LED. I can make the CPU light on the BBB 
> light up for 2.4 seconds by running the following program on the PC. The 
> fact that the CPU light turns off again indicates that there is some kind 
> of flow control that pauses the computation when the buffer fills up.
>
> import time
> import serial.tools.list_ports
>
> # connect to the BBB serial port on Windows
> x = serial.Serial('COM9')
> x.read_all()
> x.write(b'python3.6 -c "while True: print(sum(i for i in range(1000)))"\n')
> # pause forever
> time.sleep(1_000_000)
>
> If I slow down the CPU wasting command like this:
>
> import time
> import serial.tools.list_ports
>
> # connect to the BBB serial port on Windows
> x = serial.Serial('COM9')
> x.read_all()
> x.write(b'python3.6 -c "while True: print(sum(i for i in 
> range(10_000)))"\n')
> # pause forever
> time.sleep(1_000_000)
>
> Then the CPU LED stays lit for about 10x as long. This demonstrates that 
> streaming on the beagle bone is stopped by a buffer filling up rather than 
> a timer.
>
> Data loss only happens when the length of the data being written is not a 
> multiple of 4 bytes
>
> =
>
> As an example if I run the following program there is no data loss:
>
> import serial.tools.list_ports
>
> # connect to the BBB serial port on Windows
> x = serial.Serial('COM9')
> x.write(b'yes "123456"\n')
>
> while True:
>received = x.readline()
>if received != b'123456\r\n':
> print(received)
>
> But there is data loss if I run this:
>
> import serial.tools.list_ports
>
> # connect to the BBB serial port on Windows
> x = serial.Serial('COM9')
>
> x.write(b'yes "1234567"\n')
>
> while True:
> received = x.readline()
> if received != b'1234567\r\n':
> print(received)
>
> It prints:
>
>

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-04 Thread David Howlett

>> First, you're going to need to learn how to use systemd, and 
specifically, you're going to need to learn how to create, and use a 
startup service.

I happen to have used systemd startup services in a previous project so I 
can see how it would be useful for setting settings on startup.

>> Second, you need a tool that is capable of working with serial devices 
at a "low level".  Such as: 
https://manpages.debian.org/jessie/coreutils/stty.1.en.html and 
specifically this setting:

This is the tool I was looking for and could not find, thank you. The 
default settings for the serial connection I am using are:

root@beaglebone:~/reusable# stty --all
speed 9600 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = ;
eol2 = ; swtch = ; start = ^Q; stop = ^S; susp = ^Z; rprnt = 
^R;
werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
-parenb -parodd cs8 hupcl -cstopb cread clocal -crtscts
-ignbrk brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon 
-ixoff
-iuclc -ixany imaxbel -iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 
ff0
isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt
echoctl echoke

I can now use the stty command to set ixon on or off.

root@beaglebone:~/reusable# stty -ixon
root@beaglebone:~/reusable# stty --all
speed 9600 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = ;
eol2 = ; swtch = ; start = ^Q; stop = ^S; susp = ^Z; rprnt = 
^R;
werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
-parenb -parodd cs8 hupcl -cstopb cread clocal -crtscts
-ignbrk brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl -ixon 
-ixoff
-iuclc -ixany imaxbel -iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 
ff0
isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt
echoctl echoke

Unfortunately the data loss is the same regardless of whether ixon is 
enabled or disabled. Changing ixoff does not affect the data loss either.
I tried setting crtscts and cdtrdsr but this is not permitted.

root@beaglebone:~/reusable# stty crtscts
stty: standard input: unable to perform all requested operations
root@beaglebone:~/reusable# stty cdtrdsr
stty: invalid argument `cdtrdsr'

There is flow control regardless of what I do
=

There does appear to be flow control of some sort. I realised I can get 
debug information with the CPU LED. I can make the CPU light on the BBB 
light up for 2.4 seconds by running the following program on the PC. The 
fact that the CPU light turns off again indicates that there is some kind 
of flow control that pauses the computation when the buffer fills up.

import time
import serial.tools.list_ports

# connect to the BBB serial port on Windows
x = serial.Serial('COM9')
x.read_all()
x.write(b'python3.6 -c "while True: print(sum(i for i in range(1000)))"\n')
# pause forever
time.sleep(1_000_000)

If I slow down the CPU wasting command like this:

import time
import serial.tools.list_ports

# connect to the BBB serial port on Windows
x = serial.Serial('COM9')
x.read_all()
x.write(b'python3.6 -c "while True: print(sum(i for i in 
range(10_000)))"\n')
# pause forever
time.sleep(1_000_000)

Then the CPU LED stays lit for about 10x as long. This demonstrates that 
streaming on the beagle bone is stopped by a buffer filling up rather than 
a timer.

Data loss only happens when the length of the data being written is not a 
multiple of 4 bytes
=

As an example if I run the following program there is no data loss:

import serial.tools.list_ports

# connect to the BBB serial port on Windows
x = serial.Serial('COM9')
x.write(b'yes "123456"\n')

while True:
   received = x.readline()
   if received != b'123456\r\n':
print(received)

But there is data loss if I run this:

import serial.tools.list_ports

# connect to the BBB serial port on Windows
x = serial.Serial('COM9')

x.write(b'yes "1234567"\n')

while True:
received = x.readline()
if received != b'1234567\r\n':
print(received)

It prints:

b'yes "1234567"\r\n'
b'11234567\r\n'
b'11234567\r\n'
b'12234567\r\n'
b'12334567\r\n'
b'12345677\r\n'
b'\n'
b'\n'
b'\n'
b'\n'
b'\n'
b'\n'
b'12345567\r\n'
b'\n'
b'12234567\r\n'
b'\n'
b'12334567\r\n'
b'12344567\r\n'
b'12344567\r\n'
b'\n'
b'\n'
b'12344567\r\n'
b'12234567\r\n'
b'12345567\r\n'
b'12345667\r\n'
b'\n'
b'\n'
b'12234567\r\n'
b'12334567\r\n'
b'12234567\r\n'
b'12345677\r\n'
b'12334567\r\n'
b'\n'
b'12345677\r\n'
b'1234567\r\r\n'
b'\n'
b'11234567\r\n'
b'12234567\r\n'
b'123234567\r\n'
b'12234567\r\n'
b'12334567\r\n'
b'\n'

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-02 Thread William Hermans

On Tue, May 2, 2017 at 6:26 AM, David Howlett 
wrote:

> I couldn't easily find documentation on how to enable flow control on the
> beaglebone side so I chose to add a CRC32 checksum to each line. The data
> now looks like:
>
> This is not really so much a beaglebone thing, and more of a Linux thing.
The problem is documentation. As in it exists, but one part of the equation
does not necessarily have anything to do with the other. As such, it's hard
to find information if you know nothing about what you're looking for.
Trust me, I've been using Debian nearly since it's first release, and I
personally do not know how this is to be done from hands on experience.
However, I can shed some light on the subject whcih should allow you to
experiment on your own, and have something working fairly quickly if you
have a decent amount of experience with Linux in general.

First, you're going to need to learn how to use systemd, and specifically,
you're going ot need to learn how to create, and use a startup service.

Second, you need a tool that is capable of working with serial devices at a
"low level".  Such as:
https://manpages.debian.org/jessie/coreutils/stty.1.en.html and
specifically this setting:

[-]ixonenable XON/XOFF flow control

Then it's just a matter of using this tool properly form within the
service, or perhaps calling a script from the service, that sets the serial
port exactly how you want. You'll have to experiment to get things exactly
how you want / need.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/CALHSORq%2BqyhmpZ4-HyG2ORjnzujGrhhKETbv2_CRWzz3Hck%2B5w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-05-02 Thread David Howlett

I couldn't easily find documentation on how to enable flow control on the 
beaglebone side so I chose to add a CRC32 checksum to each line. The data 
now looks like:

102.04541 125 1186 3665238047
102.04588 125 1273 3232659341
102.04635 125 1273 3345150538
102.04708 125 1245 4074941927
102.04756 125 1245 943190513
102.04803 125 1303 526161833
102.04850 125 1274 3970916767
102.04898 125 1273 472341404
102.04970 125 1100 2928958186
102.05018 125 1244 2418030567
102.05066 125 1187 2253189159
102.05113 125 1157 1456340796
102.05161 125 1273 258216438
102.05234 125 1012 3972342546
102.05282 125 1214 2310087250
102.05329 125 1274 2579294272
102.05377 125 1215 1325398437

This means that corrupted lines are easy to detect. I then deal with data 
being missing later on in the analysis pipeline.

>> Here is my reading algorithm from c lib on linux it detects a pause in 
the transfer
Unfortunately the code I am writing on the PC side is called too 
infrequently for this to help. The 12k buffer can occasionally fill up and 
lose data in the gap between two calls. I could make a seperate thread 
regularly poll the windows API to see if there was new data but a checksum 
was simpler to implement as I am not familiar with threading.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/6ae68cf8-3eae-41c2-be5f-2e78a77652f5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-04-28 Thread arsi



Hi,

I use a similar algorithm on 76800 baud for RS485 Honeywell C-bus token 
ring protocol and it works without problems from java on BBB and PC..
And I noticed thatyou nowhere  call a *flush* when sending data, that 
can be the problem..



Test it with the state machine, It must work..


Here is my reading algorithm from c lib on linux it detects a pause in 
the transfer:



timeout - read timeout in seconds
usReadPause - after no incomming data on port for x micro seconds, 
return from read sequence.. (return one data packet)


jint user_com_apli_jni_Cbridge_serialReceiveInterSpacingReadPause(JNIEnv * env, 
jobject object,jint handle,jobject buffer,jint usReadPause,jint timeout){
size_t capacity = env->GetDirectBufferCapacity(buffer);
fd_set readfs;
timeval tv;
tv.tv_sec = timeout / 1000;
tv.tv_usec = 0;
void  *buff = env->GetDirectBufferAddress(buffer);
int  size = 0;
int  rd = 0;
FD_ZERO();
FD_SET(handle, );
select(handle + 1, ,NULL,NULL,  );
if  (FD_ISSET(handle, )) {
size = read(handle, buff, capacity);
rd = size;
do  {
usleep(usReadPause);
buff = ((char*) buff) + rd;
rd = read(handle, buff, capacity - size);
if  (rd > 0) {
size += rd;
}
}while  (rd > 0 && capacity - size > 0);
env->SetIntField(buffer, limitFieldId, size);
return  size;
}
return  0;
}


Arsi


*From:* David Howlett
*Sent:* Thursday, April 27, 2017 6:54PM
*To:* Beagleboard
*Subject:* [beagleboard] Re: Repeatable bug where beaglebone black loses 
bytes

from it's serial connection.



Reply to arsi
=
If I understand you correctly, you are recommending adding a 4 byte 
sequence "0x10 0x11 0x12 0x13" on a regular basis in the streaming 
data to allow the receiving PC to sync up with the stream. Is that 
what you mean? For my current application the data looks like:


10392.614241251299
10392.614691251242
10392.615141251357
10392.615581251385
10392.616281251357
10392.616741251242
10392.617181251328
10392.617651251473
10392.618101251212
10392.618801251299

It is delimited by "/r/n" which allows easy syncing up. The trouble is 
that bytes can go missing to make the above data look like:


10392.614241251299
10392.614691251242
10392.615141251357
10392.615581251385
1039216281251357
10392.616741251242
10392.617181251328
10392.617651251473
10392.618101251212
10392.618801251299

I could make my format checking stricter and I could add checksums to 
every line. This would cut down the error rate but I would prefer to 
fix the cause of the data loss if possible.









--
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/5902E548.1030302%40chello.sk.
For more options, visit https://groups.google.com/d/optout.

[beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-04-27 Thread David Howlett

Thanks for all the replies. 

Reply to Przemek Klowoski
=

I agree it is almost certainly a serial buffer overflow somewhere. By 
filling and emptying the buffers between my code on the BBB and my code on 
the PC I can detect a 12KB buffer. I believe that this 12KB buffer is in 
the BBB driver on the PC. I tried to set the driver's buffer to be bigger 
but the driver appears to ignore all calls to the WINAPI SetupComm system 
call.

I have tried OSX but I still get data loss.
I have tried to use a second BeagleBone as PC replacement but the first 
beagle bone does not show up on the list of available serial ports.

I have tried:

x = serial.Serial('COM10', xonxoff=True)
x = serial.Serial('COM10', rtscts=True)
x = serial.Serial('COM10', dsrdtr=True)

None of these cause any change in behaviour.
I have also tried sending stop commands directly:

x.write(b'\x13')  # byte 0x13 is XOFF

This does not cause any change in behaviour either.

Reply to arsi
=
If I understand you correctly, you are recommending adding a 4 byte 
sequence "0x10 0x11 0x12 0x13" on a regular basis in the streaming data to 
allow the receiving PC to sync up with the stream. Is that what you mean? 
For my current application the data looks like:

10392.61424 125 1299
10392.61469 125 1242
10392.61514 125 1357
10392.61558 125 1385
10392.61628 125 1357
10392.61674 125 1242
10392.61718 125 1328
10392.61765 125 1473
10392.61810 125 1212
10392.61880 125 1299

It is delimited by "/r/n" which allows easy syncing up. The trouble is that 
bytes can go missing to make the above data look like:

10392.61424 125 1299
10392.61469 125 1242
10392.61514 125 1357
10392.61558 125 1385
103921628 125 1357
10392.61674 125 1242
10392.61718 125 1328
10392.61765 125 1473
10392.61810 125 1212
10392.61880 125 1299

I could make my format checking stricter and I could add checksums to every 
line. This would cut down the error rate but I would prefer to fix the 
cause of the data loss if possible.

Reply to Graham
===

I have tried baud rates on the PC from 10Hz to 100GHz including common baud 
rates like 9600. The serial connection works at all baud rates tested. 
There appears to be no change in the data rate or the error rate. To rule 
out a bug in the pyserial library I have also changed the baud rate with a 
GUI terminal called "Termite 3.1". This also does not change the data rate. 
I believe that the serial port is a virtual device and all commands to 
change the baud rate are ignored by the driver. 


x = serial.Serial('COM10', baudrate=10)
x = serial.Serial('COM10', baudrate=9600)
x = serial.Serial('COM10', baudrate=100_000_000_000)

My current ideas
=

I could add checksums. I could switch from a virtual serial port to using a 
pair of physical USB-RS232 adapters. This would go: BBB USB->hardware 
RS232->PC USB. I only mention this because I have plenty of experience with 
physical RS232 and I have USB-RS232 adapters which respond to the win32 
SetupComm call to change the buffer size. I could also switch to something 
else entirely like SSH over the ethernet port.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/2c35a45c-e1dc-476a-8b18-5a6312bae8fc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-04-21 Thread Przemek Klosowski

It looks like the serial output buffer overflow. I think you're using the
USB serial port, so it's probably not lost in the USB link, but on the
Windows side the USB serial driver is presumably overwriting the output
buffer.  Would you have a Linux system somewhere to cross-check it by using
a different OS/serial driver combination?
Can you change the settings in the Windows serial port, asking for hardware
flow control?

On Fri, Apr 21, 2017 at 12:26 PM, David Howlett 
wrote:

> I should mention I am using:
> - Windows 10
> - Drivers from about 2 weeks ago
> - A beaglebone image from about 2 weeks ago
>
>
> On Friday, April 21, 2017 at 5:18:37 PM UTC+1, David Howlett wrote:
>>
>> I have spent about two days tracking down an issue in one of my machines.
>> I am streaming pressure readings from an i2c pressure sensor to a PC at
>> high speed. I have reduced the problem to a small test case. To replicate
>> the fault I connect the beaglebone black to a PC, connect to it with serial
>> over the micro USB port, login and launch a command that streams data fast.
>> On the PC run a program that collects the streamed data more slowly than it
>> is being created.
>>
>> import serial
>>
>> x = serial.Serial('COM15')
>> to_send = b'\x03yes HelloWorld\n'
>> x.write(to_send)
>> while True:
>> received = x.readline()
>> if received != b'HelloWorld\r\n':
>> print(received)
>>
>>
>> The python program above prints lines like:
>>
>> b'ld\r\n'
>> b'HelloWoroWorld\r\n'
>> b'HelloWorldorld\r\n'
>> b'HelloWorldorld\r\n'
>> b'ld\r\n'
>> b'ld\r\n'
>> b'ld\r\n'
>> b'ld\r\n'
>> b'ld\r\n'
>> b'ld\r\n'
>> b'HelloWoroWorld\r\n'
>> b'ld\r\n'
>> b'HelloWlloWorld\r\n'
>> b'ld\r\n'
>> b'HelloWorldorld\r\n'
>>
>> At about 4 lines per second. If I set the program on the beagle bone to
>> stream slowly then this issue goes away. It appears to be an issue with a
>> buffer filling up somewhere between the output of the yes command and
>> reading bytes from the windows API. The correct behaviour is that the yes
>> command should be stopped when the buffer is full to prevent the loss of
>> data, then allowed to continue when there is space in the buffer again. I
>> observe that the streaming command (in this case yes) is stopped but there
>> is still data loss happening somewhere.
>>
>> I wrote this in python because it is the language I am most comfortable
>> in but if you needed me to I could rewrite the example in some other
>> language. To run the program shown above you will need pyserial and you
>> will need to change 'COM15' to be whatever the port is called on your
>> computer.
>>
>>
>> --
> For more options, visit http://beagleboard.org/discuss
> ---
> You received this message because you are subscribed to the Google Groups
> "BeagleBoard" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to beagleboard+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/beagleboard/f4c86058-1fee-41fb-bc6f-d41f8eccb3b6%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/CAC%3D1GgHaO3GQCnKpVa3ATJ9p1VfEMkYOGvtBdEeMGb2nacR13w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

2017-04-21 Thread David Howlett

I should mention I am using:
- Windows 10
- Drivers from about 2 weeks ago
- A beaglebone image from about 2 weeks ago

On Friday, April 21, 2017 at 5:18:37 PM UTC+1, David Howlett wrote:
>
> I have spent about two days tracking down an issue in one of my machines. 
> I am streaming pressure readings from an i2c pressure sensor to a PC at 
> high speed. I have reduced the problem to a small test case. To replicate 
> the fault I connect the beaglebone black to a PC, connect to it with serial 
> over the micro USB port, login and launch a command that streams data fast. 
> On the PC run a program that collects the streamed data more slowly than it 
> is being created. 
>
> import serial
>
> x = serial.Serial('COM15')
> to_send = b'\x03yes HelloWorld\n'
> x.write(to_send)
> while True:
> received = x.readline()
> if received != b'HelloWorld\r\n':
> print(received) 
>
>
> The python program above prints lines like:
>
> b'ld\r\n'
> b'HelloWoroWorld\r\n'
> b'HelloWorldorld\r\n'
> b'HelloWorldorld\r\n'
> b'ld\r\n'
> b'ld\r\n'
> b'ld\r\n'
> b'ld\r\n'
> b'ld\r\n'
> b'ld\r\n'
> b'HelloWoroWorld\r\n'
> b'ld\r\n'
> b'HelloWlloWorld\r\n'
> b'ld\r\n'
> b'HelloWorldorld\r\n'
>
> At about 4 lines per second. If I set the program on the beagle bone to 
> stream slowly then this issue goes away. It appears to be an issue with a 
> buffer filling up somewhere between the output of the yes command and 
> reading bytes from the windows API. The correct behaviour is that the yes 
> command should be stopped when the buffer is full to prevent the loss of 
> data, then allowed to continue when there is space in the buffer again. I 
> observe that the streaming command (in this case yes) is stopped but there 
> is still data loss happening somewhere. 
>
> I wrote this in python because it is the language I am most comfortable in 
> but if you needed me to I could rewrite the example in some other language. 
> To run the program shown above you will need pyserial and you will need to 
> change 'COM15' to be whatever the port is called on your computer. 
>
>
>

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/f4c86058-1fee-41fb-bc6f-d41f8eccb3b6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

[beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

[beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

Re: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

[beagleboard] Re: Repeatable bug where beaglebone black loses bytes from it's serial connection.

23 matches

Site Navigation

Mail list logo

Footer information