Okay, problem solved. Phil was right: it was my program. Serial ports
on Linux are treated as terminal devices, since they were traditionally
used to connect terminals to Unix mainframes. Thus, they have some
settings which affects the handling of input and output. Turns out my
code wasn't being scrupulous enough with these settings, so the rfcomm
driver was doing funny stuff when it received certain characters.
Below is the reply I was writing when I discovered this; I'll retain it
for your entertainment and education.
Thanks to everyone for your responses.
---
Yeah, I'm aware that it doesn't provide framing. I'm capable of dealing
with that; I've written many different communications protocols on top
of similar stream transports.
The problem here is that bytes are being dropped somewhere. Observe
this output from my program:
Read 20 bytes: 00 C1 3B 91 07 58 08 DA 0A 53 07 F9 06 24 07 7A FE 8B FE 43
Read 1 bytes: FE
Got packet: 00 C1 3B 91 07 58 08 DA 0A 53 07 F9 06 24 07 7A FE 8B FE 43 FE
Read 21 bytes: 00 41 3E 93 07 5F 08 DC 0A 55 07 FB 06 26 07 7A FE 8B FE
43 FE
Got packet: 00 41 3E 93 07 5F 08 DC 0A 55 07 FB 06 26 07 7A FE 8B FE 43 FE
Read 13 bytes: 0B 55 07 FB 06 27 07 78 FE 7D FE 40 FE
Read 8 bytes: 00 41 43 97 07 5F 08 EB
Discarded 13 bytes of garbage
Read 13 bytes: 0A 55 07 FA 06 26 07 78 FE 7D FE 40 FE
Got packet: 00 41 43 97 07 5F 08 EB 0A 55 07 FA 06 26 07 78 FE 7D FE 40 FE
Read 21 bytes: 00 C1 45 9B 07 55 08 FC 0A 54 07 FA 06 25 07 78 FE 7D FE
40 FE
Got packet: 00 C1 45 9B 07 55 08 FC 0A 54 07 FA 06 25 07 78 FE 7D FE 40 FE
The first two packets are whole and appear to make sense by virtue of
the sensor values being approximately the same (the Shimmer was sitting
on my table). However, the next packet appears to have lost 8 bytes
from the beginning; after reading a full 21 bytes, the program detects
this, throws the 13 bytes of the partial packet away, and reads a new 13
bytes to fill up the packet.
I wrote another test program, which repeatedly sends an inquiry command
to the Shimmer and reads as much data as it can in one second. The
output looks like this:
Wrote inquiry command
Read 7 bytes
Read 5 bytes
Received data: FF 02 14 00 00 09 01 04 05 06 07 08
Wrote inquiry command
Read 1 bytes
Read 5 bytes
Received data: FF 04 05 06 07 08
Wrote inquiry command
Read 8 bytes
Read 5 bytes
Received data: FF 02 14 00 00 09 01 00 04 05 06 07 08
Wrote inquiry command
Read 4 bytes
Read 5 bytes
Received data: FF 02 14 00 04 05 06 07 08
A full, correct response would consist of the 16 bytes FF 02 14 00 00 09
01 00 01 02 03 04 05 06 07 08 (7 bytes, plus the value of the 6th byte).
Out of 100 tries, the closest was 15 bytes, with the 03 missing. In
fact, if you look at the responses above, you'll notice that the last
five bytes are always intact, with varying amounts of data missing
before them.
Epilogue (i.e. how I realized the problem): In fact, when I look through
the received data, the byte 03 never appears in it. And when I tilt the
device to its side so that the MSB of one of the accelerometer axes goes
from 04 to 03, the number of bad packets skyrockets. It so happens that
this byte corresponds to Ctrl-C; could there be a connection here...
Mikko
On 24.07.2012 20:51, steve ayer wrote:
mikko,
the framing has to happen on the receiving end. bluetooth isn't
unreliable, it was designed to replace wires; it's just that these
devices use the serial port protocol, which is stateless beyond the
confines of a single byte (unlike a network protocol, my point earlier).
to see what's going on, write a really simple receiving script (i use
pybluez) that just shows what arrived in each bluetooth transmission.
you'll see random lengths like 12, 6, 20, 8 bytes. so, you have to piece
them together.
the reason that shimmerconnect works is that it re-frames the sent bytes
by knowing how large the original packets are and re-assembling them.
i hope that this helps,
steve
On 07/24/2012 01:38 PM, Mikko Rasa wrote:
At the moment I'm using the stock boilerplate firmware, which provides
very little in the way of packet framing. It has a byte indicating the
packet type, but that's it. There's no guarantee that the rest of the
packet doesn't contain bytes with the same value, which makes it really
hard to resync after a lost byte.
My program expects to receive packets of 21 bytes, the first of which is
0x00 (indicating a data packet). It reads data from rfcomm until has 21
bytes, then checks whether the first byte is zero. If it isn't, it
assumes that there was garbage at the beginning, and scans through the
received data until it finds a zero. Then it move the rest to the
beginning of the buffer and fills it up again.
If I assume that any zero indicates the beginning of a packet, I receive
almost no corrupt packets, but I also drop valid packets where some of
the data bytes are zeroes.
Are you saying that bluetooth is inherently unreliable, and the
boilerplate firmware with its lack of proper framing is essentially
broken? How does ShimmerConnect work then? I also wonder why the
inquiry response fails so consistently and has a very limited number of
failure modes; if it was outside interference, I'd expect it to be more
random.
Also, I'm not using TCP/IP, but the lower level serial port protocol. I
know how TCP fragmentation works though.
Mikko
On 24.07.2012 20:28, steve ayer wrote:
mikko,
are you framing the original packets as the bytes come in? you realized
that just because you send 30 bytes, you don't necessarily receive them
all at once, right?
regarding phil's comments, tcp/ip doesn't break down bluetooth packets,
the bluetooth stack transmits them that way. if tcp broke packets into
random-sized chunks, then the network that carries this reply would stop
dead.
-steve
On 07/24/2012 01:16 PM, Mikko Rasa wrote:
Yes, it's a custom program. The goal is to visualize the position of
the Shimmer module (or multiple modules) in a 3D scene, but for now I'm
just dumping the data read over bluetooth to the terminal.
Is there anything particular I should check about the program? At the
moment it just opens the rfcomm device, writes the command to start
streaming data (0x07), reads the response (0xFF) and starts reading the
data. Is there some example code for Linux that's known to work?
Mikko
On 24.07.2012 19:53, Phil Reaston wrote:
Hi Mikko:
What are you using to receive the data? I think you said earlier it
was
your own program. If so, I'd check that.
Phil Reaston
Disclaimer: This email and any files transmitted with it are
confidential
and intended solely for the use of the individual or entity to whom
they
are addressed. Please note that any views or opinions presented in
this
email are solely those of the author and do not necessarily represent
those
of the company. The company accepts no liability for any damage
caused by
any virus transmitted by this email.
On Tue, Jul 24, 2012 at 9:46 AM, Mikko Rasa<[email protected]> wrote:
Okay, magnetometer is working now. Even the data transmission seems
to be
more reliable, although there are still a few errors per second (with
50Hz
sampling). Should be usable provided that I manage to improve the
filtering to accept packets with real zero bytes, or alter the
firmware to
provide better synchronization.
However, the inquiry command still fails to produce a valid
response. A
varying amount of bytes from the middle of the packet are always
lost,
sometimes starting right from the ack byte. The most common case is
that I
receive the ack byte and the last five channels (out of nine). This
is not
a huge problem in itself, since I can hardcode the channels, but it
does
indicate that something is going wrong with sending data over BT.
To eliminate possible radio interference, I grabbed my laptop and a
Shimmer and went outside to a spot with no other radio sources (that
I know
of) nearby. This had no effect on reliability.
Mikko
On 24.07.2012 18:43, Benjamin Kuris wrote:
Open your shimmer, reseat the 9DoF board. You should get
magnetometer
data using stock FW and shimmer connect when following the
quickstart
in the manual. If that fails, email [email protected] for
further assistance.
-Ben
On Tue, Jul 24, 2012 at 11:33 AM, Mikko Rasa<[email protected]> wrote:
I tried various sample rates between 5Hz and 50Hz. I'm using the
accelerometer and gyro, so six channels in total. If I enable the
magnetometer channels, I don't get any data; I guess the device is
unable to
read the sensors.
I have two Shimmer modules at my disposal, and am testing with one
at a
time
for now. In the final application I hope to use two or three
simultaneously. I have a WLAN, a couple of wireless mice and a
BT-enabled
cellphone here as well, and of course my laptop which I'm receiving
the
data
on. However, when I tested rfcomm between the cellphone (N900,
running
Linux) and the laptop, I was able to transfer large amounts of data
without
any errors (several kilobytes; I'm lucky if I get 100 sequential
error-free
bytes from the Shimmer).
This is stock boilerplate firmware, aside from whatever hacks I've
put in
while testing and might not have reverted fully. I also tried the
official
firmware from shimmer-research.com (in case my build environment
somehow
introduced errors), but it has the same problems.
The problem seems to be very data-sensitive; If I turn the Shimmer
to its
side so that the X axis accelerometer gets a low value, I receive
almost
no
zero bytes at all, and when I do manage to receive a valid packet
(i.e. a
zero byte followed by 14 non-zero bytes), the contents are badly
mangled.
Mikko
On 24.07.2012 18:11, Benjamin Kuris wrote:
What sample rate and how many channels?
How many devices in your piconet?
Any other RF in 2.4GHz band?
Are you using stock boilerplate data packets or are you sending
full
ACL packets (~128bytes data) to minimize BT overhead?
Typically you don't need to mess with the baudrate but
minimizing BT
overhead will improve results if you can accept a little bit of
latency.
I'm sure others will chime in.
-Ben
On Tue, Jul 24, 2012 at 10:57 AM, Mikko Rasa<[email protected]> wrote:
I'm trying to use a Shimmer 9DoF sensor with the boilerplate
firmware,
but
bluetooth transmission is giving me a hard time. It's dropping a
lot
of
bytes, making data streaming unreliable and certain commands
unusable.
In
particular, I can't get an intact reply from the inquiry
command to
discover
the order of channels being sent, so I have to hardcode this
into my
program
based on the boilerplate sources. With the data packets I could
use
the
packet identifier to synchronize the stream and throw out any
truncated
packets; this might spuriously reject good packets if they
contain
embedded
zero bytes, but I estimate I could recover around 80% of the
packets
this
way.
Is there anything I could do to improve the reliability? I've
already
tried
different sampling rates, as well as messing with the baud rate
used
between
the MSP430 and the BT module. Neither of these seems to have any
effect
on
the reliability, unless I change the baud rate so far out of spec
that
all
transmissions stops.
--
Mikko
______________________________**_________________
Shimmer-users mailing list
[email protected]
https://lists.eecs.harvard.**edu/mailman/listinfo/shimmer-**users<https://lists.eecs.harvard.edu/mailman/listinfo/shimmer-users>
______________________________**_________________
Shimmer-users mailing list
[email protected]
https://lists.eecs.harvard.**edu/mailman/listinfo/shimmer-**users<https://lists.eecs.harvard.edu/mailman/listinfo/shimmer-users>
_______________________________________________
Shimmer-users mailing list
[email protected]
https://lists.eecs.harvard.edu/mailman/listinfo/shimmer-users
_______________________________________________
Shimmer-users mailing list
[email protected]
https://lists.eecs.harvard.edu/mailman/listinfo/shimmer-users
