Hi Lukasz,
I am not sure I am understanding the problems you are facing. We already have
LE and BE protocols.
For example EIP is LE and the rest is generally BE. It seems that ADS/AMS is
also LE.
mspec doesn't even know about endianness.
Up till now the endianness doesn't have an effect on bit-fields or single-bit
ints.
It only starts to affect if a field goes from one byte to the next, which is
usually for (u)int and floating point values.
That's why we have created the Read/WriteBuffers to set their endianness in the
constructor.
So if you're creating a driver for ADS/AMS which is LE, then you write the
mspec according to the sequence the information is located in the transferred
bytes and have the Read/WriteBuffer handle the endianness issue.
I do see a problem when there are drivers that use mixed endianness, but we
have still to encounter such a protocol.
So I have to admit that I don't like any of the mspec changes you proposed, as
I think you are just not using the tools we have the right way.
Chris
Am 14.04.20, 00:32 schrieb "Łukasz Dywicki" <l...@code-house.org>:
Hey Niclas,
I realized how old the old things are when I started preparing
automation training for mere mortals and got into history of frames and
even cabling. Mr. Modbus and EIA-485 is definitely older than I. ;-)
Getting back to the point - yes. I been thinking how to address the byte
order in effective way. Here are two approaches I have for now:
A) My initial attempt is just a temporary buffer which is then written
in reverse order to caller. For reading it is similar - just getting N
bytes in reversed order. The hard part is.. knowing N. I had to add a
static calculation in order to allocate valid buffer sizes. I tend to
work but I'm not happy with this approach cause it involves additional work.
B) Second idea I've got is really simple and relies on code generation.
We know in which order fields are coming. Here I'm referring to a State
field which is just bunch of bits. If we would group fields in bytes and
generate code in reverse order then it has chance to work. Requirement
for that - ability to know basic field sizes upfront.
C) Try to combine above with bit-io or Read/WriteBuffers as these are
places which know actual position and state of buffers which are being
read/written.
Now, getting to two cases which are a problem. CommandId and State. So
with command id situation is simple as it is declared as enum and it is
read as uint. We know size upfront and can generate valid method call
(readIntLE).
[enum uint 16 little endian 'CommandId'
['0x00' INVALID]
['0x01' ADS_READ_DEVICE_INFO]
['0x02' ADS_READ]
['0x03' ADS_WRITE]
['0x04' ADS_READ_STATE]
['0x05' ADS_WRITE_CONTROL]
['0x06' ADS_ADD_DEVICE_NOTIFICATION]
['0x07' ADS_DELETE_DEVICE_NOTIFICATION]
['0x08' ADS_DEVICE_NOTIFICATION]
['0x09' ADS_READ_WRITE]
]
Second candidate is what I'm stuck right now sniping next cycles of
problems. So in case of State we have complex type composed from 2
bytes. A note here - instead of two bytes we might have a variable
length type which includes array or other variable section.
[type little endian 'State'
[simple bit 'broadcast' ]
[reserved int 7 '0x0' ]
[simple bit 'initCommand' ]
[simple bit 'updCommand' ]
[simple bit 'timestampAdded' ]
[simple bit 'highPriorityCommand' ]
[simple bit 'systemCommand' ]
[simple bit 'adsCommand' ]
[simple bit 'noReturn' ]
[simple bit 'response' ]
]
The order of reading big endian encoded data to impose little endian
shift would be (please correct me if I'm wrong):
1) init
2) udp
3) add timestamp
4) priority
5) system
6) ads
7) noreturn
8) response (end of byte 1)
9) broadcast
10) reserved (end of byte )
We can do same trick for writing, by re-arranging fields. By this way we
avoid any additional byte level operations.
Overall trouble with generated driver is to declare "how much" bytes
should be read and interpreted. We have precise size information at the
runtime - due to length fields, we can leverage it at generation time,
but then we won't be able to cover all cases.
I would love to keep it simple and do not break things thus I need your
advice on how to approach this problem in a valid way.
Cheers,
Łukasz
On 13.04.2020 03:26, Niclas Hedhman wrote:
> <anecdotal-rant>
> For us who were around and shaping the protocols in the 1980s, and people
> before us (and before standards like RS-232), a lot of the
"specifications"
> came out of "observation of implementation we managed to get to work",
> rather than "implement this spec". A lot was due to extreme memory
> constraints (in my case, multi-tasking operating system, serial protocol
> 187kbps, interpreted programming language with floating point ops and user
> applications in 2kB RAM and 8kB EPROM) and a general lack of information,
> like what other people were doing, sharing experiences and so on.
>
> And there were many "innovative" ways to squeeze just a little bit extra
> out of the hardware, resulting in "hard to understand" consequences. Bit
> packing was a typical one, multiple functions packed into a single byte.
> Look at page 14 in https://www.nxp.com/docs/en/data-sheet/80C31_80C32.pdf
> and read up on "UART Enahanced Mode", and we used this, i.e. 9 bits, no
> parity and clever use of address and mask to create a slave-to-slave
direct
> protocol, where the master's role was to signal which slave "owned" the
> cable. Yeah, in that 8kB ROM limitation (I think protocol was about 1kB
> ROM) and something like 150 bytes RAM for comm protocol.
>
> Could you implement a compatible device to this with PLC4X and modern
> hardware (i.e. no 8031/32 co-processor)? Possibly but bit-banging is
needed
> to support the 9bit data (+start and stop bits) and an awful lot of CPU
> cycles on something that was automatic on one of the slowest long-lived
> microcontroller ever.
> </anecdotal-rant>
>
> My point was only to highlight that some of the strange things you see in
> protocols today, have its roots in pre-standardization days. Today no one
> would go down that route, because the hardware cost nothing now (8031 +
> 8kB EPROM + 2kB static RAM + battery backup => ~$50 in 1983's currency)
and
> longevity of software is more important.
>
> Cheers
> Niclas
>
>
> On Sun, Apr 12, 2020 at 10:10 PM Christofer Dutz
<christofer.d...@c-ware.de>
> wrote:
>
>> Hi Lukasz,
>>
>> I think it really gets tricky when using BE and having some
byte-odd-sizes
>> ... I remember in the Firmata protocol there were some bitmasks and then
10
>> bit uint as BE ... not it really got tricky as the specs were written
from
>> a point of view: You read 16 bits BE and then the first6 bits mean XYZ
>> instead of describing how the bits actually travel over the wire.
>>
>> Chris
>>
>>
>>
>> Am 11.04.20, 01:21 schrieb "Łukasz Dywicki" <l...@code-house.org>:
>>
>> I've made some progress with topic by modyfing mspec and allowing
>> 'little endian' flag on fields. This moved me further to next issue -
>> which is whole type encoded little endian.
>>
>> In ADS driver such type is State, which has 2 bytes and uses 8 bits
for
>> various flags.
>> There are two cases which require different approach - reading and
>> writing. So for reading we need to swap N bytes based on type length.
>> For writing we need to alocate buffer for N bytes and swap them
before
>> writing.
>>
>> I am stuck now with freemaker templates and bit-io.
>>
>> Cheers,
>> Łukasz
>>
>>
>>
>> On 10.04.2020 17:57, Łukasz Dywicki wrote:
>> > I am doing some tests of ADS serialization.
>> >
>> > I've run into some troubles with payload which is generated with
new
>> > driver. I'm not sure if that's my fault or generated code.
>> >
>> > I did a verification of what Wireshark shows and how ads structures
>> are
>> > parsed. There is a gap I think. For example ams port number 1000
>> > (0x1027) is read as 4135.
>> >
>> > Obviously I used wrong structures while implementing protocol logic
>> in
>> > first place, but now I am uncertain of how fields are encoded. How
we
>> > mark field as little endian when rest of payload is big endian? Do
we
>> > have `uint_le`?
>> >
>> > As far I remember route creation logic I was tracking last week
used
>> > combination of LE and BE.
>> >
>> > Best regards,
>> > Łukasz
>> >
>>
>>
>>
>
