Somehow the mailing list removed all the bullet points (apparently it doesn’t
like html mails). Here again hopefully readable as plain text:
TLDR: changes in plc4x which help developing drivers across languages that have
no effect how plc4x send/receiver data or how you use the API
As of the last mail `Progress on plc4go` from 15.03.2021 I want to give another
overview what has happened between 1bd132a to f97d36f
1. Changes on read/write buffers In plc4go and plc4j
a. Note the below changes don’t have any impact on the API on
plc4x nor on the way data is written onto the wire
b. Existing buffers (which were byte base [writing and reading
bytes to a underlying reader/writer]) had an interface introduced which
abstracted the interaction from plc4x from the representation (reason for that
described in point 5).
c. Write buffer
i. 2 New methods added
1. PushContext: Indicates that the model is
about to write data which is enclosed by a logical context
2. PopContext: Indicates that the model is done
writing data which is enclosed by a logical context
ii. Every method got a logicalName and a
varargs/variadic writerArguments added
1. LogicalName: indicates the logical name of
the data which is about to be written (e.g. checksum flag, address)
2. WriterArguments: are optional hints about
the data being written (e.g. name of the enum [e.g.ADS_READ] being written, or
if a logical context is a list)
iii. The above points are completely ignored by the
byte based writers (which got renamed with the introduction of the interface to
*ByteBased) as the above changes don’t have any impact on byte based writing
(They could be used for log tracing for better debugging in the future)
d. Read buffer
i. 2 New methods added
1. PullContext: indicates that the model is
about to read data which is enclosed by a logical context
2. CloseContext: Indicates that the model is
done reading data which is enclosed by a logical context
ii. Every method got a logicalName and a
varargs/variadic readerArguments added
1. LogicalName: indicates the logical name of
the data which is about to be read (e.g. checksum flag, address)
2. ReaderArguments: are optional hints about
the data being read (e.g. name of the enum [e.g.ADS_READ] being written, or if
a logical context is a list)
iii. The above points are completely ignored by the
byte based readers (which got renamed with the introduction of the interface to
*ByteBased) as the above changes don’t have any impact on byte based writing
(They could be used for log tracing for better debugging in the future)
e. On plc4j the old methods are still available as default
methods but will be deprecated in the future (The buffers are meant to be used
in the context of the generated models which uses the new interfaces)
f. In the future I plan on adding new byte/byte[] based methods
to better encapsulate logical context on byte/byte[] fields which can then be
better rendered by implementors of the buffer interfaces (mostly required by
non byte based buffers). See my other mail on the mailing list about bytes
(https://lists.apache.org/thread.html/r5285838fd0b53d50fca34c2539b2d12ac275361d4faf5df36f3ec804@%3Cdev.plc4x.apache.org%3E).
2. Refactoring in golang SPI
a. Added default implementations for connection and codec
i. Moved default implementation into a spi/default
package
1. These can be configured with variadic
WithOption calls
ii. Moved duplicated codes into commons classes
iii. At the moment these a realized with a cycling
dependency for callback which get refactored to a non cyclic approach in the
future
b. Added default implementations for request and response
c. Removed cyclic dependency from request interceptor
cd Ported the request transaction manager from java
i At the moment this is a 1:1 port from java and
doesn’t make full use of the native golang features. The TM was required by the
s7 driver to prevent clients from overloading a plc4x (which could lead to a
huge RL crash ;) )
e. Added Ascii/Xml/Json buffers described in point 4 and 5
f. Updated goDocs
3. S7 Driver in plc4go
a. Implemented the S7 driver into plc4go
b. In the course of this I fixed several bugs and initially
with the ads drivers lead to many changes described in this mail
c. Added a s7Io_test.go (will be helpful to visualize
(https://github.com/apache/plc4x/blob/develop/plc4go/internal/plc4go/s7/s7Io_test.go)
many changes described in this mail)
4. AsciiBox/HexWriter in plc4go
a. During development of ads and s7 driver I had several
smaller bugs (e.g. padding issues which got fixed) were I had to stare at bytes
every day till I almost could identify
(https://github.com/apache/plc4x/blob/develop/plc4go/internal/plc4go/s7/s7Io_test.go#L349)
a s7Message by looking at the hex at the end of the day 🤪
b. This lead me to writing a small tool which would help me to
visually segment the message into logical context (see Point 1) to better look
for flipping bytes.
c. The AsciiBox was born which takes a string and draws a nice
box
(https://github.com/apache/plc4x/blob/develop/plc4go/internal/plc4go/spi/utils/asciiBox_test.go#L79)
(requires a boxed font which apparently GitHub doesn’t have; on boxed fonts
these are square 😉) around it (with a optional name)
d. Implementing this I added a `Box() utils.AsciiBox` to every
model which renders itself and Childs to a fixed width box (120.) which could
overflow a bit if required
e. In the initial version which doesn’t use the new buffers
(see point 4) a nice hex box could be drawn for bytes arrays.
f. Requiring to render hexdumps for data segments into a
dynamic box required a dynamic width hex dump. So I introduced hex.go which
could resize to any width. (Other than normal hex dumps this tool uses a base
10 byte index [normal use hex dumpers and render 16 bytes each line] which made
working with them easier that hex indexes [most of us count in base 10 anyway])
g. During later development a BufferBoxedBased was introduced
to render the messages true to their byte representation.
5. New xml/json writers in plc4go and plc4j
a. Important Background info: Implementing the BoxWriter (see
point 4.7) I noticed that I implemented the serializing/deserializing of
message twice (duplicated code that does the same but different is never good).
I noticed that we already had the code for serializing/deserializing messages
to/from the wire and it might be better to reuse this code for that. This
initially this lead to the introduction of the interfaces described in Point 1
(This also explains why I need a logical name and context for asciibox which
will also get now relevant for json xml). I also noticed that the xml was
imitating the byte serializing/deserializing (in golang). Even worse: Between
java and golang there were differences (Java used annotations and Jackson for
rendering / golang used a second generated implementation). This made it hard
when handling language agnostic xml test suites.
b. I introduced xml/json based writers which implemented the
buffer interfaces and deterministically (at least the should but at the moment
they need a bit improvement [e.g. plc4go json writing]) write/read xml/json
(xml has a fixed element oder json not which requires buffering on json at
least when reading). The model itself dictates when to read and to write and
the writing/reading/rendering is decoupled and encapsulated in the concrete
read/writer implementation (json/xml/ascii[go only]).
c. Whenever possible existing code was changed to use the new
xml readers/writers (e.g. point 6)
6. Updated xml representation for test suites (Parser/Driver)
a. Having now the new buffer interfaces and their xml
read/writer I could change
(https://github.com/apache/plc4x/commit/8f28d7965555ed1823425e565e5a4f5972f378ef#diff-5579ab196820eb2cdccdd7c7a429b35823e80aacd493038a1c0f13a3c025dce9R503)
the test suites.
b. The xml contains now more information (at points a bit too
much because there is no support for byte/byte[] rendering at the moment see
point 1.6).
c. Remaining java class names from test read request will be
removed in future changes and replaced by language agnostic representation
d. Updated xsd for suites to include protocol name and flavor
(which is required to reliable identify message types in golang and java)
e. Xml now contains additional attributes describing the basic
datatype and bit length (these are optional which are not required for parsing
as the buffer dictates the type on a read operation anyway but helps us humans
to get more context about the datatype. At the moment these get still validated
by the read buffers anyway to check for consistency [which will be configurable
if not already])
7. Feature update in golang api
a. Builders are now real chain able builders
i. Updated examples and documentation for the new chain
able builders
ii. Change is non breaking as the return type was
changed from void to builder
b. Added pl4go config package to configure tracing of workers
and transactions
c. Updated most golang docs in the api
8. Several Bugfixes (see commit messages in Ref compare at the bottom)
a. Refactored Testsuites to break down the big class in java
and add support for the buffers described in point 5
b. Much more
9. There are more changes contained and if you want to keep reading for
whatever reason (maybe you like reading long mails 😁) you can check the commit
descriptions (which are mostly quite verbose) in the GitHub/compare
(https://github.com/apache/plc4x/compare/1bd132a241ef7f8e6603450514f9d970c8dae91e...f97d36f51b8942b2a4e9f03328ae364edd73aa5d).
Also let me know in this tread if I overlooked a important change which isn’t
self describing or there are required features missing which don’t have a todo
in the code.
Thats for that and thanks for reading this far 😀
Sebastian
Ps: (Ref compare GitHub/compare
(https://github.com/apache/plc4x/compare/1bd132a241ef7f8e6603450514f9d970c8dae91e...f97d36f51b8942b2a4e9f03328ae364edd73aa5d))
