I am bringing back CAN stuff for PLC4J since ApacheCon Asia had forced
me to finally move forward. Code will follow shortly as I am reaching
the stabilization point.
A quick summary of changes.
1) I brought a `plc4j-transport-can` which contains a `CANTransport`.
Main role of this module is definition of CAN related APIs such as
`CANFrameBuilder`, `FrameData` which can be used by drivers who intend
to interact with CAN bus.
2) SocketCAN transport now uses generated mspec structure to map binary
form of blob we receive from JavaCAN library. By this way, at least some
portion of mapping is portable across languages.
3) CANOpen mspec defines its own frame type which is used to implement
entire protocol logic. This type has no common denominator with CAN or
socketcan frame. It represents only information specific to this
particular protocol.
4) Earlier work which been done with Sebastian and custom stacks opened
drivers to do a little bit more than just declare types. This allows
CANopen driver to inject a wire adapter from own message definition to
the desired format.
I had a fairy long battle with compiler and myself trying to abstract
things away and have a top level definition of CANFrame as well as
CANOpenFrame. In the end I failed making it flying, despite of small
successes either in transport or protocol itself.
Given that we use a lot of code generation which is unaware of language
specific syntax or features I ended up leaving these as-is.
A main reason why such approach didn't go very well was amount of
complications it created. We had then:
1) CANFrame
2) SocketCANFrame (generated)
3) SocketCANFrameWrapper extends SocketCANFrame implements CANFrame
4) CANTransport had a #CANFrameBuilder interface to hide it from driver
5) CANOpenFrame (generated)
6) Builder which could in theory bridge above type to CANFrame
As you can see, a lot of troubles. In order to keep whole thing straight
I left below:
1) FrameData
2) SocketCANFrame (generated)
3) CANOpenFrame (generated)
4) CANTransport declares few additional indicators:
a) Wire type - SocketCANFrame and associated MessageIO
b) It must return adapter from wire type to FrameData
c) It must provide frame builder back to wire type
5) There is an additional `CANDriverBase`.
By this way we are able to utilize generated code as well as use any of
further APIs coming from third parties to interact with vendor specific
transports or APIs. Most importantly, as long as transport can declare
wire encoding and provide it to driver, it will work.
Few words about motivations.
Given that CANOpen does rely on own processing logic such as serviceId
and nodeId which are delivered from CAN frame identifier (COB id) any
unification attempt around CANFrame lead to massive amount of changes in
existing protocol logic. I did not want to pollute that logic with bit
masks, int/short casts and so on. Additionaly I still have in back of my
mind compatibility/portability of logic to CAN 2.0B/CAN FD.
I simply had to look for other way.
Since we have separate root frame for CANopen and transport we can do
better tests of protocol implementation, without introducing a transport
specific encoding.
With CANFrame as an universal wire format for CANopen protocol I was
forced to do a comparison of serialized byte arrays instead of CANopen
objects. Even if it would fly somehow with CAN 2.0a (8 byte payloads), I
could not imagine this working for CAN FD (payloads up to 64 bytes).
Loosing a test visibility, especially after improvements test framework
received recently, was not acceptable.
Above approach with asymmetric API with separate wire format to write to
bus and frame data to receive from bus allows to retain a single
transformation logic from any transport. It works even if we don't have
a canonical form of a CAN frame.
Main place of work is done in a portable translation layer with a new
type called `FrameHandler`. This translation layer is responsible for
two call paths:
1) wire -> FrameData -> driver frame handler -> driver frame
2) driver frame -> driver frame handler+transport frame builder -> wire
As you can see it uses extensively CAN transport information in
combination with FrameHandler implemented by an driver.
I think that over time we can seek additional optimizations or
simplifications in above design.
For any new CAN driver independent of wire format it is necessary to:
1) do an mspec
2) extend `Plc4xCANProtocolBase` (it makes #decode method public)
3) provide a `FrameHandler` implementation specific to driver
4) wire everything together using custom stack builder
The end result of all gymnastics I made is this:
customStack.builder(trans.getMessageType(), trans.getMessageIO)
.withProtocol(cfg-> {
CANOpenProtocolLogic protocolLogic = new CANOpenProtocolLogic();
ConfigurationFactory.configure(cfg, protocolLogic);
return new CANDriverAdapter<Message, CANOpenFrame>(
protocolLogic,
trans.getMessageType(),
trans.adapter(),
new CANOpenFrameDataHandler(trans.getTransportFrameBuilder())
);
})
.withDriverContext(cfg -> new CANOpenDriverContext())
.withPacketSizeEstimator(cfg -> trans.getEstimator())
.littleEndian()
.build();
For now the CANDriverAdapter is very close to netty apis we have. Not in
terms of what it does, but in masking the sendToWire methods.
It was necessary to inject additional translation logic implemented by
driver frame handler. It is a bit unfortunate that there is no better
way, especially that ConversationContext and SendRequestContext have own
quirks which may cause additional maintenance cost in there.
With all this I guess we might be able to see if this is portable to
other drivers, especially ones which do rely on raw ethernet! :)
Best,
Łukasz
On 14.04.2021 02:36, Łukasz Dywicki wrote:
> I began to scratch surface of Driver/Protocol/Context and here is what I
> found.
>
> Currently it is *impossible* to do any customization with
> ProtocolStackConfigurer or GeneratedDriverBase without really copying
> whole classes.
> Trouble I see is:
> 1) Protocol stack configurer rely on classes instead of actual instances.
> Is there any reason why packet estimator and byte discarder are passed
> there as classes and not actual instances? These are functions which
> *should* not have side effects hence we should not require an reflection
> to create instance of these.
> 2) Driver which is asked to create protocol stack has *no access* to
> actual configuration.
> 3) Transport is "lost in transition" while creating pipelines.
> It is not retained nor accessible to stack configurer making it
> impossible to ie. make transport dependent stack configuration. As much
> as this situation is not very common. yet we either force user to pass
> extra URL parameters or let drivers do their defaults based on transport.
>
> Very simplified startup flow for driver is:
>
> generated driver initialization ->
> create configuration
> find transport ->
> configure transport
> transport -> create channel factory
> create connection ->
> create field handler
> create value handler
> create stack configurer
>
> My proposal is to switch it to:
> create configuration
> find transport ->
> configure transport
> transport -> create channel factory
> create connection ->
> create field handler
> create value handler
> create stack configurer*(configuration, transport)*
>
> Another finding I have - estimators and discarders are created by
> reflection in hope to receive configuration from URI parameters.
> Such situation so far does not happen and I think its rather rare (yet
> we can face it some day), thus I would opt for having configuration
> calls with them, however I would move that part from stack configurer
> above - to the driver. In case if driver expects configuration options
> for its wire processors it must enable it explicitly.
> This will also leave a space to create base class for CAN drivers and
> utilize transport and configuration to create estimator and discarder
> dynamically.
>
> Looking forward to hear your opinion about above changes.
>
> Best,
> Łukasz
>
> On 29.03.2021 06:43, Andreas Oswald wrote:
>> Hi Łukasz,, hi everybody,
>>
>> that sounds pretty interesting, do you have plans to to implement "higher
>> level" protocols based on CAN e.g. UDS? Maybe we should exchange ideas as my
>> daytime job ist about developing a software package that takes care of
>> Software Versions and parameter sets for electrical vans, where we rely
>> heavily on UDS. Unfortunately not all of our ECUs implement fine UDS.
>>
>> When Chris introduced me to the project I guess a bit more than two years
>> ago, Slack was the medium to use.
>>
>> Maybe someone can give me a hint about how communication is dealt with.
>>
>> Good to be back 😉
>>
>> Have a great start for the new week.
>>
>> Take care
>>
>> Andreas
>>
>> Andreas Oswald
>> Marienstraße 12
>> 52223 Stolberg
>> Germany
>>
>> ++49 2402 905 3852
>> ++49 172 426 1598 (also Signal)
>> WhatsApp/Telegram on ++49 157/38459358
>>
>> ________________________________
>> Von: Łukasz Dywicki <[email protected]>
>> Gesendet: Montag, 29. März 2021 02:18
>> An: [email protected] <[email protected]>
>> Betreff: Further updates to CAN stuff in plc4j
>>
>> Hey all,
>> I began to think how to cover basic interactions with CAN bus which
>> could serve a custom systems or ones which are closer to some narrow
>> cases. My intention here is to offer simplest possible way to become
>> CANbus client without any specific application layer protocol.
>> Results of my thought process are following.
>>
>> Driver operations: CAN as protocol is rather event based hence CAN
>> driver should not support "read" but only write and subscribe. A "read"
>> in our api indicate rather an synchronous call for which we expect an
>> answer. This is not a case with CAN where you can keep broadcasting but
>> never expect any answer back or could have multiple replies. While
>> downstream protocols can offer such functionality (canopen does) at most
>> basic level we can't provide any handling for that. Definitely it would
>> make hard to port driver over different languages.
>> If this driver gets implemented end users can use it as a very thin
>> client API which gives them access to the bus and covers very basic
>> handling of subscriptions for specific COBs (CAN Object Identifier) and
>> possibly basic IEC types via field handlers.
>>
>> Transports: an awful part of CAN is amount of APIs which can be used to
>> access a bus. The low level stuff is done by microcontrollers and hidden
>> from applications who interact with a bus. In the end socketcan, slcan,
>> usb2can or whathever-to-can are just multiple ways to publish CAN frame
>> onto physical medium.
>> I did hang on this for a moment, cause driver syntax we have is briefly:
>> <protocol>:<transport>://<transport-options>?<driver-options>
>>
>> Currently connection string for CANopen looks like this:
>> canopen:socketcan://vcan0?heartbeat=false
>> Yet for slcan or can2usb it could look like this:
>> canopen:slcan://dev/ttyUSB0;baud=50000?heartbeat=false
>> canopen:can2usb://COM4;baud=50000?heartbeat=false
>> (eventually with baud= parameter in query string).
>>
>> Whatever syntax we use to express transport and its configuration the
>> problem is later shifted to the edge of Driver and Transport. Different
>> APIs might bring different frame length estimators or discarded byte
>> handling nor filtering capabilities. Currently driver is one who is
>> responsible for that, yet in case of CAN - socketcan uses 0x00 to always
>> pad data while serial bridges do not require it at all. At present I see
>> no way to fit it without breaking some things.
>> CAN transport essentially must provide above estimators. Maybe in future
>> some other features specific to this transprot ie. message filtering
>> which seems to be very common thing. For driver it must provide a way to
>> publish "COB + byte[]" and receive "COB + byte[]".
>>
>> So far we have support only for socketcan transport. It did the job to
>> get into CAN, yet it is not the only one. Its not portable over all
>> platforms. Because of above condition canopen driver actually depends on
>> socketcan transport and its encoding of frames.
>> My long term plan in this area is extraction of CANFrameBuilder part out
>> of canopen driver to plc4j-transprot-can. This transport should define
>> CAN specific transport APIs. What do you think about that?
>>
>> Once this step is made we can bring simplistic CAN driver with very low
>> effort. It will also open a path for other CAN transports which could
>> utilize yet another hardware and serve other operating systems.
>>
>> Best,
>> Łukasz
>>
