Hi all,
Well, I was sort of dreaming of the “splitting up” could be a generic
functionality in plc4x.
This is exactly what the S7 optimizer currently does, however it’s missing one
vital function.
In Ben’s example … imagine the request was like this:
PlcReadRequest.Builder builder = plcConnection.readRequestBuilder();
builder.addItem("value-1", "400001");
builder.addItem("value-2", "400002");
builder.addItem("value-3", "400010[10]");
builder.addItem("value-4", "400020[150]");
PlcReadRequest readRequest = builder.build();
Then it could read value-1 to value-3, but value-4 can’t be read, as it already
exceeds the limit.
So we would need something that for example rewrites this query to:
PlcReadRequest.Builder builder = plcConnection.readRequestBuilder();
builder.addItem("value-1", "400001");
builder.addItem("value-2", "400002");
builder.addItem("value-3", "400010[10]");
builder.addItem("value-4-1", "400020[125]");
builder.addItem("value-4-2", "400145[25]");
PlcReadRequest readRequest = builder.build();
And then merges the results form value-4-1 and value-4-2. We should probably
just interpret these as “WORD” typed arrays and do the higher level decoding on
the joined data.
I mean … if we were reading an array of REAL values, then one real value would
be split over both tags. Only if we join the byte array first and then do the
decoding will we not be having any problems.
Chris
#
From: Niels Basjes <[email protected]>
Date: Monday, 16. January 2023 at 09:42
To: [email protected] <[email protected]>
Subject: Re: [Modbus] Handling 'too large' blocks of registers
Yes, that makes sense.
To me the question remains if the optimizer for this should be in plc4x
(which is very possible) or in the schema aware layer I mentioned.
One case for this is that in SunSpec there are some groups of logical
values / registers that _must_ be read combined. The optimizer must be
aware of this kind of requirement.
Niels
On Mon, Jan 16, 2023 at 9:34 AM Ben Hutcheson <[email protected]> wrote:
> Splitting requests using a schema shouldn't really be necessary though, as
> you still need to define the individual tags within the PLC4X Read request.
>
> So you could define each read request as one tag, each tag wouldn't be more
> than the 125 limit. Then within the optimizer it would build larger
> requests made up of these read requests. This way there's no chance of
> splitting individual tags into more than 1 request, but you get the benefit
> of merging requests into one larger one.
>
> An example below, this would then be grouped in the optimizer logic into 3
> read requests, the first request would contain values 1 and 2, the second
> just value 3 and the third only value 4.
>
> PlcReadRequest.Builder builder = plcConnection.readRequestBuilder();
> builder.addItem("value-1", "400001");
> builder.addItem("value-2", "400002");
> builder.addItem("value-3", "400010[10]");
> builder.addItem("value-4", "400020[120]");
> PlcReadRequest readRequest = builder.build();
>
>
> On Mon, Jan 16, 2023 at 9:21 AM Niels Basjes <[email protected]> wrote:
>
> > Hi,
> >
> > Yes this is exactly my idea as well.
> > Splitting a request for >125 registers into multiple requests cannot be
> > done in a generic reliable way.
> > If you know the schema (i.e. meaning of the registers) then you can
> combine
> > requests with multiple logical values into blocks of modbus registers
> that
> > stay within the 125 register limit.
> >
> > Which implies the need for a (generic) way of defining logical values and
> > the modbus registers from which they can be read (with all kinds of
> > constraints about reading them).
> > I would call this a modbus schema definition which can then be used to
> > calculate the actual read/write operations.
> >
> > The effect would be that for every type of device a schema definition is
> to
> > be created from which a client can be created/generated.
> >
> > Niels
> >
> >
> >
> >
> >
> > On Mon, Jan 16, 2023 at 8:50 AM Ben Hutcheson <[email protected]>
> > wrote:
> >
> > > Hi Niels,
> > >
> > > As Łukasz pointed out, there are a thousand different combinations of
> how
> > > Modbus requests can be arranged. I'm not sure there's a generic schema
> > that
> > > can be used to define it though.
> > >
> > > You talked originally about splitting large reads (>125 registers) up
> > into
> > > smaller registers. This logic would live in a class that implements the
> > > BaseOptimizer. This is then defined within the ModbusTCPDriver class,
> we
> > > are just using the SingleTagOptimizer for Modbus at the moment.
> > >
> > > If I were to implement this I would probably firstly try and combine
> tags
> > > that are adjacent into one request until we either encounter a break in
> > the
> > > requested area or approach the limit of ~125 registers. This way it is
> > > guaranteed that requests for individual requests are not broken into
> > > smaller requests.
> > >
> > > Kind Regards
> > >
> > > Ben
> > >
> > >
> > >
> > >
> > >
> > > On Sun, Jan 15, 2023 at 6:58 PM Niels Basjes <[email protected]> wrote:
> > >
> > > > Hi Łukasz,
> > > >
> > > > At this point I tend towards the need for a tool that allows one to
> > > define
> > > > the schema of a modbus device with all available values and code to
> > > convert
> > > > them into usable stuff (like in java an actual String or Double).
> > > > So far it seems this needs indeed flags indicating which values can
> and
> > > > must be read combined (like the sync block in SunSpec).
> > > > Then based on this a request for a set of functional values is
> > converted
> > > > into a set of modbus requests that follow these defined rules.
> > > >
> > > > Rules about an after connection delay and maximum requests per second
> > > seem
> > > > like something that should be at the modbus level (i.e. part of
> plc4x).
> > > >
> > > > Is there an existing project that does something like this?
> > > >
> > > > Niels
> > > >
> > > > On Sun, Jan 15, 2023 at 5:22 PM Łukasz Dywicki <[email protected]>
> > > > wrote:
> > > >
> > > > > I am really glad you are moving it forward and fighting it!
> > > > >
> > > > > It is fairly difficult to get modbus code aligned and working
> > > > > automatically because different devices behave differently with
> even
> > > > > most basic requests.
> > > > > I remember that one of electric meters I had to work with did
> support
> > > > > read multiple registers but did not support going over "empty"
> > indexes.
> > > > > By empty I mean these which were not declared in the
> documentation,
> > > > > even if it was still below 125 limit.
> > > > >
> > > > > Effectively I could draw three primary options to cover:
> > > > > - read continuous (just keep expanding requests up to the limit)
> > > > > - read blocks (group indexes into blocks up to limit, if they are
> > > > > continuous)
> > > > > - read single (read multiple registers is not supported)
> > > > >
> > > > > Some devices, especially older PLCs may also require extra delay
> > after
> > > > > data retrieval, so you have to wait between finishing one request
> and
> > > > > starting another. Some inverters and official dongles made by
> fairly
> > > > > large Chinese manufacturer, which brand I won't mention here, as
> well
> > > as
> > > > > primitive tcp/rtu bridges, may require "after connection delay" in
> > > order
> > > > > to establish rtu connection they use to serve tcp session.
> > > > >
> > > > > All this makes modbus option list rather long, compared to
> abilities
> > it
> > > > > gives!
> > > > >
> > > > > Cheers,
> > > > > Łukasz
> > > > >
> > > > >
> > > > > On 15.01.2023 17:05, Niels Basjes wrote:
> > > > > > Hi,
> > > > > >
> > > > > > No, I'm reading many logical values where the needed registers
> > > combined
> > > > > > exceed the 125 mark.
> > > > > > Assume my application needs a set of values and asks for such a
> > block
> > > > of
> > > > > > registers.
> > > > > > Then the modbus library that gets the request for 200 registers
> can
> > > no
> > > > > > longer reliably split it into multiple parts that yield the
> correct
> > > > > value.
> > > > > >
> > > > > > What I understand of your S7 example (I do not know this device)
> > > seems
> > > > > like
> > > > > > a device that offers a single value that exceeds the limits of
> > modbus
> > > > and
> > > > > > thus I expect they made it so you can fetch it step by step.
> > > > > >
> > > > > > I'll see if I can do a wireshark dump for you.
> > > > > >
> > > > > > What should I make the issue say? I currently do not see this as
> a
> > > bug
> > > > or
> > > > > > problem with plc4x.
> > > > > >
> > > > > > Niels
> > > > > >
> > > > > > On Sun, Jan 15, 2023 at 4:46 PM Christofer Dutz <
> > > > > [email protected]>
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Niels,
> > > > > >>
> > > > > >> I think this might be related to what happens when reading
> Strings
> > > in
> > > > S7
> > > > > >> without providing a length.
> > > > > >> In this case each string is 255 charaters long and it already
> > > exceeds
> > > > > the
> > > > > >> size-limit of 200 or so of a S7 1200.
> > > > > >>
> > > > > >> In S7 we use some code to split up multiple tags into multiple
> > > > requests,
> > > > > >> but rewriting the query to support automatic splitting of one
> > single
> > > > tag
> > > > > >> into multiple requests is quite a bit more difficult.
> > > > > >>
> > > > > >> I thought you were reading multiple tags that in sum exceeded
> the
> > > 125
> > > > > >> registers and for that case simply something similar to the S7
> > query
> > > > > >> updater should also work for other protocols.
> > > > > >>
> > > > > >> Do I understand you correctly, that in your case you want to
> read
> > > one
> > > > > tag,
> > > > > >> that is bigger?
> > > > > >>
> > > > > >> Chris
> > > > > >>
> > > > > >> From: Niels Basjes <[email protected]>
> > > > > >> Date: Sunday, 15. January 2023 at 16:38
> > > > > >> To: [email protected] <[email protected]>
> > > > > >> Subject: [Modbus] Handling 'too large' blocks of registers
> > > > > >> Hi,
> > > > > >>
> > > > > >> A few weeks ago some code of mine was merged to limit the number
> > of
> > > > > >> registers that can be requested in a single ModbusTag because
> > asking
> > > > for
> > > > > >> more than 125 registers in a single request will always fail
> > > (because
> > > > of
> > > > > >> the way modbus works).
> > > > > >> https://github.com/apache/plc4x/pull/721
> > > > > >>
> > > > > >> One of the comments was asking if it can be automated in a
> generic
> > > way
> > > > > to
> > > > > >> split a "too large" request into multiple smaller requests.
> > > > > >> So If I ask for a block of 200 registers then plc4x would simply
> > > split
> > > > > it
> > > > > >> into multiple requests (like 125 and 75 for example, or 100 and
> > 100)
> > > > and
> > > > > >> afterwards merge the resulting registers of the two requests
> back
> > > > into a
> > > > > >> single block of 200 registers for the upstream application to
> > > analyze.
> > > > > >>
> > > > > >> I said I would look into this and this is what I found while
> doing
> > > > some
> > > > > >> experimenting with my real solar converter.
> > > > > >> This device uses the SunSpec standard for defining meaning to
> > modbus
> > > > > >> registers.
> > > > > >>
> > > > > >> My current conclusion at this point is that it is impossible to
> > > handle
> > > > > this
> > > > > >> at the modbus level and I'm looking for you guys to challenge me
> > in
> > > > > this.
> > > > > >>
> > > > > >> Why do I say this?
> > > > > >> One of the logical values you can retrieve is the name of the
> > > > > Manufacturer
> > > > > >> and the Model of the device that are both stored as UTF-8 in a
> set
> > > of
> > > > 16
> > > > > >> registers (i.e. usually 32 characters).
> > > > > >>
> > https://github.com/sunspec/models/blob/master/json/model_1.json#L27
> > > > > >>
> > https://github.com/sunspec/models/blob/master/json/model_1.json#L36
> > > > > >>
> > > > > >> I found that if I try to fetch only a part of this logical
> value I
> > > get
> > > > > >> either an error (INVALID_ADDRESS) or bad data.
> > > > > >> When I fetch the entire value I get the 16 registers which hold
> > the
> > > > > correct
> > > > > >> data.
> > > > > >>
> > > > > >> Output of my test program that fetches the Model of the device
> > > (which
> > > > > is in
> > > > > >> my case "SB3.6-1AV-41" ).
> > > > > >> ModbusTag[4x40021:UINT[16]]: 0x5342 0x332E 0x362D 0x3141 0x562D
> > > > 0x3431
> > > > > >> 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
> > > 0x0000
> > > > > >> ModbusTag[4x40021:UINT[15]]: INVALID_ADDRESS
> > > > > >> ModbusTag[4x40022:UINT[15]]: 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF
> > > > 0xFFFF
> > > > > >> 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF
> > > > > >>
> > > > > >> So if I retrieve the correct block I get it.
> > > > > >> If I try to get 1 register less than I should I get an error.
> > > > > >> If I try to get the registers starting from the 2nd register I
> get
> > > > only
> > > > > >> 0xFFFF (important: NO error and a PlcResponseCode.OK !!)
> > > > > >>
> > > > > >> I expect there to be many other modbus devices that have similar
> > > > effects
> > > > > >> when the logical value requires multiple registers.
> > > > > >>
> > > > > >> So for now I come to the conclusion that fetching only part of
> the
> > > > > >> registers that make up a logical variable in general will not
> work
> > > and
> > > > > (in
> > > > > >> some cases) cannot be detected to have failed.
> > > > > >>
> > > > > >> At this point my assessment is that the only way to handle this
> is
> > > by
> > > > > >> actually knowing the meaning of the block of registers (i.e.
> which
> > > > must
> > > > > be
> > > > > >> read together in a single read) and work from there.
> > > > > >>
> > > > > >> Note that the current SunSpec definition also has the concept
> of a
> > > > > 'sync'
> > > > > >> group of points (logical values that consist of 1 or more
> > registers)
> > > > > which
> > > > > >> are grouped in the specification with the intent of "indicating
> > > > > >> that the points in the group must be read and written
> atomically",
> > > > which
> > > > > >> effectively means you cannot read them individually from the
> > modbus
> > > > > >> standpoint.
> > > > > >>
> > > > > >> Looking for your feedback on this.
> > > > > >> Do you know of existing software to generically handle this ?
> > > > > >>
> > > > > >> --
> > > > > >> Best regards / Met vriendelijke groeten,
> > > > > >>
> > > > > >> Niels Basjes
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards / Met vriendelijke groeten,
> > > >
> > > > Niels Basjes
> > > >
> > >
> >
> >
> > --
> > Best regards / Met vriendelijke groeten,
> >
> > Niels Basjes
> >
>
--
Best regards / Met vriendelijke groeten,
Niels Basjes
