Side note: The max number of registers must become configurable for any of this to work. I have an sdm630 modbus device (power meter) which clearly documents fetching at most 80 registers at a time instead of the modbus default of 125.
On Mon, Jan 16, 2023 at 12:44 PM Christofer Dutz <[email protected]> wrote: > Hi all, > > Well, I was sort of dreaming of the “splitting up” could be a generic > functionality in plc4x. > This is exactly what the S7 optimizer currently does, however it’s missing > one vital function. > In Ben’s example … imagine the request was like this: > > PlcReadRequest.Builder builder = plcConnection.readRequestBuilder(); > builder.addItem("value-1", "400001"); > builder.addItem("value-2", "400002"); > builder.addItem("value-3", "400010[10]"); > builder.addItem("value-4", "400020[150]"); > PlcReadRequest readRequest = builder.build(); > > Then it could read value-1 to value-3, but value-4 can’t be read, as it > already exceeds the limit. > So we would need something that for example rewrites this query to: > > PlcReadRequest.Builder builder = plcConnection.readRequestBuilder(); > builder.addItem("value-1", "400001"); > builder.addItem("value-2", "400002"); > builder.addItem("value-3", "400010[10]"); > builder.addItem("value-4-1", "400020[125]"); > builder.addItem("value-4-2", "400145[25]"); > PlcReadRequest readRequest = builder.build(); > > And then merges the results form value-4-1 and value-4-2. We should > probably just interpret these as “WORD” typed arrays and do the higher > level decoding on the joined data. > I mean … if we were reading an array of REAL values, then one real value > would be split over both tags. Only if we join the byte array first and > then do the decoding will we not be having any problems. > > > Chris > > # > > > From: Niels Basjes <[email protected]> > Date: Monday, 16. January 2023 at 09:42 > To: [email protected] <[email protected]> > Subject: Re: [Modbus] Handling 'too large' blocks of registers > Yes, that makes sense. > To me the question remains if the optimizer for this should be in plc4x > (which is very possible) or in the schema aware layer I mentioned. > One case for this is that in SunSpec there are some groups of logical > values / registers that _must_ be read combined. The optimizer must be > aware of this kind of requirement. > > Niels > > > On Mon, Jan 16, 2023 at 9:34 AM Ben Hutcheson <[email protected]> > wrote: > > > Splitting requests using a schema shouldn't really be necessary though, > as > > you still need to define the individual tags within the PLC4X Read > request. > > > > So you could define each read request as one tag, each tag wouldn't be > more > > than the 125 limit. Then within the optimizer it would build larger > > requests made up of these read requests. This way there's no chance of > > splitting individual tags into more than 1 request, but you get the > benefit > > of merging requests into one larger one. > > > > An example below, this would then be grouped in the optimizer logic into > 3 > > read requests, the first request would contain values 1 and 2, the second > > just value 3 and the third only value 4. > > > > PlcReadRequest.Builder builder = plcConnection.readRequestBuilder(); > > builder.addItem("value-1", "400001"); > > builder.addItem("value-2", "400002"); > > builder.addItem("value-3", "400010[10]"); > > builder.addItem("value-4", "400020[120]"); > > PlcReadRequest readRequest = builder.build(); > > > > > > On Mon, Jan 16, 2023 at 9:21 AM Niels Basjes <[email protected]> wrote: > > > > > Hi, > > > > > > Yes this is exactly my idea as well. > > > Splitting a request for >125 registers into multiple requests cannot be > > > done in a generic reliable way. > > > If you know the schema (i.e. meaning of the registers) then you can > > combine > > > requests with multiple logical values into blocks of modbus registers > > that > > > stay within the 125 register limit. > > > > > > Which implies the need for a (generic) way of defining logical values > and > > > the modbus registers from which they can be read (with all kinds of > > > constraints about reading them). > > > I would call this a modbus schema definition which can then be used to > > > calculate the actual read/write operations. > > > > > > The effect would be that for every type of device a schema definition > is > > to > > > be created from which a client can be created/generated. > > > > > > Niels > > > > > > > > > > > > > > > > > > On Mon, Jan 16, 2023 at 8:50 AM Ben Hutcheson <[email protected]> > > > wrote: > > > > > > > Hi Niels, > > > > > > > > As Łukasz pointed out, there are a thousand different combinations of > > how > > > > Modbus requests can be arranged. I'm not sure there's a generic > schema > > > that > > > > can be used to define it though. > > > > > > > > You talked originally about splitting large reads (>125 registers) up > > > into > > > > smaller registers. This logic would live in a class that implements > the > > > > BaseOptimizer. This is then defined within the ModbusTCPDriver class, > > we > > > > are just using the SingleTagOptimizer for Modbus at the moment. > > > > > > > > If I were to implement this I would probably firstly try and combine > > tags > > > > that are adjacent into one request until we either encounter a break > in > > > the > > > > requested area or approach the limit of ~125 registers. This way it > is > > > > guaranteed that requests for individual requests are not broken into > > > > smaller requests. > > > > > > > > Kind Regards > > > > > > > > Ben > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Jan 15, 2023 at 6:58 PM Niels Basjes <[email protected]> > wrote: > > > > > > > > > Hi Łukasz, > > > > > > > > > > At this point I tend towards the need for a tool that allows one to > > > > define > > > > > the schema of a modbus device with all available values and code to > > > > convert > > > > > them into usable stuff (like in java an actual String or Double). > > > > > So far it seems this needs indeed flags indicating which values can > > and > > > > > must be read combined (like the sync block in SunSpec). > > > > > Then based on this a request for a set of functional values is > > > converted > > > > > into a set of modbus requests that follow these defined rules. > > > > > > > > > > Rules about an after connection delay and maximum requests per > second > > > > seem > > > > > like something that should be at the modbus level (i.e. part of > > plc4x). > > > > > > > > > > Is there an existing project that does something like this? > > > > > > > > > > Niels > > > > > > > > > > On Sun, Jan 15, 2023 at 5:22 PM Łukasz Dywicki < > [email protected]> > > > > > wrote: > > > > > > > > > > > I am really glad you are moving it forward and fighting it! > > > > > > > > > > > > It is fairly difficult to get modbus code aligned and working > > > > > > automatically because different devices behave differently with > > even > > > > > > most basic requests. > > > > > > I remember that one of electric meters I had to work with did > > support > > > > > > read multiple registers but did not support going over "empty" > > > indexes. > > > > > > By empty I mean these which were not declared in the > > documentation, > > > > > > even if it was still below 125 limit. > > > > > > > > > > > > Effectively I could draw three primary options to cover: > > > > > > - read continuous (just keep expanding requests up to the limit) > > > > > > - read blocks (group indexes into blocks up to limit, if they are > > > > > > continuous) > > > > > > - read single (read multiple registers is not supported) > > > > > > > > > > > > Some devices, especially older PLCs may also require extra delay > > > after > > > > > > data retrieval, so you have to wait between finishing one request > > and > > > > > > starting another. Some inverters and official dongles made by > > fairly > > > > > > large Chinese manufacturer, which brand I won't mention here, as > > well > > > > as > > > > > > primitive tcp/rtu bridges, may require "after connection delay" > in > > > > order > > > > > > to establish rtu connection they use to serve tcp session. > > > > > > > > > > > > All this makes modbus option list rather long, compared to > > abilities > > > it > > > > > > gives! > > > > > > > > > > > > Cheers, > > > > > > Łukasz > > > > > > > > > > > > > > > > > > On 15.01.2023 17:05, Niels Basjes wrote: > > > > > > > Hi, > > > > > > > > > > > > > > No, I'm reading many logical values where the needed registers > > > > combined > > > > > > > exceed the 125 mark. > > > > > > > Assume my application needs a set of values and asks for such a > > > block > > > > > of > > > > > > > registers. > > > > > > > Then the modbus library that gets the request for 200 registers > > can > > > > no > > > > > > > longer reliably split it into multiple parts that yield the > > correct > > > > > > value. > > > > > > > > > > > > > > What I understand of your S7 example (I do not know this > device) > > > > seems > > > > > > like > > > > > > > a device that offers a single value that exceeds the limits of > > > modbus > > > > > and > > > > > > > thus I expect they made it so you can fetch it step by step. > > > > > > > > > > > > > > I'll see if I can do a wireshark dump for you. > > > > > > > > > > > > > > What should I make the issue say? I currently do not see this > as > > a > > > > bug > > > > > or > > > > > > > problem with plc4x. > > > > > > > > > > > > > > Niels > > > > > > > > > > > > > > On Sun, Jan 15, 2023 at 4:46 PM Christofer Dutz < > > > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > >> Hi Niels, > > > > > > >> > > > > > > >> I think this might be related to what happens when reading > > Strings > > > > in > > > > > S7 > > > > > > >> without providing a length. > > > > > > >> In this case each string is 255 charaters long and it already > > > > exceeds > > > > > > the > > > > > > >> size-limit of 200 or so of a S7 1200. > > > > > > >> > > > > > > >> In S7 we use some code to split up multiple tags into multiple > > > > > requests, > > > > > > >> but rewriting the query to support automatic splitting of one > > > single > > > > > tag > > > > > > >> into multiple requests is quite a bit more difficult. > > > > > > >> > > > > > > >> I thought you were reading multiple tags that in sum exceeded > > the > > > > 125 > > > > > > >> registers and for that case simply something similar to the S7 > > > query > > > > > > >> updater should also work for other protocols. > > > > > > >> > > > > > > >> Do I understand you correctly, that in your case you want to > > read > > > > one > > > > > > tag, > > > > > > >> that is bigger? > > > > > > >> > > > > > > >> Chris > > > > > > >> > > > > > > >> From: Niels Basjes <[email protected]> > > > > > > >> Date: Sunday, 15. January 2023 at 16:38 > > > > > > >> To: [email protected] <[email protected]> > > > > > > >> Subject: [Modbus] Handling 'too large' blocks of registers > > > > > > >> Hi, > > > > > > >> > > > > > > >> A few weeks ago some code of mine was merged to limit the > number > > > of > > > > > > >> registers that can be requested in a single ModbusTag because > > > asking > > > > > for > > > > > > >> more than 125 registers in a single request will always fail > > > > (because > > > > > of > > > > > > >> the way modbus works). > > > > > > >> https://github.com/apache/plc4x/pull/721 > > > > > > >> > > > > > > >> One of the comments was asking if it can be automated in a > > generic > > > > way > > > > > > to > > > > > > >> split a "too large" request into multiple smaller requests. > > > > > > >> So If I ask for a block of 200 registers then plc4x would > simply > > > > split > > > > > > it > > > > > > >> into multiple requests (like 125 and 75 for example, or 100 > and > > > 100) > > > > > and > > > > > > >> afterwards merge the resulting registers of the two requests > > back > > > > > into a > > > > > > >> single block of 200 registers for the upstream application to > > > > analyze. > > > > > > >> > > > > > > >> I said I would look into this and this is what I found while > > doing > > > > > some > > > > > > >> experimenting with my real solar converter. > > > > > > >> This device uses the SunSpec standard for defining meaning to > > > modbus > > > > > > >> registers. > > > > > > >> > > > > > > >> My current conclusion at this point is that it is impossible > to > > > > handle > > > > > > this > > > > > > >> at the modbus level and I'm looking for you guys to challenge > me > > > in > > > > > > this. > > > > > > >> > > > > > > >> Why do I say this? > > > > > > >> One of the logical values you can retrieve is the name of the > > > > > > Manufacturer > > > > > > >> and the Model of the device that are both stored as UTF-8 in a > > set > > > > of > > > > > 16 > > > > > > >> registers (i.e. usually 32 characters). > > > > > > >> > > > https://github.com/sunspec/models/blob/master/json/model_1.json#L27 > > > > > > >> > > > https://github.com/sunspec/models/blob/master/json/model_1.json#L36 > > > > > > >> > > > > > > >> I found that if I try to fetch only a part of this logical > > value I > > > > get > > > > > > >> either an error (INVALID_ADDRESS) or bad data. > > > > > > >> When I fetch the entire value I get the 16 registers which > hold > > > the > > > > > > correct > > > > > > >> data. > > > > > > >> > > > > > > >> Output of my test program that fetches the Model of the device > > > > (which > > > > > > is in > > > > > > >> my case "SB3.6-1AV-41" ). > > > > > > >> ModbusTag[4x40021:UINT[16]]: 0x5342 0x332E 0x362D 0x3141 > 0x562D > > > > > 0x3431 > > > > > > >> 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 > > > > 0x0000 > > > > > > >> ModbusTag[4x40021:UINT[15]]: INVALID_ADDRESS > > > > > > >> ModbusTag[4x40022:UINT[15]]: 0xFFFF 0xFFFF 0xFFFF 0xFFFF > 0xFFFF > > > > > 0xFFFF > > > > > > >> 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF > > > > > > >> > > > > > > >> So if I retrieve the correct block I get it. > > > > > > >> If I try to get 1 register less than I should I get an error. > > > > > > >> If I try to get the registers starting from the 2nd register I > > get > > > > > only > > > > > > >> 0xFFFF (important: NO error and a PlcResponseCode.OK !!) > > > > > > >> > > > > > > >> I expect there to be many other modbus devices that have > similar > > > > > effects > > > > > > >> when the logical value requires multiple registers. > > > > > > >> > > > > > > >> So for now I come to the conclusion that fetching only part of > > the > > > > > > >> registers that make up a logical variable in general will not > > work > > > > and > > > > > > (in > > > > > > >> some cases) cannot be detected to have failed. > > > > > > >> > > > > > > >> At this point my assessment is that the only way to handle > this > > is > > > > by > > > > > > >> actually knowing the meaning of the block of registers (i.e. > > which > > > > > must > > > > > > be > > > > > > >> read together in a single read) and work from there. > > > > > > >> > > > > > > >> Note that the current SunSpec definition also has the concept > > of a > > > > > > 'sync' > > > > > > >> group of points (logical values that consist of 1 or more > > > registers) > > > > > > which > > > > > > >> are grouped in the specification with the intent of > "indicating > > > > > > >> that the points in the group must be read and written > > atomically", > > > > > which > > > > > > >> effectively means you cannot read them individually from the > > > modbus > > > > > > >> standpoint. > > > > > > >> > > > > > > >> Looking for your feedback on this. > > > > > > >> Do you know of existing software to generically handle this ? > > > > > > >> > > > > > > >> -- > > > > > > >> Best regards / Met vriendelijke groeten, > > > > > > >> > > > > > > >> Niels Basjes > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Best regards / Met vriendelijke groeten, > > > > > > > > > > Niels Basjes > > > > > > > > > > > > > > > > > > -- > > > Best regards / Met vriendelijke groeten, > > > > > > Niels Basjes > > > > > > > > -- > Best regards / Met vriendelijke groeten, > > Niels Basjes > -- Best regards / Met vriendelijke groeten, Niels Basjes
