I just went back to your original mail on this thread: >At this point my assessment is that the only way to handle this is by actually knowing the meaning of the block of registers (i.e. which must be read together in a single read) and work from there.
So, I believe you are right there. From: Otto Fowler <ottobackwa...@gmail.com> <ottobackwa...@gmail.com> Reply: Otto Fowler <ottobackwa...@gmail.com> <ottobackwa...@gmail.com> Date: January 16, 2023 at 21:34:42 To: dev@plc4x.apache.org <dev@plc4x.apache.org> <dev@plc4x.apache.org> Subject: Re: [Modbus] Handling 'too large' blocks of registers So, I actually don’t think we are in disagreement. If your server implements modbus only enough to service sunspec, it probably doesn’t have ‘true’ modbus tables / memory. IE IT may only work correctly for sunspec addresses. Then the issue isn’t with how the modbus driver works, since it is to the modbus protocol and not to a device that may only implement modbus in sunspec terms. From: Niels Basjes <ni...@basjes.nl> <ni...@basjes.nl> Reply: dev@plc4x.apache.org <dev@plc4x.apache.org> <dev@plc4x.apache.org> Date: January 16, 2023 at 15:28:50 To: dev@plc4x.apache.org <dev@plc4x.apache.org> <dev@plc4x.apache.org> Subject: Re: [Modbus] Handling 'too large' blocks of registers Hi, So here I disagree with you. SunSpec is nothing more than a standardized way of giving meaning to modbus registers for a certain class of devices. It is still standard modbus to fetch that data. > If I wanted to read 40000 to 410000 by chunks of 10, I could do it and ‘just make sense’ of the returned range. So on the device I have this is simply not true: I cannot simply read any set of registers and get the correct values. I provided a packet trace here https://github.com/apache/plc4x/issues/743 so you can see for yourself what I'm getting. This is the way the modbus server in this device has been written, I expect more devices to have similar effects. Niels On Mon, Jan 16, 2023 at 6:21 PM Otto Fowler <ottobackwa...@gmail.com> wrote: > I believe you are having issues because of sunspec over modbus and not > modbus ‘generically’ > > If you wanted to read modbus as just registers, memory ranges and chunk > that there is no reason that cannot be done. If I wanted to read 40000 to > 410000 by chunks of 10, I could do it and ‘just make sense’ of the returned > range. > > I don’t think we should conflate *BASE* modbus functionality with how > models or specs that ride on top of modbus operate. I think *would* more > likely a bug in how the sunspec layer is being evaluated. > > Maybe a sunspec specific derivative of the modbus spec would be better. > > for information on how sunspec access may look : > > https://github.com/nmakel/solaredge_modbus/blob/master/src/solaredge_modbus/__init__.py > > > > > From: Niels Basjes <ni...@basjes.nl> <ni...@basjes.nl> > Reply: dev@plc4x.apache.org <dev@plc4x.apache.org> <dev@plc4x.apache.org> > Date: January 16, 2023 at 03:21:42 > To: dev@plc4x.apache.org <dev@plc4x.apache.org> <dev@plc4x.apache.org> > Subject: Re: [Modbus] Handling 'too large' blocks of registers > > Hi, > > Yes this is exactly my idea as well. > Splitting a request for >125 registers into multiple requests cannot be > done in a generic reliable way. > If you know the schema (i.e. meaning of the registers) then you can combine > requests with multiple logical values into blocks of modbus registers that > stay within the 125 register limit. > > Which implies the need for a (generic) way of defining logical values and > the modbus registers from which they can be read (with all kinds of > constraints about reading them). > I would call this a modbus schema definition which can then be used to > calculate the actual read/write operations. > > The effect would be that for every type of device a schema definition is to > be created from which a client can be created/generated. > > Niels > > > > > > On Mon, Jan 16, 2023 at 8:50 AM Ben Hutcheson <ben.hut...@gmail.com> > wrote: > > > Hi Niels, > > > > As Łukasz pointed out, there are a thousand different combinations of how > > Modbus requests can be arranged. I'm not sure there's a generic schema > that > > can be used to define it though. > > > > You talked originally about splitting large reads (>125 registers) up > into > > smaller registers. This logic would live in a class that implements the > > BaseOptimizer. This is then defined within the ModbusTCPDriver class, we > > are just using the SingleTagOptimizer for Modbus at the moment. > > > > If I were to implement this I would probably firstly try and combine tags > > that are adjacent into one request until we either encounter a break in > the > > requested area or approach the limit of ~125 registers. This way it is > > guaranteed that requests for individual requests are not broken into > > smaller requests. > > > > Kind Regards > > > > Ben > > > > > > > > > > > > On Sun, Jan 15, 2023 at 6:58 PM Niels Basjes <ni...@basjes.nl> wrote: > > > > > Hi Łukasz, > > > > > > At this point I tend towards the need for a tool that allows one to > > define > > > the schema of a modbus device with all available values and code to > > convert > > > them into usable stuff (like in java an actual String or Double). > > > So far it seems this needs indeed flags indicating which values can and > > > must be read combined (like the sync block in SunSpec). > > > Then based on this a request for a set of functional values is > converted > > > into a set of modbus requests that follow these defined rules. > > > > > > Rules about an after connection delay and maximum requests per second > > seem > > > like something that should be at the modbus level (i.e. part of plc4x). > > > > > > Is there an existing project that does something like this? > > > > > > Niels > > > > > > On Sun, Jan 15, 2023 at 5:22 PM Łukasz Dywicki <l...@code-house.org> > > > wrote: > > > > > > > I am really glad you are moving it forward and fighting it! > > > > > > > > It is fairly difficult to get modbus code aligned and working > > > > automatically because different devices behave differently with even > > > > most basic requests. > > > > I remember that one of electric meters I had to work with did support > > > > read multiple registers but did not support going over "empty" > indexes. > > > > By empty I mean these which were not declared in the documentation, > > > > even if it was still below 125 limit. > > > > > > > > Effectively I could draw three primary options to cover: > > > > - read continuous (just keep expanding requests up to the limit) > > > > - read blocks (group indexes into blocks up to limit, if they are > > > > continuous) > > > > - read single (read multiple registers is not supported) > > > > > > > > Some devices, especially older PLCs may also require extra delay > after > > > > data retrieval, so you have to wait between finishing one request and > > > > starting another. Some inverters and official dongles made by fairly > > > > large Chinese manufacturer, which brand I won't mention here, as well > > as > > > > primitive tcp/rtu bridges, may require "after connection delay" in > > order > > > > to establish rtu connection they use to serve tcp session. > > > > > > > > All this makes modbus option list rather long, compared to abilities > it > > > > gives! > > > > > > > > Cheers, > > > > Łukasz > > > > > > > > > > > > On 15.01.2023 17:05, Niels Basjes wrote: > > > > > Hi, > > > > > > > > > > No, I'm reading many logical values where the needed registers > > combined > > > > > exceed the 125 mark. > > > > > Assume my application needs a set of values and asks for such a > block > > > of > > > > > registers. > > > > > Then the modbus library that gets the request for 200 registers can > > no > > > > > longer reliably split it into multiple parts that yield the correct > > > > value. > > > > > > > > > > What I understand of your S7 example (I do not know this device) > > seems > > > > like > > > > > a device that offers a single value that exceeds the limits of > modbus > > > and > > > > > thus I expect they made it so you can fetch it step by step. > > > > > > > > > > I'll see if I can do a wireshark dump for you. > > > > > > > > > > What should I make the issue say? I currently do not see this as a > > bug > > > or > > > > > problem with plc4x. > > > > > > > > > > Niels > > > > > > > > > > On Sun, Jan 15, 2023 at 4:46 PM Christofer Dutz < > > > > christofer.d...@c-ware.de> > > > > > wrote: > > > > > > > > > >> Hi Niels, > > > > >> > > > > >> I think this might be related to what happens when reading Strings > > in > > > S7 > > > > >> without providing a length. > > > > >> In this case each string is 255 charaters long and it already > > exceeds > > > > the > > > > >> size-limit of 200 or so of a S7 1200. > > > > >> > > > > >> In S7 we use some code to split up multiple tags into multiple > > > requests, > > > > >> but rewriting the query to support automatic splitting of one > single > > > tag > > > > >> into multiple requests is quite a bit more difficult. > > > > >> > > > > >> I thought you were reading multiple tags that in sum exceeded the > > 125 > > > > >> registers and for that case simply something similar to the S7 > query > > > > >> updater should also work for other protocols. > > > > >> > > > > >> Do I understand you correctly, that in your case you want to read > > one > > > > tag, > > > > >> that is bigger? > > > > >> > > > > >> Chris > > > > >> > > > > >> From: Niels Basjes <ni...@basjes.nl> > > > > >> Date: Sunday, 15. January 2023 at 16:38 > > > > >> To: dev@plc4x.apache.org <dev@plc4x.apache.org> > > > > >> Subject: [Modbus] Handling 'too large' blocks of registers > > > > >> Hi, > > > > >> > > > > >> A few weeks ago some code of mine was merged to limit the number > of > > > > >> registers that can be requested in a single ModbusTag because > asking > > > for > > > > >> more than 125 registers in a single request will always fail > > (because > > > of > > > > >> the way modbus works). > > > > >> https://github.com/apache/plc4x/pull/721 > > > > >> > > > > >> One of the comments was asking if it can be automated in a generic > > way > > > > to > > > > >> split a "too large" request into multiple smaller requests. > > > > >> So If I ask for a block of 200 registers then plc4x would simply > > split > > > > it > > > > >> into multiple requests (like 125 and 75 for example, or 100 and > 100) > > > and > > > > >> afterwards merge the resulting registers of the two requests back > > > into a > > > > >> single block of 200 registers for the upstream application to > > analyze. > > > > >> > > > > >> I said I would look into this and this is what I found while doing > > > some > > > > >> experimenting with my real solar converter. > > > > >> This device uses the SunSpec standard for defining meaning to > modbus > > > > >> registers. > > > > >> > > > > >> My current conclusion at this point is that it is impossible to > > handle > > > > this > > > > >> at the modbus level and I'm looking for you guys to challenge me > in > > > > this. > > > > >> > > > > >> Why do I say this? > > > > >> One of the logical values you can retrieve is the name of the > > > > Manufacturer > > > > >> and the Model of the device that are both stored as UTF-8 in a set > > of > > > 16 > > > > >> registers (i.e. usually 32 characters). > > > > >> > https://github.com/sunspec/models/blob/master/json/model_1.json#L27 > > > > >> > https://github.com/sunspec/models/blob/master/json/model_1.json#L36 > > > > >> > > > > >> I found that if I try to fetch only a part of this logical value I > > get > > > > >> either an error (INVALID_ADDRESS) or bad data. > > > > >> When I fetch the entire value I get the 16 registers which hold > the > > > > correct > > > > >> data. > > > > >> > > > > >> Output of my test program that fetches the Model of the device > > (which > > > > is in > > > > >> my case "SB3.6-1AV-41" ). > > > > >> ModbusTag[4x40021:UINT[16]]: 0x5342 0x332E 0x362D 0x3141 0x562D > > > 0x3431 > > > > >> 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 > > 0x0000 > > > > >> ModbusTag[4x40021:UINT[15]]: INVALID_ADDRESS > > > > >> ModbusTag[4x40022:UINT[15]]: 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF > > > 0xFFFF > > > > >> 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF 0xFFFF > > > > >> > > > > >> So if I retrieve the correct block I get it. > > > > >> If I try to get 1 register less than I should I get an error. > > > > >> If I try to get the registers starting from the 2nd register I get > > > only > > > > >> 0xFFFF (important: NO error and a PlcResponseCode.OK !!) > > > > >> > > > > >> I expect there to be many other modbus devices that have similar > > > effects > > > > >> when the logical value requires multiple registers. > > > > >> > > > > >> So for now I come to the conclusion that fetching only part of the > > > > >> registers that make up a logical variable in general will not work > > and > > > > (in > > > > >> some cases) cannot be detected to have failed. > > > > >> > > > > >> At this point my assessment is that the only way to handle this is > > by > > > > >> actually knowing the meaning of the block of registers (i.e. which > > > must > > > > be > > > > >> read together in a single read) and work from there. > > > > >> > > > > >> Note that the current SunSpec definition also has the concept of a > > > > 'sync' > > > > >> group of points (logical values that consist of 1 or more > registers) > > > > which > > > > >> are grouped in the specification with the intent of "indicating > > > > >> that the points in the group must be read and written atomically", > > > which > > > > >> effectively means you cannot read them individually from the > modbus > > > > >> standpoint. > > > > >> > > > > >> Looking for your feedback on this. > > > > >> Do you know of existing software to generically handle this ? > > > > >> > > > > >> -- > > > > >> Best regards / Met vriendelijke groeten, > > > > >> > > > > >> Niels Basjes > > > > >> > > > > > > > > > > > > > > > > > > > > > > > -- > > > Best regards / Met vriendelijke groeten, > > > > > > Niels Basjes > > > > > > > > -- br/>Best regards / Met vriendelijke groeten, < > > Niels Basjes > -- r/>Best regards / Met vriendelijke groeten, < Niels Basjes
