No problem, I think I understand what you mean… and I agree on the device level OPC UA is maybe too much / to fat (or not well implemented :-))
> On 10.03.2021, at 15:50, Christofer Dutz <[email protected]> wrote: > > Hi Andreas, > > I didn't want to upset you. I should have been a bit more clear on this: > > In 2018, when I was travelling through the country proclaiming PLC4X, all I > heard was: But OPC-UA can do all of this (Without having tested it) ... 2019 > most of the people who tried it were already looking for alternatives. End of > 2019 and beginning of 2020 was definitely the age of MQTT (Of course OPC UA > now provided a MQTT-like communication form). > > I think OPC-UA is great for machine-to-machine communication and stuff like > dashboards. But as soon as you want to do high volume data acquisition. > Especially for use cases like machine-learning etc. the processing power of > even modern PLCs can't handle the load. > > For example, at one of the biggest German car manufacturers, they were not > able to get more than 200 datapoints every 2 seconds from a new S7 400 PLC > via OPC-UA. These are probably the most powerful ones Siemens has to offer. > If they requested 300, the PLC shut down the IO process due to overload. With > PLC4X using the S7 protocol, we were able to get 2600 datapoints in 200ms > from the same machines. > > On the client side on one of the standard VMs they were using, running the > OPC-UA stack was able to collect data on 10 PLCs with 200 datapoints. Then > the CPU power was 100% ... in our case we run PLC4X on the same VMs using S7 > and there we could collect 200 PLCs with 2600 datapoints. So also on the > client side OPC-UA is a very fat project, that requires a lot of CPU time. > > I hope this explains things a bit more. If you are just reading a few > datapoints or using OPC-UA for Server-to-Server communication the servers > will be able to cope with more. And if it doesn't you just add more nodes to > the compute cluster. > > Chris > > > > > > -----Ursprüngliche Nachricht----- > Von: Andreas Vogler <[email protected]> > Gesendet: Mittwoch, 10. März 2021 15:38 > An: [email protected] > Betreff: Re: Source Time of PLC Value? > Priorität: Niedrig > > Just a side note: it hurts to hear that OPC UA sucks badly performance-wise. > I have done tests with Milo OPC UA Client and SCADA OPC UA servers, and they > perform pretty fine. 200 MQTT clients writing values via a Gateway with OPC > UA Milo (Client) to two OPC UA servers, each OPC UA server handles 50000 > values changes per second: > https://www.linkedin.com/posts/andreas-vogler-361115122_graphql-mqtt-vertx-activity-6769029730797719552-ZkjG > > <https://www.linkedin.com/posts/andreas-vogler-361115122_graphql-mqtt-vertx-activity-6769029730797719552-ZkjG> > > > But I fully agree with MQTT - it’s simple - and I think that’s the reason why > it is so popular… > > >> On 10.03.2021, at 14:34, Christofer Dutz <[email protected]> wrote: >> >> Yes, >> >> OPC-UA has all of this ... and it's probably one of the reasons so many >> implementations differ, and you have to again interpret results. It's also >> probably a reason why OPC-UA sucks that badly performance-wise. >> >> Just saying that we should be careful what we choose as reference. In >> contrast typical MQTT doesn't have all of this and it seems to be gaining >> more and more traction, just because it doesn't have all of this overhead. >> >> Just my thoughts on this. >> >> Chris >> >> >> >> >> -----Ursprüngliche Nachricht----- >> Von: Łukasz Dywicki <[email protected]> >> Gesendet: Mittwoch, 10. März 2021 12:58 >> An: [email protected] >> Betreff: Re: AW: Source Time of PLC Value? >> >> Matthias, >> I do believe there are several points which are relevant and still not >> possible to be implemented using available PLC4X API. Metadata might not be >> most fortunate way, but it can enable or ease analytical and troubleshooting >> scenarios. >> >> Two major points I see: >> - Verification of device condition - ie. how does caller know response >> latency? This is especially relevant with frequent polling. Currently caller >> has no measures to know that and can not measure that in reliable way due to >> async nature of API. >> There is a way to implement backpressure technics with callback chains (we >> can automatically delay execute call), but first we need to bring measures >> for that. >> - Caching of values - when we do cyclic subscriptions or emulation of these >> through passive connections. There is no way to make caller code aware of >> nature of data received. Unless he code everything on his end. >> - Per operation timeout settings/automatic retry are another functionalities >> which are relevant in case of querying and writing. We do not want every >> driver to implement it, so making a metadata and/or decorator would unify >> this layer between drivers. >> >> I speak of above from my own perspective cause I did implement an >> experimental decorator API for PLC4X [1] to cope with CANopen arbitration >> issues I found in devices we interface with. It allows to catch read/write >> failures [2] and re-attempt entire operation. It is proven to work [3] >> independently of driver. Sadly it has no measures to signal its execution >> back to caller. In the end, traceability is pretty tough. >> >> Obviously OPC UA has everything sorted out, but device specific protocols >> often do not. Even a limited set of metadata can give us a way to start >> bridging the gap and making Ben's Milo/OPC UA server much closer to a real >> thing. >> >> Best, >> Łukasz >> >> [1] >> https://github.com/ConnectorIO/connectorio-addons/tree/master/bundles/ >> org.connectorio.plc4x.decorator/src/main/java/org/connectorio/plc4x/de >> corator >> [2] >> https://github.com/ConnectorIO/connectorio-addons/blob/master/bundles/ >> org.connectorio.plc4x.decorator.retry/src/main/java/org/connectorio/pl >> c4x/decorator/retry/ReadRetry.java >> [3] >> https://github.com/ConnectorIO/connectorio-addons/blob/master/bundles/ >> org.connectorio.plc4x.decorator.retry/src/test/java/org/connectorio/pl >> c4x/decorator/retry/RetryDecoratorReadTest.java >> >> >> On 08.03.2021 23:15, Matthias Milan Strljic wrote: >>> Hi all, >>> >>> i see there a point of how useful this meta information could be in >>> some scenarios. But i would rather go the route of chris. Because I >>> see there is more of a risk of an overcomplicated inhomogeneous api >>> structure. I would not say that all API should have the same java >>> structure on each platform but if you have for all the different >>> platforms other featuresets you could also just use partially the >>> code generation attempt to allow a separate meta info layer. >>> >>> And there would be the question if that is worth the additional feature? >>> I mean I know we could add especially on the OPC UA side some meta >>> information but is there more than the actual source timestamp? And >>> if there is none, would it be not better to use it as the actual >>> timestamp of the value? >>> >>> >>> Einen schönen Abend ;) >>> Greetings Matthias >>> >>> >>> >>> Am Mo., 8. März 2021 um 14:29 Uhr schrieb Christofer Dutz < >>> [email protected]>: >>> >>>> Hi, >>>> >>>> I'm generally more concerned about users expecting us to deliver >>>> feature that driver X has for driver Y too. >>>> >>>> I won't object, if you think it's worth doing. >>>> >>>> Perhaps if you could whip up an example in a feature branch? I think >>>> perhaps I was still not understanding what you propose. >>>> >>>> Would that be ok? >>>> >>>> >>>> Chris >>>> >>>> >>>> -----Ursprüngliche Nachricht----- >>>> Von: Lukas Ott <[email protected]> >>>> Gesendet: Montag, 8. März 2021 14:20 >>>> An: [email protected] >>>> Betreff: Re: AW: Source Time of PLC Value? >>>> >>>> +1 to Lukasz >>>> >>>> Am Mo., 8. März 2021 um 14:09 Uhr schrieb Łukasz Dywicki < >>>> [email protected]>: >>>> >>>>> Hold on for a second. >>>>> >>>>> Making options pushed over connection string is a long term recipe >>>>> for disaster. Apache Camel is a prime example of what could happen >>>>> if you starting with configuration with URIs and dynamic parameters. >>>>> Most of components distinguish producer (writer) and consumer >>>>> (receiver) options, most complicated components ended up with 40+ >>>>> options settable via URI. Having a mix of PlcValues which are >>>>> timestamped and >>>> 'qualified' >>>>> and not will complicate encoding logic (how we push above info to >>>>> IEC encoder/handler or decorate it?) and how we keep custom types >>>>> compatible with above? >>>>> >>>>> That is why I rather opt for extending API in a way which does not >>>>> force existing drivers to do anything. When any of community >>>>> members, regardless if its developer or user, will find it >>>>> necessary to be ported in another driver, she or he have right to >>>>> do it. It is not only yours duty to keep all drivers and languages the >>>>> same. >>>>> >>>>> Speaking of which I am also not concerned about making all >>>>> languages having the same functionality as languages tend to offer >>>>> different options which might be hard to be ported. Lets stick to >>>>> basics which are same (connection strings, field definitions, >>>>> read/write/subscribe >>>>> syntax) and let various runtime lead their own ways toward >>>> implementation. >>>>> >>>>> Key point is simple - we won't be able to maintain all languages >>>>> equally. We should not hold moving ie. go or java forward because >>>>> we can't port new API functionality to C/Python/C#/whenever. If >>>>> there is someone who *does* use above in production and *wish* to >>>>> have new API parts then that person/organization duty is to do it. >>>>> >>>>> Best, >>>>> Łukasz >>>>> >>>>> >>>>> On 08.03.2021 13:17, Christofer Dutz wrote: >>>>>> Hi, >>>>>> >>>>>> I must admit that I would be in favor of keeping it as simple as >>>>> possible (max age in the connection string) and to implement more >>>>> of the missing parts in plc4x (like the subscription simulation >>>>> layer) and hereby getting the drivers we have a bit more aligned, >>>>> than to implement more and more special API features, that only one >>>>> or two drivers support. Cause this way we're running away form the >>>>> project-promise of running a given program at any type of PLC with >>>>> any >>>> protocol. >>>>>> >>>>>> I mean ... at the current velocity the project is underway, I >>>>>> wouldn't >>>>> really want to add a new MetaData layer that probably in the end >>>>> only >>>>> 1-2 people will be implementing and which we must not only find out >>>>> how to fill with life in the first place, but also port to all of >>>>> our supported languages. >>>>>> >>>>>> If someone's willing to bring in some serious man/womenpower, I'm >>>>>> fine >>>>> with that ... or we add it to some wish-list for future extensions. >>>>> But I wouldn't want to start something like this right now. >>>>>> >>>>>> Chris >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -----Ursprüngliche Nachricht----- >>>>>> Von: Łukasz Dywicki <[email protected]> >>>>>> Gesendet: Montag, 8. März 2021 13:07 >>>>>> An: [email protected] >>>>>> Betreff: Re: Source Time of PLC Value? >>>>>> >>>>>> I have not mentioned metadata term in my answer, so credits for >>>>>> mining >>>>> it API go to you. :-) >>>>>> >>>>>> Now since you brought it I remember that JDBC has a "result set >>>>> metadata". That might be thing we still miss. The result code is >>>>> most important and available instantly, other information is >>>>> supplemental and can be read additionally. >>>>>> >>>>>> >>>>>> Best, >>>>>> Łukasz >>>>>> >>>>>> On 08.03.2021 13:01, Ben Hutcheson wrote: >>>>>>> +1 for Łukasz‘s metadata approach, it would avoid having to >>>>>>> +create >>>>>>> duplicate PLcValue classes and give us a lot of flexibility in >>>>>>> the >>>>> future. >>>>>>> >>>>>>> On Mon, Mar 8, 2021 at 5:14 AM Christofer Dutz >>>>>>> <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> sorry for being late to the party ... KNX is currently consuming >>>>>>>> all my cycles. >>>>>>>> Just wanted to add my thoughts to the discussion. >>>>>>>> >>>>>>>> Initially I thought about adding this sort of information to the >>>>>>>> API, but then I thought that we have so little protocols >>>>>>>> supporting this sort of concept that blowing up every PLCValue >>>>>>>> with this time information just would waste CPU time and Memory. >>>>>>>> >>>>>>>> When it comes to simulating subscriptions by polling in the >>>>>>>> background, I thought it would be a valid solution to provide a >>>>>>>> default age a value can have in the cache and to make it >>>>>>>> possible to override this in the connection string options. If a >>>>>>>> value is too old, >>>>> a return code of: >>>>>>>> STALE_DATA could be returned. >>>>>>>> >>>>>>>> Also, we do have an option of adding an additional set of >>>>>>>> PlcValue >>>>> types. >>>>>>>> If a driver supports this quality-of-service type of data, we >>>>>>>> could return special versions of PlcValues, that have these >>>> properties? >>>>>>>> >>>>>>>> This way we don't have to waste the CPU time and Memory for >>>>>>>> protocols that don't support this concept (We could even >>>>>>>> implement the methods on the default PlvValues to simply return >>>>>>>> some constants to the PlcValues all have the same API. >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>>> >>>>>>>> -----Ursprüngliche Nachricht----- >>>>>>>> Von: Łukasz Dywicki <[email protected]> >>>>>>>> Gesendet: Montag, 8. März 2021 10:43 >>>>>>>> An: [email protected] >>>>>>>> Betreff: Re: Source Time of PLC Value? >>>>>>>> >>>>>>>> Looking at issue we could utilize marker interfaces. >>>>>>>> At least in Java we could define Qualified and Timestamped types. >>>>>>>> >>>>>>>> Yet looking at present state of API we are unlikely to make each >>>>>>>> and every PlcValue in each and every variant (qualified and >>>>>>>> timestamped, timestamped, qualified). It would force quite a lot >>>>>>>> of >>>> wrapping. >>>>>>>> Safest option I see is to extend subscription/response API with >>>>>>>> additional method to retrieve all markers. For most of drivers >>>>>>>> it would return just an empty set. It leaves us also an open >>>>>>>> door for future drivers (or devices) to ship additional piece of >>>>>>>> metadata which >>>>> does not fit into present API. >>>>>>>> >>>>>>>> Best, >>>>>>>> Łukasz >>>>>>>> >>>>>>>> >>>>>>>> On 08.03.2021 04:30, Otto Fowler wrote: >>>>>>>>> So, maybe the extension is timestamp *and* timestamp source. >>>>>>>>> >>>>>>>>> uint64_t timestamp; >>>>>>>>> typedef enum { >>>>>>>>> /* timestamp was generated by the source device */ >>>>>>>>> SOURCE, >>>>>>>>> /* timestamp was generated by plc4x on receiving */ >>>>>>>>> GENERATED, >>>>>>>>> } TimestampQuality; >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Mar 5, 2021, at 18:04, Ben Hutcheson <[email protected]> >>>>> wrote: >>>>>>>>>> >>>>>>>>>> We do have some error codes, but it would just need to be >>>>>>>>>> extended a bit. I don't think we have one for BAD (we still >>>>>>>>>> receive data but the source has marked it bad). >>>>>>>>>> >>>>>>>>>> Timestamps I think are definitely a good idea, especially if >>>>>>>>>> we eventually DNP3. >>>>>>>>>> >>>>>>>>>> On Fri, Mar 5, 2021 at 5:39 PM Łukasz Dywicki >>>>>>>>>> <[email protected]> >>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> You have response code for each individual field which allows >>>>>>>>>>> you to determine state (OK, REMOTE_ERROR etc.). Same >>>>>>>>>>> information should be available also for subscriptions. >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Łukasz >>>>>>>>>>> >>>>>>>>>>> On 05.03.2021 21:04, Andreas Vogler wrote: >>>>>>>>>>>> Ah, forgot: and is there a status available? >>>>>>>>>>>> Valid/invalid/good/bad >>>>>>>>>>> indicator of a plc value? >>>>>>>>>>>> >>>>>>>>>>>>> On 05.03.2021, at 21:02, Andreas Vogler >>>>>>>>>>>>> <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> How can I get the source time of a PlcValue? >>>>>>>>>>>>> >>>>>>>>>>>>> a) from a subscription PlcSubscriptionEvent >>>>>>>>>>>>> b) from a read request PlcReadResponse >>>>>>>>>>>>> >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Andreas >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>> >
