Hi, I think it's a great idea.
I'd vote for having a separate project for something like this, just so it's clear that it is a historian and people can build it separately. I wouldn't want it hidden away in an obscure folder in the PLC4X repo. Cake does go with Cafe. Ben On Sun, Nov 27, 2022 at 3:09 AM Christofer Dutz <[email protected]> wrote: > Hi all, > > Trying to respond to all (except the naming question ;-) ): > > @Ryan: PLC4X Doesn’t have such a queueing, but IoTDB has … so we could be > collecting data on one node, writing that to TSFiles using the IoTDB-Client > (As Xaingdong mentioned). > These could either be online and transfer data to the server right away, > or operate in an offline-like mode and periodically fetch data in bursts > from the client (Hope that’s correct, please correct me if I’m wrong) > > @Xaiangdong: In general, PLC4X operates in an active mode, but we also > have a proposed and prototypically proven “passive-mode”. Here the software > doesn’t actively participate in the data acquisition directly, but simply > listens to the data stream and makes data available. We also have something > we call a “Data-Diode”, which technically makes interference impossible as > it lets ethernet packets flow in one direction, but nothing in the other > (Firewalls usually have the problem of letting packets pass in both > directions for established connections). We invented this, because we knew > we will never have validated and audited open-source software, to a level > that it would be certified for some of these use cases. > > So, either we can live with everything the SCADA system is already > requesting, or we add an active PLC4X node in the secure network, that > requests data, and simply ignores it, and a second – passive mode – node > sits outside the secure network to capture the information. > > I agree that this sort of thing needs to be a “product”. The Automation > Industry just doesn’t know how to work with frameworks. Ideally a one-box > thing. > Right now, most tools I have seen don’t even need the calculation or the > visualization. This is usually done on another level. Important would be > that we could be somewhat API compatible with existing products. Them > usually having SQL or REST APIs, should make it relatively easy to sort of > produce frontends to our Apache Historian, that are API compatible with > some existing industry products, so they are replaceable. > > If we wanted to add a visualization layer or a data-curation layer later > on, we should have a chat with the NiFi or StreamPipes folks as that’s what > they already have. > > For now, I would be proposing to build something that uses PLC4X for the > data-acquisition, IoTDB for the storage and build a REST frontend for this, > that’s somewhat API compatible with one of the major established products > and to add more on a step-by-step basis. > > I don’t even think it would be bad if an entity like Timecho would add > enterprise offerings, because I know, that the industry won’t “buy” > something, if there’s no commercial support or anyone, they can throw money > at, even if it’s free (That might even make them more skeptical). > > > What do you all think? Does this make sense? If yes, where would it make > sense to start working on something like this? In the IoTDB project? In the > PLC4X project, in a separate (new) project? > > > Chris > > > From: Xiangdong Huang <[email protected]> > Date: Sunday, 27. November 2022 at 06:02 > To: [email protected] <[email protected]> > Subject: Re: Building a PLC4X and IoTDB Historian? > Hi all, > > It makes sense to use PLC4X+IoTDB to build a historian software. > Actually, there are several users using IoTDB as their historian > solution... > > According to my knowledge, a historian software contains several features: > > - get data from OPC/modubs server and write to IoTDB > * use PLC4x + IoTDB-client. (we call this data collector) > > - (optional) there is a single-way network gateway for security. > * If the gateway is deployed between device and PLC4x program, then > we need to check whether plc4x supports that. > * If the gateway is deployed between IoTDB-client and IoTDB, then we > need to check wheter IoTDB-client supports that. > > - Calculation Engine, e.g., we want to write C to IoTDB when we > collect A and B from a device (for example, C=A+B/2 ) > * IoTDB's trigger supports that but we need a Drag-and-drop > programming GUI. (maybe integrating some open source projects, like > Red-node is a good idea) > > - get the latest data from IoTDB for visualization (usually users need > to draw a process flow diagram, and put the latest data on each device > icon on the diagram) > * IoTDB supports getting latest data. but we need a new software for > drawing the diagram (I do not know if there is any open source project > for this. In our real applications, we ususally buy and integrate some > other commerical software) > > - send alert message if the real time data meets some rules > * IoTDB trigger supports the rule. but currently we have no a GUI. > (IoTDB also supports Prometheus Alert Manager) > > - get the historical data > * IoTDB supports that and grafana is good at visualization > > - last but not the least, if we provide all the features to industrial > users, all the GUIs should be integrated into one entrance. > > > Best, > ----------------------------------- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > Ryan Truran <[email protected]> 于2022年11月27日周日 08:18写道： > > > > Hi folks, long time lurker… > > > > A historian would be great. Does PLC4X queue data on the machine it’s > > running on already? > > > > A common pattern in historians is to queue data on the runtime server > until > > data can be extracted to a sql server, preventing data loss, and reducing > > the need for redundancy. > > > > There are a ton of edge cases to watch out for which I can elaborate on > > further. > > > > I’m a former Industrial Controls Engineer turned software dev and can > help > > out where needed. > > > > -Ryan > > > > On Sat, Nov 26, 2022 at 10:24 AM Otto Fowler <[email protected]> > > wrote: > > > > > Would you name it “Cake”? > > > > > > From: Christofer Dutz <[email protected]> > > > <[email protected]> > > > Reply: [email protected] <[email protected]> < > [email protected]> > > > Date: November 26, 2022 at 10:36:23 > > > To: [email protected] <[email protected]> <[email protected] > >, > > > [email protected] <[email protected]> <[email protected]> > > > Subject: Building a PLC4X and IoTDB Historian? > > > > > > Hi all, > > > > > > I am currently thinking, if it wouldn’t be a good idea to build > something > > > like an open-source Historian based on PLC4X and IoTDB. > > > > > > For those of you, who don’t know what a Historian is. In the > manufacturing > > > industry they have these extremely expensive servers (usually you buy > them > > > as a bundle of hard- and software). > > > These servers are nothing else than a really crappy and brutally > expensive > > > Database for storing Time-Series data. > > > > > > The thing is most commercial products are currently really having > trouble > > > to keep up with the increasing amount of Data being sent. > > > > > > So instead of suggesting building an Historian at each customer’s > site, I > > > thought: Perhaps a ready-to-use solution based on open-source would be > a > > > good idea. > > > Not 100% sure where I’d locate such an initiative, but I would tend to > see > > > it more on the Database side. > > > > > > I think IoTDB would be the perfect storage system, all we seem to need > is > > > some sort of rest-interface that matches the industry standards for > > > querying the information and on the other side something like PLC4X to > fill > > > the database. > > > > > > Here some examples: > > > > > > > https://www.ge.com/digital/documentation/historian/version80/c_historian_apis_overview.html > > > > https://cdn.logic-control.com/docs/aveva/historian/HistorianRetrieval.pdf > > > > > > What do you folks think? > > > > > > > > > Chris > > > >
