Hi all, Trying to respond to all (except the naming question ;-) ):
@Ryan: PLC4X Doesn’t have such a queueing, but IoTDB has … so we could be collecting data on one node, writing that to TSFiles using the IoTDB-Client (As Xaingdong mentioned). These could either be online and transfer data to the server right away, or operate in an offline-like mode and periodically fetch data in bursts from the client (Hope that’s correct, please correct me if I’m wrong) @Xaiangdong: In general, PLC4X operates in an active mode, but we also have a proposed and prototypically proven “passive-mode”. Here the software doesn’t actively participate in the data acquisition directly, but simply listens to the data stream and makes data available. We also have something we call a “Data-Diode”, which technically makes interference impossible as it lets ethernet packets flow in one direction, but nothing in the other (Firewalls usually have the problem of letting packets pass in both directions for established connections). We invented this, because we knew we will never have validated and audited open-source software, to a level that it would be certified for some of these use cases. So, either we can live with everything the SCADA system is already requesting, or we add an active PLC4X node in the secure network, that requests data, and simply ignores it, and a second �C passive mode �C node sits outside the secure network to capture the information. I agree that this sort of thing needs to be a “product”. The Automation Industry just doesn’t know how to work with frameworks. Ideally a one-box thing. Right now, most tools I have seen don’t even need the calculation or the visualization. This is usually done on another level. Important would be that we could be somewhat API compatible with existing products. Them usually having SQL or REST APIs, should make it relatively easy to sort of produce frontends to our Apache Historian, that are API compatible with some existing industry products, so they are replaceable. If we wanted to add a visualization layer or a data-curation layer later on, we should have a chat with the NiFi or StreamPipes folks as that’s what they already have. For now, I would be proposing to build something that uses PLC4X for the data-acquisition, IoTDB for the storage and build a REST frontend for this, that’s somewhat API compatible with one of the major established products and to add more on a step-by-step basis. I don’t even think it would be bad if an entity like Timecho would add enterprise offerings, because I know, that the industry won’t “buy” something, if there’s no commercial support or anyone, they can throw money at, even if it’s free (That might even make them more skeptical). What do you all think? Does this make sense? If yes, where would it make sense to start working on something like this? In the IoTDB project? In the PLC4X project, in a separate (new) project? Chris From: Xiangdong Huang <[email protected]> Date: Sunday, 27. November 2022 at 06:02 To: [email protected] <[email protected]> Subject: Re: Building a PLC4X and IoTDB Historian? Hi all, It makes sense to use PLC4X+IoTDB to build a historian software. Actually, there are several users using IoTDB as their historian solution... According to my knowledge, a historian software contains several features: - get data from OPC/modubs server and write to IoTDB * use PLC4x + IoTDB-client. (we call this data collector) - (optional) there is a single-way network gateway for security. * If the gateway is deployed between device and PLC4x program, then we need to check whether plc4x supports that. * If the gateway is deployed between IoTDB-client and IoTDB, then we need to check wheter IoTDB-client supports that. - Calculation Engine, e.g., we want to write C to IoTDB when we collect A and B from a device (for example, C=A+B/2 ) * IoTDB's trigger supports that but we need a Drag-and-drop programming GUI. (maybe integrating some open source projects, like Red-node is a good idea) - get the latest data from IoTDB for visualization (usually users need to draw a process flow diagram, and put the latest data on each device icon on the diagram) * IoTDB supports getting latest data. but we need a new software for drawing the diagram (I do not know if there is any open source project for this. In our real applications, we ususally buy and integrate some other commerical software) - send alert message if the real time data meets some rules * IoTDB trigger supports the rule. but currently we have no a GUI. (IoTDB also supports Prometheus Alert Manager) - get the historical data * IoTDB supports that and grafana is good at visualization - last but not the least, if we provide all the features to industrial users, all the GUIs should be integrated into one entrance. Best, ----------------------------------- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Ryan Truran <[email protected]> 于2022年11月27日周日 08:18写道： > > Hi folks, long time lurker… > > A historian would be great. Does PLC4X queue data on the machine it’s > running on already? > > A common pattern in historians is to queue data on the runtime server until > data can be extracted to a sql server, preventing data loss, and reducing > the need for redundancy. > > There are a ton of edge cases to watch out for which I can elaborate on > further. > > I’m a former Industrial Controls Engineer turned software dev and can help > out where needed. > > -Ryan > > On Sat, Nov 26, 2022 at 10:24 AM Otto Fowler <[email protected]> > wrote: > > > Would you name it “Cake”? > > > > From: Christofer Dutz <[email protected]> > > <[email protected]> > > Reply: [email protected] <[email protected]> <[email protected]> > > Date: November 26, 2022 at 10:36:23 > > To: [email protected] <[email protected]> <[email protected]>, > > [email protected] <[email protected]> <[email protected]> > > Subject: Building a PLC4X and IoTDB Historian? > > > > Hi all, > > > > I am currently thinking, if it wouldn’t be a good idea to build something > > like an open-source Historian based on PLC4X and IoTDB. > > > > For those of you, who don’t know what a Historian is. In the manufacturing > > industry they have these extremely expensive servers (usually you buy them > > as a bundle of hard- and software). > > These servers are nothing else than a really crappy and brutally expensive > > Database for storing Time-Series data. > > > > The thing is most commercial products are currently really having trouble > > to keep up with the increasing amount of Data being sent. > > > > So instead of suggesting building an Historian at each customer’s site, I > > thought: Perhaps a ready-to-use solution based on open-source would be a > > good idea. > > Not 100% sure where I’d locate such an initiative, but I would tend to see > > it more on the Database side. > > > > I think IoTDB would be the perfect storage system, all we seem to need is > > some sort of rest-interface that matches the industry standards for > > querying the information and on the other side something like PLC4X to fill > > the database. > > > > Here some examples: > > > > https://www.ge.com/digital/documentation/historian/version80/c_historian_apis_overview.html > > https://cdn.logic-control.com/docs/aveva/historian/HistorianRetrieval.pdf > > > > What do you folks think? > > > > > > Chris > >
