I would make it IOT Historian. Historian is too generic at the apache level And that name fits: IOT DB, IOT Historian, IOT Visualize, IOT XXXX
On Mon, Nov 28, 2022 at 7:45 AM Christofer Dutz <[email protected]> wrote: > And if we already agree on a name (I think “Apache Historian” would be a > good name), > we could even already work on code with that package-name and maven > coordinates, so we don’t have to change much when coming to Apache? > > Chris > > From: Christofer Dutz <[email protected]> > Date: Monday, 28. November 2022 at 13:18 > To: [email protected] <[email protected]> > Subject: Re: Building a PLC4X and IoTDB Historian? > Hi Julian, > > Indeed, your concerns are valid regarding creating an already abandoned > TLP. > > In that case, I would opt for a private repo on GitHub which we invite > everyone willing to help work on it and as soon as we have something > workable, we make it public as part of a TLP? > > This avoids at least the fear of creating a new TLP that’s product doesn’t > even see the light of day. And as soon as we have something we want to show > the world, we join Apache? > > Would that be a compromise? > > > Chris > > From: Julian Feinauer <[email protected]> > Date: Monday, 28. November 2022 at 13:14 > To: [email protected] <[email protected]> > Subject: AW: Building a PLC4X and IoTDB Historian? > Hey, > > as I said.. we could just do it as part of one of the TLPs like PLC4X or > IoTDB easily or in a private repo as well, for sure. > I just want to avoid to create a TLP which is soon abandoned, that’s my > only concern. > Regarding all your comments I totally agree and am with you. > > Julian > > Von: Christofer Dutz <[email protected]> > Datum: Montag, 28. November 2022 um 13:07 > An: [email protected] <[email protected]> > Betreff: Re: Building a PLC4X and IoTDB Historian? > Hi Julian, > > well … I would be hesitant to try something like this without the legal > shield of the ASF … from what I’ve learned in the early days, Siemens and > Co. continuously threatened to sue me for my work on PLC4X. > The only thing that helped me then, was to say: I am doing everything > legally, so you’d have trouble succeeding with that and you also can’t sue > me but would have to sue the ASF and good luck on that not backfiring badly > on you publicly. > > I know that in the sector of drivers in the past there were several > occasions where open-source developers were annoyed so much by nuisance > law-suites, that they gave up. > > I also know Historians are silly expensive systems. I haven’t seen one > under 100k€ and the annual fees for certified solutions tend to start at > 100k€/year/production-line … so there’s a lot of money for them to fear > losing. So yes: I’m too scared to try something like this publicly without > a legal shield. > > We could however work on something like this in a protected repo at GitHub > etc. and go to TLP as soon as we have something. > > Would that be an option? > > Chris > > > From: Julian Feinauer <[email protected]> > Date: Monday, 28. November 2022 at 12:57 > To: [email protected] <[email protected]> > Subject: AW: Building a PLC4X and IoTDB Historian? > Hey Chris, > > wrt testing I meant how „sustainable” the community is, that we intend to > building. > You know the effect pretty well if you ask “How wants to start this” > everybody is like “me, me, me” and after 2 weeks you stand there alone. > And one important aspect of a TLP for me is that the community is also > “resilient” and will last “long” (whatever that means). > And this is something I think is hard to guarantee in the current state. > > Technically I have no worries, this is something we would get to work > rather easily, I see no “magics” hidden there. > > Julian > > Von: Christofer Dutz <[email protected]> > Datum: Montag, 28. November 2022 um 12:48 > An: [email protected] <[email protected]> > Betreff: Re: Building a PLC4X and IoTDB Historian? > Hi Julian, > > Well, the problem with starting at one Project as a subproject … We then > need to make everyone involved or wanting to be involved in the other > project to become committers in the other. > I wouldn’t have problems with doing it under the hood of IoTDB, personally. > > A separate Project would have the benefit of giving the industry an > impression that this is something built just for them instead of being a > side-product of something that’s already there (I have given up expecting > the Automation industry to use any form of common sense ;-) … guess that > will be my definition of Industry 5.0 … “connected everything + common > sense”) > > The incubator is intended for teaching the initial community how things > work at Apache, I guess as we’d be starting with mainly people that have > already brought projects from outside into the incubator and into TLPs, > this teaching effort will not exist in this case. Even Justin mentioned > that this would probably be a case for straight TLP. > > Regarding the testing … not sure I understand what you want to test? The > idea of an open-source historian? > I mean the need exists … I’ve seen it in multiple occasions and our > friends from IoTDB have confirmed they have built or at least seen IoTDB > serve as such a system. > Not 100% sure what you want to try out. > > But I agree … we should probably start with an (online) workshop, as this > time we’d probably be spread out a bit wider geographically, than with our > PLC4X workshops in the past. > > Chris > > > > From: Julian Feinauer <[email protected]> > Date: Monday, 28. November 2022 at 12:37 > To: [email protected] <[email protected]> > Subject: AW: Building a PLC4X and IoTDB Historian? > Hey, > > I understand your point and also agree generally. > But I personally dislike the straight to TLP route as this is something > I’d like to try out first. > And my point with the separate repo (or whatever) was just to make the > community ready for the Incubator (where a codebase and community normally > should exist). > > But the easiest way in any regard would be to start of as a subproject in > an existing PMC. This would only require to get another repo and we could > start (formally no need to even register it as a subproject). > And the repo thing could be done by you alone I think as Chair. > > Next steps would then be more of brainstorming and workshoping and > building something (whatever?). > I think the timing is quite good before Christmas because if its > intereting, I see myself contributing over the Holidays (better don’t show > this mail to my wife…). > > Best > Julian > > Von: Christofer Dutz <[email protected]> > Datum: Montag, 28. November 2022 um 09:27 > An: [email protected] <[email protected]> > Cc: Tim Mitsch <[email protected]>, [email protected] < > [email protected]> > Betreff: Re: Building a PLC4X and IoTDB Historian? > Hi Julian, > > happy you like the idea ;-) > > We would probably outclass the established ones in point of price ;-) > Right now, I think we should probably set up something that bundles IoTDB > and PLC4X and then the “code of the project” is the bubble-wrap, that > brings the two together and then provides an API that is equivalent to > existing Historian products. > I think the only way we can score points here, is if switching to it is as > easy as possible and that only works if it’s sort of a drop-in replacement. > > Currently also discussing with other board members and the IPMC chair … he > at least would not see us in the incubator. If there are enough PMC members > of other projects and Apache Members on board, we could take the straight > to TLP route. > > And I would really like to keep it at Apache and not set up something > outside. This for multiple reasons: > > * Protection (The market for the established products is huge and the > companies behind them are also huge with huge legal departments, I would > fear the same as I did with PLC4X, when I started it … don’t want to be > stuck in nuisance lawsuits, just filed for keeping us occupied) > * The satement: we want Open-Source to be accepted by the industry and > for me “Apache” is the form of Open-Source that I admire most. If it’s > built up out of Apache stuff, it should be an Apache project (If we > establish this in a way, that the industry notices Apache as a “vendor”, > this will benefit many other projects) > > Chris > > > > From: Julian Feinauer <[email protected]> > Date: Monday, 28. November 2022 at 09:17 > To: [email protected] <[email protected]> > Cc: Tim Mitsch <[email protected]>, [email protected] < > [email protected]> > Subject: AW: Building a PLC4X and IoTDB Historian? > Hi all, > > first of all.. sorry for coming in so late. > I think this is a great idea and there really is a product market fit for > such a solution. > I think technology wise its not "that complicated" to build such a system, > I think many of the details that are necessary for a good adoption of such > a software is in the areas around like documentation, marketing, more > marketing, even more and even bolder marketing and sales (how to sell an > open source project??). > > But technology wise I really like the idea and I think with the technology > one has today it is very easy to "outclass" these established systems e.g. > in performance, efficiency and als user friendliness. > The only thing which might be challenging for the future are things like > plugins or an extension system to allow users to customize their > installation. > > Regarding the "separate project" approach I am not 100% certain. > Personally, I would consider starting EITHER as a subproject in an > existing PMC (PLC4X?) or as a separate undertaking on GitHub or somewhere > else and not directly go to the Incubator or something. > Because I think we should really find out if there is enough developer > interest to build and sustain such a system. > > But I'm totally in for such a system either way! > > Julian > > PS.: Also forwarding the email to Björn and Tim who work(ed) a lot with > historians > > Von: Christofer Dutz <[email protected]> > Datum: Montag, 28. November 2022 um 08:56 > An: [email protected] <[email protected]> > Betreff: Re: Building a PLC4X and IoTDB Historian? > Hi Xiangdong, > > that’s perfect that some of you folks would be on board. I should probably > check, if there are enough PLC4X folks on board too … would be a shame if > it was just me ;-) > > And regarding the name … yeah … you are absolutely right: Historian might > really be the ideal choice … as it instantly explains what it is … A > Historian from the Apache Software Foundation and probably also no bias > towards any regional culture. > > So (Speaking to my fellow PLC4X folks … who would be on board with this?) > > > Chris > > > > From: Xiangdong Huang <[email protected]> > Date: Monday, 28. November 2022 at 04:24 > To: [email protected] <[email protected]> > Subject: Re: Building a PLC4X and IoTDB Historian? > > I guess I would sort of also vote for a separate Project. Would however > only make sense if some people from both of our projects would join in. > > Agree, I think the iotdb developers in Timecho can join. > > > And name-wise … one of the names of Historic Historians (Ideally a Greek > one) would make it into my top 10 ;-) … sort of like Hudson or Jenkins are > “famous” Buttler names ;-) > > If we consider famous names (well, there will be culture bias), I'd > like to suggest this one: https://en.wikipedia.org/wiki/Sima_Qian , as > "Records of the Grand Historian" is tooo famous (at least in China). > > BTW. naming "Apache Historian" directly may be another option. > > Best, > ----------------------------------- > Xiangdong Huang > School of Software, Tsinghua University > > > Christofer Dutz <[email protected]> 于2022年11月27日周日 22:56写道： > > > > How about Apache Ephorous? ;-) > > > > https://en.wikipedia.org/wiki/Ephorus > > Ephorus of Cyme (/ˈɛfərəs/< > https://en.wikipedia.org/wiki/Help:IPA/English>; Greek< > https://en.wikipedia.org/wiki/Greek_language>: Ἔφορος ὁ Κυμαῖος, Ephoros > ho Kymaios; c. 400 – 330 BC) was an ancient Greek< > https://en.wikipedia.org/wiki/Ancient_Greece> historian< > https://en.wikipedia.org/wiki/Historian> known for his universal history< > https://en.wikipedia.org/wiki/Universal_history>. > > > > Chris > > > > From: Christofer Dutz <[email protected]> > > Date: Sunday, 27. November 2022 at 15:28 > > To: [email protected] <[email protected]> > > Subject: Re: Building a PLC4X and IoTDB Historian? > > Hi all, > > > > I guess I would sort of also vote for a separate Project. Would however > only make sense if some people from both of our projects would join in. > > > > And name-wise … one of the names of Historic Historians (Ideally a Greek > one) would make it into my top 10 ;-) … sort of like Hudson or Jenkins are > “famous” Buttler names ;-) > > https://en.wikipedia.org/wiki/List_of_Greek_historiographers > > https://en.wikipedia.org/wiki/List_of_historians > > > > I think anything but a turnkey-ready solution will not be accepted by > anyone in the Automation industry. > > > > Chris > > > > > > > > > > From: Xiangdong Huang <[email protected]> > > Date: Sunday, 27. November 2022 at 14:05 > > To: [email protected] <[email protected]> > > Subject: Re: Building a PLC4X and IoTDB Historian? > > If we just provide a solution, or demonstration, then either is ok. > > If we want to provide an "one-box thing" (even without GUI), +1 for a > > new project. > > > > > I don’t even think it would be bad if an entity like Timecho would add > enterprise offerings, because I know, that the industry won’t “buy” > something, if there’s no commercial support or anyone, they can throw money > at, even if it’s free (That might even make them more skeptical). > > Yes, indeed. :D > > > > Best, > > ----------------------------------- > > Xiangdong Huang > > > > Ben Hutcheson <[email protected]> 于2022年11月27日周日 18:38写道： > > > > > > Hi, > > > > > > I think it's a great idea. > > > > > > I'd vote for having a separate project for something like this, just so > > > it's clear that it is a historian and people can build it separately. I > > > wouldn't want it hidden away in an obscure folder in the PLC4X repo. > > > > > > Cake does go with Cafe. > > > > > > Ben > > > > > > > > > On Sun, Nov 27, 2022 at 3:09 AM Christofer Dutz < > [email protected]> > > > wrote: > > > > > > > Hi all, > > > > > > > > Trying to respond to all (except the naming question ;-) ): > > > > > > > > @Ryan: PLC4X Doesn’t have such a queueing, but IoTDB has … so we > could be > > > > collecting data on one node, writing that to TSFiles using the > IoTDB-Client > > > > (As Xaingdong mentioned). > > > > These could either be online and transfer data to the server right > away, > > > > or operate in an offline-like mode and periodically fetch data in > bursts > > > > from the client (Hope that’s correct, please correct me if I’m wrong) > > > > > > > > @Xaiangdong: In general, PLC4X operates in an active mode, but we > also > > > > have a proposed and prototypically proven “passive-mode”. Here the > software > > > > doesn’t actively participate in the data acquisition directly, but > simply > > > > listens to the data stream and makes data available. We also have > something > > > > we call a “Data-Diode”, which technically makes interference > impossible as > > > > it lets ethernet packets flow in one direction, but nothing in the > other > > > > (Firewalls usually have the problem of letting packets pass in both > > > > directions for established connections). We invented this, because > we knew > > > > we will never have validated and audited open-source software, to a > level > > > > that it would be certified for some of these use cases. > > > > > > > > So, either we can live with everything the SCADA system is already > > > > requesting, or we add an active PLC4X node in the secure network, > that > > > > requests data, and simply ignores it, and a second – passive mode – > node > > > > sits outside the secure network to capture the information. > > > > > > > > I agree that this sort of thing needs to be a “product”. The > Automation > > > > Industry just doesn’t know how to work with frameworks. Ideally a > one-box > > > > thing. > > > > Right now, most tools I have seen don’t even need the calculation or > the > > > > visualization. This is usually done on another level. Important > would be > > > > that we could be somewhat API compatible with existing products. Them > > > > usually having SQL or REST APIs, should make it relatively easy to > sort of > > > > produce frontends to our Apache Historian, that are API compatible > with > > > > some existing industry products, so they are replaceable. > > > > > > > > If we wanted to add a visualization layer or a data-curation layer > later > > > > on, we should have a chat with the NiFi or StreamPipes folks as > that’s what > > > > they already have. > > > > > > > > For now, I would be proposing to build something that uses PLC4X for > the > > > > data-acquisition, IoTDB for the storage and build a REST frontend > for this, > > > > that’s somewhat API compatible with one of the major established > products > > > > and to add more on a step-by-step basis. > > > > > > > > I don’t even think it would be bad if an entity like Timecho would > add > > > > enterprise offerings, because I know, that the industry won’t “buy” > > > > something, if there’s no commercial support or anyone, they can > throw money > > > > at, even if it’s free (That might even make them more skeptical). > > > > > > > > > > > > What do you all think? Does this make sense? If yes, where would it > make > > > > sense to start working on something like this? In the IoTDB project? > In the > > > > PLC4X project, in a separate (new) project? > > > > > > > > > > > > Chris > > > > > > > > > > > > From: Xiangdong Huang <[email protected]> > > > > Date: Sunday, 27. November 2022 at 06:02 > > > > To: [email protected] <[email protected]> > > > > Subject: Re: Building a PLC4X and IoTDB Historian? > > > > Hi all, > > > > > > > > It makes sense to use PLC4X+IoTDB to build a historian software. > > > > Actually, there are several users using IoTDB as their historian > > > > solution... > > > > > > > > According to my knowledge, a historian software contains several > features: > > > > > > > > - get data from OPC/modubs server and write to IoTDB > > > > * use PLC4x + IoTDB-client. (we call this data collector) > > > > > > > > - (optional) there is a single-way network gateway for security. > > > > * If the gateway is deployed between device and PLC4x program, then > > > > we need to check whether plc4x supports that. > > > > * If the gateway is deployed between IoTDB-client and IoTDB, then > we > > > > need to check wheter IoTDB-client supports that. > > > > > > > > - Calculation Engine, e.g., we want to write C to IoTDB when we > > > > collect A and B from a device (for example, C=A+B/2 ) > > > > * IoTDB's trigger supports that but we need a Drag-and-drop > > > > programming GUI. (maybe integrating some open source projects, like > > > > Red-node is a good idea) > > > > > > > > - get the latest data from IoTDB for visualization (usually users > need > > > > to draw a process flow diagram, and put the latest data on each > device > > > > icon on the diagram) > > > > * IoTDB supports getting latest data. but we need a new software > for > > > > drawing the diagram (I do not know if there is any open source > project > > > > for this. In our real applications, we ususally buy and integrate > some > > > > other commerical software) > > > > > > > > - send alert message if the real time data meets some rules > > > > * IoTDB trigger supports the rule. but currently we have no a GUI. > > > > (IoTDB also supports Prometheus Alert Manager) > > > > > > > > - get the historical data > > > > * IoTDB supports that and grafana is good at visualization > > > > > > > > - last but not the least, if we provide all the features to > industrial > > > > users, all the GUIs should be integrated into one entrance. > > > > > > > > > > > > Best, > > > > ----------------------------------- > > > > Xiangdong Huang > > > > School of Software, Tsinghua University > > > > > > > > 黄向东 > > > > 清华大学 软件学院 > > > > > > > > Ryan Truran <[email protected]> 于2022年11月27日周日 08:18写道： > > > > > > > > > > Hi folks, long time lurker… > > > > > > > > > > A historian would be great. Does PLC4X queue data on the machine > it’s > > > > > running on already? > > > > > > > > > > A common pattern in historians is to queue data on the runtime > server > > > > until > > > > > data can be extracted to a sql server, preventing data loss, and > reducing > > > > > the need for redundancy. > > > > > > > > > > There are a ton of edge cases to watch out for which I can > elaborate on > > > > > further. > > > > > > > > > > I’m a former Industrial Controls Engineer turned software dev and > can > > > > help > > > > > out where needed. > > > > > > > > > > -Ryan > > > > > > > > > > On Sat, Nov 26, 2022 at 10:24 AM Otto Fowler < > [email protected]> > > > > > wrote: > > > > > > > > > > > Would you name it “Cake”? > > > > > > > > > > > > From: Christofer Dutz <[email protected]> > > > > > > <[email protected]> > > > > > > Reply: [email protected] <[email protected]> < > > > > [email protected]> > > > > > > Date: November 26, 2022 at 10:36:23 > > > > > > To: [email protected] <[email protected]> < > [email protected] > > > > >, > > > > > > [email protected] <[email protected]> < > [email protected]> > > > > > > Subject: Building a PLC4X and IoTDB Historian? > > > > > > > > > > > > Hi all, > > > > > > > > > > > > I am currently thinking, if it wouldn’t be a good idea to build > > > > something > > > > > > like an open-source Historian based on PLC4X and IoTDB. > > > > > > > > > > > > For those of you, who don’t know what a Historian is. In the > > > > manufacturing > > > > > > industry they have these extremely expensive servers (usually > you buy > > > > them > > > > > > as a bundle of hard- and software). > > > > > > These servers are nothing else than a really crappy and brutally > > > > expensive > > > > > > Database for storing Time-Series data. > > > > > > > > > > > > The thing is most commercial products are currently really having > > > > trouble > > > > > > to keep up with the increasing amount of Data being sent. > > > > > > > > > > > > So instead of suggesting building an Historian at each customer’s > > > > site, I > > > > > > thought: Perhaps a ready-to-use solution based on open-source > would be > > > > a > > > > > > good idea. > > > > > > Not 100% sure where I’d locate such an initiative, but I would > tend to > > > > see > > > > > > it more on the Database side. > > > > > > > > > > > > I think IoTDB would be the perfect storage system, all we seem > to need > > > > is > > > > > > some sort of rest-interface that matches the industry standards > for > > > > > > querying the information and on the other side something like > PLC4X to > > > > fill > > > > > > the database. > > > > > > > > > > > > Here some examples: > > > > > > > > > > > > > > > > > https://www.ge.com/digital/documentation/historian/version80/c_historian_apis_overview.html > > > > > > > > > > > https://cdn.logic-control.com/docs/aveva/historian/HistorianRetrieval.pdf > > > > > > > > > > > > What do you folks think? > > > > > > > > > > > > > > > > > > Chris > > > > > > > > > > >
