> If we get something to work in private we could easily go TLP or through the > Incubator whatever is decided then to do. +1 ----------------------------------- Xiangdong Huang School of Software, Tsinghua University
Niklas Merz <[email protected]> 于2022年11月28日周一 21:08写道： > > Hello all, > > I think this is a good compromise to get started and then publish it > with a bit of fanfare (marketing) as an Apache project to the world once > it's ready. > > I am also interested in working with you on this project. I think PLC4X > and IoTDB need something like this to become popular in the automation > world. > > Regards > Niklas > > On November 28, 2022, Julian Feinauer > <[email protected]> wrote: > > I fully agree with that. > > If we get something to work in private we could easily go TLP or > > through the Incubator whatever is decided then to do. > > > > Julian > > > > PS.: Do you setup the repo and invite myself? > > > > Von: Christofer Dutz <[email protected]> > > Datum: Montag, 28. November 2022 um 13:47 > > An: [email protected] <[email protected]> > > Betreff: Re: Building a PLC4X and IoTDB Historian? > > And if we already agree on a name (I think “Apache Historian” would be > > a good name), > > we could even already work on code with that package-name and maven > > coordinates, so we don’t have to change much when coming to Apache? > > > > Chris > > > > From: Christofer Dutz <[email protected]> > > Date: Monday, 28. November 2022 at 13:18 > > To: [email protected] <[email protected]> > > Subject: Re: Building a PLC4X and IoTDB Historian? > > Hi Julian, > > > > Indeed, your concerns are valid regarding creating an already > > abandoned TLP. > > > > In that case, I would opt for a private repo on GitHub which we invite > > everyone willing to help work on it and as soon as we have something > > workable, we make it public as part of a TLP? > > > > This avoids at least the fear of creating a new TLP that’s product > > doesn’t even see the light of day. And as soon as we have something we > > want to show the world, we join Apache? > > > > Would that be a compromise? > > > > > > Chris > > > > From: Julian Feinauer <[email protected]> > > Date: Monday, 28. November 2022 at 13:14 > > To: [email protected] <[email protected]> > > Subject: AW: Building a PLC4X and IoTDB Historian? > > Hey, > > > > as I said.. we could just do it as part of one of the TLPs like PLC4X > > or IoTDB easily or in a private repo as well, for sure. > > I just want to avoid to create a TLP which is soon abandoned, that’s > > my only concern. > > Regarding all your comments I totally agree and am with you. > > > > Julian > > > > Von: Christofer Dutz <[email protected]> > > Datum: Montag, 28. November 2022 um 13:07 > > An: [email protected] <[email protected]> > > Betreff: Re: Building a PLC4X and IoTDB Historian? > > Hi Julian, > > > > well … I would be hesitant to try something like this without the > > legal shield of the ASF … from what I’ve learned in the early days, > > Siemens and Co. continuously threatened to sue me for my work on > > PLC4X. > > The only thing that helped me then, was to say: I am doing everything > > legally, so you’d have trouble succeeding with that and you also can’t > > sue me but would have to sue the ASF and good luck on that not > > backfiring badly on you publicly. > > > > I know that in the sector of drivers in the past there were several > > occasions where open-source developers were annoyed so much by > > nuisance law-suites, that they gave up. > > > > I also know Historians are silly expensive systems. I haven’t seen one > > under 100k€ and the annual fees for certified solutions tend to start > > at 100k€/year/production-line … so there’s a lot of money for them to > > fear losing. So yes: I’m too scared to try something like this > > publicly without a legal shield. > > > > We could however work on something like this in a protected repo at > > GitHub etc. and go to TLP as soon as we have something. > > > > Would that be an option? > > > > Chris > > > > > > From: Julian Feinauer <[email protected]> > > Date: Monday, 28. November 2022 at 12:57 > > To: [email protected] <[email protected]> > > Subject: AW: Building a PLC4X and IoTDB Historian? > > Hey Chris, > > > > wrt testing I meant how „sustainable” the community is, that we intend > > to building. > > You know the effect pretty well if you ask “How wants to start this” > > everybody is like “me, me, me” and after 2 weeks you stand there > > alone. > > And one important aspect of a TLP for me is that the community is also > > “resilient” and will last “long” (whatever that means). > > And this is something I think is hard to guarantee in the current > > state. > > > > Technically I have no worries, this is something we would get to work > > rather easily, I see no “magics” hidden there. > > > > Julian > > > > Von: Christofer Dutz <[email protected]> > > Datum: Montag, 28. November 2022 um 12:48 > > An: [email protected] <[email protected]> > > Betreff: Re: Building a PLC4X and IoTDB Historian? > > Hi Julian, > > > > Well, the problem with starting at one Project as a subproject … We > > then need to make everyone involved or wanting to be involved in the > > other project to become committers in the other. > > I wouldn’t have problems with doing it under the hood of IoTDB, > > personally. > > > > A separate Project would have the benefit of giving the industry an > > impression that this is something built just for them instead of being > > a side-product of something that’s already there (I have given up > > expecting the Automation industry to use any form of common sense ;-) > > … guess that will be my definition of Industry 5.0 … “connected > > everything + common sense”) > > > > The incubator is intended for teaching the initial community how > > things work at Apache, I guess as we’d be starting with mainly people > > that have already brought projects from outside into the incubator and > > into TLPs, this teaching effort will not exist in this case. Even > > Justin mentioned that this would probably be a case for straight TLP. > > > > Regarding the testing … not sure I understand what you want to test? > > The idea of an open-source historian? > > I mean the need exists … I’ve seen it in multiple occasions and our > > friends from IoTDB have confirmed they have built or at least seen > > IoTDB serve as such a system. > > Not 100% sure what you want to try out. > > > > But I agree … we should probably start with an (online) workshop, as > > this time we’d probably be spread out a bit wider geographically, than > > with our PLC4X workshops in the past. > > > > Chris > > > > > > > > From: Julian Feinauer <[email protected]> > > Date: Monday, 28. November 2022 at 12:37 > > To: [email protected] <[email protected]> > > Subject: AW: Building a PLC4X and IoTDB Historian? > > Hey, > > > > I understand your point and also agree generally. > > But I personally dislike the straight to TLP route as this is > > something I’d like to try out first. > > And my point with the separate repo (or whatever) was just to make the > > community ready for the Incubator (where a codebase and community > > normally should exist). > > > > But the easiest way in any regard would be to start of as a subproject > > in an existing PMC. This would only require to get another repo and we > > could start (formally no need to even register it as a subproject). > > And the repo thing could be done by you alone I think as Chair. > > > > Next steps would then be more of brainstorming and workshoping and > > building something (whatever?). > > I think the timing is quite good before Christmas because if its > > intereting, I see myself contributing over the Holidays (better don’t > > show this mail to my wife…). > > > > Best > > Julian > > > > Von: Christofer Dutz <[email protected]> > > Datum: Montag, 28. November 2022 um 09:27 > > An: [email protected] <[email protected]> > > Cc: Tim Mitsch <[email protected]>, [email protected] > > <[email protected]> > > Betreff: Re: Building a PLC4X and IoTDB Historian? > > Hi Julian, > > > > happy you like the idea ;-) > > > > We would probably outclass the established ones in point of price ;-) > > Right now, I think we should probably set up something that bundles > > IoTDB and PLC4X and then the “code of the project” is the bubble-wrap, > > that brings the two together and then provides an API that is > > equivalent to existing Historian products. > > I think the only way we can score points here, is if switching to it > > is as easy as possible and that only works if it’s sort of a drop-in > > replacement. > > > > Currently also discussing with other board members and the IPMC chair > > … he at least would not see us in the incubator. If there are enough > > PMC members of other projects and Apache Members on board, we could > > take the straight to TLP route. > > > > And I would really like to keep it at Apache and not set up something > > outside. This for multiple reasons: > > > > * Protection (The market for the established products is huge and the > > companies behind them are also huge with huge legal departments, I > > would fear the same as I did with PLC4X, when I started it … don’t > > want to be stuck in nuisance lawsuits, just filed for keeping us > > occupied) > > * The satement: we want Open-Source to be accepted by the industry > > and for me “Apache” is the form of Open-Source that I admire most. If > > it’s built up out of Apache stuff, it should be an Apache project (If > > we establish this in a way, that the industry notices Apache as a > > “vendor”, this will benefit many other projects) > > > > Chris > > > > > > > > From: Julian Feinauer <[email protected]> > > Date: Monday, 28. November 2022 at 09:17 > > To: [email protected] <[email protected]> > > Cc: Tim Mitsch <[email protected]>, [email protected] > > <[email protected]> > > Subject: AW: Building a PLC4X and IoTDB Historian? > > Hi all, > > > > first of all.. sorry for coming in so late. > > I think this is a great idea and there really is a product market fit > > for such a solution. > > I think technology wise its not "that complicated" to build such a > > system, I think many of the details that are necessary for a good > > adoption of such a software is in the areas around like documentation, > > marketing, more marketing, even more and even bolder marketing and > > sales (how to sell an open source project??). > > > > But technology wise I really like the idea and I think with the > > technology one has today it is very easy to "outclass" these > > established systems e.g. in performance, efficiency and als user > > friendliness. > > The only thing which might be challenging for the future are things > > like plugins or an extension system to allow users to customize their > > installation. > > > > Regarding the "separate project" approach I am not 100% certain. > > Personally, I would consider starting EITHER as a subproject in an > > existing PMC (PLC4X?) or as a separate undertaking on GitHub or > > somewhere else and not directly go to the Incubator or something. > > Because I think we should really find out if there is enough developer > > interest to build and sustain such a system. > > > > But I'm totally in for such a system either way! > > > > Julian > > > > PS.: Also forwarding the email to Björn and Tim who work(ed) a lot > > with historians > > > > Von: Christofer Dutz <[email protected]> > > Datum: Montag, 28. November 2022 um 08:56 > > An: [email protected] <[email protected]> > > Betreff: Re: Building a PLC4X and IoTDB Historian? > > Hi Xiangdong, > > > > that’s perfect that some of you folks would be on board. I should > > probably check, if there are enough PLC4X folks on board too … would > > be a shame if it was just me ;-) > > > > And regarding the name … yeah … you are absolutely right: Historian > > might really be the ideal choice … as it instantly explains what it is > > … A Historian from the Apache Software Foundation and probably also no > > bias towards any regional culture. > > > > So (Speaking to my fellow PLC4X folks … who would be on board with > > this?) > > > > > > Chris > > > > > > > > From: Xiangdong Huang <[email protected]> > > Date: Monday, 28. November 2022 at 04:24 > > To: [email protected] <[email protected]> > > Subject: Re: Building a PLC4X and IoTDB Historian? > > > I guess I would sort of also vote for a separate Project. Would > > however only make sense if some people from both of our projects would > > join in. > > > > Agree, I think the iotdb developers in Timecho can join. > > > > > And name-wise … one of the names of Historic Historians (Ideally a > > Greek one) would make it into my top 10 ;-) … sort of like Hudson or > > Jenkins are “famous” Buttler names ;-) > > > > If we consider famous names (well, there will be culture bias), I'd > > like to suggest this one: https://en.wikipedia.org/wiki/Sima_Qian , as > > "Records of the Grand Historian" is tooo famous (at least in China). > > > > BTW. naming "Apache Historian" directly may be another option. > > > > Best, > > ----------------------------------- > > Xiangdong Huang > > School of Software, Tsinghua University > > > > > > Christofer Dutz <[email protected]> 于2022年11月27日周日 22:56写道： > > > > > > How about Apache Ephorous? ;-) > > > > > > https://en.wikipedia.org/wiki/Ephorus > > > Ephorus of Cyme > > (/ˈɛfərəs/<https://en.wikipedia.org/wiki/Help:IPA/English>; > > Greek<https://en.wikipedia.org/wiki/Greek_language>: Ἔφορος ὁ Κυμαῖος, > > Ephoros ho Kymaios; c. 400 – 330 BC) was an ancient > > Greek<https://en.wikipedia.org/wiki/Ancient_Greece> > > historian<https://en.wikipedia.org/wiki/Historian> known for his > > universal history<https://en.wikipedia.org/wiki/Universal_history>. > > > > > > Chris > > > > > > From: Christofer Dutz <[email protected]> > > > Date: Sunday, 27. November 2022 at 15:28 > > > To: [email protected] <[email protected]> > > > Subject: Re: Building a PLC4X and IoTDB Historian? > > > Hi all, > > > > > > I guess I would sort of also vote for a separate Project. Would > > however only make sense if some people from both of our projects would > > join in. > > > > > > And name-wise … one of the names of Historic Historians (Ideally a > > Greek one) would make it into my top 10 ;-) … sort of like Hudson or > > Jenkins are “famous” Buttler names ;-) > > > https://en.wikipedia.org/wiki/List_of_Greek_historiographers > > > https://en.wikipedia.org/wiki/List_of_historians > > > > > > I think anything but a turnkey-ready solution will not be accepted > > by anyone in the Automation industry. > > > > > > Chris > > > > > > > > > > > > > > > From: Xiangdong Huang <[email protected]> > > > Date: Sunday, 27. November 2022 at 14:05 > > > To: [email protected] <[email protected]> > > > Subject: Re: Building a PLC4X and IoTDB Historian? > > > If we just provide a solution, or demonstration, then either is ok. > > > If we want to provide an "one-box thing" (even without GUI), +1 for > > a > > > new project. > > > > > > > I don’t even think it would be bad if an entity like Timecho would > > add enterprise offerings, because I know, that the industry won’t > > “buy” something, if there’s no commercial support or anyone, they can > > throw money at, even if it’s free (That might even make them more > > skeptical). > > > Yes, indeed. :D > > > > > > Best, > > > ----------------------------------- > > > Xiangdong Huang > > > > > > Ben Hutcheson <[email protected]> 于2022年11月27日周日 18:38写道： > > > > > > > > Hi, > > > > > > > > I think it's a great idea. > > > > > > > > I'd vote for having a separate project for something like this, > > just so > > > > it's clear that it is a historian and people can build it > > separately. I > > > > wouldn't want it hidden away in an obscure folder in the PLC4X > > repo. > > > > > > > > Cake does go with Cafe. > > > > > > > > Ben > > > > > > > > > > > > On Sun, Nov 27, 2022 at 3:09 AM Christofer Dutz > > <[email protected]> > > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > Trying to respond to all (except the naming question ;-) ): > > > > > > > > > > @Ryan: PLC4X Doesn’t have such a queueing, but IoTDB has … so we > > could be > > > > > collecting data on one node, writing that to TSFiles using the > > IoTDB-Client > > > > > (As Xaingdong mentioned). > > > > > These could either be online and transfer data to the server > > right away, > > > > > or operate in an offline-like mode and periodically fetch data > > in bursts > > > > > from the client (Hope that’s correct, please correct me if I’m > > wrong) > > > > > > > > > > @Xaiangdong: In general, PLC4X operates in an active mode, but > > we also > > > > > have a proposed and prototypically proven “passive-mode”. Here > > the software > > > > > doesn’t actively participate in the data acquisition directly, > > but simply > > > > > listens to the data stream and makes data available. We also > > have something > > > > > we call a “Data-Diode”, which technically makes interference > > impossible as > > > > > it lets ethernet packets flow in one direction, but nothing in > > the other > > > > > (Firewalls usually have the problem of letting packets pass in > > both > > > > > directions for established connections). We invented this, > > because we knew > > > > > we will never have validated and audited open-source software, > > to a level > > > > > that it would be certified for some of these use cases. > > > > > > > > > > So, either we can live with everything the SCADA system is > > already > > > > > requesting, or we add an active PLC4X node in the secure > > network, that > > > > > requests data, and simply ignores it, and a second – passive > > mode – node > > > > > sits outside the secure network to capture the information. > > > > > > > > > > I agree that this sort of thing needs to be a “product”. The > > Automation > > > > > Industry just doesn’t know how to work with frameworks. Ideally > > a one-box > > > > > thing. > > > > > Right now, most tools I have seen don’t even need the > > calculation or the > > > > > visualization. This is usually done on another level. Important > > would be > > > > > that we could be somewhat API compatible with existing products. > > Them > > > > > usually having SQL or REST APIs, should make it relatively easy > > to sort of > > > > > produce frontends to our Apache Historian, that are API > > compatible with > > > > > some existing industry products, so they are replaceable. > > > > > > > > > > If we wanted to add a visualization layer or a data-curation > > layer later > > > > > on, we should have a chat with the NiFi or StreamPipes folks as > > that’s what > > > > > they already have. > > > > > > > > > > For now, I would be proposing to build something that uses PLC4X > > for the > > > > > data-acquisition, IoTDB for the storage and build a REST > > frontend for this, > > > > > that’s somewhat API compatible with one of the major established > > products > > > > > and to add more on a step-by-step basis. > > > > > > > > > > I don’t even think it would be bad if an entity like Timecho > > would add > > > > > enterprise offerings, because I know, that the industry won’t > > “buy” > > > > > something, if there’s no commercial support or anyone, they can > > throw money > > > > > at, even if it’s free (That might even make them more > > skeptical). > > > > > > > > > > > > > > > What do you all think? Does this make sense? If yes, where would > > it make > > > > > sense to start working on something like this? In the IoTDB > > project? In the > > > > > PLC4X project, in a separate (new) project? > > > > > > > > > > > > > > > Chris > > > > > > > > > > > > > > > From: Xiangdong Huang <[email protected]> > > > > > Date: Sunday, 27. November 2022 at 06:02 > > > > > To: [email protected] <[email protected]> > > > > > Subject: Re: Building a PLC4X and IoTDB Historian? > > > > > Hi all, > > > > > > > > > > It makes sense to use PLC4X+IoTDB to build a historian software. > > > > > Actually, there are several users using IoTDB as their historian > > > > > solution... > > > > > > > > > > According to my knowledge, a historian software contains several > > features: > > > > > > > > > > - get data from OPC/modubs server and write to IoTDB > > > > > * use PLC4x + IoTDB-client. (we call this data collector) > > > > > > > > > > - (optional) there is a single-way network gateway for security. > > > > > * If the gateway is deployed between device and PLC4x program, > > then > > > > > we need to check whether plc4x supports that. > > > > > * If the gateway is deployed between IoTDB-client and IoTDB, > > then we > > > > > need to check wheter IoTDB-client supports that. > > > > > > > > > > - Calculation Engine, e.g., we want to write C to IoTDB when we > > > > > collect A and B from a device (for example, C=A+B/2 ) > > > > > * IoTDB's trigger supports that but we need a Drag-and-drop > > > > > programming GUI. (maybe integrating some open source projects, > > like > > > > > Red-node is a good idea) > > > > > > > > > > - get the latest data from IoTDB for visualization (usually > > users need > > > > > to draw a process flow diagram, and put the latest data on each > > device > > > > > icon on the diagram) > > > > > * IoTDB supports getting latest data. but we need a new software > > for > > > > > drawing the diagram (I do not know if there is any open source > > project > > > > > for this. In our real applications, we ususally buy and > > integrate some > > > > > other commerical software) > > > > > > > > > > - send alert message if the real time data meets some rules > > > > > * IoTDB trigger supports the rule. but currently we have no a > > GUI. > > > > > (IoTDB also supports Prometheus Alert Manager) > > > > > > > > > > - get the historical data > > > > > * IoTDB supports that and grafana is good at visualization > > > > > > > > > > - last but not the least, if we provide all the features to > > industrial > > > > > users, all the GUIs should be integrated into one entrance. > > > > > > > > > > > > > > > Best, > > > > > ----------------------------------- > > > > > Xiangdong Huang > > > > > School of Software, Tsinghua University > > > > > > > > > > 黄向东 > > > > > 清华大学 软件学院 > > > > > > > > > > Ryan Truran <[email protected]> 于2022年11月27日周日 08:18写道： > > > > > > > > > > > > Hi folks, long time lurker… > > > > > > > > > > > > A historian would be great. Does PLC4X queue data on the > > machine it’s > > > > > > running on already? > > > > > > > > > > > > A common pattern in historians is to queue data on the runtime > > server > > > > > until > > > > > > data can be extracted to a sql server, preventing data loss, > > and reducing > > > > > > the need for redundancy. > > > > > > > > > > > > There are a ton of edge cases to watch out for which I can > > elaborate on > > > > > > further. > > > > > > > > > > > > I’m a former Industrial Controls Engineer turned software dev > > and can > > > > > help > > > > > > out where needed. > > > > > > > > > > > > -Ryan > > > > > > > > > > > > On Sat, Nov 26, 2022 at 10:24 AM Otto Fowler > > <[email protected]> > > > > > > wrote: > > > > > > > > > > > > > Would you name it “Cake”? > > > > > > > > > > > > > > From: Christofer Dutz <[email protected]> > > > > > > > <[email protected]> > > > > > > > Reply: [email protected] <[email protected]> < > > > > > [email protected]> > > > > > > > Date: November 26, 2022 at 10:36:23 > > > > > > > To: [email protected] <[email protected]> > > <[email protected] > > > > > >, > > > > > > > [email protected] <[email protected]> > > <[email protected]> > > > > > > > Subject: Building a PLC4X and IoTDB Historian? > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > I am currently thinking, if it wouldn’t be a good idea to > > build > > > > > something > > > > > > > like an open-source Historian based on PLC4X and IoTDB. > > > > > > > > > > > > > > For those of you, who don’t know what a Historian is. In the > > > > > manufacturing > > > > > > > industry they have these extremely expensive servers > > (usually you buy > > > > > them > > > > > > > as a bundle of hard- and software). > > > > > > > These servers are nothing else than a really crappy and > > brutally > > > > > expensive > > > > > > > Database for storing Time-Series data. > > > > > > > > > > > > > > The thing is most commercial products are currently really > > having > > > > > trouble > > > > > > > to keep up with the increasing amount of Data being sent. > > > > > > > > > > > > > > So instead of suggesting building an Historian at each > > customer’s > > > > > site, I > > > > > > > thought: Perhaps a ready-to-use solution based on open- > > source would be > > > > > a > > > > > > > good idea. > > > > > > > Not 100% sure where I’d locate such an initiative, but I > > would tend to > > > > > see > > > > > > > it more on the Database side. > > > > > > > > > > > > > > I think IoTDB would be the perfect storage system, all we > > seem to need > > > > > is > > > > > > > some sort of rest-interface that matches the industry > > standards for > > > > > > > querying the information and on the other side something > > like PLC4X to > > > > > fill > > > > > > > the database. > > > > > > > > > > > > > > Here some examples: > > > > > > > > > > > > > > > > > > > > > https://www.ge.com/digital/documentation/historian/version80/c_historian_apis_overview.html > > > > > > > > > > > > https://cdn.logic- > > control.com/docs/aveva/historian/HistorianRetrieval.pdf > > > > > > > > > > > > > > What do you folks think? > > > > > > > > > > > > > > > > > > > > > Chris > > > > > > > > > > > >
