Sorry, I’ve been crushed by IRL work, I’ll review this as soon as possible
From: Iñigo Angulo <[email protected]> <[email protected]> Reply: [email protected] <[email protected]> <[email protected]> Date: August 19, 2021 at 02:42:02 To: dev <[email protected]> <[email protected]> Cc: Gustavo Fernandez <[email protected]> <[email protected]> Subject: Re: AW: AW: Nifi integration record oriented processor for reading Hi Chris, I also had the confirmation from Apache that my ICLA is filed. My username is inigoao. Our collegue Alvaro has also sent it, I think in few days he will receive the confirmation. His username is asellart. regards, iñigo ----------------------------------------- br/>Iñigo Anngulo br/> < ZYLK.net :: consultoría.openSource br/>telf.: 7474123337 br/>Ribera de Axpe, 11 br/>Edificio A, modulo 201-203 br/>48950 Erandio (Bizkaia) br/>----------------------------------------- < ----- Mensaje original ----- De: "Christofer Dutz" <[email protected]> Para: "dev" <[email protected]>, "Gustavo Fernandez" <[email protected]> Enviados: Miércoles, 18 de Agosto 2021 14:28:00 Asunto: AW: AW: Nifi integration record oriented processor for reading Hi Gustavo, great ... I had something in the back of my mind, but I wasn't sure anymore ... then we're safe on that side ;-) Chris -----Ursprüngliche Nachricht----- Von: Gustavo Fernandez <[email protected]> br/>GGesendet: Mittwoch, 18. August 2021 12:32 An: [email protected] Betreff: Re: AW: Nifi integration record oriented processor for reading Hi Chris I sent my ICLA a month ago. The response from ASF was: [...] This message acknowledges receipt of your ICLA, which has been filed in the Apache Software Foundation records. With this message, the PLC4X PMC has been notified that your ICLA has been filed. [...] My apache user id is zgus un saludo --gustavo ----------------------------------------- Gustavo Fernández ZYLK.net :: consultoría.openSource Ribera de Axpe, 11 Edificio A, modulo 201-203 48950 Erandio (Bizkaia) movil: +34 637546184 ofic.: +34 944272119 email: gus at zylk.net skype: gus.zylk http://twitter.com/zylknet http://www.zylk.net/web/guest/web-2-0/blog ----------------------------------------- ----- Mensaje original ----- > De: "Christofer Dutz" <[email protected]> > Para: [email protected] > Enviados: Miércoles, 18 de Agosto 2021 10:49:43 > Asunto: AW: Nifi integration record oriented processor for reading > HI Inigo, > br/>> great news ... I hope Otto can spare some cyclees to review the PR from br/>> the NiFFi side ... Happy to assist with merging things back. > br/>> As it's been quite some time ... I am pretty suure we have an ICLA on br/>> file from your side, correct?? > br/>> Chris < > br/>> br/>> -----Ursprüngliche Nachricht----- < > Von: Iñigo Angulo <[email protected]> > Gesendet: Mittwoch, 18. August 2021 10:46 > An: dev <[email protected]> > Betreff: Re: Nifi integration record oriented processor for reading > br/>> Hi Otto, Chris, < > br/>> We have gone over the comments on the Nifi PR and made a commit br/>> including revisions for all (or most) toppics. > br/>> the last changes we did were related to reforciing the methods for br/>> datatype conversion (for the record scheema construction), and br/>> extending the tests. About this last one, we changed the br/>> org.apache.plc4x.nifi.BasePlc4xProcessoor class to protected. This led br/>> to renaming the test classess package to match BasePlc4xProcessor's one br/>> (""org.apache.plc4x.processors.plc4x4nifi" package was renamed to br/>> ""org.apache.plc4x.nifi"). This last commit caused a conflict on the PR, but we think it can be easily corrected when doing the merge. > br/>> At the moment, we think this nifi feature is prretty much completed. Of br/>> course there are things that can bee extended yet, which we will continue working on. > So please, if you could take a look and check if you miss something br/>> would be great. Hopefully this could be soon tested by users in the br/>> community too and get some feedback :) < > br/>> thank you < > iñigo > br/>> ----------------------------------------- < > Iñigo Angulo > br/>> ZYLK.net :: consultoría.openSource < > telf.: 747412337 > Ribera de Axpe, 11 > Edificio A, modulo 201-203 > 48950 Erandio (Bizkaia) > ----------------------------------------- > br/>> ----- Mensaje original ----- < > De: "Iñigo Angulo" <[email protected]> > Para: "dev" <[email protected]> > Enviados: Lunes, 5 de Julio 2021 12:38:14 > Asunto: Re: Nifi integration record oriented processor for reading > br/>> Hi Otto, < > br/>> thank you for the review of the PR and your coomments. > br/>> We have answered them, and will review the secttions you suggested. I br/>> would like to know if there are any tthings you want us to priorize. br/>> Maybe you have some deadlinees for the next release or something, and br/>> we could try to geet things done by this time. If you miss some task we br/>> could do here before doing the merge, please let us know. > br/>> iñigo < > br/>> ----------------------------------------- < > Iñigo Angulo > br/>> ZYLK.net :: consultoría.openSource < > telf.: 747412337 > Ribera de Axpe, 11 > Edificio A, modulo 201-203 > 48950 Erandio (Bizkaia) > ----------------------------------------- > br/>> ----- Mensaje original ----- < > De: "ottobackwards" <[email protected]> > Para: "dev" <[email protected]> > Enviados: Jueves, 24 de Junio 2021 14:40:12 > Asunto: Re: Nifi integration record oriented processor for reading > br/>> Great, < > I’m doing the review, so you should mark it ready :) > br/>> br/>> br/>> FFrom: Iñigo Angulo <[email protected]> br/>> <iangulo@@zylk.net.invalid> > Reply: [email protected] <[email protected]> br/>> <dev@@ plc4x.apache.org> > Date: June 24, 2021 at 04:52:19 > To: dev <[email protected]> <[email protected]> > Subject: Re: Nifi integration record oriented processor for reading > br/>> Hi Otto, < > br/>> I have opened the new pull request, from a deddicated branch in our br/>> fork, and a commit per contributor, thhe team members that have been working on this. > br/>> Please take a look, and let me know if you wantt me to mark it as ready br/>> to review. < > br/>> iñigo < > br/>> ----------------------------------------- br/&>Iñigo Anngulo br/> < ZYLK.net :: > consultoría.openSource br/>telf.: 7474123337 br/>Ribera de Axpe, 11 br/>> br/>Edificio A, modulo 201-203 br/&ggt;48950 Erandio (Bizkaia) > br/>----------------------------------------- < > br/>> ----- Mensaje original ----- < > De: "ottobackwards" <[email protected]> > Para: "dev" <[email protected]> > Enviados: Jueves, 17 de Junio 2021 16:38:13 > Asunto: Re: Nifi integration record oriented processor for reading > br/>> That sounds great. < > br/>>> On Jun 17, 2021, at 10:14, Iñigo Anngulo <[email protected]> wrote: >> br/>> Hi Otto, < >> br/>> So we did this workaround, we re-forked the reepo and added the > changes of the nifi-record feature on a dedicated branch. I closed the br/>> previous PR, will upload the new one on the folloowing days (we are br/>> doing some last test just in case we forggot to copy something...) >> br/>> I will keep you informed. < >> br/>> regards, < >> iñigo >> br/>> ----------------------------------------- br/>> Iñigo Angulo br/>>> br/>> < > br/>> ZYLK.net :: consultoría.openSource br/>> telf.: 747412337 br/>> br/>> Ribera de Axpe, 11 br/&ggt;> Edificio A, modulo 201-203 br/>> 48950 br/>> Erandioo > (Bizkaia) br/>> ----------------------------------------- < >> br/>> ----- Mensaje original ----- < >> De: "ottobackwards" <[email protected]> >> Para: "dev" <[email protected]> >> Enviados: Jueves, 10 de Junio 2021 16:14:54 >> Asunto: Re: Nifi integration record oriented processor for reading br/>>> br/>> Yes, that sounds great. < Thannk you br/>>> On Jun 10, 2021, at br/>>> 03:43, I&ntillde;igo Anggulo <[email protected]> > wrote: >>> br/>>> Hi Otto, < >>> br/>>> I have tried to do the rebase of our ccommits, but I am br/>>>> having < > difficulties at it... br/>>> br/>>> The issue is: we forked the repo br/>> on september 2020, and started mmaking tests and commits to our fork (on 'develop' > branch). Now I am trying to do a rebase (using git rebase -i ID) br/>> specifying the ID of our first commit. But when the fille is open for br/>> interactive mode, it gets < > ~900 commits on the develop branch (belonging to members of the PLC4X br/>> community). I think this happens because beforee opening the br/>> PullRequest, I did a 'merge upstream' with thee actual PLC4X repo, to get the updates of the code. > So, I understand that in the interactive mode file, I have to leave br/>> all commits to 'pick' (by default), and change our ccommits of the nifi feature to 'squash' > except from the first one (which also remains as 'pick'). However, br/>> when I tried this, many conflicts appear, almost one per each commit (comunity members' > commits)... >>> I may be doing something wrong (never did a rebase before...) and I > prefered just to ask, as I dont want to break or cause any conflict to br/>> the repo code.. If you see anything Im missing pllease let me know. >>> br/>>> As a workaround, I was thinking we coulld close the PR, re-do br/>>>> the < > fork of the PLC4X repo, and add the changes to the code on a dedicated br/>> 'feature-nifi-record' branch. Maybe this could mmake things clearer... >>> What do you think? >>> br/>>> thank you, < >>> iñigo >>> br/>>> ------------------------------------------ br/>>> Iñigo br/>>>> Angulo < > br/>>> br/>>> ZYLK.net :: consultoría.openSource br/>>> telf.: br/>> 747412337 br/>>&> Ribera de Axpe, 11 br/>>> Edificio A, modulo 201-203 brr/>> br/>>> 48950 Erandio > (Bizkaia) br/>>> ----------------------------------------- < >>> br/>>> ----- Mensaje original ----- < >>> De: "ottobackwards" <[email protected]> >>> Para: "dev" <[email protected]> >>> Enviados: Jueves, 27 de Mayo 2021 16:10:19 >>> Asunto: Re: Nifi integration record oriented processor for reading br/>>>> br/>>> Awesome. < br/>;>> If you can, can I ask you to: < br/>>> 1. >>> Mark the PR as ready to review in github 2. rebase or squash it to a br/>>>> single commit and force push to youur branch > to clean it up >>> br/>>> br/>>> br/>>>> On May 27, 2021, at 06:45, Iñigo Angulo > <[email protected]> wrote: >>>> br/>>>> Hi Otto, Chris < >>>> br/>>>> we have finally commited the uupdates on the Nifi Processor br/>>>>> to < > the Pull Request. The changes we have done are the following: >>>> - deducing avro datatypes from PlcResponse. Here we may check the > method (org.apache.plc4x.nifi.util.Plc4xCommon.createSchema()), in br/>> order to see if it is the best way to do it. < >>>> - we have added in the plc4j/pom.xml an "ignoredDependency" for > org.apache.nifi:nifi-standard-nar, as it is used on runtime and was br/>> rising errors during compilation. < >>>> - we have changed onScheduled method in Plc4xSourceRecordProcessor > (comparing to BaseProcessor), as we have included the posibility to br/>> have an input connection into the processor, and inddicate the target br/>> addressMap through flowfile attributes. Thhe addressMap is now created in the onTrigger method. >>>> - we have tested the performance with S7 and Modbus protocols br/>>>>> (using a < > Siemens S7-1200 and Schneider M221). We will upload an updated nifi br/>> template for both protocols, but regarding this, doo you have any br/>> testing environment to simulate PLCs?? If that is the case, we could br/>> prepare the Processors configuratiion to match these ones (connection Strings and addressMaps). >>>> br/>>>> Please take a look at the code,, any suggestion will be br/>>>>> very < > welcome. >>>> br/>>>> iñigo < >>>> br/>>>> ------------------------------------------ br/>>>> Iñigo br/>>>>> Angulo > br/>>>> br/>>>> ZYLK.net :: consultoría.openSource br/>>>> telf.: br/>> 747412337 bbr/>>>> Ribera de Axpe, 11 br/>>>> Edificio A, modulo br/>> 201-203 br/>>>> 48950 Erandio (Bizkaia) br//>>>> br/>> ----------------------------------------- < >>>> br/>>>> ----- Mensaje original ----- < >>>> De: "Iñigo Angulo" <[email protected]> >>>> Para: "dev" <[email protected]> >>>> Enviados: Miércoles, 12 de Mayo 2021 14:42:22 >>>> Asunto: Re: Nifi integration record oriented processor for reading br/>>>>> br/>>>> Hi Otto, Chris, < br/>>>> we have been working on the br/>>;>>> prrocessor to include the logic of > getting the values for variables from the PLC response. We tested it br/>> with the < > S7-1200 and seems to work fine, however we would like to make some br/>> further tests before commiting it. < >>>> br/>>>> Regarding the actual method whiich takes the datatype from br/>>>>> the < > response object, we did it in the following way: >>>> br/>>>> //PlcReadResponse readResponse Map<String, ? extends >>>> PlcValue> responseDataStructure = > readResponse.getAsPlcValue().getStruct(); >>>> for (Map.Entry<String, ? extends PlcValue> entry : > responseDataStructure.entrySet()) { >>>> br/>>>> if (entry.getValue() instanceoof PlcINT) { br/>>>> br/>>>>> builder.namme(fielldName).type().unionOf().nullBuilder().endNull().a >>>> n d().intType().endUnion().noDefault(); > br/>>>>> }}else if (entry.getValue() instanceof PlcREAL) { br/>>>>> builder.name(fieldName).type(().unionOf().nullBuilder().endNull().an >>>> d ().doubleType().endUnion().noDefault(); > br/>>>>> }} ... and so on for the rest of the classes on the package > (org.apache.plc4x.java.spi.values.*) >>>> br/>>>> //the builder object is used tto build avro schema, with > desired datatypes (for example intType()) >>>> } >>>> br/>>>> br/>>>> Is this the solution you had in mind?? If you think > there is a better way to access PlcValues, please let us know and we br/>> will update it. < >>>> br/>>>> We will upload the code soon soo that you can take a deeper > look. >>>> br/>>>> thank you!! >>>> iñigo >>>> br/>>>> ------------------------------------------ br/>>>> Iñigo br/>>>>> Angulo > br/>>>> br/>>>> ZYLK.net :: consultoría.openSource br/>>>> telf.: br/>> 747412337 bbr/>>>> Ribera de Axpe, 11 br/>>>> Edificio A, modulo br/>> 201-203 br/>>>> 48950 Erandio (Bizkaia) br//>>>> br/>> ----------------------------------------- < >>>> br/>>>> ----- Mensaje original ----- < >>>> De: "ottobackwards" <[email protected]> >>>> Para: "dev" <[email protected]> >>>> Enviados: Viernes, 30 de Abril 2021 14:50:11 >>>> Asunto: Re: Nifi integration record oriented processor for reading br/>>>>> br/>>>> Sounds liike a good plan!! >>>> br/>>>>> On Apr 30, 2021, at 05:26, Iñigo Angulo > <[email protected]> wrote: >>>>> br/>>>>> Hi Otto, Chris, < >>>>> br/>>>>> we have been reviewingg the comments on the pull request, br/>>>>>> and < > started to think about the approach of extracting values from response directly. > During next week, we will work on this and make some updates to the br/>> code with the suggestions you made. We keep you infoormed >>>>> br/>>>>> thank you, < >>>>> br/>>>>> iñigo < >>>>> br/>>>>> ------------------------------------------ br/>>>>> Iñigo > Angulo br/>>>>> br/>>>>> ZYLK.net :: consultoría.openSource br/>>>>> telf.: > 747412337 br/>>>>> Ribera de Axpe, 11 br/>>>>> Edificio A, modulo br/>> 201-203 br/>>>&>> 48950 Erandio (Bizkaia) br/>>>>> > ----------------------------------------- < >>>>> br/>>>>> ----- Mensaje originall ----- >>>>> De: "Christofer Dutz" <[email protected]> >>>>> Para: "dev" <[email protected]> >>>>> Enviados: Viernes, 23 de Abril 2021 18:10:35 >>>>> Asunto: AW: AW: Nifi integration record oriented processor for br/>>>>>> reading br/>>>&ggt;> Hi Inigo, < br/>>>>> especially if you havee a br/>>>>>> look at the KNX protocol. This < > doesn't define the usual IEC datatypes we tried to use for all normal br/>> PLC drivers. < >>>>> Here we have hundreds of datatypes that don't match any other > protocol. I think the PLCValue approach would be the simplest. >>>>> The one thing you have to keep in mind, is that you should check a > PLCValue, if it's a list (Array type) or a Structure (Which sort of br/>> relates to komplex types with a more sophisticated sstructure). >>>>> br/>>>>> Chris < >>>>> br/>>>>> br/>>>>> -----Ursprüngliche Nachricht----- < >>>>> Von: Iñigo Angulo <[email protected]> br/>>>>> Gesendet: > FFreitag, 23. April 2021 15:34 >>>>> An: dev <[email protected]> >>>>> Betreff: Re: AW: Nifi integration record oriented processor for > reading >>>>> br/>>>>> Hi Otto, Chris, < >>>>> br/>>>>> Yes, I think the approoach you propose will be best. By br/>>>>>> now, < > we are generating the schema ourselves. We have a record writer who is br/>> in charge of reading PLC values. Schema is definned previously to br/>> reading the values. We build this schema ggetting the protocol from the br/>> 'connectionString' (S7, Modbuss) and the specified variable type from the 'PLC resource address String' > containing the list of variable to read. From this we deduce the br/>> expected Avro datatype when reading, for instance, a word in S7 or a coil in Modbus. > br/>&ggt;>>> br/>>>>> However, as you mentioned, the oother approach br/>> will be much clearerr and useful. Ideally, getting the actual datatype br/>> from PLCVValue when getting the response. > Regarding this, we tried to keep the previously described 'mapping' > separated from the rest of the code, so that hopefully it can be br/>> easily replaced.. < >>>>> br/>>>>> We have done the pull rrequest, hope you can take a look br/>>>>>> at < > the code and let us know what you think. We will fill the ICLA document too. >>>>> br/>>>>> thank you < >>>>> iñigo >>>>> br/>>>>> br/>>>>> br/>>>>> >>>>> ----------------------------------------- < Iñigo Angulo br/>>>>> br/>>>>>&ggt; br/>>>>> ZYLK.net :: consultoría.openSource < >>>>> telf.: 747412337 >>>>> Ribera de Axpe, 11 >>>>> Edificio A, modulo 201-203 >>>>> 48950 Erandio (Bizkaia) >>>>> ----------------------------------------- >>>>> br/>>>>> ----- Mensaje original ----- >>>>> De: "Christofer Dutz" <[email protected]> >>>>> Para: "dev" <[email protected]> >>>>> Enviados: Jueves, 22 de Abril 2021 17:12:49 >>>>> Asunto: AW: Nifi integration record oriented processor for reading br/>>>>>> br/>>>>&ggt; Hi all, < br/>>>>> Well, you get PlcValuees from the br/>>>>>> response that wrap the < > different datatypes. So generally you shouldn't care about the detail type. >>>>> br/>>>>> However, you can call ggetObject() which returns the core > value the plc-value has ... so if it's the PLCValue for a Short, br/>> getObject will return a short value. < >>>>> br/>>>>> Does that help?? >>>>> br/>>>>> Chris < >>>>> br/>>>>> br/>>>>> -----Ursprüngliche Nachricht----- < >>>>> Von: Otto Fowler <[email protected]> >>>>> Gesendet: Donnerstag, 22. April 2021 15:21 >>>>> An: [email protected] >>>>> Betreff: Re: Nifi integration record oriented processor for br/>>>>>> reading br/>>>>&> So, you are generating the schema yourself, such br/>>>>t;>> that > downstream if they inherit schema they will just get what you br/>> generate?? And you are trying to do that by the connection string? If br/>> so, a different way I could imagine doing woould be to get the ’types’ br/>> of the data from the responses themselves. This would be more generic. br/>> The flow I could imagine ( in OnTrigger > ): >>>>> br/>>>>> DO READ < >>>>> IF NOT HAS SCHEMA >>>>> GENERATE SCHEMA FROM RESPONSE AND CACHE IN ATOMIC WRITE WITH br/>>>>>> SCHEMA br/>>>>t;> Maybe Chris can speak tto how to get the types br/>>>>t;>> from the > responses. >>>>> br/>>>>> br/>>>>>> On Apr 22, 2021, at 05:48, Iñigo Anguulo > <[email protected]> wrote: >>>>>> br/>>>>>> Hi Chris, Otto,, >>>>>> br/>>>>>> Regarding the RRecord Processor concept, i will try to br/>>>>>>&ggt; give > an overview. In Nifi, information packages are called Flowfiles, and br/>> these are the actual units of information that arre exchanged between br/>> Procesors, all along the dataflow. FFlowfiles have two sections where br/>> we can manage < > data: Attributes and Content. In the "traditional" Nifi approach, you br/>> work with both sections, extracting informatiion from the Content to br/>> the Attributes and viceversa to perfform operations. This approach br/>> could have one limitation wheen you are processing batch data (lines br/>> from a CSV file for instance), where you need to split each of the br/>> lines into ddifferent Flowfiles. Thus, a 1000 line CSV file leads to br/>> 11000 Flowfiles to process, each of them containing a single record. >>>>>> br/>>>>>> On later versioons of the product, they introduced the > Record oriented approach. This approach allows you to manage multiple br/>> records on a single FFlowfile's Content, as long as these records have all the same schema. > This means that the operations defined by the Processors are applied br/>> simultaneously to the whole content at once. FFollowing with the br/>> previous example, a 1000 line CSV file couldd produce a single Flowfile br/>> with a content of < > 1000 records. br/>>>>>> br/>>>>>> To do this, Nifi uses Avro, to br/>> serialize thee FFlowfile's Content. Then, the Record Oriented br/>> Processors use Writers and Readers to present this information in the br/>> desiired format (such as Avro, Json, CSV, etc). Basically, with the br/>&> record oriented approach, Nifi introduced multiple new Processors, and br/>> also included the Record version of many of the ""old" ones. Using this br/>> Record approach, Nifi perfomance enhances notably, specially when working with large structured information. >>>>>> br/>>>>>> The work we didd was creating a Record Oriented br/>>>>>>> Proccessor, > based on the previously existing one Plc4xSourceProcessor, to read br/>> values from the devices. We have also included a READDME on the br/>> plc4x/plc4j/integrations/apache-nifi module explaaining the Processor br/>> configuration and giving an example. Mooreover, we put a nifi template br/>> with a dataflow for testing these processors, if useful. >>>>>> br/>>>>>> Otto, regardingg the idea behind this new Processor, br/>>>>>>>; that > is right. We added the writer capability to the existing > PLC4XSourceProcessor, so that it formats the output to the desired configuration in a record manner. > At the actual implementation, we did this "protocol adaptation" from br/>> the sintax of the particular properties on Proccessor's configuration. br/>> FFor example, from connection string 's7://IP:PORT', we extract the S7 br/>> idenifier and map variaable datatypes to the actual Avro datatypes for build the record output schema. > However, here we dont have vast experience with PLC4X libraries, and br/>> for sure there will be better ways for doing this. >>>>>> Also about the Base Processor, we were thinking that maybe the br/>>>>>>> best < > approach could be to have this Base Processor, and then implement br/>> readers for particular protocols as Controller Servicces. But here br/>> also, it could be very helpful to have your oppinion. >>>>>> br/>>>>>> Lastly, regardiing the pull request, do you have any > documentation on br/>>>&ggt;>> how to do this? I mean, maybe you have br/>> defined some naming br/>&ggt;>>>> conventions, or expected structure to br/>> faaciliitate later work. At the br/>>>>>> present, we have a fork of br/>> the project where we have been working on br/>t;>>>&ggt;> these Nifi br/>> changes. We updated tthe content of our fork (fetch/merge >>>>>> upstream) about 2 weeks ago, and commited our changes to the > 'develop' br/>>>>>> branch. Do we bettter create a new branch with our commits? > how do you br/>>>&>>> prefer to receive the code? (we are not very br/>> experts on git, just in br/&ggt;>>>>> case we could cause some br/>> problemss....) >>>>>> br/>>>>>> thank you in addvance >>>>>> br/>>>>>> iñigo < >>>>>> br/>>>>>> br/>>>>>> ----------------------------------------- < br/>&ggt;>>>>> Iñigo Angulo br/>>>>>> ZYLK.net :: connsultoría.openSource >>>>>> telf.: 747412337 >>>>>> Ribera de Axpe, 11 >>>>>> Edificio A, modulo 201-203 >>>>>> 48950 Erandio (Bizkaia) >>>>>> ----------------------------------------- >>>>>> br/>>>>>> ----- Mensaje ooriginal ----- >>>>>> De: "Christofer Dutz" <[email protected]> >>>>>> Para: "dev" <[email protected]> >>>>>> Enviados: Miércoles, 21 de Abril 2021 20:01:15 >>>>>> Asunto: AW: Nifi integration record oriented processor for br/>>>>>>> reading br/>>&ggt;>>> The more I thinnk of it, br/>>>>>> Perhaps we br/>>>>>>> shouuld also think of potentiallyy providing some > information on supported configuration options. >>>>>> Wouldn't it be cool if the driver could say: "I generally have br/>>>>>>> these < > options and they have these datatypes and mean this" >>>>>> Additionally, the transports could too say: "I generally have br/>>>>>>> these < > options and they have these datatypes and mean this" >>>>>> br/>>>>>> I would be our StreamPipes friends would love something > like that? Right? >>>>>> br/>>>>>> Chris < >>>>>> br/>>>>>> br/>>>>>> -----Ursprüngliche Nachricht----- < >>>>>> Von: Otto Fowler <[email protected]> >>>>>> Gesendet: Mittwoch, 21. April 2021 17:46 >>>>>> An: [email protected] >>>>>> Betreff: Re: Nifi integration record oriented processor for br/>>>>>>> reading br/>>&>>>> Hi Inigo, < br/>>>>>> I’m a coommitter on br/>>>>>>> Apache Nifi as well as PLC44X, I would > be happy to review your processor. >>>>>> If I understand what you are saying correctly, you have a single > processor which supports record writing output? >>>>>> br/>>>>>> plc4x -> reccords >>>>>> br/>>>>>> And that you haave, for configuration purposes for that > processor created support on a per protocol basis for configuration br/>> and validation?? >>>>>> br/>>>>>> If there is perr protocol configuration / validation br/>>>>>>>; etc, > it may be better to have a base processor, and derived processors per br/>> protocol to handle those differences. < >>>>>> br/>>>>>> I look forward to seeing the code. >>>>>> br/>>>>>> br/>>>>>>> On Apr 21, 2021, at 04:05, Iñigo Angulo > <[email protected]> wrote: >>>>>>> br/>>>>>>> Hi all,, >>>>>>> br/>>>>>>> I am wrriting as we have been working on the Apache br/>>>>>&>>> Nifi > integration part of the project. We have created a Record oriented br/>> processor for reading PLC data. It is based on the prrevious existing br/>> SourceProcessor, but works with records, ussing a Nifi Writer (such as br/>> Avro, Json, and so on) to write data on flowfiles content. br/>>>>>>> br/>> br/>>>>>>> We updated the code on our fork with thee actual PLC4X git br/>> repo about 2 weeks ago, and tested it reaading values with S7 from a br/>> S7-1200 CPU from Nifi. Also, onee of our customers has recently started br/>> to use it for validaation. br/>>>>>>> br/>>>>>>> Currently, it works br/>> with S7 and Modbus oover TCP. This is becaause we had to write some br/>> classes to map connectionString annd variableList properties (sintax) br/>> of the processor to the actual protocol, to be able to build then avro br/>> schema for ooutput flowfile, taking into account variable datatypes, br/>> etcc. We only did this for > S7 and Modbus. I am sure that there is a better way to do this, so at br/>> this point you maybe could take a look to find thee best solution and br/>> avoid needing to do this mapping. br/>t;>>>>>> br/>>>>>>> If you find > this useful, we could do a ppull request to the main PLC4x repo. Let br/>> us know what you think. br/>>>>>>>t;> br/>>>>>>> best regards, >>>>>>> iñigo >>>>>>> br/>>>>>>> ------------------------------------------ >>>>>>> Iñigo Angulo >>>>>>> br/>>>>>>> ZYLK.neet :: consultoría.openSource >>>>>>> telf.: 747412337 >>>>>>> Ribera de Axpe, 11 >>>>>>> Edificio A, modulo 201-203 >>>>>>> 48950 Erandio (Bizkaia) > >>>>>> -----------------------------------------
