Sivaprasanna, FWIW, I think there might be merit to deprecating converting to Avro, but the rest I think should stay. With Avro, I feel like there is intrinsic danger in giving people that option if they're unwilling to learn how to write an Avro schema.
On Sat, Feb 23, 2019 at 1:21 PM Mike Thomsen <[email protected]> wrote: > > The number 1 thing I don't like about the Record processors is that they > require a Schema, and the complimentary processor(s?), specifically the > GetMongo one, does not require a schema. > > FWIW, we just added GetMongoRecord in 1.9.0, along with GridFS processors. > > I'll note that arguably the best reason for you to take the dive into > being able to use the Record API w/ Mongo is precisely that Mongo doesn't > even have schema on write. It's entirely possible that 9 out of 10 people > on your team write a date the right way you agreed upon and then the 1 hold > out does the polar opposite and you won't know until random, bizarre > behavior shows up. > > On Sat, Feb 23, 2019 at 12:06 PM Ryan Hendrickson < > [email protected]> wrote: > >> We often don't use the Record Processors because of the Schema requirement >> and complexity to use the LookupRecord processor. >> >> I'll refer to this email in the NiFi mailing list: "GetMongo - Pass-on >> Initial FlowFile?"... There were suggestions to use the LookupRecord >> processor, but ultimately it couldn't do what we needed to be done, so we >> had to string together a set of other processors. >> >> For us, it was easier to string together a set of processors than to >> figure >> out why LookupRecord, MongoDBLookupService, and InferAvroSchema wasn't >> getting the job done for us. >> /---success---> *ReplaceText* (Prepend JSON Key) >> ---success--> \ >> / >> \ >> *GetMongo* >> -------> *Merge Content* >> (Combine >> on Correlation Attribute Name, Binary Concat) >> \ >> / >> \---original---> *ReplaceText* (Prepend JSON Key) >> ---success--> / >> >> >> If they're marked as deprecated, I'd really like to see barrier to entry >> with the LookupRecord processors decreased. The number 1 thing I don't >> like about the Record processors is that they require a Schema, and the >> complimentary processor(s?), specifically the GetMongo one, does not >> require a schema. >> >> Ryan >> >> On Sat, Feb 23, 2019 at 11:39 AM Andrew Grande <[email protected]> >> wrote: >> >> > I'm not sure deprecating is warranted. In my experience, record based >> > processors are very powerful, but have a steep learning curve the way >> they >> > are in NiFi today, and, frankly, simple things should be dead simple. >> > >> > Now, moving the record UX towards an easy extreme affects this equation, >> > but e.g. I never open up a conversation with a new user by talking about >> > records, Schema Registry or NiFi Registry. >> > >> > Maybe there's something coming up which I'm not aware yet? Please share. >> > >> > Andrew >> > >> > On Sat, Feb 23, 2019, 7:43 AM Sivaprasanna <[email protected]> >> > wrote: >> > >> > > Team, >> > > >> > > Ever since the Record based processors were first introduced, there >> has >> > > been active development in improving the Record APIs and constant >> > interest >> > > in introducing new set of Record oriented processors. It has gone to a >> > > level where almost all the processors that deal with mainstream tech >> > have a >> > > Record based counterpart, such as the processors for MongoDB, Kafka, >> > RDBMS, >> > > HBase, etc., These record based processors have overcome the >> limitations >> > of >> > > the standard processors letting us build flows which are concise and >> > > efficient especially when we are dealing with structured data. And >> more >> > > over with the recent release of NiFi (1.9), we now have a new feature >> > that >> > > offers schema inference capability which even simplifies the process >> of >> > > building flows with such processors. Having said that, I'm wondering >> if >> > > this is a right time to raise the talk of deprecating processors which >> > the >> > > community believes has a much better record oriented counterpart, >> > covering >> > > all the functionalities currently offered by the standard processor. >> > > >> > > There are a few things that has to be talked about, like how should >> the >> > > deprecated processor be displayed in the UI, etc., but even before >> going >> > > through that route, I want to understand the community's thoughts on >> > this. >> > > >> > > Thanks, >> > > Sivaprasanna >> > > >> > >> >
