We often don't use the Record Processors because of the Schema requirement
and complexity to use the LookupRecord processor.
I'll refer to this email in the NiFi mailing list: "GetMongo - Pass-on
Initial FlowFile?"... There were suggestions to use the LookupRecord
processor, but ultimately it couldn't do what we needed to be done, so we
had to string together a set of other processors.
For us, it was easier to string together a set of processors than to figure
out why LookupRecord, MongoDBLookupService, and InferAvroSchema wasn't
getting the job done for us.
/---success---> *ReplaceText* (Prepend JSON Key)
---success--> \
/
\
*GetMongo*
-------> *Merge Content* (Combine
on Correlation Attribute Name, Binary Concat)
\
/
\---original---> *ReplaceText* (Prepend JSON Key)
---success--> /
If they're marked as deprecated, I'd really like to see barrier to entry
with the LookupRecord processors decreased. The number 1 thing I don't
like about the Record processors is that they require a Schema, and the
complimentary processor(s?), specifically the GetMongo one, does not
require a schema.
Ryan
On Sat, Feb 23, 2019 at 11:39 AM Andrew Grande <[email protected]> wrote:
> I'm not sure deprecating is warranted. In my experience, record based
> processors are very powerful, but have a steep learning curve the way they
> are in NiFi today, and, frankly, simple things should be dead simple.
>
> Now, moving the record UX towards an easy extreme affects this equation,
> but e.g. I never open up a conversation with a new user by talking about
> records, Schema Registry or NiFi Registry.
>
> Maybe there's something coming up which I'm not aware yet? Please share.
>
> Andrew
>
> On Sat, Feb 23, 2019, 7:43 AM Sivaprasanna <[email protected]>
> wrote:
>
> > Team,
> >
> > Ever since the Record based processors were first introduced, there has
> > been active development in improving the Record APIs and constant
> interest
> > in introducing new set of Record oriented processors. It has gone to a
> > level where almost all the processors that deal with mainstream tech
> have a
> > Record based counterpart, such as the processors for MongoDB, Kafka,
> RDBMS,
> > HBase, etc., These record based processors have overcome the limitations
> of
> > the standard processors letting us build flows which are concise and
> > efficient especially when we are dealing with structured data. And more
> > over with the recent release of NiFi (1.9), we now have a new feature
> that
> > offers schema inference capability which even simplifies the process of
> > building flows with such processors. Having said that, I'm wondering if
> > this is a right time to raise the talk of deprecating processors which
> the
> > community believes has a much better record oriented counterpart,
> covering
> > all the functionalities currently offered by the standard processor.
> >
> > There are a few things that has to be talked about, like how should the
> > deprecated processor be displayed in the UI, etc., but even before going
> > through that route, I want to understand the community's thoughts on
> this.
> >
> > Thanks,
> > Sivaprasanna
> >
>
