Re: ExecuteSQLRecord Bug

2019-10-08 Thread Bimal Mehta
Hi, Just following up on the below email. Is this a known issue? Happy to raise a JIRA to having it fixed in the next release or its already fixed in the later versions? Thanks On Mon, Sep 16, 2019 at 1:23 PM Bimal Mehta wrote: > Hi, > > We are using ExecuteSQLRecord processor t

ExecuteSQLRecord Bug

2019-09-16 Thread Bimal Mehta
passed below. However the inherited attributes are only passed to the first 5 flow files. They are not being passed to the other 15. Is this a bug? How do we overcome this? We are in NiFi 1.9.0 Thanks Bimal Mehta

Re: Variables to start a NiFi flow

2019-08-27 Thread Bimal Mehta
n these > GenerateFlowFile-s. > The problem is that not all "SQL" processors support flowfile input. You > can use ExecuteSQL(Record) or GenerateTableFetch in this way, but not > QueryDatabaseTable. > > Regards, > Peter > > On Mon, Aug 26, 2019 at 6:30 PM Bimal Mehta

Variables to start a NiFi flow

2019-08-26 Thread Bimal Mehta
and the flow is triggered for that table. Variable Registry comes to mind, but I am not sure how to make it work for this use case. We are using NiFi 1.9.0 as part of CDF bundle. Thanks Bimal Mehta

Re: Custom Processor Upgrade

2019-08-20 Thread Bimal Mehta
plain why it can't find the application-context.xml since I >> don't have access to your code, but I don't see why that would be >> related to moving to NAR_INF from META_INF, nothing should really be >> relying on that structure. >> >> On Thu, Aug 15, 2019 at 1:

Creating External Hive Table on Parquet Files

2019-08-20 Thread Bimal Mehta
Hi, We have a sub Flow - QueryDatabaseTable->SplitAvro->ConvertAvrotoORC->PutHDFS->ReplaceText->PutHQL. We are using ConverAvrotoORC for generating the DDL. The flow works fine. However in Cloudera distribution we are not able to query the table that has been created as the DDL creates the table

Re: Custom Processor Upgrade

2019-08-15 Thread Bimal Mehta
ar that contains your > processors, most likely in src/main/resources of the processors module > which then ends up at the root of the jar. > > On Wed, Aug 14, 2019 at 5:36 PM Bimal Mehta wrote: > >> Ahh, seems like a Springboot error. >> Is it

Re: Custom Processor Upgrade

2019-08-14 Thread Bimal Mehta
t; should be a second part to it with "Caused by " and then more of > the stacktrace that wasn't there before. > > On Wed, Aug 14, 2019 at 4:41 PM Bimal Mehta wrote: > > > > Hi Bryan, > > > > I did what you said. > > This is what I got > >

Re: Custom Processor Upgrade

2019-08-14 Thread Bimal Mehta
ot cause exception which we are missing right now. > > I think it takes about 20-30 seconds for logback to pick up the edits > to logback.xml. > > On Wed, Aug 14, 2019 at 12:53 PM Bimal Mehta wrote: > > > > For the custom processor we have, we are extending the > Abstr

Re: Custom Processor Upgrade

2019-08-14 Thread Bimal Mehta
> What is in the logs at the time you get the ghost processor? > > On Tue, Aug 13, 2019 at 10:54 PM Bimal Mehta wrote: > >> Does that mean I need to recreate the processor? Or there is some >> workaround? >> >> The processor gets unpacked and its bundled depend

Re: Custom Processor Upgrade

2019-08-13 Thread Bimal Mehta
are missing as well. On Tue, Aug 13, 2019 at 10:47 PM Bryan Bende wrote: > I don’t remember all the reasoning behind the change, but it had to do > with an issue when we upgraded Jetty... > > https://issues.apache.org/jira/browse/NIFI-5479 > > On Tue, Aug 13, 2019 at 9:47 PM

Re: Custom Processor Upgrade

2019-08-13 Thread Bimal Mehta
canvas? > > On Tue, Aug 13, 2019 at 4:54 PM Bimal Mehta wrote: > >> Hi Mike and Bryan, >> >> One of my custom processors appears as inactive in NiFi with a yellow >> triangle error. >> When I hover over it I see a message saying 'Missing Processor' v

Re: Custom Processor Upgrade

2019-08-13 Thread Bimal Mehta
of this custom processor somewhere? On Thu, Aug 8, 2019 at 9:14 AM Bimal Mehta wrote: > Thanks Mike and Bryan. > Yes it seems my template was still referring the old version. > I will have it updated now and will reimport. > Also the version of NiFi we are using is the one that comes

Re: Data Ingestion using NiFi

2019-08-13 Thread Bimal Mehta
d and PartitionRecord with them configured to > use Avro readers and writers. > > Mike > > On Tue, Aug 13, 2019 at 12:25 AM Bimal Mehta wrote: > >> Hi NiFi users, >> >> We had been using the kylo data ingest template to read the data from our >> Oracle and

Data Ingestion using NiFi

2019-08-12 Thread Bimal Mehta
spark job that will execute the validation rules and split the data. Any processor that can help achieve this? I know a few users in this group had used kylo on top of NiFi. It will be great if some of you can provide your perspective as well. Thanks in advance. Bimal Mehta

Re: Custom Processor Upgrade

2019-08-08 Thread Bimal Mehta
> there are a few critical bugs fixed in 1.9.2. > > > > On Wed, Aug 7, 2019 at 9:19 PM Bimal Mehta wrote: > >> > >> Thanks Bryan. > >> My custom processors are part of a template. However when I try to > import my template in NiFi 1.9, I get an error m

Re: Custom Processor Upgrade

2019-08-07 Thread Bimal Mehta
. > > -Bryan > > On Wed, Aug 7, 2019 at 3:18 PM Bimal Mehta wrote: > > > > Hi, > > > > If we have a custom processor that was created with NiFi 1.6, what are > the steps we need to follow to make it work in 1.9? > > Is there some sort of steps that explains the jar and pom updates we > need to do for making it work in 1.9? >

Custom Processor Upgrade

2019-08-07 Thread Bimal Mehta
Hi, If we have a custom processor that was created with NiFi 1.6, what are the steps we need to follow to make it work in 1.9? Is there some sort of steps that explains the jar and pom updates we need to do for making it work in 1.9?