Re: How to increase the processing speed of the ExtractText and ReplaceText Processor?

2016-10-24 Thread Andy LoPresto
Excellent information here. Thanks Lee and Peter. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Oct 24, 2016, at 6:57 AM, Peter Wicks (pwicks) wrote: > > Prabhu, > > Lee mentioned

Re: Can we configure NiFi to run execute process with specific Kerberos Principal?

2016-10-24 Thread Bryan Bende
Hi Ravi, I'm not very familiar with Sqoop, but from quickly reading their documentation and some other forums/blogs, it seems like the script that NiFi is calling should be doing something like the follow: kinit /bin/sqoop.sh client I would think however you execute the script successfully

Can we configure NiFi to run execute process with specific Kerberos Principal?

2016-10-24 Thread Ravi Papisetti (rpapiset)
Hi, We are planning to use "ExecuteProcess" to run a sqoop script wrapped by shell. As part of this we want NiFi to use its service principal in secure mode while submiting executing the script. Otherwise sqoop script is failed to execute saying "Caused by GSSException: No valid credentials

RE: How to increase the processing speed of the ExtractText and ReplaceText Processor?

2016-10-24 Thread Peter Wicks (pwicks)
Prabhu, Lee mentioned making sure you have good indexes, but I would caution you on this point. If you have a unique constraint then SQL Server will build an index on this automatically, but I would suggest dropping all other indexes that aren’t related to data integrity. Each time SQL Server

Re: How to increase the processing speed of the ExtractText and ReplaceText Processor?

2016-10-24 Thread Lee Laim
Hello Prabhu, 50 minutes is a good start! Now we have to *determine where the next bottleneck is *-check to see where the flow files are queueing. You can also check the "average task duration" statistic for each processor. I suspect the bottleneck is at PutSQL and will carry this assumption

start it ONCE

2016-10-24 Thread Alessio Palma
Hello All, here is a question which is driving me crazy... I'm using the PutHiveQL processor to create some tables and view into HIVE. This processor need the SQL to execute into a flowfile. I'm using a GenerateFlowFile to build an empty flowfile which is later filled using the ReplaceText