Mike,
If you don't mind could you file a JIRA for this. Frankly it sounds
like a bug to me. We should consider making a default scheduling
period of something a bit slower. Frankly just dialing back to 100 ms
would be sufficient most likely. If you agree this is a bug please
file one here: htt
Thanks all - I checked the logs and there is nothing I can see thats seems
erroneous. I increased the number of threads for the processor and added
the 10 second scheduling and it has dropped dramatically from 2.5M tasks to
300 over 5 minute period. CPU for the nifi java process is now running at
8
Incidentally, I took the information on how to do this from:
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.controller.ControllerStatusReportingTask/additionalDetails.html
On Thu, Jul 14, 2016 at 11:02 AM, Russell Bateman <
russell.bate...@perfectsearchcorp.com> wrote:
> First
If you modify a flow.xml by hand and put a plaintext value into any
sensitive property, then the next time you start NiFi and it saves the
flow.xml.gz NiFi will encrypt the plaintext value. Any change to the flow
(such as starting, stopping, reconfiguring, creating, deleting or moving a
processor)
Ashraf,
Glad to hear it fixed it. I have an existing Jira [1] filed which would include
strengthening the default sensitive key algorithm, but would also require
creating a tool to seamlessly upgrade the encrypted values from the previous
algorithm. This tool could also support migrating values
Happy to help, Brett. I always like seeing people use the software in a secure
manner. If you can, you may want to publish your tool on GitHub. While you
don’t have to submit it back for inclusion in NiFi itself (and if it’s a Ruby
tool, it may not be correct for inclusion), there are many peopl
Aaron,
My guess would be that you are hitting a Full Garbage Collection. With such a
huge Java heap, that will cause a "stop the world" pause for quite a long time.
Which garbage collector are you using? Have you tried reducing the heap from 48
GB to say 4 or 8 GB?
Thanks
-Mark
> On Jul 14, 2
Hi Andy,
I think I need a little time to review your post, but this sounds exactly like
what I was looking for. I was looking for a way to create the encrypted value
stored within the “enc{ … }” tag. Thank you for translating my question and for
the quick response!
Brett Hite
brett.h...@parson
Hi Brett,
I’m not sure I understand your question completely, so let me try to describe
it and you can correct me where I get it wrong.
You have some deployment system which uses a Ruby process to replace tokens in
a flow template with the “real” values, and one of the values that needs to be
Hello,
I have a flow file that is created from a Ruby template file (flow.xml.erb).
The template contains variables that the user can set that then get populated
when NiFi is set up. I have an EncryptContent processor and would like to
create a template variable for the Password property. Ideal
Russell,
There have been a couple of bugs in the logging for
ControllerStatusReportingTask (I am the perpetrator of one of them).
Depending on the version of NiFi you have, it is possible that your
logback.xml is correct, but that the actual logger name used by
ControllerStatusReportingTask does n
First, I'm using ControllerStatusReportingTask whose output I see in
*nifi-app.log.* This seems to do report what I want to see. However, I'd
like the status reports to go out to a different log file. I bounced NiFi
after enhancing the log-back file:
I inserted this near the top of *conf/logback.x
Thank you Joe, that was exactly what I was looking for.
Kevin
-Original Message-
From: Joe Witt [mailto:joe.w...@gmail.com]
Sent: Wednesday, July 13, 2016 4:27 PM
To: users@nifi.apache.org
Subject: Re: Increase the number of threads available to a Node
Hello
Take a look here
https://ni
Mike,
To add some context, while NiFi will intelligently schedule processors to
execute, given HandleHTTPRequest's function as a listener, it is constantly
scheduled to run, checking for a request to handle. I assume by number of
tasks, you mean the rolling count over the last 5 minutes. As ment
Andy,
Thanks for your reply. I think I must have changed the file at an earlier stage
indeed. Reverted the nifi.sensitive.props.key to an empty string and it came
back up.
Thanks,
From: Andy LoPresto
Sent: July 13, 2016 4:42:15 PM
To: users@nifi.apache.org
Su
Hi,
I'm having an issue with a small (two node) NiFi cluster where the nodes
will stop processing any queued flowfiles. I haven't seen any error
messages logged related to it, and when attempting to restart the service,
NiFi doesn't respond and the script forcibly kills it. This causes
multiple
Mike,
You can adjust the processor properties for the HandleHTTPRequest processor in
the scheduling tab.
“Concurrent tasks” limits the number of threads this processor will use
(default is 1)
“Run schedule” determines the frequency that this processor will be run
(default is ‘0 sec’ which mean
ps - also noticed it seems to generate a lot of tasks, currently 2.5M
compared to other processes in the pipeline which reports 10s of tasks.
Mike
On 14 July 2016 at 15:34, Mike Harding wrote:
> Hi All,
>
> The node in my cluster running nifi crashed due to a CPU overload event.
> After restart
Hi Dima,
You will have to create an über jar from the JDBC drivers provided by Teradata
and copy the uberjar into the lib folder of nifi.
As Matt pointed out the instructions are in the email thread. You can refer
them.
Anuj
> On Jul 14, 2016, at 9:35 AM, Matt Burgess wrote:
>
> Dima,
>
Hi All,
The node in my cluster running nifi crashed due to a CPU overload event.
After restarting I analysed the CPU consumption and found that nifi was the
issue. As you can see below it was running at 133% CPU:
PID USER PR NIVIRTRESSHR S *%CPU* %MEM TIME+ COMMAND
2031 ni
Dima,
There was a discussion on how to get the SQL processors working with
Teradata a little while ago:
http://mail-archives.apache.org/mod_mbox/nifi-users/201605.mbox/%3CCAEXY4srXZkb2pMGiOFGs%3DrSc_mHCFx%2BvjW32RjPhz_K1pMr%2B%2Bg%40mail.gmail.com%3E
Looks like it involves making a fat JAR to in
Hello, everyone,
I'm new to nifi and this mailing list, I'm evaluating if we could extract
data (tables) from Teradata to local fs or HDFS with nifi. Is that possible?
What I've done so far was creating an "Execute SQL" processor with the
query and the a database connection pooling service with t
22 matches
Mail list logo
