Chad,
I was looking into something related to this recently and I think your
description is accurate. Unfortunately ZooKeeper client relies heavily
on system properties which isn't great for talking to a bunch of
different systems like NiFi.
One thing I would be curious about, what value do you
I’m building a flow that uses ExecuteSQL to query data from Phoenix on top of a
Kerberized Ambari Metrics’ HBase (it hits the AMS Zookeeper). I ran into issues
and the logs showed NiFi (also kerberized) was getting auth failed when
connecting to ZNode.
To confirm I had all my proper AMS conf
HI Clay,
So as Bryan has said the actual connection is managed by a selector and all
this does is goes through each connection and once that connection has data
to receive it the selector then hands that over to a thread in the TCP
receiving thread pool which does then some basic TCP processing
The actual connections themselves are managed with a selector, so if
all the connections are idle there should only be one thread for the
socket.
As soon as a connection has something available to read then a thread
is spawned to start reading the connection until either no matter is
available,
Hello Edward,
So, if have of to listen to 32,000 tcp connections and I have only 80
cores, and I configure each ListenSyslog instance for 4,000 connections,
doesn't each spawn 4,000 threads behind the scene? The tcp connections will
be idle most of the time.
thanks
Clay
On Fri, Aug 2, 2019 at
Hi Clay,
Because Nifi underneath uses a thread pool for it's own threading
underneath, and each instance processor runs does so in it's own thread, I
don't see any reason why not. One thing to note that the way the ListenTCP
processor appears to have been written such that it gets all the
> I am now getting in the NiFi log file is "PutDatabaseRecord failed to
> process StandardFlowFIle Due to Unknown Table
>
> name of the table that I am trying to write to
>
> in information_schema".
I believe this is due to a mismatch between the MySQL driver version and the
MySQL version.
Hello All,
I need to listen to and process thousands of persistent TCP connections. I
have 10 nodes, each having 8 cores.
My understanding is that with existing NiFi listening processors, such as
ListnSyslog, a thread is utilized for each TCP connection. Does this scale?
Do I need to write a
I just run a single CaptureChangeMySQL processor， the cpu usage is more than
100%
Is it normal?
If so, why it is so cpu intensive?
wangl...@geekplus.com.cn
