problem with setting of Time Duration property from group variable

[l vic]

I am trying to set Time Duration property in ControlRate processor from
processor group variable.
I have INTERVAL variable set to 20 and I tried to set Time Duration as
${INTERVAL} mins
${INTERVAL:toNumber() mins}
I each case i have error like the following:
Time duration validated against '{INTERVAL mins}' is invalid because must
be of format   where duration is non-negative integer
and TimeUnit is supported Time Unit such as mins
Is there a solution to this problem?
Thank you

ExecuteSQLRecord Bug

[Bimal Mehta]

Hi,

We are using ExecuteSQLRecord processor that reads the query and certain
attributes from the previous flow,
We have the following properties set;
Max Rows Per Flow File: 5
Output Batch Size: 5

In our case we have 1 mn  records coming from the source. 20 flowfiles get
created that get passed below.
However the inherited attributes are only passed to the first 5 flow files.
They are not being passed to the other 15.
Is this a bug? How do we overcome this?

We are in NiFi 1.9.0

Thanks
Bimal Mehta

RE: NiFi cluster heavy cpu usage

[Williams, Jim]

Lei,

 

I have seen this on 1.9.2 standalone and cluster deployments.  Every time I
have seen this so far, the reason has been an input port in a process group
which is not connected by an output port.  The input port can be identified
in the user interface by drilling down to the active thread.  Disabling or
removing the input port resolves the high CPU usage.

 

 

Warm regards,

 


  

Jim Williams | Principal Database Developer


O: +1 713.341.7812 | C: +1 919.523.8767 | jwilli...@alertlogic.com |
 alertlogic.com

 


 



 

From: wangl...@geekplus.com.cn  
Sent: Monday, September 16, 2019 4:53 AM
To: users 
Cc: dev 
Subject: NiFi cluster heavy cpu usage

 

 

I deployed a nifi cluster with two virtual machines,  each  4 core, 16 GB
memroy

The cpu is more than 100% even there's no processor running.

Often the cpu is about 300% after i start some processors. 

 

[root@iZuf6agrzcolqeja3if7kbZ ~]# ps -mp 3429  -o THREAD,tid,time | sort -k2
-n -r |less 

root  171   -- - -  - - 07:08:43

root 16.1  19- futex_-  -  3598 00:40:18

root 16.1  19- futex_-  -  3569 00:40:29

root 16.0  19- futex_-  -  3597 00:40:02

root 15.9  19- futex_-  -  3603 00:39:51

root 15.9  19- futex_-  -  3601 00:39:57

root 15.9  19- - -  -  3604 00:39:57

root 15.8  19- - -  -  3713 00:39:40

root 15.7  19- - -  -  3600 00:39:27

root 15.6  19- futex_-  -  3712 00:39:00

root 15.6  19- futex_-  -  3593 00:39:05

 

 

There's some threads consumes cpu. I pick one and jstack it, often it is in
waiting state.

Any insight on this?

 

Thanks, 

Lei

 

  _  

wangl...@geekplus.com.cn  



smime.p7s
Description: S/MIME cryptographic signature

CaptureChangeMySQL log event entry exceeded max_allowed_packet

[wangl...@geekplus.com.cn]

Is there any other method except  increasing max_allowed_packet on master?
For example， i can just skip it. But how can i do that? 



2019-09-16 20:15:44,370 ERROR [Timer-Driven Process Thread-2] 
o.a.n.c.m.processors.CaptureChangeMySQL 
CaptureChangeMySQL[id=95aa3f6e-e015-1bcf-adf3-2764233af36d] Binlog connector 
communications failure: log event entry exceeded max_allowed_packet; Increase 
max_allowed_packet on master; the first event 'mysql-bin.002567' at 71521781, 
the last event read from './mysql-bin.002567' at 123, the last byte read from 
'./mysql-bin.002567' at 71521800.: 
com.github.shyiko.mysql.binlog.network.ServerException: log event entry 
exceeded max_allowed_packet; Increase max_allowed_packet on master; the first 
event 'mysql-bin.002567' at 71521781, the last event read from 
'./mysql-bin.002567' at 123, the last byte read from './mysql-bin.002567' at 
71521800.
com.github.shyiko.mysql.binlog.network.ServerException: log event entry 
exceeded max_allowed_packet; Increase max_allowed_packet on master; 




wangl...@geekplus.com.cn

NiFi cluster heavy cpu usage

[wangl...@geekplus.com.cn]

I deployed a nifi cluster with two virtual machines,  each  4 core, 16 GB  
memroy
The cpu is more than 100% even there's no processor running.
Often the cpu is about 300% after i start some processors. 

[root@iZuf6agrzcolqeja3if7kbZ ~]# ps -mp 3429  -o THREAD,tid,time | sort -k2 -n 
-r |less 
root  171   -- - -  - - 07:08:43
root 16.1  19- futex_-  -  3598 00:40:18
root 16.1  19- futex_-  -  3569 00:40:29
root 16.0  19- futex_-  -  3597 00:40:02
root 15.9  19- futex_-  -  3603 00:39:51
root 15.9  19- futex_-  -  3601 00:39:57
root 15.9  19- - -  -  3604 00:39:57
root 15.8  19- - -  -  3713 00:39:40
root 15.7  19- - -  -  3600 00:39:27
root 15.6  19- futex_-  -  3712 00:39:00
root 15.6  19- futex_-  -  3593 00:39:05


There's some threads consumes cpu. I pick one and jstack it, often it is in 
waiting state.
Any insight on this?

Thanks, 
Lei



wangl...@geekplus.com.cn