Hi everybody!
I've been testing NiFi 0.3.0 with the GetSQS processor to fetch objects
from an AWS bucket as they're created. My flow looks like this:
GetSQS
SplitJson
ExtractText
FetchS3Object
PutFile
I noticed that GetSQS causes a high amount of CPU usage - about 90% of
one core. If I turn off GetSQS, CPU usage immediately drops to 2%. If I
turn GetSQS back on with the run schedule at 10, it stays at 2%.
Would it be worth using setWaitTimeSeconds [1] to make the SQS receive a
blocking call? Alternatively, should GetSQS default to a longer run
schedule?
Cheers,
Adam
[1]
http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/sqs/model/ReceiveMessageRequest.html#setWaitTimeSeconds(java.lang.Integer)
