Adam, I suspect that getSQS is polling Amazon to check for data. It's not exactly like your standard message broker in that you have to force the poll. Anyway, throw a wait time in there and see if that fixes it. This will also help lower your monthly Amazon bill...
Adam > On Oct 19, 2015, at 11:41 PM, Adam Lamar <adamond...@gmail.com> wrote: > > Hi everybody! > > I've been testing NiFi 0.3.0 with the GetSQS processor to fetch objects from > an AWS bucket as they're created. My flow looks like this: > > GetSQS > SplitJson > ExtractText > FetchS3Object > PutFile > > I noticed that GetSQS causes a high amount of CPU usage - about 90% of one > core. If I turn off GetSQS, CPU usage immediately drops to 2%. If I turn > GetSQS back on with the run schedule at 10, it stays at 2%. > > Would it be worth using setWaitTimeSeconds [1] to make the SQS receive a > blocking call? Alternatively, should GetSQS default to a longer run schedule? > > Cheers, > Adam > > > [1] > http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/sqs/model/ReceiveMessageRequest.html#setWaitTimeSeconds(java.lang.Integer)
