Hi everybody!
I've been testing NiFi 0.3.0 with the GetSQS processor to fetch objects
from an AWS bucket as they're created. My flow looks like this:
GetSQS
SplitJson
ExtractText
FetchS3Object
PutFile
I noticed that GetSQS causes a high amount of CPU usage - about 90% of
one core. If I turn
Hello,
Just to clarify, so you are seeing the messages reach the output port and
then get removed from the queue? And on the spark side the NiFi Spark
receiver never receives anything? Or it receives message, but they have no
content?
-Bryan
On Monday, October 19, 2015, Rama Krishna Manne
wrote
gt; pretty much like:
>
> 1. ListenHTTP => captures attribute LogSrc from HTTP request header LogSrc
>
> 2. MergeContent => where Correlation Attribute Name = LogSrc /
> Attribute Strategy = Keep Only Common Attributes
>
> 3. UpdateAttribute => Updates $filename so tha
I have an flow in which messages are emitted to an output and apache-spark
will pull the messages from the port , I see the messages are pulled by
spark but cannot see the data pulled(cannot do any computations)
I tried a different way , the messages are pushed apache kafka and spark
pulls message
on Attributes
3. UpdateAttribute => Updates $filename so that it is now
data-${now():format('yyyyMMdd')}.log (e.g. data-20151019.log )
4. PutHDFS => Directory = /${LogSrc}/${now():format('/MM/dd')}
(e.g. /mydevice/2015/10/19/)
This, I believe, would result on a file named
/mydev
on Attributes
3. UpdateAttribute => Updates $filename so that it is now
data-${now():format('yyyyMMdd')}.log (e.g. data-20151019.log )
4. PutHDFS => Directory = /${LogSrc}/${now():format('/MM/dd')}
(e.g. /mydevice/2015/10/19/)
This, I believe, would result on a file named
/mydev
hi,
I have a test topic in kafka with 6 partitions. When I use default
PutKafka of nifi to put messages(let’s say 100/batch), I found it usually only
put to one partition.
I have tried the kafka key property to be ‘${nextInt():mod(6)}’ or ‘${uuid()}’,
but it doesn’t seems work. How do I
