Re: PutHDFS with mapr

2018-03-23 Thread Andre
Ravi, There are two ways of solving this. One of them (suggested to me MapR representatives) is to deploy MapR's FUSE client to your NiFi nodes, use the PutFile processor instead of PutHDFS and let the MapR client pump coordinate the API engagement with MapR-FS. This is a very clean and robust

Re: AWS CloudWatch

2018-03-23 Thread Kevin Doran
Hi Laurens, I've never done this but here are some ideas you could experiment with. Assuming the logs are coming from something like an application running on an EC2 instance, there are a number of ways you could probably expose them to NiFi without going through CloudWatch logs. There are a

AWS CloudWatch

2018-03-23 Thread Laurens Vets
Hi list, Has anyone tried to setup NiFi to get real-time CloudWatch logs somehow? I can export CloudWatch logs to S3, but it might take up to 12 hours for them to become available. I suspect the only other option is to go through AWS Kinesis Firehose to stream to S3 and have NiFi pick up the

答复: 答复: put pictures from remote server into hdfs

2018-03-23 Thread 李 磊
Thanks all! I succeed. 发件人: Andrew Grande [mailto:apere...@gmail.com] 发送时间: 2018年3月23日 21:13 收件人: users@nifi.apache.org 主题: Re: 答复: put pictures from remote server into hdfs I think the MOB expectation for HBase was around 10MB. I agree it will require some thought put in organizing the space

Re: 答复: put pictures from remote server into hdfs

2018-03-23 Thread Andrew Grande
I think the MOB expectation for HBase was around 10MB. I agree it will require some thought put in organizing the space and region server splits with column families, once this volume becomes significant. Andrew On Fri, Mar 23, 2018, 9:08 AM Mike Thomsen wrote: > Off

Re: 答复: put pictures from remote server into hdfs

2018-03-23 Thread Mike Thomsen
Off the top of my head, try PutHBaseCell for that. If you run into problems, let us know. As a side note, you should be careful about storing large binary blobs in HBase. I don't know to what extent our processors support HBase MOBs either. In general, you'll probably be alright if the pictures

Re: Store Hash data type in redis

2018-03-23 Thread Mike Thomsen
I don't think there are any processors yet for this sort of thing. I've been thinking about working on some for a while now. How would you expect that hypothetical PutRedisHash processor to work? Here are some example use cases that I've been mulling for building one: 1. Read from attributes with

Store Hash data type in redis

2018-03-23 Thread Sangavi Eswaramoorthi
Hi, I would like to store hash data in redis using NIFi. Is it achievable? Thanks, sangavi