Re: Nifi startup error after I install my first custom processor

The dependencies can definitely be confusing to wrap your head around. A good rule of thumb is that you would generally not have a direct jar dependency on anything under nifi-nar-bundles in the source tree. This would mean no jar dependencies on processors, controller services, reporting tasks

Re: Release wrangling: 1 week until our hopeful 0.4.0 release

Getting closer! https://issues.apache.org/jira/browse/NIFI-1122?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%200.4.0%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC Key stuff left: - Wrap up NIFI-655 uname/pword stuff - Make the kafka

Re: Capacity Planning : Guidelines

Venkatesh, I will note that recently, S3 has provided the ability to register notifications via SQS whenever buckets are updated. So if you want to use S3 in a scalable fashion, that is far more do-able now. Generally, the idea is to register S3 to provide a notification via SQS and then in

Capacity Planning : Guidelines

Are there any guidelines on how-to scale up/down NiFI ? (I know we don;t do autoscaling at present and nodes are independent of each other) The use-case is : 16,000 text files (csv, xml, json)/per minute totalling 150Gb are getting delivered onto a combination of FTP, S3, Local Filesystem etc.

Re: Basic Authentication

Edgardo Yep. What are some others you'd be looking for? What we're basically doing is preferring a delegated provider model. Kerberos is one we plan to knock out as well Thanks Joe On Fri, Nov 20, 2015 at 7:10 AM, Edgardo Vega wrote: > Joe, > > Yes I was looking for

Re: Capacity Planning : Guidelines

Hello So that is about 300MB/s if that really was 150Gb (not GB) per minute. If you expect each node to be able to handle say 75MB/s of throughput (which would be low but i'm being conservative) then you'd need 4 or so boxes to hit that rate. Then assume there will be surges in arrival and lulls

Re: Basic Authentication

Joe, Yes I was looking for username and password. Seems like NIFI-655 will setup the base to allow for other username/password authentication providers other than LDAP and AD. Cheers, Edgardo On Thu, Nov 19, 2015 at 5:31 PM, Joe Witt wrote: > i conflated two different

Re: Release wrangling: 1 week until our hopeful 0.4.0 release

FWIW I submitted at patch for lineendings that I don't believe will conflict with NIFI-655 On Nov 20, 2015 8:57 AM, "Joe Witt" wrote: > Getting closer! > > >

Re: Capacity Planning : Guidelines

Venkatesh, Ok. So 2.5GB/s of throughput then. That is pretty significant volume. As you mentioned much of it is text oriented so there is a good chance compression ratios will be quite considerable. Could get the edge nodes to reduce it down to say 600MB/s overall. At that point you'd need

[GitHub] nifi pull request: NIFI-1192 added support for Dynamic Properties ...

Github user olegz commented on the pull request: https://github.com/apache/nifi/pull/129#issuecomment-158542457 @trkurc @joewitt @markap14 Guys, please see the updated commit. This one has a bit more work as it includes some polishing since we discovered few more issues, one

Re: Nifi startup error after I install my first custom processor

Bryan, thanks so much for the pointers. I got is all working now and ready to start on the next processor with a whole lot more confidence. :) FWIW, maybe for someone else who needs this information, I just added a dependency for logback-classic to get more than the default slf4f-simple logging

[GitHub] nifi pull request: NIFI-1192 added support for Dynamic Properties ...

Github user olegz closed the pull request at: https://github.com/apache/nifi/pull/129 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is