Re: Nifi provenance indexing throughput if it is being used as an event store

Can I expect the Nifi search provenance part do the job for me? On Fri, 15 Feb. 2019, 13:21 Mike Thomsen Ali, > > There is a site to site publishing task for provenance that you can add as > a root controller service that would be great here. It'll just take all of > your provenance data periodic

Re: Nifi provenance indexing throughput if it is being used as an event store

Ali, There is a site to site publishing task for provenance that you can add as a root controller service that would be great here. It'll just take all of your provenance data periodically and ship it off to another NiFi server or cluster that can process all of the provenance data as blocks of JS

Re: Asymmetric push/pull throughput with S2S, possibly related to openConnectionForReceive compression?

Hi Andy, My requirement is to use https with minimum tls v1.2, https being an approved protocol. I haven't looked at websockets though, i need to do that, thank you for the suggestion. patw On Thu, Feb 14, 2019 at 12:24 PM Andy LoPresto wrote: > Pat, > > Just to clarify, your connection must

Re: Asymmetric push/pull throughput with S2S, possibly related to openConnectionForReceive compression?

Thank you Bryan, i have a lot to look at using raw protocol, will be sure to keep this setting in mind. patw On Thu, Feb 14, 2019 at 12:05 PM Bryan Bende wrote: > You can use TLS with raw s2s by setting nifi.remote.input.secure=true > > On Thu, Feb 14, 2019 at 12:56 PM Pat White > wrote: > > >

Nifi provenance indexing throughput if it is being used as an event store

Hi All, I am investigating to see how Nifi provenance can be used as an event store for a long period of time. Our use case is very burst based and sometimes we may not receive any event for a period of time and sometimes we may get burst traffic. On average we can say maybe around 1000 eps is the

Re: source build error?

If I recall correctly, the macnificent.jar is not actually stored on central.maven.org but is instead located on mvn.martiansoftware.com. Is it possible that you have a proxy of some sort that is not allowing access to that site? -- Mike On Thu, Feb 14, 2019 at 4:56 PM Otto Fowler wrote: > Yo

Re: source build error?

You should send this to d...@nifi.apache.org On February 14, 2019 at 15:10:30, l vic (lvic4...@gmail.com) wrote: Any idea what the problem is? nifi-standard-processors: *Could not resolve dependencies for project org.apache.nifi:nifi-standard-processors:jar:1.6.0: Failure to find com.martianso

source build error?

Any idea what the problem is? nifi-standard-processors: *Could not resolve dependencies for project org.apache.nifi:nifi-standard-processors:jar:1.6.0: Failure to find com.martiansoftware:macnificent:jar:0.2.0*

Re: InvokeScriptedProcessor initialize method

Hi Matt, Thank you taking the time to explain and provide necessary inputs. Regards, Harsha Sent from Outlook From: Matt Burgess Sent: Thursday, February 14, 2019 12:59 PM To: users@nifi.apache.org Subject: Re: InvokeScriptedProcessor

Re: Asymmetric push/pull throughput with S2S, possibly related to openConnectionForReceive compression?

Pat, Just to clarify, your connection must be HTTPS or it just must be secure? What about Websockets over TLS (wss://)? Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Feb 14, 2019, at 9:56 AM, Pat White wr

Re: Asymmetric push/pull throughput with S2S, possibly related to openConnectionForReceive compression?

You can use TLS with raw s2s by setting nifi.remote.input.secure=true On Thu, Feb 14, 2019 at 12:56 PM Pat White wrote: > > Thanks very much folks, definitely appreciate the feedback. > > Right, required to use tls/https connections for s2s, so raw is not an option > for me. > > Will look furthe

Re: InvokeScriptedProcessor initialize method

Harsha, You are correct, the ScriptEngine is supposed to destroy all the objects created by the script when the scripted Processor instance is destroyed (which it should be on stopped). However not all objects "like to be destroyed" and want API methods to be called on them first. This IMHO is a b

Re: Asymmetric push/pull throughput with S2S, possibly related to openConnectionForReceive compression?

Thanks very much folks, definitely appreciate the feedback. Right, required to use tls/https connections for s2s, so raw is not an option for me. Will look further at JettyServer and setIncludedMethods, thanks again. patw On Thu, Feb 14, 2019 at 11:07 AM Mark Payne wrote: > Pat, > > It appear

Re: Asymmetric push/pull throughput with S2S, possibly related to openConnectionForReceive compression?

Pat, It appears to be hard-coded, in JettyServer (full path is nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java ) Line 294 calls the gzip method, which looks like: private Handler gzip(final Handler handler

Re: Asymmetric push/pull throughput with S2S, possibly related to openConnectionForReceive compression?

I admit I haven't read this thread in detail, but is it a requirement for you to use HTTP site-to-site? I would think you could avoid this issue by using traditional raw site-to-site which is over a direct socket and not hitting Jetty. If you do want to modify Jetty's configuration you would have

Re: InvokeScriptedProcessor initialize method

Hi Matt, Thank you for the quick response. I took a while to try and understand how initialize and validate are working. I have tried a few cases and noticed that: 1. Yes the ObjectIds' are different and 2. Validate only runs periodically when the processor is stopped. When the processor

Re: Asymmetric push/pull throughput with S2S, possibly related to openConnectionForReceive compression?

...interesting. I dont have an answer but will initiate some research. Hopefully someone else replies if they know off-hand. Thanks On Thu, Feb 14, 2019 at 11:43 AM Pat White wrote: > Hi Folks, > > Could someone point me at the correct way to modify Nifi's embedded jetty > configuration settin

Re: Asymmetric push/pull throughput with S2S, possibly related to openConnectionForReceive compression?

Hi Folks, Could someone point me at the correct way to modify Nifi's embedded jetty configuration settings? Specifically i'd like to turn off jetty's automatic compression of payload. Reason for asking, think i've found my performance issue, uncompressed input to jetty is getting automatically co

Re: Nifi registry Kerberos Auth with Docker

Glad it is working for you! I replied to the points you raised inline below. On February 14, 2019 at 00:52:06, Tomislav Novosel (to.novo...@gmail.com) wrote: > Hi Kevin, > > Thank you for your suggestions. I succeeded to get everything working now. > As you described, now is everything exectly

Re: InvokeScriptedProcessor initialize method

Harsha, This is a feature/bug related to the difference between the lifecycle events handled by InvokeScriptedProcessor (ISP) itself and the scripted processor to which some lifecycle events get delegated. For normal processors (including ISP), the initialize() method is called when the processor

Re: Why RedisDistributedMapCacheClientService does not support clustered mode redis?

Redis has three modes standalone, sentinel, and cluster. Sentinel is an HA configuration with replicas and auto failover, but each node has a full copy of the data. Cluster is a sharded key space so data is divided among the nodes. In order to implement atomic compare and set operations required

InvokeScriptedProcessor initialize method

Hi All, I have a question regarding the behavior I'm noticing in InvokeScriptedProcessor. I was testing a functionality and noticed that initialize method in InvokeScriptedProcessor is called twice during the life cycle of one full run. To reproduce this I kept simple print statement in the in

Re: Why RedisDistributedMapCacheClientService does not support clustered mode redis?

I am not NiFi dev, but personally, after looking at DistributedCache processors we decided not to use them. There is nothing "distributed" about them. It will be a single point of failure in your flows, difficult to manage (you have to use NiFi to read/write/delete keys). And I also looked at Redis

Can't open summary-systems diagnostics

Hi all, I installed Nifi in cluster mode with two nodes and both secured with Kerberos for user Auth. When I want to see summary->system diagnostics I've got error: An unexpected error has occurred in UI and in the log: ERROR [NiFi Web Server-80406] o.a.nifi.web.api.config.ThrowableMapper An u

Why RedisDistributedMapCacheClientService does not support clustered mode redis?

Hi all, I am trying to configure the RedisDistributedMapCacheClientService, and nifi told me that this controller service does not support the redis running in cluster mode. In our business, we already have a redis cluster, and I do not want to set up a new redis alone for this controller service.