Re: Single Batch Per Node and back pressure

When ControlRate yields, it will yield for the full Yield Duration. So your scenario, Alexis, will allow 10 files through in the first half-second, and when the 11th arrives it will yield for 1 second (if Yield Duration = 1 sec). After that yield ends, it will allow the next 10 files through

Re: Insufficient Permissions - Unable to view the user interface - at WebUI

Hello Alexei, If you have configured an org.apache.nifi.ldap.LdapProvider in your NiFi login-identity-providers.xml for *authentication* then you will also have to configure an org.apache.nifi.ldap.tenants.LdapUserGroupProvider in your NiFi authorizers.xml for *authorization*. Some instructions

Re: Can we access Queued Duration as an attribute?

Hello James, I'm not aware of a way to access Queued Duration using expression language, but you can access the Lineage Duration information. The Getting Started Guide mentions both entryDate and lineageStartDate as immutable attributes on all flowfiles. These are numbers of milliseconds since

Re: ExecuteStreamCommand failing to unzip incoming flowfiles

treamCommand > processor? > > cat $tmptarfile >> /dev/stdout > > > On Thu, Feb 1, 2024 at 10:11 AM Michael Moser wrote: > >> Hi Jim, >> >> The ExecuteStreamCommand will only output 1 flowfile, so using it to >> unzip in this fashion won't yield th

Re: ExecuteStreamCommand failing to unzip incoming flowfiles

Hi Jim, The ExecuteStreamCommand will only output 1 flowfile, so using it to unzip in this fashion won't yield the results you need. Instead, you might try a workaround with ExecuteStreamCommand to unzip your file and then tar to repackage it. Then UnpackContent should be able to read the tar

Re: TLSv1.3 SSLContext not available on Java 11 and RHEL8

You might check if your RHEL8 was installed with FIPS mode enabled (fips-mode-setup --check). FIPS can limit available ciphers which would affect TLS in Java. -- Mike On Tue, Aug 15, 2023 at 1:38 PM Mike Thomsen wrote: > I had similar thoughts and told them to start working with different >

Re: NiFi not rolling logs

Hi Mike, You aren't alone in experiencing this. I think logback uses a pattern matcher on filename to discover files to delete. If "something" happens which causes a gap in the date pattern, then the matcher will then fail to pick up and delete files on the other side of that gap. Regards, --

Re: source build error?

If I recall correctly, the macnificent.jar is not actually stored on central.maven.org but is instead located on mvn.martiansoftware.com. Is it possible that you have a proxy of some sort that is not allowing access to that site? -- Mike On Thu, Feb 14, 2019 at 4:56 PM Otto Fowler wrote: >

Re: SSL Mutual Authentication with PutS3Object

Hello Michael, The PutS3Object processor has an SSL Context Service property which would be used to create SSL connections. You would configure one of those with your certs and keys. Since you are behind a gateway, which likely acts as a proxy, you might need to look into the proxy related

Re: A question about [MergeContent] processor

The inner workings of MergeContent is certainly a FAQ. This message [1] to the users list from a long time ago may help. I think it's still accurate. [1] - https://lists.apache.org/thread.html/5ab5d9d0bcd0eef8ace391d00f5f5678427bee4b2fbf1e48d78ea8c8@1445464430@%3Cusers.nifi.apache.org%3E

Re: How do you use a custom region?

Greetings! This JIRA ticket [1] describes the recommended work around for AWS regions that aren't in the list. -- Mike [1] - https://issues.apache.org/jira/browse/NIFI-4523 On Wed, Nov 28, 2018 at 11:44 AM Jon Logan wrote: > Andrew, > > I know there's a few regions not in the list. I'm not

Re: Who uses NiFi Cluster in Docker ?

I have done exactly what Juan Pablo Gardella suggested in my own Docker sandbox, and also a *ADDNIFI_* function. It would take a lot of cleanup and documentation in order to contribute, but if there is interest in it, then I'll see what I can do. -- Mike On Fri, Oct 19, 2018 at 1:51 PM Juan

Re: After 1.7.1 upgrade, no Provenance data is visible

Hi Peter, There was a change to provenance related access policies in 1.7.0. Check out the Migration Guide [1] for 1.7.0. It talks about what you'll need to do. [1] - https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance -- Mike On Fri, Aug 10, 2018 at 5:39 PM Peter Wicks

Re: NiFi 1.6.0 cluster stability with Site-to-Site

When I read this I thought of NIFI-4598 [1] and this may be what Joe remembers, too. If your site-to-site clients are older than 1.5.0, then maybe this is a factor? [1] - https://issues.apache.org/jira/browse/NIFI-4598 -- Mike On Fri, Aug 10, 2018 at 4:43 PM Joe Witt wrote: > Joe G > > I do

Re: Dynamically loading JARs and using them with a controller service

Hey Mike, As long as it's a controller service PropertyDescriptor that uses dynamicallyModifiesClasspath, check out the JMSConnectionFactoryProvider in the nifi-jms-bundle. -- Mike On Sat, Jul 28, 2018 at 8:52 AM Mike Thomsen wrote: > Is there a good example somewhere that shows how to use >

Re: Data not loading to Redshift

t; > I am not sure if this helped but once the Nifi IP was whitelisted on the > security group of redshift cluster, I was able to see the data in the > table. Do you think this would have made the difference? > > > > Thanks > > Aruna > > > > *From:* Michael Moser [mailto:mo

Re: Data not loading to Redshift

Hello, Unfortunately the nifi-app.log that you included did not offer any clues to why you are experiencing a problem. If you would, please edit your nifi/conf/logback.xml file and find the line below. Change the WARN to DEBUG, wait a minute for NiFi to reload the file, and then run your test

Re: ConsumeJMS with ActiveMQ over SSL in NiFi 1.4.0 (cross-post)

I placed some sample code in a Github project [1] to give you an idea of how a custom JMS ConnectionFactory provider can be created for Apache NiFi. [1] - https://github.com/mosermw/nifi-jms-providers-bundle -- Mike On Thu, Dec 7, 2017 at 7:28 AM, jhershcopf wrote:

Re: NiFi Slowness after thousand processors

Greetings! One thing that you could check is how many processors you have in the STOPPED state. The NiFi framework will perform periodic validation on all STOPPED processors, including all controller services referenced by those processors. This can have the side effect of slowing UI

Re: ConsumeJMS with ActiveMQ over SSL in NiFi 1.4.0 (cross-post)

Hi Brian, I noticed the same behavior with the JMSConnectionFactoryProvider, i.e. I could not figure out how to configure it to work with ActiveMQ SSL. I ended up writing my own ActiveMQConnectionFactoryProvider which instantiates the ActiveMQSslConnectionFactory class. It was much easier for

Re: Service controller list NiFi 0.x to 1.x

Hi Russ, In NiFi 1.2.0, there are 3 Reporting Tasks (SiteToSiteBulletionReportingTask, SiteToSiteProvenanceReportingTask, and SiteToSiteStatusReportingTask) that have the capability of using a Controller Service (StandardSSLContextService). So yes, the Global -> Controller Settings has a limited

Re: PublishAMQP had been working, now failing

James, Try leaving your PublishAMQP "SSL Context Service" property blank. In NiFi 0.6.x, PublishAMQP did not support SSL connections. If that NiFi version was working for you, then you should not use the SSL properties of PublishAMQP in NiFi 0.7.x. The "bad header" message may be RabbitMQ

Re: Threads

Greetings, Tuning NiFi for optimal performance on a specific platform depends on which processors you use. If you use a lot of Put* processors (like PutHDFS or PutS3Object) then your NiFi performance is likely going to be bound by I/O delays. In this case your NiFi system can probably handle

Re: nifi process running at 114% of node CPU !!?

I went ahead and created NIFI-2268 for this, since it was fresh in my mind. ListenHTTP calls ProcessContext.yield() whenever it doesn't have work to do, so HandleHttpRequest could do the same. -- Mike On Thu, Jul 14, 2016 at 7:00 PM, Joe Witt wrote: > Mike, > > If you

Re: Passwords in EncryptContent

If you modify a flow.xml by hand and put a plaintext value into any sensitive property, then the next time you start NiFi and it saves the flow.xml.gz NiFi will encrypt the plaintext value. Any change to the flow (such as starting, stopping, reconfiguring, creating, deleting or moving a

Re: Secure site-to-site error

Kenny, Also please check your nifi/conf/authority-providers.xml on the NCM and Nodes in your cluster. I have seen site-to-site authorization problems when this isn't setup correctly. The NCM knows about all clients that are expected in site-to-site communication via Remote Process Group, but

Re: Interaction between MergeContent parameters

Hi David, With Merge Strategy set to bin-packing algorithm and no Correlation Attribute Name set, then you are most likely using just 1 bin already. You didn't say if you had a Max Bin Age set. If you do, then MergeContent could definitely produce an output file of a few MB if you don't have a

Re: Interaction between MergeContent parameters

Hello Andre, The MergeContent processor certainly can be challenging to understand its inner workings. If you are running into the nifi.queue.swap.threshold limit of MergeContent as described in NIFI-697, then you should increase that value in the nifi.properties file and restart your NiFi