New ListHDFS Filter default in 1.23+ doesn't make sense?

Hi all, I'm testing an upgrade from NiFi 1.18/1.19 to 1.25 and ran into an issue where most ListHDFS that use a filter stopped picking up new files. A filter like .*\\.avro doesn't work anymore with the default setting of "Directories and Files". Switching the mode to "Files only" fixes the

RE: Block start/stop processors

Hi Alexandra, The workaround we use is to always use the right-click context menu on a process group you want to stop/start and never use the button in the lefthand toolbox. Right-clicking the PG also selects it, so you don't have the risk of clicking stop with no selection. Removing the

Documentation on versioned flow format and how nifi handles ids, instanceids and names when importing

Hi all, I'm faced with the challenge of deploying copies of identical flows for several "environments" on the same canvas. With the help of parameter contexts, this mostly works. Where it breaks down is the remote process groups and the remote input ports specified on the connection going to

RE: High CPU load caused by SSL certificate?

Hi all I see the attached image was removed or blocked, here is a link to it: https://i.postimg.cc/pTjM1q3q/High-CPU-SSL-Cert1.png Thanks, Isha Van: Isha Lamboo Verzonden: dinsdag 3 oktober 2023 11:12 Aan: dev@nifi.apache.org Onderwerp: High CPU load caused by SSL certificate? Hi all, I'm

High CPU load caused by SSL certificate?

Hi all, I'm investigating a strange occurrence on one of the NiFi clusters I manage. I'm mailing the dev list rather than users because it seems likely to be a bug or quirk in SSL certificate handling by NiFi or the libraries it depends on. One of our production clusters has been suffering

RE: [DISCUSS] Deprecate TLS Toolkit for Removal?

Hi David, My primary use for the TLS toolkit is for lab deployments, mostly during in-house trainings. I will miss the convenience of having a full set of keystores and truststores ready to go with a single command, but then again, a few commands in a script should replicate this well enough,

RE: DeleteHDFS behavior when idle

ngs. This is how long it will wait between > invocations if there’s nothing for it to do. > > Either way, best to file a Jira, though, to address the behavior for > running unnecessarily when there’s an incoming Connection. > > Thanks > -Mark > > > > On Jun 12, 2023,

DeleteHDFS behavior when idle

Hi all, I have a question about behavior I see on one of our NiFi 1.18 clusters that has a lot of xHDFS processors. When I look at the number of tasks in the summary, the DeleteHDFS processors have a very high number (800-1000+) of tasks even if they have nothing in their incoming queues. The

RE: Issues in Installation of nifi-1.21.0

Hi Urmila, NiFi has a web GUI, so you will need to open it in your browser. The command line window that opens when you use run-nifi-bat will stay like that for as long as you are running NiFi. To use NiFi, you should open the nifi-app.log file in the logs subdirectory. Towards the end of the

RE: [discuss] NiFi support for Hadoop ecosystem components

From the perspective of a NiFi administrator: Removing the xxxHDFS processors anytime soon (2.0) would be a huge issue for us. It shouldn't be, the last Hadoop cluster in our environments was shut down earlier this year. Hive was already gone more than a year ago. But we still have 1000+ HDFS

RE: [discuss] NiFi 1.20 and NiFi 2.0

Hi all, I would like to contribute to the migration tooling (mostly testing I suppose) when that comes up. My team's largest client has a completely template-based pipeline with external scripts replacing variable values before deploying to target clusters, so we've already started looking at

RE: [DISCUSS] Finalizing Release Goals for NiFi 2.0

Hi all, This may be too basic/self-explanatory to count as a goal at all, but will site-to-site interoperability between NiFi 1.x and NiFi 2.x nodes will be preserved? Regards, Isha -Oorspronkelijk bericht- Van: David Handermann Verzonden: zondag 11 december 2022 04:08 Aan: Otto

RE: [DISCUSS] Distributed tracing using OpenTelemetry

Hi Brian, Thank you for bringing this up. I see value here for NiFi deployments in larger (enterprise) environments, where NiFi runs alongside other integration and data management solutions that also support (or plan to support) OpenTelemetry. Moving data between on-prem legacy applications

RE: Need some feedback on how upgrading Avro might cause problems

Hi Mike, The "Infer schema" functionality in NiFi currently generates schemas with the order that will be invalid under Avro 1.9+. I noticed because I've been using that to copy-paste schemas that were "almost right" so I could manually fix them. I guess that inferred schemas should be fine

RE: Nifi toolkit encrypt single properties

ost stable integration from the perspective of an external tool is the encrypt-config.sh command and associated arguments. Regards, David Handermann On Tue, Jan 25, 2022 at 2:42 AM Isha Lamboo wrote: > Hi all, > > I hope this question is appropriate for the developers list, if not, >

Nifi toolkit encrypt single properties

of? Kind regards, Isha Lamboo

Suggestion for System administrator's guide Re: open files limit

be added to /etc/systemd/system/nifi.service [Service] LimitNOFILE=5 LimitNPROC=1 I hope this meets the bar for inclusion. If you need more information or I can contribute this in a different way, please let me know. Kind Regards, Isha Lamboo Data Engineer +31 (0)6 20 50 15 91 ­ [cid:image001