Unsubscribe

Re: Nifi flowfile monitoring

Activate prometheus exporter and you will have a metric for number of objects in the queue > On 31. Dec 2022, at 07:08, nayan sharma wrote: > > Hi Users, > Is there anyway through which I can monitor or raise alert if any flow file > got stuck in nifi queue. > > For now operation team needs

Re: Failing to start - keystore properties invalid

Hi James, Can it be that you are trying to start nifi with ssl without authentication ? Looks like that.. > On 27. Dec 2022, at 22:13, James McMahon wrote: > > Hello. I am trying to start a secure instance of nifi version 1.16.3. I am > getting this error on start attempt: > > 2022-12-27 2

Workflows deployments across environments

Hello Nifi users, Question in the round. Let’s say I have 3 Nifi instances with dev, test and prod. What is the recommend way of deploying specific workflows from one environment to another ? We are using Nifi registry + GitHub as storage for the flows. What we want to achieve: * in produc

Re: Minifi and ssl config on NiFi

ssues you may run into. > > Regards, > Matt > > >> On Apr 17, 2022, at 11:40 AM, Jorge Machado wrote: >> >> I did this on the pass and I end up switching to Nifi. I think you should >> do the same. Minifi is kind of “Dead” not being developed anymore. I

Re: Minifi and ssl config on NiFi

I did this on the pass and I end up switching to Nifi. I think you should do the same. Minifi is kind of “Dead” not being developed anymore. I found better to just switch to single instance of nifi Regards Jorge > On 17. Apr 2022, at 03:30, David Early wrote: > > We are considering using se

Re: Adding a node to a cluster

Hey, Are you using the zookeeper internally or an external one. I think you need to check at least two things: * Zookeeper connections. The node needs to register into zookeeper. * The node needs to connect to the cluster and replicate the flow files let’s say so. Overall scaling up is ok

Re: InvokeHTTP vs invalid SSL certificates

Just import the certificate into the trust store. > On 4. Mar 2022, at 13:59, Jean-Sebastien Vachon > wrote: > > Hi all, > > what is the best way to deal with invalid SSL certificates when trying to > open an URL using InvokeHTTP? > > > Thanks > > Jean-Sébastien Vachon > Co-Founder & Arc

Re: Round robin load balancing eventually stops using all nodes

Are you are a dedicated port for transferring the data or using the http protocol ? I had similar issues with a remote port connection that got solved by not using the http protocol. > On 3. Sep 2021, at 14:13, Mike Thomsen wrote: > > We have a 5 node cluster, and sometimes I've noticed that

Re: No Load Balancing since 1.13.2

Did you tried java 11 ? I have a client running a similar setup to yours but with a lower nigh version and it works fine. Maybe it is worth to try it. > On 27. Jul 2021, at 12:42, Axel Schwarz wrote: > > I did indeed, but I updated from u161 to u291, as this was the newest version > at that

Re: SocketTimeoutExceptions and JVM version

Is your input configured to be http or binary ? I can recommend using socket connection instead of https. You need an extra port but it could be worth. My last client we had similar issues. Ref: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html

Re: RecordPath...With INNER records

Hey Greene, The LookupRecord as a RecordPath as input. Check out this docs: https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html#structure or https://www.nifi.rocks/record-path-cheat-sheet/

Re: NiFI fronend disconnects after some time

How much memory do you have on your instance? I would say the fetch size should be bigger then the Max Rows… > On 1. Mar 2021, at 06:37, Vibhath Ileperuma > wrote: > > Hi all, > > I tried to fetch a large dataset from postgresql using the ExcureSQL > processor. > The configurations I used

Change internal state of a processor

Hello everyone, I think we need a tool that allows us to change the internal state of a processor. Like for example I would like to got 10 steps back from the state that has a value of 10. The resulting state would be 0 We could just overwrite what is there… Thanks

Re: Detect duplicate record reader

Hey Jeremy, Something linke this https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.DetectDuplicate/index.html

GenerateTableFetch Question

Hey again everyone, Is it possible that the GenerateTableFetch passes the max value into the table name field before it executes it ? Background for this I have a stored procedure that I pass in as table name. This needs two parameters a min and a max. Where min should be the maximum value t

Re: How to proper use DistributedCacheServer ?

rs.standard.PutDistributedMapCache/index.html> > > --- > Chris Sampson > IT Consultant > chris.samp...@naimuri.com <mailto:chris.samp...@naimuri.com> > <https://www.naimuri.com/> > > > On Fri, 12 Feb 2021 at 14:48, Jorge Machado <mailto:jom...@me.co

How to proper use DistributedCacheServer ?

Hey everyone, Is there any documentation on how to use DistributedCacheServer ? Currently from what I see this is single point of failure or does it really sync the data between nodes ? I want to have something similar to zookeeper state but not in zookeeper because it needs to be available

Re: [E] After upgrade to 1.11.4, flowController fails to start due to invalid clusterCoordinator port 0

For cluster mode check the configs that are on xml files. I had similar issues when I did not define the values. Letting them empty makes issues. Best regards Jorge CEO of Datamesh GmbH (www.dmesh.io) > On 9. Feb 2021, at 02:19, Pat White wrote: > > Thanks very much for the feedback Joe, much

Re: NIFI - Performance issues

Another thing to look would be to check if you are creating two much flow files as this hammers the disks. If you see very high memory usage it could be you are having to much data in attributes and not in the content of the flow file. > On 8. Feb 2021, at 07:07, nathan.engl...@bt.com wrote: >

Re: External Access using InvokeHTTP_Test processor and StandardSSLContextService

Hi Dan, Seems like this is a jvm issue. Try this: https://confluence.atlassian.com/kb/unable-to-connect-to-ssl-services-due-to-pkix-path-building-failed-error-779355358.html

Re: Urgent: HDFS processors throwing OOM - Compressed class space exception

How big are the files that you are trying to store? How much memory did you configure nifi ? > On 22. Jul 2020, at 06:13, Mohit Jain wrote: > > Hi team, > > I’ve been facing the issue while using any HDFS processor, e.g. - PutHDFS > throws the error - > Failed to write to HDFS due to compre

Re: Nifi - how to achieve a concurrent development and CI/CD

like your whole team developing and pushing directly to master. > > > -Eric > > On Thu., May 14, 2020, 7:02 a.m. Jorge Machado, <mailto:jom...@me.com>> wrote: > Hi, > > Managing xml is always hard I think. Last time I need to do something similar > we used

Re: Nifi - how to achieve a concurrent development and CI/CD

Hi, Managing xml is always hard I think. Last time I need to do something similar we used https://nifi.apache.org/registry.html Works pretty well It was already 2 Years ago. Maybe now there is something better > On 14. May 2020, at 15:57, Michal Slama

How to deal Schema Evolution with Dataset API

Hello everyone, One question to the community. Imagine I have this Case class Person(age: int) spark.read.parquet(“inputPath”).as[Person] After a few weeks of coding I change the class to: Case class Person(age: int, name: Option[String] = None) Then when I run th

API to put the Canvas in Read Mode

create a PR for this Jorge Machado

Re: NiFi 1.6

Voting is taking now place from RC3. Probably in the next two weeks. Jorge > On 5 Apr 2018, at 22:46, dan young wrote: > > any updates on when 1.6 is going to drop? > > dano >

Re: How to know the processors last running time?

te? > > On Mon, Mar 19, 2018 at 3:50 PM, Jorge Machado <mailto:jom...@me.com>> wrote: > you could end your flow with a logAttribute Processor for example. Would that > resolve your issues ? > > Jorge Machado > > > > > > >> On 19 Mar 2018, at

Re: How to know the processors last running time?

you could end your flow with a logAttribute Processor for example. Would that resolve your issues ? Jorge Machado > On 19 Mar 2018, at 07:57, prabhu Mahendran wrote: > > i'm having group of processors to perform some functionalities. > > I have scheduled the star

Next Release

Hi Guys, When is a new Release planed ? Is there any page where we can see a roadmap ? Thanks Jorge Machado

Re: InvokeHttp -- StandardSSLContextService Validator Exception

Any trace for us ? Working Example: Jorge Machado > On 13 Mar 2018, at 13:11, Jones, Patrick L. wrote: > > Howdy, > > I’m using a StandardSSLContextService with InvokeHttp and I get a > ValidatorException ‘unable to find valid certification path to requested >

Re: Multiple versions of .... exist.

Yes, I opened a PR: https://github.com/apache/nifi/pull/2531/files <https://github.com/apache/nifi/pull/2531/files> Jorge Machado > On 12 Mar 2018, at 09:46, Ed B wrote: > > Looks like a bug. The code should be: > > dto.setArtifact(bundle.getArtifa

Re: Multiple versions of .... exist.

After some time of debugging I found this here: How is this possible ??? Group is still null after being set.. the same happens for version. Jorge Machado > On 12 Mar 2018, at 08:46, Jorge Machado wrote: > > Hello everyone, > > I we setting up the Nifi registry are

Multiple versions of .... exist.

Hello everyone, I we setting up the Nifi registry are we are getting this error when trying to import a flow from Registry into Nifi: Multiple versions of …. exist. No exact match for default:...:unversioned. To note that we have multiple versions of the same processor in the lib folder (whic

Smarter org.apache.nifi.nar.NarUnpacker to get proper versions

to all production processors and press change version button. What is the approach to go here ? Thanks Jorge Machado

Re: Disconnect nifi node cluster via API

Hi Matt thanks for the reply, that’s is what I was looking into it ! Jorge Machado mfg > On 22 Feb 2018, at 13:55, Matt Gilman wrote: > > Jorge, > > A node must be disconnected before it can be removed. If you're looking to > invoke the REST endpoints, I would

Disconnect nifi node cluster via API

Hi guys, is there an api to disconnect a node from the cluster ? This would be great to securely automate a rolling restart. Jorge

Re: Deploying flows securely over rest api

Hi Sean, Which error are u getting. It the API has the option for it It should work. may be this helps: https://github.com/hermannpencole/nifi-config <https://github.com/hermannpencole/nifi-config> Jorge Machado > On 21 Feb 2018, at 13:40, Sean Marciniak wrote: > > He

Rolling restart of Nifi

the cluster via hostname. is this possible ? Thanks Jorge Machado