Re: PublishKafkaRecord_2_6 produces negative partition number

","type":["string","null"]},{"name":"Id","type":["string","null"]},{"name":"NumericId","type":"long"},…. > > > > Did I miss something else? > > > >

Re: Back pressure in a NiFi cluster

Hi Alexis, Backpressure is a node-level concept. If there is backpressure on one node, it does not impact other nodes. So the statement "it could be that 1 node is slower and reaches the threshold while the others run normally" is correct. HTH, Pierre Le mar. 4 juin 2024 à 14:54, Alexis Sarda-Es

Re: [JDK] NiFi 1.20.0 comaptibility with JDK21

Hi Pedro, It's hard to have a definitive answer on this as it also depends on what you're using in your flows but you may have some issues. I'd highly recommend using the latest of Apache NiFi 1.x (NiFi 1.26) if you want to run it with Java 21. Thanks, Pierre Le lun. 3 juin 2024 à 13:56, Pedro S

Re: Python Processor and relationships

Hi, Are you in a position to share the code of your python processor? Did you implement getRelationships(self) in your processor? Le ven. 26 avr. 2024, 06:52, Ashwini Kumar a écrit : > I created a Python Nifi Processor just as in the standard python > documentation. This is NOT an ExecuteScript

Re: How to upload flow definition to new 2.0 instance?

In NiFi 1.x, right click on process group, download flow definition. It'll give you a JSON file. In NiFi 2, drag and drop a process group on the canvas, and click the upload icon on the right of the name input. You'll be able to select your JSON file. HTH Le jeu. 11 avr. 2024 à 02:14, James McMah

Re: Using PutIceberg to insert into a table in S3 fails with ClassNotFoundException

I believe that you'd need to build the processor with specific profiles enabled which is not the case in the default build that we provide for convenience. In nifi-inceberg-bundle, you could do something like maven clean install -Pinclude-hadoop-aws And use the generated NARs in your NiFi deployme

[ANNOUNCE] Apache NiFi 1.25.0 Released

The Apache NiFi Team is pleased to announce the release of Apache NiFi 1.25.0. Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. https://nifi.apache.org The release artifacts can be downloaded from the project website. https://nifi.apache.org/download/

Re: NiFi Registry - DatabaseFlowPersistenceProvider and GitFlowPersistenceProvider simultaniously

typo: so you can't run the DB and Git providers simultaneously. Le mar. 16 janv. 2024 à 18:08, Bryan Bende a écrit : > Hello, > > You can have only one persistence provider defined, so you can run the > DB and Git providers simultaneously. > > Thanks, > > Bryan > > On Tue, Jan 16, 2024 at 6:52 A

Re: Embeeded zookeeper or External zookeeper for Nifi Cluster.

ierre, can you explain or indicate some doc about "ZK-less option" for > cluster on Kubernetes ? > > thanks > > Em qui., 4 de jan. de 2024 às 10:15, Pierre Villard < > pierre.villard...@gmail.com> escreveu: > >> Hi Roman, >> >> Embedded Zookeeper

Re: Hardware requirement for NIFI instance

egards, > Matt > > On Thu, Jan 4, 2024 at 8:48 AM Etienne Jouvin > wrote: > > > > Hello. > > > > I also think the problem is more about the processor, I guess ExecuteSQL. > > > > Should play with batch configuration and commit flag to commit >

Re: Hardware requirement for NIFI instance

It should be memory efficient so I think this is likely a configuration aspect of your processor. Can you share the configuration for all properties? As a side note: if NiFi ran out of memory, you'd always want to restart it because you are never sure what's the state of the JVM after an OOME. Le

Re: Embeeded zookeeper or External zookeeper for Nifi Cluster.

Hi Roman, Embedded Zookeeper is really just for convenience for easily setting up a cluster for dev/testing purposes. For production, you'd definitely want to have an external Zookeeper, or use the ZK-less option if running NiFi on Kubernetes. IMO embedded ZK should never be used for production.

Re: MiNiFi data flow parameters

Yeah I do think that what you're asking is a very legit request and it appears that parameter providers cannot be leveraged in the context of MiNiFi. We definitely need to create a properties file based parameter provider and improve the way MiNiFi can leverage parameter providers (possibly by forc

[ANNOUNCE] Apache NiFi 1.24.0 Released

The Apache NiFi Team is pleased to announce the release of Apache NiFi 1.24.0. Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. https://nifi.apache.org The release artifacts can be downloaded from the project website. https://nifi.apache.org/download.

Re: CEF parsing type error

defined long and int fields. There > should be no ambiguity around the length of it, it can be clearly deducted > to be 32 bits. > > That view is supported by the fact 64 bit integer fields have been defined > and clearly diferentiated as part of CEF 1.2. > > Cheers > > O

Re: Bug : PutGCSObject : Version NIFI 1.23.2

rarily push this config in the logback.xml but it is > not working. > > > . > . > . > > > > > > > > . > . > > It is missing something ? > > Thanks a lot for your help > > Regards > > Minh > *Envoyé:* mercredi 8 novembre 2023 à 16

Re: PutGCSObject version 1.23.2

Hi, By default the client library is doing GZIP compression on the data sent to GCS (in order to save bandwidth and limit costs). However... if your data is already compressed or in a format that would not really benefit from compression, you're going to use a lot of CPU on your NiFi nodes for no

Re: Bug : PutGCSObject : Version NIFI 1.23.2

I recall correctly this part) and that's why you got the unknown host exception. Le mer. 8 nov. 2023 à 16:52, Pierre Villard a écrit : > I believe that if you temporarily enable debug logs on the GCP client > library (via logback.xml on com.google.cloud.storage) you'd be able t

Re: Bug : PutGCSObject : Version NIFI 1.23.2

storage/v1/storage/v1/b/xxx/o?name=test/minh.txt&uploadType=resumable > > *Envoyé:* mercredi 8 novembre 2023 à 16:29 > *De:* "Pierre Villard" > *À:* users@nifi.apache.org > *Objet:* Re: Bug : PutGCSObject : Version NIFI 1.23.2 > IIRC, but not in a position to confirm

Re: CEF parsing type error

I may be able to submit a PR against ParseCEF as I did a few improvements in the past but not sure when I'll be able to get to it and how fast a new release would be made available for use in NiFi. Will try to block some time for this over the weekend. Le mer. 8 nov. 2023 à 16:22, a écrit : > O

Re: Bug : PutGCSObject : Version NIFI 1.23.2

IIRC, but not in a position to confirm right now, the right API to set would be https://*storage*-euwest1.p.googleapis.com but your host should still be able to resolve googleapis.com (I believe this can work through proxy as well). Le mer. 8 nov. 2023 à 16:16, a écrit : > Hi > > I have checked

Re: NIFI-REGISTRY Migration

Hi, You could also consider using the recently added Toolkit CLI options to export all / import all from Registry A to B. Pierre Le ven. 29 sept. 2023 à 12:13, Lars Winderling a écrit : > Hi Minh, > > you could try to employ URL aliasing >

Re: GCS processors : endpoint Override URL: " storage-xyz.p.googleapis.com"

Hi, Isn't it available starting from NiFi 1.22? https://issues.apache.org/jira/browse/NIFI-11439 Le mar. 19 sept. 2023 à 09:44, a écrit : > > Hello all, > > Is it possible to add an option in the processor GCS so we can write the > custom url for storage.googleapis.com something like " > storag

Re: AWS MSK

es. > > Nandor > > On 2023. Jul 4., at 10:53, Pierre Villard > wrote: > > No specific secret. It should be fairly straightforward. > How is your MSK configured (wrt authentication)? How are you configuring > your Kafka processors? and if you get errors, what are those? >

Re: AWS MSK

No specific secret. It should be fairly straightforward. How is your MSK configured (wrt authentication)? How are you configuring your Kafka processors? and if you get errors, what are those? Pierre Le mar. 4 juil. 2023 à 00:47, David Grootwassink a écrit : > What is the secret to connecting Ni

Re: Kafka 3 w/ KRaft and ConsumeKafka

Hey Mike, I believe it is transparent, yes. I didn't give it a try though. I know Mark P., Paul G. and David H. are looking into building Kafka 3 components while also significantly refactoring the approach. Le jeu. 15 juin 2023 à 21:53, Mike Thomsen a écrit : > Is ConsumeKafka 2.6 compatible w

Re: Cluster modification

Hi Phil, > Now you're allowed to make changes... meaning the node that was removed > from the cluster now has a flow that is no longer in-sync with the cluster > and you have to remove that node's flow before it's able to rejoin the > cluster. Was this intentional? I know there are a lot of thi

Re: How to configure PrometheusReportingTask

Hi, The property is expecting a port number. When started the reporting task will expose the metrics on that port. You then need to configure Prometheus with a job that will fetch the data from that endpoint. In other words, NiFi is not pushing the metrics, it's making the metrics available via HT

Re: Nifi flowfile monitoring

t; Thanks to all. > > El lun, 2 ene 2023 a las 10:02, Pierre Villard (< > pierre.villard...@gmail.com>) escribió: > >> I think it'd be good to take a step back. How can a flow file get stuck >> in a relationship? 1 - Is it because you have a self loop failure >> r

Re: Nifi flowfile monitoring

I think it'd be good to take a step back. How can a flow file get stuck in a relationship? 1 - Is it because you have a self loop failure relationship? 2 - Is it because you have "dead ends" with a failure relationship going to a funnel? 3 - or is it because a flow file is failing and is being roll

Re: Exacly once from NiFi to Kafka

You don't need to run the Kafka processor only on the primary node. The Kafka client will take care of doing the proper partition-consumer assignment across the nodes and threads in the NiFi cluster. We're aware of many NiFi users consuming data from Kafka at very large scale without any issue dup

Re: Workflows deployments across environments

Hi, You may find this video useful: https://www.youtube.com/watch?v=XYHMExiWM6k The title is misleading since the video does cover automated deployments across environments with NiFi Registry. Note that the recently released NiFi 1.18 version introduces the concept of parameter providers which ca

Re: Multiple GitLab accounts? (Nifi Registry)

Scania CV AB > > Phone: +46 8 553 527 27 Mobile: +46 7 008 834 76 > > Forskargatan 20, SE-151 87 Södertälje, Sweden > > stefan.jakobs...@scania.com > > > > *From:* Pierre Villard > *Sent:* den 16 juni 2022 11:37 > *To:* users@nifi.apache.org > *Subject:

Re: Multiple GitLab accounts? (Nifi Registry)

Hi Stefan, Are you looking at connecting one NiFi Registry to multiple Gitlab repositories? or have multiple users used by the NiFi Registry to perform the git operations against one repository? Pierre Le jeu. 16 juin 2022 à 08:11, Jakobsson Stefan a écrit : > Any info on this query? > > > > *

Re: Nifi 1.16.1 migration failed for encrypted of sensitive values

Hi, I recommend reading the migration guidance documentation: https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance HTH, Pierre Le mer. 4 mai 2022 à 16:46, sanjeet rath a écrit : > Hi , > > I am facing one issue in migration from 1.12 to 1.16.1 . > I have created one 1.16.1 clust

Re: Nifi XML compatibility

Hi Martin, There should not be any compatibility issues. It'd definitely not work if going from a recent version to an older one but 1.7 to 1.15 should work. Thanks, Pierre Le jeu. 17 févr. 2022 à 09:51, Martin Terreni a écrit : > Hi, > > > > I would like to know how far backwards compatibilit

Re: Using Jdbc Driver to connect with phoenix

Not saying this is a problem but, based on the screenshot, it seems you're using NiFi 1.4 which is a *very* old version. Using the latest is probably a good idea. Thanks, Pierre Le mer. 9 févr. 2022 à 18:02, ajaz rahman a écrit : > Hi, > > yes we are using phoenix-client-hbase jar. I have copie

Re: Parameters and Process Groups

Hi, You probably want to read this: https://bryanbende.com/development/2021/11/08/apache-nifi-1-15-0-parameter-context-inheritance Thanks, Pierre Le ven. 4 févr. 2022 à 13:38, James McMahon a écrit : > We have just started to employ parameter contexts rather than environment > variables. I hav

Re: Nifi 1.15.2 and 1.15.3 compilation errors

We recently changed the build to fail if it's including problematic transitive dependencies for log4j. I think those are coming with the specific versions of HBase/Hive you're using. Not a maven expert but I guess there is a way to exclude the step "enforce-banned-dependencies" from the build. Le

Re: Question bigqueryStreaming processor

Hi Franco, I believe this post contains useful information: https://stackoverflow.com/questions/43085896/update-or-delete-tables-with-streaming-buffer-in-bigquery In short: > To check if the table has a streaming buffer, check the tables.get response > for a section named streamingBuffer or, whe

Re: NiFi Fails to Reconnect to Zookeeper After an Outage

Hey Shawn, I think you're looking for https://github.com/apache/nifi/pull/5503 but it's not part of NiFi 1.15. Pierre Le mar. 30 nov. 2021 à 15:58, Shawn Weeks a écrit : > Does anyone know if the patch to the curator library ever made it into > NiFi? Still seeing this issue where once NiFi has

Re: Unable to evolve Schema with HWXRegistry

ile: +49 151 18027982 > > > > E-Mail: christian.we...@sva.de > > http://www.sva.de > > > > Geschäftsführung: Philipp Alexander, Sven Eichelbaum > > Sitz der Gesellschaft: Wiesbaden > > Registergericht: Amtsgericht Wiesbaden, HRB 10315 > -- &

Re: Unable to evolve Schema with HWXRegistry

de > http://www.sva.de > > Geschäftsführung: Philipp Alexander, Sven Eichelbaum > Sitz der Gesellschaft: Wiesbaden > Registergericht: Amtsgericht Wiesbaden, HRB 10315 > -- > *Von:* Pierre Villard > *Gesendet:* Tuesday, October 26, 2021 7:03:36 PM > *An

Re: Unable to evolve Schema with HWXRegistry

Hi Christian, Not sure to understand the overall objective. You're writing the schema reference using Confluent encoding but you have the schema in an Hortonworks Schema Registry instance? Also, what version of NiFi are you using? Thanks, Pierre Le mar. 26 oct. 2021 à 15:46, Weiss, Christian a

Re: ExtractEmailHeaders & ExtractEmailAttachments -- NoClassDefFoundError

Hi Stefan, do you mind filing a JIRA for this? it should have been fixed with https://issues.apache.org/jira/browse/NIFI-6961 Thanks! Le ven. 1 oct. 2021 à 15:11, Stefan Kok a écrit : > For anyone concerned the solution was to replace. > > javax.activation-api-1.2.0.jar > with > javax.activation

Re: UI is not as responsive...

Hi, There is a known issue with Chromium right now that affects the performances when rendering the components. That could be the reason. For Chrome, it is for Chrome 92.x. Just as a side note: a flow with 50 processors is a *very* small flow and should not be causing any issue under normal circum

Re: Disfunctional cluster with version 1.13.2

Hi, What's the version of ZK? Thanks, Pierre Le jeu. 12 août 2021 à 09:55, Axel Schwarz a écrit : > Dear all, > > after successfully battling the load balancing and installing Version > 1.13.2 again in our 3 node production environment, we experienced another > failure in the cluster resulting

Re: [PublishJMS] - Publish MQ message with MQRHF2 format

Hi, It may require a custom processor that would extend the existing one to work with IBM MQ specific capabilities. Thanks, Pierre Le mar. 10 août 2021 à 17:20, Maxime.Lézier a écrit : > > Hello, > > Is there someone who could help for this subject please ? > > Really thanks > > Max L. > > > >

Re: Provenance for SplitRecord has a wrong output claim file

Hi, I believe this is expected. If you have one XML file split into 10 JSON files, what would you expect for the output claim? You can use the provenance event to get the child flow files, and retrieve the claims from there. Also note that one claim file can contain multiple flow files but with di

Re: No Load Balancing since 1.13.2

Hi, I believe the minor u291 is known to have issues (for some of its early builds). Did you upgrade the Java version recently? Thanks, Pierre Le mar. 27 juil. 2021 à 08:07, Axel Schwarz a écrit : > Dear Community, > > we're running a secured 3 node Nifi Cluster on Java 8_u291 and Debian 7 > a

Re: Single node equivalent of cluster view

r (diagnostics)? > > > > Regards, > > > > Isha > > > > *Van:* Pierre Villard > *Verzonden:* dinsdag 20 juli 2021 14:21 > *Aan:* users@nifi.apache.org > *Onderwerp:* Re: Single node equivalent of cluster view > > > > Hi, > > > > You g

Re: Re: Re: Insufficient history, please try again later

lowController.java:3011) > etc. > > And this looks like it it happens every minute, per the snapshot frequency > I listed below. > > > Thanks, > Matt > > > - Original Message - > Subject: Re: Re: Insufficient history, please try again la

Re: Single node equivalent of cluster view

Hi, You go to the hamburger menu, then summary, then system diagnostics (bottom right of the summary view). Thanks, Pierre Le mar. 20 juil. 2021 à 13:32, Isha Lamboo a écrit : > Hi all, > > > > After working with NiFi (1.9 and 1.12) clusters for some time I’m setting > up a single node and I’m

Re: Re: Insufficient history, please try again later

Thanks, > Matt > > > - Original Message - > Subject: Re: Insufficient history, please try again later > From: "Pierre Villard" > Date: 7/19/21 12:07 pm > To: "users@nifi.apache.org" > > Hi, > > You usually need to wait a few minutes a

Re: Insufficient history, please try again later

Hi, You usually need to wait a few minutes after some data went through the component to get something to display. Are you experiencing this issue after a while even though data is going through the component? Please keep in mind that stats are removed after a restart when using the volatile repos

Re: S2S Bulletins -- Error 403

Hi, Transport protocol would need to be HTTP most likely unless you configured the right properties for S2S over RAW. Then did you make sure the input port is for site to site (not sure what version of NiFi you're using)? Also you can check in the nifi-user.log file, that's usually useful for any

Re: ConsumeMQTT Processor Broker URI from environment variable...

Hi, That's something we could easily add, feel free to open a JIRA issue for this improvement. You can also look at using parameters if that's an option for you. To reference a parameter that would be #{} instead of ${}. Thanks, Pierre Le ven. 9 avr. 2021 à 09:51, Yildiray Kabak a écrit : > De

Re: Data duplication When NIFI is restarted

Hi Vibhath, How is NiFi terminated / restarted ? Thanks, Pierre Le mer. 17 mars 2021 à 15:04, Vibhath Ileperuma a écrit : > Hi all, > > I notice that, if the NIFI instance gets terminated while a processor is > processing a flow file, that processor starts to process the flow file > again from

Re: Questions about the GetFile processor

Using the Record Writer will also be much better as you won't output one flow file per listed file. You'll have one flow file with one record per listed file, and you can then use multiple SplitRecord processors to make sure the number of flow files at one point alway remains OK. Le sam. 27 févr.

Re: Issue with GetFTP and arabic letters?

ied PutFTP and it also have some issue. >> I created a flowfile, used an UpdateAttribute to set the filename to: >> امتحان.txt used a PutFTP (with utf8 enabled) and the filename at the ftp >> server become: امتحان.txt >> >> Den tir. 23. feb. 2021 kl. 12.20 skrev P

Re: Issue with GetFTP and arabic letters?

e=امتحان.txt,size=0] > from filename FTP/امتحان.txt on remote host 192.168.1.2:21 due to > java.io.IOException: 550 The system cannot find the file specified. > ; routing to comms.failure: java.io.IOException: 550 The system cannot find > the file specified. > > > K

Re: Issue with GetFTP and arabic letters?

hese 2 processors can take > some non-English characters like Danish and Swedish character which GetFTP > can't. But they still can't read my Arabic characters. > I will try 1.13.0 > > Thanks, > Jens > > Den tir. 23. feb. 2021 kl. 08.17 skrev Pierre Villard < &

Re: Issue with GetFTP and arabic letters?

Hi Jens, You may want to try NiFi 1.13.0 with List+Fetch processors. The addition of https://issues.apache.org/jira/browse/NIFI-7685 may help. Worth a try at least. Thanks, Pierre Le mar. 23 févr. 2021 à 10:56, Jens M. Kofoed a écrit : > Dear community > > I have an issue which I hope some of

Re: How to proper use DistributedCacheServer ?

There will be an Hazelcast implementation (embedded into NiFi) that will be released with NiFi 1.13. It comes with some limitations that you need to be aware of (example: the data is not persisted in case of a cluster restart) but I believe all of this is documented in the "additional details" of t

Re: [E] After upgrade to 1.11.4, flowController fails to start due to invalid clusterCoordinator port 0

Thanks for following up and providing the explanation! Le sam. 13 févr. 2021 à 18:46, Pat White a écrit : > Hi Folks, found the reason for my issue, following up just FYI. > > My local state snapshot, on the updated 1.11.4 nodes, had bad values for > load balancer host and port, so the snapshot

Re: NIFI - Performance issues

Hi Asmath, In addition to excellent answers from Nathan and Jorge, I just want to add that NiFi should be seen as a framework enabling you to build efficient flows. But just like any framework and you can do pretty much anything you want, it can be something good but it can also be something bad.

Re: Connectivity issue (Nifi to postgres)

Hi, We'll need more information like the error message, stack trace, versions you're using for NiFi / Registry / Postgres, configuration, etc. Pierre Le ven. 29 janv. 2021 à 16:48, Arora, Meghna a écrit : > Hello Team, > We are facing some issue in NIFI connection. > > > JDBC Driver of postgre

Re: [E] Re: Lagging worker nodes

>>> I'm assuming also this is the same thing Maksym was asking about >>> yesterday. Let's try to keep the thread together as this gets discussed. >>> >>> On Thu, Jan 28, 2021 at 1:10 PM Pierre Villard < >>> pierre.villard...@gmail.com>

Re: Lagging worker nodes

Hi Zilvinas, I'm afraid we would need more details to help you out here. My first question by quickly looking at the graph would be: there is a host (green line) where the number of queued flow files is more or less constantly growing. Where in the flow are the flow files accumulating for this no

Re: NPE problem in PutSplunk processor

o describe a similar problem but with a > different processor (ExtractHL7Attributes). Might not be related, or maybe > it's copy & pasted code with same bug or some libs being used with a > problem? Just throwing out random ideas. > > > Regards, > Anders > > On Fr

Re: NPE problem in PutSplunk processor

Hi Anders, Do you have a full stacktrace to share (probably available in nifi-app.log file)? It sounds like somethings that could be easily fixed. Thanks, Pierre Le ven. 15 janv. 2021 à 14:03, Anders Synstad a écrit : > Hi. > > Anyone experienced NullPointerExceptions in PutSplunk? The only th

Re: Cross system data lineage

Hi, Just want to also mention that the integration that NiFi provides with Apache Atlas (through a dedicated reporting task) is supposed to provide this kind of end-to-end lineage across multiple components. Thanks, Pierre Le lun. 11 janv. 2021 à 18:58, Chris Sampson a écrit : > Might be worth

Re: ExtractText Improvement

Hi Eric, I do think this would be interesting. Please submit a PR when you feel this is ready for a review. Thanks, Pierre Le lun. 21 déc. 2020 à 21:56, Eric Secules a écrit : > Would this improvement be worthwhile? I'd also like to apply it to all the > regex search/replacement processors, ro

Re: files larger than queue size limit

Yes it should work just fine. The relationship backpressure settings are just soft limits: if backpressure is not enabled, then the upstream processor can be triggered even if the processor generates a huge flow file that would cause the backpressure to be enabled. The backpressure mechanism is onl

Re: GUI functions not available in secure / cluster instance

This is a known problem and, for now, the workaround is to set: nifi.security.allow.anonymous.authentication=true The property name is unfortunately misleading. This does not change anything in terms of security. What it means is that static content of the UI (css, js, png, etc) can be accessed w

Re: ForkRecord 1.12.0 duplicates flowfiles

No worries, glad you sorted it out and let us know. Pierre Le jeu. 10 déc. 2020 à 11:59, muneti a écrit : > Thanks, Pierre. > > When I have reproduced the behaviour in a simple template, i have realized > the mistake was mine: I had two overlapped success queues at the fork > processor input!!

Re: Utilizing NetflowV9 in NiFi v1.12.1

Hi, I'm aware of some people using NiFi for that but I believe they're using NARs available from external Github repositories. We don't have anything packaged OOTB in NiFi itself. For example https://github.com/simonellistonball/nifi-netflow Thanks, Pierre Le mar. 8 déc. 2020 à 20:43, Thane Dure

Re: ForkRecord 1.12.0 duplicates flowfiles

Hi Guillermo, It's not the expected behavior. Any chance you can file a JIRA and attach a template that is reproducing the issue? I can look into it and see if this is normal. Thanks, Pierre Le mer. 9 déc. 2020 à 11:53, Guillermo Muñoz Salgado a écrit : > Hi, Nifi Team!! > > We are using ForkR

Re: Ldap Cluster and Node Identity

Etienne, If you feel like there would be nice additions to have in the documentation, please feel free to share your suggestions to improve it (though a JIRA / pull request). We don't want NiFi users/administrators to have a painful process when configuring all of this. Thanks, Pierre Le jeu. 26

Re: Flow Hotspots

I recall a discussion in the past about displaying the lineage graph with different lengths for the edges of the graph based on the time between the events. I'm not a UI/UX expert and I've no idea how hard this would be to implement but it could be an option. Le ven. 20 nov. 2020 à 14:13, Mike Sof

Re: Rehosting flow files from cluster nodes to primary node only

Hi, You can configure your relationship before the ExecuteScript processor to use the load balance strategy to "single node" so that all the data gets to one node (not necessarily the primary node so you'd still keep the processor configured to be executed on all nodes). Pierre Le ven. 13 nov. 2

Re: Putdatabase- non closeable input stream- json tree reader

Hi, Could you provide a flow template reproducing the issue? Thanks, Pierre Le mar. 13 oct. 2020 à 23:20, KhajaAsmath Mohammed a écrit : > Hi, > > I am seeing lot of issues with our data when using putdatabase with > jsontree reader. JSON is valid when I checked actual data but NIFI is > compl

Re: FW: MiNiFi (Java) long term support

Hi Norbert, Outside of the roadmap of MiNiFi in the Apache community. I just want to say that this community does not provide any kind of commitment regarding long term support, it's community based and best effort. While the NiFi community is great and very active, I'd still recommend you to get

Re: Nifi takes too long to start(~ 30 minutes)

Hi, I'm surprised this is something you observe in the election process part. I've constantly seen quick startup times even with thousands of components in the flow. I'd look into the logs and maybe turn on some debug logs to find out what's going on. Pierre Le jeu. 13 août 2020 à 16:33, Joe Wit

Re: Data Provenance Stops Working

Darren - I'm using NiFi 1.11.4 for a while now (with OOTB configuration) and I do have provenance data for the flows I'm running. Le lun. 10 août 2020 à 23:24, Wyll Ingersoll < wyllys.ingers...@keepertech.com> a écrit : > Ah! That fixed my problem! > > I am running a secure/authenticated configur

Re: early eof

Can you share the full stack trace? Anything specific between your client and the ListenHTTP? Load balancer? I just did some tests and I could upload a 5GB file with no issue. I even tried with the throttling parameters so that the upload takes a very long time and it was fine as well. Le jeu. 6

Re: Monitoring FlowFiles - Lineage Duration

Hi, Both [1] and [2] could give you some ideas based on your needs and what you're using as external systems for monitoring. Overall we're working on making NiFi better on everything around monitoring, but most of the time, for specific requirements I advocate for Monitoring Driven Development [3]

Re: MiNiFi version

I don't recall the specific details but when I used MQTT processors [1] with latest version of MiNiFi Java, I *think* I could not use the latest version of NiFi NARs, I had to use code from a previous NiFi version and cherry-pick some code (open pull request) I needed for my use case [2]. Having sa

Re: NiFi 1.11.4 -- "Unable to access lib/bootstrap to create bootstrap classloader"

Hi Joseph, It looks like there is an issue with the location of the directories/files. What did you change? Did you solve your issue? Thanks, Pierre Le mar. 30 juin 2020 à 16:44, Joseph Wheeler a écrit : > Hello! > > > > I’m trying to deploy NiFi 1.11.4 to a new environment. After configuring

Re: Messed with PutS3Object, add signature on top on content

Glad you figured it out and thanks for sharing your findings with the community! Le ven. 26 juin 2020 à 16:49, Etienne Jouvin a écrit : > Hello all. > > Got it. > It was a matter of signature version. > Because I use s3Ninja, I needed to set with V2 : > [image: image.png] > > > > > > Le ven. 26

Re: Replacing a base64-encoded field in a JSON-document with its decoded/converted value

Hi, I think that would be a perfect use case for NIFI-7572 [1] once there is a solution for it. [1] https://issues.apache.org/jira/browse/NIFI-7572 Le ven. 26 juin 2020 à 08:20, Myklebust, Bjørn Magnar < bjorn.mykleb...@skatteetaten.no> a écrit : > Ok, I see. > > Thanks, Andy. > > > > Bjørn > >

Re: ConsumeKafkaRecord Performance Issue

ConsumeKafka is just taking the messages as raw bytes and writing the content to disk. ConsumeKafkaRecord is serializing/deserializing the data according to a schema and possibly converting the data into another format. That's normal to have a huge performance difference and CPU consumption. Regar

Re: ConsumeKafkaRecord Performance Issue

In addition to Mark's questions: - what version of NiFi are you using? Regarding the number of concurrent threads to use, this should be set according to the number of partitions and I recommend reading this great blog: https://bryanbende.com/development/2016/09/15/apache-nifi-and-apache-kafka No

Re: In memoriam of Jeff Storck

I can't say how much we will miss you Jeff. You were a great guy, always nice and helpful with everyone. You always went the extra mile to make things easier and more robust. RIP Jeff Le lun. 15 juin 2020 à 21:13, Jeremy Dyer a écrit : > This is shocking and heartbreaking news. Jeff was a great

Re: NIFI - Connect processors properly

Hi Asmath, You can double click on the black line of a connection to add bending points. You can add as many as you want. You can also double click on a bending point to remove it. Hope this helps, Pierre Le mer. 29 avr. 2020 à 16:14, KhajaAsmath Mohammed a écrit : > Hi, > > How do I connect t

Re: maven nifi 1.11.4 libs

Hi Josef, It can take a bit before sync is complete between repositories. It should be OK later today I'd imagine. Thanks, Pierre Le lun. 23 mars 2020 à 10:47, a écrit : > As NiFi 1.11.4 has been released yesterday, we tried to upgrade our > processors to 1.11.4. Sadly maven shows no update ye

Re: Flow file movement

Hi, Next to your ListFile/FetchFile processors, you could add a GenerateFlowFile processor with the content of one of your files for testing purposes. Thanks, Pierre Le jeu. 12 mars 2020 à 06:23, Karthick Subramanian a écrit : > Hi All, > > List File -> Fetch File -> Attrribute to CSV -> Merge

Re: Logging/Monitoring of Invalid Processors at NiFi Startup

Hi, You could use the SiteToSiteStatusReportingTask and leverage the "runStatus" field to list all the components (processors, controller services, etc) that are invalid. You could even use the QueryNiFiReportingTask to directly filter only the invalid processors and sink this information somewher

Re: Nifi -Incremental load from database

1 févr. 2020 à 13:19, KhajaAsmath Mohammed a écrit : > Hi , > > I am looking for both solutions actually. > > Thanks, > Asmath > > Sent from my iPhone > > On Feb 21, 2020, at 1:47 PM, Pierre Villard > wrote: > >  > Hi, > > When you mean incremental load

Re: Nifi -Incremental load from database

Hi, When you mean incremental load, do you mean only newly inserted lines in the source database? or do you also want to capture deletes and updates? Thanks, Pierre Le ven. 21 févr. 2020 à 05:16, KhajaAsmath Mohammed a écrit : > Hi, > > I am looking for solution to do incremental load from dat

  1   2   3   4   >