[ANNOUNCE] Apache NiFi 1.11.4 release

Hello The Apache NiFi team would like to announce the release of Apache NiFi 1.11.4. Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. Apache NiFi was made for dataflow. It supports highly configurable directed graphs of data routing, transformation,

Re: GRPC processors missing after upgrade to 1.11.3

Dirk Apologies on the grpc bundle being included. Just a mistake on my part during the RM process. Thanks Joe On Mon, Mar 9, 2020 at 9:18 AM Bryan Bende wrote: > Hello, > > We likely need Joe to confirm about grpc since he was the RM for the last > few releases, but I believe that it was

Re: Data performance with FlowFile Repo's RocksDB

Ryan By far the largest performance relevant activity is flow design itself. As a last resort I'd look at repo changes. Are you using the record processors? Does your data arrive in batches? Thanks On Thu, Sep 10, 2020 at 7:27 AM Ryan Hendrickson < ryan.andrew.hendrick...@gmail.com> wrote:

Re: Long Polling Client

Clay Have you evaluated whether InvokeHTTP will give you the desired behavior for your case - in particular with a long timeout perhaps? If you have and it doesn't do the job do you mean something which initiates a request to an HTTP server then assumes the response will remain open and it

Re: Long Polling Client

streams data back to the client on the same > connection. I am able to have the data streamed to the client if I set up a > stand alone HTTP client and issue HTTP requests but I am not able to get > getHTTP or invokeHTTP work (I don't get any messages back). > > thanks. > > On Wed,

Re: Unable to rollback changes with Nifi Registry

Juan Probably best to create a jira with all the details you provided here so someone can look into it when available Thanks On Mon, Oct 5, 2020 at 12:00 PM Juan Pablo Gardella < gardellajuanpa...@gmail.com> wrote: > Hi all, > > Does anyone know about this issue? What does it mean? > > Thanks,

Re: sslcontext certs

Michael, There is not any specific way supported or intended to combine the context used by NiFi's own HTTP server with those that would be used by processors within the flow. However, using parameter contexts here is a great way to ensure you have only a single place to update for flow

[ANNOUNCE] Apache NiFi 1.12.0 release

Hello The Apache NiFi team would like to announce the release of Apache NiFi 1.12.0. This release includes over 330 bug fixes, improvements and many new features. Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. Apache NiFi was made for dataflow. It

Re: NiFi V1.9.2 Performance

Nathan You have plenty powerful machines to hit super high speeds but what I cannot tell is how the disks are setup/capability and layout wise and relative to our three repos of importance. You'll need to share those details. That said, the design of the flow matters. The Kafka processors that

Re: NiFi V1.9.2 Performance

ConsumeKafka processor is reading > from all topics in one consumer. I’m assuming the attribute kafka.topic > attribute doesn’t exist at this stage? We use the Avro Schema Registry > Controller as we don’t have a schema registry in place yet. > > > > Kind Regards, > > > &

Re: Content Claims Filling Disk - Best practice for small files?

can you share your flow.xml.gz? On Thu, Sep 17, 2020 at 8:08 AM Ryan Hendrickson < ryan.andrew.hendrick...@gmail.com> wrote: > 1.12.0 > > Thanks, > Ryan > > On Thu, Sep 17, 2020 at 11:04 AM Joe Witt wrote: > >> Ryan >> >> What version are you usin

Re: Content Claims Filling Disk - Best practice for small files?

Ryan What version are you using? I do think we had an issue that kept items around longer than intended that has been addressed. Thanks On Thu, Sep 17, 2020 at 7:58 AM Ryan Hendrickson < ryan.andrew.hendrick...@gmail.com> wrote: > Hello, > I've got ~15 million FlowFiles, each roughly 4KB,

Re: Looking for Help! Custom Processor NAR autoloader

Very important to distinguish here... You cannot hot reload the same nar (same component version info in the manifest). However, you can hot load additional new versions and select them in the UI. If you're tossing a new copy of the nar with the same manifest metadata in the lib it wont see it

Re: NiFi 1.11.4 HDFS/HBASE Processor Errors After Kerberos Ticket Expires

If there is nothing in the logs but they stop working I suspect the issue is related the default prompt for name. Update settings in bootstrap is most likely needed. Thanks On Wed, Oct 21, 2020 at 3:36 PM Peter Turcsanyi wrote: > Are there any exception stack traces in the log when the

Re: Build Problem 1.11.4 on MacOS

Darren, I believe there were gremlins in that JDK release.. Can you please try something like 265? On Tue, Oct 20, 2020 at 8:52 AM Darren Govoni wrote: > Hi, > Seem to have this recurring problem trying to build on MacOS with > nifi-utils. Anyone have a workaround or fix for this? > > Thanks

Re: MiNiFi version

Jeremy MiNiFi Java is basically a headless version of NiFi with some added configuration ease of use items appropriate to that model. It has proven too hard to slice time for that versus NiFi so I think what will end up happening is that minifi java as a standalone source repository will just

Re: MiNiFi version

> that I can get the UUID Client ID feature, is that easy to do by just > moving the jar over or something else has to be done or I'm limited to a > specific NiFi version? > > Jeremy > > On Wed, Jul 22, 2020 at 12:52 AM Joe Witt wrote: > >> Jeremy >> >&g

Re: Urgent: HDFS processors throwing OOM - Compressed class space exception

you had already sent this urgent note to dev. now cross posted 40 minutes later. please dont On Tue, Jul 21, 2020 at 9:14 PM Mohit Jain wrote: > Hi team, > > I’ve been facing the issue while using any HDFS processor, e.g. - PutHDFS > throws the error - > Failed to write to HDFS due to

Re: Urgent: HDFS processors throwing OOM - Compressed class space exception

The files arent read into memory anyway. The class space filling up is what to focus on. On Wed, Jul 22, 2020 at 2:34 AM Mohit Jain wrote: > Those are the small files not more than 100 MB each. > Nifi is configured to 16g. > > Thanks > > Get Outlook for iOS >

Re: early eof

...I am confident far larger objects are sent via this mechanism reliably. Can you share what happens on the nifi side specifically? There should be a stack trace and log entry for this in the logs. What are the settings you have in NiFi for this processor? How much storage space does your

Re: ReplaceText - Out of memory - Requested array size exceeds VM limit

Asmath ReplaceText either loads full lines at a time or loads the entire file into memory. So keep that in mind. If you need something that only loads at worst 1-2x the length of the replacement string you're interested in then I'd recommend just using a scripted processor that does precisely

Re: Data Provenance Stops Working

soon! > > Thanks > -Mark > > [1] https://issues.apache.org/jira/browse/NIFI-7346 > [2] https://issues.apache.org/jira/browse/NIFI-7375 > > > On Aug 10, 2020, at 1:26 PM, Joe Witt wrote: > > shawn - i believe it is related to our default settings and have phoned a

Re: Data Provenance Stops Working

shawn - i believe it is related to our default settings and have phoned a friend to jump in here when able. but default retention and default sharding i *think* can result in this. You can generate a thread dump before and after the locked state to see what it is stuck/sitting on. That will help

Re: FetchSFTP: Rename file on move

Jairo You can use a PutSFTP after Fetch to place it where you want. Thanks On Tue, Aug 11, 2020 at 3:16 PM Jairo Henao wrote: > Hi community, > > Is there a way to rename a file before moving it with FetchSFTP? > > After processing a file, I need to move it to a folder and add a timestamp >

Re: NiFi Merge Content Processor Use Case

Eric I believe for now the most appropriate option available is to figure out what the maximum set of outstanding in process buckets could be and set that. Hopefully there is enough memory in the system to handle whatever that worse case could be in terms of flowfiles attributes being held in

Re: FetchSFTP Failed to fetch content for StandardFlowFileRecord

These scenarios are ripe for io race conditions between the sending process (the thing putting files in the directory) and the receiving process (nifi). It is vital to ensure the writer either changes the name when done writing (best case) and the reader only looks at that pattern OR you ensure

Re: Nifi takes too long to start(~ 30 minutes)

Mohit, You almost certainly want to take that same flow and setup a cluster on a more recent version of NiFi to compare startup times. For flows with thousands of components there are important improvements which have occurred in the past year and a half. Startup time, user perceived behavior

Re: Nifi takes too long to start(~ 30 minutes)

in the election process part. > I've constantly seen quick startup times even with thousands of components > in the flow. > I'd look into the logs and maybe turn on some debug logs to find out > what's going on. > > Pierre > > Le jeu. 13 août 2020 à 16:33, Joe Witt a

Re: NIFI /Kafka - Data loss possibility with node failures

Asmath In a traditional installation, regardless of how a NiFi cluster obtains data (kafka, ftp, HTTP calls, TCP listening, etc, ) once it is responsible for the data it has ack'd its receipt to the source(s). If that NiFi node were to become offline the data it owns is delayed. If that node

Re: MergeContent resulting in corrupted JSON

here > some,more,text,goes > null null null text,goes > more,test,goes,here > > Regarding the fork, its nothing crazy and once I get approvals (from work) > see about contributing back. > > Thanks, > Jason > > On Fri, Jul 3, 2020 at 10:19 AM Joe Witt wrote: &

Re: NiFi 1.11 - non-heap size

-1 in that case means it is on a system or configuration for which the JVM cannot get that answer. Nothing to worry about. On Thu, Jun 25, 2020 at 8:33 AM Valentina Ivanova wrote: > Hello again! > > I have been looking at the system diagnostics and noticed the values for > non-heap size

Re: NiFi 1.11 - non-heap size

Valentina Ivanova wrote: > Hi Joe, > > Thanks for the quick reply! > > What about the used and free non-heap - free non-heap is 14 MB only while > used is 224MB? > > Thanks > > Valentina > ------ > *From:* Joe Witt > *Sent:* Thu

Re: NiFi-light for analysts

That would be a fine option for those users who are capable to run maven builds. I think evolving the nifi registry and nifi integration to source all nars as needed at runtime from the registry would be the best user experience and deployment answer over time. Thanks On Mon, Jun 29, 2020 at

Re: kafka-clients version

We should make a new nar and processor name and drop ine of the older nars. Thanks On Fri, Jun 12, 2020 at 6:33 AM Bryan Bende wrote: > Gotcha. Yes, we can obviously change the version of the client lib, but > it's misleading because they were named "2_0" because they use the "2_0" > client,

Re: Nifi security of local filesystem and hdfs in multitenant hdfs use cases

er A from using Keytab A or Keytab > B directly and instead forces them to use no Keytab (which presumably will > result in authorization failure) or using the KeytabCrdentialsService, > which you can control READ access to. > > Does this help? > > Thanks > -Mark > > On Jul 30, 2020,

Re: MergeContent resulting in corrupted JSON

Jason content repo writing isnt related to zk. you mention a fork of nifi for consume kafka. have you tried using stock items/albeit without whatever feature you needed so you can narrow in on the problem? joe On Fri, Jul 3, 2020 at 7:16 AM Jason Iannone wrote: > What role does Zookeeper

Re: Detect duplicate records

I believe Robert's case is that he has records flowing through bundled in flowfiles containing one or more of them at a time and he'd like to understand on a per record level (regardless of the flowfile they're contained in) whether that record has already been seen over some time interval.

Re: Nifi 1.12.1 cluster is getting hung after few days(15 days)

Hello Please capture and share a full thread dump by running bin/nifi.sh dump. and please post these so theyre easier to read than this email system. Thanks On Thu, Jan 7, 2021 at 5:22 AM sanjeet rath wrote: > Hi All, > > Could someone please give me thoughts on the trailed mail issue, so i

Re: NIFI and Out of Memory Error

s enabled, when an out-of-memory error occurs, the JVM > crashes and produces text and binary crash files. > > Best Regards > John > > > > On Dec 3, 2020, at 10:09 AM, Joe Witt wrote: > > John, > > First, as a general rule it is usually very doable to build flows w

Re: NIFI and Out of Memory Error

John, First, as a general rule it is usually very doable to build flows which are very stream oriented rather than entire file oriented. That processor by its nature isn't friendly in this way if configured to work with large memory chunks. Alternatives often exist. Second, I do think it is

Re: Nifi 1.1.14 - too many open files after a while

I put dots in there...its just 261 or later than 271. if you have oracles i think latest is 271 and you should downgrade. thanks On Tue, Dec 15, 2020 at 6:41 AM Joe Witt wrote: > Hello > > I recommend downgrading the jvm to 2.6.1 or upgrading to after 2.7.1. > > It i

Re: Nifi 1.1.14 - too many open files after a while

> > java -version > > java version "1.8.0_271" > > Java(TM) SE Runtime Environment (build 1.8.0_271-b25) > > Java HotSpot(TM) 64-Bit Server VM (build 25.271-b25, mixed mode) > > > > *From:* Joe Witt > *Sent:* dinsdag 15 december 2020 14:34 > *To:*

Re: Nifi 1.1.14 - too many open files after a while

No known leak. Which JVM are you using? On Tue, Dec 15, 2020 at 6:18 AM Marel, J. van der (Jasper) < jasper.van.der.ma...@ing.com> wrote: > Hi, > > > > I have installed nifi 1.1.14 on a CentOS7 machine. Ulimits are set to > 5 open files for the user. > > However after a while a lot of open

Re: [Bug] Duplicate Flow Import From Registry

Eric, This is simply NiFi on your laptop (single node?) talking to a Registry running on some IaaS infrastructure? There is no load balancer/proxy/etc.. in between ? Thanks On Thu, Dec 10, 2020 at 4:17 PM Eric Secules wrote: > Hello everyone, > > My team is encountering a bug where we import

Re: [EXTERNAL] Re: horizontal merge

Geoffrey The process session is requiring you to account for all flow files you've either created or pulled from the queue. You have logic which pulls up to 2 things. It could pull one in which case you are returning. You would get the above error from that in those cases. In the case you get

Re: Provenance queries effect on processing

Eric, short version: Provenance queries can absolutely take away CPU time from the flow. longer version: They get the same priority as any other thread in nifi. Once prov queries are being executed they use CPU. I would strongly advise against any blending of the flow execution with provenance

Re: Provenance queries effect on processing

>> so what settings help control this lag? >> >> Thanks, >> Eric >> >> On Tue, Nov 3, 2020 at 2:35 PM Joe Witt wrote: >> >>> Eric, >>> >>> short version: Provenance queries can absolutely take away CPU time from >>> th

Re: Authorization Framework

be going down the road with > respect to processor security so we can keep our efforts aligned. > > In regards to your question I primarily mean authorization. Our company > already has a SSO that establishes identity credentials so these are then > used to authorize specific functions

Re: Authorization Framework

Darren You will want this thread on dev list to get traction. Also please clarify if you mean authorization or whether you mean authentication. I read all usages as meaning to discuss authentication. thanks On Wed, Nov 4, 2020 at 9:53 AM Darren Govoni wrote: > Greetings! > > We have an

Re: PutAzureBlobStorage OutOfMemoryError

Eric Your assumption is definitely what it should be. This will need someone to investigate. Please file a jira with as much detail as you can. Thanks On Wed, Oct 28, 2020 at 12:40 PM Eric Secules wrote: > Hello everyone, > > I am trying to upload a 300 MB file to azure blob storage using >

Re: There isnot show any items on Provenance GUI

Thanks for following up with the info on how you solved it. Glad it is working now. Thanks On Thu, Oct 29, 2020 at 7:13 PM 杨量（上海-技术部-开发） wrote: > Hi, > > The problem I'm having is the permission configuration. > > There should grant permission to node proxy and user to view provenance. > >

Re: NiFi Cluster 1.9.2 Content Repository

Hello The key value to watch is the 50% value. That means we will work to remove content no longer reachable in the flow until we are at 50% of the available disk space for that volume. So how big is the disk there for each node? Can you share more about what you are hoping to see happen

Re: [E] Re: NIFI show version changed *, but version control show no local changes

There is no requirement to use the registry. It simply gives you a way to store versioned flows which you can reference/use from zero or more nifi clusters/flows to help keep things in line. Many teams use this to ensure as flows are improved over time and worked through dev/test/stage/prod

Re: NiFi 1.12.1 queue balancing

Maksym, Very difficult to look at these brief/limited details and offer meaningful responses. In the example you show about the data volumes are so small that I dont even know that load balancing would kick in. But yes generally speaking the combination of load balancing and back pressure

Re: Lagging worker nodes

I'm assuming also this is the same thing Maksym was asking about yesterday. Let's try to keep the thread together as this gets discussed. On Thu, Jan 28, 2021 at 1:10 PM Pierre Villard wrote: > Hi Zilvinas, > > I'm afraid we would need more details to help you out here. > > My first question

Re: InvokeHTTP hangs after several successful calls

> The InvokeHTTP configuration is as shown below. > > > > > > On Jan 25, 2021, at 7:24 AM, Joe Witt wrote: > > Hello > > If you suspect an actual stuck/hung thread then the best course of action > is to generate a thread dump. This can be done in a couple ways but one of &

Re: [E] Re: Lagging worker nodes

>>> >>>> Hi Joe, >>>> >>>> Yes it is the same issue. We have used your advice and reduced the >>>> amount of threads on our large processors: fetch/compress/publish to a >>>> minimum and then increased gradually to 4 until the processing rate b

Re: Monitoring NiFi Root Level Controller Services

also sorry for referencing you as 'John' - not sure why I just assumed that :) On Mon, Feb 1, 2021 at 5:46 PM Joe Witt wrote: > John, > > You're using a vendor distribution of NiFi. You should contact the vendor. > > You can certainly monitor the state of a controller servi

Re: Monitoring NiFi Root Level Controller Services

John, You're using a vendor distribution of NiFi. You should contact the vendor. You can certainly monitor the state of a controller service via the REST API. They should either be enabled or not enabled. Thanks On Mon, Feb 1, 2021 at 5:39 PM jgunvaldson wrote: > Hi All, > > Root level

Re: Regarding jira issue: NIFI-7856

Sanjeet If I understand correctly you are trying to backport a fix we have completed in the 1.13.x line to your 1.12.x line and asking for assistance with that. I recommend you upgrade to 1.13.x instead of trying to backport. However, yes on quick read your stated approach does seem fine.

Re: Broken pipe write failed errors

What JVM are you using? Thanks On Sat, May 29, 2021 at 11:16 AM Juan Pablo Gardella < gardellajuanpa...@gmail.com> wrote: > Not related to Nifi, but I faced the same type of issue for endpoints > behind a proxy which takes more than 30 seconds to answer. Fixed by > replacing Apache Http client

Re: Broken pipe write failed errors

a 8 with > 1.9.2. > > On Sat, May 29, 2021, 14:21 Joe Witt wrote: > >> What JVM are you using? >> >> Thanks >> >> On Sat, May 29, 2021 at 11:16 AM Juan Pablo Gardella < >> gardellajuanpa...@gmail.com> wrote: >> >>> Not related to Nifi, b

Re: Taking Huge Time for Connect all Nodes to NIFI cluster

I dont recommend splitting the clusters for this. Lets figure out what is happening for your case. On Tue, Jun 29, 2021 at 10:25 PM Modepalli Venkata Avinash < modepalli.avin...@subex.com> wrote: > Hi Joe, > > > > Thanks for your feedback. > > Yes, working on to split into multiple clusters.

Re: PYFI Python Nifi Clone

Sounds fun and looks cool. But do not violate the marks such as do not use the Apache NiFi logo. Thanks On Sat, Jul 10, 2021 at 9:38 AM Darren Govoni wrote: > Hi!, >Just sharing a fun project I'll post on github soon. I'm creating a > pure python clone of Nifi that separates the UI

Re: Taking Huge Time for Connect all Nodes to NIFI cluster

Hello A cluster that size should be fine. We did make various improvement to cluster behavior and startup times though. What prevents you from moving to 1.13? How many flowfiles are in the repo when restarting is taking that long? thanks On Tue, Jun 29, 2021 at 7:38 AM Joe Obernberger <

Re: Insufficient Permission to Clear Queue

Shawn Yeah probably not our best response to help you figure out and resolve the issue on your own. Please file a JIRA with what the settings where, what you tried and received, and how you fixed and why it was confusing. Just so whoever picks that up will have the context and can make it more

Re: After upgrade to 1.11.4, flowController fails to start due to invalid clusterCoordinator port 0

PatW I'd triple-check to ensure there are no weird/special/unexpected characters in your nifi.properties file. These are often not obvious in default text views so you might need to explicitly set some view to expose them. Yeah this is certainly not a great user experience - we give you just

Re:

il may contain QAD proprietary information and should be treated > as confidential. > > > On Fri, Mar 26, 2021 at 3:26 PM Joe Witt wrote: > >> We would need to know which nice version, Java version, and so you’re >> on. From the jetty output this appears potentially quite old. P

Re:

We would need to know which nice version, Java version, and so you’re on. >From the jetty output this appears potentially quite old. Please try on 1.13.2 Thanks On Fri, Mar 26, 2021 at 5:15 AM Ralph Vercauteren wrote: > Hi All, > > I get the next error connecting to the nifi website: >

Re: ExtractText 1.13.1 only one regex evaluated on CSV file?

Please avoid 1.13.1. Grab 1.13.2 On Wed, Mar 24, 2021 at 11:33 PM Hendrik Ruijter < hendrik.ruij...@verisure.com> wrote: > [Sending again since I cannot see this mail in the archive] > > Hello, ExtractText 1.12.1 evaluates the regex for each flowfile and adds > attributes to each flowfile.

Re: SplitText issue

Michal - It is on track to be available within about 3 hours. On Fri, Mar 19, 2021 at 7:35 AM Michal Tomaszewski wrote: > > Thanks! > > Regards, > > Michal > > > > From: Mark Payne > Sent: Friday, March 19, 2021 3:01 PM > To: users@nifi.apache.org > Subject: Re: SplitText issue > > > > Michal,

Re: Writing parquet files to S3

Not responding to the real question in the thread but "I'm using NIFI 1.13.1.". Please switch to 1.13.2 right away due to a regression in 1.13.1 On Mon, Mar 22, 2021 at 12:24 AM Vibhath Ileperuma wrote: > > Hi Bryan, > > I'm planning to add these generated parquet files to an impala S3 table.

Re: Creating recursive missing folders with PutSmbFile

pretty sure SMB is super popular - it is just that for the cases we typically engage in SMB isn't used as the protocol to access data :) Agree with the rest of that Thanks On Mon, Mar 22, 2021 at 11:13 AM Mark Payne wrote: > > Jens, > > In order to review & merge a PR, there are two important

[ANNOUNCE] Apache NiFi 1.13.1 release

Hello The Apache NiFi team would like to announce the release of Apache NiFi 1.13.1. This is a stability focused release including nearly 50 bug fixes and improvements. Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. Apache NiFi was made for

[ANNOUNCE] Apache NiFi 1.13.1 release

Hello The Apache NiFi team would like to announce the release of Apache NiFi 1.13.1. This is a stability focused release including nearly 50 bug fixes and improvements. Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. Apache NiFi was made for

Re: Any known issue on SplitRecord?

Juan We found a bug in 1.13.1 today as reported here https://issues.apache.org/jira/browse/NIFI-8337 and https://issues.apache.org/jira/browse/NIFI-8334. We will have a 1.13.2 out asap to fix this and the regression now has tests to prevent it in the future. Thanks Joe On Wed, Mar 17, 2021 at

Regression in 1.13.1 - creating 1.13.2

All Based on two reports received today we've found a regression in process session handling in NiFi 1.13.1. This is fixed on main and we will have a 1.13.2 RC1 up for vote in a couple hours. The core issue found is https://issues.apache.org/jira/browse/NIFI-8337. This is the same root

Re: Any known issue on SplitRecord?

I should clarify that I am not positive it is the same issue but it is certainly possible especially if this worked in 1.13.0 Thanks On Wed, Mar 17, 2021 at 8:46 PM Joe Witt wrote: > Juan > > We found a bug in 1.13.1 today as reported here > https://issues.apache.org/jira/brow

Re: Any known issue on SplitRecord?

npa...@gmail.com> wrote: > Wow that is fast! You are awesome, thanks Joe. I will test it. > > Juan > > On Thu, 18 Mar 2021 at 00:47, Joe Witt wrote: > >> Juan >> >> We found a bug in 1.13.1 today as reported here >> https://issues.apache.org/jira/browse

Re: Questions about the GetFile processor

Hello Yeah when there are a ton (50k or more) of files in a directory performance is *horrible*. If you can put them into some subdirs to divide it up then it will go a lot faster. Thanks On Fri, Feb 26, 2021 at 7:30 PM Jean-Sebastien Vachon < jsvac...@brizodata.com> wrote: > Hi again, > > I

Re: NIFI - Topic pattern names

See http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-kafka-2-6-nar/1.13.0/org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_2_6/index.html Look at 'Topic Name(s)' and 'Topic Name Format' with a provided naming pattern. Thanks On Tue, Feb 23, 2021 at 1:55 PM KhajaAsmath

Re: NiFi 1.11.4 Custom Processor Development

> -Original Message- > From: Joe Witt [mailto:joe.w...@gmail.com] > Sent: 13 April 2021 18:16 > To: users@nifi.apache.org > Subject: Re: NiFi 1.11.4 Custom Processor Development > > Nathan > > Nothing has occurred that should result in issues with using custom >

Re: NiFi 1.11.4 Custom Processor Development

t experienced anything yet, but would > hate to spend ages in debugging an issue that is the result of this! > > Kind Regards, > > Nathan > > From: Joe Witt > Sent: Tuesday, April 13, 2021 5:48 pm > To: users@nifi.apache.org > Su

Re: NiFi 1.11.4 Custom Processor Development

Hello In moving to support maven 3.8 we found only https based repos are allowed by default. In addressing this we updated the build to move away from bintray and another location as well. This is now the case when you build master. You could try to cherry pick and edit the key commits or hang

Re: Nifi throws an error when reading a large csv file

How large is each line expected to be? You could have a massive line or much larger than thought of. Or you could be creating far more flowfiles than intended. If you cut the file in size does it work better? Will need more data to help narrow in but obviously we're all very interested to know

[ANNOUNCE] Apache NiFi 1.13.0 release

Hello The Apache NiFi team would like to announce the release of Apache NiFi 1.13.0. This release includes 260 new features, bug fixes and improvements. Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. Apache NiFi was made for dataflow. It supports

[ANNOUNCE] Apache NiFi 1.13.0 release

Hello The Apache NiFi team would like to announce the release of Apache NiFi 1.13.0. This release includes 260 new features, bug fixes and improvements. Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. Apache NiFi was made for dataflow. It supports

Re: Re: Disfunctional cluster with version 1.13.2

If restarting is helping you then generally it should be easily found. Is mem usage spiking? Are thread dumps revealing? On Tue, Aug 17, 2021 at 6:12 AM Ryan Hendrickson < ryan.andrew.hendrick...@gmail.com> wrote: > We've rolled back to 1.11.4 in a couple scenarios. We have also setup a >

Re: Make NiFi Flow Read Only on Disconnect

Shawn So that is one direction (being more restrictive). Another direction is to simply ditch the logic we have and allow changes and simply update the disconnected node when it rejoins. We have all kinds of super complex super duper awesome logic in there to help prevent users from getting

Re: Make NiFi Flow Read Only on Disconnect

n > > -----Original Message- > From: Joe Witt > Sent: Thursday, August 26, 2021 10:23 AM > To: users@nifi.apache.org > Subject: Re: Make NiFi Flow Read Only on Disconnect > > Shawn > > So that is one direction (being more restrictive). Another direction is to > simply

Re: Nifi & rtsp or rtmp

Valentina\ Can you describe what you want to be able to do with the streaming video? Thanks On Mon, Aug 30, 2021 at 6:46 AM Valentina Ivanova wrote: > > Hello! > > I am tasked to handle streaming video in rtmp or rtsp and have a quite short > time to set it up. So I am wondering if Nifi is

Re: Deadlock in loop

Hello This case should work very well. Please share the details of the flow configuration. Can you download a flow template and share that? thanks On Wed, Aug 18, 2021 at 8:20 AM Aurélien Mazoyer wrote: > Hi, > > I have a nifi flow that reads zip files. For each non-zip file it performs >

Re: Round robin load balancing eventually stops using all nodes

Ryan If this is so easily replicated for you it should be trivially found and fixed most likely. Please share, for each node in your cluster, both a thread dump and heap dump within 30 mins of startup and again after 24 hours. This will allow us to see the delta and if there appears to be any

Re: Frozen Relationships - Remote Process Groups

Ryan Dont think that is a known issue/pattern we've heard much of. Definitely will need a stack track set when you observe it. Ideally you get a stack dump when all looks good, then a stack trace once the freeze had started, then again after some time within the freeze again. Share those along

Re: odd performance behavior 1.14

Scott Nope this sounds pretty dang unique What JVM? May need to attach a profiler. I have seen buried exceptions happening at massive rates causing horrid performance among a few other scenarios but nothing specific to 1.14 thanks On Sat, Jul 31, 2021 at 4:01 PM scott wrote: > Hi All, > I

Re: CryptographicHashContent calculates 2 differents sha256 hashes on the same content

Jens, Also what type of file system/storage system are you running NiFi on in this case? We'll need to know this for the NiFi content/flowfile/provenance repositories? Is it NFS? Thanks On Wed, Oct 20, 2021 at 11:14 AM Joe Witt wrote: > > Jens, > > And to further narrow this down

Re: CryptographicHashContent calculates 2 differents sha256 hashes on the same content

ifi itself you can make this happen or cannot? Thanks On Wed, Oct 20, 2021 at 11:08 AM Joe Witt wrote: > > Jens, > > "After fetching a FlowFile-stream file and unpacked it back into NiFi > I calculate a sha256. 1 minutes later I recalculate the sha256 on the > ex

Re: CryptographicHashContent calculates 2 differents sha256 hashes on the same content

> flow that does this, but not sure if the email server will strip out the > attachment or not. > > This way we remove any possibility of actual corruption between the two nifi > instances. If we can still see corruption / different hashes within a single > nifi instance, then

Re: Trouble starting docker container

Glad you found the problem and shared the resolution. Thanks On Mon, Oct 18, 2021 at 8:58 AM Jean-Sebastien Vachon < jsvac...@brizodata.com> wrote: > I fixed my problem... it was related to this > > >

Re: Round robin load balancing eventually stops using all nodes

Ryan, Regarding NIFI-9236 the JIRA captures it well but sounds like there is now a better understanding of how it works and what options exist to better view details. Regarding Load Balancing: NIFI-7081 is largely about the scenario whereby in load balancing cases nodes which are slower

<    3   4   5   6   7   8   9   10   >