Re: NIFI - Regex replace Help

Hi Asmath, If I understood correctly, you could try a single \\N|\" simply join the regexes with a pipe. I hope that helps Best, Lars On 6 November 2020 17:35:05 CET, KhajaAsmath Mohammed wrote: >Hi, > >I have e replacetext processor with below settings. I want to replace >them >with only

Re: A few quick questions

Hi Robert, addressing your second question: I also use global contexts in places where this is appropriate. Still, for globally shared values used within groups that have a nested context, you could go with environment variables or system properties, for non-sensitive values. For sensitive

Re: Handling floating point numbers

Hi Vibhath, that is the usual problem with floating point numbers. The only ways to fix that I can imagine: * store them as numeric/decimal values in your postgres db (might be hard to apply at this stage), or * you might be able to round and cast them in your postgres-query to

Re: Negative off heap memory?

the non-heap usage is really 2.81816384E8 for you. So 281 MB. And there is no max value set. Thanks -Mark On Aug 4, 2021, at 1:19 PM, Lars Winderling mailto:lars.winderl...@posteo.de>> wrote: Dear community, using the QueryNiFiReportingTask on NiFi:1.13.2 on java:8, debian:10, w

Negative off heap memory?

Dear community, using the QueryNiFiReportingTask on NiFi:1.13.2 on java:8, debian:10, with G1 enabled as GC, I see something like this:   "jvm_heap_used" : 4.26262096E8,   "jvm_heap_usage" : 0.2646583418051402, *  "jvm_non_heap_usage" : -2.81816384E8,* The non heap usage is negative by

Re: NiFi Queue Monitoring

Scott, you could use tls client cert auth, maybe including appropriate user-mapping. Since you have been using ldap, you maybe can use the dn as cert subject as-is. Only be aware that whitespace handling in the subject dn might differ between nifi and your ldap. We're also running nifi secured

Re: Expression language within scripts in ExecuteScript

James, maybe just use? Import socket socket.gethostname() It will give you rather the hostname, but that should also help for distinguishing between nodes. Best, Lars On 17 July 2021 22:25:47 CEST, James McMahon wrote: >Looking closely at the flowFiles I am creating,in the subsequent

Re: NIFI-REGISTRY Migration

Hi Minh, you could try to employ URL aliasing as explained in the docs. This way, the registry will store only an alias instead of the actual registry address. When migrating to another host, make

Re: Requesting Apache NiFi Help/Guidelines

Hi Ben, I think your regex should read krb5cc_.* , thus including a dot. Best Lars On 20 June 2022 20:02:42 CEST, "Ben .T.George" wrote: >Hello, > >i have moved files under /home/ansible, which is home directory for the >user ansible and the user i used to connect to remote server, also

Re: Re[2]: Minifi and ssl config on NiFi

Hi Dave, you could use a (custom) CA for your client certs, so only the CA-cert would need to be trusted. And for policies, you could use an LDAP group and base policies on that. Downside is that NiFi currently doesn't offer certificate revocation afaik, so it might not be applicable to you.

Duplicate UUID after clone events in NiFi 1.16.3

Dear community, we have recently upgraded from 1.14.0 to 1.16.3. And the automatic UUID generation on CLONE events seems to have changed. When sending a flowfile from a source (processor or port) using the same relationship to multiple sinks, the cloned flowfiles all have the same UUID.

Re: Duplicate UUID after clone events in NiFi 1.16.3

I have just found https://issues.apache.org/jira/browse/NIFI-10203, it's a know issue resolved in 1.17.0. sorry for spamming the list. On 22-09-01 13:41, Lars Winderling wrote: Dear community, we have recently upgraded from 1.14.0 to 1.16.3. And the automatic UUID generation on CLONE events

Trigger content eviction manually?

Dear community, sometimes our content repository grows out of bounds. Since it has been separated on disk from the rest of NiFi, we can still use the NiFi UI and empty the respective queues. However, the disk remains jammed. Sometimes, it gets cleaned up after a few mintes, but most of the

Re: ExecuteStreamCommand fails to extract archived files

Hi Jim, have you checked whether NiFi can actually access the given paths (/bin/7za and /mnt/in)? You might use a script processor just to check both paths exist. If that all works, you might want to dump the processor config as json by either downloading the flow definition or manually

Re: Trigger content eviction manually?

I'm using 1.16.3 from upstream (no custom build) on java 11 temurin, debian 10, virtualized, no docker setup. On 13 September 2022 13:37:15 CEST, Joe Witt wrote: >Lars > >What version are you using? > >Thanks > >On Tue, Sep 13, 2022 at 3:11 AM Lars Winderling >wr

Re: Trigger content eviction manually?

default before was like 1MB and what we'd see is we'd hang on to >large content way longer than we intended because some queue had one >tiny object in it. So that value became really important. > >If you're on 1MB change to 50KB and see what happens. > >Thanks > >On Tue, Sep 1

Re: Trigger content eviction manually?

.archive.max.retention.period=7 days >nifi.content.repository.archive.max.usage.percentage=50% >nifi.content.repository.archive.enabled=true >nifi.content.repository.always.sync=false > >Thanks > >On Tue, Sep 13, 2022 at 7:04 AM Lars Winderling > wrote: >> >> I'm using 1.16.3 from upstream (no custo

Re: Trigger content eviction manually?

ior was the answer.  I owe the community a blog on this Thanks On Tue, Sep 13, 2022 at 9:57 AM Lars Winderling wrote: Sorry, misread the jira. We're still on the old default value. Thank you for being persistant about it. I will try it tomorrow with the lower value and get back to

Re: Trigger content eviction manually?

1:08 PM, Lars Winderling wrote: …and guess what I did :-) the joys of remote working. just put my kids to bed, and here you are! # Content Repository nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository nifi.content.claim.max.appendable.size=10 MB

Re: Expected mergerecord performance

Hi Richard, it's not related, but for the logical types timestamp-millis you should use a "long" instead of a "string" (cf https://avro.apache.org/docs/1.11.1/specification/#timestamp-millisecond-precision) afaik. Best, Lars On 21 December 2022 08:29:54 CET, Richard Beare wrote: >I have

Re: NiFi not rolling logs

artup. I have >a Jira ticket up to fix this for 1.x and 2.x and will likely have it up >today. Should definitely be ready for 1.23 > >On Sat, Jul 8, 2023 at 4:17 AM Lars Winderling >wrote: > >> Dear NiFiers, we have been bugged so much by overflowing logfiles, and >&

Re: NiFi not rolling logs

from my iPhone On Jul 8, 2023, at 9:26 AM, Lars Winderling wrote:  Hi Mike, thanks for the advice. Our NiFi instances are running for week, if not months. Often times until the next update, so the startup option will bring much benefit, I fear, or am I mistaken

Re: NiFi not rolling logs

Dear NiFiers, we have been bugged so much by overflowing logfiles, and nothing has ever helped. I thought it was just my lack of skills...especially when NiFi has some issues and keeps on spilling stacktraces with high frequency to disk, it eats up space quickly. I have created cronjobs that

Re: [EXTERNAL] Re: Processor ID/Processor Name as a default attribute

Hi folks, I think you can obtain the same information via provenance. Or am I mssing something? It sounds to me like you want to do manual debugging instead of automizing it, so provenance should be easily accessible. Enjoy your day and take care! Best, Lars On 26 May 2023 11:21:13 CEST, James

Re: [EXTERNAL] Re: Processor ID/Processor Name as a default attribute

ing an easy to >trace "source processor" to look for. > >BUT, if you do things differently/better and feel like sharing some tips, >I'd be very appreciative, even if you took this off the mailing list/direct >mail. > >thanks again > >On Fri, May 26, 2023 at 11:54 

Re: BufferedReader best option to search through large flowfiles?

Hi Jim, RouteText works in a line-by-line fashion, so that shouldn't exhaust memory (unless for /very/ long lines). Other processors such as ReplaceText have the option to choose whether you want to stream lines, or slurp the whole file at once. Best, Lars On 23-06-05 14:49, James McMahon

Re: BufferedReader best option to search through large flowfiles?

Hi James, in case the NiFi processors such as ExtractText, ReplaceText and RouteOnContent (maybe multiple in a row/in parallel) do not match your use case, I'd definitely go with a bufferend reader and line wise processing. Afaik you can get it as easily as     new

Re: [NIFI 1.23.2] Insecure Cipher Provider Algorithm

Hi Quentin, I second these findings. I'm getting the same error on 1.23.2 using the same ciphers.   deb: 11   java: 17.0.7 Temurin Best, Lars On 23-12-13 14:58, Quentin HORNEMAN GUTTON wrote: Hello, I’m facing an issue after upgrading NiFi 1.13.2 to 1.23.2. I have a warn log with Insecure

Re: Finding slow down in processing

Hi Aaron, is the number of threads set sufficiently high? Once I set it too low by accident on a very powerful machine, and when we got more and more flows, at some point NiFi slowed down tremendously. By increasing threads to the recommend setting (a few per core, cf. admin docs) we got NiFi