Re: Help with tailing json log files

2020-04-01 Thread Jens M. Kofoed
Hi again Sorry, the file format is not en JSON. It is in XML, my mistake. Playing with to many things at the same time regards Jens Den ons. 1. apr. 2020 kl. 11.59 skrev Jens M. Kofoed : > Hi > > I need your help to crack this nut. > An application is making logfiles in json forma

Help with tailing json log files

2020-04-01 Thread Jens M. Kofoed
Hi I need your help to crack this nut. An application is making logfiles in json format, with \n\r after each tag. The application doesn't log into a single file (eventlog.xml) and rollover. It write new logfiles every hour (eventlog_-mm-dd_hh.xml), if and only if new data is going to be

How to block a flow until first file is finish

2020-03-24 Thread Jens M. Kofoed
Hi I'm trying to build a flow, where a putfile process is only allow to write a file, if the previous file has finished the following block in the flow. But I can't works it out with my wait notify. Since the first file can't go through the wait block because the notify is coming after the wait

How to build a flow for this use case

2020-03-24 Thread Jens M. Kofoed
Hi I've been using NIFI for a time now, but only for simple flows. But now I got a new use case which is working outside nifi as a cron job, but I would like the flow to be inside nifi. I have to split data files into separate files based on zipcodes, or just zipcodes of interest I'm using a

The build in help pages in NIFI are slow on offline systems

2020-03-24 Thread Jens M. Kofoed
Hi My NIFI server has no connection to the internet, and if you try to open the documentation with in NIFI it takes 40-60 sec before the pages shows up. There are 2 links to https://fonts.googleapis.com/css? and 1 link to

Re: How to block a flow until first file is finish

2020-03-24 Thread Jens M. Kofoed
does the "real" processing. > > > *Chris Sampson* > IT Consultant > *Tel:* 07867 843 675 > chris.samp...@naimuri.com > > > > On Tue, 24 Mar 2020 at 10:05, Jens M. Kofoed > wrote: > >> Hi >> >> I'm trying to build a flow, where a putfile proc

Problem with InvokeHTTP not timing out

2020-10-13 Thread Jens M. Kofoed
/NIFI-7899 Kind regards Jens M. Kofoed

Re: Detect duplicate records

2020-08-15 Thread Jens M. Kofoed
. A DISTINCT which takes 2 min. and 45 sec. only takes about 15 sec with a union. kind regards. I don't know which engine is in NIFI. Jens M. Kofoed Den lør. 15. aug. 2020 kl. 18.08 skrev Matt Burgess : > In addition to the SO answer, if you know all the fields in the > record, you can use Query

Re: How to split json subarrays and keep root

2020-09-29 Thread Jens M. Kofoed
t > Fields" set to "true", I think that does what you're looking to do. > > Regards, > Matt > > [1] > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.12.1/org.apache.nifi.processors.standard.ForkRecord/index.html > > On Fri

How do I split and flatten this JSON document

2020-09-24 Thread Jens M. Kofoed
:value2, key3: { key3a:value3a, key3b:value3b } key4:{ key4a1:value4a1, key4a2:value4a2 } } Record 2 { key1:value1, key2:value2, key3: { key3a:value3a, key3b:value3b } key4:{ key4b1:value4b1, key4b2:value4b2 } } Kind regards Jens M. Kofoed

How to split json subarrays and keep root

2020-09-24 Thread Jens M. Kofoed
:value2, key3: { key3a:value3a, key3b:value3b } key4:{ key4a1:value4a1, key4a2:value4a2 } } Record 2 { key1:value1, key2:value2, key3: { key3a:value3a, key3b:value3b } key4:{ key4b1:value4b1, key4b2:value4b2 } } Kind regards Jens M. Kofoed

Need help converting xml data

2020-05-28 Thread Jens M. Kofoed
ke this: "Part3": { "Details": { "additionalInfo": { "Location": "New York", "Company": "A Company" } } } Please help... Kind regards Jens M. Kofoed

Unable to revert local changes with registry

2020-10-28 Thread Jens M. Kofoed
omeone who knows how to fix this? kind regards Jens M. Kofoed

Re: need parse first few lines to customize a header for csv

2020-08-01 Thread Jens M. Kofoed
Hi What about the ReplaceText. If you know it will always be 4 header lines, you could use a regex like this: ^topic.*?$ ^.*?$ ^.*?$ ^.*?$ or using 4 ReplaceText after each other, only replacing First-Line. regards Jens M. Kofoed Den fre. 31. jul. 2020 kl. 21.09 skrev gemlan123 : > I have s

Need help troubleshooting wait-notify

2020-08-02 Thread Jens M. Kofoed
Hi I have created a wait-notify gate which is working fine for days, but suddenly the wait process is not releasing any files. I am using redis as the distributed cachemap, and I was looking in redis to find the release identifier but I could not find it. Which seems odd to me. I have release

Re: Nested groups for LdapUserGroupProvider

2020-07-24 Thread Jens M. Kofoed
Hi From my knowledge and playing with ldap and nifi. Nifi “imports” users and groups into nifi and nifi does not support groups in groups. In my setup it looks like it imports groups first. Next it imports users. If a user is memberOf an imported group it will be connected to the group in nifi.

Re: Nested groups for LdapUserGroupProvider

2020-07-25 Thread Jens M. Kofoed
the >> config settings to the correct spot within our LDAP forest…initially we >> didn’t point it correctly and only saw first-level members, after a bit of >> trial and error, finally got nested groups working, and we’ve tested down 5 >> levels of nesting. >> >> >&

Re: Nested groups for LdapUserGroupProvider

2020-07-26 Thread Jens M. Kofoed
ad to solve > that problem? > > Best regards, > Moncef > > Le sam. 25 juil. 2020 à 13:23, Jens M. Kofoed a > écrit : > >> Hi >> >> I have been looking through the source code for LdapUserGroupProvider in >> nifi: >> https://github.com/apache/nifi/blo

Re: Detect duplicate records

2020-08-16 Thread Jens M. Kofoed
something like a redis cache that would tell me >>>> either first time the record was seen or not and update the cache >>>> accordingly. Guessing nothing like that for records exists at this point? >>>> >>>> We've used DetectDuplicate to do this for entire fl

1 minutes vs 5 minutes Metrics via Prometheus

2020-12-08 Thread Jens M. Kofoed
on the processor is showing the last 5 minutes and it seems like it is the same 5 minutes stat that is being reported at the metrics instead of the 1 minutes snap. Is it possible to configure the Prometheus reporting task to report the 1 minutes snap instead? kind regards Jens M. Kofoed

Monitor queue duration?

2021-01-12 Thread Jens M. Kofoed
Hi I would like to monitor the queue duration for the input to some Merge Content processors. I need to check if files are queued up for long time. If so it might be because a file is missing. The queue duration information is not part of the Prometheus metriks nor the site-to-site reports.

Re: NIFI show version changed *, but version control show no local changes

2021-01-27 Thread Jens M. Kofoed
good to try with the > latest since there have been significant improvements around this area > in the last few releases. > >> On Wed, Jan 27, 2021 at 5:45 AM Jens M. Kofoed >> wrote: >> >> Hi >> >> We have a situation where process groups in NIFI shows

Re: [E] Re: NIFI show version changed *, but version control show no local changes

2021-01-27 Thread Jens M. Kofoed
1 but we do not use NiFi Registry, I >> wonder if that's the requirement to use the registry? >> >>> >> >>> On Wed, Jan 27, 2021 at 2:25 PM Bryan Bende wrote: >> >>>> >> >>>> Please specify the versions of NiFi and NiFi Regi

Re: [E] Re: NIFI show version changed *, but version control show no local changes

2021-01-27 Thread Jens M. Kofoed
not use NiFi Registry, I > wonder if that's the requirement to use the registry? > >>> > >>> On Wed, Jan 27, 2021 at 2:25 PM Bryan Bende wrote: > >>>> > >>>> Please specify the versions of NiFi and NiFi Registry. If it is not > >>>

Re: [E] Re: NIFI show version changed *, but version control show no local changes

2021-01-27 Thread Jens M. Kofoed
I have tried to import all version flows to another NIFI server. It shows the same picture. kind regards Jens Den tor. 28. jan. 2021 kl. 07.40 skrev Jens M. Kofoed < jmkofoed@gmail.com>: > Hi > > I have tried to compare the 11 flows which is in the same bucket. > All

Re: [E] Re: NIFI show version changed *, but version control show no local changes

2021-01-28 Thread Jens M. Kofoed
12. PG-Root is still showing it has local changes, but it has not. Kind regards Jens Den tor. 28. jan. 2021 kl. 07.51 skrev Jens M. Kofoed < jmkofoed@gmail.com>: > I have tried to import all version flows to another NIFI server. It shows > the same picture. > > kind regard

Single FlowFile Per Node courses bug in nested version control

2021-01-28 Thread Jens M. Kofoed
Hey I have an issue with nested PG where the root PG continuously is showing it has local changes but it has not. I have now found why, and how you can reproduce the bug 1. Create an empty Process Group (PG-Root) 2. Add version control to the PG-Root 3. Add an empty PG (PG-Sub) in PG-Root 4. Add

Need help to insert complete content of a file into JSONB field in postgres

2021-06-09 Thread Jens M. Kofoed
Dear community I'm struggling with inserting files into a Postgres DB. It is the whole content of a file which has to be inserted into one field in one record. Not each line/record in the field. The PutSQL process expect the content of an incoming FlowFile to be the SQL command to execute. Not

Re: Round Robin not working NiFi 1.13.2

2021-06-10 Thread Jens M. Kofoed
e the culprit :) It sounds like your other nodes > can't connect to node3 over port 8443. Have you verified that the port is > open? Same question for all other ports configured in your nifi.properties. > > On Thu, Jun 10, 2021 at 10:08 AM Jens M. Kofoed > wrote: > >> Hi Joe >

Re: Round Robin not working NiFi 1.13.2

2021-06-10 Thread Jens M. Kofoed
gt; > On Thu, Jun 10, 2021 at 10:29 AM Jens M. Kofoed > wrote: > >> I have the same error for node2 as well. >> All 3 nodes can talk to each other. If I use a remote process group and >> connect to an "remote" input port, everything works fine. This is a wo

Re: Round Robin not working NiFi 1.13.2

2021-06-10 Thread Jens M. Kofoed
cluster.load.balance.address in order to be explicit. > > On Thu, Jun 10, 2021 at 10:45 AM Jens M. Kofoed > wrote: > >> Hi Joe >> >> I just found out that port 6342 is bound to localhost. Why >> In the last build NIFI is bound to localhost as standard if not &g

Round Robin not working NiFi 1.13.2

2021-06-10 Thread Jens M. Kofoed
Dear community I have created a 3 node cluster with NiFi 1.13.2, java 8 on a ubuntu 20.04. I have a ListFTP Process running on primary node only -> FetchFTP with Round Robin on the connection. But if I stop the FetchFTP Process and looking at the queue all flowfiles are listed to be on the same

Re: Round Robin not working NiFi 1.13.2

2021-06-10 Thread Jens M. Kofoed
. 15.36 skrev Joe Gresock : > Hi Jens, > > Out of curiosity, when you run the FetchFTP processor, what does the > Status History of that processor show? Is the processor processing files > on all of your nodes or just the primary? > > On Thu, Jun 10, 2021 at 9:07 AM Je

BUG??? NiFi 1.13.2 Cluster Load Balance binds to localhost and is not accessible

2021-06-10 Thread Jens M. Kofoed
Dear Community I have installed and configured a 3 node secured NiFi cluster with NiFi 1.13.2, Java 8 on Ubuntu 20.04. I was wondering why the cluster didn't load balance flowfiles after I configured Round Robins between a ListFTP and FetchFTP Process. (Other mails earlier today: Round Robin not

Re: BUG??? NiFi 1.13.2 Cluster Load Balance binds to localhost and is not accessible

2021-06-10 Thread Jens M. Kofoed
r. 10. jun. 2021 kl. 19.51 skrev Jens M. Kofoed < jmkofoed@gmail.com>: > Joe, Sorry if there is a space in the mail. There is no spaces in the > original config. I had to change the original address with an anonymous > > /jens > > Den 10. jun. 2021 kl. 19.31 skrev Joe G

Re: BUG??? NiFi 1.13.2 Cluster Load Balance binds to localhost and is not accessible

2021-06-10 Thread Jens M. Kofoed
ad.balance.host > property and see what happens? > >> On Thu, Jun 10, 2021 at 1:23 PM Jens M. Kofoed >> wrote: >> Dear Community >> >> I have installed and configured a 3 node secured NiFi cluster with NiFi >> 1.13.2, Java 8 on Ubuntu 20.04. >> I was

Re: Using a corporate SSL signed certificate

2021-06-15 Thread Jens M. Kofoed
This works fine for my. Kind regards Jens M. Kofoed Den man. 14. jun. 2021 kl. 15.39 skrev QUEVILLON EMMANUEL - EXT-SAFRAN ENGINEERING SERVICES (SAFRAN) : > Hi list, > > > > We are trying to set a nifi secure installation using a SSL singed > certificate by our corporate

Re: BUG??? NiFi 1.13.2 Cluster Load Balance binds to localhost and is not accessible

2021-06-10 Thread Jens M. Kofoed
/util/NiFiProperties.java: nifi.cluster.load.balance.address kind regards Jens Den fre. 11. jun. 2021 kl. 06.54 skrev Jens M. Kofoed < jmkofoed@gmail.com>: > I found out that another user has reported the same bug in JIRA: > https://issues.apache.org/jira/browse/NIFI-8643 > > s

Re: BUG??? NiFi 1.13.2 Cluster Load Balance binds to localhost and is not accessible

2021-06-10 Thread Jens M. Kofoed
I found out that another user has reported the same bug in JIRA: https://issues.apache.org/jira/browse/NIFI-8643 so I'm not the only one, seeing this problem. Kind regards Jens Den tor. 10. jun. 2021 kl. 20.08 skrev Jens M. Kofoed < jmkofoed@gmail.com>: > I have also tried to no

Re: BUG??? NiFi 1.13.2 Cluster Load Balance binds to localhost and is not accessible

2021-06-11 Thread Jens M. Kofoed
This is a bug. There are a mismatch between documentation, nifi.properties and the java class handling the properties https://issues.apache.org/jira/browse/NIFI-8643 kind regards Jens Den fre. 11. jun. 2021 kl. 07.51 skrev Jens M. Kofoed < jmkofoed@gmail.com>: > Adding

Re: Round Robin not working NiFi 1.13.2

2021-06-11 Thread Jens M. Kofoed
This is a bug. There are a mismatch between documentation, nifi.properties and the java class handling the properties https://issues.apache.org/jira/browse/NIFI-8643 kind regards Jens Den tor. 10. jun. 2021 kl. 17.01 skrev Jens M. Kofoed < jmkofoed@gmail.com>: > In the begin

NIFI show version changed *, but version control show no local changes

2021-01-27 Thread Jens M. Kofoed
/NIFIREG-437? kind regards Jens M. Kofoed

Re: Creating recursive missing folders with PutSmbFile

2021-03-22 Thread Jens M. Kofoed
and diffidently not then we had to make changes. So i order to get access to Windows servers, I have configured MS IIS with a FTP Site. To be able to use smb shares, I hope it would be much easier. kind regards Jens M. Kofoed Den man. 22. mar. 2021 kl. 19.19 skrev Joe Witt : > pretty sure SMB is su

Daylightsaving issue with ListFTP and PutFTP Processor

2021-04-06 Thread Jens M. Kofoed
he lastmodified time, PutFTP was happy again. I will try to see if can reproduce the issues. Kind regards Jens M. Kofoed

Creating recursive missing folders with PutSmbFile

2021-03-22 Thread Jens M. Kofoed
included in 1.13, but unfortunately it is not. Kind regards Jens M. Kofoed

Count records where value match x

2021-03-01 Thread Jens M. Kofoed
s to make multiple outputs depending of different statements and next use a CalculateRecordStats to calculate the result. But here I will end up with multiple flowfiles and not one final result. Is it possible to do it in one straight flow? kind regards Jens M. Kofoed

Re: Issue with QueryRecord failing when data is missing

2021-02-25 Thread Jens M. Kofoed
. And why did I not think of it when I sat and tore my hair out of my head :-) As a workaround I have used a ValidateRecord and a RouteOnAttribute using the record.count. So many thanks for the answer. Kind regards Jens M. Kofoed Den tor. 25. feb. 2021 kl. 16.28 skrev Matt Burgess : > Jens, >

Re: Count records where value match x

2021-03-02 Thread Jens M. Kofoed
Hi Mark Many thanks for you replay. I have never used a "where" clauses inside a count in a select statement so I tried it. But it doesn't work all 4 fields has the same value which is equal to the total amount of records. if it had worked it would have been very very nice. Thanks Jens

PartitionsRecord based on part of a string

2021-03-02 Thread Jens M. Kofoed
ionRecord Please help Kind regards Jens M. Kofoed

Re: Count records where value match x

2021-03-03 Thread Jens M. Kofoed
ll end) as count_queuedBytesProcent, count(case when isBackPressureEnabled = 'true' then 1 else null end) as count_isBackPressureEnabled FROM FLOWFILE kind regards Jens Den ons. 3. mar. 2021 kl. 06.44 skrev Jens M. Kofoed : > Hi Mark > > Many thanks for you replay. I have never used a "w

Issue with QueryRecord failing when data is missing

2021-02-24 Thread Jens M. Kofoed
: java.sql.SQLException: Error while preparing statement [SELECT colA FROM FLOWFILE WHERE colC = true] Is this a bug? kind regards Jens M. Kofoed

Re: Issue with GetFTP and arabic letters?

2021-02-22 Thread Jens M. Kofoed
at least. > > Thanks, > Pierre > > Le mar. 23 févr. 2021 à 10:56, Jens M. Kofoed a > écrit : > >> Dear community >> >> I have an issue which I hope some of you might be able to help me with or >> point me in the right direction. >> We have a FTP se

Issue with GetFTP and arabic letters?

2021-02-22 Thread Jens M. Kofoed
from the NiFi server to the FTP server I see the same behavior, Arabic letters is showing as ??? But since FileZilla can see Arabic letters correctly, I can't see that the issue is at the FTP server. So please help Kind regards Jens M. Kofoed

Why is totalSizeCap not a default parameter in the logback.xml file

2021-02-24 Thread Jens M. Kofoed
30 *10GB* true %date %level [%thread] %logger{40} %msg%n Please add this as a new default parameter kind regards Jens M. Kofoed

Re: Issue with GetFTP and arabic letters?

2021-02-23 Thread Jens M. Kofoed
FetchFTP. > We may have a 1.13.1 release coming soon so it would likely be included. > > Thanks, > Pierre > >> Le mar. 23 févr. 2021 à 15:30, Jens M. Kofoed a >> écrit : >> Jira bug created: https://issues.apache.org/jira/browse/NIFI-8220 >> >> ki

Re: Issue with GetFTP and arabic letters?

2021-02-23 Thread Jens M. Kofoed
s Den tir. 23. feb. 2021 kl. 09.14 skrev Pierre Villard < pierre.villard...@gmail.com>: > Let us know if this still doesn't work, we will definitely want to look > into it! > > Le mar. 23 févr. 2021 à 11:47, Jens M. Kofoed a > écrit : > >> Hi Pierre >> >>

Re: Issue with GetFTP and arabic letters?

2021-02-23 Thread Jens M. Kofoed
Villard < pierre.villard...@gmail.com>: > Do you mind filing a JIRA? I had a quick look and the fix should be fairly > straightforward. Will give it a try to confirm. > > Thanks, > Pierre > > Le mar. 23 févr. 2021 à 15:09, Jens M. Kofoed a > écrit : > >> Hi &

Re: Issue with GetFTP and arabic letters?

2021-02-23 Thread Jens M. Kofoed
Jira bug created: https://issues.apache.org/jira/browse/NIFI-8220 kind regards Jens M. Kofoed Den tir. 23. feb. 2021 kl. 12.28 skrev Jens M. Kofoed < jmkofoed@gmail.com>: > no problem, > > By the way I just tried PutFTP and it also have some issue. > I create

Re: Creating recursive missing folders with PutSmbFile

2021-04-20 Thread Jens M. Kofoed
. I went into the folder for the SMB processors, and build the NAR for that specific folder which works. I copied the new NAR file to my NIFI server for testing the new processor, but I can see it does not include the change made for the PR Kind regards Jens M. Kofoed Den man. 22. mar. 2021 kl

Re: Creating recursive missing folders with PutSmbFile

2021-04-20 Thread Jens M. Kofoed
indows Administrators hates to setup new ftp services every time a new server comes in place. So the PutSMB processor is very much needed. Next are some minor changes to the GetSMB ( https://issues.apache.org/jira/projects/NIFI/issues/NIFI-7908) which I will try to see if I can fix. Kind regards Jens M.

Re: Deadlock in loop

2021-08-20 Thread Jens M. Kofoed
ong there are files in the PG no new files will enter. Regards Jens M. Kofoed > Den 20. aug. 2021 kl. 09.49 skrev Aurélien Mazoyer : > > Hi Shawn, > > Thanks for your answer: that sounds good. However I did not manage to remove > the backpressure on queues in Nifi... Is it possib

Re: No Load Balancing since 1.13.2

2021-08-22 Thread Jens M. Kofoed
get the correct IP via nslookup Kind regards Jens M. Kofoed Den fre. 6. aug. 2021 kl. 15.48 skrev Mark Payne : > Jens, > > You’re right - my mistake, the change from > “nifi.cluster.load.balance.address” to “nifi.cluster.load.balance.host” was > in 1.14.0, not early on.

Re: ${hostname(true)} returns localhost

2021-09-09 Thread Jens M. Kofoed
u. > > Thanks > Shawn > > From: Jens M. Kofoed > Sent: Thursday, September 9, 2021 8:25 AM > To: users@nifi.apache.org > Subject: Re: ${hostname(true)} returns localhost > > Thanks Shawn > > Some how the hostname was added to the /etc/hosts file

Re: ${hostname(true)} returns localhost

2021-09-09 Thread Jens M. Kofoed
> > Thanks > Shawn > > From: Jens M. Kofoed > Sent: Thursday, September 9, 2021 5:53 AM > To: users@nifi.apache.org > Subject: ${hostname(true)} returns localhost > > Hi > > I'm using nifi 1.14.0 and have a single node cluster and a 3 node cluster. At > th

${hostname(true)} returns localhost

2021-09-09 Thread Jens M. Kofoed
Hi I'm using nifi 1.14.0 and have a single node cluster and a 3 node cluster. At the single node cluster the ${hostname(true)} returns the FQDN but at my 3 node cluster it returns "localhost". Any ideas why??? Kind regards Jens M. Kofoed

How can HAProxy detect if a node is disconneted

2021-09-09 Thread Jens M. Kofoed
UI is still available and http listener is also active. Is it possible via EL to detect if nodes cluster status? If so, I could create a PG to listen for health check and response a status code depending on the nodes cluster status. Kind regards Jens M. Kofoed

Re: Re: Re: No Load Balancing since 1.13.2

2021-07-29 Thread Jens M. Kofoed
defined "nifi.cluster.node.address" with the hostname and not an ip adress and the "nifi.cluster.load.balance.address" with the ip address of the server. And triple check the configuration at all servers :-) Kind Regards Jens M. Kofoed Den tor. 29. jul. 2021 kl. 10.11 skrev A

Re: No Load Balancing since 1.13.2

2021-08-05 Thread Jens M. Kofoed
m running netstat -l tcp 0 0 localhost:6342 0.0.0.0:* LISTEN Kind regards Jens M. Kofoed Den tor. 5. aug. 2021 kl. 23.08 skrev Mark Payne : > Axel, > > I think that I can help clarify some of these things. > > First of all: nifi.cluster.load.balance.host vs.

Re: Need help to create avro schema for arrays with tags

2021-07-28 Thread Jens M. Kofoed
Expect Records as Array: true Attribute Prefix:prefix_ Field Name for Content: field_name_for_content The JSON Output is the following: { "field_with_attribute" : { "attr" : null, "field_name_for_content" : "content of field" } }

XML Reader: Tags with Attributes not working

2021-07-28 Thread Jens M. Kofoed
} } ] } Expect Records as Array: false Attribute Prefix:prefix_ Field Name for Content: field_name_for_content The JSON Output is the following: { "field_with_attribute" : { "attr" : null, "field_name_for_content" : "content of field" } } Kind regards Jens M. Kofoed

Can't connect to Redis Sentinels after upgrading to v.1.14.0

2021-08-05 Thread Jens M. Kofoed
is the property "SSL Context Service" RedisConnectionPoolService. It says if configured it will use ssl to etablish a secure connection, if not configured til will not be secure. It is not configured. Please any help :-) kind regards Jens M. Kofoed

Re: CryptographicHashContent calculates 2 differents sha256 hashes on the same content

2021-10-20 Thread Jens M. Kofoed
test flows to see if I can find any reason why it behaves as it does. Kind Regards Jens M. Kofoed > Den 20. okt. 2021 kl. 16.39 skrev Mark Payne : > > Jens, > > Thanks for sharing the images. > > I tried to setup a test to reproduce the issue. I’ve had it running f

Re: Nifi and Registry behind Citrix ADC.

2021-10-19 Thread Jens M. Kofoed
Only if you want other ways to authenticate users. I have setup our NIFI systems to talk with our MS AD via ldaps, and defined different AD groups which in nifi has different policy rules. Some people can manage every thing, others can only start/stop specific processors in specific process

Re: CryptographicHashContent calculates 2 differents sha256 hashes on the same content

2021-11-01 Thread Jens M. Kofoed
> Update from hour 67. Still lookin' good. > > > > Will advise. > > > > Thanks > > > > On Thu, Oct 28, 2021 at 8:08 AM Jens M. Kofoed > wrote: > > > > > > Many many thanks  Joe for looking into this. My test flow was > running for 6 days be

Re: CryptographicHashContent calculates 2 differents sha256 hashes on the same content

2021-10-28 Thread Jens M. Kofoed
ny sign of trouble. Will keep running for another > week or so if I can. > > Thanks > >> On Wed, Oct 27, 2021 at 12:42 PM Jens M. Kofoed >> wrote: >> >> The Physical hosts with VMWare is using the vmfs but the vm machines running >> at hosts can’t

Re: CryptographicHashContent calculates 2 differents sha256 hashes on the same content

2021-11-03 Thread Jens M. Kofoed
4 hours (7.6 days) in and zero issues. >> >> Will need to turn this off soon but wanted to give a final update. >> Looks great. Given the information on your system there appears to be >> something we dont understand related to the virtual file system >> involved or something

Re: Nifi secured cluster can't send heartbeat between nodes

2021-10-27 Thread Jens M. Kofoed
Hi I don’t know if this will help. But the certificate used by nifi needs be both a server auth and client auth. Normally certificates are only one of them. The nifi certificate use the server auth for the web ui and when other servers connect to it. It is using the client auth when nifi talks

Re: CryptographicHashContent calculates 2 differents sha256 hashes on the same content

2021-10-27 Thread Jens M. Kofoed
useful/necessary on certain storage >>>> subsystems. >>>> >>>> Thanks >>>> >>>>> On Tue, Oct 26, 2021 at 12:05 PM Joe Witt wrote: >>>>> Ignore "For the scenario where you can replicate this please share the >>

Re: NiFi Queue Monitoring

2021-07-27 Thread Jens M. Kofoed
could be added to the wiki. If not direct copied to the wiki, at least with a link. regards Jens M. Kofoed Den ons. 28. jul. 2021 kl. 00.15 skrev Matt Burgess : > I’m planning on doing one all about QueryNiFiReportingTask and the > RecordSinks, I can include this use case if you like, but

Re: Re: No Load Balancing since 1.13.2

2021-07-28 Thread Jens M. Kofoed
Jens M. Kofoed Den ons. 28. jul. 2021 kl. 11.17 skrev Axel Schwarz : > Just tried Java 11. But still does not work. Nothing changed. :( > > --- Ursprüngliche Nachricht --- > Von: Jorge Machado > Datum: 27.07.2021 13:08:55 > An: users@nifi.apache.org, Axel Schwarz >

Re: Provenance for SplitRecord has a wrong output claim file

2021-07-28 Thread Jens M. Kofoed
Thanks Pierre your are right, I didn't think of that. what should the output claim file be if there are multiple outputs. The output claim file is off course the original file :-) silly me regards Jens M. Kofoed Den ons. 28. jul. 2021 kl. 11.10 skrev Pierre Villard < pierre.villard...@gmail.

Provenance for SplitRecord has a wrong output claim file

2021-07-28 Thread Jens M. Kofoed
a convertRecord process the output claim file is equal to the file coming out of the process Kind regards Jens M. Kofoed

RE:Need help to create avro schema for arrays with tags

2021-07-28 Thread Jens M. Kofoed
ame": "name","type": "string"},{"name": "value","type":["null","string","int","boolean"]}]}}}]}},{ name":"objectIdentification","type":["null",{ "name":"objectIdentificationRecord","type":"record","fields":[{"name":"objectId","type":["null","int"]},{"name":"objectType","type":["null","string"]}]}]}]}]} Is is almost there. All I'm missing is the value for the tagX fields. Please help regards Jens M. Kofoed

Need help to create avro schema for arrays with tags

2021-07-28 Thread Jens M. Kofoed
" objType " } } or "object" : { "objectDetails" : { "additionalInfo" : [ {"name":"tag1", "value":"value1"}, { "name":"tag2", "value":"value2"}, { "name":"tag3", "value":"value3"} ] }, "objectIdentification" : { "objectId" : 1, "objectType" : " objType " } } Kind regards Jens M. Kofoed

InvokeHTTP performance - max out at 50 post per sec.

2022-03-04 Thread Jens M. Kofoed
ind regards Jens M. Kofoed

Re: InvokeHTTP performance - max out at 50 post per sec.

2022-03-04 Thread Jens M. Kofoed
Hi Isha Thanks for your reply. I have tried to play with the scheduling, and it doesn't help much. So I will use the concurrent task. Had hoped there where other things which could be tweaked to get it to run faster. kind regards Jens M. Kofoed Den fre. 4. mar. 2022 kl. 10.16 skrev Isha Lamboo

ListFTP gets NullPointerException with files in DaylightSavingTime

2022-03-28 Thread Jens M. Kofoed
regards Jens M. Kofoed

Re: PutTCP in version 1.15.3 runs with no inputs

2022-02-01 Thread Jens M. Kofoed
Hi Chris Thanks for your reply. My work around is in fact, that I set the scheduling to 1 sec. And hopefully it is fixed and well tested to the next release. I just can't understand why a bug like this should come in a new release. Kind regards Jens M. Kofoed Den tir. 1. feb. 2022 kl. 12.24

Not able to view or download content from queue

2022-02-01 Thread Jens M. Kofoed
rsing media type '' Kind regards Jens M. Kofoed

PutTCP in version 1.15.3 runs with no inputs

2022-02-01 Thread Jens M. Kofoed
continuously without any incoming flow files. Here the process has only being running for a couple of seconds and already have run 3,721,047 tasks. [image: image.png] kind regards Jens M. Kofoed

Re: Help : LoadBalancer

2023-09-07 Thread Jens M. Kofoed
-cluster01.foo.bar pointing to the vip address to keepalived. In your nifi-properties files you would have so set a proxy host address: nifi.web.proxy.host: "nifi-cluster01.foo.bar:8443" This setup is working for me. Kind regards Jens M. Kofoed Den ons. 6. sep. 2023 kl. 16.17 skrev

Re: Help : LoadBalancer

2023-09-07 Thread Jens M. Kofoed
reply. You don't need 2 HAProxy nodes, one can do the job. But it will be one point of failure. You can also just use dns round robin to point to two haproxy nodes, or dive in to the use of keepalived. Kind regards Jens M. Kofoed Den tors. 7. sep. 2023 kl. 13.32 skrev : > > Hello Jens

Load balancing just stopped working in NIFI 1.16.1

2022-05-19 Thread Jens M. Kofoed
Hi I have a 3 node test cluster running v. 1.16.1. Which has been working fine, with no errors. But i doesn't do much, since it is my test cluster. But now I am struggling with load balance connection refuse between nodes. Both node 2 and 3 are refusing load balancing connections, even after

Re: Load balancing just stopped working in NIFI 1.16.1

2022-05-19 Thread Jens M. Kofoed
try deleting the state/ directory on the nodes and restarting. > Perhaps somehow it’s got an old value cached and is trying to communicate > with the wrong port? > > Thanks > -Mark > > >> On May 19, 2022, at 7:42 AM, Jens M. Kofoed wrote: >> >> Hi >>

sftp processors start giving SSH_MSG_UNIMPLEMENTED errors after moving to v. 1.16.1

2022-05-04 Thread Jens M. Kofoed
servers. Any idea why? I have tried to debug it, but have not been able to find any reasons. I've changed the processors to only have 1 concurrent task instead of old settings with 3-5 since we also change from single instance to cluster. Kind regards Jens M. Kofoed

Re: sftp processors start giving SSH_MSG_UNIMPLEMENTED errors after moving to v. 1.16.1

2022-05-05 Thread Jens M. Kofoed
Hi David A JIRA ticket have been made: https://issues.apache.org/jira/browse/NIFI-9989 including a track trace. If more information is needed, please say so :-) Kind regards Jens M. Kofoed Den ons. 4. maj 2022 kl. 14.29 skrev David Handermann < exceptionfact...@apache.org>: >

StandardOauth2AccessTokenProvider gets "token not active"

2022-08-29 Thread Jens M. Kofoed
I can't see if all nodes in the cluster is refreshing the token at the same time, or if it's only the primary nodes which refresh. If all nodes are refreshing could it be that one nodes is slower than the others to refresh, and that the old tokens gets invalid after the first node has refreshed it? Kind regards Jens M. Kofoed

Need help to merge all records in cluster into one flowfile

2022-08-30 Thread Jens M. Kofoed
mergeRecord outputs 2 flowfiles. Any ideas how to force all into one flowfile. Kind regards Jens M. Kofoed

Re: Need help to merge all records in cluster into one flowfile

2022-08-31 Thread Jens M. Kofoed
There, it will show a “Details” field, which will tell you why it merged > the data in the bin. > Once you understand why it’s merging the data with only 2 FlowFiles, you > should be to understand how to adjust your configuration to avoid doing > that. > > Thanks > -Mark > > &

Re: Need help to merge all records in cluster into one flowfile

2022-08-31 Thread Jens M. Kofoed
is invalid: 'Component' is invalid because Failed to perform validation due to java.lang.NumberFormatException: For input string: "" I got the same error if I just tried to set the EL to: ${record.count} Is this a bug??? Kind regards Jens Den ons. 31. aug. 2022 kl. 09.24 skrev Jens

  1   2   >