AW: Processor with cron scheduling in middle of flow

2023-02-22 Thread Dobbernack, Harald (Key-Work)
Just responding to this part: > You should not be using CRON driven for any processors in the middle of a > flow. In fact, we really > should probably just disable that all together. Please don't disable this! We actually use CRON for some of our PutSFTP Processors as there are servicetimes of

AW: Thank You community

2022-10-04 Thread Dobbernack, Harald (Key-Work)
| Perhaps I will explain this better in a medium Post. oh please do! Am looking forward to reading about your adapting flow solution! But also enjoy your free time! Von: Carlos Manuel Fernandes (DSI) Gesendet: Freitag, 30. September 2022 17:11 An: users@nifi.apache.org Betreff: Thank You

AW: Delete FlowFile content

2021-05-20 Thread Dobbernack, Harald (Key-Work)
My choice in the past for such a case was AttributesToJSON with Destination=flowfile-content thinking that this really quickly gets rid of large content, albeit replacing it with a small json … don’t know if this is better or worse performance wise than ReplaceText, but it does get the job

AW: Some retry flowfile questions

2021-04-23 Thread Dobbernack, Harald (Key-Work)
Mark, thank you so much for this great explanation! Harald Von: Mark Payne Gesendet: Donnerstag, 22. April 2021 22:32 An: users@nifi.apache.org Betreff: Re: Some retry flowfile questions Geoff, The difference between penalization and yielding is whether the failure is data-dependent or not.

AW: GetFTP Socket timeouts

2021-04-06 Thread Dobbernack, Harald (Key-Work)
Please avoid using 1.13.1 (session handling broken) and use 1.13.2 (I have not read your entire post, just jumped at the version) Von: Samu Gesendet: Dienstag, 6. April 2021 07:04 An: users@nifi.apache.org Betreff: GetFTP Socket timeouts I am using Nifi 1.13.1 and GetFTP gives me socket

AW: Strict order of flow files in a cluster

2021-04-01 Thread Dobbernack, Harald (Key-Work)
essed simultaneously." In the tooltip: "Only a single FlowFile is to be allowed to enter the Process Group at a time on each node in the cluster." So it's not accros the cluster. This is also the behaviour I notice after a quick test. On 01/04/2021, 10:24, "Dobbernack, Harald (K

AW: Strict order of flow files in a cluster

2021-04-01 Thread Dobbernack, Harald (Key-Work)
I suppose the feature 'Ability to specify group level flow file concurrency - for instance run a single flow file end to end before running another for traditional job handling' available from Version 1.12 upward should be helpful here (have not tried myself yet) -Ursprüngliche

AW: Change time zone in NiFi

2020-11-26 Thread Dobbernack, Harald (Key-Work)
In https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#format you can see how to add your timezone, examples are in ‘Table 26. format Examples’ Hope that helps. Von: naga satish Gesendet: Donnerstag, 26. November 2020 10:28 An: users@nifi.apache.org Betreff: Change time

AW: Triggered fetching of multiple files from FTP - how?

2020-07-09 Thread Dobbernack, Harald (Key-Work)
Hi, you can use a listftp to know the filenames and then trigger fetchfile via wait processor as soon as it's ok to fetch them Kind Regards harald Von: Luca Giovannini Gesendet: Donnerstag, 9. Juli 2020 15:25 An: users@nifi.apache.org Betreff: Triggered fetching of multiple files from FTP

AW: NiFi 1.11 - High repository storage usage

2020-06-25 Thread Dobbernack, Harald (Key-Work)
* When I check the size of the respective folders on the disk they don't even add up to 12,24 GB shown on the screenshot. I'm guessing you have other stuff on the drive or partition as well and not only the repositories. Von: Dobbernack, Harald (Key-Work) Gesendet: Donnerstag, 25. Juni

AW: NiFi 1.11 - High repository storage usage

2020-06-25 Thread Dobbernack, Harald (Key-Work)
Hi, presumably all three respositories are on the same partition or drive? I believe the screenshot view you posted shows what the OS reports as the space usage of the whole partition/drive on which the repositories are sitting. Best practice would be to place each repository on its own

AW: Not Seeing Provenance data

2020-04-09 Thread Dobbernack, Harald (Key-Work)
. Thanks -Mark [1] https://issues.apache.org/jira/browse/NIFI-7346 On Apr 9, 2020, at 11:14 AM, Dobbernack, Harald (Key-Work) mailto:harald.dobbern...@key-work.de>> wrote: Hi Mark, I can confirm after testing that if no provenance event has been generated in a time greater than t

AW: Not Seeing Provenance data

2020-04-09 Thread Dobbernack, Harald (Key-Work)
file”.’ fits very nicely to my test. As a workaround we’re going to set a greater nifi.provenance.repository.max.storage.time until this can be resolved. Thanks again for looking into this. Harald Von: Dobbernack, Harald (Key-Work) Gesendet: Donnerstag, 9. April 2020 15:22 An: users

AW: Not Seeing Provenance data

2020-04-09 Thread Dobbernack, Harald (Key-Work)
-Mark On Apr 9, 2020, at 4:32 AM, Dobbernack, Harald (Key-Work) mailto:harald.dobbern...@key-work.de>> wrote: What I noticed is that as long as provenance is working there will be *.prov files in the directory. When Provenance isn’t working these files are not to be seen. Maybe so

AW: Not Seeing Provenance data

2020-04-09 Thread Dobbernack, Harald (Key-Work)
, Harald (Key-Work) Gesendet: Donnerstag, 9. April 2020 10:27 An: users@nifi.apache.org Betreff: AW: Not Seeing Provenance data This is something I experience too from time to time. My quick and dirty workaround is stop nifi, delete everything in the provenance directory, restart

AW: Not Seeing Provenance data

2020-04-09 Thread Dobbernack, Harald (Key-Work)
This is something I experience too from time to time. My quick and dirty workaround is stop nifi, delete everything in the provenance directory, restart Then Provenance is usable again (of course only with data since the delete) . I'm hoping very much there is a better way, someone can

Re: AvroSchemaRegistry reflects changed or new Schemas only after NiFi Restart?

2020-03-10 Thread Dobbernack, Harald (Key-Work)
A new Jira has been opened concerning this: https://issues.apache.org/jira/browse/NIFI-7242 On 2020/03/06 09:36:35, "Dobbernack, Harald (Key-Work)" wrote: > Hi,> > > > > when testing a new flow on NiFi 1.11.1 I got an Error stating a missing > S

AvroSchemaRegistry reflects changed or new Schemas only after NiFi Restart?

2020-03-06 Thread Dobbernack, Harald (Key-Work)
Hi, when testing a new flow on NiFi 1.11.1 I got an Error stating a missing Schema that I had actually added to the AvroSchemaRegistry beforehand. I then restarted the NiFi Service and everything worked then, the flow then had no problems. I also experienced that changes to an already existing

AW: Logging/Monitoring of Invalid Processors at NiFi Startup

2020-03-05 Thread Dobbernack, Harald (Key-Work)
Pierre Le mer. 4 mars 2020 à 16:44, <mailto:josef.zahn...@swisscom.com> a écrit : With logback.xml you can finetune logmessages, but don't ask me the details __. Cheers Josef On 04.03.20, 16:35, "Dobbernack, Harald (Key-Work)" <mailto:harald.dobbern...@key-work.de> wrot

AW: Logging/Monitoring of Invalid Processors at NiFi Startup

2020-03-04 Thread Dobbernack, Harald (Key-Work)
e-mail isn’t the right medium to alert/monitor, eg. If you have massiv issues it would flood your e-mail account completely with warnings – and I don’t think that you want that. Cheers Josef From: "Dobbernack, Harald (Key-Work)" <mailto:harald.dobbern...@key-work.de> Reply t

Logging/Monitoring of Invalid Processors at NiFi Startup

2020-03-04 Thread Dobbernack, Harald (Key-Work)
Our standalone Nifi 1.11.1 on Debian 10.2 will on service startup not throw an Error or Warning into the NiFi log if it deams a processor invalid, for example if a samba mount is not available and the listfile or getfile processors cannot reach the mount. If, on the other hand, the processors