Re: FetchSFTP: Rename file on move

Thanks for the suggestions. The combination of UpdateAttribute (Setting the new filename) with PutSFTP works fine! On Tue, Aug 11, 2020 at 5:59 PM Andy LoPresto wrote: > You can also use an UpdateAttribute processor to change the “filename” > attribute, which is what any “file persistence”

Re: FetchSFTP: Rename file on move

You can also use an UpdateAttribute processor to change the “filename” attribute, which is what any “file persistence” processor (PutSFTP, PutFile, etc.) will use when writing the file out. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com He/Him PGP Fingerprint: 70EC B3E5 98A6

Re: FetchSFTP: Rename file on move

Jairo You can use a PutSFTP after Fetch to place it where you want. Thanks On Tue, Aug 11, 2020 at 3:16 PM Jairo Henao wrote: > Hi community, > > Is there a way to rename a file before moving it with FetchSFTP? > > After processing a file, I need to move it to a folder and add a timestamp >

FetchSFTP: Rename file on move

Hi community, Is there a way to rename a file before moving it with FetchSFTP? After processing a file, I need to move it to a folder and add a timestamp suffix to it. The file in the source always has the same name, but I need that when moving it they are not overwritten. Any ideas or is it

Re: ReplaceText - Out of memory - Requested array size exceeds VM limit

Asmath ReplaceText either loads full lines at a time or loads the entire file into memory. So keep that in mind. If you need something that only loads at worst 1-2x the length of the replacement string you're interested in then I'd recommend just using a scripted processor that does precisely

ReplaceText - Out of memory - Requested array size exceeds VM limit

Hi , I have a file that is throwing an error when looking for particular string and replace it with other string. Requested array size exceeds VM limit Any suggestions for this? File is around 800 MB. Thanks, Asmath

Re: Get all available variables in the InvokeScriptedProcessor

Although this is an "unnatural" use of Groovy (and a conversation much better suited for the dev list :), it is possible to get at a map of defined variables (key and value). This counts on particular implementations of the API and that there is no SecurityManager installed in the JVM so Groovy

Re: NIFI /Kafka - Data loss possibility with node failures

Thanks Joe. This is really helpful. On Tue, Aug 11, 2020 at 9:33 AM Joe Witt wrote: > Asmath > > In a traditional installation, regardless of how a NiFi cluster obtains > data (kafka, ftp, HTTP calls, TCP listening, etc, ) once it is > responsible for the data it has ack'd its receipt to

Re: NIFI /Kafka - Data loss possibility with node failures

Asmath In a traditional installation, regardless of how a NiFi cluster obtains data (kafka, ftp, HTTP calls, TCP listening, etc, ) once it is responsible for the data it has ack'd its receipt to the source(s). If that NiFi node were to become offline the data it owns is delayed. If that node

NIFI /Kafka - Data loss possibility with node failures

Hi, [image: image.png] we have 3 node NIFI clusters and due to some reasons NODE 2 and NODE 3 were disconnected when the flow was running . Consume kafka was reading data from all node settings and loading the data into the database. In the above scenario, is there a possibility of loss of

InvokeGRPC with External Proto

Hi, I need to wire an external gRPC service into Nifi - could anyone provide some pointers on how to change the service IDL / proto within the InvokeGRPC processor to support the existing external gRPC service proto definitions? Am trying to avoid having to writing a middleware processor to

Re: Data Provenance Stops Working

Darren - I'm using NiFi 1.11.4 for a while now (with OOTB configuration) and I do have provenance data for the flows I'm running. Le lun. 10 août 2020 à 23:24, Wyll Ingersoll < wyllys.ingers...@keepertech.com> a écrit : > Ah! That fixed my problem! > > I am running a secure/authenticated