Thanks Joe and Pierre,
Yep, that was the way to go indeed and thanks for your confirmation, time and
replies [😊]
I've moved matching the static part to the ListSFTP regex with File Filter
Regex = test.file.\d{8}-\d{6}.csv
and matched with Pierre's expression to simplify the route condition. Further
down in GetSFTP, I had to specify ${path}/${filename} to fetch the file.
Best regards,
________________________________
From: Pierre Villard <pierre.villard...@gmail.com>
Sent: April 21, 2016 1:18 PM
To: users@nifi.apache.org
Subject: Re: datetime argument to the GetSFTP processor
Hi Ashraf,
I believe you are right, RouteOnAttribute is certainly what you should use.
Regarding the expression, it sounds good to me. Depending of the filename
format characteristics, may be you can directly use
${filename:contains( ${now():toNumber():minus(86400000):format('yyyymmdd')} )}
2016-04-21 19:11 GMT+02:00 Ashraf Hasson
<ahas...@windmobile.ca<mailto:ahas...@windmobile.ca>>:
Hi there,
Okay, so I should use RouteOnAttribute I think.
Here's the source filename: test.file.20160420-015931.csv
I've configured the RouteOnAttribute to route the `filename` to success when
the following is matched:
property: filename == value:
${filename:substringAfter('test.file.'):startsWith(
${now():toNumber():minus(86400000):format('yyyymmdd')} )}
Maybe this is not the right way but I'd highly appreciate some help.
Thanks,
________________________________________
From: Ashraf Hasson
Sent: April 21, 2016 12:50 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: Re: datetime argument to the GetSFTP processor
Hi James,
Thanks for your reply.
The source has multiple files generated per day and I'm interested in one of
those, so filtering is required I guess.
I don't know how to filter based on the filename, was trying to pipe things
like this: ListSFTP -> UpdateAttribute -> FetchSFTP
but I'm sure there's a better way to filter. Can you help by giving me a
pointer or two please?
I also tried to pipe directly from ListSFTP -> FetchSFTP and used
${now():toNumber():minus(86400000):format('yyyymmdd')}-*.csv but then this
doesn't match the remote file guessing because FetchSFTP doesn't do globing.
Thanks,
________________________________________
From: Joe Witt <joe.w...@gmail.com<mailto:joe.w...@gmail.com>>
Sent: April 15, 2016 1:35 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: Re: datetime argument to the GetSFTP processor
Ahh - great idea James!
On Fri, Apr 15, 2016 at 1:34 PM, James Wing
<jvw...@gmail.com<mailto:jvw...@gmail.com>> wrote:
> Have you looked at a combination of ListSFTP -> FetchSFTP? I believe
> ListSFTP has built-in support for tracking recent files, and it might
> satisfy your use case. If not, you can certainly filter the listed files by
> the "filename" attribute before calling FetchSFTP.
>
> On Fri, Apr 15, 2016 at 7:40 AM, Ashraf Hasson
> <ahas...@windmobile.ca<mailto:ahas...@windmobile.ca>>
> wrote:
>>
>> Hi there,
>>
>>
>> I've been trying to use the GetSFTP processor to fetch files that have a
>> 'yyyymmdd' part but the only option I have is to use Java Regular
>> Expression, not sure how to specify today or yesterday's date dynamically.
>>
>>
>> I thought an alternative is to use a locally generated file that contains
>> the datetime string, use a GetFile + UpdateAttribute to pass in a custom
>> attribute to the GetSFTP processor that way, but sounded a bit cumbersome
>> tbh.
>>
>>
>> Any thoughts/hints please?
>>
>>
>> Thanks,
>>
>> Ashraf
>
>
