Hi, David.
Sorry for the misunderstanding, my fault. Firstly, we tried using ListSFTP
and FetchSFTP, and when it didn't work, we tried another option (GetSFTP),
and I pasted the wrong stack trace. So, i've done the following tests:
- ListSFTP + FetchSFTP: Error in ListSFTP [1]
- GetSFTP: Error [2]
- Generate flowfile + FetchSFTP with the name of the symlink in the *Remote
File* property: OK, the file is downloaded.
So, it seems the issue is in ListSFTP and GetSFTP, but FetchSFTP works
fine.
Thanks. Regards
--
Guille
[1]
2022-02-03 17:36:45,466 ERROR [Timer-Driven Process Thread-8]
o.a.nifi.processors.standard.ListSFTP
ListSFTP[id=64443154-ac76-1736-9e49-f2ca388dfbdf] Unable to get listing
from *****.gz; skipping: java.io.FileNotFoundException: Could not perform
listing on *****.gz because could not find the file on the remote server
java.io.FileNotFoundException: Could not perform listing on *****.gz
because could not find the file on the remote server
at
org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:350)
at
org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:365)
at
org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:262)
at
org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:120)
at
org.apache.nifi.processors.standard.ListSFTP.performListing(ListSFTP.java:150)
at
org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:112)
at
org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:750)
at
org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:525)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1273)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
at
org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:63)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2]
2022-02-03 17:39:14,714 ERROR [Timer-Driven Process Thread-27]
o.a.nifi.processors.standard.GetSFTP
GetSFTP[id=c0300c77-017e-1000-ffff-fffffa9c1f31] Unable to get listing from
*****.gz; skipping: java.io.FileNotFoundException: Could not perform
listing on *****.gz because could not find the file on the remote server
java.io.FileNotFoundException: Could not perform listing on *****.gz
because could not find the file on the remote server
at
org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:350)
at
org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:365)
at
org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:262)
at
org.apache.nifi.processors.standard.GetFileTransfer.fetchListing(GetFileTransfer.java:299)
at
org.apache.nifi.processors.standard.GetFileTransfer.onTrigger(GetFileTransfer.java:126)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1273)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
at
org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:63)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
El jue, 3 feb 2022 a las 17:09, David Handermann (<
[email protected]>) escribió:
> Hi Guille,
>
> Thanks for raising this issue and providing a stack trace. You mentioned
> using ListSFTP and FetchSFTP, but the stack references GetFileTransfer,
> which corresponds to GetSFTP.
>
> Can you confirm the same error using FetchSFTP? If you can confirm the
> same issue with FetchSFTP, it would be very helpful to add those details on
> the newer Jira issue NIFI-6699.
>
> NiFi SFTP processors switched to a different SSH library after the
> resolution of NIFI-5560, so it is possible that some changes may be
> necessary. However, it would be helpful to confirm whether this is an
> issue with FetchSFTP, GetSFTP, or both processors.
>
> Regards,
> David Handermann
>
> On Thu, Feb 3, 2022 at 9:57 AM Guillermo Muñoz Salgado <[email protected]>
> wrote:
>
>> Hi all,
>>
>> We are developing a use case in which we have to get some files from a
>> server. We have implemented it by the listSFTP + FetchSFTP way in a 3 nodes
>> cluster running nifi 1.15.3. But we are having some issues when what
>> we want to get are symlinks instead of files. We have set true the property
>> *"Follow
>> symlink" *but we get the same results. Are we doing something wrong? Or
>> is it a bug or a known issue? We have found this issue [1] but it is old
>> and resolved and this other one [2], that is older and unresolved. We're
>> not sure if they are related to this behaviour or not.
>>
>> I paste our error log:
>>
>> 2022-02-03 16:27:41,002 ERROR [Timer-Driven Process Thread-18]
>> o.a.nifi.processors.standard.GetSFTP
>> GetSFTP[id=c0300c77-017e-1000-ffff-fff-ffa9c1f31] Unable to get listing
>> from testfile.gz; skipping: java.io.FileNotFoundException: Could not
>> perform listing on testfile.gz because could not find the file on the
>> remote server
>> java.io.FileNotFoundException: Could not perform listing on testfile.gz
>> because could not find the file on the remote server
>> at
>> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:350)
>> at
>> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:365)
>> at
>> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:262)
>> at
>> org.apache.nifi.processors.standard.GetFileTransfer.fetchListing(GetFileTransfer.java:299)
>> at
>> org.apache.nifi.processors.standard.GetFileTransfer.onTrigger(GetFileTransfer.java:126)
>> at
>> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>> at
>> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1273)
>> at
>> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
>> at
>> org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:63)
>> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>>
>> Thanks in advance
>> --
>> Guille
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-5560
>> [2] https://issues.apache.org/jira/browse/NIFI-6699
>>
>
--
Guillermo Muñoz Salgado
