[jira] [Comment Edited] (NIFI-5764) Allow ListSftp connection parameter
[
https://issues.apache.org/jira/browse/NIFI-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676491#comment-16676491
]
dav edited comment on NIFI-5764 at 11/6/18 10:08 AM:
-
Hi [~ijokarumawak],
this is the dump of the node 2 of our cluster. if it helps you, i can give you
the other 2 nodes dumps.
Let we know.
Thank you
[^dumpone]
was (Author: dav):
Hi [~ijokarumawak],
this is the dump of the node 2 of our cluster. if it helps you, i can give you
the other 2 nodes dumps.
Let we know.[^dumpone]
> Allow ListSftp connection parameter
> ---
>
> Key: NIFI-5764
> URL: https://issues.apache.org/jira/browse/NIFI-5764
> Project: Apache NiFi
> Issue Type: Wish
> Components: Extensions
>Affects Versions: 1.6.0
>Reporter: dav
>Priority: Critical
> Labels: SFTP, customization, sftp
> Attachments: dumpone
>
>
> ListSftp and other Sftp processors should be able to add parameters
> (like [-B buffer_size] [-b batchfile] [-c cipher]
> [-D sftp_server_path] [-F ssh_config] [-i identity_file] [-l limit]
> [-o ssh_option] [-P port] [-R num_requests] [-S program]
> [-s subsystem | sftp_server] host
> sftp [user@]host[:file ...]
> sftp [user@]host[:dir[/]]
> sftp -b batchfile [user@]host)
> in order to edit the type of connection on Sftp Server.
> For instance, I have this error on nifi:
> 2018-10-29 11:06:09,462 ERROR [Timer-Driven Process Thread-5]
> SimpleProcessLogger.java:254
> ListSFTP[id=766ac418-27ce-335a-5b13-52abe3495592] Failed to perform listing
> on remote host due to java.io.IOException: Failed to obtain connection to
> remote host due to com.jcraft.jsch.JSchException: Auth fail: {}
> java.io.IOException: Failed to obtain connection to remote host due to
> com.jcraft.jsch.JSchException: Auth fail
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getChannel(SFTPTransfer.java:468)
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:192)
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:156)
> at
> org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:105)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:401)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1147)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:175)
> at
> org.apache.nifi.controller.scheduling.QuartzSchedulingAgent$2.run(QuartzSchedulingAgent.java:140)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: com.jcraft.jsch.JSchException: Auth fail
> at com.jcraft.jsch.Session.connect(Session.java:519)
> at com.jcraft.jsch.Session.connect(Session.java:183)
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getChannel(SFTPTransfer.java:448)
> ... 15 common frames omitted
> This can be avoided by connect to Sftp server with this string:
> *sftp -o “controlmaster auto” username@sftp_server*
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-5764) Allow ListSftp connection parameter
[
https://issues.apache.org/jira/browse/NIFI-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675468#comment-16675468
]
Alfredo De Luca edited comment on NIFI-5764 at 11/5/18 5:45 PM:
Hi [~ijokarumawak]. Thanks for that but I can't fix that issue that we have. So
i did a few test (not with NiFi) and if I use a controlmaster on my ssh
connection I don't get this error.
Caused by: com.jcraft.jsch.JSchException: Auth fail
Any idea/thoughts?
Cheers
was (Author: alfredo.deluca):
Hi [~ijokarumawak]. Thanks for that but I can't fix that issue that we have. So
i did a few test (not with NiFi) and if I use a controlmaster on my ssh
connection I don't get this error.
Caused by: com.jcraft.jsch.JSchException: Auth fail
Any idea/thoughts?
Cheerd
> Allow ListSftp connection parameter
> ---
>
> Key: NIFI-5764
> URL: https://issues.apache.org/jira/browse/NIFI-5764
> Project: Apache NiFi
> Issue Type: Wish
> Components: Extensions
>Affects Versions: 1.6.0
>Reporter: dav
>Priority: Critical
> Labels: SFTP, customization, sftp
>
> ListSftp and other Sftp processors should be able to add parameters
> (like [-B buffer_size] [-b batchfile] [-c cipher]
> [-D sftp_server_path] [-F ssh_config] [-i identity_file] [-l limit]
> [-o ssh_option] [-P port] [-R num_requests] [-S program]
> [-s subsystem | sftp_server] host
> sftp [user@]host[:file ...]
> sftp [user@]host[:dir[/]]
> sftp -b batchfile [user@]host)
> in order to edit the type of connection on Sftp Server.
> For instance, I have this error on nifi:
> 2018-10-29 11:06:09,462 ERROR [Timer-Driven Process Thread-5]
> SimpleProcessLogger.java:254
> ListSFTP[id=766ac418-27ce-335a-5b13-52abe3495592] Failed to perform listing
> on remote host due to java.io.IOException: Failed to obtain connection to
> remote host due to com.jcraft.jsch.JSchException: Auth fail: {}
> java.io.IOException: Failed to obtain connection to remote host due to
> com.jcraft.jsch.JSchException: Auth fail
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getChannel(SFTPTransfer.java:468)
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:192)
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:156)
> at
> org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:105)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:401)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1147)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:175)
> at
> org.apache.nifi.controller.scheduling.QuartzSchedulingAgent$2.run(QuartzSchedulingAgent.java:140)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: com.jcraft.jsch.JSchException: Auth fail
> at com.jcraft.jsch.Session.connect(Session.java:519)
> at com.jcraft.jsch.Session.connect(Session.java:183)
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getChannel(SFTPTransfer.java:448)
> ... 15 common frames omitted
> This can be avoided by connect to Sftp server with this string:
> *sftp -o “controlmaster auto” username@sftp_server*
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-5764) Allow ListSftp connection parameter
[
https://issues.apache.org/jira/browse/NIFI-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669499#comment-16669499
]
Koji Kawamura edited comment on NIFI-5764 at 10/31/18 2:15 AM:
---
The underlying Jsch library used by NiFi does not support openssh ControlMaster
feature.
https://stackoverflow.com/questions/35975099/is-it-possible-to-add-controlmaster-to-a-jsch-connection
was (Author: ijokarumawak):
The underlying Jsch library used by NiFi does not support openssh ControlMaster
feature.
https://stackoverflow.com/questions/35975099/is-it-possible-to-add-controlmaster-to-a-jsch-connection
If the connection issue comes from the fact that ListSFTP establishes new
connection at each onTrigger, then we may be able to reuse the same
SFTPTransfer instance at ListSFTP processor to mitigate the issue.
Also, current code doesn't seem to call SFTPTransfer.close(), wondering if that
leads to the connection issue, too.
FetchSFTP pools SFTPTransfer instances to reuse. Pooling is implemented at its
superclass, FetchFileTransfer.
> Allow ListSftp connection parameter
> ---
>
> Key: NIFI-5764
> URL: https://issues.apache.org/jira/browse/NIFI-5764
> Project: Apache NiFi
> Issue Type: Wish
> Components: Extensions
>Affects Versions: 1.6.0
>Reporter: dav
>Priority: Critical
> Labels: SFTP, customization, sftp
>
> ListSftp and other Sftp processors should be able to add parameters
> (like [-B buffer_size] [-b batchfile] [-c cipher]
> [-D sftp_server_path] [-F ssh_config] [-i identity_file] [-l limit]
> [-o ssh_option] [-P port] [-R num_requests] [-S program]
> [-s subsystem | sftp_server] host
> sftp [user@]host[:file ...]
> sftp [user@]host[:dir[/]]
> sftp -b batchfile [user@]host)
> in order to edit the type of connection on Sftp Server.
> For instance, I have this error on nifi:
> 2018-10-29 11:06:09,462 ERROR [Timer-Driven Process Thread-5]
> SimpleProcessLogger.java:254
> ListSFTP[id=766ac418-27ce-335a-5b13-52abe3495592] Failed to perform listing
> on remote host due to java.io.IOException: Failed to obtain connection to
> remote host due to com.jcraft.jsch.JSchException: Auth fail: {}
> java.io.IOException: Failed to obtain connection to remote host due to
> com.jcraft.jsch.JSchException: Auth fail
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getChannel(SFTPTransfer.java:468)
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:192)
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:156)
> at
> org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:105)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:401)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1147)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:175)
> at
> org.apache.nifi.controller.scheduling.QuartzSchedulingAgent$2.run(QuartzSchedulingAgent.java:140)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: com.jcraft.jsch.JSchException: Auth fail
> at com.jcraft.jsch.Session.connect(Session.java:519)
> at com.jcraft.jsch.Session.connect(Session.java:183)
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getChannel(SFTPTransfer.java:448)
> ... 15 common frames omitted
> This can be avoided by connect to Sftp server with this string:
> *sftp -o “controlmaster auto” username@sftp_server*
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
