[
https://issues.apache.org/jira/browse/MINIFICPP-542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16828934#comment-16828934
]
Daniel Bakai edited comment on MINIFICPP-542 at 4/29/19 7:47 AM:
-----------------------------------------------------------------
[~phrocker]
I’ve looked at curl an other possible implementations (libssh, libssh2,
OpenSSH). curl itself uses libssh2 internally. I’ve found that while we could
probably provide most of the things we need using just curl, we wouldn’t have
the control I would be comfortable with. curl works in a “one download or
upload operation on a file or directory, and random commands before and after
that” model: you specify one upload or download, and other commands to execute
before that (CURLOPT_QUOTE) and after that (CURLOPT_POSTQUOTE). However, you
don’t really have much control over these commands. You can either flag them as
fatal, if failed, or non-fatal, if failed, but after executing the whole
operation, you don’t know the result for each of these commands, and if one of
them fails, you don’t know which one did. You also don’t have the range of
parameterization available for commands which are available natively in the
protocol and also provided by libssh2, and some of which I see would be useful
in implementing the processors. Because we need to execute many of these
commands aside from the main upload or download action, this is not too
practical.
OpenSSH doesn’t have a convenient interface for in-memory SFTP, libssh is under
LGPL.
libssh2 provides a stable and user-friendly interface, and curl’s
implementation doesn’t really add any significant business logic to that, so I
would prefer to use libssh2 only, if not for the need for proxy support:
libssh2 can't itself connect to a proxy.
curl's proxy support is excellent, on the other hand, and fortunately we can
use CURLOPT_CONNECTONLY and CURLOPT_ACTIVESOCKET to sort out all the proxy
operations and give us a socket to use afterwards. We can then give that socket
to libssh2, and implement our SFTP functionality with it.
I think this is our best option, so I’ve created a standalone tool for testing
it, that can use curl to connect to a proxy, then download and upload files
through SFTP using libssh2, and it works with both SOCKS and HTTP (CONNECT)
proxies.
Today I plan to start the work of integrating this into a processor.
was (Author: bakaid):
[~phrocker]
I’ve looked at curl an other possible implementations (libssh, libssh2,
OpenSSH). curl itself uses libssh2 internally. I’ve found that while we could
probably provide most of the things we need using just curl, we wouldn’t have
the control I would be comfortable with. curl works in a “one download or
upload operation on a file or directory, and random commands before and after
that” model: you specify one upload or download, and other commands to execute
before that (CURLOPT_QUOTE) and after that (CURLOPT_POSTQUOTE). However, you
don’t really have much control over these commands. You can either flag them as
fatal, if failed, or non-fatal, if failed, but after executing the whole
operation, you don’t know the result for each of these commands, and if one of
them fails, you don’t know which one did. You also don’t have the range of
parameterization available for commands which are available natively in the
protocol and also provided by libssh2, and some of which I see would be useful
in implementing the processors. Because we need to execute many of these
commands aside from the main upload or download action, this is not too
practical.
OpenSSH doesn’t have a convenient interface for in-memory SFTP, libssh is under
LGPL.
libssh2 provides a stable and user-friendly interface, and curl’s
implementation doesn’t really add any significant business logic to that, so I
would prefer to use libssh2 only, if not for the need for proxy support:
libssh2 can't itself connect to a proxy.
curl's proxy support is excellent, on the other hand, and fortunately we can
use CURLOPT_CONNECTONLY and CURLOPT_ACTIVESOCKET to sort out all the proxy
operations and give us a socket to use afterwards. We can then give that socket
to libssh2, and implement our SFTP functionality with it.
I think this is our best option, so I’ve created a standalone tool for testing
it, that can use curl to connect to a proxy, then download and upload files
using through SFTP using libssh2, and it works with both SOCKS and HTTP
(CONNECT) proxies.
Today I plan to start the work of integrating this into a processor.
> Add PutSFTP Support
> -------------------
>
> Key: MINIFICPP-542
> URL: https://issues.apache.org/jira/browse/MINIFICPP-542
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
> Reporter: Mr TheSegfault
> Assignee: Daniel Bakai
> Priority: Major
> Labels: easyfix, newbie
> Fix For: 0.7.0
>
>
> SFTP is an easy operation to add through the use of libCURL
>
> Processors the can be added:
> (priority) PutSFTP
> ListSFTP
> FetchSFTP
> GetSFTP
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
