ryanczarny opened a new issue, #31300:
URL: https://github.com/apache/airflow/issues/31300
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
I am running Airflow v2.3.2 / Python 3.10 from the Docker Image below.
`apache/airflow:2.3.2-python3.10`
The Docker Image has set `paramiko==2.7.2` in order to address the
authentication issues that had been seen in testing.
When calling the sftp, I am using the following:
```
sftp = SFTPHook("connection|sftp")
sftp.look_for_keys = False
sftp.get_conn()
```
I have also tried it without the `sftp.look_for_keys` line.
In the Connections within the Airflow UI, I have configured the Extra
section as follows:
```
{
"private_key": "privatekeyinfo",
"no_host_key_check": true
}
```
When I test the connection within the UI, it reports Connection successfully
tested. However, when the script that calls the Hook runs, I receive the
following:
```
[TIMESTAMP] {transport.py:1819} INFO - Connected (version 2.0, client
dropbear)
[TIMESTAMP] {transport.py:1819} INFO - Authentication (password) failed.
```
I have also attempted to pass the "host_key" in the Extras field but get the
same Authentication error.
To be explicit, I have tried the following -
- sftp.look_for_keys = False and "no_host_key_check": true
- sftp.look_for_keys = False and "host_key": "host_key_value"
- #sftp.look_for_keys = False and "no_host_key_check": true
- #sftp.look_for_keys = False and "host_key": "host_key_value"
- Connections in the Airflow is successful for "no_host_key_check": true
in Extras
- Connections in the Airflow is successful for "host_key":
"host_key_value" in Extras
Referenced SO questions -
[Airflow SFTPHook - No hostkey for host
found](https://stackoverflow.com/questions/54325376/airflow-sftphook-no-hostkey-for-host-found)
[Paramiko AuthenticationException
issue](https://stackoverflow.com/questions/4135261/paramiko-authenticationexception-issue)
[Verify host key with
pysftp](https://stackoverflow.com/questions/38939454/verify-host-key-with-pysftp)
["Failed to load HostKeys" warning while connecting to SFTP server with
pysftp](https://stackoverflow.com/questions/56521549/failed-to-load-hostkeys-warning-while-connecting-to-sftp-server-with-pysftp)
[How to use Airflow to SSH into a server with RSA public/private
keys?](https://stackoverflow.com/questions/59428238/how-to-use-airflow-to-ssh-into-a-server-with-rsa-public-private-keys)
["No hostkey for host ***** found" when connecting to SFTP server with
pysftp using private
key](https://stackoverflow.com/questions/53864260/no-hostkey-for-host-found-when-connecting-to-sftp-server-with-pysftp-usi)
Additional Logging from Paramiko -
```
[TIMESTAMP] {transport.py:1819} DEBUG - starting thread (client mode):
0x9e33d000
[TIMESTAMP] {transport.py:1819} DEBUG - Local version/idstring:
SSH-2.0-paramiko_2.7.2
[TIMESTAMP] {transport.py:1819} DEBUG - Remote version/idstring:
SSH-2.0-dropbear [SERVER]
[TIMESTAMP] {transport.py:1819} INFO - Connected (version 2.0, client
dropbear)
[TIMESTAMP] {transport.py:1819} DEBUG - kex
algos:['diffie-hellman-group1-sha1', 'diffie-hellman-group14-sha256',
'diffie-hellman-group14-sha1'] server key:['ssh-dss', 'ssh-rsa'] client
encrypt:['blowfish-cbc', 'aes128-ctr', 'aes128-cbc', '3des-cbc'] server
encrypt:['blowfish-cbc', 'aes128-ctr', 'aes128-cbc', '3des-cbc'] client
mac:['hmac-sha1', 'hmac-md5-96', 'hmac-sha1-96', 'hmac-md5'] server
mac:['hmac-sha1', 'hmac-md5-96', 'hmac-sha1-96', 'hmac-md5'] client
compress:['none'] server compress:['none'] client lang:[''] server lang:['']
kex follows?False
[TIMESTAMP] {transport.py:1819} DEBUG - Kex agreed:
diffie-hellman-group14-sha256
[TIMESTAMP] {transport.py:1819} DEBUG - HostKey agreed: ssh-rsa
[TIMESTAMP] {transport.py:1819} DEBUG - Cipher agreed: aes128-ctr
[TIMESTAMP] {transport.py:1819} DEBUG - MAC agreed: hmac-sha1
[TIMESTAMP] {transport.py:1819} DEBUG - Compression agreed: none
[TIMESTAMP] {transport.py:1819} DEBUG - kex engine KexGroup14SHA256
specified hash_algo <built-in function openssl_sha256>
[TIMESTAMP] {transport.py:1819} DEBUG - Switch to new keys ...
[TIMESTAMP] {transport.py:1819} DEBUG - Attempting password auth...
[TIMESTAMP] {transport.py:1819} DEBUG - userauth is OK
[TIMESTAMP] {transport.py:1819} INFO - Authentication (password) failed.
```
### What you think should happen instead
Authentication should verify and allow for the SFTPHook to perform as
expected.
### How to reproduce
1. Get an SFTP connection that requires a private key
2. Add the connection to the `Connections` in Airflow
3. Add the private key in the `Extras` field
4. Test both setting `"no_host_key_check": true` and
`"host_key"="public_key"`
5. Test the connection and receive a `Connected` Result (green banner across
the top)
6. Create a Python DAG
7. Import `from airflow.providers.sftp.hooks.sftp import SFTPHook`
8. access the connection in the script `sftp = SFTPHook("conn_name|sftp")`
9. Establish the connection `sftp.get_conn()`
10. Try the following with and without `sftp.look_for_keys = False`
` sftp.store_file('./'+filename,filename,confirm=True)`
### Operating System
apache/airflow:2.3.2-python3.10
### Versions of Apache Airflow Providers
apache-airflow-providers-sftp==4.2.4
apache-airflow-providers-ssh==3.6.0
### Deployment
Other 3rd-party Helm chart
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
