vasu2809 opened a new issue, #29241:
URL: https://github.com/apache/airflow/issues/29241
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
Google Composer Version - 2.1.3
Airflow Version - 2.3.4
We are trying to perform certain operations (bash) via Google Composer tasks
SSHOperator - When we run parallel Composer DAGs - which means many SSH hook
operator tasks trying to establish SSH connection to a Google Compute Engine VM
at the same time - we intermittently get errors related to the 2 errors given
below
We feel this intermittent issue is being caused due to maybe some timeout
parameter (banner timeout parameter) in paramiko or ssh hook operator - can you
let us know how we could overcome this?
**Code base for one Composer Task**
create_transfer_run_directory = SSHOperator(
task_id="create_transfer_run_directory",
ssh_hook=ComputeEngineSSHHook(
instance_name=GCE_INSTANCE,
zone=GCE_ZONE,
use_oslogin=True,
use_iap_tunnel=False,
use_internal_ip=True,
),
command=f"sudo mkdir -p {transfer_run_directory}/"
'{{ ti.xcom_pull(task_ids="load_config", key="transfer_id") }}',
dag=dag,
)
**Issue 1**
[2023-01-30, 16:51:12 UTC] {compute_ssh.py:258} INFO - Failed to connect.
Waiting 1s to retry
[2023-01-30, 16:51:15 UTC] {transport.py:1874} INFO - Connected (version
2.0, client OpenSSH_8.9p1)
[2023-01-30, 16:51:15 UTC] {transport.py:1874} INFO - Authentication
(publickey) failed.
[2023-01-30, 16:51:15 UTC] {taskinstance.py:1904} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/providers/ssh/operators/ssh.py",
line 157, in execute
with self.get_ssh_client() as ssh_client:
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/providers/ssh/operators/ssh.py",
line 124, in get_ssh_client
return self.get_hook().get_conn()
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/compute_ssh.py",
line 232, in get_conn
sshclient = self._connect_to_instance(user, hostname, privkey,
proxy_command)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/compute_ssh.py",
line 245, in _connect_to_instance
client.connect(
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/compute_ssh.py",
line 50, in connect
return super().connect(*args, **kwargs)
File "/opt/python3.8/lib/python3.8/site-packages/paramiko/client.py", line
450, in connect
self._auth(
File "/opt/python3.8/lib/python3.8/site-packages/paramiko/client.py", line
781, in _auth
raise saved_exception
File "/opt/python3.8/lib/python3.8/site-packages/paramiko/client.py", line
681, in _auth
self._transport.auth_publickey(username, pkey)
File "/opt/python3.8/lib/python3.8/site-packages/paramiko/transport.py",
line 1635, in auth_publickey
return self.auth_handler.wait_for_response(my_event)
File
"/opt/python3.8/lib/python3.8/site-packages/paramiko/auth_handler.py", line
259, in wait_for_response
raise e
paramiko.ssh_exception.AuthenticationException: Authentication failed
**Issue 2**
[2023-01-30, 16�45 UTC] {transport.py:1872} ERROR - File
"/opt/python3.8/lib/python3.8/site-packages/paramiko/transport.py", line 2094,
in run
[2023-01-30, 16�45 UTC] {transport.py:1872} ERROR - self._check_banner()
[2023-01-30, 16�45 UTC] {transport.py:1872} ERROR - File
"/opt/python3.8/lib/python3.8/site-packages/paramiko/transport.py", line 2275,
in _check_banner
[2023-01-30, 16�45 UTC] {transport.py:1872} ERROR - raise SSHException(
[2023-01-30, 16�45 UTC] {transport.py:1872} ERROR -
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
https://stackoverflow.com/questions/25609153/paramiko-error-reading-ssh-protocol-banner
### What you think should happen instead
All SSH connections are successful instead of some failing intermittently
### How to reproduce
_No response_
### Operating System
Composer Kubernetes Cluster
### Versions of Apache Airflow Providers
Airflow Version - 2.3.4
### Deployment
Composer
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
