dabla commented on issue #47201: URL: https://github.com/apache/airflow/issues/47201#issuecomment-2691069740
> > Duplicate of [#47129](https://github.com/apache/airflow/issues/47129) ? > > Yes. I think this will continue to break for the users depending upon how they are using it at the moment. For example we have a wrapper around SFTPHook that will allow us to do multiple interactions with SFTP server with one open connection. For instance, check if certain file matches the pattern, send the file to different server(and additionally create directory on the destination server) and then delete the file from source. > > ``` > def my_file_transfer_function(): > srcConnectionWrapper = FTPConnection(source) > destConnectionWrapper = FTPConnection(dest) > srcConnectionWrapper.file_exsist(src_file_path) > srcConnectionWrapper.sftp_client.getfo(src_file_path, destConnectionWrapper.sftp_client.open( > dest_file_path, 'wb')) > srcConnectionWrapper.delete_file(src_file_path) > ``` > > So now we need to open three connections to source and 2 connections to destination server if we were to do the same using 5.1.0. I know this is very specific implementation but I am trying to say that we can have all kinds of implementation which could break if we change the get_conn() in sftphook The reason why this [PR](https://github.com/apache/airflow/pull/44247) changed the implementation was that we discovered connection leaking due to the caching of the connection in the SFTPHook. Also even if the connection is closed using the close_conn method of the SFTPHook, the underlying SSHClient isn't closed and caused connections leaks. In you example above, I would then opt to get the context managed connection and call the underlying SFTPClient methods directly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
