[
https://issues.apache.org/jira/browse/HDFS-12325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135633#comment-16135633
]
Hudson commented on HDFS-12325:
-------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12219 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/12219/])
HDFS-12325. SFTPFileSystem operations should restore cwd. Contributed by (arp:
rev 736ceab2f58fb9ab5907c5b5110bd44384038e6b)
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/sftp/SFTPFileSystem.java
> SFTPFileSystem operations should restore cwd
> --------------------------------------------
>
> Key: HDFS-12325
> URL: https://issues.apache.org/jira/browse/HDFS-12325
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Namit Maheshwari
> Assignee: Chen Liang
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: HDFS-12325.001.patch
>
>
> We've seen a case where writing to {{SFTPFileSystem}} led to unexpected
> behaviour:
> Given a directory ./data with more than one files in it, the steps it took to
> get this error was simply:
> {code}
> hdfs dfs -fs sftp://x.y.z -mkdir dir0
> hdfs dfs -fs sftp://x.y.z -copyFromLocal data dir0
> hdfs dfs -fs sftp://x.y.z -ls -R dir0
> {code}
> But not all files show up as in the ls output, in fact more often just one
> single file shows up in that path...
> Digging deeper, we found that rename, mkdirs and create operations in
> {{SFTPFileSystem}} are changing the current working directory during it's
> execution. For example in create there are:
> {code}
> client.cd(parent.toUri().getPath());
> os = client.put(f.getName());
> {code}
> The issue here is {{SFTPConnectionPool}} is caching SFTP sessions (in
> {{idleConnections}}), which contains their current working directory. So
> after these operations, the sessions will be put back to cache with a changed
> working directory. This accumulates in each call and ends up causing
> unexpected weird behaviour. Basically this error happens when processing
> multiple file system objects in one operation, and relative path is being
> used.
> The fix here is to restore the current working directory of the SFTP sessions.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
