Andrei Iatsuk created HADOOP-20079:
--------------------------------------
Summary: SFTPInputStream reads one byte per call: override
read(byte[], int, int)
Key: HADOOP-20079
URL: https://issues.apache.org/jira/browse/HADOOP-20079
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 3.4.1, 3.5.0, 3.3.6, 3.3.1
Reporter: Andrei Iatsuk
{{SFTPInputStream}} overrides only {{read()}}. It does not override
{{read(byte[], int, int)}}, so the default implementation from
{{java.io.InputStream}} is used, and it calls {{read()}} for every byte.
Because of this, every bulk read from an {{sftp://}} path goes byte by byte:
{{IOUtils.copyBytes}}, {{readFully}}, positioned reads,
{{BufferedInputStream}}. For each byte there is a synchronized call, the checks
for closed stream, EOF and seek, a one-byte read in JSch and a statistics
update. The read is CPU-bound and very slow, about 6 MB/s on localhost. A
bigger buffer on the caller side does not help, because all wrappers end in the
same default method.
This is a regression from HADOOP-14566. In branch-3.2 {{SFTPInputStream}} had
{{read(byte[], int, int)}}, which called the JSch stream directly. HADOOP-14566
rewrote the class to support seek, and this method was removed. So all releases
since 3.3.1 have the problem, and trunk too.
Fix: add {{read(byte[], int, int)}} again. It passes the whole buffer to the
JSch stream and updates {{pos}}, {{nextPos}} and the statistics in the same way
as {{read()}}.
Benchmark: sequential read of a 1 GiB file from a local OpenSSH sftp server,
with {{FileSystem.open()}} and {{IOUtils.copyBytes()}} with a 64 KB buffer:
||Build||Run 1||Run 2||Run 3||Mean||
|trunk (28c018cdafe)|6.0 MB/s|6.0 MB/s|6.0 MB/s|6.0 MB/s|
|with fix|530.5 MB/s|554.7 MB/s|539.7 MB/s|541.6 MB/s|
Corretto 21.0.13, 16 vCPU. Runs were interleaved: before, after, before, after,
...
I will open a PR with the fix and a unit test.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
