This is an automated email from the ASF dual-hosted git repository.
stevel pushed a change to branch feature-HADOOP-18028-s3a-prefetch-branch-3.3
in repository https://gitbox.apache.org/repos/asf/hadoop.git
discard d149c68dc1c HADOOP-18416. fix ITestS3AIOStatisticsContext test failure
(#4931)
discard 2ffffc48c4c HADOOP-18377. hadoop-aws build to add a -prefetch profile
to run all tests with prefetching (#4914)
discard 6711ccd02d5 HADOOP-18466. Limit the findbugs suppression
IS2_INCONSISTENT_SYNC to S3AFileSystem field (#4926)
discard c3950769ac0 HADOOP-18455. S3A prefetching executor should be closed
(#4879)
discard e6b05ae8c54 HADOOP-18186. s3a prefetching to use
SemaphoredDelegatingExecutor for submitting work (#4796)
discard 3d362d0b3c9 HADOOP-18380. fs.s3a.prefetch.block.size to be read
through longBytesOption (#4762)
discard 5633755f4d8 HADOOP-18028. High performance S3A input stream (#4752)
add d7d36b9d2ac HADOOP-18689. Bump jettison from 1.5.3 to 1.5.4 in
/hadoop-project (#5502) (#5586)
add 0b56be3ca45 MAPREDUCE-7437. MR Fetcher class to use an AtomicInteger
to generate IDs. (#5579)
add 919c3f615b4 HADOOP-18660. Filesystem Spelling Mistake (#5475).
add 4d052a2456f HDFS-16672. Fix lease interval comparison in
BlockReportLeaseManager (#4598). Contributed by dzcxzl.
new d81676e1266 HADOOP-18028. High performance S3A input stream (#4752)
new a01427792ee HADOOP-18380. fs.s3a.prefetch.block.size to be read
through longBytesOption (#4762)
new ca52512f957 HADOOP-18186. s3a prefetching to use
SemaphoredDelegatingExecutor for submitting work (#4796)
new 267c7f9dbc3 HADOOP-18455. S3A prefetching executor should be closed
(#4879)
new 1f49861fa38 HADOOP-18466. Limit the findbugs suppression
IS2_INCONSISTENT_SYNC to S3AFileSystem field (#4926)
new 5aa200e8ada HADOOP-18377. hadoop-aws build to add a -prefetch profile
to run all tests with prefetching (#4914)
new 8102d3c2806 HADOOP-18416. fix ITestS3AIOStatisticsContext test failure
(#4931)
new 36f67f06cba HADOOP-18378. Implement lazy seek in S3A prefetching.
(#4955)
new fd25b56953e HADOOP-18531. Fix assertion failure in
ITestS3APrefetchingInputStream (#5149)
new b031fc86210 HADOOP-18189 S3APrefetchingInputStream to support status
probes when closed (#5036)
new 2c6f01c1b50 HADOOP-18351. Reduce excess logging of errors during S3A
prefetching reads (#5274)
new caf58937a0a HADOOP-18410. S3AInputStream.unbuffer() does not release
http connections -prefetch changes(#4766)
new 5a90d989724 HADOOP-18482. ITestS3APrefetchingInputStream to skip if
CSV test file unavailable (#4983)
new eb326403ac1 HADOOP-18399. S3A Prefetch - SingleFilePerBlockCache to
use LocalDirAllocator (#5054)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (d149c68dc1c)
\
N -- N -- N
refs/heads/feature-HADOOP-18028-s3a-prefetch-branch-3.3 (eb326403ac1)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 14 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
LICENSE-binary | 2 +-
.../main/java/org/apache/hadoop/fs/FileSystem.java | 4 +-
.../apache/hadoop/fs/impl/prefetch/BlockCache.java | 8 +-
.../fs/impl/prefetch/CachingBlockManager.java | 40 ++++--
.../hadoop/fs/impl/prefetch/FilePosition.java | 4 +-
.../fs/impl/prefetch/SingleFilePerBlockCache.java | 87 +++++++++---
.../site/markdown/filesystem/pathcapabilities.md | 4 +-
.../hadoop/fs/impl/prefetch/TestBlockCache.java | 11 +-
.../hadoop/fs/impl/prefetch/TestFilePosition.java | 33 ++++-
.../apache/hadoop/fs/http/server/HttpFSServer.java | 8 +-
.../lib/servlet/FileSystemReleaseFilter.java | 4 +-
.../blockmanagement/BlockReportLeaseManager.java | 2 +-
.../hadoop/mapreduce/task/reduce/Fetcher.java | 5 +-
.../src/site/markdown/manifest_committer.md | 4 +-
.../hadoop/mapred/uploader/FrameworkUploader.java | 4 +-
hadoop-project/pom.xml | 2 +-
hadoop-tools/hadoop-aws/pom.xml | 5 +
.../org/apache/hadoop/fs/s3a/S3AFileSystem.java | 27 +++-
.../org/apache/hadoop/fs/s3a/S3AInputStream.java | 1 +
.../org/apache/hadoop/fs/s3a/S3AReadOpContext.java | 2 +-
.../fs/s3a/prefetch/S3ACachingBlockManager.java | 12 +-
.../fs/s3a/prefetch/S3ACachingInputStream.java | 125 ++++++++---------
.../fs/s3a/prefetch/S3AInMemoryInputStream.java | 12 +-
.../fs/s3a/prefetch/S3APrefetchingInputStream.java | 53 +++++--
.../fs/s3a/prefetch/S3ARemoteInputStream.java | 54 +++++---
.../hadoop/fs/s3a/prefetch/S3ARemoteObject.java | 108 ++-------------
.../apache/hadoop/fs/s3a/s3guard/S3GuardTool.java | 2 +-
.../markdown/tools/hadoop-aws/delegation_tokens.md | 4 +-
.../site/markdown/tools/hadoop-aws/prefetching.md | 24 ++--
.../tools/hadoop-aws/troubleshooting_s3a.md | 4 +-
.../fs/s3a/ITestS3APrefetchingCacheFiles.java | 144 +++++++++++++++++++
.../fs/s3a/ITestS3APrefetchingInputStream.java | 152 +++++++++++++++++----
...m.java => ITestRoleDelegationInFilesystem.java} | 4 +-
...ava => ITestSessionDelegationInFilesystem.java} | 6 +-
.../fs/s3a/performance/ITestUnbufferDraining.java | 2 +
.../hadoop/fs/s3a/prefetch/S3APrefetchFakes.java | 27 +++-
.../s3a/prefetch/TestS3ACachingBlockManager.java | 34 +++--
.../fs/s3a/prefetch/TestS3ARemoteInputStream.java | 48 +++++--
38 files changed, 723 insertions(+), 349 deletions(-)
create mode 100644
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3APrefetchingCacheFiles.java
rename
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/{ITestRoleDelegationInFileystem.java
=> ITestRoleDelegationInFilesystem.java} (95%)
rename
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/{ITestSessionDelegationInFileystem.java
=> ITestSessionDelegationInFilesystem.java} (99%)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-commits-h...@hadoop.apache.org
