This is an automated email from the ASF dual-hosted git repository.
stevel pushed a change to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
from 313608e HADOOP-16232. Fix errors in the checkstyle configration xmls.
Contributed by Wanqiang Ji.
new 1b64b0f HADOOP-15407. HADOOP-15540. Support Windows Azure Storage -
Blob file system "ABFS" in Hadoop: Core Commit.
new 37426c2 HADOOP-15560. ABFS: removed dependency injection and
unnecessary dependencies. Contributed by Da Zhou.
new b537d8c HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests.
Contributed by Steve Loughran and Da Zhou.
new 5da4bb8 HADOOP-15659. Code changes for bug fix and new tests.
Contributed by Da Zhou.
new e320b64 Fixing issue due to commit 2b2399d6 after rebase onto trunk.
new 92e37a0 HADOOP-15669. ABFS: Improve HTTPS Performance. Contributed by
Vishwajeet Dusane.
new 9d47230 HADOOP-15660. ABFS: Add support for OAuth Contributed by Da
Zhou, Rajeev Bansal, and Junhua Gu.
new e727c87 HADOOP-15661. ABFS: Add support for ACL. Contributed by
Junhua Gu and Da Zhou.
new 08d3d50 HADOOP 15688. ABFS: InputStream wrapped in FSDataInputStream
twice. Contributed by Sean Mackrory.
new b59741f HADOOP-15682. ABFS: Add support for StreamCapabilities. Fix
javadoc and checkstyle. Contributed by Thomas Marquardt.
new 92f3f9c HADOOP-15692. ABFS: extensible support for custom oauth.
Contributed by Junhua Gu and Rajeev Bansal.
new 3bd089d HADOOP-15663. ABFS: Simplify configuration. Contributed by Da
Zhou.
new 12c621e HADOOP-15664. ABFS: Reduce test run time via parallelization
and grouping. Contributed by Da Zhou.
new 305e750 HADOOP-15703. ABFS - Implement client-side throttling.
Contributed by Sneha Varma and Thomas Marquardt.
new 8bb713e Fixing findbugs and license issues related to:
new a192d85 HADOOP-15728. ABFS: Add backward compatibility to handle
Unsupported Operation for storage account with no namespace feature.
new 36b2bab HADOOP-15740. ABFS: Check variable names during
initialization of AbfsClientThrottlingIntercept. Contributed by Sneha Varma.
new e52f4a1 HADOOP-15694. ABFS: Allow OAuth credentials to not be tied to
accounts. Contributed by Sean Mackrory.
new 263094f HADOOP-15753. ABFS: support path "abfs://mycluster/file/path"
Contributed by Da Zhou.
new 3599cd3 HADOOP-15757. ABFS: remove dependency on common-codec Base64.
Contributed by Da Zhou.
new 94d0de2 HADOOP-15744. AbstractContractAppendTest fails against HDFS
on HADOOP-15407 branch. Contributed by Steve Loughran.
new 26ce4e9 HADOOP-15715. ITestAzureBlobFileSystemE2E timing out with
non-scale timeout of 10 min. Contributed by Da Zhou
new 8acab6c HADOOP-15714. Tune abfs/wasb parallel and sequential test
execution. Contributed by Da Zhou.
new 1efa1b7 HADOOP-15719. Fail-fast when using OAuth over http.
Contributed by Da Zhou.
new 10e149d HADOOP-15769. ABFS: distcp tests are always skipped.
Contributed by Steve Loughran
new 920b77c HADOOP-15773. Fixing checkstyle and other issues raised by
Yetus.
new 6c7c8b8 HADOOP-15704. Mark ABFS extension package and interfaces as
LimitedPrivate/Unstable. Contributed by Steve Loughran.
new 4f3cc1d HADOOP-15778. ABFS: Fix client side throttling for read.
Contributed by Sneha Varma.
new bde00d5 HADOOP-15723. ABFS: Ranger Support. Contributed by Yuan Gao.
new ee0fa0d HADOOP-15739. ABFS: remove unused maven dependencies and add
used undeclared dependencies. Contributed by Da Zhou.
new 0c5074a HADOOP-15793. ABFS: Skip unsupported test cases when non
namespace enabled in ITestAzureBlobFileSystemAuthorization Contributed by Yuan
Gao.
new d5e4c40 HADOOP-15801. ABFS: Fixing skipUserGroupMetadata in
AzureBlobFileSystemStore. Contributed by Da Zhou
new e3033a5 HADOOP-15792. typo in
AzureBlobFileSystem.getIsNamespaceEnabeld. Contributed by Abhishek Modi.
new 1b08e8d HADOOP-15795. Make HTTPS the default protocol for ABFS.
Contributed by Da Zhou.
new 73bbada HADOOP-15809. ABFS: better exception handling when making
getAccessToken call. Contributed by Da Zhou
new 885c726 HADOOP-15825. ABFS: Enable some tests for namespace not
enabled account using OAuth. Contributed by Da Zhou.
new 8330f18 HADOOP-15823. ABFS: Stop requiring client ID and tenant ID
for MSI (Contributed by Da Zhou via Daniel Templeton)
new d370981 HADOOP-15846. ABFS: fix mask related bugs in setAcl,
modifyAclEntries and removeAclEntries.
new 9f220c4 HADOOP-15812. ABFS: Improve AbfsRestOperationException format
to ensure full msg can be displayed on console.
new 9e1d39b HADOOP-15876. Use keySet().removeAll() to remove multiple
keys from Map in AzureBlobFileSystemStore
new afa5705 HADOOP-15872. ABFS: Update to target 2018-11-09 REST version
for ADLS Gen 2.
new 9cf1fb6 HADOOP-15940. ABFS: For HNS account, avoid unnecessary get
call when doing Rename.
new 6f200a6 HADOOP-15968. ABFS: add try catch for UGI failure when
initializing ABFS.
new 06e6a67 HADOOP-15969. ABFS: getNamespaceEnabled can fail blocking
user access thru ACLs.
new 38a36d4 HADOOP-15972 ABFS: reduce list page size to to 500.
new eaaf283 HADOOP-16004. ABFS: Convert 404 error response in
AbfsInputStream and AbfsOutPutStream to FileNotFoundException.
new 175a69e HADOOP-15860. ABFS: Throw exception when directory / file
name ends with a period (.). Contributed by Shweta Yakkali.
new 4a17610 HADOOP-16040. ABFS: Bug fix for tolerateOobAppends
configuration.
new a0b99f4 HADOOP-15662. Better exception handling of DNS errors.
new 34dbac8 HADOOP-15975. ABFS: remove timeout check for DELETE and
RENAME.
new 00b990d HADOOP-16044. ABFS: Better exception handling of DNS errors
followup
new f799c46 HADOOP-16048. ABFS: Fix Date format parser.
new 0bbbb9c HADOOP-16041. Include Hadoop version in User-Agent string for
ABFS. Contributed by Shweta Yakkali.
new 80c8b77 HADOOP-15954. ABFS: Enable owner and group conversion for MSI
and login user using OAuth.
new c2d7429 HADOOP-16136. ABFS: Should only transform username to short
name
new 024a694 HADOOP-16169. ABFS: Bug fix for getPathProperties.
new 662a75d HADOOP-16182. Update abfs storage back-end with "close" flag
when application is done writing to a file.
new 18f66a0 HADOOP-16205 Backport ABFS driver from trunk to branch 2.0:
Fix build and test failures.
new 8900c7f HADOOP-16205 Backport ABFS driver from trunk to branch 2.
The 59 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitignore | 1 +
.../src/main/resources/core-default.xml | 37 +
.../src/site/markdown/filesystem/filesystem.md | 11 +-
.../hadoop/conf/TestCommonConfigurationFields.java | 3 +
.../fs/contract/AbstractContractConcatTest.java | 50 +-
.../AbstractContractGetFileStatusTest.java | 42 +-
.../fs/contract/AbstractContractMkdirTest.java | 8 +
.../hadoop/fs/contract/AbstractFSContract.java | 2 -
.../hadoop/fs/contract/ContractTestUtils.java | 19 +-
hadoop-project/pom.xml | 18 +
hadoop-tools/hadoop-azure/pom.xml | 418 +++++-
.../src/config/checkstyle-suppressions.xml | 49 +
.../hadoop/fs/azure/ClientThrottlingAnalyzer.java | 2 +-
.../java/org/apache/hadoop/fs/azurebfs/Abfs.java} | 44 +-
.../hadoop/fs/azurebfs/AbfsConfiguration.java | 638 +++++++++
.../java/org/apache/hadoop/fs/azurebfs/Abfss.java | 46 +
.../hadoop/fs/azurebfs/AzureBlobFileSystem.java | 1086 +++++++++++++++
.../fs/azurebfs/AzureBlobFileSystemStore.java | 1041 +++++++++++++++
.../fs/azurebfs/SecureAzureBlobFileSystem.java} | 37 +-
.../fs/azurebfs/constants/AbfsHttpConstants.java | 95 ++
.../fs/azurebfs/constants/ConfigurationKeys.java | 114 ++
.../constants/FileSystemConfigurations.java | 71 +
.../azurebfs/constants/FileSystemUriSchemes.java | 42 +
.../constants/HttpHeaderConfigurations.java | 63 +
.../fs/azurebfs/constants/HttpQueryParams.java | 42 +
.../hadoop/fs/azurebfs/constants/package-info.java | 22 +
.../ConfigurationValidationAnnotations.java | 104 ++
.../contracts/annotations/package-info.java | 22 +
.../diagnostics/ConfigurationValidator.java} | 33 +-
.../contracts/diagnostics/package-info.java | 22 +
.../exceptions/AbfsRestOperationException.java | 103 ++
.../exceptions/AzureBlobFileSystemException.java | 56 +
.../ConfigurationPropertyNotFoundException.java} | 32 +-
.../FileSystemOperationUnhandledException.java | 33 +
.../InvalidAbfsRestOperationException.java | 40 +
.../exceptions/InvalidAclOperationException.java | 33 +
.../InvalidConfigurationValueException.java} | 35 +-
.../InvalidFileSystemPropertyException.java | 33 +
.../exceptions/InvalidUriAuthorityException.java | 33 +
.../contracts/exceptions/InvalidUriException.java} | 31 +-
.../exceptions/KeyProviderException.java} | 40 +-
.../contracts/exceptions/TimeoutException.java | 33 +
.../exceptions/TokenAccessProviderException.java | 36 +
.../contracts/exceptions/package-info.java | 22 +
.../hadoop/fs/azurebfs/contracts/package-info.java | 22 +
.../contracts/services/AzureServiceErrorCode.java | 115 ++
.../contracts/services/ListResultEntrySchema.java | 239 ++++
.../contracts/services/ListResultSchema.java | 58 +
.../contracts/services/ReadBufferStatus.java | 29 +
.../azurebfs/contracts/services/package-info.java | 22 +
.../Base64StringConfigurationBasicValidator.java | 50 +
.../BooleanConfigurationBasicValidator.java | 50 +
.../diagnostics/ConfigurationBasicValidator.java | 67 +
.../IntegerConfigurationBasicValidator.java | 68 +
.../LongConfigurationBasicValidator.java | 63 +
.../StringConfigurationBasicValidator.java | 43 +
.../fs/azurebfs/diagnostics/package-info.java | 22 +
.../extensions/AbfsAuthorizationException.java} | 39 +-
.../fs/azurebfs/extensions/AbfsAuthorizer.java | 57 +
.../extensions/CustomDelegationTokenManager.java | 70 +
.../extensions/CustomTokenProviderAdaptee.java | 75 ++
.../fs/azurebfs/extensions/package-info.java | 32 +
.../fs/azurebfs/oauth2/AccessTokenProvider.java | 98 ++
.../fs/azurebfs/oauth2/AzureADAuthenticator.java | 346 +++++
.../hadoop/fs/azurebfs/oauth2/AzureADToken.java | 47 +
.../azurebfs/oauth2/ClientCredsTokenProvider.java | 62 +
.../oauth2/CustomTokenProviderAdapter.java | 58 +
.../fs/azurebfs/oauth2/IdentityTransformer.java | 279 ++++
.../fs/azurebfs/oauth2/MsiTokenProvider.java | 48 +
.../hadoop/fs/azurebfs/oauth2/QueryParams.java | 69 +
.../oauth2/RefreshTokenBasedTokenProvider.java | 57 +
.../azurebfs/oauth2/UserPasswordTokenProvider.java | 56 +
.../hadoop/fs/azurebfs/oauth2/package-info.java | 18 +
.../org/apache/hadoop/fs/azurebfs/package.html | 31 +
.../security/AbfsDelegationTokenIdentifier.java} | 51 +-
.../security/AbfsDelegationTokenManager.java | 88 ++
.../fs/azurebfs/security/AbfsTokenRenewer.java | 96 ++
.../hadoop/fs/azurebfs/security/package-info.java | 23 +
.../hadoop/fs/azurebfs/services/AbfsAclHelper.java | 278 ++++
.../hadoop/fs/azurebfs/services/AbfsClient.java | 589 ++++++++
.../services/AbfsClientThrottlingAnalyzer.java} | 82 +-
.../services/AbfsClientThrottlingIntercept.java | 135 ++
.../fs/azurebfs/services/AbfsHttpHeader.java | 40 +
.../fs/azurebfs/services/AbfsHttpOperation.java | 446 +++++++
.../fs/azurebfs/services/AbfsInputStream.java | 391 ++++++
.../fs/azurebfs/services/AbfsOutputStream.java | 398 ++++++
.../fs/azurebfs/services/AbfsPermission.java | 114 ++
.../fs/azurebfs/services/AbfsRestOperation.java | 208 +++
.../azurebfs/services/AbfsRestOperationType.java | 43 +
.../fs/azurebfs/services/AbfsUriQueryBuilder.java | 64 +
.../hadoop/fs/azurebfs/services/AuthType.java | 27 +
.../azurebfs/services/ExponentialRetryPolicy.java | 144 ++
.../hadoop/fs/azurebfs/services/KeyProvider.java | 43 +
.../hadoop/fs/azurebfs/services/ReadBuffer.java | 139 ++
.../fs/azurebfs/services/ReadBufferManager.java | 395 ++++++
.../fs/azurebfs/services/ReadBufferWorker.java | 72 +
.../fs/azurebfs/services/SharedKeyCredentials.java | 510 +++++++
.../services/ShellDecryptionKeyProvider.java | 71 +
.../fs/azurebfs/services/SimpleKeyProvider.java | 54 +
.../hadoop/fs/azurebfs/services/package-info.java | 22 +
.../apache/hadoop/fs/azurebfs/utils/Base64.java | 329 +++++
.../fs/azurebfs/utils/SSLSocketFactoryEx.java | 241 ++++
.../apache/hadoop/fs/azurebfs/utils/UriUtils.java | 78 ++
.../hadoop/fs/azurebfs/utils/package-info.java | 22 +
...rg.apache.hadoop.security.token.TokenIdentifier | 1 +
.../org.apache.hadoop.security.token.TokenRenewer | 1 +
.../hadoop-azure/src/site/markdown/abfs.md | 82 ++
.../src/site/markdown/testing_azure.md | 230 +++-
.../fs/azure/AzureBlobStorageTestAccount.java | 22 +-
.../ITestFileSystemOperationExceptionMessage.java | 3 +-
.../fs/azure/ITestNativeFileSystemStatistics.java | 99 ++
.../fs/azure/ITestWasbUriAndConfiguration.java | 26 -
.../fs/azure/NativeAzureFileSystemBaseTest.java | 78 +-
.../fs/azure/integration/AzureTestConstants.java | 6 +-
.../fs/azure/integration/AzureTestUtils.java | 69 +
.../fs/azure/metrics/TestRollingWindowAverage.java | 4 +-
.../fs/azurebfs/AbstractAbfsIntegrationTest.java | 345 +++++
.../hadoop/fs/azurebfs/AbstractAbfsScaleTest.java | 59 +
.../fs/azurebfs/AbstractAbfsTestWithTimeout.java | 70 +
.../apache/hadoop/fs/azurebfs/ITestAbfsClient.java | 82 ++
.../fs/azurebfs/ITestAbfsIdentityTransformer.java | 304 +++++
.../fs/azurebfs/ITestAbfsReadWriteAndSeek.java | 89 ++
.../azurebfs/ITestAbfsRestOperationException.java | 75 ++
.../azurebfs/ITestAzureBlobFileSystemAppend.java | 79 ++
.../ITestAzureBlobFileSystemAuthorization.java | 418 ++++++
.../ITestAzureBlobFileSystemBackCompat.java | 86 ++
.../fs/azurebfs/ITestAzureBlobFileSystemCopy.java | 96 ++
.../azurebfs/ITestAzureBlobFileSystemCreate.java | 107 ++
.../azurebfs/ITestAzureBlobFileSystemDelete.java | 144 ++
.../fs/azurebfs/ITestAzureBlobFileSystemE2E.java | 242 ++++
.../azurebfs/ITestAzureBlobFileSystemE2EScale.java | 120 ++
.../ITestAzureBlobFileSystemFileStatus.java | 125 ++
.../azurebfs/ITestAzureBlobFileSystemFinalize.java | 62 +
.../fs/azurebfs/ITestAzureBlobFileSystemFlush.java | 382 ++++++
.../ITestAzureBlobFileSystemInitAndCreate.java | 52 +
.../ITestAzureBlobFileSystemListStatus.java | 245 ++++
.../ITestAzureBlobFileSystemMkDir.java} | 44 +-
.../fs/azurebfs/ITestAzureBlobFileSystemOauth.java | 178 +++
.../ITestAzureBlobFileSystemPermission.java | 108 ++
.../ITestAzureBlobFileSystemRandomRead.java | 586 ++++++++
.../azurebfs/ITestAzureBlobFileSystemRename.java | 152 +++
.../ITestAzureBlobFileSystemRenameUnicode.java | 98 ++
.../fs/azurebfs/ITestAzureBlobFilesystemAcl.java | 1410 ++++++++++++++++++++
.../hadoop/fs/azurebfs/ITestClientUrlScheme.java | 101 ++
.../fs/azurebfs/ITestFileSystemInitialization.java | 77 ++
.../fs/azurebfs/ITestFileSystemProperties.java | 119 ++
.../fs/azurebfs/ITestFileSystemRegistration.java | 113 ++
.../fs/azurebfs/ITestGetNameSpaceEnabled.java | 103 ++
.../fs/azurebfs/ITestOauthOverAbfsScheme.java | 63 +
.../fs/azurebfs/ITestWasbAbfsCompatibility.java | 194 +++
.../TestAbfsConfigurationFieldsValidation.java | 179 +++
.../fs/azurebfs/TestAccountConfiguration.java | 285 ++++
.../azurebfs/constants/TestConfigurationKeys.java | 42 +
.../hadoop/fs/azurebfs/constants/package-info.java | 22 +
.../azurebfs/contract/ABFSContractTestBinding.java | 67 +
.../azurebfs/contract/AbfsFileSystemContract.java | 64 +
.../ITestAbfsFileSystemContractAppend.java | 61 +
.../ITestAbfsFileSystemContractConcat.java | 51 +
.../ITestAbfsFileSystemContractCreate.java | 52 +
.../ITestAbfsFileSystemContractDelete.java | 52 +
.../ITestAbfsFileSystemContractDistCp.java | 52 +
.../ITestAbfsFileSystemContractGetFileStatus.java | 51 +
.../contract/ITestAbfsFileSystemContractMkdir.java | 52 +
.../contract/ITestAbfsFileSystemContractOpen.java | 52 +
.../ITestAbfsFileSystemContractRename.java | 52 +
.../ITestAbfsFileSystemContractRootDirectory.java | 57 +
.../ITestAbfsFileSystemContractSecureDistCp.java | 49 +
.../contract/ITestAbfsFileSystemContractSeek.java | 52 +
.../ITestAbfsFileSystemContractSetTimes.java | 51 +
.../contract/ITestAzureBlobFileSystemBasics.java | 105 ++
.../hadoop/fs/azurebfs/contract/package-info.java | 22 +
.../diagnostics/TestConfigurationValidators.java | 121 ++
.../fs/azurebfs/diagnostics/package-info.java | 22 +
.../fs/azurebfs/extensions/MockAbfsAuthorizer.java | 90 ++
.../fs/azurebfs/extensions/package-info.java | 22 +
.../apache/hadoop/fs/azurebfs/package-info.java | 22 +
.../fs/azurebfs/services/TestAbfsClient.java | 94 ++
.../services/TestAbfsClientThrottlingAnalyzer.java | 177 +++
.../fs/azurebfs/services/TestQueryParams.java | 72 +
.../services/TestShellDecryptionKeyProvider.java | 92 ++
.../hadoop/fs/azurebfs/services/package-info.java | 22 +
.../hadoop/fs/azurebfs/utils/AbfsTestUtils.java | 85 ++
.../hadoop/fs/azurebfs/utils/AclTestHelpers.java | 119 ++
.../hadoop/fs/azurebfs/utils/Parallelized.java | 60 +
.../hadoop/fs/azurebfs/utils/TestUriUtils.java | 48 +
.../hadoop/fs/azurebfs/utils/package-info.java | 22 +
.../hadoop-azure/src/test/resources/abfs.xml | 64 +
.../hadoop-azure/src/test/resources/azure-test.xml | 56 +-
.../src/test/resources/log4j.properties | 35 +
189 files changed, 21605 insertions(+), 462 deletions(-)
create mode 100644
hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml
copy
hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java
=> main/java/org/apache/hadoop/fs/azurebfs/Abfs.java} (50%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/Abfss.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java
copy
hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java
=> main/java/org/apache/hadoop/fs/azurebfs/SecureAzureBlobFileSystem.java}
(50%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/AbfsHttpConstants.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/ConfigurationKeys.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemConfigurations.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemUriSchemes.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/HttpHeaderConfigurations.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/HttpQueryParams.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/annotations/ConfigurationValidationAnnotations.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/annotations/package-info.java
copy
hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java
=>
main/java/org/apache/hadoop/fs/azurebfs/contracts/diagnostics/ConfigurationValidator.java}
(50%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/diagnostics/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/AbfsRestOperationException.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/AzureBlobFileSystemException.java
copy
hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java
=>
main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/ConfigurationPropertyNotFoundException.java}
(50%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/FileSystemOperationUnhandledException.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidAbfsRestOperationException.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidAclOperationException.java
copy
hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java
=>
main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidConfigurationValueException.java}
(50%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidFileSystemPropertyException.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidUriAuthorityException.java
copy
hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java
=>
main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidUriException.java}
(50%)
copy
hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java
=>
main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/KeyProviderException.java}
(50%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/TimeoutException.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/TokenAccessProviderException.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/AzureServiceErrorCode.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/ListResultEntrySchema.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/ListResultSchema.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/ReadBufferStatus.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/Base64StringConfigurationBasicValidator.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/BooleanConfigurationBasicValidator.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/ConfigurationBasicValidator.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/IntegerConfigurationBasicValidator.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/LongConfigurationBasicValidator.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/StringConfigurationBasicValidator.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/package-info.java
copy
hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java
=>
main/java/org/apache/hadoop/fs/azurebfs/extensions/AbfsAuthorizationException.java}
(50%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/AbfsAuthorizer.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/CustomDelegationTokenManager.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/CustomTokenProviderAdaptee.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/AccessTokenProvider.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/AzureADAuthenticator.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/AzureADToken.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/ClientCredsTokenProvider.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/CustomTokenProviderAdapter.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/IdentityTransformer.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/MsiTokenProvider.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/QueryParams.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/RefreshTokenBasedTokenProvider.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/UserPasswordTokenProvider.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/package.html
copy
hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java
=>
main/java/org/apache/hadoop/fs/azurebfs/security/AbfsDelegationTokenIdentifier.java}
(50%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/security/AbfsDelegationTokenManager.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/security/AbfsTokenRenewer.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/security/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsAclHelper.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/{azure/ClientThrottlingAnalyzer.java
=> azurebfs/services/AbfsClientThrottlingAnalyzer.java} (81%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingIntercept.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpHeader.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsPermission.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperationType.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsUriQueryBuilder.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AuthType.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ExponentialRetryPolicy.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/KeyProvider.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ReadBuffer.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ReadBufferManager.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ReadBufferWorker.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/SharedKeyCredentials.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ShellDecryptionKeyProvider.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/SimpleKeyProvider.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/Base64.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/SSLSocketFactoryEx.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/site/markdown/abfs.md
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/ITestNativeFileSystemStatistics.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/AbstractAbfsIntegrationTest.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/AbstractAbfsScaleTest.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/AbstractAbfsTestWithTimeout.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsClient.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsIdentityTransformer.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsReadWriteAndSeek.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsRestOperationException.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAppend.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAuthorization.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemBackCompat.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemCopy.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemCreate.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemDelete.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemE2E.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemE2EScale.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemFileStatus.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemFinalize.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemFlush.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemInitAndCreate.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemListStatus.java
copy
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/{azure/metrics/TestRollingWindowAverage.java
=> azurebfs/ITestAzureBlobFileSystemMkDir.java} (50%)
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemOauth.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemPermission.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemRandomRead.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemRename.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemRenameUnicode.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFilesystemAcl.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestClientUrlScheme.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestFileSystemInitialization.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestFileSystemProperties.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestFileSystemRegistration.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestGetNameSpaceEnabled.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestOauthOverAbfsScheme.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestWasbAbfsCompatibility.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAbfsConfigurationFieldsValidation.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAccountConfiguration.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/constants/TestConfigurationKeys.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/constants/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ABFSContractTestBinding.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/AbfsFileSystemContract.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractAppend.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractConcat.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractCreate.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractDelete.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractDistCp.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractGetFileStatus.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractMkdir.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractOpen.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractRename.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractRootDirectory.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractSecureDistCp.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractSeek.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractSetTimes.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAzureBlobFileSystemBasics.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/diagnostics/TestConfigurationValidators.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/diagnostics/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/MockAbfsAuthorizer.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClient.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClientThrottlingAnalyzer.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestQueryParams.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestShellDecryptionKeyProvider.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/package-info.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/AbfsTestUtils.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/AclTestHelpers.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/Parallelized.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/TestUriUtils.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/resources/abfs.xml
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
