[
https://issues.apache.org/jira/browse/HADOOP-17912?focusedWorklogId=788876&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-788876
]
ASF GitHub Bot logged work on HADOOP-17912:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 08/Jul/22 08:12
Start Date: 08/Jul/22 08:12
Worklog Time Spent: 10m
Work Description: pranavsaxena-microsoft commented on PR #3440:
URL: https://github.com/apache/hadoop/pull/3440#issuecomment-1178690768
Following commits are added to make the PR have commits of trunk which were
added after the branch HADOOP-17912 was cut from the apache/hadoop:trunk.:
1.
[16973d4](https://github.com/apache/hadoop/pull/3440/commits/16973d461d79a1516f8a0683cb4753c490e154d2)
2.
https://github.com/apache/hadoop/pull/3440/commits/c27c28dd3f51159c7a4e72e33fd7805c48c47f55
3.
https://github.com/apache/hadoop/pull/3440/commits/0e700ca21251a9f26252346442a9d0a91d6a6c42
4.
https://github.com/apache/hadoop/pull/3440/commits/4164ee2e1535d85f7a39260fbe4d70b0fa3697b9
##Testing:
This was tested by running integrated tests with the Azure-storage accounts
(US EAST).
Non-HNS Endpoint: pranavsaxenanonhns.blob.core.windows.net
HNS Endpoint: pranavsaxenahns.dfs.core.windows.net
```
:::: AGGREGATED TEST RESULT ::::
HNS-OAuth
========================
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399
Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 107, Failures: 1, Errors: 0, Skipped: 2
[INFO] Results:
[INFO]
[WARNING] Tests run: 574, Failures: 0, Errors: 0, Skipped: 26
[INFO] Results:
[INFO]
[WARNING] Tests run: 332, Failures: 0, Errors: 0, Skipped: 41
HNS-SharedKey
========================
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399
Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 107, Failures: 1, Errors: 0, Skipped: 2
[INFO] Results:
[INFO]
[WARNING] Tests run: 574, Failures: 0, Errors: 0, Skipped: 26
[INFO] Results:
[INFO]
[WARNING] Tests run: 332, Failures: 0, Errors: 0, Skipped: 41
NonHNS-SharedKey
========================
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399
Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 107, Failures: 1, Errors: 0, Skipped: 2
[INFO] Results:
[INFO]
[WARNING] Tests run: 559, Failures: 0, Errors: 0, Skipped: 268
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
ITestAbfsRenameStageFailure>TestRenameStageFailure.testResilienceAsExpected:126
[resilient commit support] expected:<[tru]e> but was:<[fals]e>
[ERROR]
ITestAbfsTerasort.test_110_teragen:244->executeStage:211->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89
teragen(1000,
abfs://[email protected]/ITestAbfsTerasort/sortin)
failed expected:<0> but was:<1>
[ERROR] Errors:
[ERROR] ITestAbfsJobThroughManifestCommitter.test_0420_validateJob »
OutputValidation ...
[ERROR] ITestAbfsManifestCommitProtocol.testCommitLifecycle »
OutputValidation `abfs:/...
[ERROR] ITestAbfsManifestCommitProtocol.testCommitterWithDuplicatedCommit
» OutputValidation
[ERROR] ITestAbfsManifestCommitProtocol.testConcurrentCommitTaskWithSubDir
» OutputValidation
[ERROR] ITestAbfsManifestCommitProtocol.testMapFileOutputCommitter »
OutputValidation ...
[ERROR] ITestAbfsManifestCommitProtocol.testOutputFormatIntegration »
OutputValidation
[ERROR] ITestAbfsManifestCommitProtocol.testParallelJobsToAdjacentPaths »
OutputValidation
[ERROR] ITestAbfsManifestCommitProtocol.testTwoTaskAttemptsCommit »
OutputValidation `...
[INFO]
[ERROR] Tests run: 332, Failures: 2, Errors: 8, Skipped: 46
AppendBlob-HNS-OAuth
========================
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399
Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 107, Failures: 1, Errors: 0, Skipped: 2
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
ITestAzureBlobFileSystemMainOperation>FSMainOperationsBaseTest.testRenameDirectoryAsNonExistentDirectory:984->FSMainOperationsBaseTest.doTestRenameDirectoryAsNonExistentDirectory:1004
Nested file2 exists: THIS SUCCEEDED WHEN RAN INDIVIDUALLY
[INFO]
[ERROR] Tests run: 574, Failures: 1, Errors: 0, Skipped: 26
[INFO] Results:
[INFO]
[WARNING] Tests run: 332, Failures: 0, Errors: 0, Skipped: 41
Time taken: 62 mins 42 secs.
```
Issue Time Tracking
-------------------
Worklog Id: (was: 788876)
Time Spent: 50m (was: 40m)
> ABFS: Support for Encryption Context
> ------------------------------------
>
> Key: HADOOP-17912
> URL: https://issues.apache.org/jira/browse/HADOOP-17912
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.3.1
> Reporter: Sumangala Patki
> Assignee: Sumangala Patki
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Support for customer-provided encryption keys at the file level, superceding
> the global (account-level) key use in HADOOP-17536.
> ABFS driver will support an "EncryptionContext" plugin for retrieving
> encryption information, the implementation for which should be provided by
> the client. The keys/context retrieved will be sent via request headers to
> the server, which will store the encryption context. Subsequent REST calls to
> server that access data/user metadata of the file will require fetching the
> encryption context through a GetFileProperties call and retrieving the key
> from the custom provider, before sending the request.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
