This is an automated email from the ASF dual-hosted git repository.
adoroszlai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ozone.git
The following commit(s) were added to refs/heads/master by this push:
new 5c3aa01 HDDS-6273. Amend doc SecuringTDE.md (#3047)
5c3aa01 is described below
commit 5c3aa0185bfc43edc046274f39e32267dd6fad27
Author: Siyao Meng <[email protected]>
AuthorDate: Tue Feb 8 04:57:14 2022 -0800
HDDS-6273. Amend doc SecuringTDE.md (#3047)
---
hadoop-hdds/docs/content/security/SecuringTDE.md | 48 +++++++++++++++++-----
.../docs/content/security/SecuringTDE.zh.md | 4 +-
2 files changed, 39 insertions(+), 13 deletions(-)
diff --git a/hadoop-hdds/docs/content/security/SecuringTDE.md
b/hadoop-hdds/docs/content/security/SecuringTDE.md
index c85da0f..2a98984 100644
--- a/hadoop-hdds/docs/content/security/SecuringTDE.md
+++ b/hadoop-hdds/docs/content/security/SecuringTDE.md
@@ -58,10 +58,10 @@ To create an encrypted bucket, client need to:
* Assign the encryption key to a bucket.
```bash
- ozone sh bucket create -k encKey /vol/encryptedBucket
+ ozone sh bucket create -k encKey /vol/encryptedbucket
```
-After this command, all data written to the _encryptedBucket_ will be encrypted
+After this command, all data written to the _encryptedbucket_ will be encrypted
via the encKey and while reading the clients will talk to Key Management
Server and read the key and decrypt it. In other words, the data stored
inside Ozone is always encrypted. The fact that data is encrypted at rest
@@ -71,20 +71,47 @@ will be completely transparent to the clients and end users.
There are two ways to create an encrypted bucket that can be accessed via S3
Gateway.
-####1. Create a bucket using shell under "/s3v" volume
+#### Option 1. Create a bucket using shell under "/s3v" volume
```bash
- ozone sh bucket create -k encKey /s3v/encryptedBucket
+ ozone sh bucket create -k encKey --layout=OBJECT_STORE /s3v/encryptedbucket
```
-####2. Create a link to an encrypted bucket under "/s3v" volume
+
+#### Option 2. Create a link to an encrypted bucket under "/s3v" volume
```bash
- ozone sh bucket create -k encKey /vol/encryptedBucket
- ozone sh bucket link /vol/encryptedBucket /s3v/linkencryptedbucket
+ ozone sh bucket create -k encKey --layout=OBJECT_STORE /vol/encryptedbucket
+ ozone sh bucket link /vol/encryptedbucket /s3v/linkencryptedbucket
```
-Note: An encrypted bucket cannot be created via S3 APIs. It must be done using
Ozone shell commands as shown above.
+
+Note 1: An encrypted bucket cannot be created via S3 APIs. It must be done
using Ozone shell commands as shown above.
After creating an encrypted bucket, all the keys added to this bucket using
s3g will be encrypted.
+Note 2: `--layout=OBJECT_STORE` is specified in the above examples
+for full compatibility with S3 (which is the default value for the `--layout`
+argument, but explicitly added here to make a point).
+
+Bucket created with the `OBJECT_STORE` type will NOT be accessible via
+HCFS (ofs or o3fs) at all. And such access will be rejected. For instance:
+
+ ```bash
+ $ ozone fs -ls ofs://ozone1/s3v/encryptedbucket/
+ -ls: Bucket: encryptedbucket has layout: OBJECT_STORE, which does not
support file system semantics. Bucket Layout must be FILE_SYSTEM_OPTIMIZED or
LEGACY.
+ ```
+
+ ```bash
+ $ ozone fs -ls o3fs://encryptedbucket.s3v.ozone1/
+ 22/02/07 00:00:00 WARN fs.FileSystem: Failed to initialize fileystem
o3fs://encryptedbucket.s3v.ozone1/: java.lang.IllegalArgumentException: Bucket:
encryptedbucket has layout: OBJECT_STORE, which does not support file system
semantics. Bucket Layout must be FILE_SYSTEM_OPTIMIZED or LEGACY.
+ -ls: Bucket: encryptedbucket has layout: OBJECT_STORE, which does not
support file system semantics. Bucket Layout must be FILE_SYSTEM_OPTIMIZED or
LEGACY.
+ ```
+
+If one wants the bucket to be accessible from both S3G and HCFS (ofs and o3fs)
+at the same time, use `--layout=FILE_SYSTEM_OPTIMIZED` instead.
+
+However, in buckets with `FILE_SYSTEM_OPTIMIZED` layout, some irregular S3 key
+names may be rejected or normalized, which can be undesired.
+See [Prefix based File System Optimization]({{< relref
"../feature/PrefixFSO.md" >}}) for more information.
+
In non-secure mode, the user running the S3Gateway daemon process is the proxy
user,
while in secure mode the S3Gateway Kerberos principal
(ozone.s3g.kerberos.principal) is the proxy user.
S3Gateway proxy's all the users accessing the encrypted buckets to decrypt the
key.
@@ -111,12 +138,11 @@ The below two configurations must be added to the
kms-site.xml to allow the S3Ga
This is the host where the S3Gateway is running. Set this to '*' to
allow
requests from any hosts to be proxied.
</description>
-
</property>
-
```
-###KMS Authorization
+### KMS Authorization
+
If Ranger authorization is enabled for KMS, then decrypt key permission should
be given to
access key id user(currently access key is kerberos principal) to decrypt the
encrypted key
to read/write a key in the encrypted bucket.
diff --git a/hadoop-hdds/docs/content/security/SecuringTDE.zh.md
b/hadoop-hdds/docs/content/security/SecuringTDE.zh.md
index bda5e76..d7a2911 100644
--- a/hadoop-hdds/docs/content/security/SecuringTDE.zh.md
+++ b/hadoop-hdds/docs/content/security/SecuringTDE.zh.md
@@ -49,7 +49,7 @@ hadoop.security.key.provider.path | KMS uri. <br> 比如
kms://http@kms-host:96
* 将加密密钥分配给桶
```bash
- ozone sh bucket create -k encKey /vol/encryptedBucket
+ ozone sh bucket create -k encKey /vol/encryptedbucket
```
-这条命令执行后，所以写往 _encryptedBucket_ 的数据都会用 encKey 进行加密，当读取里面的数据时，客户端通过 KMS
获取密钥进行解密。换句话说，Ozone 中存储的数据一直是加密的，但用户和客户端对此完全无感知。
+这条命令执行后，所以写往 _encryptedbucket_ 的数据都会用 encKey 进行加密，当读取里面的数据时，客户端通过 KMS
获取密钥进行解密。换句话说，Ozone 中存储的数据一直是加密的，但用户和客户端对此完全无感知。
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
