Hello Alexey Serbin, Kudu Jenkins, Andrew Wong, Bankim Bhavsar,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17974
to look at the new patch set (#3).
Change subject: [encryption] KUDU-3331 Encrypt file system
......................................................................
[encryption] KUDU-3331 Encrypt file system
de02a34 introduced encryption support to Env in a self-contained way,
but it's not used across Kudu.
This commit integrates this encryption support into the project and
modifies several test suites to also run tests with encryption enabled.
I also changed the way the "encrypted" flag in *FileOptions works:
previously, the default was false, as files were not encrypted by
default. As this commit introduces the "--encrypt_data_at_rest" flag,
which is false by default, the encrypted flag now means that a file is
encrypted or not if data at rest encryption is enabled. The default is
now true, which results in less code change and not having to complicate
FileCache.
When encryption is enabled, the following files are encrypted:
- WAL and blocks as they contain user data, which is considered
sensitive.
- LBM block metadata as FileCache is used to access them, so by
encrypting these files, we can keep FileCache simpler contains block
location
Instance and block manager instance files don't need to be encrypted,
and neither do consensus and tablet metadata files.
As most PBC files are non-encrypted, PBUtil disables encryption by
default. This leads to an edge-case in the PBC CLI tool, which can dump
and edit PBC files. This needs to be able to recognize whether the file
is encrypted. As encryption headers are not yet implemented, I added a
hack which checks the file name and treats the file as encrypted if it
ends with ".metadata" (LBM block metadata extension) and non-encrypted
otherwise.
Change-Id: I909d0c4af0c1fca0d14c99a6627842dbe2ed7524
---
M src/kudu/consensus/consensus_meta-test.cc
M src/kudu/fs/log_block_manager-test.cc
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/integration-tests/security-itest.cc
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/mini-cluster/external_mini_cluster.h
M src/kudu/postgres/mini_postgres.cc
M src/kudu/ranger/ranger_client.cc
M src/kudu/security/test/mini_kdc.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_pbc.cc
M src/kudu/util/env.cc
M src/kudu/util/env.h
M src/kudu/util/env_posix.cc
M src/kudu/util/file_cache-test.cc
M src/kudu/util/pb_util-test.cc
M src/kudu/util/pb_util.cc
M src/kudu/util/rolling_log.cc
M src/kudu/util/yamlreader-test.cc
19 files changed, 272 insertions(+), 93 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/74/17974/3
--
To view, visit http://gerrit.cloudera.org:8080/17974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I909d0c4af0c1fca0d14c99a6627842dbe2ed7524
Gerrit-Change-Number: 17974
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Bukor <[email protected]>
Gerrit-Reviewer: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Andrew Wong <[email protected]>
Gerrit-Reviewer: Bankim Bhavsar <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
