Hello Alexey Serbin, Kudu Jenkins, Andrew Wong, Bankim Bhavsar, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17974 to look at the new patch set (#4). Change subject: [encryption] KUDU-3331 Encrypt file system ...................................................................... [encryption] KUDU-3331 Encrypt file system de02a34 introduced encryption support to Env in a self-contained way, but it's not used across Kudu. This commit integrates this encryption support into the project and modifies several test suites to also run tests with encryption enabled. I also changed the way the "encrypted" flag in *FileOptions works: previously, the default was false, as files were not encrypted by default. As this commit introduces the "--encrypt_data_at_rest" flag, which is false by default, the encrypted flag now means that a file is encrypted or not if data at rest encryption is enabled. The default is now true, which results in less code change and not having to complicate FileCache. When encryption is enabled, the following files are encrypted: - WAL and blocks as they contain user data, which is considered sensitive. - LBM block metadata as FileCache is used to access them, so by encrypting these files, we can keep FileCache simpler as it doesn't have to be able to handle both encrypted and non-encrypted files. Instance and block manager instance files don't need to be encrypted, and neither do consensus and tablet metadata files. As most PBC files are non-encrypted, PBUtil disables encryption by default. This leads to an edge-case in the PBC CLI tool, which can dump and edit PBC files. This needs to be able to recognize whether the file is encrypted. As encryption headers are not yet implemented, I added a hack which checks the file name and treats the file as encrypted if it ends with ".metadata" (LBM block metadata extension) and non-encrypted otherwise. Change-Id: I909d0c4af0c1fca0d14c99a6627842dbe2ed7524 --- M src/kudu/consensus/consensus_meta-test.cc M src/kudu/fs/log_block_manager-test.cc M src/kudu/integration-tests/raft_consensus-itest.cc M src/kudu/integration-tests/security-itest.cc M src/kudu/mini-cluster/external_mini_cluster.cc M src/kudu/mini-cluster/external_mini_cluster.h M src/kudu/postgres/mini_postgres.cc M src/kudu/ranger/ranger_client.cc M src/kudu/security/test/mini_kdc.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_pbc.cc M src/kudu/util/env-test.cc M src/kudu/util/env.cc M src/kudu/util/env.h M src/kudu/util/env_posix.cc M src/kudu/util/file_cache-test.cc M src/kudu/util/pb_util-test.cc M src/kudu/util/pb_util.cc M src/kudu/util/rolling_log.cc M src/kudu/util/yamlreader-test.cc 20 files changed, 282 insertions(+), 103 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/74/17974/4 -- To view, visit http://gerrit.cloudera.org:8080/17974 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I909d0c4af0c1fca0d14c99a6627842dbe2ed7524 Gerrit-Change-Number: 17974 Gerrit-PatchSet: 4 Gerrit-Owner: Attila Bukor <abu...@apache.org> Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com> Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com> Gerrit-Reviewer: Attila Bukor <abu...@apache.org> Gerrit-Reviewer: Bankim Bhavsar <ban...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins (120)