Hi Nikolay, Has anybody started looking into your request? As I understand the plan is to get TDE Phase 1 released in 2.7, right? https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89067473
Could you also list what needs to be done at Phase 2 and how much time it might take. -- Denis On Thu, Aug 9, 2018 at 8:48 AM Nikolay Izhikov <nizhi...@apache.org> wrote: > Hello, Igniters. > > I want to share with you TDE implementation details. > I think it can simplify review and acception of TDE implementation. > This mail is copy of wiki page [1]. > > Please, share your thoughts. > > TDE is ready for review [2]. > Please, let me know, who is able to make final review. > > This document describes some internal details of TDE.Phase 1 > implementation [3]. > I suggest that reader familiar with the general design described in IEP-18 > [4]. > > > Cache group key management and node join enhancements: > > 1. GridEncryptionManager encapsulates all logic related to key > management: > a. All group encryption keys are stored in MetaStore. > > b. Joining node sends to cluster: > * Master key digest. > * All group keys saved locally (Map<Integer, > byte[]>). Keys send over a network in encrypted form. > > c. Coordinator on new node join: > * Compares new node master key digest with the > local master key digest. > If differs then new node join is rejected. > > * Compares local and received group keys. > If some key differs then new node join is > rejected. > > d. All server nodes: > * If some of received keys are new then they save > locally. > > 2. Dynamic cache creation: > a. On server node - Encryption key is generated and added > to DynamicCacheChangeRequest. > > b. On client node: > * Prior to creation of DynamicCacheChangeRequest > we have to get new encryption key from some server node. > * New request added to solve this - > GenerateEncryptionKeyRequest. > > > WAL Record encryption: > > > 1. New type of WAL record created – EncryptedRecord. > > 2. EncryptedRecord is a container that stores some > WalRecordCacheGroupAware in encrypted form inside. > > 3. Write: > Each record for an encrypted group that implements > WalRecordCacheGroupAware written to WAL in encrypted form. > Instead of that record we write EncryptedRecord with plain record > inside(PageSnapshot, PageDeltaRecord, etc). > > 4. Read: There are 3 different cases on EncryptedRecord read: > a. WAL restore – we read EncryptedRecord, decrypt internal > record and return it. > > b. Offline WAL iteration(StandaloneWalRecordsIterator) - > EncryptionSpi is null so wecan’t decrypt EncryptedRecord and just return it > from an iterator. > > c. Meta storage restore process – On node start we restore > MetaStore. > When we do it no encryption keys are available, because > they are stored in MetaStore. > So we can’t decrypt EncryptedRecord and just return it > from an iterator. > We don't need decrypted record on this step to operate > properly. > > > Page encryption: > > > 1. We have to write on disk pages aligned on 2^n (2kb, 4kb, etc) > for gain maximum perfrormance. > > 2. There is a 16 byte overhead for and AES CBC mode. > > 3. So we have to preserve 16 bytes in page in memory to get > encrypted page size equal to 2^n when written it to disk. > > 4. PageIO has many methods with pageSize parameter. > So for encrypted cache groups page size is calculated as > cfg.getPageSize() - 16 byte. > > > [1] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89067473 > [2] https://github.com/apache/ignite/pull/4167 > [3] https://issues.apache.org/jira/browse/IGNITE-8485 > [4] > https://cwiki.apache.org/confluence/display/IGNITE/IEP-18%3A+Transparent+Data+Encryption >
