Hi Arun, Thanks for your patience. I have a related question In my application i need to encrypt/decrypt files from the map reduce phase and i need to support key rotation. Can i access the KMS from the map/reduce phase to retrieve the key material from the key alias which i retrieve from the FileEncryptionInfo class stored in the extended attribute of the file(for decryption)? Any pointers to how i can plug in various key providers such as java keystore would be appreciated. Is the toString method used to store the FileEncryptionInfo into the extended attribute if so how is the FileEncryptionInfo object retrieved back from the String. Thanks again for your help. Sitaraman
On Sun, Jun 14, 2015 at 12:53 PM, Arun Suresh <asur...@apache.org> wrote: > Apologize if I wasn't clear > > > Is the EZ key version same as an alias for the key? > yup > > > the EDEK along with the EZ key version is stored in the FIleInfo > FileInfo contains both EDEK and EZ key version. The FileInfo (you can look > at the *org.apache.hadoop.fs.FileEncryptionInfo* class for more info) > object is stored as the value of the extended attribute of that file. > > > How is the KeyMaterial derived from the KeyAlias and where is the > mapping between > the two stored? Is it in the KMS? > Yup. KMS extends the *org.apache.hadoop.crypto.key.KeyProvider* class. You > can take a look at it or a concrete implementation such as > JavaKeyStoreProvider for more information. > > Also, you should probably direct questions related to HDFS encryption to > hdfs-dev@hadoop.apache.org > > Cheers > -Arun > > > > On Sun, Jun 14, 2015 at 12:11 AM, Sitaraman Vilayannur < > vrsitaramanietfli...@gmail.com> wrote: > > > Hi Arun, > > Thanks for your response. > > Could you explain this a bit further for me.... > > Is the EZ key version same as an alias for the key? > > The EDEK is stored in the extended attributes of the file and the EZkey > > Version is stored > > in the FileInfo why is the EZKey Version not stored in the extended > > attributes too. > > Where is the FileInfo object persisted? Is it in the NameNode? > > How is the KeyMaterial derived from the KeyAlias and where is the mapping > > between the two stored? Is it in the KMS? > > Thanks much for your help in this. > > Sitaraman > > > > On Sun, Jun 14, 2015 at 12:14 PM, Arun Suresh <asur...@cloudera.com> > > wrote: > > > > > Hello Sitaraman, > > > > > > It is the EZ key "version" that is used to generate the EDEK (and which > > is > > > ultimately stored in the encrypted file's extended attributes > > > '*raw.hdfs.crypto.encryption.info > > > <http://raw.hdfs.crypto.encryption.info>*'), not really the the EZ key > > > itself (which is stored in the directory's extended attribute ‘ > > > *raw.hdfs.crypto.encryption.zone*’). > > > > > > Essentially, each file in a directory has a unique EDEK.. and an EDEK > is > > is > > > generated with the current version of the directory EZ key. The EDEK > > along > > > with the EZ key version is stored in the FIleInfo. While decrypting, > both > > > these are passed on to the KMS which provides the client with the DEK > > which > > > can be used to decrypt the file. > > > > > > Hope this clarifies things. > > > > > > Cheers > > > -Arun > > > > > > On Sat, Jun 13, 2015 at 9:51 PM, Sitaraman Vilayannur < > > > vrsitaramanietfli...@gmail.com> wrote: > > > > > > > HDFSDataatRestEncryption.pdf says the following about key > > > rotation..(please > > > > see appended below at the end of the mail) > > > > If the existing files do not have their EDEKs reencrypted using the > new > > > > ezkeyid, how would the existing files be decrypted? That is where is > > the > > > > mapping between files and its EZKey (for after key rotation different > > > files > > > > have different EZKeys)ids stored and how is it retrieved? > > > > Thanks > > > > Sitaraman > > > > > > > > Key Rotation > > > > When the administrator causes a key rotation of the EZkey > > > > in the KMS, the encryption zone’s EZkey > > > > (stored in the encryption zone directory’s > > > raw.hdfs.crypto.encryption.zone > > > > extended attribute) gets the new keyid and version (only the version > > > > changes). Any new files > > > > created in the encryption zone have their DEKs encrypted using the > new > > > key > > > > version. Existing > > > > files do not have their EDEKs reencrypted using the new ezkeyid/ > > > > version, but this will be considered as a future enhancement. Note > > that a > > > > key rotation only needs to causes a reencryption of the DEK, not a > > > > reencryption of the underlying file. > > > > > > > > > >
