Hi Sitaraman, A key name can have multiple versions. When you roll a key (via its name), a new version is created. When you fetch a key via name, you get the current version. You can also explicitly fetch a particular key version.
I think what you term a "key alias" is the key name. Regarding FEInfo conversion between a PB and xattr, see these lines: https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java#L2993 https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java#L1174 Hope this helps, Andrew On Sun, Jun 14, 2015 at 5:48 AM, Sitaraman Vilayannur < vrsitaramanietfli...@gmail.com> wrote: > Hi Arun, > FileEncryptionInfo has both a getKeyName and a getKeyVersionName. What > distinguishes the concept of keyname and key version. > It appears to me that the keyname is closer to key alias than a key > version. What is key version? Thanks much. > Sitaraman > > On Sun, Jun 14, 2015 at 2:07 PM, Sitaraman Vilayannur < > vrsitaramanietfli...@gmail.com> wrote: > > > Hi Arun, > > Thanks for your patience. I have a related question In my application i > > need to encrypt/decrypt files > > from the map reduce phase and i need to support key rotation. Can i > > access the KMS from the map/reduce > > phase to retrieve the key material from the key alias which i retrieve > > from the FileEncryptionInfo class stored in the > > extended attribute of the file(for decryption)? > > Any pointers to how i can plug in various key providers such as java > > keystore would be appreciated. > > Is the toString method used to store the FileEncryptionInfo into the > > extended attribute if so how is the > > FileEncryptionInfo object retrieved back from the String. > > Thanks again for your help. > > Sitaraman > > > > On Sun, Jun 14, 2015 at 12:53 PM, Arun Suresh <asur...@apache.org> > wrote: > > > >> Apologize if I wasn't clear > >> > >> > Is the EZ key version same as an alias for the key? > >> yup > >> > >> > the EDEK along with the EZ key version is stored in the FIleInfo > >> FileInfo contains both EDEK and EZ key version. The FileInfo (you can > look > >> at the *org.apache.hadoop.fs.FileEncryptionInfo* class for more info) > >> object is stored as the value of the extended attribute of that file. > >> > >> > How is the KeyMaterial derived from the KeyAlias and where is the > >> mapping between > >> the two stored? Is it in the KMS? > >> Yup. KMS extends the *org.apache.hadoop.crypto.key.KeyProvider* class. > You > >> can take a look at it or a concrete implementation such as > >> JavaKeyStoreProvider for more information. > >> > >> Also, you should probably direct questions related to HDFS encryption to > >> hdfs-dev@hadoop.apache.org > >> > >> Cheers > >> -Arun > >> > >> > >> > >> On Sun, Jun 14, 2015 at 12:11 AM, Sitaraman Vilayannur < > >> vrsitaramanietfli...@gmail.com> wrote: > >> > >> > Hi Arun, > >> > Thanks for your response. > >> > Could you explain this a bit further for me.... > >> > Is the EZ key version same as an alias for the key? > >> > The EDEK is stored in the extended attributes of the file and the > EZkey > >> > Version is stored > >> > in the FileInfo why is the EZKey Version not stored in the extended > >> > attributes too. > >> > Where is the FileInfo object persisted? Is it in the NameNode? > >> > How is the KeyMaterial derived from the KeyAlias and where is the > >> mapping > >> > between the two stored? Is it in the KMS? > >> > Thanks much for your help in this. > >> > Sitaraman > >> > > >> > On Sun, Jun 14, 2015 at 12:14 PM, Arun Suresh <asur...@cloudera.com> > >> > wrote: > >> > > >> > > Hello Sitaraman, > >> > > > >> > > It is the EZ key "version" that is used to generate the EDEK (and > >> which > >> > is > >> > > ultimately stored in the encrypted file's extended attributes > >> > > '*raw.hdfs.crypto.encryption.info > >> > > <http://raw.hdfs.crypto.encryption.info>*'), not really the the EZ > >> key > >> > > itself (which is stored in the directory's extended attribute ‘ > >> > > *raw.hdfs.crypto.encryption.zone*’). > >> > > > >> > > Essentially, each file in a directory has a unique EDEK.. and an > EDEK > >> is > >> > is > >> > > generated with the current version of the directory EZ key. The EDEK > >> > along > >> > > with the EZ key version is stored in the FIleInfo. While decrypting, > >> both > >> > > these are passed on to the KMS which provides the client with the > DEK > >> > which > >> > > can be used to decrypt the file. > >> > > > >> > > Hope this clarifies things. > >> > > > >> > > Cheers > >> > > -Arun > >> > > > >> > > On Sat, Jun 13, 2015 at 9:51 PM, Sitaraman Vilayannur < > >> > > vrsitaramanietfli...@gmail.com> wrote: > >> > > > >> > > > HDFSDataatRestEncryption.pdf says the following about key > >> > > rotation..(please > >> > > > see appended below at the end of the mail) > >> > > > If the existing files do not have their EDEKs reencrypted using > the > >> new > >> > > > ezkeyid, how would the existing files be decrypted? That is where > is > >> > the > >> > > > mapping between files and its EZKey (for after key rotation > >> different > >> > > files > >> > > > have different EZKeys)ids stored and how is it retrieved? > >> > > > Thanks > >> > > > Sitaraman > >> > > > > >> > > > Key Rotation > >> > > > When the administrator causes a key rotation of the EZkey > >> > > > in the KMS, the encryption zone’s EZkey > >> > > > (stored in the encryption zone directory’s > >> > > raw.hdfs.crypto.encryption.zone > >> > > > extended attribute) gets the new keyid and version (only the > version > >> > > > changes). Any new files > >> > > > created in the encryption zone have their DEKs encrypted using the > >> new > >> > > key > >> > > > version. Existing > >> > > > files do not have their EDEKs reencrypted using the new ezkeyid/ > >> > > > version, but this will be considered as a future enhancement. Note > >> > that a > >> > > > key rotation only needs to causes a reencryption of the DEK, not a > >> > > > reencryption of the underlying file. > >> > > > > >> > > > >> > > >> > > > > >
