All, A few comments on the topics raised below.
1) All nodes that mount an encrypted file system, and also the nodes with management roles on the file system will need access to the keys have the proper setup (RKM.conf, etc). Edward is correct that there was some change in behavior, introduced in 4.2.1 . Before the change, a mount would fail unless RKM.conf is present on the node. In addition, once a policy with encryption rules was applied, nodes without the proper encryption setup would unmount the file system. With the change, the error gets delayed to when encrypted files are accessed. The change in behavior was introduced based on feedback that unmounting the file system in that case was too drastic in that scenario. >> So on a node which I don't want to have access to any encrypted files, do I just need to have an empty RKM.conf file? All nodes which mount an encrypted file system should have proper setup for encryption, even for a node from where only unencrypted files are being accessed. 2) >> Encrypted file systems do not store data in large MD blocks. Makes sense. This means large MD blocks aren't as useful as they are in unencrypted FS, if you are using this. Correct. Data is not stored in the inode for encrypted files. On the other hand, since encryption metadata is stored as an extended attribute in the inode, 4K inodes are still recommended -- especially in cases where a more complicated encryption policy is used. 3) >> Having at least one backup SKLM server is a good idea. "kmipServerUri[N+1]" in the conf. While the documentation claims the FS can continue operation once it caches the MEK if an SKLM server goes away, in operation this does NOT work as you may expect. Your users still need access to the FEKs for the files your clients work on. Logs will fill with Key <key> could not be fetched. errors. Using a backup key server is strongly recommended. While it's true that the files may still be accessed for a while if the key server becomes unreachable, this was not something to be counted on. First because keys (MEK) may expire at any time, requiring the key to be retrieved from the key server again. Second because a file may require a key may be needed that has not been cached before. 4) >> Secondly ... (and maybe I'm misunderstanding the docs here) For the Policy https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.2/com.ibm.spectru m.scale.v4r22.doc/bl1adv_encryptionpolicyrules.htm KEYS ('Keyname'[, 'Keyname', ... ]) KeyId:RkmId RkmId should match the stanza name in RKM.conf? Correct. >> If so, it would be useful if the docs used the same names in the examples (RKMKMIP3 vs rkmname3) And KeyId should match a "Key UUID" in SKLM? Correct. We'll review the documentation to ensure that the meaning of the RkmId in the examples is clear. 5) >> Third. My understanding from talking to various IBM people is that we need ISKLM entitlements for NSD Servers, Protocol nodes and AFM gateways (probably), do we have to do any kind of node registration in ISKLM? Or is this purely based on the certificates being distributed to clients and keys are mapped in ISKLM to the client cert to determine if the node is able to request the key? I'll work on getting clarifications from the ISKLM folks on this aspect. Felipe ---- Felipe Knop [email protected] GPFS Development and Security IBM Systems IBM Building 008 2455 South Rd, Poughkeepsie, NY 12601 (845) 433-9314 T/L 293-9314 From: "Wahl, Edward" <[email protected]> To: gpfsug main discussion list <[email protected]> Date: 04/06/2017 10:55 AM Subject: Re: [gpfsug-discuss] Spectrum Scale Encryption Sent by: [email protected] This is rather dependant on SS version. So what used to happen before 4.2.2.* is that a client would be unable to mount the filesystem in question and would give an error in the mmfs.log.latest for an SGPanic, In 4.2.2.* It appears it will now mount the file system and then give errors on file access instead. (just tested this on 4.2.2.3) I'll have to read through the changelogs looking for this one. Depending on your policy for encryption then, this might be exactly what you want, but I REALLY REALLY dislike this behaviour. To me this means clients can now mount an encrypted FS now and then fail during operation. If I get a client node that comes up improperly, user work will start, and it will fail with "Operation not permitted" errors on file access. I imagine my batch system could run through a massive amount of jobs on a bad client without anyone noticing immeadiately. Yet another thing we now have to monitor now I guess. *shrug* A couple other gotcha's we've seen with Encryption: Encrypted file systems do not store data in large MD blocks. Makes sense. This means large MD blocks aren't as useful as they are in unencrypted FS, if you are using this. Having at least one backup SKLM server is a good idea. "kmipServerUri[N+1]" in the conf. While the documentation claims the FS can continue operation once it caches the MEK if an SKLM server goes away, in operation this does NOT work as you may expect. Your users still need access to the FEKs for the files your clients work on. Logs will fill with Key <key> could not be fetched. errors. Ed Wahl OSC ________________________________________ From: [email protected] [[email protected]] on behalf of Simon Thompson (Research Computing - IT Services) [[email protected]] Sent: Thursday, April 06, 2017 4:20 AM To: [email protected] Subject: [gpfsug-discuss] Spectrum Scale Encryption We are currently looking at adding encryption to our deployment for some of our data sets and for some of our nodes. Apologies in advance if some of this is a bit vague, we're not yet at the point where we can test this stuff out, so maybe some of it will become clear when we try it out. For a node that we don't want to have access to any encrypted data, what do we need to set up? According to the docs: https://www.ibm.com/support/knowledgecenter/STXKQY_4.2.2/com.ibm.spectrum.s cale.v4r22.doc/bl1adv_encryption_prep.htm "After the file system is configured with encryption policy rules, the file system is considered encrypted. From that point on, each node that has access to that file system must have an RKM.conf file present. Otherwise, the file system might not be mounted or might become unmounted." So on a node which I don't want to have access to any encrypted files, do I just need to have an empty RKM.conf file? (If this is the case, would be good to have this added to the docs) Secondly ... (and maybe I'm misunderstanding the docs here) For the Policy https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.2/com.ibm.spectru m.scale.v4r22.doc/bl1adv_encryptionpolicyrules.htm KEYS ('Keyname'[, 'Keyname', ... ]) KeyId:RkmId RkmId should match the stanza name in RKM.conf? If so, it would be useful if the docs used the same names in the examples (RKMKMIP3 vs rkmname3) And KeyId should match a "Key UUID" in SKLM? Third. My understanding from talking to various IBM people is that we need ISKLM entitlements for NSD Servers, Protocol nodes and AFM gateways (probably), do we have to do any kind of node registration in ISKLM? Or is this purely based on the certificates being distributed to clients and keys are mapped in ISKLM to the client cert to determine if the node is able to request the key? Thanks Simon _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
