I think it's clearly too late to bring comments about the actual algorithm being used. But I'll post my comments anyways, even though I expect no substantive changes to be made to the algorithm.
I'm cc'ing Ulrich Drepper, the author of http://opensolaris.org/os/community/arc/caselog/2007/642/materials/algorithm-spec-txt/ That file does not list the e-mail addresses of other members of the discussion group. I've cc'ed Don Cragun (participant from Sun); Casper Dik was already cc'ed. Please forward this to the other participants from IBM and Red Hat if appropriate. - The note in step 3 is incorrect. The rationale for not adding the $5$ or $6$ prefix should not be that it adds no entropy for neither does the salt. The rationale should be that adding a constant sub-string adds no security (the salt isn't random, but it isn't constant either), and possibly adds a weakness (known plaintext in all cases, as opposed to the salt which known plaintext, but different for every user). - Step 11(a) has a typo: "a) for a 1-digit add digest B to digest " should read: "a) for a 1-digit add digest B to digest A" - Step 11 is new, but no justification is given. Furthermore, adding "digest B" to digest A is very different from adding "the password string" to digest A. Add to this the fact that password length distributions won't be random, and I worry about what step 11 does to the security of this function. I'm not a cryptographer, so I'm not sure whether to be concerned about step 11. I'd like to see review of step 11 (in context, of course) by a subject matter expert. - Was PBKDF2 (RFC2898) with HMAC-SHA-256/512 considered? If not, why not? PBKDF2 with HMAC-SHA-256/512 would have a number of benefits: - PBKDF2 has been around longer (at least since March 1999) - PBKDF2 already has variable round counts - HMAC is arguably the right structure when one has a secret value (password) and non-secret values (salt) and wishes to produce a digest of the two. HMAC-MD5 is still considered sufficiently strong that the recent breaks of MD5 are not disastrous to protocols using HMAC-MD5. Similarly for HMAC-SHA-1. - Use of off-the-shelf cryptographic algorithms and protocols is better than inventing new ones. For example, it simplifies review by non-cryptographers. Implementations exist (abound?), including open source ones. Kerberos V, for example, uses PBKDF2 w/ HMAC-SHA-1 to derive AES keys from passwords (RFC3962), and there are at least three open source implementations of this of distinct lineage, and, IIUC, all of the participants in the discussion of this algorithm ship one of these or a derivative of one. - Why a difference of only 3 orders of magnitude between ROUNDS_MIN and ROUNDS_MAX? That does not sound like much to me. What is the expected useful lifetime of this algorithm? I think clients should have a _local_ value of ROUNDS_MAX that is appropriate given the available local resources. I.e., a slow cell phone shouldn't accept a round count from a third party that would cause it to spend years computing this function!. This issue came up when the IETF KRB-WG worked on RFC3962, and I think it's come up in the discussion of other protocols in the IETF (SASL, I think). See RFC3962 and/or IETF KRB WG mailing list archives for a discussion of this matter. Nico --