pitrou commented on PR #259: URL: https://github.com/apache/parquet-format/pull/259#issuecomment-2151699531
How about the following wording to better explain the limit: > Encryption security of the two supported AES modes rely on the uniqueness of the *(encryption key, nonce)* pair accross all invocations of the encryption function. If a given nonce is reused by mistake with the same encryption key, then an attacker could get critical information about the encryption key and potentially break the data encrypted with this particular key. > Since this specification requires nonces to be random-generated, this implies a statistical limit to the number of cipher invocations for a given encryption key. By convention, the section 8 of the NIST SP 800-38D document limits the number of cipher invocations to 2^32 to ensure that the probability of nonce reuse stays below 2^-32. > In Parquet, each encrypted module consists of a separate cipher invocation. Moreover, the vast majority of encrypted modules are page headers and page payloads. This therefore translates into the following conventional limits: > * If data in the Parquet file is encrypted using the file's footer key, then there should be no more than 2^31 pages in a given file, and no more than 2^31 pages accross all files encrypted with the same footer key (in any case, it is recommended to never reuse the same footer key accross files); > * If data in the Parquet file is encrypted using each column's encryption key, then there should no more than 2^31 pages in each given column. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
