ggershinsky commented on PR #259: URL: https://github.com/apache/parquet-format/pull/259#issuecomment-2152274863
> > Encryption security of the two supported AES modes relies on the uniqueness of the _(encryption key, nonce)_ pair accross all invocations of the encryption function. If a given nonce is reused by mistake with the same encryption key, then an attacker could get critical information about the encryption key and potentially break the data encrypted with this particular key. > > > Since this specification requires nonces to be random-generated, this implies a statistical limit to the number of cipher invocations for a given encryption key. By convention, the section 8 of the NIST SP 800-38D document limits the number of cipher invocations to 2^32 to ensure that the probability of nonce reuse stays below 2^-32. The ultimate source of this Parquet limitation is the NIST spec, section 8.3 (more specifically, this text : "_The total number of invocations of the authenticated encryption function shall not exceed 2^32, including all IV lengths and all instances of the authenticated encryption function with the given key_"). The Parquet spec can ref/site this text, but shouldn't supply an explanation (must be very careful here - best be left to the official NIST explanation in the section 8 and in the referenced papers). Our spec will also be more compact then, no overlap with the NIST doc. > > > In Parquet, each encrypted module consists of a separate cipher invocation. Moreover, the vast majority of encrypted modules are page headers and page payloads. This therefore translates into the following conventional limits: > > > * If data in the Parquet file is encrypted using the file's footer key, then there should be no more than 2^31 pages in a given file, and no more than 2^31 pages accross all files encrypted with the same footer key (in any case, it is recommended to never reuse the same footer key accross files); > > * If data in the Parquet file is encrypted using each column's encryption key, then there should no more than 2^31 pages in each given column. This is not entirely accurate. Some columns can be encrypted with the footer key. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
