Re: [cryptopp-users] Re: XTS does not get the expected result for non-byte-aligned NIST vectors
Will you support bit-oriented XTS in the next release？ The values seem non-sensical to me. Where did 137-bit data unit come from? That's going to be a sector size in real life, so its going to be 512, 2048, 4096, 64k, etc. >>> In practice, there seems to be no such use case. 在2021年5月20日星期四 UTC+8 下午7:37:31 写道： > On Thu, May 20, 2021 at 4:02 AM yu li wrote: > > > > Thank you for your reply. > > > > All test vector in > https://github.com/weidai11/cryptopp/blob/master/TestVectors/xts.txt. are > byte-aligned . I've tested these vectors locally, and they're all pass. But > it doesn't have a non-byte-aligned vector like datalen=250bit in > https://github.com/weidai11/cryptopp/blob/master/TestVectors/xts.txt. We > can found non-byte-aligned vector in > https://csrc.nist.gov/Projects/Cryptographic-Algorithm-Validation-Program/CAVP-TESTING-BLOCK-CIPHER-MODES#XTS > > . All byte-aligned vector in > https://csrc.nist.gov/Projects/Cryptographic-Algorithm-Validation-Program/CAVP-TESTING-BLOCK-CIPHER-MODES#XTS > > can pass， but all non-byte-aligned vector are fail. > > My bad Yu. I took alignment to mean type alignment. Sorry about that. > > XTS mode in Crypto++ is byte oriented, not bit oriented. We could > possibly change that. > > I'm looking at > https://csrc.nist.gov/CSRC/media/Projects/Cryptographic-Algorithm-Validation-Program/documents/aes/XTSVS.pdf > , > and it says (from p. 6): > > An implementation may support a data unit length that is not a > multiple of 8 bits. In this case, the plaintext (PT) and ciphertext > (CT) will be represented in the request, sample, and response files by > a bit string padded with zeros on the right to the next byte boundary, > in hexadecimal. For example, suppose an implementation supports a 137 > bit data unit. The first 128-bit block consists of the first (i.e., > leftmost) 128 bits. If the second, nine-bit partial block is > 011011011, then in the request and sample files it will be padded with > seven zeros on the right – 0110 1101 1000 – and represented as > 6d80 (hex). Response files values should be formatted the same way. > > The values seem non-sensical to me. Where did 137-bit data unit come > from? That's going to be a sector size in real life, so its going to > be 512, 2048, 4096, 64k, etc. > > Jeff > -- You received this message because you are subscribed to the Google Groups "Crypto++ Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to cryptopp-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/cryptopp-users/a1766513-d46f-4b74-b6a0-b9d2184535b9n%40googlegroups.com.
Re: [cryptopp-users] Re: XTS does not get the expected result for non-byte-aligned NIST vectors
On Thu, May 20, 2021 at 4:02 AM yu li wrote: > > Thank you for your reply. > > All test vector in > https://github.com/weidai11/cryptopp/blob/master/TestVectors/xts.txt. are > byte-aligned . I've tested these vectors locally, and they're all pass. But > it doesn't have a non-byte-aligned vector like datalen=250bit in > https://github.com/weidai11/cryptopp/blob/master/TestVectors/xts.txt. We can > found non-byte-aligned vector in > https://csrc.nist.gov/Projects/Cryptographic-Algorithm-Validation-Program/CAVP-TESTING-BLOCK-CIPHER-MODES#XTS > . All byte-aligned vector in > https://csrc.nist.gov/Projects/Cryptographic-Algorithm-Validation-Program/CAVP-TESTING-BLOCK-CIPHER-MODES#XTS > can pass， but all non-byte-aligned vector are fail. My bad Yu. I took alignment to mean type alignment. Sorry about that. XTS mode in Crypto++ is byte oriented, not bit oriented. We could possibly change that. I'm looking at https://csrc.nist.gov/CSRC/media/Projects/Cryptographic-Algorithm-Validation-Program/documents/aes/XTSVS.pdf, and it says (from p. 6): An implementation may support a data unit length that is not a multiple of 8 bits. In this case, the plaintext (PT) and ciphertext (CT) will be represented in the request, sample, and response files by a bit string padded with zeros on the right to the next byte boundary, in hexadecimal. For example, suppose an implementation supports a 137 bit data unit. The first 128-bit block consists of the first (i.e., leftmost) 128 bits. If the second, nine-bit partial block is 011011011, then in the request and sample files it will be padded with seven zeros on the right – 0110 1101 1000 – and represented as 6d80 (hex). Response files values should be formatted the same way. The values seem non-sensical to me. Where did 137-bit data unit come from? That's going to be a sector size in real life, so its going to be 512, 2048, 4096, 64k, etc. Jeff -- You received this message because you are subscribed to the Google Groups "Crypto++ Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to cryptopp-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/cryptopp-users/CAH8yC8mVJ0tPby4uJdQK6xtC%3D1ag7jqvV1fFKf-nU1mb94e0zg%40mail.gmail.com.
[cryptopp-users] Re: XTS does not get the expected result for non-byte-aligned NIST vectors
Thank you for your reply. All test vector in https://github.com/weidai11/cryptopp/blob/master/TestVectors/xts.txt. are byte-aligned . I've tested these vectors locally, and they're all pass. But it doesn't have a non-byte-aligned vector like datalen=250bit in https://github.com/weidai11/cryptopp/blob/master/TestVectors/xts.txt. We can found non-byte-aligned vector in https://csrc.nist.gov/Projects/Cryptographic-Algorithm-Validation-Program/CAVP-TESTING-BLOCK-CIPHER-MODES#XTS . All byte-aligned vector in https://csrc.nist.gov/Projects/Cryptographic-Algorithm-Validation-Program/CAVP-TESTING-BLOCK-CIPHER-MODES#XTS can pass， but all non-byte-aligned vector are fail. 在2021年5月20日星期四 UTC+8 下午1:07:48 写道： > On Thursday, May 20, 2021 at 12:38:14 AM UTC-4 Jeffrey Walton wrote: > >> On Wednesday, May 19, 2021 at 11:53:54 PM UTC-4 yu...@nvidia.com wrote: >> >>> I intend to use Crypto++ XTS-AES for encrypting data. I tested XTS-AES >>> with the following NIST vector and got an unexpected >>> CT= 54a8629d76db46d0c516fca52c9c903baa3a635ddd56f09760f63252c8b46140 . >>> >>> I call the following function: >>> XTS_Mode< AES >::Encryption e; >>> e.SetKeyWithIV(in_key, KeySize, in_iv, IVSize); >>> StringSource(in_plain_txt,svSize(plain_txt, 1),true, >>> new StreamTransformationFilter(e, >>> new StringSink(cipher), >>> StreamTransformationFilter::NO_PADDING >>> ) // StreamTransformationFilter >>> ); // StringSource >>> >>> My questions: >>> >>>- Does Crypto++ XTS-AES not support non-byte aligned data >>>encryption and decryption >>> >>> >> XTS mode should support non-aligned data. For SSE, it uses _mm_loadu_si128 >> and _mm_storeu_si128, which are unaligned loads and stores. The class >> also uses 'GetWord(false, LITTLE_ENDIAN_ORDER, ...)' and >> 'PutWord(false, LITTLE_ENDIAN_ORDER, ...)'. The 'false' says the >> data is not aligned, so a memcpy is used. Also see >> https://github.com/weidai11/cryptopp/blob/master/xts.cpp. >> >> We've seen some trouble with bad code generation when two pointers happen >> to be the same in strcipher.cpp. But I don't believe XTS uses it. Also see >> https://github.com/weidai11/cryptopp/issues/1010. >> > > Something else that might be a factor... The head notes in xts.h ( > https://github.com/weidai11/cryptopp/blob/master/xts.h) says: > > /// \details XTS mode is a wide block mode defined by IEEE > P1619-2008. NIST > /// SP-800-38E approves the mode for storage devices citing IEEE > 1619-2007. > /// IEEE 1619-2007 provides both a reference implementation and test > vectors. > /// The IEEE reference implementation fails to arrive at the expected > result > /// for some test vectors. > > That's a bad sign :( > > I keep the programs that are used to generate test vectors so we can > always establish provenance if needed. The program is located in one of my > testing GitHubs. For XTS, it is located at > https://github.com/noloader/cryptopp-test. > > The test vectors we use are located at > https://github.com/weidai11/cryptopp/blob/master/TestVectors/xts.txt. > Looking through the test vector, it looks like we use both the XTS test > vectors, the XTS reference implementation and Botan. Botan is Jack Lloyd's > Botan (https://github.com/randombit/botan), and we use to to add > additional test vectors when we feel there are gaps. > > Jeff > -- You received this message because you are subscribed to the Google Groups "Crypto++ Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to cryptopp-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/cryptopp-users/2cdb34ba-208c-4de8-a9a5-dee081f8c836n%40googlegroups.com.
[cryptopp-users] Re: XTS does not get the expected result for non-byte-aligned NIST vectors
On Thursday, May 20, 2021 at 12:38:14 AM UTC-4 Jeffrey Walton wrote: > On Wednesday, May 19, 2021 at 11:53:54 PM UTC-4 yu...@nvidia.com wrote: > >> I intend to use Crypto++ XTS-AES for encrypting data. I tested XTS-AES >> with the following NIST vector and got an unexpected >> CT= 54a8629d76db46d0c516fca52c9c903baa3a635ddd56f09760f63252c8b46140 . >> >> I call the following function: >> XTS_Mode< AES >::Encryption e; >> e.SetKeyWithIV(in_key, KeySize, in_iv, IVSize); >> StringSource(in_plain_txt,svSize(plain_txt, 1),true, >> new StreamTransformationFilter(e, >> new StringSink(cipher), >> StreamTransformationFilter::NO_PADDING >> ) // StreamTransformationFilter >> ); // StringSource >> >> My questions: >> >>- Does Crypto++ XTS-AES not support non-byte aligned data >>encryption and decryption >> >> > XTS mode should support non-aligned data. For SSE, it uses _mm_loadu_si128 > and _mm_storeu_si128, which are unaligned loads and stores. The class > also uses 'GetWord(false, LITTLE_ENDIAN_ORDER, ...)' and > 'PutWord(false, LITTLE_ENDIAN_ORDER, ...)'. The 'false' says the > data is not aligned, so a memcpy is used. Also see > https://github.com/weidai11/cryptopp/blob/master/xts.cpp. > > We've seen some trouble with bad code generation when two pointers happen > to be the same in strcipher.cpp. But I don't believe XTS uses it. Also see > https://github.com/weidai11/cryptopp/issues/1010. > Something else that might be a factor... The head notes in xts.h (https://github.com/weidai11/cryptopp/blob/master/xts.h) says: /// \details XTS mode is a wide block mode defined by IEEE P1619-2008. NIST /// SP-800-38E approves the mode for storage devices citing IEEE 1619-2007. /// IEEE 1619-2007 provides both a reference implementation and test vectors. /// The IEEE reference implementation fails to arrive at the expected result /// for some test vectors. That's a bad sign :( I keep the programs that are used to generate test vectors so we can always establish provenance if needed. The program is located in one of my testing GitHubs. For XTS, it is located at https://github.com/noloader/cryptopp-test. The test vectors we use are located at https://github.com/weidai11/cryptopp/blob/master/TestVectors/xts.txt. Looking through the test vector, it looks like we use both the XTS test vectors, the XTS reference implementation and Botan. Botan is Jack Lloyd's Botan (https://github.com/randombit/botan), and we use to to add additional test vectors when we feel there are gaps. Jeff -- You received this message because you are subscribed to the Google Groups "Crypto++ Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to cryptopp-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/cryptopp-users/22c62ccc-dc12-4ffb-93fd-a96b14003c17n%40googlegroups.com.
[cryptopp-users] Re: XTS does not get the expected result for non-byte-aligned NIST vectors
On Wednesday, May 19, 2021 at 11:53:54 PM UTC-4 yu...@nvidia.com wrote: > I intend to use Crypto++ XTS-AES for encrypting data. I tested XTS-AES > with the following NIST vector and got an unexpected > CT= 54a8629d76db46d0c516fca52c9c903baa3a635ddd56f09760f63252c8b46140 . > > I call the following function: > XTS_Mode< AES >::Encryption e; > e.SetKeyWithIV(in_key, KeySize, in_iv, IVSize); > StringSource(in_plain_txt,svSize(plain_txt, 1),true, > new StreamTransformationFilter(e, > new StringSink(cipher), > StreamTransformationFilter::NO_PADDING > ) // StreamTransformationFilter > ); // StringSource > > My questions: > >- Does Crypto++ XTS-AES not support non-byte aligned data >encryption and decryption > > XTS mode should support non-aligned data. For SSE, it uses _mm_loadu_si128 and _mm_storeu_si128, which are unaligned loads and stores. The class also uses 'GetWord(false, LITTLE_ENDIAN_ORDER, ...)' and 'PutWord(false, LITTLE_ENDIAN_ORDER, ...)'. The 'false' says the data is not aligned, so a memcpy is used. Also see https://github.com/weidai11/cryptopp/blob/master/xts.cpp. We've seen some trouble with bad code generation when two pointers happen to be the same in strcipher.cpp. But I don't believe XTS uses it. Also see https://github.com/weidai11/cryptopp/issues/1010. Do you have a reproducer? Jeff -- You received this message because you are subscribed to the Google Groups "Crypto++ Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to cryptopp-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/cryptopp-users/02885d8a-b28d-4e65-9153-5409d288632dn%40googlegroups.com.
