Re: [lng-odp] [APO-NEXT PATCHv6 1/3] api: random: add explicit controls over random data

[Bill Fischofer]

On Wed, Dec 7, 2016 at 8:21 AM, Savolainen, Petri (Nokia - FI/Espoo)
 wrote:
>
>> >> +int32_t odp_random_repeatable_data(uint8_t *buf, uint32_t len,
>> >> +odp_random_kind_t kind, uint32_t
>> *seed);
>> >>
>> >
>> > If pseudo/deterministic is not a good term for this, then I think it's
>> better to profile it strictly for testing (all production cases are
>> covered by odp_random_data()). Also for testing the kind is not needed, we
>> just say that: "this is good only for testing, do not use for production".
>> If kind would be there, it would give two ways to achieve "cryptograptic
>> quality" randoms and that's not the intention of the call. Also a larger
>> seed enables better randomness of the output.
>>
>> Actually if you want to test the implementation of a cryptographic
>> algorithm you need repeatable cryptographic data as input. This is
>> just being consistent across the API. You would never use the
>> repeatable variant in production and I don't think that would be a
>> point of confusion. The term "cryptographic quality" simply refers to
>> the statistical properties of the data, and nothing more should be
>> read into that term.
>>
>
> OK. This is then the compromise
>
> /**
>  * Generate repeatable random data for testing purposes
>  *
>  * For testing purposes it is often useful to generate "random" sequences
>  * that are repeatable...
>  *
>  * This function should be used only for testing purposes, use 
> odp_random_data()
>  * for production.
>  */
> int32_t odp_random_test_data(uint8_t *buf, uint32_t len, odp_random_kind_t 
> kind, uint64_t *seed);
>
>
> -Petri

I'll send a v7 with these changes.

>
>
>> >
>> >
>> > /**
>> >  * Generate repeatable random data for testing purposes
>> >  *
>> >  * For testing purposes it is often useful to generate "random"
>> sequences
>> >  * that are repeatable...
>> >  *
>> >  * This function should be used only for testing purposes, use
>> odp_random_data()
>> >  * for production.
>> >  */
>> > int32_t odp_random_test_data(uint8_t *buf, uint32_t len, uint64_t
>> *seed);
>> >
>> >
>> > -Petri
>> >

Re: [lng-odp] [APO-NEXT PATCHv6 1/3] api: random: add explicit controls over random data

[Savolainen, Petri (Nokia - FI/Espoo)]

> >> +int32_t odp_random_repeatable_data(uint8_t *buf, uint32_t len,
> >> +odp_random_kind_t kind, uint32_t
> *seed);
> >>
> >
> > If pseudo/deterministic is not a good term for this, then I think it's
> better to profile it strictly for testing (all production cases are
> covered by odp_random_data()). Also for testing the kind is not needed, we
> just say that: "this is good only for testing, do not use for production".
> If kind would be there, it would give two ways to achieve "cryptograptic
> quality" randoms and that's not the intention of the call. Also a larger
> seed enables better randomness of the output.
> 
> Actually if you want to test the implementation of a cryptographic
> algorithm you need repeatable cryptographic data as input. This is
> just being consistent across the API. You would never use the
> repeatable variant in production and I don't think that would be a
> point of confusion. The term "cryptographic quality" simply refers to
> the statistical properties of the data, and nothing more should be
> read into that term.
> 

OK. This is then the compromise

/**
 * Generate repeatable random data for testing purposes
 *
 * For testing purposes it is often useful to generate "random" sequences
 * that are repeatable...
 *
 * This function should be used only for testing purposes, use odp_random_data()
 * for production.
 */
int32_t odp_random_test_data(uint8_t *buf, uint32_t len, odp_random_kind_t 
kind, uint64_t *seed);


-Petri


> >
> >
> > /**
> >  * Generate repeatable random data for testing purposes
> >  *
> >  * For testing purposes it is often useful to generate "random"
> sequences
> >  * that are repeatable...
> >  *
> >  * This function should be used only for testing purposes, use
> odp_random_data()
> >  * for production.
> >  */
> > int32_t odp_random_test_data(uint8_t *buf, uint32_t len, uint64_t
> *seed);
> >
> >
> > -Petri
> >

Re: [lng-odp] [APO-NEXT PATCHv6 1/3] api: random: add explicit controls over random data

[Bill Fischofer]

On Wed, Dec 7, 2016 at 3:10 AM, Savolainen, Petri (Nokia - FI/Espoo)
<petri.savolai...@nokia-bell-labs.com> wrote:
>
>
>> -Original Message-
>> From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of Bill
>> Fischofer
>> Sent: Monday, December 05, 2016 11:29 PM
>> To: lng-odp@lists.linaro.org
>> Subject: [lng-odp] [APO-NEXT PATCHv6 1/3] api: random: add explicit
>> controls over random data
>>
>> Rework the odp_random_data() API to replace the use_entropy with an
>> explicit odp_random_kind parameter that controls the type of random
>> desired. Two new APIs are also introduced:
>>
>> - odp_random_max_kind() returns the maximum kind of random data available
>>
>> - odp_random_repeatable_data() permits applications to generate repeatable
>>   random sequences for testing purposes
>>
>> Because the type signature of odp_random_data() is changed, the
>> implementation and validation tests are included here for bisectability.
>
> This does not need to be mentioned in git log, since it's our rule anyway 
> (that every commit must build and pass tests). Actually, it would be cleaner 
> to include only the minimum amount of implementation changes into the API 
> patch: the new odp_random_data() implementation and the test for that (single 
> line change).

OK

>
>
>
>> --- a/include/odp/api/spec/random.h
>> +++ b/include/odp/api/spec/random.h
>> @@ -24,18 +24,68 @@ extern "C" {
>>   */
>>
>>  /**
>> + * Random kind selector
>> + */
>> +typedef enum {
>> + /** Basic random, presumably pseudo-random generated by SW */
>> + ODP_RANDOM_BASIC,
>> + /** Cryptographic quality random */
>> + ODP_RANDOM_CRYPTO,
>> + /** True random, generated from a HW entropy source */
>> + ODP_RANDOM_TRUE,
>> +} odp_random_kind_t;
>> +
>> +/**
>> + * Query random max kind
>> + *
>
> This would be a good place to state the order of kinds: which is the "max" 
> end the enum.
>
>> + * @return kind The maximum odp_random_kind_t supported by this
>> implementation
>> + */
>> +odp_random_kind_t odp_random_max_kind(void);
>> +
>> +/**
>>   * Generate random byte data
>>   *
>> + * The intent in supporting different kinds of random data is to allow
>> + * tradeoffs between performance and the quality of random data needed.
>> The
>> + * assumption is that basic random is cheap while true random is
>> relatively
>> + * expensive in terms of time to generate, with cryptographic random
>> being
>> + * something in between. Implementations that support highly efficient
>> true
>> + * random are free to use this for all requested kinds. So it is always
>> + * permissible to "upgrade" a random data request, but never to
>> "downgrade"
>> + * such requests.
>> + *
>>   * @param[out]buf   Output buffer
>> - * @param size  Size of output buffer
>> - * @param use_entropy   Use entropy
>> + * @param len   Length of output buffer in bytes
>> + * @param kind  Specifies the type of random data required.
>> Request
>> + *  is expected to fail if the implementation is
>> unable to
>> + *  provide the requested type.
>> + *
>> + * @return Number of bytes written
>> + * @retval <0 on failure
>> + */
>> +int32_t odp_random_data(uint8_t *buf, uint32_t len, odp_random_kind_t
>> kind);
>> +
>> +/**
>> + * Generate repeatable random byte data
>> + *
>> + * For testing purposes it is often useful to generate "random" sequences
>> + * that are repeatable. This is accomplished by supplying a seed value
>> that
>> + * is used for pseudo-random data generation. The caller provides
>>   *
>> - * @todo Define the implication of the use_entropy parameter
>> + * @param[out]buf  Output buffer
>> + * @param len  Length of output buffer in bytes
>> + * @param kind Specifies the type of random data required.
>> Request
>> + * will fail if the implementation is unable to
>> provide
>> + * repeatable random of the requested type. This is
>> + * always true for true random and may be true for
>> + * cryptographic random.
>> + * @param[in,out] seed Seed value to use
>>   *
>>   * @return Number of bytes written
>>   * @retval <0 on failure
>>   */
>> -int32_t odp_ran

Re: [lng-odp] [APO-NEXT PATCHv6 1/3] api: random: add explicit controls over random data

[Savolainen, Petri (Nokia - FI/Espoo)]

> -Original Message-
> From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of Bill
> Fischofer
> Sent: Monday, December 05, 2016 11:29 PM
> To: lng-odp@lists.linaro.org
> Subject: [lng-odp] [APO-NEXT PATCHv6 1/3] api: random: add explicit
> controls over random data
> 
> Rework the odp_random_data() API to replace the use_entropy with an
> explicit odp_random_kind parameter that controls the type of random
> desired. Two new APIs are also introduced:
> 
> - odp_random_max_kind() returns the maximum kind of random data available
> 
> - odp_random_repeatable_data() permits applications to generate repeatable
>   random sequences for testing purposes
> 
> Because the type signature of odp_random_data() is changed, the
> implementation and validation tests are included here for bisectability.

This does not need to be mentioned in git log, since it's our rule anyway (that 
every commit must build and pass tests). Actually, it would be cleaner to 
include only the minimum amount of implementation changes into the API patch: 
the new odp_random_data() implementation and the test for that (single line 
change).



> --- a/include/odp/api/spec/random.h
> +++ b/include/odp/api/spec/random.h
> @@ -24,18 +24,68 @@ extern "C" {
>   */
> 
>  /**
> + * Random kind selector
> + */
> +typedef enum {
> + /** Basic random, presumably pseudo-random generated by SW */
> + ODP_RANDOM_BASIC,
> + /** Cryptographic quality random */
> + ODP_RANDOM_CRYPTO,
> + /** True random, generated from a HW entropy source */
> + ODP_RANDOM_TRUE,
> +} odp_random_kind_t;
> +
> +/**
> + * Query random max kind
> + *

This would be a good place to state the order of kinds: which is the "max" end 
the enum.

> + * @return kind The maximum odp_random_kind_t supported by this
> implementation
> + */
> +odp_random_kind_t odp_random_max_kind(void);
> +
> +/**
>   * Generate random byte data
>   *
> + * The intent in supporting different kinds of random data is to allow
> + * tradeoffs between performance and the quality of random data needed.
> The
> + * assumption is that basic random is cheap while true random is
> relatively
> + * expensive in terms of time to generate, with cryptographic random
> being
> + * something in between. Implementations that support highly efficient
> true
> + * random are free to use this for all requested kinds. So it is always
> + * permissible to "upgrade" a random data request, but never to
> "downgrade"
> + * such requests.
> + *
>   * @param[out]buf   Output buffer
> - * @param size  Size of output buffer
> - * @param use_entropy   Use entropy
> + * @param len   Length of output buffer in bytes
> + * @param kind  Specifies the type of random data required.
> Request
> + *  is expected to fail if the implementation is
> unable to
> + *  provide the requested type.
> + *
> + * @return Number of bytes written
> + * @retval <0 on failure
> + */
> +int32_t odp_random_data(uint8_t *buf, uint32_t len, odp_random_kind_t
> kind);
> +
> +/**
> + * Generate repeatable random byte data
> + *
> + * For testing purposes it is often useful to generate "random" sequences
> + * that are repeatable. This is accomplished by supplying a seed value
> that
> + * is used for pseudo-random data generation. The caller provides
>   *
> - * @todo Define the implication of the use_entropy parameter
> + * @param[out]buf  Output buffer
> + * @param len  Length of output buffer in bytes
> + * @param kind Specifies the type of random data required.
> Request
> + * will fail if the implementation is unable to
> provide
> + * repeatable random of the requested type. This is
> + * always true for true random and may be true for
> + * cryptographic random.
> + * @param[in,out] seed Seed value to use
>   *
>   * @return Number of bytes written
>   * @retval <0 on failure
>   */
> -int32_t odp_random_data(uint8_t *buf, int32_t size, odp_bool_t
> use_entropy);
> +int32_t odp_random_repeatable_data(uint8_t *buf, uint32_t len,
> +odp_random_kind_t kind, uint32_t *seed);
> 

If pseudo/deterministic is not a good term for this, then I think it's better 
to profile it strictly for testing (all production cases are covered by 
odp_random_data()). Also for testing the kind is not needed, we just say that: 
"this is good only for testing, do not use for production". If kind would be 
there, it would give two ways to achieve &

Re: [lng-odp] [APO-NEXT PATCHv6 1/3] api: random: add explicit controls over random data

[Bill Fischofer]

Sorry about the typo in the patch subject. Obviously should be
API-NEXT. I'll correct in v7 if that's needed.

On Mon, Dec 5, 2016 at 3:28 PM, Bill Fischofer
 wrote:
> Rework the odp_random_data() API to replace the use_entropy with an
> explicit odp_random_kind parameter that controls the type of random
> desired. Two new APIs are also introduced:
>
> - odp_random_max_kind() returns the maximum kind of random data available
>
> - odp_random_repeatable_data() permits applications to generate repeatable
>   random sequences for testing purposes
>
> Because the type signature of odp_random_data() is changed, the
> implementation and validation tests are included here for bisectability.
>
> Signed-off-by: Bill Fischofer 
> ---
> Changes in v6:
> - Add odp_random_max_kind() API instead of adding this to the
>   odp_crypto_capability() API.
> - Rename odp_random_seeded_data() to odp_random_repeatable_data()
> - Merge API defs, implementation, and validation to preserve bisectability
>
> Changes in v5:
> - Change return type from int to int32_t for random APIs
>
> Changes in v4:
> - Normalize random API signatures with other ODP APIs
> - Add new odp_random_seeded_data() API for repeatable random data generation
> - Add additional tests for new odp_random_seeded_data() API
> - Break out crypto section of User Guide to its own sub-document
> - Add User Guide docuemntation for ODP random data API.
>
> Changes in v3:
> - Address commments by Petri
> - Rename ODP_RAND_NORMAL to ODP_RANDOM_BASIC to avoid confusion with the
> mathematical term "normal"
>
>  include/odp/api/spec/random.h   | 58 
> +++--
>  platform/linux-generic/odp_crypto.c | 50 ++---
>  test/common_plat/validation/api/random/random.c | 54 ++-
>  test/common_plat/validation/api/random/random.h |  2 +
>  4 files changed, 154 insertions(+), 10 deletions(-)
>
> diff --git a/include/odp/api/spec/random.h b/include/odp/api/spec/random.h
> index 00fa15b..bc130f5 100644
> --- a/include/odp/api/spec/random.h
> +++ b/include/odp/api/spec/random.h
> @@ -24,18 +24,68 @@ extern "C" {
>   */
>
>  /**
> + * Random kind selector
> + */
> +typedef enum {
> +   /** Basic random, presumably pseudo-random generated by SW */
> +   ODP_RANDOM_BASIC,
> +   /** Cryptographic quality random */
> +   ODP_RANDOM_CRYPTO,
> +   /** True random, generated from a HW entropy source */
> +   ODP_RANDOM_TRUE,
> +} odp_random_kind_t;
> +
> +/**
> + * Query random max kind
> + *
> + * @return kind The maximum odp_random_kind_t supported by this 
> implementation
> + */
> +odp_random_kind_t odp_random_max_kind(void);
> +
> +/**
>   * Generate random byte data
>   *
> + * The intent in supporting different kinds of random data is to allow
> + * tradeoffs between performance and the quality of random data needed. The
> + * assumption is that basic random is cheap while true random is relatively
> + * expensive in terms of time to generate, with cryptographic random being
> + * something in between. Implementations that support highly efficient true
> + * random are free to use this for all requested kinds. So it is always
> + * permissible to "upgrade" a random data request, but never to "downgrade"
> + * such requests.
> + *
>   * @param[out]buf   Output buffer
> - * @param size  Size of output buffer
> - * @param use_entropy   Use entropy
> + * @param len   Length of output buffer in bytes
> + * @param kind  Specifies the type of random data required. Request
> + *  is expected to fail if the implementation is unable 
> to
> + *  provide the requested type.
> + *
> + * @return Number of bytes written
> + * @retval <0 on failure
> + */
> +int32_t odp_random_data(uint8_t *buf, uint32_t len, odp_random_kind_t kind);
> +
> +/**
> + * Generate repeatable random byte data
> + *
> + * For testing purposes it is often useful to generate "random" sequences
> + * that are repeatable. This is accomplished by supplying a seed value that
> + * is used for pseudo-random data generation. The caller provides
>   *
> - * @todo Define the implication of the use_entropy parameter
> + * @param[out]buf  Output buffer
> + * @param len  Length of output buffer in bytes
> + * @param kind Specifies the type of random data required. Request
> + * will fail if the implementation is unable to provide
> + * repeatable random of the requested type. This is
> + * always true for true random and may be true for
> + * cryptographic random.
> + * @param[in,out] seed Seed value to use
>   *
>   * @return Number of bytes written
>   * @retval <0 on failure
>   */
> -int32_t odp_random_data(uint8_t *buf, int32_t size, odp_bool_t use_entropy);
> +int32_t odp_random_repeatable_data(uint8_t *buf, 

[lng-odp] [APO-NEXT PATCHv6 1/3] api: random: add explicit controls over random data

[Bill Fischofer]

Rework the odp_random_data() API to replace the use_entropy with an
explicit odp_random_kind parameter that controls the type of random
desired. Two new APIs are also introduced:

- odp_random_max_kind() returns the maximum kind of random data available

- odp_random_repeatable_data() permits applications to generate repeatable
  random sequences for testing purposes

Because the type signature of odp_random_data() is changed, the
implementation and validation tests are included here for bisectability.

Signed-off-by: Bill Fischofer 
---
Changes in v6:
- Add odp_random_max_kind() API instead of adding this to the
  odp_crypto_capability() API.
- Rename odp_random_seeded_data() to odp_random_repeatable_data()
- Merge API defs, implementation, and validation to preserve bisectability

Changes in v5:
- Change return type from int to int32_t for random APIs

Changes in v4:
- Normalize random API signatures with other ODP APIs
- Add new odp_random_seeded_data() API for repeatable random data generation
- Add additional tests for new odp_random_seeded_data() API
- Break out crypto section of User Guide to its own sub-document
- Add User Guide docuemntation for ODP random data API.

Changes in v3:
- Address commments by Petri
- Rename ODP_RAND_NORMAL to ODP_RANDOM_BASIC to avoid confusion with the
mathematical term "normal"

 include/odp/api/spec/random.h   | 58 +++--
 platform/linux-generic/odp_crypto.c | 50 ++---
 test/common_plat/validation/api/random/random.c | 54 ++-
 test/common_plat/validation/api/random/random.h |  2 +
 4 files changed, 154 insertions(+), 10 deletions(-)

diff --git a/include/odp/api/spec/random.h b/include/odp/api/spec/random.h
index 00fa15b..bc130f5 100644
--- a/include/odp/api/spec/random.h
+++ b/include/odp/api/spec/random.h
@@ -24,18 +24,68 @@ extern "C" {
  */
 
 /**
+ * Random kind selector
+ */
+typedef enum {
+   /** Basic random, presumably pseudo-random generated by SW */
+   ODP_RANDOM_BASIC,
+   /** Cryptographic quality random */
+   ODP_RANDOM_CRYPTO,
+   /** True random, generated from a HW entropy source */
+   ODP_RANDOM_TRUE,
+} odp_random_kind_t;
+
+/**
+ * Query random max kind
+ *
+ * @return kind The maximum odp_random_kind_t supported by this implementation
+ */
+odp_random_kind_t odp_random_max_kind(void);
+
+/**
  * Generate random byte data
  *
+ * The intent in supporting different kinds of random data is to allow
+ * tradeoffs between performance and the quality of random data needed. The
+ * assumption is that basic random is cheap while true random is relatively
+ * expensive in terms of time to generate, with cryptographic random being
+ * something in between. Implementations that support highly efficient true
+ * random are free to use this for all requested kinds. So it is always
+ * permissible to "upgrade" a random data request, but never to "downgrade"
+ * such requests.
+ *
  * @param[out]buf   Output buffer
- * @param size  Size of output buffer
- * @param use_entropy   Use entropy
+ * @param len   Length of output buffer in bytes
+ * @param kind  Specifies the type of random data required. Request
+ *  is expected to fail if the implementation is unable to
+ *  provide the requested type.
+ *
+ * @return Number of bytes written
+ * @retval <0 on failure
+ */
+int32_t odp_random_data(uint8_t *buf, uint32_t len, odp_random_kind_t kind);
+
+/**
+ * Generate repeatable random byte data
+ *
+ * For testing purposes it is often useful to generate "random" sequences
+ * that are repeatable. This is accomplished by supplying a seed value that
+ * is used for pseudo-random data generation. The caller provides
  *
- * @todo Define the implication of the use_entropy parameter
+ * @param[out]buf  Output buffer
+ * @param len  Length of output buffer in bytes
+ * @param kind Specifies the type of random data required. Request
+ * will fail if the implementation is unable to provide
+ * repeatable random of the requested type. This is
+ * always true for true random and may be true for
+ * cryptographic random.
+ * @param[in,out] seed Seed value to use
  *
  * @return Number of bytes written
  * @retval <0 on failure
  */
-int32_t odp_random_data(uint8_t *buf, int32_t size, odp_bool_t use_entropy);
+int32_t odp_random_repeatable_data(uint8_t *buf, uint32_t len,
+  odp_random_kind_t kind, uint32_t *seed);
 
 /**
  * @}
diff --git a/platform/linux-generic/odp_crypto.c 
b/platform/linux-generic/odp_crypto.c
index 7e686ff..a731528 100644
--- a/platform/linux-generic/odp_crypto.c
+++ b/platform/linux-generic/odp_crypto.c
@@ -4,6 +4,7 @@
  * SPDX-License-Identifier: BSD-3-Clause
  */
 
+#include 
 #include 
 #include 
 #include 
@@