[dpdk-dev] [PATCH v6 9/9] table: align rte table hash structs for cache line size
Thanks Cristian and Thomas for your feedback. I have taken your suggestions and sent out v7. Please check if the new patch is fine. Thanks, Gowrishankar On Thursday 08 September 2016 03:10 PM, Dumitrescu, Cristian wrote: > >> -Original Message- >> From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com] >> Sent: Thursday, September 8, 2016 10:36 AM >> To: Gowrishankar Muthukrishnan >> Cc: dev at dpdk.org; Dumitrescu, Cristian ; >> Chao Zhu ; Richardson, Bruce >> ; Ananyev, Konstantin >> ; Pradeep >> Subject: Re: [dpdk-dev] [PATCH v6 9/9] table: align rte table hash structs >> for >> cache line size >> >> 2016-08-31 17:29, Dumitrescu, Cristian: >>> From: Gowrishankar Muthukrishnan >>>> rte table hash structs rte_bucket_4_8, rte_bucket_4_16 and >>>> rte_bucket_4_32 have >>>> to be cache aligned as required by their corresponding hash create >> functions >>>> rte_table_hash_create_key8_lru etc. >>> Hi Gowrishankar, >>> >>> My understanding is you are trying to work around the check invoked by >> the hash table create functions that verifies the size of the bucket header >> structure is a multiple of the cache line, right? >>> Given that the size of this structure is 1x, 2x or 3x times 64 bytes, the >>> check >> passes on IA CPUs (cache line of 64 bytes; explicit alignment to cache line >> size >> is not needed in order to make the size of the structure a multiple of cache >> line), but not on PPC CPUs (cache line of 128 bytes), correct? >>> I don't think your proposal provides the best way to fix this issue, since >> your code leads to a considerable increase in the memory consumption used >> per bucket in most cases: >>> - 100% more memory for 8-byte key hash table >>> - 0% more for 16-byte key hash table (code does not fix anything, >> explicit alignment is not needed) >>> - 50% more for 32-byte key hash table >>> >>> I suggest you simply change the check: instead of validating this data >> structure is a multiple of cache line size, validate it is a multiple of 64 >> bytes. >> >> Any news please? >> The whole series is blocked for this patch. >> Should we expect a v7? > Yes, I think we should. Small fix for a considerable benefit. > >
[dpdk-dev] [PATCH v6 9/9] table: align rte table hash structs for cache line size
2016-08-31 17:29, Dumitrescu, Cristian: > From: Gowrishankar Muthukrishnan > > rte table hash structs rte_bucket_4_8, rte_bucket_4_16 and > > rte_bucket_4_32 have > > to be cache aligned as required by their corresponding hash create functions > > rte_table_hash_create_key8_lru etc. > > Hi Gowrishankar, > > My understanding is you are trying to work around the check invoked by the > hash table create functions that verifies the size of the bucket header > structure is a multiple of the cache line, right? > > Given that the size of this structure is 1x, 2x or 3x times 64 bytes, the > check passes on IA CPUs (cache line of 64 bytes; explicit alignment to cache > line size is not needed in order to make the size of the structure a multiple > of cache line), but not on PPC CPUs (cache line of 128 bytes), correct? > > I don't think your proposal provides the best way to fix this issue, since > your code leads to a considerable increase in the memory consumption used per > bucket in most cases: > - 100% more memory for 8-byte key hash table > - 0% more for 16-byte key hash table (code does not fix anything, > explicit alignment is not needed) > - 50% more for 32-byte key hash table > > I suggest you simply change the check: instead of validating this data > structure is a multiple of cache line size, validate it is a multiple of 64 > bytes. Any news please? The whole series is blocked for this patch. Should we expect a v7?
[dpdk-dev] [PATCH v6 9/9] table: align rte table hash structs for cache line size
> -Original Message- > From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com] > Sent: Thursday, September 8, 2016 10:36 AM > To: Gowrishankar Muthukrishnan > Cc: dev at dpdk.org; Dumitrescu, Cristian ; > Chao Zhu ; Richardson, Bruce > ; Ananyev, Konstantin > ; Pradeep > Subject: Re: [dpdk-dev] [PATCH v6 9/9] table: align rte table hash structs for > cache line size > > 2016-08-31 17:29, Dumitrescu, Cristian: > > From: Gowrishankar Muthukrishnan > > > rte table hash structs rte_bucket_4_8, rte_bucket_4_16 and > > > rte_bucket_4_32 have > > > to be cache aligned as required by their corresponding hash create > functions > > > rte_table_hash_create_key8_lru etc. > > > > Hi Gowrishankar, > > > > My understanding is you are trying to work around the check invoked by > the hash table create functions that verifies the size of the bucket header > structure is a multiple of the cache line, right? > > > > Given that the size of this structure is 1x, 2x or 3x times 64 bytes, the > > check > passes on IA CPUs (cache line of 64 bytes; explicit alignment to cache line > size > is not needed in order to make the size of the structure a multiple of cache > line), but not on PPC CPUs (cache line of 128 bytes), correct? > > > > I don't think your proposal provides the best way to fix this issue, since > your code leads to a considerable increase in the memory consumption used > per bucket in most cases: > > - 100% more memory for 8-byte key hash table > > - 0% more for 16-byte key hash table (code does not fix anything, > explicit alignment is not needed) > > - 50% more for 32-byte key hash table > > > > I suggest you simply change the check: instead of validating this data > structure is a multiple of cache line size, validate it is a multiple of 64 > bytes. > > Any news please? > The whole series is blocked for this patch. > Should we expect a v7? Yes, I think we should. Small fix for a considerable benefit.
[dpdk-dev] [PATCH v6 9/9] table: align rte table hash structs for cache line size
> -Original Message-
> From: Gowrishankar Muthukrishnan
> [mailto:gowrishankar.m at linux.vnet.ibm.com]
> Sent: Tuesday, August 16, 2016 11:28 AM
> To: dev at dpdk.org
> Cc: Chao Zhu ; Richardson, Bruce
> ; Ananyev, Konstantin
> ; Thomas Monjalon
> ; Dumitrescu, Cristian
> ; Pradeep
> Subject: [PATCH v6 9/9] table: align rte table hash structs for cache line
> size
>
> rte table hash structs rte_bucket_4_8, rte_bucket_4_16 and
> rte_bucket_4_32 have
> to be cache aligned as required by their corresponding hash create functions
> rte_table_hash_create_key8_lru etc.
>
> Signed-off-by: Gowrishankar Muthukrishnan
>
> ---
> lib/librte_table/rte_table_hash_key16.c | 4 ++--
> lib/librte_table/rte_table_hash_key32.c | 4 ++--
> lib/librte_table/rte_table_hash_key8.c | 2 +-
> 3 files changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/lib/librte_table/rte_table_hash_key16.c
> b/lib/librte_table/rte_table_hash_key16.c
> index b7e000f..2102326 100644
> --- a/lib/librte_table/rte_table_hash_key16.c
> +++ b/lib/librte_table/rte_table_hash_key16.c
> @@ -68,10 +68,10 @@ struct rte_bucket_4_16 {
> uint64_t next_valid;
>
> /* Cache line 1 */
> - uint64_t key[4][2];
> + uint64_t key[4][2] __rte_cache_aligned;
>
> /* Cache line 2 */
> - uint8_t data[0];
> + uint8_t data[0] __rte_cache_aligned;
> };
>
> struct rte_table_hash {
> diff --git a/lib/librte_table/rte_table_hash_key32.c
> b/lib/librte_table/rte_table_hash_key32.c
> index a7aba49..619f63a 100644
> --- a/lib/librte_table/rte_table_hash_key32.c
> +++ b/lib/librte_table/rte_table_hash_key32.c
> @@ -68,10 +68,10 @@ struct rte_bucket_4_32 {
> uint64_t next_valid;
>
> /* Cache lines 1 and 2 */
> - uint64_t key[4][4];
> + uint64_t key[4][4] __rte_cache_aligned;
>
> /* Cache line 3 */
> - uint8_t data[0];
> + uint8_t data[0] __rte_cache_aligned;
> };
>
> struct rte_table_hash {
> diff --git a/lib/librte_table/rte_table_hash_key8.c
> b/lib/librte_table/rte_table_hash_key8.c
> index e2e2bdc..4d5e0cd 100644
> --- a/lib/librte_table/rte_table_hash_key8.c
> +++ b/lib/librte_table/rte_table_hash_key8.c
> @@ -68,7 +68,7 @@ struct rte_bucket_4_8 {
> uint64_t key[4];
>
> /* Cache line 1 */
> - uint8_t data[0];
> + uint8_t data[0] __rte_cache_aligned;
> };
>
> struct rte_table_hash {
> --
> 1.9.1
Hi Gowrishankar,
My understanding is you are trying to work around the check invoked by the hash
table create functions that verifies the size of the bucket header structure is
a multiple of the cache line, right?
Given that the size of this structure is 1x, 2x or 3x times 64 bytes, the check
passes on IA CPUs (cache line of 64 bytes; explicit alignment to cache line
size is not needed in order to make the size of the structure a multiple of
cache line), but not on PPC CPUs (cache line of 128 bytes), correct?
I don't think your proposal provides the best way to fix this issue, since your
code leads to a considerable increase in the memory consumption used per bucket
in most cases:
- 100% more memory for 8-byte key hash table
- 0% more for 16-byte key hash table (code does not fix anything,
explicit alignment is not needed)
- 50% more for 32-byte key hash table
I suggest you simply change the check: instead of validating this data
structure is a multiple of cache line size, validate it is a multiple of 64
bytes.
Regards,
Cristian
[dpdk-dev] [PATCH v6 9/9] table: align rte table hash structs for cache line size
rte table hash structs rte_bucket_4_8, rte_bucket_4_16 and rte_bucket_4_32 have
to be cache aligned as required by their corresponding hash create functions
rte_table_hash_create_key8_lru etc.
Signed-off-by: Gowrishankar Muthukrishnan
---
lib/librte_table/rte_table_hash_key16.c | 4 ++--
lib/librte_table/rte_table_hash_key32.c | 4 ++--
lib/librte_table/rte_table_hash_key8.c | 2 +-
3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/lib/librte_table/rte_table_hash_key16.c
b/lib/librte_table/rte_table_hash_key16.c
index b7e000f..2102326 100644
--- a/lib/librte_table/rte_table_hash_key16.c
+++ b/lib/librte_table/rte_table_hash_key16.c
@@ -68,10 +68,10 @@ struct rte_bucket_4_16 {
uint64_t next_valid;
/* Cache line 1 */
- uint64_t key[4][2];
+ uint64_t key[4][2] __rte_cache_aligned;
/* Cache line 2 */
- uint8_t data[0];
+ uint8_t data[0] __rte_cache_aligned;
};
struct rte_table_hash {
diff --git a/lib/librte_table/rte_table_hash_key32.c
b/lib/librte_table/rte_table_hash_key32.c
index a7aba49..619f63a 100644
--- a/lib/librte_table/rte_table_hash_key32.c
+++ b/lib/librte_table/rte_table_hash_key32.c
@@ -68,10 +68,10 @@ struct rte_bucket_4_32 {
uint64_t next_valid;
/* Cache lines 1 and 2 */
- uint64_t key[4][4];
+ uint64_t key[4][4] __rte_cache_aligned;
/* Cache line 3 */
- uint8_t data[0];
+ uint8_t data[0] __rte_cache_aligned;
};
struct rte_table_hash {
diff --git a/lib/librte_table/rte_table_hash_key8.c
b/lib/librte_table/rte_table_hash_key8.c
index e2e2bdc..4d5e0cd 100644
--- a/lib/librte_table/rte_table_hash_key8.c
+++ b/lib/librte_table/rte_table_hash_key8.c
@@ -68,7 +68,7 @@ struct rte_bucket_4_8 {
uint64_t key[4];
/* Cache line 1 */
- uint8_t data[0];
+ uint8_t data[0] __rte_cache_aligned;
};
struct rte_table_hash {
--
1.9.1
