Re: [PATCH linux-next v3] mlx5: wrong page mask if CONFIG_ARCH_DMA_ADDR_T_64BIT enabled for 32Bit architectures

2015-04-15 Thread Honggang LI 
On Wed, Apr 15, 2015 at 08:27:23AM +0300, Or Gerlitz wrote:
> On 4/15/2015 6:19 AM, Honggang Li wrote:
> >Fix(bf0bf77 mlx5: Support communicating arbitrary host page size to firmware)
> 
> This isn't the way to write the fix note, do it that way
> 
> Fixes: bf0bf77f6519 ('mlx5: Support communicating arbitrary host
> page size to firmware')
> 

Fixed it and new patch (v4) had been sent out. Please review it. Hope it
is the final one.

thanks

> >
> >Signed-off-by: Honggang Li
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH linux-next v3] mlx5: wrong page mask if CONFIG_ARCH_DMA_ADDR_T_64BIT enabled for 32Bit architectures

2015-04-15 Thread Or Gerlitz 

On 4/15/2015 6:19 AM, Honggang Li wrote:

Fix(bf0bf77 mlx5: Support communicating arbitrary host page size to firmware)


This isn't the way to write the fix note, do it that way

Fixes: bf0bf77f6519 ('mlx5: Support communicating arbitrary host page 
size to firmware')




Signed-off-by: Honggang Li


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH linux-next v3] mlx5: wrong page mask if CONFIG_ARCH_DMA_ADDR_T_64BIT enabled for 32Bit architectures

2015-04-15 Thread Honggang LI 
On Wed, Apr 15, 2015 at 08:27:23AM +0300, Or Gerlitz wrote:
 On 4/15/2015 6:19 AM, Honggang Li wrote:
 Fix(bf0bf77 mlx5: Support communicating arbitrary host page size to firmware)
 
 This isn't the way to write the fix note, do it that way
 
 Fixes: bf0bf77f6519 ('mlx5: Support communicating arbitrary host
 page size to firmware')
 

Fixed it and new patch (v4) had been sent out. Please review it. Hope it
is the final one.

thanks

 
 Signed-off-by: Honggang Liho...@redhat.com
 
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH linux-next v3] mlx5: wrong page mask if CONFIG_ARCH_DMA_ADDR_T_64BIT enabled for 32Bit architectures

2015-04-15 Thread Or Gerlitz 

On 4/15/2015 6:19 AM, Honggang Li wrote:

Fix(bf0bf77 mlx5: Support communicating arbitrary host page size to firmware)


This isn't the way to write the fix note, do it that way

Fixes: bf0bf77f6519 ('mlx5: Support communicating arbitrary host page 
size to firmware')




Signed-off-by: Honggang Liho...@redhat.com


--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH linux-next v3] mlx5: wrong page mask if CONFIG_ARCH_DMA_ADDR_T_64BIT enabled for 32Bit architectures

2015-04-14 Thread Eli Cohen 
Acked-by: Eli Cohen 

On Wed, Apr 15, 2015 at 11:19:17AM +0800, Honggang Li wrote:
> If CONFIG_ARCH_DMA_ADDR_T_64BIT enabled for x86 systems and physical
> memory is more than 4GB, dma_map_page may return a valid memory
> address which greater than 0x. As a result, the mlx5 device page
> allocator RB tree will be initialized with valid addresses greater than
> 0xfff.
> 
> However, (addr & PAGE_MASK) set the high four bytes to zeros. So, it's
> impossible for the function, free_4k, to release the pages whose
> addresses greater than 4GB. Memory leaks. And mlx5_ib module can't
> release the pages when user try to remove the module, as a result,
> system hang.
> 
> [root@rdma05 root]# dmesg  | grep addr | head
> addr = 3fe384000
> addr & PAGE_MASK =  fe384000
> [root@rdma05 root]# rmmod mlx5_ib   < hang on
> 
> -- cosnole log -
> mlx5_ib :04:00.0: irq 138 for MSI/MSI-X
>   alloc irq_desc for 139 on node -1
>   alloc kstat_irqs on node -1
> mlx5_ib :04:00.0: irq 139 for MSI/MSI-X
> :04:00.0:free_4k:221:(pid 1519): page not found
> :04:00.0:free_4k:221:(pid 1519): page not found
> :04:00.0:free_4k:221:(pid 1519): page not found
> :04:00.0:free_4k:221:(pid 1519): page not found
> -- cosnole log -
> 
> Fix(bf0bf77 mlx5: Support communicating arbitrary host page size to
> firmware)
> 
> Signed-off-by: Honggang Li 
> ---
>  .../net/ethernet/mellanox/mlx5/core/pagealloc.c|   10 ++
>  1 files changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c 
> b/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
> index df22383..8a64542 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
> @@ -211,26 +211,28 @@ static int alloc_4k(struct mlx5_core_dev *dev, u64 
> *addr)
>   return 0;
>  }
>  
> +#define MLX5_U64_4K_PAGE_MASK ((~(u64)0U) << PAGE_SHIFT)
> +
>  static void free_4k(struct mlx5_core_dev *dev, u64 addr)
>  {
>   struct fw_page *fwp;
>   int n;
>  
> - fwp = find_fw_page(dev, addr & PAGE_MASK);
> + fwp = find_fw_page(dev, addr & MLX5_U64_4K_PAGE_MASK);
>   if (!fwp) {
>   mlx5_core_warn(dev, "page not found\n");
>   return;
>   }
>  
> - n = (addr & ~PAGE_MASK) >> MLX5_ADAPTER_PAGE_SHIFT;
> + n = (addr & ~MLX5_U64_4K_PAGE_MASK) >> MLX5_ADAPTER_PAGE_SHIFT;
>   fwp->free_count++;
>   set_bit(n, >bitmask);
>   if (fwp->free_count == MLX5_NUM_4K_IN_PAGE) {
>   rb_erase(>rb_node, >priv.page_root);
>   if (fwp->free_count != 1)
>   list_del(>list);
> - dma_unmap_page(>pdev->dev, addr & PAGE_MASK, PAGE_SIZE,
> -DMA_BIDIRECTIONAL);
> + dma_unmap_page(>pdev->dev, addr & MLX5_U64_4K_PAGE_MASK,
> +PAGE_SIZE, DMA_BIDIRECTIONAL);
>   __free_page(fwp->page);
>   kfree(fwp);
>   } else if (fwp->free_count == 1) {
> -- 
> 1.7.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[PATCH linux-next v3] mlx5: wrong page mask if CONFIG_ARCH_DMA_ADDR_T_64BIT enabled for 32Bit architectures

2015-04-14 Thread Honggang Li 
If CONFIG_ARCH_DMA_ADDR_T_64BIT enabled for x86 systems and physical
memory is more than 4GB, dma_map_page may return a valid memory
address which greater than 0x. As a result, the mlx5 device page
allocator RB tree will be initialized with valid addresses greater than
0xfff.

However, (addr & PAGE_MASK) set the high four bytes to zeros. So, it's
impossible for the function, free_4k, to release the pages whose
addresses greater than 4GB. Memory leaks. And mlx5_ib module can't
release the pages when user try to remove the module, as a result,
system hang.

[root@rdma05 root]# dmesg  | grep addr | head
addr = 3fe384000
addr & PAGE_MASK =  fe384000
[root@rdma05 root]# rmmod mlx5_ib   < hang on

-- cosnole log -
mlx5_ib :04:00.0: irq 138 for MSI/MSI-X
  alloc irq_desc for 139 on node -1
  alloc kstat_irqs on node -1
mlx5_ib :04:00.0: irq 139 for MSI/MSI-X
:04:00.0:free_4k:221:(pid 1519): page not found
:04:00.0:free_4k:221:(pid 1519): page not found
:04:00.0:free_4k:221:(pid 1519): page not found
:04:00.0:free_4k:221:(pid 1519): page not found
-- cosnole log -

Fix(bf0bf77 mlx5: Support communicating arbitrary host page size to
firmware)

Signed-off-by: Honggang Li 
---
 .../net/ethernet/mellanox/mlx5/core/pagealloc.c|   10 ++
 1 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c 
b/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
index df22383..8a64542 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
@@ -211,26 +211,28 @@ static int alloc_4k(struct mlx5_core_dev *dev, u64 *addr)
return 0;
 }
 
+#define MLX5_U64_4K_PAGE_MASK ((~(u64)0U) << PAGE_SHIFT)
+
 static void free_4k(struct mlx5_core_dev *dev, u64 addr)
 {
struct fw_page *fwp;
int n;
 
-   fwp = find_fw_page(dev, addr & PAGE_MASK);
+   fwp = find_fw_page(dev, addr & MLX5_U64_4K_PAGE_MASK);
if (!fwp) {
mlx5_core_warn(dev, "page not found\n");
return;
}
 
-   n = (addr & ~PAGE_MASK) >> MLX5_ADAPTER_PAGE_SHIFT;
+   n = (addr & ~MLX5_U64_4K_PAGE_MASK) >> MLX5_ADAPTER_PAGE_SHIFT;
fwp->free_count++;
set_bit(n, >bitmask);
if (fwp->free_count == MLX5_NUM_4K_IN_PAGE) {
rb_erase(>rb_node, >priv.page_root);
if (fwp->free_count != 1)
list_del(>list);
-   dma_unmap_page(>pdev->dev, addr & PAGE_MASK, PAGE_SIZE,
-  DMA_BIDIRECTIONAL);
+   dma_unmap_page(>pdev->dev, addr & MLX5_U64_4K_PAGE_MASK,
+  PAGE_SIZE, DMA_BIDIRECTIONAL);
__free_page(fwp->page);
kfree(fwp);
} else if (fwp->free_count == 1) {
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[PATCH linux-next v3] mlx5: wrong page mask if CONFIG_ARCH_DMA_ADDR_T_64BIT enabled for 32Bit architectures

2015-04-14 Thread Honggang Li 
If CONFIG_ARCH_DMA_ADDR_T_64BIT enabled for x86 systems and physical
memory is more than 4GB, dma_map_page may return a valid memory
address which greater than 0x. As a result, the mlx5 device page
allocator RB tree will be initialized with valid addresses greater than
0xfff.

However, (addr  PAGE_MASK) set the high four bytes to zeros. So, it's
impossible for the function, free_4k, to release the pages whose
addresses greater than 4GB. Memory leaks. And mlx5_ib module can't
release the pages when user try to remove the module, as a result,
system hang.

[root@rdma05 root]# dmesg  | grep addr | head
addr = 3fe384000
addr  PAGE_MASK =  fe384000
[root@rdma05 root]# rmmod mlx5_ib    hang on

-- cosnole log -
mlx5_ib :04:00.0: irq 138 for MSI/MSI-X
  alloc irq_desc for 139 on node -1
  alloc kstat_irqs on node -1
mlx5_ib :04:00.0: irq 139 for MSI/MSI-X
:04:00.0:free_4k:221:(pid 1519): page not found
:04:00.0:free_4k:221:(pid 1519): page not found
:04:00.0:free_4k:221:(pid 1519): page not found
:04:00.0:free_4k:221:(pid 1519): page not found
-- cosnole log -

Fix(bf0bf77 mlx5: Support communicating arbitrary host page size to
firmware)

Signed-off-by: Honggang Li ho...@redhat.com
---
 .../net/ethernet/mellanox/mlx5/core/pagealloc.c|   10 ++
 1 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c 
b/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
index df22383..8a64542 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
@@ -211,26 +211,28 @@ static int alloc_4k(struct mlx5_core_dev *dev, u64 *addr)
return 0;
 }
 
+#define MLX5_U64_4K_PAGE_MASK ((~(u64)0U)  PAGE_SHIFT)
+
 static void free_4k(struct mlx5_core_dev *dev, u64 addr)
 {
struct fw_page *fwp;
int n;
 
-   fwp = find_fw_page(dev, addr  PAGE_MASK);
+   fwp = find_fw_page(dev, addr  MLX5_U64_4K_PAGE_MASK);
if (!fwp) {
mlx5_core_warn(dev, page not found\n);
return;
}
 
-   n = (addr  ~PAGE_MASK)  MLX5_ADAPTER_PAGE_SHIFT;
+   n = (addr  ~MLX5_U64_4K_PAGE_MASK)  MLX5_ADAPTER_PAGE_SHIFT;
fwp-free_count++;
set_bit(n, fwp-bitmask);
if (fwp-free_count == MLX5_NUM_4K_IN_PAGE) {
rb_erase(fwp-rb_node, dev-priv.page_root);
if (fwp-free_count != 1)
list_del(fwp-list);
-   dma_unmap_page(dev-pdev-dev, addr  PAGE_MASK, PAGE_SIZE,
-  DMA_BIDIRECTIONAL);
+   dma_unmap_page(dev-pdev-dev, addr  MLX5_U64_4K_PAGE_MASK,
+  PAGE_SIZE, DMA_BIDIRECTIONAL);
__free_page(fwp-page);
kfree(fwp);
} else if (fwp-free_count == 1) {
-- 
1.7.1

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH linux-next v3] mlx5: wrong page mask if CONFIG_ARCH_DMA_ADDR_T_64BIT enabled for 32Bit architectures

2015-04-14 Thread Eli Cohen 
Acked-by: Eli Cohen e...@mellanox.com

On Wed, Apr 15, 2015 at 11:19:17AM +0800, Honggang Li wrote:
 If CONFIG_ARCH_DMA_ADDR_T_64BIT enabled for x86 systems and physical
 memory is more than 4GB, dma_map_page may return a valid memory
 address which greater than 0x. As a result, the mlx5 device page
 allocator RB tree will be initialized with valid addresses greater than
 0xfff.
 
 However, (addr  PAGE_MASK) set the high four bytes to zeros. So, it's
 impossible for the function, free_4k, to release the pages whose
 addresses greater than 4GB. Memory leaks. And mlx5_ib module can't
 release the pages when user try to remove the module, as a result,
 system hang.
 
 [root@rdma05 root]# dmesg  | grep addr | head
 addr = 3fe384000
 addr  PAGE_MASK =  fe384000
 [root@rdma05 root]# rmmod mlx5_ib    hang on
 
 -- cosnole log -
 mlx5_ib :04:00.0: irq 138 for MSI/MSI-X
   alloc irq_desc for 139 on node -1
   alloc kstat_irqs on node -1
 mlx5_ib :04:00.0: irq 139 for MSI/MSI-X
 :04:00.0:free_4k:221:(pid 1519): page not found
 :04:00.0:free_4k:221:(pid 1519): page not found
 :04:00.0:free_4k:221:(pid 1519): page not found
 :04:00.0:free_4k:221:(pid 1519): page not found
 -- cosnole log -
 
 Fix(bf0bf77 mlx5: Support communicating arbitrary host page size to
 firmware)
 
 Signed-off-by: Honggang Li ho...@redhat.com
 ---
  .../net/ethernet/mellanox/mlx5/core/pagealloc.c|   10 ++
  1 files changed, 6 insertions(+), 4 deletions(-)
 
 diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c 
 b/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
 index df22383..8a64542 100644
 --- a/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
 +++ b/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
 @@ -211,26 +211,28 @@ static int alloc_4k(struct mlx5_core_dev *dev, u64 
 *addr)
   return 0;
  }
  
 +#define MLX5_U64_4K_PAGE_MASK ((~(u64)0U)  PAGE_SHIFT)
 +
  static void free_4k(struct mlx5_core_dev *dev, u64 addr)
  {
   struct fw_page *fwp;
   int n;
  
 - fwp = find_fw_page(dev, addr  PAGE_MASK);
 + fwp = find_fw_page(dev, addr  MLX5_U64_4K_PAGE_MASK);
   if (!fwp) {
   mlx5_core_warn(dev, page not found\n);
   return;
   }
  
 - n = (addr  ~PAGE_MASK)  MLX5_ADAPTER_PAGE_SHIFT;
 + n = (addr  ~MLX5_U64_4K_PAGE_MASK)  MLX5_ADAPTER_PAGE_SHIFT;
   fwp-free_count++;
   set_bit(n, fwp-bitmask);
   if (fwp-free_count == MLX5_NUM_4K_IN_PAGE) {
   rb_erase(fwp-rb_node, dev-priv.page_root);
   if (fwp-free_count != 1)
   list_del(fwp-list);
 - dma_unmap_page(dev-pdev-dev, addr  PAGE_MASK, PAGE_SIZE,
 -DMA_BIDIRECTIONAL);
 + dma_unmap_page(dev-pdev-dev, addr  MLX5_U64_4K_PAGE_MASK,
 +PAGE_SIZE, DMA_BIDIRECTIONAL);
   __free_page(fwp-page);
   kfree(fwp);
   } else if (fwp-free_count == 1) {
 -- 
 1.7.1
 
 --
 To unsubscribe from this list: send the line unsubscribe linux-rdma in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/