[PATCH net] ibmvnic: Fix error recovery on login failure

[John Allen]

Testing has uncovered a failure case that is not handled properly. In the
event that a login fails and we are not able to recover on the spot, we
return 0 from do_reset, preventing any error recovery code from being
triggered.  Additionally, the state is set to "probed" meaning that when we
are able to trigger the error recovery, the driver always comes up in the
probed state. To handle the case properly, we need to return a failure code
here and set the adapter state to the state that we entered the reset in
indicating the state that we would like to come out of the recovery reset
in.

Signed-off-by: John Allen 
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index d0e196b..c1e23bb 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1825,8 +1825,8 @@ static int do_reset(struct ibmvnic_adapter *adapter,
 
rc = ibmvnic_login(netdev);
if (rc) {
-   adapter->state = VNIC_PROBED;
-   return 0;
+   adapter->state = reset_state;
+   return rc;
}
 
if (adapter->reset_reason == VNIC_RESET_CHANGE_PARAM ||

[PATCH net] ibmvnic: Disable irqs before exiting reset from closed state

[John Allen]

When the driver is closed, all the associated irqs are disabled. In the
event that the driver exits a reset in the closed state, we should be
consistent with the state we are in directly after a close. So before we
exit the reset routine, all irqs should be disabled as well. This will
prevent the irqs from being enabled twice in this case and reporting a
number of noisy warning traces.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 0389a7a..b492af6 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1167,19 +1167,11 @@ static void clean_tx_pools(struct ibmvnic_adapter 
*adapter)
}
 }

-static void ibmvnic_cleanup(struct net_device *netdev)
+static void ibmvnic_disable_irqs(struct ibmvnic_adapter *adapter)
 {
-   struct ibmvnic_adapter *adapter = netdev_priv(netdev);
+   struct net_device *netdev = adapter->netdev;
int i;

-   /* ensure that transmissions are stopped if called by do_reset */
-   if (adapter->resetting)
-   netif_tx_disable(netdev);
-   else
-   netif_tx_stop_all_queues(netdev);
-
-   ibmvnic_napi_disable(adapter);
-
if (adapter->tx_scrq) {
for (i = 0; i < adapter->req_tx_queues; i++)
if (adapter->tx_scrq[i]->irq) {
@@ -1198,6 +1190,21 @@ static void ibmvnic_cleanup(struct net_device *netdev)
}
}
}
+}
+
+static void ibmvnic_cleanup(struct net_device *netdev)
+{
+   struct ibmvnic_adapter *adapter = netdev_priv(netdev);
+
+   /* ensure that transmissions are stopped if called by do_reset */
+   if (adapter->resetting)
+   netif_tx_disable(netdev);
+   else
+   netif_tx_stop_all_queues(netdev);
+
+   ibmvnic_napi_disable(adapter);
+   ibmvnic_disable_irqs(adapter);
+
clean_rx_pools(adapter);
clean_tx_pools(adapter);
 }
@@ -1772,6 +1779,7 @@ static int do_reset(struct ibmvnic_adapter *adapter,
}
}

+   ibmvnic_disable_irqs(adapter);
adapter->state = VNIC_CLOSED;

if (reset_state == VNIC_CLOSED)

[PATCH net-next] ibmvnic: Fix reset return from closed state

[John Allen]

The case in which we handle a reset from the state where the device is
closed seems to be bugged for all types of reset. For most types of reset
we currently exit the reset routine correctly, but don't set the state to
indicate that we are back in the "closed" state. For some specific cases,
we don't exit the reset routine at all and resetting will cause a closed
device to be opened.

This patch fixes the problem by unconditionally checking the reset_state
and correctly setting the adapter state before returning.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 7be4b06..b103a7e 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1714,12 +1714,14 @@ static int do_reset(struct ibmvnic_adapter *adapter,
rc = reset_rx_pools(adapter);
if (rc)
return rc;
-
-   if (reset_state == VNIC_CLOSED)
-   return 0;
}
}

+   adapter->state = VNIC_CLOSED;
+
+   if (reset_state == VNIC_CLOSED)
+   return 0;
+
rc = __ibmvnic_open(netdev);
if (rc) {
if (list_empty(>rwi_list))

Re: [PATCH net-next v2] ibmvnic: Bail from ibmvnic_open if driver is already open

[John Allen]

On 03/12/2018 03:10 PM, Andrew Lunn wrote:
>> The problem here is that our routine to change the mtu does a full reset on
>> the driver meaning that in the process we go from effectively "open" to
>> "closed" to "open" again.
>>
>> Consider the scenario where we change the mtu by running "ifdown 
>> ",
>> editing the ifcfg file with the new mtu, and finally running "ifup 
>> > In this case, we call ibmvnic_close from the ifdown and as a result of the 
>> ifup,
>> we do the reset for the mtu change (which puts us back in the "open" state) 
>> and
>> call ibmvnic_open. After the reset, we are already in the "open" state and 
>> the
>> following call to open is redundant.
> 
> Hi John
> 
> So you are saying "ip link set mtu 4242 eth1" on a down interface will
> put it up. That i would say is broken. You should be fixing this,
> rather than papering over the cracks.

That's a good point. I'll work on a fix to address that.

-John

> 
>   Andrew
> 
> 

Re: [PATCH net-next v2] ibmvnic: Bail from ibmvnic_open if driver is already open

[John Allen]

On 03/12/2018 02:33 PM, Andrew Lunn wrote:
> On Mon, Mar 12, 2018 at 02:19:52PM -0500, John Allen wrote:
>> If the driver is already in the "open" state, don't attempt the procedure
>> for opening the driver.
>>
>> Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
>> ---
>> v2: Unlock reset_lock mutex before returning.
>>
>> diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
>> b/drivers/net/ethernet/ibm/ibmvnic.c
>> index 7be4b06..9a5e8ac 100644
>> --- a/drivers/net/ethernet/ibm/ibmvnic.c
>> +++ b/drivers/net/ethernet/ibm/ibmvnic.c
>> @@ -1057,6 +1057,11 @@ static int ibmvnic_open(struct net_device *netdev)
>>
>>  mutex_lock(>reset_lock);
>>
>> +if (adapter->state == VNIC_OPEN) {
>> +mutex_unlock(>reset_lock);
>> +return 0;
>> +}
> 
> Hi John
> 
> This looks better.
> 
> But how did you get ibmvnic_open() to be called twice? Is the actual
> issue that __ibmvnic_close() does not kill off the
> adapter->ibmvnic_reset workqueue, so that a reset can happen after
> close() has been called?

The problem here is that our routine to change the mtu does a full reset on
the driver meaning that in the process we go from effectively "open" to
"closed" to "open" again.

Consider the scenario where we change the mtu by running "ifdown ",
editing the ifcfg file with the new mtu, and finally running "ifup <interface".
In this case, we call ibmvnic_close from the ifdown and as a result of the ifup,
we do the reset for the mtu change (which puts us back in the "open" state) and
call ibmvnic_open. After the reset, we are already in the "open" state and the
following call to open is redundant. The check in ibmvnic_open to see if we're
in the open state is really just insurance to guard against any case like this
where we're in the open state from the driver perspective and something decides
to trigger the drivers open routine.

-John

> 
>  Andrew
> 
> 

[PATCH net-next v2] ibmvnic: Bail from ibmvnic_open if driver is already open

[John Allen]

If the driver is already in the "open" state, don't attempt the procedure
for opening the driver.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
v2: Unlock reset_lock mutex before returning.

diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 7be4b06..9a5e8ac 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1057,6 +1057,11 @@ static int ibmvnic_open(struct net_device *netdev)

mutex_lock(>reset_lock);

+   if (adapter->state == VNIC_OPEN) {
+   mutex_unlock(>reset_lock);
+   return 0;
+   }
+
if (adapter->state != VNIC_CLOSED) {
rc = ibmvnic_login(netdev);
if (rc) {

Re: [PATCH net-next] ibmvnic: Bail from ibmvnic_open if driver is already open

[John Allen]

On 03/12/2018 02:01 PM, Andrew Lunn wrote:
> On Mon, Mar 12, 2018 at 01:47:51PM -0500, John Allen wrote:
>> If the driver is already in the "open" state, don't attempt the procedure
>> for opening the driver.
>>
>> Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
>> ---
>> diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
>> b/drivers/net/ethernet/ibm/ibmvnic.c
>> index 7be4b06..98c4f75 100644
>> --- a/drivers/net/ethernet/ibm/ibmvnic.c
>> +++ b/drivers/net/ethernet/ibm/ibmvnic.c
>> @@ -1057,6 +1057,9 @@ static int ibmvnic_open(struct net_device *netdev)
>>
>>  mutex_lock(>reset_lock);
>>
>> +if (adapter->state == VNIC_OPEN)
>> +return 0;
>> +
> 
> I've not looked at the actual code, but what about the lock it just
> took and does not unlock before returning?
Ah yes, I missed that. I will update and send a new version.

> 
>   Andrew
> 

[PATCH net-next] ibmvnic: Bail from ibmvnic_open if driver is already open

[John Allen]

If the driver is already in the "open" state, don't attempt the procedure
for opening the driver.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 7be4b06..98c4f75 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1057,6 +1057,9 @@ static int ibmvnic_open(struct net_device *netdev)

mutex_lock(>reset_lock);

+   if (adapter->state == VNIC_OPEN)
+   return 0;
+
if (adapter->state != VNIC_CLOSED) {
rc = ibmvnic_login(netdev);
if (rc) {

[PATCH net] ibmvnic: Fix rx queue cleanup for non-fatal resets

[John Allen]

At some point, a check was added to exit the polling routine during resets.
This makes sense for most reset conditions, but for a non-fatal error, we
expect the polling routine to continue running to properly clean up the rx
queues. This patch checks if we are performing a non-fatal reset and if we
are, continues normal polling operation.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 8c3058d5d191..2a26b2ece7fe 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1831,7 +1831,8 @@ static int ibmvnic_poll(struct napi_struct *napi, int 
budget)
u16 offset;
u8 flags = 0;

-   if (unlikely(adapter->resetting)) {
+   if (unlikely(adapter->resetting &&
+adapter->reset_reason != VNIC_RESET_NON_FATAL)) {
enable_scrq_irq(adapter, adapter->rx_scrq[scrq_num]);
napi_complete_done(napi, frames_processed);
return frames_processed;

[PATCH net v2 3/3] ibmvnic: Allocate and request vpd in init_resources

[John Allen]

In reset events in which our memory allocations need to be reallocated,
VPD data is being freed, but never reallocated. This can cause issues if
we later attempt to access that memory or reset and attempt to free the
memory. This patch moves the allocation of the VPD data to init_resources
so that it will be symmetrically freed during release resources.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index bb56460..f0dbb76 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -867,7 +867,7 @@ static int ibmvnic_get_vpd(struct ibmvnic_adapter *adapter)
if (adapter->vpd->buff)
len = adapter->vpd->len;

-   reinit_completion(>fw_done);
+   init_completion(>fw_done);
crq.get_vpd_size.first = IBMVNIC_CRQ_CMD;
crq.get_vpd_size.cmd = GET_VPD_SIZE;
ibmvnic_send_crq(adapter, );
@@ -929,6 +929,13 @@ static int init_resources(struct ibmvnic_adapter *adapter)
if (!adapter->vpd)
return -ENOMEM;

+   /* Vital Product Data (VPD) */
+   rc = ibmvnic_get_vpd(adapter);
+   if (rc) {
+   netdev_err(netdev, "failed to initialize Vital Product Data 
(VPD)\n");
+   return rc;
+   }
+
adapter->map_id = 1;
adapter->napi = kcalloc(adapter->req_rx_queues,
sizeof(struct napi_struct), GFP_KERNEL);
@@ -1002,7 +1009,7 @@ static int __ibmvnic_open(struct net_device *netdev)
 static int ibmvnic_open(struct net_device *netdev)
 {
struct ibmvnic_adapter *adapter = netdev_priv(netdev);
-   int rc, vpd;
+   int rc;

mutex_lock(>reset_lock);

@@ -1030,11 +1037,6 @@ static int ibmvnic_open(struct net_device *netdev)
rc = __ibmvnic_open(netdev);
netif_carrier_on(netdev);

-   /* Vital Product Data (VPD) */
-   vpd = ibmvnic_get_vpd(adapter);
-   if (vpd)
-   netdev_err(netdev, "failed to initialize Vital Product Data 
(VPD)\n");
-
mutex_unlock(>reset_lock);

return rc;

[PATCH net v2 2/3] ibmvnic: Revert to previous mtu when unsupported value requested

[John Allen]

If we request an unsupported mtu value, the vnic server will suggest a
different value. Currently we take the suggested value without question
and login with that value. However, the behavior doesn't seem completely
sane as attempting to change the mtu to some specific value will change
the mtu to some completely different value most of the time. This patch
fixes the issue by logging in with the previously used mtu value and
printing an error message saying that the given mtu is unsupported.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index f8f1396..bb56460 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -3608,7 +3608,17 @@ static void handle_request_cap_rsp(union ibmvnic_crq 
*crq,
 *req_value,
 (long int)be64_to_cpu(crq->request_capability_rsp.
   number), name);
-   *req_value = be64_to_cpu(crq->request_capability_rsp.number);
+
+   if (be16_to_cpu(crq->request_capability_rsp.capability) ==
+   REQ_MTU) {
+   pr_err("mtu of %llu is not supported. Reverting.\n",
+  *req_value);
+   *req_value = adapter->fallback.mtu;
+   } else {
+   *req_value =
+   be64_to_cpu(crq->request_capability_rsp.number);
+   }
+
ibmvnic_send_req_caps(adapter, 1);
return;
default:

[PATCH net v2 1/3] ibmvnic: Modify buffer size and number of queues on failover

[John Allen]

Using newer backing devices can cause the required padding at the end of
buffer as well as the number of queues to change after a failover.
Since we currently assume that these values never change, after a
failover to a backing device with different capabilities, we can get
errors from the vnic server, attempt to free long term buffers that are
no longer there, or not free long term buffers that should be freed.

This patch resolves the issue by checking whether any of these values
change, and if so perform the necessary re-allocations.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
v2: Added the line to free the long term buff in reset rx pools which
caused me to hit a couple of other problems. On a failover, the number
of queues can also change which doesn't get updated in the current
reset code. Added some extra logic in do reset to check for changed
number of queues and added some logic to release pools to ensure that
it is only releasing pools that were allocated in the init pools
routines.
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 461014b7ccdd..12559c63c226 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -411,6 +411,10 @@ static int reset_rx_pools(struct ibmvnic_adapter *adapter)
struct ibmvnic_rx_pool *rx_pool;
int rx_scrqs;
int i, j, rc;
+   u64 *size_array;
+
+   size_array = (u64 *)((u8 *)(adapter->login_rsp_buf) +
+   be32_to_cpu(adapter->login_rsp_buf->off_rxadd_buff_size));

rx_scrqs = be32_to_cpu(adapter->login_rsp_buf->num_rxadd_subcrqs);
for (i = 0; i < rx_scrqs; i++) {
@@ -418,7 +422,17 @@ static int reset_rx_pools(struct ibmvnic_adapter *adapter)

netdev_dbg(adapter->netdev, "Re-setting rx_pool[%d]\n", i);

-   rc = reset_long_term_buff(adapter, _pool->long_term_buff);
+   if (rx_pool->buff_size != be64_to_cpu(size_array[i])) {
+   free_long_term_buff(adapter, _pool->long_term_buff);
+   rx_pool->buff_size = be64_to_cpu(size_array[i]);
+   alloc_long_term_buff(adapter, _pool->long_term_buff,
+rx_pool->size *
+rx_pool->buff_size);
+   } else {
+   rc = reset_long_term_buff(adapter,
+ _pool->long_term_buff);
+   }
+
if (rc)
return rc;

@@ -440,14 +454,12 @@ static int reset_rx_pools(struct ibmvnic_adapter *adapter)
 static void release_rx_pools(struct ibmvnic_adapter *adapter)
 {
struct ibmvnic_rx_pool *rx_pool;
-   int rx_scrqs;
int i, j;

if (!adapter->rx_pool)
return;

-   rx_scrqs = be32_to_cpu(adapter->login_rsp_buf->num_rxadd_subcrqs);
-   for (i = 0; i < rx_scrqs; i++) {
+   for (i = 0; i < adapter->num_active_rx_pools; i++) {
rx_pool = >rx_pool[i];

netdev_dbg(adapter->netdev, "Releasing rx_pool[%d]\n", i);
@@ -470,6 +482,7 @@ static void release_rx_pools(struct ibmvnic_adapter 
*adapter)

kfree(adapter->rx_pool);
adapter->rx_pool = NULL;
+   adapter->num_active_rx_pools = 0;
 }

 static int init_rx_pools(struct net_device *netdev)
@@ -494,6 +507,8 @@ static int init_rx_pools(struct net_device *netdev)
return -1;
}

+   adapter->num_active_rx_pools = 0;
+
for (i = 0; i < rxadd_subcrqs; i++) {
rx_pool = >rx_pool[i];

@@ -537,6 +552,8 @@ static int init_rx_pools(struct net_device *netdev)
rx_pool->next_free = 0;
}

+   adapter->num_active_rx_pools = rxadd_subcrqs;
+
return 0;
 }

@@ -587,13 +604,12 @@ static void release_vpd_data(struct ibmvnic_adapter 
*adapter)
 static void release_tx_pools(struct ibmvnic_adapter *adapter)
 {
struct ibmvnic_tx_pool *tx_pool;
-   int i, tx_scrqs;
+   int i;

if (!adapter->tx_pool)
return;

-   tx_scrqs = be32_to_cpu(adapter->login_rsp_buf->num_txsubm_subcrqs);
-   for (i = 0; i < tx_scrqs; i++) {
+   for (i = 0; i < adapter->num_active_tx_pools; i++) {
netdev_dbg(adapter->netdev, "Releasing tx_pool[%d]\n", i);
tx_pool = >tx_pool[i];
kfree(tx_pool->tx_buff);
@@ -604,6 +620,7 @@ static void release_tx_pools(struct ibmvnic_adapter 
*adapter)

kfree(adapter->tx_pool);
adapter->tx_pool = NULL;
+   adapter->num_active_tx_pools = 0;
 }

 static int init_tx_pools(struct net_device *netdev)
@@ -620,6 +637,8 @@ static int init_tx_pools(struct net_device *netdev)
if (!adapter->tx_pool)
   

[PATCH net v2 0/3] ibmvnic: Reset behavior fixes

[John Allen]

This patchset fixes a number of issues related to ibmvnic reset uncovered
from testing new Power9 machines with Everglades adapters and the new
functionality to change mtu and other parameters in the driver.

Changes since v1:
-In patch 1/3, added the line to free the long term buffers before
allocating a new one. This change inadvertently uncovered the problem
that the number of queues can change after a failover as well. To fix
this, we check whether or not the number of queues has changed in
do_reset and if they have, we do a full release and init of the queues.
-In patch 1/3, added variables to the adapter struct to track how
many rx/tx pools have actually been allocated and modify the release
pools routines to use these values rather than the possibly incorrect
req_rx/tx_queues values.

John Allen (3):
  ibmvnic: Modify buffer size and number of queues on failover
  ibmvnic: Revert to previous mtu when unsupported value requested
  ibmvnic: Allocate and request vpd in init_resources

 drivers/net/ethernet/ibm/ibmvnic.c | 73 ++
 drivers/net/ethernet/ibm/ibmvnic.h |  2 ++
 2 files changed, 60 insertions(+), 15 deletions(-)

-- 
2.9.5

Re: [PATCH 1/3 net] ibmvnic: Modify buffer size on failover

[John Allen]

On 01/15/2018 03:11 PM, John Allen wrote:
> Using newer backing devices can cause the required padding at the end of
> rx buffers to change. Currently we assume that the size of buffers will
> never change, but in the case that we failover from a backing device with
> smaller padding requirement to a backing device with a larger padding
> requirement, the vnic server will fail to post rx buffers due to
> inadequate space in our rx pool. This patch fixes the issue by checking
> whether or not the buffer size has changed on a reset and if it has,
> reallocate the buffer.
> 
> Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
> ---
> diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
> b/drivers/net/ethernet/ibm/ibmvnic.c
> index b676fa9..5b68a28 100644
> --- a/drivers/net/ethernet/ibm/ibmvnic.c
> +++ b/drivers/net/ethernet/ibm/ibmvnic.c
> @@ -412,13 +412,25 @@ static int reset_rx_pools(struct ibmvnic_adapter 
> *adapter)
>   int rx_scrqs;
>   int i, j, rc;
> 
> + size_array = (u64 *)((u8 *)(adapter->login_rsp_buf) +
> + be32_to_cpu(adapter->login_rsp_buf->off_rxadd_buff_size));
> +
>   rx_scrqs = be32_to_cpu(adapter->login_rsp_buf->num_rxadd_subcrqs);
>   for (i = 0; i < rx_scrqs; i++) {
>   rx_pool = >rx_pool[i];
> 
>   netdev_dbg(adapter->netdev, "Re-setting rx_pool[%d]\n", i);
> 
> - rc = reset_long_term_buff(adapter, _pool->long_term_buff);
> + if (rx_pool->buff_size != be64_to_cpu(size_array[i])) {
> + rx_pool->buff_size = be64_to_cpu(size_array[i]);
> + rc = alloc_long_term_buff(adapter,
> +   _pool->long_term_buff,
> +   rx_pool->size *
> +   rx_pool->buff_size);

We should be freeing the long_term_buff here before allocating a new one.
I will send a new version with the changes. Please ignore this version.

-John

> + } else {
> + rc = reset_long_term_buff(adapter,
> +   _pool->long_term_buff);
> + }
>   if (rc)
>   return rc;
> 

[PATCH 3/3 net] ibmvnic: Allocate and request vpd in init_resources

[John Allen]

In reset events in which our memory allocations need to be reallocated,
VPD data is being freed, but never reallocated. This can cause issues if
we later attempt to access that memory or reset and attempt to free the
memory. This patch moves the allocation of the VPD data to init_resources
so that it will be symmetrically freed during release resources.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index bb56460..f0dbb76 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -867,7 +867,7 @@ static int ibmvnic_get_vpd(struct ibmvnic_adapter *adapter)
if (adapter->vpd->buff)
len = adapter->vpd->len;

-   reinit_completion(>fw_done);
+   init_completion(>fw_done);
crq.get_vpd_size.first = IBMVNIC_CRQ_CMD;
crq.get_vpd_size.cmd = GET_VPD_SIZE;
ibmvnic_send_crq(adapter, );
@@ -929,6 +929,13 @@ static int init_resources(struct ibmvnic_adapter *adapter)
if (!adapter->vpd)
return -ENOMEM;

+   /* Vital Product Data (VPD) */
+   rc = ibmvnic_get_vpd(adapter);
+   if (rc) {
+   netdev_err(netdev, "failed to initialize Vital Product Data 
(VPD)\n");
+   return rc;
+   }
+
adapter->map_id = 1;
adapter->napi = kcalloc(adapter->req_rx_queues,
sizeof(struct napi_struct), GFP_KERNEL);
@@ -1002,7 +1009,7 @@ static int __ibmvnic_open(struct net_device *netdev)
 static int ibmvnic_open(struct net_device *netdev)
 {
struct ibmvnic_adapter *adapter = netdev_priv(netdev);
-   int rc, vpd;
+   int rc;

mutex_lock(>reset_lock);

@@ -1030,11 +1037,6 @@ static int ibmvnic_open(struct net_device *netdev)
rc = __ibmvnic_open(netdev);
netif_carrier_on(netdev);

-   /* Vital Product Data (VPD) */
-   vpd = ibmvnic_get_vpd(adapter);
-   if (vpd)
-   netdev_err(netdev, "failed to initialize Vital Product Data 
(VPD)\n");
-
mutex_unlock(>reset_lock);

return rc;

[PATCH 2/3 net] ibmvnic: Revert to previous mtu when unsupported value requested

[John Allen]

If we request an unsupported mtu value, the vnic server will suggest a
different value. Currently we take the suggested value without question
and login with that value. However, the behavior doesn't seem completely
sane as attempting to change the mtu to some specific value will change
the mtu to some completely different value most of the time. This patch
fixes the issue by logging in with the previously used mtu value and
printing an error message saying that the given mtu is unsupported.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index f8f1396..bb56460 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -3608,7 +3608,17 @@ static void handle_request_cap_rsp(union ibmvnic_crq 
*crq,
 *req_value,
 (long int)be64_to_cpu(crq->request_capability_rsp.
   number), name);
-   *req_value = be64_to_cpu(crq->request_capability_rsp.number);
+
+   if (be16_to_cpu(crq->request_capability_rsp.capability) ==
+   REQ_MTU) {
+   pr_err("mtu of %llu is not supported. Reverting.\n",
+  *req_value);
+   *req_value = adapter->fallback.mtu;
+   } else {
+   *req_value =
+   be64_to_cpu(crq->request_capability_rsp.number);
+   }
+
ibmvnic_send_req_caps(adapter, 1);
return;
default:

[PATCH 1/3 net] ibmvnic: Modify buffer size on failover

[John Allen]

Using newer backing devices can cause the required padding at the end of
rx buffers to change. Currently we assume that the size of buffers will
never change, but in the case that we failover from a backing device with
smaller padding requirement to a backing device with a larger padding
requirement, the vnic server will fail to post rx buffers due to
inadequate space in our rx pool. This patch fixes the issue by checking
whether or not the buffer size has changed on a reset and if it has,
reallocate the buffer.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index b676fa9..5b68a28 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -412,13 +412,25 @@ static int reset_rx_pools(struct ibmvnic_adapter *adapter)
int rx_scrqs;
int i, j, rc;

+   size_array = (u64 *)((u8 *)(adapter->login_rsp_buf) +
+   be32_to_cpu(adapter->login_rsp_buf->off_rxadd_buff_size));
+
rx_scrqs = be32_to_cpu(adapter->login_rsp_buf->num_rxadd_subcrqs);
for (i = 0; i < rx_scrqs; i++) {
rx_pool = >rx_pool[i];

netdev_dbg(adapter->netdev, "Re-setting rx_pool[%d]\n", i);

-   rc = reset_long_term_buff(adapter, _pool->long_term_buff);
+   if (rx_pool->buff_size != be64_to_cpu(size_array[i])) {
+   rx_pool->buff_size = be64_to_cpu(size_array[i]);
+   rc = alloc_long_term_buff(adapter,
+ _pool->long_term_buff,
+ rx_pool->size *
+ rx_pool->buff_size);
+   } else {
+   rc = reset_long_term_buff(adapter,
+ _pool->long_term_buff);
+   }
if (rc)
return rc;

[PATCH 0/3 net] ibmvnic: Reset behavior fixes

[John Allen]

This patchset fixes a number of issues related to ibmvnic reset uncovered
from testing new Power9 machines with Everglades adapters and the new
functionality to change mtu and other parameters in the driver.

This set should be applied after Thomas Falcon's patch,
"[PATCH net] ibmvnic: Fix pending MAC address changes" which is currently
submitted awaiting acceptance to the net tree.

John Allen (3):
  ibmvnic: Modify buffer size on failover
  ibmvnic: Revert to previous mtu when unsupported value requested
  ibmvnic: Allocate and request vpd in init_resources

 drivers/net/ethernet/ibm/ibmvnic.c | 42 ++
 1 file changed, 33 insertions(+), 9 deletions(-)

-- 
1.8.3.1

Re: [PATCH net] ibmvnic: Fix pending MAC address changes

[John Allen]

On 01/10/2018 07:39 PM, Thomas Falcon wrote:
> Due to architecture limitations, the IBM VNIC client driver is unable
> to perform MAC address changes unless the device has "logged in" to
> its backing device. Currently, pending MAC changes are handled before
> login, resulting in an error and failure to change the MAC address.
> Moving that chunk to the end of the ibmvnic_login function, when we are
> sure that it was successful, fixes that.
> 
> The MAC address can be changed when the device is up or down, so
> only check if the device is in a "PROBED" state before setting the
> MAC address.
> 
> Fixes: c26eba03e407 ("ibmvnic: Update reset infrastructure to support tunable 
> parameters")
> Signed-off-by: Thomas Falcon <tlfal...@linux.vnet.ibm.com>
> ---

Reviewed-by: John Allen <jal...@linux.vnet.ibm.com>

>  drivers/net/ethernet/ibm/ibmvnic.c | 13 +++--
>  1 file changed, 7 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
> b/drivers/net/ethernet/ibm/ibmvnic.c
> index 6911b7c..b676fa9 100644
> --- a/drivers/net/ethernet/ibm/ibmvnic.c
> +++ b/drivers/net/ethernet/ibm/ibmvnic.c
> @@ -757,6 +757,12 @@ static int ibmvnic_login(struct net_device *netdev)
>   }
>   } while (adapter->renegotiate);
> 
> + /* handle pending MAC address changes after successful login */
> + if (adapter->mac_change_pending) {
> + __ibmvnic_set_mac(netdev, >desired.mac);
> + adapter->mac_change_pending = false;
> + }
> +
>   return 0;
>  }
> 
> @@ -994,11 +1000,6 @@ static int ibmvnic_open(struct net_device *netdev)
> 
>   mutex_lock(>reset_lock);
> 
> - if (adapter->mac_change_pending) {
> - __ibmvnic_set_mac(netdev, >desired.mac);
> - adapter->mac_change_pending = false;
> - }
> -
>   if (adapter->state != VNIC_CLOSED) {
>   rc = ibmvnic_login(netdev);
>   if (rc) {
> @@ -1532,7 +1533,7 @@ static int ibmvnic_set_mac(struct net_device *netdev, 
> void *p)
>   struct ibmvnic_adapter *adapter = netdev_priv(netdev);
>   struct sockaddr *addr = p;
> 
> - if (adapter->state != VNIC_OPEN) {
> + if (adapter->state == VNIC_PROBED) {
>   memcpy(>desired.mac, addr, sizeof(struct sockaddr));
>   adapter->mac_change_pending = true;
>   return 0;
> 

[PATCH v3 net-next 2/2] ibmvnic: Fix failover error path for non-fatal resets

[John Allen]

For all non-fatal reset conditions, the hypervisor will send a failover when
we attempt to initialize the crq and the vnic client is expected to handle
that failover instead of the existing non-fatal reset. To handle this, we
need to return from init with a return code that indicates that we have hit
this case.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 3d02801..d0cff28 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1507,7 +1507,7 @@ static int do_reset(struct ibmvnic_adapter *adapter,

rc = ibmvnic_init(adapter);
if (rc)
-   return 0;
+   return IBMVNIC_INIT_FAILED;

/* If the adapter was in PROBE state prior to the reset,
 * exit here.
@@ -1610,7 +1610,7 @@ static void __ibmvnic_reset(struct work_struct *work)
while (rwi) {
rc = do_reset(adapter, rwi, reset_state);
kfree(rwi);
-   if (rc)
+   if (rc && rc != IBMVNIC_INIT_FAILED)
break;

rwi = get_next_rwi(adapter);
diff --git a/drivers/net/ethernet/ibm/ibmvnic.h 
b/drivers/net/ethernet/ibm/ibmvnic.h
index 27107f3..4670af8 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.h
+++ b/drivers/net/ethernet/ibm/ibmvnic.h
@@ -30,6 +30,8 @@
 #define IBMVNIC_DRIVER_VERSION "1.0.1"
 #define IBMVNIC_INVALID_MAP-1
 #define IBMVNIC_STATS_TIMEOUT  1
+#define IBMVNIC_INIT_FAILED2
+
 /* basic structures plus 100 2k buffers */
 #define IBMVNIC_IO_ENTITLEMENT_DEFAULT 610305

[PATCH v3 net-next 1/2] ibmvnic: Update reset infrastructure to support tunable parameters

[John Allen]

Update ibmvnic reset infrastructure to include a new reset option that will
allow changing of tunable parameters. There currently is no way to request
different capabilities from the vnic server on the fly so this patch
achieves this by resetting the driver and attempting to log in with the
requested changes. If the reset operation fails, the old values of the
tunable parameters are stored in the "fallback" struct and we attempt to
login with the fallback values.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 11eba82..3d02801 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -115,6 +115,7 @@ static union sub_crq *ibmvnic_next_scrq(struct 
ibmvnic_adapter *,
 static int init_sub_crq_irqs(struct ibmvnic_adapter *adapter);
 static int ibmvnic_init(struct ibmvnic_adapter *);
 static void release_crq_queue(struct ibmvnic_adapter *);
+static int __ibmvnic_set_mac(struct net_device *netdev, struct sockaddr *p);

 struct ibmvnic_stat {
char name[ETH_GSTRING_LEN];
@@ -926,6 +927,11 @@ static int ibmvnic_open(struct net_device *netdev)

mutex_lock(>reset_lock);

+   if (adapter->mac_change_pending) {
+   __ibmvnic_set_mac(netdev, >desired.mac);
+   adapter->mac_change_pending = false;
+   }
+
if (adapter->state != VNIC_CLOSED) {
rc = ibmvnic_login(netdev);
if (rc) {
@@ -1426,7 +1432,7 @@ static void ibmvnic_set_multi(struct net_device *netdev)
}
 }

-static int ibmvnic_set_mac(struct net_device *netdev, void *p)
+static int __ibmvnic_set_mac(struct net_device *netdev, struct sockaddr *p)
 {
struct ibmvnic_adapter *adapter = netdev_priv(netdev);
struct sockaddr *addr = p;
@@ -1444,6 +1450,22 @@ static int ibmvnic_set_mac(struct net_device *netdev, 
void *p)
return 0;
 }

+static int ibmvnic_set_mac(struct net_device *netdev, void *p)
+{
+   struct ibmvnic_adapter *adapter = netdev_priv(netdev);
+   struct sockaddr *addr = p;
+
+   if (adapter->state != VNIC_OPEN) {
+   memcpy(>desired.mac, addr, sizeof(struct sockaddr));
+   adapter->mac_change_pending = true;
+   return 0;
+   }
+
+   __ibmvnic_set_mac(netdev, addr);
+
+   return 0;
+}
+
 /**
  * do_reset returns zero if we are able to keep processing reset events, or
  * non-zero if we hit a fatal error and must halt.
@@ -1470,6 +1492,13 @@ static int do_reset(struct ibmvnic_adapter *adapter,
if (rc)
return rc;

+   if (adapter->reset_reason == VNIC_RESET_CHANGE_PARAM ||
+   adapter->wait_for_reset) {
+   release_resources(adapter);
+   release_sub_crqs(adapter);
+   release_crq_queue(adapter);
+   }
+
if (adapter->reset_reason != VNIC_RESET_NON_FATAL) {
/* remove the closed state so when we call open it appears
 * we are coming from the probed state.
@@ -1492,16 +1521,23 @@ static int do_reset(struct ibmvnic_adapter *adapter,
return 0;
}

-   rc = reset_tx_pools(adapter);
-   if (rc)
-   return rc;
+   if (adapter->reset_reason == VNIC_RESET_CHANGE_PARAM ||
+   adapter->wait_for_reset) {
+   rc = init_resources(adapter);
+   if (rc)
+   return rc;
+   } else {
+   rc = reset_tx_pools(adapter);
+   if (rc)
+   return rc;

-   rc = reset_rx_pools(adapter);
-   if (rc)
-   return rc;
+   rc = reset_rx_pools(adapter);
+   if (rc)
+   return rc;

-   if (reset_state == VNIC_CLOSED)
-   return 0;
+   if (reset_state == VNIC_CLOSED)
+   return 0;
+   }
}

rc = __ibmvnic_open(netdev);
@@ -1561,7 +1597,7 @@ static void __ibmvnic_reset(struct work_struct *work)
struct ibmvnic_adapter *adapter;
struct net_device *netdev;
u32 reset_state;
-   int rc;
+   int rc = 0;

adapter = container_of(work, struct ibmvnic_adapter, ibmvnic_reset);
netdev = adapter->netdev;
@@ -1580,6 +1616,12 @@ static void __ibmvnic_reset(struct work_struct *work)
rwi = get_next_rwi(adapter);
}

+   if (adapter->wait_for_reset) {
+   adapter->wait_for_reset = false;
+   adapter->reset_done_rc = rc;
+   complete(>reset_done);
+   }
+
if (rc) {
netdev_dbg(adapter->netdev, "Reset failed\

[PATCH v3 net-next 0/2] ibmvnic: Tunable parameter support

[John Allen]

This series implements support for changing tunable parameters such as the
mtu, number of tx/rx queues, and number of buffers per queue via ethtool
and ifconfig.

v2: -Fix conflict with Tom's recently applied TSO/SG patches
v3: -Initialize rc in __ibmvnic_reset fixing build warning
-Fix buggy behavior with pending mac changes. Use boolean flag
 to track if mac change is needed on open rather than relying on
 the desired->mac pointer.
-Directly include tunable structs in the adapter struct rather
 than keeping pointers, eliminating the need to directly allocate
 them.

John Allen (2):
  ibmvnic: Update reset infrastructure to support tunable parameters
  ibmvnic: Fix failover error path for non-fatal resets

 drivers/net/ethernet/ibm/ibmvnic.c | 217 -
 drivers/net/ethernet/ibm/ibmvnic.h |  26 -
 2 files changed, 213 insertions(+), 30 deletions(-)

-- 
1.8.3.1

[PATCH v2 net-next 2/2] ibmvnic: Fix failover error path for non-fatal resets

[John Allen]

For all non-fatal reset conditions, the hypervisor will send a failover when
we attempt to initialize the crq and the vnic client is expected to handle
that failover instead of the existing non-fatal reset. To handle this, we
need to return from init with a return code that indicates that we have hit
this case.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index c2c4a5b..fa6b2ad 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1506,7 +1506,7 @@ static int do_reset(struct ibmvnic_adapter *adapter,

rc = ibmvnic_init(adapter);
if (rc)
-   return 0;
+   return IBMVNIC_INIT_FAILED;

/* If the adapter was in PROBE state prior to the reset,
 * exit here.
@@ -1609,7 +1609,7 @@ static void __ibmvnic_reset(struct work_struct *work)
while (rwi) {
rc = do_reset(adapter, rwi, reset_state);
kfree(rwi);
-   if (rc)
+   if (rc && rc != IBMVNIC_INIT_FAILED)
break;

rwi = get_next_rwi(adapter);
diff --git a/drivers/net/ethernet/ibm/ibmvnic.h 
b/drivers/net/ethernet/ibm/ibmvnic.h
index 8de998a..b5aaa6e 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.h
+++ b/drivers/net/ethernet/ibm/ibmvnic.h
@@ -30,6 +30,8 @@
 #define IBMVNIC_DRIVER_VERSION "1.0.1"
 #define IBMVNIC_INVALID_MAP-1
 #define IBMVNIC_STATS_TIMEOUT  1
+#define IBMVNIC_INIT_FAILED2
+
 /* basic structures plus 100 2k buffers */
 #define IBMVNIC_IO_ENTITLEMENT_DEFAULT 610305

[PATCH v2 net-next 1/2] ibmvnic: Update reset infrastructure to support tunable parameters

[John Allen]

Update ibmvnic reset infrastructure to include a new reset option that will
allow changing of tunable parameters. There currently is no way to request
different capabilities from the vnic server on the fly so this patch
achieves this by resetting the driver and attempting to log in with the
requested changes. If the reset operation fails, the old values of the
tunable parameters are stored in the "fallback" struct and we attempt to
login with the fallback values.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 11eba82..c2c4a5b 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -115,6 +115,7 @@ static union sub_crq *ibmvnic_next_scrq(struct 
ibmvnic_adapter *,
 static int init_sub_crq_irqs(struct ibmvnic_adapter *adapter);
 static int ibmvnic_init(struct ibmvnic_adapter *);
 static void release_crq_queue(struct ibmvnic_adapter *);
+static int __ibmvnic_set_mac(struct net_device *netdev, struct sockaddr *p);

 struct ibmvnic_stat {
char name[ETH_GSTRING_LEN];
@@ -926,6 +927,11 @@ static int ibmvnic_open(struct net_device *netdev)

mutex_lock(>reset_lock);

+   if (adapter->desired->mac) {
+   __ibmvnic_set_mac(netdev, adapter->desired->mac);
+   adapter->desired->mac = NULL;
+   }
+
if (adapter->state != VNIC_CLOSED) {
rc = ibmvnic_login(netdev);
if (rc) {
@@ -1426,7 +1432,7 @@ static void ibmvnic_set_multi(struct net_device *netdev)
}
 }

-static int ibmvnic_set_mac(struct net_device *netdev, void *p)
+static int __ibmvnic_set_mac(struct net_device *netdev, struct sockaddr *p)
 {
struct ibmvnic_adapter *adapter = netdev_priv(netdev);
struct sockaddr *addr = p;
@@ -1444,6 +1450,21 @@ static int ibmvnic_set_mac(struct net_device *netdev, 
void *p)
return 0;
 }

+static int ibmvnic_set_mac(struct net_device *netdev, void *p)
+{
+   struct ibmvnic_adapter *adapter = netdev_priv(netdev);
+   struct sockaddr *addr = p;
+
+   if (adapter->state != VNIC_OPEN) {
+   memcpy(adapter->desired->mac, addr, sizeof(struct sockaddr));
+   return 0;
+   }
+
+   __ibmvnic_set_mac(netdev, addr);
+
+   return 0;
+}
+
 /**
  * do_reset returns zero if we are able to keep processing reset events, or
  * non-zero if we hit a fatal error and must halt.
@@ -1470,6 +1491,13 @@ static int do_reset(struct ibmvnic_adapter *adapter,
if (rc)
return rc;

+   if (adapter->reset_reason == VNIC_RESET_CHANGE_PARAM ||
+   adapter->wait_for_reset) {
+   release_resources(adapter);
+   release_sub_crqs(adapter);
+   release_crq_queue(adapter);
+   }
+
if (adapter->reset_reason != VNIC_RESET_NON_FATAL) {
/* remove the closed state so when we call open it appears
 * we are coming from the probed state.
@@ -1492,16 +1520,23 @@ static int do_reset(struct ibmvnic_adapter *adapter,
return 0;
}

-   rc = reset_tx_pools(adapter);
-   if (rc)
-   return rc;
+   if (adapter->reset_reason == VNIC_RESET_CHANGE_PARAM ||
+   adapter->wait_for_reset) {
+   rc = init_resources(adapter);
+   if (rc)
+   return rc;
+   } else {
+   rc = reset_tx_pools(adapter);
+   if (rc)
+   return rc;

-   rc = reset_rx_pools(adapter);
-   if (rc)
-   return rc;
+   rc = reset_rx_pools(adapter);
+   if (rc)
+   return rc;

-   if (reset_state == VNIC_CLOSED)
-   return 0;
+   if (reset_state == VNIC_CLOSED)
+   return 0;
+   }
}

rc = __ibmvnic_open(netdev);
@@ -1580,6 +1615,12 @@ static void __ibmvnic_reset(struct work_struct *work)
rwi = get_next_rwi(adapter);
}

+   if (adapter->wait_for_reset) {
+   adapter->wait_for_reset = false;
+   adapter->reset_done_rc = rc;
+   complete(>reset_done);
+   }
+
if (rc) {
netdev_dbg(adapter->netdev, "Reset failed\n");
free_all_rwi(adapter);
@@ -1759,9 +1800,42 @@ static void ibmvnic_netpoll_controller(struct net_device 
*dev)
 }
 #endif

+static int wait_for_reset(struct ibmvnic_adapter *adapter)
+{
+   adapter->fallback->mtu = adapter->req_mtu;
+   adapter->fallback->rx_queues = adapter->req_rx_queues;
+  

[PATCH v2 net-next 0/2] ibmvnic: Tunable parameter support

[John Allen]

This series implements support for changing tunable parameters such as the
mtu, number of tx/rx queues, and number of buffers per queue via ethtool
and ifconfig.

v2: Fix conflict with Tom's recently applied TSO/SG patches

John Allen (2):
  ibmvnic: Update reset infrastructure to support tunable parameters
  ibmvnic: Fix failover error path for non-fatal resets

 drivers/net/ethernet/ibm/ibmvnic.c | 222 -
 drivers/net/ethernet/ibm/ibmvnic.h |  24 +++-
 2 files changed, 216 insertions(+), 30 deletions(-)

-- 
1.8.3.1

[PATCH net-next 2/2] ibmvnic: Fix failover error path for non-fatal resets

[John Allen]

For all non-fatal reset conditions, the hypervisor will send a failover when
we attempt to initialize the crq and the vnic client is expected to handle
that failover instead of the existing non-fatal reset. To handle this, we
need to return from init with a return code that indicates that we have hit
this case.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index ebbb083..c914d5a 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1452,7 +1452,7 @@ static int do_reset(struct ibmvnic_adapter *adapter,

rc = ibmvnic_init(adapter);
if (rc)
-   return 0;
+   return IBMVNIC_INIT_FAILED;

/* If the adapter was in PROBE state prior to the reset,
 * exit here.
@@ -1555,7 +1555,7 @@ static void __ibmvnic_reset(struct work_struct *work)
while (rwi) {
rc = do_reset(adapter, rwi, reset_state);
kfree(rwi);
-   if (rc)
+   if (rc && rc != IBMVNIC_INIT_FAILED)
break;

rwi = get_next_rwi(adapter);
diff --git a/drivers/net/ethernet/ibm/ibmvnic.h 
b/drivers/net/ethernet/ibm/ibmvnic.h
index 5347a7c..46eed65 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.h
+++ b/drivers/net/ethernet/ibm/ibmvnic.h
@@ -30,6 +30,8 @@
 #define IBMVNIC_DRIVER_VERSION "1.0.1"
 #define IBMVNIC_INVALID_MAP-1
 #define IBMVNIC_STATS_TIMEOUT  1
+#define IBMVNIC_INIT_FAILED2
+
 /* basic structures plus 100 2k buffers */
 #define IBMVNIC_IO_ENTITLEMENT_DEFAULT 610305

[PATCH net-next 1/2] ibmvnic: Update reset infrastructure to support tunable parameters

[John Allen]

Update ibmvnic reset infrastructure to include a new reset option that will
allow changing of tunable parameters. There currently is no way to request
different capabilities from the vnic server on the fly so this patch
achieves this by resetting the driver and attempting to log in with the
requested changes. If the reset operation fails, the old values of the
tunable parameters are stored in the "fallback" struct and we attempt to
login with the fallback values.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 4bc14a9..36898b0 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -115,6 +115,7 @@ static union sub_crq *ibmvnic_next_scrq(struct 
ibmvnic_adapter *,
 static int init_sub_crq_irqs(struct ibmvnic_adapter *adapter);
 static int ibmvnic_init(struct ibmvnic_adapter *);
 static void release_crq_queue(struct ibmvnic_adapter *);
+static int __ibmvnic_set_mac(struct net_device *netdev, struct sockaddr *p);

 struct ibmvnic_stat {
char name[ETH_GSTRING_LEN];
@@ -910,6 +911,11 @@ static int ibmvnic_open(struct net_device *netdev)

mutex_lock(>reset_lock);

+   if (adapter->desired->mac) {
+   __ibmvnic_set_mac(netdev, adapter->desired->mac);
+   adapter->desired->mac = NULL;
+   }
+
if (adapter->state != VNIC_CLOSED) {
rc = ibmvnic_login(netdev);
if (rc) {
@@ -1372,7 +1378,7 @@ static void ibmvnic_set_multi(struct net_device *netdev)
}
 }

-static int ibmvnic_set_mac(struct net_device *netdev, void *p)
+static int __ibmvnic_set_mac(struct net_device *netdev, struct sockaddr *p)
 {
struct ibmvnic_adapter *adapter = netdev_priv(netdev);
struct sockaddr *addr = p;
@@ -1390,6 +1396,21 @@ static int ibmvnic_set_mac(struct net_device *netdev, 
void *p)
return 0;
 }

+static int ibmvnic_set_mac(struct net_device *netdev, void *p)
+{
+   struct ibmvnic_adapter *adapter = netdev_priv(netdev);
+   struct sockaddr *addr = p;
+
+   if (adapter->state != VNIC_OPEN) {
+   memcpy(adapter->desired->mac, addr, sizeof(struct sockaddr));
+   return 0;
+   }
+
+   __ibmvnic_set_mac(netdev, addr);
+
+   return 0;
+}
+
 /**
  * do_reset returns zero if we are able to keep processing reset events, or
  * non-zero if we hit a fatal error and must halt.
@@ -1416,6 +1437,13 @@ static int do_reset(struct ibmvnic_adapter *adapter,
if (rc)
return rc;

+   if (adapter->reset_reason == VNIC_RESET_CHANGE_PARAM ||
+   adapter->wait_for_reset) {
+   release_resources(adapter);
+   release_sub_crqs(adapter);
+   release_crq_queue(adapter);
+   }
+
if (adapter->reset_reason != VNIC_RESET_NON_FATAL) {
/* remove the closed state so when we call open it appears
 * we are coming from the probed state.
@@ -1438,16 +1466,23 @@ static int do_reset(struct ibmvnic_adapter *adapter,
return 0;
}

-   rc = reset_tx_pools(adapter);
-   if (rc)
-   return rc;
+   if (adapter->reset_reason == VNIC_RESET_CHANGE_PARAM ||
+   adapter->wait_for_reset) {
+   rc = init_resources(adapter);
+   if (rc)
+   return rc;
+   } else {
+   rc = reset_tx_pools(adapter);
+   if (rc)
+   return rc;

-   rc = reset_rx_pools(adapter);
-   if (rc)
-   return rc;
+   rc = reset_rx_pools(adapter);
+   if (rc)
+   return rc;

-   if (reset_state == VNIC_CLOSED)
-   return 0;
+   if (reset_state == VNIC_CLOSED)
+   return 0;
+   }
}

rc = __ibmvnic_open(netdev);
@@ -1526,6 +1561,12 @@ static void __ibmvnic_reset(struct work_struct *work)
rwi = get_next_rwi(adapter);
}

+   if (adapter->wait_for_reset) {
+   adapter->wait_for_reset = false;
+   adapter->reset_done_rc = rc;
+   complete(>reset_done);
+   }
+
if (rc) {
netdev_dbg(adapter->netdev, "Reset failed\n");
free_all_rwi(adapter);
@@ -1705,9 +1746,42 @@ static void ibmvnic_netpoll_controller(struct net_device 
*dev)
 }
 #endif

+static int wait_for_reset(struct ibmvnic_adapter *adapter)
+{
+   adapter->fallback->mtu = adapter->req_mtu;
+   adapter->fallback->rx_queues = adapter->req_rx_queues;
+  

[PATCH net-next 0/2] ibmvnic: Tunable parameter support

[John Allen]

This series implements support for changing tunable parameters such as the
mtu, number of tx/rx queues, and number of buffers per queue via ethtool
and ifconfig.

John Allen (2):
  ibmvnic: Update reset infrastructure to support tunable parameters
  ibmvnic: Fix failover error path for non-fatal resets

 drivers/net/ethernet/ibm/ibmvnic.c | 222 -
 drivers/net/ethernet/ibm/ibmvnic.h |  24 +++-
 2 files changed, 216 insertions(+), 30 deletions(-)

-- 
1.8.3.1

[PATCH net-next] ibmvnic: Improve output for unsupported stats

[John Allen]

The vnic server can report -1 in the event that a given statistic is not
supported. Currently, the -1 value is implicitly cast to an unsigned
integer and appears through the ethtool -S output as a very large number.
This patch improves this behavior by reporting 0 in the event that a
given statistic is not supported.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index cb8182f..b8ad2db 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1862,6 +1862,7 @@ static void ibmvnic_get_ethtool_stats(struct net_device 
*dev,
 {
struct ibmvnic_adapter *adapter = netdev_priv(dev);
union ibmvnic_crq crq;
+   u64 stat;
int i, j;

memset(, 0, sizeof(crq));
@@ -1876,9 +1877,11 @@ static void ibmvnic_get_ethtool_stats(struct net_device 
*dev,
ibmvnic_send_crq(adapter, );
wait_for_completion(>stats_done);

-   for (i = 0; i < ARRAY_SIZE(ibmvnic_stats); i++)
-   data[i] = be64_to_cpu(IBMVNIC_GET_STAT(adapter,
-   ibmvnic_stats[i].offset));
+   for (i = 0; i < ARRAY_SIZE(ibmvnic_stats); i++) {
+   stat = be64_to_cpu(IBMVNIC_GET_STAT(adapter,
+   ibmvnic_stats[i].offset));
+   data[i] = stat == -1 ? 0 : stat;
+   }

for (j = 0; j < adapter->req_tx_queues; j++) {
data[i] = adapter->tx_stats_buffers[j].packets;

[PATCH net-next] ibmvnic: Report rx buffer return codes as netdev_dbg

[John Allen]

Reporting any return code for a receive buffer as an "rx error" only
produces alarming noise and the only values that have been observed to be
used in this field are not error conditions. Change this to a netdev_dbg
with a more descriptive message.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 5932160..99576ba 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1579,7 +1579,8 @@ static int ibmvnic_poll(struct napi_struct *napi, int 
budget)
  rx_comp.correlator);
/* do error checking */
if (next->rx_comp.rc) {
-   netdev_err(netdev, "rx error %x\n", next->rx_comp.rc);
+   netdev_dbg(netdev, "rx buffer returned with rc %x\n",
+  be16_to_cpu(next->rx_comp.rc));
/* free the entry */
next->rx_comp.first = 0;
remove_buff_from_pool(adapter, rx_buff);

[PATCH net-next 4/4] ibmvnic: Implement .get_channels

[John Allen]

Implement .get_channels (ethtool -l) functionality

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 1cc5db94e40f..130aee7b5c32 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1741,6 +1741,21 @@ static void ibmvnic_get_ringparam(struct net_device 
*netdev,
ring->rx_jumbo_pending = 0;
 }

+static void ibmvnic_get_channels(struct net_device *netdev,
+struct ethtool_channels *channels)
+{
+   struct ibmvnic_adapter *adapter = netdev_priv(netdev);
+
+   channels->max_rx = adapter->max_rx_queues;
+   channels->max_tx = adapter->max_tx_queues;
+   channels->max_other = 0;
+   channels->max_combined = 0;
+   channels->rx_count = adapter->req_rx_queues;
+   channels->tx_count = adapter->req_tx_queues;
+   channels->other_count = 0;
+   channels->combined_count = 0;
+}
+
 static void ibmvnic_get_strings(struct net_device *dev, u32 stringset, u8 
*data)
 {
struct ibmvnic_adapter *adapter = netdev_priv(dev);
@@ -1837,6 +1852,7 @@ static const struct ethtool_ops ibmvnic_ethtool_ops = {
.set_msglevel   = ibmvnic_set_msglevel,
.get_link   = ibmvnic_get_link,
.get_ringparam  = ibmvnic_get_ringparam,
+   .get_channels   = ibmvnic_get_channels,
.get_strings= ibmvnic_get_strings,
.get_sset_count = ibmvnic_get_sset_count,
.get_ethtool_stats  = ibmvnic_get_ethtool_stats,

[PATCH net-next 3/4] ibmvnic: Implement .get_ringparam

[John Allen]

Implement .get_ringparam (ethtool -g) functionality

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 285ea23bac6a..1cc5db94e40f 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1729,12 +1729,14 @@ static u32 ibmvnic_get_link(struct net_device *netdev)
 static void ibmvnic_get_ringparam(struct net_device *netdev,
  struct ethtool_ringparam *ring)
 {
-   ring->rx_max_pending = 0;
-   ring->tx_max_pending = 0;
+   struct ibmvnic_adapter *adapter = netdev_priv(netdev);
+
+   ring->rx_max_pending = adapter->max_rx_add_entries_per_subcrq;
+   ring->tx_max_pending = adapter->max_tx_entries_per_subcrq;
ring->rx_mini_max_pending = 0;
ring->rx_jumbo_max_pending = 0;
-   ring->rx_pending = 0;
-   ring->tx_pending = 0;
+   ring->rx_pending = adapter->req_rx_add_entries_per_subcrq;
+   ring->tx_pending = adapter->req_tx_entries_per_subcrq;
ring->rx_mini_pending = 0;
ring->rx_jumbo_pending = 0;
 }

[PATCH net-next 2/4] ibmvnic: Convert vnic server reported statistics to cpu endian

[John Allen]

The vnic server reports the statistics buffer in big endian format and must
be converted to cpu endian in order to be displayed correctly on little
endian lpars.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index a264c6992328..285ea23bac6a 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1807,7 +1807,8 @@ static void ibmvnic_get_ethtool_stats(struct net_device 
*dev,
wait_for_completion(>stats_done);

for (i = 0; i < ARRAY_SIZE(ibmvnic_stats); i++)
-   data[i] = IBMVNIC_GET_STAT(adapter, ibmvnic_stats[i].offset);
+   data[i] = be64_to_cpu(IBMVNIC_GET_STAT(adapter,
+   ibmvnic_stats[i].offset));

for (j = 0; j < adapter->req_tx_queues; j++) {
data[i] = adapter->tx_stats_buffers[j].packets;

[PATCH net-next 1/4] ibmvnic: Implement per-queue statistics reporting

[John Allen]

Add counters to report number of packets, bytes, and dropped packets for
each transmit queue and number of packets, bytes, and interrupts for each
receive queue. Modify ethtool callbacks to report the new statistics.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index a3e694679635..a264c6992328 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -346,6 +346,31 @@ static void replenish_pools(struct ibmvnic_adapter 
*adapter)
}
 }

+static void release_stats_buffers(struct ibmvnic_adapter *adapter)
+{
+   kfree(adapter->tx_stats_buffers);
+   kfree(adapter->rx_stats_buffers);
+}
+
+static int init_stats_buffers(struct ibmvnic_adapter *adapter)
+{
+   adapter->tx_stats_buffers =
+   kcalloc(adapter->req_tx_queues,
+   sizeof(struct ibmvnic_tx_queue_stats),
+   GFP_KERNEL);
+   if (!adapter->tx_stats_buffers)
+   return -ENOMEM;
+
+   adapter->rx_stats_buffers =
+   kcalloc(adapter->req_rx_queues,
+   sizeof(struct ibmvnic_rx_queue_stats),
+   GFP_KERNEL);
+   if (!adapter->rx_stats_buffers)
+   return -ENOMEM;
+
+   return 0;
+}
+
 static void release_stats_token(struct ibmvnic_adapter *adapter)
 {
struct device *dev = >vdev->dev;
@@ -686,6 +711,7 @@ static void release_resources(struct ibmvnic_adapter 
*adapter)
release_rx_pools(adapter);

release_stats_token(adapter);
+   release_stats_buffers(adapter);
release_error_buffers(adapter);

if (adapter->napi) {
@@ -763,6 +789,10 @@ static int init_resources(struct ibmvnic_adapter *adapter)
if (rc)
return rc;

+   rc = init_stats_buffers(adapter);
+   if (rc)
+   return rc;
+
rc = init_stats_token(adapter);
if (rc)
return rc;
@@ -1245,6 +1275,9 @@ static int ibmvnic_xmit(struct sk_buff *skb, struct 
net_device *netdev)
netdev->stats.tx_packets += tx_packets;
adapter->tx_send_failed += tx_send_failed;
adapter->tx_map_failed += tx_map_failed;
+   adapter->tx_stats_buffers[queue_num].packets += tx_packets;
+   adapter->tx_stats_buffers[queue_num].bytes += tx_bytes;
+   adapter->tx_stats_buffers[queue_num].dropped_packets += tx_dropped;

return ret;
 }
@@ -1585,6 +1618,8 @@ static int ibmvnic_poll(struct napi_struct *napi, int 
budget)
napi_gro_receive(napi, skb); /* send it up */
netdev->stats.rx_packets++;
netdev->stats.rx_bytes += length;
+   adapter->rx_stats_buffers[scrq_num].packets++;
+   adapter->rx_stats_buffers[scrq_num].bytes += length;
frames_processed++;
}

@@ -1706,6 +1741,7 @@ static void ibmvnic_get_ringparam(struct net_device 
*netdev,

 static void ibmvnic_get_strings(struct net_device *dev, u32 stringset, u8 
*data)
 {
+   struct ibmvnic_adapter *adapter = netdev_priv(dev);
int i;

if (stringset != ETH_SS_STATS)
@@ -1713,13 +1749,39 @@ static void ibmvnic_get_strings(struct net_device *dev, 
u32 stringset, u8 *data)

for (i = 0; i < ARRAY_SIZE(ibmvnic_stats); i++, data += ETH_GSTRING_LEN)
memcpy(data, ibmvnic_stats[i].name, ETH_GSTRING_LEN);
+
+   for (i = 0; i < adapter->req_tx_queues; i++) {
+   snprintf(data, ETH_GSTRING_LEN, "tx%d_packets", i);
+   data += ETH_GSTRING_LEN;
+
+   snprintf(data, ETH_GSTRING_LEN, "tx%d_bytes", i);
+   data += ETH_GSTRING_LEN;
+
+   snprintf(data, ETH_GSTRING_LEN, "tx%d_dropped_packets", i);
+   data += ETH_GSTRING_LEN;
+   }
+
+   for (i = 0; i < adapter->req_rx_queues; i++) {
+   snprintf(data, ETH_GSTRING_LEN, "rx%d_packets", i);
+   data += ETH_GSTRING_LEN;
+
+   snprintf(data, ETH_GSTRING_LEN, "rx%d_bytes", i);
+   data += ETH_GSTRING_LEN;
+
+   snprintf(data, ETH_GSTRING_LEN, "rx%d_interrupts", i);
+   data += ETH_GSTRING_LEN;
+   }
 }

 static int ibmvnic_get_sset_count(struct net_device *dev, int sset)
 {
+   struct ibmvnic_adapter *adapter = netdev_priv(dev);
+
switch (sset) {
case ETH_SS_STATS:
-   return ARRAY_SIZE(ibmvnic_stats);
+   return ARRAY_SIZE(ibmvnic_stats) +
+  adapter->req_tx_queues * NUM_TX_STATS +
+  adapter->req_rx_queues * NUM_RX_STATS;
default:
return -EOPNOTSUPP;
}
@@ -1730,7 

[PATCH net-next 0/4] ibmvnic: Improve ethtool functionality

[John Allen]

This patch series improves ibmvnic ethtool functionality by adding support
for ethtool -l and -g options, correcting existing statistics reporting,
and augmenting the existing statistics with counters for each tx and rx
queue.

John Allen (4):
  ibmvnic: Implement per-queue statistics reporting
  ibmvnic: Convert vnic server reported statistics to cpu endian
  ibmvnic: Implement .get_ringparam
  ibmvnic: Implement .get_channels

 drivers/net/ethernet/ibm/ibmvnic.c | 115 ++---
 drivers/net/ethernet/ibm/ibmvnic.h |  17 ++
 2 files changed, 125 insertions(+), 7 deletions(-)

-- 
2.12.3

[PATCH net-next] ibmvnic: Check for transport event on driver resume

[John Allen]

On resume, the ibmvnic driver will fail to resume normal operation.
The main crq gets closed on suspend by the vnic server and doesn't get
reopened again as the interrupt for the transport event that would reset
the main crq comes in after the driver has been suspended.

This patch resolves the issue by removing the calls to kick the receive
interrupts handlers and instead directly invoking the main crq interrupt
handler. This will ensure that we see the transport event necessary to
properly resume the driver.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index a3e694679635..9d8af464dc44 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -3851,10 +3851,7 @@ static int ibmvnic_resume(struct device *dev)
if (adapter->state != VNIC_OPEN)
return 0;

-   /* kick the interrupt handlers just in case we lost an interrupt */
-   for (i = 0; i < adapter->req_rx_queues; i++)
-   ibmvnic_interrupt_rx(adapter->rx_scrq[i]->irq,
-adapter->rx_scrq[i]);
+   tasklet_schedule(>tasklet);

return 0;
 }

[PATCH net-next] ibmvnic: Return from ibmvnic_resume if not in VNIC_OPEN state

[John Allen]

If the ibmvnic driver is not in the VNIC_OPEN state, return from
ibmvnic_resume callback. If we are not in the VNIC_OPEN state, interrupts
may not be initialized and directly calling the interrupt handler will
cause a crash.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 722daf5..0135095 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -3859,6 +3859,9 @@ static int ibmvnic_resume(struct device *dev)
struct ibmvnic_adapter *adapter = netdev_priv(netdev);
int i;

+   if (adapter->state != VNIC_OPEN)
+   return 0;
+
/* kick the interrupt handlers just in case we lost an interrupt */
for (i = 0; i < adapter->req_rx_queues; i++)
ibmvnic_interrupt_rx(adapter->rx_scrq[i]->irq,

[PATCH net] ibmvnic: Return failure on attempted mtu change

[John Allen]

Changing the mtu is currently not supported in the ibmvnic driver.

Implement .ndo_change_mtu in the driver so that attempting to use ifconfig
to change the mtu will fail and present the user with an error message.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 4f2d329..8ff6c74 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1468,6 +1468,11 @@ static void ibmvnic_netpoll_controller(struct net_device 
*dev)
 }
 #endif

+static int ibmvnic_change_mtu(struct net_device *netdev, int new_mtu)
+{
+   return -EOPNOTSUPP;
+}
+
 static const struct net_device_ops ibmvnic_netdev_ops = {
.ndo_open   = ibmvnic_open,
.ndo_stop   = ibmvnic_close,
@@ -1479,6 +1484,7 @@ static void ibmvnic_netpoll_controller(struct net_device 
*dev)
 #ifdef CONFIG_NET_POLL_CONTROLLER
.ndo_poll_controller= ibmvnic_netpoll_controller,
 #endif
+   .ndo_change_mtu = ibmvnic_change_mtu,
 };

 /* ethtool functions */

[PATCH net-next] ibmvnic: Track state of adapter napis

[John Allen]

Track the state of ibmvnic napis. The driver can get into states where it
can be reset when napis are already disabled and attempting to disable them
again will cause the driver to hang.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 4f2d329..594ee6d 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -675,8 +675,12 @@ static int __ibmvnic_open(struct net_device *netdev)
adapter->state = VNIC_OPENING;
replenish_pools(adapter);

-   for (i = 0; i < adapter->req_rx_queues; i++)
-   napi_enable(>napi[i]);
+   if (adapter->napi_disabled) {
+   for (i = 0; i < adapter->req_rx_queues; i++)
+   napi_enable(>napi[i]);
+
+   adapter->napi_disabled = false;
+   }

/* We're ready to receive frames, enable the sub-crq interrupts and
 * set the logical link state to up
@@ -780,9 +784,11 @@ static int __ibmvnic_close(struct net_device *netdev)
adapter->state = VNIC_CLOSING;
netif_tx_stop_all_queues(netdev);

-   if (adapter->napi) {
+   if (!adapter->napi_disabled) {
for (i = 0; i < adapter->req_rx_queues; i++)
napi_disable(>napi[i]);
+
+   adapter->napi_disabled = true;
}

clean_tx_pools(adapter);
@@ -3540,6 +3546,9 @@ static int ibmvnic_probe(struct vio_dev *dev, const 
struct vio_device_id *id)
free_netdev(netdev);
return rc;
}
+
+   adapter->napi_disabled = true;
+
dev_info(>dev, "ibmvnic registered\n");

adapter->state = VNIC_PROBED;
diff --git a/drivers/net/ethernet/ibm/ibmvnic.h 
b/drivers/net/ethernet/ibm/ibmvnic.h
index 4702b48..12b2400 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.h
+++ b/drivers/net/ethernet/ibm/ibmvnic.h
@@ -1031,4 +1031,5 @@ struct ibmvnic_adapter {
struct list_head rwi_list;
struct work_struct ibmvnic_reset;
bool resetting;
+   bool napi_disabled;
 };

[PATCH net 3/4] ibmvnic: Move ibmvnic adapter intialization to its own routine

[John Allen]

The intialization of the ibmvnic driver with respect to the virtual
server it connects to should be moved to its own routine. This will
alolow the driver to initiate this process from places outside of
the drivers probe routine.

Signed-off-by: Nathan Fontenot 
---
 drivers/net/ethernet/ibm/ibmvnic.c | 110 -
 1 file changed, 60 insertions(+), 50 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index b6606d8..f8cc572 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -3785,14 +3785,65 @@ static void handle_crq_init_rsp(struct work_struct 
*work)
dev_err(dev, "Passive initialization was not successful\n");
 }
 
-static int ibmvnic_probe(struct vio_dev *dev, const struct vio_device_id *id)
+static int ibmvnic_init(struct ibmvnic_adapter *adapter)
 {
+   struct device *dev = >vdev->dev;
unsigned long timeout = msecs_to_jiffies(3);
+   struct dentry *ent;
+   char buf[17]; /* debugfs name buf */
+   int rc;
+
+   rc = ibmvnic_init_crq_queue(adapter);
+   if (rc) {
+   dev_err(dev, "Couldn't initialize crq. rc=%d\n", rc);
+   return rc;
+   }
+
+   adapter->stats_token = dma_map_single(dev, >stats,
+ sizeof(struct ibmvnic_statistics),
+ DMA_FROM_DEVICE);
+   if (dma_mapping_error(dev, adapter->stats_token)) {
+   ibmvnic_release_crq_queue(adapter);
+   dev_err(dev, "Couldn't map stats buffer\n");
+   return -ENOMEM;
+   }
+
+   snprintf(buf, sizeof(buf), "ibmvnic_%x", adapter->vdev->unit_address);
+   ent = debugfs_create_dir(buf, NULL);
+   if (!ent || IS_ERR(ent)) {
+   dev_info(dev, "debugfs create directory failed\n");
+   adapter->debugfs_dir = NULL;
+   } else {
+   adapter->debugfs_dir = ent;
+   ent = debugfs_create_file("dump", S_IRUGO,
+ adapter->debugfs_dir,
+ adapter->netdev, _dump_ops);
+   if (!ent || IS_ERR(ent)) {
+   dev_info(dev, "debugfs create dump file failed\n");
+   adapter->debugfs_dump = NULL;
+   } else {
+   adapter->debugfs_dump = ent;
+   }
+   }
+
+   init_completion(>init_done);
+   ibmvnic_send_crq_init(adapter);
+   if (!wait_for_completion_timeout(>init_done, timeout)) {
+   dev_err(dev, "Initialization sequence timed out\n");
+   if (adapter->debugfs_dir && !IS_ERR(adapter->debugfs_dir))
+   debugfs_remove_recursive(adapter->debugfs_dir);
+   ibmvnic_release_crq_queue(adapter);
+   return -1;
+   }
+
+   return 0;
+}
+
+static int ibmvnic_probe(struct vio_dev *dev, const struct vio_device_id *id)
+{
struct ibmvnic_adapter *adapter;
struct net_device *netdev;
unsigned char *mac_addr_p;
-   struct dentry *ent;
-   char buf[17]; /* debugfs name buf */
int rc;
 
dev_dbg(>dev, "entering ibmvnic_probe for UA 0x%x\n",
@@ -3830,69 +3881,28 @@ static int ibmvnic_probe(struct vio_dev *dev, const 
struct vio_device_id *id)
 
spin_lock_init(>stats_lock);
 
-   rc = ibmvnic_init_crq_queue(adapter);
-   if (rc) {
-   dev_err(>dev, "Couldn't initialize crq. rc=%d\n", rc);
-   goto free_netdev;
-   }
-
INIT_LIST_HEAD(>errors);
INIT_LIST_HEAD(>inflight);
spin_lock_init(>error_list_lock);
spin_lock_init(>inflight_lock);
 
-   adapter->stats_token = dma_map_single(>dev, >stats,
- sizeof(struct ibmvnic_statistics),
- DMA_FROM_DEVICE);
-   if (dma_mapping_error(>dev, adapter->stats_token)) {
-   if (!firmware_has_feature(FW_FEATURE_CMO))
-   dev_err(>dev, "Couldn't map stats buffer\n");
-   rc = -ENOMEM;
-   goto free_crq;
-   }
-
-   snprintf(buf, sizeof(buf), "ibmvnic_%x", dev->unit_address);
-   ent = debugfs_create_dir(buf, NULL);
-   if (!ent || IS_ERR(ent)) {
-   dev_info(>dev, "debugfs create directory failed\n");
-   adapter->debugfs_dir = NULL;
-   } else {
-   adapter->debugfs_dir = ent;
-   ent = debugfs_create_file("dump", S_IRUGO, adapter->debugfs_dir,
- netdev, _dump_ops);
-   if (!ent || IS_ERR(ent)) {
-   dev_info(>dev,
-"debugfs create dump file failed\n");
-   adapter->debugfs_dump = NULL;
-   } else {
-   

[PATCH net 2/4] ibmvnic: Move login to its own routine

[John Allen]

Move the code that handles login and renegotiation of ibmvnic
capabilities to its own routine.

Signed-off-by: Nathan Fontenot 
---
 drivers/net/ethernet/ibm/ibmvnic.c | 28 
 1 file changed, 20 insertions(+), 8 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 7588c12..b6606d8 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -371,18 +371,11 @@ static void free_rx_pool(struct ibmvnic_adapter *adapter,
pool->rx_buff = NULL;
 }
 
-static int ibmvnic_open(struct net_device *netdev)
+static int ibmvnic_login(struct net_device *netdev)
 {
struct ibmvnic_adapter *adapter = netdev_priv(netdev);
unsigned long timeout = msecs_to_jiffies(3);
struct device *dev = >vdev->dev;
-   struct ibmvnic_tx_pool *tx_pool;
-   union ibmvnic_crq crq;
-   int rxadd_subcrqs;
-   u64 *size_array;
-   int tx_subcrqs;
-   int rc = 0;
-   int i, j;
 
do {
if (adapter->renegotiate) {
@@ -407,6 +400,25 @@ static int ibmvnic_open(struct net_device *netdev)
}
} while (adapter->renegotiate);
 
+   return 0;
+}
+
+static int ibmvnic_open(struct net_device *netdev)
+{
+   struct ibmvnic_adapter *adapter = netdev_priv(netdev);
+   struct device *dev = >vdev->dev;
+   struct ibmvnic_tx_pool *tx_pool;
+   union ibmvnic_crq crq;
+   int rxadd_subcrqs;
+   u64 *size_array;
+   int tx_subcrqs;
+   int rc = 0;
+   int i, j;
+
+   rc = ibmvnic_login(netdev);
+   if (rc)
+   return rc;
+
rc = netif_set_real_num_tx_queues(netdev, adapter->req_tx_queues);
if (rc) {
dev_err(dev, "failed to set the number of tx queues\n");
-- 
2.7.4

[PATCH net 4/4] ibmvnic: Correct ibmvnic handling of device open/close

[John Allen]

When closing the ibmvnic device we need to release the resources used
in communicating to the virtual I/O server. These need to be
re-negotiated with the server at open time.

This patch moves the releasing of resources a separate routine
and updates the open and close handlers to release all resources at
close and re-negotiate and allocate these resources at open.

Signed-off-by: Nathan Fontenot 
---
 drivers/net/ethernet/ibm/ibmvnic.c | 114 +
 drivers/net/ethernet/ibm/ibmvnic.h |   1 +
 2 files changed, 67 insertions(+), 48 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index f8cc572..3d73182 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -113,6 +113,8 @@ static void send_request_unmap(struct ibmvnic_adapter *, 
u8);
 static void send_login(struct ibmvnic_adapter *adapter);
 static void send_cap_queries(struct ibmvnic_adapter *adapter);
 static int init_sub_crq_irqs(struct ibmvnic_adapter *adapter);
+static int ibmvnic_init(struct ibmvnic_adapter *);
+static void ibmvnic_release_crq_queue(struct ibmvnic_adapter *);
 
 struct ibmvnic_stat {
char name[ETH_GSTRING_LEN];
@@ -415,6 +417,12 @@ static int ibmvnic_open(struct net_device *netdev)
int rc = 0;
int i, j;
 
+   if (adapter->is_closed) {
+   rc = ibmvnic_init(adapter);
+   if (rc)
+   return rc;
+   }
+
rc = ibmvnic_login(netdev);
if (rc)
return rc;
@@ -525,6 +533,7 @@ static int ibmvnic_open(struct net_device *netdev)
ibmvnic_send_crq(adapter, );
 
netif_tx_start_all_queues(netdev);
+   adapter->is_closed = false;
 
return 0;
 
@@ -564,21 +573,12 @@ static int ibmvnic_open(struct net_device *netdev)
return -ENOMEM;
 }
 
-static int ibmvnic_close(struct net_device *netdev)
+static void ibmvnic_release_resources(struct ibmvnic_adapter *adapter)
 {
-   struct ibmvnic_adapter *adapter = netdev_priv(netdev);
struct device *dev = >vdev->dev;
-   union ibmvnic_crq crq;
+   int tx_scrqs, rx_scrqs;
int i;
 
-   adapter->closing = true;
-
-   for (i = 0; i < adapter->req_rx_queues; i++)
-   napi_disable(>napi[i]);
-
-   if (!adapter->failover)
-   netif_tx_stop_all_queues(netdev);
-
if (adapter->bounce_buffer) {
if (!dma_mapping_error(dev, adapter->bounce_buffer_dma)) {
dma_unmap_single(>vdev->dev,
@@ -591,33 +591,70 @@ static int ibmvnic_close(struct net_device *netdev)
adapter->bounce_buffer = NULL;
}
 
-   memset(, 0, sizeof(crq));
-   crq.logical_link_state.first = IBMVNIC_CRQ_CMD;
-   crq.logical_link_state.cmd = LOGICAL_LINK_STATE;
-   crq.logical_link_state.link_state = IBMVNIC_LOGICAL_LNK_DN;
-   ibmvnic_send_crq(adapter, );
+   tx_scrqs = be32_to_cpu(adapter->login_rsp_buf->num_txsubm_subcrqs);
+   for (i = 0; i < tx_scrqs; i++) {
+   struct ibmvnic_tx_pool *tx_pool = >tx_pool[i];
 
-   for (i = 0; i < be32_to_cpu(adapter->login_rsp_buf->num_txsubm_subcrqs);
-i++) {
-   kfree(adapter->tx_pool[i].tx_buff);
-   free_long_term_buff(adapter,
-   >tx_pool[i].long_term_buff);
-   kfree(adapter->tx_pool[i].free_map);
+   kfree(tx_pool->tx_buff);
+   free_long_term_buff(adapter, _pool->long_term_buff);
+   kfree(tx_pool->free_map);
}
kfree(adapter->tx_pool);
adapter->tx_pool = NULL;
 
-   for (i = 0; i < be32_to_cpu(adapter->login_rsp_buf->num_rxadd_subcrqs);
-i++) {
-   free_rx_pool(adapter, >rx_pool[i]);
-   free_long_term_buff(adapter,
-   >rx_pool[i].long_term_buff);
+   rx_scrqs = be32_to_cpu(adapter->login_rsp_buf->num_rxadd_subcrqs);
+   for (i = 0; i < rx_scrqs; i++) {
+   struct ibmvnic_rx_pool *rx_pool = >rx_pool[i];
+
+   free_rx_pool(adapter, rx_pool);
+   free_long_term_buff(adapter, _pool->long_term_buff);
}
kfree(adapter->rx_pool);
adapter->rx_pool = NULL;
 
-   adapter->closing = false;
+   release_sub_crqs(adapter);
+   ibmvnic_release_crq_queue(adapter);
+
+   if (adapter->debugfs_dir && !IS_ERR(adapter->debugfs_dir))
+   debugfs_remove_recursive(adapter->debugfs_dir);
+
+   if (adapter->stats_token)
+   dma_unmap_single(dev, adapter->stats_token,
+sizeof(struct ibmvnic_statistics),
+DMA_FROM_DEVICE);
+
+   if (adapter->ras_comps)
+   dma_free_coherent(dev, adapter->ras_comp_num *
+ sizeof(struct ibmvnic_fw_component),
+   

[PATCH net 0/4] ibmvnic: Initialization fixes and improvements

[John Allen]

These patches resolve issues with the ibmvnic initialization process.

John Allen (1):
  ibmvnic: Move login and queue negotiation into ibmvnic_open

Nathan Fontenot (3):
  ibmvnic: Move login to its own routine
  ibmvnic: Move ibmvnic adapter intialization to its own routine
  ibmvnic: Correct ibmvnic handling of device open/close

 drivers/net/ethernet/ibm/ibmvnic.c | 320 +
 drivers/net/ethernet/ibm/ibmvnic.h |   1 +
 2 files changed, 180 insertions(+), 141 deletions(-)

-- 
2.7.4

[PATCH net 1/4] ibmvnic: Move login and queue negotiation into ibmvnic_open

[John Allen]

VNIC server expects LINK_STATE_UP to be sent within 30s of the login. If we
exceed the timeout, VNIC server will attempt to fail over. Since time
between probe and open of the device is indeterminate, move login and queue
negotiation into ibmvnic open so we can guarantee that login and sending
LINK_STATE_UP occur within the 30s window.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
 drivers/net/ethernet/ibm/ibmvnic.c | 88 +++---
 1 file changed, 43 insertions(+), 45 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index b23d654..7588c12 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -110,6 +110,9 @@ static int ibmvnic_poll(struct napi_struct *napi, int data);
 static void send_map_query(struct ibmvnic_adapter *adapter);
 static void send_request_map(struct ibmvnic_adapter *, dma_addr_t, __be32, u8);
 static void send_request_unmap(struct ibmvnic_adapter *, u8);
+static void send_login(struct ibmvnic_adapter *adapter);
+static void send_cap_queries(struct ibmvnic_adapter *adapter);
+static int init_sub_crq_irqs(struct ibmvnic_adapter *adapter);
 
 struct ibmvnic_stat {
char name[ETH_GSTRING_LEN];
@@ -371,14 +374,51 @@ static void free_rx_pool(struct ibmvnic_adapter *adapter,
 static int ibmvnic_open(struct net_device *netdev)
 {
struct ibmvnic_adapter *adapter = netdev_priv(netdev);
+   unsigned long timeout = msecs_to_jiffies(3);
struct device *dev = >vdev->dev;
struct ibmvnic_tx_pool *tx_pool;
union ibmvnic_crq crq;
int rxadd_subcrqs;
u64 *size_array;
int tx_subcrqs;
+   int rc = 0;
int i, j;
 
+   do {
+   if (adapter->renegotiate) {
+   adapter->renegotiate = false;
+   release_sub_crqs_no_irqs(adapter);
+
+   reinit_completion(>init_done);
+   send_cap_queries(adapter);
+   if (!wait_for_completion_timeout(>init_done,
+timeout)) {
+   dev_err(dev, "Capabilities query timeout\n");
+   return -1;
+   }
+   }
+
+   reinit_completion(>init_done);
+   send_login(adapter);
+   if (!wait_for_completion_timeout(>init_done,
+timeout)) {
+   dev_err(dev, "Login timeout\n");
+   return -1;
+   }
+   } while (adapter->renegotiate);
+
+   rc = netif_set_real_num_tx_queues(netdev, adapter->req_tx_queues);
+   if (rc) {
+   dev_err(dev, "failed to set the number of tx queues\n");
+   return -1;
+   }
+
+   rc = init_sub_crq_irqs(adapter);
+   if (rc) {
+   dev_err(dev, "failed to initialize sub crq irqs\n");
+   return -1;
+   }
+
rxadd_subcrqs =
be32_to_cpu(adapter->login_rsp_buf->num_rxadd_subcrqs);
tx_subcrqs =
@@ -508,6 +548,7 @@ static int ibmvnic_open(struct net_device *netdev)
for (i = 0; i < adapter->req_rx_queues; i++)
napi_disable(>napi[i]);
 alloc_napi_failed:
+   release_sub_crqs(adapter);
return -ENOMEM;
 }
 
@@ -3421,8 +3462,7 @@ static void ibmvnic_handle_crq(union ibmvnic_crq *crq,
dma_unmap_single(dev, adapter->ip_offload_ctrl_tok,
 sizeof(adapter->ip_offload_ctrl),
 DMA_TO_DEVICE);
-   /* We're done with the queries, perform the login */
-   send_login(adapter);
+   complete(>init_done);
break;
case REQUEST_RAS_COMP_NUM_RSP:
netdev_dbg(netdev, "Got Request RAS Comp Num Response\n");
@@ -3702,26 +3742,6 @@ static void handle_crq_init_rsp(struct work_struct *work)
goto task_failed;
}
 
-   do {
-   if (adapter->renegotiate) {
-   adapter->renegotiate = false;
-   release_sub_crqs_no_irqs(adapter);
-
-   reinit_completion(>init_done);
-   send_cap_queries(adapter);
-   if (!wait_for_completion_timeout(>init_done,
-timeout)) {
-   dev_err(dev, "Passive init timeout\n");
-   goto task_failed;
-   }
-   }
-   } while (adapter->renegotiate);
-   rc = init_sub_crq_irqs(adapter);
-
-   if (rc)
-   goto task_failed;
-
-   netdev->real_num_tx_queues = adapte

[PATCH net] ibmvnic: Start completion queue negotiation at server-provided optimum values

[John Allen]

Use the opt_* fields to determine the starting point for negotiating the
number of tx/rx completion queues with the vnic server. These contain the
number of queues that the vnic server estimates that it will be able to
allocate. While renegotiation may still occur, using the opt_* fields will
reduce the number of times this needs to happen and will prevent driver
probe timeout on systems using large numbers of ibmvnic client devices per
vnic port.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index d54405b4..ee66164 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1493,9 +1493,8 @@ static void init_sub_crqs(struct ibmvnic_adapter 
*adapter, int retry)
adapter->max_rx_add_entries_per_subcrq > entries_page ?
entries_page : adapter->max_rx_add_entries_per_subcrq;

-   /* Choosing the maximum number of queues supported by firmware*/
-   adapter->req_tx_queues = adapter->max_tx_queues;
-   adapter->req_rx_queues = adapter->max_rx_queues;
+   adapter->req_tx_queues = adapter->opt_tx_comp_sub_queues;
+   adapter->req_rx_queues = adapter->opt_rx_comp_queues;
adapter->req_rx_add_queues = adapter->max_rx_add_queues;

adapter->req_mtu = adapter->max_mtu;

Re: [PATCH net 3/4] ibmvnic: simplify and improve driver probe function

[John Allen]

On 07/06/2016 03:35 PM, Thomas Falcon wrote:
> @@ -3560,6 +3608,7 @@ static const struct file_operations ibmvnic_dump_ops = {
> 
>  static int ibmvnic_probe(struct vio_dev *dev, const struct vio_device_id *id)
>  {
> + unsigned long timeout = msecs_to_jiffies(3);
>   struct ibmvnic_adapter *adapter;
>   struct net_device *netdev;
>   unsigned char *mac_addr_p;
> @@ -3638,30 +3687,26 @@ static int ibmvnic_probe(struct vio_dev *dev, const 
> struct vio_device_id *id)
>   ibmvnic_send_crq_init(adapter);
> 
>   init_completion(>init_done);
> - wait_for_completion(>init_done);
> + if (!wait_for_completion_timeout(>init_done, timeout))
> + return 0;
> 
>   do {
> - adapter->renegotiate = false;
> -
> - init_sub_crqs(adapter, 0);
> - reinit_completion(>init_done);
> - wait_for_completion(>init_done);
> -
>   if (adapter->renegotiate) {
> - release_sub_crqs(adapter);
> + adapter->renegotiate = false;
> + release_sub_crqs_no_irqs(adapter);
>   send_cap_queries(adapter);
> 
>   reinit_completion(>init_done);
> - wait_for_completion(>init_done);
> + if (!wait_for_completion_timeout(>init_done,
> +  timeout))
> + return 0;
>   }
>   } while (adapter->renegotiate);

Instead of a do-while here, this should just be simplified to a while loop.

while (adapter->renegotiate) {
...
}

will serve the same purpose as

do {
if (adapter->renegotiate) {
...
}
} while (adapter->renegotiate);

Same goes for the similar loop in patch 4/4.

Re: [PATCH net-next] ibmvnic: Defer tx completion processing using a wait queue

[John Allen]

On 04/12/2016 03:12 PM, Eric Dumazet wrote:
> On Tue, 2016-04-12 at 14:38 -0500, John Allen wrote:
>> Moves tx completion processing out of interrupt context, deferring work
>> using a wait queue. With this work now deferred, we must account for the
>> possibility that skbs can be sent faster than we can process completion
>> requests in which case the tx buffer will overflow. If the tx buffer is
>> full, ibmvnic_xmit will return NETDEV_TX_BUSY and stop the current tx
>> queue. Subsequently, the queue will be restarted in ibmvnic_complete_tx
>> when all pending tx completion requests have been cleared.
> 
> 1) Why is this needed ?

In the current ibmvnic implementation, tx completion processing is done in
interrupt context. Depending on the load, this can block further
interrupts for a long time. This patch just creates a bottom half so that
when a tx completion interrupt comes in, we can defer the majority of the
work and exit interrupt context quickly.

> 
> 2) If it is needed, why is this not done in a generic way, so that other
> drivers can use this ?

I'm still fairly new to network driver development so I'm not in tune with
the needs of other drivers. My assumption was that the wait queue data
structure was a reasonably generic way to handle something like this. Is
there a more appropriate/generic way of implementing a bottom half for
this purpose?

-John
> 
> Thanks.
> 
> 
> 
> 

[PATCH net-next] ibmvnic: Defer tx completion processing using a wait queue

[John Allen]

Moves tx completion processing out of interrupt context, deferring work
using a wait queue. With this work now deferred, we must account for the
possibility that skbs can be sent faster than we can process completion
requests in which case the tx buffer will overflow. If the tx buffer is
full, ibmvnic_xmit will return NETDEV_TX_BUSY and stop the current tx
queue. Subsequently, the queue will be restarted in ibmvnic_complete_tx
when all pending tx completion requests have been cleared.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 864cb21..641e340 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -105,6 +105,7 @@ static int pending_scrq(struct ibmvnic_adapter *,
struct ibmvnic_sub_crq_queue *);
 static union sub_crq *ibmvnic_next_scrq(struct ibmvnic_adapter *,
struct ibmvnic_sub_crq_queue *);
+static int ibmvnic_tx_work(void *data);
 static int ibmvnic_poll(struct napi_struct *napi, int data);
 static void send_map_query(struct ibmvnic_adapter *adapter);
 static void send_request_map(struct ibmvnic_adapter *, dma_addr_t, __be32, u8);
@@ -437,6 +438,17 @@ static int ibmvnic_open(struct net_device *netdev)

tx_pool->consumer_index = 0;
tx_pool->producer_index = 0;
+
+   init_waitqueue_head(_pool->ibmvnic_tx_comp_q);
+   tx_pool->work_thread =
+   kthread_run(ibmvnic_tx_work, adapter->tx_scrq[i],
+   "%s_%s_%d",
+   IBMVNIC_NAME, adapter->netdev->name, i);
+   if (IS_ERR(tx_pool->work_thread)) {
+   dev_err(dev, "Couldn't create kernel thread: %ld\n",
+   PTR_ERR(tx_pool->work_thread));
+   goto thread_failed;
+   }
}
adapter->bounce_buffer_size =
(netdev->mtu + ETH_HLEN - 1) / PAGE_SIZE + 1;
@@ -477,6 +489,9 @@ bounce_map_failed:
 bounce_alloc_failed:
i = tx_subcrqs - 1;
kfree(adapter->tx_pool[i].free_map);
+thread_failed:
+   for (j = 0; j < i; j++)
+   kthread_stop(adapter->tx_pool[j].work_thread);
 tx_fm_alloc_failed:
free_long_term_buff(adapter, >tx_pool[i].long_term_buff);
 tx_ltb_alloc_failed:
@@ -731,6 +746,16 @@ static int ibmvnic_xmit(struct sk_buff *skb, struct 
net_device *netdev)
}

index = tx_pool->free_map[tx_pool->consumer_index];
+   /* tx queue full */
+   if ((index + 1) % adapter->max_tx_entries_per_subcrq ==
+   tx_pool->free_map[tx_pool->producer_index]) {
+   netif_tx_stop_queue(netdev_get_tx_queue(netdev, queue_num));
+   tx_send_failed++;
+   tx_dropped++;
+   ret = NETDEV_TX_BUSY;
+   goto out;
+   }
+
offset = index * adapter->req_mtu;
dst = tx_pool->long_term_buff.buff + offset;
memset(dst, 0, adapter->req_mtu);
@@ -1314,6 +1339,7 @@ static int ibmvnic_complete_tx(struct ibmvnic_adapter 
*adapter,
 {
struct device *dev = >vdev->dev;
struct ibmvnic_tx_buff *txbuff;
+   struct netdev_queue *txq;
union sub_crq *next;
int index;
int i, j;
@@ -1361,6 +1387,10 @@ restart_loop:
next->tx_comp.first = 0;
}

+   txq = netdev_get_tx_queue(adapter->netdev, scrq->pool_index);
+   if (netif_tx_queue_stopped(txq))
+   netif_tx_wake_queue(txq);
+
enable_scrq_irq(adapter, scrq);

if (pending_scrq(adapter, scrq)) {
@@ -1371,13 +1401,35 @@ restart_loop:
return 0;
 }

+static int ibmvnic_tx_work(void *data)
+{
+   int rc;
+   struct ibmvnic_sub_crq_queue *scrq = data;
+   struct ibmvnic_adapter *adapter = scrq->adapter;
+
+   while (1) {
+   rc = wait_event_interruptible(adapter->
+ tx_pool[scrq->pool_index].
+ ibmvnic_tx_comp_q,
+ pending_scrq(adapter, scrq));
+   BUG_ON(rc);
+
+   if (kthread_should_stop())
+   break;
+
+   disable_scrq_irq(adapter, scrq);
+
+   ibmvnic_complete_tx(adapter, scrq);
+   }
+   return 0;
+}
+
 static irqreturn_t ibmvnic_interrupt_tx(int irq, void *instance)
 {
struct ibmvnic_sub_crq_queue *scrq = instance;
struct ibmvnic_adapter *adapter = scrq->adapter;

-   disable_scrq_irq(adapter, scrq);
-   ibmvnic_complete_tx(adapter, scrq);
+   wake_up(>tx_pool[scrq->pool_index].ibmvnic_tx_comp_q);

return IRQ_HANDLED;
 }

[PATCH net-next] ibmvnic: Enable use of multiple tx/rx scrqs

[John Allen]

Enables the use of multiple transmit and receive scrqs allowing the ibmvnic
driver to take advantage of multiqueue functionality. To achieve this, the
driver must implement the process of negotiating the maximum number of
queues allowed by the server. Initially, the driver will attempt to login
with the maximum number of tx and rx queues supported by the server. If
the server fails to allocate the requested number of scrqs, it will return
partial success in the login response. In this case, we must reinitiate
the login process from the request capabilities stage and attempt to login
requesting fewer scrqs.

Signed-off-by: John Allen <jal...@linux.vnet.ibm.com>
---
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c 
b/drivers/net/ethernet/ibm/ibmvnic.c
index 21bccf6..6e9b91d 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -800,11 +800,12 @@ static int ibmvnic_xmit(struct sk_buff *skb, struct 
net_device *netdev)
ret = NETDEV_TX_BUSY;
goto out;
}
-   lpar_rc = send_subcrq_indirect(adapter, handle_array[0],
+   lpar_rc = send_subcrq_indirect(adapter, handle_array[queue_num],
   (u64)tx_buff->indir_dma,
   (u64)num_entries);
} else {
-   lpar_rc = send_subcrq(adapter, handle_array[0], _crq);
+   lpar_rc = send_subcrq(adapter, handle_array[queue_num],
+ _crq);
}
if (lpar_rc != H_SUCCESS) {
dev_err(dev, "tx failed with code %ld\n", lpar_rc);
@@ -989,7 +990,7 @@ restart_poll:
netdev->stats.rx_bytes += length;
frames_processed++;
}
-   replenish_pools(adapter);
+   replenish_rx_pool(adapter, >rx_pool[scrq_num]);

if (frames_processed < budget) {
enable_scrq_irq(adapter, adapter->rx_scrq[scrq_num]);
@@ -1426,9 +1427,9 @@ static void init_sub_crqs(struct ibmvnic_adapter 
*adapter, int retry)
entries_page : adapter->max_rx_add_entries_per_subcrq;

/* Choosing the maximum number of queues supported by firmware*/
-   adapter->req_tx_queues = adapter->min_tx_queues;
-   adapter->req_rx_queues = adapter->min_rx_queues;
-   adapter->req_rx_add_queues = adapter->min_rx_add_queues;
+   adapter->req_tx_queues = adapter->max_tx_queues;
+   adapter->req_rx_queues = adapter->max_rx_queues;
+   adapter->req_rx_add_queues = adapter->max_rx_add_queues;

adapter->req_mtu = adapter->max_mtu;
}
@@ -1776,13 +1777,11 @@ static void send_login(struct ibmvnic_adapter *adapter)
goto buf_map_failed;
}

-   rsp_buffer_size =
-   sizeof(struct ibmvnic_login_rsp_buffer) +
-   sizeof(u64) * (adapter->req_tx_queues +
-  adapter->req_rx_queues *
-  adapter->req_rx_add_queues + adapter->
-  req_rx_add_queues) +
-   sizeof(u8) * (IBMVNIC_TX_DESC_VERSIONS);
+   rsp_buffer_size = sizeof(struct ibmvnic_login_rsp_buffer) +
+ sizeof(u64) * adapter->req_tx_queues +
+ sizeof(u64) * adapter->req_rx_queues +
+ sizeof(u64) * adapter->req_rx_queues +
+ sizeof(u8) * IBMVNIC_TX_DESC_VERSIONS;

login_rsp_buffer = kmalloc(rsp_buffer_size, GFP_ATOMIC);
if (!login_rsp_buffer)
@@ -2401,6 +2400,16 @@ static int handle_login_rsp(union ibmvnic_crq 
*login_rsp_crq,
dma_unmap_single(dev, adapter->login_rsp_buf_token,
 adapter->login_rsp_buf_sz, DMA_BIDIRECTIONAL);

+   /* If the number of queues requested can't be allocated by the
+* server, the login response will return with code 1. We will need
+* to resend the login buffer with fewer queues requested.
+*/
+   if (login_rsp_crq->generic.rc.code) {
+   adapter->renegotiate = true;
+   complete(>init_done);
+   return 0;
+   }
+
netdev_dbg(adapter->netdev, "Login Response Buffer:\n");
for (i = 0; i < (adapter->login_rsp_buf_sz - 1) / 8 + 1; i++) {
netdev_dbg(adapter->netdev, "%016lx\n",
@@ -3627,15 +3636,22 @@ static int ibmvnic_probe(struct vio_dev *dev, const 
struct vio_device_id *id)

init_completion(>init_done);
wait_for_completion(>init_done);
+
+   do {
+   adapter->renegotiate = false;

-   /* needed to pull init_sub_crqs outside of an interrupt context
-* because it creates IRQ mappings for the subCRQ queues, causing
-