>I'm confused. Which one is faster?
Sorry for the confusion, Michael. The one with return 1 has better
throughput.
Thanks
Shirley Ma___
general mailing list
[EMAIL PROTECTED]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscri
_MCAST_RUN, &priv->flags);
- netif_carrier_on(dev);
}
int ipoib_mcast_start_thread(struct net_device *dev)
(See attached file: ipoib-link.patch)
Thanks
Shirley Ma
ipoib-link.patch
Description: Binary data
___
openib-general mailing list
oland Dreier <[EMAIL PROTECTED]> wrote on 02/27/2007 02:41:44 PM:
> > So the IBV_CQ_REPORT_MISSED_EVENTS has been part of OFED-1.2 already?
I can
> > generate the patch for all ULPs to use this for review. Do you need me
to
> > do that?
>
> No, it's not in OFED 1.2 or the upstream kernel.
Roland Dreier <[EMAIL PROTECTED]> wrote on 02/27/2007 02:35:34 PM:
> I don't think this applies any more since Sean's multicast stuff was
> merged. I didn't realize you wanted to get this merged upstream --
> anyway, can you please regenerate the patch against the latest kernel?
>
> Thanks
Su
Roland Dreier <[EMAIL PROTECTED]> wrote on 02/26/2007 02:36:26 PM:
> No way, it's way too late at this point to change the kernel-user ABI,
> let alone change all ULPs.
>
> - R.
Hello Roland,
So the IBV_CQ_REPORT_MISSED_EVENTS has been part of OFED-1.2 already? I can
generate the patch for a
or default IPv6 link local
solicited address) when bringing the interface up. It does impact IPoIB
usability in large node cluster when MCG LIDs are limited.
Thanks
Shirley Ma
- Forwarded by Shirley Ma/Beaverton/IBM on 02/27/0
Roland Dreier <[EMAIL PROTECTED]> wrote on 02/27/2007 01:40:36 PM:
> Shirley, can you clarify why doing dma_alloc_coherent() in the kernel
> helps on your Cell blade? It really seems that dma_alloc_coherent()
> just allocates some memory and then does dma_map(DMA_BIDIRECTIONAL),
> which woul
Roland Dreier <[EMAIL PROTECTED]> wrote on 02/26/2007 02:09:48 PM:
> > That would be great. We hit a similar problem in our cluster test --
data
> > corruption because of this race.
>
> On what platform?
>
> - R.
On our cell blade + PCI-e Mellanox.
Thanks
Shirley Ma
> Hmm, OK. Then I will do my best to make sure we get a fix for this
> into 2.6.22.
That would be great. We hit a similar problem in our cluster test -- data
corruption because of this race.
Thanks
Shirley Ma___
openib-general mailing list
openib-g
Roland,
Yes. It would be good to reduce number of interrupts by changing all upper
layer protocols to use:
poll CQ
notify CQ, rotting packet notification
poll again
instead of
notify CQ
poll CQ
If possible this can be in OFED-1.2?
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW
If your subnet is already has a SM running. Please look at the ifconfig
output. If the interface ib0 is UP but not RUNNING, you can't ping since
the carrier is not ON. Also look at /var/log/messages to see whether there
is any errors.
Thanks
Shirley Ma
IBM Linux Technology Center
153
Roland Dreier <[EMAIL PROTECTED]> wrote on 02/16/2007 01:55:06 PM:
> > Two MCGs groups must be establised before IPoIB link up, one is
broadcast
> > for IPv4, one is all hosts multicast for IPv6. So Node A is a member
of all
> > hosts address, the patch directs ND sends to all hosts, so no
Roland Dreier <[EMAIL PROTECTED]> wrote on 02/16/2007 09:25:30 AM:
> > Even SM supports 1000 MCGs, it's still not sufficitent for 250 nodes
> > cluster, each node have 4 links for IPv6 without any scope/global IPv6
> > address configured.(250*4+ a few default MCGs) There will be a MCG
over
before IPoIB link up, one is broadcast
for IPv4, one is all hosts multicast for IPv6. So Node A is a member of all
hosts address, the patch directs ND sends to all hosts, so node A responses
it.
Thanks
Shirley Ma
___
openib-general mailing list
openib
Roland Dreier <[EMAIL PROTECTED]> wrote on 02/16/2007 09:49:24 AM:
> > For the successful join, ND sends to the node directly, for the
failure
> > join, ND sends to all hosts addr. So ND will work no matter whether
the
> > join OK or not, that's the patch does.
>
> But what if the full-memb
Roland,
>I really don't like this approach, since it can break things in very
>subtle ways (eg suppose one node fails to join its solicited node
>group, but then a later node wants to talk to it and succeeds in
>joining the solicited node group as a send-only member -- since the
>first node is
hanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638
Roland Dreier
<[EMAIL P
Hello, Roland,
We have a customer issue regarding IPv6oIB. In the subnet, there are
limited number of MCGs supported. So when there are multiple IPv6 addresses
are assigned to one interface, each IPv6 address will have one unique
solicited-node address (depends on their groupID). Then in a la
fully, but
all hosts multicast group join failure. Then ib interface can be UP, but
not RUNNING. The interface couldn't work at all.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503
Roland,
Could you please review this patch when you have time? I am looking
forward to seeing your comments to address a customer issue. Appreciate
your help.
Thanks
Shirley Ma___
openib-general mailing list
openib-general@openib.org
http://op
in_task(void *dev_ptr
ipoib_dbg_mcast(priv, "successfully joined all multicast groups\n");
clear_bit(IPOIB_MCAST_RUN, &priv->flags);
- netif_carrier_on(dev);
}
int ipoib_mcast_start_thread(struct net_device *dev)
(See attached file: ipoib-multicast.patch)
Thanks
Shirley Ma
IBM
IPoIB link other than that determined by the
broadcast group membership.
Shirley Ma
Shirley
Ma/Beaverton/IBM@
When bringing IPoIB interface up, I hit default group multicast join
failure. (This could be fixed in SM set up?)
ib0: multicast join failed for , status -22
Then the interface was UP but not RUNNING. So the nodes couldn't ping each
other. I think the right behavior of the interface should
Roland Dreier <[EMAIL PROTECTED]> wrote on 11/16/2006 11:26:31 AM:
> > What I have found in ehca driver, n! = t, does't mean it's empty. If
poll
> > again, there are still some packets in cq. IB_CQ_REPORT_mISSED_EVENTS
most
> > of the time reports 1. It relies on netif_rx_reschedule() retu
>I will rerun my test to use n! = 0 to see any difference here.
It should be n == 0 to indicate empty.
Thanks
Shirley Ma___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please v
Roland Dreier <[EMAIL PROTECTED]> wrote on 11/14/2006 03:18:23 PM:
> Shirley> The rotting packet situation consistently happens for
> Shirley> ehca driver. The napi could poll forever with your
> Shirley> original patch. That's the reason I defer the rotting
> Shirley> packet proc
From the code work through, if defering rotting packet process by return (missed_event && netif_rx_reschedule(dev, 0)); Then the same dev->poll can be added to per cpu poll list twice: one is from netif_rx_reschedule, one is from napi return 1. That might explains packets out of order: when one po
Roland,
Ignore my previous email, test_and_set_bit is atomic operation and has the memeory barrier already.
Thanks
Shirley Ma___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe,
Roland,
I think there is a barrier might be needed in checking LINK SCHED state, like smp_mb_before_clear_bit() and smp_mb_after_clear_bit(), otherwise the netif_rx_reschedule() for rotting packet and next interrupt netif_rx_schedule() could be running in the time. If the interrupt is round-robin
Roland Dreier <[EMAIL PROTECTED]> wrote on 11/13/2006 08:45:52 AM:
> > Sorry I was not intend to send previous email. Anyway I accidently sent it
> > out. What I thought was there would be a problem, if the missed_event
> > always return to 1. Then this napi poll would keep forever.
>
> Well,
Roland,
Sorry I was not intend to send previous email. Anyway I accidently sent it out. What I thought was there would be a problem, if the missed_event always return to 1. Then this napi poll would keep forever. How about defer the rotting packets process later? like this:
>
> + if
p; netif_rx_reschedule(dev, 0);
> +
Thanks
Shirley Ma
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
> I would really like to understand why ehca does worse with NAPI. In
> my tests both mthca and ipath exhibit various degrees of improvement
> depending on the test -- but I've never seen performance get worse.
> This is the main thing holding back merging NAPI.
>
> Does the NAPI patch help mthc
XT_COMP, &missed_event);
+ if (unlikely(missed_event) && netif_rx_reschedule(dev, 0))
+ goto repoll;
+ netif_rx_complete(dev);
+
+ return 0;
+ }
Any other reason, calling netif_rx_complete() while still possibably within napi?
Thanks
Shirley Ma
IBM Linux Technology
nk throughput with one UD QP. Now I couldn't get it anymore on the new kernel. I was struggling with TCP window limits on the new kernel. Do you have any hint?
thanks
Shirley Ma
IBM Linux Technology Center
___
openib-general mailing list
openib-gen
Retest several times, this hack patch only fixed the none scaling code. I thought I tested both scaling and none scaling, it seems I made a mistake, I might configure and test none scaling configuration twice in previous run.
thanks
Shirley Ma
IBM Linux Technology Center
Someone who knows how
> ehca works might be able to spot the error.
>
> - R.
The oprofile data (with your napi + this hack patch) looks good, it reduced cpu utilization significantly. (I was wrong about cpu utilization.) I will talk with ehca team regarding this missing event hint
x27;s why it helps. This is really a hack, doesn't address the problem. It sacrificed cpu utilization and gained the performance back. I need to understand how ehca reports missing event, there might be some delay there?
Thanks
Shirley Ma
IBM Linux Technology Center_
. I will run oprofile tomorrow to see the difference. I think with these extra repolls, the cpu utilization would be much higher.
Thanks
Shirley Ma
IBM Linux Technology Center___
openib-general mailing list
openib-general@openib.org
http://openib.org
send queue size=recve queue size = 1K.
Thanks
Shirley Ma
IBM Linux Technology Center___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/lis
t; to be part of a major release.
Then we need to wait for the new release or find an alternative way which I doubt.
Thanks
Shirley Ma
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
T
Roland Dreier <[EMAIL PROTECTED]> wrote on 10/19/2006 07:51:31 AM:
> Shirley> I am not sure whether this already has an answer. The
> Shirley> justification is madvise(..., MADV_DONTFORK) is used to
> Shirley> make fork() work for verbs consumers in the recent
> Shirley> packages.
update from 1.0.3?
>
> --
> Doug Ledford <[EMAIL PROTECTED]>
I am not sure whether this already has an answer.
The justification is madvise(..., MADV_DONTFORK) is used to make fork() work for verbs consumers in the recent packages. I hope same patch will be in libehca.
thanks
Shi
thing.
That's I am trying to understand now.
Yes, the send side rate dropped significant, cpu usage lower as well.
> Does changing the netdev weight value affect anything?
>
> - R.
No, it doesn't.
Thanks
Shirley Ma
IBM Linux Technology Center___
hput performance is not good. (I am not fully conviced the huge latency is because of raising NAPI in thread context.) Then I tried ehca no scaling driver, the latency looks good, but the throughtput is still a problem. We are working on these issues. Hopefully we can get the answer so
Hi, Roland,
There were a couple errors and warning when I applied this patch to OFED-1.1-rc7.
1. ehca_req_notify_cq() in ehca_iverbs.h is not updated.
2. *maybe_missed_event = ipz_qeit_is_valid(my_cq->ipz_queue) should be =ipz_qeit_is_valid(&my_cq->ipz_queue)
3. a compile warning this line retur
Parks Fields <[EMAIL PROTECTED]> wrote on 10/17/2006 01:12:48 PM:
>
> >
> >No, it's never a good idea to turn off TCP or IP checksums. That
> >leads to possibilities of silent data corruption too easily.
>
> I totally agree...
Have we ever seen silent data corruption in CHECKSUM_HW?
Thanks
Sh
What I suggested here is when it's connected mode with large MTU, set ib interface flag to CHECKSUM_UNNECESSARY. But this only works on packets not being routed off-net at the TCP layer.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax):
"Michael S. Tsirkin" <[EMAIL PROTECTED]> wrote on 10/16/2006 11:12:03 PM:
> Quoting r. Shirley Ma <[EMAIL PROTECTED]>:
> > /* can be added later once ipoib support sg
> > .get_sg = ethtool_op_get_sg,
> > .set_sg = ethtool_op_set_sg,
> > */
>
I am going to add below ethtool ops in ipoib. Anything comments? Once ethtool support is added, GSO will be get/set directly through ethtool as Michael pointed out earlier.
static struct ethtool_ops ipoib_ethtool_ops = {
.get_settings = ipoib_get_settings,
.set_settings
Roland,
Don't know why I have trouble to get this patch from your git tree. Do you mind to post this patch here so I can test the performance over ehca?
Thanks
Shirley Ma___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailma
ing correctly -- this is other work that is
> independent of GSO? Is the plan to add a new optional driver method
> that extends hard_start_xmit() to accept multiple packets?
>
> - R.
Yes, you are right. It is the new work indendent of GSO. I hope I have the BW to do all the work o
Roland Dreier <[EMAIL PROTECTED]> wrote on 10/16/2006 10:37:12 AM:
> Shirley> Good. Then after enabling GSO, we can chain multiple
> Shirley> packets together in IPoIB for one doorbell to send large
> Shirley> packet.
>
> How does that work? GSO doesn't change the hard_start_xmit()
Good. Then after enabling GSO, we can chain multiple packets together in IPoIB for one doorbell to send large packet.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638___
openib
Hi, Roland,
If we only support GSO enablement in ethtool, there is no problem. What I meant is anything related to MAC address in ethtool utility needs to be updated for IB device.
Do you like the idea to add ethtool support in IPoIB? Do you want me to work on this?
Thanks
Shirley Ma___
[EMAIL PROTECTED] wrote on 09/28/2006 09:11:47 AM:
> Michael> Looked pretty simple on the outset, but oh well. Keep us
> Michael> posted.
>
> I just work slowly.
>
> Anyway I don't think this is that urgent -- we've dumped enough stuff
> into 2.6.19, so I think this should wait for 2.6.
I have created a patch to monitor CQ. That wasn't the reason for performance drop. I couldn't see any race from the output.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638_
lity to support ibX interface. Now it only supports ethX.
thanks
Shirley Ma
IBM Linux Technology Center___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openi
Hi, Eli,
Eli cohen <[EMAIL PROTECTED]> wrote on 09/26/2006 11:35:26 PM:
> On Tue, 2006-09-26 at 21:34 -0700, Shirley Ma wrote:
>
> > NAPI patch moves ipoib poll from hardware interrupt context to softirq
> > context. It would reduce the hardware interrupts, reduce
"Michael S. Tsirkin" <[EMAIL PROTECTED]> wrote on 09/26/2006 11:23:16 PM:
> Quoting r. Shirley Ma <[EMAIL PROTECTED]>:
> > We
> > are implementing multiple EQs suport for one adapter now.
>
> I think with MSI we can have a per-interface EQ in mthca.
&g
"Michael S. Tsirkin" <[EMAIL PROTECTED]> wrote on 09/26/2006 09:59:30 PM:
> I still hope ehca NAPI performance can be fixed. But if not, maybe we should
> have the low level driver set a disable_napi flag rather than have users play
> with module options.
>
> --
> MST
I forgot to mention these N
"Michael S. Tsirkin" <[EMAIL PROTECTED]> wrote on 09/26/2006 09:59:30 PM:
> Quoting r. Shirley Ma <[EMAIL PROTECTED]>:
> > Subject: Re: [PATCH] IB/ipoib: NAPI
> >
> > We did some touch test on ehca driver, we saw performance drop somehow.
>
&g
Hi, Roland,
> Shirley> It can be a configuration option to enable/disable NAPI,
> Shirley> just like other network device.
>
> But is there any reason to keep the non-NAPI code around? I hate to
> have two codepaths to maintain.
If you would like to maintain one code path only, then w
utilization. But I still question about the BW improvement. I did see various performance with the same test under the same condition.
Have you tested this patch with different message sizes, different socket sizes? Are these results consistent better?
Thanks
Shirley Ma
IBM Li
> Shirley> Since linux 2.6.18 supports GSO, I have patched IPoIB to
> Shirley> enable GSO, but haven't tested the performance yet. Has
> Shirley> anyone tried already?
>
> No, I don't think anyone looked at that yet. Could you post your
> patch? What is required? Supporting gather/
As will generate a CQ event in this case, although it's not
> strictly required by the IB spec. How will ehca behave?
>
> - R.
That could be the reason. I did see mthca poll empty entry, but not on ehca. I will confirm this with ehca team.
Thanks
Shirley Ma
IBM Linux Technology Cent
We did some touch test on ehca driver, we saw performance drop somehow. I strongly recommand NAPI as a configurable option in ipoib. So customers can turn on/off based on their configurations.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax
to build the trunk on 2.6.18.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
Since linux 2.6.18 supports GSO, I have patched IPoIB to enable GSO, but haven't tested the performance yet. Has anyone tried already?
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503
> This patch implements NAPI for iopib. It is a draft implementation.
> I would like your opinion on whether we need a module parameter
> to control if NAPI should be activated or not.
It can be a configuration option to enable/disable NAPI, just like other network device.
Thanks
Shirl
Roland,
We had a simple version of NAPI patch. We saw the performance improvement on mthca but not ehca. We will test this NAPI patch on ehca when it's available to see how's the performance.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Roland,
>Can you post a recipe to reproduce the crash?
It happened on 32 nodes cluster (each
node has 8 dual core cpus) running IBM applications over IPoIB.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503)
Roland,
We have seen several skb panic under
heavy stress 48 hour test. I wonder whether there are duplicated or corrupted
cookies received from device driver to reuse skb buff, since skb buff ring
is indexed by wr_id.
Is that possible?
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW
Michael,
I will apply this patch. This patch would reduce the race, not address the problem.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638___
openib-general mailing list
openib
Ohmm. That's a myth. So this problem
is hardware independent, right?
It's not easy to reproduce it. ifconfig
up and down stress test could hit this problem occasionally.
thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax):
On Fri, 2006-06-02 at 16:15 -0700, Roland Dreier wrote:
> > 2. during unicast arp skb retransmission, unicast_arp_send() appended
> > the skb on the list, while ipoib_flush_paths() calling path_free() to
> > free the same skb from the list.
>
> I think I see what's going on. the skb ends up be
ff *skb)
{
if (skb->list) {
printk(KERN_WARNING "Warning: kfree_skb passed an skb still "
"on a list (from %p).\n", NET_CALLER(skb));
BUG();
}
The patch will fix both problems by using priv->lock to protect
erface down. It's safe to use priv->lock, because
skb_queue_len(&path->queue) <
IPOIB_MAX_PATH_REC_QUEUE, which is 3.
Signed-off-by: Shirley Ma <[EMAIL PROTECTED]>
diff -urpN infiniband/ulp/ipoib/ipoib_main.c
infiniband-skb/ulp/ipoib/ipoib_main.c
--- infiniband/ulp/ipoi
nd_arp() and ipoib_flush_paths() when
bring interface down. It's safe to use priv->lock, because
skb_queue_len(&path->queue) <
IPOIB_MAX_PATH_REC_QUEUE, which is 3.
Here is the patch. Please review it and let me know if there is a
problem to apply this patch.
Signed-off-by: Shir
Roland Dreier <[EMAIL PROTECTED]> wrote on 05/26/2006
04:20:02 PM:
> (Also the default TX ring size is 64, not 128, isn't it?)
>
> - R.
Yes.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(
ill use my unix account to send out patches.
>Also, if you're interested, you could try the patch below and see how
it does on your tests.
Sure. I will test it after this weekend.
Did you see send queue overrun
with tx_ring default size 128?
Thanks
Shirley Ma
IBM Linux Technology Cent
in ipoib send
if (unlikely(!*to_ipoib_neigh(skb->dst->neighbour)))
kref_get();
in ipoib completion:
if (unlikely(!*to_ipoib_neigh(skb->dst->neighbour)))
ipoib_put_ah();
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phon
b->dst->neighbour)))
kref_put();
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/lis
for 5HZ, the
list is empty.
I could implment it differently, like
use RCU list with cache-friendly.
I thought it's not worth it before since
i didn't see the gain.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6
when device driver send queue is full, and depends on TCP retransmission.
This implementation would cause TCP fast trans, slow start, and packets
out of orders.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638
__
here
could improve IPoIB from 20% - 80% unidirectional and doubled bidirectional.
As you mentioned I need help to repolish
these patches. I am glad that you give all these valuable inputs of my
patches. Thanks lots here.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverto
riv->rx_ring);
@@ -896,10 +881,8 @@ void ipoib_dev_cleanup(struct net_device
ipoib_ib_dev_cleanup(dev);
kfree(priv->rx_ring);
- kfree(priv->tx_ring);
priv->rx_ring = NULL;
- priv->tx_ring = NULL;
}
static void ipoib_setup(struct net_device *dev)
@@ -
Oops, I missed one pair of spin_lock_irqsave()/spin_lock_irqrestore()
to protect send_list in ipoib_ib_handle_send_wc().
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638___
openib
ipoib_dev_cleanup(struct net_device
ipoib_ib_dev_cleanup(dev);
kfree(priv->rx_ring);
- kfree(priv->tx_ring);
priv->rx_ring = NULL;
- priv->tx_ring = NULL;
}
static void ipoib_setup(struct net_device *dev)
@@ -944,6 +927,7 @@ static void
e tx_ring
patch with atomic operation for you
to review, let's discuss the AH reap
solution later.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638
___
openib-general m
that will help. How does RCU know when it's
safe
> to free an AH?
With tx_ring removal patch, RCU can
be done.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638___
openib-gen
Roland,
My idea is to remove this AH reap thread.
We can use RCU to do the same work without lots of coding. Do you agree?
And in the reap AH code, tx_tail/tx_head
isn't consistently protected by tx_lock. It uses priv->lock.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll
ch is broken (leads to incorrect lifetimes for
> AHs), and in any case makes the send fast path slower.
>
> - R.
That's a value point. This problem will be addressed
in next tx_ring removal
patch. The kref_put was called in ipoib_ib_handle_send_wc().
Thanks
Shirley Ma
IBM Li
.)
{
...
if (path->ah)
{
kref_get(&path->ah->ref);
neigh->ah = path->ah;
ipoib_send(dev, skb, path->ah...
}
Please correct me if I am wrong.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97
hirley> prevent it to be freed?
>
> If that were true then why would we want to reference count sends
at
> all? The whole point is that a path might be destroyed before
the
> send is executed.
>
> - R.
It is true for unicast, it has a reference count before ipoib_send(). I
nee
nd
> request.
>
> - R.
I thought the path holding another AH
reference to prevent it to be freed?
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638
___
openib-general
n.c 2006-05-23
09:31:49.0 -0700
@@ -957,7 +957,6 @@ static void ipoib_setup(struct net_devic
INIT_WORK(&priv->mcast_task,
ipoib_mcast_join_task, priv->dev);
INIT_WORK(&priv->flush_task,
ipoib_ib_dev_flush, priv->dev);
Eric,
I have no problem with splitting CQ,
you can refer my IPoIB splitting CQ patch.
Could you share your code here so we
can give you some suggestions?
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578
R.
The stack is 4k, not 8K anymore. I think we can still
use IPOIB_NUM_SEND_WC as 4. I modified mthca_XXX_post_send to remove lock
totally before(since sender is exclusive), and found that lock didn't impact
performance too much.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverto
overrun
3. remove tx_ring
4. replace ipoib_reap_ah with kref_get()/kref_put()
5. remove rx_ring
6. change poll_cq from interrupt conext
to thread context, multiple threads support on both send and recv
7. tunable poll interval parameters
to sycn hardare driver
Thanks
Shirley Ma
IBM Linux Technology
1 - 100 of 337 matches
Mail list logo
