Do you want the debug from the IO, Compute, or Both? On Thu, Jul 10, 2008 at 7:01 AM, Eli Cohen <[EMAIL PROTECTED]> wrote:
> On Thu, Jul 10, 2008 at 04:30:11AM -0400, Michael Di Domenico wrote: > > I'm having a bit of a weird problem that i cannot figure out. If anyone > can > > help from the community it would be appreciated. > > Here's the packet flow > > > > cn(ib0)->io(ib0)->io(eth5)->pan(*) > > > > cn = compute node > > io = io node > > pan = panasas storage network > > > > We have 12 shelves of panasas network storage on a seperate network, > which > > is being fronted by bridge servers which are routing IPoIB traffic to 10G > > ethernet traffic. We're using Mellanox Connect-X Ethernet/IB adapters > > everwhere. We're running Ofed 1.3.1 and the latest firmwares for IB/Eth > > everywhere. > > > > Here's the problem. I can mount the storage on the compute nodes, but if > i > > try to send anything more then 50MB of data via dd. I seem to loose the > ARP > > entries for the compute nodes on the IO servers. This seems to happen > > whether I use the filesystem or a netperf run from the compute node to > the > > panasas storage > > > > I can run netperf between the compute node and io node and get full IPoIB > > line rate with no issues > > I can run netperf between the io node and the panasas storage and get > full > > 10G ethernet line rate with no issues > > > > When looking at the TCP traces, i can clearly see that a big chunk of > data > > is sent between the end-points and then it stalls. Immediately after the > > stall is an ARP request and then another chunk of data, and this scenario > > repeats over and over. > > > > Any thoughts or questions? > > > > Michael, > could you repeat the experiment with debugging enabled? For IPoIB, this > can be done as follows: > > echo 1 > /sys/module/ib_ipoib/parameters/debug_level > echo 1 > /sys/module/ib_ipoib/parameters/macast_debug_level > > Please send the output of dmesg after the failure. > > Thanks. >
_______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
