Hello.
I'm going to raise a different, although slightly related issue; please
advise me if it should be sent off-list or create another "thread".
I am also using Topspin IB PCI-X cards and TS120 switch, with Scientific
Linux EL4.2 (2.6.9-11.ELsmp) on Opteron x86_64 - HP DL145G2 nodes -
(Vikrant is SLES9 a typo? is it Scientific Linux EL?).
IPoIB is running fine (I get around 680MB/s from netperf at 70% CPU usage)
but as sson as I try to run anything that uses IB SDP (I think it uses
RDMA), such as tests in topspin/bin/ or topspin/sbin/hca_stress, the
machines where the test is run crash with an IB module oops
So I am running PVFS but on the Gb boards; I don't want to use PVFS-IB
while the IB stack is "unreliable"...
Is someone experience the same troubles?
I have been advised to replace the Topspin/Cisco with the openIB stack,
but I had yet no window to halt the nodes...
Best Regards,
paulo lopes
> Hello Pete
>
> Thanks for the info. I think we'll have to stick to one protocol only in
> the face of these issues.
>
> The pvfs2-IB cluster is up finally and there is some speedup as compared
> to a pvfs2-TCP setup on a different cluster. But Im seeing one strange
> thing on the IB setup: the pvfs2-client-core is using up 100% CPU even
> in the absence of file-system activity. I saw on pvfs2 FAQs one entry
> related to pvfs2-server CPU usage, but none related to
> pvfs2-client-core. Have other people run into a similar problem? Note
> that on the pvfs2-TCP cluster, which is similarly configured except for
> not having Infiniband, pvfs2-client-core is quite well behaved.
>
> The pvfs2 is setup as follows:
> IO servers: node{0-3}
> metadata servers : node0
> protocol : IB
> clients: node{0-3}
> note that Im running the clients on the same machines as the servers,
> each a dual proc.
> OS : Linux SLES9 x86_64
> kernel: 2.6.5-7.244-smp
>
> Let me know if anyone has faced this problem or has a solution.
>
> Thanks
> Vikrant
--
Paulo Afonso Lopes | Tel: +351- 21 294 8536
Departamento de Informática | 294 8300 ext.10763
Faculdade de Ciências e Tecnologia | Fax: +351- 21 294 8541
Universidade Nova de Lisboa | e-mail: [EMAIL PROTECTED]
2829-516 Caparica, PORTUGAL
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
