terry watson a e'crit :
Hi all,
I will be performing some testing of partitioning used as a security control.
Am I right in believing that IBIS will be able to set partition table values of
the local compute node I am logged on to, even though they are not using
OpenSM, but rather a SM on a switch? Could I then attempt to access a partition
that I was originally excluded from accessing?
I am new to Infiniband technology and would also appreciate a response from an
expert who has views on the strength of the security that partitioning provides
in separating two clusters that should have no interaction whatsoever.
Thanks,
Dave
_________________________________________________________________
Discover the new Windows Vista
http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
The partitions are only managed by the subnet manager - either opensm
running on a node into the fabric or an embedded subnet manager on a switch.
For opensm , partitions are defined into a configuration file
/etc/opensm/partitions.conf, for a embedded subnet manager, you have to
configure the partitions using the CLI or GUI provided by the switch.
Defining a partition is mainly choosing a pkey and ports nodes with
their membership (limited or not).
The subnet manager assigned the pkeys to the ports of the node when ib
kernel modules are loaded. You can see the partitions the IB port belong
to by ( I mean those defined by the subnet manager) :
# grep -v 0x0000 /sys/class/infiniband/mthca0/ports/1/pkeys/*
/sys/class/infiniband/mthca0/ports/1/pkeys/0:0xffff
/sys/class/infiniband/mthca0/ports/1/pkeys/1:0x8001
/sys/class/infiniband/mthca0/ports/1/pkeys/2:0x8002
/sys/class/infiniband/mthca0/ports/1/pkeys/3:0x8003
/sys/class/infiniband/mthca0/ports/1/pkeys/4:0x8010
A port may belong to many partitions. Nodes (ports) may have different
partitions configurations. Partitions order for a port is not always the
same ( it may depend on the chronology of partition declarations in the
subnet manager)
Over these partitions, you can define new IP (IP over IB) interfaces by
creating files like /etc/sysconfig/network-scripts/ifcfg-ib0.8002 :
# cat /etc/sysconfig/network-scripts/ifcfg-ib0.8002
DEVICE=ib0.8002
BOOTPROTO=static
IPADDR=XXX.YYY.ZZZ.TTT
NETMASK=255.255.255.0
NETWORK=255.255.255.0
ONBOOT=yes
The openibd script create the child interface and configure it at system
startup using some special devices to do that :
echo $pkey > /sys/class/net/ib0/create_child
But this command creates only a child interface on the node, but
communications on this interface will not work until you add the port
node to the corresponding partition into the subnet manager
configuration. Then you will see the pkey appearing automatically into
files /sys/class/infiniband/mthca0/ports/1/pkeys/* on the node.
[EMAIL PROTECTED] ~]# echo 0x8009 > /sys/class/net/ib0/create_child
[EMAIL PROTECTED] ~]# dmesg | grep 8009
divert: not allocating divert_blk for non-ethernet device ib0.8009
[EMAIL PROTECTED] ~]# grep -v 0x0000
/sys/class/infiniband/mthca0/ports/1/pkeys/*
/sys/class/infiniband/mthca0/ports/1/pkeys/0:0xffff
/sys/class/infiniband/mthca0/ports/1/pkeys/1:0x8001
/sys/class/infiniband/mthca0/ports/1/pkeys/2:0x8002
/sys/class/infiniband/mthca0/ports/1/pkeys/3:0x8003
/sys/class/infiniband/mthca0/ports/1/pkeys/4:0x8010
[EMAIL PROTECTED] ~]# ifconfig -a | grep 8009
ib0.8009 Link encap:UNSPEC HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
[EMAIL PROTECTED] ~]# echo 0x8009 > /sys/class/net/ib0/delete_child
[EMAIL PROTECTED] ~]# dmesg | grep 8009
divert: not allocating divert_blk for non-ethernet device ib0.8009
divert: no divert_blk to free, ib0.8009 not ethernet
To use MPI with partitions, you have also to configure it (in the
configuration file) . For MVAPICH you must use VIADEV_DEFAULT_PKEY_IX
or VIADEV_DEFAULT_PKEY in the config file :
/usr/mpi/gcc/mvapich-1.0.0/etc/mvapich.conf . AT CEA, I'm using
VIADEV_DEFAULT_PKEY (pkey value)
as we have nodes with different partitions configurations.
Hoping this will help you.
Regards
Philippe Gregoire CEA/DAM
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
