Hi
I'm seeing some strange behavior with networking inside my LDom
environment, and was hoping that somebody has experienced the same
issue. And hopefully have an idea how to fix it :-)
First, here is some background information.. The system is a T5240
running Solaris 10u5, and LDom 1.0.3. There are 5 LDoms configured,
and all are using each of three virtual switches and each switch is
connected to a single network port.
The primary domain has 8 VCPUs and 8GB of RAM allocated to it. Nothing
else is running in it. Each LDom is configured with 24 VCPUs, 3 MAUs,
at least 22GB of RAM and three virtual nic.
Every now and then a LDom will drop off our network for no apparent
reason. The LDom is still running, I am able to connect to the console
and the vnet0 interface is still plumbed. I cannot ping the LDom from
the outside or from the primary domain. Generally if I leave the
system for a period of time (say 10-15mins) everything comes back to
life.
Nothing of relevance shows up in the logs of either the LDom or the
primary domain, aside from the inability connect to our NIS or NFS
servers.
Below is a listing of the configuration. If anybody needs more
information please let me know. Any ideas would be helpful!
Scott
torsun01sx:/root# ldm list-bindings -e
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv SP 8 8G 2.8% 20h 36m
MAC
00:14:4f:e8:a9:b8
VCPU
VID PID UTIL STRAND
0 0 1.7% 100%
1 1 1.5% 100%
2 2 0.3% 100%
3 3 0.1% 100%
4 4 1.8% 100%
5 5 0.3% 100%
6 6 0.1% 100%
7 7 0.2% 100%
MAU
ID CPUSET
0 (0, 1, 2, 3, 4, 5, 6, 7)
MEMORY
RA PA SIZE
0xe000000 0xe000000 8G
IO
DEVICE PSEUDONYM OPTIONS
pci at 400 pci_0
pci at 500 pci_1
VCC
NAME PORT-RANGE
primary-vcc0 5001-5100
CLIENT PORT LDC
torld-soln02 at primary-vcc0 5001 17
torld-soln01 at primary-vcc0 5002 26
torld-qa02 at primary-vcc0 5003 33
torld-qa01 at primary-vcc0 5004 40
torld-soleng01 at primary-vcc0 5005 46
VSW
NAME MAC NET-DEV DEVICE MODE
primary-vsw1 00:14:4f:f9:ef:b6 nxge1 switch at 0
PEER MAC LDC
vnet0 at torld-soln02 00:14:4f:fb:ae:3c 11
vnet2 at torld-soln01 00:14:4f:f9:08:7e 21
vnet1 at torld-qa02 00:14:4f:fb:7d:0e 28
vnet2 at torld-qa01 00:14:4f:fb:0d:8a 36
vnet1 at torld-soleng01 00:14:4f:f9:64:9b 42
NAME MAC NET-DEV DEVICE MODE
primary-vsw2 00:14:4f:fa:a2:aa nxge2 switch at 1
PEER MAC LDC
vnet1 at torld-soln02 00:14:4f:fb:c2:74 12
vnet0 at torld-soln01 00:14:4f:f8:44:52 18
vnet2 at torld-qa02 00:14:4f:f9:8f:26 29
vnet0 at torld-qa01 00:14:4f:f9:7c:be 34
vnet2 at torld-soleng01 00:14:4f:fa:28:34 43
NAME MAC NET-DEV DEVICE MODE
primary-vsw3 00:14:4f:fb:22:8b nxge3 switch at 2
PEER MAC LDC
vnet2 at torld-soln02 00:14:4f:fa:3c:e3 14
vnet1 at torld-soln01 00:14:4f:fb:66:d4 19
vnet0 at torld-qa02 00:14:4f:f9:e9:26 27
vnet1 at torld-qa01 00:14:4f:fa:a6:e1 35
vnet0 at torld-soleng01 00:14:4f:fb:05:47 41
VDS
NAME VOLUME OPTIONS DEVICE
primary-vds0 torld-soln02_dsk01 /data/ldom/
torld-soln02/dsk01.img
torld-qa01_dsk01 /data/ldom/
torld-qa01/dsk01.img
torld-qa01_dsk02 /data/ldom/
torld-qa01/dsk02.img
torld-qa02_dsk01 /data/ldom/
torld-qa02/dsk01.img
torld-qa02_dsk02 /data/ldom/
torld-qa02/dsk02.img
torld-soln01_dsk01 /data/ldom/
torld-soln01/dsk01.img
torld-soleng01_dsk01 /data/ldom/
torld-soleng01/dsk01.img
CLIENT VOLUME LDC
torld-soln02_dsk01 at torld-soln02 torld-soln02_dsk01 15
torld-soln01_dsk01 at torld-soln01 torld-soln01_dsk01 24
torld-qa02_dsk01 at torld-qa02 torld-qa02_dsk01 30
torld-qa02_dsk02 at torld-qa02 torld-qa02_dsk02 31
torld-qa01_dsk01 at torld-qa01 torld-qa01_dsk01 37
torld-qa01_dsk02 at torld-qa01 torld-qa01_dsk02 38
torld-soleng01_dsk01 at torld-soleng01 torld-soleng01_dsk01 44
VLDC
NAME
primary-vldc3
CLIENT DESC LDC
SP spds 20
SP sunvts 6
SP sunmc 7
SP explorer 8
SP led 9
SP flashupdate 10
SP system-management 13
NAME
primary-vldc0
CLIENT DESC LDC
hvctl at primary hvctl 0
ds at primary domain-services 1
vldcc1 at primary fma 3
ds at torld-soln02 domain-services 16
ds at torld-soln01 domain-services 25
ds at torld-qa02 domain-services 32
ds at torld-qa01 domain-services 39
ds at torld-soleng01 domain-services 45
VLDCC
NAME SERVICE DESC LDC
ds primary-vldc0 at primary domain-services 2
vldcc2 SP spfma 5
vldcc1 primary-vldc0 at primary ldmfma 4
VCONS
NAME SERVICE PORT LDC
SP
------------------------------------------------------------------------------
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
torld-qa01 active -n--- 5004 24 24G 1.0% 15h 1m
MAC
00:14:4f:f8:22:3c
VCPU
VID PID UTIL STRAND
0 80 0.0% 100%
1 81 26% 100%
2 82 0.0% 100%
3 83 5.1% 100%
4 84 0.0% 100%
5 85 89% 100%
6 86 0.0% 100%
7 87 0.0% 100%
8 88 0.5% 100%
9 89 0.3% 100%
10 90 0.0% 100%
11 91 0.0% 100%
12 92 0.0% 100%
13 93 0.1% 100%
14 94 0.0% 100%
15 95 0.0% 100%
16 96 0.2% 100%
17 97 0.0% 100%
18 98 0.0% 100%
19 99 0.0% 100%
20 100 0.0% 100%
21 101 0.0% 100%
22 102 0.0% 100%
23 103 0.0% 100%
MAU
ID CPUSET
10 (80, 81, 82, 83, 84, 85, 86, 87)
11 (88, 89, 90, 91, 92, 93, 94, 95)
12 (96, 97, 98, 99, 100, 101, 102, 103)
MEMORY
RA PA SIZE
0xe000000 0x140e000000 24G
NETWORK
NAME SERVICE DEVICE MAC
vnet0 primary-vsw2 at primary network at 0
00:14:4f:f9:7c:be
PEER MAC LDC
primary-vsw2 at primary 00:14:4f:fa:a2:aa 0
vnet1 at torld-soln02 00:14:4f:fb:c2:74 1
vnet0 at torld-soln01 00:14:4f:f8:44:52 2
vnet2 at torld-qa02 00:14:4f:f9:8f:26 3
vnet2 at torld-soleng01 00:14:4f:fa:28:34 18
NAME SERVICE DEVICE MAC
vnet1 primary-vsw3 at primary network at 1
00:14:4f:fa:a6:e1
PEER MAC LDC
primary-vsw3 at primary 00:14:4f:fb:22:8b 4
vnet2 at torld-soln02 00:14:4f:fa:3c:e3 5
vnet1 at torld-soln01 00:14:4f:fb:66:d4 6
vnet0 at torld-qa02 00:14:4f:f9:e9:26 7
vnet0 at torld-soleng01 00:14:4f:fb:05:47 16
NAME SERVICE DEVICE MAC
vnet2 primary-vsw1 at primary network at 2
00:14:4f:fb:0d:8a
PEER MAC LDC
primary-vsw1 at primary 00:14:4f:f9:ef:b6 8
vnet0 at torld-soln02 00:14:4f:fb:ae:3c 9
vnet2 at torld-soln01 00:14:4f:f9:08:7e 10
vnet1 at torld-qa02 00:14:4f:fb:7d:0e 11
vnet1 at torld-soleng01 00:14:4f:f9:64:9b 17
DISK
NAME VOLUME TOUT DEVICE
SERVER LDC
torld-qa01_dsk01 torld-qa01_dsk01 at primary-vds0 disk at 0
primary 12
torld-qa01_dsk02 torld-qa01_dsk02 at primary-vds0 disk at 1
primary 13
VLDCC
NAME SERVICE DESC LDC
ds primary-vldc0 at primary domain-services 14
VCONS
NAME SERVICE PORT LDC
torld-qa01 primary-vcc0 at primary 5004 15
------------------------------------------------------------------------------
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
torld-qa02 active -n--- 5003 24 24G 0.1% 15h 35m
MAC
00:14:4f:fb:03:f5
VCPU
VID PID UTIL STRAND
0 56 1.2% 100%
1 57 49% 100%
2 58 0.0% 100%
3 59 9.9% 100%
4 60 0.0% 100%
5 61 79% 100%
6 62 0.0% 100%
7 63 0.0% 100%
8 64 5.0% 100%
9 65 0.0% 100%
10 66 0.0% 100%
11 67 0.0% 100%
12 68 0.0% 100%
13 69 0.0% 100%
14 70 0.0% 100%
15 71 0.0% 100%
16 72 0.0% 100%
17 73 0.0% 100%
18 74 0.0% 100%
19 75 0.0% 100%
20 76 0.0% 100%
21 77 0.0% 100%
22 78 0.0% 100%
23 79 0.0% 100%
MAU
ID CPUSET
7 (56, 57, 58, 59, 60, 61, 62, 63)
8 (64, 65, 66, 67, 68, 69, 70, 71)
9 (72, 73, 74, 75, 76, 77, 78, 79)
MEMORY
RA PA SIZE
0xe000000 0xe0e000000 24G
NETWORK
NAME SERVICE DEVICE MAC
vnet0 primary-vsw3 at primary network at 0
00:14:4f:f9:e9:26
PEER MAC LDC
primary-vsw3 at primary 00:14:4f:fb:22:8b 0
vnet2 at torld-soln02 00:14:4f:fa:3c:e3 1
vnet1 at torld-soln01 00:14:4f:fb:66:d4 2
vnet1 at torld-qa01 00:14:4f:fa:a6:e1 14
vnet0 at torld-soleng01 00:14:4f:fb:05:47 16
NAME SERVICE DEVICE MAC
vnet1 primary-vsw1 at primary network at 1
00:14:4f:fb:7d:0e
PEER MAC LDC
primary-vsw1 at primary 00:14:4f:f9:ef:b6 3
vnet0 at torld-soln02 00:14:4f:fb:ae:3c 4
vnet2 at torld-soln01 00:14:4f:f9:08:7e 5
vnet2 at torld-qa01 00:14:4f:fb:0d:8a 15
vnet1 at torld-soleng01 00:14:4f:f9:64:9b 17
NAME SERVICE DEVICE MAC
vnet2 primary-vsw2 at primary network at 2
00:14:4f:f9:8f:26
PEER MAC LDC
primary-vsw2 at primary 00:14:4f:fa:a2:aa 6
vnet1 at torld-soln02 00:14:4f:fb:c2:74 7
vnet0 at torld-soln01 00:14:4f:f8:44:52 8
vnet0 at torld-qa01 00:14:4f:f9:7c:be 13
vnet2 at torld-soleng01 00:14:4f:fa:28:34 18
DISK
NAME VOLUME TOUT DEVICE
SERVER LDC
torld-qa02_dsk01 torld-qa02_dsk01 at primary-vds0 disk at 0
primary 9
torld-qa02_dsk02 torld-qa02_dsk02 at primary-vds0 disk at 1
primary 10
VLDCC
NAME SERVICE DESC LDC
ds primary-vldc0 at primary domain-services 11
VCONS
NAME SERVICE PORT LDC
torld-qa02 primary-vcc0 at primary 5003 12
------------------------------------------------------------------------------
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
torld-soleng01 active -n--- 5005 24 22G 26% 37m
MAC
00:14:4f:fa:4c:a1
VCPU
VID PID UTIL STRAND
0 104 0.0% 100%
1 105 51% 100%
2 106 0.0% 100%
3 107 10% 100%
4 108 0.0% 100%
5 109 78% 100%
6 110 0.0% 100%
7 111 0.0% 100%
8 112 0.0% 100%
9 113 0.0% 100%
10 114 0.0% 100%
11 115 0.0% 100%
12 116 0.0% 100%
13 117 0.0% 100%
14 118 0.0% 100%
15 119 0.0% 100%
16 120 0.0% 100%
17 121 0.0% 100%
18 122 0.0% 100%
19 123 0.0% 100%
20 124 0.0% 100%
21 125 0.0% 100%
22 126 0.0% 100%
23 127 0.0% 100%
MAU
ID CPUSET
13 (104, 105, 106, 107, 108, 109, 110, 111)
14 (112, 113, 114, 115, 116, 117, 118, 119)
15 (120, 121, 122, 123, 124, 125, 126, 127)
MEMORY
RA PA SIZE
0xe000000 0x1a0e000000 22G
NETWORK
NAME SERVICE DEVICE MAC
vnet0 primary-vsw3 at primary network at 0
00:14:4f:fb:05:47
PEER MAC LDC
primary-vsw3 at primary 00:14:4f:fb:22:8b 0
vnet2 at torld-soln02 00:14:4f:fa:3c:e3 1
vnet1 at torld-soln01 00:14:4f:fb:66:d4 2
vnet0 at torld-qa02 00:14:4f:f9:e9:26 3
vnet1 at torld-qa01 00:14:4f:fa:a6:e1 4
NAME SERVICE DEVICE MAC
vnet1 primary-vsw1 at primary network at 1
00:14:4f:f9:64:9b
PEER MAC LDC
primary-vsw1 at primary 00:14:4f:f9:ef:b6 5
vnet0 at torld-soln02 00:14:4f:fb:ae:3c 6
vnet2 at torld-soln01 00:14:4f:f9:08:7e 7
vnet1 at torld-qa02 00:14:4f:fb:7d:0e 8
vnet2 at torld-qa01 00:14:4f:fb:0d:8a 9
NAME SERVICE DEVICE MAC
vnet2 primary-vsw2 at primary network at 2
00:14:4f:fa:28:34
PEER MAC LDC
primary-vsw2 at primary 00:14:4f:fa:a2:aa 10
vnet1 at torld-soln02 00:14:4f:fb:c2:74 11
vnet0 at torld-soln01 00:14:4f:f8:44:52 12
vnet2 at torld-qa02 00:14:4f:f9:8f:26 13
vnet0 at torld-qa01 00:14:4f:f9:7c:be 14
DISK
NAME VOLUME TOUT DEVICE
SERVER LDC
torld-soleng01_dsk01 torld-soleng01_dsk01 at primary-vds0
disk at 0 primary 15
VLDCC
NAME SERVICE DESC LDC
ds primary-vldc0 at primary domain-services 16
VCONS
NAME SERVICE PORT LDC
torld-soleng01 primary-vcc0 at primary 5005 17
------------------------------------------------------------------------------
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
torld-soln01 active -n--- 5002 24 24G 0.1% 15h 46m
MAC
00:14:4f:f9:69:4a
VCPU
VID PID UTIL STRAND
0 32 0.0% 100%
1 33 52% 100%
2 34 0.0% 100%
3 35 10% 100%
4 36 0.0% 100%
5 37 78% 100%
6 38 0.0% 100%
7 39 0.0% 100%
8 40 0.0% 100%
9 41 0.0% 100%
10 42 0.0% 100%
11 43 0.0% 100%
12 44 0.0% 100%
13 45 0.0% 100%
14 46 0.0% 100%
15 47 0.0% 100%
16 48 0.0% 100%
17 49 0.0% 100%
18 50 0.0% 100%
19 51 0.0% 100%
20 52 0.0% 100%
21 53 0.0% 100%
22 54 0.0% 100%
23 55 0.0% 100%
MAU
ID CPUSET
4 (32, 33, 34, 35, 36, 37, 38, 39)
5 (40, 41, 42, 43, 44, 45, 46, 47)
6 (48, 49, 50, 51, 52, 53, 54, 55)
MEMORY
RA PA SIZE
0xe000000 0x80e000000 24G
NETWORK
NAME SERVICE DEVICE MAC
vnet0 primary-vsw2 at primary network at 0
00:14:4f:f8:44:52
PEER MAC LDC
primary-vsw2 at primary 00:14:4f:fa:a2:aa 0
vnet1 at torld-soln02 00:14:4f:fb:c2:74 1
vnet2 at torld-qa02 00:14:4f:f9:8f:26 11
vnet0 at torld-qa01 00:14:4f:f9:7c:be 12
vnet2 at torld-soleng01 00:14:4f:fa:28:34 17
NAME SERVICE DEVICE MAC
vnet1 primary-vsw3 at primary network at 1
00:14:4f:fb:66:d4
PEER MAC LDC
primary-vsw3 at primary 00:14:4f:fb:22:8b 2
vnet2 at torld-soln02 00:14:4f:fa:3c:e3 3
vnet0 at torld-qa02 00:14:4f:f9:e9:26 9
vnet1 at torld-qa01 00:14:4f:fa:a6:e1 13
vnet0 at torld-soleng01 00:14:4f:fb:05:47 15
NAME SERVICE DEVICE MAC
vnet2 primary-vsw1 at primary network at 2
00:14:4f:f9:08:7e
PEER MAC LDC
primary-vsw1 at primary 00:14:4f:f9:ef:b6 4
vnet0 at torld-soln02 00:14:4f:fb:ae:3c 5
vnet1 at torld-qa02 00:14:4f:fb:7d:0e 10
vnet2 at torld-qa01 00:14:4f:fb:0d:8a 14
vnet1 at torld-soleng01 00:14:4f:f9:64:9b 16
DISK
NAME VOLUME TOUT DEVICE
SERVER LDC
torld-soln01_dsk01 torld-soln01_dsk01 at primary-vds0 disk at 0
primary 6
VLDCC
NAME SERVICE DESC LDC
ds primary-vldc0 at primary domain-services 7
VCONS
NAME SERVICE PORT LDC
torld-soln01 primary-vcc0 at primary 5002 8
------------------------------------------------------------------------------
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
torld-soln02 active -n--- 5001 24 24G 0.1% 19h 40m
MAC
00:14:4f:fb:d1:3d
VCPU
VID PID UTIL STRAND
0 8 0.0% 100%
1 9 50% 100%
2 10 0.0% 100%
3 11 10% 100%
4 12 0.0% 100%
5 13 79% 100%
6 14 0.0% 100%
7 15 0.0% 100%
8 16 1.0% 100%
9 17 0.0% 100%
10 18 0.0% 100%
11 19 0.0% 100%
12 20 0.0% 100%
13 21 0.0% 100%
14 22 0.0% 100%
15 23 0.0% 100%
16 24 0.0% 100%
17 25 0.0% 100%
18 26 0.0% 100%
19 27 0.0% 100%
20 28 0.0% 100%
21 29 0.0% 100%
22 30 0.0% 100%
23 31 0.0% 100%
MAU
ID CPUSET
1 (8, 9, 10, 11, 12, 13, 14, 15)
2 (16, 17, 18, 19, 20, 21, 22, 23)
3 (24, 25, 26, 27, 28, 29, 30, 31)
MEMORY
RA PA SIZE
0xe000000 0x20e000000 24G
NETWORK
NAME SERVICE DEVICE MAC
vnet0 primary-vsw1 at primary network at 0
00:14:4f:fb:ae:3c
PEER MAC LDC
primary-vsw1 at primary 00:14:4f:f9:ef:b6 0
vnet2 at torld-soln01 00:14:4f:f9:08:7e 8
vnet1 at torld-qa02 00:14:4f:fb:7d:0e 10
vnet2 at torld-qa01 00:14:4f:fb:0d:8a 14
vnet1 at torld-soleng01 00:14:4f:f9:64:9b 16
NAME SERVICE DEVICE MAC
vnet1 primary-vsw2 at primary network at 1
00:14:4f:fb:c2:74
PEER MAC LDC
primary-vsw2 at primary 00:14:4f:fa:a2:aa 1
vnet0 at torld-soln01 00:14:4f:f8:44:52 6
vnet2 at torld-qa02 00:14:4f:f9:8f:26 11
vnet0 at torld-qa01 00:14:4f:f9:7c:be 12
vnet2 at torld-soleng01 00:14:4f:fa:28:34 17
NAME SERVICE DEVICE MAC
vnet2 primary-vsw3 at primary network at 2
00:14:4f:fa:3c:e3
PEER MAC LDC
primary-vsw3 at primary 00:14:4f:fb:22:8b 2
vnet1 at torld-soln01 00:14:4f:fb:66:d4 7
vnet0 at torld-qa02 00:14:4f:f9:e9:26 9
vnet1 at torld-qa01 00:14:4f:fa:a6:e1 13
vnet0 at torld-soleng01 00:14:4f:fb:05:47 15
DISK
NAME VOLUME TOUT DEVICE
SERVER LDC
torld-soln02_dsk01 torld-soln02_dsk01 at primary-vds0 disk at 0
primary 3
VLDCC
NAME SERVICE DESC LDC
ds primary-vldc0 at primary domain-services 4
VCONS
NAME SERVICE PORT LDC
torld-soln02 primary-vcc0 at primary 5001 5
torsun01sx:/root#
