On 03/11/10 03:10 PM, Andrey Yasnogor wrote: > > Tom, help me to understand what let you to conclude it was a storage > problem and not Solaris/NFS issue? > I'm not convinced this being the case here, but did open an SR. > Tnx.
Oh, I used a pretty simple heuristic: 1) You mentioned Unified Storage in the title 2) We've been asked to start all such issues with SRs. :-> > > On 3/11/10 3:37 PM, "Tom Haynes" <Thomas.Haynes at Sun.COM> wrote: > > Andrey Yasnogor wrote: > > Anyone? > > > > Andrey, > > You should follow the support model in place for getting help on > the 7210. > > Thanks, > Tom > > > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS > op OP_OPEN got error NFS4ERR_EXPIRED causing recovery action > NR_CLIENTID. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS > Starting recovery for mount /zones/zone2/root/orafiles (mi > 0x600647b3000 mi > > _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 ./qa1/redo > (0x600543b2c68), rnode_pt2 ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8) > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 273629 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS > Recovery done for mount /zones/zone2/root/orafiles (mi > 0x600647b3000) on se > > rver xx.xx.xx.xx, rnode_pt1 ./qa1/redo (0x600543b2c68), rnode_pt2 > ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8) > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 978989 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]NFSMAPID_DOMAIN does not match the > server: xx.xx.xx.xx domain > > Mar 11 08:44:00 v890-01-zn2 Please check configuration > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS > op OP_WRITE got error NFS4ERR_EXPIRED causing recovery action > NR_CLIENTID. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS > Starting recovery for mount /zones/zone2/root/orafiles (mi > 0x600647b3000 mi > > _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 > ./qa1jp/redo/controlqa1jp02.ctl (0x600671dac68), rnode_pt2 <null > string> (0x0) > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for > file ./qa1/sysaux01.dbf (rnode_pt 0x300 > > 05308f70) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 13091 lost its locks on file > ./qa1/sysaux01.dbf (rnode_pt: 30005308f70) > > due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 941083 kern.info] NOTICE: > NFS4 FACT SHEET: > > Mar 11 08:44:00 v890-01-zn2 Action: NR_CLIENTID > > Mar 11 08:44:00 v890-01-zn2 NFS4 error: NFS4ERR_EXPIRED > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for > file ./qa1/qrtz_data01.dbf (rnode_pt 0x > > 30005306410) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 13091 lost its locks on file > ./qa1/qrtz_data01.dbf (rnode_pt: 300053064 > > 10) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/arc_jp_data01.dbf (rnode_pt > > 0x3002aaf9728) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/arc_jp_data01.dbf (rnode_pt: 3002aa > > f9728) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for > file ./qa1/tools01.dbf (rnode_pt 0x3000 > > 5308050) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 13091 lost its locks on file > ./qa1/tools01.dbf (rnode_pt: 30005308050) > > due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/il_data02.dbf (rnode_pt 0x3 > > 00216757e0) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/il_data02.dbf (rnode_pt: 300216757e > > 0) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/streams_ts_01.dbf (rnode_pt > > 0x30014dffbf0) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/streams_ts_01.dbf (rnode_pt: 30014d > > ffbf0) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 6048 for > file ./qa1jp/redo/controlqa1jp02.ctl (rn > > ode_pt 0x600671dac68) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 6048 lost its locks on file > ./qa1jp/redo/controlqa1jp02.ctl (rnode_pt: > > 600671dac68) due to NFS recovery error (NFS4ERR_NO_GRACE) on > server xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/system01.dbf (rnode_pt 0x30 > > 02166bbc0) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/system01.dbf (rnode_pt: 3002166bbc0 > > ) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/tools01.dbf (rnode_pt 0x300 > > 2aaed730) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/tools01.dbf (rnode_pt: 3002aaed730) > > due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/qrtz_jp_data01.dbf (rnode_p > > t 0x3002aae7738) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/qrtz_jp_data01.dbf (rnode_pt: 3002a > > ae7738) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for > file ./qa1/ARC_INDEX01.dbf (rnode_pt 0x > > 6005ee8e018) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 13091 lost its locks on file > ./qa1/ARC_INDEX01.dbf (rnode_pt: 6005ee8e0 > > 18) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/il_data_in_01.dbf (rnode_pt > > 0x3003115b778) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/il_data_in_01.dbf (rnode_pt: 300311 > > 5b778) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/il_data03.dbf (rnode_pt 0x3 > > 0021665bc8) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/il_data03.dbf (rnode_pt: 30021665bc > > 8) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/il_data_in_02.dbf (rnode_pt > > 0x300299177a8) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/il_data_in_02.dbf (rnode_pt: 300299 > > 177a8) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/undotbs01.dbf (rnode_pt 0x3 > > 003113f758) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/undotbs01.dbf (rnode_pt: 3003113f75 > > 8) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/il_data01.dbf (rnode_pt 0x3 > > 003115f768) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/il_data01.dbf (rnode_pt: 3003115f76 > > 8) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/arc_jp_index01.dbf (rnode_p > > t 0x300299017b0) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/arc_jp_index01.dbf (rnode_pt: 30029 > > 9017b0) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 6048 for > file ./qa1jp/controlqa1jp03.ctl (rnode_p > > t 0x3002167d7d0) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 6048 lost its locks on file > ./qa1jp/controlqa1jp03.ctl (rnode_pt: 30021 > > 67d7d0) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/sysaux01.dbf (rnode_pt 0x60 > > 05e84c178) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/sysaux01.dbf (rnode_pt: 6005e84c178 > > ) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for > file ./qa1/streams_ts_01.dbf (rnode_pt > > 0x30005307ac0) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 13091 lost its locks on file > ./qa1/streams_ts_01.dbf (rnode_pt: 3000530 > > 7ac0) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for > file ./qa1/system01.dbf (rnode_pt 0x300 > > 05308418) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 13091 lost its locks on file > ./qa1/system01.dbf (rnode_pt: 30005308418) > > due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > > > > > > ------------------------------------------------------------------------ > Disclaimer: This electronic mail and any attachments are confidential > and may be privileged. If you are not the intended recipient, please > notify the sender immediately by replying to this email, and destroy > all copies of this email and any attachments. Thank you.
