[nfs-discuss] Oracle DB crashed on NFS4 built on 7210 Unified Storage
On 03/11/10 03:10 PM, Andrey Yasnogor wrote: > > Tom, help me to understand what let you to conclude it was a storage > problem and not Solaris/NFS issue? > I'm not convinced this being the case here, but did open an SR. > Tnx. Oh, I used a pretty simple heuristic: 1) You mentioned Unified Storage in the title 2) We've been asked to start all such issues with SRs. :-> > > On 3/11/10 3:37 PM, "Tom Haynes" wrote: > > Andrey Yasnogor wrote: > > Anyone? > > > > Andrey, > > You should follow the support model in place for getting help on > the 7210. > > Thanks, > Tom > > > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS > op OP_OPEN got error NFS4ERR_EXPIRED causing recovery action > NR_CLIENTID. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS > Starting recovery for mount /zones/zone2/root/orafiles (mi > 0x600647b3000 mi > > _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 ./qa1/redo > (0x600543b2c68), rnode_pt2 ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8) > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 273629 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS > Recovery done for mount /zones/zone2/root/orafiles (mi > 0x600647b3000) on se > > rver xx.xx.xx.xx, rnode_pt1 ./qa1/redo (0x600543b2c68), rnode_pt2 > ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8) > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 978989 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]NFSMAPID_DOMAIN does not match the > server: xx.xx.xx.xx domain > > Mar 11 08:44:00 v890-01-zn2 Please check configuration > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS > op OP_WRITE got error NFS4ERR_EXPIRED causing recovery action > NR_CLIENTID. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS > Starting recovery for mount /zones/zone2/root/orafiles (mi > 0x600647b3000 mi > > _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 > ./qa1jp/redo/controlqa1jp02.ctl (0x600671dac68), rnode_pt2 string> (0x0) > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for > file ./qa1/sysaux01.dbf (rnode_pt 0x300 > > 05308f70) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 13091 lost its locks on file > ./qa1/sysaux01.dbf (rnode_pt: 30005308f70) > > due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 941083 kern.info] NOTICE: > NFS4 FACT SHEET: > > Mar 11 08:44:00 v890-01-zn2 Action: NR_CLIENTID > > Mar 11 08:44:00 v890-01-zn2 NFS4 error: NFS4ERR_EXPIRED > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for > file ./qa1/qrtz_data01.dbf (rnode_pt 0x > > 30005306410) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 13091 lost its locks on file > ./qa1/qrtz_data01.dbf (rnode_pt: 300053064 > > 10) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for > file ./qa1jp/arc_jp_data01.dbf (rnode_pt > > 0x3002aaf9728) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 5811 lost its locks on file > ./qa1jp/arc_jp_data01.dbf (rnode_pt: 3002aa > > f9728) due to NFS recovery error (NFS4ERR_NO_GRACE) on server > xx.xx.xx.xx. > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for > file ./qa1/tools01.dbf (rnode_pt 0x3000 > > 5308050) on (server xx.xx.xx.xx): error 10033 > > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: > [NFS4][Server: xx.xx.xx.xx][Mntpt: > /zones/zone2/root/orafiles]Process 13091 lost its locks on file > ./qa1/tools01.dbf (rnode_pt: 30005308050
[nfs-discuss] Oracle DB crashed on NFS4 built on 7210 Unified Storage
Andrey Yasnogor wrote: > Anyone? > Andrey, You should follow the support model in place for getting help on the 7210. Thanks, Tom > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS op OP_OPEN got error > NFS4ERR_EXPIRED causing recovery action NR_CLIENTID. > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Starting recovery for > mount /zones/zone2/root/orafiles (mi 0x600647b3000 mi > _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 ./qa1/redo > (0x600543b2c68), rnode_pt2 ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8) > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 273629 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Recovery done for mount > /zones/zone2/root/orafiles (mi 0x600647b3000) on se > rver xx.xx.xx.xx, rnode_pt1 ./qa1/redo (0x600543b2c68), rnode_pt2 > ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8) > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 978989 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFSMAPID_DOMAIN does not match > the server: xx.xx.xx.xx domain > Mar 11 08:44:00 v890-01-zn2 Please check configuration > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS op OP_WRITE got error > NFS4ERR_EXPIRED causing recovery action NR_CLIENTID. > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Starting recovery for > mount /zones/zone2/root/orafiles (mi 0x600647b3000 mi > _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 > ./qa1jp/redo/controlqa1jp02.ctl (0x600671dac68), rnode_pt2 (0x0) > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid > 13091 for file ./qa1/sysaux01.dbf (rnode_pt 0x300 > 05308f70) on (server xx.xx.xx.xx): error 10033 > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks > on file ./qa1/sysaux01.dbf (rnode_pt: 30005308f70) > due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 941083 kern.info] NOTICE: NFS4 FACT > SHEET: > Mar 11 08:44:00 v890-01-zn2 Action: NR_CLIENTID > Mar 11 08:44:00 v890-01-zn2 NFS4 error: NFS4ERR_EXPIRED > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid > 13091 for file ./qa1/qrtz_data01.dbf (rnode_pt 0x > 30005306410) on (server xx.xx.xx.xx): error 10033 > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks > on file ./qa1/qrtz_data01.dbf (rnode_pt: 300053064 > 10) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid > 5811 for file ./qa1jp/arc_jp_data01.dbf (rnode_pt > 0x3002aaf9728) on (server xx.xx.xx.xx): error 10033 > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on > file ./qa1jp/arc_jp_data01.dbf (rnode_pt: 3002aa > f9728) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid > 13091 for file ./qa1/tools01.dbf (rnode_pt 0x3000 > 5308050) on (server xx.xx.xx.xx): error 10033 > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks > on file ./qa1/tools01.dbf (rnode_pt: 30005308050) > due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid > 5811 for file ./qa1jp/il_data02.dbf (rnode_pt 0x3 > 00216757e0) on (server xx.xx.xx.xx): error 10033 > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on > file ./qa1jp/il_data02.dbf (rnode_pt: 300216757e > 0) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid > 5811 for file ./qa1jp/streams_ts_01.dbf (rnode_pt > 0x3
[nfs-discuss] Oracle DB crashed on NFS4 built on 7210 Unified Storage
Tom, help me to understand what let you to conclude it was a storage problem and not Solaris/NFS issue? I'm not convinced this being the case here, but did open an SR. Tnx. On 3/11/10 3:37 PM, "Tom Haynes" wrote: Andrey Yasnogor wrote: > Anyone? > Andrey, You should follow the support model in place for getting help on the 7210. Thanks, Tom > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS op OP_OPEN got error > NFS4ERR_EXPIRED causing recovery action NR_CLIENTID. > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Starting recovery for > mount /zones/zone2/root/orafiles (mi 0x600647b3000 mi > _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 ./qa1/redo > (0x600543b2c68), rnode_pt2 ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8) > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 273629 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Recovery done for mount > /zones/zone2/root/orafiles (mi 0x600647b3000) on se > rver xx.xx.xx.xx, rnode_pt1 ./qa1/redo (0x600543b2c68), rnode_pt2 > ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8) > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 978989 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFSMAPID_DOMAIN does not match > the server: xx.xx.xx.xx domain > Mar 11 08:44:00 v890-01-zn2 Please check configuration > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS op OP_WRITE got error > NFS4ERR_EXPIRED causing recovery action NR_CLIENTID. > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Starting recovery for > mount /zones/zone2/root/orafiles (mi 0x600647b3000 mi > _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 > ./qa1jp/redo/controlqa1jp02.ctl (0x600671dac68), rnode_pt2 (0x0) > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid > 13091 for file ./qa1/sysaux01.dbf (rnode_pt 0x300 > 05308f70) on (server xx.xx.xx.xx): error 10033 > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks > on file ./qa1/sysaux01.dbf (rnode_pt: 30005308f70) > due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 941083 kern.info] NOTICE: NFS4 FACT > SHEET: > Mar 11 08:44:00 v890-01-zn2 Action: NR_CLIENTID > Mar 11 08:44:00 v890-01-zn2 NFS4 error: NFS4ERR_EXPIRED > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid > 13091 for file ./qa1/qrtz_data01.dbf (rnode_pt 0x > 30005306410) on (server xx.xx.xx.xx): error 10033 > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks > on file ./qa1/qrtz_data01.dbf (rnode_pt: 300053064 > 10) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid > 5811 for file ./qa1jp/arc_jp_data01.dbf (rnode_pt > 0x3002aaf9728) on (server xx.xx.xx.xx): error 10033 > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on > file ./qa1jp/arc_jp_data01.dbf (rnode_pt: 3002aa > f9728) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid > 13091 for file ./qa1/tools01.dbf (rnode_pt 0x3000 > 5308050) on (server xx.xx.xx.xx): error 10033 > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks > on file ./qa1/tools01.dbf (rnode_pt: 30005308050) > due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid > 5811 for file ./qa1jp/il_data02.dbf (rnode_pt 0x3 > 00216757e0) on (server xx.xx.xx.xx): error 10033 > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: > xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on > file ./qa1jp/il_data02.dbf (rnode_pt: 300216757e > 0) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. > Mar 11 08
[nfs-discuss] Oracle DB crashed on NFS4 built on 7210 Unified Storage
Anyone? Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS op OP_OPEN got error NFS4ERR_EXPIRED causing recovery action NR_CLIENTID. Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Starting recovery for mount /zones/zone2/root/orafiles (mi 0x600647b3000 mi _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 ./qa1/redo (0x600543b2c68), rnode_pt2 ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8) Mar 11 08:44:00 v890-01-zn2 nfs: [ID 273629 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Recovery done for mount /zones/zone2/root/orafiles (mi 0x600647b3000) on se rver xx.xx.xx.xx, rnode_pt1 ./qa1/redo (0x600543b2c68), rnode_pt2 ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8) Mar 11 08:44:00 v890-01-zn2 nfs: [ID 978989 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFSMAPID_DOMAIN does not match the server: xx.xx.xx.xx domain Mar 11 08:44:00 v890-01-zn2 Please check configuration Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS op OP_WRITE got error NFS4ERR_EXPIRED causing recovery action NR_CLIENTID. Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Starting recovery for mount /zones/zone2/root/orafiles (mi 0x600647b3000 mi _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 ./qa1jp/redo/controlqa1jp02.ctl (0x600671dac68), rnode_pt2 (0x0) Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for file ./qa1/sysaux01.dbf (rnode_pt 0x300 05308f70) on (server xx.xx.xx.xx): error 10033 Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks on file ./qa1/sysaux01.dbf (rnode_pt: 30005308f70) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. Mar 11 08:44:00 v890-01-zn2 nfs: [ID 941083 kern.info] NOTICE: NFS4 FACT SHEET: Mar 11 08:44:00 v890-01-zn2 Action: NR_CLIENTID Mar 11 08:44:00 v890-01-zn2 NFS4 error: NFS4ERR_EXPIRED Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for file ./qa1/qrtz_data01.dbf (rnode_pt 0x 30005306410) on (server xx.xx.xx.xx): error 10033 Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks on file ./qa1/qrtz_data01.dbf (rnode_pt: 300053064 10) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for file ./qa1jp/arc_jp_data01.dbf (rnode_pt 0x3002aaf9728) on (server xx.xx.xx.xx): error 10033 Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on file ./qa1jp/arc_jp_data01.dbf (rnode_pt: 3002aa f9728) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for file ./qa1/tools01.dbf (rnode_pt 0x3000 5308050) on (server xx.xx.xx.xx): error 10033 Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks on file ./qa1/tools01.dbf (rnode_pt: 30005308050) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for file ./qa1jp/il_data02.dbf (rnode_pt 0x3 00216757e0) on (server xx.xx.xx.xx): error 10033 Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on file ./qa1jp/il_data02.dbf (rnode_pt: 300216757e 0) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx. Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for file ./qa1jp/streams_ts_01.dbf (rnode_pt 0x30014dffbf0) on (server xx.xx.xx.xx): error 10033 Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on file ./qa1jp/streams_ts_01.dbf (rnode_pt: 30014d ffbf0) due to NFS re
