[nfs-discuss] Oracle DB crashed on NFS4 built on 7210 Unified Storage

2010-03-12 Thread Tom Haynes
On 03/11/10 03:10 PM, Andrey Yasnogor wrote:
>
> Tom, help me to understand what let you to conclude it was a storage 
> problem and not Solaris/NFS issue?
> I'm not convinced this being the case here, but did open an SR.
> Tnx.

Oh, I used a pretty simple heuristic:

1) You mentioned Unified Storage in the title
2) We've been asked to start all such issues with SRs.

:->




>
> On 3/11/10 3:37 PM, "Tom Haynes"  wrote:
>
> Andrey Yasnogor wrote:
> > Anyone?
> >
>
> Andrey,
>
> You should follow the support model in place for getting help on
> the 7210.
>
> Thanks,
> Tom
>
>
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS
> op OP_OPEN got error NFS4ERR_EXPIRED causing recovery action
> NR_CLIENTID.
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS
> Starting recovery for mount /zones/zone2/root/orafiles (mi
> 0x600647b3000 mi
> > _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 ./qa1/redo
> (0x600543b2c68), rnode_pt2 ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8)
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 273629 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS
> Recovery done for mount /zones/zone2/root/orafiles (mi
> 0x600647b3000) on se
> > rver xx.xx.xx.xx, rnode_pt1 ./qa1/redo (0x600543b2c68), rnode_pt2
> ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8)
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 978989 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt:
> /zones/zone2/root/orafiles]NFSMAPID_DOMAIN does not match the
> server: xx.xx.xx.xx domain
> > Mar 11 08:44:00 v890-01-zn2 Please check configuration
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS
> op OP_WRITE got error NFS4ERR_EXPIRED causing recovery action
> NR_CLIENTID.
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS
> Starting recovery for mount /zones/zone2/root/orafiles (mi
> 0x600647b3000 mi
> > _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1
> ./qa1jp/redo/controlqa1jp02.ctl (0x600671dac68), rnode_pt2  string> (0x0)
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt:
> /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for
> file ./qa1/sysaux01.dbf (rnode_pt 0x300
> > 05308f70) on (server xx.xx.xx.xx): error 10033
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt:
> /zones/zone2/root/orafiles]Process 13091 lost its locks on file
> ./qa1/sysaux01.dbf (rnode_pt: 30005308f70)
> >  due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 941083 kern.info] NOTICE:
> NFS4 FACT SHEET:
> > Mar 11 08:44:00 v890-01-zn2  Action: NR_CLIENTID
> > Mar 11 08:44:00 v890-01-zn2  NFS4 error: NFS4ERR_EXPIRED
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt:
> /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for
> file ./qa1/qrtz_data01.dbf (rnode_pt 0x
> > 30005306410) on (server xx.xx.xx.xx): error 10033
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt:
> /zones/zone2/root/orafiles]Process 13091 lost its locks on file
> ./qa1/qrtz_data01.dbf (rnode_pt: 300053064
> > 10) due to NFS recovery error (NFS4ERR_NO_GRACE) on server
> xx.xx.xx.xx.
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt:
> /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 5811 for
> file ./qa1jp/arc_jp_data01.dbf (rnode_pt
> >  0x3002aaf9728) on (server xx.xx.xx.xx): error 10033
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt:
> /zones/zone2/root/orafiles]Process 5811 lost its locks on file
> ./qa1jp/arc_jp_data01.dbf (rnode_pt: 3002aa
> > f9728) due to NFS recovery error (NFS4ERR_NO_GRACE) on server
> xx.xx.xx.xx.
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt:
> /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 13091 for
> file ./qa1/tools01.dbf (rnode_pt 0x3000
> > 5308050) on (server xx.xx.xx.xx): error 10033
> > Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE:
> [NFS4][Server: xx.xx.xx.xx][Mntpt:
> /zones/zone2/root/orafiles]Process 13091 lost its locks on file
> ./qa1/tools01.dbf (rnode_pt: 30005308050

[nfs-discuss] Oracle DB crashed on NFS4 built on 7210 Unified Storage

2010-03-11 Thread Tom Haynes
Andrey Yasnogor wrote:
> Anyone? 
>   

Andrey,

You should follow the support model in place for getting help on
the 7210.

Thanks,
Tom


> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS op OP_OPEN got error 
> NFS4ERR_EXPIRED causing recovery action NR_CLIENTID.
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Starting recovery for 
> mount /zones/zone2/root/orafiles (mi 0x600647b3000 mi
> _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 ./qa1/redo 
> (0x600543b2c68), rnode_pt2 ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8)
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 273629 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Recovery done for mount 
> /zones/zone2/root/orafiles (mi 0x600647b3000) on se
> rver xx.xx.xx.xx, rnode_pt1 ./qa1/redo (0x600543b2c68), rnode_pt2 
> ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8)
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 978989 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFSMAPID_DOMAIN does not match 
> the server: xx.xx.xx.xx domain
> Mar 11 08:44:00 v890-01-zn2 Please check configuration
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS op OP_WRITE got error 
> NFS4ERR_EXPIRED causing recovery action NR_CLIENTID.
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Starting recovery for 
> mount /zones/zone2/root/orafiles (mi 0x600647b3000 mi
> _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 
> ./qa1jp/redo/controlqa1jp02.ctl (0x600671dac68), rnode_pt2  (0x0)
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
> 13091 for file ./qa1/sysaux01.dbf (rnode_pt 0x300
> 05308f70) on (server xx.xx.xx.xx): error 10033
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks 
> on file ./qa1/sysaux01.dbf (rnode_pt: 30005308f70)
>  due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 941083 kern.info] NOTICE: NFS4 FACT 
> SHEET: 
> Mar 11 08:44:00 v890-01-zn2  Action: NR_CLIENTID 
> Mar 11 08:44:00 v890-01-zn2  NFS4 error: NFS4ERR_EXPIRED   
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
> 13091 for file ./qa1/qrtz_data01.dbf (rnode_pt 0x
> 30005306410) on (server xx.xx.xx.xx): error 10033
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks 
> on file ./qa1/qrtz_data01.dbf (rnode_pt: 300053064
> 10) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
> 5811 for file ./qa1jp/arc_jp_data01.dbf (rnode_pt
>  0x3002aaf9728) on (server xx.xx.xx.xx): error 10033
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on 
> file ./qa1jp/arc_jp_data01.dbf (rnode_pt: 3002aa
> f9728) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
> 13091 for file ./qa1/tools01.dbf (rnode_pt 0x3000
> 5308050) on (server xx.xx.xx.xx): error 10033
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks 
> on file ./qa1/tools01.dbf (rnode_pt: 30005308050) 
> due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
> 5811 for file ./qa1jp/il_data02.dbf (rnode_pt 0x3
> 00216757e0) on (server xx.xx.xx.xx): error 10033
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on 
> file ./qa1jp/il_data02.dbf (rnode_pt: 300216757e
> 0) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
> 5811 for file ./qa1jp/streams_ts_01.dbf (rnode_pt
>  0x3

[nfs-discuss] Oracle DB crashed on NFS4 built on 7210 Unified Storage

2010-03-11 Thread Andrey Yasnogor

Tom, help me to understand what let you to conclude it was a storage problem 
and not Solaris/NFS issue?
I'm not convinced this being the case here, but did open an SR.
Tnx.

On 3/11/10 3:37 PM, "Tom Haynes"  wrote:

Andrey Yasnogor wrote:
> Anyone?
>

Andrey,

You should follow the support model in place for getting help on
the 7210.

Thanks,
Tom


> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS op OP_OPEN got error 
> NFS4ERR_EXPIRED causing recovery action NR_CLIENTID.
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Starting recovery for 
> mount /zones/zone2/root/orafiles (mi 0x600647b3000 mi
> _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 ./qa1/redo 
> (0x600543b2c68), rnode_pt2 ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8)
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 273629 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Recovery done for mount 
> /zones/zone2/root/orafiles (mi 0x600647b3000) on se
> rver xx.xx.xx.xx, rnode_pt1 ./qa1/redo (0x600543b2c68), rnode_pt2 
> ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8)
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 978989 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFSMAPID_DOMAIN does not match 
> the server: xx.xx.xx.xx domain
> Mar 11 08:44:00 v890-01-zn2 Please check configuration
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS op OP_WRITE got error 
> NFS4ERR_EXPIRED causing recovery action NR_CLIENTID.
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Starting recovery for 
> mount /zones/zone2/root/orafiles (mi 0x600647b3000 mi
> _recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 
> ./qa1jp/redo/controlqa1jp02.ctl (0x600671dac68), rnode_pt2  (0x0)
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
> 13091 for file ./qa1/sysaux01.dbf (rnode_pt 0x300
> 05308f70) on (server xx.xx.xx.xx): error 10033
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks 
> on file ./qa1/sysaux01.dbf (rnode_pt: 30005308f70)
>  due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 941083 kern.info] NOTICE: NFS4 FACT 
> SHEET:
> Mar 11 08:44:00 v890-01-zn2  Action: NR_CLIENTID
> Mar 11 08:44:00 v890-01-zn2  NFS4 error: NFS4ERR_EXPIRED
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
> 13091 for file ./qa1/qrtz_data01.dbf (rnode_pt 0x
> 30005306410) on (server xx.xx.xx.xx): error 10033
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks 
> on file ./qa1/qrtz_data01.dbf (rnode_pt: 300053064
> 10) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
> 5811 for file ./qa1jp/arc_jp_data01.dbf (rnode_pt
>  0x3002aaf9728) on (server xx.xx.xx.xx): error 10033
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on 
> file ./qa1jp/arc_jp_data01.dbf (rnode_pt: 3002aa
> f9728) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
> 13091 for file ./qa1/tools01.dbf (rnode_pt 0x3000
> 5308050) on (server xx.xx.xx.xx): error 10033
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks 
> on file ./qa1/tools01.dbf (rnode_pt: 30005308050)
> due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
> 5811 for file ./qa1jp/il_data02.dbf (rnode_pt 0x3
> 00216757e0) on (server xx.xx.xx.xx): error 10033
> Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
> xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on 
> file ./qa1jp/il_data02.dbf (rnode_pt: 300216757e
> 0) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
> Mar 11 08

[nfs-discuss] Oracle DB crashed on NFS4 built on 7210 Unified Storage

2010-03-11 Thread Andrey Yasnogor
Anyone? 

Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS op OP_OPEN got error 
NFS4ERR_EXPIRED causing recovery action NR_CLIENTID.
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Starting recovery for mount 
/zones/zone2/root/orafiles (mi 0x600647b3000 mi
_recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 ./qa1/redo (0x600543b2c68), 
rnode_pt2 ./qa1/redo/controlqa102.ctl (0x6005ae7a4c8)
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 273629 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Recovery done for mount 
/zones/zone2/root/orafiles (mi 0x600647b3000) on se
rver xx.xx.xx.xx, rnode_pt1 ./qa1/redo (0x600543b2c68), rnode_pt2 
./qa1/redo/controlqa102.ctl (0x6005ae7a4c8)
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 978989 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFSMAPID_DOMAIN does not match 
the server: xx.xx.xx.xx domain
Mar 11 08:44:00 v890-01-zn2 Please check configuration
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS op OP_WRITE got error 
NFS4ERR_EXPIRED causing recovery action NR_CLIENTID.
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 581112 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]NFS Starting recovery for mount 
/zones/zone2/root/orafiles (mi 0x600647b3000 mi
_recovflags [0x1]) on server xx.xx.xx.xx, rnode_pt1 
./qa1jp/redo/controlqa1jp02.ctl (0x600671dac68), rnode_pt2  (0x0)
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
13091 for file ./qa1/sysaux01.dbf (rnode_pt 0x300
05308f70) on (server xx.xx.xx.xx): error 10033
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks on 
file ./qa1/sysaux01.dbf (rnode_pt: 30005308f70)
 due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 941083 kern.info] NOTICE: NFS4 FACT SHEET: 
Mar 11 08:44:00 v890-01-zn2  Action: NR_CLIENTID 
Mar 11 08:44:00 v890-01-zn2  NFS4 error: NFS4ERR_EXPIRED   
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
13091 for file ./qa1/qrtz_data01.dbf (rnode_pt 0x
30005306410) on (server xx.xx.xx.xx): error 10033
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks on 
file ./qa1/qrtz_data01.dbf (rnode_pt: 300053064
10) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
5811 for file ./qa1jp/arc_jp_data01.dbf (rnode_pt
 0x3002aaf9728) on (server xx.xx.xx.xx): error 10033
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on 
file ./qa1jp/arc_jp_data01.dbf (rnode_pt: 3002aa
f9728) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
13091 for file ./qa1/tools01.dbf (rnode_pt 0x3000
5308050) on (server xx.xx.xx.xx): error 10033
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 13091 lost its locks on 
file ./qa1/tools01.dbf (rnode_pt: 30005308050) 
due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
5811 for file ./qa1jp/il_data02.dbf (rnode_pt 0x3
00216757e0) on (server xx.xx.xx.xx): error 10033
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on 
file ./qa1jp/il_data02.dbf (rnode_pt: 300216757e
0) due to NFS recovery error (NFS4ERR_NO_GRACE) on server xx.xx.xx.xx.
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 257235 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Couldn't reclaim lock for pid 
5811 for file ./qa1jp/streams_ts_01.dbf (rnode_pt
 0x30014dffbf0) on (server xx.xx.xx.xx): error 10033
Mar 11 08:44:00 v890-01-zn2 nfs: [ID 269213 kern.info] NOTICE: [NFS4][Server: 
xx.xx.xx.xx][Mntpt: /zones/zone2/root/orafiles]Process 5811 lost its locks on 
file ./qa1jp/streams_ts_01.dbf (rnode_pt: 30014d
ffbf0) due to NFS re