Process running which is either from gpfs or using it. We used to see it when we ran torque from gpfs and forgot to stop it before shutting down gpfs.
Or if you have dsmrecalld (for example) running on the node. Simon ________________________________________ From: [email protected] [[email protected]] on behalf of Tejas Rao [[email protected]] Sent: 29 June 2016 16:58 To: gpfsug main discussion list Subject: Re: [gpfsug-discuss] mount: /dev/gpfs01 already mounted or /gpfs01 busy "umount -l" returns with "not mounted" message. See below. [root@ ~]# umount -l /gpfs01 umount: /gpfs01: not mounted [root@ ~]# umount -l /dev/gpfs01 umount: /dev/gpfs01: not mounted [root@ ~]# [root@ ~]# /usr/lpp/mmfs/bin/mmmount gpfs01 Wed Jun 29 11:56:58 EDT 2016: mmmount: Mounting file systems ... mount: /dev/gpfs01 already mounted or /gpfs01 busy mmmount: Command failed. Examine previous error messages to determine cause. Thanks. On 6/29/2016 11:46, Buterbaugh, Kevin L wrote: Hi Tejas, One thing that I have had occasional success with is a lazy unmount - i.e. “unmount -l /gpfs01”. Doesn’t help more of the time than it does help in my experience, but it’s one thing I try before bouncing a node. HTHAL… Kevin On Jun 29, 2016, at 10:43 AM, Tejas Rao <<mailto:[email protected]>[email protected]<mailto:[email protected]>> wrote: Hi all, I cannot get the filesystem mounted again on few nodes. [root@ ~]# /usr/lpp/mmfs/bin/mmmount gpfs01 Wed Jun 29 11:29:50 EDT 2016: mmmount: Mounting file systems ... mount: /dev/gpfs01 already mounted or /gpfs01 busy mmmount: Command failed. Examine previous error messages to determine cause. Things I have tried and checked. 1. Checked /proc/mounts and /etc/mtab and lsof /dev/gpfs01 and both /dev/gpfs01 and /gpfs01 are not seen anywhere. 2. Tried mmshutdown/mmstartup and it fails to unload mmfs modules. See below. 3. mmumount gpfs01 -f A reboot of the client would probably fix it but as the client is used in production, this is not easy to do. Are there any tricks/hacks to remount the filesystem again without rebooting the client. [root@~]# /usr/lpp/mmfs/bin/mmshutdown Wed Jun 29 11:10:52 EDT 2016: mmshutdown: Starting force unmount of GPFS file systems Wed Jun 29 11:10:57 EDT 2016: mmshutdown: Shutting down GPFS daemons Shutting down! 'shutdown' command about to kill process 11431 Unloading modules from /lib/modules/2.6.32-504.8.1.el6.x86_64/extra mmfsenv: Module mmfslinux is still in use. Unmount all GPFS file systems and issue the command: mmfsadm cleanup Unloading module mmfs26 ERROR: Module mmfs26 is in use mmfsenv: Error unloading module mmfs26. Unloading module mmfslinux ERROR: Module mmfslinux is in use by mmfs26 mmfsenv: Error unloading module mmfslinux. Wed Jun 29 11:11:03 EDT 2016: mmshutdown: Finished [root@~]# Thanks. _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org<http://spectrumscale.org> http://gpfsug.org/mailman/listinfo/gpfsug-discuss — Kevin Buterbaugh - Senior System Administrator Vanderbilt University - Advanced Computing Center for Research and Education [email protected]<mailto:[email protected]> - (615)875-9633 _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
