Re: [Lustre-discuss] ne2scan and fsfind
Hello I advise you to have a look at Robinhood, which has a ton of features to handle Lustre file management: http://robinhood.sf.net/ http://robinhood.sourceforge.net And if you think this does not fit your needs I will be very interested to know why. Regards, Aurélien Degrémont Andrus, Brian Contractor a écrit : Hello all! I have been googling around trying to find some good tools to handle the large fileystems we have grown here and see reference to ‘ne2scan’ and ‘fsfind’. They sound great, but I am not finding a place to get them. Does anyone out there know where these gems are hidden? Brian Andrus ITACS/Research Computing Naval Postgraduate School Monterey, California voice: 831-656-6238 ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
[Lustre-discuss] Kernel Panic error after lustre 2.0 installation
Hi there, I have got an error after installing lustre 2.0 on the MGS server with RAID controller card. The server OS is centOS 5.4 x86_64 and has 1.2TB storage which has configured by RAID 1+0. After installing lustre rpm packages and rebooting machine, I face with the errors below at linux startup: Unable to access resume device (LABEL=SWAP-sda3) mount: could not find filesystem 'dev/root' setuproot: moving /dev failed: No such file or directory setuproot: error mounting /proc: No such file or directory setuproot: error mounting /sys: No such file or directory swirchroot: mount failed: No such file or directory Kernel Panic - not syncing: Attempted to kill init! I have no problem with the original kernel installed by centos. I guessed this may be related to RAID controller card driver which may not loaded by the patched lustre kernel. so I have added the driver into the initrd.img file. But it didn't solve the problem. Should I install the lustre by building the source? Or any other clue to this problem? Thanks in advance... ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
Re: [Lustre-discuss] Kernel Panic error after lustre 2.0 installation
The driver name is arcmsr.ko and I extracted it from driver.img included in RAID controller's CD. The following text file may clarify better: ftp://areca.starline.de/RaidCards/AP_Drivers/Linux/DRIVER/RedHat/FedoraCore/Redhat-Fedora-core8/1.20.0X.15/Intel/readme.txt Please tell me, if you need more information about this issue... On Thu, Feb 17, 2011 at 11:33 PM, Brian J. Murrell br...@whamcloud.comwrote: On Thu, 2011-02-17 at 23:26 +0330, Arya Mazaheri wrote: Hi there, Hi, Unable to access resume device (LABEL=SWAP-sda3) mount: could not find filesystem 'dev/root' setuproot: moving /dev failed: No such file or directory setuproot: error mounting /proc: No such file or directory setuproot: error mounting /sys: No such file or directory swirchroot: mount failed: No such file or directory Kernel Panic - not syncing: Attempted to kill init! I have no problem with the original kernel installed by centos. I guessed this may be related to RAID controller card driver which may not loaded by the patched lustre kernel. That seems like a reasonable conclusion given the information available. so I have added the driver into the initrd.img file. Where did you get the driver from? What is the name of the driver? But it didn't solve the problem. Depending on where it came from, yes, it might not. Should I install the lustre by building the source? That may be required, but not necessarily required. We need more information. b. ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss signature.asc Description: PGP signature ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
Re: [Lustre-discuss] Kernel Panic error after lustre 2.0 installation
Yep. All you have to do is rebuild the driver for the Lustre kernel. First, bring the system back up with the non-Lustre kernel. See the bottom of the readme: # cd /usr/src/linux/drivers/scsi/arcmsr (suppose /usr/src/linux is the soft-link for /usr/src/kernel/2.6.23.1-42.fc8-i386) # make -C /lib/modules/`uname -r`/build CONFIG_SCSI_ARCMSR=m SUBDIRS=$PWD modules # insmod arcmsr.ko Except instead of uname -r substitute the lustre kernel's 'uname -r', as you want to build for the Lustre kernel. Be sure you have the Lustre kernel-devel RPM installed. Note that the insmod will not work (you already have it for the running kernel, and the one you built for the Lustre kernel will not work). You will need to rebuild the initrd for the Lustre kernel (see the other instructions in the readme, using the Lustre kernel). Kevin Arya Mazaheri wrote: The driver name is arcmsr.ko and I extracted it from driver.img included in RAID controller's CD. The following text file may clarify better: ftp://areca.starline.de/RaidCards/AP_Drivers/Linux/DRIVER/RedHat/FedoraCore/Redhat-Fedora-core8/1.20.0X.15/Intel/readme.txt Please tell me, if you need more information about this issue... On Thu, Feb 17, 2011 at 11:33 PM, Brian J. Murrell br...@whamcloud.com mailto:br...@whamcloud.com wrote: On Thu, 2011-02-17 at 23:26 +0330, Arya Mazaheri wrote: Hi there, Hi, Unable to access resume device (LABEL=SWAP-sda3) mount: could not find filesystem 'dev/root' setuproot: moving /dev failed: No such file or directory setuproot: error mounting /proc: No such file or directory setuproot: error mounting /sys: No such file or directory swirchroot: mount failed: No such file or directory Kernel Panic - not syncing: Attempted to kill init! I have no problem with the original kernel installed by centos. I guessed this may be related to RAID controller card driver which may not loaded by the patched lustre kernel. That seems like a reasonable conclusion given the information available. so I have added the driver into the initrd.img file. Where did you get the driver from? What is the name of the driver? But it didn't solve the problem. Depending on where it came from, yes, it might not. Should I install the lustre by building the source? That may be required, but not necessarily required. We need more information. b. ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org mailto:Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
Re: [Lustre-discuss] Lustre client error
To figure out which OST is which, use e2label /dev/sdX (or e2label /dev/mapper/mpath7) which will print the OST index in hex. If clients run out of space, but there is space left, see Bug 22755 (mostly fixed in Lustre 1.8.4). Lustre assigns the OST index at file creation time. Lustre will avoid full OSTs, but once a file is created any growth must be accommodated by the initial OST assignment(s). Deactivating the OST on the MDS will prevent new allocations, but they shouldn't be happening anyway. You can copy/rename some large files to put them on another OST which will free up space on the full OST (move will not allocate new space, just change the directory name). Kevin Jagga Soorma wrote: This OST is 100% now with only 12GB remaining and something is actively writing to this volume. What would be the appropriate thing to do in this scenario? If I set this to read only on the mds then some of my clients start hanging up. Should I be running lfs find -O OST_UID /lustre and then move the files out of this filesystem and re-add them back? But then there is no gurantee that they will not be written to this specific OST. Any help would be greately appreciated. Thanks, -J On Tue, Feb 15, 2011 at 3:05 PM, Jagga Soorma jagg...@gmail.com mailto:jagg...@gmail.com wrote: I might be looking at the wrong OST. What is the best way to map the actual /dev/mapper/mpath[X] to what OST ID is used for that volume? Thanks, -J On Tue, Feb 15, 2011 at 3:01 PM, Jagga Soorma jagg...@gmail.com mailto:jagg...@gmail.com wrote: Also, it looks like the client is reporting a different %used compared to the oss server itself: client: reshpc101:~ # lfs df -h | grep -i 0007 reshpcfs-OST0007_UUID 2.0T 1.7T202.7G 84% /reshpcfs[OST:7] oss: /dev/mapper/mpath72.0T 1.9T 40G 98% /gnet/lustre/oss02/mpath7 Here is how the data seems to be distributed on one of the OSS's: -- /dev/mapper/mpath52.0T 1.2T 688G 65% /gnet/lustre/oss02/mpath5 /dev/mapper/mpath62.0T 1.7T 224G 89% /gnet/lustre/oss02/mpath6 /dev/mapper/mpath72.0T 1.9T 41G 98% /gnet/lustre/oss02/mpath7 /dev/mapper/mpath82.0T 1.3T 671G 65% /gnet/lustre/oss02/mpath8 /dev/mapper/mpath92.0T 1.3T 634G 67% /gnet/lustre/oss02/mpath9 -- -J On Tue, Feb 15, 2011 at 2:37 PM, Jagga Soorma jagg...@gmail.com mailto:jagg...@gmail.com wrote: I did deactivate this OST on the MDS server. So how would I deal with a OST filling up? The OST's don't seem to be filling up evenly either. How does lustre handle a OST that is at 100%? Would it not use this specific OST for writes if there are other OST available with capacity? Thanks, -J On Tue, Feb 15, 2011 at 11:45 AM, Andreas Dilger adil...@whamcloud.com mailto:adil...@whamcloud.com wrote: On 2011-02-15, at 12:20, Cliff White wrote: Client situation depends on where you deactivated the OST - if you deactivate on the MDS only, clients should be able to read. What is best to do when an OST fills up really depends on what else you are doing at the time, and how much control you have over what the clients are doing and other things. If you can solve the space issue with a quick rm -rf, best to leave it online, likewise if all your clients are trying to bang on it and failing, best to turn things off. YMMV In theory, with 1.8 the full OST should be skipped for new object allocations, but this is not robust in the face of e.g. a single very large file being written to the OST that takes it from average usage to being full. On Tue, Feb 15, 2011 at 10:57 AM, Jagga Soorma jagg...@gmail.com mailto:jagg...@gmail.com wrote: Hi Guys, One of my clients got a hung lustre mount this morning and I saw the following errors in my logs: -- ..snip.. Feb 15 09:38:07 reshpc116 kernel: LustreError: 11-0: an error occurred while communicating with 10.0.250.47@o2ib3. The ost_write operation failed with -28 Feb 15 09:38:07 reshpc116 kernel: LustreError: Skipped 4755836 previous similar messages Feb 15 09:48:07 reshpc116 kernel: LustreError: 11-0: an
[Lustre-discuss] Is this setup possible
Hi, Is this setup possible.I want to install a patchless client on the client nodes. Following are details :- 1) Server has no OFED ; client has OFED 1.5.2. 2) Client / Server has Lustre 1.8.3. 3) Client Linux version : 2.6.30 Server Linux Version : 2.6.18-164.11.1 Is it possible that if I install different Linux version of Client/Server but same Lustre version on Client/Server the things will work fine. Also since I want to install patchless client on the client side do I need to restrict the Lustre setup to some specific version of the Linux Kernel. Thanks, Vilobh ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
[Lustre-discuss] Fwd: Query regarding Lustre setup
Hi, Is this setup possible.I want to install a patchless client on the client nodes. Following are details :- 1) Server has no OFED ; client has OFED 1.5.2. 2) Client / Server has Lustre 1.8.3. 3) Client Linux version : 2.6.30 Server Linux Version : 2.6.18-164.11.1 Is it possible that if I install different Linux version of Client/Server but same Lustre version on Client/Server the things will work fine. Also since I want to install patchless client on the client side do I need to restrict the Lustre setup to some specific version of the Linux Kernel. Thanks, Vilobh ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
[Lustre-discuss] Query regarding Lustre Setup
Hi, Is this setup possible.I want to install a patchless client on the client nodes. Following are details :- 1) Server has no OFED ; client has OFED 1.5.2. 2) Client / Server has Lustre 1.8.3. 3) Client Linux version : 2.6.30 Server Linux Version : 2.6.18-164.11.1 Is it possible that if I install different Linux version of Client/Server but same Lustre version on Client/Server the things will work fine. Also since I want to install patchless client on the client side do I need to restrict the Lustre setup to some specific version of the Linux Kernel. Thanks, Vilobh ** ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
Re: [Lustre-discuss] Is this setup possible
All the nodes have to run the same network type, so they can talk to one another. If client is runnng Infiniband, server must also run Infiniband, in most cases. See the Lustre Manual for information on Lustre Routing. Clients and server can run different versions of Lustre. You need to run a version of the kernel supported by Lustre, even with patchless clients. See Lustre release notes for kernel versions. cliffw On Thu, Feb 17, 2011 at 2:54 PM, vilobh meshram meshram.vil...@gmail.comwrote: Hi, Is this setup possible.I want to install a patchless client on the client nodes. Following are details :- 1) Server has no OFED ; client has OFED 1.5.2. 2) Client / Server has Lustre 1.8.3. 3) Client Linux version : 2.6.30 Server Linux Version : 2.6.18-164.11.1 Is it possible that if I install different Linux version of Client/Server but same Lustre version on Client/Server the things will work fine. Also since I want to install patchless client on the client side do I need to restrict the Lustre setup to some specific version of the Linux Kernel. Thanks, Vilobh ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
