If you're using your Isilon for NFS backend, I would make the following recommendations: * Use multiple mountpoints to the Isilon from your TSM server (e.g. Isilon Client). I would use IP addresses or direct IP connection to the underlying Isilon nodes to spread the load out against the entire Isilon cluster (manually). * Bypass SmartConnect, if that's a licensed thing for you. Don't worry about failing over your mount points using smartconnect for this. In the worst case emergencies, you're only impacting backups processing and can easily reconfigure the mountpoint manually in case of a node outage. If it's that important, then use a smartconnect policy that makes sense for TSM deployments (probably round-robin, not one of those fancier load-balancing options). * Use many mount points and directories on the TSM server in the devclass. I have 16 filesystems (aka mounts) per each TSM server devclass. * Use subdirectories, as suggested in a earlier post, not whole new exports, although that should work too, to separate the mount points so you don't run into NFS client locking issues. * Predefine your file volume sizes. I think directory container pools might be a better option to use in the future, but that forces you into dedup, IIRC.
Thanks! SF On Mon, May 14, 2018 at 10:25 AM, Skylar Thompson <skyl...@uw.edu> wrote: > No, this is an NFS server setting, but I'm not sure that it's tunable on > Isilon. On Linux and Solaris, it defaults to some very low value, which is > fine for sequential I/O but really slows down random I/O. On Linux, > RPCNFSDCOUNT can be tuned from the default of 8 to 512, which is fine as > long as the NFS server is running on dedicated hardware. > > On Mon, May 14, 2018 at 10:06:31AM -0400, Zoltan Forray wrote: > > We did some quick research and "NFS thread" controls don't apply in our > > situation and can't be set. Or are you referring to the mountlimit value > > for the devclass? > > > > On Mon, May 14, 2018 at 9:31 AM, Skylar Thompson <skyl...@uw.edu> wrote: > > > > > This sounds pretty good to me. If you can, I would boost your NFS > thread > > > count past the number of CPUs that you have, since a lot of NFS is just > > > waiting for the disks to respond. You still need a thread for that, > but it > > > won't consume much CPU. > > > > > > On Mon, May 14, 2018 at 08:27:27AM -0400, Zoltan Forray wrote: > > > > Very interesting. This supports my idea on how I want to layout the > > > > new/replacement server. The old server is only 16-threads and > certainly > > > > could not handle dedup (we can't afford any appliances like DD) > since it > > > is > > > > bucking under the current backups traffic. The new server has > 72-threads > > > as > > > > well as 100TB internal disk. My idea is to use the fast internal > 100TB > > > > disk for inbound traffic and deduping and use the 200TB NFS/ISILON > for > > > > nextstoragepool (trying to get completely off 3592-tape storage for > > > onsite > > > > backups). Plus the DB will be on SSD. > > > > > > > > Any thoughts on this configuration? > > > > > > > > On Sun, May 13, 2018 at 8:38 PM, Harris, Steven < > > > > steven.har...@btfinancialgroup.com> wrote: > > > > > > > > > Zoltan > > > > > > > > > > I have a similar issue TSM 7.1.1.300 AIX -> Data Domain. Have dual > > > 10Gb > > > > > links, but can only get ~4000 writes/sec and 120MB/sec throughput. > AIX > > > only > > > > > supports NFS3, and as others have pointed out in this forum > recently, > > > the > > > > > stack does not have a good reputation. > > > > > > > > > > I'm finding that the heavy NFS load has other knock on effects, > e.g. > > > > > TSMManager keeps reporting the instance offline when it's very > busy as > > > it > > > > > gets a network error on some of its regular queries, but these > work ok > > > when > > > > > load is light. Also getting a lot of Severed/reconnected sessions. > > > > > CPU/IO/Paging are not a problem. > > > > > > > > > > Cheers > > > > > > > > > > Steve > > > > > > > > > > Steven Harris > > > > > TSM Admin/Consultant > > > > > Canberra Australia > > > > > > > > > > -----Original Message----- > > > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > Behalf > > > Of > > > > > Zoltan Forray > > > > > Sent: Saturday, 12 May 2018 1:39 AM > > > > > To: ADSM-L@VM.MARIST.EDU > > > > > Subject: [ADSM-L] ISILON storage/FILE DEVCLASS performance issues > > > > > > > > > > Folks, > > > > > > > > > > ISP 7.1.7.300 on RHEL 6 10G connectivity > > > > > > > > > > We need some guidance on trying to figure out why ISP/TSM write > > > perform to > > > > > ISILON storage (via FILE DEVCLASS) is so horrible. > > > > > > > > > > We recently attached 200TB of ISILON storage to this server so we > could > > > > > empty the 36TB of onboard disk drives to move this server to new > > > hardware. > > > > > > > > > > However, per my OS and SAN folks, we are only seeing 1Gbs level of > data > > > > > movement from the ISP server. Doing a regular file copy to this > same > > > > > storage peaks at 10Gbs speeds. > > > > > > > > > > So what, if anything, are we doing wrong when it comes to > configuring > > > the > > > > > storage for ISP to use? Are there some secret > > > controls/settings/options to > > > > > tell it to use the storage at max-speeds? > > > > > > > > > > We tried changing the Est/Max capacity thinking larger files would > > > reduce > > > > > the overhead of allocating new pieces constantly. Changed the > Mount > > > Limit > > > > > to a bigger number. Nothing has helped. > > > > > > > > > > Only thing uses the storage right now is migrations from the > original > > > disk > > > > > stgpool. > > > > > > > > > > > > > > > > > > > > -- > > > > > *Zoltan Forray* > > > > > Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator > Xymon > > > > > Monitor Administrator VMware Administrator Virginia Commonwealth > > > University > > > > > UCC/Office of Technology Services www.ucc.vcu.edu zfor...@vcu.edu > - > > > > > 804-828-4807 Don't be a phishing victim - VCU and other reputable > > > > > organizations will never use email to request that you reply with > your > > > > > password, social security number or confidential personal > information. > > > For > > > > > more details visit http://phishing.vcu.edu/ > > > > > > > > > > > > > > > This message and any attachment is confidential and may be > privileged > > > or > > > > > otherwise protected from disclosure. You should immediately delete > the > > > > > message if you are not the intended recipient. If you have received > > > this > > > > > email by mistake please delete it from your system; you should not > > > copy the > > > > > message or disclose its content to anyone. > > > > > > > > > > This electronic communication may contain general financial product > > > advice > > > > > but should not be relied upon or construed as a recommendation of > any > > > > > financial product. The information has been prepared without taking > > > into > > > > > account your objectives, financial situation or needs. You should > > > consider > > > > > the Product Disclosure Statement relating to the financial product > and > > > > > consult your financial adviser before making a decision about > whether > > > to > > > > > acquire, hold or dispose of a financial product. > > > > > > > > > > For further details on the financial product please go to > > > > > http://www.bt.com.au > > > > > > > > > > Past performance is not a reliable indicator of future performance. > > > > > > > > > > > > > > > > > > > > > -- > > > > *Zoltan Forray* > > > > Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator > > > > Xymon Monitor Administrator > > > > VMware Administrator > > > > Virginia Commonwealth University > > > > UCC/Office of Technology Services > > > > www.ucc.vcu.edu > > > > zfor...@vcu.edu - 804-828-4807 > > > > Don't be a phishing victim - VCU and other reputable organizations > will > > > > never use email to request that you reply with your password, social > > > > security number or confidential personal information. For more > details > > > > visit http://phishing.vcu.edu/ > > > > > > -- > > > -- Skylar Thompson (skyl...@u.washington.edu) > > > -- Genome Sciences Department, System Administrator > > > -- Foege Building S046, (206)-685-7354 > > > -- University of Washington School of Medicine > > > > > > > > > > > -- > > *Zoltan Forray* > > Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator > > Xymon Monitor Administrator > > VMware Administrator > > Virginia Commonwealth University > > UCC/Office of Technology Services > > www.ucc.vcu.edu > > zfor...@vcu.edu - 804-828-4807 > > Don't be a phishing victim - VCU and other reputable organizations will > > never use email to request that you reply with your password, social > > security number or confidential personal information. For more details > > visit http://phishing.vcu.edu/ > > -- > -- Skylar Thompson (skyl...@u.washington.edu) > -- Genome Sciences Department, System Administrator > -- Foege Building S046, (206)-685-7354 > -- University of Washington School of Medicine >
