Some additional questions about benchmarking CIFS servers
Hi, During some discussions yesterday with some folks, I realized that a couple of additional deficiencies that NetBench has are: 1. All the clients connect at the beginning of the benchmark, and do not disconnect until the end of the benchmark. Connection handling is an important aspect of a CIFS benchmark, and while you do not expect every client to be disconnecting and reconnecting every five seconds, you do expect a certain connection load, perhaps 5% of the overall clients would connect and disconnect from the server twice during the benchmark. Secondly, the NetBench benchmark does not disconnect after the setup phase. Indeed, the way it works seems to prevent this (you net use the server manually or through a script on each client), however, servers will cache on a per-client basis and the server may have cached the files that were created in the setup phase. 2. Likewise, logon handling is not explored at all by the NetBench benchmark. To what extent is it important to have a schedule of accounts and cycle through them (and to what extent can rpcclient be used to create accounts on the server)? 3. NetBench uses a very small number of SMB operations as far as I can tell (12 or 13) and does not explore some areas that I feel should be used, like locking (guess what I hacked into my program last night). There are others, however, like change notify, etc. To what extent do people feel that these are relevant to a CIFS benchmark, and what other areas have I missed. Regards - Richard Sharpe, [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED]
Re: Some additional questions about benchmarking CIFS servers
On Sat, Jun 22, 2002 at 02:25:41AM +0930, Richard Sharpe wrote: To what extent do people feel that these are relevant to a CIFS benchmark, and what other areas have I missed. Well in defense of NetBench (I can't believe I'm saying this :-) it doesn't attempt to give full coverage of SMB, that would be the job of a conformance tester. It only tries to shovel data to the server as fast as possible, trying to create realistic capacity situations. Just having a simple, Open Source/Free Software benchmark that would do the same would be a good start, we can argue about how to extend it once we have buy in that the new benchmark makes a good NetBench replacement (ie. can we get the magazines to run it instead of NetBench). Jeremy.
Re: Some additional questions about benchmarking CIFS servers
NetBench is a protocol agnostic benchmark. You can connect via CIFS, NFS, NCP, etc. Having said that, I don't think NetBench is that great of a benchmarking utility. Response times are at least as important as bandwidth. http://www.netapp.com/ftp/usenix-nt97.pdf has some good info on how latency can looked at when using NetBench (I haven't looked at recent versions of NetBench, which seem to have some response time stuff added in. Maybe it does all of that now). I'm hoping to do some benchmarking soon. The University I work for is currently evaluating NetWare (what we currently use), Windows 2000 and Samba for our future file/print solution. We've got 3 Dell 2550 servers, and a Dell|EMC Clariion 4500 which I'm hoping I'll be allowed to use for some benchmarking. I might be able to use up to about 200 lab workstations. My plan is to compare NetWare 6.0, Windows 2000 Advanced Server and Samba, with all running on identical servers connected to SAN hardware. There would be 3 NetWare benchmarks - NCP to NSS filesystem, CIFS to traditional filesystem CIFS to NSS filesystem Windows 2000 Advanced server would just be CIFS to NTFS (FAT doesn't have ACLs) Samba would run on Linux, and would compare all journaling filesystems that support ACLs. That's the plan, at least. I'm hoping to get it approved. I'm really curious to see NetWare vs. Samba numbers. From NetBench results I've found on the net, it seems like they both have close to twice the peak bandwith of Windows 2000, but I haven't seen any results of CIFS access to NetWare. One thing I would like to test for, but I'm not quite sure how to go about it, is how many concurrent connections can each solution support? NetBench doesn't really tell you that. It tells you how the server performs with a given number of active users. Most users are idle most of the time. If we swtich from our current NetWare environment to Linux/Samba, it would be nice to know if we need more servers, the same # of servers, or less servers. I think I know the answer to that if we go with Windows... Well, I don't actually - that's why I'd like to be able to test it.
Re: Some additional questions about benchmarking CIFS servers
On Sat, 22 Jun 2002, Richard Sharpe wrote: During some discussions yesterday with some folks, I realized that a couple of additional deficiencies that NetBench has are: 1. All the clients connect at the beginning of the benchmark, and do not disconnect until the end of the benchmark. Connection handling is an important aspect of a CIFS benchmark, and while you do not expect every client to be disconnecting and reconnecting every five seconds, you do expect a certain connection load, perhaps 5% of the overall clients would connect and disconnect from the server twice during the benchmark. I'll second that! One of the main problems we had a year ago on our main fileserver (Solaris, Sun 450, up to 1,000 simultaneous smbd connections) when we tried moving from Samba 2.0.x to 2.2.x (and then through the 2.2.x series) was the avalanching of smbds. Contention on connections.tdb seemed to be a contributory factor. I understand that this was compounded by oplock-related issues, but nevertheless, we were typically averaging one operation per second on the database. (Student classrooms spread across campus; burst activity as lectures finished; etc.) The machine was already loaded with the routine activity of established smbds. So in our experience and environment, the connect/disconnect activity seems to be a significant factor in loading a Samba server. Hope that helps. -- : David LeeI.T. Service : : Systems Programmer Computer Centre : : University of Durham : : http://www.dur.ac.uk/t.d.lee/South Road: : Durham: : Phone: +44 191 374 2882 U.K. :
