Re: hardware requirements per hits
On Wed, Aug 19, 2009 at 12:47 AM, Subhan Malickmali...@illinois.edu wrote: On 8/17/09 10:15 PM, Fajar A. Nugraha wrote: Here are some pointers from my experience though: - syslog query logging is expensive. NEVER enable it. If you need to log client queries, log it directly to file instead. I would like to hear more about why this is so. We are currently debating sending query logs to a remote syslog server to enhance some security tools. It depends on your requirement. In my case, sending query log to syslog makes disk I/O the bottleneck. Not really sure why logging to file directly fix this issue, perhaps syslog does a sync() for every line or something. We are running BIND 9.6.1-P1 with multithreading enabled on RHEL 4 (2 dual-core 2.8 GHz Opterons with 1MB cache, 4G of RAM). I have run some tests and while there is some queries/sec hit, the RTTs are not terrible. Queries per second: 2425.385916 qps I got around 6000 qps on a smiliar test. Jinmei mentioned something about getting 24k qps on a 4-way Opteron. Again, it depends on your requirement. If your load is low enough, you might be able to live with performance penalty imposed by syslog. -- Fajar ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: hardware requirements per hits
On 8/17/09 10:15 PM, Fajar A. Nugraha wrote: Here are some pointers from my experience though: - syslog query logging is expensive. NEVER enable it. If you need to log client queries, log it directly to file instead. On Wed, Aug 19, 2009 at 12:47 AM, Subhan Malickmali...@illinois.edu wrote: I would like to hear more about why this is so. We are currently debating sending query logs to a remote syslog server to enhance some security tools. On 19.08.09 16:11, Fajar A. Nugraha wrote: It depends on your requirement. In my case, sending query log to syslog makes disk I/O the bottleneck. Not really sure why logging to file directly fix this issue, perhaps syslog does a sync() for every line or something. yes, unless you configure it not to do so. See man page of your syslog version. There's also some overhead when sending log line to another process either via network or local socket and parsing that line in the another process. Logging to file is just faster and more reliable unless you use remote logging features of syslog. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. One World. One Web. One Program. - Microsoft promotional advertisement Ein Volk, ein Reich, ein Fuhrer! - Adolf Hitler ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
RE: hardware requirements per hits
@Bill, @Kevin @Fajar Thanks for your interesting explanations. I assumed that Bind (alone and regardless of other variables), I assumed it has a limitation for a certain number of hits per hardware. After reading your reply, now I understand that Bind can handle same load on less hardware requirement but in different environment, please correct me if I'm wrong. I admit when I said hit I didn't think of difference between cached and other types so thanks again. Regards, Alans -Original Message- From: Fajar A. Nugraha [mailto:fa...@fajar.net] Sent: Tuesday, August 18, 2009 6:15 AM To: Alans Cc: bind-users@lists.isc.org Subject: Re: hardware requirements per hits On Mon, Aug 17, 2009 at 8:50 PM, Alansbatpowe...@yahoo.co.uk wrote: @Matus: let me put it in this way, if I want to create a budget for next year for example, then I should know what upgrades I need for next year (estimated needs), and let's assume dns queries increase monthly by x hits, now, if I know how many hits will make me upgrade cpu and memory then I can find out my cpu and memory needs for next year, hope this explain to you why my question is not usless, at least for me. I'll be happy if you tell me another way to know my needs for next year. I'm assuming you already have a running DNS server? In that case I'd simply gather stats from it. What kind of hardware it currently has, how much is current CPU and disk load, how many queries per second it currently serves, etc. Based on that you can have a rough estimate as to what you'd need to upgrade. Here are some pointers from my experience though: - syslog query logging is expensive. NEVER enable it. If you need to log client queries, log it directly to file instead. - disk I/O can be a serious bottleneck. If that's the case consider disable logging. - BIND would generally work better with faster CPU compared to multiple CPUs/cores, e.g. 1 x 3GHZ CPU could outperform 2 x 1.5GHz CPU. - memory cache can speedup things to a point. Try allocating about 2-4G when you're handing lots of clients. Those are very general pointers though, YMMV. You might find it easier to simply add aonther server instead of upgrading. -- Fajar ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: hardware requirements per hits
On 8/17/09 10:15 PM, Fajar A. Nugraha wrote: Here are some pointers from my experience though: - syslog query logging is expensive. NEVER enable it. If you need to log client queries, log it directly to file instead. I would like to hear more about why this is so. We are currently debating sending query logs to a remote syslog server to enhance some security tools. We are running BIND 9.6.1-P1 with multithreading enabled on RHEL 4 (2 dual-core 2.8 GHz Opterons with 1MB cache, 4G of RAM). I have run some tests and while there is some queries/sec hit, the RTTs are not terrible. - queryperf -d input/queries.log.34 -s hidden.server -q 200 -l 300 -b 32 running 9.6.1-P1 with querylogging on and directed to remote syslog Statistics: Parse input file: multiple times Run time limit: 300 seconds Ran through file: 0 times Queries sent: 746189 queries Queries completed:739501 queries Queries lost: 6688 queries Queries delayed(?): 0 queries RTT max: 5.003000 sec RTT min: 0.000197 sec RTT average: 0.036471 sec RTT std deviation:0.204566 sec RTT out of range: 1 queries Percentage completed: 99.10% Percentage lost:0.90% Started at: Tue Aug 18 11:22:50 2009 Finished at: Tue Aug 18 11:27:55 2009 Ran for: 304.900344 seconds Queries per second: 2425.385916 qps - cache flushed queryperf -d input/queries.log.34 -s hidden.server -q 200 -l 300 -b 32 running 9.6.1-P1 with querylogging on and directed to local disk Statistics: Parse input file: multiple times Run time limit: 300 seconds Ran through file: 0 times Queries sent: 982436 queries Queries completed:973645 queries Queries lost: 8791 queries Queries delayed(?): 0 queries RTT max: 4.999350 sec RTT min: 0.000219 sec RTT average: 0.016778 sec RTT std deviation:0.152307 sec RTT out of range: 0 queries Percentage completed: 99.11% Percentage lost:0.89% Started at: Tue Aug 18 11:29:08 2009 Finished at: Tue Aug 18 11:34:13 2009 Ran for: 304.979150 seconds Queries per second: 3192.496930 qps - cache flushed queryperf -d input/queries.log.34 -s hidden.server -q 200 -l 300 -b 32 running 9.6.1-P1 with querylogging off Statistics: Parse input file: multiple times Run time limit: 300 seconds Ran through file: 0 times Queries sent: 1027578 queries Queries completed:1018243 queries Queries lost: 9335 queries Queries delayed(?): 0 queries RTT max: 5.043680 sec RTT min: 0.08 sec RTT average: 0.013455 sec RTT std deviation:0.142308 sec RTT out of range: 1 queries Percentage completed: 99.09% Percentage lost:0.91% Started at: Tue Aug 18 11:35:27 2009 Finished at: Tue Aug 18 11:40:32 2009 Ran for: 304.932400 seconds Queries per second: 3339.241747 qps - This server is a caching-forwarder with max-cache-ttl (and max-ncache-ttl) set to 15 mins. It has 4G of memory with no limit enforced in named.conf. version: 9.6.1-P1 CPUs found: 4 worker threads: 4 number of zones: 12 debug level: 0 xfers running: 0 xfers deferred: 0 soa queries in progress: 0 query logging is ON recursive clients: 0/5900/6000 tcp clients: 0/100 server is up and running ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: hardware requirements per hits
On 17.08.09 08:43, Alans wrote: Ok, please tell me what details you need. I guess number of hits is related to increase in CPU usage, for example 1000 simultaneously hits will make cpu usage 10% and so on, I need that number. I'm afraid this ias jusst useless question. The only usefull question is what hardware you need to be able to process your traffic. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Support bacteria - they're the only culture some people have. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
RE: hardware requirements per hits
@Matus: let me put it in this way, if I want to create a budget for next year for example, then I should know what upgrades I need for next year (estimated needs), and let's assume dns queries increase monthly by x hits, now, if I know how many hits will make me upgrade cpu and memory then I can find out my cpu and memory needs for next year, hope this explain to you why my question is not usless, at least for me. I'll be happy if you tell me another way to know my needs for next year. Alans -Original Message- From: bind-users-boun...@lists.isc.org [mailto:bind-users-boun...@lists.isc.org] On Behalf Of Matus UHLAR - fantomas Sent: Monday, August 17, 2009 12:18 PM To: bind-users@lists.isc.org Subject: Re: hardware requirements per hits On 17.08.09 08:43, Alans wrote: Ok, please tell me what details you need. I guess number of hits is related to increase in CPU usage, for example 1000 simultaneously hits will make cpu usage 10% and so on, I need that number. I'm afraid this ias jusst useless question. The only usefull question is what hardware you need to be able to process your traffic. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Support bacteria - they're the only culture some people have. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
RE: hardware requirements per hits
Alans batpowe...@yahoo.co.uk said: @Matus: let me put it in this way, if I want to create a budget for next year for example, then I should know what upgrades I need for next year (estimated needs), and let's assume dns queries increase monthly by x hits, now, if I know how many hits will make me upgrade cpu and memory then I can find out my cpu and memory needs for next year, hope this explain to you why my question is not usless, at least for me. I'll be happy if you tell me another way to know my needs for next year. I won't necessarily go so far as to say this question is useless, but it is almost impossible to answer. You aren't telling us what you current situation is; what you are seeing for a query load, whether you are running a server for authoritative data or a resolving server, if you are using DNSSEC, etc. This is to say nothing about your current hardware; CPU, memory, network/Internet connectivity, etc. Also, are your running anything else on the same platform as your DNS server? My best suggestion is to test your environment yourself. Run queryperf (or, Nominum's dnsperf, for authoritative servers, and resperf, for caching resolvers, tools) to determine what sort of level your servers can support. Then, compare this result to the level of service that you are currently seeing. This will give you an idea of what level of service your systems can provide. If the maximum performance cannot meet your expectations, then you need an upgrade. If they do meed your expectations, then you are ok. Simple enough. But, some questions that only you can answer for your situation. Who is querying your server? What queries are you currently receiving and answering? How are your servers currently performing? What is the bottle neck that you are seeing? Is it CPU? Memory? Disk I/O? Network? (Bottleneck is sort of a bad term here. A better phrase is What part of the system is limiting your performance?) How long does it take for your server to completely start AND is this a problem for you? (More zones, and large zones, makes for a slower startup. But, if it starts fast enough for you then it is acceptable.) I used to run a major DNS server on a microVAX II handling 500 queries per second (a LONG time ago). BIND doesn't necessarily require much CPU. It can require lots of memory. (And if you are running other services on the same system, then these other services have to share memory with your DNS server.) Rarely, is disk I/O an issue, unless you are seeing excessive swapping/paging, which says you need more memory. Network or Internet connectivity isn't normally an issue either (DNS traffic doesn't have to be excessive if your systems are reasonably configured.) If you are using DNSSEC, or are planning on it, you can expect to use more CPU. Again, all of these things are things that you can determine yourself by testing your server performance. I remember a quote from somewhere a long time ago. If you don't know how your system is running now when things are good, how do you expect to be able to say what is causing a problem in the future when things are bad? (I suspect that this cam from System Performance Tuning by Mike Loukides, O'Reilly Assc. My copy is quite old but still useful.) Know how your system is performing BEFORE there is a problem. Bill Larson ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: hardware requirements per hits
And this is why we try to keep beancounters away from the technical folks... There are just so many variables here. Are all the clients looking up essentially the *same* names? Or are the new clients looking up *different* names than the old ones? This has an impact on cache hit ratio, which then has an cascading impact on network traffic, query residency, latency, etc. Will the next big Internet fad involve 5 DNS names? Or 50? Or 250? How frequently will those names be looked up, and, is the target app conducive to DNS-based load-balancing with low TTL values? What OS are you running on? Does it scale well with volume? How about the network implementation? Is it sockets-based? How well does it manage its buffers? Do you have competent administrators watching your systems, tuning them for performance? Proactively? Reactively? Would you consider jumping ship to another OS that scales better? Consider the migration costs and logistics. What about the option of buying more servers as volume increases, rather than trying to upgrade the existing servers? Do you have rack space? Electrical? Air conditioning? Backup generator? Site licenses? How's your monitoring and event-management infrastructure? Can it handle an influx of new servers? Are your client apps well-behaved enough that they can tolerate a failover from one resolver to another in case of a single-point failure? How about two failovers, in the case of a multiple-point failure? Are your network architects competent enough to implement anycast? I could go on... - Kevin Alans wrote: @Matus: let me put it in this way, if I want to create a budget for next year for example, then I should know what upgrades I need for next year (estimated needs), and let's assume dns queries increase monthly by x hits, now, if I know how many hits will make me upgrade cpu and memory then I can find out my cpu and memory needs for next year, hope this explain to you why my question is not usless, at least for me. I'll be happy if you tell me another way to know my needs for next year. Alans -Original Message- From: bind-users-boun...@lists.isc.org [mailto:bind-users-boun...@lists.isc.org] On Behalf Of Matus UHLAR - fantomas Sent: Monday, August 17, 2009 12:18 PM To: bind-users@lists.isc.org Subject: Re: hardware requirements per hits On 17.08.09 08:43, Alans wrote: Ok, please tell me what details you need. I guess number of hits is related to increase in CPU usage, for example 1000 simultaneously hits will make cpu usage 10% and so on, I need that number. I'm afraid this ias jusst useless question. The only usefull question is what hardware you need to be able to process your traffic. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: hardware requirements per hits
On Mon, Aug 17, 2009 at 8:50 PM, Alansbatpowe...@yahoo.co.uk wrote: @Matus: let me put it in this way, if I want to create a budget for next year for example, then I should know what upgrades I need for next year (estimated needs), and let's assume dns queries increase monthly by x hits, now, if I know how many hits will make me upgrade cpu and memory then I can find out my cpu and memory needs for next year, hope this explain to you why my question is not usless, at least for me. I'll be happy if you tell me another way to know my needs for next year. I'm assuming you already have a running DNS server? In that case I'd simply gather stats from it. What kind of hardware it currently has, how much is current CPU and disk load, how many queries per second it currently serves, etc. Based on that you can have a rough estimate as to what you'd need to upgrade. Here are some pointers from my experience though: - syslog query logging is expensive. NEVER enable it. If you need to log client queries, log it directly to file instead. - disk I/O can be a serious bottleneck. If that's the case consider disable logging. - BIND would generally work better with faster CPU compared to multiple CPUs/cores, e.g. 1 x 3GHZ CPU could outperform 2 x 1.5GHz CPU. - memory cache can speedup things to a point. Try allocating about 2-4G when you're handing lots of clients. Those are very general pointers though, YMMV. You might find it easier to simply add aonther server instead of upgrading. -- Fajar ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: hardware requirements per hits
Alans wrote: Hi, I want to know when we need hardware upgrade. How many queries will use 50% of cpu and memory? FYI this question is impossible to answer without a lot more details. Doug ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
RE: hardware requirements per hits
Ok, please tell me what details you need. I guess number of hits is related to increase in CPU usage, for example 1000 simultaneously hits will make cpu usage 10% and so on, I need that number. Regards, Alans -Original Message- From: Doug Barton [mailto:do...@dougbarton.us] Sent: Monday, August 17, 2009 7:08 AM To: Alans Cc: bind-users@lists.isc.org Subject: Re: hardware requirements per hits Alans wrote: Hi, I want to know when we need hardware upgrade. How many queries will use 50% of cpu and memory? FYI this question is impossible to answer without a lot more details. Doug ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
