Hi! > > Fine, this is a lot better now. Since you're running at 2000 concurrent > connections, the impact on the cache is noticeable (at 32kB per connection > for haproxy, it's 64MB of RAM possibly touched each second, maybe only 16MB > since requests are short and fit in a single page). Could you recheck at > only 250 concurrent connections in total (125 per ab) ? This usually is > the optimal point I observe. I'm not saying that it should be your target, > but we're chasing the issues :-) >
Using one/two clients with ab2 -c 250 -H 'Accept-Encoding: None' -n 100000000 http://testhost I can get: Requests per second: 25360.45 [#/sec] (mean) Connection Times (ms) min mean[+/-sd] median max Connect: 0 6 94.8 3 9017 Processing: 0 4 16.7 3 5002 Waiting: 0 0 10.3 0 5002 Total: 0 10 96.3 6 9022 Percentage of the requests served within a certain time (ms) 50% 6 66% 7 75% 8 80% 8 90% 9 95% 10 98% 11 99% 13 100% 9022 (longest request) Maximum session rate is 49000 (exact value). Also, I've found reason why SYN packets were lost. It looks like it happened because of slow interrupt handling (same ksoftirq/0 at 100%), ifconfig eth0 reports dropped packets. I've tried to set ethtool -g eth0 rx 2040 and dropped packets are gone at least according to ifconfig. Also I've upgraded kernel from 2.6.38-r6 (gentoo) to 2.6.39-r3 (gentoo) - nothing changed. At all. haproxy version is 1.4.8. Altering somaxconn also did change anything. Only changing affinity of irq/haproxy is affecting system, maximum rate changes from 25k to 49k. And that's it... I'm including sysctl -a output, but I think all this happens because of some troubles with bnx2 driver - I just don't see any explanation why 70-80Mbs are saturating haproxy and irq handling (lost packets!). I have an option to try 'High Performance 1000PT Intel Network Card' could it be any better or I should try find solution for current configuration? My final task is to handle DDoS attacks with flexible and robust filter available. Haproxy is already helping me to stay alive under ~8-10k DDoS bots (I'm using two servers and DNS RR in production), but attackers are not sleeping and I'm expecting attacks to continue with more bots. I bet they will stop at 20-25k bots. Such botnet will generate approx. 500k session rate. and ~1Gbps bandwidth so I was dreaming to handle it on this one server with two NIC's bonded giving me 2Gbps for traffic:) > > Typical output of one of two ab2 running is: > > > > Server Software: > > Server Hostname: nohost > > Server Port: 80 > > > > Document Path: / > > Document Length: 0 bytes > > > > Concurrency Level: 1000 > > Time taken for tests: 470.484 seconds > > Complete requests: 10000000 > > Failed requests: 0 > > Write errors: 0 > > Total transferred: 0 bytes > > HTML transferred: 0 bytes > > Requests per second: 21254.72 [#/sec] (mean) > > Time per request: 47.048 [ms] (mean) > > Time per request: 0.047 [ms] (mean, across all concurrent requests) > > Transfer rate: 0.00 [Kbytes/sec] received > > > > Connection Times (ms) > > min mean[+/-sd] median max > > Connect: 0 34 275.9 11 21086 > > This one means there is packet loss on SYN packets. Some requests > take up to 4 SYN to pass (0+3+6+9 seconds). Clearly something is > wrong, either on the network or more likely net.core.somaxconn. > You have to restart haproxy after you change this default setting. > > Does "dmesg" say anything on either the clients or the proxy machine ? > > > Processing: 0 13 17.8 11 784 > > Waiting: 0 0 0.0 0 0 > > Total: 2 47 276.9 22 21305 > > > > Percentage of the requests served within a certain time (ms) > > 50% 22 > > 66% 26 > > 75% 28 > > 80% 30 > > 90% 37 > > 95% 41 > > 98% 47 > > 99% 266 > > 100% 21305 (longest request) > > > > Typical output of vmstat is: > > dex9 ipv4 # vmstat 1 > > procs -----------memory---------- ---swap-- -----io---- -system-- > > ----cpu---- > > r b swpd free buff cache si so bi bo in cs us sy id > > wa > > 1 0 0 131771328 46260 64016 0 0 2 1 865 503 1 6 > > 94 0 > > 1 0 0 131770688 46260 64024 0 0 0 0 40496 6323 1 9 > > 90 0 > > OK, so 1% user, 9% system, 90% idle, 0% wait at 40k int/s. Since this is > scaled to 100% for all cores, it means that we're saturating a core in the > system (which is expected with short connections). > > I don't remember if I asked you what version of haproxy and what kernel you > were using. Possibly that some TCP options can improve things a bit. > > > Also, I've checked version of NIC's firmware: > > dex9 ipv4 # ethtool -i eth0 > > driver: bnx2 > > version: 2.0.21 > > firmware-version: 6.2.12 bc 5.2.3 > > bus-info: 0000:01:00.0 > > OK, let's hope it's fine. I remember having seen apparently good results > with version 4.4 from what I recall, so this one should be OK. > > > Moreover, I've tried launching two ab2 localy: > > dex9 ipv4 # ab2 -c 1000 -H 'Accept-Encoding: None' -n 10000000 > > http://localweb/ > > This is ApacheBench, Version 2.3 <$Revision: 655654 $> > > Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ > > Licensed to The Apache Software Foundation, http://www.apache.org/ > > > > Benchmarking inbet.cc (be patient) > > Completed 1000000 requests > > Completed 2000000 requests > > ^C > > > > Server Software: > > Server Hostname: localweb > > Server Port: 80 > > > > Document Path: / > > Document Length: 0 bytes > > > > Concurrency Level: 1000 > > Time taken for tests: 104.583 seconds > > Complete requests: 2141673 > > Failed requests: 0 > > Write errors: 0 > > Total transferred: 0 bytes > > HTML transferred: 0 bytes > > Requests per second: 20478.13 [#/sec] (mean) > > Time per request: 48.833 [ms] (mean) > > Time per request: 0.049 [ms] (mean, across all concurrent requests) > > Transfer rate: 0.00 [Kbytes/sec] received > > > > Connection Times (ms) > > min mean[+/-sd] median max > > Connect: 0 38 352.7 6 21073 > > Processing: 0 10 49.7 7 14919 > > Waiting: 0 0 0.0 0 0 > > Total: 1 48 365.8 13 21078 > > > > Percentage of the requests served within a certain time (ms) > > 50% 13 > > 66% 19 > > 75% 26 > > 80% 35 > > 90% 36 > > 95% 37 > > 98% 39 > > 99% 67 > > 100% 21078 (longest request) > > > > Two such ab2 processes are running both at 100% and saturating haproxy to > > 100%. 'Cur' session rate is also around 40-44k/s. > > Fine, so those are the exact same numbers, with the same issue with packet > losses. > > > Should I get rid of dell r410 and replace it with Core i5?:)) Being serious, > > is there any other tips or tricks I can try? To see those amazing 100k/s > > session rate? > > Two things to test first as indicated above : > 1) retest with less concurrency from ab to see if things improve > 2) increase /proc/sys/net/core/somaxconn to 10000 or so > > Next, if things don't get any better, please post the output of sysctl -a. > > Hmmm please also note that when reaching 300k connections/s on the core i5, > it was done with 10Gb NICs which have an extremely low latency and nice TCP > stateless optimizations. I'm used to see much better results with them than > with gig NICs even at sub-gig rate. But anyway, more than 100k is expected > from such a machine. > > For instance, I'm attaching a capture of a test I caught one year ago on my > PC (Core 2 duo 2.66 GHz at that time), and which exhibits 212ksess/s. I > think it was a bench of TCP connections, not HTTP sessions, but still even > if we double the number of TCP packets exchanged over the wire, we should > still see more than 100k on this machine. > > Regards, > Willy >
kernel.sched_child_runs_first = 0 kernel.sched_rt_period_us = 1000000 kernel.sched_rt_runtime_us = 950000 kernel.panic = 0 kernel.core_uses_pid = 0 kernel.core_pattern = core kernel.core_pipe_limit = 0 kernel.tainted = 0 kernel.real-root-dev = 0 kernel.print-fatal-signals = 0 kernel.ctrl-alt-del = 0 kernel.modprobe = /sbin/modprobe kernel.modules_disabled = 0 kernel.hotplug = kernel.acct = 4 2 30 kernel.sysrq = 1 kernel.cad_pid = 1 kernel.threads-max = 2063940 kernel.random.poolsize = 4096 kernel.random.entropy_avail = 141 kernel.random.read_wakeup_threshold = 64 kernel.random.write_wakeup_threshold = 128 kernel.random.boot_id = 0cc1e281-5dfa-46fe-a2d1-1201743821fc kernel.random.uuid = 28284437-566f-442c-87db-46e253f428ba kernel.overflowuid = 65534 kernel.overflowgid = 65534 kernel.pid_max = 32768 kernel.panic_on_oops = 0 kernel.printk = 1 4 1 7 kernel.printk_ratelimit = 5 kernel.printk_ratelimit_burst = 10 kernel.printk_delay = 0 kernel.dmesg_restrict = 0 kernel.kptr_restrict = 0 kernel.ngroups_max = 65536 kernel.unknown_nmi_panic = 0 kernel.panic_on_unrecovered_nmi = 0 kernel.panic_on_io_nmi = 0 kernel.bootloader_type = 113 kernel.bootloader_version = 1 kernel.kstack_depth_to_print = 12 kernel.io_delay_type = 0 kernel.randomize_va_space = 1 kernel.acpi_video_flags = 0 kernel.compat-log = 1 kernel.max_lock_depth = 1024 kernel.poweroff_cmd = /sbin/poweroff kernel.keys.maxkeys = 200 kernel.keys.maxbytes = 20000 kernel.keys.root_maxkeys = 200 kernel.keys.root_maxbytes = 20000 kernel.keys.gc_delay = 300 kernel.perf_event_paranoid = 1 kernel.perf_event_mlock_kb = 516 kernel.perf_event_max_sample_rate = 100000 kernel.blk_iopoll = 1 kernel.vsyscall64 = 1 kernel.ostype = Linux kernel.osrelease = 2.6.39-gentoo-r3 kernel.version = #1 SMP Tue Jul 19 01:03:31 Local time zone must be set--see zic kernel.hostname = dex9 kernel.domainname = (none) kernel.shmmax = 33554432 kernel.shmall = 2097152 kernel.shmmni = 4096 kernel.msgmax = 8192 kernel.msgmni = 32768 kernel.msgmnb = 16384 kernel.sem = 250 32000 32 128 kernel.auto_msgmni = 1 kernel.pty.max = 4096 kernel.pty.nr = 1 vm.overcommit_memory = 0 vm.panic_on_oom = 0 vm.oom_kill_allocating_task = 0 vm.oom_dump_tasks = 1 vm.overcommit_ratio = 50 vm.page-cluster = 3 vm.dirty_background_ratio = 10 vm.dirty_background_bytes = 0 vm.dirty_ratio = 20 vm.dirty_bytes = 0 vm.dirty_writeback_centisecs = 500 vm.dirty_expire_centisecs = 3000 vm.nr_pdflush_threads = 0 vm.swappiness = 60 vm.lowmem_reserve_ratio = 256 256 32 vm.drop_caches = 0 vm.min_free_kbytes = 46018 vm.percpu_pagelist_fraction = 0 vm.max_map_count = 65530 vm.laptop_mode = 0 vm.block_dump = 0 vm.vfs_cache_pressure = 100 vm.legacy_va_layout = 0 vm.stat_interval = 1 vm.mmap_min_addr = 4096 vm.scan_unevictable_pages = 0 fs.inode-nr = 9479 303 fs.inode-state = 9479 303 0 0 0 0 0 fs.file-nr = 192 0 13209180 fs.file-max = 13209180 fs.nr_open = 1048576 fs.dentry-state = 10050 8346 45 0 0 0 fs.overflowuid = 65534 fs.overflowgid = 65534 fs.leases-enable = 1 fs.dir-notify-enable = 1 fs.lease-break-time = 45 fs.aio-nr = 0 fs.aio-max-nr = 65536 fs.inotify.max_user_instances = 128 fs.inotify.max_user_watches = 8192 fs.inotify.max_queued_events = 16384 fs.epoll.max_user_watches = 27052482 fs.suid_dumpable = 0 fs.pipe-max-size = 1048576 fs.quota.lookups = 0 fs.quota.drops = 0 fs.quota.reads = 0 fs.quota.writes = 0 fs.quota.cache_hits = 0 fs.quota.allocated_dquots = 0 fs.quota.free_dquots = 0 fs.quota.syncs = 0 fs.quota.warnings = 1 fs.xfs.irix_sgid_inherit = 0 fs.xfs.irix_symlink_mode = 0 fs.xfs.panic_mask = 0 fs.xfs.error_level = 3 fs.xfs.xfssyncd_centisecs = 3000 fs.xfs.inherit_sync = 1 fs.xfs.inherit_nodump = 1 fs.xfs.inherit_noatime = 1 fs.xfs.xfsbufd_centisecs = 100 fs.xfs.age_buffer_centisecs = 1500 fs.xfs.inherit_nosymlinks = 0 fs.xfs.rotorstep = 1 fs.xfs.inherit_nodefrag = 1 fs.xfs.filestream_centisecs = 3000 fs.xfs.stats_clear = 0 fs.ocfs2.nm.hb_ctl_path = /sbin/ocfs2_hb_ctl fs.mqueue.queues_max = 256 fs.mqueue.msg_max = 10 fs.mqueue.msgsize_max = 8192 fs.nfs.nlm_grace_period = 0 fs.nfs.nlm_timeout = 10 fs.nfs.nlm_udpport = 0 fs.nfs.nlm_tcpport = 0 fs.nfs.nsm_use_hostnames = 0 fs.nfs.nsm_local_state = 0 fs.nfs.nfs_mountpoint_timeout = 500 fs.nfs.nfs_congestion_kb = 262144 debug.exception-trace = 1 dev.scsi.logging_level = 0 dev.raid.speed_limit_min = 1000 dev.raid.speed_limit_max = 200000 dev.mac_hid.mouse_button_emulation = 0 dev.mac_hid.mouse_button2_keycode = 97 dev.mac_hid.mouse_button3_keycode = 100 dev.cdrom.info = CD-ROM information, Id: cdrom.c 3.20 2003/12/17 dev.cdrom.info = dev.cdrom.info = drive name: sr0 dev.cdrom.info = drive speed: 0 dev.cdrom.info = drive # of slots: 1 dev.cdrom.info = Can close tray: 0 dev.cdrom.info = Can open tray: 0 dev.cdrom.info = Can lock tray: 1 dev.cdrom.info = Can change speed: 1 dev.cdrom.info = Can select disk: 0 dev.cdrom.info = Can read multisession: 1 dev.cdrom.info = Can read MCN: 1 dev.cdrom.info = Reports media changed: 1 dev.cdrom.info = Can play audio: 1 dev.cdrom.info = Can write CD-R: 0 dev.cdrom.info = Can write CD-RW: 0 dev.cdrom.info = Can read DVD: 1 dev.cdrom.info = Can write DVD-R: 0 dev.cdrom.info = Can write DVD-RAM: 0 dev.cdrom.info = Can read MRW: 1 dev.cdrom.info = Can write MRW: 1 dev.cdrom.info = Can write RAM: 1 dev.cdrom.info = dev.cdrom.info = dev.cdrom.autoclose = 1 dev.cdrom.autoeject = 0 dev.cdrom.debug = 0 dev.cdrom.lock = 1 dev.cdrom.check_media = 0 dev.parport.default.timeslice = 200 dev.parport.default.spintime = 500 fbcondecor = /sbin/fbcondecor_helper net.netfilter.nf_log.0 = NONE net.netfilter.nf_log.1 = NONE net.netfilter.nf_log.2 = NONE net.netfilter.nf_log.3 = NONE net.netfilter.nf_log.4 = NONE net.netfilter.nf_log.5 = NONE net.netfilter.nf_log.6 = NONE net.netfilter.nf_log.7 = NONE net.netfilter.nf_log.8 = NONE net.netfilter.nf_log.9 = NONE net.netfilter.nf_log.10 = NONE net.netfilter.nf_log.11 = NONE net.netfilter.nf_log.12 = NONE net.core.somaxconn = 30000 net.core.xfrm_aevent_etime = 10 net.core.xfrm_aevent_rseqth = 2 net.core.xfrm_larval_drop = 1 net.core.xfrm_acq_expires = 30 net.core.wmem_max = 131071 net.core.rmem_max = 131071 net.core.wmem_default = 126976 net.core.rmem_default = 126976 net.core.dev_weight = 64 net.core.netdev_max_backlog = 1000 net.core.netdev_tstamp_prequeue = 1 net.core.message_cost = 5 net.core.message_burst = 10 net.core.optmem_max = 20480 net.core.rps_sock_flow_entries = 0 net.core.netdev_budget = 300 net.core.warnings = 1 net.ipv4.route.gc_thresh = 524288 net.ipv4.route.max_size = 8388608 net.ipv4.route.gc_min_interval = 0 net.ipv4.route.gc_min_interval_ms = 500 net.ipv4.route.gc_timeout = 300 net.ipv4.route.gc_interval = 60 net.ipv4.route.redirect_load = 2 net.ipv4.route.redirect_number = 9 net.ipv4.route.redirect_silence = 2048 net.ipv4.route.error_cost = 100 net.ipv4.route.error_burst = 500 net.ipv4.route.gc_elasticity = 8 net.ipv4.route.mtu_expires = 600 net.ipv4.route.min_pmtu = 552 net.ipv4.route.min_adv_mss = 256 net.ipv4.neigh.default.mcast_solicit = 3 net.ipv4.neigh.default.ucast_solicit = 3 net.ipv4.neigh.default.app_solicit = 0 net.ipv4.neigh.default.retrans_time = 100 net.ipv4.neigh.default.base_reachable_time = 30 net.ipv4.neigh.default.delay_first_probe_time = 5 net.ipv4.neigh.default.gc_stale_time = 60 net.ipv4.neigh.default.unres_qlen = 3 net.ipv4.neigh.default.proxy_qlen = 64 net.ipv4.neigh.default.anycast_delay = 100 net.ipv4.neigh.default.proxy_delay = 80 net.ipv4.neigh.default.locktime = 100 net.ipv4.neigh.default.retrans_time_ms = 1000 net.ipv4.neigh.default.base_reachable_time_ms = 30000 net.ipv4.neigh.default.gc_interval = 30 net.ipv4.neigh.default.gc_thresh1 = 128 net.ipv4.neigh.default.gc_thresh2 = 512 net.ipv4.neigh.default.gc_thresh3 = 1024 net.ipv4.neigh.lo.mcast_solicit = 3 net.ipv4.neigh.lo.ucast_solicit = 3 net.ipv4.neigh.lo.app_solicit = 0 net.ipv4.neigh.lo.retrans_time = 100 net.ipv4.neigh.lo.base_reachable_time = 30 net.ipv4.neigh.lo.delay_first_probe_time = 5 net.ipv4.neigh.lo.gc_stale_time = 60 net.ipv4.neigh.lo.unres_qlen = 3 net.ipv4.neigh.lo.proxy_qlen = 64 net.ipv4.neigh.lo.anycast_delay = 100 net.ipv4.neigh.lo.proxy_delay = 80 net.ipv4.neigh.lo.locktime = 100 net.ipv4.neigh.lo.retrans_time_ms = 1000 net.ipv4.neigh.lo.base_reachable_time_ms = 30000 net.ipv4.neigh.eth0.mcast_solicit = 3 net.ipv4.neigh.eth0.ucast_solicit = 3 net.ipv4.neigh.eth0.app_solicit = 0 net.ipv4.neigh.eth0.retrans_time = 100 net.ipv4.neigh.eth0.base_reachable_time = 30 net.ipv4.neigh.eth0.delay_first_probe_time = 5 net.ipv4.neigh.eth0.gc_stale_time = 60 net.ipv4.neigh.eth0.unres_qlen = 3 net.ipv4.neigh.eth0.proxy_qlen = 64 net.ipv4.neigh.eth0.anycast_delay = 100 net.ipv4.neigh.eth0.proxy_delay = 80 net.ipv4.neigh.eth0.locktime = 100 net.ipv4.neigh.eth0.retrans_time_ms = 1000 net.ipv4.neigh.eth0.base_reachable_time_ms = 30000 net.ipv4.neigh.eth1.mcast_solicit = 3 net.ipv4.neigh.eth1.ucast_solicit = 3 net.ipv4.neigh.eth1.app_solicit = 0 net.ipv4.neigh.eth1.retrans_time = 100 net.ipv4.neigh.eth1.base_reachable_time = 30 net.ipv4.neigh.eth1.delay_first_probe_time = 5 net.ipv4.neigh.eth1.gc_stale_time = 60 net.ipv4.neigh.eth1.unres_qlen = 3 net.ipv4.neigh.eth1.proxy_qlen = 64 net.ipv4.neigh.eth1.anycast_delay = 100 net.ipv4.neigh.eth1.proxy_delay = 80 net.ipv4.neigh.eth1.locktime = 100 net.ipv4.neigh.eth1.retrans_time_ms = 1000 net.ipv4.neigh.eth1.base_reachable_time_ms = 30000 net.ipv4.neigh.bond0.mcast_solicit = 3 net.ipv4.neigh.bond0.ucast_solicit = 3 net.ipv4.neigh.bond0.app_solicit = 0 net.ipv4.neigh.bond0.retrans_time = 100 net.ipv4.neigh.bond0.base_reachable_time = 30 net.ipv4.neigh.bond0.delay_first_probe_time = 5 net.ipv4.neigh.bond0.gc_stale_time = 60 net.ipv4.neigh.bond0.unres_qlen = 3 net.ipv4.neigh.bond0.proxy_qlen = 64 net.ipv4.neigh.bond0.anycast_delay = 100 net.ipv4.neigh.bond0.proxy_delay = 80 net.ipv4.neigh.bond0.locktime = 100 net.ipv4.neigh.bond0.retrans_time_ms = 1000 net.ipv4.neigh.bond0.base_reachable_time_ms = 30000 net.ipv4.tcp_timestamps = 1 net.ipv4.tcp_window_scaling = 1 net.ipv4.tcp_sack = 1 net.ipv4.tcp_retrans_collapse = 1 net.ipv4.ip_default_ttl = 64 net.ipv4.ip_no_pmtu_disc = 0 net.ipv4.ip_nonlocal_bind = 0 net.ipv4.tcp_syn_retries = 5 net.ipv4.tcp_synack_retries = 5 net.ipv4.tcp_max_orphans = 131072 net.ipv4.tcp_max_tw_buckets = 1048576 net.ipv4.ip_dynaddr = 0 net.ipv4.tcp_keepalive_time = 7200 net.ipv4.tcp_keepalive_probes = 9 net.ipv4.tcp_keepalive_intvl = 75 net.ipv4.tcp_retries1 = 3 net.ipv4.tcp_retries2 = 15 net.ipv4.tcp_fin_timeout = 60 net.ipv4.tcp_syncookies = 1 net.ipv4.tcp_tw_recycle = 0 net.ipv4.tcp_abort_on_overflow = 0 net.ipv4.tcp_stdurg = 0 net.ipv4.tcp_rfc1337 = 0 net.ipv4.tcp_max_syn_backlog = 1024 net.ipv4.ip_local_port_range = 32768 61000 net.ipv4.ip_local_reserved_ports = net.ipv4.igmp_max_memberships = 20 net.ipv4.igmp_max_msf = 10 net.ipv4.inet_peer_threshold = 65664 net.ipv4.inet_peer_minttl = 120 net.ipv4.inet_peer_maxttl = 600 net.ipv4.inet_peer_gc_mintime = 10 net.ipv4.inet_peer_gc_maxtime = 120 net.ipv4.tcp_orphan_retries = 0 net.ipv4.tcp_fack = 1 net.ipv4.tcp_reordering = 3 net.ipv4.tcp_ecn = 2 net.ipv4.tcp_dsack = 1 net.ipv4.tcp_mem = 12383616 16511488 24767232 net.ipv4.tcp_wmem = 4096 16384 4194304 net.ipv4.tcp_rmem = 4096 87380 4194304 net.ipv4.tcp_app_win = 31 net.ipv4.tcp_adv_win_scale = 2 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_frto = 2 net.ipv4.tcp_frto_response = 0 net.ipv4.tcp_low_latency = 0 net.ipv4.tcp_no_metrics_save = 0 net.ipv4.tcp_moderate_rcvbuf = 1 net.ipv4.tcp_tso_win_divisor = 3 net.ipv4.tcp_congestion_control = cubic net.ipv4.tcp_abc = 0 net.ipv4.tcp_mtu_probing = 0 net.ipv4.tcp_base_mss = 512 net.ipv4.tcp_workaround_signed_windows = 0 net.ipv4.tcp_dma_copybreak = 4096 net.ipv4.tcp_slow_start_after_idle = 1 net.ipv4.tcp_available_congestion_control = cubic reno net.ipv4.tcp_allowed_congestion_control = cubic reno net.ipv4.tcp_max_ssthresh = 0 net.ipv4.tcp_cookie_size = 0 net.ipv4.tcp_thin_linear_timeouts = 0 net.ipv4.tcp_thin_dupack = 0 net.ipv4.udp_mem = 12383616 16511488 24767232 net.ipv4.udp_rmem_min = 4096 net.ipv4.udp_wmem_min = 4096 net.ipv4.conf.all.forwarding = 0 net.ipv4.conf.all.mc_forwarding = 0 net.ipv4.conf.all.accept_redirects = 1 net.ipv4.conf.all.secure_redirects = 1 net.ipv4.conf.all.shared_media = 1 net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.all.send_redirects = 1 net.ipv4.conf.all.accept_source_route = 0 net.ipv4.conf.all.accept_local = 0 net.ipv4.conf.all.src_valid_mark = 0 net.ipv4.conf.all.proxy_arp = 0 net.ipv4.conf.all.medium_id = 0 net.ipv4.conf.all.bootp_relay = 0 net.ipv4.conf.all.log_martians = 0 net.ipv4.conf.all.tag = 0 net.ipv4.conf.all.arp_filter = 0 net.ipv4.conf.all.arp_announce = 0 net.ipv4.conf.all.arp_ignore = 0 net.ipv4.conf.all.arp_accept = 0 net.ipv4.conf.all.arp_notify = 0 net.ipv4.conf.all.proxy_arp_pvlan = 0 net.ipv4.conf.all.disable_xfrm = 0 net.ipv4.conf.all.disable_policy = 0 net.ipv4.conf.all.force_igmp_version = 0 net.ipv4.conf.all.promote_secondaries = 0 net.ipv4.conf.default.forwarding = 0 net.ipv4.conf.default.mc_forwarding = 0 net.ipv4.conf.default.accept_redirects = 1 net.ipv4.conf.default.secure_redirects = 1 net.ipv4.conf.default.shared_media = 1 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.default.send_redirects = 1 net.ipv4.conf.default.accept_source_route = 1 net.ipv4.conf.default.accept_local = 0 net.ipv4.conf.default.src_valid_mark = 0 net.ipv4.conf.default.proxy_arp = 0 net.ipv4.conf.default.medium_id = 0 net.ipv4.conf.default.bootp_relay = 0 net.ipv4.conf.default.log_martians = 0 net.ipv4.conf.default.tag = 0 net.ipv4.conf.default.arp_filter = 0 net.ipv4.conf.default.arp_announce = 0 net.ipv4.conf.default.arp_ignore = 0 net.ipv4.conf.default.arp_accept = 0 net.ipv4.conf.default.arp_notify = 0 net.ipv4.conf.default.proxy_arp_pvlan = 0 net.ipv4.conf.default.disable_xfrm = 0 net.ipv4.conf.default.disable_policy = 0 net.ipv4.conf.default.force_igmp_version = 0 net.ipv4.conf.default.promote_secondaries = 0 net.ipv4.conf.lo.forwarding = 0 net.ipv4.conf.lo.mc_forwarding = 0 net.ipv4.conf.lo.accept_redirects = 1 net.ipv4.conf.lo.secure_redirects = 1 net.ipv4.conf.lo.shared_media = 1 net.ipv4.conf.lo.rp_filter = 1 net.ipv4.conf.lo.send_redirects = 1 net.ipv4.conf.lo.accept_source_route = 1 net.ipv4.conf.lo.accept_local = 0 net.ipv4.conf.lo.src_valid_mark = 0 net.ipv4.conf.lo.proxy_arp = 0 net.ipv4.conf.lo.medium_id = 0 net.ipv4.conf.lo.bootp_relay = 0 net.ipv4.conf.lo.log_martians = 0 net.ipv4.conf.lo.tag = 0 net.ipv4.conf.lo.arp_filter = 0 net.ipv4.conf.lo.arp_announce = 0 net.ipv4.conf.lo.arp_ignore = 0 net.ipv4.conf.lo.arp_accept = 0 net.ipv4.conf.lo.arp_notify = 0 net.ipv4.conf.lo.proxy_arp_pvlan = 0 net.ipv4.conf.lo.disable_xfrm = 1 net.ipv4.conf.lo.disable_policy = 1 net.ipv4.conf.lo.force_igmp_version = 0 net.ipv4.conf.lo.promote_secondaries = 0 net.ipv4.conf.eth0.forwarding = 0 net.ipv4.conf.eth0.mc_forwarding = 0 net.ipv4.conf.eth0.accept_redirects = 1 net.ipv4.conf.eth0.secure_redirects = 1 net.ipv4.conf.eth0.shared_media = 1 net.ipv4.conf.eth0.rp_filter = 1 net.ipv4.conf.eth0.send_redirects = 1 net.ipv4.conf.eth0.accept_source_route = 1 net.ipv4.conf.eth0.accept_local = 0 net.ipv4.conf.eth0.src_valid_mark = 0 net.ipv4.conf.eth0.proxy_arp = 0 net.ipv4.conf.eth0.medium_id = 0 net.ipv4.conf.eth0.bootp_relay = 0 net.ipv4.conf.eth0.log_martians = 0 net.ipv4.conf.eth0.tag = 0 net.ipv4.conf.eth0.arp_filter = 0 net.ipv4.conf.eth0.arp_announce = 0 net.ipv4.conf.eth0.arp_ignore = 0 net.ipv4.conf.eth0.arp_accept = 0 net.ipv4.conf.eth0.arp_notify = 0 net.ipv4.conf.eth0.proxy_arp_pvlan = 0 net.ipv4.conf.eth0.disable_xfrm = 0 net.ipv4.conf.eth0.disable_policy = 0 net.ipv4.conf.eth0.force_igmp_version = 0 net.ipv4.conf.eth0.promote_secondaries = 0 net.ipv4.conf.eth1.forwarding = 0 net.ipv4.conf.eth1.mc_forwarding = 0 net.ipv4.conf.eth1.accept_redirects = 1 net.ipv4.conf.eth1.secure_redirects = 1 net.ipv4.conf.eth1.shared_media = 1 net.ipv4.conf.eth1.rp_filter = 1 net.ipv4.conf.eth1.send_redirects = 1 net.ipv4.conf.eth1.accept_source_route = 1 net.ipv4.conf.eth1.accept_local = 0 net.ipv4.conf.eth1.src_valid_mark = 0 net.ipv4.conf.eth1.proxy_arp = 0 net.ipv4.conf.eth1.medium_id = 0 net.ipv4.conf.eth1.bootp_relay = 0 net.ipv4.conf.eth1.log_martians = 0 net.ipv4.conf.eth1.tag = 0 net.ipv4.conf.eth1.arp_filter = 0 net.ipv4.conf.eth1.arp_announce = 0 net.ipv4.conf.eth1.arp_ignore = 0 net.ipv4.conf.eth1.arp_accept = 0 net.ipv4.conf.eth1.arp_notify = 0 net.ipv4.conf.eth1.proxy_arp_pvlan = 0 net.ipv4.conf.eth1.disable_xfrm = 0 net.ipv4.conf.eth1.disable_policy = 0 net.ipv4.conf.eth1.force_igmp_version = 0 net.ipv4.conf.eth1.promote_secondaries = 0 net.ipv4.conf.bond0.forwarding = 0 net.ipv4.conf.bond0.mc_forwarding = 0 net.ipv4.conf.bond0.accept_redirects = 1 net.ipv4.conf.bond0.secure_redirects = 1 net.ipv4.conf.bond0.shared_media = 1 net.ipv4.conf.bond0.rp_filter = 1 net.ipv4.conf.bond0.send_redirects = 1 net.ipv4.conf.bond0.accept_source_route = 1 net.ipv4.conf.bond0.accept_local = 0 net.ipv4.conf.bond0.src_valid_mark = 0 net.ipv4.conf.bond0.proxy_arp = 0 net.ipv4.conf.bond0.medium_id = 0 net.ipv4.conf.bond0.bootp_relay = 0 net.ipv4.conf.bond0.log_martians = 0 net.ipv4.conf.bond0.tag = 0 net.ipv4.conf.bond0.arp_filter = 0 net.ipv4.conf.bond0.arp_announce = 0 net.ipv4.conf.bond0.arp_ignore = 0 net.ipv4.conf.bond0.arp_accept = 0 net.ipv4.conf.bond0.arp_notify = 0 net.ipv4.conf.bond0.proxy_arp_pvlan = 0 net.ipv4.conf.bond0.disable_xfrm = 0 net.ipv4.conf.bond0.disable_policy = 0 net.ipv4.conf.bond0.force_igmp_version = 0 net.ipv4.conf.bond0.promote_secondaries = 0 net.ipv4.ip_forward = 0 net.ipv4.xfrm4_gc_thresh = 4194304 net.ipv4.ipfrag_high_thresh = 262144 net.ipv4.ipfrag_low_thresh = 196608 net.ipv4.ipfrag_time = 30 net.ipv4.icmp_echo_ignore_all = 0 net.ipv4.icmp_echo_ignore_broadcasts = 1 net.ipv4.icmp_ignore_bogus_error_responses = 1 net.ipv4.icmp_errors_use_inbound_ifaddr = 0 net.ipv4.icmp_ratelimit = 1000 net.ipv4.icmp_ratemask = 6168 net.ipv4.rt_cache_rebuild_count = 4 net.ipv4.ipfrag_secret_interval = 600 net.ipv4.ipfrag_max_dist = 64 net.token-ring.rif_timeout = 60000 net.ipv6.neigh.default.mcast_solicit = 3 net.ipv6.neigh.default.ucast_solicit = 3 net.ipv6.neigh.default.app_solicit = 0 net.ipv6.neigh.default.retrans_time = 100 net.ipv6.neigh.default.base_reachable_time = 30 net.ipv6.neigh.default.delay_first_probe_time = 5 net.ipv6.neigh.default.gc_stale_time = 60 net.ipv6.neigh.default.unres_qlen = 3 net.ipv6.neigh.default.proxy_qlen = 64 net.ipv6.neigh.default.anycast_delay = 100 net.ipv6.neigh.default.proxy_delay = 80 net.ipv6.neigh.default.locktime = 0 net.ipv6.neigh.default.retrans_time_ms = 1000 net.ipv6.neigh.default.base_reachable_time_ms = 30000 net.ipv6.neigh.default.gc_interval = 30 net.ipv6.neigh.default.gc_thresh1 = 128 net.ipv6.neigh.default.gc_thresh2 = 512 net.ipv6.neigh.default.gc_thresh3 = 1024 net.ipv6.neigh.lo.mcast_solicit = 3 net.ipv6.neigh.lo.ucast_solicit = 3 net.ipv6.neigh.lo.app_solicit = 0 net.ipv6.neigh.lo.retrans_time = 100 net.ipv6.neigh.lo.base_reachable_time = 30 net.ipv6.neigh.lo.delay_first_probe_time = 5 net.ipv6.neigh.lo.gc_stale_time = 60 net.ipv6.neigh.lo.unres_qlen = 3 net.ipv6.neigh.lo.proxy_qlen = 64 net.ipv6.neigh.lo.anycast_delay = 100 net.ipv6.neigh.lo.proxy_delay = 80 net.ipv6.neigh.lo.locktime = 0 net.ipv6.neigh.lo.retrans_time_ms = 1000 net.ipv6.neigh.lo.base_reachable_time_ms = 30000 net.ipv6.neigh.eth0.mcast_solicit = 3 net.ipv6.neigh.eth0.ucast_solicit = 3 net.ipv6.neigh.eth0.app_solicit = 0 net.ipv6.neigh.eth0.retrans_time = 100 net.ipv6.neigh.eth0.base_reachable_time = 30 net.ipv6.neigh.eth0.delay_first_probe_time = 5 net.ipv6.neigh.eth0.gc_stale_time = 60 net.ipv6.neigh.eth0.unres_qlen = 3 net.ipv6.neigh.eth0.proxy_qlen = 64 net.ipv6.neigh.eth0.anycast_delay = 100 net.ipv6.neigh.eth0.proxy_delay = 80 net.ipv6.neigh.eth0.locktime = 0 net.ipv6.neigh.eth0.retrans_time_ms = 1000 net.ipv6.neigh.eth0.base_reachable_time_ms = 30000 net.ipv6.neigh.eth1.mcast_solicit = 3 net.ipv6.neigh.eth1.ucast_solicit = 3 net.ipv6.neigh.eth1.app_solicit = 0 net.ipv6.neigh.eth1.retrans_time = 100 net.ipv6.neigh.eth1.base_reachable_time = 30 net.ipv6.neigh.eth1.delay_first_probe_time = 5 net.ipv6.neigh.eth1.gc_stale_time = 60 net.ipv6.neigh.eth1.unres_qlen = 3 net.ipv6.neigh.eth1.proxy_qlen = 64 net.ipv6.neigh.eth1.anycast_delay = 100 net.ipv6.neigh.eth1.proxy_delay = 80 net.ipv6.neigh.eth1.locktime = 0 net.ipv6.neigh.eth1.retrans_time_ms = 1000 net.ipv6.neigh.eth1.base_reachable_time_ms = 30000 net.ipv6.neigh.bond0.mcast_solicit = 3 net.ipv6.neigh.bond0.ucast_solicit = 3 net.ipv6.neigh.bond0.app_solicit = 0 net.ipv6.neigh.bond0.retrans_time = 100 net.ipv6.neigh.bond0.base_reachable_time = 30 net.ipv6.neigh.bond0.delay_first_probe_time = 5 net.ipv6.neigh.bond0.gc_stale_time = 60 net.ipv6.neigh.bond0.unres_qlen = 3 net.ipv6.neigh.bond0.proxy_qlen = 64 net.ipv6.neigh.bond0.anycast_delay = 100 net.ipv6.neigh.bond0.proxy_delay = 80 net.ipv6.neigh.bond0.locktime = 0 net.ipv6.neigh.bond0.retrans_time_ms = 1000 net.ipv6.neigh.bond0.base_reachable_time_ms = 30000 net.ipv6.xfrm6_gc_thresh = 1024 net.ipv6.conf.all.forwarding = 0 net.ipv6.conf.all.hop_limit = 64 net.ipv6.conf.all.mtu = 1280 net.ipv6.conf.all.accept_ra = 1 net.ipv6.conf.all.accept_redirects = 1 net.ipv6.conf.all.autoconf = 1 net.ipv6.conf.all.dad_transmits = 1 net.ipv6.conf.all.router_solicitations = 3 net.ipv6.conf.all.router_solicitation_interval = 4 net.ipv6.conf.all.router_solicitation_delay = 1 net.ipv6.conf.all.force_mld_version = 0 net.ipv6.conf.all.use_tempaddr = 0 net.ipv6.conf.all.temp_valid_lft = 604800 net.ipv6.conf.all.temp_prefered_lft = 86400 net.ipv6.conf.all.regen_max_retry = 5 net.ipv6.conf.all.max_desync_factor = 600 net.ipv6.conf.all.max_addresses = 16 net.ipv6.conf.all.accept_ra_defrtr = 1 net.ipv6.conf.all.accept_ra_pinfo = 1 net.ipv6.conf.all.accept_ra_rtr_pref = 1 net.ipv6.conf.all.router_probe_interval = 60 net.ipv6.conf.all.proxy_ndp = 0 net.ipv6.conf.all.accept_source_route = 0 net.ipv6.conf.all.disable_ipv6 = 0 net.ipv6.conf.all.accept_dad = 1 net.ipv6.conf.all.force_tllao = 0 net.ipv6.conf.default.forwarding = 0 net.ipv6.conf.default.hop_limit = 64 net.ipv6.conf.default.mtu = 1280 net.ipv6.conf.default.accept_ra = 1 net.ipv6.conf.default.accept_redirects = 1 net.ipv6.conf.default.autoconf = 1 net.ipv6.conf.default.dad_transmits = 1 net.ipv6.conf.default.router_solicitations = 3 net.ipv6.conf.default.router_solicitation_interval = 4 net.ipv6.conf.default.router_solicitation_delay = 1 net.ipv6.conf.default.force_mld_version = 0 net.ipv6.conf.default.use_tempaddr = 0 net.ipv6.conf.default.temp_valid_lft = 604800 net.ipv6.conf.default.temp_prefered_lft = 86400 net.ipv6.conf.default.regen_max_retry = 5 net.ipv6.conf.default.max_desync_factor = 600 net.ipv6.conf.default.max_addresses = 16 net.ipv6.conf.default.accept_ra_defrtr = 1 net.ipv6.conf.default.accept_ra_pinfo = 1 net.ipv6.conf.default.accept_ra_rtr_pref = 1 net.ipv6.conf.default.router_probe_interval = 60 net.ipv6.conf.default.proxy_ndp = 0 net.ipv6.conf.default.accept_source_route = 0 net.ipv6.conf.default.disable_ipv6 = 0 net.ipv6.conf.default.accept_dad = 1 net.ipv6.conf.default.force_tllao = 0 net.ipv6.conf.lo.forwarding = 0 net.ipv6.conf.lo.hop_limit = 64 net.ipv6.conf.lo.mtu = 16436 net.ipv6.conf.lo.accept_ra = 1 net.ipv6.conf.lo.accept_redirects = 1 net.ipv6.conf.lo.autoconf = 1 net.ipv6.conf.lo.dad_transmits = 1 net.ipv6.conf.lo.router_solicitations = 3 net.ipv6.conf.lo.router_solicitation_interval = 4 net.ipv6.conf.lo.router_solicitation_delay = 1 net.ipv6.conf.lo.force_mld_version = 0 net.ipv6.conf.lo.use_tempaddr = -1 net.ipv6.conf.lo.temp_valid_lft = 604800 net.ipv6.conf.lo.temp_prefered_lft = 86400 net.ipv6.conf.lo.regen_max_retry = 5 net.ipv6.conf.lo.max_desync_factor = 600 net.ipv6.conf.lo.max_addresses = 16 net.ipv6.conf.lo.accept_ra_defrtr = 1 net.ipv6.conf.lo.accept_ra_pinfo = 1 net.ipv6.conf.lo.accept_ra_rtr_pref = 1 net.ipv6.conf.lo.router_probe_interval = 60 net.ipv6.conf.lo.proxy_ndp = 0 net.ipv6.conf.lo.accept_source_route = 0 net.ipv6.conf.lo.disable_ipv6 = 0 net.ipv6.conf.lo.accept_dad = -1 net.ipv6.conf.lo.force_tllao = 0 net.ipv6.conf.eth0.forwarding = 0 net.ipv6.conf.eth0.hop_limit = 64 net.ipv6.conf.eth0.mtu = 1500 net.ipv6.conf.eth0.accept_ra = 1 net.ipv6.conf.eth0.accept_redirects = 1 net.ipv6.conf.eth0.autoconf = 1 net.ipv6.conf.eth0.dad_transmits = 1 net.ipv6.conf.eth0.router_solicitations = 3 net.ipv6.conf.eth0.router_solicitation_interval = 4 net.ipv6.conf.eth0.router_solicitation_delay = 1 net.ipv6.conf.eth0.force_mld_version = 0 net.ipv6.conf.eth0.use_tempaddr = 0 net.ipv6.conf.eth0.temp_valid_lft = 604800 net.ipv6.conf.eth0.temp_prefered_lft = 86400 net.ipv6.conf.eth0.regen_max_retry = 5 net.ipv6.conf.eth0.max_desync_factor = 600 net.ipv6.conf.eth0.max_addresses = 16 net.ipv6.conf.eth0.accept_ra_defrtr = 1 net.ipv6.conf.eth0.accept_ra_pinfo = 1 net.ipv6.conf.eth0.accept_ra_rtr_pref = 1 net.ipv6.conf.eth0.router_probe_interval = 60 net.ipv6.conf.eth0.proxy_ndp = 0 net.ipv6.conf.eth0.accept_source_route = 0 net.ipv6.conf.eth0.disable_ipv6 = 0 net.ipv6.conf.eth0.accept_dad = 1 net.ipv6.conf.eth0.force_tllao = 0 net.ipv6.conf.eth1.forwarding = 0 net.ipv6.conf.eth1.hop_limit = 64 net.ipv6.conf.eth1.mtu = 1500 net.ipv6.conf.eth1.accept_ra = 1 net.ipv6.conf.eth1.accept_redirects = 1 net.ipv6.conf.eth1.autoconf = 1 net.ipv6.conf.eth1.dad_transmits = 1 net.ipv6.conf.eth1.router_solicitations = 3 net.ipv6.conf.eth1.router_solicitation_interval = 4 net.ipv6.conf.eth1.router_solicitation_delay = 1 net.ipv6.conf.eth1.force_mld_version = 0 net.ipv6.conf.eth1.use_tempaddr = 0 net.ipv6.conf.eth1.temp_valid_lft = 604800 net.ipv6.conf.eth1.temp_prefered_lft = 86400 net.ipv6.conf.eth1.regen_max_retry = 5 net.ipv6.conf.eth1.max_desync_factor = 600 net.ipv6.conf.eth1.max_addresses = 16 net.ipv6.conf.eth1.accept_ra_defrtr = 1 net.ipv6.conf.eth1.accept_ra_pinfo = 1 net.ipv6.conf.eth1.accept_ra_rtr_pref = 1 net.ipv6.conf.eth1.router_probe_interval = 60 net.ipv6.conf.eth1.proxy_ndp = 0 net.ipv6.conf.eth1.accept_source_route = 0 net.ipv6.conf.eth1.disable_ipv6 = 0 net.ipv6.conf.eth1.accept_dad = 1 net.ipv6.conf.eth1.force_tllao = 0 net.ipv6.conf.bond0.forwarding = 0 net.ipv6.conf.bond0.hop_limit = 64 net.ipv6.conf.bond0.mtu = 1500 net.ipv6.conf.bond0.accept_ra = 1 net.ipv6.conf.bond0.accept_redirects = 1 net.ipv6.conf.bond0.autoconf = 1 net.ipv6.conf.bond0.dad_transmits = 1 net.ipv6.conf.bond0.router_solicitations = 3 net.ipv6.conf.bond0.router_solicitation_interval = 4 net.ipv6.conf.bond0.router_solicitation_delay = 1 net.ipv6.conf.bond0.force_mld_version = 0 net.ipv6.conf.bond0.use_tempaddr = 0 net.ipv6.conf.bond0.temp_valid_lft = 604800 net.ipv6.conf.bond0.temp_prefered_lft = 86400 net.ipv6.conf.bond0.regen_max_retry = 5 net.ipv6.conf.bond0.max_desync_factor = 600 net.ipv6.conf.bond0.max_addresses = 16 net.ipv6.conf.bond0.accept_ra_defrtr = 1 net.ipv6.conf.bond0.accept_ra_pinfo = 1 net.ipv6.conf.bond0.accept_ra_rtr_pref = 1 net.ipv6.conf.bond0.router_probe_interval = 60 net.ipv6.conf.bond0.proxy_ndp = 0 net.ipv6.conf.bond0.accept_source_route = 0 net.ipv6.conf.bond0.disable_ipv6 = 0 net.ipv6.conf.bond0.accept_dad = 1 net.ipv6.conf.bond0.force_tllao = 0 net.ipv6.ip6frag_high_thresh = 262144 net.ipv6.ip6frag_low_thresh = 196608 net.ipv6.ip6frag_time = 60 net.ipv6.route.gc_thresh = 1024 net.ipv6.route.max_size = 4096 net.ipv6.route.gc_min_interval = 0 net.ipv6.route.gc_timeout = 60 net.ipv6.route.gc_interval = 30 net.ipv6.route.gc_elasticity = 9 net.ipv6.route.mtu_expires = 600 net.ipv6.route.min_adv_mss = 1220 net.ipv6.route.gc_min_interval_ms = 500 net.ipv6.icmp.ratelimit = 1000 net.ipv6.bindv6only = 0 net.ipv6.ip6frag_secret_interval = 600 net.ipv6.mld_max_msf = 64 net.unix.max_dgram_qlen = 10 abi.vsyscall32 = 2 sunrpc.rpc_debug = 0 sunrpc.nfs_debug = 0 sunrpc.nfsd_debug = 0 sunrpc.nlm_debug = 0 sunrpc.transports = tcp 1048576 sunrpc.transports = udp 32768 sunrpc.udp_slot_table_entries = 16 sunrpc.tcp_slot_table_entries = 16 sunrpc.min_resvport = 665 sunrpc.max_resvport = 1023 sunrpc.tcp_fin_timeout = 15
