[gentoo-user] Rate limiting TCP connections...
I've a Netgear DG834G router - and I connect two machines to it using Ethernet... one Gentoo; one Windows... it works reasonably well... I hit a snag when downloading a large file from Gentoo - for example a multi-meg portage archive. At such times, the Windows PC seems to be given a rather unfair share of the bandwidth... with the Gentoo box getting 60K/s but the windows PC sometimes even failing to establish DNS lookups before they time-out. Is there a straightforward way to make my Gentoo box 'play fair' and not hog 100% of the bandwidth? Does anyone else have this problem?
Re: [gentoo-user] Rate limiting TCP connections...
Norberto Bensa wrote: Ah!! But Windows (XP) uses TC by default. It doesn't use 20% of the network bandwidth unless you tweak some registry setting and/or disable QoS in network properties. That sounds like a fine plan for me... but, erm, how does it know? Both Linux and Xp talk to my router at 100mbps - and my router talks to the outside world at 0.5mbps... so, while I'd be entirely happy to cap both machines at 80mbps, I don't see why this would have any effect on the competition for the 0.5mbps to the outside world. What's more to the point, it doesn't seem to be Linux competing with Xp, per se - but rather Linux competing with Linux - since my LAN works great - and I can communicate at will between Xp and Linux - it is only when Linux's bind competes with Linux's wget that I see a problem. This is with two processes on the same PC. Why? Is pretty obvious what's happening: your Linux box is eating all the bandwidth with the MB download because _by_default_ Linux doesn't do any TC at all. If the iptables thingy was too aggressive, try a --limit-rate (or --rate-limit; I can't never get it right) in wget. I presume this is what you mean (taken from "man iptables"): --limit rate Maximum average matching rate: specified as a number, with an optional `/second', `/minute', `/hour', or `/day' suffix; the default is 3/hour. This looks as if I can limit the rate at which my linux box talks on my LAN - but this isn't what I need to do. Interestingly, long downloads from two competing WinXp boxes don't cause a problem - but both will max-out my available download capacity... suggesting to me that fixed rate-limiting is not what is called for...
Re: [gentoo-user] Re: Rate limiting TCP connections...
Holger Hoffstaette wrote: On Sun, 17 Aug 2008 22:53:23 -0300, Norberto Bensa wrote Ah!! But Windows (XP) uses TC by default. It doesn't use 20% of the network bandwidth unless you tweak some registry setting and/or disable QoS in network properties. This is not the case. Please read: http://support.microsoft.com/default.aspx?scid=kb;EN-US;q31 The last paragraph could not be more clear on this. Thanks... I was pretty sure that reserving a proportion of my LAN bandwidth wouldn't help - though I didn't have that reference to hand. I'd have been happy to rate-limit to 80mbps if that would have helped - though I saw no reason that it should. Much more likely is that the Gentoo (Linux) TCP Window sizing is more aggressive and/or the Windows app/stack does not do the same. The OP should ensure that window sizing is turned on (might be off) and bump up the initial window size. This sounds far more plausible to me... it certainly seems to be something to do with the TCP stack on Gentoo. I've had a look at http://gentoo-wiki.com/HOWTO_TCP_Tuning - which was interesting - but didn't suggest any clear answers to me. From sysctl, I discovered that tcp_congestion_control is "cubic" - which isn't even an option documented by the wiki... I wonder, does the "C" in BIC mean "cubic" - making these one and the same? The LWN article suggests not. Is "cubic" the defaut for everyone? Is this Gentoo specific? Is there a reason I've had it chosen over BIC for me by Gentoo magic? (I've not done any TCP tuning myself - everything is default on this score... I built my own kernel - but only to enable unrelated options.) (I think) I'm using a vanilla kernel configuration with respect to networking... uname reports "2.6.23-gentoo-r3". $ syctl net.iv4 net.ipv4.tcp_window_scaling = 1 net.ipv4.tcp_sack = 1 net.ipv4.tcp_retrans_collapse = 1 net.ipv4.ip_forward = 1 net.ipv4.ip_default_ttl = 64 net.ipv4.ip_no_pmtu_disc = 0 net.ipv4.ip_nonlocal_bind = 0 net.ipv4.tcp_syn_retries = 5 net.ipv4.tcp_synack_retries = 5 net.ipv4.tcp_max_orphans = 32768 net.ipv4.tcp_max_tw_buckets = 18 net.ipv4.ipfrag_high_thresh = 262144 net.ipv4.ipfrag_low_thresh = 196608 net.ipv4.ip_dynaddr = 0 net.ipv4.ipfrag_time = 30 net.ipv4.tcp_keepalive_time = 7200 net.ipv4.tcp_keepalive_probes = 9 net.ipv4.tcp_keepalive_intvl = 75 net.ipv4.tcp_retries1 = 3 net.ipv4.tcp_retries2 = 15 net.ipv4.tcp_fin_timeout = 60 net.ipv4.tcp_tw_recycle = 0 net.ipv4.tcp_abort_on_overflow = 0 net.ipv4.tcp_stdurg = 0 net.ipv4.tcp_rfc1337 = 0 net.ipv4.tcp_max_syn_backlog = 1024 net.ipv4.ip_local_port_range = 3276861000 net.ipv4.icmp_echo_ignore_all = 0 net.ipv4.icmp_echo_ignore_broadcasts = 1 net.ipv4.icmp_ignore_bogus_error_responses = 1 net.ipv4.icmp_errors_use_inbound_ifaddr = 0 net.ipv4.route.min_delay = 2 net.ipv4.route.max_delay = 10 net.ipv4.route.gc_thresh = 32768 net.ipv4.route.max_size = 524288 net.ipv4.route.gc_min_interval = 0 net.ipv4.route.gc_min_interval_ms = 500 net.ipv4.route.gc_timeout = 300 net.ipv4.route.gc_interval = 60 net.ipv4.route.redirect_load = 5 net.ipv4.route.redirect_number = 9 net.ipv4.route.redirect_silence = 5120 net.ipv4.route.error_cost = 250 net.ipv4.route.error_burst = 1250 net.ipv4.route.gc_elasticity = 8 net.ipv4.route.mtu_expires = 600 net.ipv4.route.min_pmtu = 552 net.ipv4.route.min_adv_mss = 256 net.ipv4.route.secret_interval = 600 net.ipv4.igmp_max_memberships = 20 net.ipv4.igmp_max_msf = 10 net.ipv4.inet_peer_threshold = 65664 net.ipv4.inet_peer_minttl = 120 net.ipv4.inet_peer_maxttl = 600 net.ipv4.inet_peer_gc_mintime = 10 net.ipv4.inet_peer_gc_maxtime = 120 net.ipv4.tcp_orphan_retries = 0 net.ipv4.tcp_fack = 1 net.ipv4.tcp_reordering = 3 net.ipv4.tcp_ecn = 0 net.ipv4.tcp_dsack = 1 net.ipv4.tcp_mem = 96576128768 193152 net.ipv4.tcp_wmem = 409616384 4120576 net.ipv4.tcp_rmem = 409687380 4120576 net.ipv4.tcp_app_win = 31 net.ipv4.tcp_adv_win_scale = 2 net.ipv4.icmp_ratelimit = 250 net.ipv4.icmp_ratemask = 6168 net.ipv4.tcp_tw_reuse = 0 net.ipv4.tcp_frto = 0 net.ipv4.tcp_frto_response = 0 net.ipv4.tcp_low_latency = 0 net.ipv4.ipfrag_secret_interval = 600 net.ipv4.ipfrag_max_dist = 64 net.ipv4.tcp_no_metrics_save = 0 net.ipv4.tcp_moderate_rcvbuf = 1 net.ipv4.tcp_tso_win_divisor = 3 net.ipv4.tcp_congestion_control = cubic net.ipv4.tcp_abc = 0 net.ipv4.tcp_mtu_probing = 0 net.ipv4.tcp_base_mss = 512 net.ipv4.tcp_workaround_signed_windows = 0 net.ipv4.tcp_slow_start_after_idle = 1 net.ipv4.tcp_available_congestion_control = cubic reno net.ipv4.tcp_allowed_congestion_control = cubic reno net.ipv4.tcp_max_ssthresh = 0 net.ipv4.neigh.default.mcast_solicit = 3 net.ipv4.neigh.default.ucast_solicit = 3 net.ipv4.neigh.default.app_solicit = 0 net.ipv4.neigh.default.retrans_time = 100 net.ipv4.neigh.default.base_reachable_time = 30 net.ipv4.neigh.default.delay_first_probe_time = 5 net.ipv4.neigh.default.gc_stale_time = 60 net.ipv4.neigh.default.unres_qlen = 3 net.ipv4.neigh.default.pr
Re: [gentoo-user] Rate limiting TCP connections...
Norberto Bensa wrote: Nope. "fixed rate limiting" is not the answer. You need QoS at the router level, but if it doesn't support it, you'll need to change how your Linux box talks and listen to internet packages. That's what I said -more or less- on my first reply. I'm a believer in doing things the easiest way... and while I can see that manually specifying limits on bandwidth use from Linux on an explicit address-range basis would "work" - it is not an appealing approach. Let's make an experiment: 1. Terminate all downloads and activity on the internet. 2. Restart your bind (so it flushes its cache) 3. in XP1 download something huge (an ISO image) from one souce in the internet and wait 'til it is at full speed (does it go up to 0.5Mb??) 4. in XP2 start to ping different sources. Does XP2 lost packets? If I do my downloading from XP (using Linux as my nameserver) everything works perfectly. My downloads max-out my ADSL connection - and not only can I ping other hosts concurrently, but I can surf the web and bandwidth is shared fairly between competing applications. My router is a "Netgear Wireless ADSL Firewall Router" - it seems pretty common... and I've not found other people moaning that it has problems... For me, it only has problems when accessed from my Linux box.
Re: [gentoo-user] Rate limiting TCP connections...
Mick wrote: I think that the problem is associated with the way that the Linux box treats bind requests. Other than QoS which will try to allocate some bandwidth to bind packets, or nice which will elevate bind's processes - you may want to check your kernel's IO scheduler and set it to something that will give each process an equal bite of the cherry. Trial & error may get you there. I don't think QOS is the right answer - since I've never seen this problem before - and I've never used QOS on Linux before. I think that the scheduler might explain things far better. I half-remember something about the inclusion of the new "CFS" scheduler when I compiled the kernel - maybe the default changed when I moved to the 2.6.23-gentoo-r3 kernel from - erm - some older 2.6 version many months ago. Maybe I should upgrade to the latest kernel (I'm reluctant to do this in a hurry - since I've lost my notes on which kernel options I'd activated - and it's a 'live' box I'm using on a day-to-day basis that I'd rather not break. Doah!) A workaround to avoid WinXP name requests timing out is to manually set at your WinXP clients the Netgear's IP address as their secondary DNS server. That's a fantastic hack... I like the idea... though, obviously, if it were trivial, I'd prefer Bind on gentoo not to hang. ;) My router is a "Netgear Wireless ADSL Firewall Router" - it seems pretty common... and I've not found other people moaning that it has problems... For me, it only has problems when accessed from my Linux box. I used to run a Netgear DG834 and did not notice anything like this. After a few seconds the Gentoo and WinXP clients would share the bandwidth - irrespective of which one started downloading first. WinXP might have been slightly more hesitant to start with, but after say 30 seconds it would even out. However, this was with wired full duplex connections. Wireless is half duplex, transmit and receive happens sequentially not in parallel - when downloading on the Gentoo goes at full pelt it may take longer for inbound packets to get to bind and this could make the rather short TTL that MSWindows has to time out. That's encouraging. Always good to know that there are no 'known' bugs... I was caught out by this a few years ago with a DLink router - where I assumed it was my FreeBSD box at fault - but, actually, the router had broken firmware. PS. Have you tried this with two Linux clients (use Knoppix on one of your MSWindows boxen)? No, I could, I guess - but I'm 95% convinced now that this is an I/O scheduling issue on my Gentoo box - and is not an issue with my router. Many thanks.
Re: [gentoo-user] Rate limiting TCP connections...
Neil Bothwick wrote: Maybe I should upgrade to the latest kernel (I'm reluctant to do this in a hurry - since I've lost my notes on which kernel options I'd activated Copy the current config over and run make oldconfig. There's no need to take notes when the system keeps track for you - and it's a 'live' box I'm using on a day-to-day basis that I'd rather not break. Doah!) Installing a new kernel won't cause the old one to break, no matter how badly you misconfigure it. Ack. (and thanks) I sort-of knew that... though I'd have had to read the docs to remember "make oldconfig". With systems I need, I'm exceptionally cautious... I've never forgiven myself for the decade's acquisition of 10GB of data... that I accidentally wiped thinking I was backing it up. It's a sinking feeling like no other. I'll give the kernel a shot, but only when I've a day or two to recover if I screw-up. :-)
Re: [gentoo-user] Rate limiting TCP connections...
Neil Bothwick wrote: That reminds me of the time I carefully backup up everything from partition A to partition B, and verified it, before reformatting partition B instead of A. You've quite a way to go to get my experience. In my 10GB was about 50mb of only-copy ASCII - the air turned blue for miles around... I was not a happy bunny. ;) Something is seriously wrong with your GRUB setup if it takes a day to select the previous kernel :( Nope... it's all pretty vanilla... I expect it to take a few minutes - but I'm not attempting it unless I've got a day to fix it if I make an error... that way I won't do something daft because I'm interrupted and trying to field a dozen things at once. I'm a strong believer in Murphy's law.
[gentoo-user] (Slightly OT) A question about Wikis (maybe)
I'm considering a project, and am keen not to end up re-inventing the wheel. I'm looking to use collaborative techniques to put together a 'knowledge base'... I require: * Collaborative editing to grow the number of 'records' held. * For the 'records' (pages) to be of a standard form - so that data of a similar 'type' can be processed (summarised and filtered) based upon any attribute (field). The 'records' - however need to be flexible - in the sense that a field might be a sequence of records (recursively) in a table... The 'type' of a particular record will be known, and should have a common appearance and layout that can be edited independent of the data. It must be possible to extend the type of records after data has been collected... as the system evolves... but these fundamental changes need only be possible for an administrator. * Support for a hierarchy of users - such that only the submitter/members of senior groups can view new data until it is approved by a member of a senior group. * Support for public comment & discussion on every page - a threaded forum approach would be fine. * (Ideally - not sure how this would be used) Support to drag in data from third-party sites either using RSS or using web-services. * Email notification of changes to pages where an interest is registered. * Full version management. I've briefly looked at Twiki and Xwiki (which show some promise - but I'm not sure they're up to the job.) I'm familiar with Ruby on Rails - though I suspect that it is too low-level for my purposes. I would like to avoid focusing on the implementation details as much as possible and focus on the design of the collaborative system using the highest-level RAD approach I available today. Have others addressed a similar problem (using gentoo)?
