On Sun, 11 Apr 2010 15:23:16 +0200 Olaf Selke <olaf.se...@blutmagie.de> wrote: >Scott Bennett schrieb: >> >> I see I missed the implication in Olaf's main complaint above, which >> is that the authorities are advertising more capacity for his node than >> his node is advertising. > >relax, I'm not complaining. We are all volunteers not being payed for >writing tor code. > >> Checking the current (i.e., valid-after >> 2010-04-11 10:00:00) consensus, I see that the authorities have decided >> to volunteer 2560 KB/s on behalf of node blutmagie, which is indeed greater >> than 2500 KB/s, though only 2.4% greater. > >yes, I noticed that without mentioning it on this list. It appears to be >a common misunderstanding in coding between 2^10 and 10^3. > >> (For some reason, my current >> directory files don't seem to contain a descriptor for blutmagie at all. >> I don't know why, but I assume that it will prove to be a temporary >> situation.) > >did my node stop publishing its descriptor again? Traffic has dropped >about 80% within the last hours. Have a look at the attached graph.
Oh, jeez. I thought that one had been fixed a while back. Sigh. Or maybe it's not just *one* bug. > >> But the total exit usage question cannot be answered at present because >> nothing reports that information at present. > >maybe I take your advice and add php code at blutmagie tns to sum up the >extra-info average rate data and print the so calculated bandwidth >instead of max observed one. You might communicate with Kasimir Gabert about that. I think he said some months ago that he was going to do that for his torstatus stuff, so what you want might already be written. > >Regarding superpages: Is it possible to figure out how much cpu time >being wasted in address translation with on-board means like vmstat? But I don't see how because a successful translation stays in the hardware and causes no interrupt. The kernel only sees something when the translation fails. FreeBSD has had steadily growing HWPMC support since 6.0, so I just looked through some of the HWPMC man pages. There are counters for instruction cache misses and data cache misses, but I didn't see any counters at all that involved TLB-related events. That doesn't mean that there aren't any, just that I didn't find any documentation of any. On a moment's thought, that does seem a trifle odd, given that there are counters for lots of strange things, including other performance hits that have far milder consequences per event (e.g., logical CPU cycle lockouts caused by conflict with the other logical CPU in a core, IIRC) than TLB misses have. If I had the Intel processor manuals that describe the various counters that the chips support, I'd have a better clue where to look in the FreeBSD documentation or maybe how to query the hardware itself (using a small utility that is part of the base system) to find out whether there are any TLB-related event counters available. You might dig around in your system to see whether LINUX shows support for any TLB-related event counters. Something else you might check on if you're considering adding scraps of code to tor to use LINUX's "huge" page support is whether "huge" pages get page-fixed (a.k.a. "wired"). FreeBSD's superpages don't. If the kernel decides it has to snatch any base page frames out of a superpage, it simply demotes a superpage to the next smaller page size supported on that processor type, then demotes one of those, etc. until it can free up individual base pages. That means it can't cause the sort of problem that tying up several hundred megabytes of page frames by fixing a large process's pages into them can create for the rest of whatever is running in the same machine because a superpage's base (i.e., underlying, smallest sized) page frames can always be freed for reuse by something else if the system really needs them. On i386 and amd64 architectures, there are only two page sizes (4 KB and 4 MB) available anyway, so there's only one step available in either the promotion direction or the demotion direction. (ia64 (i.e., Itanium) has a several more steps available, IIRC, and I think the alpha processors have about five.) In any case, your Xeon(s) ought to be able to benefit considerably from running your gargantuan tor process in 4 MB pages instead of 4 KB pages. I'm not sure about Xeons, but the regular, non-server i386 and amd64 chips by Intel only have 64 TLB entries for instruction (i.e., text) pages and 64 TLB entries for data (i.e., data and stack) pages, so that means if an instruction working set or a data working set exceeds 256 KB, then the process running with base (4 KB) pages will end up thrashing the relevant TLB(s), stalling the processor every time while the MMU walks through the page table. If you have HTT-enabled Xeons, then remember that the two logical CPUs in each core share the same MMU and L1 and L2 caches, as well as the same bus to and from main memory, which can further increase the frequency and cost of TLB misses as the two threads tramp around in different parts of your system simultaneously. >I'm certainly not going to migrate my tor node to Windows. Never ever! ;-) > 8<Q Really, I was only kidding. :-) But, AFAIK, FreeBSD and some versions of Windows are the only OS that support automatic, transparent superpages. I think Solaris for x86 supports superpages somehow, but my dim recollection of the information I ran across said that they required some programmer intervention. Nevertheless, if you have to keep your system a LINUX system, it would be worth it in your case to see about adding whatever code might be needed to use the "huge" pages facilities. I have a suspicion that it might turn out to be a fairly small number of lines of extra code to set up essentially all of tor's memory as huge pages. The trick would be to make sure that anywhere that tor requests more memory from the OS it also puts the newly allocated memory into huge pages (probably easy enough) *and* frees them again each time an entire huge page is emptied (the trickier part). Anyway, whatever you decide to try, please, please report back here any results of the experiment. Scott Bennett, Comm. ASMELG, CFIAG ********************************************************************** * Internet: bennett at cs.niu.edu * *--------------------------------------------------------------------* * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * * -- Gov. John Hancock, New York Journal, 28 January 1790 * ********************************************************************** *********************************************************************** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talk in the body. http://archives.seul.org/or/talk/