From your trace it appears that you have NPTL turned off. This means each thread has a pid and shows up in top. There are various disadvantages of this, but the main advantage is that if you do a stack dump and translate the pid's into hexadecimal, you can cross-match the pid's against the java tid's. Please could you find out which threads are using all the CPU? For example in the attached picture, 28142 (6dee) is the top item, using 38%.
Right now my node is using 30-50% of one core (2.2GHz atm). And that's with heavy logging, and queued requests. There have in the past been various causes for 100% CPU usage, in approximate order of popularity: - Running out of memory causing constant garbage collection. (easily checked with wrapper.java.additional=-Xloggc:freenet.loggc). - Bugs causing infinite loops. - Heavy logging (check your config settings). - FEC encodes/decodes (not likely if no queued stuff/clients). - Resuming requests on startup (again not likely, but check that downloads.dat.gz is empty, it should be ~21 bytes if so). Most likely your problem is caused by one of the above issues IMHO. Could you please get a stack dump and another one of those thread/load dumps simultaneously? Then add "wrapper.java.additional=-Xloggc:freenet.loggc" to your wrapper.conf. Don't restart the node yet as the thread/load dump may suggest something I'd want more info on. More comments below... On Thursday 21 February 2008 00:47, you wrote: > Hi, > > I'm sorry to annoy you with that (I know that a lot of users probably > already did) , but I have various questions about the node, the main > one being "why does it use so much resources ?!". > > There is a reason behind this question : I'll have to shut down my > node on my dedicated server because, honestly, having the CPU used at > 100%, 3/4 of the memory used all the time, and a load always >= 2 > even when no downloads/insertions are running is not acceptable for me > anymore :( (it's a celeron 2,6Ghz with 256MB of memory) If you could try to help me with some debugging first that'd be awesome. If not you're just moaning. :) I'm assuming you also run More Important Things on the machine mentioned? 100% CPU should not be a reason to decomission it by itself, unless power consumption is a big deal. > See by yourself: > http://zeus.kwain.net/~jflesch/chronos.jpg > http://chronos.kwain.net/~jflesch/wrapper.log (I have these messages > almost each hour in my wrapper.log, and after some restarts, the > wrapper stops the node). Does not explain why it restarts the node in the first place. But it does suggest a memory problem. > Note to toad : can you remove my ref from the seednodes file please ? > > I have the feeling that the CPU use is somehow related to the number > of threads used by the node. And when I see the number of threads used > by the node, I can only wonder "damn ! why does it need so much > threads ?!". I mean, an usual network application only need some few > threads for each socket : > - One thread reading what's coming on the socket, and starting a new > thread for each message read (using a thread poll like you do if > needed) > - If you have a lot of things to send, one thread used to send all the > data that are put in a queue (or any more sophisticated structure). > When I look at the node stats, I see a lot of threads who, according > to their names, are probably waiting for a specific message on the > socket or waiting to send their own message on it. > Moreover, I think that by reducing drastically the number of threads, > the node could spare a lot of memory currently used in the threads > stacks. So, would it possible with the current freenet code ? (I tried > to determine that by myself, but I must admit that the code of the > node is quite huge). I very much doubt that thread stacks are *that* big a problem. Java is designed to be multi-threaded. And there are good reasons to be multi-threaded. Most of our threads are described below: - We do indeed have one thread per UDP socket to handle incoming data. - We also have one thread to handle sending data (and a few other administrative tasks). - We have threads for FEC encoding. - We have threads for sending requests and inserts (and announcements). Receiving requests and inserts uses a thread for a short period and then goes asynchronous. The point is, this is core logic, it is *really* important that it work well. The code is vastly simpler by using a thread per request rather than a state machine, and in terms of getting good debugged code, simplicity matters. If we have to reduce the number of threads used by requests, the best solution is probably to pull in a continuations library and use that, but so far I haven't seen any reason to do so. - We have threads for block transfer. This can be reduced; right now it is a good deal more complicated than it needs to be. It will happen when we do some major link level rewriting. - We have threads for FCP and HTTP - one per connection and one per server socket. This is easiest, and on an idle node it's just 2 threads (3 if console is enabled). - There are a few housekeeping jobs that still have dedicated threads even though they sleep most of the time e.g. DNSRequester. Feel free to convert them into ticker jobs. > > About the memory use, I have a similar question : Where all the memory > gave to the node goes ?! Before, I was believing that most of the > memory was used to index the datastore. But a node with a default > configuration only use 20MB of memory for this task, so for what are > used the remaining 100MB? BDB allows you to control some but not all of its memory usage. The part it lets you control is 20MB, it uses a significant amount beyond that (maybe 50%). On all nodes I've profiled, most of the remaining memory is used for queued requests. I have no idea why a node with no queued downloads would use a lot of memory or CPU, my experience suggests this is a bug. > > I'm asking these questions because I really think that if these issues > could be solve, then the bandwidth available over freenet could become > really more interesting (more bandwidth => more spreading of freenet > around the world => more probabilities to reach the countries for > which Freenet is designed). Freenet is quite capable of reaching the sort of bandwidth that is readily available to most users right now. For example on my node, with a 40kB/sec output limit: * Total Input: 4.15 GiB (33.4 KiB/sec) * Total Output: 4.37 GiB (35.2 KiB/sec) * Payload Output: 2.80 GiB (22.5 KiB/sec)(64%) Dedicated hosts on 100Mbps lines are really not that interesting for a fully distributed system. > Moreover, I'm used to see Freenet just > like another layer above UDP/IP, but I can't imagine having a network > layer using so much memory/CPU (and that's regrettable because I > really believe that distributed storage can be the future of > networking). It is possible to go over the top with optimising. Programmer time is very expensive compared to RAM. The average new PC has 1GB of RAM, maybe 512M for a really low-end system. Having said that, by all means we should deal with all the low-hanging fruit; Freenet should not use more memory and CPU than it needs to, for several reasons: - People in the places that need Freenet most may have low-end hardware. - Freenet's impact on a desktop system needs to be minimal if people are to continue to use it and allow it to run 24x7 in the background. - Upstream bandwidth is rising, albeit slowly in most places. - A few geeks will run it on low-end dedicated servers, whether on their home LAN or at Bytemark etc.
pgpJRHT44GaZ4.pgp
Description: PGP signature
_______________________________________________ Devl mailing list [email protected] http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
