On Sat, Feb 08, 2003 at 02:48:12PM -0800, Ian Clarke wrote: > > Pro: > > * The thread-limit - each HTTP connection costs a connection, and we > > cannot afford them, because we can't use as many threads in a single > > VM as we want to. > > a) Why not? Because we tried it before, and if we increase the default threadlimit everyone gets OutOfMemoryException's. > b) This won't be an issue when me move to nio
Only if we move BOTH the node AND fproxy to nio. I don't think the latter is terribly important. And we will still use threads. Moving the WHOLE node to nio would be a mammoth task for very little gain, or so Oskar told me once. We would just move the important bits. What is left will occupy much less threads than the current situation, but will still occupy threads, and therefore compete for resources with Fproxy. > > > * In general, Fproxy etc wouldn't interfere with Fred's resources > > The computer as a whole would have less resources, since it would have two VMs > running. Resource management within the Freenet VM is just a question of doing > things right. How much less? Can you quantify it somehow? It would be possible to shut down the client VM when it was not being used, if that was required. In any case, the client VM shouldn't use much resources when it is idle, unlike the node VM, which is never idle on a healthy permanent node, although it has peaks of activity. This underlines the point that the node and the clients have very little in common, that they should be separated in order to make freenet more modular and to make it easier to create alternate node implementations, over and above any immediate material benefits. > > > * Possible to restart Fproxy without restarting Fred > > Why would this be so valuable? Because it can take a really long time to restart fred on a large datastore, and there isn't much I can do about it beyond what we have now (the index files - but the index files have to checkpoint, which means locking the datastore while cloning the hashtable...). > > > * Easier to implement alternate interfaces such as a browser plugin if > > _everything_ is exposed by FCP. > > True, but we don't need to run Fproxy in a separate VM just to expose > everything via FCP. The original intent was to have both a FCP sockets > interface (as we do), and a parallel FCP Java API (as we don't really) which > could allow plugins to run in Fred's JVM. Separating them forces the issue. > > > * Easier to make alternate implementations of the node. Or make radical > > changes to Fred without making corresponding changes to Fproxy - for > > example, the long-heralded nio. > > If the API which FProxy uses to talk to the node parallels FCP then this > will be the case anyway. No it won't. An alternate node implementation would have to implement shitloads of stuff, for example FEC. We could put FEC, for example, into a middle layer of "high level FCP", provided by the fproxy process. > > > * People who only use third party FCP clients don't need to run the > > Fproxy VM at all. > > They wouldn't need to run the FProxy plugin either. Please explain what if anything you mean by plugins. I am interested in the idea of third party plugins to implement specific services such as a NIM2email gateway... > > > * Possible to create a package with only the node, and a package with > > only Fproxy, if wanted. > > We could create a package which didn't include the FProxy plugin if we wanted > too. It would be a complete kludge. > > > * Considerably easier to run the client side on a different machine to > > the server side. > > Than what? Pointing your web browser at something other than 127.0.0.1? > > > Comments? > > I don't think any of the benefits outweigh the additional resource consumption > that would be required for two JVMs. We take enough crap over resource usage > as it is. Our resource usage problems are concerned in two areas primarily: * CPU usage. This is caused by bugs, by not having nio and by being run on systems that are far too old. * Usage of threads. This would be fixed by NIO, but only if we apply NIO to both fproxy and the node. It would be mitigated significantly by separating the fproxy VM from the node VM, not in terms of the number of actual threads used on the machine but in terms of the ridiculously crude resource usage indicators Java forces us to use (the thread limit). Finally, implementing NIO properly would/will probably take a significant amount of time, code changes and debugging. It should be implemented eventually, but separating fproxy from the node is a relatively easy thing that we could do in a relatively short time, if it is a good thing. I think it is. The client code shares very little with the node that can't be used from the library out of the VM, or ported across FCP. Also, I don't see that the FEC commands should be a requirement for a working node implementation - we could have the node talk "low level FCP" (basic FCP, without the FEC commands) and FNP, and the client provide a proxy FCP port that passes through low level commands and implements high level ones - such as the FEC commands, or future high-level FCP commands. > > Ian. > > -- > Ian Clarke ian@[freenetproject.org|locut.us|cematics.com] > Latest Project http://locut.us/ > Personal Homepage http://locut.us/ian/ -- Matthew Toseland [EMAIL PROTECTED][EMAIL PROTECTED] Full time freenet hacker. http://freenetproject.org/ Freenet Distribution Node (temporary) at http://amphibian.dyndns.org:8889/KHdtkpR1UhY/ ICTHUS.
msg06650/pgp00000.pgp
Description: PGP signature
