How about a compromize? Seperate config files for the node and the client, a Main class for the client that can be run directly independantly of the node, but by default is run as a thread in the Fred VM?
On Sat, Feb 08, 2003 at 11:15:16PM +0000, Matthew Toseland wrote: > On Sat, Feb 08, 2003 at 02:48:12PM -0800, Ian Clarke wrote: > > > Pro: > > > * The thread-limit - each HTTP connection costs a connection, and we > > > cannot afford them, because we can't use as many threads in a single > > > VM as we want to. > > > > a) Why not? > Because we tried it before, and if we increase the default threadlimit > everyone gets OutOfMemoryException's. > > b) This won't be an issue when me move to nio > > Only if we move BOTH the node AND fproxy to nio. I don't think the > latter is terribly important. And we will still use threads. Moving the > WHOLE node to nio would be a mammoth task for very little gain, or so > Oskar told me once. We would just move the important bits. What is left > will occupy much less threads than the current situation, but will still > occupy threads, and therefore compete for resources with Fproxy. > > > > > * In general, Fproxy etc wouldn't interfere with Fred's resources > > > > The computer as a whole would have less resources, since it would have two VMs > > running. Resource management within the Freenet VM is just a question of doing > > things right. > > How much less? Can you quantify it somehow? > > It would be possible to shut down the client VM when it was not being > used, if that was required. In any case, the client VM shouldn't use > much resources when it is idle, unlike the node VM, which is never idle > on a healthy permanent node, although it has peaks of activity. This > underlines the point that the node and the clients have very little in > common, that they should be separated in order to make freenet more > modular and to make it easier to create alternate node implementations, > over and above any immediate material benefits. > > > > > > * Possible to restart Fproxy without restarting Fred > > > > Why would this be so valuable? > Because it can take a really long time to restart fred on a large > datastore, and there isn't much I can do about it beyond what we have > now (the index files - but the index files have to checkpoint, which > means locking the datastore while cloning the hashtable...). > > > > > * Easier to implement alternate interfaces such as a browser plugin if > > > _everything_ is exposed by FCP. > > > > True, but we don't need to run Fproxy in a separate VM just to expose > > everything via FCP. The original intent was to have both a FCP sockets > > interface (as we do), and a parallel FCP Java API (as we don't really) which > > could allow plugins to run in Fred's JVM. > > Separating them forces the issue. > > > > > * Easier to make alternate implementations of the node. Or make radical > > > changes to Fred without making corresponding changes to Fproxy - for > > > example, the long-heralded nio. > > > > If the API which FProxy uses to talk to the node parallels FCP then this > > will be the case anyway. > > No it won't. An alternate node implementation would have to implement > shitloads of stuff, for example FEC. We could put FEC, for example, into > a middle layer of "high level FCP", provided by the fproxy process. > > > > > * People who only use third party FCP clients don't need to run the > > > Fproxy VM at all. > > > > They wouldn't need to run the FProxy plugin either. > > Please explain what if anything you mean by plugins. I am interested in > the idea of third party plugins to implement specific services such as a > NIM2email gateway... > > > > > > * Possible to create a package with only the node, and a package with > > > only Fproxy, if wanted. > > > > We could create a package which didn't include the FProxy plugin if we wanted > > too. > > It would be a complete kludge. > > > > > * Considerably easier to run the client side on a different machine to > > > the server side. > > > > Than what? Pointing your web browser at something other than 127.0.0.1? > > > > > Comments? > > > > I don't think any of the benefits outweigh the additional resource consumption > > that would be required for two JVMs. We take enough crap over resource usage > > as it is. > Our resource usage problems are concerned in two areas primarily: > * CPU usage. This is caused by bugs, by not having nio and by being run > on systems that are far too old. > * Usage of threads. This would be fixed by NIO, but only if we apply NIO > to both fproxy and the node. It would be mitigated significantly by > separating the fproxy VM from the node VM, not in terms of the number > of actual threads used on the machine but in terms of the ridiculously > crude resource usage indicators Java forces us to use (the thread > limit). > > Finally, implementing NIO properly would/will probably take a > significant amount of time, code changes and debugging. It should be > implemented eventually, but separating fproxy from the node is a > relatively easy thing that we could do in a relatively short time, if it > is a good thing. I think it is. The client code shares very little with > the node that can't be used from the library out of the VM, or ported > across FCP. Also, I don't see that the FEC commands should be a > requirement for a working node implementation - we could have the node > talk "low level FCP" (basic FCP, without the FEC commands) and FNP, and > the client provide a proxy FCP port that passes through low level > commands and implements high level ones - such as the FEC commands, or > future high-level FCP commands. > > > > Ian. > > > > -- > > Ian Clarke ian@[freenetproject.org|locut.us|cematics.com] > > Latest Project http://locut.us/ > > Personal Homepage http://locut.us/ian/ > > > -- > Matthew Toseland > [EMAIL PROTECTED][EMAIL PROTECTED] > Full time freenet hacker. > http://freenetproject.org/ > Freenet Distribution Node (temporary) at >http://amphibian.dyndns.org:8889/KHdtkpR1UhY/ > ICTHUS. -- Matthew Toseland [EMAIL PROTECTED][EMAIL PROTECTED] Full time freenet hacker. http://freenetproject.org/ Freenet Distribution Node (temporary) at http://amphibian.dyndns.org:8889/5zk9P2XjVHA/ ICTHUS.
msg06657/pgp00000.pgp
Description: PGP signature
