On Tuesday 27 Mar 2012 10:21:58 Chetan Hosmani wrote: > Hello, > > I have been idling on the IRC channel for quite some time now. The > response from freenet is really good. > > For my GSoC application I have been working on a proposal for the > transport plugin. Although the response from freenet is "this is a > very hard project", I have tried my best to understand the codebase of > freenet and the exact purpose of this project. In particular I have > spoken to Arnebab, toad_ and nextgens regarding this assignment and > from them have gained a good insight on what needs to be done. > > Based on their information and some research on the project this is my > present standing. Some of it might still be incorrect. > > Firstly Freenet presently runs extensively on UDP based sockets. The > communication happens at several layers and with different mechanisms > i.e sockets, streams, reliable packets, UDP, so on... The major > problem is that the code has been integrated very tightly. For e.g. > NodeCrypto class uses only UDPSocketHandler for communication. So this > means that the data cryptography and communication at the transport > layer (using UDP in this case) are grouped very tightly. > > This means that a major refactoring of the code is needed. This task > is supposed to be the hard part (where prior freenet experience is > needed). > Changes will definitely encompass refactoring - Node, NodeCrpyto, > UdpSocketHandler and other related dependencies. > For this I plan to do a very thorough research and practice on the > core functionality of freenet way before the coding period begins, so > I know the exact task at hand. > I ll obviously be at the mercy of the community. > > On the other hand a lot of work has been completed. For eg. > implementations of OutgoingPacketMangler and IncomingPacketFIlter > allow packets defined for any transport protocol. This is also > mentioned here - "Last year's work on new packet format should really > help although some transports (really small packets e.g. pretending to > be Skype) will still need to do their own splitting/reassembly (this > should probably happen within the node too, although it should be > possible to turn it off). " > Streams have better support: https://bugs.freenetproject.org/view.php?id=2214 > > Secondly once this is achieved, UDP will become an individual > transport plugin and similarly the framework will support users to > write their own transport plugin. Now this means the cryptography and > packet modifications are done in a different level, and hence the > developer need not bother about them. As part of the GSoC project I > will be required to make this change and also in the process develop > TCP transport plugin. > Here I think I am more comfortable, and I think my existing knowledge > of sockets should get me through. > > Thirdly, some other objectives as toad_ mentioned as important, > include ways to deal with having multiple connections open to the same > peer at the same time. Presently haven't thought about this, and don't > know that much about freenet for the exact need for this. > > And apart from this (some confusion regarding this) is implementation > of other application level protocols like HTTP, VoIP and so on. Now > this can "become" easy if protocols like TCP are enabled. Also as > mentioned in the project page is the ability for communications to > pretend to be of other protocols. Again I believe it means that an > example plugin needs to be developed. > This part of the project would spill outside the deadline but it can > get direct contribution from the community. > > The application period has now started, so I ll be turning in mine. > But I was hoping I could clarify a few things. > > I know this is beyond what can be finished in three months. *Please > give me your opinion on this proposal and what I should do. * > > Also as nextgens mentioned, this project would be very hard for me, I > would like to know if I should continue researching more or probably > give something else a shot. I still have a week to go either way. But > I am aware this requires a lot of effort and knowledge and I am ready > for that. For now I will try and fix a bug. > > Thank you > > PS: Comments, including "you don't know shit" or "go watch TV" are welcome :)
Okay, major issues: Re crypto, we need two (or more) separate APIs that a transport plugin can register itself with, the obvious ones at this point being: - Streams. - Unreliable packets. Each of these transport types will need: - Crypto - both setting up the connection and once it is connected. - Authentication. - Bottom part of the retransmission logic. (Probably not needed for streams). - Code to actually drive the transports. (For a packet-based transport, we probably expect the plugin to provide its own send/receive loops; for a stream-based transport we'd probably want it to give us an InputStream/OutputStream pair for each connection) Some of this code can be shared. For example, the actual maths/crypto in connection setup would probably be very similar, but the structure of the code could be quite different. Message queueing before messages are split is the same. Formatting a Message into a set of bytes is the same. Retransmission logic - splitting up messages, resending them when they're not acknowledged etc - I'm not sure about, we have code to split up messages and track what parts of them have been sent, acknowledged etc, which could be reused by any lossy packet transport, streams might just send whole messages at once but would probably still need some sort of cumulative acknowledgements (i.e. "we have received up to message #<number>")... There are definite advantages to having a global structure for messages in flight, so we can send them down any connection and avoid losing messages when one breaks - and in fact, most of the code currently assumes that either the messages are delivered successfully, or the connection is restarted or lost. This is one of the reasons why we need to look at how to have multiple connections simultaneously as part of transport plugins. One of the other reasons is we may have multiple transports available and may try to set them all up simultaneously - and in some cases, it may be true that either one of them is faster than the others, or that we get more bandwidth by using all of them. In any case the current code will not handle this properly, and it is inevitable that problems will happen with multiple transports (we barely manage with multiple IP addresses!). There is no reason to require more than a TCP plugin for a Summer of Code. But the API must support more advanced plugins and make it reasonably easy to write them, by doing as much as possible of the hard stuff in the node. So the core thing would be to refactor, and to make a TCP plugin; a really simple stego plugin, e.g. that relays data to a web server until it sees a keyword, would be nice, but IMHO shouldn't be *required* by the GSoC task. Hopefully a mentor can be found. I really don't have time, nextgens would probably have a pretty good idea about the code, possibly one or two other people might ...
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Devl mailing list [email protected] https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
