On Sunday 13 Mar 2011 22:44:30 Andrea Mambrini wrote: > Hi toad, > I'm thinking on working as a Google Summer of Code project on the > implementation of transport plugins to make freenet to run over TCP (and > maybe HTTP) and to implement some steganography techniques to hide the > traffic from network analyzers. > > Do you think it is feasible in 4 months? In the mailing list Arne wrote that > it might be too big project. What do you think? Maybe it would be more > feasibile just the transport plugin part?
It would be a large project. It may be possible to split it up a bit and still get something useful out of it though. There are many components and issues to consider: - General infrastructure: Coordinating multiple connections per peer, for instance; e.g. should messages be sent down one connection if another fails??? - Transport plugins infrastructure: Plugin interfaces, plugin registration, basic APIs, node references, how to represent link-level pubkeys if needed, etc. - Transport plugins must be as easy to implement as possible, so it is essential that Freenet itself does most of the work. The packet transport should be able to register itself and have freenet deal with packing, encryption, retransmission (if necessary) and so on - with it possible for the plugin to turn off and take over specific parts e.g. retranmission. - Several different plugin types, each with its own interface and registration API, and parameters. For instance: -- TCP-style streams: This is just a stream in each direction. Encryption will be different, MACs will need to be periodic (or streamy using authenticating ciphers), packing could be similar, no need for retransmission but we probably still want acks. -- Lossy packets: Similar to current UDP code. The plugin should just ask for a packet in a given size range, and report an incoming one; the plugin's job is to wrap it with appropriate headers to make it look like something other than Freenet, or to get through a network obstacle. Freenet will implement the same as it does now - encryption, retransmission, packing, etc. -- Non-lossy packets where the external protocol provides a retransmission system. Has its own difficulties, of course... - For packet-style transports, there may be a maximum size, which could be very small (as in something looking like Skype), or very large; advanced steganographic plugins may force a specific packet size or even generate traffic to match a model of what it is pretending to be, sending packets regardless of whether there is anything to put in them. - The packet format changes from the last GSoC have now been deployed and have separated the size of messages from the size of packets, allowing us to efficiently pack parts of messages into any sized packets, but even so, very small packets (skype-sized e.g. 27 bytes) would be a problem, as there isn't really enough space for the packet number, the MAC etc. Anything 100 bytes upwards can be handled pretty efficiently though. - Variable latency might be an issue for some transports, and even variable bandwidth ... eventually. Long term, some plugins will want to send stuff even when we don't have anything to send, so we may want to be able to create useful packets even when we don't have data to transport, e.g. transferring bloom filters (related to bloom filter sharing) or (depending on trust level etc) parts of our datastore. We might even want to implement a special case of low bandwidth transport plugins just to exchange IP addresses and help the others to get set up. Now, how do we define a feasible four month student project out of all of that? Some ideas for breaking it up into somewhat manageable chunks... Transport plugins and lossy packets: (probably the whole of one SoC IMHO): - General infrastructure. E.g. handling multiple connections between two nodes cleanly. - Basic transport plugins infrastructure. - API for lossy packet based plugins, and driver code to talk to that API (e.g. should each plugin implement its own send loops and manage traffic between them? how do we manage bandwidth usage - simplest is just to grant access to the global token bucket??? do we want to have a single PacketSender thread still, and a sendPacket() method on the plugin guaranteed to return quickly??? this stuff will determine the form of the API - probably it is best to let the plugin manage the sends, even though that increases its complexity slightly...) - Refactor the existing code to use the API for lossy packet based plugins. - Implement a really simple wrapper-style lossy packet plugin, which pretends to be some UDP application with medium-sized packets (several hundred bytes), and adds headers sufficient to convince traffic classifiers. Streams: - API for stream-based plugins, and driver code (stream transports will use streams so probably two threads per connection) - All the heavy lifting on the node side: Connection setup (basically similar to existing setup but very different interfaces), stream encryption, authentication, acks/how to deal with not needing to retransmit, etc. - A simple TCP-based plugin which doesn't do any stealth. Immediately useful for places where we don't have UDP but have outgoing (or even incoming) TCP. - An HTTP CONNECT plugin (some proxies allow you to switch to a plain TCP stream), or a "stealth TCP" plugin (where data is generally proxied to a local service e.g. a web server, but if the magic string, which depends on the node identity, the date and time etc, is seen, we switch to Freenet traffic. - BONUS: A true HTTP plugin, using either one or two connections (arguably two is better for latency), which would be able to get through proxies and thus provide connectivity even when there is only HTTP proxied access. Small packets and Skype: - Use the Skype library/API to send data through a real Skype instance. This might be a stream or a lossless datagram service, I don't recall. - Design a scheme for making small packets workable e.g. with distributed/higher level HMAC's, and make a fake-skype transport. I suspect that the first part would be more than enough for a Summer of Code project: Infrastructure and a simple cloaked lossy packet plugin. The fact that it involves major refactoring might be a problem, although the APIs aren't all that bad now IMHO, and some of the students we've had in recent years have been so good that they could certainly have pulled it off (with reasonable help and explanations from mentors), so it might not be a big problem. I am doubtful about just going for TCP without refactoring the packet-based transports to use the new infrastructure; clearly something would need to be done with the existing code, especially if (as I believe is appropriate) we are reusing most of the code.
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Devl mailing list [email protected] http://freenetproject.org/cgi-bin/mailman/listinfo/devl
