On Mon, Sep 05, 2005 at 10:56:33PM +0200, freenetwork at web.de wrote: > >So, a little more concrete: > > [snip] > > Maybe I'm simply ignorant and/or deaf, dumb or something, but I've got some > questions. > > Meta: > I followed your publish/subscribe mails that went over devl-list but was not > able to find the mail which describes the need for the proposed p/s system. > * What is it for? > * As you might have noticed, you were the only one that said something to > this p/s proposal. Nobody (Ian once) said something about it. Maybe there are > others that don't see the use of it and grind their minds?
Sorry, we weren't very public about it, although it has been discussed a few times. Briefly: On a working routing fabric, such as that which should be provided by 0.7 with oskar's new algorithm, we can build various services: - Insert and retrieve data (0.5 semantics) - Notify when a piece of data is inserted (passive requests) - Messaging from one node to another across the network via location (e.g. to tell them your new IP address; not very reliable as locations can change) - Inserting data pointers (maybe; inverse passive requests) - Publish/subscribe. This is a 1:many, very close to real time, reasonably reliable, means of sending a stream of data around anonymously. A user can create a stream, and then post messages to it. Another user can subscribe to that stream, over the network, using the stream key, which will be PSK@<blah...>/<name> . > > Tech: > Okay, if I understood correctly, stream-participating nodes are built up and > bound together like a tree lying above the network, linked together by a > stream ID and a stream routing table for every stream that's crosing the node. Pretty much yes. > If another node wants to participate, it is enrolled into the tree. If a > message comes in, it is routed up to the root node, then all the way through > the tree to reach all stream participants. > * What are the security and anonymity aspects when subscribers span up a > tree? Isn't that somewhat trace- or probeable? How so? > * There was a streaming test implementation in the old 0.4 aera made by fish > featuring FECed chunks of data that were adressed like edition freesites > (there was some additional meta IIRC). So you could skip within that stream, > old unwanted parts of the stream would fall Indeed. Which never really worked across the network, was absurdly inefficient, and at best could have achieved latencies of minutes. I think we estimated that 90 seconds might be achievable at one point. Failure tables meant that half an hour was a realistic minimum at the time, but are no longer a problem. > out of the network and new ones are inserted by the "sender" for all > interested clients to be received as datachunks using the normal freenet data > retrival scheme. What's the gain of this p/s thing? It's a LOT quicker and a lot more efficient. It should be possible to run IRC on it. We don't need to poll keys, which also takes a massive load off the network. > * Is this complex p/s architecture needed? Can't the functions be done with > "pure" datamoving? What do you mean? Requests and inserts? In theory, but in a grossly inefficient and slow way. > * Are the datachunks cached and routed in the same way as normal freenet CHK > data? They are not cached in the same way, but the whole thing depends on routing. > * Why an additional tree-structure if ((n)ng-)routing exists? The function of the tree is to fan out the messages, so we don't have individual connections to each client from the root node. On each node in the tree, the data is cached, and a normal subscribe request will terminate when it reaches it. > * How is routing affected by this "treerouting"? It isn't. The tree uses routing. > * Are those streams to be understood as "shoutcast" audio streams? as > newsfeeds? as read-only filestreams? IRC-style streams? What are their main > use and what are they geared for? bulk transfer? low latency? Anything from a non-polling RSS to IRC. Low latency. NOT bulk transfer, in general - you insert a key for that. We will probably have severe limits on transfer rate, as the data used to move stuff around could be used for other purposes e.g. normal requests. > * What if the "alchemistic" (as you like to call such things) limit of 32 > streams isn't sufficient? Why 32 anyway? Why not 65536 or +inf? or 4? So you can keep the caches all in RAM, mainly. Also to serve as a load limiter. It's alchemy, essentially. > * What if DNFs are common? or at least often enough to have the tree break > into pieces having to Resubscribe over and over? Then routing doesn't work, and ordinary requests won't work either. This depends on working routing. However everything suggests that oskar's new routing algorithm CAN work. > > Workflow: > You seem to be using at least some of your valuable (you being the only one > that is able to bring us 0.7) time on p/s. Volunteers are always appreciated. :) > * What priority is it? Maybe this could be considered 0.7.2 stuff and get > 0.7.0 running first? I am of the view that something new and fun is important to the goal of getting some money and enthusiasm so we can continue to work on the rest of 0.7. IRC over Freenet is something we've never been able to do. And furthermore, IRC is a great way to establish a community, as shown by IIP and I2P. Certainly there is loads to do on 0.7; it just seems best to work on this first. We have basic insert/request working, we need splitfiles, and SSKs, and FCPv2, and fproxy, and rate limiting, and so on. > > I hope you can ease my bogged head. -- Matthew J Toseland - toad at amphibian.dyndns.org Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: <https://emu.freenetproject.org/pipermail/tech/attachments/20050905/0d7738a4/attachment.pgp>
