This is description of my work on node to node file transfers. Most of the work where made a couple of weeks ago. The last few weeks I have been very busy but now I have got some time hour to put it together.
I have chosen an architecture where the file transfers are stored in the request queue. The advantage of this is twofold. It makes it easy to send and receive files over FCP. It is also makes it easy to make the transfers appear on the uploads and downloads pages, which is where the users expect to find them. To achieve this two new subclasses of ClientRequest has been created, ClientSend and ClientReceive. A ClientSend is created when a user offers a file to a friend. When it is created a a N2N file offer message is sent to the friend. This results in a UserAlert being registered at the friend. If the friend accepts the offer a ClientReceive is created and a N2N file offer accept is sent. If the friend rejects the offer a N2N file offer rejected is sent and nothing is created in the request queue. Previously two Maps where used i DarknetPeerNode to keep track of sent and received file offers. The Map with sent file offers have been removed. The entries in it have been replaced with ClientSend entries in the request queue which are persistent. The Map with received offers has been kept since a ClientReceive is only created after an offer has been accepted. This Map is recreated at startup by reading file offers from the extra peer data directory. Since ClientSender uses a BulkTransmitter and a PartiallyReceivedBulk it needs references to the MessageCore, a DarknetPeerNode and to the node to node ByteCounter. These references are made transient and are not stored in the database. Instead entries for the byte counter and the message core have been added to the ClientContext. A reference to the PeerMananger has also been added to the ClientContext. The ClientSender stores the node identity and used the PeerMananger to look up the DarknetPeerNode. ClientReceiver works in a similar way. ClientSend finds out about new progress by ClientSender by listening on ClientEvents. ClientSender generates a SplitFileProgressEvents when the transfer of a block have succeeded. For N2N file transfers there is no concept of split files, but using the existing event makes it easy to generate SimpleProgress-messages which can be sent to FCP-clients. The following conventions have been followed to apply a SplitFileProgressEvent to direct file transfers - Required always equals Total - FinalizedTotal always equals true. - Failed and FatallyFailed equals zero. ClientReceive and ClientReceiver communicates in the same way. To create a ClientSend which sends a file offer the ClientSend FCP-message can be used ClientSend Filename=name of file ClientToken=arbitrary string Global=true of false Verbosity=0 or 1 Identifier=arbitrary string NodeIdentifier=node identifier EndMessage To find file offers a user can use the WatchFeeds FCP-message WatchFeeds EndMessage If there is a file being offered this returns a ReceivedFileOfferFeed which includes such information as uid, filename, mimetype, node identifier and size. To create ClientReceive which sends file offer accepted the ClientReceive FCP-message can be used. ClientReceive Uid=long from RecievedFileOfferFeed Global=true of false ClientToken=arbitrary string Verbosity=0 or 1 Identifier=arbitrary string NodeIdentifier=node identifier EndMessage In repy to ClientSend and ClientReceive messages a PersistentSend and a PersistentReceive is returned respectively. At the moment N2N file transfers always use forever persistence. Does it make sense to allow them to have reboot and connection persistence as well? It is also only possible to send and receive files to the hard drive of the node. This is work in progress. It will take some time before it works properly. At the moment it can serve as an evaluation of one possible architecture. The major problems that I haven't solved are related to threading. File transfers works under ideal conditions, if both users are online when the file offer is accepted. There should be some way to automatically check if a file transfer can be started. At the sender the condition could be that the friend is online and that the file offer has been accepted. At the receiver the condition only needs to be that friend is online. How should this be achieved? Should a RequestScheduler be used or is there a more simple way?. There should also be a way to resume failed transfers. Since the data is sent and received sequentially and every block is acknowledged the only thing needed to restart a transfer is the number of blocks sent/received and this is stored in the ClientSender and ClientReceiver. For this purpose I have created a constructor in PartiallyReceivedBulk that takes an integer and sets the n first bits in the BitArray to 1. The way the BulkTransmitter informs the ClientSender also needs to be fixed. At the moment the BulkTransmitter is given a DBJobRunner when it is created and it creates a new DBJob every time a block has been successfully sent. This doesn't only lead to extremely ugly code but also to very poor performance. The same problem also exists with the BulkReceiver and ClientReceiver. The exception handling is also poor, some exceptions are not handled, others are handled at the wrong place. Maybe the new exceptions SendException and ReceiveException should be created to mimic InsertException and FetchException? The code is available at http://github.com/ljb/fred-staging I will be very busy the next two weeks and won't be doing any coding. Later however I might have a few hours over every week that I could use to work on persistent file transfers. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20090830/bee4cb61/attachment.pgp>
