On Apr 2, 2005 1:40 AM, Christian Biere <[EMAIL PROTECTED]> wrote: > So far Gtk-Gnutella can only perform an overlap check to verify > the consistency of partial downloads and the final SHA1 check. With > TTH we could verify the consistency of each single chunk and therefore > drop bad ones and recognize bad sources.
I've read the THEX link you sent now and understand a little better. It is a very good idea. It presents some interesting implementation problems though that will need some meditation. > > Could be simply size chunks based on past performance of source, merge > > any chunks that are below the minimum size (assuming they are > > contiguous), then let each source have its own chunk? > > Sounds reasonable. If you peek at Bittorrent, the optimum size is > probably somewhere between 16-256 KiB. Of course, BT uses pipelining > so that smaller chunks don't cause a latency pause. Now I just have to figure out how to determine past performance. The logical way hasn't been working for me. Go figure. > > Are there that many people that start a huge number of > > downloads that aren't almost instantly finished? I'd imagine that the > > only people who actually _start_ a huge number are those that can > > finish a huge number in short order. The slow-computer-dial-up users > > only start three or so at a time, right? > > Well, that's another problem in Gtk-Gnutella. Queued downloads and > active ones are subject to the same download slot count. So people > tend to allow dozens or hundreds of downloads at a time to prevent > having nothing but idle queued downloads. It's also not really > possible to manually queue/preference downloads. Keep in mind that > sources need to be kept fresh, so it's cannot be as blatantly > simply as just pausing them. Yeah, it seems more logical to me that idle queued downloads would be automatically retried again after a delay, but others in our queue are also tried in parallel. Once we got our maximum of _active_ downloads, then all of our idle queued would be queued on our end again. Is there a reason it doesn't work this way? Was that paragraph even parsable? > > > If you have some time to work on Gnutella, I'd rather encourage > > > you to try getting TTH support to fly. > > > Point me at documentation if you have it sitting around. If not I'll > > google after work. > > A lot of necessary code is already there - albeit not test, AFAIK. > The next step would be wiring this code i.e., finding the right > places and adapting the current download logic - which is unfortunately > comparatively complex. Yeah, I've been neck-deep in the download logic so far. I think that I've almost figured out the important bits enough to try hacking the stuff we've been talking about. Now how long it will take the hacks to actually _work_ is anyone's guess. > Some documentation is here: > THEX: http://www.open-content.net/specs/draft-jchapweske-thex-02.html > Partial File Sharing Protocol: doc/gnutella/PFSP Thanks. > I'm not sure what's the best place to put the tigertree. One > possibility is to use the fileinfo infrastructure and just > put it into the partial file. Otherwise, the tigertree needs > to put into a separate file which is somewhat uglier to handle. > One further issue is that Gtk-Gnutella cannot download files > without knowing the filesize in advance. There were some > half-baken attempts to fix this but it's very bug at the moment. I wasn't even thinking of where to store stuff when gtk-gnutella is closed. That _is_ an interesting problem. If we stick it in the partial file, we risk breaking the ability to view/read/hear the partial. I know that mplayer is pretty good about playing movies/music even if it has garbage at the beginning, but I don't think we should assume that everyone will be using such forgiving programs. Then again, potentially having two files per partial seems very ... ugly to me. Cannot download without knowing the filesize in advance? Ouch. I can see where that might be a problem though, given untrusted sources. > > > At the moment, I suppose doing it vice-versa would be more effective > > > i.e., use the fast sources for random chunks. Another thing that > > > would speed thing up is pipelining. > > > That seems illogical to me, so I'm sure that I'm missing something. I > > see no way in which it is better for the individual user to have the > > random chunks downloaded faster. I see a potential tiny improvement to > > the network as a whole, but not one that I would even begin to > > classify as more important than the benefit to the user. > > Well, yes, I was thinking about the network as a whole. I don't > have hard facts on this, though. In general, what's good for the > network should be good for the user, the reverse does not apply. It's fast becomming clear that some testing will need to be done to be sure either way. I can appreciate the focus on the network health. But for the record, the reason I'm leaning the other way is that I see a sure-fire win for the user (at least this user, and I'm guessing that I'm not _that_ rare of a use-case) versus an unsure win for the network. I can see where it might help the network overall, but I'm just not sure. > -- > Christian ~Daniel ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Gtk-gnutella-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/gtk-gnutella-devel
