On 10/27/06, toad <toad at amphibian.dyndns.org> wrote: > On Fri, Oct 27, 2006 at 05:26:23PM +0200, bbackde at googlemail.com wrote: > > On 10/27/06, toad <toad at amphibian.dyndns.org> wrote: > > > > > >Of course. It's been suggested for ease of integration with other > > >systems that we provide one or more enforced checksums in the top-level > > >metadata. If we can spare the CPU cycles we could provide the full range > > >- SHA-256, SHA-1, MD5. But realistically probably just SHA-256, which > > >hopefully others will use anyway. This isn't entirely trivial to > > >implement; how important is it? > > > > Not important for now. Frost uses SHA-256 anyway.SHA-1 and MD5 are > > definetly outdated. > > MD5 is practically broken, at least for birthday attacks. SHA-1 is > theoretically broken. However both are used by other filesharing > networks.
I know. We could do it better with SHA-256. > > > > >> If you want to implement it please tell me about this. The reason is > > >> that the new Frost will rely on the fact that the same file creates > > >> the same CHK key. I assume if you add a checksum to metadata the CHK > > >> key changes... and yes I know its alpha and everything could change > > >> later, but then I will have to reset the filebase for Frost and start > > >> with a new filelist. I think also thaw and freemule would have a hit > > >> when the CHK keys for same content change. > > > > > >Well ... the same data might yield a different key, but unless the top > > >level manifest has disappeared, it would still propagate the original > > >insert. I doubt it would affect Thaw. As a general rule, metadata > > >changes will always be back-compatible, and are likely to be infrequent, > > >but they may happen. > > > > All my comments base on the fact that the top level manifest is lost > > and noone else knows the original memi and filename. Thats the worst > > case that I try to circumvent. Maybe its not worth the effort because > > this most likely never happens. But on 0.5 it happened often... > > The original MIME should be guessable. If nobody knows the original > filename, then what _do_ they know about the file? Anything? They know the original filename, its inside a signed frost post. But, you know users. They renamed the file, and I think they will not rename the file to its original name when reinserting it... either you force them to do something, or they will not do it. Or you design it fail-safe :) (I know what I say, I'm a user too) > > > > >> You mentioned one thing that sounds really good to me. If it would be > > >> possible to have one CHK key for the plain complete binary file (octet > > >> stream, no mime type, no filename) and multiple different CHKs that > > >> transport additional informations (filename,mime), this would be > > >> really great. This offers the most flexiblity, allows to compare files > > >> if content is the same and more. If the only disadvantage is that you > > >> have to insert 1 additional metadata CHK, then please do it. For 100+ > > >> MB files this makes no difference... > > >> What do you think? Could this be done? > > > > > >And for small files, it makes a huge difference. This requires an > > >arbitrary threshold in the node. How do we determine one? > > > > No problem if it is not done. Frost will use the plain approach, see > > below... > > > > >I don't see why MIME should vary from inserter to inserter really. Only > > >broken clients use application/octet-stream. :) > > > > Again here: I think about the worst case where a user renames the file > > extension for some reason. If this happens and the file CHK key > > changes, the whole Frost filesharing would be affected, because Frost > > only tracks 1 CHK key for 1 unique file. Maintaining many CHK keys for > > the same file would be possible, but requires to enhance Frost (my > > problem) and maybe requires alot of requests to the node for the > > different keys (node workload problem) in case most of the CHK keys > > are not retrieveable. > > So Frost already assigns each file a unique identifier, being its > SHA-256? Interesting. The purpose being presumably to support multiple > filenames and MIME types for the same file? I mean, exactly why is it > beneficial to index them by SHA-256 rather than by key? - independency from node and maybe changing CHK keys *g* - take load from node by computing SHA inside Frost JVM - SHA computation does not need an expensive encode - SHA is verifyable with external tools > > > > >You can force it to work this way if you want by inserting a CHK with no > > >filename, and then inserting a redirect to it. But you wouldn't at > > >present be able to ask the node for the redirect target. > > > > Because the file sharing system is cleaner with 1 CHK key per file, > > Frost will continue to insert _shared files_ without a name and > > without a mime type. Manually added (not shared) files will get a mime > > type. Using no additional informations hopefully ensures that the CHK > > key is the same for the same file. > > Another point for me is that IMHO shared binary files do not need a > > mime type. They are not intendet to be downloaded in a browser. If you > > download it in a browser this works well too and prevents that the > > browser opens the file automatically (more anonymous). You can still > > save the file and open it as usual. > > If you save it, then > a) You have to figure out the filename. > b) You have to figure out the MIME type. > c) You aren't warned when the operating system auto-detects the MIME > type, launches Adobe Acrobat and reports back to the Bad Guys. > > Really, having a MIME type is better. > > Having said that, pretty much all filesharing is inherently insecure > even over Freenet, because most interesting file formats allow for > web-bugs. :( a) is done by Frost, user can choose one of the filenames the uploaders used b) and c) true, but this is the todays situation for most apps. the file extension determines the content type. I see no need for mime types there. Except in browser they are not used or remembered anywhere. Once the file is on your filesystem only the extension and the content is needed. Correct me if I'm wrong, but at least this is true for windows. I don't know if ext3 saves any mime type for a file.... > > > For me the download of binaries > > from freenet is the same as FTP, emule, bittorrent, ... you get a > > octet stream and thats it. No need for mime types here. For html, yes. > > Needed because intented to be shown in browser directly... > > I'm not convinced of the need for such a sharp dividing line. Its no need, but thats the current situation... > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.5 (GNU/Linux) > > iD8DBQFFQisHA9rUluQ9pFARAqosAKC+Oy6ootB0JltS0X78tsjZvI/HGACfU97+ > /esbF10XRA/EfuMMnQWd7Vs= > =S4nD > -----END PGP SIGNATURE----- > > > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > >
