What do we want the default heuristics to be for containers? Presently, we have: Any file below 64kB is considered for inclusion in a container. Only one container will be inserted per freesite. It is allowed to grow up to 2MB, then files are rejected and inserted separately.
Obviously this sucks: - An activelink will pull the whole 2MB container. This means indexes can take a very long time to load! - Unused sites are supposed to fall out of freenet; activelinks subvert this, but maybe they should only fetch part of the site, especially given the first point. - Larger sites could benefit from more files being included in more containers. And the current rules impose an arbitrary cutoff. - Larger files may sometimes be profitably included in the archive. Proposed solution: - Any freesite which fits entirely into a container of 256kB will be inserted as a single container. - For larger sites, if the activelink (indicated by a flag) plus the HTML plus the CSS fits into a container of 512kB, it should be inserted as a single container. The rest of the site may be inserted as a single container (excluding very large files - how large?). - For even larger sites, we try to divide it up mostly by subdirectory, with a limit of 512kB per container. Obviously these heuristics can be overridden by clients, which can specify which container each file goes into. It has also been suggested that we offer a choice of heuristics to clients (by type, by directory etc). Benefits: - Small freesites retain the advantages of being a single container. And benefit relative to the current situation because larger files can be included as long as the overall site is within the limit. - Large freesites benefit because they can still be containerised (this can be a huge benefit). - Indexes load faster, and the activelink only preloads the html for most sites (so the images will fall off if they are not accessed). Ideas? Details? Objections? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20061113/1148cda3/attachment.pgp>
