On Sunday, 22 May 2016 at 16:06:07 UTC, Marco Leise wrote:
There are a few open questions, for me at least. Are there other proposals with good properties? Are there any objections against the benchmarking method or the license? Can the implementation be in D? If not, should we link against the system liblz4.so/a or copy the C code into the backend? Is a pre-filled dictionary worthwhile? If so, which words should go in it? The latter is important because changes to the Dlang mangling need to be reflected in other projects outside our direct control, too.
How it's being benchmarked does make me question it a bit. I avoided doing multi-threading so my results are as they would run on 1 CPU/Core, while adding multi-threading probably isn't too hard to do (at least in D). Then there's the wonder of how many cores, and other details. A compressor designed to run in parallel will naturally be in it's own element there. This of course assumes we are allowed to in the first place; The compiler may already be using those cores for other tasks: Scanning other files, optimizing code, CTFE, etc, to which point speed gained in compression may not actually translate to an overall speedup if you are stealing CPU cycles.
As for a dictionary and being primed, I'm trying to figure out what would be in it myself. Commonly used templates and keywords that appear in symbols are so far my best guess. I'd avoid library names unless they'd appear in the symbols (so 'std.core' might not mean much, while 'unittestL' does). Then again I don't have a good sample of what I'd actually encounter live, maybe every major library import would be good thing to have.
