Hi there,

the task is simple: count word occurrences from stdin (around 150mb in this case) and print sorted results to stdout in a somewhat idiomatic fashion.

Now, d is quite elegant while maintaining high performance compared to both c and c++, but I, as a complete beginner, can't identify where the 10x memory usage (~300mb, see results below) is coming from.

Unicode overhead? Internal buffer? Is something slurping the whole file? Assoc array allocations? Couldn't find huge allocs with dmd -vgc and -profile=gc either. What did I do wrong?

```d ===================================================================
void main()
    import std.stdio, std.algorithm, std.range;

    int[string] count;
    foreach(const word; stdin.byLine.map!splitter.joiner) {

    //or even:
    //foreach(line; stdin.byLine) {
    //    foreach(const word; line.splitter) {
    //        ++count[word];
    //    }

        .sort!((a, b) => a.value > b.value)
        .each!(a => writefln("%d %s", a.value, a.key));

```c++ (for reference) =================================================
#include <iostream>
#include <vector>
#include <unordered_map>
#include <algorithm>

using namespace std;

int main() {
    string s;
    unordered_map<string, int> count;
    while (cin >> s) {

    vector<pair<string, int>> temp {begin(count), end(count)};
    sort(begin(temp), end(temp),
        [](const auto& a, const auto& b) {return b.second < a.second;});
    for (const auto& elem : temp) {
        cout << elem.second << " " << elem.first << '\n';

Results on an old celeron dual core (wall clock and res mem):
0:08.78, 313732 kb <= d dmd
0:08.25, 318084 kb <= d ldc
0:08.38, 38512 kb  <= c++ idiomatic (above)
0:07.76, 30276 kb  <= c++ boost
0:08.42, 26756 kb  <= c verbose, hand-rolled hashtable

Mem and time measured like so:
/usr/bin/time -v $cmd < input >/dev/null

Input words file creation (around 300k * 50 words):
tr '\n' ' ' < /usr/share/dict/$lang > joined
for i in {1..50}; do cat joined >> input; done

word count sample output:
[... snip ...]
50 ironsmith
50 gloried
50 quindecagon
50 directory's
50 hydrobiological

Compilation flags:
dmd -O -release -mcpu=native -ofwc-d-dmd wc.d
ldc2 -O3 -release -flto=full -mcpu=native -ofwc-d-ldc wc.d
clang -std=c11 -O3 -march=native -flto -o wp-c-clang wp.c
clang++ -std=c++17 -O3 -march=native -flto -o wp-cpp-clang wp-boost.cpp

dmd: v2.082.1
ldc: 1.12.0 (based on DMD v2.082.1 and LLVM 6.0.1)
llvm/clang: 6.0.1

Reply via email to