I tend to use bash_history as a way to remember specific commands to run for various tasks, often
quite irregularly, and thus keep HISTFILESIZE quite large (800k lines in my .bash_history, for
32MiB). bash/readline handle this just fine, except that I often have something like 50 instances of
bash running in various terminal emulators. This leads to something like 7.5GB of memory dedicated
to bash instances (around 150MB per instance), with most of it going to history storage.
While various optimizations to improve the space efficiency of history would be welcome, ultimately
this really feels like a task for Linux's samepage merging logic. If the history strings themselves
were put into contiguous memory (okay, not incredibly trivial, but arenas aren't so complicated), it
should be fairly straightforward to just `madvise(addr, length, MADV_MERGEABLE)` and let KSM handle
the rest.
I can of course set a lower HISTSIZE than HISTFILESIZE and set histappend to only load some of
history in memory, but even the last 10k lines of history leaves me often grep'ing bash_history
manually. Maybe I'm missing some obvious way to reduce redundant many-bash-instance memory usage,
though?
Thanks,
Matt
- MADV_MERGEABLE for history Matt Corallo
-