I tend to use bash_history as a way to remember specific commands to run for various tasks, often quite irregularly, and thus keep HISTFILESIZE quite large (800k lines in my .bash_history, for 32MiB). bash/readline handle this just fine, except that I often have something like 50 instances of bash running in various terminal emulators. This leads to something like 7.5GB of memory dedicated to bash instances (around 150MB per instance), with most of it going to history storage.

While various optimizations to improve the space efficiency of history would be welcome, ultimately this really feels like a task for Linux's samepage merging logic. If the history strings themselves were put into contiguous memory (okay, not incredibly trivial, but arenas aren't so complicated), it should be fairly straightforward to just `madvise(addr, length, MADV_MERGEABLE)` and let KSM handle the rest.

I can of course set a lower HISTSIZE than HISTFILESIZE and set histappend to only load some of history in memory, but even the last 10k lines of history leaves me often grep'ing bash_history manually. Maybe I'm missing some obvious way to reduce redundant many-bash-instance memory usage, though?

Thanks,
Matt

Reply via email to