On Thu, Mar 19, 2026 at 07:46:24AM +0000, Sad Clouds via Postfix-users wrote:
> Am I correct in thinking that both hash and texthash have similar
> lookup performance, but hash is better suited to very large tables,
> since the database does not need to be loaded entirely to RAM and can
> be demand-paged by the OS?
Yes, "texthash" is not suitable for very large tables. That said, what
"very large" means may not be intuitive. On my modest NUC system:
$ for i in {1..10000}; do printf "foo%d bar%d\n" $i $i; done > /tmp/th.txt
$ time postmap -q 'foo5000' texthash:/tmp/th.txt
bar5000
real 0m0.017s
user 0m0.010s
sys 0m0.006s
$ for i in {1..100000}; do printf "foo%d bar%d\n" $i $i; done > /tmp/th.txt
$ time postmap -q 'foo50000' texthash:/tmp/th.txt
bar50000
real 0m0.072s
user 0m0.052s
sys 0m0.018s
Essentially all the time taken is loading the table, not doing the lookups:
$ for i in 50000; do printf "foo%d\n" $i; done | time postmap -q -
texthash:/tmp/th.txt | wc -l
1
0.05user 0.01system 0:00.07elapsed 95%CPU (0avgtext+0avgdata
26428maxresident)k
0inputs+0outputs (0major+4758minor)pagefaults 0swaps
$ for i in {50000..50100}; do printf "foo%d\n" $i; done | time postmap -q -
texthash:/tmp/th.txt | wc -l
0.05user 0.01system 0:00.07elapsed 95%CPU (0avgtext+0avgdata
26612maxresident)k
0inputs+0outputs (0major+4758minor)pagefaults 0swaps
101
That startup cost is amortised through process reuse ($max_use). If you
have more than sufficient RAM to store the table $default_process_limit
times (or whatever the master.cf process limit is for the service in
question), texthash may be fine.
That 26MB of RAM for "postmap" might be 2.6GB if 100 processes used
similarly large tables. That's probably the main barrier to using
texthash with sufficiently large source files.
--
Viktor. 🇺🇦 Слава Україні!
_______________________________________________
Postfix-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
