Hi misc@,
I have a program, a DNS server. It has a database to hold internal data.
Right now it's very inneficient in the way it uses memory. Let me explain.
If you know what an RRSET is it's all the RR records under one name. Like in
the OpenBSD.ORG name there is a SOA, NS, A RR's and so on. That's an RRSET.
Internally in my program I reserve space for an RRSET to all the maximum
records (around maximum 15 A RR's or so) and there is 15 or so types of
RR's in support with my program. As you can guess the RRSET becomes huge.
When designing the database for this I thought "hey what if I mmap this space
and only those parts that get written to it will actually be used". And that's
what I did. Only I run into a wall. With close to 30000 RRSET's I get this
in top:
Memory: Real: 1233M/3468M act/tot Free: 28G Cache: 1197M Swap: 0K/32G
PID USERNAME PRI NICE SIZE RES STATE WAIT TIME CPU COMMAND
74465 _ddd 2 0 13G 163M sleep/1 select 0:01 0.00% delphinusdn
31148 _ddd 2 0 13G 1116K sleep/1 select 0:01 0.00% delphinusdn
...
So you see the SIZE is 13 GB, but the actual resident memory is 163M. So now
I come to my final question:
Is there a way to claim back part of that 13G?
If not is there a way to overcommit? I googled on this and many people think
it's bad and I don't want to start a religious war right now.
In the worst case scenario I'll have to rewrite my underlying database, and I'm
a little lazy, if I do have to do that it won't be for another year+.
I looked at madvise(2) and I'm a little confused as to what I can use for this.
Best regards,
-peter
