[CouchDB Version : 0.11]
[OS: RHEL 5.5]
Hi all,
We are seeing some odd behaviour with CouchDb not releasing files after it
has compacted them meaning we are seeing disk space continue to increase
until the service is restarted even though we have regular compaction /
cleanup.
Typical scenario : File system alert for disk space ..
df says 5.5.GB used ..
df -h | grep -P 'Filesystem|db'
Filesystem Size Used Avail Use% Mounted on
8.0G 5.5G 2.2G 72% /var/lib/db
.. but the files on the disk (and in Futon) say only 793MB is being used ..
du -sh /var/lib/db/*
793M /var/lib/db/couchdb
.. running lsof shows the files marked as deleted but still there (I'm
assuming beam still has some kind of handle still open to them) ..
lsof | grep -P 'COMMAND|/var/lib/db/couchdb/'
COMMAND PID USER FD TYPE DEVICE SIZE
NODE NAME
beam.smp 21352 couchdb 12u REG 253,10 4183
1210374
/var/lib/db/couchdb/.active_context_design/8efe9584a944f4b24b97f134f541d80f.view
beam.smp 21352 couchdb 16u REG 253,10 612039080
1204227 /var/lib/db/couchdb/active_context.couch (deleted)
beam.smp 21352 couchdb 20u REG 253,10 180699242
1204237 /var/lib/db/couchdb/active_context.couch (deleted)
beam.smp 21352 couchdb 21u REG 253,10 3133540
2023429 /var/lib/db/couchdb/thesaurus.couch (deleted)
beam.smp 21352 couchdb 25u REG 253,10 71422052
2023428 /var/lib/db/couchdb/active_context_schedule.couch (deleted)
beam.smp 21352 couchdb 26u REG 253,10 71372900
2056193 /var/lib/db/couchdb/active_context_schedule.couch
beam.smp 21352 couchdb 28u REG 253,10 621252717
1204238 /var/lib/db/couchdb/active_context.couch
beam.smp 21352 couchdb 29u REG 253,10 1740903
90113 /var/lib/db/couchdb/mapping_cache.couch
beam.smp 21352 couchdb 30u REG 253,10 3133540
1204253 /var/lib/db/couchdb/thesaurus.couch (deleted)
beam.smp 21352 couchdb 31u REG 253,10 2318439
1204240 /var/lib/db/couchdb/mappings.couch (deleted)
beam.smp 21352 couchdb 32u REG 253,10 983150
2004994
/var/lib/db/couchdb/.mappings_design/f9964c50b95c167c8ac3d8d3b4c76a4b.view
beam.smp 21352 couchdb 33u REG 253,10 3133540
2056194 /var/lib/db/couchdb/thesaurus.couch
beam.smp 21352 couchdb 34u REG 253,10 70742116
1204231 /var/lib/db/couchdb/active_context_schedule.couch (deleted)
beam.smp 21352 couchdb 44u REG 253,10 1740903
2023427 /var/lib/db/couchdb/mapping_cache.couch (deleted)
beam.smp 21352 couchdb 45u REG 253,10 65396836
1994756 /var/lib/db/couchdb/active_context_schedule.couch (deleted)
beam.smp 21352 couchdb 47u REG 253,10 71229540
1204243 /var/lib/db/couchdb/active_context_schedule.couch (deleted)
beam.smp 21352 couchdb 48u REG 253,10 240254078
1204226 /var/lib/db/couchdb/active_context.couch (deleted)
beam.smp 21352 couchdb 49u REG 253,10 2232423
1204241 /var/lib/db/couchdb/mappings.couch
beam.smp 21352 couchdb 51u REG 253,10 96903353
1210371
/var/lib/db/couchdb/.active_context_design/b406f7bb6d01b6d50324f14cde588324.view
beam.smp 21352 couchdb 52u REG 253,10 1192034
1249282
/var/lib/db/couchdb/.mapping_cache_design/8bbf80a6e1ca8f2c71ebc90c730e7190.view
beam.smp 21352 couchdb 53u REG 253,10 1622135
2004993
/var/lib/db/couchdb/.mappings_design/1ff69004b575837aae9d4147162197d4.view
beam.smp 21352 couchdb 54u REG 253,10 4172
1208327
/var/lib/db/couchdb/.thesaurus_design/73fe79b20ff06e341257fa78231ca87e.view
beam.smp 21352 couchdb 55u REG 253,10 13496483
1208325
/var/lib/db/couchdb/.thesaurus_design/fcc08d6e41c54c0d470c5984ea9eb6e3.view
beam.smp 21352 couchdb 56u REG 253,10 12959902
1208326
/var/lib/db/couchdb/.thesaurus_design/c06195da19f34ced37fa1ab91e663c73.view
beam.smp 21352 couchdb 57u REG 253,10 70254692
1204236 /var/lib/db/couchdb/active_context_schedule.couch (deleted)
beam.smp 21352 couchdb 58u REG 253,10 581730
1210373
/var/lib/db/couchdb/.active_context_schedule_design/c64c047dcff74ea2122356efb1042759.view
beam.smp 21352 couchdb 67u REG 253,10 1818727
1204247 /var/lib/db/couchdb/mapping_cache.couch (deleted)
beam.smp 21352 couchdb 78u REG 253,10 609905146
1204234 /var/lib/db/couchdb/active_context.couch (deleted)
beam.smp 21352 couchdb 80u REG 253,10 545919085
1994753 /var/lib/db/couchdb/active_context.couch (deleted)
beam.smp 21352 couchdb 81u REG 253,10 71032932
1204249 /var/lib/db/couchdb/active_context_schedule.couch (deleted)
beam.smp 21352 couchdb 84u REG 253,10 548866111
1204239 /var/lib/db/couchdb/active_context.couch (deleted)
beam.smp 21352 couchdb 85u REG 253,10 564612402
1204233 /var/lib/db/couchdb/active_context.couch (deleted)
beam.smp 21352 couchdb 87u REG 253,10 615996179
1204228 /var/lib/db/couchdb/active_context.couch (deleted)
beam.smp 21352 couchdb 102u REG 253,10 561111149
2023425 /var/lib/db/couchdb/active_context.couch (deleted)
Has anyone seen this behaviour before or knows of a way around this ?
Currently we have to restart the couchdb service in order to restore the
disk space.
Any help/advice is much appreciated.
Regards
Stephen
