H--o-l edited a comment on issue #1621: Black matter in shard files URL: https://github.com/apache/couchdb/issues/1621#issuecomment-426681587 Hi ! I'm working with @adrienverge and I continue to investigate the "black matter" issue. As @adrienverge said, shard files (`db.1520354435.couch`) constantly keep growing (by several kB per day), even though there are no requests done to them. This is a problem because we have *many* small, untouched databases. Compaction helps to free space on a regular basis, but it takes a bunch of cpu time. So we would like to understand what is happening in our `.couch` files. In a node of our cluster we monitored output of `strings /var/lib/couchd/shards/xxx/db.1520354435.couch`. In one night, it had ~800 new rows like this one: ``` kv_nodel ?_local/shard-sync-VNaRvSQnMLpUSzV8iZvcow-IOHL5NIdglwNQCT0DL_V0Ah seqa} ], target_uuid cef8fe0a2eac1f0f7af0d5e0c56fbbd8 7 historyh @;couchdb@node1l 142-216 -$!7 source x2a65d174d772861e4cd085e9e347729- $ timestamp!7| 2018-10-02T22:21:13.181557Zjjjj ~Jd TgKo_QqiB9XPEFr08HvUC2Q 98-214 M/Uf|e0dc03c296d3847efd92e964053e5f1b 7 P15:23:01.533200Zjjjjj kp_nodel ?_local/shard-sync-IOHL5NIdglwNQCT0DL_V0A-gKo_QqiB9XPEFr08HvUC2Qh -IOHLFj db_headera size_infob nilb cef8fe0a2eac1f0f7af0d5e0c56fbbd8l <couchdb@node2a ``` As we have ~34k dbs, multiplied by 8 shards by node, it quickly sum up and takes a lot of space and a lot of compaction time, which sometimes ends up saturating our disks and CPU. Can you advise us ?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
