Re: Disk full
Indeed puzzling.
If you delete the database (DELETE /dbname) and if this succeeds (2xx response)
then all of the db data is deleted fully. If you think you're seeing data
persisting after deletion you have a problem (the delete is failing, or you're
not really deleting the db, or something extremely strange is happening).
Another cause of invisible bloat would be failed writes (especially ones with
attachment data) as we'll write the data as we go but if the write then fails
that leaves the partial write in the file with nothing pointing back at it.
Compaction will clean that up, of course.
Compaction is essential in practically all cases. You could maybe get away with
disabling it if you don't create, update or delete a document but even in that
case the files will grow on restart (and perhaps when the db is closed and
reopened?) as we'll append a new database footer.
--
Robert Samuel Newson
rnew...@apache.org
On Thu, 2 May 2019, at 18:02, Adam Kocoloski wrote:
> Hi Willem,
>
> Good question. CouchDB has a 100% copy-on-write storage engine,
> including for all updates to btree nodes, etc. so any updates to the
> database will necessarily increase the file size before compaction.
> Looking at your info I don’t see a heavy source of updates, so it is a
> little puzzling.
>
> Adam
>
>
> > On May 2, 2019, at 12:53 PM, Willem Bison wrote:
> >
> > Hi Adam,
> >
> > I ran "POST compact" on the DB mentioned in my post and 'disk_size' went
> > from 729884227 (yes, it had grown that much in 1 hour !?) to 1275480.
> >
> > Wow.
> >
> > I disabled compacting because I thought it was useless in our case since
> > the db's and the docs are so small. I do wonder how it is possible for a db
> > to grow so much when its being deleted several times a week. What is all
> > the 'air' ?
> >
> > On Thu, 2 May 2019 at 18:31, Adam Kocoloski wrote:
> >
> >> Hi Willem,
> >>
> >> Compaction would certainly reduce your storage space. You have such a
> >> small number of documents in these databases that it would be a fast
> >> operation. Did you try it and run into issues?
> >>
> >> Changing cluster.q shouldn’t affect the overall storage consumption.
> >>
> >> Adam
> >>
> >>> On May 2, 2019, at 12:15 PM, Willem Bison wrote:
> >>>
> >>> Hi,
> >>>
> >>> Our CouchDb 2.3.1 standalone server (AWS Ubuntu 18.04) is using a lot of
> >>> disk space, so much so that it regularly causes a disk full and a crash.
> >>>
> >>> The server contains approximately 100 databases each with a reported
> >>> (Fauxton) size of less than 2.5Mb and less than 250 docs. Yesterday the
> >>> 'shards' folders combined exceeded a total 14G causing the server to
> >> crash.
> >>>
> >>> The server is configured with
> >>> cluster.n = 1 and
> >>> cluster.q = 8
> >>> because that was suggested during setup.
> >>>
> >>> When I write this the 'shards' folders look like this:
> >>> /var/lib/couchdb/shards# du -hs *
> >>> 869M -1fff
> >>> 1.4G 2000-3fff
> >>> 207M 4000-5fff
> >>> 620M 6000-7fff
> >>> 446M 8000-9fff
> >>> 458M a000-bfff
> >>> 400M c000-dfff
> >>> 549M e000-
> >>>
> >>> One of the largest files is this:
> >>> curl localhost:5984/xxx_1590
> >>> {
> >>> "db_name": "xxx_1590",
> >>> "purge_seq":
> >>>
> >> "0-g1FTeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFNSApBMqv___39WIgMedXksQJKhAUgBlc4nRu0DiFoC5iYpgOy3J9L-BRAz9-NXm8iQJE_YYgeQxfFEWnwAYvF9oNosADncXo4",
> >>> "update_seq":
> >>>
> >> "3132-g1FWeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFMiQ5L8sxI18ChKUgCSSfYgdUkMDNw1-JQ6gJTGg42UxacuAaSuHqxOAo-6PBYgydAApIBK52clchNUuwCidn9Wog5BtQcgau9nJQoTVPsAohboXsksAJuwX9Y",
> >>> "sizes": {
> >>> "file": 595928643,
> >>> "external": 462778,
> >>> "active": 1393380
> >>> },
> >>> "other": {
> >>> "data_size": 462778
> >>> },
> >>> "doc_del_count": 0,
> >>> "doc_count": 74,
> >>> "disk_size": 595928643,
> >>> "disk_format_version": 7,
> >>> "data_size": 1393380,
> >>> "compact_running": false,
> >>> "cluster": {
> >>> "q": 8,
> >>> "n": 1,
> >>> "w": 1,
> >>> "r": 1
> >>> },
> >>> "instance_start_time": "0"
> >>> }
> >>>
> >>> curl localhost:5984/xxx_1590/_local_docs
> >>> {"total_rows":null,"offset":null,"rows":[
> >>>
> >> {"id":"_local/189d9109518d1a2167b06ca9639af5f2ba16f0a5","key":"_local/189d9109518d1a2167b06ca9639af5f2ba16f0a5","value":{"rev":"0-3022"}},
> >>>
> >> {"id":"_local/7b3e0d929201afcea44b237b5b3e86b35ff924c6","key":"_local/7b3e0d929201afcea44b237b5b3e86b35ff924c6","value":{"rev":"0-18"}},
> >>>
> >> {"id":"_local/7da4a2aaebc84d01ba0e2906ac0fcb82d96bfe05","key":"_local/7da4a2aaebc84d01ba0e2906ac0fcb82d96bfe05","value":{"rev":"0-3749"}},
> >>>
> >> {"id":"_local/9619b06f20d26b076e4060d050dc8e3bde878920","key":"_local/9619b06f20d26b076e4060d050dc8e3bde878920","value":{"rev":"0-172"}}
> >>> ]}
> >>>
> >>> Each database push/pull replicates
Re: Disk full
Hi Willem,
Good question. CouchDB has a 100% copy-on-write storage engine, including for
all updates to btree nodes, etc. so any updates to the database will
necessarily increase the file size before compaction. Looking at your info I
don’t see a heavy source of updates, so it is a little puzzling.
Adam
> On May 2, 2019, at 12:53 PM, Willem Bison wrote:
>
> Hi Adam,
>
> I ran "POST compact" on the DB mentioned in my post and 'disk_size' went
> from 729884227 (yes, it had grown that much in 1 hour !?) to 1275480.
>
> Wow.
>
> I disabled compacting because I thought it was useless in our case since
> the db's and the docs are so small. I do wonder how it is possible for a db
> to grow so much when its being deleted several times a week. What is all
> the 'air' ?
>
> On Thu, 2 May 2019 at 18:31, Adam Kocoloski wrote:
>
>> Hi Willem,
>>
>> Compaction would certainly reduce your storage space. You have such a
>> small number of documents in these databases that it would be a fast
>> operation. Did you try it and run into issues?
>>
>> Changing cluster.q shouldn’t affect the overall storage consumption.
>>
>> Adam
>>
>>> On May 2, 2019, at 12:15 PM, Willem Bison wrote:
>>>
>>> Hi,
>>>
>>> Our CouchDb 2.3.1 standalone server (AWS Ubuntu 18.04) is using a lot of
>>> disk space, so much so that it regularly causes a disk full and a crash.
>>>
>>> The server contains approximately 100 databases each with a reported
>>> (Fauxton) size of less than 2.5Mb and less than 250 docs. Yesterday the
>>> 'shards' folders combined exceeded a total 14G causing the server to
>> crash.
>>>
>>> The server is configured with
>>> cluster.n = 1 and
>>> cluster.q = 8
>>> because that was suggested during setup.
>>>
>>> When I write this the 'shards' folders look like this:
>>> /var/lib/couchdb/shards# du -hs *
>>> 869M -1fff
>>> 1.4G 2000-3fff
>>> 207M 4000-5fff
>>> 620M 6000-7fff
>>> 446M 8000-9fff
>>> 458M a000-bfff
>>> 400M c000-dfff
>>> 549M e000-
>>>
>>> One of the largest files is this:
>>> curl localhost:5984/xxx_1590
>>> {
>>> "db_name": "xxx_1590",
>>> "purge_seq":
>>>
>> "0-g1FTeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFNSApBMqv___39WIgMedXksQJKhAUgBlc4nRu0DiFoC5iYpgOy3J9L-BRAz9-NXm8iQJE_YYgeQxfFEWnwAYvF9oNosADncXo4",
>>> "update_seq":
>>>
>> "3132-g1FWeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFMiQ5L8sxI18ChKUgCSSfYgdUkMDNw1-JQ6gJTGg42UxacuAaSuHqxOAo-6PBYgydAApIBK52clchNUuwCidn9Wog5BtQcgau9nJQoTVPsAohboXsksAJuwX9Y",
>>> "sizes": {
>>> "file": 595928643,
>>> "external": 462778,
>>> "active": 1393380
>>> },
>>> "other": {
>>> "data_size": 462778
>>> },
>>> "doc_del_count": 0,
>>> "doc_count": 74,
>>> "disk_size": 595928643,
>>> "disk_format_version": 7,
>>> "data_size": 1393380,
>>> "compact_running": false,
>>> "cluster": {
>>> "q": 8,
>>> "n": 1,
>>> "w": 1,
>>> "r": 1
>>> },
>>> "instance_start_time": "0"
>>> }
>>>
>>> curl localhost:5984/xxx_1590/_local_docs
>>> {"total_rows":null,"offset":null,"rows":[
>>>
>> {"id":"_local/189d9109518d1a2167b06ca9639af5f2ba16f0a5","key":"_local/189d9109518d1a2167b06ca9639af5f2ba16f0a5","value":{"rev":"0-3022"}},
>>>
>> {"id":"_local/7b3e0d929201afcea44b237b5b3e86b35ff924c6","key":"_local/7b3e0d929201afcea44b237b5b3e86b35ff924c6","value":{"rev":"0-18"}},
>>>
>> {"id":"_local/7da4a2aaebc84d01ba0e2906ac0fcb82d96bfe05","key":"_local/7da4a2aaebc84d01ba0e2906ac0fcb82d96bfe05","value":{"rev":"0-3749"}},
>>>
>> {"id":"_local/9619b06f20d26b076e4060d050dc8e3bde878920","key":"_local/9619b06f20d26b076e4060d050dc8e3bde878920","value":{"rev":"0-172"}}
>>> ]}
>>>
>>> Each database push/pull replicates with a small number of clients (< 10).
>>> Most of the documents contain orders that are shortlived. We throw away
>> all
>>> db's 3 times a week as a brute force purge.
>>> Compacting has been disabled because it takes too much cpu and was
>>> considered useless in our case (small db's, purging).
>>>
>>> I read this:
>>> https://github.com/apache/couchdb/issues/1621
>>> but I'm not sure how it helps me.
>>>
>>> These are my questions:
>>> How is it possible that such a small db occupies so much space?
>>> What can I do to reduce this?
>>> Would changing 'cluster.q' have any effect or would the same amount of
>>> bytes be used in less folders? (am I correct in assuming that cluster.q
>>> 1
>>> is pointless in standalone configuration?)
>>>
>>> Thanks!
>>> Willem
>>
>>
Re: Disk full
Hi Adam,
I ran "POST compact" on the DB mentioned in my post and 'disk_size' went
from 729884227 (yes, it had grown that much in 1 hour !?) to 1275480.
Wow.
I disabled compacting because I thought it was useless in our case since
the db's and the docs are so small. I do wonder how it is possible for a db
to grow so much when its being deleted several times a week. What is all
the 'air' ?
On Thu, 2 May 2019 at 18:31, Adam Kocoloski wrote:
> Hi Willem,
>
> Compaction would certainly reduce your storage space. You have such a
> small number of documents in these databases that it would be a fast
> operation. Did you try it and run into issues?
>
> Changing cluster.q shouldn’t affect the overall storage consumption.
>
> Adam
>
> > On May 2, 2019, at 12:15 PM, Willem Bison wrote:
> >
> > Hi,
> >
> > Our CouchDb 2.3.1 standalone server (AWS Ubuntu 18.04) is using a lot of
> > disk space, so much so that it regularly causes a disk full and a crash.
> >
> > The server contains approximately 100 databases each with a reported
> > (Fauxton) size of less than 2.5Mb and less than 250 docs. Yesterday the
> > 'shards' folders combined exceeded a total 14G causing the server to
> crash.
> >
> > The server is configured with
> > cluster.n = 1 and
> > cluster.q = 8
> > because that was suggested during setup.
> >
> > When I write this the 'shards' folders look like this:
> > /var/lib/couchdb/shards# du -hs *
> > 869M -1fff
> > 1.4G 2000-3fff
> > 207M 4000-5fff
> > 620M 6000-7fff
> > 446M 8000-9fff
> > 458M a000-bfff
> > 400M c000-dfff
> > 549M e000-
> >
> > One of the largest files is this:
> > curl localhost:5984/xxx_1590
> > {
> >"db_name": "xxx_1590",
> >"purge_seq":
> >
> "0-g1FTeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFNSApBMqv___39WIgMedXksQJKhAUgBlc4nRu0DiFoC5iYpgOy3J9L-BRAz9-NXm8iQJE_YYgeQxfFEWnwAYvF9oNosADncXo4",
> >"update_seq":
> >
> "3132-g1FWeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFMiQ5L8sxI18ChKUgCSSfYgdUkMDNw1-JQ6gJTGg42UxacuAaSuHqxOAo-6PBYgydAApIBK52clchNUuwCidn9Wog5BtQcgau9nJQoTVPsAohboXsksAJuwX9Y",
> >"sizes": {
> >"file": 595928643,
> >"external": 462778,
> >"active": 1393380
> >},
> >"other": {
> >"data_size": 462778
> >},
> >"doc_del_count": 0,
> >"doc_count": 74,
> >"disk_size": 595928643,
> >"disk_format_version": 7,
> >"data_size": 1393380,
> >"compact_running": false,
> >"cluster": {
> >"q": 8,
> >"n": 1,
> >"w": 1,
> >"r": 1
> >},
> >"instance_start_time": "0"
> > }
> >
> > curl localhost:5984/xxx_1590/_local_docs
> > {"total_rows":null,"offset":null,"rows":[
> >
> {"id":"_local/189d9109518d1a2167b06ca9639af5f2ba16f0a5","key":"_local/189d9109518d1a2167b06ca9639af5f2ba16f0a5","value":{"rev":"0-3022"}},
> >
> {"id":"_local/7b3e0d929201afcea44b237b5b3e86b35ff924c6","key":"_local/7b3e0d929201afcea44b237b5b3e86b35ff924c6","value":{"rev":"0-18"}},
> >
> {"id":"_local/7da4a2aaebc84d01ba0e2906ac0fcb82d96bfe05","key":"_local/7da4a2aaebc84d01ba0e2906ac0fcb82d96bfe05","value":{"rev":"0-3749"}},
> >
> {"id":"_local/9619b06f20d26b076e4060d050dc8e3bde878920","key":"_local/9619b06f20d26b076e4060d050dc8e3bde878920","value":{"rev":"0-172"}}
> > ]}
> >
> > Each database push/pull replicates with a small number of clients (< 10).
> > Most of the documents contain orders that are shortlived. We throw away
> all
> > db's 3 times a week as a brute force purge.
> > Compacting has been disabled because it takes too much cpu and was
> > considered useless in our case (small db's, purging).
> >
> > I read this:
> > https://github.com/apache/couchdb/issues/1621
> > but I'm not sure how it helps me.
> >
> > These are my questions:
> > How is it possible that such a small db occupies so much space?
> > What can I do to reduce this?
> > Would changing 'cluster.q' have any effect or would the same amount of
> > bytes be used in less folders? (am I correct in assuming that cluster.q
> > 1
> > is pointless in standalone configuration?)
> >
> > Thanks!
> > Willem
>
>
Re: Disk full
Hi Willem,
Compaction would certainly reduce your storage space. You have such a small
number of documents in these databases that it would be a fast operation. Did
you try it and run into issues?
Changing cluster.q shouldn’t affect the overall storage consumption.
Adam
> On May 2, 2019, at 12:15 PM, Willem Bison wrote:
>
> Hi,
>
> Our CouchDb 2.3.1 standalone server (AWS Ubuntu 18.04) is using a lot of
> disk space, so much so that it regularly causes a disk full and a crash.
>
> The server contains approximately 100 databases each with a reported
> (Fauxton) size of less than 2.5Mb and less than 250 docs. Yesterday the
> 'shards' folders combined exceeded a total 14G causing the server to crash.
>
> The server is configured with
> cluster.n = 1 and
> cluster.q = 8
> because that was suggested during setup.
>
> When I write this the 'shards' folders look like this:
> /var/lib/couchdb/shards# du -hs *
> 869M -1fff
> 1.4G 2000-3fff
> 207M 4000-5fff
> 620M 6000-7fff
> 446M 8000-9fff
> 458M a000-bfff
> 400M c000-dfff
> 549M e000-
>
> One of the largest files is this:
> curl localhost:5984/xxx_1590
> {
>"db_name": "xxx_1590",
>"purge_seq":
> "0-g1FTeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFNSApBMqv___39WIgMedXksQJKhAUgBlc4nRu0DiFoC5iYpgOy3J9L-BRAz9-NXm8iQJE_YYgeQxfFEWnwAYvF9oNosADncXo4",
>"update_seq":
> "3132-g1FWeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFMiQ5L8sxI18ChKUgCSSfYgdUkMDNw1-JQ6gJTGg42UxacuAaSuHqxOAo-6PBYgydAApIBK52clchNUuwCidn9Wog5BtQcgau9nJQoTVPsAohboXsksAJuwX9Y",
>"sizes": {
>"file": 595928643,
>"external": 462778,
>"active": 1393380
>},
>"other": {
>"data_size": 462778
>},
>"doc_del_count": 0,
>"doc_count": 74,
>"disk_size": 595928643,
>"disk_format_version": 7,
>"data_size": 1393380,
>"compact_running": false,
>"cluster": {
>"q": 8,
>"n": 1,
>"w": 1,
>"r": 1
>},
>"instance_start_time": "0"
> }
>
> curl localhost:5984/xxx_1590/_local_docs
> {"total_rows":null,"offset":null,"rows":[
> {"id":"_local/189d9109518d1a2167b06ca9639af5f2ba16f0a5","key":"_local/189d9109518d1a2167b06ca9639af5f2ba16f0a5","value":{"rev":"0-3022"}},
> {"id":"_local/7b3e0d929201afcea44b237b5b3e86b35ff924c6","key":"_local/7b3e0d929201afcea44b237b5b3e86b35ff924c6","value":{"rev":"0-18"}},
> {"id":"_local/7da4a2aaebc84d01ba0e2906ac0fcb82d96bfe05","key":"_local/7da4a2aaebc84d01ba0e2906ac0fcb82d96bfe05","value":{"rev":"0-3749"}},
> {"id":"_local/9619b06f20d26b076e4060d050dc8e3bde878920","key":"_local/9619b06f20d26b076e4060d050dc8e3bde878920","value":{"rev":"0-172"}}
> ]}
>
> Each database push/pull replicates with a small number of clients (< 10).
> Most of the documents contain orders that are shortlived. We throw away all
> db's 3 times a week as a brute force purge.
> Compacting has been disabled because it takes too much cpu and was
> considered useless in our case (small db's, purging).
>
> I read this:
> https://github.com/apache/couchdb/issues/1621
> but I'm not sure how it helps me.
>
> These are my questions:
> How is it possible that such a small db occupies so much space?
> What can I do to reduce this?
> Would changing 'cluster.q' have any effect or would the same amount of
> bytes be used in less folders? (am I correct in assuming that cluster.q > 1
> is pointless in standalone configuration?)
>
> Thanks!
> Willem
Disk full
Hi,
Our CouchDb 2.3.1 standalone server (AWS Ubuntu 18.04) is using a lot of
disk space, so much so that it regularly causes a disk full and a crash.
The server contains approximately 100 databases each with a reported
(Fauxton) size of less than 2.5Mb and less than 250 docs. Yesterday the
'shards' folders combined exceeded a total 14G causing the server to crash.
The server is configured with
cluster.n = 1 and
cluster.q = 8
because that was suggested during setup.
When I write this the 'shards' folders look like this:
/var/lib/couchdb/shards# du -hs *
869M -1fff
1.4G 2000-3fff
207M 4000-5fff
620M 6000-7fff
446M 8000-9fff
458M a000-bfff
400M c000-dfff
549M e000-
One of the largest files is this:
curl localhost:5984/xxx_1590
{
"db_name": "xxx_1590",
"purge_seq":
"0-g1FTeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFNSApBMqv___39WIgMedXksQJKhAUgBlc4nRu0DiFoC5iYpgOy3J9L-BRAz9-NXm8iQJE_YYgeQxfFEWnwAYvF9oNosADncXo4",
"update_seq":
"3132-g1FWeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFMiQ5L8sxI18ChKUgCSSfYgdUkMDNw1-JQ6gJTGg42UxacuAaSuHqxOAo-6PBYgydAApIBK52clchNUuwCidn9Wog5BtQcgau9nJQoTVPsAohboXsksAJuwX9Y",
"sizes": {
"file": 595928643,
"external": 462778,
"active": 1393380
},
"other": {
"data_size": 462778
},
"doc_del_count": 0,
"doc_count": 74,
"disk_size": 595928643,
"disk_format_version": 7,
"data_size": 1393380,
"compact_running": false,
"cluster": {
"q": 8,
"n": 1,
"w": 1,
"r": 1
},
"instance_start_time": "0"
}
curl localhost:5984/xxx_1590/_local_docs
{"total_rows":null,"offset":null,"rows":[
{"id":"_local/189d9109518d1a2167b06ca9639af5f2ba16f0a5","key":"_local/189d9109518d1a2167b06ca9639af5f2ba16f0a5","value":{"rev":"0-3022"}},
{"id":"_local/7b3e0d929201afcea44b237b5b3e86b35ff924c6","key":"_local/7b3e0d929201afcea44b237b5b3e86b35ff924c6","value":{"rev":"0-18"}},
{"id":"_local/7da4a2aaebc84d01ba0e2906ac0fcb82d96bfe05","key":"_local/7da4a2aaebc84d01ba0e2906ac0fcb82d96bfe05","value":{"rev":"0-3749"}},
{"id":"_local/9619b06f20d26b076e4060d050dc8e3bde878920","key":"_local/9619b06f20d26b076e4060d050dc8e3bde878920","value":{"rev":"0-172"}}
]}
Each database push/pull replicates with a small number of clients (< 10).
Most of the documents contain orders that are shortlived. We throw away all
db's 3 times a week as a brute force purge.
Compacting has been disabled because it takes too much cpu and was
considered useless in our case (small db's, purging).
I read this:
https://github.com/apache/couchdb/issues/1621
but I'm not sure how it helps me.
These are my questions:
How is it possible that such a small db occupies so much space?
What can I do to reduce this?
Would changing 'cluster.q' have any effect or would the same amount of
bytes be used in less folders? (am I correct in assuming that cluster.q > 1
is pointless in standalone configuration?)
Thanks!
Willem
Re: Shards-directory still big after deleting big database in Fauxton
Glad this worked out. Quick tip then, unless you run this on an 8-core (or more) machine, you might want to look into reducing your q for this database. q=2 or $num_cores is a good rule of thumb. You can use our couch-continuum tool to migrate an existing db: https://npmjs.com/couch-continuum Cheers Jan — > On 2. May 2019, at 14:17, Frank Röhm wrote: > > OK, I found it. > In the 8 shards subdirectories (from -1fff to e000-) > there was still 8 frwiki directories (frwiki.1510609658.couch) with each 5 GB. > I deleted them with: > > find . -name frwiki.1510609658.couch -delete > > from the shards dir and gone they are. > Hopefully it won’t affect my CouchDB, but as I heard this is very robust ;) > > I think I can stick to the v2.x now, no need to downgrade now, ouff. > > frank > >> Am 02.05.2019 um 07:25 schrieb Joan Touzet : >> >> Look for a .deleted directory under your data/ directory. The files may not >> have been deleted but moved aside due to the enable_database_recovery >> setting, or because the DB was still in use when you restarted CouchDB. >> >> Another useful command is: >> >> $ du -sh /opt/couchdb/data/* >> >> which should tell you where the storage is being used. Does this show >> anything useful to you? >> >> -Joan >> >>> On 2019-05-01 2:22 p.m., Frank Walter wrote: >>> Hello >>> I have CouchDB v2.3.1 (on Ubuntu 14.04) and I use it only for creating >>> Wikipedia databases with mwscrape. >>> My shards folder was too big, over 50 GB big, so I deleted one big db >>> (frwiki) which had 32 GB in Fauxton. That db is gone now. >>> After this, I thought now my shards folder should be about 20 GB but it >>> is still 52 GB. >>> I don't find any documentation about that in the CouchDB Doc. >>> I restarted CouchDB (/etc/init.d/couchdb restart) but nothing changes. >>> How can I reduce the size of shards? How can I get rid of this ghost-db? >>> My next step would be, if I cannot solve this issue, to uninstall >>> CouchDB 2.x and reinstall 1.x, because I dont need that feature of >>> cluster server anyway. I see only inconvenience for my use. >>> Thanks >>> frank >
Re: Shards-directory still big after deleting big database in Fauxton
OK, I found it. In the 8 shards subdirectories (from -1fff to e000-) there was still 8 frwiki directories (frwiki.1510609658.couch) with each 5 GB. I deleted them with: find . -name frwiki.1510609658.couch -delete from the shards dir and gone they are. Hopefully it won’t affect my CouchDB, but as I heard this is very robust ;) I think I can stick to the v2.x now, no need to downgrade now, ouff. frank > Am 02.05.2019 um 07:25 schrieb Joan Touzet : > > Look for a .deleted directory under your data/ directory. The files may not > have been deleted but moved aside due to the enable_database_recovery > setting, or because the DB was still in use when you restarted CouchDB. > > Another useful command is: > > $ du -sh /opt/couchdb/data/* > > which should tell you where the storage is being used. Does this show > anything useful to you? > > -Joan > > On 2019-05-01 2:22 p.m., Frank Walter wrote: >> Hello >> I have CouchDB v2.3.1 (on Ubuntu 14.04) and I use it only for creating >> Wikipedia databases with mwscrape. >> My shards folder was too big, over 50 GB big, so I deleted one big db >> (frwiki) which had 32 GB in Fauxton. That db is gone now. >> After this, I thought now my shards folder should be about 20 GB but it >> is still 52 GB. >> I don't find any documentation about that in the CouchDB Doc. >> I restarted CouchDB (/etc/init.d/couchdb restart) but nothing changes. >> How can I reduce the size of shards? How can I get rid of this ghost-db? >> My next step would be, if I cannot solve this issue, to uninstall >> CouchDB 2.x and reinstall 1.x, because I dont need that feature of >> cluster server anyway. I see only inconvenience for my use. >> Thanks >> frank
