Precalculating index sizes is difficult because indexes store only unique values of tokens, then lists of pointers into the documents. So the total size depends on the total number of distinct values *and* the total document count.
Generally its not advisable to be running so low on disk that adding a single index will kill you ... if your that low then you are waiting for disaster. If you want to experiment with sizing I suggest using AWS and creating a server there where you can experiment without affecting production. Then when your done experimenting you can just kill it off and not pay any more. ----------------------------------------------------------------------------- David Lee Lead Engineer MarkLogic Corporation [email protected] Phone: +1 812-482-5224 Cell: +1 812-630-7622 www.marklogic.com<http://www.marklogic.com/> From: [email protected] [mailto:[email protected]] On Behalf Of RAVINDER MAAN Sent: Tuesday, January 28, 2014 2:00 PM To: [email protected] Subject: Re: [MarkLogic Dev General] General Digest, Vol 115, Issue 32 The problem is that whenever I add new index I cant estimate how much extra disk space is required for new index.If there were a way to get size of an existing index than before adding a new index I will be able to estimate(using existing index size) how much disk space I need for new indexes. So if there is not enough disk space I would know in advance that adding new index might cause problems or effect performance. On 28 Jan 2014 18:04, <[email protected]<mailto:[email protected]>> wrote: Send General mailing list submissions to [email protected]<mailto:[email protected]> To subscribe or unsubscribe via the World Wide Web, visit http://developer.marklogic.com/mailman/listinfo/general or, via email, send a message with subject or body 'help' to [email protected]<mailto:[email protected]> You can reach the person managing the list at [email protected]<mailto:[email protected]> When replying, please edit your Subject line so it is more specific than "Re: Contents of General digest..." Today's Topics: 1. Re: Size of an index (RAVINDER MAAN) (Paul M) ---------------------------------------------------------------------- Message: 1 Date: Mon, 27 Jan 2014 10:28:43 -0800 (PST) From: Paul M <[email protected]<mailto:[email protected]>> Subject: Re: [MarkLogic Dev General] Size of an index (RAVINDER MAAN) To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Message-ID: <[email protected]<mailto:[email protected]>> Content-Type: text/plain; charset="iso-8859-1" What tasks are you trying to accomplish that requires knowing the size of an index on disk? There may be? solutions other than definitive size of index. -Paul ________________________________ From: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> To: [email protected]<mailto:[email protected]> Sent: Saturday, January 25, 2014 3:00 PM Subject: General Digest, Vol 115, Issue 31 Send General mailing list submissions to ??? [email protected]<mailto:[email protected]> To subscribe or unsubscribe via the World Wide Web, visit ??? http://developer.marklogic.com/mailman/listinfo/general or, via email, send a message with subject or body 'help' to ??? [email protected]<mailto:[email protected]> You can reach the person managing the list at ??? [email protected]<mailto:[email protected]> When replying, please edit your Subject line so it is more specific than "Re: Contents of General digest..." Today's Topics: ? 1. Size of an index (RAVINDER MAAN) ? 2. Re: Size of an index (Danny Sokolsky) ? 3. Re: Size of an index (Geert J.) ? 4. Re: Size of an index (Michael Blakeley) ---------------------------------------------------------------------- Message: 1 Date: Fri, 24 Jan 2014 19:41:27 +0000 From: RAVINDER MAAN <[email protected]<mailto:[email protected]>> Subject: [MarkLogic Dev General] Size of an index To: [email protected]<mailto:[email protected]> Message-ID: ??? <CAFuTbAJ=m7g+YHYS-p3ztii16GK8Bxem=cs6wz0zrmkhqrb...@mail.gmail.com<mailto:[email protected]>> Content-Type: text/plain; charset="iso-8859-1" Hi all Is threre any way to find the size of an index on disk? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://developer.marklogic.com/pipermail/general/attachments/20140124/98c5aaaa/attachment-0001.html ------------------------------ Message: 2 Date: Fri, 24 Jan 2014 21:12:33 +0000 From: Danny Sokolsky <[email protected]<mailto:[email protected]>> Subject: Re: [MarkLogic Dev General] Size of an index To: MarkLogic Developer Discussion <[email protected]<mailto:[email protected]>> Message-ID: ??? <d4f697174c071d46b891b38bfc6eafa6149...@exchg10-be01.marklogic.com<mailto:d4f697174c071d46b891b38bfc6eafa6149...@exchg10-be01.marklogic.com>> Content-Type: text/plain; charset="us-ascii" You can look at the size of the database, either in the database status page or using xdmp:forest-status and/or xdmp:forest-counts for each forest in the database.? If you add an index or change index settings for a database, you can compare the sizes before and after (although merges will make the size somewhat variable). -Danny From: [email protected]<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of RAVINDER MAAN Sent: Friday, January 24, 2014 11:41 AM To: [email protected]<mailto:[email protected]> Subject: [MarkLogic Dev General] Size of an index Hi all Is threre any way to find the size of an index on disk? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://developer.marklogic.com/pipermail/general/attachments/20140124/19e06872/attachment-0001.html ------------------------------ Message: 3 Date: Fri, 24 Jan 2014 23:13:59 +0100 From: "Geert J." <[email protected]<mailto:[email protected]>> Subject: Re: [MarkLogic Dev General] Size of an index To: MarkLogic Developer Discussion <[email protected]<mailto:[email protected]>> Message-ID: <[email protected]<mailto:[email protected]>> Content-Type: text/plain; charset="iso-8859-1" Hi Ravinder, The forest files in the MarkLogic data directory have a direct relation to the indexes, so you could also simply look there. Most notably the ones with a hash at the beginning of their name. You can read the type directly from the filenames. Guessing which file belongs to which index is harder. There is a kind of logic, but that goes beyond my comprehension. Maybe someone from inside MarkLogic would be willing to reveal some details about this? Cheers, Geert *Van:* [email protected]<mailto:[email protected]> [mailto:general-<mailto:general-> [email protected]<mailto:[email protected]>] *Namens *Danny Sokolsky *Verzonden:* vrijdag 24 januari 2014 22:13 *Aan:* MarkLogic Developer Discussion *Onderwerp:* Re: [MarkLogic Dev General] Size of an index You can look at the size of the database, either in the database status page or using xdmp:forest-status and/or xdmp:forest-counts for each forest in the database.? If you add an index or change index settings for a database, you can compare the sizes before and after (although merges will make the size somewhat variable). -Danny *From:* [email protected]<mailto:[email protected]> [ mailto:[email protected]<mailto:[email protected]><[email protected]<mailto:[email protected]>>] *On Behalf Of *RAVINDER MAAN *Sent:* Friday, January 24, 2014 11:41 AM *To:* [email protected]<mailto:[email protected]> *Subject:* [MarkLogic Dev General] Size of an index Hi all Is threre any way to find the size of an index on disk? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://developer.marklogic.com/pipermail/general/attachments/20140124/6f77fbc4/attachment-0001.html ------------------------------ Message: 4 Date: Fri, 24 Jan 2014 14:28:04 -0800 From: Michael Blakeley <[email protected]<mailto:[email protected]>> Subject: Re: [MarkLogic Dev General] Size of an index To: MarkLogic Developer Discussion <[email protected]<mailto:[email protected]>> Message-ID: <[email protected]<mailto:[email protected]>> Content-Type: text/plain; charset=iso-8859-1 Those individual files are scalar indexes: element-value, geospatial, etc. All the default indexing goes into the hash-based universal index, which is even more opaque. -- Mike On 24 Jan 2014, at 14:13 , Geert J. <[email protected]<mailto:[email protected]>> wrote: > Hi Ravinder, >? > The forest files in the MarkLogic data directory have a direct relation to > the indexes, so you could also simply look there. Most notably the ones with > a hash at the beginning of their name. You can read the type directly from > the filenames. Guessing which file belongs to which index is harder. There is > a kind of logic, but that goes beyond my comprehension. Maybe someone from > inside MarkLogic would be willing to reveal some details about this? >? > Cheers, > Geert >? > Van: > [email protected]<mailto:[email protected]> > > [mailto:[email protected]<mailto:[email protected]>] > NamensDanny Sokolsky > Verzonden: vrijdag 24 januari 2014 22:13 > Aan: MarkLogic Developer Discussion > Onderwerp: Re: [MarkLogic Dev General] Size of an index >? > You can look at the size of the database, either in the database status page > or using xdmp:forest-status and/or xdmp:forest-counts for each forest in the > database.? If you add an index or change index settings for a database, you > can compare the sizes before and after (although merges will make the size > somewhat variable). >? > -Danny >? > From: > [email protected]<mailto:[email protected]> > > [mailto:[email protected]<mailto:[email protected]>] > On Behalf Of RAVINDER MAAN > Sent: Friday, January 24, 2014 11:41 AM > To: [email protected]<mailto:[email protected]> > Subject: [MarkLogic Dev General] Size of an index >? > Hi all > Is threre any way to find the size of an index on disk? > > Thanks > > _______________________________________________ > General mailing list > [email protected]<mailto:[email protected]> > http://developer.marklogic.com/mailman/listinfo/general ------------------------------ _______________________________________________ General mailing list [email protected]<mailto:[email protected]> http://developer.marklogic.com/mailman/listinfo/general End of General Digest, Vol 115, Issue 31 **************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: http://developer.marklogic.com/pipermail/general/attachments/20140127/bf908b2d/attachment-0001.html ------------------------------ _______________________________________________ General mailing list [email protected]<mailto:[email protected]> http://developer.marklogic.com/mailman/listinfo/general End of General Digest, Vol 115, Issue 32 ****************************************
_______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
