Each forest will check the existing stands against the merge policy whenever a forest event happens. Typically this means a new stand on-disk: an in-memory stand filled up, or a merge finished.
At the end of an update-heavy process - reindexing for example - it's typical to have a large number of deleted fragments in the older stands. These will merge out eventually, but it's a function of the rate at which new stands arrive and their size. The amount of time doesn't matter. This behavior is a large driver of the "3x forest size for merge space" rule of thumb. Or, as Damon mentioned, you can force a merge. The https://github.com/mblakele/threx tool does this automatically when run as a scheduled task, periodically checking its own heuristics to decide whether or not a merge would be worthwhile. -- Mike On 25 Jun 2013, at 12:18 , Alex Milowski <[email protected]> wrote: > Right. I knew I could force a merge. > > Data is constantly being added to this database. I expected it to merge > automatically at some point. > > I rarely if ever do an update. Almost everything is insertion of new data. > > > On Tue, Jun 25, 2013 at 12:02 PM, Damon Feldman <[email protected]> > wrote: > Alex, > > > > Merges are only triggered by document updates, so when you start adding or > changing data the system will check merge policies and decide if it should > merge or not. In the meantime you can force a merge for the entire database > or particular forests on the admin GUI. > > > > Yours, > > Damon > > > > -- > > Damon Feldman > > Sr. Principal Consultant, MarkLogic > > > > From: [email protected] > [mailto:[email protected]] On Behalf Of Alex Milowski > Sent: Tuesday, June 25, 2013 2:16 PM > To: General Mark Logic Developer Discussion > Subject: [MarkLogic Dev General] Merge Policy / Large Delete > > > > After deleting a large amount of content, I have large forests (100+GB) on > disk with large amounts deleted fragments (60+ GB). I didn't notice any > merging going on and I expected this to clear itself up eventually. > > > > After waiting for quite awhile (many days), it didn't do anything by itself. > > > > The merge policy for the database is: > > > > merge priority = lower, max size = 0, min size = 1024, min ration = 2, > timestamp = 0 > > > > The database has three forests that are roughly balanced. As such, there is > 180GB of deleted fragments. > > > > I've gone in a manually requested the forests to merge, doing them one at a > time. > > > > Is there a reason why the merge didn't happen all by itself? > > > > -- > --Alex Milowski > "The excellence of grammar as a guide is proportional to the paucity of the > inflexions, i.e. to the degree of analysis effected by the language > considered." > > Bertrand Russell in a footnote of Principles of Mathematics > > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general > > > > > -- > --Alex Milowski > "The excellence of grammar as a guide is proportional to the paucity of the > inflexions, i.e. to the degree of analysis effected by the language > considered." > > Bertrand Russell in a footnote of Principles of Mathematics > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
