Thanks, Wayne and Mike. If we ever need to get rid of the last few, we now know how.
-Brent -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Michael Blakeley Sent: Friday, September 26, 2014 2:13 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] unable to merge all deleted fragments out of the database Thanks, Wayne. It sounds like Brent's problem would be solved if the merge buttons on the database and forest pages used xdmp:request-timestamp() automatically. Of course that also might make it easier to produce the original problems.... -- Mike On 26 Sep 2014, at 10:44 , Wayne Feick <[email protected]> wrote: > If you pass an explicit timestamp to xdmp:merge() (e.g. the return > value of xdmp:request-timestamp()) the server will discard all deleted > fragments. > > The hour window was added to avoid XDMP-OLDSTAMP errors that had > cropped up in some of our stress testing, most commonly for replica > databases, but also causing transaction retries for non-replica databases. > > We've done some tuning of the change since then (e.g. not holding on > to the last hour of deleted fragments after a reindex), and we may do > some further tuning so this is less surprising to people. > > Wayne. > > > > On 09/26/2014 09:41 AM, Michael Blakeley wrote: >>> From what I've seen there now seems to be an offset from the merge >>> timestamp. I think it's about an hour. No idea why this was introduced, but >>> try waiting an hour and then merge again. >> >> -- Mike >> >> On 26 Sep 2014, at 08:33 , Brent Hartwig <[email protected]> wrote: >> >>> Hello, and Happy Friday! >>> >>> While attempting to calculate an expansion ratio, a client and I initiated >>> manual merges of a ML 7.0-3 database, then recorded the size. We did see >>> the database size decrease, as well as merge-related entries in the log; >>> however, the number of deleted fragments never reached zero. This is >>> different than I remember from previous versions. A scan of release notes >>> for versions 5, 6, and 7 didn't turn up anything. >>> >>> The merge timestamp is zero, and the system would have been relatively >>> quiet when merge was requested. >>> >>> The number of remaining deleted fragments was relatively low, and did >>> decrease after some merges, just never to zero. Given the size of our >>> documents, these few deleted fragments would not have significantly skewed >>> the results. Nonetheless, curiosity has the better of me. >>> >>> After the second test of the day, there were ~65K documents taking up ~150 >>> MB. There were 879 deleted fragments after a merge. >>> >>> Might this be a display bug, or does the merge process ultimately decide >>> just how far to go? >>> >>> Thanks much. >>> >>> -Brent >>> >>> Brent Hartwig, Solutions Architect | RSI Content Solutions >>> _______________________________________________ >>> General mailing list >>> [email protected] >>> http://developer.marklogic.com/mailman/listinfo/general >> _______________________________________________ >> General mailing list >> [email protected] >> http://developer.marklogic.com/mailman/listinfo/general > > -- > Wayne Feick > Principal Engineer > MarkLogic Corporation > [email protected] > Phone: +1 650 655 2378 > www.marklogic.com > > This e-mail and any accompanying attachments are confidential. The > information is intended solely for the use of the individual to whom it is > addressed. Any review, disclosure, copying, distribution, or use of this > e-mail communication by others is strictly prohibited. If you are not the > intended recipient, please notify us immediately by returning this message to > the sender and delete all copies. Thank you for your cooperation. > > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general > _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
