Comments inline: On Wed, Jun 10, 2015 at 11:45 AM, Shawn Heisey <apa...@elyograg.org> wrote:
> On 6/9/2015 10:46 PM, Shalin Shekhar Mangar wrote: > > That comment about commitReserveDuration is not accurate. It has nothing > > to do with the size of the index. It is the time for which commits are > > reserved even if no request has accessed them. A commit and its related > > files will never be deleted as long as a request is fetching them. The > > only reason why you would increase commitReserveDuration is when you > > have a cross datacenter repeater setup and requests to fetch index files > > are made more than 10 seconds apart by the repeater (due to whatever > > reasons). Heavy GC can also cause the commitReserveDuration to be > exceeded. > > I've reverted the reference guide change. Once I can figure out exactly > what happens here, I'll work on the wiki page as well and figure out > whether the ref guide needs any new edits. > > I'm finding the code very difficult to follow, which is pretty normal > for me -- there are a lot of inheritance levels and it takes a lot of > skill and time to decipher it. > > I'd like to write something in the docs that explains in simple terms > when a user would actually need to increase this value, so I need to > understand what events will reset the deletion timer? > > Is it separate calls to the replication handler, to get a file list, > request a file, etc? If it is, then if the index is dozens of gigabytes > and optimized to a single segment, there will be multiple files that > take a lot longer than the 10 second default to transfer. > If the time between multiple API calls such "filelist", "filecontent" exceeds 10 seconds then the commit is released (i.e. free to be deleted). > If the timer is continually reset throughout the actual transfer of a > file, then this is a lot less brittle than I had originally suspected. > I can't find any evidence that this is how it works, but as I said > before, the code is hard to follow. > When the actual transfer of a file starts, then the whole commit point is "saved" i.e. it won't be released until the entire file transfer is complete (this is different from a "reserve" which is time based). At the end of the transfer of a single file, the reserve is again extended by the commitReserveDuration to allow for the next file to be fetched. In my practical experience, increasing the commitReserveDuration in a well-behaved Solr install has only been necessary for repeater setups. You can see this code in ReplicationHandler.DirectoryFileStream.write() method. > > Thanks, > Shawn > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > -- Regards, Shalin Shekhar Mangar.
