To add: Current best practice is to do a hard commit with openSearcher=false every minute or so. AutoCommit is great for this. It shouldn't affect your overall indexing performance and it will constrain the transaction log.
- Mark On Feb 28, 2013, at 8:35 PM, Erick Erickson <erickerick...@gmail.com> wrote: > A new tlog gets created and the current one is closed on a hard commit > (openSearcher=true or false doesn't matter). The old one will be kept > around for a bit, I suspect if you'd done it a third time the first one > would go away. For all I know, the code might read if (tlog docs > 100) and > you may be hitting an off-by-one situation. > > But you're right. If you follow older practices and _never_ commit until > the end of a really long indexing process, you'll see these logs grow huge. > Even worse, anytime they're replayed they'll then re-index a lot of > documents. > > So I'd try the experiment over again with 110 docs.... But I can say > they're getting purged on my machine quite regularly with some stress > testing I'm doing. > > Best, > Erick > > > On Thu, Feb 28, 2013 at 7:59 AM, adityab <aditya_ba...@yahoo.com> wrote: > >> whats the life cycle of a tlog file. Is it purged after commit (even with >> soft commit) ? >> I posted 100 docs to solr (standalone) did hard commit. Observed a new tlog >> file is created. >> re-posted the same 100 docs and did hard commit. Observed a new tlog file >> is >> created. Old one still exists. >> >> When do they get purged. Concern is we have at least 20K docs published >> every 2hrs so need to understand if its safe to put them in a different >> location where we can have a script to purge old files at regular interval. >> >> thanks >> >> >> >> >> -- >> View this message in context: >> http://lucene.472066.n3.nabble.com/A-few-operations-questions-about-the-tlog-UpdateLog-tp4042560p4043618.html >> Sent from the Solr - User mailing list archive at Nabble.com. >>
