Re: Solr 4.2 Startup Detects Corrupt Log And is Really Slow to Start

2013-04-22 Thread Umesh Prasad
Sorry for late reply. I was trying to change our indexing pipeline and do explicit intermediate commits for each core. That turned out to be a bit more work that I have time for. So, I do want to explore hard commits. I tried solr-host:port/solr/core/*update?commit=true* . But there is no

Re: Solr 4.2 Startup Detects Corrupt Log And is Really Slow to Start

2013-04-17 Thread Erick Erickson
How big are you transaction logs? They can be replayed on startup. They are truncated and a new one started when you do a hard commit (openSearcher true or false doesn't matter). So a quick test of this theory would be to just stop your indexing process, issue a hard commit on all your cores and

Re: Solr 4.2 Startup Detects Corrupt Log And is Really Slow to Start

2013-04-17 Thread Umesh Prasad
Thanks Erick. Couple of Questions : Our transaction logs are huge as we have disabled auto commit. The biggest one is 6.1 GB. *567M*autosuggest/data/tlog *22M* avmediaCore/data/tlog *388M*booksCore/data/tlog *4.9G * books/data/tlog *6.1G * mp3-downloads/data/tlog ( 150 % of index

Re: Solr 4.2 Startup Detects Corrupt Log And is Really Slow to Start

2013-04-17 Thread Shawn Heisey
On 4/17/2013 10:29 AM, Umesh Prasad wrote: We use DIH and have turned off the Auto commit because we have to sometimes build index from Scratch (clean=true) and we not want to Our master server sees a lot of restarts, sometimes 2-3 times a day. It polls other Data Sources for updates which are

Re: Solr 4.2 Startup Detects Corrupt Log And is Really Slow to Start

2013-04-17 Thread Mark Miller
On Apr 17, 2013, at 1:42 PM, Shawn Heisey s...@elyograg.org wrote: On 4/17/2013 10:29 AM, Umesh Prasad wrote: We use DIH and have turned off the Auto commit because we have to sometimes build index from Scratch (clean=true) and we not want to Our master server sees a lot of restarts,

Re: Solr 4.2 Startup Detects Corrupt Log And is Really Slow to Start

2013-04-17 Thread Shawn Heisey
On 4/17/2013 11:56 AM, Mark Miller wrote: There is one additional caveat - when you disable the updateLog, you have to switch to MMapDirectoryFactory instead of NRTCachingDirectoryFactory. The NRT directory implementation will cache a portion of a commit (including hard commits) into RAM

Solr 4.2 Startup Detects Corrupt Log And is Really Slow to Start

2013-04-16 Thread Umesh Prasad
Hi, We are migrating to Solr 4.2 from Solr 3.6 and Solr 4.2 is throwing Exception on Restart. What is More, it take a hell lot of Time ( More than one hour to get Up and Running) THE exception After Restart ... = Apr 16, 2013 4:47:31 PM