Understanding autoSoftCommit

2012-09-24 Thread Trym R. Møller

Hi

On my windows workstation I have tried to index a document into a 
SolrCloud instance with the following special configuration:

autoCommit
maxTime120/maxTime
/autoCommit
autoSoftCommit
maxTime60/maxTime
/autoSoftCommit
...
updateLog
  str name=dir${solr.data.dir:}/str
/updateLog
That is commit every 20 minutes and soft commit every 10 minutes.

Right after indexing I can find the document using /get (and not using 
/search) and after 10 minutes I can find it as well using /search.
If I stop Solr using Ctrl+C or kill -9 (from my cygwin console) before 
the 10 minutes have passed and starts Solr again then I can find the 
document using both /get and /search.


Are there any scenarios where I will loose an indexed document before 
either commit or soft commit is trigged?

And does the transaction log have anything to do with this...

Thanks in advance.

Best regards Trym


Re: Understanding autoSoftCommit

2012-09-24 Thread Mark Miller
autoCommit (hard commit) is basically just to reduce how much RAM is
needed for the transaction log. You should generally use it with
openSearcher=false and don't need to use it for visibility.

It's also not required for durability due to the transaction log.

Soft commit should be used for visibility. It's also got nothing to do
with durability.

For durability, the idea is that if Solr accepts your update, it's in.
And yes, the transaction log is part of that.

- Mark

On Mon, Sep 24, 2012 at 7:12 AM, Trym R. Møller t...@sigmat.dk wrote:
 Hi

 On my windows workstation I have tried to index a document into a SolrCloud
 instance with the following special configuration:
 autoCommit
 maxTime120/maxTime
 /autoCommit
 autoSoftCommit
 maxTime60/maxTime
 /autoSoftCommit
 ...
 updateLog
   str name=dir${solr.data.dir:}/str
 /updateLog
 That is commit every 20 minutes and soft commit every 10 minutes.

 Right after indexing I can find the document using /get (and not using
 /search) and after 10 minutes I can find it as well using /search.
 If I stop Solr using Ctrl+C or kill -9 (from my cygwin console) before the
 10 minutes have passed and starts Solr again then I can find the document
 using both /get and /search.

 Are there any scenarios where I will loose an indexed document before either
 commit or soft commit is trigged?
 And does the transaction log have anything to do with this...

 Thanks in advance.

 Best regards Trym



-- 
- Mark