[
https://issues.apache.org/jira/browse/SOLR-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096346#comment-13096346
]
Yonik Seeley commented on SOLR-2565:
------------------------------------
So now that commits no longer block adds, we should revisit what the best
defaults are.
The transaction logging in SOLR-2700 needs to keep track of uncommitted
documents - hence if it doesn't affect performance too much, we should probably
commit more often some how. Autocommit based on number of documents doesn't
work well across the broad spectrum of users (what value would work well for
twitter indexers and book indexers). A size-based approach would probably work
best, but we don't have that. Maybe a time based approach? That would limit
the transaction log size to the number of documents indexable in a given time
period, which should be roughly proportional to the document size. I guess
such a time period should be somewhere between 10 and 60 seconds? A lot of
data can be indexed in 60 sec, and the goal is to limit the transaction log
size while not impacting performance too much due to increased commit frequency.
The other issue is soft commits... should we configure a soft commitWithin by
default (prob within the range of 1-10 sec)?
> Prevent IW#close and cut over to IW#commit
> ------------------------------------------
>
> Key: SOLR-2565
> URL: https://issues.apache.org/jira/browse/SOLR-2565
> Project: Solr
> Issue Type: Improvement
> Components: update
> Affects Versions: 4.0
> Reporter: Simon Willnauer
> Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-2565-revert.patch, SOLR-2565.patch,
> SOLR-2565.patch, SOLR-2565.patch, SOLR-2565__HuperDuperAutoCommitTest.patch,
> dump.txt, fix+hossmans-test.patch, slowtests.txt
>
>
> Spinnoff from SOLR-2193. We already have a branch to work on this issue here
> https://svn.apache.org/repos/asf/lucene/dev/branches/solr2193
> The main goal here is to prevent solr from closing the IW and use IW#commit
> instead. AFAIK the main issues here are:
> The update handler needs an overhaul.
> A few goals I think we might want to look at:
> 1. Expose the SolrIndexWriter in the api or add the proper abstractions to
> get done what we now do with special casing:
> 2. Stop closing the IndexWriter and start using commit (still lazy IW init
> though).
> 3. Drop iwAccess, iwCommit locks and sync mostly at the Lucene level.
> 4. Address the current issues we face because multiple original/'reloaded'
> cores can have a different IndexWriter on the same index.
> Eventually this is a preparation for NRT support in Solr which I will create
> a followup issue for.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
