Re: Best way to check Solr index for completeness

2010-09-29 Thread Dennis Gearon
://www.yert.com/film.php --- On Tue, 9/28/10, dshvadskiy dshvads...@gmail.com wrote: From: dshvadskiy dshvads...@gmail.com Subject: Re: Best way to check Solr index for completeness To: solr-user@lucene.apache.org Date: Tuesday, September 28, 2010, 2:11 PM That will certainly work for most recent

Re: Best way to check Solr index for completeness

2010-09-29 Thread Peter Karich
How long does it take to get 1000 docs? Why not ensure this while indexing? I think besides your suggestion or the suggestion of Luke there is no other way... Regards, Peter. Hello, What would be the best way to check Solr index against original system (Database) to make sure index is up to

Re: Best way to check Solr index for completeness

2010-09-29 Thread dshvadskiy
TermComponent and compare them with hash calculated on database record. -- View this message in context: http://lucene.472066.n3.nabble.com/Best-way-to-check-Solr-index-for-completeness-tp1598626p1602597.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Best way to check Solr index for completeness

2010-09-29 Thread dshvadskiy
Regenerating index is a slow operation due to limitation of the source systems. We run several complex SQL statements to generate 1 Solr document. Full reindex takes about 24 hours. -- View this message in context: http://lucene.472066.n3.nabble.com/Best-way-to-check-Solr-index

Re: Best way to check Solr index for completeness

2010-09-29 Thread dshvadskiy
.472066.n3.nabble.com/Best-way-to-check-Solr-index-for-completeness-tp1598626p1603108.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Best way to check Solr index for completeness

2010-09-29 Thread Walter Underwood
still like the idea of using TermComponent and will use it in the future if number of docs in the index will grow. Thanks for all suggestions. Dmitriy -- View this message in context: http://lucene.472066.n3.nabble.com/Best-way-to-check-Solr-index-for-completeness-tp1598626p1603108.html

Re: Best way to check Solr index for completeness

2010-09-29 Thread Erick Erickson
-way-to-check-Solr-index-for-completeness-tp1598626p1602597.html Sent from the Solr - User mailing list archive at Nabble.com.

Best way to check Solr index for completeness

2010-09-28 Thread Dmitriy Shvadskiy
Hello, What would be the best way to check Solr index against original system (Database) to make sure index is up to date? I can use Solr fields like Id and timestamp to check against appropriate fields in database. Our index currently contains over 2 mln documents across several cores. Pulling

Re: Best way to check Solr index for completeness

2010-09-28 Thread Luke Crouch
Is there a 1:1 ratio of db records to solr documents? If so, couldn't you simply select the most recent updated record from the db and check to make sure the corresponding solr doc has the same timestamp? -L On Tue, Sep 28, 2010 at 3:48 PM, Dmitriy Shvadskiy dshvads...@gmail.comwrote: Hello,

Re: Best way to check Solr index for completeness

2010-09-28 Thread dshvadskiy
://lucene.472066.n3.nabble.com/Best-way-to-check-Solr-index-for-completeness-tp1598626p1598733.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Best way to check Solr index for completeness

2010-09-28 Thread Erick Erickson
in context: http://lucene.472066.n3.nabble.com/Best-way-to-check-Solr-index-for-completeness-tp1598626p1598733.html Sent from the Solr - User mailing list archive at Nabble.com.