Re: the version of a Lucene index changes after an optimize?
Thank you Erik for the information you gave me. I will test the version of the index in order to know when I need to refresh the component. Best Regards, gquaire - Jouve ITS France -- View this message in context: http://lucene.472066.n3.nabble.com/the-version-of-a-Lucene-index-changes-after-an-optimize-tp3143822p3148275.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: the version of a Lucene index changes after an optimize?
If a document is deleted, the terms are left in the index, the document is just *marked* as deleted. So anything that traverses the terms will pick up terms from old (deleted) documents. An optimize will remove the "stale" data, so I should think that your component will have to be refreshed on an optimize! I'm pretty sure that the version number is changed after an optimize, as the segments files may be completely re-written and the index is, indeed, different. It should be a pretty easy thing to verify... Best Erick On Thu, Jul 7, 2011 at 4:50 AM, gquaire wrote: > Thanks Eric for your reply,. > > To answer to your question, I'm currently developing a kind of > TermsComponent which is able to merge the terms of several fields and have > the ability to reach a position in the list with a random access . To do > that, I construct a merged list of terms from the Lucene Index for these > fields. I need to rebuild this list each time the index has been modified. > If an optimize changes the Lucene index data, so I have to detect it as I do > for classical upates. Can I use the version number of the index to detect > such modifications? > > Best regards, > > gquaire > > 2011/7/6 Erick Erickson [via Lucene] < > ml-node+3145453-901143774-377...@n3.nabble.com> > >> I question this point: >> "But, if only an optimize has happened (the data in the index >> didn't change), the component doesn't need to be modified" >> >> An optimize may, for instance, change the internal Lucene >> document IDs. What is your component doing? >> >> Also, optimize should be a fairly rare occurrence. I'm wondering >> if it's worth the hassle to detect it. >> >> Best >> Erick >> >> On Wed, Jul 6, 2011 at 3:37 AM, gquaire <[hidden >> email]<http://user/SendEmail.jtp?type=node&node=3145453&i=0>> >> wrote: >> >> > Hello everybody, >> > >> > I am new in this forum and I need your expertize on Solr-Lucene. >> > I'm currently develop a new component for Solr for a professional >> project. >> > This component has to be refreshed when some mofications have been >> applied >> > in the index. But, if only an optimize has happened (the data in the >> index >> > didn't change), the component doesn't need to be modified. To do that, >> I'm >> > testing the version number stored in the index which can be retrieved by >> the >> > IndexReader class with "IndexReader.getCurrentVersion()". But, I need to >> > know if the version number is incremented after an optimize operation. >> Can >> > you tell me if it is the case? >> > If it is, how can I detect that the data have changed in the index ? >> > >> > Thanks for your help! >> > >> > gquaire >> > >> > >> > >> > - >> > Jouve ITS France >> > -- >> > View this message in context: >> http://lucene.472066.n3.nabble.com/the-version-of-a-Lucene-index-changes-after-an-optimize-tp3143822p3143822.html >> > Sent from the Solr - User mailing list archive at Nabble.com. >> > >> >> >> -- >> If you reply to this email, your message will be added to the discussion >> below: >> >> http://lucene.472066.n3.nabble.com/the-version-of-a-Lucene-index-changes-after-an-optimize-tp3143822p3145453.html >> To unsubscribe from the version of a Lucene index changes after an >> optimize?, click >> here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3143822&code=Z3F1YWlyZS5wcm9AZ21haWwuY29tfDMxNDM4MjJ8LTE0MDIwODk0OTg=>. >> >> > > > - > Jouve ITS France > -- > View this message in context: > http://lucene.472066.n3.nabble.com/the-version-of-a-Lucene-index-changes-after-an-optimize-tp3143822p3147556.html > Sent from the Solr - User mailing list archive at Nabble.com.
Re: the version of a Lucene index changes after an optimize?
Thanks Eric for your reply,. To answer to your question, I'm currently developing a kind of TermsComponent which is able to merge the terms of several fields and have the ability to reach a position in the list with a random access . To do that, I construct a merged list of terms from the Lucene Index for these fields. I need to rebuild this list each time the index has been modified. If an optimize changes the Lucene index data, so I have to detect it as I do for classical upates. Can I use the version number of the index to detect such modifications? Best regards, gquaire 2011/7/6 Erick Erickson [via Lucene] < ml-node+3145453-901143774-377...@n3.nabble.com> > I question this point: > "But, if only an optimize has happened (the data in the index > didn't change), the component doesn't need to be modified" > > An optimize may, for instance, change the internal Lucene > document IDs. What is your component doing? > > Also, optimize should be a fairly rare occurrence. I'm wondering > if it's worth the hassle to detect it. > > Best > Erick > > On Wed, Jul 6, 2011 at 3:37 AM, gquaire <[hidden > email]<http://user/SendEmail.jtp?type=node&node=3145453&i=0>> > wrote: > > > Hello everybody, > > > > I am new in this forum and I need your expertize on Solr-Lucene. > > I'm currently develop a new component for Solr for a professional > project. > > This component has to be refreshed when some mofications have been > applied > > in the index. But, if only an optimize has happened (the data in the > index > > didn't change), the component doesn't need to be modified. To do that, > I'm > > testing the version number stored in the index which can be retrieved by > the > > IndexReader class with "IndexReader.getCurrentVersion()". But, I need to > > know if the version number is incremented after an optimize operation. > Can > > you tell me if it is the case? > > If it is, how can I detect that the data have changed in the index ? > > > > Thanks for your help! > > > > gquaire > > > > > > > > - > > Jouve ITS France > > -- > > View this message in context: > http://lucene.472066.n3.nabble.com/the-version-of-a-Lucene-index-changes-after-an-optimize-tp3143822p3143822.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > > > -- > If you reply to this email, your message will be added to the discussion > below: > > http://lucene.472066.n3.nabble.com/the-version-of-a-Lucene-index-changes-after-an-optimize-tp3143822p3145453.html > To unsubscribe from the version of a Lucene index changes after an > optimize?, click > here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3143822&code=Z3F1YWlyZS5wcm9AZ21haWwuY29tfDMxNDM4MjJ8LTE0MDIwODk0OTg=>. > > - Jouve ITS France -- View this message in context: http://lucene.472066.n3.nabble.com/the-version-of-a-Lucene-index-changes-after-an-optimize-tp3143822p3147556.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: the version of a Lucene index changes after an optimize?
I question this point: "But, if only an optimize has happened (the data in the index didn't change), the component doesn't need to be modified" An optimize may, for instance, change the internal Lucene document IDs. What is your component doing? Also, optimize should be a fairly rare occurrence. I'm wondering if it's worth the hassle to detect it. Best Erick On Wed, Jul 6, 2011 at 3:37 AM, gquaire wrote: > Hello everybody, > > I am new in this forum and I need your expertize on Solr-Lucene. > I'm currently develop a new component for Solr for a professional project. > This component has to be refreshed when some mofications have been applied > in the index. But, if only an optimize has happened (the data in the index > didn't change), the component doesn't need to be modified. To do that, I'm > testing the version number stored in the index which can be retrieved by the > IndexReader class with "IndexReader.getCurrentVersion()". But, I need to > know if the version number is incremented after an optimize operation. Can > you tell me if it is the case? > If it is, how can I detect that the data have changed in the index ? > > Thanks for your help! > > gquaire > > > > ----- > Jouve ITS France > -- > View this message in context: > http://lucene.472066.n3.nabble.com/the-version-of-a-Lucene-index-changes-after-an-optimize-tp3143822p3143822.html > Sent from the Solr - User mailing list archive at Nabble.com. >
the version of a Lucene index changes after an optimize?
Hello everybody, I am new in this forum and I need your expertize on Solr-Lucene. I'm currently develop a new component for Solr for a professional project. This component has to be refreshed when some mofications have been applied in the index. But, if only an optimize has happened (the data in the index didn't change), the component doesn't need to be modified. To do that, I'm testing the version number stored in the index which can be retrieved by the IndexReader class with "IndexReader.getCurrentVersion()". But, I need to know if the version number is incremented after an optimize operation. Can you tell me if it is the case? If it is, how can I detect that the data have changed in the index ? Thanks for your help! gquaire - Jouve ITS France -- View this message in context: http://lucene.472066.n3.nabble.com/the-version-of-a-Lucene-index-changes-after-an-optimize-tp3143822p3143822.html Sent from the Solr - User mailing list archive at Nabble.com.
