Re: In Place Updates not work as expected
Hi, That’s how you build regular document. Incremental/atomic updates need to use update commands. Did not check latest Solrj, so maybe there is built in way of doing that, but quick googling showed how it can be achieved: SolrInputDocument doc2 = new SolrInputDocument(); MapfpValue2 = new HashMap (); fpValue2.put("add","fp2"); doc2.setField("FACTURES_PRODUIT", fpValue2); HTH, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 16 Mar 2018, at 15:36, mganeshs wrote: > > Hi Emir, > > It's normal setfield and addDocument > > for ex. > in a for loop > solrInputDocument.setField(sFieldId, fieldValue); > and after this, we add the created document. > solrClient.add(collectionName, solrInputDocuments); > > I just want to know whether, we need to do something specific for in-place > updates ? > > Kindly let me know, > > Regards, > > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: In Place Updates not work as expected
Hi Emir, It's normal setfield and addDocument for ex. in a for loop solrInputDocument.setField(sFieldId, fieldValue); and after this, we add the created document. solrClient.add(collectionName, solrInputDocuments); I just want to know whether, we need to do something specific for in-place updates ? Kindly let me know, Regards, -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: In Place Updates not work as expected
Hi, Can you share part of code where you prepare update. Thanks, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 14 Mar 2018, at 15:27, mganeshswrote: > > Hi Emir, > > I am using solrj to update the document. Is there any spl API to be used for > in place Updates ? > > Yes are we are updating in Batch of 1000 documents. > > As I mentioned before, since I am updating only docvalues i expect it should > update in faster than updating normal field. Isn't it ? > > Regards, > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: In Place Updates not work as expected
On 3/14/2018 8:27 AM, mganeshs wrote: As I mentioned before, since I am updating only docvalues i expect it should update in faster than updating normal field. Isn't it ? Maybe. But not always. To do an in-place update, Solr must rewrite the docValues data for that field in that segment. It must write this data for *EVERY* document in that segment which has that field. If the segment has a few documents, this will almost certainly be very fast. But if the segment with the document you are updating is large and contains enough documents that there are a million unique values for that field, then Solr is going to have to gather those million values from the existing data, then write a new file containing a million values. This isn't going to be fast ... and a standard update probably would be faster. Segments are part of the organization of a Lucene index. Solr is a Lucene application. Thanks, Shawn
Re: In Place Updates not work as expected
Hi Emir, I am using solrj to update the document. Is there any spl API to be used for in place Updates ? Yes are we are updating in Batch of 1000 documents. As I mentioned before, since I am updating only docvalues i expect it should update in faster than updating normal field. Isn't it ? Regards, -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: In Place Updates not work as expected
Hi, Did you confirm that it actually does in place update? In case of in place update, after update (maybe try single) only doc values file should change (if my understanding is right). Do you update a full document or some test doc with a single field? Do you batch updates or send one by one? Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 16 Feb 2018, at 05:19, mganeshswrote: > > All, > > I have (say 1M, in real time it would be more even) solr documents which has > lot of fields and it's bit huge. We have a functionality, where we need to > go and update a specific field or add new field in to that document. Since > we have to do this for all 1M documents, it's taking up more time and it's > not acceptable. > > So we thought of using "In Place Updates". > > As per documentation, we have made sure it's following this criteria > --- > *An atomic update operation is performed using this approach only when the > fields to be updated meet these three conditions: > > are non-indexed (indexed="false"), non-stored (stored="false"), single > valued (multiValued="false") numeric docValues (docValues="true") fields; > > the _version_ field is also a non-indexed, non-stored single valued > docValues field; and, > > copy targets of updated fields, if any, are also non-indexed, non-stored > single valued numeric docValues fields.* > --- > To check whether it's working as expected, > * First we tried to update a normal field and it took around 1.5 Hours to > update all 1M docs, as the complete documents is getting re-indexed. > > * We also tried to update the docvalue field and it also took around 1.5 > hours to complete for 1M docs. > > As in the second case, we are updating docvalue field type, and as it won't > re-index the complete document, isn't that should take lesser time ? > > What could be going wrong ? I am using Sorl 6.5.1. Is this a bug or expected > behavior ? > > Regards, > > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
In Place Updates not work as expected
All, I have (say 1M, in real time it would be more even) solr documents which has lot of fields and it's bit huge. We have a functionality, where we need to go and update a specific field or add new field in to that document. Since we have to do this for all 1M documents, it's taking up more time and it's not acceptable. So we thought of using "In Place Updates". As per documentation, we have made sure it's following this criteria --- *An atomic update operation is performed using this approach only when the fields to be updated meet these three conditions: are non-indexed (indexed="false"), non-stored (stored="false"), single valued (multiValued="false") numeric docValues (docValues="true") fields; the _version_ field is also a non-indexed, non-stored single valued docValues field; and, copy targets of updated fields, if any, are also non-indexed, non-stored single valued numeric docValues fields.* --- To check whether it's working as expected, * First we tried to update a normal field and it took around 1.5 Hours to update all 1M docs, as the complete documents is getting re-indexed. * We also tried to update the docvalue field and it also took around 1.5 hours to complete for 1M docs. As in the second case, we are updating docvalue field type, and as it won't re-index the complete document, isn't that should take lesser time ? What could be going wrong ? I am using Sorl 6.5.1. Is this a bug or expected behavior ? Regards, -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html