Re: Solr documents update on index
Hi, But i'm indexing rss feeds. I want that solr indexes that without change the existing information of a document with the same uniqueKey. The best approach is that solr updates the doc if changes are detected, but i can leave without that. I really would like that solr does not update the document if it already exists. I'm using the DataImportScheduler to solr index launch the scheduled index. Appreciate any possible help. On Sep 6, 2013, at 9:16 AM, Shalin Shekhar Mangar wrote: > Yes, if a document with the same key exists, then the old document > will be deleted and replaced with the new document. You can also > partially update documents (we call it atomic updates) which reads the > old document from local index, updates it according to the request and > then replaces the old document with the new one. > > See > https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+Index+Handlers#UploadingDatawithIndexHandlers-UpdatingOnlyPartofaDocument > > On Fri, Sep 6, 2013 at 1:03 AM, Luis Portela Afonso > wrote: >> Hi, >> >> I'm having a problem when solr indexes. >> It is updating documents already indexed. Is this a normal behavior? >> If a document with the same key already exists is it supposed to be updated? >> I has thinking that is supposed to just update if the information on the >> rss has changed. >> >> Appreciate your help >> >> -- >> Sent from Gmail Mobile > > > > -- > Regards, > Shalin Shekhar Mangar. smime.p7s Description: S/MIME cryptographic signature
Re: Solr documents update on index
Yes, if a document with the same key exists, then the old document will be deleted and replaced with the new document. You can also partially update documents (we call it atomic updates) which reads the old document from local index, updates it according to the request and then replaces the old document with the new one. See https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+Index+Handlers#UploadingDatawithIndexHandlers-UpdatingOnlyPartofaDocument On Fri, Sep 6, 2013 at 1:03 AM, Luis Portela Afonso wrote: > Hi, > > I'm having a problem when solr indexes. > It is updating documents already indexed. Is this a normal behavior? > If a document with the same key already exists is it supposed to be updated? > I has thinking that is supposed to just update if the information on the > rss has changed. > > Appreciate your help > > -- > Sent from Gmail Mobile -- Regards, Shalin Shekhar Mangar.
Solr documents update on index
Hi, I'm having a problem when solr indexes. It is updating documents already indexed. Is this a normal behavior? If a document with the same key already exists is it supposed to be updated? I has thinking that is supposed to just update if the information on the rss has changed. Appreciate your help -- Sent from Gmail Mobile
Re: Trigger documents update in a collection
Hi, Doable with a custom Update Request Processor, yes. Otis Solr & ElasticSearch Support http://sematext.com/ On Apr 15, 2013 3:14 PM, "Francois Perron" < francois.per...@wantedanalytics.com> wrote: > Hi all, > > I want to use Solr4 as a NoSQL. > > My 'ideal' workflow is to add/update documents in a collection (NoSQL) and > automatically update changes in another collection with more specific > search capabilities. The nosql collection will contains all my documents > (750M docs). The 'searchable' collection will only contain a subset of > this collection (active documents based on a field). > > Is it possible ? > > Thank you
Trigger documents update in a collection
Hi all, I want to use Solr4 as a NoSQL. My 'ideal' workflow is to add/update documents in a collection (NoSQL) and automatically update changes in another collection with more specific search capabilities. The nosql collection will contains all my documents (750M docs). The 'searchable' collection will only contain a subset of this collection (active documents based on a field). Is it possible ? Thank you
Re: Documents update
Created file, reloaded solr - externalfilefield works fine, if i change change external files and do "curl http://127.0.0.1:4900/solr/site/update -H "Content-Type: text/xml" --data-binary ''" then no thanges are made. If i start solr without external files and then create them - they are not working.. What is wrong? PS: Solr 3.2 > http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html > On Tuesday 31 May 2011 15:41:32 Denis Kuzmenok wrote: >> Flags are stored to filter results and it's pretty highloaded, it's >> working fine, but i can't update index very often just to make flags >> up to time =\ >> Where can i read about using external fields / files? >> >> > And it wouldn't work unless all the data is stored anyway. Currently >> > there's no way to update a single field in a document, although there's >> > work being done in that direction (see the "column stride" JIRA). >> > >> > What do you want to do with these fields? If it's to influence scoring, >> > you could look at external fields. >> > >> > If the flags are a selection criteria, it's...harder. What are the flags >> > used for? Could you consider essentially storing a map of the >> > uniqueKey's and flags in a "special" document and having your app >> > read that document and merge the results with the output? If this seems >> > irrelevant, a more complete statement of the use-case would be helpful. >> > >> > Best >> > Erick
Re: Documents update
> Will it be slow if there are 3-5 million key/value rows? AFAIK it shouldn't affect search time significantly as Solr caches it in memory after you reloading Solr core / issuing commit. But obviously you need more memory and commit/reload will take more time.
Re: Documents update
Will it be slow if there are 3-5 million key/value rows? > http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html > On Tuesday 31 May 2011 15:41:32 Denis Kuzmenok wrote: >> Flags are stored to filter results and it's pretty highloaded, it's >> working fine, but i can't update index very often just to make flags >> up to time =\ >> Where can i read about using external fields / files?
Re: Documents update
http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html On Tuesday 31 May 2011 15:41:32 Denis Kuzmenok wrote: > Flags are stored to filter results and it's pretty highloaded, it's > working fine, but i can't update index very often just to make flags > up to time =\ > Where can i read about using external fields / files? > > > And it wouldn't work unless all the data is stored anyway. Currently > > there's no way to update a single field in a document, although there's > > work being done in that direction (see the "column stride" JIRA). > > > > What do you want to do with these fields? If it's to influence scoring, > > you could look at external fields. > > > > If the flags are a selection criteria, it's...harder. What are the flags > > used for? Could you consider essentially storing a map of the > > uniqueKey's and flags in a "special" document and having your app > > read that document and merge the results with the output? If this seems > > irrelevant, a more complete statement of the use-case would be helpful. > > > > Best > > Erick -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350
Re: Documents update
Flags are stored to filter results and it's pretty highloaded, it's working fine, but i can't update index very often just to make flags up to time =\ Where can i read about using external fields / files? > And it wouldn't work unless all the data is stored anyway. Currently there's > no way to update a single field in a document, although there's work being > done in that direction (see the "column stride" JIRA). > What do you want to do with these fields? If it's to influence scoring, you > could look at external fields. > If the flags are a selection criteria, it's...harder. What are the flags > used for? Could you consider essentially storing a map of the > uniqueKey's and flags in a "special" document and having your app > read that document and merge the results with the output? If this seems > irrelevant, a more complete statement of the use-case would be helpful. > Best > Erick
Re: Documents update
And it wouldn't work unless all the data is stored anyway. Currently there's no way to update a single field in a document, although there's work being done in that direction (see the "column stride" JIRA). What do you want to do with these fields? If it's to influence scoring, you could look at external fields. If the flags are a selection criteria, it's...harder. What are the flags used for? Could you consider essentially storing a map of the uniqueKey's and flags in a "special" document and having your app read that document and merge the results with the output? If this seems irrelevant, a more complete statement of the use-case would be helpful. Best Erick On Fri, May 27, 2011 at 4:33 AM, Denis Kuzmenok wrote: > I'm using 3.1 now. Indexing lasts for a few hours, and have big > plain size. Getting all documents would be rather slow :( > > >> Not with 1.4, but apparently there is a patch for trunk. Not >> sure if it is in 3.1. > >> If you are on 1.4, you could first query Solr to get the data >> for the document to be changed, change the modified values, >> and make a complete XML, including all fields, for post.jar. > >> Regards, >> Gora > > > > >
Re: Documents update
I'm using 3.1 now. Indexing lasts for a few hours, and have big plain size. Getting all documents would be rather slow :( > Not with 1.4, but apparently there is a patch for trunk. Not > sure if it is in 3.1. > If you are on 1.4, you could first query Solr to get the data > for the document to be changed, change the modified values, > and make a complete XML, including all fields, for post.jar. > Regards, > Gora
Re: Documents update
2011/5/27 Denis Kuzmenok : > Hi. > > I have and indexed database which is indexed few times a day and > contain tinyint flag (like is_enabled, is_active, etc), and content > isn't changed too often, but flags are. > So if i index via post.jar only flags then entire document is deleted > and there's only unique key and flags. > Is there any way to index certain columns, and not to change all > document? [...] Not with 1.4, but apparently there is a patch for trunk. Not sure if it is in 3.1. If you are on 1.4, you could first query Solr to get the data for the document to be changed, change the modified values, and make a complete XML, including all fields, for post.jar. Regards, Gora
Documents update
Hi. I have and indexed database which is indexed few times a day and contain tinyint flag (like is_enabled, is_active, etc), and content isn't changed too often, but flags are. So if i index via post.jar only flags then entire document is deleted and there's only unique key and flags. Is there any way to index certain columns, and not to change all document? Thanks
