If you dont' plan on filtering/ sorting and/or faceting on fast-changing fields it would be better to store them outside of solr/lucene in my opinion.
If you must: for indexing-performance reasons you will probably end up with maintaining seperate indices (1 for slow-changing/static fields and 1 for fast-changing-fields) . You frequently commit the fast-changing -index to incorporate the changes in current_price. Afterwards you have 2 options I believe: 1. use parallelreader to query the seperate indices directly. Afaik, this is not (completely) integrated in Solr... I wouldn't recommend it. 2. after you commit the fast-changing-index, merge with the static-index. You're left with 1 fresh index, which you can push to your slave-servers. (all this in regular interverals) Disadvatages: - In any way, you must be very careful with maintaining multiple parallel indexes with the purpose of treating them as one. For instance document inserts must be done exactly in the same order, otherwise the indices go 'out-of-sync' and are unusable. - higher maintenance - there is always a time-window in which the current_price values are stale. If that's within reqs that's ok. The other path, which I recommend, would be to store the current_price outside of solr (like you're currently doing) but instead of using a relational db, try looking into persistent key-value stores. Many of them exist and a lot of progress has been made in the last couple of years. For simple key-lookups (what you need as far as I can tell) they really blow every relational db out of the water (considering the same hardware of course) We're currently using Tokyo Cabinet with the server-frontend Tokyo Tyrant and seeing almost a 5x increased in lookup performance compared to our previous kv-store memcachedDB which is based on BerkelyDB. Memcachedb was already several times faster than our mysql-setup (although not optimally tuned) . to sum things up: use the best tools for what they were meant to do. - index/search --> solr/ lucene without a doubt. - kv-lookup --> consensus is still forming, and a lot of players (with a lot of different types of functionality) but if all you need is simple key-value-lookup, I would go for Tokyo Cabinet (TC) / Tyrant at the moment. Please note that TC and competitors aren't just some code/ hobby projects but are usually born out of a real need at huge websites / social networks such as TC which is born from mixi (big social network in Japan) . So at least you're in good company.. for kv-stores I would suggest to begin your research at: http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores/ (beginning 2009) http://randomfoo.net/2009/04/20/some-notes-on-distributed-key-stores (half 2009) and get a feel of the kv-playing field. Hope this (pretty long) post helps, Geert-Jan 2010/3/17 Krzysztof Grodzicki <krzysztof.grodzi...@iterate.pl> > Hi Mortiz, > > You can take a look on the project ZOIE - > http://code.google.com/p/zoie/. I think it's that what are you looking > for. > > br > Krzysztof > > On Wed, Mar 17, 2010 at 9:49 AM, Moritz Mädler <m...@moritz-maedler.de> > wrote: > > Hi List, > > > > we are running a marketplace which has about a comparable functionality > like ebay (auctions, fixed-price items etc). > > The items are placed on the market by users who want to sell their goods. > > > > Currently we are using Sphinx as an indexing engine, but, as Sphinx > returns only document ids we have to make a > > database-query to fetch the data to display. This massively decreases > performance as we have to do two requests to > > display data. > > > > I heard that Solr is able to return a complete dataset and we hope a > switch to Solr can boost perfomance. > > A critical question is left and i was not able to find a solution for it > in the docs: Is it possible to update attributes directly in the > > index? > > An example for better illustration: > > We have an index which holds all the auctions (containing auctionid, > auction title) with its current prices(field: current_price). When a user > places a new bid, > > is it possible to update the attribute 'current_price' directly in the > index so that we can fetch the current_price from Solr and not from the > database? > > > > I hope you understood my problem. It would be kind if someone can point > me to the right direction. > > > > Thanks alot! > > > > Moritz >
