1- SOLR 4.6 2- We do but right now I am talking about plain keyword queries just sorted by date. Once this is better will start looking into caches which we already changed a little. 3- As I said the contents are not stored in this index. Some other metadata fields are but with normal queries its super fast so I guess even if I change there it will be a minor difference. We have SSD and quite fast too. 4- That's something we need to do but even in low workload those queries take a lot of time 5- Every 10 mins and currently no auto warming as user queries are rarely same and also once its fully warmed those queries are still slow. 6- Nops.
On Thu, Mar 13, 2014 at 5:38 PM, Dmitry Kan <solrexp...@gmail.com> wrote: > 1. What is your solr version? In 4.x family the proximity searches have > been optimized among other query types. > 2. Do you use the filter queries? What is the situation with the cache > utilization ratios? Optimize (= i.e. bump up the respective cache sizes) if > you have low hitratios and many evictions. > 3. Can you avoid storing some fields and only index them? When the field is > stored and it is retrieved in the result, there are couple of disk seeks > per field=> search slows down. Consider SSD disks. > 4. Do you monitor your system in terms of RAM / cache stats / GC? Do you > observe STW GC pauses? > 5. How often do you commit & do you have the autowarming / external warming > configured? > 6. If you use faceting, consider storing DocValues for facet fields. > > some solr wiki docs: > > https://wiki.apache.org/solr/SolrPerformanceProblems?highlight=%28%28SolrPerformanceFactors%29%29 > > > > > > On Thu, Mar 13, 2014 at 8:52 AM, Salman Akram < > salman.ak...@northbaysolutions.net> wrote: > > > Well some of the searches take minutes. > > > > Below are some stats about this particular index that I am talking about: > > > > Index size = 400GB (Using CommonGrams so without that the index is around > > 180GB) > > Position File = 280GB > > Total Docs = 170 million (just indexed for searching - for highlighting > > contents are stored in another index) > > Avg Doc Size = Few hundred KBs > > RAM = 384GB (it has other indexes too but still OS cache can have 60-80% > of > > the total index cached) > > > > Phrase queries run pretty fast with CG but complex versions of wildcard > and > > proximity queries can be really slow. I know using CG will make them slow > > but they just take too long. By default sorting is on date but users have > > few other parameters too on which they can sort. > > > > I wanted to avoid creating multiple indexes (maybe based on years) but > > seems that to search on partial data that's the only feasible way. > > > > > > > > > > On Wed, Mar 12, 2014 at 2:47 PM, Dmitry Kan <solrexp...@gmail.com> > wrote: > > > > > As Hoss pointed out above, different projects have different > > requirements. > > > Some want to sort by date of ingestion reverse, which means that having > > > posting lists organized in a reverse order with the early termination > is > > > the way to go (no such feature in Solr directly). Some other projects > > want > > > to collect all docs matching a query, and then sort by rank, but you > > cannot > > > guarantee, that the most recently inserted document is the most > relevant > > in > > > terms of your ranking. > > > > > > > > > Do your current searches take too long? > > > > > > > > > On Tue, Mar 11, 2014 at 11:51 AM, Salman Akram < > > > salman.ak...@northbaysolutions.net> wrote: > > > > > > > Its a long video and I will definitely go through it but it seems > this > > is > > > > not possible with SOLR as it is? > > > > > > > > I just thought it would be quite a common issue; I mean generally for > > > > search engines its more important to show the first page results, > > rather > > > > than using timeAllowed which might not even return a single result. > > > > > > > > Thanks! > > > > > > > > > > > > -- > > > > Regards, > > > > > > > > Salman Akram > > > > > > > > > > > > > > > > -- > > > Dmitry > > > Blog: http://dmitrykan.blogspot.com > > > Twitter: http://twitter.com/dmitrykan > > > > > > > > > > > -- > > Regards, > > > > Salman Akram > > > > > > -- > Dmitry > Blog: http://dmitrykan.blogspot.com > Twitter: http://twitter.com/dmitrykan > -- Regards, Salman Akram
