High system cpu usage while starting solr
Hello, Im running few solr cores on one pretty good server. After some time i discover that restarting solr makes queries last longer. What i see is that after restart jvm usage is realy low and raise slowly while system cpu ussage is high. My select queries are realy slow during that time. After few days when jvm grab some MORE memory system drops down. java settings -Xmx6144m -XX:+UseConcMarkSweepGC -XX:+PrintGC -XX:+PrintGCDetails I have 8GB ram HELP! -- View this message in context: http://lucene.472066.n3.nabble.com/High-system-cpu-usage-while-starting-solr-tp4168124.html Sent from the Solr - User mailing list archive at Nabble.com.
Fast question about replication between solr version
I cant't find anywhere inofrmation about replication between solr version. Is it possible replicate between example solr 4.2 and 4.10 without any problems? What i need is to upgrade version and don't wona re-index all from beginning. -- View this message in context: http://lucene.472066.n3.nabble.com/Fast-question-about-replication-between-solr-version-tp4159575.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: slow performance on simple filter
I'm really at dead point. Mine indeks is 5,6GM and about 8mln documments. Field i'm using for filter is simple as hell. field name=class_name type=string indexed=true stored=true multiValued=false/ Can it be that other fields affect my search if i only do filter query? solr/puls-objects-prod/select?q=*%3A*fq=class_name:License mine results: int name=QTime831/int lst name=params str name=q*:*/str str name=fqclass_name:License/str /lst /lst result name=response numFound=8655108 start=0 -- View this message in context: http://lucene.472066.n3.nabble.com/slow-performance-on-simple-filter-tp4135613p4140728.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: slow performance on simple filter
i show you my full query it's rly simple one q=*:* and fq=class_name:CdnFile debug q shows that process of q takes so long. single filter is critical here. And cache is ot an option here. It work well but i need to know why such sipmple filter can takes so long. arr name=parsed_filter_queries strclass_name:CdnFile/str /arr lst name=timing double name=time421.0/double lst name=prepare double name=time1.0/double lst name=query double name=time1.0/double /lst lst name=facet double name=time0.0/double /lst lst name=mlt double name=time0.0/double /lst lst name=highlight double name=time0.0/double /lst lst name=stats double name=time0.0/double /lst lst name=debug double name=time0.0/double /lst /lst lst name=process double name=time420.0/double lst name=query double name=time415.0/double -- View this message in context: http://lucene.472066.n3.nabble.com/slow-performance-on-simple-filter-tp4135613p4139050.html Sent from the Solr - User mailing list archive at Nabble.com.
slow performance on simple filter
Hey,
I got pretty big index with about 7 mln doccuments.
I got pretty slow query when i ask about common word. Nothing changes when i
ask by q or fq.
params={fl=score,class_nameindent=trueq=*:*wt=xmlfq=class_name:CdnFile}
hits=5594978 status=0 QTime=408
It't not about my hardware tho. I was sure solr is blazing fast if i do such
simple queries.
Why my query is so slow? Its because of to frequent worl CdnFile?
PS.
I index only this field into elasticsearch and im getting same responses
like in 0,08s
--
View this message in context:
http://lucene.472066.n3.nabble.com/slow-performance-on-simple-filter-tp4135613.html
Sent from the Solr - User mailing list archive at Nabble.com.
solr 3.6 hang for few seconds, need help
Hi,
Im running solr 3.6 on tomcat, under some traffic about 20r/s
I got 6 different cores on it.
I was testing one by quering every 1 second with simple request and time
param.
INFO: [core1] webapp=/solr3.4-tomcat path=/select params= ... 1:55:05 ...
Apr 29, 2013 1:55:06 PM org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=2
commit{dir=/vol/solr3.4-tomcat/core2/index,segFN=segments_4ecd4,version=1331656373519,generation=7387672,filenames=[_1yxp2.fdx,
_4herl.nrm, _2inup_17f5.del, _1yxp2.fd
commit{dir=/vol/solr3.4-tomcat/core2/index,segFN=segments_4ecdh,version=1331656373568,generation=7387685,filenames=[_1yxp2.fdx,
_4herl.nrm, _2inup_17f5.del, _1yxp2.fd
Apr 29, 2013 1:55:06 PM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1331656373568
Apr 29, 2013 1:55:06 PM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit(optimize=false,waitFlush=false,waitSearcher=true,expungeDeletes=false)
Apr 29, 2013 1:55:06 PM org.apache.solr.core.SolrCore execute
INFO: [core1] webapp=/solr3.4-tomcat path=/select params=... 1:55:06 ...
Apr 29, 2013 1:55:14 PM org.apache.solr.core.SolrCore execute
INFO: [core1] webapp=/solr3.4-tomcat path=/select params=... 1:55:07 ...
Apr 29, 2013 1:55:15 PM org.apache.solr.core.SolrCore execute
INFO: [core1] webapp=/solr3.4-tomcat path=/select params=... 1:55:15 ...
query from 1:55:15 was executed at 1:15:14.
Between 1:55:06 - 1:55:17 there is nothing in solr and tomcat logs.
What could hapen here? I'm getting that hang every some time.
Does commiting, or something could stop me from searching?
--
View this message in context:
http://lucene.472066.n3.nabble.com/solr-3-6-hang-for-few-seconds-need-help-tp4059760.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud new zookeper node on different ip/ replicate between two clasters
This is good sollution. One thing here is rly unyoing. The double indexing. Is there a way to replicate to another dc? Seams solrcloud cant use his ealier replication. Would be nice if i can replicate somehow between two soulrcloud. -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-new-zookeper-node-on-different-ip-replicate-between-two-clasters-tp4039101p4039791.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud new zookeper node on different ip/ replicate between two clasters
Thx Mark The replication handler can be setup to replicate to another dc. Erm, i dont get it. I can setup replication between two solr cloud this way or just solrcloud-solr? You can also put nodes in both dcs Indexing will slow rly much if I understad well solrcluoud replica and leader (replication is real-time ). Worst is when by accident Zoo will elect leader in other dc. Zoo could use obserwers here bit it will only makes things more comlicated too. I have not yet found the multi dc zk solution. Only smth called obserwers help a bit in my case. Zoo called obserwers dont vote thay are jusr like points of read. It would be good here, but after one dc down i need fully working zoo and obserwers doesnt support to change to follower. About zoo conf is ofc big problem, but configuration doesnt change much so two zoo quorum in bots dc are ok imo. I know other systems use something like having a tie breaker node in Europe Yeah, i want run my own cloud and want to have failover in amazon. I'm from europe :) -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-new-zookeper-node-on-different-ip-replicate-between-two-clasters-tp4039101p4039808.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud new zookeper node on different ip/ replicate between two clasters
I dont think its so simple. First I need to have at last 3 zoo to keep failver for one server. Second after one zoo die, i need restart of all solrs. Maybe i define simply question. Two data centers. How to replicate two solr claster between two datacenters? In no SolrClaud there is repeater, if i connect all SolrCloud nodes in one claster between dc i will make lots of trafiick between them. I dont mention about that i will get leader elected in wrong datacanter eventually. How can i have two claster of solr and replicate them between two datacenters? -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-new-zookeper-node-on-different-ip-replicate-between-two-clasters-tp4039101p4039217.html Sent from the Solr - User mailing list archive at Nabble.com.
Way to lock solr for incoming writes
Is there a way to lock solr for writes? I don't wona use solr integrated backup because i'm using ceph claster. What I need is to have consistent data for few seconds to make backup. -- View this message in context: http://lucene.472066.n3.nabble.com/Way-to-lock-solr-for-incoming-writes-tp4033873.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud removing shard (how to not loose data)
Mark, I know i still have access to data and i can woke ap shard again. What i want to do is. I have 3 shards on 3 nodes, one on each. Now i discower that i dont need 3 nodes and i want only 2. So i want to remove shard and put data from it to these who left. Is there way to index that data without force index it again? -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-removing-shard-how-to-not-loose-data-tp4032138p4032459.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud removing shard (how to not loose data)
Seams I'm to lazy. I found this http://wiki.apache.org/solr/MergingSolrIndexes, and it works rly. -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-removing-shard-how-to-not-loose-data-tp4032138p4032508.html Sent from the Solr - User mailing list archive at Nabble.com.
SolrCloud removing shard (how to not loose data)
Lets say i got one collection with 3 shards. Every shard contains indexed data. I want to unload one shard. Is there any way for data from unloaded shard to be not lost? How to remove shard with data withoud loosing them? -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-removing-shard-how-to-not-loose-data-tp4032138.html Sent from the Solr - User mailing list archive at Nabble.com.
auto warm up cache and new data
Please help me understand that. What wil happen if if have cached data and thay change after comit and i have autowarm set up. Old cached data will be still accesible in cache so i will get old data? That means if autowarm copy all needed data to new cache probably i will never see new data? Cache in solr expire only with searcher going down right? -- View this message in context: http://lucene.472066.n3.nabble.com/auto-warm-up-cache-and-new-data-tp3940963p3940963.html Sent from the Solr - User mailing list archive at Nabble.com.
Different solr config under tomcat.
Is there a way to use different path for solrconfig.xml, like solrconfig_slave.xml for instance under tomcat? I dont want run cores. I know how to do it with cores, but i want to have single instance. Is there any parameter which i can use to say tomcat to use solrconfig_slave.xml? Pls help -- View this message in context: http://lucene.472066.n3.nabble.com/Different-solr-config-under-tomcat-tp3918465p3918465.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr 3.5 and indexing performance
Ok i found it. Its becouse of Hunspell which now is in solr. Somehow when im using it by myself in 3.4 it is a lot of faster then one from 3.5. Dont know about differences, but is there any way i use my old Google Hunspell jar? -- View this message in context: http://lucene.472066.n3.nabble.com/solr-3-5-and-indexing-performance-tp3766653p3769139.html Sent from the Solr - User mailing list archive at Nabble.com.
solr 3.5 and indexing performance
Hello,
I wanted to switch to new version of solr, exactelly to 3.5 but im getting
big drop of indexing speed.
I'm using 3.1 and after few tests i discower that 3.4 do it a lot of better
then 3.5
My schema is really simple few field using text type field
/fieldType name=text class=solr.TextField
positionIncrementGap=100 autoGeneratePhraseQueries=true
analyzer type=index
tokenizer class=solr.WhitespaceTokenizerFactory/
filter class=solr.StopFilterFactory
ignoreCase=true
words=stopwords.txt
enablePositionIncrements=true
/
filter class=solr.WordDelimiterFilterFactory
generateWordParts=1 generateNumberParts=1 catenateWords=1
catenateNumbers=1 catenateAll=0 splitOnCaseChange=1/
filter class=solr.LowerCaseFilterFactory/
filter class=solr.KeywordMarkerFilterFactory
protected=protwords.txt/
filter class=solr.HunspellStemFilterFactory
dictionary=pl_PL.dic affix=pl_PL.aff/
filter class=solr.ASCIIFoldingFilterFactory/
/analyzer
analyzer type=query
tokenizer class=solr.WhitespaceTokenizerFactory/
filter class=solr.SynonymFilterFactory synonyms=synonyms.txt
ignoreCase=true expand=true/
filter class=solr.StopFilterFactory
ignoreCase=true
words=stopwords.txt
enablePositionIncrements=true
/
filter class=solr.WordDelimiterFilterFactory
generateWordParts=1 generateNumberParts=1 catenateWords=0
catenateNumbers=0 catenateAll=0 splitOnCaseChange=1/
filter class=solr.LowerCaseFilterFactory/
filter class=solr.KeywordMarkerFilterFactory
protected=protwords.txt/
filter class=solr.HunspellStemFilterFactory
dictionary=pl_PL.dic affix=pl_PL.aff/
filter class=solr.ASCIIFoldingFilterFactory/
/analyzer
/fieldType
/
All data and configuration are the same, same schema, solrconfig, same
jetty.
*SOLR 3.5*
/Feb 22, 2012 3:40:33 PM org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1
commit{dir=/vol/home/mciurla/proj/solr/accordion3.5/example/solr/data/index,segFN=segments_bl,version=1329831219365,generation=417,filenames=[_a5.fdx,
_52.fdx, _aq.frq,
_a5.fdt, _cr.nrm, _52.fnm, _a5.prx, segments_bl, _52.fdt, _7k.tii, _cr.frq,
_a5.tis, _cr.fdt, _a5.nrm, _cr.prx, _cp.prx, _cr.fdx, _cn.nrm, _52.tvf,
_cp.fnm, _co.tii, _52.tvd, _8
o.tvx, _co.tis, _8o.tii, _a5.fnm, _8o.tvd, _7k.tis, _8o.tvf, _bb.tis,
_7k.fdx, _7k.fdt, _7k.frq, _bb.tii, _cn.frq, _co.prx, _aq.tii, _cq.fdx,
_52.tii, _cm.tis, _cq.fdt, _aq.tis,
_52.tis, _aq.tvx, _co.nrm, _bb.prx, _cm.tii, _cr.fnm, _aq.tvf, _bb_3.del,
_aq.tvd, _cm.frq, _cp.nrm, _cq.tis, _52.prx, _cn.tis, _8o.fnm, _cl.nrm,
_cl.fnm, _a5.tii, _cn.tii, _cq
.tii, _cp.tis, _cp.fdt, _cl.fdt, _cl.prx, _aq.fdt, _cl.fdx, _cr.tis,
_co.frq, _7k.fnm, _cq.frq, _bb.fnm, _cr.tii, _cp.fdx, _cp.tii, _aq.fdx,
_cq.tvd, _8o.fdt, _cq.tvf, _52.nrm,
_8o.nrm, _aq.fnm, _8o.prx, _co.tvd, _cq.tvx, _52.frq, _bb.nrm, _bb.fdt,
_cp.tvf, _a5.tvx, _cp.tvd, _cn.tvx, _7k.nrm, _bb.fdx, _cm.tvx, _cm.fdx,
_cl.tvf, _cp.tvx, _co.fdx, _cl.tv
d, _cn.tvf, _a5.frq, _cm.fdt, _a5.tvf, _co.fdt, _a5.tvd, _cp.frq, _cn.fdt,
_cm.nrm, _7k_d.del, _cn.fdx, _52_1e.del, _7k.prx, _8o.fdx, _cn.prx, _cl.tis,
_cq.nrm, _7k.tvx, _cq.prx
, _cn.tvd, _cl.tii, _cm.fnm, _7k.tvd, _cm.prx, _8o.tis, _cm.tvf, _52.tvx,
_7k.tvf, _cl.tvx, _cm.tvd, _a5_9.del, _bb.tvf, _bb.tvd, _cr.tvd, _co.tvf,
_bb.tvx, _cr.tvf, _co.fnm, _a
q.prx, _cl.frq, _cq.fnm, _aq_9.del, _bb.frq, _8o.frq, _aq.nrm, _co.tvx,
_8o_t.del, _cr.tvx, _cn.fnm, _cl_6.del]
Feb 22, 2012 3:40:33 PM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1329831219365
Feb 22, 2012 3:40:47 PM org.apache.solr.update.processor.LogUpdateProcessor
finish
*INFO: {add=[2271874, 2271875, 2271876, 2271877, 2271878, 2271879, 2271880,
2271881, ... (100 adds)]} 0 14213*
Feb 22, 2012 3:40:47 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={} status=0 QTime=14213
/
when on solr 3.4
/Feb 22, 2012 3:42:56 PM org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1
commit{dir=/vol/home/mciurla/proj/solr/accordion3.4/example/solr/data/index,segFN=segments_29,version=1329918470592,generation=81,filenames=[_2b.tvf,
_2c.tvx, _2d.tvf, _2f.tvx, _2d.tvd, _15.prx, _15.frq, _2b.tvd, _2c.nrm,
_20.fnm, _2b.tvx, _2c.fdx, _2c.prx, _2f.tii, _2f.tvf, _20.tvx, _2b.fnm,
_2c.fdt, _2d.tis, _15.fdt, _20.frq, _2d.tvx, _2f.tvd, _15.fdx, _15.fnm,
_2c.tvf, _2e.frq, _2e.prx, _2c.tvd, _2b.frq, _20.tvd, _2c.fnm, _20.tvf,
_2e.tvf, _2e.nrm, _20.tis, _2b.prx, _20.tii, _2e.tvd, _15.tis, _2f.frq,
_15.tii, _2e.tvx, _2e.tii, _2c.tis, _2c.frq, _2e.fdx, _2f.prx, _2f.fnm,
_15.tvx, _2e.fdt, _15.tvf, _2b.tis, _2c.tii, _2d.prx, _2d.fnm, _20.fdx,
_2b.tii, _2e.tis, _20.fdt, _2d.frq, _2b.nrm, _15.tvd, _15_b.del, _2b.fdt,
_2f.nrm, _2d.fdx,
Re: solr 3.5 and indexing performance
i got it all commnented in updateHandler, im prety sure there is no default autocommit updateHandler class=solr.DirectUpdateHandler2 iorixxx wrote I wanted to switch to new version of solr, exactelly to 3.5 but im getting big drop of indexing speed. Could it be autoCommit configuration in solrconfig.xml? -- View this message in context: http://lucene.472066.n3.nabble.com/solr-3-5-and-indexing-performance-tp3766653p3767843.html Sent from the Solr - User mailing list archive at Nabble.com.
SOLR
Is there any way to score not being affected by duplicated input in query? When i have record with field title: The GIRL with the dragon tattoo If query is: girl it get less score then girl girl girl. It find word in the same position, why score is growing? I need it to know if record i found is close to user query. Best would be for me to know how much % match with that title field. -- View this message in context: http://lucene.472066.n3.nabble.com/SOLR-tp3731831p3731831.html Sent from the Solr - User mailing list archive at Nabble.com.
