Solr 8.5.2 indexing issue
Hi,
I have the following document which fails to get indexed.
{
"asset_id":"add-ons:576deefef7453a9189aa039b66500eb2",
"reference_url":"modeling-a-high-speed-backplane-part-3-4-port-s-parameters-to-differential-tdr-and-tdt.html"}
I am not sure what is so special about the content in the reference_url
field.
reference_url field is defined as follows in schema:
It throws the following error.
Status:
{"data":{"responseHeader":{"status":400,"QTime":18},"error":{"metadata":["error-class","org.apache.solr.common.SolrException","root-error-class","java.lang.IndexOutOfBoundsException"],"msg":"Exception
writing document id add-ons:576deefef7453a9189aa039b66500eb2 to the index;
possible analysis
error.","code":400}},"status":400,"config":{"method":"POST","transformRequest":[null],"transformResponse":[null],"jsonpCallbackParam":"callback","headers":{"Content-type":"application/json","Accept":"application/json,
text/plain, */*","X-Requested-With":"XMLHttpRequest"},"data":"[{\n
\"asset_id\":\"add-ons:576deefef7453a9189aa039b66500eb2\",\n
\"reference_url\":\"modeling-a-high-speed-backplane-part-3-4-port-s-parameters-to-differential-tdr-and-tdt.html\"}]","url":"add-ons/update","params":{"wt":"json","_":1593304427428,"commitWithin":1000,"overwrite":true},"timeout":1},"statusText":"Bad
Request","xhrStatus":"complete","resource":{"0":"[","1":"{","2":"\n","3":"
","4":" ","5":" ","6":" ","7":" ","8":" ","9":" ","10":"
","11":"\"","12":"a","13":"s","14":"s","15":"e","16":"t","17":"_","18":"i","19":"d","20":"\"","21":":","22":"\"","23":"a","24":"d","25":"d","26":"-","27":"o","28":"n","29":"s","30":":","31":"5","32":"7","33":"6","34":"d","35":"e","36":"e","37":"f","38":"e","39":"f","40":"7","41":"4","42":"5","43":"3","44":"a","45":"9","46":"1","47":"8","48":"9","49":"a","50":"a","51":"0","52":"3","53":"9","54":"b","55":"6","56":"6","57":"5","58":"0","59":"0","60":"e","61":"b","62":"2","63":"\"","64":",","65":"\n","66":"
","67":" ","68":" ","69":" ","70":" ","71":" ","72":" ","73":"
","74":"\"","75":"r","76":"e","77":"f","78":"e","79":"r","80":"e","81":"n","82":"c","83":"e","84":"_","85":"u","86":"r","87":"l","88":"\"","89":":","90":"\"","91":"m","92":"o","93":"d","94":"e","95":"l","96":"i","97":"n","98":"g","99":"-","100":"a","101":"-","102":"h","103":"i","104":"g","105":"h","106":"-","107":"s","108":"p","109":"e","110":"e","111":"d","112":"-","113":"b","114":"a","115":"c","116":"k","117":"p","118":"l","119":"a","120":"n","121":"e","122":"-","123":"p","124":"a","125":"r","126":"t","127":"-","128":"3","129":"-","130":"4","131":"-","132":"p","133":"o","134":"r","135":"t","136":"-","137":"s","138":"-","139":"p","140":"a","141":"r","142":"a","143":"m","144":"e","145":"t","146":"e","147":"r","148":"s","149":"-","150":"t","151":"o","152":"-","153":"d","154":"i","155":"f","156":"f","157":"e","158":"r","159":"e","160":"n","161":"t","162":"i","163":"a","164":"l","165":"-","166":"t","167":"d","168":"r","169":"-","170":"a","171":"n","172":"d","173":"-","174":"t","175":"d","176":"t","177":".","178":"h","179":"t","180":"m","181":"l","182":"\"","183":"}","184":"]"}}
Re: Solr heap Old generation grows and it is not recovered by G1GC
Hi, Just summarizing: I've experimented using different sized of filtercache and documentcache, after removing any maxRamMB. Now the heap seems to behave as expected, that is, it grows, then GC (not full one) kicks in multiple times and keep the used heap under control. eventually full GC may kick in and the size goes down a little more. Previously, when I had maxRamMB specified, the heap would grow considerably (for a search returning about 300K docs) and after that it would not go down again (and those docs were never again requested). This did not work well. I looked at the heapdump and saw all the caches (filter, document, one type per core), so if you have multiple shards you may have to be very careful not to increase the cache sizes, because they apply to each core. I still think there is something strange when a search returns a large number of docs - the G1GC didn't seem to handle that very well in some cases (when maxRamMB was specified), but that may be the symptom and not the cause. Thanks for the help. Reinaldo On Sat, Jun 27, 2020 at 4:29 AM Zisis T. wrote: > Hi Reinaldo, > > Glad that helped. I've had several sleepless nights with Solr clusters > failing spectacularly in production due to that but I still cannot say that > the problem is completely away. > > Did you check in the heap dump if you have cache memory leaks as described > in https://issues.apache.org/jira/browse/SOLR-12743? > > Say you have 4 cache instances (filterCache, documentCache etc) per core > and > you have 5 Solr cores you should not see more than 20 CaffeineCache > instances in your dump. > > Unfortunately I still cannot determine what exactly triggers this memory > leak although since I removed the maxRAMMB setting I've not seen similar > behavior for more than a month now in production. > > The weird thing is that I was running on Solr 7.5.0 for quite some time > without any issues and it was at some point in time that those problems > started appearing... > > > > -- > Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html >
Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr
Rather than getting rid of the terminology, we should get rid of the standalone mode Solr altogether. I totally understand that SolrCloud is broken in many ways today, but we should attempt to fix it and have it as the only mode in Solr. On Wed, 24 Jun, 2020, 8:17 pm Mike Drob, wrote: > Brend, > > I appreciate that you are trying to examine this issue from multiple sides > and consider future implications, but I don’t think that is a stirring > argument. By analogy, if we are out of eggs and my wife asks me to go to > the store to get some, refusing to do so on the basis that she might call > me while I’m there and also ask me to get milk would not be reasonable. > > What will come next may be an interesting question philosophically, but we > are not discussing abstract concepts here. There is a concrete issue > identified, and we’re soliciting input in how best to address it. > > Thank you for the suggestion of "guide/follower" > > Mike > > On Wed, Jun 24, 2020 at 6:30 AM Bernd Fehling < > bernd.fehl...@uni-bielefeld.de> wrote: > > > I'm following this thread now for a while and I can understand > > the wish to change some naming/wording/speech in one or the other > > programs but I always get back to the one question: > > "Is it the weapon which kills people or the hand controlled by > > the mind which fires the weapon?" > > > > The thread started with slave - slavery, then turned over to master > > and followed by leader (for me as a german... you know). > > What will come next? > > > > And more over, we now discuss about changes in the source code and > > due to this there need to be changes to the documentation. > > What about the books people wrote about this programs and source code, > > should we force this authors to rewrite their books? > > May be we should file a request to all web search engines to reject > > all stored content about these "banned" words? > > And contact all web hosters about providing bad content. > > > > To sum things up, within my 40 years of computer science and writing > > programs I have never had a nanosecond any thoughts about words > > like master, slave, leader, ... other than thinking about computers > > and programming. > > > > Just my 2 cents. > > > > For what it is worth, I tend to guide/follower if there "must be" any > > changes. > > > > Bernd > > >
Re: solrj - get metrics from all nodes
For people who is also looking for the solution - you can append "node=node_name" in metrics request to get specific data of node. If anyone know how to get the data if all the nodes together, please kindly share, thanks. Regards, Chien -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Solr heap Old generation grows and it is not recovered by G1GC
Hi Reinaldo, Glad that helped. I've had several sleepless nights with Solr clusters failing spectacularly in production due to that but I still cannot say that the problem is completely away. Did you check in the heap dump if you have cache memory leaks as described in https://issues.apache.org/jira/browse/SOLR-12743? Say you have 4 cache instances (filterCache, documentCache etc) per core and you have 5 Solr cores you should not see more than 20 CaffeineCache instances in your dump. Unfortunately I still cannot determine what exactly triggers this memory leak although since I removed the maxRAMMB setting I've not seen similar behavior for more than a month now in production. The weird thing is that I was running on Solr 7.5.0 for quite some time without any issues and it was at some point in time that those problems started appearing... -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
