Re: SolrCloud and exernal file fields

Mikhail I appreciate your input, it's very useful :) On Wed, Nov 21, 2012 at 6:30 AM, Mikhail Khludnev < mkhlud...@griddynamics.com> wrote: > Martin, > This deployment seems a little bit confusing to me. You have 16-way fairy > virtual "box", and send 16 request for really heavy operation at the

Re: is it possible to save the search query?

Hi Mark, Thanks a lot for your reply. I do not have much knowledge regarding xslt. Could you kindly specify how do i use xslt to extract relevant information. Also how can i drive solr response to a file. Any guidance regarding this would help. Thanks and regards, Romita From: geeky2 To

Re: Using SolrCloud for update often lose response and get 503 error

Hi 200 threads sounds excessive. Maybe you are DOSing your Solr cluster. Try with fewer threads and increase until you get to the point of diminishing returns. You may simply be consuming all available container threads, which you could increase, but that may not make indexing any faster. Otis

Re: Weird Behaviour on Solr 5x (SolrCloud)

well... i find a way to avoid this... i dont know if it is the correct way or i am simply bypassing the problem instead of fixing it.. when i delete the data/ folders contents before restarting, it can get the index information from cloud without any problem... so it is the way how solrcloud work

Re: SolrCloud and exernal file fields

On Wed, Nov 21, 2012 at 2:07 AM, Martin Koch wrote: > I'm not sure about the mmap directory or where that > would be configured in solr - can you explain that? > You can check it at Solr Admin/Statistics/core/searcher/stats/readerDir should be org.apache.lucene.store.MMapDirectory -- Sincerel

Re: Weird Behaviour on Solr 5x (SolrCloud)

is this because of zookeeper's load balancer or something like that? because the results are returning totally randomly... - Zeki ama calismiyor... Calissa yapar... -- View this message in context: http://lucene.472066.n3.nabble.com/Weird-Behaviour-on-Solr-5x-SolrCloud-tp4021219p4021500.ht

Re: relative token count in a query result

Hello, Have you tried to implement your own Collector and pass it into IndexSearch.search()? Collector has a reference to the current scorer, and therefore presumably can access tf info from TermQueryScorer: org.apache.lucene.search.TermScorer.freq(). Then collector can just sum these tfs. Be awa

Re: SolrCloud and exernal file fields

Martin, This deployment seems a little bit confusing to me. You have 16-way fairy virtual "box", and send 16 request for really heavy operation at the same moment, it does not surprise me that you loosing it for some period of time. At that time you should have more than 16 in load average metrics.

SolrCloud across datacenter

hi I m looking into using single SolrCloud spanning multiple data centers. And it would be mainly used as key, value store (NOSQL) type install. We would like to have flexibility to take out the entire data center for entire day or so. Also our app design is such that delay in soft commit of reco

Re: Replication Backup

Hi Eva, I think you just need to configure the Solr instance on your Windows and point it to your Solr master. It will then copy the index from the master periodically. Please see http://search-lucene.com/?q=solr+replication+backup for some more info about doing backups - you don't need rsync. O

Re: SolrCloud(5x) - Detects all of the Solr insrances on a machine

Mark Miller-3 wrote > It must be passes with -D as a system prop with the default setup. > That feeds hostPort in solr.xml. If you use etc/jetty.xml, but sure to > still pass it on the cmd line or also put the port in solr.xml for > hostPort. > - Mark basically I should add the port info to the so

Re: SOLR USING 100% percent CPU and not responding after a while

Hi, I looked at the spreadsheet and the graph and it looks like that's memory for the whole OS. What you want to look at is the JVM heap and GC counts and timings there, esp. when you say performance sinks. At that time also look at your query, filter, and document caches and see if evictions go

Re: SolrCloud(5x) - Detects all of the Solr insrances on a machine

It must be passes with -D as a system prop with the default setup. That feeds hostPort in solr.xml. If you use etc/jetty.xml, but sure to still pass it on the cmd line or also put the port in solr.xml for hostPort. - Mark On Tue, Nov 20, 2012 at 10:11 PM, deniz wrote: > Mark Miller-3 wrote >> Ho

Re: SolrCloud(5x) - Detects all of the Solr insrances on a machine

Mark Miller-3 wrote > How are you specifying the port? I don't see jetty.port in there. That > is critical - it sets the hostPort in solr.xml. > - Mark setting it with -Djetty.port=blabla or directly in etc/jetty.xml - Zeki ama calismiyor... Calissa yapar... -- View this message in context:

Re: SolrCloud(5x) - Detects all of the Solr insrances on a machine

How are you specifying the port? I don't see jetty.port in there. That is critical - it sets the hostPort in solr.xml. - Mark On Tue, Nov 20, 2012 at 9:42 PM, deniz wrote: > so another test result: > > i have set up a similar environment on another virtual machine which is > running on the same

Re: SolrCloud(5x) - Detects all of the Solr insrances on a machine

so another test result: i have set up a similar environment on another virtual machine which is running on the same hard machine with my previous example... so basically my standalone solr is running on virtual1:8983 and i set up 3 solr instances which are on virtual2:8995,8996,8997... those vir

Re: Weird negative query responses

On 11/20/2012 1:27 PM, Ravi Solr wrote: Can somebody kindly clarify how negative queries work. I having this weird issue with an analyzed text field. I want to find all docs which don't have a value in the 'body' field. The field definition and query i am using is given below. Can somebody tell m

Re: Custom ranking solutions?

Hi Dan, Thanks! I'm using boost query to solve this problem. Floyd 2012/11/21 Daniel Rosher > Hi > > The product function query needs a valuesource, not the pseudo score field. > > You probably need something like (with Solr 4.0): > > q={!lucene}*:*&sort=product(query($q),2) desc,score > de

Re: Ranking by sorting score and rankingField better or by product(score, rankingField)?

Hi Chris, Thanks! Before your great suggestions, I give up using function query to calculate product of score and rankingField and using exactly the same with your boost query solution. Of course it works fine. The next step will be design suitable function to output a ranking value that also cons

Re: SolrCloud(5x) - Detects all of the Solr insrances on a machine

well here are more details about my starting commands This is the standalone SolrServer: (port 8983) java -server -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:+UseCompressedStrings -Dcom.sun.management.jmxremote -d64 -Xmx4096m -Dcom.sun.management.jmxremote.port= -Dcom.sun.manage

Re: is there a way to prevent abusing rows parameter

You could set an "invariant" parameter value, but that would mean they can't give an override. It might be a useful addition to Solr to have a maximum value (specified as an invariant). You could also simply add your own Solr "search component" that checked and maxed the &rows. -- Jack Kru

is there a way to prevent abusing rows parameter

silly question is there any configuration value I can set to prevent someone from entering a bad value for the rows parameter? ie to prevent something like "&rows=1" from crashing my servers? the server I am looking at is a solr v3.6 -- View this message in context: http://lucene.47

Re: SolrCloud(5x) - Detects all of the Solr insrances on a machine

You would have to be giving it the address of zookeeper - otherwise how would the other nodes know about it? Something is fishy. - Mark On Nov 20, 2012, at 3:56 AM, deniz wrote: > Hello, > > I am running a Solr instance (4.0), without invoking anything about > zookeeper and solrcloud, as a s

Re: Can Solr v1.4 and v4.0 co-exist in Tomcat?

Thanks, that helped with a different issue I was having. However the Admin UI issue is a known issue with IE9 (and possibly IE8) https://issues.apache.org/jira/browse/SOLR-3876 https://issues.apache.org/jira/browse/SOLR-3841 basically IE is not supported for the Solr Admin UI at this time. I insta

Re: SolrCloud: CloudSolrServer Zookeeper error.

Thanks, Mark! I´ll try tunning timeout settings and then check out if everything is allright. Regards, - Luis Cappa. 2012/11/20 Mark Miller > Take a look at the FAQ for SolrCloud: > http://wiki.apache.org/solr/SolrCloud#FAQ > > - Mark > > On Nov 20, 2012, at 7:00 AM, Luis Cappa Banda wrote

Re: SolrCloud and exernal file fields

Mikhail PSB On Tue, Nov 20, 2012 at 7:22 PM, Mikhail Khludnev < mkhlud...@griddynamics.com> wrote: > Martin, > > Please find additional question from me below. > > Simone, > > I'm sorry for hijacking your thread. The only what I've heard about it at > recent ApacheCon sessions is that Zookeeper

Re: [Solrj] How can I get unique field name?

There is no absolute requirement that a Solr schema have a unique key field, so you could get a null value for the field. -- Jack Krupansky -Original Message- From: zakaria benzidalmal Sent: Tuesday, November 20, 2012 6:02 AM To: solr-user@lucene.apache.org Subject: Re: [Solrj] How ca

Re: solr4 MULTIPOLYGON search syntax

Thanks to both of you. I got it working as follows after your helpful posts. I get this polygon search working. fq=geo:"Intersects(POLYGON((149.4023 -34.6072, 149.4023 -34.8690, 149.9022 -34.8690, 149.9022 -34.6072, 149.4023 -34.6072)))" AND I can get this search working (as in results returned)

relative token count in a query result

Hello, earlier, I was trying to retrieve the total token count per index http://lucene.472066.n3.nabble.com/how-to-retrieve-total-token-count-per-collection-index-td4000161.html . now, I would like to have a token (word) count within the document-set (resulting of a query), both for the matchi

Re: solr4 MULTIPOLYGON search syntax

Hi jend, You need an extra layer of parenthesis for MultiPolygon. I see that you opened up with MULTIPOLYGON((… instead of MULTIPOLYGON(((… Of course ensure you balance your parenthesis. For examples of WKT, see Wikipedia: http://en.wikipedia.org/wiki/Well-known_text ~ David On Nov 19, 2

Re: SOLR USING 100% percent CPU and not responding after a while

Hello! How about some information how Garbage Collector works during the time when you have Solr unresponsive ? -- Regards, Rafał Kuć Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch > Hello, > We were doing fine with memory, comparitively. This was in a one > hour sp

Fwd: Scotas Newsletter #1

Hi all, i think that this email could be interesting for you. See you! Julian -- Forwarded message -- From: Scotas newsletter Date: 2012/11/20 Subject: Newsletter #1 To: Julian ** Is this email not displaying correctly? View it in your browser

Re: Reduce QueryComponent prepare time

Markus, It seems you faced the challenge of optimizing complex eDisMax code for your particular usecase, which is not so common. I can not help with these coding, just can share some experience: we have mind blowing queries too - they spawns many fields and enumerate many phrase shingles. We have

Re: SolrCloud and exernal file fields

Martin, Please find additional question from me below. Simone, I'm sorry for hijacking your thread. The only what I've heard about it at recent ApacheCon sessions is that Zookeeper is supposed to replicate those files as configs under solr home. And I'm really looking forward to know how it work

Re: Solr 4:How to call a updateRequestProcessorChain during the /dataimport?

: Following is the my dataimport handler, which is already having : 'update.chain'. I think I can't give more than one update.chain in this Correct, you can only specify a single chain when doing an update. You have to decide which parts of your existing "dedupe" chain you want to use inconjunc

Re: Ranking by sorting score and rankingField better or by product(score, rankingField)?

: But the sort=product(score, rankingField) is not working in my test. What : probably wrong? the problem is "score" is not a field or a function -- Solr doesn't know exactly what "score" you want it to use there (scores from which query?) You either need to refrence the query in the function (

Replication Backup

Hi All, It takes a long time to reindex our database and so I'd like to be able to backup the solr server. I'm running solr 3.6.1 using tomcat on debian squeeze and I'd like to be able to backup to a windows server that contains the rest of our backups. There isn't much free space on the solr ser

Re: SolrCloud: CloudSolrServer Zookeeper error.

Take a look at the FAQ for SolrCloud: http://wiki.apache.org/solr/SolrCloud#FAQ - Mark On Nov 20, 2012, at 7:00 AM, Luis Cappa Banda wrote: > Hello everyone, > > I have developed a NRT Java app that indexes documents using a > CloudSolrServer instance. During my tests I have a SolrCloud compos

Need a link to working SolrJ java web app example

Anyone have an example of a working web application using Solr or SolrJ? I am looking for a working example that Queries data. Would be helpful if it was written using Spring, Struts or JSF but not a requirement. -- View this message in context: http://lucene.472066.n3.nabble.com/Need-a-link-

Re: High Slave CPU Intermittently After Replication

Re: Can Solr v1.4 and v4.0 co-exist in Tomcat?

Our setup generally follows these instructions: http://wiki.apache.org/solr/SolrTomcat#Multiple_Solr_Webapps BTW, we're running on Tomcat 6. You'll definitely want to undo your changes to the Tomcat lib directory since you don't want anything Solr-related shared across your web apps. On Nov 2

RE: Additional field informations?

You could use dynamicFields to map the language as well german title english title etc... Yorgos Mamakis Backend Developer www.europeana.eu Email: yorgos.mama...@kb.nl Skype: fuabap -Original Message- From: Rafal Kuc [mailto:r@solr.pl] Sent: Tue 11/20/2012 5:26 PM To: solr-use

Re: Additional field informations?

Hello! I think that's not possible out of the box in Solr. -- Regards, Rafał Kuć Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch > Hello all, > We import xml documents to solr with solrj. We use xsl to proccess the > "objects" to fields. > We got the language inform

Re: Custom ranking solutions?

Hi The product function query needs a valuesource, not the pseudo score field. You probably need something like (with Solr 4.0): q={!lucene}*:*&sort=product(query($q),2) desc,score desc&fl=score,_score_:product(query($q),2),[explain] Cheers, Dan On Tue, Nov 20, 2012 at 2:29 AM, Floyd Wu wrote

Re: Additional field informations?

Hi, Have a look at DocTransformers http://wiki.apache.org/solr/DocTransformers and ExplainAugmenterFactory as an example Cheers, Dan On Tue, Nov 20, 2012 at 3:08 PM, Sebastian Hofmann wrote: > Hello all, > We import xml documents to solr with solrj. We use xsl to proccess the > "objects" to f

Re: Ranking by sorting score and rankingField better or by product(score, rankingField)?

Thanks for the report, Floyd, the Solr Reference Guide has been changed to say the default is $SOLR_HOME/data rather than $SOLR_HOME/data/index. Currently the ExternalFileField javadoc says: S

Re: solr4 MULTIPOLYGON search syntax

I think you may be missing a set of parenthesis, instead of MULTIPOLYGON((point,point,point)),((point,point,point)) you need MULTIPOLYGON(((point,point,point)),((point,point,point))) -- View this message in context: http://lucene.472066.n3.nabble.com/solr4-MULTIPOLYGON-search-syntax-tp4021199p

Re: [Solrj] is there any utility class to get core's search base URL?

I explain, I am writing a custom response writer. in the resulting xml I would like to add to each result element a solr fatch link for eg. http://host:port/solr/core/select?q=id: How can I get the search url in a response writer class? Is there any way to build this url? Regards. __

Additional field informations?

Hello all, We import xml documents to solr with solrj. We use xsl to proccess the "objects" to fields. We got the language informations in our "objects". After xsl out Documents look like this: ... german title english title french title ...

Re: Can Solr v1.4 and v4.0 co-exist in Tomcat?

I renamed the 4.0 .war file solr4.war and put it in the \lib directory of Tomcat (where solr.war already exists for version 1.4.1). I have a new core created using the solr example that came with 4.0. It looks like I can run queries and do a few other things using the URL, but the Solr Admin UI do

Re: is it possible to save the search query?

Hello, i think you are asking two questions here - i'll see if i can give you some simple examples for both 1) how can i pull data from a solr search result set and compare it to another for analysis? one way - might be to drive the results in to files and then use xslt to extract relevant infor

RE: Reduce QueryComponent prepare time

Hi, Profiling pointed me directly to the method i already suspected: ExtendedDismaxQParser.parse(). I added manual timers in parts of the method and made sure the timers add up to the QueryComponent prepare time. After starting Solr there's one small part taking almost 100ms on a fast machine w

Re: Apply clustering to field names?

(Sorry for spamming) It does not solve the whole issue though. I'm still looking for a way to "cluster the terms of a field". 2012/11/20 Per Fredelius > I see now that the TermsComponent > supply > a lot of the data I was looking for. > > // Per > >

Re: Apply clustering to field names?

I see now that the TermsComponent supply a lot of the data I was looking for. // Per 2012/11/20 Per Fredelius > Hello Solr users, > > I'm new at using Solr, working with it for my thesis. I have a > configuration up and running, doing the basic stuff,

Re: SolrCloud and exernal file fields

Hi Mikhail Please see answers below. On Tue, Nov 20, 2012 at 12:28 PM, Mikhail Khludnev < mkhlud...@griddynamics.com> wrote: > Martin, > > Thank you for telling your own "war-story". It's really useful for > community. > The first question might seems not really conscious, but would you tell me

Re: DIH delta import handle sql

On 20 November 2012 19:09, Ramo Karahasan wrote: [...] > is delta-import using the same SQL statement as the full-import ? in the > delta-import I sometimes get wrong categories to my product. If I run the > full-import directly afterwards, the categories are set correct. It depends on how you ha

DIH delta import handle sql

Hi, is delta-import using the same SQL statement as the full-import ? in the delta-import I sometimes get wrong categories to my product. If I run the full-import directly afterwards, the categories are set correct. Another thing is, that indexing with the same command returns sometimes 1.6

Re: Optimizing SOLR

On 20 November 2012 14:59, gauravmunjal wrote: > Hi, > > I used the example from the tutorial of Apache SOLR 4.0 and implemented the > same on around 8 million documents. The document has three fields namely > title, id and features. There 8 million docs are basically html files > converted to XML

Optimizing SOLR

Hi, I used the example from the tutorial of Apache SOLR 4.0 and implemented the same on around 8 million documents. The document has three fields namely title, id and features. There 8 million docs are basically html files converted to XML files where title contains the title of the html page and

SolrCloud: CloudSolrServer Zookeeper error.

Hello everyone, I have developed a NRT Java app that indexes documents using a CloudSolrServer instance. During my tests I have a SolrCloud composed by only one shard and two Solr server instances. During the several tests I did found just in one case the following error: * * *12-11-20 12:54:03,6

Re: SolrCloud and exernal file fields

Martin, Thank you for telling your own "war-story". It's really useful for community. The first question might seems not really conscious, but would you tell me what blocks searching during EFF reload, when it's triggered by handler or by listener? I don't really get the sentence about sequential

[Solrj] is there any utility class to get core's search base URL?

Hi all, I have a simple question to ask: Is there any utility class to get core's search base URL? Or: a utility class to build core search query URLs? Regards. __ Zakaria BENZIDALMAL

Re: [Solrj] How can I get unique field name?

Thank you Mikhail, Yes it does. I can access it throu the SolrQueryRequest object. this.uniqueKeyFieldName = req.getSchema().getUniqueKeyField().getName(); 2012/11/20 Mikhail Khludnev > Hello Zakharia, > > org.apache.solr.schema.IndexSchema.getUniqueKeyField() > Does it help? > > > > On Tue,

Re: [Solrj] How can I get unique field name?

Hello Zakharia, org.apache.solr.schema.IndexSchema.getUniqueKeyField() Does it help? On Tue, Nov 20, 2012 at 2:40 PM, zakaria benzidalmal wrote: > Hi all, > > I am writing a custom query response writer and I would like to handle the > unique key field without knowing his actual name to stay g

[Solrj] How can I get unique field name?

Hi all, I am writing a custom query response writer and I would like to handle the unique key field without knowing his actual name to stay generic. My question is: how can I get the uniqueKey fieldname of a result document? Regards. __ Zakaria BENZIDALMAL

Re: CloudSolrServer and LBHttpSolrServer: setting BinaryResponseParser and BinaryRequestWriter.

Hello, Sandopolus! First let me thank you a lot your last e-mail: it showed me the way to resolve this bug/problem. I´ve implemented a custom BinaryLBHttpSolrServer, as you suggested, but with your implementation I had the same problems. I think that it is due to HttpClientUtil.createClient(null).

Re: is it possible to save the search query?

Hi Amit, Could you please explain more on how to use a user cookie to construct a session. Thanks and regards, Romita From: Amit Nithian To: "solr-user@lucene.apache.org" , Date: 11/08/2012 06:21 PM Subject:Re: is it possible to save the search query? Are you trying to do

Re: Solr filter using data from the database

Hello Colleagues, Recently I had talk at ApacheCon about this problem. Both proposed approaches are definitely work. Frequent updates http://goo.gl/xGPMUsometimes cost too much. Filters http://goo.gl/mMvRQ works slow starting from thousand of keys and might have low hit ratio. One of the promising

Re: SolrCloud and exernal file fields

Solr 4.0 does support using EFFs, but it might not give you what you're hoping fore. We tried using Solr Cloud, and have given up again. The EFF is placed in the parent of the index directory in each core; each core reads the entire EFF and picks out the IDs that it is responsible for. In the cu

Re: Inserting many documents and update relations

Hello, I propose to join docs externally eg in tiny rdbms, just put ids there and keep content in files. Then DIH, I believe and only believe, should be able to build full document representation with joined entities. As an alternative you can index document as is with id-references between them i

SolrCloud(5x) - Detects all of the Solr insrances on a machine

Hello, I am running a Solr instance (4.0), without invoking anything about zookeeper and solrcloud, as a standalone server on a machine. then for testing Solr 5x trunk, i have set 2 Solr (5x) instances, running with -DzkHost= someaddress:port and when i check zookeeper logs, i can see that the s