Re: Fault tolerant Solr replication architecture

2012-05-21 Thread findbestopensource
Hi Parvin, Fault tolerant architecture is something you need to decide on your requirement. At some point of time there may require some manual intervention to recover from crash. You need to see how much percentage you could support fault tolerant. It certainly may not be 100. We could handle

Re: using Carrot2 custom ITokenizerFactory

2012-05-21 Thread Stanislaw Osinski
Hi Koji, Dawid came up with a simple fix for this, it's committed to trunk and 3.6 branch. Staszek On Sun, May 20, 2012 at 5:15 PM, Koji Sekiguchi k...@r.email.ne.jp wrote: Hi Staszek, Thank you for the fix so quickly! As a trial, I set: str

Re: No Effect of omitNorms and omitTermFreqAndPositions when using MLT handler?

2012-05-21 Thread Ravish Bhagdev
Ahh, this is because I have to override DefaultSimilarity to turn off tf/idf scoring? But this will apply to all the fields and general search on text fields as well? Is there a way to apply custom similarity to specific field types or fields only? Is there no way of turning TF/IDF off without

org.apache.solr.common.SolrException: ERROR: [doc=null] missing required field: id

2012-05-21 Thread Tolga
Hi, I am getting this error: [doc=null] missing required field: id request: http://localhost:8983/solr/update?wt=javabinversion=2 at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:430) at

Re: No Effect of omitNorms and omitTermFreqAndPositions when using MLT handler?

2012-05-21 Thread Ravish Bhagdev
I found this: https://issues.apache.org/jira/browse/LUCENE-2236 So, it seems this feature is not supported in Solr 1.4 at all. Is there any possible work around? If not, I'll have to consider splitting my schema into two which will be quite a big change :( - Ravish On Mon, May 21, 2012 at

Re: org.apache.solr.common.SolrException: ERROR: [doc=null] missing required field: id

2012-05-21 Thread Michael Kuhlmann
Am 21.05.2012 12:07, schrieb Tolga: Hi, I am getting this error: [doc=null] missing required field: id [...] I've got this entry in schema.xml: field name=id type=string stored=true indexed=true/ What to do? Simply make sure that every document you're sending to Solr contains this id

Re: org.apache.solr.common.SolrException: ERROR: [doc=null] missing required field: id

2012-05-21 Thread Tolga
How do I verify it exists? I've been crawling the same site and it wasn't giving an error on Thursday. Regards, On 5/21/12 1:20 PM, Michael Kuhlmann wrote: Am 21.05.2012 12:07, schrieb Tolga: Hi, I am getting this error: [doc=null] missing required field: id [...] I've got this entry

Re: org.apache.solr.common.SolrException: ERROR: [doc=null] missing required field: id

2012-05-21 Thread Michael Kuhlmann
Am 21.05.2012 12:40, schrieb Tolga: How do I verify it exists? I've been crawling the same site and it wasn't giving an error on Thursday. It depends on what you're doing. Are you using nutch? -Kuli

Re: org.apache.solr.common.SolrException: ERROR: [doc=null] missing required field: id

2012-05-21 Thread Tolga
Yes. On 5/21/12 1:49 PM, Michael Kuhlmann wrote: Am 21.05.2012 12:40, schrieb Tolga: How do I verify it exists? I've been crawling the same site and it wasn't giving an error on Thursday. It depends on what you're doing. Are you using nutch? -Kuli

Re: Not able to use the highlighting feature! Want to return snippets of text

2012-05-21 Thread Ahmet Arslan
text:abstracthl=truehl.fl=textf.text.hl.snippets=2f.text.hl.fragsize=200debugQuery=true Three things to check: 1-) See your text field declared as suitable for highlighting. http://wiki.apache.org/solr/FieldOptionsByUseCase 2-) Increase hl.maxAnalyzedChars=Integer.MAX 3-) Increase

Re: Not able to use the highlighting feature! Want to return snippets of text

2012-05-21 Thread Jack Krupansky
Take a look at the /browse request handler in the example solrconfig.xml and compare how it does highlighting to what you are doing. There are a lot of little details, so maybe even one might be missing. Also, you can only highlight stored fields, so make sure that text is stored. In the Solr

Facing problem to integrate UIMA in SOLR

2012-05-21 Thread dsy99
Hello all, I am facing problem to integrate the UIMA in SOLR. I followed the following steps, provided in README file shipped along with Uima to integrate it in Solr Step1. I set lib/ tags in solrconfig.xml appropriately to point the jar files. lib dir=../../contrib/uima/lib / lib

Re: Indexing Searching MySQL table with Hindi and English data

2012-05-21 Thread KP Sanjailal
Hi, Thank you so much for replying. The MySQL database server is running on a Fedora Core 12 Machine with Hindi Language Support enabled. Details of the database are - ENGINE=MyISAM and DEFAULT CHARSET=utf8 Data is imported using the Solr DataImportHandler (mysql jdbc driver). In the

Re: org.apache.solr.common.SolrException: ERROR: [doc=null] missing required field: id

2012-05-21 Thread Jack Krupansky
Solr appears to force your UniqueKey field to be required even though you don't have an explicit required=true attribute. As a debugging aid, try adding default=missing to your id field definition and then you can query on id:missing and see what data is being indexed without an id. But, it

Re: Indexing Searching MySQL table with Hindi and English data

2012-05-21 Thread Jack Krupansky
Is it possible that your text editor/display does not support UTF-8 encoding? Assuming the data is properly encoded, do you have the encoding=UTF-8 attribute in your DIH dataSource tag? -- Jack Krupansky -Original Message- From: KP Sanjailal Sent: Monday, May 21, 2012 7:37 AM To:

Re: problem in replication

2012-05-21 Thread shinkanze
hi Tomas , My queries are complex ,i am faceting on many fields ,and using highlighting and using boosts etc in the same query . auto warming takes hell lot of time hence i have removed it . -- View this message in context:

no css on browse UI when multicore

2012-05-21 Thread Aleksander Akerø
Hi The css files from the browse GUI in solr 3.6 does not seem to work properly when solr is deployed with multiple cores and I can’t figure out how to solve this. I know this have been an issue in solr but I thought it was fixed in the newer versions. Any answers or pointers on how to get

boost function parameter (bf) ignores character escaping

2012-05-21 Thread mail
Hey, I'm running solr (3.5.0.2011.11.30.16.37.06) and have encountered what I think is a bug with the boost function (bf) parameter. I've used sunspot (for use of solr with rails) which allows managing dynamic fields, which by default creates fields like

Re: no css on browse UI when multicore

2012-05-21 Thread Erik Hatcher
On May 21, 2012, at 08:11 , Aleksander Akerø wrote: The css files from the browse GUI in solr 3.6 does not seem to work properly when solr is deployed with multiple cores and I can’t figure out how to solve this. I know this have been an issue in solr but I thought it was fixed in the newer

UI

2012-05-21 Thread Tolga
Hi, Can you recommend a good PHP UI to search? Is SolrPHPClient good?

Re: boost function parameter (bf) ignores character escaping

2012-05-21 Thread Erik Hatcher
Yeah, a bug report would be good. But really this is a Sunspot bug report. Field names should NOT have :'s in them. Field names should stick to standard Java identifier rules, otherwise it's escaping madness. You could try something like this as a workaround:

Re: boost function parameter (bf) ignores character escaping

2012-05-21 Thread Jack Krupansky
Quoting from the new trunk example schema: field names should consist of alphanumeric or underscore characters only and not start with a digit. This is not currently strictly enforced, but other field names will not have first class support from all components and back

RE: Solr Single Core vs Multiple Cores installation for localization

2012-05-21 Thread Ivan Hrytsyuk
We intend to have separate, language specific search UI. At the moment we like solution with separate cores more because it is more flexible. But as a rule flexibility costs in terms of performance and we would like to know that price. Jack, what did you mean by 'Managing a bunch of small and

RE: no css on browse UI when multicore

2012-05-21 Thread Aleksander Akerø
Ok, thanks a bunch! I think the url's are set up properly but we have sort of made our own solrconfig files so it's probably the file handler then. I will look into that, but I'm 99.999% sure that this was my problem. Again, thank you for the quick reply! -Original Message- From: Erik

Re: Fault tolerant Solr replication architecture

2012-05-21 Thread Jeremy Taylor
Have you looked at DataStax Enterprise? On May 21, 2012 12:25 AM, Parvin Gasimzade parvin.gasimz...@gmail.com wrote: Hi, I am using solr with replication. I have one master that indexes data and two slaves which pulls index from master and responds to the queries. My question is, how can i

Re: using Carrot2 custom ITokenizerFactory

2012-05-21 Thread Koji Sekiguchi
My problem was gone. Thanks Staszek and Dawid! koji -- Query Log Visualizer for Apache Solr http://soleami.com/ (12/05/21 18:11), Stanislaw Osinski wrote: Hi Koji, Dawid came up with a simple fix for this, it's committed to trunk and 3.6 branch. Staszek

RE: SolrCloud deduplication

2012-05-21 Thread Markus Jelsma
Hi, SOLR-2822 seems to work just fine as long as the SignatureProcessor precedes the DistributedProcessor in the update chain. Thanks, Markus -Original message- From:Mark Miller markrmil...@gmail.com Sent: Fri 18-May-2012 16:05 To: solr-user@lucene.apache.org; Markus Jelsma

Re: Duplicate documents being added even with unique key

2012-05-21 Thread Parmeley, Michael
Changing my field type to string for my uniquekey field solved the problem. Thanks to Jack and Erik for the fix! On May 18, 2012, at 5:33 PM, Jack Krupansky wrote: Typically the uniqueKey field is a string field type (your schema uses text_general), although I don't think it is supposed to

RE: SolrCloud deduplication

2012-05-21 Thread Markus Jelsma
Hi again, It seemed to work fine but in the end duplicates are not overwritten. We first run the SignatureProcessor and then the DistributedProcessor. If we do it the other way around the digest field receives multiple values and throws errors. Is there anything else we can do or another patch

Re: Question about wildcards

2012-05-21 Thread Anderson vasconcelos
Hi. In debug mode, the generated query was: str name=rawquerystringfield:*2231-7/str str name=querystringfield:*2231-7/str str name=parsedqueryfield:*2231-7/str str name=parsedquery_toStringfield:*2231-7/str The analisys of indexing the text .2231-7 produces this result:

Re: Question about wildcards

2012-05-21 Thread Jack Krupansky
Before Solr 3.6, which added MultiTermAwareComponent for analyzers, the presence of a wildcard completely short-circuited (prevented) the query-time analysis, so you have to manually emulate all steps of the query analyzer yourself if you want to do a wildcard. Even with 3.6, not all filters

Re: Question about wildcards

2012-05-21 Thread Anderson vasconcelos
I change the fieldtype of field to the follow: fieldType name=text_ws class=solr.TextField positionIncrementGap=100 analyzertokenizer class=solr.WhitespaceTokenizerFactory//analyzer /fieldType As you see, i just keep the WhitespaceTokenizerFactory. That's works. Now i could find using

Re: Question about wildcards

2012-05-21 Thread Jack Krupansky
And, generally when I see a field that has values like .2231-7, it should be a string field rather than tokenized text. As a string, you can then do straight wildcards without surprises. -- Jack Krupansky -Original Message- From: Jack Krupansky Sent: Monday, May 21, 2012 11:23

Re: Question about wildcards

2012-05-21 Thread Anderson vasconcelos
Thanks all for the explanations. Anderson 2012/5/21 Jack Krupansky j...@basetechnology.com And, generally when I see a field that has values like .2231-7, it should be a string field rather than tokenized text. As a string, you can then do straight wildcards without surprises. -- Jack

RE: SolrCloud deduplication

2012-05-21 Thread Markus Jelsma
https://issues.apache.org/jira/browse/SOLR-3473 -Original message- From:Mark Miller markrmil...@gmail.com Sent: Mon 21-May-2012 18:11 To: solr-user@lucene.apache.org Subject: Re: SolrCloud deduplication Looking again at the SignatureUpdateProcessor code, I think that indeed this

Re: boost function parameter (bf) ignores character escaping

2012-05-21 Thread Jack Krupansky
I think there is a way in sunspot to give an explicit name to a field so that sunspot doesn't generate class-namecolonfield-name for field names. I think it is the :as function, such as: string :name, :as = :name_s So, you can then refer to name in your ruby code and name_s will be the field

RE: trunk cloud ui not working

2012-05-21 Thread Phil Hoy
After further investigation I have found that it is not a problem on firefox, only chrome and IE. Phil -Original Message- Sent: 21 May 2012 18:05 To: solr-user@lucene.apache.org Subject: trunk cloud ui not working Hi, I am running from the trunk and the localhost:8983/solr/#/~cloud

Re: trunk cloud ui not working

2012-05-21 Thread Mark Miller
What OS? I was just trying trunk and looking at that view on Chrome on OSX and Linux and did not see an issue. On May 21, 2012, at 1:15 PM, Phil Hoy wrote: After further investigation I have found that it is not a problem on firefox, only chrome and IE. Phil -Original Message-

Re: Not able to use the highlighting feature! Want to return snippets of text

2012-05-21 Thread 12rad
The field I am trying to highlight is stored. field name=text type=text_en required=false compressed=false omitNorms=false indexed=true stored=true multiValued=true termVectors=true termPositions=true termOffsets=true/ In the searchHandler i've set the parameters as follows:

Re: Fault tolerant Solr replication architecture

2012-05-21 Thread Jan Høydahl
Parvin, What you are looking for is already available in the bleeding edge, unreleased version of Solr, which will become version 4.0 sometime later this year. You can download it at [1] and test it out. The feature is called SolrCloud [2] and it replaces the old replication mechanism in 1.x

Re: CloudSolrServer not working with standalone Zookeeper

2012-05-21 Thread Daniel Brügge
Ok, it seems that a maven dependency to zookeeper version 3.3 broke this. Now it connects to the zk instance. Thanks. On Mon, May 21, 2012 at 5:31 PM, Daniel Brügge daniel.brue...@googlemail.com wrote: Thanks for your feedback. I don't know. I've tried just now with the newest trunk version

Re: Lucene FieldCache - Out of memory exception

2012-05-21 Thread Chris Hostetter
: I am using solr 1.3 with jdk 1.5.0_14 and weblogic 10MP1 application server : on Solaris. I use embedded solr server. More details : FWIW: Solr 1.3 is *REALLY* old ... do not be suprised if much of the info you are given (or read) doesn't apply. : - some mail threads on this forum seem to

Re: Not able to use the highlighting feature! Want to return snippets of text

2012-05-21 Thread Rahul Warawdekar
Hi, Can you please provide the definitions of the following 3 objects from your solrconfig.xml ? str name =hl.fragListBuildersimple/str str name =hl.fragmentsBuildercolored/str str name=hl.fragmenterregex/str For eg, the simple hl.fragListBuilder should be defined as mentioned below in your

SolrJ: clusters, labels, docs - search results

2012-05-21 Thread okayndc
Hello, Was wondering how to access the cluster labels, and docs(ids) via SolrJ? I have added the following: query.seParam(q, userQuery); query.setParam(clustering, true); query.setParam(qt, /core2/clustering); query.setParam(carrot.title, title); But how to access the

Re: Not able to use the highlighting feature! Want to return snippets of text

2012-05-21 Thread 12rad
For the fragListBuilder it's fragListBuilder name=simple default=true class=solr.highlight.SimpleFragListBuilder/ fragment builder is fragmentsBuilder name=colored class=solr.highlight.ScoreOrderFragmentsBuilder

Re: Solr Facets and doc count for a term

2012-05-21 Thread Chris Hostetter
: Is there a way to not only get the number of times a term appears for : a particular field (faceting) as well as the number of documents that : were associated with a particular term? So for instance if I had the : following docs Nope... faceting is associated with _sets_ of documents, so

Re: Not able to use the highlighting feature! Want to return snippets of text

2012-05-21 Thread Rahul Warawdekar
Hi, I believe, in your colored fragmentsBuilder definition, you have not mentioned anything in your pre and post tags and that may be the reason that you are getting snippets of text, without highlighting. Please refer http://wiki.apache.org/solr/HighlightingParameters and check the

how to join 3 tables to pull required data

2012-05-21 Thread srini
I am having a situation where I need to join 3 tables to pull the required information. Can anyone throw me some ideas!!! select A.sid, B.cid, C.NAME from table1 A, table2 B, table3 C where A.sid= C.sid and A.oid = B.oid and C.typeid = 5 and C.flag = 0 and B.cid= 1000; Can you please provide

Remote streaming - posting a URL which is password protected

2012-05-21 Thread 12rad
I want to post index a http document that is password protected. It has a username name login. I tried doing this curl -u username:password http://localhost:8983/solr/update/extract?literal.id=doc900commit=true; -F stream.url=http://somewebsite.com/docs/DOC2609 but it just indexes the login

Solr mail dataimporter cannot be found

2012-05-21 Thread Emma Bo Liu
Hi, I want to index emails using solr. I put the user name, password, hostname in data-config.xml under mail folder. This is a valid email but when I run in url http://localhost:8983/solr/mail/dataimport?command=full-import It said cannot access mail/dataimporter reason: no found. But when i

Re: Remote streaming - posting a URL which is password protected

2012-05-21 Thread Jan Høydahl
Hi, Using curl -u will only attempt to log in to Jetty/Solr, which is not password protected I assume. What you really would like is for the HTTP call which Solr does based on stream.url to attempt a login. Such functionality is not implemented as far as I know. You may try the syntax

Re: UI

2012-05-21 Thread Johannes Goll
yes, I am using this library and it works perfectly so far. If something does not work you can just modify it http://code.google.com/p/solr-php-client/ Johannes 2012/5/21 Tolga to...@ozses.net: Hi, Can you recommend a good PHP UI to search? Is SolrPHPClient good?

Re: Newbie with Carrot2?

2012-05-21 Thread Chris Hostetter
: Subject: Newbie with Carrot2? : References: 35E48F3294A0416A8F476E9C173321F3@msrvcn04 : In-Reply-To: 35E48F3294A0416A8F476E9C173321F3@msrvcn04 https://people.apache.org/~hossman/#threadhijack Thread Hijacking on Mailing Lists When starting a new discussion on a mailing list, please do not

Re: Date format in the schema.xml

2012-05-21 Thread Chris Hostetter
: Subject: Date format in the schema.xml : References: 1336981696.60953.yahoomailclas...@web121705.mail.ne1.yahoo.com : In-Reply-To: 1336981696.60953.yahoomailclas...@web121705.mail.ne1.yahoo.com https://people.apache.org/~hossman/#threadhijack Thread Hijacking on Mailing Lists When starting a

Re: UI

2012-05-21 Thread Damien Camilleri
My favourite php library is solarium. Everything OOP. I've tried a few. http://www.solarium-project.org/ Sent from my iPhone On 21/05/2012, at 6:44 PM, Johannes Goll johannes.g...@gmail.com wrote: yes, I am using this library and it works perfectly so far. If something does not work you can

Re: Solr 3.6.0 problem with multi-core and json

2012-05-21 Thread Chris Hostetter
: I should clarify the error a bit. When I make a select request on my first : core (called core0) using the wt=json parameter I get a 400 response with : the explanation undefined field: gid. The field gid is not defined in the : schema.xml file of my first core. But, it is defined in the

SolrCloud: how to index documents into a specific core and how to search against that core?

2012-05-21 Thread Yandong Yao
Hi Guys, I use following command to start solr cloud according to solr cloud wiki. yydzero:example bjcoe$ java -Dbootstrap_confdir=./solr/conf -Dcollection.configName=myconf -DzkRun -DnumShards=2 -jar start.jar yydzero:example2 bjcoe$ java -Djetty.port=7574 -DzkHost=localhost:9983 -jar start.jar

Date boosting mlt results - possible?

2012-05-21 Thread John Pettitt
Specifically if I'm doing a query using the solr mlt handler (http://wiki.apache.org/solr/MoreLikeThisHandler) and stream.body to supply the source doc is there any way to boost result documents based on document age? I already know how to do that for a regular query using dismax

Re: And results before Or results

2012-05-21 Thread Chris Hostetter
: I want to have a strick enforcement that In case of a 3 word search, those : results that match all 3 term should be presented ahead of those that match : 2 terms when I set mm=2. : : I have seen quite some cases where, those results that match 2 out of 3 : words appear ahead of those matching

Re: UI

2012-05-21 Thread Bill Bell
The php.net plugin is the best. SolrPHPClient is missing several features. Sent from my Mobile device 720-256-8076 On May 21, 2012, at 6:35 AM, Tolga to...@ozses.net wrote: Hi, Can you recommend a good PHP UI to search? Is SolrPHPClient good?

Re: SolrCloud: how to index documents into a specific core and how to search against that core?

2012-05-21 Thread Darren Govoni
Why do you want to control what gets indexed into a core and then knowing what core to search? That's the kind of knowing that SolrCloud solves. In SolrCloud, it handles the distribution of documents across shards and retrieves them regardless of which node is searched from. That is the point of

Re: adding an OR to a fq makes some doc that matched not match anymore

2012-05-21 Thread Chris Hostetter
: - /suggest?q=suggest_terms:lap*fq=type:Pfq=(-type:B) : numFound=1 : doc, so adding a doc will also fulfill right? : /suggest?q=suggest_terms:lap*fq=type:Pfq=(-type:B OR name:aa) : numFound=0 : : is there a logical explanation??

Re: And results before Or results

2012-05-21 Thread Karthick Duraisamy Soundararaj
Interesting, omitTf=true eventhough it would give strict enforcement, wouldnt it affect the relevancy? Like, I am wondering if the ordering amongst the three word matches would be not as good as it would be when we have omitNorms=trueomitTf=true. Do you have an idea? On Mon, May 21, 2012 at 8:51

RE: Advanced search with results matrix

2012-05-21 Thread Chris Hostetter
: No, it's not just one single query, rather, as I've mentioned before, it's : combination of searches with result count for each combination. Explained : in detail below: : 1) (SQL Server OR SQL) : 2) (Visual Basic OR VB.NET) : 3) (Java AND JavaScript) : 4) (SQL Server OR SQL) AND (Visual Basic

Re: And results before Or results

2012-05-21 Thread Chris Hostetter
: Interesting, omitTf=true eventhough it would give strict enforcement, : wouldnt it affect the relevancy? Like, I am wondering if the ordering : amongst the three word matches would be not as good as it would be when we : have omitNorms=trueomitTf=true. Do you have an idea? It will *absolutely*

Re: Indexing files using multi-cores - could not fix after many retries

2012-05-21 Thread Gora Mohanty
On 22 May 2012 05:12, sudarshan chakravarthy.sudars...@gmail.com wrote: [...] requestHandler name=/update/csv                  class=solr.CSVRequestHandler                  startup=lazy / [...] Response: html head meta http-equiv=Content-Type content=text/html; charset=ISO-8859-1/