Query regarding Solr ClassCastException on client side for Java
Hi
I am using apache solr 3.3 on windows 7 with JDK 1.6.
I am facing very strange problem. My code makes a query to the solr server in
Java, the code is pasted below:
public QueryResponse getSearchResponse(String query, String url,
String...facets){
try{
SolrQuery query1 = new SolrQuery();
query1.setQuery(query);
if(facets != null){
if(facets.length 0){
query1.setFacet(true);
query1.setRows(0);
for(int i=0; i facets.length; i++){
query1.addFacetField(facets[i]);
}
}
}
log.info(The URL is + url);
query1.setFacetMinCount(1);
query1.setRows(45);
query1.setIncludeScore(false);
log.info(The query is + query1.getQuery());
SolrServer solrServer = new CommonsHttpSolrServer(url);
QueryResponse queryResponse = solrServer.query(query1);
if(queryResponse == null){
queryResponse = new QueryResponse();
log.info(The query response is null);
}
solrServer.commit();
return queryResponse;
}catch(Exception ex){
log.error(Error fetching data from Solr -!! + ex.toString());
ex.printStackTrace();
return new QueryResponse();
}
}
At the line, QueryResponse queryResponse = solrServer.query(query1); Class Cast
Exception is thrown and no data is fetched. I suspect this has something to do
with Solrj client distribution.
I have pasted the exception as well, where Solr war is reporting that there
were 25 records hit, but the client log from my code reports class cast
exception. The exception is pasted below:
INFO: [] webapp=/solr path=/select params={facet=truefacet.mincount=1q=*:*fac
et.field=grp_idwt=javabinrows=45version=2.2} hits=25 status=0 QTime=2
1822816 [http-8085-5] ERROR com.nb.philips.util.SolrjUtil - Error fetching data
from Solr -!! org.apache.solr.client.solrj.SolrServerException: Error executing
query
org.apache.solr.client.solrj.SolrServerException: Error executing query
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryReques
t.java:96)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:109)
at com.nb.philips.util.SolrjUtil.getSearchResponse(SolrjUtil.java:70)
at com.nb.philips.HomePageBO.getHomePageVO(HomePageBO.java:57)
at com.nb.philips.HomePageBO.getGroupsSortedByRank(HomePageBO.java:41)
at org.apache.jsp.includes.dashboard_jsp._jspService(dashboard_jsp.java:
98)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper
.java:386)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:3
13)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appl
icationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationF
ilterChain.java:206)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDisp
atcher.java:646)
at org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationD
ispatcher.java:551)
at org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDis
patcher.java:488)
at org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary
.java:968)
at org.apache.jsp.pages.productDetails_jsp._jspService(productDetails_js
p.java:1917)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper
.java:386)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:3
13)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appl
icationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationF
ilterChain.java:206)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDisp
atcher.java:646)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(Applica
tionDispatcher.java:436)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationD
ispatcher.java:374)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDis
patcher.java:302)
at
Re: Solr request filter and indexing process
Might wanna check your resources, like memory and diskspace. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-request-filter-and-indexing-process-tp3212907p3213512.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr request filter and indexing process
thank you for reply my message. But the developer machine is a new Dell PC with windows7 and 4G memory. If I add 10 document through SolrPHPClient's addDocuments function with XML format, It will takens about more than five seconds. This speed must be unacceptable to our user. I have already disabled the optimize operation after update, Have any other factors is related ? 2011/7/31 O. Klein kl...@octoweb.nl Might wanna check your resources, like memory and diskspace. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-request-filter-and-indexing-process-tp3212907p3213512.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr request filter and indexing process
I don't have much experience with Solr on windows, but have you given Solr enough java heap size etc? Im assuming you are running Solr stand alone. Maybe try and run it in Tomcat? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-request-filter-and-indexing-process-tp3212907p3213650.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr request filter and indexing process
Thanks for reply ,the Solr server uses dataimporthandler, it can import 60 thousands records(about 4G index files) properly in about 30 minutes. So I think the memory is enough. Do you know some performance factors about SolrPhpClient ? 2011/7/31 O. Klein kl...@octoweb.nl I don't have much experience with Solr on windows, but have you given Solr enough java heap size etc? Im assuming you are running Solr stand alone. Maybe try and run it in Tomcat? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-request-filter-and-indexing-process-tp3212907p3213650.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr request filter and indexing process
The first thing that comes to mind is to check whether you are committing after every insert. A number of things may happen when you commit, including merges, rebuilding the spelling dictionary (is this still true in 3.3? maybe not). It's better to commit after a batch of inserts. -Mike On 7/31/2011 11:49 AM, 于浩 wrote: Thanks for reply ,the Solr server uses dataimporthandler, it can import 60 thousands records(about 4G index files) properly in about 30 minutes. So I think the memory is enough. Do you know some performance factors about SolrPhpClient ? 2011/7/31 O. Kleinkl...@octoweb.nl I don't have much experience with Solr on windows, but have you given Solr enough java heap size etc? Im assuming you are running Solr stand alone. Maybe try and run it in Tomcat? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-request-filter-and-indexing-process-tp3212907p3213650.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr can not index F**K!
One of our clients (a hot girl!) brought this to our attention: In this document there are many f* words: http://sec.gov/Archives/edgar/data/1474227/00014742271032/d424b3.htm and we have indexed it with latest version of Solr (ver 3.3). But, we if we search F**K, it does not return the document back! We have tried to index it with different text types, but still not working. Any idea why F* can not be indexed - being censored by the government? :D -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-can-not-index-F-K-tp3214246p3214246.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr request filter and indexing process
: page is loaded. Then I found a problem: some request will return : statusCode:0 and QTime:0, The solr has accepted the request, but It does not : return a result document. If I send each request one by one manually ,It : will return the result. But If I send the request frequently in a very : short times, It will return nothing only statusCode:0 and QTime:0. : I think this may be a stratege for solr. but i can't find any documents or : discussions on the internet. there is no stratgy like whta you describe in solr -- requests should not behave differnetly depending on wether you send them one at a time, or if you send them all concurrently -- many parallel requests may bog down the server, causing the responses to take longer, but you should not see different results from solr. you have not relaly given us much information at all to go on to make suggestsion about the problem you are seeing -- at a minimum some server log showing the types of requests you are making and whta *solr* says about those requests in it's logs is neccesary, but it may also be helpful if you show us the client code you are using to make the requests. My suspicioun is that Solr isn't recieving the requests you think you are sending. Please review this wiki page carefully before sending any replies... http://wiki.apache.org/solr/UsingMailingLists -Hoss
Re: Query regarding Solr ClassCastException on client side for Java
: I am using apache solr 3.3 on windows 7 with JDK 1.6.
First question: what version of SolrJ are you using? is it also 3.3?
: At the line, QueryResponse queryResponse = solrServer.query(query1);
: Class Cast Exception is thrown and no data is fetched. I suspect this
: has something to do with Solrj client distribution.
...
: INFO: [] webapp=/solr path=/select
params={facet=truefacet.mincount=1q=*:*fac
: et.field=grp_idwt=javabinrows=45version=2.2} hits=25 status=0 QTime=2
Assuming that URL still causes the same problem, can you try loading it in
a browser (using wt=xml instead of wt=javabin) and posting the results
to the list? It would help if you could also add echoParams=all to the
URL and/or show us your requestHandler configuration.
I haven't dug into the code, but based on your stack trace i suspect maybe
there is a bug somewhere in the NamedListCodec that assumes Map keys will
always be Strings -- would like to see the data being returned to try and
unerstand why tests are failing or no one else has reported seeing the
same problem you are since that request seems fairly simple and straight
forward (unless ou have some interesting defaults in your config)
: Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
jav
: a.lang.String
: at
org.apache.solr.common.util.NamedListCodec.readOrderedMap(NamedListCo
: dec.java:97)
: at
org.apache.solr.common.util.NamedListCodec.readVal(NamedListCodec.jav
: a:154)
: at
org.apache.solr.common.util.NamedListCodec.readSolrDocument(NamedList
: Codec.java:284)
: at
org.apache.solr.common.util.NamedListCodec.readVal(NamedListCodec.jav
: a:171)
: at
org.apache.solr.common.util.NamedListCodec.readArray(NamedListCodec.j
: ava:367)
: at
org.apache.solr.common.util.NamedListCodec.readVal(NamedListCodec.jav
: a:153)
: at
org.apache.solr.common.util.NamedListCodec.readSolrDocumentList(Named
: ListCodec.java:301)
: at
org.apache.solr.common.util.NamedListCodec.readVal(NamedListCodec.jav
: a:172)
: at
org.apache.solr.common.util.NamedListCodec.readOrderedMap(NamedListCo
: dec.java:98)
: at
org.apache.solr.common.util.NamedListCodec.readVal(NamedListCodec.jav
: a:154)
: at
org.apache.solr.common.util.NamedListCodec.unmarshal(NamedListCodec.j
: ava:89)
: at
org.apache.solr.client.solrj.impl.BinaryResponseParser.processRespons
: e(BinaryResponseParser.java:39)
-Hoss
Re: Solr can not index F**K!
That seems a little far fetched, have you checked your analysis? François On Jul 31, 2011, at 4:58 PM, randohi wrote: One of our clients (a hot girl!) brought this to our attention: In this document there are many f* words: http://sec.gov/Archives/edgar/data/1474227/00014742271032/d424b3.htm and we have indexed it with latest version of Solr (ver 3.3). But, we if we search F**K, it does not return the document back! We have tried to index it with different text types, but still not working. Any idea why F* can not be indexed - being censored by the government? :D -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-can-not-index-F-K-tp3214246p3214246.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr can not index F**K!
Check your Stop words list On Jul 31, 2011 6:25 PM, François Schiettecatte fschietteca...@gmail.com wrote: That seems a little far fetched, have you checked your analysis? François On Jul 31, 2011, at 4:58 PM, randohi wrote: One of our clients (a hot girl!) brought this to our attention: In this document there are many f* words: http://sec.gov/Archives/edgar/data/1474227/00014742271032/d424b3.htm and we have indexed it with latest version of Solr (ver 3.3). But, we if we search F**K, it does not return the document back! We have tried to index it with different text types, but still not working. Any idea why F* can not be indexed - being censored by the government? :D -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-can-not-index-F-K-tp3214246p3214246.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr can not index F**K!
Indeed, the analysis will show if the term is a stop word, the term gets removed by the stop filter, turning on verbose output shows that. François On Jul 31, 2011, at 6:27 PM, Shashi Kant wrote: Check your Stop words list On Jul 31, 2011 6:25 PM, François Schiettecatte fschietteca...@gmail.com wrote: That seems a little far fetched, have you checked your analysis? François On Jul 31, 2011, at 4:58 PM, randohi wrote: One of our clients (a hot girl!) brought this to our attention: In this document there are many f* words: http://sec.gov/Archives/edgar/data/1474227/00014742271032/d424b3.htm and we have indexed it with latest version of Solr (ver 3.3). But, we if we search F**K, it does not return the document back! We have tried to index it with different text types, but still not working. Any idea why F* can not be indexed - being censored by the government? :D -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-can-not-index-F-K-tp3214246p3214246.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: positionIncrementGap in schema.xml
It is not in stopwords list. The file was indexed with the following setting:
Field Type: text_en
Properties: Indexed, Tokenized, Stored, Multivalued
Schema: Indexed, Tokenized, Stored, Multivalued
Index: Indexed, Tokenized, Stored
Position Increment Gap: 100
Index Analyzer: org.apache.solr.analysis.TokenizerChain Details
Tokenizer Class: org.apache.solr.analysis.StandardTokenizerFactory
Filters:
1. org.apache.solr.analysis.StopFilterFactory args:{words:
stopwords_en.txt ignoreCase: true enablePositionIncrements: true
luceneMatchVersion: LUCENE_33 }
2. org.apache.solr.analysis.LowerCaseFilterFactory
args:{luceneMatchVersion: LUCENE_33 }
3. org.apache.solr.analysis.EnglishPossessiveFilterFactory
args:{luceneMatchVersion: LUCENE_33 }
4. org.apache.solr.analysis.KeywordMarkerFilterFactory args:{protected:
protwords.txt luceneMatchVersion: LUCENE_33 }
5. org.apache.solr.analysis.PorterStemFilterFactory
args:{luceneMatchVersion: LUCENE_33 }
There are 1376 tokens indexed, and f**k is not one of them.
--
View this message in context:
http://lucene.472066.n3.nabble.com/positionIncrementGap-in-schema-xml-tp488338p3214469.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr can not index F**K!
It is not in stopwords list. The file was indexed with the following setting:
Field Type: text_en
Properties: Indexed, Tokenized, Stored, Multivalued
Schema: Indexed, Tokenized, Stored, Multivalued
Index: Indexed, Tokenized, Stored
Position Increment Gap: 100
Index Analyzer: org.apache.solr.analysis.TokenizerChain Details
Tokenizer Class: org.apache.solr.analysis.StandardTokenizerFactory
Filters:
1. org.apache.solr.analysis.StopFilterFactory args:{words:
stopwords_en.txt ignoreCase: true enablePositionIncrements: true
luceneMatchVersion: LUCENE_33 }
2. org.apache.solr.analysis.LowerCaseFilterFactory
args:{luceneMatchVersion: LUCENE_33 }
3. org.apache.solr.analysis.EnglishPossessiveFilterFactory
args:{luceneMatchVersion: LUCENE_33 }
4. org.apache.solr.analysis.KeywordMarkerFilterFactory args:{protected:
protwords.txt luceneMatchVersion: LUCENE_33 }
5. org.apache.solr.analysis.PorterStemFilterFactory
args:{luceneMatchVersion: LUCENE_33 }
There are 1376 tokens indexed, and f**k is not one of them.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-can-not-index-F-K-tp3214246p3214472.html
Sent from the Solr - User mailing list archive at Nabble.com.
imported to solr
The indexed contents of 100 sites were imported to solr from nutch using: bin/nutch solrindex http://127.0.0.1:8983/solr/ crawl/crawldb crawl/linkdb crawl/segments/* now, a solr admin search for 'photography' includes these results: doc float name=score0.12570743/float float name=boost1.0440307/float str name=digest94d97f2806240d18d67cafe9c34f94e1/str str name=idhttp://www.galleryhopper.org//str str name=segment.../str str name=titleGallery Hopper: Todd Walker's photography ephemera. Read, enjoy, share, discard./str date name=tstamp.../date str name=urlhttp://www.galleryhopper.org//str /doc but highlighting options are on the title field not page text. My question: Where is the stored parsetext content of the pages? What is the solr command to send it from nutch with url/id key? The information is contained in the crawl segments with solr id field matching nutch url. Thanks.
Re: Solr can not index F**K!
On 7/31/2011 7:29 PM, randohi wrote:
org.apache.solr.analysis.KeywordMarkerFilterFactory args:{protected:
protwords.txt luceneMatchVersion: LUCENE_33 }
Could something be going on here? What's in your protwords.txt ?
-Mike
Re: Solr can not index F**K!
The content of protwords.txt: dontstems zwhacky -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-can-not-index-F-K-tp3214246p3214525.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr can not index F**K!
I hate to be the PC guy, but seriously, did this have to be said? On Mon, Aug 1, 2011 at 6:58 AM, randohi rand...@lawyer.com wrote: One of our clients (a hot girl!) -- E: mark.man...@gmail.com T: http://www.twitter.com/neurotic W: www.compoundtheory.com cf.Objective(ANZ) + Flex - Nov 17, 18 - Melbourne Australia http://www.cfobjective.com.au
Re: Solr can not index F**K!
: http://sec.gov/Archives/edgar/data/1474227/00014742271032/d424b3.htm My money is on maxFieldLength in solrconfig.xml... http://www.lucidimagination.com/search/?q=maxFieldLength#/p:solr -Hoss
Re: changing the root directory where solrCloud stores info inside zookeeper File system
Sorry for the delay on this response Yatir - I missed this email. I need to be more diligent ;) You can chroot any path by tacking it on the ZooKeeper address. So if the address is 127.0.0.1:3002, then you can use the /solr sub-tree by using the address 127.0.0.1:3002/nestedisokay/solr - Mark Miller lucidimagination.com On Jul 26, 2011, at 4:52 AM, Yatir Ben Shlomo wrote: Hi! I am using solrCloud with a zookeeper ensamble of 3. I noticed that solcOuld stores information direclt under the root dir in the ZooKeepr file system: \config \live_nodes \ collections In my setup Zookeepr is also used by other modules so I would like solrCLoud to store everything under /solrCLoud/ or something similar Is there a property for that or do I need to custom code it ? Thanks
Re: Solr can not index F**K!
Yes, that is where the problem comes! Many many thanks to everyone! -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-can-not-index-F-K-tp3214246p3214666.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: An idea for an intersection type of filter query
: fq AND fq AND fq AND (fqu OR fqu OR fqu) : : It would be awesome to have a syntax that creates arbitrarily complex and : nested AND/OR combinations, but that would be a MAJOR undertaking. The logic : I've mentioned above seems to be the most useful you could get with just : having the one additional parameter. You can get pure union by just using the syntax isn't really the hard part. where things get tricky is in the internals of th SolrIndexSearcher and SearchHandler so that you cache those fqu params independently and then union the results, particularly when those fq/fqu params need to be part of the cache key for the queryResultCache ... a lot of little changes to the internals. It's been discussed at a high level a sporadically over the years, but no one has had the drive/energy/knowledge to dig into the guts and make it work. Having built several custom faceting components over the years (that apply special biz rules) i can tell you that generating DocSets and then computing unions/intersections is easy and efficient (the SolrIndexSearcher/SolrCache/DocSet APIs are really straight forward), but anytime you want to then use that DocSet to constrain a DocList ... you run into complications. -Hoss
Re: Solr 3.2.0 is not writing log
In etc/jetty.xml, at line 204, you should remove the comment (delete line 204 and line 218) 204 -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-3-2-0-is-not-writing-log-tp3208945p3214774.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Query on multi valued field
have you tried multi:1 and multi:2 and multi:3 ? 2011/7/29 rajini maski rajinima...@gmail.com Hi All, I have a specific requirement in the multi-valued field type.The requirement is as follows There is a multivalued field in each document which can have mutliple elements or single element. For Eg: Consider that following are the documents matched for say q= *:* *DOC1* doc arr name=multi str1/str /arr /doc * * *DOC2* doc arr name=multi str1/str str3/str str4/str /arr /doc *DOC3* doc arr name=multi str1/str str2/str /arr /doc The query is get only those documents which have multiple elements for that multivalued field. I.e, doc 2 and 3 should be returned from the above set.. Is there anyway to achieve this? Awaiting reply, Thanks Regards, Rajani -- *Alexei Martchenko* | *CEO* | Superdownloads ale...@superdownloads.com.br | ale...@martchenko.com.br | (11) 5083.1018/5080.3535/5080.3533
Re: Solr Incremental Indexing
There could be multiple ways of getting this done, and the exact one depends a lot on factors like - what system are you using? How realtime the change has to be reflected back into the system? How is the indexing/replication done? Usually, in cases where the tolerance is about 6hrs (i.e. your DB change wont be reflected in Solr Index for as high as 6hrs), you can set up a cron job to be triggered every 6 hrs. It will see all the changes made between that time, and update Index and commit it. In cases, where a more real time requirement, there could be a trigger in the application (and not at the db level), which would fork a process to update Solr about this change by means of delayed task. If using this approach, it is suggested to use autocommit every N documents, N could be anything depending your app. *Pranav Prakash* temet nosce Twitter http://twitter.com/pranavprakash | Blog http://blog.myblive.com | Google http://www.google.com/profiles/pranny On Sun, Jul 31, 2011 at 02:32, Alexei Martchenko ale...@superdownloads.com.br wrote: I always have a field in my databases called datelastmodified, so whenever I update that record, i set it to getdate() - mssql func - and then get all latest records order by that field. 2011/7/29 Mohammed Lateef Hussain mohammedlateefh...@gmail.com Hi Need some help in Solr incremental indexing approch. I have built my Solr index using SolrJ API and now want to update the index whenever any changes has been made in database. My requirement is not to use DB triggers to call any update events. I want to update my index on the fly whenever my application updates any record in database. Note: My indexing logic to get the required data from DB is some what complex and involves many tables. Please suggest me how can I proceed here. Thanks Lateef -- *Alexei Martchenko* | *CEO* | Superdownloads ale...@superdownloads.com.br | ale...@martchenko.com.br | (11) 5083.1018/5080.3535/5080.3533
