Solr mlt, ContentStream and mlt.fl

2011-07-09 Thread kare...@gmail.com
Hi,

I have recently upgraded to latest Solr4, rewrote the config files and
recreated the index.  However now I am having trouble with the MLT
handler and I am not sure whether this is by design or a mistake I am
doing.

If I am using MLT on indexed documents, everything works as expected,
I can set  mlt.fl to multiple fields etc.  But when I use stream.body
instead of an indexed document, I can only use exactly one field for
mlt.fl, otherwise an error is returned.

To sum up:
WORKS: 
.../solr/core1/select?stream.body=consumer%20electronicsqt=mltmlt.fl=title
WORKS: 
.../solr/core1/select?stream.body=consumer%20electronicsqt=mltmlt.fl=body
WORKS: .../solr/core1/select?q=id:12345qt=mltmlt.fl=body
DOES NOT WORK: 
.../solr/core1/select?stream.body=consumer%20electronicsqt=mltmlt.fl=title,body

Searching on the error I found some references to CachingTokenFilter
on lucene mailing lists, but I do not really understand whether this
problem is caused by my misconfiguration or a bug in MLT.


Error output from Solr example with MLT enabled:
java.io.IOException: Stream closed
at java.io.StringReader.ensureOpen(StringReader.java:56)
at java.io.StringReader.read(StringReader.java:90)
at 
org.apache.solr.schema.FieldType$DefaultAnalyzer$1.incrementToken(FieldType.java:417)
at 
org.apache.lucene.search.similar.MoreLikeThis.addTermFrequencies(MoreLikeThis.java:891)
at 
org.apache.lucene.search.similar.MoreLikeThis.retrieveTerms(MoreLikeThis.java:963)
at 
org.apache.lucene.search.similar.MoreLikeThis.like(MoreLikeThis.java:634)
at 
org.apache.solr.handler.MoreLikeThisHandler$MoreLikeThisHelper.getMoreLikeThis(MoreLikeThisHandler.java:370)
at 
org.apache.solr.handler.MoreLikeThisHandler.handleRequestBody(MoreLikeThisHandler.java:149)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:353)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at 
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at 
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at 
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at 
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at 
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at 
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at 
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at 
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at 
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at 
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at 
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

My actual error output:
Stream closed java.io.IOException: Stream closed at
java.io.StringReader.ensureOpen(StringReader.java:56) at
java.io.StringReader.read(StringReader.java:90) at
org.apache.lucene.analysis.CharReader.read(CharReader.java:54) at
org.apache.lucene.analysis.charfilter.MappingCharFilter.read(MappingCharFilter.java:126)
at 
org.apache.lucene.analysis.pattern.PatternReplaceCharFilter.read(PatternReplaceCharFilter.java:127)
at 
org.apache.lucene.analysis.standard.StandardTokenizerImpl.zzRefill(StandardTokenizerImpl.java:803)
at 
org.apache.lucene.analysis.standard.StandardTokenizerImpl.getNextToken(StandardTokenizerImpl.java:1010)
at 
org.apache.lucene.analysis.standard.StandardTokenizer.incrementToken(StandardTokenizer.java:168)
at 
org.apache.lucene.analysis.util.FilteringTokenFilter.incrementToken(FilteringTokenFilter.java:50)
at 
org.apache.lucene.analysis.miscellaneous.WordDelimiterFilter.incrementToken(WordDelimiterFilter.java:224)
at 
org.apache.lucene.analysis.core.LowerCaseFilter.incrementToken(LowerCaseFilter.java:54)
at 
org.apache.lucene.analysis.compound.CompoundWordTokenFilterBase.incrementToken(CompoundWordTokenFilterBase.java:157)
at 
org.apache.lucene.analysis.snowball.SnowballFilter.incrementToken(SnowballFilter.java:77)
at 

Result Grouping always returns grouped output

2011-05-27 Thread kare...@gmail.com
Hello,

I am using the latest nightly build of Solr 4.0 and I would like to
use grouping/field collapsing while maintaining compatibility with my
current parser.  I am using the regular webinterface to test it, the
same commands like in the wiki, just with the field names matching my
dataset.

Grouping itself works, group=true and group.field return the expected
results, but neither group.main=true or group.format=simple seem to
change anything.

Do I have to include something special in solrconconfig.xml or
scheme.xml to make the simple output work?

Thanks for any hints,
K