Need help in understanding solr clustering component
I was looking(in process of making my own) into solr's default clustering
component for carrot2. In the clustering component class there are 2 methods
where the clustering algorithms are called:
in the overridden process method
SolrDocumentList solrDocList = SolrPluginUtils.docListToSolrDocumentList(
results.docList, rb.req.getSearcher(),
engine.getFieldsToLoad(rb.req),docIds);
Object clusters = engine.cluster(rb.getQuery(), solrDocList, docIds,
rb.req);
rb.rsp.add("clusters", clusters);
And once again in the finishStage method
Map docIds = null;
Object clusters = engine.cluster(rb.getQuery(), solrDocList, docIds,
rb.req);
rb.rsp.add("clusters", clusters);
Now my question is the process method works not on the complete result query
but on the shards and finish stage once when all the results have been
aggregated, then why does we call the clustering algorithms twice and adding
it to the resulted cluster? Am I missing something?
Wont it create too many labels if in the worst case none of the cluster
labels match?
P.S Please correct me if I am wrong.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Need-help-in-understanding-solr-clustering-component-tp4334400.html
Sent from the Solr - User mailing list archive at Nabble.com.
Solr Clustering component different results than Carrot workbench
Though I am interacting with Dawid (creator of Carrot2) on Carrot2 mailing list however just wanted to post my problem to a wider audience. I am using Solr 4.7 (on both windows and linux) and saved my lingo-attributes.xml file from the workbench which I am using in Solr. Note that for testing I am just having one solr Index and all the queries are getting fired on that. Now the clusters that I am getting are good in the workbench (carrot) but pathetic in Solr. In the logs (jetty) I can see: Loaded Solr resource: clustering/carrot2/lingo-attributes.xml, so that indicates that my attribute file is being loaded. I am really confused what is accounting for the difference in the two outputs (workbench vs Solr). Again to reiterate the data sources are same (just one solr index and same queries with 100 results). This is happening on both Linux and Windows. Given below is my search component and request handler configuration: lingo org.carrot2.clustering.lingo.LingoClusteringAlgorithm 30 clustering/carrot2 true true org.carrot2.clustering.lingo.LingoClusteringAlgorithm clustering/carrot2 film_id description true false 100 clustering
Re: Problem in including both clustering component and spellchecker for solr search results at the same time
Markus, i did like it * default true false 1 default true false 1 * I hope i have done things correctly. But when i run solr server i am getting exception *org.apache.solr.common.SolrException: Unknown Search Component: clusteringComponent* - Thanks & Regards Romi -- View this message in context: http://lucene.472066.n3.nabble.com/Problem-in-including-both-clustering-component-and-spellchecker-for-solr-search-results-at-the-same-e-tp3128864p3136756.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem in including both clustering component and spellchecker for solr search results at the same time
This example loads two fictional components. Use spellcheck and clustering instead. 704 705 708 709 explicit 710 10 711 712 716 725 730 746 754 758 764 > would you please give me an example for custom request handler > > - > Thanks & Regards > Romi > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Problem-in-including-both-clustering-co > mponent-and-spellchecker-for-solr-search-results-at-the-same-e-tp3128864p31 > 28893.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem in including both clustering component and spellchecker for solr search results at the same time
would you please give me an example for custom request handler - Thanks & Regards Romi -- View this message in context: http://lucene.472066.n3.nabble.com/Problem-in-including-both-clustering-component-and-spellchecker-for-solr-search-results-at-the-same-e-tp3128864p3128893.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem in including both clustering component and spellchecker for solr search results at the same time
Use a custom request handler and define both components as in the example for these individual request handlers. > I want to include both clustering and spellchecker in my search results. > but at a time i am able to include only one. Only one, with which > i am setting default=true. than how can i include both > clustering and spellchecker both for my results. > > > - > Thanks & Regards > Romi > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Problem-in-including-both-clustering-co > mponent-and-spellchecker-for-solr-search-results-at-the-same-e-tp3128864p31 > 28864.html Sent from the Solr - User mailing list archive at Nabble.com.
Problem in including both clustering component and spellchecker for solr search results at the same time
I want to include both clustering and spellchecker in my search results. but at a time i am able to include only one. Only one, with which i am setting default=true. than how can i include both clustering and spellchecker both for my results. - Thanks & Regards Romi -- View this message in context: http://lucene.472066.n3.nabble.com/Problem-in-including-both-clustering-component-and-spellchecker-for-solr-search-results-at-the-same-e-tp3128864p3128864.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: what is solr clustering component
> > and my second question is does clustering effect indexes. > No, it doesn't. Clustering is performed only on the search results produced by Solr, it doesn't change anything in the index. Cheers, Staszek
Re: what is solr clustering component
thanks iorixxx, i changed my configuration to include clustering in search results. in my xml format search results i got a tag , to show this clusters in to search results do i need to parse this xml. and my second question is does clustering effect indexes. - Thanks & Regards Romi -- View this message in context: http://lucene.472066.n3.nabble.com/what-is-solr-clustering-component-tp3121484p3124627.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: what is solr clustering component
> I just went through solr wiki page > for clustering. But i am not getting what > is the benefit of using clustering. Can anyone tell me what > is actually > clusering and what its use in indexing and searching. > does it effect search results?? > Please reply It is for search result clustering. Try the demo with the query word jaguar. http://search.carrot2.org/stable/search It generates clusters and labels. (on the left)
what is solr clustering component
I just went through solr wiki page for clustering. But i am not getting what is the benefit of using clustering. Can anyone tell me what is actually clusering and what its use in indexing and searching. does it effect search results?? Please reply - Thanks & Regards Romi -- View this message in context: http://lucene.472066.n3.nabble.com/what-is-solr-clustering-component-tp3121484p3121484.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: assit with the Clustering component in Solr/Lucene
Thanks much Stan, Ramdev On May 16, 2011, at 11:38 AM, Stanislaw Osinski wrote: Both of the clustering algorithms that ship with Solr (Lingo and STC) are designed to allow one document to appear in more than one cluster, which actually does make sense in many scenarios. There's no easy way to force them to produce hard clusterings because this would require a complete change in the way the algorithms work. If you need each document to belong to exactly one cluster, you'd have to post-process the clusters to remove the redundant document assignments. On the second thought, I have a simple implementation of k-means clustering that could do hard clustering for you. It's not available yet, it will most probably be part of the next major release of Carrot2 (the package that does the clustering). Please watch this issue http://issues.carrot2.org/browse/CARROT-791 to get updates on this. Just to let you know: Carrot2 3.5.0 has landed in Solr trunk and branch_3x, so you can use the bisecting k-means clustering algorithm (org.carrot2.clustering.kmeans.BisectingKMeansClusteringAlgorithm) which will produce non-overlapping clusters for you. The downside of this simple implementation of k-means is that, for the time being, it produces one-word cluster labels rather than phrases as Lingo and STC. Cheers, S.
Re: assit with the Clustering component in Solr/Lucene
> > Both of the clustering algorithms that ship with Solr (Lingo and STC) are >> designed to allow one document to appear in more than one cluster, which >> actually does make sense in many scenarios. There's no easy way to force >> them to produce hard clusterings because this would require a complete >> change in the way the algorithms work. If you need each document to belong >> to exactly one cluster, you'd have to post-process the clusters to remove >> the redundant document assignments. >> > > On the second thought, I have a simple implementation of k-means clustering > that could do hard clustering for you. It's not available yet, it will most > probably be part of the next major release of Carrot2 (the package that does > the clustering). Please watch this issue > http://issues.carrot2.org/browse/CARROT-791 to get updates on this. > Just to let you know: Carrot2 3.5.0 has landed in Solr trunk and branch_3x, so you can use the bisecting k-means clustering algorithm (org.carrot2.clustering.kmeans.BisectingKMeansClusteringAlgorithm) which will produce non-overlapping clusters for you. The downside of this simple implementation of k-means is that, for the time being, it produces one-word cluster labels rather than phrases as Lingo and STC. Cheers, S.
Re: assit with the Clustering component in Solr/Lucene
Thanks for the confirmation, I'll take a look at the issue. S. On Thu, Mar 31, 2011 at 17:24, wrote: > That did make a difference, I now see the exact number of cluster i see > from the workbench. > I am of course interested in why the config changes did not have much > effect. However, I am happy that by adding the threshold to my request URL > produces the desired results > > let me know if I can do any more tests and I will do so. Thanks much > > Ramdev > > > > On Mar 31, 2011, at 10:18 AM, Stanislaw Osinski wrote: > > > I added the parameter as you suggested. >> (LingoClusteringAlgorithm.clusterMergingThreshold) into the searchComponent >> section that describes the Clustering module >> Changing the value of the parameter did not have any effect on my search >> results. >> >> However, when I used the Carrot2 workbench, I could see the effect of >> changing the value. (from 6 clusters it went down to 2 clusters) >> > > Interesting... Can you, for the sake of debugging, append > &LingoClusteringAlgorithm.clusterMergingThreshold=0.0 to your request URL? > > S. > > >
Re: assit with the Clustering component in Solr/Lucene
That did make a difference, I now see the exact number of cluster i see from the workbench. I am of course interested in why the config changes did not have much effect. However, I am happy that by adding the threshold to my request URL produces the desired results let me know if I can do any more tests and I will do so. Thanks much Ramdev On Mar 31, 2011, at 10:18 AM, Stanislaw Osinski wrote: I added the parameter as you suggested. (LingoClusteringAlgorithm.clusterMergingThreshold) into the searchComponent section that describes the Clustering module Changing the value of the parameter did not have any effect on my search results. However, when I used the Carrot2 workbench, I could see the effect of changing the value. (from 6 clusters it went down to 2 clusters) Interesting... Can you, for the sake of debugging, append &LingoClusteringAlgorithm.clusterMergingThreshold=0.0 to your request URL? S.
Re: assit with the Clustering component in Solr/Lucene
> I added the parameter as you suggested. > (LingoClusteringAlgorithm.clusterMergingThreshold) into the searchComponent > section that describes the Clustering module > Changing the value of the parameter did not have any effect on my search > results. > > However, when I used the Carrot2 workbench, I could see the effect of > changing the value. (from 6 clusters it went down to 2 clusters) > Interesting... Can you, for the sake of debugging, append &LingoClusteringAlgorithm.clusterMergingThreshold=0.0 to your request URL? S.
Re: assit with the Clustering component in Solr/Lucene
Hi Staszek:
I added the parameter as you suggested.
(LingoClusteringAlgorithm.clusterMergingThreshold) into the searchComponent
section that describes the Clustering module
Changing the value of the parameter did not have any effect on my search
results.
However, when I used the Carrot2 workbench, I could see the effect of changing
the value. (from 6 clusters it went down to 2 clusters)
here is the XML snippet for the searchComponent:
default
org.carrot2.clustering.lingo.LingoClusteringAlgorithm
20
0.0
I would appreciate any insights into this behavior.
Thanks
Ramdev
On Mar 30, 2011, at 11:51 AM, Stanislaw Osinski wrote:
Hi Ramdev,
Both of the clustering algorithms that ship with Solr (Lingo and STC)
are designed to allow one document to appear in more than one cluster, which
actually does make sense in many scenarios. There's no easy way to force them
to produce hard clusterings because this would require a complete change in the
way the algorithms work. If you need each document to belong to exactly one
cluster, you'd have to post-process the clusters to remove the redundant
document assignments. Alternatively, in case of the Lingo algorithm, you can
try lowering the "LingoClusteringAlgorithm.clusterMergingThreshold" to some
value in the range of 0.2--0.5. If you do that, clusters containing overlapping
documents will get merged. For more information about this attribute, see here:
http://download.carrot2.org/stable/manual/#section.attribute.LingoClusteringAlgorithm.clusterMergingThreshold.
Cheers,
Staszek
On Wed, Mar 30, 2011 at 18:21, Markus Jelsma
wrote:
Yes, you can set engine specific parameters. Check the comments
in your
snippety.
> Hi:
> I recently included the CLustering component into Solr and
updated the
> requestHandler accordingly (in solrconfig.xml). Snippet of
the Config for
> the CLuserting:
>
>name="clusteringComponent"
> enable="${solr.clustering.enabled:false}"
>
class="org.apache.solr.handler.clustering.ClusteringComponent" >
>
>
>
> default
>
>
name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgori
> thm
> 20
>
>
> stc
>
name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm<
> /str>
>
>
> snippet of the Config for requestHandler
>default="true">
>
>explicit
>
>true
>default
>true
>
>headline
>pi
>
>headline
>
>true
>
>
>
>false
>
>
> clusteringComponent
>
>
>
>
> When I perform a search, I see that the Cluster section
within the Solr
> results shows me results that are not quite consistent. There
are two
> documents that are reported in two different documents
>
> Are there parameters that can be set that will prevent this
from happening
> ?
>
>
> Thanks much
>
> Ramdev
Re: assit with the Clustering component in Solr/Lucene
> Both of the clustering algorithms that ship with Solr (Lingo and STC) are > designed to allow one document to appear in more than one cluster, which > actually does make sense in many scenarios. There's no easy way to force > them to produce hard clusterings because this would require a complete > change in the way the algorithms work. If you need each document to belong > to exactly one cluster, you'd have to post-process the clusters to remove > the redundant document assignments. > On the second thought, I have a simple implementation of k-means clustering that could do hard clustering for you. It's not available yet, it will most probably be part of the next major release of Carrot2 (the package that does the clustering). Please watch this issue http://issues.carrot2.org/browse/CARROT-791 to get updates on this. Cheers, S.
Re: assit with the Clustering component in Solr/Lucene
Hi Ramdev,
Both of the clustering algorithms that ship with Solr (Lingo and STC) are
designed to allow one document to appear in more than one cluster, which
actually does make sense in many scenarios. There's no easy way to force
them to produce hard clusterings because this would require a complete
change in the way the algorithms work. If you need each document to belong
to exactly one cluster, you'd have to post-process the clusters to remove
the redundant document assignments. Alternatively, in case of the Lingo
algorithm, you can try lowering the
"LingoClusteringAlgorithm.clusterMergingThreshold" to some value in the
range of 0.2--0.5. If you do that, clusters containing overlapping documents
will get merged. For more information about this attribute, see here:
http://download.carrot2.org/stable/manual/#section.attribute.LingoClusteringAlgorithm.clusterMergingThreshold
.
Cheers,
Staszek
On Wed, Mar 30, 2011 at 18:21, Markus Jelsma wrote:
> Yes, you can set engine specific parameters. Check the comments in your
> snippety.
>
> > Hi:
> > I recently included the CLustering component into Solr and updated the
> > requestHandler accordingly (in solrconfig.xml). Snippet of the Config for
> > the CLuserting:
> >
> >> name="clusteringComponent"
> > enable="${solr.clustering.enabled:false}"
> > class="org.apache.solr.handler.clustering.ClusteringComponent" >
> >
> >
> >
> > default
> >
> >>
> name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgori
> > thm
> >name="LingoClusteringAlgorithm.desiredClusterCountBase">20
> >
> >
> > stc
> >>
> name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm<
> > /str>
> >
> >
> > snippet of the Config for requestHandler
> >> default="true">
> >
> >explicit
> >
> >true
> >default
> >true
> >
> >headline
> >pi
> >
> >headline
> >
> >true
> >
> >
> >
> >false
> >
> >
> > clusteringComponent
> >
> >
> >
> >
> > When I perform a search, I see that the Cluster section within the Solr
> > results shows me results that are not quite consistent. There are two
> > documents that are reported in two different documents
> >
> > Are there parameters that can be set that will prevent this from
> happening
> > ?
> >
> >
> > Thanks much
> >
> > Ramdev
>
Re: assit with the Clustering component in Solr/Lucene
Yes, you can set engine specific parameters. Check the comments in your
snippety.
> Hi:
> I recently included the CLustering component into Solr and updated the
> requestHandler accordingly (in solrconfig.xml). Snippet of the Config for
> the CLuserting:
>
>name="clusteringComponent"
> enable="${solr.clustering.enabled:false}"
> class="org.apache.solr.handler.clustering.ClusteringComponent" >
>
>
>
> default
>
>name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgori
> thm
> 20
>
>
> stc
>name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm<
> /str>
>
>
> snippet of the Config for requestHandler
>default="true">
>
>explicit
>
>true
>default
>true
>
>headline
>pi
>
>headline
>
>true
>
>
>
>false
>
>
> clusteringComponent
>
>
>
>
> When I perform a search, I see that the Cluster section within the Solr
> results shows me results that are not quite consistent. There are two
> documents that are reported in two different documents
>
> Are there parameters that can be set that will prevent this from happening
> ?
>
>
> Thanks much
>
> Ramdev
assit with the Clustering component in Solr/Lucene
Hi: I recently included the CLustering component into Solr and updated the requestHandler accordingly (in solrconfig.xml). Snippet of the Config for the CLuserting: default org.carrot2.clustering.lingo.LingoClusteringAlgorithm 20 stc org.carrot2.clustering.stc.STCClusteringAlgorithm snippet of the Config for requestHandler explicit true default true headline pi headline true false clusteringComponent When I perform a search, I see that the Cluster section within the Solr results shows me results that are not quite consistent. There are two documents that are reported in two different documents Are there parameters that can be set that will prevent this from happening ? Thanks much Ramdev
Re: carrot2 clustering component error
I've seen that before on a 3.1 check out after i compiled the clustering component, copied the jars and started Solr. For some reason , recompiling didn't work and doing an ant clean in front didn't fix it either. Updating to a revision i knew did work also failed. I just removed the entire checkout and took it back in, repeated my steps and it works fine now. > help me out of this error: > > java.lang.NoClassDefFoundError: org/apache/solr/util/plugin/SolrCoreAware
carrot2 clustering component error
help me out of this error: java.lang.NoClassDefFoundError: org/apache/solr/util/plugin/SolrCoreAware
Re: Carrot2 clustering component
Hi, I think the exception is caused by the fact that you're trying to use the latest version of Carrot2 with Solr 1.4.x. There are two alternative solutions here: * as described in http://wiki.apache.org/solr/ClusteringComponent, invoke "ant get-libraries" to get the compatible JAR files. or * use the latest version of Carrot2 with Solr 1.4.x by installing the compatibility package, documentation is here: http://download.carrot2.org/stable/manual/#section.solr Cheers, Staszek On Tue, Jan 18, 2011 at 13:36, Isha Garg wrote: > Hi, >Can anyone help me to solve the error: > Class org.carrot2.util.pool.SoftUnboundedPool does not implement the > requested interface org.carrot2.util.pool.IParameterizedPool >at > org.carrot2.core.PoolingProcessingComponentManager.(PoolingProcessingComponentManager.java:77) >at > org.carrot2.core.PoolingProcessingComponentManager.(PoolingProcessingComponentManager.java:62) >at org.carrot2.core.ControllerFactory.create(ControllerFactory.java:158) >at > org.carrot2.core.ControllerFactory.createPooling(ControllerFactory.java:71) >at > org.apache.solr.handler.clustering.carrot2.CarrotClusteringEngine.(CarrotClusteringEngine.java:61) >at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) >at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) >at java.lang.reflect.Constructor.newInstance(Constructor.java:513) >at java.lang.Class.newInstance0(Class.java:355) >at java.lang.Class.newInstance(Class.java:308) >at > org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:396) >at > org.apache.solr.handler.clustering.ClusteringComponent.inform(ClusteringComponent.java:121) >at > org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:486) >at org.apache.solr.core.SolrCore.(SolrCore.java:588) >at > org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:137) >at > org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:83) >at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) >at > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >at > org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) >at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) >at > org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) >at > org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) >at > org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) >at > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >at > org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) >at > org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) >at > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >at > org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) >at > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >at > org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) >at org.mortbay.jetty.Server.doStart(Server.java:210) >at > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) >at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >at java.lang.reflect.Method.invoke(Method.java:597) >at org.mortbay.start.Main.invokeMain(Main.java:183) >at org.mortbay.start.Main.start(Main.java:497) >at org.mortbay.start.Main.main(Main.java:115) > 18 Jan, 2011 6:03:30 PM org.apache.solr.common.SolrException log > SEVERE: java.lang.IncompatibleClassChangeError: Class > org.carrot2.util.pool.SoftUnboundedPool does not implement the requested > interface org.carrot2.util.pool.IParameterizedPool >at > org.carrot2.core.PoolingProcessingComponentManager.(PoolingProcessingComponentManager.java:77) >at > org.carrot2.core.PoolingProcessingComponentManager.(PoolingProcessingComponentManager.java:62) >at org.carrot2.core.ControllerFactory.create(ControllerFactory.java:158) >at > org.carrot2.core.ControllerFactory.createPooling(ControllerFactory.java:71) >at > org.apache.solr.handler.clustering.carrot2.CarrotClusteringEngine.(CarrotClusteringEngine.java:61) >at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) >at > sun.refl
Carrot2 clustering component
Hi, Can anyone help me to solve the error: Class org.carrot2.util.pool.SoftUnboundedPool does not implement the requested interface org.carrot2.util.pool.IParameterizedPool at org.carrot2.core.PoolingProcessingComponentManager.(PoolingProcessingComponentManager.java:77) at org.carrot2.core.PoolingProcessingComponentManager.(PoolingProcessingComponentManager.java:62) at org.carrot2.core.ControllerFactory.create(ControllerFactory.java:158) at org.carrot2.core.ControllerFactory.createPooling(ControllerFactory.java:71) at org.apache.solr.handler.clustering.carrot2.CarrotClusteringEngine.(CarrotClusteringEngine.java:61) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at java.lang.Class.newInstance0(Class.java:355) at java.lang.Class.newInstance(Class.java:308) at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:396) at org.apache.solr.handler.clustering.ClusteringComponent.inform(ClusteringComponent.java:121) at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:486) at org.apache.solr.core.SolrCore.(SolrCore.java:588) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:137) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:83) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) at org.mortbay.jetty.Server.doStart(Server.java:210) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.mortbay.start.Main.invokeMain(Main.java:183) at org.mortbay.start.Main.start(Main.java:497) at org.mortbay.start.Main.main(Main.java:115) 18 Jan, 2011 6:03:30 PM org.apache.solr.common.SolrException log SEVERE: java.lang.IncompatibleClassChangeError: Class org.carrot2.util.pool.SoftUnboundedPool does not implement the requested interface org.carrot2.util.pool.IParameterizedPool at org.carrot2.core.PoolingProcessingComponentManager.(PoolingProcessingComponentManager.java:77) at org.carrot2.core.PoolingProcessingComponentManager.(PoolingProcessingComponentManager.java:62) at org.carrot2.core.ControllerFactory.create(ControllerFactory.java:158) at org.carrot2.core.ControllerFactory.createPooling(ControllerFactory.java:71) at org.apache.solr.handler.clustering.carrot2.CarrotClusteringEngine.(CarrotClusteringEngine.java:61) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at java.lang.Class.newInstance0(Class.java:355) at java.lang.Class.newInstance(Class.java:308) at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:396) at org.apache.solr.handler.clustering.ClusteringComponent.inform(ClusteringComponent.java:121) at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:486) at org.apache.solr.core.SolrCore.(SolrCore.java:588) at org.apache.so
Re: Carrot2 clustering component
Isha, Next, you need to run the actual search so Carrot2 has some search results to cluster. Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message > From: Isha Garg > To: solr-user@lucene.apache.org > Sent: Tue, January 18, 2011 1:54:39 AM > Subject: Re: Carrot2 clustering component > > On Tuesday 18 January 2011 11:12 AM, Otis Gospodnetic wrote: > > Isha, > > > > You'll get more and better help if you provide more details about what you >have > > done, what you have tried, what isn't working, what errors or behaviour > > you >are > > seeing, etc. > > > > Otis > > > > Sematext ::http://sematext.com/ :: Solr - Lucene - Nutch > > Lucene ecosystem search ::http://search-lucene.com/ > > > > > > > > - Original Message > > > >> From: Isha Garg > >> To:solr-user@lucene.apache.org > >> Sent: Tue, January 18, 2011 12:38:03 AM > >> Subject: Carrot2 clustering component > >> > >> Hi, > >> I am not able to understand the caarot2 clustering component from > >> > >> http://wiki.apache.org/solr/ClusteringComponent > >> > >> please provide me more detailed information if someone had already > >> worked >on > >> this. How to run this and use this during search query. > >> > >> > >> Thanks! > >> Isha > >> > >> > >> > >> > >> > I had downloaded some jar files compatible with solr1.4 including: > carrot2-core-3.4.2.jar > guava-r05.jar > hppc-0.3.1.jar > jackson-core-asl-1.5.2.jar > mahout-collections-0.3.jar > jackson-mapper-asl-1.5.2.jar > log4j-1.2.14.jar > mahout-math-0.3.jar > simple-xml-2.3.5.jar > And placed them at contrib/clustering/lib > > Then changed the solr.config as: > > > > > explicit > > > default > true > > headin > id > > text > > true > > > > false > > > > clustering > > > > > > > > default > > name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm > > 20 > > > stc > name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm > > > > > And then run solr using command: > java -Dsolr.clustering.enabled=true -jar start.jar > > > Now can you tell me where i am wrong ?? what else should i >do? > > >
Re: Carrot2 clustering component
On Tuesday 18 January 2011 11:12 AM, Otis Gospodnetic wrote: Isha, You'll get more and better help if you provide more details about what you have done, what you have tried, what isn't working, what errors or behaviour you are seeing, etc. Otis Sematext ::http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search ::http://search-lucene.com/ - Original Message From: Isha Garg To:solr-user@lucene.apache.org Sent: Tue, January 18, 2011 12:38:03 AM Subject: Carrot2 clustering component Hi, I am not able to understand the caarot2 clustering component from http://wiki.apache.org/solr/ClusteringComponent please provide me more detailed information if someone had already worked on this. How to run this and use this during search query. Thanks! Isha I had downloaded some jar files compatible with solr1.4 including: carrot2-core-3.4.2.jar guava-r05.jar hppc-0.3.1.jar jackson-core-asl-1.5.2.jar mahout-collections-0.3.jar jackson-mapper-asl-1.5.2.jar log4j-1.2.14.jar mahout-math-0.3.jar simple-xml-2.3.5.jar And placed them at contrib/clustering/lib Then changed the solr.config as: explicit default true headin id text true false clustering default org.carrot2.clustering.lingo.LingoClusteringAlgorithm 20 stc org.carrot2.clustering.stc.STCClusteringAlgorithm And then run solr using command: java -Dsolr.clustering.enabled=true -jar start.jar Now can you tell me where i am wrong ?? what else should i do?
Re: Carrot2 clustering component
Isha, You'll get more and better help if you provide more details about what you have done, what you have tried, what isn't working, what errors or behaviour you are seeing, etc. Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message > From: Isha Garg > To: solr-user@lucene.apache.org > Sent: Tue, January 18, 2011 12:38:03 AM > Subject: Carrot2 clustering component > > Hi, >I am not able to understand the caarot2 clustering component from > > http://wiki.apache.org/solr/ClusteringComponent > > please provide me more detailed information if someone had already worked on >this. How to run this and use this during search query. > > > Thanks! > Isha > > > >
Carrot2 clustering component
Hi, I am not able to understand the caarot2 clustering component from http://wiki.apache.org/solr/ClusteringComponent please provide me more detailed information if someone had already worked on this. How to run this and use this during search query. Thanks! Isha
Carrot2 clustering Component
Hi, Please tell me how can I get the libraries and plugins for carrot2 clustering component in solr1.4.Tell me the site from where i can get them. Thanks! Isha
Re: specifying the doc id in clustering component
Yes, that's the approach I'm taking right now. I do a lookup the doc ids in the resultset to find the matching document. I can live with the manual lookup, I wanted to see if it would be possible to pick a custom field to represent the document in the docs array. Thanks for contributing the plugin to solr! @tommychheng Programmer and UC Irvine Graduate Student Find a great grad school based on research interests: http://gradschoolnow.com On 8/19/10 10:51 PM, Stanislaw Osinski wrote: The solr schema has the fields, id, name and desc. I would like to get docs:["name Field here" ] instead of the doc Id field as in "docs":["200066", "195650", The idea behind using the document ids was that based on them you could access the individual documents' content, including the other fields, right from the "response" field. Using ids limits duplication in the response text as a whole. Is it possible to use this approach in your application? Staszek
Re: specifying the doc id in clustering component
> The solr schema has the fields, id, name and desc. > > I would like to get docs:["name Field here" ] instead of the doc Id > field as in > "docs":["200066", "195650", > The idea behind using the document ids was that based on them you could access the individual documents' content, including the other fields, right from the "response" field. Using ids limits duplication in the response text as a whole. Is it possible to use this approach in your application? Staszek
Re: specifying the doc id in clustering component
The solr schema has the fields, id, name and desc. I would like to get docs:["name Field here" ] instead of the doc Id field as in "docs":["200066", "195650", On Wednesday, August 18, 2010, Stanislaw Osinski wrote: > Hi Tommy, > > I'm using the clustering component with solr 1.4. >> >> The response is given by the id field in the doc array like: >> "labels":["Devices"], >> "docs":["200066", >> "195650", >> "204850", >> Is there a way to change the doc label to be another field? >> >> i couldn't this option in http://wiki.apache.org/solr/ClusteringComponent > > > I'm not sure if I get you right. The "labels" field is generated by the > clustering engine, it's a description of the group (cluster) of documents. > The description is usually a phrase or a number of phrases. The "docs" field > lists the ids of documents that the algorithm assigned to the cluster. > > Can you give an example of the input and output you'd expect? > > Thanks! > > Stanislaw >
Re: specifying the doc id in clustering component
Hi Tommy, I'm using the clustering component with solr 1.4. > > The response is given by the id field in the doc array like: >"labels":["Devices"], >"docs":["200066", > "195650", > "204850", > Is there a way to change the doc label to be another field? > > i couldn't this option in http://wiki.apache.org/solr/ClusteringComponent I'm not sure if I get you right. The "labels" field is generated by the clustering engine, it's a description of the group (cluster) of documents. The description is usually a phrase or a number of phrases. The "docs" field lists the ids of documents that the algorithm assigned to the cluster. Can you give an example of the input and output you'd expect? Thanks! Stanislaw
specifying the doc id in clustering component
I'm using the clustering component with solr 1.4. The response is given by the id field in the doc array like: "labels":["Devices"], "docs":["200066", "195650", "204850", Is there a way to change the doc label to be another field? i couldn't this option in http://wiki.apache.org/solr/ClusteringComponent -- @tommychheng Programmer and UC Irvine Graduate Student Find a great grad school based on research interests: http://gradschoolnow.com
Re: clustering component
Hey thanks Stanislaw! I'm going to try this against the current trunk tonight and see what happens. Matt On Wed, Jul 28, 2010 at 8:41 AM, Stanislaw Osinski < stanislaw.osin...@carrotsearch.com> wrote: > > The patch should also work with trunk, but I haven't verified it yet. > > > > I've just added a patch against solr trunk to > https://issues.apache.org/jira/browse/SOLR-1804. > > S. >
Re: clustering component
> The patch should also work with trunk, but I haven't verified it yet. > I've just added a patch against solr trunk to https://issues.apache.org/jira/browse/SOLR-1804. S.
Re: clustering component
Hi Matt, I'm attempting to get the carrot based clustering component (in trunk) to > work. I see that the clustering contrib has been disabled for the time > being. Does anyone know if this will be re-enabled soon, or even better, > know how I could get it working as it is? > I've recently created a patch to update the clustering algorithms in branch_3x: https://issues.apache.org/jira/browse/SOLR-1804 The patch should also work with trunk, but I haven't verified it yet. S.
clustering component
Hi, I'm attempting to get the carrot based clustering component (in trunk) to work. I see that the clustering contrib has been disabled for the time being. Does anyone know if this will be re-enabled soon, or even better, know how I could get it working as it is? Thanks, Matt
Re: How to Combine Dismax Query Handler and Clustering Component
: How do we combine clustering component and Dismax query handler? The dismax *handler* is now just the SearchHandler with defType=dismax ... so if you follow the examples for setting up the clustering component on an instance of SearchHandler, all you have to do is configure that instance to use the DismaxQParserPlugin by using defType=dismax as a (default or invarient) query param. -Hoss
How to Combine Dismax Query Handler and Clustering Component
Hi, How do we combine clustering component and Dismax query handler? Regards, allahbaksh
