[
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011563#comment-13011563
]
Stephen Weiss commented on SOLR-236:
------------------------------------
Just be careful Yuriy, there are reasons why this thing is not in Solr 1.4.1
already :-) The code does not scale particularly well beyond a few million
documents, especially if you use the version that preserves totals and paging.
It was enough to keep my software from being scrapped, but if you plan on
scaling much past that point any time soon, you may need to start thinking
about alternative solutions. I know I certainly am... I have a sinking worry
my application may outgrow the limits of this patch's stability before
something truly production ready comes to fore, possibly even this year if
growth continues. However, given that the very concept of grouping is critical
to the site that I support with SOLR, and attempts to provide the same
functionality without actually grouping have failed repeatedly over the past
few months, it is very sadly starting to look like I will have to cut very
useful features (to no end of complaints, I'm sure) in order to ensure it's
overall stability unless some miracle happens. Mama always told me I should
have learned Java!
Long story short, if you don't have to have this patch yet, and your software
hasn't been written to do anything like this yet, I would not start doing it
now! You will regret it when you run out of options later on and your servers
start crashing all over the place. See if you can keep it under wraps until a
real release comes out with it.
> Field collapsing
> ----------------
>
> Key: SOLR-236
> URL: https://issues.apache.org/jira/browse/SOLR-236
> Project: Solr
> Issue Type: New Feature
> Components: search
> Affects Versions: 1.3
> Reporter: Emmanuel Keller
> Assignee: Shalin Shekhar Mangar
> Fix For: Next
>
> Attachments: DocSetScoreCollector.java,
> NonAdjacentDocumentCollapser.java, NonAdjacentDocumentCollapserTest.java,
> SOLR-236-1_4_1-NPEfix.patch, SOLR-236-1_4_1-paging-totals-working.patch,
> SOLR-236-1_4_1.patch, SOLR-236-FieldCollapsing.patch,
> SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch,
> SOLR-236-branch_3x.patch, SOLR-236-distinctFacet.patch, SOLR-236-trunk.patch,
> SOLR-236-trunk.patch, SOLR-236-trunk.patch, SOLR-236-trunk.patch,
> SOLR-236-trunk.patch, SOLR-236.patch, SOLR-236.patch, SOLR-236.patch,
> SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, SOLR-236.patch,
> SOLR-236.patch, SOLR-236_collapsing.patch, SOLR-236_collapsing.patch,
> collapsing-patch-to-1.3.0-dieter.patch, collapsing-patch-to-1.3.0-ivan.patch,
> collapsing-patch-to-1.3.0-ivan_2.patch,
> collapsing-patch-to-1.3.0-ivan_3.patch, field-collapse-3.patch,
> field-collapse-4-with-solrj.patch, field-collapse-5.patch,
> field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch,
> field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch,
> field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch,
> field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch,
> field-collapse-5.patch, field-collapse-5.patch,
> field-collapse-solr-236-2.patch, field-collapse-solr-236.patch,
> field-collapsing-extended-592129.patch, field_collapsing_1.1.0.patch,
> field_collapsing_1.3.patch, field_collapsing_dsteigerwald.diff,
> field_collapsing_dsteigerwald.diff, field_collapsing_dsteigerwald.diff,
> quasidistributed.additional.patch, solr-236.patch
>
>
> This patch include a new feature called "Field collapsing".
> "Used in order to collapse a group of results with similar value for a given
> field to a single entry in the result set. Site collapsing is a special case
> of this, where all results for a given web site is collapsed into one or two
> entries in the result set, typically with an associated "more documents from
> this site" link. See also Duplicate detection."
> http://www.fastsearch.com/glossary.aspx?m=48&amid=299
> The implementation add 3 new query parameters (SolrParams):
> "collapse.field" to choose the field used to group results
> "collapse.type" normal (default value) or adjacent
> "collapse.max" to select how many continuous results are allowed before
> collapsing
> TODO (in progress):
> - More documentation (on source code)
> - Test cases
> Two patches:
> - "field_collapsing.patch" for current development version
> - "field_collapsing_1.1.0.patch" for Solr-1.1.0
> P.S.: Feedback and misspelling correction are welcome ;-)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
