from:"Emmanuel Keller"

[
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Emmanuel Keller updated SOLR-236:
-

Attachment: field_collapsing_1.3.patch

Here is the patch for solr 1.3 rev 589395.

I made some performance improvment. No more cache. We are using bitdocset or
hashdocset using solrconfig.hashdocsetmaxsize variable.

Regards,
Emmanuel Keller.

Field collapsing

Key: SOLR-236
URL: https://issues.apache.org/jira/browse/SOLR-236
Project: Solr
Issue Type: New Feature
Components: search
Affects Versions: 1.3
Reporter: Emmanuel Keller
Attachments: field_collapsing_1.1.0.patch,
field_collapsing_1.3.patch, SOLR-236-FieldCollapsing.patch,
SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch

This patch include a new feature called Field collapsing.
Used in order to collapse a group of results with similar value for a given
field to a single entry in the result set. Site collapsing is a special case
of this, where all results for a given web site is collapsed into one or two
entries in the result set, typically with an associated more documents from
this site link. See also Duplicate detection.
http://www.fastsearch.com/glossary.aspx?m=48amid=299
The implementation add 3 new query parameters (SolrParams):
collapse.field to choose the field used to group results
collapse.type normal (default value) or adjacent
collapse.max to select how many continuous results are allowed before
collapsing
TODO (in progress):
- More documentation (on source code)
- Test cases
Two patches:
- field_collapsing.patch for current development version
- field_collapsing_1.1.0.patch for Solr-1.1.0
P.S.: Feedback and misspelling correction are welcome ;-)

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Issue Comment Edited: (SOLR-236) Field collapsing

[
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12538339
]

ekeller edited comment on SOLR-236 at 10/28/07 1:55 PM:

Here is the patch for solr 1.3 rev 589395.

I made some performance improvement. No more cache. I use bitdocset or
hashdocset depending on solrconfig.hashdocsetmaxsize variable.

Regards,
Emmanuel Keller.

was (Author: ekeller):
Here is the patch for solr 1.3 rev 589395.

I made some performance improvment. No more cache. We are using bitdocset or
hashdocset using solrconfig.hashdocsetmaxsize variable.

Regards,
Emmanuel Keller.

Field collapsing

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing

[
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Emmanuel Keller updated SOLR-236:
-

Attachment: field_collapsing_1.3.patch

Thank you Yonik !
Here is the complete version.

P.S.: It's time to go to bed in Europe ...

Emmanuel.

Field collapsing

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

Attachment: (was: field_collapsing_1.3.patch)

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.3
Reporter: Emmanuel Keller
 Attachments: field_collapsing_1.1.0.patch, 
 field_collapsing_1.3.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing

2007-06-27 Thread Emmanuel Keller (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

Attachment: SOLR-236-FieldCollapsing.patch

This new patch resolves a performance issues.
I have added time informations for monitoring performances:

str name=time57/5/str

The first value is the elapsed time (in milliseconds) needed to compute 
collapsed informations (CollapseFilter.ajacentCollapse method).
The second value is the elapsed time needed to compute results informations 
(CollapseFilter.getMoreResults method).

We are using Solr (with collapsing patch) on a large index in production 
environnment (120GB with more than 3 000 000 documents).

P.S.: This time, the patch is relative to the solr root directory.

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.3
Reporter: Emmanuel Keller
 Attachments: field_collapsing_1.1.0.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236-FieldCollapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing

2007-06-11 Thread Emmanuel Keller (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

Attachment: SOLR-236-FieldCollapsing.patch

New release:
- Fieldcollapsing added on DisMaxRequestHandler
- Types are correctly handled on collapsed field

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller
 Attachments: field_collapsing_1.1.0.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236-FieldCollapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version (1.2)
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing

2007-06-11 Thread Emmanuel Keller (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

Attachment: (was: SOLR-236-FieldCollapsing.patch)

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller
 Attachments: field_collapsing_1.1.0.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version (1.2)
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing

2007-06-11 Thread Emmanuel Keller (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

  Description: 
This patch include a new feature called Field collapsing.

Used in order to collapse a group of results with similar value for a given 
field to a single entry in the result set. Site collapsing is a special case of 
this, where all results for a given web site is collapsed into one or two 
entries in the result set, typically with an associated more documents from 
this site link. See also Duplicate detection.
http://www.fastsearch.com/glossary.aspx?m=48amid=299

The implementation add 3 new query parameters (SolrParams):
collapse.field to choose the field used to group results
collapse.type normal (default value) or adjacent
collapse.max to select how many continuous results are allowed before 
collapsing

TODO (in progress):
- More documentation (on source code)
- Test cases

Two patches:
- field_collapsing.patch for current development version
- field_collapsing_1.1.0.patch for Solr-1.1.0


P.S.: Feedback and misspelling correction are welcome ;-)

  was:
This patch include a new feature called Field collapsing.

Used in order to collapse a group of results with similar value for a given 
field to a single entry in the result set. Site collapsing is a special case of 
this, where all results for a given web site is collapsed into one or two 
entries in the result set, typically with an associated more documents from 
this site link. See also Duplicate detection.
http://www.fastsearch.com/glossary.aspx?m=48amid=299

The implementation add 3 new query parameters (SolrParams):
collapse.field to choose the field used to group results
collapse.type normal (default value) or adjacent
collapse.max to select how many continuous results are allowed before 
collapsing

TODO (in progress):
- More documentation (on source code)
- Test cases

Two patches:
- field_collapsing.patch for current development version (1.2)
- field_collapsing_1.1.0.patch for Solr-1.1.0


P.S.: Feedback and misspelling correction are welcome ;-)

Affects Version/s: (was: 1.2)
   1.3

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.3
Reporter: Emmanuel Keller
 Attachments: field_collapsing_1.1.0.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing

[
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Emmanuel Keller updated SOLR-236:
-

Attachment: SOLR-236-FieldCollapsing.patch

Sorry, my last post was buggy. Here is the correct one. There is no more
exception now.
About tokens, if any token matches within the field it will collapse.
When I start implementing collapsing, my need was to to group documents having
exact identical field.

I believe that faceting has identical behavior. Lookt at Graphic card as
example:
http://localhost:8983/solr/select/?q=cat:graphic%20cardversion=2.2start=0rows=10indent=onfacet=truefacet.field=cat

I will try to maintain the wiki page.

Field collapsing

Key: SOLR-236
URL: https://issues.apache.org/jira/browse/SOLR-236
Project: Solr
Issue Type: New Feature
Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller
Attachments: collapse_field.patch, collapse_field.patch,
field_collapsing.patch, field_collapsing.patch, field_collapsing.patch,
field_collapsing_1.1.0.patch, SOLR-236-FieldCollapsing.patch,
SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch

This patch include a new feature called Field collapsing.
Used in order to collapse a group of results with similar value for a given
field to a single entry in the result set. Site collapsing is a special case
of this, where all results for a given web site is collapsed into one or two
entries in the result set, typically with an associated more documents from
this site link. See also Duplicate detection.
http://www.fastsearch.com/glossary.aspx?m=48amid=299
The implementation add 3 new query parameters (SolrParams):
collapse.field to choose the field used to group results
collapse.type normal (default value) or adjacent
collapse.max to select how many continuous results are allowed before
collapsing
TODO (in progress):
- More documentation (on source code)
- Test cases
Two patches:
- field_collapsing.patch for current development version (1.2)
- field_collapsing_1.1.0.patch for Solr-1.1.0
P.S.: Feedback and misspelling correction are welcome ;-)

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

Attachment: (was: collapse_field.patch)

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller
 Attachments: field_collapsing_1.1.0.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version (1.2)
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

Attachment: (was: field_collapsing.patch)

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller
 Attachments: field_collapsing_1.1.0.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version (1.2)
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

Attachment: (was: field_collapsing.patch)

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller
 Attachments: field_collapsing_1.1.0.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version (1.2)
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

Attachment: (was: field_collapsing.patch)

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller
 Attachments: field_collapsing_1.1.0.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version (1.2)
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

Attachment: (was: SOLR-236-FieldCollapsing.patch)

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller
 Attachments: field_collapsing_1.1.0.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version (1.2)
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-236) Field collapsing

[
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501578
]

Emmanuel Keller commented on SOLR-236:
--

Adjacent collapsing is useful because it preserves the pertinence of the sort.
The sorting is not modified. I copy the current sort to do a new search.

I am currently working on taking care of type field (int).

Field collapsing

This patch include a new feature called Field collapsing.
Used in order to collapse a group of results with similar value for a given
field to a single entry in the result set. Site collapsing is a special case
of this, where all results for a given web site is collapsed into one or two
entries in the result set, typically with an associated more documents from
this site link. See also Duplicate detection.
http://www.fastsearch.com/glossary.aspx?m=48amid=299
The implementation add 3 new query parameters (SolrParams):
collapse.field to choose the field used to group results
collapse.type normal (default value) or adjacent
collapse.max to select how many continuous results are allowed before
collapsing
TODO (in progress):
- More documentation (on source code)
- Test cases
Two patches:
- field_collapsing.patch for current development version (1.2)
- field_collapsing_1.1.0.patch for Solr-1.1.0
P.S.: Feedback and misspelling correction are welcome ;-)

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-236) Field collapsing


[ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501583
 ] 

Emmanuel Keller commented on SOLR-236:
--

Correct, except that collapse result is only used as filter to the final result 
to hide collapsed documents.

P.S.: Sorry, if my answers are a little short, I am not perfectly fluent in 
english.

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller
 Attachments: field_collapsing_1.1.0.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version (1.2)
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing

2007-06-04 Thread Emmanuel Keller (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

Attachment: SOLR-236-FieldCollapsing.patch

Right, It's more useful.

This new version includes the result as you expect it.

You should add the following constraint on the wiki: The collapsing field must 
be un-tokenized.

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller
 Attachments: collapse_field.patch, collapse_field.patch, 
 field_collapsing.patch, field_collapsing.patch, field_collapsing.patch, 
 field_collapsing_1.1.0.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236-FieldCollapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version (1.2)
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing

2007-05-19 Thread Emmanuel Keller (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Emmanuel Keller updated SOLR-236:
-

Attachment: field_collapsing.patch

The last version of the patch.

- Results are now cached using CollapseCache (a new instance of SolrCache
added on solrconfig.xml)
- The parameter collapse has been removed.

This version has been fully tested.

Feedbacks are welcome.

Field collapsing

This patch include a new feature called Field collapsing.
Used in order to collapse a group of results with similar value for a given
field to a single entry in the result set. Site collapsing is a special case
of this, where all results for a given web site is collapsed into one or two
entries in the result set, typically with an associated more documents from
this site link. See also Duplicate detection.
http://www.fastsearch.com/glossary.aspx?m=48amid=299
The implementation add 4 new query parameters (SolrParams):
collapse set to true to enable collapsing.
collapse.field to choose the field used to group results
collapse.type normal (default value) or adjacent
collapse.max to select how many continuous results are allowed before
collapsing
TODO (in progress):
- More documentation (on source code)
- Test cases

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing

2007-05-19 Thread Emmanuel Keller (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

Attachment: field_collapsing_1.1.0.patch

I still maintain a version for the release 1.1.0 (The version we used on our 
production environment).

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller
 Attachments: collapse_field.patch, collapse_field.patch, 
 field_collapsing.patch, field_collapsing.patch, field_collapsing.patch, 
 field_collapsing_1.1.0.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 4 new query parameters (SolrParams):
 collapse set to true to enable collapsing.
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-236) Field collapsing

2007-05-18 Thread Emmanuel Keller (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12496805
]

Emmanuel Keller commented on SOLR-236:
--

You're right. As collapse.field is a required field, we don't need more
information. My first idea was to copy the behavior of facet.

Field collapsing

This patch include a new feature called Field collapsing.
Used in order to collapse a group of results with similar value for a given
field to a single entry in the result set. Site collapsing is a special case
of this, where all results for a given web site is collapsed into one or two
entries in the result set, typically with an associated more documents from
this site link. See also Duplicate detection.
http://www.fastsearch.com/glossary.aspx?m=48amid=299
The implementation add 4 new query parameters (SolrParams):
collapse set to true to enable collapsing.
collapse.field to choose the field used to group results
collapse.type normal (default value) or adjacent
collapse.max to select how many continuous results are allowed before
collapsing
TODO (in progress):
- More documentation (on source code)
- Test cases

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing

2007-05-14 Thread Emmanuel Keller (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Emmanuel Keller updated SOLR-236:
-

Attachment: field_collapsing.patch

Corrects a bug on the previous version when using a value greater than 1 as
collapse.max parameter.

Field collapsing

This patch include a new feature called Field collapsing.
Used in order to collapse a group of results with similar value for a given
field to a single entry in the result set. Site collapsing is a special case
of this, where all results for a given web site is collapsed into one or two
entries in the result set, typically with an associated more documents from
this site link. See also Duplicate detection.
http://www.fastsearch.com/glossary.aspx?m=48amid=299
The implementation add 3 new query parameters (SolrParams):
collapse set to true to enable collapsing.
collapse.field to choose the field used to group results
collapse.max to select how many continuous results are allowed before
collapsing
TODO (in progress):
- More documentation (on source code)
- Test cases

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-236) Field collapsing

2007-05-13 Thread Emmanuel Keller (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12495376
]

Emmanuel Keller commented on SOLR-236:
--

Yonik,

You are right, only adjacent documents are collapsed.
I work on a large index ( 2.000.000 documents) growing every day. The first
goal was to group results, preserving score ranking and achieving good
performances. This light implementation meets our needs.
I am currently working on a second implementation taking care of the semantics.

P.S.: Congratulations for this great application.

Field collapsing

This patch include a new feature called Field collapsing.
Used in order to collapse a group of results with similar value for a given
field to a single entry in the result set. Site collapsing is a special case
of this, where all results for a given web site is collapsed into one or two
entries in the result set, typically with an associated more documents from
this site link. See also Duplicate detection.
http://www.fastsearch.com/glossary.aspx?m=48amid=299
The implementation add 3 new query parameters (SolrParams):
collapse set to true to enable collapsing.
collapse.field to choose the field used to group results
collapse.max to select how many continuous results are allowed before
collapsing
TODO (in progress):
- More documentation (on source code)
- Test cases

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing

2007-05-13 Thread Emmanuel Keller (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Emmanuel Keller updated SOLR-236:
-

Attachment: field_collapsing.patch

This release is more conform with the semantics of field collapsing.

Parameters are:

collapse=true // enable collapsing
collapse.field=[field] // indexed field used for collapsing
collapse.max=[integer] // Start collapsing after n document
collapse.type=[normal|adjacent] // Default value is normal

- adjacent collapse only consecutive documents.
- normal collapse all documents having equal collapsing field.

Field collapsing

This patch include a new feature called Field collapsing.
Used in order to collapse a group of results with similar value for a given
field to a single entry in the result set. Site collapsing is a special case
of this, where all results for a given web site is collapsed into one or two
entries in the result set, typically with an associated more documents from
this site link. See also Duplicate detection.
http://www.fastsearch.com/glossary.aspx?m=48amid=299
The implementation add 3 new query parameters (SolrParams):
collapse set to true to enable collapsing.
collapse.field to choose the field used to group results
collapse.max to select how many continuous results are allowed before
collapsing
TODO (in progress):
- More documentation (on source code)
- Test cases

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-237) Field collapsing

2007-05-12 Thread Emmanuel Keller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12495244
 ] 

Emmanuel Keller commented on SOLR-237:
--

Ryan,

I used the following svn 
path:http://svn.apache.org/repos/asf/lucene/solr/tags/release-1.1.0
Last changed revision: 489774

Shoud I use this one ?
http://svn.apache.org/repos/asf/lucene/solr/branches/branch-1.1
Last changed revision: 488066

Thanks for you reply
Emmanuel.

 Field collapsing
 

 Key: SOLR-237
 URL: https://issues.apache.org/jira/browse/SOLR-237
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.1.0
Reporter: Emmanuel Keller
 Attachments: field_collapsing_1.1.0.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse set to true to enable collapsing.
 collapse.field to choose the field used to group results
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-237) Field collapsing

2007-05-12 Thread Emmanuel Keller (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-237:
-

Attachment: field_collapsing-1.1.patch

Patch from http://svn.apache.org/repos/asf/lucene/solr/branches/branch-1.1

 Field collapsing
 

 Key: SOLR-237
 URL: https://issues.apache.org/jira/browse/SOLR-237
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.1.0
Reporter: Emmanuel Keller
 Attachments: field_collapsing-1.1.patch, field_collapsing_1.1.0.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse set to true to enable collapsing.
 collapse.field to choose the field used to group results
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (SOLR-236) Field collapsing

Field collapsing


 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller


This patch include a new feature called Field collapsing.

Used in order to collapse a group of results with similar value for a given 
field to a single entry in the result set. Site collapsing is a special case of 
this, where all results for a given web site is collapsed into one or two 
entries in the result set, typically with an associated more documents from 
this site link. See also Duplicate detection.
http://www.fastsearch.com/glossary.aspx?m=48amid=299

The implementation add 3 new query parameters (SolrParams):
collapse set to true to enable collapsing.
collapse.field to choose the field used to group results
collapse.max to select how many continuous results are allowed before 
collapsing

TODO (in progress):
- More documentation (on source code)
- Test cases


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-236) Field collapsing


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

Attachment: collapse_field.patch

Field Collapsing

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller
 Attachments: collapse_field.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse set to true to enable collapsing.
 collapse.field to choose the field used to group results
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Re: Field collapsing functionality

2007-05-11 Thread Emmanuel Keller


Thank you for your help !

Here is the patch:
https://issues.apache.org/jira/browse/SOLR-236

Functionnal, but not yet fully tested.
Working well on my index (more that 2.000.000 entries).

I am working on test cases now.

Best regards,
Emmanuel.


On 5/11/07, Ryan McKinley [EMAIL PROTECTED] wrote:



Yes, this would be great!

Here is an old related topic:
http://www.nabble.com/result-grouping--tf2910425.html#a8131895

If you need any help with JIRA etc, feel free to ask

ryan


Otis Gospodnetic wrote:
 Emmanuel,

 This sounds useful!
 Here is everything you'll need to know:
http://wiki.apache.org/solr/HowToContribute

 Thanks,
 Otis
 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Simpy -- http://www.simpy.com/  -  Tag  -  Search  -  Share

 - Original Message 
 From: Emmanuel Keller [EMAIL PROTECTED]
 To: solr-dev@lucene.apache.org
 Sent: Thursday, May 10, 2007 5:19:09 PM
 Subject: Field collapsing functionality

 Hi,

 My name is Emmanuel Keller. I am an engineer working as technical
manager
 for a french company.

 For some projects, I identified that Lucene was the search engine we
need.
 I worked hard to successfully integrate Solr on the first project:
 http://www.usinenouvelle.com/expo

 For the next project, (not yet online), I needed a collapse
functionality,
 described here: http://www.fastsearch.com/glossary.aspx?m=48amid=299.

 Finally, I did it on Solr-1.1.1-dev.

 If you are agree, I propose to commit the version for the current trunk
 version.

 Sincerely yours,
 Emmanuel.


 P.S.: English is not my native language.







--
Emmanuel Keller.
Directeur des Systèmes d'Information
GISI Interactive
12-14 rue Médéric - 75017 PARIS
tél. : 33 (0)1 56 79 41 30
fax : 33 (0)1 43 80 44 28
mobile : 33 (0)6 84 09 99 05
e.mail : [EMAIL PROTECTED]
http://www.usinenouvelle.com

[jira] Updated: (SOLR-236) Field collapsing


 [ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated SOLR-236:
-

Attachment: collapse_field.patch

Remplacing HashDocSet by BitDocSet for hasMoreResult for better performances

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.2
Reporter: Emmanuel Keller
 Attachments: collapse_field.patch, collapse_field.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse set to true to enable collapsing.
 collapse.field to choose the field used to group results
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (SOLR-237) Field collapsing

Field collapsing


 Key: SOLR-237
 URL: https://issues.apache.org/jira/browse/SOLR-237
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.1.0
Reporter: Emmanuel Keller


This patch include a new feature called Field collapsing.

Used in order to collapse a group of results with similar value for a given 
field to a single entry in the result set. Site collapsing is a special case of 
this, where all results for a given web site is collapsed into one or two 
entries in the result set, typically with an associated more documents from 
this site link. See also Duplicate detection.
http://www.fastsearch.com/glossary.aspx?m=48amid=299

The implementation add 3 new query parameters (SolrParams):
collapse set to true to enable collapsing.
collapse.field to choose the field used to group results
collapse.max to select how many continuous results are allowed before 
collapsing

TODO (in progress):
- More documentation (on source code)
- Test cases 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-237) Field collapsing