date:20100218

java.io.IOException: read past EOF
--

 Key: SOLR-1778
 URL: https://issues.apache.org/jira/browse/SOLR-1778
 Project: Solr
  Issue Type: Bug
Affects Versions: 1.4
Reporter: Yonik Seeley
Priority: Critical
 Fix For: 1.5


A query with relevancy scores of all zeros produces an invalid doclist that 
includes sentinel values 2147483647 and causes Solr to request that invalid 
docid from Lucene which results in a java.io.IOException: read past EOF

http://search.lucidimagination.com/search/document/2d5359c0e0d103be/java_io_ioexception_read_past_eof_after_solr_1_4_0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Assigned: (SOLR-1778) java.io.IOException: read past EOF


 [ 
https://issues.apache.org/jira/browse/SOLR-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley reassigned SOLR-1778:
--

Assignee: Yonik Seeley

 java.io.IOException: read past EOF
 --

 Key: SOLR-1778
 URL: https://issues.apache.org/jira/browse/SOLR-1778
 Project: Solr
  Issue Type: Bug
Affects Versions: 1.4
Reporter: Yonik Seeley
Assignee: Yonik Seeley
Priority: Critical
 Fix For: 1.5


 A query with relevancy scores of all zeros produces an invalid doclist that 
 includes sentinel values 2147483647 and causes Solr to request that invalid 
 docid from Lucene which results in a java.io.IOException: read past EOF
 http://search.lucidimagination.com/search/document/2d5359c0e0d103be/java_io_ioexception_read_past_eof_after_solr_1_4_0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1778) java.io.IOException: read past EOF


[ 
https://issues.apache.org/jira/browse/SOLR-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835209#action_12835209
 ] 

Yonik Seeley commented on SOLR-1778:


This is a lucene bug: LUCENE-2270

 java.io.IOException: read past EOF
 --

 Key: SOLR-1778
 URL: https://issues.apache.org/jira/browse/SOLR-1778
 Project: Solr
  Issue Type: Bug
Affects Versions: 1.4
Reporter: Yonik Seeley
Assignee: Yonik Seeley
Priority: Critical
 Fix For: 1.5


 A query with relevancy scores of all zeros produces an invalid doclist that 
 includes sentinel values 2147483647 and causes Solr to request that invalid 
 docid from Lucene which results in a java.io.IOException: read past EOF
 http://search.lucidimagination.com/search/document/2d5359c0e0d103be/java_io_ioexception_read_past_eof_after_solr_1_4_0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-236) Field collapsing


[ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835230#action_12835230
 ] 

Peter Karich commented on SOLR-236:
---

We are facing OutOfMemory problems too. We are using 
https://issues.apache.org/jira/secure/attachment/12425775/field-collapse-5.patch

 Are you using any other features besides plain collapsing? The field collapse 
 cache gets large very quickly,
 I suggest you turn it off (if you are using it). Also you can try to make 
 your filterCache smaller.

How can I turn off the collapse cache or make the filterCache smaller?
Are there other workarounds? E.g. via using a special version of the patch ?

I read that it could help to specify collapse.maxdocs but this didn't help in 
our case ... could collapse.type=adjacent help here?  
(https://issues.apache.org/jira/browse/SOLR-236?focusedCommentId=12495376page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12495376)

What do you think?

BTW: We really like this patch and would like to use it !! :-)

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.3
Reporter: Emmanuel Keller
Assignee: Shalin Shekhar Mangar
 Fix For: 1.5

 Attachments: collapsing-patch-to-1.3.0-dieter.patch, 
 collapsing-patch-to-1.3.0-ivan.patch, collapsing-patch-to-1.3.0-ivan_2.patch, 
 collapsing-patch-to-1.3.0-ivan_3.patch, field-collapse-3.patch, 
 field-collapse-4-with-solrj.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-solr-236-2.patch, field-collapse-solr-236.patch, 
 field-collapsing-extended-592129.patch, field_collapsing_1.1.0.patch, 
 field_collapsing_1.3.patch, field_collapsing_dsteigerwald.diff, 
 field_collapsing_dsteigerwald.diff, field_collapsing_dsteigerwald.diff, 
 quasidistributed.additional.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, 
 SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, solr-236.patch, 
 SOLR-236_collapsing.patch, SOLR-236_collapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-236) Field collapsing


[ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835258#action_12835258
 ] 

Peter Karich commented on SOLR-236:
---

Trying the latest patch from 1th Feb 2010 compiles against solr-2010-02-13 from 
nightly build but does not work. If I query 

http://searchdev05:15100/cs-bidcs/select?q=*:*collapse.field=myfield

it fails with: 

{noformat} 

HTTP Status 500 - null java.lang.NullPointerException at 
org.apache.solr.schema.FieldType.toExternal(FieldType.java:329) at 
org.apache.solr.schema.FieldType.storedToReadable(FieldType.java:348) at 
org.apache.solr.search.fieldcollapse.collector.AbstractCollapseCollector.getCollapseGroupResult(AbstractCollapseCollector.java:58)
 at 
org.apache.solr.search.fieldcollapse.collector.DocumentGroupCountCollapseCollectorFactory$DocumentCountCollapseCollector.getResult(DocumentGroupCountCollapseCollectorFactory.ja
va:84) at 
org.apache.solr.search.fieldcollapse.AbstractDocumentCollapser.getCollapseInfo(AbstractDocumentCollapser.java:193)
 at 
org.apache.solr.handler.component.CollapseComponent.doProcess(CollapseComponent.java:192)
 at 
org.apache.solr.handler.component.CollapseComponent.process(CollapseComponent.java:127)
 at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
 at
...
 {noformat} 


I only need the OutOfMemory problem solved ... :-(

 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.3
Reporter: Emmanuel Keller
Assignee: Shalin Shekhar Mangar
 Fix For: 1.5

 Attachments: collapsing-patch-to-1.3.0-dieter.patch, 
 collapsing-patch-to-1.3.0-ivan.patch, collapsing-patch-to-1.3.0-ivan_2.patch, 
 collapsing-patch-to-1.3.0-ivan_3.patch, field-collapse-3.patch, 
 field-collapse-4-with-solrj.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-solr-236-2.patch, field-collapse-solr-236.patch, 
 field-collapsing-extended-592129.patch, field_collapsing_1.1.0.patch, 
 field_collapsing_1.3.patch, field_collapsing_dsteigerwald.diff, 
 field_collapsing_dsteigerwald.diff, field_collapsing_dsteigerwald.diff, 
 quasidistributed.additional.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, 
 SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, solr-236.patch, 
 SOLR-236_collapsing.patch, SOLR-236_collapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to select how many continuous results are allowed before 
 collapsing
 TODO (in progress):
 - More documentation (on source code)
 - Test cases
 Two patches:
 - field_collapsing.patch for current development version
 - field_collapsing_1.1.0.patch for Solr-1.1.0
 P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Issue Comment Edited: (SOLR-236) Field collapsing


[ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835258#action_12835258
 ] 

Peter Karich edited comment on SOLR-236 at 2/18/10 4:06 PM:


Trying the latest patch from 1th Feb 2010 compiles against solr-2010-02-13 from 
nightly build but does not work. If I query 

http://server/cs-bidcs/select?q=*:*collapse.field=myfield

it fails with: 

{noformat} 

HTTP Status 500 - null java.lang.NullPointerException at 
org.apache.solr.schema.FieldType.toExternal(FieldType.java:329) at 
org.apache.solr.schema.FieldType.storedToReadable(FieldType.java:348) at 
org.apache.solr.search.fieldcollapse.collector.AbstractCollapseCollector.getCollapseGroupResult(AbstractCollapseCollector.java:58)
 at 
org.apache.solr.search.fieldcollapse.collector.DocumentGroupCountCollapseCollectorFactory$DocumentCountCollapseCollector.getResult(DocumentGroupCountCollapseCollectorFactory.ja
va:84) at 
org.apache.solr.search.fieldcollapse.AbstractDocumentCollapser.getCollapseInfo(AbstractDocumentCollapser.java:193)
 at 
org.apache.solr.handler.component.CollapseComponent.doProcess(CollapseComponent.java:192)
 at 
org.apache.solr.handler.component.CollapseComponent.process(CollapseComponent.java:127)
 at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
 at
...
 {noformat} 


I only need the OutOfMemory problem solved ... :-(

  was (Author: peathal):
Trying the latest patch from 1th Feb 2010 compiles against solr-2010-02-13 
from nightly build but does not work. If I query 

http://searchdev05:15100/cs-bidcs/select?q=*:*collapse.field=myfield

it fails with: 

{noformat} 

HTTP Status 500 - null java.lang.NullPointerException at 
org.apache.solr.schema.FieldType.toExternal(FieldType.java:329) at 
org.apache.solr.schema.FieldType.storedToReadable(FieldType.java:348) at 
org.apache.solr.search.fieldcollapse.collector.AbstractCollapseCollector.getCollapseGroupResult(AbstractCollapseCollector.java:58)
 at 
org.apache.solr.search.fieldcollapse.collector.DocumentGroupCountCollapseCollectorFactory$DocumentCountCollapseCollector.getResult(DocumentGroupCountCollapseCollectorFactory.ja
va:84) at 
org.apache.solr.search.fieldcollapse.AbstractDocumentCollapser.getCollapseInfo(AbstractDocumentCollapser.java:193)
 at 
org.apache.solr.handler.component.CollapseComponent.doProcess(CollapseComponent.java:192)
 at 
org.apache.solr.handler.component.CollapseComponent.process(CollapseComponent.java:127)
 at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
 at
...
 {noformat} 


I only need the OutOfMemory problem solved ... :-(
  
 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.3
Reporter: Emmanuel Keller
Assignee: Shalin Shekhar Mangar
 Fix For: 1.5

 Attachments: collapsing-patch-to-1.3.0-dieter.patch, 
 collapsing-patch-to-1.3.0-ivan.patch, collapsing-patch-to-1.3.0-ivan_2.patch, 
 collapsing-patch-to-1.3.0-ivan_3.patch, field-collapse-3.patch, 
 field-collapse-4-with-solrj.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-solr-236-2.patch, field-collapse-solr-236.patch, 
 field-collapsing-extended-592129.patch, field_collapsing_1.1.0.patch, 
 field_collapsing_1.3.patch, field_collapsing_dsteigerwald.diff, 
 field_collapsing_dsteigerwald.diff, field_collapsing_dsteigerwald.diff, 
 quasidistributed.additional.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, 
 SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, solr-236.patch, 
 SOLR-236_collapsing.patch, SOLR-236_collapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent

[jira] Issue Comment Edited: (SOLR-236) Field collapsing


[ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835258#action_12835258
 ] 

Peter Karich edited comment on SOLR-236 at 2/18/10 4:06 PM:


Trying the latest patch from 1th Feb 2010 compiles against solr-2010-02-13 from 
nightly build but does not work. If I query 

http://server/solr-app/select?q=*:*collapse.field=myfield

it fails with: 

{noformat} 

HTTP Status 500 - null java.lang.NullPointerException at 
org.apache.solr.schema.FieldType.toExternal(FieldType.java:329) at 
org.apache.solr.schema.FieldType.storedToReadable(FieldType.java:348) at 
org.apache.solr.search.fieldcollapse.collector.AbstractCollapseCollector.getCollapseGroupResult(AbstractCollapseCollector.java:58)
 at 
org.apache.solr.search.fieldcollapse.collector.DocumentGroupCountCollapseCollectorFactory$DocumentCountCollapseCollector.getResult(DocumentGroupCountCollapseCollectorFactory.ja
va:84) at 
org.apache.solr.search.fieldcollapse.AbstractDocumentCollapser.getCollapseInfo(AbstractDocumentCollapser.java:193)
 at 
org.apache.solr.handler.component.CollapseComponent.doProcess(CollapseComponent.java:192)
 at 
org.apache.solr.handler.component.CollapseComponent.process(CollapseComponent.java:127)
 at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
 at
...
 {noformat} 


I only need the OutOfMemory problem solved ... :-(

  was (Author: peathal):
Trying the latest patch from 1th Feb 2010 compiles against solr-2010-02-13 
from nightly build but does not work. If I query 

http://server/cs-bidcs/select?q=*:*collapse.field=myfield

it fails with: 

{noformat} 

HTTP Status 500 - null java.lang.NullPointerException at 
org.apache.solr.schema.FieldType.toExternal(FieldType.java:329) at 
org.apache.solr.schema.FieldType.storedToReadable(FieldType.java:348) at 
org.apache.solr.search.fieldcollapse.collector.AbstractCollapseCollector.getCollapseGroupResult(AbstractCollapseCollector.java:58)
 at 
org.apache.solr.search.fieldcollapse.collector.DocumentGroupCountCollapseCollectorFactory$DocumentCountCollapseCollector.getResult(DocumentGroupCountCollapseCollectorFactory.ja
va:84) at 
org.apache.solr.search.fieldcollapse.AbstractDocumentCollapser.getCollapseInfo(AbstractDocumentCollapser.java:193)
 at 
org.apache.solr.handler.component.CollapseComponent.doProcess(CollapseComponent.java:192)
 at 
org.apache.solr.handler.component.CollapseComponent.process(CollapseComponent.java:127)
 at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
 at
...
 {noformat} 


I only need the OutOfMemory problem solved ... :-(
  
 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.3
Reporter: Emmanuel Keller
Assignee: Shalin Shekhar Mangar
 Fix For: 1.5

 Attachments: collapsing-patch-to-1.3.0-dieter.patch, 
 collapsing-patch-to-1.3.0-ivan.patch, collapsing-patch-to-1.3.0-ivan_2.patch, 
 collapsing-patch-to-1.3.0-ivan_3.patch, field-collapse-3.patch, 
 field-collapse-4-with-solrj.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-solr-236-2.patch, field-collapse-solr-236.patch, 
 field-collapsing-extended-592129.patch, field_collapsing_1.1.0.patch, 
 field_collapsing_1.3.patch, field_collapsing_dsteigerwald.diff, 
 field_collapsing_dsteigerwald.diff, field_collapsing_dsteigerwald.diff, 
 quasidistributed.additional.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, 
 SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, solr-236.patch, 
 SOLR-236_collapsing.patch, SOLR-236_collapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max to

[jira] Issue Comment Edited: (SOLR-236) Field collapsing


[ 
https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835258#action_12835258
 ] 

Peter Karich edited comment on SOLR-236 at 2/18/10 4:07 PM:


Trying the latest patch from 1th Feb 2010. It compiles against solr-2010-02-13 
from nightly build dir, but does not work. If I query 

http://server/solr-app/select?q=*:*collapse.field=myfield

it fails with: 

{noformat} 

HTTP Status 500 - null java.lang.NullPointerException at 
org.apache.solr.schema.FieldType.toExternal(FieldType.java:329) at 
org.apache.solr.schema.FieldType.storedToReadable(FieldType.java:348) at 
org.apache.solr.search.fieldcollapse.collector.AbstractCollapseCollector.getCollapseGroupResult(AbstractCollapseCollector.java:58)
 at 
org.apache.solr.search.fieldcollapse.collector.DocumentGroupCountCollapseCollectorFactory$DocumentCountCollapseCollector.getResult(DocumentGroupCountCollapseCollectorFactory.ja
va:84) at 
org.apache.solr.search.fieldcollapse.AbstractDocumentCollapser.getCollapseInfo(AbstractDocumentCollapser.java:193)
 at 
org.apache.solr.handler.component.CollapseComponent.doProcess(CollapseComponent.java:192)
 at 
org.apache.solr.handler.component.CollapseComponent.process(CollapseComponent.java:127)
 at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
 at
...
 {noformat} 


I only need the OutOfMemory problem solved ... :-(

  was (Author: peathal):
Trying the latest patch from 1th Feb 2010 compiles against solr-2010-02-13 
from nightly build but does not work. If I query 

http://server/solr-app/select?q=*:*collapse.field=myfield

it fails with: 

{noformat} 

HTTP Status 500 - null java.lang.NullPointerException at 
org.apache.solr.schema.FieldType.toExternal(FieldType.java:329) at 
org.apache.solr.schema.FieldType.storedToReadable(FieldType.java:348) at 
org.apache.solr.search.fieldcollapse.collector.AbstractCollapseCollector.getCollapseGroupResult(AbstractCollapseCollector.java:58)
 at 
org.apache.solr.search.fieldcollapse.collector.DocumentGroupCountCollapseCollectorFactory$DocumentCountCollapseCollector.getResult(DocumentGroupCountCollapseCollectorFactory.ja
va:84) at 
org.apache.solr.search.fieldcollapse.AbstractDocumentCollapser.getCollapseInfo(AbstractDocumentCollapser.java:193)
 at 
org.apache.solr.handler.component.CollapseComponent.doProcess(CollapseComponent.java:192)
 at 
org.apache.solr.handler.component.CollapseComponent.process(CollapseComponent.java:127)
 at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
 at
...
 {noformat} 


I only need the OutOfMemory problem solved ... :-(
  
 Field collapsing
 

 Key: SOLR-236
 URL: https://issues.apache.org/jira/browse/SOLR-236
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 1.3
Reporter: Emmanuel Keller
Assignee: Shalin Shekhar Mangar
 Fix For: 1.5

 Attachments: collapsing-patch-to-1.3.0-dieter.patch, 
 collapsing-patch-to-1.3.0-ivan.patch, collapsing-patch-to-1.3.0-ivan_2.patch, 
 collapsing-patch-to-1.3.0-ivan_3.patch, field-collapse-3.patch, 
 field-collapse-4-with-solrj.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-5.patch, field-collapse-5.patch, 
 field-collapse-solr-236-2.patch, field-collapse-solr-236.patch, 
 field-collapsing-extended-592129.patch, field_collapsing_1.1.0.patch, 
 field_collapsing_1.3.patch, field_collapsing_dsteigerwald.diff, 
 field_collapsing_dsteigerwald.diff, field_collapsing_dsteigerwald.diff, 
 quasidistributed.additional.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch, 
 SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, 
 SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, solr-236.patch, 
 SOLR-236_collapsing.patch, SOLR-236_collapsing.patch


 This patch include a new feature called Field collapsing.
 Used in order to collapse a group of results with similar value for a given 
 field to a single entry in the result set. Site collapsing is a special case 
 of this, where all results for a given web site is collapsed into one or two 
 entries in the result set, typically with an associated more documents from 
 this site link. See also Duplicate detection.
 http://www.fastsearch.com/glossary.aspx?m=48amid=299
 The implementation add 3 new query parameters (SolrParams):
 collapse.field to choose the field used to group results
 collapse.type normal (default value) or adjacent
 collapse.max

[jira] Reopened: (SOLR-1695) Missleading error message when adding docs with missing/multiple value(s) for uniqueKey field


 [ 
https://issues.apache.org/jira/browse/SOLR-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley reopened SOLR-1695:



reopening - this broke the build.

  Missleading error message when adding docs with missing/multiple value(s) 
 for uniqueKey field
 --

 Key: SOLR-1695
 URL: https://issues.apache.org/jira/browse/SOLR-1695
 Project: Solr
  Issue Type: Improvement
Reporter: Hoss Man
Assignee: Hoss Man
Priority: Minor
 Fix For: 1.5


 Sometimes users don't seem to notice/understand the uniqueKey/ declaration 
 in the example schema, and the error message they get if their documents 
 don't include that field is confusing...
 {code}
 org.apache.solr.common.SolrException: Document [null] missing required field: 
 id
 {code}
 ...because they get an almost identical error even if they remove 
 {{required=true}} from {{field name=id /}} in their schema.xml file.
 We should improve the error message so it's clear when a Document is missing 
 the uniqueKeyField (not just a required field) so they know the 
 terminology to look for in diagnosing the problem.
 http://old.nabble.com/solr-1.4-csv-import-Document-missing-required-field%3A-id-to26990048.html#a26990779

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1687) add param for limiting start and rows params


[ 
https://issues.apache.org/jira/browse/SOLR-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835288#action_12835288
 ] 

Yonik Seeley commented on SOLR-1687:


There are a lot of params that people may want to restrict control of... and 
I'm not sure it makes sense to add mins and maxes for all of them.  
Traditionally, this type of stuff  is delegated to the front-end clients to 
restrict.  Would it make more sense to add an optional component to check 
restrictions?  The restrictions could optionally be in the config for the 
component and thus wouldn't have to be looked up and parsed for every request.

 add param for limiting start and rows params
 

 Key: SOLR-1687
 URL: https://issues.apache.org/jira/browse/SOLR-1687
 Project: Solr
  Issue Type: Improvement
Reporter: Hoss Man
 Attachments: SOLR-1687.patch


 conventional wisdom is that it doesn't make sense to paginate with huge 
 pages, or to drill down deep into high numbered pages -- features like 
 faceting tend to be a better UI experience, and less intensive on solr.
 At the moment, Sold adminstrators can use invariant params to hardcode the 
 rows param to something reasonable, but unless they only want to allow 
 users to look at page one, the can't do much to lock down the start param 
 expect inforce these rules in the client code
 we should add new params that set an upper bound on both of these, which can 
 then be specified as default/invarient params in solrconfig.xml

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Re: poly fields in fieldcache

2010-02-18 Thread Grant Ingersoll

For sorting or just in general?

Sorting currently is not supported for PointType, etc. (although it probably 
could be if someone wanted to define it, although I'm not a 100% sure what it 
would mean in all cases).

In reality, Poly Fields are just semantic sugar around other fields, at least 
that is how the current ones are implemented.  So, I suppose they could be made 
to work w/ the field cache, if they don't already.  I'd have to look 
specifically, but they do support loading via a ValueSource.

Is there something specific you are looking to do?

-Grant

On Feb 17, 2010, at 6:12 PM, patrick o'leary wrote:

 Quick question:
 Can poly fields be accessed through fieldCache ?

Re: poly fields in fieldcache

2010-02-18 Thread patrick o'leary

Cool, we want to examine certain fields of a docset which currently is a
multivalued field, obviously only the first value gets loaded into field
cache.

But if a poly field that can be loaded into FC, then that will work, we can
extend FC to return an Field[] and make that cache aware.


Sorting on multivalued is definitely a subjective matter that a function
query would rock in, having an FC or VS that supports is would make that
much easier, like say events where an event can have multiple dates,
sort_date_compared(performance_dates, NOW)

Or even distances from a poly, polyDistance(convexHull, point) or
polyDistance(center, point) etc..


On Thu, Feb 18, 2010 at 10:40 AM, Grant Ingersoll gsing...@apache.orgwrote:

 For sorting or just in general?

 Sorting currently is not supported for PointType, etc. (although it
 probably could be if someone wanted to define it, although I'm not a 100%
 sure what it would mean in all cases).

 In reality, Poly Fields are just semantic sugar around other fields, at
 least that is how the current ones are implemented.  So, I suppose they
 could be made to work w/ the field cache, if they don't already.  I'd have
 to look specifically, but they do support loading via a ValueSource.

 Is there something specific you are looking to do?

 -Grant

 On Feb 17, 2010, at 6:12 PM, patrick o'leary wrote:

  Quick question:
  Can poly fields be accessed through fieldCache ?

[jira] Commented: (SOLR-1695) Missleading error message when adding docs with missing/multiple value(s) for uniqueKey field

[
https://issues.apache.org/jira/browse/SOLR-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835334#action_12835334
]

Hoss Man commented on SOLR-1695:

Doh!

Note to self: don't just run the tests, remember to look at the results as well.

The DocumentBuilderTest failures make sense: they use a schema with uniqueKey
defined, but add docs w/o that field to test other behaviors of toDocument.
They passed prior to this change because the only tested to toDocument method
in isolation, andthe test for a missing uniqueKey was missing from that method.
I think it's safe to consider these tests broken as written, since toDocument
does do schema validation -- it just wasn't doing the uniqueKey validation
before. So i'll modify those tests to include a value for the uniqueKey field

the ConvertedLegacyTest failure confuses me though ... it also adds docs w/o a
uniqueKey field even though the schema requires one, but they do full adds so
it's not obvious from the surface why it was ever passing before ... i want to
think about that a little more before just fixing' the test -- it may be
masking another bug.

Missleading error message when adding docs with missing/multiple value(s)
for uniqueKey field
--

Key: SOLR-1695
URL: https://issues.apache.org/jira/browse/SOLR-1695
Project: Solr
Issue Type: Improvement
Reporter: Hoss Man
Assignee: Hoss Man
Priority: Minor
Fix For: 1.5

Sometimes users don't seem to notice/understand the uniqueKey/ declaration
in the example schema, and the error message they get if their documents
don't include that field is confusing...
{code}
org.apache.solr.common.SolrException: Document [null] missing required field:
id
{code}
...because they get an almost identical error even if they remove
{{required=true}} from {{field name=id /}} in their schema.xml file.
We should improve the error message so it's clear when a Document is missing
the uniqueKeyField (not just a required field) so they know the
terminology to look for in diagnosing the problem.
http://old.nabble.com/solr-1.4-csv-import-Document-missing-required-field%3A-id-to26990048.html#a26990779

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (SOLR-1779) DistanceUtils.parsePoint doesn't handle dimensions 2 properly

2010-02-18 Thread Grant Ingersoll (JIRA)

DistanceUtils.parsePoint doesn't handle dimensions  2 properly
---

 Key: SOLR-1779
 URL: https://issues.apache.org/jira/browse/SOLR-1779
 Project: Solr
  Issue Type: Bug
Reporter: Grant Ingersoll
Priority: Trivial


As the title says.  Here's the fix:

{code}
Index: DistanceUtils.java
===
--- DistanceUtils.java  (revision 911529)
+++ DistanceUtils.java  (working copy)
@@ -140,7 +140,7 @@
 while (start  end  externalVal.charAt(start) == ' ') start++;
 while (end  start  externalVal.charAt(end - 1) == ' ') end--;
 out[i] = externalVal.substring(start, end);
-start = idx + 1;
+start = end + 1;
 end = externalVal.indexOf(',', start);
 if (end == -1) {
   end = externalVal.length();
@@ -180,7 +180,7 @@
 while (start  end  externalVal.charAt(start) == ' ') start++;
 while (end  start  externalVal.charAt(end - 1) == ' ') end--;
 out[i] = Double.parseDouble(externalVal.substring(start, end));
-start = idx + 1;
+start = end + 1;
 end = externalVal.indexOf(',', start);
 if (end == -1) {
   end = externalVal.length();
{code}

Will commit now, but am going to check in a test as part of SOLR-1568, which I 
have open w/ lots of other changes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1695) Missleading error message when adding docs with missing/multiple value(s) for uniqueKey field


[ 
https://issues.apache.org/jira/browse/SOLR-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835343#action_12835343
 ] 

Yonik Seeley commented on SOLR-1695:


bq. the ConvertedLegacyTest failure confuses me though

schema.xml does not require the id field, and the failing add explicitly says 
allowDups=false (legacy speak for overwrite=false)


  Missleading error message when adding docs with missing/multiple value(s) 
 for uniqueKey field
 --

 Key: SOLR-1695
 URL: https://issues.apache.org/jira/browse/SOLR-1695
 Project: Solr
  Issue Type: Improvement
Reporter: Hoss Man
Assignee: Hoss Man
Priority: Minor
 Fix For: 1.5


 Sometimes users don't seem to notice/understand the uniqueKey/ declaration 
 in the example schema, and the error message they get if their documents 
 don't include that field is confusing...
 {code}
 org.apache.solr.common.SolrException: Document [null] missing required field: 
 id
 {code}
 ...because they get an almost identical error even if they remove 
 {{required=true}} from {{field name=id /}} in their schema.xml file.
 We should improve the error message so it's clear when a Document is missing 
 the uniqueKeyField (not just a required field) so they know the 
 terminology to look for in diagnosing the problem.
 http://old.nabble.com/solr-1.4-csv-import-Document-missing-required-field%3A-id-to26990048.html#a26990779

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter


[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835347#action_12835347
 ] 

Yonik Seeley commented on SOLR-1568:


I'm not a fan of optional units... I think we should just standardize on 
something (meters perhaps) and stick with it.
This is a programmatic API, not a user API, and it's not a hardship for a 
programmer to convert to whatever units are appropriate.



 Implement Spatial Filter
 

 Key: SOLR-1568
 URL: https://issues.apache.org/jira/browse/SOLR-1568
 Project: Solr
  Issue Type: New Feature
Reporter: Grant Ingersoll
Assignee: Grant Ingersoll
Priority: Minor
 Fix For: 1.5

 Attachments: CartesianTierQParserPlugin.java, SOLR-1568.patch


 Given an index with spatial information (either as a geohash, 
 SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
 able to pass in a filter query that takes in the field name, lat, lon and 
 distance and produces an appropriate Filter (i.e. one that is aware of the 
 underlying field type for use by Solr. 
 The interface _could_ look like:
 {code}
 fq={!sfilt dist=20}location:49.32,-79.0
 {code}
 or it could be:
 {code}
 fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
 {code}
 or:
 {code}
 fq={!sfilt p=49.32,-79.0 f=location dist=20}
 {code}
 or:
 {code}
 fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
 {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1695) Missleading error message when adding docs with missing/multiple value(s) for uniqueKey field


[ 
https://issues.apache.org/jira/browse/SOLR-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835364#action_12835364
 ] 

Hoss Man commented on SOLR-1695:


Hmmm ok so the reason the legacy test passed prior to this change is that 
DirectUpdateHandler2 (and DirectUpdateHandler from what i can tell) don't 
bother checking for a uniqueKey (or for multiple uniqueKeys) if 
allowDups=true (which it is in the line of ConvertedLEgacyTest that's 
failing).

So the question becomes: Is it a bug that DUH(2) allow docs w/o a uniqueKey 
field just because allowDups=true?

If it's not a bug, then this entire patch should probably be rolled back -- but 
personally It feels like it really is a bug: if a schema declares a uniqueKey 
field, then just because a particular add command says allowDups=true doesn't 
mean that docs w/o an id (or with multiple ids) should be allowed in to the 
index -- those docs will need meaningful ids if/when a later commit does want 
to override them (consider the case of doing an initial build w/ allowDups=true 
for speed, and then incremental updates w/ allowDups=false ... the index needs 
to be internally consistent.

Actually: I'm just going to roll this entire patch back either way -- we can 
improve the error messages generated by DirectUpdateHandler2 and eliminate the 
redundant uniqueKey check in DocumentBuilder.toDocument.  As a separate issue 
we can consider whether DUH2 is buggy.

  Missleading error message when adding docs with missing/multiple value(s) 
 for uniqueKey field
 --

 Key: SOLR-1695
 URL: https://issues.apache.org/jira/browse/SOLR-1695
 Project: Solr
  Issue Type: Improvement
Reporter: Hoss Man
Assignee: Hoss Man
Priority: Minor
 Fix For: 1.5


 Sometimes users don't seem to notice/understand the uniqueKey/ declaration 
 in the example schema, and the error message they get if their documents 
 don't include that field is confusing...
 {code}
 org.apache.solr.common.SolrException: Document [null] missing required field: 
 id
 {code}
 ...because they get an almost identical error even if they remove 
 {{required=true}} from {{field name=id /}} in their schema.xml file.
 We should improve the error message so it's clear when a Document is missing 
 the uniqueKeyField (not just a required field) so they know the 
 terminology to look for in diagnosing the problem.
 http://old.nabble.com/solr-1.4-csv-import-Document-missing-required-field%3A-id-to26990048.html#a26990779

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Re: svn commit: r911534 - /lucene/solr/trunk/src/java/org/apache/solr/search/function/distance/DistanceUtils.java

2010-02-18 Thread Chris Hostetter


: + idex = end;

typo? ... it's causing a compile error.


-Hoss

[jira] Commented: (SOLR-1695) Missleading error message when adding docs with missing/multiple value(s) for uniqueKey field


[ 
https://issues.apache.org/jira/browse/SOLR-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835373#action_12835373
 ] 

Hoss Man commented on SOLR-1695:


bq. schema.xml does not require the id field, and the failing add explicitly 
says allowDups=false (legacy speak for overwrite=false)

...it doesn't require id but it does declare id as the uniqueKey field ... 
even if it's allowing dups shouldn't it ensure that the docs has 1 and only one 
value for the uniqueKey field?

  Missleading error message when adding docs with missing/multiple value(s) 
 for uniqueKey field
 --

 Key: SOLR-1695
 URL: https://issues.apache.org/jira/browse/SOLR-1695
 Project: Solr
  Issue Type: Improvement
Reporter: Hoss Man
Assignee: Hoss Man
Priority: Minor
 Fix For: 1.5


 Sometimes users don't seem to notice/understand the uniqueKey/ declaration 
 in the example schema, and the error message they get if their documents 
 don't include that field is confusing...
 {code}
 org.apache.solr.common.SolrException: Document [null] missing required field: 
 id
 {code}
 ...because they get an almost identical error even if they remove 
 {{required=true}} from {{field name=id /}} in their schema.xml file.
 We should improve the error message so it's clear when a Document is missing 
 the uniqueKeyField (not just a required field) so they know the 
 terminology to look for in diagnosing the problem.
 http://old.nabble.com/solr-1.4-csv-import-Document-missing-required-field%3A-id-to26990048.html#a26990779

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1695) Missleading error message when adding docs with missing/multiple value(s) for uniqueKey field

[
https://issues.apache.org/jira/browse/SOLR-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835381#action_12835381
]

Yonik Seeley commented on SOLR-1695:

bq. ...it doesn't require id but it does declare id as the uniqueKey field
... even if it's allowing dups shouldn't it ensure that the docs has 1 and only
one value for the uniqueKey field?

That depends... it makes sense for normal documents, but I've seen people
that add a few auxillary documents to their index that used different fields,
including the id field. But it's not like you gain greater power by allowing
that - those usecases could be covered by forcing the user to come up with a
uniqueKey value too... it would just be a little more work.

Missleading error message when adding docs with missing/multiple value(s)
for uniqueKey field
--

Key: SOLR-1695
URL: https://issues.apache.org/jira/browse/SOLR-1695
Project: Solr
Issue Type: Improvement
Reporter: Hoss Man
Assignee: Hoss Man
Priority: Minor
Fix For: 1.5

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Re: svn commit: r911534 - /lucene/solr/trunk/src/java/org/apache/solr/search/function/distance/DistanceUtils.java

2010-02-18 Thread Grant Ingersoll

Haste makes waste.  I'll check in a test for all this stuff too in the near 
future, but that test is too intertwined with other changes at the moment.

On Feb 18, 2010, at 2:56 PM, Chris Hostetter wrote:

 
 : +   idex = end;
 
 typo? ... it's causing a compile error.
 
 
 -Hoss

[jira] Resolved: (SOLR-1695) Missleading error message when adding docs with missing/multiple value(s) for uniqueKey field


 [ 
https://issues.apache.org/jira/browse/SOLR-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man resolved SOLR-1695.


Resolution: Fixed

Committed revision 911595.

rolledback the changes to DocumentBuilder and improved the existing error 
messages in UpdateHandler instead.

  Missleading error message when adding docs with missing/multiple value(s) 
 for uniqueKey field
 --

 Key: SOLR-1695
 URL: https://issues.apache.org/jira/browse/SOLR-1695
 Project: Solr
  Issue Type: Improvement
Reporter: Hoss Man
Assignee: Hoss Man
Priority: Minor
 Fix For: 1.5


 Sometimes users don't seem to notice/understand the uniqueKey/ declaration 
 in the example schema, and the error message they get if their documents 
 don't include that field is confusing...
 {code}
 org.apache.solr.common.SolrException: Document [null] missing required field: 
 id
 {code}
 ...because they get an almost identical error even if they remove 
 {{required=true}} from {{field name=id /}} in their schema.xml file.
 We should improve the error message so it's clear when a Document is missing 
 the uniqueKeyField (not just a required field) so they know the 
 terminology to look for in diagnosing the problem.
 http://old.nabble.com/solr-1.4-csv-import-Document-missing-required-field%3A-id-to26990048.html#a26990779

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (SOLR-1780) existence of exactly one value for uniqueKey field is not checked when overwrite=false or allowDups=true

existence of exactly one value for uniqueKey field is not checked when 
overwrite=false or allowDups=true


 Key: SOLR-1780
 URL: https://issues.apache.org/jira/browse/SOLR-1780
 Project: Solr
  Issue Type: Bug
Reporter: Hoss Man


As noted in SOLR-1695, DirectUpdateHandler(2) when a document is added, the 
uniqueKey field is only asserted to contain exactly one value if 
overwrite=true.  If overwrite=false (or allowDups=true) then the uniqueKey 
field is not checked at all.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1777) fields with sortMissingLast don't sort correctly


[ 
https://issues.apache.org/jira/browse/SOLR-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835424#action_12835424
 ] 

Hoss Man commented on SOLR-1777:


Yonik: just to be verify: was this bug was introduced in Solr 1.4? ... 
presumably because of the changes to per segment collecting?

(that's the way the Affects Version/s is marked, but i want to sanity check 
in case it was actually a more fundamental problem affecting earlier versions 
of Solr as well).

 fields with sortMissingLast don't sort correctly
 

 Key: SOLR-1777
 URL: https://issues.apache.org/jira/browse/SOLR-1777
 Project: Solr
  Issue Type: Bug
  Components: search
Affects Versions: 1.4
Reporter: Yonik Seeley
Assignee: Yonik Seeley
Priority: Critical
 Fix For: 1.5

 Attachments: SOLR-1777.patch, SOLR-1777.patch


 field types with the sortMissingLast=true attribute can have results sorted 
 incorrectly.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1777) fields with sortMissingLast don't sort correctly


[ 
https://issues.apache.org/jira/browse/SOLR-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835451#action_12835451
 ] 

Yonik Seeley commented on SOLR-1777:


bq. Yonik: just to be verify: was this bug was introduced in Solr 1.4?... 
presumably because of the changes to per segment collecting? 

Yep.  The per-segment collecting and the FieldComparator changes caused us to 
rewrite all of our custom comparators... and this one had bugs.


 fields with sortMissingLast don't sort correctly
 

 Key: SOLR-1777
 URL: https://issues.apache.org/jira/browse/SOLR-1777
 Project: Solr
  Issue Type: Bug
  Components: search
Affects Versions: 1.4
Reporter: Yonik Seeley
Assignee: Yonik Seeley
Priority: Critical
 Fix For: 1.5

 Attachments: SOLR-1777.patch, SOLR-1777.patch


 field types with the sortMissingLast=true attribute can have results sorted 
 incorrectly.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-1724) Real Basic Core Management with Zookeeper


 [ 
https://issues.apache.org/jira/browse/SOLR-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Rutherglen updated SOLR-1724:
---

Attachment: SOLR-1724.patch

* No-commit

* NodeCoresManagerTest.testInstallCores works

* There's HDFS test cases using MiniDFSCluster



 Real Basic Core Management with Zookeeper
 -

 Key: SOLR-1724
 URL: https://issues.apache.org/jira/browse/SOLR-1724
 Project: Solr
  Issue Type: New Feature
  Components: multicore
Affects Versions: 1.4
Reporter: Jason Rutherglen
 Fix For: 1.5

 Attachments: commons-lang-2.4.jar, gson-1.4.jar, 
 hadoop-0.20.2-dev-core.jar, hadoop-0.20.2-dev-test.jar, SOLR-1724.patch, 
 SOLR-1724.patch, SOLR-1724.patch, SOLR-1724.patch


 Though we're implementing cloud, I need something real soon I can
 play with and deploy. So this'll be a patch that only deploys
 new cores, and that's about it. The arch is real simple:
 On Zookeeper there'll be a directory that contains files that
 represent the state of the cores of a given set of servers which
 will look like the following:
 /production/cores-1.txt
 /production/cores-2.txt
 /production/core-host-1-actual.txt (ephemeral node per host)
 Where each core-N.txt file contains:
 hostname,corename,instanceDir,coredownloadpath
 coredownloadpath is a URL such as file://, http://, hftp://, hdfs://, ftp://, 
 etc
 and
 core-host-actual.txt contains:
 hostname,corename,instanceDir,size
 Everytime a new core-N.txt file is added, the listening host
 finds it's entry in the list and begins the process of trying to
 match the entries. Upon completion, it updates it's
 /core-host-1-actual.txt file to it's completed state or logs an error.
 When all host actual files are written (without errors), then a
 new core-1-actual.txt file is written which can be picked up by
 another process that can create a new core proxy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1724) Real Basic Core Management with Zookeeper


[ 
https://issues.apache.org/jira/browse/SOLR-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835490#action_12835490
 ] 

Jason Rutherglen commented on SOLR-1724:


I need to figure out how integrate this with the Solr Cloud distributed search 
stuff... Hmm... Maybe I'll start with the Solr Cloud test cases?

 Real Basic Core Management with Zookeeper
 -

 Key: SOLR-1724
 URL: https://issues.apache.org/jira/browse/SOLR-1724
 Project: Solr
  Issue Type: New Feature
  Components: multicore
Affects Versions: 1.4
Reporter: Jason Rutherglen
 Fix For: 1.5

 Attachments: commons-lang-2.4.jar, gson-1.4.jar, 
 hadoop-0.20.2-dev-core.jar, hadoop-0.20.2-dev-test.jar, SOLR-1724.patch, 
 SOLR-1724.patch, SOLR-1724.patch, SOLR-1724.patch


 Though we're implementing cloud, I need something real soon I can
 play with and deploy. So this'll be a patch that only deploys
 new cores, and that's about it. The arch is real simple:
 On Zookeeper there'll be a directory that contains files that
 represent the state of the cores of a given set of servers which
 will look like the following:
 /production/cores-1.txt
 /production/cores-2.txt
 /production/core-host-1-actual.txt (ephemeral node per host)
 Where each core-N.txt file contains:
 hostname,corename,instanceDir,coredownloadpath
 coredownloadpath is a URL such as file://, http://, hftp://, hdfs://, ftp://, 
 etc
 and
 core-host-actual.txt contains:
 hostname,corename,instanceDir,size
 Everytime a new core-N.txt file is added, the listening host
 finds it's entry in the list and begins the process of trying to
 match the entries. Upon completion, it updates it's
 /core-host-1-actual.txt file to it's completed state or logs an error.
 When all host actual files are written (without errors), then a
 new core-1-actual.txt file is written which can be picked up by
 another process that can create a new core proxy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-1724) Real Basic Core Management with Zookeeper


 [ 
https://issues.apache.org/jira/browse/SOLR-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Rutherglen updated SOLR-1724:
---

Attachment: SOLR-1724.patch

Updated to HEAD

 Real Basic Core Management with Zookeeper
 -

 Key: SOLR-1724
 URL: https://issues.apache.org/jira/browse/SOLR-1724
 Project: Solr
  Issue Type: New Feature
  Components: multicore
Affects Versions: 1.4
Reporter: Jason Rutherglen
 Fix For: 1.5

 Attachments: commons-lang-2.4.jar, gson-1.4.jar, 
 hadoop-0.20.2-dev-core.jar, hadoop-0.20.2-dev-test.jar, SOLR-1724.patch, 
 SOLR-1724.patch, SOLR-1724.patch, SOLR-1724.patch, SOLR-1724.patch


 Though we're implementing cloud, I need something real soon I can
 play with and deploy. So this'll be a patch that only deploys
 new cores, and that's about it. The arch is real simple:
 On Zookeeper there'll be a directory that contains files that
 represent the state of the cores of a given set of servers which
 will look like the following:
 /production/cores-1.txt
 /production/cores-2.txt
 /production/core-host-1-actual.txt (ephemeral node per host)
 Where each core-N.txt file contains:
 hostname,corename,instanceDir,coredownloadpath
 coredownloadpath is a URL such as file://, http://, hftp://, hdfs://, ftp://, 
 etc
 and
 core-host-actual.txt contains:
 hostname,corename,instanceDir,size
 Everytime a new core-N.txt file is added, the listening host
 finds it's entry in the list and begins the process of trying to
 match the entries. Upon completion, it updates it's
 /core-host-1-actual.txt file to it's completed state or logs an error.
 When all host actual files are written (without errors), then a
 new core-1-actual.txt file is written which can be picked up by
 another process that can create a new core proxy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1724) Real Basic Core Management with Zookeeper


[ 
https://issues.apache.org/jira/browse/SOLR-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12835513#action_12835513
 ] 

Jason Rutherglen commented on SOLR-1724:


I need to add the deletion policy before I can test this in a real environment, 
otherwise bunches of useless files will pile up in ZK.

 Real Basic Core Management with Zookeeper
 -

 Key: SOLR-1724
 URL: https://issues.apache.org/jira/browse/SOLR-1724
 Project: Solr
  Issue Type: New Feature
  Components: multicore
Affects Versions: 1.4
Reporter: Jason Rutherglen
 Fix For: 1.5

 Attachments: commons-lang-2.4.jar, gson-1.4.jar, 
 hadoop-0.20.2-dev-core.jar, hadoop-0.20.2-dev-test.jar, SOLR-1724.patch, 
 SOLR-1724.patch, SOLR-1724.patch, SOLR-1724.patch, SOLR-1724.patch


 Though we're implementing cloud, I need something real soon I can
 play with and deploy. So this'll be a patch that only deploys
 new cores, and that's about it. The arch is real simple:
 On Zookeeper there'll be a directory that contains files that
 represent the state of the cores of a given set of servers which
 will look like the following:
 /production/cores-1.txt
 /production/cores-2.txt
 /production/core-host-1-actual.txt (ephemeral node per host)
 Where each core-N.txt file contains:
 hostname,corename,instanceDir,coredownloadpath
 coredownloadpath is a URL such as file://, http://, hftp://, hdfs://, ftp://, 
 etc
 and
 core-host-actual.txt contains:
 hostname,corename,instanceDir,size
 Everytime a new core-N.txt file is added, the listening host
 finds it's entry in the list and begins the process of trying to
 match the entries. Upon completion, it updates it's
 /core-host-1-actual.txt file to it's completed state or logs an error.
 When all host actual files are written (without errors), then a
 new core-1-actual.txt file is written which can be picked up by
 another process that can create a new core proxy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-1724) Real Basic Core Management with Zookeeper