from:"Charles Sanders"

Re: query keyword but no result (solr 8)

2019-05-14 Thread Charles Sanders

Derrick, 
This makes me think you do not have a default query field identified. Look in 
your solrcong.xml file for the requesthandler you are calling and see if it has 
the "df" parameter set. Should look something like: 
text 

Hope this helps! 

- Original Message -

From: "Derrick Cui"  
To: solr-user@lucene.apache.org 
Sent: Monday, May 13, 2019 10:52:13 PM 
Subject: query keyword but no result (solr 8) 

Hi, 

I am trying to setup solrcloud, I can index a few documents successfully. 
but I cannot get result if I search keyword(without field). if I use 
field:keyword, I can get result. 

any idea why I get this issue? 

Thank you 

-- 
Regards, 

Derrick Cui 
Email: derrick...@gmail.com

DocTransformer [explain] not working in Solr 5

2016-05-26 Thread Charles Sanders

Not able to get the DocTransformer [explain] to work in Solr 5. I'm sure I'm 
doing something wrong. But I'm following the example in the documentation. 
https://cwiki.apache.org/confluence/display/solr/Transforming+Result+Documents 

Other transformers ( [docid] and [shard] ) are working as expected. Anyone know 
how to make this work? I've used it in the past with Solr 4. Any information 
greatly appreciated. Query and results follow. 

Thanks, 
Charles 

http://localhost:8983/solr/access_shard1_replica1/select?q=generic 
=1=id,[docid],[shard],[explain 
style=nl]=json=true=edismax=allTitle^3 allText^2 
contents=true 


{
  "responseHeader":{
"status":0,
"QTime":13,
"params":{
  "q":"generic\n",
  "defType":"edismax",
  "indent":"true",
  "qf":"allTitle^3 allText^2 contents",
  "fl":"id,[docid],[shard],[explain style=nl]",
  "rows":"1",
  "wt":"json",
  "debugQuery":"true"}},
  "response":{"numFound":5,"start":0,"maxScore":2.890213,"docs":[
  {
"id":"99",
"[docid]":70,
"[shard]":"http://10.13.49.9:8983/solr/access_shard1_replica1/"}]
  },
  "spellcheck":{
"suggestions":[],
"collations":[]},
  "debug":{
"track":{
  "rid":"-access_shard1_replica1-1464265350378-8",
  "EXECUTE_QUERY":{
"http://10.13.49.9:8983/solr/access_shard1_replica1/":{
  "QTime":"1",
  "ElapsedTime":"5",
  "RequestPurpose":"GET_TOP_IDS",
  "NumFound":"5",
  
"Response":"{responseHeader={status=0,QTime=1,params={df=allText,distrib=false,spellcheck.dictionary=andreasAutoComplete,fl=[uri,
 
score],shards.purpose=4,spellcheck.maxCollations=5,fsv=true,shard.url=http://10.13.49.9:8983/solr/access_shard1_replica1/,rid=-access_shard1_replica1-1464265350378-8,defType=edismax,qf=allTitle^3
 allText^2 contents,wt=javabin,debug=[false, timing, 
track],qt=/select,start=0,rows=1,version=2,shards.qt=/select,q=generic\n,enableElevation=false,spellcheck=true,requestPurpose=GET_TOP_IDS,NOW=1464265350377,spellcheck.onlyMorePopular=true,isShard=true,spellcheck.count=5,debugQuery=false,spellcheck.collate=true}},response={numFound=5,start=0,maxScore=2.890213,docs=[SolrDocument{uri=http://foo/bar/Generic/99,
 
score=2.890213}]},sort_values={},spellcheck={suggestions={},collations={},originalTerms=[generic]},debug={timing={time=1.0,prepare={time=0.0,query={time=0.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},elevator={time=0.0},autoComplete={time=0.0},debug={time=0.0}},process={time=0.0,query={time=0.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},elevator={time=0.0},autoComplete={time=0.0},debug={time=0.0}"}},
  "GET_FIELDS":{
"http://10.13.49.9:8983/solr/access_shard1_replica1/":{
  "QTime":"2",
  "ElapsedTime":"4",
  "RequestPurpose":"GET_FIELDS,GET_DEBUG",
  "NumFound":"1",
  
"Response":"{responseHeader={status=0,QTime=2,params={df=allText,distrib=false,spellcheck.dictionary=andreasAutoComplete,fl=[id,[docid],[shard],[explain
 style=nl], 
uri],shards.purpose=320,spellcheck.maxCollations=5,shard.url=http://10.13.49.9:8983/solr/access_shard1_replica1/,rid=-access_shard1_replica1-1464265350378-8,defType=edismax,qf=allTitle^3
 allText^2 contents,wt=javabin,debug=[timing, 
track],qt=/select,rows=1,version=2,shards.qt=/select,q=generic\n,enableElevation=false,spellcheck=false,requestPurpose=GET_FIELDS,GET_DEBUG,NOW=1464265350377,spellcheck.onlyMorePopular=true,ids=http://foo/bar/Generic/99,isShard=true,spellcheck.count=5,debugQuery=true,spellcheck.collate=true}},response={numFound=1,start=0,docs=[SolrDocument{id=99,
 [docid]=70, 
[shard]=http://10.13.49.9:8983/solr/access_shard1_replica1/}]},debug={rawquerystring=generic\n,querystring=generic\n,parsedquery=(+DisjunctionMaxQuery((allTitle:gener^3.0
 | contents:gener | 
allText:gener^2.0)))/no_coord,parsedquery_toString=+(allTitle:gener^3.0 | 
contents:gener | 
allText:gener^2.0),explain={http://foo/bar/Generic/99=\n2.890213 = max of:\n  
2.890213 = weight(allTitle:gener^3.0 in 70) [DefaultSimilarity], result of:\n   
 2.890213 = fieldWeight in 70, product of:\n  1.0 = tf(freq=1.0), with freq 
of:\n1.0 = termFreq=1.0\n  4.624341 = idf(docFreq=1, maxDocs=75)\n  
0.625 = fieldNorm(doc=70)\n  0.33601448 = weight(contents:gener in 70) 
[DefaultSimilarity], result of:\n0.33601448 = score(doc=70,freq=1.0), 
product of:\n  0.2541428 = queryWeight, product of:\n3.5257287 = 
idf(docFreq=5, maxDocs=75)\n0.07208234 = queryNorm\n  1.3221483 = 
fieldWeight in 70, product of:\n1.0 = tf(freq=1.0), with freq of:\n 
 1.0 = termFreq=1.0\n3.5257287 = idf(docFreq=5, maxDocs=75)\n   
 0.375 =

Re: How to set multivalued false, using SolrJ

2016-04-11 Thread Charles Sanders

Hello, 
Multivalued fields are controlled by the schema. You need to define your field 
in the schema file as 'not' a multivalue field. Here are a couple of examples 
of field definitions, one multivalued, the other not. 

 
 

If you do not explicitly define your field, then solr will use default 
definitions, which are probably storing the field as multivalued. 


Charles 





- Original Message -

From: "巩学超"  
To: solr-user@lucene.apache.org 
Sent: Monday, April 11, 2016 7:58:35 AM 
Subject: How to set multivalued false, using SolrJ 

Hello， 
Can you do me a favour, I use solrJ to index, but I get all the Field is 
multivalued. How can I set my Field to not multivalued, can you tell me how to 
setting use solrJ.

Re: SyntaxError - Block Join Parent Query

2016-03-24 Thread Charles Sanders

I tried this on another machine with a clean index. I was able to get the query 
to work. Thank you. 

Couple of related questions. 
1) I was able to get this to work on a single shard machine. But I am not able 
to get this query to work on Solr with two shards (SorlCloud). Any reason why 
this does not work with SolrCloud? 
2) The query pattern you supplied does not appear in the documentation. Do you 
know of any reason why the information in the documentation does not work and 
does not mention your pattern? 
https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-BlockJoinQueryParsers
 

Thanks again. 


- Original Message -

From: "Mikhail Khludnev" <mkhlud...@griddynamics.com> 
To: "solr-user" <solr-user@lucene.apache.org> 
Sent: Thursday, March 24, 2016 11:31:11 AM 
Subject: Re: SyntaxError - Block Join Parent Query 

I suggest to add debugQuery=true and fl=*,[child ...] doc transformer. And 
come back with response. 

On Thu, Mar 24, 2016 at 3:23 PM, Charles Sanders <csand...@redhat.com> 
wrote: 

> Ah yes. Thank you. Made the correction and I do not get the SyntaxError. 
> However, it does not apply the child filter. The query should return only 
> TestParent4. But it is returning TestParent2, TestParent3 and TestParent4. 
> All of these meet the parent portion of the query (+blue). But only 
> TestParent4 should meet the child portion of the query. 
> 
> q=+blue +{!parent 
> which="documentKind:TestParent"v=$childq}=portal_product:("red hat") 
> 
> 
> { 
> "responseHeader":{ 
> "status":0, 
> "QTime":15, 
> "params":{ 
> "indent":"true", 
> "q":" blue {!parent which=\"documentKind:TestParent\"v=$childq}", 
> "childq":"portal_product:(\"red hat\")", 
> "wt":"json"}}, 
> "response":{"numFound":2733,"start":0,"maxScore":3.0138793,"docs":[ 
> { 
> "documentKind":"TestParent", 
> "uri":"https://ping/pong/testparent4;, 
> "view_uri":"https://ping/pong/testparent4;, 
> "id":"TestParent4", 
> "allTitle":"blue", 
> "sortTitle":"blue", 
> "_version_":1529622873461751808, 
> "_root_":["https://ping/pong/testparent4;], 
> "timestamp":"2016-03-23T19:40:48.211Z", 
> "language":"en"}, 
> { 
> "documentKind":"TestParent", 
> "uri":"https://ping/pong/testparent3;, 
> "view_uri":"https://ping/pong/testparent3;, 
> "id":"TestParent3", 
> "allTitle":"blue", 
> "sortTitle":"blue", 
> "_version_":1529622308758487040, 
> "_root_":["https://ping/pong/testparent3;], 
> "timestamp":"2016-03-23T19:31:49.668Z", 
> "language":"en"}, 
> { 
> "documentKind":"TestParent", 
> "uri":"https://ping/pong/testparent2;, 
> "view_uri":"https://ping/pong/testparent2;, 
> "id":"TestParent2", 
> "allTitle":"blue", 
> "sortTitle":"blue", 
> "_version_":1529622293809987584, 
> "_root_":["https://ping/pong/testparent2;], 
> "timestamp":"2016-03-23T19:31:35.408Z", 
> "language":"en"} 
> 
> - Original Message - 
> 
> From: "Mikhail Khludnev" <mkhlud...@griddynamics.com> 
> To: "solr-user" <solr-user@lucene.apache.org> 
> Sent: Wednesday, March 23, 2016 5:34:31 PM 
> Subject: Re: SyntaxError - Block Join Parent Query 
> 
> On Thu, Mar 24, 2016 at 12:16 AM, Charles Sanders <csand...@redhat.com> 
> wrote: 
> 
> > Thanks for the quick reply. But I'm not sure I understand. Did I do 
> > something wrong?? 
> > 
> Absolutely 
> 'portal_product"red hat")' 
> You omit a colon and opening bracket after a field name, don't you? 
> 
> 
> > 
> > 
> > 
> /select?q=+blue%20+{!parent%20which=%22documentKind:TestParent%22%20v=$childq}=portal_product%22red%20hat%22)
>  
> > 
> >  
> >  
> >  400  
> >  2  
> >  
> >  
> > blue {!parent which="documentKind:TestParent" v=$childq} 
> >  
> >  portal_product"red hat")  
> >  
> >  
> >  
> >  
> > org.apache.solr.search.SyntaxError: Cannot parse 'portal_product"red 
> > hat")': Encountered "

Re: SyntaxError - Block Join Parent Query

2016-03-24 Thread Charles Sanders

Ah yes. Thank you. Made the correction and I do not get the SyntaxError. 
However, it does not apply the child filter. The query should return only 
TestParent4. But it is returning TestParent2, TestParent3 and TestParent4. All 
of these meet the parent portion of the query (+blue). But only TestParent4 
should meet the child portion of the query. 

q=+blue +{!parent 
which="documentKind:TestParent"v=$childq}=portal_product:("red hat") 


{
  "responseHeader":{
"status":0,
"QTime":15,
"params":{
  "indent":"true",
  "q":" blue  {!parent which=\"documentKind:TestParent\"v=$childq}",
  "childq":"portal_product:(\"red hat\")",
  "wt":"json"}},
  "response":{"numFound":2733,"start":0,"maxScore":3.0138793,"docs":[
  {
"documentKind":"TestParent",
"uri":"https://ping/pong/testparent4;,
"view_uri":"https://ping/pong/testparent4;,
"id":"TestParent4",
"allTitle":"blue",
"sortTitle":"blue",
"_version_":1529622873461751808,
"_root_":["https://ping/pong/testparent4;],
"timestamp":"2016-03-23T19:40:48.211Z",
"language":"en"},
  {
"documentKind":"TestParent",
"uri":"https://ping/pong/testparent3;,
"view_uri":"https://ping/pong/testparent3;,
"id":"TestParent3",
"allTitle":"blue",
"sortTitle":"blue",
"_version_":1529622308758487040,
"_root_":["https://ping/pong/testparent3;],
"timestamp":"2016-03-23T19:31:49.668Z",
"language":"en"},
  {
"documentKind":"TestParent",
"uri":"https://ping/pong/testparent2;,
"view_uri":"https://ping/pong/testparent2;,
"id":"TestParent2",
"allTitle":"blue",
"sortTitle":"blue",
"_version_":1529622293809987584,
"_root_":["https://ping/pong/testparent2;],
"timestamp":"2016-03-23T19:31:35.408Z",
"language":"en"} 

- Original Message -

From: "Mikhail Khludnev" <mkhlud...@griddynamics.com> 
To: "solr-user" <solr-user@lucene.apache.org> 
Sent: Wednesday, March 23, 2016 5:34:31 PM 
Subject: Re: SyntaxError - Block Join Parent Query 

On Thu, Mar 24, 2016 at 12:16 AM, Charles Sanders <csand...@redhat.com> 
wrote: 

> Thanks for the quick reply. But I'm not sure I understand. Did I do 
> something wrong?? 
> 
Absolutely 
'portal_product"red hat")' 
You omit a colon and opening bracket after a field name, don't you? 


> 
> 
> /select?q=+blue%20+{!parent%20which=%22documentKind:TestParent%22%20v=$childq}=portal_product%22red%20hat%22)
>  
> 
>  
>  
>  400  
>  2  
>  
>  
> blue {!parent which="documentKind:TestParent" v=$childq} 
>  
>  portal_product"red hat")  
>  
>  
>  
>  
> org.apache.solr.search.SyntaxError: Cannot parse 'portal_product"red 
> hat")': Encountered " ")" ") "" at line 1, column 23. Was expecting one of: 
>   ...  ...  ... "+" ... "-" ...  ... "(" ... 
> "*" ... "^" ...  ...  ...  ...  ... 
>  ...  ... "[" ... "{" ...  ...  ... 
>  
>  400  
>  
>  
> 
> - Original Message - 
> 
> From: "Mikhail Khludnev" <mkhlud...@griddynamics.com> 
> To: "solr-user" <solr-user@lucene.apache.org> 
> Sent: Wednesday, March 23, 2016 5:02:29 PM 
> Subject: Re: SyntaxError - Block Join Parent Query 
> 
> On Wed, Mar 23, 2016 at 11:09 PM, Charles Sanders <csand...@redhat.com> 
> wrote: 
> 
> > I'm getting a SyntaxError which I do not understand when I execute a 
> block 
> > join parent query. I'm running Solr5.2.1, with 2 shards. The problem 
> > appears to be in that portion of the query that filters the child 
> document. 
> > Any insight as to where I made the error is greatly appreciated. 
> > 
> > This query produces an error: 
> > q=+blue +{!parent which="documentKind:TestParent"}portal_product:("red 
> > hat") 
> > -- should return TestParent4 
> >

Re: SyntaxError - Block Join Parent Query

2016-03-23 Thread Charles Sanders

Thanks for the quick reply. But I'm not sure I understand. Did I do something 
wrong?? 

/select?q=+blue%20+{!parent%20which=%22documentKind:TestParent%22%20v=$childq}=portal_product%22red%20hat%22)
 

 
 
 400  
 2  
 
 
blue {!parent which="documentKind:TestParent" v=$childq} 
 
 portal_product"red hat")  
 
 
 
 
org.apache.solr.search.SyntaxError: Cannot parse 'portal_product"red hat")': 
Encountered " ")" ") "" at line 1, column 23. Was expecting one of:   
...  ...  ... "+" ... "-" ...  ... "(" ... "*" ... "^" ... 
 ...  ...  ...  ...  ... 
 ... "[" ... "{" ...  ...  ... 
 
 400  
 
 

- Original Message -

From: "Mikhail Khludnev" <mkhlud...@griddynamics.com> 
To: "solr-user" <solr-user@lucene.apache.org> 
Sent: Wednesday, March 23, 2016 5:02:29 PM 
Subject: Re: SyntaxError - Block Join Parent Query 

On Wed, Mar 23, 2016 at 11:09 PM, Charles Sanders <csand...@redhat.com> 
wrote: 

> I'm getting a SyntaxError which I do not understand when I execute a block 
> join parent query. I'm running Solr5.2.1, with 2 shards. The problem 
> appears to be in that portion of the query that filters the child document. 
> Any insight as to where I made the error is greatly appreciated. 
> 
> This query produces an error: 
> q=+blue +{!parent which="documentKind:TestParent"}portal_product:("red 
> hat") 
> -- should return TestParent4 
> 
q=+blue +{!parent which="documentKind:TestParent" 
v=$childq}=portal_product:("red hat") 


> However, this query works: 
> q=+blue +{!parent which="documentKind:TestParent"}portal_product:rhel 
> -- should return TestParent2 
> 
> Sample data and schema information below: 
> { 
> "documentKind": "TestParent", 
> "uri": "https://ping/pong/testparent1;, 
> "view_uri": "https://ping/pong/testparent1;, 
> "id": "TestParent1", 
> "allTitle": "gold", 
> "allText": "gold", 
> "contents": "gold", 
> "_childDocuments_": [ 
> { 
> "documentKind": "TestChild", 
> "uri": "testchild1", 
> "id": "testchild1", 
> "portal_product_version": "6", 
> "portal_product": "rhel" 
> } 
> ] 
> } 
> 
> { 
> "documentKind": "TestParent", 
> "uri": "https://ping/pong/testparent2;, 
> "view_uri": "https://ping/pong/testparent2;, 
> "id": "TestParent2", 
> "allTitle": "blue", 
> "allText": "blue", 
> "contents": "blue", 
> "_childDocuments_": [ 
> { 
> "documentKind": "TestChild", 
> "uri": "testchild2", 
> "id": "testchild2", 
> "portal_product_version": "6", 
> "portal_product": "rhel" 
> } 
> ] 
> } 
> 
> { 
> "documentKind": "TestParent", 
> "uri": "https://ping/pong/testparent3;, 
> "view_uri": "https://ping/pong/testparent3;, 
> "id": "TestParent3", 
> "allTitle": "blue", 
> "allText": "blue", 
> "contents": "blue", 
> "_childDocuments_": [ 
> { 
> "documentKind": "TestChild", 
> "uri": "testchild3", 
> "id": "testchild3", 
> "portal_product_version": "3", 
> "portal_product": "rhev" 
> } 
> ] 
> } 
> 
> { 
> "documentKind": "TestParent", 
> "uri": "https://ping/pong/testparent4;, 
> "view_uri": "https://ping/pong/testparent4;, 
> "id": "TestParent4", 
> "allTitle": "blue", 
> "allText": "blue", 
> "contents": "blue", 
> "_childDocuments_": [ 
> { 
> "documentKind": "TestChild", 
> "uri": "testchild4", 
> "id": "testchild4", 
> "portal_product_version": "3", 
> "portal_product": "red hat" 
> } 
> ] 
> } 
> 
>  required="true" /> 
>  required="true"/> 
>  
>  
>  
>  multiValued="true" termVectors="true" /> 
>  multiValued="true" termVectors="true" /> 
>  stored="true"/> 
>  
> 



-- 
Sincerely yours 
Mikhail Khludnev 
Principal Engineer, 
Grid Dynamics 

<http://www.griddynamics.com> 
<mkhlud...@griddynamics.com>

SyntaxError - Block Join Parent Query

2016-03-23 Thread Charles Sanders

I'm getting a SyntaxError which I do not understand when I execute a block join 
parent query. I'm running Solr5.2.1, with 2 shards. The problem appears to be 
in that portion of the query that filters the child document. Any insight as to 
where I made the error is greatly appreciated. 

This query produces an error: 
q=+blue +{!parent which="documentKind:TestParent"}portal_product:("red hat") 
-- should return TestParent4 

However, this query works: 
q=+blue +{!parent which="documentKind:TestParent"}portal_product:rhel 
-- should return TestParent2 

Sample data and schema information below: 
{ 
"documentKind": "TestParent", 
"uri": "https://ping/pong/testparent1;, 
"view_uri": "https://ping/pong/testparent1;, 
"id": "TestParent1", 
"allTitle": "gold", 
"allText": "gold", 
"contents": "gold", 
"_childDocuments_": [ 
{ 
"documentKind": "TestChild", 
"uri": "testchild1", 
"id": "testchild1", 
"portal_product_version": "6", 
"portal_product": "rhel" 
} 
] 
} 

{ 
"documentKind": "TestParent", 
"uri": "https://ping/pong/testparent2;, 
"view_uri": "https://ping/pong/testparent2;, 
"id": "TestParent2", 
"allTitle": "blue", 
"allText": "blue", 
"contents": "blue", 
"_childDocuments_": [ 
{ 
"documentKind": "TestChild", 
"uri": "testchild2", 
"id": "testchild2", 
"portal_product_version": "6", 
"portal_product": "rhel" 
} 
] 
} 

{ 
"documentKind": "TestParent", 
"uri": "https://ping/pong/testparent3;, 
"view_uri": "https://ping/pong/testparent3;, 
"id": "TestParent3", 
"allTitle": "blue", 
"allText": "blue", 
"contents": "blue", 
"_childDocuments_": [ 
{ 
"documentKind": "TestChild", 
"uri": "testchild3", 
"id": "testchild3", 
"portal_product_version": "3", 
"portal_product": "rhev" 
} 
] 
} 

{ 
"documentKind": "TestParent", 
"uri": "https://ping/pong/testparent4;, 
"view_uri": "https://ping/pong/testparent4;, 
"id": "TestParent4", 
"allTitle": "blue", 
"allText": "blue", 
"contents": "blue", 
"_childDocuments_": [ 
{ 
"documentKind": "TestChild", 
"uri": "testchild4", 
"id": "testchild4", 
"portal_product_version": "3", 
"portal_product": "red hat" 
} 
] 
}

Re: Problem with solr.LengthFilterFactory

2015-05-18 Thread Charles Sanders

Jack, 
Thanks for the information. If I understand this correctly, the White space 
tokenizer will break a single token of size 300 into two tokens, one of size 
256 and the other of size 44. If this is true, then for the single test 
document I have used, in the index in the portal_package field, I should see 
two tokens rather than one large single token. 

If my understanding is correct, then why in my production system, where we 
occasionally get a single very large token, do I see this error? 
Caused by: java.lang.IllegalArgumentException: Document contains at least one 
immense term in field=portal_package (whose UTF8 encoding is longer than the 
max length 32766) 

The existence of this error would lead me to conclude that a very large single 
token is making its way through the white space tokenizer and filters to the 
index where it is rejected. 

I'm afraid my understanding is not complete. Can you fill in the gaps? 

Thanks, 
Charles 


- Original Message -

From: Jack Krupansky jack.krupan...@gmail.com 
To: solr-user@lucene.apache.org 
Sent: Friday, May 15, 2015 4:31:22 PM 
Subject: Re: Problem with solr.LengthFilterFactory 

Sorry that my brain has turned to mush... the issue you are hitting is due 
to a known, undocumented limit in the whitespace tokenizer: 

https://issues.apache.org/jira/browse/LUCENE-5785 
White space tokenizer has undocumented limit of 256 characters per token 

If you look at the parsed query you will see that two query terms were 
generated. This is because the whitespace tokenizer will simply split long 
tokens every 256 characters. So, your filter will never see a long term. 

There is a note on the Jira (evidently by me!) that you can use the pattern 
tokenizer as a workaround. But... if your term is a string anyway, you 
could just use the keyword tokenizer. 


-- Jack Krupansky 

On Fri, May 15, 2015 at 4:06 PM, Charles Sanders csand...@redhat.com 
wrote: 

 Shawn, 
 Thanks a bunch for working with me on this. 
 
 I have deleted all records from my index. Stopped solr. Made the schema 
 changes as requested. Started solr. Then insert the one test record. Then 
 search. Still see the same results. No portal_package is not the unique 
 key, its uri. Which is a string field. 
 
 field name=portal_package type=text_std indexed=true stored=true 
 multiValued=true/ 
 
 fieldType name=text_std class=solr.TextField 
 positionIncrementGap=100 
 tokenizer class=solr.WhitespaceTokenizerFactory/ 
 filter class=solr.LengthFilterFactory min=1 max=300 / 
 /fieldType 
 
 { 
 documentKind: test, 
 uri: test300, 
 id: test300, 
 portal_package:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
  
 } 
 
 
 { 
 responseHeader: { 
 status: 0, 
 QTime: 47, 
 params: { 
 spellcheck: true, 
 enableElevation: false, 
 df: allText, 
 echoParams: all, 
 spellcheck.maxCollations: 5, 
 spellcheck.dictionary: andreasAutoComplete, 
 spellcheck.count: 5, 
 spellcheck.collate: true, 
 spellcheck.onlyMorePopular: true, 
 rows: 10, 
 indent: true, 
 q: 
 portal_package:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
  
 _: 1431719989047, 
 debug: query, 
 wt: json 
 } 
 }, 
 response: { 
 numFound: 1, 
 start: 0, 
 docs: [ 
 { 
 documentKind: test, 
 uri: test300, 
 id: test300, 
 portal_package: [ 
 
 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
  
 ], 
 _version_: 1501267024421060600, 
 timestamp: 2015-05-15T19:56:43.247Z, 
 language: en 
 } 
 ] 
 }, 
 debug: { 
 rawquerystring: 
 portal_package

Re: Problem with solr.LengthFilterFactory

2015-05-18 Thread Charles Sanders

Shawn, 
Sorry about my mistake. I have rerun the test with the correct fieldType 
definition. Still have the same results. See test information below. 

field name=portal_package type=text_std indexed=true stored=true 
multiValued=true/ 

fieldType name=text_std class=solr.TextField positionIncrementGap=100 
analyzer 
tokenizer class=solr.WhitespaceTokenizerFactory/ 
filter class=solr.LengthFilterFactory min=1 max=300 / 
/analyzer 
/fieldType 

{ 
documentKind: test, 
uri: test300, 
id: test300, 
portal_package: 
12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 
} 


{ 
responseHeader: { 
status: 0, 
QTime: 43, 
params: { 
indent: true, 
q: 
portal_package:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
 
_: 1431951254019, 
wt: json 
} 
}, 
response: { 
numFound: 1, 
start: 0, 
docs: [ 
{ 
documentKind: test, 
uri: test300, 
id: test300, 
portal_package: [ 
12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 
], 
_version_: 1501509688204722200, 
timestamp: 2015-05-18T12:13:45.464Z, 
language: en 
} 
] 
} 
} 

- Original Message -

From: Shawn Heisey apa...@elyograg.org 
To: solr-user@lucene.apache.org 
Sent: Friday, May 15, 2015 4:22:10 PM 
Subject: Re: Problem with solr.LengthFilterFactory 

On 5/15/2015 2:06 PM, Charles Sanders wrote: 
 I have deleted all records from my index. Stopped solr. Made the schema 
 changes as requested. Started solr. Then insert the one test record. Then 
 search. Still see the same results. No portal_package is not the unique key, 
 its uri. Which is a string field. 
 
 field name=portal_package type=text_std indexed=true stored=true 
 multiValued=true/ 
 
 fieldType name=text_std class=solr.TextField positionIncrementGap=100 
 tokenizer class=solr.WhitespaceTokenizerFactory/ 
 filter class=solr.LengthFilterFactory min=1 max=300 / 
 /fieldType 

You got rid of analyzer and it's closing tag entirely, that's not 
going to work. This is what you need, and you'll need to reindex the 
doc again: 

fieldType name=text_std class=solr.TextField positionIncrementGap=100 
analyzer 
tokenizer class=solr.WhitespaceTokenizerFactory/ 
filter class=solr.LengthFilterFactory min=1 max=300/ 
/analyzer 
/fieldType 

Here's a temporary paste that you can copy/paste: 

http://apaste.info/NXn 

Thanks, 
Shawn

Re: Problem with solr.LengthFilterFactory

2015-05-18 Thread Charles Sanders

No, the field has always been text. And from the error, its obviously passing a 
very large token to the index, regardless of the tokenizer and filter. 

So I guess I will have to tokenize and filter the text before I send it to 
solr, since solr is not able to properly handle a very large token. More custom 
code. 

Thanks everyone for your help. 
Charles 


- Original Message -

From: Jack Krupansky jack.krupan...@gmail.com 
To: solr-user@lucene.apache.org 
Sent: Monday, May 18, 2015 12:00:37 PM 
Subject: Re: Problem with solr.LengthFilterFactory 

Sorry for not spotting that earlier. Lucene itself does have such a limit. 
No way around it - an individual term is limited to 32K-2 bytes. Lucene is 
designed for searching of terms, not large blob storage. 

Maybe you defined that field as a string originally and later updated your 
schema to a text field, so that Lucene still knows it as an unanalyzed 
string field? You need to delete the index and start over if you want to 
change the field types like that. 

-- Jack Krupansky 

On Mon, May 18, 2015 at 8:33 AM, Charles Sanders csand...@redhat.com 
wrote: 

 Jack, 
 Thanks for the information. If I understand this correctly, the White 
 space tokenizer will break a single token of size 300 into two tokens, one 
 of size 256 and the other of size 44. If this is true, then for the single 
 test document I have used, in the index in the portal_package field, I 
 should see two tokens rather than one large single token. 
 
 If my understanding is correct, then why in my production system, where we 
 occasionally get a single very large token, do I see this error? 
 Caused by: java.lang.IllegalArgumentException: Document contains at least 
 one immense term in field=portal_package (whose UTF8 encoding is longer 
 than the max length 32766) 
 
 The existence of this error would lead me to conclude that a very large 
 single token is making its way through the white space tokenizer and 
 filters to the index where it is rejected. 
 
 I'm afraid my understanding is not complete. Can you fill in the gaps? 
 
 Thanks, 
 Charles 
 
 
 - Original Message - 
 
 From: Jack Krupansky jack.krupan...@gmail.com 
 To: solr-user@lucene.apache.org 
 Sent: Friday, May 15, 2015 4:31:22 PM 
 Subject: Re: Problem with solr.LengthFilterFactory 
 
 Sorry that my brain has turned to mush... the issue you are hitting is due 
 to a known, undocumented limit in the whitespace tokenizer: 
 
 https://issues.apache.org/jira/browse/LUCENE-5785 
 White space tokenizer has undocumented limit of 256 characters per token 
 
 If you look at the parsed query you will see that two query terms were 
 generated. This is because the whitespace tokenizer will simply split long 
 tokens every 256 characters. So, your filter will never see a long term. 
 
 There is a note on the Jira (evidently by me!) that you can use the pattern 
 tokenizer as a workaround. But... if your term is a string anyway, you 
 could just use the keyword tokenizer. 
 
 
 -- Jack Krupansky 
 
 On Fri, May 15, 2015 at 4:06 PM, Charles Sanders csand...@redhat.com 
 wrote: 
 
  Shawn, 
  Thanks a bunch for working with me on this. 
  
  I have deleted all records from my index. Stopped solr. Made the schema 
  changes as requested. Started solr. Then insert the one test record. Then 
  search. Still see the same results. No portal_package is not the unique 
  key, its uri. Which is a string field. 
  
  field name=portal_package type=text_std indexed=true stored=true 
  multiValued=true/ 
  
  fieldType name=text_std class=solr.TextField 
  positionIncrementGap=100 
  tokenizer class=solr.WhitespaceTokenizerFactory/ 
  filter class=solr.LengthFilterFactory min=1 max=300 / 
  /fieldType 
  
  { 
  documentKind: test, 
  uri: test300, 
  id: test300, 
  
 portal_package:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
  
  } 
  
  
  { 
  responseHeader: { 
  status: 0, 
  QTime: 47, 
  params: { 
  spellcheck: true, 
  enableElevation: false, 
  df: allText, 
  echoParams: all, 
  spellcheck.maxCollations: 5, 
  spellcheck.dictionary: andreasAutoComplete, 
  spellcheck.count: 5, 
  spellcheck.collate: true, 
  spellcheck.onlyMorePopular: true, 
  rows: 10, 
  indent: true, 
  q: 
  
 portal_package:123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345

Re: Problem with solr.LengthFilterFactory

2015-05-15 Thread Charles Sanders

Yes, that is what I am seeing. Looking in the code myself, I see no reason for 
this behavior. That is why I assumed I was doing something very wrong. 

Below I have included an example. I set the max length to 300. I insert a 
record with a single token of 500 characters. I expect the token to be removed 
and not included in the index. When I query using the large token, the record 
is returned. I can see the same result using the analysis page in the solr 
console. 

He is a test example: 

field name=portal_package type=text_std indexed=true stored=true 
multiValued=true/ 

fieldType name=text_std class=solr.TextField positionIncrementGap=100 
analyzer type=index 
tokenizer class=solr.WhitespaceTokenizerFactory/ 
filter class=solr.LengthFilterFactory min=1 max=300 / 
/analyzer 
/fieldType 


A test record: 

{ 
documentKind: test, 
uri: test300, 
id: test300, 
portal_package: 
12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 
} 


Query result: 

{ 
responseHeader: { 
status: 0, 
QTime: 55, 
params: { 
indent: true, 
q: 
portal_package:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
 
_: 1431704135745, 
wt: json 
} 
}, 
response: { 
numFound: 1, 
start: 0, 
docs: [ 
{ 
documentKind: test, 
uri: test300, 
id: test300, 
portal_package: [ 
12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 
], 
_version_: 1501249997589446700, 
timestamp: 2015-05-15T15:26:05.205Z, 
language: en 
} 
] 
} 
} 





- Original Message -

From: Shawn Heisey apa...@elyograg.org 
To: solr-user@lucene.apache.org 
Sent: Friday, May 15, 2015 11:13:14 AM 
Subject: Re: Problem with solr.LengthFilterFactory 

On 5/15/2015 8:49 AM, Charles Sanders wrote: 
 I'm seeing a problem with the LengthFilter. It appears to work fine until I 
 increase the max value above 254. At the point it stops removing the very 
 large token from the stream. As a result I get the error: 
 java.lang.IllegalArgumentException: Document contains at least one immense 
 term.. UTF8 encoding is longer than the max length 32766 
 
 I'm certain I'm doing this wrong. Can someone please show me the light. :) 
 
 fieldType name=text_std class=solr.TextField positionIncrementGap=100 
 analyzer type=index 
 tokenizer class=solr.WhitespaceTokenizerFactory/ 
 filter class=solr.LengthFilterFactory min=1 max=254 / 
 /analyzer 
 /fieldType 

So with max=254, you don't get the error? Looking at the code for 
LengthFilter, I can't see any way for it to behave differently with a 
max of 254 vs. a max of 255 or higher. All of the interfaces and 
classes involved use int for length, which means it should work 
perfectly with numbers above 254. 

Thanks, 
Shawn

Re: Problem with solr.LengthFilterFactory

2015-05-15 Thread Charles Sanders

Agree 100%. The value returned is the value stored. Not affected by the 
analyzer. 

However, I searched for that token. See my query? I would expect the analyzer 
to remove the large token. So that when I search for the large token I would 
find nothing. Rather it returns my record. 

Am I missing something here? 


- Original Message -

From: Jack Krupansky jack.krupan...@gmail.com 
To: solr-user@lucene.apache.org 
Sent: Friday, May 15, 2015 11:56:51 AM 
Subject: Re: Problem with solr.LengthFilterFactory 

The returned value is the stored or original source value - only the 
indexed terms are affected by token filtering. 

You could use an update processor if you want to adjust the actual source 
value, such as the truncate processor to truncate long source values: 

http://lucene.apache.org/solr/5_1_0/solr-core/org/apache/solr/update/processor/TruncateFieldUpdateProcessorFactory.html
 


-- Jack Krupansky 

On Fri, May 15, 2015 at 11:38 AM, Charles Sanders csand...@redhat.com 
wrote: 

 Yes, that is what I am seeing. Looking in the code myself, I see no reason 
 for this behavior. That is why I assumed I was doing something very wrong. 
 
 Below I have included an example. I set the max length to 300. I insert a 
 record with a single token of 500 characters. I expect the token to be 
 removed and not included in the index. When I query using the large token, 
 the record is returned. I can see the same result using the analysis page 
 in the solr console. 
 
 He is a test example: 
 
 field name=portal_package type=text_std indexed=true stored=true 
 multiValued=true/ 
 
 fieldType name=text_std class=solr.TextField 
 positionIncrementGap=100 
 analyzer type=index 
 tokenizer class=solr.WhitespaceTokenizerFactory/ 
 filter class=solr.LengthFilterFactory min=1 max=300 / 
 /analyzer 
 /fieldType 
 
 
 A test record: 
 
 { 
 documentKind: test, 
 uri: test300, 
 id: test300, 
 portal_package: 
 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
  
 } 
 
 
 Query result: 
 
 { 
 responseHeader: { 
 status: 0, 
 QTime: 55, 
 params: { 
 indent: true, 
 q: 
 portal_package:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
  
 _: 1431704135745, 
 wt: json 
 } 
 }, 
 response: { 
 numFound: 1, 
 start: 0, 
 docs: [ 
 { 
 documentKind: test, 
 uri: test300, 
 id: test300, 
 portal_package: [ 
 
 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
  
 ], 
 _version_: 1501249997589446700, 
 timestamp: 2015-05-15T15:26:05.205Z, 
 language: en 
 } 
 ] 
 } 
 } 
 
 
 
 
 
 - Original Message - 
 
 From: Shawn Heisey apa...@elyograg.org 
 To: solr-user@lucene.apache.org 
 Sent: Friday, May 15, 2015 11:13:14 AM 
 Subject: Re: Problem with solr.LengthFilterFactory 
 
 On 5/15/2015 8:49 AM, Charles Sanders wrote: 
  I'm seeing a problem with the LengthFilter. It appears to work fine 
 until I increase the max value above 254. At the point it stops removing 
 the very large token from the stream. As a result I get the error: 
  java.lang.IllegalArgumentException: Document contains at least one 
 immense term.. UTF8 encoding is longer than the max length 32766 
  
  I'm certain I'm doing this wrong. Can someone please show me the light. 
 :) 
  
  fieldType name=text_std class=solr.TextField 
 positionIncrementGap=100 
  analyzer type=index 
  tokenizer class=solr.WhitespaceTokenizerFactory/ 
  filter class=solr.LengthFilterFactory min=1 max=254 / 
  /analyzer 
  /fieldType 
 
 So with max=254, you don't get the error? Looking at the code for 
 LengthFilter, I can't see any way for it to behave differently with a 
 max of 254 vs. a max of 255 or higher. All of the interfaces

Problem with solr.LengthFilterFactory

2015-05-15 Thread Charles Sanders

I'm seeing a problem with the LengthFilter. It appears to work fine until I 
increase the max value above 254. At the point it stops removing the very large 
token from the stream. As a result I get the error: 
java.lang.IllegalArgumentException: Document contains at least one immense 
term.. UTF8 encoding is longer than the max length 32766 

I'm certain I'm doing this wrong. Can someone please show me the light. :) 

fieldType name=text_std class=solr.TextField positionIncrementGap=100 
analyzer type=index 
tokenizer class=solr.WhitespaceTokenizerFactory/ 
filter class=solr.LengthFilterFactory min=1 max=254 / 
/analyzer 
/fieldType 


Solr Version - 4.8.1 


-Charles

Re: Problem with solr.LengthFilterFactory

2015-05-15 Thread Charles Sanders

Shawn, 
Thanks a bunch for working with me on this. 

I have deleted all records from my index. Stopped solr. Made the schema changes 
as requested. Started solr. Then insert the one test record. Then search. Still 
see the same results. No portal_package is not the unique key, its uri. Which 
is a string field. 

field name=portal_package type=text_std indexed=true stored=true 
multiValued=true/ 

fieldType name=text_std class=solr.TextField positionIncrementGap=100 
tokenizer class=solr.WhitespaceTokenizerFactory/ 
filter class=solr.LengthFilterFactory min=1 max=300 / 
/fieldType 

{ 
documentKind: test, 
uri: test300, 
id: test300, 
portal_package:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 
} 


{ 
responseHeader: { 
status: 0, 
QTime: 47, 
params: { 
spellcheck: true, 
enableElevation: false, 
df: allText, 
echoParams: all, 
spellcheck.maxCollations: 5, 
spellcheck.dictionary: andreasAutoComplete, 
spellcheck.count: 5, 
spellcheck.collate: true, 
spellcheck.onlyMorePopular: true, 
rows: 10, 
indent: true, 
q: 
portal_package:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
 
_: 1431719989047, 
debug: query, 
wt: json 
} 
}, 
response: { 
numFound: 1, 
start: 0, 
docs: [ 
{ 
documentKind: test, 
uri: test300, 
id: test300, 
portal_package: [ 
12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 
], 
_version_: 1501267024421060600, 
timestamp: 2015-05-15T19:56:43.247Z, 
language: en 
} 
] 
}, 
debug: { 
rawquerystring: 
portal_package:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
 
querystring: 
portal_package:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
 
parsedquery: 
portal_package:1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456
 
portal_package:7890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
 
parsedquery_toString: 
portal_package:1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456
 
portal_package:7890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
 
QParser: LuceneQParser 
} 
}

Re: Problem with solr.LengthFilterFactory

2015-05-15 Thread Charles Sanders

Ran the same test as below. Added echoParams=all and debug=query. Thanks for 
the help! 

Results 

{ 
responseHeader: { 
status: 0, 
QTime: 46, 
params: { 
spellcheck: true, 
enableElevation: false, 
df: allText, 
echoParams: all, 
spellcheck.maxCollations: 5, 
spellcheck.dictionary: andreasAutoComplete, 
spellcheck.count: 5, 
spellcheck.collate: true, 
spellcheck.onlyMorePopular: true, 
rows: 10, 
indent: true, 
q: 
portal_package:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
 
_: 1431715486263, 
debug: query, 
wt: json 
} 
}, 
response: { 
numFound: 1, 
start: 0, 
docs: [ 
{ 
documentKind: test, 
uri: test300, 
id: test300, 
portal_package: [ 
12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 
], 
_version_: 1501262378591846400, 
timestamp: 2015-05-15T18:42:52.625Z, 
language: en 
} 
] 
}, 
debug: { 
rawquerystring: 
portal_package:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
 
querystring: 
portal_package:12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
 
parsedquery: 
portal_package:123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345
 
portal_package:67890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
 
parsedquery_toString: 
portal_package:123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345
 
portal_package:67890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890,
 
QParser: LuceneQParser 
} 
} 




- Original Message -

From: Shawn Heisey apa...@elyograg.org 
To: solr-user@lucene.apache.org 
Sent: Friday, May 15, 2015 2:36:03 PM 
Subject: Re: Problem with solr.LengthFilterFactory 

On 5/15/2015 10:04 AM, Charles Sanders wrote: 
 Agree 100%. The value returned is the value stored. Not affected by the 
 analyzer. 
 
 However, I searched for that token. See my query? I would expect the analyzer 
 to remove the large token. So that when I search for the large token I would 
 find nothing. Rather it returns my record. 
 
 Am I missing something here? 

Can you add echoParams=all and debug=query to your request and send the 
response you get? 

Thanks, 
Shawn

Re: Query Elevation Component only elevates one document in elevateIds list

2015-02-08 Thread Charles Sanders

Ah that did it. I was adding them as an array of strings. Actually setting it 
in SolrJ. 
params.set(QueryElevationParams.IDS, sa); 

Where sa is a String[]. Don't do that. :) 

Much thanks Joel. 


- Original Message -

From: Joel Bernstein joels...@gmail.com 
To: solr-user@lucene.apache.org 
Sent: Friday, February 6, 2015 2:54:20 PM 
Subject: Re: Query Elevation Component only elevates one document in elevateIds 
list 

Try sending the elevateIds as a comma delimited list. 

Joel Bernstein 
Search Engineer at Heliosearch 

On Fri, Feb 6, 2015 at 2:17 PM, Charles Sanders csand...@redhat.com wrote: 

 Using the Query Elevation Component in solr 4.8 and the elevateIds 
 parameter to force 3 documents to the top of the query results. Query and 
 results included below. In the results, you will see only one document of 
 the three is in the query results. It is the first document and it is 
 marked as elevated. The other two are not in the results. 
 
 I've read the documentation here: 
 
 https://archive.apache.org/dist/lucene/solr/ref-guide/apache-solr-ref-guide-4.8.pdf
  
 
 and here: 
 
 https://cwiki.apache.org/confluence/display/solr/The+Query+Elevation+Component
  
 
 Based on the documentation, I think I have done this correctly, but the 
 results are not as expected. Any thoughts? 
 
 -Charles 
 
 
  Query ** 
 
 $HOST/solr/collection1/select?enableElevation=truefl=uri,allTitlefl=%5Belevated%5Dstart=0q=oracleqf=allTitleqf=allTextelevateIds=https%3A%2F%
  
 2Fapi.access.devgssci.devlab.phx1.redhat.com 
 %2Frs%2Farticles%2F216093elevateIds=https%3A%2F% 
 2Fapi.access.devgssci.devlab.phx1.redhat.com 
 %2Frs%2Farticles%2F395013elevateIds=https%3A%2F% 
 2Fapi.access.devgssci.devlab.phx1.redhat.com 
 %2Frs%2Farticles%2F725843wt=jsondefType=edismaxrows=10indent=trueforceElevation=true
  
 
 ** Query - decode so its easier to read * 
 
 $HOST/solr/collection1/select?enableElevation=truefl=uri,allTitlefl=[elevated]start=0q=oracleqf=allTitleqf=allTextelevateIds=
  
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/216093elevateIds=https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/395013elevateIds=https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/725843wt=jsondefType=edismaxrows=10indent=trueforceElevation=true
  
 
 ** Results ** 
 { 
 responseHeader:{ 
 status:0, 
 QTime:274, 
 params:{ 
 enableElevation:true, 
 fl:[uri,allTitle, 
 [elevated]], 
 indent:true, 
 start:0, 
 q:oracle, 
 forceElevation:true, 
 qf:[allTitle, 
 allText], 
 wt:json, 
 elevateIds:[ 
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/216093;, 
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/395013;, 
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/725843;], 
 rows:10, 
 defType:edismax}}, 
 response:{numFound:95055,start:0,maxScore:5.220278,docs:[ 
 { 
 uri: 
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/216093;, 
 allTitle:Storage Management for the Oracle Database on Red Hat 
 Enterprise Linux 6: Using ASM With or Without ASMLib, 
 [elevated]:true}, 
 { 
 uri: 
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/838843
  
 , 
 allTitle:Oracle Salt, 
 [elevated]:false}, 
 { 
 uri: 
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/838413
  
 , 
 allTitle:Oracle iProcurement, 
 [elevated]:false}, 
 { 
 uri: 
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/839063
  
 , 
 allTitle:Oracle TSAM, 
 [elevated]:false}, 
 { 
 uri: 
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/839073
  
 , 
 allTitle:Oracle Tuxedo, 
 [elevated]:false}, 
 { 
 uri: 
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/838953
  
 , 
 allTitle:Oracle Sourcing, 
 [elevated]:false}, 
 { 
 uri: 
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/837383
  
 , 
 allTitle:Oracle Coherence, 
 [elevated]:false}, 
 { 
 uri: 
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/837363
  
 , 
 allTitle:Oracle Clinical, 
 [elevated]:false}, 
 { 
 uri: 
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/837443
  
 , 
 allTitle:Oracle Database, 
 [elevated]:false}, 
 { 
 uri: 
 https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/820373
  
 , 
 allTitle:Oracle RAC, 
 [elevated]:false}] 
 }, 
 spellcheck:{ 
 suggestions:[]}}

Query Elevation Component only elevates one document in elevateIds list

2015-02-06 Thread Charles Sanders

Using the Query Elevation Component in solr 4.8 and the elevateIds parameter to 
force 3 documents to the top of the query results. Query and results included 
below. In the results, you will see only one document of the three is in the 
query results. It is the first document and it is marked as elevated. The other 
two are not in the results. 

I've read the documentation here: 
https://archive.apache.org/dist/lucene/solr/ref-guide/apache-solr-ref-guide-4.8.pdf
 

and here: 
https://cwiki.apache.org/confluence/display/solr/The+Query+Elevation+Component 

Based on the documentation, I think I have done this correctly, but the results 
are not as expected. Any thoughts? 

-Charles 


 Query ** 
$HOST/solr/collection1/select?enableElevation=truefl=uri,allTitlefl=%5Belevated%5Dstart=0q=oracleqf=allTitleqf=allTextelevateIds=https%3A%2F%2Fapi.access.devgssci.devlab.phx1.redhat.com%2Frs%2Farticles%2F216093elevateIds=https%3A%2F%2Fapi.access.devgssci.devlab.phx1.redhat.com%2Frs%2Farticles%2F395013elevateIds=https%3A%2F%2Fapi.access.devgssci.devlab.phx1.redhat.com%2Frs%2Farticles%2F725843wt=jsondefType=edismaxrows=10indent=trueforceElevation=true
 

** Query - decode so its easier to read * 
$HOST/solr/collection1/select?enableElevation=truefl=uri,allTitlefl=[elevated]start=0q=oracleqf=allTitleqf=allTextelevateIds=https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/216093elevateIds=https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/395013elevateIds=https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/725843wt=jsondefType=edismaxrows=10indent=trueforceElevation=true
 

** Results ** 
{ 
responseHeader:{ 
status:0, 
QTime:274, 
params:{ 
enableElevation:true, 
fl:[uri,allTitle, 
[elevated]], 
indent:true, 
start:0, 
q:oracle, 
forceElevation:true, 
qf:[allTitle, 
allText], 
wt:json, 
elevateIds:[https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/216093;,
 
https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/395013;, 
https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/725843;], 
rows:10, 
defType:edismax}}, 
response:{numFound:95055,start:0,maxScore:5.220278,docs:[ 
{ 
uri:https://api.access.devgssci.devlab.phx1.redhat.com/rs/articles/216093;, 
allTitle:Storage Management for the Oracle Database on Red Hat Enterprise 
Linux 6: Using ASM With or Without ASMLib, 
[elevated]:true}, 
{ 
uri:https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/838843;,
 
allTitle:Oracle Salt, 
[elevated]:false}, 
{ 
uri:https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/838413;,
 
allTitle:Oracle iProcurement, 
[elevated]:false}, 
{ 
uri:https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/839063;,
 
allTitle:Oracle TSAM, 
[elevated]:false}, 
{ 
uri:https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/839073;,
 
allTitle:Oracle Tuxedo, 
[elevated]:false}, 
{ 
uri:https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/838953;,
 
allTitle:Oracle Sourcing, 
[elevated]:false}, 
{ 
uri:https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/837383;,
 
allTitle:Oracle Coherence, 
[elevated]:false}, 
{ 
uri:https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/837363;,
 
allTitle:Oracle Clinical, 
[elevated]:false}, 
{ 
uri:https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/837443;,
 
allTitle:Oracle Database, 
[elevated]:false}, 
{ 
uri:https://api.access.devgssci.devlab.phx1.redhat.com/rs/ecosystem/software/820373;,
 
allTitle:Oracle RAC, 
[elevated]:false}] 
}, 
spellcheck:{ 
suggestions:[]}}

Re: Suggester Example In Documentation Not Working

2015-01-23 Thread Charles Sanders

Well, I'm running LucidWorks 2.9.1 which contains Solr 4.8. 

I initially was working with the Solr documentation: 
http://archive.apache.org/dist/lucene/solr/ref-guide/apache-solr-ref-guide-4.8.pdf
 

However, you will notice on page 228, under the section Suggester, it gives 
an example of a suggester search component using solr.SpellCheckComponet. 

Then our support rep at LucidWorks said we should use the documentation found 
here: 
https://cwiki.apache.org/confluence/display/solr/Suggester 

This documentation is for Solr 5.0, however, you will notice the statement: 
Solr has long had the autosuggest functionality, but Solr 4.7 introduced a new 
approach based on a dedicated SuggestComponent .  

So it would appear the solr.SuggestComponent has been around since 4.7, but the 
documentation has not caught up with the changes. Which is the source of a 
little confusion. 

Nevertheless, I had hoped to find a simple working example that I could use as 
a starting point to get the solr.SuggestComponent working so that I might play 
around with it and make it do what I want. The suggester appears to have many 
parameters and options, of which, several contain little or no explanation. 

No problem. I will just have to invest a little more time to unravel how the 
component works and how I can best use it. 

Thanks for your reply. 


- Original Message -

From: Chris Hostetter hossman_luc...@fucit.org 
To: solr-user@lucene.apache.org 
Sent: Thursday, January 22, 2015 12:50:46 PM 
Subject: Re: Suggester Example In Documentation Not Working 


1) which version of Solr are you using? (note that the online HTML ref 
guide is a DRARFT that applies to 5.0 - you may want to review the 
specific released version of the ref guide that applies to your version of 
solr: http://archive.apache.org/dist/lucene/solr/ref-guide/ 

2) the behavior of the suggester is very specific to the contents of the 
dictionary built -- the examples on that page apply to the example docs 
included with solr -- hence the techproduct data, and the example queries 
for input like elec suggesting electronics 

no where on that page is an example using the query kern -- wether or 
not that input would return a suggestion is going to be entirely dependent 
on wether the dictionary you built contains any similar terms to suggest. 

if you can please post more details about your documents -- ideally a full 
set of all the documents in your index (using a small test index of 
course) that may help to understand the results you are getting. 



: Date: Thu, 22 Jan 2015 11:14:43 -0500 (EST) 
: From: Charles Sanders csand...@redhat.com 
: Reply-To: solr-user@lucene.apache.org 
: To: solr-user@lucene.apache.org 
: Subject: Suggester Example In Documentation Not Working 
: 
: Attempting to follow the documentation found here: 
: https://cwiki.apache.org/confluence/display/solr/Suggester 
: 
: The example given in the documentation is not working. See below my 
configuration. I only changed the field names to those in my schema. Can anyone 
provide an example for this component that actually works? 
: 
: searchComponent name=suggest class=solr.SuggestComponent 
: lst name=suggester 
: str name=namemySuggester/str 
: str name=lookupImplFuzzyLookupFactory/str 
: str name=dictionaryImplDocumentDictionaryFactory/str 
: str name=fieldsugg_allText/str 
: str name=weightFieldsuggestWeight/str 
: str name=suggestAnalyzerFieldTypestring/str 
: /lst 
: /searchComponent 
: 
: requestHandler name=/suggest class=solr.SearchHandler startup=lazy 
: lst name=defaults 
: str name=suggesttrue/str 
: str name=suggest.count10/str 
: str name=suggest.buildtrue/str 
: /lst 
: arr name=components 
: strsuggest/str 
: /arr 
: /requestHandler 
: 
: field name=sugg_allText type=string indexed=true multiValued=true 
stored=false/ 
: field name=suggestWeight type=long indexed=true stored=true 
default=1 / 
: 
: 
: 
http://localhost:/solr/collection1/suggest?suggest=truesuggest.build=truesuggest.dictionary=mySuggesterwt=jsonsuggest.q=kern
 
: 
: 
{responseHeader:{status:0,QTime:4},command:build,suggest:{mySuggester:{kern:{numFound:0,suggestions:[]
 
: 

-Hoss 
http://www.lucidworks.com/

Suggester Example In Documentation Not Working

2015-01-22 Thread Charles Sanders

Attempting to follow the documentation found here: 
https://cwiki.apache.org/confluence/display/solr/Suggester 

The example given in the documentation is not working. See below my 
configuration. I only changed the field names to those in my schema. Can anyone 
provide an example for this component that actually works? 

searchComponent name=suggest class=solr.SuggestComponent 
lst name=suggester 
str name=namemySuggester/str 
str name=lookupImplFuzzyLookupFactory/str 
str name=dictionaryImplDocumentDictionaryFactory/str 
str name=fieldsugg_allText/str 
str name=weightFieldsuggestWeight/str 
str name=suggestAnalyzerFieldTypestring/str 
/lst 
/searchComponent 

requestHandler name=/suggest class=solr.SearchHandler startup=lazy 
lst name=defaults 
str name=suggesttrue/str 
str name=suggest.count10/str 
str name=suggest.buildtrue/str 
/lst 
arr name=components 
strsuggest/str 
/arr 
/requestHandler 

field name=sugg_allText type=string indexed=true multiValued=true 
stored=false/ 
field name=suggestWeight type=long indexed=true stored=true 
default=1 / 


http://localhost:/solr/collection1/suggest?suggest=truesuggest.build=truesuggest.dictionary=mySuggesterwt=jsonsuggest.q=kern
 

{responseHeader:{status:0,QTime:4},command:build,suggest:{mySuggester:{kern:{numFound:0,suggestions:[]

Re: SpellCheck (AutoComplete) Not Working In Distributed Environment

2015-01-13 Thread Charles Sanders

Still not able to get my autoComplete component to work in a distributed 
environment. Works fine on a non-distributed system. Also, on the distributed 
system, if I include distrib=false, it works. 

I have tried shards.qt and shards parameters, but they make no difference. I 
should add, I am running SolrCloud and ZooKeeper, if that makes any difference. 
I have played around with this quite a bit, but nothing seems to work. 

When I add shards.qt=/ac {the name of the request handler}, I get an error in 
the solr logs. It simply states: java.lang.NullPointerException. That's it 
nothing more. This is listed as logger SolrCore and SolrDispatchFilter. 

Any ideas, suggestions on how I can troubleshoot and find the problem? Is there 
something specific I should look for? 

Please find attached text file with relevant information from schema.xml and 
sorlconfig.xml. 

Any help greatly appreciated! Thanks, 
-Charles 



- Original Message -

From: Erick Erickson erickerick...@gmail.com 
To: solr-user@lucene.apache.org 
Sent: Tuesday, December 30, 2014 6:07:13 PM 
Subject: Re: SpellCheck (AutoComplete) Not Working In Distributed Environment 

Did you try the shards parameter? See: 
https://cwiki.apache.org/confluence/display/solr/Spell+Checking#SpellChecking-DistributedSpellCheck
 

On Tue, Dec 30, 2014 at 2:20 PM, Charles Sanders csand...@redhat.com wrote: 
 I'm running Solr 4.8 in a distributed environment (2 shards). I have added 
 the spellcheck component to my request handler. In my test system, which is 
 not distributed, it works. But when I move it to the Dev box, which is 
 distributed, 2 shards, it is not working. Is there something additional I 
 must do to get this to work in a distributed environment? 
 
 requestHandler default=true name=standard class=solr.SearchHandler 
 !-- default values for query parameters can be specified, these 
 will be overridden by parameters in the request 
 -- 
 lst name=defaults 
 str name=echoParamsexplicit/str 
 int name=rows10/int 
 str name=dfallText/str 
 !-- default autocomplete settings for this search request handler -- 
 str name=spellchecktrue/str 
 str name=spellcheck.dictionaryandreasAutoComplete/str 
 str name=spellcheck.onlyMorePopulartrue/str 
 str name=spellcheck.count5/str 
 str name=spellcheck.collatetrue/str 
 str name=spellcheck.maxCollations5/str 
 /lst 
 arr name=last-components 
 strautoComplete/str 
 /arr 
 /requestHandler 
 
 searchComponent name=autoComplete class=solr.SpellCheckComponent 
 lst name=spellchecker 
 str name=nameandreasAutoComplete/str 
 str name=classnameorg.apache.solr.spelling.suggest.Suggester/str 
 str 
 name=lookupImplorg.apache.solr.spelling.suggest.tst.TSTLookupFactory/str 
 str name=fieldsugg_allText/str 
 str name=buildOnCommittrue/str 
 float name=threshold.005/float 
 str name=queryAnalyzerFieldTypetext_suggest/str 
 /lst 
 /searchComponent 
 
 
 Any help greatly appreciated! Thanks, 
 -Charles 
 
 
 

* Schema.xml ***
field name=issue_suggest type=text_suggest indexed=true stored=false/
field name=sugg_allText type=text_suggest indexed=true 
multiValued=true stored=false/

fieldType name=text_suggest class=solr.TextField 
positionIncrementGap=100
  analyzer type=index
tokenizer class=solr.StandardTokenizerFactory/
filter class=solr.LowerCaseFilterFactory/
  /analyzer
  analyzer type=query
tokenizer class=solr.StandardTokenizerFactory/
filter class=solr.LowerCaseFilterFactory/
  /analyzer
/fieldType


 Solrconfig.xml ***

!-- Auto-Complete component --
searchComponent name=autoComplete class=solr.SpellCheckComponent
lst name=spellchecker
str name=nameandreasAutoComplete/str
str 
name=classnameorg.apache.solr.spelling.suggest.Suggester/str
str 
name=lookupImplorg.apache.solr.spelling.suggest.tst.TSTLookupFactory/str   
   
str name=fieldsugg_allText/str
str name=buildOnCommittrue/str
float name=threshold.005/float
str name=queryAnalyzerFieldTypetext_suggest/str
/lst
lst name=spellchecker
str name=namerecommendationsAutoComplete/str
str 
name=classnameorg.apache.solr.spelling.suggest.Suggester/str
str 
name=lookupImplorg.apache.solr.spelling.suggest.tst.TSTLookupFactory/str   
   
str name=fieldissue_suggest/str
str name=buildOnCommittrue/str
float name=threshold.005/float
str name=queryAnalyzerFieldTypetext_suggest/str
/lst
/searchComponent

requestHandler name=/ac class=solr.SearchHandler
lst name=defaults
str name=spellchecktrue/str

Re: SpellCheck (AutoComplete) Not Working In Distributed Environment

2014-12-31 Thread Charles Sanders

Got it. Thanks for your help everyone. 

- Original Message -

From: Shawn Heisey apa...@elyograg.org 
To: solr-user@lucene.apache.org 
Sent: Tuesday, December 30, 2014 7:16:59 PM 
Subject: Re: SpellCheck (AutoComplete) Not Working In Distributed Environment 

On 12/30/2014 5:03 PM, Charles Sanders wrote: 
 Thanks for the suggestion. 

 I did not do that originally because the documentation states: 
 This parameter is not required for the /select request handler. 

 Which is what I am using. But I gave it a go, even though I'm not certain of 
 the shard names. Now I have a NPE. 

 solr/collection1/select?q=kernel+prows=1wt=jsonindent=trueshards.qt=/acshards=shard1,shard2

If this is not SolrCloud, then the shards parameter must include most of 
the full base URL for each shard that you will be querying. You can 
only use a bare shard name if you're running SolrCloud. 

The shards.qt parameter that you have used means that when the shards 
are consulted, the /ac handler will be used rather than /select. 

Here's an example of a shards parameter that will combine results from 
three cores on two machines. When not running SolrCloud, this is how 
you do distributed searching: 

shards=idxa2.example.com:8981/solr/ai-inclive,idxa1.example.com:8981/solr/ai-0live,idxa2.example.com:8981/solr/ai-1live

SolrCloud hides almost all of this complexity. 

Thanks, 
Shawn

Filter Queries Do Not Allow OR operator ??

2014-12-31 Thread Charles Sanders

I'm running solr 4.8. On my test system I ran the query 
http://localhost:/solr/collection1/select?q=*:*fq=-hasPublishedRevision:truerows=100fl=documentKind,
 hasPublishedRevisionwt=jsonindent=true 

The results are as expected. It returns 59 documents of various documentKinds, 
all having a hasPublishedRevision value of false or none at all (just not 
true). 

Now I run the query 
http://localhost:/solr/collection1/select?q=*:*fq=-hasPublishedRevision:true
 OR documentKind:Solutionrows=100fl=documentKind, 
hasPublishedRevisionwt=jsonindent=true 

This does not return the expected results. This returns only 5 documents. All 
have documentKind as Solution. The results returned are consistent with an AND 
condition in the filter query not an OR. 

I expected the filter query used to have returned all documents that do not 
have a hasPublishedRevision value of true (regardless of documentKind), as well 
as all documents with a documentKind of Solution (regardless to 
hasPublishedRevision value). 

Obviously filter queries do not support 'any' boolean logic. I will have to 
find a work around for this issue. 

-Charles

Re: Filter Queries Do Not Allow OR operator ??

2014-12-31 Thread Charles Sanders

Shawn, 
Thanks. Not what I expected (still thinking in database terms), but makes sense 
based on the results. I understand now and I think I have the filter I need. 

Charles 


- Original Message -

From: Shawn Heisey apa...@elyograg.org 
To: solr-user@lucene.apache.org 
Sent: Wednesday, December 31, 2014 12:56:02 PM 
Subject: Re: Filter Queries Do Not Allow OR operator ?? 

On 12/31/2014 10:45 AM, Charles Sanders wrote: 
 I'm running solr 4.8. On my test system I ran the query 
 http://localhost:/solr/collection1/select?q=*:*fq=-hasPublishedRevision:truerows=100fl=documentKind,
  hasPublishedRevisionwt=jsonindent=true 
 
 The results are as expected. It returns 59 documents of various 
 documentKinds, all having a hasPublishedRevision value of false or none at 
 all (just not true). 
 
 Now I run the query 
 http://localhost:/solr/collection1/select?q=*:*fq=-hasPublishedRevision:true
  OR documentKind:Solutionrows=100fl=documentKind, 
 hasPublishedRevisionwt=jsonindent=true 
 
 This does not return the expected results. This returns only 5 documents. All 
 have documentKind as Solution. The results returned are consistent with an 
 AND condition in the filter query not an OR. 
 
 I expected the filter query used to have returned all documents that do not 
 have a hasPublishedRevision value of true (regardless of documentKind), as 
 well as all documents with a documentKind of Solution (regardless to 
 hasPublishedRevision value). 
 
 Obviously filter queries do not support 'any' boolean logic. I will have to 
 find a work around for this issue. 

Your filter query starts with a negative clause. A simple truism for 
Solr: You cannot subtract from nothing. If the negative query is 
simple enough, Solr can detect it and implicitly add the *:* (all 
documents) for the negative clause to subtract from, but as soon as 
there is any complexity at all, Solr is not able to recognize the 
situation and repair it automatically. 

Try one of these filters instead. I am not sure exactly what you're 
after, so you will have to pick the one that does what you need: 

fq=*:* -(hasPublishRevision:true OR documentKind:Solution) 
fq=(*:* -hasPublishRevision:true) OR documentKind:Solution 

Thanks, 
Shawn

SpellCheck (AutoComplete) Not Working In Distributed Environment

2014-12-30 Thread Charles Sanders

I'm running Solr 4.8 in a distributed environment (2 shards). I have added the 
spellcheck component to my request handler. In my test system, which is not 
distributed, it works. But when I move it to the Dev box, which is distributed, 
2 shards, it is not working. Is there something additional I must do to get 
this to work in a distributed environment? 

requestHandler default=true name=standard class=solr.SearchHandler 
!-- default values for query parameters can be specified, these 
will be overridden by parameters in the request 
-- 
lst name=defaults 
str name=echoParamsexplicit/str 
int name=rows10/int 
str name=dfallText/str 
!-- default autocomplete settings for this search request handler -- 
str name=spellchecktrue/str 
str name=spellcheck.dictionaryandreasAutoComplete/str 
str name=spellcheck.onlyMorePopulartrue/str 
str name=spellcheck.count5/str 
str name=spellcheck.collatetrue/str 
str name=spellcheck.maxCollations5/str 
/lst 
arr name=last-components 
strautoComplete/str 
/arr 
/requestHandler 

searchComponent name=autoComplete class=solr.SpellCheckComponent 
lst name=spellchecker 
str name=nameandreasAutoComplete/str 
str name=classnameorg.apache.solr.spelling.suggest.Suggester/str 
str 
name=lookupImplorg.apache.solr.spelling.suggest.tst.TSTLookupFactory/str 
str name=fieldsugg_allText/str 
str name=buildOnCommittrue/str 
float name=threshold.005/float 
str name=queryAnalyzerFieldTypetext_suggest/str 
/lst 
/searchComponent 


Any help greatly appreciated! Thanks, 
-Charles

Re: SpellCheck (AutoComplete) Not Working In Distributed Environment

2014-12-30 Thread Charles Sanders

Thanks for the suggestion. 

I did not do that originally because the documentation states: 
This parameter is not required for the /select request handler. 

Which is what I am using. But I gave it a go, even though I'm not certain of 
the shard names. Now I have a NPE. 

solr/collection1/select?q=kernel+prows=1wt=jsonindent=trueshards.qt=/acshards=shard1,shard2
 

{
  responseHeader:{
status:500,
QTime:12,
params:{
  shards:shard1,shard2,
  indent:true,
  shards.qt:/ac,
  q:kernel p,
  wt:json,
  rows:1}},
  error:{
trace:java.lang.NullPointerException\n\tat 
org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:901)\n\tat
 
org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:686)\n\tat
 
org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:665)\n\tat
 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:325)\n\tat
 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)\n\tat
 org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)\n\tat 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:787)\n\tat
 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:431)\n\tat
 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)\n\tat
 
com.lucid.servlet.LweSolrDispatchFilter.doFilter(LweSolrDispatchFilter.java:202)\n\tat
 
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)\n\tat
 
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)\n\tat
 
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)\n\tat
 
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)\n\tat
 
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)\n\tat
 com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)\n\tat 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)\n\tat
 
org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:212)\n\tat
 
org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:179)\n\tat
 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)\n\tat
 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)\n\tat
 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)\n\tat
 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)\n\tat
 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)\n\tat
 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)\n\tat
 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)\n\tat 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)\n\tat
 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)\n\tat
 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)\n\tat
 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)\n\tat
 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)\n\tat
 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)\n\tat
 org.eclipse.jetty.server.Server.handle(Server.java:351)\n\tat 
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)\n\tat
 
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)\n\tat
 
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)\n\tat
 
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)\n\tat
 org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634)\n\tat 
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)\n\tat 
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)\n\tat
 
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)\n\tat
 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)\n\tat
 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)\n\tat
 java.lang.Thread.run(Thread.java:744)\n,
code:500}} 



- Original Message -

From: Erick Erickson erickerick...@gmail.com 
To: solr-user@lucene.apache.org 
Sent: Tuesday, December 30, 2014 6:07:13 PM 
Subject: Re: SpellCheck (AutoComplete) Not Working In Distributed Environment 

Did you try the shards parameter? See: 
https://cwiki.apache.org/confluence/display/solr/Spell+Checking#SpellChecking-DistributedSpellCheck
 

On Tue, Dec 30, 2014 at 2:20 PM, Charles Sanders csand

Re: query keyword but no result (solr 8)

DocTransformer [explain] not working in Solr 5

Re: How to set multivalued false, using SolrJ

Re: SyntaxError - Block Join Parent Query

Re: SyntaxError - Block Join Parent Query

Re: SyntaxError - Block Join Parent Query

SyntaxError - Block Join Parent Query

Re: Problem with solr.LengthFilterFactory

Re: Problem with solr.LengthFilterFactory

Re: Problem with solr.LengthFilterFactory

Re: Problem with solr.LengthFilterFactory

Re: Problem with solr.LengthFilterFactory

Problem with solr.LengthFilterFactory

Re: Problem with solr.LengthFilterFactory

Re: Problem with solr.LengthFilterFactory

Re: Query Elevation Component only elevates one document in elevateIds list

Query Elevation Component only elevates one document in elevateIds list

Re: Suggester Example In Documentation Not Working

Suggester Example In Documentation Not Working

Re: SpellCheck (AutoComplete) Not Working In Distributed Environment

Re: SpellCheck (AutoComplete) Not Working In Distributed Environment

Filter Queries Do Not Allow OR operator ??

Re: Filter Queries Do Not Allow OR operator ??

SpellCheck (AutoComplete) Not Working In Distributed Environment

Re: SpellCheck (AutoComplete) Not Working In Distributed Environment

25 matches

Site Navigation

Mail list logo

Footer information