applicability of schema on document
I am trying to understand how the schema and it's field types gets applied
to a document. Is it based on a document Id e.g. how to specify solr to
store fields like age as integer, dateofbirth as date but not as a string
or vice versa. Also, is it possible to change the field type at runtime?
Direct update using below command adds the values even when the id doesn't
exist:
curl http://localhost:8983/solr/update\?commit\=true -H
'Content-type:application/json' -d '[{dob: {add:
[2014-02-12T12:00:00Z, 2014-07-16T12:00:00Z]}, id: }]'
My doubts in the above scenario are:
- Is it taking some default types based on parsed values?
- Is it also possible to store multiple types in a single field?
- What are the rules for schema less doc
Above scenario is tried using solr.war in example/webapps Following are
multiple schema files in example directory
./example/example-DIH/solr/db/conf/schema.xml
./example/example-DIH/solr/mail/conf/schema.xml
./example/example-DIH/solr/rss/conf/schema.xml
./example/example-DIH/solr/solr/conf/schema.xml
./example/example-DIH/solr/tika/conf/schema.xml
./example/example-schemaless/solr/collection1/conf/schema.xml
./example/multicore/core0/conf/schema.xml
./example/multicore/core1/conf/schema.xml
./example/solr/collection1/conf/schema.xml
Any documentation or unit tests describing the flow, creating and using the
schema will be helpful.
Thanks
Anurag
demo app explaining solr features
I am wondering if there is any demo app that can demonstrate all the features/capabilities of solr. My intention is to understand, use and play around all the features supported by solr. Also looking to explore how solr fits with NLP (like open-nlp), different datastores (like cassandra, mongodb, arangoDB, couchDB), ML engines(mahout, prediction.io etc) and caching servers (redis, memcache). Thanks Anurag
Re: java.lang.NumberFormatException: For input string: string;#-6.872515521, 53.28853084
And how is the schema field declared. Seems like it's a TrieDoubleField, which should be a simple floating point value. You should be using the spatial field types. -- Jack Krupansky -Original Message- From: Erick Erickson Sent: Friday, September 26, 2014 12:20 PM To: solr-user@lucene.apache.org Subject: Re: java.lang.NumberFormatException: For input string: string;#-6.872515521, 53.28853084 It looks like the data is, literally, string;#-6.872515521, 53.28853084 or maybe #-6.872515521, 53.28853084 either way the data isn't in anything like the format expected. Of course I may be mis-reading this, but it looks like your input process isn't doing what you expect. How are you sending the data to Solr? Best, Erick On Fri, Sep 26, 2014 at 7:00 AM, lalitjangra lalit.j.jan...@gmail.com wrote: Hi, I am trying to index latitude and longitude data into solr but getting error as below. ERROR - 2014-09-26 13:44:16.503; org.apache.solr.common.SolrException; org.apache.solr.common.SolrException: ERROR: [doc=http://testirishwaterportal/sites/am/ass/asi/agg/ami/Lists/Waste Waste Water Pumping Station/DispForm.aspx?ID=841] Error adding field 'gis_x0020_coordinate'='string;#-6.872515521, 53.28853084' msg=For input string: string;#-6.872515521, 53.28853084 at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:167) at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:215) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51) at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:569) at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:705) at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435) at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100) at org.apache.solr.handler.extraction.ExtractingDocumentLoader.doAdd(ExtractingDocumentLoader.java:121) at org.apache.solr.handler.extraction.ExtractingDocumentLoader.addDoc(ExtractingDocumentLoader.java:126) at org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:228) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:710) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:413) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:368) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at
Expunging Deletes
I'm running into memory issues and wondering if I should be using expungeDeletes on commits. The server in question at the moment has 450k documents in the collection and represents 15GB on disk. There are also 700k+ Deleted Docs and I'm guessing that is part of the disk space consumption but I am not having any luck getting that cleared out. I noticed the expungeDeletes=false in some of the log output related to commit but didn't try setting it to true yet. Will this clear those deleted documents and recover that space? Or should something else already be managing that but maybe isn't configured correctly? Our data is user specific data, each customer has their own database structure so it varies with each user. They also add/remove data fairly frequently in many cases. To compare another collection of the same data type, there are 1M documents and about 120k deleted docs but disk space is only 6.3GB. Hoping someone can share some advice about how to manage this. Thanks, Eric
fq syntax for requiring all multiValued field values to be within a list?
Hello, I've attempted to figure this out from reading the documentation but without much luck. I looked for a comprehensive query syntax specification (e.g., with BNF and a list of operator semantics) but I'm unable to find such a document (does such a thing exist? or is the syntax too much of a moving target?) I'm using 4.6.1, if that makes a difference, though upgrading is an option if it necessary to make this work. I've got a multiValued field color, which describes the colors of item in the database. Items can have zero or more colors. What I want is to be able to filter out all hits that contain colors not within a constraining list, i.e., something like NOT (color NOT IN (red,yellow,green)). So the following would be passed by the filter: (no value for 'color') color: red color: red, color: green whereas these would be excluded: color: red, color: blue color: magenta Nothing I've come up with so far, e.g. -(-color: red -color: green), seems to work. I've also looked into using a function query but it seems to lack operators for dealing with string multivalued fields. Ideas? Thanks, Bill
Re: fq syntax for requiring all multiValued field values to be within a list?
On Sat, Sep 27, 2014 at 2:52 PM, White, Bill bwh...@ptfs.com wrote: Hello, I've attempted to figure this out from reading the documentation but without much luck. I looked for a comprehensive query syntax specification (e.g., with BNF and a list of operator semantics) but I'm unable to find such a document (does such a thing exist? or is the syntax too much of a moving target?) I'm using 4.6.1, if that makes a difference, though upgrading is an option if it necessary to make this work. I've got a multiValued field color, which describes the colors of item in the database. Items can have zero or more colors. What I want is to be able to filter out all hits that contain colors not within a constraining list, i.e., something like NOT (color NOT IN (red,yellow,green)). So the following would be passed by the filter: (no value for 'color') color: red color: red, color: green whereas these would be excluded: color: red, color: blue color: magenta You're looking for things that either match red, yellow, or green, or have no color: color:(red yellow green) OR (*:* -color:*) -Yonik http://heliosearch.org - native code faceting, facet functions, sub-facets, off-heap data
Re: fq syntax for requiring all multiValued field values to be within a list?
Not just that. I'm looking for things which match either red or yellow or green, but do NOT match ANY other color. I can probably drop the requirement related to having no color. On Sat, Sep 27, 2014 at 3:28 PM, Yonik Seeley yo...@heliosearch.com wrote: On Sat, Sep 27, 2014 at 2:52 PM, White, Bill bwh...@ptfs.com wrote: Hello, I've attempted to figure this out from reading the documentation but without much luck. I looked for a comprehensive query syntax specification (e.g., with BNF and a list of operator semantics) but I'm unable to find such a document (does such a thing exist? or is the syntax too much of a moving target?) I'm using 4.6.1, if that makes a difference, though upgrading is an option if it necessary to make this work. I've got a multiValued field color, which describes the colors of item in the database. Items can have zero or more colors. What I want is to be able to filter out all hits that contain colors not within a constraining list, i.e., something like NOT (color NOT IN (red,yellow,green)). So the following would be passed by the filter: (no value for 'color') color: red color: red, color: green whereas these would be excluded: color: red, color: blue color: magenta You're looking for things that either match red, yellow, or green, or have no color: color:(red yellow green) OR (*:* -color:*) -Yonik http://heliosearch.org - native code faceting, facet functions, sub-facets, off-heap data
Re: fq syntax for requiring all multiValued field values to be within a list?
Sorry, color is multivalued, so a given record might be both blue and red. I don't want those to show up in the results. On Sat, Sep 27, 2014 at 3:36 PM, White, Bill bwh...@ptfs.com wrote: Not just that. I'm looking for things which match either red or yellow or green, but do NOT match ANY other color. I can probably drop the requirement related to having no color. On Sat, Sep 27, 2014 at 3:28 PM, Yonik Seeley yo...@heliosearch.com wrote: On Sat, Sep 27, 2014 at 2:52 PM, White, Bill bwh...@ptfs.com wrote: Hello, I've attempted to figure this out from reading the documentation but without much luck. I looked for a comprehensive query syntax specification (e.g., with BNF and a list of operator semantics) but I'm unable to find such a document (does such a thing exist? or is the syntax too much of a moving target?) I'm using 4.6.1, if that makes a difference, though upgrading is an option if it necessary to make this work. I've got a multiValued field color, which describes the colors of item in the database. Items can have zero or more colors. What I want is to be able to filter out all hits that contain colors not within a constraining list, i.e., something like NOT (color NOT IN (red,yellow,green)). So the following would be passed by the filter: (no value for 'color') color: red color: red, color: green whereas these would be excluded: color: red, color: blue color: magenta You're looking for things that either match red, yellow, or green, or have no color: color:(red yellow green) OR (*:* -color:*) -Yonik http://heliosearch.org - native code faceting, facet functions, sub-facets, off-heap data
Re: fq syntax for requiring all multiValued field values to be within a list?
On Sat, Sep 27, 2014 at 3:36 PM, White, Bill bwh...@ptfs.com wrote: Sorry, color is multivalued, so a given record might be both blue and red. I don't want those to show up in the results. I think the only way currently (out of the box) is to enumerate the other possible colors to exclude them. color:(red yellow green) -color:(blue cyan xxx) -Yonik http://heliosearch.org - native code faceting, facet functions, sub-facets, off-heap data On Sat, Sep 27, 2014 at 3:36 PM, White, Bill bwh...@ptfs.com wrote: Not just that. I'm looking for things which match either red or yellow or green, but do NOT match ANY other color. I can probably drop the requirement related to having no color. On Sat, Sep 27, 2014 at 3:28 PM, Yonik Seeley yo...@heliosearch.com wrote: On Sat, Sep 27, 2014 at 2:52 PM, White, Bill bwh...@ptfs.com wrote: Hello, I've attempted to figure this out from reading the documentation but without much luck. I looked for a comprehensive query syntax specification (e.g., with BNF and a list of operator semantics) but I'm unable to find such a document (does such a thing exist? or is the syntax too much of a moving target?) I'm using 4.6.1, if that makes a difference, though upgrading is an option if it necessary to make this work. I've got a multiValued field color, which describes the colors of item in the database. Items can have zero or more colors. What I want is to be able to filter out all hits that contain colors not within a constraining list, i.e., something like NOT (color NOT IN (red,yellow,green)). So the following would be passed by the filter: (no value for 'color') color: red color: red, color: green whereas these would be excluded: color: red, color: blue color: magenta You're looking for things that either match red, yellow, or green, or have no color: color:(red yellow green) OR (*:* -color:*) -Yonik http://heliosearch.org - native code faceting, facet functions, sub-facets, off-heap data
Re: fq syntax for requiring all multiValued field values to be within a list?
Hmm, that won't work since color is free-form. Is there a way to invoke (via fq) a user-defined function (hopefully defined as part of the fq syntax, but alternatively, written in Java) and have it applied to the resultset? On Sat, Sep 27, 2014 at 3:41 PM, Yonik Seeley yo...@heliosearch.com wrote: On Sat, Sep 27, 2014 at 3:36 PM, White, Bill bwh...@ptfs.com wrote: Sorry, color is multivalued, so a given record might be both blue and red. I don't want those to show up in the results. I think the only way currently (out of the box) is to enumerate the other possible colors to exclude them. color:(red yellow green) -color:(blue cyan xxx) -Yonik http://heliosearch.org - native code faceting, facet functions, sub-facets, off-heap data On Sat, Sep 27, 2014 at 3:36 PM, White, Bill bwh...@ptfs.com wrote: Not just that. I'm looking for things which match either red or yellow or green, but do NOT match ANY other color. I can probably drop the requirement related to having no color. On Sat, Sep 27, 2014 at 3:28 PM, Yonik Seeley yo...@heliosearch.com wrote: On Sat, Sep 27, 2014 at 2:52 PM, White, Bill bwh...@ptfs.com wrote: Hello, I've attempted to figure this out from reading the documentation but without much luck. I looked for a comprehensive query syntax specification (e.g., with BNF and a list of operator semantics) but I'm unable to find such a document (does such a thing exist? or is the syntax too much of a moving target?) I'm using 4.6.1, if that makes a difference, though upgrading is an option if it necessary to make this work. I've got a multiValued field color, which describes the colors of item in the database. Items can have zero or more colors. What I want is to be able to filter out all hits that contain colors not within a constraining list, i.e., something like NOT (color NOT IN (red,yellow,green)). So the following would be passed by the filter: (no value for 'color') color: red color: red, color: green whereas these would be excluded: color: red, color: blue color: magenta You're looking for things that either match red, yellow, or green, or have no color: color:(red yellow green) OR (*:* -color:*) -Yonik http://heliosearch.org - native code faceting, facet functions, sub-facets, off-heap data
Re: fq syntax for requiring all multiValued field values to be within a list?
On Sat, Sep 27, 2014 at 11:36 PM, White, Bill bwh...@ptfs.com wrote:
but do NOT match ANY other color.
Bill, I miss the whole picture, it's worth to rephrase the problem in one
sentence.
But regarding the quote above, you can try to use exclusive ranges
https://lucene.apache.org/core/4_6_0/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Range_Searches
fq=-color:({* TO green} {green TO red} {red TO *})
just don't forget to build ranges alphabetically
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
Re: fq syntax for requiring all multiValued field values to be within a list?
OK, let me try phrasing it better.
How do I exclude from search, any result which contains any value for
multivalued field 'color' which is not within a given constraint set
(e.g., red, green, yellow, burnt sienna), given that I do not what
any of the other possible values of 'color' are?
In pseudocode:
for all x in result.color
if x not in (red,green,yellow, burnt sienna)
filter out result
I don't see how range queries would work since I have no control over the
possible values of 'color', e.g., there could be a valid color lemon
yellow between green and red, and I don't want a result which has
(color: red, color: lemon yellow)
On Sat, Sep 27, 2014 at 4:02 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
On Sat, Sep 27, 2014 at 11:36 PM, White, Bill bwh...@ptfs.com wrote:
but do NOT match ANY other color.
Bill, I miss the whole picture, it's worth to rephrase the problem in one
sentence.
But regarding the quote above, you can try to use exclusive ranges
https://lucene.apache.org/core/4_6_0/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Range_Searches
fq=-color:({* TO green} {green TO red} {red TO *})
just don't forget to build ranges alphabetically
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
Re: fq syntax for requiring all multiValued field values to be within a list?
On Sat, Sep 27, 2014 at 3:46 PM, White, Bill bwh...@ptfs.com wrote: Hmm, that won't work since color is free-form. Is there a way to invoke (via fq) a user-defined function (hopefully defined as part of the fq syntax, but alternatively, written in Java) and have it applied to the resultset? https://wiki.apache.org/solr/SolrPlugins#QParserPlugin -Yonik http://heliosearch.org - native code faceting, facet functions, sub-facets, off-heap data
Re: fq syntax for requiring all multiValued field values to be within a list?
Thanks! On Sat, Sep 27, 2014 at 4:18 PM, Yonik Seeley yo...@heliosearch.com wrote: On Sat, Sep 27, 2014 at 3:46 PM, White, Bill bwh...@ptfs.com wrote: Hmm, that won't work since color is free-form. Is there a way to invoke (via fq) a user-defined function (hopefully defined as part of the fq syntax, but alternatively, written in Java) and have it applied to the resultset? https://wiki.apache.org/solr/SolrPlugins#QParserPlugin -Yonik http://heliosearch.org - native code faceting, facet functions, sub-facets, off-heap data
Re: fq syntax for requiring all multiValued field values to be within a list?
indeed!
the exclusive range {green TO red} matches to the lemon yellow
hence, the negation suppresses it from appearing
fq=-color:{green TO red}
then you need to suppress eg black and white also
fq=-color:({* TO green} {green TO red} {red TO *})
I have no control over the
possible values of 'color',
You don't need to control possible values, you just suppressing any values
beside of the given green and red.
Mind that either green or red passes that negation of exclusive ranges
disjunction.
On Sun, Sep 28, 2014 at 12:15 AM, White, Bill bwh...@ptfs.com wrote:
OK, let me try phrasing it better.
How do I exclude from search, any result which contains any value for
multivalued field 'color' which is not within a given constraint set
(e.g., red, green, yellow, burnt sienna), given that I do not what
any of the other possible values of 'color' are?
In pseudocode:
for all x in result.color
if x not in (red,green,yellow, burnt sienna)
filter out result
I don't see how range queries would work since I have no control over the
possible values of 'color', e.g., there could be a valid color lemon
yellow between green and red, and I don't want a result which has
(color: red, color: lemon yellow)
On Sat, Sep 27, 2014 at 4:02 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
On Sat, Sep 27, 2014 at 11:36 PM, White, Bill bwh...@ptfs.com wrote:
but do NOT match ANY other color.
Bill, I miss the whole picture, it's worth to rephrase the problem in one
sentence.
But regarding the quote above, you can try to use exclusive ranges
https://lucene.apache.org/core/4_6_0/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Range_Searches
fq=-color:({* TO green} {green TO red} {red TO *})
just don't forget to build ranges alphabetically
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
Re: Append children documents for nested document
It's never possible. You have to reindex whole block. On Tue, Sep 16, 2014 at 6:20 PM, bradhill99 bradhil...@yahoo.com wrote: Anyone can help me about this? or Solr not support adding additional children documents to existed parent document? -- View this message in context: http://lucene.472066.n3.nabble.com/Append-children-documents-for-nested-document-tp4157087p4159152.html Sent from the Solr - User mailing list archive at Nabble.com. -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
Re: fq syntax for requiring all multiValued field values to be within a list?
Heh... very clever, Mikhail!
-Yonik
http://heliosearch.org - native code faceting, facet functions,
sub-facets, off-heap data
On Sat, Sep 27, 2014 at 4:43 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
indeed!
the exclusive range {green TO red} matches to the lemon yellow
hence, the negation suppresses it from appearing
fq=-color:{green TO red}
then you need to suppress eg black and white also
fq=-color:({* TO green} {green TO red} {red TO *})
I have no control over the
possible values of 'color',
You don't need to control possible values, you just suppressing any values
beside of the given green and red.
Mind that either green or red passes that negation of exclusive ranges
disjunction.
On Sun, Sep 28, 2014 at 12:15 AM, White, Bill bwh...@ptfs.com wrote:
OK, let me try phrasing it better.
How do I exclude from search, any result which contains any value for
multivalued field 'color' which is not within a given constraint set
(e.g., red, green, yellow, burnt sienna), given that I do not what
any of the other possible values of 'color' are?
In pseudocode:
for all x in result.color
if x not in (red,green,yellow, burnt sienna)
filter out result
I don't see how range queries would work since I have no control over the
possible values of 'color', e.g., there could be a valid color lemon
yellow between green and red, and I don't want a result which has
(color: red, color: lemon yellow)
On Sat, Sep 27, 2014 at 4:02 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
On Sat, Sep 27, 2014 at 11:36 PM, White, Bill bwh...@ptfs.com wrote:
but do NOT match ANY other color.
Bill, I miss the whole picture, it's worth to rephrase the problem in one
sentence.
But regarding the quote above, you can try to use exclusive ranges
https://lucene.apache.org/core/4_6_0/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Range_Searches
fq=-color:({* TO green} {green TO red} {red TO *})
just don't forget to build ranges alphabetically
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
Re: fq syntax for requiring all multiValued field values to be within a list?
Hmm. If I understand correctly this builds a set out of open intervals
(exclusive ranges), that's a great idea!
It doesn't seem to work for me, though; fq=-color:({* TO red} {red TO *})
is giving me results with color=burnt sienna
The field is defined as field name=color type=string indexed=true
stored=true multiValued=true /
On Sat, Sep 27, 2014 at 4:43 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
indeed!
the exclusive range {green TO red} matches to the lemon yellow
hence, the negation suppresses it from appearing
fq=-color:{green TO red}
then you need to suppress eg black and white also
fq=-color:({* TO green} {green TO red} {red TO *})
I have no control over the
possible values of 'color',
You don't need to control possible values, you just suppressing any values
beside of the given green and red.
Mind that either green or red passes that negation of exclusive ranges
disjunction.
On Sun, Sep 28, 2014 at 12:15 AM, White, Bill bwh...@ptfs.com wrote:
OK, let me try phrasing it better.
How do I exclude from search, any result which contains any value for
multivalued field 'color' which is not within a given constraint set
(e.g., red, green, yellow, burnt sienna), given that I do not
what
any of the other possible values of 'color' are?
In pseudocode:
for all x in result.color
if x not in (red,green,yellow, burnt sienna)
filter out result
I don't see how range queries would work since I have no control over the
possible values of 'color', e.g., there could be a valid color lemon
yellow between green and red, and I don't want a result which has
(color: red, color: lemon yellow)
On Sat, Sep 27, 2014 at 4:02 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
On Sat, Sep 27, 2014 at 11:36 PM, White, Bill bwh...@ptfs.com wrote:
but do NOT match ANY other color.
Bill, I miss the whole picture, it's worth to rephrase the problem in
one
sentence.
But regarding the quote above, you can try to use exclusive ranges
https://lucene.apache.org/core/4_6_0/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Range_Searches
fq=-color:({* TO green} {green TO red} {red TO *})
just don't forget to build ranges alphabetically
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
Re: fq syntax for requiring all multiValued field values to be within a list?
hm. try to convert it to query q=-color:({* TO red} {red TO *}) and check
the explanation from debugQuery=true
I tried to play with my index
q: *:*,
facet.field: swatchColors_string_mv,
fq: -swatchColors_string_mv:({* TO RED} {RED TO *}),
I got the following facets:
facet_fields: {
swatchColors_string_mv: [
RED,
122,
BLACK,
0,
BLUE,
0,
BROWN,
0,
GREEN,
0,
so, it works for me at least...
On Sun, Sep 28, 2014 at 12:54 AM, White, Bill bwh...@ptfs.com wrote:
Hmm. If I understand correctly this builds a set out of open intervals
(exclusive ranges), that's a great idea!
It doesn't seem to work for me, though; fq=-color:({* TO red} {red TO *})
is giving me results with color=burnt sienna
The field is defined as field name=color type=string indexed=true
stored=true multiValued=true /
On Sat, Sep 27, 2014 at 4:43 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
indeed!
the exclusive range {green TO red} matches to the lemon yellow
hence, the negation suppresses it from appearing
fq=-color:{green TO red}
then you need to suppress eg black and white also
fq=-color:({* TO green} {green TO red} {red TO *})
I have no control over the
possible values of 'color',
You don't need to control possible values, you just suppressing any
values
beside of the given green and red.
Mind that either green or red passes that negation of exclusive ranges
disjunction.
On Sun, Sep 28, 2014 at 12:15 AM, White, Bill bwh...@ptfs.com wrote:
OK, let me try phrasing it better.
How do I exclude from search, any result which contains any value for
multivalued field 'color' which is not within a given constraint set
(e.g., red, green, yellow, burnt sienna), given that I do not
what
any of the other possible values of 'color' are?
In pseudocode:
for all x in result.color
if x not in (red,green,yellow, burnt sienna)
filter out result
I don't see how range queries would work since I have no control over
the
possible values of 'color', e.g., there could be a valid color lemon
yellow between green and red, and I don't want a result which has
(color: red, color: lemon yellow)
On Sat, Sep 27, 2014 at 4:02 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
On Sat, Sep 27, 2014 at 11:36 PM, White, Bill bwh...@ptfs.com
wrote:
but do NOT match ANY other color.
Bill, I miss the whole picture, it's worth to rephrase the problem in
one
sentence.
But regarding the quote above, you can try to use exclusive ranges
https://lucene.apache.org/core/4_6_0/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Range_Searches
fq=-color:({* TO green} {green TO red} {red TO *})
just don't forget to build ranges alphabetically
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
Re: fq syntax for requiring all multiValued field values to be within a list?
It worked for me once I changed to
-color:({* TO red} OR {red TO *})
I'm not sure why the OR is needed, maybe it's my version? (4.6.1)
On Sat, Sep 27, 2014 at 5:22 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
hm. try to convert it to query q=-color:({* TO red} {red TO *}) and check
the explanation from debugQuery=true
I tried to play with my index
q: *:*,
facet.field: swatchColors_string_mv,
fq: -swatchColors_string_mv:({* TO RED} {RED TO *}),
I got the following facets:
facet_fields: {
swatchColors_string_mv: [
RED,
122,
BLACK,
0,
BLUE,
0,
BROWN,
0,
GREEN,
0,
so, it works for me at least...
On Sun, Sep 28, 2014 at 12:54 AM, White, Bill bwh...@ptfs.com wrote:
Hmm. If I understand correctly this builds a set out of open intervals
(exclusive ranges), that's a great idea!
It doesn't seem to work for me, though; fq=-color:({* TO red} {red TO
*})
is giving me results with color=burnt sienna
The field is defined as field name=color type=string indexed=true
stored=true multiValued=true /
On Sat, Sep 27, 2014 at 4:43 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
indeed!
the exclusive range {green TO red} matches to the lemon yellow
hence, the negation suppresses it from appearing
fq=-color:{green TO red}
then you need to suppress eg black and white also
fq=-color:({* TO green} {green TO red} {red TO *})
I have no control over the
possible values of 'color',
You don't need to control possible values, you just suppressing any
values
beside of the given green and red.
Mind that either green or red passes that negation of exclusive ranges
disjunction.
On Sun, Sep 28, 2014 at 12:15 AM, White, Bill bwh...@ptfs.com wrote:
OK, let me try phrasing it better.
How do I exclude from search, any result which contains any value for
multivalued field 'color' which is not within a given constraint
set
(e.g., red, green, yellow, burnt sienna), given that I do not
what
any of the other possible values of 'color' are?
In pseudocode:
for all x in result.color
if x not in (red,green,yellow, burnt sienna)
filter out result
I don't see how range queries would work since I have no control over
the
possible values of 'color', e.g., there could be a valid color lemon
yellow between green and red, and I don't want a result which
has
(color: red, color: lemon yellow)
On Sat, Sep 27, 2014 at 4:02 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
On Sat, Sep 27, 2014 at 11:36 PM, White, Bill bwh...@ptfs.com
wrote:
but do NOT match ANY other color.
Bill, I miss the whole picture, it's worth to rephrase the problem
in
one
sentence.
But regarding the quote above, you can try to use exclusive ranges
https://lucene.apache.org/core/4_6_0/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Range_Searches
fq=-color:({* TO green} {green TO red} {red TO *})
just don't forget to build ranges alphabetically
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
Re: fq syntax for requiring all multiValued field values to be within a list?
http://wiki.apache.org/solr/SchemaXml#Default_query_parser_operator ?
once again, debugQuery=true perfectly explains what's going on with q.
On Sun, Sep 28, 2014 at 1:24 AM, White, Bill bwh...@ptfs.com wrote:
It worked for me once I changed to
-color:({* TO red} OR {red TO *})
I'm not sure why the OR is needed, maybe it's my version? (4.6.1)
On Sat, Sep 27, 2014 at 5:22 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
hm. try to convert it to query q=-color:({* TO red} {red TO *}) and check
the explanation from debugQuery=true
I tried to play with my index
q: *:*,
facet.field: swatchColors_string_mv,
fq: -swatchColors_string_mv:({* TO RED} {RED TO *}),
I got the following facets:
facet_fields: {
swatchColors_string_mv: [
RED,
122,
BLACK,
0,
BLUE,
0,
BROWN,
0,
GREEN,
0,
so, it works for me at least...
On Sun, Sep 28, 2014 at 12:54 AM, White, Bill bwh...@ptfs.com wrote:
Hmm. If I understand correctly this builds a set out of open intervals
(exclusive ranges), that's a great idea!
It doesn't seem to work for me, though; fq=-color:({* TO red} {red TO
*})
is giving me results with color=burnt sienna
The field is defined as field name=color type=string
indexed=true
stored=true multiValued=true /
On Sat, Sep 27, 2014 at 4:43 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
indeed!
the exclusive range {green TO red} matches to the lemon yellow
hence, the negation suppresses it from appearing
fq=-color:{green TO red}
then you need to suppress eg black and white also
fq=-color:({* TO green} {green TO red} {red TO *})
I have no control over the
possible values of 'color',
You don't need to control possible values, you just suppressing any
values
beside of the given green and red.
Mind that either green or red passes that negation of exclusive
ranges
disjunction.
On Sun, Sep 28, 2014 at 12:15 AM, White, Bill bwh...@ptfs.com
wrote:
OK, let me try phrasing it better.
How do I exclude from search, any result which contains any value
for
multivalued field 'color' which is not within a given constraint
set
(e.g., red, green, yellow, burnt sienna), given that I do
not
what
any of the other possible values of 'color' are?
In pseudocode:
for all x in result.color
if x not in (red,green,yellow, burnt sienna)
filter out result
I don't see how range queries would work since I have no control
over
the
possible values of 'color', e.g., there could be a valid color
lemon
yellow between green and red, and I don't want a result which
has
(color: red, color: lemon yellow)
On Sat, Sep 27, 2014 at 4:02 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
On Sat, Sep 27, 2014 at 11:36 PM, White, Bill bwh...@ptfs.com
wrote:
but do NOT match ANY other color.
Bill, I miss the whole picture, it's worth to rephrase the
problem
in
one
sentence.
But regarding the quote above, you can try to use exclusive
ranges
https://lucene.apache.org/core/4_6_0/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Range_Searches
fq=-color:({* TO green} {green TO red} {red TO *})
just don't forget to build ranges alphabetically
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
Re: fq syntax for requiring all multiValued field values to be within a list?
Yes, that was it, thank you!
On Sat, Sep 27, 2014 at 5:28 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
http://wiki.apache.org/solr/SchemaXml#Default_query_parser_operator ?
once again, debugQuery=true perfectly explains what's going on with q.
On Sun, Sep 28, 2014 at 1:24 AM, White, Bill bwh...@ptfs.com wrote:
It worked for me once I changed to
-color:({* TO red} OR {red TO *})
I'm not sure why the OR is needed, maybe it's my version? (4.6.1)
On Sat, Sep 27, 2014 at 5:22 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
hm. try to convert it to query q=-color:({* TO red} {red TO *}) and
check
the explanation from debugQuery=true
I tried to play with my index
q: *:*,
facet.field: swatchColors_string_mv,
fq: -swatchColors_string_mv:({* TO RED} {RED TO *}),
I got the following facets:
facet_fields: {
swatchColors_string_mv: [
RED,
122,
BLACK,
0,
BLUE,
0,
BROWN,
0,
GREEN,
0,
so, it works for me at least...
On Sun, Sep 28, 2014 at 12:54 AM, White, Bill bwh...@ptfs.com wrote:
Hmm. If I understand correctly this builds a set out of open
intervals
(exclusive ranges), that's a great idea!
It doesn't seem to work for me, though; fq=-color:({* TO red} {red
TO
*})
is giving me results with color=burnt sienna
The field is defined as field name=color type=string
indexed=true
stored=true multiValued=true /
On Sat, Sep 27, 2014 at 4:43 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
indeed!
the exclusive range {green TO red} matches to the lemon yellow
hence, the negation suppresses it from appearing
fq=-color:{green TO red}
then you need to suppress eg black and white also
fq=-color:({* TO green} {green TO red} {red TO *})
I have no control over the
possible values of 'color',
You don't need to control possible values, you just suppressing any
values
beside of the given green and red.
Mind that either green or red passes that negation of exclusive
ranges
disjunction.
On Sun, Sep 28, 2014 at 12:15 AM, White, Bill bwh...@ptfs.com
wrote:
OK, let me try phrasing it better.
How do I exclude from search, any result which contains any value
for
multivalued field 'color' which is not within a given constraint
set
(e.g., red, green, yellow, burnt sienna), given that I do
not
what
any of the other possible values of 'color' are?
In pseudocode:
for all x in result.color
if x not in (red,green,yellow, burnt sienna)
filter out result
I don't see how range queries would work since I have no control
over
the
possible values of 'color', e.g., there could be a valid color
lemon
yellow between green and red, and I don't want a result
which
has
(color: red, color: lemon yellow)
On Sat, Sep 27, 2014 at 4:02 PM, Mikhail Khludnev
mkhlud...@griddynamics.com wrote:
On Sat, Sep 27, 2014 at 11:36 PM, White, Bill bwh...@ptfs.com
wrote:
but do NOT match ANY other color.
Bill, I miss the whole picture, it's worth to rephrase the
problem
in
one
sentence.
But regarding the quote above, you can try to use exclusive
ranges
https://lucene.apache.org/core/4_6_0/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Range_Searches
fq=-color:({* TO green} {green TO red} {red TO *})
just don't forget to build ranges alphabetically
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
SolrCould read-only replicas
Hi- I have been reading up on SolrCloud and it seems that it is not possible to have a cross-datacenter read-only slave anymore but wanted to ask here to be sure. We currently have a pre Solr 4.0 installation with the master instance in our US mid-west datacenter. The datacenter in Europe has read-replicas which pull data using solr.ReplicationHandler. We wanted to upgrade to SolrCloud. As far as I have been able to figure out, with SolrCloud you cannot have a read-only replica anymore. A replica has to be able to become a leader and writes against all replicas for a shard have to succeed. Because of the a strong consistency model across replicas, it seems that replicas cannot be across datacenters anymore. So my question is, how can we have a read-ony replica in a remote datacenter in Solr 4.0+ similar to pre Solr 4.0? Is it not possible anymore without doing it all yourself? cheers, Tikoo
Re: SolrCould read-only replicas
Well, replicas _can_ cross data centers, there's nothing explicitly preventing it. But the network latency will cause interesting problems, not to mention ZooKeeper Quorum sensing being challenging. But your description is accurate, there's no such thing as a read only replica in SolrCloud. That said, if you're just looking at doing the same thing you're doing pre Solr 4.0, then you can just go ahead and set it up like your older version; replication still works just like it did before, and it's not going away. In fact, it's used in SolrCloud to handle replica syncing under certain circumstances. SolrCloud is an alternate model that overcomes some of the pain points of managing (particularly sharded) installations that have things like NRT requirements etc. But there's nothing in Solr 4.x that _requires_ that you operate in cloud mode, you can use master/slave just like 3.x for instance if it suits your problem better. Best, Erick On Sat, Sep 27, 2014 at 7:42 PM, Sandeep Tikoo sti...@digitalriver.com wrote: Hi- I have been reading up on SolrCloud and it seems that it is not possible to have a cross-datacenter read-only slave anymore but wanted to ask here to be sure. We currently have a pre Solr 4.0 installation with the master instance in our US mid-west datacenter. The datacenter in Europe has read-replicas which pull data using solr.ReplicationHandler. We wanted to upgrade to SolrCloud. As far as I have been able to figure out, with SolrCloud you cannot have a read-only replica anymore. A replica has to be able to become a leader and writes against all replicas for a shard have to succeed. Because of the a strong consistency model across replicas, it seems that replicas cannot be across datacenters anymore. So my question is, how can we have a read-ony replica in a remote datacenter in Solr 4.0+ similar to pre Solr 4.0? Is it not possible anymore without doing it all yourself? cheers, Tikoo
Re: demo app explaining solr features
On Sat, Sep 27, 2014 at 12:26 PM, Anurag Sharma anura...@gmail.com wrote: I am wondering if there is any demo app that can demonstrate all the features/capabilities of solr. My intention is to understand, use and play around all the features supported by solr. https://lucene.apache.org/solr/4_10_0/tutorial.html -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
