Re: SOLR 5.4.0?

[Ere Maijala]

Well, for us SOLR-8418 is a major issue. I haven't encountered other 
issues, but that one was sort of a show-stopper.


--Ere

31.12.2015, 7.27, William Bell kirjoitti:

How is SOLR 5.4.0 ? I heard there was a quick 5.4.1 coming out?

Any major issues?



--
Ere Maijala
Kansalliskirjasto / The National Library of Finland

Re: Having replica will slow down Solr?

[Zheng Lin Edwin Yeo]

Hi DavidPhilip,

What are your memory configs: xms and xmx values, available ram size?
> I'm using Solr 5.3.0, so I've only set the -Xmx value at 14g. My
available RAM size is 64GB

Were you also indexing at the time when the search response was so slow? In
the sense, were you committing often with openSearcher=true, in any case?
> I wasn't doing any indexing at that time the search response was so slow.

What is the QTime for consequent searches? Are the caches enabled?
> The QTime for consequent searches are quite fast, usually less than
500ms. The caches are enabled.

Regards,
Edwin


On 30 December 2015 at 16:40, davidphilip cherian <
davidphilipcher...@gmail.com> wrote:

> Few questions:
>
> What are your memory configs: xms and xmx values, available ram size?
>
> Were you also indexing at the time when the search response was so slow? In
> the sense, were you committing often with openSearcher=true, in any case?
>
> What is the QTime for consequent searches? Are the caches enabled?
>
>
>
>
>
>
> On Wed, Dec 30, 2015 at 1:50 PM, Binoy Dalal 
> wrote:
>
> > If I understand the cloud setup correctly having a replica will serve for
> > redundancy and won't slow your searches down.
> > In your case for the slow response times, are you returning any large
> > stored fields?
> >
> > On Wed, 30 Dec 2015, 13:26 Zheng Lin Edwin Yeo 
> > wrote:
> >
> > > Hi,
> > >
> > > I would like to find out, will having a replica slow down the search
> for
> > > Solr?
> > >
> > > Currently, I'm having 1 shard and a replicationFactor of 2 using Solr
> > > 5.3.0. I'm running SolrCloud, with 3 external ZooKeeper using ZooKeeper
> > > 3.4.6, and my index size is 183GB.
> > >
> > > I have been getting QTime of more than 3000ms for my basic search
> > function,
> > > even without adding other things like faceting or highlighting.
> > >
> > > Regards,
> > > Edwin
> > >
> > --
> > Regards,
> > Binoy Dalal
> >
>

Re: Having replica will slow down Solr?

[Binoy Dalal]

1) If your stored fields are very large, it will lead to high response
times since solr will have to decompress the field and then return the
whole thing. I had faced such an issue with one of my implementations where
we had stored fields around 100KB of text per document and the response
times were through the roof. To get around this, we simply highlighted the
stored field being returned and returned the highlights. That got us to
sub-100ms response times. See if this is something you can do.
2) Your subsequent searches will be quicker since your caches are warmed up
and some of your queries then return from the cache itself. For the initial
stages, you should warm up your cache before a new searcher opens by using
either the autowarming feature or explicitly using warming queries to
populate the cache. This should give you some information:
https://wiki.apache.org/solr/SolrCaching

On Thu, Dec 31, 2015 at 8:59 AM Zheng Lin Edwin Yeo 
wrote:

> Hi DavidPhilip,
>
> What are your memory configs: xms and xmx values, available ram size?
> > I'm using Solr 5.3.0, so I've only set the -Xmx value at 14g. My
> available RAM size is 64GB
>
> Were you also indexing at the time when the search response was so slow? In
> the sense, were you committing often with openSearcher=true, in any case?
> > I wasn't doing any indexing at that time the search response was so slow.
>
> What is the QTime for consequent searches? Are the caches enabled?
> > The QTime for consequent searches are quite fast, usually less than
> 500ms. The caches are enabled.
>
> Regards,
> Edwin
>
>
> On 30 December 2015 at 16:40, davidphilip cherian <
> davidphilipcher...@gmail.com> wrote:
>
> > Few questions:
> >
> > What are your memory configs: xms and xmx values, available ram size?
> >
> > Were you also indexing at the time when the search response was so slow?
> In
> > the sense, were you committing often with openSearcher=true, in any case?
> >
> > What is the QTime for consequent searches? Are the caches enabled?
> >
> >
> >
> >
> >
> >
> > On Wed, Dec 30, 2015 at 1:50 PM, Binoy Dalal 
> > wrote:
> >
> > > If I understand the cloud setup correctly having a replica will serve
> for
> > > redundancy and won't slow your searches down.
> > > In your case for the slow response times, are you returning any large
> > > stored fields?
> > >
> > > On Wed, 30 Dec 2015, 13:26 Zheng Lin Edwin Yeo 
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I would like to find out, will having a replica slow down the search
> > for
> > > > Solr?
> > > >
> > > > Currently, I'm having 1 shard and a replicationFactor of 2 using Solr
> > > > 5.3.0. I'm running SolrCloud, with 3 external ZooKeeper using
> ZooKeeper
> > > > 3.4.6, and my index size is 183GB.
> > > >
> > > > I have been getting QTime of more than 3000ms for my basic search
> > > function,
> > > > even without adding other things like faceting or highlighting.
> > > >
> > > > Regards,
> > > > Edwin
> > > >
> > > --
> > > Regards,
> > > Binoy Dalal
> > >
> >
>
-- 
Regards,
Binoy Dalal

Re: Stable Versions in Solr 4

[Shyam R]

I will always look around here for versions / new functionality or fixes /
release notes

https://issues.apache.org/jira/browse/SOLR/?selectedTab=com.atlassian.jira.jira-projects-plugin:changelog-panel

Thanks

On Thu, Dec 31, 2015 at 4:05 AM, Shawn Heisey  wrote:

> On 12/28/2015 5:12 AM, abhi Abhishek wrote:
> >i am trying to determine stable version of SOLR 4. is there a blog
> which
> > we can refer.. i understand we can read through Release Notes. I am
> > interested in user reviews and challenges seen with various versions of
> > SOLR 4.
>
> Here's some information about Solr version numbers, with X.Y.Z providing
> the legend:  X is the major version number.  Major versions are released
> very infrequently.  Y tracks the minor version number.  Minor releases
> are made quite frequently.  Z is incremented with bugfix releases.  Most
> of the time, the third number in the version is zero.
>
> Every release of Solr that you can download from the official mirror
> network is built from a version control branch that is known as the
> stable branch.  Currently that is branch_5x, at some point in the future
> it will be branch_6x.
>
> The goal of the stable branch is to always be in a state where a viable
> release candidate could be created.  That's why it's called the stable
> branch.  If all of the tests in the included test suite are passing,
> that's a good sign that there are no major problems.  It's no guarantee,
> just a good sign.
>
> All releases have bugs, but unless those bugs are very nasty, they do
> not get fixed until the next minor version.  When the bugs are
> particularly bad, there might be a bugfix release.
>
> It sounds like you're trying to decide which release you should use.
> The answer to that question is usually very easy -- the latest version,
> which is currently 5.4.0.  Right after a new release happens, the best
> choice might be the newest bugfix release of the previous minor version.
>
> The pace of development is very high in Solr.  Each new minor version
> includes new features and enhancements.  The sum total of the
> differences between 4.0 and 4.10 is greater than the difference between
> 4.10 and 5.0.
>
> I would not recommend using a 4.x release at this time.  The 4.x line
> went into maintenance mode ten months ago with the release of 5.0.  The
> community is now focused on 5.x versions.  If you mention a problem with
> a 4.x version now, the first thing you'll be told is that you need to
> upgrade, because unless the bug you're experiencing is a showstopper
> that affects a wide variety of users, it will not be fixed in 4.x.  If
> it is a major bug that affects a large number of users, it will only be
> fixed a version like 4.10.5 -- a bugfix release on the last minor 4.x
> version.
>
> Thanks,
> Shawn
>



-- 
Ph: 9845704792

Re: Testing Solr configuration, schema, and other fields

[Alexandre Rafalovitch]

I might be just confused here, but I am not sure what your bottle neck
actually is. You seem to know your critical path already, so how can we
help?

Starting new solr core from given configuration directory is easy. Catching
hard errors from that is probably just gripping logs or a custom logger.

And you don't seem to be talking about lint style soft sanity checks, but
rather the initialization stopping hard checks.

What is the next step you are stuck on?

Regards,
   Alex
On 31 Dec 2015 3:09 am, "Davis, Daniel (NIH/NLM) [C]" 
wrote:

> At my organization, I want to create a tool that allows users to keep a
> solr configuration as a Git repository.   Then, I want my Continuous
> Integration environment to take some branch of the git repository and
> "publish" it into ZooKeeper/SolrCloud.
>
> Working on my own, it is only a very small pain to note foolish errors
> I've made, fix them, and restart.However, I want my users to be able to
> edit their own Solr schema and config *most* of the time, at least on
> development servers.They will not have command-line access to these
> servers, and I want to avoid endless restarts.
>
> I'm not interested in fighting to maintain such a useless thing as a
> DTD/XSD without community support; what I really want to know is whether
> Solr will start and can index some sample documents.   I'm wondering
> whether I might be able to build a tool to fire up an EmbeddedSolrServer
> and capture error messages/exceptions in a reasonable way. This tool
> could then be run by my users before they commit to git, and then again by
> the CI server before it "publishes" the configuration to
> ZooKeeper/SolrCloud.
>
> Any suggestions?
>
> Dan Davis, Systems/Applications Architect (Contractor),
> Office of Computer and Communications Systems,
> National Library of Medicine, NIH
>
>

SOLR 5.4.0?

[William Bell]

How is SOLR 5.4.0 ? I heard there was a quick 5.4.1 coming out?

Any major issues?

-- 
Bill Bell
billnb...@gmail.com
cell 720-256-8076

Using post.jr for indexing in Solr 5.4.0

[Zheng Lin Edwin Yeo]

Hi,

Would like to find out, is there any changes to the post.jar for Solr 5.4?
I tried to use the post.jar from Solr 5.3.0, and it gives me the following
error during indexing:

java.lang.NoSuchMethodError:
org.apache.solr.h
andler.extraction.ExtractingDocumentLoader.init(Lorg/apache/solr/request
/SolrQueryRequest;Lorg/apache/solr/update/processor/UpdateRequestProcessor;Lorg/
apache/tika/config/TikaConfig;Lorg/apache/solr/handler/extraction/ParseContextCo
nfig;Lorg/apache/solr/handler/extraction/SolrContentHandlerFactory;)Vjava.lang.RuntimeException: java.lang.NoSuchMethodError:
org.apache
.solr.handler.extraction.ExtractingDocumentLoader.init(Lorg/apache/solr/
request/SolrQueryRequest;Lorg/apache/solr/update/processor/UpdateRequestProcesso
r;Lorg/apache/tika/config/TikaConfig;Lorg/apache/solr/handler/extraction/ParseCo
ntextConfig;Lorg/apache/solr/handler/extraction/SolrContentHandlerFactory;)V
at
org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:611)

at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:472)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte
r.java:222)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte
r.java:181)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1652)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java
:585)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j
ava:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.jav
a:577)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandl
er.java:223)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandl
er.java:1127)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:
515)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandle
r.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandle
r.java:1061)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j
ava:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(Cont
extHandlerCollection.java:215)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerColl
ection.java:110)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper
.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.jav
a:257)
at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java
:540)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPoo
l.java:635)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool
.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoSuchMethodError:
org.apache.solr.handler.extraction.Extra
ctingDocumentLoader.init(Lorg/apache/solr/request/SolrQueryRequest;Lorg/
apache/solr/update/processor/UpdateRequestProcessor;Lorg/apache/tika/config/Tika
Config;Lorg/apache/solr/handler/extraction/ParseContextConfig;Lorg/apache/solr/h
andler/extraction/SolrContentHandlerFactory;)V
at
org.apache.solr.handler.extraction.ExtractingRequestHandler.newLoader
(ExtractingRequestHandler.java:130)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Co
ntentStreamHandlerBase.java:59)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandl
erBase.java:156)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2073)
at
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:658)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:457)
... 22 more
500


Regards,
Edwin

Re: Teiid with Solr - using any other engine except the SolrDefaultQueryEngine

[Alexandre Rafalovitch]

Are you trying to do federated search? What about carrot? Not the one that
ships with Solr, the parent project.

Regards,
   Alex
On 31 Dec 2015 12:21 am, "Mark Horninger" 
wrote:

> I have gotten Teiid and Solr wired up, but it seems like the only way to
> query is with the default Solr Query Engine, and nothing else.  In asking
> Dr. Google, this is a data black hole.  The more I look at it, the more I
> think I'm going to end up having to write a custom translator.  Is there
> anyone else out there who has had this challenge, and if so, how did you
> overcome it?
>
> Thanks In Advance!
>
> -Mark H.
>
>
> [GrayHair]
> GHS Confidentiality Notice
>
> This e-mail message, including any attachments, is for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized review, use, disclosure or distribution of
> this information is prohibited, and may be punishable by law. If this was
> sent to you in error, please notify the sender by reply e-mail and destroy
> all copies of the original message.
>
> GrayHair Software 
>
>

Re: multi term analyzer error

[Erick Erickson]

Right, you may be one of the few people to actually implement your own
multiTerm analyzer function
despite the fact that this has been in the code for years!

If you look at the factories and see if they implement the
"MultiTermAwareComponent" interface,
and PatternReplaceCharFitlerFactory does _not_. Thus it can't be used
in a multiTerm
analysis chain.

A bit of background here. The whole "MultiTermAwareComponent" was
implemented to handle
simple cases that were causing endless questions. For instance,
anything with a wildcard would
do no analysis. Thus people would define a field with, say,
LowerCaseFilterFactory and then ask
"Why don't we find 'Eric*'  when Erick is in the field?" The answer
was that "wildcard terms are not
sent through the analysis chain, you have to do those kinds of
transformations in the client." This
was not terribly satisfactory...

There are various sound reasons why "doing the right thing" with
wildcards in a filter that
breaks a single token into two or more tokens this is very hard in the
general case. Any filter
that generates two or more tokens is impossible to get right. Does
this mean both tokens should
be wildcards? The first? The second? Neither? Any decision is the
wrong decision. And don't
even get me started on something like Ngrams or Shingles.

OK, finally answering your question. The only filters that are
multi-term aware are ones that
are _guaranteed_ to produce one and only one token from any input token.
PatternReplaceCharFilterFactory cannot honor that contract so I'm
pretty sure that's what's
causing your error. Assuming the substitutions you're doing would work
on the whole string,
you might be able to use PatterhReplaceCharFilterFactory since that
operates on the whole
input string rather than the tokens and thus could be used.

But I have to ask "why are you implementing a multiTerm analyzer"?
What is the use-case you're
trying to solve? Because from your example, it looks like you're
trying to search over a string-type
(untokenized) input and if so this not the right approach at all.

Best,
Erick

On Tue, Dec 29, 2015 at 10:16 PM, Eyal  Naamati
 wrote:
> Hi Ahmet,
> Yes there is a space in my example.
> This is my multiterm analyzer:
>
> 
>  pattern="\-" replacement="\_" />
> 
> 
> 
>
> Thanks!
>
> Eyal Naamati
> Alma Developer
> Tel: +972-2-6499313
> Mobile: +972-547915255
> eyal.naam...@exlibrisgroup.com
>
> www.exlibrisgroup.com
>
> -Original Message-
> From: Ahmet Arslan [mailto:iori...@yahoo.com.INVALID]
> Sent: Tuesday, December 29, 2015 5:18 PM
> To: solr-user@lucene.apache.org
> Subject: Re: multi term analyzer error
>
> Hi Eyal,
>
> What is your analyzer definition for multi-term?
> In your example, is star charter separated from the term by a space?
>
>
> Ahmet
>
> On Tuesday, December 29, 2015 3:26 PM, Eyal Naamati 
>  wrote:
>
>
>
>
> Hi,
>
> I defined a multi-term analyzer to my analysis chain, and it works as I 
> expect. However, for some queries (for example '* or 'term *') I get an 
> exception "analyzer returned no terms for multiTerm term". These queries work 
> when I don't customize a multi-term analyzer.
> My question: is there a way to handle this in the analyzer configuration (in 
> my schema.xml)? I realize that I can also change the query I am sending the 
> analyzer, but that is difficult for me since there are many places in our 
> program that use this.
> Thanks!
>
> Eyal Naamati
> Alma Developer
> Tel: +972-2-6499313
> Mobile: +972-547915255
> eyal.naam...@exlibrisgroup.com
>
> www.exlibrisgroup.com

Re: Parameter Substitution / Macro Expansion

[Yonik Seeley]

On Wed, Dec 30, 2015 at 10:30 AM, Alessandro Benedetti
 wrote:
>  debugging I confirm you are right Yonik, but also that the defaults
> are already applied at this point :
> org/apache/solr/request/json/RequestUtil.java:158
>
> Could be a resolution of properties that happens at solrconfig, xml level ?

Ah, good call - that must be it!
I wonder if there is a way to escape it at that level (config parsing)
so that it binds later (request param substitution)?

-Yonik

Teiid with Solr - using any other engine except the SolrDefaultQueryEngine

[Mark Horninger]

I have gotten Teiid and Solr wired up, but it seems like the only way to query 
is with the default Solr Query Engine, and nothing else.  In asking Dr. Google, 
this is a data black hole.  The more I look at it, the more I think I'm going 
to end up having to write a custom translator.  Is there anyone else out there 
who has had this challenge, and if so, how did you overcome it?

Thanks In Advance!

-Mark H.


[GrayHair]
GHS Confidentiality Notice

This e-mail message, including any attachments, is for the sole use of the 
intended recipient(s) and may contain confidential and privileged information. 
Any unauthorized review, use, disclosure or distribution of this information is 
prohibited, and may be punishable by law. If this was sent to you in error, 
please notify the sender by reply e-mail and destroy all copies of the original 
message.

GrayHair Software 

Testing Solr configuration, schema, and other fields

[Davis, Daniel (NIH/NLM) [C]]

At my organization, I want to create a tool that allows users to keep a solr 
configuration as a Git repository.   Then, I want my Continuous Integration 
environment to take some branch of the git repository and "publish" it into 
ZooKeeper/SolrCloud.

Working on my own, it is only a very small pain to note foolish errors I've 
made, fix them, and restart.However, I want my users to be able to edit 
their own Solr schema and config *most* of the time, at least on development 
servers.They will not have command-line access to these servers, and I want 
to avoid endless restarts.

I'm not interested in fighting to maintain such a useless thing as a DTD/XSD 
without community support; what I really want to know is whether Solr will 
start and can index some sample documents.   I'm wondering whether I might be 
able to build a tool to fire up an EmbeddedSolrServer and capture error 
messages/exceptions in a reasonable way. This tool could then be run by my 
users before they commit to git, and then again by the CI server before it 
"publishes" the configuration to ZooKeeper/SolrCloud.

Any suggestions?

Dan Davis, Systems/Applications Architect (Contractor),
Office of Computer and Communications Systems,
National Library of Medicine, NIH

RE: Testing Solr configuration, schema, and other fields

[Davis, Daniel (NIH/NLM) [C]]

Your bottom line point is that EmbeddedSolrServer is different, and some 
configurations will not work on it where they would work on a SolrCloud.   This 
is well taken.   Maybe creating a new collection on existing dev nodes could be 
done.

As far as VDI and Puppet.   My requirements are different because my 
organization is different.   I would prefer not to go into how different.   I 
have written puppet modules for other system configurations, tested them on AWS 
EC2, and yet those modules have not been adopted by my organization.


-Original Message-
From: Mark Horninger [mailto:mhornin...@grayhairsoftware.com] 
Sent: Wednesday, December 30, 2015 3:25 PM
To: solr-user@lucene.apache.org
Subject: RE: Testing Solr configuration, schema, and other fields

Daniel,


Sounds almost like you're reinventing the wheel.  Could you possibly automate 
this through puppet or Chef?  With a VDI environment, then all you would need 
to do is build a new VM Node based on original setup.  Then you can just roll 
out the node as one of the zk nodes.

Just a thought on that subject.

v/r,

-Mark H.

-Original Message-
From: Davis, Daniel (NIH/NLM) [C] [mailto:daniel.da...@nih.gov]
Sent: Wednesday, December 30, 2015 3:10 PM
To: solr-user@lucene.apache.org
Subject: Testing Solr configuration, schema, and other fields

At my organization, I want to create a tool that allows users to keep a solr 
configuration as a Git repository.   Then, I want my Continuous Integration 
environment to take some branch of the git repository and "publish" it into 
ZooKeeper/SolrCloud.

Working on my own, it is only a very small pain to note foolish errors I've 
made, fix them, and restart.However, I want my users to be able to edit 
their own Solr schema and config *most* of the time, at least on development 
servers.They will not have command-line access to these servers, and I want 
to avoid endless restarts.

I'm not interested in fighting to maintain such a useless thing as a DTD/XSD 
without community support; what I really want to know is whether Solr will 
start and can index some sample documents.   I'm wondering whether I might be 
able to build a tool to fire up an EmbeddedSolrServer and capture error 
messages/exceptions in a reasonable way. This tool could then be run by my 
users before they commit to git, and then again by the CI server before it 
"publishes" the configuration to ZooKeeper/SolrCloud.

Any suggestions?

Dan Davis, Systems/Applications Architect (Contractor), Office of Computer and 
Communications Systems, National Library of Medicine, NIH

[GrayHair]
GHS Confidentiality Notice

This e-mail message, including any attachments, is for the sole use of the 
intended recipient(s) and may contain confidential and privileged information. 
Any unauthorized review, use, disclosure or distribution of this information is 
prohibited, and may be punishable by law. If this was sent to you in error, 
please notify the sender by reply e-mail and destroy all copies of the original 
message.

GrayHair Software 

RE: Testing Solr configuration, schema, and other fields

[Mark Horninger]

Daniel,


Sounds almost like you're reinventing the wheel.  Could you possibly automate 
this through puppet or Chef?  With a VDI environment, then all you would need 
to do is build a new VM Node based on original setup.  Then you can just roll 
out the node as one of the zk nodes.

Just a thought on that subject.

v/r,

-Mark H.

-Original Message-
From: Davis, Daniel (NIH/NLM) [C] [mailto:daniel.da...@nih.gov]
Sent: Wednesday, December 30, 2015 3:10 PM
To: solr-user@lucene.apache.org
Subject: Testing Solr configuration, schema, and other fields

At my organization, I want to create a tool that allows users to keep a solr 
configuration as a Git repository.   Then, I want my Continuous Integration 
environment to take some branch of the git repository and "publish" it into 
ZooKeeper/SolrCloud.

Working on my own, it is only a very small pain to note foolish errors I've 
made, fix them, and restart.However, I want my users to be able to edit 
their own Solr schema and config *most* of the time, at least on development 
servers.They will not have command-line access to these servers, and I want 
to avoid endless restarts.

I'm not interested in fighting to maintain such a useless thing as a DTD/XSD 
without community support; what I really want to know is whether Solr will 
start and can index some sample documents.   I'm wondering whether I might be 
able to build a tool to fire up an EmbeddedSolrServer and capture error 
messages/exceptions in a reasonable way. This tool could then be run by my 
users before they commit to git, and then again by the CI server before it 
"publishes" the configuration to ZooKeeper/SolrCloud.

Any suggestions?

Dan Davis, Systems/Applications Architect (Contractor), Office of Computer and 
Communications Systems, National Library of Medicine, NIH

[GrayHair]
GHS Confidentiality Notice

This e-mail message, including any attachments, is for the sole use of the 
intended recipient(s) and may contain confidential and privileged information. 
Any unauthorized review, use, disclosure or distribution of this information is 
prohibited, and may be punishable by law. If this was sent to you in error, 
please notify the sender by reply e-mail and destroy all copies of the original 
message.

GrayHair Software 

RE: Testing Solr configuration, schema, and other fields

[Davis, Daniel (NIH/NLM) [C]]

I think of enterprise search as very similar to RDBMS:

- It belongs in the backend behind your app.
- Each project ought to control its own schema and data.

So, I want the configset for each team's Solr collections to be stored in our 
Git server just as the RDBMS schema is if a developer is using a framework or a 
couple of SQL files, scripts, and a VERSION table.It ought to be that easy.


-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com] 
Sent: Wednesday, December 30, 2015 5:37 PM
To: solr-user 
Subject: Re: Testing Solr configuration, schema, and other fields

Yeah, the notion of DTDs have gone around several times but always founder on 
the fact that you can, say, define your own Filter with it's own set of 
parameters etc. Sure, you can make a generic DTD that accommodates this, but 
then it becomes so general as to be little more than a syntax checker.

The managed schema stuff allows modifications of the schema via REST calls and 
there is some equivalent functionality for solrconfig.xml, but the interesting 
bit about that is that then your VCS is not the "one true source" of the 
configs, it almost goes backwards: Modify the configs in Zookeeper then check 
in to Git.
And even that doesn't really solve, say, putting default search fields in 
solrconfig.xml that do not exist in the schema file.

Frankly what I usually do when heavily editing either one is just do it on my 
local laptop, either stand alone or SolrCloud, _then_ check it in and/or test 
it on my cloud setup. So I guess the take-away is that I don't have any very 
good solution here.

Best,
Erick


On Wed, Dec 30, 2015 at 1:10 PM, Davis, Daniel (NIH/NLM) [C] 
 wrote:
> Your bottom line point is that EmbeddedSolrServer is different, and some 
> configurations will not work on it where they would work on a SolrCloud.   
> This is well taken.   Maybe creating a new collection on existing dev nodes 
> could be done.
>
> As far as VDI and Puppet.   My requirements are different because my 
> organization is different.   I would prefer not to go into how different.   I 
> have written puppet modules for other system configurations, tested them on 
> AWS EC2, and yet those modules have not been adopted by my organization.
>
>
> -Original Message-
> From: Mark Horninger [mailto:mhornin...@grayhairsoftware.com]
> Sent: Wednesday, December 30, 2015 3:25 PM
> To: solr-user@lucene.apache.org
> Subject: RE: Testing Solr configuration, schema, and other fields
>
> Daniel,
>
>
> Sounds almost like you're reinventing the wheel.  Could you possibly automate 
> this through puppet or Chef?  With a VDI environment, then all you would need 
> to do is build a new VM Node based on original setup.  Then you can just roll 
> out the node as one of the zk nodes.
>
> Just a thought on that subject.
>
> v/r,
>
> -Mark H.
>
> -Original Message-
> From: Davis, Daniel (NIH/NLM) [C] [mailto:daniel.da...@nih.gov]
> Sent: Wednesday, December 30, 2015 3:10 PM
> To: solr-user@lucene.apache.org
> Subject: Testing Solr configuration, schema, and other fields
>
> At my organization, I want to create a tool that allows users to keep a solr 
> configuration as a Git repository.   Then, I want my Continuous Integration 
> environment to take some branch of the git repository and "publish" it into 
> ZooKeeper/SolrCloud.
>
> Working on my own, it is only a very small pain to note foolish errors I've 
> made, fix them, and restart.However, I want my users to be able to edit 
> their own Solr schema and config *most* of the time, at least on development 
> servers.They will not have command-line access to these servers, and I 
> want to avoid endless restarts.
>
> I'm not interested in fighting to maintain such a useless thing as a DTD/XSD 
> without community support; what I really want to know is whether Solr will 
> start and can index some sample documents.   I'm wondering whether I might be 
> able to build a tool to fire up an EmbeddedSolrServer and capture error 
> messages/exceptions in a reasonable way. This tool could then be run by 
> my users before they commit to git, and then again by the CI server before it 
> "publishes" the configuration to ZooKeeper/SolrCloud.
>
> Any suggestions?
>
> Dan Davis, Systems/Applications Architect (Contractor), Office of 
> Computer and Communications Systems, National Library of Medicine, NIH
>
> [GrayHair]
> GHS Confidentiality Notice
>
> This e-mail message, including any attachments, is for the sole use of the 
> intended recipient(s) and may contain confidential and privileged 
> information. Any unauthorized review, use, disclosure or distribution of this 
> information is prohibited, and may be punishable by law. If this was sent to 
> you in error, please notify the sender by reply e-mail and destroy all copies 
> of the original message.
>
> GrayHair Software 
>

Re: Issue with Join

[William Bell]

Yeah I ran it. It appears to only have an issue if the field for the core
on the outside is MultiValue, and the inside field is single value, and the
order of the field has the code not as the first entry. Like:

 practicing_specialties_codes:
>  [
> - "PS1010",
> - "PS282",
> - "*PS1226*"
> ]
>  }

Here is the output of the debug:


   - join:
   {
  - {!join from=prac_spec_code to=practicing_specialties_codes
  fromIndex=autosuggest}*:*:
  {
 - time: 24,
 - fromSetSize: 170179,
 - toSetSize: 3194542,
 - fromTermCount: 1091,
 - fromTermTotalDf: 1091,
 - fromTermDirectCount: 1091,
 - fromTermHits: 1091,
 - fromTermHitsTotalDf: 1091,
 - toTermHits: 1069,
 - toTermHitsTotalDf: 4002538,
 - toTermDirectCount: 697,
 - smallSetsDeferred: 1,
 - toSetDocsAdded: 4002538
 }
  },


On Wed, Dec 30, 2015 at 2:02 AM, Mikhail Khludnev <
mkhlud...@griddynamics.com> wrote:

> Hello,
>
> Have you considered to pass NOT {!join from=prac_spec_code
> to=practicing_specialties_codes
>  fromIndex=autosuggest}*:* as a q and look at debug output.
> Also there was a trick for dealing with pure negative queries q=*:*
> -foo:bar though it might be obsolete.
>
> On Tue, Dec 29, 2015 at 7:02 AM, William Bell  wrote:
>
> > I have having issues with {!join}. If the core have multiValued field and
> > the inner join does not have a multiValued field it does not find the
> > ones...
> >
> > Solr 5.3.1... 5.3.1
> >
> > Example.
> >
> > PS1226 is in practicing_specialties_codes in providersearch core. This
> > field is multiValued.
> >
> > in the autosuggest core there is NOT a field for PS1226 in there. This
> > field is called prac_spec_code and is single values.
> >
> >
> >
> >
> http://localhost:8983/solr/providersearch/select?q=*%3A*=json=true=practicing_specialties_codes:PS1226=practicing_specialties_codes
> >
> > I get:
> >
> >
> >- docs:
> >[
> >   -
> >   {
> >  - practicing_specialties_codes:
> >  [
> > - "PS1010",
> > - "PS282",
> > - "PS1226"
> > ]
> >  }
> >   ]
> >
> >
> >
> > In autosuggest there is nothing:
> >
> >
> >
> http://localhost:8983/solr/autosuggest/select?q=*%3A*=json=true=prac_spec_code:PS1226=prac_spec_code
> >
> > Nothing.
> >
> > Then a join should find what is in providersearch but missing in
> > autosuggest.
> >
> >
> >
> http://localhost:8983/solr/providersearch/select?debugQuery=true=json=*:*=10=practicing_specialties_codes:PS1226=practicing_specialties_codes=NOT%20{!join%20from=prac_spec_code%20to=practicing_specialties_codes%20fromIndex=autosuggest}auto_type:PRACSPEC
> >
> > or
> >
> >
> >
> http://hgsolr2sl1:8983/solr/providersearch/select?debugQuery=true=json=*:*=10=practicing_specialties_codes=NOT%20{!join%20from=prac_spec_code%20to=practicing_specialties_codes%20fromIndex=autosuggest}auto_type:PRACSPEC
> >
> > or
> >
> >
> >
> http://hgsolr2sl1:8983/solr/providersearch/select?debugQuery=true=json=*:*=10=practicing_specialties_codes=NOT%20{!join%20from=prac_spec_code%20to=practicing_specialties_codes%20fromIndex=autosuggest}*:*
> >
> > I also tried *:* AND NOT {!join}
> >
> > I get 0 results. This seems to be a bug.
> >
> > {
> >
> >- responseHeader:
> >{
> >   - status: 0,
> >   - QTime: 178,
> >   - params:
> >   {
> >  - q: "*:*",
> >  - fl: "practicing_specialties_codes",
> >  - fq: "NOT {!join from=prac_spec_code
> >  to=practicing_specialties_codes fromIndex=autosuggest}*:*",
> >  - rows: "10",
> >  - wt: "json",
> >  - debugQuery: "true"
> >  }
> >   },
> >- response:
> >{
> >   - numFound: 0,
> >   - start: 0,
> >   - docs: [ ]
> >   },
> >- debug:
> >{
> >   - rawquerystring: "*:*",
> >   - querystring: "*:*",
> >   - parsedquery: "MatchAllDocsQuery(*:*)",
> >   - parsedquery_toString: "*:*",
> >   - explain: { },
> >   - QParser: "LuceneQParser",
> >   - filter_queries:
> >   [
> >  - "NOT {!join from=prac_spec_code
> to=practicing_specialties_codes
> >  fromIndex=autosuggest}*:*"
> >  ],
> >   - parsed_filter_queries:
> >   [
> >  - "-JoinQuery({!join from=prac_spec_code
> >  to=practicing_specialties_codes fromIndex=autosuggest}*:*)"
> >  ],
> >   - timing:
> >   {
> >  - time: 177,
> >  - prepare:
> >  {
> > - time: 0,
> > - query:
> > {
> >- time: 0
> >},
> > - facet:
> > {
> >- time: 0
> >},
> > - facet_module:
> > {
> >- time: 0
> >},
> > - mlt:
> >   

Error importing data - java.util.concurrent.RejectedExecutionException

[Jeff Chastain]

I will preface this with the fact that I am still pretty new to both Solr and 
Tomcat, so hopefully this is something obvious to somebody out there.  I have 
two 4.3.10 Solr servers set up in separate contexts, running on a Tomcat 7 
application server on Windows 2012.  When I attempt to import data from a SQL 
server into a collection on one of the Solr instances, no documents are created 
and the log files when run at full debug level show the following .

--

DEBUG - 2015-12-30 13:24:53.469; 
org.apache.solr.update.processor.LogUpdateProcessor; PRE_UPDATE add{,id=216885} 
{{params(optimize=true=true=true=true=false=full-import=false=json),defaults(config=db-data-config.xml)}}
WARN  - 2015-12-30 13:24:53.469; org.apache.solr.handler.dataimport.SolrWriter; 
Error creating document : SolrInputDocument(fields: [memberId=**, 
location=**,**, longitude=**, lastName=**, status=**, 
latitude=**, id=**, firstName=**, _version_=1522019276914950145])
org.apache.solr.common.SolrException: Exception writing document id ** to 
the index; possible analysis error.
 at 
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:168)
 at 
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
 at 
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
 at 
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:926)
 at 
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1080)
 at 
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:692)
 at 
org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
 at org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:71)
 at 
org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:265)
 at 
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:511)
 at 
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:415)
 at 
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:330)
 at 
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
 at 
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
 at 
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
 at 
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461)
Caused by: java.util.concurrent.RejectedExecutionException: Task 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@15afacfe 
rejected from 
java.util.concurrent.ScheduledThreadPoolExecutor@132e86ed[Terminated, pool size 
= 0, active threads = 0, queued tasks = 0, completed tasks = 1]
 at 
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(Unknown 
Source)
 at java.util.concurrent.ThreadPoolExecutor.reject(Unknown Source)
 at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(Unknown 
Source)
 at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(Unknown 
Source)
 at 
org.apache.solr.update.CommitTracker._scheduleCommitWithin(CommitTracker.java:150)
 at 
org.apache.solr.update.CommitTracker._scheduleCommitWithinIfNeeded(CommitTracker.java:118)
 at 
org.apache.solr.update.CommitTracker.addedDocument(CommitTracker.java:169)
 at 
org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:275)
 at 
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:164)
 ... 15 more

--

I am not sure where to even start looking here, but the server appears to be 
running fine with ample CPU and memory room.  I have doubled the RAM available 
to the Tomcat server (1024 on start, 4096 for the max).

On the Solr side, I have checked the data shown against the schema for the 
collection and everything appears to line up.

I am at a loss here ... can anybody offer a pointer?

Thanks,
-- Jeff

Re: Stable Versions in Solr 4

[Shawn Heisey]

On 12/28/2015 5:12 AM, abhi Abhishek wrote:
>i am trying to determine stable version of SOLR 4. is there a blog which
> we can refer.. i understand we can read through Release Notes. I am
> interested in user reviews and challenges seen with various versions of
> SOLR 4.

Here's some information about Solr version numbers, with X.Y.Z providing
the legend:  X is the major version number.  Major versions are released
very infrequently.  Y tracks the minor version number.  Minor releases
are made quite frequently.  Z is incremented with bugfix releases.  Most
of the time, the third number in the version is zero.

Every release of Solr that you can download from the official mirror
network is built from a version control branch that is known as the
stable branch.  Currently that is branch_5x, at some point in the future
it will be branch_6x.

The goal of the stable branch is to always be in a state where a viable
release candidate could be created.  That's why it's called the stable
branch.  If all of the tests in the included test suite are passing,
that's a good sign that there are no major problems.  It's no guarantee,
just a good sign.

All releases have bugs, but unless those bugs are very nasty, they do
not get fixed until the next minor version.  When the bugs are
particularly bad, there might be a bugfix release.

It sounds like you're trying to decide which release you should use.
The answer to that question is usually very easy -- the latest version,
which is currently 5.4.0.  Right after a new release happens, the best
choice might be the newest bugfix release of the previous minor version.

The pace of development is very high in Solr.  Each new minor version
includes new features and enhancements.  The sum total of the
differences between 4.0 and 4.10 is greater than the difference between
4.10 and 5.0.

I would not recommend using a 4.x release at this time.  The 4.x line
went into maintenance mode ten months ago with the release of 5.0.  The
community is now focused on 5.x versions.  If you mention a problem with
a 4.x version now, the first thing you'll be told is that you need to
upgrade, because unless the bug you're experiencing is a showstopper
that affects a wide variety of users, it will not be fixed in 4.x.  If
it is a major bug that affects a large number of users, it will only be
fixed a version like 4.10.5 -- a bugfix release on the last minor 4.x
version.

Thanks,
Shawn

Re: Testing Solr configuration, schema, and other fields

[Erick Erickson]

Yeah, the notion of DTDs have gone around several times but always founder
on the fact that you can, say, define your own Filter with it's own set of
parameters etc. Sure, you can make a generic DTD that accommodates
this, but then it becomes so general as to be little more than a syntax checker.

The managed schema stuff allows modifications of the schema via REST calls
and there is some equivalent functionality for solrconfig.xml, but the
interesting
bit about that is that then your VCS is not the "one true source" of
the configs,
it almost goes backwards: Modify the configs in Zookeeper then check in to Git.
And even that doesn't really solve, say, putting default search fields in
solrconfig.xml that do not exist in the schema file.

Frankly what I usually do when heavily editing either one is just do
it on my local
laptop, either stand alone or SolrCloud, _then_ check it in and/or test it on
my cloud setup. So I guess the take-away is that I don't have any very good
solution here.

Best,
Erick


On Wed, Dec 30, 2015 at 1:10 PM, Davis, Daniel (NIH/NLM) [C]
 wrote:
> Your bottom line point is that EmbeddedSolrServer is different, and some 
> configurations will not work on it where they would work on a SolrCloud.   
> This is well taken.   Maybe creating a new collection on existing dev nodes 
> could be done.
>
> As far as VDI and Puppet.   My requirements are different because my 
> organization is different.   I would prefer not to go into how different.   I 
> have written puppet modules for other system configurations, tested them on 
> AWS EC2, and yet those modules have not been adopted by my organization.
>
>
> -Original Message-
> From: Mark Horninger [mailto:mhornin...@grayhairsoftware.com]
> Sent: Wednesday, December 30, 2015 3:25 PM
> To: solr-user@lucene.apache.org
> Subject: RE: Testing Solr configuration, schema, and other fields
>
> Daniel,
>
>
> Sounds almost like you're reinventing the wheel.  Could you possibly automate 
> this through puppet or Chef?  With a VDI environment, then all you would need 
> to do is build a new VM Node based on original setup.  Then you can just roll 
> out the node as one of the zk nodes.
>
> Just a thought on that subject.
>
> v/r,
>
> -Mark H.
>
> -Original Message-
> From: Davis, Daniel (NIH/NLM) [C] [mailto:daniel.da...@nih.gov]
> Sent: Wednesday, December 30, 2015 3:10 PM
> To: solr-user@lucene.apache.org
> Subject: Testing Solr configuration, schema, and other fields
>
> At my organization, I want to create a tool that allows users to keep a solr 
> configuration as a Git repository.   Then, I want my Continuous Integration 
> environment to take some branch of the git repository and "publish" it into 
> ZooKeeper/SolrCloud.
>
> Working on my own, it is only a very small pain to note foolish errors I've 
> made, fix them, and restart.However, I want my users to be able to edit 
> their own Solr schema and config *most* of the time, at least on development 
> servers.They will not have command-line access to these servers, and I 
> want to avoid endless restarts.
>
> I'm not interested in fighting to maintain such a useless thing as a DTD/XSD 
> without community support; what I really want to know is whether Solr will 
> start and can index some sample documents.   I'm wondering whether I might be 
> able to build a tool to fire up an EmbeddedSolrServer and capture error 
> messages/exceptions in a reasonable way. This tool could then be run by 
> my users before they commit to git, and then again by the CI server before it 
> "publishes" the configuration to ZooKeeper/SolrCloud.
>
> Any suggestions?
>
> Dan Davis, Systems/Applications Architect (Contractor), Office of Computer 
> and Communications Systems, National Library of Medicine, NIH
>
> [GrayHair]
> GHS Confidentiality Notice
>
> This e-mail message, including any attachments, is for the sole use of the 
> intended recipient(s) and may contain confidential and privileged 
> information. Any unauthorized review, use, disclosure or distribution of this 
> information is prohibited, and may be punishable by law. If this was sent to 
> you in error, please notify the sender by reply e-mail and destroy all copies 
> of the original message.
>
> GrayHair Software 
>

Add me to the Solr ContributorsGroup

[Saïd Radhouani]

Hi - I'd appreciate if you could add me to the Contributor Group. Here are
my account info :

- Name: Saïd Radhouani
- User name: radhouani
- email: said.radhou...@gmail.com

Thanks,
-Saïd

Re: Add me to the Solr ContributorsGroup

[Erick Erickson]

Done

On Wed, Dec 30, 2015 at 5:36 PM, Saïd Radhouani
 wrote:
> Hi - I'd appreciate if you could add me to the Contributor Group. Here are
> my account info :
>
> - Name: Saïd Radhouani
> - User name: radhouani
> - email: said.radhou...@gmail.com
>
> Thanks,
> -Saïd

Re: Having replica will slow down Solr?

[Zheng Lin Edwin Yeo]

Hi Binoy,

The only large stored field I have is content, which is indexed from the
content of the rich text documents.

Regards,
Edwin


On 30 December 2015 at 16:20, Binoy Dalal  wrote:

> If I understand the cloud setup correctly having a replica will serve for
> redundancy and won't slow your searches down.
> In your case for the slow response times, are you returning any large
> stored fields?
>
> On Wed, 30 Dec 2015, 13:26 Zheng Lin Edwin Yeo 
> wrote:
>
> > Hi,
> >
> > I would like to find out, will having a replica slow down the search for
> > Solr?
> >
> > Currently, I'm having 1 shard and a replicationFactor of 2 using Solr
> > 5.3.0. I'm running SolrCloud, with 3 external ZooKeeper using ZooKeeper
> > 3.4.6, and my index size is 183GB.
> >
> > I have been getting QTime of more than 3000ms for my basic search
> function,
> > even without adding other things like faceting or highlighting.
> >
> > Regards,
> > Edwin
> >
> --
> Regards,
> Binoy Dalal
>

Re: Using post.jr for indexing in Solr 5.4.0

[davidphilip cherian]

Did you try using the latest post jar (shipped with 5.4 version) to index
same docs?
What happened? Did you get the same error or no?



On Thu, Dec 31, 2015 at 11:12 AM, Zheng Lin Edwin Yeo 
wrote:

> Hi,
>
> Would like to find out, is there any changes to the post.jar for Solr 5.4?
> I tried to use the post.jar from Solr 5.3.0, and it gives me the following
> error during indexing:
>
> java.lang.NoSuchMethodError:
> org.apache.solr.h
>
> andler.extraction.ExtractingDocumentLoader.init(Lorg/apache/solr/request
>
> /SolrQueryRequest;Lorg/apache/solr/update/processor/UpdateRequestProcessor;Lorg/
>
> apache/tika/config/TikaConfig;Lorg/apache/solr/handler/extraction/ParseContextCo
>
> nfig;Lorg/apache/solr/handler/extraction/SolrContentHandlerFactory;)V name="trace">java.lang.RuntimeException: java.lang.NoSuchMethodError:
> org.apache
>
> .solr.handler.extraction.ExtractingDocumentLoader.init(Lorg/apache/solr/
>
> request/SolrQueryRequest;Lorg/apache/solr/update/processor/UpdateRequestProcesso
>
> r;Lorg/apache/tika/config/TikaConfig;Lorg/apache/solr/handler/extraction/ParseCo
>
> ntextConfig;Lorg/apache/solr/handler/extraction/SolrContentHandlerFactory;)V
> at
> org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:611)
>
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:472)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte
> r.java:222)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte
> r.java:181)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
> Handler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java
> :585)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j
> ava:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.jav
> a:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandl
> er.java:223)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandl
> er.java:1127)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:
> 515)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandle
> r.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandle
> r.java:1061)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j
> ava:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(Cont
> extHandlerCollection.java:215)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerColl
> ection.java:110)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper
> .java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.jav
> a:257)
> at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java
> :540)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPoo
> l.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool
> .java:555)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NoSuchMethodError:
> org.apache.solr.handler.extraction.Extra
>
> ctingDocumentLoader.init(Lorg/apache/solr/request/SolrQueryRequest;Lorg/
>
> apache/solr/update/processor/UpdateRequestProcessor;Lorg/apache/tika/config/Tika
>
> Config;Lorg/apache/solr/handler/extraction/ParseContextConfig;Lorg/apache/solr/h
> andler/extraction/SolrContentHandlerFactory;)V
> at
> org.apache.solr.handler.extraction.ExtractingRequestHandler.newLoader
> (ExtractingRequestHandler.java:130)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Co
> ntentStreamHandlerBase.java:59)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandl
> erBase.java:156)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2073)
> at
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:658)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:457)
> ... 22 more
> 500
> 
>
> Regards,
> Edwin
>

Re: Having replica will slow down Solr?

[davidphilip cherian]

Few questions:

What are your memory configs: xms and xmx values, available ram size?

Were you also indexing at the time when the search response was so slow? In
the sense, were you committing often with openSearcher=true, in any case?

What is the QTime for consequent searches? Are the caches enabled?






On Wed, Dec 30, 2015 at 1:50 PM, Binoy Dalal  wrote:

> If I understand the cloud setup correctly having a replica will serve for
> redundancy and won't slow your searches down.
> In your case for the slow response times, are you returning any large
> stored fields?
>
> On Wed, 30 Dec 2015, 13:26 Zheng Lin Edwin Yeo 
> wrote:
>
> > Hi,
> >
> > I would like to find out, will having a replica slow down the search for
> > Solr?
> >
> > Currently, I'm having 1 shard and a replicationFactor of 2 using Solr
> > 5.3.0. I'm running SolrCloud, with 3 external ZooKeeper using ZooKeeper
> > 3.4.6, and my index size is 183GB.
> >
> > I have been getting QTime of more than 3000ms for my basic search
> function,
> > even without adding other things like faceting or highlighting.
> >
> > Regards,
> > Edwin
> >
> --
> Regards,
> Binoy Dalal
>

Re: How to achieve join like behavior on solr-cloud

[Alok Bhandari]

Hi Dennis ,

thanks for your reply. As I wanted this for some production system so may
not be able to upgrade to under-development branch of solr.

but thanks a lot for pointing me to this possible approach.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-achieve-join-like-behavior-on-solr-cloud-tp4247703p4247896.html
Sent from the Solr - User mailing list archive at Nabble.com.

Maximum number of values in a Solr multi-valued field

[Vishnu perumal]

Hi,

I am using SOLR 4.10. I have around 350M documents indexed in SOLR. In my
case, i have one multi-valued field to store the generated ids and also
this field is searchable. Would large number of values (100K or 200K or 1M)
in this multi-valued field pose any problem for SOLR or is there any limit
in number of values in a multi-valued field?

Field:


Any thoughts of this would be appreciated.


Thank you

Re: Having replica will slow down Solr?

[Binoy Dalal]

If I understand the cloud setup correctly having a replica will serve for
redundancy and won't slow your searches down.
In your case for the slow response times, are you returning any large
stored fields?

On Wed, 30 Dec 2015, 13:26 Zheng Lin Edwin Yeo  wrote:

> Hi,
>
> I would like to find out, will having a replica slow down the search for
> Solr?
>
> Currently, I'm having 1 shard and a replicationFactor of 2 using Solr
> 5.3.0. I'm running SolrCloud, with 3 external ZooKeeper using ZooKeeper
> 3.4.6, and my index size is 183GB.
>
> I have been getting QTime of more than 3000ms for my basic search function,
> even without adding other things like faceting or highlighting.
>
> Regards,
> Edwin
>
-- 
Regards,
Binoy Dalal

Re: Issue with Join

[Mikhail Khludnev]

Hello,

Have you considered to pass NOT {!join from=prac_spec_code
to=practicing_specialties_codes
 fromIndex=autosuggest}*:* as a q and look at debug output.
Also there was a trick for dealing with pure negative queries q=*:*
-foo:bar though it might be obsolete.

On Tue, Dec 29, 2015 at 7:02 AM, William Bell  wrote:

> I have having issues with {!join}. If the core have multiValued field and
> the inner join does not have a multiValued field it does not find the
> ones...
>
> Solr 5.3.1... 5.3.1
>
> Example.
>
> PS1226 is in practicing_specialties_codes in providersearch core. This
> field is multiValued.
>
> in the autosuggest core there is NOT a field for PS1226 in there. This
> field is called prac_spec_code and is single values.
>
>
>
> http://localhost:8983/solr/providersearch/select?q=*%3A*=json=true=practicing_specialties_codes:PS1226=practicing_specialties_codes
>
> I get:
>
>
>- docs:
>[
>   -
>   {
>  - practicing_specialties_codes:
>  [
> - "PS1010",
> - "PS282",
> - "PS1226"
> ]
>  }
>   ]
>
>
>
> In autosuggest there is nothing:
>
>
> http://localhost:8983/solr/autosuggest/select?q=*%3A*=json=true=prac_spec_code:PS1226=prac_spec_code
>
> Nothing.
>
> Then a join should find what is in providersearch but missing in
> autosuggest.
>
>
> http://localhost:8983/solr/providersearch/select?debugQuery=true=json=*:*=10=practicing_specialties_codes:PS1226=practicing_specialties_codes=NOT%20{!join%20from=prac_spec_code%20to=practicing_specialties_codes%20fromIndex=autosuggest}auto_type:PRACSPEC
>
> or
>
>
> http://hgsolr2sl1:8983/solr/providersearch/select?debugQuery=true=json=*:*=10=practicing_specialties_codes=NOT%20{!join%20from=prac_spec_code%20to=practicing_specialties_codes%20fromIndex=autosuggest}auto_type:PRACSPEC
>
> or
>
>
> http://hgsolr2sl1:8983/solr/providersearch/select?debugQuery=true=json=*:*=10=practicing_specialties_codes=NOT%20{!join%20from=prac_spec_code%20to=practicing_specialties_codes%20fromIndex=autosuggest}*:*
>
> I also tried *:* AND NOT {!join}
>
> I get 0 results. This seems to be a bug.
>
> {
>
>- responseHeader:
>{
>   - status: 0,
>   - QTime: 178,
>   - params:
>   {
>  - q: "*:*",
>  - fl: "practicing_specialties_codes",
>  - fq: "NOT {!join from=prac_spec_code
>  to=practicing_specialties_codes fromIndex=autosuggest}*:*",
>  - rows: "10",
>  - wt: "json",
>  - debugQuery: "true"
>  }
>   },
>- response:
>{
>   - numFound: 0,
>   - start: 0,
>   - docs: [ ]
>   },
>- debug:
>{
>   - rawquerystring: "*:*",
>   - querystring: "*:*",
>   - parsedquery: "MatchAllDocsQuery(*:*)",
>   - parsedquery_toString: "*:*",
>   - explain: { },
>   - QParser: "LuceneQParser",
>   - filter_queries:
>   [
>  - "NOT {!join from=prac_spec_code to=practicing_specialties_codes
>  fromIndex=autosuggest}*:*"
>  ],
>   - parsed_filter_queries:
>   [
>  - "-JoinQuery({!join from=prac_spec_code
>  to=practicing_specialties_codes fromIndex=autosuggest}*:*)"
>  ],
>   - timing:
>   {
>  - time: 177,
>  - prepare:
>  {
> - time: 0,
> - query:
> {
>- time: 0
>},
> - facet:
> {
>- time: 0
>},
> - facet_module:
> {
>- time: 0
>},
> - mlt:
> {
>- time: 0
>},
> - highlight:
> {
>- time: 0
>},
> - stats:
> {
>- time: 0
>},
> - expand:
> {
>- time: 0
>},
> - debug:
> {
>- time: 0
>}
> },
>  - process:
>  {
> - time: 177,
> - query:
> {
>- time: 177
>},
> - facet:
> {
>- time: 0
>},
> - facet_module:
> {
>- time: 0
>},
> - mlt:
> {
>- time: 0
>},
> - highlight:
> {
>- time: 0
>},
> - stats:
> {
>- time: 0
>},
> - expand:
> {
>- time: 0
>},
> - debug:
> {
>- time: 0
>}
> }
>  }
>   }
>
> }
>
>
>
>
> --
> Bill Bell
> billnb...@gmail.com
> cell 720-256-8076
>



-- 
Sincerely yours
Mikhail Khludnev

Re: Parameter Substitution / Macro Expansion

[Erik Hatcher]

I dunno why that doesn't work but try {!mlt ... v=$master_id} and set a default 
value in your config for master_id and that should at least work. 

   Erik

> On Dec 30, 2015, at 08:37, Alessandro Benedetti  wrote:
> 
> I was taking a look to this article from Yonik [1] .
> I like the feature and actually it could useful for some use cases of mine.
> Started to play with it in Solr 5.4 .
> 
> I verified it is working if you pass both the query and the parameters to
> substitute as request parameters.
> 
> I verified it is working if you pass both the query and the parameters to
> substitute as defaults or appends for the request handler.
> 
> Currently I am not able to make it working the following scenario :
> solrconfig.xml
> 
> 
>>  
>>   ...
>>  {!boost b=recip(dist(2,0,star_rating,0,
>> *${star_rating:3}*),1,10,10)}{!boost b=recip(geodist(lat_lon,*${lat:0}*,
>> *${lon:0}*),2,200,20)}{!mlt
>> qf=name,description,facilities,resort,region,dest_level_2 mintf=1 mindf=5
>> maxqt=50}*${master_id:0}*
>>   ...
>>  
>>  
> 
> 
> 
> Then I pass in the request the parameter to substitute , like :
> master_id=1 .
> Unfortunately is always getting the default one.
> Am i doing something wrong ?
> If it is known limitation I would be happy to contribute a fix, what are
> the components involved ?
> 
> Cheers
> 
> [1] http://yonik.com/solr-query-parameter-substitution/
> 
> -- 
> --
> 
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
> 
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
> 
> William Blake - Songs of Experience -1794 England

Re: Maximum number of values in a Solr multi-valued field

[Upayavira]

I am not aware of a specific limit to the number of values in a
multivalued field. Essentially, a multi-valued field is the same as a
very long tokenised field - so a document containing a full Shakespeare
play for example.

If you are doing searches against a document set of this type, then you
will be going from terms to documents. In this case, a single one of
your terms will map back to your document, in which case, it really
doesn't matter how many such terms map back to the document, performance
will be comparable.

However, if you are attempting faceting, for example, when you want to
calculate the facet counts for this field, you will need to increment
the bucket for every value for your field for each document, and in such
a case you can expect performance to collapse.

Upayavira

On Wed, Dec 30, 2015, at 10:50 AM, Vishnu perumal wrote:
> Hi,
> 
> I am using SOLR 4.10. I have around 350M documents indexed in SOLR. In my
> case, i have one multi-valued field to store the generated ids and also
> this field is searchable. Would large number of values (100K or 200K or
> 1M)
> in this multi-valued field pose any problem for SOLR or is there any
> limit
> in number of values in a multi-valued field?
> 
> Field:
>  required="false" multiValued="true" />
> 
> Any thoughts of this would be appreciated.
> 
> 
> Thank you

Parameter Substitution / Macro Expansion

[Alessandro Benedetti]

I was taking a look to this article from Yonik [1] .
I like the feature and actually it could useful for some use cases of mine.
Started to play with it in Solr 5.4 .

I verified it is working if you pass both the query and the parameters to
substitute as request parameters.

I verified it is working if you pass both the query and the parameters to
substitute as defaults or appends for the request handler.

Currently I am not able to make it working the following scenario :
solrconfig.xml


>   
>...
>   {!boost b=recip(dist(2,0,star_rating,0,
> *${star_rating:3}*),1,10,10)}{!boost b=recip(geodist(lat_lon,*${lat:0}*,
> *${lon:0}*),2,200,20)}{!mlt
> qf=name,description,facilities,resort,region,dest_level_2 mintf=1 mindf=5
> maxqt=50}*${master_id:0}*
>...
>   
>   



Then I pass in the request the parameter to substitute , like :
master_id=1 .
Unfortunately is always getting the default one.
Am i doing something wrong ?
If it is known limitation I would be happy to contribute a fix, what are
the components involved ?

Cheers

[1] http://yonik.com/solr-query-parameter-substitution/

-- 
--

Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: Parameter Substitution / Macro Expansion

[Yonik Seeley]

On Wed, Dec 30, 2015 at 8:37 AM, Alessandro Benedetti
 wrote:
> I was taking a look to this article from Yonik [1] .
> I like the feature and actually it could useful for some use cases of mine.
> Started to play with it in Solr 5.4 .
>
> I verified it is working if you pass both the query and the parameters to
> substitute as request parameters.
>
> I verified it is working if you pass both the query and the parameters to
> substitute as defaults or appends for the request handler.
>
> Currently I am not able to make it working the following scenario :
> solrconfig.xml
>
> 
>>   
>>...
>>   {!boost b=recip(dist(2,0,star_rating,0,
>> *${star_rating:3}*),1,10,10)}{!boost b=recip(geodist(lat_lon,*${lat:0}*,
>> *${lon:0}*),2,200,20)}{!mlt
>> qf=name,description,facilities,resort,region,dest_level_2 mintf=1 mindf=5
>> maxqt=50}*${master_id:0}*
>>...
>>   
>>   
>
>
>
> Then I pass in the request the parameter to substitute , like :
> master_id=1 .
> Unfortunately is always getting the default one.

Hmmm, that's odd.
Macro expansion is done after defaults/appends/invariants are added to
the params, so it should work (although we lack a test for this).

See RequestUtil.java:
  newMap = MacroExpander.expand(newMap);

If you add echoParams=all to your request, that currently shows the
params post-expansion to verify you're actually getting the default.

-Yonik

Re: Parameter Substitution / Macro Expansion

[Alessandro Benedetti]

Erick,
your suggestion worked :




Hotels Similarity
edismax
5
*,score
active:true
explicit
json
{!boost
b=recip(dist(2,0,star_rating,0,$star_rating),1,10,10)}{!boost
b=recip(geodist(lat_lon,$lat,$lon),2,200,20)}{!mlt
qf=name,description,facilities,resort,region,dest_level_2 mintf=1
mindf=5 maxqt=50 v=$master_id}

0
3
0
0



Yonik, running debugQuery=on i verified that the default were added before,
so basically I can see the default replaced but the one I am sending in the
request param take no effect ( as actually the macro disappeared , replaced
by the default) .
Is maybe because I am inside a block with different query parsers ?

Cheers


On 30 December 2015 at 15:14, Yonik Seeley  wrote:

> On Wed, Dec 30, 2015 at 8:37 AM, Alessandro Benedetti
>  wrote:
> > I was taking a look to this article from Yonik [1] .
> > I like the feature and actually it could useful for some use cases of
> mine.
> > Started to play with it in Solr 5.4 .
> >
> > I verified it is working if you pass both the query and the parameters to
> > substitute as request parameters.
> >
> > I verified it is working if you pass both the query and the parameters to
> > substitute as defaults or appends for the request handler.
> >
> > Currently I am not able to make it working the following scenario :
> > solrconfig.xml
> >
> > 
> >>   
> >>...
> >>   {!boost b=recip(dist(2,0,star_rating,0,
> >> *${star_rating:3}*),1,10,10)}{!boost b=recip(geodist(lat_lon,*${lat:0}*,
> >> *${lon:0}*),2,200,20)}{!mlt
> >> qf=name,description,facilities,resort,region,dest_level_2 mintf=1
> mindf=5
> >> maxqt=50}*${master_id:0}*
> >>...
> >>   
> >>   
> >
> >
> >
> > Then I pass in the request the parameter to substitute , like :
> > master_id=1 .
> > Unfortunately is always getting the default one.
>
> Hmmm, that's odd.
> Macro expansion is done after defaults/appends/invariants are added to
> the params, so it should work (although we lack a test for this).
>
> See RequestUtil.java:
>   newMap = MacroExpander.expand(newMap);
>
> If you add echoParams=all to your request, that currently shows the
> params post-expansion to verify you're actually getting the default.
>
> -Yonik
>



-- 
--

Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: Parameter Substitution / Macro Expansion

[Alessandro Benedetti]

 debugging I confirm you are right Yonik, but also that the defaults
are already applied at this point :
org/apache/solr/request/json/RequestUtil.java:158

Could be a resolution of properties that happens at solrconfig, xml level ?
It verifies if any java properties is in there when running solr, and
because of that the defaults are already applied ?

Cheers

On 30 December 2015 at 15:20, Alessandro Benedetti 
wrote:

> Erick,
> your suggestion worked :
>
> 
> 
> 
> Hotels Similarity
> edismax
> 5
> *,score
> active:true
> explicit
> json
> {!boost 
> b=recip(dist(2,0,star_rating,0,$star_rating),1,10,10)}{!boost 
> b=recip(geodist(lat_lon,$lat,$lon),2,200,20)}{!mlt 
> qf=name,description,facilities,resort,region,dest_level_2 mintf=1 mindf=5 
> maxqt=50 v=$master_id}
> 
> 0
> 3
> 0
> 0
> 
> 
>
> Yonik, running debugQuery=on i verified that the default were added
> before, so basically I can see the default replaced but the one I am
> sending in the request param take no effect ( as actually the macro
> disappeared , replaced by the default) .
> Is maybe because I am inside a block with different query parsers ?
>
> Cheers
>
>
> On 30 December 2015 at 15:14, Yonik Seeley  wrote:
>
>> On Wed, Dec 30, 2015 at 8:37 AM, Alessandro Benedetti
>>  wrote:
>> > I was taking a look to this article from Yonik [1] .
>> > I like the feature and actually it could useful for some use cases of
>> mine.
>> > Started to play with it in Solr 5.4 .
>> >
>> > I verified it is working if you pass both the query and the parameters
>> to
>> > substitute as request parameters.
>> >
>> > I verified it is working if you pass both the query and the parameters
>> to
>> > substitute as defaults or appends for the request handler.
>> >
>> > Currently I am not able to make it working the following scenario :
>> > solrconfig.xml
>> >
>> > 
>> >>   
>> >>...
>> >>   {!boost b=recip(dist(2,0,star_rating,0,
>> >> *${star_rating:3}*),1,10,10)}{!boost
>> b=recip(geodist(lat_lon,*${lat:0}*,
>> >> *${lon:0}*),2,200,20)}{!mlt
>> >> qf=name,description,facilities,resort,region,dest_level_2 mintf=1
>> mindf=5
>> >> maxqt=50}*${master_id:0}*
>> >>...
>> >>   
>> >>   
>> >
>> >
>> >
>> > Then I pass in the request the parameter to substitute , like :
>> > master_id=1 .
>> > Unfortunately is always getting the default one.
>>
>> Hmmm, that's odd.
>> Macro expansion is done after defaults/appends/invariants are added to
>> the params, so it should work (although we lack a test for this).
>>
>> See RequestUtil.java:
>>   newMap = MacroExpander.expand(newMap);
>>
>> If you add echoParams=all to your request, that currently shows the
>> params post-expansion to verify you're actually getting the default.
>>
>> -Yonik
>>
>
>
>
> --
> --
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>



-- 
--

Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: Parameter Substitution / Macro Expansion

[Alessandro Benedetti]

Reference :
https://wiki.apache.org/solr/SolrConfigXml#System_property_substitution

On 30 December 2015 at 15:30, Alessandro Benedetti 
wrote:

>  debugging I confirm you are right Yonik, but also that the defaults
> are already applied at this point :
> org/apache/solr/request/json/RequestUtil.java:158
>
> Could be a resolution of properties that happens at solrconfig, xml level ?
> It verifies if any java properties is in there when running solr, and
> because of that the defaults are already applied ?
>
> Cheers
>
> On 30 December 2015 at 15:20, Alessandro Benedetti 
> wrote:
>
>> Erick,
>> your suggestion worked :
>>
>> 
>> 
>> 
>> Hotels Similarity
>> edismax
>> 5
>> *,score
>> active:true
>> explicit
>> json
>> {!boost 
>> b=recip(dist(2,0,star_rating,0,$star_rating),1,10,10)}{!boost 
>> b=recip(geodist(lat_lon,$lat,$lon),2,200,20)}{!mlt 
>> qf=name,description,facilities,resort,region,dest_level_2 mintf=1 mindf=5 
>> maxqt=50 v=$master_id}
>> 
>> 0
>> 3
>> 0
>> 0
>> 
>> 
>>
>> Yonik, running debugQuery=on i verified that the default were added
>> before, so basically I can see the default replaced but the one I am
>> sending in the request param take no effect ( as actually the macro
>> disappeared , replaced by the default) .
>> Is maybe because I am inside a block with different query parsers ?
>>
>> Cheers
>>
>>
>> On 30 December 2015 at 15:14, Yonik Seeley  wrote:
>>
>>> On Wed, Dec 30, 2015 at 8:37 AM, Alessandro Benedetti
>>>  wrote:
>>> > I was taking a look to this article from Yonik [1] .
>>> > I like the feature and actually it could useful for some use cases of
>>> mine.
>>> > Started to play with it in Solr 5.4 .
>>> >
>>> > I verified it is working if you pass both the query and the parameters
>>> to
>>> > substitute as request parameters.
>>> >
>>> > I verified it is working if you pass both the query and the parameters
>>> to
>>> > substitute as defaults or appends for the request handler.
>>> >
>>> > Currently I am not able to make it working the following scenario :
>>> > solrconfig.xml
>>> >
>>> > 
>>> >>   
>>> >>...
>>> >>   {!boost b=recip(dist(2,0,star_rating,0,
>>> >> *${star_rating:3}*),1,10,10)}{!boost
>>> b=recip(geodist(lat_lon,*${lat:0}*,
>>> >> *${lon:0}*),2,200,20)}{!mlt
>>> >> qf=name,description,facilities,resort,region,dest_level_2 mintf=1
>>> mindf=5
>>> >> maxqt=50}*${master_id:0}*
>>> >>...
>>> >>   
>>> >>   
>>> >
>>> >
>>> >
>>> > Then I pass in the request the parameter to substitute , like :
>>> > master_id=1 .
>>> > Unfortunately is always getting the default one.
>>>
>>> Hmmm, that's odd.
>>> Macro expansion is done after defaults/appends/invariants are added to
>>> the params, so it should work (although we lack a test for this).
>>>
>>> See RequestUtil.java:
>>>   newMap = MacroExpander.expand(newMap);
>>>
>>> If you add echoParams=all to your request, that currently shows the
>>> params post-expansion to verify you're actually getting the default.
>>>
>>> -Yonik
>>>
>>
>>
>>
>> --
>> --
>>
>> Benedetti Alessandro
>> Visiting card : http://about.me/alessandro_benedetti
>>
>> "Tyger, tyger burning bright
>> In the forests of the night,
>> What immortal hand or eye
>> Could frame thy fearful symmetry?"
>>
>> William Blake - Songs of Experience -1794 England
>>
>
>
>
> --
> --
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>



-- 
--

Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: Changing Solr Schema with Data

[Salman Ansari]

Thanks.

Salman

On Tue, Dec 29, 2015 at 9:33 PM, Binoy Dalal  wrote:

> What shalin says is solid and will work with solr 5.x as well as 3.x
> You could do a little poc if you want to be absolutely certain. Shouldn't
> take you very long.
> Your only concern will be that your old docs won't be matched against
> queries matched against the newly added fields.
>
> On Tue, 29 Dec 2015, 23:38 Salman Ansari  wrote:
>
> > Thanks guys for your responses.
> >
> > @Shalin: Do you have a documentation that explains this? Moreover, is it
> > only for Solr 5+ or is it still applicable to Solr 3+? I am asking this
> as
> > I am working in a team and in some of our projects we are using old Solr
> > versions and I need to convince the guys that this is possible in the old
> > Solr as well.
> >
> > Thanks for your help.
> >
> > Regards,
> > Salman
> >
> >
> > On Tue, Dec 29, 2015 at 9:44 AM, Shalin Shekhar Mangar <
> > shalinman...@gmail.com> wrote:
> >
> > > Adding new fields is not a problem. You can continue to use your
> > > existing index with the new schema.
> > >
> > > On Tue, Dec 29, 2015 at 1:58 AM, Salman Ansari <
> salman.rah...@gmail.com>
> > > wrote:
> > > > You can say that we are not removing any fields (so the old data
> should
> > > not
> > > > get affected), however, we need to add new fields (which new data
> will
> > > > have). Does that answer your question?
> > > >
> > > >
> > > > Regards,
> > > > Salman
> > > >
> > > > On Mon, Dec 28, 2015 at 9:58 PM, Alexandre Rafalovitch <
> > > arafa...@gmail.com>
> > > > wrote:
> > > >
> > > >> Is the schema change affects the data you want to keep?
> > > >> 
> > > >> Newsletter and resources for Solr beginners and intermediates:
> > > >> http://www.solr-start.com/
> > > >>
> > > >>
> > > >> On 29 December 2015 at 01:48, Salman Ansari <
> salman.rah...@gmail.com>
> > > >> wrote:
> > > >> > Hi,
> > > >> >
> > > >> > I am facing an issue where I need to change Solr schema but I have
> > > >> crucial
> > > >> > data that I don't want to delete. Is there a way where I can
> change
> > > the
> > > >> > schema of the index while keeping the data intact?
> > > >> >
> > > >> > Regards,
> > > >> > Salman
> > > >>
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Shalin Shekhar Mangar.
> > >
> >
> --
> Regards,
> Binoy Dalal
>