Re: eDismax query syntax question

[Andrea Gazzarini]

Hi Webster,
what does the query debug say? if you set debug=true in the request you can
have a better idea about how the two queries get interpreted

Andrea

On Mon, 15 Jun 2020 at 16:01, Webster Homer <
webster.ho...@milliporesigma.com> wrote:

> Markus,
> Thanks, for the reference, but that doesn't answer my question. If - is a
> special character, it's not consistently special. In my example
> "3-DIMETHYL" behaves quite differently than ")-PYRIMIDINE".  If I escape
> the closing parenthesis the following minus no longer behaves specially.
> The referred article does not even mention parenthesis, but it changes the
> behavior of the following "-" if it is escaped. In "3-DIMETHYL" the minus
> is not special.
>
> These all fix the problem:
> 1,3-DIMETHYL-5-(3-PHENYL-ALLYLIDENE\)-PYRIMIDINE-2,4,6-TRIONE
> 1,3-DIMETHYL-5-(3-PHENYL-ALLYLIDENE)\-PYRIMIDINE-2,4,6-TRIONE
> 1,3-DIMETHYL-5-\(3-PHENYL-ALLYLIDENE\)-PYRIMIDINE-2,4,6-TRIONE
>
> Only the minus following the parenthesis is treated as a NOT.
> Are parentheses special? They're not mentioned in the eDismax
> documentation.
>
> -Original Message-
> From: Markus Jelsma 
> Sent: Saturday, June 13, 2020 4:57 AM
> To: solr-user@lucene.apache.org
> Subject: RE: eDismax query syntax question
>
> Hello,
>
> These are special characters, if you don't need them, you must escape them.
>
> See top of the article:
>
> https://lucene.apache.org/solr/guide/8_5/the-extended-dismax-query-parser.html
>
> Markus
>
>
>
>
> -Original message-
> > From:Webster Homer 
> > Sent: Friday 12th June 2020 22:09
> > To: solr-user@lucene.apache.org
> > Subject: eDismax query syntax question
> >
> > Recently we found strange behavior in a query. We use eDismax as the
> query parser.
> >
> > This is the query term:
> > 1,3-DIMETHYL-5-(3-PHENYL-ALLYLIDENE)-PYRIMIDINE-2,4,6-TRIONE
> >
> > It should hit one document in our index. It does not. However, if you
> use the Dismax query parser it does match the record.
> >
> > The problem seems to involve the parenthesis and the dashes. If you
> > escape the dash after the parenthesis it matches
> > 1,3-DIMETHYL-5-(3-PHENYL-ALLYLIDENE)\-PYRIMIDINE-2,4,6-TRIONE
> >
> > I thought that eDismax and Dismax escaped all lucene special characters
> before passing the query to lucene. Although I also remember reading that +
> and - can have special significance in a query if preceded with white
> space. I can find very little documentation on either query parser in how
> they work.
> >
> > Is this expected behavior or is this a bug? If expected, where can I
> find documentation?
> >
> >
> >
> > This message and any attachment are confidential and may be privileged
> or otherwise protected from disclosure. If you are not the intended
> recipient, you must not copy this message or attachment or disclose the
> contents to any other person. If you have received this transmission in
> error, please notify the sender immediately and delete the message and any
> attachment from your system. Merck KGaA, Darmstadt, Germany and any of its
> subsidiaries do not accept liability for any omissions or errors in this
> message which may arise as a result of E-Mail-transmission or for damages
> resulting from any unauthorized changes of the content of this message and
> any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
> subsidiaries do not guarantee that this message is free of viruses and does
> not accept liability for any damages caused by any virus transmitted
> therewith.
> >
> >
> >
> > Click http://www.merckgroup.com/disclaimer to access the German,
> French, Spanish and Portuguese versions of this disclaimer.
> >
>
>
> This message and any attachment are confidential and may be privileged or
> otherwise protected from disclosure. If you are not the intended recipient,
> you must not copy this message or attachment or disclose the contents to
> any other person. If you have received this transmission in error, please
> notify the sender immediately and delete the message and any attachment
> from your system. Merck KGaA, Darmstadt, Germany and any of its
> subsidiaries do not accept liability for any omissions or errors in this
> message which may arise as a result of E-Mail-transmission or for damages
> resulting from any unauthorized changes of the content of this message and
> any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
> subsidiaries do not guarantee that this message is free of viruses and does
> not accept liability for any damages caused by any virus transmitted
> therewith.
>
>
>
> Click http://www.merckgroup.com/disclaimer to access the German, French,
> Spanish and Portuguese versions of this disclaimer.
>
-- 
Andrea Gazzarini
*Search Consultant, R Software Engineer*


www.sease.io

email: a.gazzar...@sease.io
cell: +39 349 513 86 25

Re: json.facet throws ClassCastException

[Andrea Gazzarini]

Hi, problem should be caused by missing surrounding curly brackets.
That is, your query is

json.facet=prod:{type:terms,field:product,mincount:1,limit:8}

instead it should be

json.facet=*{*prod:{type:terms,field:product,mincount:1,limit:8}*}*

that causes the wrong interpretation of the "json/facet" parameter 
(String instead of Map)


Cheers,
Andrea

On 04/10/2019 22:55, Mikhail Khludnev wrote:

Gosh, obviously. see the clue
https://github.com/apache/lucene-solr/blob/7d3dcd220f92f25a997cf1559a91b6d9e1b57c6d/solr/core/src/java/org/apache/solr/search/facet/FacetModule.java#L78

On Fri, Oct 4, 2019 at 10:47 PM Webster Homer <
webster.ho...@milliporesigma.com> wrote:


Sometimes it comes back in the reply
"java.lang.ClassCastException: java.lang.String cannot be cast to
java.util.Map\n\tat
org.apache.solr.search.facet.FacetModule.prepare(FacetModule.java:78)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:269)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)\n\tat
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:710)\n\tat
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:516)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1751)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\n\tat
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:169)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat
org.eclipse.jetty.server.Server.handle(Server.java:534)\n\tat
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)\n\tat
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)\n\tat
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)\n\tat
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)\n\tat
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\n\tat
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\n\tat
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\n\tat
java.lang.Thread.run(Thread.java:748)\n",

-Original Message-
From: Mikhail Khludnev 
Sent: Friday, October 04, 2019 2:28 PM
To: solr-user 
Subject: Re: json.facet throws ClassCastException

Hello, Webster.

Have you managed to capture stacktrace?

On Fri, Oct 4, 2019 at 8:24 PM Webster Homer <
webster.ho...@milliporesigma.com> wrote:


I'm trying to understand what is wrong with my query or collection.

I have a functioning solr schema and collection. I'm running Solr 7.2

When I run with a facet.field it works, but if I change it to use a
json.facet it throws a class cast exception.

json.facet=prod:{type:terms,field:product,mincount:1,limit:8}

java.lang.String cannot be cast to java.util.Map

The product field is defined as


And lowercase is defined as:

   
 
 
   
 

I don't have enough information to understand what its complaining about.

Thanks
This message and any attachment are confidential and may be privileged
or otherwise protected from disclosure. If you are not the intended
recipient, you must not copy this message or attachment or disclose
the contents to any other person. If you have 

Re: Synonym filters memory usage

[Andrea Gazzarini]

That sounds really strange to me.
Segments are created gradually depending on changes applied to the 
index, while the Schema should have a completely different lifecycle, 
independent from that.
If that is true, that would mean each time a new segment is created Solr 
would instantiate a new Schema instance (or at least, assuming this is 
valid only for synonyms, one SynonymFilterFactory, one SynonymFilter, 
one SynonymMap), which again, sounds really strange.


Thanks for the point, I'll check and I'll let you know

Cheers,
Andrea

On 30/09/2019 09:58, Bernd Fehling wrote:

Yes, I think so.
While integrating a Thesaurus as synonyms.txt I saw massive memory usage.
A heap dump and analysis with MemoryAnalyzer pointed out that the
SynonymMap took 3 times a huge amount of memory, together with each
opened index segment.
Just try it and check that by yourself with heap dump and MemoryAnalyzer.

Regards
Bernd


Am 30.09.19 um 09:44 schrieb Andrea Gazzarini:
mmm, ok for the core but are you sure things in this case are working 
per-segment? I would expect a FilterFactory instance per index, 
initialized at schema loading time.


On 30/09/2019 09:04, Bernd Fehling wrote:

And I think this is per core per index segment.

2 cores per instance, each core with 3 index segments, sums up to 6 
times

the 2 SynonymMaps. Results in 12 times SynonymMaps.

Regards
Bernd


Am 30.09.19 um 08:41 schrieb Andrea Gazzarini:

  Hi,
looking at the stateful nature of SynonymGraphFilter/FilterFactory 
classes,

the answer should be 2 times (one time per type instance).
The SynonymMap, which internally holds the synonyms table, is a 
private
member of the filter factory and it is loaded each time the factory 
needs

to create a type.

Best,
Andrea

On 29/09/2019 23:49, Dominique Bejean wrote:

Hi,

My concern is about memory used by synonym filter, especially if 
synonyms

resources files are large.

If in my schema, there are two field types "TypeSyno1" and "TypeSyno2"
using synonym filter with the same synonyms files.
For each of these two field types there are two fields

Field1 type is TypeSyno1
Field2 type is TypeSyno1
Field3 type is TypeSyno2
Field4 type is TypeSyno2

How many times is the synonym file loaded in memory ?
4 times, so one time per field ?
2 times, so one time per instanciated type ?

Regards

Dominique




--
Andrea Gazzarini
/Search Consultant, R Software Engineer/

Sease Ltd

mobile: +39 349 513 86 25
email: a.gazzar...@sease.io

Re: Synonym filters memory usage

[Andrea Gazzarini]

mmm, ok for the core but are you sure things in this case are working 
per-segment? I would expect a FilterFactory instance per index, 
initialized at schema loading time.


On 30/09/2019 09:04, Bernd Fehling wrote:

And I think this is per core per index segment.

2 cores per instance, each core with 3 index segments, sums up to 6 times
the 2 SynonymMaps. Results in 12 times SynonymMaps.

Regards
Bernd


Am 30.09.19 um 08:41 schrieb Andrea Gazzarini:

  Hi,
looking at the stateful nature of SynonymGraphFilter/FilterFactory 
classes,

the answer should be 2 times (one time per type instance).
The SynonymMap, which internally holds the synonyms table, is a private
member of the filter factory and it is loaded each time the factory 
needs

to create a type.

Best,
Andrea

On 29/09/2019 23:49, Dominique Bejean wrote:

Hi,

My concern is about memory used by synonym filter, especially if 
synonyms

resources files are large.

If in my schema, there are two field types "TypeSyno1" and "TypeSyno2"
using synonym filter with the same synonyms files.
For each of these two field types there are two fields

Field1 type is TypeSyno1
Field2 type is TypeSyno1
Field3 type is TypeSyno2
Field4 type is TypeSyno2

How many times is the synonym file loaded in memory ?
4 times, so one time per field ?
2 times, so one time per instanciated type ?

Regards

Dominique


--
Andrea Gazzarini
/Search Consultant, R Software Engineer/

Sease Ltd

mobile: +39 349 513 86 25
email: a.gazzar...@sease.io

Re: Synonym filters memory usage

[Andrea Gazzarini]

On 30/09/2019 09:04, Bernd Fehling wrote:

And I think this is per core per index segment.

2 cores per instance, each core with 3 index segments, sums up to 6 times
the 2 SynonymMaps. Results in 12 times SynonymMaps.

Regards
Bernd


Am 30.09.19 um 08:41 schrieb Andrea Gazzarini:

  Hi,
looking at the stateful nature of SynonymGraphFilter/FilterFactory 
classes,

the answer should be 2 times (one time per type instance).
The SynonymMap, which internally holds the synonyms table, is a private
member of the filter factory and it is loaded each time the factory 
needs

to create a type.

Best,
Andrea

On 29/09/2019 23:49, Dominique Bejean wrote:

Hi,

My concern is about memory used by synonym filter, especially if 
synonyms

resources files are large.

If in my schema, there are two field types "TypeSyno1" and "TypeSyno2"
using synonym filter with the same synonyms files.
For each of these two field types there are two fields

Field1 type is TypeSyno1
Field2 type is TypeSyno1
Field3 type is TypeSyno2
Field4 type is TypeSyno2

How many times is the synonym file loaded in memory ?
4 times, so one time per field ?
2 times, so one time per instanciated type ?

Regards

Dominique


--
Andrea Gazzarini
/Search Consultant, R Software Engineer/

Sease Ltd

mobile: +39 349 513 86 25
email: a.gazzar...@sease.io

Re: Synonym filters memory usage

[Andrea Gazzarini]

 Hi,
looking at the stateful nature of SynonymGraphFilter/FilterFactory classes,
the answer should be 2 times (one time per type instance).
The SynonymMap, which internally holds the synonyms table, is a private
member of the filter factory and it is loaded each time the factory needs
to create a type.

Best,
Andrea

-- 
Andrea Gazzarini
*Search Consultant, R Software Engineer*


www.sease.io

email: a.gazzar...@sease.io
cell: +39 349 513 86 25

On 29/09/2019 23:49, Dominique Bejean wrote:

Hi,

My concern is about memory used by synonym filter, especially if synonyms
resources files are large.

If in my schema, there are two field types "TypeSyno1" and "TypeSyno2"
using synonym filter with the same synonyms files.
For each of these two field types there are two fields

Field1 type is TypeSyno1
Field2 type is TypeSyno1
Field3 type is TypeSyno2
Field4 type is TypeSyno2

How many times is the synonym file loaded in memory ?
4 times, so one time per field ?
2 times, so one time per instanciated type ?

Regards

Dominique

Re: req.getCore().getCoreContainer().getCore(core_name) is returning null - Solr 8.2.0

[Andrea Gazzarini]

I remember ZK coordinates (hosts, ports and root) are set as system
properties in Solr nodes (please open the admin console and see their
names). So, it would be just a matter of

System.getProperty(ZK ensemble coordinates|root)

Prior to go in that direction: I don't know/remember if there's some ZK
Solr specific class where they can be asked. If that class exists, it would
be a better way, otherwise you can go with the system property approach.

Andrea

On Thu, 29 Aug 2019, 21:32 Arnold Bronley,  wrote:

> @Andrea: I agree with you. Do you know if there is a way to initialize
> SolrCloudClient directly from some information that I get
> from SolrQueryRequest or from AddUpdateCommand object?
>
> @Erick: Thank you for the information about
> StatelessScriptUpdateProcessorFactory.
>
> "In your situation, add this _before_ the update is distributed and instead
> of
> coreB, ask for collectionB."
>
> Right, but how do I ask for for collectionB?
>
> "Next, you want to get the value from “coreB”. Don’t do that, get it from
> _collection_ B."
>
> Right, but how do I get value _collection_B?
>
>
>
> On Thu, Aug 29, 2019 at 2:17 PM Erick Erickson 
> wrote:
>
> > Have you looked at using one of the update processors?
> >
> > Consider StatelessScriptUpdateProcessorFactory for instance. You can do
> > anything
> > you’d like to do in a script (Groovy, Postscript. Python I think, and
> > others). See:
> > ./example/files/conf/update-script.js for one example.
> >
> > You put it in your solrconfig file in the update handler, then put the
> > script in your
> > conf directory and push it to ZK and the rest is automagical.
> >
> > There are a bunch of other update processors that you can use that are
> also
> > pretty much by configuration, but the one I referenced is the one that is
> > the
> > most general-purpose.
> >
> > In your situation, add this _before_ the update is distributed and
> instead
> > of
> > coreB, ask for collectionB.
> >
> > Distributed updates go like this:
> > 1. the doc gets routed to a leader for a shard
> > 2. the doc gets forwarded to each replica.
> >
> > Now, depending on where you put the update processor (and you’ll have to
> > dig a bit. Much of this distribution logic is implicit, but you can
> > explicitly
> > define it in solrconfig.xml), this either happens  _before_ the docs are
> > sent
> > to the rest of the replicas or _after_ the docs arrive at each replica.
> > From what
> > you’ve described, you want to do this before distribution so all copies
> > have
> > the new field. You don’t care what replica is the leader. You don’t care
> > how many
> > other replicas exist or where they are. You don’t even care if there’s
> any
> > replica hosting this particular collection on the node that does this, it
> > happens
> > before distribution.
> >
> > Next, you want to get the value from “coreB”. Don’t do that, get it from
> > _collection_ B. Since you have the doc ID (presumably the ),
> > using get-by-id instead of a standard query will be very efficient. I can
> > imagine
> > under very heavy load this might introduce too much overhead, but it’s
> > where I’d start.
> >
> > Best,
> > Erick
> >
> > > On Aug 29, 2019, at 1:45 PM, Arnold Bronley 
> > wrote:
> > >
> > > I can't use  CloudSolrClient  because I need to intercept the incoming
> > > indexing request and then add one more field to it. All this happens on
> > > Solr side and not client side.
> > >
> > > On Thu, Aug 29, 2019 at 1:05 PM Andrea Gazzarini  >
> > > wrote:
> > >
> > >> Hi Arnold,
> > >> why don't you use solrj (in this case a CloudSolrClient) instead of
> > dealing
> > >> with such low-level details? The actual location of the document you
> are
> > >> looking for would be completely abstracted.
> > >>
> > >> Best,
> > >> Andrea
> > >>
> > >> On Thu, 29 Aug 2019, 18:50 Arnold Bronley, 
> > >> wrote:
> > >>
> > >>> So, here is the problem that I am trying to solve. I am moving from
> > Solr
> > >>> master-slave architecture to SolrCloud architecture. I have one
> custom
> > >> Solr
> > >>> plugin that does following:
> > >>>
> > >>> 1. When a document (say document with unique id doc1)is getting
> indexed
> > >> to
> > >>> a core say core A then this plugin adds one m

Re: req.getCore().getCoreContainer().getCore(core_name) is returning null - Solr 8.2.0

[Andrea Gazzarini]

"client" and "server" side depends just on the perspective. In my opinion
it is not a black/white and can have different shapes. In your case, I
believe your component, which is on Solr side, can plays both roles (i.e.
"server" component for collection A and client component for collection B)

Andrea

On Thu, 29 Aug 2019, 19:46 Arnold Bronley,  wrote:

> I can't use  CloudSolrClient  because I need to intercept the incoming
> indexing request and then add one more field to it. All this happens on
> Solr side and not client side.
>
> On Thu, Aug 29, 2019 at 1:05 PM Andrea Gazzarini 
> wrote:
>
> > Hi Arnold,
> > why don't you use solrj (in this case a CloudSolrClient) instead of
> dealing
> > with such low-level details? The actual location of the document you are
> > looking for would be completely abstracted.
> >
> > Best,
> > Andrea
> >
> > On Thu, 29 Aug 2019, 18:50 Arnold Bronley, 
> > wrote:
> >
> > > So, here is the problem that I am trying to solve. I am moving from
> Solr
> > > master-slave architecture to SolrCloud architecture. I have one custom
> > Solr
> > > plugin that does following:
> > >
> > > 1. When a document (say document with unique id doc1)is getting indexed
> > to
> > > a core say core A then this plugin adds one more field to the indexing
> > > request. It fetches this new field from core B. Core B in our case
> > > maintains popularity score field for each document which gets
> calculated
> > in
> > > a different project. It fetches the popularity score from score B for
> > doc1
> > > and adds it to indexing request.
> > > 2. In following code, dataInfo.dataSource is the name of the core B.
> > >
> > > I can use the name of the core B like collection_shard1_replica_n21 and
> > it
> > > works. But it is not a good solution. What if I had a multiple shards
> for
> > > core B? In that case the the doc1 that I am trying to find might not be
> > > present in collection_shard1_replica_n21.
> > >
> > > So is there something like,
> > >
> > > SolrCollecton dataCollection = getCollection(dataInfo.dataSource);
> > >
> > > @Override
> > > public void processAdd(AddUpdateCommand cmd) throws IOException {
> > >SolrInputDocument doc = cmd.getSolrInputDocument();
> > >String uniqueId = getUniqueId(doc);
> > >
> > >SolrCore dataCore =
> > > req.getCore().getCoreContainer().getCore(dataInfo.dataSource);
> > >
> > >if (dataCore == null){
> > >LOG.error("Solr core '{}' to use as data source could not be
> > > found!  "
> > >+ "Please check if it is loaded.", dataInfo.dataSource);
> > >} else{
> > >
> > >   Document sourceDoc = getSourceDocument(dataCore, uniqueId);
> > >
> > >   if (sourceDoc != null){
> > >
> > >   populateDocToBeAddedFromSourceDoc(doc,sourceDoc);
> > >   }
> > >}
> > >
> > >// pass it up the chain
> > >super.processAdd(cmd);
> > > }
> > >
> > >
> > > On Wed, Aug 28, 2019 at 6:15 PM Erick Erickson <
> erickerick...@gmail.com>
> > > wrote:
> > >
> > > > No, you cannot just use the collection name. Replicas are just cores.
> > > > You can host many replicas of a single collection on a single Solr
> node
> > > > in a single CoreContainer (there’s only one per Solr JVM). If you
> just
> > > > specified a collection name how would the code have any clue which
> > > > of the possibilities to return?
> > > >
> > > > The name is in the form collection_shard1_replica_n21
> > > >
> > > > How do you know where the doc you’re working on? Put the ID through
> > > > the hashing mechanism.
> > > >
> > > > This isn’t the same at all if you’re running stand-alone, then
> there’s
> > > only
> > > > one name.
> > > >
> > > > But as I indicated above, your ask for just using the collection name
> > > isn’t
> > > > going to work by definition.
> > > >
> > > > So perhaps this is an XY problem. You’re asking about getCore, which
> is
> > > > a very specific, low-level concept. What are you trying to do at a
> > higher
> > > > level? Why do you think you need to get a core? What do you w

Re: req.getCore().getCoreContainer().getCore(core_name) is returning null - Solr 8.2.0

[Andrea Gazzarini]

Hi Arnold,
why don't you use solrj (in this case a CloudSolrClient) instead of dealing
with such low-level details? The actual location of the document you are
looking for would be completely abstracted.

Best,
Andrea

On Thu, 29 Aug 2019, 18:50 Arnold Bronley,  wrote:

> So, here is the problem that I am trying to solve. I am moving from Solr
> master-slave architecture to SolrCloud architecture. I have one custom Solr
> plugin that does following:
>
> 1. When a document (say document with unique id doc1)is getting indexed to
> a core say core A then this plugin adds one more field to the indexing
> request. It fetches this new field from core B. Core B in our case
> maintains popularity score field for each document which gets calculated in
> a different project. It fetches the popularity score from score B for doc1
> and adds it to indexing request.
> 2. In following code, dataInfo.dataSource is the name of the core B.
>
> I can use the name of the core B like collection_shard1_replica_n21 and it
> works. But it is not a good solution. What if I had a multiple shards for
> core B? In that case the the doc1 that I am trying to find might not be
> present in collection_shard1_replica_n21.
>
> So is there something like,
>
> SolrCollecton dataCollection = getCollection(dataInfo.dataSource);
>
> @Override
> public void processAdd(AddUpdateCommand cmd) throws IOException {
>SolrInputDocument doc = cmd.getSolrInputDocument();
>String uniqueId = getUniqueId(doc);
>
>SolrCore dataCore =
> req.getCore().getCoreContainer().getCore(dataInfo.dataSource);
>
>if (dataCore == null){
>LOG.error("Solr core '{}' to use as data source could not be
> found!  "
>+ "Please check if it is loaded.", dataInfo.dataSource);
>} else{
>
>   Document sourceDoc = getSourceDocument(dataCore, uniqueId);
>
>   if (sourceDoc != null){
>
>   populateDocToBeAddedFromSourceDoc(doc,sourceDoc);
>   }
>}
>
>// pass it up the chain
>super.processAdd(cmd);
> }
>
>
> On Wed, Aug 28, 2019 at 6:15 PM Erick Erickson 
> wrote:
>
> > No, you cannot just use the collection name. Replicas are just cores.
> > You can host many replicas of a single collection on a single Solr node
> > in a single CoreContainer (there’s only one per Solr JVM). If you just
> > specified a collection name how would the code have any clue which
> > of the possibilities to return?
> >
> > The name is in the form collection_shard1_replica_n21
> >
> > How do you know where the doc you’re working on? Put the ID through
> > the hashing mechanism.
> >
> > This isn’t the same at all if you’re running stand-alone, then there’s
> only
> > one name.
> >
> > But as I indicated above, your ask for just using the collection name
> isn’t
> > going to work by definition.
> >
> > So perhaps this is an XY problem. You’re asking about getCore, which is
> > a very specific, low-level concept. What are you trying to do at a higher
> > level? Why do you think you need to get a core? What do you want to _do_
> > with the doc that you need the core it resides in?
> >
> > Best,
> > Erick
> >
> > > On Aug 28, 2019, at 5:28 PM, Arnold Bronley 
> > wrote:
> > >
> > > Wait, would I need to use core name like  collection1_shard1_replica_n4
> > > etc/? Can't I use collection name? What if  I have multiple shards, how
> > > would I know where does the document that I am working with lives in
> > > currently.
> > > I would rather prefer to use collection name and expect the core
> > > information to be abstracted out that way.
> > >
> > > On Wed, Aug 28, 2019 at 5:13 PM Erick Erickson <
> erickerick...@gmail.com>
> > > wrote:
> > >
> > >> Hmmm, should work. What is your core_name? There’s strings like
> > >> collection1_shard1_replica_n4 and core_node6. Are you sure you’re
> using
> > the
> > >> right one?
> > >>
> > >>> On Aug 28, 2019, at 3:56 PM, Arnold Bronley  >
> > >> wrote:
> > >>>
> > >>> Hi,
> > >>>
> > >>> In a custom Solr plugin code,
> > >>> req.getCore().getCoreContainer().getCore(core_name) is returning null
> > >> even
> > >>> if core by name core_name is loaded and up in Solr. req is object
> > >>> of SolrQueryRequest class. I am using Solr 8.2.0 in SolrCloud mode.
> > >>>
> > >>> Any ideas on why this might be the case?
> > >>
> > >>
> >
> >
>

Re: Solr master issue

[Andrea Gazzarini]

I bet this is the problem:

java.nio.file.NoSuchFileException: /solr-m/server/solr/sitecore_c

Do you have any idea about why Solr us not finding that datafile?

Andrea

On Wed, 21 Aug 2019, 14:41 Atita Arora,  wrote:

> I think it would be useful to provide additional details like solr version,
> when did you start seeing the problem? Did you upgrade lately?
> Did you change anything?
>
> On Wed, Aug 21, 2019 at 2:17 PM Akreeti Agarwal  wrote:
>
> > Hi,
> >
> > I am facing issue on my solr master, getting this in logs:
> >
> > 2019-08-19 22:29:55.573 ERROR (qtp59559151-101) [   ]
> o.a.s.s.HttpSolrCall
> > null:org.apache.solr.common.SolrException: SolrCore
> > 'sitecore_contents_index' is not available due to init failure: Error
> > opening new searcher
> > at
> > org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1071)
> > at
> > org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:252)
> > at
> > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:414)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> > at
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> > at
> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> > at
> >
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> > at
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> > at
> org.eclipse.jetty.server.Server.handle(Server.java:499)
> > at
> > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> > at
> >
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> > at org.eclipse.jetty.io
> > .AbstractConnection$2.run(AbstractConnection.java:540)
> > at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> > at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> > at java.lang.Thread.run(Thread.java:748)
> > Caused by: org.apache.solr.common.SolrException: Error opening new
> searcher
> > at
> org.apache.solr.core.SolrCore.(SolrCore.java:820)
> > at
> org.apache.solr.core.SolrCore.(SolrCore.java:658)
> > at
> > org.apache.solr.core.CoreContainer.create(CoreContainer.java:820)
> > at
> > org.apache.solr.core.CoreContainer.access$000(CoreContainer.java:90)
> > at
> > org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:473)
> > at
> > org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:464)
> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> >
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > ... 1 more
> > Caused by: org.apache.solr.common.SolrException: Error opening new
> searcher
> > at
> > org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1696)
> > at
> > org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1807)
> > at
> > org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:914)
> > at
> org.apache.solr.core.SolrCore.(SolrCore.java:793)
> > ... 10 more
> > Caused by: java.nio.file.NoSuchFileException:
> > 

Re: modify query response plugin

[Andrea Gazzarini]

I'm not really sure I got what you want to achieve, but after reading

" I am trying to manipulate the data that I already have in the response,
...
I can see how I can access the response and construct a new one."

an option to consider could be a query response writer [1].

Cheers,
Andrea

[1] https://lucene.apache.org/solr/guide/6_6/response-writers.html


On Thu, 8 Aug 2019, 23:34 Atita Arora,  wrote:

> Isn't it resolved by simply adding the desired pre/post tags in highlighter
> request?
>
> On Thu, Aug 8, 2019 at 11:20 PM Moyer, Brett  wrote:
>
> > Highlight? What about using the Highlighter?
> > https://lucene.apache.org/solr/guide/6_6/highlighting.html
> >
> > Brett Moyer
> > Manager, Sr. Technical Lead | TFS Technology
> >   Public Production Support
> >   Digital Search & Discovery
> >
> > 8625 Andrew Carnegie Blvd | 4th floor
> > Charlotte, NC 28263
> > Tel: 704.988.4508
> > Fax: 704.988.4907
> > bmo...@tiaa.org
> >
> >
> > -Original Message-
> > From: Maria Muslea 
> > Sent: Thursday, August 8, 2019 1:28 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: modify query response plugin
> >
> > Thank you for your response. I believe that the Tagger is used for NER,
> > which is different than what I am trying to do.
> > It is also available only with Solr 7 and I would need this to work with
> > version 6.5.0.
> >
> > I am trying to manipulate the data that I already have in the response,
> > and I can't find a good example of a plugin that does something similar,
> so
> > I can see how I can access the response and construct a new one.
> >
> > Your help is greatly appreciated.
> >
> > Thank you,
> > Maria
> >
> > On Tue, Aug 6, 2019 at 3:19 PM Erik Hatcher 
> > wrote:
> >
> > > I think you’re looking for the Solr Tagger, described here:
> > > https://lucidworks.com/post/solr-tagger-improving-relevancy/
> > >
> > > > On Aug 6, 2019, at 16:04, Maria Muslea 
> wrote:
> > > >
> > > > Hi,
> > > >
> > > > I am trying to implement a plugin that will modify my query
> > > > response. For example, I would like to execute a query that will
> > return something like:
> > > >
> > > > {...
> > > > "description":"flights at LAX",
> > > > "highlight":"airport;11;3"
> > > > ...}
> > > > This is information that I have in my document, so I can return it.
> > > >
> > > > Now, I would like the plugin to intercept the result, do some
> > > > processing
> > > on
> > > > it, and return something like:
> > > >
> > > > {...
> > > > "description":"flights at LAX",
> > > > "highlight":{
> > > >   "concept":"airport",
> > > >   "description":"flights at LAX"
> > > > ...}
> > > >
> > > > I looked at some RequestHandler implementations, but I can't find
> > > > any sample code that would help me with this. Would this type of
> > > > plugin be handled by a RequestHandler? Could you maybe point me to a
> > > > sample plugin that does something similar?
> > > >
> > > > I would really appreciate your help.
> > > >
> > > > Thank you,
> > > > Maria
> > >
> > *
> > This e-mail may contain confidential or privileged information.
> > If you are not the intended recipient, please notify the sender
> > immediately and then delete it.
> >
> > TIAA
> > *
> >
>

Re: searching only within a date range

[Andrea Gazzarini]

Hi Mark, you are using a "range facet" which is a "query-shape" feature, 
it doesn't have any constraint on the results (i.e. it doesn't filter at 
all).
You need to add a filter query [1] with a date range clause (e.g. 
fq=field:[ TO or *>]).


Best,
Andrea

[1] 
https://lucene.apache.org/solr/guide/6_6/common-query-parameters.html#CommonQueryParameters-Thefq_FilterQuery_Parameter

[2] https://lucene.apache.org/solr/guide/6_6/working-with-dates.html

On 07/06/2019 14:02, Mark Fenbers - NOAA Federal wrote:

Hello!

I have a search setup and it works fine.  I search a text field called
"logtext" in a database table.  My Java code is like this:

SolrQuery query - new SolrQuery();
query.setQuery(searchWord);
query.setParam("df", "logtext");

Then I execute the search... and it works just great.  But now I want to
add a constraint to only search for the "searchWord" within a certain range
of time -- given timestamps in the column called "posttime".  So, I added
the code in bold below:

SolrQuery query - new SolrQuery();
query.setQuery(searchWord);
*query.setFacet(true);*
*query.addDateRangeFacet("posttime", new Date(System.currentTimeMillis() -
1000L * 86400L * 365L), new Date(System.currentTimeMillis()), "+1DAY"); /*
from 1 year ago to present) */*
query.setParam("df", "logtext");

But this gives me a complaint: *undefined field: "posttime"* so I clearly
do not understand the arguments needed to addDateRangeFacet().  Can someone
help me determine the proper code for doing what I want?

Further, I am puzzled about the "gap" argument [last one in
addDateRangeFacet()].  What does this do?  I used +1DAY, but I really have
no idea the purpose of this.  I haven't found any documentation that
explains this well.

Mark

Solrj, Json Facets, (Date) stats facets

[Andrea Gazzarini]

Good morning guys, I have a questions about Solrj and JSON facets.

I'm using Solr 7.7.1 and I'm sending a request like this:

json.facet={x:'max(iterationTimestamp)'}

where "iterationTimestamp" is a solr.DatePointField. The JSON response 
correctly includes what I'm expecting:


    "facets": {
    "count": 8,
    "x": "1973-09-20T17:33:18.700Z"
    }

but Solrj doesn't. Specifically, the jsonFacetingResponse contains only 
the domainCount attribute (8).
Looking at the code I see that in NestableJsonFacet a stats is taken in 
account only if the corresponding value is an instance of Number (and x 
in the example above is a java.util.Date).


Is that expected? Is there a way (other than dealing with nested 
NamedLists) for retrieving that value?


Cheers,
Andrea

Re: Solr Index Size after reindex

[Andrea Gazzarini]

Yes, those numbers are different and that should explain the different 
size. I think you should be able to find some information in the 
Alfresco or Solr log. There must be a reason about the missing content. 
For example, are those numbers coming from two comparable snapshots? In 
other words, I imagine that at a given moment X you rsync-ed the two servers


 * 5.365.213 is the numDocs you got just after the sync, isn't it?
 * 4.537.651 is the numDocs you got in the staging server after the
   reindexing isn't it? Are you sure the whole reindexing is completed?

MaxDocs is the number of documents you have in the index including the 
deleted docs not yet cleared by a merge. In the console you should also 
see the "Deleted docs" count which should be equal to (maxdocs - numdocs)


Ciao

Andrea

On 08/02/2019 15:53, Mathieu Menard wrote:


Hi Andrea,

I’ve checked this information and here is the result:



PRODUCTION



STAGING

*numDocs*



5.365.213



4.537.651

*MaxDoc*



5.845.469



5.129.556

It seems that there is more than 800.00 docs in PRODUCTION that will 
explain the size of indexes more important. But there is a thing that 
I don’t understand, we have copied the DB and the contenstore the 
numDocs for the two environments should be the same no?


Could you also explain me the meaning of the maxDocs value pleases?

Thanks

Matthieu

*From:*Andrea Gazzarini [mailto:a.gazzar...@sease.io]
*Sent:* vendredi 8 février 2019 14:54
*To:* solr-user@lucene.apache.org
*Subject:* Re: Solr Index Size after reindex

Hi Mathieu,
what about the docs in the two infrastructures? Do they have the same 
numbers (numdocs / maxdocs)? Any meaningful message (error or not) in 
log files?


Andrea

On 08/02/2019 14:19, Mathieu Menard wrote:

Hello,

I would like to have your point of view about an observation we
have made on our two alfresco install (Production and Staging
environment) and more specifically on the size of our solr indexes
on these two environments.

Regularly we do a rsync between the Production and the Staging
environment, we make a copy of the Alfresco’s DB and a copy of the
entire contenstore after that we reindex all the alfresco content.

We have noticed that for the production environment we have 19 Gb
of indexes while in the staging we have “only” 11. Gb of indexes.
We have some difficulties to understand this difference because we
assume that the indexes optimization in the same for a full
reindex or for the normal use of solr.

I’ve verified the configuration between the two solr instances and
I don’t see any differences could you help me to better understand
 this phenomenon.

Here you can find some information about our two environment, if
you need more details, I will give you as soon as possible:



PRODUCTION



STAGING

Alfresco version



5.1.1.4



5.1.1.4

Solr Version





Java version





Linux Machine



See Staging_caracteristics.txt file in attachment



See Staging_caracteristics.txt file in attachment

Please let me know if you any other information I will sent it to
you rapidly.

Kind Regards

Matthieu

Re: Solr Index Size after reindex

[Andrea Gazzarini]

Hi Mathieu,
what about the docs in the two infrastructures? Do they have the same 
numbers (numdocs / maxdocs)? Any meaningful message (error or not) in 
log files?


Andrea

On 08/02/2019 14:19, Mathieu Menard wrote:


Hello,

I would like to have your point of view about an observation we have 
made on our two alfresco install (Production and Staging environment) 
and more specifically on the size of our solr indexes on these two 
environments.


Regularly we do a rsync between the Production and the Staging 
environment, we make a copy of the Alfresco’s DB and a copy of the 
entire contenstore after that we reindex all the alfresco content.


We have noticed that for the production environment we have 19 Gb of 
indexes while in the staging we have “only” 11. Gb of indexes. We have 
some difficulties to understand this difference because we assume that 
the indexes optimization in the same for a full reindex or for the 
normal use of solr.


I’ve verified the configuration between the two solr instances and I 
don’t see any differences could you help me to better understand  this 
phenomenon.


Here you can find some information about our two environment, if you 
need more details, I will give you as soon as possible:




PRODUCTION



STAGING

Alfresco version



5.1.1.4



5.1.1.4

Solr Version





Java version





Linux Machine



See Staging_caracteristics.txt file in attachment



See Staging_caracteristics.txt file in attachment

Please let me know if you any other information I will sent it to you 
rapidly.


Kind Regards

Matthieu

Re: copyfield not working

[Andrea Gazzarini]

Hi Jay, the text analysis always operates on the indexed content. The 
stored content of a filed is left untouched unless you do something 
before it gets indexed (e.g. on client side or by an 
UpdateRequestProcessor).


Cheers,
Andrea

On 14/01/2019 08:46, Jay Potharaju wrote:

Hi,
I have a copy field in which i am copying the contents of text_en field to
another custom field.
After indexing i was expecting any of the special characters in the
paragraph to be removed, but it does not look like that is happening. The
copied content is same as the what is there in the source. I ran analysis
...looks like the pattern matching works as expected and the special
characters are removed.

Any suggestions?
  <
charFilter class="solr.PatternReplaceCharFilterFactory" pattern=
"['!#\$%'\(\)\*+,-\./:;=?@\[\]\^_`{|}~!@#$%^*]" /><
filter class="solr.EnglishPossessiveFilterFactory"/>   

Thanks
Jay

Re: Solr Request Handler

[Andrea Gazzarini]

Hi,
What Alexander said is right, but if in your scenario you would still go
for that, you could try this [1], that should fit your need.

Best,
Andrea

[1]  https://github.com/SeaseLtd/composite-request-handler

On Mon, 3 Dec 2018, 13:26 Alexandre Rafalovitch  You should not be exposing Solr directly to the client, but treating
> it more as a database.
>
> Given that, why would you not write your processing code in that
> middle-ware layer?
>
> Regards,
>Alex.
> On Mon, 3 Dec 2018 at 06:43, Lucky Sharma  wrote:
> >
> > Hi have one scenario,
> > where I need to make a sequential call to the solr. Both the requests
> > are sequential and output of the first will be required to set some
> > params of the second search request.
> >
> > So for such scenarios, I am writing a plugin which will internally
> > handle both the requests and gives the final response as output.
> >
> > Is this the correct approach?
> > The only assumption I am making here is that both the cores are
> > locally available.
> >
> > --
> > Warm Regards,
> >
> > Lucky Sharma
> > Contact No: +91 9821559918
>

Re: qf an pf parameters question

[Andrea Gazzarini]

Hi, you may be interested in the mm [1] parameter, which in this case 
shout be set to 100%. However, if your requirements are more complicated 
than this, an mm=100% could have some unwanted side-effects because it's 
very "rigid".


Best,
Andrea

[1] 
https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Themm_MinimumShouldMatch_Parameter


On 16/10/18 13:36, Kempelen, Ákos wrote:

Hello,

I use edismax qf and pf parameters, but for some field it would be desired to 
match ONLY  if terms are in the same phrase.
However, the  qf parameter does not work with phrases, it is just for 
"filtering", and pf params affects only scoring.

For example:
qf=fulltext subject
pf=fulltext subject^5

subjects in one document1:
- harry potter
- star wars

q=harry star
will match document1, but it is a wrong result.

Any idea how to solve this?
Akos

Re: Query with exact number of tokens

[Andrea Gazzarini]

Oops, sorry...too much rush in reading, I didn't read the second part.

Please forget my answer ;)

Andrea

On 21/09/18 15:52, Andrea Gazzarini wrote:

Hi Sergio,
assuming that you don't want to disable tokenisation (otherwise you 
can define the indexed field as a string and search it as a whole),
in "Relevant Search" the authors describe a cool approach using the so 
called "Sentinel Tokens", which are symbolic tokens representing the 
beginning and the end of a value (field value or query).


SENTINEL_BEGINSENTINEL_END

Those tokens could be injected at index and query time so the returned 
matches will be effectively "exact" matches. Matching docs will have 
exact values matching (that actually depends on the text analysis you 
apply) and the sentinels in the expected place (beginning + end)


Best,
Andrea

On 21/09/18 15:00, marotosg wrote:

Hi,

I have to search for company names where my first requirement is to find
only exact matches on the company name.

For instance if I search for "CENTURY BANCORP, INC." I shouldn't find 
"NEW

CENTURY BANCORP, INC."
because the result company has the extra keyword "NEW".

I can't use exact match because the sequence of tokens may differ. 
Basically
I need to find results where the  tokens are the same in any order 
and the

number of tokens match.

I have no idea if it's possible as include in the query the number of 
tokens

and solr field has that info within to match it.

Thanks for your help
Sergio



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Query with exact number of tokens

[Andrea Gazzarini]

Hi Sergio,
assuming that you don't want to disable tokenisation (otherwise you can 
define the indexed field as a string and search it as a whole),
in "Relevant Search" the authors describe a cool approach using the so 
called "Sentinel Tokens", which are symbolic tokens representing the 
beginning and the end of a value (field value or query).


SENTINEL_BEGINSENTINEL_END

Those tokens could be injected at index and query time so the returned 
matches will be effectively "exact" matches. Matching docs will have 
exact values matching (that actually depends on the text analysis you 
apply) and the sentinels in the expected place (beginning + end)


Best,
Andrea

On 21/09/18 15:00, marotosg wrote:

Hi,

I have to search for company names where my first requirement is to find
only exact matches on the company name.

For instance if I search for "CENTURY BANCORP, INC." I shouldn't find "NEW
CENTURY BANCORP, INC."
because the result company has the extra keyword "NEW".

I can't use exact match because the sequence of tokens may differ. Basically
I need to find results where the  tokens are the same in any order and the
number of tokens match.

I have no idea if it's possible as include in the query the number of tokens
and solr field has that info within to match it.

Thanks for your help
Sergio



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Explode kind of function in Solr

[Andrea Gazzarini]

Hi,
as far as I know, this is not possible. Solr is document-oriented, 
meaning with that its datamodel works using a "document" level of 
granularity. If you index A, you get A.

I see a couple of chances (maybe someone else could add other options):

 * index exactly what you need: in your case I would index *3 docs*
   with a grouping criteria *"1"* (it is your current id, which cannot
   be an id in this case)
 * index 1 document with multivalued "phone" id (like in your example),
   and have an intermediate API layer or directly the frontend which
   "expands" the result as you want.

Best,
Andrea

On 13/09/18 12:20, Rushikesh Garadade wrote:

Hello All,
Is there any functionality in solr that can convert (explode) results from
1 document to many docuement.
*Example: *
Lets say I have doc:
{
id:1,
phone: [11,22,33]
}

when I query to solr with id=1 I want result as below:
[{
id:1,
phone:11
},
{
id:1,
phone:22
},
{
d:1,
phone:33
}]

Please let me know if this is possible in Solr , if Yes how?

Thanks,
Rushikesh Garadade

Re: 20180913 - Clarification about Limitation

[Andrea Gazzarini]

Hi Rekna,
I think nobody can seriously answer to your questions. The only 
"serious" answer to your questions, which doesn't help you, is "it 
depends"; specifically, it depends on what is / are your goal / goals, 
context and so on.


Is not possible, at least in my opinion, to provide such answers under 
an absolute perspective because they shouldn't make so much sense. Of 
course, there are some "hard" limits (if you have 10GB of available disk 
space, your index cannot be greater than that) but I believe you can 
ignore that: it's better if you focus on (and provide us some other 
details about) your objectives.


Best,
Andrea

On 13/09/18 10:07, Rekha wrote:

Hi Solr Team,
I am new to SOLR. I need following clarification from you.
How many documents can be stored in one core?   Is 
there any limit for number of fields per document?   How many 
Cores can be created in on SOLR?Is there any other 
limitation is there based on the Disk storage size? I mean some of the database has 
the 10 GM limit, I have asked like that. Can we use SOLR as a 
database?  
Thanks,Rekha Karthick

Re: Silent error from DataImportHandler

[Andrea Gazzarini]

I cannot give you detailed instructions as I don't have in front of me a 
Solr console with the dataimport enabled, but I remember that there's a 
detailed section which reports a lot of information.


In the meantime: shooting in the dark, if the query is working and 
"total rows fetched" = 0, then the query is not working for some reason: 
can you check if some clause in your SQL include < or > ? They need to 
be escaped ( )


Andrea


On 10/09/2018 17:22, Monique Monteiro wrote:

This is shown in the section "Raw Debug-Response".

On Mon, Sep 10, 2018 at 12:20 PM Andrea Gazzarini 
wrote:


Hi Monique, this is the output; when you check the debug checkbox
another section is printed

Andrea

On 10/09/2018 17:19, Monique Monteiro wrote:

Text:

{ "responseHeader": { "status": 0, "QTime": 463 }, "initArgs": [

"defaults",

[ "config", "data-cnpj-config.xml" ] ], "command": "full-import", "mode":
"debug", "documents": [], "verbose-output": [], "status": "idle","
importResponse": "", "statusMessages": { "Time Elapsed": "0:0:0.432",

"Total

Requests made to DataSource": "1", "Total Rows Fetched": "0", "Total
Documents Processed": "0", "Total Documents Skipped": "0", "Full Dump
Started": "2018-09-10 12:04:03", "Full Import failed": "2018-09-10

12:04:04"

} }

On Mon, Sep 10, 2018 at 12:08 PM Andrea Gazzarini 
wrote:


Hi Monique,
I think you cannot attach files / images, please post, if available, the
url of the image or a text description.

Andrea

On 10/09/2018 17:05, Monique Monteiro wrote:

Hi Andrea,

Solr console doesn't return a very different information even with
debug mode enabled:

image.png

On Mon, Sep 10, 2018 at 12:00 PM Andrea Gazzarini
mailto:a.gazzar...@sease.io>> wrote:

  You can check the solr.log or the solr-console.log. Another option
  is to
  activate the debug mode in the Solr console before running the
  data import.

  Andrea

  On 10/09/2018 16:57, Monique Monteiro wrote:
  > Hi all,
  >
  > I have a data import handler configured with an Oracle SQL query
  which
  > works like a charm.  However, when I have the same query
  configured in
  > Solr's data import handler, nothing happens, and it returns:
  >
  >
  >
  >  "*Total Requests made to DataSource*": "1",
  >
  >  "*Total Rows Fetched*": "0",
  >
  >  "*Total Documents Processed*": "0",
  >
  >  "*Total Documents Skipped*": "0",
  >
  > "Full Dump Started": "2018-09-06 18:15:59", "Full Import
  failed": "2018-09-06
  > 18:16:02"
  >
  > Has anyone any ideas about what may be happening?  Is there any
  log file
  > which can tell the error?
  >
  > Thanks in advance,
  >



--
Monique Monteiro
Twitter: http://twitter.com/monilouise

Re: Silent error from DataImportHandler

[Andrea Gazzarini]

Hi Monique, this is the output; when you check the debug checkbox 
another section is printed


Andrea

On 10/09/2018 17:19, Monique Monteiro wrote:

Text:

{ "responseHeader": { "status": 0, "QTime": 463 }, "initArgs": [ "defaults",
[ "config", "data-cnpj-config.xml" ] ], "command": "full-import", "mode":
"debug", "documents": [], "verbose-output": [], "status": "idle","
importResponse": "", "statusMessages": { "Time Elapsed": "0:0:0.432", "Total
Requests made to DataSource": "1", "Total Rows Fetched": "0", "Total
Documents Processed": "0", "Total Documents Skipped": "0", "Full Dump
Started": "2018-09-10 12:04:03", "Full Import failed": "2018-09-10 12:04:04"
} }

On Mon, Sep 10, 2018 at 12:08 PM Andrea Gazzarini 
wrote:


Hi Monique,
I think you cannot attach files / images, please post, if available, the
url of the image or a text description.

Andrea

On 10/09/2018 17:05, Monique Monteiro wrote:

Hi Andrea,

Solr console doesn't return a very different information even with
debug mode enabled:

image.png

On Mon, Sep 10, 2018 at 12:00 PM Andrea Gazzarini
mailto:a.gazzar...@sease.io>> wrote:

 You can check the solr.log or the solr-console.log. Another option
 is to
 activate the debug mode in the Solr console before running the
 data import.

 Andrea

 On 10/09/2018 16:57, Monique Monteiro wrote:
 > Hi all,
 >
 > I have a data import handler configured with an Oracle SQL query
 which
 > works like a charm.  However, when I have the same query
 configured in
 > Solr's data import handler, nothing happens, and it returns:
 >
 >
 >
 >  "*Total Requests made to DataSource*": "1",
 >
 >  "*Total Rows Fetched*": "0",
 >
 >  "*Total Documents Processed*": "0",
 >
 >  "*Total Documents Skipped*": "0",
 >
 > "Full Dump Started": "2018-09-06 18:15:59", "Full Import
 failed": "2018-09-06
 > 18:16:02"
 >
 > Has anyone any ideas about what may be happening?  Is there any
 log file
 > which can tell the error?
 >
 > Thanks in advance,
 >



--
Monique Monteiro
Twitter: http://twitter.com/monilouise

Re: Silent error from DataImportHandler

[Andrea Gazzarini]

Hi Monique,
I think you cannot attach files / images, please post, if available, the 
url of the image or a text description.


Andrea

On 10/09/2018 17:05, Monique Monteiro wrote:

Hi Andrea,

Solr console doesn't return a very different information even with 
debug mode enabled:


image.png

On Mon, Sep 10, 2018 at 12:00 PM Andrea Gazzarini 
mailto:a.gazzar...@sease.io>> wrote:


You can check the solr.log or the solr-console.log. Another option
is to
activate the debug mode in the Solr console before running the
data import.

Andrea

On 10/09/2018 16:57, Monique Monteiro wrote:
> Hi all,
>
> I have a data import handler configured with an Oracle SQL query
which
> works like a charm.  However, when I have the same query
configured in
> Solr's data import handler, nothing happens, and it returns:
>
>
>
>      "*Total Requests made to DataSource*": "1",
>
>      "*Total Rows Fetched*": "0",
>
>      "*Total Documents Processed*": "0",
>
>      "*Total Documents Skipped*": "0",
>
> "Full Dump Started": "2018-09-06 18:15:59", "Full Import
failed": "2018-09-06
> 18:16:02"
>
> Has anyone any ideas about what may be happening?  Is there any
log file
> which can tell the error?
>
> Thanks in advance,
>



--
Monique Monteiro
Twitter: http://twitter.com/monilouise

Re: Silent error from DataImportHandler

[Andrea Gazzarini]

You can check the solr.log or the solr-console.log. Another option is to 
activate the debug mode in the Solr console before running the data import.


Andrea

On 10/09/2018 16:57, Monique Monteiro wrote:

Hi all,

I have a data import handler configured with an Oracle SQL query which
works like a charm.  However, when I have the same query configured in
Solr's data import handler, nothing happens, and it returns:



 "*Total Requests made to DataSource*": "1",

 "*Total Rows Fetched*": "0",

 "*Total Documents Processed*": "0",

 "*Total Documents Skipped*": "0",

"Full Dump Started": "2018-09-06 18:15:59", "Full Import failed": "2018-09-06
18:16:02"

Has anyone any ideas about what may be happening?  Is there any log file
which can tell the error?

Thanks in advance,

Re: SynonimGraphFilter expands wrong synonims

[Andrea Gazzarini]

And as you probably already checked, inserting the proper 
*tokenizerFactory* also expands the right synonym line:


q = (body:"Cytosolic 5'-nucleotidase II"  OR body:"EC 3.1.3.5")

parsedQuery = SpanOrQuery(spanOr([body:p49902, spanNear([body:cytosol, 
body:purin, body:5, body:nucleotidas], 0, true), spanNear([body:ec, 
body:3.1.3.5], 0, true), spanNear([body:cytosol, body:5, 
body:nucleotidas, body:ii], 0, true)])) SpanOrQuery(spanOr([body:p49902, 
spanNear([body:cytosol, body:purin, body:5, body:nucleotidas], 0, true), 
spanNear([body:cytosol, body:5, body:nucleotidas, body:ii], 0, true), 
spanNear([body:ec, body:3.1.3.5], 0, true)]))


Best,
Andrea

On 05/09/18 16:10, Andrea Gazzarini wrote:


You're right, my answer forgot to mention the *tokenizerFactory* 
parameter that you can add in the filter declaration. But, differently 
from what you think the default tokenizer used for parsing the 
synonyms _is not_ the tokenizer of the current analyzer 
(StandardTokenizer in your example) but WhitespaceTokenizer. See here 
[1] for a complete description of the filter capabilities.


So instead of switching the analyzer tokenizer you could also add a 
tokenizerFactory="solr.StandardTokenizerFactory" in the synonym filter 
declaration.


Best,
Andrea

[1] 
https://lucene.apache.org/solr/guide/6_6/filter-descriptions.html#FilterDescriptions-SynonymGraphFilter


On 05/09/2018 15:58, Danilo Tomasoni wrote:

Hi Andrea,

thank you for your answer.

About the second question: The standardTokenizer should be applied 
also to the phrase query, so the ' and - symbols should be removed 
even there, and this should allow a match in the synonim file isn't it?


With an example:


in phrase query:

"Cytosolic 5'-nucleotidase II" -> standardTokenizer -> Cytosolic, 5, 
nucleotidase, II



in synonym parsing:

...,Cytosolic 5'-nucleotidase II,... -> standardTokenizer -> 
Cytosolic, 5, nucleotidase, II



So the two graphs should match.. or I'm wrong?
Thank you
Danilo

ody:On 05/09/2018 13:23, Andrea Gazzarini wrote:

Hi Danilo,
let's see if this can help you (I'm sorry for the poor debugging, 
I'm reading & writing from my mobile): the first issue should have 
something to do with synonym overlapping and since I'm very curious 
about what it is happening, I will be more precise when I will be in 
front of a laptop.


The second: I guess the main problem is the StandardTokenizer, which 
removes the ' and - symbols. That should be the reason why you don't 
have any synonym detection. You should replace it with a 
WhitespaceTokenizer but, be aware that if you do that, the 
apostrophe in the document ( ′ ) is not the same symbol ( ' ) you've 
used in the query and in the synonyms file, so you need to replace 
it somewhere (in the document and/or in the query) otherwise you 
won't have any match.


HTH
Gazza

On 05/09/2018 12:19, Danilo Tomasoni wrote:

Hello to all,

I have an issue related to synonimgraphfilter expanding the wrong 
synonims for a phrase-term at query time.


I have a dictionary with the following lines

P49902,Cytosolic purine 5'-nucleotidase,EC 3.1.3.5,Cytosolic 
5'-nucleotidase II
A8K9N1,Glucosidase\, beta\, acid 3,Cytosolic,Glucosidase\, beta\, 
acid 3,Cytosolic\, isoform CRA_b,cDNA FLJ78196\, highly similar to 
Homo sapiens glucosidase\, beta\, acid 3,cytosolic,GBA3\, 
mRNA,cDNA\, FLJ93688\, Homo sapiens glucosidase\, beta\, acid 
3,cytosolic,GBA3\, mRNA


and two documents

{"body":"8. The method of claim 6 wherein said method inhibits at 
least one 5′-nucleotidase chosen from cytosolic 5′-nucleotidase II 
(cN-II), cytosolic 5′-nucleotidase IA (cN-IA), cytosolic 
5′-nucleotidase IB (cN-IB), cytosolic 5′-nucleotidase IMA 
(cN-IIIA), cytosolic 5′-nucleotidase NIB (cN-IIIB), 
ecto-5′-nucleotidase (eN, CD73), cytosolic 5′(3′)-deoxynucleotidase 
(cdN) and mitochondrial 5′(3′)-deoxynucleotidase (mdN)."}
{"body":"Trichomonosis caused by the flagellate protozoan 
Trichomonas vaginalis represents the most prevalent nonviral 
sexually transmitted disease worldwide (WHO-DRHR 2012). In women, 
the symptoms are cyclic and often worsen around the menstruation 
period. In men, trichomonosis is largely asymptomatic and these men 
are considered to be carriers of T. vaginalis (Petrin et al. 1998). 
This infection has been associated with birth outcomes (Klebanoff 
et al. 2001), infertility (Grodstein et al. 1993), cervical and 
prostate cancer (Viikki et al. 2000, Sutcliffe et al. 2012) and 
pelvic inflammatory disease (Cherpes et al. 2006). Importantly, T. 
vaginalis is a co-factor in human immunodeficiency virus 
transmission and acquisition (Sorvillo et al. 2001, Van Der Pol et 
al. 2008). Therefore, it is important to study the host-parasite 
relationship to understand T. vaginalis infection and pathogenesis. 
Colonisation of the mucosa by T. vaginalis is a complex multi-step 
process that involves distinct mechanisms 

Re: regarding 'sharedlib' in solr

[Andrea Gazzarini]

Hi, please expand a bit. Specifically:

 * what are those text files? Configuration files? You want something
   like a central point where to manage things like stopwords, synonyms?
 * I don't think the shareLib folder has been created for this usage.
   However, please post the complete message you're receiving, together
   with the stacktrace, if available.

Andrea

On 07/09/18 12:02, Abhishek Agarwal wrote:

Hi,

I want to share a folder containing text files in solr among different
cores so if the folder is updated ,so it would reflect in all the cores
having path specified but the problem I am facing is that , i am using
sharedlib in solr.xml and specifying the default path there.And also I am
updating schema.xml of my core but when I am loading core ,it is giving
error 'unsafe loading' .and not getting reload. Please help me in this.

Re: SynonimGraphFilter expands wrong synonims

[Andrea Gazzarini]

You're right, my answer forgot to mention the *tokenizerFactory* 
parameter that you can add in the filter declaration. But, differently 
from what you think the default tokenizer used for parsing the synonyms 
_is not_ the tokenizer of the current analyzer (StandardTokenizer in 
your example) but WhitespaceTokenizer. See here [1] for a complete 
description of the filter capabilities.


So instead of switching the analyzer tokenizer you could also add a 
tokenizerFactory="solr.StandardTokenizerFactory" in the synonym filter 
declaration.


Best,
Andrea

[1] 
https://lucene.apache.org/solr/guide/6_6/filter-descriptions.html#FilterDescriptions-SynonymGraphFilter


On 05/09/2018 15:58, Danilo Tomasoni wrote:

Hi Andrea,

thank you for your answer.

About the second question: The standardTokenizer should be applied 
also to the phrase query, so the ' and - symbols should be removed 
even there, and this should allow a match in the synonim file isn't it?


With an example:


in phrase query:

"Cytosolic 5'-nucleotidase II" -> standardTokenizer -> Cytosolic, 5, 
nucleotidase, II



in synonym parsing:

...,Cytosolic 5'-nucleotidase II,... -> standardTokenizer -> 
Cytosolic, 5, nucleotidase, II



So the two graphs should match.. or I'm wrong?
Thank you
Danilo

ody:On 05/09/2018 13:23, Andrea Gazzarini wrote:

Hi Danilo,
let's see if this can help you (I'm sorry for the poor debugging, I'm 
reading & writing from my mobile): the first issue should have 
something to do with synonym overlapping and since I'm very curious 
about what it is happening, I will be more precise when I will be in 
front of a laptop.


The second: I guess the main problem is the StandardTokenizer, which 
removes the ' and - symbols. That should be the reason why you don't 
have any synonym detection. You should replace it with a 
WhitespaceTokenizer but, be aware that if you do that, the apostrophe 
in the document ( ′ ) is not the same symbol ( ' ) you've used in the 
query and in the synonyms file, so you need to replace it somewhere 
(in the document and/or in the query) otherwise you won't have any 
match.


HTH
Gazza

On 05/09/2018 12:19, Danilo Tomasoni wrote:

Hello to all,

I have an issue related to synonimgraphfilter expanding the wrong 
synonims for a phrase-term at query time.


I have a dictionary with the following lines

P49902,Cytosolic purine 5'-nucleotidase,EC 3.1.3.5,Cytosolic 
5'-nucleotidase II
A8K9N1,Glucosidase\, beta\, acid 3,Cytosolic,Glucosidase\, beta\, 
acid 3,Cytosolic\, isoform CRA_b,cDNA FLJ78196\, highly similar to 
Homo sapiens glucosidase\, beta\, acid 3,cytosolic,GBA3\, 
mRNA,cDNA\, FLJ93688\, Homo sapiens glucosidase\, beta\, acid 
3,cytosolic,GBA3\, mRNA


and two documents

{"body":"8. The method of claim 6 wherein said method inhibits at 
least one 5′-nucleotidase chosen from cytosolic 5′-nucleotidase II 
(cN-II), cytosolic 5′-nucleotidase IA (cN-IA), cytosolic 
5′-nucleotidase IB (cN-IB), cytosolic 5′-nucleotidase IMA (cN-IIIA), 
cytosolic 5′-nucleotidase NIB (cN-IIIB), ecto-5′-nucleotidase (eN, 
CD73), cytosolic 5′(3′)-deoxynucleotidase (cdN) and mitochondrial 
5′(3′)-deoxynucleotidase (mdN)."}
{"body":"Trichomonosis caused by the flagellate protozoan 
Trichomonas vaginalis represents the most prevalent nonviral 
sexually transmitted disease worldwide (WHO-DRHR 2012). In women, 
the symptoms are cyclic and often worsen around the menstruation 
period. In men, trichomonosis is largely asymptomatic and these men 
are considered to be carriers of T. vaginalis (Petrin et al. 1998). 
This infection has been associated with birth outcomes (Klebanoff et 
al. 2001), infertility (Grodstein et al. 1993), cervical and 
prostate cancer (Viikki et al. 2000, Sutcliffe et al. 2012) and 
pelvic inflammatory disease (Cherpes et al. 2006). Importantly, T. 
vaginalis is a co-factor in human immunodeficiency virus 
transmission and acquisition (Sorvillo et al. 2001, Van Der Pol et 
al. 2008). Therefore, it is important to study the host-parasite 
relationship to understand T. vaginalis infection and pathogenesis. 
Colonisation of the mucosa by T. vaginalis is a complex multi-step 
process that involves distinct mechanisms (Alderete et al. 2004). 
The parasite interacts with mucin (Lehker & Sweeney 1999), adheres 
to vaginal epithelial cells (VECs) in a process mediated by adhesion 
proteins (AP120, AP65, AP51, AP33 and AP23) and undergoes dramatic 
morphological changes from a pyriform to an amoeboid form (Engbring 
& Alderete 1998, Kucknoor et al. 2005, Moreno-Brito et al. 2005). 
After adhesion to VECs, the synthesis and gene expression of 
adhesins are increased (Kucknoor et al. 2005). These mechanisms must 
be tightly regulated and iron plays a pivotal role in this 
regulation. Iron is an essential element for all living organisms, 
from the most primitive to the most complex, as a component of 
haeme, iron-sulphur clusters 

Re: SynonimGraphFilter expands wrong synonims

[Andrea Gazzarini]

Hi Danilo,
let's see if this can help you (I'm sorry for the poor debugging, I'm 
reading & writing from my mobile): the first issue should have something 
to do with synonym overlapping and since I'm very curious about what it 
is happening, I will be more precise when I will be in front of a laptop.


The second: I guess the main problem is the StandardTokenizer, which 
removes the ' and - symbols. That should be the reason why you don't 
have any synonym detection. You should replace it with a 
WhitespaceTokenizer but, be aware that if you do that, the apostrophe in 
the document ( ′ ) is not the same symbol ( ' ) you've used in the query 
and in the synonyms file, so you need to replace it somewhere (in the 
document and/or in the query) otherwise you won't have any match.


HTH
Gazza

On 05/09/2018 12:19, Danilo Tomasoni wrote:

Hello to all,

I have an issue related to synonimgraphfilter expanding the wrong 
synonims for a phrase-term at query time.


I have a dictionary with the following lines

P49902,Cytosolic purine 5'-nucleotidase,EC 3.1.3.5,Cytosolic 
5'-nucleotidase II
A8K9N1,Glucosidase\, beta\, acid 3,Cytosolic,Glucosidase\, beta\, acid 
3,Cytosolic\, isoform CRA_b,cDNA FLJ78196\, highly similar to Homo 
sapiens glucosidase\, beta\, acid 3,cytosolic,GBA3\, mRNA,cDNA\, 
FLJ93688\, Homo sapiens glucosidase\, beta\, acid 3,cytosolic,GBA3\, mRNA


and two documents

{"body":"8. The method of claim 6 wherein said method inhibits at 
least one 5′-nucleotidase chosen from cytosolic 5′-nucleotidase II 
(cN-II), cytosolic 5′-nucleotidase IA (cN-IA), cytosolic 
5′-nucleotidase IB (cN-IB), cytosolic 5′-nucleotidase IMA (cN-IIIA), 
cytosolic 5′-nucleotidase NIB (cN-IIIB), ecto-5′-nucleotidase (eN, 
CD73), cytosolic 5′(3′)-deoxynucleotidase (cdN) and mitochondrial 
5′(3′)-deoxynucleotidase (mdN)."}
{"body":"Trichomonosis caused by the flagellate protozoan Trichomonas 
vaginalis represents the most prevalent nonviral sexually transmitted 
disease worldwide (WHO-DRHR 2012). In women, the symptoms are cyclic 
and often worsen around the menstruation period. In men, trichomonosis 
is largely asymptomatic and these men are considered to be carriers of 
T. vaginalis (Petrin et al. 1998). This infection has been associated 
with birth outcomes (Klebanoff et al. 2001), infertility (Grodstein et 
al. 1993), cervical and prostate cancer (Viikki et al. 2000, Sutcliffe 
et al. 2012) and pelvic inflammatory disease (Cherpes et al. 2006). 
Importantly, T. vaginalis is a co-factor in human immunodeficiency 
virus transmission and acquisition (Sorvillo et al. 2001, Van Der Pol 
et al. 2008). Therefore, it is important to study the host-parasite 
relationship to understand T. vaginalis infection and pathogenesis. 
Colonisation of the mucosa by T. vaginalis is a complex multi-step 
process that involves distinct mechanisms (Alderete et al. 2004). The 
parasite interacts with mucin (Lehker & Sweeney 1999), adheres to 
vaginal epithelial cells (VECs) in a process mediated by adhesion 
proteins (AP120, AP65, AP51, AP33 and AP23) and undergoes dramatic 
morphological changes from a pyriform to an amoeboid form (Engbring & 
Alderete 1998, Kucknoor et al. 2005, Moreno-Brito et al. 2005). After 
adhesion to VECs, the synthesis and gene expression of adhesins are 
increased (Kucknoor et al. 2005). These mechanisms must be tightly 
regulated and iron plays a pivotal role in this regulation. Iron is an 
essential element for all living organisms, from the most primitive to 
the most complex, as a component of haeme, iron-sulphur clusters and a 
variety of proteins. Iron is known to contribute to biological 
functions such as DNA and RNA synthesis, oxygen transport and 
metabolic reactions. T. vaginalis has developed multiple iron uptake 
systems such as receptors for hololactoferrin, haemoglobin (HB), 
haemin (HM) and haeme binding as well as adhesins to erythrocytes and 
epithelial cells (Moreno-Brito et al. 2005, Ardalan et al. 2009). Iron 
plays a crucial role in the pathogenesis of trichomonosis by 
increasing cytoadherence and modulating resistance to complement 
lyses, ligation to the extracellular matrix and the expression of 
proteases (Figueroa-Angulo et al. 2012). In agreement with this role, 
the symptoms of trichomonosis worsen after menstruation. In addition, 
iron also influences nucleotide hydrolysis in T. vaginalis (Tasca et 
al. 2005, de Jesus et al. 2006). The extracellular concentrations of 
ATP and adenosine can markedly increase under several conditions such 
as inflammation and hypoxia as well as in the presence of pathogens 
(Robson et al. 2006, Sansom 2012). In the extracellular medium, these 
nucleotides can act as immunomodulators by triggering immunological 
effects. Extracellular ATP acts as a proinflammatory immune-mediator 
by triggering multiple immunological effects on cell types such as 
neutrophils, macrophages, dendritic cells and lymphocytes (Bours et 
al. 2006). In this sense, ATP and 

Re: Contextual Synonym Filter

[Andrea Gazzarini]

Hi Luca,
I believe this is not an easy task to do passing through Solr/Lucene 
internals; did you try to use what Solr offers out of the box?
For example, you could define several fields associated where each 
corresponding field type uses a different synonym set. So you would have


 * F1 -> FT1 -> SYNSET1
 * F2 -> FT2 -> SYNSET2
 * ...

if you query using *fq=F1:something* then the *FT1* (and the *SYNSET1*) 
will be used, if you query using *fq=F2:something* then the *FT2* (and 
the *SYNSET2*) will be used, and so on


I don't know your context so my suggestion could be absolutely in the 
wrong path


Best,
Andrea

On 03/09/2018 15:41, Vergantini Luca wrote:


I need to create a contextual Synonym Filter:

I need that the Synonym Filter load different synonym configuration 
based on the fq query parameter.


I’ve already modified the SynonymGraphFilterFactory to load from DB 
(this is another requirement) but I can’t understand how to make the 
fq parameter arrive to the Factory.


Maybe I need a Query Parser plugin?

Please help



Luca Vergantini

Whitehall Reply
Via del Giorgione, 59
00147 - Roma - ITALY
phone: +39 06 844341
l.vergant...@reply.it 
www.reply.it

Whitehall Reply

Re: Solr7 embeded req: Bad content type error

[Andrea Gazzarini]

Hi Alfonso, thank you, too: I've learned another stuff ;)

Andrea

On 29/08/18 14:31, Alfonso Noriega wrote:

Thanks for your time and help Andrea,
I guess we should try to use the Json API
<https://lucene.apache.org/solr/guide/7_4/json-request-api.html> provided
by Solr and figure out a way to do so with SolrJ.


On Wed, 29 Aug 2018 at 14:21, Andrea Gazzarini  wrote:


Well, I don't know the actual reason why the behavior is different
between Cloud and Embedded client: maybe things are different because in
the Embedded Solr HTTP is not involved at all, but I'm just shooting in
the dark.

I'm not aware about POST capabilities you mentioned, sorry

Andrea


On 29/08/2018 14:07, Alfonso Noriega wrote:

Yes, I realized that changing the method to GET solves the issue but it

is

intentionally set to POST as in a real case scenario we had the issue of
users creating too long queries which where facing the REST length

limits.

I think a possible solution would be to send the Solr params as a json in
the request body, but I am not sure if SolrJ supports this.

Alfonso.

On Wed, 29 Aug 2018 at 13:46, Andrea Gazzarini 

wrote:

I think that's the issue: just guessing because I do not have the code
in front of me.

POST requests put the query in the request body, and the
EmbeddedSolrServer expects to find a valid JSON. Did you try to remove
the Method param?

Andrea


On 29/08/2018 13:12, Alfonso Noriega wrote:

Hi Andrea,

Thanks for your help, something which is relevant and I forgot to

mention

before is that the requests are always a POST method.
As mentioned before, it is not a single query which fails but all of

the

requests done to the  search handler.

final SolrQuery q = new SolrQuery("!( _id_:"+

doc.getId()+")AND(_root_:"+

doc.getId()+")");
final QueryResponse query = solrClient.query(q,

SolrRequest.METHOD.POST);

Regarding the embedded server instantiation:
.
.
.
final CoreContainer container =

CoreContainer.createAndLoad(tmpSolrHome,

tmpSolrConfig);
return new SolrClientWrapper(container, CORE_NAME, tmpSolrHome);
.
.
.
private class SolrClientWrapper extends EmbeddedSolrServer {

   private final Path temporaryFolder;

   public SolrClientWrapper(CoreContainer coreContainer, String
coreName, Path temporaryFolder) {
   super(coreContainer, coreName);
   this.temporaryFolder = temporaryFolder;
   }
.
.
.

The whole embedded server instantiation can be seen here
<

https://github.com/RBMHTechnology/vind/blob/master/server/embedded-solr-server/src/main/java/com/rbmhtechnology/vind/solr/backend/EmbeddedSolrServerProvider.java

.

Best,
Alfonso.

On Wed, 29 Aug 2018 at 12:57, Andrea Gazzarini 

wrote:

Hi Alfonso,
could you please paste an extract of the client code? Specifically

those

few lines where you create the SolrQuery with params.

The line you mentioned is dealing with ContentStream which as far as I
remember wraps the request body, and not the request params. So as
request body Solr expects a valid JSON payload, but in your case, if I
got you, you're sending plain query parameters (through SolrJ).

Best,
Andrea

On 29/08/2018 12:45, Alfonso Noriega wrote:

Hi,
I am implementing a migration of Vind
<https://github.com/RBMHTechnology/vind> library from solr 5 to

7.4.0

and I

am facing an error which I have no idea how to solve...

The library provides a wrapper (and some extra stuff) to develop

search

tools over Solr and uses SolrJ to access it, more info about it can

be

seen

in the public repo, but basically all requests are done to solr

through a

client provided by solrj.
Actually, the migration is done and the tests running against a cloud
instance are working perfectly fine. The issue arises when testing

against

an embedded server (which is the default case as you may not have a

local

solr instance running), in which case every request fails throwing

the

following exception:
*org.apache.solr.common.SolrException: Bad contentType for search

handler

:application/javabin request={q=!(+_id_:P3)AND(_root_:P3)}*

* at


org.apache.solr.request.json.RequestUtil.processParams(RequestUtil.java:73)*

* at


org.apache.solr.util.SolrPluginUtils.setDefaults(SolrPluginUtils.java:167)*

* at


org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:196)*

* at org.apache.solr.core.SolrCore.execute(SolrCore.java:2539)*
* at


org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:191)*

* at

org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)*

* at

org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974)*

* at

org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:990)*

After some debugging I have seen that the embedded solr server uses
*org.apache.solr.request.json.RequestUtil* to process the parameters

which

invariable expects a '*application/json' *contentType but SolrJ

prodcues

Re: Solr7 embeded req: Bad content type error

[Andrea Gazzarini]

Well, I don't know the actual reason why the behavior is different 
between Cloud and Embedded client: maybe things are different because in 
the Embedded Solr HTTP is not involved at all, but I'm just shooting in 
the dark.


I'm not aware about POST capabilities you mentioned, sorry

Andrea


On 29/08/2018 14:07, Alfonso Noriega wrote:

Yes, I realized that changing the method to GET solves the issue but it is
intentionally set to POST as in a real case scenario we had the issue of
users creating too long queries which where facing the REST length limits.
I think a possible solution would be to send the Solr params as a json in
the request body, but I am not sure if SolrJ supports this.

Alfonso.

On Wed, 29 Aug 2018 at 13:46, Andrea Gazzarini  wrote:


I think that's the issue: just guessing because I do not have the code
in front of me.

POST requests put the query in the request body, and the
EmbeddedSolrServer expects to find a valid JSON. Did you try to remove
the Method param?

Andrea


On 29/08/2018 13:12, Alfonso Noriega wrote:

Hi Andrea,

Thanks for your help, something which is relevant and I forgot to mention
before is that the requests are always a POST method.
As mentioned before, it is not a single query which fails but all of the
requests done to the  search handler.

final SolrQuery q = new SolrQuery("!( _id_:"+ doc.getId()+")AND(_root_:"+
doc.getId()+")");
final QueryResponse query = solrClient.query(q, SolrRequest.METHOD.POST);

Regarding the embedded server instantiation:
.
.
.
final CoreContainer container = CoreContainer.createAndLoad(tmpSolrHome,
tmpSolrConfig);
return new SolrClientWrapper(container, CORE_NAME, tmpSolrHome);
.
.
.
private class SolrClientWrapper extends EmbeddedSolrServer {

  private final Path temporaryFolder;

  public SolrClientWrapper(CoreContainer coreContainer, String
coreName, Path temporaryFolder) {
  super(coreContainer, coreName);
  this.temporaryFolder = temporaryFolder;
  }
.
.
.

The whole embedded server instantiation can be seen here
<

https://github.com/RBMHTechnology/vind/blob/master/server/embedded-solr-server/src/main/java/com/rbmhtechnology/vind/solr/backend/EmbeddedSolrServerProvider.java

.

Best,
Alfonso.

On Wed, 29 Aug 2018 at 12:57, Andrea Gazzarini 

wrote:

Hi Alfonso,
could you please paste an extract of the client code? Specifically those
few lines where you create the SolrQuery with params.

The line you mentioned is dealing with ContentStream which as far as I
remember wraps the request body, and not the request params. So as
request body Solr expects a valid JSON payload, but in your case, if I
got you, you're sending plain query parameters (through SolrJ).

Best,
Andrea

On 29/08/2018 12:45, Alfonso Noriega wrote:

Hi,
I am implementing a migration of Vind
<https://github.com/RBMHTechnology/vind> library from solr 5 to 7.4.0

and I

am facing an error which I have no idea how to solve...

The library provides a wrapper (and some extra stuff) to develop search
tools over Solr and uses SolrJ to access it, more info about it can be

seen

in the public repo, but basically all requests are done to solr

through a

client provided by solrj.
Actually, the migration is done and the tests running against a cloud
instance are working perfectly fine. The issue arises when testing

against

an embedded server (which is the default case as you may not have a

local

solr instance running), in which case every request fails throwing the
following exception:
*org.apache.solr.common.SolrException: Bad contentType for search

handler

:application/javabin request={q=!(+_id_:P3)AND(_root_:P3)}*

* at


org.apache.solr.request.json.RequestUtil.processParams(RequestUtil.java:73)*

* at


org.apache.solr.util.SolrPluginUtils.setDefaults(SolrPluginUtils.java:167)*

* at


org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:196)*

* at org.apache.solr.core.SolrCore.execute(SolrCore.java:2539)*
* at


org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:191)*

* at

org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)*

* at

org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974)*

* at

org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:990)*

After some debugging I have seen that the embedded solr server uses
*org.apache.solr.request.json.RequestUtil* to process the parameters

which

invariable expects a '*application/json' *contentType but SolrJ

prodcues

as

*'application/javabin'.*
Any ideas on how to fix this issue?

The library has in the pom defined the following solr depedencies :
*.*
*.*
*.*

*7.4.0*
*.*
*.*
*.*
**
*org.apache.solr*
*solr-core*
*${solr.version}*
**
**
*org.apache.solr*
*solr-solrj*
*${solr.version}*
**

Thanks!

Re: Solr7 embeded req: Bad content type error

[Andrea Gazzarini]

I think that's the issue: just guessing because I do not have the code 
in front of me.


POST requests put the query in the request body, and the 
EmbeddedSolrServer expects to find a valid JSON. Did you try to remove 
the Method param?


Andrea


On 29/08/2018 13:12, Alfonso Noriega wrote:

Hi Andrea,

Thanks for your help, something which is relevant and I forgot to mention
before is that the requests are always a POST method.
As mentioned before, it is not a single query which fails but all of the
requests done to the  search handler.

final SolrQuery q = new SolrQuery("!( _id_:"+ doc.getId()+")AND(_root_:"+
doc.getId()+")");
final QueryResponse query = solrClient.query(q, SolrRequest.METHOD.POST);

Regarding the embedded server instantiation:
.
.
.
final CoreContainer container = CoreContainer.createAndLoad(tmpSolrHome,
tmpSolrConfig);
return new SolrClientWrapper(container, CORE_NAME, tmpSolrHome);
.
.
.
private class SolrClientWrapper extends EmbeddedSolrServer {

 private final Path temporaryFolder;

 public SolrClientWrapper(CoreContainer coreContainer, String
coreName, Path temporaryFolder) {
 super(coreContainer, coreName);
 this.temporaryFolder = temporaryFolder;
 }
.
.
.

The whole embedded server instantiation can be seen here
<https://github.com/RBMHTechnology/vind/blob/master/server/embedded-solr-server/src/main/java/com/rbmhtechnology/vind/solr/backend/EmbeddedSolrServerProvider.java>
.

Best,
Alfonso.

On Wed, 29 Aug 2018 at 12:57, Andrea Gazzarini  wrote:


Hi Alfonso,
could you please paste an extract of the client code? Specifically those
few lines where you create the SolrQuery with params.

The line you mentioned is dealing with ContentStream which as far as I
remember wraps the request body, and not the request params. So as
request body Solr expects a valid JSON payload, but in your case, if I
got you, you're sending plain query parameters (through SolrJ).

Best,
Andrea

On 29/08/2018 12:45, Alfonso Noriega wrote:

Hi,
I am implementing a migration of Vind
<https://github.com/RBMHTechnology/vind> library from solr 5 to 7.4.0

and I

am facing an error which I have no idea how to solve...

The library provides a wrapper (and some extra stuff) to develop search
tools over Solr and uses SolrJ to access it, more info about it can be

seen

in the public repo, but basically all requests are done to solr through a
client provided by solrj.
Actually, the migration is done and the tests running against a cloud
instance are working perfectly fine. The issue arises when testing

against

an embedded server (which is the default case as you may not have a local
solr instance running), in which case every request fails throwing the
following exception:
*org.apache.solr.common.SolrException: Bad contentType for search handler
:application/javabin request={q=!(+_id_:P3)AND(_root_:P3)}*

* at


org.apache.solr.request.json.RequestUtil.processParams(RequestUtil.java:73)*

* at


org.apache.solr.util.SolrPluginUtils.setDefaults(SolrPluginUtils.java:167)*

* at


org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:196)*

* at org.apache.solr.core.SolrCore.execute(SolrCore.java:2539)*
* at


org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:191)*

* at

org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)*

* at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974)*
* at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:990)*

After some debugging I have seen that the embedded solr server uses
*org.apache.solr.request.json.RequestUtil* to process the parameters

which

invariable expects a '*application/json' *contentType but SolrJ prodcues

as

*'application/javabin'.*
Any ideas on how to fix this issue?

The library has in the pom defined the following solr depedencies :
*.*
*.*
*.*

*7.4.0*
*.*
*.*
*.*
**
*org.apache.solr*
*solr-core*
*${solr.version}*
**
**
*org.apache.solr*
*solr-solrj*
*${solr.version}*
**

Thanks!

Re: Solr7 embeded req: Bad content type error

[Andrea Gazzarini]

Hi Alfonso,
could you please paste an extract of the client code? Specifically those 
few lines where you create the SolrQuery with params.


The line you mentioned is dealing with ContentStream which as far as I 
remember wraps the request body, and not the request params. So as 
request body Solr expects a valid JSON payload, but in your case, if I 
got you, you're sending plain query parameters (through SolrJ).


Best,
Andrea

On 29/08/2018 12:45, Alfonso Noriega wrote:

Hi,
I am implementing a migration of Vind
 library from solr 5 to 7.4.0 and I
am facing an error which I have no idea how to solve...

The library provides a wrapper (and some extra stuff) to develop search
tools over Solr and uses SolrJ to access it, more info about it can be seen
in the public repo, but basically all requests are done to solr through a
client provided by solrj.
Actually, the migration is done and the tests running against a cloud
instance are working perfectly fine. The issue arises when testing against
an embedded server (which is the default case as you may not have a local
solr instance running), in which case every request fails throwing the
following exception:
*org.apache.solr.common.SolrException: Bad contentType for search handler
:application/javabin request={q=!(+_id_:P3)AND(_root_:P3)}*

* at
org.apache.solr.request.json.RequestUtil.processParams(RequestUtil.java:73)*
* at
org.apache.solr.util.SolrPluginUtils.setDefaults(SolrPluginUtils.java:167)*
* at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:196)*
* at org.apache.solr.core.SolrCore.execute(SolrCore.java:2539)*
* at
org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:191)*
* at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)*
* at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974)*
* at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:990)*

After some debugging I have seen that the embedded solr server uses
*org.apache.solr.request.json.RequestUtil* to process the parameters which
invariable expects a '*application/json' *contentType but SolrJ prodcues as
*'application/javabin'.*
Any ideas on how to fix this issue?

The library has in the pom defined the following solr depedencies :
*.*
*.*
*.*

*7.4.0*
*.*
*.*
*.*
**
*org.apache.solr*
*solr-core*
*${solr.version}*
**
**
*org.apache.solr*
*solr-solrj*
*${solr.version}*
**

Thanks!

Re: why the group feature is in QueryComponet ,but GroupComponent ?

[Andrea Gazzarini]

A search component is something which contributes to the overall 
returned response. With "contributes" I mean "adds something".
The highlighting is an example of such behavior: it depends on the query 
component and on top of a set of search results it enriches the response 
with an additional section ("highlighting").


The context is not the same when you enable the grouping feature, as it 
doesn't contribute with an additional section, it actualy changes the 
query response shape. That's the reason why (I think) it is not a 
SearchComponent: you should consider that more like a variant of the 
usual query behavior.


Just curiosity: can I ask you why you care about that? Do you have a use 
case where you need both raw and grouped docs? Again, just curiosity.


Best,
Andrea

On 20/08/18 11:30, shreck wrote:




--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Multi-word Synonyms - how does sow parameter work?

[Andrea Gazzarini]

Hi Roy, I think you miss the autoGeneratePhraseQueries=true in the field 
type definition.
I was on a slightly different use case when I met your same issue (I was 
using synonyms expansion at query time) and honestly I didn't understand 
why this is not the default and implicit behavior. In other words, like 
you, I can't imagine a scenario where I would a multi-terms synonym be 
destructured in multiple OR clauses.


Best,
Andrea

On 16/08/18 02:07, Roy Lim wrote:

I am not using edismax (eventually I would like to get there) but I'm just
testing with standard query right now.  Original posting:

I'm trying to figure out why the multi-word synonym expansion is not
working correctly (or, at least what I'm misunderstanding).  Specifically,
when I test a standard query with Solr Admin it appears to still split on
whitespace.

Here is my setup:
- Solr 7.2.1
- synonym example: LCD => liquid crystal display
- q=myfield:LCD
- added parameter: sow=false
- myfield schema looks like (analyzer both applicable to index and query
time):


   
 
 
 ...


When debugging the query, Solr Admin shows the parsed query as:

myfield:liquid myfield:crystal myfield:display


(default operator being OR), as you can see it would incorrectly match on
any of those words, but not all, which is what I would expect...

Should it not do a phrase query search for the exact translated synonym,
"liquid crystal display"?



On Wed, Aug 15, 2018 at 5:01 PM, Doug Turnbull <
dturnb...@opensourceconnections.com> wrote:


Also share your fieldType settings for myfield as well from your schema
On Wed, Aug 15, 2018 at 8:00 PM Doug Turnbull <
dturnb...@opensourceconnections.com> wrote:


Aside from the screenshot issue, one  thing to check: are you searching
with defType=edismax ?

As in
q=lcd=myfield=false=edismax

?

Also sow=false should the the default on Solr 7 and above

Doug

On Wed, Aug 15, 2018 at 6:27 PM Roy Lim  wrote:


I'm trying to figure out why the multi-word synonym expansion is not
working
correctly.  Specifically, when I test a standard query with Solr Admin

it

is
still splitting on whitespace.

Here is my setup:
- Solr 7.2.1
- synonym LCD => liquid crystal display
- q=myfield:LCD
- added: sow=false
- myfield looks like:


Solr Admin shows the parsed query looks like:

myfield:liquid myfield:crystal myfield:display

(default operator being OR), which would incorrectly match documents

with

any of those words, but not all, which is what I would expect...





--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html


--
CTO, OpenSource Connections
Author, Relevant Search
http://o19s.com/doug


--
CTO, OpenSource Connections
Author, Relevant Search
http://o19s.com/doug

Re: Is Running the Same Filters on Index and Query Redundant?

[Andrea Gazzarini]

You're welcome, great to hear you have less doubts.
I see you're using the SynonymGraphFilter followed by a StopFilter at 
query time: have a look at this post [1], you might find some useful info.


Best,
Andrea

[1] https://sease.io/2018/07/combining-synonyms-and-stopwords.html

On 15/08/18 21:47, Zimmermann, Thomas wrote:

Hi Andrea,

Thanks so much. I wasn¹t thinking in the correct perspective on the query
portion of the analyzer, but your explanation makes perfect sense. In my
head I imagine the result set of the query being transformed by the
filters, but in actuality the filter is being applied to the query itself
before processing. This makes sense on my end and I think it answer my
questions.

Excellent point on the html strip factory. I¹ll evaluate our use cases.

This was all brought about by switching from the deprecated synonym and
word delimiter factories to the new graph based factories, where we
stopped filtering on insert for those and switched to filtering on query
based on recommendations from the Solr Doc.

Thanks,
TZ

On 8/15/18, 3:17 PM, "Andrea Gazzarini"  wrote:


Hi Thomas,
as you know, the two analyzers play in a different moment, with a
different input and a different goal for the corresponding output:

  * index analyzer: input is a field value, output is used for building
the index
  * query analyzer: input is a (user) query string, output is used for
building a (Solr) query

At index time a term dictionary is built, and a retrieval time the
output query tries to find a match in that dictionary. I wouldn't call
it "redundancy" because even if the filter is the same, it is applied to
a different input and it has a different goal.

Some filters must be present both at index at query time because
otherwise you won't find any match: if you put a lowercase filter only
on the index side, queries with uppercase chars won't find any match.
Some others don't (one example is the SynonymGraphFilter you've used
only at query time). In general, everything depends on your needs and
it's perfectly valid to have symmetric (index analyzer = query analyzer)
and asymmetric text analysis (index analyzer != query analyzer).

Without knowing your context is very hard to guess if there's something
wrong in the configuration. What is the part of the analyzers you think
is redundant?

On top of that: in your chain the HTMLStripCharFilterFactory applied at
query time is something unusual, because while it makes perfectly sense
at index time (where I guess you index some HTML source), at query time
I can't imagine a scenario where the user inputs queries containing HTML
tags.

Best,
Andrea

On 15/08/18 20:43, Zimmermann, Thomas wrote:

Hi,

We have the text field below configured on fields that are both stored
and indexed. It seems to me that applying the same filters on both index
and query would be redundant, and perhaps a waste of processing on the
retrieval side if the filter work was already done on the index side. Is
this a fair statement to make? Should I only be applying filters on one
end of the transaction?

Thanks,
TZ


 



  

  

  

  

  

  





  

  

  

  

  

  

  

  



  

Re: Is Running the Same Filters on Index and Query Redundant?

[Andrea Gazzarini]

Hi Thomas,
as you know, the two analyzers play in a different moment, with a 
different input and a different goal for the corresponding output:


 * index analyzer: input is a field value, output is used for building
   the index
 * query analyzer: input is a (user) query string, output is used for
   building a (Solr) query

At index time a term dictionary is built, and a retrieval time the 
output query tries to find a match in that dictionary. I wouldn't call 
it "redundancy" because even if the filter is the same, it is applied to 
a different input and it has a different goal.


Some filters must be present both at index at query time because 
otherwise you won't find any match: if you put a lowercase filter only 
on the index side, queries with uppercase chars won't find any match. 
Some others don't (one example is the SynonymGraphFilter you've used 
only at query time). In general, everything depends on your needs and 
it's perfectly valid to have symmetric (index analyzer = query analyzer) 
and asymmetric text analysis (index analyzer != query analyzer).


Without knowing your context is very hard to guess if there's something 
wrong in the configuration. What is the part of the analyzers you think 
is redundant?


On top of that: in your chain the HTMLStripCharFilterFactory applied at 
query time is something unusual, because while it makes perfectly sense 
at index time (where I guess you index some HTML source), at query time 
I can't imagine a scenario where the user inputs queries containing HTML 
tags.


Best,
Andrea

On 15/08/18 20:43, Zimmermann, Thomas wrote:

Hi,

We have the text field below configured on fields that are both stored and 
indexed. It seems to me that applying the same filters on both index and query 
would be redundant, and perhaps a waste of processing on the retrieval side if 
the filter work was already done on the index side. Is this a fair statement to 
make? Should I only be applying filters on one end of the transaction?

Thanks,
TZ




   

 

 

 

 

 

 

   

   

 

 

 

 

 

 

 

 

   

 

Re: Can We Alias field "_text_" to "text" in Schema

[Andrea Gazzarini]

Hi,
field names with both leading and trailing underscores are reserved [1] 
so you it would be better to avoid that.
I cannot tell you what exactly the problem is, using such naming; I 
remember I had troubles with function queries, so, in general, I would 
follow that advice.


Best,
Andrea

[1] 
https://lucene.apache.org/solr/guide/6_6/defining-fields.html#DefiningFields-FieldProperties


On 08/08/18 13:17, THADC wrote:

Hello,

We recently migrated from solr 4.7 to 7.3. We are having a problem with
legacy java code that we do not have control over where searches are coded
to query text as "text". Again, we do not the luxury in this instance to
update the code, so we were wondering whether there is a way to define an
alias field with name _text_ to text.

Any reply is greatly appreciated.

Thanks!



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr 7.4.0 : Enabling search on all keys by default

[Andrea Gazzarini]

Hi Rajnish,
yes, you can use a general-generic field (that's the price of having 
such "centralization") where, by means of the *copyField* directive, all 
fields are copied there.
Then, you can use that field as a default search field (df parameter) in 
your RequestHandler.






...

Best,
Andrea

On 08/08/18 08:55, Rajnish Kumar CHAUHAN wrote:


Hello team

I am using Solr-7.4.0 and on Solr we application , it is not possible 
to search on all attributes by default as was the case with Solr-6.4.0


I have to mentioned key name for searching .

How can we enable search on all keys by default.

Thanks.

Description: Description: Description: Description: Description: 
Description: logo_big5


Rajnish CHAUAHN> |TINA: 199#6133

ICT | MnQ

Re: Ignored fields and copyfield

[Andrea Gazzarini]

Hi John,
Yes, it's possible.

Andrea

On Mon, 6 Aug 2018, 22:47 John Davis,  wrote:

> Hi there,
> If a field is set as "ignored" (indexed=false, stored=false) can it be used
> for another field as part of copyfield directive which might index/store
> it.
>
> John
>

Re: Solr Server crashes when requesting a result with too large resultRows

[Andrea Gazzarini]

Yes, but 581 is the final number you got in the response, which is the 
result of the main query intersected with the filter query so I wouldn't 
take in account this number. The main and the filter query are executed 
separately, so I guess (but I'm guessing because I don't know these 
internals) that's here where the "rows" parameter matters.


Again, I'm guessing, I'm sure some Solr committer here can explain you 
how things are working.


Best,
Andrea

On 31/07/18 11:12, Fette, Georg wrote:

Hi Andrea,
I agree that receiving too much data in one request is bad. But I was 
surprised that the query works with a lower but still very large rows 
parameter and that there is a threshold at which it crashes the 
server. Furthermore, it seems that the reason for the crash is not the 
size of the actual results because those are only 581.

Greetings
Georg

Am 31.07.2018 um 10:53 schrieb Andrea Gazzarini:

Hi Georg,
I would say, without knowing your context, that this is not what Solr 
is supposed to do. You're asking to load everything in a single 
request/response and this poses a problem.
Since I guess that, even we assume it works, you should then iterate 
those results one by one or in blocks, an option would be to do this 
part (block scrolling) using Solr [2].

I suggest you to have a look at

 * the export endpoint [1]
 * the cursor API [2]

Best,
Andrea

[1] https://lucene.apache.org/solr/guide/6_6/exporting-result-sets.html
[2] 
https://lucene.apache.org/solr/guide/6_6/pagination-of-results.html#fetching-a-large-number-of-sorted-results-cursors


On 31/07/18 10:44, Georg Fette wrote:

Hello,
We run the server version 7.3.1. on a machine with 32GB RAM in a 
mode having -10g.

When requesting a query with
q={!boost 
b=sv_int_catalog_count_document}string_catalog_aliases:(*2*)=string_field_type:catalog_entry=2147483647
the server takes all available memory up to 10GB and is then no 
longer accessible with one processor at 100%.
When we reduce the rows parameter to 1000 the query works. The 
query returns only 581 results.
The documentation at 
https://wiki.apache.org/solr/CommonQueryParameters states that as 
the "rows" parameter a "ridiculously large value" may be used, but 
this could pose a problem. The number we used was Int.max from Java.

Greetings
Georg

Re: Solr Server crashes when requesting a result with too large resultRows

[Andrea Gazzarini]

Hi Georg,
I would say, without knowing your context, that this is not what Solr is 
supposed to do. You're asking to load everything in a single 
request/response and this poses a problem.
Since I guess that, even we assume it works, you should then iterate 
those results one by one or in blocks, an option would be to do this 
part (block scrolling) using Solr [2].

I suggest you to have a look at

 * the export endpoint [1]
 * the cursor API [2]

Best,
Andrea

[1] https://lucene.apache.org/solr/guide/6_6/exporting-result-sets.html
[2] 
https://lucene.apache.org/solr/guide/6_6/pagination-of-results.html#fetching-a-large-number-of-sorted-results-cursors


On 31/07/18 10:44, Georg Fette wrote:

Hello,
We run the server version 7.3.1. on a machine with 32GB RAM in a mode 
having -10g.

When requesting a query with
q={!boost 
b=sv_int_catalog_count_document}string_catalog_aliases:(*2*)=string_field_type:catalog_entry=2147483647
the server takes all available memory up to 10GB and is then no longer 
accessible with one processor at 100%.
When we reduce the rows parameter to 1000 the query works. The 
query returns only 581 results.
The documentation at 
https://wiki.apache.org/solr/CommonQueryParameters states that as the 
"rows" parameter a "ridiculously large value" may be used, but this 
could pose a problem. The number we used was Int.max from Java.

Greetings
Georg

Re: java.lang.OutOfMemoryError indexing xlsm and xlsx file

[Andrea Gazzarini]

Hi Mario, could you please share your settings (e.g. OS, JVM memory, 
System memory)?


Andrea

On 27/07/18 11:36, Bisonti Mario wrote:

Hallo
I obtain the error indexing a .xlsm or .xlsx file of 11 MB

What could I do?

Thanks a lot
Mario

2018-07-27 11:08:25.634 WARN  (qtp1521083627-99) [   x:core_share] 
o.e.j.s.HttpChannel /solr/core_share/update/extract
java.lang.OutOfMemoryError
 at 
java.base/java.lang.AbstractStringBuilder.hugeCapacity(AbstractStringBuilder.java:188)
 at 
java.base/java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:180)
 at 
java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:147)
 at 
java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:660)
 at java.base/java.lang.StringBuilder.append(StringBuilder.java:195)
 at 
org.apache.solr.handler.extraction.SolrContentHandler.characters(SolrContentHandler.java:302)
 at 
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
 at 
org.apache.tika.sax.SecureContentHandler.characters(SecureContentHandler.java:270)
 at 
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
 at 
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
 at 
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
 at 
org.apache.tika.sax.SafeContentHandler.access$001(SafeContentHandler.java:46)
 at 
org.apache.tika.sax.SafeContentHandler$1.write(SafeContentHandler.java:82)
 at 
org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:140)
 at 
org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:287)
 at 
org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:279)
 at 
org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:306)
 at 
org.apache.tika.parser.microsoft.ooxml.OOXMLTikaBodyPartHandler.run(OOXMLTikaBodyPartHandler.java:147)
 at 
org.apache.tika.parser.microsoft.ooxml.OOXMLWordAndPowerPointTextHandler.handleEndOfRun(OOXMLWordAndPowerPointTextHandler.java:468)
 at 
org.apache.tika.parser.microsoft.ooxml.OOXMLWordAndPowerPointTextHandler.endElement(OOXMLWordAndPowerPointTextHandler.java:450)
 at 
org.apache.tika.sax.ContentHandlerDecorator.endElement(ContentHandlerDecorator.java:136)
 at 
org.apache.tika.sax.ContentHandlerDecorator.endElement(ContentHandlerDecorator.java:136)
 at 
java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
 at 
java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1714)
 at 
java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2879)
 at 
java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
 at 
java.xml/com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
 at 
java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:532)
 at 
java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:888)
 at 
java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:824)
 at 
java.xml/com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
 at 
java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
 at 
java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:635)
 at 
java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:324)
 at java.xml/javax.xml.parsers.SAXParser.parse(SAXParser.java:197)
 at 
org.apache.tika.parser.microsoft.ooxml.AbstractOOXMLExtractor.handleGeneralTextContainingPart(AbstractOOXMLExtractor.java:506)
 at 
org.apache.tika.parser.microsoft.ooxml.XSSFExcelExtractorDecorator.processShapes(XSSFExcelExtractorDecorator.java:279)
 at 
org.apache.tika.parser.microsoft.ooxml.XSSFExcelExtractorDecorator.buildXHTML(XSSFExcelExtractorDecorator.java:185)
 at 
org.apache.tika.parser.microsoft.ooxml.AbstractOOXMLExtractor.getXHTML(AbstractOOXMLExtractor.java:135)
 at 
org.apache.tika.parser.microsoft.ooxml.XSSFExcelExtractorDecorator.getXHTML(XSSFExcelExtractorDecorator.java:120)
 at 
org.apache.tika.parser.microsoft.ooxml.OOXMLExtractorFactory.parse(OOXMLExtractorFactory.java:143)
 at 

Re: Question

[Andrea Gazzarini]

Hi Driss,
I think the answer to the first question is yes, but I guess It doesn't
help you so much.
Second and third questions: "It depends", you should describe better your
contest, narrowing questions ad much as possibile ("how can web do It" is
definitely top much generic)

Best,
Andrea


Il lun 23 lug 2018, 15:18 Driss Khalil  ha scritto:

> Hi,
> I'm new to Solr and I just want to know if it's possible to handle
> Unstrcutured data in solr .If yes how can we do it ? Do we need it to
> combine it with something else?
>
>
>
>
>
> *Driss KHALIL*
>
> Responsable prospection & sponsoring, Forum GENI Entreprises.
>
> Elève ingénieur en Génie Logiciel, ENSIAS.
> GSM: (+212) 06 62 52 83 26
>
> [image: https://www.linkedin.com/in/driss-khalil-b3aab4151/]
> 
>

Re: synonyms question

[Andrea Gazzarini]

Hi Ennio,
could you please share:

 * your configuration (specifically the field type declaration in your
   schema)
 * the query (please add debug=true) and the corresponding query response

Best,
Andrea

On 17/07/18 17:35, Ennio Bozzetti wrote:

I'm trying to get my synonyms to work, but for this one keyword I cannot get it 
to work.

I added the following to my synonyms file.

fiber,fibre

But when I search for fiber or fibre it does not work.

Fiber is the American English spelling and Fibre is the British English 
spelling.

My field type is set to text_en would that be why?

Thanks,

Ennio Bozzetti
Senior Web Programmer
THORLABS
(973) 300-2561
www.thorlabs.com

Re: solr nested schema

[Andrea Gazzarini]

Hi,
Please expand a bit your needs, because the answer to your question could
be different.

Specifically: is that nesting needed only for visualization purposes? Could
you please expand your access pattern (i.e. queries requirements)?

Even if Solr supports nested documents (just google "Solr nested
documents") , depending on the answers above, it could be something that
you don't need.

Best,
Andrea

Il sab 14 lug 2018, 13:27 sara hajili  ha scritto:

> hi
> i need add nested document in solr . how can i do this ?
> how add nested document field in schema ?
> for example i need to add this :
> {
>   "id":12,
>'topic":[ {
>code:1,
> name:"hi"
> },
> {
> code:2,
>  name:"hi"
> }
>   ]
> }
>

Re: copy field

[Andrea Gazzarini]

You're welcome,
but keep in mind what Erik said: it seems you don't need that information
at "view" level so maybe with range faceting you could get what you need
without creating additional (and redundant) fields.

Andrea

Il ven 13 lug 2018, 07:04 Anil  ha scritto:

> Thanks Andrea. i will write update processor in index pipe line.
>
> I feel this is very good feature to support.
>
> Thanks,
> Anil
>
> On 12 July 2018 at 22:59, Andrea Gazzarini  wrote:
>
> > Hi Anil,
> > The copy Field directive is not what you're looking for because it
> doesn't
> > change the stored value of a field.
> >
> > What you need is an Update Request Processor, which is a kind of
> > interceptor in the indexing chain (i.e. It allows you to change an
> incoming
> > document before it gets indexed).
> > Unfortunately, as far as I know there's not an available processor for
> > doing what you need in the example you described, but consider that
> writing
> > a new processor is a trivial thing.
> >
> > Andrea
> >
> > Il gio 12 lug 2018, 19:23 Anil  ha scritto:
> >
> > > HI,
> > >
> > > i have a date field which needs to copied to different field with
> > different
> > > format/value. is there any way  to achieve this using copy field ? or
> > needs
> > > to be done when creating solr document itself.
> > >
> > > lets say createdDate is 10-23-2017 10:15:00, it needs to be copied to
> > > transformedDate field as  10-23-2017.
> > >
> > > please help. thanks.
> > >
> > > Regards,
> > > Anil
> > >
> >
>

Re: copy field

[Andrea Gazzarini]

Hi Anil,
The copy Field directive is not what you're looking for because it doesn't
change the stored value of a field.

What you need is an Update Request Processor, which is a kind of
interceptor in the indexing chain (i.e. It allows you to change an incoming
document before it gets indexed).
Unfortunately, as far as I know there's not an available processor for
doing what you need in the example you described, but consider that writing
a new processor is a trivial thing.

Andrea

Il gio 12 lug 2018, 19:23 Anil  ha scritto:

> HI,
>
> i have a date field which needs to copied to different field with different
> format/value. is there any way  to achieve this using copy field ? or needs
> to be done when creating solr document itself.
>
> lets say createdDate is 10-23-2017 10:15:00, it needs to be copied to
> transformedDate field as  10-23-2017.
>
> please help. thanks.
>
> Regards,
> Anil
>

Re: solr filter query on text field

[Andrea Gazzarini]

The syntax is valid in all those three examples, the right one depends on
what you need.

The first query executes a proximity search (you can think to a phrase
search, for simplicity) so it returns no result because probably you don't
have any matching docs with that whole literal.

The second is querying the my_text_field for all terms which compose the
value between parenthesis. You can think to a query where each term is an
optional clause, something like mytextfield:jurassic OR mytextfiekd:park...
(it's not exactly an OR but this could give you the idea=

The third example is not doing what you think. My_text_field is used only
with the first term (Jurassic) while the others are using the default
field. Something like mytextfield:jurassic OR defaultfield:park OR
defaultfield:the That's the reason  you have so many results (I guess
the default field is a catch-all field)

Sorry for typos I'm using my mobile

Andrea

Il mer 11 lug 2018, 17:54 Wei  ha scritto:

> Hi,
>
> I am running filter query on a field of text_general type and see
> completely different results for the following queries:
>
>fq= my_text_field:"Jurassic park the movie"   returns 0
> result
>
>fq= my_text_field:(Jurassic park the movie)   returns 20
> result
>
>fq= my_text_field:Jurassic park the movie  returns
> thousands of results
>
>
> Which one is the correct syntax? I am confused why the first query doesn't
> have any match at all.  I also thought 2 and 3 are the same, but turns out
> quite different.
>
>
> Thanks,
> Wei
>

Re: Error in solr Plugin

[Andrea Gazzarini]

Hi,
I mean you should use Maven which would pickup, starting from a number 
(e.g. 6.6.1), all the correct dependencies you need for developing the 
plugin.
Yes, the "top" libraries (e.g. Solr and Lucene) should have the same 
version but on top of that, the plugin could require some other direct 
or indirect dependency, and here there's no a magic rule for determining 
which version of such dependency you would need. So again: please use 
Maven (or Gradle).


About the second point: generally speaking, if you're targeting Solr 
6.6.1 your plugin will live in that context, with all the beauties 
coming with the 6.6.1 version. And here, you can imagine why you cannot 
use something that is not yet there, something that will be implemented 
in a later version.


On top of that, if you meant the TokenStream.incrementToken method 
please check better: this is part of the core Lucene API and unless I 
didn't get you, you will find it also in past versions.


Best,
Andrea

On 24/06/18 14:35, Zahra Aminolroaya wrote:

Thanks Andrea. Do you mean all of my jar file versions should be 6.6.1?

The lucene-core7 had some useful functions like  incrementToken which I
could not find in previous versions because of that I used this version.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Error in solr Plugin

[Andrea Gazzarini]

Hi Zahra,
I think your guessing it's right: I see some mess in libraries versions. 
If I got you


 * the target platform is Solr 6.6.1
 * the compile classpath includes solr-core-4.1.0, 1.4.0 (!) and lucene
   7.4.0?

If that is correct, with a ClassCastException you're just scraping the 
surface of all errors you can get :) As a rule of thumb you should 
always work with the same version of the target Solr (because that 
version will be the runtime container of your plugin).


I suggest you to avoid putting libraries manually in the classpath; you 
should use a dependency management tool like Maven or Gradle, set the 
target Solr version once and have the required dependencies 
automatically set up.


Let me know if you need some help with the last point.

Best,
Andrea

On 24/06/18 14:07, Zahra Aminolroaya wrote:

I am using solr 6.6.1. I want to write my own analyzer for the field type
"text_general" in schema. the field type in schema is as follows:
   
 
   
   
   
  
 
When I test the filter in Java, everything is alright; However, when I start
my solr I get the following error:

Caused by: java.lang.ClassCastException: class
normalizing.myNormalizerFilterFactory
at java.lang.Class.asSubclass(Unknown Source)
at
org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:539)
at
org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:624)
at
org.apache.solr.schema.FieldTypePluginLoader$3.create(FieldTypePluginLoader.java:397)
at
org.apache.solr.schema.FieldTypePluginLoader$3.create(FieldTypePluginLoader.java:390)
at
org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:152)
... 20 more

I used the jar files in my lib as follows: solr-core-4.1.0,
slf4j-api-1.6.6, lucene-core-7.4.0, apache-solr-core-1.4.0, apache-lucene.

Why do I get this error? Is it becuase of the wrong jar file versions
especially the lucene-core-7.4.0 as my lucene version is 6.6.1?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Spring Boot missing required field:

[Andrea Gazzarini]

Hi Rushikesh,
If the issue is: "when I set required=true Solr says the field is 
missing, and if I set required="false" I have no problem at all, but 
Solr documents have no value for that field", then trust me, the field 
is missing.


I see two possible points where the issue could be:

 * client side: you say "I have value associated with field"; although
   I think you're saying this because you're really sure about that, I
   suggest you to print out the document just before it is sent to
   Solr. 99% I bet you won't find a value for that field, otherwise at
   last we are sure what is obvious for you it is obvious for the
   machine as well
 * Solr side: are you using some UpdateRequestProcessor in the index
   chain? In case, Is it possible that one of these components is
   removing for somewhat reason that field from the incoming document?

Cheers,
Gazza

On 19/06/18 15:19, Rushikesh Garadade wrote:

Yes Andrea,
I have already tried that, I have value associated with field.
This issue is coming when I set 'required="true"' . If i remove this then
everything works fine. I am not getting why this issue occurs when I set
required="true".

Can you please provide me some pointers to look see what may be the reason.

Thanks,
Rushikesh Garadade

On Sat, Jun 9, 2018 at 2:56 PM Andrea Gazzarini 
wrote:


Hi Rushikesh,
I bet your client is not doing what you think. The error is clear, the
incoming document doesn't have that field.

I would investigate more on the client side. Without entering in
interesting fields like unit testing, I guess the old and good
System.out.println, just before sending the document to Solr, could help
you a lot here.

Best,
Andrea

On Sat, 9 Jun 2018, 10:08 Rushikesh Garadade, 
Hi,
I am using solr 7.3 with java spring boot. I schema of my collection I

have

set schema as

indexed="true"

required="true" stored="true"/>
I have done all other necessary settings required for projects to run.

When I run a application and trying to insert document via CODE. I am
getting error "*missing required field : mailReceiveDate *". Although I
have provided the field value.
Following details code error of the same


Caused by:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:

Error

from server at http://:8983/solr:
[doc=8ac2bcf6-7a56-4fed-b83e-7ccc00454088] missing required field:
mailReceiveDate
at



org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:612)

~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
sarowe - 2018-03-02 15:09:35]
at



org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)

~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
sarowe - 2018-03-02 15:09:35]
at



org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)

~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
sarowe - 2018-03-02 15:09:35]
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
sarowe - 2018-03-02 15:09:35]
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
sarowe - 2018-03-02 15:09:35]
at



org.springframework.data.solr.core.SolrTemplate.lambda$saveBean$2(SolrTemplate.java:219)

~[spring-data-solr-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at



org.springframework.data.solr.core.SolrTemplate.execute(SolrTemplate.java:166)

~[spring-data-solr-3.0.7.RELEASE.jar:3.0.7.RELEASE]
... 58 common frames omitted


Please let me know what can be the issue?

Thanks,
Rushikesh Garadade

Re: Spring Boot missing required field:

[Andrea Gazzarini]

Hi Rushikesh,
I bet your client is not doing what you think. The error is clear, the
incoming document doesn't have that field.

I would investigate more on the client side. Without entering in
interesting fields like unit testing, I guess the old and good
System.out.println, just before sending the document to Solr, could help
you a lot here.

Best,
Andrea

On Sat, 9 Jun 2018, 10:08 Rushikesh Garadade, 
wrote:

> Hi,
> I am using solr 7.3 with java spring boot. I schema of my collection I have
> set schema as
>  required="true" stored="true"/>
> I have done all other necessary settings required for projects to run.
>
> When I run a application and trying to insert document via CODE. I am
> getting error "*missing required field : mailReceiveDate *". Although I
> have provided the field value.
> Following details code error of the same
>
>
> Caused by:
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
> from server at http://:8983/solr:
> [doc=8ac2bcf6-7a56-4fed-b83e-7ccc00454088] missing required field:
> mailReceiveDate
> at
>
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:612)
> ~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
> sarowe - 2018-03-02 15:09:35]
> at
>
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
> ~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
> sarowe - 2018-03-02 15:09:35]
> at
>
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
> ~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
> sarowe - 2018-03-02 15:09:35]
> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
> ~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
> sarowe - 2018-03-02 15:09:35]
> at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
> ~[solr-solrj-6.6.3.jar:6.6.3 d1e9bbd333ea55cfa0c75d324424606e857a775b -
> sarowe - 2018-03-02 15:09:35]
> at
>
> org.springframework.data.solr.core.SolrTemplate.lambda$saveBean$2(SolrTemplate.java:219)
> ~[spring-data-solr-3.0.7.RELEASE.jar:3.0.7.RELEASE]
> at
>
> org.springframework.data.solr.core.SolrTemplate.execute(SolrTemplate.java:166)
> ~[spring-data-solr-3.0.7.RELEASE.jar:3.0.7.RELEASE]
> ... 58 common frames omitted
>
>
> Please let me know what can be the issue?
>
> Thanks,
> Rushikesh Garadade
>

Re: Apache and Apache Solr together

[Andrea Gazzarini]

Hi,
all what you need is in Solr, better: all what you need is Solr.

Solr is a server which exposes its services through HTTP, so you don't 
need Apache at all (at least for a training course).


Best,
Andrea

On 06/06/18 08:57, azharuddin wrote:

I've got a question: I came across  Apache Solr
   as requirement for a module
I'm installing and even after reading the documentation on Apache Solr's
official homepage I'm still not sure whether Apache runs alongside regular
Apache or does it require it own server? If it does work alongside Apache,
is there any known issues/problems that I should be aware of? How would this
architecture (Apache and Apache Solr) in terms of file system and serving
pages? I'm sorry if the question might sound silly but I'm very new to the
whole server-side programming/setup world.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Parent product show in search result

[Andrea Gazzarini]

Hi,
as far as I remember the Magento integration (at least the connector 
version I worked with) doesn't have such capabilities, so if I remember 
well and it is still valid, your need would require some custom (client) 
code.


The alternative, which would move the search workflow entirely in Solr, 
is to use a "composite" request handler which would execute the logic 
you described. Everything would be completely transparent to the caller 
(i.e. Magento).


You can find a working version of such component here [1].
It will exactly do what you need:

 * Run a first query with the parent filter query
 * in case of zero results, it will execute another query with the
   children filter

Again, everything would be transparent to the call, which would issue 
one single query.


Best,
Andrea

[1] https://github.com/SeaseLtd/composite-request-handler

On 05/06/18 10:09, Charlie Hull wrote:

On 04/06/2018 17:15, Apurba Hazra wrote:

Hello,

We are implementing solr search for our webseite using magento.

Our requirement is, in search result page we have to show only parent
product not all child product if the parent exist, otherwise we have to
show child product.

Will you please tell us how we can do that. Should we change setting in
solr panel as well as magento admin panel.

Please advice us, it's very urgent.


Hi,

How and more importantly *if* you can do this will depend on how Solr 
has been integrated with Magento. Magento documentation, mailing lists 
etc. should be your first port of call.


Best

Charlie



*Thanks & Regards,*
*Apurba Hazra*

*Project Manager*

*Navigator Software Pvt. Ltd.*
Web Applications /  Enterprise Mobility & Mobile Apps / Cloud 
Solutions /
E-Commerce / Bespoke and Product development / Enterprise CMS / 
Online POS /
VOIP Solutions / Internet Marketing / Business Intelligence & 
Analytics /

Dedicated Hiring Solutions.

www.needdevelopers.com
www.boostmysale.com
www.navsoft.in

20 Dr. E Moses Road, Mahalakshmi, Mumbai 400020
205 & 206 Haute Street Bldg., 86A Topsia Road; Kolkata 700046
Tel: (+91-33) 40259595 <00913340259595>

Re: A good KV store/plugins to go with Solr

[Andrea Gazzarini]

Hi Sam, I have been in a similar scenario (not recently so my answer could
be outdated). As far as I remember caching, at least in that scenario,
didn't help so much, probably because the field size.

So we went with the second option: a custom SearchComponent connected with
Redis. I'm not aware if such component is available somewhere but, trust
me, it's a very easy thing to write.

Best,
Andrea

On Mon, 4 Jun 2018, 20:45 Sambhav Kothari,  wrote:

> Hi everyone,
>
> We at MetaBrainz are trying to scale our solr cloud instance but are
> hitting a bottle-neck.
>
> Each of the documents in our solr index is accompanied by a '_store' field
> that store our API compatible response for that document (which is
> basically parsed and displayed by our custom response writer).
>
> The main problem is that this field is very large (It takes upto 60-70% of
> our index) and because of this, Solr is struggling to keep up with our
> required reqs/s.
>
> Any ideas on how to improve upon this?
>
> I have a couple of options in mind -
>
> 1. Use caches extensively.
> 2. Have solr return only a doc id and fetch the response string from a KV
> store/fast db.
>
> About 2 - are there any solr plugins will allow me to do this?
>
> Thanks,
> Sam
>

Re: Mysterious Solr crash

[Andrea Gazzarini]

Looking at the stack trace, which seem truncated, I would start from here

o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Early EOF
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:190)

Could you please expand a bit your context (e.g. solr version, cloud / no
cloud, xml data size, indexing client type). Also, the whole stacktrace
(unless it is huge), should be helpful, too.

Best,
Andrea


On 3 Jun 2018 6:18 pm, "Zheng Lin Edwin Yeo"  wrote:

Hi,

Which version of Solr are you running on? And also, are you running on
SolrCloud or Standalone Solr?

Regards,
Edwin


On 3 June 2018 at 21:52, Nawab Zada Asad Iqbal  wrote:

> Good morning
>
> I am running a batch indexing job and Solr core mysteriously shut down
> without any particular error. How can I investigate this? I am focusing on
> the line which mentions "Shutting down CoreContainer instance".
>
> There are errors soon after that, but they seemed to be caused by the core
> shutting down and not the other way round; although i can be wrong.
>
>
> Thanks
> Nawab
>
> 
> Jun 03, 2018 03:02:02 AM INFO  (qtp761960786-1049) [   x:filesearch]
> o.a.s.u.p.LogUpdateProcessorFactory [filesearch]  webapp=/solr
> path=/update
> params={commit=false}{add=[file_30594113079, file_221594113074,
> file_92594113074, file_93594113076, file_94594113078, file_155594113071,
> file_285594113073, file_286594113075, file_13694113071, file_16694113077]}
> 0 201
> Jun 03, 2018 03:02:02 AM INFO  (qtp761960786-911) [   x:filesearch]
> o.a.s.u.p.LogUpdateProcessorFactory [filesearch]  webapp=/solr
> path=/update
> params={commit=false}{add=[file_98928108921, file_113038108926,
> file_114038108928, file_46038108921, file_48038108925, folder_48038108925,
> file_100138108926, file_291138108921, file_292138108923,
file_33138108921]}
> 0 113
> Jun 03, 2018 03:02:02 AM INFO  (Thread-0) [   ] o.a.s.c.CoreContainer
> Shutting down CoreContainer instance=1021436681
> Jun 03, 2018 03:02:02 AM INFO  (Thread-0) [   ] o.a.s.m.SolrMetricManager
> Closing metric reporters for registry=solr.node, tag=null
> Jun 03, 2018 03:02:02 AM INFO  (Thread-0) [   ] o.a.s.m.SolrMetricManager
> Closing metric reporters for registry=solr.jvm, tag=null
> Jun 03, 2018 03:02:02 AM INFO  (Thread-0) [   ] o.a.s.m.SolrMetricManager
> Closing metric reporters for registry=solr.jetty, tag=null
> Jun 03, 2018 03:02:02 AM INFO  (qtp761960786-851) [   x:filesearch]
> o.a.s.u.p.LogUpdateProcessorFactory [filesearch]  webapp=/solr
> path=/update
> params={commit=false}{add=[file_34202892465, file_35202892467,
> file_226202892462]} 0 75
> Jun 03, 2018 03:02:02 AM INFO  (qtp761960786-1124) [   x:filesearch]
> o.a.s.u.p.LogUpdateProcessorFactory [filesearch]  webapp=/solr
> path=/update
> params={commit=false}{add=[folder_31457908173, file_32457908175,
> folder_32457908175, file_33457908177, file_34457908179, file_94457908170,
> folder_20557908177, file_19557908175]} 0 115
> Jun 03, 2018 03:02:02 AM ERROR (qtp761960786-851) [   x:filesearch]
> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Early EOF
> at org.apache.solr.handler.loader.XMLLoader.load(
> XMLLoader.java:190)
> at
> org.apache.solr.handler.UpdateRequestHandler$1.load(
> UpdateRequestHandler.java:97)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(
> ContentStreamHandlerBase.java:68)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(
> RequestHandlerBase.java:177)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2484)
> at
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
> at org.apache.solr.servlet.HttpSolrCall.call(
> HttpSolrCall.java:526)
>

Re: Weird behavioural differences between pf in dismax and edismax

[Andrea Gazzarini]

Hi Sam,
I noticed the same behaviour. Looking at the code it seems that it is 
expected: the two classes (ExtendedDisMaxQParser and DisMaxQParser) 
don't have a direct inheritance relationships and the methods which deal 
with the PF parameter are different. Specifically, the 
DismaxQParser.getPhraseQuery seems to produce the query phrase 
regardless the number of terms that compose the query (and this matches 
with the observed behaviour), while the ExtendedDismax seems to take in 
account this aspect .


I agree with you, it results in a different behaviour, even for those 
single-word queries that output more than one terms (e.g. putting a pf 
clause with a q=hello-world and a field in qf which uses a 
StandardTokenizer or a WordDelimiterFilter in the analyzer).


About the reason of such different implementation, I don't know, maybe 
someone else here is able to help you.


Best,
Andrea

On 27/05/18 15:14, Sambhav Kothari wrote:

Hello,

I experienced a weird behaviour with dismax and edismax query parsers.
Dismax will include pf boosts when we query something that has just a
single word, edismax on the other hand will not include pf boosts.

The result is that a dismax and an edismax handler with the same set of
defaults, return different results for single word queries (eg. "Hello")
but the same results for multi word queries (eg. "Hello Wold")

Is this expected?

Regards,
Sam

Re: Index and query time suggester behavior in a SolrCloud environment

[Andrea Gazzarini]

Ah great, many thanks again!


On 20/04/17 17:09, Shalin Shekhar Mangar wrote:

Hi Andrea,

Looks like I have you some bad information. I looked at the code and
ran a test locally. The suggest.build and suggest.reload params are in
fact distributed across to all shards but only to one replica of each
shard. This is still bad enough and you should use buildOnOptimize as
suggested but I just wanted to correct the wrong information I gave
earlier.

On Thu, Apr 20, 2017 at 6:23 PM, Andrea Gazzarini <gxs...@gmail.com> wrote:

Perfect, I don't need NRT at this moment so that fits perfectly

Thanks,
Andrea


On 20/04/17 14:37, Shalin Shekhar Mangar wrote:

Yeah, if it is just once a day then you can afford to do an optimize.
For a more NRT indexing approach, I wouldn't recommend optimize at
all.

On Thu, Apr 20, 2017 at 5:29 PM, Andrea Gazzarini <gxs...@gmail.com>
wrote:

Ok, many thanks

I see / read that it should be better to rely on the background merging
instead of issuing explicit optimizes, but I think in this case one
optimize
in a day it shouldn't be a problem.

Did I get you correctly?

Thanks again,
Andrea


On 20/04/17 13:17, Shalin Shekhar Mangar wrote:

Can the client not send an optimize command explicitly after all
indexing/deleting is complete?

Re: Index and query time suggester behavior in a SolrCloud environment

[Andrea Gazzarini]

Perfect, I don't need NRT at this moment so that fits perfectly

Thanks,
Andrea

On 20/04/17 14:37, Shalin Shekhar Mangar wrote:

Yeah, if it is just once a day then you can afford to do an optimize.
For a more NRT indexing approach, I wouldn't recommend optimize at
all.

On Thu, Apr 20, 2017 at 5:29 PM, Andrea Gazzarini <gxs...@gmail.com> wrote:

Ok, many thanks

I see / read that it should be better to rely on the background merging
instead of issuing explicit optimizes, but I think in this case one optimize
in a day it shouldn't be a problem.

Did I get you correctly?

Thanks again,
Andrea


On 20/04/17 13:17, Shalin Shekhar Mangar wrote:

Can the client not send an optimize command explicitly after all
indexing/deleting is complete?

Re: Index and query time suggester behavior in a SolrCloud environment

[Andrea Gazzarini]

Ok, many thanks

I see / read that it should be better to rely on the background merging 
instead of issuing explicit optimizes, but I think in this case one 
optimize in a day it shouldn't be a problem.


Did I get you correctly?

Thanks again,
Andrea

On 20/04/17 13:17, Shalin Shekhar Mangar wrote:

Can the client not send an optimize command explicitly after all
indexing/deleting is complete?

Re: Index and query time suggester behavior in a SolrCloud environment

[Andrea Gazzarini]

Hi Shalin,
many thanks for your response. This is my scenario:

 * I build my index once in a day, it could be a delta or a full
   re-index.In any case, that takes some time;
 * I have an auto-commit (hard, no soft-commits) set to a given period
   and during the indexing cycle, several hard commits are executed. So
   the buildOnCommit (I guess) it's not an option because it will
   rebuild that suggest index several times.

But I have a doubt on the second point: the reference guide says:

/"Use buildOnCommit to rebuild the dictionary with every soft-commit"/

As I said, I have no soft-commits only hard-commits: does the rebuild 
happen after hard commits (with buildOnCommit=true)?


The other option, buildOnOptimize, makes me curious: in the scenario 
above, let's say documents are indexed / deleted every morning at 4am, 
in a window that takes 1 max 3 hours, how can I build the suggest index 
(more or less) just after that window? I'm ok if the build happens after 
a reasonable delay (e.g. 1, max 2 hours)


Many thanks,
Andrea


On 20/04/17 11:11, Shalin Shekhar Mangar wrote:

Comments inline:


On Wed, Apr 19, 2017 at 2:46 PM, Andrea Gazzarini <gxs...@gmail.com> wrote:

Hi,
any help out there?

BTW I forgot the Solr version: 6.5.0

Thanks,
Andrea


On 18/04/17 11:45, Andrea Gazzarini wrote:

Hi,
I have a project, with SolrCloud, where I'm going to use the Suggester
component (BlendedInfixLookupFactory with DocumentDictionaryFactory).
Some info:

   * I will have a suggest-only collection, with no NRT requirements
 (indexes will be updated with a daily frequency)
   * I'm not yet sure about the replication factor (I have to do some
 checks)
   * I'm using Solrj on the client side

After reading some documentation I have a couple of doubts:

   * how the *suggest.build* command is working? Can I issue this
 command towards just one node, and have that node forward the
 request to the other nodes (so each of them can build its own
 suggester index portion)?

The suggest.build only builds locally in the node to which you sent
the request. This makes it a bit tricky because if you send that
command with just the collection name, it will be resolved to a local
core and executed there. The safest/easiest way is to set
buildOnCommit or buildOnOptimize in the suggester configuration.


   * how things are working at query time? Can I use send a request
 with only suggest.q=... to my /suggest request handler and get
 back distributed suggestions?

The SuggestComponent works in distributed mode and it will request and
merge results from all shards.


Thanks in advance
Andrea

Re: Index and query time suggester behavior in a SolrCloud environment

[Andrea Gazzarini]

Hi,
any help out there?

BTW I forgot the Solr version: 6.5.0

Thanks,
Andrea

On 18/04/17 11:45, Andrea Gazzarini wrote:

Hi,
I have a project, with SolrCloud, where I'm going to use the Suggester 
component (BlendedInfixLookupFactory with DocumentDictionaryFactory).

Some info:

  * I will have a suggest-only collection, with no NRT requirements
(indexes will be updated with a daily frequency)
  * I'm not yet sure about the replication factor (I have to do some
checks)
  * I'm using Solrj on the client side

After reading some documentation I have a couple of doubts:

  * how the *suggest.build* command is working? Can I issue this
command towards just one node, and have that node forward the
request to the other nodes (so each of them can build its own
suggester index portion)?
  * how things are working at query time? Can I use send a request
with only suggest.q=... to my /suggest request handler and get
back distributed suggestions?

Thanks in advance
Andrea

Index and query time suggester behavior in a SolrCloud environment

[Andrea Gazzarini]

Hi,
I have a project, with SolrCloud, where I'm going to use the Suggester 
component (BlendedInfixLookupFactory with DocumentDictionaryFactory).

Some info:

 * I will have a suggest-only collection, with no NRT requirements
   (indexes will be updated with a daily frequency)
 * I'm not yet sure about the replication factor (I have to do some checks)
 * I'm using Solrj on the client side

After reading some documentation I have a couple of doubts:

 * how the *suggest.build* command is working? Can I issue this command
   towards just one node, and have that node forward the request to the
   other nodes (so each of them can build its own suggester index portion)?
 * how things are working at query time? Can I use send a request with
   only suggest.q=... to my /suggest request handler and get back
   distributed suggestions?

Thanks in advance
Andrea

Re: How to get field names of dynamic field

[Andrea Gazzarini]

I can see those names in the "Schema  browser" of the admin UI, so I guess
using the (lucene?) API it shouldn't be hard to get this info.

I don' know if the schema api (or some other service) offer this service

Andrea

On 14 Apr 2017 10:03, "Midas A"  wrote:

> Hi,
>
>
> Can i get all the field created for dynamic field in solr .
>
> Like
> my dynamic field is by_*
>
> and i have index
> by_color
> by_size ..
> etc
>
> I want to retrieve all these field name .
> Is there any way to do this  based on some query
>

Re: KeywordTokenizer and multiValued field

[Andrea Gazzarini]

Hi Wunder,
I think it's the first option: if you have 3 values then the analyzer 
chain is executed three times.


Andrea

On 12/04/17 18:45, Walter Underwood wrote:

Does the KeywordTokenizer make each value into a unitary string or does it take 
the whole list of values and make that a single string?

I really hope it is the former. I can’t find this in the docs (including 
JavaDocs).

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

Re: Autosuggestion

[Andrea Gazzarini]

Hi,
I think you got an old post. I would have a look at the built-in 
feature, first. These posts can help you to get a quick overview:


https://cwiki.apache.org/confluence/display/solr/Suggester
http://alexbenedetti.blogspot.it/2015/07/solr-you-complete-me.html
https://lucidworks.com/2015/03/04/solr-suggester/

HTH,
Andrea

On 12/04/17 14:43, OTH wrote:

Hello,

Is there any recommended way to achieve auto-suggestion in textboxes using
Solr?

I'm new to Solr, but right now I have achieved this functionality by using
an example I found online, doing this:

I added a copy field, which is of the following type:

   
 
   
   
 
 
   
   
 
   

In the search box, after each character is typed, the above field is
queried, and the results are shown in a drop-down list.

However, this is performing quite slow.  I'm not sure if that has to do
with the front-end code, or because I'm not using the recommended approach
in terms of how I'm using Solr.  Is there any other recommended way to use
Solr to achieve this functionality?

Thanks

Re: query the data contain key value

[Andrea Gazzarini]

Solr doesn't know about "null" values. Using a schema which declares all 
those fields (id, eventA, eventB, eventC) and indexing those 2 documents


1,eventA=2,eventC=3
2,eventB=1,eventA=1

You already get a situation "similar" to what you want. I said "Similar" 
because you won't have any null value: a given field value for a given 
document won't exist at all:


 * ID 1: A=2, C=3
 * ID 2: A=1,B=1

If I misunderstood the question, please expand (from a functional point 
of view) what you want to achieve.


Best,
Andrea

On 15/03/17 09:44, tkg_cangkul wrote:

Hi, i have data sample to index it into solr.
below is the format of my sample data :

id,field1,field2
1,eventA=2,eventC=3
2,eventB=1,eventA=1

how can i query or indexing that data format in solr to get the result 
like this :


id,eventA,eventB,eventC
1,2,null,3
2,1,1,null

is there any way to do that?

pls help

Re: query the data contain key value

[Andrea Gazzarini]

Solr doesn't know about "null" values. Using a schema which declares all 
those fields (id, eventA, eventB, eventC) and indexing those 2 documents


1,eventA=2,eventC=3
2,eventB=1,eventA=1

You already get a situation "similar" to what you want. I said "Similar" 
because you won't have any null value: a given field value for a given 
document won't exist at all:


 * ID 1: A=2, C=3
 * ID 2: A=1,B=1

If I misunderstood the question, please expand (from a functional point 
of view) what you want to achieve.


Best,
Andrea

On 15/03/17 09:44, tkg_cangkul wrote:

Hi, i have data sample to index it into solr.
below is the format of my sample data :

id,field1,field2
1,eventA=2,eventC=3
2,eventB=1,eventA=1

how can i query or indexing that data format in solr to get the result 
like this :


id,eventA,eventB,eventC
1,2,null,3
2,1,1,null

is there any way to do that?

pls help

Re: SOLR 4.8.0 Master/Slave Replication Issues

[Andrea Gazzarini]

Hi Scott,
that could depend on a lot of things. Some questions:

 * What is your commit policy? Explicit / auto / soft / hard ...
 * "Other days things are off by a small amount between master and
   slave"...what do you mean exactly? What is the behaviour you see in
   terms of index versions between nodes?
 * Any error / message in the log file?

Honestly I never used Solr 4.8.0 (this specific version) but I have a 
lot of customers with other 4.x (mostly 4.10.4) master / slave 
configuration and they don't have such problems.


Best,
Andrea

On 09/03/17 22:26, Pouliot, Scott wrote:

So we've been having oddball replication issues between master and slave (Not 
SOLRCloud) for a few weeks now.  Some days...everything lines up just fine.  
Other days things are off by a small amount between master and slave.

Here is the config on the MASTER server replication config:


   
 
   
   startup
   commit

   
   00:00:10
 
 
 
 1
   

Here is the SLAVE server replication config:


 

   
   http://server01:8080/solr/${solr.core.name}

   
   00:01:00
 
   

It seems very sporadic at best.  Sometimes I can manually click "Replicate Now" from the 
replication page in the SOLR UI and it will work, sometimes it does nothing.  Sometimes I can go to 
the master server, and if the "Optimize" button is available and I click itit will 
replicate as soon as the Optimization is done.

Is this a bug in SOLR?  Could we possibly have some settings out of whack here? 
 I've been digging around online for a bit and not finding much info here.

This is on an older version of SOLR though, so wondering if maybe it used to be 
a bug?

Thanks!

Scott

Re: How to create solr custom filter

[Andrea Gazzarini]

Hi Mugeesh,
my fault: a point is missing there, as suggested

/"//*-ea *//was not specified but "/

//

You need to add the "-ea" VM argument. If you are in Eclipse,

/Run >> Run Configurations/

then in the dialog that appears, select the run configuration 
corresponding to that class (StartDevSolr), and click on the second tab 
("Arguments"). There you will find two text areas, type *-ea* in the "VM 
Arguments" textarea.


HTH
Andrea

On 08/02/17 06:09, Mugeesh Husain wrote:

thanks andrea for your help, I created few solr plugin that working fine, but
still i am stuck to debug the code using eclipse, as you mentioned below
url.http://andreagazzarini.blogspot.in/2016/11/quickly-debug-your-solr-add-on.htmlIn
this url, i could not run the junit code, i couldn't run StartDevSolr
file(its dosn't showing me junit run/debug), when I  removed abstract method
from StartDevSolr class the  it showing me below error,  Assertions
mismatch: -ea was not specified but -Dtests.asserts=trueFeb 08, 2017
10:30:29 AM com.carrotsearch.randomizedtesting.RandomizedRunner
runSuiteSEVERE: Panic: RunListener hook shouldn't throw
exceptions.java.lang.NullPointerException   at
org.apache.lucene.util.RunListenerPrintReproduceInfo.printDebuggingInformation(RunListenerPrintReproduceInfo.java:131)
at
org.apache.lucene.util.RunListenerPrintReproduceInfo.testRunFinished(RunListenerPrintReproduceInfo.java:118)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:706)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:591)I
tried above code using solr 6.2.0.I am newone for jnuit may be i am getting
this issue, if you have any more debug url, let me know.or  suggest me?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-create-solr-custom-filter-tp4317767p4319258.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: custom plugin version

[Andrea Gazzarini]

Hi Zaccheo,
I don't think this is possible, this is something related with the 
classloader behavior, and even if there's a "priority" rule in the JVM, 
I wouldn't rely on that in my application.
That could be good in a dev environment where you can specify the 
"order" of the imported libraries (e.g. Eclipse), but definitely not so 
good outside (IMO).


As far as I know, there's no a built-in way to declare the version of 
custom components, but you could adopt the same approach of Lucene, with 
something like a Version class that drives the behavior of your component.

In this way you will have

 * always one jar (better: unique classes FQNs), so no classloader issues
 * a behavior that changes depending on the configuration

Best,
Andrea

On 03/02/17 10:57, Zaccheo Bagnati wrote:

Hi all,
I developed a custom DocTransformer that is loaded from a .jar in the core
"lib" directory. It works but I have now a problem with versioning:
1. if lib directory contains different versions of the same .jar which one
is loaded? I tried putting both myplugins-1.0.0.jar and myplugins-1.0.1.jar
and I noticed that the oldest one is loaded. Is there a way to force
specific jar version to be loaded in solrconfig?
2. More in general: is it possible to expose in solr the version number for
custom plugins?
Thank you in advance

Re: How to create solr custom filter

[Andrea Gazzarini]

On top of what Alessandro already told you, here's a brief post [1] that 
can be useful for setting up your dev environment.


HTH
Andrea

[1] 
http://andreagazzarini.blogspot.it/2016/11/quickly-debug-your-solr-add-on.html 



On 30/01/17 11:16, alessandro.benedetti wrote:

Generally speaking I assume you want to write a Solr plugin [1] ( or you are
forking Solr instead?)
So you need to package your jar.
You can setup your IDE as you usually do, and use a building framework such
as Maven/Gradle.
After you have your jar you can easily import it in Solr, specifying the
location in the solrconfig.xml.



[1] https://wiki.apache.org/solr/SolrPlugins,
http://shulhi.com/creating-solr-plugin/




--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-create-solr-custom-filter-tp4317767p4317784.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Stable releases of Solr

[Andrea Gazzarini]

Hi Deepak,
the latest version is the 6.3.0 and I guess it is the best to pick up. 
Keep in mind that 3.6.1 => 6.3.0 is definitely a big jump.


In general, I think once a version is made available, that means it is 
(hopefully) stable.


Best,
Andrea

On 16/12/16 08:10, Deepak Kumar Gupta wrote:

Hi,

I am planning to upgrade lucene version in my codebase from 3.6.1
What is the latest stable version to which I can upgrade it?
Is 6.3.X stable?

Thanks,
Deepak

Re: [ANN] InvisibleQueriesRequestHandler

[Andrea Gazzarini]

Hi Charlie,
Great to hear that! I never worked in a Drupal / Hybris -> Solr
integration. So it seems things sound moreless like the Magento scenario
That means what I did could make a sense and most important, it could be
useful for someone.

Best,
Andrea

On 5 Dec 2016 18:08, "Charlie Hull" <char...@flax.co.uk> wrote:

> On 05/12/2016 09:18, Andrea Gazzarini wrote:
>
>> Hi guys,
>> I developed this handler [1] while doing some work on a Magento ->  Solr
>> project.
>>
>> If someone is interested (this is a post [2] where I briefly explain the
>> goal), or wants to contribute with some idea / improvement, feel free to
>> give me a shout or a feedback.
>>
>> Best,
>> Andrea
>>
>> [1] https://github.com/agazzarini/invisible-queries-request-handler
>> [2]
>> https://andreagazzarini.blogspot.it/2016/12/composing-and-
>> reusing-request-handlers.html
>>
>> We like this idea: we've seen plenty of systems where it's hard to change
> what the container system using Solr is doing (e.g. Hybris, Drupal...) so
> to be able to run multiple searches in Solr itself is very useful. Nice one!
>
> Charlie
>
>
> --
> Charlie Hull
> Flax - Open Source Enterprise Search
>
> tel/fax: +44 (0)8700 118334
> mobile:  +44 (0)7767 825828
> web: www.flax.co.uk
>
>

Re: [ANN] InvisibleQueriesRequestHandler

[Andrea Gazzarini]

Hi Erik,
interesting approach, but, please correct me if I didn't get you, this is
different because

- it requires some kind of control on the client side and for example, in
Magento you don't have that: it is not aware about group responses or
arbitrary facet queries (BTW the Magento/Solr connector officially supports
Solr 3.6.2 :( )

- it always executes all group / facet queries without any conditional /
cascading logic between them

Again please correct me if I misunderstood your approach.

Thanks for the hint
Andrea

On 5 Dec 2016 19:14, "Erik Hatcher" <erik.hatc...@gmail.com> wrote:

Another technique for this is to use Grouping’s `group.query` a few times,
with exact to fuzzier types of queries and get it all back in one
response.  So you _can_ run multiple searches in a single query already :)

I’ve used a similar technique with faceting and `facet.query` to give you
counts of exact to fuzzier types of queries to get the counts.

Erik



> On Dec 5, 2016, at 9:08 AM, Charlie Hull <char...@flax.co.uk> wrote:
>
> On 05/12/2016 09:18, Andrea Gazzarini wrote:
>> Hi guys,
>> I developed this handler [1] while doing some work on a Magento ->  Solr
>> project.
>>
>> If someone is interested (this is a post [2] where I briefly explain the
>> goal), or wants to contribute with some idea / improvement, feel free to
>> give me a shout or a feedback.
>>
>> Best,
>> Andrea
>>
>> [1] https://github.com/agazzarini/invisible-queries-request-handler
>> [2]
>> https://andreagazzarini.blogspot.it/2016/12/composing-
and-reusing-request-handlers.html
>>
> We like this idea: we've seen plenty of systems where it's hard to change
what the container system using Solr is doing (e.g. Hybris, Drupal...) so
to be able to run multiple searches in Solr itself is very useful. Nice one!
>
> Charlie
>
>
> --
> Charlie Hull
> Flax - Open Source Enterprise Search
>
> tel/fax: +44 (0)8700 118334
> mobile:  +44 (0)7767 825828
> web: www.flax.co.uk
>

[ANN] InvisibleQueriesRequestHandler

[Andrea Gazzarini]

Hi guys,
I developed this handler [1] while doing some work on a Magento ->  Solr
project.

If someone is interested (this is a post [2] where I briefly explain the
goal), or wants to contribute with some idea / improvement, feel free to
give me a shout or a feedback.

Best,
Andrea

[1] https://github.com/agazzarini/invisible-queries-request-handler
[2]
https://andreagazzarini.blogspot.it/2016/12/composing-and-reusing-request-handlers.html

Issue in the analysis chain

[Andrea Gazzarini]

Hi,
I found a strange behavior with the MappingCharFilterFactory in Solr 
*6.2.1*. Definitely curious if I'm missing something or someone else met 
that.


I have a (index and query) chain composed as follows:

mapping="mapping-FoldToASCII.txt"/>


...

The mapping-FoldToASCII.txt is the exact file that you can find in the 
Solr download bundle, I didn't add any mapping.
I started having some search issues and after checking, I saw that some 
characters with diacritics weren't replaced. I isolated one of those 
cases and tried to see what's happen in the analysis page.


As expected, the characters weren't replaced so I tried char by char. 
Nothing, it doesn't work.

An example

I pasted īà in the "Field Value (Index)" box. The *ī* char is the 
unicode *\u012b* which is already mapped in the mapping-FoldToASCII.txt


Without the "Verbose Output" flag [1]

 * I see an empty space beside the MCF (where instead I'd expect to see
   the "i", "a" replaced characters)
 * the KeywordTokenizer reports exactly my input "īà" so it seems the
   MCF didn't make any change to the source input

However, if I turn the "Verbose Output" flag on [2]

 * You can see that the MCF is working (i.e. ī becomes i, and à becomes a)
 * But the KeywordTokenizer is still ignoring that and it produces īà

I tried the same with a Solr 4.7.1 instance and as you can see [3] it 
works as I would expect


Any help would be warmly appreciated

Best,
Andrea

[1] https://drive.google.com/file/d/0B82QaJKoMzvWb3dLcW80ME5wdXc/view
[2] https://drive.google.com/file/d/0B82QaJKoMzvWN2lNSF9JQUhPZ3c/view
[3] https://drive.google.com/file/d/0B82QaJKoMzvWeHRzUnU3MGFtY2s/view

Re: solr hierarchical search hyponyms hypernyms

[Andrea Gazzarini]

Hi Francesco,

On 29/09/16 10:47, marcyborg wrote:

Hi Andrea,
Thanks very much for your complete reply.
You're right, I'm new about Solr, so I'm sorry if'm asking trivial
questions, or I'not exaustive in my questions!

About the scenario, I try to explain it:
I have to load the thesaurus in Solr core, and the user would be able to
query that thesaurus, when searching a keyword.
Getting into the details: I search a keyword T, if this T has BT and/or NT,
I'd like to retrive that terms, and show that.
If you mean "user enters term T; in the thesaurus it is associated to a 
BT and NT1, NT2; I want to expand the search using all those terms" then 
I think the most trivial thing you can do is a simple Java standalone 
program (or whatever language you prefer) that loads the thesaurus and 
convert it in the plain synonyms format. Then, you can see in the Solr 
reference guide [1] how to configure that (the default configuration 
should already have set up, if I remember well, just a matter of 
replacing the default synynoms.txt file included in the example).


I suggest you to start reading the reference guide and then go deeper 
into the synonyms topic, which can be very tricky (the "pain in the ass" 
Hoss mentioned in his answer).


Best,
Andrea

[1] 
https://cwiki.apache.org/confluence/display/solr/Filter+Descriptions#FilterDescriptions-SynonymFilter

I hope this clarifies the scenario!

Ciao,
Francesco



--
View this message in context: 
http://lucene.472066.n3.nabble.com/solr-hierarchical-search-hyponyms-hypernyms-tp4298385p4298569.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to use Solr 5.3 using Maven ?

[Andrea Gazzarini]

Hi,
here [1] you can find one way to do that. You can start such class as a 
JUnit test or a simple main, using Maven or not.


Best,
Andrea

[1] 
http://stackoverflow.com/questions/31521345/solr-5-integration-tests-with-maven#33189271


On 28/09/16 15:13, todhanda wrote:

I am using Solr 5.3, and would need to get it working with Apache
Maven(3.1.1).

Is there any way the Maven can be configured to start and/or stop the Solr
5.3 (may be by using any maven plugin) ?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-use-Solr-5-3-using-Maven-tp4298430.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: solr hierarchical search hyponyms hypernyms

[Andrea Gazzarini]

Hi Francesco,
I think an information it's missing here: what are you trying to do 
concretely? "Using Solr as semantic search engine" means at the same 
time everything and nothing :) and (I guess) it involves something more 
than a thesaurus.
Keeping things simple, and assuming your only concern is the thesaurus, 
what is your search requirement / scenario / expectation? I mean, let's 
assume you can index / manage the thesaurus hierarchy, something like


BT
---> T
--> NT1
--> NT2
--> NT3

What do you want to do with such data?
When the user search T you want a query expansion to T,BT?
Or you want to store that information at index time?
Or you want to reduce all T to BT?
What about the narrower terms? How do you imagine their role within the 
search context?


In any case, there's nothing (AFAIK) in Solr that understands SKOS / 
RDF. In the links you posted it seems someone already went in that 
direction but I don't know those add-ons and (see above) I don't know 
your needs.


Ciao,
Andrea

On 28/09/16 12:00, marcyborg wrote:

Hi Andrea,
Thanks very much for your helpful link!
Is there a way to contact Hoss?

About that link, I have two doubts:
-) I'm using Solr (5.4.1) in schemaless mode, and I can't manage the
schema.xml in order to customize the SynonymFilterFactory;
-) If I copy my thesaurus in the synonyms.txt file, I lose all the hierarchy
terms;

According to your experience, these two plugins can be helpful for my
purpose?
http://redlink.co/adding-semantic-search-to-apache-solr/
http://www.opensemanticsearch.org/

Best,
Francesco



--
View this message in context: 
http://lucene.472066.n3.nabble.com/solr-hierarchical-search-hyponyms-hypernyms-tp4298385p4298401.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: solr hierarchical search hyponyms hypernyms

[Andrea Gazzarini]

Hi, I don't believe there's something for doing that in Solr and 
personally I'm not aware if someone developed such filter.


Please have a look at this exchange [1], where Hoss gave some useful 
hints about this topic.


Best,
Andrea

[1] http://osdir.com/ml/solr-user.lucene.apache.org/2010-12/msg00470.html


On 28/09/16 10:25, marcyborg wrote:

Hi,
Refering to this post Does Lucene / Solr support hypernyms and hyponyms?, I
want to use Solr as a semantic search engine, in order to do a hierarchical
search (narrower term, broader term), based on a thesaurus (which is in
XML/RDF format).

I've not found any tokenizers or filters being able of doing this
https://cwiki.apache.org/confluence/display/solr/Tokenizers

Have you any suggestions helping me?

Thanks a lot.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/solr-hierarchical-search-hyponyms-hypernyms-tp4298385.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Load a java class on start up

[Andrea Gazzarini]

You're welcome ;) is that close to what you were looking for?
On 2 Jul 2016 11:53, "Mark Robinson" <mark123lea...@gmail.com> wrote:

> Thanks much Andrea esp. for the suggestion of SolrCoreAware!
>
>
>
> Best,
> Mark.
>
> On Thu, Jun 30, 2016 at 10:23 AM, Andrea Gazzarini <gxs...@gmail.com>
> wrote:
>
> > Hi,
> > the lifecycle of your Solr extension (i.e. the component) is not
> something
> > that's up to you.
> > Before designing the component you should read the framework docs [1], in
> > order to understand the context where it will live, once deployed.
> >
> > There's nothing, as far as I know, other than the component callbacks
> > (e.g. the inform, init methods) that can help you to manage the lifecycle
> > of a custom class you're using within the component. Look at the
> > SolrCoreAware [2] interface, maybe it could fit your needs.
> > From what you write it seems you could need something like a singleton
> > (which is often an anti-pattern in distributed environment) , but without
> > further details I'm just shooting in the dark
> >
> > In addition: you wrote a component so I guess it shouldn't be so hard to
> > have a look at one of the existing built-in components. I'm quite sure
> they
> > already met (and solved) a similar issue.
> >
> > Best,
> > Andrea
> >
> > [1]
> >
> https://lucene.apache.org/solr/6_1_0/solr-core/org/apache/solr/handler/component/SearchComponent.html
> > [2] https://wiki.apache.org/solr/SolrPlugins#SolrCoreAware
> >
> >
> > On 30/06/16 16:00, Mark Robinson wrote:
> >
> >> Hi,
> >>
> >> I have a java OBJECT which I need to load once.
> >> I have written a java custom component, which I have added in
> >> "last-components" in solrconfig.xml, from which I want to access the
> above
> >> mentioned OBJECT when each search request comes in.
> >>
> >> Is there a way I can load a java object on server/ instance startup?
> >> OR
> >> Load it when the first call comes to SOLR?
> >>
> >> For the time being I created that Java object inside the custom
> component
> >> itself; but it is loaded each time a search request comes in.
> >>
> >> Could some one pls give some pointers on how my above requirement can be
> >> achieved in SOLR?
> >>
> >> Thanks!
> >> Mark
> >>
> >>
> >
>

Re: Load a java class on start up

[Andrea Gazzarini]

Hi,
the lifecycle of your Solr extension (i.e. the component) is not 
something that's up to you.
Before designing the component you should read the framework docs [1], 
in order to understand the context where it will live, once deployed.


There's nothing, as far as I know, other than the component callbacks 
(e.g. the inform, init methods) that can help you to manage the 
lifecycle of a custom class you're using within the component. Look at 
the SolrCoreAware [2] interface, maybe it could fit your needs.
From what you write it seems you could need something like a singleton 
(which is often an anti-pattern in distributed environment) , but 
without further details I'm just shooting in the dark


In addition: you wrote a component so I guess it shouldn't be so hard to 
have a look at one of the existing built-in components. I'm quite sure 
they already met (and solved) a similar issue.


Best,
Andrea

[1] 
https://lucene.apache.org/solr/6_1_0/solr-core/org/apache/solr/handler/component/SearchComponent.html

[2] https://wiki.apache.org/solr/SolrPlugins#SolrCoreAware

On 30/06/16 16:00, Mark Robinson wrote:

Hi,

I have a java OBJECT which I need to load once.
I have written a java custom component, which I have added in
"last-components" in solrconfig.xml, from which I want to access the above
mentioned OBJECT when each search request comes in.

Is there a way I can load a java object on server/ instance startup?
OR
Load it when the first call comes to SOLR?

For the time being I created that Java object inside the custom component
itself; but it is loaded each time a search request comes in.

Could some one pls give some pointers on how my above requirement can be
achieved in SOLR?

Thanks!
Mark

Re: Stemming and Managed Schema

[Andrea Gazzarini]

Sure, this is the API reference [1] where you can see, you can add types 
and fields


Andrea

[1] https://cwiki.apache.org/confluence/display/solr/Schema+API


On 03/06/16 17:07, Jamal, Sarfaraz wrote:

Hi Guys,

I found the following article:
http://thinknook.com/keyword-stemming-and-lemmatisation-with-apache-solr-2013-08-02/

And I want to do stemming on one of our fields.

However, I am using a Managed Schema and I am unsure how to add these two 
blocks to it -

I know there is an API for managed schemas, would that support these additions?

Thanks!

Sas

Re: Information for solr-user@lucene.apache.org

[Andrea Gazzarini]

Hi Carl,
This address is valid, any subscribed user received a copy of your email.

solr-user@lucene.apache.org

Andrea
On 21 May 2016 15:10, "Carl Roberts"  wrote:

> And, these response are just weird.  Do they mean this user list is
> obsolete?  is solr no longer supported via a user list where we can ask
> questions?
>
> On 5/21/16 9:08 AM, solr-user-h...@lucene.apache.org wrote:
>
>> Hi! This is the ezmlm program. I'm managing the
>> solr-user@lucene.apache.org mailing list.
>>
>> I'm working for my owner, who can be reached
>> at solr-user-ow...@lucene.apache.org.
>>
>> No information has been provided for this list.
>>
>> --- Administrative commands for the solr-user list ---
>>
>> I can handle administrative requests automatically. Please
>> do not send them to the list address! Instead, send
>> your message to the correct command address:
>>
>> To subscribe to the list, send a message to:
>> 
>>
>> To remove your address from the list, send a message to:
>> 
>>
>> Send mail to the following for info and FAQ for this list:
>> 
>> 
>>
>> Similar addresses exist for the digest list:
>> 
>> 
>>
>> To get messages 123 through 145 (a maximum of 100 per request), mail:
>> 
>>
>> To get an index with subject and author for messages 123-456 , mail:
>> 
>>
>> They are always returned as sets of 100, max 2000 per request,
>> so you'll actually get 100-499.
>>
>> To receive all messages with the same subject as message 12345,
>> send a short message to:
>> 
>>
>> The messages should contain one line or word of text to avoid being
>> treated as sp@m, but I will ignore their content.
>> Only the ADDRESS you send to is important.
>>
>> You can start a subscription for an alternate address,
>> for example "john@host.domain", just add a hyphen and your
>> address (with '=' instead of '@') after the command word:
>> 

Re: Re-ranking query: issue with sort criteria and how to disable it

[Andrea Gazzarini]

Hi Joel,

just created [1] a new issue for that.

Many thanks again

Andrea

[1] https://issues.apache.org/jira/browse/SOLR-9095


On 06/05/16 20:21, Joel Bernstein wrote:

Maybe one ticket would work. Something like: "ReRanker should gracefully
handle sorts without score". Then you can describe the two scenarios. It
might be that these problems are tackled outside of the
ReRankQParserPlugin. Possibly the QueryComponent could add some logic that
would tack on the secondary score sort or remove the reRanker.

Joel Bernstein
http://joelsolr.blogspot.com/

On Fri, May 6, 2016 at 1:39 PM, Andrea Gazzarini <gxs...@gmail.com> wrote:


Hi Joel,
many thanks for the response and sorry for this late reply.

About the first question, I can open a JIRA for that. Instead, for
disabling the component I think it would be useful to add

- an automatic behaviour: if the sort criteria excludes the score the
re-ranking could be automatically excluded
- a parameter / flag (something like *rr=true*) which enables / disables
the reranking. In this way such behaviour could be also driven on the
client side

What do you think? I guess this should be another JIRA

Best,
Andrea


On Fri, May 6, 2016 at 3:32 PM, Joel Bernstein <joels...@gmail.com> wrote:


I would consider the NPE when sort by score is not included a bug. There

is

the work around, that you mentioned, which is to have a compound sort

which

includes score.

The second issue though of disabling the ReRanker when someone doesn't
include a sort by score, would be a new feature of the ReRanker. I think
it's a good idea but it's not implemented yet.

I'm not sure if anyone has any ideas about conditionally adding the
ReRanker using configurations?

Joel Bernstein
http://joelsolr.blogspot.com/

On Fri, May 6, 2016 at 4:10 AM, Andrea Gazzarini <gxs...@gmail.com>

wrote:

Hi guys,
I have a Solr 4.10.4 instance with a RequestHandler that has a

re-ranking

query configured like this:


 dismax
 ...
 {!boost b=someFunction() v=$q}
 {!rerank reRankQuery=$rqq reRankDocs=60
reRankWeight=1.2}
 score desc


Everything is working until the client sends a sort params that doesn't
include the score field. So if for example the request contains

"sort=price

asc" then a NullPointerException is thrown:
/
//09:46:08,548 ERROR [org.apache.solr.core.SolrCore]
java.lang.NullPointerException//
//[INFO] [talledLocalContainer] at


org.apache.lucene.search.TopFieldCollector$OneComparatorScoringMaxScoreCollector.collect(TopFieldCollector.java:291)//

//[INFO] [talledLocalContainer] at


org.apache.solr.search.ReRankQParserPlugin$ReRankCollector.collect(ReRankQParserPlugin.java:263)//

//[INFO] [talledLocalContainer] at


org.apache.solr.search.SolrIndexSearcher.sortDocSet(SolrIndexSearcher.java:1999)//

//[INFO] [talledLocalContainer] at


org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1423)//

//[INFO] [talledLocalContainer] at


org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:514)//

//[INFO] [talledLocalContainer] at


org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:484)//

//[INFO] [talledLocalContainer] at


org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)//

//[INFO] [talledLocalContainer] at


org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)

/The only way to avoid this exception is to _explicitly_ add th/e

"score

desc" /value to the incoming field (i.e. sort=price asc, score desc).

In

this way I get no exception. I said "explicitly" because adding an
"appends" section in my handler


 score desc


Even I don't know if that could solve my problem, in practice it is
completely ignoring (i.e. I'm still getting the NPE above).
However, when I explicitly add "sort=price asc, score desc", as
consequence of the re-ranking, the top 60 results, although I said to

Solr

"order by price", are still shuffled and that's not what I want.

On top of that I have two questions:

  * Any idea about the exception above?
  * How can I disable the re-ranking query in case the order is not by
score?

About the second question, I'm thinking to the following solutions, but
I'm not sure if there's a better way to do that.

1. Create another request handler, which is basically a clone of the
handler above but without the re-ranking stuff
2. Use local params for the reRankDocs...


 dismax
 ...
 {!boost b=someFunction() v=$q}
 {!rerank reRankQuery=$rqq reRankDocs=*$rrd*
reRankWeight=1.2}
*60*
 score desc


...and have (in case of sorting by something different from the score)

the

client sending an additional params "rdd=0". This is working but I

still

need to explicitly declare "sort=price asc, score desc"

Any thoughts?

Best,
Andrea

Re: Re-ranking query: issue with sort criteria and how to disable it

[Andrea Gazzarini]

Hi Joel,
many thanks for the response and sorry for this late reply.

About the first question, I can open a JIRA for that. Instead, for
disabling the component I think it would be useful to add

- an automatic behaviour: if the sort criteria excludes the score the
re-ranking could be automatically excluded
- a parameter / flag (something like *rr=true*) which enables / disables
the reranking. In this way such behaviour could be also driven on the
client side

What do you think? I guess this should be another JIRA

Best,
Andrea


On Fri, May 6, 2016 at 3:32 PM, Joel Bernstein <joels...@gmail.com> wrote:

> I would consider the NPE when sort by score is not included a bug. There is
> the work around, that you mentioned, which is to have a compound sort which
> includes score.
>
> The second issue though of disabling the ReRanker when someone doesn't
> include a sort by score, would be a new feature of the ReRanker. I think
> it's a good idea but it's not implemented yet.
>
> I'm not sure if anyone has any ideas about conditionally adding the
> ReRanker using configurations?
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Fri, May 6, 2016 at 4:10 AM, Andrea Gazzarini <gxs...@gmail.com> wrote:
>
> > Hi guys,
> > I have a Solr 4.10.4 instance with a RequestHandler that has a re-ranking
> > query configured like this:
> >
> > 
> > dismax
> > ...
> > {!boost b=someFunction() v=$q}
> > {!rerank reRankQuery=$rqq reRankDocs=60
> > reRankWeight=1.2}
> > score desc
> > 
> >
> > Everything is working until the client sends a sort params that doesn't
> > include the score field. So if for example the request contains
> "sort=price
> > asc" then a NullPointerException is thrown:
> > /
> > //09:46:08,548 ERROR [org.apache.solr.core.SolrCore]
> > java.lang.NullPointerException//
> > //[INFO] [talledLocalContainer] at
> >
> org.apache.lucene.search.TopFieldCollector$OneComparatorScoringMaxScoreCollector.collect(TopFieldCollector.java:291)//
> > //[INFO] [talledLocalContainer] at
> >
> org.apache.solr.search.ReRankQParserPlugin$ReRankCollector.collect(ReRankQParserPlugin.java:263)//
> > //[INFO] [talledLocalContainer] at
> >
> org.apache.solr.search.SolrIndexSearcher.sortDocSet(SolrIndexSearcher.java:1999)//
> > //[INFO] [talledLocalContainer] at
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1423)//
> > //[INFO] [talledLocalContainer] at
> >
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:514)//
> > //[INFO] [talledLocalContainer] at
> >
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:484)//
> > //[INFO] [talledLocalContainer] at
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)//
> > //[INFO] [talledLocalContainer] at
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> >
> > /The only way to avoid this exception is to _explicitly_ add th/e "score
> > desc" /value to the incoming field (i.e. sort=price asc, score desc). In
> > this way I get no exception. I said "explicitly" because adding an
> > "appends" section in my handler
> >
> > 
> > score desc
> > 
> >
> > Even I don't know if that could solve my problem, in practice it is
> > completely ignoring (i.e. I'm still getting the NPE above).
> > However, when I explicitly add "sort=price asc, score desc", as
> > consequence of the re-ranking, the top 60 results, although I said to
> Solr
> > "order by price", are still shuffled and that's not what I want.
> >
> > On top of that I have two questions:
> >
> >  * Any idea about the exception above?
> >  * How can I disable the re-ranking query in case the order is not by
> >score?
> >
> > About the second question, I'm thinking to the following solutions, but
> > I'm not sure if there's a better way to do that.
> >
> > 1. Create another request handler, which is basically a clone of the
> > handler above but without the re-ranking stuff
> > 2. Use local params for the reRankDocs...
> >
> > 
> > dismax
> > ...
> > {!boost b=someFunction() v=$q}
> > {!rerank reRankQuery=$rqq reRankDocs=*$rrd*
> > reRankWeight=1.2}
> > *60*
> > score desc
> > 
> >
> > ...and have (in case of sorting by something different from the score)
> the
> > client sending an additional params "rdd=0". This is working but I still
> > need to explicitly declare "sort=price asc, score desc"
> >
> > Any thoughts?
> >
> > Best,
> > Andrea
> >
> >
>

Re-ranking query: issue with sort criteria and how to disable it

[Andrea Gazzarini]

Hi guys,
I have a Solr 4.10.4 instance with a RequestHandler that has a 
re-ranking query configured like this:



dismax
...
{!boost b=someFunction() v=$q}
{!rerank reRankQuery=$rqq reRankDocs=60 
reRankWeight=1.2}

score desc


Everything is working until the client sends a sort params that doesn't 
include the score field. So if for example the request contains 
"sort=price asc" then a NullPointerException is thrown:

/
//09:46:08,548 ERROR [org.apache.solr.core.SolrCore] 
java.lang.NullPointerException//
//[INFO] [talledLocalContainer] at 
org.apache.lucene.search.TopFieldCollector$OneComparatorScoringMaxScoreCollector.collect(TopFieldCollector.java:291)//
//[INFO] [talledLocalContainer] at 
org.apache.solr.search.ReRankQParserPlugin$ReRankCollector.collect(ReRankQParserPlugin.java:263)//
//[INFO] [talledLocalContainer] at 
org.apache.solr.search.SolrIndexSearcher.sortDocSet(SolrIndexSearcher.java:1999)//
//[INFO] [talledLocalContainer] at 
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1423)//
//[INFO] [talledLocalContainer] at 
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:514)//
//[INFO] [talledLocalContainer] at 
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:484)//
//[INFO] [talledLocalContainer] at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)//
//[INFO] [talledLocalContainer] at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)


/The only way to avoid this exception is to _explicitly_ add th/e "score 
desc" /value to the incoming field (i.e. sort=price asc, score desc). In 
this way I get no exception. I said "explicitly" because adding an 
"appends" section in my handler



score desc


Even I don't know if that could solve my problem, in practice it is 
completely ignoring (i.e. I'm still getting the NPE above).
However, when I explicitly add "sort=price asc, score desc", as 
consequence of the re-ranking, the top 60 results, although I said to 
Solr "order by price", are still shuffled and that's not what I want.


On top of that I have two questions:

 * Any idea about the exception above?
 * How can I disable the re-ranking query in case the order is not by
   score?

About the second question, I'm thinking to the following solutions, but 
I'm not sure if there's a better way to do that.


1. Create another request handler, which is basically a clone of the 
handler above but without the re-ranking stuff

2. Use local params for the reRankDocs...


dismax
...
{!boost b=someFunction() v=$q}
{!rerank reRankQuery=$rqq reRankDocs=*$rrd* 
reRankWeight=1.2}

*60*
score desc


...and have (in case of sorting by something different from the score) 
the client sending an additional params "rdd=0". This is working but I 
still need to explicitly declare "sort=price asc, score desc"


Any thoughts?

Best,
Andrea

Re: copy field source not working in solr schema.xml

[Andrea Gazzarini]

Although what you pasted isn't the complete schema I guess you miss a

 wrote:

> Error :
>  org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Could not load conf for core demo7: copyField dest :'i_member_id' is not an
> explicit field and doesn't match a dynamicField.. Schema file is
> /opt/solr/example/solr/demo7/conf/schema.xml
>
> My schema.xml :
>
> required="true" multiValued="false" />
>  stored="true" required="false" />
>
> 
> 
>
> Please help me.
> Thanks in advance.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/copy-field-source-not-working-in-solr-schema-xml-tp4273355.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: failed to connect solr server through my website

[Andrea Gazzarini]

Hi Adel,
As far as I know, the mailing list doesn't allow attachments. Please paste
the relevant part of your log

Andrea
On 28 Mar 2016 11:18, "Adel Mohamed Khalifa" 
wrote:

> Hello All,
>
>
>
> I failed to connect solr server through my website, I attached my solr log
> if anyone can help me please?
>
>
>
>
> *Regards,**Adel Khalifa | *Developer *| **Saudisoft-Egypt | ** Tel: *+2
> 023 303 2037 - *ext *112 | *M *+2 01149247744 | *Fax *+2 023 303 2036 | 
> *Follow
> us on **[image: Description: cid:image001.gif@01D11A1D.D81CE0C0]*
> *
>  |
> **[image: Description: cid:image002.gif@01D11A1D.D81CE0C0]*
> * | **[image: Description:
> cid:image003.gif@01D11A1D.D81CE0C0]*
> 
> *[image: Description: cid:image004.jpg@01D11A1D.D81CE0C0]*
>
>
>
>
>

RE: Making my own search website

[Andrea Gazzarini]

Hi Adel,
Is it possible that something (i.e. inbound or outbound firewall) forbids
the access to Solr from that machine?

Andrea
On 27 Mar 2016 14:46, "Adel Mohamed Khalifa" <a.moha...@saudisoft.com>
wrote:

> Thanks Andrea,
>
>
>
> I want to say if my string is wrong it would not connect when I connect
> from windows it’s the same code, I did not change in it.
>
>
>
> SOLR_SERVER_URL=http://172.16.0.72:8983/solr/SearchCore
>
>
>
> Regards,
> Adel Khalifa
>
>
>
>
>
> From: Andrea Gazzarini [mailto:gxs...@gmail.com]
> Sent: Sunday, March 27, 2016 2:22 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Making my own search website
>
>
>
> Hi Adel,
> Absolutely not sure what's happening on (Solr) server side, the first
> thing that comes on my mind is: if you're correctly accessing the solr
> admin console that means the string you're getting in that resource bundle
> is wrong. I'd print out that value in order to make sure about the
> correctness of that address or I would hard code the correct string itself.
>
> Best,
> Andrea
>
> P.S.: don't create a SolrServer instance for each http request. Better to
> define a member instance, initialize it in the init() method and close in
> the destroy()...that adheres much better to the servlet threading model.
>
> Hello All,
>
>
>
> First, I want to apology for my complexity needs before and that I cannot
> asking obviously.
>
>
>
> So Now, I create my Solr server and indexing I can log on it directly
> throw ( http://ip:port/solr/# <http://ip:port/solr/> ) from ubuntu server
> or my windows laptop.
>
> I build my website on netbeans using J2EE on my windows laptop and add a
> servlet to access the solr server for searching and when I run it it works
> good.
>
>
>
> But the problem occurred when I try to run on it on the Ubuntu server it
> stopped at : -- SolrServer server = new HttpSolrServer(ip:port/solr/core);
>
> And it does not give me any exception.
>
>
>
> I attached the ajax code that I use to call the servlet and also attached
> the servlet code.
>
>
>
> Regards,
> Adel Khalifa | Developer | Saudisoft-Egypt |  Tel: +2 023 303 2037 - ext
> 112 | M +2 01149247744 <tel:%2B2%2001149247744>  | Fax +2 023 303 2036 |
> Follow us on  <
> https://www.facebook.com/pages/Saudisoft-Co-Ltd/289968997768973?ref_type=bookmark>
> Description: cid:image001.gif@01D11A1D.D81CE0C0 |  <
> https://twitter.com/Saudisoft> Description:
> cid:image002.gif@01D11A1D.D81CE0C0 |  <
> https://www.linkedin.com/company/77017?trk=vsrp_companies_res_name=VSRPsearchId%3A1489659901402995947155%2CVSRPtargetId%3A77017%2CVSRPcmpt%3Aprimary>
> Description: cid:image003.gif@01D11A1D.D81CE0C0
> Description: cid:image004.jpg@01D11A1D.D81CE0C0
>
>
>
>
>
>

Re: Making my own search website

[Andrea Gazzarini]

Hi Adel,
Absolutely not sure what's happening on (Solr) server side, the first thing
that comes on my mind is: if you're correctly accessing the solr admin
console that means the string you're getting in that resource bundle is
wrong. I'd print out that value in order to make sure about the correctness
of that address or I would hard code the correct string itself.

Best,
Andrea

P.S.: don't create a SolrServer instance for each http request. Better to
define a member instance, initialize it in the init() method and close in
the destroy()...that adheres much better to the servlet threading model.

Hello All,



First, I want to apology for my complexity needs before and that I cannot
asking obviously.



So Now, I create my Solr server and indexing I can log on it directly throw
( http://ip:port/solr/#) from ubuntu server or my windows laptop.

I build my website on netbeans using J2EE on my windows laptop and add a
servlet to access the solr server for searching and when I run it it works
good.



But the problem occurred when I try to run on it on the Ubuntu server it
stopped at : -- SolrServer server = new HttpSolrServer(ip:port/solr/core);

And it does not give me any exception.



I attached the ajax code that I use to call the servlet and also attached
the servlet code.




*Regards,**Adel Khalifa | *Developer *| **Saudisoft-Egypt | ** Tel: *+2 023
303 2037 - *ext *112 | *M *+2 01149247744 | *Fax *+2 023 303 2036 | *Follow
us on **[image: Description: cid:image001.gif@01D11A1D.D81CE0C0]*
*
|
**[image: Description: cid:image002.gif@01D11A1D.D81CE0C0]*
* | **[image: Description:
cid:image003.gif@01D11A1D.D81CE0C0]*

*[image: Description: cid:image004.jpg@01D11A1D.D81CE0C0]*

Re: Solr event listerns

[Andrea Gazzarini]

Hi Anil,
If with "Application" you mean an external / remote Application then
there's no such pre-built mechanism to do that.

The event is not notifies "in" your application or "to" your application.
Solr knows nothing about it.

Specifically, event listeners allows you to do something when a given event
happens...what exactly is "something" it's up to you; for instance you
could (asynch) send a http message to your API, send a message to a queue,
send a GCM message to some mobile device, or in general: everything :) but
again, this is up to you.

Andrea
On 26 Mar 2016 15:46, "Anil" <anilk...@gmail.com> wrote:

> HI Alex,
>
> i am still no clear how an event is notified in my application if it
> listener is configured in SolrConfig.xml (centralized solr server). can you
> please clarify?
>
> Sorry for dumb question.
>
> Thanks,
> Anil
>
> On 26 March 2016 at 19:48, Andrea Gazzarini <gxs...@gmail.com> wrote:
>
> > Hi Anil,
> > see here [1] where I did some data pre-loading at solr startup. The post
> > has been written in 2014 so maybe something could have a different name
> > (class, methods)
> >
> > Best,
> > Andrea
> >
> > [1]
> >
> >
> http://andreagazzarini.blogspot.it/2014/11/preloading-data-at-solr-startup.html
> > On 26 Mar 2016 15:14, "Anil" <anilk...@gmail.com> wrote:
> >
> > > Thanks Alex. Let me digg more in that area.
> > >
> > > On 26 March 2016 at 19:40, Andrea Gazzarini <gxs...@gmail.com> wrote:
> > >
> > > > Event listeners are custom classes so you could do
> > "anything"however
> > > I
> > > > guess the event firing is synchronous so the listener logic should be
> > > > no-blocking and as fast as possible.
> > > >
> > > > But this is my guess, I hadn't look at the code. Instead, if the
> > listener
> > > > invocation is asynch then forget my comment, the answer is yes.
> > > >
> > > > Best,
> > > > Andrea
> > > > On 26 Mar 2016 15:03, "Anil" <anilk...@gmail.com> wrote:
> > > >
> > > > > Can solr notify other systems on commit of document ?
> > > > >
> > > > > On 26 March 2016 at 13:59, Alexandre Rafalovitch <
> arafa...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Well, the updated documents are not visible until the commit. So,
> > > > > > perhaps you just need the commit listeners then. Those could be
> any
> > > > > > external scripts I believe.
> > > > > >
> > > > > > Regards,
> > > > > >Alex.
> > > > > > 
> > > > > > Newsletter and resources for Solr beginners and intermediates:
> > > > > > http://www.solr-start.com/
> > > > > >
> > > > > >
> > > > > > On 26 March 2016 at 17:14, Anil <anilk...@gmail.com> wrote:
> > > > > > > Thanks Alex.
> > > > > > >
> > > > > > > i have gone through it. i am building a use case where cache
> has
> > to
> > > > > > updated
> > > > > > > in my application when there is update or addition of solr
> > document
> > > > > > happens.
> > > > > > >
> > > > > > > i am not sure event listeners  of Update handler would fit the
> > use
> > > > > case.
> > > > > > >
> > > > > > > What do you say ? please share your ideas.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Anil
> > > > > > >
> > > > > > > On 26 March 2016 at 10:26, Alexandre Rafalovitch <
> > > arafa...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > >> Have you looked at UpdateRequestProcessors?
> > > > > > >>
> > > > > > >> Plus check solrconfig.xml for listeners on commit.
> > > > > > >>
> > > > > > >> Regards,
> > > > > > >> Alex
> > > > > > >> On 26 Mar 2016 3:54 pm, "Anil" <anilk...@gmail.com> wrote:
> > > > > > >>
> > > > > > >> > HI,
> > > > > > >> >
> > > > > > >> > Does solr support event (document create, update, delete)
> > > > listeners
> > > > > ?
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Anil
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Solr event listerns

[Andrea Gazzarini]

Hi Anil,
see here [1] where I did some data pre-loading at solr startup. The post
has been written in 2014 so maybe something could have a different name
(class, methods)

Best,
Andrea

[1]
http://andreagazzarini.blogspot.it/2014/11/preloading-data-at-solr-startup.html
On 26 Mar 2016 15:14, "Anil" <anilk...@gmail.com> wrote:

> Thanks Alex. Let me digg more in that area.
>
> On 26 March 2016 at 19:40, Andrea Gazzarini <gxs...@gmail.com> wrote:
>
> > Event listeners are custom classes so you could do "anything"however
> I
> > guess the event firing is synchronous so the listener logic should be
> > no-blocking and as fast as possible.
> >
> > But this is my guess, I hadn't look at the code. Instead, if the listener
> > invocation is asynch then forget my comment, the answer is yes.
> >
> > Best,
> > Andrea
> > On 26 Mar 2016 15:03, "Anil" <anilk...@gmail.com> wrote:
> >
> > > Can solr notify other systems on commit of document ?
> > >
> > > On 26 March 2016 at 13:59, Alexandre Rafalovitch <arafa...@gmail.com>
> > > wrote:
> > >
> > > > Well, the updated documents are not visible until the commit. So,
> > > > perhaps you just need the commit listeners then. Those could be any
> > > > external scripts I believe.
> > > >
> > > > Regards,
> > > >Alex.
> > > > 
> > > > Newsletter and resources for Solr beginners and intermediates:
> > > > http://www.solr-start.com/
> > > >
> > > >
> > > > On 26 March 2016 at 17:14, Anil <anilk...@gmail.com> wrote:
> > > > > Thanks Alex.
> > > > >
> > > > > i have gone through it. i am building a use case where cache has to
> > > > updated
> > > > > in my application when there is update or addition of solr document
> > > > happens.
> > > > >
> > > > > i am not sure event listeners  of Update handler would fit the use
> > > case.
> > > > >
> > > > > What do you say ? please share your ideas.
> > > > >
> > > > > Thanks,
> > > > > Anil
> > > > >
> > > > > On 26 March 2016 at 10:26, Alexandre Rafalovitch <
> arafa...@gmail.com
> > >
> > > > wrote:
> > > > >
> > > > >> Have you looked at UpdateRequestProcessors?
> > > > >>
> > > > >> Plus check solrconfig.xml for listeners on commit.
> > > > >>
> > > > >> Regards,
> > > > >> Alex
> > > > >> On 26 Mar 2016 3:54 pm, "Anil" <anilk...@gmail.com> wrote:
> > > > >>
> > > > >> > HI,
> > > > >> >
> > > > >> > Does solr support event (document create, update, delete)
> > listeners
> > > ?
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Anil
> > > > >> >
> > > > >>
> > > >
> > >
> >
>

Re: Solr event listerns

[Andrea Gazzarini]

Event listeners are custom classes so you could do "anything"however I
guess the event firing is synchronous so the listener logic should be
no-blocking and as fast as possible.

But this is my guess, I hadn't look at the code. Instead, if the listener
invocation is asynch then forget my comment, the answer is yes.

Best,
Andrea
On 26 Mar 2016 15:03, "Anil"  wrote:

> Can solr notify other systems on commit of document ?
>
> On 26 March 2016 at 13:59, Alexandre Rafalovitch 
> wrote:
>
> > Well, the updated documents are not visible until the commit. So,
> > perhaps you just need the commit listeners then. Those could be any
> > external scripts I believe.
> >
> > Regards,
> >Alex.
> > 
> > Newsletter and resources for Solr beginners and intermediates:
> > http://www.solr-start.com/
> >
> >
> > On 26 March 2016 at 17:14, Anil  wrote:
> > > Thanks Alex.
> > >
> > > i have gone through it. i am building a use case where cache has to
> > updated
> > > in my application when there is update or addition of solr document
> > happens.
> > >
> > > i am not sure event listeners  of Update handler would fit the use
> case.
> > >
> > > What do you say ? please share your ideas.
> > >
> > > Thanks,
> > > Anil
> > >
> > > On 26 March 2016 at 10:26, Alexandre Rafalovitch 
> > wrote:
> > >
> > >> Have you looked at UpdateRequestProcessors?
> > >>
> > >> Plus check solrconfig.xml for listeners on commit.
> > >>
> > >> Regards,
> > >> Alex
> > >> On 26 Mar 2016 3:54 pm, "Anil"  wrote:
> > >>
> > >> > HI,
> > >> >
> > >> > Does solr support event (document create, update, delete) listeners
> ?
> > >> >
> > >> > Thanks,
> > >> > Anil
> > >> >
> > >>
> >
>

Re: Replacing a document in Solr5

[Andrea Gazzarini]

That has nothing to do with your topic: addField adds a new value for a
given field in a SolrInputDocument, while setField replaces any existing
value (of a given field, regardless what is the existing value, I mean,
regardless if that field has zero, one or more values).

SolrInputDocument document = new SolrInputDocument();

document.set("id", 32872382); // the id field has now one value:  32872382

document.add("author", "B. Meyer") // the author field has one value. In
this case, being the first value, add() and set() behave in the the same way

document.add("author", "A. Yersu") // Now the author field has two values
document.set("author", "I.UUhash") // That will replace the existing two
values with this value.


solrClient.add(document); // here, You are sending  document with 1 id and
1 author



Those are methods of SolrInputDocument; when you call them, you're changing
the state of a local transfer object (the SolrInputDocument instance).
Before sending that to Solr using solrClient.add(SolrInputDocument) you can
do whatever you want with that instance (i.e. removing, adding, setting
values). The "document" representation that Solr will see is the state of
the instance that you pass to solrClient.add(...)

Best,
Andrea


2015-12-19 8:48 GMT+01:00 Debraj Manna <subharaj.ma...@gmail.com>:

> Ok. Then what is the difference between addField
> <
> http://github.com/apache/lucene-solr/tree/lucene_solr_5_3_1/solr/solrj/src/java/org/apache/solr/common/SolrInputDocument.java#L150
> >
> & setField
> <
> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/SolrInputDocument.html#setField-java.lang.String-java.lang.Object-float-
> >
> ?
>
> On Sat, Dec 19, 2015 at 1:04 PM, Andrea Gazzarini <a.gazzar...@gmail.com>
> wrote:
>
> > As far as I know, this is how Solr works (e.g. it replaces the whole
> > document): how do you replace only a part of a document?
> >
> > Just send a SolrInputDocument with an existing (i.e. already indexed) id
> > and the document (on Solr) will be replaced.
> >
> > Andrea
> >
> > 2015-12-19 8:16 GMT+01:00 Debraj Manna <subharaj.ma...@gmail.com>:
> >
> > > Can someone let me know how can I replace a document on each update in
> > Solr
> > > 5.2.1 using SolrJ? I don;t want to update parts of the document. On
> doing
> > > update it should replace the entire document.
> > >
> >
>

Re: Replacing a document in Solr5

[Andrea Gazzarini]

As far as I know, this is how Solr works (e.g. it replaces the whole
document): how do you replace only a part of a document?

Just send a SolrInputDocument with an existing (i.e. already indexed) id
and the document (on Solr) will be replaced.

Andrea

2015-12-19 8:16 GMT+01:00 Debraj Manna :

> Can someone let me know how can I replace a document on each update in Solr
> 5.2.1 using SolrJ? I don;t want to update parts of the document. On doing
> update it should replace the entire document.
>