Slave replicating partially before write operation completes on master

[Geepalem]

Hi Team,

We have below critical issue in our project. Please suggest how to fix this
issue.

Whenever indexing happening on master, slave replicating documents
partially. Due to this, search results on website are showing wrongly.

During indexing on master, I have observed below versions on master and
slave


MASTER
-
Master (searching) version - 1
Master (Replicable) version -  2

SLAVE
---
Master (searching) version - 1
Master (Replicable) version -  2
Slave (Searching) version - 2


Once indexing gets completed, versions on both master and slave are same as
3 and also data replicated completely to slave.

Below is replication cong used in  master and slave 

  

  commit
  optimize
  commit  
  schema.xml
  00:00:10

2

  16

  




 

 
  http://localhost:8983/solr/global_web_index/replication
 
  00:00:20

  


Can any one please advice how to solve this issue as this is creating
escalations on project.

NOTE: As of now, I am disabling replication on slave before indexing to
avoid this issue.


Thanks  in advance!


Regards,
G. Naresh Kumar



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Slave-replicating-partially-before-write-operation-completes-on-master-tp4331453.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Enable https for Solr

[Zheng Lin Edwin Yeo]

Hi Shawn,

The same problem is happening in both SolrJ and SimplePostTool.
I got the same error for both.

Regards,
Edwin


On 22 April 2017 at 20:25, Shawn Heisey  wrote:

> On 4/22/2017 1:45 AM, Zheng Lin Edwin Yeo wrote:
> > Hi Steve,
> >
> > Yes, I have set that in solr.in.cmd.
>
> The *client* is where the certificate validation problem is happening.
> The client (you said SolrJ, but the stacktrace indicates it is
> SimplePostTool) is what needs the keystore-related system properties.
>
> Thanks,
> Shawn
>
>

Re: Modify solr score

[Erik Hatcher]

This may be suggesting a solution that is too experimental or using the wrong 
hammer for the job, but to me it sounds like you could use “payloads” for this 
type of ranking of terms relationship to a document.   

See SOLR-1485 for the recent work I’ve been doing (and aim to get committed 
soon).   You could index documents in this way:

   id, weighted_terms_dpf
   1, A|5.0 B|95.0
2,A|88.7 B|0.1

And then search for “A” and use the 88.7 value to factor into the score or 
sorting.  

Erik



> On Apr 21, 2017, at 12:35 PM, tstusr  wrote:
> 
> Since we report the score, we think there will be some relation between them.
> As far as we know scoring (and then ranking) are calculated based on tf-idf.
> 
> What we want to do is to make a qualitative ranking, it means, according to
> one topic we will tag documents as "very related", "fairly related" or "poor
> related". So, we select some documents completely unrelated to a topic.
> 
> On a very related document we found a ratio of ~2% of words that reports
> ~0.85 of score (what we think is related to ranking). On a test document we
> found a ratio of less than 0.01% and the score is heigher than the first
> one. What we expect is that documents not related (those ones with less
> ratio) report lower scores so we can then use them as minimum and create the
> scale.
> 
> We came with multiply (of affect in some way) the default rank solr provide
> us with the ratio of documents so unrelated documents will be penalized
> while those with higher ratio values will be overrated.
> 
> Greetings, and thanks for your help.
> 
> 
> 
> 
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Modify-solr-score-tp4331300p4331315.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: Data Changes Logging

[Walter Underwood]

Do a range search on that field with the desired date range. Request rows=0. 
Compare the numFound to the total docs.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)


> On Apr 22, 2017, at 8:40 AM, Rick Leir  wrote:
> 
> Preeti,
> Suppose you have a field named 'modifiedDate' and you update it when you 
> change the 'column' (i assume you mean some field in your document). Then you 
> need to decide how often to check for alerts. When you check for an alert you 
> would do a select with a bit of date math to see how many of your documents 
> have a recent modifiedDate. I am leaving out the details.
> Cheers -- Rick
> 
> On April 21, 2017 12:28:21 AM EDT, Preeti Bhat  
> wrote:
>> Hi All,
>> 
>> We got a peculiar requirement from client, not sure whether the SOLR
>> supports it or not. We would like to have alerts in place if we have
>> the changes on the particular column for more than specific threshold
>> on said day.
>> For example: Say, we have column say "Name" on which we have say around
>> 30% changes we would like an alert. I am not sure whether we have some
>> logging mechanism to get this done. Any ideas would be appreciated.
>> 
>> 
>> Thanks and Regards,
>> Preeti
>> 
>> 
>> 
>> NOTICE TO RECIPIENTS: This communication may contain confidential
>> and/or privileged information. If you are not the intended recipient
>> (or have received this communication in error) please notify the sender
>> and it-supp...@shoregrp.com immediately, and destroy this
>> communication. Any unauthorized copying, disclosure or distribution of
>> the material in this communication is strictly forbidden. Any views or
>> opinions presented in this email are solely those of the author and do
>> not necessarily represent those of the company. Finally, the recipient
>> should check this email and any attachments for the presence of
>> viruses. The company accepts no liability for any damage caused by any
>> virus transmitted by this email.
> 
> -- 
> Sorry for being brief. Alternate email is rickleir at yahoo dot com

Re: Data Changes Logging

[Rick Leir]

Preeti,
Suppose you have a field named 'modifiedDate' and you update it when you change 
the 'column' (i assume you mean some field in your document). Then you need to 
decide how often to check for alerts. When you check for an alert you would do 
a select with a bit of date math to see how many of your documents have a 
recent modifiedDate. I am leaving out the details.
Cheers -- Rick

On April 21, 2017 12:28:21 AM EDT, Preeti Bhat  wrote:
>Hi All,
>
>We got a peculiar requirement from client, not sure whether the SOLR
>supports it or not. We would like to have alerts in place if we have
>the changes on the particular column for more than specific threshold
>on said day.
>For example: Say, we have column say "Name" on which we have say around
>30% changes we would like an alert. I am not sure whether we have some
>logging mechanism to get this done. Any ideas would be appreciated.
>
>
>Thanks and Regards,
>Preeti
>
>
>
>NOTICE TO RECIPIENTS: This communication may contain confidential
>and/or privileged information. If you are not the intended recipient
>(or have received this communication in error) please notify the sender
>and it-supp...@shoregrp.com immediately, and destroy this
>communication. Any unauthorized copying, disclosure or distribution of
>the material in this communication is strictly forbidden. Any views or
>opinions presented in this email are solely those of the author and do
>not necessarily represent those of the company. Finally, the recipient
>should check this email and any attachments for the presence of
>viruses. The company accepts no liability for any damage caused by any
>virus transmitted by this email.

-- 
Sorry for being brief. Alternate email is rickleir at yahoo dot com 

Re: Enable https for Solr

[Shawn Heisey]

On 4/22/2017 1:45 AM, Zheng Lin Edwin Yeo wrote:
> Hi Steve,
>
> Yes, I have set that in solr.in.cmd.

The *client* is where the certificate validation problem is happening. 
The client (you said SolrJ, but the stacktrace indicates it is
SimplePostTool) is what needs the keystore-related system properties.

Thanks,
Shawn

Re: Enable https for Solr

[Zheng Lin Edwin Yeo]

Hi Steve,

Yes, I have set that in solr.in.cmd.

Regards,
Edwin

On 22 April 2017 at 01:51, Steve Rowe  wrote:

> Edwin,
>
> Did you set the required keystore/truststore/password system properties?
> See the example at  Enabling+SSL#EnablingSSL-IndexadocumentusingCloudSolrClient>
>
> --
> Steve
> www.lucidworks.com
>
> > On Apr 21, 2017, at 1:44 PM, Zheng Lin Edwin Yeo 
> wrote:
> >
> > Thank you Steve.
> >
> > I have managed to set up the SSL, and the query via https is working now.
> >
> > However, I am getting this error when I tried to do indexing using
> SolrJ. I
> > have already changed the URL to pass using https.
> >
> > What could be the reason that causes this?
> >
> > javax.net.ssl.SSLHandshakeException: sun.security.validator.
> > ValidatorException:
> > PKIX path building failed: sun.security.provider.certpath.
> > SunCertPathBuilderExce
> > ption: unable to find valid certification path to requested target
> >at sun.security.ssl.Alerts.getSSLException(Unknown Source)
> >at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
> >at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
> >at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
> >at sun.security.ssl.ClientHandshaker.serverCertificate(Unknown
> > Source)
> >at sun.security.ssl.ClientHandshaker.processMessage(Unknown
> Source)
> >at sun.security.ssl.Handshaker.processLoop(Unknown Source)
> >at sun.security.ssl.Handshaker.process_record(Unknown Source)
> >at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
> >at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown
> > Source
> > )
> >at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
> >at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
> >at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown
> > Source)
> >at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnec
> > tion.connect
> > (Unknown Source)
> >at sun.net.www.protocol.http.HttpURLConnection.
> getInputStream0(Unknown
> > S
> > ource)
> >at sun.net.www.protocol.http.HttpURLConnection.
> getInputStream(Unknown
> > So
> > urce)
> >at sun.net.www.protocol.https.HttpsURLConnectionImpl.
> > getInputStream(Unkn
> > own Source)
> >at org.apache.solr.util.SimplePostTool.doHttpGet(
> > SimplePostTool.java:1702)
> >at org.apache.solr.util.SimplePostTool.main(SimplePostTool.j
> > ava:256)
> > Caused by: sun.security.validator.ValidatorException: PKIX path building
> > failed:
> > sun.security.provider.certpath.SunCertPathBuilderException: unable to
> find
> > vali
> > d certification path to requested target
> >at sun.security.validator.PKIXValidator.doBuild(Unknown Source)
> >at sun.security.validator.PKIXValidator.engineValidate(Unknown
> > Source)
> >at sun.security.validator.Validator.validate(Unknown Source)
> >at sun.security.ssl.X509TrustManagerImpl.validate(Unknown Source)
> >at sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown
> > Source)
> >at sun.security.ssl.X509TrustManagerImpl.
> checkServerTrusted(Unknown
> > Sour
> > ce)
> >... 15 more
> > Caused by: sun.security.provider.certpath.SunCertPathBuilderException:
> > unable to
> > find valid certification path to requested target
> >at sun.security.provider.certpath.SunCertPathBuilder.
> build(Unknown
> > Sourc
> > e)
> >at sun.security.provider.certpath.SunCertPathBuilder.
> > engineBuild(Unknown
> > Source)
> >at java.security.cert.CertPathBuilder.build(Unknown Source)
> >... 21 more
> > javax.net.ssl.SSLHandshakeException: sun.security.validator.
> > ValidatorException:
> > PKIX path building failed: sun.security.provider.certpath.
> > SunCertPathBuilderExce
> > ption: unable to find valid certification path to requested target
> >at sun.security.ssl.Alerts.getSSLException(Unknown Source)
> >at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
> >at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
> >at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
> >at sun.security.ssl.ClientHandshaker.serverCertificate(Unknown
> > Source)
> >at sun.security.ssl.ClientHandshaker.processMessage(Unknown
> Source)
> >at sun.security.ssl.Handshaker.processLoop(Unknown Source)
> >at sun.security.ssl.Handshaker.process_record(Unknown Source)
> >at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
> >at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown
> > Source
> > )
> >at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
> >at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
> >at