restoring an index from restore command

[GOURAUD Emmanuel]

Hi there, 

I'm currently trying to restore a core from a snapshot, and it does not work as 
excepted (in my case). 

My context: 
I'm using solr 5.3.0 
i used named snapshot and specific location to ensure collection integrity a a 
time in a single directory 

below, i will describe backup/restore for a specific core 

backup command: 
http://10.0.2.15:8080/solr/test_shard2_replica1/replication?wt=json=backup=%2Fhome%2Fsolr%2Ftest%2F20151128-073603009560=test_shard2_replica1
 

it has created the snapshot with all data: 
[vagrant@centos7 solr-5.3.0]$ ls -lh 
/home/solr/test/20151128-073603009560/snapshot.test_shard1_replica2/ 
total 32K 
-rw-r--r-- 1 solr solr 346 Nov 28 07:36 _5.fdt 
-rw-r--r-- 1 solr solr 84 Nov 28 07:36 _5.fdx 
-rw-r--r-- 1 solr solr 431 Nov 28 07:36 _5.fnm 
-rw-r--r-- 1 solr solr 110 Nov 28 07:36 _5_Lucene50_0.doc 
-rw-r--r-- 1 solr solr 697 Nov 28 07:36 _5_Lucene50_0.tim 
-rw-r--r-- 1 solr solr 171 Nov 28 07:36 _5_Lucene50_0.tip 
-rw-r--r-- 1 solr solr 432 Nov 28 07:36 _5.si 
-rw-r--r-- 1 solr solr 165 Nov 28 07:36 segments_7 

i flush all data from the same core and restart the solr5 instance and then try 
to restore from the previous snapshot: 
[vagrant@centos7 solr-5.3.0]$ curl 
"http://10.20.1.16:8082/solr/test_shard1_replica2/replication?wt=json=restore=%2Fhome%2Fsolr%2Ftest%2F20151128-073603009560=test_shard1_replica2;
 
{"responseHeader":{"status":0,"QTime":53},"status":"OK"} 

all seems to be OK, let's verify 

[vagrant@centos7 solr-5.3.0]$ curl 
"http://10.20.1.16:8082/solr/test_shard1_replica2/replication?wt=json=restorestatus"{"responseHeader":{"status":0,"QTime":2},"restorestatus":{"snapshotName":"snapshot.test_shard1_replica2","status":"success"}}
 

great!! what about logs: 

[...skipped data...] 
2015-11-28 07:42:39.128 INFO 
(restoreExecutor-8-thread-1-processing-x:test_shard1_replica2 r:core_node4 
test_shard1_replica2 n:10.20.1.16:8082_solr snapshot.test_shard1_replic 
a2 s:shard1 c:test //home//solr//test//20151128-073603009560) [c:test s:shard1 
r:core_node4 x:test_shard1_replica2] o.a.s.h.RestoreCore Copying file _5.fdt to 
restore directory 

2015-11-28 07:42:39.129 INFO 
(restoreExecutor-8-thread-1-processing-x:test_shard1_replica2 r:core_node4 
test_shard1_replica2 n:10.20.1.16:8082_solr snapshot.test_shard1_replic 
a2 s:shard1 c:test //home//solr//test//20151128-073603009560) [c:test s:shard1 
r:core_node4 x:test_shard1_replica2] o.a.s.h.RestoreCore Copying file 
_5_Lucene50_0.tip to restor 
e directory 
2015-11-28 07:42:39.129 INFO 
(restoreExecutor-8-thread-1-processing-x:test_shard1_replica2 r:core_node4 
test_shard1_replica2 n:10.20.1.16:8082_solr snapshot.test_shard1_replic 
a2 s:shard1 c:test //home//solr//test//20151128-073603009560) [c:test s:shard1 
r:core_node4 x:test_shard1_replica2] o.a.s.h.RestoreCore Copying file _5.si to 
restore directory 
2015-11-28 07:42:39.130 INFO 
(restoreExecutor-8-thread-1-processing-x:test_shard1_replica2 r:core_node4 
test_shard1_replica2 n:10.20.1.16:8082_solr snapshot.test_shard1_replic 
a2 s:shard1 c:test //home//solr//test//20151128-073603009560) [c:test s:shard1 
r:core_node4 x:test_shard1_replica2] o.a.s.h.RestoreCore Copying file 
_5_Lucene50_0.doc to restor 
e directory 
2015-11-28 07:42:39.132 INFO 
(restoreExecutor-8-thread-1-processing-x:test_shard1_replica2 r:core_node4 
test_shard1_replica2 n:10.20.1.16:8082_solr snapshot.test_shard1_replic 
a2 s:shard1 c:test //home//solr//test//20151128-073603009560) [c:test s:shard1 
r:core_node4 x:test_shard1_replica2] o.a.s.h.RestoreCore Copying file 
_5_Lucene50_0.tim to restor 
e directory 
2015-11-28 07:42:39.134 INFO 
(restoreExecutor-8-thread-1-processing-x:test_shard1_replica2 r:core_node4 
test_shard1_replica2 n:10.20.1.16:8082_solr snapshot.test_shard1_replic 
a2 s:shard1 c:test //home//solr//test//20151128-073603009560) [c:test s:shard1 
r:core_node4 x:test_shard1_replica2] o.a.s.h.RestoreCore Copying file _5.fdx to 
restore directory 
[...skipped data...] 

Wow , i love solr ;) 

now, let's have a look at data directory: 

[vagrant@centos7 solr-5.3.0]$ ls -lh 
/usr/local/third/solr-5.3.0/server/solr/test_shard1_replica2/data/* 
-rw-r--r-- 1 solr solr 92 Nov 28 07:42 
/usr/local/third/solr-5.3.0/server/solr/test_shard1_replica2/data/index.properties
 

/usr/local/third/solr-5.3.0/server/solr/test_shard1_replica2/data/restore.snapshot.test_shard1_replica2:
 
total 0 
-rw-r--r-- 1 solr solr 0 Nov 28 07:42 write.lock 

/usr/local/third/solr-5.3.0/server/solr/test_shard1_replica2/data/tlog: 
total 0 

Hum, no data, what did i miss? 

in order to workaround, i can restore from filesystem, but this behavior is 
making me mad :) 

thanks for your help ;) 

Re: Solrcloud with Zookeeper in production

[Brian Narsi]

I suggest the documentation Apache Solr Reference Guide for general
learning.

https://cwiki.apache.org/confluence/display/solr/Using+ZooKeeper+to+Manage+Configuration+Files
https://cwiki.apache.org/confluence/display/solr/Solr+Field+Types
https://wiki.apache.org/solr/SolrConfigXml

And if you run into a problem or have specific questions, this email list
is excellent.


On Sat, Nov 28, 2015 at 12:56 AM, Mugeesh Husain  wrote:

> thanks narsi,
>
> could you share some link for configuring these config files
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solrcloud-with-Zookeeper-in-production-tp4242162p4242543.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Single-sharded SolrCloud vs Lucene indexing speed

[Zisis Tachtsidis]

I'm conducting some indexing experiments in SolrCloud and I want to confirm
my conclusions and ask for suggestions on how to improve performance.

My setup includes a single-sharded collection with 1 additional replica in
SolrCloud 5.3.1. I'm using SolrJ and the indexing speed refers to the actual
SolrJ call that adds the document. I've run some indexing tests and it seems
that Lucene indexing is equal to or better than Solr's in all cases. In all
cases the same documents are sent to both Lucene and the same analysis
is performed on the documents. 

- 2 replicas, leader is a replica on a machine under heavy load => ~3x
slower than Lucene.
- 2 replicas, leader is a replica on a machine under light load => ~2x
slower than Lucene.
- 1 replica on a machine under light load => indexing speed similar to
Lucene.

Conclusions
(*) It seems that the slowest replica determines the indexing speed. 
(*) It gets even worse if the slowest replica is the leader. This is
justified if it's true that only after the leader finishes indexing it
forwards the request to the remaining replicas.

Regarding improvements
(*) I'm indexing pretty big documents 0.5MB

Re: Setting up Solr on multiple machines

[Salman Ansari]

I have started with one Zookeeper to test things and I have the following
questions

1) In my zoo.cfg I have defined
tickTime=4000
dataDir=C:\\Solr\\Zookeeper\\zookeeper-3.4.6\\data
clientPort=2183

the strange thing is that it picks up dataDir and clientPort but always
keeps tickTime = 3000. Any idea why?

2) It is clear from the documentation how to create an ensemble of
Zookeepers on 3 machines but what I am not sure about is how to
  a)  Setup actual Solr on 2 machines (is it just installing Solr on
each server and then passing the same zookeeper ensemble)?
  b) How to (using Solr Cloud) create 2 shards spread on 2 machines
with each machine having a replica of the other for high availability. So
server1 will have shard1 and replica2 and server2 will have shard2 and
replica1?

Comments and feedback are appreciated.

Regards,
Salman


On Fri, Nov 27, 2015 at 5:52 AM, Zheng Lin Edwin Yeo 
wrote:

> Yes, the ZooKeeper is Windows compatible.
>
> You can follow the guide, just need to replace the Linux commands with the
> Windows commands and paths
>
> Regards,
> Edwin
>
>
> On 26 November 2015 at 20:56, Alessandro Benedetti 
> wrote:
>
> > I think it should be straightforward following the Solr wiki :
> >
> >
> >
> https://cwiki.apache.org/confluence/display/solr/Setting+Up+an+External+ZooKeeper+Ensemble
> >
> > I never played much in details with zookeeper ( never tried on a windows
> > machine), but I assume it is windows compatible ( I can see binaries
> .cmd )
> >
> > Cheers
> >
> > On 26 November 2015 at 12:38, Salman Ansari 
> > wrote:
> >
> > > Hi,
> > >
> > > I have seen the guide of setting up Solr on one machine as well as
> > setting
> > > it up on multiple machines on Liunx. Is there a good guide of how to
> > setup
> > > Solr on multiple machines on Windows Server with Zookeeper ensemble? My
> > > structure is as follows
> > >
> > > 1) 3 machines will have Zookeeper to create an ensemble
> > > 2) 2 of these machines will have Solr installed (with each having a
> > replica
> > > of other to provide high availability)
> > >
> > > Any link/article that provides such a guide?
> > >
> > > Regards,
> > > Salman
> > >
> >
> >
> >
> > --
> > --
> >
> > Benedetti Alessandro
> > Visiting card : http://about.me/alessandro_benedetti
> >
> > "Tyger, tyger burning bright
> > In the forests of the night,
> > What immortal hand or eye
> > Could frame thy fearful symmetry?"
> >
> > William Blake - Songs of Experience -1794 England
> >
>