Re: Bringing Old Collections Up Again

2016-05-03 Thread Shawn Heisey
On 5/2/2016 5:16 AM, Salman Ansari wrote:
> After several trials, it did start Solr on both machines but *non of the
> previous collections came back normally.* When I look at the admin page, it
> shows errors as follows
>
> *[Collection_name]_shard2_replica2:*
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Index locked for write for core '[Collection_name]_shard2_replica2'. Solr
> now longer supports forceful unlocking via 'unlockOnStartup'. Please verify
> locks manually!

I thought I had already sent this, but looks like it's still hanging
around.  Erick said much the same thing in his reply.

It sounds like Solr is being forcibly killed, not gracefully shut down,
when the machine is rebooted after installing updates.  When Solr is
forcibly killed, you probably need to find and delete a file named
"write.lock" in each of your index directories.  If that file exists
before Solr starts, Solr will not be able to lock the index and will not
load that index.  Restarting Solr after fixing the problem might be the
only solution.

You should not allow these systems to automatically reboot for updates,
or you should see if you can extend the amount of time that the reboot
will wait for processes to shut down normally before forcibly killing
them.  A graceful shutdown of a Solr process handling a lot of large
indexes can take quite a while.

A question for pondering:  Should the start script check for
index/write.lock files and fail to start (with a clear error message) if
any are found?  If we do that, we probably also need a "cleanlocks"
command on the solr script (and probably the init script) to delete
those files.

Thanks,
Shawn



Bringing Old Collections Up Again

2016-05-02 Thread Salman Ansari
Hi,

I am hosting Zookeeper ensemble and Solr servers on Microsoft cloud
(Azure). From time to time machines are forced to restart to install
updates. Recently, this happened again and it caused Zookeeper ensemble and
Solr instances to go down. When the machines came back up again. I tried
the following

1) Started Zookeeper on all machines using the following command
zkServer.cmd (on all three machines)

2) Started Solr on two of those machines using

solr.cmd start -c -p 8983 -h [server1_name] -z
"[server1_ip]:2181,[server2_name]:2181,[server3_name]:2181"
solr.cmd start -c -p 8983 -h [server2_name] -z
"[server2_ip]:2181,[server1_name]:2181,[server3_name]:2181"
solr.cmd start -c -p 7574 -h [server1_name] -z
"[server1_ip]:2181,[server2_name]:2181,[server3_name]:2181"
solr.cmd start -c -p 7574 -h [server2_name] -z
"[server2_ip]:2181,[server1_name]:2181,[server3_name]:2181"

After several trials, it did start Solr on both machines but *non of the
previous collections came back normally.* When I look at the admin page, it
shows errors as follows

*[Collection_name]_shard2_replica2:*
org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
Index locked for write for core '[Collection_name]_shard2_replica2'. Solr
now longer supports forceful unlocking via 'unlockOnStartup'. Please verify
locks manually!

So probably I am doing something wrong or there is a different way of
bringing old collections up.

Appreciate your comments/feedback regarding this.

Regards,
Salman