RE: Solr Index data lost
Just to close this thread – It looks like it’s working fine now. Not sure what mistake I had done last time. But now, the index data is still persistent on the pen drive even after server shutdown and restarting it on a different machine where the pen drive is plugged in. Thanks for all your help.. Regards Vijay From: Vijaya Narayana Reddy Bhoomi Reddy [mailto:vijaya.bhoomire...@whishworks.com] Sent: 21 April 2015 09:22 To: solr-user@lucene.apache.org Subject: Re: Solr Index data lost Shawn, Yes, I had used java -jar start.jar. I haven't tried moving it to a local hard disk, as I wanted to work on two machines (work and home). So was using a pen drive as the index storage. Yesterday, I did the complete indexing and then unplugged the drive from work machine and connected to my personal laptop. Data folder didn't exist. Erick, As per your earlier suggestion, I am using Tika and SolrJ to index the data (both binary and as well database content) and the same had been committed using the SolrJ UpdataRequest. I was able to see the data in the admin UI screen and even performed some searches on the index and it worked fine. Thanks Regards Vijay On 21 April 2015 at 00:42, Erick Erickson erickerick...@gmail.com mailto:erickerick...@gmail.com wrote: Did you commit before you unplugged the drive? Were you able to see data in the admin UI _before_ you unplugged the drive? Best, Erick On Mon, Apr 20, 2015 at 3:58 PM, Vijay Bhoomireddy vijaya.bhoomire...@whishworks.com mailto:vijaya.bhoomire...@whishworks.com wrote: Shawn, I haven’t changed any DirectoryFactory setting in the solrconfig.xml as I am using in a local setup and using the default configurations. Device has been unmounted successfully (confirmed through windows message in the lower right corner). I am using Solr-4.10.2. I simply run a Ctrl-C command in the windows Command prompt to stop Solr, in the same window where it was started earlier. Please correct me if something has been done not in the correct fashion. Thanks Regards Vijay -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org mailto:apa...@elyograg.org ] Sent: 20 April 2015 22:34 To: solr-user@lucene.apache.org mailto:solr-user@lucene.apache.org Subject: Re: Solr Index data lost On 4/20/2015 2:55 PM, Vijay Bhoomireddy wrote: I have configured Solr example server on a pen drive. I have indexed some content. The data directory was under example/solr/collection1/data which is the default one. After indexing, I stopped the Solr server and unplugged the pen drive and reconnected the same. Now, when I navigate to the SolrAdmin UI, I cannot see any data in the index. Any pointers please? In this case, though the installation was on a pen-drive, I think it shouldn't matter to Solr on where the data directory is. So I believe this data folder wiping has happened due to server shutdown. Will the data folder be wiped off if the server is restarted or stopped? How to save the index data between machine failures or planned maintenances? If you are using the default Directory implementation in your solrconfig.xml (NRTCachingDirectoryFactory for 4.x and later, MMapDirectoryFactory for newer 3.x versions), then everything should be persisted correctly. Did you properly unmount/eject the removable volume before you unplugged it? On a non-windows OS, you might also want to run the 'sync' command. If you didn't do the unmount/eject, you can't be sure that the filesystem was properly closed and fully up-to-date on the device. What version of Solr did you use and how exactly did you start Solr and the example? How did you stop Solr? Thanks, Shawn -- The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS. -- The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
Re: Solr Index data lost
Shawn, Yes, I had used java -jar start.jar. I haven't tried moving it to a local hard disk, as I wanted to work on two machines (work and home). So was using a pen drive as the index storage. Yesterday, I did the complete indexing and then unplugged the drive from work machine and connected to my personal laptop. Data folder didn't exist. Erick, As per your earlier suggestion, I am using Tika and SolrJ to index the data (both binary and as well database content) and the same had been committed using the SolrJ UpdataRequest. I was able to see the data in the admin UI screen and even performed some searches on the index and it worked fine. Thanks Regards Vijay On 21 April 2015 at 00:42, Erick Erickson erickerick...@gmail.com wrote: Did you commit before you unplugged the drive? Were you able to see data in the admin UI _before_ you unplugged the drive? Best, Erick On Mon, Apr 20, 2015 at 3:58 PM, Vijay Bhoomireddy vijaya.bhoomire...@whishworks.com wrote: Shawn, I haven’t changed any DirectoryFactory setting in the solrconfig.xml as I am using in a local setup and using the default configurations. Device has been unmounted successfully (confirmed through windows message in the lower right corner). I am using Solr-4.10.2. I simply run a Ctrl-C command in the windows Command prompt to stop Solr, in the same window where it was started earlier. Please correct me if something has been done not in the correct fashion. Thanks Regards Vijay -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: 20 April 2015 22:34 To: solr-user@lucene.apache.org Subject: Re: Solr Index data lost On 4/20/2015 2:55 PM, Vijay Bhoomireddy wrote: I have configured Solr example server on a pen drive. I have indexed some content. The data directory was under example/solr/collection1/data which is the default one. After indexing, I stopped the Solr server and unplugged the pen drive and reconnected the same. Now, when I navigate to the SolrAdmin UI, I cannot see any data in the index. Any pointers please? In this case, though the installation was on a pen-drive, I think it shouldn't matter to Solr on where the data directory is. So I believe this data folder wiping has happened due to server shutdown. Will the data folder be wiped off if the server is restarted or stopped? How to save the index data between machine failures or planned maintenances? If you are using the default Directory implementation in your solrconfig.xml (NRTCachingDirectoryFactory for 4.x and later, MMapDirectoryFactory for newer 3.x versions), then everything should be persisted correctly. Did you properly unmount/eject the removable volume before you unplugged it? On a non-windows OS, you might also want to run the 'sync' command. If you didn't do the unmount/eject, you can't be sure that the filesystem was properly closed and fully up-to-date on the device. What version of Solr did you use and how exactly did you start Solr and the example? How did you stop Solr? Thanks, Shawn -- The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS. -- The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
Re: Solr Index data lost
On 4/20/2015 2:55 PM, Vijay Bhoomireddy wrote: I have configured Solr example server on a pen drive. I have indexed some content. The data directory was under example/solr/collection1/data which is the default one. After indexing, I stopped the Solr server and unplugged the pen drive and reconnected the same. Now, when I navigate to the SolrAdmin UI, I cannot see any data in the index. Any pointers please? In this case, though the installation was on a pen-drive, I think it shouldn't matter to Solr on where the data directory is. So I believe this data folder wiping has happened due to server shutdown. Will the data folder be wiped off if the server is restarted or stopped? How to save the index data between machine failures or planned maintenances? If you are using the default Directory implementation in your solrconfig.xml (NRTCachingDirectoryFactory for 4.x and later, MMapDirectoryFactory for newer 3.x versions), then everything should be persisted correctly. Did you properly unmount/eject the removable volume before you unplugged it? On a non-windows OS, you might also want to run the 'sync' command. If you didn't do the unmount/eject, you can't be sure that the filesystem was properly closed and fully up-to-date on the device. What version of Solr did you use and how exactly did you start Solr and the example? How did you stop Solr? Thanks, Shawn
RE: Solr Index data lost
Shawn, I haven’t changed any DirectoryFactory setting in the solrconfig.xml as I am using in a local setup and using the default configurations. Device has been unmounted successfully (confirmed through windows message in the lower right corner). I am using Solr-4.10.2. I simply run a Ctrl-C command in the windows Command prompt to stop Solr, in the same window where it was started earlier. Please correct me if something has been done not in the correct fashion. Thanks Regards Vijay -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: 20 April 2015 22:34 To: solr-user@lucene.apache.org Subject: Re: Solr Index data lost On 4/20/2015 2:55 PM, Vijay Bhoomireddy wrote: I have configured Solr example server on a pen drive. I have indexed some content. The data directory was under example/solr/collection1/data which is the default one. After indexing, I stopped the Solr server and unplugged the pen drive and reconnected the same. Now, when I navigate to the SolrAdmin UI, I cannot see any data in the index. Any pointers please? In this case, though the installation was on a pen-drive, I think it shouldn't matter to Solr on where the data directory is. So I believe this data folder wiping has happened due to server shutdown. Will the data folder be wiped off if the server is restarted or stopped? How to save the index data between machine failures or planned maintenances? If you are using the default Directory implementation in your solrconfig.xml (NRTCachingDirectoryFactory for 4.x and later, MMapDirectoryFactory for newer 3.x versions), then everything should be persisted correctly. Did you properly unmount/eject the removable volume before you unplugged it? On a non-windows OS, you might also want to run the 'sync' command. If you didn't do the unmount/eject, you can't be sure that the filesystem was properly closed and fully up-to-date on the device. What version of Solr did you use and how exactly did you start Solr and the example? How did you stop Solr? Thanks, Shawn -- The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
Re: Solr Index data lost
On 4/20/2015 4:58 PM, Vijay Bhoomireddy wrote: I haven’t changed any DirectoryFactory setting in the solrconfig.xml as I am using in a local setup and using the default configurations. Device has been unmounted successfully (confirmed through windows message in the lower right corner). I am using Solr-4.10.2. I simply run a Ctrl-C command in the windows Command prompt to stop Solr, in the same window where it was started earlier. You didn't say what command you used to start Solr, but based on using Ctrl-C to stop it, I am guessing it was java -jar start.jar rather than the bin/solr script that was new with 4.10. The Ctrl-C stop method should result in a graceful shutdown of Solr. There should be no problems with persistence based on all this. Any data that was in the index before you stopped Solr should be there after you start it back up. I would not expect to see any problems with a pen drive other than speed, even if it's a fat32 filesystem rather than NTFS. Does it behave differently if you move it to a local hard disk? Thanks, Shawn
Re: Solr Index data lost
Did you commit before you unplugged the drive? Were you able to see data in the admin UI _before_ you unplugged the drive? Best, Erick On Mon, Apr 20, 2015 at 3:58 PM, Vijay Bhoomireddy vijaya.bhoomire...@whishworks.com wrote: Shawn, I haven’t changed any DirectoryFactory setting in the solrconfig.xml as I am using in a local setup and using the default configurations. Device has been unmounted successfully (confirmed through windows message in the lower right corner). I am using Solr-4.10.2. I simply run a Ctrl-C command in the windows Command prompt to stop Solr, in the same window where it was started earlier. Please correct me if something has been done not in the correct fashion. Thanks Regards Vijay -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: 20 April 2015 22:34 To: solr-user@lucene.apache.org Subject: Re: Solr Index data lost On 4/20/2015 2:55 PM, Vijay Bhoomireddy wrote: I have configured Solr example server on a pen drive. I have indexed some content. The data directory was under example/solr/collection1/data which is the default one. After indexing, I stopped the Solr server and unplugged the pen drive and reconnected the same. Now, when I navigate to the SolrAdmin UI, I cannot see any data in the index. Any pointers please? In this case, though the installation was on a pen-drive, I think it shouldn't matter to Solr on where the data directory is. So I believe this data folder wiping has happened due to server shutdown. Will the data folder be wiped off if the server is restarted or stopped? How to save the index data between machine failures or planned maintenances? If you are using the default Directory implementation in your solrconfig.xml (NRTCachingDirectoryFactory for 4.x and later, MMapDirectoryFactory for newer 3.x versions), then everything should be persisted correctly. Did you properly unmount/eject the removable volume before you unplugged it? On a non-windows OS, you might also want to run the 'sync' command. If you didn't do the unmount/eject, you can't be sure that the filesystem was properly closed and fully up-to-date on the device. What version of Solr did you use and how exactly did you start Solr and the example? How did you stop Solr? Thanks, Shawn -- The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
Solr Index data lost
Hi, I have configured Solr example server on a pen drive. I have indexed some content. The data directory was under example/solr/collection1/data which is the default one. After indexing, I stopped the Solr server and unplugged the pen drive and reconnected the same. Now, when I navigate to the SolrAdmin UI, I cannot see any data in the index. Any pointers please? In this case, though the installation was on a pen-drive, I think it shouldn't matter to Solr on where the data directory is. So I believe this data folder wiping has happened due to server shutdown. Will the data folder be wiped off if the server is restarted or stopped? How to save the index data between machine failures or planned maintenances? Thanks Regards Vijay -- The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
