> When I restart Solr How exactly are you restarting Solr? Are you running a "bin/solr restart"? Or is Solr already shut down and you're just starting it back up with a "bin/solr start <other-args>"? Depending on how Solr was shut down, you might be running into a bit of a known-issue with Solr's HDFS support. Solr creates lock files for each index, to restrict who can write to that index in the interest of avoiding race conditions and protecting against file corruption. Often when Solr crashes or is shut down abruptly (via a "kill -9") it doesn't have time to clean up these lock files and it fails to start up the next time because it is still locked out from touching that index. This might be what you're running in to. In which case you could carefully make sure that no Solr nodes are using the index in question, delete the lock file manually out of HDFS, and try starting Solr again.
The advice above is what we usually tell people with write.lock issues on HDFS...though some elements of the stack trace you provided make me wonder whether you're seeing the same exact problem. Your stack trace has a NullPointerException, and a "Filesystem Closed" error (typically seen when a Java object gets closed too early and may indicate a bug). I'm not used to seeing either of these associated with the "standard" write.lock issues. What version of Solr are you seeing this on? Best regards, Jason On Thu, Mar 14, 2019 at 5:28 AM VAIBHAV SHUKLA shuklavaibha...@yahoo.in <vaibhavshukla2...@gmail.com> wrote: > > When I restart Solr it throws the following error. Solr collection indexed to > pdf in hdfs throws error during solr restart. > > > > Error > > java.util.concurrent.ExecutionException: > org.apache.solr.common.SolrException: Unable to create core [PDFIndex] > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at > org.apache.solr.core.CoreContainer.lambda$load$6(CoreContainer.java:594) > at > com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.solr.common.SolrException: Unable to create core > [PDFIndex] > at > org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:966) > at > org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:565) > at > com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197) > ... 5 more > Caused by: org.apache.solr.common.SolrException: Index dir > 'hdfs://192.168.1.16:8020/PDFIndex/data/index/' of core 'PDFIndex' is already > locked. The most likely cause is another Solr server (or another solr core in > this server) also configured to use this directory; other possible causes may > be specific to lockType: hdfs > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:977) > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830) > at > org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:950) > ... 7 more > Caused by: org.apache.lucene.store.LockObtainFailedException: Index dir > 'hdfs://192.168.1.16:8020/PDFIndex/data/index/' of core 'PDFIndex' is already > locked. The most likely cause is another Solr server (or another solr core in > this server) also configured to use this directory; other possible causes may > be specific to lockType: hdfs > at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:712) > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:923) > ... 9 more > 2018-12-22 07:55:13.431 ERROR > (OldIndexDirectoryCleanupThreadForCore-PDFIndex) [ x:PDFIndex] > o.a.s.c.HdfsDirectoryFactory Error checking for old index directories to > clean-up. > java.io.IOException: Filesystem closed > at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:808) > at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2083) > at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2069) > at > org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:791) > at > org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106) > at > org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:853) > at > org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:849) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:860) > at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1517) > at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1557) > at > org.apache.solr.core.HdfsDirectoryFactory.cleanupOldIndexDirectories(HdfsDirectoryFactory.java:546) > at > org.apache.solr.core.SolrCore.lambda$cleanupOldIndexDirectories$19(SolrCore.java:3050) > at java.lang.Thread.run(Thread.java:748) > 2018-12-22 07:55:13.433 ERROR > (OldIndexDirectoryCleanupThreadForCore-PDFIndex) [ x:PDFIndex] > o.a.s.c.SolrCore Failed to cleanup old index directories for core PDFIndex > java.lang.NullPointerException > at > org.apache.solr.core.HdfsDirectoryFactory.cleanupOldIndexDirectories(HdfsDirectoryFactory.java:564) > at > org.apache.solr.core.SolrCore.lambda$cleanupOldIndexDirectories$19(SolrCore.java:3050) > at java.lang.Thread.run(Thread.java:748) > > I have created the collection in Solr which will index the pdf files and this > collection is indexing all the pdf in HDFS. > > Thanks & Regards > Vaibhav Shukla > Sent from Mail for Windows 10 >
