Hi Tamer, Actually you can skip the check with a “hack”:
You can override the check by enforcing number of threads and number of merges at same time by setting this on your own config when using ConcurrentMergeScheduler: https://goo.gl/5QJpMh If you override maxThreadCount and maxMergeCount in the CMS instance, the check is not executed. You may pass the CMS config using IndexWriterConfig. In addition to find out the real issue, we need more information: The problem I had was that I cannot say where it exactly stopped for your case, because your “stack” trace-like output had no line numbers. It would be better to run “jstack <pid>” on command line when it hangs to see the line number in Lucene code. There are 2 places that might hang: Listing the mount points on its own or listing their block device properties inside “/sys/block” folder. Just to be sure: I hope you don’t place your indexes on NFS mounts? Aren’t you? Uwe ----- Uwe Schindler Achterdiek 19, D-28357 Bremen http://www.thetaphi.de eMail: [email protected] > -----Original Message----- > From: Tamer Gur [mailto:[email protected]] > Sent: Wednesday, July 12, 2017 4:57 PM > To: [email protected]; Uwe Schindler <[email protected]> > Subject: Re: stucked indexing process > > thanks Uwe for reply. we are indexing data in a cluster where there are > many mount points so it is possible that one them has issue or slowness > when this check first tried but now when i execute "mount" it is > responding all the mount points. > > I was wondering is there any configuration to skip this SSD check? > > Tamer > > On 12/07/2017 14:15, Uwe Schindler wrote: > > > > Hi, > > > > to figure out if you system is using an SSD drive for the index > > directory, the merge scheduler has to get the underlying mount point > > of the index directory. As there is no direct lookup for that, it > > needs to list all mount points in the system with a Java7 FS function. > > And that seems to hang for some reason. Could it be that you have a > > mount (like NFS or CIFS) that no longer responds? > > > > Just list all with “cat /proc/mounts” or the “mount” command and check > > if any of them is stuck or no longer responding. > > > > Uwe > > > > ----- > > > > Uwe Schindler > > > > Achterdiek 19, D-28357 Bremen > > > > http://www.thetaphi.de <http://www.thetaphi.de/> > > > > eMail: [email protected] > > > > *From:*Tamer Gur [mailto:[email protected]] > > *Sent:* Wednesday, July 12, 2017 12:29 PM > > *To:* [email protected] > > *Subject:* stucked indexing process > > > > Hi all, > > > > we are having an issue in our indexing pipeline time to time our > > indexing process are stucked. Following text&picture is from jvisualvm > > and it seems process is waiting at > > sun.nio.fs.UnixFileSystem$FileStoreIterator.hasNext() method all the > > time. we are using lucene 5.4.1 and java 1.8.0_65-b17. > > > > what can be the reason of this? > > > > Many Thanks > > > > Tamer > > > > text version > > > > " > > > org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.<init > >()","100.0","73509067","73509067","3" > > " > > > org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.<init > >()","100.0","73509067","73509067","3" > > " > > > org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.addC > ategory()","100.0","73509067","73509067","3" > > " > > > org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.inter > nalAddCategory()","100.0","73509067","73509067","3" > > " > > > org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.addC > ategoryDocument()","100.0","73509067","73509067","3" > > " > > > org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.getT > axoArrays()","100.0","73509067","73509067","3" > > " > > > org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.initR > eaderManager()","100.0","73509067","73509067","3" > > " > > > org.apache.lucene.index.ReaderManager.<init>()","100.0","73509067","7350 > 9067","3" > > " > > > org.apache.lucene.index.DirectoryReader.open()","100.0","73509067","7350 > 9067","3" > > " > > > org.apache.lucene.index.IndexWriter.getReader()","100.0","73509067","735 > 09067","3" > > " > > > org.apache.lucene.index.IndexWriter.maybeMerge()","100.0","73509067","7 > 3509067","3" > > " > > > org.apache.lucene.index.ConcurrentMergeScheduler.merge()","100.0","7350 > 9067","73509067","3" > > " > > > org.apache.lucene.index.ConcurrentMergeScheduler.initDynamicDefaults()", > "100.0","73509067","73509067","3" > > " > > org.apache.lucene.util.IOUtils.spins()","100.0","73509067","73509067","3" > > " > > org.apache.lucene.util.IOUtils.spins()","100.0","73509067","73509067","3" > > " > > > org.apache.lucene.util.IOUtils.spinsLinux()","100.0","73509067","73509067", > "3" > > " > > > org.apache.lucene.util.IOUtils.getFileStore()","100.0","73509067","73509067" > ,"3" > > " > > > sun.nio.fs.UnixFileSystem$FileStoreIterator.hasNext()","100.0","73509067"," > 73509067","3" > > > > image version > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
