Hi Team, I am using aws efs to store a lucene index. While indexing for around 200 millions records. The efs read iops went till 20,097 GB's and the cost for efs went too high. Whereas the write iops was only 56GB
In my use case i am committing every 100k records (because in my test scenarios committing per million was taking a lot of time) Below is the snippet on how i am instantiating lucene writter FSDirectory indexDirectory = NIOFSDirectory.open(indexDir.toPath()); StandardAnalyzer analyzer = new StandardAnalyzer(); IndexWriterConfig indexWriterConfig = new IndexWriterConfig(analyzer); writter = new IndexWriter(indexDirectory, indexWriterConfig); Can someone please help to understand why such huge reads are happening? and how to mitigate such issues. Thanks in advance Gopal Sharma
