I've been trying to bulk index about 11 million PDFs, and while profiling our Solr instance, I noticed that all of the threads that are processing indexing requests are constantly blocking each other during this call:
http-8080-Processor39 [BLOCKED] CPU time: 9:35 java.util.Collections$SynchronizedMap.get(Object) org.pdfbox.pdmodel.font.PDFont.getAFM() org.pdfbox.pdmodel.font.PDSimpleFont.getFontHeight(byte[], int, int) org.pdfbox.util.PDFStreamEngine.showString(byte[]) org.pdfbox.util.operator.ShowTextGlyph.process(PDFOperator, List) org.pdfbox.util.PDFStreamEngine.processOperator(PDFOperator, List) org.pdfbox.util.PDFStreamEngine.processSubStream(PDPage, PDResources, COSStream) org.pdfbox.util.PDFStreamEngine.processStream(PDPage, PDResources, COSStream) org.pdfbox.util.PDFTextStripper.processPage(PDPage, COSStream) org.pdfbox.util.PDFTextStripper.processPages(List) org.pdfbox.util.PDFTextStripper.writeText(PDDocument, Writer) org.pdfbox.util.PDFTextStripper.getText(PDDocument) org.apache.tika.parser.pdf.PDF2XHTML.process(PDDocument, ContentHandler, Metadata) org.apache.tika.parser.pdf.PDFParser.parse(InputStream, ContentHandler, Metadata) org.apache.tika.parser.CompositeParser.parse(InputStream, ContentHandler, Metadata) org.apache.tika.parser.AutoDetectParser.parse(InputStream, ContentHandler, Metadata) org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(SolrQueryRequest, SolrQueryResponse, ContentStream) org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(SolrQueryRequest, SolrQueryResponse) org.apache.solr.handler.RequestHandlerBase.handleRequest(SolrQueryRequest, SolrQueryResponse) org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(SolrQueryRequest, SolrQueryResponse) org.apache.solr.core.SolrCore.execute(SolrRequestHandler, SolrQueryRequest, SolrQueryResponse) org.apache.solr.servlet.SolrDispatchFilter.execute(HttpServletRequest, SolrRequestHandler, SolrQueryRequest, SolrQueryResponse) org.apache.solr.servlet.SolrDispatchFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) org.apache.catalina.core.StandardWrapperValve.invoke(Request, Response) org.apache.catalina.core.StandardContextValve.invoke(Request, Response) org.apache.catalina.core.StandardHostValve.invoke(Request, Response) org.apache.catalina.valves.ErrorReportValve.invoke(Request, Response) org.apache.catalina.core.StandardEngineValve.invoke(Request, Response) org.apache.catalina.connector.CoyoteAdapter.service(Request, Response) org.apache.coyote.http11.Http11Processor.process(InputStream, OutputStream) org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(TcpConnection, Object[]) org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(Socket, TcpConnection, Object[]) org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(Object[]) org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run() java.lang.Thread.run() Has anyone run into this before? Any ideas on how to reduce the contention? Thanks, Gio.
