[ https://issues.apache.org/jira/browse/SOLR-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-4227: ---------------------------------------- Attachment: SOLR-4227.patch After SOLR-1565, the relevant code lies in RequestWriter. I'll commit this shortly. > StreamingUpdateSolrServer does not buffer OutputStreamWriter with > BufferedWriter, causing encoding explosion > ------------------------------------------------------------------------------------------------------------ > > Key: SOLR-4227 > URL: https://issues.apache.org/jira/browse/SOLR-4227 > Project: Solr > Issue Type: Improvement > Affects Versions: 3.2 > Environment: Java 1.6, Linux. I am running SOLR 3.2, but the code > doesn't seem different in 3.5. > Reporter: Conrad Herrmann > Fix For: 4.7 > > Attachments: SOLR-4227.patch > > > org.apache.solr.client.solrj.impl.StreamingUpdateSolrServer line 112 is: > OutputStreamWriter writer = new OutputStreamWriter(out, "UTF-8"); > and then we call > req.writeXML( writer ); > Because the writer is not buffered, this causes the XML writer to call the > UTF-8 encoder for each atom being written, like in > org.apache.solr.common.util.XML.writeXML: > out.write('<'); > This causes the stream encoder to allocate a char array to hold it, and > sun.nio.cs.StreamEncoder.implWrite allocates a CharBuffer to wrap it. All > just for one character. > This is particularly a problem when you have a lot of threads (100?) writing > to the SOLR server, they rapidly eat up all the CPU. > It would be helpful to allocate the writer as a BufferedWriter, so encoding > only happens when you flush. JavaDoc for OutputStreamWriter recommends this: > "For top efficiency, consider wrapping an OutputStreamWriter within a > BufferedWriter so as to avoid frequent converter invocations." -- This message was sent by Atlassian JIRA (v6.1.5#6160) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org