Chris M. Hostetter created SOLR-16265:
-----------------------------------------
Summary: reduce memory usage of ContentWriter based requests in
Http2SolrClient
Key: SOLR-16265
URL: https://issues.apache.org/jira/browse/SOLR-16265
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Chris M. Hostetter
I recently noticed the code below exists in {{Http2SolrClient.createRequest}}...
{code}
if (contentWriter != null) {
Request req = httpClient.newRequest(url +
wparams.toQueryString()).method(method);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
contentWriter.write(baos);
// TODO reduce memory usage
return req.content(
new BytesContentProvider(contentWriter.getContentType(),
baos.toByteArray()));
{code}
* AFAICT there is no (other) existing jira discussing this TODO
* This method is called for most "simple" HTTP2 based requests
** {{Http2SolrClient}} or {{CloudHttp2SolrClient}} -- but not
{{ConcurrentUpdateHttp2SolrClient}}
* This block triggers for anything with a {{ContentWriter}}
** ie: all {{UpdateRequests}} ... and in theory other custom requests
* Part of the issue seems to be that this code repurposes the {{ContentWriter}}
"push" style API into a "pull" style Jetty client API
** Even though {{Http2SolrClient}} has other code used only by
{{ConcurrentUpdateHttp2SolrClient}} ({{initOutStream(...)}}) which does
leverage a "push" style Jetty client API: {{OutputStreamContentProvider}}
* But more silly: we make one (serialized) {{byte[]}} of the data in memory
inside the {{ByteArrayOutputStream}} then we call {{toByteArray()}} which makes
a second copy of the {{byte[]}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
