David Smiley created SOLR-8937:
----------------------------------
Summary: bin/post (SimplePostTool) should stream/chunk if size not
known
Key: SOLR-8937
URL: https://issues.apache.org/jira/browse/SOLR-8937
Project: Solr
Issue Type: Improvement
Components: SimplePostTool
Reporter: David Smiley
Assignee: David Smiley
{{SimplePostTool.postData()}} takes a {{Long length}} arg that is sometimes
null -- for stdin and for processing some web pages for links. If it's null,
we should this on the underlying HttpURLConnection:
{{urlc.setChunkedStreamingMode(-1);}}. I have found that I can't use bin/post
to stream large files from stdin since it the JDK's implementation wants to put
the whole thing in memory first. Sadly this is true of {{curl}} as well. When
I set chunk mode, it works. (FYI -1 tells it to use its default chunk size --
4k).
FYI I'm using stdin because I've got massive JSON files zipped up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
