[
https://issues.apache.org/jira/browse/SOLR-8937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley updated SOLR-8937:
-------------------------------
Attachment: SOLR_8937_post_streaming.patch
> bin/post (SimplePostTool) should stream/chunk if size not known
> ---------------------------------------------------------------
>
> Key: SOLR-8937
> URL: https://issues.apache.org/jira/browse/SOLR-8937
> Project: Solr
> Issue Type: Improvement
> Components: SimplePostTool
> Reporter: David Smiley
> Assignee: David Smiley
> Attachments: SOLR_8937_post_streaming.patch
>
>
> {{SimplePostTool.postData()}} takes a {{Long length}} arg that is sometimes
> null -- for stdin and for processing some web pages for links. If it's null,
> we should this on the underlying HttpURLConnection:
> {{urlc.setChunkedStreamingMode(-1);}}. I have found that I can't use
> bin/post to stream large files from stdin since it the JDK's implementation
> wants to put the whole thing in memory first. Sadly this is true of {{curl}}
> as well. When I set chunk mode, it works. (FYI -1 tells it to use its
> default chunk size -- 4k).
> FYI I'm using stdin because I've got massive JSON files zipped up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
