Tim Allison created NUTCH-3020:
----------------------------------
Summary: ParseSegment should check for protocol's flags for
truncation
Key: NUTCH-3020
URL: https://issues.apache.org/jira/browse/NUTCH-3020
Project: Nutch
Issue Type: Task
Reporter: Tim Allison
As discussed on the user list, several protocols can identify when a fetch has
been truncated. ParseSegment only checks for the number of bytes fetched vs the
http length header (if it exists). We should modify ParseSegment to check for
notification of truncation from the protocols.
I noticed this specifically with okhttp, but other protocols may flag
truncation as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
