[
https://issues.apache.org/jira/browse/NUTCH-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17783365#comment-17783365
]
Hudson commented on NUTCH-3020:
-------------------------------
SUCCESS: Integrated in Jenkins build Nutch ยป Nutch-trunk #140 (See
[https://ci-builds.apache.org/job/Nutch/job/Nutch-trunk/140/])
NUTCH-3020 -- ParseSegment should check for okhttp's truncation flag (#794)
(github:
[https://github.com/apache/nutch/commit/90849124d757fb0417ea90576e88b1f55da616f1])
* (edit) src/java/org/apache/nutch/parse/ParseSegment.java
* (add) src/test/org/apache/nutch/parse/TestParseSegment.java
> ParseSegment should check for protocol's flags for truncation
> -------------------------------------------------------------
>
> Key: NUTCH-3020
> URL: https://issues.apache.org/jira/browse/NUTCH-3020
> Project: Nutch
> Issue Type: Task
> Reporter: Tim Allison
> Priority: Minor
> Fix For: 1.20
>
>
> As discussed on the user list, several protocols can identify when a fetch
> has been truncated. ParseSegment only checks for the number of bytes fetched
> vs the http length header (if it exists). We should modify ParseSegment to
> check for notification of truncation from the protocols.
> I noticed this specifically with okhttp, but other protocols may flag
> truncation as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
