This is an automated email from the ASF dual-hosted git repository.
snagel pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/nutch.git.
from b11082a Merge pull request #345 from
sebastian-nagel/NUTCH-2595-crawler-commons-0.10
add 32860a5 NUTCH-2576 HTTP protocol implementation based on okhttp -
derived from storm-crawler#443 (contributed by Julien Nioche)
add 659e1c8 NUTCH-2576 HTTP protocol implementation based on okhttp -
fix: copy buffer(s) entirely to response (readFully vs. read)
add f598db7 NUTCH-2576 HTTP protocol implementation based on okhttp - do
not catch IOException thrown by okhttp, to be caught by calling method
(FetcherThread)
add dbdb40b NUTCH-2576 HTTP protocol implementation based on okhttp - set
Cookie header (same as in protocol-http)
add 466a0ed NUTCH-2576 HTTP protocol implementation based on okhttp -
port unit tests for NUTCH-2549 from protocol-http - ignore failing unit tests
add f1aa728 NUTCH-2576 HTTP protocol implementation based on okhttp -
change port number of test server used for unit tests to avoid error with
concurrently running tests of protocol-http
new 4bcaeeb Merge pull request #328 from
sebastian-nagel/nutch-2576-protocol-okhttp
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
build.xml | 4 +
conf/nutch-default.xml | 22 ++
.../org/apache/nutch/metadata/HttpHeaders.java | 4 +
.../org/apache/nutch/net/protocols/Response.java | 26 ++
src/plugin/build.xml | 3 +
.../apache/nutch/protocol/http/api/HttpBase.java | 64 ++++-
.../nutch/protocol/htmlunit/HttpResponse.java | 4 +-
.../apache/nutch/protocol/http/HttpResponse.java | 23 +-
.../{protocol-http => protocol-okhttp}/build.xml | 2 +-
.../{index-metadata => protocol-okhttp}/ivy.xml | 1 +
.../jsp/basic-http.jsp | 0
.../jsp/brokenpage.jsp | 0
.../jsp/redirect301.jsp | 0
.../jsp/redirect302.jsp | 0
.../{protocol-http => protocol-okhttp}/plugin.xml | 22 +-
.../org/apache/nutch/protocol/okhttp/OkHttp.java | 248 ++++++++++++++++
.../nutch/protocol/okhttp/OkHttpResponse.java | 191 +++++++++++++
.../nutch/protocol/okhttp}/package-info.java | 5 +-
.../src/test/conf/nutch-site-test.xml | 21 +-
.../protocol/okhttp/TestBadServerResponses.java | 318 +++++++++++++++++++++
.../nutch/protocol/okhttp/TestProtocolOkHttp.java} | 9 +-
21 files changed, 912 insertions(+), 55 deletions(-)
copy src/plugin/{protocol-http => protocol-okhttp}/build.xml (97%)
copy src/plugin/{index-metadata => protocol-okhttp}/ivy.xml (94%)
copy src/plugin/{protocol-http => protocol-okhttp}/jsp/basic-http.jsp (100%)
copy src/plugin/{protocol-http => protocol-okhttp}/jsp/brokenpage.jsp (100%)
copy src/plugin/{protocol-http => protocol-okhttp}/jsp/redirect301.jsp (100%)
copy src/plugin/{protocol-http => protocol-okhttp}/jsp/redirect302.jsp (100%)
copy src/plugin/{protocol-http => protocol-okhttp}/plugin.xml (70%)
create mode 100755
src/plugin/protocol-okhttp/src/java/org/apache/nutch/protocol/okhttp/OkHttp.java
create mode 100644
src/plugin/protocol-okhttp/src/java/org/apache/nutch/protocol/okhttp/OkHttpResponse.java
copy src/{java/org/apache/nutch/scoring =>
plugin/protocol-okhttp/src/java/org/apache/nutch/protocol/okhttp}/package-info.java
(83%)
copy src/plugin/{protocol-httpclient =>
protocol-okhttp}/src/test/conf/nutch-site-test.xml (75%)
create mode 100644
src/plugin/protocol-okhttp/src/test/org/apache/nutch/protocol/okhttp/TestBadServerResponses.java
copy
src/plugin/{protocol-http/src/test/org/apache/nutch/protocol/http/TestProtocolHttp.java
=>
protocol-okhttp/src/test/org/apache/nutch/protocol/okhttp/TestProtocolOkHttp.java}
(96%)
--
To stop receiving notification emails like this one, please contact
sna...@apache.org.
