[jira] [Commented] (NUTCH-2725) Plugin lib-http to support per-host configurable cookies

2019-07-29 Thread Hudson (JIRA) 


[ 
https://issues.apache.org/jira/browse/NUTCH-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16895166#comment-16895166
 ] 

Hudson commented on NUTCH-2725:
---

FAILURE: Integrated in Jenkins build Nutch-trunk #3630 (See 
[https://builds.apache.org/job/Nutch-trunk/3630/])
NUTCH-2725 Plugin lib-http to support per-host configurable cookies (markus: 
[https://github.com/apache/nutch/commit/54f73bf78ded8b66ba262270d069232417bbe391])
* (edit) 
src/plugin/protocol-okhttp/src/java/org/apache/nutch/protocol/okhttp/OkHttpResponse.java
* (edit) 
src/plugin/protocol-httpclient/src/java/org/apache/nutch/protocol/httpclient/HttpResponse.java
* (edit) 
src/plugin/protocol-http/src/java/org/apache/nutch/protocol/http/HttpResponse.java
* (edit) conf/nutch-default.xml
* (edit) 
src/plugin/lib-http/src/java/org/apache/nutch/protocol/http/api/HttpBase.java
* (add) conf/cookies.txt


> Plugin lib-http to support per-host configurable cookies
> 
>
> Key: NUTCH-2725
> URL: https://issues.apache.org/jira/browse/NUTCH-2725
> Project: Nutch
>  Issue Type: Improvement
>  Components: protocol
>Affects Versions: 1.15
>Reporter: Markus Jelsma
>Assignee: Markus Jelsma
>Priority: Major
> Fix For: 1.16
>
> Attachments: NUTCH-2725.patch, NUTCH-2725.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (NUTCH-2725) Plugin lib-http to support per-host configurable cookies

2019-07-29 Thread Markus Jelsma (JIRA) 


[ 
https://issues.apache.org/jira/browse/NUTCH-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16895139#comment-16895139
 ] 

Markus Jelsma commented on NUTCH-2725:
--

Committed a67c9bee..54f73bf7  master -> master

Thanks Sebastian!


> Plugin lib-http to support per-host configurable cookies
> 
>
> Key: NUTCH-2725
> URL: https://issues.apache.org/jira/browse/NUTCH-2725
> Project: Nutch
>  Issue Type: Improvement
>  Components: protocol
>Affects Versions: 1.15
>Reporter: Markus Jelsma
>Assignee: Markus Jelsma
>Priority: Major
> Fix For: 1.16
>
> Attachments: NUTCH-2725.patch, NUTCH-2725.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (NUTCH-2725) Plugin lib-http to support per-host configurable cookies

2019-07-29 Thread Sebastian Nagel (JIRA) 


[ 
https://issues.apache.org/jira/browse/NUTCH-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16895052#comment-16895052
 ] 

Sebastian Nagel commented on NUTCH-2725:


+1 looks good!

> Plugin lib-http to support per-host configurable cookies
> 
>
> Key: NUTCH-2725
> URL: https://issues.apache.org/jira/browse/NUTCH-2725
> Project: Nutch
>  Issue Type: Improvement
>  Components: protocol
>Affects Versions: 1.15
>Reporter: Markus Jelsma
>Assignee: Markus Jelsma
>Priority: Major
> Fix For: 1.16
>
> Attachments: NUTCH-2725.patch, NUTCH-2725.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (NUTCH-2725) Plugin lib-http to support per-host configurable cookies

2019-07-25 Thread Markus Jelsma (JIRA) 


[ 
https://issues.apache.org/jira/browse/NUTCH-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16892873#comment-16892873
 ] 

Markus Jelsma commented on NUTCH-2725:
--

Addressed all three points. Thanks Sebastian!

> Plugin lib-http to support per-host configurable cookies
> 
>
> Key: NUTCH-2725
> URL: https://issues.apache.org/jira/browse/NUTCH-2725
> Project: Nutch
>  Issue Type: Improvement
>  Components: protocol
>Affects Versions: 1.15
>Reporter: Markus Jelsma
>Assignee: Markus Jelsma
>Priority: Major
> Fix For: 1.16
>
> Attachments: NUTCH-2725.patch, NUTCH-2725.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (NUTCH-2725) Plugin lib-http to support per-host configurable cookies

2019-07-25 Thread Sebastian Nagel (JIRA) 


[ 
https://issues.apache.org/jira/browse/NUTCH-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16892858#comment-16892858
 ] 

Sebastian Nagel commented on NUTCH-2725:


Hi [~markus17] looks good and works. A few minor points:
* converting the URL object to a String, then parsing it again doesn't seem 
efficient (could just pass the URL object itself):
{code}
cookie = http.getCookie(url.toString());
...
public String getCookie(String url) {
   if (hostCookies != null) {
 return hostCookies.get(URLUtil.getHost(url));
   }
...
{code}
* comment lines in the cookies.txt file cause an exception and the rest of the 
file is ignored (should generally report and skip invalid lines and continue):
{noformat}
2019-07-25 16:58:24,052 WARN  http.Http - Failed to read 
http.agent.host.cookie.file cookies.txt: 
java.lang.ArrayIndexOutOfBoundsException: 1
at 
org.apache.nutch.protocol.http.api.HttpBase.setConf(HttpBase.java:278)
{noformat}
* could add "http.agent.host.cookie.file" to nutch-default.xml

> Plugin lib-http to support per-host configurable cookies
> 
>
> Key: NUTCH-2725
> URL: https://issues.apache.org/jira/browse/NUTCH-2725
> Project: Nutch
>  Issue Type: Improvement
>  Components: protocol
>Affects Versions: 1.15
>Reporter: Markus Jelsma
>Assignee: Markus Jelsma
>Priority: Major
> Fix For: 1.16
>
> Attachments: NUTCH-2725.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)