i started reimplementing it with 4.3.3 and I though I should consult to avoid rewriting it a 3rd time. :-P
When using HttpClient I would like to use the same HttpContext between requests to maintain authentication state and because the fetcher thread is unaware of the specific protocol in use (and it shouldn't) then I thought to make the HttpContext object thread static in order to reuse it between fetches. I noticed the protocol object is usually cached and as such i can make the HttpContext simply a member of the class but I wouldn't want to rely on the fact the protocol object is cached because it might change in the future. Is there a better way to maintain state/context in Protocol objects other than thread static members? On Sat, Apr 5, 2014 at 9:37 AM, d_k <[email protected]> wrote: > Alright. I'll look into it. Thanks! > > > On Sat, Apr 5, 2014 at 12:39 AM, Sebastian Nagel < > [email protected]> wrote: > >> > Define 'addressing'. :-) >> > I didn't refactor because I don't really know which direction will be >> the >> > right direction for that plugin. So in a way the plugin is still the >> same. >> > All I did was to change all the API calls to httpclient 4.1.1 and check >> > that the tests still run (it wasn't as easy as it sounds. :-P ) >> >> That's at least something. Unfortunately, I never had a closer look to >> the httpclient >> plugin, and cannot estimate what level of rewriting is required. >> >> > So what you are saying is that I can make the protocol-httpclient use >> the >> > latest 4.3.x version without breaking anything? >> >> Yes, it should be possible. It happens just often that different versions >> of a lib are used. >> >> > So what do you say? Should I redo it with 4.2.6? Go straight for 4.3.x? >> > I would like to be able to provide a patch for 2.2.1 users and trunk >> users >> > considering i'm a 2.2.1 user myself. >> > What would be the correct approach? >> >> Go straight for 4.3.x and not depend on indirectly on the Solr version. >> >> > What exactly do I need to change and where? >> src/plugin/protocol-httpclient/ivy.xml >> -> add as dependency >> src/plugin/protocol-httpclient/plugin.xml >> -> add as library >> -> add also transitive dependencies >> >> The best was is to have a look at another plugin, e.g., >> indexer-elastic >> >> > Will I still be able to use >> > Eclipse or will it break because Eclipse won't know how to provide the >> > correct dependency? >> >> You have to update the dependencies: >> - if you use IvyDE : add the ivy.xml as IvyDE lib to Java build path >> - if "ant eclipse": change ivy.xml, close the Eclipse project, >> call "ant eclipse", open project again and press F5 "Refresh" >> >> Sebastian >> >> On 04/04/2014 10:56 PM, d_k wrote: >> > On Fri, Apr 4, 2014 at 11:28 PM, Sebastian Nagel < >> [email protected] >> >> wrote: >> > >> >> Hi, >> >> >> >> does it mean you are (also) addressing NUTCH-1086? Would be great, >> >> since this issue is waiting for a solution since long! >> >> >> > >> > Define 'addressing'. :-) >> > I didn't refactor because I don't really know which direction will be >> the >> > right direction for that plugin. So in a way the plugin is still the >> same. >> > All I did was to change all the API calls to httpclient 4.1.1 and check >> > that the tests still run (it wasn't as easy as it sounds. :-P ) >> > >> > >> >>> The reason I picked version 4.1.1 and not the latest is because I >> noticed >> >>> it is already in the build/lib dir and I wasn't sure I can use two >> >> versions >> >>> of the jar with the same namespace without creating conflicts. >> >> >> >> You should be able to use any version of httpclient, but it must be >> >> registered as dependency in the plugin's ivy.xml >> >> (src/plugin/protocol-httpclient/ivy.xml), >> >> not in the "main" ivy/ivy.xml. >> >> >> > >> > Actually I didn't change any ivy xml. I just changed the code to use the >> > new imports and it must have picked up the dependencies by itself. I >> used >> > Eclipse so maybe it has something to do with it. >> > >> > >> >> Each plugin gets its own class loader to solve the problem of >> conflicting >> >> dependencies, see >> >> >> https://wiki.apache.org/nutch/WhatsTheProblemWithPluginsAndClass-loading >> >> >> > >> > So what you are saying is that I can make the protocol-httpclient use >> the >> > latest 4.3.x version without breaking anything? >> > What exactly do I need to change and where? Will I still be able to use >> > Eclipse or will it break because Eclipse won't know how to provide the >> > correct dependency? >> > >> > >> >> I didn't check 2.2.1, but in head of 2.x httpclient 4.2.6 is a >> dependency >> >> of a dependency (solrj) of the indexer-solr plugin. The upgrade has >> been >> >> done >> >> with NUTCH-1568. >> >> >> > >> > So what do you say? Should I redo it with 4.2.6? Go straight for 4.3.x? >> > I would like to be able to provide a patch for 2.2.1 users and trunk >> users >> > considering i'm a 2.2.1 user myself. >> > What would be the correct approach? >> > >> > >> > >> >> Sebastian >> >> >> >> On 04/04/2014 04:14 PM, d_k wrote: >> >>> I've written a patch for the 2.2.1 source code that upgrades the >> >>> protocol-httpclient to httpclient 4.1.1 >> >>> >> >>> Unfortunately I had to adjust the test because currently httpclient >> 4.1.1 >> >>> does not support authenticating with different credentials against >> >>> different realms in the same domain: >> >>> HTTPCLIENT-1490<https://issues.apache.org/jira/browse/HTTPCLIENT-1490 >> > >> >>> . >> >>> >> >>> The reason I picked version 4.1.1 and not the latest is because I >> noticed >> >>> it is already in the build/lib dir and I wasn't sure I can use two >> >> versions >> >>> of the jar with the same namespace without creating conflicts. >> >>> >> >>> My questions are: >> >>> 1) Anyone needs this patch or did I took the wrong path in choosing >> >> 4.1.1? >> >>> 2) If so, under what JIRA issue should I submit it? NUTCH-751? >> >> NUTCH-1086? >> >>> something else? new issue? >> >>> >> >> >> >> >> > >> >> >
