Hi All,

The issue got resolved with enabling of ivy.

As mentioned below, I was trying to enable selenium plugin for crawling dynamic 
rendered content of javascript for the websites having https enabled 
(https://nseindia.com/live_market/dynaContent/live_analysis/top_gainers_losers.htm?cat=L,
 
https://nseindia.com/live_market/dynaContent/live_analysis/top_gainers_losers.htm?cat=G).

I have followed all the instructions mentioned @ 
https://github.com/apache/nutch/tree/master/src/plugin/protocol-selenium.
keep the plugin, protocol-httpclient along with protocol-selenium, in 
nutch-site.xml @NUTCH_HOME/conf as the crawling websites are of https.
Enabled selenium.take.screenshot and the selenium is running as well.

When I started crawling, I don't see javascript data fetched from the websites 
as well selenium screen captured.

Had any one tried the same, pls do let me know, Thanks!

Below versions were used
Apache nutch version: 1.12
FireFox version: 60.3.0
Selenium version: 3.4.0 (standalone)

Thanks & Regards
Venkata MR
+91 98455 77125

From: Venkata MR
Sent: 04 December 2018 08:35
To: user@nutch.apache.org
Subject: Enable selenium Plugin

Hi,

As part of crawling javascript dynamic content, I was trying to enable selenium 
plugin to apache nutch 1.15.
I was referring to git link 
https://github.com/apache/nutch/tree/master/src/plugin/protocol-selenium to 
enable selenium plugin.

I was getting below error, couldn't able to move further. Any pointers.

[ivy:resolve] commons-lang#commons-lang;2.6 in default: excluding 
commons-lang#commons-lang;2.6!commons-lang.jar
[ivy:resolve] com.google.protobuf#protobuf-java;2.5.0 in default: excluding 
com.google.protobuf#protobuf-java;2.5.0!protobuf-java.jar(bundle)
[ivy:resolve] org.apache.httpcomponents#httpcore;4.4.7 in default: excluding 
org.apache.httpcomponents#httpcore;4.4.7!httpcore.jar
[ivy:resolve] ERROR: impossible to get artifacts when data has not been loaded. 
IvyNode = javax.measure#unit-api;1.0
[ivy:resolve]
[ivy:resolve] :: problems summary ::
[ivy:resolve] :::: WARNINGS
[ivy:resolve]           module not found: 
javax.measure#unit-api;working@LP-5CD7311YGR
[ivy:resolve]   ==== local: tried
[ivy:resolve]     
C:\Users\venkata.mr\.ivy2/local/javax.measure/unit-api/working@LP-5CD7311YGR/ivys/ivy.xml
[ivy:resolve]     -- artifact 
javax.measure#unit-api;working@LP-5CD7311YGR!unit-api.jar:
[ivy:resolve]     
C:\Users\venkata.mr\.ivy2/local/javax.measure/unit-api/working@LP-5CD7311YGR/jars/unit-api.jar
[ivy:resolve]   ==== maven2: tried
[ivy:resolve]     
http://repo1.maven.org/maven2/javax/measure/unit-api/working@LP-5CD7311YGR/unit-api-work...@lp-5cd7311ygr.pom
[ivy:resolve]     -- artifact 
javax.measure#unit-api;working@LP-5CD7311YGR!unit-api.jar:

Thanks & Regards
Venkata MR
+91 98455 77125

::DISCLAIMER::
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended 
for the named recipient(s) only. E-mail transmission is not guaranteed to be 
secure or error-free as information could be intercepted, corrupted, lost, 
destroyed, arrive late or incomplete, or may contain viruses in transmission. 
The e mail and its contents (with or without referred errors) shall therefore 
not attach any liability on the originator or HCL or its affiliates. Views or 
opinions, if any, presented in this email are solely those of the author and 
may not necessarily reflect the views or opinions of HCL or its affiliates. Any 
form of reproduction, dissemination, copying, disclosure, modification, 
distribution and / or publication of this message without the prior written 
consent of authorized representative of HCL is strictly prohibited. If you have 
received this email in error please delete it and notify the sender 
immediately. Before opening any email and/or attachments, please check them for 
viruses and other defects.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Reply via email to