Hi, When you install the patch, did you see any fails? No fail is tolerated. I am guessing there is something wrong with ivy.xml. I am suggesting that checkout ALL files in Nutch and then try it again.
Best, Jiaxin On Tuesday, February 17, 2015, Jaydeep Bagrecha <[email protected]> wrote: > Hi all, > I am trying to install and build selenium with nutch1.10 on Mac Yosemite. > > having following error after downloading selenium patch( > https://issues.apache.org/jira/browse/NUTCH-1933) and while using “ant > runtime” command (as mentioned by Jiaxin below).Any suggestions to avoid it? > > error: package org.openqa.selenium does not exist > [javac] import org.openqa.selenium.By; > [javac] ^ > error: package org.openqa.selenium does not exist > [javac] import org.openqa.selenium.WebDriver; > [javac] ^ > error: package org.openqa.selenium.firefox does not exist > [javac] import org.openqa.selenium.firefox.FirefoxDriver; > [javac] ^ > error: package org.openqa.selenium.firefox does not exist > [javac] import org.openqa.selenium.firefox.FirefoxProfile; > error: cannot find symbol > [javac] public static ThreadLocal<WebDriver> threadWebDriver = new > ThreadLocal<WebDriver>() { > [javac] ^ > [javac] symbol: class WebDriver > [javac] location: class HttpWebClient > error: cannot find symbol > [javac] protected WebDriver initialValue() > [javac] ^ > [javac] symbol: class WebDriver > error: cannot find symbol > [javac] FirefoxProfile profile = new FirefoxProfile(); > [javac] ^ > [javac] symbol: class FirefoxProfile > error: cannot find symbol > [javac] WebDriver driver = new FirefoxDriver(profile); > [javac] ^ > [javac] symbol: class FirefoxDriver > error: cannot find symbol > [javac] driver = new FirefoxDriver(); > [javac] ^ > [javac] symbol: class FirefoxDriver > [javac] location: class HttpWebClient > > error: cannot find symbol > [javac] new WebDriverWait(driver, 3); > [javac] ^ > [javac] symbol: class WebDriverWait > [javac] location: class HttpWebClient > > error: cannot find symbol > [javac] String innerHtml = > driver.findElement(By.tagName("body")).getAttribute("innerHTML"); > [javac] ^ > [javac] symbol: variable By > [javac] location: class HttpWebClient > > Thanks, > Jaydeep > > On Feb 12, 2015, at 11:37 PM, Jiaxin Ye <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > > Sure. I will do it once I confirm it works... > > On Thursday, February 12, 2015, Mattmann, Chris A (3980) < > [email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > >> This is great, Jiaxin, can you please make a wiki page on the Nutch >> wiki that has this information? >> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> Chris Mattmann, Ph.D. >> Chief Architect >> Instrument Software and Science Data Systems Section (398) >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >> Office: 168-519, Mailstop: 168-527 >> Email: [email protected] >> WWW: http://sunset.usc.edu/~mattmann/ >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> Adjunct Associate Professor, Computer Science Department >> University of Southern California, Los Angeles, CA 90089 USA >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> >> >> >> >> >> >> -----Original Message----- >> From: Jiaxin Ye <[email protected]> >> Reply-To: "[email protected]" <[email protected]> >> Date: Thursday, February 12, 2015 at 9:39 PM >> To: "[email protected]" <[email protected]> >> Subject: Nutch-Selenium in Nutch 1.10 >> >> >Hi Li, Shuo. You are so right. I finished installing and successfully run >> >the butch with selenium and Firefox. I have a question though, does your >> >Firefox plug out for always all the urls we crawled? >> > >> > >> >Hi Prof Mattmann. I think here is the way we install selenium on MAC with >> >OS higher than 10.6 I think... >> > >> > >> >1. Download XQuatz, it's a dmp file, install it directly >> >2. Download Nutch 1.10 >> >3. Download the patch and put it on the Nutch project directory >> >4. patch -p0 < THE PATCH NAME >> >5. DO NOT update the build.xml and the ivy.xml as the selenium tutorial >> >in the github told you. The patch basically updated those .xml file for >> >us. And the patch also installs lib-selenium and protocol selenium for us >> >(Correct me if >> > I am wrong) >> >6. Update tika dependency if needed >> >7. Go to the Nutch project directory and run ant runtime >> >8. Download Firefox >> >9. Open a new terminal and type >> > xvfb -screen scrn 1024x758x34 (I think you can set it smaller if you >> >want...) >> > There should be some errors after entering the command (for me at >> >least). Manually sudo create a /tmp/.X11-unix folder, and then set the >> >mode to 1777. Rerun the command. xvfb should be working. >> >10. Go to nutch > runtime > local and run the crawling command >> > >> > >> >Hope it helps. :) >> > >> > >> >Best, >> >Jiaxin >> > >> > >> > >> > >> > >> >On Thu, Feb 12, 2015 at 1:08 PM, Shuo Li >> ><[email protected] <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >> > >> >I think I have possibly finished installing. >> > >> > >> >What you need to do: >> >0. git status and checkout what you have modified. >> >1. patch -p0 < YOUR_PATCH_FILE >> >2. ant clean jar >> >3. ant runtime >> > >> > >> >Will try crawling using selenium later on. Hope this helped. >_< >> > >> > >> >On Thu, Feb 12, 2015 at 9:20 AM, Mattmann, Chris A (3980) >> ><[email protected] >> ><javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >> > >> >Yes I believe you need to install X11 - why don't you try and report back >> >what you find thanks. >> > >> >Sent from my iPhone >> > >> >On Feb 12, 2015, at 8:28 AM, Jiaxin Ye <[email protected] >> ><javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >> > >> > >> > >> >Hi professor, but can we use Selenium on Mac? >> > >> >On Thursday, February 12, 2015, Mattmann, Chris A (3980) >> ><[email protected] >> ><javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >> > >> >You need Selenium Jiaxin, in order to crawl dynamic pages in the >> >polar dataset you have been assigned in my CSCI 572 search engines class. >> > >> >The instructions for integrating Selenium with Nutch 1.10-trunk >> >are here: >> > >> >https://issues.apache.org/jira/browse/NUTCH-1933 >> > >> > >> >Cheers, >> >Chris >> > >> > >> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> >Chris Mattmann, Ph.D. >> >Chief Architect >> >Instrument Software and Science Data Systems Section (398) >> >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >> >Office: 168-519, Mailstop: 168-527 >> >Email: [email protected] >> >WWW: http://sunset.usc.edu/~mattmann/ >> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> >Adjunct Associate Professor, Computer Science Department >> >University of Southern California, Los Angeles, CA 90089 USA >> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> > >> > >> > >> > >> > >> > >> >-----Original Message----- >> >From: Jiaxin Ye <[email protected]> >> >Reply-To: "[email protected]" <[email protected]> >> >Date: Thursday, February 12, 2015 at 12:46 AM >> >To: "[email protected]" <[email protected]> >> >Subject: Re: Nutch-Selenium in Nutch 1.10 >> > >> >>Well, good choice. I am thinking changing to ubuntu now. The thing is >> why >> >>do we need Selenium anyway? Just easier to perform crawling? >> >> >> >>On Thu, Feb 12, 2015 at 12:25 AM, Shuo Li >> >><[email protected]> wrote: >> >> >> >>Interestingly, I'm a mac user but I don't want to screw my laptop so I'm >> >>using vagrant with Ubuntu Trusty. It doesn't have GUI but Xvfb can still >> >>be installed properly. The issue would be I don't know how to integrate >> >>Selenium with Nutch 1.10. >> >> >> >>On Thu, Feb 12, 2015 at 12:04 AM, Jiaxin Ye >> >><[email protected]> wrote: >> >> >> >>Hi all, >> >> >> >> >> >>Anyone here knows where to find the setup tutorial for Selenium on Mac >> ?? >> >>I find it difficult to install Xvfb on mac. >> >> >> >> >> >>Best, >> >>Jiaxin >> >> >> >> >> >>On Tue, Feb 10, 2015 at 9:42 PM, Sapnashri Suresh >> >><[email protected]> wrote: >> >> >> >>Hi Shuo Li, >> >> >> >> >> >>We were facing a similar issue. Prof. Mattman suggested we look into >> this >> >>patch for Selenium on Nutch 1.10 : >> >>https://issues.apache.org/jira/browse/NUTCH-1933. >> >> >> >> >> >>Hope this helps! >> >> >> >> >> >>Thanks, >> >>Sapna >> >> >> >>On Tue, Feb 10, 2015 at 9:36 PM, Shuo Li >> >><[email protected]> wrote: >> >> >> >>Yop, >> >> >> >> >> >>I'm trying to install selenium in Nutch 1.10. However, this error pops >> >>out: >> >> >> >> >> >>error: package org.apache.nutch.storage does not exist >> >> >> >> >> >> >> >>I can only find this package in Nutch 2.x. Is there a way to use >> Selenium >> >>in 1.10? >> >> >> >> >> >>Any advice would be appreciated. >> >> >> >> >> >>Regards, >> >>Shuo Li >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>-- >> >>Graduate Student >> >>MS in CS (Data Science) >> >>Viterbi School of Engineering >> >>University of Southern California >> >> >> >> >> >>Phone: >> >>+1 650-307-9848 <tel:%2B1%20650-307-9848> <tel:%2B1%20650-307-9848> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> >> >
