When I tried to run the mentioned script I go exception. Would you please
tell me what caused that?
WARN security.UserGroupInformation: PriviledgedActionException as:hdfs
(auth:SIMPLE) cause:java.io.IOException: Not a file:
hdfs://namenode:8020/hbase
14/05/19 17:16:31 WARN security.UserGroupInformation:
PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.io.IOException:
Not a file: hdfs://namenode:8020/hbase
14/05/19 17:16:31 ERROR crawl.Injector: Injector: java.io.IOException: Not
a file: hdfs://namenode:8020/hbase
at
org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:277)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:520)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:512)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:833)
at org.apache.nutch.crawl.Injector.inject(Injector.java:279)
at org.apache.nutch.crawl.Injector.run(Injector.java:316)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.nutch.crawl.Injector.main(Injector.java:306)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
On Mon, May 19, 2014 at 5:10 PM, Ali Nazemian <[email protected]> wrote:
> Thank you very much.
>
>
> On Mon, May 19, 2014 at 5:06 PM, Julien Nioche <
> [email protected]> wrote:
>
>> there is no crawl class any more. Re-crawl class meant "regarding the
>> crawl
>> class". Again there is now a script for it
>>
>>
>> On 19 May 2014 13:08, Ali Nazemian <[email protected]> wrote:
>>
>> > Hi,
>> > Unfortunately I did not find any re-crawl class.
>> >
>> >
>> > On Mon, May 19, 2014 at 3:44 PM, Julien Nioche <
>> > [email protected]> wrote:
>> >
>> > > Hi Ali
>> > >
>> > > See http://wiki.apache.org/nutch/NutchHadoopSingleNodeTutorial for
>> the
>> > > hadoop cluster part.
>> > > Re- Crawl class : there is now a script for crawling, see
>> bin/crawl.sh.
>> > It
>> > > is easier to modify than the all in one Crawl class and gives a good
>> > > understanding of the underlying processing steps.
>> > >
>> > > Best
>> > >
>> > > Julien
>> > >
>> > >
>> > > On 19 May 2014 11:55, Ali Nazemian <[email protected]> wrote:
>> > >
>> > > > Hi,
>> > > > I was wondering how can I run nutch 1.8 job on hadoop cluster? As
>> far
>> > as
>> > > > know for running nutch 1.7 job on hadoop cluster we could use
>> > > > org.apache.nutch.crawl.Crawl class. Since this class was deprecated
>> and
>> > > > removed from nutch 1.8 what class would be responsible for crawling
>> > job?
>> > > > Which class should I put in hadoop command?
>> > > > Best regards.
>> > > > --
>> > > > A.Nazemian
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > >
>> > > Open Source Solutions for Text Engineering
>> > >
>> > > http://digitalpebble.blogspot.com/
>> > > http://www.digitalpebble.com
>> > > http://twitter.com/digitalpebble
>> > >
>> >
>> >
>> >
>> > --
>> > A.Nazemian
>> >
>>
>>
>>
>> --
>>
>> Open Source Solutions for Text Engineering
>>
>> http://digitalpebble.blogspot.com/
>> http://www.digitalpebble.com
>> http://twitter.com/digitalpebble
>>
>
>
>
> --
> A.Nazemian
>
--
A.Nazemian
