Hi,
I have also tried this :
bin/nutch index -D solr.server.url="
https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc97b4177a_600f_4040_9309_e632c116443f/solr/localWebCollection/";
-D solr.auth=true -D solr.auth.username="User" -D solr.auth.password="PASS"
-D solr.server.type="cloud" final/crawl/crawldb -linkdb final/crawl/linkdb
-dir final/crawl/segments/20*
the result is:
Indexing 250 documents
Deleting 0 documents
Indexing 250 documents
Indexer: java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836)
at org.apache.nutch.indexer.IndexingJob.index(IndexingJob.java:145)
at org.apache.nutch.indexer.IndexingJob.run(IndexingJob.java:222)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.nutch.indexer.IndexingJob.main(IndexingJob.java:231)
any helps on how to fix it?
Thanks,
Bests
Shakiba Davari
On Tue, Jul 26, 2016 at 5:42 PM, shakiba davari <[email protected]>
wrote:
> Hi,
> I am trying to run the following index command:
>
> bin/nutch index -D solr.server.url="
> https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc97b4177a_600f_4040_9309_e632c116443f/solr/localWebCollection/";
> -D solr.auth=true -D solr.auth.username="USER" -D solr.auth.password="PASS"
> -D solr.server.type="cloud" -D solr.zookeeper.url = "
> https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc97b4177a_600f_4040_9309_e632c116443f/config/map-config
> "
> final/crawl/crawldb -linkdb final/crawl -dir final/crawl/segments/
>
> First Question is that I am not sure what exactly should I use for -D
> solr.zookeeper.url, I guess it should be to my configuration,right?
>
> And second is that and I get this :
>
>
> Indexer: java.io.IOException: No FileSystem for scheme: https
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2385)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2392)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2431)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2413)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
> at
> org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:256)
> at
> org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:228)
> at
> org.apache.hadoop.mapred.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:45)
> at
> org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:304)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:520)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:512)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:833)
> at org.apache.nutch.indexer.IndexingJob.index(IndexingJob.java:145)
> at org.apache.nutch.indexer.IndexingJob.run(IndexingJob.java:222)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.nutch.indexer.IndexingJob.main(IndexingJob.java:231)
>
>
> I have seen this happens when using solrindex, but why now? Anyone knows
> what the problem is?
>
> Bests
> Shakiba Davari
>
>
