On Wednesday, December 14, 2016 at 8:14:39 AM UTC-8, Masood Rehman wrote: > > I have a scrapy project '*tutorial*' in the following directory > > *C:\wamp64\www\tutorial>* > > the project directory structure is given below. > > > <https://lh3.googleusercontent.com/-oSxzXOpFxQ0/WFFtRJmTw_I/AAAAAAAAAJE/rfVJe7wEHvEXi9SzNjtI4RyYxUvfedRfwCLcB/s1600/quotes_spider.jpg> > > > > > > > > > > > > I want to run the spider through my custom python script *runspiders.py* > > from __future__ import print_function > import scrapy > from scrapy.crawler import CrawlerProcess > from scrapy.utils.project import get_project_settings > > > def main(): > process = CrawlerProcess(get_project_settings()) > process.crawl("quotes") > process.start() > > if __name__ == '__main__' : main() > > The spider runs correctly when the custom python script run from inside > the scrapy project folder e.g > > *C:\wamp64\www\tutorial>python runspiders.py* > > But the scrapy raise the following exception when the custom python script > run from outside the project folder e.g > > *C:\wamp64\www>python tutorial/runspiders.py* > > File "C:\Python27\lib\site-packages\scrapy\spiderloader.py", line 43, in > load > raise KeyError("Spider not found: {}".format(spider_name)) > KeyError: 'Spider not found: quotes' > > Thank you in advance. >
The reason is because Scrapy looks for the scrapy.cfg file, which contains your project structure and project name. Running it outside your project folder where there isn't a scrapy.cfg will not work. -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users+unsubscr...@googlegroups.com. To post to this group, send email to scrapy-users@googlegroups.com. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
