Hi there, welcome to the scapy community! The online documentation has lots of code snippets with valid import statements.
For example for CrawlSpider, you can check https://docs.scrapy.org/en/latest/topics/spiders.html#crawlspider-example which shows these imports: import scrapy from scrapy.spiders import CrawlSpider, Rule from scrapy.linkextractors import LinkExtractor You also have the Index (https://docs.scrapy.org/en/latest/genindex.html) to go to the different classes and methods. (Most classes are documented on https://docs.scrapy.org and listed in the index) I was about to take LinkExtractor as an example, but in fact it's an alias for scrapy.linkextractors.lxmlhtml.LxmlLinkExtractor See https://docs.scrapy.org/en/latest/topics/link-extractors.html#module-scrapy.linkextractors Let's take Rule as example. In the index, you can find it on https://docs.scrapy.org/en/latest/genindex.html#R Then you follow https://docs.scrapy.org/en/latest/topics/spiders.html#scrapy.spiders.Rule and you can see *class*scrapy.spiders.Rule(*link_extractor*, *callback=None*, *cb_kwargs=None*, *follow=None*, *process_links=None*, *process_request=None*) So to use the Rule class and instances in your code, you'll have to import something like this: from scrapy.spiders import Rule Hope this helps. /Paul. On Thursday, January 5, 2017 at 10:26:50 AM UTC+1, field.hat.red wrote: > > Hey, everyone. I'm new to scrapy and there are things I do not > understand. > The question I have is this. > When I create a spider how to figure out what to import? Maybe I'm a > Python really do not know, do not argue. > For example. > > rules = ( > Rule(LinkExtractor(allow=(r"href=\"\/courses\/\d*\"", )), > callback='parse'), > ) > > Doesn't work for me, if I don't do from scrapy.contrib.spiders import > CrawlSpider, Rule > > Here's how I understand what and when to import? > > Thank you. > > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users+unsubscr...@googlegroups.com. To post to this group, send email to scrapy-users@googlegroups.com. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
