Thanks! This is what I was looking for :) Andre
On Thursday, April 6, 2017 at 8:03:46 AM UTC-7, Jakob de Maeyer wrote: > > Ah, my apologies, I'm an idiot. ;) Of course the settings attribute only > gets set *after* the __init__() call. > > In that case, if you need the settings inside __init__(), it's probably > simplest to override from_crawler and pass the settings object to > __init__(): > > class MySpider(scrapy.Spider): > name = "myspider" > def __init__(self, settings, *args, **kwargs): > super(MySpider, self).__init__(*args, **kwargs) > if settings['FOO']: > # Enter panic mode > pass > @classmethod > def from_crawler(cls, crawler, *args, **kwargs): > spider = cls(crawler.settings, *args, **kwargs) > spider._set_crawler(crawler) > return spider > > > Sorry about the noise ;), > -Jakob > > > On Thu, Apr 6, 2017 at 4:48 PM Andre King <andre.a...@gmail.com > <javascript:>> wrote: > >> Hi Jakob, >> >> Actually, I get an error that the spider has no attribute "settings". >> Please see the small project I created to show this: >> https://github.com/andreking/hounder >> >> - Andre >> >> >> On Thursday, April 6, 2017 at 1:28:32 AM UTC-7, Jakob de Maeyer wrote: >>> >>> Hey Andre, >>> >>> the default from_crawler method will set the Settings object as settings >>> attribute of your spider, so you can just do this in your init (and leave >>> from_crawler untouched): >>> >>> def __init__(self, *args, **kwargs): >>> super(MySpider, self).__init__(*args, **kwargs) >>> if self.settings['FOO']: >>> # Enter panic mode >>> >>> Cheers!, >>> -Jakob >>> >>> >>> On Thursday, April 6, 2017 at 1:12:11 AM UTC+2, Andre King wrote: >>>> >>>> Actually this brings up the problem that my command-line args get >>>> clobbered, unless I perform a check: if 'foo' in kwargs, do nothing. It >>>> seems like I'm doing something wrong. Any tips? >>>> >>>> On Wednesday, April 5, 2017 at 4:00:27 PM UTC-7, Andre King wrote: >>>>> >>>>> Just answered my own questions: >>>>> >>>>> (1) Do NOT call get_project_settings() in spider's __init__ method >>>>> because: this will retrieve only the settings in settings.py, not any >>>>> custom spider or command line settings. >>>>> (2) In order to pass arguments into the spider's __init__ method from >>>>> the from_crawler method, make sure both methods accept *args and **kwargs >>>>> as input. Then you can do something like this in from_crawler: >>>>> >>>>> my_setting = crawler.settings.get('MY_SETTING') >>>>> >>>>> kwargs['foo'] = my_setting >>>>> >>>>> obj = super(MySpider, self).from_crawler(crawler, *args, **kwargs) >>>>> >>>>> return obj >>>>> >>>>> On Wednesday, April 5, 2017 at 3:33:55 PM UTC-7, Andre King wrote: >>>>>> >>>>>> So far, I've read that settings should be gotten in the from_crawler >>>>>> method and passed in as args or kwargs to the spider __init__ method. >>>>>> >>>>>> I wasn't able to figure out how to do this, so my solution has been >>>>>> to simply call get_project_settings() in the spider's __init__ method >>>>>> instead and go from there. >>>>>> >>>>>> Is there anything wrong with this approach? If so, can someone >>>>>> provide an example as to how to pass settings to __init__ from >>>>>> from_crawler? >>>>>> >>>>>> Thanks, >>>>>> Andre >>>>>> >>>>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "scrapy-users" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/scrapy-users/U2DNE1a2Yd0/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> scrapy-users...@googlegroups.com <javascript:>. >> To post to this group, send email to scrapy...@googlegroups.com >> <javascript:>. >> Visit this group at https://groups.google.com/group/scrapy-users. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users+unsubscr...@googlegroups.com. To post to this group, send email to scrapy-users@googlegroups.com. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
