Hi Jakob,
Actually, I get an error that the spider has no attribute "settings".
Please see the small project I created to show
this: https://github.com/andreking/hounder
- Andre
On Thursday, April 6, 2017 at 1:28:32 AM UTC-7, Jakob de Maeyer wrote:
>
> Hey Andre,
>
> the default from_crawler method will set the Settings object as settings
> attribute of your spider, so you can just do this in your init (and leave
> from_crawler untouched):
>
> def __init__(self, *args, **kwargs):
> super(MySpider, self).__init__(*args, **kwargs)
> if self.settings['FOO']:
> # Enter panic mode
>
> Cheers!,
> -Jakob
>
>
> On Thursday, April 6, 2017 at 1:12:11 AM UTC+2, Andre King wrote:
>>
>> Actually this brings up the problem that my command-line args get
>> clobbered, unless I perform a check: if 'foo' in kwargs, do nothing. It
>> seems like I'm doing something wrong. Any tips?
>>
>> On Wednesday, April 5, 2017 at 4:00:27 PM UTC-7, Andre King wrote:
>>>
>>> Just answered my own questions:
>>>
>>> (1) Do NOT call get_project_settings() in spider's __init__ method
>>> because: this will retrieve only the settings in settings.py, not any
>>> custom spider or command line settings.
>>> (2) In order to pass arguments into the spider's __init__ method from
>>> the from_crawler method, make sure both methods accept *args and **kwargs
>>> as input. Then you can do something like this in from_crawler:
>>>
>>> my_setting = crawler.settings.get('MY_SETTING')
>>>
>>> kwargs['foo'] = my_setting
>>>
>>> obj = super(MySpider, self).from_crawler(crawler, *args, **kwargs)
>>>
>>> return obj
>>>
>>> On Wednesday, April 5, 2017 at 3:33:55 PM UTC-7, Andre King wrote:
>>>>
>>>> So far, I've read that settings should be gotten in the from_crawler
>>>> method and passed in as args or kwargs to the spider __init__ method.
>>>>
>>>> I wasn't able to figure out how to do this, so my solution has been to
>>>> simply call get_project_settings() in the spider's __init__ method instead
>>>> and go from there.
>>>>
>>>> Is there anything wrong with this approach? If so, can someone provide
>>>> an example as to how to pass settings to __init__ from from_crawler?
>>>>
>>>> Thanks,
>>>> Andre
>>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to scrapy-users+unsubscr...@googlegroups.com.
To post to this group, send email to scrapy-users@googlegroups.com.
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
