Hi,
Yesterday I updated to to Scrapy 1.0.
I was setting variables from within the start_requests so that they were
updated based on some parameters passed into the spider. I did this by
doing something similar to:
self.settings['DOMAIN_DEPTHS'] = 4
Then, I would use this in a custom middleware using code like:
class DomainDepthMiddleware(object):
def __init__(self, domain_depths, default_depth,
domain_pagecounts, domain_pagecount_max):
self.domain_depths = domain_depths
@classmethod
def from_crawler(cls, crawler):
settings = crawler.settings
domain_depths = settings.getdict('DOMAIN_DEPTHS', 0)
return cls(domain_depths)
Now, when I try to access it, it is being set to zero. There are some
other things that make me believe that the way that I am setting these
settings as a dictionary is incorrect. I read about the using the
settings.set('VARIABLE', "value"). Is this what I need to move all my code
to for scrapy 1.0? If so, how are you supposed to set something like the
following.
self.settings['VARIABLE_NAME']['KEY'] = 'string_value'
while adhering to the syntax of settings.set(.....) ?
Thanks.
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
