0down votefavorite
<https://stackoverflow.com/questions/45780889/how-do-i-know-scrapy-custom-settings-are-applied-log-shows-general-settings?noredirect=1#>
I use scrapy 1.3 According to documentation I can declare settings for each
specific spider simple enough. Here is my code
class BhSpider(scrapy.Spider):
name = "code"
custom_settings = {'CONCURRENT_REQUESTS' : '20',
'FEED_EXPORT_FIELDS' :['price', 'stock','partnumber','sku',
'name' ,'manufacture','attribute','distributor','upc','descr',
'p_url','main_image','images']}
allowed_domains = ***
But according to logs project settings are used. How do i know project
settings are overcomed by this specific spider settings?
here is the log
2017-08-20 13:47:51 [scrapy.utils.log] INFO: Scrapy 1.3.2 started (bot: pc)
2017-08-20 13:47:51 [scrapy.utils.log] INFO: Overridden settings:
{'NEWSPIDER_MODULE': 'pc.spiders'
, 'FEED_URI': 'test_output1.csv', 'CONCURRENT_REQUESTS': 200, 'SPIDER_MODULES':
['pc.spiders'], 'BO
T_NAME': 'pc', 'USER_AGENT': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36
(KHTML, like Gecko) C
hrome/59.0.3071.115 Safari/537.36', 'FEED_FORMAT': 'csv', 'FEED_EXPORT_FIELDS':
['price', 'stock',
'partnumber', 'sku', 'name', 'manufacture', 'attribute', 'distributor', 'upc',
'descr']}
2017-08-20 13:47:51 [scrapy.middleware] INFO: Enabled extensions:
In the output file Export fields seem to be correct( as in custom settings,
though log does not show that). How can i know that specific settings are
enabled?
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to scrapy-users+unsubscr...@googlegroups.com.
To post to this group, send email to scrapy-users@googlegroups.com.
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
