Hey there,
I’m trying to figure out is it possible to dynamically assign item names
for some while rest items are defined in items.py file, here’s sample code:
from scrapy.loader import ItemLoaderfrom w3lib.html import
replace_escape_charsfrom scrapy.loader.processors import Compose,
MapComposefrom some_spider.items import someItems
class someSpider(Spider):
name = 'domain'
allowed_domains = ['domain.com']
# rest of code...
def parse(self, response):
# code...
l = ItemLoader(item=someItems(), response=response)
l.default_output_processor = MapCompose(lambda v: v.strip(),
replace_escape_chars)
# title is defined in items.py page
l.add_xpath('title', '//h1/text()')
# unknown_field is unknown item name
unknown_fields = response.xpath('//*[@class="foo"]/text()').extract()
for unknown_field in unknown_fields:
l.add_value('unknown_field', unknown_field)
return l.load_item()
I searched for solutions for this kind of setup but none of solutions
online work. Any help is appreciated, thanks.
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
