Hi,
Please try a following:
def parse(self, response):
sel = Selector(response)
sites = sel.xpath('//h3/a')
items = []
for site in sites:
item = GrantsItem()
item['title'] = site.xpath('text()').extract()
item['url'] = site.xpath('@href').extract()
items.append(item)
return items
This loops through all 'h3' elements that have 'a' element as a child.
The original code looped through all 'a' elements, but many such
elements are not related to grants.
Also in the loop the xpath selector needs to be applied to the 'site',
not to the top level 'sel'. This way the selector extracts information
from each link that was matched by '//h3/a'.
I hope this helps,
Jan
On Tue, Feb 4, 2014 at 8:23 AM, Portia Burton <[email protected]> wrote:
> I'm building a scrapy application that looks for film grants. I'm having
> two problems. First, I'm not sure exactly which logic I should use when
> looking for grant on the page. I'm *only* looking for available grants.
> http://www.filmindependent.org/labs-and-programs/grants-and-awards/
>
> Second, I've used firebug to help me determine which dom elements I should
> retrieve, but when I try to save the scraped file into a json file, all I
> get are empty dictionaries in items.json. I have spent many days pouring
> through the documentation
> http://doc.scrapy.org/en/latest/topics/selectors.html, but still not sure
> what I'm doing wrong.
>
> --
> You received this message because you are subscribed to the Google Groups
> "scrapy-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at http://groups.google.com/group/scrapy-users.
> For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/groups/opt_out.
