Interestingly, XPath 1.0 has starts-with() but does not have ends-with() http://www.w3.org/TR/xpath/
You might be interesting in CSS3 "Substring matching attribute selectors" http://www.w3.org/TR/css3-selectors/#attribute-substrings and you can try: div[class$=content] that would work for class attribute ending with "content" Check this sample python shell session: >>> import scrapy >>> html = """ ... <div class="main-content">main content</div> ... <div class="main-content-2">main content 2</div> ... <div class="content-first">content first</div> ... <div class="blog-content">blog content</div> ... <div class="content">content</div> ... """ >>> selector = scrapy.Selector(text=html) >>> selector.xpath('//div[contains(@class, "content")]').extract() [u'<div class="main-content">main content</div>', u'<div class="main-content-2">main content 2</div>', u'<div class="content-first">content first</div>', u'<div class="blog-content">blog content</div>'] >>> selector.css('div[class$="-content"]').extract() [u'<div class="main-content">main content</div>', u'<div class="blog-content">blog content</div>'] >>> selector.css('div[class$="content"]').extract() [u'<div class="main-content">main content</div>', u'<div class="blog-content">blog content</div>', u'<div class="content">content</div>'] >>> On Monday, March 2, 2015 at 6:04:02 PM UTC+1, [email protected] wrote: > > Is there any way to use the *contains *(or any other function) to extract > divs that have a class that contains a certain word? Something like: > > //div[contains(@class, "content")] #<-- not even sure that works, but > you get the idea,right? > > But it would capture classes like *main-content*, *body-content*, > *blog-content*, etc? I am looking for any kind of wildcard or > functionality beyond a simple match. Thank you. > > jeff. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
