Hi Valdir,
The code that you provided works great in shell when testing, but it
doesn’t work when I run the actual spider, here’s part of spider code I've
implemented after your suggestion:
```python
def parse_app_lists(self, response):
"""Returns page with 42 days results instead of default 7 days."""
yield FormRequest.from_response(response,
formdata={'RdoTimeLimit': '42'},
dont_filter=True,
callback=self.parse_page)
def parse_page(self, response):
open_in_browser(response)
```
The output that gets opened in Firefox looks like this:
So basically it doesn’t get loaded for some reason.
I’ve also set the ```DOWNLOAD_DELAY=2``` so forms can get loaded properly.
четвртак, 24. децембар 2015. 14.53.47 UTC+1, Valdir Stumm Junior је
написао/ла:
>
> Hey! I think that FormRequest is using the first form on the page (that
> text field from the top). You can tell FormRequest to use the second form
> through an xpath expression, like this:
>
> fr = FormRequest.from_response(response,
> formdata={'RdoTimeLimit': '42'},
> dont_filter=True, formxpath="(//form)[2]")
>
> On Thu, Dec 24, 2015 at 11:13 AM, Mario <laki-paki...@hotmail.com
> <javascript:>> wrote:
>
>> Hi guys,
>>
>> Here’s what I’m trying to accomplish:
>>
>> [image: ss]
>>
>> This is url: http://www.eplanning.ie/CarlowCC/SearchListing/RECEIVED
>>
>> Step 1(from image): Select 42 days instead of 7 days default.
>>
>> Step 2(from image): Click on search button for results.
>>
>> I realize that this is possible with Selenium but I’m almost certain that
>> this can be done with Scrapy by setting proper form/s.
>>
>> I’ve tried mimicking calls in scrapy shell, here’s just one of many calls
>> that I used for selecting 42 days limit:
>>
>> scrapy shell 'http://www.eplanning.ie/CarlowCC/SearchListing/RECEIVED'
>>
>> from scrapy.http import FormRequest
>>
>> fr = FormRequest.from_response(response,
>> formdata={'RdoTimeLimit': '42'},
>> dont_filter=True)
>>
>> fetch(fr)
>>
>> view(response)
>>
>> Here’s the output:
>>
>> [image: ss]
>>
>> Thanks in advance for any help, and merry Christmas everybody!
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "scrapy-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to scrapy-users...@googlegroups.com <javascript:>.
>> To post to this group, send email to scrapy...@googlegroups.com
>> <javascript:>.
>> Visit this group at https://groups.google.com/group/scrapy-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> [image: Scrapinghub] <https://scrapinghub.com>
>
> Valdir Stumm Junior
> Developer Evangelist, Scrapinghub
> [image: Skype]stummjr[image: Twitter] <https://twitter.com/stummjr>[image:
> Github] <https://github.com/stummjr>[image: Twitter]
> <https://twitter.com/scrapinghub>[image: LinkedIn]
> <https://www.linkedin.com/company/scrapinghub>[image: Github]
> <https://github.com/scrapinghub>
>
> *We turn web content into structured data. Lead maintainers of Scrapy
> <http://scrapy.org>.*
>
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to scrapy-users+unsubscr...@googlegroups.com.
To post to this group, send email to scrapy-users@googlegroups.com.
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
