I don't see the "return req". Is it there?
On Thu, Jan 16, 2014 at 8:39 PM, Justin Yek <[email protected]> wrote: > Hello, > > I'm trying to scrape an ASP.NET website. The steps are as follows: > > 1. Submit a form with stock code = '00001' at > http://www.hkexnews.hk/listedco/listconews/advancedsearch/search_active_main.aspx. > > 2. Scrape data in the page that returns > > The callback function after_post does not get called so I'm not sure if I > am setting up the FormRequest properly. Appreciate any help. > > --- > > class HKFilingSpider(BaseSpider): > name = "hkex_filing" > allowed_domains = ["http://www.hkexnews.hk/";] > start_urls = [" > http://www.hkexnews.hk/listedco/listconews/advancedsearch/search_active_main.aspx > "] > > def parse(self, response): > print "Submitting form..." > sel = Selector(response) > req = FormRequest.from_response(response, > formname='aspnetForm', > formdata={ > '__VIEWSTATEENCRYPTED':'', > 'ctl00$txt_today':'20140116', > 'ctl00$hfStatus':'AEM', > 'ctl00$hfAlert':'', > 'ctl00$txt_stock_code':'00001', > 'ctl00$txt_stock_name':'', > > 'ctl00$rdo_SelectDocType':'rbAll', > 'ctl00$sel_tier_1':'-2', > > 'ctl00$sel_DocTypePrior2006':'-1', > 'ctl00$sel_tier_2_group':'-2', > 'ctl00$sel_tier_2':'-2', > 'ctl00$ddlTierTwo':'23,1,3', > 'ctl00$ddlTierTwoGroup':'10,2', > 'ctl00$txtKeyWord':'', > > 'ctl00$rdo_SelectDateOfRelease':'rbManualRange', > > 'ctl00$sel_DateOfReleaseFrom_d':'01', > > 'ctl00$sel_DateOfReleaseFrom_m':'04', > > 'ctl00$sel_DateOfReleaseFrom_y':'1999', > > 'ctl00$sel_DateOfReleaseTo_d':'16', > > 'ctl00$sel_DateOfReleaseTo_m':'01', > > 'ctl00$sel_DateOfReleaseTo_y':'2014', > > 'ctl00$sel_defaultDateRange':'SevenDays', > > 'ctl00$rdo_SelectSortBy':'rbDateTime', > }, > callback=self.after_post) > > def after_post(self, response): > # does not get called > > -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/groups/opt_out.
