Hi Theodis,
you can do
scrapy crawl spidername -a username=john -a password=secret
and then
class LoginSpider(Spider):
name = 'example.com'
start_urls = ['http://www.example.com/users/login.php']
def parse(self, response):
return [FormRequest.from_response(response,
formdata={'username': self.username, 'password':
self.password},
callback=self.after_login)]
def after_login(self, response):
# check login succeed before going on
if "authentication failed" in response.body:
self.log("Login failed", level=log.ERROR)
return
# continue scraping with authenticated session...
Hope it helps.
/Paul.
On Monday, January 13, 2014 9:53:11 PM UTC+1, Theodis Butler wrote:
>
> How would I pass username and password from the command line? Sorry not an
> expert python programmer. Any help is greatly appreciated!
>
> Could I pass the arguments like: scrapy crawl spidername -a
> username=john,password=secret ??
>
>
> class LoginSpider(Spider):
> name = 'example.com'
> start_urls = ['http://www.example.com/users/login.php']
>
> def parse(self, response):
> return [FormRequest.from_response(response,
> formdata={'username': 'john', 'password': 'secret'},
> callback=self.after_login)]
>
> def after_login(self, response):
> # check login succeed before going on
> if "authentication failed" in response.body:
> self.log("Login failed", level=log.ERROR)
> return
>
> # continue scraping with authenticated session...
>
>
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/groups/opt_out.
