I am writing a crawler using scrapy that logs in a php app and crawls it.
With the help of loginform <https://github.com/scrapy/loginform> utility, I
was able to log into 13 apps excluding a few like zencart, openemr, magneto
etc. I am describing below the problems I am facing with these apps. Any
suggestions/insight on any of them will be very helpful.
1) OpenEMR.
The login form is simple and was polluted by loginform utility. However, on
making a post request using FormRequest, the html response received
indicates that "OpenEMR needs Javascriot to perform user authentication".
Does this mean that the filling forms using FormRequest won't help me log
in the site ever?
2) Zencart
The login page has 2 forms (login and create account). The loginform
utility finds the correct form and populates it. However, the submit button
has no name so cannot be passed in the formdata (as done by loginform). To
side-step this, I set dont_click to true. But since there exists another
form on the page with compulsary fields, an error is returned ("error while
filling form"). The next thing I tried was using the clickdata attribute in
the form request. The submit button has the code <input alt="Sign In"
titlte = "Sign In" src="...." type="image">. On setting the
clickdata={"type":"image","alt":"Sign In"}, I get an error saying "no
clickable element matching {"type":"image","alt":"Sign In"} found". Is this
submit button not clickable then? If so, I don't know what to do next.
3) Magneto
The submit button is not inside <input> but inside <button>. I gave its id
in the clickdata but it doesn't work?
Any ideas will be helpful.
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
