I concur with the comments below; I've run into very similar sorts of
things. A couple of comments:
(1) Using proxomitron -- this is a great tool, and the SSL support does
work, for the most part. I have, however seen situations where the
leading "HTTP" on the HTTP response status line seems to get dropped
when proxomitron is in the middle. This tends to confuse the browser,
and causes other side-effects.
(2) To help with the hidden field & click coordinates issues, I wrote a
simple LWP-based character-mode browser, with SSL support. It uses
HTML::Form to generate well-formed (no pun intended) posts, including
button click coordinates of {1,1}. As it goes along, it logs all the
requests to a file for later re-use. It uses readline support to prompt
the user to pick from multiple forms and/or hyperlinks on a page, and to
allow the user to fill in form field values.
(3) I've encountered multiple types of "re-directs" that need to be
handled, all within the same application(!):
(a) std. HTTP redirects, which can be handled by LWP as detailed below
(b) "meta HTTP-equiv refresh" tags in the HTML <head> section.
(c) javasript "location=" commands
While probably not correct for every situation, I have logic in my
"browser" that looks for (b) & (c) types & automatically "follows" these
redirects.
-David
[EMAIL PROTECTED]
> -----Original Message-----
> From: Doug Monroe [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, March 14, 2002 9:18 AM
> To: [EMAIL PROTECTED]
> Cc: Steve Sapovits; Vicki Brown
> Subject: Re: Using LWP to talk to Javascript?
>
> All great points! Thanks Steve!
> Below, I've included a few followup comments to this great summary.
> --
> Doug Monroe
>
> Steve Sapovits wrote:
> >
> > Vicki Brown wrote:
> >
> > > For my current contract we have a CGI on server A that users can
log
> into
> > > directly. It controls their experience.
> > >
> > > We have a second CGI on server B, behind the firewall. In case
anyone
> is
> > > familiar with it, CGI B is a forum/discussion app called "Web
> Crossing".
> > >
> > > I have been asked (if I can) to set things up so that CGI A
controls
> CGI B
> > > remotely. The user will think they are interacting with
CGI/server A
> at all
> > > times.
> > >
> > > I am using libWWW to pass requests back and forth, using
> LWP::UserAgent.
> > > Everything works spiffily ... except if CGI B wants to do
something
> with
> > > Javascript ...
> >
> > JavaScript is by default run in the browser. Since LWP::UserAgent
is
> > emulating the browser, it would have to understand JavaScript to do
> > anything meaningful with the JavaScript it receives from any CGI.
> >
> > For JavaScript that is just set up to invoke a server side URL (your
> > first example) you may get away with what you need: Recognize the
> > onClick action, and have LWP::UserAgent invoke the URL. If what
you're
> > seeing in LWP as a response is different than what you see as a
browser,
> > there are several considerations to think about:
> >
> > (1) What you're "seeing" in the GUI browser may be the result of
> JavaScript
> > being run there on page receipt. You need to look first at
HTTP
> level
> > data and see if your responses are identical.
> >
> > (2) The server may be serving a different result to LWP because it
knows
> > enough to think it doesn't support JavaScript (maybe something
> silly
> > like looking at a user agent header, etc.).
> >
> > (3) The browser may be carrying cookies around that you're not
carrying
> around
> > in your LWP application.
> >
> > (4) The browser may be carrying around hidden fields that you're not
in
> > recognizing and setting in your LWP application.
> >
> > Outside of Javascript, the bulk of my problems fall into the #3 and
#4
> areas.
> >
> > A good place to start is comparing HTTP headers going back and forth
in
> each
> > case.
>
> bit of a braindump follows...FWIW
>
> TIP#1
> I'll echo support to the value of examining headers. At times with JS
in
> the
> mix, you need to see exactly what URL is being posted to. I've found a
> convenient tool to examine headers passed by "real" browsers (in order
to
> better emulate same with perl) for the wintel side would be
proxomitron
> (proxomitron.org). FWIW- proxomitron v4.1 now allows examination of
HTTPS
> traffic (with some added SSL dll's). Users on wintel and *nix side
could
> also
> use ethereal/tethereal (www.ethereal.com) to snoop HTTP/HTTPS packets
>
> TIP#2
> RE: "hidden" fields
> Along with "regular" hidden fields as mentioned by Steve above, I
found
> one
> app I was trying to get working with, shall we say "perl middleware",
had
> to
> do with JS reformatting input data into a totally different formfield
> name/value pair passed to the server. Example:
> if (document.x.tmpDOB.value.length != 10)
> {
> alert("Please enter Date of Birth as
MM/DD/YYYY");
> document.x.tmpDOB.focus(); return false;
> } else {
> var yyyy =
> document.x.tmpDOB.value.substring(6,10);
> var mm =
document.x.tmpDOB.value.substring(0,2);
> var dd =
document.x.tmpDOB.value.substring(3,5);
> document.x.DOB.value = yyyy + mm + dd;
> }
> in this case, the tmpDOB was -entered- by user as MM/DD/YYYY, but the
> server
> apparently wants it as YYMMDD, so they used JS to do that transform
and
> pass
> the value as DOB (vs. tmpDOB). You must emulate that by passing the
data
> in
> the way the JS would be doing it.
>
> TIP#3
> Not strictly JS-related but I've run into things like using images for
> "submit" buttons. In some cases, the server expects to see x/y
> coordinates
> passed from the mouse "click" on the image %-}
> So you must pass name/value pairs for that like-
> image.x=13&image.y=18
>
> semi-related...
> <INPUT TYPE="SUBMIT" NAME="search" VALUE="Search">
> Sometimes expects the submit button name/value to be passed:
> search=Search
>
> In -some- cases....the server owners went to great lengths to only
-show-
> submit buttons -if- JS is on:
> <SCRIPT LANGUAGE="JavaScript">
> document.write("<input type=\"image\" src=\"/images/submit.gif\"
> alt=\"Login\"
> id=\"img1\" name=\"img1\"> " +
> "<input type=\"image\" src=\"/images/clear.gif\" onClick=\"return
> Clear()\"
> alt=\"Reset\" id=img2 name=img2>");
> </SCRIPT>
>
> Again, not strictly JS-related but:
> TIP#4
> Sometimes you -need- to pass HTTP_REFERER data, e.g.:
> $request->referer('http://www.example.com/logon.html');
>
> TIP#5
> # use as needed to show/debug header info
> use LWP::Debug qw(+);
>
> TIP#6
> Sometimes you need to follow redirects:
> # use as needed to allow LWP to follow redirects
> BEGIN { *LWP::UserAgent::redirect_ok = sub { 1 } }
>
> Point in all this is...LWP won't magically interpret JS for you, so
-you-
> (the
> programmer) must examine the -source- of the URL you're trying to
eumlate
> a
> POST/GET -from-. Keep in mind that -includes- carefully examining
source
> that
> is called into an HTML page like:
> <SCRIPT LANGUAGE="JavaScript" SRC="/scripts/filename.js">
>
> Security side effect: these comments should serve to reinforce the
notion
> that
> we -all- need to "defensively" program our server-side apps. Do NOT
assume
> a
> well behaved "legit" browser is all you will ever encounter "in the
wild".
> Two simple but critical points being:
> - NEVER blindly trust user/form input
> - NEVER assume user/form input has passed thru your JS validation
> routines
> There are people like us "out there" ;)
