Re: Simulating a browser session
From: Brandon Cruz [EMAIL PROTECTED] Sent: Thursday, January 16, 2003 10:23 AM Subject: Simulating a browser session Does anyone have familiarity with using JAVA to simulate a browser session and navigate multiple pages in a https based web application, storing session cookies, doing form posts, etc.? Whenever wierd stuff like this starts happening, I've found that it's really helpful to have some kind of man in the middle processes to watch what's coming and going over the wire, like packet sniffers, or a logging proxy, etc. Anything that you can stick between a working browser and the actual application that can give insight to the comings and goings of the requests, and what's in the headers. Of course, then I realized that your going over SSL, which kind of makes that difficult to do. Any chance you can run over HTTP within a controlled environment? While it's nice to apply standards, and assume that things SHOULD work, they don't really help when what should work doesn't, and then it's all reverse engineering. I can't really give specifics, as a lot of it is platform dependent, and I've done a bunch of one off hacks and what not in the past. But, the most basic tool, if you can get this out of the SSL environment, would be to use a packet sniffer to watch a working transaction. These can reveal a lot of subtle details in these cases. Regards, Will Hartung ([EMAIL PROTECTED]) -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: Simulating a browser session
This might be of interest: http://www.innovation.ch/java/HTTPClient/ John -Original Message- From: Brandon Cruz [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 16, 2003 1:24 PM To: Tomcat Users List Subject: Simulating a browser session Does anyone have familiarity with using JAVA to simulate a browser session and navigate multiple pages in a https based web application, storing session cookies, doing form posts, etc.? We are trying to do an https post, but the application is acting as if the parameters are never passed. We have done the exact same thing before at another website with no problems. The only difference is that the one that does not work is using session cookies, although we are able to successfully get and set the cookies. In both situations, the web application we are trying to navigate is ASP based. We are trying to integrate one of our partners applications to ours and could really use any information, web pages to reference, anything. We're out of ideas. Thanks in advance and sorry for the off topic post, please reply to me directly. Brandon -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Simulating a browser session
Brandon, 9 times out of 10 it is my cookie, or ssl, or other non-complient logic in my code, but otherwise, this may be you, ... 1 times out of 10 when I have trouble interacting with a webapplication, it is caused by the useragent setting i pass. My steps to diagnose my problems are: 1: Try to work with the web app in internet explorer 5.5 2: trace interactions with site using cURL acting as IE5.5 3: trace interactions with site using cURL acting as the suspect agent my favorite usage is: curl -viLsA 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; .NET CLR 1.0.3705)' url -F for form values -b -c for cookies -k for ssl issues -u for user auth -o for output file ( i like the screen better ) On Thu, 16 Jan 2003, Brandon Cruz wrote: Does anyone have familiarity with using JAVA to simulate a browser session and navigate multiple pages in a https based web application, storing session cookies, doing form posts, etc.? We are trying to do an https post, but the application is acting as if the parameters are never passed. We have done the exact same thing before at another website with no problems. The only difference is that the one that does not work is using session cookies, although we are able to successfully get and set the cookies. In both situations, the web application we are trying to navigate is ASP based. We are trying to integrate one of our partners applications to ours and could really use any information, web pages to reference, anything. We're out of ideas. Thanks in advance and sorry for the off topic post, please reply to me directly. Brandon -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - - - Jason Pyeron http://www.pyerotechnics.com - - Owner Lead Pyerotechnics Development, Inc. - - +1 410 808 6646 (c) 500 West University Parkway #1S - - +1 410 467 2266 (f) Baltimore, Maryland 21210-3253 - - - -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, purge the message from your system and notify the sender immediately. Any other use of the email by you is prohibited. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Simulating a browser session
I like this HttpClient instead ... http://jakarta.apache.org/commons/httpclient/index.html -Tim Turner, John wrote: This might be of interest: http://www.innovation.ch/java/HTTPClient/ John -Original Message- From: Brandon Cruz [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 16, 2003 1:24 PM To: Tomcat Users List Subject: Simulating a browser session Does anyone have familiarity with using JAVA to simulate a browser session and navigate multiple pages in a https based web application, storing session cookies, doing form posts, etc.? We are trying to do an https post, but the application is acting as if the parameters are never passed. We have done the exact same thing before at another website with no problems. The only difference is that the one that does not work is using session cookies, although we are able to successfully get and set the cookies. In both situations, the web application we are trying to navigate is ASP based. We are trying to integrate one of our partners applications to ours and could really use any information, web pages to reference, anything. We're out of ideas. Thanks in advance and sorry for the off topic post, please reply to me directly. Brandon -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: Simulating a browser session
Nice. Someone at Jakarta needs to work on their search engine placement. ;) John -Original Message- From: Tim Funk [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 16, 2003 2:08 PM To: Tomcat Users List Subject: Re: Simulating a browser session I like this HttpClient instead ... http://jakarta.apache.org/commons/httpclient/index.html -Tim Turner, John wrote: This might be of interest: http://www.innovation.ch/java/HTTPClient/ John -Original Message- From: Brandon Cruz [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 16, 2003 1:24 PM To: Tomcat Users List Subject: Simulating a browser session Does anyone have familiarity with using JAVA to simulate a browser session and navigate multiple pages in a https based web application, storing session cookies, doing form posts, etc.? We are trying to do an https post, but the application is acting as if the parameters are never passed. We have done the exact same thing before at another website with no problems. The only difference is that the one that does not work is using session cookies, although we are able to successfully get and set the cookies. In both situations, the web application we are trying to navigate is ASP based. We are trying to integrate one of our partners applications to ours and could really use any information, web pages to reference, anything. We're out of ideas. Thanks in advance and sorry for the off topic post, please reply to me directly. Brandon -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: Simulating a browser session
Jason, Thanks for the info. What exactly is this cURL program? Is it something I can run from a linux command line or a program I can download? Anyway, I really appreciate the info! Brandon -Original Message- From: Jason Pyeron [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 16, 2003 12:30 PM To: [EMAIL PROTECTED] Subject: Re: Simulating a browser session Brandon, 9 times out of 10 it is my cookie, or ssl, or other non-complient logic in my code, but otherwise, this may be you, ... 1 times out of 10 when I have trouble interacting with a webapplication, it is caused by the useragent setting i pass. My steps to diagnose my problems are: 1: Try to work with the web app in internet explorer 5.5 2: trace interactions with site using cURL acting as IE5.5 3: trace interactions with site using cURL acting as the suspect agent my favorite usage is: curl -viLsA 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; .NET CLR 1.0.3705)' url -F for form values -b -c for cookies -k for ssl issues -u for user auth -o for output file ( i like the screen better ) On Thu, 16 Jan 2003, Brandon Cruz wrote: Does anyone have familiarity with using JAVA to simulate a browser session and navigate multiple pages in a https based web application, storing session cookies, doing form posts, etc.? We are trying to do an https post, but the application is acting as if the parameters are never passed. We have done the exact same thing before at another website with no problems. The only difference is that the one that does not work is using session cookies, although we are able to successfully get and set the cookies. In both situations, the web application we are trying to navigate is ASP based. We are trying to integrate one of our partners applications to ours and could really use any information, web pages to reference, anything. We're out of ideas. Thanks in advance and sorry for the off topic post, please reply to me directly. Brandon -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - - - Jason Pyeron http://www.pyerotechnics.com - - Owner Lead Pyerotechnics Development, Inc. - - +1 410 808 6646 (c) 500 West University Parkway #1S - - +1 410 467 2266 (f) Baltimore, Maryland 21210-3253 - - - -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, purge the message from your system and notify the sender immediately. Any other use of the email by you is prohibited. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Simulating a browser session
Brandon: curl is a command line utility similar to wget (which you probably already have). it's available for download on sourceforge.net. Michael Della Bitta Columbia University -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Simulating a browser session
From: Brandon Cruz [EMAIL PROTECTED] Sent: Thursday, January 16, 2003 11:52 AM Subject: RE: Simulating a browser session Thanks for the info. What exactly is this cURL program? Is it something I can run from a linux command line or a program I can download? Anyway, I really appreciate the info! It's like wget, a command line http client. Another thing to look at is perhaps tweaking the log files on the host (for apache, that would be access.log) to try and get request information out of it. But, either way, get the request details nailed down and the rest falls in place. Regards, Will Hartung ([EMAIL PROTECTED]) -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: Simulating a browser session
You could try using the HTTPClient library: http://www.innovation.ch/java/HTTPClient/ or you can try using HTTPUnit: http://httpunit.sourceforge.net/ Will. -Original Message- From: Jason Pyeron [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 16, 2003 10:30 AM To: [EMAIL PROTECTED] Subject: Re: Simulating a browser session Brandon, 9 times out of 10 it is my cookie, or ssl, or other non-complient logic in my code, but otherwise, this may be you, ... 1 times out of 10 when I have trouble interacting with a webapplication, it is caused by the useragent setting i pass. My steps to diagnose my problems are: 1: Try to work with the web app in internet explorer 5.5 2: trace interactions with site using cURL acting as IE5.5 3: trace interactions with site using cURL acting as the suspect agent my favorite usage is: curl -viLsA 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; .NET CLR 1.0.3705)' url -F for form values -b -c for cookies -k for ssl issues -u for user auth -o for output file ( i like the screen better ) On Thu, 16 Jan 2003, Brandon Cruz wrote: Does anyone have familiarity with using JAVA to simulate a browser session and navigate multiple pages in a https based web application, storing session cookies, doing form posts, etc.? We are trying to do an https post, but the application is acting as if the parameters are never passed. We have done the exact same thing before at another website with no problems. The only difference is that the one that does not work is using session cookies, although we are able to successfully get and set the cookies. In both situations, the web application we are trying to navigate is ASP based. We are trying to integrate one of our partners applications to ours and could really use any information, web pages to reference, anything. We're out of ideas. Thanks in advance and sorry for the off topic post, please reply to me directly. Brandon -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - - - Jason Pyeron http://www.pyerotechnics.com - - Owner Lead Pyerotechnics Development, Inc. - - +1 410 808 6646 (c) 500 West University Parkway #1S - - +1 410 467 2266 (f) Baltimore, Maryland 21210-3253 - - - -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, purge the message from your system and notify the sender immediately. Any other use of the email by you is prohibited. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: Simulating a browser session
cURL can be obtained from http://curl.haxx.se/ for windows, cygwin, linux, etc. -jason On Thu, 16 Jan 2003, Brandon Cruz wrote: Jason, Thanks for the info. What exactly is this cURL program? Is it something I can run from a linux command line or a program I can download? Anyway, I really appreciate the info! Brandon -Original Message- From: Jason Pyeron [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 16, 2003 12:30 PM To: [EMAIL PROTECTED] Subject: Re: Simulating a browser session Brandon, 9 times out of 10 it is my cookie, or ssl, or other non-complient logic in my code, but otherwise, this may be you, ... 1 times out of 10 when I have trouble interacting with a webapplication, it is caused by the useragent setting i pass. My steps to diagnose my problems are: 1: Try to work with the web app in internet explorer 5.5 2: trace interactions with site using cURL acting as IE5.5 3: trace interactions with site using cURL acting as the suspect agent my favorite usage is: curl -viLsA 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; .NET CLR 1.0.3705)' url -F for form values -b -c for cookies -k for ssl issues -u for user auth -o for output file ( i like the screen better ) On Thu, 16 Jan 2003, Brandon Cruz wrote: Does anyone have familiarity with using JAVA to simulate a browser session and navigate multiple pages in a https based web application, storing session cookies, doing form posts, etc.? We are trying to do an https post, but the application is acting as if the parameters are never passed. We have done the exact same thing before at another website with no problems. The only difference is that the one that does not work is using session cookies, although we are able to successfully get and set the cookies. In both situations, the web application we are trying to navigate is ASP based. We are trying to integrate one of our partners applications to ours and could really use any information, web pages to reference, anything. We're out of ideas. Thanks in advance and sorry for the off topic post, please reply to me directly. Brandon -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - - - Jason Pyeron http://www.pyerotechnics.com - - Owner Lead Pyerotechnics Development, Inc. - - +1 410 808 6646 (c) 500 West University Parkway #1S - - +1 410 467 2266 (f) Baltimore, Maryland 21210-3253 - - - -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, purge the message from your system and notify the sender immediately. Any other use of the email by you is prohibited. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - - - Jason Pyeron http://www.pyerotechnics.com - - Owner Lead Pyerotechnics Development, Inc. - - +1 410 808 6646 (c) 500 West University Parkway #1S - - +1 410 467 2266 (f) Baltimore, Maryland 21210-3253 - - - -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, purge the message from your system and notify the sender immediately. Any other use of the email by you is prohibited. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
