Thanks Roland. I tried treating them as footers to no avail. I still don't get access to the actual body of the response.
Here's the wire log. As you can see from below, the "data" portion of the response (which should be considered the body) is being skipped over to get to the headers. The problem is that attempting to grab the data as headers, footers or body is not working. It's like the method is throwing those lines away (but the wire output thinks they are header rows). [INFO] DreamAPIRequest - -Extracting activity for campaign: test mailing: test01 [DEBUG] HttpClient - -Java version: 1.5.0_10 [DEBUG] HttpClient - -Java vendor: Sun Microsystems Inc. [DEBUG] HttpClient - -Java class path: C:\Documents and Settings\cmorris\workspace\AMI_2.0_Local\target;C:\Documents and Settings\cmorris\workspace\AMI_2.0_Local\target\classes;C:\Documents and Settings\cmorris\.m2\repository\commons-codec\commons-codec\1.2\commons- codec-1.2.jar;C:\Documents and Settings\cmorris\.m2\repository\commons-httpclient\commons-httpclient\3. 0.1\commons-httpclient-3.0.1.jar;C:\Documents and Settings\cmorris\.m2\repository\commons-io\commons-io\1.3.1\commons-io-1 .3.1.jar;C:\Documents and Settings\cmorris\.m2\repository\commons-logging\commons-logging\1.0.3\co mmons-logging-1.0.3.jar;C:\Documents and Settings\cmorris\.m2\repository\commons-vfs\commons-vfs\1.0\commons-vfs- 1.0.jar;C:\Documents and Settings\cmorris\.m2\repository\jdom\jdom\1.0\jdom-1.0.jar;C:\Documents and Settings\cmorris\.m2\repository\junit\junit\3.8.1\junit-3.8.1.jar;C:\Doc uments and Settings\cmorris\workspace\AMI_2.0_Local\jar\commons-httpclient-3.0.1.ja r;C:\Documents and Settings\cmorris\workspace\AMI_2.0_Local\jar\commons-logging-1.0.3.jar;C :\Documents and Settings\cmorris\workspace\AMI_2.0_Local\jar\commons-vfs-20070719.jar;C: \Documents and Settings\cmorris\workspace\AMI_2.0_Local\jar\jdom-1.0.jar;C:\Documents and Settings\cmorris\workspace\AMI_2.0_Local\jar\xerces.jar;C:\Documents and Settings\cmorris\workspace\AMI_2.0_Local\jar\commons-codec-1.2.jar;C:\Do cuments and Settings\cmorris\workspace\AMI_2.0_Local\jar\commons-net-1.4.1.jar;C:\Do cuments and Settings\cmorris\workspace\AMI_2.0_Local\jar\jsch-0.1.32.jar [DEBUG] HttpClient - -Operating system name: Windows XP [DEBUG] HttpClient - -Operating system architecture: x86 [DEBUG] HttpClient - -Operating system version: 5.1 [DEBUG] HttpClient - -SUN 1.5: SUN (DSA key/parameter generation; DSA signing; SHA-1, MD5 digests; SecureRandom; X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX CertPathBuilder; LDAP, Collection CertStores) [DEBUG] HttpClient - -SunRsaSign 1.5: Sun RSA signature provider [DEBUG] HttpClient - -SunJSSE 1.5: Sun JSSE provider(PKCS12, SunX509 key/trust factories, SSLv3, TLSv1) [DEBUG] HttpClient - -SunJCE 1.5: SunJCE Provider (implements RSA, DES, Triple DES, AES, Blowfish, ARCFOUR, RC2, PBE, Diffie-Hellman, HMAC) [DEBUG] HttpClient - -SunJGSS 1.0: Sun (Kerberos v5) [DEBUG] HttpClient - -SunSASL 1.5: Sun SASL provider(implements client mechanisms for: DIGEST-MD5, GSSAPI, EXTERNAL, PLAIN, CRAM-MD5; server mechanisms for: DIGEST-MD5, GSSAPI, CRAM-MD5) [DEBUG] DefaultHttpParams - -Set parameter http.useragent = Jakarta Commons-HttpClient/3.0.1 [DEBUG] DefaultHttpParams - -Set parameter http.protocol.version = HTTP/1.1 [DEBUG] DefaultHttpParams - -Set parameter http.connection-manager.class = class org.apache.commons.httpclient.SimpleHttpConnectionManager [DEBUG] DefaultHttpParams - -Set parameter http.protocol.cookie-policy = rfc2109 [DEBUG] DefaultHttpParams - -Set parameter http.protocol.element-charset = US-ASCII [DEBUG] DefaultHttpParams - -Set parameter http.protocol.content-charset = ISO-8859-1 [DEBUG] DefaultHttpParams - -Set parameter http.method.retry-handler = [EMAIL PROTECTED] [DEBUG] DefaultHttpParams - -Set parameter http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE, dd-MMM-yy HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy HH:mm:ss z, EEE, dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy HH:mm:ss z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE dd-MMM-yy HH:mm:ss z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z, EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy HH:mm:ss z] [INFO] HttpAPIRequest - -postRequest posting to: http://dweb.bfi0.com:9000/api/ActivityDownload.mpl [INFO] HttpAPIRequest - -HTTP Post contains the following parameters: [INFO] HttpAPIRequest - -parameter: headers_first = 0 [INFO] HttpAPIRequest - -parameter: client_name = xxxxxxxxxxxx [INFO] HttpAPIRequest - -parameter: client_token = xxxxxxxxxxxxx [INFO] HttpAPIRequest - -parameter: output_field_delimiter = || [INFO] HttpAPIRequest - -parameter: campaign_name = test [INFO] HttpAPIRequest - -parameter: mailing_name = test01 [INFO] HttpAPIRequest - -parameter: start_date = 20070901 [INFO] HttpAPIRequest - -parameter: end_date = 20070930 [INFO] HttpAPIRequest - -parameter: ret_mod_name = Y [INFO] HttpAPIRequest - -parameter: ret_link_name = Y [INFO] HttpAPIRequest - -parameter: profile_field_names = CUST_ID [DEBUG] HttpConnection - -Open connection to xxxx.xxxx.xxx:9000 [DEBUG] header - ->> "POST /api/ActivityDownload.mpl HTTP/1.1[\r][\n]" [DEBUG] HttpMethodBase - -Adding Host request header [DEBUG] HttpMethodBase - -Default charset used: ISO-8859-1 [DEBUG] HttpMethodBase - -Default charset used: ISO-8859-1 [DEBUG] header - ->> "User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7[\r][\n]" [DEBUG] header - ->> "Keep-Alive: 300[\r][\n]" [DEBUG] header - ->> "Connection: keep-alive[\r][\n]" [DEBUG] header - ->> "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plai n;q=0.8,image/png,*/*;q=0.5[\r][\n]" [DEBUG] header - ->> "Accept-Language: en-us,en;q=0.5[\r][\n]" [DEBUG] header - ->> "Accept-Encoding: gzip,deflate[\r][\n]" [DEBUG] header - ->> "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7[\r][\n]" [DEBUG] header - ->> "Host: xxxx.xxx.xxx:9000[\r][\n]" [DEBUG] header - ->> "Content-Length: 238[\r][\n]" [DEBUG] header - ->> "Content-Type: application/x-www-form-urlencoded[\r][\n]" [DEBUG] header - ->> "[\r][\n]" [DEBUG] EntityEnclosingMethod - -Request body sent [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||1||20070 911 111040||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||1||20070 911 111602||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||0||20070 911 111602||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||0||20070 911 111602||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||0||20070 911 111603||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||SEND||H||S||1||||||||1||2007091 1 110746||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||SEND||H||S||1||||||||1||2007091 1 110748||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||SEND||H||S||1||||||||1||2007091 1 110749||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||SEND||H||S||1||||||||1||2007091 1 110751||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||SEND||H||S||1||||||||1||2007091 1 110752||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||R||280988||http://www. epsiloninteractive.com||TOPLEVEL||test_linkname_html||1||20070911 111120||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||0||20070 912 152737||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||R||280988||http://www. epsiloninteractive.com||TOPLEVEL||test_linkname_html||0||20070912 152742||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||1||20070 913 221549||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||R||280988||http://www. epsiloninteractive.com||TOPLEVEL||test_linkname_html||1||20070913 221553||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||R||280988||http://www. epsiloninteractive.com||TOPLEVEL||test_linkname_html||0||20070913 221602||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||0||20070 914 102710||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||0||20070 918 165151||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||1||20070 918 170651||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||0||20070 918 170651||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||1||20070 918 172650||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||0||20070 918 172650||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||0||20070 918 172743||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||T||C||DEF||||||||1||20070 918 174925||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||T||O||DEF||||||||1||20070 918 175044||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||SEND||H||S||1||||||||0||2007091 8 170117||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||SEND||H||S||1||||||||0||2007091 8 170122||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||SEND||H||S||1||||||||0||2007091 8 170124||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||SEND||H||S||1||||||||0||2007091 8 170125||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||SEND||H||S||1||||||||0||2007091 8 170126||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||0||20070 918 190149||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||R||280988||http://www. epsiloninteractive.com||TOPLEVEL||test_linkname_html||0||20070918 190154||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||R||331121||http://www. epsiloninteractive.com||TEST_MODULE||test_module_link_name_html||1||2007 0918 190158||[\r][\n]" [DEBUG] header - -<< "test01||[EMAIL PROTECTED]||[EMAIL PROTECTED]||HVWB||H||G||DEF||||||||0||20070 919 113605||[\r][\n]" [DEBUG] header - -<< "HTTP/1.1 200 OK[\r][\n]" [DEBUG] header - -<< "Date: Thu, 27 Sep 2007 15:38:28 GMT[\r][\n]" [DEBUG] header - -<< "Server: Apache[\r][\n]" [DEBUG] header - -<< "BFI-Status: OK[\r][\n]" [DEBUG] header - -<< "BFI-Status-String: [\r][\n]" [DEBUG] header - -<< "BFI-Number-Records: 34[\r][\n]" [DEBUG] header - -<< "Connection: close[\r][\n]" [DEBUG] header - -<< "Transfer-Encoding: chunked[\r][\n]" [DEBUG] header - -<< "Content-Type: text/plain[\r][\n]" [WARN] HttpMethodBase - -Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.test|| [DEBUG] HttpMethodBase - -Buffering response body [DEBUG] HttpMethodBase - -Should close connection in response to directive: close [DEBUG] HttpConnection - -Releasing connection back to connection manager. [INFO] HttpAPIRequest - -postRequest received the following response: HTTP/1.1 200 OK [DEBUG] HttpAPIRequest - -response header: Date: Thu, 27 Sep 2007 15:38:28 GMT [DEBUG] HttpAPIRequest - -response header: Server: Apache [DEBUG] HttpAPIRequest - -response header: BFI-Status: OK [DEBUG] HttpAPIRequest - -response header: BFI-Status-String: [DEBUG] HttpAPIRequest - -response header: BFI-Number-Records: 34 [DEBUG] HttpAPIRequest - -response header: Connection: close [DEBUG] HttpAPIRequest - -response header: Transfer-Encoding: chunked [DEBUG] HttpAPIRequest - -response header: Content-Type: text/plain [DEBUG] HttpAPIRequest - -response body: [INFO] DreamAPIRequest - -BFI-Status: OK [INFO] DreamAPIRequest - -BFI-Status-String: [INFO] DreamAPIRequest - -BFI-Number-Records: 34 [INFO] FileUtility - -Writing string buffer to File: email_response.datAppend Flag: false Chris Morris Sorry about the footer... I have no control over what the exchange server appends to my messages on my behalf... -----Original Message----- From: Roland Weber [mailto:[EMAIL PROTECTED] Sent: Thursday, September 27, 2007 9:33 AM To: HttpClient User Discussion Subject: Re: HTTP Response contains headers last Morris, Chris wrote: > I'm working with an API that returns the HTTP Response Headers after the > response body. This confuses the HTTPClient and PostMethod into > thinking that there is no body. We have code to handle footers in ChunkedInputStream. However, this is the first time that I encounter footers in the wild, so maybe that code is buggy. Could you please post a wire log, including headers and content? Feel free to obfuscate any sensitive information and cut the contents. http://jakarta.apache.org/httpcomponents/httpclient-3.x/logging.html > Is there a way for me to get direct > access to the response input stream so that I can parse it out myself? You can hack through the code of course. The socket's stream is not accessible directly, because the ChunkedInputStream has to parse the "chunked" encoding. Trailers are only allowed with chunked encoding. If you have to hack through the code anyway, you should rather fix what is broken than work around what is already there. > The information contained in this communication is > confidential, and is intended only for the sole use of > the recipient named above. [...] Always funny to read these footers on mails sent to public mailing lists... cheers, Roland --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] The information contained in this communication is confidential, and is intended only for the sole use of the recipient named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please re-send this communication to the sender and delete the original message or any copy of it from your computer system. Thank you. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
