Aha, since the response isn't a valid HTTP message (ICY 200 OK) the DefaultResponseParser loops looking for a valid protocol which will never come. Setting the maxGarbageLines in my request fixed the issue.
Thanks! --Josh On Sun, Mar 25, 2012 at 7:04 AM, Oleg Kalnichevski <[email protected]> wrote: > On Sat, 2012-03-24 at 17:15 -0700, Josh Gordineer wrote: > > We do count the bytes we read from the inputstream but in our case we > don't > > get the HttpResponse back from the execute so we don't have a chance to > > stop the execution while reading. > > > > HttpClient returns a HttpResponse object as soon as it receives an HTTP > message head consisting of a status line and response headers. The > message body is not processed inside the #execute method. If HttpClient > does not return from the method, it means the server has sent no > response at all. > > Besides, one can abort the request at any point of its execution by > using HttpUriRequest#abort method. > > Hope this helps > > Oleg > > > Am I missing something obvious about how to access the inputstream from > the > > entity? > > > > Here's a quick sample of our usage in this case: > > > > import java.io.IOException; > > import java.io.InputStream; > > > > import org.apache.http.HttpResponse; > > import org.apache.http.client.HttpClient; > > import org.apache.http.client.methods.HttpGet; > > import org.apache.http.impl.client.DefaultHttpClient; > > > > public class StreamTest { > > > > public static void main(String[] args) { > > HttpClient httpclient = new DefaultHttpClient(); > > HttpGet get = new HttpGet( > > "http://scfire-dtc-aa04.stream.aol.com:80/stream/1030/7.html";); > > try { > > HttpResponse response = httpclient.execute(get); // does not return > > InputStream stream = response.getEntity().getContent(); > > // read stream and count bytes, exit if we go above our max bytes > > } catch (IOException e) { > > e.printStackTrace(); > > } > > } > > } > > > > > > > > > > On Sat, Mar 24, 2012 at 4:34 AM, Oleg Kalnichevski <[email protected]> > wrote: > > > > > On Fri, 2012-03-23 at 17:05 -0700, Josh Gordineer wrote: > > > > I have a question about how to protect against requests to streamed > > > > sources. A project background is that we allow execution on our > servers > > > of > > > > outbound http requests based on user input. So in essence we need to > > > > protect against abuse by adding restrictions on response size/time > etc. > > > We > > > > have done this successfully in the past by adding a counter to the > > > > inputstream to make sure data is below an arbitrary max size > however, we > > > > have come across a case when a user inputs (either incorrectly or > > > > maliciously) tries to fetch an audio stream which causes our read > thread > > > to > > > > continue endlessly reading the content (details of the actual feed > and > > > > stacktrace below). This request hangs getting the HttpResponse from > the > > > > httpclient.execute(request) call. Ideally we could read the response > > > > header however since the client is hanging at the execute line I > don't > > > have > > > > the handle to fetch the headers. > > > > > > > > Basically we need identify these requests and kill them prior to > causing > > > > our machine to churn reading data. My first idea is adding a hook in > > > > httpclient to track the size of the request we are reading and kill > it > > > > after it gets past a threshold however it wasn't clear to me how to > do so > > > > with httpclient (I looked at creating a specialization of > > > > DefaultResponseParser however I didn't see how to instantiate my > > > version). > > > > > > > > Any suggestions are welcome. Thanks in advance! > > > > > > > > > > Why can't you just read from the input stream, count bytes read, and > > > abort the request in case the count exceeds a certain limit? > > > > > > Oleg > > > > > > > --Josh > > > > > > > > Detailed info: > > > > > > > > $ curl -v > http://scfire-dtc-aa04.stream.aol.com:80/stream/1030/7.html > > > > * About to connect() to scfire-dtc-aa04.stream.aol.com port 80 (#0) > > > > * Trying 205.188.234.4... connected > > > > * Connected to scfire-dtc-aa04.stream.aol.com (205.188.234.4) port > 80 > > > (#0) > > > > > GET /stream/1030/7.html HTTP/1.1 > > > > > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 > > > > OpenSSL/0.9.8r zlib/1.2.3 > > > > > Host: scfire-dtc-aa04.stream.aol.com > > > > > Accept: */* > > > > > > > > > ICY 200 OK > > > > icy-notice1: <BR>This stream requires <a href=" > http://www.winamp.com/ > > > > ">Winamp</a><BR> > > > > icy-notice2: Firehose Ultravox/SHOUTcast Relay Server/Linux > v2.6.0<BR> > > > > icy-name: RADIOUP.COM - THE HITLIST (formely 108.fm) - #1 FOR ALL > HIT > > > MUSIC > > > > icy-genre: Top 40 Pop Rap Hip Hop Top40 > > > > icy-url: http://www.radioup.com/ > > > > content-type: audio/mpeg > > > > icy-pub: 1 > > > > icy-br: 128 > > > > > > > > Sstack dump for the read thread (versions httpclient-4.1.3.jar > > > > httpcore-4.1.4.jar): > > > > > > > > "Instance-thread-1" prio=10 tid=0x89ae5c00 nid=0xe04 runnable > > > [0x03785000] > > > > java.lang.Thread.State: RUNNABLE > > > > at java.net.SocketInputStream.socketRead0(Native Method) > > > > at java.net.SocketInputStream.read(SocketInputStream.java:129) > > > > at > > > > > > > > org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149) > > > > at > > > > > > > > org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111) > > > > at > > > > > > > > org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264) > > > > at > > > > > > > > org.apache.http.impl.conn.LoggingSessionInputBuffer.readLine(LoggingSessionInputBuffer.java:115) > > > > at > > > > > > > > org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98) > > > > at > > > > > > > > org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252) > > > > at > > > > > > > > org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282) > > > > at > > > > > > > > org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247) > > > > at > > > > > > > > org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216) > > > > at > > > > > > > > org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298) > > > > at > > > > > > > > org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) > > > > at > > > > > > > > org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647) > > > > at > > > > > > > > org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464) > > > > at > > > > > > > > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820) > > > > at > > > > > > > > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754) > > > > at > > > > > > > > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732) > > > > <snip> > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
