We do count the bytes we read from the inputstream but in our case we don't
get the HttpResponse back from the execute so we don't have a chance to
stop the execution while reading.
Am I missing something obvious about how to access the inputstream from the
entity?
Here's a quick sample of our usage in this case:
import java.io.IOException;
import java.io.InputStream;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
public class StreamTest {
public static void main(String[] args) {
HttpClient httpclient = new DefaultHttpClient();
HttpGet get = new HttpGet(
"http://scfire-dtc-aa04.stream.aol.com:80/stream/1030/7.html";);
try {
HttpResponse response = httpclient.execute(get); // does not return
InputStream stream = response.getEntity().getContent();
// read stream and count bytes, exit if we go above our max bytes
} catch (IOException e) {
e.printStackTrace();
}
}
}
On Sat, Mar 24, 2012 at 4:34 AM, Oleg Kalnichevski <[email protected]> wrote:
> On Fri, 2012-03-23 at 17:05 -0700, Josh Gordineer wrote:
> > I have a question about how to protect against requests to streamed
> > sources. A project background is that we allow execution on our servers
> of
> > outbound http requests based on user input. So in essence we need to
> > protect against abuse by adding restrictions on response size/time etc.
> We
> > have done this successfully in the past by adding a counter to the
> > inputstream to make sure data is below an arbitrary max size however, we
> > have come across a case when a user inputs (either incorrectly or
> > maliciously) tries to fetch an audio stream which causes our read thread
> to
> > continue endlessly reading the content (details of the actual feed and
> > stacktrace below). This request hangs getting the HttpResponse from the
> > httpclient.execute(request) call. Ideally we could read the response
> > header however since the client is hanging at the execute line I don't
> have
> > the handle to fetch the headers.
> >
> > Basically we need identify these requests and kill them prior to causing
> > our machine to churn reading data. My first idea is adding a hook in
> > httpclient to track the size of the request we are reading and kill it
> > after it gets past a threshold however it wasn't clear to me how to do so
> > with httpclient (I looked at creating a specialization of
> > DefaultResponseParser however I didn't see how to instantiate my
> version).
> >
> > Any suggestions are welcome. Thanks in advance!
> >
>
> Why can't you just read from the input stream, count bytes read, and
> abort the request in case the count exceeds a certain limit?
>
> Oleg
>
> > --Josh
> >
> > Detailed info:
> >
> > $ curl -v http://scfire-dtc-aa04.stream.aol.com:80/stream/1030/7.html
> > * About to connect() to scfire-dtc-aa04.stream.aol.com port 80 (#0)
> > * Trying 205.188.234.4... connected
> > * Connected to scfire-dtc-aa04.stream.aol.com (205.188.234.4) port 80
> (#0)
> > > GET /stream/1030/7.html HTTP/1.1
> > > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7
> > OpenSSL/0.9.8r zlib/1.2.3
> > > Host: scfire-dtc-aa04.stream.aol.com
> > > Accept: */*
> > >
> > ICY 200 OK
> > icy-notice1: <BR>This stream requires <a href="http://www.winamp.com/
> > ">Winamp</a><BR>
> > icy-notice2: Firehose Ultravox/SHOUTcast Relay Server/Linux v2.6.0<BR>
> > icy-name: RADIOUP.COM - THE HITLIST (formely 108.fm) - #1 FOR ALL HIT
> MUSIC
> > icy-genre: Top 40 Pop Rap Hip Hop Top40
> > icy-url: http://www.radioup.com/
> > content-type: audio/mpeg
> > icy-pub: 1
> > icy-br: 128
> >
> > Sstack dump for the read thread (versions httpclient-4.1.3.jar
> > httpcore-4.1.4.jar):
> >
> > "Instance-thread-1" prio=10 tid=0x89ae5c00 nid=0xe04 runnable
> [0x03785000]
> > java.lang.Thread.State: RUNNABLE
> > at java.net.SocketInputStream.socketRead0(Native Method)
> > at java.net.SocketInputStream.read(SocketInputStream.java:129)
> > at
> >
> org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
> > at
> >
> org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
> > at
> >
> org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264)
> > at
> >
> org.apache.http.impl.conn.LoggingSessionInputBuffer.readLine(LoggingSessionInputBuffer.java:115)
> > at
> >
> org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
> > at
> >
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
> > at
> >
> org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282)
> > at
> >
> org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
> > at
> >
> org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216)
> > at
> >
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
> > at
> >
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> > at
> >
> org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
> > at
> >
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
> > at
> >
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
> > at
> >
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
> > at
> >
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
> > <snip>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>
