[
https://issues.apache.org/jira/browse/HTTPCORE-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16182687#comment-16182687
]
Michael Heemskerk commented on HTTPCORE-491:
--------------------------------------------
bq. Should this setting be configurable?
Not sure; it's really only meant to protect against OOMEs. If the 256kb seems a
bit on the high size, I can go with a smaller default initial buffer size. Note
that the buffer will still grow unrestricted.
bq. Should there be an upper limit as well?
It's hard to pick an arbitrary number that would be a safe upper limit (in that
it won't break someone's use case). Which means you'd have to make the upper
limit configurable. However, since you can pass any
{{HttpAsyncResponseConsumer}}, I'm not convinced adding a {{maxEntitySize}}
config option somewhere makes sense.
If you wanted to go with a bounded response consumer, you could go with your
own implementation.
Here's the one I ended up with:
{code:java}
public class BoundedAsyncResponseConsumer extends
AbstractAsyncResponseConsumer<HttpResponse> {
// limit the amount of memory that is pre-allocated based on the reported
Content-Length. Let's be a bit paranoid
private static final int MAX_INITIAL_BUFFER_SIZE = 256 * 1024;
private final int maxEntitySize;
private volatile BoundedInputBuffer buf;
private volatile HttpResponse response;
BoundedAsyncResponseConsumer(int maxEntitySize) {
this.maxEntitySize = maxEntitySize;
}
protected HttpResponse buildResult(HttpContext context) {
return response;
}
protected void onContentReceived(ContentDecoder decoder, IOControl ioctrl)
throws IOException {
Asserts.notNull(buf, "Content buffer");
try {
buf.consumeContent(decoder);
} catch (BufferFullException e) {
throw new EntityTooLargeException(response,
"Entity content is too long; larger than " + maxEntitySize
+ " bytes");
}
}
protected void onEntityEnclosed(HttpEntity entity, ContentType contentType)
throws IOException {
int length = Math.min(Ints.saturatedCast(entity.getContentLength()),
maxEntitySize);
if (length < 0L) {
// start with a 4k buffer
length = Math.min(4096, maxEntitySize);
}
int initialBufferSize = Math.min(MAX_INITIAL_BUFFER_SIZE, length);
buf = new BoundedInputBuffer(initialBufferSize, maxEntitySize, new
HeapByteBufferAllocator());
Asserts.notNull(response, "response");
response.setEntity(new ContentBufferEntity(entity, buf));
}
protected void onResponseReceived(HttpResponse response) throws IOException
{
this.response = response;
}
protected void releaseResources() {
this.response = null;
this.buf = null;
}
private static class BoundedInputBuffer extends SimpleInputBuffer {
private final int maxSize;
BoundedInputBuffer(int initialSize, int maxSize, ByteBufferAllocator
allocator) {
super(Math.min(maxSize, initialSize), allocator);
this.maxSize = maxSize;
}
@Override
protected void expand() {
int capacity = buffer.capacity();
int newCapacity = capacity < 2 ? 2 : capacity + (capacity >>> 1);
if (newCapacity < capacity) {
// must be integer overflow
newCapacity = Integer.MAX_VALUE;
}
ensureCapacity(newCapacity);
}
@Override
protected void ensureCapacity(int requiredCapacity) {
if (buffer.capacity() == maxSize && requiredCapacity > maxSize) {
throw new BufferFullException();
}
super.ensureCapacity(Math.min(requiredCapacity, maxSize));
}
}
private static class BufferFullException extends RuntimeException {
}
}
{code}
> BasicAsyncResponseConsumer can easily be tricked into triggering an OOME
> ------------------------------------------------------------------------
>
> Key: HTTPCORE-491
> URL: https://issues.apache.org/jira/browse/HTTPCORE-491
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore NIO
> Affects Versions: 4.4.7
> Reporter: Michael Heemskerk
> Fix For: 4.4.8
>
>
> When using {{BasicAsyncResponseConsumer}} to consume a response, the consumer
> initializes its {{SimpleInputBuffer}} with the value reported on the
> response's {{Content-Length}} header.
> It's easy to spoof a response with a very large (but smaller than
> Integer.MAX_VALUE) {{Content-Length}} header and have the client pre-allocate
> a massive buffer, triggering an OOME.
> Since {{SimpleInputBuffer}} already expands-on-demand, it would be trivial to
> cap the initial buffer size to some reasonable limit (256k or even 1M)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
