I've basically replaced the use of byte array (byte[]) and
ByteArrayOutputStream class with a collection of non-direct (or direct,
whichever you prefer) ByteArray objects and created a pool mechanism for
the ByteArray objects. All read and write operations on the protocol are
directly attached to ByteArray instead of using temporary byte array
and/or ByteArrayOutputStream. Hence, avoiding excessive 'buffer-copy'
and incremental garbage collection overheads.
This is still in testing stage. Especially how it performs when using
Servlet-based implementation where it uses OutputStream/InputStream
interface for communicating with the underlying communication socket.
On 26-10-10 00:41, Bryan Duxbury wrote:
I'd love to hear your ideas for reducing the memory overhead of thrift
objects.
On Mon, Oct 25, 2010 at 10:36 AM, Andeka Putra<andeka.pu...@gmail.com>wrote:
I'm trying to. The problem is I've done quite a major work from the
original thrift code that I have to fit them in to the current branch.
Btw, the goal to my project is to simplify Java implementation of thrift
which focus more on reducing memory footprint and increase heap re-usability
cycle. Let me try to figure this out.
regards,
Andeka Putra
On 25-10-10 23:02, Bryan Duxbury wrote:
If you have an alternate implementation handy, please attach it to the
issue.
On Mon, Oct 25, 2010 at 3:42 AM, Andeka Putra<andeka.pu...@gmail.com
wrote:
Hi Mathias, my thinking exactly..
In fact, I'm writing my own code that modified implementation of thrift
over HTTP to use Apache HTTPComponent that is also cookie-aware. I can
start
committing the changes as soon as I get write access to the repo.
regards,
Andeka Putra
On 25-10-10 16:45, Mathias Herberts (JIRA) wrote:
Under heavy load, THttpClient may fail with "too many open files"
-----------------------------------------------------------------
Key: THRIFT-970
URL: https://issues.apache.org/jira/browse/THRIFT-970
Project: Thrift
Issue Type: Bug
Components: Java - Library
Affects Versions: 0.5, 0.4, 0.3, 0.2
Environment: All
Reporter: Mathias Herberts
THttpClient uses URL.openConnection which returns a HttpUrlConnection
instance for each message transmission.
HttpUrlConnection supposedly pools connections to the server. While
stress
testing an application, we've noticed that after several thousands
requests,
THttpClient would fail with a "Too many open files" error thrown in
java.net.Socket.createImpl called from THttpClient.flush.
As the underlying connection to the server is internally handled by
HttpUrlConnection and the JVM, there is unfortunately not much that can
be
done to remedy this problem while still using HttpUrlConnection.
I propose a new implementation of THttpClient which uses Apache
HttpClient
from Http Components instead of HttpUrlConnection.
