Thrif messages have to be buffered into memory. I suggest to use paging n rows at a time to keep latency predictable.
On Friday, April 22, 2016, Tenghuan He <[email protected]> wrote: > Hi there > > I have a PostgreSQL database on my server machine, which can only be > queried locally. Now I want to query the data from another machine. It > seems that Thrift RPC is a natural and good choice. However I found there > is some problem when the ResultSet is very large, say there are millions of > rows in the ResultSet. Since ResultSet is not serializable and serializing > it is meaningless. Returning the materialized rows all at once consumes two > much memory. > I consider returning such as 1000 rows each time > Is there any other idea or advice? > > Thanks in advance > > Tenghuan He > -- Sorry this was sent from mobile. Will do less grammar and spell check than usual.
