[
https://issues.apache.org/jira/browse/CALCITE-3333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabriel Reid updated CALCITE-3333:
----------------------------------
Summary: Add time-based ResultSet frame size limiting (was: Add time-based
of ResultSet frame size limiting)
> Add time-based ResultSet frame size limiting
> --------------------------------------------
>
> Key: CALCITE-3333
> URL: https://issues.apache.org/jira/browse/CALCITE-3333
> Project: Calcite
> Issue Type: New Feature
> Components: avatica
> Reporter: Gabriel Reid
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The size of a single JDBC ResultSet frame returned in a single
> {{prepareAndExecute}} or {{fetch}} invocation is currently 100, meaning that
> each retrieval of a portion of a ResultSet will send 100 rows over the wire.
> This frame size may be too big in some situations, and too small in other
> situations.
> If the underlying data source being queried can provide thousands of (small)
> records per second, then only reading them at 100 per RPC call will be
> unnecessarily slow.
> On the other hand, if the underlying data source is only providing records at
> a rate of 1 per second, then it will take 100 seconds for each RPC call to
> return, which can lead to timeouts (particularly if Avatica server is sitting
> behind a proxy that has a strict request timeout).
> The main factors to take into account when finding an ideal size of frame to
> return for each RPC call are:
> * make the frames small enough that they don't overload either Avatica server
> or the client with overly large amounts of data at one time
> * make the frames large enough so that the percentage of total query time
> that is spent only on RPC overhead is minimized
> The general idea of this ticket is to add a pluggable "frame size limiting"
> functionality so that frame size limiting can be done based on the number of
> rows, number of bytes, amount of time spent building a frame, or any other
> property or combination of properties.
> Note that CALCITE-2322 contains some work to allow configuring the size of a
> single frame on a Connection or Statement (via the {{setFetchSize}} method),
> although it's not yet merged in. That ticket would also be useful, and does
> not conflict with the general intent of this ticket.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
