Alexey Serbin created KUDU-1693:
-----------------------------------
Summary: Flush write operations on per-TS basis and add
corresponding limit on the buffer space
Key: KUDU-1693
URL: https://issues.apache.org/jira/browse/KUDU-1693
Project: Kudu
Issue Type: Improvement
Components: client
Affects Versions: 1.0.0
Reporter: Alexey Serbin
Grouping write operations on per-tablet-server basis would be beneficial for
'one-out-of-many lagging tablet server' scenario. There, all tablet servers
for a table perform good except for one which runs slow due to some reason
(excessive IO, network issues, failing disk, etc.). The problem is that the
lagging server hinders buffers turnaround: a buffer is considered 'flushed' and
its space is reclaimed at once when _all_ operations in the buffer are
completed. So, if 1000 operations are flushed but there is 1 operation still
in progress, the whole buffer space is 'locked' and cannot be used.
Accordingly, introducing per-tablet-server buffer limit for write operations
would help to address scenarios with concurrent writes into tables with very
different partition factors (like 2 and 100). E.g., the incoming operations
for tables with very different partition factors are intermixed in the context
of the same session. The problem is that setting the total buffer space limit
high is fine for the writes into the table with many partitions (assuming those
writes are evenly distributed across participating tablets), but it may be over
the server-side's limit for max transaction size if those writes are targeted
for a table with a few partitions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
