The 8TB limit isn't a hard one, it's just a reflection of the scale that Kudu developers commonly test. Beyond 8TB we can't vouch for Kudu's stability and performance. For example, we know that as the amount of on-disk data grows, node restart times get longer and longer (see KUDU-2014 for some ideas on how to improve that). Furthermore, as tablets accrue more data blocks, their superblocks become larger, raising the minimum amount of I/O for any operation that rewrites a superblock (such as a flush or compaction). Lastly, the tablet copy protocol used in rereplication tries to copy the entire superblock in one RPC message; if the superblock is too large, it'll run up against the default 50 MB RPC transfer size (see src/kudu/rpc/transfer.cc).
These examples are just off the top of my head; there may be others lurking. So this goes back to what I led with: beyond the recommended limit we aren't quite sure how Kudu's performance and stability are affected. All that said, you're welcome to try it out and report back with your findings. On Thu, Aug 2, 2018 at 7:23 AM Quanlong Huang <huang_quanl...@126.com> wrote: > > Hi all, > > In the document of "Known Issues and Limitations", it's recommended that > "maximum amount of stored data, post-replication and post-compression, per > tablet server is 8TB". How is the 8TB calculated? > > We have some machines each with 15 * 4TB spinning disk drives and 256GB RAM, > 48 cpu cores. Does it mean the other 52(= 15 * 4 - 8) TB space is recommended > to leave for other systems? We prefer to make the machine dedicated to Kudu. > Can tablet server leverage the whole space efficiently? > > Thanks, > Quanlong
