> We do archiving data in Order to make assumptions on it in future. So, yes > we expect to grow continously. In the mean time I learned to go for > predictable grow per partition rather than unpredictable large > partitioning. So today we are growing 250.000.000 Records per Day going > into a single table and heading towards to about 100 times that number this > year. A Partition will grow one Record a Day, which should give us good > horizontal scaleability, but means 250.000.000 to 25.000.000.000 > partitions. Hope this Numbers should not make me feel uncomfortable :) >
There will be some additional tuning to do at around ~200 million partitions per table per node. Specifically bloom filters and index summaries. Depending on partition size and read access patterns, tuning compression settings will have a big effect as well given the volume.
