hi, all, We have a system with 17 nodes, with a big table about 28T in size. We use native hbase bulkloader (importtsv) to load data, and it generated a lot of hfiles, on each Region Server there are about 800 hfiles. We turned off Major Compact, but the Minor compaction is running due to so many hfile. The problem is, after the initial loading, there are about 80% disk space used, when minor compaction is going on, we notice the disk space is reducing rapidly until all disk spaces are used and hbase went down.
We try to change the hbase.hstore.compactionThreshold to 2000, but the minor compaction is still triggered. The system is CDH 5.7, HBase is 1.2. Could anyone help to give us some suggestions? We are really stuck. Thanks in advance. Thanks, Ming -----Original Message----- From: Andrzej [mailto:[email protected]] Sent: Friday, August 25, 2017 11:55 PM To: [email protected] Subject: How to optimal using column family? I need database to store transactions. Each transaction can have many inputs and outputs, usually 1-3 but sometimes about 100. Previously in sql I have two main tables: first table had one row per transaction and store common transaction things; second had one row per one input/output. But now are column families. I need two tables or family input and output and many columns like input:col57, output:col26 ? Which is better efficient?
