Will Berkeley created KUDU-2701:
-----------------------------------
Summary: Compaction loop resulting from small rowset compaction
policy (KUDU-1400)
Key: KUDU-2701
URL: https://issues.apache.org/jira/browse/KUDU-2701
Project: Kudu
Issue Type: Bug
Affects Versions: 1.9.0
Reporter: Will Berkeley
Assignee: Will Berkeley
I saw an instance of rowset compaction looping, accomplishing nothing but
wearing out the disk. The offending compaction's logging is
{noformat}
I0213 14:23:51.062127 9268 maintenance_manager.cc:306] P
09d6bf7a02124145b43f43cb7a667b3d: Scheduling
CompactRowSetsOp(39aba63834b441e0b26d7aa5949f92ae): perf score=0.025948
I0213 14:23:55.630163 8904 maintenance_manager.cc:501] P
09d6bf7a02124145b43f43cb7a667b3d:
CompactRowSetsOp(39aba63834b441e0b26d7aa5949f92ae) complete. Timing: real
12.968s user 10.285s sys 0.578s Metrics:
{"bytes_written":235680573,"cfile_cache_hit":32,"cfile_cache_hit_bytes":44035,"cfile_cache_miss":5750,"cfile_cache_miss_bytes":246197139,"cfile_init":24,"data
dirs.queue_time_us":162819,"data dirs.run_cpu_time_us":1864,"data
dirs.run_wall_time_us":1461840,"drs_written":8,"fdatasync":25,"fdatasync_us":1217638,"lbm_read_time_us":79002,"lbm_reads_lt_1ms":5846,"lbm_write_time_us":1043597,"lbm_writes_1-10_ms":2,"lbm_writes_10-100_ms":10,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":30276,"mutex_wait_us":1,"num_input_rowsets":8,"rows_written":46640210,"spinlock_wait_cycles":1664,"thread_start_us":453,"threads_started":14,"wal-append.queue_time_us":49}
{noformat}
The situation is that about 8 16MB rowsets are calculated to compact to 4 32MB
rowsets, but get written out as 8 16MB rowsets. The trick is that the rowsets
*aren't* 16MB...they are really 32MB-- 16MB is the size of the base data and
redos, which is what we use as the "size" for compaction purposes.
Probably, we should use "everything" ({{Rowset::OnDiskSize()}}) for the
KUDU-1400 calculations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
