I don't remember seeing anything like that, except that if your data isn't compressible then it will probably take a little longer to insert your data. The vmstat and top outputs are pretty useless in this case, as we can't tell exactly what's going on in the processes. I suggest you take a look at the region server logs, comparing runs when your families are LZOed and when they aren't. Specifically look for the time it takes to flush and the time it takes to compact.
J-D On Wed, Aug 11, 2010 at 12:36 AM, Zheng Lv <[email protected]> wrote: > Hello Everyone, > I installed lzo in our cluster. And I found that a insert job which > normally takes 2 minutes will take 20 minutes now. > > The following is part of vmstat logs on the working regionserver when > the insert job is running: > procs -----------memory---------- ---swap-- -----io---- --system-- > -----cpu------ > r b swpd free buff cache si so bi bo in cs us sy id > wa st > 1 0 2300556 651964 13792 1399784 0 0 0 0 1533 22829 10 7 > 83 0 0 > 1 0 2300460 651360 13792 1400312 100 0 100 0 1144 25314 11 8 > 81 0 0 > 1 0 2300460 650616 13792 1400888 0 0 0 168 1574 24416 10 8 > 82 0 0 > 1 0 2300444 650036 13792 1401252 36 0 36 0 1541 24622 10 8 > 82 0 0 > 1 0 2300440 649044 13800 1401788 0 0 0 3032 1166 22469 11 8 > 80 1 0 > 1 0 2300388 648764 13800 1402296 32 0 32 0 1540 20633 9 7 > 83 0 0 > 2 0 2300292 648316 13804 1402684 100 0 100 0 1124 24694 10 8 > 82 0 0 > 1 0 2300292 647612 13804 1403664 0 0 0 0 1568 25410 10 8 > 82 0 0 > 1 0 2300288 647076 13804 1404132 0 0 0 0 1132 25667 10 8 > 81 0 0 > 2 0 2300284 646060 13812 1404804 32 0 32 3044 1563 23355 10 8 > 81 1 0 > 1 0 2300232 645588 13812 1405500 32 0 32 0 1140 24416 11 8 > 81 0 0 > 2 0 2300124 645116 13812 1405856 132 0 132 0 1588 24845 10 8 > 82 0 0 > > The following is result of "top": > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ > COMMAND > > 28603 root 24 0 1549m 433m 11m S 91.1 11.0 64:27.35 > java > > 17979 root 24 0 1555m 179m 9384 S 54.2 4.6 11:33.44 > java > > 5284 root 23 0 918m 260m 11m S 3.7 6.6 0:48.24 > java > > 19007 root 18 0 4502m 764m 10m S 2.3 19.4 20:02.20 java > and the first two are "dadtanode" and "regionserver". > > I'm sorry there is no more infomation about this I can give. Anyone has any > experience about this? > Thanks a lot. > LvZheng. >
