thanks for your response. Here I provide an example since I was looking into it. Like you mentioned it can be done but it takes some effort.
I see this in the regionserver logs: 4:21:32.107 PM INFO org.apache.hadoop.hbase.regionserver.HRegion Starting compaction on d in region <table>,\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1407770330344.061ddbc4f79c44d6660d0aba6ded1323. 4:21:32.107 PM INFO org.apache.hadoop.hbase.regionserver.HStore Starting compaction of 10 file(s) in d of <table>,\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1407770330344.061ddbc4f79c44d6660d0aba6ded1323. into tmpdir=hdfs://hmaster.in.wellcentive.com:8020/hbase/data/default/<table>/061ddbc4f79c44d6660d0aba6ded1323/.tmp, totalSize=2.0 G 4:25:35.755 PM INFO org.apache.hadoop.hbase.regionserver.HStore Completed compaction of 10 file(s) in d of <table>,\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1407770330344.061ddbc4f79c44d6660d0aba6ded1323. into e7471cf82de4417491e631f7b7f0d67a(size=2.0 G), total size for store is 51.4 G. This selection was in queue for 0sec, and took 4mins, 3sec to execute. 4:25:35.755 PM INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread Completed compaction: Request = regionName=<table>,\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1407770330344.061ddbc4f79c44d6660d0aba6ded1323., storeName=d, fileCount=10, fileSize=2.0 G, priority=42, time=4989389638797549; duration=4mins, 3sec and I checked the HBase .tmp dir and after the logs show completed then I can correlate the results. I can do this before the compaction completes by process of elimination since the other file is much larger than 2GB, but it would not be obvious if there were two files both smaller than 2GB. [hdfs@hmaster gmarquez]$ hdfs dfs -ls /hbase/data/default/<table>/061ddbc4f79c44d6660d0aba6ded1323/.tmp Found 2 items -rwxr-xr-x 3 hbase hbase 24561844224 2014-08-11 15:30 /hbase/data/default/<table>/061ddbc4f79c44d6660d0aba6ded1323/.tmp/22d1b6f857e4458d8f6c875d4f48a575 -rwxr-xr-x 3 hbase hbase 1744830464 2014-08-11 16:21 /hbase/data/default/<table>/061ddbc4f79c44d6660d0aba6ded1323/.tmp/e7471cf82de4417491e631f7b7f0d67a process of elimination leads me to that second file name 'e7471cf82de4417491e631f7b7f0d67a' because the first one is already much bigger than 2GB, but this is a bit tedious as you can tell. On Mon, Aug 11, 2014 at 4:31 PM, Geovanie Marquez < [email protected]> wrote: > nice, just took a look at this path in my system. I see the growth but > that would only apply to major_compactions where all files are picked up > and compacted because minor compactions won't be trackable this way without > knowing which files were selected by the compaction algorithm. > > is there a method for getting progress on these? > > > > On Mon, Aug 11, 2014 at 3:14 PM, Bryan Beaudreault < > [email protected]> wrote: > >> I've also seen that on 0.94.x. The compaction progress is almost never >> accurate. >> >> I've taken to doing: >> >> hdfs -du /hbase/<table>/<region>/.tmp >> >> to track the new hfile size, expecting it to be approximately the same >> size >> as total hfiles pre-compaction. >> >> >> On Mon, Aug 11, 2014 at 3:11 PM, Geovanie Marquez < >> [email protected]> wrote: >> >> > *Attribute Name* *Value* >> > HBase Version 0.98.1-cdh5.1.0, rUnknown >> > HBase Compiled Sat Jul 12 08:20:49 PDT 2014, jenkins >> > Hadoop Version 2.3.0-cdh5.1.0, r8e266e052e423af592871e2dfe09d54c03f6a0e8 >> > Hadoop Compiled 2014-07-12T13:49Z, jenkins >> > >> > >> > On Mon, Aug 11, 2014 at 12:56 PM, Ted Yu <[email protected]> wrote: >> > >> > > Which release of hbase are you using ? >> > > >> > > Thanks >> > > >> > > >> > > On Mon, Aug 11, 2014 at 9:49 AM, Geovanie Marquez < >> > > [email protected]> wrote: >> > > >> > > > What is the meaning of the progress value below? >> > > > >> > > > >> > > > >> > > > I see the information broken down below in the following HBase UI. >> > > > >> > > > >> > > >> > >> http://region-server-ip:60030/rs-status?filter=general#regionCompactStats >> > > > >> > > > >> > > > >> > > > Region Name Num. >> > > > >> > > > >> > > > >> > > >> > >> <tablename>,`\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1398130030457.2dddb2b81d70ee4b4aaf39ca830beb41. >> > > > >> > > > >> > > > >> > > > Compacting KVsNum. >> > > > >> > > > 13617 >> > > > >> > > > >> > > > >> > > > Compacted KVsCompaction >> > > > >> > > > 9221037 >> > > > >> > > > >> > > > >> > > > Progress >> > > > >> > > > 67717.09% >> > > > >> > > > >> > > > >> > > > Another example: >> > > > >> > > > Compacting KVsNum. Compacted KVsCompaction Progress >> > > > >> > > > 357792875 1624293960 453.98% >> > > > >> > > > >> > > > The progress seems non-sensical. How should I interpret it? I >> notice >> > > > something more than 100% most of the time. >> > > > >> > > >> > >> > >
