?????? a problem of long STW because of GC ref-proc

[zheng wang]

Yes,it will be remission by your advise,but there only get request in our 
business,so 16KB is better.
IMO,the locks of offset will always be used,so is the strong reference a better 
choice?




------------------ ???????? ------------------
??????: "OpenInx"<open...@gmail.com>;
????????: 2019??9??30??(??????) ????9:46
??????: "Hbase-User"<user@hbase.apache.org>;

????: Re: a problem of long STW because of GC ref-proc



Seems your block size is very small (16KB), so there will be
70*1024*1024/16=4587520 block (at most) in your BucketCache.
For each block, the RS will maintain a soft reference idLock and a
BucketEntry in its bucket cache.  So maybe you can try to
enlarge the block size ?

On Sun, Sep 29, 2019 at 10:14 PM zheng wang <18031...@qq.com> wrote:

> Hi~
>
>
> My live cluster env config below:
> hbase version:cdh6.0.1(apache hbase2.0.0)
> hbase config: bucketCache(70g),blocksize(16k)
>
>
> java version:1.8.0_51
> javaconfig:heap(32g),-XX:+UseG1GC  -XX:MaxGCPauseMillis=100
> -XX:+ParallelRefProcEnabled
>
>
> About 1-2days ,regionServer would occur a old gen gc that cost 1~2s in
> remark phase:
>
>
> 2019-09-29T01:55:45.186+0800: 365222.053:
> [GC remark
>         2019-09-29T01:55:45.186+0800: 365222.053:
>         [Finalize Marking, 0.0016327 secs]
>         2019-09-29T01:55:45.188+0800: 365222.054:
>         [GC ref-proc
>                 2019-09-29T01:55:45.188+0800: 365222.054: [SoftReference,
> 1264586 refs, 0.3151392 secs]
>                 2019-09-29T01:55:45.503+0800: 365222.370: [WeakReference,
> 4317 refs, 0.0024381 secs]
>                 2019-09-29T01:55:45.505+0800: 365222.372: [FinalReference,
> 9791 refs, 0.0037445 secs]
>                 2019-09-29T01:55:45.509+0800: 365222.376:
> [PhantomReference, 0 refs, 1963 refs, 0.0018941 secs]
>                 2019-09-29T01:55:45.511+0800: 365222.378: [JNI Weak
> Reference, 0.0001156 secs]
>         , 1.4554361 secs]
>         2019-09-29T01:55:46.643+0800: 365223.510:
>         [Unloading, 0.0211370 secs]
> , 1.4851728 secs]
>
> The SoftReference seems used by offsetLock in BucketCache, there is two
> questions :
> 1:SoftReference proc cost 0.31s,but why GC ref-proc cost 1.45s at all?
> 2:Is this a good choice to use SoftReference here?