答复: Region Server OutOfMemory Error

Could you retry with -XX:+HeapDumpOnOutOfMemoryError ? the heap dump will make the thing clear 发件人: Shuai Lin linshuai2...@gmail.com 发送时间: 2015年1月6日 19:32 收件人: user@hbase.apache.org 主题: Region Server OutOfMemory Error Hi all, We have a hbase cluster of 5

答复: Region server not accept connections intermittently

Coud you try with -XX:+PrintGCApplicationStoppedTime vm parameter ? the hung from vm side was not caused by GC always Thanks, 发件人: Rural Hunter [ruralhun...@gmail.com] 发送时间: 2014年7月8日 14:06 收件人: user@hbase.apache.org 主题: Region server not accept

答复: White papers/Solution implemented to use HIVE/HBASE as OLTP

borthakur.com/ftp/RealtimeHadoopSigmod2011.pdf Thanks, 发件人: Ramasubramanian Narayanan [ramasubramanian.naraya...@gmail.com] 发送时间: 2014年6月9日 13:27 收件人: u...@hive.apache.org; user@hbase.apache.org 主题: White papers/Solution implemented to use HIVE/HBASE as

答复: HBase logging paused for a long time then RS crashed

2014-04-03 08:44:14,391 WARN [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 37907ms What did gc log say around that timestamp? And have you enable -XX:+PrintGCApplicationStoppedTime ? With it at least you can tell there's a long

答复: control HBase stop / start from supervisord

Hi Mathan, Maybe you could get some ideas from: https://github.com/XiaoMi/minos we do use the modified supervisor to start/stop HBase on all our cluster:) Thanks, 发件人: Nick Dimiduk [ndimi...@gmail.com] 发送时间: 2014年1月8日 2:32 收件人: mathan kumar 抄送: hbase-user

答复: one table flushes at much smaller sizes than other?

Do you observe some log like Too many hlogs before flushing a few dozen KB log ? Just want to double-check it's weather a forcing flush issue or not:) Thanks, Liang 发件人: Norbert Burger [norbert.bur...@gmail.com] 发送时间: 2013年12月27日 5:52 收件人:

答复: Upgrade from HBase 0.94.6-cdh4.4.0 to 0.94.14

It's a CDH related question, let me forward this thread to the right list. Thanks, 发件人: Kristoffer Sjögren [sto...@gmail.com] 发送时间: 2013年12月21日 23:01 收件人: user@hbase.apache.org 主题: Upgrade from HBase 0.94.6-cdh4.4.0 to 0.94.14 Hi We are running HBase

答复: Should we set the property hbase.local.dir to a list of directories on different disks to spread I/O?

Please take a look at dfs.datanode.data.dir in hdfs-site.xml. P.S. hbase.local.dir supports one dir only, don't specify it with a list:) Thanks, 发件人: Tao Xiao [xiaotao.cs@gmail.com] 发送时间: 2013年12月20日 15:11 收件人: user 主题: Should we set the property

答复: 答复: Should we set the property hbase.local.dir to a list of directories on different disks to spread I/O?

to a list of directories on different disks to spread I/O? Since hbase.local.dir specifes local directory, why doesn't it specify multiple directories on different disks to spread I/O, just as what dfs.datanode.data.dir does ? 2013/12/20 谢良 xieli...@xiaomi.com Please take a look

答复: What cause region server to timeout other than long gc?

Maybe you can try to add -XX:+PrintGCApplicationStoppedTime, then if other ops(not gc) caused the long safepoint duration, you could find the log. btw, did you have a high load during that time:) Best, Liang 发件人: Henry Hung [ythu...@winbond.com] 发送时间:

答复: How can I insert large image or video into HBase?

Do you care about low latency? if so, then maybe it's not a good choice to store big file into hbase, especially few G size, that's definitely will bring a GC hurt:) Best, Liang 发件人: Roman Nikitchenko [ro...@nikitchenko.dp.ua] 发送时间: 2013年10月23日 14:50

答复: You Are Dead Exception due to promotion failure

The OS will pick up a process to kill according to an internal oom score,weight...(very likely is HBase if it ran long time and consumed lots of memory) 发件人: prakash kadel [prakash.ka...@gmail.com] 发送时间: 2013年10月8日 6:13 收件人: user@hbase.apache.org 主题: Re:

答复: flushing + compactions after config change

If reached memstore global up-limit, you'll find Blocking updates on in your files(see MemStoreFlusher.reclaimMemStoreMemory); If it's caused by too many log files, you'll find Too many hlogs: logs=(see HLog.cleanOldLogs) Hope it's helpful for you:) Best, Liang

答复: 答复: flushing + compactions after config change

btw, don't use CMSIncrementalMode, iirc, it had been removed from hotspot upstream accually. 发件人: Viral Bajaria [viral.baja...@gmail.com] 发送时间: 2013年6月27日 18:08 收件人: user@hbase.apache.org 主题: Re: 答复: flushing + compactions after config change I do have a

答复: Possibility of using timestamp as row key in HBase

Or maybe you could try to revert your rowkey:) 发件人: yun peng [pengyunm...@gmail.com] 发送时间: 2013年6月20日 5:59 收件人: user@hbase.apache.org 主题: Re: Possibility of using timestamp as row key in HBase Thanks for the reply. The idea is interesting, but in practice,

答复: hbase jvm problem

Would you have a chance to test w/o UseAdaptiveSizePolicy option? There's a related hotspot issue discussed several days ago: http://marc.info/?l=openjdk-serviceability-devm=136367606426463w=1 Best, Liang 发件人: jian fan [xiaofanhb...@gmail.com] 发送时间:

答复: Does HBase RegionServer benefit from OS Page Cache

Maybe we should adopt some ideas from RDBMS ? In MySQL area: Innodb storage engine has a buffer pool(just like current block cache), caches both compressed and uncompressed pages in latest innodb version, it brings about adaptive LRU algorithm, see

答复: HBase CheckSum vs Hadoop CheckSum

comments in line Regards, Liang 发件人: Jean-Marc Spaggiari [jean-m...@spaggiari.org] 发送时间: 2013年2月26日 20:03 收件人: user 主题: HBase CheckSum vs Hadoop CheckSum Hi, Quick question. When we are activating the short circuit read in HBase, it's recommanded to

答复: GC frequency

Of course, you'll hit the nightmarish CMS fragement easier if NewSize too low:) Generally speaking, most of YGC should be less than 5ms for a normal size heap. maybe your load is too high or there're vm options be misconfigured ? 发件人: Varun Sharma

答复: 答复: GC frequency

to clean out 512m - now, the only thing that I am afraid with reducing the size of new gen is higher frequency and the chances of more frequent promotion failures. On Thu, Feb 21, 2013 at 12:10 AM, 谢良 xieli...@xiaomi.com wrote: Of course, you'll hit the nightmarish CMS fragement easier if NewSize

答复: 答复: 答复: GC frequency

is a good choice:) else, you should analyse your gc log and tweak vm option On Thu, Feb 21, 2013 at 12:50 AM, 谢良 xieli...@xiaomi.com wrote: Here is a good formula to estimate: http://blog.ragozin.info/2011/06/understanding-gc-pauses-in-jvm-hotspots.html Hope it helpful

答复: GC pause issues

Hi Varun, Please note if you try to increase new generation size, then the ParNew time will be up accordingly, and CMS YGC is also a STW. could you have a try to reduce memstore size to a smaller value, e.g. 128m or 256m ? Regards, Liang 发件人: Varun

答复: Region server Memory Use is double the -Xmx setting

Please set -XX:MaxDirectMemorySize explicitly, else the default is taking the value like -Xmx in currenty JDK6, at least for jdk1.6.30+ Best Regards, Liang 发件人: Buckley,Ron [buckl...@oclc.org] 发送时间: 2013年1月23日 5:17 收件人: user@hbase.apache.org 主题: Region

答复: Storing images in Hbase

Just out of curiousity, why not considering a blob storage system ? Best Regards, Liang 发件人: kavishahuja [kavishah...@yahoo.com] 发送时间: 2013年1月5日 18:11 收件人: user@hbase.apache.org 主题: Storing images in Hbase *Hello EVERYBODY first of all, a happy new year

答复: 答复: Storing images in Hbase

: 2013年1月6日 13:45 收件人: user@hbase.apache.org Cc: user@hbase.apache.org 主题: Re: 答复: Storing images in Hbase IMHO Use dfs unread for blobs and use Hbase for meta data Sent from my iPhone On Jan 5, 2013, at 7:58 PM, 谢良 xieli...@xiaomi.com wrote: Just out of curiousity, why not considering a blob

答复: 答复: Multiple regionservers on a single node

regionservers on a single node On Fri, Dec 7, 2012 at 10:58 AM, 谢良 xieli...@xiaomi.com wrote: Emm, have you tried to tune your GC deeply? please provide the exactly VM options and jdk version and GC logs.. In our test cluster this week, i managed to reduce the longest STW from 22+ seconds(Xmx20G

答复: Re:Re: 答复: Multiple regionservers on a single node

人: user@hbase.apache.org 主题: Re:Re: 答复: Multiple regionservers on a single node Can you share your GC command options here? 在 2012-12-11 06:21:08，Adrien Mogenet adrien.moge...@gmail.com 写道： On Fri, Dec 7, 2012 at 10:58 AM, 谢良 xieli...@xiaomi.com wrote: Emm, have you tried to tune your GC

答复: Multiple regionservers on a single node

Emm, have you tried to tune your GC deeply? please provide the exactly VM options and jdk version and GC logs.. In our test cluster this week, i managed to reduce the longest STW from 22+ seconds(Xmx20G) to 1.1s(Xmx48G) under a very heavy YCSB stress long-term-testing. Also it would be better

About dfs.datanode.readahead.bytes

Is there anybody had benchmarked dfs.datanode.readahead.bytes parameter in HBase scenario ? My option is that we should tune it to a smaller value(e.g. on Hadoop 2.0), like 512k or sth else, especially high random read request in pretty huge dataset. Dose it make sense ?

答复: hbase.client.scanner.timeout.period not being respected

Did you rebounce your server cluster ? Per HregionServer.java code : this.scannerLeaseTimeoutPeriod = conf.getInt(HConstants.HBASE_CLIENT_SCANNER_TIMEOUT_PERIOD, HConstants.DEFAULT_HBASE_CLIENT_SCANNER_TIMEOUT_PERIOD); seems this parameter is used by server side as well I am not an

答复: could not start HMaster

Is there any complain in HDFS log ? 发件人: yulin...@dell.com [yulin...@dell.com] 发送时间: 2012年10月16日 4:35 收件人: user@hbase.apache.org 主题: RE: could not start HMaster No, I don't think so. This is a dedicated testing machine and no automatic cleaning up on the

答复: [Stand alone - distributed mode] HBase master isn't initializing completely

Is there any WARNING/ERROR in HDFS logfile ? Please ensure zkhdfs are in healthy status firstly Could you provide your hbse version, it'll be great:) 发件人: techbuddy [techbuddy...@gmail.com] 发送时间: 2012年10月12日 5:11 收件人: user@hbase.apache.org 主题: [Stand alone

答复: hmaster down cause by zookeeper?

Hi Xiang, It's not the root cause, if you skim through sendBuffer impl in NIOServerCnxn.java, you'll find there's a catch statement finally to log all exception, no throw again. IMHO, the hbase master log file is the right place you need to dive:) 发件人: