Re: How to limit a single row being filled with millions of columus?

[Phil Yang]

We have hbase.hstore.compaction.kv.max to setBatch on compaction and
default value is 10, which means we will write each 10 cells to writer. I
think it can prevent using too much heap while compacting?

Thanks,
Phil


2016-12-06 15:29 GMT+08:00 Guanghao Zhang :

> Now the scan context has size limit. And scan can break between cells. This
> should help for this. What is the version of your cluster?
>
> 2016-12-06 13:35 GMT+08:00 聪聪 <175998...@qq.com>:
>
> > I am glad to receive your reply！How can I find a big row quickly? If not
> > ,when we proceed major compact,the regionserver dose not work all the
> time .
> >
> >
> >
> >
> >
> >
> >
> > -- 原始邮件 --
> > 发件人: "Guanghao Zhang";;
> > 发送时间: 2016年12月6日(星期二) 中午12:13
> > 收件人: "user";
> >
> > 主题: Re: How to limit a single row being filled with millions of columus?
> >
> >
> >
> > There are a config hbase.table.max.rowsize but only used for user
> get/scan.
> > It will throw RowTooBigException when you scan a big row with millions of
> > columns. But it doesn't work for compact. If i am not wrong, there are
> not
> > a way to prevent a single row being filled with millions of columns.
> >
> > 2016-12-06 11:52 GMT+08:00 聪聪 <175998...@qq.com>:
> >
> > > Recently, I have a problem that confused me a long time. The problem is
> > > that as we all know in HBase,there are millions of columns in a single
> > row.
> > > Full gc will happen when region proceeds major compact and it results
> in
> > > regionserver and hbase not working. Is there any good way to prevent a
> > > single row being put(wrote) columns overmuch?
> > > Hope to you soon!
> >
>

Re: How to limit a single row being filled with millions of columus?

[Guanghao Zhang]

Now the scan context has size limit. And scan can break between cells. This
should help for this. What is the version of your cluster?

2016-12-06 13:35 GMT+08:00 聪聪 <175998...@qq.com>:

> I am glad to receive your reply！How can I find a big row quickly? If not
> ,when we proceed major compact,the regionserver dose not work all the time .
>
>
>
>
>
>
>
> -- 原始邮件 --
> 发件人: "Guanghao Zhang";;
> 发送时间: 2016年12月6日(星期二) 中午12:13
> 收件人: "user";
>
> 主题: Re: How to limit a single row being filled with millions of columus?
>
>
>
> There are a config hbase.table.max.rowsize but only used for user get/scan.
> It will throw RowTooBigException when you scan a big row with millions of
> columns. But it doesn't work for compact. If i am not wrong, there are not
> a way to prevent a single row being filled with millions of columns.
>
> 2016-12-06 11:52 GMT+08:00 聪聪 <175998...@qq.com>:
>
> > Recently, I have a problem that confused me a long time. The problem is
> > that as we all know in HBase,there are millions of columns in a single
> row.
> > Full gc will happen when region proceeds major compact and it results in
> > regionserver and hbase not working. Is there any good way to prevent a
> > single row being put(wrote) columns overmuch?
> > Hope to you soon!
>

Re: High CPU Utilization by meta region

[Timothy Brown]

We discovered that we were running the canary on all of our RegionServers.
These were reading from meta and causing the CPU usage. When we stopped the
canary, the CPU utilization on the RS that only contained the meta region
looked like the usage on the other RegionServers.

On Thu, Dec 1, 2016 at 8:43 PM, Stack  wrote:

> On Thu, Dec 1, 2016 at 3:48 PM, Timothy Brown  wrote:
> >
> > >
> > >
> > If you turn on RPC-level TRACE logging for a minute or so, anything about
> > > the client addresses that seems interesting?
> >
> >
> > Nothing seemed interesting to me but you may have a different opinion.
> > Here's the logs http://pastebin.com/FE8qVNH4
> >
> >
> Yeah. Nothing in there. There is no TRACE logging in there.
>
> St.Ack
>

?????? How to limit a single row being filled with millions of columus?

[????]

I am glad to receive your reply??How can I find a big row quickly? If not ,when 
we proceed major compact,the regionserver dose not work all the time .







--  --
??: "Guanghao Zhang";;
: 2016??12??6??(??) 12:13
??: "user"; 

: Re: How to limit a single row being filled with millions of columus?



There are a config hbase.table.max.rowsize but only used for user get/scan.
It will throw RowTooBigException when you scan a big row with millions of
columns. But it doesn't work for compact. If i am not wrong, there are not
a way to prevent a single row being filled with millions of columns.

2016-12-06 11:52 GMT+08:00  <175998...@qq.com>:

> Recently, I have a problem that confused me a long time. The problem is
> that as we all know in HBase,there are millions of columns in a single row.
> Full gc will happen when region proceeds major compact and it results in
> regionserver and hbase not working. Is there any good way to prevent a
> single row being put(wrote) columns overmuch?
> Hope to you soon!

?????? How to limit a single row being filled with millions of columus?

[????]

How can I find a big row quickly? If not ,when we proceed major compact,the 
regionserver dose not work all the time 
I am glad to receive your reply??How can I find a big row quickly? If not ,when 
we proceed major compact,the regionserver dose not work all the time .


--  --
??: "Guanghao Zhang";;
: 2016??12??6??(??) 12:13
??: "user"; 

: Re: How to limit a single row being filled with millions of columus?



There are a config hbase.table.max.rowsize but only used for user get/scan.
It will throw RowTooBigException when you scan a big row with millions of
columns. But it doesn't work for compact. If i am not wrong, there are not
a way to prevent a single row being filled with millions of columns.

2016-12-06 11:52 GMT+08:00  <175998...@qq.com>:

> Recently, I have a problem that confused me a long time. The problem is
> that as we all know in HBase,there are millions of columns in a single row.
> Full gc will happen when region proceeds major compact and it results in
> regionserver and hbase not working. Is there any good way to prevent a
> single row being put(wrote) columns overmuch?
> Hope to you soon!

Re: How to limit a single row being filled with millions of columus?

[Guanghao Zhang]

There are a config hbase.table.max.rowsize but only used for user get/scan.
It will throw RowTooBigException when you scan a big row with millions of
columns. But it doesn't work for compact. If i am not wrong, there are not
a way to prevent a single row being filled with millions of columns.

2016-12-06 11:52 GMT+08:00 聪聪 <175998...@qq.com>:

> Recently, I have a problem that confused me a long time. The problem is
> that as we all know in HBase,there are millions of columns in a single row.
> Full gc will happen when region proceeds major compact and it results in
> regionserver and hbase not working. Is there any good way to prevent a
> single row being put(wrote) columns overmuch?
> Hope to you soon!

Re: How to limit a single row being filled with millions of columns?

[Ted Yu]

bq. to prevent a single row being put(wrote) columns over much?

Can you clarify the above ?

Don't you have control over the schema ?

Thanks

On Mon, Dec 5, 2016 at 7:52 PM, 聪聪 <175998...@qq.com> wrote:

> Recently, I have a problem that confused me a long time. The problem is
> that as we all know in HBase,there are millions of columns in a single row.
> Full gc will happen when region proceeds major compact and it results in
> regionserver and hbase not working. Is there any good way to prevent a
> single row being put(wrote) columns overmuch?
> Hope to you soon!

How to limit a single row being filled with millions of columus?

[????]

Recently, I have a problem that confused me a long time. The problem is that as 
we all know in HBase,there are millions of columns in a single row. Full gc 
will happen when region proceeds major compact and it results in regionserver 
and hbase not working. Is there any good way to prevent a single row being 
put(wrote) columns overmuch?
Hope to you soon!